2024-05-27T07:15:20.621 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-27T07:15:20.621 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/postgres}} --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662/orig.config.yaml 2024-05-27T07:15:20.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T07:15:20.768 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662 branch: wip-vshankar-testing-20240521.063558-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/postgres}} email: vshankar@redhat.com first_in_suite: false job_id: '7727662' kernel: flavor: default kdb: 1 sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240521.063558-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240521.063558-debug sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 owner: scheduled_vshankar@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2499 sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 sleep_before_teardown: 0 subset: 573/1024 suite: fs suite_branch: wip-vshankar-testing-20240521.063558-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLnrBzFmEIEtKeA+lEW+u0rHlHWk5rzyGdDfz/n5q0mJ4zjjvxBJzSK+Z/A9EKgG1QHf6GXqOqvZbkQ2lahfx6k= smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAe0V5u+mSphn1wYbTwkRP+EIdOZ1ANnr9/LCtR9MYbsXjfRK54AQ99HJpVWq4WXacYUH41Cr2dgs8XaomVF3MI= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6IKF4DDtlBP8J23+ozl5s8gQ9fCsRv1FTLCn4VosZsoQAHWM2urnK9+bQEjVbbWBRb0yGCQ05P9ctoMnzODYA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-27_05:52:49 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-27T07:15:20.768 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa; will attempt to use it 2024-05-27T07:15:20.769 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks 2024-05-27T07:15:20.769 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-27T07:15:20.770 INFO:teuthology.task.internal:Checking packages... 2024-05-27T07:15:20.789 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '38f9d900c83c63799fbdbe61acc9a11b0d3554a6' 2024-05-27T07:15:20.789 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-27T07:15:20.790 INFO:teuthology.packaging:ref: None 2024-05-27T07:15:20.790 INFO:teuthology.packaging:tag: None 2024-05-27T07:15:20.790 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240521.063558-debug 2024-05-27T07:15:20.790 INFO:teuthology.packaging:sha1: 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:15:20.790 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240521.063558-debug 2024-05-27T07:15:21.046 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3872.g38f9d900 2024-05-27T07:15:21.047 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-27T07:15:21.056 INFO:teuthology.task.internal:no buildpackages task found 2024-05-27T07:15:21.056 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-27T07:15:21.118 INFO:teuthology.task.internal:Saving configuration 2024-05-27T07:15:21.139 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-27T07:15:21.186 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-27T07:15:21.210 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 07:09:37.973616', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLnrBzFmEIEtKeA+lEW+u0rHlHWk5rzyGdDfz/n5q0mJ4zjjvxBJzSK+Z/A9EKgG1QHf6GXqOqvZbkQ2lahfx6k='} 2024-05-27T07:15:21.231 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 07:09:37.974565', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAe0V5u+mSphn1wYbTwkRP+EIdOZ1ANnr9/LCtR9MYbsXjfRK54AQ99HJpVWq4WXacYUH41Cr2dgs8XaomVF3MI='} 2024-05-27T07:15:21.257 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 07:09:37.972176', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF6IKF4DDtlBP8J23+ozl5s8gQ9fCsRv1FTLCn4VosZsoQAHWM2urnK9+bQEjVbbWBRb0yGCQ05P9ctoMnzODYA='} 2024-05-27T07:15:21.257 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-27T07:15:21.262 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-27T07:15:21.263 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-27T07:15:21.263 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-27T07:15:21.263 INFO:teuthology.run_tasks:Running task console_log... 2024-05-27T07:15:21.379 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f184e4f7ac0>, signals=[15]) 2024-05-27T07:15:21.379 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-27T07:15:21.387 INFO:teuthology.task.internal:Opening connections... 2024-05-27T07:15:21.387 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T07:15:21.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:15:21.463 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-05-27T07:15:21.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:15:21.541 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-05-27T07:15:21.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:15:21.624 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-27T07:15:21.631 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-05-27T07:15:21.649 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-05-27T07:15:21.649 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-05-27T07:15:21.706 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T07:15:21.707 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T07:15:21.708 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-05-27T07:15:21.729 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-05-27T07:15:21.747 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-05-27T07:15:21.747 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-05-27T07:15:21.804 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-05-27T07:15:21.804 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-05-27T07:15:21.804 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-05-27T07:15:21.804 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-05-27T07:15:21.805 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-05-27T07:15:21.805 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-05-27T07:15:21.805 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T07:15:21.805 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-05-27T07:15:21.805 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-05-27T07:15:21.805 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T07:15:21.805 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-05-27T07:15:21.805 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T07:15:21.805 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T07:15:21.805 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T07:15:21.806 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-05-27T07:15:21.855 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-05-27T07:15:21.873 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-05-27T07:15:21.874 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T07:15:21.931 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T07:15:21.932 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-05-27T07:15:21.962 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-27T07:15:21.972 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-27T07:15:22.028 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-27T07:15:22.028 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-05-27T07:15:22.030 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-05-27T07:15:22.033 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-05-27T07:15:22.049 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-27T07:15:22.095 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-27T07:15:22.095 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-05-27T07:15:22.097 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-05-27T07:15:22.100 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-05-27T07:15:22.215 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-27T07:15:22.296 INFO:teuthology.run_tasks:Running task kernel... 2024-05-27T07:15:22.313 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'} 2024-05-27T07:15:22.313 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}, 'mon.c': {'flavor': 'default', 'kdb': 1, 'sha1': 'b9459a26b0888785d05c47deb35abc9d0354ab93'}}, timeout 300 2024-05-27T07:15:22.314 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T07:15:22.314 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T07:15:22.315 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T07:15:22.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:15:22.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:15:22.332 DEBUG:teuthology.task.kernel:sha1 for mon.a is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:22.332 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T07:15:22.332 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-05-27T07:15:22.333 DEBUG:teuthology.task.kernel:sha1 for mon.b is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:22.333 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T07:15:22.333 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-05-27T07:15:22.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:15:22.334 DEBUG:teuthology.task.kernel:sha1 for mon.c is b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:22.334 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "b9459a26b0888785d05c47deb35abc9d0354ab93"... 2024-05-27T07:15:22.334 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-05-27T07:15:22.389 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-437.el9.x86_64 2024-05-27T07:15:22.389 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:22.389 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T07:15:22.390 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:22.394 INFO:teuthology.orchestra.run.smithi136.stdout:5.14.0-437.el9.x86_64 2024-05-27T07:15:22.394 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-437.el9.x86_64 2024-05-27T07:15:22.394 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:22.394 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T07:15:22.394 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:22.395 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:22.395 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-27T07:15:22.395 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:22.555 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum upgrade -y linux-firmware 2024-05-27T07:15:22.557 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum upgrade -y linux-firmware 2024-05-27T07:15:22.606 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum upgrade -y linux-firmware 2024-05-27T07:15:24.663 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.1 MB 00:00 2024-05-27T07:15:25.098 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-05-27T07:15:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.1 MB 00:01 2024-05-27T07:15:26.481 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-05-27T07:15:26.875 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-05-27T07:15:27.218 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-05-27T07:15:30.706 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-05-27T07:15:31.102 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.2 MB 00:00 2024-05-27T07:15:31.693 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-05-27T07:15:32.528 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 17 kB 00:00 2024-05-27T07:15:33.015 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 17 kB 00:01 2024-05-27T07:15:33.569 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 17 kB 00:00 2024-05-27T07:15:34.083 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-05-27T07:15:35.212 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-27T07:15:35.539 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 21 MB 00:02 2024-05-27T07:15:39.250 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-05-27T07:15:40.629 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 5.8 kB/s | 1.7 kB 00:00 2024-05-27T07:15:40.697 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-27T07:15:41.233 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-05-27T07:15:41.234 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-05-27T07:15:41.234 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-05-27T07:15:41.353 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.c... 2024-05-27T07:15:41.354 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:41.515 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T07:15:41.515 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T07:15:41.517 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi136.front.sepia.ceph.com to complete... 2024-05-27T07:15:42.617 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-05-27T07:15:42.618 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-05-27T07:15:42.618 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-05-27T07:15:42.713 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-27T07:15:42.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:42.727 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-05-27T07:15:42.728 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-05-27T07:15:42.728 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-05-27T07:15:42.829 INFO:teuthology.task.kernel:Downloading kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-27T07:15:42.830 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b9459a26b0888785d05c47deb35abc9d0354ab93 2024-05-27T07:15:42.862 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T07:15:42.862 DEBUG:teuthology.orchestra.run.smithi129:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T07:15:42.864 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi129.front.sepia.ceph.com to complete... 2024-05-27T07:15:43.052 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ 2024-05-27T07:15:43.052 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_gb9459a26b088-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-27T07:15:43.055 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi090.front.sepia.ceph.com to complete... 2024-05-27T07:15:57.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27 07:15:57 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T07:15:57.770 INFO:teuthology.orchestra.run.smithi136.stderr:FINISHED --2024-05-27 07:15:57-- 2024-05-27T07:15:57.770 INFO:teuthology.orchestra.run.smithi136.stderr:Total wall clock time: 16s 2024-05-27T07:15:57.770 INFO:teuthology.orchestra.run.smithi136.stderr:Downloaded: 1 files, 340M in 16s (21.4 MB/s) 2024-05-27T07:15:57.773 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.c... 2024-05-27T07:15:57.773 DEBUG:teuthology.orchestra.run.smithi136:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T07:15:59.855 INFO:teuthology.orchestra.run.smithi136.stdout:Verifying... ######################################## 2024-05-27T07:15:59.992 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing... ######################################## 2024-05-27T07:16:00.120 INFO:teuthology.orchestra.run.smithi136.stdout:Updating / installing... 2024-05-27T07:16:08.089 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T07:16:15.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27 07:16:15 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T07:16:15.783 INFO:teuthology.orchestra.run.smithi129.stderr:FINISHED --2024-05-27 07:16:15-- 2024-05-27T07:16:15.783 INFO:teuthology.orchestra.run.smithi129.stderr:Total wall clock time: 33s 2024-05-27T07:16:15.783 INFO:teuthology.orchestra.run.smithi129.stderr:Downloaded: 1 files, 340M in 32s (10.5 MB/s) 2024-05-27T07:16:15.786 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.b... 2024-05-27T07:16:15.786 DEBUG:teuthology.orchestra.run.smithi129:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T07:16:17.876 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying... ######################################## 2024-05-27T07:16:18.026 INFO:teuthology.orchestra.run.smithi129.stdout:Preparing... ######################################## 2024-05-27T07:16:18.136 INFO:teuthology.orchestra.run.smithi129.stdout:Updating / installing... 2024-05-27T07:16:19.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27 07:16:19 URL:https://3.chacra.ceph.com/r/kernel/testing/b9459a26b0888785d05c47deb35abc9d0354ab93/centos/9/flavors/default/x86_64/kernel-6.9.0_gb9459a26b088-1.x86_64.rpm [356863406/356863406] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-27T07:16:19.896 INFO:teuthology.orchestra.run.smithi090.stderr:FINISHED --2024-05-27 07:16:19-- 2024-05-27T07:16:19.896 INFO:teuthology.orchestra.run.smithi090.stderr:Total wall clock time: 37s 2024-05-27T07:16:19.896 INFO:teuthology.orchestra.run.smithi090.stderr:Downloaded: 1 files, 340M in 36s (9.39 MB/s) 2024-05-27T07:16:19.911 INFO:teuthology.task.kernel:Installing kernel b9459a26b0888785d05c47deb35abc9d0354ab93 on mon.a... 2024-05-27T07:16:19.912 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-27T07:16:22.640 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying... ######################################## 2024-05-27T07:16:22.778 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing... ######################################## 2024-05-27T07:16:22.893 INFO:teuthology.orchestra.run.smithi090.stdout:Updating / installing... 2024-05-27T07:16:26.146 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T07:16:30.827 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-6.9.0_gb9459a26b088-1 ######################################## 2024-05-27T07:16:47.798 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T07:16:47.798 DEBUG:teuthology.orchestra.run.smithi136:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T07:16:47.882 INFO:teuthology.orchestra.run.smithi136.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T07:16:47.882 INFO:teuthology.orchestra.run.smithi136.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T07:16:47.882 INFO:teuthology.orchestra.run.smithi136.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T07:16:47.882 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T07:16:47.882 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T07:16:47.882 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T07:16:47.882 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T07:16:47.883 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T07:16:47.884 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T07:16:47.885 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T07:16:47.886 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T07:16:47.887 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T07:16:47.888 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T07:16:47.889 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T07:16:47.890 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T07:16:47.891 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T07:16:47.892 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T07:16:47.893 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T07:16:47.894 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T07:16:47.895 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T07:16:47.896 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T07:16:47.897 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T07:16:47.898 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T07:16:47.899 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T07:16:47.900 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T07:16:47.901 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T07:16:47.902 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T07:16:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T07:16:47.904 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T07:16:47.905 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T07:16:47.906 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T07:16:47.907 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T07:16:47.908 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T07:16:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T07:16:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T07:16:47.911 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T07:16:47.912 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T07:16:47.913 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T07:16:47.914 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T07:16:47.915 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T07:16:47.916 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T07:16:47.917 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T07:16:47.918 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T07:16:47.919 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T07:16:47.920 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T07:16:47.921 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T07:16:47.922 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T07:16:47.923 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T07:16:47.924 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T07:16:47.925 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T07:16:47.926 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T07:16:47.927 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T07:16:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T07:16:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T07:16:47.930 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T07:16:47.931 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T07:16:47.932 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T07:16:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T07:16:47.934 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T07:16:47.935 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T07:16:47.936 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T07:16:47.937 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T07:16:47.938 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T07:16:47.939 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T07:16:47.940 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T07:16:47.941 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T07:16:47.942 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T07:16:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T07:16:47.944 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T07:16:47.945 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T07:16:47.946 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T07:16:47.947 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T07:16:47.948 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T07:16:47.949 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T07:16:47.950 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T07:16:47.951 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T07:16:47.952 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T07:16:47.953 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T07:16:47.954 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T07:16:47.955 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T07:16:47.956 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T07:16:47.957 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T07:16:47.958 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T07:16:47.959 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T07:16:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T07:16:47.961 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T07:16:47.962 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T07:16:47.963 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T07:16:47.964 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T07:16:47.965 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T07:16:47.966 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T07:16:47.967 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T07:16:47.968 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T07:16:47.969 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T07:16:47.970 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T07:16:47.971 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T07:16:47.972 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T07:16:47.973 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T07:16:47.974 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T07:16:47.975 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T07:16:47.976 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T07:16:47.977 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T07:16:47.978 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T07:16:47.979 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T07:16:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T07:16:47.981 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T07:16:47.982 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T07:16:47.983 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T07:16:47.984 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T07:16:47.985 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T07:16:47.986 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T07:16:47.987 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T07:16:47.988 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T07:16:47.989 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T07:16:47.990 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T07:16:47.991 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T07:16:47.992 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T07:16:47.993 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T07:16:47.994 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T07:16:47.995 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T07:16:47.996 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T07:16:47.997 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T07:16:47.998 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T07:16:47.999 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T07:16:48.000 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T07:16:48.001 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T07:16:48.002 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T07:16:48.003 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T07:16:48.004 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T07:16:48.005 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T07:16:48.006 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T07:16:48.007 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T07:16:48.008 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T07:16:48.009 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T07:16:48.010 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T07:16:48.011 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T07:16:48.012 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T07:16:48.013 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T07:16:48.014 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T07:16:48.015 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T07:16:48.016 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T07:16:48.017 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T07:16:48.018 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T07:16:48.019 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T07:16:48.020 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T07:16:48.021 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T07:16:48.022 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T07:16:48.023 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T07:16:48.024 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T07:16:48.025 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T07:16:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T07:16:48.027 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T07:16:48.027 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T07:16:48.027 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T07:16:48.027 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T07:16:48.027 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T07:16:48.027 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T07:16:48.027 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T07:16:48.027 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T07:16:48.027 INFO:teuthology.orchestra.run.smithi136.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:16:48.031 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T07:16:48.032 DEBUG:teuthology.orchestra.run.smithi136:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T07:16:48.053 INFO:teuthology.orchestra.run.smithi136.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout:fi 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout:for file in vmlinuz System.map config; do 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout: fi 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout:done 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:16:48.054 INFO:teuthology.orchestra.run.smithi136.stdout:fi 2024-05-27T07:16:48.055 INFO:teuthology.orchestra.run.smithi136.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T07:16:48.055 INFO:teuthology.orchestra.run.smithi136.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T07:16:48.055 INFO:teuthology.orchestra.run.smithi136.stdout:fi 2024-05-27T07:16:48.055 INFO:teuthology.orchestra.run.smithi136.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T07:16:48.055 INFO:teuthology.orchestra.run.smithi136.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:16:48.055 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T07:16:48.055 INFO:teuthology.orchestra.run.smithi136.stdout:fi 2024-05-27T07:16:48.056 DEBUG:teuthology.orchestra.run.smithi136:> sudo rpm -qi grub2-tools 2024-05-27T07:16:48.127 INFO:teuthology.orchestra.run.smithi136.stdout:Name : grub2-tools 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Epoch : 1 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Version : 2.06 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Release : 68.el9 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: x86_64 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Group : Unspecified 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Size : 8264254 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:License : GPLv3+ 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T07:16:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:Packager : builder@centos.org 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout:Vendor : CentOS 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout:Summary : Support tools for GRUB. 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout:Description : 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout:hardware devices. 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:16:48.129 INFO:teuthology.orchestra.run.smithi136.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T07:16:48.130 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T07:16:48.130 INFO:teuthology.task.kernel:Updating grub on smithi136 to boot 6.9.0-gb9459a26b088 2024-05-27T07:16:48.130 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T07:16:48.683 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2024-05-27T07:16:50.089 INFO:teuthology.orchestra.run.smithi136.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T07:16:50.122 INFO:teuthology.orchestra.run.smithi136.stderr:done 2024-05-27T07:16:50.125 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-05-27T07:16:50.153 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.39ucJ10nbj 2024-05-27T07:16:50.154 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.39ucJ10nbj 2024-05-27T07:16:50.227 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.39ucJ10nbj 2024-05-27T07:16:50.372 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.39ucJ10nbj is 6KB 2024-05-27T07:16:50.421 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.39ucJ10nbj 2024-05-27T07:16:50.435 DEBUG:teuthology.orchestra.run.smithi136:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T07:16:50.507 INFO:teuthology.orchestra.run.smithi136.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T07:16:50.507 INFO:teuthology.orchestra.run.smithi136.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T07:16:50.508 INFO:teuthology.orchestra.run.smithi136.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T07:16:50.508 INFO:teuthology.orchestra.run.smithi136.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T07:16:50.509 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T07:16:50.682 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2024-05-27T07:17:03.295 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi129.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T07:17:03.295 DEBUG:teuthology.orchestra.run.smithi129:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T07:17:03.365 INFO:teuthology.orchestra.run.smithi129.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T07:17:03.365 INFO:teuthology.orchestra.run.smithi129.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T07:17:03.366 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T07:17:03.367 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T07:17:03.368 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T07:17:03.369 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T07:17:03.370 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T07:17:03.371 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T07:17:03.372 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T07:17:03.373 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T07:17:03.374 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T07:17:03.375 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T07:17:03.376 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T07:17:03.377 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T07:17:03.378 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T07:17:03.379 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T07:17:03.380 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T07:17:03.381 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T07:17:03.382 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T07:17:03.383 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T07:17:03.384 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T07:17:03.385 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T07:17:03.386 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T07:17:03.387 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T07:17:03.388 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T07:17:03.389 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T07:17:03.390 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T07:17:03.391 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T07:17:03.392 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T07:17:03.393 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T07:17:03.394 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T07:17:03.395 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T07:17:03.396 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T07:17:03.397 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T07:17:03.398 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T07:17:03.399 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T07:17:03.400 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T07:17:03.401 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T07:17:03.402 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T07:17:03.403 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T07:17:03.404 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T07:17:03.405 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T07:17:03.406 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T07:17:03.407 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T07:17:03.408 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T07:17:03.409 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T07:17:03.410 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T07:17:03.411 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T07:17:03.412 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T07:17:03.413 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T07:17:03.414 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T07:17:03.415 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T07:17:03.416 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T07:17:03.417 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T07:17:03.418 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T07:17:03.419 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T07:17:03.420 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T07:17:03.421 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T07:17:03.422 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T07:17:03.423 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T07:17:03.424 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T07:17:03.425 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T07:17:03.426 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T07:17:03.427 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T07:17:03.428 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T07:17:03.429 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T07:17:03.430 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T07:17:03.431 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T07:17:03.432 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T07:17:03.433 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T07:17:03.434 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T07:17:03.435 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T07:17:03.436 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T07:17:03.437 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T07:17:03.438 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T07:17:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T07:17:03.440 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T07:17:03.441 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T07:17:03.442 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T07:17:03.443 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T07:17:03.444 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T07:17:03.445 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T07:17:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T07:17:03.447 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T07:17:03.448 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T07:17:03.449 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T07:17:03.450 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T07:17:03.451 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T07:17:03.452 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T07:17:03.453 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T07:17:03.454 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T07:17:03.455 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T07:17:03.456 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T07:17:03.456 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T07:17:03.456 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T07:17:03.456 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T07:17:03.456 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T07:17:03.456 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T07:17:03.456 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T07:17:03.456 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T07:17:03.456 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T07:17:03.456 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T07:17:03.457 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T07:17:03.458 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T07:17:03.459 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T07:17:03.460 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T07:17:03.461 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T07:17:03.462 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T07:17:03.463 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T07:17:03.464 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T07:17:03.465 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T07:17:03.466 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T07:17:03.467 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T07:17:03.468 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T07:17:03.469 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T07:17:03.470 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T07:17:03.471 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T07:17:03.472 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T07:17:03.473 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T07:17:03.474 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T07:17:03.475 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T07:17:03.476 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T07:17:03.477 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T07:17:03.478 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T07:17:03.479 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T07:17:03.480 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T07:17:03.481 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T07:17:03.482 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T07:17:03.483 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T07:17:03.484 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T07:17:03.485 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T07:17:03.486 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T07:17:03.487 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T07:17:03.488 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T07:17:03.489 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T07:17:03.490 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T07:17:03.491 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T07:17:03.492 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T07:17:03.493 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T07:17:03.494 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T07:17:03.495 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T07:17:03.496 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T07:17:03.497 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T07:17:03.498 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T07:17:03.499 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T07:17:03.500 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T07:17:03.501 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T07:17:03.502 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T07:17:03.502 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T07:17:03.502 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T07:17:03.502 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T07:17:03.502 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T07:17:03.502 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T07:17:03.502 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T07:17:03.502 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T07:17:03.502 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T07:17:03.503 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T07:17:03.504 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T07:17:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T07:17:03.506 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T07:17:03.507 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T07:17:03.508 INFO:teuthology.orchestra.run.smithi129.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:17:03.512 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T07:17:03.512 DEBUG:teuthology.orchestra.run.smithi129:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T07:17:03.535 INFO:teuthology.orchestra.run.smithi129.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T07:17:03.535 INFO:teuthology.orchestra.run.smithi129.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:17:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:17:03.535 INFO:teuthology.orchestra.run.smithi129.stdout:fi 2024-05-27T07:17:03.535 INFO:teuthology.orchestra.run.smithi129.stdout:for file in vmlinuz System.map config; do 2024-05-27T07:17:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T07:17:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T07:17:03.535 INFO:teuthology.orchestra.run.smithi129.stdout: fi 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout:done 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout:fi 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout:fi 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T07:17:03.536 INFO:teuthology.orchestra.run.smithi129.stdout:fi 2024-05-27T07:17:03.537 DEBUG:teuthology.orchestra.run.smithi129:> sudo rpm -qi grub2-tools 2024-05-27T07:17:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Name : grub2-tools 2024-05-27T07:17:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Epoch : 1 2024-05-27T07:17:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Version : 2.06 2024-05-27T07:17:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Release : 68.el9 2024-05-27T07:17:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Architecture: x86_64 2024-05-27T07:17:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T07:17:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Group : Unspecified 2024-05-27T07:17:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:Size : 8264254 2024-05-27T07:17:03.610 INFO:teuthology.orchestra.run.smithi129.stdout:License : GPLv3+ 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:Packager : builder@centos.org 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:Vendor : CentOS 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:Summary : Support tools for GRUB. 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:Description : 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T07:17:03.611 INFO:teuthology.orchestra.run.smithi129.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T07:17:03.612 INFO:teuthology.orchestra.run.smithi129.stdout:hardware devices. 2024-05-27T07:17:03.612 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:17:03.612 INFO:teuthology.orchestra.run.smithi129.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T07:17:03.613 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T07:17:03.613 INFO:teuthology.task.kernel:Updating grub on smithi129 to boot 6.9.0-gb9459a26b088 2024-05-27T07:17:03.613 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T07:17:04.229 INFO:teuthology.orchestra.run.smithi129.stderr:Generating grub configuration file ... 2024-05-27T07:17:05.697 INFO:teuthology.orchestra.run.smithi129.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T07:17:05.728 INFO:teuthology.orchestra.run.smithi129.stderr:done 2024-05-27T07:17:05.731 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-05-27T07:17:05.768 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.gXhjPKTgOK 2024-05-27T07:17:05.769 DEBUG:teuthology.orchestra.run.smithi129:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gXhjPKTgOK 2024-05-27T07:17:05.837 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.gXhjPKTgOK 2024-05-27T07:17:05.978 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.gXhjPKTgOK is 6KB 2024-05-27T07:17:06.026 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.gXhjPKTgOK 2024-05-27T07:17:06.040 DEBUG:teuthology.orchestra.run.smithi129:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T07:17:06.101 INFO:teuthology.orchestra.run.smithi129.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T07:17:06.101 INFO:teuthology.orchestra.run.smithi129.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T07:17:06.101 INFO:teuthology.orchestra.run.smithi129.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T07:17:06.101 INFO:teuthology.orchestra.run.smithi129.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T07:17:06.102 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T07:17:06.261 DEBUG:teuthology.orchestra.run.smithi129:> sudo shutdown -r now 2024-05-27T07:17:12.964 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi090.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-27T07:17:12.964 DEBUG:teuthology.orchestra.run.smithi090:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-27T07:17:13.056 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/System.map-6.9.0-gb9459a26b088 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/config-6.9.0-gb9459a26b088 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/vmlinuz-6.9.0-gb9459a26b088 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/System.map 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/config 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-27T07:17:13.057 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-27T07:17:13.058 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/arch/x86/kvm/kvm.ko 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ansi_cprng.ko 2024-05-27T07:17:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/anubis.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/arc4.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_memcpy.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_pq.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_tx.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/async_xor.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/async_tx/raid6test.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authenc.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/authencesn.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blake2b_generic.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_common.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/blowfish_generic.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/camellia_generic.ko 2024-05-27T07:17:13.060 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast5_generic.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast6_generic.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cast_common.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/ccm.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cmac.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crc32_generic.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_engine.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/crypto_user.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/cts.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/des_generic.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/dh_generic.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/echainiv.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/essiv.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/fcrypt.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/khazad.ko 2024-05-27T07:17:13.061 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/lz4hc.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/md4.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/michael_mic.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcbc.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/pcrypt.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/rmd160.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/seed.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/serpent_generic.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tcrypt.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/tea.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_common.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/twofish_generic.ko 2024-05-27T07:17:13.062 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/vmac.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/wp512.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xcbc.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xor.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/crypto/xxhash_generic.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/acpi_pad.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/ec_sys.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/platform_profile.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbs.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/sbshc.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/acpi/video.ko 2024-05-27T07:17:13.063 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/acard-ahci.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ahci_platform.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/ata_generic.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/libahci_platform.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_acpi.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ali.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_amd.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_artop.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atiixp.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_atp867x.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd640.ko 2024-05-27T07:17:13.064 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cmd64x.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_cypress.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_efar.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt366.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt37x.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it8213.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_it821x.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_jmicron.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_marvell.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_mpiix.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_netcell.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ninja32.ko 2024-05-27T07:17:13.065 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87410.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_ns87415.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_oldpiix.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_opti.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_optidma.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pcmcia.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_piccolo.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_rdc.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sch.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_serverworks.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sil680.ko 2024-05-27T07:17:13.066 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sis.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_sl82c105.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_triflex.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pata_via.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/pdc_adma.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_inic162x.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_mv.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_nv.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_promise.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_qstor.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sil24.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sis.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_svw.ko 2024-05-27T07:17:13.067 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_sx4.ko 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_uli.ko 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_via.ko 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ata/sata_vsc.ko 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/auxdisplay/ks0108.ko 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/bcma/bcma.ko 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block 2024-05-27T07:17:13.068 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/aoe/aoe.ko 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/brd.ko 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/drbd/drbd.ko 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/floppy.ko 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/loop.ko 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/nbd.ko 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/null_blk/null_blk.ko 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/pktcdvd.ko 2024-05-27T07:17:13.069 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/rbd.ko 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/virtio_blk.ko 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/xen-blkfront.ko 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/block/zram/zram.ko 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hangcheck-timer.ko 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-27T07:17:13.070 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/via-rng.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/lp.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/mwave/mwave.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/ppdev.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tlclk.ko 2024-05-27T07:17:13.071 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/uv_mmtimer.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/char/virtio_console.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-27T07:17:13.072 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/ccp/ccp.ko 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-27T07:17:13.073 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-aes.ko 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/padlock-sha.ko 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dca/dca.ko 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-27T07:17:13.074 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/amd64_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/e752x_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/edac_mce_amd.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3000_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i3200_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5100_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i5400_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7300_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i7core_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/i82975x_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/ie31200_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/sb_edac.ko 2024-05-27T07:17:13.075 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/edac/x38_edac.ko 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/edd.ko 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-ich.ko 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-appleir.ko 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-aureal.ko 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-axff.ko 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-dr.ko 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elecom.ko 2024-05-27T07:17:13.076 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-elo.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-emsff.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gaff.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gt683r.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-gyration.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-holtekff.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-hyperv.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-icade.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-keytouch.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-kye.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lcpower.ko 2024-05-27T07:17:13.077 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-led.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-lenovo.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-multitouch.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-ortek.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-penmount.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-petalynx.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-picolcd.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-pl.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-plantronics.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-primax.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-rmi.ko 2024-05-27T07:17:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-common.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-roccat.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-saitek.ko 2024-05-27T07:17:13.079 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-samsung.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sjoy.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sony.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-speedlink.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-steelseries.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-sunplus.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tivo.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-tmff.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-topseed.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-twinhan.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-uclogic.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-waltop.ko 2024-05-27T07:17:13.080 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-wiimote.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-xinmo.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zpff.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/hid-zydacron.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/uhid.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hid/wacom.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_balloon.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_utils.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hv/hv_vmbus.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/abituguru3.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-27T07:17:13.081 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7414.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ad7418.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adc128d818.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1025.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1026.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1029.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm1031.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adm9240.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ads7828.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7410.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7411.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7462.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7470.ko 2024-05-27T07:17:13.082 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7475.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/adt7x10.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/amc6821.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/applesmc.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asb100.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asc7621.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/atxp1.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/coretemp.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/dme1737.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds1621.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ds620.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc1403.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/emc6w201.ko 2024-05-27T07:17:13.083 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71805f.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f71882fg.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/f75375s.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fam15h_power.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/fschmd.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g760a.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/g762.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl518sm.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/gl520sm.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5500_temp.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/i5k_amb.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmaem.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ibmpex.ko 2024-05-27T07:17:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina209.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ina2xx.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/it87.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k10temp.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/k8temp.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lineage-pem.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm63.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm73.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm75.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm77.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm78.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm80.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm83.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm85.ko 2024-05-27T07:17:13.085 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm87.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm90.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm92.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm93.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95234.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95241.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/lm95245.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc2945.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4151.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4215.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4222.ko 2024-05-27T07:17:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4245.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4260.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/ltc4261.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max16065.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1619.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max1668.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max197.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6639.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6650.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/max6697.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/mcp3021.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6683.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775-core.ko 2024-05-27T07:17:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct6775.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/nct7802.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87360.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pc87427.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pcf8591.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-27T07:17:13.088 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/powr1220.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5627.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch5636.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht15.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sht21.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/shtc1.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/sis5595.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47b397.ko 2024-05-27T07:17:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m1.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/smsc47m192.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/thmc50.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp102.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp103.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp401.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/tmp421.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via-cputemp.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/via686a.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt1211.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/vt8231.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627ehf.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83627hf.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83781d.ko 2024-05-27T07:17:13.090 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83791d.ko 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83792d.ko 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83793.ko 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83795.ko 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l785ts.ko 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/hwmon/w83l786ng.ko 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-27T07:17:13.091 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-27T07:17:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-dev.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-mux.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-smbus.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/i2c/i2c-stub.ko 2024-05-27T07:17:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_core.ko 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4 2024-05-27T07:17:13.094 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-27T07:17:13.095 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/gameport/gameport.ko 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/joydev.ko 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/appletouch.ko 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/bcm5974.ko 2024-05-27T07:17:13.096 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/cyapatp.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/sermouse.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/altera_ps2.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/arc_ps2.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/serio/serio_raw.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/input/sparse-keymap.ko 2024-05-27T07:17:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-blinkm.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lm3530.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-lp3944.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/leds-ss4200.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-27T07:17:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/bcache/bcache.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-bio-prison.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache-smq.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-cache.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-crypt.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-delay.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-flakey.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-log-userspace.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-multipath.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-queue-length.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-raid.ko 2024-05-27T07:17:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-round-robin.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-service-time.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-switch.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-thin-pool.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/dm-verity.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid0.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid1.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid10.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/md/raid456.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptbase.ko 2024-05-27T07:17:13.100 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptctl.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptfc.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptlan.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptsas.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptscsih.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/message/fusion/mptspi.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_ich.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/lpc_sch.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/mfd-core.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/sm501.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/viperboard.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/vx855.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/mfd/wl1273-core.ko 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc 2024-05-27T07:17:13.101 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds9802als.ko 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/apds990x.ko 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/bh1770glc.ko 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/cb710/cb710.ko 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/at24.ko 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/eeprom/max6875.ko 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/enclosure.ko 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/hpilo.ko 2024-05-27T07:17:13.102 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29003.ko 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/isl29020.ko 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-me.ko 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei-txe.ko 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/mei/mei.ko 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-27T07:17:13.103 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_7xx1.ko 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tifm_core.ko 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/tsl2550.ko 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_balloon.ko 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/bonding/bonding.ko 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-27T07:17:13.104 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/dummy.ko 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/eql.ko 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-27T07:17:13.105 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-27T07:17:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-27T07:17:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-27T07:17:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-27T07:17:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/dnet.ko 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ethoc.ko 2024-05-27T07:17:13.110 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/fealnx.ko 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-27T07:17:13.111 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/jme.ko 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox 2024-05-27T07:17:13.112 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-27T07:17:13.113 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-27T07:17:13.114 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-27T07:17:13.115 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-27T07:17:13.116 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-27T07:17:13.117 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/geneve.ko 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ifb.ko 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvlan.ko 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/macvtap.ko 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio 2024-05-27T07:17:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/mii.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/net_failover.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/netconsole.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/nlmon.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/amd.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-27T07:17:13.119 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/bcm87xx.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/broadcom.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/cicada.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/davicom.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/dp83640.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/et1011c.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/icplus.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/lxt.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/marvell.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/micrel.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/national.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/phylink.ko 2024-05-27T07:17:13.120 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/qsemi.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/realtek.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/smsc.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/ste10Xp.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/phy/vitesse.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_async.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppoe.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pppox.ko 2024-05-27T07:17:13.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/ppp/pptp.ko 2024-05-27T07:17:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip 2024-05-27T07:17:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slhc.ko 2024-05-27T07:17:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/slip/slip.ko 2024-05-27T07:17:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/sungem_phy.ko 2024-05-27T07:17:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tap.ko 2024-05-27T07:17:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team 2024-05-27T07:17:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team.ko 2024-05-27T07:17:13.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-27T07:17:13.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-27T07:17:13.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-27T07:17:13.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_random.ko 2024-05-27T07:17:13.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/tun.ko 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/veth.ko 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/virtio_net.ko 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/vxlan/vxlan.ko 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/net/xen-netfront.ko 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb 2024-05-27T07:17:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ntb/ntb.ko 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme-core.ko 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/nvme/host/nvme.ko 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport.ko 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_cs.ko 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_pc.ko 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/parport/parport_serial.ko 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie 2024-05-27T07:17:13.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pci/xen-pcifront.ko 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/i82092.ko 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/pd6729.ko 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/acerhdf.ko 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-27T07:17:13.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/hdaps.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/intel_ips.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-27T07:17:13.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/platform/x86/wmi.ko 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-27T07:17:13.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/clients/pps_parport.ko 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/pps/pps_core.ko 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp.ko 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ptp/ptp_kvm.ko 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ras/amd/fmpm.ko 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-27T07:17:13.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-em3027.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-27T07:17:13.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-max6900.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/rtc/rtc-x1205.ko 2024-05-27T07:17:13.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-9xxx.ko 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-sas.ko 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/3w-xxxx.ko 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/BusLogic.ko 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/a100u2w.ko 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/advansys.ko 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/am53c974.ko 2024-05-27T07:17:13.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/atp870u.ko 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bfa/bfa.ko 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ch.ko 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi 2024-05-27T07:17:13.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dc395x.ko 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/dmx3191d.ko 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r 2024-05-27T07:17:13.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/esp_scsi.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/fnic/fnic.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hpsa.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hptiop.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/hv_storvsc.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/initio.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ipr.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ips.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/isci/isci.ko 2024-05-27T07:17:13.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libfc/libfc.ko 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi.ko 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/libsas/libsas.ko 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid.ko 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-27T07:17:13.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/mvumi.ko 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/pmcraid.ko 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla1280.ko 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx 2024-05-27T07:17:13.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/raid_class.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_debug.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/ses.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/st.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/stex.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-27T07:17:13.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/virtio_scsi.ko 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/scsi/wd719x.ko 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ssb/ssb.ko 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_file.ko 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_iblock.ko 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_mod.ko 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_pscsi.ko 2024-05-27T07:17:13.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/target_core_user.ko 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-27T07:17:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless 2024-05-27T07:17:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_gsm.ko 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/n_hdlc.ko 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/nozomi.ko 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/arc_uart.ko 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host 2024-05-27T07:17:13.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio.ko 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_aec.ko 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_cif.ko 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_pci_generic.ko 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/uio/uio_sercos3.ko 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-acm.ko 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usblp.ko 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/class/usbtmc.ko 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host 2024-05-27T07:17:13.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/mdc800.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/image/microtek.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/adutux.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/appledisplay.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi26.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/emi62.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ezusb.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/idmouse.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/iowarrior.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/ldusb.ko 2024-05-27T07:17:13.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/legousbtower.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usb3503.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usblcd.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/uss720.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/misc/yurex.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/phy/phy-generic.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/aircable.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ark3116.ko 2024-05-27T07:17:13.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ch341.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cp210x.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cyberjack.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/empeg.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/io_ti.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipaq.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ipw.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ir-usb.ko 2024-05-27T07:17:13.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mct_u232.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7720.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/mos7840.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/navman.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/omninet.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/opticon.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/option.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/oti6858.ko 2024-05-27T07:17:13.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/pl2303.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcaux.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/qcserial.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/quatech2.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/safe_serial.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/sierra.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ssu100.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/symbolserial.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_debug.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-27T07:17:13.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/visor.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/whiteheat.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/uas.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-karma.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-27T07:17:13.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/storage/usb-storage.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio.ko 2024-05-27T07:17:13.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost.ko 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_net.ko 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/vhost/vhost_scsi.ko 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/apple_bl.ko 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/lcd.ko 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio.ko 2024-05-27T07:17:13.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_balloon.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_mmio.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/virtio/virtio_ring.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2482.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/masters/ds2490.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-27T07:17:13.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/w1/wire.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-27T07:17:13.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/hpwdt.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/i6300esb.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ib700wdt.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ibmasr.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/it87_wdt.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/machzwd.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/nv_tco.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-27T07:17:13.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/softdog.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/via_wdt.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/wdt_pci.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/watchdog/xen_wdt.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-evtchn.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntalloc.ko 2024-05-27T07:17:13.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-gntdev.ko 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xen-privcmd.ko 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/9p/9p.ko 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/affs/affs.ko 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/befs/befs.ko 2024-05-27T07:17:13.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/binfmt_misc.ko 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/btrfs/btrfs.ko 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cachefiles/cachefiles.ko 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ceph/ceph.ko 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/coda/coda.ko 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/cramfs/cramfs.ko 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/dlm/dlm.ko 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs 2024-05-27T07:17:13.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/f2fs/f2fs.ko 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/fat.ko 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/msdos.ko 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fat/vfat.ko 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/cuse.ko 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/fuse/fuse.ko 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/gfs2/gfs2.ko 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs 2024-05-27T07:17:13.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfs/hfs.ko 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/hfsplus/hfsplus.ko 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/isofs/isofs.ko 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/jfs/jfs.ko 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/lockd/lockd.ko 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/minix/minix.ko 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/netfs/netfs.ko 2024-05-27T07:17:13.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfs.ko 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv3.ko 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs/nfsv4.ko 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/grace.ko 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfs_common/nfs_acl.ko 2024-05-27T07:17:13.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nfsd/nfsd.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nilfs2/nilfs2.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-celtic.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-centeuro.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-croatian.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-cyrillic.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-gaelic.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-greek.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-iceland.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-inuit.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-roman.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-romanian.ko 2024-05-27T07:17:13.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/mac-turkish.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1250.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1251.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp1255.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp737.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp775.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp850.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp852.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp855.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp857.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp860.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp861.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp862.ko 2024-05-27T07:17:13.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp863.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp864.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp865.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp866.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp869.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp874.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp932.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp936.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp949.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_cp950.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_euc-jp.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-1.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-13.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-14.ko 2024-05-27T07:17:13.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-15.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-2.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-3.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-4.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-5.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-6.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-7.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_iso8859-9.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-r.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-ru.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_koi8-u.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/nls/nls_utf8.ko 2024-05-27T07:17:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2.ko 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/overlayfs/overlay.ko 2024-05-27T07:17:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/pstore/ramoops.ko 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/reiserfs/reiserfs.ko 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/romfs/romfs.ko 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/client/cifs.ko 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_arc4.ko 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/smb/common/cifs_md4.ko 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/squashfs/squashfs.ko 2024-05-27T07:17:13.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/sysv/sysv.ko 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/udf/udf.ko 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/ufs/ufs.ko 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/fs/xfs/xfs.ko 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/rcu/rcutorture.ko 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/torture.ko 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace 2024-05-27T07:17:13.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/asn1_encoder.ko 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc-itu-t.ko 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crc8.ko 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libarc4.ko 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/crypto/libdes.ko 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/libcrc32c.ko 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lru_cache.ko 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4_compress.ko 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/lz4/lz4hc_compress.ko 2024-05-27T07:17:13.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/math/cordic.ko 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/raid6/raid6_pq.ko 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_bm.ko 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_fsm.ko 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/ts_kmp.ko 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/lib/zstd/zstd_compress.ko 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/mm/hwpoison-inject.ko 2024-05-27T07:17:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/6lowpan.ko 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_dest.ko 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_fragment.ko 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_hop.ko 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_mobility.ko 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_routing.ko 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/6lowpan/nhc_udp.ko 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/garp.ko 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/mrp.ko 2024-05-27T07:17:13.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/p8022.ko 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/psnap.ko 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/802/stp.ko 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/8021q/8021q.ko 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet.ko 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_fd.ko 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_rdma.ko 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/9p/9pnet_virtio.ko 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/batman-adv/batman-adv.ko 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge 2024-05-27T07:17:13.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/br_netfilter.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/bridge.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-27T07:17:13.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/ebtables.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-27T07:17:13.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ceph/libceph.ko 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/failover.ko 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/core/pktgen.ko 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp.ko 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_diag.ko 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv4.ko 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dccp/dccp_ipv6.ko 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dns_resolver/dns_resolver.ko 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa 2024-05-27T07:17:13.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/dsa_core.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_brcm.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_dsa.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_none.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/dsa/tag_trailer.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ah4.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/esp4.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/fou.ko 2024-05-27T07:17:13.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/gre.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/inet_diag.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_gre.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_tunnel.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ip_vti.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipcomp.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/ipip.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-27T07:17:13.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-27T07:17:13.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_bic.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_dctcp.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_diag.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_highspeed.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_htcp.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_hybla.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_illinois.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_lp.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_scalable.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_vegas.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_veno.ko 2024-05-27T07:17:13.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_westwood.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tcp_yeah.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/tunnel4.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_diag.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/udp_tunnel.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ah6.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/esp6.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/fou6.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_tunnel.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ip6_vti.ko 2024-05-27T07:17:13.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/ipcomp6.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-27T07:17:13.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/sit.ko 2024-05-27T07:17:13.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/tunnel6.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/key/af_key.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_core.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_eth.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ip6.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_netlink.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/l2tp/l2tp_ppp.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/llc/llc.ko 2024-05-27T07:17:13.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mac802154/mac802154.ko 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/mpls/mpls_gso.ko 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set.ko 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-27T07:17:13.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-27T07:17:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conncount.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-27T07:17:13.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table.ko 2024-05-27T07:17:13.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_log_syslog.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_irc.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_sip.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nf_tables.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_log.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-27T07:17:13.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_chain_nat.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_compat.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_ct.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fib.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_hash.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_limit.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_log.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_masq.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_nat.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_numgen.ko 2024-05-27T07:17:13.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_queue.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_quota.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_redir.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_reject_inet.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_socket.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/nft_tunnel.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_AUDIT.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_CT.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_DSCP.ko 2024-05-27T07:17:13.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HL.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_HMARK.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LED.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_LOG.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NETMAP.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFLOG.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_RATEEST.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_SECMARK.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-27T07:17:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TEE.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TPROXY.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_TRACE.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_addrtype.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_bpf.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cgroup.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cluster.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_comment.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connbytes.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlabel.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connlimit.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_connmark.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_conntrack.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_cpu.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dccp.ko 2024-05-27T07:17:13.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_devgroup.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_dscp.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ecn.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_esp.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hashlimit.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_helper.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_hl.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipcomp.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_iprange.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_ipvs.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_l2tp.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_length.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_limit.ko 2024-05-27T07:17:13.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mac.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_mark.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_multiport.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nat.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_nfacct.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_osf.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_owner.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_physdev.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_pkttype.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_policy.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_quota.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_rateest.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_realm.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_recent.ko 2024-05-27T07:17:13.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_sctp.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_set.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_socket.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_state.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_statistic.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_string.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_tcpmss.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_time.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netfilter/xt_u32.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/netlink/netlink_diag.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/nsh/nsh.ko 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch 2024-05-27T07:17:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/openvswitch.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-geneve.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-gre.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/openvswitch/vport-vxlan.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/packet/af_packet_diag.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_rdma.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/rds/rds_tcp.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_csum.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_gact.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_mirred.ko 2024-05-27T07:17:13.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_nat.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_pedit.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_police.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_simple.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_skbedit.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/act_vlan.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_basic.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_bpf.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_flow.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_fw.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_route.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/cls_u32.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_cmp.ko 2024-05-27T07:17:13.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_ipset.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_meta.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_nbyte.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_text.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/em_u32.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_choke.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_codel.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_drr.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_fq.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_gred.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hfsc.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_hhf.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_htb.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_ingress.ko 2024-05-27T07:17:13.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_mqprio_lib.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_multiq.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_netem.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_pie.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_plug.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_prio.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_qfq.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_red.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfb.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_sfq.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_tbf.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sched/sch_teql.ko 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp 2024-05-27T07:17:13.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp.ko 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sctp/sctp_diag.ko 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/sunrpc.ko 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc.ko 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/tipc/tipc_diag.ko 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/unix/unix_diag.ko 2024-05-27T07:17:13.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock.ko 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys 2024-05-27T07:17:13.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/kernel/security/keys/trusted-keys/trusted.ko 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.alias.bin 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.alias.bin 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.bin 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.builtin.modinfo 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.dep.bin 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.devname 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.order 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.softdep 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/modules.symbols.bin 2024-05-27T07:17:13.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:17:13.207 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-gb9459a26b088 2024-05-27T07:17:13.207 DEBUG:teuthology.orchestra.run.smithi090:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-27T07:17:13.231 INFO:teuthology.orchestra.run.smithi090.stdout:postinstall scriptlet (using /bin/sh): 2024-05-27T07:17:13.231 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:17:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/bin/kernel-install add 6.9.0-gb9459a26b088 /lib/modules/6.9.0-gb9459a26b088/vmlinuz 2024-05-27T07:17:13.231 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-05-27T07:17:13.231 INFO:teuthology.orchestra.run.smithi090.stdout:for file in vmlinuz System.map config; do 2024-05-27T07:17:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: if ! cmp --silent "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088"; then 2024-05-27T07:17:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: cp "/lib/modules/6.9.0-gb9459a26b088/${file}" "/boot/${file}-6.9.0-gb9459a26b088" 2024-05-27T07:17:13.231 INFO:teuthology.orchestra.run.smithi090.stdout: fi 2024-05-27T07:17:13.231 INFO:teuthology.orchestra.run.smithi090.stdout:done 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -d "/lib/modules/6.9.0-gb9459a26b088/dtb" ] && \ 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: ! diff -rq "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" >/dev/null 2>&1; then 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: rm -rf "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: cp -r "/lib/modules/6.9.0-gb9459a26b088/dtb" "/boot/dtb-6.9.0-gb9459a26b088" 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout:if [ ! -e "/lib/modules/6.9.0-gb9459a26b088/modules.dep" ]; then 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/sbin/depmod 6.9.0-gb9459a26b088 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-install remove 6.9.0-gb9459a26b088 2024-05-27T07:17:13.232 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-05-27T07:17:13.233 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -qi grub2-tools 2024-05-27T07:17:13.306 INFO:teuthology.orchestra.run.smithi090.stdout:Name : grub2-tools 2024-05-27T07:17:13.306 INFO:teuthology.orchestra.run.smithi090.stdout:Epoch : 1 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Version : 2.06 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Release : 68.el9 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: x86_64 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Group : Unspecified 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Size : 8264254 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:License : GPLv3+ 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Packager : builder@centos.org 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:Vendor : CentOS 2024-05-27T07:17:13.307 INFO:teuthology.orchestra.run.smithi090.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T07:17:13.308 INFO:teuthology.orchestra.run.smithi090.stdout:Summary : Support tools for GRUB. 2024-05-27T07:17:13.308 INFO:teuthology.orchestra.run.smithi090.stdout:Description : 2024-05-27T07:17:13.308 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:17:13.308 INFO:teuthology.orchestra.run.smithi090.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T07:17:13.308 INFO:teuthology.orchestra.run.smithi090.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T07:17:13.308 INFO:teuthology.orchestra.run.smithi090.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T07:17:13.308 INFO:teuthology.orchestra.run.smithi090.stdout:hardware devices. 2024-05-27T07:17:13.308 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:17:13.308 INFO:teuthology.orchestra.run.smithi090.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T07:17:13.309 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T07:17:13.309 INFO:teuthology.task.kernel:Updating grub on smithi090 to boot 6.9.0-gb9459a26b088 2024-05-27T07:17:13.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T07:17:13.924 INFO:teuthology.orchestra.run.smithi090.stderr:Generating grub configuration file ... 2024-05-27T07:17:15.632 INFO:teuthology.orchestra.run.smithi090.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T07:17:15.656 INFO:teuthology.orchestra.run.smithi090.stderr:done 2024-05-27T07:17:15.659 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-05-27T07:17:15.685 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.uCiZz9uqVF 2024-05-27T07:17:15.685 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uCiZz9uqVF 2024-05-27T07:17:15.758 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.uCiZz9uqVF 2024-05-27T07:17:15.910 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.uCiZz9uqVF is 6KB 2024-05-27T07:17:15.957 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.uCiZz9uqVF 2024-05-27T07:17:15.971 DEBUG:teuthology.orchestra.run.smithi090:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T07:17:16.040 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T07:17:16.041 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T07:17:16.041 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T07:17:16.041 INFO:teuthology.orchestra.run.smithi090.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088.conf 2024-05-27T07:17:16.042 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-gb9459a26b088 2024-05-27T07:17:16.183 DEBUG:teuthology.orchestra.run.smithi090:> sudo shutdown -r now 2024-05-27T07:17:20.689 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-05-27T07:17:20.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-05-27T07:17:20.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:17:36.278 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-27T07:17:36.279 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-05-27T07:17:36.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:17:39.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-05-27T07:17:46.193 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-27T07:17:46.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-05-27T07:17:46.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:17:48.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-05-27T07:17:48.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:17:51.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-05-27T07:17:54.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-05-27T07:18:03.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-05-27T07:18:03.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:18:03.848 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-05-27T07:18:03.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:18:04.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-05-27T07:18:06.978 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-05-27T07:18:13.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-05-27T07:18:13.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:18:16.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-05-27T07:18:18.982 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-05-27T07:18:18.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:18:21.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-05-27T07:18:28.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-05-27T07:18:28.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:18:31.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-05-27T07:18:36.733 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-05-27T07:18:36.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:18:37.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-05-27T07:18:38.494 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-05-27T07:18:39.156 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-05-27T07:18:39.156 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.9.0-gb9459a26b088"... 2024-05-27T07:18:39.156 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-05-27T07:18:39.198 INFO:teuthology.orchestra.run.smithi136.stdout:6.9.0-gb9459a26b088 2024-05-27T07:18:39.198 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T07:18:39.198 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T07:18:39.198 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T07:18:40.198 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-27T07:18:40.199 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T07:18:40.371 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2024-05-27T07:18:40.426 DEBUG:teuthology.parallel:result is None 2024-05-27T07:18:46.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-05-27T07:18:46.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:18:52.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-05-27T07:18:52.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:18:55.337 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-05-27T07:18:55.927 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-05-27T07:18:55.927 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-gb9459a26b088"... 2024-05-27T07:18:55.927 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-05-27T07:18:55.967 INFO:teuthology.orchestra.run.smithi129.stdout:6.9.0-gb9459a26b088 2024-05-27T07:18:55.968 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T07:18:55.968 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T07:18:55.968 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T07:18:56.968 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-27T07:18:56.969 DEBUG:teuthology.orchestra.run.smithi129:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T07:18:57.077 INFO:teuthology.orchestra.run.smithi129.stdout:ttyS1 2024-05-27T07:18:57.147 DEBUG:teuthology.parallel:result is None 2024-05-27T07:19:05.246 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-05-27T07:19:23.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-05-27T07:19:23.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:19:23.723 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-05-27T07:19:24.687 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-05-27T07:19:24.687 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-gb9459a26b088"... 2024-05-27T07:19:24.687 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-05-27T07:19:24.730 INFO:teuthology.orchestra.run.smithi090.stdout:6.9.0-gb9459a26b088 2024-05-27T07:19:24.730 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-gb9459a26b088 vs 6.9.0-gb9459a26b088 2024-05-27T07:19:24.730 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T07:19:24.730 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T07:19:25.731 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-27T07:19:25.731 DEBUG:teuthology.orchestra.run.smithi090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-27T07:19:25.969 INFO:teuthology.orchestra.run.smithi090.stdout:ttyS1 2024-05-27T07:19:26.028 DEBUG:teuthology.parallel:result is None 2024-05-27T07:19:26.028 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-27T07:19:26.036 INFO:teuthology.task.internal:Creating test directory... 2024-05-27T07:19:26.037 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T07:19:26.044 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T07:19:26.053 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T07:19:26.113 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-27T07:19:26.119 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-27T07:19:26.147 INFO:teuthology.task.internal:Creating archive directory... 2024-05-27T07:19:26.147 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T07:19:26.153 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T07:19:26.160 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T07:19:26.243 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-27T07:19:26.251 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-27T07:19:26.251 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T07:19:26.256 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T07:19:26.263 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T07:19:26.330 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:19:26.351 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:19:26.358 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:19:26.363 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:19:26.384 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:19:26.396 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T07:19:26.400 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-27T07:19:26.408 INFO:teuthology.task.internal:Configuring sudo... 2024-05-27T07:19:26.408 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T07:19:26.413 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T07:19:26.421 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T07:19:26.503 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-27T07:19:26.511 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-27T07:19:26.511 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T07:19:26.517 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T07:19:26.524 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T07:19:26.582 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:19:26.674 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:19:26.758 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:19:26.829 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:19:26.897 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:19:26.897 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T07:19:26.958 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:19:27.046 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:19:27.148 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:19:27.232 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:19:27.318 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:19:27.318 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T07:19:27.400 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:19:27.486 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T07:19:27.588 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:19:27.670 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T07:19:27.756 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:19:27.756 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T07:19:27.839 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-05-27T07:19:27.845 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-05-27T07:19:27.852 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-05-27T07:19:27.944 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T07:19:27.967 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T07:19:27.990 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T07:19:28.398 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-27T07:19:28.406 INFO:teuthology.task.internal:Starting timer... 2024-05-27T07:19:28.406 INFO:teuthology.run_tasks:Running task pcp... 2024-05-27T07:19:28.445 INFO:teuthology.run_tasks:Running task selinux... 2024-05-27T07:19:28.478 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-27T07:19:28.478 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-05-27T07:19:28.677 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-05-27T07:19:28.683 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-05-27T07:19:28.848 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-05-27T07:19:28.853 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-05-27T07:19:28.983 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-05-27T07:19:28.989 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-27T07:19:28.989 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-05-27T07:19:29.046 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-05-27T07:19:29.046 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-05-27T07:19:29.104 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-05-27T07:19:29.104 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-05-27T07:19:29.163 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-05-27T07:19:29.164 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi129.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive'} 2024-05-27T07:19:29.164 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T07:19:29.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:19:29.234 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T07:19:29.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:19:29.313 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T07:19:29.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T07:19:29.389 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-27T07:19:29.389 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-05-27T07:19:29.476 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-05-27T07:19:29.551 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-05-27T07:19:29.627 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-27T07:19:29.637 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-27T07:19:29.639 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-27T07:19:29.758 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-27T07:19:29.768 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-27T07:19:29.769 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com,smithi129.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-27T07:26:00.451 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2024-05-27T07:26:00.453 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-05-27T07:26:00.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:26:00.566 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-05-27T07:26:00.728 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-05-27T07:26:00.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-05-27T07:26:00.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:26:00.834 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-05-27T07:26:00.974 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-05-27T07:26:00.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-05-27T07:26:00.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T07:26:01.077 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-05-27T07:26:01.234 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-05-27T07:26:01.235 INFO:teuthology.run_tasks:Running task clock... 2024-05-27T07:26:01.245 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-27T07:26:01.245 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T07:26:01.246 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T07:26:01.254 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T07:26:01.254 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T07:26:01.261 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T07:26:01.261 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T07:26:01.342 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T07:26:01.348 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T07:26:01.351 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T07:26:01.384 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T07:26:01.392 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T07:26:01.393 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T07:26:01.457 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-05-27T07:26:01.465 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-05-27T07:26:01.466 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-05-27T07:26:01.485 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-05-27T07:26:01.496 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-05-27T07:26:01.497 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-05-27T07:26:01.525 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T07:26:01.536 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T07:26:01.538 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T07:26:01.561 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T07:26:01.577 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T07:26:01.578 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T07:26:01.694 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-05-27T07:26:01.701 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T07:26:01.702 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-05-27T07:26:01.702 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.702 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.702 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.702 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.716 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-05-27T07:26:01.718 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-05-27T07:26:01.724 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T07:26:01.724 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-05-27T07:26:01.724 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.724 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.724 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.724 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.727 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T07:26:01.727 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-05-27T07:26:01.727 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.727 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.728 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.728 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T07:26:01.729 INFO:teuthology.run_tasks:Running task install... 2024-05-27T07:26:01.737 DEBUG:teuthology.task.install:project ceph 2024-05-27T07:26:01.737 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6'}} 2024-05-27T07:26:01.737 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6'} 2024-05-27T07:26:01.738 INFO:teuthology.task.install:Using flavor: default 2024-05-27T07:26:01.743 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-27T07:26:01.743 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-27T07:26:01.744 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'tag': None, 'wait_for_package': False} 2024-05-27T07:26:01.744 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:26:01.746 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'tag': None, 'wait_for_package': False} 2024-05-27T07:26:01.747 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:26:01.747 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'tag': None, 'wait_for_package': False} 2024-05-27T07:26:01.748 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:26:01.891 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/ 2024-05-27T07:26:01.892 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3872.g38f9d900 2024-05-27T07:26:01.904 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/ 2024-05-27T07:26:01.904 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3872.g38f9d900 2024-05-27T07:26:01.969 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/ 2024-05-27T07:26:01.969 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3872.g38f9d900 2024-05-27T07:26:02.003 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T07:26:02.003 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:26:02.004 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T07:26:02.019 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T07:26:02.019 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:26:02.019 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T07:26:02.095 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T07:26:02.095 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T07:26:02.101 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T07:26:02.102 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T07:26:02.143 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T07:26:02.143 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:26:02.143 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T07:26:02.223 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T07:26:02.231 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T07:26:02.234 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T07:26:02.234 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T07:26:02.331 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T07:26:02.337 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T07:26:02.362 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T07:26:02.408 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-05-27T07:26:02.410 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-05-27T07:26:02.412 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-05-27T07:26:02.414 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-05-27T07:26:02.473 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T07:26:02.551 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-05-27T07:26:02.555 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-05-27T07:26:02.721 INFO:teuthology.orchestra.run.smithi136.stdout:45 files removed 2024-05-27T07:26:02.726 INFO:teuthology.orchestra.run.smithi090.stdout:45 files removed 2024-05-27T07:26:02.760 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T07:26:02.764 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T07:26:02.868 INFO:teuthology.orchestra.run.smithi129.stdout:45 files removed 2024-05-27T07:26:02.906 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T07:26:03.580 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 216 kB/s | 84 kB 00:00 2024-05-27T07:26:03.702 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 168 kB/s | 84 kB 00:00 2024-05-27T07:26:03.809 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 178 kB/s | 84 kB 00:00 2024-05-27T07:26:04.039 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-05-27T07:26:04.153 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-05-27T07:26:04.285 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-27T07:26:04.581 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2024-05-27T07:26:04.720 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-05-27T07:26:04.844 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-05-27T07:26:05.230 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-27T07:26:05.534 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-27T07:26:07.248 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-27T07:26:09.160 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 8.0 MB/s | 19 MB 00:02 2024-05-27T07:26:09.852 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 1.6 MB/s | 8.1 MB 00:05 2024-05-27T07:26:11.806 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-05-27T07:26:13.196 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 19 MB 00:02 2024-05-27T07:26:13.732 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 17 kB 00:00 2024-05-27T07:26:14.019 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.2 MB 00:00 2024-05-27T07:26:15.669 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 17 kB 00:00 2024-05-27T07:26:16.281 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 21 MB 00:02 2024-05-27T07:26:17.919 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-27T07:26:20.363 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.2 MB 00:03 2024-05-27T07:26:22.517 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-05-27T07:26:22.983 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 17 kB 00:01 2024-05-27T07:26:24.088 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-05-27T07:26:24.268 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-05-27T07:26:25.010 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:26:25.012 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:26:25.032 INFO:teuthology.orchestra.run.smithi136.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T07:26:25.033 INFO:teuthology.orchestra.run.smithi136.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T07:26:25.034 INFO:teuthology.orchestra.run.smithi136.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T07:26:25.036 INFO:teuthology.orchestra.run.smithi136.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T07:26:25.038 INFO:teuthology.orchestra.run.smithi136.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T07:26:25.041 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T07:26:25.046 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T07:26:25.114 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-05-27T07:26:25.140 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-05-27T07:26:25.140 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-05-27T07:26:25.140 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-05-27T07:26:25.140 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-05-27T07:26:25.140 INFO:teuthology.orchestra.run.smithi136.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T07:26:25.140 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 6.6 k 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.4 M 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 863 k 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 147 k 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 1.5 M 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 143 k 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 3.5 M 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 7.4 M 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 49 k 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 10 M 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 43 M 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 273 k 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 764 k 2024-05-27T07:26:25.141 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 224 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 27 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 18 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 32 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 755 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 127 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T07:26:25.142 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 161 k 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 320 k 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 300 k 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 99 k 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 86 k 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.0 M 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 169 k 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T07:26:25.143 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.3 M 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.1 M 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-05-27T07:26:25.144 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T07:26:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T07:26:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T07:26:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 20 M 2024-05-27T07:26:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 34 k 2024-05-27T07:26:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 2.3 M 2024-05-27T07:26:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 274 k 2024-05-27T07:26:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 4.9 M 2024-05-27T07:26:25.145 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 18 M 2024-05-27T07:26:25.146 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 16 k 2024-05-27T07:26:25.146 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 25 k 2024-05-27T07:26:25.146 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T07:26:25.146 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T07:26:25.146 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T07:26:25.146 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T07:26:25.146 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T07:26:25.146 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T07:26:25.146 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T07:26:25.146 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 157 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 498 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.2 M 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T07:26:25.147 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T07:26:25.148 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 45 k 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 136 k 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T07:26:25.149 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T07:26:25.150 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T07:26:25.150 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T07:26:25.150 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T07:26:25.150 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T07:26:25.150 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T07:26:25.150 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T07:26:25.150 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T07:26:25.151 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T07:26:25.151 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T07:26:25.151 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T07:26:25.151 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T07:26:25.151 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T07:26:25.151 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T07:26:25.151 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T07:26:25.152 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T07:26:25.152 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T07:26:25.152 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T07:26:25.152 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T07:26:25.152 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T07:26:25.152 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T07:26:25.152 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T07:26:25.153 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T07:26:25.153 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T07:26:25.153 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T07:26:25.153 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T07:26:25.153 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T07:26:25.153 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T07:26:25.153 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T07:26:25.154 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T07:26:25.154 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T07:26:25.154 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T07:26:25.154 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T07:26:25.154 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T07:26:25.154 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T07:26:25.155 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T07:26:25.155 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T07:26:25.155 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T07:26:25.155 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T07:26:25.155 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T07:26:25.155 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T07:26:25.155 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T07:26:25.156 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T07:26:25.156 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T07:26:25.156 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T07:26:25.156 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T07:26:25.156 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T07:26:25.156 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T07:26:25.156 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T07:26:25.156 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T07:26:25.157 INFO:teuthology.orchestra.run.smithi136.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T07:26:25.157 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T07:26:25.157 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T07:26:25.157 INFO:teuthology.orchestra.run.smithi136.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T07:26:25.157 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T07:26:25.157 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T07:26:25.157 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-05-27T07:26:25.158 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T07:26:25.158 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:26:25.158 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-05-27T07:26:25.158 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-05-27T07:26:25.158 INFO:teuthology.orchestra.run.smithi136.stdout:Install 148 Packages 2024-05-27T07:26:25.158 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 11 Packages 2024-05-27T07:26:25.158 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:26:25.159 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 219 M 2024-05-27T07:26:25.159 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-05-27T07:26:26.393 INFO:teuthology.orchestra.run.smithi136.stdout:(1/159): ceph-19.0.0-3872.g38f9d900.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-05-27T07:26:26.505 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:26:26.507 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:26:26.527 INFO:teuthology.orchestra.run.smithi129.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T07:26:26.529 INFO:teuthology.orchestra.run.smithi129.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T07:26:26.530 INFO:teuthology.orchestra.run.smithi129.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T07:26:26.532 INFO:teuthology.orchestra.run.smithi129.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T07:26:26.534 INFO:teuthology.orchestra.run.smithi129.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T07:26:26.537 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T07:26:26.538 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T07:26:26.610 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-05-27T07:26:26.633 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 6.6 k 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.4 M 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 863 k 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 147 k 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 1.5 M 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 143 k 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 3.5 M 2024-05-27T07:26:26.634 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 7.4 M 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 49 k 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 10 M 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 43 M 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 273 k 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 764 k 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 224 k 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 27 k 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 18 k 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T07:26:26.635 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 32 k 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 755 k 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 127 k 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 161 k 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 320 k 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 300 k 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 99 k 2024-05-27T07:26:26.636 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 86 k 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.0 M 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 169 k 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.3 M 2024-05-27T07:26:26.637 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.1 M 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 20 M 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 34 k 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 2.3 M 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 274 k 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 4.9 M 2024-05-27T07:26:26.638 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 18 M 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 16 k 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 25 k 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 157 k 2024-05-27T07:26:26.639 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 498 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.2 M 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T07:26:26.640 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 45 k 2024-05-27T07:26:26.641 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 136 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T07:26:26.642 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T07:26:26.643 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T07:26:26.644 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T07:26:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:26:26.646 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-05-27T07:26:26.647 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-05-27T07:26:26.647 INFO:teuthology.orchestra.run.smithi129.stdout:Install 148 Packages 2024-05-27T07:26:26.647 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 11 Packages 2024-05-27T07:26:26.647 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:26:26.647 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 219 M 2024-05-27T07:26:26.647 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-05-27T07:26:26.985 INFO:teuthology.orchestra.run.smithi136.stdout:(2/159): ceph-fuse-19.0.0-3872.g38f9d900.el9.x8 1.4 MB/s | 863 kB 00:00 2024-05-27T07:26:27.093 INFO:teuthology.orchestra.run.smithi136.stdout:(3/159): ceph-immutable-object-cache-19.0.0-387 1.3 MB/s | 147 kB 00:00 2024-05-27T07:26:27.431 INFO:teuthology.orchestra.run.smithi129.stdout:(1/159): ceph-19.0.0-3872.g38f9d900.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-05-27T07:26:27.452 INFO:teuthology.orchestra.run.smithi136.stdout:(4/159): ceph-common-19.0.0-3872.g38f9d900.el9. 14 MB/s | 20 MB 00:01 2024-05-27T07:26:27.585 INFO:teuthology.orchestra.run.smithi136.stdout:(5/159): ceph-mgr-19.0.0-3872.g38f9d900.el9.x86 11 MB/s | 1.5 MB 00:00 2024-05-27T07:26:27.944 INFO:teuthology.orchestra.run.smithi136.stdout:(6/159): ceph-mon-19.0.0-3872.g38f9d900.el9.x86 14 MB/s | 4.9 MB 00:00 2024-05-27T07:26:27.948 INFO:teuthology.orchestra.run.smithi129.stdout:(2/159): ceph-fuse-19.0.0-3872.g38f9d900.el9.x8 1.6 MB/s | 863 kB 00:00 2024-05-27T07:26:28.011 INFO:teuthology.orchestra.run.smithi136.stdout:(7/159): ceph-mds-19.0.0-3872.g38f9d900.el9.x86 2.5 MB/s | 2.3 MB 00:00 2024-05-27T07:26:28.381 INFO:teuthology.orchestra.run.smithi129.stdout:(3/159): ceph-immutable-object-cache-19.0.0-387 340 kB/s | 147 kB 00:00 2024-05-27T07:26:28.553 INFO:teuthology.orchestra.run.smithi136.stdout:(8/159): ceph-base-19.0.0-3872.g38f9d900.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-05-27T07:26:28.653 INFO:teuthology.orchestra.run.smithi136.stdout:(9/159): ceph-selinux-19.0.0-3872.g38f9d900.el9 254 kB/s | 25 kB 00:00 2024-05-27T07:26:29.137 INFO:teuthology.orchestra.run.smithi136.stdout:(10/159): ceph-osd-19.0.0-3872.g38f9d900.el9.x8 15 MB/s | 18 MB 00:01 2024-05-27T07:26:29.212 INFO:teuthology.orchestra.run.smithi136.stdout:(11/159): cephfs-mirror-19.0.0-3872.g38f9d900.e 3.0 MB/s | 224 kB 00:00 2024-05-27T07:26:29.263 INFO:teuthology.orchestra.run.smithi136.stdout:(12/159): cephfs-shell-19.0.0-3872.g38f9d900.el 552 kB/s | 27 kB 00:00 2024-05-27T07:26:29.315 INFO:teuthology.orchestra.run.smithi129.stdout:(4/159): ceph-base-19.0.0-3872.g38f9d900.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-05-27T07:26:29.329 INFO:teuthology.orchestra.run.smithi136.stdout:(13/159): libcephfs-devel-19.0.0-3872.g38f9d900 482 kB/s | 32 kB 00:00 2024-05-27T07:26:29.382 INFO:teuthology.orchestra.run.smithi129.stdout:(5/159): ceph-mds-19.0.0-3872.g38f9d900.el9.x86 2.3 MB/s | 2.3 MB 00:00 2024-05-27T07:26:29.456 INFO:teuthology.orchestra.run.smithi136.stdout:(14/159): libcephfs2-19.0.0-3872.g38f9d900.el9. 5.9 MB/s | 755 kB 00:00 2024-05-27T07:26:29.530 INFO:teuthology.orchestra.run.smithi136.stdout:(15/159): libcephsqlite-19.0.0-3872.g38f9d900.e 2.1 MB/s | 157 kB 00:00 2024-05-27T07:26:29.605 INFO:teuthology.orchestra.run.smithi136.stdout:(16/159): librados-devel-19.0.0-3872.g38f9d900. 1.7 MB/s | 127 kB 00:00 2024-05-27T07:26:29.615 INFO:teuthology.orchestra.run.smithi129.stdout:(6/159): ceph-mgr-19.0.0-3872.g38f9d900.el9.x86 5.0 MB/s | 1.5 MB 00:00 2024-05-27T07:26:29.697 INFO:teuthology.orchestra.run.smithi136.stdout:(17/159): libradosstriper1-19.0.0-3872.g38f9d90 5.3 MB/s | 498 kB 00:00 2024-05-27T07:26:30.289 INFO:teuthology.orchestra.run.smithi136.stdout:(18/159): librgw2-19.0.0-3872.g38f9d900.el9.x86 8.7 MB/s | 5.2 MB 00:00 2024-05-27T07:26:30.348 INFO:teuthology.orchestra.run.smithi136.stdout:(19/159): python3-ceph-argparse-19.0.0-3872.g38 786 kB/s | 45 kB 00:00 2024-05-27T07:26:30.377 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-05-27T07:26:30.415 INFO:teuthology.orchestra.run.smithi136.stdout:(20/159): python3-ceph-common-19.0.0-3872.g38f9 2.0 MB/s | 136 kB 00:00 2024-05-27T07:26:30.498 INFO:teuthology.orchestra.run.smithi136.stdout:(21/159): python3-cephfs-19.0.0-3872.g38f9d900. 1.9 MB/s | 161 kB 00:00 2024-05-27T07:26:30.582 INFO:teuthology.orchestra.run.smithi136.stdout:(22/159): python3-rados-19.0.0-3872.g38f9d900.e 3.8 MB/s | 320 kB 00:00 2024-05-27T07:26:30.682 INFO:teuthology.orchestra.run.smithi136.stdout:(23/159): python3-rbd-19.0.0-3872.g38f9d900.el9 3.0 MB/s | 300 kB 00:00 2024-05-27T07:26:30.758 INFO:teuthology.orchestra.run.smithi136.stdout:(24/159): python3-rgw-19.0.0-3872.g38f9d900.el9 1.3 MB/s | 99 kB 00:00 2024-05-27T07:26:30.825 INFO:teuthology.orchestra.run.smithi136.stdout:(25/159): rbd-fuse-19.0.0-3872.g38f9d900.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-27T07:26:31.217 INFO:teuthology.orchestra.run.smithi136.stdout:(26/159): rbd-mirror-19.0.0-3872.g38f9d900.el9. 7.8 MB/s | 3.0 MB 00:00 2024-05-27T07:26:31.325 INFO:teuthology.orchestra.run.smithi136.stdout:(27/159): rbd-nbd-19.0.0-3872.g38f9d900.el9.x86 2.0 MB/s | 169 kB 00:00 2024-05-27T07:26:31.367 INFO:teuthology.orchestra.run.smithi136.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3872.g 519 kB/s | 34 kB 00:00 2024-05-27T07:26:31.453 INFO:teuthology.orchestra.run.smithi136.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3872.g38f9d90 1.7 MB/s | 143 kB 00:00 2024-05-27T07:26:31.901 INFO:teuthology.orchestra.run.smithi136.stdout:(30/159): ceph-mgr-dashboard-19.0.0-3872.g38f9d 7.9 MB/s | 3.5 MB 00:00 2024-05-27T07:26:32.060 INFO:teuthology.orchestra.run.smithi136.stdout:(31/159): ceph-radosgw-19.0.0-3872.g38f9d900.el 2.5 MB/s | 10 MB 00:04 2024-05-27T07:26:32.147 INFO:teuthology.orchestra.run.smithi129.stdout:(7/159): ceph-mon-19.0.0-3872.g38f9d900.el9.x86 1.8 MB/s | 4.9 MB 00:02 2024-05-27T07:26:32.326 INFO:teuthology.orchestra.run.smithi136.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3872.g38 1.0 MB/s | 274 kB 00:00 2024-05-27T07:26:32.435 INFO:teuthology.orchestra.run.smithi136.stdout:(33/159): ceph-mgr-rook-19.0.0-3872.g38f9d900.e 458 kB/s | 49 kB 00:00 2024-05-27T07:26:32.535 INFO:teuthology.orchestra.run.smithi136.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3872.g3 166 kB/s | 16 kB 00:00 2024-05-27T07:26:32.786 INFO:teuthology.orchestra.run.smithi136.stdout:(35/159): ceph-mgr-diskprediction-local-19.0.0- 8.4 MB/s | 7.4 MB 00:00 2024-05-27T07:26:32.821 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:26:32.822 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T07:26:32.829 INFO:teuthology.orchestra.run.smithi136.stdout:(36/159): ceph-volume-19.0.0-3872.g38f9d900.el9 933 kB/s | 273 kB 00:00 2024-05-27T07:26:32.843 INFO:teuthology.orchestra.run.smithi090.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T07:26:32.844 INFO:teuthology.orchestra.run.smithi090.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T07:26:32.845 INFO:teuthology.orchestra.run.smithi090.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T07:26:32.847 INFO:teuthology.orchestra.run.smithi090.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T07:26:32.849 INFO:teuthology.orchestra.run.smithi090.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T07:26:32.853 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T07:26:32.854 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T07:26:32.912 INFO:teuthology.orchestra.run.smithi136.stdout:(37/159): cephadm-19.0.0-3872.g38f9d900.el9.noa 6.0 MB/s | 764 kB 00:00 2024-05-27T07:26:32.916 INFO:teuthology.orchestra.run.smithi129.stdout:(8/159): ceph-osd-19.0.0-3872.g38f9d900.el9.x86 5.4 MB/s | 18 MB 00:03 2024-05-27T07:26:32.925 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-05-27T07:26:32.937 INFO:teuthology.orchestra.run.smithi136.stdout:(38/159): cephfs-top-19.0.0-3872.g38f9d900.el9. 171 kB/s | 18 kB 00:00 2024-05-27T07:26:32.946 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-05-27T07:26:32.946 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-05-27T07:26:32.946 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-05-27T07:26:32.946 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-05-27T07:26:32.946 INFO:teuthology.orchestra.run.smithi090.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T07:26:32.947 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 6.6 k 2024-05-27T07:26:32.947 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.4 M 2024-05-27T07:26:32.947 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 863 k 2024-05-27T07:26:32.947 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 147 k 2024-05-27T07:26:32.947 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 1.5 M 2024-05-27T07:26:32.947 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 143 k 2024-05-27T07:26:32.947 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 3.5 M 2024-05-27T07:26:32.947 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 7.4 M 2024-05-27T07:26:32.947 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 49 k 2024-05-27T07:26:32.948 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 10 M 2024-05-27T07:26:32.948 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 43 M 2024-05-27T07:26:32.948 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 273 k 2024-05-27T07:26:32.948 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 764 k 2024-05-27T07:26:32.948 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 224 k 2024-05-27T07:26:32.948 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 27 k 2024-05-27T07:26:32.948 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 18 k 2024-05-27T07:26:32.948 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T07:26:32.948 INFO:teuthology.orchestra.run.smithi090.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T07:26:32.948 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 32 k 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 755 k 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 127 k 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 161 k 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 320 k 2024-05-27T07:26:32.949 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 300 k 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 99 k 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 86 k 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.0 M 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 169 k 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T07:26:32.950 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.3 M 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 3.1 M 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 20 M 2024-05-27T07:26:32.951 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 34 k 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 2.3 M 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 274 k 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 4.9 M 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 18 M 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3872.g38f9d900.el9 ceph-noarch 16 k 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 25 k 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T07:26:32.952 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 157 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 498 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 5.2 M 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T07:26:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T07:26:32.954 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T07:26:32.955 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T07:26:32.955 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T07:26:32.955 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T07:26:32.955 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 45 k 2024-05-27T07:26:32.955 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.0.0-3872.g38f9d900.el9 ceph 136 k 2024-05-27T07:26:32.955 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T07:26:32.955 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T07:26:32.955 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T07:26:32.955 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T07:26:32.956 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T07:26:32.957 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T07:26:32.958 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T07:26:32.958 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T07:26:32.958 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T07:26:32.958 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T07:26:32.958 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T07:26:32.958 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T07:26:32.958 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T07:26:32.958 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T07:26:32.958 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T07:26:32.959 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T07:26:32.960 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T07:26:32.960 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T07:26:32.960 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T07:26:32.960 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T07:26:32.960 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T07:26:32.960 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T07:26:32.960 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T07:26:32.960 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T07:26:32.960 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout:Install 148 Packages 2024-05-27T07:26:32.961 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 11 Packages 2024-05-27T07:26:32.962 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:26:32.962 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 219 M 2024-05-27T07:26:32.962 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-05-27T07:26:32.991 INFO:teuthology.orchestra.run.smithi129.stdout:(9/159): ceph-selinux-19.0.0-3872.g38f9d900.el9 340 kB/s | 25 kB 00:00 2024-05-27T07:26:33.020 INFO:teuthology.orchestra.run.smithi136.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 747 kB/s | 80 kB 00:00 2024-05-27T07:26:33.079 INFO:teuthology.orchestra.run.smithi136.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-05-27T07:26:33.121 INFO:teuthology.orchestra.run.smithi136.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-05-27T07:26:33.155 INFO:teuthology.orchestra.run.smithi136.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-05-27T07:26:33.605 INFO:teuthology.orchestra.run.smithi090.stdout:(1/159): ceph-19.0.0-3872.g38f9d900.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-27T07:26:33.608 INFO:teuthology.orchestra.run.smithi129.stdout:(10/159): ceph-common-19.0.0-3872.g38f9d900.el9 3.0 MB/s | 20 MB 00:06 2024-05-27T07:26:33.613 INFO:teuthology.orchestra.run.smithi136.stdout:(43/159): ceph-test-19.0.0-3872.g38f9d900.el9.x 8.7 MB/s | 43 MB 00:04 2024-05-27T07:26:33.655 INFO:teuthology.orchestra.run.smithi136.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 213 kB/s | 106 kB 00:00 2024-05-27T07:26:33.683 INFO:teuthology.orchestra.run.smithi129.stdout:(11/159): cephfs-mirror-19.0.0-3872.g38f9d900.e 3.0 MB/s | 224 kB 00:00 2024-05-27T07:26:33.714 INFO:teuthology.orchestra.run.smithi136.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 2.1 MB/s | 1.2 MB 00:00 2024-05-27T07:26:33.739 INFO:teuthology.orchestra.run.smithi136.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-27T07:26:33.758 INFO:teuthology.orchestra.run.smithi129.stdout:(12/159): cephfs-shell-19.0.0-3872.g38f9d900.el 365 kB/s | 27 kB 00:00 2024-05-27T07:26:33.773 INFO:teuthology.orchestra.run.smithi136.stdout:(47/159): python3-pycparser-2.20-6.el9.noarch.r 855 kB/s | 135 kB 00:00 2024-05-27T07:26:33.798 INFO:teuthology.orchestra.run.smithi136.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-27T07:26:33.867 INFO:teuthology.orchestra.run.smithi129.stdout:(13/159): libcephfs-devel-19.0.0-3872.g38f9d900 295 kB/s | 32 kB 00:00 2024-05-27T07:26:34.251 INFO:teuthology.orchestra.run.smithi129.stdout:(14/159): libcephfs2-19.0.0-3872.g38f9d900.el9. 1.9 MB/s | 755 kB 00:00 2024-05-27T07:26:34.305 INFO:teuthology.orchestra.run.smithi090.stdout:(2/159): ceph-fuse-19.0.0-3872.g38f9d900.el9.x8 1.2 MB/s | 863 kB 00:00 2024-05-27T07:26:34.331 INFO:teuthology.orchestra.run.smithi136.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 201 kB/s | 107 kB 00:00 2024-05-27T07:26:34.359 INFO:teuthology.orchestra.run.smithi129.stdout:(15/159): libcephsqlite-19.0.0-3872.g38f9d900.e 1.4 MB/s | 157 kB 00:00 2024-05-27T07:26:34.440 INFO:teuthology.orchestra.run.smithi136.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-05-27T07:26:34.476 INFO:teuthology.orchestra.run.smithi129.stdout:(16/159): librados-devel-19.0.0-3872.g38f9d900. 1.1 MB/s | 127 kB 00:00 2024-05-27T07:26:34.582 INFO:teuthology.orchestra.run.smithi136.stdout:(51/159): automake-1.16.2-8.el9.noarch.rpm 818 kB/s | 689 kB 00:00 2024-05-27T07:26:34.624 INFO:teuthology.orchestra.run.smithi136.stdout:(52/159): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-05-27T07:26:34.691 INFO:teuthology.orchestra.run.smithi136.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-05-27T07:26:34.718 INFO:teuthology.orchestra.run.smithi129.stdout:(17/159): libradosstriper1-19.0.0-3872.g38f9d90 2.0 MB/s | 498 kB 00:00 2024-05-27T07:26:34.724 INFO:teuthology.orchestra.run.smithi136.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-05-27T07:26:34.800 INFO:teuthology.orchestra.run.smithi136.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-05-27T07:26:34.834 INFO:teuthology.orchestra.run.smithi136.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-05-27T07:26:34.909 INFO:teuthology.orchestra.run.smithi136.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-27T07:26:34.934 INFO:teuthology.orchestra.run.smithi136.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 453 kB/s | 45 kB 00:00 2024-05-27T07:26:34.963 INFO:teuthology.orchestra.run.smithi090.stdout:(3/159): ceph-common-19.0.0-3872.g38f9d900.el9. 12 MB/s | 20 MB 00:01 2024-05-27T07:26:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:(4/159): ceph-immutable-object-cache-19.0.0-387 213 kB/s | 147 kB 00:00 2024-05-27T07:26:35.185 INFO:teuthology.orchestra.run.smithi136.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-05-27T07:26:35.197 INFO:teuthology.orchestra.run.smithi090.stdout:(5/159): ceph-mds-19.0.0-3872.g38f9d900.el9.x86 9.7 MB/s | 2.3 MB 00:00 2024-05-27T07:26:35.218 INFO:teuthology.orchestra.run.smithi136.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 2.0 MB/s | 582 kB 00:00 2024-05-27T07:26:35.294 INFO:teuthology.orchestra.run.smithi136.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-05-27T07:26:35.359 INFO:teuthology.orchestra.run.smithi136.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 339 kB/s | 47 kB 00:00 2024-05-27T07:26:35.411 INFO:teuthology.orchestra.run.smithi136.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-05-27T07:26:35.470 INFO:teuthology.orchestra.run.smithi136.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-27T07:26:35.520 INFO:teuthology.orchestra.run.smithi136.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 326 kB/s | 35 kB 00:00 2024-05-27T07:26:35.648 INFO:teuthology.orchestra.run.smithi090.stdout:(6/159): ceph-mon-19.0.0-3872.g38f9d900.el9.x86 11 MB/s | 4.9 MB 00:00 2024-05-27T07:26:35.727 INFO:teuthology.orchestra.run.smithi129.stdout:(18/159): librgw2-19.0.0-3872.g38f9d900.el9.x86 5.1 MB/s | 5.2 MB 00:01 2024-05-27T07:26:35.766 INFO:teuthology.orchestra.run.smithi090.stdout:(7/159): ceph-base-19.0.0-3872.g38f9d900.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-05-27T07:26:35.802 INFO:teuthology.orchestra.run.smithi129.stdout:(19/159): python3-ceph-argparse-19.0.0-3872.g38 612 kB/s | 45 kB 00:00 2024-05-27T07:26:35.886 INFO:teuthology.orchestra.run.smithi129.stdout:(20/159): python3-ceph-common-19.0.0-3872.g38f9 1.6 MB/s | 136 kB 00:00 2024-05-27T07:26:35.983 INFO:teuthology.orchestra.run.smithi090.stdout:(8/159): ceph-mgr-19.0.0-3872.g38f9d900.el9.x86 1.5 MB/s | 1.5 MB 00:00 2024-05-27T07:26:35.994 INFO:teuthology.orchestra.run.smithi129.stdout:(21/159): python3-cephfs-19.0.0-3872.g38f9d900. 1.5 MB/s | 161 kB 00:00 2024-05-27T07:26:36.058 INFO:teuthology.orchestra.run.smithi090.stdout:(9/159): ceph-selinux-19.0.0-3872.g38f9d900.el9 339 kB/s | 25 kB 00:00 2024-05-27T07:26:36.079 INFO:teuthology.orchestra.run.smithi136.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 7.8 MB/s | 4.7 MB 00:00 2024-05-27T07:26:36.153 INFO:teuthology.orchestra.run.smithi129.stdout:(22/159): ceph-radosgw-19.0.0-3872.g38f9d900.el 2.5 MB/s | 10 MB 00:04 2024-05-27T07:26:36.195 INFO:teuthology.orchestra.run.smithi129.stdout:(23/159): python3-rados-19.0.0-3872.g38f9d900.e 1.6 MB/s | 320 kB 00:00 2024-05-27T07:26:36.195 INFO:teuthology.orchestra.run.smithi136.stdout:(67/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.9 MB/s | 3.0 MB 00:01 2024-05-27T07:26:36.270 INFO:teuthology.orchestra.run.smithi129.stdout:(24/159): python3-rgw-19.0.0-3872.g38f9d900.el9 1.3 MB/s | 99 kB 00:00 2024-05-27T07:26:36.321 INFO:teuthology.orchestra.run.smithi136.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-05-27T07:26:36.362 INFO:teuthology.orchestra.run.smithi129.stdout:(25/159): rbd-fuse-19.0.0-3872.g38f9d900.el9.x8 949 kB/s | 86 kB 00:00 2024-05-27T07:26:36.404 INFO:teuthology.orchestra.run.smithi129.stdout:(26/159): python3-rbd-19.0.0-3872.g38f9d900.el9 1.2 MB/s | 300 kB 00:00 2024-05-27T07:26:36.429 INFO:teuthology.orchestra.run.smithi136.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-27T07:26:36.563 INFO:teuthology.orchestra.run.smithi129.stdout:(27/159): rbd-nbd-19.0.0-3872.g38f9d900.el9.x86 1.0 MB/s | 169 kB 00:00 2024-05-27T07:26:36.680 INFO:teuthology.orchestra.run.smithi129.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3872.g 295 kB/s | 34 kB 00:00 2024-05-27T07:26:36.680 INFO:teuthology.orchestra.run.smithi136.stdout:(70/159): postgresql-contrib-13.14-1.el9.x86_64 1.8 MB/s | 911 kB 00:00 2024-05-27T07:26:36.847 INFO:teuthology.orchestra.run.smithi129.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3872.g38f9d90 862 kB/s | 143 kB 00:00 2024-05-27T07:26:36.989 INFO:teuthology.orchestra.run.smithi136.stdout:(71/159): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-05-27T07:26:37.097 INFO:teuthology.orchestra.run.smithi136.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 816 kB/s | 87 kB 00:00 2024-05-27T07:26:37.164 INFO:teuthology.orchestra.run.smithi136.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 2.1 MB/s | 1.0 MB 00:00 2024-05-27T07:26:37.217 INFO:teuthology.orchestra.run.smithi090.stdout:(10/159): ceph-radosgw-19.0.0-3872.g38f9d900.el 7.0 MB/s | 10 MB 00:01 2024-05-27T07:26:37.281 INFO:teuthology.orchestra.run.smithi136.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 528 kB/s | 61 kB 00:00 2024-05-27T07:26:37.305 INFO:teuthology.orchestra.run.smithi129.stdout:(30/159): rbd-mirror-19.0.0-3872.g38f9d900.el9. 3.2 MB/s | 3.0 MB 00:00 2024-05-27T07:26:37.398 INFO:teuthology.orchestra.run.smithi136.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.1 MB/s | 244 kB 00:00 2024-05-27T07:26:37.493 INFO:teuthology.orchestra.run.smithi090.stdout:(11/159): ceph-osd-19.0.0-3872.g38f9d900.el9.x8 9.6 MB/s | 18 MB 00:01 2024-05-27T07:26:37.515 INFO:teuthology.orchestra.run.smithi136.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-05-27T07:26:37.526 INFO:teuthology.orchestra.run.smithi090.stdout:(12/159): cephfs-mirror-19.0.0-3872.g38f9d900.e 728 kB/s | 224 kB 00:00 2024-05-27T07:26:37.551 INFO:teuthology.orchestra.run.smithi090.stdout:(13/159): cephfs-shell-19.0.0-3872.g38f9d900.el 470 kB/s | 27 kB 00:00 2024-05-27T07:26:37.593 INFO:teuthology.orchestra.run.smithi090.stdout:(14/159): libcephfs-devel-19.0.0-3872.g38f9d900 479 kB/s | 32 kB 00:00 2024-05-27T07:26:37.624 INFO:teuthology.orchestra.run.smithi136.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 239 kB/s | 26 kB 00:00 2024-05-27T07:26:37.669 INFO:teuthology.orchestra.run.smithi090.stdout:(15/159): libcephfs2-19.0.0-3872.g38f9d900.el9. 6.3 MB/s | 755 kB 00:00 2024-05-27T07:26:37.702 INFO:teuthology.orchestra.run.smithi090.stdout:(16/159): libcephsqlite-19.0.0-3872.g38f9d900.e 1.4 MB/s | 157 kB 00:00 2024-05-27T07:26:37.732 INFO:teuthology.orchestra.run.smithi136.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 178 kB/s | 19 kB 00:00 2024-05-27T07:26:37.736 INFO:teuthology.orchestra.run.smithi090.stdout:(17/159): librados-devel-19.0.0-3872.g38f9d900. 1.9 MB/s | 127 kB 00:00 2024-05-27T07:26:37.841 INFO:teuthology.orchestra.run.smithi136.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-05-27T07:26:37.853 INFO:teuthology.orchestra.run.smithi090.stdout:(18/159): libradosstriper1-19.0.0-3872.g38f9d90 3.2 MB/s | 498 kB 00:00 2024-05-27T07:26:37.945 INFO:teuthology.orchestra.run.smithi090.stdout:(19/159): python3-ceph-argparse-19.0.0-3872.g38 495 kB/s | 45 kB 00:00 2024-05-27T07:26:38.004 INFO:teuthology.orchestra.run.smithi136.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 214 kB/s | 35 kB 00:00 2024-05-27T07:26:38.046 INFO:teuthology.orchestra.run.smithi090.stdout:(20/159): python3-ceph-common-19.0.0-3872.g38f9 1.3 MB/s | 136 kB 00:00 2024-05-27T07:26:38.146 INFO:teuthology.orchestra.run.smithi090.stdout:(21/159): python3-cephfs-19.0.0-3872.g38f9d900. 1.6 MB/s | 161 kB 00:00 2024-05-27T07:26:38.206 INFO:teuthology.orchestra.run.smithi129.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3872.g38f9d 2.6 MB/s | 3.5 MB 00:01 2024-05-27T07:26:38.217 INFO:teuthology.orchestra.run.smithi136.stdout:(81/159): python3-babel-2.9.1-2.el9.noarch.rpm 5.3 MB/s | 6.0 MB 00:01 2024-05-27T07:26:38.263 INFO:teuthology.orchestra.run.smithi090.stdout:(22/159): librgw2-19.0.0-3872.g38f9d900.el9.x86 9.8 MB/s | 5.2 MB 00:00 2024-05-27T07:26:38.305 INFO:teuthology.orchestra.run.smithi090.stdout:(23/159): python3-rados-19.0.0-3872.g38f9d900.e 2.0 MB/s | 320 kB 00:00 2024-05-27T07:26:38.306 INFO:teuthology.orchestra.run.smithi129.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3872.g38 2.7 MB/s | 274 kB 00:00 2024-05-27T07:26:38.339 INFO:teuthology.orchestra.run.smithi090.stdout:(24/159): python3-rbd-19.0.0-3872.g38f9d900.el9 3.9 MB/s | 300 kB 00:00 2024-05-27T07:26:38.373 INFO:teuthology.orchestra.run.smithi090.stdout:(25/159): python3-rgw-19.0.0-3872.g38f9d900.el9 1.5 MB/s | 99 kB 00:00 2024-05-27T07:26:38.375 INFO:teuthology.orchestra.run.smithi136.stdout:(82/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 4.4 MB 00:02 2024-05-27T07:26:38.398 INFO:teuthology.orchestra.run.smithi090.stdout:(26/159): rbd-fuse-19.0.0-3872.g38f9d900.el9.x8 1.4 MB/s | 86 kB 00:00 2024-05-27T07:26:38.465 INFO:teuthology.orchestra.run.smithi090.stdout:(27/159): rbd-nbd-19.0.0-3872.g38f9d900.el9.x86 2.5 MB/s | 169 kB 00:00 2024-05-27T07:26:38.493 INFO:teuthology.orchestra.run.smithi129.stdout:(33/159): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-05-27T07:26:38.540 INFO:teuthology.orchestra.run.smithi090.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3872.g 462 kB/s | 34 kB 00:00 2024-05-27T07:26:38.567 INFO:teuthology.orchestra.run.smithi136.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 717 kB/s | 250 kB 00:00 2024-05-27T07:26:38.587 INFO:teuthology.orchestra.run.smithi129.stdout:(34/159): ceph-mgr-rook-19.0.0-3872.g38f9d900.e 176 kB/s | 49 kB 00:00 2024-05-27T07:26:38.607 INFO:teuthology.orchestra.run.smithi090.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3872.g38f9d90 2.1 MB/s | 143 kB 00:00 2024-05-27T07:26:38.615 INFO:teuthology.orchestra.run.smithi129.stdout:(35/159): ceph-prometheus-alerts-19.0.0-3872.g3 135 kB/s | 16 kB 00:00 2024-05-27T07:26:38.699 INFO:teuthology.orchestra.run.smithi129.stdout:(36/159): ceph-volume-19.0.0-3872.g38f9d900.el9 2.4 MB/s | 273 kB 00:00 2024-05-27T07:26:38.717 INFO:teuthology.orchestra.run.smithi090.stdout:(30/159): rbd-mirror-19.0.0-3872.g38f9d900.el9. 8.9 MB/s | 3.0 MB 00:00 2024-05-27T07:26:38.758 INFO:teuthology.orchestra.run.smithi129.stdout:(37/159): cephfs-top-19.0.0-3872.g38f9d900.el9. 316 kB/s | 18 kB 00:00 2024-05-27T07:26:38.759 INFO:teuthology.orchestra.run.smithi136.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 404 kB/s | 77 kB 00:00 2024-05-27T07:26:38.784 INFO:teuthology.orchestra.run.smithi136.stdout:(85/159): python3-oauthlib-3.1.1-5.el9.noarch.r 545 kB/s | 222 kB 00:00 2024-05-27T07:26:38.833 INFO:teuthology.orchestra.run.smithi129.stdout:(38/159): cephadm-19.0.0-3872.g38f9d900.el9.noa 3.4 MB/s | 764 kB 00:00 2024-05-27T07:26:38.875 INFO:teuthology.orchestra.run.smithi129.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 692 kB/s | 80 kB 00:00 2024-05-27T07:26:38.876 INFO:teuthology.orchestra.run.smithi136.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 363 kB/s | 42 kB 00:00 2024-05-27T07:26:38.942 INFO:teuthology.orchestra.run.smithi129.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-27T07:26:38.976 INFO:teuthology.orchestra.run.smithi090.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3872.g38f9d 9.6 MB/s | 3.5 MB 00:00 2024-05-27T07:26:38.992 INFO:teuthology.orchestra.run.smithi129.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-05-27T07:26:39.043 INFO:teuthology.orchestra.run.smithi129.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-05-27T07:26:39.051 INFO:teuthology.orchestra.run.smithi090.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3872.g38 3.6 MB/s | 274 kB 00:00 2024-05-27T07:26:39.085 INFO:teuthology.orchestra.run.smithi129.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.6 MB/s | 106 kB 00:00 2024-05-27T07:26:39.110 INFO:teuthology.orchestra.run.smithi090.stdout:(33/159): ceph-mgr-rook-19.0.0-3872.g38f9d900.e 858 kB/s | 49 kB 00:00 2024-05-27T07:26:39.143 INFO:teuthology.orchestra.run.smithi129.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.2 MB 00:00 2024-05-27T07:26:39.152 INFO:teuthology.orchestra.run.smithi136.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 578 kB/s | 159 kB 00:00 2024-05-27T07:26:39.168 INFO:teuthology.orchestra.run.smithi090.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3872.g3 284 kB/s | 16 kB 00:00 2024-05-27T07:26:39.202 INFO:teuthology.orchestra.run.smithi129.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-27T07:26:39.210 INFO:teuthology.orchestra.run.smithi136.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 633 kB/s | 269 kB 00:00 2024-05-27T07:26:39.235 INFO:teuthology.orchestra.run.smithi129.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-27T07:26:39.244 INFO:teuthology.orchestra.run.smithi090.stdout:(35/159): ceph-volume-19.0.0-3872.g38f9d900.el9 3.6 MB/s | 273 kB 00:00 2024-05-27T07:26:39.269 INFO:teuthology.orchestra.run.smithi129.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-05-27T07:26:39.319 INFO:teuthology.orchestra.run.smithi136.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 473 kB/s | 51 kB 00:00 2024-05-27T07:26:39.386 INFO:teuthology.orchestra.run.smithi090.stdout:(36/159): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-05-27T07:26:39.428 INFO:teuthology.orchestra.run.smithi136.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 499 kB/s | 54 kB 00:00 2024-05-27T07:26:39.445 INFO:teuthology.orchestra.run.smithi090.stdout:(37/159): cephadm-19.0.0-3872.g38f9d900.el9.noa 3.7 MB/s | 764 kB 00:00 2024-05-27T07:26:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:(38/159): cephfs-top-19.0.0-3872.g38f9d900.el9. 169 kB/s | 18 kB 00:00 2024-05-27T07:26:39.578 INFO:teuthology.orchestra.run.smithi090.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 606 kB/s | 80 kB 00:00 2024-05-27T07:26:39.587 INFO:teuthology.orchestra.run.smithi136.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 643 kB/s | 279 kB 00:00 2024-05-27T07:26:39.637 INFO:teuthology.orchestra.run.smithi090.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-05-27T07:26:39.688 INFO:teuthology.orchestra.run.smithi090.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-05-27T07:26:39.695 INFO:teuthology.orchestra.run.smithi136.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-05-27T07:26:39.721 INFO:teuthology.orchestra.run.smithi090.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-27T07:26:39.763 INFO:teuthology.orchestra.run.smithi090.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-27T07:26:39.812 INFO:teuthology.orchestra.run.smithi136.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 381 kB/s | 44 kB 00:00 2024-05-27T07:26:39.814 INFO:teuthology.orchestra.run.smithi090.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-05-27T07:26:39.848 INFO:teuthology.orchestra.run.smithi090.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-05-27T07:26:39.881 INFO:teuthology.orchestra.run.smithi090.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-27T07:26:39.915 INFO:teuthology.orchestra.run.smithi090.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-05-27T07:26:39.966 INFO:teuthology.orchestra.run.smithi090.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 8.1 MB/s | 689 kB 00:00 2024-05-27T07:26:40.024 INFO:teuthology.orchestra.run.smithi090.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-05-27T07:26:40.050 INFO:teuthology.orchestra.run.smithi090.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-05-27T07:26:40.083 INFO:teuthology.orchestra.run.smithi090.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 375 kB/s | 22 kB 00:00 2024-05-27T07:26:40.117 INFO:teuthology.orchestra.run.smithi090.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-05-27T07:26:40.143 INFO:teuthology.orchestra.run.smithi090.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 554 kB/s | 33 kB 00:00 2024-05-27T07:26:40.185 INFO:teuthology.orchestra.run.smithi090.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 445 kB/s | 18 kB 00:00 2024-05-27T07:26:40.228 INFO:teuthology.orchestra.run.smithi090.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 272 kB/s | 11 kB 00:00 2024-05-27T07:26:40.229 INFO:teuthology.orchestra.run.smithi136.stdout:(94/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.3 MB/s | 5.1 MB 00:02 2024-05-27T07:26:40.254 INFO:teuthology.orchestra.run.smithi136.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 500 kB/s | 221 kB 00:00 2024-05-27T07:26:40.278 INFO:teuthology.orchestra.run.smithi090.stdout:(56/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 160 kB 00:00 2024-05-27T07:26:40.338 INFO:teuthology.orchestra.run.smithi136.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 333 kB/s | 36 kB 00:00 2024-05-27T07:26:40.353 INFO:teuthology.orchestra.run.smithi090.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-05-27T07:26:40.363 INFO:teuthology.orchestra.run.smithi136.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-05-27T07:26:40.378 INFO:teuthology.orchestra.run.smithi090.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 454 kB/s | 45 kB 00:00 2024-05-27T07:26:40.429 INFO:teuthology.orchestra.run.smithi090.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-05-27T07:26:40.455 INFO:teuthology.orchestra.run.smithi136.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-05-27T07:26:40.479 INFO:teuthology.orchestra.run.smithi090.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-05-27T07:26:40.480 INFO:teuthology.orchestra.run.smithi136.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 496 kB/s | 58 kB 00:00 2024-05-27T07:26:40.505 INFO:teuthology.orchestra.run.smithi090.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 341 kB/s | 25 kB 00:00 2024-05-27T07:26:40.530 INFO:teuthology.orchestra.run.smithi090.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 961 kB/s | 47 kB 00:00 2024-05-27T07:26:40.553 INFO:teuthology.orchestra.run.smithi129.stdout:(48/159): ceph-test-19.0.0-3872.g38f9d900.el9.x 5.7 MB/s | 43 MB 00:07 2024-05-27T07:26:40.564 INFO:teuthology.orchestra.run.smithi136.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 868 kB/s | 94 kB 00:00 2024-05-27T07:26:40.572 INFO:teuthology.orchestra.run.smithi090.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-05-27T07:26:40.606 INFO:teuthology.orchestra.run.smithi090.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-27T07:26:40.611 INFO:teuthology.orchestra.run.smithi129.stdout:(49/159): automake-1.16.2-8.el9.noarch.rpm 501 kB/s | 689 kB 00:01 2024-05-27T07:26:40.631 INFO:teuthology.orchestra.run.smithi090.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 609 kB/s | 35 kB 00:00 2024-05-27T07:26:40.681 INFO:teuthology.orchestra.run.smithi136.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 319 kB/s | 64 kB 00:00 2024-05-27T07:26:40.703 INFO:teuthology.orchestra.run.smithi129.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 658 kB/s | 943 kB 00:01 2024-05-27T07:26:40.706 INFO:teuthology.orchestra.run.smithi136.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 318 kB/s | 45 kB 00:00 2024-05-27T07:26:40.790 INFO:teuthology.orchestra.run.smithi090.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-05-27T07:26:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-05-27T07:26:40.965 INFO:teuthology.orchestra.run.smithi090.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-05-27T07:26:41.007 INFO:teuthology.orchestra.run.smithi090.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 7.7 MB/s | 911 kB 00:00 2024-05-27T07:26:41.033 INFO:teuthology.orchestra.run.smithi090.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-27T07:26:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-05-27T07:26:41.158 INFO:teuthology.orchestra.run.smithi090.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-27T07:26:41.229 INFO:teuthology.orchestra.run.smithi129.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 35 kB/s | 22 kB 00:00 2024-05-27T07:26:41.309 INFO:teuthology.orchestra.run.smithi090.stdout:(73/159): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-05-27T07:26:41.323 INFO:teuthology.orchestra.run.smithi136.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-05-27T07:26:41.351 INFO:teuthology.orchestra.run.smithi090.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.5 MB/s | 61 kB 00:00 2024-05-27T07:26:41.370 INFO:teuthology.orchestra.run.smithi129.stdout:(52/159): boost-program-options-1.75.0-8.el9.x8 131 kB/s | 107 kB 00:00 2024-05-27T07:26:41.401 INFO:teuthology.orchestra.run.smithi090.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-27T07:26:41.453 INFO:teuthology.orchestra.run.smithi090.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-27T07:26:41.482 INFO:teuthology.orchestra.run.smithi136.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 3.4 MB/s | 548 kB 00:00 2024-05-27T07:26:41.532 INFO:teuthology.orchestra.run.smithi136.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 6.1 MB/s | 309 kB 00:00 2024-05-27T07:26:41.574 INFO:teuthology.orchestra.run.smithi136.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 476 kB/s | 19 kB 00:00 2024-05-27T07:26:41.587 INFO:teuthology.orchestra.run.smithi129.stdout:(53/159): flex-2.6.4-9.el9.x86_64.rpm 355 kB/s | 313 kB 00:00 2024-05-27T07:26:41.741 INFO:teuthology.orchestra.run.smithi136.stdout:(107/159): libarrow-9.0.0-11.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-05-27T07:26:41.762 INFO:teuthology.orchestra.run.smithi090.stdout:(77/159): python3-babel-2.9.1-2.el9.noarch.rpm 9.9 MB/s | 6.0 MB 00:00 2024-05-27T07:26:41.783 INFO:teuthology.orchestra.run.smithi136.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 640 kB/s | 26 kB 00:00 2024-05-27T07:26:41.787 INFO:teuthology.orchestra.run.smithi090.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 77 kB/s | 26 kB 00:00 2024-05-27T07:26:41.812 INFO:teuthology.orchestra.run.smithi090.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 387 kB/s | 19 kB 00:00 2024-05-27T07:26:41.825 INFO:teuthology.orchestra.run.smithi136.stdout:(109/159): protobuf-compiler-3.14.0-13.el9.x86_ 773 kB/s | 863 kB 00:01 2024-05-27T07:26:41.846 INFO:teuthology.orchestra.run.smithi090.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-27T07:26:41.850 INFO:teuthology.orchestra.run.smithi136.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 733 kB/s | 49 kB 00:00 2024-05-27T07:26:41.880 INFO:teuthology.orchestra.run.smithi090.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 527 kB/s | 35 kB 00:00 2024-05-27T07:26:41.892 INFO:teuthology.orchestra.run.smithi136.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-05-27T07:26:41.930 INFO:teuthology.orchestra.run.smithi090.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-05-27T07:26:41.942 INFO:teuthology.orchestra.run.smithi136.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 582 kB/s | 67 kB 00:00 2024-05-27T07:26:41.945 INFO:teuthology.orchestra.run.smithi129.stdout:(54/159): flexiblas-3.0.4-8.el9.x86_64.rpm 46 kB/s | 33 kB 00:00 2024-05-27T07:26:41.993 INFO:teuthology.orchestra.run.smithi136.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-05-27T07:26:41.997 INFO:teuthology.orchestra.run.smithi090.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-27T07:26:42.035 INFO:teuthology.orchestra.run.smithi136.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-27T07:26:42.039 INFO:teuthology.orchestra.run.smithi090.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-27T07:26:42.069 INFO:teuthology.orchestra.run.smithi136.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-27T07:26:42.081 INFO:teuthology.orchestra.run.smithi090.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-05-27T07:26:42.123 INFO:teuthology.orchestra.run.smithi136.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 369 kB/s | 32 kB 00:00 2024-05-27T07:26:42.131 INFO:teuthology.orchestra.run.smithi090.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 5.4 MB/s | 269 kB 00:00 2024-05-27T07:26:42.153 INFO:teuthology.orchestra.run.smithi136.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-05-27T07:26:42.178 INFO:teuthology.orchestra.run.smithi136.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.1 MB/s | 172 kB 00:00 2024-05-27T07:26:42.179 INFO:teuthology.orchestra.run.smithi129.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 31 kB/s | 18 kB 00:00 2024-05-27T07:26:42.212 INFO:teuthology.orchestra.run.smithi136.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 6.1 MB/s | 358 kB 00:00 2024-05-27T07:26:42.245 INFO:teuthology.orchestra.run.smithi136.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-05-27T07:26:42.248 INFO:teuthology.orchestra.run.smithi090.stdout:(87/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-27T07:26:42.279 INFO:teuthology.orchestra.run.smithi136.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-05-27T07:26:42.282 INFO:teuthology.orchestra.run.smithi090.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-27T07:26:42.324 INFO:teuthology.orchestra.run.smithi090.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-05-27T07:26:42.349 INFO:teuthology.orchestra.run.smithi090.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 772 kB/s | 51 kB 00:00 2024-05-27T07:26:42.354 INFO:teuthology.orchestra.run.smithi136.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-05-27T07:26:42.375 INFO:teuthology.orchestra.run.smithi090.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-27T07:26:42.413 INFO:teuthology.orchestra.run.smithi136.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-05-27T07:26:42.417 INFO:teuthology.orchestra.run.smithi090.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-27T07:26:42.438 INFO:teuthology.orchestra.run.smithi136.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-05-27T07:26:42.459 INFO:teuthology.orchestra.run.smithi090.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-05-27T07:26:42.463 INFO:teuthology.orchestra.run.smithi136.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 358 kB/s | 18 kB 00:00 2024-05-27T07:26:42.489 INFO:teuthology.orchestra.run.smithi136.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 468 kB/s | 23 kB 00:00 2024-05-27T07:26:42.509 INFO:teuthology.orchestra.run.smithi090.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.4 MB/s | 221 kB 00:00 2024-05-27T07:26:42.514 INFO:teuthology.orchestra.run.smithi136.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 393 kB/s | 19 kB 00:00 2024-05-27T07:26:42.531 INFO:teuthology.orchestra.run.smithi136.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 477 kB/s | 20 kB 00:00 2024-05-27T07:26:42.551 INFO:teuthology.orchestra.run.smithi090.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 878 kB/s | 36 kB 00:00 2024-05-27T07:26:42.571 INFO:teuthology.orchestra.run.smithi129.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 18 kB/s | 11 kB 00:00 2024-05-27T07:26:42.581 INFO:teuthology.orchestra.run.smithi136.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-05-27T07:26:42.593 INFO:teuthology.orchestra.run.smithi090.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-27T07:26:42.598 INFO:teuthology.orchestra.run.smithi136.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 702 kB/s | 46 kB 00:00 2024-05-27T07:26:42.624 INFO:teuthology.orchestra.run.smithi136.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-05-27T07:26:42.644 INFO:teuthology.orchestra.run.smithi090.stdout:(97/159): socat-1.7.4.1-5.el9.x86_64.rpm 6.1 MB/s | 305 kB 00:00 2024-05-27T07:26:42.646 INFO:teuthology.orchestra.run.smithi129.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.3 MB/s | 3.0 MB 00:01 2024-05-27T07:26:42.649 INFO:teuthology.orchestra.run.smithi136.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-27T07:26:42.682 INFO:teuthology.orchestra.run.smithi136.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-05-27T07:26:42.686 INFO:teuthology.orchestra.run.smithi090.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-05-27T07:26:42.708 INFO:teuthology.orchestra.run.smithi136.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 284 kB/s | 16 kB 00:00 2024-05-27T07:26:42.729 INFO:teuthology.orchestra.run.smithi090.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-05-27T07:26:42.733 INFO:teuthology.orchestra.run.smithi136.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-27T07:26:42.758 INFO:teuthology.orchestra.run.smithi136.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 571 kB/s | 28 kB 00:00 2024-05-27T07:26:42.771 INFO:teuthology.orchestra.run.smithi090.stdout:(100/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-27T07:26:42.784 INFO:teuthology.orchestra.run.smithi136.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 621 kB/s | 31 kB 00:00 2024-05-27T07:26:42.818 INFO:teuthology.orchestra.run.smithi136.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-05-27T07:26:42.821 INFO:teuthology.orchestra.run.smithi090.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 909 kB/s | 45 kB 00:00 2024-05-27T07:26:42.842 INFO:teuthology.orchestra.run.smithi136.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-27T07:26:42.868 INFO:teuthology.orchestra.run.smithi136.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 722 kB/s | 36 kB 00:00 2024-05-27T07:26:42.894 INFO:teuthology.orchestra.run.smithi136.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-27T07:26:42.927 INFO:teuthology.orchestra.run.smithi136.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-05-27T07:26:42.952 INFO:teuthology.orchestra.run.smithi136.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-27T07:26:42.986 INFO:teuthology.orchestra.run.smithi136.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-05-27T07:26:43.011 INFO:teuthology.orchestra.run.smithi136.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 345 kB/s | 20 kB 00:00 2024-05-27T07:26:43.045 INFO:teuthology.orchestra.run.smithi136.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-27T07:26:43.172 INFO:teuthology.orchestra.run.smithi129.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 174 kB/s | 160 kB 00:00 2024-05-27T07:26:43.172 INFO:teuthology.orchestra.run.smithi090.stdout:(102/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-05-27T07:26:43.329 INFO:teuthology.orchestra.run.smithi136.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-05-27T07:26:43.339 INFO:teuthology.orchestra.run.smithi129.stdout:(59/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 59 kB/s | 45 kB 00:00 2024-05-27T07:26:43.656 INFO:teuthology.orchestra.run.smithi090.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 27 kB/s | 22 kB 00:00 2024-05-27T07:26:43.762 INFO:teuthology.orchestra.run.smithi136.stdout:(148/159): librbd1-19.0.0-3872.g38f9d900.el9.x8 7.2 MB/s | 3.1 MB 00:00 2024-05-27T07:26:43.781 INFO:teuthology.orchestra.run.smithi129.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 586 kB/s | 662 kB 00:01 2024-05-27T07:26:43.946 INFO:teuthology.orchestra.run.smithi136.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-27T07:26:43.964 INFO:teuthology.orchestra.run.smithi090.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-05-27T07:26:43.972 INFO:teuthology.orchestra.run.smithi129.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 40 kB/s | 25 kB 00:00 2024-05-27T07:26:44.042 INFO:teuthology.orchestra.run.smithi090.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 4.1 MB/s | 309 kB 00:00 2024-05-27T07:26:44.054 INFO:teuthology.orchestra.run.smithi136.stdout:(150/159): librados2-19.0.0-3872.g38f9d900.el9. 3.3 MB/s | 3.3 MB 00:01 2024-05-27T07:26:44.130 INFO:teuthology.orchestra.run.smithi136.stdout:(151/159): NetworkManager-libnm-1.47.90-1.el9.x 9.9 MB/s | 1.8 MB 00:00 2024-05-27T07:26:44.155 INFO:teuthology.orchestra.run.smithi136.stdout:(152/159): NetworkManager-team-1.47.90-1.el9.x8 417 kB/s | 41 kB 00:00 2024-05-27T07:26:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 147 kB/s | 19 kB 00:00 2024-05-27T07:26:44.189 INFO:teuthology.orchestra.run.smithi136.stdout:(153/159): NetworkManager-tui-1.47.90-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-05-27T07:26:44.214 INFO:teuthology.orchestra.run.smithi136.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 636 kB/s | 37 kB 00:00 2024-05-27T07:26:44.248 INFO:teuthology.orchestra.run.smithi136.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-05-27T07:26:44.248 INFO:teuthology.orchestra.run.smithi129.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 542 kB/s | 582 kB 00:01 2024-05-27T07:26:44.274 INFO:teuthology.orchestra.run.smithi090.stdout:(107/159): protobuf-compiler-3.14.0-13.el9.x86_ 785 kB/s | 863 kB 00:01 2024-05-27T07:26:44.281 INFO:teuthology.orchestra.run.smithi136.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.8 MB/s | 260 kB 00:00 2024-05-27T07:26:44.348 INFO:teuthology.orchestra.run.smithi136.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 14 MB/s | 1.3 MB 00:00 2024-05-27T07:26:44.399 INFO:teuthology.orchestra.run.smithi090.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-05-27T07:26:44.449 INFO:teuthology.orchestra.run.smithi136.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-05-27T07:26:44.498 INFO:teuthology.orchestra.run.smithi129.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 66 kB/s | 47 kB 00:00 2024-05-27T07:26:44.524 INFO:teuthology.orchestra.run.smithi090.stdout:(109/159): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-27T07:26:44.558 INFO:teuthology.orchestra.run.smithi090.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 321 kB/s | 49 kB 00:00 2024-05-27T07:26:44.584 INFO:teuthology.orchestra.run.smithi090.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-27T07:26:44.653 INFO:teuthology.orchestra.run.smithi090.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-27T07:26:44.718 INFO:teuthology.orchestra.run.smithi090.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.2 MB/s | 839 kB 00:00 2024-05-27T07:26:44.759 INFO:teuthology.orchestra.run.smithi090.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-05-27T07:26:44.793 INFO:teuthology.orchestra.run.smithi090.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 624 kB/s | 43 kB 00:00 2024-05-27T07:26:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 485 kB/s | 32 kB 00:00 2024-05-27T07:26:44.852 INFO:teuthology.orchestra.run.smithi090.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-05-27T07:26:44.886 INFO:teuthology.orchestra.run.smithi090.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-27T07:26:44.928 INFO:teuthology.orchestra.run.smithi090.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 4.8 MB/s | 358 kB 00:00 2024-05-27T07:26:44.962 INFO:teuthology.orchestra.run.smithi090.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.2 MB/s | 303 kB 00:00 2024-05-27T07:26:44.987 INFO:teuthology.orchestra.run.smithi090.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 3.6 MB/s | 206 kB 00:00 2024-05-27T07:26:45.011 INFO:teuthology.orchestra.run.smithi129.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 282 kB/s | 292 kB 00:01 2024-05-27T07:26:45.097 INFO:teuthology.orchestra.run.smithi090.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-05-27T07:26:45.130 INFO:teuthology.orchestra.run.smithi090.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-05-27T07:26:45.156 INFO:teuthology.orchestra.run.smithi090.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 189 kB/s | 11 kB 00:00 2024-05-27T07:26:45.165 INFO:teuthology.orchestra.run.smithi129.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 53 kB/s | 35 kB 00:00 2024-05-27T07:26:45.181 INFO:teuthology.orchestra.run.smithi090.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 358 kB/s | 18 kB 00:00 2024-05-27T07:26:45.206 INFO:teuthology.orchestra.run.smithi090.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 470 kB/s | 23 kB 00:00 2024-05-27T07:26:45.241 INFO:teuthology.orchestra.run.smithi129.stdout:(66/159): lua-5.4.4-4.el9.x86_64.rpm 204 kB/s | 188 kB 00:00 2024-05-27T07:26:45.249 INFO:teuthology.orchestra.run.smithi090.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 295 kB/s | 19 kB 00:00 2024-05-27T07:26:45.301 INFO:teuthology.orchestra.run.smithi090.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 212 kB/s | 20 kB 00:00 2024-05-27T07:26:45.366 INFO:teuthology.orchestra.run.smithi090.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-05-27T07:26:45.391 INFO:teuthology.orchestra.run.smithi090.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 569 kB/s | 46 kB 00:00 2024-05-27T07:26:45.425 INFO:teuthology.orchestra.run.smithi090.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-27T07:26:45.459 INFO:teuthology.orchestra.run.smithi090.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 893 kB/s | 58 kB 00:00 2024-05-27T07:26:45.501 INFO:teuthology.orchestra.run.smithi090.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-05-27T07:26:45.526 INFO:teuthology.orchestra.run.smithi090.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 260 kB/s | 16 kB 00:00 2024-05-27T07:26:45.559 INFO:teuthology.orchestra.run.smithi090.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-27T07:26:45.593 INFO:teuthology.orchestra.run.smithi090.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 431 kB/s | 28 kB 00:00 2024-05-27T07:26:45.627 INFO:teuthology.orchestra.run.smithi090.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-05-27T07:26:45.661 INFO:teuthology.orchestra.run.smithi090.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-05-27T07:26:45.694 INFO:teuthology.orchestra.run.smithi090.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 917 kB/s | 59 kB 00:00 2024-05-27T07:26:45.728 INFO:teuthology.orchestra.run.smithi090.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 543 kB/s | 36 kB 00:00 2024-05-27T07:26:45.762 INFO:teuthology.orchestra.run.smithi090.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 774 kB/s | 51 kB 00:00 2024-05-27T07:26:45.796 INFO:teuthology.orchestra.run.smithi090.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-05-27T07:26:45.829 INFO:teuthology.orchestra.run.smithi090.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.4 MB/s | 90 kB 00:00 2024-05-27T07:26:45.871 INFO:teuthology.orchestra.run.smithi090.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-05-27T07:26:45.898 INFO:teuthology.orchestra.run.smithi090.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 317 kB/s | 20 kB 00:00 2024-05-27T07:26:45.957 INFO:teuthology.orchestra.run.smithi090.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-27T07:26:46.049 INFO:teuthology.orchestra.run.smithi090.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-27T07:26:46.340 INFO:teuthology.orchestra.run.smithi129.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 3.6 MB/s | 4.7 MB 00:01 2024-05-27T07:26:46.391 INFO:teuthology.orchestra.run.smithi090.stdout:(148/159): librbd1-19.0.0-3872.g38f9d900.el9.x8 9.1 MB/s | 3.1 MB 00:00 2024-05-27T07:26:46.516 INFO:teuthology.orchestra.run.smithi129.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-05-27T07:26:46.524 INFO:teuthology.orchestra.run.smithi090.stdout:(149/159): librados2-19.0.0-3872.g38f9d900.el9. 6.0 MB/s | 3.3 MB 00:00 2024-05-27T07:26:46.641 INFO:teuthology.orchestra.run.smithi090.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 9.3 MB/s | 2.3 MB 00:00 2024-05-27T07:26:46.657 INFO:teuthology.orchestra.run.smithi129.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.0 MB/s | 4.4 MB 00:01 2024-05-27T07:26:46.683 INFO:teuthology.orchestra.run.smithi090.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-27T07:26:46.734 INFO:teuthology.orchestra.run.smithi090.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 4.9 MB/s | 248 kB 00:00 2024-05-27T07:26:46.801 INFO:teuthology.orchestra.run.smithi090.stdout:(153/159): NetworkManager-libnm-1.47.90-1.el9.x 6.6 MB/s | 1.8 MB 00:00 2024-05-27T07:26:46.826 INFO:teuthology.orchestra.run.smithi090.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 402 kB/s | 37 kB 00:00 2024-05-27T07:26:46.860 INFO:teuthology.orchestra.run.smithi090.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-05-27T07:26:46.894 INFO:teuthology.orchestra.run.smithi090.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 3.8 MB/s | 260 kB 00:00 2024-05-27T07:26:46.994 INFO:teuthology.orchestra.run.smithi090.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-05-27T07:26:47.086 INFO:teuthology.orchestra.run.smithi090.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-05-27T07:26:47.466 INFO:teuthology.orchestra.run.smithi129.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 140 kB/s | 133 kB 00:00 2024-05-27T07:26:47.525 INFO:teuthology.orchestra.run.smithi129.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 770 kB/s | 911 kB 00:01 2024-05-27T07:26:48.075 INFO:teuthology.orchestra.run.smithi129.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 4.3 MB/s | 5.8 MB 00:01 2024-05-27T07:26:48.284 INFO:teuthology.orchestra.run.smithi129.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 115 kB/s | 87 kB 00:00 2024-05-27T07:26:48.393 INFO:teuthology.orchestra.run.smithi090.stdout:(159/159): ceph-test-19.0.0-3872.g38f9d900.el9. 3.5 MB/s | 43 MB 00:12 2024-05-27T07:26:48.401 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-05-27T07:26:48.401 INFO:teuthology.orchestra.run.smithi090.stdout:Total 14 MB/s | 219 MB 00:15 2024-05-27T07:26:48.626 INFO:teuthology.orchestra.run.smithi129.stdout:(74/159): protobuf-3.14.0-13.el9.x86_64.rpm 892 kB/s | 1.0 MB 00:01 2024-05-27T07:26:49.034 INFO:teuthology.orchestra.run.smithi129.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 82 kB/s | 61 kB 00:00 2024-05-27T07:26:49.418 INFO:teuthology.orchestra.run.smithi129.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 4.4 MB/s | 6.0 MB 00:01 2024-05-27T07:26:49.535 INFO:teuthology.orchestra.run.smithi129.stdout:(77/159): python3-devel-3.9.18-3.el9.x86_64.rpm 269 kB/s | 244 kB 00:00 2024-05-27T07:26:49.960 INFO:teuthology.orchestra.run.smithi129.stdout:(78/159): python3-jinja2-2.11.3-6.el9.noarch.rp 269 kB/s | 248 kB 00:00 2024-05-27T07:26:50.160 INFO:teuthology.orchestra.run.smithi129.stdout:(79/159): python3-jsonpatch-1.21-16.el9.noarch. 35 kB/s | 26 kB 00:00 2024-05-27T07:26:50.185 INFO:teuthology.orchestra.run.smithi129.stdout:(80/159): python3-jsonpointer-2.0-4.el9.noarch. 29 kB/s | 19 kB 00:00 2024-05-27T07:26:50.844 INFO:teuthology.orchestra.run.smithi129.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 51 kB/s | 35 kB 00:00 2024-05-27T07:26:50.878 INFO:teuthology.orchestra.run.smithi129.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 188 kB/s | 172 kB 00:00 2024-05-27T07:26:51.629 INFO:teuthology.orchestra.run.smithi129.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.7 MB/s | 5.1 MB 00:01 2024-05-27T07:26:51.634 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-05-27T07:26:51.761 INFO:teuthology.orchestra.run.smithi129.stdout:(84/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 274 kB/s | 250 kB 00:00 2024-05-27T07:26:51.787 INFO:teuthology.orchestra.run.smithi129.stdout:(85/159): python3-oauthlib-3.1.1-5.el9.noarch.r 245 kB/s | 222 kB 00:00 2024-05-27T07:26:51.889 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-05-27T07:26:51.889 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-05-27T07:26:52.454 INFO:teuthology.orchestra.run.smithi129.stdout:(86/159): python3-packaging-20.9-5.el9.noarch.r 94 kB/s | 77 kB 00:00 2024-05-27T07:26:52.612 INFO:teuthology.orchestra.run.smithi129.stdout:(87/159): python3-prettytable-0.7.2-27.el9.noar 51 kB/s | 42 kB 00:00 2024-05-27T07:26:52.787 INFO:teuthology.orchestra.run.smithi129.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 270 kB/s | 269 kB 00:00 2024-05-27T07:26:53.321 INFO:teuthology.orchestra.run.smithi129.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 183 kB/s | 159 kB 00:00 2024-05-27T07:26:53.571 INFO:teuthology.orchestra.run.smithi129.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 65 kB/s | 51 kB 00:00 2024-05-27T07:26:53.680 INFO:teuthology.orchestra.run.smithi129.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 262 kB/s | 279 kB 00:01 2024-05-27T07:26:54.055 INFO:teuthology.orchestra.run.smithi129.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 73 kB/s | 54 kB 00:00 2024-05-27T07:26:54.372 INFO:teuthology.orchestra.run.smithi129.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 60 kB/s | 42 kB 00:00 2024-05-27T07:26:54.616 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-05-27T07:26:54.617 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-05-27T07:26:54.747 INFO:teuthology.orchestra.run.smithi129.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 64 kB/s | 44 kB 00:00 2024-05-27T07:26:55.271 INFO:teuthology.orchestra.run.smithi129.stdout:(95/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-27T07:26:55.306 INFO:teuthology.orchestra.run.smithi129.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 237 kB/s | 221 kB 00:00 2024-05-27T07:26:55.648 INFO:teuthology.orchestra.run.smithi129.stdout:(97/159): qatlib-service-24.02.0-1.el9.x86_64.r 40 kB/s | 36 kB 00:00 2024-05-27T07:26:55.981 INFO:teuthology.orchestra.run.smithi129.stdout:(98/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 67 kB/s | 47 kB 00:00 2024-05-27T07:26:56.307 INFO:teuthology.orchestra.run.smithi129.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 305 kB/s | 305 kB 00:00 2024-05-27T07:26:56.365 INFO:teuthology.orchestra.run.smithi129.stdout:(100/159): uuid-1.6.2-55.el9.x86_64.rpm 81 kB/s | 58 kB 00:00 2024-05-27T07:26:56.799 INFO:teuthology.orchestra.run.smithi129.stdout:(101/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 115 kB/s | 94 kB 00:00 2024-05-27T07:26:57.024 INFO:teuthology.orchestra.run.smithi129.stdout:(102/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 89 kB/s | 64 kB 00:00 2024-05-27T07:26:57.049 INFO:teuthology.orchestra.run.smithi129.stdout:(103/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 66 kB/s | 45 kB 00:00 2024-05-27T07:26:57.075 INFO:teuthology.orchestra.run.smithi129.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-05-27T07:26:57.621 INFO:teuthology.orchestra.run.smithi129.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 566 kB/s | 309 kB 00:00 2024-05-27T07:26:57.658 INFO:teuthology.orchestra.run.smithi129.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 902 kB/s | 548 kB 00:00 2024-05-27T07:26:57.717 INFO:teuthology.orchestra.run.smithi129.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 205 kB/s | 19 kB 00:00 2024-05-27T07:26:57.826 INFO:teuthology.orchestra.run.smithi129.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 241 kB/s | 26 kB 00:00 2024-05-27T07:26:57.926 INFO:teuthology.orchestra.run.smithi129.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 490 kB/s | 49 kB 00:00 2024-05-27T07:26:58.026 INFO:teuthology.orchestra.run.smithi129.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 679 kB/s | 67 kB 00:00 2024-05-27T07:26:58.127 INFO:teuthology.orchestra.run.smithi129.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-05-27T07:26:58.160 INFO:teuthology.orchestra.run.smithi129.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-05-27T07:26:58.252 INFO:teuthology.orchestra.run.smithi129.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 839 kB 00:00 2024-05-27T07:26:58.336 INFO:teuthology.orchestra.run.smithi129.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-05-27T07:26:58.361 INFO:teuthology.orchestra.run.smithi129.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-05-27T07:26:58.459 INFO:teuthology.orchestra.run.smithi129.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 262 kB/s | 32 kB 00:00 2024-05-27T07:26:58.487 INFO:teuthology.orchestra.run.smithi129.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-05-27T07:26:58.562 INFO:teuthology.orchestra.run.smithi129.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-27T07:26:58.596 INFO:teuthology.orchestra.run.smithi129.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-05-27T07:26:58.671 INFO:teuthology.orchestra.run.smithi129.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-05-27T07:26:58.704 INFO:teuthology.orchestra.run.smithi129.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-05-27T07:26:58.805 INFO:teuthology.orchestra.run.smithi129.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-27T07:26:58.872 INFO:teuthology.orchestra.run.smithi129.stdout:(123/159): protobuf-compiler-3.14.0-13.el9.x86_ 468 kB/s | 863 kB 00:01 2024-05-27T07:26:58.905 INFO:teuthology.orchestra.run.smithi129.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-05-27T07:26:59.104 INFO:teuthology.orchestra.run.smithi129.stdout:(125/159): python3-jaraco-collections-3.0.0-8.e 117 kB/s | 23 kB 00:00 2024-05-27T07:26:59.131 INFO:teuthology.orchestra.run.smithi129.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 68 kB/s | 18 kB 00:00 2024-05-27T07:26:59.198 INFO:teuthology.orchestra.run.smithi129.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 210 kB/s | 19 kB 00:00 2024-05-27T07:26:59.223 INFO:teuthology.orchestra.run.smithi129.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 215 kB/s | 20 kB 00:00 2024-05-27T07:26:59.390 INFO:teuthology.orchestra.run.smithi129.stdout:(129/159): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-05-27T07:26:59.416 INFO:teuthology.orchestra.run.smithi129.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 242 kB/s | 46 kB 00:00 2024-05-27T07:26:59.491 INFO:teuthology.orchestra.run.smithi129.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 791 kB/s | 79 kB 00:00 2024-05-27T07:26:59.566 INFO:teuthology.orchestra.run.smithi129.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 385 kB/s | 58 kB 00:00 2024-05-27T07:26:59.600 INFO:teuthology.orchestra.run.smithi129.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-27T07:26:59.650 INFO:teuthology.orchestra.run.smithi129.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-05-27T07:26:59.734 INFO:teuthology.orchestra.run.smithi129.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 676 kB/s | 90 kB 00:00 2024-05-27T07:26:59.759 INFO:teuthology.orchestra.run.smithi129.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 261 kB/s | 28 kB 00:00 2024-05-27T07:26:59.826 INFO:teuthology.orchestra.run.smithi129.stdout:(137/159): python3-grpcio-1.46.7-10.el9.x86_64. 1.8 MB/s | 2.0 MB 00:01 2024-05-27T07:26:59.851 INFO:teuthology.orchestra.run.smithi129.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 264 kB/s | 31 kB 00:00 2024-05-27T07:26:59.926 INFO:teuthology.orchestra.run.smithi129.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-05-27T07:26:59.952 INFO:teuthology.orchestra.run.smithi129.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 359 kB/s | 36 kB 00:00 2024-05-27T07:26:59.977 INFO:teuthology.orchestra.run.smithi129.stdout:(141/159): python3-rsa-4.9-2.el9.noarch.rpm 394 kB/s | 59 kB 00:00 2024-05-27T07:27:00.022 INFO:teuthology.orchestra.run.smithi129.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 557 kB/s | 51 kB 00:00 2024-05-27T07:27:00.053 INFO:teuthology.orchestra.run.smithi129.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-27T07:27:00.077 INFO:teuthology.orchestra.run.smithi129.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 898 kB/s | 90 kB 00:00 2024-05-27T07:27:00.153 INFO:teuthology.orchestra.run.smithi129.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 201 kB/s | 20 kB 00:00 2024-05-27T07:27:00.187 INFO:teuthology.orchestra.run.smithi129.stdout:(146/159): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-05-27T07:27:00.212 INFO:teuthology.orchestra.run.smithi129.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-05-27T07:27:00.296 INFO:teuthology.orchestra.run.smithi129.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-27T07:27:00.479 INFO:teuthology.orchestra.run.smithi129.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-27T07:27:00.580 INFO:teuthology.orchestra.run.smithi129.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-05-27T07:27:00.622 INFO:teuthology.orchestra.run.smithi129.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-27T07:27:00.663 INFO:teuthology.orchestra.run.smithi129.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 5.9 MB/s | 248 kB 00:00 2024-05-27T07:27:00.705 INFO:teuthology.orchestra.run.smithi129.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 901 kB/s | 37 kB 00:00 2024-05-27T07:27:00.747 INFO:teuthology.orchestra.run.smithi129.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 5.0 MB/s | 207 kB 00:00 2024-05-27T07:27:00.798 INFO:teuthology.orchestra.run.smithi129.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.2 MB/s | 260 kB 00:00 2024-05-27T07:27:00.881 INFO:teuthology.orchestra.run.smithi129.stdout:(156/159): librbd1-19.0.0-3872.g38f9d900.el9.x8 4.6 MB/s | 3.1 MB 00:00 2024-05-27T07:27:00.948 INFO:teuthology.orchestra.run.smithi129.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-05-27T07:27:01.056 INFO:teuthology.orchestra.run.smithi129.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-05-27T07:27:01.198 INFO:teuthology.orchestra.run.smithi129.stdout:(159/159): librados2-19.0.0-3872.g38f9d900.el9. 3.3 MB/s | 3.3 MB 00:01 2024-05-27T07:27:01.206 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-05-27T07:27:01.206 INFO:teuthology.orchestra.run.smithi129.stdout:Total 6.3 MB/s | 219 MB 00:34 2024-05-27T07:27:01.388 INFO:teuthology.orchestra.run.smithi136.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 758 kB/s | 16 MB 00:21 2024-05-27T07:27:01.392 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-05-27T07:27:01.395 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-05-27T07:27:01.396 INFO:teuthology.orchestra.run.smithi136.stdout:Total 6.1 MB/s | 219 MB 00:36 2024-05-27T07:27:01.504 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T07:27:01.591 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T07:27:01.715 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T07:27:01.775 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T07:27:01.815 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T07:27:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T07:27:02.339 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T07:27:02.488 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:27:02.668 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:27:02.698 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:27:02.825 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:27:02.864 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:27:02.969 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T07:27:03.337 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T07:27:03.397 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T07:27:03.422 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:27:03.573 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:27:03.598 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:27:03.717 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:27:03.741 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:27:03.802 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:27:04.096 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:27:04.419 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:27:04.440 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-05-27T07:27:04.542 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-05-27T07:27:04.694 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-05-27T07:27:04.694 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-05-27T07:27:04.779 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:27:04.793 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-05-27T07:27:04.793 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-05-27T07:27:04.872 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:27:05.064 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:27:05.221 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:27:05.593 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:27:05.782 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T07:27:05.821 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T07:27:05.881 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T07:27:05.998 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T07:27:06.059 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T07:27:06.136 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T07:27:06.188 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T07:27:06.228 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T07:27:06.348 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T07:27:06.418 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T07:27:06.474 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T07:27:06.542 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T07:27:06.658 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T07:27:06.763 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 33/170 2024-05-27T07:27:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:27:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 35/170 2024-05-27T07:27:06.981 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T07:27:07.048 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T07:27:07.159 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T07:27:07.337 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-05-27T07:27:07.337 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T07:27:07.338 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-05-27T07:27:07.402 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-05-27T07:27:07.403 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-05-27T07:27:07.407 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T07:27:08.109 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T07:27:08.212 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T07:27:08.250 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T07:27:08.299 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T07:27:08.365 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T07:27:08.430 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 46/170 2024-05-27T07:27:08.511 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T07:27:08.738 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T07:27:08.814 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T07:27:08.945 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T07:27:09.043 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T07:27:09.139 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T07:27:09.182 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T07:27:09.221 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T07:27:09.288 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T07:27:09.329 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T07:27:09.391 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T07:27:09.459 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T07:27:09.657 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T07:27:09.706 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T07:27:09.752 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T07:27:09.876 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T07:27:09.924 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T07:27:10.018 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T07:27:10.059 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T07:27:10.092 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T07:27:10.286 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T07:27:10.462 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T07:27:10.516 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T07:27:10.554 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T07:27:10.600 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T07:27:10.716 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T07:27:10.775 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T07:27:10.821 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T07:27:10.867 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T07:27:10.916 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T07:27:10.992 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T07:27:11.040 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T07:27:11.192 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T07:27:11.252 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T07:27:11.298 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T07:27:11.369 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T07:27:11.517 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T07:27:11.585 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T07:27:12.552 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T07:27:12.621 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T07:27:12.655 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T07:27:12.868 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T07:27:13.329 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T07:27:13.421 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T07:27:13.552 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T07:27:13.592 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T07:27:13.843 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-05-27T07:27:13.855 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T07:27:13.886 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-05-27T07:27:13.896 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T07:27:13.968 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T07:27:13.980 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T07:27:14.047 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T07:27:14.049 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T07:27:14.051 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T07:27:14.151 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T07:27:14.162 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T07:27:14.208 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T07:27:14.223 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T07:27:14.240 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T07:27:14.254 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T07:27:14.352 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T07:27:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T07:27:14.796 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T07:27:14.830 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T07:27:14.905 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:27:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:27:15.014 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T07:27:15.089 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:27:15.123 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:27:15.140 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:27:15.154 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T07:27:15.169 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:27:15.250 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:27:15.289 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:27:15.295 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:27:15.330 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:27:15.384 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T07:27:15.411 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T07:27:15.738 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T07:27:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T07:27:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T07:27:15.811 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T07:27:15.836 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:27:15.867 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:27:15.998 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:27:16.019 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:27:16.029 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:27:16.052 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:27:16.123 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:27:16.158 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:27:16.171 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 15/170 2024-05-27T07:27:16.195 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:27:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:27:16.263 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T07:27:16.514 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:27:16.566 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:27:16.770 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T07:27:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:27:16.884 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T07:27:16.892 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T07:27:16.921 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T07:27:16.965 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T07:27:17.052 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:27:17.131 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:27:17.138 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:27:17.216 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T07:27:17.310 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:27:17.349 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T07:27:17.375 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:27:17.397 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:27:17.413 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:27:17.515 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:27:17.554 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:27:17.589 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 104/170 2024-05-27T07:27:17.655 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T07:27:17.726 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:27:17.838 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T07:27:17.915 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T07:27:17.944 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T07:27:17.998 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T07:27:18.048 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T07:27:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T07:27:18.104 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T07:27:18.130 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T07:27:18.149 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T07:27:18.212 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T07:27:18.213 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T07:27:18.235 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T07:27:18.260 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:27:18.260 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T07:27:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T07:27:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T07:27:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T07:27:18.386 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:27:18.406 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T07:27:18.412 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 108/170 2024-05-27T07:27:18.438 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T07:27:18.472 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T07:27:18.478 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T07:27:18.504 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T07:27:18.558 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T07:27:18.597 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T07:27:18.664 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T07:27:18.668 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T07:27:18.703 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T07:27:18.716 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 33/170 2024-05-27T07:27:18.758 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T07:27:18.770 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:27:18.824 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 35/170 2024-05-27T07:27:18.869 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T07:27:18.916 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 33/170 2024-05-27T07:27:18.945 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T07:27:18.968 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:27:19.004 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T07:27:19.006 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 35/170 2024-05-27T07:27:19.114 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T07:27:19.146 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T07:27:19.210 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T07:27:19.284 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T07:27:19.310 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T07:27:19.346 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T07:27:19.488 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T07:27:19.554 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T07:27:20.043 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T07:27:20.143 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T07:27:20.179 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T07:27:20.221 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T07:27:20.238 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T07:27:20.274 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T07:27:20.321 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 46/170 2024-05-27T07:27:20.343 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T07:27:20.387 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T07:27:20.393 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T07:27:20.427 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T07:27:20.485 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T07:27:20.539 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 46/170 2024-05-27T07:27:20.613 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T07:27:20.614 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T07:27:20.674 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T07:27:20.832 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T07:27:20.833 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T07:27:20.899 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T07:27:20.912 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T07:27:21.022 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T07:27:21.022 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T07:27:21.027 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:27:21.077 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:27:21.103 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T07:27:21.191 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T07:27:21.233 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T07:27:21.270 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T07:27:21.343 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T07:27:21.364 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T07:27:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T07:27:21.419 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T07:27:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T07:27:21.509 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T07:27:21.516 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T07:27:21.549 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T07:27:21.616 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T07:27:21.678 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T07:27:21.715 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T07:27:21.765 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T07:27:21.769 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:27:21.818 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T07:27:21.823 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:27:21.909 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T07:27:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T07:27:21.964 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T07:27:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T07:27:22.012 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:27:22.012 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T07:27:22.012 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T07:27:22.012 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:22.017 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T07:27:22.049 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:27:22.054 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T07:27:22.096 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T07:27:22.129 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T07:27:22.147 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T07:27:22.188 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T07:27:22.233 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T07:27:22.270 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T07:27:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T07:27:22.343 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T07:27:22.498 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T07:27:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T07:27:22.569 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T07:27:22.607 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T07:27:22.654 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T07:27:22.680 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T07:27:22.721 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T07:27:22.751 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T07:27:22.774 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T07:27:22.792 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T07:27:22.822 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T07:27:22.839 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T07:27:22.869 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T07:27:22.915 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T07:27:22.946 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T07:27:22.976 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T07:27:23.022 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T07:27:23.023 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T07:27:23.069 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T07:27:23.075 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T07:27:23.118 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T07:27:23.195 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T07:27:23.219 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T07:27:23.241 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T07:27:23.279 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T07:27:23.331 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T07:27:23.382 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T07:27:23.411 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T07:27:23.446 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T07:27:23.490 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T07:27:23.559 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T07:27:23.573 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T07:27:23.647 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T07:27:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T07:27:23.776 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T07:27:24.608 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T07:27:24.663 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T07:27:24.696 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T07:27:24.728 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T07:27:24.784 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T07:27:24.817 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T07:27:24.818 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T07:27:24.879 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T07:27:24.938 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T07:27:24.940 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T07:27:25.000 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T07:27:25.052 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T07:27:25.053 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T07:27:25.099 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T07:27:25.152 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T07:27:25.195 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T07:27:25.334 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T07:27:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T07:27:25.417 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T07:27:25.442 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T07:27:25.466 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T07:27:25.540 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T07:27:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T07:27:26.404 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T07:27:26.427 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T07:27:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T07:27:28.034 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T07:27:28.135 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T07:27:28.139 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T07:27:28.170 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T07:27:28.222 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T07:27:28.238 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T07:27:28.273 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T07:27:28.309 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T07:27:28.610 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T07:27:28.690 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T07:27:28.720 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:27:28.734 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:27:28.838 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:27:28.874 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 104/170 2024-05-27T07:27:28.928 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T07:27:29.127 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 103/170 2024-05-27T07:27:29.163 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 104/170 2024-05-27T07:27:29.221 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T07:27:29.472 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T07:27:29.511 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:27:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:27:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 108/170 2024-05-27T07:27:29.826 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T07:27:29.859 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:27:29.984 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 107/170 2024-05-27T07:27:30.011 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 108/170 2024-05-27T07:27:32.314 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:27:32.360 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:27:32.653 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:27:32.708 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:27:33.023 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:27:33.080 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:27:33.240 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:27:33.240 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T07:27:33.240 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T07:27:33.240 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:33.306 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:27:33.380 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 109/170 2024-05-27T07:27:33.428 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:27:33.599 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 110/170 2024-05-27T07:27:33.599 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T07:27:33.599 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T07:27:33.599 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:27:33.637 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:27:37.115 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:27:37.115 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-05-27T07:27:37.115 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-05-27T07:27:37.116 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-05-27T07:27:37.116 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-05-27T07:27:37.116 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-05-27T07:27:37.116 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-05-27T07:27:37.116 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-05-27T07:27:37.116 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:37.348 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:27:37.393 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:27:37.394 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:37.394 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T07:27:37.394 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:39.119 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:27:39.196 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:27:39.196 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:39.196 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T07:27:39.196 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:27:39.196 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:27:39.196 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:39.460 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:27:39.533 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:27:39.533 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:39.533 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T07:27:39.533 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:27:39.533 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:27:39.533 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:40.070 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:27:40.150 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:27:40.150 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:40.151 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T07:27:40.151 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:27:40.151 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:27:40.151 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:40.287 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T07:27:40.351 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T07:27:40.425 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T07:27:40.663 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T07:27:40.726 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T07:27:40.812 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T07:27:40.920 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T07:27:41.089 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T07:27:41.143 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T07:27:41.183 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T07:27:41.218 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T07:27:41.907 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:27:41.951 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:27:42.016 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 128/170 2024-05-27T07:27:42.136 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 129/170 2024-05-27T07:27:42.174 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:27:43.492 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:27:43.536 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:27:44.633 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:27:44.694 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:27:44.858 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:27:45.040 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 133/170 2024-05-27T07:27:45.072 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:27:45.146 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:27:45.146 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:45.146 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T07:27:45.146 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:27:45.146 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:27:45.147 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:45.197 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:27:45.234 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:27:45.341 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 136/170 2024-05-27T07:27:45.409 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 137/170 2024-05-27T07:27:46.470 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T07:27:46.502 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:27:46.584 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:27:46.584 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-05-27T07:27:46.584 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-05-27T07:27:46.584 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-05-27T07:27:46.584 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-05-27T07:27:46.584 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-05-27T07:27:46.584 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-05-27T07:27:46.584 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-05-27T07:27:46.585 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:46.592 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:27:46.592 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:46.593 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T07:27:46.593 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:27:46.593 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:27:46.593 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:46.606 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 111/170 2024-05-27T07:27:46.606 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-05-27T07:27:46.606 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-05-27T07:27:46.606 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-05-27T07:27:46.606 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-05-27T07:27:46.607 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-05-27T07:27:46.607 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-05-27T07:27:46.607 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-05-27T07:27:46.607 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:27:46.661 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:27:46.762 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:27:46.762 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:46.762 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T07:27:46.762 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:46.793 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:27:46.835 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:27:46.835 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:46.835 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T07:27:46.835 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:46.843 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:27:46.845 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:27:46.890 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 112/170 2024-05-27T07:27:46.890 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:46.890 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T07:27:46.890 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:27:46.943 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:27:46.943 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:46.943 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T07:27:46.943 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:27:46.943 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:27:46.943 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:47.329 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:27:47.420 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:27:47.420 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:47.420 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T07:27:47.420 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:27:47.421 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:27:47.421 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:48.512 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:27:48.614 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:27:48.614 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:48.614 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T07:27:48.615 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:27:48.615 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:27:48.615 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:27:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 113/170 2024-05-27T07:27:48.750 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:48.750 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T07:27:48.750 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:27:48.750 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T07:27:48.750 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:27:48.862 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:27:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:27:48.951 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:48.951 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T07:27:48.951 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:27:48.952 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:27:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:49.029 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:27:49.102 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 114/170 2024-05-27T07:27:49.103 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:49.103 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T07:27:49.103 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:27:49.103 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T07:27:49.103 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:27:49.463 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:27:49.524 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:27:49.525 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:49.525 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T07:27:49.525 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:27:49.525 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:27:49.525 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:49.621 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:27:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T07:27:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 115/170 2024-05-27T07:27:49.689 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:49.689 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T07:27:49.689 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:27:49.689 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T07:27:49.689 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:27:49.727 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T07:27:49.802 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T07:27:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T07:27:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T07:27:49.974 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T07:27:50.042 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T07:27:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T07:27:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T07:27:50.210 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T07:27:50.267 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T07:27:50.285 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T07:27:50.353 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T07:27:50.463 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T07:27:50.463 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T07:27:50.516 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T07:27:50.566 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T07:27:50.598 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T07:27:50.629 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T07:27:50.692 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T07:27:50.753 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T07:27:50.789 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T07:27:51.440 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:27:51.479 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:27:51.534 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 128/170 2024-05-27T07:27:51.598 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:27:51.634 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 127/170 2024-05-27T07:27:51.642 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 129/170 2024-05-27T07:27:51.680 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:27:51.686 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 128/170 2024-05-27T07:27:51.787 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 129/170 2024-05-27T07:27:51.836 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:27:52.062 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 143/170 2024-05-27T07:27:52.115 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 144/170 2024-05-27T07:27:52.168 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 145/170 2024-05-27T07:27:52.203 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T07:27:52.241 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T07:27:52.514 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 148/170 2024-05-27T07:27:52.577 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T07:27:52.612 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 150/170 2024-05-27T07:27:52.651 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T07:27:52.710 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T07:27:52.769 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:27:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:27:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:27:53.151 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 130/170 2024-05-27T07:27:53.193 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:27:53.748 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:27:53.802 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:27:53.857 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:27:54.134 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:27:54.166 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:27:54.285 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 131/170 2024-05-27T07:27:54.322 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:27:54.326 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:27:54.479 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 132/170 2024-05-27T07:27:54.533 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 133/170 2024-05-27T07:27:54.568 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:27:54.640 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:27:54.640 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:54.640 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T07:27:54.640 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:27:54.641 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:27:54.641 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:27:54.658 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 133/170 2024-05-27T07:27:54.683 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:27:54.702 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:27:54.741 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:27:54.753 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 134/170 2024-05-27T07:27:54.753 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:54.753 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T07:27:54.753 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:27:54.753 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T07:27:54.754 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:27:54.854 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 135/170 2024-05-27T07:27:54.886 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 136/170 2024-05-27T07:27:54.967 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 136/170 2024-05-27T07:27:54.971 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 137/170 2024-05-27T07:27:55.039 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 137/170 2024-05-27T07:27:55.207 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 155/170 2024-05-27T07:27:55.313 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T07:27:55.366 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T07:27:55.509 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T07:27:55.550 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T07:27:55.584 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:27:55.618 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T07:27:55.660 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T07:27:55.660 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:27:55.803 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:27:55.847 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T07:27:55.852 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:27:55.992 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:27:56.033 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T07:27:56.063 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T07:27:56.076 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:27:56.076 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:27:56.113 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:27:56.120 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T07:27:56.138 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:27:56.164 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:27:56.164 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:56.164 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T07:27:56.164 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:27:56.164 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:27:56.164 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:56.229 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:27:56.232 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 139/170 2024-05-27T07:27:56.232 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:56.233 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T07:27:56.233 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:27:56.233 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T07:27:56.233 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:27:56.265 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:27:56.308 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:27:56.308 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:56.308 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T07:27:56.308 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:56.351 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 140/170 2024-05-27T07:27:56.351 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:56.351 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T07:27:56.351 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:27:56.392 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:27:56.410 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:27:56.426 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:27:56.440 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:27:56.462 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:27:56.462 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:56.463 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T07:27:56.463 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:27:56.463 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:27:56.463 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:56.559 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 141/170 2024-05-27T07:27:56.559 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:56.559 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T07:27:56.560 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:27:56.560 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T07:27:56.560 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:27:56.565 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:27:56.622 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T07:27:56.731 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:27:56.822 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:27:56.901 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:27:56.902 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:56.902 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T07:27:56.902 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:27:56.902 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:27:56.902 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:27:56.973 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:27:56.989 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:27:56.989 INFO:teuthology.orchestra.run.smithi090.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T07:27:56.989 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:27:57.062 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 142/170 2024-05-27T07:27:57.062 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T07:27:57.062 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T07:27:57.062 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:27:57.066 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T07:27:57.066 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:28:01.566 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 143/170 2024-05-27T07:28:01.623 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 144/170 2024-05-27T07:28:01.678 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 145/170 2024-05-27T07:28:01.718 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T07:28:01.754 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 143/170 2024-05-27T07:28:01.756 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T07:28:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 144/170 2024-05-27T07:28:01.861 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 145/170 2024-05-27T07:28:01.895 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T07:28:01.933 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T07:28:02.044 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 148/170 2024-05-27T07:28:02.106 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T07:28:02.139 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 150/170 2024-05-27T07:28:02.180 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T07:28:02.224 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 148/170 2024-05-27T07:28:02.231 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T07:28:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T07:28:02.281 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:28:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 150/170 2024-05-27T07:28:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T07:28:02.400 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T07:28:02.450 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:28:03.228 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:28:03.266 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:28:03.286 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:28:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:28:03.343 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:28:03.376 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T07:28:04.111 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 1/170 2024-05-27T07:28:04.111 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2/170 2024-05-27T07:28:04.111 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 3/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 4/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 5/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 6/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 7/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 11/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 12/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 13/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:28:04.112 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 15/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 16/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 17/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 18/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 19/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 20/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 21/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 22/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 23/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 24/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 25/170 2024-05-27T07:28:04.113 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 26/170 2024-05-27T07:28:04.114 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 27/170 2024-05-27T07:28:04.114 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 28/170 2024-05-27T07:28:04.114 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 29/170 2024-05-27T07:28:04.114 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 30/170 2024-05-27T07:28:04.114 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 31/170 2024-05-27T07:28:04.114 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 32/170 2024-05-27T07:28:04.114 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 33/170 2024-05-27T07:28:04.129 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:28:04.129 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 35/170 2024-05-27T07:28:04.129 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 36/170 2024-05-27T07:28:04.129 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 37/170 2024-05-27T07:28:04.129 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 38/170 2024-05-27T07:28:04.129 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 39/170 2024-05-27T07:28:04.130 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T07:28:04.130 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T07:28:04.130 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T07:28:04.130 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T07:28:04.130 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T07:28:04.130 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T07:28:04.130 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T07:28:04.130 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T07:28:04.130 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T07:28:04.130 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T07:28:04.131 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T07:28:04.131 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T07:28:04.131 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T07:28:04.131 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T07:28:04.131 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T07:28:04.131 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T07:28:04.131 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T07:28:04.131 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T07:28:04.131 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T07:28:04.132 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T07:28:04.133 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T07:28:04.134 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T07:28:04.135 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T07:28:04.136 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T07:28:04.137 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T07:28:04.138 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 149/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 151/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T07:28:04.139 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T07:28:04.140 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T07:28:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 155/170 2024-05-27T07:28:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 155/170 2024-05-27T07:28:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T07:28:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T07:28:04.847 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T07:28:04.908 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T07:28:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T07:28:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T07:28:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:28:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T07:28:05.035 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T07:28:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T07:28:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:28:05.082 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T07:28:05.124 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:28:05.158 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T07:28:05.199 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T07:28:05.199 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:28:05.230 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:28:05.276 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T07:28:05.276 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:28:05.342 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T07:28:05.388 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T07:28:05.388 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:28:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:28:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T07:28:05.525 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T07:28:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:28:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:28:05.563 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T07:28:05.602 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:28:05.602 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:28:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:28:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:28:05.935 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T07:28:05.935 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:28:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:28:06.089 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T07:28:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T07:28:06.134 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T07:28:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:28:06.253 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:28:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T07:28:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:28:06.299 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-05-27T07:28:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T07:28:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T07:28:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T07:28:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T07:28:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T07:28:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T07:28:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T07:28:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T07:28:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T07:28:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T07:28:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T07:28:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T07:28:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T07:28:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T07:28:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T07:28:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T07:28:06.312 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T07:28:06.312 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:28:06.312 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-05-27T07:28:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:28:06.487 INFO:teuthology.orchestra.run.smithi136.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T07:28:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:28:06.521 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T07:28:06.521 INFO:teuthology.orchestra.run.smithi129.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-27T07:28:06.521 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:28:07.022 DEBUG:teuthology.parallel:result is None 2024-05-27T07:28:12.307 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 1/170 2024-05-27T07:28:12.307 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2/170 2024-05-27T07:28:12.307 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 3/170 2024-05-27T07:28:12.307 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 4/170 2024-05-27T07:28:12.307 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 5/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 6/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 7/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 11/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 12/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 13/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 15/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 16/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 17/170 2024-05-27T07:28:12.308 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 18/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 19/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 20/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 21/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 22/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 23/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 24/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 25/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 26/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 27/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 28/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 29/170 2024-05-27T07:28:12.309 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 30/170 2024-05-27T07:28:12.322 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 31/170 2024-05-27T07:28:12.322 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 32/170 2024-05-27T07:28:12.322 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 33/170 2024-05-27T07:28:12.322 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:28:12.322 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 35/170 2024-05-27T07:28:12.322 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 36/170 2024-05-27T07:28:12.322 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 37/170 2024-05-27T07:28:12.322 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 38/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 39/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T07:28:12.323 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T07:28:12.324 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T07:28:12.325 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T07:28:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T07:28:12.327 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T07:28:12.328 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T07:28:12.329 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T07:28:12.330 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T07:28:12.331 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 149/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 151/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T07:28:12.332 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T07:28:12.333 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T07:28:12.333 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T07:28:12.333 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T07:28:12.333 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T07:28:12.333 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T07:28:12.333 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T07:28:12.333 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T07:28:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 1/170 2024-05-27T07:28:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2/170 2024-05-27T07:28:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 3/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 4/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3872.g38f9d 5/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 6/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 7/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 8/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 9/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 10/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 11/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 12/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 13/170 2024-05-27T07:28:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 14/170 2024-05-27T07:28:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_ 15/170 2024-05-27T07:28:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 16/170 2024-05-27T07:28:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 17/170 2024-05-27T07:28:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_6 18/170 2024-05-27T07:28:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86 19/170 2024-05-27T07:28:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 20/170 2024-05-27T07:28:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el 21/170 2024-05-27T07:28:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.0.0-3872.g38f9d900.el9. 22/170 2024-05-27T07:28:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_6 23/170 2024-05-27T07:28:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 24/170 2024-05-27T07:28:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 25/170 2024-05-27T07:28:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 26/170 2024-05-27T07:28:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 27/170 2024-05-27T07:28:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 28/170 2024-05-27T07:28:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 29/170 2024-05-27T07:28:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900. 30/170 2024-05-27T07:28:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noa 31/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.n 32/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f 33/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el 34/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 35/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.e 36/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 37/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 38/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 39/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T07:28:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T07:28:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T07:28:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T07:28:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T07:28:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T07:28:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T07:28:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T07:28:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T07:28:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T07:28:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T07:28:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T07:28:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T07:28:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T07:28:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T07:28:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T07:28:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T07:28:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 149/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 151/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T07:28:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T07:28:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T07:28:12.504 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T07:28:14.539 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.540 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.541 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T07:28:14.542 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T07:28:14.543 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T07:28:14.544 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T07:28:14.545 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T07:28:14.546 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T07:28:14.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T07:28:14.548 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.549 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T07:28:14.550 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T07:28:14.551 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T07:28:14.551 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.551 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.551 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.551 INFO:teuthology.orchestra.run.smithi136.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T07:28:14.551 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T07:28:14.551 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T07:28:14.551 INFO:teuthology.orchestra.run.smithi136.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T07:28:14.551 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T07:28:14.551 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T07:28:14.552 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T07:28:14.552 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:28:14.552 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T07:28:14.777 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.778 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.779 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top-2:19.0.0-3872.g38f9d900.el9.noarch 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T07:28:14.780 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.781 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T07:28:14.782 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.783 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T07:28:14.784 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T07:28:14.785 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T07:28:14.786 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T07:28:14.787 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.0.0-3872.g38f9d900.el9.x86_64 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T07:28:14.788 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T07:28:14.789 INFO:teuthology.orchestra.run.smithi129.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T07:28:14.789 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T07:28:14.789 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T07:28:14.789 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T07:28:14.789 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-05-27T07:28:14.789 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-05-27T07:28:15.200 DEBUG:teuthology.parallel:result is None 2024-05-27T07:28:15.485 DEBUG:teuthology.parallel:result is None 2024-05-27T07:28:15.486 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:28:15.721 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T07:28:15.794 INFO:teuthology.orchestra.run.smithi090.stdout:19.0.0-3872.g38f9d900.el9 2024-05-27T07:28:15.795 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3872.g38f9d900.el9 2024-05-27T07:28:15.795 INFO:teuthology.task.install:The correct ceph version 19.0.0-3872.g38f9d900 is installed. 2024-05-27T07:28:15.797 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:28:15.938 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T07:28:15.989 INFO:teuthology.orchestra.run.smithi129.stdout:19.0.0-3872.g38f9d900.el9 2024-05-27T07:28:15.989 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3872.g38f9d900.el9 2024-05-27T07:28:15.989 INFO:teuthology.task.install:The correct ceph version 19.0.0-3872.g38f9d900 is installed. 2024-05-27T07:28:15.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:28:16.217 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T07:28:16.269 INFO:teuthology.orchestra.run.smithi136.stdout:19.0.0-3872.g38f9d900.el9 2024-05-27T07:28:16.269 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3872.g38f9d900.el9 2024-05-27T07:28:16.269 INFO:teuthology.task.install:The correct ceph version 19.0.0-3872.g38f9d900 is installed. 2024-05-27T07:28:16.271 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-27T07:28:16.272 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:28:16.272 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T07:28:16.365 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:28:16.365 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T07:28:16.429 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:28:16.429 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T07:28:16.493 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-27T07:28:16.494 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:28:16.494 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T07:28:16.584 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T07:28:16.711 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:28:16.711 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T07:28:16.780 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T07:28:16.881 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:28:16.881 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T07:28:16.947 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T07:28:17.051 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-27T07:28:17.051 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:28:17.052 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T07:28:17.141 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T07:28:17.271 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:28:17.271 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T07:28:17.348 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T07:28:17.463 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:28:17.464 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T07:28:17.529 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T07:28:17.632 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-27T07:28:17.633 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:28:17.633 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T07:28:17.721 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T07:28:17.804 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:28:17.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T07:28:17.872 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T07:28:17.973 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:28:17.973 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T07:28:18.035 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T07:28:18.155 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-27T07:28:18.251 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'subvols': {'create': 2}} 2024-05-27T07:28:18.251 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:28:18.252 INFO:tasks.cephadm:Cluster fsid is b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:28:18.252 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-27T07:28:18.252 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.90', 'mon.b': '172.21.15.129', 'mon.c': '172.21.15.136'} 2024-05-27T07:28:18.252 INFO:tasks.cephadm:First mon is mon.a on smithi090 2024-05-27T07:28:18.252 INFO:tasks.cephadm:First mgr is x 2024-05-27T07:28:18.252 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-27T07:28:18.252 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-05-27T07:28:18.343 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname $(hostname -s) 2024-05-27T07:28:18.409 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-05-27T07:28:18.478 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-27T07:28:18.479 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:28:18.633 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79587/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3872-g38f9d900', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3872.g38f9d900'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-21 08:08:15.155937', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240521.063558-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '38f9d900c83c63799fbdbe61acc9a11b0d3554a6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79583/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3872-g38f9d900', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3872.g38f9d900'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-21 07:54:08.730060', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240521.063558-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-27T07:28:18.850 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-vshankar-testing-20240521.063558-debug, sha1 38f9d900c83c63799fbdbe61acc9a11b0d3554a6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:28:18.853 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm 2024-05-27T07:28:18.853 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm 2024-05-27T07:28:18.853 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:19.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 782786 May 27 07:28 /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:19.383 DEBUG:teuthology.orchestra.run.smithi129:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:20.092 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r--. 1 ubuntu ubuntu 782786 May 27 07:28 /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:20.093 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240521.063558-debug/38f9d900c83c63799fbdbe61acc9a11b0d3554a6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:20.867 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 782786 May 27 07:28 /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:20.868 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:20.927 DEBUG:teuthology.orchestra.run.smithi129:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:20.986 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T07:28:21.060 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 on all hosts... 2024-05-27T07:28:21.060 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 pull 2024-05-27T07:28:21.068 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 pull 2024-05-27T07:28:21.075 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 pull 2024-05-27T07:28:22.185 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T07:28:22.274 INFO:teuthology.orchestra.run.smithi129.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T07:28:22.374 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T07:29:12.248 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-05-27T07:29:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:29:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "image_id": "e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799", 2024-05-27T07:29:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "repo_digests": [ 2024-05-27T07:29:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6" 2024-05-27T07:29:12.248 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-05-27T07:29:12.248 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-05-27T07:29:18.067 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-05-27T07:29:18.067 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:29:18.067 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799", 2024-05-27T07:29:18.067 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-05-27T07:29:18.067 INFO:teuthology.orchestra.run.smithi136.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6" 2024-05-27T07:29:18.067 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-05-27T07:29:18.067 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-05-27T07:29:24.891 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:29:24.892 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:29:24.892 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799", 2024-05-27T07:29:24.892 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-05-27T07:29:24.892 INFO:teuthology.orchestra.run.smithi090.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6" 2024-05-27T07:29:24.892 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T07:29:24.892 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:29:24.925 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-05-27T07:29:25.001 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph 2024-05-27T07:29:25.090 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-05-27T07:29:25.185 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-05-27T07:29:25.263 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /etc/ceph 2024-05-27T07:29:25.348 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-05-27T07:29:25.436 INFO:tasks.cephadm:Writing seed config... 2024-05-27T07:29:25.437 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-27T07:29:25.437 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-27T07:29:25.437 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-27T07:29:25.438 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-27T07:29:25.438 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-27T07:29:25.438 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-27T07:29:25.438 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-27T07:29:25.438 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-27T07:29:25.438 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-27T07:29:25.438 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-27T07:29:25.438 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-27T07:29:25.438 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-27T07:29:25.439 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-27T07:29:25.440 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-27T07:29:25.441 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-27T07:29:25.441 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-27T07:29:25.441 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-27T07:29:25.441 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-27T07:29:25.441 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-27T07:29:25.441 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-27T07:29:25.441 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-27T07:29:25.441 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-27T07:29:25.442 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:29:25.442 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-27T07:29:25.489 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b0214938-1bfa-11ef-bc9b-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-27T07:29:25.489 DEBUG:teuthology.orchestra.run.smithi090:mon.a> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.a.service 2024-05-27T07:29:25.538 DEBUG:teuthology.orchestra.run.smithi090:mgr.x> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.x.service 2024-05-27T07:29:25.585 INFO:tasks.cephadm:Bootstrapping... 2024-05-27T07:29:25.585 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 -v bootstrap --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.90 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:29:25.856 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-05-27T07:29:25.857 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6', '-v', 'bootstrap', '--fsid', 'b0214938-1bfa-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.90', '--skip-admin-label'] 2024-05-27T07:29:25.857 INFO:teuthology.orchestra.run.smithi090.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-27T07:29:25.857 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying podman|docker is present... 2024-05-27T07:29:25.916 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 5.0.2 2024-05-27T07:29:25.916 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying lvm2 is present... 2024-05-27T07:29:25.916 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying time synchronization is in place... 2024-05-27T07:29:25.932 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T07:29:25.932 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T07:29:25.949 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T07:29:25.949 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-05-27T07:29:25.972 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-05-27T07:29:25.992 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-05-27T07:29:25.992 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-05-27T07:29:25.992 INFO:teuthology.orchestra.run.smithi090.stdout:Repeating the final host check... 2024-05-27T07:29:26.048 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 5.0.2 2024-05-27T07:29:26.048 INFO:teuthology.orchestra.run.smithi090.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-27T07:29:26.048 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl is present 2024-05-27T07:29:26.048 INFO:teuthology.orchestra.run.smithi090.stdout:lvcreate is present 2024-05-27T07:29:26.063 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T07:29:26.063 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T07:29:26.079 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T07:29:26.079 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-05-27T07:29:26.099 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-05-27T07:29:26.118 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-05-27T07:29:26.119 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-05-27T07:29:26.119 INFO:teuthology.orchestra.run.smithi090.stdout:Host looks OK 2024-05-27T07:29:26.119 INFO:teuthology.orchestra.run.smithi090.stdout:Cluster fsid: b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:29:26.120 INFO:teuthology.orchestra.run.smithi090.stdout:Acquiring lock 140303437143584 on /run/cephadm/b0214938-1bfa-11ef-bc9b-c7b262605968.lock 2024-05-27T07:29:26.120 INFO:teuthology.orchestra.run.smithi090.stdout:Lock 140303437143584 acquired on /run/cephadm/b0214938-1bfa-11ef-bc9b-c7b262605968.lock 2024-05-27T07:29:26.121 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 3300 ... 2024-05-27T07:29:26.122 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 6789 ... 2024-05-27T07:29:26.123 INFO:teuthology.orchestra.run.smithi090.stdout:Base mon IP(s) is [172.21.15.90:3300, 172.21.15.90:6789], mon addrv is [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-05-27T07:29:26.137 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.90 metric 100 2024-05-27T07:29:26.137 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.90 metric 100 2024-05-27T07:29:26.148 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-27T07:29:26.148 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-27T07:29:26.149 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-05-27T07:29:26.159 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-27T07:29:26.159 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-27T07:29:26.159 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T07:29:26.160 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-27T07:29:26.160 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-05-27T07:29:26.160 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T07:29:26.162 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-05-27T07:29:26.162 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-05-27T07:29:26.162 INFO:teuthology.orchestra.run.smithi090.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-27T07:29:26.165 INFO:teuthology.orchestra.run.smithi090.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-27T07:29:26.167 INFO:teuthology.orchestra.run.smithi090.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T07:29:27.055 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 2024-05-27T07:29:27.056 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6... 2024-05-27T07:29:27.056 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Getting image source signatures 2024-05-27T07:29:27.056 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:cf46015854c6cdc81d3273f73f8f001fe4a1edc6ed77e464da8c9d5e01e9b108 2024-05-27T07:29:27.056 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:6ac36e7c57f227110df5cd65cfdca1cc05e0b17df6748637a6aa04c5c80f89c6 2024-05-27T07:29:27.057 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying config sha256:e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 2024-05-27T07:29:27.057 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-27T07:29:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:ceph: stdout ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:29:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph version: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:29:28.351 INFO:teuthology.orchestra.run.smithi090.stdout:Extracting ceph user uid/gid from container image... 2024-05-27T07:29:29.441 INFO:teuthology.orchestra.run.smithi090.stdout:stat: stdout 167 167 2024-05-27T07:29:29.441 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial keys... 2024-05-27T07:29:30.590 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQBZNlRmd78OOhAA232aktw9rVQ3mtOtqUSIEw== 2024-05-27T07:29:31.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQBbNlRm3e3ZDRAALr+JjFKPEORn3Ic8tWTjwA== 2024-05-27T07:29:33.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQBcNlRmIz1OHRAA4FvlK8eDFGFmowvvFyrthw== 2024-05-27T07:29:33.028 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial monmap... 2024-05-27T07:29:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T07:29:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-27T07:29:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:29:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T07:29:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool for a [v2:172.21.15.90:3300,v1:172.21.15.90:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T07:29:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:setting min_mon_release = quincy 2024-05-27T07:29:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: set fsid to b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:29:34.179 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T07:29:34.179 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:29:34.180 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mon... 2024-05-27T07:29:46.938 INFO:teuthology.orchestra.run.smithi090.stdout:create mon.a on 2024-05-27T07:29:47.336 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-27T07:29:47.580 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-27T07:29:47.875 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b0214938-1bfa-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-b0214938-1bfa-11ef-bc9b-c7b262605968.target. 2024-05-27T07:29:47.876 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b0214938-1bfa-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-b0214938-1bfa-11ef-bc9b-c7b262605968.target. 2024-05-27T07:29:48.268 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.a 2024-05-27T07:29:48.268 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-05-27T07:29:48.553 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b0214938-1bfa-11ef-bc9b-c7b262605968.target.wants/ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@.service. 2024-05-27T07:29:49.204 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:48 smithi090 podman[31036]: 2024-05-27 07:29:48.957014393 +0000 UTC m=+0.134271170 container create 67fb3141fa833eec410efa3953ef165f71bcfd840cb76373a444468d82598dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-27T07:29:49.370 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T07:29:49.371 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-05-27T07:29:49.388 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T07:29:49.388 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-05-27T07:29:49.388 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-05-27T07:29:49.389 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T07:29:49.389 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon to start... 2024-05-27T07:29:49.389 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon... 2024-05-27T07:29:49.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:49 smithi090 podman[31036]: 2024-05-27 07:29:49.253012305 +0000 UTC m=+0.430269089 container init 67fb3141fa833eec410efa3953ef165f71bcfd840cb76373a444468d82598dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-05-27T07:29:49.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:49 smithi090 podman[31036]: 2024-05-27 07:29:49.263373671 +0000 UTC m=+0.440630455 container start 67fb3141fa833eec410efa3953ef165f71bcfd840cb76373a444468d82598dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T07:29:49.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:49 smithi090 bash[31036]: 67fb3141fa833eec410efa3953ef165f71bcfd840cb76373a444468d82598dec 2024-05-27T07:29:49.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:49 smithi090 systemd[1]: Started Ceph mon.a for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:29:49.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:49 smithi090 ceph-mon[31054]: mkfs b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:29:49.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:49 smithi090 ceph-mon[31054]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T07:29:50.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T07:29:50.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: monmap epoch 1 2024-05-27T07:29:50.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:29:50.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: last_changed 2024-05-27T07:29:33.566998+0000 2024-05-27T07:29:50.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:29:50.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: min_mon_release 19 (squid) 2024-05-27T07:29:50.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: election_strategy: 1 2024-05-27T07:29:50.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:29:50.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: fsmap 2024-05-27T07:29:50.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T07:29:50.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: mgrmap e1: no daemons active 2024-05-27T07:29:50.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:50 smithi090 ceph-mon[31054]: from='client.? 172.21.15.90:0/2106263351' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-27T07:29:50.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout cluster: 2024-05-27T07:29:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout id: b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:29:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-27T07:29:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:29:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout services: 2024-05-27T07:29:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.72298s) 2024-05-27T07:29:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-27T07:29:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-27T07:29:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:29:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout data: 2024-05-27T07:29:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pgs: 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.044+0000 7f80fc800700 1 Processor -- start 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.044+0000 7f80fc800700 1 -- start start 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.044+0000 7f80fc800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f410c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.044+0000 7f80fc800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f80f410c650 con 0x7f80f410bc50 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.046+0000 7f80f9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f410c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.046+0000 7f80f9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f410c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34520/0 (socket says 172.21.15.90:34520) 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.046+0000 7f80f9e00700 1 -- 172.21.15.90:0/2256740301 learned_addr learned my addr 172.21.15.90:0/2256740301 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:29:50.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.046+0000 7f80f9e00700 1 -- 172.21.15.90:0/2256740301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f410cdd0 con 0x7f80f410bc50 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.047+0000 7f80f9e00700 1 --2- 172.21.15.90:0/2256740301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f410c060 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f80e80098a0 tx=0x7f80e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb7c18315278ca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.047+0000 7f80f8a00700 1 -- 172.21.15.90:0/2256740301 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e8004160 con 0x7f80f410bc50 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.047+0000 7f80f8a00700 1 -- 172.21.15.90:0/2256740301 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e8004300 con 0x7f80f410bc50 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.047+0000 7f80f8a00700 1 -- 172.21.15.90:0/2256740301 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e80036a0 con 0x7f80f410bc50 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.048+0000 7f80fc800700 1 -- 172.21.15.90:0/2256740301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 msgr2=0x7f80f410c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.048+0000 7f80fc800700 1 --2- 172.21.15.90:0/2256740301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f410c060 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f80e80098a0 tx=0x7f80e8009bb0 comp rx=0 tx=0).stop 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.048+0000 7f80fc800700 1 -- 172.21.15.90:0/2256740301 shutdown_connections 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.048+0000 7f80fc800700 1 --2- 172.21.15.90:0/2256740301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f410c060 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.048+0000 7f80fc800700 1 -- 172.21.15.90:0/2256740301 >> 172.21.15.90:0/2256740301 conn(0x7f80f407f0e0 msgr2=0x7f80f4109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.049+0000 7f80fc800700 1 -- 172.21.15.90:0/2256740301 shutdown_connections 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.049+0000 7f80fc800700 1 -- 172.21.15.90:0/2256740301 wait complete. 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.050+0000 7f80fc800700 1 Processor -- start 2024-05-27T07:29:50.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.050+0000 7f80fc800700 1 -- start start 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.050+0000 7f80fc800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.050+0000 7f80fc800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f80f410d300 con 0x7f80f410bc50 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.051+0000 7f80f9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.051+0000 7f80f9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34526/0 (socket says 172.21.15.90:34526) 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.051+0000 7f80f9e00700 1 -- 172.21.15.90:0/2106263351 learned_addr learned my addr 172.21.15.90:0/2106263351 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.051+0000 7f80f9e00700 1 -- 172.21.15.90:0/2106263351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f419fc70 con 0x7f80f410bc50 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.051+0000 7f80f9e00700 1 --2- 172.21.15.90:0/2106263351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f419f730 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f80e8000c00 tx=0x7f80e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.052+0000 7f80eea00700 1 -- 172.21.15.90:0/2106263351 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e8003db0 con 0x7f80f410bc50 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.052+0000 7f80eea00700 1 -- 172.21.15.90:0/2106263351 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e8011630 con 0x7f80f410bc50 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.052+0000 7f80fc800700 1 -- 172.21.15.90:0/2106263351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f419ff00 con 0x7f80f410bc50 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.052+0000 7f80eea00700 1 -- 172.21.15.90:0/2106263351 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e8011950 con 0x7f80f410bc50 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.052+0000 7f80fc800700 1 -- 172.21.15.90:0/2106263351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f41a2b10 con 0x7f80f410bc50 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.052+0000 7f80eea00700 1 -- 172.21.15.90:0/2106263351 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80e8011af0 con 0x7f80f410bc50 2024-05-27T07:29:50.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.052+0000 7f80eea00700 1 -- 172.21.15.90:0/2106263351 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80e801c790 con 0x7f80f410bc50 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.053+0000 7f80fc800700 1 -- 172.21.15.90:0/2106263351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f4064e10 con 0x7f80f410bc50 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.057+0000 7f80eea00700 1 -- 172.21.15.90:0/2106263351 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f80e8032020 con 0x7f80f410bc50 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.136+0000 7f80fc800700 1 -- 172.21.15.90:0/2106263351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f80f402e8e0 con 0x7f80f410bc50 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.137+0000 7f80eea00700 1 -- 172.21.15.90:0/2106263351 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7f80e8004260 con 0x7f80f410bc50 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.138+0000 7f80fc800700 1 -- 172.21.15.90:0/2106263351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 msgr2=0x7f80f419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.138+0000 7f80fc800700 1 --2- 172.21.15.90:0/2106263351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f419f730 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f80e8000c00 tx=0x7f80e8004750 comp rx=0 tx=0).stop 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.138+0000 7f80fc800700 1 -- 172.21.15.90:0/2106263351 shutdown_connections 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.138+0000 7f80fc800700 1 --2- 172.21.15.90:0/2106263351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80f410bc50 0x7f80f419f730 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.138+0000 7f80fc800700 1 -- 172.21.15.90:0/2106263351 >> 172.21.15.90:0/2106263351 conn(0x7f80f407f0e0 msgr2=0x7f80f4195e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.138+0000 7f80fc800700 1 -- 172.21.15.90:0/2106263351 shutdown_connections 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:50.138+0000 7f80fc800700 1 -- 172.21.15.90:0/2106263351 wait complete. 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:mon is available 2024-05-27T07:29:50.776 INFO:teuthology.orchestra.run.smithi090.stdout:Assimilating anything we can from ceph.conf... 2024-05-27T07:29:51.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:51 smithi090 ceph-mon[31054]: from='client.? 172.21.15.90:0/4011768788' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T07:29:51.954 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:51 smithi090 ceph-mon[31054]: from='client.? 172.21.15.90:0/4011768788' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-27T07:29:52.363 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:29:52.364 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T07:29:52.364 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:29:52.364 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-27T07:29:52.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-05-27T07:29:52.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T07:29:52.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T07:29:52.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T07:29:52.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T07:29:52.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:29:52.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T07:29:52.365 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T07:29:52.368 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:29:52.368 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.424+0000 7f47f7000700 1 Processor -- start 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.424+0000 7f47f7000700 1 -- start start 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.425+0000 7f47f7000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f007d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.425+0000 7f47f7000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f47f007d780 con 0x7f47f007ee10 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.425+0000 7f47efe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f007d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:52.369 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.425+0000 7f47efe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f007d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34540/0 (socket says 172.21.15.90:34540) 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.426+0000 7f47efe00700 1 -- 172.21.15.90:0/267016933 learned_addr learned my addr 172.21.15.90:0/267016933 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.426+0000 7f47efe00700 1 -- 172.21.15.90:0/267016933 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f007d900 con 0x7f47f007ee10 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.426+0000 7f47efe00700 1 --2- 172.21.15.90:0/267016933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f007d240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f47e00098a0 tx=0x7f47e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76e77530d00c056 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.427+0000 7f47ef400700 1 -- 172.21.15.90:0/267016933 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47e0004160 con 0x7f47f007ee10 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.427+0000 7f47ef400700 1 -- 172.21.15.90:0/267016933 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e0004300 con 0x7f47f007ee10 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.427+0000 7f47ef400700 1 -- 172.21.15.90:0/267016933 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47e00036a0 con 0x7f47f007ee10 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.428+0000 7f47f7000700 1 -- 172.21.15.90:0/267016933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 msgr2=0x7f47f007d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.428+0000 7f47f7000700 1 --2- 172.21.15.90:0/267016933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f007d240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f47e00098a0 tx=0x7f47e0009bb0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.428+0000 7f47f7000700 1 -- 172.21.15.90:0/267016933 shutdown_connections 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.428+0000 7f47f7000700 1 --2- 172.21.15.90:0/267016933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f007d240 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.370 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.428+0000 7f47f7000700 1 -- 172.21.15.90:0/267016933 >> 172.21.15.90:0/267016933 conn(0x7f47f01050b0 msgr2=0x7f47f0107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:52.371 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.428+0000 7f47f7000700 1 -- 172.21.15.90:0/267016933 shutdown_connections 2024-05-27T07:29:52.371 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.428+0000 7f47f7000700 1 -- 172.21.15.90:0/267016933 wait complete. 2024-05-27T07:29:52.371 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.429+0000 7f47f7000700 1 Processor -- start 2024-05-27T07:29:52.371 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.429+0000 7f47f7000700 1 -- start start 2024-05-27T07:29:52.371 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.430+0000 7f47f7000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:52.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.430+0000 7f47f7000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f47f007de30 con 0x7f47f007ee10 2024-05-27T07:29:52.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.430+0000 7f47efe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:52.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.430+0000 7f47efe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34544/0 (socket says 172.21.15.90:34544) 2024-05-27T07:29:52.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.430+0000 7f47efe00700 1 -- 172.21.15.90:0/4011768788 learned_addr learned my addr 172.21.15.90:0/4011768788 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:29:52.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.431+0000 7f47efe00700 1 -- 172.21.15.90:0/4011768788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f019fce0 con 0x7f47f007ee10 2024-05-27T07:29:52.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.431+0000 7f47efe00700 1 --2- 172.21.15.90:0/4011768788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f019f7a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f47e0000c00 tx=0x7f47e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:52.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.431+0000 7f47ed600700 1 -- 172.21.15.90:0/4011768788 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47e0003db0 con 0x7f47f007ee10 2024-05-27T07:29:52.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.431+0000 7f47ed600700 1 -- 172.21.15.90:0/4011768788 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e0011630 con 0x7f47f007ee10 2024-05-27T07:29:52.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.431+0000 7f47ed600700 1 -- 172.21.15.90:0/4011768788 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47e0011950 con 0x7f47f007ee10 2024-05-27T07:29:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.431+0000 7f47f7000700 1 -- 172.21.15.90:0/4011768788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f019ff70 con 0x7f47f007ee10 2024-05-27T07:29:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.431+0000 7f47f7000700 1 -- 172.21.15.90:0/4011768788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f01a2b80 con 0x7f47f007ee10 2024-05-27T07:29:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.432+0000 7f47ed600700 1 -- 172.21.15.90:0/4011768788 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f47e0011af0 con 0x7f47f007ee10 2024-05-27T07:29:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.432+0000 7f47ed600700 1 -- 172.21.15.90:0/4011768788 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f47e001c450 con 0x7f47f007ee10 2024-05-27T07:29:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.432+0000 7f47f7000700 1 -- 172.21.15.90:0/4011768788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f0064e10 con 0x7f47f007ee10 2024-05-27T07:29:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.435+0000 7f47ed600700 1 -- 172.21.15.90:0/4011768788 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f47e0032020 con 0x7f47f007ee10 2024-05-27T07:29:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.512+0000 7f47f7000700 1 -- 172.21.15.90:0/4011768788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f47f002dc30 con 0x7f47f007ee10 2024-05-27T07:29:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.518+0000 7f47ed600700 1 -- 172.21.15.90:0/4011768788 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f47e0004260 con 0x7f47f007ee10 2024-05-27T07:29:52.377 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.520+0000 7f47ed600700 1 -- 172.21.15.90:0/4011768788 <== mon.0 v2:172.21.15.90:3300/0 8 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f47e001c6a0 con 0x7f47f007ee10 2024-05-27T07:29:52.378 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.520+0000 7f47f7000700 1 -- 172.21.15.90:0/4011768788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 msgr2=0x7f47f019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:52.378 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.520+0000 7f47f7000700 1 --2- 172.21.15.90:0/4011768788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f019f7a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f47e0000c00 tx=0x7f47e0004750 comp rx=0 tx=0).stop 2024-05-27T07:29:52.378 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.520+0000 7f47f7000700 1 -- 172.21.15.90:0/4011768788 shutdown_connections 2024-05-27T07:29:52.378 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.520+0000 7f47f7000700 1 --2- 172.21.15.90:0/4011768788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47f007ee10 0x7f47f019f7a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:52.378 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.520+0000 7f47f7000700 1 -- 172.21.15.90:0/4011768788 >> 172.21.15.90:0/4011768788 conn(0x7f47f01050b0 msgr2=0x7f47f0105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:52.378 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.520+0000 7f47f7000700 1 -- 172.21.15.90:0/4011768788 shutdown_connections 2024-05-27T07:29:52.378 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:51.520+0000 7f47f7000700 1 -- 172.21.15.90:0/4011768788 wait complete. 2024-05-27T07:29:52.378 INFO:teuthology.orchestra.run.smithi090.stdout:Generating new minimal ceph.conf... 2024-05-27T07:29:53.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:53 smithi090 ceph-mon[31054]: from='client.? 172.21.15.90:0/1167568838' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:29:53.904 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.084+0000 7f32bf800700 1 Processor -- start 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.085+0000 7f32bf800700 1 -- start start 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.085+0000 7f32bf800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b8112c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.085+0000 7f32bf800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8113200 con 0x7f32b81107f0 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.086+0000 7f32bce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b8112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.086+0000 7f32bce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b8112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34554/0 (socket says 172.21.15.90:34554) 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.086+0000 7f32bce00700 1 -- 172.21.15.90:0/1699383589 learned_addr learned my addr 172.21.15.90:0/1699383589 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.086+0000 7f32bce00700 1 -- 172.21.15.90:0/1699383589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b8113380 con 0x7f32b81107f0 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.087+0000 7f32bce00700 1 --2- 172.21.15.90:0/1699383589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b8112c10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0098a0 tx=0x7f32ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ea8868059b6bcd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.087+0000 7f32b7400700 1 -- 172.21.15.90:0/1699383589 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32ac004160 con 0x7f32b81107f0 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.087+0000 7f32b7400700 1 -- 172.21.15.90:0/1699383589 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f32ac004300 con 0x7f32b81107f0 2024-05-27T07:29:53.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.087+0000 7f32b7400700 1 -- 172.21.15.90:0/1699383589 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32ac0119e0 con 0x7f32b81107f0 2024-05-27T07:29:53.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.088+0000 7f32bf800700 1 -- 172.21.15.90:0/1699383589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 msgr2=0x7f32b8112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:53.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.088+0000 7f32bf800700 1 --2- 172.21.15.90:0/1699383589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b8112c10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0098a0 tx=0x7f32ac009bb0 comp rx=0 tx=0).stop 2024-05-27T07:29:53.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.088+0000 7f32bf800700 1 -- 172.21.15.90:0/1699383589 shutdown_connections 2024-05-27T07:29:53.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.088+0000 7f32bf800700 1 --2- 172.21.15.90:0/1699383589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b8112c10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:53.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.088+0000 7f32bf800700 1 -- 172.21.15.90:0/1699383589 >> 172.21.15.90:0/1699383589 conn(0x7f32b807f0b0 msgr2=0x7f32b8081540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:53.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.088+0000 7f32bf800700 1 -- 172.21.15.90:0/1699383589 shutdown_connections 2024-05-27T07:29:53.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.089+0000 7f32bf800700 1 -- 172.21.15.90:0/1699383589 wait complete. 2024-05-27T07:29:53.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.090+0000 7f32bf800700 1 Processor -- start 2024-05-27T07:29:53.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.090+0000 7f32bf800700 1 -- start start 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.090+0000 7f32bf800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b81a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.090+0000 7f32bf800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81143a0 con 0x7f32b81107f0 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.091+0000 7f32bce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b81a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.091+0000 7f32bce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b81a4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34562/0 (socket says 172.21.15.90:34562) 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.091+0000 7f32bce00700 1 -- 172.21.15.90:0/1167568838 learned_addr learned my addr 172.21.15.90:0/1167568838 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.091+0000 7f32bce00700 1 -- 172.21.15.90:0/1167568838 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b81a48a0 con 0x7f32b81107f0 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.091+0000 7f32bce00700 1 --2- 172.21.15.90:0/1167568838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b81a4360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0116a0 tx=0x7f32ac011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.092+0000 7f32b5600700 1 -- 172.21.15.90:0/1167568838 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32ac003970 con 0x7f32b81107f0 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.092+0000 7f32b5600700 1 -- 172.21.15.90:0/1167568838 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f32ac003b10 con 0x7f32b81107f0 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.092+0000 7f32b5600700 1 -- 172.21.15.90:0/1167568838 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32ac019550 con 0x7f32b81107f0 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.092+0000 7f32bf800700 1 -- 172.21.15.90:0/1167568838 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b81a4b30 con 0x7f32b81107f0 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.092+0000 7f32bf800700 1 -- 172.21.15.90:0/1167568838 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b81a7740 con 0x7f32b81107f0 2024-05-27T07:29:53.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.092+0000 7f32b5600700 1 -- 172.21.15.90:0/1167568838 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f32ac019b50 con 0x7f32b81107f0 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.092+0000 7f32b5600700 1 -- 172.21.15.90:0/1167568838 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f32ac00b720 con 0x7f32b81107f0 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.094+0000 7f32bf800700 1 -- 172.21.15.90:0/1167568838 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b8064e10 con 0x7f32b81107f0 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.096+0000 7f32b5600700 1 -- 172.21.15.90:0/1167568838 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f32ac01aa50 con 0x7f32b81107f0 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.172+0000 7f32bf800700 1 -- 172.21.15.90:0/1167568838 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f32b81a7960 con 0x7f32b81107f0 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.173+0000 7f32b5600700 1 -- 172.21.15.90:0/1167568838 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f32ac01a430 con 0x7f32b81107f0 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.175+0000 7f32bf800700 1 -- 172.21.15.90:0/1167568838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 msgr2=0x7f32b81a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.175+0000 7f32bf800700 1 --2- 172.21.15.90:0/1167568838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b81a4360 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0116a0 tx=0x7f32ac011a60 comp rx=0 tx=0).stop 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.175+0000 7f32bf800700 1 -- 172.21.15.90:0/1167568838 shutdown_connections 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.175+0000 7f32bf800700 1 --2- 172.21.15.90:0/1167568838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32b81107f0 0x7f32b81a4360 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.175+0000 7f32bf800700 1 -- 172.21.15.90:0/1167568838 >> 172.21.15.90:0/1167568838 conn(0x7f32b807f0b0 msgr2=0x7f32b807f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.175+0000 7f32bf800700 1 -- 172.21.15.90:0/1167568838 shutdown_connections 2024-05-27T07:29:53.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:53.175+0000 7f32bf800700 1 -- 172.21.15.90:0/1167568838 wait complete. 2024-05-27T07:29:53.913 INFO:teuthology.orchestra.run.smithi090.stdout:Restarting the monitor... 2024-05-27T07:29:54.204 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:53 smithi090 systemd[1]: Stopping Ceph mon.a for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:29:54.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:54 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a[31050]: 2024-05-27T07:29:54.372+0000 7faf65e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T07:29:54.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:54 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a[31050]: 2024-05-27T07:29:54.372+0000 7faf65e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-27T07:29:54.937 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:54 smithi090 podman[31304]: 2024-05-27 07:29:54.648999955 +0000 UTC m=+0.401529035 container died 67fb3141fa833eec410efa3953ef165f71bcfd840cb76373a444468d82598dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-27T07:29:55.204 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:55 smithi090 podman[31304]: 2024-05-27 07:29:55.094283087 +0000 UTC m=+0.846812160 container cleanup 67fb3141fa833eec410efa3953ef165f71bcfd840cb76373a444468d82598dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T07:29:55.204 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:55 smithi090 bash[31304]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a 2024-05-27T07:29:55.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:55 smithi090 podman[31315]: 2024-05-27 07:29:55.292270057 +0000 UTC m=+0.629857236 container remove 67fb3141fa833eec410efa3953ef165f71bcfd840cb76373a444468d82598dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:29:55.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:55 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-27T07:29:55.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:55 smithi090 systemd[1]: Stopped Ceph mon.a for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:29:55.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:55 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.a.service: Consumed 1.870s CPU time. 2024-05-27T07:29:55.954 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:55 smithi090 systemd[1]: Starting Ceph mon.a for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:29:56.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 podman[31411]: 2024-05-27 07:29:56.07665052 +0000 UTC m=+0.117491981 container create 217e3dbbb3134b31b0553cfe6ba5d74730b6c90e0cfd8fd03f76cb29fffb8223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T07:29:56.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 podman[31411]: 2024-05-27 07:29:55.99291442 +0000 UTC m=+0.033755859 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:29:56.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 podman[31411]: 2024-05-27 07:29:56.292322819 +0000 UTC m=+0.333164302 container init 217e3dbbb3134b31b0553cfe6ba5d74730b6c90e0cfd8fd03f76cb29fffb8223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240521, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:29:56.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 podman[31411]: 2024-05-27 07:29:56.302810416 +0000 UTC m=+0.343651898 container start 217e3dbbb3134b31b0553cfe6ba5d74730b6c90e0cfd8fd03f76cb29fffb8223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-27T07:29:56.389 INFO:teuthology.orchestra.run.smithi090.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-27T07:29:56.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 bash[31411]: 217e3dbbb3134b31b0553cfe6ba5d74730b6c90e0cfd8fd03f76cb29fffb8223 2024-05-27T07:29:56.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 systemd[1]: Started Ceph mon.a for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:29:56.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T07:29:56.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev), process ceph-mon, pid 2 2024-05-27T07:29:56.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: pidfile_write: ignore empty --pid-file 2024-05-27T07:29:56.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: load: jerasure load: lrc 2024-05-27T07:29:56.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: RocksDB version: 7.9.2 2024-05-27T07:29:56.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Git sha 0 2024-05-27T07:29:56.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Compile date 2024-05-21 07:00:12 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: DB SUMMARY 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: DB Session ID: WBOT8XFV4Z36TVLAB6BY 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: CURRENT file: CURRENT 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104572 ; 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.error_if_exists: 0 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.create_if_missing: 0 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.paranoid_checks: 1 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T07:29:56.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.env: 0x558c813a4500 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.info_log: 0x558c99bf2040 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.statistics: (nil) 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.use_fsync: 0 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_log_file_size: 0 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.allow_fallocate: 1 2024-05-27T07:29:56.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T07:29:56.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T07:29:56.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.use_direct_reads: 0 2024-05-27T07:29:56.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T07:29:56.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T07:29:56.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.db_log_dir: 2024-05-27T07:29:56.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.wal_dir: 2024-05-27T07:29:56.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T07:29:56.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.write_buffer_manager: 0x558c99abc8c0 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T07:29:56.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.unordered_write: 0 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.row_cache: None 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.wal_filter: None 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.two_write_queues: 0 2024-05-27T07:29:56.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.wal_compression: 0 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.atomic_flush: 0 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.log_readahead_size: 0 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T07:29:56.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_background_jobs: 2 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_background_compactions: -1 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_subcompactions: 1 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_open_files: -1 2024-05-27T07:29:56.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_background_flushes: -1 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Compression algorithms supported: 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: kZSTD supported: 0 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: kXpressCompression supported: 0 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: kZlibCompression supported: 1 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: kSnappyCompression supported: 1 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: kLZ4Compression supported: 1 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: kBZip2Compression supported: 0 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T07:29:56.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.merge_operator: 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_filter: None 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558c99bf2200) 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-05-27T07:29:56.686 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_type: 0 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_index_type: 0 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_shortening: 1 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: checksum: 4 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: no_block_cache: 0 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache: 0x558c99aef1f0 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_options: 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: capacity : 536870912 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_shard_bits : 4 2024-05-27T07:29:56.687 INFO:journalctl@ceph.mon.a.smithi090.stdout: strict_capacity_limit : 0 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_compressed: (nil) 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: persistent_cache: (nil) 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size: 4096 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size_deviation: 10 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_restart_interval: 16 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_block_restart_interval: 1 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: metadata_block_size: 4096 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: partition_filters: 0 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: use_delta_encoding: 1 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: filter_policy: bloomfilter 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: whole_key_filtering: 1 2024-05-27T07:29:56.688 INFO:journalctl@ceph.mon.a.smithi090.stdout: verify_compression: 0 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout: format_version: 5 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_index_compression: 1 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_align: 0 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout: max_auto_readahead_size: 262144 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout: prepopulate_block_cache: 0 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout: initial_auto_readahead_size: 8192 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compression: NoCompression 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T07:29:56.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.num_levels: 7 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:29:56.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T07:29:56.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T07:29:56.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T07:29:56.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.inplace_update_support: 0 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.bloom_locality: 0 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.max_successive_merges: 0 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T07:29:56.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.ttl: 2592000 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.enable_blob_files: false 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.min_blob_size: 0 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T07:29:56.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7a8b5dee-8d99-4350-a040-f81b050398b5 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794996435065, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794996437503, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99811, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97693, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14036, "raw_average_key_size": 49, "raw_value_size": 90031, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716794996, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7a8b5dee-8d99-4350-a040-f81b050398b5", "db_session_id": "WBOT8XFV4Z36TVLAB6BY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716794996437771, "job": 1, "event": "recovery_finished"} 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558c99ad3c00 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: DB pointer 0x558c9e8c6000 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T07:29:56.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** DB Stats ** 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: L0 2/0 99.33 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: Sum 2/0 99.33 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T07:29:56.697 INFO:journalctl@ceph.mon.a.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 63.4 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T07:29:56.698 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative compaction: 0.00 GB write, 13.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval compaction: 0.00 GB write, 13.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache BinnedLRUCache@0x558c99aef1f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: starting mon.a rank 0 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: mon.a@-1(???) e1 preinit fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: mon.a@-1(???).mds e1 new map 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: mon.a@-1(???).mds e1 print_map 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: e1 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:29:56.699 INFO:journalctl@ceph.mon.a.smithi090.stdout: legacy client fscid: -1 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout: No filesystems configured 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: monmap epoch 1 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: last_changed 2024-05-27T07:29:33.566998+0000 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: min_mon_release 19 (squid) 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: election_strategy: 1 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:29:56.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: fsmap 2024-05-27T07:29:56.701 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T07:29:56.701 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:56 smithi090 ceph-mon[31429]: mgrmap e1: no daemons active 2024-05-27T07:29:57.812 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.067+0000 7fdf5f000700 1 Processor -- start 2024-05-27T07:29:57.812 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.067+0000 7fdf5f000700 1 -- start start 2024-05-27T07:29:57.812 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.068+0000 7fdf5f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf58112c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:57.812 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.068+0000 7fdf5f000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58113200 con 0x7fdf581107f0 2024-05-27T07:29:57.812 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.068+0000 7fdf57e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf58112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:57.814 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.068+0000 7fdf57e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf58112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34572/0 (socket says 172.21.15.90:34572) 2024-05-27T07:29:57.814 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.068+0000 7fdf57e00700 1 -- 172.21.15.90:0/1809891059 learned_addr learned my addr 172.21.15.90:0/1809891059 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:29:57.814 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.069+0000 7fdf57e00700 1 -- 172.21.15.90:0/1809891059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf58113380 con 0x7fdf581107f0 2024-05-27T07:29:57.814 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.069+0000 7fdf57e00700 1 --2- 172.21.15.90:0/1809891059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf58112c10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf480098a0 tx=0x7fdf48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05404e3f6ccd7f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:57.814 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.070+0000 7fdf56a00700 1 -- 172.21.15.90:0/1809891059 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf48004160 con 0x7fdf581107f0 2024-05-27T07:29:57.814 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.070+0000 7fdf56a00700 1 -- 172.21.15.90:0/1809891059 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdf48004300 con 0x7fdf581107f0 2024-05-27T07:29:57.814 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.070+0000 7fdf56a00700 1 -- 172.21.15.90:0/1809891059 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf480119e0 con 0x7fdf581107f0 2024-05-27T07:29:57.814 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.070+0000 7fdf5f000700 1 -- 172.21.15.90:0/1809891059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 msgr2=0x7fdf58112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:57.814 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.070+0000 7fdf5f000700 1 --2- 172.21.15.90:0/1809891059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf58112c10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf480098a0 tx=0x7fdf48009bb0 comp rx=0 tx=0).stop 2024-05-27T07:29:57.814 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.071+0000 7fdf5f000700 1 -- 172.21.15.90:0/1809891059 shutdown_connections 2024-05-27T07:29:57.815 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.071+0000 7fdf5f000700 1 --2- 172.21.15.90:0/1809891059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf58112c10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:57.815 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.071+0000 7fdf5f000700 1 -- 172.21.15.90:0/1809891059 >> 172.21.15.90:0/1809891059 conn(0x7fdf5807f0b0 msgr2=0x7fdf58081540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:57.815 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.071+0000 7fdf5f000700 1 -- 172.21.15.90:0/1809891059 shutdown_connections 2024-05-27T07:29:57.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.071+0000 7fdf5f000700 1 -- 172.21.15.90:0/1809891059 wait complete. 2024-05-27T07:29:57.817 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.072+0000 7fdf5f000700 1 Processor -- start 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.072+0000 7fdf5f000700 1 -- start start 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.073+0000 7fdf5f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf581a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.073+0000 7fdf5f000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581143a0 con 0x7fdf581107f0 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.073+0000 7fdf57e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf581a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.073+0000 7fdf57e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf581a4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34580/0 (socket says 172.21.15.90:34580) 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.073+0000 7fdf57e00700 1 -- 172.21.15.90:0/3986973442 learned_addr learned my addr 172.21.15.90:0/3986973442 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.073+0000 7fdf57e00700 1 -- 172.21.15.90:0/3986973442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf581a48a0 con 0x7fdf581107f0 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.074+0000 7fdf57e00700 1 --2- 172.21.15.90:0/3986973442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf581a4360 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf480116a0 tx=0x7fdf48011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.074+0000 7fdf54c00700 1 -- 172.21.15.90:0/3986973442 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf48003970 con 0x7fdf581107f0 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.074+0000 7fdf5f000700 1 -- 172.21.15.90:0/3986973442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf581a4b30 con 0x7fdf581107f0 2024-05-27T07:29:57.818 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.074+0000 7fdf54c00700 1 -- 172.21.15.90:0/3986973442 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdf48003b10 con 0x7fdf581107f0 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.074+0000 7fdf5f000700 1 -- 172.21.15.90:0/3986973442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf581a7740 con 0x7fdf581107f0 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.074+0000 7fdf54c00700 1 -- 172.21.15.90:0/3986973442 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf48029060 con 0x7fdf581107f0 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.074+0000 7fdf54c00700 1 -- 172.21.15.90:0/3986973442 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdf480197e0 con 0x7fdf581107f0 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.074+0000 7fdf54c00700 1 -- 172.21.15.90:0/3986973442 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf4800b720 con 0x7fdf581107f0 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.075+0000 7fdf5f000700 1 -- 172.21.15.90:0/3986973442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf58064e10 con 0x7fdf581107f0 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.078+0000 7fdf54c00700 1 -- 172.21.15.90:0/3986973442 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fdf48024d50 con 0x7fdf581107f0 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.154+0000 7fdf5f000700 1 -- 172.21.15.90:0/3986973442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fdf581a7a30 con 0x7fdf581107f0 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.159+0000 7fdf54c00700 1 -- 172.21.15.90:0/3986973442 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fdf48019cb0 con 0x7fdf581107f0 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.160+0000 7fdf5f000700 1 -- 172.21.15.90:0/3986973442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 msgr2=0x7fdf581a4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.160+0000 7fdf5f000700 1 --2- 172.21.15.90:0/3986973442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf581a4360 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdf480116a0 tx=0x7fdf48011a60 comp rx=0 tx=0).stop 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.160+0000 7fdf5f000700 1 -- 172.21.15.90:0/3986973442 shutdown_connections 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.160+0000 7fdf5f000700 1 --2- 172.21.15.90:0/3986973442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf581107f0 0x7fdf581a4360 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.160+0000 7fdf5f000700 1 -- 172.21.15.90:0/3986973442 >> 172.21.15.90:0/3986973442 conn(0x7fdf5807f0b0 msgr2=0x7fdf5807f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:29:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.160+0000 7fdf5f000700 1 -- 172.21.15.90:0/3986973442 shutdown_connections 2024-05-27T07:29:57.820 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:29:57.160+0000 7fdf5f000700 1 -- 172.21.15.90:0/3986973442 wait complete. 2024-05-27T07:29:57.820 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-27T07:29:57.846 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:29:57.846 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mgr... 2024-05-27T07:29:57.848 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-27T07:29:57.849 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-27T07:29:58.302 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.x 2024-05-27T07:29:58.302 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-05-27T07:29:58.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:29:58 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3986973442' entity='client.admin' 2024-05-27T07:29:58.554 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b0214938-1bfa-11ef-bc9b-c7b262605968.target.wants/ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@.service. 2024-05-27T07:29:58.888 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:29:58 smithi090 systemd[1]: Starting Ceph mgr.x for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:29:59.145 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:29:58 smithi090 podman[31641]: 2024-05-27 07:29:58.933705774 +0000 UTC m=+0.109651939 container create da1e5558b99ab619c617cb673d704721834096a11216cde713f8d5f33ca1034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-27T07:29:59.145 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:29:58 smithi090 podman[31641]: 2024-05-27 07:29:58.855321467 +0000 UTC m=+0.031267606 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 2024-05-27T07:29:59.145 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:29:59 smithi090 podman[31641]: 2024-05-27 07:29:59.153676785 +0000 UTC m=+0.329622922 container init da1e5558b99ab619c617cb673d704721834096a11216cde713f8d5f33ca1034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T07:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T07:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-05-27T07:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T07:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-05-27T07:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-05-27T07:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T07:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T07:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-05-27T07:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T07:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-05-27T07:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-05-27T07:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-27T07:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr to start... 2024-05-27T07:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr... 2024-05-27T07:29:59.453 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:29:59 smithi090 podman[31641]: 2024-05-27 07:29:59.163491452 +0000 UTC m=+0.339437634 container start da1e5558b99ab619c617cb673d704721834096a11216cde713f8d5f33ca1034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5) 2024-05-27T07:29:59.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:29:59 smithi090 bash[31641]: da1e5558b99ab619c617cb673d704721834096a11216cde713f8d5f33ca1034d 2024-05-27T07:29:59.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:29:59 smithi090 systemd[1]: Started Ceph mgr.x for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:29:59.740 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:29:59 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:29:59.486+0000 7f36d85bc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:29:59.740 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:29:59 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:29:59.748+0000 7f36d85bc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:29:59.996 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:29:59 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:29:59.884+0000 7f36d85bc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:30:00.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:00 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3564623559' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T07:30:00.703 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:00.301+0000 7f36d85bc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:30:00.704 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:00.419+0000 7f36d85bc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:30:00.727 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-05-27T07:30:00.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T07:30:00.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T07:30:00.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T07:30:00.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T07:29:49.416565+0000", 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.005+0000 7fa293400700 1 Processor -- start 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.005+0000 7fa293400700 1 -- start start 2024-05-27T07:30:00.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.006+0000 7fa293400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c10c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.006+0000 7fa293400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c07ce40 con 0x7fa28c10a520 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.006+0000 7fa290a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c10c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.007+0000 7fa290a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c10c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49394/0 (socket says 172.21.15.90:49394) 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.007+0000 7fa290a00700 1 -- 172.21.15.90:0/347594118 learned_addr learned my addr 172.21.15.90:0/347594118 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.007+0000 7fa290a00700 1 -- 172.21.15.90:0/347594118 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c10ce80 con 0x7fa28c10a520 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.007+0000 7fa290a00700 1 --2- 172.21.15.90:0/347594118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c10c940 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2800098a0 tx=0x7fa280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e68a7cb583c46b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.008+0000 7fa28b400700 1 -- 172.21.15.90:0/347594118 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa280004160 con 0x7fa28c10a520 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.008+0000 7fa28b400700 1 -- 172.21.15.90:0/347594118 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa280004300 con 0x7fa28c10a520 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.008+0000 7fa28b400700 1 -- 172.21.15.90:0/347594118 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2800119e0 con 0x7fa28c10a520 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.009+0000 7fa293400700 1 -- 172.21.15.90:0/347594118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 msgr2=0x7fa28c10c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.009+0000 7fa293400700 1 --2- 172.21.15.90:0/347594118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c10c940 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa2800098a0 tx=0x7fa280009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:00.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.009+0000 7fa293400700 1 -- 172.21.15.90:0/347594118 shutdown_connections 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.009+0000 7fa293400700 1 --2- 172.21.15.90:0/347594118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c10c940 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.009+0000 7fa293400700 1 -- 172.21.15.90:0/347594118 >> 172.21.15.90:0/347594118 conn(0x7fa28c103f60 msgr2=0x7fa28c1063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.009+0000 7fa293400700 1 -- 172.21.15.90:0/347594118 shutdown_connections 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.009+0000 7fa293400700 1 -- 172.21.15.90:0/347594118 wait complete. 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.010+0000 7fa293400700 1 Processor -- start 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.010+0000 7fa293400700 1 -- start start 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.011+0000 7fa293400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.011+0000 7fa293400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c10dd80 con 0x7fa28c10a520 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.011+0000 7fa290a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.011+0000 7fa290a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c19dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49410/0 (socket says 172.21.15.90:49410) 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.011+0000 7fa290a00700 1 -- 172.21.15.90:0/3564623559 learned_addr learned my addr 172.21.15.90:0/3564623559 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.011+0000 7fa290a00700 1 -- 172.21.15.90:0/3564623559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c19e230 con 0x7fa28c10a520 2024-05-27T07:30:00.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.012+0000 7fa290a00700 1 --2- 172.21.15.90:0/3564623559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c19dcf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa280003f20 tx=0x7fa280011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.012+0000 7fa289600700 1 -- 172.21.15.90:0/3564623559 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2800038b0 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.012+0000 7fa289600700 1 -- 172.21.15.90:0/3564623559 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa280003a50 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.012+0000 7fa293400700 1 -- 172.21.15.90:0/3564623559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28c19e4c0 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.012+0000 7fa289600700 1 -- 172.21.15.90:0/3564623559 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa280019550 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.012+0000 7fa293400700 1 -- 172.21.15.90:0/3564623559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28c1a10d0 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.012+0000 7fa289600700 1 -- 172.21.15.90:0/3564623559 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa280019b90 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.012+0000 7fa289600700 1 -- 172.21.15.90:0/3564623559 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa280024730 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.013+0000 7fa293400700 1 -- 172.21.15.90:0/3564623559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28c064e10 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.016+0000 7fa289600700 1 -- 172.21.15.90:0/3564623559 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa280043ec0 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.095+0000 7fa293400700 1 -- 172.21.15.90:0/3564623559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa28c1a12a0 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.096+0000 7fa289600700 1 -- 172.21.15.90:0/3564623559 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa280032030 con 0x7fa28c10a520 2024-05-27T07:30:00.744 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.097+0000 7fa293400700 1 -- 172.21.15.90:0/3564623559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 msgr2=0x7fa28c19dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:00.745 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.097+0000 7fa293400700 1 --2- 172.21.15.90:0/3564623559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c19dcf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa280003f20 tx=0x7fa280011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:00.745 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.098+0000 7fa293400700 1 -- 172.21.15.90:0/3564623559 shutdown_connections 2024-05-27T07:30:00.745 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.098+0000 7fa293400700 1 --2- 172.21.15.90:0/3564623559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa28c10a520 0x7fa28c19dcf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:00.745 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.098+0000 7fa293400700 1 -- 172.21.15.90:0/3564623559 >> 172.21.15.90:0/3564623559 conn(0x7fa28c103f60 msgr2=0x7fa28c1047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:00.745 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.098+0000 7fa293400700 1 -- 172.21.15.90:0/3564623559 shutdown_connections 2024-05-27T07:30:00.745 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:00.098+0000 7fa293400700 1 -- 172.21.15.90:0/3564623559 wait complete. 2024-05-27T07:30:00.745 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (1/15)... 2024-05-27T07:30:02.448 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:02.015+0000 7f36d85bc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:30:02.448 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:02.242+0000 7f36d85bc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:30:02.448 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:02.457+0000 7f36d85bc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:30:03.165 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:02.841+0000 7f36d85bc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:30:03.166 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:02.967+0000 7f36d85bc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:30:03.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:03 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:03.230+0000 7f36d85bc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:30:03.804 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:03 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1857193420' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T07:30:03.804 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:03 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:03.503+0000 7f36d85bc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:30:04.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:30:04.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:30:04.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:30:04.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T07:30:04.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T07:30:04.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-27T07:30:04.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T07:30:04.094 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:04.094 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T07:30:04.094 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T07:30:04.094 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:04.094 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T07:30:04.094 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:04.094 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T07:30:04.095 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T07:30:04.095 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T07:30:04.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T07:30:04.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T07:30:04.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T07:30:04.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:04.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T07:30:04.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T07:30:04.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T07:30:04.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T07:30:04.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T07:30:04.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T07:30:04.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T07:30:04.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T07:30:04.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T07:30:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T07:30:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T07:30:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T07:30:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T07:30:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T07:30:04.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T07:30:04.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T07:30:04.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T07:30:04.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T07:30:04.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T07:30:04.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:04.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:30:04.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:04.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T07:30:04.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:04.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T07:29:49.416565+0000", 2024-05-27T07:30:04.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:30:04.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:04.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T07:30:04.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:30:04.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.397+0000 7f60fd800700 1 Processor -- start 2024-05-27T07:30:04.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.397+0000 7f60fd800700 1 -- start start 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.398+0000 7f60fd800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f807d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.398+0000 7f60fd800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f60f807d780 con 0x7f60f807ee10 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.398+0000 7f60f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f807d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.398+0000 7f60f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f807d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49418/0 (socket says 172.21.15.90:49418) 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.398+0000 7f60f7e00700 1 -- 172.21.15.90:0/3174032964 learned_addr learned my addr 172.21.15.90:0/3174032964 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.399+0000 7f60f7e00700 1 -- 172.21.15.90:0/3174032964 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f807d900 con 0x7f60f807ee10 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.399+0000 7f60f7e00700 1 --2- 172.21.15.90:0/3174032964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f807d240 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f60e80098a0 tx=0x7f60e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10df8f4e46f4eb27 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.399+0000 7f60f6a00700 1 -- 172.21.15.90:0/3174032964 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60e8004160 con 0x7f60f807ee10 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.400+0000 7f60f6a00700 1 -- 172.21.15.90:0/3174032964 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f60e8004300 con 0x7f60f807ee10 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.400+0000 7f60f6a00700 1 -- 172.21.15.90:0/3174032964 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60e80119e0 con 0x7f60f807ee10 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.400+0000 7f60fd800700 1 -- 172.21.15.90:0/3174032964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 msgr2=0x7f60f807d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:04.111 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.400+0000 7f60fd800700 1 --2- 172.21.15.90:0/3174032964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f807d240 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f60e80098a0 tx=0x7f60e8009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.401+0000 7f60fd800700 1 -- 172.21.15.90:0/3174032964 shutdown_connections 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.401+0000 7f60fd800700 1 --2- 172.21.15.90:0/3174032964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f807d240 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.401+0000 7f60fd800700 1 -- 172.21.15.90:0/3174032964 >> 172.21.15.90:0/3174032964 conn(0x7f60f8104f90 msgr2=0x7f60f81073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.401+0000 7f60fd800700 1 -- 172.21.15.90:0/3174032964 shutdown_connections 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.401+0000 7f60fd800700 1 -- 172.21.15.90:0/3174032964 wait complete. 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.402+0000 7f60fd800700 1 Processor -- start 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.402+0000 7f60fd800700 1 -- start start 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.402+0000 7f60fd800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.402+0000 7f60fd800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f60f810fe10 con 0x7f60f807ee10 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.403+0000 7f60f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.403+0000 7f60f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f819fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49432/0 (socket says 172.21.15.90:49432) 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.403+0000 7f60f7e00700 1 -- 172.21.15.90:0/1857193420 learned_addr learned my addr 172.21.15.90:0/1857193420 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:04.112 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.403+0000 7f60f7e00700 1 -- 172.21.15.90:0/1857193420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f81a0330 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.403+0000 7f60f7e00700 1 --2- 172.21.15.90:0/1857193420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f819fdf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60e80119b0 tx=0x7f60e8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.403+0000 7f60f4c00700 1 -- 172.21.15.90:0/1857193420 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60e8003af0 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.404+0000 7f60f4c00700 1 -- 172.21.15.90:0/1857193420 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f60e801a8a0 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.404+0000 7f60fd800700 1 -- 172.21.15.90:0/1857193420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f81a05c0 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.404+0000 7f60f4c00700 1 -- 172.21.15.90:0/1857193420 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60e8024830 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.404+0000 7f60fd800700 1 -- 172.21.15.90:0/1857193420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f81a31d0 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.404+0000 7f60f4c00700 1 -- 172.21.15.90:0/1857193420 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f60e8022020 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.404+0000 7f60f4c00700 1 -- 172.21.15.90:0/1857193420 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f60e80195e0 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.405+0000 7f60fd800700 1 -- 172.21.15.90:0/1857193420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f8064e10 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.408+0000 7f60f4c00700 1 -- 172.21.15.90:0/1857193420 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f60e801aa50 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.483+0000 7f60fd800700 1 -- 172.21.15.90:0/1857193420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f60f81a33a0 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.484+0000 7f60f4c00700 1 -- 172.21.15.90:0/1857193420 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f60e8032030 con 0x7f60f807ee10 2024-05-27T07:30:04.113 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.486+0000 7f60fd800700 1 -- 172.21.15.90:0/1857193420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 msgr2=0x7f60f819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:04.114 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.486+0000 7f60fd800700 1 --2- 172.21.15.90:0/1857193420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f819fdf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60e80119b0 tx=0x7f60e8011a90 comp rx=0 tx=0).stop 2024-05-27T07:30:04.114 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.486+0000 7f60fd800700 1 -- 172.21.15.90:0/1857193420 shutdown_connections 2024-05-27T07:30:04.114 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.486+0000 7f60fd800700 1 --2- 172.21.15.90:0/1857193420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60f807ee10 0x7f60f819fdf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:04.114 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.486+0000 7f60fd800700 1 -- 172.21.15.90:0/1857193420 >> 172.21.15.90:0/1857193420 conn(0x7f60f8104f90 msgr2=0x7f60f81068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:04.114 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.486+0000 7f60fd800700 1 -- 172.21.15.90:0/1857193420 shutdown_connections 2024-05-27T07:30:04.114 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:03.486+0000 7f60fd800700 1 -- 172.21.15.90:0/1857193420 wait complete. 2024-05-27T07:30:04.114 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (2/15)... 2024-05-27T07:30:04.453 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:04 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:04.330+0000 7f36d85bc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:30:04.953 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:04 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:04.619+0000 7f36d85bc200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:30:05.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:05 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:05.183+0000 7f36d85bc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:30:05.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:05 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:05.311+0000 7f36d85bc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:30:06.203 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:05 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:05.775+0000 7f36d85bc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:30:06.527 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:06 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:06.234+0000 7f36d85bc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:30:06.527 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:06 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:06.363+0000 7f36d85bc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:30:06.839 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:06 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:06.704+0000 7f36d85bc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:30:07.204 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:06 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3351724854' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T07:30:07.204 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:06 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:06.892+0000 7f36d85bc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:30:07.204 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:07.023+0000 7f36d85bc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:30:07.703 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:07.424+0000 7f36d85bc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:30:07.704 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:07.547+0000 7f36d85bc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:30:07.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:30:07.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:30:07.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:30:07.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T07:30:07.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T07:30:07.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T07:30:07.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T07:30:07.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:07.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T07:30:07.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T07:30:07.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-05-27T07:30:07.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:07.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T07:30:07.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T07:30:07.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:07.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-27T07:30:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T07:30:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T07:30:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T07:30:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T07:30:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T07:30:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T07:30:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T07:30:07.746 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T07:30:07.746 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T07:30:07.746 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T07:30:07.746 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:07.746 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T07:30:07.746 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T07:30:07.746 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T07:30:07.746 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T07:30:07.746 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T07:30:07.746 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T07:30:07.747 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T07:30:07.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T07:30:07.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T07:30:07.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T07:30:07.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:07.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:30:07.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T07:29:49.416565+0000", 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.757+0000 7f9d6f800700 1 Processor -- start 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.757+0000 7f9d6f800700 1 -- start start 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.757+0000 7f9d6f800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d68109e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.757+0000 7f9d6f800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6810a450 con 0x7f9d68109a50 2024-05-27T07:30:07.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.758+0000 7f9d6ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d68109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.758+0000 7f9d6ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d68109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49434/0 (socket says 172.21.15.90:49434) 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.758+0000 7f9d6ce00700 1 -- 172.21.15.90:0/1869391626 learned_addr learned my addr 172.21.15.90:0/1869391626 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.758+0000 7f9d6ce00700 1 -- 172.21.15.90:0/1869391626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6810ac20 con 0x7f9d68109a50 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.759+0000 7f9d6ce00700 1 --2- 172.21.15.90:0/1869391626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d68109e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0098a0 tx=0x7f9d5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3b969f874f83fe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.759+0000 7f9d67400700 1 -- 172.21.15.90:0/1869391626 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d5c004160 con 0x7f9d68109a50 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.759+0000 7f9d67400700 1 -- 172.21.15.90:0/1869391626 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9d5c004300 con 0x7f9d68109a50 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.759+0000 7f9d67400700 1 -- 172.21.15.90:0/1869391626 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d5c0119e0 con 0x7f9d68109a50 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.760+0000 7f9d6f800700 1 -- 172.21.15.90:0/1869391626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 msgr2=0x7f9d68109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.760+0000 7f9d6f800700 1 --2- 172.21.15.90:0/1869391626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d68109e60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0098a0 tx=0x7f9d5c009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.760+0000 7f9d6f800700 1 -- 172.21.15.90:0/1869391626 shutdown_connections 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.760+0000 7f9d6f800700 1 --2- 172.21.15.90:0/1869391626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d68109e60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:07.755 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.760+0000 7f9d6f800700 1 -- 172.21.15.90:0/1869391626 >> 172.21.15.90:0/1869391626 conn(0x7f9d681050b0 msgr2=0x7f9d68107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.760+0000 7f9d6f800700 1 -- 172.21.15.90:0/1869391626 shutdown_connections 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.760+0000 7f9d6f800700 1 -- 172.21.15.90:0/1869391626 wait complete. 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.761+0000 7f9d6f800700 1 Processor -- start 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.761+0000 7f9d6f800700 1 -- start start 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.762+0000 7f9d6f800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d6819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.762+0000 7f9d6f800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6810dfd0 con 0x7f9d68109a50 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.762+0000 7f9d6ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d6819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.762+0000 7f9d6ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d6819ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49446/0 (socket says 172.21.15.90:49446) 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.762+0000 7f9d6ce00700 1 -- 172.21.15.90:0/3351724854 learned_addr learned my addr 172.21.15.90:0/3351724854 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.762+0000 7f9d6ce00700 1 -- 172.21.15.90:0/3351724854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6819e330 con 0x7f9d68109a50 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.763+0000 7f9d6ce00700 1 --2- 172.21.15.90:0/3351724854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d6819ddf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0046e0 tx=0x7f9d5c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:07.756 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.763+0000 7f9d65600700 1 -- 172.21.15.90:0/3351724854 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d5c0039e0 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.763+0000 7f9d65600700 1 -- 172.21.15.90:0/3351724854 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f9d5c003b80 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.763+0000 7f9d6f800700 1 -- 172.21.15.90:0/3351724854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6819e5c0 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.763+0000 7f9d65600700 1 -- 172.21.15.90:0/3351724854 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d5c024700 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.763+0000 7f9d6f800700 1 -- 172.21.15.90:0/3351724854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d681a1190 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.763+0000 7f9d65600700 1 -- 172.21.15.90:0/3351724854 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9d5c022020 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.764+0000 7f9d65600700 1 -- 172.21.15.90:0/3351724854 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9d5c02dd40 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.764+0000 7f9d6f800700 1 -- 172.21.15.90:0/3351724854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d68064e10 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.767+0000 7f9d65600700 1 -- 172.21.15.90:0/3351724854 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9d5c01ab30 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.846+0000 7f9d6f800700 1 -- 172.21.15.90:0/3351724854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9d681a1360 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.847+0000 7f9d65600700 1 -- 172.21.15.90:0/3351724854 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f9d5c02d3f0 con 0x7f9d68109a50 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.848+0000 7f9d6f800700 1 -- 172.21.15.90:0/3351724854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 msgr2=0x7f9d6819ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.848+0000 7f9d6f800700 1 --2- 172.21.15.90:0/3351724854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d6819ddf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0046e0 tx=0x7f9d5c011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:07.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.848+0000 7f9d6f800700 1 -- 172.21.15.90:0/3351724854 shutdown_connections 2024-05-27T07:30:07.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.848+0000 7f9d6f800700 1 --2- 172.21.15.90:0/3351724854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d68109a50 0x7f9d6819ddf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:07.758 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.848+0000 7f9d6f800700 1 -- 172.21.15.90:0/3351724854 >> 172.21.15.90:0/3351724854 conn(0x7f9d681050b0 msgr2=0x7f9d68105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:07.759 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.849+0000 7f9d6f800700 1 -- 172.21.15.90:0/3351724854 shutdown_connections 2024-05-27T07:30:07.759 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:06.849+0000 7f9d6f800700 1 -- 172.21.15.90:0/3351724854 wait complete. 2024-05-27T07:30:07.759 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (3/15)... 2024-05-27T07:30:08.580 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:08 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:08.303+0000 7f36d85bc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:30:08.954 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:08 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:08.588+0000 7f36d85bc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:30:08.954 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: Activating manager daemon x 2024-05-27T07:30:08.954 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: mgrmap e2: x(active, starting, since 0.0118526s) 2024-05-27T07:30:08.954 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: from='mgr.14100 172.21.15.90:0/1981081788' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:30:08.954 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: from='mgr.14100 172.21.15.90:0/1981081788' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:30:08.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: from='mgr.14100 172.21.15.90:0/1981081788' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:30:08.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: from='mgr.14100 172.21.15.90:0/1981081788' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:30:08.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: from='mgr.14100 172.21.15.90:0/1981081788' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:30:08.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: Manager daemon x is now available 2024-05-27T07:30:08.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: from='mgr.14100 172.21.15.90:0/1981081788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:30:08.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: from='mgr.14100 172.21.15.90:0/1981081788' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:30:08.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:08 smithi090 ceph-mon[31429]: from='mgr.14100 172.21.15.90:0/1981081788' entity='mgr.x' 2024-05-27T07:30:09.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:09 smithi090 ceph-mon[31429]: from='mgr.14100 172.21.15.90:0/1981081788' entity='mgr.x' 2024-05-27T07:30:09.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:09 smithi090 ceph-mon[31429]: from='mgr.14100 172.21.15.90:0/1981081788' entity='mgr.x' 2024-05-27T07:30:09.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:09 smithi090 ceph-mon[31429]: mgrmap e3: x(active, since 1.02323s) 2024-05-27T07:30:11.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:30:11.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:30:11.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:30:11.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T07:30:11.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T07:30:11.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T07:30:11.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T07:30:11.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T07:30:11.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T07:30:11.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:11.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T07:29:49.416565+0000", 2024-05-27T07:30:11.457 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.456+0000 7fe003a00700 1 Processor -- start 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.456+0000 7fe003a00700 1 -- start start 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.456+0000 7fe003a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc07dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.456+0000 7fe003a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc07e010 con 0x7fdffc07f6a0 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.457+0000 7fe001000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc07dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.457+0000 7fe001000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc07dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38864/0 (socket says 172.21.15.90:38864) 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.457+0000 7fe001000700 1 -- 172.21.15.90:0/869315117 learned_addr learned my addr 172.21.15.90:0/869315117 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.457+0000 7fe001000700 1 -- 172.21.15.90:0/869315117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc07e190 con 0x7fdffc07f6a0 2024-05-27T07:30:11.458 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.458+0000 7fe001000700 1 --2- 172.21.15.90:0/869315117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc07dad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdff00098a0 tx=0x7fdff0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f072a524aba932d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.458+0000 7fdffb400700 1 -- 172.21.15.90:0/869315117 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff0004160 con 0x7fdffc07f6a0 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.458+0000 7fdffb400700 1 -- 172.21.15.90:0/869315117 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdff0004300 con 0x7fdffc07f6a0 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.458+0000 7fdffb400700 1 -- 172.21.15.90:0/869315117 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff00119e0 con 0x7fdffc07f6a0 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.459+0000 7fe003a00700 1 -- 172.21.15.90:0/869315117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 msgr2=0x7fdffc07dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.459+0000 7fe003a00700 1 --2- 172.21.15.90:0/869315117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc07dad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdff00098a0 tx=0x7fdff0009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.459+0000 7fe003a00700 1 -- 172.21.15.90:0/869315117 shutdown_connections 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.459+0000 7fe003a00700 1 --2- 172.21.15.90:0/869315117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc07dad0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.459+0000 7fe003a00700 1 -- 172.21.15.90:0/869315117 >> 172.21.15.90:0/869315117 conn(0x7fdffc105070 msgr2=0x7fdffc107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.459+0000 7fe003a00700 1 -- 172.21.15.90:0/869315117 shutdown_connections 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.459+0000 7fe003a00700 1 -- 172.21.15.90:0/869315117 wait complete. 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.460+0000 7fe003a00700 1 Processor -- start 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.460+0000 7fe003a00700 1 -- start start 2024-05-27T07:30:11.459 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.461+0000 7fe003a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc19ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.461+0000 7fe003a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc10ff90 con 0x7fdffc07f6a0 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.461+0000 7fe001000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc19ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.461+0000 7fe001000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc19ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38874/0 (socket says 172.21.15.90:38874) 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.461+0000 7fe001000700 1 -- 172.21.15.90:0/2725908605 learned_addr learned my addr 172.21.15.90:0/2725908605 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.461+0000 7fe001000700 1 -- 172.21.15.90:0/2725908605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc1a0490 con 0x7fdffc07f6a0 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.462+0000 7fe001000700 1 --2- 172.21.15.90:0/2725908605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc19ff50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdff00116a0 tx=0x7fdff0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.462+0000 7fdff9600700 1 -- 172.21.15.90:0/2725908605 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff0003970 con 0x7fdffc07f6a0 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.462+0000 7fdff9600700 1 -- 172.21.15.90:0/2725908605 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdff0003b10 con 0x7fdffc07f6a0 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.462+0000 7fdff9600700 1 -- 172.21.15.90:0/2725908605 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdff0019550 con 0x7fdffc07f6a0 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.462+0000 7fe003a00700 1 -- 172.21.15.90:0/2725908605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdffc1a0720 con 0x7fdffc07f6a0 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.462+0000 7fe003a00700 1 -- 172.21.15.90:0/2725908605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdffc1a3330 con 0x7fdffc07f6a0 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.463+0000 7fe003a00700 1 -- 172.21.15.90:0/2725908605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdffc064e10 con 0x7fdffc07f6a0 2024-05-27T07:30:11.460 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.463+0000 7fdff9600700 1 -- 172.21.15.90:0/2725908605 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7fdff00197c0 con 0x7fdffc07f6a0 2024-05-27T07:30:11.461 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.464+0000 7fdff9600700 1 --2- 172.21.15.90:0/2725908605 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7fdfe403e350 0x7fdfe4040860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:11.461 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.464+0000 7fdff9600700 1 -- 172.21.15.90:0/2725908605 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdff0053760 con 0x7fdffc07f6a0 2024-05-27T07:30:11.461 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.464+0000 7fdffbe00700 1 --2- 172.21.15.90:0/2725908605 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7fdfe403e350 0x7fdfe4040860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:11.461 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.465+0000 7fdffbe00700 1 --2- 172.21.15.90:0/2725908605 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7fdfe403e350 0x7fdfe4040860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8006fd0 tx=0x7fdfe8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:11.461 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.471+0000 7fdff9600700 1 -- 172.21.15.90:0/2725908605 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdff001a430 con 0x7fdffc07f6a0 2024-05-27T07:30:11.461 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.654+0000 7fdff9600700 1 -- 172.21.15.90:0/2725908605 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fdff001aa50 con 0x7fdffc07f6a0 2024-05-27T07:30:11.461 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.797+0000 7fe003a00700 1 -- 172.21.15.90:0/2725908605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fdffc1a3500 con 0x7fdffc07f6a0 2024-05-27T07:30:11.462 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.798+0000 7fdff9600700 1 -- 172.21.15.90:0/2725908605 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdff0051400 con 0x7fdffc07f6a0 2024-05-27T07:30:11.462 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.801+0000 7fe003a00700 1 -- 172.21.15.90:0/2725908605 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7fdfe403e350 msgr2=0x7fdfe4040860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:11.462 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.801+0000 7fe003a00700 1 --2- 172.21.15.90:0/2725908605 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7fdfe403e350 0x7fdfe4040860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8006fd0 tx=0x7fdfe8006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:11.462 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.801+0000 7fe003a00700 1 -- 172.21.15.90:0/2725908605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 msgr2=0x7fdffc19ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:11.462 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.801+0000 7fe003a00700 1 --2- 172.21.15.90:0/2725908605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc19ff50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdff00116a0 tx=0x7fdff0011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:11.462 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.801+0000 7fe003a00700 1 -- 172.21.15.90:0/2725908605 shutdown_connections 2024-05-27T07:30:11.462 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.801+0000 7fe003a00700 1 --2- 172.21.15.90:0/2725908605 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7fdfe403e350 0x7fdfe4040860 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:11.462 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.801+0000 7fe003a00700 1 --2- 172.21.15.90:0/2725908605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdffc07f6a0 0x7fdffc19ff50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:11.462 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.801+0000 7fe003a00700 1 -- 172.21.15.90:0/2725908605 >> 172.21.15.90:0/2725908605 conn(0x7fdffc105070 msgr2=0x7fdffc105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:11.462 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.801+0000 7fe003a00700 1 -- 172.21.15.90:0/2725908605 shutdown_connections 2024-05-27T07:30:11.463 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:10.801+0000 7fe003a00700 1 -- 172.21.15.90:0/2725908605 wait complete. 2024-05-27T07:30:11.463 INFO:teuthology.orchestra.run.smithi090.stdout:mgr is available 2024-05-27T07:30:11.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:11 smithi090 ceph-mon[31429]: mgrmap e4: x(active, since 2s) 2024-05-27T07:30:11.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:11 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2725908605' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T07:30:12.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:12 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/66829865' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T07:30:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:30:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T07:30:13.086 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:30:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-27T07:30:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T07:30:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T07:30:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T07:30:13.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T07:30:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:30:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T07:30:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T07:30:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-05-27T07:30:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T07:30:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T07:30:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T07:30:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T07:30:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T07:30:13.089 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T07:30:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.151+0000 7f260ec00700 1 Processor -- start 2024-05-27T07:30:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.151+0000 7f260ec00700 1 -- start start 2024-05-27T07:30:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.152+0000 7f260ec00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260810bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.152+0000 7f260ec00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f260810c490 con 0x7f260810ba90 2024-05-27T07:30:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.152+0000 7f260d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260810bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.152+0000 7f260d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260810bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38882/0 (socket says 172.21.15.90:38882) 2024-05-27T07:30:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.152+0000 7f260d800700 1 -- 172.21.15.90:0/3150896827 learned_addr learned my addr 172.21.15.90:0/3150896827 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.153+0000 7f260d800700 1 -- 172.21.15.90:0/3150896827 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260810cc60 con 0x7f260810ba90 2024-05-27T07:30:13.092 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.153+0000 7f260d800700 1 --2- 172.21.15.90:0/3150896827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260810bea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0098a0 tx=0x7f25fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ee526cd913fab7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.154+0000 7f2607e00700 1 -- 172.21.15.90:0/3150896827 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25fc004160 con 0x7f260810ba90 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.154+0000 7f2607e00700 1 -- 172.21.15.90:0/3150896827 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f25fc004300 con 0x7f260810ba90 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.154+0000 7f2607e00700 1 -- 172.21.15.90:0/3150896827 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25fc0119e0 con 0x7f260810ba90 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.154+0000 7f260ec00700 1 -- 172.21.15.90:0/3150896827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 msgr2=0x7f260810bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.154+0000 7f260ec00700 1 --2- 172.21.15.90:0/3150896827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260810bea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0098a0 tx=0x7f25fc009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.155+0000 7f260ec00700 1 -- 172.21.15.90:0/3150896827 shutdown_connections 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.155+0000 7f260ec00700 1 --2- 172.21.15.90:0/3150896827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260810bea0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.155+0000 7f260ec00700 1 -- 172.21.15.90:0/3150896827 >> 172.21.15.90:0/3150896827 conn(0x7f26081074c0 msgr2=0x7f26081098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.155+0000 7f260ec00700 1 -- 172.21.15.90:0/3150896827 shutdown_connections 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.155+0000 7f260ec00700 1 -- 172.21.15.90:0/3150896827 wait complete. 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.156+0000 7f260ec00700 1 Processor -- start 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.156+0000 7f260ec00700 1 -- start start 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.157+0000 7f260ec00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:13.093 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.157+0000 7f260ec00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2608110010 con 0x7f260810ba90 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.157+0000 7f260d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.157+0000 7f260d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260819fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38884/0 (socket says 172.21.15.90:38884) 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.157+0000 7f260d800700 1 -- 172.21.15.90:0/66829865 learned_addr learned my addr 172.21.15.90:0/66829865 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.157+0000 7f260d800700 1 -- 172.21.15.90:0/66829865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26081a0370 con 0x7f260810ba90 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.158+0000 7f260d800700 1 --2- 172.21.15.90:0/66829865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260819fe30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009fb0 tx=0x7f25fc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.158+0000 7f2606000700 1 -- 172.21.15.90:0/66829865 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25fc0038b0 con 0x7f260810ba90 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.158+0000 7f2606000700 1 -- 172.21.15.90:0/66829865 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f25fc003a50 con 0x7f260810ba90 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.158+0000 7f2606000700 1 -- 172.21.15.90:0/66829865 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25fc019550 con 0x7f260810ba90 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.158+0000 7f260ec00700 1 -- 172.21.15.90:0/66829865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26081a0600 con 0x7f260810ba90 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.158+0000 7f260ec00700 1 -- 172.21.15.90:0/66829865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26081a3210 con 0x7f260810ba90 2024-05-27T07:30:13.097 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.159+0000 7f260ec00700 1 -- 172.21.15.90:0/66829865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2608064e10 con 0x7f260810ba90 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.159+0000 7f2606000700 1 -- 172.21.15.90:0/66829865 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f25fc01a430 con 0x7f260810ba90 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.160+0000 7f2606000700 1 --2- 172.21.15.90:0/66829865 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f25f003e420 0x7f25f0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.160+0000 7f2606000700 1 -- 172.21.15.90:0/66829865 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f25fc053aa0 con 0x7f260810ba90 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.160+0000 7f260ce00700 1 --2- 172.21.15.90:0/66829865 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f25f003e420 0x7f25f0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.161+0000 7f260ce00700 1 --2- 172.21.15.90:0/66829865 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f25f003e420 0x7f25f0040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25f4006fd0 tx=0x7f25f4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.166+0000 7f2606000700 1 -- 172.21.15.90:0/66829865 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25fc0534c0 con 0x7f260810ba90 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.393+0000 7f260ec00700 1 -- 172.21.15.90:0/66829865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f26081a4a10 con 0x7f260810ba90 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.396+0000 7f2606000700 1 -- 172.21.15.90:0/66829865 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f25fc0196f0 con 0x7f260810ba90 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.398+0000 7f260ec00700 1 -- 172.21.15.90:0/66829865 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f25f003e420 msgr2=0x7f25f0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.398+0000 7f260ec00700 1 --2- 172.21.15.90:0/66829865 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f25f003e420 0x7f25f0040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25f4006fd0 tx=0x7f25f4006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.399+0000 7f260ec00700 1 -- 172.21.15.90:0/66829865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 msgr2=0x7f260819fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:13.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.399+0000 7f260ec00700 1 --2- 172.21.15.90:0/66829865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260819fe30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009fb0 tx=0x7f25fc011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.399+0000 7f260ec00700 1 -- 172.21.15.90:0/66829865 shutdown_connections 2024-05-27T07:30:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.399+0000 7f260ec00700 1 --2- 172.21.15.90:0/66829865 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f25f003e420 0x7f25f0040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.399+0000 7f260ec00700 1 --2- 172.21.15.90:0/66829865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f260810ba90 0x7f260819fe30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.399+0000 7f260ec00700 1 -- 172.21.15.90:0/66829865 >> 172.21.15.90:0/66829865 conn(0x7f26081074c0 msgr2=0x7f2608108d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.399+0000 7f260ec00700 1 -- 172.21.15.90:0/66829865 shutdown_connections 2024-05-27T07:30:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:12.399+0000 7f260ec00700 1 -- 172.21.15.90:0/66829865 wait complete. 2024-05-27T07:30:13.099 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling cephadm module... 2024-05-27T07:30:14.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:14 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3716559051' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-27T07:30:14.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: ignoring --setuser ceph since I am not root 2024-05-27T07:30:14.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: ignoring --setgroup ceph since I am not root 2024-05-27T07:30:14.734 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:14.471+0000 7fd1de4d1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:30:14.734 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:14.741+0000 7fd1de4d1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:30:14.764 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.737+0000 7f0296400700 1 Processor -- start 2024-05-27T07:30:14.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.737+0000 7f0296400700 1 -- start start 2024-05-27T07:30:14.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.737+0000 7f0296400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029010c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:14.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.737+0000 7f0296400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f029010c650 con 0x7f029010bc50 2024-05-27T07:30:14.765 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.738+0000 7f028f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029010c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:14.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.738+0000 7f028f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029010c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38892/0 (socket says 172.21.15.90:38892) 2024-05-27T07:30:14.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.738+0000 7f028f400700 1 -- 172.21.15.90:0/2381890069 learned_addr learned my addr 172.21.15.90:0/2381890069 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:14.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.738+0000 7f028f400700 1 -- 172.21.15.90:0/2381890069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029010cdd0 con 0x7f029010bc50 2024-05-27T07:30:14.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.739+0000 7f028f400700 1 --2- 172.21.15.90:0/2381890069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029010c060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f02800098a0 tx=0x7f0280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9af3381b42b260 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:14.766 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.739+0000 7f028ea00700 1 -- 172.21.15.90:0/2381890069 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0280004160 con 0x7f029010bc50 2024-05-27T07:30:14.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.739+0000 7f028ea00700 1 -- 172.21.15.90:0/2381890069 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0280004300 con 0x7f029010bc50 2024-05-27T07:30:14.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.739+0000 7f028ea00700 1 -- 172.21.15.90:0/2381890069 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02800119e0 con 0x7f029010bc50 2024-05-27T07:30:14.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.740+0000 7f0296400700 1 -- 172.21.15.90:0/2381890069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 msgr2=0x7f029010c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:14.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.740+0000 7f0296400700 1 --2- 172.21.15.90:0/2381890069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029010c060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f02800098a0 tx=0x7f0280009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:14.767 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.740+0000 7f0296400700 1 -- 172.21.15.90:0/2381890069 shutdown_connections 2024-05-27T07:30:14.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.740+0000 7f0296400700 1 --2- 172.21.15.90:0/2381890069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029010c060 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:14.768 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.740+0000 7f0296400700 1 -- 172.21.15.90:0/2381890069 >> 172.21.15.90:0/2381890069 conn(0x7f029007f0e0 msgr2=0x7f0290109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:14.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.740+0000 7f0296400700 1 -- 172.21.15.90:0/2381890069 shutdown_connections 2024-05-27T07:30:14.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.740+0000 7f0296400700 1 -- 172.21.15.90:0/2381890069 wait complete. 2024-05-27T07:30:14.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.741+0000 7f0296400700 1 Processor -- start 2024-05-27T07:30:14.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.742+0000 7f0296400700 1 -- start start 2024-05-27T07:30:14.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.742+0000 7f0296400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:14.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.742+0000 7f0296400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0290110180 con 0x7f029010bc50 2024-05-27T07:30:14.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.742+0000 7f028f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:14.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.742+0000 7f028f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029019ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38898/0 (socket says 172.21.15.90:38898) 2024-05-27T07:30:14.769 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.742+0000 7f028f400700 1 -- 172.21.15.90:0/3716559051 learned_addr learned my addr 172.21.15.90:0/3716559051 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:14.771 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.743+0000 7f028f400700 1 -- 172.21.15.90:0/3716559051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02901a04e0 con 0x7f029010bc50 2024-05-27T07:30:14.771 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.743+0000 7f028f400700 1 --2- 172.21.15.90:0/3716559051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029019ffa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0280009fb0 tx=0x7f0280011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:14.771 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.743+0000 7f028cc00700 1 -- 172.21.15.90:0/3716559051 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02800038b0 con 0x7f029010bc50 2024-05-27T07:30:14.771 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.743+0000 7f028cc00700 1 -- 172.21.15.90:0/3716559051 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f0280003a50 con 0x7f029010bc50 2024-05-27T07:30:14.771 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.743+0000 7f028cc00700 1 -- 172.21.15.90:0/3716559051 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028001a5c0 con 0x7f029010bc50 2024-05-27T07:30:14.771 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.743+0000 7f0296400700 1 -- 172.21.15.90:0/3716559051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02901a0770 con 0x7f029010bc50 2024-05-27T07:30:14.771 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.743+0000 7f0296400700 1 -- 172.21.15.90:0/3716559051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02901a3380 con 0x7f029010bc50 2024-05-27T07:30:14.771 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.745+0000 7f0296400700 1 -- 172.21.15.90:0/3716559051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0290064e10 con 0x7f029010bc50 2024-05-27T07:30:14.771 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.745+0000 7f028cc00700 1 -- 172.21.15.90:0/3716559051 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f0280024a60 con 0x7f029010bc50 2024-05-27T07:30:14.771 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.745+0000 7f028cc00700 1 --2- 172.21.15.90:0/3716559051 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f027403e3d0 0x7f02740408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:14.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.745+0000 7f028cc00700 1 -- 172.21.15.90:0/3716559051 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0280053b60 con 0x7f029010bc50 2024-05-27T07:30:14.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.746+0000 7f0286a00700 1 --2- 172.21.15.90:0/3716559051 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f027403e3d0 0x7f02740408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:14.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.746+0000 7f0286a00700 1 --2- 172.21.15.90:0/3716559051 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f027403e3d0 0x7f02740408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f027800ad30 tx=0x7f02780093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:14.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:13.751+0000 7f028cc00700 1 -- 172.21.15.90:0/3716559051 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f028002c350 con 0x7f029010bc50 2024-05-27T07:30:14.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.037+0000 7f0296400700 1 -- 172.21.15.90:0/3716559051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f02901a3690 con 0x7f029010bc50 2024-05-27T07:30:14.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.091+0000 7f028cc00700 1 -- 172.21.15.90:0/3716559051 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f0280053500 con 0x7f029010bc50 2024-05-27T07:30:14.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.095+0000 7f0296400700 1 -- 172.21.15.90:0/3716559051 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f027403e3d0 msgr2=0x7f02740408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:14.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.095+0000 7f0296400700 1 --2- 172.21.15.90:0/3716559051 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f027403e3d0 0x7f02740408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f027800ad30 tx=0x7f02780093f0 comp rx=0 tx=0).stop 2024-05-27T07:30:14.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.095+0000 7f0296400700 1 -- 172.21.15.90:0/3716559051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 msgr2=0x7f029019ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:14.773 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.096+0000 7f0296400700 1 --2- 172.21.15.90:0/3716559051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029019ffa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0280009fb0 tx=0x7f0280011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:14.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.096+0000 7f0296400700 1 -- 172.21.15.90:0/3716559051 shutdown_connections 2024-05-27T07:30:14.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.096+0000 7f0296400700 1 --2- 172.21.15.90:0/3716559051 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f027403e3d0 0x7f02740408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:14.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.096+0000 7f0296400700 1 --2- 172.21.15.90:0/3716559051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f029010bc50 0x7f029019ffa0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:14.774 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.096+0000 7f0296400700 1 -- 172.21.15.90:0/3716559051 >> 172.21.15.90:0/3716559051 conn(0x7f029007f0e0 msgr2=0x7f02901091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:14.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.097+0000 7f0296400700 1 -- 172.21.15.90:0/3716559051 shutdown_connections 2024-05-27T07:30:14.775 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:14.097+0000 7f0296400700 1 -- 172.21.15.90:0/3716559051 wait complete. 2024-05-27T07:30:15.091 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:14.884+0000 7fd1de4d1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:30:15.408 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:15.292+0000 7fd1de4d1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:30:15.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:15 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3716559051' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-27T07:30:15.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:15 smithi090 ceph-mon[31429]: mgrmap e5: x(active, since 5s) 2024-05-27T07:30:15.664 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:15.415+0000 7fd1de4d1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:30:16.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:16 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2781024371' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T07:30:16.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:30:16.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-27T07:30:16.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T07:30:16.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-27T07:30:16.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T07:30:16.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:30:16.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.463+0000 7f24ffe00700 1 Processor -- start 2024-05-27T07:30:16.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.463+0000 7f24ffe00700 1 -- start start 2024-05-27T07:30:16.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.464+0000 7f24ffe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250010c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.464+0000 7f24ffe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f250007ce80 con 0x7f250010a350 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.464+0000 7f24fea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250010c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.464+0000 7f24fea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250010c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38936/0 (socket says 172.21.15.90:38936) 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.464+0000 7f24fea00700 1 -- 172.21.15.90:0/4192553887 learned_addr learned my addr 172.21.15.90:0/4192553887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.465+0000 7f24fea00700 1 -- 172.21.15.90:0/4192553887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250010ccb0 con 0x7f250010a350 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.465+0000 7f24fea00700 1 --2- 172.21.15.90:0/4192553887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250010c770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f24f00098a0 tx=0x7f24f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ee5832b60ae70 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.465+0000 7f24fd600700 1 -- 172.21.15.90:0/4192553887 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f0004160 con 0x7f250010a350 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.465+0000 7f24fd600700 1 -- 172.21.15.90:0/4192553887 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f24f0004300 con 0x7f250010a350 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.465+0000 7f24fd600700 1 -- 172.21.15.90:0/4192553887 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f00119e0 con 0x7f250010a350 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.466+0000 7f24ffe00700 1 -- 172.21.15.90:0/4192553887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 msgr2=0x7f250010c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:16.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.466+0000 7f24ffe00700 1 --2- 172.21.15.90:0/4192553887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250010c770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f24f00098a0 tx=0x7f24f0009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.466+0000 7f24ffe00700 1 -- 172.21.15.90:0/4192553887 shutdown_connections 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.466+0000 7f24ffe00700 1 --2- 172.21.15.90:0/4192553887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250010c770 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.466+0000 7f24ffe00700 1 -- 172.21.15.90:0/4192553887 >> 172.21.15.90:0/4192553887 conn(0x7f2500103d90 msgr2=0x7f2500106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.466+0000 7f24ffe00700 1 -- 172.21.15.90:0/4192553887 shutdown_connections 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.466+0000 7f24ffe00700 1 -- 172.21.15.90:0/4192553887 wait complete. 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.467+0000 7f24ffe00700 1 Processor -- start 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.468+0000 7f24ffe00700 1 -- start start 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.468+0000 7f24ffe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.468+0000 7f24ffe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f250010dc10 con 0x7f250010a350 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.468+0000 7f24fea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.468+0000 7f24fea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38948/0 (socket says 172.21.15.90:38948) 2024-05-27T07:30:16.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.468+0000 7f24fea00700 1 -- 172.21.15.90:0/2781024371 learned_addr learned my addr 172.21.15.90:0/2781024371 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.468+0000 7f24fea00700 1 -- 172.21.15.90:0/2781024371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250019e0f0 con 0x7f250010a350 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.469+0000 7f24fea00700 1 --2- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250019dbb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24f00119b0 tx=0x7f24f0011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.469+0000 7f24f7400700 1 -- 172.21.15.90:0/2781024371 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f0003af0 con 0x7f250010a350 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.469+0000 7f24f7400700 1 -- 172.21.15.90:0/2781024371 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f24f001a8a0 con 0x7f250010a350 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.469+0000 7f24ffe00700 1 -- 172.21.15.90:0/2781024371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250019e380 con 0x7f250010a350 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.469+0000 7f24f7400700 1 -- 172.21.15.90:0/2781024371 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24f0024830 con 0x7f250010a350 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.469+0000 7f24ffe00700 1 -- 172.21.15.90:0/2781024371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25001a0f90 con 0x7f250010a350 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.470+0000 7f24ffe00700 1 -- 172.21.15.90:0/2781024371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2500064e10 con 0x7f250010a350 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.471+0000 7f24f7400700 1 -- 172.21.15.90:0/2781024371 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f24f0022020 con 0x7f250010a350 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.471+0000 7f24f7400700 1 --2- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f24e403e4c0 0x7f24e40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:16.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.471+0000 7f24f7400700 1 -- 172.21.15.90:0/2781024371 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f24f0053690 con 0x7f250010a350 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.471+0000 7f24fe000700 1 -- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f24e403e4c0 msgr2=0x7f24e40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/410158065 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.471+0000 7f24fe000700 1 --2- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f24e403e4c0 0x7f24e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.477+0000 7f24f7400700 1 -- 172.21.15.90:0/2781024371 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f24f00533f0 con 0x7f250010a350 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.672+0000 7f24fe000700 1 -- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f24e403e4c0 msgr2=0x7f24e40409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/410158065 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.672+0000 7f24fe000700 1 --2- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f24e403e4c0 0x7f24e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.768+0000 7f24ffe00700 1 -- 172.21.15.90:0/2781024371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f25001a1610 con 0x7f250010a350 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.769+0000 7f24f7400700 1 -- 172.21.15.90:0/2781024371 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f24f001aae0 con 0x7f250010a350 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.773+0000 7f24ffe00700 1 -- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f24e403e4c0 msgr2=0x7f24e40409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.773+0000 7f24ffe00700 1 --2- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f24e403e4c0 0x7f24e40409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.773+0000 7f24ffe00700 1 -- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 msgr2=0x7f250019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.773+0000 7f24ffe00700 1 --2- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250019dbb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f24f00119b0 tx=0x7f24f0011a90 comp rx=0 tx=0).stop 2024-05-27T07:30:16.408 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.773+0000 7f24ffe00700 1 -- 172.21.15.90:0/2781024371 shutdown_connections 2024-05-27T07:30:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.773+0000 7f24ffe00700 1 --2- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7f24e403e4c0 0x7f24e40409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.773+0000 7f24ffe00700 1 --2- 172.21.15.90:0/2781024371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f250010a350 0x7f250019dbb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.773+0000 7f24ffe00700 1 -- 172.21.15.90:0/2781024371 >> 172.21.15.90:0/2781024371 conn(0x7f2500103d90 msgr2=0x7f25001046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.773+0000 7f24ffe00700 1 -- 172.21.15.90:0/2781024371 shutdown_connections 2024-05-27T07:30:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:15.773+0000 7f24ffe00700 1 -- 172.21.15.90:0/2781024371 wait complete. 2024-05-27T07:30:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-05-27T07:30:16.409 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 5... 2024-05-27T07:30:17.240 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:16.994+0000 7fd1de4d1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:30:17.241 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:17.224+0000 7fd1de4d1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:30:17.641 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:17.440+0000 7fd1de4d1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:30:17.935 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:17.820+0000 7fd1de4d1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:30:17.936 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:17.942+0000 7fd1de4d1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:30:18.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:18.211+0000 7fd1de4d1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:30:18.889 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:18.471+0000 7fd1de4d1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:30:19.568 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:19.283+0000 7fd1de4d1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:30:19.568 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:19.575+0000 7fd1de4d1200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:30:20.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:20.134+0000 7fd1de4d1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:30:20.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:20.260+0000 7fd1de4d1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:30:21.181 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:20.733+0000 7fd1de4d1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:30:21.181 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:21.188+0000 7fd1de4d1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:30:21.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:21.314+0000 7fd1de4d1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:30:21.944 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:21.641+0000 7fd1de4d1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:30:21.945 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:21.821+0000 7fd1de4d1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:30:21.945 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:21.951+0000 7fd1de4d1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:30:22.704 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:22 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:22.370+0000 7fd1de4d1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:30:22.704 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:22 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:22.493+0000 7fd1de4d1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:30:23.536 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:23 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:23.253+0000 7fd1de4d1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:30:23.906 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:23 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:23.543+0000 7fd1de4d1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:30:23.906 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:23 smithi090 ceph-mon[31429]: Active manager daemon x restarted 2024-05-27T07:30:23.906 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:23 smithi090 ceph-mon[31429]: Activating manager daemon x 2024-05-27T07:30:23.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:23 smithi090 ceph-mon[31429]: osdmap e2: 0 total, 0 up, 0 in 2024-05-27T07:30:23.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:23 smithi090 ceph-mon[31429]: mgrmap e6: x(active, starting, since 0.0103203s) 2024-05-27T07:30:23.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:23 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:30:23.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:23 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:30:23.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:23 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:30:23.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:23 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:30:23.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:23 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:30:23.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:23 smithi090 ceph-mon[31429]: Manager daemon x is now available 2024-05-27T07:30:24.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:24 smithi090 ceph-mon[31429]: Found migration_current of "None". Setting to last migration. 2024-05-27T07:30:24.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:24 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:24.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:24 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:24.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:24 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:30:24.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:24 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:30:24.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:24 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:30:24.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:24 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:30:24.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:24 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:24.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:24 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:24.942 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:24 smithi090 ceph-mon[31429]: mgrmap e7: x(active, since 1.0161s) 2024-05-27T07:30:25.339 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.038+0000 7ff48ee00700 1 Processor -- start 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.038+0000 7ff48ee00700 1 -- start start 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.039+0000 7ff48ee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48810c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.039+0000 7ff48ee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff48807ce80 con 0x7ff48810a500 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.039+0000 7ff487e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48810c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.039+0000 7ff487e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48810c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38962/0 (socket says 172.21.15.90:38962) 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.039+0000 7ff487e00700 1 -- 172.21.15.90:0/236869672 learned_addr learned my addr 172.21.15.90:0/236869672 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.040+0000 7ff487e00700 1 -- 172.21.15.90:0/236869672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48810ce60 con 0x7ff48810a500 2024-05-27T07:30:25.347 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.040+0000 7ff487e00700 1 --2- 172.21.15.90:0/236869672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48810c920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4780098a0 tx=0x7ff478009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c52f25d22014cd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.040+0000 7ff486a00700 1 -- 172.21.15.90:0/236869672 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff478004160 con 0x7ff48810a500 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.040+0000 7ff486a00700 1 -- 172.21.15.90:0/236869672 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff478004300 con 0x7ff48810a500 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.040+0000 7ff486a00700 1 -- 172.21.15.90:0/236869672 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4780119e0 con 0x7ff48810a500 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.041+0000 7ff48ee00700 1 -- 172.21.15.90:0/236869672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 msgr2=0x7ff48810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.041+0000 7ff48ee00700 1 --2- 172.21.15.90:0/236869672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48810c920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4780098a0 tx=0x7ff478009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.041+0000 7ff48ee00700 1 -- 172.21.15.90:0/236869672 shutdown_connections 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.041+0000 7ff48ee00700 1 --2- 172.21.15.90:0/236869672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48810c920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.041+0000 7ff48ee00700 1 -- 172.21.15.90:0/236869672 >> 172.21.15.90:0/236869672 conn(0x7ff488103fa0 msgr2=0x7ff488106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.041+0000 7ff48ee00700 1 -- 172.21.15.90:0/236869672 shutdown_connections 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.042+0000 7ff48ee00700 1 -- 172.21.15.90:0/236869672 wait complete. 2024-05-27T07:30:25.348 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.043+0000 7ff48ee00700 1 Processor -- start 2024-05-27T07:30:25.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.043+0000 7ff48ee00700 1 -- start start 2024-05-27T07:30:25.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.043+0000 7ff48ee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48819ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:25.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.043+0000 7ff48ee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff48810ddc0 con 0x7ff48810a500 2024-05-27T07:30:25.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.043+0000 7ff487e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48819ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:25.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.043+0000 7ff487e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48819ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38976/0 (socket says 172.21.15.90:38976) 2024-05-27T07:30:25.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.043+0000 7ff487e00700 1 -- 172.21.15.90:0/63015642 learned_addr learned my addr 172.21.15.90:0/63015642 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:25.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.044+0000 7ff487e00700 1 -- 172.21.15.90:0/63015642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4881a0480 con 0x7ff48810a500 2024-05-27T07:30:25.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.044+0000 7ff487e00700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48819ff40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff4780116a0 tx=0x7ff478011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:25.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.044+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff478003970 con 0x7ff48810a500 2024-05-27T07:30:25.349 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.044+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7ff478003b10 con 0x7ff48810a500 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.044+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff478019550 con 0x7ff48810a500 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.044+0000 7ff48ee00700 1 -- 172.21.15.90:0/63015642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4881a0710 con 0x7ff48810a500 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.045+0000 7ff48ee00700 1 -- 172.21.15.90:0/63015642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4881a3320 con 0x7ff48810a500 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.046+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7ff4780197c0 con 0x7ff48810a500 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.046+0000 7ff484c00700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 0x7ff46c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.046+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 --> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff46c0410e0 con 0x7ff46c03e4c0 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.046+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff478053990 con 0x7ff48810a500 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.047+0000 7ff487400700 1 -- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 msgr2=0x7ff46c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/410158065 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.047+0000 7ff487400700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 0x7ff46c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.248+0000 7ff487400700 1 -- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 msgr2=0x7ff46c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/410158065 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.248+0000 7ff487400700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 0x7ff46c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:30:25.350 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.649+0000 7ff487400700 1 -- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 msgr2=0x7ff46c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/410158065 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:17.649+0000 7ff487400700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 0x7ff46c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:18.450+0000 7ff487400700 1 -- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 msgr2=0x7ff46c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/410158065 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:18.450+0000 7ff487400700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 0x7ff46c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:20.052+0000 7ff487400700 1 -- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 msgr2=0x7ff46c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/410158065 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:20.052+0000 7ff487400700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 0x7ff46c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:23.255+0000 7ff487400700 1 -- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 msgr2=0x7ff46c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/410158065 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:23.255+0000 7ff487400700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 0x7ff46c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:23.554+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7ff478010d60 con 0x7ff48810a500 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:23.554+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 msgr2=0x7ff46c0409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:23.554+0000 7ff484c00700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 0x7ff46c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.558+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7ff478054cc0 con 0x7ff48810a500 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.558+0000 7ff484c00700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7ff46c041e00 0x7ff46c044220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:25.351 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.558+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 --> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff4780068a0 con 0x7ff46c041e00 2024-05-27T07:30:25.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.562+0000 7ff487400700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7ff46c041e00 0x7ff46c044220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:25.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.564+0000 7ff487400700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7ff46c041e00 0x7ff46c044220 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff474003a10 tx=0x7ff4740092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:25.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.565+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 <== mgr.14120 v2:172.21.15.90:6800/2742579272 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff4780068a0 con 0x7ff46c041e00 2024-05-27T07:30:25.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.581+0000 7ff48ee00700 1 -- 172.21.15.90:0/63015642 --> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff488064e10 con 0x7ff46c041e00 2024-05-27T07:30:25.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.582+0000 7ff484c00700 1 -- 172.21.15.90:0/63015642 <== mgr.14120 v2:172.21.15.90:6800/2742579272 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff488064e10 con 0x7ff46c041e00 2024-05-27T07:30:25.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.582+0000 7ff48ee00700 1 -- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7ff46c041e00 msgr2=0x7ff46c044220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:25.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.582+0000 7ff48ee00700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7ff46c041e00 0x7ff46c044220 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff474003a10 tx=0x7ff4740092b0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.582+0000 7ff48ee00700 1 -- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 msgr2=0x7ff48819ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:25.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.582+0000 7ff48ee00700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48819ff40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff4780116a0 tx=0x7ff478011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:25.352 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.583+0000 7ff48ee00700 1 -- 172.21.15.90:0/63015642 shutdown_connections 2024-05-27T07:30:25.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.583+0000 7ff48ee00700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7ff46c041e00 0x7ff46c044220 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.583+0000 7ff48ee00700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff48810a500 0x7ff48819ff40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.583+0000 7ff48ee00700 1 --2- 172.21.15.90:0/63015642 >> [v2:172.21.15.90:6800/410158065,v1:172.21.15.90:6801/410158065] conn(0x7ff46c03e4c0 0x7ff46c0409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:25.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.583+0000 7ff48ee00700 1 -- 172.21.15.90:0/63015642 >> 172.21.15.90:0/63015642 conn(0x7ff488103fa0 msgr2=0x7ff488108420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:25.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.583+0000 7ff48ee00700 1 -- 172.21.15.90:0/63015642 shutdown_connections 2024-05-27T07:30:25.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:24.583+0000 7ff48ee00700 1 -- 172.21.15.90:0/63015642 wait complete. 2024-05-27T07:30:25.353 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 5 is available 2024-05-27T07:30:25.353 INFO:teuthology.orchestra.run.smithi090.stdout:Setting orchestrator backend to cephadm... 2024-05-27T07:30:25.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:25 smithi090 ceph-mon[31429]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T07:30:25.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:25 smithi090 ceph-mon[31429]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T07:30:25.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:25 smithi090 ceph-mon[31429]: [27/May/2024:07:30:24] ENGINE Bus STARTING 2024-05-27T07:30:25.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:25 smithi090 ceph-mon[31429]: [27/May/2024:07:30:24] ENGINE Serving on http://172.21.15.90:8765 2024-05-27T07:30:25.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:25 smithi090 ceph-mon[31429]: [27/May/2024:07:30:24] ENGINE Serving on https://172.21.15.90:7150 2024-05-27T07:30:25.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:25 smithi090 ceph-mon[31429]: [27/May/2024:07:30:24] ENGINE Bus STARTED 2024-05-27T07:30:25.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:25 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:30:26.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:26 smithi090 ceph-mon[31429]: mgrmap e8: x(active, since 2s) 2024-05-27T07:30:26.908 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:26 smithi090 ceph-mon[31429]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:26.908 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:26 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:26.908 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:26 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:30:26.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.988+0000 7fa418e00700 1 Processor -- start 2024-05-27T07:30:26.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.988+0000 7fa418e00700 1 -- start start 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.988+0000 7fa418e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41410c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.988+0000 7fa418e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa41410c650 con 0x7fa41410bc50 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.989+0000 7fa411e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41410c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.989+0000 7fa411e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41410c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54086/0 (socket says 172.21.15.90:54086) 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.989+0000 7fa411e00700 1 -- 172.21.15.90:0/937881474 learned_addr learned my addr 172.21.15.90:0/937881474 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.989+0000 7fa411e00700 1 -- 172.21.15.90:0/937881474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41410cdd0 con 0x7fa41410bc50 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.990+0000 7fa411e00700 1 --2- 172.21.15.90:0/937881474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41410c060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4040098a0 tx=0x7fa404009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6e3d1d8d2a05c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.990+0000 7fa410a00700 1 -- 172.21.15.90:0/937881474 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa404004160 con 0x7fa41410bc50 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.990+0000 7fa410a00700 1 -- 172.21.15.90:0/937881474 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa404004300 con 0x7fa41410bc50 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.990+0000 7fa410a00700 1 -- 172.21.15.90:0/937881474 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4040119e0 con 0x7fa41410bc50 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.991+0000 7fa418e00700 1 -- 172.21.15.90:0/937881474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 msgr2=0x7fa41410c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:26.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.991+0000 7fa418e00700 1 --2- 172.21.15.90:0/937881474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41410c060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa4040098a0 tx=0x7fa404009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.991+0000 7fa418e00700 1 -- 172.21.15.90:0/937881474 shutdown_connections 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.991+0000 7fa418e00700 1 --2- 172.21.15.90:0/937881474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41410c060 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.991+0000 7fa418e00700 1 -- 172.21.15.90:0/937881474 >> 172.21.15.90:0/937881474 conn(0x7fa41407f0e0 msgr2=0x7fa414109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.991+0000 7fa418e00700 1 -- 172.21.15.90:0/937881474 shutdown_connections 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.991+0000 7fa418e00700 1 -- 172.21.15.90:0/937881474 wait complete. 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.992+0000 7fa418e00700 1 Processor -- start 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.992+0000 7fa418e00700 1 -- start start 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.992+0000 7fa418e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41419ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.992+0000 7fa418e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa414110180 con 0x7fa41410bc50 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.993+0000 7fa411e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41419ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.993+0000 7fa411e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41419ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54100/0 (socket says 172.21.15.90:54100) 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.993+0000 7fa411e00700 1 -- 172.21.15.90:0/3426011318 learned_addr learned my addr 172.21.15.90:0/3426011318 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:26.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.993+0000 7fa411e00700 1 -- 172.21.15.90:0/3426011318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4141a04e0 con 0x7fa41410bc50 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.993+0000 7fa411e00700 1 --2- 172.21.15.90:0/3426011318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41419ffa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa404009fb0 tx=0x7fa404011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.993+0000 7fa40aa00700 1 -- 172.21.15.90:0/3426011318 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4040038b0 con 0x7fa41410bc50 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.993+0000 7fa40aa00700 1 -- 172.21.15.90:0/3426011318 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa404003a50 con 0x7fa41410bc50 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.993+0000 7fa418e00700 1 -- 172.21.15.90:0/3426011318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4141a0770 con 0x7fa41410bc50 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.994+0000 7fa40aa00700 1 -- 172.21.15.90:0/3426011318 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa40401a5c0 con 0x7fa41410bc50 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.994+0000 7fa418e00700 1 -- 172.21.15.90:0/3426011318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4141a3380 con 0x7fa41410bc50 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.995+0000 7fa418e00700 1 -- 172.21.15.90:0/3426011318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa414064e10 con 0x7fa41410bc50 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.995+0000 7fa40aa00700 1 -- 172.21.15.90:0/3426011318 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa40401a8c0 con 0x7fa41410bc50 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.995+0000 7fa40aa00700 1 --2- 172.21.15.90:0/3426011318 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa3f803e420 0x7fa3f8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.995+0000 7fa40aa00700 1 -- 172.21.15.90:0/3426011318 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa404053e30 con 0x7fa41410bc50 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.996+0000 7fa411400700 1 --2- 172.21.15.90:0/3426011318 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa3f803e420 0x7fa3f8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:25.996+0000 7fa411400700 1 --2- 172.21.15.90:0/3426011318 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa3f803e420 0x7fa3f8040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa400006fd0 tx=0x7fa400006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:26.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.001+0000 7fa40aa00700 1 -- 172.21.15.90:0/3426011318 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa40402c350 con 0x7fa41410bc50 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.234+0000 7fa418e00700 1 -- 172.21.15.90:0/3426011318 --> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fa4141098b0 con 0x7fa3f803e420 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.248+0000 7fa40aa00700 1 -- 172.21.15.90:0/3426011318 <== mgr.14120 v2:172.21.15.90:6800/2742579272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa4141098b0 con 0x7fa3f803e420 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.251+0000 7fa418e00700 1 -- 172.21.15.90:0/3426011318 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa3f803e420 msgr2=0x7fa3f8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.252+0000 7fa418e00700 1 --2- 172.21.15.90:0/3426011318 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa3f803e420 0x7fa3f8040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa400006fd0 tx=0x7fa400006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.252+0000 7fa418e00700 1 -- 172.21.15.90:0/3426011318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 msgr2=0x7fa41419ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.252+0000 7fa418e00700 1 --2- 172.21.15.90:0/3426011318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41419ffa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa404009fb0 tx=0x7fa404011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.252+0000 7fa418e00700 1 -- 172.21.15.90:0/3426011318 shutdown_connections 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.252+0000 7fa418e00700 1 --2- 172.21.15.90:0/3426011318 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa3f803e420 0x7fa3f8040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.252+0000 7fa418e00700 1 --2- 172.21.15.90:0/3426011318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41410bc50 0x7fa41419ffa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.252+0000 7fa418e00700 1 -- 172.21.15.90:0/3426011318 >> 172.21.15.90:0/3426011318 conn(0x7fa41407f0e0 msgr2=0x7fa4141091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.252+0000 7fa418e00700 1 -- 172.21.15.90:0/3426011318 shutdown_connections 2024-05-27T07:30:26.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:26.252+0000 7fa418e00700 1 -- 172.21.15.90:0/3426011318 wait complete. 2024-05-27T07:30:28.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:28 smithi090 ceph-mon[31429]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:28.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-27T07:30:28.497 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.606+0000 7fa653600700 1 Processor -- start 2024-05-27T07:30:28.497 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.606+0000 7fa653600700 1 -- start start 2024-05-27T07:30:28.497 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.606+0000 7fa653600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c10c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:28.497 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.606+0000 7fa653600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c07d6d0 con 0x7fa64c10a550 2024-05-27T07:30:28.497 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.607+0000 7fa650c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c10c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:28.498 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.607+0000 7fa650c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c10c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54104/0 (socket says 172.21.15.90:54104) 2024-05-27T07:30:28.498 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.607+0000 7fa650c00700 1 -- 172.21.15.90:0/114983997 learned_addr learned my addr 172.21.15.90:0/114983997 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:28.498 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.608+0000 7fa650c00700 1 -- 172.21.15.90:0/114983997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c10ceb0 con 0x7fa64c10a550 2024-05-27T07:30:28.498 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.608+0000 7fa650c00700 1 --2- 172.21.15.90:0/114983997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c10c970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098a0 tx=0x7fa640009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866cfc7620b3b18a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:28.498 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.608+0000 7fa64b400700 1 -- 172.21.15.90:0/114983997 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa640004160 con 0x7fa64c10a550 2024-05-27T07:30:28.499 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.608+0000 7fa64b400700 1 -- 172.21.15.90:0/114983997 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa640004300 con 0x7fa64c10a550 2024-05-27T07:30:28.500 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.608+0000 7fa64b400700 1 -- 172.21.15.90:0/114983997 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6400119e0 con 0x7fa64c10a550 2024-05-27T07:30:28.500 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.609+0000 7fa653600700 1 -- 172.21.15.90:0/114983997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 msgr2=0x7fa64c10c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:28.500 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.609+0000 7fa653600700 1 --2- 172.21.15.90:0/114983997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c10c970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098a0 tx=0x7fa640009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:28.500 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.610+0000 7fa653600700 1 -- 172.21.15.90:0/114983997 shutdown_connections 2024-05-27T07:30:28.500 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.610+0000 7fa653600700 1 --2- 172.21.15.90:0/114983997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c10c970 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:28.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.610+0000 7fa653600700 1 -- 172.21.15.90:0/114983997 >> 172.21.15.90:0/114983997 conn(0x7fa64c103f50 msgr2=0x7fa64c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:28.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.610+0000 7fa653600700 1 -- 172.21.15.90:0/114983997 shutdown_connections 2024-05-27T07:30:28.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.610+0000 7fa653600700 1 -- 172.21.15.90:0/114983997 wait complete. 2024-05-27T07:30:28.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.611+0000 7fa653600700 1 Processor -- start 2024-05-27T07:30:28.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.611+0000 7fa653600700 1 -- start start 2024-05-27T07:30:28.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.611+0000 7fa653600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:28.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.611+0000 7fa653600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c10ddb0 con 0x7fa64c10a550 2024-05-27T07:30:28.501 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.612+0000 7fa650c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.612+0000 7fa650c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c19dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54106/0 (socket says 172.21.15.90:54106) 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.612+0000 7fa650c00700 1 -- 172.21.15.90:0/1120950449 learned_addr learned my addr 172.21.15.90:0/1120950449 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.612+0000 7fa650c00700 1 -- 172.21.15.90:0/1120950449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c19e260 con 0x7fa64c10a550 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.612+0000 7fa650c00700 1 --2- 172.21.15.90:0/1120950449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c19dd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6400116a0 tx=0x7fa640011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.613+0000 7fa649600700 1 -- 172.21.15.90:0/1120950449 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa640003970 con 0x7fa64c10a550 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.613+0000 7fa649600700 1 -- 172.21.15.90:0/1120950449 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa640003b10 con 0x7fa64c10a550 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.613+0000 7fa653600700 1 -- 172.21.15.90:0/1120950449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c19e4f0 con 0x7fa64c10a550 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.613+0000 7fa649600700 1 -- 172.21.15.90:0/1120950449 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa640019550 con 0x7fa64c10a550 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.613+0000 7fa653600700 1 -- 172.21.15.90:0/1120950449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c1a10f0 con 0x7fa64c10a550 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.614+0000 7fa653600700 1 -- 172.21.15.90:0/1120950449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa64c064e10 con 0x7fa64c10a550 2024-05-27T07:30:28.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.614+0000 7fa649600700 1 -- 172.21.15.90:0/1120950449 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa64001a430 con 0x7fa64c10a550 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.615+0000 7fa649600700 1 --2- 172.21.15.90:0/1120950449 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa63403e470 0x7fa634040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.615+0000 7fa649600700 1 -- 172.21.15.90:0/1120950449 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa6400538d0 con 0x7fa64c10a550 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.615+0000 7fa64be00700 1 --2- 172.21.15.90:0/1120950449 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa63403e470 0x7fa634040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.616+0000 7fa64be00700 1 --2- 172.21.15.90:0/1120950449 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa63403e470 0x7fa634040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa638006fd0 tx=0x7fa638006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.621+0000 7fa649600700 1 -- 172.21.15.90:0/1120950449 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa64001a720 con 0x7fa64c10a550 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.853+0000 7fa653600700 1 -- 172.21.15.90:0/1120950449 --> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa64c104f70 con 0x7fa63403e470 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.855+0000 7fa649600700 1 -- 172.21.15.90:0/1120950449 <== mgr.14120 v2:172.21.15.90:6800/2742579272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa64c104f70 con 0x7fa63403e470 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.857+0000 7fa653600700 1 -- 172.21.15.90:0/1120950449 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa63403e470 msgr2=0x7fa634040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.857+0000 7fa653600700 1 --2- 172.21.15.90:0/1120950449 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa63403e470 0x7fa634040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa638006fd0 tx=0x7fa638006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.857+0000 7fa653600700 1 -- 172.21.15.90:0/1120950449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 msgr2=0x7fa64c19dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.858+0000 7fa653600700 1 --2- 172.21.15.90:0/1120950449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c19dd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa6400116a0 tx=0x7fa640011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.858+0000 7fa653600700 1 -- 172.21.15.90:0/1120950449 shutdown_connections 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.858+0000 7fa653600700 1 --2- 172.21.15.90:0/1120950449 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa63403e470 0x7fa634040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:28.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.858+0000 7fa653600700 1 --2- 172.21.15.90:0/1120950449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa64c10a550 0x7fa64c19dd20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:28.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.858+0000 7fa653600700 1 -- 172.21.15.90:0/1120950449 >> 172.21.15.90:0/1120950449 conn(0x7fa64c103f50 msgr2=0x7fa64c104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:28.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.858+0000 7fa653600700 1 -- 172.21.15.90:0/1120950449 shutdown_connections 2024-05-27T07:30:28.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:27.858+0000 7fa653600700 1 -- 172.21.15.90:0/1120950449 wait complete. 2024-05-27T07:30:28.506 INFO:teuthology.orchestra.run.smithi090.stdout:Generating ssh key... 2024-05-27T07:30:29.930 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: Generating public/private rsa key pair. 2024-05-27T07:30:29.930 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: Your identification has been saved in /tmp/tmp861ddinh/key. 2024-05-27T07:30:29.930 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: Your public key has been saved in /tmp/tmp861ddinh/key.pub. 2024-05-27T07:30:29.930 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: The key fingerprint is: 2024-05-27T07:30:29.930 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: SHA256:6El2pQxbkAEmGPtDDoVdUZzXJhEl+NIi0iug51qGguo ceph-b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:30:29.930 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: The key's randomart image is: 2024-05-27T07:30:29.930 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: +---[RSA 3072]----+ 2024-05-27T07:30:29.930 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: |.=o.==+=+=. | 2024-05-27T07:30:29.930 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: |oo.o =.o.o | 2024-05-27T07:30:29.931 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: |o .. .+.o. | 2024-05-27T07:30:29.931 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: |.=. o o*oo | 2024-05-27T07:30:29.931 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: |..+. o=oS | 2024-05-27T07:30:29.931 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: |o.o..+ o | 2024-05-27T07:30:29.931 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: |+oo. o | 2024-05-27T07:30:29.931 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: |oo. | 2024-05-27T07:30:29.932 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: |+E | 2024-05-27T07:30:29.932 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: +----[SHA256]-----+ 2024-05-27T07:30:30.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.161+0000 7f77f0e00700 1 Processor -- start 2024-05-27T07:30:30.250 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.161+0000 7f77f0e00700 1 -- start start 2024-05-27T07:30:30.250 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.162+0000 7f77f0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec112b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.162+0000 7f77f0e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec113090 con 0x7f77ec110730 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.162+0000 7f77eb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec112b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.163+0000 7f77eb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec112b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55508/0 (socket says 172.21.15.90:55508) 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.163+0000 7f77eb400700 1 -- 172.21.15.90:0/3794915175 learned_addr learned my addr 172.21.15.90:0/3794915175 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.163+0000 7f77eb400700 1 -- 172.21.15.90:0/3794915175 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec113210 con 0x7f77ec110730 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.164+0000 7f77eb400700 1 --2- 172.21.15.90:0/3794915175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec112b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0098a0 tx=0x7f77dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e9359273d528bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.164+0000 7f77eaa00700 1 -- 172.21.15.90:0/3794915175 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77dc004160 con 0x7f77ec110730 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.164+0000 7f77eaa00700 1 -- 172.21.15.90:0/3794915175 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f77dc004300 con 0x7f77ec110730 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.164+0000 7f77eaa00700 1 -- 172.21.15.90:0/3794915175 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77dc0119e0 con 0x7f77ec110730 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.165+0000 7f77f0e00700 1 -- 172.21.15.90:0/3794915175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 msgr2=0x7f77ec112b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.165+0000 7f77f0e00700 1 --2- 172.21.15.90:0/3794915175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec112b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0098a0 tx=0x7f77dc009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.165+0000 7f77f0e00700 1 -- 172.21.15.90:0/3794915175 shutdown_connections 2024-05-27T07:30:30.251 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.165+0000 7f77f0e00700 1 --2- 172.21.15.90:0/3794915175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec112b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:30.252 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.165+0000 7f77f0e00700 1 -- 172.21.15.90:0/3794915175 >> 172.21.15.90:0/3794915175 conn(0x7f77ec07efd0 msgr2=0x7f77ec0813f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:30.252 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.165+0000 7f77f0e00700 1 -- 172.21.15.90:0/3794915175 shutdown_connections 2024-05-27T07:30:30.252 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.165+0000 7f77f0e00700 1 -- 172.21.15.90:0/3794915175 wait complete. 2024-05-27T07:30:30.252 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.166+0000 7f77f0e00700 1 Processor -- start 2024-05-27T07:30:30.252 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.167+0000 7f77f0e00700 1 -- start start 2024-05-27T07:30:30.252 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.167+0000 7f77f0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec1a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:30.252 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.167+0000 7f77f0e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1141e0 con 0x7f77ec110730 2024-05-27T07:30:30.252 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.167+0000 7f77eb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec1a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:30.252 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.167+0000 7f77eb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec1a4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55524/0 (socket says 172.21.15.90:55524) 2024-05-27T07:30:30.253 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.167+0000 7f77eb400700 1 -- 172.21.15.90:0/1017892020 learned_addr learned my addr 172.21.15.90:0/1017892020 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:30.253 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.168+0000 7f77eb400700 1 -- 172.21.15.90:0/1017892020 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec1a46c0 con 0x7f77ec110730 2024-05-27T07:30:30.253 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.168+0000 7f77eb400700 1 --2- 172.21.15.90:0/1017892020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec1a4180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0116a0 tx=0x7f77dc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:30.253 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.168+0000 7f77e8c00700 1 -- 172.21.15.90:0/1017892020 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77dc003970 con 0x7f77ec110730 2024-05-27T07:30:30.253 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.168+0000 7f77e8c00700 1 -- 172.21.15.90:0/1017892020 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f77dc003b10 con 0x7f77ec110730 2024-05-27T07:30:30.253 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.168+0000 7f77f0e00700 1 -- 172.21.15.90:0/1017892020 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ec1a4950 con 0x7f77ec110730 2024-05-27T07:30:30.253 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.168+0000 7f77e8c00700 1 -- 172.21.15.90:0/1017892020 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77dc019550 con 0x7f77ec110730 2024-05-27T07:30:30.253 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.168+0000 7f77f0e00700 1 -- 172.21.15.90:0/1017892020 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ec1a7560 con 0x7f77ec110730 2024-05-27T07:30:30.253 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.169+0000 7f77f0e00700 1 -- 172.21.15.90:0/1017892020 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ec064e10 con 0x7f77ec110730 2024-05-27T07:30:30.254 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.170+0000 7f77e8c00700 1 -- 172.21.15.90:0/1017892020 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f77dc0197e0 con 0x7f77ec110730 2024-05-27T07:30:30.254 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.170+0000 7f77e8c00700 1 --2- 172.21.15.90:0/1017892020 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f77d003e4c0 0x7f77d00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:30.254 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.170+0000 7f77e8c00700 1 -- 172.21.15.90:0/1017892020 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f77dc053880 con 0x7f77ec110730 2024-05-27T07:30:30.254 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.171+0000 7f77e3e00700 1 --2- 172.21.15.90:0/1017892020 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f77d003e4c0 0x7f77d00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:30.254 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.171+0000 7f77e3e00700 1 --2- 172.21.15.90:0/1017892020 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f77d003e4c0 0x7f77d00409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77d4006fd0 tx=0x7f77d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:30.254 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.176+0000 7f77e8c00700 1 -- 172.21.15.90:0/1017892020 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f77dc016260 con 0x7f77ec110730 2024-05-27T07:30:30.254 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.409+0000 7f77f0e00700 1 -- 172.21.15.90:0/1017892020 --> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f77ec080fa0 con 0x7f77d003e4c0 2024-05-27T07:30:30.254 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.599+0000 7f77e8c00700 1 -- 172.21.15.90:0/1017892020 <== mgr.14120 v2:172.21.15.90:6800/2742579272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f77ec080fa0 con 0x7f77d003e4c0 2024-05-27T07:30:30.254 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.602+0000 7f77f0e00700 1 -- 172.21.15.90:0/1017892020 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f77d003e4c0 msgr2=0x7f77d00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:30.254 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.602+0000 7f77f0e00700 1 --2- 172.21.15.90:0/1017892020 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f77d003e4c0 0x7f77d00409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77d4006fd0 tx=0x7f77d4006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:30.255 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.602+0000 7f77f0e00700 1 -- 172.21.15.90:0/1017892020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 msgr2=0x7f77ec1a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:30.255 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.602+0000 7f77f0e00700 1 --2- 172.21.15.90:0/1017892020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec1a4180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0116a0 tx=0x7f77dc011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:30.255 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.602+0000 7f77f0e00700 1 -- 172.21.15.90:0/1017892020 shutdown_connections 2024-05-27T07:30:30.255 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.602+0000 7f77f0e00700 1 --2- 172.21.15.90:0/1017892020 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f77d003e4c0 0x7f77d00409d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:30.255 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.602+0000 7f77f0e00700 1 --2- 172.21.15.90:0/1017892020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77ec110730 0x7f77ec1a4180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:30.255 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.602+0000 7f77f0e00700 1 -- 172.21.15.90:0/1017892020 >> 172.21.15.90:0/1017892020 conn(0x7f77ec07efd0 msgr2=0x7f77ec080890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:30.255 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.602+0000 7f77f0e00700 1 -- 172.21.15.90:0/1017892020 shutdown_connections 2024-05-27T07:30:30.255 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:29.602+0000 7f77f0e00700 1 -- 172.21.15.90:0/1017892020 wait complete. 2024-05-27T07:30:30.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:30 smithi090 ceph-mon[31429]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:30.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:30 smithi090 ceph-mon[31429]: Generating ssh key... 2024-05-27T07:30:30.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:30 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:30.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:30 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:31.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:31 smithi090 ceph-mon[31429]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:31.866 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy7HkqLpW6HfI2MMU0CTJa2Z+3KhtD5H/qszNaqdj0me9lMm+Ha7nXDhYnUiLzm69P8FzAQtiTjqLuB0rckKM4svzvxNff46MUsRk67fBtrylqA71RKbnOmUtdphGLS1DFhiyeHXKLD0eo+qPLGKbgVe+xFeOLvxetpelijgmwlYjICC5g7aTe6GbZ+Tw+27N9treDQqgSfz8fIOEO7gaeJdiJ5StUyC2jrR2iDob5ZvP2vKHYKAvfPMDW93wGKpEMVKU4cDdh36oEQnM90/IHLwFW6v6Qr1oaoDZrAiELvKR9BOcQPTL3U2EPQGFXoCYAx8l5HREDxG4xYcDThQYJ58C7pFoMehazaOCi7bYFdVNyGskOyg8uiVTndOGk32bavvRl2FR2mI/l7ruXwLNWx7PVTvxxo45UxBhmMyjwCsXxpJ3AqW77gLk+rav7UtoDCxPoiCh1DTWIl1H09q+bAUecgizs/+5IH4ajTDx45O32RYwYh7H/RYz0P4/t1U0= ceph-b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:30:31.866 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.871+0000 7f5432a00700 1 Processor -- start 2024-05-27T07:30:31.866 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.871+0000 7f5432a00700 1 -- start start 2024-05-27T07:30:31.866 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.871+0000 7f5432a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c10be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:31.866 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.871+0000 7f5432a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f542c10c470 con 0x7f542c10ba70 2024-05-27T07:30:31.866 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.872+0000 7f5431600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c10be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.872+0000 7f5431600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c10be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55528/0 (socket says 172.21.15.90:55528) 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.872+0000 7f5431600700 1 -- 172.21.15.90:0/3071126585 learned_addr learned my addr 172.21.15.90:0/3071126585 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.872+0000 7f5431600700 1 -- 172.21.15.90:0/3071126585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542c10cc40 con 0x7f542c10ba70 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.873+0000 7f5431600700 1 --2- 172.21.15.90:0/3071126585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c10be80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f54200098a0 tx=0x7f5420009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca4cc967c5682d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.873+0000 7f542be00700 1 -- 172.21.15.90:0/3071126585 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5420004160 con 0x7f542c10ba70 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.873+0000 7f542be00700 1 -- 172.21.15.90:0/3071126585 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5420004300 con 0x7f542c10ba70 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.873+0000 7f542be00700 1 -- 172.21.15.90:0/3071126585 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54200119e0 con 0x7f542c10ba70 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.874+0000 7f5432a00700 1 -- 172.21.15.90:0/3071126585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 msgr2=0x7f542c10be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.874+0000 7f5432a00700 1 --2- 172.21.15.90:0/3071126585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c10be80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f54200098a0 tx=0x7f5420009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.874+0000 7f5432a00700 1 -- 172.21.15.90:0/3071126585 shutdown_connections 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.874+0000 7f5432a00700 1 --2- 172.21.15.90:0/3071126585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c10be80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:31.867 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.874+0000 7f5432a00700 1 -- 172.21.15.90:0/3071126585 >> 172.21.15.90:0/3071126585 conn(0x7f542c07efd0 msgr2=0x7f542c07f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.874+0000 7f5432a00700 1 -- 172.21.15.90:0/3071126585 shutdown_connections 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.874+0000 7f5432a00700 1 -- 172.21.15.90:0/3071126585 wait complete. 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.875+0000 7f5432a00700 1 Processor -- start 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.875+0000 7f5432a00700 1 -- start start 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.876+0000 7f5432a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.876+0000 7f5432a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f542c10fff0 con 0x7f542c10ba70 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.876+0000 7f5431600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.876+0000 7f5431600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c19fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55536/0 (socket says 172.21.15.90:55536) 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.876+0000 7f5431600700 1 -- 172.21.15.90:0/949211256 learned_addr learned my addr 172.21.15.90:0/949211256 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.877+0000 7f5431600700 1 -- 172.21.15.90:0/949211256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542c1a0330 con 0x7f542c10ba70 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.877+0000 7f5431600700 1 --2- 172.21.15.90:0/949211256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c19fdf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f54200046e0 tx=0x7f5420011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:31.868 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.877+0000 7f542a000700 1 -- 172.21.15.90:0/949211256 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54200039e0 con 0x7f542c10ba70 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.877+0000 7f542a000700 1 -- 172.21.15.90:0/949211256 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5420003b80 con 0x7f542c10ba70 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.877+0000 7f5432a00700 1 -- 172.21.15.90:0/949211256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542c1a05c0 con 0x7f542c10ba70 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.877+0000 7f542a000700 1 -- 172.21.15.90:0/949211256 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5420024700 con 0x7f542c10ba70 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.877+0000 7f5432a00700 1 -- 172.21.15.90:0/949211256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542c1a31d0 con 0x7f542c10ba70 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.879+0000 7f542a000700 1 -- 172.21.15.90:0/949211256 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5420022020 con 0x7f542c10ba70 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.879+0000 7f5432a00700 1 -- 172.21.15.90:0/949211256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f542c064e10 con 0x7f542c10ba70 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.879+0000 7f542a000700 1 --2- 172.21.15.90:0/949211256 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f541403e4c0 0x7f54140409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.879+0000 7f542a000700 1 -- 172.21.15.90:0/949211256 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f54200539d0 con 0x7f542c10ba70 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.879+0000 7f5430c00700 1 --2- 172.21.15.90:0/949211256 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f541403e4c0 0x7f54140409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.880+0000 7f5430c00700 1 --2- 172.21.15.90:0/949211256 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f541403e4c0 0x7f54140409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5418006fd0 tx=0x7f5418006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:30.885+0000 7f542a000700 1 -- 172.21.15.90:0/949211256 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54200172b0 con 0x7f542c10ba70 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.117+0000 7f5432a00700 1 -- 172.21.15.90:0/949211256 --> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f542c109170 con 0x7f541403e4c0 2024-05-27T07:30:31.869 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.119+0000 7f542a000700 1 -- 172.21.15.90:0/949211256 <== mgr.14120 v2:172.21.15.90:6800/2742579272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f542c109170 con 0x7f541403e4c0 2024-05-27T07:30:31.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.123+0000 7f5432a00700 1 -- 172.21.15.90:0/949211256 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f541403e4c0 msgr2=0x7f54140409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:31.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.123+0000 7f5432a00700 1 --2- 172.21.15.90:0/949211256 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f541403e4c0 0x7f54140409d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5418006fd0 tx=0x7f5418006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:31.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.123+0000 7f5432a00700 1 -- 172.21.15.90:0/949211256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 msgr2=0x7f542c19fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:31.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.123+0000 7f5432a00700 1 --2- 172.21.15.90:0/949211256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c19fdf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f54200046e0 tx=0x7f5420011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:31.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.123+0000 7f5432a00700 1 -- 172.21.15.90:0/949211256 shutdown_connections 2024-05-27T07:30:31.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.123+0000 7f5432a00700 1 --2- 172.21.15.90:0/949211256 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f541403e4c0 0x7f54140409d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:31.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.123+0000 7f5432a00700 1 --2- 172.21.15.90:0/949211256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f542c10ba70 0x7f542c19fdf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:31.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.123+0000 7f5432a00700 1 -- 172.21.15.90:0/949211256 >> 172.21.15.90:0/949211256 conn(0x7f542c07efd0 msgr2=0x7f542c108a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:31.870 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.123+0000 7f5432a00700 1 -- 172.21.15.90:0/949211256 shutdown_connections 2024-05-27T07:30:31.871 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:31.123+0000 7f5432a00700 1 -- 172.21.15.90:0/949211256 wait complete. 2024-05-27T07:30:31.871 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-27T07:30:31.871 INFO:teuthology.orchestra.run.smithi090.stdout:Adding key to root@localhost authorized_keys... 2024-05-27T07:30:31.871 INFO:teuthology.orchestra.run.smithi090.stdout:Adding host smithi090... 2024-05-27T07:30:33.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:33 smithi090 ceph-mon[31429]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:34.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:34 smithi090 ceph-mon[31429]: Deploying cephadm binary to smithi090 2024-05-27T07:30:39.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Added host 'smithi090' with addr '172.21.15.90' 2024-05-27T07:30:39.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.525+0000 7fa53d800700 1 Processor -- start 2024-05-27T07:30:39.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.525+0000 7fa53d800700 1 -- start start 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.525+0000 7fa53d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa538112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.525+0000 7fa53d800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa538113230 con 0x7fa5381108d0 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.525+0000 7fa536a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa538112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.525+0000 7fa536a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa538112cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55542/0 (socket says 172.21.15.90:55542) 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.525+0000 7fa536a00700 1 -- 172.21.15.90:0/2048792796 learned_addr learned my addr 172.21.15.90:0/2048792796 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.526+0000 7fa536a00700 1 -- 172.21.15.90:0/2048792796 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5381133b0 con 0x7fa5381108d0 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.526+0000 7fa536a00700 1 --2- 172.21.15.90:0/2048792796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa538112cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa5280098a0 tx=0x7fa528009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ba5c1326fda189 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.527+0000 7fa536000700 1 -- 172.21.15.90:0/2048792796 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa528004160 con 0x7fa5381108d0 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.527+0000 7fa536000700 1 -- 172.21.15.90:0/2048792796 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa528004300 con 0x7fa5381108d0 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.527+0000 7fa536000700 1 -- 172.21.15.90:0/2048792796 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5280119e0 con 0x7fa5381108d0 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.528+0000 7fa53d800700 1 -- 172.21.15.90:0/2048792796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 msgr2=0x7fa538112cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.528+0000 7fa53d800700 1 --2- 172.21.15.90:0/2048792796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa538112cf0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa5280098a0 tx=0x7fa528009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:39.552 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.528+0000 7fa53d800700 1 -- 172.21.15.90:0/2048792796 shutdown_connections 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.528+0000 7fa53d800700 1 --2- 172.21.15.90:0/2048792796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa538112cf0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.528+0000 7fa53d800700 1 -- 172.21.15.90:0/2048792796 >> 172.21.15.90:0/2048792796 conn(0x7fa53807f080 msgr2=0x7fa538081530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.528+0000 7fa53d800700 1 -- 172.21.15.90:0/2048792796 shutdown_connections 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.528+0000 7fa53d800700 1 -- 172.21.15.90:0/2048792796 wait complete. 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.529+0000 7fa53d800700 1 Processor -- start 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.529+0000 7fa53d800700 1 -- start start 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.530+0000 7fa53d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa5381a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.530+0000 7fa53d800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381143d0 con 0x7fa5381108d0 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.530+0000 7fa536a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa5381a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.530+0000 7fa536a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa5381a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55546/0 (socket says 172.21.15.90:55546) 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.530+0000 7fa536a00700 1 -- 172.21.15.90:0/4245226561 learned_addr learned my addr 172.21.15.90:0/4245226561 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.530+0000 7fa536a00700 1 -- 172.21.15.90:0/4245226561 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5381a4840 con 0x7fa5381108d0 2024-05-27T07:30:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.531+0000 7fa536a00700 1 --2- 172.21.15.90:0/4245226561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa5381a4300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa5280116a0 tx=0x7fa528011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.531+0000 7fa52f400700 1 -- 172.21.15.90:0/4245226561 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa528003970 con 0x7fa5381108d0 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.531+0000 7fa52f400700 1 -- 172.21.15.90:0/4245226561 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fa528003b10 con 0x7fa5381108d0 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.531+0000 7fa53d800700 1 -- 172.21.15.90:0/4245226561 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5381a4ad0 con 0x7fa5381108d0 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.531+0000 7fa52f400700 1 -- 172.21.15.90:0/4245226561 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa528019550 con 0x7fa5381108d0 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.532+0000 7fa53d800700 1 -- 172.21.15.90:0/4245226561 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381a76e0 con 0x7fa5381108d0 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.533+0000 7fa52f400700 1 -- 172.21.15.90:0/4245226561 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fa5280197c0 con 0x7fa5381108d0 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.533+0000 7fa53d800700 1 -- 172.21.15.90:0/4245226561 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa538064e10 con 0x7fa5381108d0 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.533+0000 7fa52f400700 1 --2- 172.21.15.90:0/4245226561 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa51c03e4c0 0x7fa51c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.533+0000 7fa52f400700 1 -- 172.21.15.90:0/4245226561 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa5280538b0 con 0x7fa5381108d0 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.533+0000 7fa52fe00700 1 --2- 172.21.15.90:0/4245226561 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa51c03e4c0 0x7fa51c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:39.554 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.534+0000 7fa52fe00700 1 --2- 172.21.15.90:0/4245226561 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa51c03e4c0 0x7fa51c0409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa520006fd0 tx=0x7fa520006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.540+0000 7fa52f400700 1 -- 172.21.15.90:0/4245226561 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa528016260 con 0x7fa5381108d0 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:32.771+0000 7fa53d800700 1 -- 172.21.15.90:0/4245226561 --> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}) -- 0x7fa538080050 con 0x7fa51c03e4c0 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.675+0000 7fa52f400700 1 -- 172.21.15.90:0/4245226561 <== mgr.14120 v2:172.21.15.90:6800/2742579272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fa538080050 con 0x7fa51c03e4c0 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.680+0000 7fa53d800700 1 -- 172.21.15.90:0/4245226561 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa51c03e4c0 msgr2=0x7fa51c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.680+0000 7fa53d800700 1 --2- 172.21.15.90:0/4245226561 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa51c03e4c0 0x7fa51c0409d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa520006fd0 tx=0x7fa520006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.680+0000 7fa53d800700 1 -- 172.21.15.90:0/4245226561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 msgr2=0x7fa5381a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.680+0000 7fa53d800700 1 --2- 172.21.15.90:0/4245226561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa5381a4300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa5280116a0 tx=0x7fa528011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.680+0000 7fa53d800700 1 -- 172.21.15.90:0/4245226561 shutdown_connections 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.680+0000 7fa53d800700 1 --2- 172.21.15.90:0/4245226561 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fa51c03e4c0 0x7fa51c0409d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.680+0000 7fa53d800700 1 --2- 172.21.15.90:0/4245226561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5381108d0 0x7fa5381a4300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.680+0000 7fa53d800700 1 -- 172.21.15.90:0/4245226561 >> 172.21.15.90:0/4245226561 conn(0x7fa53807f080 msgr2=0x7fa53807f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.680+0000 7fa53d800700 1 -- 172.21.15.90:0/4245226561 shutdown_connections 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:38.680+0000 7fa53d800700 1 -- 172.21.15.90:0/4245226561 wait complete. 2024-05-27T07:30:39.555 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mon service... 2024-05-27T07:30:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:39 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:39 smithi090 ceph-mon[31429]: Added host smithi090 2024-05-27T07:30:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:39 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:30:41.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:41 smithi090 ceph-mon[31429]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:41.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:41 smithi090 ceph-mon[31429]: Saving service mon spec with placement count:5 2024-05-27T07:30:41.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:41 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:41.880 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-27T07:30:41.880 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.297+0000 7f36e7200700 1 Processor -- start 2024-05-27T07:30:41.880 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.297+0000 7f36e7200700 1 -- start start 2024-05-27T07:30:41.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.298+0000 7f36e7200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e0109e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:41.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.298+0000 7f36e7200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f36e010a450 con 0x7f36e0109a50 2024-05-27T07:30:41.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.298+0000 7f36dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e0109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.299+0000 7f36dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e0109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41364/0 (socket says 172.21.15.90:41364) 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.299+0000 7f36dfe00700 1 -- 172.21.15.90:0/2943332388 learned_addr learned my addr 172.21.15.90:0/2943332388 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.299+0000 7f36dfe00700 1 -- 172.21.15.90:0/2943332388 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e010ac20 con 0x7f36e0109a50 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.299+0000 7f36dfe00700 1 --2- 172.21.15.90:0/2943332388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e0109e60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36d00098a0 tx=0x7f36d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdadc780c1557e0b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.300+0000 7f36dea00700 1 -- 172.21.15.90:0/2943332388 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36d0004160 con 0x7f36e0109a50 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.300+0000 7f36dea00700 1 -- 172.21.15.90:0/2943332388 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f36d0004300 con 0x7f36e0109a50 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.300+0000 7f36dea00700 1 -- 172.21.15.90:0/2943332388 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36d00119e0 con 0x7f36e0109a50 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.301+0000 7f36e7200700 1 -- 172.21.15.90:0/2943332388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 msgr2=0x7f36e0109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.301+0000 7f36e7200700 1 --2- 172.21.15.90:0/2943332388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e0109e60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36d00098a0 tx=0x7f36d0009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.301+0000 7f36e7200700 1 -- 172.21.15.90:0/2943332388 shutdown_connections 2024-05-27T07:30:41.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.301+0000 7f36e7200700 1 --2- 172.21.15.90:0/2943332388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e0109e60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.301+0000 7f36e7200700 1 -- 172.21.15.90:0/2943332388 >> 172.21.15.90:0/2943332388 conn(0x7f36e01050b0 msgr2=0x7f36e0107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.301+0000 7f36e7200700 1 -- 172.21.15.90:0/2943332388 shutdown_connections 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.301+0000 7f36e7200700 1 -- 172.21.15.90:0/2943332388 wait complete. 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.302+0000 7f36e7200700 1 Processor -- start 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.302+0000 7f36e7200700 1 -- start start 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.302+0000 7f36e7200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e007ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.302+0000 7f36e7200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f36e010dfd0 con 0x7f36e0109a50 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.303+0000 7f36dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e007ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.303+0000 7f36dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e007ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41376/0 (socket says 172.21.15.90:41376) 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.303+0000 7f36dfe00700 1 -- 172.21.15.90:0/2207047804 learned_addr learned my addr 172.21.15.90:0/2207047804 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:41.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.303+0000 7f36dfe00700 1 -- 172.21.15.90:0/2207047804 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e007efb0 con 0x7f36e0109a50 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.303+0000 7f36dfe00700 1 --2- 172.21.15.90:0/2207047804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e007ea70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f36d00046e0 tx=0x7f36d0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.304+0000 7f36dcc00700 1 -- 172.21.15.90:0/2207047804 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36d00039e0 con 0x7f36e0109a50 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.304+0000 7f36dcc00700 1 -- 172.21.15.90:0/2207047804 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f36d0003b80 con 0x7f36e0109a50 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.304+0000 7f36e7200700 1 -- 172.21.15.90:0/2207047804 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e007d200 con 0x7f36e0109a50 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.304+0000 7f36dcc00700 1 -- 172.21.15.90:0/2207047804 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36d0024700 con 0x7f36e0109a50 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.304+0000 7f36e7200700 1 -- 172.21.15.90:0/2207047804 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e007d6e0 con 0x7f36e0109a50 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.305+0000 7f36dcc00700 1 -- 172.21.15.90:0/2207047804 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f36d0022020 con 0x7f36e0109a50 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.306+0000 7f36e7200700 1 -- 172.21.15.90:0/2207047804 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c0005360 con 0x7f36e0109a50 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.306+0000 7f36dcc00700 1 --2- 172.21.15.90:0/2207047804 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f36c403e470 0x7f36c4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.306+0000 7f36dcc00700 1 -- 172.21.15.90:0/2207047804 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f36d00547d0 con 0x7f36e0109a50 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.306+0000 7f36df400700 1 --2- 172.21.15.90:0/2207047804 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f36c403e470 0x7f36c4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.307+0000 7f36df400700 1 --2- 172.21.15.90:0/2207047804 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f36c403e470 0x7f36c4040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f36cc006fd0 tx=0x7f36cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:41.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.313+0000 7f36dcc00700 1 -- 172.21.15.90:0/2207047804 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36d00172b0 con 0x7f36e0109a50 2024-05-27T07:30:41.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.575+0000 7f36e7200700 1 -- 172.21.15.90:0/2207047804 --> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f36c0000bf0 con 0x7f36c403e470 2024-05-27T07:30:41.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.588+0000 7f36dcc00700 1 -- 172.21.15.90:0/2207047804 <== mgr.14120 v2:172.21.15.90:6800/2742579272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f36c0000bf0 con 0x7f36c403e470 2024-05-27T07:30:41.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.592+0000 7f36e7200700 1 -- 172.21.15.90:0/2207047804 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f36c403e470 msgr2=0x7f36c4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:41.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.592+0000 7f36e7200700 1 --2- 172.21.15.90:0/2207047804 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f36c403e470 0x7f36c4040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f36cc006fd0 tx=0x7f36cc006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:41.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.592+0000 7f36e7200700 1 -- 172.21.15.90:0/2207047804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 msgr2=0x7f36e007ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:41.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.592+0000 7f36e7200700 1 --2- 172.21.15.90:0/2207047804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e007ea70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f36d00046e0 tx=0x7f36d0011a60 comp rx=0 tx=0).stop 2024-05-27T07:30:41.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.592+0000 7f36e7200700 1 -- 172.21.15.90:0/2207047804 shutdown_connections 2024-05-27T07:30:41.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.592+0000 7f36e7200700 1 --2- 172.21.15.90:0/2207047804 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f36c403e470 0x7f36c4040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:41.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.592+0000 7f36e7200700 1 --2- 172.21.15.90:0/2207047804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e0109a50 0x7f36e007ea70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:41.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.592+0000 7f36e7200700 1 -- 172.21.15.90:0/2207047804 >> 172.21.15.90:0/2207047804 conn(0x7f36e01050b0 msgr2=0x7f36e0105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:41.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.592+0000 7f36e7200700 1 -- 172.21.15.90:0/2207047804 shutdown_connections 2024-05-27T07:30:41.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:40.592+0000 7f36e7200700 1 -- 172.21.15.90:0/2207047804 wait complete. 2024-05-27T07:30:41.892 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mgr service... 2024-05-27T07:30:43.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:43 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:43.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:43 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:43.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:43 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:43.750 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-27T07:30:43.750 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.745+0000 7f5cc0000700 1 Processor -- start 2024-05-27T07:30:43.750 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.746+0000 7f5cc0000700 1 -- start start 2024-05-27T07:30:43.750 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.746+0000 7f5cc0000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb807d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:43.750 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.746+0000 7f5cc0000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb807d760 con 0x7f5cb807edf0 2024-05-27T07:30:43.750 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.747+0000 7f5cbd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb807d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:43.750 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.747+0000 7f5cbd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb807d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41384/0 (socket says 172.21.15.90:41384) 2024-05-27T07:30:43.750 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.747+0000 7f5cbd600700 1 -- 172.21.15.90:0/982650169 learned_addr learned my addr 172.21.15.90:0/982650169 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.747+0000 7f5cbd600700 1 -- 172.21.15.90:0/982650169 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb807d8e0 con 0x7f5cb807edf0 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.748+0000 7f5cbd600700 1 --2- 172.21.15.90:0/982650169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb807d220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0098a0 tx=0x7f5cac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce290ac898651b6a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.748+0000 7f5cb7e00700 1 -- 172.21.15.90:0/982650169 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cac004160 con 0x7f5cb807edf0 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.748+0000 7f5cb7e00700 1 -- 172.21.15.90:0/982650169 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cac004300 con 0x7f5cb807edf0 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.748+0000 7f5cb7e00700 1 -- 172.21.15.90:0/982650169 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cac011960 con 0x7f5cb807edf0 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.749+0000 7f5cc0000700 1 -- 172.21.15.90:0/982650169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 msgr2=0x7f5cb807d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.749+0000 7f5cc0000700 1 --2- 172.21.15.90:0/982650169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb807d220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0098a0 tx=0x7f5cac009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.749+0000 7f5cc0000700 1 -- 172.21.15.90:0/982650169 shutdown_connections 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.749+0000 7f5cc0000700 1 --2- 172.21.15.90:0/982650169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb807d220 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.749+0000 7f5cc0000700 1 -- 172.21.15.90:0/982650169 >> 172.21.15.90:0/982650169 conn(0x7f5cb81050b0 msgr2=0x7f5cb8107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.749+0000 7f5cc0000700 1 -- 172.21.15.90:0/982650169 shutdown_connections 2024-05-27T07:30:43.751 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.749+0000 7f5cc0000700 1 -- 172.21.15.90:0/982650169 wait complete. 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.750+0000 7f5cc0000700 1 Processor -- start 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.750+0000 7f5cc0000700 1 -- start start 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.751+0000 7f5cc0000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb819ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.751+0000 7f5cc0000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb8110030 con 0x7f5cb807edf0 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.751+0000 7f5cbd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb819ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.751+0000 7f5cbd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb819ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41394/0 (socket says 172.21.15.90:41394) 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.751+0000 7f5cbd600700 1 -- 172.21.15.90:0/4238648124 learned_addr learned my addr 172.21.15.90:0/4238648124 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.752+0000 7f5cbd600700 1 -- 172.21.15.90:0/4238648124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb81a04a0 con 0x7f5cb807edf0 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.752+0000 7f5cbd600700 1 --2- 172.21.15.90:0/4238648124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb819ff60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5cac011670 tx=0x7f5cac011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.752+0000 7f5cb6000700 1 -- 172.21.15.90:0/4238648124 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cac01f070 con 0x7f5cb807edf0 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.752+0000 7f5cb6000700 1 -- 172.21.15.90:0/4238648124 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cac003b20 con 0x7f5cb807edf0 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.752+0000 7f5cc0000700 1 -- 172.21.15.90:0/4238648124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb81a0730 con 0x7f5cb807edf0 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.752+0000 7f5cb6000700 1 -- 172.21.15.90:0/4238648124 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cac019580 con 0x7f5cb807edf0 2024-05-27T07:30:43.752 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.752+0000 7f5cc0000700 1 -- 172.21.15.90:0/4238648124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb81a3340 con 0x7f5cb807edf0 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.754+0000 7f5cc0000700 1 -- 172.21.15.90:0/4238648124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb8064e10 con 0x7f5cb807edf0 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.754+0000 7f5cb6000700 1 -- 172.21.15.90:0/4238648124 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5cac022070 con 0x7f5cb807edf0 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.754+0000 7f5cb6000700 1 --2- 172.21.15.90:0/4238648124 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f5ca003e470 0x7f5ca0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.755+0000 7f5cb6000700 1 -- 172.21.15.90:0/4238648124 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5cac053a80 con 0x7f5cb807edf0 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.755+0000 7f5cbcc00700 1 --2- 172.21.15.90:0/4238648124 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f5ca003e470 0x7f5ca0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.756+0000 7f5cbcc00700 1 --2- 172.21.15.90:0/4238648124 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f5ca003e470 0x7f5ca0040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4006fd0 tx=0x7f5ca4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:42.760+0000 7f5cb6000700 1 -- 172.21.15.90:0/4238648124 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5cac0172b0 con 0x7f5cb807edf0 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.016+0000 7f5cc0000700 1 -- 172.21.15.90:0/4238648124 --> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5cb8106050 con 0x7f5ca003e470 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.025+0000 7f5cb6000700 1 -- 172.21.15.90:0/4238648124 <== mgr.14120 v2:172.21.15.90:6800/2742579272 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5cb8106050 con 0x7f5ca003e470 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.028+0000 7f5cc0000700 1 -- 172.21.15.90:0/4238648124 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f5ca003e470 msgr2=0x7f5ca0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.028+0000 7f5cc0000700 1 --2- 172.21.15.90:0/4238648124 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f5ca003e470 0x7f5ca0040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4006fd0 tx=0x7f5ca4006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:43.753 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.028+0000 7f5cc0000700 1 -- 172.21.15.90:0/4238648124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 msgr2=0x7f5cb819ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:43.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.028+0000 7f5cc0000700 1 --2- 172.21.15.90:0/4238648124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb819ff60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5cac011670 tx=0x7f5cac011a30 comp rx=0 tx=0).stop 2024-05-27T07:30:43.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.028+0000 7f5cc0000700 1 -- 172.21.15.90:0/4238648124 shutdown_connections 2024-05-27T07:30:43.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.028+0000 7f5cc0000700 1 --2- 172.21.15.90:0/4238648124 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f5ca003e470 0x7f5ca0040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:43.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.028+0000 7f5cc0000700 1 --2- 172.21.15.90:0/4238648124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cb807edf0 0x7f5cb819ff60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:43.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.028+0000 7f5cc0000700 1 -- 172.21.15.90:0/4238648124 >> 172.21.15.90:0/4238648124 conn(0x7f5cb81050b0 msgr2=0x7f5cb8105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:43.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.029+0000 7f5cc0000700 1 -- 172.21.15.90:0/4238648124 shutdown_connections 2024-05-27T07:30:43.754 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:43.029+0000 7f5cc0000700 1 -- 172.21.15.90:0/4238648124 wait complete. 2024-05-27T07:30:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:44 smithi090 ceph-mon[31429]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:30:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:44 smithi090 ceph-mon[31429]: Saving service mgr spec with placement count:2 2024-05-27T07:30:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:45 smithi090 ceph-mon[31429]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:30:45.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:45 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3383528147' entity='client.admin' 2024-05-27T07:30:45.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.386+0000 7f1100a00700 1 Processor -- start 2024-05-27T07:30:45.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.387+0000 7f1100a00700 1 -- start start 2024-05-27T07:30:45.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.387+0000 7f1100a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc10c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:45.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.387+0000 7f1100a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc07ce80 con 0x7f10fc10a4e0 2024-05-27T07:30:45.650 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.387+0000 7f10f9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc10c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.387+0000 7f10f9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc10c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41408/0 (socket says 172.21.15.90:41408) 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.387+0000 7f10f9e00700 1 -- 172.21.15.90:0/2996321317 learned_addr learned my addr 172.21.15.90:0/2996321317 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.388+0000 7f10f9e00700 1 -- 172.21.15.90:0/2996321317 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc10ce40 con 0x7f10fc10a4e0 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.388+0000 7f10f9e00700 1 --2- 172.21.15.90:0/2996321317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc10c900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0098a0 tx=0x7f10ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d7017ba3eb62c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.388+0000 7f10f8a00700 1 -- 172.21.15.90:0/2996321317 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10ec004160 con 0x7f10fc10a4e0 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.388+0000 7f10f8a00700 1 -- 172.21.15.90:0/2996321317 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10ec004300 con 0x7f10fc10a4e0 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.388+0000 7f10f8a00700 1 -- 172.21.15.90:0/2996321317 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10ec011960 con 0x7f10fc10a4e0 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.389+0000 7f1100a00700 1 -- 172.21.15.90:0/2996321317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 msgr2=0x7f10fc10c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.389+0000 7f1100a00700 1 --2- 172.21.15.90:0/2996321317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc10c900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0098a0 tx=0x7f10ec009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.389+0000 7f1100a00700 1 -- 172.21.15.90:0/2996321317 shutdown_connections 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.389+0000 7f1100a00700 1 --2- 172.21.15.90:0/2996321317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc10c900 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.389+0000 7f1100a00700 1 -- 172.21.15.90:0/2996321317 >> 172.21.15.90:0/2996321317 conn(0x7f10fc103f60 msgr2=0x7f10fc1063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:45.651 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.389+0000 7f1100a00700 1 -- 172.21.15.90:0/2996321317 shutdown_connections 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.390+0000 7f1100a00700 1 -- 172.21.15.90:0/2996321317 wait complete. 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.390+0000 7f1100a00700 1 Processor -- start 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.391+0000 7f1100a00700 1 -- start start 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.391+0000 7f1100a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.391+0000 7f1100a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc10dde0 con 0x7f10fc10a4e0 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.391+0000 7f10f9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.391+0000 7f10f9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc19dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41412/0 (socket says 172.21.15.90:41412) 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.391+0000 7f10f9e00700 1 -- 172.21.15.90:0/3383528147 learned_addr learned my addr 172.21.15.90:0/3383528147 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.392+0000 7f10f9e00700 1 -- 172.21.15.90:0/3383528147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc19e280 con 0x7f10fc10a4e0 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.392+0000 7f10f9e00700 1 --2- 172.21.15.90:0/3383528147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc19dd40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10ec011670 tx=0x7f10ec011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.392+0000 7f10f2a00700 1 -- 172.21.15.90:0/3383528147 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10ec003aa0 con 0x7f10fc10a4e0 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.392+0000 7f10f2a00700 1 -- 172.21.15.90:0/3383528147 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10ec01b8d0 con 0x7f10fc10a4e0 2024-05-27T07:30:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.392+0000 7f10f2a00700 1 -- 172.21.15.90:0/3383528147 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10ec0197f0 con 0x7f10fc10a4e0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.392+0000 7f1100a00700 1 -- 172.21.15.90:0/3383528147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc19e510 con 0x7f10fc10a4e0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.392+0000 7f1100a00700 1 -- 172.21.15.90:0/3383528147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc1a1110 con 0x7f10fc10a4e0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.393+0000 7f10f2a00700 1 -- 172.21.15.90:0/3383528147 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f10ec019990 con 0x7f10fc10a4e0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.393+0000 7f1100a00700 1 -- 172.21.15.90:0/3383528147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10fc064e10 con 0x7f10fc10a4e0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.394+0000 7f10f2a00700 1 --2- 172.21.15.90:0/3383528147 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f10e003e470 0x7f10e0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.394+0000 7f10f2a00700 1 -- 172.21.15.90:0/3383528147 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f10ec053f40 con 0x7f10fc10a4e0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.394+0000 7f10f9400700 1 --2- 172.21.15.90:0/3383528147 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f10e003e470 0x7f10e0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.395+0000 7f10f9400700 1 --2- 172.21.15.90:0/3383528147 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f10e003e470 0x7f10e0040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f10e8006fd0 tx=0x7f10e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.399+0000 7f10f2a00700 1 -- 172.21.15.90:0/3383528147 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10ec01f030 con 0x7f10fc10a4e0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.629+0000 7f1100a00700 1 -- 172.21.15.90:0/3383528147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f10fc1a1420 con 0x7f10fc10a4e0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.638+0000 7f10f2a00700 1 -- 172.21.15.90:0/3383528147 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f10ec053890 con 0x7f10fc10a4e0 2024-05-27T07:30:45.653 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.642+0000 7f1100a00700 1 -- 172.21.15.90:0/3383528147 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f10e003e470 msgr2=0x7f10e0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:45.654 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.642+0000 7f1100a00700 1 --2- 172.21.15.90:0/3383528147 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f10e003e470 0x7f10e0040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f10e8006fd0 tx=0x7f10e8006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:45.654 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.642+0000 7f1100a00700 1 -- 172.21.15.90:0/3383528147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 msgr2=0x7f10fc19dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:45.654 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.642+0000 7f1100a00700 1 --2- 172.21.15.90:0/3383528147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc19dd40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10ec011670 tx=0x7f10ec011a30 comp rx=0 tx=0).stop 2024-05-27T07:30:45.654 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.642+0000 7f1100a00700 1 -- 172.21.15.90:0/3383528147 shutdown_connections 2024-05-27T07:30:45.654 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.642+0000 7f1100a00700 1 --2- 172.21.15.90:0/3383528147 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f10e003e470 0x7f10e0040980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:45.654 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.642+0000 7f1100a00700 1 --2- 172.21.15.90:0/3383528147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10fc10a4e0 0x7f10fc19dd40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:45.654 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.642+0000 7f1100a00700 1 -- 172.21.15.90:0/3383528147 >> 172.21.15.90:0/3383528147 conn(0x7f10fc103f60 msgr2=0x7f10fc104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:45.654 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.643+0000 7f1100a00700 1 -- 172.21.15.90:0/3383528147 shutdown_connections 2024-05-27T07:30:45.654 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:44.643+0000 7f1100a00700 1 -- 172.21.15.90:0/3383528147 wait complete. 2024-05-27T07:30:46.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:46 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:46.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:46 smithi090 ceph-mon[31429]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:30:47.244 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.312+0000 7f5847200700 1 Processor -- start 2024-05-27T07:30:47.244 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.312+0000 7f5847200700 1 -- start start 2024-05-27T07:30:47.244 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.312+0000 7f5847200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f5848110530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:47.244 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.312+0000 7f5847200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5848110b40 con 0x7f58481100e0 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.313+0000 7f5845e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f5848110530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.313+0000 7f5845e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f5848110530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41428/0 (socket says 172.21.15.90:41428) 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.313+0000 7f5845e00700 1 -- 172.21.15.90:0/19491122 learned_addr learned my addr 172.21.15.90:0/19491122 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.313+0000 7f5845e00700 1 -- 172.21.15.90:0/19491122 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5848110fb0 con 0x7f58481100e0 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.314+0000 7f5845e00700 1 --2- 172.21.15.90:0/19491122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f5848110530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f58380098a0 tx=0x7f5838009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b189e1e4e08624c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.314+0000 7f5844a00700 1 -- 172.21.15.90:0/19491122 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5838004160 con 0x7f58481100e0 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.314+0000 7f5844a00700 1 -- 172.21.15.90:0/19491122 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5838004300 con 0x7f58481100e0 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.314+0000 7f5844a00700 1 -- 172.21.15.90:0/19491122 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5838011960 con 0x7f58481100e0 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.315+0000 7f5847200700 1 -- 172.21.15.90:0/19491122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 msgr2=0x7f5848110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.315+0000 7f5847200700 1 --2- 172.21.15.90:0/19491122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f5848110530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f58380098a0 tx=0x7f5838009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:47.245 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.315+0000 7f5847200700 1 -- 172.21.15.90:0/19491122 shutdown_connections 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.315+0000 7f5847200700 1 --2- 172.21.15.90:0/19491122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f5848110530 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.315+0000 7f5847200700 1 -- 172.21.15.90:0/19491122 >> 172.21.15.90:0/19491122 conn(0x7f584807efd0 msgr2=0x7f58480813f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.315+0000 7f5847200700 1 -- 172.21.15.90:0/19491122 shutdown_connections 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.315+0000 7f5847200700 1 -- 172.21.15.90:0/19491122 wait complete. 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.316+0000 7f5847200700 1 Processor -- start 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.316+0000 7f5847200700 1 -- start start 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.316+0000 7f5847200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f58481a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.316+0000 7f5847200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f58481143f0 con 0x7f58481100e0 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.317+0000 7f5845e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f58481a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.317+0000 7f5845e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f58481a4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41436/0 (socket says 172.21.15.90:41436) 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.317+0000 7f5845e00700 1 -- 172.21.15.90:0/1638846521 learned_addr learned my addr 172.21.15.90:0/1638846521 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.317+0000 7f5845e00700 1 -- 172.21.15.90:0/1638846521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58481a46a0 con 0x7f58481100e0 2024-05-27T07:30:47.246 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.317+0000 7f5845e00700 1 --2- 172.21.15.90:0/1638846521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f58481a4160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5838011670 tx=0x7f5838011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.318+0000 7f583ea00700 1 -- 172.21.15.90:0/1638846521 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5838003a60 con 0x7f58481100e0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.318+0000 7f583ea00700 1 -- 172.21.15.90:0/1638846521 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f583801b8d0 con 0x7f58481100e0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.318+0000 7f5847200700 1 -- 172.21.15.90:0/1638846521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58481a4930 con 0x7f58481100e0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.318+0000 7f583ea00700 1 -- 172.21.15.90:0/1638846521 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5838023750 con 0x7f58481100e0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.318+0000 7f5847200700 1 -- 172.21.15.90:0/1638846521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58481a7540 con 0x7f58481100e0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.319+0000 7f5847200700 1 -- 172.21.15.90:0/1638846521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5848064e10 con 0x7f58481100e0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.319+0000 7f583ea00700 1 -- 172.21.15.90:0/1638846521 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5838023a00 con 0x7f58481100e0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.320+0000 7f583ea00700 1 --2- 172.21.15.90:0/1638846521 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f582c03e470 0x7f582c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.320+0000 7f583ea00700 1 -- 172.21.15.90:0/1638846521 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5838054250 con 0x7f58481100e0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.320+0000 7f5845400700 1 --2- 172.21.15.90:0/1638846521 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f582c03e470 0x7f582c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.321+0000 7f5845400700 1 --2- 172.21.15.90:0/1638846521 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f582c03e470 0x7f582c040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5834006fd0 tx=0x7f5834006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.325+0000 7f583ea00700 1 -- 172.21.15.90:0/1638846521 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f583801f030 con 0x7f58481100e0 2024-05-27T07:30:47.247 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.572+0000 7f5847200700 1 -- 172.21.15.90:0/1638846521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f58481a7710 con 0x7f58481100e0 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.579+0000 7f583ea00700 1 -- 172.21.15.90:0/1638846521 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f5838019e20 con 0x7f58481100e0 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.583+0000 7f5847200700 1 -- 172.21.15.90:0/1638846521 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f582c03e470 msgr2=0x7f582c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.583+0000 7f5847200700 1 --2- 172.21.15.90:0/1638846521 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f582c03e470 0x7f582c040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5834006fd0 tx=0x7f5834006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.583+0000 7f5847200700 1 -- 172.21.15.90:0/1638846521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 msgr2=0x7f58481a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.583+0000 7f5847200700 1 --2- 172.21.15.90:0/1638846521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f58481a4160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5838011670 tx=0x7f5838011a30 comp rx=0 tx=0).stop 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.583+0000 7f5847200700 1 -- 172.21.15.90:0/1638846521 shutdown_connections 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.583+0000 7f5847200700 1 --2- 172.21.15.90:0/1638846521 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f582c03e470 0x7f582c040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.583+0000 7f5847200700 1 --2- 172.21.15.90:0/1638846521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58481100e0 0x7f58481a4160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.583+0000 7f5847200700 1 -- 172.21.15.90:0/1638846521 >> 172.21.15.90:0/1638846521 conn(0x7f584807efd0 msgr2=0x7f58480807b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.583+0000 7f5847200700 1 -- 172.21.15.90:0/1638846521 shutdown_connections 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:46.583+0000 7f5847200700 1 -- 172.21.15.90:0/1638846521 wait complete. 2024-05-27T07:30:47.248 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling the dashboard module... 2024-05-27T07:30:47.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:47 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1638846521' entity='client.admin' 2024-05-27T07:30:48.760 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:48 smithi090 ceph-mon[31429]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:30:48.760 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:48 smithi090 ceph-mon[31429]: from='mgr.14120 172.21.15.90:0/1645165104' entity='mgr.x' 2024-05-27T07:30:48.760 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:48 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1182724816' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-27T07:30:49.013 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:48 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: ignoring --setuser ceph since I am not root 2024-05-27T07:30:49.014 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:48 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: ignoring --setgroup ceph since I am not root 2024-05-27T07:30:49.014 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:49 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:49.020+0000 7f1ccc96c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:30:49.524 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:49 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:49.275+0000 7f1ccc96c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:30:49.524 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:49 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:49.409+0000 7f1ccc96c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:30:49.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:49 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1182724816' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-27T07:30:49.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:49 smithi090 ceph-mon[31429]: mgrmap e9: x(active, since 25s) 2024-05-27T07:30:50.204 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:49 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:49.818+0000 7f1ccc96c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:30:50.204 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:49 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:49.935+0000 7f1ccc96c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:30:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.927+0000 7f79a3e00700 1 Processor -- start 2024-05-27T07:30:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.927+0000 7f79a3e00700 1 -- start start 2024-05-27T07:30:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.928+0000 7f79a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a407d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.928+0000 7f79a3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f79a407d780 con 0x7f79a407ee10 2024-05-27T07:30:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.928+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a407d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.928+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a407d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41440/0 (socket says 172.21.15.90:41440) 2024-05-27T07:30:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.928+0000 7f79a2a00700 1 -- 172.21.15.90:0/2389959696 learned_addr learned my addr 172.21.15.90:0/2389959696 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:50.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.929+0000 7f79a2a00700 1 -- 172.21.15.90:0/2389959696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a407d900 con 0x7f79a407ee10 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.929+0000 7f79a2a00700 1 --2- 172.21.15.90:0/2389959696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a407d240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f79940098a0 tx=0x7f7994009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4b8d918b491358 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.929+0000 7f79a2000700 1 -- 172.21.15.90:0/2389959696 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7994004160 con 0x7f79a407ee10 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.929+0000 7f79a2000700 1 -- 172.21.15.90:0/2389959696 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7994004300 con 0x7f79a407ee10 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.929+0000 7f79a2000700 1 -- 172.21.15.90:0/2389959696 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7994011960 con 0x7f79a407ee10 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.930+0000 7f79a3e00700 1 -- 172.21.15.90:0/2389959696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 msgr2=0x7f79a407d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.930+0000 7f79a3e00700 1 --2- 172.21.15.90:0/2389959696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a407d240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f79940098a0 tx=0x7f7994009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.930+0000 7f79a3e00700 1 -- 172.21.15.90:0/2389959696 shutdown_connections 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.930+0000 7f79a3e00700 1 --2- 172.21.15.90:0/2389959696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a407d240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.930+0000 7f79a3e00700 1 -- 172.21.15.90:0/2389959696 >> 172.21.15.90:0/2389959696 conn(0x7f79a4104f90 msgr2=0x7f79a41073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.931+0000 7f79a3e00700 1 -- 172.21.15.90:0/2389959696 shutdown_connections 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.931+0000 7f79a3e00700 1 -- 172.21.15.90:0/2389959696 wait complete. 2024-05-27T07:30:50.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.932+0000 7f79a3e00700 1 Processor -- start 2024-05-27T07:30:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.932+0000 7f79a3e00700 1 -- start start 2024-05-27T07:30:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.932+0000 7f79a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.932+0000 7f79a3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f79a410fe50 con 0x7f79a407ee10 2024-05-27T07:30:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.932+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:50.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.932+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41452/0 (socket says 172.21.15.90:41452) 2024-05-27T07:30:50.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.932+0000 7f79a2a00700 1 -- 172.21.15.90:0/1182724816 learned_addr learned my addr 172.21.15.90:0/1182724816 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.933+0000 7f79a2a00700 1 -- 172.21.15.90:0/1182724816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a41a0350 con 0x7f79a407ee10 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.933+0000 7f79a2a00700 1 --2- 172.21.15.90:0/1182724816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a419fe10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7994011670 tx=0x7f7994011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.933+0000 7f799b400700 1 -- 172.21.15.90:0/1182724816 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7994003a60 con 0x7f79a407ee10 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.933+0000 7f79a3e00700 1 -- 172.21.15.90:0/1182724816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a41a05e0 con 0x7f79a407ee10 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.933+0000 7f799b400700 1 -- 172.21.15.90:0/1182724816 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f799401b8d0 con 0x7f79a407ee10 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.934+0000 7f79a3e00700 1 -- 172.21.15.90:0/1182724816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a41a31f0 con 0x7f79a407ee10 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.934+0000 7f799b400700 1 -- 172.21.15.90:0/1182724816 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7994004300 con 0x7f79a407ee10 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.935+0000 7f799b400700 1 -- 172.21.15.90:0/1182724816 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7994023650 con 0x7f79a407ee10 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.935+0000 7f79a3e00700 1 -- 172.21.15.90:0/1182724816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a4064e10 con 0x7f79a407ee10 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.935+0000 7f799b400700 1 --2- 172.21.15.90:0/1182724816 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f798803e610 0x7f7988040b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.936+0000 7f799b400700 1 -- 172.21.15.90:0/1182724816 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7994053fb0 con 0x7f79a407ee10 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.936+0000 7f799be00700 1 --2- 172.21.15.90:0/1182724816 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f798803e610 0x7f7988040b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:50.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.937+0000 7f799be00700 1 --2- 172.21.15.90:0/1182724816 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f798803e610 0x7f7988040b20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f798c006fd0 tx=0x7f798c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:47.942+0000 7f799b400700 1 -- 172.21.15.90:0/1182724816 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f799401f030 con 0x7f79a407ee10 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.237+0000 7f79a3e00700 1 -- 172.21.15.90:0/1182724816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f79a41a33c0 con 0x7f79a407ee10 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.623+0000 7f799b400700 1 -- 172.21.15.90:0/1182724816 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f799400b3e0 con 0x7f79a407ee10 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.628+0000 7f799b400700 1 -- 172.21.15.90:0/1182724816 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f799402c910 con 0x7f79a407ee10 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.629+0000 7f79a3e00700 1 -- 172.21.15.90:0/1182724816 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f798803e610 msgr2=0x7f7988040b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.629+0000 7f79a3e00700 1 --2- 172.21.15.90:0/1182724816 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f798803e610 0x7f7988040b20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f798c006fd0 tx=0x7f798c006e40 comp rx=0 tx=0).stop 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.629+0000 7f79a3e00700 1 -- 172.21.15.90:0/1182724816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 msgr2=0x7f79a419fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.629+0000 7f79a3e00700 1 --2- 172.21.15.90:0/1182724816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a419fe10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7994011670 tx=0x7f7994011a30 comp rx=0 tx=0).stop 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.630+0000 7f79a3e00700 1 -- 172.21.15.90:0/1182724816 shutdown_connections 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.630+0000 7f79a3e00700 1 --2- 172.21.15.90:0/1182724816 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f798803e610 0x7f7988040b20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.630+0000 7f79a3e00700 1 --2- 172.21.15.90:0/1182724816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a407ee10 0x7f79a419fe10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.630+0000 7f79a3e00700 1 -- 172.21.15.90:0/1182724816 >> 172.21.15.90:0/1182724816 conn(0x7f79a4104f90 msgr2=0x7f79a41068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.631+0000 7f79a3e00700 1 -- 172.21.15.90:0/1182724816 shutdown_connections 2024-05-27T07:30:50.317 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:48.631+0000 7f79a3e00700 1 -- 172.21.15.90:0/1182724816 wait complete. 2024-05-27T07:30:51.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:51 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3801901565' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T07:30:51.941 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:51 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:51.559+0000 7f1ccc96c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:30:51.941 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:51 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:51.790+0000 7f1ccc96c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:30:51.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:30:51.951 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-27T07:30:51.951 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T07:30:51.951 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-27T07:30:51.951 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T07:30:51.951 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:30:51.951 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.940+0000 7fea93000700 1 Processor -- start 2024-05-27T07:30:51.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.941+0000 7fea93000700 1 -- start start 2024-05-27T07:30:51.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.941+0000 7fea93000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c10c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:51.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.941+0000 7fea93000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c10cc10 con 0x7fea8c07d560 2024-05-27T07:30:51.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.942+0000 7fea8be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c10c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:51.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.942+0000 7fea8be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c10c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47882/0 (socket says 172.21.15.90:47882) 2024-05-27T07:30:51.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.942+0000 7fea8be00700 1 -- 172.21.15.90:0/1251754062 learned_addr learned my addr 172.21.15.90:0/1251754062 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:51.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.942+0000 7fea8be00700 1 -- 172.21.15.90:0/1251754062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c10cd90 con 0x7fea8c07d560 2024-05-27T07:30:51.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.943+0000 7fea8be00700 1 --2- 172.21.15.90:0/1251754062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c10c600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0098a0 tx=0x7fea7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a37649decb8754 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:51.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.943+0000 7fea8aa00700 1 -- 172.21.15.90:0/1251754062 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea7c004160 con 0x7fea8c07d560 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.943+0000 7fea8aa00700 1 -- 172.21.15.90:0/1251754062 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea7c004300 con 0x7fea8c07d560 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.943+0000 7fea8aa00700 1 -- 172.21.15.90:0/1251754062 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea7c011960 con 0x7fea8c07d560 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.944+0000 7fea93000700 1 -- 172.21.15.90:0/1251754062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 msgr2=0x7fea8c10c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.944+0000 7fea93000700 1 --2- 172.21.15.90:0/1251754062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c10c600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0098a0 tx=0x7fea7c009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.944+0000 7fea93000700 1 -- 172.21.15.90:0/1251754062 shutdown_connections 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.944+0000 7fea93000700 1 --2- 172.21.15.90:0/1251754062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c10c600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.944+0000 7fea93000700 1 -- 172.21.15.90:0/1251754062 >> 172.21.15.90:0/1251754062 conn(0x7fea8c103f90 msgr2=0x7fea8c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.944+0000 7fea93000700 1 -- 172.21.15.90:0/1251754062 shutdown_connections 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.944+0000 7fea93000700 1 -- 172.21.15.90:0/1251754062 wait complete. 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.945+0000 7fea93000700 1 Processor -- start 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.945+0000 7fea93000700 1 -- start start 2024-05-27T07:30:51.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.946+0000 7fea93000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c19ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.946+0000 7fea93000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c10ddf0 con 0x7fea8c07d560 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.946+0000 7fea8be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c19ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.946+0000 7fea8be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c19ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47888/0 (socket says 172.21.15.90:47888) 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.946+0000 7fea8be00700 1 -- 172.21.15.90:0/3801901565 learned_addr learned my addr 172.21.15.90:0/3801901565 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.946+0000 7fea8be00700 1 -- 172.21.15.90:0/3801901565 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c1a04b0 con 0x7fea8c07d560 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.947+0000 7fea8be00700 1 --2- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c19ff70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fea7c011670 tx=0x7fea7c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.947+0000 7fea88c00700 1 -- 172.21.15.90:0/3801901565 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea7c01f070 con 0x7fea8c07d560 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.947+0000 7fea88c00700 1 -- 172.21.15.90:0/3801901565 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea7c003b20 con 0x7fea8c07d560 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.947+0000 7fea93000700 1 -- 172.21.15.90:0/3801901565 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8c1a0740 con 0x7fea8c07d560 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.947+0000 7fea88c00700 1 -- 172.21.15.90:0/3801901565 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea7c019580 con 0x7fea8c07d560 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.947+0000 7fea93000700 1 -- 172.21.15.90:0/3801901565 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8c1a3350 con 0x7fea8c07d560 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.948+0000 7fea93000700 1 -- 172.21.15.90:0/3801901565 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8c064e10 con 0x7fea8c07d560 2024-05-27T07:30:51.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.949+0000 7fea88c00700 1 -- 172.21.15.90:0/3801901565 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fea7c022070 con 0x7fea8c07d560 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.949+0000 7fea88c00700 1 --2- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fea7003e4c0 0x7fea700409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.949+0000 7fea88c00700 1 -- 172.21.15.90:0/3801901565 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea7c053ac0 con 0x7fea8c07d560 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.949+0000 7fea8b400700 1 -- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fea7003e4c0 msgr2=0x7fea700409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2742579272 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.949+0000 7fea8b400700 1 --2- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fea7003e4c0 0x7fea700409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:50.957+0000 7fea88c00700 1 -- 172.21.15.90:0/3801901565 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea7c01ba80 con 0x7fea8c07d560 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.150+0000 7fea8b400700 1 -- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fea7003e4c0 msgr2=0x7fea700409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2742579272 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.150+0000 7fea8b400700 1 --2- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fea7003e4c0 0x7fea700409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.234+0000 7fea93000700 1 -- 172.21.15.90:0/3801901565 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fea8c0643c0 con 0x7fea8c07d560 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.235+0000 7fea88c00700 1 -- 172.21.15.90:0/3801901565 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fea7c02ca40 con 0x7fea8c07d560 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.238+0000 7fea93000700 1 -- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fea7003e4c0 msgr2=0x7fea700409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.238+0000 7fea93000700 1 --2- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fea7003e4c0 0x7fea700409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.238+0000 7fea93000700 1 -- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 msgr2=0x7fea8c19ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:51.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.238+0000 7fea93000700 1 --2- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c19ff70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fea7c011670 tx=0x7fea7c011a30 comp rx=0 tx=0).stop 2024-05-27T07:30:51.961 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.238+0000 7fea93000700 1 -- 172.21.15.90:0/3801901565 shutdown_connections 2024-05-27T07:30:51.961 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.238+0000 7fea93000700 1 --2- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7fea7003e4c0 0x7fea700409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:51.961 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.238+0000 7fea93000700 1 --2- 172.21.15.90:0/3801901565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea8c07d560 0x7fea8c19ff70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:51.961 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.238+0000 7fea93000700 1 -- 172.21.15.90:0/3801901565 >> 172.21.15.90:0/3801901565 conn(0x7fea8c103f90 msgr2=0x7fea8c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:51.961 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.238+0000 7fea93000700 1 -- 172.21.15.90:0/3801901565 shutdown_connections 2024-05-27T07:30:51.961 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:51.239+0000 7fea93000700 1 -- 172.21.15.90:0/3801901565 wait complete. 2024-05-27T07:30:51.962 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-05-27T07:30:51.962 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 9... 2024-05-27T07:30:52.204 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:52 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:52.013+0000 7f1ccc96c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:30:52.544 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:52 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:52.421+0000 7f1ccc96c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:30:52.801 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:52 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:52.552+0000 7f1ccc96c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:30:53.060 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:52 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:52.809+0000 7f1ccc96c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:30:53.060 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:53 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:53.068+0000 7f1ccc96c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:30:54.138 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:53 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:53.855+0000 7f1ccc96c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:30:54.138 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:54 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:54.146+0000 7f1ccc96c200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:30:54.829 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:54 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:54.710+0000 7f1ccc96c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:30:55.136 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:54 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:54.846+0000 7f1ccc96c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:30:55.454 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:55 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:55.338+0000 7f1ccc96c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:30:55.933 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:55 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:55.810+0000 7f1ccc96c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:30:55.933 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:55 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:55.941+0000 7f1ccc96c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:30:56.575 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:56.271+0000 7f1ccc96c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:30:56.575 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:56.451+0000 7f1ccc96c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:30:56.575 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:56.582+0000 7f1ccc96c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:30:57.455 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:56.982+0000 7f1ccc96c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:30:57.456 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:57.107+0000 7f1ccc96c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:30:58.143 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:57.861+0000 7f1ccc96c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:30:58.143 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:30:58 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:30:58.151+0000 7f1ccc96c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:30:58.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:58 smithi090 ceph-mon[31429]: Active manager daemon x restarted 2024-05-27T07:30:58.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:58 smithi090 ceph-mon[31429]: Activating manager daemon x 2024-05-27T07:30:58.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:58 smithi090 ceph-mon[31429]: osdmap e3: 0 total, 0 up, 0 in 2024-05-27T07:30:58.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:58 smithi090 ceph-mon[31429]: mgrmap e10: x(active, starting, since 0.0136327s) 2024-05-27T07:30:58.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:30:58.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:30:58.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:30:58.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:30:58.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:30:58.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:58 smithi090 ceph-mon[31429]: Manager daemon x is now available 2024-05-27T07:30:59.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:30:59.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:30:59.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:30:59.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:59 smithi090 ceph-mon[31429]: [27/May/2024:07:30:58] ENGINE Bus STARTING 2024-05-27T07:30:59.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:30:59 smithi090 ceph-mon[31429]: mgrmap e11: x(active, since 1.01009s) 2024-05-27T07:30:59.824 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-05-27T07:30:59.825 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-27T07:30:59.825 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T07:30:59.825 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-05-27T07:30:59.825 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.584+0000 7f4e61e00700 1 Processor -- start 2024-05-27T07:30:59.825 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.584+0000 7f4e61e00700 1 -- start start 2024-05-27T07:30:59.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.584+0000 7f4e61e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c07d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:59.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.584+0000 7f4e61e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c07d850 con 0x7f4e5c07ee50 2024-05-27T07:30:59.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.585+0000 7f4e5aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c07d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:59.833 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.585+0000 7f4e5aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c07d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47890/0 (socket says 172.21.15.90:47890) 2024-05-27T07:30:59.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.585+0000 7f4e5aa00700 1 -- 172.21.15.90:0/2836462150 learned_addr learned my addr 172.21.15.90:0/2836462150 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:59.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.585+0000 7f4e5aa00700 1 -- 172.21.15.90:0/2836462150 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5c07d9d0 con 0x7f4e5c07ee50 2024-05-27T07:30:59.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.585+0000 7f4e5aa00700 1 --2- 172.21.15.90:0/2836462150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c07d280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0098a0 tx=0x7f4e4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298aca04077f1f11 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:59.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.586+0000 7f4e59600700 1 -- 172.21.15.90:0/2836462150 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e4c004160 con 0x7f4e5c07ee50 2024-05-27T07:30:59.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.586+0000 7f4e59600700 1 -- 172.21.15.90:0/2836462150 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e4c004300 con 0x7f4e5c07ee50 2024-05-27T07:30:59.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.586+0000 7f4e59600700 1 -- 172.21.15.90:0/2836462150 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e4c011960 con 0x7f4e5c07ee50 2024-05-27T07:30:59.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.587+0000 7f4e61e00700 1 -- 172.21.15.90:0/2836462150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 msgr2=0x7f4e5c07d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:59.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.587+0000 7f4e61e00700 1 --2- 172.21.15.90:0/2836462150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c07d280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0098a0 tx=0x7f4e4c009bb0 comp rx=0 tx=0).stop 2024-05-27T07:30:59.834 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.587+0000 7f4e61e00700 1 -- 172.21.15.90:0/2836462150 shutdown_connections 2024-05-27T07:30:59.835 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.587+0000 7f4e61e00700 1 --2- 172.21.15.90:0/2836462150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c07d280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:59.836 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.587+0000 7f4e61e00700 1 -- 172.21.15.90:0/2836462150 >> 172.21.15.90:0/2836462150 conn(0x7f4e5c1050c0 msgr2=0x7f4e5c107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:59.836 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.587+0000 7f4e61e00700 1 -- 172.21.15.90:0/2836462150 shutdown_connections 2024-05-27T07:30:59.836 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.587+0000 7f4e61e00700 1 -- 172.21.15.90:0/2836462150 wait complete. 2024-05-27T07:30:59.836 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.588+0000 7f4e61e00700 1 Processor -- start 2024-05-27T07:30:59.836 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.588+0000 7f4e61e00700 1 -- start start 2024-05-27T07:30:59.836 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.589+0000 7f4e61e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c1a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:59.836 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.589+0000 7f4e61e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5c110070 con 0x7f4e5c07ee50 2024-05-27T07:30:59.836 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.589+0000 7f4e5aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c1a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.589+0000 7f4e5aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c1a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47892/0 (socket says 172.21.15.90:47892) 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.589+0000 7f4e5aa00700 1 -- 172.21.15.90:0/474466369 learned_addr learned my addr 172.21.15.90:0/474466369 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.589+0000 7f4e5aa00700 1 -- 172.21.15.90:0/474466369 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5c1a05b0 con 0x7f4e5c07ee50 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.589+0000 7f4e5aa00700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c1a0070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c011670 tx=0x7f4e4c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.590+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e4c003a60 con 0x7f4e5c07ee50 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.590+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e4c01b8d0 con 0x7f4e5c07ee50 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.590+0000 7f4e61e00700 1 -- 172.21.15.90:0/474466369 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5c1a0840 con 0x7f4e5c07ee50 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.590+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e4c023790 con 0x7f4e5c07ee50 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.590+0000 7f4e61e00700 1 -- 172.21.15.90:0/474466369 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5c1a3450 con 0x7f4e5c07ee50 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.591+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f4e4c023a90 con 0x7f4e5c07ee50 2024-05-27T07:30:59.837 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.592+0000 7f4e60a00700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 0x7f4e400409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:59.838 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.592+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 --> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4e400410e0 con 0x7f4e4003e4c0 2024-05-27T07:30:59.838 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.592+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4e4c0540a0 con 0x7f4e5c07ee50 2024-05-27T07:30:59.838 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.592+0000 7f4e5a000700 1 -- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 msgr2=0x7f4e400409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2742579272 2024-05-27T07:30:59.838 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.592+0000 7f4e5a000700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 0x7f4e400409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:30:59.838 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.793+0000 7f4e5a000700 1 -- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 msgr2=0x7f4e400409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2742579272 2024-05-27T07:30:59.838 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:52.793+0000 7f4e5a000700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 0x7f4e400409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:30:59.838 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:53.194+0000 7f4e5a000700 1 -- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 msgr2=0x7f4e400409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2742579272 2024-05-27T07:30:59.838 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:53.194+0000 7f4e5a000700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 0x7f4e400409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T07:30:59.838 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:53.995+0000 7f4e5a000700 1 -- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 msgr2=0x7f4e400409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2742579272 2024-05-27T07:30:59.839 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:53.995+0000 7f4e5a000700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 0x7f4e400409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T07:30:59.839 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:55.597+0000 7f4e5a000700 1 -- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 msgr2=0x7f4e400409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2742579272 2024-05-27T07:30:59.839 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:55.597+0000 7f4e5a000700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 0x7f4e400409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T07:30:59.839 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:58.162+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f4e4c02c3f0 con 0x7f4e5c07ee50 2024-05-27T07:30:59.839 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:58.163+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 msgr2=0x7f4e400409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:30:59.839 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:58.163+0000 7f4e60a00700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 0x7f4e400409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:59.839 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.161+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f4e4c055360 con 0x7f4e5c07ee50 2024-05-27T07:30:59.839 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.162+0000 7f4e60a00700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e4c053c30 0x7f4e40043fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:30:59.839 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.162+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4e4c00af30 con 0x7f4e4c053c30 2024-05-27T07:30:59.839 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.162+0000 7f4e5a000700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e4c053c30 0x7f4e40043fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:30:59.840 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.163+0000 7f4e5a000700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e4c053c30 0x7f4e40043fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4e48003a10 tx=0x7f4e480092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:30:59.840 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.166+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f4e4c00af30 con 0x7f4e4c053c30 2024-05-27T07:30:59.840 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.180+0000 7f4e61e00700 1 -- 172.21.15.90:0/474466369 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4e5c064e10 con 0x7f4e4c053c30 2024-05-27T07:30:59.840 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.181+0000 7f4e60a00700 1 -- 172.21.15.90:0/474466369 <== mgr.14152 v2:172.21.15.90:6800/2854405112 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f4e5c064e10 con 0x7f4e4c053c30 2024-05-27T07:30:59.840 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.181+0000 7f4e61e00700 1 -- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e4c053c30 msgr2=0x7f4e40043fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:59.840 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.181+0000 7f4e61e00700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e4c053c30 0x7f4e40043fc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4e48003a10 tx=0x7f4e480092b0 comp rx=0 tx=0).stop 2024-05-27T07:30:59.840 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.182+0000 7f4e61e00700 1 -- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 msgr2=0x7f4e5c1a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:30:59.840 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.182+0000 7f4e61e00700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c1a0070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c011670 tx=0x7f4e4c011a30 comp rx=0 tx=0).stop 2024-05-27T07:30:59.840 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.182+0000 7f4e61e00700 1 -- 172.21.15.90:0/474466369 shutdown_connections 2024-05-27T07:30:59.840 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.182+0000 7f4e61e00700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e4c053c30 0x7f4e40043fc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:59.841 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.182+0000 7f4e61e00700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e5c07ee50 0x7f4e5c1a0070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:59.841 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.182+0000 7f4e61e00700 1 --2- 172.21.15.90:0/474466369 >> [v2:172.21.15.90:6800/2742579272,v1:172.21.15.90:6801/2742579272] conn(0x7f4e4003e4c0 0x7f4e400409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:30:59.841 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.182+0000 7f4e61e00700 1 -- 172.21.15.90:0/474466369 >> 172.21.15.90:0/474466369 conn(0x7f4e5c1050c0 msgr2=0x7f4e5c105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:30:59.841 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.182+0000 7f4e61e00700 1 -- 172.21.15.90:0/474466369 shutdown_connections 2024-05-27T07:30:59.841 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:30:59.182+0000 7f4e61e00700 1 -- 172.21.15.90:0/474466369 wait complete. 2024-05-27T07:30:59.841 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 9 is available 2024-05-27T07:30:59.841 INFO:teuthology.orchestra.run.smithi090.stdout:Generating a dashboard self-signed certificate... 2024-05-27T07:31:00.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:00 smithi090 ceph-mon[31429]: [27/May/2024:07:30:58] ENGINE Serving on https://172.21.15.90:7150 2024-05-27T07:31:00.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:00 smithi090 ceph-mon[31429]: [27/May/2024:07:30:59] ENGINE Serving on http://172.21.15.90:8765 2024-05-27T07:31:00.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:00 smithi090 ceph-mon[31429]: [27/May/2024:07:30:59] ENGINE Bus STARTED 2024-05-27T07:31:00.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:00 smithi090 ceph-mon[31429]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T07:31:00.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:00 smithi090 ceph-mon[31429]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T07:31:01.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:01 smithi090 ceph-mon[31429]: mgrmap e12: x(active, since 2s) 2024-05-27T07:31:02.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-27T07:31:02.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.091+0000 7f88ede00700 1 Processor -- start 2024-05-27T07:31:02.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.091+0000 7f88ede00700 1 -- start start 2024-05-27T07:31:02.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.091+0000 7f88ede00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e8109e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:02.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.091+0000 7f88ede00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f88e810a430 con 0x7f88e8109a30 2024-05-27T07:31:02.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.092+0000 7f88e6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e8109e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:02.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.092+0000 7f88e6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e8109e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39668/0 (socket says 172.21.15.90:39668) 2024-05-27T07:31:02.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.092+0000 7f88e6a00700 1 -- 172.21.15.90:0/609335876 learned_addr learned my addr 172.21.15.90:0/609335876 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:02.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.092+0000 7f88e6a00700 1 -- 172.21.15.90:0/609335876 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e810ac00 con 0x7f88e8109a30 2024-05-27T07:31:02.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.093+0000 7f88e6a00700 1 --2- 172.21.15.90:0/609335876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e8109e40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88d80098a0 tx=0x7f88d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de8409a5e3312ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:02.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.093+0000 7f88e5600700 1 -- 172.21.15.90:0/609335876 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88d8004160 con 0x7f88e8109a30 2024-05-27T07:31:02.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.093+0000 7f88e5600700 1 -- 172.21.15.90:0/609335876 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88d8004300 con 0x7f88e8109a30 2024-05-27T07:31:02.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.093+0000 7f88e5600700 1 -- 172.21.15.90:0/609335876 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88d8011960 con 0x7f88e8109a30 2024-05-27T07:31:02.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.094+0000 7f88ede00700 1 -- 172.21.15.90:0/609335876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 msgr2=0x7f88e8109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:02.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.094+0000 7f88ede00700 1 --2- 172.21.15.90:0/609335876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e8109e40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88d80098a0 tx=0x7f88d8009bb0 comp rx=0 tx=0).stop 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.094+0000 7f88ede00700 1 -- 172.21.15.90:0/609335876 shutdown_connections 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.094+0000 7f88ede00700 1 --2- 172.21.15.90:0/609335876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e8109e40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.094+0000 7f88ede00700 1 -- 172.21.15.90:0/609335876 >> 172.21.15.90:0/609335876 conn(0x7f88e8105090 msgr2=0x7f88e8107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.094+0000 7f88ede00700 1 -- 172.21.15.90:0/609335876 shutdown_connections 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.094+0000 7f88ede00700 1 -- 172.21.15.90:0/609335876 wait complete. 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.095+0000 7f88ede00700 1 Processor -- start 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.095+0000 7f88ede00700 1 -- start start 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.096+0000 7f88ede00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e819dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.096+0000 7f88ede00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f88e810dfb0 con 0x7f88e8109a30 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.096+0000 7f88e6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e819dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:02.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.096+0000 7f88e6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e819dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39678/0 (socket says 172.21.15.90:39678) 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.096+0000 7f88e6a00700 1 -- 172.21.15.90:0/1381168819 learned_addr learned my addr 172.21.15.90:0/1381168819 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.096+0000 7f88e6a00700 1 -- 172.21.15.90:0/1381168819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e819e2c0 con 0x7f88e8109a30 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.097+0000 7f88e6a00700 1 --2- 172.21.15.90:0/1381168819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e819dd80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88d8011980 tx=0x7f88d8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.097+0000 7f88eca00700 1 -- 172.21.15.90:0/1381168819 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88d801f070 con 0x7f88e8109a30 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.097+0000 7f88eca00700 1 -- 172.21.15.90:0/1381168819 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f88d8003aa0 con 0x7f88e8109a30 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.097+0000 7f88eca00700 1 -- 172.21.15.90:0/1381168819 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88d8019560 con 0x7f88e8109a30 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.097+0000 7f88ede00700 1 -- 172.21.15.90:0/1381168819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e819e550 con 0x7f88e8109a30 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.097+0000 7f88ede00700 1 -- 172.21.15.90:0/1381168819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e81a1160 con 0x7f88e8109a30 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.098+0000 7f88ede00700 1 -- 172.21.15.90:0/1381168819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e8064e10 con 0x7f88e8109a30 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.099+0000 7f88eca00700 1 -- 172.21.15.90:0/1381168819 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f88d8022070 con 0x7f88e8109a30 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.099+0000 7f88eca00700 1 --2- 172.21.15.90:0/1381168819 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f88cc03e510 0x7f88cc040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:02.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.099+0000 7f88eca00700 1 -- 172.21.15.90:0/1381168819 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f88d8053a90 con 0x7f88e8109a30 2024-05-27T07:31:02.103 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.099+0000 7f88e6000700 1 --2- 172.21.15.90:0/1381168819 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f88cc03e510 0x7f88cc040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:02.103 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.100+0000 7f88e6000700 1 --2- 172.21.15.90:0/1381168819 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f88cc03e510 0x7f88cc040a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f88d4006fd0 tx=0x7f88d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:02.103 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.105+0000 7f88eca00700 1 -- 172.21.15.90:0/1381168819 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88d8023e50 con 0x7f88e8109a30 2024-05-27T07:31:02.103 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.345+0000 7f88ede00700 1 -- 172.21.15.90:0/1381168819 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f88e81a1330 con 0x7f88cc03e510 2024-05-27T07:31:02.103 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.455+0000 7f88eca00700 1 -- 172.21.15.90:0/1381168819 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f88e81a1330 con 0x7f88cc03e510 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.458+0000 7f88ede00700 1 -- 172.21.15.90:0/1381168819 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f88cc03e510 msgr2=0x7f88cc040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.458+0000 7f88ede00700 1 --2- 172.21.15.90:0/1381168819 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f88cc03e510 0x7f88cc040a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f88d4006fd0 tx=0x7f88d4006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.458+0000 7f88ede00700 1 -- 172.21.15.90:0/1381168819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 msgr2=0x7f88e819dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.458+0000 7f88ede00700 1 --2- 172.21.15.90:0/1381168819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e819dd80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88d8011980 tx=0x7f88d8011a60 comp rx=0 tx=0).stop 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.459+0000 7f88ede00700 1 -- 172.21.15.90:0/1381168819 shutdown_connections 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.459+0000 7f88ede00700 1 --2- 172.21.15.90:0/1381168819 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f88cc03e510 0x7f88cc040a20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.459+0000 7f88ede00700 1 --2- 172.21.15.90:0/1381168819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88e8109a30 0x7f88e819dd80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.459+0000 7f88ede00700 1 -- 172.21.15.90:0/1381168819 >> 172.21.15.90:0/1381168819 conn(0x7f88e8105090 msgr2=0x7f88e8105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.459+0000 7f88ede00700 1 -- 172.21.15.90:0/1381168819 shutdown_connections 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:01.459+0000 7f88ede00700 1 -- 172.21.15.90:0/1381168819 wait complete. 2024-05-27T07:31:02.104 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial admin user... 2024-05-27T07:31:02.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:02 smithi090 ceph-mon[31429]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:31:02.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:02.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:02.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:03.940 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$0gLRSZeveFyVnDfWYwXSWO6arUPXxS4itCkPQ75xyWiBIek3WZ55e", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716795063, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-27T07:31:03.941 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.720+0000 7f616e400700 1 Processor -- start 2024-05-27T07:31:03.941 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.721+0000 7f616e400700 1 -- start start 2024-05-27T07:31:03.941 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.721+0000 7f616e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f616810c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:03.941 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.721+0000 7f616e400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f616807ce80 con 0x7f616810a4e0 2024-05-27T07:31:03.941 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.721+0000 7f6167400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f616810c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:03.941 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.722+0000 7f6167400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f616810c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39680/0 (socket says 172.21.15.90:39680) 2024-05-27T07:31:03.942 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.722+0000 7f6167400700 1 -- 172.21.15.90:0/1644527534 learned_addr learned my addr 172.21.15.90:0/1644527534 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:03.942 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.722+0000 7f6167400700 1 -- 172.21.15.90:0/1644527534 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616810ce40 con 0x7f616810a4e0 2024-05-27T07:31:03.942 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.722+0000 7f6167400700 1 --2- 172.21.15.90:0/1644527534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f616810c900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61580098a0 tx=0x7f6158009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56dff471b6d45eaa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:03.942 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.723+0000 7f6166000700 1 -- 172.21.15.90:0/1644527534 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6158004160 con 0x7f616810a4e0 2024-05-27T07:31:03.942 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.723+0000 7f6166000700 1 -- 172.21.15.90:0/1644527534 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6158004300 con 0x7f616810a4e0 2024-05-27T07:31:03.942 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.723+0000 7f6166000700 1 -- 172.21.15.90:0/1644527534 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6158011960 con 0x7f616810a4e0 2024-05-27T07:31:03.942 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.724+0000 7f616e400700 1 -- 172.21.15.90:0/1644527534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 msgr2=0x7f616810c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:03.943 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.724+0000 7f616e400700 1 --2- 172.21.15.90:0/1644527534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f616810c900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f61580098a0 tx=0x7f6158009bb0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.943 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.724+0000 7f616e400700 1 -- 172.21.15.90:0/1644527534 shutdown_connections 2024-05-27T07:31:03.943 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.724+0000 7f616e400700 1 --2- 172.21.15.90:0/1644527534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f616810c900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.943 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.724+0000 7f616e400700 1 -- 172.21.15.90:0/1644527534 >> 172.21.15.90:0/1644527534 conn(0x7f6168103f60 msgr2=0x7f61681063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:03.943 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.724+0000 7f616e400700 1 -- 172.21.15.90:0/1644527534 shutdown_connections 2024-05-27T07:31:03.943 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.724+0000 7f616e400700 1 -- 172.21.15.90:0/1644527534 wait complete. 2024-05-27T07:31:03.944 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.725+0000 7f616e400700 1 Processor -- start 2024-05-27T07:31:03.944 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.726+0000 7f616e400700 1 -- start start 2024-05-27T07:31:03.944 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.726+0000 7f616e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f6168109c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:03.944 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.726+0000 7f616e400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f616810dde0 con 0x7f616810a4e0 2024-05-27T07:31:03.945 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.726+0000 7f6167400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f6168109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:03.945 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.726+0000 7f6167400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f6168109c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39686/0 (socket says 172.21.15.90:39686) 2024-05-27T07:31:03.945 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.726+0000 7f6167400700 1 -- 172.21.15.90:0/3549879653 learned_addr learned my addr 172.21.15.90:0/3549879653 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:03.945 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.727+0000 7f6167400700 1 -- 172.21.15.90:0/3549879653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616810a1a0 con 0x7f616810a4e0 2024-05-27T07:31:03.945 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.727+0000 7f6167400700 1 --2- 172.21.15.90:0/3549879653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f6168109c60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6158011670 tx=0x7f6158011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:03.945 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.727+0000 7f615fe00700 1 -- 172.21.15.90:0/3549879653 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f615801f070 con 0x7f616810a4e0 2024-05-27T07:31:03.945 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.727+0000 7f615fe00700 1 -- 172.21.15.90:0/3549879653 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6158003b20 con 0x7f616810a4e0 2024-05-27T07:31:03.946 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.728+0000 7f616e400700 1 -- 172.21.15.90:0/3549879653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61681083f0 con 0x7f616810a4e0 2024-05-27T07:31:03.946 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.728+0000 7f616e400700 1 -- 172.21.15.90:0/3549879653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61681088d0 con 0x7f616810a4e0 2024-05-27T07:31:03.946 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.728+0000 7f615fe00700 1 -- 172.21.15.90:0/3549879653 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6158019580 con 0x7f616810a4e0 2024-05-27T07:31:03.946 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.729+0000 7f615fe00700 1 -- 172.21.15.90:0/3549879653 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f6158022070 con 0x7f616810a4e0 2024-05-27T07:31:03.946 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.730+0000 7f615fe00700 1 --2- 172.21.15.90:0/3549879653 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f614c03e520 0x7f614c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:03.946 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.730+0000 7f615fe00700 1 -- 172.21.15.90:0/3549879653 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6158054ad0 con 0x7f616810a4e0 2024-05-27T07:31:03.946 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.730+0000 7f616e400700 1 -- 172.21.15.90:0/3549879653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6148005360 con 0x7f616810a4e0 2024-05-27T07:31:03.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.730+0000 7f6166a00700 1 --2- 172.21.15.90:0/3549879653 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f614c03e520 0x7f614c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:03.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.731+0000 7f6166a00700 1 --2- 172.21.15.90:0/3549879653 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f614c03e520 0x7f614c040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6154006fd0 tx=0x7f6154006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:03.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.739+0000 7f615fe00700 1 -- 172.21.15.90:0/3549879653 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61580172b0 con 0x7f616810a4e0 2024-05-27T07:31:03.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:02.992+0000 7f616e400700 1 -- 172.21.15.90:0/3549879653 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f6148002430 con 0x7f614c03e520 2024-05-27T07:31:03.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.232+0000 7f615fe00700 1 -- 172.21.15.90:0/3549879653 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f6148002430 con 0x7f614c03e520 2024-05-27T07:31:03.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.236+0000 7f616e400700 1 -- 172.21.15.90:0/3549879653 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f614c03e520 msgr2=0x7f614c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:03.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.236+0000 7f616e400700 1 --2- 172.21.15.90:0/3549879653 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f614c03e520 0x7f614c040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6154006fd0 tx=0x7f6154006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:03.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.236+0000 7f616e400700 1 -- 172.21.15.90:0/3549879653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 msgr2=0x7f6168109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:03.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.236+0000 7f616e400700 1 --2- 172.21.15.90:0/3549879653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f6168109c60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6158011670 tx=0x7f6158011a30 comp rx=0 tx=0).stop 2024-05-27T07:31:03.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.236+0000 7f616e400700 1 -- 172.21.15.90:0/3549879653 shutdown_connections 2024-05-27T07:31:03.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.236+0000 7f616e400700 1 --2- 172.21.15.90:0/3549879653 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f614c03e520 0x7f614c040a30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.236+0000 7f616e400700 1 --2- 172.21.15.90:0/3549879653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f616810a4e0 0x7f6168109c60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:03.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.236+0000 7f616e400700 1 -- 172.21.15.90:0/3549879653 >> 172.21.15.90:0/3549879653 conn(0x7f6168103f60 msgr2=0x7f6168104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:03.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.237+0000 7f616e400700 1 -- 172.21.15.90:0/3549879653 shutdown_connections 2024-05-27T07:31:03.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:03.237+0000 7f616e400700 1 -- 172.21.15.90:0/3549879653 wait complete. 2024-05-27T07:31:03.949 INFO:teuthology.orchestra.run.smithi090.stdout:Fetching dashboard port number... 2024-05-27T07:31:04.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:04 smithi090 ceph-mon[31429]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:31:04.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:04 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:04.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:04 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:05.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:05 smithi090 ceph-mon[31429]: mgrmap e13: x(active, since 6s) 2024-05-27T07:31:05.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:05 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/685249433' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-27T07:31:06.446 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 8443 2024-05-27T07:31:06.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.589+0000 7f5103000700 1 Processor -- start 2024-05-27T07:31:06.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.589+0000 7f5103000700 1 -- start start 2024-05-27T07:31:06.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.589+0000 7f5103000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc07d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:06.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.589+0000 7f5103000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc07d830 con 0x7f50fc07ee30 2024-05-27T07:31:06.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.590+0000 7f50fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc07d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:06.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.590+0000 7f50fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc07d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39694/0 (socket says 172.21.15.90:39694) 2024-05-27T07:31:06.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.590+0000 7f50fbe00700 1 -- 172.21.15.90:0/3621026932 learned_addr learned my addr 172.21.15.90:0/3621026932 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.590+0000 7f50fbe00700 1 -- 172.21.15.90:0/3621026932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc07d9b0 con 0x7f50fc07ee30 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.591+0000 7f50fbe00700 1 --2- 172.21.15.90:0/3621026932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc07d260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0098a0 tx=0x7f50ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1016bf154a5b8edf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.591+0000 7f50faa00700 1 -- 172.21.15.90:0/3621026932 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ec004160 con 0x7f50fc07ee30 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.591+0000 7f50faa00700 1 -- 172.21.15.90:0/3621026932 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50ec004300 con 0x7f50fc07ee30 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.591+0000 7f50faa00700 1 -- 172.21.15.90:0/3621026932 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ec011960 con 0x7f50fc07ee30 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.592+0000 7f5103000700 1 -- 172.21.15.90:0/3621026932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 msgr2=0x7f50fc07d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.592+0000 7f5103000700 1 --2- 172.21.15.90:0/3621026932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc07d260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0098a0 tx=0x7f50ec009bb0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.592+0000 7f5103000700 1 -- 172.21.15.90:0/3621026932 shutdown_connections 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.592+0000 7f5103000700 1 --2- 172.21.15.90:0/3621026932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc07d260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.592+0000 7f5103000700 1 -- 172.21.15.90:0/3621026932 >> 172.21.15.90:0/3621026932 conn(0x7f50fc1050b0 msgr2=0x7f50fc107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.592+0000 7f5103000700 1 -- 172.21.15.90:0/3621026932 shutdown_connections 2024-05-27T07:31:06.452 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.592+0000 7f5103000700 1 -- 172.21.15.90:0/3621026932 wait complete. 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.593+0000 7f5103000700 1 Processor -- start 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.593+0000 7f5103000700 1 -- start start 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.594+0000 7f5103000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc19ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.594+0000 7f5103000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc110050 con 0x7f50fc07ee30 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.594+0000 7f50fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc19ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.594+0000 7f50fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc19ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39700/0 (socket says 172.21.15.90:39700) 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.594+0000 7f50fbe00700 1 -- 172.21.15.90:0/685249433 learned_addr learned my addr 172.21.15.90:0/685249433 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.594+0000 7f50fbe00700 1 -- 172.21.15.90:0/685249433 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc1a0500 con 0x7f50fc07ee30 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.595+0000 7f50fbe00700 1 --2- 172.21.15.90:0/685249433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc19ffc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50ec011670 tx=0x7f50ec011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.595+0000 7f50f8c00700 1 -- 172.21.15.90:0/685249433 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ec01f070 con 0x7f50fc07ee30 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.595+0000 7f50f8c00700 1 -- 172.21.15.90:0/685249433 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50ec003b20 con 0x7f50fc07ee30 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.595+0000 7f5103000700 1 -- 172.21.15.90:0/685249433 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc1a0790 con 0x7f50fc07ee30 2024-05-27T07:31:06.453 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.595+0000 7f50f8c00700 1 -- 172.21.15.90:0/685249433 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ec019580 con 0x7f50fc07ee30 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.595+0000 7f5103000700 1 -- 172.21.15.90:0/685249433 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc1a33a0 con 0x7f50fc07ee30 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.596+0000 7f5103000700 1 -- 172.21.15.90:0/685249433 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50fc064e10 con 0x7f50fc07ee30 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.596+0000 7f50f8c00700 1 -- 172.21.15.90:0/685249433 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f50ec022070 con 0x7f50fc07ee30 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.597+0000 7f50f8c00700 1 --2- 172.21.15.90:0/685249433 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50e003e5c0 0x7f50e0040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.597+0000 7f50f8c00700 1 -- 172.21.15.90:0/685249433 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f50ec053c00 con 0x7f50fc07ee30 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.597+0000 7f50fb400700 1 --2- 172.21.15.90:0/685249433 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50e003e5c0 0x7f50e0040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.598+0000 7f50fb400700 1 --2- 172.21.15.90:0/685249433 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50e003e5c0 0x7f50e0040ad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f50e8006fd0 tx=0x7f50e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.603+0000 7f50f8c00700 1 -- 172.21.15.90:0/685249433 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50ec02cd00 con 0x7f50fc07ee30 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.829+0000 7f5103000700 1 -- 172.21.15.90:0/685249433 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f50fc1a3570 con 0x7f50fc07ee30 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.830+0000 7f50f8c00700 1 -- 172.21.15.90:0/685249433 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f50ec01ba80 con 0x7f50fc07ee30 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.832+0000 7f5103000700 1 -- 172.21.15.90:0/685249433 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50e003e5c0 msgr2=0x7f50e0040ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.832+0000 7f5103000700 1 --2- 172.21.15.90:0/685249433 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50e003e5c0 0x7f50e0040ad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f50e8006fd0 tx=0x7f50e8006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:06.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.832+0000 7f5103000700 1 -- 172.21.15.90:0/685249433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 msgr2=0x7f50fc19ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:06.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.832+0000 7f5103000700 1 --2- 172.21.15.90:0/685249433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc19ffc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50ec011670 tx=0x7f50ec011a30 comp rx=0 tx=0).stop 2024-05-27T07:31:06.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.833+0000 7f5103000700 1 -- 172.21.15.90:0/685249433 shutdown_connections 2024-05-27T07:31:06.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.833+0000 7f5103000700 1 --2- 172.21.15.90:0/685249433 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50e003e5c0 0x7f50e0040ad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.833+0000 7f5103000700 1 --2- 172.21.15.90:0/685249433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50fc07ee30 0x7f50fc19ffc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:06.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.833+0000 7f5103000700 1 -- 172.21.15.90:0/685249433 >> 172.21.15.90:0/685249433 conn(0x7f50fc1050b0 msgr2=0x7f50fc105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:06.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.833+0000 7f5103000700 1 -- 172.21.15.90:0/685249433 shutdown_connections 2024-05-27T07:31:06.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:04.833+0000 7f5103000700 1 -- 172.21.15.90:0/685249433 wait complete. 2024-05-27T07:31:06.477 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T07:31:06.478 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-05-27T07:31:06.502 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T07:31:06.502 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-05-27T07:31:06.503 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-05-27T07:31:06.503 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-27T07:31:06.509 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph Dashboard is now available at: 2024-05-27T07:31:06.509 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: URL: https://smithi090.front.sepia.ceph.com:8443/ 2024-05-27T07:31:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: User: admin 2024-05-27T07:31:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: Password: ijfyic7cpv 2024-05-27T07:31:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:06.510 INFO:teuthology.orchestra.run.smithi090.stdout:Saving cluster configuration to /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config directory 2024-05-27T07:31:06.513 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling autotune for osd_memory_target 2024-05-27T07:31:08.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.247+0000 7fd410000700 1 Processor -- start 2024-05-27T07:31:08.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.247+0000 7fd410000700 1 -- start start 2024-05-27T07:31:08.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.247+0000 7fd410000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd408112c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:08.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.247+0000 7fd410000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd408113200 con 0x7fd4081107f0 2024-05-27T07:31:08.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.248+0000 7fd40d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd408112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:08.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.248+0000 7fd40d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd408112c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39702/0 (socket says 172.21.15.90:39702) 2024-05-27T07:31:08.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.248+0000 7fd40d600700 1 -- 172.21.15.90:0/1586723875 learned_addr learned my addr 172.21.15.90:0/1586723875 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:08.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.248+0000 7fd40d600700 1 -- 172.21.15.90:0/1586723875 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd408113380 con 0x7fd4081107f0 2024-05-27T07:31:08.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.249+0000 7fd40d600700 1 --2- 172.21.15.90:0/1586723875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd408112c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0098a0 tx=0x7fd3fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c196cc8da046e077 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.249+0000 7fd407e00700 1 -- 172.21.15.90:0/1586723875 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc004160 con 0x7fd4081107f0 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.249+0000 7fd407e00700 1 -- 172.21.15.90:0/1586723875 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3fc004300 con 0x7fd4081107f0 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.249+0000 7fd407e00700 1 -- 172.21.15.90:0/1586723875 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc011960 con 0x7fd4081107f0 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.250+0000 7fd410000700 1 -- 172.21.15.90:0/1586723875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 msgr2=0x7fd408112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.250+0000 7fd410000700 1 --2- 172.21.15.90:0/1586723875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd408112c10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0098a0 tx=0x7fd3fc009bb0 comp rx=0 tx=0).stop 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.250+0000 7fd410000700 1 -- 172.21.15.90:0/1586723875 shutdown_connections 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.250+0000 7fd410000700 1 --2- 172.21.15.90:0/1586723875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd408112c10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.250+0000 7fd410000700 1 -- 172.21.15.90:0/1586723875 >> 172.21.15.90:0/1586723875 conn(0x7fd40807f0b0 msgr2=0x7fd408081540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.250+0000 7fd410000700 1 -- 172.21.15.90:0/1586723875 shutdown_connections 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.250+0000 7fd410000700 1 -- 172.21.15.90:0/1586723875 wait complete. 2024-05-27T07:31:08.144 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.251+0000 7fd410000700 1 Processor -- start 2024-05-27T07:31:08.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.252+0000 7fd410000700 1 -- start start 2024-05-27T07:31:08.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.252+0000 7fd410000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd4081a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:08.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.252+0000 7fd410000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd4081143e0 con 0x7fd4081107f0 2024-05-27T07:31:08.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.252+0000 7fd40d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd4081a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:08.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.252+0000 7fd40d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd4081a4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39718/0 (socket says 172.21.15.90:39718) 2024-05-27T07:31:08.147 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.252+0000 7fd40d600700 1 -- 172.21.15.90:0/532304088 learned_addr learned my addr 172.21.15.90:0/532304088 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.253+0000 7fd40d600700 1 -- 172.21.15.90:0/532304088 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4081a48c0 con 0x7fd4081107f0 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.253+0000 7fd40d600700 1 --2- 172.21.15.90:0/532304088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd4081a4380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc011670 tx=0x7fd3fc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.253+0000 7fd406000700 1 -- 172.21.15.90:0/532304088 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc01f070 con 0x7fd4081107f0 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.253+0000 7fd406000700 1 -- 172.21.15.90:0/532304088 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3fc003b20 con 0x7fd4081107f0 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.253+0000 7fd406000700 1 -- 172.21.15.90:0/532304088 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3fc019580 con 0x7fd4081107f0 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.253+0000 7fd410000700 1 -- 172.21.15.90:0/532304088 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4081a4b50 con 0x7fd4081107f0 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.254+0000 7fd410000700 1 -- 172.21.15.90:0/532304088 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4081a7760 con 0x7fd4081107f0 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.255+0000 7fd410000700 1 -- 172.21.15.90:0/532304088 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd408064e10 con 0x7fd4081107f0 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.255+0000 7fd406000700 1 -- 172.21.15.90:0/532304088 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd3fc022070 con 0x7fd4081107f0 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.255+0000 7fd406000700 1 --2- 172.21.15.90:0/532304088 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd3f003e5c0 0x7fd3f0040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:08.148 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.255+0000 7fd406000700 1 -- 172.21.15.90:0/532304088 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd3fc053c80 con 0x7fd4081107f0 2024-05-27T07:31:08.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.256+0000 7fd40cc00700 1 --2- 172.21.15.90:0/532304088 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd3f003e5c0 0x7fd3f0040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:08.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.256+0000 7fd40cc00700 1 --2- 172.21.15.90:0/532304088 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd3f003e5c0 0x7fd3f0040ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4006fd0 tx=0x7fd3f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:08.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.261+0000 7fd406000700 1 -- 172.21.15.90:0/532304088 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3fc0172b0 con 0x7fd4081107f0 2024-05-27T07:31:08.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.492+0000 7fd410000700 1 -- 172.21.15.90:0/532304088 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fd4081a7930 con 0x7fd4081107f0 2024-05-27T07:31:08.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.493+0000 7fd406000700 1 -- 172.21.15.90:0/532304088 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fd3fc01ba80 con 0x7fd4081107f0 2024-05-27T07:31:08.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.496+0000 7fd410000700 1 -- 172.21.15.90:0/532304088 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd3f003e5c0 msgr2=0x7fd3f0040ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:08.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.497+0000 7fd410000700 1 --2- 172.21.15.90:0/532304088 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd3f003e5c0 0x7fd3f0040ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4006fd0 tx=0x7fd3f4006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.497+0000 7fd410000700 1 -- 172.21.15.90:0/532304088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 msgr2=0x7fd4081a4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.497+0000 7fd410000700 1 --2- 172.21.15.90:0/532304088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd4081a4380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc011670 tx=0x7fd3fc011a30 comp rx=0 tx=0).stop 2024-05-27T07:31:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.497+0000 7fd410000700 1 -- 172.21.15.90:0/532304088 shutdown_connections 2024-05-27T07:31:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.497+0000 7fd410000700 1 --2- 172.21.15.90:0/532304088 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd3f003e5c0 0x7fd3f0040ad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.497+0000 7fd410000700 1 --2- 172.21.15.90:0/532304088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4081107f0 0x7fd4081a4380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.497+0000 7fd410000700 1 -- 172.21.15.90:0/532304088 >> 172.21.15.90:0/532304088 conn(0x7fd40807f0b0 msgr2=0x7fd40807f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.497+0000 7fd410000700 1 -- 172.21.15.90:0/532304088 shutdown_connections 2024-05-27T07:31:08.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:07.497+0000 7fd410000700 1 -- 172.21.15.90:0/532304088 wait complete. 2024-05-27T07:31:09.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.849+0000 7f56c6200700 1 Processor -- start 2024-05-27T07:31:09.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.849+0000 7f56c6200700 1 -- start start 2024-05-27T07:31:09.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.850+0000 7f56c6200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c0109e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:09.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.850+0000 7f56c6200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56c010a410 con 0x7f56c0109a10 2024-05-27T07:31:09.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.850+0000 7f56bf400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c0109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:09.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.850+0000 7f56bf400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c0109e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54648/0 (socket says 172.21.15.90:54648) 2024-05-27T07:31:09.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.850+0000 7f56bf400700 1 -- 172.21.15.90:0/4285366599 learned_addr learned my addr 172.21.15.90:0/4285366599 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:09.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.851+0000 7f56bf400700 1 -- 172.21.15.90:0/4285366599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c010abe0 con 0x7f56c0109a10 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.851+0000 7f56bf400700 1 --2- 172.21.15.90:0/4285366599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c0109e20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f56b00098a0 tx=0x7f56b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1288c9855d1054 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.851+0000 7f56bea00700 1 -- 172.21.15.90:0/4285366599 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56b0004160 con 0x7f56c0109a10 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.851+0000 7f56bea00700 1 -- 172.21.15.90:0/4285366599 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56b0004300 con 0x7f56c0109a10 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.851+0000 7f56bea00700 1 -- 172.21.15.90:0/4285366599 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56b0011960 con 0x7f56c0109a10 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.852+0000 7f56c6200700 1 -- 172.21.15.90:0/4285366599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 msgr2=0x7f56c0109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.852+0000 7f56c6200700 1 --2- 172.21.15.90:0/4285366599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c0109e20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f56b00098a0 tx=0x7f56b0009bb0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.853+0000 7f56c6200700 1 -- 172.21.15.90:0/4285366599 shutdown_connections 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.853+0000 7f56c6200700 1 --2- 172.21.15.90:0/4285366599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c0109e20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.853+0000 7f56c6200700 1 -- 172.21.15.90:0/4285366599 >> 172.21.15.90:0/4285366599 conn(0x7f56c0105050 msgr2=0x7f56c0107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.853+0000 7f56c6200700 1 -- 172.21.15.90:0/4285366599 shutdown_connections 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.853+0000 7f56c6200700 1 -- 172.21.15.90:0/4285366599 wait complete. 2024-05-27T07:31:09.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.854+0000 7f56c6200700 1 Processor -- start 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.854+0000 7f56c6200700 1 -- start start 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.854+0000 7f56c6200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.854+0000 7f56c6200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56c010df90 con 0x7f56c0109a10 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.855+0000 7f56bf400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.855+0000 7f56bf400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c019dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54652/0 (socket says 172.21.15.90:54652) 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.855+0000 7f56bf400700 1 -- 172.21.15.90:0/3064778975 learned_addr learned my addr 172.21.15.90:0/3064778975 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.855+0000 7f56bf400700 1 -- 172.21.15.90:0/3064778975 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c019e2d0 con 0x7f56c0109a10 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.855+0000 7f56bf400700 1 --2- 172.21.15.90:0/3064778975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c019dd90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f56b0011980 tx=0x7f56b0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.856+0000 7f56bcc00700 1 -- 172.21.15.90:0/3064778975 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56b001f070 con 0x7f56c0109a10 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.856+0000 7f56bcc00700 1 -- 172.21.15.90:0/3064778975 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56b0003aa0 con 0x7f56c0109a10 2024-05-27T07:31:09.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.856+0000 7f56bcc00700 1 -- 172.21.15.90:0/3064778975 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56b0019560 con 0x7f56c0109a10 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.856+0000 7f56c6200700 1 -- 172.21.15.90:0/3064778975 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c019e560 con 0x7f56c0109a10 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.856+0000 7f56c6200700 1 -- 172.21.15.90:0/3064778975 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c01a1170 con 0x7f56c0109a10 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.857+0000 7f56c6200700 1 -- 172.21.15.90:0/3064778975 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c0064e10 con 0x7f56c0109a10 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.857+0000 7f56bcc00700 1 -- 172.21.15.90:0/3064778975 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f56b0022070 con 0x7f56c0109a10 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.858+0000 7f56bcc00700 1 --2- 172.21.15.90:0/3064778975 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f56a403e5b0 0x7f56a4040ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.858+0000 7f56bcc00700 1 -- 172.21.15.90:0/3064778975 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f56b0053c00 con 0x7f56c0109a10 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.858+0000 7f56b6a00700 1 --2- 172.21.15.90:0/3064778975 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f56a403e5b0 0x7f56a4040ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.859+0000 7f56b6a00700 1 --2- 172.21.15.90:0/3064778975 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f56a403e5b0 0x7f56a4040ac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006fd0 tx=0x7f56a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:08.864+0000 7f56bcc00700 1 -- 172.21.15.90:0/3064778975 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56b0023aa0 con 0x7f56c0109a10 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.210+0000 7f56c6200700 1 -- 172.21.15.90:0/3064778975 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f56c01a1340 con 0x7f56c0109a10 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.215+0000 7f56bcc00700 1 -- 172.21.15.90:0/3064778975 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f56b001ba80 con 0x7f56c0109a10 2024-05-27T07:31:09.855 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.218+0000 7f56c6200700 1 -- 172.21.15.90:0/3064778975 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f56a403e5b0 msgr2=0x7f56a4040ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.218+0000 7f56c6200700 1 --2- 172.21.15.90:0/3064778975 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f56a403e5b0 0x7f56a4040ac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006fd0 tx=0x7f56a8006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.218+0000 7f56c6200700 1 -- 172.21.15.90:0/3064778975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 msgr2=0x7f56c019dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.218+0000 7f56c6200700 1 --2- 172.21.15.90:0/3064778975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c019dd90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f56b0011980 tx=0x7f56b0011a60 comp rx=0 tx=0).stop 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.218+0000 7f56c6200700 1 -- 172.21.15.90:0/3064778975 shutdown_connections 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.218+0000 7f56c6200700 1 --2- 172.21.15.90:0/3064778975 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f56a403e5b0 0x7f56a4040ac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.218+0000 7f56c6200700 1 --2- 172.21.15.90:0/3064778975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c0109a10 0x7f56c019dd90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.218+0000 7f56c6200700 1 -- 172.21.15.90:0/3064778975 >> 172.21.15.90:0/3064778975 conn(0x7f56c0105050 msgr2=0x7f56c0105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.219+0000 7f56c6200700 1 -- 172.21.15.90:0/3064778975 shutdown_connections 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-05-27T07:31:09.219+0000 7f56c6200700 1 -- 172.21.15.90:0/3064778975 wait complete. 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-27T07:31:09.856 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout:Or, if you are only running a single cluster on this host: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: ceph telemetry on 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout:For more information see: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:09.857 INFO:teuthology.orchestra.run.smithi090.stdout:Bootstrap complete. 2024-05-27T07:31:09.956 INFO:tasks.cephadm:Fetching config... 2024-05-27T07:31:09.956 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:31:09.956 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-27T07:31:10.001 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-27T07:31:10.001 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:31:10.001 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-27T07:31:10.086 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-27T07:31:10.086 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:31:10.086 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-27T07:31:10.209 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-27T07:31:10.209 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:31:10.210 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-27T07:31:10.261 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-27T07:31:10.262 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy7HkqLpW6HfI2MMU0CTJa2Z+3KhtD5H/qszNaqdj0me9lMm+Ha7nXDhYnUiLzm69P8FzAQtiTjqLuB0rckKM4svzvxNff46MUsRk67fBtrylqA71RKbnOmUtdphGLS1DFhiyeHXKLD0eo+qPLGKbgVe+xFeOLvxetpelijgmwlYjICC5g7aTe6GbZ+Tw+27N9treDQqgSfz8fIOEO7gaeJdiJ5StUyC2jrR2iDob5ZvP2vKHYKAvfPMDW93wGKpEMVKU4cDdh36oEQnM90/IHLwFW6v6Qr1oaoDZrAiELvKR9BOcQPTL3U2EPQGFXoCYAx8l5HREDxG4xYcDThQYJ58C7pFoMehazaOCi7bYFdVNyGskOyg8uiVTndOGk32bavvRl2FR2mI/l7ruXwLNWx7PVTvxxo45UxBhmMyjwCsXxpJ3AqW77gLk+rav7UtoDCxPoiCh1DTWIl1H09q+bAUecgizs/+5IH4ajTDx45O32RYwYh7H/RYz0P4/t1U0= ceph-b0214938-1bfa-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T07:31:10.356 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy7HkqLpW6HfI2MMU0CTJa2Z+3KhtD5H/qszNaqdj0me9lMm+Ha7nXDhYnUiLzm69P8FzAQtiTjqLuB0rckKM4svzvxNff46MUsRk67fBtrylqA71RKbnOmUtdphGLS1DFhiyeHXKLD0eo+qPLGKbgVe+xFeOLvxetpelijgmwlYjICC5g7aTe6GbZ+Tw+27N9treDQqgSfz8fIOEO7gaeJdiJ5StUyC2jrR2iDob5ZvP2vKHYKAvfPMDW93wGKpEMVKU4cDdh36oEQnM90/IHLwFW6v6Qr1oaoDZrAiELvKR9BOcQPTL3U2EPQGFXoCYAx8l5HREDxG4xYcDThQYJ58C7pFoMehazaOCi7bYFdVNyGskOyg8uiVTndOGk32bavvRl2FR2mI/l7ruXwLNWx7PVTvxxo45UxBhmMyjwCsXxpJ3AqW77gLk+rav7UtoDCxPoiCh1DTWIl1H09q+bAUecgizs/+5IH4ajTDx45O32RYwYh7H/RYz0P4/t1U0= ceph-b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:31:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:10 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3064778975' entity='client.admin' 2024-05-27T07:31:10.387 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy7HkqLpW6HfI2MMU0CTJa2Z+3KhtD5H/qszNaqdj0me9lMm+Ha7nXDhYnUiLzm69P8FzAQtiTjqLuB0rckKM4svzvxNff46MUsRk67fBtrylqA71RKbnOmUtdphGLS1DFhiyeHXKLD0eo+qPLGKbgVe+xFeOLvxetpelijgmwlYjICC5g7aTe6GbZ+Tw+27N9treDQqgSfz8fIOEO7gaeJdiJ5StUyC2jrR2iDob5ZvP2vKHYKAvfPMDW93wGKpEMVKU4cDdh36oEQnM90/IHLwFW6v6Qr1oaoDZrAiELvKR9BOcQPTL3U2EPQGFXoCYAx8l5HREDxG4xYcDThQYJ58C7pFoMehazaOCi7bYFdVNyGskOyg8uiVTndOGk32bavvRl2FR2mI/l7ruXwLNWx7PVTvxxo45UxBhmMyjwCsXxpJ3AqW77gLk+rav7UtoDCxPoiCh1DTWIl1H09q+bAUecgizs/+5IH4ajTDx45O32RYwYh7H/RYz0P4/t1U0= ceph-b0214938-1bfa-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T07:31:10.505 INFO:teuthology.orchestra.run.smithi129.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy7HkqLpW6HfI2MMU0CTJa2Z+3KhtD5H/qszNaqdj0me9lMm+Ha7nXDhYnUiLzm69P8FzAQtiTjqLuB0rckKM4svzvxNff46MUsRk67fBtrylqA71RKbnOmUtdphGLS1DFhiyeHXKLD0eo+qPLGKbgVe+xFeOLvxetpelijgmwlYjICC5g7aTe6GbZ+Tw+27N9treDQqgSfz8fIOEO7gaeJdiJ5StUyC2jrR2iDob5ZvP2vKHYKAvfPMDW93wGKpEMVKU4cDdh36oEQnM90/IHLwFW6v6Qr1oaoDZrAiELvKR9BOcQPTL3U2EPQGFXoCYAx8l5HREDxG4xYcDThQYJ58C7pFoMehazaOCi7bYFdVNyGskOyg8uiVTndOGk32bavvRl2FR2mI/l7ruXwLNWx7PVTvxxo45UxBhmMyjwCsXxpJ3AqW77gLk+rav7UtoDCxPoiCh1DTWIl1H09q+bAUecgizs/+5IH4ajTDx45O32RYwYh7H/RYz0P4/t1U0= ceph-b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:31:10.563 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy7HkqLpW6HfI2MMU0CTJa2Z+3KhtD5H/qszNaqdj0me9lMm+Ha7nXDhYnUiLzm69P8FzAQtiTjqLuB0rckKM4svzvxNff46MUsRk67fBtrylqA71RKbnOmUtdphGLS1DFhiyeHXKLD0eo+qPLGKbgVe+xFeOLvxetpelijgmwlYjICC5g7aTe6GbZ+Tw+27N9treDQqgSfz8fIOEO7gaeJdiJ5StUyC2jrR2iDob5ZvP2vKHYKAvfPMDW93wGKpEMVKU4cDdh36oEQnM90/IHLwFW6v6Qr1oaoDZrAiELvKR9BOcQPTL3U2EPQGFXoCYAx8l5HREDxG4xYcDThQYJ58C7pFoMehazaOCi7bYFdVNyGskOyg8uiVTndOGk32bavvRl2FR2mI/l7ruXwLNWx7PVTvxxo45UxBhmMyjwCsXxpJ3AqW77gLk+rav7UtoDCxPoiCh1DTWIl1H09q+bAUecgizs/+5IH4ajTDx45O32RYwYh7H/RYz0P4/t1U0= ceph-b0214938-1bfa-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T07:31:10.679 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy7HkqLpW6HfI2MMU0CTJa2Z+3KhtD5H/qszNaqdj0me9lMm+Ha7nXDhYnUiLzm69P8FzAQtiTjqLuB0rckKM4svzvxNff46MUsRk67fBtrylqA71RKbnOmUtdphGLS1DFhiyeHXKLD0eo+qPLGKbgVe+xFeOLvxetpelijgmwlYjICC5g7aTe6GbZ+Tw+27N9treDQqgSfz8fIOEO7gaeJdiJ5StUyC2jrR2iDob5ZvP2vKHYKAvfPMDW93wGKpEMVKU4cDdh36oEQnM90/IHLwFW6v6Qr1oaoDZrAiELvKR9BOcQPTL3U2EPQGFXoCYAx8l5HREDxG4xYcDThQYJ58C7pFoMehazaOCi7bYFdVNyGskOyg8uiVTndOGk32bavvRl2FR2mI/l7ruXwLNWx7PVTvxxo45UxBhmMyjwCsXxpJ3AqW77gLk+rav7UtoDCxPoiCh1DTWIl1H09q+bAUecgizs/+5IH4ajTDx45O32RYwYh7H/RYz0P4/t1U0= ceph-b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:31:10.734 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-27T07:31:11.159 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:14.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.220+0000 7fd941600700 1 -- 172.21.15.90:0/2399388037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93c102b00 msgr2=0x7fd93c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:14.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.220+0000 7fd941600700 1 --2- 172.21.15.90:0/2399388037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93c102b00 0x7fd93c102f10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009910 tx=0x7fd92c009c20 comp rx=0 tx=0).stop 2024-05-27T07:31:14.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.220+0000 7fd941600700 1 -- 172.21.15.90:0/2399388037 shutdown_connections 2024-05-27T07:31:14.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.220+0000 7fd941600700 1 --2- 172.21.15.90:0/2399388037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93c102b00 0x7fd93c102f10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.220+0000 7fd941600700 1 -- 172.21.15.90:0/2399388037 >> 172.21.15.90:0/2399388037 conn(0x7fd93c0fe120 msgr2=0x7fd93c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:14.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.220+0000 7fd941600700 1 -- 172.21.15.90:0/2399388037 shutdown_connections 2024-05-27T07:31:14.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.220+0000 7fd941600700 1 -- 172.21.15.90:0/2399388037 wait complete. 2024-05-27T07:31:14.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.221+0000 7fd941600700 1 Processor -- start 2024-05-27T07:31:14.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.221+0000 7fd941600700 1 -- start start 2024-05-27T07:31:14.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.221+0000 7fd941600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93c102b00 0x7fd93c19b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:14.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.221+0000 7fd941600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c108970 con 0x7fd93c102b00 2024-05-27T07:31:14.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.222+0000 7fd93aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93c102b00 0x7fd93c19b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:14.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.222+0000 7fd93aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93c102b00 0x7fd93c19b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54672/0 (socket says 172.21.15.90:54672) 2024-05-27T07:31:14.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.222+0000 7fd93aa00700 1 -- 172.21.15.90:0/455782111 learned_addr learned my addr 172.21.15.90:0/455782111 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:14.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.222+0000 7fd93aa00700 1 -- 172.21.15.90:0/455782111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c19bde0 con 0x7fd93c102b00 2024-05-27T07:31:14.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.223+0000 7fd93aa00700 1 --2- 172.21.15.90:0/455782111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93c102b00 0x7fd93c19b8a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd92c005950 tx=0x7fd92c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:14.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.223+0000 7fd933400700 1 -- 172.21.15.90:0/455782111 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd92c027070 con 0x7fd93c102b00 2024-05-27T07:31:14.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.223+0000 7fd933400700 1 -- 172.21.15.90:0/455782111 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd92c00bdf0 con 0x7fd93c102b00 2024-05-27T07:31:14.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.223+0000 7fd941600700 1 -- 172.21.15.90:0/455782111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93c19c070 con 0x7fd93c102b00 2024-05-27T07:31:14.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.223+0000 7fd933400700 1 -- 172.21.15.90:0/455782111 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd92c00fdb0 con 0x7fd93c102b00 2024-05-27T07:31:14.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.223+0000 7fd941600700 1 -- 172.21.15.90:0/455782111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd93c19ec30 con 0x7fd93c102b00 2024-05-27T07:31:14.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.225+0000 7fd933400700 1 -- 172.21.15.90:0/455782111 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd92c02a070 con 0x7fd93c102b00 2024-05-27T07:31:14.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.225+0000 7fd941600700 1 -- 172.21.15.90:0/455782111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93c1090f0 con 0x7fd93c102b00 2024-05-27T07:31:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.225+0000 7fd933400700 1 --2- 172.21.15.90:0/455782111 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd92003e570 0x7fd920040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.225+0000 7fd933400700 1 -- 172.21.15.90:0/455782111 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd92c053c30 con 0x7fd93c102b00 2024-05-27T07:31:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.226+0000 7fd933e00700 1 --2- 172.21.15.90:0/455782111 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd92003e570 0x7fd920040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:14.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.226+0000 7fd933e00700 1 --2- 172.21.15.90:0/455782111 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd92003e570 0x7fd920040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd924006fd0 tx=0x7fd924006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:14.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.232+0000 7fd933400700 1 -- 172.21.15.90:0/455782111 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd92c013070 con 0x7fd93c102b00 2024-05-27T07:31:14.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.517+0000 7fd941600700 1 -- 172.21.15.90:0/455782111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fd93c19f080 con 0x7fd93c102b00 2024-05-27T07:31:14.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.522+0000 7fd933400700 1 -- 172.21.15.90:0/455782111 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fd92c00f460 con 0x7fd93c102b00 2024-05-27T07:31:14.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.525+0000 7fd941600700 1 -- 172.21.15.90:0/455782111 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd92003e570 msgr2=0x7fd920040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:14.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.525+0000 7fd941600700 1 --2- 172.21.15.90:0/455782111 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd92003e570 0x7fd920040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd924006fd0 tx=0x7fd924006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:14.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.525+0000 7fd941600700 1 -- 172.21.15.90:0/455782111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93c102b00 msgr2=0x7fd93c19b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:14.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.525+0000 7fd941600700 1 --2- 172.21.15.90:0/455782111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93c102b00 0x7fd93c19b8a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd92c005950 tx=0x7fd92c005810 comp rx=0 tx=0).stop 2024-05-27T07:31:14.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.525+0000 7fd941600700 1 -- 172.21.15.90:0/455782111 shutdown_connections 2024-05-27T07:31:14.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.525+0000 7fd941600700 1 --2- 172.21.15.90:0/455782111 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd92003e570 0x7fd920040a80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.525+0000 7fd941600700 1 --2- 172.21.15.90:0/455782111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93c102b00 0x7fd93c19b8a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:14.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.525+0000 7fd941600700 1 -- 172.21.15.90:0/455782111 >> 172.21.15.90:0/455782111 conn(0x7fd93c0fe120 msgr2=0x7fd93c06ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:14.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.525+0000 7fd941600700 1 -- 172.21.15.90:0/455782111 shutdown_connections 2024-05-27T07:31:14.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:14.525+0000 7fd941600700 1 -- 172.21.15.90:0/455782111 wait complete. 2024-05-27T07:31:15.193 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-27T07:31:15.194 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-27T07:31:15.764 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:15 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/455782111' entity='client.admin' 2024-05-27T07:31:15.882 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:17.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:17.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:17.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:31:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:31:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:17.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.946+0000 7ff7ad200700 1 -- 172.21.15.90:0/2616914470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7a810a1d0 msgr2=0x7ff7a810a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:17.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.946+0000 7ff7ad200700 1 --2- 172.21.15.90:0/2616914470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7a810a1d0 0x7ff7a810a5e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff798009910 tx=0x7ff798009c20 comp rx=0 tx=0).stop 2024-05-27T07:31:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.947+0000 7ff7ad200700 1 -- 172.21.15.90:0/2616914470 shutdown_connections 2024-05-27T07:31:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.947+0000 7ff7ad200700 1 --2- 172.21.15.90:0/2616914470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7a810a1d0 0x7ff7a810a5e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.947+0000 7ff7ad200700 1 -- 172.21.15.90:0/2616914470 >> 172.21.15.90:0/2616914470 conn(0x7ff7a8079240 msgr2=0x7ff7a807b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.947+0000 7ff7ad200700 1 -- 172.21.15.90:0/2616914470 shutdown_connections 2024-05-27T07:31:17.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.947+0000 7ff7ad200700 1 -- 172.21.15.90:0/2616914470 wait complete. 2024-05-27T07:31:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.948+0000 7ff7ad200700 1 Processor -- start 2024-05-27T07:31:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.948+0000 7ff7ad200700 1 -- start start 2024-05-27T07:31:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.949+0000 7ff7ad200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7a810a1d0 0x7ff7a819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.949+0000 7ff7ad200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a8110040 con 0x7ff7a810a1d0 2024-05-27T07:31:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.949+0000 7ff7a5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7a810a1d0 0x7ff7a819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.949+0000 7ff7a5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7a810a1d0 0x7ff7a819fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54684/0 (socket says 172.21.15.90:54684) 2024-05-27T07:31:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.949+0000 7ff7a5e00700 1 -- 172.21.15.90:0/4186596837 learned_addr learned my addr 172.21.15.90:0/4186596837 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:17.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.950+0000 7ff7a5e00700 1 -- 172.21.15.90:0/4186596837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a81a0170 con 0x7ff7a810a1d0 2024-05-27T07:31:17.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.950+0000 7ff7a5e00700 1 --2- 172.21.15.90:0/4186596837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7a810a1d0 0x7ff7a819fc30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff798005760 tx=0x7ff798004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:17.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.950+0000 7ff79ea00700 1 -- 172.21.15.90:0/4186596837 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff798027070 con 0x7ff7a810a1d0 2024-05-27T07:31:17.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.950+0000 7ff79ea00700 1 -- 172.21.15.90:0/4186596837 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff79800bdf0 con 0x7ff7a810a1d0 2024-05-27T07:31:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.950+0000 7ff79ea00700 1 -- 172.21.15.90:0/4186596837 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff79800fd70 con 0x7ff7a810a1d0 2024-05-27T07:31:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.950+0000 7ff7ad200700 1 -- 172.21.15.90:0/4186596837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a81a0400 con 0x7ff7a810a1d0 2024-05-27T07:31:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.951+0000 7ff7ad200700 1 -- 172.21.15.90:0/4186596837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a81a3010 con 0x7ff7a810a1d0 2024-05-27T07:31:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.952+0000 7ff7ad200700 1 -- 172.21.15.90:0/4186596837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a81107c0 con 0x7ff7a810a1d0 2024-05-27T07:31:17.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.952+0000 7ff79ea00700 1 -- 172.21.15.90:0/4186596837 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff79802a070 con 0x7ff7a810a1d0 2024-05-27T07:31:17.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.952+0000 7ff79ea00700 1 --2- 172.21.15.90:0/4186596837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff78c03e510 0x7ff78c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:17.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.952+0000 7ff79ea00700 1 -- 172.21.15.90:0/4186596837 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff7980539b0 con 0x7ff7a810a1d0 2024-05-27T07:31:17.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.953+0000 7ff7a5400700 1 --2- 172.21.15.90:0/4186596837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff78c03e510 0x7ff78c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:17.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.953+0000 7ff7a5400700 1 --2- 172.21.15.90:0/4186596837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff78c03e510 0x7ff78c040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff794006fd0 tx=0x7ff794006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:17.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:17.958+0000 7ff79ea00700 1 -- 172.21.15.90:0/4186596837 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff798013070 con 0x7ff7a810a1d0 2024-05-27T07:31:18.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.190+0000 7ff7ad200700 1 -- 172.21.15.90:0/4186596837 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff7a802e6f0 con 0x7ff78c03e510 2024-05-27T07:31:18.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.199+0000 7ff79ea00700 1 -- 172.21.15.90:0/4186596837 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff7a802e6f0 con 0x7ff78c03e510 2024-05-27T07:31:18.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.202+0000 7ff7ad200700 1 -- 172.21.15.90:0/4186596837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff78c03e510 msgr2=0x7ff78c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:18.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.202+0000 7ff7ad200700 1 --2- 172.21.15.90:0/4186596837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff78c03e510 0x7ff78c040a20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff794006fd0 tx=0x7ff794006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:18.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.202+0000 7ff7ad200700 1 -- 172.21.15.90:0/4186596837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7a810a1d0 msgr2=0x7ff7a819fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:18.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.202+0000 7ff7ad200700 1 --2- 172.21.15.90:0/4186596837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7a810a1d0 0x7ff7a819fc30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff798005760 tx=0x7ff798004f60 comp rx=0 tx=0).stop 2024-05-27T07:31:18.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.202+0000 7ff7ad200700 1 -- 172.21.15.90:0/4186596837 shutdown_connections 2024-05-27T07:31:18.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.202+0000 7ff7ad200700 1 --2- 172.21.15.90:0/4186596837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff78c03e510 0x7ff78c040a20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.202+0000 7ff7ad200700 1 --2- 172.21.15.90:0/4186596837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7a810a1d0 0x7ff7a819fc30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:18.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.202+0000 7ff7ad200700 1 -- 172.21.15.90:0/4186596837 >> 172.21.15.90:0/4186596837 conn(0x7ff7a8079240 msgr2=0x7ff7a8079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:18.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.202+0000 7ff7ad200700 1 -- 172.21.15.90:0/4186596837 shutdown_connections 2024-05-27T07:31:18.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:18.202+0000 7ff7ad200700 1 -- 172.21.15.90:0/4186596837 wait complete. 2024-05-27T07:31:18.976 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi129 2024-05-27T07:31:18.977 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:31:18.977 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.conf 2024-05-27T07:31:19.034 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:31:19.035 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:31:19.130 INFO:tasks.cephadm:Adding host smithi129 to orchestrator... 2024-05-27T07:31:19.130 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch host add smithi129 2024-05-27T07:31:19.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:19 smithi090 ceph-mon[31429]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:19.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:19 smithi090 ceph-mon[31429]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:31:19.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:19.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:31:19.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:31:19.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:31:19.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:19 smithi090 ceph-mon[31429]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-27T07:31:19.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:19 smithi090 ceph-mon[31429]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:31:19.536 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:20.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:20 smithi090 ceph-mon[31429]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:31:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.654+0000 7f196e400700 1 -- 172.21.15.90:0/474347582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1968104d20 msgr2=0x7f1968105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.654+0000 7f196e400700 1 --2- 172.21.15.90:0/474347582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1968104d20 0x7f1968105130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1958009910 tx=0x7f1958009c20 comp rx=0 tx=0).stop 2024-05-27T07:31:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.654+0000 7f196e400700 1 -- 172.21.15.90:0/474347582 shutdown_connections 2024-05-27T07:31:21.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.654+0000 7f196e400700 1 --2- 172.21.15.90:0/474347582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1968104d20 0x7f1968105130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.654+0000 7f196e400700 1 -- 172.21.15.90:0/474347582 >> 172.21.15.90:0/474347582 conn(0x7f19680785a0 msgr2=0x7f1968078a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.654+0000 7f196e400700 1 -- 172.21.15.90:0/474347582 shutdown_connections 2024-05-27T07:31:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.654+0000 7f196e400700 1 -- 172.21.15.90:0/474347582 wait complete. 2024-05-27T07:31:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.655+0000 7f196e400700 1 Processor -- start 2024-05-27T07:31:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.656+0000 7f196e400700 1 -- start start 2024-05-27T07:31:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.656+0000 7f196e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1968104d20 0x7f19681a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.656+0000 7f196e400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f196810dfd0 con 0x7f1968104d20 2024-05-27T07:31:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.656+0000 7f1967400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1968104d20 0x7f19681a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:21.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.656+0000 7f1967400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1968104d20 0x7f19681a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52746/0 (socket says 172.21.15.90:52746) 2024-05-27T07:31:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.656+0000 7f1967400700 1 -- 172.21.15.90:0/4238041204 learned_addr learned my addr 172.21.15.90:0/4238041204 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.657+0000 7f1967400700 1 -- 172.21.15.90:0/4238041204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19681a1c30 con 0x7f1968104d20 2024-05-27T07:31:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.657+0000 7f1967400700 1 --2- 172.21.15.90:0/4238041204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1968104d20 0x7f19681a16f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1958006010 tx=0x7f1958005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:21.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.657+0000 7f195fe00700 1 -- 172.21.15.90:0/4238041204 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1958026070 con 0x7f1968104d20 2024-05-27T07:31:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.657+0000 7f195fe00700 1 -- 172.21.15.90:0/4238041204 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f195800b750 con 0x7f1968104d20 2024-05-27T07:31:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.657+0000 7f195fe00700 1 -- 172.21.15.90:0/4238041204 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1958003810 con 0x7f1968104d20 2024-05-27T07:31:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.658+0000 7f196e400700 1 -- 172.21.15.90:0/4238041204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19681a1ec0 con 0x7f1968104d20 2024-05-27T07:31:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.658+0000 7f196e400700 1 -- 172.21.15.90:0/4238041204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196819b910 con 0x7f1968104d20 2024-05-27T07:31:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.659+0000 7f196e400700 1 -- 172.21.15.90:0/4238041204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196806b620 con 0x7f1968104d20 2024-05-27T07:31:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.660+0000 7f195fe00700 1 -- 172.21.15.90:0/4238041204 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f195800bbb0 con 0x7f1968104d20 2024-05-27T07:31:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.660+0000 7f195fe00700 1 --2- 172.21.15.90:0/4238041204 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f194c03e570 0x7f194c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.660+0000 7f195fe00700 1 -- 172.21.15.90:0/4238041204 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f195801e070 con 0x7f1968104d20 2024-05-27T07:31:21.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.661+0000 7f1966a00700 1 --2- 172.21.15.90:0/4238041204 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f194c03e570 0x7f194c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:21.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.661+0000 7f1966a00700 1 --2- 172.21.15.90:0/4238041204 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f194c03e570 0x7f194c040a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1954006fd0 tx=0x7f1954006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:21.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.666+0000 7f195fe00700 1 -- 172.21.15.90:0/4238041204 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1958013070 con 0x7f1968104d20 2024-05-27T07:31:21.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:21.901+0000 7f196e400700 1 -- 172.21.15.90:0/4238041204 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}) -- 0x7f196819ce40 con 0x7f194c03e570 2024-05-27T07:31:21.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:21 smithi090 ceph-mon[31429]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:31:21.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:21 smithi090 ceph-mon[31429]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:21.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:21 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:21.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:21 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:21.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:21 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:23.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:23 smithi090 ceph-mon[31429]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:31:23.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:23 smithi090 ceph-mon[31429]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:24.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:24 smithi090 ceph-mon[31429]: Deploying cephadm binary to smithi129 2024-05-27T07:31:25.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:25 smithi090 ceph-mon[31429]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:27.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:27 smithi090 ceph-mon[31429]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:27.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.994+0000 7f195fe00700 1 -- 172.21.15.90:0/4238041204 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f196819ce40 con 0x7f194c03e570 2024-05-27T07:31:27.990 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi129' with addr '172.21.15.129' 2024-05-27T07:31:27.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.998+0000 7f196e400700 1 -- 172.21.15.90:0/4238041204 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f194c03e570 msgr2=0x7f194c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:27.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.998+0000 7f196e400700 1 --2- 172.21.15.90:0/4238041204 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f194c03e570 0x7f194c040a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1954006fd0 tx=0x7f1954006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:27.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.998+0000 7f196e400700 1 -- 172.21.15.90:0/4238041204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1968104d20 msgr2=0x7f19681a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:27.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.998+0000 7f196e400700 1 --2- 172.21.15.90:0/4238041204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1968104d20 0x7f19681a16f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1958006010 tx=0x7f1958005710 comp rx=0 tx=0).stop 2024-05-27T07:31:27.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.999+0000 7f196e400700 1 -- 172.21.15.90:0/4238041204 shutdown_connections 2024-05-27T07:31:27.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.999+0000 7f196e400700 1 --2- 172.21.15.90:0/4238041204 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f194c03e570 0x7f194c040a80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:27.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.999+0000 7f196e400700 1 --2- 172.21.15.90:0/4238041204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1968104d20 0x7f19681a16f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:27.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.999+0000 7f196e400700 1 -- 172.21.15.90:0/4238041204 >> 172.21.15.90:0/4238041204 conn(0x7f19680785a0 msgr2=0x7f1968109f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:27.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.999+0000 7f196e400700 1 -- 172.21.15.90:0/4238041204 shutdown_connections 2024-05-27T07:31:27.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:27.999+0000 7f196e400700 1 -- 172.21.15.90:0/4238041204 wait complete. 2024-05-27T07:31:28.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T07:31:29.168 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:29.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:28 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:29.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:28 smithi090 ceph-mon[31429]: Added host smithi129 2024-05-27T07:31:29.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:28 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:31:29.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:28 smithi090 ceph-mon[31429]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:29.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:28 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:30.901 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:30 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:31.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.182+0000 7f45c2600700 1 -- 172.21.15.90:0/2618536717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45bc105fb0 msgr2=0x7f45bc1063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:31.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.182+0000 7f45c2600700 1 --2- 172.21.15.90:0/2618536717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45bc105fb0 0x7f45bc1063c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009910 tx=0x7f45ac009c20 comp rx=0 tx=0).stop 2024-05-27T07:31:31.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.183+0000 7f45c2600700 1 -- 172.21.15.90:0/2618536717 shutdown_connections 2024-05-27T07:31:31.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.183+0000 7f45c2600700 1 --2- 172.21.15.90:0/2618536717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45bc105fb0 0x7f45bc1063c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.183+0000 7f45c2600700 1 -- 172.21.15.90:0/2618536717 >> 172.21.15.90:0/2618536717 conn(0x7f45bc101840 msgr2=0x7f45bc103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:31.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.183+0000 7f45c2600700 1 -- 172.21.15.90:0/2618536717 shutdown_connections 2024-05-27T07:31:31.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.183+0000 7f45c2600700 1 -- 172.21.15.90:0/2618536717 wait complete. 2024-05-27T07:31:31.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.184+0000 7f45c2600700 1 Processor -- start 2024-05-27T07:31:31.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.184+0000 7f45c2600700 1 -- start start 2024-05-27T07:31:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.185+0000 7f45c2600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45bc105fb0 0x7f45bc19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.185+0000 7f45c2600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc110150 con 0x7f45bc105fb0 2024-05-27T07:31:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.185+0000 7f45bb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45bc105fb0 0x7f45bc19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.185+0000 7f45bb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45bc105fb0 0x7f45bc19bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41318/0 (socket says 172.21.15.90:41318) 2024-05-27T07:31:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.185+0000 7f45bb400700 1 -- 172.21.15.90:0/3090384209 learned_addr learned my addr 172.21.15.90:0/3090384209 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.185+0000 7f45bb400700 1 -- 172.21.15.90:0/3090384209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc19c060 con 0x7f45bc105fb0 2024-05-27T07:31:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.186+0000 7f45bb400700 1 --2- 172.21.15.90:0/3090384209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45bc105fb0 0x7f45bc19bb20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45ac006010 tx=0x7f45ac005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.186+0000 7f45b3e00700 1 -- 172.21.15.90:0/3090384209 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45ac026070 con 0x7f45bc105fb0 2024-05-27T07:31:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.186+0000 7f45b3e00700 1 -- 172.21.15.90:0/3090384209 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45ac00b6f0 con 0x7f45bc105fb0 2024-05-27T07:31:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.186+0000 7f45b3e00700 1 -- 172.21.15.90:0/3090384209 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45ac003810 con 0x7f45bc105fb0 2024-05-27T07:31:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.186+0000 7f45c2600700 1 -- 172.21.15.90:0/3090384209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc19c350 con 0x7f45bc105fb0 2024-05-27T07:31:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.186+0000 7f45c2600700 1 -- 172.21.15.90:0/3090384209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc1a0360 con 0x7f45bc105fb0 2024-05-27T07:31:31.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.187+0000 7f45c2600700 1 -- 172.21.15.90:0/3090384209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45bc107460 con 0x7f45bc105fb0 2024-05-27T07:31:31.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.188+0000 7f45b3e00700 1 -- 172.21.15.90:0/3090384209 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f45ac029070 con 0x7f45bc105fb0 2024-05-27T07:31:31.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.188+0000 7f45b3e00700 1 --2- 172.21.15.90:0/3090384209 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f45a003e570 0x7f45a0040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:31.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.188+0000 7f45b3e00700 1 -- 172.21.15.90:0/3090384209 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f45ac01e070 con 0x7f45bc105fb0 2024-05-27T07:31:31.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.189+0000 7f45baa00700 1 --2- 172.21.15.90:0/3090384209 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f45a003e570 0x7f45a0040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:31.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.189+0000 7f45baa00700 1 --2- 172.21.15.90:0/3090384209 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f45a003e570 0x7f45a0040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f45a8006fd0 tx=0x7f45a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:31.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.194+0000 7f45b3e00700 1 -- 172.21.15.90:0/3090384209 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45ac013070 con 0x7f45bc105fb0 2024-05-27T07:31:31.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.432+0000 7f45c2600700 1 -- 172.21.15.90:0/3090384209 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45bc19ce60 con 0x7f45a003e570 2024-05-27T07:31:31.427 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:31.427 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}] 2024-05-27T07:31:31.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.434+0000 7f45b3e00700 1 -- 172.21.15.90:0/3090384209 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f45bc19ce60 con 0x7f45a003e570 2024-05-27T07:31:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.437+0000 7f45c2600700 1 -- 172.21.15.90:0/3090384209 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f45a003e570 msgr2=0x7f45a0040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.437+0000 7f45c2600700 1 --2- 172.21.15.90:0/3090384209 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f45a003e570 0x7f45a0040a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f45a8006fd0 tx=0x7f45a8006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.437+0000 7f45c2600700 1 -- 172.21.15.90:0/3090384209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45bc105fb0 msgr2=0x7f45bc19bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.437+0000 7f45c2600700 1 --2- 172.21.15.90:0/3090384209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45bc105fb0 0x7f45bc19bb20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45ac006010 tx=0x7f45ac005710 comp rx=0 tx=0).stop 2024-05-27T07:31:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.437+0000 7f45c2600700 1 -- 172.21.15.90:0/3090384209 shutdown_connections 2024-05-27T07:31:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.437+0000 7f45c2600700 1 --2- 172.21.15.90:0/3090384209 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f45a003e570 0x7f45a0040a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.437+0000 7f45c2600700 1 --2- 172.21.15.90:0/3090384209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45bc105fb0 0x7f45bc19bb20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.437+0000 7f45c2600700 1 -- 172.21.15.90:0/3090384209 >> 172.21.15.90:0/3090384209 conn(0x7f45bc101840 msgr2=0x7f45bc10bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.437+0000 7f45c2600700 1 -- 172.21.15.90:0/3090384209 shutdown_connections 2024-05-27T07:31:31.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:31.437+0000 7f45c2600700 1 -- 172.21.15.90:0/3090384209 wait complete. 2024-05-27T07:31:31.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:31 smithi090 ceph-mon[31429]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:32.128 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-05-27T07:31:32.128 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:31:32.128 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-05-27T07:31:32.186 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:31:32.187 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:31:32.283 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-05-27T07:31:32.283 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch host add smithi136 2024-05-27T07:31:32.729 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:33.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:32 smithi090 ceph-mon[31429]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:31:33.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:32 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:33.934 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:33 smithi090 ceph-mon[31429]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:34.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.845+0000 7ff197000700 1 -- 172.21.15.90:0/760767081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff19010a3c0 msgr2=0x7ff19010a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:34.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.845+0000 7ff197000700 1 --2- 172.21.15.90:0/760767081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff19010a3c0 0x7ff19010a7d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff180009910 tx=0x7ff180009c20 comp rx=0 tx=0).stop 2024-05-27T07:31:34.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.845+0000 7ff197000700 1 -- 172.21.15.90:0/760767081 shutdown_connections 2024-05-27T07:31:34.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.845+0000 7ff197000700 1 --2- 172.21.15.90:0/760767081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff19010a3c0 0x7ff19010a7d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:34.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.845+0000 7ff197000700 1 -- 172.21.15.90:0/760767081 >> 172.21.15.90:0/760767081 conn(0x7ff1900799e0 msgr2=0x7ff1901085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:34.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.845+0000 7ff197000700 1 -- 172.21.15.90:0/760767081 shutdown_connections 2024-05-27T07:31:34.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.846+0000 7ff197000700 1 -- 172.21.15.90:0/760767081 wait complete. 2024-05-27T07:31:34.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.847+0000 7ff197000700 1 Processor -- start 2024-05-27T07:31:34.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.847+0000 7ff197000700 1 -- start start 2024-05-27T07:31:34.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.847+0000 7ff197000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff19010a3c0 0x7ff19019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:34.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.847+0000 7ff197000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff190114560 con 0x7ff19010a3c0 2024-05-27T07:31:34.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.848+0000 7ff18fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff19010a3c0 0x7ff19019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:34.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.848+0000 7ff18fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff19010a3c0 0x7ff19019fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41340/0 (socket says 172.21.15.90:41340) 2024-05-27T07:31:34.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.848+0000 7ff18fe00700 1 -- 172.21.15.90:0/2287376514 learned_addr learned my addr 172.21.15.90:0/2287376514 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:34.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.848+0000 7ff18fe00700 1 -- 172.21.15.90:0/2287376514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1901a03c0 con 0x7ff19010a3c0 2024-05-27T07:31:34.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.849+0000 7ff18fe00700 1 --2- 172.21.15.90:0/2287376514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff19010a3c0 0x7ff19019fe80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff180006010 tx=0x7ff180005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:34.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.849+0000 7ff18cc00700 1 -- 172.21.15.90:0/2287376514 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff180026070 con 0x7ff19010a3c0 2024-05-27T07:31:34.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.849+0000 7ff18cc00700 1 -- 172.21.15.90:0/2287376514 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff18000b6f0 con 0x7ff19010a3c0 2024-05-27T07:31:34.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.849+0000 7ff197000700 1 -- 172.21.15.90:0/2287376514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1901a06b0 con 0x7ff19010a3c0 2024-05-27T07:31:34.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.849+0000 7ff18cc00700 1 -- 172.21.15.90:0/2287376514 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff180003810 con 0x7ff19010a3c0 2024-05-27T07:31:34.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.849+0000 7ff197000700 1 -- 172.21.15.90:0/2287376514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1901a4660 con 0x7ff19010a3c0 2024-05-27T07:31:34.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.850+0000 7ff197000700 1 -- 172.21.15.90:0/2287376514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19010b870 con 0x7ff19010a3c0 2024-05-27T07:31:34.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.850+0000 7ff18cc00700 1 -- 172.21.15.90:0/2287376514 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff180029070 con 0x7ff19010a3c0 2024-05-27T07:31:34.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.851+0000 7ff18cc00700 1 --2- 172.21.15.90:0/2287376514 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff17403e570 0x7ff174040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:34.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.851+0000 7ff18cc00700 1 -- 172.21.15.90:0/2287376514 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff18001e070 con 0x7ff19010a3c0 2024-05-27T07:31:34.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.851+0000 7ff18f400700 1 --2- 172.21.15.90:0/2287376514 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff17403e570 0x7ff174040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:34.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.852+0000 7ff18f400700 1 --2- 172.21.15.90:0/2287376514 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff17403e570 0x7ff174040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff17c006fd0 tx=0x7ff17c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:34.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:34.857+0000 7ff18cc00700 1 -- 172.21.15.90:0/2287376514 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff180013070 con 0x7ff19010a3c0 2024-05-27T07:31:35.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:35.093+0000 7ff197000700 1 -- 172.21.15.90:0/2287376514 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}) -- 0x7ff1901a1190 con 0x7ff17403e570 2024-05-27T07:31:36.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:35 smithi090 ceph-mon[31429]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:37.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:36 smithi090 ceph-mon[31429]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:31:37.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:37.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:37.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:37.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:37.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:31:37.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:31:37.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:31:38.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:37 smithi090 ceph-mon[31429]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:38.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:37 smithi090 ceph-mon[31429]: Deploying cephadm binary to smithi136 2024-05-27T07:31:38.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:37 smithi090 ceph-mon[31429]: Updating smithi129:/etc/ceph/ceph.conf 2024-05-27T07:31:40.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:39 smithi090 ceph-mon[31429]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:41.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.174+0000 7ff18cc00700 1 -- 172.21.15.90:0/2287376514 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff1901a1190 con 0x7ff17403e570 2024-05-27T07:31:41.167 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-05-27T07:31:41.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:40 smithi090 ceph-mon[31429]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:31:41.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.178+0000 7ff197000700 1 -- 172.21.15.90:0/2287376514 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff17403e570 msgr2=0x7ff174040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.178+0000 7ff197000700 1 --2- 172.21.15.90:0/2287376514 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff17403e570 0x7ff174040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff17c006fd0 tx=0x7ff17c006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.178+0000 7ff197000700 1 -- 172.21.15.90:0/2287376514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff19010a3c0 msgr2=0x7ff19019fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.178+0000 7ff197000700 1 --2- 172.21.15.90:0/2287376514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff19010a3c0 0x7ff19019fe80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff180006010 tx=0x7ff180005710 comp rx=0 tx=0).stop 2024-05-27T07:31:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.179+0000 7ff197000700 1 -- 172.21.15.90:0/2287376514 shutdown_connections 2024-05-27T07:31:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.179+0000 7ff197000700 1 --2- 172.21.15.90:0/2287376514 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff17403e570 0x7ff174040a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.179+0000 7ff197000700 1 --2- 172.21.15.90:0/2287376514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff19010a3c0 0x7ff19019fe80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.179+0000 7ff197000700 1 -- 172.21.15.90:0/2287376514 >> 172.21.15.90:0/2287376514 conn(0x7ff1900799e0 msgr2=0x7ff19010bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:41.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.179+0000 7ff197000700 1 -- 172.21.15.90:0/2287376514 shutdown_connections 2024-05-27T07:31:41.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:41.179+0000 7ff197000700 1 -- 172.21.15.90:0/2287376514 wait complete. 2024-05-27T07:31:41.924 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T07:31:42.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:41 smithi090 ceph-mon[31429]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:42.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:41 smithi090 ceph-mon[31429]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:31:42.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:42.351 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:43.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:42 smithi090 ceph-mon[31429]: Added host smithi136 2024-05-27T07:31:43.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:42 smithi090 ceph-mon[31429]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:31:43.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:42 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:43.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:42 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:43.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:42 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:43.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:42 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:31:44.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:43 smithi090 ceph-mon[31429]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:44.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:43 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:44.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.280+0000 7f7bace00700 1 -- 172.21.15.90:0/1809432152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba8077ce0 msgr2=0x7f7ba80780f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:44.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.280+0000 7f7bace00700 1 --2- 172.21.15.90:0/1809432152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba8077ce0 0x7f7ba80780f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009910 tx=0x7f7b98009c20 comp rx=0 tx=0).stop 2024-05-27T07:31:44.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.280+0000 7f7bace00700 1 -- 172.21.15.90:0/1809432152 shutdown_connections 2024-05-27T07:31:44.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.280+0000 7f7bace00700 1 --2- 172.21.15.90:0/1809432152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba8077ce0 0x7f7ba80780f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.280+0000 7f7bace00700 1 -- 172.21.15.90:0/1809432152 >> 172.21.15.90:0/1809432152 conn(0x7f7ba8101840 msgr2=0x7f7ba8103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:44.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.281+0000 7f7bace00700 1 -- 172.21.15.90:0/1809432152 shutdown_connections 2024-05-27T07:31:44.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.281+0000 7f7bace00700 1 -- 172.21.15.90:0/1809432152 wait complete. 2024-05-27T07:31:44.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.282+0000 7f7bace00700 1 Processor -- start 2024-05-27T07:31:44.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.282+0000 7f7bace00700 1 -- start start 2024-05-27T07:31:44.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.282+0000 7f7bace00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba8077ce0 0x7f7ba819fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:44.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.282+0000 7f7bace00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba81145b0 con 0x7f7ba8077ce0 2024-05-27T07:31:44.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.283+0000 7f7ba5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba8077ce0 0x7f7ba819fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:44.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.283+0000 7f7ba5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba8077ce0 0x7f7ba819fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43148/0 (socket says 172.21.15.90:43148) 2024-05-27T07:31:44.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.283+0000 7f7ba5e00700 1 -- 172.21.15.90:0/1960169569 learned_addr learned my addr 172.21.15.90:0/1960169569 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:44.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.283+0000 7f7ba5e00700 1 -- 172.21.15.90:0/1960169569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba81a0400 con 0x7f7ba8077ce0 2024-05-27T07:31:44.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.283+0000 7f7ba5e00700 1 --2- 172.21.15.90:0/1960169569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba8077ce0 0x7f7ba819fec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b98005660 tx=0x7f7b98005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:44.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.284+0000 7f7b9ea00700 1 -- 172.21.15.90:0/1960169569 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b98026070 con 0x7f7ba8077ce0 2024-05-27T07:31:44.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.284+0000 7f7b9ea00700 1 -- 172.21.15.90:0/1960169569 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b9800b750 con 0x7f7ba8077ce0 2024-05-27T07:31:44.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.284+0000 7f7b9ea00700 1 -- 172.21.15.90:0/1960169569 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b980037b0 con 0x7f7ba8077ce0 2024-05-27T07:31:44.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.284+0000 7f7bace00700 1 -- 172.21.15.90:0/1960169569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba81a0690 con 0x7f7ba8077ce0 2024-05-27T07:31:44.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.284+0000 7f7bace00700 1 -- 172.21.15.90:0/1960169569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba81a0af0 con 0x7f7ba8077ce0 2024-05-27T07:31:44.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.285+0000 7f7bace00700 1 -- 172.21.15.90:0/1960169569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba8079420 con 0x7f7ba8077ce0 2024-05-27T07:31:44.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.285+0000 7f7b9ea00700 1 -- 172.21.15.90:0/1960169569 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7b98029070 con 0x7f7ba8077ce0 2024-05-27T07:31:44.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.286+0000 7f7b9ea00700 1 --2- 172.21.15.90:0/1960169569 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7b8c03e510 0x7f7b8c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:44.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.286+0000 7f7b9ea00700 1 -- 172.21.15.90:0/1960169569 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7b9801e070 con 0x7f7ba8077ce0 2024-05-27T07:31:44.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.286+0000 7f7ba5400700 1 --2- 172.21.15.90:0/1960169569 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7b8c03e510 0x7f7b8c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:44.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.287+0000 7f7ba5400700 1 --2- 172.21.15.90:0/1960169569 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7b8c03e510 0x7f7b8c040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7b94006fd0 tx=0x7f7b94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:44.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.293+0000 7f7b9ea00700 1 -- 172.21.15.90:0/1960169569 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7b98013070 con 0x7f7ba8077ce0 2024-05-27T07:31:44.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.531+0000 7f7bace00700 1 -- 172.21.15.90:0/1960169569 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7ba81a1220 con 0x7f7b8c03e510 2024-05-27T07:31:44.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.533+0000 7f7b9ea00700 1 -- 172.21.15.90:0/1960169569 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f7ba81a1220 con 0x7f7b8c03e510 2024-05-27T07:31:44.526 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:31:44.526 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-05-27T07:31:44.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.536+0000 7f7bace00700 1 -- 172.21.15.90:0/1960169569 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7b8c03e510 msgr2=0x7f7b8c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:44.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.536+0000 7f7bace00700 1 --2- 172.21.15.90:0/1960169569 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7b8c03e510 0x7f7b8c040a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7b94006fd0 tx=0x7f7b94006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:44.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.536+0000 7f7bace00700 1 -- 172.21.15.90:0/1960169569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba8077ce0 msgr2=0x7f7ba819fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:44.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.536+0000 7f7bace00700 1 --2- 172.21.15.90:0/1960169569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba8077ce0 0x7f7ba819fec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b98005660 tx=0x7f7b98005740 comp rx=0 tx=0).stop 2024-05-27T07:31:44.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.536+0000 7f7bace00700 1 -- 172.21.15.90:0/1960169569 shutdown_connections 2024-05-27T07:31:44.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.536+0000 7f7bace00700 1 --2- 172.21.15.90:0/1960169569 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7b8c03e510 0x7f7b8c040a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.536+0000 7f7bace00700 1 --2- 172.21.15.90:0/1960169569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ba8077ce0 0x7f7ba819fec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:44.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.536+0000 7f7bace00700 1 -- 172.21.15.90:0/1960169569 >> 172.21.15.90:0/1960169569 conn(0x7f7ba8101840 msgr2=0x7f7ba8102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:44.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.536+0000 7f7bace00700 1 -- 172.21.15.90:0/1960169569 shutdown_connections 2024-05-27T07:31:44.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:44.536+0000 7f7bace00700 1 -- 172.21.15.90:0/1960169569 wait complete. 2024-05-27T07:31:45.240 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-27T07:31:45.241 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-05-27T07:31:45.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:45 smithi090 ceph-mon[31429]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:45.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:45.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:45 smithi090 ceph-mon[31429]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:31:45.673 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:31:47.433 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:47 smithi090 ceph-mon[31429]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:47.433 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:47.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.697+0000 7fd718200700 1 -- 172.21.15.90:0/3869159094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd710075b50 msgr2=0x7fd710075f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:47.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.697+0000 7fd718200700 1 --2- 172.21.15.90:0/3869159094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd710075b50 0x7fd710075f60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd704009910 tx=0x7fd704009c20 comp rx=0 tx=0).stop 2024-05-27T07:31:47.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.698+0000 7fd718200700 1 -- 172.21.15.90:0/3869159094 shutdown_connections 2024-05-27T07:31:47.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.698+0000 7fd718200700 1 --2- 172.21.15.90:0/3869159094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd710075b50 0x7fd710075f60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:47.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.698+0000 7fd718200700 1 -- 172.21.15.90:0/3869159094 >> 172.21.15.90:0/3869159094 conn(0x7fd7100ff690 msgr2=0x7fd710101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:47.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.698+0000 7fd718200700 1 -- 172.21.15.90:0/3869159094 shutdown_connections 2024-05-27T07:31:47.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.698+0000 7fd718200700 1 -- 172.21.15.90:0/3869159094 wait complete. 2024-05-27T07:31:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.699+0000 7fd718200700 1 Processor -- start 2024-05-27T07:31:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.699+0000 7fd718200700 1 -- start start 2024-05-27T07:31:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.700+0000 7fd718200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd710075b50 0x7fd71019ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.700+0000 7fd718200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd7101145f0 con 0x7fd710075b50 2024-05-27T07:31:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.700+0000 7fd715800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd710075b50 0x7fd71019ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:47.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.700+0000 7fd715800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd710075b50 0x7fd71019ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43168/0 (socket says 172.21.15.90:43168) 2024-05-27T07:31:47.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.700+0000 7fd715800700 1 -- 172.21.15.90:0/1719855834 learned_addr learned my addr 172.21.15.90:0/1719855834 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:31:47.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.700+0000 7fd715800700 1 -- 172.21.15.90:0/1719855834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7101a0460 con 0x7fd710075b50 2024-05-27T07:31:47.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.701+0000 7fd715800700 1 --2- 172.21.15.90:0/1719855834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd710075b50 0x7fd71019ff20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd704006010 tx=0x7fd704005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:47.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.701+0000 7fd70e000700 1 -- 172.21.15.90:0/1719855834 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd704026070 con 0x7fd710075b50 2024-05-27T07:31:47.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.701+0000 7fd70e000700 1 -- 172.21.15.90:0/1719855834 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd70400b750 con 0x7fd710075b50 2024-05-27T07:31:47.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.701+0000 7fd718200700 1 -- 172.21.15.90:0/1719855834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7101a06f0 con 0x7fd710075b50 2024-05-27T07:31:47.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.701+0000 7fd70e000700 1 -- 172.21.15.90:0/1719855834 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7040038a0 con 0x7fd710075b50 2024-05-27T07:31:47.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.701+0000 7fd718200700 1 -- 172.21.15.90:0/1719855834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7101a0b50 con 0x7fd710075b50 2024-05-27T07:31:47.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.702+0000 7fd718200700 1 -- 172.21.15.90:0/1719855834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd710077290 con 0x7fd710075b50 2024-05-27T07:31:47.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.702+0000 7fd70e000700 1 -- 172.21.15.90:0/1719855834 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd704029070 con 0x7fd710075b50 2024-05-27T07:31:47.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.703+0000 7fd70e000700 1 --2- 172.21.15.90:0/1719855834 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd6f803e570 0x7fd6f8040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:47.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.703+0000 7fd70e000700 1 -- 172.21.15.90:0/1719855834 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd70401e070 con 0x7fd710075b50 2024-05-27T07:31:47.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.704+0000 7fd714e00700 1 --2- 172.21.15.90:0/1719855834 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd6f803e570 0x7fd6f8040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:47.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.704+0000 7fd714e00700 1 --2- 172.21.15.90:0/1719855834 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd6f803e570 0x7fd6f8040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc006fd0 tx=0x7fd6fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:47.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.709+0000 7fd70e000700 1 -- 172.21.15.90:0/1719855834 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd704013070 con 0x7fd710075b50 2024-05-27T07:31:47.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:47.946+0000 7fd718200700 1 -- 172.21.15.90:0/1719855834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd7101a1280 con 0x7fd710075b50 2024-05-27T07:31:48.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.371+0000 7fd70e000700 1 -- 172.21.15.90:0/1719855834 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fd704015350 con 0x7fd710075b50 2024-05-27T07:31:48.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.375+0000 7fd718200700 1 -- 172.21.15.90:0/1719855834 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd6f803e570 msgr2=0x7fd6f8040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:48.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.375+0000 7fd718200700 1 --2- 172.21.15.90:0/1719855834 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd6f803e570 0x7fd6f8040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc006fd0 tx=0x7fd6fc006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.375+0000 7fd718200700 1 -- 172.21.15.90:0/1719855834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd710075b50 msgr2=0x7fd71019ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.375+0000 7fd718200700 1 --2- 172.21.15.90:0/1719855834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd710075b50 0x7fd71019ff20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd704006010 tx=0x7fd704005710 comp rx=0 tx=0).stop 2024-05-27T07:31:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.375+0000 7fd718200700 1 -- 172.21.15.90:0/1719855834 shutdown_connections 2024-05-27T07:31:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.375+0000 7fd718200700 1 --2- 172.21.15.90:0/1719855834 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd6f803e570 0x7fd6f8040a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.375+0000 7fd718200700 1 --2- 172.21.15.90:0/1719855834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd710075b50 0x7fd71019ff20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.375+0000 7fd718200700 1 -- 172.21.15.90:0/1719855834 >> 172.21.15.90:0/1719855834 conn(0x7fd7100ff690 msgr2=0x7fd710102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.376+0000 7fd718200700 1 -- 172.21.15.90:0/1719855834 shutdown_connections 2024-05-27T07:31:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:31:48.376+0000 7fd718200700 1 -- 172.21.15.90:0/1719855834 wait complete. 2024-05-27T07:31:48.369 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2024-05-27T07:31:48.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:48 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1719855834' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T07:31:49.012 INFO:tasks.cephadm:Adding mon.a on smithi090 2024-05-27T07:31:49.012 INFO:tasks.cephadm:Adding mon.b on smithi129 2024-05-27T07:31:49.012 INFO:tasks.cephadm:Adding mon.c on smithi136 2024-05-27T07:31:49.012 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi090:172.21.15.90=a;smithi129:172.21.15.129=b;smithi136:172.21.15.136=c' 2024-05-27T07:31:49.414 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T07:31:49.519 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /etc/ceph/ceph.conf 2024-05-27T07:31:49.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:49 smithi090 ceph-mon[31429]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:49.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:49 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1719855834' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T07:31:49.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:49 smithi090 ceph-mon[31429]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:31:51.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:51 smithi090 ceph-mon[31429]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:51.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:51.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:51.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:51.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:51.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:31:51.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:31:51.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:31:52.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.156+0000 7f42e6200700 1 -- 172.21.15.136:0/3560465153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e0104760 msgr2=0x7f42e0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:52.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.156+0000 7f42e6200700 1 --2- 172.21.15.136:0/3560465153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e0104760 0x7f42e0106b80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009910 tx=0x7f42d0009c20 comp rx=0 tx=0).stop 2024-05-27T07:31:52.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.156+0000 7f42e6200700 1 -- 172.21.15.136:0/3560465153 shutdown_connections 2024-05-27T07:31:52.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.156+0000 7f42e6200700 1 --2- 172.21.15.136:0/3560465153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e0104760 0x7f42e0106b80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:52.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.156+0000 7f42e6200700 1 -- 172.21.15.136:0/3560465153 >> 172.21.15.136:0/3560465153 conn(0x7f42e00fe120 msgr2=0x7f42e01005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:52.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.157+0000 7f42e6200700 1 -- 172.21.15.136:0/3560465153 shutdown_connections 2024-05-27T07:31:52.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.157+0000 7f42e6200700 1 -- 172.21.15.136:0/3560465153 wait complete. 2024-05-27T07:31:52.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.158+0000 7f42e6200700 1 Processor -- start 2024-05-27T07:31:52.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.158+0000 7f42e6200700 1 -- start start 2024-05-27T07:31:52.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.158+0000 7f42e6200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e0104760 0x7f42e0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:52.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.158+0000 7f42e6200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f42e0109a70 con 0x7f42e0104760 2024-05-27T07:31:52.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.159+0000 7f42df400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e0104760 0x7f42e0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:52.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.159+0000 7f42df400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e0104760 0x7f42e0199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.136:33986/0 (socket says 172.21.15.136:33986) 2024-05-27T07:31:52.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.159+0000 7f42df400700 1 -- 172.21.15.136:0/1637828755 learned_addr learned my addr 172.21.15.136:0/1637828755 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-05-27T07:31:52.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.159+0000 7f42df400700 1 -- 172.21.15.136:0/1637828755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e0199b80 con 0x7f42e0104760 2024-05-27T07:31:52.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.160+0000 7f42df400700 1 --2- 172.21.15.136:0/1637828755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e0104760 0x7f42e0199640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005790 tx=0x7f42d0004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:52.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.160+0000 7f42dcc00700 1 -- 172.21.15.136:0/1637828755 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d0027070 con 0x7f42e0104760 2024-05-27T07:31:52.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.160+0000 7f42e6200700 1 -- 172.21.15.136:0/1637828755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e0199e10 con 0x7f42e0104760 2024-05-27T07:31:52.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.160+0000 7f42e6200700 1 -- 172.21.15.136:0/1637828755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e019ca20 con 0x7f42e0104760 2024-05-27T07:31:52.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.161+0000 7f42dcc00700 1 -- 172.21.15.136:0/1637828755 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42d0004830 con 0x7f42e0104760 2024-05-27T07:31:52.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.161+0000 7f42dcc00700 1 -- 172.21.15.136:0/1637828755 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d000fa40 con 0x7f42e0104760 2024-05-27T07:31:52.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.162+0000 7f42e6200700 1 -- 172.21.15.136:0/1637828755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42bc005360 con 0x7f42e0104760 2024-05-27T07:31:52.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.162+0000 7f42dcc00700 1 -- 172.21.15.136:0/1637828755 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f42d002a070 con 0x7f42e0104760 2024-05-27T07:31:52.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.163+0000 7f42dcc00700 1 --2- 172.21.15.136:0/1637828755 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42c403e570 0x7f42c4040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:52.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.163+0000 7f42dcc00700 1 -- 172.21.15.136:0/1637828755 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f42d0053a10 con 0x7f42e0104760 2024-05-27T07:31:52.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.163+0000 7f42d7e00700 1 --2- 172.21.15.136:0/1637828755 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42c403e570 0x7f42c4040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:52.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.164+0000 7f42d7e00700 1 --2- 172.21.15.136:0/1637828755 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42c403e570 0x7f42c4040a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f42c8006fd0 tx=0x7f42c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:52.171 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.179+0000 7f42dcc00700 1 -- 172.21.15.136:0/1637828755 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42d0013070 con 0x7f42e0104760 2024-05-27T07:31:52.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.437+0000 7f42e6200700 1 -- 172.21.15.136:0/1637828755 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi129:172.21.15.129=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}) -- 0x7f42bc000cb0 con 0x7f42c403e570 2024-05-27T07:31:52.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.457+0000 7f42dcc00700 1 -- 172.21.15.136:0/1637828755 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f42bc000cb0 con 0x7f42c403e570 2024-05-27T07:31:52.449 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mon update... 2024-05-27T07:31:52.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.460+0000 7f42e6200700 1 -- 172.21.15.136:0/1637828755 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42c403e570 msgr2=0x7f42c4040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:52.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.460+0000 7f42e6200700 1 --2- 172.21.15.136:0/1637828755 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42c403e570 0x7f42c4040a80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f42c8006fd0 tx=0x7f42c8006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:52.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.460+0000 7f42e6200700 1 -- 172.21.15.136:0/1637828755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e0104760 msgr2=0x7f42e0199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:52.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.460+0000 7f42e6200700 1 --2- 172.21.15.136:0/1637828755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e0104760 0x7f42e0199640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f42d0005790 tx=0x7f42d0004f60 comp rx=0 tx=0).stop 2024-05-27T07:31:52.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.460+0000 7f42e6200700 1 -- 172.21.15.136:0/1637828755 shutdown_connections 2024-05-27T07:31:52.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.460+0000 7f42e6200700 1 --2- 172.21.15.136:0/1637828755 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42c403e570 0x7f42c4040a80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:52.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.460+0000 7f42e6200700 1 --2- 172.21.15.136:0/1637828755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e0104760 0x7f42e0199640 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:52.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.460+0000 7f42e6200700 1 -- 172.21.15.136:0/1637828755 >> 172.21.15.136:0/1637828755 conn(0x7f42e00fe120 msgr2=0x7f42e00feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:52.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.460+0000 7f42e6200700 1 -- 172.21.15.136:0/1637828755 shutdown_connections 2024-05-27T07:31:52.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:52.460+0000 7f42e6200700 1 -- 172.21.15.136:0/1637828755 wait complete. 2024-05-27T07:31:52.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:52 smithi090 ceph-mon[31429]: Updating smithi136:/etc/ceph/ceph.conf 2024-05-27T07:31:53.227 DEBUG:teuthology.orchestra.run.smithi129:mon.b> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.b.service 2024-05-27T07:31:53.235 DEBUG:teuthology.orchestra.run.smithi136:mon.c> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.c.service 2024-05-27T07:31:53.240 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T07:31:53.240 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T07:31:53.674 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:31:53.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:53 smithi090 ceph-mon[31429]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:31:53.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:53 smithi090 ceph-mon[31429]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:53.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:53 smithi090 ceph-mon[31429]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi129:172.21.15.129=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:31:53.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:53 smithi090 ceph-mon[31429]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi129:172.21.15.129=b;smithi136:172.21.15.136=c;count:3 2024-05-27T07:31:53.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:53 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:53.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:53 smithi090 ceph-mon[31429]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:31:54.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:54 smithi090 ceph-mon[31429]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:31:54.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:54.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:54.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:31:54.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:31:54.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:31:55.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:55 smithi090 ceph-mon[31429]: Deploying daemon mon.c on smithi136 2024-05-27T07:31:55.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:55 smithi090 ceph-mon[31429]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:56.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.840+0000 7fd163e00700 1 -- 172.21.15.136:0/4124751597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd164109990 msgr2=0x7fd164109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:56.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.840+0000 7fd163e00700 1 --2- 172.21.15.136:0/4124751597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd164109990 0x7fd164109da0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd154009910 tx=0x7fd154009c20 comp rx=0 tx=0).stop 2024-05-27T07:31:56.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.840+0000 7fd163e00700 1 -- 172.21.15.136:0/4124751597 shutdown_connections 2024-05-27T07:31:56.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.840+0000 7fd163e00700 1 --2- 172.21.15.136:0/4124751597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd164109990 0x7fd164109da0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:56.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.840+0000 7fd163e00700 1 -- 172.21.15.136:0/4124751597 >> 172.21.15.136:0/4124751597 conn(0x7fd1640ff480 msgr2=0x7fd1641018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:56.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.840+0000 7fd163e00700 1 -- 172.21.15.136:0/4124751597 shutdown_connections 2024-05-27T07:31:56.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.840+0000 7fd163e00700 1 -- 172.21.15.136:0/4124751597 wait complete. 2024-05-27T07:31:56.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.841+0000 7fd163e00700 1 Processor -- start 2024-05-27T07:31:56.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.842+0000 7fd163e00700 1 -- start start 2024-05-27T07:31:56.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.842+0000 7fd163e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd164109990 0x7fd164075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:56.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.842+0000 7fd163e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd16410dd90 con 0x7fd164109990 2024-05-27T07:31:56.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.843+0000 7fd162a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd164109990 0x7fd164075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:56.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.843+0000 7fd162a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd164109990 0x7fd164075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.136:34002/0 (socket says 172.21.15.136:34002) 2024-05-27T07:31:56.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.843+0000 7fd162a00700 1 -- 172.21.15.136:0/333470925 learned_addr learned my addr 172.21.15.136:0/333470925 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-05-27T07:31:56.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.843+0000 7fd162a00700 1 -- 172.21.15.136:0/333470925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd164077900 con 0x7fd164109990 2024-05-27T07:31:56.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.844+0000 7fd162a00700 1 --2- 172.21.15.136:0/333470925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd164109990 0x7fd164075ab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1540092e0 tx=0x7fd154005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:56.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.844+0000 7fd15b400700 1 -- 172.21.15.136:0/333470925 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd154026070 con 0x7fd164109990 2024-05-27T07:31:56.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.844+0000 7fd15b400700 1 -- 172.21.15.136:0/333470925 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd15400b6f0 con 0x7fd164109990 2024-05-27T07:31:56.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.844+0000 7fd163e00700 1 -- 172.21.15.136:0/333470925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd164075ff0 con 0x7fd164109990 2024-05-27T07:31:56.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.844+0000 7fd15b400700 1 -- 172.21.15.136:0/333470925 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1540036c0 con 0x7fd164109990 2024-05-27T07:31:56.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.844+0000 7fd163e00700 1 -- 172.21.15.136:0/333470925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1640764b0 con 0x7fd164109990 2024-05-27T07:31:56.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.845+0000 7fd163e00700 1 -- 172.21.15.136:0/333470925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1641050a0 con 0x7fd164109990 2024-05-27T07:31:56.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.846+0000 7fd15b400700 1 -- 172.21.15.136:0/333470925 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd154029070 con 0x7fd164109990 2024-05-27T07:31:56.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.847+0000 7fd15b400700 1 --2- 172.21.15.136:0/333470925 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd14803e510 0x7fd148040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:31:56.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.847+0000 7fd15b400700 1 -- 172.21.15.136:0/333470925 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd15401e070 con 0x7fd164109990 2024-05-27T07:31:56.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.848+0000 7fd162000700 1 --2- 172.21.15.136:0/333470925 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd14803e510 0x7fd148040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:31:56.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.849+0000 7fd162000700 1 --2- 172.21.15.136:0/333470925 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd14803e510 0x7fd148040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006fd0 tx=0x7fd14c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:31:56.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:56.864+0000 7fd15b400700 1 -- 172.21.15.136:0/333470925 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd154013070 con 0x7fd164109990 2024-05-27T07:31:57.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.189+0000 7fd163e00700 1 -- 172.21.15.136:0/333470925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd164077010 con 0x7fd164109990 2024-05-27T07:31:57.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.191+0000 7fd15b400700 1 -- 172.21.15.136:0/333470925 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fd154029dd0 con 0x7fd164109990 2024-05-27T07:31:57.183 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:31:57.183 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","modified":"2024-05-27T07:29:33.566998Z","created":"2024-05-27T07:29:33.566998Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T07:31:57.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.194+0000 7fd163e00700 1 -- 172.21.15.136:0/333470925 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd14803e510 msgr2=0x7fd148040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:57.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.194+0000 7fd163e00700 1 --2- 172.21.15.136:0/333470925 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd14803e510 0x7fd148040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006fd0 tx=0x7fd14c006e40 comp rx=0 tx=0).stop 2024-05-27T07:31:57.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.194+0000 7fd163e00700 1 -- 172.21.15.136:0/333470925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd164109990 msgr2=0x7fd164075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:31:57.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.194+0000 7fd163e00700 1 --2- 172.21.15.136:0/333470925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd164109990 0x7fd164075ab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1540092e0 tx=0x7fd154005710 comp rx=0 tx=0).stop 2024-05-27T07:31:57.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.194+0000 7fd163e00700 1 -- 172.21.15.136:0/333470925 shutdown_connections 2024-05-27T07:31:57.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.194+0000 7fd163e00700 1 --2- 172.21.15.136:0/333470925 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd14803e510 0x7fd148040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:57.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.194+0000 7fd163e00700 1 --2- 172.21.15.136:0/333470925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd164109990 0x7fd164075ab0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:31:57.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.194+0000 7fd163e00700 1 -- 172.21.15.136:0/333470925 >> 172.21.15.136:0/333470925 conn(0x7fd1640ff480 msgr2=0x7fd164101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:31:57.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.194+0000 7fd163e00700 1 -- 172.21.15.136:0/333470925 shutdown_connections 2024-05-27T07:31:57.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:31:57.195+0000 7fd163e00700 1 -- 172.21.15.136:0/333470925 wait complete. 2024-05-27T07:31:57.188 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-05-27T07:31:57.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:57 smithi090 ceph-mon[31429]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:31:57.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:57 smithi090 ceph-mon[31429]: from='client.? 172.21.15.136:0/333470925' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:31:59.094 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T07:31:59.094 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T07:31:59.522 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:31:59.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:31:59 smithi090 ceph-mon[31429]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:00.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:00 smithi136 podman[35287]: 2024-05-27 07:32:00.94945691 +0000 UTC m=+0.136322966 container create 8986a2754a27244f159dae5aa81c04cc2b5896b038b6e40fc308967753e37f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T07:32:01.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 podman[35287]: 2024-05-27 07:32:01.212082952 +0000 UTC m=+0.398949094 container init 8986a2754a27244f159dae5aa81c04cc2b5896b038b6e40fc308967753e37f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-27T07:32:01.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 podman[35287]: 2024-05-27 07:32:01.223048305 +0000 UTC m=+0.409914407 container start 8986a2754a27244f159dae5aa81c04cc2b5896b038b6e40fc308967753e37f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:32:01.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 bash[35287]: 8986a2754a27244f159dae5aa81c04cc2b5896b038b6e40fc308967753e37f06 2024-05-27T07:32:01.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 systemd[1]: Started Ceph mon.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:32:01.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T07:32:01.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev), process ceph-mon, pid 2 2024-05-27T07:32:01.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pidfile_write: ignore empty --pid-file 2024-05-27T07:32:01.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: load: jerasure load: lrc 2024-05-27T07:32:01.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: RocksDB version: 7.9.2 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Git sha 0 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Compile date 2024-05-21 07:00:12 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: DB SUMMARY 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: DB Session ID: QC89JILHI21ZWCQUR9JT 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: CURRENT file: CURRENT 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.error_if_exists: 0 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.create_if_missing: 0 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.paranoid_checks: 1 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T07:32:01.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T07:32:01.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T07:32:01.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.env: 0x559938342500 2024-05-27T07:32:01.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T07:32:01.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.info_log: 0x55996e5a6040 2024-05-27T07:32:01.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T07:32:01.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.statistics: (nil) 2024-05-27T07:32:01.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.use_fsync: 0 2024-05-27T07:32:01.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_log_file_size: 0 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.allow_fallocate: 1 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.use_direct_reads: 0 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.db_log_dir: 2024-05-27T07:32:01.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.wal_dir: 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.write_buffer_manager: 0x55996e4708c0 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T07:32:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T07:32:01.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T07:32:01.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T07:32:01.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T07:32:01.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T07:32:01.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T07:32:01.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.unordered_write: 0 2024-05-27T07:32:01.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T07:32:01.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T07:32:01.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T07:32:01.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T07:32:01.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.row_cache: None 2024-05-27T07:32:01.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.wal_filter: None 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.two_write_queues: 0 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.wal_compression: 0 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.atomic_flush: 0 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.log_readahead_size: 0 2024-05-27T07:32:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T07:32:01.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T07:32:01.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T07:32:01.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T07:32:01.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T07:32:01.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T07:32:01.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T07:32:01.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_background_jobs: 2 2024-05-27T07:32:01.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_background_compactions: -1 2024-05-27T07:32:01.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_subcompactions: 1 2024-05-27T07:32:01.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_open_files: -1 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T07:32:01.503 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_background_flushes: -1 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Compression algorithms supported: 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: kZSTD supported: 0 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: kXpressCompression supported: 0 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: kZlibCompression supported: 1 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: kSnappyCompression supported: 1 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: kLZ4Compression supported: 1 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: kBZip2Compression supported: 0 2024-05-27T07:32:01.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.merge_operator: 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_filter: None 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55996e5a6200) 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout: cache_index_and_filter_blocks: 1 2024-05-27T07:32:01.505 INFO:journalctl@ceph.mon.c.smithi136.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: pin_top_level_index_and_filter: 1 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: index_type: 0 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: data_block_index_type: 0 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: index_shortening: 1 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: checksum: 4 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: no_block_cache: 0 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_cache: 0x55996e4a31f0 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_cache_name: BinnedLRUCache 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_cache_options: 2024-05-27T07:32:01.506 INFO:journalctl@ceph.mon.c.smithi136.stdout: capacity : 536870912 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: num_shard_bits : 4 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: strict_capacity_limit : 0 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: high_pri_pool_ratio: 0.000 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_cache_compressed: (nil) 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: persistent_cache: (nil) 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_size: 4096 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_size_deviation: 10 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_restart_interval: 16 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: index_block_restart_interval: 1 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: metadata_block_size: 4096 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: partition_filters: 0 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: use_delta_encoding: 1 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: filter_policy: bloomfilter 2024-05-27T07:32:01.507 INFO:journalctl@ceph.mon.c.smithi136.stdout: whole_key_filtering: 1 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout: verify_compression: 0 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout: read_amp_bytes_per_bit: 0 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout: format_version: 5 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout: enable_index_compression: 1 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_align: 0 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout: max_auto_readahead_size: 262144 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout: prepopulate_block_cache: 0 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout: initial_auto_readahead_size: 8192 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compression: NoCompression 2024-05-27T07:32:01.508 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.num_levels: 7 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:32:01.509 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:32:01.510 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T07:32:01.511 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T07:32:01.512 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.inplace_update_support: 0 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.bloom_locality: 0 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.max_successive_merges: 0 2024-05-27T07:32:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.ttl: 2592000 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.enable_blob_files: false 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.min_blob_size: 0 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e444933e-bec7-42e5-b5a1-53a7c5e3819d 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716795121359843, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716795121360888, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716795121, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e444933e-bec7-42e5-b5a1-53a7c5e3819d", "db_session_id": "QC89JILHI21ZWCQUR9JT", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-27T07:32:01.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716795121361291, "job": 1, "event": "recovery_finished"} 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55996e487c00 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: DB pointer 0x55997327a000 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout: ** DB Stats ** 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:32:01.516 INFO:journalctl@ceph.mon.c.smithi136.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: ** Compaction Stats [default] ** 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: ** Compaction Stats [default] ** 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:32:01.517 INFO:journalctl@ceph.mon.c.smithi136.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T07:32:01.518 INFO:journalctl@ceph.mon.c.smithi136.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:32:01.518 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-05-27T07:32:01.518 INFO:journalctl@ceph.mon.c.smithi136.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T07:32:01.518 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-05-27T07:32:01.518 INFO:journalctl@ceph.mon.c.smithi136.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:32:01.518 INFO:journalctl@ceph.mon.c.smithi136.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T07:32:01.518 INFO:journalctl@ceph.mon.c.smithi136.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T07:32:01.518 INFO:journalctl@ceph.mon.c.smithi136.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T07:32:01.519 INFO:journalctl@ceph.mon.c.smithi136.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T07:32:01.519 INFO:journalctl@ceph.mon.c.smithi136.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T07:32:01.519 INFO:journalctl@ceph.mon.c.smithi136.stdout: Cumulative compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:32:01.519 INFO:journalctl@ceph.mon.c.smithi136.stdout: Interval compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:32:01.519 INFO:journalctl@ceph.mon.c.smithi136.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T07:32:01.519 INFO:journalctl@ceph.mon.c.smithi136.stdout: Block cache BinnedLRUCache@0x55996e4a31f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-27T07:32:01.519 INFO:journalctl@ceph.mon.c.smithi136.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: using public_addr v2:172.21.15.136:0/0 -> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: starting mon.c rank -1 at public addrs [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] at bind addrs [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(???) e0 preinit fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).mds e1 new map 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout: e1 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:32:01.520 INFO:journalctl@ceph.mon.c.smithi136.stdout: legacy client fscid: -1 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout: No filesystems configured 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:32:01.521 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Added host smithi129 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.522 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Deploying cephadm binary to smithi136 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Updating smithi129:/etc/ceph/ceph.conf 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:01.523 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Added host smithi136 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:32:01.524 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1719855834' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1719855834' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:01.525 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Updating smithi136:/etc/ceph/ceph.conf 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi129:172.21.15.129=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi129:172.21.15.129=b;smithi136:172.21.15.136=c;count:3 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:01.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: Deploying daemon mon.c on smithi136 2024-05-27T07:32:01.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: from='client.? 172.21.15.136:0/333470925' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:32:01.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:01.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:01 smithi136 ceph-mon[35324]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-27T07:32:06.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:32:06.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: mon.a calling monitor election 2024-05-27T07:32:06.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:06.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: mon.c calling monitor election 2024-05-27T07:32:06.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:06.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: monmap epoch 2 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: last_changed 2024-05-27T07:32:01.412516+0000 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: min_mon_release 19 (squid) 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: election_strategy: 1 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: fsmap 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: mgrmap e13: x(active, since 68s) 2024-05-27T07:32:06.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: overall HEALTH_OK 2024-05-27T07:32:06.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:06.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:32:06.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: mon.a calling monitor election 2024-05-27T07:32:06.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: mon.c calling monitor election 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: monmap epoch 2 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: last_changed 2024-05-27T07:32:01.412516+0000 2024-05-27T07:32:06.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:32:06.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: min_mon_release 19 (squid) 2024-05-27T07:32:06.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: election_strategy: 1 2024-05-27T07:32:06.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:32:06.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-05-27T07:32:06.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: fsmap 2024-05-27T07:32:06.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:06.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: mgrmap e13: x(active, since 68s) 2024-05-27T07:32:06.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: overall HEALTH_OK 2024-05-27T07:32:06.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:07.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:07 smithi090 ceph-mon[31429]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:07.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:07.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:07.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:07.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:07 smithi090 ceph-mon[31429]: Deploying daemon mon.b on smithi129 2024-05-27T07:32:07.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:07.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:07 smithi136 ceph-mon[35324]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:07.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:07.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:07.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:07.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:07 smithi136 ceph-mon[35324]: Deploying daemon mon.b on smithi129 2024-05-27T07:32:07.970 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:07.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.974+0000 7f6ce5000700 1 -- 172.21.15.136:0/1244104672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc40062d0 msgr2=0x7f6cc40087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:07.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.974+0000 7f6ce5000700 1 --2- 172.21.15.136:0/1244104672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc40062d0 0x7f6cc40087b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0104ce0 tx=0x7f6cd000fe90 comp rx=0 tx=0).stop 2024-05-27T07:32:07.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.975+0000 7f6ce5000700 1 -- 172.21.15.136:0/1244104672 shutdown_connections 2024-05-27T07:32:07.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.975+0000 7f6ce5000700 1 --2- 172.21.15.136:0/1244104672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc40062d0 0x7f6cc40087b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:07.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.975+0000 7f6ce5000700 1 -- 172.21.15.136:0/1244104672 >> 172.21.15.136:0/1244104672 conn(0x7f6ce00ff760 msgr2=0x7f6ce0101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:07.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.975+0000 7f6ce5000700 1 -- 172.21.15.136:0/1244104672 shutdown_connections 2024-05-27T07:32:07.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.975+0000 7f6ce5000700 1 -- 172.21.15.136:0/1244104672 wait complete. 2024-05-27T07:32:07.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.978+0000 7f6ce5000700 1 Processor -- start 2024-05-27T07:32:07.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.979+0000 7f6ce5000700 1 -- start start 2024-05-27T07:32:07.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.980+0000 7f6ce5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc40062d0 0x7f6ce019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:07.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.980+0000 7f6ce5000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce019fdd0 0x7f6ce0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:07.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.980+0000 7f6ce5000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce010ba80 con 0x7f6cc40062d0 2024-05-27T07:32:07.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.980+0000 7f6ce5000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce010b900 con 0x7f6ce019fdd0 2024-05-27T07:32:07.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.981+0000 7f6cdde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc40062d0 0x7f6ce019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:07.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.981+0000 7f6cdde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc40062d0 0x7f6ce019f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.136:32840/0 (socket says 172.21.15.136:32840) 2024-05-27T07:32:07.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.981+0000 7f6cdde00700 1 -- 172.21.15.136:0/1664945826 learned_addr learned my addr 172.21.15.136:0/1664945826 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-05-27T07:32:07.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.981+0000 7f6cd5400700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce019fdd0 0x7f6ce0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:07.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.982+0000 7f6cd5400700 1 -- 172.21.15.136:0/1664945826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce019fdd0 msgr2=0x7f6ce0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-27T07:32:07.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.982+0000 7f6cd5400700 1 -- 172.21.15.136:0/1664945826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce019fdd0 msgr2=0x7f6ce0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T07:32:07.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.982+0000 7f6cd5400700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce019fdd0 0x7f6ce0199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T07:32:07.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.982+0000 7f6cdde00700 1 -- 172.21.15.136:0/1664945826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce019fdd0 msgr2=0x7f6ce0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:07.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.982+0000 7f6cd5400700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce019fdd0 0x7f6ce0199ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:32:07.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.982+0000 7f6cdde00700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce019fdd0 0x7f6ce0199ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:07.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.982+0000 7f6cdde00700 1 -- 172.21.15.136:0/1664945826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce01a03a0 con 0x7f6cc40062d0 2024-05-27T07:32:07.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.983+0000 7f6cdde00700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc40062d0 0x7f6ce019f890 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0005920 tx=0x7f6cd001da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:07.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.983+0000 7f6cd7400700 1 -- 172.21.15.136:0/1664945826 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6cd0005400 con 0x7f6cc40062d0 2024-05-27T07:32:07.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.983+0000 7f6cd7400700 1 -- 172.21.15.136:0/1664945826 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6cd00055a0 con 0x7f6cc40062d0 2024-05-27T07:32:07.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.983+0000 7f6ce5000700 1 -- 172.21.15.136:0/1664945826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce019a050 con 0x7f6cc40062d0 2024-05-27T07:32:07.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.984+0000 7f6cd7400700 1 -- 172.21.15.136:0/1664945826 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6cd0011730 con 0x7f6cc40062d0 2024-05-27T07:32:07.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.984+0000 7f6ce5000700 1 -- 172.21.15.136:0/1664945826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce019a5e0 con 0x7f6cc40062d0 2024-05-27T07:32:07.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.986+0000 7f6cd7400700 1 -- 172.21.15.136:0/1664945826 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6cd0011a30 con 0x7f6cc40062d0 2024-05-27T07:32:07.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.986+0000 7f6ce5000700 1 -- 172.21.15.136:0/1664945826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce0105360 con 0x7f6cc40062d0 2024-05-27T07:32:07.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.986+0000 7f6cd7400700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6cbc03e2b0 0x7f6cbc0407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:07.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.987+0000 7f6cd7400700 1 -- 172.21.15.136:0/1664945826 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6cd0056600 con 0x7f6cc40062d0 2024-05-27T07:32:07.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.989+0000 7f6cd5400700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6cbc03e2b0 0x7f6cbc0407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:07.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:07.993+0000 7f6cd5400700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6cbc03e2b0 0x7f6cbc0407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00ffc90 tx=0x7f6cc8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:07.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.004+0000 7f6cd7400700 1 -- 172.21.15.136:0/1664945826 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6cd002d2b0 con 0x7f6cc40062d0 2024-05-27T07:32:08.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.338+0000 7f6ce5000700 1 -- 172.21.15.136:0/1664945826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6ce019afe0 con 0x7f6cc40062d0 2024-05-27T07:32:08.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.340+0000 7f6cd7400700 1 -- 172.21.15.136:0/1664945826 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f6cd0011d20 con 0x7f6cc40062d0 2024-05-27T07:32:08.333 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:32:08.334 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":2,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","modified":"2024-05-27T07:32:01.412516Z","created":"2024-05-27T07:29:33.566998Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-27T07:32:08.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 -- 172.21.15.136:0/1664945826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6cbc03e2b0 msgr2=0x7f6cbc0407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:08.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6cbc03e2b0 0x7f6cbc0407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00ffc90 tx=0x7f6cc8008040 comp rx=0 tx=0).stop 2024-05-27T07:32:08.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 -- 172.21.15.136:0/1664945826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc40062d0 msgr2=0x7f6ce019f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:08.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc40062d0 0x7f6ce019f890 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0005920 tx=0x7f6cd001da60 comp rx=0 tx=0).stop 2024-05-27T07:32:08.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 -- 172.21.15.136:0/1664945826 shutdown_connections 2024-05-27T07:32:08.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6cbc03e2b0 0x7f6cbc0407c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:08.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cc40062d0 0x7f6ce019f890 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:08.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 --2- 172.21.15.136:0/1664945826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce019fdd0 0x7f6ce0199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:08.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 -- 172.21.15.136:0/1664945826 >> 172.21.15.136:0/1664945826 conn(0x7f6ce00ff760 msgr2=0x7f6ce0107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:08.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 -- 172.21.15.136:0/1664945826 shutdown_connections 2024-05-27T07:32:08.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:08.343+0000 7f6ce5000700 1 -- 172.21.15.136:0/1664945826 wait complete. 2024-05-27T07:32:08.337 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 2 2024-05-27T07:32:08.704 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:08 smithi136 ceph-mon[35324]: from='client.? 172.21.15.136:0/1664945826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:32:08.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:08 smithi090 ceph-mon[31429]: from='client.? 172.21.15.136:0/1664945826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:32:09.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:09 smithi090 ceph-mon[31429]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:09.992 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:09 smithi136 ceph-mon[35324]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:10.090 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T07:32:10.090 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T07:32:10.536 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:32:11.646 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:11 smithi136 ceph-mon[35324]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:11.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:11 smithi090 ceph-mon[31429]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.209 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:11 smithi129 podman[35235]: 2024-05-27 07:32:11.936563517 +0000 UTC m=+0.131722395 container create 714573e46108e9f94f2c5e187085efcddfc01454ca1d0f990d16a014d64cddb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:32:12.210 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 podman[35235]: 2024-05-27 07:32:12.214988043 +0000 UTC m=+0.410146913 container init 714573e46108e9f94f2c5e187085efcddfc01454ca1d0f990d16a014d64cddb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-b, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-27T07:32:12.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 podman[35235]: 2024-05-27 07:32:12.224618203 +0000 UTC m=+0.419777074 container start 714573e46108e9f94f2c5e187085efcddfc01454ca1d0f990d16a014d64cddb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T07:32:12.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 bash[35235]: 714573e46108e9f94f2c5e187085efcddfc01454ca1d0f990d16a014d64cddb4 2024-05-27T07:32:12.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 systemd[1]: Started Ceph mon.b for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:32:12.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T07:32:12.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev), process ceph-mon, pid 2 2024-05-27T07:32:12.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pidfile_write: ignore empty --pid-file 2024-05-27T07:32:12.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: load: jerasure load: lrc 2024-05-27T07:32:12.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: RocksDB version: 7.9.2 2024-05-27T07:32:12.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Git sha 0 2024-05-27T07:32:12.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Compile date 2024-05-21 07:00:12 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: DB SUMMARY 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: DB Session ID: AZX58BVSBQQR0VZDA09F 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: CURRENT file: CURRENT 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.error_if_exists: 0 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.create_if_missing: 0 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.paranoid_checks: 1 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T07:32:12.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.env: 0x55808e8a7500 2024-05-27T07:32:12.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T07:32:12.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.info_log: 0x5580a49e6040 2024-05-27T07:32:12.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T07:32:12.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.statistics: (nil) 2024-05-27T07:32:12.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.use_fsync: 0 2024-05-27T07:32:12.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_log_file_size: 0 2024-05-27T07:32:12.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T07:32:12.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T07:32:12.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T07:32:12.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T07:32:12.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.allow_fallocate: 1 2024-05-27T07:32:12.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T07:32:12.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T07:32:12.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.use_direct_reads: 0 2024-05-27T07:32:12.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T07:32:12.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T07:32:12.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.db_log_dir: 2024-05-27T07:32:12.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.wal_dir: 2024-05-27T07:32:12.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T07:32:12.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T07:32:12.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T07:32:12.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T07:32:12.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T07:32:12.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T07:32:12.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T07:32:12.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.write_buffer_manager: 0x5580a48b08c0 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.unordered_write: 0 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T07:32:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.row_cache: None 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.wal_filter: None 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.two_write_queues: 0 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.wal_compression: 0 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.atomic_flush: 0 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T07:32:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.log_readahead_size: 0 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_background_jobs: 2 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_background_compactions: -1 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_subcompactions: 1 2024-05-27T07:32:12.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_open_files: -1 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T07:32:12.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_background_flushes: -1 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Compression algorithms supported: 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: kZSTD supported: 0 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: kXpressCompression supported: 0 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: kZlibCompression supported: 1 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: kSnappyCompression supported: 1 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: kLZ4Compression supported: 1 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: kBZip2Compression supported: 0 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T07:32:12.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.merge_operator: 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_filter: None 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5580a49e6200) 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout: cache_index_and_filter_blocks: 1 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T07:32:12.568 INFO:journalctl@ceph.mon.b.smithi129.stdout: pin_top_level_index_and_filter: 1 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: index_type: 0 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: data_block_index_type: 0 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: index_shortening: 1 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: checksum: 4 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: no_block_cache: 0 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache: 0x5580a48e31f0 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache_name: BinnedLRUCache 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache_options: 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: capacity : 536870912 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: num_shard_bits : 4 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: strict_capacity_limit : 0 2024-05-27T07:32:12.569 INFO:journalctl@ceph.mon.b.smithi129.stdout: high_pri_pool_ratio: 0.000 2024-05-27T07:32:12.570 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache_compressed: (nil) 2024-05-27T07:32:12.570 INFO:journalctl@ceph.mon.b.smithi129.stdout: persistent_cache: (nil) 2024-05-27T07:32:12.570 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_size: 4096 2024-05-27T07:32:12.570 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_size_deviation: 10 2024-05-27T07:32:12.570 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_restart_interval: 16 2024-05-27T07:32:12.570 INFO:journalctl@ceph.mon.b.smithi129.stdout: index_block_restart_interval: 1 2024-05-27T07:32:12.570 INFO:journalctl@ceph.mon.b.smithi129.stdout: metadata_block_size: 4096 2024-05-27T07:32:12.570 INFO:journalctl@ceph.mon.b.smithi129.stdout: partition_filters: 0 2024-05-27T07:32:12.570 INFO:journalctl@ceph.mon.b.smithi129.stdout: use_delta_encoding: 1 2024-05-27T07:32:12.570 INFO:journalctl@ceph.mon.b.smithi129.stdout: filter_policy: bloomfilter 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout: whole_key_filtering: 1 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout: verify_compression: 0 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout: read_amp_bytes_per_bit: 0 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout: format_version: 5 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout: enable_index_compression: 1 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_align: 0 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout: max_auto_readahead_size: 262144 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout: prepopulate_block_cache: 0 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout: initial_auto_readahead_size: 8192 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T07:32:12.571 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compression: NoCompression 2024-05-27T07:32:12.572 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T07:32:12.572 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T07:32:12.572 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T07:32:12.573 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.num_levels: 7 2024-05-27T07:32:12.573 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T07:32:12.573 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T07:32:12.573 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T07:32:12.573 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T07:32:12.573 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T07:32:12.573 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T07:32:12.574 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T07:32:12.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T07:32:12.576 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T07:32:12.577 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.inplace_update_support: 0 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.bloom_locality: 0 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.max_successive_merges: 0 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T07:32:12.578 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.ttl: 2592000 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.enable_blob_files: false 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.min_blob_size: 0 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T07:32:12.579 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2424d34c-f9f3-4d0e-b47a-4cce34cfe844 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716795132355080, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716795132356303, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716795132, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2424d34c-f9f3-4d0e-b47a-4cce34cfe844", "db_session_id": "AZX58BVSBQQR0VZDA09F", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716795132356681, "job": 1, "event": "recovery_finished"} 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5580a48c7c00 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: DB pointer 0x5580a96ba000 2024-05-27T07:32:12.580 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** DB Stats ** 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** Compaction Stats [default] ** 2024-05-27T07:32:12.581 INFO:journalctl@ceph.mon.b.smithi129.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** Compaction Stats [default] ** 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-05-27T07:32:12.582 INFO:journalctl@ceph.mon.b.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: Block cache BinnedLRUCache@0x5580a48e31f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T07:32:12.583 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: using public_addr v2:172.21.15.129:0/0 -> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: starting mon.b rank -1 at public addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] at bind addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(???) e0 preinit fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).mds e1 new map 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout: e1 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout: legacy client fscid: -1 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout: No filesystems configured 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-27T07:32:12.584 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Added host smithi129 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.585 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:32:12.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Deploying cephadm binary to smithi136 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Updating smithi129:/etc/ceph/ceph.conf 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Added host smithi136 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.587 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1719855834' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1719855834' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T07:32:12.588 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Updating smithi136:/etc/ceph/ceph.conf 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi129:172.21.15.129=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:12.589 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi129:172.21.15.129=b;smithi136:172.21.15.136=c;count:3 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Deploying daemon mon.c on smithi136 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='client.? 172.21.15.136:0/333470925' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:32:12.590 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.a calling monitor election 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.c calling monitor election 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: monmap epoch 2 2024-05-27T07:32:12.591 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: last_changed 2024-05-27T07:32:01.412516+0000 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: min_mon_release 19 (squid) 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: election_strategy: 1 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: fsmap 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mgrmap e13: x(active, since 68s) 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: overall HEALTH_OK 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.592 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:12.593 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:12.593 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:12.593 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: Deploying daemon mon.b on smithi129 2024-05-27T07:32:12.593 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:12.593 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: from='client.? 172.21.15.136:0/1664945826' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T07:32:12.593 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.593 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:12.593 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:12 smithi129 ceph-mon[35254]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-05-27T07:32:17.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:17.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:32:17.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: mon.a calling monitor election 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: mon.c calling monitor election 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: monmap epoch 3 2024-05-27T07:32:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: last_changed 2024-05-27T07:32:12.418519+0000 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: min_mon_release 19 (squid) 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: election_strategy: 1 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: fsmap 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: mgrmap e13: x(active, since 79s) 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: overall HEALTH_OK 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:17.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:17.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:32:17.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: mon.a calling monitor election 2024-05-27T07:32:17.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: mon.c calling monitor election 2024-05-27T07:32:17.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: monmap epoch 3 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: last_changed 2024-05-27T07:32:12.418519+0000 2024-05-27T07:32:17.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: min_mon_release 19 (squid) 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: election_strategy: 1 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: fsmap 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: mgrmap e13: x(active, since 79s) 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: overall HEALTH_OK 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:18.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.432+0000 7fa19d800700 1 -- 172.21.15.136:0/2363373975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa17c005c90 msgr2=0x7fa17c008170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:18.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.432+0000 7fa19d800700 1 --2- 172.21.15.136:0/2363373975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa17c005c90 0x7fa17c008170 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa1981072d0 tx=0x7fa18800ff00 comp rx=0 tx=0).stop 2024-05-27T07:32:18.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.432+0000 7fa19d800700 1 -- 172.21.15.136:0/2363373975 shutdown_connections 2024-05-27T07:32:18.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.432+0000 7fa19d800700 1 --2- 172.21.15.136:0/2363373975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa17c005c90 0x7fa17c008170 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.432+0000 7fa19d800700 1 -- 172.21.15.136:0/2363373975 >> 172.21.15.136:0/2363373975 conn(0x7fa198101960 msgr2=0x7fa198103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:18.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.432+0000 7fa19d800700 1 -- 172.21.15.136:0/2363373975 shutdown_connections 2024-05-27T07:32:18.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.432+0000 7fa19d800700 1 -- 172.21.15.136:0/2363373975 wait complete. 2024-05-27T07:32:18.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.436+0000 7fa19d800700 1 Processor -- start 2024-05-27T07:32:18.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.436+0000 7fa19d800700 1 -- start start 2024-05-27T07:32:18.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.437+0000 7fa19d800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa17c005c90 0x7fa19819be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:18.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.437+0000 7fa19d800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa19819c3c0 0x7fa1981a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:18.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.437+0000 7fa19d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1981a0bf0 0x7fa1981a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:18.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.438+0000 7fa19d800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa19810dca0 con 0x7fa1981a0bf0 2024-05-27T07:32:18.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.438+0000 7fa19d800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa19810db20 con 0x7fa19819c3c0 2024-05-27T07:32:18.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.438+0000 7fa19d800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa19810de20 con 0x7fa17c005c90 2024-05-27T07:32:18.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.438+0000 7fa18e000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa19819c3c0 0x7fa1981a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:18.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.438+0000 7fa18e000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa19819c3c0 0x7fa1981a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.136:42382/0 (socket says 172.21.15.136:42382) 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.438+0000 7fa18e000700 1 -- 172.21.15.136:0/3822451192 learned_addr learned my addr 172.21.15.136:0/3822451192 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa196a00700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa17c005c90 0x7fa19819be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa197400700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1981a0bf0 0x7fa1981a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa18e000700 1 -- 172.21.15.136:0/3822451192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa19819c3c0 msgr2=0x7fa1981a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa18e000700 1 -- 172.21.15.136:0/3822451192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa19819c3c0 msgr2=0x7fa1981a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa18e000700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa19819c3c0 0x7fa1981a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa18e000700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa19819c3c0 0x7fa1981a06b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa196a00700 1 -- 172.21.15.136:0/3822451192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa19819c3c0 msgr2=0x7fa1981a06b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa196a00700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa19819c3c0 0x7fa1981a06b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa196a00700 1 -- 172.21.15.136:0/3822451192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1981a0bf0 msgr2=0x7fa1981a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa196a00700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1981a0bf0 0x7fa1981a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.439+0000 7fa196a00700 1 -- 172.21.15.136:0/3822451192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1981a8960 con 0x7fa17c005c90 2024-05-27T07:32:18.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.440+0000 7fa196a00700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa17c005c90 0x7fa19819be80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa188021000 tx=0x7fa18801da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:18.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.440+0000 7fa18fe00700 1 -- 172.21.15.136:0/3822451192 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1880054e0 con 0x7fa17c005c90 2024-05-27T07:32:18.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.440+0000 7fa18fe00700 1 -- 172.21.15.136:0/3822451192 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa188005680 con 0x7fa17c005c90 2024-05-27T07:32:18.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.440+0000 7fa19d800700 1 -- 172.21.15.136:0/3822451192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1981a8b30 con 0x7fa17c005c90 2024-05-27T07:32:18.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.440+0000 7fa18fe00700 1 -- 172.21.15.136:0/3822451192 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18802f8b0 con 0x7fa17c005c90 2024-05-27T07:32:18.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.440+0000 7fa19d800700 1 -- 172.21.15.136:0/3822451192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1981a90c0 con 0x7fa17c005c90 2024-05-27T07:32:18.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.442+0000 7fa18fe00700 1 -- 172.21.15.136:0/3822451192 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa18802fa50 con 0x7fa17c005c90 2024-05-27T07:32:18.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.442+0000 7fa19d800700 1 -- 172.21.15.136:0/3822451192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa198064e10 con 0x7fa17c005c90 2024-05-27T07:32:18.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.443+0000 7fa18fe00700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa17403e9a0 0x7fa174040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:18.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.443+0000 7fa18fe00700 1 -- 172.21.15.136:0/3822451192 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa188055ac0 con 0x7fa17c005c90 2024-05-27T07:32:18.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.444+0000 7fa18e000700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa17403e9a0 0x7fa174040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:18.438 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.445+0000 7fa18e000700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa17403e9a0 0x7fa174040eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa198101e20 tx=0x7fa180008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:18.442 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.449+0000 7fa18fe00700 1 -- 172.21.15.136:0/3822451192 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa18801c370 con 0x7fa17c005c90 2024-05-27T07:32:18.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:18 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:18.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:18 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:18.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:18 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:18.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.779+0000 7fa19d800700 1 -- 172.21.15.136:0/3822451192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa1981a9330 con 0x7fa17c005c90 2024-05-27T07:32:18.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.779+0000 7fa18fe00700 1 -- 172.21.15.136:0/3822451192 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7fa188055e20 con 0x7fa17c005c90 2024-05-27T07:32:18.773 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-05-27T07:32:18.773 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":3,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","modified":"2024-05-27T07:32:12.418519Z","created":"2024-05-27T07:29:33.566998Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-27T07:32:18.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 -- 172.21.15.136:0/3822451192 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa17403e9a0 msgr2=0x7fa174040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:18.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa17403e9a0 0x7fa174040eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa198101e20 tx=0x7fa180008040 comp rx=0 tx=0).stop 2024-05-27T07:32:18.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 -- 172.21.15.136:0/3822451192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa17c005c90 msgr2=0x7fa19819be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:18.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa17c005c90 0x7fa19819be80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa188021000 tx=0x7fa18801da60 comp rx=0 tx=0).stop 2024-05-27T07:32:18.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 -- 172.21.15.136:0/3822451192 shutdown_connections 2024-05-27T07:32:18.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa17403e9a0 0x7fa174040eb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1981a0bf0 0x7fa1981a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa17c005c90 0x7fa19819be80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 --2- 172.21.15.136:0/3822451192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa19819c3c0 0x7fa1981a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:18.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 -- 172.21.15.136:0/3822451192 >> 172.21.15.136:0/3822451192 conn(0x7fa198101960 msgr2=0x7fa198109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:18.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 -- 172.21.15.136:0/3822451192 shutdown_connections 2024-05-27T07:32:18.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:18.782+0000 7fa19d800700 1 -- 172.21.15.136:0/3822451192 wait complete. 2024-05-27T07:32:18.777 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 3 2024-05-27T07:32:18.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:18 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:18.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:18 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:18.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:18 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:19.414 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-27T07:32:19.414 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-05-27T07:32:19.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: mon.a calling monitor election 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: mon.c calling monitor election 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:19.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: monmap epoch 3 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: last_changed 2024-05-27T07:32:12.418519+0000 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: min_mon_release 19 (squid) 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: election_strategy: 1 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: fsmap 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:19.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: mgrmap e13: x(active, since 79s) 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: overall HEALTH_OK 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: mon.b calling monitor election 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: mon.b calling monitor election 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: mon.c calling monitor election 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: mon.a calling monitor election 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: monmap epoch 3 2024-05-27T07:32:19.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: last_changed 2024-05-27T07:32:12.418519+0000 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: min_mon_release 19 (squid) 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: election_strategy: 1 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: fsmap 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: mgrmap e13: x(active, since 81s) 2024-05-27T07:32:19.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:19 smithi129 ceph-mon[35254]: overall HEALTH_OK 2024-05-27T07:32:19.655 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: mon.b calling monitor election 2024-05-27T07:32:19.655 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: mon.b calling monitor election 2024-05-27T07:32:19.655 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: mon.c calling monitor election 2024-05-27T07:32:19.655 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: mon.a calling monitor election 2024-05-27T07:32:19.655 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T07:32:19.655 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: monmap epoch 3 2024-05-27T07:32:19.655 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:19.655 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: last_changed 2024-05-27T07:32:12.418519+0000 2024-05-27T07:32:19.656 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:32:19.656 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: min_mon_release 19 (squid) 2024-05-27T07:32:19.656 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: election_strategy: 1 2024-05-27T07:32:19.656 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:32:19.656 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-05-27T07:32:19.656 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-05-27T07:32:19.656 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: fsmap 2024-05-27T07:32:19.656 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:19.656 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: mgrmap e13: x(active, since 81s) 2024-05-27T07:32:19.656 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:19 smithi136 ceph-mon[35324]: overall HEALTH_OK 2024-05-27T07:32:19.856 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:19.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: mon.b calling monitor election 2024-05-27T07:32:19.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: mon.b calling monitor election 2024-05-27T07:32:19.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: mon.c calling monitor election 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: mon.a calling monitor election 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: monmap epoch 3 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: last_changed 2024-05-27T07:32:12.418519+0000 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: created 2024-05-27T07:29:33.566998+0000 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: min_mon_release 19 (squid) 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: election_strategy: 1 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: fsmap 2024-05-27T07:32:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T07:32:19.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: mgrmap e13: x(active, since 81s) 2024-05-27T07:32:19.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:19 smithi090 ceph-mon[31429]: overall HEALTH_OK 2024-05-27T07:32:21.134 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.134 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.134 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.135 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.135 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:21.135 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:21.135 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:21.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:21.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:21.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:21.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:21.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:21.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:32:21.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.953+0000 7f85f5400700 1 -- 172.21.15.90:0/1718769016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85f0103de0 msgr2=0x7f85f01041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:21.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.953+0000 7f85f5400700 1 --2- 172.21.15.90:0/1718769016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85f0103de0 0x7f85f01041f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009910 tx=0x7f85e0009c20 comp rx=0 tx=0).stop 2024-05-27T07:32:21.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.953+0000 7f85f5400700 1 -- 172.21.15.90:0/1718769016 shutdown_connections 2024-05-27T07:32:21.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.953+0000 7f85f5400700 1 --2- 172.21.15.90:0/1718769016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85f0103de0 0x7f85f01041f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:21.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.953+0000 7f85f5400700 1 -- 172.21.15.90:0/1718769016 >> 172.21.15.90:0/1718769016 conn(0x7f85f00ff690 msgr2=0x7f85f0101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:21.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.953+0000 7f85f5400700 1 -- 172.21.15.90:0/1718769016 shutdown_connections 2024-05-27T07:32:21.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.953+0000 7f85f5400700 1 -- 172.21.15.90:0/1718769016 wait complete. 2024-05-27T07:32:21.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.954+0000 7f85f5400700 1 Processor -- start 2024-05-27T07:32:21.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.955+0000 7f85f5400700 1 -- start start 2024-05-27T07:32:21.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.955+0000 7f85f5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85f0103de0 0x7f85f019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:21.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.955+0000 7f85f5400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85f019fcb0 0x7f85f0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:21.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.955+0000 7f85f5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85f0199ed0 0x7f85f019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:21.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.955+0000 7f85f5400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010df80 con 0x7f85f0103de0 2024-05-27T07:32:21.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.955+0000 7f85f5400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010de00 con 0x7f85f019fcb0 2024-05-27T07:32:21.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.955+0000 7f85f5400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85f010e100 con 0x7f85f0199ed0 2024-05-27T07:32:21.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.955+0000 7f85ee800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85f0103de0 0x7f85f019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:21.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.955+0000 7f85ee800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85f0103de0 0x7f85f019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38426/0 (socket says 172.21.15.90:38426) 2024-05-27T07:32:21.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.955+0000 7f85ee800700 1 -- 172.21.15.90:0/129805374 learned_addr learned my addr 172.21.15.90:0/129805374 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:32:21.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.956+0000 7f85e7e00700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85f019fcb0 0x7f85f0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:21.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.956+0000 7f85ee800700 1 -- 172.21.15.90:0/129805374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85f0199ed0 msgr2=0x7f85f019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:21.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.956+0000 7f85ee800700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85f0199ed0 0x7f85f019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:21.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.956+0000 7f85ee800700 1 -- 172.21.15.90:0/129805374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85f019fcb0 msgr2=0x7f85f0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:21.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.956+0000 7f85ee800700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85f019fcb0 0x7f85f0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:21.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.956+0000 7f85ee800700 1 -- 172.21.15.90:0/129805374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f01a6600 con 0x7f85f0103de0 2024-05-27T07:32:21.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.956+0000 7f85e7e00700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85f019fcb0 0x7f85f0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:32:21.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.956+0000 7f85ee800700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85f0103de0 0x7f85f019f770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f85e000f690 tx=0x7f85e000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:21.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.957+0000 7f85e7400700 1 -- 172.21.15.90:0/129805374 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0027070 con 0x7f85f0103de0 2024-05-27T07:32:21.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.957+0000 7f85e7400700 1 -- 172.21.15.90:0/129805374 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85e0005da0 con 0x7f85f0103de0 2024-05-27T07:32:21.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.957+0000 7f85f5400700 1 -- 172.21.15.90:0/129805374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f01a6890 con 0x7f85f0103de0 2024-05-27T07:32:21.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.957+0000 7f85e7400700 1 -- 172.21.15.90:0/129805374 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0017440 con 0x7f85f0103de0 2024-05-27T07:32:21.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.957+0000 7f85f5400700 1 -- 172.21.15.90:0/129805374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f01a6da0 con 0x7f85f0103de0 2024-05-27T07:32:21.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.958+0000 7f85f5400700 1 -- 172.21.15.90:0/129805374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f0105290 con 0x7f85f0103de0 2024-05-27T07:32:21.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.958+0000 7f85e7400700 1 -- 172.21.15.90:0/129805374 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f85e002a070 con 0x7f85f0103de0 2024-05-27T07:32:21.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-27T07:32:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: Updating smithi129:/etc/ceph/ceph.conf 2024-05-27T07:32:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: Updating smithi136:/etc/ceph/ceph.conf 2024-05-27T07:32:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:21 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:21.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.959+0000 7f85e7400700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f85d403e9a0 0x7f85d4040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:21.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.959+0000 7f85e7400700 1 -- 172.21.15.90:0/129805374 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f85e0053800 con 0x7f85f0103de0 2024-05-27T07:32:21.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.959+0000 7f85e7e00700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f85d403e9a0 0x7f85d4040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:21.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.960+0000 7f85e7e00700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f85d403e9a0 0x7f85d4040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f85f010b070 tx=0x7f85d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:21.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:21.968+0000 7f85e7400700 1 -- 172.21.15.90:0/129805374 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85e0013070 con 0x7f85f0103de0 2024-05-27T07:32:22.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.198+0000 7f85f5400700 1 -- 172.21.15.90:0/129805374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f85f01a70f0 con 0x7f85f0103de0 2024-05-27T07:32:22.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.198+0000 7f85e7400700 1 -- 172.21.15.90:0/129805374 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f85e000e3e0 con 0x7f85f0103de0 2024-05-27T07:32:22.192 INFO:teuthology.orchestra.run.smithi090.stdout:# minimal ceph.conf for b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:22.192 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-05-27T07:32:22.193 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:32:22.193 INFO:teuthology.orchestra.run.smithi090.stdout: mon_host = [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-05-27T07:32:22.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 -- 172.21.15.90:0/129805374 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f85d403e9a0 msgr2=0x7f85d4040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f85d403e9a0 0x7f85d4040eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f85f010b070 tx=0x7f85d8008040 comp rx=0 tx=0).stop 2024-05-27T07:32:22.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 -- 172.21.15.90:0/129805374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85f0103de0 msgr2=0x7f85f019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:22.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85f0103de0 0x7f85f019f770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f85e000f690 tx=0x7f85e000f770 comp rx=0 tx=0).stop 2024-05-27T07:32:22.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 -- 172.21.15.90:0/129805374 shutdown_connections 2024-05-27T07:32:22.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f85d403e9a0 0x7f85d4040eb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85f0199ed0 0x7f85f019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85f0103de0 0x7f85f019f770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 --2- 172.21.15.90:0/129805374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85f019fcb0 0x7f85f0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:22.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 -- 172.21.15.90:0/129805374 >> 172.21.15.90:0/129805374 conn(0x7f85f00ff690 msgr2=0x7f85f0109910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:22.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.202+0000 7f85f5400700 1 -- 172.21.15.90:0/129805374 shutdown_connections 2024-05-27T07:32:22.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:22.203+0000 7f85f5400700 1 -- 172.21.15.90:0/129805374 wait complete. 2024-05-27T07:32:22.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:22.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-27T07:32:22.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: Updating smithi129:/etc/ceph/ceph.conf 2024-05-27T07:32:22.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: Updating smithi136:/etc/ceph/ceph.conf 2024-05-27T07:32:22.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:21 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: Updating smithi129:/etc/ceph/ceph.conf 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: Updating smithi136:/etc/ceph/ceph.conf 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:21 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:22.927 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-27T07:32:22.928 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:32:22.928 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T07:32:22.990 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:32:22.990 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:32:23.092 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:32:23.092 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T07:32:23.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:23.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:23.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:23.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:23.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/129805374' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:23.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T07:32:23.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:23.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:32:23.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:23.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:22 smithi090 ceph-mon[31429]: Reconfiguring daemon mon.a on smithi090 2024-05-27T07:32:23.170 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/129805374' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:23.171 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:22 smithi129 ceph-mon[35254]: Reconfiguring daemon mon.a on smithi090 2024-05-27T07:32:23.186 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:32:23.186 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:32:23.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:23.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/129805374' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:23.243 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:22 smithi136 ceph-mon[35324]: Reconfiguring daemon mon.a on smithi090 2024-05-27T07:32:23.315 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:32:23.315 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T07:32:23.405 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:32:23.406 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:32:23.533 INFO:tasks.cephadm:Adding mgr.x on smithi090 2024-05-27T07:32:23.533 INFO:tasks.cephadm:Adding mgr.y on smithi129 2024-05-27T07:32:23.533 INFO:tasks.cephadm:Adding mgr.z on smithi136 2024-05-27T07:32:23.534 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi090=x;smithi129=y;smithi136=z' 2024-05-27T07:32:24.006 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:32:25.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:25 smithi129 ceph-mon[35254]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:25.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:25.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:25.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:25 smithi129 ceph-mon[35254]: Reconfiguring mon.b (monmap changed)... 2024-05-27T07:32:25.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:25.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:32:25.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:25.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:25 smithi129 ceph-mon[35254]: Reconfiguring daemon mon.b on smithi129 2024-05-27T07:32:25.611 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:25 smithi136 ceph-mon[35324]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:25.611 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:25.612 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:25.612 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:25 smithi136 ceph-mon[35324]: Reconfiguring mon.b (monmap changed)... 2024-05-27T07:32:25.612 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:25.612 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:32:25.612 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:25.612 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:25 smithi136 ceph-mon[35324]: Reconfiguring daemon mon.b on smithi129 2024-05-27T07:32:25.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:25 smithi090 ceph-mon[31429]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:25.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:25.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:25.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:25 smithi090 ceph-mon[31429]: Reconfiguring mon.b (monmap changed)... 2024-05-27T07:32:25.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:25.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:32:25.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:25.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:25 smithi090 ceph-mon[31429]: Reconfiguring daemon mon.b on smithi129 2024-05-27T07:32:25.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.966+0000 7f91ab800700 1 -- 172.21.15.136:0/609597423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91a410a320 msgr2=0x7f91a410a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:25.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.967+0000 7f91ab800700 1 --2- 172.21.15.136:0/609597423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91a410a320 0x7f91a410a730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9198009910 tx=0x7f9198009c20 comp rx=0 tx=0).stop 2024-05-27T07:32:25.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.967+0000 7f91ab800700 1 -- 172.21.15.136:0/609597423 shutdown_connections 2024-05-27T07:32:25.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.967+0000 7f91ab800700 1 --2- 172.21.15.136:0/609597423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91a410a320 0x7f91a410a730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:25.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.967+0000 7f91ab800700 1 -- 172.21.15.136:0/609597423 >> 172.21.15.136:0/609597423 conn(0x7f91a4079380 msgr2=0x7f91a407b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:25.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.967+0000 7f91ab800700 1 -- 172.21.15.136:0/609597423 shutdown_connections 2024-05-27T07:32:25.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.967+0000 7f91ab800700 1 -- 172.21.15.136:0/609597423 wait complete. 2024-05-27T07:32:25.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.968+0000 7f91ab800700 1 Processor -- start 2024-05-27T07:32:25.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.968+0000 7f91ab800700 1 -- start start 2024-05-27T07:32:25.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.969+0000 7f91ab800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a410a320 0x7f91a419feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:25.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.969+0000 7f91ab800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91a41a03f0 0x7f91a41a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:25.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.969+0000 7f91ab800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91a41a58c0 0x7f91a41a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:25.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.969+0000 7f91ab800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4110190 con 0x7f91a41a58c0 2024-05-27T07:32:25.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.969+0000 7f91ab800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4110010 con 0x7f91a41a03f0 2024-05-27T07:32:25.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.969+0000 7f91ab800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f91a4110310 con 0x7f91a410a320 2024-05-27T07:32:25.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.969+0000 7f91a3e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91a41a03f0 0x7f91a41a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:25.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.970+0000 7f91a8e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a410a320 0x7f91a419feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:25.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.970+0000 7f91a3e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91a41a03f0 0x7f91a41a5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.136:55482/0 (socket says 172.21.15.136:55482) 2024-05-27T07:32:25.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.970+0000 7f91a3e00700 1 -- 172.21.15.136:0/3611090451 learned_addr learned my addr 172.21.15.136:0/3611090451 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-05-27T07:32:25.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.970+0000 7f91a9800700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91a41a58c0 0x7f91a41a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:25.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.970+0000 7f91a8e00700 1 -- 172.21.15.136:0/3611090451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91a41a03f0 msgr2=0x7f91a41a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:25.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.970+0000 7f91a8e00700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91a41a03f0 0x7f91a41a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:25.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.970+0000 7f91a8e00700 1 -- 172.21.15.136:0/3611090451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91a41a58c0 msgr2=0x7f91a41a7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:25.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.970+0000 7f91a8e00700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91a41a58c0 0x7f91a41a7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:25.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.970+0000 7f91a8e00700 1 -- 172.21.15.136:0/3611090451 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a41a8250 con 0x7f91a410a320 2024-05-27T07:32:25.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.970+0000 7f91a3e00700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91a41a03f0 0x7f91a41a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:32:25.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.971+0000 7f91a8e00700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a410a320 0x7f91a419feb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9198005920 tx=0x7f9198004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:25.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.971+0000 7f91a1600700 1 -- 172.21.15.136:0/3611090451 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198027070 con 0x7f91a410a320 2024-05-27T07:32:25.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.971+0000 7f91a1600700 1 -- 172.21.15.136:0/3611090451 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9198005da0 con 0x7f91a410a320 2024-05-27T07:32:25.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.971+0000 7f91ab800700 1 -- 172.21.15.136:0/3611090451 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a41a84e0 con 0x7f91a410a320 2024-05-27T07:32:25.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.971+0000 7f91a1600700 1 -- 172.21.15.136:0/3611090451 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198022c00 con 0x7f91a410a320 2024-05-27T07:32:25.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.971+0000 7f91ab800700 1 -- 172.21.15.136:0/3611090451 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a41a8a10 con 0x7f91a410a320 2024-05-27T07:32:25.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.972+0000 7f91a1600700 1 -- 172.21.15.136:0/3611090451 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f919802a070 con 0x7f91a410a320 2024-05-27T07:32:25.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.972+0000 7f91ab800700 1 -- 172.21.15.136:0/3611090451 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a4064e10 con 0x7f91a410a320 2024-05-27T07:32:25.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.973+0000 7f91a1600700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f918c03e9a0 0x7f918c040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:25.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.973+0000 7f91a1600700 1 -- 172.21.15.136:0/3611090451 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9198053b20 con 0x7f91a410a320 2024-05-27T07:32:25.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.974+0000 7f91a3e00700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f918c03e9a0 0x7f918c040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:25.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.975+0000 7f91a3e00700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f918c03e9a0 0x7f918c040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91a41a0f60 tx=0x7f91940092c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:25.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:25.980+0000 7f91a1600700 1 -- 172.21.15.136:0/3611090451 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9198013070 con 0x7f91a410a320 2024-05-27T07:32:26.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.218+0000 7f91ab800700 1 -- 172.21.15.136:0/3611090451 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi129=y;smithi136=z", "target": ["mon-mgr", ""]}) -- 0x7f91a407b1f0 con 0x7f918c03e9a0 2024-05-27T07:32:26.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.233+0000 7f91a1600700 1 -- 172.21.15.136:0/3611090451 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f91a407b1f0 con 0x7f918c03e9a0 2024-05-27T07:32:26.227 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mgr update... 2024-05-27T07:32:26.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.236+0000 7f91ab800700 1 -- 172.21.15.136:0/3611090451 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f918c03e9a0 msgr2=0x7f918c040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.236+0000 7f91ab800700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f918c03e9a0 0x7f918c040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f91a41a0f60 tx=0x7f91940092c0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.236+0000 7f91ab800700 1 -- 172.21.15.136:0/3611090451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a410a320 msgr2=0x7f91a419feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:26.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.236+0000 7f91ab800700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a410a320 0x7f91a419feb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9198005920 tx=0x7f9198004f80 comp rx=0 tx=0).stop 2024-05-27T07:32:26.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.236+0000 7f91ab800700 1 -- 172.21.15.136:0/3611090451 shutdown_connections 2024-05-27T07:32:26.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.236+0000 7f91ab800700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f918c03e9a0 0x7f918c040eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.236+0000 7f91ab800700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91a41a58c0 0x7f91a41a7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.236+0000 7f91ab800700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91a410a320 0x7f91a419feb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.236+0000 7f91ab800700 1 --2- 172.21.15.136:0/3611090451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f91a41a03f0 0x7f91a41a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:26.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.236+0000 7f91ab800700 1 -- 172.21.15.136:0/3611090451 >> 172.21.15.136:0/3611090451 conn(0x7f91a4079380 msgr2=0x7f91a407aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:26.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.237+0000 7f91ab800700 1 -- 172.21.15.136:0/3611090451 shutdown_connections 2024-05-27T07:32:26.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:32:26.237+0000 7f91ab800700 1 -- 172.21.15.136:0/3611090451 wait complete. 2024-05-27T07:32:26.954 DEBUG:teuthology.orchestra.run.smithi129:mgr.y> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.y.service 2024-05-27T07:32:26.962 DEBUG:teuthology.orchestra.run.smithi136:mgr.z> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.z.service 2024-05-27T07:32:26.969 INFO:tasks.cephadm:Deploying OSDs... 2024-05-27T07:32:26.969 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:32:26.969 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T07:32:27.025 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T07:32:27.026 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-05-27T07:32:27.079 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T07:32:27.080 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:27.080 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-05-27T07:32:27.080 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:27.080 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:27.080 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-05-27 07:31:14.625902609 +0000 2024-05-27T07:32:27.080 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-05-27 07:25:06.158019329 +0000 2024-05-27T07:32:27.080 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-05-27 07:25:06.158019329 +0000 2024-05-27T07:32:27.080 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-05-27 07:25:06.158019329 +0000 2024-05-27T07:32:27.080 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T07:32:27.159 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-05-27T07:32:27.160 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-05-27T07:32:27.160 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000528299 s, 969 kB/s 2024-05-27T07:32:27.163 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T07:32:27.218 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-05-27T07:32:27.266 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T07:32:27.266 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:27.266 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-27T07:32:27.266 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:27.266 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:27.266 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-05-27 07:31:14.673901876 +0000 2024-05-27T07:32:27.266 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-05-27 07:25:06.693011111 +0000 2024-05-27T07:32:27.266 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-05-27 07:25:06.693011111 +0000 2024-05-27T07:32:27.266 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-05-27 07:25:06.693011111 +0000 2024-05-27T07:32:27.267 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T07:32:27.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:27.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi129=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:27.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: Saving service mgr spec with placement smithi090=x;smithi129=y;smithi136=z;count:3 2024-05-27T07:32:27.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:27.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:27.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:27.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: Reconfiguring mon.c (monmap changed)... 2024-05-27T07:32:27.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:27.308 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:32:27.308 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:27.308 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:27 smithi129 ceph-mon[35254]: Reconfiguring daemon mon.c on smithi136 2024-05-27T07:32:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-05-27T07:32:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-05-27T07:32:27.337 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000348053 s, 1.5 MB/s 2024-05-27T07:32:27.340 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T07:32:27.396 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-05-27T07:32:27.445 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T07:32:27.445 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:27.445 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-27T07:32:27.445 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:27.445 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:27.445 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-05-27 07:31:14.720901159 +0000 2024-05-27T07:32:27.445 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-05-27 07:25:07.293001895 +0000 2024-05-27T07:32:27.445 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-05-27 07:25:07.293001895 +0000 2024-05-27T07:32:27.445 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-05-27 07:25:07.293001895 +0000 2024-05-27T07:32:27.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T07:32:27.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:27.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi129=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:27.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: Saving service mgr spec with placement smithi090=x;smithi129=y;smithi136=z;count:3 2024-05-27T07:32:27.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:27.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:27.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:27.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: Reconfiguring mon.c (monmap changed)... 2024-05-27T07:32:27.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:27.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:32:27.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:27.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:27 smithi090 ceph-mon[31429]: Reconfiguring daemon mon.c on smithi136 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi129=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: Saving service mgr spec with placement smithi090=x;smithi129=y;smithi136=z;count:3 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: Reconfiguring mon.c (monmap changed)... 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:27.483 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:27 smithi136 ceph-mon[35324]: Reconfiguring daemon mon.c on smithi136 2024-05-27T07:32:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-05-27T07:32:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-05-27T07:32:27.520 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000342751 s, 1.5 MB/s 2024-05-27T07:32:27.524 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T07:32:27.624 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-05-27T07:32:27.675 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T07:32:27.675 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:27.675 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-05-27T07:32:27.675 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:27.675 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:27.675 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-05-27 07:31:14.775900319 +0000 2024-05-27T07:32:27.675 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-05-27 07:25:07.864993110 +0000 2024-05-27T07:32:27.675 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-05-27 07:25:07.864993110 +0000 2024-05-27T07:32:27.675 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-05-27 07:25:07.864993110 +0000 2024-05-27T07:32:27.676 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T07:32:27.750 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-05-27T07:32:27.750 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-05-27T07:32:27.750 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000347596 s, 1.5 MB/s 2024-05-27T07:32:27.754 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T07:32:27.808 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-05-27T07:32:27.808 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T07:32:27.866 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T07:32:27.867 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-05-27T07:32:27.921 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T07:32:27.921 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:27.921 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-27T07:32:27.921 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:27.921 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:27.921 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-05-27 07:32:11.411676096 +0000 2024-05-27T07:32:27.921 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-05-27 07:25:08.233912449 +0000 2024-05-27T07:32:27.921 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-05-27 07:25:08.233912449 +0000 2024-05-27T07:32:27.921 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-05-27 07:25:08.233912449 +0000 2024-05-27T07:32:27.922 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T07:32:27.998 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-05-27T07:32:27.998 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-05-27T07:32:27.998 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000335862 s, 1.5 MB/s 2024-05-27T07:32:28.002 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T07:32:28.056 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-05-27T07:32:28.103 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T07:32:28.104 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:28.104 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-27T07:32:28.104 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:28.104 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:28.104 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-05-27 07:32:11.411676096 +0000 2024-05-27T07:32:28.104 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-05-27 07:25:08.833899603 +0000 2024-05-27T07:32:28.104 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-05-27 07:25:08.833899603 +0000 2024-05-27T07:32:28.104 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-05-27 07:25:08.833899603 +0000 2024-05-27T07:32:28.104 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T07:32:28.184 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-05-27T07:32:28.184 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-05-27T07:32:28.184 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000363964 s, 1.4 MB/s 2024-05-27T07:32:28.187 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T07:32:28.240 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-05-27T07:32:28.290 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T07:32:28.290 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:28.290 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-27T07:32:28.290 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:28.290 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:28.290 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-05-27 07:32:11.412676075 +0000 2024-05-27T07:32:28.290 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-05-27 07:25:09.422886992 +0000 2024-05-27T07:32:28.291 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-05-27 07:25:09.422886992 +0000 2024-05-27T07:32:28.291 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-05-27 07:25:09.422886992 +0000 2024-05-27T07:32:28.291 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T07:32:28.368 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-05-27T07:32:28.368 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-05-27T07:32:28.368 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000365088 s, 1.4 MB/s 2024-05-27T07:32:28.372 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T07:32:28.428 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-05-27T07:32:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T07:32:28.476 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:28.476 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-05-27T07:32:28.477 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:28.477 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:28.477 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-05-27 07:32:11.413676053 +0000 2024-05-27T07:32:28.477 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-05-27 07:25:09.991874809 +0000 2024-05-27T07:32:28.477 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-05-27 07:25:09.991874809 +0000 2024-05-27T07:32:28.477 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-05-27 07:25:09.991874809 +0000 2024-05-27T07:32:28.477 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T07:32:28.556 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-05-27T07:32:28.557 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-05-27T07:32:28.557 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000373166 s, 1.4 MB/s 2024-05-27T07:32:28.560 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T07:32:28.613 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-05-27T07:32:28.613 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T07:32:28.659 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T07:32:28.659 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-05-27T07:32:28.745 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T07:32:28.745 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:28.745 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-27T07:32:28.746 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:28.746 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:28.746 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-05-27 07:32:00.009950177 +0000 2024-05-27T07:32:28.746 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-05-27 07:25:09.876835218 +0000 2024-05-27T07:32:28.746 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-05-27 07:25:09.876835218 +0000 2024-05-27T07:32:28.746 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-05-27 07:25:09.876835218 +0000 2024-05-27T07:32:28.746 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T07:32:28.861 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-05-27T07:32:28.861 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-05-27T07:32:28.861 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000385746 s, 1.3 MB/s 2024-05-27T07:32:28.864 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T07:32:28.912 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-05-27T07:32:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T07:32:29.004 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:29.004 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-05-27T07:32:29.004 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:29.004 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:29.004 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-05-27 07:32:00.010950155 +0000 2024-05-27T07:32:29.005 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-05-27 07:25:10.452822475 +0000 2024-05-27T07:32:29.005 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-05-27 07:25:10.452822475 +0000 2024-05-27T07:32:29.005 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-05-27 07:25:10.452822475 +0000 2024-05-27T07:32:29.005 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T07:32:29.073 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-05-27T07:32:29.073 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-05-27T07:32:29.073 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000320699 s, 1.6 MB/s 2024-05-27T07:32:29.076 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T07:32:29.126 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-05-27T07:32:29.171 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T07:32:29.171 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:29.171 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-05-27T07:32:29.171 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:29.171 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:29.171 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-05-27 07:32:00.011950133 +0000 2024-05-27T07:32:29.171 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-05-27 07:25:11.021809887 +0000 2024-05-27T07:32:29.171 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-05-27 07:25:11.021809887 +0000 2024-05-27T07:32:29.172 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-05-27 07:25:11.021809887 +0000 2024-05-27T07:32:29.172 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T07:32:29.273 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-05-27T07:32:29.273 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-05-27T07:32:29.274 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.0003502 s, 1.5 MB/s 2024-05-27T07:32:29.277 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T07:32:29.318 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-05-27T07:32:29.399 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T07:32:29.400 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T07:32:29.400 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-27T07:32:29.400 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T07:32:29.400 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T07:32:29.400 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-05-27 07:32:00.011950133 +0000 2024-05-27T07:32:29.400 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-05-27 07:25:11.614796768 +0000 2024-05-27T07:32:29.400 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-05-27 07:25:11.614796768 +0000 2024-05-27T07:32:29.400 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-05-27 07:25:11.614796768 +0000 2024-05-27T07:32:29.400 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T07:32:29.500 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-05-27T07:32:29.500 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-05-27T07:32:29.500 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000268502 s, 1.9 MB/s 2024-05-27T07:32:29.503 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T07:32:29.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:29.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:29.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:29.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:29.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:29.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:29.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:29.526 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:32:29.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:32:29.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:32:29.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:29.527 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:29 smithi136 ceph-mon[35324]: Deploying daemon mgr.z on smithi136 2024-05-27T07:32:29.554 INFO:tasks.cephadm:Deploying osd.0 on smithi090 with /dev/vg_nvme/lv_4... 2024-05-27T07:32:29.555 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T07:32:29.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:29.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:29.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:29.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:29.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:29.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:29.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:29.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:32:29.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:32:29.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:32:29.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:29.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:29 smithi129 ceph-mon[35254]: Deploying daemon mgr.z on smithi136 2024-05-27T07:32:29.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:29.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:29.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:29.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:29.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:29.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:29.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:29.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:32:29.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:32:29.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:32:29.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:29.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:29 smithi090 ceph-mon[31429]: Deploying daemon mgr.z on smithi136 2024-05-27T07:32:29.982 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:31.391 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:31 smithi136 systemd[1]: Starting Ceph mgr.z for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:32:31.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:31 smithi090 ceph-mon[31429]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:31.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:31 smithi136 ceph-mon[35324]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:31.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:31 smithi129 ceph-mon[35254]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:32.037 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:31 smithi136 podman[37035]: 2024-05-27 07:32:31.684899783 +0000 UTC m=+0.034459118 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:32:32.037 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:31 smithi136 podman[37035]: 2024-05-27 07:32:31.794921063 +0000 UTC m=+0.144480434 container create 550d9598331ba373b37628fec767c806384e1b0d46e628974528999503b002df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-27T07:32:32.037 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:32 smithi136 podman[37035]: 2024-05-27 07:32:32.0439564 +0000 UTC m=+0.393515814 container init 550d9598331ba373b37628fec767c806384e1b0d46e628974528999503b002df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T07:32:32.362 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:32 smithi136 podman[37035]: 2024-05-27 07:32:32.053729478 +0000 UTC m=+0.403288819 container start 550d9598331ba373b37628fec767c806384e1b0d46e628974528999503b002df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:32:32.362 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:32 smithi136 bash[37035]: 550d9598331ba373b37628fec767c806384e1b0d46e628974528999503b002df 2024-05-27T07:32:32.362 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:32 smithi136 systemd[1]: Started Ceph mgr.z for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:32:32.362 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:32 smithi136 ceph-mgr[37053]: -- 172.21.15.136:0/414392654 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db691845a0 con 0x55db6df0e000 2024-05-27T07:32:32.596 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T07:32:32.596 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T07:32:32.596 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-05-27T07:32:32.596 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-05-27T07:32:32.596 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0334331 s, 314 MB/s 2024-05-27T07:32:32.596 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-05-27T07:32:32.620 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:32.367+0000 7f0d3cfd7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:32:32.620 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:32.626+0000 7f0d3cfd7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:32:32.992 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:32.760+0000 7f0d3cfd7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:32:33.492 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:33.159+0000 7f0d3cfd7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:32:33.493 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:33.274+0000 7f0d3cfd7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:32:33.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:33 smithi136 ceph-mon[35324]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:33.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:33.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:33.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:33.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:32:33.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:32:33.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:32:33.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:33.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:33 smithi136 ceph-mon[35324]: Deploying daemon mgr.y on smithi129 2024-05-27T07:32:33.544 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:33 smithi129 ceph-mon[35254]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:33.544 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:33.544 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:33.545 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:33.545 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:32:33.545 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:32:33.545 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:32:33.545 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:33.545 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:33 smithi129 ceph-mon[35254]: Deploying daemon mgr.y on smithi129 2024-05-27T07:32:33.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:33 smithi090 ceph-mon[31429]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:33.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:33.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:33.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:33.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:32:33.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T07:32:33.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:32:33.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:33.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:33 smithi090 ceph-mon[31429]: Deploying daemon mgr.y on smithi129 2024-05-27T07:32:33.649 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-05-27T07:32:34.087 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:32:35.242 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:34.852+0000 7f0d3cfd7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:32:35.242 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:35.091+0000 7f0d3cfd7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:32:35.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:35 smithi129 ceph-mon[35254]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:35.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:35 smithi090 ceph-mon[31429]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:35.680 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:35.309+0000 7f0d3cfd7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:32:35.680 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:35 smithi136 ceph-mon[35324]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:35.766 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:35 smithi129 podman[36615]: 2024-05-27 07:32:35.76071633 +0000 UTC m=+0.367726494 container init d65cd30f1ca7885796c1f524960c7b5d47667eada5f6ac848b02eae2c2f57c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-27T07:32:35.766 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:35 smithi129 podman[36615]: 2024-05-27 07:32:35.772303323 +0000 UTC m=+0.379313403 container start d65cd30f1ca7885796c1f524960c7b5d47667eada5f6ac848b02eae2c2f57c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:32:35.992 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:35.686+0000 7f0d3cfd7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:32:35.992 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:35.809+0000 7f0d3cfd7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:32:36.057 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:35 smithi129 bash[36615]: d65cd30f1ca7885796c1f524960c7b5d47667eada5f6ac848b02eae2c2f57c57 2024-05-27T07:32:36.057 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:35 smithi129 systemd[1]: Started Ceph mgr.y for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:32:36.057 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:35 smithi129 ceph-mgr[36634]: -- 172.21.15.129:0/1183565736 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f5cdd1e4e0 con 0x55f5d2aaa900 2024-05-27T07:32:36.057 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:35 smithi129 ceph-mgr[36634]: -- 172.21.15.129:0/1183565736 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5cdd205a0 con 0x55f5d2aaa900 2024-05-27T07:32:36.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.142+0000 7f2225a00700 1 -- 172.21.15.90:0/104487013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f222006c2c0 msgr2=0x7f2220106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:36.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.143+0000 7f2225a00700 1 --2- 172.21.15.90:0/104487013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f222006c2c0 0x7f2220106ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2210009990 tx=0x7f2210009ca0 comp rx=0 tx=0).stop 2024-05-27T07:32:36.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.143+0000 7f2225a00700 1 -- 172.21.15.90:0/104487013 shutdown_connections 2024-05-27T07:32:36.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.143+0000 7f2225a00700 1 --2- 172.21.15.90:0/104487013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2220109fe0 0x7f222010c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:36.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.143+0000 7f2225a00700 1 --2- 172.21.15.90:0/104487013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f222006c2c0 0x7f2220106ea0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:36.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.143+0000 7f2225a00700 1 --2- 172.21.15.90:0/104487013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2220107470 0x7f2220109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:36.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.143+0000 7f2225a00700 1 -- 172.21.15.90:0/104487013 >> 172.21.15.90:0/104487013 conn(0x7f22200fe870 msgr2=0x7f2220100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:32:36.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.143+0000 7f2225a00700 1 -- 172.21.15.90:0/104487013 shutdown_connections 2024-05-27T07:32:36.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.143+0000 7f2225a00700 1 -- 172.21.15.90:0/104487013 wait complete. 2024-05-27T07:32:36.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.144+0000 7f2225a00700 1 Processor -- start 2024-05-27T07:32:36.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.144+0000 7f2225a00700 1 -- start start 2024-05-27T07:32:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f2225a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f222006c2c0 0x7f222019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f2225a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2220107470 0x7f222019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f2225a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2220109fe0 0x7f22201a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f2225a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f222010fcb0 con 0x7f2220109fe0 2024-05-27T07:32:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f2225a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f222010fb30 con 0x7f222006c2c0 2024-05-27T07:32:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f2225a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f222010fe30 con 0x7f2220107470 2024-05-27T07:32:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f221f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2220109fe0 0x7f22201a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f221f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2220109fe0 0x7f22201a3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53934/0 (socket says 172.21.15.90:53934) 2024-05-27T07:32:36.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f221f400700 1 -- 172.21.15.90:0/3795373261 learned_addr learned my addr 172.21.15.90:0/3795373261 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:32:36.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f221ea00700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f222006c2c0 0x7f222019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:36.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f221e000700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2220107470 0x7f222019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f221f400700 1 -- 172.21.15.90:0/3795373261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2220107470 msgr2=0x7f222019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f221f400700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2220107470 0x7f222019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.145+0000 7f221f400700 1 -- 172.21.15.90:0/3795373261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f222006c2c0 msgr2=0x7f222019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.146+0000 7f221f400700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f222006c2c0 0x7f222019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.146+0000 7f221f400700 1 -- 172.21.15.90:0/3795373261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22201a4590 con 0x7f2220109fe0 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.146+0000 7f221ea00700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f222006c2c0 0x7f222019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.146+0000 7f221f400700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2220109fe0 0x7f22201a3f10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2218007ee0 tx=0x7f221800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.146+0000 7f2217400700 1 -- 172.21.15.90:0/3795373261 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221800dbd0 con 0x7f2220109fe0 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.146+0000 7f2217400700 1 -- 172.21.15.90:0/3795373261 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f221800dd70 con 0x7f2220109fe0 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.146+0000 7f2225a00700 1 -- 172.21.15.90:0/3795373261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22201a4880 con 0x7f2220109fe0 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.146+0000 7f2217400700 1 -- 172.21.15.90:0/3795373261 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22180058d0 con 0x7f2220109fe0 2024-05-27T07:32:36.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.146+0000 7f2225a00700 1 -- 172.21.15.90:0/3795373261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22201a4dc0 con 0x7f2220109fe0 2024-05-27T07:32:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.148+0000 7f2225a00700 1 -- 172.21.15.90:0/3795373261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22200512b0 con 0x7f2220109fe0 2024-05-27T07:32:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.148+0000 7f2217400700 1 -- 172.21.15.90:0/3795373261 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f221801a020 con 0x7f2220109fe0 2024-05-27T07:32:36.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.148+0000 7f2217400700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f220403e750 0x7f2204040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:32:36.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.148+0000 7f2217400700 1 -- 172.21.15.90:0/3795373261 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2218052180 con 0x7f2220109fe0 2024-05-27T07:32:36.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.149+0000 7f221ea00700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f220403e750 0x7f2204040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:32:36.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.149+0000 7f221ea00700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f220403e750 0x7f2204040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f221001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:32:36.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.156+0000 7f2217400700 1 -- 172.21.15.90:0/3795373261 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2218010070 con 0x7f2220109fe0 2024-05-27T07:32:36.317 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:36.068+0000 7f0d3cfd7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:32:36.317 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:36.324+0000 7f0d3cfd7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:32:36.334 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:36 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:36.083+0000 7fd198cba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:32:36.334 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:36 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:36.341+0000 7fd198cba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:32:36.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:36.398+0000 7f2225a00700 1 -- 172.21.15.90:0/3795373261 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f222010ace0 con 0x7f220403e750 2024-05-27T07:32:36.807 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:36 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:36.475+0000 7fd198cba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:32:37.306 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:37 smithi129 ceph-mon[35254]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:37.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:37.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:37 smithi129 ceph-mon[35254]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:37.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:32:37.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:32:37.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:37.307 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:36 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:36.910+0000 7fd198cba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:32:37.308 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:37 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:37.034+0000 7fd198cba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:32:37.328 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:37 smithi090 ceph-mon[31429]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:37.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:37.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:37 smithi090 ceph-mon[31429]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:37.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:32:37.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:32:37.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:37.469 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:37 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:37.161+0000 7f0d3cfd7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:32:37.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:37.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:37 smithi136 ceph-mon[35324]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:37.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:37.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:37 smithi136 ceph-mon[35324]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:32:37.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:32:37.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:32:37.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:37.742 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:37 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:37.475+0000 7f0d3cfd7200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:32:38.187 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:38 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:38.056+0000 7f0d3cfd7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:32:38.492 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:38 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:38.194+0000 7f0d3cfd7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:32:38.992 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:38 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:38.668+0000 7f0d3cfd7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:32:39.056 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:38 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:38.728+0000 7fd198cba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:32:39.057 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:38 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:38.971+0000 7fd198cba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:32:39.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:39 smithi136 ceph-mon[35324]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:39.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.493 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:39.118+0000 7f0d3cfd7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:32:39.493 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:39.245+0000 7f0d3cfd7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:32:39.556 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:39 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:39.195+0000 7fd198cba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:32:39.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:39 smithi129 ceph-mon[35254]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:39.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:39 smithi090 ceph-mon[31429]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:39.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:39.878 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:39.570+0000 7f0d3cfd7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:32:39.879 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:39.754+0000 7f0d3cfd7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:32:39.879 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:39.885+0000 7f0d3cfd7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:32:39.981 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:39 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:39.608+0000 7fd198cba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:32:39.981 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:39 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:39.733+0000 7fd198cba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:32:39.982 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:39 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:39.988+0000 7fd198cba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:32:40.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:40.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:40.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:40.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:40 smithi136 ceph-mon[35324]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T07:32:40.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:32:40.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:32:40.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:40.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:40 smithi136 ceph-mon[35324]: Reconfiguring daemon mgr.x on smithi090 2024-05-27T07:32:40.494 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:40 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:40.291+0000 7f0d3cfd7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:32:40.494 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:40 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:40.416+0000 7f0d3cfd7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:32:40.557 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:40 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:40.245+0000 7fd198cba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:32:40.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:40.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:40.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:40.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:40 smithi129 ceph-mon[35254]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T07:32:40.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:32:40.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:32:40.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:40.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:40 smithi129 ceph-mon[35254]: Reconfiguring daemon mgr.x on smithi090 2024-05-27T07:32:40.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:40.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:40.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:40.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:40 smithi090 ceph-mon[31429]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T07:32:40.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T07:32:40.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T07:32:40.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:40.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:40 smithi090 ceph-mon[31429]: Reconfiguring daemon mgr.x on smithi090 2024-05-27T07:32:41.307 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:41 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:41.052+0000 7fd198cba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:32:41.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:41 smithi129 ceph-mon[35254]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:41.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:41 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2607276604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81e03104-5d6d-4892-a8c1-6a186048c90c"}]: dispatch 2024-05-27T07:32:41.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:41 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2607276604' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81e03104-5d6d-4892-a8c1-6a186048c90c"}]': finished 2024-05-27T07:32:41.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:41 smithi129 ceph-mon[35254]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T07:32:41.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:32:41.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:41 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1522300378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:32:41.464 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:41 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:41.179+0000 7f0d3cfd7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:32:41.465 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:41 smithi136 ceph-mon[35324]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:41.465 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:41 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2607276604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81e03104-5d6d-4892-a8c1-6a186048c90c"}]: dispatch 2024-05-27T07:32:41.465 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:41 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2607276604' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81e03104-5d6d-4892-a8c1-6a186048c90c"}]': finished 2024-05-27T07:32:41.465 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:41 smithi136 ceph-mon[35324]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T07:32:41.465 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:32:41.465 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:41 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1522300378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:32:41.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:41 smithi090 ceph-mon[31429]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:41.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:41 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2607276604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "81e03104-5d6d-4892-a8c1-6a186048c90c"}]: dispatch 2024-05-27T07:32:41.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:41 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2607276604' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "81e03104-5d6d-4892-a8c1-6a186048c90c"}]': finished 2024-05-27T07:32:41.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:41 smithi090 ceph-mon[31429]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T07:32:41.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:32:41.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:41 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1522300378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:32:41.742 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:32:41 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:32:41.471+0000 7f0d3cfd7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:32:41.806 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:41 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:41.344+0000 7fd198cba200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:32:42.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:42.261+0000 7f2217400700 1 -- 172.21.15.90:0/3795373261 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f22180133f0 con 0x7f2220109fe0 2024-05-27T07:32:42.263 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:41 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:41.904+0000 7fd198cba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:32:42.263 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:42 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:42.031+0000 7fd198cba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:32:42.557 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:42 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:42.504+0000 7fd198cba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:32:42.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:42 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:32:42.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:42 smithi129 ceph-mon[35254]: Standby manager daemon z started 2024-05-27T07:32:42.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:42 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:32:42.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:42 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:32:42.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:42 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:32:42.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:42 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:32:42.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:42 smithi090 ceph-mon[31429]: Standby manager daemon z started 2024-05-27T07:32:42.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:42 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:32:42.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:42 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:32:42.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:42 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:32:42.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:42 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:32:42.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:42 smithi136 ceph-mon[35324]: Standby manager daemon z started 2024-05-27T07:32:42.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:42 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:32:42.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:42 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:32:42.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:42 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.136:0/2711091626' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:32:43.267 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:42 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:42.956+0000 7fd198cba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:32:43.267 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:43.080+0000 7fd198cba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:32:43.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:43 smithi129 ceph-mon[35254]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:43.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:43 smithi129 ceph-mon[35254]: mgrmap e14: x(active, since 104s), standbys: z 2024-05-27T07:32:43.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:43 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:32:43.557 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:43.401+0000 7fd198cba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:32:43.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:43 smithi090 ceph-mon[31429]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:43.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:43 smithi090 ceph-mon[31429]: mgrmap e14: x(active, since 104s), standbys: z 2024-05-27T07:32:43.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:43 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:32:43.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:43 smithi136 ceph-mon[35324]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:43.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:43 smithi136 ceph-mon[35324]: mgrmap e14: x(active, since 104s), standbys: z 2024-05-27T07:32:43.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:43 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:32:44.057 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:43.580+0000 7fd198cba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:32:44.057 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:43.710+0000 7fd198cba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:32:44.556 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:44 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:44.116+0000 7fd198cba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:32:44.557 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:44 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:44.239+0000 7fd198cba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:32:45.273 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:45 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:45.003+0000 7fd198cba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:32:45.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:45 smithi090 ceph-mon[31429]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:45.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:45 smithi129 ceph-mon[35254]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:45.557 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:32:45 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:32:45.295+0000 7fd198cba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:32:45.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:45 smithi136 ceph-mon[35324]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:46.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:32:46.310+0000 7f2217400700 1 -- 172.21.15.90:0/3795373261 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f2218004ad0 con 0x7f2220109fe0 2024-05-27T07:32:46.419 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:46 smithi090 ceph-mon[31429]: Standby manager daemon y started 2024-05-27T07:32:46.419 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:46 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:32:46.419 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:46 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:32:46.419 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:46 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:32:46.419 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:46 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:32:46.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:46 smithi129 ceph-mon[35254]: Standby manager daemon y started 2024-05-27T07:32:46.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:46 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:32:46.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:46 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:32:46.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:46 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:32:46.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:46 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:32:46.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:46 smithi136 ceph-mon[35324]: Standby manager daemon y started 2024-05-27T07:32:46.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:46 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:32:46.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:46 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:32:46.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:46 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:32:46.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:46 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.129:0/3197246115' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:32:47.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:47 smithi090 ceph-mon[31429]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:47.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:47 smithi090 ceph-mon[31429]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-05-27T07:32:47.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:32:47.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:47.457 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:47.457 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:47.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:47 smithi129 ceph-mon[35254]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:47.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:47 smithi129 ceph-mon[35254]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-05-27T07:32:47.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:32:47.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:47.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:47.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:47.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:47 smithi136 ceph-mon[35324]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:47.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:47 smithi136 ceph-mon[35324]: mgrmap e15: x(active, since 108s), standbys: z, y 2024-05-27T07:32:47.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:32:47.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:47.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:47.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:48.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:48 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:48.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:48 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:48.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:48 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:48.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:48 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:48.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:48 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:48.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:48 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:48.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:48 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:48.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:48 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:32:48.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:48 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:49.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:49 smithi129 ceph-mon[35254]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:49.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:49 smithi090 ceph-mon[31429]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:49.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:49 smithi136 ceph-mon[35324]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:50.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T07:32:50.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:50.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:50 smithi129 ceph-mon[35254]: Deploying daemon osd.0 on smithi090 2024-05-27T07:32:50.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T07:32:50.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:50.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:50 smithi090 ceph-mon[31429]: Deploying daemon osd.0 on smithi090 2024-05-27T07:32:50.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T07:32:50.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:32:50.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:50 smithi136 ceph-mon[35324]: Deploying daemon osd.0 on smithi090 2024-05-27T07:32:51.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:51 smithi090 ceph-mon[31429]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:51.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:51 smithi136 ceph-mon[35324]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:51.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:51 smithi129 ceph-mon[35254]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:53.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:53 smithi090 ceph-mon[31429]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:53.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:53 smithi136 ceph-mon[35324]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:53.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:53 smithi129 ceph-mon[35254]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:55.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:55 smithi090 ceph-mon[31429]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:55.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:55 smithi136 ceph-mon[35324]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:55.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:55 smithi129 ceph-mon[35254]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:57.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:57 smithi090 ceph-mon[31429]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:57.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:57 smithi136 ceph-mon[35324]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:57.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:57 smithi129 ceph-mon[35254]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:59.412 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:59.413 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:59.413 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:59.413 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:32:59 smithi090 ceph-mon[31429]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:59.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:59.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:59.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:59.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:32:59 smithi136 ceph-mon[35324]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:32:59.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:59.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:32:59.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:32:59.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:32:59 smithi129 ceph-mon[35254]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:33:01.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:01 smithi090 ceph-mon[31429]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:33:01.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:01 smithi129 ceph-mon[35254]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:33:01.742 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:01 smithi136 ceph-mon[35324]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:33:02.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:02.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:02.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:02.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:02.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:02.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:02 smithi090 ceph-mon[31429]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T07:33:02.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:02.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:02.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:02.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:02.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:02.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:02 smithi136 ceph-mon[35324]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T07:33:02.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:02.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:02.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:02.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:02.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:02.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:02 smithi129 ceph-mon[35254]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T07:33:03.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.250+0000 7f2217400700 1 -- 172.21.15.90:0/3795373261 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f222010ace0 con 0x7f220403e750 2024-05-27T07:33:03.244 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 0 on host 'smithi090' 2024-05-27T07:33:03.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 -- 172.21.15.90:0/3795373261 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f220403e750 msgr2=0x7f2204040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:03.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f220403e750 0x7f2204040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2210006010 tx=0x7f221001a040 comp rx=0 tx=0).stop 2024-05-27T07:33:03.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 -- 172.21.15.90:0/3795373261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2220109fe0 msgr2=0x7f22201a3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:03.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2220109fe0 0x7f22201a3f10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2218007ee0 tx=0x7f221800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:33:03.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 -- 172.21.15.90:0/3795373261 shutdown_connections 2024-05-27T07:33:03.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f220403e750 0x7f2204040c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:03.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2220109fe0 0x7f22201a3f10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:03.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f222006c2c0 0x7f222019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:03.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 --2- 172.21.15.90:0/3795373261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2220107470 0x7f222019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:03.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 -- 172.21.15.90:0/3795373261 >> 172.21.15.90:0/3795373261 conn(0x7f22200fe870 msgr2=0x7f2220100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:03.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 -- 172.21.15.90:0/3795373261 shutdown_connections 2024-05-27T07:33:03.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:33:03.255+0000 7f2225a00700 1 -- 172.21.15.90:0/3795373261 wait complete. 2024-05-27T07:33:03.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:03 smithi136 ceph-mon[35324]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:33:03.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:03 smithi136 ceph-mon[35324]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T07:33:03.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:03 smithi136 ceph-mon[35324]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T07:33:03.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:03 smithi136 ceph-mon[35324]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:33:03.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:03.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:03.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:03.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:03.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:03 smithi129 ceph-mon[35254]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:33:03.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:03 smithi129 ceph-mon[35254]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T07:33:03.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:03 smithi129 ceph-mon[35254]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T07:33:03.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:03 smithi129 ceph-mon[35254]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:33:03.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:03.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:03.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:03.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:03.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:03 smithi090 ceph-mon[31429]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:33:03.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:03 smithi090 ceph-mon[31429]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T07:33:03.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:03 smithi090 ceph-mon[31429]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T07:33:03.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:03 smithi090 ceph-mon[31429]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:33:03.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:03.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:03.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:03.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:04.015 DEBUG:teuthology.orchestra.run.smithi090:osd.0> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.0.service 2024-05-27T07:33:04.020 INFO:tasks.cephadm:Deploying osd.1 on smithi129 with /dev/vg_nvme/lv_4... 2024-05-27T07:33:04.020 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T07:33:04.452 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:33:04.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:04 smithi090 ceph-mon[31429]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:33:04.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:04 smithi090 ceph-mon[31429]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T07:33:04.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:04 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:04.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:04 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:04.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:04 smithi136 ceph-mon[35324]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:33:04.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:04 smithi136 ceph-mon[35324]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T07:33:04.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:04 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:04.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:04 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:04.779 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:04 smithi129 ceph-mon[35254]: from='osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:33:04.779 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:04 smithi129 ceph-mon[35254]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T07:33:04.779 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:04 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:04.779 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:04 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:05.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:05 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:33:05.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:05 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:33:05.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:05 smithi090 ceph-mon[31429]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:33:05.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:05 smithi090 ceph-mon[31429]: osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] boot 2024-05-27T07:33:05.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:05 smithi090 ceph-mon[31429]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T07:33:05.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:05 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:05.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:05 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:33:05.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:05 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:33:05.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:05 smithi136 ceph-mon[35324]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:33:05.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:05 smithi136 ceph-mon[35324]: osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] boot 2024-05-27T07:33:05.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:05 smithi136 ceph-mon[35324]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T07:33:05.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:05 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:05.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:05 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:33:05.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:05 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:33:05.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:05 smithi129 ceph-mon[35254]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T07:33:05.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:05 smithi129 ceph-mon[35254]: osd.0 [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] boot 2024-05-27T07:33:05.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:05 smithi129 ceph-mon[35254]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T07:33:05.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:05 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:33:06.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:06 smithi129 ceph-mon[35254]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T07:33:06.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:06 smithi090 ceph-mon[31429]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T07:33:06.992 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:06 smithi136 ceph-mon[35324]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T07:33:07.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:07 smithi129 ceph-mon[35254]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:07.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:07 smithi090 ceph-mon[31429]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:07.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:07 smithi136 ceph-mon[35324]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:08.109 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T07:33:08.109 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T07:33:08.109 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-05-27T07:33:08.109 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-05-27T07:33:08.110 INFO:teuthology.orchestra.run.smithi129.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0325754 s, 322 MB/s 2024-05-27T07:33:08.110 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-05-27T07:33:08.964 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_4 2024-05-27T07:33:09.381 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:33:09.703 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:09 smithi129 ceph-mon[35254]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:09.703 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:09 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi090 2024-05-27T07:33:09.703 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.703 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.703 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:09.703 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:09 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi090 to 17089M 2024-05-27T07:33:09.703 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.703 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:09.703 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:09.703 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:09 smithi090 ceph-mon[31429]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:09.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:09 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi090 2024-05-27T07:33:09.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:09.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:09 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi090 to 17089M 2024-05-27T07:33:09.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:09.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:09.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:09 smithi136 ceph-mon[35324]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:09.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:09 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi090 2024-05-27T07:33:09.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:09.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:09 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi090 to 17089M 2024-05-27T07:33:09.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:09.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:09.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:09.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.653+0000 7fd59a800700 1 -- 172.21.15.129:0/1943746377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5941074b0 msgr2=0x7fd5941078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:11.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.653+0000 7fd59a800700 1 --2- 172.21.15.129:0/1943746377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5941074b0 0x7fd5941078c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd580009990 tx=0x7fd580009ca0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.653+0000 7fd59a800700 1 -- 172.21.15.129:0/1943746377 shutdown_connections 2024-05-27T07:33:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.653+0000 7fd59a800700 1 --2- 172.21.15.129:0/1943746377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd594107e50 0x7fd59410e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.653+0000 7fd59a800700 1 --2- 172.21.15.129:0/1943746377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd594106380 0x7fd594106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.653+0000 7fd59a800700 1 --2- 172.21.15.129:0/1943746377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5941074b0 0x7fd5941078c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.653+0000 7fd59a800700 1 -- 172.21.15.129:0/1943746377 >> 172.21.15.129:0/1943746377 conn(0x7fd594101a00 msgr2=0x7fd594103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.653+0000 7fd59a800700 1 -- 172.21.15.129:0/1943746377 shutdown_connections 2024-05-27T07:33:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.653+0000 7fd59a800700 1 -- 172.21.15.129:0/1943746377 wait complete. 2024-05-27T07:33:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.654+0000 7fd59a800700 1 Processor -- start 2024-05-27T07:33:11.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.655+0000 7fd59a800700 1 -- start start 2024-05-27T07:33:11.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.655+0000 7fd59a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd594106380 0x7fd59419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.655+0000 7fd59a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5941074b0 0x7fd59419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.655+0000 7fd59a800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd594107e50 0x7fd5941a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.655+0000 7fd59a800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd594111cf0 con 0x7fd594106380 2024-05-27T07:33:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.655+0000 7fd59a800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd594111b70 con 0x7fd594107e50 2024-05-27T07:33:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.655+0000 7fd59a800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd594111e70 con 0x7fd5941074b0 2024-05-27T07:33:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd594107e50 0x7fd5941a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd594107e50 0x7fd5941a3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34558/0 (socket says 172.21.15.129:34558) 2024-05-27T07:33:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599e00700 1 -- 172.21.15.129:0/3512519466 learned_addr learned my addr 172.21.15.129:0/3512519466 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-05-27T07:33:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd598a00700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5941074b0 0x7fd59419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:11.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599400700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd594106380 0x7fd59419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:11.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599e00700 1 -- 172.21.15.129:0/3512519466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5941074b0 msgr2=0x7fd59419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:11.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599e00700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5941074b0 0x7fd59419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599e00700 1 -- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd594106380 msgr2=0x7fd59419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:11.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599e00700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd594106380 0x7fd59419c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599e00700 1 -- 172.21.15.129:0/3512519466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5941a4370 con 0x7fd594107e50 2024-05-27T07:33:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599400700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd594106380 0x7fd59419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:33:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.656+0000 7fd599e00700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd594107e50 0x7fd5941a3d80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00c900 tx=0x7fd58c00cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.657+0000 7fd592000700 1 -- 172.21.15.129:0/3512519466 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c007b00 con 0x7fd594107e50 2024-05-27T07:33:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.657+0000 7fd592000700 1 -- 172.21.15.129:0/3512519466 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd58c00ce50 con 0x7fd594107e50 2024-05-27T07:33:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.657+0000 7fd59a800700 1 -- 172.21.15.129:0/3512519466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5941a4660 con 0x7fd594107e50 2024-05-27T07:33:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.657+0000 7fd592000700 1 -- 172.21.15.129:0/3512519466 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c018600 con 0x7fd594107e50 2024-05-27T07:33:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.657+0000 7fd59a800700 1 -- 172.21.15.129:0/3512519466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5941a4b50 con 0x7fd594107e50 2024-05-27T07:33:11.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.658+0000 7fd59a800700 1 -- 172.21.15.129:0/3512519466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5940512b0 con 0x7fd594107e50 2024-05-27T07:33:11.657 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.663+0000 7fd592000700 1 -- 172.21.15.129:0/3512519466 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fd58c00f450 con 0x7fd594107e50 2024-05-27T07:33:11.657 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:11 smithi129 ceph-mon[35254]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:11.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.664+0000 7fd592000700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd57c0b27f0 0x7fd57c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:11.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.664+0000 7fd592000700 1 -- 172.21.15.129:0/3512519466 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fd58c0e0ed0 con 0x7fd594107e50 2024-05-27T07:33:11.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.665+0000 7fd599400700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd57c0b27f0 0x7fd57c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:11.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.667+0000 7fd599400700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd57c0b27f0 0x7fd57c0b4d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5941073b0 tx=0x7fd588009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:11.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.668+0000 7fd592000700 1 -- 172.21.15.129:0/3512519466 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd58c014030 con 0x7fd594107e50 2024-05-27T07:33:11.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:11 smithi090 ceph-mon[31429]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:11.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:11 smithi136 ceph-mon[35324]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:11.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:11.909+0000 7fd59a800700 1 -- 172.21.15.129:0/3512519466 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd594063bf0 con 0x7fd57c0b27f0 2024-05-27T07:33:12.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:33:12.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:33:12.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:12.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:33:12.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:33:12.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:12.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:33:12.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:33:12.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:13.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:13 smithi090 ceph-mon[31429]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:13.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:13 smithi090 ceph-mon[31429]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:13.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:13 smithi136 ceph-mon[35324]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:13.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:13 smithi136 ceph-mon[35324]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:13.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:13 smithi129 ceph-mon[35254]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:13.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:13 smithi129 ceph-mon[35254]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:15.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:15 smithi136 ceph-mon[35324]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:15.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:15 smithi129 ceph-mon[35254]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:15.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:15 smithi090 ceph-mon[31429]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:16.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:16 smithi136 ceph-mon[35324]: from='client.? 172.21.15.129:0/2713619046' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a"}]: dispatch 2024-05-27T07:33:16.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:16 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a"}]: dispatch 2024-05-27T07:33:16.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:16 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a"}]': finished 2024-05-27T07:33:16.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:16 smithi136 ceph-mon[35324]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T07:33:16.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:16 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:16.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:16 smithi129 ceph-mon[35254]: from='client.? 172.21.15.129:0/2713619046' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a"}]: dispatch 2024-05-27T07:33:16.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:16 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a"}]: dispatch 2024-05-27T07:33:16.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:16 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a"}]': finished 2024-05-27T07:33:16.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:16 smithi129 ceph-mon[35254]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T07:33:16.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:16 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:16.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:16 smithi090 ceph-mon[31429]: from='client.? 172.21.15.129:0/2713619046' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a"}]: dispatch 2024-05-27T07:33:16.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:16 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a"}]: dispatch 2024-05-27T07:33:16.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:16 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a"}]': finished 2024-05-27T07:33:16.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:16 smithi090 ceph-mon[31429]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T07:33:16.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:17.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:17 smithi136 ceph-mon[35324]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:17.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:17 smithi136 ceph-mon[35324]: from='client.? 172.21.15.129:0/3470288585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:33:17.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:17 smithi129 ceph-mon[35254]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:17.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:17 smithi129 ceph-mon[35254]: from='client.? 172.21.15.129:0/3470288585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:33:17.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:17 smithi090 ceph-mon[31429]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:17.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:17 smithi090 ceph-mon[31429]: from='client.? 172.21.15.129:0/3470288585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:33:19.671 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:19 smithi129 ceph-mon[35254]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:19.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:19 smithi136 ceph-mon[35324]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:19.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:19 smithi090 ceph-mon[31429]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:21.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:21 smithi136 ceph-mon[35324]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:21.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:21 smithi129 ceph-mon[35254]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:21.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:21 smithi090 ceph-mon[31429]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:23.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:23 smithi136 ceph-mon[35324]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:23.744 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:23 smithi129 ceph-mon[35254]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:23.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:23 smithi090 ceph-mon[31429]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:25.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:25 smithi090 ceph-mon[31429]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:25.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T07:33:25.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:25.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:25 smithi129 ceph-mon[35254]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:25.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T07:33:25.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:25.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:25 smithi136 ceph-mon[35324]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:25.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T07:33:25.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:26.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:26 smithi129 ceph-mon[35254]: Deploying daemon osd.1 on smithi129 2024-05-27T07:33:26.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:26 smithi090 ceph-mon[31429]: Deploying daemon osd.1 on smithi129 2024-05-27T07:33:26.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:26 smithi136 ceph-mon[35324]: Deploying daemon osd.1 on smithi129 2024-05-27T07:33:27.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:27 smithi129 ceph-mon[35254]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:27.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:27 smithi090 ceph-mon[31429]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:27.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:27 smithi136 ceph-mon[35324]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:29.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:29 smithi129 ceph-mon[35254]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:29.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:29 smithi090 ceph-mon[31429]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:29.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:29 smithi136 ceph-mon[35324]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:31.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:31 smithi129 ceph-mon[35254]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:32.612 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:31 smithi136 ceph-mon[35324]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:32.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:31 smithi090 ceph-mon[31429]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:33.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:33 smithi129 ceph-mon[35254]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:33.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:33 smithi090 ceph-mon[31429]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:33.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:33 smithi136 ceph-mon[35324]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:35.207 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:35 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:35.207 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:35 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:35.207 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:35 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:35.207 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:35 smithi090 ceph-mon[31429]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:35.298 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:35 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:35.298 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:35 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:35.298 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:35 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:35.298 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:35 smithi129 ceph-mon[35254]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:35.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:35 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:35.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:35 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:35.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:35 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:35.493 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:35 smithi136 ceph-mon[35324]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:37.307 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:37 smithi129 ceph-mon[35254]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:37.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:37 smithi090 ceph-mon[31429]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:37.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:37 smithi136 ceph-mon[35324]: pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:38.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:38 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:38.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:38 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:38.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:38 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:38.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:38 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:38.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:38 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:38.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:38 smithi129 ceph-mon[35254]: from='osd.1 [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:33:38.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:38 smithi129 ceph-mon[35254]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:33:38.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:38 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:38.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:38 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:38.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:38 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:38.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:38 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:38.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:38 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:38.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:38 smithi090 ceph-mon[31429]: from='osd.1 [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:33:38.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:38 smithi090 ceph-mon[31429]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:33:38.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:38 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:38.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:38 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:38.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:38 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:38.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:38 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:38.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:38 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:38.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:38 smithi136 ceph-mon[35324]: from='osd.1 [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:33:38.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:38 smithi136 ceph-mon[35324]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T07:33:39.509 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:39 smithi129 ceph-mon[35254]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:39.509 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:39 smithi129 ceph-mon[35254]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T07:33:39.509 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:39 smithi129 ceph-mon[35254]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T07:33:39.509 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:39.510 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:39 smithi129 ceph-mon[35254]: from='osd.1 [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:33:39.510 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:39 smithi129 ceph-mon[35254]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:33:39.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.680+0000 7fd592000700 1 -- 172.21.15.129:0/3512519466 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd594063bf0 con 0x7fd57c0b27f0 2024-05-27T07:33:39.675 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 1 on host 'smithi129' 2024-05-27T07:33:39.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.685+0000 7fd59a800700 1 -- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd57c0b27f0 msgr2=0x7fd57c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:39.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.685+0000 7fd59a800700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd57c0b27f0 0x7fd57c0b4d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5941073b0 tx=0x7fd588009040 comp rx=0 tx=0).stop 2024-05-27T07:33:39.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.685+0000 7fd59a800700 1 -- 172.21.15.129:0/3512519466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd594107e50 msgr2=0x7fd5941a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:39.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.685+0000 7fd59a800700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd594107e50 0x7fd5941a3d80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00c900 tx=0x7fd58c00cc10 comp rx=0 tx=0).stop 2024-05-27T07:33:39.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.686+0000 7fd59a800700 1 -- 172.21.15.129:0/3512519466 shutdown_connections 2024-05-27T07:33:39.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.686+0000 7fd59a800700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd57c0b27f0 0x7fd57c0b4d00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:39.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.686+0000 7fd59a800700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd594107e50 0x7fd5941a3d80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:39.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.686+0000 7fd59a800700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd594106380 0x7fd59419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:39.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.686+0000 7fd59a800700 1 --2- 172.21.15.129:0/3512519466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5941074b0 0x7fd59419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.686+0000 7fd59a800700 1 -- 172.21.15.129:0/3512519466 >> 172.21.15.129:0/3512519466 conn(0x7fd594101a00 msgr2=0x7fd59410a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.686+0000 7fd59a800700 1 -- 172.21.15.129:0/3512519466 shutdown_connections 2024-05-27T07:33:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:33:39.686+0000 7fd59a800700 1 -- 172.21.15.129:0/3512519466 wait complete. 2024-05-27T07:33:39.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:39 smithi090 ceph-mon[31429]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:39.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:39 smithi090 ceph-mon[31429]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T07:33:39.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:39 smithi090 ceph-mon[31429]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T07:33:39.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:39.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:39 smithi090 ceph-mon[31429]: from='osd.1 [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:33:39.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:39 smithi090 ceph-mon[31429]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:33:39.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:39 smithi136 ceph-mon[35324]: pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:39.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:39 smithi136 ceph-mon[35324]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T07:33:39.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:39 smithi136 ceph-mon[35324]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T07:33:39.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:39.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:39 smithi136 ceph-mon[35324]: from='osd.1 [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:33:39.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:39 smithi136 ceph-mon[35324]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:33:40.459 DEBUG:teuthology.orchestra.run.smithi129:osd.1> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.1.service 2024-05-27T07:33:40.464 INFO:tasks.cephadm:Deploying osd.2 on smithi136 with /dev/vg_nvme/lv_4... 2024-05-27T07:33:40.464 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T07:33:40.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:40 smithi090 ceph-mon[31429]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:33:40.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:40 smithi090 ceph-mon[31429]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T07:33:40.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:40.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:40.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:40.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:40.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:40.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:40 smithi090 ceph-mon[31429]: from='osd.1 ' entity='osd.1' 2024-05-27T07:33:40.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:40 smithi136 ceph-mon[35324]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:33:40.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:40 smithi136 ceph-mon[35324]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T07:33:40.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:40.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:40.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:40.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:40.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:40.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:40 smithi136 ceph-mon[35324]: from='osd.1 ' entity='osd.1' 2024-05-27T07:33:40.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:40 smithi129 ceph-mon[35254]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:33:40.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:40 smithi129 ceph-mon[35254]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T07:33:40.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:40.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:40.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:33:40.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:40.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:40.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:40 smithi129 ceph-mon[35254]: from='osd.1 ' entity='osd.1' 2024-05-27T07:33:40.910 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:33:41.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:41 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:33:41.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:41 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:33:41.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:41 smithi090 ceph-mon[31429]: pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:41.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:41.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:41 smithi090 ceph-mon[31429]: osd.1 [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] boot 2024-05-27T07:33:41.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:41 smithi090 ceph-mon[31429]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T07:33:41.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:41.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:41 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:33:41.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:41 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:33:41.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:41 smithi136 ceph-mon[35324]: pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:41.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:41.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:41 smithi136 ceph-mon[35324]: osd.1 [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] boot 2024-05-27T07:33:41.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:41 smithi136 ceph-mon[35324]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T07:33:41.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:41.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:41 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:33:41.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:41 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:33:41.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:41 smithi129 ceph-mon[35254]: pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T07:33:41.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:41.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:41 smithi129 ceph-mon[35254]: osd.1 [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] boot 2024-05-27T07:33:41.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:41 smithi129 ceph-mon[35254]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T07:33:41.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:33:43.657 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T07:33:43.657 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T07:33:43.657 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-05-27T07:33:43.657 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-05-27T07:33:43.658 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0337619 s, 311 MB/s 2024-05-27T07:33:43.658 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-05-27T07:33:43.658 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:43 smithi136 ceph-mon[35324]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:43.658 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:43 smithi136 ceph-mon[35324]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T07:33:43.673 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:43 smithi129 ceph-mon[35254]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:43.673 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:43 smithi129 ceph-mon[35254]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T07:33:43.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:43 smithi090 ceph-mon[31429]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:43.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:43 smithi090 ceph-mon[31429]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T07:33:44.438 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_4 2024-05-27T07:33:44.879 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:33:45.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:45 smithi090 ceph-mon[31429]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:45.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:45 smithi136 ceph-mon[35324]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:45.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:45 smithi129 ceph-mon[35254]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:46.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.877+0000 7f33c1800700 1 -- 172.21.15.136:0/4076635121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33bc104330 msgr2=0x7f33bc104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:46.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.877+0000 7f33c1800700 1 --2- 172.21.15.136:0/4076635121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33bc104330 0x7f33bc104740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009990 tx=0x7f33ac009ca0 comp rx=0 tx=0).stop 2024-05-27T07:33:46.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.878+0000 7f33c1800700 1 -- 172.21.15.136:0/4076635121 shutdown_connections 2024-05-27T07:33:46.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.878+0000 7f33c1800700 1 --2- 172.21.15.136:0/4076635121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33bc105e00 0x7f33bc10c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:46.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.878+0000 7f33c1800700 1 --2- 172.21.15.136:0/4076635121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33bc104330 0x7f33bc104740 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:46.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.878+0000 7f33c1800700 1 --2- 172.21.15.136:0/4076635121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33bc105460 0x7f33bc105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:46.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.878+0000 7f33c1800700 1 -- 172.21.15.136:0/4076635121 >> 172.21.15.136:0/4076635121 conn(0x7f33bc0ff9b0 msgr2=0x7f33bc101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:33:46.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.878+0000 7f33c1800700 1 -- 172.21.15.136:0/4076635121 shutdown_connections 2024-05-27T07:33:46.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.878+0000 7f33c1800700 1 -- 172.21.15.136:0/4076635121 wait complete. 2024-05-27T07:33:46.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.879+0000 7f33c1800700 1 Processor -- start 2024-05-27T07:33:46.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.879+0000 7f33c1800700 1 -- start start 2024-05-27T07:33:46.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.879+0000 7f33c1800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33bc104330 0x7f33bc19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:46.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.879+0000 7f33c1800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33bc105460 0x7f33bc19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:46.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.879+0000 7f33c1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33bc105e00 0x7f33bc1a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:46.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.879+0000 7f33c1800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc10fc70 con 0x7f33bc105e00 2024-05-27T07:33:46.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.879+0000 7f33c1800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc10faf0 con 0x7f33bc104330 2024-05-27T07:33:46.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.879+0000 7f33c1800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc10fdf0 con 0x7f33bc105460 2024-05-27T07:33:46.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33ba000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33bc105460 0x7f33bc19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:46.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33ba000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33bc105460 0x7f33bc19cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:37576/0 (socket says 172.21.15.136:37576) 2024-05-27T07:33:46.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33ba000700 1 -- 172.21.15.136:0/3767314438 learned_addr learned my addr 172.21.15.136:0/3767314438 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33baa00700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33bc104330 0x7f33bc19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33bb400700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33bc105e00 0x7f33bc1a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33ba000700 1 -- 172.21.15.136:0/3767314438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33bc104330 msgr2=0x7f33bc19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33ba000700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33bc104330 0x7f33bc19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33ba000700 1 -- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33bc105e00 msgr2=0x7f33bc1a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33ba000700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33bc105e00 0x7f33bc1a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33ba000700 1 -- 172.21.15.136:0/3767314438 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc1a4520 con 0x7f33bc105460 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.880+0000 7f33bb400700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33bc105e00 0x7f33bc1a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.881+0000 7f33ba000700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33bc105460 0x7f33bc19cf90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f33a800c9b0 tx=0x7f33a800cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.881+0000 7f33b3400700 1 -- 172.21.15.136:0/3767314438 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8007b00 con 0x7f33bc105460 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.881+0000 7f33b3400700 1 -- 172.21.15.136:0/3767314438 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33a8007ca0 con 0x7f33bc105460 2024-05-27T07:33:46.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.881+0000 7f33c1800700 1 -- 172.21.15.136:0/3767314438 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc1a4810 con 0x7f33bc105460 2024-05-27T07:33:46.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.881+0000 7f33b3400700 1 -- 172.21.15.136:0/3767314438 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8018660 con 0x7f33bc105460 2024-05-27T07:33:46.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.881+0000 7f33c1800700 1 -- 172.21.15.136:0/3767314438 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc1a4d50 con 0x7f33bc105460 2024-05-27T07:33:46.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.883+0000 7f33c1800700 1 -- 172.21.15.136:0/3767314438 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33bc0512b0 con 0x7f33bc105460 2024-05-27T07:33:46.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.886+0000 7f33b3400700 1 -- 172.21.15.136:0/3767314438 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f33a800f450 con 0x7f33bc105460 2024-05-27T07:33:46.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.887+0000 7f33b3400700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33a00b27f0 0x7f33a00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:33:46.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.888+0000 7f33b3400700 1 -- 172.21.15.136:0/3767314438 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f33a80e0ad0 con 0x7f33bc105460 2024-05-27T07:33:46.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.888+0000 7f33baa00700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33a00b27f0 0x7f33a00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:33:46.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.890+0000 7f33baa00700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33a00b27f0 0x7f33a00b4d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33ac006010 tx=0x7f33ac01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:33:46.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:46.894+0000 7f33b3400700 1 -- 172.21.15.136:0/3767314438 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33a80e2e20 con 0x7f33bc105460 2024-05-27T07:33:47.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:33:47.136+0000 7f33c1800700 1 -- 172.21.15.136:0/3767314438 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f33bc077100 con 0x7f33a00b27f0 2024-05-27T07:33:47.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:47.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi129 2024-05-27T07:33:47.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:47.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi129 to 17089M 2024-05-27T07:33:47.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:47.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:47.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:33:47.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:33:47.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:47.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:47.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi129 2024-05-27T07:33:47.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:47.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi129 to 17089M 2024-05-27T07:33:47.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:47.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:47.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:33:47.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:33:47.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:47.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:47.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi129 2024-05-27T07:33:47.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi129 to 17089M 2024-05-27T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:33:47.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:33:48.557 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:48 smithi129 ceph-mon[35254]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:48.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:48 smithi136 ceph-mon[35324]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:48.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:48 smithi090 ceph-mon[31429]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:33:49.671 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:49 smithi136 ceph-mon[35324]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:49.698 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:49 smithi129 ceph-mon[35254]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:49.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:49 smithi090 ceph-mon[31429]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:51.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:51 smithi136 ceph-mon[35324]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:51.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:51 smithi136 ceph-mon[35324]: from='client.? 172.21.15.136:0/1861720823' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b859166-8b57-42a1-852f-93a38ab9102c"}]: dispatch 2024-05-27T07:33:51.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:51 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b859166-8b57-42a1-852f-93a38ab9102c"}]: dispatch 2024-05-27T07:33:51.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:51 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b859166-8b57-42a1-852f-93a38ab9102c"}]': finished 2024-05-27T07:33:51.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:51 smithi136 ceph-mon[35324]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T07:33:51.741 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:51 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:33:51.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:51 smithi129 ceph-mon[35254]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:51.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:51 smithi129 ceph-mon[35254]: from='client.? 172.21.15.136:0/1861720823' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b859166-8b57-42a1-852f-93a38ab9102c"}]: dispatch 2024-05-27T07:33:51.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:51 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b859166-8b57-42a1-852f-93a38ab9102c"}]: dispatch 2024-05-27T07:33:51.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:51 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b859166-8b57-42a1-852f-93a38ab9102c"}]': finished 2024-05-27T07:33:51.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:51 smithi129 ceph-mon[35254]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T07:33:51.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:51 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:33:51.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:51 smithi090 ceph-mon[31429]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:51.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:51 smithi090 ceph-mon[31429]: from='client.? 172.21.15.136:0/1861720823' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b859166-8b57-42a1-852f-93a38ab9102c"}]: dispatch 2024-05-27T07:33:51.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:51 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b859166-8b57-42a1-852f-93a38ab9102c"}]: dispatch 2024-05-27T07:33:51.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:51 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b859166-8b57-42a1-852f-93a38ab9102c"}]': finished 2024-05-27T07:33:51.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:51 smithi090 ceph-mon[31429]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T07:33:51.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:33:52.743 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:52 smithi136 ceph-mon[35324]: from='client.? 172.21.15.136:0/4216142869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:33:52.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:52 smithi129 ceph-mon[35254]: from='client.? 172.21.15.136:0/4216142869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:33:52.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:52 smithi090 ceph-mon[31429]: from='client.? 172.21.15.136:0/4216142869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:33:53.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:53 smithi129 ceph-mon[35254]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:53.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:53 smithi090 ceph-mon[31429]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:53.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:53 smithi136 ceph-mon[35324]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:55.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:55 smithi129 ceph-mon[35254]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:55.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:55 smithi090 ceph-mon[31429]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:55.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:55 smithi136 ceph-mon[35324]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:57.703 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:57 smithi136 ceph-mon[35324]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:57.807 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:57 smithi129 ceph-mon[35254]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:57.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:57 smithi090 ceph-mon[31429]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:59.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:33:59 smithi129 ceph-mon[35254]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:59.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:33:59 smithi090 ceph-mon[31429]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:33:59.966 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:33:59 smithi136 ceph-mon[35324]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:00.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T07:34:00.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:00.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T07:34:00.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:00.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T07:34:00.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:01.664 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:01 smithi136 ceph-mon[35324]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:01.664 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:01 smithi136 ceph-mon[35324]: Deploying daemon osd.2 on smithi136 2024-05-27T07:34:01.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:01 smithi129 ceph-mon[35254]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:01.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:01 smithi129 ceph-mon[35254]: Deploying daemon osd.2 on smithi136 2024-05-27T07:34:01.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:01 smithi090 ceph-mon[31429]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:01.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:01 smithi090 ceph-mon[31429]: Deploying daemon osd.2 on smithi136 2024-05-27T07:34:03.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:03 smithi136 ceph-mon[35324]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:03.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:03 smithi129 ceph-mon[35254]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:03.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:03 smithi090 ceph-mon[31429]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:05.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:05 smithi129 ceph-mon[35254]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:05.867 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:05 smithi136 ceph-mon[35324]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:05.919 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:05 smithi090 ceph-mon[31429]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:07.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:07 smithi129 ceph-mon[35254]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:07.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:07 smithi090 ceph-mon[31429]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:07.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:07 smithi136 ceph-mon[35324]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:09.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:09 smithi090 ceph-mon[31429]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:09.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:09.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:09.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:09.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:09 smithi136 ceph-mon[35324]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:09.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:09.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:09.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:10.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:09 smithi129 ceph-mon[35254]: pgmap v100: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:10.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:10.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:10.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:11.877 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:11 smithi136 ceph-mon[35324]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:11.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:11 smithi090 ceph-mon[31429]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:12.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:11 smithi129 ceph-mon[35254]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:13.602 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:13 smithi136 ceph-mon[35324]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:13.602 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:13 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:13.602 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:13 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:13.602 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:13 smithi136 ceph-mon[35324]: from='osd.2 [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:34:13.602 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:13 smithi136 ceph-mon[35324]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:34:13.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:13 smithi090 ceph-mon[31429]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:13.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:13 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:13.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:13 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:13.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:13 smithi090 ceph-mon[31429]: from='osd.2 [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:34:13.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:13 smithi090 ceph-mon[31429]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:34:13.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:13 smithi129 ceph-mon[35254]: pgmap v102: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:13.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:13 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:13.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:13 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:13.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:13 smithi129 ceph-mon[35254]: from='osd.2 [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:34:13.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:13 smithi129 ceph-mon[35254]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T07:34:14.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.350+0000 7f33b3400700 1 -- 172.21.15.136:0/3767314438 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f33bc077100 con 0x7f33a00b27f0 2024-05-27T07:34:14.346 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 2 on host 'smithi136' 2024-05-27T07:34:14.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.354+0000 7f33c1800700 1 -- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33a00b27f0 msgr2=0x7f33a00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:14.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.354+0000 7f33c1800700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33a00b27f0 0x7f33a00b4d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33ac006010 tx=0x7f33ac01a040 comp rx=0 tx=0).stop 2024-05-27T07:34:14.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.354+0000 7f33c1800700 1 -- 172.21.15.136:0/3767314438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33bc105460 msgr2=0x7f33bc19cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:14.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.354+0000 7f33c1800700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33bc105460 0x7f33bc19cf90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f33a800c9b0 tx=0x7f33a800cd70 comp rx=0 tx=0).stop 2024-05-27T07:34:14.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.354+0000 7f33c1800700 1 -- 172.21.15.136:0/3767314438 shutdown_connections 2024-05-27T07:34:14.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.355+0000 7f33c1800700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33a00b27f0 0x7f33a00b4d00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:14.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.355+0000 7f33c1800700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33bc105e00 0x7f33bc1a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:14.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.355+0000 7f33c1800700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33bc104330 0x7f33bc19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:14.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.355+0000 7f33c1800700 1 --2- 172.21.15.136:0/3767314438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33bc105460 0x7f33bc19cf90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:14.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.355+0000 7f33c1800700 1 -- 172.21.15.136:0/3767314438 >> 172.21.15.136:0/3767314438 conn(0x7f33bc0ff9b0 msgr2=0x7f33bc0759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:14.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.355+0000 7f33c1800700 1 -- 172.21.15.136:0/3767314438 shutdown_connections 2024-05-27T07:34:14.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:34:14.355+0000 7f33c1800700 1 -- 172.21.15.136:0/3767314438 wait complete. 2024-05-27T07:34:14.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:14.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:14.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:14.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T07:34:14.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T07:34:14.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:14.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: from='osd.2 [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:34:14.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:34:14.494 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:14.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:14.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:14.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:14.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:14.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:14.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T07:34:14.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T07:34:14.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:14.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: from='osd.2 [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:34:14.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:34:14.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:14.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:14.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:14.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:14.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:14.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:14.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T07:34:14.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T07:34:14.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:14.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: from='osd.2 [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:34:14.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:34:14.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:14.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:14.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:15.160 DEBUG:teuthology.orchestra.run.smithi136:osd.2> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.2.service 2024-05-27T07:34:15.165 INFO:tasks.cephadm:Deploying osd.3 on smithi090 with /dev/vg_nvme/lv_3... 2024-05-27T07:34:15.165 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T07:34:15.411 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 07:34:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2[40317]: 2024-05-27T07:34:15.312+0000 7f8645800700 -1 osd.2 0 waiting for initial osdmap 2024-05-27T07:34:15.411 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 07:34:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2[40317]: 2024-05-27T07:34:15.327+0000 7f863b600700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T07:34:15.623 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:34:15.729 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:15 smithi136 ceph-mon[35324]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:15.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:15 smithi136 ceph-mon[35324]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:34:15.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:15 smithi136 ceph-mon[35324]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T07:34:15.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:15.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:15.730 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:15 smithi136 ceph-mon[35324]: from='osd.2 ' entity='osd.2' 2024-05-27T07:34:15.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:15 smithi090 ceph-mon[31429]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:15.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:15 smithi090 ceph-mon[31429]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:34:15.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:15 smithi090 ceph-mon[31429]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T07:34:15.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:15.919 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:15.919 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:15 smithi090 ceph-mon[31429]: from='osd.2 ' entity='osd.2' 2024-05-27T07:34:16.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:15 smithi129 ceph-mon[35254]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:16.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:15 smithi129 ceph-mon[35254]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:34:16.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:15 smithi129 ceph-mon[35254]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T07:34:16.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:16.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:16.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:15 smithi129 ceph-mon[35254]: from='osd.2 ' entity='osd.2' 2024-05-27T07:34:16.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:16 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:34:16.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:16 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:34:16.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:16 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:16.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:16 smithi136 ceph-mon[35324]: osd.2 [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] boot 2024-05-27T07:34:16.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:16 smithi136 ceph-mon[35324]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T07:34:16.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:16 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:16.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:16 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:34:16.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:16 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:34:16.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:16.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:16 smithi090 ceph-mon[31429]: osd.2 [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] boot 2024-05-27T07:34:16.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:16 smithi090 ceph-mon[31429]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T07:34:16.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:17.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:16 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:34:17.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:16 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:34:17.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:16 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:17.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:16 smithi129 ceph-mon[35254]: osd.2 [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] boot 2024-05-27T07:34:17.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:16 smithi129 ceph-mon[35254]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T07:34:17.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:16 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:34:17.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:17 smithi136 ceph-mon[35324]: pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:17.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:17 smithi136 ceph-mon[35324]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T07:34:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:17 smithi090 ceph-mon[31429]: pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:17 smithi090 ceph-mon[31429]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T07:34:18.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:17 smithi129 ceph-mon[35254]: pgmap v106: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T07:34:18.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:17 smithi129 ceph-mon[35254]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T07:34:18.389 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T07:34:18.389 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T07:34:18.389 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-05-27T07:34:18.389 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-05-27T07:34:18.390 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0326932 s, 321 MB/s 2024-05-27T07:34:18.390 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-05-27T07:34:18.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:18 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:34:18.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:18 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:34:19.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:18 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:34:19.266 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-05-27T07:34:19.693 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:34:19.954 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:19 smithi129 ceph-mon[35254]: pgmap v109: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:19.955 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T07:34:19.955 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:19 smithi129 ceph-mon[35254]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T07:34:19.955 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:34:19.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:19 smithi090 ceph-mon[31429]: pgmap v109: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:19.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T07:34:19.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:19 smithi090 ceph-mon[31429]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T07:34:19.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:34:19.958 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 07:34:19 smithi090 sudo[50164]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:34:19.958 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 07:34:19 smithi090 sudo[50164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:19.958 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 07:34:19 smithi090 sudo[50164]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:19.959 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 07:34:19 smithi090 sudo[50164]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:19.959 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 07:34:19 smithi090 sudo[50167]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:34:19.959 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 07:34:19 smithi090 sudo[50167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:19.959 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 07:34:19 smithi090 sudo[50167]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:19.959 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 07:34:19 smithi090 sudo[50167]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:19.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:19 smithi136 ceph-mon[35324]: pgmap v109: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:19.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:19 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T07:34:19.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:19 smithi136 ceph-mon[35324]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T07:34:19.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:19 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T07:34:20.309 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 07:34:19 smithi129 sudo[43983]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:34:20.309 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 07:34:19 smithi129 sudo[43983]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:20.309 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 07:34:19 smithi129 sudo[43983]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:20.309 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 07:34:19 smithi129 sudo[43983]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:20.309 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 07:34:20 smithi129 sudo[43986]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:34:20.309 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 07:34:20 smithi129 sudo[43986]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:20.309 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 07:34:20 smithi129 sudo[43986]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:20.309 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 07:34:20 smithi129 sudo[43986]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:20.463 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44425]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:34:20.463 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44425]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:20.463 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44425]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:20.463 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44425]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:20.463 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44428]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:34:20.463 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44428]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:20.463 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44428]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:20.463 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44428]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:20.660 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 sudo[50191]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:34:20.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 sudo[50191]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:20.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 sudo[50191]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:20.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 sudo[50191]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:20.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 sudo[50215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:34:20.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 sudo[50215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:20.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 sudo[50215]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:20.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 sudo[50215]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:20.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44431]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44431]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44431]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44431]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44434]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44434]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44434]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 sudo[44434]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:34:20.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:34:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:34:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:34:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:34:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:34:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:34:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:34:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:34:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:34:20.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T07:34:20.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T07:34:20.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:34:20.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:34:20.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:34:20.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:34:20.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:34:20.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:34:20.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:34:20.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:34:20.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:34:20.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:34:21.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 sudo[44021]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T07:34:21.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 sudo[44021]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:21.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 sudo[44021]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 sudo[44021]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 sudo[44024]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 sudo[44024]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 sudo[44024]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 sudo[44024]: pam_unix(sudo:session): session closed for user root 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:34:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:34:21.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:34:21.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:34:21.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:34:21.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:34:21.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.901+0000 7f55da400700 1 -- 172.21.15.90:0/191197711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55d4110980 msgr2=0x7f55d4112da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:21.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.901+0000 7f55da400700 1 --2- 172.21.15.90:0/191197711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55d4110980 0x7f55d4112da0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00b0d0 tx=0x7f55cc00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:34:21.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.901+0000 7f55da400700 1 -- 172.21.15.90:0/191197711 shutdown_connections 2024-05-27T07:34:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.901+0000 7f55da400700 1 --2- 172.21.15.90:0/191197711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55d4110980 0x7f55d4112da0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.901+0000 7f55da400700 1 --2- 172.21.15.90:0/191197711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d40776d0 0x7f55d4075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.901+0000 7f55da400700 1 --2- 172.21.15.90:0/191197711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55d40760e0 0x7f55d4076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.901+0000 7f55da400700 1 -- 172.21.15.90:0/191197711 >> 172.21.15.90:0/191197711 conn(0x7f55d40ff9b0 msgr2=0x7f55d4101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.901+0000 7f55da400700 1 -- 172.21.15.90:0/191197711 shutdown_connections 2024-05-27T07:34:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.901+0000 7f55da400700 1 -- 172.21.15.90:0/191197711 wait complete. 2024-05-27T07:34:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.902+0000 7f55da400700 1 Processor -- start 2024-05-27T07:34:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.903+0000 7f55da400700 1 -- start start 2024-05-27T07:34:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.903+0000 7f55da400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d40760e0 0x7f55d41a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.903+0000 7f55da400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55d40776d0 0x7f55d41a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.903+0000 7f55da400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55d4110980 0x7f55d41a8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.903+0000 7f55da400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f55d4116250 con 0x7f55d40776d0 2024-05-27T07:34:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.903+0000 7f55da400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55d41160d0 con 0x7f55d4110980 2024-05-27T07:34:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.903+0000 7f55da400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55d41163d0 con 0x7f55d40760e0 2024-05-27T07:34:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.903+0000 7f55cbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55d40776d0 0x7f55d41a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.903+0000 7f55cbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55d40776d0 0x7f55d41a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51900/0 (socket says 172.21.15.90:51900) 2024-05-27T07:34:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.903+0000 7f55cbe00700 1 -- 172.21.15.90:0/3150475826 learned_addr learned my addr 172.21.15.90:0/3150475826 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:34:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55d3400700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d40760e0 0x7f55d41a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55d3e00700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55d4110980 0x7f55d41a8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55cbe00700 1 -- 172.21.15.90:0/3150475826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d40760e0 msgr2=0x7f55d41a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55cbe00700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d40760e0 0x7f55d41a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55cbe00700 1 -- 172.21.15.90:0/3150475826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55d4110980 msgr2=0x7f55d41a8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55cbe00700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55d4110980 0x7f55d41a8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55cbe00700 1 -- 172.21.15.90:0/3150475826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d41a88f0 con 0x7f55d40776d0 2024-05-27T07:34:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55cbe00700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55d40776d0 0x7f55d41a12d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00d950 tx=0x7f55bc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:21.899 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:21 smithi090 ceph-mon[31429]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:21.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:21 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi136 2024-05-27T07:34:21.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:21 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi136 to 17089M 2024-05-27T07:34:21.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:21 smithi090 ceph-mon[31429]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:34:21.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:21 smithi090 ceph-mon[31429]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:34:21.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:21 smithi090 ceph-mon[31429]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T07:34:21.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:21 smithi090 ceph-mon[31429]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T07:34:21.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55d0c00700 1 -- 172.21.15.90:0/3150475826 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc016070 con 0x7f55d40776d0 2024-05-27T07:34:21.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55d0c00700 1 -- 172.21.15.90:0/3150475826 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f55bc010460 con 0x7f55d40776d0 2024-05-27T07:34:21.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.904+0000 7f55d0c00700 1 -- 172.21.15.90:0/3150475826 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00f620 con 0x7f55d40776d0 2024-05-27T07:34:21.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.905+0000 7f55da400700 1 -- 172.21.15.90:0/3150475826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d41a8be0 con 0x7f55d40776d0 2024-05-27T07:34:21.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.905+0000 7f55da400700 1 -- 172.21.15.90:0/3150475826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d41a90d0 con 0x7f55d40776d0 2024-05-27T07:34:21.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.906+0000 7f55da400700 1 -- 172.21.15.90:0/3150475826 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d40512b0 con 0x7f55d40776d0 2024-05-27T07:34:21.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.908+0000 7f55d0c00700 1 -- 172.21.15.90:0/3150475826 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f55bc019070 con 0x7f55d40776d0 2024-05-27T07:34:21.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.910+0000 7f55d0c00700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55b80b2880 0x7f55b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:21.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.910+0000 7f55d0c00700 1 -- 172.21.15.90:0/3150475826 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 3237+0+0 (secure 0 0 0) 0x7f55bc0e0b20 con 0x7f55d40776d0 2024-05-27T07:34:21.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.910+0000 7f55d3400700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55b80b2880 0x7f55b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:21.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.911+0000 7f55d3400700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55b80b2880 0x7f55b80b4d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55c4005950 tx=0x7f55c40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:21.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:21.915+0000 7f55d0c00700 1 -- 172.21.15.90:0/3150475826 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f55bc0a9b70 con 0x7f55d40776d0 2024-05-27T07:34:21.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:21 smithi136 ceph-mon[35324]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:21.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:21 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi136 2024-05-27T07:34:21.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:21 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi136 to 17089M 2024-05-27T07:34:21.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:21 smithi136 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:34:21.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:21 smithi136 ceph-mon[35324]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:34:21.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:21 smithi136 ceph-mon[35324]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T07:34:21.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:21 smithi136 ceph-mon[35324]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T07:34:22.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:21 smithi129 ceph-mon[35254]: pgmap v112: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:22.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:21 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi136 2024-05-27T07:34:22.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:21 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi136 to 17089M 2024-05-27T07:34:22.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:21 smithi129 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T07:34:22.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:21 smithi129 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T07:34:22.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:21 smithi129 ceph-mon[35254]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T07:34:22.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:21 smithi129 ceph-mon[35254]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-05-27T07:34:22.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:22.157+0000 7f55da400700 1 -- 172.21.15.90:0/3150475826 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f55d4063bf0 con 0x7f55b80b2880 2024-05-27T07:34:22.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:34:22.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:34:22.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:22.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:34:22.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:34:22.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:23.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:34:23.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:34:23.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:23.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:23 smithi090 ceph-mon[31429]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:34:23.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:23 smithi090 ceph-mon[31429]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:23.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:23 smithi136 ceph-mon[35324]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:34:23.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:23 smithi136 ceph-mon[35324]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:24.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:23 smithi129 ceph-mon[35254]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:34:24.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:23 smithi129 ceph-mon[35254]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:25.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:25 smithi090 ceph-mon[31429]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:25.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:25 smithi136 ceph-mon[35324]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:26.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:25 smithi129 ceph-mon[35254]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:26.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:26 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2454077072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a92e6906-1cfb-4f68-9ab7-2bc35221478f"}]: dispatch 2024-05-27T07:34:26.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:26 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2454077072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a92e6906-1cfb-4f68-9ab7-2bc35221478f"}]': finished 2024-05-27T07:34:26.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:26 smithi090 ceph-mon[31429]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T07:34:26.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:26 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:26.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:26 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2454077072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a92e6906-1cfb-4f68-9ab7-2bc35221478f"}]: dispatch 2024-05-27T07:34:26.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:26 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2454077072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a92e6906-1cfb-4f68-9ab7-2bc35221478f"}]': finished 2024-05-27T07:34:26.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:26 smithi136 ceph-mon[35324]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T07:34:26.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:26 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:27.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:26 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2454077072' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a92e6906-1cfb-4f68-9ab7-2bc35221478f"}]: dispatch 2024-05-27T07:34:27.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:26 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2454077072' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a92e6906-1cfb-4f68-9ab7-2bc35221478f"}]': finished 2024-05-27T07:34:27.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:26 smithi129 ceph-mon[35254]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T07:34:27.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:26 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:27.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:27 smithi090 ceph-mon[31429]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:27.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:27 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3219202861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:34:27.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:27 smithi136 ceph-mon[35324]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:27.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:27 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3219202861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:34:28.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:27 smithi129 ceph-mon[35254]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:28.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:27 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3219202861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:34:29.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:29 smithi090 ceph-mon[31429]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:29.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:29 smithi136 ceph-mon[35324]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:30.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:29 smithi129 ceph-mon[35254]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:31.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:31 smithi090 ceph-mon[31429]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:31.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:31 smithi136 ceph-mon[35324]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:32.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:31 smithi129 ceph-mon[35254]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:33.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:33 smithi090 ceph-mon[31429]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:34.003 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:33 smithi136 ceph-mon[35324]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:34.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:33 smithi129 ceph-mon[35254]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:35.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:35 smithi136 ceph-mon[35324]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:36.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:35 smithi129 ceph-mon[35254]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:36.207 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:35 smithi090 ceph-mon[31429]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:36.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:36 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T07:34:36.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:36 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:37.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T07:34:37.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:37.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:36 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T07:34:37.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:36 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:37.916 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:37 smithi090 ceph-mon[31429]: Deploying daemon osd.3 on smithi090 2024-05-27T07:34:37.916 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:37 smithi090 ceph-mon[31429]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:37.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:37 smithi136 ceph-mon[35324]: Deploying daemon osd.3 on smithi090 2024-05-27T07:34:37.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:37 smithi136 ceph-mon[35324]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:38.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:37 smithi129 ceph-mon[35254]: Deploying daemon osd.3 on smithi090 2024-05-27T07:34:38.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:37 smithi129 ceph-mon[35254]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:40.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:39 smithi129 ceph-mon[35254]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:40.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:39 smithi090 ceph-mon[31429]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:40.244 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:39 smithi136 ceph-mon[35324]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:41.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:41 smithi090 ceph-mon[31429]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:42.046 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:41 smithi136 ceph-mon[35324]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:42.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:41 smithi129 ceph-mon[35254]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:43.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:43 smithi136 ceph-mon[35324]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:44.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:43 smithi129 ceph-mon[35254]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:44.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:43 smithi090 ceph-mon[31429]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:46.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:45 smithi090 ceph-mon[31429]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:46.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:46.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:46.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:46.244 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:45 smithi136 ceph-mon[35324]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:46.244 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:46.244 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:46.244 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:46.308 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:45 smithi129 ceph-mon[35254]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:46.308 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:46.308 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:46.308 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:48.152 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:47 smithi090 ceph-mon[31429]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:48.244 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:47 smithi136 ceph-mon[35324]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:48.308 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:47 smithi129 ceph-mon[35254]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:49.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:49 smithi129 ceph-mon[35254]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:49.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:49 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:49.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:49 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:49.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:49 smithi129 ceph-mon[35254]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T07:34:49.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:49 smithi090 ceph-mon[31429]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:49.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:49 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:49.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:49 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:49.913 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:49 smithi090 ceph-mon[31429]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T07:34:49.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:49 smithi136 ceph-mon[35324]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:49.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:49 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:49.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:49 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:49.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:49 smithi136 ceph-mon[35324]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T07:34:50.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.412+0000 7f55d0c00700 1 -- 172.21.15.90:0/3150475826 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f55d4063bf0 con 0x7f55b80b2880 2024-05-27T07:34:50.409 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 3 on host 'smithi090' 2024-05-27T07:34:50.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.416+0000 7f55da400700 1 -- 172.21.15.90:0/3150475826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55b80b2880 msgr2=0x7f55b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.416+0000 7f55da400700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55b80b2880 0x7f55b80b4d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55c4005950 tx=0x7f55c40058e0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.416+0000 7f55da400700 1 -- 172.21.15.90:0/3150475826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55d40776d0 msgr2=0x7f55d41a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.416+0000 7f55da400700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55d40776d0 0x7f55d41a12d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00d950 tx=0x7f55bc00dd10 comp rx=0 tx=0).stop 2024-05-27T07:34:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.416+0000 7f55da400700 1 -- 172.21.15.90:0/3150475826 shutdown_connections 2024-05-27T07:34:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.416+0000 7f55da400700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55b80b2880 0x7f55b80b4d90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.417+0000 7f55da400700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55d4110980 0x7f55d41a8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.417+0000 7f55da400700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d40760e0 0x7f55d41a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.417+0000 7f55da400700 1 --2- 172.21.15.90:0/3150475826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55d40776d0 0x7f55d41a12d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.417+0000 7f55da400700 1 -- 172.21.15.90:0/3150475826 >> 172.21.15.90:0/3150475826 conn(0x7f55d40ff9b0 msgr2=0x7f55d4103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.417+0000 7f55da400700 1 -- 172.21.15.90:0/3150475826 shutdown_connections 2024-05-27T07:34:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:34:50.417+0000 7f55da400700 1 -- 172.21.15.90:0/3150475826 wait complete. 2024-05-27T07:34:50.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:50.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:50.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:50.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:50 smithi090 ceph-mon[31429]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T07:34:50.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:50 smithi090 ceph-mon[31429]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T07:34:50.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:50 smithi090 ceph-mon[31429]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:34:50.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:50.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:50.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:50.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:50.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:50.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:50.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:50.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:50 smithi129 ceph-mon[35254]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T07:34:50.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:50 smithi129 ceph-mon[35254]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T07:34:50.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:50 smithi129 ceph-mon[35254]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:34:50.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:50.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:50.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:50.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:50.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:50.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:50.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:50.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:50 smithi136 ceph-mon[35324]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T07:34:50.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:50 smithi136 ceph-mon[35324]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T07:34:50.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:50 smithi136 ceph-mon[35324]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:34:50.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:50.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:50.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:50.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:34:51.184 DEBUG:teuthology.orchestra.run.smithi090:osd.3> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.3.service 2024-05-27T07:34:51.188 INFO:tasks.cephadm:Deploying osd.4 on smithi129 with /dev/vg_nvme/lv_3... 2024-05-27T07:34:51.188 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T07:34:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:34:51.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:51 smithi090 ceph-mon[31429]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:51.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:51 smithi090 ceph-mon[31429]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:34:51.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:51 smithi090 ceph-mon[31429]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T07:34:51.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:51.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:51.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:51 smithi090 ceph-mon[31429]: osdmap e26: 4 total, 3 up, 4 in 2024-05-27T07:34:51.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:51 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:51.799 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:51 smithi129 ceph-mon[35254]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:51.800 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:51 smithi129 ceph-mon[35254]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:34:51.800 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:51 smithi129 ceph-mon[35254]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T07:34:51.800 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:51 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:51.800 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:51 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:51.800 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:51 smithi129 ceph-mon[35254]: osdmap e26: 4 total, 3 up, 4 in 2024-05-27T07:34:51.800 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:51 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:51.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:51 smithi136 ceph-mon[35324]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:51.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:51 smithi136 ceph-mon[35324]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:34:51.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:51 smithi136 ceph-mon[35324]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T07:34:51.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:51 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:51.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:51 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:51.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:51 smithi136 ceph-mon[35324]: osdmap e26: 4 total, 3 up, 4 in 2024-05-27T07:34:51.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:51 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:52.208 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 07:34:51 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3[52519]: 2024-05-27T07:34:51.740+0000 7f667ea00700 -1 osd.3 0 waiting for initial osdmap 2024-05-27T07:34:52.208 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 07:34:51 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3[52519]: 2024-05-27T07:34:51.759+0000 7f6674800700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T07:34:52.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:52 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:34:52.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:52 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:34:52.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:52 smithi129 ceph-mon[35254]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' 2024-05-27T07:34:52.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:52.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:52 smithi129 ceph-mon[35254]: osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] boot 2024-05-27T07:34:52.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:52 smithi129 ceph-mon[35254]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T07:34:52.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:52.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:52 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:34:52.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:52 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:34:52.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:52 smithi090 ceph-mon[31429]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' 2024-05-27T07:34:52.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:52.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:52 smithi090 ceph-mon[31429]: osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] boot 2024-05-27T07:34:52.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:52 smithi090 ceph-mon[31429]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T07:34:52.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:52.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:52 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:34:52.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:52 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:34:52.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:52 smithi136 ceph-mon[35324]: from='osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486]' entity='osd.3' 2024-05-27T07:34:52.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:52.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:52 smithi136 ceph-mon[35324]: osd.3 [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] boot 2024-05-27T07:34:52.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:52 smithi136 ceph-mon[35324]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T07:34:52.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:34:53.899 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:53 smithi129 ceph-mon[35254]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:53.900 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:53 smithi129 ceph-mon[35254]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T07:34:54.079 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:53 smithi136 ceph-mon[35324]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:54.079 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:53 smithi136 ceph-mon[35324]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T07:34:54.207 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:53 smithi090 ceph-mon[31429]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-27T07:34:54.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:53 smithi090 ceph-mon[31429]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T07:34:54.692 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T07:34:54.692 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T07:34:54.692 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-05-27T07:34:54.692 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-05-27T07:34:54.692 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0368752 s, 284 MB/s 2024-05-27T07:34:54.692 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-05-27T07:34:55.528 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_3 2024-05-27T07:34:55.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:55 smithi090 ceph-mon[31429]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:34:55.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:55 smithi090 ceph-mon[31429]: osdmap e29: 4 total, 4 up, 4 in 2024-05-27T07:34:55.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:55 smithi129 ceph-mon[35254]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:34:55.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:55 smithi129 ceph-mon[35254]: osdmap e29: 4 total, 4 up, 4 in 2024-05-27T07:34:55.953 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:34:55.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:55 smithi136 ceph-mon[35324]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:34:55.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:55 smithi136 ceph-mon[35324]: osdmap e29: 4 total, 4 up, 4 in 2024-05-27T07:34:57.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:34:57.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi090 2024-05-27T07:34:57.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:34:57.614 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:34:57.614 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi090 to 8544M 2024-05-27T07:34:57.614 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.614 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:57.614 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:57.614 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:34:57.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi090 2024-05-27T07:34:57.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:34:57.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:34:57.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi090 to 8544M 2024-05-27T07:34:57.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:57.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:57.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:34:57.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi090 2024-05-27T07:34:57.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:34:57.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:34:57.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi090 to 8544M 2024-05-27T07:34:57.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:57.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:57.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:34:57.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:34:58.182 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.188+0000 7f8133000700 1 -- 172.21.15.129:0/2448907988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f812c107670 msgr2=0x7f812c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.188+0000 7f8133000700 1 --2- 172.21.15.129:0/2448907988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f812c107670 0x7f812c107a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8118009990 tx=0x7f8118009ca0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.188+0000 7f8133000700 1 -- 172.21.15.129:0/2448907988 shutdown_connections 2024-05-27T07:34:58.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.188+0000 7f8133000700 1 --2- 172.21.15.129:0/2448907988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f812c108010 0x7f812c10e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.188+0000 7f8133000700 1 --2- 172.21.15.129:0/2448907988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f812c106540 0x7f812c106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.188+0000 7f8133000700 1 --2- 172.21.15.129:0/2448907988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f812c107670 0x7f812c107a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.188+0000 7f8133000700 1 -- 172.21.15.129:0/2448907988 >> 172.21.15.129:0/2448907988 conn(0x7f812c101b80 msgr2=0x7f812c104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:34:58.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.188+0000 7f8133000700 1 -- 172.21.15.129:0/2448907988 shutdown_connections 2024-05-27T07:34:58.183 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.188+0000 7f8133000700 1 -- 172.21.15.129:0/2448907988 wait complete. 2024-05-27T07:34:58.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.189+0000 7f8133000700 1 Processor -- start 2024-05-27T07:34:58.184 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.189+0000 7f8133000700 1 -- start start 2024-05-27T07:34:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f8133000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f812c106540 0x7f812c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f8133000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f812c107670 0x7f812c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f8133000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f812c108010 0x7f812c1a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f8133000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f812c111eb0 con 0x7f812c108010 2024-05-27T07:34:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f8133000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f812c111d30 con 0x7f812c106540 2024-05-27T07:34:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f8133000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f812c112030 con 0x7f812c107670 2024-05-27T07:34:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f812be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f812c106540 0x7f812c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f812be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f812c106540 0x7f812c19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52672/0 (socket says 172.21.15.129:52672) 2024-05-27T07:34:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f812be00700 1 -- 172.21.15.129:0/2119721027 learned_addr learned my addr 172.21.15.129:0/2119721027 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-05-27T07:34:58.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f812b400700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f812c107670 0x7f812c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.190+0000 7f8131000700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f812c108010 0x7f812c1a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f812be00700 1 -- 172.21.15.129:0/2119721027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f812c107670 msgr2=0x7f812c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f812be00700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f812c107670 0x7f812c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f812be00700 1 -- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f812c108010 msgr2=0x7f812c1a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f812be00700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f812c108010 0x7f812c1a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f812be00700 1 -- 172.21.15.129:0/2119721027 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c1a45c0 con 0x7f812c106540 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f8131000700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f812c108010 0x7f812c1a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f812be00700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f812c106540 0x7f812c19cab0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f811c00c970 tx=0x7f811c00cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f8128c00700 1 -- 172.21.15.129:0/2119721027 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c004220 con 0x7f812c106540 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f8128c00700 1 -- 172.21.15.129:0/2119721027 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f811c007d90 con 0x7f812c106540 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f8128c00700 1 -- 172.21.15.129:0/2119721027 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811c003c80 con 0x7f812c106540 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f8133000700 1 -- 172.21.15.129:0/2119721027 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812c1a48b0 con 0x7f812c106540 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.191+0000 7f8133000700 1 -- 172.21.15.129:0/2119721027 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812c1a4df0 con 0x7f812c106540 2024-05-27T07:34:58.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.193+0000 7f8133000700 1 -- 172.21.15.129:0/2119721027 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f812c0512b0 con 0x7f812c106540 2024-05-27T07:34:58.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.195+0000 7f8128c00700 1 -- 172.21.15.129:0/2119721027 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f811c0043c0 con 0x7f812c106540 2024-05-27T07:34:58.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.197+0000 7f8128c00700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81100b2900 0x7f81100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:34:58.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.197+0000 7f8128c00700 1 -- 172.21.15.129:0/2119721027 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f811c0e1010 con 0x7f812c106540 2024-05-27T07:34:58.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.198+0000 7f812b400700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81100b2900 0x7f81100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:34:58.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.199+0000 7f812b400700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81100b2900 0x7f81100b4e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8118006010 tx=0x7f811801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:34:58.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.200+0000 7f8128c00700 1 -- 172.21.15.129:0/2119721027 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f811c014030 con 0x7f812c106540 2024-05-27T07:34:58.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:34:58.439+0000 7f8133000700 1 -- 172.21.15.129:0/2119721027 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f812c063bf0 con 0x7f81100b2900 2024-05-27T07:34:59.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:59 smithi129 ceph-mon[35254]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:34:59.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:59 smithi129 ceph-mon[35254]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:34:59.612 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:34:59.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:34:59.613 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:34:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:59.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:59 smithi090 ceph-mon[31429]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:34:59.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:59 smithi090 ceph-mon[31429]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:34:59.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:34:59.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:34:59.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:34:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:34:59.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:59 smithi136 ceph-mon[35324]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:34:59.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:59 smithi136 ceph-mon[35324]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:34:59.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:34:59.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:34:59.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:34:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:01.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:01 smithi090 ceph-mon[31429]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-27T07:35:01.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:01 smithi136 ceph-mon[35324]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-27T07:35:01.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:01 smithi129 ceph-mon[35254]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-27T07:35:03.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:03 smithi129 ceph-mon[35254]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-27T07:35:03.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:03 smithi129 ceph-mon[35254]: from='client.? 172.21.15.129:0/3432505304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a730ea3e-47fa-46e2-b3af-dba7d3ac49c4"}]: dispatch 2024-05-27T07:35:03.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:03 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a730ea3e-47fa-46e2-b3af-dba7d3ac49c4"}]: dispatch 2024-05-27T07:35:03.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:03 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a730ea3e-47fa-46e2-b3af-dba7d3ac49c4"}]': finished 2024-05-27T07:35:03.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:03 smithi129 ceph-mon[35254]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T07:35:03.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:03.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:03 smithi090 ceph-mon[31429]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-27T07:35:03.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:03 smithi090 ceph-mon[31429]: from='client.? 172.21.15.129:0/3432505304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a730ea3e-47fa-46e2-b3af-dba7d3ac49c4"}]: dispatch 2024-05-27T07:35:03.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:03 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a730ea3e-47fa-46e2-b3af-dba7d3ac49c4"}]: dispatch 2024-05-27T07:35:03.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:03 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a730ea3e-47fa-46e2-b3af-dba7d3ac49c4"}]': finished 2024-05-27T07:35:03.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:03 smithi090 ceph-mon[31429]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T07:35:03.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:03.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:03 smithi136 ceph-mon[35324]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-27T07:35:03.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:03 smithi136 ceph-mon[35324]: from='client.? 172.21.15.129:0/3432505304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a730ea3e-47fa-46e2-b3af-dba7d3ac49c4"}]: dispatch 2024-05-27T07:35:03.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:03 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a730ea3e-47fa-46e2-b3af-dba7d3ac49c4"}]: dispatch 2024-05-27T07:35:03.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:03 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a730ea3e-47fa-46e2-b3af-dba7d3ac49c4"}]': finished 2024-05-27T07:35:03.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:03 smithi136 ceph-mon[35324]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T07:35:03.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:04.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:04 smithi090 ceph-mon[31429]: from='client.? 172.21.15.129:0/1763432506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:35:04.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:04 smithi136 ceph-mon[35324]: from='client.? 172.21.15.129:0/1763432506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:35:04.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:04 smithi129 ceph-mon[35254]: from='client.? 172.21.15.129:0/1763432506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:35:05.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:05 smithi090 ceph-mon[31429]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T07:35:05.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:05 smithi136 ceph-mon[35324]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T07:35:05.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:05 smithi129 ceph-mon[35254]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-27T07:35:07.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:07 smithi090 ceph-mon[31429]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:35:07.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:07 smithi136 ceph-mon[35324]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:35:07.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:07 smithi129 ceph-mon[35254]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:35:09.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:09 smithi129 ceph-mon[35254]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:09.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:09 smithi090 ceph-mon[31429]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:09.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:09 smithi136 ceph-mon[35324]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:11.558 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:11 smithi129 ceph-mon[35254]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:11.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:11 smithi090 ceph-mon[31429]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:11.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:11 smithi136 ceph-mon[35324]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:12.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T07:35:12.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:12.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T07:35:12.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:12.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T07:35:12.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:13.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:13 smithi090 ceph-mon[31429]: Deploying daemon osd.4 on smithi129 2024-05-27T07:35:13.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:13 smithi090 ceph-mon[31429]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:13.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:13 smithi136 ceph-mon[35324]: Deploying daemon osd.4 on smithi129 2024-05-27T07:35:13.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:13 smithi136 ceph-mon[35324]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:13.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:13 smithi129 ceph-mon[35254]: Deploying daemon osd.4 on smithi129 2024-05-27T07:35:13.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:13 smithi129 ceph-mon[35254]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:15.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:15 smithi090 ceph-mon[31429]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:15.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:15 smithi136 ceph-mon[35324]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:15.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:15 smithi129 ceph-mon[35254]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:17.681 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:17 smithi129 ceph-mon[35254]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:17.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:17 smithi090 ceph-mon[31429]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:17.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:17 smithi136 ceph-mon[35324]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:19.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:19 smithi090 ceph-mon[31429]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:19.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:19 smithi136 ceph-mon[35324]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:19.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:19 smithi129 ceph-mon[35254]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:21.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:21 smithi090 ceph-mon[31429]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:21.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:21 smithi136 ceph-mon[35324]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:21.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:21 smithi129 ceph-mon[35254]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:22.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:22.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:22.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:22 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:22.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:22.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:22.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:22 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:22.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:22.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:22.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:22 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:23.665 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:23 smithi129 ceph-mon[35254]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:23.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:23 smithi090 ceph-mon[31429]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:23.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:23 smithi136 ceph-mon[35324]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:25.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:25 smithi090 ceph-mon[31429]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:25.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:25.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:25.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:25.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:35:25.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:25.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:25 smithi136 ceph-mon[35324]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:25.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:25.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:25.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:25.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:35:25.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:26.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:25 smithi129 ceph-mon[35254]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:26.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:26.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:26.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:26.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:35:26.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:26.808 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:26 smithi129 ceph-mon[35254]: from='osd.4 [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:35:26.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:26 smithi129 ceph-mon[35254]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:35:26.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:26 smithi090 ceph-mon[31429]: from='osd.4 [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:35:26.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:26 smithi090 ceph-mon[31429]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:35:26.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:26 smithi136 ceph-mon[35324]: from='osd.4 [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:35:26.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:26 smithi136 ceph-mon[35324]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T07:35:27.401 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.405+0000 7f8128c00700 1 -- 172.21.15.129:0/2119721027 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f812c063bf0 con 0x7f81100b2900 2024-05-27T07:35:27.401 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 4 on host 'smithi129' 2024-05-27T07:35:27.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.409+0000 7f8133000700 1 -- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81100b2900 msgr2=0x7f81100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.409+0000 7f8133000700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81100b2900 0x7f81100b4e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8118006010 tx=0x7f811801a040 comp rx=0 tx=0).stop 2024-05-27T07:35:27.405 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.409+0000 7f8133000700 1 -- 172.21.15.129:0/2119721027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f812c106540 msgr2=0x7f812c19cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:27.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.409+0000 7f8133000700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f812c106540 0x7f812c19cab0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f811c00c970 tx=0x7f811c00cd30 comp rx=0 tx=0).stop 2024-05-27T07:35:27.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.409+0000 7f8133000700 1 -- 172.21.15.129:0/2119721027 shutdown_connections 2024-05-27T07:35:27.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.409+0000 7f8133000700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81100b2900 0x7f81100b4e10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.409+0000 7f8133000700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f812c108010 0x7f812c1a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.409+0000 7f8133000700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f812c106540 0x7f812c19cab0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.409+0000 7f8133000700 1 --2- 172.21.15.129:0/2119721027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f812c107670 0x7f812c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:27.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.409+0000 7f8133000700 1 -- 172.21.15.129:0/2119721027 >> 172.21.15.129:0/2119721027 conn(0x7f812c101b80 msgr2=0x7f812c10a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:27.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.410+0000 7f8133000700 1 -- 172.21.15.129:0/2119721027 shutdown_connections 2024-05-27T07:35:27.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:35:27.410+0000 7f8133000700 1 -- 172.21.15.129:0/2119721027 wait complete. 2024-05-27T07:35:27.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:27 smithi090 ceph-mon[31429]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:27.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:27 smithi090 ceph-mon[31429]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T07:35:27.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:27 smithi090 ceph-mon[31429]: from='osd.4 [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:35:27.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:27 smithi090 ceph-mon[31429]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T07:35:27.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:27.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:27 smithi090 ceph-mon[31429]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:35:27.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:27.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:27.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:27.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:27 smithi136 ceph-mon[35324]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:27.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:27 smithi136 ceph-mon[35324]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T07:35:27.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:27 smithi136 ceph-mon[35324]: from='osd.4 [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:35:27.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:27 smithi136 ceph-mon[35324]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T07:35:27.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:27.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:27 smithi136 ceph-mon[35324]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:35:27.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:27.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:27.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:28.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:27 smithi129 ceph-mon[35254]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:28.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:27 smithi129 ceph-mon[35254]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T07:35:28.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:27 smithi129 ceph-mon[35254]: from='osd.4 [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:35:28.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:27 smithi129 ceph-mon[35254]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T07:35:28.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:28.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:27 smithi129 ceph-mon[35254]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:35:28.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:28.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:28.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:28.203 DEBUG:teuthology.orchestra.run.smithi129:osd.4> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.4.service 2024-05-27T07:35:28.207 INFO:tasks.cephadm:Deploying osd.5 on smithi136 with /dev/vg_nvme/lv_3... 2024-05-27T07:35:28.208 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T07:35:28.636 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:35:28.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:28 smithi129 ceph-mon[35254]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:35:28.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:28 smithi129 ceph-mon[35254]: osdmap e32: 5 total, 4 up, 5 in 2024-05-27T07:35:28.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:28 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:28.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:28 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:28.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:28 smithi090 ceph-mon[31429]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:35:28.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:28 smithi090 ceph-mon[31429]: osdmap e32: 5 total, 4 up, 5 in 2024-05-27T07:35:28.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:28 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:28.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:28 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:28.968 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:28 smithi136 ceph-mon[35324]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:35:28.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:28 smithi136 ceph-mon[35324]: osdmap e32: 5 total, 4 up, 5 in 2024-05-27T07:35:28.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:28 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:28.969 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:28 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:29.793 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:29 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:35:29.794 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:29 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:35:29.794 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:29 smithi129 ceph-mon[35254]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:29.794 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:29.794 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:29 smithi129 ceph-mon[35254]: osd.4 [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] boot 2024-05-27T07:35:29.794 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:29 smithi129 ceph-mon[35254]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T07:35:29.794 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:29.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:29 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:35:29.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:29 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:35:29.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:29 smithi136 ceph-mon[35324]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:29.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:29.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:29 smithi136 ceph-mon[35324]: osd.4 [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] boot 2024-05-27T07:35:29.837 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:29 smithi136 ceph-mon[35324]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T07:35:29.837 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:29.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:29 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:35:29.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:29 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:35:29.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:29 smithi090 ceph-mon[31429]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-27T07:35:29.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:29.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:29 smithi090 ceph-mon[31429]: osd.4 [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] boot 2024-05-27T07:35:29.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:29 smithi090 ceph-mon[31429]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T07:35:29.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:35:30.744 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:30 smithi136 ceph-mon[35324]: osdmap e34: 5 total, 5 up, 5 in 2024-05-27T07:35:30.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:30 smithi129 ceph-mon[35254]: osdmap e34: 5 total, 5 up, 5 in 2024-05-27T07:35:30.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:30 smithi090 ceph-mon[31429]: osdmap e34: 5 total, 5 up, 5 in 2024-05-27T07:35:31.384 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T07:35:31.384 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T07:35:31.385 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-05-27T07:35:31.385 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-05-27T07:35:31.385 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.037516 s, 280 MB/s 2024-05-27T07:35:31.385 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-05-27T07:35:31.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:31 smithi090 ceph-mon[31429]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:31.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:31 smithi136 ceph-mon[35324]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:32.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:31 smithi129 ceph-mon[35254]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:32.219 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_3 2024-05-27T07:35:32.649 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:35:33.801 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:33 smithi136 ceph-mon[35324]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:33.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:33.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:33.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:35:33.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:35:33.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:33.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:33.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:35:33.802 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:33 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:33.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:33 smithi090 ceph-mon[31429]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:33.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:33.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:33.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:35:33.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:35:33.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:33.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:33.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:35:33.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:33 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:34.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:33 smithi129 ceph-mon[35254]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:34.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:34.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:34.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:35:34.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:35:34.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:34.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:34.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:35:34.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:33 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:34.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.732+0000 7fe411400700 1 -- 172.21.15.136:0/3473077765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe40c0760e0 msgr2=0x7fe40c076570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:34.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.732+0000 7fe411400700 1 --2- 172.21.15.136:0/3473077765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe40c0760e0 0x7fe40c076570 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4009990 tx=0x7fe3f4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:35:34.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.733+0000 7fe411400700 1 -- 172.21.15.136:0/3473077765 shutdown_connections 2024-05-27T07:35:34.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.733+0000 7fe411400700 1 --2- 172.21.15.136:0/3473077765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe40c10bff0 0x7fe40c10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:34.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.733+0000 7fe411400700 1 --2- 172.21.15.136:0/3473077765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe40c0776d0 0x7fe40c075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:34.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.733+0000 7fe411400700 1 --2- 172.21.15.136:0/3473077765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe40c0760e0 0x7fe40c076570 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:34.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.733+0000 7fe411400700 1 -- 172.21.15.136:0/3473077765 >> 172.21.15.136:0/3473077765 conn(0x7fe40c0ff7a0 msgr2=0x7fe40c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:35:34.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.733+0000 7fe411400700 1 -- 172.21.15.136:0/3473077765 shutdown_connections 2024-05-27T07:35:34.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.733+0000 7fe411400700 1 -- 172.21.15.136:0/3473077765 wait complete. 2024-05-27T07:35:34.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.734+0000 7fe411400700 1 Processor -- start 2024-05-27T07:35:34.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.734+0000 7fe411400700 1 -- start start 2024-05-27T07:35:34.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.734+0000 7fe411400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe40c0760e0 0x7fe40c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:34.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.734+0000 7fe411400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe40c0776d0 0x7fe40c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:34.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe411400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe40c10bff0 0x7fe40c1a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:34.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe411400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c111c50 con 0x7fe40c0776d0 2024-05-27T07:35:34.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe411400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c111ad0 con 0x7fe40c10bff0 2024-05-27T07:35:34.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe411400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c111dd0 con 0x7fe40c0760e0 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe40be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe40c0760e0 0x7fe40c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe40be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe40c0760e0 0x7fe40c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:60454/0 (socket says 172.21.15.136:60454) 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe40be00700 1 -- 172.21.15.136:0/257741531 learned_addr learned my addr 172.21.15.136:0/257741531 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe410a00700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe40c10bff0 0x7fe40c1a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe403e00700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe40c0776d0 0x7fe40c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe40be00700 1 -- 172.21.15.136:0/257741531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe40c10bff0 msgr2=0x7fe40c1a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe40be00700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe40c10bff0 0x7fe40c1a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe40be00700 1 -- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe40c0776d0 msgr2=0x7fe40c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe40be00700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe40c0776d0 0x7fe40c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe40be00700 1 -- 172.21.15.136:0/257741531 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c1a42f0 con 0x7fe40c0760e0 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe410a00700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe40c10bff0 0x7fe40c1a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:34.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.735+0000 7fe403e00700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe40c0776d0 0x7fe40c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:35:34.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.736+0000 7fe40be00700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe40c0760e0 0x7fe40c19c820 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00c9e0 tx=0x7fe3fc00cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:34.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.736+0000 7fe409600700 1 -- 172.21.15.136:0/257741531 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc007aa0 con 0x7fe40c0760e0 2024-05-27T07:35:34.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.736+0000 7fe409600700 1 -- 172.21.15.136:0/257741531 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3fc007c40 con 0x7fe40c0760e0 2024-05-27T07:35:34.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.736+0000 7fe409600700 1 -- 172.21.15.136:0/257741531 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc018660 con 0x7fe40c0760e0 2024-05-27T07:35:34.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.736+0000 7fe411400700 1 -- 172.21.15.136:0/257741531 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40c1a45e0 con 0x7fe40c0760e0 2024-05-27T07:35:34.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.736+0000 7fe411400700 1 -- 172.21.15.136:0/257741531 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40c1a4ad0 con 0x7fe40c0760e0 2024-05-27T07:35:34.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.737+0000 7fe411400700 1 -- 172.21.15.136:0/257741531 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe40c0512b0 con 0x7fe40c0760e0 2024-05-27T07:35:34.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.740+0000 7fe409600700 1 -- 172.21.15.136:0/257741531 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe3fc018800 con 0x7fe40c0760e0 2024-05-27T07:35:34.735 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:34 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi129 2024-05-27T07:35:34.736 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:34 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi129 to 8544M 2024-05-27T07:35:34.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.741+0000 7fe409600700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe3f00b2980 0x7fe3f00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:35:34.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.742+0000 7fe409600700 1 -- 172.21.15.136:0/257741531 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fe3fc0e12a0 con 0x7fe40c0760e0 2024-05-27T07:35:34.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.742+0000 7fe403e00700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe3f00b2980 0x7fe3f00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:35:34.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.743+0000 7fe403e00700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe3f00b2980 0x7fe3f00b4e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4000c00 tx=0x7fe3f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:35:34.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.748+0000 7fe409600700 1 -- 172.21.15.136:0/257741531 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3fc054df0 con 0x7fe40c0760e0 2024-05-27T07:35:34.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:34 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi129 2024-05-27T07:35:34.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:34 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi129 to 8544M 2024-05-27T07:35:34.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:35:34.992+0000 7fe411400700 1 -- 172.21.15.136:0/257741531 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe40c063bf0 con 0x7fe3f00b2980 2024-05-27T07:35:35.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:34 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi129 2024-05-27T07:35:35.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:34 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi129 to 8544M 2024-05-27T07:35:35.947 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:35 smithi136 ceph-mon[35324]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:35.948 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:35 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:35:35.948 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:35 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:35:35.948 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:35 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:35.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:35 smithi090 ceph-mon[31429]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:35.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:35 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:35:35.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:35 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:35:35.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:35 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:36.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:35 smithi129 ceph-mon[35254]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:36.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:35 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:35:36.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:35 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:35:36.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:35 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:36.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:36 smithi090 ceph-mon[31429]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:35:36.977 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:36 smithi136 ceph-mon[35324]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:35:37.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:36 smithi129 ceph-mon[35254]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:35:37.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:37 smithi090 ceph-mon[31429]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:37.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:37 smithi136 ceph-mon[35324]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:38.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:37 smithi129 ceph-mon[35254]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:39.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:39 smithi090 ceph-mon[31429]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:39.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:39 smithi090 ceph-mon[31429]: from='client.? 172.21.15.136:0/2992601745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d6ab904-e641-4ebf-98f8-39396f44f205"}]: dispatch 2024-05-27T07:35:39.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:39 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d6ab904-e641-4ebf-98f8-39396f44f205"}]: dispatch 2024-05-27T07:35:39.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:39 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d6ab904-e641-4ebf-98f8-39396f44f205"}]': finished 2024-05-27T07:35:39.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:39 smithi090 ceph-mon[31429]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T07:35:39.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:35:39.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:39 smithi136 ceph-mon[35324]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:39.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:39 smithi136 ceph-mon[35324]: from='client.? 172.21.15.136:0/2992601745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d6ab904-e641-4ebf-98f8-39396f44f205"}]: dispatch 2024-05-27T07:35:39.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:39 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d6ab904-e641-4ebf-98f8-39396f44f205"}]: dispatch 2024-05-27T07:35:39.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:39 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d6ab904-e641-4ebf-98f8-39396f44f205"}]': finished 2024-05-27T07:35:39.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:39 smithi136 ceph-mon[35324]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T07:35:39.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:35:40.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:39 smithi129 ceph-mon[35254]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:40.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:39 smithi129 ceph-mon[35254]: from='client.? 172.21.15.136:0/2992601745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d6ab904-e641-4ebf-98f8-39396f44f205"}]: dispatch 2024-05-27T07:35:40.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:39 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d6ab904-e641-4ebf-98f8-39396f44f205"}]: dispatch 2024-05-27T07:35:40.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:39 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d6ab904-e641-4ebf-98f8-39396f44f205"}]': finished 2024-05-27T07:35:40.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:39 smithi129 ceph-mon[35254]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T07:35:40.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:35:40.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:40 smithi136 ceph-mon[35324]: from='client.? 172.21.15.136:0/1695778325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:35:41.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:40 smithi129 ceph-mon[35254]: from='client.? 172.21.15.136:0/1695778325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:35:41.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:40 smithi090 ceph-mon[31429]: from='client.? 172.21.15.136:0/1695778325' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:35:41.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:41 smithi136 ceph-mon[35324]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:42.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:41 smithi129 ceph-mon[35254]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:42.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:41 smithi090 ceph-mon[31429]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:43.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:43 smithi136 ceph-mon[35324]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:44.058 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:43 smithi129 ceph-mon[35254]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:44.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:43 smithi090 ceph-mon[31429]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:45.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:45 smithi136 ceph-mon[35324]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:46.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:45 smithi129 ceph-mon[35254]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:46.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:45 smithi090 ceph-mon[31429]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:47.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:47 smithi136 ceph-mon[35324]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:48.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:47 smithi129 ceph-mon[35254]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:48.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:47 smithi090 ceph-mon[31429]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:48.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:48 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T07:35:48.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:48 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:49.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:48 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T07:35:49.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:48 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:49.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:48 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T07:35:49.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:48 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:35:49.908 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:49 smithi136 ceph-mon[35324]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:49.908 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:49 smithi136 ceph-mon[35324]: Deploying daemon osd.5 on smithi136 2024-05-27T07:35:50.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:49 smithi129 ceph-mon[35254]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:50.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:49 smithi129 ceph-mon[35254]: Deploying daemon osd.5 on smithi136 2024-05-27T07:35:50.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:49 smithi090 ceph-mon[31429]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:50.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:49 smithi090 ceph-mon[31429]: Deploying daemon osd.5 on smithi136 2024-05-27T07:35:51.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:51 smithi136 ceph-mon[35324]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:52.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:51 smithi129 ceph-mon[35254]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:52.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:51 smithi090 ceph-mon[31429]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:54.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:53 smithi090 ceph-mon[31429]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:54.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:53 smithi129 ceph-mon[35254]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:54.083 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:53 smithi136 ceph-mon[35324]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:56.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:55 smithi129 ceph-mon[35254]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:56.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:55 smithi090 ceph-mon[31429]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:56.244 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:55 smithi136 ceph-mon[35324]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:58.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:57 smithi129 ceph-mon[35254]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:58.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:58.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:58.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:58.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:57 smithi090 ceph-mon[31429]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:58.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:58.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:58.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:35:58.244 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:57 smithi136 ceph-mon[35324]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:35:58.244 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:58.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:35:58.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:36:00.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:35:59 smithi129 ceph-mon[35254]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:00.157 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:35:59 smithi136 ceph-mon[35324]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:00.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:35:59 smithi090 ceph-mon[31429]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:01.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:01 smithi090 ceph-mon[31429]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:01.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:01.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:01.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:01.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:01.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:01.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:01 smithi090 ceph-mon[31429]: from='osd.5 [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:36:01.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:01 smithi090 ceph-mon[31429]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:36:01.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:01 smithi136 ceph-mon[35324]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:01.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:01.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:01.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:01.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:01.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:01.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:01 smithi136 ceph-mon[35324]: from='osd.5 [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:36:01.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:01 smithi136 ceph-mon[35324]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:36:02.070 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:01 smithi129 ceph-mon[35254]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:02.070 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:02.070 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:02.070 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:02.070 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:02.070 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:02.070 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:01 smithi129 ceph-mon[35254]: from='osd.5 [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:36:02.070 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:01 smithi129 ceph-mon[35254]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T07:36:02.911 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 5 on host 'smithi136' 2024-05-27T07:36:02.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.915+0000 7fe409600700 1 -- 172.21.15.136:0/257741531 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe40c063bf0 con 0x7fe3f00b2980 2024-05-27T07:36:02.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.919+0000 7fe411400700 1 -- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe3f00b2980 msgr2=0x7fe3f00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:02.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.919+0000 7fe411400700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe3f00b2980 0x7fe3f00b4e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4000c00 tx=0x7fe3f4005fd0 comp rx=0 tx=0).stop 2024-05-27T07:36:02.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.919+0000 7fe411400700 1 -- 172.21.15.136:0/257741531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe40c0760e0 msgr2=0x7fe40c19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:02.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.919+0000 7fe411400700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe40c0760e0 0x7fe40c19c820 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00c9e0 tx=0x7fe3fc00cda0 comp rx=0 tx=0).stop 2024-05-27T07:36:02.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.919+0000 7fe411400700 1 -- 172.21.15.136:0/257741531 shutdown_connections 2024-05-27T07:36:02.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.919+0000 7fe411400700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe3f00b2980 0x7fe3f00b4e90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:02.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.919+0000 7fe411400700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe40c10bff0 0x7fe40c1a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:02.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.919+0000 7fe411400700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe40c0760e0 0x7fe40c19c820 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:02.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.919+0000 7fe411400700 1 --2- 172.21.15.136:0/257741531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe40c0776d0 0x7fe40c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:02.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.919+0000 7fe411400700 1 -- 172.21.15.136:0/257741531 >> 172.21.15.136:0/257741531 conn(0x7fe40c0ff7a0 msgr2=0x7fe40c107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:02.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.920+0000 7fe411400700 1 -- 172.21.15.136:0/257741531 shutdown_connections 2024-05-27T07:36:02.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:36:02.920+0000 7fe411400700 1 -- 172.21.15.136:0/257741531 wait complete. 2024-05-27T07:36:03.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:02 smithi129 ceph-mon[35254]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T07:36:03.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:02 smithi129 ceph-mon[35254]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T07:36:03.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:03.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:02 smithi129 ceph-mon[35254]: from='osd.5 [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:36:03.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:02 smithi129 ceph-mon[35254]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:36:03.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:02 smithi090 ceph-mon[31429]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T07:36:03.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:02 smithi090 ceph-mon[31429]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T07:36:03.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:03.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:02 smithi090 ceph-mon[31429]: from='osd.5 [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:36:03.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:02 smithi090 ceph-mon[31429]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:36:03.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:02 smithi136 ceph-mon[35324]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T07:36:03.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:02 smithi136 ceph-mon[35324]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T07:36:03.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:03.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:02 smithi136 ceph-mon[35324]: from='osd.5 [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:36:03.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:02 smithi136 ceph-mon[35324]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:36:03.876 DEBUG:teuthology.orchestra.run.smithi136:osd.5> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.5.service 2024-05-27T07:36:03.882 INFO:tasks.cephadm:Deploying osd.6 on smithi090 with /dev/vg_nvme/lv_2... 2024-05-27T07:36:03.882 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T07:36:04.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:03 smithi129 ceph-mon[35254]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:04.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:03 smithi129 ceph-mon[35254]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:36:04.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:03 smithi129 ceph-mon[35254]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T07:36:04.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:04.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:04.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:36:04.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:04.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:04.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:03 smithi136 ceph-mon[35324]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:04.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:03 smithi136 ceph-mon[35324]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:36:04.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:03 smithi136 ceph-mon[35324]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T07:36:04.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:04.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:04.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:36:04.168 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:04.168 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:04.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:03 smithi090 ceph-mon[31429]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:04.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:03 smithi090 ceph-mon[31429]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:36:04.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:03 smithi090 ceph-mon[31429]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T07:36:04.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:04.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:04.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:36:04.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:04.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:36:05.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:04 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:36:05.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:04 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:36:05.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:04 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:05.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:04 smithi090 ceph-mon[31429]: osd.5 [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] boot 2024-05-27T07:36:05.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:04 smithi090 ceph-mon[31429]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T07:36:05.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:04 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:05.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:04 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:36:05.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:04 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:36:05.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:04 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:05.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:04 smithi136 ceph-mon[35324]: osd.5 [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] boot 2024-05-27T07:36:05.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:04 smithi136 ceph-mon[35324]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T07:36:05.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:04 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:05.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:04 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:36:05.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:04 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:36:05.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:04 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:05.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:04 smithi129 ceph-mon[35254]: osd.5 [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] boot 2024-05-27T07:36:05.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:04 smithi129 ceph-mon[35254]: osdmap e38: 6 total, 6 up, 6 in 2024-05-27T07:36:05.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:04 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:36:05.919 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:05 smithi136 ceph-mon[35324]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:05.920 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:05 smithi136 ceph-mon[35324]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T07:36:05.921 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:05 smithi090 ceph-mon[31429]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:05.921 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:05 smithi090 ceph-mon[31429]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T07:36:06.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:05 smithi129 ceph-mon[35254]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-27T07:36:06.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:05 smithi129 ceph-mon[35254]: osdmap e39: 6 total, 6 up, 6 in 2024-05-27T07:36:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T07:36:07.086 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T07:36:07.087 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-05-27T07:36:07.087 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-05-27T07:36:07.087 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0329815 s, 318 MB/s 2024-05-27T07:36:07.087 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-05-27T07:36:07.087 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:06 smithi090 ceph-mon[31429]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T07:36:07.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:06 smithi136 ceph-mon[35324]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T07:36:07.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:06 smithi129 ceph-mon[35254]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T07:36:07.926 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-05-27T07:36:07.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:07 smithi136 ceph-mon[35324]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:07.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:07 smithi090 ceph-mon[31429]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:08.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:07 smithi129 ceph-mon[35254]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:08.361 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:36:09.890 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: pgmap v185: 1 pgs: 1 activating+degraded; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T07:36:09.890 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi136 2024-05-27T07:36:09.890 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:09.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:09.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:09.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:09.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi136 to 8544M 2024-05-27T07:36:09.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:09.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:09.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:09.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:09.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:09 smithi090 ceph-mon[31429]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-27T07:36:09.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: pgmap v185: 1 pgs: 1 activating+degraded; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T07:36:09.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi136 2024-05-27T07:36:09.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:09.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:09.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:09.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:09.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi136 to 8544M 2024-05-27T07:36:09.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:09.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:09.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:09.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:09.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:09 smithi136 ceph-mon[35324]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-27T07:36:10.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: pgmap v185: 1 pgs: 1 activating+degraded; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi136 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi136 to 8544M 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:09 smithi129 ceph-mon[35254]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-27T07:36:10.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.461+0000 7f078cc00700 1 -- 172.21.15.90:0/2594157716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f078810ab00 msgr2=0x7f078810af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:10.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.461+0000 7f078cc00700 1 --2- 172.21.15.90:0/2594157716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f078810ab00 0x7f078810af10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0778009990 tx=0x7f0778009ca0 comp rx=0 tx=0).stop 2024-05-27T07:36:10.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.461+0000 7f078cc00700 1 -- 172.21.15.90:0/2594157716 shutdown_connections 2024-05-27T07:36:10.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.461+0000 7f078cc00700 1 --2- 172.21.15.90:0/2594157716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f078810c410 0x7f0788112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:10.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.461+0000 7f078cc00700 1 --2- 172.21.15.90:0/2594157716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f078810ab00 0x7f078810af10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:10.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.461+0000 7f078cc00700 1 --2- 172.21.15.90:0/2594157716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f078810ba70 0x7f078810be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:10.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.461+0000 7f078cc00700 1 -- 172.21.15.90:0/2594157716 >> 172.21.15.90:0/2594157716 conn(0x7f07880799b0 msgr2=0x7f078807be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:10.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.462+0000 7f078cc00700 1 -- 172.21.15.90:0/2594157716 shutdown_connections 2024-05-27T07:36:10.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.462+0000 7f078cc00700 1 -- 172.21.15.90:0/2594157716 wait complete. 2024-05-27T07:36:10.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.463+0000 7f078cc00700 1 Processor -- start 2024-05-27T07:36:10.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.463+0000 7f078cc00700 1 -- start start 2024-05-27T07:36:10.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.463+0000 7f078cc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f078810ab00 0x7f07881a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:10.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.463+0000 7f078cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f078810ba70 0x7f07881a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:10.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.463+0000 7f078cc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f078810c410 0x7f07881a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:10.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.463+0000 7f078cc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0788116230 con 0x7f078810ba70 2024-05-27T07:36:10.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.463+0000 7f078cc00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07881160b0 con 0x7f078810c410 2024-05-27T07:36:10.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.463+0000 7f078cc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07881163b0 con 0x7f078810ab00 2024-05-27T07:36:10.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.463+0000 7f0785400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f078810ba70 0x7f07881a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:10.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0785400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f078810ba70 0x7f07881a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41376/0 (socket says 172.21.15.90:41376) 2024-05-27T07:36:10.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0785400700 1 -- 172.21.15.90:0/3197241672 learned_addr learned my addr 172.21.15.90:0/3197241672 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:36:10.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0785e00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f078810ab00 0x7f07881a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:10.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0786800700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f078810c410 0x7f07881a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:10.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0785400700 1 -- 172.21.15.90:0/3197241672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f078810ab00 msgr2=0x7f07881a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:10.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0785400700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f078810ab00 0x7f07881a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:10.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0785400700 1 -- 172.21.15.90:0/3197241672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f078810c410 msgr2=0x7f07881a82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:10.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0785400700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f078810c410 0x7f07881a82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:10.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0785400700 1 -- 172.21.15.90:0/3197241672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07881a88d0 con 0x7f078810ba70 2024-05-27T07:36:10.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0785e00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f078810ab00 0x7f07881a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:36:10.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.464+0000 7f0785400700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f078810ba70 0x7f07881a1340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f077400d950 tx=0x7f077400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:10.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.465+0000 7f077ea00700 1 -- 172.21.15.90:0/3197241672 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774016070 con 0x7f078810ba70 2024-05-27T07:36:10.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.465+0000 7f077ea00700 1 -- 172.21.15.90:0/3197241672 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0774010460 con 0x7f078810ba70 2024-05-27T07:36:10.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.465+0000 7f078cc00700 1 -- 172.21.15.90:0/3197241672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07881a8bc0 con 0x7f078810ba70 2024-05-27T07:36:10.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.465+0000 7f077ea00700 1 -- 172.21.15.90:0/3197241672 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077400f620 con 0x7f078810ba70 2024-05-27T07:36:10.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.465+0000 7f078cc00700 1 -- 172.21.15.90:0/3197241672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07881a9150 con 0x7f078810ba70 2024-05-27T07:36:10.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.466+0000 7f078cc00700 1 -- 172.21.15.90:0/3197241672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07880512b0 con 0x7f078810ba70 2024-05-27T07:36:10.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.469+0000 7f077ea00700 1 -- 172.21.15.90:0/3197241672 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0774019070 con 0x7f078810ba70 2024-05-27T07:36:10.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.471+0000 7f077ea00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f076c0b2900 0x7f076c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:10.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.472+0000 7f077ea00700 1 -- 172.21.15.90:0/3197241672 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4533+0+0 (secure 0 0 0) 0x7f07740e0f60 con 0x7f078810ba70 2024-05-27T07:36:10.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.472+0000 7f0785e00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f076c0b2900 0x7f076c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:10.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.473+0000 7f0785e00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f076c0b2900 0x7f076c0b4e10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0778003820 tx=0x7f077801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:10.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.473+0000 7f077ea00700 1 -- 172.21.15.90:0/3197241672 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f07740a9aa0 con 0x7f078810ba70 2024-05-27T07:36:10.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:10.717+0000 7f078cc00700 1 -- 172.21.15.90:0/3197241672 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0788063bf0 con 0x7f076c0b2900 2024-05-27T07:36:11.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:11 smithi090 ceph-mon[31429]: pgmap v186: 1 pgs: 1 activating+degraded; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T07:36:11.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:11 smithi090 ceph-mon[31429]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:36:11.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:36:11.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:36:11.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:11.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:11 smithi136 ceph-mon[35324]: pgmap v186: 1 pgs: 1 activating+degraded; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T07:36:11.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:11 smithi136 ceph-mon[35324]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:36:11.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:36:11.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:36:11.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:12.068 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:11 smithi129 ceph-mon[35254]: pgmap v186: 1 pgs: 1 activating+degraded; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T07:36:12.068 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:11 smithi129 ceph-mon[35254]: from='client.14337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:36:12.068 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:36:12.068 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:36:12.068 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:12.924 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:12 smithi090 ceph-mon[31429]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-27T07:36:12.925 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:12 smithi090 ceph-mon[31429]: Cluster is now healthy 2024-05-27T07:36:12.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:12 smithi136 ceph-mon[35324]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-27T07:36:12.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:12 smithi136 ceph-mon[35324]: Cluster is now healthy 2024-05-27T07:36:13.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:12 smithi129 ceph-mon[35254]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-27T07:36:13.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:12 smithi129 ceph-mon[35254]: Cluster is now healthy 2024-05-27T07:36:13.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:13 smithi090 ceph-mon[31429]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T07:36:13.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:13 smithi136 ceph-mon[35324]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T07:36:14.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:13 smithi129 ceph-mon[35254]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T07:36:15.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:15 smithi090 ceph-mon[31429]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-27T07:36:15.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:15 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1395087542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f675d248-884d-4bbb-88ab-6edab1eafcda"}]: dispatch 2024-05-27T07:36:15.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:15 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1395087542' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f675d248-884d-4bbb-88ab-6edab1eafcda"}]': finished 2024-05-27T07:36:15.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:15 smithi090 ceph-mon[31429]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T07:36:15.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:15.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:15 smithi136 ceph-mon[35324]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-27T07:36:15.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:15 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1395087542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f675d248-884d-4bbb-88ab-6edab1eafcda"}]: dispatch 2024-05-27T07:36:15.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:15 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1395087542' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f675d248-884d-4bbb-88ab-6edab1eafcda"}]': finished 2024-05-27T07:36:15.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:15 smithi136 ceph-mon[35324]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T07:36:15.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:16.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:15 smithi129 ceph-mon[35254]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-27T07:36:16.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:15 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1395087542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f675d248-884d-4bbb-88ab-6edab1eafcda"}]: dispatch 2024-05-27T07:36:16.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:15 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1395087542' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f675d248-884d-4bbb-88ab-6edab1eafcda"}]': finished 2024-05-27T07:36:16.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:15 smithi129 ceph-mon[35254]: osdmap e41: 7 total, 6 up, 7 in 2024-05-27T07:36:16.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:16.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:16 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2002783416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:36:16.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:16 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2002783416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:36:17.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:16 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2002783416' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:36:17.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:17 smithi090 ceph-mon[31429]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:36:17.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:17 smithi136 ceph-mon[35324]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:36:18.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:17 smithi129 ceph-mon[35254]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:36:19.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:19 smithi090 ceph-mon[31429]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:36:19.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:19 smithi136 ceph-mon[35324]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:36:20.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:19 smithi129 ceph-mon[35254]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:36:21.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:21 smithi090 ceph-mon[31429]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:36:21.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:21 smithi136 ceph-mon[35324]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:36:22.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:21 smithi129 ceph-mon[35254]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:36:23.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:23 smithi090 ceph-mon[31429]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:23.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:23 smithi136 ceph-mon[35324]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:24.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:23 smithi129 ceph-mon[35254]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:24.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:24 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T07:36:24.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:24 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:24.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:24 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T07:36:24.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:24 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:25.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:24 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T07:36:25.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:24 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:25.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:25 smithi090 ceph-mon[31429]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:25.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:25 smithi090 ceph-mon[31429]: Deploying daemon osd.6 on smithi090 2024-05-27T07:36:25.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:25 smithi136 ceph-mon[35324]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:25.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:25 smithi136 ceph-mon[35324]: Deploying daemon osd.6 on smithi090 2024-05-27T07:36:26.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:25 smithi129 ceph-mon[35254]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:26.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:25 smithi129 ceph-mon[35254]: Deploying daemon osd.6 on smithi090 2024-05-27T07:36:27.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:27 smithi090 ceph-mon[31429]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:27.994 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:27 smithi136 ceph-mon[35324]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:28.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:27 smithi129 ceph-mon[35254]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:29.933 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:29 smithi090 ceph-mon[31429]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:29.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:29 smithi136 ceph-mon[35324]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:30.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:29 smithi129 ceph-mon[35254]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:31.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:31 smithi136 ceph-mon[35324]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:32.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:31 smithi129 ceph-mon[35254]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:32.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:31 smithi090 ceph-mon[31429]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:33.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:33 smithi090 ceph-mon[31429]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:33.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:33 smithi136 ceph-mon[35324]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:34.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:33 smithi129 ceph-mon[35254]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:35.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:34 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:35.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:34 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:35.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:34 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:36:35.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:34 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:35.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:34 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:35.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:34 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:36:35.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:34 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:35.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:34 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:35.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:34 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:36:36.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:35 smithi090 ceph-mon[31429]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:36.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:35 smithi136 ceph-mon[35324]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:36.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:35 smithi129 ceph-mon[35254]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:38.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:37 smithi090 ceph-mon[31429]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:38.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:38.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:38.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:38.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:38.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:38.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:37 smithi136 ceph-mon[35324]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:38.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:38.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:38.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:38.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:38.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:38.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:37 smithi129 ceph-mon[35254]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:38.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:38.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:38.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:38.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:38.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:39.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:38 smithi090 ceph-mon[31429]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T07:36:39.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:38 smithi136 ceph-mon[35324]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T07:36:39.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:38 smithi129 ceph-mon[35254]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T07:36:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.920+0000 7f077ea00700 1 -- 172.21.15.90:0/3197241672 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0788063bf0 con 0x7f076c0b2900 2024-05-27T07:36:39.917 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 6 on host 'smithi090' 2024-05-27T07:36:39.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.924+0000 7f078cc00700 1 -- 172.21.15.90:0/3197241672 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f076c0b2900 msgr2=0x7f076c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.924+0000 7f078cc00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f076c0b2900 0x7f076c0b4e10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0778003820 tx=0x7f077801a040 comp rx=0 tx=0).stop 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.924+0000 7f078cc00700 1 -- 172.21.15.90:0/3197241672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f078810ba70 msgr2=0x7f07881a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.924+0000 7f078cc00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f078810ba70 0x7f07881a1340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f077400d950 tx=0x7f077400dd10 comp rx=0 tx=0).stop 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.925+0000 7f078cc00700 1 -- 172.21.15.90:0/3197241672 shutdown_connections 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.925+0000 7f078cc00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f076c0b2900 0x7f076c0b4e10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.925+0000 7f078cc00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f078810c410 0x7f07881a82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.925+0000 7f078cc00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f078810ab00 0x7f07881a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.925+0000 7f078cc00700 1 --2- 172.21.15.90:0/3197241672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f078810ba70 0x7f07881a1340 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.925+0000 7f078cc00700 1 -- 172.21.15.90:0/3197241672 >> 172.21.15.90:0/3197241672 conn(0x7f07880799b0 msgr2=0x7f078810eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.925+0000 7f078cc00700 1 -- 172.21.15.90:0/3197241672 shutdown_connections 2024-05-27T07:36:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:36:39.925+0000 7f078cc00700 1 -- 172.21.15.90:0/3197241672 wait complete. 2024-05-27T07:36:40.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:39 smithi090 ceph-mon[31429]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:40.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:39 smithi090 ceph-mon[31429]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T07:36:40.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:39 smithi090 ceph-mon[31429]: osdmap e42: 7 total, 6 up, 7 in 2024-05-27T07:36:40.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:39 smithi090 ceph-mon[31429]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:36:40.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:40.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:40.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:40.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:36:40.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:39 smithi136 ceph-mon[35324]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:40.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:39 smithi136 ceph-mon[35324]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T07:36:40.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:39 smithi136 ceph-mon[35324]: osdmap e42: 7 total, 6 up, 7 in 2024-05-27T07:36:40.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:39 smithi136 ceph-mon[35324]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:36:40.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:40.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:40.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:40.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:36:40.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:39 smithi129 ceph-mon[35254]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:40.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:39 smithi129 ceph-mon[35254]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T07:36:40.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:39 smithi129 ceph-mon[35254]: osdmap e42: 7 total, 6 up, 7 in 2024-05-27T07:36:40.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:39 smithi129 ceph-mon[35254]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:36:40.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:40.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:40.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:40.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:36:40.750 DEBUG:teuthology.orchestra.run.smithi090:osd.6> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.6.service 2024-05-27T07:36:40.754 INFO:tasks.cephadm:Deploying osd.7 on smithi129 with /dev/vg_nvme/lv_2... 2024-05-27T07:36:40.754 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T07:36:40.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:40 smithi090 ceph-mon[31429]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:36:40.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:40 smithi090 ceph-mon[31429]: osdmap e43: 7 total, 6 up, 7 in 2024-05-27T07:36:40.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:40.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:40.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:40 smithi090 ceph-mon[31429]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:40.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:40 smithi090 ceph-mon[31429]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' 2024-05-27T07:36:40.959 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 07:36:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T07:36:40.840+0000 7fe32e800700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T07:36:41.194 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:36:41.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:40 smithi136 ceph-mon[35324]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:36:41.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:40 smithi136 ceph-mon[35324]: osdmap e43: 7 total, 6 up, 7 in 2024-05-27T07:36:41.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:41.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:41.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:40 smithi136 ceph-mon[35324]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:41.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:40 smithi136 ceph-mon[35324]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' 2024-05-27T07:36:41.309 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:40 smithi129 ceph-mon[35254]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:36:41.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:40 smithi129 ceph-mon[35254]: osdmap e43: 7 total, 6 up, 7 in 2024-05-27T07:36:41.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:41.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:41.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:40 smithi129 ceph-mon[35254]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T07:36:41.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:40 smithi129 ceph-mon[35254]: from='osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683]' entity='osd.6' 2024-05-27T07:36:42.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:41 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:36:42.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:41 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:36:42.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:42.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:41 smithi129 ceph-mon[35254]: osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] boot 2024-05-27T07:36:42.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:41 smithi129 ceph-mon[35254]: osdmap e44: 7 total, 7 up, 7 in 2024-05-27T07:36:42.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:42.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:41 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:36:42.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:41 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:36:42.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:42.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:41 smithi090 ceph-mon[31429]: osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] boot 2024-05-27T07:36:42.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:41 smithi090 ceph-mon[31429]: osdmap e44: 7 total, 7 up, 7 in 2024-05-27T07:36:42.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:42.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:41 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:36:42.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:41 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:36:42.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:42.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:41 smithi136 ceph-mon[35324]: osd.6 [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] boot 2024-05-27T07:36:42.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:41 smithi136 ceph-mon[35324]: osdmap e44: 7 total, 7 up, 7 in 2024-05-27T07:36:42.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:36:44.190 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:43 smithi129 ceph-mon[35254]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:44.190 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:43 smithi129 ceph-mon[35254]: osdmap e45: 7 total, 7 up, 7 in 2024-05-27T07:36:44.191 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T07:36:44.191 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T07:36:44.191 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-05-27T07:36:44.191 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-05-27T07:36:44.193 INFO:teuthology.orchestra.run.smithi129.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0343768 s, 305 MB/s 2024-05-27T07:36:44.193 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 2024-05-27T07:36:44.193 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-05-27T07:36:44.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:43 smithi090 ceph-mon[31429]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:44.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:43 smithi090 ceph-mon[31429]: osdmap e45: 7 total, 7 up, 7 in 2024-05-27T07:36:44.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:43 smithi136 ceph-mon[35324]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:44.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:43 smithi136 ceph-mon[35324]: osdmap e45: 7 total, 7 up, 7 in 2024-05-27T07:36:45.104 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_2 2024-05-27T07:36:45.536 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:36:46.000 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:45 smithi129 ceph-mon[35254]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:46.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:45 smithi090 ceph-mon[31429]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:46.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:45 smithi136 ceph-mon[35324]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:47.254 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi090 2024-05-27T07:36:47.254 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.254 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.254 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:47.254 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:47.254 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:47.254 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi090 to 5696M 2024-05-27T07:36:47.255 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.255 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:47.255 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:47.255 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.255 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:47 smithi129 ceph-mon[35254]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:47.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi090 2024-05-27T07:36:47.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:47.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi090 to 5696M 2024-05-27T07:36:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:47 smithi090 ceph-mon[31429]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:47.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi090 2024-05-27T07:36:47.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:47.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:47.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:36:47.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi090 to 5696M 2024-05-27T07:36:47.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:47.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:36:47.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:36:47.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:47 smithi136 ceph-mon[35324]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:47.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.832+0000 7fd38be00700 1 -- 172.21.15.129:0/4215764581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410c410 msgr2=0x7fd384112cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:47.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.832+0000 7fd38be00700 1 --2- 172.21.15.129:0/4215764581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410c410 0x7fd384112cc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b0a0 tx=0x7fd37c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:36:47.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.832+0000 7fd38be00700 1 -- 172.21.15.129:0/4215764581 shutdown_connections 2024-05-27T07:36:47.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.832+0000 7fd38be00700 1 --2- 172.21.15.129:0/4215764581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410c410 0x7fd384112cc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:47.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.832+0000 7fd38be00700 1 --2- 172.21.15.129:0/4215764581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38410ab00 0x7fd38410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:47.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.832+0000 7fd38be00700 1 --2- 172.21.15.129:0/4215764581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410ba70 0x7fd38410be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:47.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.832+0000 7fd38be00700 1 -- 172.21.15.129:0/4215764581 >> 172.21.15.129:0/4215764581 conn(0x7fd3840799b0 msgr2=0x7fd38407be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:36:47.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.832+0000 7fd38be00700 1 -- 172.21.15.129:0/4215764581 shutdown_connections 2024-05-27T07:36:47.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.833+0000 7fd38be00700 1 -- 172.21.15.129:0/4215764581 wait complete. 2024-05-27T07:36:47.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.833+0000 7fd38be00700 1 Processor -- start 2024-05-27T07:36:47.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.834+0000 7fd38be00700 1 -- start start 2024-05-27T07:36:47.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.834+0000 7fd38be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38410ab00 0x7fd3841a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:47.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.834+0000 7fd38be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410ba70 0x7fd3841a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:47.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.834+0000 7fd38be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410c410 0x7fd3841a8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:47.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.834+0000 7fd38be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3841162b0 con 0x7fd38410ba70 2024-05-27T07:36:47.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.834+0000 7fd38be00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd384116130 con 0x7fd38410c410 2024-05-27T07:36:47.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.834+0000 7fd38be00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd384116430 con 0x7fd38410ab00 2024-05-27T07:36:47.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.834+0000 7fd389e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410c410 0x7fd3841a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:47.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.834+0000 7fd389e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410c410 0x7fd3841a8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34934/0 (socket says 172.21.15.129:34934) 2024-05-27T07:36:47.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.834+0000 7fd389e00700 1 -- 172.21.15.129:0/982796735 learned_addr learned my addr 172.21.15.129:0/982796735 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.835+0000 7fd388a00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410ba70 0x7fd3841a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.835+0000 7fd389400700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38410ab00 0x7fd3841a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.835+0000 7fd389e00700 1 -- 172.21.15.129:0/982796735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38410ab00 msgr2=0x7fd3841a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.835+0000 7fd389e00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38410ab00 0x7fd3841a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.835+0000 7fd389e00700 1 -- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410ba70 msgr2=0x7fd3841a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.835+0000 7fd389e00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410ba70 0x7fd3841a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.835+0000 7fd389e00700 1 -- 172.21.15.129:0/982796735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3841a8900 con 0x7fd38410c410 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.835+0000 7fd388a00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410ba70 0x7fd3841a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.835+0000 7fd389e00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410c410 0x7fd3841a8280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b830 tx=0x7fd37c00bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.835+0000 7fd382000700 1 -- 172.21.15.129:0/982796735 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c021070 con 0x7fd38410c410 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.836+0000 7fd382000700 1 -- 172.21.15.129:0/982796735 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd37c007c80 con 0x7fd38410c410 2024-05-27T07:36:47.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.836+0000 7fd38be00700 1 -- 172.21.15.129:0/982796735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3841a8b90 con 0x7fd38410c410 2024-05-27T07:36:47.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.836+0000 7fd382000700 1 -- 172.21.15.129:0/982796735 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c01b7b0 con 0x7fd38410c410 2024-05-27T07:36:47.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.836+0000 7fd38be00700 1 -- 172.21.15.129:0/982796735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3841a8fe0 con 0x7fd38410c410 2024-05-27T07:36:47.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.837+0000 7fd38be00700 1 -- 172.21.15.129:0/982796735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3840512b0 con 0x7fd38410c410 2024-05-27T07:36:47.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.840+0000 7fd382000700 1 -- 172.21.15.129:0/982796735 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd37c024070 con 0x7fd38410c410 2024-05-27T07:36:47.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.842+0000 7fd382000700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd36c0b2880 0x7fd36c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:36:47.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.842+0000 7fd382000700 1 -- 172.21.15.129:0/982796735 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7fd37c00f020 con 0x7fd38410c410 2024-05-27T07:36:47.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.843+0000 7fd389400700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd36c0b2880 0x7fd36c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:36:47.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.844+0000 7fd389400700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd36c0b2880 0x7fd36c0b4d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd378005950 tx=0x7fd3780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:36:47.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:47.845+0000 7fd382000700 1 -- 172.21.15.129:0/982796735 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd37c02b020 con 0x7fd38410c410 2024-05-27T07:36:48.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:36:48.085+0000 7fd38be00700 1 -- 172.21.15.129:0/982796735 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd384063bf0 con 0x7fd36c0b2880 2024-05-27T07:36:48.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:48 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:36:48.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:48 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:36:48.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:48 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:48.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:48 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:36:48.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:48 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:36:48.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:48 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:48.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:48 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:36:48.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:48 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:36:48.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:48 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:36:49.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:49 smithi129 ceph-mon[35254]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:36:49.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:49 smithi129 ceph-mon[35254]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:49.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:49 smithi090 ceph-mon[31429]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:36:49.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:49 smithi090 ceph-mon[31429]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:49.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:49 smithi136 ceph-mon[35324]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:36:49.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:49 smithi136 ceph-mon[35324]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:51.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:51 smithi090 ceph-mon[31429]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:51.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:51 smithi136 ceph-mon[35324]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:51.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:51 smithi129 ceph-mon[35254]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:53.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:53 smithi090 ceph-mon[31429]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:53.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:53 smithi090 ceph-mon[31429]: from='client.? 172.21.15.129:0/4108429783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a66106-a43b-4eb3-adf2-5291094db025"}]: dispatch 2024-05-27T07:36:53.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:53 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a66106-a43b-4eb3-adf2-5291094db025"}]: dispatch 2024-05-27T07:36:53.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:53 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22a66106-a43b-4eb3-adf2-5291094db025"}]': finished 2024-05-27T07:36:53.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:53 smithi090 ceph-mon[31429]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T07:36:53.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:53 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:36:53.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:53 smithi136 ceph-mon[35324]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:53.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:53 smithi136 ceph-mon[35324]: from='client.? 172.21.15.129:0/4108429783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a66106-a43b-4eb3-adf2-5291094db025"}]: dispatch 2024-05-27T07:36:53.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:53 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a66106-a43b-4eb3-adf2-5291094db025"}]: dispatch 2024-05-27T07:36:53.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:53 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22a66106-a43b-4eb3-adf2-5291094db025"}]': finished 2024-05-27T07:36:53.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:53 smithi136 ceph-mon[35324]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T07:36:53.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:53 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:36:53.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:53 smithi129 ceph-mon[35254]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:53.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:53 smithi129 ceph-mon[35254]: from='client.? 172.21.15.129:0/4108429783' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a66106-a43b-4eb3-adf2-5291094db025"}]: dispatch 2024-05-27T07:36:53.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:53 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a66106-a43b-4eb3-adf2-5291094db025"}]: dispatch 2024-05-27T07:36:53.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:53 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22a66106-a43b-4eb3-adf2-5291094db025"}]': finished 2024-05-27T07:36:53.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:53 smithi129 ceph-mon[35254]: osdmap e46: 8 total, 7 up, 8 in 2024-05-27T07:36:53.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:53 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:36:54.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:54 smithi090 ceph-mon[31429]: from='client.? 172.21.15.129:0/3480042131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:36:54.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:54 smithi136 ceph-mon[35324]: from='client.? 172.21.15.129:0/3480042131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:36:54.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:54 smithi129 ceph-mon[35254]: from='client.? 172.21.15.129:0/3480042131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:36:55.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:55 smithi090 ceph-mon[31429]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:55.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:55 smithi136 ceph-mon[35324]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:55.809 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:55 smithi129 ceph-mon[35254]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:57.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:57 smithi129 ceph-mon[35254]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:57.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:57 smithi090 ceph-mon[31429]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:57.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:57 smithi136 ceph-mon[35324]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:59.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:36:59 smithi129 ceph-mon[35254]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:59.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:36:59 smithi090 ceph-mon[31429]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:36:59.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:36:59 smithi136 ceph-mon[35324]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:01.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:01 smithi090 ceph-mon[31429]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:01.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:01 smithi136 ceph-mon[35324]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:01.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:01 smithi129 ceph-mon[35254]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:02.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T07:37:02.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:02.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T07:37:02.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:02.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T07:37:02.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:03.551 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:03 smithi129 ceph-mon[35254]: Deploying daemon osd.7 on smithi129 2024-05-27T07:37:03.551 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:03 smithi129 ceph-mon[35254]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:03.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:03 smithi090 ceph-mon[31429]: Deploying daemon osd.7 on smithi129 2024-05-27T07:37:03.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:03 smithi090 ceph-mon[31429]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:03.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:03 smithi136 ceph-mon[35324]: Deploying daemon osd.7 on smithi129 2024-05-27T07:37:03.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:03 smithi136 ceph-mon[35324]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:05.559 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:05 smithi129 ceph-mon[35254]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:05.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:05 smithi090 ceph-mon[31429]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:05.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:05 smithi136 ceph-mon[35324]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:07.501 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:07 smithi129 ceph-mon[35254]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:07.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:07 smithi090 ceph-mon[31429]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:07.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:07 smithi136 ceph-mon[35324]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:09.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:09 smithi090 ceph-mon[31429]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:09.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:09 smithi136 ceph-mon[35324]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:09.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:09 smithi129 ceph-mon[35254]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:11.586 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:11 smithi129 ceph-mon[35254]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:11.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:11 smithi090 ceph-mon[31429]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:11.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:11 smithi136 ceph-mon[35324]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:13.059 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:13.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:13.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:13.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:13.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:13.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:13.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:13.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:13.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:14.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:13 smithi129 ceph-mon[35254]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:14.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:13 smithi090 ceph-mon[31429]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:14.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:13 smithi136 ceph-mon[35324]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:16.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:15 smithi129 ceph-mon[35254]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:16.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:16.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:16.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:15 smithi090 ceph-mon[31429]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:16.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:16.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:16.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:15 smithi136 ceph-mon[35324]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:16.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:16.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:17.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:16 smithi090 ceph-mon[31429]: from='osd.7 [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:37:17.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:16 smithi090 ceph-mon[31429]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:37:17.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:16 smithi136 ceph-mon[35324]: from='osd.7 [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:37:17.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:16 smithi136 ceph-mon[35324]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:37:17.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:16 smithi129 ceph-mon[35254]: from='osd.7 [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:37:17.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:16 smithi129 ceph-mon[35254]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T07:37:17.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.564+0000 7fd382000700 1 -- 172.21.15.129:0/982796735 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd384063bf0 con 0x7fd36c0b2880 2024-05-27T07:37:17.560 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 7 on host 'smithi129' 2024-05-27T07:37:17.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.568+0000 7fd38be00700 1 -- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd36c0b2880 msgr2=0x7fd36c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:17.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.568+0000 7fd38be00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd36c0b2880 0x7fd36c0b4d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd378005950 tx=0x7fd3780058e0 comp rx=0 tx=0).stop 2024-05-27T07:37:17.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.568+0000 7fd38be00700 1 -- 172.21.15.129:0/982796735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410c410 msgr2=0x7fd3841a8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:17.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.568+0000 7fd38be00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410c410 0x7fd3841a8280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b830 tx=0x7fd37c00bd70 comp rx=0 tx=0).stop 2024-05-27T07:37:17.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.568+0000 7fd38be00700 1 -- 172.21.15.129:0/982796735 shutdown_connections 2024-05-27T07:37:17.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.568+0000 7fd38be00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd36c0b2880 0x7fd36c0b4d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:17.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.568+0000 7fd38be00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410c410 0x7fd3841a8280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:17.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.568+0000 7fd38be00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38410ab00 0x7fd3841a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:17.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.568+0000 7fd38be00700 1 --2- 172.21.15.129:0/982796735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410ba70 0x7fd3841a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:17.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.568+0000 7fd38be00700 1 -- 172.21.15.129:0/982796735 >> 172.21.15.129:0/982796735 conn(0x7fd3840799b0 msgr2=0x7fd38410eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:17.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.569+0000 7fd38be00700 1 -- 172.21.15.129:0/982796735 shutdown_connections 2024-05-27T07:37:17.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:37:17.569+0000 7fd38be00700 1 -- 172.21.15.129:0/982796735 wait complete. 2024-05-27T07:37:18.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: from='osd.7 [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T07:37:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: from='osd.7 [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:37:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T07:37:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:37:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:18.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:18.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:18.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:18.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:18.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T07:37:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: from='osd.7 [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:37:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: osdmap e47: 8 total, 7 up, 8 in 2024-05-27T07:37:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:37:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:17 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:18.512 DEBUG:teuthology.orchestra.run.smithi129:osd.7> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.7.service 2024-05-27T07:37:18.518 INFO:tasks.cephadm:Deploying osd.8 on smithi136 with /dev/vg_nvme/lv_2... 2024-05-27T07:37:18.518 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T07:37:18.966 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:37:19.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:18 smithi129 ceph-mon[35254]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:37:19.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:18 smithi129 ceph-mon[35254]: osdmap e48: 8 total, 7 up, 8 in 2024-05-27T07:37:19.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:18 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:19.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:18 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:19.164 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:18 smithi136 ceph-mon[35324]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:37:19.164 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:18 smithi136 ceph-mon[35324]: osdmap e48: 8 total, 7 up, 8 in 2024-05-27T07:37:19.164 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:18 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:19.164 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:18 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:19.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:18 smithi090 ceph-mon[31429]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:37:19.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:18 smithi090 ceph-mon[31429]: osdmap e48: 8 total, 7 up, 8 in 2024-05-27T07:37:19.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:18 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:19.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:18 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:20.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:19 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:37:20.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:19 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:37:20.018 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:19 smithi129 ceph-mon[35254]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:20.018 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:20.018 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:19 smithi129 ceph-mon[35254]: osd.7 [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] boot 2024-05-27T07:37:20.018 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:19 smithi129 ceph-mon[35254]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T07:37:20.018 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:20.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:19 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:37:20.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:19 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:37:20.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:19 smithi136 ceph-mon[35324]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:20.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:19 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:20.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:19 smithi136 ceph-mon[35324]: osd.7 [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] boot 2024-05-27T07:37:20.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:19 smithi136 ceph-mon[35324]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T07:37:20.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:19 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:20.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:19 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:37:20.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:19 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:37:20.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:19 smithi090 ceph-mon[31429]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T07:37:20.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:20.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:19 smithi090 ceph-mon[31429]: osd.7 [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] boot 2024-05-27T07:37:20.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:19 smithi090 ceph-mon[31429]: osdmap e49: 8 total, 8 up, 8 in 2024-05-27T07:37:20.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:37:21.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:20 smithi129 ceph-mon[35254]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T07:37:21.117 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:20 smithi136 ceph-mon[35324]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T07:37:21.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:20 smithi090 ceph-mon[31429]: osdmap e50: 8 total, 8 up, 8 in 2024-05-27T07:37:22.006 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T07:37:22.007 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T07:37:22.007 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-05-27T07:37:22.007 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-05-27T07:37:22.007 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0336369 s, 312 MB/s 2024-05-27T07:37:22.007 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-05-27T07:37:22.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:21 smithi090 ceph-mon[31429]: pgmap v231: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:22.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:21 smithi090 ceph-mon[31429]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-27T07:37:22.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:21 smithi090 ceph-mon[31429]: osdmap e51: 8 total, 8 up, 8 in 2024-05-27T07:37:22.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:21 smithi136 ceph-mon[35324]: pgmap v231: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:22.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:21 smithi136 ceph-mon[35324]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-27T07:37:22.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:21 smithi136 ceph-mon[35324]: osdmap e51: 8 total, 8 up, 8 in 2024-05-27T07:37:22.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:21 smithi129 ceph-mon[35254]: pgmap v231: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:22.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:21 smithi129 ceph-mon[35254]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-27T07:37:22.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:21 smithi129 ceph-mon[35254]: osdmap e51: 8 total, 8 up, 8 in 2024-05-27T07:37:22.873 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_2 2024-05-27T07:37:23.298 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:37:23.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:23 smithi136 ceph-mon[35324]: pgmap v233: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:23.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:23.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:23.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:37:23.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:37:23.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:37:23.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:23.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:23.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:23.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:24.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:23 smithi090 ceph-mon[31429]: pgmap v233: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:24.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:24.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:24.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:37:24.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:37:24.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:37:24.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:24.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:24.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:24.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:24.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:23 smithi129 ceph-mon[35254]: pgmap v233: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:37:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:37:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:37:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:25.081 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:24 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi129 2024-05-27T07:37:25.081 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:24 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi129 to 5696M 2024-05-27T07:37:25.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:24 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi129 2024-05-27T07:37:25.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:24 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi129 to 5696M 2024-05-27T07:37:25.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:24 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi129 2024-05-27T07:37:25.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:24 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi129 to 5696M 2024-05-27T07:37:25.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.656+0000 7f8983000700 1 -- 172.21.15.136:0/1392043917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897c107790 msgr2=0x7f897c109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:25.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.656+0000 7f8983000700 1 --2- 172.21.15.136:0/1392043917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897c107790 0x7f897c109bb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8964009990 tx=0x7f8964009ca0 comp rx=0 tx=0).stop 2024-05-27T07:37:25.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.656+0000 7f8983000700 1 -- 172.21.15.136:0/1392043917 shutdown_connections 2024-05-27T07:37:25.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.656+0000 7f8983000700 1 --2- 172.21.15.136:0/1392043917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897c10a0f0 0x7f897c10c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:25.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.656+0000 7f8983000700 1 --2- 172.21.15.136:0/1392043917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f897c104e30 0x7f897c107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:25.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.656+0000 7f8983000700 1 --2- 172.21.15.136:0/1392043917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897c107790 0x7f897c109bb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:25.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.656+0000 7f8983000700 1 -- 172.21.15.136:0/1392043917 >> 172.21.15.136:0/1392043917 conn(0x7f897c0fe870 msgr2=0x7f897c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:25.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.656+0000 7f8983000700 1 -- 172.21.15.136:0/1392043917 shutdown_connections 2024-05-27T07:37:25.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.656+0000 7f8983000700 1 -- 172.21.15.136:0/1392043917 wait complete. 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.657+0000 7f8983000700 1 Processor -- start 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.657+0000 7f8983000700 1 -- start start 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f8983000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f897c104e30 0x7f897c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f8983000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897c107790 0x7f897c19d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f8983000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897c10a0f0 0x7f897c1a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f8983000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10fd00 con 0x7f897c104e30 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f8983000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10fb80 con 0x7f897c107790 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f8983000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f897c10fe80 con 0x7f897c10a0f0 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f8981000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897c10a0f0 0x7f897c1a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f8981000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897c10a0f0 0x7f897c1a3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:43194/0 (socket says 172.21.15.136:43194) 2024-05-27T07:37:25.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f8981000700 1 -- 172.21.15.136:0/1795181619 learned_addr learned my addr 172.21.15.136:0/1795181619 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-05-27T07:37:25.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f897be00700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f897c104e30 0x7f897c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:25.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.658+0000 7f8973400700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897c107790 0x7f897c19d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:25.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.659+0000 7f8981000700 1 -- 172.21.15.136:0/1795181619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897c107790 msgr2=0x7f897c19d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:25.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.659+0000 7f8981000700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897c107790 0x7f897c19d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:25.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.659+0000 7f8981000700 1 -- 172.21.15.136:0/1795181619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f897c104e30 msgr2=0x7f897c19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:25.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.659+0000 7f8981000700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f897c104e30 0x7f897c19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:25.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.659+0000 7f8981000700 1 -- 172.21.15.136:0/1795181619 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897c1a4670 con 0x7f897c10a0f0 2024-05-27T07:37:25.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.659+0000 7f8981000700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897c10a0f0 0x7f897c1a3ff0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f897400ada0 tx=0x7f897400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:25.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.659+0000 7f8979600700 1 -- 172.21.15.136:0/1795181619 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974017020 con 0x7f897c10a0f0 2024-05-27T07:37:25.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.660+0000 7f8979600700 1 -- 172.21.15.136:0/1795181619 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f897400cd60 con 0x7f897c10a0f0 2024-05-27T07:37:25.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.660+0000 7f8979600700 1 -- 172.21.15.136:0/1795181619 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974005280 con 0x7f897c10a0f0 2024-05-27T07:37:25.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.660+0000 7f8983000700 1 -- 172.21.15.136:0/1795181619 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897c1a4960 con 0x7f897c10a0f0 2024-05-27T07:37:25.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.660+0000 7f8983000700 1 -- 172.21.15.136:0/1795181619 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897c1a4ed0 con 0x7f897c10a0f0 2024-05-27T07:37:25.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.661+0000 7f8983000700 1 -- 172.21.15.136:0/1795181619 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897c1a5160 con 0x7f897c10a0f0 2024-05-27T07:37:25.660 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.663+0000 7f8979600700 1 -- 172.21.15.136:0/1795181619 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f897400c8d0 con 0x7f897c10a0f0 2024-05-27T07:37:25.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.665+0000 7f8979600700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89600b2880 0x7f89600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:37:25.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.665+0000 7f8979600700 1 -- 172.21.15.136:0/1795181619 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5397+0+0 (secure 0 0 0) 0x7f89740e0f10 con 0x7f897c10a0f0 2024-05-27T07:37:25.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.666+0000 7f897be00700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89600b2880 0x7f89600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:37:25.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.667+0000 7f897be00700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89600b2880 0x7f89600b4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f897c06c020 tx=0x7f896c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:37:25.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.670+0000 7f8979600700 1 -- 172.21.15.136:0/1795181619 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8974021020 con 0x7f897c10a0f0 2024-05-27T07:37:25.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:25.911+0000 7f8983000700 1 -- 172.21.15.136:0/1795181619 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f897c1034d0 con 0x7f89600b2880 2024-05-27T07:37:26.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:25 smithi090 ceph-mon[31429]: pgmap v234: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:26.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:37:26.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:37:26.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:26.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:25 smithi136 ceph-mon[35324]: pgmap v234: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:26.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:37:26.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:37:26.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:26.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:25 smithi129 ceph-mon[35254]: pgmap v234: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:26.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:37:26.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:37:26.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:27.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:26 smithi090 ceph-mon[31429]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:37:27.224 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:26 smithi136 ceph-mon[35324]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:37:27.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:26 smithi129 ceph-mon[35254]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:37:28.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:27 smithi090 ceph-mon[31429]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-27T07:37:28.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:27 smithi090 ceph-mon[31429]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-27T07:37:28.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:27 smithi090 ceph-mon[31429]: Cluster is now healthy 2024-05-27T07:37:28.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:27 smithi136 ceph-mon[35324]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-27T07:37:28.245 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:27 smithi136 ceph-mon[35324]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-27T07:37:28.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:27 smithi136 ceph-mon[35324]: Cluster is now healthy 2024-05-27T07:37:28.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:27 smithi129 ceph-mon[35254]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-27T07:37:28.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:27 smithi129 ceph-mon[35254]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-27T07:37:28.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:27 smithi129 ceph-mon[35254]: Cluster is now healthy 2024-05-27T07:37:29.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:28 smithi136 ceph-mon[35324]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-05-27T07:37:29.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:28 smithi129 ceph-mon[35254]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-05-27T07:37:29.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:28 smithi090 ceph-mon[31429]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-05-27T07:37:30.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:30 smithi090 ceph-mon[31429]: from='client.? 172.21.15.136:0/4205615845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4148e784-16f8-48b6-8521-ed7556044566"}]: dispatch 2024-05-27T07:37:30.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:30 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4148e784-16f8-48b6-8521-ed7556044566"}]: dispatch 2024-05-27T07:37:30.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:30 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4148e784-16f8-48b6-8521-ed7556044566"}]': finished 2024-05-27T07:37:30.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:30 smithi090 ceph-mon[31429]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T07:37:30.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:30 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:30.495 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:30 smithi136 ceph-mon[35324]: from='client.? 172.21.15.136:0/4205615845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4148e784-16f8-48b6-8521-ed7556044566"}]: dispatch 2024-05-27T07:37:30.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:30 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4148e784-16f8-48b6-8521-ed7556044566"}]: dispatch 2024-05-27T07:37:30.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:30 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4148e784-16f8-48b6-8521-ed7556044566"}]': finished 2024-05-27T07:37:30.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:30 smithi136 ceph-mon[35324]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T07:37:30.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:30 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:30.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:30 smithi129 ceph-mon[35254]: from='client.? 172.21.15.136:0/4205615845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4148e784-16f8-48b6-8521-ed7556044566"}]: dispatch 2024-05-27T07:37:30.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:30 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4148e784-16f8-48b6-8521-ed7556044566"}]: dispatch 2024-05-27T07:37:30.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:30 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4148e784-16f8-48b6-8521-ed7556044566"}]': finished 2024-05-27T07:37:30.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:30 smithi129 ceph-mon[35254]: osdmap e52: 9 total, 8 up, 9 in 2024-05-27T07:37:30.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:30 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:31.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:31 smithi090 ceph-mon[31429]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-27T07:37:31.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:31 smithi090 ceph-mon[31429]: from='client.? 172.21.15.136:0/4007657497' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:37:31.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:31 smithi136 ceph-mon[35324]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-27T07:37:31.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:31 smithi136 ceph-mon[35324]: from='client.? 172.21.15.136:0/4007657497' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:37:31.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:31 smithi129 ceph-mon[35254]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-27T07:37:31.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:31 smithi129 ceph-mon[35254]: from='client.? 172.21.15.136:0/4007657497' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:37:33.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:33 smithi090 ceph-mon[31429]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:37:33.734 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:33 smithi136 ceph-mon[35324]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:37:33.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:33 smithi129 ceph-mon[35254]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:37:35.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:35 smithi090 ceph-mon[31429]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:37:35.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:35 smithi136 ceph-mon[35324]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:37:35.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:35 smithi129 ceph-mon[35254]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-27T07:37:37.696 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:37 smithi136 ceph-mon[35324]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:37.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:37 smithi090 ceph-mon[31429]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:37.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:37 smithi129 ceph-mon[35254]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:39.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:39 smithi090 ceph-mon[31429]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:39.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:39 smithi136 ceph-mon[35324]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:39.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:39 smithi129 ceph-mon[35254]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:40.692 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T07:37:40.692 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:40.692 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:40 smithi136 ceph-mon[35324]: Deploying daemon osd.8 on smithi136 2024-05-27T07:37:40.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T07:37:40.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:40.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:40 smithi090 ceph-mon[31429]: Deploying daemon osd.8 on smithi136 2024-05-27T07:37:40.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T07:37:40.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:40.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:40 smithi129 ceph-mon[35254]: Deploying daemon osd.8 on smithi136 2024-05-27T07:37:41.558 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:41 smithi136 ceph-mon[35324]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:41.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:41 smithi090 ceph-mon[31429]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:41.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:41 smithi129 ceph-mon[35254]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:43.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:43 smithi090 ceph-mon[31429]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:43.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:43 smithi136 ceph-mon[35324]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:43.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:43 smithi129 ceph-mon[35254]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:45.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:45 smithi136 ceph-mon[35324]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:45.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:45 smithi090 ceph-mon[31429]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:45.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:45 smithi129 ceph-mon[35254]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:47.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:47 smithi090 ceph-mon[31429]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:47.711 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:47 smithi136 ceph-mon[35324]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:47.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:47 smithi129 ceph-mon[35254]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:49.863 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:49 smithi136 ceph-mon[35324]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:49.863 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:49 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:49.863 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:49 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:49.863 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:49 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:49.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:49 smithi090 ceph-mon[31429]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:49.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:49 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:49.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:49 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:49.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:49 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:50.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:49 smithi129 ceph-mon[35254]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:50.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:49 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:50.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:49 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:50.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:49 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:51.956 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:51 smithi136 ceph-mon[35324]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:51.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:51 smithi090 ceph-mon[31429]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:52.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:51 smithi129 ceph-mon[35254]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:53.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:53 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:53.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:53 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:53.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:53 smithi136 ceph-mon[35324]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:53.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:53 smithi136 ceph-mon[35324]: from='osd.8 [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:37:53.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:53 smithi136 ceph-mon[35324]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:37:53.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:53 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:53.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:53 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:53.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:53 smithi090 ceph-mon[31429]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:53.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:53 smithi090 ceph-mon[31429]: from='osd.8 [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:37:53.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:53 smithi090 ceph-mon[31429]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:37:53.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:53 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:53.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:53 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:53.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:53 smithi129 ceph-mon[35254]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:53.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:53 smithi129 ceph-mon[35254]: from='osd.8 [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:37:53.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:53 smithi129 ceph-mon[35254]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T07:37:54.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.097+0000 7f8979600700 1 -- 172.21.15.136:0/1795181619 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f897c1034d0 con 0x7f89600b2880 2024-05-27T07:37:54.094 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 8 on host 'smithi136' 2024-05-27T07:37:54.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 -- 172.21.15.136:0/1795181619 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89600b2880 msgr2=0x7f89600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89600b2880 0x7f89600b4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f897c06c020 tx=0x7f896c009190 comp rx=0 tx=0).stop 2024-05-27T07:37:54.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 -- 172.21.15.136:0/1795181619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897c10a0f0 msgr2=0x7f897c1a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:37:54.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897c10a0f0 0x7f897c1a3ff0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f897400ada0 tx=0x7f897400c330 comp rx=0 tx=0).stop 2024-05-27T07:37:54.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 -- 172.21.15.136:0/1795181619 shutdown_connections 2024-05-27T07:37:54.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89600b2880 0x7f89600b4d90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897c10a0f0 0x7f897c1a3ff0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f897c104e30 0x7f897c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 --2- 172.21.15.136:0/1795181619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897c107790 0x7f897c19d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:37:54.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 -- 172.21.15.136:0/1795181619 >> 172.21.15.136:0/1795181619 conn(0x7f897c0fe870 msgr2=0x7f897c102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:37:54.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 -- 172.21.15.136:0/1795181619 shutdown_connections 2024-05-27T07:37:54.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:37:54.101+0000 7f8983000700 1 -- 172.21.15.136:0/1795181619 wait complete. 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: from='osd.8 [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:54.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:54.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T07:37:54.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T07:37:54.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:54.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: from='osd.8 [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:37:54.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:37:54.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:54.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:54.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:54.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:54.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:54.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:54.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T07:37:54.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: osdmap e53: 9 total, 8 up, 9 in 2024-05-27T07:37:54.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:54.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: from='osd.8 [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:37:54.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:37:54.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:37:54.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:37:54.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:54.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:54.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:37:54.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:37:54.894 DEBUG:teuthology.orchestra.run.smithi136:osd.8> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.8.service 2024-05-27T07:37:54.899 INFO:tasks.cephadm:Deploying osd.9 on smithi090 with /dev/vg_nvme/lv_1... 2024-05-27T07:37:54.899 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T07:37:55.337 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:37:55.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:55 smithi090 ceph-mon[31429]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:37:55.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:55 smithi090 ceph-mon[31429]: osdmap e54: 9 total, 8 up, 9 in 2024-05-27T07:37:55.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:55 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:55.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:55 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:55.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:55 smithi090 ceph-mon[31429]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:55.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:55 smithi090 ceph-mon[31429]: osd.8 [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] boot 2024-05-27T07:37:55.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:55 smithi090 ceph-mon[31429]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T07:37:55.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:55 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:55.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:55 smithi136 ceph-mon[35324]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:37:55.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:55 smithi136 ceph-mon[35324]: osdmap e54: 9 total, 8 up, 9 in 2024-05-27T07:37:55.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:55 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:55.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:55 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:55.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:55 smithi136 ceph-mon[35324]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:55.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:55 smithi136 ceph-mon[35324]: osd.8 [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] boot 2024-05-27T07:37:55.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:55 smithi136 ceph-mon[35324]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T07:37:55.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:55 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:55.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:55 smithi129 ceph-mon[35254]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:37:55.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:55 smithi129 ceph-mon[35254]: osdmap e54: 9 total, 8 up, 9 in 2024-05-27T07:37:55.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:55 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:55.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:55 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:55.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:55 smithi129 ceph-mon[35254]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T07:37:55.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:55 smithi129 ceph-mon[35254]: osd.8 [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] boot 2024-05-27T07:37:55.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:55 smithi129 ceph-mon[35254]: osdmap e55: 9 total, 9 up, 9 in 2024-05-27T07:37:55.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:55 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:37:56.324 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:56 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:37:56.325 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:56 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:37:56.325 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:56 smithi136 ceph-mon[35324]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T07:37:56.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:56 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:37:56.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:56 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:37:56.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:56 smithi129 ceph-mon[35254]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T07:37:56.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:56 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:37:56.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:56 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:37:56.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:56 smithi090 ceph-mon[31429]: osdmap e56: 9 total, 9 up, 9 in 2024-05-27T07:37:57.262 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:57 smithi090 ceph-mon[31429]: pgmap v255: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:37:57.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:57 smithi136 ceph-mon[35324]: pgmap v255: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:37:57.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:57 smithi129 ceph-mon[35254]: pgmap v255: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:37:58.259 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T07:37:58.259 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T07:37:58.259 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-05-27T07:37:58.259 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-05-27T07:37:58.259 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0373026 s, 281 MB/s 2024-05-27T07:37:58.259 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-05-27T07:37:58.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:58 smithi136 ceph-mon[35324]: osdmap e57: 9 total, 9 up, 9 in 2024-05-27T07:37:58.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:58 smithi129 ceph-mon[35254]: osdmap e57: 9 total, 9 up, 9 in 2024-05-27T07:37:58.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:58 smithi090 ceph-mon[31429]: osdmap e57: 9 total, 9 up, 9 in 2024-05-27T07:37:59.028 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_1 2024-05-27T07:37:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:37:59.475 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:37:59 smithi136 ceph-mon[35324]: pgmap v257: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:37:59.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:37:59 smithi129 ceph-mon[35254]: pgmap v257: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:37:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:37:59 smithi090 ceph-mon[31429]: pgmap v257: 1 pgs: 1 peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:00.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi136 2024-05-27T07:38:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi136 to 5696M 2024-05-27T07:38:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:00.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi136 2024-05-27T07:38:00.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:00.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:00.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:00.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:00.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:00.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi136 to 5696M 2024-05-27T07:38:00.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:00.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:00.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:00.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:01.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi136 2024-05-27T07:38:01.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi136 to 5696M 2024-05-27T07:38:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:01.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.582+0000 7f266d400700 1 -- 172.21.15.90:0/2168267452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668102c20 msgr2=0x7f2668105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:01.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.582+0000 7f266d400700 1 --2- 172.21.15.90:0/2168267452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668102c20 0x7f2668105040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2658009990 tx=0x7f2658009ca0 comp rx=0 tx=0).stop 2024-05-27T07:38:01.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.582+0000 7f266d400700 1 -- 172.21.15.90:0/2168267452 shutdown_connections 2024-05-27T07:38:01.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.582+0000 7f266d400700 1 --2- 172.21.15.90:0/2168267452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2668107ee0 0x7f26681082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:01.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.582+0000 7f266d400700 1 --2- 172.21.15.90:0/2168267452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668102c20 0x7f2668105040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:01.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.582+0000 7f266d400700 1 --2- 172.21.15.90:0/2168267452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2668105580 0x7f26681079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:01.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.582+0000 7f266d400700 1 -- 172.21.15.90:0/2168267452 >> 172.21.15.90:0/2168267452 conn(0x7f26680fe870 msgr2=0x7f2668100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:01.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.582+0000 7f266d400700 1 -- 172.21.15.90:0/2168267452 shutdown_connections 2024-05-27T07:38:01.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.582+0000 7f266d400700 1 -- 172.21.15.90:0/2168267452 wait complete. 2024-05-27T07:38:01.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.583+0000 7f266d400700 1 Processor -- start 2024-05-27T07:38:01.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.584+0000 7f266d400700 1 -- start start 2024-05-27T07:38:01.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.584+0000 7f266d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668102c20 0x7f26681a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.584+0000 7f266d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2668105580 0x7f26681a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.584+0000 7f266d400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2668107ee0 0x7f26681a8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.584+0000 7f266d400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2668116210 con 0x7f2668102c20 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.584+0000 7f266d400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2668116090 con 0x7f2668107ee0 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.584+0000 7f266d400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2668116390 con 0x7f2668105580 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.584+0000 7f2666800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668102c20 0x7f26681a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.584+0000 7f2666800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668102c20 0x7f26681a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44438/0 (socket says 172.21.15.90:44438) 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.584+0000 7f2666800700 1 -- 172.21.15.90:0/728821406 learned_addr learned my addr 172.21.15.90:0/728821406 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.585+0000 7f265fe00700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2668105580 0x7f26681a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.585+0000 7f2667200700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2668107ee0 0x7f26681a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.585+0000 7f2666800700 1 -- 172.21.15.90:0/728821406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2668105580 msgr2=0x7f26681a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.585+0000 7f2666800700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2668105580 0x7f26681a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.585+0000 7f2666800700 1 -- 172.21.15.90:0/728821406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2668107ee0 msgr2=0x7f26681a8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.585+0000 7f2666800700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2668107ee0 0x7f26681a8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:01.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.585+0000 7f2666800700 1 -- 172.21.15.90:0/728821406 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26681a89a0 con 0x7f2668102c20 2024-05-27T07:38:01.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.585+0000 7f2666800700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668102c20 0x7f26681a0e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f26580098e0 tx=0x7f265800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:01.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.586+0000 7f265f400700 1 -- 172.21.15.90:0/728821406 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2658026070 con 0x7f2668102c20 2024-05-27T07:38:01.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.586+0000 7f265f400700 1 -- 172.21.15.90:0/728821406 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265800f460 con 0x7f2668102c20 2024-05-27T07:38:01.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.586+0000 7f266d400700 1 -- 172.21.15.90:0/728821406 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26681a8c30 con 0x7f2668102c20 2024-05-27T07:38:01.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.586+0000 7f265f400700 1 -- 172.21.15.90:0/728821406 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265800fe60 con 0x7f2668102c20 2024-05-27T07:38:01.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.586+0000 7f266d400700 1 -- 172.21.15.90:0/728821406 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26681a90e0 con 0x7f2668102c20 2024-05-27T07:38:01.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.587+0000 7f266d400700 1 -- 172.21.15.90:0/728821406 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26680512b0 con 0x7f2668102c20 2024-05-27T07:38:01.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.590+0000 7f265f400700 1 -- 172.21.15.90:0/728821406 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2658029070 con 0x7f2668102c20 2024-05-27T07:38:01.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.592+0000 7f265f400700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f264c0b2900 0x7f264c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:01.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.593+0000 7f265f400700 1 -- 172.21.15.90:0/728821406 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7f26580e2bd0 con 0x7f2668102c20 2024-05-27T07:38:01.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.593+0000 7f265fe00700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f264c0b2900 0x7f264c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:01.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.593+0000 7f265fe00700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f264c0b2900 0x7f264c0b4e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f26681a1ef0 tx=0x7f2650009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:01.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.595+0000 7f265f400700 1 -- 172.21.15.90:0/728821406 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2658030080 con 0x7f2668102c20 2024-05-27T07:38:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:01.836+0000 7f266d400700 1 -- 172.21.15.90:0/728821406 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2668063bf0 con 0x7f264c0b2900 2024-05-27T07:38:01.902 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:01 smithi136 ceph-mon[35324]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:38:01.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:01 smithi090 ceph-mon[31429]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:38:02.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:01 smithi129 ceph-mon[35254]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T07:38:02.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:02 smithi090 ceph-mon[31429]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:38:02.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:38:02.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:38:02.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:02 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:02.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:02 smithi136 ceph-mon[35324]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:38:02.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:38:02.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:38:02.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:02 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:03.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:02 smithi129 ceph-mon[35254]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:38:03.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:38:03.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:38:03.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:02 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:03.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:03 smithi090 ceph-mon[31429]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-27T07:38:03.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:03 smithi136 ceph-mon[35324]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-27T07:38:04.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:03 smithi129 ceph-mon[35254]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-27T07:38:05.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:05 smithi090 ceph-mon[31429]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-27T07:38:05.995 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:05 smithi136 ceph-mon[35324]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-27T07:38:06.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:05 smithi129 ceph-mon[35254]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-27T07:38:06.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:06 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3944594695' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14e386d1-ab82-4e8d-baa4-14d02137a192"}]: dispatch 2024-05-27T07:38:06.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:06 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3944594695' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14e386d1-ab82-4e8d-baa4-14d02137a192"}]': finished 2024-05-27T07:38:06.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:06 smithi090 ceph-mon[31429]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T07:38:06.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:06.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:06 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3944594695' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14e386d1-ab82-4e8d-baa4-14d02137a192"}]: dispatch 2024-05-27T07:38:06.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:06 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3944594695' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14e386d1-ab82-4e8d-baa4-14d02137a192"}]': finished 2024-05-27T07:38:06.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:06 smithi136 ceph-mon[35324]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T07:38:06.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:07.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:06 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3944594695' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14e386d1-ab82-4e8d-baa4-14d02137a192"}]: dispatch 2024-05-27T07:38:07.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:06 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3944594695' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14e386d1-ab82-4e8d-baa4-14d02137a192"}]': finished 2024-05-27T07:38:07.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:06 smithi129 ceph-mon[35254]: osdmap e58: 10 total, 9 up, 10 in 2024-05-27T07:38:07.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:06 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:07.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:07 smithi090 ceph-mon[31429]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-27T07:38:07.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:07 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1441030658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:38:08.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:07 smithi136 ceph-mon[35324]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-27T07:38:08.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:07 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1441030658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:38:08.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:07 smithi129 ceph-mon[35254]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-27T07:38:08.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:07 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1441030658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:38:09.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:09 smithi090 ceph-mon[31429]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:38:09.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:09 smithi136 ceph-mon[35324]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:38:10.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:09 smithi129 ceph-mon[35254]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:38:11.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:11 smithi090 ceph-mon[31429]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:11.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:11 smithi136 ceph-mon[35324]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:12.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:11 smithi129 ceph-mon[35254]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:13.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:13 smithi090 ceph-mon[31429]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:13.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:13 smithi136 ceph-mon[35324]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:14.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:13 smithi129 ceph-mon[35254]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:15.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:15 smithi090 ceph-mon[31429]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:15.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:15 smithi136 ceph-mon[35324]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:16.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:15 smithi129 ceph-mon[35254]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:16.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T07:38:16.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:16.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:16 smithi090 ceph-mon[31429]: Deploying daemon osd.9 on smithi090 2024-05-27T07:38:16.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:16 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T07:38:16.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:16 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:16.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:16 smithi136 ceph-mon[35324]: Deploying daemon osd.9 on smithi090 2024-05-27T07:38:17.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:16 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T07:38:17.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:16 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:17.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:16 smithi129 ceph-mon[35254]: Deploying daemon osd.9 on smithi090 2024-05-27T07:38:17.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:17 smithi090 ceph-mon[31429]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:17.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:17 smithi136 ceph-mon[35324]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:18.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:17 smithi129 ceph-mon[35254]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:19.904 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:19 smithi090 ceph-mon[31429]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:19.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:19 smithi136 ceph-mon[35324]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:20.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:19 smithi129 ceph-mon[35254]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:21.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:21 smithi090 ceph-mon[31429]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:21.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:21 smithi136 ceph-mon[35324]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:22.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:21 smithi129 ceph-mon[35254]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:23.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:23 smithi090 ceph-mon[31429]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:23.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:23 smithi136 ceph-mon[35324]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:24.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:23 smithi129 ceph-mon[35254]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:25.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:25 smithi136 ceph-mon[35324]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:25.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:25.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:38:25.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:25 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:26.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:25 smithi129 ceph-mon[35254]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:26.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:26.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:38:26.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:25 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:26.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:25 smithi090 ceph-mon[31429]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:26.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:26.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:38:26.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:25 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:27.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:27 smithi136 ceph-mon[35324]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:28.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:27 smithi129 ceph-mon[35254]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:28.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:27 smithi090 ceph-mon[31429]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:30.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:29 smithi090 ceph-mon[31429]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:30.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:30.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:29 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:30.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:29 smithi090 ceph-mon[31429]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T07:38:30.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:29 smithi136 ceph-mon[35324]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:30.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:30.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:29 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:30.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:29 smithi136 ceph-mon[35324]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T07:38:30.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:29 smithi129 ceph-mon[35254]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:30.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:30.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:29 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:30.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:29 smithi129 ceph-mon[35254]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T07:38:30.732 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 9 on host 'smithi090' 2024-05-27T07:38:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.735+0000 7f265f400700 1 -- 172.21.15.90:0/728821406 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2668063bf0 con 0x7f264c0b2900 2024-05-27T07:38:30.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 -- 172.21.15.90:0/728821406 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f264c0b2900 msgr2=0x7f264c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f264c0b2900 0x7f264c0b4e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f26681a1ef0 tx=0x7f2650009260 comp rx=0 tx=0).stop 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 -- 172.21.15.90:0/728821406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668102c20 msgr2=0x7f26681a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668102c20 0x7f26681a0e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f26580098e0 tx=0x7f265800ba90 comp rx=0 tx=0).stop 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 -- 172.21.15.90:0/728821406 shutdown_connections 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f264c0b2900 0x7f264c0b4e10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2668107ee0 0x7f26681a8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2668102c20 0x7f26681a0e40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 --2- 172.21.15.90:0/728821406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2668105580 0x7f26681a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 -- 172.21.15.90:0/728821406 >> 172.21.15.90:0/728821406 conn(0x7f26680fe870 msgr2=0x7f2668103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 -- 172.21.15.90:0/728821406 shutdown_connections 2024-05-27T07:38:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:38:30.739+0000 7f266d400700 1 -- 172.21.15.90:0/728821406 wait complete. 2024-05-27T07:38:31.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:30 smithi090 ceph-mon[31429]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T07:38:31.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:30 smithi090 ceph-mon[31429]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T07:38:31.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:30 smithi090 ceph-mon[31429]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:38:31.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:30 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:31.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:30 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:31.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:30 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:31.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:30 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:31.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:30 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:31.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:30 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:31.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:30 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:38:31.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:30 smithi136 ceph-mon[35324]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T07:38:31.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:30 smithi136 ceph-mon[35324]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T07:38:31.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:30 smithi136 ceph-mon[35324]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:38:31.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:30 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:31.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:30 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:31.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:30 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:31.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:30 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:31.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:30 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:31.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:30 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:31.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:30 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:38:31.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:30 smithi129 ceph-mon[35254]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T07:38:31.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:30 smithi129 ceph-mon[35254]: osdmap e59: 10 total, 9 up, 10 in 2024-05-27T07:38:31.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:30 smithi129 ceph-mon[35254]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-05-27T07:38:31.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:30 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:31.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:30 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:31.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:30 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:31.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:30 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:31.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:30 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:31.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:30 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:31.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:30 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:38:31.522 DEBUG:teuthology.orchestra.run.smithi090:osd.9> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.9.service 2024-05-27T07:38:31.529 INFO:tasks.cephadm:Deploying osd.10 on smithi129 with /dev/vg_nvme/lv_1... 2024-05-27T07:38:31.530 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T07:38:31.972 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:38:32.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:31 smithi090 ceph-mon[31429]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:32.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:31 smithi090 ceph-mon[31429]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:38:32.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:31 smithi090 ceph-mon[31429]: osdmap e60: 10 total, 9 up, 10 in 2024-05-27T07:38:32.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:31 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:32.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:31 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:32.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:31 smithi136 ceph-mon[35324]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:32.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:31 smithi136 ceph-mon[35324]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:38:32.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:31 smithi136 ceph-mon[35324]: osdmap e60: 10 total, 9 up, 10 in 2024-05-27T07:38:32.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:31 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:32.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:31 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:32.294 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:31 smithi129 ceph-mon[35254]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:32.295 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:31 smithi129 ceph-mon[35254]: from='osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-05-27T07:38:32.295 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:31 smithi129 ceph-mon[35254]: osdmap e60: 10 total, 9 up, 10 in 2024-05-27T07:38:32.295 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:31 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:32.295 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:31 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:33.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:32 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:38:33.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:32 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:38:33.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:32 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:33.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:32 smithi090 ceph-mon[31429]: osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] boot 2024-05-27T07:38:33.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:32 smithi090 ceph-mon[31429]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T07:38:33.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:32 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:33.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:32 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:38:33.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:32 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:38:33.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:32 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:33.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:32 smithi136 ceph-mon[35324]: osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] boot 2024-05-27T07:38:33.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:32 smithi136 ceph-mon[35324]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T07:38:33.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:32 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:33.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:32 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:38:33.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:32 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:38:33.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:32 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:33.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:32 smithi129 ceph-mon[35254]: osd.9 [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] boot 2024-05-27T07:38:33.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:32 smithi129 ceph-mon[35254]: osdmap e61: 10 total, 10 up, 10 in 2024-05-27T07:38:33.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:32 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:38:34.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:33 smithi129 ceph-mon[35254]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:34.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:33 smithi129 ceph-mon[35254]: osdmap e62: 10 total, 10 up, 10 in 2024-05-27T07:38:34.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:33 smithi090 ceph-mon[31429]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:34.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:33 smithi090 ceph-mon[31429]: osdmap e62: 10 total, 10 up, 10 in 2024-05-27T07:38:34.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:33 smithi136 ceph-mon[35324]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T07:38:34.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:33 smithi136 ceph-mon[35324]: osdmap e62: 10 total, 10 up, 10 in 2024-05-27T07:38:35.029 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T07:38:35.029 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T07:38:35.029 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-05-27T07:38:35.029 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-05-27T07:38:35.030 INFO:teuthology.orchestra.run.smithi129.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0340035 s, 308 MB/s 2024-05-27T07:38:35.030 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-05-27T07:38:35.971 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:35 smithi129 ceph-mon[35254]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:35.975 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_1 2024-05-27T07:38:36.194 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:35 smithi090 ceph-mon[31429]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:36.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:35 smithi136 ceph-mon[35324]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:36.405 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T07:38:37.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:37.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi090 2024-05-27T07:38:37.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi090 to 4272M 2024-05-27T07:38:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:37 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:37.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi090 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi090 to 4272M 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:37 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:37.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi090 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi090 to 4272M 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:38:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:37 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:38:38.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.832+0000 7fd835e00700 1 -- 172.21.15.129:0/2656387124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd830105e20 msgr2=0x7fd83010c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.832+0000 7fd835e00700 1 --2- 172.21.15.129:0/2656387124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd830105e20 0x7fd83010c6d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd828009990 tx=0x7fd828009ca0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.832+0000 7fd835e00700 1 -- 172.21.15.129:0/2656387124 shutdown_connections 2024-05-27T07:38:38.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.832+0000 7fd835e00700 1 --2- 172.21.15.129:0/2656387124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd830105e20 0x7fd83010c6d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.832+0000 7fd835e00700 1 --2- 172.21.15.129:0/2656387124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd830104350 0x7fd830104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.832+0000 7fd835e00700 1 --2- 172.21.15.129:0/2656387124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd830105480 0x7fd830105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.832+0000 7fd835e00700 1 -- 172.21.15.129:0/2656387124 >> 172.21.15.129:0/2656387124 conn(0x7fd8300ff9b0 msgr2=0x7fd830101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:38:38.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.833+0000 7fd835e00700 1 -- 172.21.15.129:0/2656387124 shutdown_connections 2024-05-27T07:38:38.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.833+0000 7fd835e00700 1 -- 172.21.15.129:0/2656387124 wait complete. 2024-05-27T07:38:38.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.834+0000 7fd835e00700 1 Processor -- start 2024-05-27T07:38:38.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.834+0000 7fd835e00700 1 -- start start 2024-05-27T07:38:38.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.834+0000 7fd835e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd830104350 0x7fd83019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.834+0000 7fd835e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd830105480 0x7fd83019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.834+0000 7fd835e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd830105e20 0x7fd8301a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.834+0000 7fd835e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010fcb0 con 0x7fd830104350 2024-05-27T07:38:38.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.834+0000 7fd835e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010fb30 con 0x7fd830105480 2024-05-27T07:38:38.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.834+0000 7fd835e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd83010fe30 con 0x7fd830105e20 2024-05-27T07:38:38.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.834+0000 7fd826000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd830105480 0x7fd83019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.835+0000 7fd826000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd830105480 0x7fd83019cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49282/0 (socket says 172.21.15.129:49282) 2024-05-27T07:38:38.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.835+0000 7fd826000700 1 -- 172.21.15.129:0/940472681 learned_addr learned my addr 172.21.15.129:0/940472681 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-05-27T07:38:38.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.835+0000 7fd82ea00700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd830104350 0x7fd83019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.835+0000 7fd82f400700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd830105e20 0x7fd8301a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.835+0000 7fd826000700 1 -- 172.21.15.129:0/940472681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd830105e20 msgr2=0x7fd8301a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.835+0000 7fd826000700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd830105e20 0x7fd8301a3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.835+0000 7fd826000700 1 -- 172.21.15.129:0/940472681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd830104350 msgr2=0x7fd83019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:38:38.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.835+0000 7fd826000700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd830104350 0x7fd83019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:38:38.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.835+0000 7fd826000700 1 -- 172.21.15.129:0/940472681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8301a4510 con 0x7fd830105480 2024-05-27T07:38:38.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.835+0000 7fd826000700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd830105480 0x7fd83019cf80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd8180060b0 tx=0x7fd818007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:38.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.836+0000 7fd834a00700 1 -- 172.21.15.129:0/940472681 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81801a070 con 0x7fd830105480 2024-05-27T07:38:38.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.836+0000 7fd834a00700 1 -- 172.21.15.129:0/940472681 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd81800de20 con 0x7fd830105480 2024-05-27T07:38:38.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.836+0000 7fd835e00700 1 -- 172.21.15.129:0/940472681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8301a4800 con 0x7fd830105480 2024-05-27T07:38:38.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.836+0000 7fd834a00700 1 -- 172.21.15.129:0/940472681 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818014690 con 0x7fd830105480 2024-05-27T07:38:38.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.836+0000 7fd835e00700 1 -- 172.21.15.129:0/940472681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301a4d40 con 0x7fd830105480 2024-05-27T07:38:38.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.837+0000 7fd835e00700 1 -- 172.21.15.129:0/940472681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8300512b0 con 0x7fd830105480 2024-05-27T07:38:38.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.840+0000 7fd834a00700 1 -- 172.21.15.129:0/940472681 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd81801d070 con 0x7fd830105480 2024-05-27T07:38:38.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.841+0000 7fd834a00700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8140b2880 0x7fd8140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:38:38.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.841+0000 7fd834a00700 1 -- 172.21.15.129:0/940472681 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7fd8180e1870 con 0x7fd830105480 2024-05-27T07:38:38.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.842+0000 7fd82ea00700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8140b2880 0x7fd8140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:38:38.840 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.843+0000 7fd82ea00700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8140b2880 0x7fd8140b4d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd820005fd0 tx=0x7fd820005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:38:38.841 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:38.844+0000 7fd834a00700 1 -- 172.21.15.129:0/940472681 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd818010070 con 0x7fd830105480 2024-05-27T07:38:39.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:38:39.079+0000 7fd835e00700 1 -- 172.21.15.129:0/940472681 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd830077100 con 0x7fd8140b2880 2024-05-27T07:38:39.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:39 smithi090 ceph-mon[31429]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:39.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:38:39.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:38:39.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:39 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:39.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:39 smithi136 ceph-mon[35324]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:39.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:38:39.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:38:39.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:39 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:39.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:39 smithi129 ceph-mon[35254]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:39.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:38:39.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:38:39.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:39 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:40.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:40 smithi090 ceph-mon[31429]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:38:40.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:40 smithi136 ceph-mon[35324]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:38:40.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:40 smithi129 ceph-mon[35254]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:38:41.560 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:41 smithi129 ceph-mon[35254]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:41.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:41 smithi090 ceph-mon[31429]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:41.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:41 smithi136 ceph-mon[35324]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:43.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:43 smithi090 ceph-mon[31429]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:43.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:43 smithi090 ceph-mon[31429]: from='client.? 172.21.15.129:0/4030209633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6"}]: dispatch 2024-05-27T07:38:43.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:43 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6"}]: dispatch 2024-05-27T07:38:43.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:43 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6"}]': finished 2024-05-27T07:38:43.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:43 smithi090 ceph-mon[31429]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T07:38:43.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:43 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:38:43.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:43 smithi136 ceph-mon[35324]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:43.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:43 smithi136 ceph-mon[35324]: from='client.? 172.21.15.129:0/4030209633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6"}]: dispatch 2024-05-27T07:38:43.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:43 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6"}]: dispatch 2024-05-27T07:38:43.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:43 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6"}]': finished 2024-05-27T07:38:43.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:43 smithi136 ceph-mon[35324]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T07:38:43.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:43 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:38:43.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:43 smithi129 ceph-mon[35254]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:43.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:43 smithi129 ceph-mon[35254]: from='client.? 172.21.15.129:0/4030209633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6"}]: dispatch 2024-05-27T07:38:43.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:43 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6"}]: dispatch 2024-05-27T07:38:43.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:43 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6"}]': finished 2024-05-27T07:38:43.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:43 smithi129 ceph-mon[35254]: osdmap e63: 11 total, 10 up, 11 in 2024-05-27T07:38:43.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:43 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:38:44.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:44 smithi090 ceph-mon[31429]: from='client.? 172.21.15.129:0/2283442486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:38:44.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:44 smithi136 ceph-mon[35324]: from='client.? 172.21.15.129:0/2283442486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:38:44.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:44 smithi129 ceph-mon[35254]: from='client.? 172.21.15.129:0/2283442486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:38:45.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:45 smithi090 ceph-mon[31429]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:45.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:45 smithi136 ceph-mon[35324]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:45.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:45 smithi129 ceph-mon[35254]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:47.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:47 smithi090 ceph-mon[31429]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:47.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:47 smithi136 ceph-mon[35324]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:47.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:47 smithi129 ceph-mon[35254]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:49.678 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:49 smithi129 ceph-mon[35254]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:49.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:49 smithi090 ceph-mon[31429]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:49.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:49 smithi136 ceph-mon[35324]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:51.481 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:51 smithi129 ceph-mon[35254]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:51.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:51 smithi090 ceph-mon[31429]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:51.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:51 smithi136 ceph-mon[35324]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:53.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:53 smithi090 ceph-mon[31429]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:53.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:53 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T07:38:53.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:53 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:53.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:53 smithi136 ceph-mon[35324]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:53.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:53 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T07:38:53.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:53 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:53.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:53 smithi129 ceph-mon[35254]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:53.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:53 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T07:38:53.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:53 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:38:54.519 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:54 smithi129 ceph-mon[35254]: Deploying daemon osd.10 on smithi129 2024-05-27T07:38:54.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:54 smithi090 ceph-mon[31429]: Deploying daemon osd.10 on smithi129 2024-05-27T07:38:54.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:54 smithi136 ceph-mon[35324]: Deploying daemon osd.10 on smithi129 2024-05-27T07:38:55.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:55 smithi090 ceph-mon[31429]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:55.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:55 smithi136 ceph-mon[35324]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:55.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:55 smithi129 ceph-mon[35254]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:57.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:57 smithi090 ceph-mon[31429]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:57.709 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:57 smithi129 ceph-mon[35254]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:57.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:57 smithi136 ceph-mon[35324]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:59.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:38:59 smithi090 ceph-mon[31429]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:59.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:38:59 smithi136 ceph-mon[35324]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:38:59.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:38:59 smithi129 ceph-mon[35254]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:01.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:01 smithi090 ceph-mon[31429]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:01.711 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:01 smithi129 ceph-mon[35254]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:01.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:01 smithi136 ceph-mon[35324]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:03.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:03 smithi129 ceph-mon[35254]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:03.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:03.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:03.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:03 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:03.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:03 smithi090 ceph-mon[31429]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:03.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:03.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:03.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:03 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:03.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:03 smithi136 ceph-mon[35324]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:03.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:03.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:03.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:03 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:05.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:05 smithi129 ceph-mon[35254]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:05.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:05 smithi090 ceph-mon[31429]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:05.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:05 smithi136 ceph-mon[35324]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:07.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:07 smithi129 ceph-mon[35254]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:07.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:07 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:07.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:07 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:07.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:07 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:07.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:07 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:07.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:07 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:07.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:07 smithi129 ceph-mon[35254]: from='osd.10 [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:39:07.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:07 smithi129 ceph-mon[35254]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:39:07.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:07 smithi090 ceph-mon[31429]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:07.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:07.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:07.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:07.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:07.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:07.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:07 smithi090 ceph-mon[31429]: from='osd.10 [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:39:07.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:07 smithi090 ceph-mon[31429]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:39:07.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:07 smithi136 ceph-mon[35324]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:07.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:07.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:07.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:07.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:07.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:07.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:07 smithi136 ceph-mon[35324]: from='osd.10 [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:39:07.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:07 smithi136 ceph-mon[35324]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T07:39:08.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.363+0000 7fd834a00700 1 -- 172.21.15.129:0/940472681 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd830077100 con 0x7fd8140b2880 2024-05-27T07:39:08.362 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 10 on host 'smithi129' 2024-05-27T07:39:08.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 -- 172.21.15.129:0/940472681 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8140b2880 msgr2=0x7fd8140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:08.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8140b2880 0x7fd8140b4d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd820005fd0 tx=0x7fd820005e50 comp rx=0 tx=0).stop 2024-05-27T07:39:08.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 -- 172.21.15.129:0/940472681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd830105480 msgr2=0x7fd83019cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:08.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd830105480 0x7fd83019cf80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd8180060b0 tx=0x7fd818007f90 comp rx=0 tx=0).stop 2024-05-27T07:39:08.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 -- 172.21.15.129:0/940472681 shutdown_connections 2024-05-27T07:39:08.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8140b2880 0x7fd8140b4d90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:08.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd830105e20 0x7fd8301a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:08.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd830104350 0x7fd83019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:08.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 --2- 172.21.15.129:0/940472681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd830105480 0x7fd83019cf80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:08.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 -- 172.21.15.129:0/940472681 >> 172.21.15.129:0/940472681 conn(0x7fd8300ff9b0 msgr2=0x7fd8300759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:08.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 -- 172.21.15.129:0/940472681 shutdown_connections 2024-05-27T07:39:08.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-05-27T07:39:08.368+0000 7fd835e00700 1 -- 172.21.15.129:0/940472681 wait complete. 2024-05-27T07:39:08.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:08 smithi090 ceph-mon[31429]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T07:39:08.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:08 smithi090 ceph-mon[31429]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T07:39:08.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:08 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:08.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:08 smithi090 ceph-mon[31429]: from='osd.10 [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:39:08.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:08 smithi090 ceph-mon[31429]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:39:08.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:08 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:08.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:08 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:08.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:08 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:08.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:08 smithi136 ceph-mon[35324]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T07:39:08.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:08 smithi136 ceph-mon[35324]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T07:39:08.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:08 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:08.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:08 smithi136 ceph-mon[35324]: from='osd.10 [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:39:08.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:08 smithi136 ceph-mon[35324]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:39:08.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:08 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:08.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:08 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:08.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:08 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:09.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:08 smithi129 ceph-mon[35254]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T07:39:09.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:08 smithi129 ceph-mon[35254]: osdmap e64: 11 total, 10 up, 11 in 2024-05-27T07:39:09.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:08 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:09.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:08 smithi129 ceph-mon[35254]: from='osd.10 [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:39:09.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:08 smithi129 ceph-mon[35254]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-05-27T07:39:09.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:08 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:09.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:08 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:09.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:08 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:09.227 DEBUG:teuthology.orchestra.run.smithi129:osd.10> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.10.service 2024-05-27T07:39:09.231 INFO:tasks.cephadm:Deploying osd.11 on smithi136 with /dev/vg_nvme/lv_1... 2024-05-27T07:39:09.231 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T07:39:09.670 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:39:09.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:09 smithi129 ceph-mon[35254]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:09.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:09 smithi129 ceph-mon[35254]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:39:09.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:09 smithi129 ceph-mon[35254]: osdmap e65: 11 total, 10 up, 11 in 2024-05-27T07:39:09.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:09.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:09.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:09 smithi090 ceph-mon[31429]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:09.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:09 smithi090 ceph-mon[31429]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:39:09.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:09 smithi090 ceph-mon[31429]: osdmap e65: 11 total, 10 up, 11 in 2024-05-27T07:39:09.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:09.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:09.987 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:09 smithi136 ceph-mon[35324]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-27T07:39:09.987 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:09 smithi136 ceph-mon[35324]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-05-27T07:39:09.987 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:09 smithi136 ceph-mon[35324]: osdmap e65: 11 total, 10 up, 11 in 2024-05-27T07:39:09.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:09.988 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:10.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:10 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:39:10.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:10 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:39:10.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:10 smithi090 ceph-mon[31429]: osd.10 [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] boot 2024-05-27T07:39:10.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:10 smithi090 ceph-mon[31429]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T07:39:10.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:10 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:10.981 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:10 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:39:10.981 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:10 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:39:10.981 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:10 smithi129 ceph-mon[35254]: osd.10 [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] boot 2024-05-27T07:39:10.981 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:10 smithi129 ceph-mon[35254]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T07:39:10.981 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:10 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:10.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:10 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:39:10.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:10 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:39:10.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:10 smithi136 ceph-mon[35324]: osd.10 [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] boot 2024-05-27T07:39:10.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:10 smithi136 ceph-mon[35324]: osdmap e66: 11 total, 11 up, 11 in 2024-05-27T07:39:10.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:10 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:39:11.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:11 smithi090 ceph-mon[31429]: pgmap v302: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:11.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:11 smithi090 ceph-mon[31429]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T07:39:11.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:11 smithi136 ceph-mon[35324]: pgmap v302: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:11.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:11 smithi136 ceph-mon[35324]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T07:39:12.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:11 smithi129 ceph-mon[35254]: pgmap v302: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:12.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:11 smithi129 ceph-mon[35254]: osdmap e67: 11 total, 11 up, 11 in 2024-05-27T07:39:12.644 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T07:39:12.644 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T07:39:12.644 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-05-27T07:39:12.644 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-05-27T07:39:12.644 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0361017 s, 290 MB/s 2024-05-27T07:39:12.644 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-05-27T07:39:12.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:12 smithi090 ceph-mon[31429]: osdmap e68: 11 total, 11 up, 11 in 2024-05-27T07:39:12.996 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:12 smithi129 ceph-mon[35254]: osdmap e68: 11 total, 11 up, 11 in 2024-05-27T07:39:12.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:12 smithi136 ceph-mon[35324]: osdmap e68: 11 total, 11 up, 11 in 2024-05-27T07:39:13.584 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_1 2024-05-27T07:39:13.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:13 smithi136 ceph-mon[35324]: pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:14.027 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:39:14.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:13 smithi129 ceph-mon[35254]: pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:14.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:13 smithi090 ceph-mon[31429]: pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:15.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi129 2024-05-27T07:39:15.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi129 to 4272M 2024-05-27T07:39:15.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:15.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:15.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:14 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.214 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi129 2024-05-27T07:39:15.214 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.214 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.214 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.215 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.215 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.215 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.215 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi129 to 4272M 2024-05-27T07:39:15.215 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.215 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:15.215 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:15.215 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:14 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi129 2024-05-27T07:39:15.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:15.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi129 to 4272M 2024-05-27T07:39:15.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:15.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:15.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:15.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:14 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:16.115 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:15 smithi136 ceph-mon[35324]: pgmap v306: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:16.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:15 smithi090 ceph-mon[31429]: pgmap v306: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:16.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:15 smithi129 ceph-mon[35254]: pgmap v306: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:16.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.365+0000 7fca27200700 1 -- 172.21.15.136:0/1576232203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca2810bff0 msgr2=0x7fca2810e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:16.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.365+0000 7fca27200700 1 --2- 172.21.15.136:0/1576232203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca2810bff0 0x7fca2810e500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fca2000b0a0 tx=0x7fca2000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:39:16.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.365+0000 7fca27200700 1 -- 172.21.15.136:0/1576232203 shutdown_connections 2024-05-27T07:39:16.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.365+0000 7fca27200700 1 --2- 172.21.15.136:0/1576232203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca2810bff0 0x7fca2810e500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:16.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.365+0000 7fca27200700 1 --2- 172.21.15.136:0/1576232203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca280776d0 0x7fca28075ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:16.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.365+0000 7fca27200700 1 --2- 172.21.15.136:0/1576232203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca280760e0 0x7fca28076570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:16.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.365+0000 7fca27200700 1 -- 172.21.15.136:0/1576232203 >> 172.21.15.136:0/1576232203 conn(0x7fca280ff7a0 msgr2=0x7fca28101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:16.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.365+0000 7fca27200700 1 -- 172.21.15.136:0/1576232203 shutdown_connections 2024-05-27T07:39:16.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.365+0000 7fca27200700 1 -- 172.21.15.136:0/1576232203 wait complete. 2024-05-27T07:39:16.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.367+0000 7fca27200700 1 Processor -- start 2024-05-27T07:39:16.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.367+0000 7fca27200700 1 -- start start 2024-05-27T07:39:16.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.367+0000 7fca27200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca280760e0 0x7fca2819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:16.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.367+0000 7fca27200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca280776d0 0x7fca2819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:16.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.367+0000 7fca27200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca2810bff0 0x7fca281a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:16.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca27200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca28111d20 con 0x7fca280760e0 2024-05-27T07:39:16.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca27200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fca28111ba0 con 0x7fca280776d0 2024-05-27T07:39:16.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca27200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fca28111ea0 con 0x7fca2810bff0 2024-05-27T07:39:16.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca26800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca2810bff0 0x7fca281a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:16.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca26800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca2810bff0 0x7fca281a3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:34558/0 (socket says 172.21.15.136:34558) 2024-05-27T07:39:16.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca26800700 1 -- 172.21.15.136:0/3821748398 learned_addr learned my addr 172.21.15.136:0/3821748398 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-05-27T07:39:16.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca25400700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca280776d0 0x7fca2819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:16.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca25e00700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca280760e0 0x7fca2819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:16.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca26800700 1 -- 172.21.15.136:0/3821748398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca280776d0 msgr2=0x7fca2819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:16.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca26800700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca280776d0 0x7fca2819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:16.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca26800700 1 -- 172.21.15.136:0/3821748398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca280760e0 msgr2=0x7fca2819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:16.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca26800700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca280760e0 0x7fca2819c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:16.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca26800700 1 -- 172.21.15.136:0/3821748398 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca281a4370 con 0x7fca2810bff0 2024-05-27T07:39:16.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.368+0000 7fca25400700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca280776d0 0x7fca2819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:16.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.369+0000 7fca26800700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca2810bff0 0x7fca281a3d80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca200062a0 tx=0x7fca2000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:16.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.369+0000 7fca1ea00700 1 -- 172.21.15.136:0/3821748398 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2001f030 con 0x7fca2810bff0 2024-05-27T07:39:16.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.369+0000 7fca27200700 1 -- 172.21.15.136:0/3821748398 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca281a4600 con 0x7fca2810bff0 2024-05-27T07:39:16.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.369+0000 7fca1ea00700 1 -- 172.21.15.136:0/3821748398 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca20007ac0 con 0x7fca2810bff0 2024-05-27T07:39:16.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.369+0000 7fca1ea00700 1 -- 172.21.15.136:0/3821748398 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20004840 con 0x7fca2810bff0 2024-05-27T07:39:16.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.369+0000 7fca27200700 1 -- 172.21.15.136:0/3821748398 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca281a4b80 con 0x7fca2810bff0 2024-05-27T07:39:16.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.371+0000 7fca27200700 1 -- 172.21.15.136:0/3821748398 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca280512b0 con 0x7fca2810bff0 2024-05-27T07:39:16.373 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.376+0000 7fca1ea00700 1 -- 172.21.15.136:0/3821748398 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fca20003a00 con 0x7fca2810bff0 2024-05-27T07:39:16.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.377+0000 7fca1ea00700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fca0c0b2900 0x7fca0c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:16.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.378+0000 7fca1ea00700 1 -- 172.21.15.136:0/3821748398 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7fca2000f020 con 0x7fca2810bff0 2024-05-27T07:39:16.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.378+0000 7fca25e00700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fca0c0b2900 0x7fca0c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:16.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.379+0000 7fca25e00700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fca0c0b2900 0x7fca0c0b4e10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fca18006fd0 tx=0x7fca18008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:16.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.380+0000 7fca1ea00700 1 -- 172.21.15.136:0/3821748398 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca200ab470 con 0x7fca2810bff0 2024-05-27T07:39:16.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:16.623+0000 7fca27200700 1 -- 172.21.15.136:0/3821748398 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fca28063bf0 con 0x7fca0c0b2900 2024-05-27T07:39:17.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:39:17.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:39:17.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:16 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:17.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:16 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:39:17.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:16 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:39:17.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:16 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:17.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:16 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T07:39:17.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:16 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T07:39:17.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:16 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:17 smithi090 ceph-mon[31429]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-27T07:39:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:17 smithi090 ceph-mon[31429]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:39:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:17 smithi136 ceph-mon[35324]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-27T07:39:18.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:17 smithi136 ceph-mon[35324]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:39:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:17 smithi129 ceph-mon[35254]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-27T07:39:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:17 smithi129 ceph-mon[35254]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:39:20.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:19 smithi090 ceph-mon[31429]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T07:39:20.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:19 smithi136 ceph-mon[35324]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T07:39:20.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:19 smithi129 ceph-mon[35254]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T07:39:21.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:20 smithi090 ceph-mon[31429]: from='client.? 172.21.15.136:0/4281636028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d748f6bf-b26a-4a17-9b1b-51e78f7d5708"}]: dispatch 2024-05-27T07:39:21.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:20 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d748f6bf-b26a-4a17-9b1b-51e78f7d5708"}]: dispatch 2024-05-27T07:39:21.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:20 smithi090 ceph-mon[31429]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d748f6bf-b26a-4a17-9b1b-51e78f7d5708"}]': finished 2024-05-27T07:39:21.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:20 smithi090 ceph-mon[31429]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T07:39:21.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:21.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:20 smithi136 ceph-mon[35324]: from='client.? 172.21.15.136:0/4281636028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d748f6bf-b26a-4a17-9b1b-51e78f7d5708"}]: dispatch 2024-05-27T07:39:21.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:20 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d748f6bf-b26a-4a17-9b1b-51e78f7d5708"}]: dispatch 2024-05-27T07:39:21.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:20 smithi136 ceph-mon[35324]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d748f6bf-b26a-4a17-9b1b-51e78f7d5708"}]': finished 2024-05-27T07:39:21.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:20 smithi136 ceph-mon[35324]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T07:39:21.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:21.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:20 smithi129 ceph-mon[35254]: from='client.? 172.21.15.136:0/4281636028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d748f6bf-b26a-4a17-9b1b-51e78f7d5708"}]: dispatch 2024-05-27T07:39:21.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:20 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d748f6bf-b26a-4a17-9b1b-51e78f7d5708"}]: dispatch 2024-05-27T07:39:21.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:20 smithi129 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d748f6bf-b26a-4a17-9b1b-51e78f7d5708"}]': finished 2024-05-27T07:39:21.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:20 smithi129 ceph-mon[35254]: osdmap e69: 12 total, 11 up, 12 in 2024-05-27T07:39:21.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:22.060 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:21 smithi129 ceph-mon[35254]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T07:39:22.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:21 smithi129 ceph-mon[35254]: from='client.? 172.21.15.136:0/1690138965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:39:22.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:21 smithi090 ceph-mon[31429]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T07:39:22.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:21 smithi090 ceph-mon[31429]: from='client.? 172.21.15.136:0/1690138965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:39:22.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:21 smithi136 ceph-mon[35324]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T07:39:22.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:21 smithi136 ceph-mon[35324]: from='client.? 172.21.15.136:0/1690138965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T07:39:23.310 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:23 smithi129 ceph-mon[35254]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:39:23.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:23 smithi090 ceph-mon[31429]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:39:23.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:23 smithi136 ceph-mon[35324]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:39:25.651 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:25 smithi136 ceph-mon[35324]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:39:25.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:25 smithi090 ceph-mon[31429]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:39:25.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:25 smithi129 ceph-mon[35254]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T07:39:27.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:27 smithi090 ceph-mon[31429]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:27.726 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:27 smithi136 ceph-mon[35324]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:27.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:27 smithi129 ceph-mon[35254]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:29.477 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:29 smithi136 ceph-mon[35324]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:29.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:29 smithi090 ceph-mon[31429]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:29.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:29 smithi129 ceph-mon[35254]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:31.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:31 smithi129 ceph-mon[35254]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:31.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:31 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T07:39:31.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:31 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:31.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:31 smithi129 ceph-mon[35254]: Deploying daemon osd.11 on smithi136 2024-05-27T07:39:31.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:31 smithi090 ceph-mon[31429]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:31.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:31 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T07:39:31.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:31 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:31.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:31 smithi090 ceph-mon[31429]: Deploying daemon osd.11 on smithi136 2024-05-27T07:39:31.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:31 smithi136 ceph-mon[35324]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:31.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:31 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T07:39:31.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:31 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:31.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:31 smithi136 ceph-mon[35324]: Deploying daemon osd.11 on smithi136 2024-05-27T07:39:33.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:33 smithi136 ceph-mon[35324]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:33.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:33 smithi090 ceph-mon[31429]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:33.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:33 smithi129 ceph-mon[35254]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:35.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:35 smithi090 ceph-mon[31429]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:35.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:35 smithi136 ceph-mon[35324]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:35.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:35 smithi129 ceph-mon[35254]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:37.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:37 smithi090 ceph-mon[31429]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:37.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:37 smithi136 ceph-mon[35324]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:37.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:37 smithi129 ceph-mon[35254]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:39.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:39 smithi090 ceph-mon[31429]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:39.731 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:39 smithi136 ceph-mon[35324]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:39.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:39 smithi129 ceph-mon[35254]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:40.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:40.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:40.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:40 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:40.711 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:40.711 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:40.711 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:40 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:40.810 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:40.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:40.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:40 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:41.469 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:41 smithi136 ceph-mon[35324]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:41.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:41 smithi090 ceph-mon[31429]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:41.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:41 smithi129 ceph-mon[35254]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:43.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:43 smithi090 ceph-mon[31429]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:43.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:43 smithi136 ceph-mon[35324]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:43.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:43 smithi129 ceph-mon[35254]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:45.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:45.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:45.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:45.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:45.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:45.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:45 smithi136 ceph-mon[35324]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:45.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:45 smithi136 ceph-mon[35324]: from='osd.11 [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T07:39:45.167 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:45 smithi136 ceph-mon[35324]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T07:39:45.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:45.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:45.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:45.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:45.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:45.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:45 smithi090 ceph-mon[31429]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:45.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:45 smithi090 ceph-mon[31429]: from='osd.11 [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T07:39:45.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:45 smithi090 ceph-mon[31429]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T07:39:45.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:45.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:45.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:45.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:45.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:45.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:45 smithi129 ceph-mon[35254]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:45.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:45 smithi129 ceph-mon[35254]: from='osd.11 [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T07:39:45.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:45 smithi129 ceph-mon[35254]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T07:39:45.902 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 11 on host 'smithi136' 2024-05-27T07:39:46.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.902+0000 7fca1ea00700 1 -- 172.21.15.136:0/3821748398 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fca28063bf0 con 0x7fca0c0b2900 2024-05-27T07:39:46.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 -- 172.21.15.136:0/3821748398 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fca0c0b2900 msgr2=0x7fca0c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fca0c0b2900 0x7fca0c0b4e10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fca18006fd0 tx=0x7fca18008040 comp rx=0 tx=0).stop 2024-05-27T07:39:46.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 -- 172.21.15.136:0/3821748398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca2810bff0 msgr2=0x7fca281a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:46.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca2810bff0 0x7fca281a3d80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca200062a0 tx=0x7fca2000bd70 comp rx=0 tx=0).stop 2024-05-27T07:39:46.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 -- 172.21.15.136:0/3821748398 shutdown_connections 2024-05-27T07:39:46.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fca0c0b2900 0x7fca0c0b4e10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca2810bff0 0x7fca281a3d80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca280760e0 0x7fca2819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 --2- 172.21.15.136:0/3821748398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca280776d0 0x7fca2819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:46.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 -- 172.21.15.136:0/3821748398 >> 172.21.15.136:0/3821748398 conn(0x7fca280ff7a0 msgr2=0x7fca28107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:46.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 -- 172.21.15.136:0/3821748398 shutdown_connections 2024-05-27T07:39:46.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:39:45.907+0000 7fca27200700 1 -- 172.21.15.136:0/3821748398 wait complete. 2024-05-27T07:39:46.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:46 smithi090 ceph-mon[31429]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T07:39:46.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:46 smithi090 ceph-mon[31429]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T07:39:46.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:46 smithi090 ceph-mon[31429]: from='osd.11 [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:39:46.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:46 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:46.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:46 smithi090 ceph-mon[31429]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:39:46.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:46 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:46.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:46 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:46.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:46 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:46.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:46 smithi136 ceph-mon[35324]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T07:39:46.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:46 smithi136 ceph-mon[35324]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T07:39:46.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:46 smithi136 ceph-mon[35324]: from='osd.11 [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:39:46.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:46 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:46.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:46 smithi136 ceph-mon[35324]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:39:46.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:46 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:46.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:46 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:46.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:46 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:46.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:46 smithi129 ceph-mon[35254]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T07:39:46.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:46 smithi129 ceph-mon[35254]: osdmap e70: 12 total, 11 up, 12 in 2024-05-27T07:39:46.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:46 smithi129 ceph-mon[35254]: from='osd.11 [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:39:46.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:46 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:46.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:46 smithi129 ceph-mon[35254]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-05-27T07:39:46.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:46 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:46.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:46 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:46.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:46 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:39:46.706 DEBUG:teuthology.orchestra.run.smithi136:osd.11> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.11.service 2024-05-27T07:39:46.713 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-27T07:39:46.713 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-05-27T07:39:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:39:47.216 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:47 smithi136 ceph-mon[35324]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:39:47.216 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:47 smithi136 ceph-mon[35324]: osdmap e71: 12 total, 11 up, 12 in 2024-05-27T07:39:47.216 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:47.216 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:47 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:47.216 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:47 smithi136 ceph-mon[35324]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:47.216 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:47 smithi136 ceph-mon[35324]: from='osd.11 ' entity='osd.11' 2024-05-27T07:39:47.217 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 07:39:46 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T07:39:46.947+0000 7fbb4b200700 -1 osd.11 0 waiting for initial osdmap 2024-05-27T07:39:47.217 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 07:39:46 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T07:39:46.977+0000 7fbb41000700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T07:39:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:47 smithi090 ceph-mon[31429]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:39:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:47 smithi090 ceph-mon[31429]: osdmap e71: 12 total, 11 up, 12 in 2024-05-27T07:39:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:47 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:47.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:47 smithi090 ceph-mon[31429]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:47.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:47 smithi090 ceph-mon[31429]: from='osd.11 ' entity='osd.11' 2024-05-27T07:39:47.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:47 smithi129 ceph-mon[35254]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-05-27T07:39:47.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:47 smithi129 ceph-mon[35254]: osdmap e71: 12 total, 11 up, 12 in 2024-05-27T07:39:47.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:47.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:47 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:47.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:47 smithi129 ceph-mon[35254]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-05-27T07:39:47.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:47 smithi129 ceph-mon[35254]: from='osd.11 ' entity='osd.11' 2024-05-27T07:39:48.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:48 smithi136 ceph-mon[35324]: purged_snaps scrub starts 2024-05-27T07:39:48.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:48 smithi136 ceph-mon[35324]: purged_snaps scrub ok 2024-05-27T07:39:48.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:48 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:48.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:48 smithi136 ceph-mon[35324]: osd.11 [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] boot 2024-05-27T07:39:48.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:48 smithi136 ceph-mon[35324]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T07:39:48.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:48 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:48.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:48 smithi090 ceph-mon[31429]: purged_snaps scrub starts 2024-05-27T07:39:48.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:48 smithi090 ceph-mon[31429]: purged_snaps scrub ok 2024-05-27T07:39:48.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:48 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:48.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:48 smithi090 ceph-mon[31429]: osd.11 [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] boot 2024-05-27T07:39:48.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:48 smithi090 ceph-mon[31429]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T07:39:48.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:48 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:48.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:48 smithi129 ceph-mon[35254]: purged_snaps scrub starts 2024-05-27T07:39:48.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:48 smithi129 ceph-mon[35254]: purged_snaps scrub ok 2024-05-27T07:39:48.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:48 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:48.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:48 smithi129 ceph-mon[35254]: osd.11 [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] boot 2024-05-27T07:39:48.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:48 smithi129 ceph-mon[35254]: osdmap e72: 12 total, 12 up, 12 in 2024-05-27T07:39:48.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:48 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:39:49.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:49 smithi136 ceph-mon[35324]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:49.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.426+0000 7f0f7e400700 1 -- 172.21.15.90:0/892660152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f78105e00 msgr2=0x7f0f7810c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:49.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.426+0000 7f0f7e400700 1 --2- 172.21.15.90:0/892660152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f78105e00 0x7f0f7810c6b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000b0a0 tx=0x7f0f7000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.426+0000 7f0f7e400700 1 -- 172.21.15.90:0/892660152 shutdown_connections 2024-05-27T07:39:49.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.426+0000 7f0f7e400700 1 --2- 172.21.15.90:0/892660152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f78105e00 0x7f0f7810c6b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.426+0000 7f0f7e400700 1 --2- 172.21.15.90:0/892660152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f78104330 0x7f0f78104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.426+0000 7f0f7e400700 1 --2- 172.21.15.90:0/892660152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f78105460 0x7f0f78105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.426+0000 7f0f7e400700 1 -- 172.21.15.90:0/892660152 >> 172.21.15.90:0/892660152 conn(0x7f0f780ff9b0 msgr2=0x7f0f78101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:49.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.426+0000 7f0f7e400700 1 -- 172.21.15.90:0/892660152 shutdown_connections 2024-05-27T07:39:49.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.426+0000 7f0f7e400700 1 -- 172.21.15.90:0/892660152 wait complete. 2024-05-27T07:39:49.425 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:49 smithi090 ceph-mon[31429]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:49.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.428+0000 7f0f7e400700 1 Processor -- start 2024-05-27T07:39:49.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.428+0000 7f0f7e400700 1 -- start start 2024-05-27T07:39:49.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.428+0000 7f0f7e400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f78104330 0x7f0f7819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:49.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.428+0000 7f0f7e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f78105460 0x7f0f7819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:49.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.428+0000 7f0f7e400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f78105e00 0x7f0f781a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.428+0000 7f0f7e400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7810fc90 con 0x7f0f78105460 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.428+0000 7f0f7e400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7810fb10 con 0x7f0f78104330 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.428+0000 7f0f7e400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7810fe10 con 0x7f0f78105e00 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f6fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f78105460 0x7f0f7819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f6fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f78105460 0x7f0f7819cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38172/0 (socket says 172.21.15.90:38172) 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f6fe00700 1 -- 172.21.15.90:0/4165598913 learned_addr learned my addr 172.21.15.90:0/4165598913 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f77400700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f78104330 0x7f0f7819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f77e00700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f78105e00 0x7f0f781a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f6fe00700 1 -- 172.21.15.90:0/4165598913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f78105e00 msgr2=0x7f0f781a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f6fe00700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f78105e00 0x7f0f781a3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f6fe00700 1 -- 172.21.15.90:0/4165598913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f78104330 msgr2=0x7f0f7819ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f6fe00700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f78104330 0x7f0f7819ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f6fe00700 1 -- 172.21.15.90:0/4165598913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f781a4570 con 0x7f0f78105460 2024-05-27T07:39:49.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.429+0000 7f0f77400700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f78104330 0x7f0f7819ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:49.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.430+0000 7f0f6fe00700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f78105460 0x7f0f7819cf50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000e9a0 tx=0x7f0f6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:49.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.430+0000 7f0f74c00700 1 -- 172.21.15.90:0/4165598913 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6000cbd0 con 0x7f0f78105460 2024-05-27T07:39:49.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.430+0000 7f0f7e400700 1 -- 172.21.15.90:0/4165598913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f781a4800 con 0x7f0f78105460 2024-05-27T07:39:49.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.430+0000 7f0f74c00700 1 -- 172.21.15.90:0/4165598913 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f6000cd70 con 0x7f0f78105460 2024-05-27T07:39:49.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.430+0000 7f0f74c00700 1 -- 172.21.15.90:0/4165598913 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60018680 con 0x7f0f78105460 2024-05-27T07:39:49.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.430+0000 7f0f7e400700 1 -- 172.21.15.90:0/4165598913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f781a4d90 con 0x7f0f78105460 2024-05-27T07:39:49.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.431+0000 7f0f7e400700 1 -- 172.21.15.90:0/4165598913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f780512b0 con 0x7f0f78105460 2024-05-27T07:39:49.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.434+0000 7f0f74c00700 1 -- 172.21.15.90:0/4165598913 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0f60010430 con 0x7f0f78105460 2024-05-27T07:39:49.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.436+0000 7f0f74c00700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0f5c0b2900 0x7f0f5c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:49.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.436+0000 7f0f74c00700 1 -- 172.21.15.90:0/4165598913 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f0f600e2350 con 0x7f0f78105460 2024-05-27T07:39:49.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.436+0000 7f0f77400700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0f5c0b2900 0x7f0f5c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:49.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.437+0000 7f0f77400700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0f5c0b2900 0x7f0f5c0b4e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f68005950 tx=0x7f0f680058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:49.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.439+0000 7f0f74c00700 1 -- 172.21.15.90:0/4165598913 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f600aa470 con 0x7f0f78105460 2024-05-27T07:39:49.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:49 smithi129 ceph-mon[35254]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:49.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.674+0000 7f0f7e400700 1 -- 172.21.15.90:0/4165598913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0f781a51b0 con 0x7f0f78105460 2024-05-27T07:39:49.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.675+0000 7f0f74c00700 1 -- 172.21.15.90:0/4165598913 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f0f600a9d70 con 0x7f0f78105460 2024-05-27T07:39:49.672 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:39:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.679+0000 7f0f7e400700 1 -- 172.21.15.90:0/4165598913 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0f5c0b2900 msgr2=0x7f0f5c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.679+0000 7f0f7e400700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0f5c0b2900 0x7f0f5c0b4e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f68005950 tx=0x7f0f680058e0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.679+0000 7f0f7e400700 1 -- 172.21.15.90:0/4165598913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f78105460 msgr2=0x7f0f7819cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.679+0000 7f0f7e400700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f78105460 0x7f0f7819cf50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000e9a0 tx=0x7f0f6000ecb0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.679+0000 7f0f7e400700 1 -- 172.21.15.90:0/4165598913 shutdown_connections 2024-05-27T07:39:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.680+0000 7f0f7e400700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0f5c0b2900 0x7f0f5c0b4e10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.680+0000 7f0f7e400700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f78105e00 0x7f0f781a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.680+0000 7f0f7e400700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f78104330 0x7f0f7819ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.680+0000 7f0f7e400700 1 --2- 172.21.15.90:0/4165598913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f78105460 0x7f0f7819cf50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:49.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.680+0000 7f0f7e400700 1 -- 172.21.15.90:0/4165598913 >> 172.21.15.90:0/4165598913 conn(0x7f0f780ff9b0 msgr2=0x7f0f780759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:49.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.680+0000 7f0f7e400700 1 -- 172.21.15.90:0/4165598913 shutdown_connections 2024-05-27T07:39:49.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:49.680+0000 7f0f7e400700 1 -- 172.21.15.90:0/4165598913 wait complete. 2024-05-27T07:39:50.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:50 smithi090 ceph-mon[31429]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T07:39:50.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:50 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/4165598913' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T07:39:50.462 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:50 smithi129 ceph-mon[35254]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T07:39:50.463 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:50 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/4165598913' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T07:39:50.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:50 smithi136 ceph-mon[35324]: osdmap e73: 12 total, 12 up, 12 in 2024-05-27T07:39:50.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:50 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/4165598913' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T07:39:50.594 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1716795587,"num_in_osds":12,"osd_in_since":1716795560,"num_remapped_pgs":0} 2024-05-27T07:39:50.595 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:39:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:39:51.452 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:51 smithi090 ceph-mon[31429]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:51.460 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:51 smithi136 ceph-mon[35324]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:51.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:51 smithi129 ceph-mon[35254]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: Detected new or changed devices on smithi136 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi136 to 4272M 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:52.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:52 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:52.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: Detected new or changed devices on smithi136 2024-05-27T07:39:52.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:52.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:52.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:52.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:52.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:52.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:52.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi136 to 4272M 2024-05-27T07:39:52.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:52.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:52.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:52.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:52 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:53.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: Detected new or changed devices on smithi136 2024-05-27T07:39:53.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:53.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:53.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:53.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:53.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:53.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:39:53.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi136 to 4272M 2024-05-27T07:39:53.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:53.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:39:53.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:39:53.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:52 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:39:53.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.483+0000 7f29eae00700 1 -- 172.21.15.90:0/3920980587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e4077780 msgr2=0x7f29e4110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:53.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.483+0000 7f29eae00700 1 --2- 172.21.15.90:0/3920980587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e4077780 0x7f29e4110630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009990 tx=0x7f29d0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.483+0000 7f29eae00700 1 -- 172.21.15.90:0/3920980587 shutdown_connections 2024-05-27T07:39:53.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.483+0000 7f29eae00700 1 --2- 172.21.15.90:0/3920980587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4110b70 0x7f29e4112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.483+0000 7f29eae00700 1 --2- 172.21.15.90:0/3920980587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e4076e30 0x7f29e4077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.483+0000 7f29eae00700 1 --2- 172.21.15.90:0/3920980587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e4077780 0x7f29e4110630 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.483+0000 7f29eae00700 1 -- 172.21.15.90:0/3920980587 >> 172.21.15.90:0/3920980587 conn(0x7f29e40ffe90 msgr2=0x7f29e41022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:53.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.484+0000 7f29eae00700 1 -- 172.21.15.90:0/3920980587 shutdown_connections 2024-05-27T07:39:53.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.484+0000 7f29eae00700 1 -- 172.21.15.90:0/3920980587 wait complete. 2024-05-27T07:39:53.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.485+0000 7f29eae00700 1 Processor -- start 2024-05-27T07:39:53.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.485+0000 7f29eae00700 1 -- start start 2024-05-27T07:39:53.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.485+0000 7f29eae00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e4076e30 0x7f29e419d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.485+0000 7f29eae00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4077780 0x7f29e419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.485+0000 7f29eae00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e4110b70 0x7f29e41a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.485+0000 7f29eae00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4116660 con 0x7f29e4110b70 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.485+0000 7f29eae00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41164e0 con 0x7f29e4077780 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.485+0000 7f29eae00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f29e41167e0 con 0x7f29e4076e30 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e4110b70 0x7f29e41a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e4110b70 0x7f29e41a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38198/0 (socket says 172.21.15.90:38198) 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e8e00700 1 -- 172.21.15.90:0/3636423495 learned_addr learned my addr 172.21.15.90:0/3636423495 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e3e00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e4076e30 0x7f29e419d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e3400700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4077780 0x7f29e419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:53.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e8e00700 1 -- 172.21.15.90:0/3636423495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e4076e30 msgr2=0x7f29e419d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e8e00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e4076e30 0x7f29e419d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e8e00700 1 -- 172.21.15.90:0/3636423495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4077780 msgr2=0x7f29e419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e8e00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4077780 0x7f29e419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e8e00700 1 -- 172.21.15.90:0/3636423495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e41a1fc0 con 0x7f29e4110b70 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.486+0000 7f29e3400700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4077780 0x7f29e419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.487+0000 7f29e8e00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e4110b70 0x7f29e41a1920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00b9e0 tx=0x7f29dc00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.487+0000 7f29e0c00700 1 -- 172.21.15.90:0/3636423495 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc019070 con 0x7f29e4110b70 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.487+0000 7f29eae00700 1 -- 172.21.15.90:0/3636423495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e41a22b0 con 0x7f29e4110b70 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.487+0000 7f29e0c00700 1 -- 172.21.15.90:0/3636423495 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29dc00cdc0 con 0x7f29e4110b70 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.487+0000 7f29e0c00700 1 -- 172.21.15.90:0/3636423495 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc014e50 con 0x7f29e4110b70 2024-05-27T07:39:53.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.487+0000 7f29eae00700 1 -- 172.21.15.90:0/3636423495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e41a9bd0 con 0x7f29e4110b70 2024-05-27T07:39:53.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.488+0000 7f29eae00700 1 -- 172.21.15.90:0/3636423495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e40512b0 con 0x7f29e4110b70 2024-05-27T07:39:53.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.493+0000 7f29e0c00700 1 -- 172.21.15.90:0/3636423495 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f29dc01c070 con 0x7f29e4110b70 2024-05-27T07:39:53.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.494+0000 7f29e0c00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f29c80b2a20 0x7f29c80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:53.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.494+0000 7f29e0c00700 1 -- 172.21.15.90:0/3636423495 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f29dc0e15e0 con 0x7f29e4110b70 2024-05-27T07:39:53.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.494+0000 7f29e3e00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f29c80b2a20 0x7f29c80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:53.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.495+0000 7f29e3e00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f29c80b2a20 0x7f29c80b4f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009a10 tx=0x7f29d4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:53.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.497+0000 7f29e0c00700 1 -- 172.21.15.90:0/3636423495 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29dc0a9700 con 0x7f29e4110b70 2024-05-27T07:39:53.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.734+0000 7f29eae00700 1 -- 172.21.15.90:0/3636423495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29e419e450 con 0x7f29e4110b70 2024-05-27T07:39:53.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.735+0000 7f29e0c00700 1 -- 172.21.15.90:0/3636423495 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18432 (secure 0 0 0) 0x7f29dc0a9000 con 0x7f29e4110b70 2024-05-27T07:39:53.733 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:39:53.733 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":73,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:39:49.141652+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:39:53.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.738+0000 7f29eae00700 1 -- 172.21.15.90:0/3636423495 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f29c80b2a20 msgr2=0x7f29c80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:53.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.738+0000 7f29eae00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f29c80b2a20 0x7f29c80b4f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009a10 tx=0x7f29d4009190 comp rx=0 tx=0).stop 2024-05-27T07:39:53.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.738+0000 7f29eae00700 1 -- 172.21.15.90:0/3636423495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e4110b70 msgr2=0x7f29e41a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:53.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.738+0000 7f29eae00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e4110b70 0x7f29e41a1920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00b9e0 tx=0x7f29dc00bcf0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.739+0000 7f29eae00700 1 -- 172.21.15.90:0/3636423495 shutdown_connections 2024-05-27T07:39:53.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.739+0000 7f29eae00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f29c80b2a20 0x7f29c80b4f30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.739+0000 7f29eae00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29e4110b70 0x7f29e41a1920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.739+0000 7f29eae00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29e4076e30 0x7f29e419d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.739+0000 7f29eae00700 1 --2- 172.21.15.90:0/3636423495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f29e4077780 0x7f29e419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:53.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.739+0000 7f29eae00700 1 -- 172.21.15.90:0/3636423495 >> 172.21.15.90:0/3636423495 conn(0x7f29e40ffe90 msgr2=0x7f29e4101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:53.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.739+0000 7f29eae00700 1 -- 172.21.15.90:0/3636423495 shutdown_connections 2024-05-27T07:39:53.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:53.739+0000 7f29eae00700 1 -- 172.21.15.90:0/3636423495 wait complete. 2024-05-27T07:39:53.763 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:53 smithi090 ceph-mon[31429]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:53.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:53 smithi136 ceph-mon[35324]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:54.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:53 smithi129 ceph-mon[35254]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:54.656 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T07:34:18.340712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T07:39:54.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-27T07:39:54.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:54 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3636423495' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:39:54.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:54 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3636423495' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:39:55.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:54 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3636423495' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:39:55.099 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:39:55.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:55 smithi090 ceph-mon[31429]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:55.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:55 smithi136 ceph-mon[35324]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:56.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:55 smithi129 ceph-mon[35254]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:57.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.436+0000 7f5923000700 1 -- 172.21.15.90:0/432467264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f591c10af40 msgr2=0x7f591c10b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:57.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.436+0000 7f5923000700 1 --2- 172.21.15.90:0/432467264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f591c10af40 0x7f591c10b350 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f59080099c0 tx=0x7f5908009cd0 comp rx=0 tx=0).stop 2024-05-27T07:39:57.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.436+0000 7f5923000700 1 -- 172.21.15.90:0/432467264 shutdown_connections 2024-05-27T07:39:57.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.436+0000 7f5923000700 1 --2- 172.21.15.90:0/432467264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c10b890 0x7f591c1131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:57.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.436+0000 7f5923000700 1 --2- 172.21.15.90:0/432467264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f591c106610 0x7f591c106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:57.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.436+0000 7f5923000700 1 --2- 172.21.15.90:0/432467264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f591c10af40 0x7f591c10b350 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:57.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.436+0000 7f5923000700 1 -- 172.21.15.90:0/432467264 >> 172.21.15.90:0/432467264 conn(0x7f591c102020 msgr2=0x7f591c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:57.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.436+0000 7f5923000700 1 -- 172.21.15.90:0/432467264 shutdown_connections 2024-05-27T07:39:57.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.436+0000 7f5923000700 1 -- 172.21.15.90:0/432467264 wait complete. 2024-05-27T07:39:57.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.437+0000 7f5923000700 1 Processor -- start 2024-05-27T07:39:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.437+0000 7f5923000700 1 -- start start 2024-05-27T07:39:57.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.438+0000 7f5923000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c106610 0x7f591c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:57.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.438+0000 7f5923000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f591c10af40 0x7f591c19d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:57.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.438+0000 7f5923000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f591c10b890 0x7f591c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:57.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.438+0000 7f5923000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1163a0 con 0x7f591c10af40 2024-05-27T07:39:57.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.438+0000 7f5923000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f591c116220 con 0x7f591c106610 2024-05-27T07:39:57.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.438+0000 7f5923000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f591c116520 con 0x7f591c10b890 2024-05-27T07:39:57.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.438+0000 7f5921200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f591c10af40 0x7f591c19d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:57.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.438+0000 7f5921200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f591c10af40 0x7f591c19d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38218/0 (socket says 172.21.15.90:38218) 2024-05-27T07:39:57.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.438+0000 7f5921200700 1 -- 172.21.15.90:0/2103915564 learned_addr learned my addr 172.21.15.90:0/2103915564 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:39:57.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.438+0000 7f5921c00700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c106610 0x7f591c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:57.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f5922600700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f591c10b890 0x7f591c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:57.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f5921200700 1 -- 172.21.15.90:0/2103915564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f591c10b890 msgr2=0x7f591c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:57.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f5921200700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f591c10b890 0x7f591c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:57.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f5921200700 1 -- 172.21.15.90:0/2103915564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c106610 msgr2=0x7f591c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:57.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f5921200700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c106610 0x7f591c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:57.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f5921200700 1 -- 172.21.15.90:0/2103915564 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c1a1b70 con 0x7f591c10af40 2024-05-27T07:39:57.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f5921200700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f591c10af40 0x7f591c19d2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f590800b580 tx=0x7f590800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:57.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f591a000700 1 -- 172.21.15.90:0/2103915564 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908027070 con 0x7f591c10af40 2024-05-27T07:39:57.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f591a000700 1 -- 172.21.15.90:0/2103915564 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f59080056d0 con 0x7f591c10af40 2024-05-27T07:39:57.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f5923000700 1 -- 172.21.15.90:0/2103915564 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c1a1e00 con 0x7f591c10af40 2024-05-27T07:39:57.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.439+0000 7f591a000700 1 -- 172.21.15.90:0/2103915564 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59080173f0 con 0x7f591c10af40 2024-05-27T07:39:57.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.440+0000 7f5923000700 1 -- 172.21.15.90:0/2103915564 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c1a2330 con 0x7f591c10af40 2024-05-27T07:39:57.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.441+0000 7f5923000700 1 -- 172.21.15.90:0/2103915564 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591c0512b0 con 0x7f591c10af40 2024-05-27T07:39:57.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.443+0000 7f591a000700 1 -- 172.21.15.90:0/2103915564 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f590802a070 con 0x7f591c10af40 2024-05-27T07:39:57.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.445+0000 7f591a000700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f59040b2900 0x7f59040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:39:57.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.445+0000 7f591a000700 1 -- 172.21.15.90:0/2103915564 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f59080e2fc0 con 0x7f591c10af40 2024-05-27T07:39:57.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.445+0000 7f5921c00700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f59040b2900 0x7f59040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:39:57.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.446+0000 7f5921c00700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f59040b2900 0x7f59040b4e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5910005ea0 tx=0x7f5910005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:39:57.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.448+0000 7f591a000700 1 -- 172.21.15.90:0/2103915564 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f590805fdf0 con 0x7f591c10af40 2024-05-27T07:39:57.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.680+0000 7f5923000700 1 -- 172.21.15.90:0/2103915564 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f591c19e0b0 con 0x7f591c10af40 2024-05-27T07:39:57.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.681+0000 7f591a000700 1 -- 172.21.15.90:0/2103915564 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f5908013070 con 0x7f591c10af40 2024-05-27T07:39:57.679 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-05-27T07:39:57.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.684+0000 7f5923000700 1 -- 172.21.15.90:0/2103915564 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f59040b2900 msgr2=0x7f59040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:57.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.684+0000 7f5923000700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f59040b2900 0x7f59040b4e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5910005ea0 tx=0x7f5910005e30 comp rx=0 tx=0).stop 2024-05-27T07:39:57.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.684+0000 7f5923000700 1 -- 172.21.15.90:0/2103915564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f591c10af40 msgr2=0x7f591c19d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:39:57.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.684+0000 7f5923000700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f591c10af40 0x7f591c19d2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f590800b580 tx=0x7f590800f740 comp rx=0 tx=0).stop 2024-05-27T07:39:57.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.685+0000 7f5923000700 1 -- 172.21.15.90:0/2103915564 shutdown_connections 2024-05-27T07:39:57.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.685+0000 7f5923000700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f59040b2900 0x7f59040b4e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:57.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.685+0000 7f5923000700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f591c10b890 0x7f591c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:57.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.685+0000 7f5923000700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f591c106610 0x7f591c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:57.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.685+0000 7f5923000700 1 --2- 172.21.15.90:0/2103915564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f591c10af40 0x7f591c19d2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:39:57.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.685+0000 7f5923000700 1 -- 172.21.15.90:0/2103915564 >> 172.21.15.90:0/2103915564 conn(0x7f591c102020 msgr2=0x7f591c10a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:39:57.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.685+0000 7f5923000700 1 -- 172.21.15.90:0/2103915564 shutdown_connections 2024-05-27T07:39:57.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:39:57.685+0000 7f5923000700 1 -- 172.21.15.90:0/2103915564 wait complete. 2024-05-27T07:39:57.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:57 smithi090 ceph-mon[31429]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:57.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:57 smithi136 ceph-mon[35324]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:58.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:57 smithi129 ceph-mon[35254]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:58.502 INFO:tasks.cephadm:Adding mds.a on smithi090 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.d on smithi090 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.g on smithi090 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.j on smithi090 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.b on smithi129 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.e on smithi129 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.h on smithi129 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.k on smithi129 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.c on smithi136 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.f on smithi136 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.i on smithi136 2024-05-27T07:39:58.503 INFO:tasks.cephadm:Adding mds.l on smithi136 2024-05-27T07:39:58.504 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l' 2024-05-27T07:39:58.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:58 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2103915564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:39:58.945 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T07:39:58.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:58 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2103915564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:39:59.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:58 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2103915564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:39:59.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:39:59 smithi136 ceph-mon[35324]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:39:59.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:39:59 smithi090 ceph-mon[31429]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:00.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:39:59 smithi129 ceph-mon[35254]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:00.942 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:00 smithi136 ceph-mon[35324]: overall HEALTH_OK 2024-05-27T07:40:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:00 smithi129 ceph-mon[35254]: overall HEALTH_OK 2024-05-27T07:40:01.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:00 smithi090 ceph-mon[31429]: overall HEALTH_OK 2024-05-27T07:40:01.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.305+0000 7eff71600700 1 -- 172.21.15.136:0/1382016004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6c10c250 msgr2=0x7eff6c112b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:01.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.305+0000 7eff71600700 1 --2- 172.21.15.136:0/1382016004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6c10c250 0x7eff6c112b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7eff6400b0a0 tx=0x7eff6400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:40:01.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.305+0000 7eff71600700 1 -- 172.21.15.136:0/1382016004 shutdown_connections 2024-05-27T07:40:01.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.305+0000 7eff71600700 1 --2- 172.21.15.136:0/1382016004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6c10c250 0x7eff6c112b00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:01.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.305+0000 7eff71600700 1 --2- 172.21.15.136:0/1382016004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff6c10a780 0x7eff6c10ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:01.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.305+0000 7eff71600700 1 --2- 172.21.15.136:0/1382016004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff6c10b8b0 0x7eff6c10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:01.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.305+0000 7eff71600700 1 -- 172.21.15.136:0/1382016004 >> 172.21.15.136:0/1382016004 conn(0x7eff6c079830 msgr2=0x7eff6c1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:01.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.306+0000 7eff71600700 1 -- 172.21.15.136:0/1382016004 shutdown_connections 2024-05-27T07:40:01.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.306+0000 7eff71600700 1 -- 172.21.15.136:0/1382016004 wait complete. 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.307+0000 7eff71600700 1 Processor -- start 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.307+0000 7eff71600700 1 -- start start 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.307+0000 7eff71600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6c10a780 0x7eff6c1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.307+0000 7eff71600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff6c10b8b0 0x7eff6c1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.307+0000 7eff71600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff6c10c250 0x7eff6c1a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.307+0000 7eff71600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1160f0 con 0x7eff6c10c250 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.307+0000 7eff71600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c115f70 con 0x7eff6c10b8b0 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.307+0000 7eff71600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c116270 con 0x7eff6c10a780 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.308+0000 7eff6be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6c10a780 0x7eff6c1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.308+0000 7eff6be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6c10a780 0x7eff6c1a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:47710/0 (socket says 172.21.15.136:47710) 2024-05-27T07:40:01.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.308+0000 7eff6be00700 1 -- 172.21.15.136:0/2132380281 learned_addr learned my addr 172.21.15.136:0/2132380281 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-05-27T07:40:01.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.308+0000 7eff70c00700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff6c10c250 0x7eff6c1a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:01.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.308+0000 7eff6b400700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff6c10b8b0 0x7eff6c1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:01.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.308+0000 7eff6be00700 1 -- 172.21.15.136:0/2132380281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff6c10b8b0 msgr2=0x7eff6c1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:01.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.308+0000 7eff6be00700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff6c10b8b0 0x7eff6c1a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:01.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.308+0000 7eff6be00700 1 -- 172.21.15.136:0/2132380281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff6c10c250 msgr2=0x7eff6c1a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:01.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.308+0000 7eff6be00700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff6c10c250 0x7eff6c1a80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:01.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.308+0000 7eff6be00700 1 -- 172.21.15.136:0/2132380281 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c1a8770 con 0x7eff6c10a780 2024-05-27T07:40:01.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.309+0000 7eff6be00700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6c10a780 0x7eff6c1a0c10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00d950 tx=0x7eff5c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:01.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.309+0000 7eff68c00700 1 -- 172.21.15.136:0/2132380281 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c017070 con 0x7eff6c10a780 2024-05-27T07:40:01.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.309+0000 7eff68c00700 1 -- 172.21.15.136:0/2132380281 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff5c00f690 con 0x7eff6c10a780 2024-05-27T07:40:01.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.309+0000 7eff71600700 1 -- 172.21.15.136:0/2132380281 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c1a8a60 con 0x7eff6c10a780 2024-05-27T07:40:01.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.309+0000 7eff68c00700 1 -- 172.21.15.136:0/2132380281 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0039e0 con 0x7eff6c10a780 2024-05-27T07:40:01.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.309+0000 7eff71600700 1 -- 172.21.15.136:0/2132380281 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c1a8ff0 con 0x7eff6c10a780 2024-05-27T07:40:01.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.311+0000 7eff71600700 1 -- 172.21.15.136:0/2132380281 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6c0512b0 con 0x7eff6c10a780 2024-05-27T07:40:01.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.313+0000 7eff68c00700 1 -- 172.21.15.136:0/2132380281 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7eff5c003b80 con 0x7eff6c10a780 2024-05-27T07:40:01.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.315+0000 7eff68c00700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7eff500b2900 0x7eff500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:01.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.315+0000 7eff68c00700 1 -- 172.21.15.136:0/2132380281 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7eff5c0e1fa0 con 0x7eff6c10a780 2024-05-27T07:40:01.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.316+0000 7eff6b400700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7eff500b2900 0x7eff500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:01.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.317+0000 7eff6b400700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7eff500b2900 0x7eff500b4e10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7eff6c1a1cc0 tx=0x7eff5800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:01.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.318+0000 7eff68c00700 1 -- 172.21.15.136:0/2132380281 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff5c020030 con 0x7eff6c10a780 2024-05-27T07:40:01.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.563+0000 7eff71600700 1 -- 172.21.15.136:0/2132380281 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}) -- 0x7eff6c063eb0 con 0x7eff500b2900 2024-05-27T07:40:01.581 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.583+0000 7eff68c00700 1 -- 172.21.15.136:0/2132380281 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7eff6c063eb0 con 0x7eff500b2900 2024-05-27T07:40:01.581 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mds.all update... 2024-05-27T07:40:01.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.586+0000 7eff71600700 1 -- 172.21.15.136:0/2132380281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7eff500b2900 msgr2=0x7eff500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:01.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.586+0000 7eff71600700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7eff500b2900 0x7eff500b4e10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7eff6c1a1cc0 tx=0x7eff5800b410 comp rx=0 tx=0).stop 2024-05-27T07:40:01.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.586+0000 7eff71600700 1 -- 172.21.15.136:0/2132380281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6c10a780 msgr2=0x7eff6c1a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:01.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.586+0000 7eff71600700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6c10a780 0x7eff6c1a0c10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00d950 tx=0x7eff5c00dc60 comp rx=0 tx=0).stop 2024-05-27T07:40:01.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.587+0000 7eff71600700 1 -- 172.21.15.136:0/2132380281 shutdown_connections 2024-05-27T07:40:01.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.587+0000 7eff71600700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7eff500b2900 0x7eff500b4e10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:01.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.587+0000 7eff71600700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7eff6c10c250 0x7eff6c1a80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:01.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.587+0000 7eff71600700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6c10a780 0x7eff6c1a0c10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:01.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.587+0000 7eff71600700 1 --2- 172.21.15.136:0/2132380281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff6c10b8b0 0x7eff6c1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:01.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.587+0000 7eff71600700 1 -- 172.21.15.136:0/2132380281 >> 172.21.15.136:0/2132380281 conn(0x7eff6c079830 msgr2=0x7eff6c10eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:01.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.587+0000 7eff71600700 1 -- 172.21.15.136:0/2132380281 shutdown_connections 2024-05-27T07:40:01.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-05-27T07:40:01.587+0000 7eff71600700 1 -- 172.21.15.136:0/2132380281 wait complete. 2024-05-27T07:40:01.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:01 smithi136 ceph-mon[35324]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:01.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:01.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:40:01.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:01.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:40:01.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:01.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:01.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:01.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:02.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:01 smithi129 ceph-mon[35254]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:02.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:02.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:40:02.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:02.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:40:02.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:02.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:02.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:02.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:02.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:01 smithi090 ceph-mon[31429]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:02.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:02.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:40:02.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:02.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:40:02.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:02.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:02.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:02.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:02.428 DEBUG:teuthology.orchestra.run.smithi090:mds.a> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.a.service 2024-05-27T07:40:02.436 DEBUG:teuthology.orchestra.run.smithi090:mds.d> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service 2024-05-27T07:40:02.482 DEBUG:teuthology.orchestra.run.smithi090:mds.g> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.g.service 2024-05-27T07:40:02.528 DEBUG:teuthology.orchestra.run.smithi090:mds.j> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service 2024-05-27T07:40:02.574 DEBUG:teuthology.orchestra.run.smithi129:mds.b> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.b.service 2024-05-27T07:40:02.582 DEBUG:teuthology.orchestra.run.smithi129:mds.e> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.e.service 2024-05-27T07:40:02.628 DEBUG:teuthology.orchestra.run.smithi129:mds.h> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.h.service 2024-05-27T07:40:02.673 DEBUG:teuthology.orchestra.run.smithi129:mds.k> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.k.service 2024-05-27T07:40:02.720 DEBUG:teuthology.orchestra.run.smithi136:mds.c> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service 2024-05-27T07:40:02.728 DEBUG:teuthology.orchestra.run.smithi136:mds.f> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.f.service 2024-05-27T07:40:02.775 DEBUG:teuthology.orchestra.run.smithi136:mds.i> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.i.service 2024-05-27T07:40:02.820 DEBUG:teuthology.orchestra.run.smithi136:mds.l> sudo journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.l.service 2024-05-27T07:40:02.865 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-27T07:40:02.865 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:40:02.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:02 smithi090 ceph-mon[31429]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:40:02.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:02 smithi090 ceph-mon[31429]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-05-27T07:40:02.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:02 smithi090 ceph-mon[31429]: Deploying daemon mds.l on smithi136 2024-05-27T07:40:02.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:02 smithi136 ceph-mon[35324]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:40:02.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:02 smithi136 ceph-mon[35324]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-05-27T07:40:02.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:02 smithi136 ceph-mon[35324]: Deploying daemon mds.l on smithi136 2024-05-27T07:40:03.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:02 smithi129 ceph-mon[35254]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:40:03.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:02 smithi129 ceph-mon[35254]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-05-27T07:40:03.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:02 smithi129 ceph-mon[35254]: Deploying daemon mds.l on smithi136 2024-05-27T07:40:03.314 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:03.984 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:03 smithi136 ceph-mon[35324]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:04.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:03 smithi129 ceph-mon[35254]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:04.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:03 smithi090 ceph-mon[31429]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:05.128 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 07:40:04 smithi136 systemd[1]: Starting Ceph mds.l for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:05.497 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 07:40:05 smithi136 podman[65403]: 2024-05-27 07:40:05.034895379 +0000 UTC m=+0.034283529 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:05.497 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 07:40:05 smithi136 podman[65403]: 2024-05-27 07:40:05.179509246 +0000 UTC m=+0.178897431 container create d8a33d5999d14dbf015667b2f29d72a243f96f44ffa4eda0847cde7e13eb3cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:40:05.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.543+0000 7fdc1ca00700 1 -- 172.21.15.90:0/1731508624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc1810c5a0 msgr2=0x7fdc1810c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:05.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.543+0000 7fdc1ca00700 1 --2- 172.21.15.90:0/1731508624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc1810c5a0 0x7fdc1810c9b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009990 tx=0x7fdc08009ca0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.543+0000 7fdc1ca00700 1 -- 172.21.15.90:0/1731508624 shutdown_connections 2024-05-27T07:40:05.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.543+0000 7fdc1ca00700 1 --2- 172.21.15.90:0/1731508624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc18106eb0 0x7fdc18113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.543+0000 7fdc1ca00700 1 --2- 172.21.15.90:0/1731508624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc1810c5a0 0x7fdc1810c9b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.543+0000 7fdc1ca00700 1 --2- 172.21.15.90:0/1731508624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc18106560 0x7fdc18106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.543+0000 7fdc1ca00700 1 -- 172.21.15.90:0/1731508624 >> 172.21.15.90:0/1731508624 conn(0x7fdc18102040 msgr2=0x7fdc18104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:05.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.543+0000 7fdc1ca00700 1 -- 172.21.15.90:0/1731508624 shutdown_connections 2024-05-27T07:40:05.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.543+0000 7fdc1ca00700 1 -- 172.21.15.90:0/1731508624 wait complete. 2024-05-27T07:40:05.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.544+0000 7fdc1ca00700 1 Processor -- start 2024-05-27T07:40:05.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.544+0000 7fdc1ca00700 1 -- start start 2024-05-27T07:40:05.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.545+0000 7fdc1ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc18106560 0x7fdc1819d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:05.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.545+0000 7fdc1ca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc18106eb0 0x7fdc1819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:05.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.545+0000 7fdc1ca00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc1810c5a0 0x7fdc181a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:05.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.545+0000 7fdc1ca00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdc18116680 con 0x7fdc18106560 2024-05-27T07:40:05.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.545+0000 7fdc1ca00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc18116500 con 0x7fdc18106eb0 2024-05-27T07:40:05.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.545+0000 7fdc1ca00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc18116800 con 0x7fdc1810c5a0 2024-05-27T07:40:05.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.545+0000 7fdc15e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc18106560 0x7fdc1819d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:05.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.545+0000 7fdc15e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc18106560 0x7fdc1819d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51190/0 (socket says 172.21.15.90:51190) 2024-05-27T07:40:05.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.545+0000 7fdc15e00700 1 -- 172.21.15.90:0/2597999951 learned_addr learned my addr 172.21.15.90:0/2597999951 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:40:05.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.545+0000 7fdc16800700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc1810c5a0 0x7fdc181a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:05.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc15400700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc18106eb0 0x7fdc1819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:05.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc15e00700 1 -- 172.21.15.90:0/2597999951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc1810c5a0 msgr2=0x7fdc181a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:05.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc15e00700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc1810c5a0 0x7fdc181a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc15e00700 1 -- 172.21.15.90:0/2597999951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc18106eb0 msgr2=0x7fdc1819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:05.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc15e00700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc18106eb0 0x7fdc1819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc15e00700 1 -- 172.21.15.90:0/2597999951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc181a1f80 con 0x7fdc18106560 2024-05-27T07:40:05.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc15e00700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc18106560 0x7fdc1819d0b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdc080098e0 tx=0x7fdc080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:05.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc0ea00700 1 -- 172.21.15.90:0/2597999951 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08004830 con 0x7fdc18106560 2024-05-27T07:40:05.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc0ea00700 1 -- 172.21.15.90:0/2597999951 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc0800f7e0 con 0x7fdc18106560 2024-05-27T07:40:05.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc0ea00700 1 -- 172.21.15.90:0/2597999951 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08005310 con 0x7fdc18106560 2024-05-27T07:40:05.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.546+0000 7fdc1ca00700 1 -- 172.21.15.90:0/2597999951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc181a2210 con 0x7fdc18106560 2024-05-27T07:40:05.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.547+0000 7fdc1ca00700 1 -- 172.21.15.90:0/2597999951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc181a26c0 con 0x7fdc18106560 2024-05-27T07:40:05.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.548+0000 7fdc1ca00700 1 -- 172.21.15.90:0/2597999951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1810e960 con 0x7fdc18106560 2024-05-27T07:40:05.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.552+0000 7fdc0ea00700 1 -- 172.21.15.90:0/2597999951 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdc0800fc90 con 0x7fdc18106560 2024-05-27T07:40:05.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.553+0000 7fdc0ea00700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbfc0b2900 0x7fdbfc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.554+0000 7fdc0ea00700 1 -- 172.21.15.90:0/2597999951 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fdc080e31e0 con 0x7fdc18106560 2024-05-27T07:40:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.554+0000 7fdc15400700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbfc0b2900 0x7fdbfc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.554+0000 7fdc15400700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbfc0b2900 0x7fdbfc0b4e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc04006fd0 tx=0x7fdc04008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:05.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.556+0000 7fdc0ea00700 1 -- 172.21.15.90:0/2597999951 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc08013070 con 0x7fdc18106560 2024-05-27T07:40:05.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.852+0000 7fdc1ca00700 1 -- 172.21.15.90:0/2597999951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc1819e410 con 0x7fdc18106560 2024-05-27T07:40:05.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:05 smithi090 ceph-mon[31429]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:05.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:05 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.136:6832/1595668128,v1:172.21.15.136:6833/1595668128] up:boot 2024-05-27T07:40:05.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:05 smithi090 ceph-mon[31429]: fsmap 1 up:standby 2024-05-27T07:40:05.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:05 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:40:05.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:05 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:05.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:05 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.853+0000 7fdc0ea00700 1 -- 172.21.15.90:0/2597999951 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fdc080ab390 con 0x7fdc18106560 2024-05-27T07:40:05.852 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:40:05.853 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-27T07:40:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 -- 172.21.15.90:0/2597999951 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbfc0b2900 msgr2=0x7fdbfc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbfc0b2900 0x7fdbfc0b4e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc04006fd0 tx=0x7fdc04008040 comp rx=0 tx=0).stop 2024-05-27T07:40:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 -- 172.21.15.90:0/2597999951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc18106560 msgr2=0x7fdc1819d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc18106560 0x7fdc1819d0b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdc080098e0 tx=0x7fdc080047b0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 -- 172.21.15.90:0/2597999951 shutdown_connections 2024-05-27T07:40:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbfc0b2900 0x7fdbfc0b4e10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc1810c5a0 0x7fdc181a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc18106560 0x7fdc1819d0b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 --2- 172.21.15.90:0/2597999951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc18106eb0 0x7fdc1819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 -- 172.21.15.90:0/2597999951 >> 172.21.15.90:0/2597999951 conn(0x7fdc18102040 msgr2=0x7fdc18103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 -- 172.21.15.90:0/2597999951 shutdown_connections 2024-05-27T07:40:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:05.857+0000 7fdc1ca00700 1 -- 172.21.15.90:0/2597999951 wait complete. 2024-05-27T07:40:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 2 2024-05-27T07:40:05.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:05 smithi136 ceph-mon[35324]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:05.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:05 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.136:6832/1595668128,v1:172.21.15.136:6833/1595668128] up:boot 2024-05-27T07:40:05.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:05 smithi136 ceph-mon[35324]: fsmap 1 up:standby 2024-05-27T07:40:05.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:05 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:40:05.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:05 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:05.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:05 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:05.998 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 07:40:05 smithi136 podman[65403]: 2024-05-27 07:40:05.50047797 +0000 UTC m=+0.499866162 container init d8a33d5999d14dbf015667b2f29d72a243f96f44ffa4eda0847cde7e13eb3cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-l, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:40:05.998 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 07:40:05 smithi136 podman[65403]: 2024-05-27 07:40:05.510508281 +0000 UTC m=+0.509896489 container start d8a33d5999d14dbf015667b2f29d72a243f96f44ffa4eda0847cde7e13eb3cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-27T07:40:05.998 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 07:40:05 smithi136 bash[65403]: d8a33d5999d14dbf015667b2f29d72a243f96f44ffa4eda0847cde7e13eb3cbd 2024-05-27T07:40:05.998 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 07:40:05 smithi136 systemd[1]: Started Ceph mds.l for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:05.998 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 07:40:05 smithi136 ceph-mds[65422]: -- 172.21.15.136:0/1134464271 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a3a03da40 con 0x559a3a085a80 2024-05-27T07:40:05.998 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 07:40:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-l[65418]: starting mds.l at 2024-05-27T07:40:06.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:05 smithi129 ceph-mon[35254]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:06.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:05 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.136:6832/1595668128,v1:172.21.15.136:6833/1595668128] up:boot 2024-05-27T07:40:06.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:05 smithi129 ceph-mon[35254]: fsmap 1 up:standby 2024-05-27T07:40:06.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:05 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:40:06.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:05 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:06.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:05 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:06.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:06.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:06.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:06.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:06.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:06 smithi090 ceph-mon[31429]: Deploying daemon mds.k on smithi129 2024-05-27T07:40:06.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:06 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2597999951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:07.041 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:06 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:07.041 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:06 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:07.041 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:06 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:07.041 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:06 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:07.041 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:06 smithi129 ceph-mon[35254]: Deploying daemon mds.k on smithi129 2024-05-27T07:40:07.042 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:06 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2597999951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:07.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:07.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:07.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:07.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:07.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:06 smithi136 ceph-mon[35324]: Deploying daemon mds.k on smithi129 2024-05-27T07:40:07.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:06 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2597999951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:08.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:07 smithi129 ceph-mon[35254]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:08.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:07 smithi090 ceph-mon[31429]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:08.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:07 smithi136 ceph-mon[35324]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:08.676 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:40:09.122 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:09.512 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 07:40:09 smithi129 systemd[1]: Starting Ceph mds.k for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:09.799 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 07:40:09 smithi129 podman[64360]: 2024-05-27 07:40:09.417505155 +0000 UTC m=+0.032191201 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:09.799 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 07:40:09 smithi129 podman[64360]: 2024-05-27 07:40:09.533536696 +0000 UTC m=+0.148222766 container create 446c1375720aca54b2ca1777d833303bfd4487436a85fc29b60dfe7c84649563 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-27T07:40:09.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:09 smithi090 ceph-mon[31429]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:10.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:09 smithi129 ceph-mon[35254]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:10.062 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 07:40:09 smithi129 podman[64360]: 2024-05-27 07:40:09.829125127 +0000 UTC m=+0.443811277 container init 446c1375720aca54b2ca1777d833303bfd4487436a85fc29b60dfe7c84649563 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:40:10.062 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 07:40:09 smithi129 podman[64360]: 2024-05-27 07:40:09.839731849 +0000 UTC m=+0.454417926 container start 446c1375720aca54b2ca1777d833303bfd4487436a85fc29b60dfe7c84649563 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-k, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521) 2024-05-27T07:40:10.062 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 07:40:09 smithi129 bash[64360]: 446c1375720aca54b2ca1777d833303bfd4487436a85fc29b60dfe7c84649563 2024-05-27T07:40:10.062 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 07:40:09 smithi129 systemd[1]: Started Ceph mds.k for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:10.062 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 07:40:09 smithi129 ceph-mds[64379]: -- 172.21.15.129:0/2676878282 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f04fc3ba40 con 0x55f04fc83a80 2024-05-27T07:40:10.062 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 07:40:09 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-k[64375]: starting mds.k at 2024-05-27T07:40:10.246 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:09 smithi136 ceph-mon[35324]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:11.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.211+0000 7f9b13c00700 1 -- 172.21.15.90:0/3614614206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b0c1045e0 msgr2=0x7f9b0c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.211+0000 7f9b13c00700 1 --2- 172.21.15.90:0/3614614206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b0c1045e0 0x7f9b0c1049f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9b000099c0 tx=0x7f9b00009cd0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.211+0000 7f9b13c00700 1 -- 172.21.15.90:0/3614614206 shutdown_connections 2024-05-27T07:40:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.211+0000 7f9b13c00700 1 --2- 172.21.15.90:0/3614614206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b0c109860 0x7f9b0c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.211+0000 7f9b13c00700 1 --2- 172.21.15.90:0/3614614206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b0c1045e0 0x7f9b0c1049f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.211+0000 7f9b13c00700 1 --2- 172.21.15.90:0/3614614206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b0c108f10 0x7f9b0c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.211+0000 7f9b13c00700 1 -- 172.21.15.90:0/3614614206 >> 172.21.15.90:0/3614614206 conn(0x7f9b0c100030 msgr2=0x7f9b0c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.211+0000 7f9b13c00700 1 -- 172.21.15.90:0/3614614206 shutdown_connections 2024-05-27T07:40:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.211+0000 7f9b13c00700 1 -- 172.21.15.90:0/3614614206 wait complete. 2024-05-27T07:40:11.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.212+0000 7f9b13c00700 1 Processor -- start 2024-05-27T07:40:11.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.213+0000 7f9b13c00700 1 -- start start 2024-05-27T07:40:11.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.213+0000 7f9b13c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b0c1045e0 0x7f9b0c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.213+0000 7f9b13c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b0c108f10 0x7f9b0c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.213+0000 7f9b13c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b0c109860 0x7f9b0c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.213+0000 7f9b13c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c114390 con 0x7f9b0c108f10 2024-05-27T07:40:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.213+0000 7f9b13c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c114210 con 0x7f9b0c109860 2024-05-27T07:40:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.213+0000 7f9b13c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c114510 con 0x7f9b0c1045e0 2024-05-27T07:40:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.213+0000 7f9b0be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b0c108f10 0x7f9b0c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.213+0000 7f9b0be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b0c108f10 0x7f9b0c19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57464/0 (socket says 172.21.15.90:57464) 2024-05-27T07:40:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.213+0000 7f9b0be00700 1 -- 172.21.15.90:0/1531054737 learned_addr learned my addr 172.21.15.90:0/1531054737 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:40:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.214+0000 7f9b11200700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b0c1045e0 0x7f9b0c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.214+0000 7f9b11c00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b0c109860 0x7f9b0c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:11.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.214+0000 7f9b0be00700 1 -- 172.21.15.90:0/1531054737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b0c1045e0 msgr2=0x7f9b0c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:11.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.214+0000 7f9b0be00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b0c1045e0 0x7f9b0c19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.214+0000 7f9b0be00700 1 -- 172.21.15.90:0/1531054737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b0c109860 msgr2=0x7f9b0c1a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.214+0000 7f9b0be00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b0c109860 0x7f9b0c1a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.214+0000 7f9b0be00700 1 -- 172.21.15.90:0/1531054737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c1a1ef0 con 0x7f9b0c108f10 2024-05-27T07:40:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.214+0000 7f9b11c00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b0c109860 0x7f9b0c1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.214+0000 7f9b0be00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b0c108f10 0x7f9b0c19d5c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9af800d980 tx=0x7f9af800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.215+0000 7f9b09600700 1 -- 172.21.15.90:0/1531054737 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8017070 con 0x7f9b0c108f10 2024-05-27T07:40:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.215+0000 7f9b09600700 1 -- 172.21.15.90:0/1531054737 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9af8010460 con 0x7f9b0c108f10 2024-05-27T07:40:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.215+0000 7f9b09600700 1 -- 172.21.15.90:0/1531054737 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af800de60 con 0x7f9b0c108f10 2024-05-27T07:40:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.215+0000 7f9b13c00700 1 -- 172.21.15.90:0/1531054737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c1a21e0 con 0x7f9b0c108f10 2024-05-27T07:40:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.215+0000 7f9b13c00700 1 -- 172.21.15.90:0/1531054737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0c1a9b00 con 0x7f9b0c108f10 2024-05-27T07:40:11.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.216+0000 7f9b13c00700 1 -- 172.21.15.90:0/1531054737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b0c0512b0 con 0x7f9b0c108f10 2024-05-27T07:40:11.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.221+0000 7f9b09600700 1 -- 172.21.15.90:0/1531054737 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9af8010610 con 0x7f9b0c108f10 2024-05-27T07:40:11.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.222+0000 7f9b09600700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f9af40b2900 0x7f9af40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:11.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.223+0000 7f9b09600700 1 -- 172.21.15.90:0/1531054737 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f9af80e18b0 con 0x7f9b0c108f10 2024-05-27T07:40:11.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.223+0000 7f9b11200700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f9af40b2900 0x7f9af40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:11.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.224+0000 7f9b11200700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f9af40b2900 0x7f9af40b4e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9b00000c00 tx=0x7f9b00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:11.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.225+0000 7f9b09600700 1 -- 172.21.15.90:0/1531054737 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9af80a9a50 con 0x7f9b0c108f10 2024-05-27T07:40:11.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:11.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:11.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:11.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:11.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:11.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:11.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: Deploying daemon mds.j on smithi090 2024-05-27T07:40:11.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:11.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] up:boot 2024-05-27T07:40:11.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: fsmap 2 up:standby 2024-05-27T07:40:11.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:40:11.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:11.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:11.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:11.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:11.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:11.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:11.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: Deploying daemon mds.j on smithi090 2024-05-27T07:40:11.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:11.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] up:boot 2024-05-27T07:40:11.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: fsmap 2 up:standby 2024-05-27T07:40:11.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:40:11.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:11.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:11.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:11.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:11.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:11.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:11.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: Deploying daemon mds.j on smithi090 2024-05-27T07:40:11.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:11.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] up:boot 2024-05-27T07:40:11.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: fsmap 2 up:standby 2024-05-27T07:40:11.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:40:11.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.512+0000 7f9b13c00700 1 -- 172.21.15.90:0/1531054737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b0c19e380 con 0x7f9b0c108f10 2024-05-27T07:40:11.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.513+0000 7f9b09600700 1 -- 172.21.15.90:0/1531054737 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f9af80a9350 con 0x7f9b0c108f10 2024-05-27T07:40:11.511 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:40:11.511 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T07:40:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.516+0000 7f9b13c00700 1 -- 172.21.15.90:0/1531054737 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f9af40b2900 msgr2=0x7f9af40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.516+0000 7f9b13c00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f9af40b2900 0x7f9af40b4e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9b00000c00 tx=0x7f9b00005fb0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.516+0000 7f9b13c00700 1 -- 172.21.15.90:0/1531054737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b0c108f10 msgr2=0x7f9b0c19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.516+0000 7f9b13c00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b0c108f10 0x7f9b0c19d5c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9af800d980 tx=0x7f9af800dd40 comp rx=0 tx=0).stop 2024-05-27T07:40:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.517+0000 7f9b13c00700 1 -- 172.21.15.90:0/1531054737 shutdown_connections 2024-05-27T07:40:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.517+0000 7f9b13c00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f9af40b2900 0x7f9af40b4e10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.517+0000 7f9b13c00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b0c109860 0x7f9b0c1a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.517+0000 7f9b13c00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b0c1045e0 0x7f9b0c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.517+0000 7f9b13c00700 1 --2- 172.21.15.90:0/1531054737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b0c108f10 0x7f9b0c19d5c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.517+0000 7f9b13c00700 1 -- 172.21.15.90:0/1531054737 >> 172.21.15.90:0/1531054737 conn(0x7f9b0c100030 msgr2=0x7f9b0c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.517+0000 7f9b13c00700 1 -- 172.21.15.90:0/1531054737 shutdown_connections 2024-05-27T07:40:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:11.517+0000 7f9b13c00700 1 -- 172.21.15.90:0/1531054737 wait complete. 2024-05-27T07:40:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 3 2024-05-27T07:40:12.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:12 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1531054737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:12.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:12 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1531054737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:12.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:12 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1531054737' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:13.399 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:13 smithi136 ceph-mon[35324]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:13.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:13 smithi090 ceph-mon[31429]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:13.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:13 smithi129 ceph-mon[35254]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:13.710 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:40:13 smithi090 systemd[1]: Starting Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:14.154 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:40:13 smithi090 podman[71000]: 2024-05-27 07:40:13.77417348 +0000 UTC m=+0.034884097 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:14.155 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:40:13 smithi090 podman[71000]: 2024-05-27 07:40:13.87618422 +0000 UTC m=+0.136894870 container create 164daf34dda43fce4b9f3a8a26cc2a57dc18bb8024218ae9224b1e98567eb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-27T07:40:14.155 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:40:14 smithi090 podman[71000]: 2024-05-27 07:40:14.155636017 +0000 UTC m=+0.416346737 container init 164daf34dda43fce4b9f3a8a26cc2a57dc18bb8024218ae9224b1e98567eb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:40:14.460 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:40:14 smithi090 podman[71000]: 2024-05-27 07:40:14.168462518 +0000 UTC m=+0.429173161 container start 164daf34dda43fce4b9f3a8a26cc2a57dc18bb8024218ae9224b1e98567eb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T07:40:14.461 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:40:14 smithi090 bash[71000]: 164daf34dda43fce4b9f3a8a26cc2a57dc18bb8024218ae9224b1e98567eb554 2024-05-27T07:40:14.461 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:40:14 smithi090 systemd[1]: Started Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:14.461 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:40:14 smithi090 ceph-mds[71019]: -- 172.21.15.90:0/1147054367 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc5b671a40 con 0x55cc5b6b9a80 2024-05-27T07:40:14.461 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:40:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: starting mds.j at 2024-05-27T07:40:15.344 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:40:15.356 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:15.356 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6834/2091177441,v1:172.21.15.90:6835/2091177441] up:boot 2024-05-27T07:40:15.356 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: fsmap 3 up:standby 2024-05-27T07:40:15.356 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:40:15.357 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:15.357 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:15.357 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:15.357 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:15.357 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:15.357 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:15.357 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:15 smithi136 ceph-mon[35324]: Deploying daemon mds.d on smithi090 2024-05-27T07:40:15.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:15.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6834/2091177441,v1:172.21.15.90:6835/2091177441] up:boot 2024-05-27T07:40:15.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: fsmap 3 up:standby 2024-05-27T07:40:15.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:40:15.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:15.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:15.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:15.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:15.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:15.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:15.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:15 smithi090 ceph-mon[31429]: Deploying daemon mds.d on smithi090 2024-05-27T07:40:15.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:15.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6834/2091177441,v1:172.21.15.90:6835/2091177441] up:boot 2024-05-27T07:40:15.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: fsmap 3 up:standby 2024-05-27T07:40:15.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:40:15.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:15.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:15.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:15.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:15.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:15.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:15.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:15 smithi129 ceph-mon[35254]: Deploying daemon mds.d on smithi090 2024-05-27T07:40:16.668 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:17.361 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:40:17 smithi090 systemd[1]: Starting Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:17.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:17 smithi090 ceph-mon[31429]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:17.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:17 smithi136 ceph-mon[35324]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:17.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:17 smithi129 ceph-mon[35254]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:18.501 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:40:18 smithi090 podman[71400]: 2024-05-27 07:40:18.135294137 +0000 UTC m=+0.036320182 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:18.838 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:40:18 smithi090 podman[71400]: 2024-05-27 07:40:18.548445331 +0000 UTC m=+0.449471450 container create 54c7cd984db6c32eaba3cb732331877a47c71c78c58cd937fa4ac97944b25443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-27T07:40:18.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.896+0000 7f34df200700 1 -- 172.21.15.90:0/1374898849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e0109670 msgr2=0x7f34e0110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:18.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.896+0000 7f34df200700 1 --2- 172.21.15.90:0/1374898849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e0109670 0x7f34e0110fd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f34d800b0d0 tx=0x7f34d800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:40:18.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.896+0000 7f34df200700 1 -- 172.21.15.90:0/1374898849 shutdown_connections 2024-05-27T07:40:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.896+0000 7f34df200700 1 --2- 172.21.15.90:0/1374898849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e0109670 0x7f34e0110fd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.896+0000 7f34df200700 1 --2- 172.21.15.90:0/1374898849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e01043f0 0x7f34e0104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.896+0000 7f34df200700 1 --2- 172.21.15.90:0/1374898849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e0108d20 0x7f34e0109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.896+0000 7f34df200700 1 -- 172.21.15.90:0/1374898849 >> 172.21.15.90:0/1374898849 conn(0x7f34e0079d00 msgr2=0x7f34e007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.896+0000 7f34df200700 1 -- 172.21.15.90:0/1374898849 shutdown_connections 2024-05-27T07:40:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.896+0000 7f34df200700 1 -- 172.21.15.90:0/1374898849 wait complete. 2024-05-27T07:40:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.897+0000 7f34df200700 1 Processor -- start 2024-05-27T07:40:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.897+0000 7f34df200700 1 -- start start 2024-05-27T07:40:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.897+0000 7f34df200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e01043f0 0x7f34e019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34df200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e0108d20 0x7f34e019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34df200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0109670 0x7f34e01a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34df200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01141a0 con 0x7f34e01043f0 2024-05-27T07:40:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34df200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0114020 con 0x7f34e0108d20 2024-05-27T07:40:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34df200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0114320 con 0x7f34e0109670 2024-05-27T07:40:18.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34dde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e01043f0 0x7f34e019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:18.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34dde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e01043f0 0x7f34e019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44310/0 (socket says 172.21.15.90:44310) 2024-05-27T07:40:18.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34dde00700 1 -- 172.21.15.90:0/3114163763 learned_addr learned my addr 172.21.15.90:0/3114163763 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:40:18.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34dd400700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e0108d20 0x7f34e019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:18.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34dde00700 1 -- 172.21.15.90:0/3114163763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0109670 msgr2=0x7f34e01a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:18.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34dde00700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0109670 0x7f34e01a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:18.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34dde00700 1 -- 172.21.15.90:0/3114163763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e0108d20 msgr2=0x7f34e019d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:18.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34dde00700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e0108d20 0x7f34e019d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:18.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34dde00700 1 -- 172.21.15.90:0/3114163763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e01a1c40 con 0x7f34e01043f0 2024-05-27T07:40:18.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.898+0000 7f34dd400700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e0108d20 0x7f34e019d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:18.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.899+0000 7f34dde00700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e01043f0 0x7f34e019cd70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f34d000c920 tx=0x7f34d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:18.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.899+0000 7f34d6a00700 1 -- 172.21.15.90:0/3114163763 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0007b00 con 0x7f34e01043f0 2024-05-27T07:40:18.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.899+0000 7f34d6a00700 1 -- 172.21.15.90:0/3114163763 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34d0004cc0 con 0x7f34e01043f0 2024-05-27T07:40:18.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.899+0000 7f34d6a00700 1 -- 172.21.15.90:0/3114163763 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d00054f0 con 0x7f34e01043f0 2024-05-27T07:40:18.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.899+0000 7f34df200700 1 -- 172.21.15.90:0/3114163763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e01a1f30 con 0x7f34e01043f0 2024-05-27T07:40:18.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.899+0000 7f34df200700 1 -- 172.21.15.90:0/3114163763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e01a9850 con 0x7f34e01043f0 2024-05-27T07:40:18.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.900+0000 7f34df200700 1 -- 172.21.15.90:0/3114163763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e01058a0 con 0x7f34e01043f0 2024-05-27T07:40:18.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.905+0000 7f34d6a00700 1 -- 172.21.15.90:0/3114163763 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f34d0007ca0 con 0x7f34e01043f0 2024-05-27T07:40:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.906+0000 7f34d6a00700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34c40b2900 0x7f34c40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:18.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.906+0000 7f34d6a00700 1 -- 172.21.15.90:0/3114163763 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f34d00e1d20 con 0x7f34e01043f0 2024-05-27T07:40:18.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.907+0000 7f34dd400700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34c40b2900 0x7f34c40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:18.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.907+0000 7f34dd400700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34c40b2900 0x7f34c40b4e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34cc005fd0 tx=0x7f34cc005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:18.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:18.909+0000 7f34d6a00700 1 -- 172.21.15.90:0/3114163763 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f34d0014030 con 0x7f34e01043f0 2024-05-27T07:40:19.119 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:40:18 smithi090 podman[71400]: 2024-05-27 07:40:18.997713528 +0000 UTC m=+0.898739633 container init 54c7cd984db6c32eaba3cb732331877a47c71c78c58cd937fa4ac97944b25443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-05-27T07:40:19.119 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:40:19 smithi090 podman[71400]: 2024-05-27 07:40:19.007499042 +0000 UTC m=+0.908525131 container start 54c7cd984db6c32eaba3cb732331877a47c71c78c58cd937fa4ac97944b25443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-27T07:40:19.119 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:40:19 smithi090 bash[71400]: 54c7cd984db6c32eaba3cb732331877a47c71c78c58cd937fa4ac97944b25443 2024-05-27T07:40:19.119 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:40:19 smithi090 systemd[1]: Started Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:19.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.202+0000 7f34df200700 1 -- 172.21.15.90:0/3114163763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34e00512b0 con 0x7f34e01043f0 2024-05-27T07:40:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.205+0000 7f34d6a00700 1 -- 172.21.15.90:0/3114163763 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f34d00a9f60 con 0x7f34e01043f0 2024-05-27T07:40:19.204 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:40:19.204 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-27T07:40:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.208+0000 7f34df200700 1 -- 172.21.15.90:0/3114163763 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34c40b2900 msgr2=0x7f34c40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.208+0000 7f34df200700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34c40b2900 0x7f34c40b4e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f34cc005fd0 tx=0x7f34cc005d40 comp rx=0 tx=0).stop 2024-05-27T07:40:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.208+0000 7f34df200700 1 -- 172.21.15.90:0/3114163763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e01043f0 msgr2=0x7f34e019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.208+0000 7f34df200700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e01043f0 0x7f34e019cd70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f34d000c920 tx=0x7f34d000cce0 comp rx=0 tx=0).stop 2024-05-27T07:40:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.208+0000 7f34df200700 1 -- 172.21.15.90:0/3114163763 shutdown_connections 2024-05-27T07:40:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.208+0000 7f34df200700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34c40b2900 0x7f34c40b4e10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.208+0000 7f34df200700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34e0109670 0x7f34e01a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.208+0000 7f34df200700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34e01043f0 0x7f34e019cd70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.208+0000 7f34df200700 1 --2- 172.21.15.90:0/3114163763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e0108d20 0x7f34e019d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:19.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.208+0000 7f34df200700 1 -- 172.21.15.90:0/3114163763 >> 172.21.15.90:0/3114163763 conn(0x7f34e0079d00 msgr2=0x7f34e0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:19.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.209+0000 7f34df200700 1 -- 172.21.15.90:0/3114163763 shutdown_connections 2024-05-27T07:40:19.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:19.209+0000 7f34df200700 1 -- 172.21.15.90:0/3114163763 wait complete. 2024-05-27T07:40:19.208 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 4 2024-05-27T07:40:19.377 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:40:19 smithi090 ceph-mds[71457]: -- 172.21.15.90:0/2857108526 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567ce13da40 con 0x5567ce185600 2024-05-27T07:40:19.377 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:40:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: starting mds.d at 2024-05-27T07:40:19.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:19 smithi090 ceph-mon[31429]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:19.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:19.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:19 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3114163763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:19.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:19.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:19.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:19.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:19.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:19 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:19 smithi136 ceph-mon[35324]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:19 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:19 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3114163763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:19 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:19 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:19 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:19 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:19.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:19 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:19.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:19 smithi129 ceph-mon[35254]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:19.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:19.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:19 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3114163763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:19.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:19.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:19.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:19.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:19.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:19 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:20.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:20 smithi129 ceph-mon[35254]: Deploying daemon mds.e on smithi129 2024-05-27T07:40:20.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:20 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6836/3818984519,v1:172.21.15.90:6837/3818984519] up:boot 2024-05-27T07:40:20.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:20 smithi129 ceph-mon[35254]: fsmap 4 up:standby 2024-05-27T07:40:20.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:20 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:40:20.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:20 smithi090 ceph-mon[31429]: Deploying daemon mds.e on smithi129 2024-05-27T07:40:20.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:20 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6836/3818984519,v1:172.21.15.90:6837/3818984519] up:boot 2024-05-27T07:40:20.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:20 smithi090 ceph-mon[31429]: fsmap 4 up:standby 2024-05-27T07:40:20.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:20 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:40:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:20 smithi136 ceph-mon[35324]: Deploying daemon mds.e on smithi129 2024-05-27T07:40:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:20 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6836/3818984519,v1:172.21.15.90:6837/3818984519] up:boot 2024-05-27T07:40:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:20 smithi136 ceph-mon[35324]: fsmap 4 up:standby 2024-05-27T07:40:20.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:20 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:40:21.561 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:21 smithi129 ceph-mon[35254]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:21.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:21 smithi090 ceph-mon[31429]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:21.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:21 smithi136 ceph-mon[35324]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:22.034 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:40:22.454 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:22.796 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 07:40:22 smithi129 systemd[1]: Starting Ceph mds.e for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:23.062 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 07:40:22 smithi129 podman[64670]: 2024-05-27 07:40:22.702758931 +0000 UTC m=+0.036311074 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:23.062 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 07:40:22 smithi129 podman[64670]: 2024-05-27 07:40:22.822442961 +0000 UTC m=+0.155995129 container create 7f2d3fbd6f3076203e80762f0382e6df5549ecf0f273dde6213361d36dd10378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-27T07:40:23.399 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 07:40:23 smithi129 podman[64670]: 2024-05-27 07:40:23.101617861 +0000 UTC m=+0.435170059 container init 7f2d3fbd6f3076203e80762f0382e6df5549ecf0f273dde6213361d36dd10378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-e, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T07:40:23.400 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 07:40:23 smithi129 podman[64670]: 2024-05-27 07:40:23.111283633 +0000 UTC m=+0.444835775 container start 7f2d3fbd6f3076203e80762f0382e6df5549ecf0f273dde6213361d36dd10378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:40:23.400 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 07:40:23 smithi129 ceph-mds[64689]: -- 172.21.15.129:0/1972034890 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55da90945a40 con 0x55da9098d600 2024-05-27T07:40:23.400 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 07:40:23 smithi129 bash[64670]: 7f2d3fbd6f3076203e80762f0382e6df5549ecf0f273dde6213361d36dd10378 2024-05-27T07:40:23.400 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 07:40:23 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-e[64685]: starting mds.e at 2024-05-27T07:40:23.400 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 07:40:23 smithi129 systemd[1]: Started Ceph mds.e for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:23.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:23 smithi090 ceph-mon[31429]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:23.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:23.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:23.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:23.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:23.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:23.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:23 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:23.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:23 smithi136 ceph-mon[35324]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:23.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:23.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:23.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:23.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:23.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:23.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:23 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:23.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:23 smithi129 ceph-mon[35254]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:23.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:23.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:23.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:23.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:23.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:23.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:23 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:24.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.633+0000 7fea6e000700 1 -- 172.21.15.90:0/2218781473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea68103a70 msgr2=0x7fea68110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:24.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.633+0000 7fea6e000700 1 --2- 172.21.15.90:0/2218781473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea68103a70 0x7fea68110700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fea54009990 tx=0x7fea54009ca0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.633+0000 7fea6e000700 1 -- 172.21.15.90:0/2218781473 shutdown_connections 2024-05-27T07:40:24.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.633+0000 7fea6e000700 1 --2- 172.21.15.90:0/2218781473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea68110c40 0x7fea68113060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.633+0000 7fea6e000700 1 --2- 172.21.15.90:0/2218781473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea68103090 0x7fea681034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.633+0000 7fea6e000700 1 --2- 172.21.15.90:0/2218781473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea68103a70 0x7fea68110700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.633+0000 7fea6e000700 1 -- 172.21.15.90:0/2218781473 >> 172.21.15.90:0/2218781473 conn(0x7fea680fef20 msgr2=0x7fea68101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:24.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.633+0000 7fea6e000700 1 -- 172.21.15.90:0/2218781473 shutdown_connections 2024-05-27T07:40:24.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.633+0000 7fea6e000700 1 -- 172.21.15.90:0/2218781473 wait complete. 2024-05-27T07:40:24.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.634+0000 7fea6e000700 1 Processor -- start 2024-05-27T07:40:24.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea6e000700 1 -- start start 2024-05-27T07:40:24.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea6e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea68103090 0x7fea6819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:24.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea6e000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea68103a70 0x7fea6819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:24.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea6e000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea68110c40 0x7fea681a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:24.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea6e000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fea68116570 con 0x7fea68103090 2024-05-27T07:40:24.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea6e000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fea681163f0 con 0x7fea68110c40 2024-05-27T07:40:24.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea6e000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea681166f0 con 0x7fea68103a70 2024-05-27T07:40:24.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea67400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea68103090 0x7fea6819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:24.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea67400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea68103090 0x7fea6819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44352/0 (socket says 172.21.15.90:44352) 2024-05-27T07:40:24.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea67400700 1 -- 172.21.15.90:0/1559028154 learned_addr learned my addr 172.21.15.90:0/1559028154 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:40:24.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.635+0000 7fea66a00700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea68103a70 0x7fea6819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:24.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.636+0000 7fea67e00700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea68110c40 0x7fea681a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:24.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.636+0000 7fea67400700 1 -- 172.21.15.90:0/1559028154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea68103a70 msgr2=0x7fea6819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:24.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.636+0000 7fea67400700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea68103a70 0x7fea6819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.636+0000 7fea67400700 1 -- 172.21.15.90:0/1559028154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea68110c40 msgr2=0x7fea681a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.636+0000 7fea67400700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea68110c40 0x7fea681a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.636+0000 7fea67400700 1 -- 172.21.15.90:0/1559028154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea681a1ed0 con 0x7fea68103090 2024-05-27T07:40:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.636+0000 7fea67e00700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea68110c40 0x7fea681a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.636+0000 7fea67400700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea68103090 0x7fea6819d000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c920 tx=0x7fea5800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.637+0000 7fea5fe00700 1 -- 172.21.15.90:0/1559028154 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58007b00 con 0x7fea68103090 2024-05-27T07:40:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.637+0000 7fea5fe00700 1 -- 172.21.15.90:0/1559028154 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea58004cc0 con 0x7fea68103090 2024-05-27T07:40:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.637+0000 7fea5fe00700 1 -- 172.21.15.90:0/1559028154 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58005440 con 0x7fea68103090 2024-05-27T07:40:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.637+0000 7fea6e000700 1 -- 172.21.15.90:0/1559028154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea681a21c0 con 0x7fea68103090 2024-05-27T07:40:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.637+0000 7fea6e000700 1 -- 172.21.15.90:0/1559028154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea681a9ae0 con 0x7fea68103090 2024-05-27T07:40:24.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.638+0000 7fea6e000700 1 -- 172.21.15.90:0/1559028154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea6806c910 con 0x7fea68103090 2024-05-27T07:40:24.638 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:24 smithi136 ceph-mon[35324]: Deploying daemon mds.f on smithi136 2024-05-27T07:40:24.638 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:24 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] up:boot 2024-05-27T07:40:24.639 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:24 smithi136 ceph-mon[35324]: fsmap 5 up:standby 2024-05-27T07:40:24.639 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:24 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:40:24.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.641+0000 7fea5fe00700 1 -- 172.21.15.90:0/1559028154 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fea58004750 con 0x7fea68103090 2024-05-27T07:40:24.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.642+0000 7fea5fe00700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fea4c0b2900 0x7fea4c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:24.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:24 smithi090 ceph-mon[31429]: Deploying daemon mds.f on smithi136 2024-05-27T07:40:24.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:24 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] up:boot 2024-05-27T07:40:24.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:24 smithi090 ceph-mon[31429]: fsmap 5 up:standby 2024-05-27T07:40:24.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:24 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:40:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.643+0000 7fea5fe00700 1 -- 172.21.15.90:0/1559028154 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fea580e1d20 con 0x7fea68103090 2024-05-27T07:40:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.643+0000 7fea66a00700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fea4c0b2900 0x7fea4c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:24.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.643+0000 7fea66a00700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fea4c0b2900 0x7fea4c0b4e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fea5400b5c0 tx=0x7fea54005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.645+0000 7fea5fe00700 1 -- 172.21.15.90:0/1559028154 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea58014030 con 0x7fea68103090 2024-05-27T07:40:24.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:24 smithi129 ceph-mon[35254]: Deploying daemon mds.f on smithi136 2024-05-27T07:40:24.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:24 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] up:boot 2024-05-27T07:40:24.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:24 smithi129 ceph-mon[35254]: fsmap 5 up:standby 2024-05-27T07:40:24.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:24 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:40:24.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.926+0000 7fea6e000700 1 -- 172.21.15.90:0/1559028154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea6819e360 con 0x7fea68103090 2024-05-27T07:40:24.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.927+0000 7fea5fe00700 1 -- 172.21.15.90:0/1559028154 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7fea580a9ec0 con 0x7fea68103090 2024-05-27T07:40:24.926 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:40:24.927 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-27T07:40:24.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.931+0000 7fea6e000700 1 -- 172.21.15.90:0/1559028154 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fea4c0b2900 msgr2=0x7fea4c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:24.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.931+0000 7fea6e000700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fea4c0b2900 0x7fea4c0b4e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fea5400b5c0 tx=0x7fea54005fd0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.931+0000 7fea6e000700 1 -- 172.21.15.90:0/1559028154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea68103090 msgr2=0x7fea6819d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:24.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.931+0000 7fea6e000700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea68103090 0x7fea6819d000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea5800c920 tx=0x7fea5800cc30 comp rx=0 tx=0).stop 2024-05-27T07:40:24.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.931+0000 7fea6e000700 1 -- 172.21.15.90:0/1559028154 shutdown_connections 2024-05-27T07:40:24.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.931+0000 7fea6e000700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fea4c0b2900 0x7fea4c0b4e10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.931+0000 7fea6e000700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea68110c40 0x7fea681a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.931+0000 7fea6e000700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea68103090 0x7fea6819d000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.931+0000 7fea6e000700 1 --2- 172.21.15.90:0/1559028154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea68103a70 0x7fea6819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:24.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.931+0000 7fea6e000700 1 -- 172.21.15.90:0/1559028154 >> 172.21.15.90:0/1559028154 conn(0x7fea680fef20 msgr2=0x7fea68112900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:24.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.932+0000 7fea6e000700 1 -- 172.21.15.90:0/1559028154 shutdown_connections 2024-05-27T07:40:24.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:24.932+0000 7fea6e000700 1 -- 172.21.15.90:0/1559028154 wait complete. 2024-05-27T07:40:24.932 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 6 2024-05-27T07:40:25.654 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:25 smithi136 ceph-mon[35324]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:25.654 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:25 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1559028154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:25.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:25 smithi090 ceph-mon[31429]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:25.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:25 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1559028154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:25.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:25 smithi129 ceph-mon[35254]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:25.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:25 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1559028154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:26.747 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 07:40:26 smithi136 systemd[1]: Starting Ceph mds.f for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:27.146 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 07:40:26 smithi136 podman[65717]: 2024-05-27 07:40:26.726086446 +0000 UTC m=+0.034761918 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:27.146 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 07:40:26 smithi136 podman[65717]: 2024-05-27 07:40:26.850636394 +0000 UTC m=+0.159311918 container create 1f254d421592e4d00f769a07edca15cd7a2a75996d8fc2b8bd6980af270641d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:40:27.146 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 07:40:27 smithi136 podman[65717]: 2024-05-27 07:40:27.146398795 +0000 UTC m=+0.455074346 container init 1f254d421592e4d00f769a07edca15cd7a2a75996d8fc2b8bd6980af270641d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:40:27.452 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 07:40:27 smithi136 podman[65717]: 2024-05-27 07:40:27.157768374 +0000 UTC m=+0.466443904 container start 1f254d421592e4d00f769a07edca15cd7a2a75996d8fc2b8bd6980af270641d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-27T07:40:27.452 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 07:40:27 smithi136 bash[65717]: 1f254d421592e4d00f769a07edca15cd7a2a75996d8fc2b8bd6980af270641d2 2024-05-27T07:40:27.453 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 07:40:27 smithi136 systemd[1]: Started Ceph mds.f for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:27.453 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 07:40:27 smithi136 ceph-mds[65736]: -- 172.21.15.136:0/2154677109 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8073d1a40 con 0x55b807419600 2024-05-27T07:40:27.453 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 07:40:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-f[65732]: starting mds.f at 2024-05-27T07:40:27.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:27 smithi090 ceph-mon[31429]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:27.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:27.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:27.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:27.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:27.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:27 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:27.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:27 smithi136 ceph-mon[35324]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:27.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:27.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:27.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:27.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:27.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:27 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:27.810 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:40:27.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:27 smithi129 ceph-mon[35254]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:27.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:27.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:27.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:27.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:27.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:27 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:28.271 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:28.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:28 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:28.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:28 smithi090 ceph-mon[31429]: Deploying daemon mds.c on smithi136 2024-05-27T07:40:28.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:28 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.136:6834/1419480441,v1:172.21.15.136:6835/1419480441] up:boot 2024-05-27T07:40:28.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:28 smithi090 ceph-mon[31429]: fsmap 6 up:standby 2024-05-27T07:40:28.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:28 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:40:28.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:28 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:28.718 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:28 smithi136 ceph-mon[35324]: Deploying daemon mds.c on smithi136 2024-05-27T07:40:28.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:28 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.136:6834/1419480441,v1:172.21.15.136:6835/1419480441] up:boot 2024-05-27T07:40:28.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:28 smithi136 ceph-mon[35324]: fsmap 6 up:standby 2024-05-27T07:40:28.719 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:28 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:40:28.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:28 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:28.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:28 smithi129 ceph-mon[35254]: Deploying daemon mds.c on smithi136 2024-05-27T07:40:28.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:28 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.136:6834/1419480441,v1:172.21.15.136:6835/1419480441] up:boot 2024-05-27T07:40:28.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:28 smithi129 ceph-mon[35254]: fsmap 6 up:standby 2024-05-27T07:40:28.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:28 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:40:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:29 smithi090 ceph-mon[31429]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:29.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:29 smithi136 ceph-mon[35324]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:29.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:29 smithi129 ceph-mon[35254]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:30.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.510+0000 7f6ac1200700 1 -- 172.21.15.90:0/4043423358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6abc106eb0 msgr2=0x7f6abc113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:30.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.510+0000 7f6ac1200700 1 --2- 172.21.15.90:0/4043423358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6abc106eb0 0x7f6abc113340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400b0a0 tx=0x7f6ab400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.510+0000 7f6ac1200700 1 -- 172.21.15.90:0/4043423358 shutdown_connections 2024-05-27T07:40:30.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.510+0000 7f6ac1200700 1 --2- 172.21.15.90:0/4043423358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6abc106eb0 0x7f6abc113340 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.510+0000 7f6ac1200700 1 --2- 172.21.15.90:0/4043423358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6abc10c5a0 0x7f6abc10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.510+0000 7f6ac1200700 1 --2- 172.21.15.90:0/4043423358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6abc106560 0x7f6abc106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.510+0000 7f6ac1200700 1 -- 172.21.15.90:0/4043423358 >> 172.21.15.90:0/4043423358 conn(0x7f6abc102040 msgr2=0x7f6abc104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:30.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.510+0000 7f6ac1200700 1 -- 172.21.15.90:0/4043423358 shutdown_connections 2024-05-27T07:40:30.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.510+0000 7f6ac1200700 1 -- 172.21.15.90:0/4043423358 wait complete. 2024-05-27T07:40:30.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.511+0000 7f6ac1200700 1 Processor -- start 2024-05-27T07:40:30.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.512+0000 7f6ac1200700 1 -- start start 2024-05-27T07:40:30.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.512+0000 7f6ac1200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6abc106560 0x7f6abc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.512+0000 7f6ac1200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6abc106eb0 0x7f6abc19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.512+0000 7f6ac1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6abc10c5a0 0x7f6abc1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.512+0000 7f6ac1200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc116580 con 0x7f6abc10c5a0 2024-05-27T07:40:30.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.512+0000 7f6ac1200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc116400 con 0x7f6abc106560 2024-05-27T07:40:30.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.512+0000 7f6ac1200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc116700 con 0x7f6abc106eb0 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.512+0000 7f6aba800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6abc10c5a0 0x7f6abc1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.512+0000 7f6aba800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6abc10c5a0 0x7f6abc1a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34376/0 (socket says 172.21.15.90:34376) 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.512+0000 7f6aba800700 1 -- 172.21.15.90:0/2368305161 learned_addr learned my addr 172.21.15.90:0/2368305161 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.513+0000 7f6ab9400700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6abc106eb0 0x7f6abc19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.513+0000 7f6ab9e00700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6abc106560 0x7f6abc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.513+0000 7f6aba800700 1 -- 172.21.15.90:0/2368305161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6abc106eb0 msgr2=0x7f6abc19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.513+0000 7f6aba800700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6abc106eb0 0x7f6abc19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.513+0000 7f6aba800700 1 -- 172.21.15.90:0/2368305161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6abc106560 msgr2=0x7f6abc19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.513+0000 7f6aba800700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6abc106560 0x7f6abc19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.513+0000 7f6aba800700 1 -- 172.21.15.90:0/2368305161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6abc1a1d40 con 0x7f6abc10c5a0 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.513+0000 7f6ab9e00700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6abc106560 0x7f6abc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.513+0000 7f6aba800700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6abc10c5a0 0x7f6abc1a16a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400b830 tx=0x7f6ab4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:30.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.514+0000 7f6ab2a00700 1 -- 172.21.15.90:0/2368305161 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4021070 con 0x7f6abc10c5a0 2024-05-27T07:40:30.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.514+0000 7f6ab2a00700 1 -- 172.21.15.90:0/2368305161 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ab4007b80 con 0x7f6abc10c5a0 2024-05-27T07:40:30.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.514+0000 7f6ab2a00700 1 -- 172.21.15.90:0/2368305161 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40052a0 con 0x7f6abc10c5a0 2024-05-27T07:40:30.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.514+0000 7f6ac1200700 1 -- 172.21.15.90:0/2368305161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6abc1a1fd0 con 0x7f6abc10c5a0 2024-05-27T07:40:30.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.514+0000 7f6ac1200700 1 -- 172.21.15.90:0/2368305161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6abc1a2460 con 0x7f6abc10c5a0 2024-05-27T07:40:30.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.515+0000 7f6ac1200700 1 -- 172.21.15.90:0/2368305161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6abc10e960 con 0x7f6abc10c5a0 2024-05-27T07:40:30.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.518+0000 7f6ab2a00700 1 -- 172.21.15.90:0/2368305161 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6ab4024070 con 0x7f6abc10c5a0 2024-05-27T07:40:30.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.519+0000 7f6ab2a00700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6aa00b2900 0x7f6aa00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:30.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.520+0000 7f6ab2a00700 1 -- 172.21.15.90:0/2368305161 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f6ab400f020 con 0x7f6abc10c5a0 2024-05-27T07:40:30.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.520+0000 7f6ab9e00700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6aa00b2900 0x7f6aa00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:30.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.520+0000 7f6ab9e00700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6aa00b2900 0x7f6aa00b4e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6aac005e10 tx=0x7f6aac005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:30.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.523+0000 7f6ab2a00700 1 -- 172.21.15.90:0/2368305161 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ab402b020 con 0x7f6abc10c5a0 2024-05-27T07:40:30.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.802+0000 7f6ac1200700 1 -- 172.21.15.90:0/2368305161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6abc19e1d0 con 0x7f6abc10c5a0 2024-05-27T07:40:30.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.803+0000 7f6ab2a00700 1 -- 172.21.15.90:0/2368305161 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f6ab40ab9a0 con 0x7f6abc10c5a0 2024-05-27T07:40:30.801 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:40:30.801 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-27T07:40:30.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.806+0000 7f6ac1200700 1 -- 172.21.15.90:0/2368305161 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6aa00b2900 msgr2=0x7f6aa00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:30.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.806+0000 7f6ac1200700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6aa00b2900 0x7f6aa00b4e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6aac005e10 tx=0x7f6aac005d60 comp rx=0 tx=0).stop 2024-05-27T07:40:30.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.806+0000 7f6ac1200700 1 -- 172.21.15.90:0/2368305161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6abc10c5a0 msgr2=0x7f6abc1a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:30.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.806+0000 7f6ac1200700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6abc10c5a0 0x7f6abc1a16a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400b830 tx=0x7f6ab4004d00 comp rx=0 tx=0).stop 2024-05-27T07:40:30.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.807+0000 7f6ac1200700 1 -- 172.21.15.90:0/2368305161 shutdown_connections 2024-05-27T07:40:30.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.807+0000 7f6ac1200700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6aa00b2900 0x7f6aa00b4e10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.807+0000 7f6ac1200700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6abc10c5a0 0x7f6abc1a16a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.807+0000 7f6ac1200700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6abc106560 0x7f6abc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.807+0000 7f6ac1200700 1 --2- 172.21.15.90:0/2368305161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6abc106eb0 0x7f6abc19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:30.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.807+0000 7f6ac1200700 1 -- 172.21.15.90:0/2368305161 >> 172.21.15.90:0/2368305161 conn(0x7f6abc102040 msgr2=0x7f6abc1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:30.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.807+0000 7f6ac1200700 1 -- 172.21.15.90:0/2368305161 shutdown_connections 2024-05-27T07:40:30.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:30.807+0000 7f6ac1200700 1 -- 172.21.15.90:0/2368305161 wait complete. 2024-05-27T07:40:30.806 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 7 2024-05-27T07:40:31.137 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:40:30 smithi136 systemd[1]: Starting Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:31.459 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:40:31 smithi136 podman[66017]: 2024-05-27 07:40:31.044506393 +0000 UTC m=+0.036629169 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:31.459 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:40:31 smithi136 podman[66017]: 2024-05-27 07:40:31.157508703 +0000 UTC m=+0.149631512 container create 5faa54b04d554c51f7adc1353246546154ace0d76017d9cba59763a1fcc84cc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:40:31.459 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:40:31 smithi136 podman[66017]: 2024-05-27 07:40:31.461293287 +0000 UTC m=+0.453416157 container init 5faa54b04d554c51f7adc1353246546154ace0d76017d9cba59763a1fcc84cc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-27T07:40:31.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:31 smithi136 ceph-mon[35324]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:31.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:31 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2368305161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:31.747 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:40:31 smithi136 podman[66017]: 2024-05-27 07:40:31.471326892 +0000 UTC m=+0.463449672 container start 5faa54b04d554c51f7adc1353246546154ace0d76017d9cba59763a1fcc84cc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS) 2024-05-27T07:40:31.747 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:40:31 smithi136 bash[66017]: 5faa54b04d554c51f7adc1353246546154ace0d76017d9cba59763a1fcc84cc9 2024-05-27T07:40:31.747 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:40:31 smithi136 systemd[1]: Started Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:31.747 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:40:31 smithi136 ceph-mds[66035]: -- 172.21.15.136:0/1687207367 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556674e81a40 con 0x556674ec9600 2024-05-27T07:40:31.748 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:40:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: starting mds.c at 2024-05-27T07:40:31.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:31 smithi129 ceph-mon[35254]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:31.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:31 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2368305161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:31.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:31 smithi090 ceph-mon[31429]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:31.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:31 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2368305161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:32.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:32 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:32.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:32 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:32.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:32 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:32.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:32 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:32.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:32 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:32.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:32 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:32.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:32 smithi090 ceph-mon[31429]: Deploying daemon mds.h on smithi129 2024-05-27T07:40:32.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:32 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.136:6836/3523286296,v1:172.21.15.136:6837/3523286296] up:boot 2024-05-27T07:40:32.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:32 smithi090 ceph-mon[31429]: fsmap 7 up:standby 2024-05-27T07:40:32.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:32 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:40:32.990 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:32 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:32.990 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:32 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:32.991 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:32 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:32.991 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:32 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:32.991 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:32 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:32.991 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:32 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:32.991 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:32 smithi129 ceph-mon[35254]: Deploying daemon mds.h on smithi129 2024-05-27T07:40:32.991 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:32 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.136:6836/3523286296,v1:172.21.15.136:6837/3523286296] up:boot 2024-05-27T07:40:32.991 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:32 smithi129 ceph-mon[35254]: fsmap 7 up:standby 2024-05-27T07:40:32.991 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:32 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:40:32.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:32 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:32.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:32 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:32.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:32 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:32.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:32 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:32.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:32 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:32.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:32 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:32.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:32 smithi136 ceph-mon[35324]: Deploying daemon mds.h on smithi129 2024-05-27T07:40:32.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:32 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.136:6836/3523286296,v1:172.21.15.136:6837/3523286296] up:boot 2024-05-27T07:40:32.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:32 smithi136 ceph-mon[35324]: fsmap 7 up:standby 2024-05-27T07:40:32.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:32 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:40:33.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:40:33.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:33 smithi136 ceph-mon[35324]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:34.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:33 smithi129 ceph-mon[35254]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:34.116 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:34.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:33 smithi090 ceph-mon[31429]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:35.311 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 07:40:35 smithi129 systemd[1]: Starting Ceph mds.h for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:35.706 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 07:40:35 smithi129 podman[64974]: 2024-05-27 07:40:35.295522601 +0000 UTC m=+0.031925846 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:35.707 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 07:40:35 smithi129 podman[64974]: 2024-05-27 07:40:35.419754381 +0000 UTC m=+0.156157663 container create 37f5f08697cf618a164706ae58025204dcbca4b7d03561aae8e7f82b61a20aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-h, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-27T07:40:35.707 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 07:40:35 smithi129 podman[64974]: 2024-05-27 07:40:35.70731273 +0000 UTC m=+0.443716046 container init 37f5f08697cf618a164706ae58025204dcbca4b7d03561aae8e7f82b61a20aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-h, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:40:35.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:35 smithi136 ceph-mon[35324]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:36.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:35 smithi090 ceph-mon[31429]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:36.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:35 smithi129 ceph-mon[35254]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:36.062 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 07:40:35 smithi129 podman[64974]: 2024-05-27 07:40:35.717297642 +0000 UTC m=+0.453700941 container start 37f5f08697cf618a164706ae58025204dcbca4b7d03561aae8e7f82b61a20aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-h, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:40:36.062 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 07:40:35 smithi129 bash[64974]: 37f5f08697cf618a164706ae58025204dcbca4b7d03561aae8e7f82b61a20aaa 2024-05-27T07:40:36.062 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 07:40:35 smithi129 systemd[1]: Started Ceph mds.h for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:36.062 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 07:40:35 smithi129 ceph-mds[64993]: -- 172.21.15.129:0/1199217082 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56391e209a40 con 0x56391e251600 2024-05-27T07:40:36.062 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 07:40:35 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-h[64989]: starting mds.h at 2024-05-27T07:40:36.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.292+0000 7f20a9800700 1 -- 172.21.15.90:0/1475540056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20a4076670 msgr2=0x7f20a4110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:36.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.292+0000 7f20a9800700 1 --2- 172.21.15.90:0/1475540056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20a4076670 0x7f20a4110460 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f208c009990 tx=0x7f208c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.292+0000 7f20a9800700 1 -- 172.21.15.90:0/1475540056 shutdown_connections 2024-05-27T07:40:36.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.292+0000 7f20a9800700 1 --2- 172.21.15.90:0/1475540056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20a41109a0 0x7f20a4112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.292+0000 7f20a9800700 1 --2- 172.21.15.90:0/1475540056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a4075d20 0x7f20a4076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.292+0000 7f20a9800700 1 --2- 172.21.15.90:0/1475540056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20a4076670 0x7f20a4110460 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.292+0000 7f20a9800700 1 -- 172.21.15.90:0/1475540056 >> 172.21.15.90:0/1475540056 conn(0x7f20a4079d00 msgr2=0x7f20a407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.293+0000 7f20a9800700 1 -- 172.21.15.90:0/1475540056 shutdown_connections 2024-05-27T07:40:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.293+0000 7f20a9800700 1 -- 172.21.15.90:0/1475540056 wait complete. 2024-05-27T07:40:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.294+0000 7f20a9800700 1 Processor -- start 2024-05-27T07:40:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.294+0000 7f20a9800700 1 -- start start 2024-05-27T07:40:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a9800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20a4075d20 0x7f20a419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a9800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20a4076670 0x7f20a419d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a9800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a41109a0 0x7f20a41a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a9800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4116350 con 0x7f20a4075d20 2024-05-27T07:40:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a9800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41161d0 con 0x7f20a4076670 2024-05-27T07:40:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a9800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41164d0 con 0x7f20a41109a0 2024-05-27T07:40:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20a4075d20 0x7f20a419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20a4075d20 0x7f20a419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34386/0 (socket says 172.21.15.90:34386) 2024-05-27T07:40:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a3e00700 1 -- 172.21.15.90:0/2144144520 learned_addr learned my addr 172.21.15.90:0/2144144520 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:40:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a8e00700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a41109a0 0x7f20a41a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f209b400700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20a4076670 0x7f20a419d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a3e00700 1 -- 172.21.15.90:0/2144144520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a41109a0 msgr2=0x7f20a41a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a3e00700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a41109a0 0x7f20a41a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a3e00700 1 -- 172.21.15.90:0/2144144520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20a4076670 msgr2=0x7f20a419d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a3e00700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20a4076670 0x7f20a419d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a3e00700 1 -- 172.21.15.90:0/2144144520 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a41a1bc0 con 0x7f20a4075d20 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.295+0000 7f20a8e00700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a41109a0 0x7f20a41a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.296+0000 7f20a3e00700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20a4075d20 0x7f20a419cda0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f209400d980 tx=0x7f209400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.296+0000 7f20a1600700 1 -- 172.21.15.90:0/2144144520 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094016070 con 0x7f20a4075d20 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.296+0000 7f20a1600700 1 -- 172.21.15.90:0/2144144520 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2094004020 con 0x7f20a4075d20 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.296+0000 7f20a9800700 1 -- 172.21.15.90:0/2144144520 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a41a1eb0 con 0x7f20a4075d20 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.296+0000 7f20a1600700 1 -- 172.21.15.90:0/2144144520 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094004bb0 con 0x7f20a4075d20 2024-05-27T07:40:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.296+0000 7f20a9800700 1 -- 172.21.15.90:0/2144144520 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a41a23f0 con 0x7f20a4075d20 2024-05-27T07:40:36.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.297+0000 7f20a9800700 1 -- 172.21.15.90:0/2144144520 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a4077290 con 0x7f20a4075d20 2024-05-27T07:40:36.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.302+0000 7f20a1600700 1 -- 172.21.15.90:0/2144144520 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f209400bb60 con 0x7f20a4075d20 2024-05-27T07:40:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.303+0000 7f20a1600700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20880b2880 0x7f20880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:36.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.303+0000 7f20a1600700 1 -- 172.21.15.90:0/2144144520 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f20940e21b0 con 0x7f20a4075d20 2024-05-27T07:40:36.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.303+0000 7f209b400700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20880b2880 0x7f20880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:36.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.304+0000 7f209b400700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20880b2880 0x7f20880b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f208c003760 tx=0x7f208c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:36.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.306+0000 7f20a1600700 1 -- 172.21.15.90:0/2144144520 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f209401f020 con 0x7f20a4075d20 2024-05-27T07:40:36.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.586+0000 7f20a9800700 1 -- 172.21.15.90:0/2144144520 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20a402e5e0 con 0x7f20a4075d20 2024-05-27T07:40:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.587+0000 7f20a1600700 1 -- 172.21.15.90:0/2144144520 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7f20940aa2d0 con 0x7f20a4075d20 2024-05-27T07:40:36.585 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:40:36.585 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-27T07:40:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 -- 172.21.15.90:0/2144144520 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20880b2880 msgr2=0x7f20880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20880b2880 0x7f20880b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f208c003760 tx=0x7f208c005fb0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 -- 172.21.15.90:0/2144144520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20a4075d20 msgr2=0x7f20a419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20a4075d20 0x7f20a419cda0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f209400d980 tx=0x7f209400dd40 comp rx=0 tx=0).stop 2024-05-27T07:40:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 -- 172.21.15.90:0/2144144520 shutdown_connections 2024-05-27T07:40:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20880b2880 0x7f20880b4d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a41109a0 0x7f20a41a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20a4075d20 0x7f20a419cda0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 --2- 172.21.15.90:0/2144144520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20a4076670 0x7f20a419d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 -- 172.21.15.90:0/2144144520 >> 172.21.15.90:0/2144144520 conn(0x7f20a4079d00 msgr2=0x7f20a4101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 -- 172.21.15.90:0/2144144520 shutdown_connections 2024-05-27T07:40:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:36.590+0000 7f20a9800700 1 -- 172.21.15.90:0/2144144520 wait complete. 2024-05-27T07:40:36.589 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 8 2024-05-27T07:40:37.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:36 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:37.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:36 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:37.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:36 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:37.179 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:36 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:37.180 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:36 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:37.180 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:36 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:37.180 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:36 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2144144520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:37.180 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:36 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] up:boot 2024-05-27T07:40:37.180 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:36 smithi136 ceph-mon[35324]: fsmap 8 up:standby 2024-05-27T07:40:37.180 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:36 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:40:37.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:37.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:37.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:37.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:37.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:37.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:37.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:36 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2144144520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:37.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:36 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] up:boot 2024-05-27T07:40:37.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:36 smithi090 ceph-mon[31429]: fsmap 8 up:standby 2024-05-27T07:40:37.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:36 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:40:37.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:36 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:37.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:36 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:37.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:36 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:37.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:36 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:37.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:36 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:37.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:36 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:37.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:36 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2144144520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:37.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:36 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] up:boot 2024-05-27T07:40:37.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:36 smithi129 ceph-mon[35254]: fsmap 8 up:standby 2024-05-27T07:40:37.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:36 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:40:38.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:37 smithi090 ceph-mon[31429]: Deploying daemon mds.i on smithi136 2024-05-27T07:40:38.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:37 smithi090 ceph-mon[31429]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:38.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:37 smithi136 ceph-mon[35324]: Deploying daemon mds.i on smithi136 2024-05-27T07:40:38.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:37 smithi136 ceph-mon[35324]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:38.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:37 smithi129 ceph-mon[35254]: Deploying daemon mds.i on smithi136 2024-05-27T07:40:38.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:37 smithi129 ceph-mon[35254]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:39.424 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:40:39.498 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 07:40:39 smithi136 systemd[1]: Starting Ceph mds.i for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:39.852 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 07:40:39 smithi136 podman[66316]: 2024-05-27 07:40:39.44006418 +0000 UTC m=+0.035651537 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:39.852 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 07:40:39 smithi136 podman[66316]: 2024-05-27 07:40:39.557636711 +0000 UTC m=+0.153224103 container create 8a9624ce7aa2241feddc3891b4cbd66f25e383606001130866c9d6c5f828a8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-i, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-27T07:40:39.852 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 07:40:39 smithi136 podman[66316]: 2024-05-27 07:40:39.853617197 +0000 UTC m=+0.449204643 container init 8a9624ce7aa2241feddc3891b4cbd66f25e383606001130866c9d6c5f828a8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T07:40:39.881 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:40.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:39 smithi090 ceph-mon[31429]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:40.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:39 smithi136 ceph-mon[35324]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:40.247 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 07:40:39 smithi136 podman[66316]: 2024-05-27 07:40:39.865095922 +0000 UTC m=+0.460683287 container start 8a9624ce7aa2241feddc3891b4cbd66f25e383606001130866c9d6c5f828a8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, ceph=True, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T07:40:40.247 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 07:40:39 smithi136 ceph-mds[66336]: -- 172.21.15.136:0/3351539930 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac98d1ba40 con 0x55ac98d63600 2024-05-27T07:40:40.247 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 07:40:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-i[66332]: starting mds.i at 2024-05-27T07:40:40.247 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 07:40:39 smithi136 bash[66316]: 8a9624ce7aa2241feddc3891b4cbd66f25e383606001130866c9d6c5f828a8eb 2024-05-27T07:40:40.247 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 07:40:40 smithi136 systemd[1]: Started Ceph mds.i for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:40.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:39 smithi129 ceph-mon[35254]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:41.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:41.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:41.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:41.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:41.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:41.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:41.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: Deploying daemon mds.g on smithi090 2024-05-27T07:40:41.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:41.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] up:boot 2024-05-27T07:40:41.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: fsmap 9 up:standby 2024-05-27T07:40:41.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:41 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:40:41.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:41.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:41.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:41.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:41.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:41.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:41.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: Deploying daemon mds.g on smithi090 2024-05-27T07:40:41.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:41.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] up:boot 2024-05-27T07:40:41.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: fsmap 9 up:standby 2024-05-27T07:40:41.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:41 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:40:41.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:41.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:41.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:41.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:41.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:41.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:41.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: Deploying daemon mds.g on smithi090 2024-05-27T07:40:41.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:41.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] up:boot 2024-05-27T07:40:41.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: fsmap 9 up:standby 2024-05-27T07:40:41.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:41 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:40:42.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.246+0000 7f876b600700 1 -- 172.21.15.90:0/3866607956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8764108730 msgr2=0x7f8764110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.246+0000 7f876b600700 1 --2- 172.21.15.90:0/3866607956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8764108730 0x7f8764110090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f875c00b0d0 tx=0x7f875c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.246+0000 7f876b600700 1 -- 172.21.15.90:0/3866607956 shutdown_connections 2024-05-27T07:40:42.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.246+0000 7f876b600700 1 --2- 172.21.15.90:0/3866607956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8764108730 0x7f8764110090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.246+0000 7f876b600700 1 --2- 172.21.15.90:0/3866607956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87641034b0 0x7f87641038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.246+0000 7f876b600700 1 --2- 172.21.15.90:0/3866607956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8764107de0 0x7f87641081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.246+0000 7f876b600700 1 -- 172.21.15.90:0/3866607956 >> 172.21.15.90:0/3866607956 conn(0x7f87640fedb0 msgr2=0x7f87641011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.246+0000 7f876b600700 1 -- 172.21.15.90:0/3866607956 shutdown_connections 2024-05-27T07:40:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.246+0000 7f876b600700 1 -- 172.21.15.90:0/3866607956 wait complete. 2024-05-27T07:40:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.248+0000 7f876b600700 1 Processor -- start 2024-05-27T07:40:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.248+0000 7f876b600700 1 -- start start 2024-05-27T07:40:42.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.248+0000 7f876b600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87641034b0 0x7f876419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f876b600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764107de0 0x7f876419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f876b600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8764108730 0x7f8764199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f876b600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8764113260 con 0x7f8764108730 2024-05-27T07:40:42.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f876b600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87641130e0 con 0x7f87641034b0 2024-05-27T07:40:42.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f876b600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87641133e0 con 0x7f8764107de0 2024-05-27T07:40:42.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8769600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8764108730 0x7f8764199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8769600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8764108730 0x7f8764199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44174/0 (socket says 172.21.15.90:44174) 2024-05-27T07:40:42.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8769600700 1 -- 172.21.15.90:0/2423312674 learned_addr learned my addr 172.21.15.90:0/2423312674 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:40:42.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8763e00700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764107de0 0x7f876419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8768c00700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87641034b0 0x7f876419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8769600700 1 -- 172.21.15.90:0/2423312674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764107de0 msgr2=0x7f876419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8769600700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764107de0 0x7f876419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8769600700 1 -- 172.21.15.90:0/2423312674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87641034b0 msgr2=0x7f876419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8769600700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87641034b0 0x7f876419f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8769600700 1 -- 172.21.15.90:0/2423312674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876419a170 con 0x7f8764108730 2024-05-27T07:40:42.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.249+0000 7f8763e00700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764107de0 0x7f876419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:40:42.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.250+0000 7f8769600700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8764108730 0x7f8764199ac0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f875c007d10 tx=0x7f875c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:42.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.250+0000 7f8761600700 1 -- 172.21.15.90:0/2423312674 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c003e10 con 0x7f8764108730 2024-05-27T07:40:42.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.250+0000 7f8761600700 1 -- 172.21.15.90:0/2423312674 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f875c005500 con 0x7f8764108730 2024-05-27T07:40:42.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.250+0000 7f8761600700 1 -- 172.21.15.90:0/2423312674 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c01aaa0 con 0x7f8764108730 2024-05-27T07:40:42.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.250+0000 7f876b600700 1 -- 172.21.15.90:0/2423312674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876419a400 con 0x7f8764108730 2024-05-27T07:40:42.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.250+0000 7f876b600700 1 -- 172.21.15.90:0/2423312674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876419a890 con 0x7f8764108730 2024-05-27T07:40:42.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.251+0000 7f876b600700 1 -- 172.21.15.90:0/2423312674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87640512b0 con 0x7f8764108730 2024-05-27T07:40:42.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.254+0000 7f8761600700 1 -- 172.21.15.90:0/2423312674 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f875c0056b0 con 0x7f8764108730 2024-05-27T07:40:42.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.255+0000 7f8761600700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f874c0b2900 0x7f874c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:42.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.256+0000 7f8761600700 1 -- 172.21.15.90:0/2423312674 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f875c00f030 con 0x7f8764108730 2024-05-27T07:40:42.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.256+0000 7f8768c00700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f874c0b2900 0x7f874c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:42.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.256+0000 7f8768c00700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f874c0b2900 0x7f874c0b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8758005950 tx=0x7f87580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:42.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.258+0000 7f8761600700 1 -- 172.21.15.90:0/2423312674 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f875c0ad5b0 con 0x7f8764108730 2024-05-27T07:40:42.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.548+0000 7f876b600700 1 -- 172.21.15.90:0/2423312674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f876410b1c0 con 0x7f8764108730 2024-05-27T07:40:42.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.549+0000 7f8761600700 1 -- 172.21.15.90:0/2423312674 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7f875c0aceb0 con 0x7f8764108730 2024-05-27T07:40:42.546 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:40:42.546 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34442,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T07:40:42.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.551+0000 7f876b600700 1 -- 172.21.15.90:0/2423312674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f874c0b2900 msgr2=0x7f874c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.551+0000 7f876b600700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f874c0b2900 0x7f874c0b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8758005950 tx=0x7f87580058e0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.551+0000 7f876b600700 1 -- 172.21.15.90:0/2423312674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8764108730 msgr2=0x7f8764199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:42.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.551+0000 7f876b600700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8764108730 0x7f8764199ac0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f875c007d10 tx=0x7f875c007df0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.551+0000 7f876b600700 1 -- 172.21.15.90:0/2423312674 shutdown_connections 2024-05-27T07:40:42.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.551+0000 7f876b600700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f874c0b2900 0x7f874c0b4e10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.551+0000 7f876b600700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8764108730 0x7f8764199ac0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.551+0000 7f876b600700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87641034b0 0x7f876419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.551+0000 7f876b600700 1 --2- 172.21.15.90:0/2423312674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764107de0 0x7f876419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:42.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.552+0000 7f876b600700 1 -- 172.21.15.90:0/2423312674 >> 172.21.15.90:0/2423312674 conn(0x7f87640fedb0 msgr2=0x7f87641075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:42.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.552+0000 7f876b600700 1 -- 172.21.15.90:0/2423312674 shutdown_connections 2024-05-27T07:40:42.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:42.552+0000 7f876b600700 1 -- 172.21.15.90:0/2423312674 wait complete. 2024-05-27T07:40:42.550 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 10 2024-05-27T07:40:43.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:43 smithi090 ceph-mon[31429]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:43.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:43 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2423312674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:43.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:43 smithi136 ceph-mon[35324]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:43.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:43 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2423312674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:43.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:43 smithi129 ceph-mon[35254]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:43.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:43 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2423312674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:44.417 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 07:40:44 smithi090 systemd[1]: Starting Ceph mds.g for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:44.695 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 07:40:44 smithi090 podman[72389]: 2024-05-27 07:40:44.322803902 +0000 UTC m=+0.031631777 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:44.695 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 07:40:44 smithi090 podman[72389]: 2024-05-27 07:40:44.432938669 +0000 UTC m=+0.141766580 container create 37b82db357a226fa30efec82b638b0acf291c459e2cdf33e812a5d5abe48a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-g, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T07:40:44.695 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 07:40:44 smithi090 podman[72389]: 2024-05-27 07:40:44.696571916 +0000 UTC m=+0.405399854 container init 37b82db357a226fa30efec82b638b0acf291c459e2cdf33e812a5d5abe48a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-27T07:40:44.947 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 07:40:44 smithi090 podman[72389]: 2024-05-27 07:40:44.707591954 +0000 UTC m=+0.416419838 container start 37b82db357a226fa30efec82b638b0acf291c459e2cdf33e812a5d5abe48a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-g, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T07:40:44.947 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 07:40:44 smithi090 ceph-mds[72408]: -- 172.21.15.90:0/176820859 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56025f331a40 con 0x56025f379a80 2024-05-27T07:40:45.211 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 07:40:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-g[72404]: starting mds.g at 2024-05-27T07:40:45.211 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 07:40:45 smithi090 bash[72389]: 37b82db357a226fa30efec82b638b0acf291c459e2cdf33e812a5d5abe48a047 2024-05-27T07:40:45.211 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 07:40:45 smithi090 systemd[1]: Started Ceph mds.g for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:45.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:45 smithi090 ceph-mon[31429]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:45.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:45.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:45.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:45.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:45.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:45.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:45 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:45.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:40:45.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:45 smithi136 ceph-mon[35324]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:45.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:45.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:45.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:45.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:45.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:45.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:45 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:45.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:45 smithi129 ceph-mon[35254]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:45.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:45.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:45.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:45.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:45.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:45.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:45 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:46.191 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:46.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:46 smithi090 ceph-mon[31429]: Deploying daemon mds.a on smithi090 2024-05-27T07:40:46.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:46 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6838/767662982,v1:172.21.15.90:6839/767662982] up:boot 2024-05-27T07:40:46.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:46 smithi090 ceph-mon[31429]: fsmap 10 up:standby 2024-05-27T07:40:46.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:46 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:40:46.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:46 smithi136 ceph-mon[35324]: Deploying daemon mds.a on smithi090 2024-05-27T07:40:46.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:46 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6838/767662982,v1:172.21.15.90:6839/767662982] up:boot 2024-05-27T07:40:46.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:46 smithi136 ceph-mon[35324]: fsmap 10 up:standby 2024-05-27T07:40:46.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:46 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:40:46.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:46 smithi129 ceph-mon[35254]: Deploying daemon mds.a on smithi090 2024-05-27T07:40:46.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:46 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6838/767662982,v1:172.21.15.90:6839/767662982] up:boot 2024-05-27T07:40:46.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:46 smithi129 ceph-mon[35254]: fsmap 10 up:standby 2024-05-27T07:40:46.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:46 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:40:47.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:47 smithi090 ceph-mon[31429]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:47.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:47 smithi136 ceph-mon[35324]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:47.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:47 smithi129 ceph-mon[35254]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:49.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.491+0000 7f53d3000700 1 -- 172.21.15.90:0/1054477417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53cc115360 msgr2=0x7f53cc117780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:49.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.491+0000 7f53d3000700 1 --2- 172.21.15.90:0/1054477417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53cc115360 0x7f53cc117780 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f53c400b0a0 tx=0x7f53c400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.491+0000 7f53d3000700 1 -- 172.21.15.90:0/1054477417 shutdown_connections 2024-05-27T07:40:49.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.491+0000 7f53d3000700 1 --2- 172.21.15.90:0/1054477417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53cc115360 0x7f53cc117780 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.491+0000 7f53d3000700 1 --2- 172.21.15.90:0/1054477417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53cc075b50 0x7f53cc075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.491+0000 7f53d3000700 1 --2- 172.21.15.90:0/1054477417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53cc0764a0 0x7f53cc114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.491+0000 7f53d3000700 1 -- 172.21.15.90:0/1054477417 >> 172.21.15.90:0/1054477417 conn(0x7f53cc100030 msgr2=0x7f53cc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:49.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.491+0000 7f53d3000700 1 -- 172.21.15.90:0/1054477417 shutdown_connections 2024-05-27T07:40:49.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.491+0000 7f53d3000700 1 -- 172.21.15.90:0/1054477417 wait complete. 2024-05-27T07:40:49.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.492+0000 7f53d3000700 1 Processor -- start 2024-05-27T07:40:49.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.492+0000 7f53d3000700 1 -- start start 2024-05-27T07:40:49.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.493+0000 7f53d3000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53cc075b50 0x7f53cc1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:49.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.493+0000 7f53d3000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53cc0764a0 0x7f53cc1a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.493+0000 7f53d3000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53cc115360 0x7f53cc1a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.493+0000 7f53d3000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc11a960 con 0x7f53cc0764a0 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.493+0000 7f53d3000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc11a7e0 con 0x7f53cc075b50 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.493+0000 7f53d3000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc11aae0 con 0x7f53cc115360 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.493+0000 7f53cb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53cc0764a0 0x7f53cc1a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.493+0000 7f53cb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53cc0764a0 0x7f53cc1a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59516/0 (socket says 172.21.15.90:59516) 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.493+0000 7f53cb400700 1 -- 172.21.15.90:0/1281623016 learned_addr learned my addr 172.21.15.90:0/1281623016 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.494+0000 7f53cbe00700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53cc075b50 0x7f53cc1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.494+0000 7f53d1000700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53cc115360 0x7f53cc1a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.494+0000 7f53cb400700 1 -- 172.21.15.90:0/1281623016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53cc115360 msgr2=0x7f53cc1a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:49.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.494+0000 7f53cb400700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53cc115360 0x7f53cc1a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.494+0000 7f53cb400700 1 -- 172.21.15.90:0/1281623016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53cc075b50 msgr2=0x7f53cc1a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.494+0000 7f53cb400700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53cc075b50 0x7f53cc1a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.494+0000 7f53cb400700 1 -- 172.21.15.90:0/1281623016 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc1a61c0 con 0x7f53cc0764a0 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.494+0000 7f53cb400700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53cc0764a0 0x7f53cc1a1890 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f53b800d950 tx=0x7f53b800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.494+0000 7f53c8c00700 1 -- 172.21.15.90:0/1281623016 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8016070 con 0x7f53cc0764a0 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.495+0000 7f53c8c00700 1 -- 172.21.15.90:0/1281623016 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f53b8010460 con 0x7f53cc0764a0 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.495+0000 7f53d3000700 1 -- 172.21.15.90:0/1281623016 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53cc1a64b0 con 0x7f53cc0764a0 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.495+0000 7f53c8c00700 1 -- 172.21.15.90:0/1281623016 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b800f620 con 0x7f53cc0764a0 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.495+0000 7f53d3000700 1 -- 172.21.15.90:0/1281623016 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53cc1addd0 con 0x7f53cc0764a0 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.496+0000 7f53d3000700 1 -- 172.21.15.90:0/1281623016 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53cc0512b0 con 0x7f53cc0764a0 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.499+0000 7f53c8c00700 1 -- 172.21.15.90:0/1281623016 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f53b8019070 con 0x7f53cc0764a0 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.501+0000 7f53c8c00700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f53b00b2900 0x7f53b00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:49.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.501+0000 7f53c8c00700 1 -- 172.21.15.90:0/1281623016 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f53b80e1950 con 0x7f53cc0764a0 2024-05-27T07:40:49.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.501+0000 7f53cbe00700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f53b00b2900 0x7f53b00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:49.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.502+0000 7f53cbe00700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f53b00b2900 0x7f53b00b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f53bc005950 tx=0x7f53bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:49.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.506+0000 7f53c8c00700 1 -- 172.21.15.90:0/1281623016 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53b80a9a70 con 0x7f53cc0764a0 2024-05-27T07:40:49.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:49 smithi090 ceph-mon[31429]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:49.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:49 smithi136 ceph-mon[35324]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:49.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:49 smithi129 ceph-mon[35254]: pgmap v358: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:49.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.816+0000 7f53d3000700 1 -- 172.21.15.90:0/1281623016 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53cc1a2650 con 0x7f53cc0764a0 2024-05-27T07:40:49.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.817+0000 7f53c8c00700 1 -- 172.21.15.90:0/1281623016 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f53b80a9370 con 0x7f53cc0764a0 2024-05-27T07:40:49.815 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:40:49.815 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14556,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34442,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-27T07:40:49.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.820+0000 7f53d3000700 1 -- 172.21.15.90:0/1281623016 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f53b00b2900 msgr2=0x7f53b00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:49.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.820+0000 7f53d3000700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f53b00b2900 0x7f53b00b4e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f53bc005950 tx=0x7f53bc0058e0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.820+0000 7f53d3000700 1 -- 172.21.15.90:0/1281623016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53cc0764a0 msgr2=0x7f53cc1a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:49.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.820+0000 7f53d3000700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53cc0764a0 0x7f53cc1a1890 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f53b800d950 tx=0x7f53b800dd10 comp rx=0 tx=0).stop 2024-05-27T07:40:49.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.821+0000 7f53d3000700 1 -- 172.21.15.90:0/1281623016 shutdown_connections 2024-05-27T07:40:49.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.821+0000 7f53d3000700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f53b00b2900 0x7f53b00b4e10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.821+0000 7f53d3000700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53cc115360 0x7f53cc1a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.821+0000 7f53d3000700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53cc075b50 0x7f53cc1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.821+0000 7f53d3000700 1 --2- 172.21.15.90:0/1281623016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53cc0764a0 0x7f53cc1a1890 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:49.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.821+0000 7f53d3000700 1 -- 172.21.15.90:0/1281623016 >> 172.21.15.90:0/1281623016 conn(0x7f53cc100030 msgr2=0x7f53cc100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:49.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.821+0000 7f53d3000700 1 -- 172.21.15.90:0/1281623016 shutdown_connections 2024-05-27T07:40:49.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:49.821+0000 7f53d3000700 1 -- 172.21.15.90:0/1281623016 wait complete. 2024-05-27T07:40:49.819 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 11 2024-05-27T07:40:50.918 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:50.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:50.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:50.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:50.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:50.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:50.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: Deploying daemon mds.b on smithi129 2024-05-27T07:40:50.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1281623016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:50.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6840/24895501,v1:172.21.15.90:6841/24895501] up:boot 2024-05-27T07:40:50.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: fsmap 11 up:standby 2024-05-27T07:40:50.919 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:50 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:40:50.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:50.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:50.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:50.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:50.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:50.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:50.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: Deploying daemon mds.b on smithi129 2024-05-27T07:40:50.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1281623016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:50.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6840/24895501,v1:172.21.15.90:6841/24895501] up:boot 2024-05-27T07:40:50.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: fsmap 11 up:standby 2024-05-27T07:40:50.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:50 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:40:50.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:50.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:50.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:50.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T07:40:50.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T07:40:50.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:40:50.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: Deploying daemon mds.b on smithi129 2024-05-27T07:40:50.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1281623016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:50.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6840/24895501,v1:172.21.15.90:6841/24895501] up:boot 2024-05-27T07:40:50.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: fsmap 11 up:standby 2024-05-27T07:40:50.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:50 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:40:51.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:51 smithi090 ceph-mon[31429]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:51.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:51 smithi136 ceph-mon[35324]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:52.012 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:51 smithi129 ceph-mon[35254]: pgmap v359: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:52.684 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:40:53.099 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:53.218 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:40:52 smithi129 systemd[1]: Starting Ceph mds.b for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:40:53.470 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:40:53 smithi129 podman[65285]: 2024-05-27 07:40:53.125161689 +0000 UTC m=+0.034171657 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:40:53.470 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:40:53 smithi129 podman[65285]: 2024-05-27 07:40:53.291856459 +0000 UTC m=+0.200866465 container create d2c1c48c587b0486490ba25dd8e795c0ca11879a64868bbac2bd5ff185aea35a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-27T07:40:53.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:53 smithi090 ceph-mon[31429]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:53.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:53 smithi136 ceph-mon[35324]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:54.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:53 smithi129 ceph-mon[35254]: pgmap v360: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:54.062 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:40:53 smithi129 podman[65285]: 2024-05-27 07:40:53.587411677 +0000 UTC m=+0.496421727 container init d2c1c48c587b0486490ba25dd8e795c0ca11879a64868bbac2bd5ff185aea35a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:40:54.062 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:40:53 smithi129 podman[65285]: 2024-05-27 07:40:53.597916597 +0000 UTC m=+0.506926597 container start d2c1c48c587b0486490ba25dd8e795c0ca11879a64868bbac2bd5ff185aea35a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T07:40:54.062 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:40:53 smithi129 bash[65285]: d2c1c48c587b0486490ba25dd8e795c0ca11879a64868bbac2bd5ff185aea35a 2024-05-27T07:40:54.062 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:40:53 smithi129 systemd[1]: Started Ceph mds.b for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:40:54.062 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:40:53 smithi129 ceph-mds[65304]: -- 172.21.15.129:0/3647120842 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55701fb15a40 con 0x55701fb5d180 2024-05-27T07:40:54.062 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:40:53 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b[65300]: starting mds.b at 2024-05-27T07:40:54.934 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:54.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:54.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:54.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:54.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:54.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:40:54.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:54 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] up:boot 2024-05-27T07:40:54.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:54 smithi090 ceph-mon[31429]: fsmap 12 up:standby 2024-05-27T07:40:54.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:54 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:40:55.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:40:55.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:54 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] up:boot 2024-05-27T07:40:55.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:54 smithi129 ceph-mon[35254]: fsmap 12 up:standby 2024-05-27T07:40:55.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:54 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:40:55.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.172+0000 7f3471000700 1 -- 172.21.15.90:0/3133940087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c115170 msgr2=0x7f346c117590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:55.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.172+0000 7f3471000700 1 --2- 172.21.15.90:0/3133940087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c115170 0x7f346c117590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f346400b0a0 tx=0x7f346400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.172+0000 7f3471000700 1 -- 172.21.15.90:0/3133940087 shutdown_connections 2024-05-27T07:40:55.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.172+0000 7f3471000700 1 --2- 172.21.15.90:0/3133940087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c115170 0x7f346c117590 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.172+0000 7f3471000700 1 --2- 172.21.15.90:0/3133940087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f346c075b50 0x7f346c075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.172+0000 7f3471000700 1 --2- 172.21.15.90:0/3133940087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c0764a0 0x7f346c114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.172+0000 7f3471000700 1 -- 172.21.15.90:0/3133940087 >> 172.21.15.90:0/3133940087 conn(0x7f346c079d00 msgr2=0x7f346c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:55.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.173+0000 7f3471000700 1 -- 172.21.15.90:0/3133940087 shutdown_connections 2024-05-27T07:40:55.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.173+0000 7f3471000700 1 -- 172.21.15.90:0/3133940087 wait complete. 2024-05-27T07:40:55.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.174+0000 7f3471000700 1 Processor -- start 2024-05-27T07:40:55.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.174+0000 7f3471000700 1 -- start start 2024-05-27T07:40:55.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.174+0000 7f3471000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f346c075b50 0x7f346c1a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:55.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.174+0000 7f3471000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c0764a0 0x7f346c1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:55.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.174+0000 7f3471000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c115170 0x7f346c1a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:55.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f3471000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f346c11a780 con 0x7f346c115170 2024-05-27T07:40:55.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f3471000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f346c11a600 con 0x7f346c075b50 2024-05-27T07:40:55.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f3471000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f346c11a900 con 0x7f346c0764a0 2024-05-27T07:40:55.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f346c075b50 0x7f346c1a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:55.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c115170 0x7f346c1a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:55.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f346c075b50 0x7f346c1a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49000/0 (socket says 172.21.15.90:49000) 2024-05-27T07:40:55.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346b400700 1 -- 172.21.15.90:0/2480653230 learned_addr learned my addr 172.21.15.90:0/2480653230 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:40:55.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346aa00700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c0764a0 0x7f346c1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:55.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346b400700 1 -- 172.21.15.90:0/2480653230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c0764a0 msgr2=0x7f346c1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:55.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346b400700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c0764a0 0x7f346c1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346b400700 1 -- 172.21.15.90:0/2480653230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c115170 msgr2=0x7f346c1a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:55.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346b400700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c115170 0x7f346c1a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346b400700 1 -- 172.21.15.90:0/2480653230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c1a60c0 con 0x7f346c075b50 2024-05-27T07:40:55.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.175+0000 7f346be00700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c115170 0x7f346c1a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:40:55.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.176+0000 7f346b400700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f346c075b50 0x7f346c1a11f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f345c00e9a0 tx=0x7f345c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:55.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.176+0000 7f3463e00700 1 -- 172.21.15.90:0/2480653230 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c00ca10 con 0x7f346c075b50 2024-05-27T07:40:55.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.176+0000 7f3463e00700 1 -- 172.21.15.90:0/2480653230 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f345c00cbb0 con 0x7f346c075b50 2024-05-27T07:40:55.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.176+0000 7f3463e00700 1 -- 172.21.15.90:0/2480653230 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c018680 con 0x7f346c075b50 2024-05-27T07:40:55.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.176+0000 7f3471000700 1 -- 172.21.15.90:0/2480653230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c1a6350 con 0x7f346c075b50 2024-05-27T07:40:55.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.176+0000 7f3471000700 1 -- 172.21.15.90:0/2480653230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c1a6840 con 0x7f346c075b50 2024-05-27T07:40:55.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.177+0000 7f3471000700 1 -- 172.21.15.90:0/2480653230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f346c077290 con 0x7f346c075b50 2024-05-27T07:40:55.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.180+0000 7f3463e00700 1 -- 172.21.15.90:0/2480653230 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f345c010430 con 0x7f346c075b50 2024-05-27T07:40:55.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.182+0000 7f3463e00700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34500b2880 0x7f34500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:55.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.182+0000 7f3463e00700 1 -- 172.21.15.90:0/2480653230 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f345c0e23b0 con 0x7f346c075b50 2024-05-27T07:40:55.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.182+0000 7f346aa00700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34500b2880 0x7f34500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:55.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.183+0000 7f346aa00700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34500b2880 0x7f34500b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f346c116cf0 tx=0x7f3458009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:55.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.187+0000 7f3463e00700 1 -- 172.21.15.90:0/2480653230 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f345c0aa4d0 con 0x7f346c075b50 2024-05-27T07:40:55.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:40:55.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:54 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] up:boot 2024-05-27T07:40:55.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:54 smithi136 ceph-mon[35324]: fsmap 12 up:standby 2024-05-27T07:40:55.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:54 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:40:55.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.470+0000 7f3471000700 1 -- 172.21.15.90:0/2480653230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f346c1a2550 con 0x7f346c075b50 2024-05-27T07:40:55.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.472+0000 7f3463e00700 1 -- 172.21.15.90:0/2480653230 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7f345c0a9dd0 con 0x7f346c075b50 2024-05-27T07:40:55.469 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:40:55.469 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14556,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34442,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-27T07:40:55.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.474+0000 7f3471000700 1 -- 172.21.15.90:0/2480653230 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34500b2880 msgr2=0x7f34500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:55.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.474+0000 7f3471000700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34500b2880 0x7f34500b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f346c116cf0 tx=0x7f3458009260 comp rx=0 tx=0).stop 2024-05-27T07:40:55.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.474+0000 7f3471000700 1 -- 172.21.15.90:0/2480653230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f346c075b50 msgr2=0x7f346c1a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:55.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.474+0000 7f3471000700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f346c075b50 0x7f346c1a11f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f345c00e9a0 tx=0x7f345c00ecb0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.475+0000 7f3471000700 1 -- 172.21.15.90:0/2480653230 shutdown_connections 2024-05-27T07:40:55.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.475+0000 7f3471000700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f34500b2880 0x7f34500b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.475+0000 7f3471000700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f346c115170 0x7f346c1a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.475+0000 7f3471000700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f346c075b50 0x7f346c1a11f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.475+0000 7f3471000700 1 --2- 172.21.15.90:0/2480653230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f346c0764a0 0x7f346c1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:55.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.475+0000 7f3471000700 1 -- 172.21.15.90:0/2480653230 >> 172.21.15.90:0/2480653230 conn(0x7f346c079d00 msgr2=0x7f346c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:55.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.475+0000 7f3471000700 1 -- 172.21.15.90:0/2480653230 shutdown_connections 2024-05-27T07:40:55.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:55.475+0000 7f3471000700 1 -- 172.21.15.90:0/2480653230 wait complete. 2024-05-27T07:40:55.473 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 13 2024-05-27T07:40:55.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:55 smithi136 ceph-mon[35324]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:55.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:55 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:55 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:55.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:55 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2480653230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:56.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:55 smithi129 ceph-mon[35254]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:56.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:55 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:56.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:55 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:56.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:55 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2480653230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:56.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:55 smithi090 ceph-mon[31429]: pgmap v361: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:56.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:55 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:56.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:55 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:56.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:55 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2480653230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:40:56.302 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-27T07:40:57.070 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:40:58.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:57 smithi090 ceph-mon[31429]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:58.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:57 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:57 smithi136 ceph-mon[35324]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:58.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:57 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:57 smithi129 ceph-mon[35254]: pgmap v362: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:58.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:58.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:57 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:59.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.539+0000 7f3395a00700 1 -- 172.21.15.90:0/2336677100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33901034b0 msgr2=0x7f33901038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:59.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.539+0000 7f3395a00700 1 --2- 172.21.15.90:0/2336677100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33901034b0 0x7f33901038c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f33800099c0 tx=0x7f3380009cd0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.539+0000 7f3395a00700 1 -- 172.21.15.90:0/2336677100 shutdown_connections 2024-05-27T07:40:59.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.539+0000 7f3395a00700 1 --2- 172.21.15.90:0/2336677100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3390108730 0x7f3390110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.539+0000 7f3395a00700 1 --2- 172.21.15.90:0/2336677100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33901034b0 0x7f33901038c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.539+0000 7f3395a00700 1 --2- 172.21.15.90:0/2336677100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3390107de0 0x7f33901081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.539+0000 7f3395a00700 1 -- 172.21.15.90:0/2336677100 >> 172.21.15.90:0/2336677100 conn(0x7f33900fedb0 msgr2=0x7f33901011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:59.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.539+0000 7f3395a00700 1 -- 172.21.15.90:0/2336677100 shutdown_connections 2024-05-27T07:40:59.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.539+0000 7f3395a00700 1 -- 172.21.15.90:0/2336677100 wait complete. 2024-05-27T07:40:59.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.541+0000 7f3395a00700 1 Processor -- start 2024-05-27T07:40:59.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.541+0000 7f3395a00700 1 -- start start 2024-05-27T07:40:59.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.541+0000 7f3395a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33901034b0 0x7f339019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:59.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.541+0000 7f3395a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3390107de0 0x7f339019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:59.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.542+0000 7f3395a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3390108730 0x7f339019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:59.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.542+0000 7f3395a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3390113260 con 0x7f3390108730 2024-05-27T07:40:59.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.542+0000 7f3395a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33901130e0 con 0x7f33901034b0 2024-05-27T07:40:59.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.542+0000 7f3395a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33901133e0 con 0x7f3390107de0 2024-05-27T07:40:59.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.542+0000 7f3387e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3390107de0 0x7f339019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:59.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.542+0000 7f3387e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3390107de0 0x7f339019b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:60926/0 (socket says 172.21.15.90:60926) 2024-05-27T07:40:59.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.542+0000 7f338f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3390108730 0x7f339019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:59.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.542+0000 7f3387e00700 1 -- 172.21.15.90:0/2791235938 learned_addr learned my addr 172.21.15.90:0/2791235938 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:40:59.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.543+0000 7f338ea00700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33901034b0 0x7f339019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:59.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.543+0000 7f338f400700 1 -- 172.21.15.90:0/2791235938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3390107de0 msgr2=0x7f339019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:59.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.543+0000 7f338f400700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3390107de0 0x7f339019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.543+0000 7f338f400700 1 -- 172.21.15.90:0/2791235938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33901034b0 msgr2=0x7f339019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:59.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.543+0000 7f338f400700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33901034b0 0x7f339019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.543+0000 7f338f400700 1 -- 172.21.15.90:0/2791235938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339019fc10 con 0x7f3390108730 2024-05-27T07:40:59.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.543+0000 7f338f400700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3390108730 0x7f339019f570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3388007e90 tx=0x7f338800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:59.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.544+0000 7f3387400700 1 -- 172.21.15.90:0/2791235938 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388009e20 con 0x7f3390108730 2024-05-27T07:40:59.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.544+0000 7f3387400700 1 -- 172.21.15.90:0/2791235938 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f338800fc40 con 0x7f3390108730 2024-05-27T07:40:59.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.544+0000 7f3395a00700 1 -- 172.21.15.90:0/2791235938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339019ff00 con 0x7f3390108730 2024-05-27T07:40:59.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.544+0000 7f3387400700 1 -- 172.21.15.90:0/2791235938 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388005020 con 0x7f3390108730 2024-05-27T07:40:59.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.544+0000 7f3395a00700 1 -- 172.21.15.90:0/2791235938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33900ff1c0 con 0x7f3390108730 2024-05-27T07:40:59.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.545+0000 7f3395a00700 1 -- 172.21.15.90:0/2791235938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3390104960 con 0x7f3390108730 2024-05-27T07:40:59.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.549+0000 7f3387400700 1 -- 172.21.15.90:0/2791235938 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f338801a420 con 0x7f3390108730 2024-05-27T07:40:59.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.551+0000 7f3387400700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33740b2880 0x7f33740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:40:59.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.551+0000 7f3387400700 1 -- 172.21.15.90:0/2791235938 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f33880e5e50 con 0x7f3390108730 2024-05-27T07:40:59.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.552+0000 7f338ea00700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33740b2880 0x7f33740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:40:59.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.552+0000 7f338ea00700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33740b2880 0x7f33740b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3380000c00 tx=0x7f3380005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:40:59.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.553+0000 7f3387400700 1 -- 172.21.15.90:0/2791235938 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33880adf70 con 0x7f3390108730 2024-05-27T07:40:59.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:59.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:40:59 smithi090 ceph-mon[31429]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:59.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:59.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:40:59 smithi136 ceph-mon[35324]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:40:59.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.801+0000 7f3395a00700 1 -- 172.21.15.90:0/2791235938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f339019c0a0 con 0x7f3390108730 2024-05-27T07:40:59.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.802+0000 7f3387400700 1 -- 172.21.15.90:0/2791235938 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f3388004ad0 con 0x7f3390108730 2024-05-27T07:40:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:40:59.799 INFO:teuthology.orchestra.run.smithi090.stdout:[ 2024-05-27T07:40:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T07:40:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "poolnum": 1, 2024-05-27T07:40:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "poolname": ".mgr" 2024-05-27T07:40:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T07:40:59.800 INFO:teuthology.orchestra.run.smithi090.stdout:] 2024-05-27T07:40:59.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.804+0000 7f3395a00700 1 -- 172.21.15.90:0/2791235938 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33740b2880 msgr2=0x7f33740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:59.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.804+0000 7f3395a00700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33740b2880 0x7f33740b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3380000c00 tx=0x7f3380005fb0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.804+0000 7f3395a00700 1 -- 172.21.15.90:0/2791235938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3390108730 msgr2=0x7f339019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:40:59.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.804+0000 7f3395a00700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3390108730 0x7f339019f570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3388007e90 tx=0x7f338800f740 comp rx=0 tx=0).stop 2024-05-27T07:40:59.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.805+0000 7f3395a00700 1 -- 172.21.15.90:0/2791235938 shutdown_connections 2024-05-27T07:40:59.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.805+0000 7f3395a00700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f33740b2880 0x7f33740b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.805+0000 7f3395a00700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3390108730 0x7f339019f570 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.805+0000 7f3395a00700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33901034b0 0x7f339019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.805+0000 7f3395a00700 1 --2- 172.21.15.90:0/2791235938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3390107de0 0x7f339019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:40:59.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.805+0000 7f3395a00700 1 -- 172.21.15.90:0/2791235938 >> 172.21.15.90:0/2791235938 conn(0x7f33900fedb0 msgr2=0x7f339006b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:40:59.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.805+0000 7f3395a00700 1 -- 172.21.15.90:0/2791235938 shutdown_connections 2024-05-27T07:40:59.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:40:59.805+0000 7f3395a00700 1 -- 172.21.15.90:0/2791235938 wait complete. 2024-05-27T07:40:59.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:40:59.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:40:59 smithi129 ceph-mon[35254]: pgmap v363: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:00.592 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-27T07:41:00.592 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-27T07:41:00.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:00.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:00.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:00 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2791235938' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T07:41:00.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:00.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:00.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:00 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2791235938' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T07:41:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:00 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2791235938' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T07:41:01.987 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:01 smithi090 ceph-mon[31429]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:01.988 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:41:01.988 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:41:01.988 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:01.988 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:01 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:41:01.996 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:01 smithi136 ceph-mon[35324]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:01.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:41:01.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:41:01.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:01.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:01 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:41:02.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:01 smithi129 ceph-mon[35254]: pgmap v364: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:02.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:41:02.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:41:02.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:02.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:01 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:41:03.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.118+0000 7f3ca9c00700 1 -- 172.21.15.90:0/970735069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca4106eb0 msgr2=0x7f3ca4113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:03.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.118+0000 7f3ca9c00700 1 --2- 172.21.15.90:0/970735069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca4106eb0 0x7f3ca4113340 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00b0a0 tx=0x7f3c9c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:03.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.118+0000 7f3ca9c00700 1 -- 172.21.15.90:0/970735069 shutdown_connections 2024-05-27T07:41:03.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.118+0000 7f3ca9c00700 1 --2- 172.21.15.90:0/970735069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca4106eb0 0x7f3ca4113340 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:03.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.118+0000 7f3ca9c00700 1 --2- 172.21.15.90:0/970735069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ca410c5a0 0x7f3ca410c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:03.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.118+0000 7f3ca9c00700 1 --2- 172.21.15.90:0/970735069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca4106560 0x7f3ca4106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:03.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.118+0000 7f3ca9c00700 1 -- 172.21.15.90:0/970735069 >> 172.21.15.90:0/970735069 conn(0x7f3ca4102080 msgr2=0x7f3ca41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:03.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.118+0000 7f3ca9c00700 1 -- 172.21.15.90:0/970735069 shutdown_connections 2024-05-27T07:41:03.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.118+0000 7f3ca9c00700 1 -- 172.21.15.90:0/970735069 wait complete. 2024-05-27T07:41:03.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.119+0000 7f3ca9c00700 1 Processor -- start 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.119+0000 7f3ca9c00700 1 -- start start 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.120+0000 7f3ca9c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca4106560 0x7f3ca419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.120+0000 7f3ca9c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ca4106eb0 0x7f3ca419d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.120+0000 7f3ca9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca410c5a0 0x7f3ca41a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.120+0000 7f3ca9c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca4116650 con 0x7f3ca410c5a0 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.120+0000 7f3ca9c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41164d0 con 0x7f3ca4106eb0 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.120+0000 7f3ca9c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca41167d0 con 0x7f3ca4106560 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.120+0000 7f3ca3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca410c5a0 0x7f3ca41a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.120+0000 7f3ca3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca410c5a0 0x7f3ca41a1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50228/0 (socket says 172.21.15.90:50228) 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.120+0000 7f3ca3400700 1 -- 172.21.15.90:0/3816754719 learned_addr learned my addr 172.21.15.90:0/3816754719 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.120+0000 7f3ca2000700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ca4106eb0 0x7f3ca419d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:03.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.121+0000 7f3ca2a00700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca4106560 0x7f3ca419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:03.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.121+0000 7f3ca3400700 1 -- 172.21.15.90:0/3816754719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca4106560 msgr2=0x7f3ca419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:03.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.121+0000 7f3ca3400700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca4106560 0x7f3ca419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:03.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.121+0000 7f3ca3400700 1 -- 172.21.15.90:0/3816754719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ca4106eb0 msgr2=0x7f3ca419d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:03.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.121+0000 7f3ca3400700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ca4106eb0 0x7f3ca419d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:03.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.121+0000 7f3ca3400700 1 -- 172.21.15.90:0/3816754719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca41a1e10 con 0x7f3ca410c5a0 2024-05-27T07:41:03.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.121+0000 7f3ca2000700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ca4106eb0 0x7f3ca419d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:03.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.121+0000 7f3ca3400700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca410c5a0 0x7f3ca41a1770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00b070 tx=0x7f3c9c004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:03.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.121+0000 7f3c9b400700 1 -- 172.21.15.90:0/3816754719 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c021070 con 0x7f3ca410c5a0 2024-05-27T07:41:03.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.122+0000 7f3c9b400700 1 -- 172.21.15.90:0/3816754719 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c9c007c00 con 0x7f3ca410c5a0 2024-05-27T07:41:03.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.122+0000 7f3ca9c00700 1 -- 172.21.15.90:0/3816754719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca41a20a0 con 0x7f3ca410c5a0 2024-05-27T07:41:03.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.122+0000 7f3c9b400700 1 -- 172.21.15.90:0/3816754719 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c004780 con 0x7f3ca410c5a0 2024-05-27T07:41:03.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.122+0000 7f3ca9c00700 1 -- 172.21.15.90:0/3816754719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca41a9a20 con 0x7f3ca410c5a0 2024-05-27T07:41:03.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.123+0000 7f3ca9c00700 1 -- 172.21.15.90:0/3816754719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca410e960 con 0x7f3ca410c5a0 2024-05-27T07:41:03.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.127+0000 7f3c9b400700 1 -- 172.21.15.90:0/3816754719 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3c9c003f20 con 0x7f3ca410c5a0 2024-05-27T07:41:03.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.128+0000 7f3c9b400700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3c880b2900 0x7f3c880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:03.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.129+0000 7f3c9b400700 1 -- 172.21.15.90:0/3816754719 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f3c9c00f020 con 0x7f3ca410c5a0 2024-05-27T07:41:03.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.129+0000 7f3ca2a00700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3c880b2900 0x7f3c880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:03.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.130+0000 7f3ca2a00700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3c880b2900 0x7f3c880b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c94006fd0 tx=0x7f3c94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:03.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.132+0000 7f3c9b400700 1 -- 172.21.15.90:0/3816754719 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c9c02a020 con 0x7f3ca410c5a0 2024-05-27T07:41:03.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.370+0000 7f3ca9c00700 1 -- 172.21.15.90:0/3816754719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3ca4107c80 con 0x7f3ca410c5a0 2024-05-27T07:41:03.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.757+0000 7f3c9b400700 1 -- 172.21.15.90:0/3816754719 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f3c9c0ab940 con 0x7f3ca410c5a0 2024-05-27T07:41:03.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.762+0000 7f3ca9c00700 1 -- 172.21.15.90:0/3816754719 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3c880b2900 msgr2=0x7f3c880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:03.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.762+0000 7f3ca9c00700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3c880b2900 0x7f3c880b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c94006fd0 tx=0x7f3c94008040 comp rx=0 tx=0).stop 2024-05-27T07:41:03.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.762+0000 7f3ca9c00700 1 -- 172.21.15.90:0/3816754719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca410c5a0 msgr2=0x7f3ca41a1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:03.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.762+0000 7f3ca9c00700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca410c5a0 0x7f3ca41a1770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00b070 tx=0x7f3c9c004e00 comp rx=0 tx=0).stop 2024-05-27T07:41:03.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.762+0000 7f3ca9c00700 1 -- 172.21.15.90:0/3816754719 shutdown_connections 2024-05-27T07:41:03.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.762+0000 7f3ca9c00700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3c880b2900 0x7f3c880b4e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:03.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.762+0000 7f3ca9c00700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ca410c5a0 0x7f3ca41a1770 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:03.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.762+0000 7f3ca9c00700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ca4106560 0x7f3ca419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:03.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.762+0000 7f3ca9c00700 1 --2- 172.21.15.90:0/3816754719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ca4106eb0 0x7f3ca419d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:03.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.762+0000 7f3ca9c00700 1 -- 172.21.15.90:0/3816754719 >> 172.21.15.90:0/3816754719 conn(0x7f3ca4102080 msgr2=0x7f3ca41047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:03.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.763+0000 7f3ca9c00700 1 -- 172.21.15.90:0/3816754719 shutdown_connections 2024-05-27T07:41:03.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:03.763+0000 7f3ca9c00700 1 -- 172.21.15.90:0/3816754719 wait complete. 2024-05-27T07:41:03.764 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_metadata' created 2024-05-27T07:41:03.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:03 smithi136 ceph-mon[35324]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:03.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:03 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3816754719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T07:41:04.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:03 smithi129 ceph-mon[35254]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:04.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:03 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3816754719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T07:41:04.194 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:03 smithi090 ceph-mon[31429]: pgmap v365: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:04.194 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:03 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3816754719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T07:41:04.680 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-27T07:41:04.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:04 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3816754719' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T07:41:04.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:04 smithi090 ceph-mon[31429]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T07:41:04.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:04 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:04.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:04 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:04.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:04 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:04.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:04 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:04 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3816754719' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T07:41:05.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:04 smithi129 ceph-mon[35254]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T07:41:05.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:04 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:04 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:04 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:04 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.127 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:05.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:04 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3816754719' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T07:41:05.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:04 smithi136 ceph-mon[35324]: osdmap e74: 12 total, 12 up, 12 in 2024-05-27T07:41:05.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:04 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:04 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:04 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:04 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:05 smithi090 ceph-mon[31429]: pgmap v367: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:05.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:05 smithi090 ceph-mon[31429]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T07:41:05.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:05 smithi090 ceph-mon[31429]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:41:05.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:05 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:05 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:05 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:41:05.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:05 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:41:05.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:05 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:05.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:05 smithi090 ceph-mon[31429]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T07:41:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:05 smithi129 ceph-mon[35254]: pgmap v367: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:05 smithi129 ceph-mon[35254]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T07:41:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:05 smithi129 ceph-mon[35254]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:41:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:05 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:05 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:05 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:41:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:05 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:41:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:05 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:05 smithi129 ceph-mon[35254]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T07:41:06.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:05 smithi136 ceph-mon[35324]: pgmap v367: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:06.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:05 smithi136 ceph-mon[35324]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T07:41:06.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:05 smithi136 ceph-mon[35324]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:41:06.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:05 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:06.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:05 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:06.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:05 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:41:06.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:05 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:41:06.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:05 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:41:06.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:05 smithi136 ceph-mon[35324]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T07:41:07.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.349+0000 7f061de00700 1 -- 172.21.15.90:0/673883070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f061810b2d0 msgr2=0x7f0618117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.349+0000 7f061de00700 1 --2- 172.21.15.90:0/673883070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f061810b2d0 0x7f0618117800 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f061000b0a0 tx=0x7f061000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.349+0000 7f061de00700 1 -- 172.21.15.90:0/673883070 shutdown_connections 2024-05-27T07:41:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.349+0000 7f061de00700 1 --2- 172.21.15.90:0/673883070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f061810b2d0 0x7f0618117800 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.349+0000 7f061de00700 1 --2- 172.21.15.90:0/673883070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618110960 0x7f0618110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.349+0000 7f061de00700 1 --2- 172.21.15.90:0/673883070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f061810a980 0x7f061810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.349+0000 7f061de00700 1 -- 172.21.15.90:0/673883070 >> 172.21.15.90:0/673883070 conn(0x7f0618077c70 msgr2=0x7f06180780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.349+0000 7f061de00700 1 -- 172.21.15.90:0/673883070 shutdown_connections 2024-05-27T07:41:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.349+0000 7f061de00700 1 -- 172.21.15.90:0/673883070 wait complete. 2024-05-27T07:41:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.350+0000 7f061de00700 1 Processor -- start 2024-05-27T07:41:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.350+0000 7f061de00700 1 -- start start 2024-05-27T07:41:07.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.351+0000 7f061de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f061810a980 0x7f06181a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:07.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.351+0000 7f061de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f061810b2d0 0x7f06181a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:07.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.351+0000 7f061de00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618110960 0x7f06181a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:07.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.351+0000 7f061de00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f061811a930 con 0x7f061810b2d0 2024-05-27T07:41:07.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.351+0000 7f061de00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f061811a7b0 con 0x7f0618110960 2024-05-27T07:41:07.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.351+0000 7f061de00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f061811aab0 con 0x7f061810a980 2024-05-27T07:41:07.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.351+0000 7f060e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f061810b2d0 0x7f06181a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.351+0000 7f060e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f061810b2d0 0x7f06181a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50242/0 (socket says 172.21.15.90:50242) 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.351+0000 7f060e000700 1 -- 172.21.15.90:0/3164366136 learned_addr learned my addr 172.21.15.90:0/3164366136 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.351+0000 7f0616a00700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f061810a980 0x7f06181a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f060e000700 1 -- 172.21.15.90:0/3164366136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f061810a980 msgr2=0x7f06181a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f060e000700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f061810a980 0x7f06181a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f060e000700 1 -- 172.21.15.90:0/3164366136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618110960 msgr2=0x7f06181a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f060e000700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618110960 0x7f06181a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f060e000700 1 -- 172.21.15.90:0/3164366136 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06181a61b0 con 0x7f061810b2d0 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f060e000700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f061810b2d0 0x7f06181a1880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f060000d980 tx=0x7f060000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f061ca00700 1 -- 172.21.15.90:0/3164366136 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0600017070 con 0x7f061810b2d0 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f061ca00700 1 -- 172.21.15.90:0/3164366136 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0600010460 con 0x7f061810b2d0 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f061de00700 1 -- 172.21.15.90:0/3164366136 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06181a64a0 con 0x7f061810b2d0 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f061ca00700 1 -- 172.21.15.90:0/3164366136 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060000de60 con 0x7f061810b2d0 2024-05-27T07:41:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.352+0000 7f061de00700 1 -- 172.21.15.90:0/3164366136 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06181addc0 con 0x7f061810b2d0 2024-05-27T07:41:07.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.354+0000 7f061de00700 1 -- 172.21.15.90:0/3164366136 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06180512b0 con 0x7f061810b2d0 2024-05-27T07:41:07.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.357+0000 7f061ca00700 1 -- 172.21.15.90:0/3164366136 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0600010610 con 0x7f061810b2d0 2024-05-27T07:41:07.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.358+0000 7f061ca00700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05fc0b2900 0x7f05fc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:07.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.358+0000 7f061ca00700 1 -- 172.21.15.90:0/3164366136 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f06000e1a10 con 0x7f061810b2d0 2024-05-27T07:41:07.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.358+0000 7f0616a00700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05fc0b2900 0x7f05fc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:07.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.359+0000 7f0616a00700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05fc0b2900 0x7f05fc0b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0608005950 tx=0x7f06080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:07.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.361+0000 7f061ca00700 1 -- 172.21.15.90:0/3164366136 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f06000a9a40 con 0x7f061810b2d0 2024-05-27T07:41:07.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.593+0000 7f061de00700 1 -- 172.21.15.90:0/3164366136 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f061802e5e0 con 0x7f061810b2d0 2024-05-27T07:41:07.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.786+0000 7f061ca00700 1 -- 172.21.15.90:0/3164366136 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v77) ==== 171+0+0 (secure 0 0 0) 0x7f06000a9340 con 0x7f061810b2d0 2024-05-27T07:41:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.791+0000 7f061de00700 1 -- 172.21.15.90:0/3164366136 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05fc0b2900 msgr2=0x7f05fc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.791+0000 7f061de00700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05fc0b2900 0x7f05fc0b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0608005950 tx=0x7f06080058e0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.791+0000 7f061de00700 1 -- 172.21.15.90:0/3164366136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f061810b2d0 msgr2=0x7f06181a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.791+0000 7f061de00700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f061810b2d0 0x7f06181a1880 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f060000d980 tx=0x7f060000dd40 comp rx=0 tx=0).stop 2024-05-27T07:41:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.791+0000 7f061de00700 1 -- 172.21.15.90:0/3164366136 shutdown_connections 2024-05-27T07:41:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.791+0000 7f061de00700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05fc0b2900 0x7f05fc0b4e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.791+0000 7f061de00700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618110960 0x7f06181a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.791+0000 7f061de00700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f061810a980 0x7f06181a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.791+0000 7f061de00700 1 --2- 172.21.15.90:0/3164366136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f061810b2d0 0x7f06181a1880 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.791+0000 7f061de00700 1 -- 172.21.15.90:0/3164366136 >> 172.21.15.90:0/3164366136 conn(0x7f0618077c70 msgr2=0x7f061807b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.792+0000 7f061de00700 1 -- 172.21.15.90:0/3164366136 shutdown_connections 2024-05-27T07:41:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:07.792+0000 7f061de00700 1 -- 172.21.15.90:0/3164366136 wait complete. 2024-05-27T07:41:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data' created 2024-05-27T07:41:08.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:07 smithi129 ceph-mon[35254]: pgmap v370: 33 pgs: 14 active+clean, 1 creating+activating, 6 creating+peering, 12 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:08.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:07 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3164366136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-27T07:41:08.191 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:07 smithi090 ceph-mon[31429]: pgmap v370: 33 pgs: 14 active+clean, 1 creating+activating, 6 creating+peering, 12 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:08.191 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:07 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3164366136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-27T07:41:08.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:07 smithi136 ceph-mon[35324]: pgmap v370: 33 pgs: 14 active+clean, 1 creating+activating, 6 creating+peering, 12 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:08.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:07 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3164366136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-27T07:41:08.731 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-27T07:41:08.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:08 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3164366136' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-27T07:41:08.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:08 smithi090 ceph-mon[31429]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T07:41:08.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:08 smithi090 ceph-mon[31429]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T07:41:09.184 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:09.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:08 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3164366136' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-27T07:41:09.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:08 smithi136 ceph-mon[35324]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T07:41:09.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:08 smithi136 ceph-mon[35324]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T07:41:09.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:08 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3164366136' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-27T07:41:09.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:08 smithi129 ceph-mon[35254]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T07:41:09.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:08 smithi129 ceph-mon[35254]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T07:41:09.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:09 smithi090 ceph-mon[31429]: pgmap v372: 97 pgs: 26 active+clean, 1 creating+activating, 23 creating+peering, 47 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:09.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:09 smithi090 ceph-mon[31429]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T07:41:10.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:09 smithi136 ceph-mon[35324]: pgmap v372: 97 pgs: 26 active+clean, 1 creating+activating, 23 creating+peering, 47 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:10.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:09 smithi136 ceph-mon[35324]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T07:41:10.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:09 smithi129 ceph-mon[35254]: pgmap v372: 97 pgs: 26 active+clean, 1 creating+activating, 23 creating+peering, 47 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:10.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:09 smithi129 ceph-mon[35254]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T07:41:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.412+0000 7f0e5fa00700 1 -- 172.21.15.90:0/162050995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e5810a980 msgr2=0x7f0e5810ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.412+0000 7f0e5fa00700 1 --2- 172.21.15.90:0/162050995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e5810a980 0x7f0e5810ad90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009990 tx=0x7f0e40009ca0 comp rx=0 tx=0).stop 2024-05-27T07:41:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.412+0000 7f0e5fa00700 1 -- 172.21.15.90:0/162050995 shutdown_connections 2024-05-27T07:41:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.412+0000 7f0e5fa00700 1 --2- 172.21.15.90:0/162050995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5810b2d0 0x7f0e58117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.412+0000 7f0e5fa00700 1 --2- 172.21.15.90:0/162050995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e58110960 0x7f0e58110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.412+0000 7f0e5fa00700 1 --2- 172.21.15.90:0/162050995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e5810a980 0x7f0e5810ad90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.412+0000 7f0e5fa00700 1 -- 172.21.15.90:0/162050995 >> 172.21.15.90:0/162050995 conn(0x7f0e58077c70 msgr2=0x7f0e580780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:11.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.412+0000 7f0e5fa00700 1 -- 172.21.15.90:0/162050995 shutdown_connections 2024-05-27T07:41:11.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.412+0000 7f0e5fa00700 1 -- 172.21.15.90:0/162050995 wait complete. 2024-05-27T07:41:11.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.414+0000 7f0e5fa00700 1 Processor -- start 2024-05-27T07:41:11.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.414+0000 7f0e5fa00700 1 -- start start 2024-05-27T07:41:11.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.414+0000 7f0e5fa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5810a980 0x7f0e581a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:11.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.414+0000 7f0e5fa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e5810b2d0 0x7f0e581a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:11.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.414+0000 7f0e5fa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58110960 0x7f0e581a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:11.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.414+0000 7f0e5fa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5811aa60 con 0x7f0e58110960 2024-05-27T07:41:11.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.414+0000 7f0e5fa00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5811a8e0 con 0x7f0e5810b2d0 2024-05-27T07:41:11.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.414+0000 7f0e5fa00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5811abe0 con 0x7f0e5810a980 2024-05-27T07:41:11.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e5da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58110960 0x7f0e581a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:11.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e5da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58110960 0x7f0e581a5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49006/0 (socket says 172.21.15.90:49006) 2024-05-27T07:41:11.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e5da00700 1 -- 172.21.15.90:0/1573992777 learned_addr learned my addr 172.21.15.90:0/1573992777 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:11.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e5d000700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5810a980 0x7f0e581a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:11.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e4fe00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e5810b2d0 0x7f0e581a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:11.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e5da00700 1 -- 172.21.15.90:0/1573992777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5810a980 msgr2=0x7f0e581a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:11.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e5da00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5810a980 0x7f0e581a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:11.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e5da00700 1 -- 172.21.15.90:0/1573992777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e5810b2d0 msgr2=0x7f0e581a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:11.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e5da00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e5810b2d0 0x7f0e581a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e5da00700 1 -- 172.21.15.90:0/1573992777 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e581a63e0 con 0x7f0e58110960 2024-05-27T07:41:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.415+0000 7f0e4fe00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e5810b2d0 0x7f0e581a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.416+0000 7f0e5da00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58110960 0x7f0e581a5d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000b970 tx=0x7f0e5000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.416+0000 7f0e56000700 1 -- 172.21.15.90:0/1573992777 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50019070 con 0x7f0e58110960 2024-05-27T07:41:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.416+0000 7f0e56000700 1 -- 172.21.15.90:0/1573992777 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e5000cdc0 con 0x7f0e58110960 2024-05-27T07:41:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.416+0000 7f0e5fa00700 1 -- 172.21.15.90:0/1573992777 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e581a66d0 con 0x7f0e58110960 2024-05-27T07:41:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.416+0000 7f0e56000700 1 -- 172.21.15.90:0/1573992777 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50014e30 con 0x7f0e58110960 2024-05-27T07:41:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.416+0000 7f0e5fa00700 1 -- 172.21.15.90:0/1573992777 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e581adff0 con 0x7f0e58110960 2024-05-27T07:41:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.417+0000 7f0e5fa00700 1 -- 172.21.15.90:0/1573992777 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5810c070 con 0x7f0e58110960 2024-05-27T07:41:11.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.420+0000 7f0e56000700 1 -- 172.21.15.90:0/1573992777 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0e5001c070 con 0x7f0e58110960 2024-05-27T07:41:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.422+0000 7f0e56000700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e3c0b2880 0x7f0e3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:11.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.422+0000 7f0e56000700 1 -- 172.21.15.90:0/1573992777 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f0e500e1880 con 0x7f0e58110960 2024-05-27T07:41:11.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.422+0000 7f0e5d000700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e3c0b2880 0x7f0e3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:11.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.423+0000 7f0e5d000700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e3c0b2880 0x7f0e3c0b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e5807d0e0 tx=0x7f0e48009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:11.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.425+0000 7f0e56000700 1 -- 172.21.15.90:0/1573992777 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e500a9750 con 0x7f0e58110960 2024-05-27T07:41:11.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.707+0000 7f0e5fa00700 1 -- 172.21.15.90:0/1573992777 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0e5802e5e0 con 0x7f0e58110960 2024-05-27T07:41:11.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.844+0000 7f0e56000700 1 -- 172.21.15.90:0/1573992777 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f0e500a9050 con 0x7f0e58110960 2024-05-27T07:41:11.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.848+0000 7f0e5fa00700 1 -- 172.21.15.90:0/1573992777 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e3c0b2880 msgr2=0x7f0e3c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:11.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.848+0000 7f0e5fa00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e3c0b2880 0x7f0e3c0b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e5807d0e0 tx=0x7f0e48009190 comp rx=0 tx=0).stop 2024-05-27T07:41:11.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.848+0000 7f0e5fa00700 1 -- 172.21.15.90:0/1573992777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58110960 msgr2=0x7f0e581a5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:11.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.848+0000 7f0e5fa00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58110960 0x7f0e581a5d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000b970 tx=0x7f0e5000bc80 comp rx=0 tx=0).stop 2024-05-27T07:41:11.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.850+0000 7f0e5fa00700 1 -- 172.21.15.90:0/1573992777 shutdown_connections 2024-05-27T07:41:11.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.850+0000 7f0e5fa00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e3c0b2880 0x7f0e3c0b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:11.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.850+0000 7f0e5fa00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e58110960 0x7f0e581a5d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:11.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.850+0000 7f0e5fa00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5810a980 0x7f0e581a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:11.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.850+0000 7f0e5fa00700 1 --2- 172.21.15.90:0/1573992777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e5810b2d0 0x7f0e581a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:11.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.850+0000 7f0e5fa00700 1 -- 172.21.15.90:0/1573992777 >> 172.21.15.90:0/1573992777 conn(0x7f0e58077c70 msgr2=0x7f0e5807b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:11.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.850+0000 7f0e5fa00700 1 -- 172.21.15.90:0/1573992777 shutdown_connections 2024-05-27T07:41:11.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:11.850+0000 7f0e5fa00700 1 -- 172.21.15.90:0/1573992777 wait complete. 2024-05-27T07:41:11.850 INFO:teuthology.orchestra.run.smithi090.stderr:new fs with metadata pool 2 and data pool 3 2024-05-27T07:41:11.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:11 smithi090 ceph-mon[31429]: pgmap v375: 97 pgs: 56 active+clean, 17 creating+peering, 24 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:11.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:11 smithi090 ceph-mon[31429]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:41:11.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:11 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1573992777' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T07:41:11.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:11 smithi090 ceph-mon[31429]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T07:41:12.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:11 smithi136 ceph-mon[35324]: pgmap v375: 97 pgs: 56 active+clean, 17 creating+peering, 24 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:12.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:11 smithi136 ceph-mon[35324]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:41:12.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:11 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1573992777' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T07:41:12.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:11 smithi136 ceph-mon[35324]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T07:41:12.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:11 smithi129 ceph-mon[35254]: pgmap v375: 97 pgs: 56 active+clean, 17 creating+peering, 24 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:12.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:11 smithi129 ceph-mon[35254]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:41:12.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:11 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1573992777' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T07:41:12.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:11 smithi129 ceph-mon[35254]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T07:41:12.748 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-27T07:41:12.748 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-27T07:41:12.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:12 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1573992777' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T07:41:12.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:12 smithi090 ceph-mon[31429]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T07:41:12.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:12 smithi090 ceph-mon[31429]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T07:41:12.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:12 smithi090 ceph-mon[31429]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T07:41:13.149 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:12 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1573992777' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T07:41:13.149 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:12 smithi136 ceph-mon[35324]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T07:41:13.150 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:12 smithi136 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T07:41:13.150 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:12 smithi136 ceph-mon[35324]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T07:41:13.202 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:13.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:12 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1573992777' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T07:41:13.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:12 smithi129 ceph-mon[35254]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T07:41:13.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:12 smithi129 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T07:41:13.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:12 smithi129 ceph-mon[35254]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T07:41:13.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:13 smithi090 ceph-mon[31429]: pgmap v377: 97 pgs: 80 active+clean, 17 creating+peering; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-27T07:41:13.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:13 smithi090 ceph-mon[31429]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T07:41:13.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:13 smithi090 ceph-mon[31429]: Cluster is now healthy 2024-05-27T07:41:13.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:13 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] up:active 2024-05-27T07:41:13.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:13 smithi090 ceph-mon[31429]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:41:13.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:13 smithi090 ceph-mon[31429]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:41:14.182 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:13 smithi136 ceph-mon[35324]: pgmap v377: 97 pgs: 80 active+clean, 17 creating+peering; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-27T07:41:14.182 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:13 smithi136 ceph-mon[35324]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T07:41:14.182 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:13 smithi136 ceph-mon[35324]: Cluster is now healthy 2024-05-27T07:41:14.182 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:13 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] up:active 2024-05-27T07:41:14.182 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:13 smithi136 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:41:14.183 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:13 smithi136 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:41:14.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:13 smithi129 ceph-mon[35254]: pgmap v377: 97 pgs: 80 active+clean, 17 creating+peering; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-27T07:41:14.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:13 smithi129 ceph-mon[35254]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T07:41:14.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:13 smithi129 ceph-mon[35254]: Cluster is now healthy 2024-05-27T07:41:14.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:13 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] up:active 2024-05-27T07:41:14.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:13 smithi129 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:41:14.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:13 smithi129 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:41:15.061 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:41:14 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b[65300]: 2024-05-27T07:41:14.690+0000 7f8484000700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-27T07:41:15.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.336+0000 7f31cce00700 1 -- 172.21.15.90:0/177361856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31c8075ad0 msgr2=0x7f31c8075ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:15.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.336+0000 7f31cce00700 1 --2- 172.21.15.90:0/177361856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31c8075ad0 0x7f31c8075ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009990 tx=0x7f31b8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.337+0000 7f31cce00700 1 -- 172.21.15.90:0/177361856 shutdown_connections 2024-05-27T07:41:15.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.337+0000 7f31cce00700 1 --2- 172.21.15.90:0/177361856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31c8110c00 0x7f31c8113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.337+0000 7f31cce00700 1 --2- 172.21.15.90:0/177361856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31c8075ad0 0x7f31c8075ee0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.337+0000 7f31cce00700 1 --2- 172.21.15.90:0/177361856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8076420 0x7f31c81106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.337+0000 7f31cce00700 1 -- 172.21.15.90:0/177361856 >> 172.21.15.90:0/177361856 conn(0x7f31c80ffe60 msgr2=0x7f31c81022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:15.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.337+0000 7f31cce00700 1 -- 172.21.15.90:0/177361856 shutdown_connections 2024-05-27T07:41:15.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.337+0000 7f31cce00700 1 -- 172.21.15.90:0/177361856 wait complete. 2024-05-27T07:41:15.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.338+0000 7f31cce00700 1 Processor -- start 2024-05-27T07:41:15.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.339+0000 7f31cce00700 1 -- start start 2024-05-27T07:41:15.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.339+0000 7f31cce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31c8075ad0 0x7f31c819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:15.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.339+0000 7f31cce00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31c8076420 0x7f31c819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:15.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.339+0000 7f31cce00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8110c00 0x7f31c81a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:15.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.339+0000 7f31cce00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8116600 con 0x7f31c8075ad0 2024-05-27T07:41:15.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.339+0000 7f31cce00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8116480 con 0x7f31c8110c00 2024-05-27T07:41:15.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.339+0000 7f31cce00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f31c8116780 con 0x7f31c8076420 2024-05-27T07:41:15.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31c8075ad0 0x7f31c819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:15.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31c8075ad0 0x7f31c819cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49022/0 (socket says 172.21.15.90:49022) 2024-05-27T07:41:15.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c5e00700 1 -- 172.21.15.90:0/3244091313 learned_addr learned my addr 172.21.15.90:0/3244091313 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:15.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c5400700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31c8076420 0x7f31c819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:15.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c6800700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8110c00 0x7f31c81a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:15.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c5e00700 1 -- 172.21.15.90:0/3244091313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31c8076420 msgr2=0x7f31c819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:15.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c5e00700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31c8076420 0x7f31c819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c5e00700 1 -- 172.21.15.90:0/3244091313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8110c00 msgr2=0x7f31c81a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:15.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c5e00700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8110c00 0x7f31c81a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c5e00700 1 -- 172.21.15.90:0/3244091313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c81a1df0 con 0x7f31c8075ad0 2024-05-27T07:41:15.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.340+0000 7f31c5400700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31c8076420 0x7f31c819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:41:15.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.341+0000 7f31c5e00700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31c8075ad0 0x7f31c819cfd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31b8000c00 tx=0x7f31b80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:15.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.341+0000 7f31bea00700 1 -- 172.21.15.90:0/3244091313 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8027070 con 0x7f31c8075ad0 2024-05-27T07:41:15.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.341+0000 7f31cce00700 1 -- 172.21.15.90:0/3244091313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c81a2080 con 0x7f31c8075ad0 2024-05-27T07:41:15.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.341+0000 7f31bea00700 1 -- 172.21.15.90:0/3244091313 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31b800fc80 con 0x7f31c8075ad0 2024-05-27T07:41:15.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.341+0000 7f31bea00700 1 -- 172.21.15.90:0/3244091313 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b8022d60 con 0x7f31c8075ad0 2024-05-27T07:41:15.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.341+0000 7f31cce00700 1 -- 172.21.15.90:0/3244091313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c81a25b0 con 0x7f31c8075ad0 2024-05-27T07:41:15.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.343+0000 7f31cce00700 1 -- 172.21.15.90:0/3244091313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c80512b0 con 0x7f31c8075ad0 2024-05-27T07:41:15.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.345+0000 7f31bea00700 1 -- 172.21.15.90:0/3244091313 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f31b802a070 con 0x7f31c8075ad0 2024-05-27T07:41:15.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.347+0000 7f31bea00700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f31ac0b2880 0x7f31ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:15.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.347+0000 7f31bea00700 1 -- 172.21.15.90:0/3244091313 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f31b80e33d0 con 0x7f31c8075ad0 2024-05-27T07:41:15.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.347+0000 7f31c5400700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f31ac0b2880 0x7f31ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:15.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.348+0000 7f31c5400700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f31ac0b2880 0x7f31ac0b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31b4005fd0 tx=0x7f31b4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:15.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.350+0000 7f31bea00700 1 -- 172.21.15.90:0/3244091313 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31b8013070 con 0x7f31c8075ad0 2024-05-27T07:41:15.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.594+0000 7f31cce00700 1 -- 172.21.15.90:0/3244091313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f31c819e330 con 0x7f31c8075ad0 2024-05-27T07:41:15.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.882+0000 7f31bea00700 1 -- 172.21.15.90:0/3244091313 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) ==== 157+0+0 (secure 0 0 0) 0x7f31b80ab2c0 con 0x7f31c8075ad0 2024-05-27T07:41:15.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.888+0000 7f31cce00700 1 -- 172.21.15.90:0/3244091313 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f31ac0b2880 msgr2=0x7f31ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.888+0000 7f31cce00700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f31ac0b2880 0x7f31ac0b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f31b4005fd0 tx=0x7f31b4005e30 comp rx=0 tx=0).stop 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.888+0000 7f31cce00700 1 -- 172.21.15.90:0/3244091313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31c8075ad0 msgr2=0x7f31c819cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.888+0000 7f31cce00700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31c8075ad0 0x7f31c819cfd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31b8000c00 tx=0x7f31b80036f0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.889+0000 7f31cce00700 1 -- 172.21.15.90:0/3244091313 shutdown_connections 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.889+0000 7f31cce00700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f31ac0b2880 0x7f31ac0b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.889+0000 7f31cce00700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c8110c00 0x7f31c81a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.889+0000 7f31cce00700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f31c8075ad0 0x7f31c819cfd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.889+0000 7f31cce00700 1 --2- 172.21.15.90:0/3244091313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31c8076420 0x7f31c819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.889+0000 7f31cce00700 1 -- 172.21.15.90:0/3244091313 >> 172.21.15.90:0/3244091313 conn(0x7f31c80ffe60 msgr2=0x7f31c81004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.889+0000 7f31cce00700 1 -- 172.21.15.90:0/3244091313 shutdown_connections 2024-05-27T07:41:15.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:15.889+0000 7f31cce00700 1 -- 172.21.15.90:0/3244091313 wait complete. 2024-05-27T07:41:16.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:15 smithi090 ceph-mon[31429]: pgmap v378: 97 pgs: 97 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-27T07:41:16.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:15 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3244091313' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-27T07:41:16.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:15 smithi136 ceph-mon[35324]: pgmap v378: 97 pgs: 97 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-27T07:41:16.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:15 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3244091313' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-27T07:41:16.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:15 smithi129 ceph-mon[35254]: pgmap v378: 97 pgs: 97 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 1 op/s 2024-05-27T07:41:16.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:15 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3244091313' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-27T07:41:16.975 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-27T07:41:17.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:16 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3244091313' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-27T07:41:17.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:16 smithi090 ceph-mon[31429]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T07:41:17.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:16 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3244091313' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-27T07:41:17.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:16 smithi136 ceph-mon[35324]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T07:41:17.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:16 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3244091313' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-27T07:41:17.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:16 smithi129 ceph-mon[35254]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T07:41:17.428 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:18.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:17 smithi090 ceph-mon[31429]: pgmap v380: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-05-27T07:41:18.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:17 smithi136 ceph-mon[35324]: pgmap v380: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-05-27T07:41:18.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:17 smithi129 ceph-mon[35254]: pgmap v380: 97 pgs: 97 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-05-27T07:41:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.584+0000 7fcec2c00700 1 -- 172.21.15.90:0/1350354746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcebc10a7b0 msgr2=0x7fcebc10abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.584+0000 7fcec2c00700 1 --2- 172.21.15.90:0/1350354746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcebc10a7b0 0x7fcebc10abc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009990 tx=0x7fcea8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:41:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.584+0000 7fcec2c00700 1 -- 172.21.15.90:0/1350354746 shutdown_connections 2024-05-27T07:41:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.584+0000 7fcec2c00700 1 --2- 172.21.15.90:0/1350354746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcebc10b100 0x7fcebc117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.584+0000 7fcec2c00700 1 --2- 172.21.15.90:0/1350354746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcebc10f6b0 0x7fcebc10fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.584+0000 7fcec2c00700 1 --2- 172.21.15.90:0/1350354746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcebc10a7b0 0x7fcebc10abc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.584+0000 7fcec2c00700 1 -- 172.21.15.90:0/1350354746 >> 172.21.15.90:0/1350354746 conn(0x7fcebc076af0 msgr2=0x7fcebc076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.584+0000 7fcec2c00700 1 -- 172.21.15.90:0/1350354746 shutdown_connections 2024-05-27T07:41:19.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.584+0000 7fcec2c00700 1 -- 172.21.15.90:0/1350354746 wait complete. 2024-05-27T07:41:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.586+0000 7fcec2c00700 1 Processor -- start 2024-05-27T07:41:19.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.586+0000 7fcec2c00700 1 -- start start 2024-05-27T07:41:19.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.586+0000 7fcec2c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcebc10a7b0 0x7fcebc1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:19.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.586+0000 7fcec2c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcebc10b100 0x7fcebc1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:19.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.586+0000 7fcec2c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcebc10f6b0 0x7fcebc1a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:19.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.586+0000 7fcec2c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc11a7a0 con 0x7fcebc10a7b0 2024-05-27T07:41:19.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.586+0000 7fcec2c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc11a620 con 0x7fcebc10b100 2024-05-27T07:41:19.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.586+0000 7fcec2c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc11a920 con 0x7fcebc10f6b0 2024-05-27T07:41:19.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.587+0000 7fcec1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcebc10a7b0 0x7fcebc1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:19.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.587+0000 7fcec1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcebc10a7b0 0x7fcebc1a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34904/0 (socket says 172.21.15.90:34904) 2024-05-27T07:41:19.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.587+0000 7fcec1800700 1 -- 172.21.15.90:0/1277414386 learned_addr learned my addr 172.21.15.90:0/1277414386 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:19.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.587+0000 7fcec0e00700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcebc10b100 0x7fcebc1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:19.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.587+0000 7fcec2200700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcebc10f6b0 0x7fcebc1a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:19.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.587+0000 7fcec1800700 1 -- 172.21.15.90:0/1277414386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcebc10f6b0 msgr2=0x7fcebc1a59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:19.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.587+0000 7fcec1800700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcebc10f6b0 0x7fcebc1a59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:19.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.587+0000 7fcec1800700 1 -- 172.21.15.90:0/1277414386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcebc10b100 msgr2=0x7fcebc1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:19.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.587+0000 7fcec1800700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcebc10b100 0x7fcebc1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:19.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.587+0000 7fcec1800700 1 -- 172.21.15.90:0/1277414386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcebc1a5f90 con 0x7fcebc10a7b0 2024-05-27T07:41:19.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.588+0000 7fcec1800700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcebc10a7b0 0x7fcebc1a1170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fceb000c9b0 tx=0x7fceb000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:19.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.588+0000 7fceba000700 1 -- 172.21.15.90:0/1277414386 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0007b00 con 0x7fcebc10a7b0 2024-05-27T07:41:19.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.588+0000 7fceba000700 1 -- 172.21.15.90:0/1277414386 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fceb0007ca0 con 0x7fcebc10a7b0 2024-05-27T07:41:19.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.588+0000 7fcec2c00700 1 -- 172.21.15.90:0/1277414386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcebc1a6280 con 0x7fcebc10a7b0 2024-05-27T07:41:19.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.588+0000 7fceba000700 1 -- 172.21.15.90:0/1277414386 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0018660 con 0x7fcebc10a7b0 2024-05-27T07:41:19.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.588+0000 7fcec2c00700 1 -- 172.21.15.90:0/1277414386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcebc1a67c0 con 0x7fcebc10a7b0 2024-05-27T07:41:19.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.589+0000 7fcec2c00700 1 -- 172.21.15.90:0/1277414386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcebc0512b0 con 0x7fcebc10a7b0 2024-05-27T07:41:19.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.592+0000 7fceba000700 1 -- 172.21.15.90:0/1277414386 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fceb000f450 con 0x7fcebc10a7b0 2024-05-27T07:41:19.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.593+0000 7fceba000700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcea40b2880 0x7fcea40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:19.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.594+0000 7fceba000700 1 -- 172.21.15.90:0/1277414386 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8277+0+0 (secure 0 0 0) 0x7fceb00e22e0 con 0x7fcebc10a7b0 2024-05-27T07:41:19.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.594+0000 7fcec0e00700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcea40b2880 0x7fcea40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:19.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.594+0000 7fcec0e00700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcea40b2880 0x7fcea40b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcea800b5c0 tx=0x7fcea8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:19.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.601+0000 7fceba000700 1 -- 172.21.15.90:0/1277414386 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fceb0014030 con 0x7fcebc10a7b0 2024-05-27T07:41:19.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:19.836+0000 7fcec2c00700 1 -- 172.21.15.90:0/1277414386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcebc02e5e0 con 0x7fcebc10a7b0 2024-05-27T07:41:20.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:19 smithi090 ceph-mon[31429]: pgmap v381: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:41:20.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:19 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1277414386' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T07:41:20.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:19 smithi136 ceph-mon[35324]: pgmap v381: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:41:20.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:19 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1277414386' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T07:41:20.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:19 smithi129 ceph-mon[35254]: pgmap v381: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:41:20.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:19 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1277414386' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T07:41:20.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.950+0000 7fceba000700 1 -- 172.21.15.90:0/1277414386 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) ==== 228+0+0 (secure 0 0 0) 0x7fceb00a9f70 con 0x7fcebc10a7b0 2024-05-27T07:41:20.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.955+0000 7fcec2c00700 1 -- 172.21.15.90:0/1277414386 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcea40b2880 msgr2=0x7fcea40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:20.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.955+0000 7fcec2c00700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcea40b2880 0x7fcea40b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcea800b5c0 tx=0x7fcea8005fb0 comp rx=0 tx=0).stop 2024-05-27T07:41:20.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.955+0000 7fcec2c00700 1 -- 172.21.15.90:0/1277414386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcebc10a7b0 msgr2=0x7fcebc1a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:20.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.955+0000 7fcec2c00700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcebc10a7b0 0x7fcebc1a1170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fceb000c9b0 tx=0x7fceb000cd70 comp rx=0 tx=0).stop 2024-05-27T07:41:20.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.956+0000 7fcec2c00700 1 -- 172.21.15.90:0/1277414386 shutdown_connections 2024-05-27T07:41:20.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.956+0000 7fcec2c00700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcea40b2880 0x7fcea40b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:20.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.956+0000 7fcec2c00700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcebc10f6b0 0x7fcebc1a59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:20.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.956+0000 7fcec2c00700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcebc10a7b0 0x7fcebc1a1170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:20.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.956+0000 7fcec2c00700 1 --2- 172.21.15.90:0/1277414386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcebc10b100 0x7fcebc1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:20.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.956+0000 7fcec2c00700 1 -- 172.21.15.90:0/1277414386 >> 172.21.15.90:0/1277414386 conn(0x7fcebc076af0 msgr2=0x7fcebc1088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:20.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.956+0000 7fcec2c00700 1 -- 172.21.15.90:0/1277414386 shutdown_connections 2024-05-27T07:41:20.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:20.956+0000 7fcec2c00700 1 -- 172.21.15.90:0/1277414386 wait complete. 2024-05-27T07:41:20.956 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data_ec' created 2024-05-27T07:41:21.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:20 smithi090 ceph-mon[31429]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T07:41:21.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:20 smithi136 ceph-mon[35324]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T07:41:21.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:20 smithi129 ceph-mon[35254]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T07:41:21.934 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-27T07:41:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:21 smithi090 ceph-mon[31429]: pgmap v383: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:41:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:21 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1277414386' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T07:41:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:21 smithi090 ceph-mon[31429]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T07:41:22.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:21 smithi136 ceph-mon[35324]: pgmap v383: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:41:22.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:21 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1277414386' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T07:41:22.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:21 smithi136 ceph-mon[35324]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T07:41:22.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:21 smithi129 ceph-mon[35254]: pgmap v383: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T07:41:22.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:21 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1277414386' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T07:41:22.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:21 smithi129 ceph-mon[35254]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T07:41:22.386 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:23.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:22 smithi129 ceph-mon[35254]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T07:41:23.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:22 smithi136 ceph-mon[35324]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T07:41:23.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:22 smithi090 ceph-mon[31429]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T07:41:24.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:23 smithi090 ceph-mon[31429]: pgmap v386: 129 pgs: 1 creating+activating, 5 creating+peering, 24 unknown, 99 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-05-27T07:41:24.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:23 smithi090 ceph-mon[31429]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:41:24.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:23 smithi090 ceph-mon[31429]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T07:41:24.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:23 smithi136 ceph-mon[35324]: pgmap v386: 129 pgs: 1 creating+activating, 5 creating+peering, 24 unknown, 99 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-05-27T07:41:24.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:23 smithi136 ceph-mon[35324]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:41:24.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:23 smithi136 ceph-mon[35324]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T07:41:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:23 smithi129 ceph-mon[35254]: pgmap v386: 129 pgs: 1 creating+activating, 5 creating+peering, 24 unknown, 99 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2024-05-27T07:41:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:23 smithi129 ceph-mon[35254]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T07:41:24.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:23 smithi129 ceph-mon[35254]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T07:41:24.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.709+0000 7f2d8e200700 1 -- 172.21.15.90:0/441595851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d88109860 msgr2=0x7f2d881111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:24.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.709+0000 7f2d8e200700 1 --2- 172.21.15.90:0/441595851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d88109860 0x7f2d881111c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000b0d0 tx=0x7f2d8000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.709+0000 7f2d8e200700 1 -- 172.21.15.90:0/441595851 shutdown_connections 2024-05-27T07:41:24.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.709+0000 7f2d8e200700 1 --2- 172.21.15.90:0/441595851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d88109860 0x7f2d881111c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.709+0000 7f2d8e200700 1 --2- 172.21.15.90:0/441595851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d881045e0 0x7f2d881049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.709+0000 7f2d8e200700 1 --2- 172.21.15.90:0/441595851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d88108f10 0x7f2d88109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.709+0000 7f2d8e200700 1 -- 172.21.15.90:0/441595851 >> 172.21.15.90:0/441595851 conn(0x7f2d88100030 msgr2=0x7f2d88102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:24.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.709+0000 7f2d8e200700 1 -- 172.21.15.90:0/441595851 shutdown_connections 2024-05-27T07:41:24.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.709+0000 7f2d8e200700 1 -- 172.21.15.90:0/441595851 wait complete. 2024-05-27T07:41:24.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.711+0000 7f2d8e200700 1 Processor -- start 2024-05-27T07:41:24.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.711+0000 7f2d8e200700 1 -- start start 2024-05-27T07:41:24.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.711+0000 7f2d8e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d881045e0 0x7f2d8819d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:24.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.711+0000 7f2d8e200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d88108f10 0x7f2d8819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:24.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.711+0000 7f2d8e200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d88109860 0x7f2d881a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:24.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.711+0000 7f2d8e200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88114480 con 0x7f2d881045e0 2024-05-27T07:41:24.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.711+0000 7f2d8e200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88114300 con 0x7f2d88108f10 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.711+0000 7f2d8e200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88114600 con 0x7f2d88109860 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.711+0000 7f2d87400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d881045e0 0x7f2d8819d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.712+0000 7f2d87400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d881045e0 0x7f2d8819d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34922/0 (socket says 172.21.15.90:34922) 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.712+0000 7f2d87400700 1 -- 172.21.15.90:0/563228123 learned_addr learned my addr 172.21.15.90:0/563228123 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.712+0000 7f2d7ea00700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d88108f10 0x7f2d8819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.712+0000 7f2d87e00700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d88109860 0x7f2d881a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.712+0000 7f2d87400700 1 -- 172.21.15.90:0/563228123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d88109860 msgr2=0x7f2d881a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.712+0000 7f2d87400700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d88109860 0x7f2d881a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.712+0000 7f2d87400700 1 -- 172.21.15.90:0/563228123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d88108f10 msgr2=0x7f2d8819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.712+0000 7f2d87400700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d88108f10 0x7f2d8819d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.712+0000 7f2d87400700 1 -- 172.21.15.90:0/563228123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d881a2040 con 0x7f2d881045e0 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.712+0000 7f2d87400700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d881045e0 0x7f2d8819d170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800b530 tx=0x7f2d7800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:24.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.713+0000 7f2d84c00700 1 -- 172.21.15.90:0/563228123 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78015020 con 0x7f2d881045e0 2024-05-27T07:41:24.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.713+0000 7f2d8e200700 1 -- 172.21.15.90:0/563228123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d881a2330 con 0x7f2d881045e0 2024-05-27T07:41:24.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.713+0000 7f2d84c00700 1 -- 172.21.15.90:0/563228123 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d78011cc0 con 0x7f2d881045e0 2024-05-27T07:41:24.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.713+0000 7f2d84c00700 1 -- 172.21.15.90:0/563228123 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78010d30 con 0x7f2d881045e0 2024-05-27T07:41:24.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.713+0000 7f2d8e200700 1 -- 172.21.15.90:0/563228123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d881a9c50 con 0x7f2d881045e0 2024-05-27T07:41:24.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.714+0000 7f2d8e200700 1 -- 172.21.15.90:0/563228123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d880512b0 con 0x7f2d881045e0 2024-05-27T07:41:24.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.716+0000 7f2d84c00700 1 -- 172.21.15.90:0/563228123 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2d78018020 con 0x7f2d881045e0 2024-05-27T07:41:24.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.719+0000 7f2d84c00700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f2d6c0b2900 0x7f2d6c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:24.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.719+0000 7f2d84c00700 1 -- 172.21.15.90:0/563228123 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f2d780e2220 con 0x7f2d881045e0 2024-05-27T07:41:24.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.719+0000 7f2d7ea00700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f2d6c0b2900 0x7f2d6c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:24.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.720+0000 7f2d7ea00700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f2d6c0b2900 0x7f2d6c0b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2d70006fd0 tx=0x7f2d70008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:24.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.723+0000 7f2d84c00700 1 -- 172.21.15.90:0/563228123 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d7801f020 con 0x7f2d881045e0 2024-05-27T07:41:24.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.961+0000 7f2d8e200700 1 -- 172.21.15.90:0/563228123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f2d8819e4d0 con 0x7f2d881045e0 2024-05-27T07:41:24.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.993+0000 7f2d84c00700 1 -- 172.21.15.90:0/563228123 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) ==== 169+0+0 (secure 0 0 0) 0x7f2d8819e4d0 con 0x7f2d881045e0 2024-05-27T07:41:24.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.996+0000 7f2d8e200700 1 -- 172.21.15.90:0/563228123 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f2d6c0b2900 msgr2=0x7f2d6c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:24.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.996+0000 7f2d8e200700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f2d6c0b2900 0x7f2d6c0b4e10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2d70006fd0 tx=0x7f2d70008040 comp rx=0 tx=0).stop 2024-05-27T07:41:24.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.996+0000 7f2d8e200700 1 -- 172.21.15.90:0/563228123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d881045e0 msgr2=0x7f2d8819d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:24.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.996+0000 7f2d8e200700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d881045e0 0x7f2d8819d170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800b530 tx=0x7f2d7800b8f0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.996+0000 7f2d8e200700 1 -- 172.21.15.90:0/563228123 shutdown_connections 2024-05-27T07:41:24.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.996+0000 7f2d8e200700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f2d6c0b2900 0x7f2d6c0b4e10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.997+0000 7f2d8e200700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d88109860 0x7f2d881a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.997+0000 7f2d8e200700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d881045e0 0x7f2d8819d170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.997+0000 7f2d8e200700 1 --2- 172.21.15.90:0/563228123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d88108f10 0x7f2d8819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:24.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.997+0000 7f2d8e200700 1 -- 172.21.15.90:0/563228123 >> 172.21.15.90:0/563228123 conn(0x7f2d88100030 msgr2=0x7f2d8810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:24.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.997+0000 7f2d8e200700 1 -- 172.21.15.90:0/563228123 shutdown_connections 2024-05-27T07:41:24.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:24.997+0000 7f2d8e200700 1 -- 172.21.15.90:0/563228123 wait complete. 2024-05-27T07:41:24.999 INFO:teuthology.orchestra.run.smithi090.stderr:set pool 4 allow_ec_overwrites to true 2024-05-27T07:41:25.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:24 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/563228123' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-27T07:41:25.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:24 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/563228123' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-27T07:41:25.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:24 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/563228123' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-27T07:41:25.801 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-27T07:41:26.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:25 smithi136 ceph-mon[35324]: pgmap v388: 129 pgs: 1 creating+activating, 5 creating+peering, 15 unknown, 108 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:26.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:25 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/563228123' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-27T07:41:26.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:25 smithi136 ceph-mon[35324]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T07:41:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:26.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:25 smithi129 ceph-mon[35254]: pgmap v388: 129 pgs: 1 creating+activating, 5 creating+peering, 15 unknown, 108 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:26.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:25 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/563228123' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-27T07:41:26.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:25 smithi129 ceph-mon[35254]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T07:41:26.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:25 smithi090 ceph-mon[31429]: pgmap v388: 129 pgs: 1 creating+activating, 5 creating+peering, 15 unknown, 108 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:26.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:25 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/563228123' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-27T07:41:26.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:25 smithi090 ceph-mon[31429]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T07:41:27.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:27 smithi090 ceph-mon[31429]: pgmap v390: 129 pgs: 1 creating+activating, 5 creating+peering, 123 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:27.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:27 smithi129 ceph-mon[35254]: pgmap v390: 129 pgs: 1 creating+activating, 5 creating+peering, 123 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:27.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:27 smithi136 ceph-mon[35324]: pgmap v390: 129 pgs: 1 creating+activating, 5 creating+peering, 123 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:28.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.591+0000 7f5834c00700 1 -- 172.21.15.90:0/2660897310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583010b2d0 msgr2=0x7f5830117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:28.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.591+0000 7f5834c00700 1 --2- 172.21.15.90:0/2660897310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583010b2d0 0x7f5830117800 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f582800b0a0 tx=0x7f582800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:28.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.591+0000 7f5834c00700 1 -- 172.21.15.90:0/2660897310 shutdown_connections 2024-05-27T07:41:28.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.591+0000 7f5834c00700 1 --2- 172.21.15.90:0/2660897310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583010b2d0 0x7f5830117800 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:28.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.591+0000 7f5834c00700 1 --2- 172.21.15.90:0/2660897310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5830110960 0x7f5830110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:28.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.591+0000 7f5834c00700 1 --2- 172.21.15.90:0/2660897310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f583010a980 0x7f583010ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:28.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.591+0000 7f5834c00700 1 -- 172.21.15.90:0/2660897310 >> 172.21.15.90:0/2660897310 conn(0x7f5830077c70 msgr2=0x7f58300780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:28.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.591+0000 7f5834c00700 1 -- 172.21.15.90:0/2660897310 shutdown_connections 2024-05-27T07:41:28.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.591+0000 7f5834c00700 1 -- 172.21.15.90:0/2660897310 wait complete. 2024-05-27T07:41:28.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.592+0000 7f5834c00700 1 Processor -- start 2024-05-27T07:41:28.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.592+0000 7f5834c00700 1 -- start start 2024-05-27T07:41:28.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.593+0000 7f5834c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583010a980 0x7f58301a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:28.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.593+0000 7f5834c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f583010b2d0 0x7f58301a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:28.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.593+0000 7f5834c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5830110960 0x7f58301a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:28.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.593+0000 7f5834c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f583011a930 con 0x7f583010a980 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.593+0000 7f5834c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f583011a7b0 con 0x7f583010b2d0 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.593+0000 7f5834c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f583011aab0 con 0x7f5830110960 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.593+0000 7f582de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583010a980 0x7f58301a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.593+0000 7f582de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583010a980 0x7f58301a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34952/0 (socket says 172.21.15.90:34952) 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.593+0000 7f582de00700 1 -- 172.21.15.90:0/1762306614 learned_addr learned my addr 172.21.15.90:0/1762306614 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f582d400700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f583010b2d0 0x7f58301a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f582e800700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5830110960 0x7f58301a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f582de00700 1 -- 172.21.15.90:0/1762306614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5830110960 msgr2=0x7f58301a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f582de00700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5830110960 0x7f58301a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f582de00700 1 -- 172.21.15.90:0/1762306614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f583010b2d0 msgr2=0x7f58301a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f582de00700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f583010b2d0 0x7f58301a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:28.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f582de00700 1 -- 172.21.15.90:0/1762306614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58301a61e0 con 0x7f583010a980 2024-05-27T07:41:28.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f582de00700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583010a980 0x7f58301a13c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f582000c9e0 tx=0x7f582000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:28.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f5826a00700 1 -- 172.21.15.90:0/1762306614 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820007aa0 con 0x7f583010a980 2024-05-27T07:41:28.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f5826a00700 1 -- 172.21.15.90:0/1762306614 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5820007c40 con 0x7f583010a980 2024-05-27T07:41:28.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.594+0000 7f5834c00700 1 -- 172.21.15.90:0/1762306614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58301a64d0 con 0x7f583010a980 2024-05-27T07:41:28.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.595+0000 7f5826a00700 1 -- 172.21.15.90:0/1762306614 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820018660 con 0x7f583010a980 2024-05-27T07:41:28.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.595+0000 7f5834c00700 1 -- 172.21.15.90:0/1762306614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58301a6a10 con 0x7f583010a980 2024-05-27T07:41:28.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.596+0000 7f5834c00700 1 -- 172.21.15.90:0/1762306614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58300512b0 con 0x7f583010a980 2024-05-27T07:41:28.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.599+0000 7f5826a00700 1 -- 172.21.15.90:0/1762306614 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5820018800 con 0x7f583010a980 2024-05-27T07:41:28.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.600+0000 7f5826a00700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f58140b2880 0x7f58140b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:28.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.601+0000 7f5826a00700 1 -- 172.21.15.90:0/1762306614 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7f58200e25b0 con 0x7f583010a980 2024-05-27T07:41:28.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.601+0000 7f582d400700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f58140b2880 0x7f58140b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:28.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.602+0000 7f582d400700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f58140b2880 0x7f58140b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f581c005950 tx=0x7f581c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:28.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.603+0000 7f5826a00700 1 -- 172.21.15.90:0/1762306614 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5820054df0 con 0x7f583010a980 2024-05-27T07:41:28.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:28.887+0000 7f5834c00700 1 -- 172.21.15.90:0/1762306614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f583002e5e0 con 0x7f583010a980 2024-05-27T07:41:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.438+0000 7f5826a00700 1 -- 172.21.15.90:0/1762306614 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f5820014030 con 0x7f583010a980 2024-05-27T07:41:29.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.443+0000 7f5834c00700 1 -- 172.21.15.90:0/1762306614 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f58140b2880 msgr2=0x7f58140b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:29.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.443+0000 7f5834c00700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f58140b2880 0x7f58140b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f581c005950 tx=0x7f581c0058e0 comp rx=0 tx=0).stop 2024-05-27T07:41:29.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.443+0000 7f5834c00700 1 -- 172.21.15.90:0/1762306614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583010a980 msgr2=0x7f58301a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:29.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.443+0000 7f5834c00700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583010a980 0x7f58301a13c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f582000c9e0 tx=0x7f582000cda0 comp rx=0 tx=0).stop 2024-05-27T07:41:29.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.443+0000 7f5834c00700 1 -- 172.21.15.90:0/1762306614 shutdown_connections 2024-05-27T07:41:29.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.443+0000 7f5834c00700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f58140b2880 0x7f58140b4d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:29.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.443+0000 7f5834c00700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5830110960 0x7f58301a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:29.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.443+0000 7f5834c00700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583010a980 0x7f58301a13c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:29.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.443+0000 7f5834c00700 1 --2- 172.21.15.90:0/1762306614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f583010b2d0 0x7f58301a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:29.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.443+0000 7f5834c00700 1 -- 172.21.15.90:0/1762306614 >> 172.21.15.90:0/1762306614 conn(0x7f5830077c70 msgr2=0x7f583007b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:29.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.444+0000 7f5834c00700 1 -- 172.21.15.90:0/1762306614 shutdown_connections 2024-05-27T07:41:29.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:29.444+0000 7f5834c00700 1 -- 172.21.15.90:0/1762306614 wait complete. 2024-05-27T07:41:29.444 INFO:teuthology.orchestra.run.smithi090.stderr:added data pool 4 to fsmap 2024-05-27T07:41:29.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:29 smithi090 ceph-mon[31429]: pgmap v391: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:29.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:29 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1762306614' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-27T07:41:29.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:29 smithi136 ceph-mon[35324]: pgmap v391: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:29.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:29 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1762306614' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-27T07:41:29.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:29 smithi129 ceph-mon[35254]: pgmap v391: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:29.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:29 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1762306614' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-27T07:41:30.299 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:41:30.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:30 smithi090 ceph-mon[31429]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T07:41:30.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:30 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1762306614' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-27T07:41:30.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:30 smithi090 ceph-mon[31429]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:41:30.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:30 smithi136 ceph-mon[35324]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T07:41:30.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:30 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1762306614' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-27T07:41:30.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:30 smithi136 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:41:30.753 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:30.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:30 smithi129 ceph-mon[35254]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T07:41:30.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:30 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1762306614' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-27T07:41:30.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:30 smithi129 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:41:31.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:31 smithi090 ceph-mon[31429]: pgmap v393: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:31.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:31 smithi090 ceph-mon[31429]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T07:41:31.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:31 smithi090 ceph-mon[31429]: Cluster is now healthy 2024-05-27T07:41:31.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:31 smithi136 ceph-mon[35324]: pgmap v393: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:31.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:31 smithi136 ceph-mon[35324]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T07:41:31.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:31 smithi136 ceph-mon[35324]: Cluster is now healthy 2024-05-27T07:41:31.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:31 smithi129 ceph-mon[35254]: pgmap v393: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:31.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:31 smithi129 ceph-mon[35254]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-27T07:41:31.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:31 smithi129 ceph-mon[35254]: Cluster is now healthy 2024-05-27T07:41:32.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.898+0000 7fac30c00700 1 -- 172.21.15.90:0/4056614892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac2c110960 msgr2=0x7fac2c110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.898+0000 7fac30c00700 1 --2- 172.21.15.90:0/4056614892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac2c110960 0x7fac2c110d70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009990 tx=0x7fac1c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.898+0000 7fac30c00700 1 -- 172.21.15.90:0/4056614892 shutdown_connections 2024-05-27T07:41:32.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.898+0000 7fac30c00700 1 --2- 172.21.15.90:0/4056614892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c10b2d0 0x7fac2c117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.898+0000 7fac30c00700 1 --2- 172.21.15.90:0/4056614892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac2c110960 0x7fac2c110d70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.898+0000 7fac30c00700 1 --2- 172.21.15.90:0/4056614892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2c10a980 0x7fac2c10ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.898+0000 7fac30c00700 1 -- 172.21.15.90:0/4056614892 >> 172.21.15.90:0/4056614892 conn(0x7fac2c077c70 msgr2=0x7fac2c0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:32.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.898+0000 7fac30c00700 1 -- 172.21.15.90:0/4056614892 shutdown_connections 2024-05-27T07:41:32.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.898+0000 7fac30c00700 1 -- 172.21.15.90:0/4056614892 wait complete. 2024-05-27T07:41:32.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.899+0000 7fac30c00700 1 Processor -- start 2024-05-27T07:41:32.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.899+0000 7fac30c00700 1 -- start start 2024-05-27T07:41:32.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.900+0000 7fac30c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac2c10a980 0x7fac2c1a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.900+0000 7fac30c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c10b2d0 0x7fac2c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.900+0000 7fac30c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2c110960 0x7fac2c1a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.900+0000 7fac30c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c11a910 con 0x7fac2c10a980 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.900+0000 7fac30c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c11a790 con 0x7fac2c10b2d0 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.900+0000 7fac30c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c11aa90 con 0x7fac2c110960 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.900+0000 7fac29e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac2c10a980 0x7fac2c1a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.900+0000 7fac29e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac2c10a980 0x7fac2c1a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46974/0 (socket says 172.21.15.90:46974) 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.900+0000 7fac29e00700 1 -- 172.21.15.90:0/432916955 learned_addr learned my addr 172.21.15.90:0/432916955 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac21400700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c10b2d0 0x7fac2c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac2a800700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2c110960 0x7fac2c1a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac29e00700 1 -- 172.21.15.90:0/432916955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2c110960 msgr2=0x7fac2c1a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac29e00700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2c110960 0x7fac2c1a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac29e00700 1 -- 172.21.15.90:0/432916955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c10b2d0 msgr2=0x7fac2c1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:32.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac29e00700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c10b2d0 0x7fac2c1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:32.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac29e00700 1 -- 172.21.15.90:0/432916955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c1a6250 con 0x7fac2c10a980 2024-05-27T07:41:32.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac21400700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c10b2d0 0x7fac2c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:41:32.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac2a800700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2c110960 0x7fac2c1a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:41:32.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac29e00700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac2c10a980 0x7fac2c1a1380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000c00 tx=0x7fac1c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:32.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac23400700 1 -- 172.21.15.90:0/432916955 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c027070 con 0x7fac2c10a980 2024-05-27T07:41:32.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac23400700 1 -- 172.21.15.90:0/432916955 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac1c00fdb0 con 0x7fac2c10a980 2024-05-27T07:41:32.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac23400700 1 -- 172.21.15.90:0/432916955 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0173f0 con 0x7fac2c10a980 2024-05-27T07:41:32.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.901+0000 7fac30c00700 1 -- 172.21.15.90:0/432916955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c1a64e0 con 0x7fac2c10a980 2024-05-27T07:41:32.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.902+0000 7fac30c00700 1 -- 172.21.15.90:0/432916955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c1ade60 con 0x7fac2c10a980 2024-05-27T07:41:32.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.903+0000 7fac30c00700 1 -- 172.21.15.90:0/432916955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2c10c070 con 0x7fac2c10a980 2024-05-27T07:41:32.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.906+0000 7fac23400700 1 -- 172.21.15.90:0/432916955 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fac1c02a070 con 0x7fac2c10a980 2024-05-27T07:41:32.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.907+0000 7fac23400700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac100b2900 0x7fac100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:32.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.908+0000 7fac23400700 1 -- 172.21.15.90:0/432916955 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fac1c0e39f0 con 0x7fac2c10a980 2024-05-27T07:41:32.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.908+0000 7fac21400700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac100b2900 0x7fac100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:32.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.909+0000 7fac21400700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac100b2900 0x7fac100b4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac14005fd0 tx=0x7fac14005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:32.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:32.912+0000 7fac23400700 1 -- 172.21.15.90:0/432916955 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac1c031080 con 0x7fac2c10a980 2024-05-27T07:41:33.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.193+0000 7fac30c00700 1 -- 172.21.15.90:0/432916955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac2c0512e0 con 0x7fac2c10a980 2024-05-27T07:41:33.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.194+0000 7fac23400700 1 -- 172.21.15.90:0/432916955 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11806 (secure 0 0 0) 0x7fac2c0512e0 con 0x7fac2c10a980 2024-05-27T07:41:33.192 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:41:33.192 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14556,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34442,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:41:28.890122+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34472,"qdb_cluster":[34472]},"id":1}]} 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 -- 172.21.15.90:0/432916955 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac100b2900 msgr2=0x7fac100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac100b2900 0x7fac100b4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fac14005fd0 tx=0x7fac14005dc0 comp rx=0 tx=0).stop 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 -- 172.21.15.90:0/432916955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac2c10a980 msgr2=0x7fac2c1a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac2c10a980 0x7fac2c1a1380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000c00 tx=0x7fac1c0047b0 comp rx=0 tx=0).stop 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 -- 172.21.15.90:0/432916955 shutdown_connections 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac100b2900 0x7fac100b4e10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac2c110960 0x7fac2c1a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac2c10a980 0x7fac2c1a1380 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 --2- 172.21.15.90:0/432916955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac2c10b2d0 0x7fac2c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 -- 172.21.15.90:0/432916955 >> 172.21.15.90:0/432916955 conn(0x7fac2c077c70 msgr2=0x7fac2c07bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:33.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 -- 172.21.15.90:0/432916955 shutdown_connections 2024-05-27T07:41:33.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:33.198+0000 7fac30c00700 1 -- 172.21.15.90:0/432916955 wait complete. 2024-05-27T07:41:33.197 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-05-27T07:41:33.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:33 smithi090 ceph-mon[31429]: pgmap v394: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:33.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:33 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/432916955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:41:33.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:33 smithi136 ceph-mon[35324]: pgmap v394: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:33.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:33 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/432916955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:41:33.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:33 smithi129 ceph-mon[35254]: pgmap v394: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:33.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:33 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/432916955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:41:34.070 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:41:34.516 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:35.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:35 smithi090 ceph-mon[31429]: pgmap v395: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:35.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:35 smithi136 ceph-mon[35324]: pgmap v395: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:35.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:35 smithi129 ceph-mon[35254]: pgmap v395: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.875+0000 7fdbd4600700 1 -- 172.21.15.90:0/2645485643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbcc110960 msgr2=0x7fdbcc110d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.875+0000 7fdbd4600700 1 --2- 172.21.15.90:0/2645485643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbcc110960 0x7fdbcc110d70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009990 tx=0x7fdbc0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:41:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.875+0000 7fdbd4600700 1 -- 172.21.15.90:0/2645485643 shutdown_connections 2024-05-27T07:41:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.875+0000 7fdbd4600700 1 --2- 172.21.15.90:0/2645485643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbcc10b2d0 0x7fdbcc117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.875+0000 7fdbd4600700 1 --2- 172.21.15.90:0/2645485643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbcc110960 0x7fdbcc110d70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.875+0000 7fdbd4600700 1 --2- 172.21.15.90:0/2645485643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbcc10a980 0x7fdbcc10ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.875+0000 7fdbd4600700 1 -- 172.21.15.90:0/2645485643 >> 172.21.15.90:0/2645485643 conn(0x7fdbcc077c70 msgr2=0x7fdbcc0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.875+0000 7fdbd4600700 1 -- 172.21.15.90:0/2645485643 shutdown_connections 2024-05-27T07:41:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.875+0000 7fdbd4600700 1 -- 172.21.15.90:0/2645485643 wait complete. 2024-05-27T07:41:36.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.877+0000 7fdbd4600700 1 Processor -- start 2024-05-27T07:41:36.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.877+0000 7fdbd4600700 1 -- start start 2024-05-27T07:41:36.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.877+0000 7fdbd4600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbcc10a980 0x7fdbcc1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:36.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.877+0000 7fdbd4600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbcc10b2d0 0x7fdbcc1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:36.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.877+0000 7fdbd4600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbcc110960 0x7fdbcc1a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:36.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.877+0000 7fdbd4600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc11a960 con 0x7fdbcc10b2d0 2024-05-27T07:41:36.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.877+0000 7fdbd4600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc11a7e0 con 0x7fdbcc110960 2024-05-27T07:41:36.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.877+0000 7fdbd4600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc11aae0 con 0x7fdbcc10a980 2024-05-27T07:41:36.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.878+0000 7fdbd1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbcc10b2d0 0x7fdbcc1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:36.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.878+0000 7fdbd1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbcc10b2d0 0x7fdbcc1a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46998/0 (socket says 172.21.15.90:46998) 2024-05-27T07:41:36.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.878+0000 7fdbd1200700 1 -- 172.21.15.90:0/2951788620 learned_addr learned my addr 172.21.15.90:0/2951788620 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:36.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.878+0000 7fdbd1c00700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbcc10a980 0x7fdbcc1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:36.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.878+0000 7fdbd2600700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbcc110960 0x7fdbcc1a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:36.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.878+0000 7fdbd1200700 1 -- 172.21.15.90:0/2951788620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbcc10a980 msgr2=0x7fdbcc1a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:36.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.878+0000 7fdbd1200700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbcc10a980 0x7fdbcc1a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:36.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.878+0000 7fdbd1200700 1 -- 172.21.15.90:0/2951788620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbcc110960 msgr2=0x7fdbcc1a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:36.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.878+0000 7fdbd1200700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbcc110960 0x7fdbcc1a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:36.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.878+0000 7fdbd1200700 1 -- 172.21.15.90:0/2951788620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc1a6230 con 0x7fdbcc10b2d0 2024-05-27T07:41:36.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.879+0000 7fdbd1200700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbcc10b2d0 0x7fdbcc1a19b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c9e0 tx=0x7fdbb800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:36.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.879+0000 7fdbca000700 1 -- 172.21.15.90:0/2951788620 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8007aa0 con 0x7fdbcc10b2d0 2024-05-27T07:41:36.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.879+0000 7fdbca000700 1 -- 172.21.15.90:0/2951788620 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbb8007c40 con 0x7fdbcc10b2d0 2024-05-27T07:41:36.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.879+0000 7fdbd4600700 1 -- 172.21.15.90:0/2951788620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbcc1a6520 con 0x7fdbcc10b2d0 2024-05-27T07:41:36.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.879+0000 7fdbca000700 1 -- 172.21.15.90:0/2951788620 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8018660 con 0x7fdbcc10b2d0 2024-05-27T07:41:36.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.879+0000 7fdbd4600700 1 -- 172.21.15.90:0/2951788620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbcc1a6a60 con 0x7fdbcc10b2d0 2024-05-27T07:41:36.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.880+0000 7fdbd4600700 1 -- 172.21.15.90:0/2951788620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbcc0512b0 con 0x7fdbcc10b2d0 2024-05-27T07:41:36.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.883+0000 7fdbca000700 1 -- 172.21.15.90:0/2951788620 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdbb8018800 con 0x7fdbcc10b2d0 2024-05-27T07:41:36.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.884+0000 7fdbca000700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbb40b2880 0x7fdbb40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:36.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.885+0000 7fdbca000700 1 -- 172.21.15.90:0/2951788620 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fdbb80e2670 con 0x7fdbcc10b2d0 2024-05-27T07:41:36.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.885+0000 7fdbd1c00700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbb40b2880 0x7fdbb40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:36.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.888+0000 7fdbd1c00700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbb40b2880 0x7fdbb40b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000b5c0 tx=0x7fdbc0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:36.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:36.888+0000 7fdbca000700 1 -- 172.21.15.90:0/2951788620 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdbb8054df0 con 0x7fdbcc10b2d0 2024-05-27T07:41:37.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.127+0000 7fdbd4600700 1 -- 172.21.15.90:0/2951788620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbcc1a2770 con 0x7fdbcc10b2d0 2024-05-27T07:41:37.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.130+0000 7fdbca000700 1 -- 172.21.15.90:0/2951788620 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24104 (secure 0 0 0) 0x7fdbb8014030 con 0x7fdbcc10b2d0 2024-05-27T07:41:37.129 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:41:37.129 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":87,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:41:28.889771+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:41:37.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.133+0000 7fdbd4600700 1 -- 172.21.15.90:0/2951788620 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbb40b2880 msgr2=0x7fdbb40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.133+0000 7fdbd4600700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbb40b2880 0x7fdbb40b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000b5c0 tx=0x7fdbc0005fb0 comp rx=0 tx=0).stop 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.133+0000 7fdbd4600700 1 -- 172.21.15.90:0/2951788620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbcc10b2d0 msgr2=0x7fdbcc1a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.133+0000 7fdbd4600700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbcc10b2d0 0x7fdbcc1a19b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c9e0 tx=0x7fdbb800cda0 comp rx=0 tx=0).stop 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.133+0000 7fdbd4600700 1 -- 172.21.15.90:0/2951788620 shutdown_connections 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.133+0000 7fdbd4600700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdbb40b2880 0x7fdbb40b4d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.133+0000 7fdbd4600700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbcc110960 0x7fdbcc1a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.133+0000 7fdbd4600700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbcc10a980 0x7fdbcc1a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.133+0000 7fdbd4600700 1 --2- 172.21.15.90:0/2951788620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbcc10b2d0 0x7fdbcc1a19b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.133+0000 7fdbd4600700 1 -- 172.21.15.90:0/2951788620 >> 172.21.15.90:0/2951788620 conn(0x7fdbcc077c70 msgr2=0x7fdbcc07bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.134+0000 7fdbd4600700 1 -- 172.21.15.90:0/2951788620 shutdown_connections 2024-05-27T07:41:37.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:37.134+0000 7fdbd4600700 1 -- 172.21.15.90:0/2951788620 wait complete. 2024-05-27T07:41:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:37 smithi090 ceph-mon[31429]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:37 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2951788620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:41:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:37 smithi136 ceph-mon[35324]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:37.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:37 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2951788620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:41:37.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:37 smithi129 ceph-mon[35254]: pgmap v396: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:37 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2951788620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:41:38.013 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:41:38.448 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:39.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:39 smithi090 ceph-mon[31429]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:39.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:39 smithi136 ceph-mon[35324]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:39.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:39 smithi129 ceph-mon[35254]: pgmap v397: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:40.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.744+0000 7f7318000700 1 -- 172.21.15.90:0/154793282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7310106eb0 msgr2=0x7f7310113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.744+0000 7f7318000700 1 --2- 172.21.15.90:0/154793282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7310106eb0 0x7f7310113340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f730800b0a0 tx=0x7f730800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.745+0000 7f7318000700 1 -- 172.21.15.90:0/154793282 shutdown_connections 2024-05-27T07:41:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.745+0000 7f7318000700 1 --2- 172.21.15.90:0/154793282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7310106eb0 0x7f7310113340 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.745+0000 7f7318000700 1 --2- 172.21.15.90:0/154793282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f731010c5a0 0x7f731010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.745+0000 7f7318000700 1 --2- 172.21.15.90:0/154793282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7310106560 0x7f7310106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.745+0000 7f7318000700 1 -- 172.21.15.90:0/154793282 >> 172.21.15.90:0/154793282 conn(0x7f7310102040 msgr2=0x7f7310104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.745+0000 7f7318000700 1 -- 172.21.15.90:0/154793282 shutdown_connections 2024-05-27T07:41:40.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.745+0000 7f7318000700 1 -- 172.21.15.90:0/154793282 wait complete. 2024-05-27T07:41:40.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.746+0000 7f7318000700 1 Processor -- start 2024-05-27T07:41:40.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.746+0000 7f7318000700 1 -- start start 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7318000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7310106560 0x7f731019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7318000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7310106eb0 0x7f731019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7318000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f731010c5a0 0x7f73101a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7318000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7310116580 con 0x7f7310106560 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7318000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7310116400 con 0x7f7310106eb0 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7318000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7310116700 con 0x7f731010c5a0 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7315600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7310106560 0x7f731019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7315600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7310106560 0x7f731019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56068/0 (socket says 172.21.15.90:56068) 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7315600700 1 -- 172.21.15.90:0/350433718 learned_addr learned my addr 172.21.15.90:0/350433718 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7314c00700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7310106eb0 0x7f731019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:40.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.747+0000 7f7316000700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f731010c5a0 0x7f73101a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:40.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f7315600700 1 -- 172.21.15.90:0/350433718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f731010c5a0 msgr2=0x7f73101a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:40.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f7315600700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f731010c5a0 0x7f73101a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:40.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f7315600700 1 -- 172.21.15.90:0/350433718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7310106eb0 msgr2=0x7f731019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:40.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f7315600700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7310106eb0 0x7f731019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:40.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f7315600700 1 -- 172.21.15.90:0/350433718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73101a1da0 con 0x7f7310106560 2024-05-27T07:41:40.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f7315600700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7310106560 0x7f731019ced0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f730400d980 tx=0x7f730400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:40.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f730e000700 1 -- 172.21.15.90:0/350433718 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304017070 con 0x7f7310106560 2024-05-27T07:41:40.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f730e000700 1 -- 172.21.15.90:0/350433718 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7304010460 con 0x7f7310106560 2024-05-27T07:41:40.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f730e000700 1 -- 172.21.15.90:0/350433718 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730400de60 con 0x7f7310106560 2024-05-27T07:41:40.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f7318000700 1 -- 172.21.15.90:0/350433718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73101a2090 con 0x7f7310106560 2024-05-27T07:41:40.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.748+0000 7f7318000700 1 -- 172.21.15.90:0/350433718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73101a99b0 con 0x7f7310106560 2024-05-27T07:41:40.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.749+0000 7f7318000700 1 -- 172.21.15.90:0/350433718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73100512b0 con 0x7f7310106560 2024-05-27T07:41:40.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.752+0000 7f730e000700 1 -- 172.21.15.90:0/350433718 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7304010610 con 0x7f7310106560 2024-05-27T07:41:40.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.754+0000 7f730e000700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f72f80b2900 0x7f72f80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:40.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.754+0000 7f730e000700 1 -- 172.21.15.90:0/350433718 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f73040e1f40 con 0x7f7310106560 2024-05-27T07:41:40.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.754+0000 7f7314c00700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f72f80b2900 0x7f72f80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:40.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.755+0000 7f7314c00700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f72f80b2900 0x7f72f80b4e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f72fc005950 tx=0x7f72fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:40.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.756+0000 7f730e000700 1 -- 172.21.15.90:0/350433718 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73040a9a40 con 0x7f7310106560 2024-05-27T07:41:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:40.997+0000 7f7318000700 1 -- 172.21.15.90:0/350433718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f731019e230 con 0x7f7310106560 2024-05-27T07:41:41.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.001+0000 7f730e000700 1 -- 172.21.15.90:0/350433718 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24104 (secure 0 0 0) 0x7f73040a9340 con 0x7f7310106560 2024-05-27T07:41:41.002 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:41:41.003 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":87,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:41:28.889771+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:41:41.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.004+0000 7f7318000700 1 -- 172.21.15.90:0/350433718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f72f80b2900 msgr2=0x7f72f80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:41.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.004+0000 7f7318000700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f72f80b2900 0x7f72f80b4e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f72fc005950 tx=0x7f72fc0058e0 comp rx=0 tx=0).stop 2024-05-27T07:41:41.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.004+0000 7f7318000700 1 -- 172.21.15.90:0/350433718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7310106560 msgr2=0x7f731019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:41.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.005+0000 7f7318000700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7310106560 0x7f731019ced0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f730400d980 tx=0x7f730400dd40 comp rx=0 tx=0).stop 2024-05-27T07:41:41.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.005+0000 7f7318000700 1 -- 172.21.15.90:0/350433718 shutdown_connections 2024-05-27T07:41:41.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.005+0000 7f7318000700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f72f80b2900 0x7f72f80b4e10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:41.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.005+0000 7f7318000700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f731010c5a0 0x7f73101a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:41.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.005+0000 7f7318000700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7310106560 0x7f731019ced0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:41.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.005+0000 7f7318000700 1 --2- 172.21.15.90:0/350433718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7310106eb0 0x7f731019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:41.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.005+0000 7f7318000700 1 -- 172.21.15.90:0/350433718 >> 172.21.15.90:0/350433718 conn(0x7f7310102040 msgr2=0x7f73101037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:41.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.005+0000 7f7318000700 1 -- 172.21.15.90:0/350433718 shutdown_connections 2024-05-27T07:41:41.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:41.005+0000 7f7318000700 1 -- 172.21.15.90:0/350433718 wait complete. 2024-05-27T07:41:41.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:41 smithi090 ceph-mon[31429]: pgmap v398: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:41.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:41 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/350433718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:41:41.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:41 smithi136 ceph-mon[35324]: pgmap v398: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:41.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:41 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/350433718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:41:41.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:41 smithi129 ceph-mon[35254]: pgmap v398: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:41.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:41 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/350433718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:41:41.969 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:41:42.408 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:43.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:43 smithi090 ceph-mon[31429]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:43.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:43 smithi136 ceph-mon[35324]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:43.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:43 smithi129 ceph-mon[35254]: pgmap v399: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:44.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.587+0000 7f1b13000700 1 -- 172.21.15.90:0/2672792310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b0c108750 msgr2=0x7f1b0c1100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:44.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.587+0000 7f1b13000700 1 --2- 172.21.15.90:0/2672792310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b0c108750 0x7f1b0c1100b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400b0d0 tx=0x7f1b0400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.587+0000 7f1b13000700 1 -- 172.21.15.90:0/2672792310 shutdown_connections 2024-05-27T07:41:44.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.587+0000 7f1b13000700 1 --2- 172.21.15.90:0/2672792310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b0c108750 0x7f1b0c1100b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.587+0000 7f1b13000700 1 --2- 172.21.15.90:0/2672792310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b0c1034d0 0x7f1b0c1038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.587+0000 7f1b13000700 1 --2- 172.21.15.90:0/2672792310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b0c107e00 0x7f1b0c108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.587+0000 7f1b13000700 1 -- 172.21.15.90:0/2672792310 >> 172.21.15.90:0/2672792310 conn(0x7f1b0c0fef20 msgr2=0x7f1b0c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:44.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.587+0000 7f1b13000700 1 -- 172.21.15.90:0/2672792310 shutdown_connections 2024-05-27T07:41:44.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.587+0000 7f1b13000700 1 -- 172.21.15.90:0/2672792310 wait complete. 2024-05-27T07:41:44.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.589+0000 7f1b13000700 1 Processor -- start 2024-05-27T07:41:44.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.589+0000 7f1b13000700 1 -- start start 2024-05-27T07:41:44.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.589+0000 7f1b13000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b0c1034d0 0x7f1b0c1a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:44.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b13000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b0c107e00 0x7f1b0c1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:44.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b13000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b0c108750 0x7f1b0c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:44.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b13000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c113230 con 0x7f1b0c107e00 2024-05-27T07:41:44.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b13000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c1130b0 con 0x7f1b0c1034d0 2024-05-27T07:41:44.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b13000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c1133b0 con 0x7f1b0c108750 2024-05-27T07:41:44.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b03e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b0c107e00 0x7f1b0c1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:44.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b03e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b0c107e00 0x7f1b0c1a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56092/0 (socket says 172.21.15.90:56092) 2024-05-27T07:41:44.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b03e00700 1 -- 172.21.15.90:0/394453302 learned_addr learned my addr 172.21.15.90:0/394453302 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:44.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b0be00700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b0c1034d0 0x7f1b0c1a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:44.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b11000700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b0c108750 0x7f1b0c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:44.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b03e00700 1 -- 172.21.15.90:0/394453302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b0c108750 msgr2=0x7f1b0c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:44.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b03e00700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b0c108750 0x7f1b0c19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b03e00700 1 -- 172.21.15.90:0/394453302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b0c1034d0 msgr2=0x7f1b0c1a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:44.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b03e00700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b0c1034d0 0x7f1b0c1a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.590+0000 7f1b03e00700 1 -- 172.21.15.90:0/394453302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c19b1e0 con 0x7f1b0c107e00 2024-05-27T07:41:44.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.591+0000 7f1b03e00700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b0c107e00 0x7f1b0c1a0e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1af400d980 tx=0x7f1af400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:44.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.591+0000 7f1b09600700 1 -- 172.21.15.90:0/394453302 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4017070 con 0x7f1b0c107e00 2024-05-27T07:41:44.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.591+0000 7f1b09600700 1 -- 172.21.15.90:0/394453302 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1af4010460 con 0x7f1b0c107e00 2024-05-27T07:41:44.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.591+0000 7f1b09600700 1 -- 172.21.15.90:0/394453302 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af400de60 con 0x7f1b0c107e00 2024-05-27T07:41:44.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.591+0000 7f1b13000700 1 -- 172.21.15.90:0/394453302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0c19b4d0 con 0x7f1b0c107e00 2024-05-27T07:41:44.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.591+0000 7f1b13000700 1 -- 172.21.15.90:0/394453302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0c1a7770 con 0x7f1b0c107e00 2024-05-27T07:41:44.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.592+0000 7f1b13000700 1 -- 172.21.15.90:0/394453302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0c0512b0 con 0x7f1b0c107e00 2024-05-27T07:41:44.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.597+0000 7f1b09600700 1 -- 172.21.15.90:0/394453302 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1af4010610 con 0x7f1b0c107e00 2024-05-27T07:41:44.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.598+0000 7f1b09600700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1af00b2900 0x7f1af00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:44.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.599+0000 7f1b09600700 1 -- 172.21.15.90:0/394453302 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f1af40e1e60 con 0x7f1b0c107e00 2024-05-27T07:41:44.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.599+0000 7f1b0be00700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1af00b2900 0x7f1af00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:44.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.599+0000 7f1b0be00700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1af00b2900 0x7f1af00b4e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1afc005950 tx=0x7f1afc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:44.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.602+0000 7f1b09600700 1 -- 172.21.15.90:0/394453302 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1af40a99c0 con 0x7f1b0c107e00 2024-05-27T07:41:44.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.888+0000 7f1b13000700 1 -- 172.21.15.90:0/394453302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b0c02e5e0 con 0x7f1b0c107e00 2024-05-27T07:41:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.889+0000 7f1b09600700 1 -- 172.21.15.90:0/394453302 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11806 (secure 0 0 0) 0x7f1af40a92c0 con 0x7f1b0c107e00 2024-05-27T07:41:44.888 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:41:44.888 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14556,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34406,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34442,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:41:28.890122+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34472},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34472,"qdb_cluster":[34472]},"id":1}]} 2024-05-27T07:41:44.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 -- 172.21.15.90:0/394453302 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1af00b2900 msgr2=0x7f1af00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:44.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1af00b2900 0x7f1af00b4e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1afc005950 tx=0x7f1afc0058e0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 -- 172.21.15.90:0/394453302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b0c107e00 msgr2=0x7f1b0c1a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b0c107e00 0x7f1b0c1a0e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1af400d980 tx=0x7f1af400dd40 comp rx=0 tx=0).stop 2024-05-27T07:41:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 -- 172.21.15.90:0/394453302 shutdown_connections 2024-05-27T07:41:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1af00b2900 0x7f1af00b4e10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b0c108750 0x7f1b0c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b0c1034d0 0x7f1b0c1a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 --2- 172.21.15.90:0/394453302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b0c107e00 0x7f1b0c1a0e20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:44.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 -- 172.21.15.90:0/394453302 >> 172.21.15.90:0/394453302 conn(0x7f1b0c0fef20 msgr2=0x7f1b0c06c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:44.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 -- 172.21.15.90:0/394453302 shutdown_connections 2024-05-27T07:41:44.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:44.893+0000 7f1b13000700 1 -- 172.21.15.90:0/394453302 wait complete. 2024-05-27T07:41:44.892 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-05-27T07:41:45.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:45 smithi090 ceph-mon[31429]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:45.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:45 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/394453302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:41:45.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:45 smithi136 ceph-mon[35324]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:45.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:45 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/394453302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:41:45.766 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:41:45.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:45 smithi129 ceph-mon[35254]: pgmap v400: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:45.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:45 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/394453302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:41:46.220 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:47.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:47 smithi090 ceph-mon[31429]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:47.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:47 smithi136 ceph-mon[35324]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:47.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:47 smithi129 ceph-mon[35254]: pgmap v401: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.639+0000 7f2030a00700 1 -- 172.21.15.90:0/2159619933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c10e6f0 msgr2=0x7f202c110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.639+0000 7f2030a00700 1 --2- 172.21.15.90:0/2159619933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c10e6f0 0x7f202c110bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f202400b0a0 tx=0x7f202400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.640+0000 7f2030a00700 1 -- 172.21.15.90:0/2159619933 shutdown_connections 2024-05-27T07:41:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.640+0000 7f2030a00700 1 --2- 172.21.15.90:0/2159619933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c10e6f0 0x7f202c110bd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.640+0000 7f2030a00700 1 --2- 172.21.15.90:0/2159619933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202c105280 0x7f202c105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.640+0000 7f2030a00700 1 --2- 172.21.15.90:0/2159619933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f202c105bd0 0x7f202c10e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.640+0000 7f2030a00700 1 -- 172.21.15.90:0/2159619933 >> 172.21.15.90:0/2159619933 conn(0x7f202c078ab0 msgr2=0x7f202c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.640+0000 7f2030a00700 1 -- 172.21.15.90:0/2159619933 shutdown_connections 2024-05-27T07:41:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.640+0000 7f2030a00700 1 -- 172.21.15.90:0/2159619933 wait complete. 2024-05-27T07:41:48.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.641+0000 7f2030a00700 1 Processor -- start 2024-05-27T07:41:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.641+0000 7f2030a00700 1 -- start start 2024-05-27T07:41:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.641+0000 7f2030a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f202c105280 0x7f202c102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.641+0000 7f2030a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c105bd0 0x7f202c1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.641+0000 7f2030a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202c10e6f0 0x7f202c1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.641+0000 7f2030a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f202c114100 con 0x7f202c105bd0 2024-05-27T07:41:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.641+0000 7f2030a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f202c113f80 con 0x7f202c105280 2024-05-27T07:41:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.641+0000 7f2030a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f202c114280 con 0x7f202c10e6f0 2024-05-27T07:41:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.642+0000 7f202aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c105bd0 0x7f202c1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.642+0000 7f202aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c105bd0 0x7f202c1034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56114/0 (socket says 172.21.15.90:56114) 2024-05-27T07:41:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.642+0000 7f202aa00700 1 -- 172.21.15.90:0/8684033 learned_addr learned my addr 172.21.15.90:0/8684033 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:48.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.642+0000 7f202b400700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f202c105280 0x7f202c102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:48.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.642+0000 7f202be00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202c10e6f0 0x7f202c1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.642+0000 7f202aa00700 1 -- 172.21.15.90:0/8684033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202c10e6f0 msgr2=0x7f202c1039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.642+0000 7f202aa00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202c10e6f0 0x7f202c1039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.642+0000 7f202aa00700 1 -- 172.21.15.90:0/8684033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f202c105280 msgr2=0x7f202c102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.642+0000 7f202aa00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f202c105280 0x7f202c102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.642+0000 7f202aa00700 1 -- 172.21.15.90:0/8684033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c1a8e70 con 0x7f202c105bd0 2024-05-27T07:41:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.643+0000 7f202aa00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c105bd0 0x7f202c1034b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f201800d950 tx=0x7f201800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.643+0000 7f2023e00700 1 -- 172.21.15.90:0/8684033 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018016070 con 0x7f202c105bd0 2024-05-27T07:41:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.643+0000 7f2023e00700 1 -- 172.21.15.90:0/8684033 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2018010460 con 0x7f202c105bd0 2024-05-27T07:41:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.643+0000 7f2030a00700 1 -- 172.21.15.90:0/8684033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202c1a9100 con 0x7f202c105bd0 2024-05-27T07:41:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.643+0000 7f2023e00700 1 -- 172.21.15.90:0/8684033 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201800f620 con 0x7f202c105bd0 2024-05-27T07:41:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.643+0000 7f2030a00700 1 -- 172.21.15.90:0/8684033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202c1a9690 con 0x7f202c105bd0 2024-05-27T07:41:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.644+0000 7f2030a00700 1 -- 172.21.15.90:0/8684033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202c0512b0 con 0x7f202c105bd0 2024-05-27T07:41:48.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.648+0000 7f2023e00700 1 -- 172.21.15.90:0/8684033 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2018019070 con 0x7f202c105bd0 2024-05-27T07:41:48.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.650+0000 7f2023e00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20100b2900 0x7f20100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:48.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.650+0000 7f2023e00700 1 -- 172.21.15.90:0/8684033 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f20180e20c0 con 0x7f202c105bd0 2024-05-27T07:41:48.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.650+0000 7f202b400700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20100b2900 0x7f20100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:48.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.651+0000 7f202b400700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20100b2900 0x7f20100b4e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f201c005950 tx=0x7f201c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:48.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.652+0000 7f2023e00700 1 -- 172.21.15.90:0/8684033 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f20180a9b40 con 0x7f202c105bd0 2024-05-27T07:41:48.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.894+0000 7f2030a00700 1 -- 172.21.15.90:0/8684033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f202c02e5e0 con 0x7f202c105bd0 2024-05-27T07:41:48.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.897+0000 7f2023e00700 1 -- 172.21.15.90:0/8684033 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24104 (secure 0 0 0) 0x7f20180a9440 con 0x7f202c105bd0 2024-05-27T07:41:48.895 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:41:48.895 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":87,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:41:28.889771+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:41:48.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.900+0000 7f2030a00700 1 -- 172.21.15.90:0/8684033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20100b2900 msgr2=0x7f20100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:48.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.900+0000 7f2030a00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20100b2900 0x7f20100b4e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f201c005950 tx=0x7f201c0058e0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.900+0000 7f2030a00700 1 -- 172.21.15.90:0/8684033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c105bd0 msgr2=0x7f202c1034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.900+0000 7f2030a00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c105bd0 0x7f202c1034b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f201800d950 tx=0x7f201800dd10 comp rx=0 tx=0).stop 2024-05-27T07:41:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.900+0000 7f2030a00700 1 -- 172.21.15.90:0/8684033 shutdown_connections 2024-05-27T07:41:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.900+0000 7f2030a00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f20100b2900 0x7f20100b4e10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.900+0000 7f2030a00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202c10e6f0 0x7f202c1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.900+0000 7f2030a00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f202c105280 0x7f202c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.900+0000 7f2030a00700 1 --2- 172.21.15.90:0/8684033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f202c105bd0 0x7f202c1034b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.900+0000 7f2030a00700 1 -- 172.21.15.90:0/8684033 >> 172.21.15.90:0/8684033 conn(0x7f202c078ab0 msgr2=0x7f202c0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.901+0000 7f2030a00700 1 -- 172.21.15.90:0/8684033 shutdown_connections 2024-05-27T07:41:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:48.901+0000 7f2030a00700 1 -- 172.21.15.90:0/8684033 wait complete. 2024-05-27T07:41:49.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-27T07:41:11.710115+0000', 'modified': '2024-05-27T07:41:28.890122+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34472}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34472': {'gid': 34472, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.129:6839/2893900972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 2893900972}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 2893900972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34472, 'qdb_cluster': [34472]} 2024-05-27T07:41:49.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-27T07:41:49.704 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T07:41:49.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:49 smithi136 ceph-mon[35324]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:49.777 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:49 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/8684033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:41:49.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:49 smithi090 ceph-mon[31429]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:49.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:49 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/8684033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:41:49.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:49 smithi129 ceph-mon[35254]: pgmap v402: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:49.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:49 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/8684033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:41:50.160 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:51.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:51 smithi136 ceph-mon[35324]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:51.809 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:51 smithi090 ceph-mon[31429]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:51.811 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:51 smithi129 ceph-mon[35254]: pgmap v403: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:52.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.407+0000 7fe489600700 1 -- 172.21.15.90:0/2828489264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe484075d20 msgr2=0x7fe484076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:52.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.407+0000 7fe489600700 1 --2- 172.21.15.90:0/2828489264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe484075d20 0x7fe484076130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe474009990 tx=0x7fe474009ca0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.407+0000 7fe489600700 1 -- 172.21.15.90:0/2828489264 shutdown_connections 2024-05-27T07:41:52.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.407+0000 7fe489600700 1 --2- 172.21.15.90:0/2828489264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe484110b90 0x7fe484112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.407+0000 7fe489600700 1 --2- 172.21.15.90:0/2828489264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe484075d20 0x7fe484076130 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.407+0000 7fe489600700 1 --2- 172.21.15.90:0/2828489264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe484076670 0x7fe484110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.407+0000 7fe489600700 1 -- 172.21.15.90:0/2828489264 >> 172.21.15.90:0/2828489264 conn(0x7fe484100030 msgr2=0x7fe484102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:52.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.407+0000 7fe489600700 1 -- 172.21.15.90:0/2828489264 shutdown_connections 2024-05-27T07:41:52.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.407+0000 7fe489600700 1 -- 172.21.15.90:0/2828489264 wait complete. 2024-05-27T07:41:52.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.408+0000 7fe489600700 1 Processor -- start 2024-05-27T07:41:52.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.408+0000 7fe489600700 1 -- start start 2024-05-27T07:41:52.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.409+0000 7fe489600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe484075d20 0x7fe48419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.409+0000 7fe489600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe484076670 0x7fe48419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.409+0000 7fe489600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe484110b90 0x7fe4841a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.409+0000 7fe489600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe484116580 con 0x7fe484075d20 2024-05-27T07:41:52.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.409+0000 7fe489600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe484116400 con 0x7fe484110b90 2024-05-27T07:41:52.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.409+0000 7fe489600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe484116700 con 0x7fe484076670 2024-05-27T07:41:52.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.409+0000 7fe482a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe484075d20 0x7fe48419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.409+0000 7fe482a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe484075d20 0x7fe48419cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49906/0 (socket says 172.21.15.90:49906) 2024-05-27T07:41:52.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.409+0000 7fe482a00700 1 -- 172.21.15.90:0/153411703 learned_addr learned my addr 172.21.15.90:0/153411703 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.410+0000 7fe482000700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe484076670 0x7fe48419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.410+0000 7fe483400700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe484110b90 0x7fe4841a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.410+0000 7fe482a00700 1 -- 172.21.15.90:0/153411703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe484076670 msgr2=0x7fe48419d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.410+0000 7fe482a00700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe484076670 0x7fe48419d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.410+0000 7fe482a00700 1 -- 172.21.15.90:0/153411703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe484110b90 msgr2=0x7fe4841a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.410+0000 7fe482a00700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe484110b90 0x7fe4841a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.410+0000 7fe482a00700 1 -- 172.21.15.90:0/153411703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4841a1dd0 con 0x7fe484075d20 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.410+0000 7fe483400700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe484110b90 0x7fe4841a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.410+0000 7fe482a00700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe484075d20 0x7fe48419cfb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe474000c00 tx=0x7fe474005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.411+0000 7fe47b400700 1 -- 172.21.15.90:0/153411703 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474027070 con 0x7fe484075d20 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.411+0000 7fe47b400700 1 -- 172.21.15.90:0/153411703 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe47400fb80 con 0x7fe484075d20 2024-05-27T07:41:52.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.411+0000 7fe489600700 1 -- 172.21.15.90:0/153411703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4841a2060 con 0x7fe484075d20 2024-05-27T07:41:52.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.411+0000 7fe47b400700 1 -- 172.21.15.90:0/153411703 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474022c30 con 0x7fe484075d20 2024-05-27T07:41:52.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.411+0000 7fe489600700 1 -- 172.21.15.90:0/153411703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4841a2540 con 0x7fe484075d20 2024-05-27T07:41:52.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.415+0000 7fe47b400700 1 -- 172.21.15.90:0/153411703 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe47402a070 con 0x7fe484075d20 2024-05-27T07:41:52.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.416+0000 7fe498106b80 1 Processor -- start 2024-05-27T07:41:52.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.417+0000 7fe498106b80 1 -- start start 2024-05-27T07:41:52.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.417+0000 7fe498106b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d82f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.417+0000 7fe498106b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d6b0df0 0x55ff2d82f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.417+0000 7fe498106b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d82fe20 0x55ff2d83c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.417+0000 7fe498106b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d75ddf0 con 0x55ff2d79c9a0 2024-05-27T07:41:52.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.417+0000 7fe498106b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d74d990 con 0x55ff2d82fe20 2024-05-27T07:41:52.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.417+0000 7fe498106b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d7519d0 con 0x55ff2d6b0df0 2024-05-27T07:41:52.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.418+0000 7fe47b400700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe4680b2880 0x7fe4680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.418+0000 7fe483400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d82f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.418+0000 7fe482000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d82fe20 0x55ff2d83c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.418+0000 7fe47b400700 1 -- 172.21.15.90:0/153411703 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fe4740e3440 con 0x7fe484075d20 2024-05-27T07:41:52.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.418+0000 7fe483400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d82f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49912/0 (socket says 172.21.15.90:49912) 2024-05-27T07:41:52.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.418+0000 7fe482000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d82fe20 0x55ff2d83c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48964/0 (socket says 172.21.15.90:48964) 2024-05-27T07:41:52.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.418+0000 7fe483400700 1 -- 172.21.15.90:0/1398130275 learned_addr learned my addr 172.21.15.90:0/1398130275 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:52.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.418+0000 7fe483400700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe4680b2880 0x7fe4680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.418+0000 7fe482a00700 1 --2- 172.21.15.90:0/1398130275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d6b0df0 0x55ff2d82f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.419+0000 7fe483400700 1 -- 172.21.15.90:0/1398130275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d6b0df0 msgr2=0x55ff2d82f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:52.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.419+0000 7fe483400700 1 --2- 172.21.15.90:0/1398130275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d6b0df0 0x55ff2d82f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.419+0000 7fe483400700 1 -- 172.21.15.90:0/1398130275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d82fe20 msgr2=0x55ff2d83c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:52.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.419+0000 7fe483400700 1 --2- 172.21.15.90:0/1398130275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d82fe20 0x55ff2d83c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.419+0000 7fe483400700 1 -- 172.21.15.90:0/1398130275 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff2d7163d0 con 0x55ff2d79c9a0 2024-05-27T07:41:52.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.419+0000 7fe483400700 1 --2- 172.21.15.90:0/1398130275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d82f2e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe47c012de0 tx=0x7fe47c0165e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee250ed44d50cd57 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:52.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.419+0000 7fe483400700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe4680b2880 0x7fe4680b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe47c016930 tx=0x7fe47c00a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:52.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.419+0000 7fe46fe00700 1 -- 172.21.15.90:0/1398130275 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c01d070 con 0x55ff2d79c9a0 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.420+0000 7fe46fe00700 1 -- 172.21.15.90:0/1398130275 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe47c018d80 con 0x55ff2d79c9a0 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.420+0000 7fe46fe00700 1 -- 172.21.15.90:0/1398130275 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c010040 con 0x55ff2d79c9a0 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.420+0000 7fe498106b80 1 -- 172.21.15.90:0/1398130275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 msgr2=0x55ff2d82f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.421+0000 7fe498106b80 1 --2- 172.21.15.90:0/1398130275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d82f2e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe47c012de0 tx=0x7fe47c0165e0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.421+0000 7fe498106b80 1 -- 172.21.15.90:0/1398130275 shutdown_connections 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.421+0000 7fe498106b80 1 --2- 172.21.15.90:0/1398130275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d82fe20 0x55ff2d83c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.421+0000 7fe498106b80 1 --2- 172.21.15.90:0/1398130275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d82f2e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.421+0000 7fe498106b80 1 --2- 172.21.15.90:0/1398130275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d6b0df0 0x55ff2d82f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.421+0000 7fe498106b80 1 -- 172.21.15.90:0/1398130275 >> 172.21.15.90:0/1398130275 conn(0x55ff2d74a340 msgr2=0x55ff2d82b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.421+0000 7fe498106b80 1 -- 172.21.15.90:0/1398130275 shutdown_connections 2024-05-27T07:41:52.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.421+0000 7fe498106b80 1 -- 172.21.15.90:0/1398130275 wait complete. 2024-05-27T07:41:52.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.422+0000 7fe498106b80 1 -- start start 2024-05-27T07:41:52.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.422+0000 7fe498106b80 1 Processor -- start 2024-05-27T07:41:52.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.422+0000 7fe498106b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d839750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.422+0000 7fe498106b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d79c9a0 0x55ff2d839c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe498106b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d83a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe498106b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d653f50 con 0x55ff2d6b0df0 2024-05-27T07:41:52.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe498106b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d6ee710 con 0x55ff2d79c9a0 2024-05-27T07:41:52.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe498106b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d83cad0 con 0x55ff2d82fe20 2024-05-27T07:41:52.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe498106b80 10 client.0 _ll_register_callbacks cb 0x55ff2d6f7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:52.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe482a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d839750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe482a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d839750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49918/0 (socket says 172.21.15.90:49918) 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe482a00700 1 -- 172.21.15.90:0/1577934020 learned_addr learned my addr 172.21.15.90:0/1577934020 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe482000700 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d79c9a0 0x55ff2d839c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe482a00700 1 -- 172.21.15.90:0/1577934020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 msgr2=0x55ff2d83a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe482a00700 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d83a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe482a00700 1 -- 172.21.15.90:0/1577934020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d79c9a0 msgr2=0x55ff2d839c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe482a00700 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d79c9a0 0x55ff2d839c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.423+0000 7fe482a00700 1 -- 172.21.15.90:0/1577934020 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff2d69d880 con 0x55ff2d6b0df0 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.424+0000 7fe482a00700 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d839750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe4740e0ca0 tx=0x7fe4740aa250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc8e2475bdf64cb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.424+0000 7fe46cc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.424+0000 7fe46cc00700 1 -- 172.21.15.90:0/1577934020 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740aaf50 con 0x55ff2d6b0df0 2024-05-27T07:41:52.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.424+0000 7fe46cc00700 1 -- 172.21.15.90:0/1577934020 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fe47400fd30 con 0x55ff2d6b0df0 2024-05-27T07:41:52.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.424+0000 7fe498106b80 1 -- 172.21.15.90:0/1577934020 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe47c018d80 con 0x55ff2d6b0df0 2024-05-27T07:41:52.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.424+0000 7fe46cc00700 1 -- 172.21.15.90:0/1577934020 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fe4740cd0e0 con 0x55ff2d6b0df0 2024-05-27T07:41:52.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.424+0000 7fe498106b80 10 client.14664 fetch_fsmap learned FSMap version 17 2024-05-27T07:41:52.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.425+0000 7fe498106b80 1 -- 172.21.15.90:0/1577934020 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55ff2d71e010 con 0x55ff2d6b0df0 2024-05-27T07:41:52.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.425+0000 7fe46cc00700 1 -- 172.21.15.90:0/1577934020 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe47c018d80 con 0x55ff2d6b0df0 2024-05-27T07:41:52.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.425+0000 7fe46cc00700 1 -- 172.21.15.90:0/1577934020 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fe4740b3190 con 0x55ff2d6b0df0 2024-05-27T07:41:52.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.425+0000 7fe498106b80 10 client.14664 fetch_fsmap finished waiting for FSMap version 17 2024-05-27T07:41:52.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.425+0000 7fe498106b80 10 client.14664 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:41:52.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.425+0000 7fe498106b80 20 client.14664 populate_metadata read hostname 'smithi090' 2024-05-27T07:41:52.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.425+0000 7fe498106b80 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ff2d849a80 0x55ff2d84aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.425+0000 7fe498106b80 4 client.14664 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:41:52.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.425+0000 7fe498106b80 1 -- 172.21.15.90:0/1577934020 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff2d75fd40 con 0x55ff2d849a80 2024-05-27T07:41:52.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.426+0000 7fe482000700 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ff2d849a80 0x55ff2d84aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.427+0000 7fe482000700 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ff2d849a80 0x55ff2d84aef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ee6f3d6dd8a44f8 server_cookie=66a6ce5487ce9377 in_seq=0 out_seq=0 2024-05-27T07:41:52.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.427+0000 7fe46cc00700 10 client.14664 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:41:52.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe46cc00700 1 -- 172.21.15.90:0/1577934020 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ff2d75fd40 con 0x55ff2d849a80 2024-05-27T07:41:52.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe46cc00700 10 client.14664 handle_command_reply: tid=0 2024-05-27T07:41:52.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe498106b80 1 client.14664 shutdown 2024-05-27T07:41:52.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe498106b80 1 -- 172.21.15.90:0/1577934020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 msgr2=0x55ff2d839750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:52.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe498106b80 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d839750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe4740e0ca0 tx=0x7fe4740aa250 comp rx=0 tx=0).stop 2024-05-27T07:41:52.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe498106b80 1 -- 172.21.15.90:0/1577934020 shutdown_connections 2024-05-27T07:41:52.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe498106b80 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ff2d849a80 0x55ff2d84aef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe498106b80 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d83a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe498106b80 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d839750 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe498106b80 1 --2- 172.21.15.90:0/1577934020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d79c9a0 0x55ff2d839c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe498106b80 1 -- 172.21.15.90:0/1577934020 >> 172.21.15.90:0/1577934020 conn(0x55ff2d74a340 msgr2=0x55ff2d82f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:52.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.430+0000 7fe46cc00700 0 client.14664 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:41:52.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.431+0000 7fe498106b80 1 -- 172.21.15.90:0/1577934020 shutdown_connections 2024-05-27T07:41:52.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.431+0000 7fe498106b80 1 -- 172.21.15.90:0/1577934020 wait complete. 2024-05-27T07:41:52.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.431+0000 7fe498106b80 20 client.14664 trim_cache size 0 max 16384 2024-05-27T07:41:52.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.454+0000 7fe498106b80 1 Processor -- start 2024-05-27T07:41:52.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.455+0000 7fe498106b80 1 -- start start 2024-05-27T07:41:52.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.455+0000 7fe498106b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d834f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.455+0000 7fe498106b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d79c9a0 0x55ff2d82b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.455+0000 7fe498106b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d845d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.455+0000 7fe498106b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d849230 con 0x55ff2d6b0df0 2024-05-27T07:41:52.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.455+0000 7fe498106b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d849900 con 0x55ff2d79c9a0 2024-05-27T07:41:52.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.455+0000 7fe498106b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d849fd0 con 0x55ff2d82fe20 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.455+0000 7fe482a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d834f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.455+0000 7fe482a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d834f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49928/0 (socket says 172.21.15.90:49928) 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.455+0000 7fe482a00700 1 -- 172.21.15.90:0/634295264 learned_addr learned my addr 172.21.15.90:0/634295264 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.456+0000 7fe483400700 1 --2- 172.21.15.90:0/634295264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d845d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.456+0000 7fe482000700 1 --2- 172.21.15.90:0/634295264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d79c9a0 0x55ff2d82b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.456+0000 7fe482a00700 1 -- 172.21.15.90:0/634295264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 msgr2=0x55ff2d845d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.456+0000 7fe482a00700 1 --2- 172.21.15.90:0/634295264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d845d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.456+0000 7fe482a00700 1 -- 172.21.15.90:0/634295264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d79c9a0 msgr2=0x55ff2d82b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.456+0000 7fe482a00700 1 --2- 172.21.15.90:0/634295264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d79c9a0 0x55ff2d82b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.456+0000 7fe482a00700 1 -- 172.21.15.90:0/634295264 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff2d8462b0 con 0x55ff2d6b0df0 2024-05-27T07:41:52.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.456+0000 7fe482a00700 1 --2- 172.21.15.90:0/634295264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d834f40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4740b1010 tx=0x7fe4740b3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef002cf02d49479 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:52.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.456+0000 7fe46f400700 1 -- 172.21.15.90:0/634295264 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740aaf50 con 0x55ff2d6b0df0 2024-05-27T07:41:52.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.457+0000 7fe46f400700 1 -- 172.21.15.90:0/634295264 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4740c4070 con 0x55ff2d6b0df0 2024-05-27T07:41:52.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.457+0000 7fe46f400700 1 -- 172.21.15.90:0/634295264 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740aec00 con 0x55ff2d6b0df0 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.458+0000 7fe498106b80 1 -- 172.21.15.90:0/634295264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 msgr2=0x55ff2d834f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.458+0000 7fe498106b80 1 --2- 172.21.15.90:0/634295264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d834f40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4740b1010 tx=0x7fe4740b3040 comp rx=0 tx=0).stop 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.458+0000 7fe498106b80 1 -- 172.21.15.90:0/634295264 shutdown_connections 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.458+0000 7fe498106b80 1 --2- 172.21.15.90:0/634295264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d845d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.458+0000 7fe498106b80 1 --2- 172.21.15.90:0/634295264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d6b0df0 0x55ff2d834f40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.458+0000 7fe498106b80 1 --2- 172.21.15.90:0/634295264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d79c9a0 0x55ff2d82b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.458+0000 7fe498106b80 1 -- 172.21.15.90:0/634295264 >> 172.21.15.90:0/634295264 conn(0x55ff2d74a340 msgr2=0x55ff2d839710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.458+0000 7fe498106b80 1 -- 172.21.15.90:0/634295264 shutdown_connections 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.458+0000 7fe498106b80 1 -- 172.21.15.90:0/634295264 wait complete. 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.459+0000 7fe498106b80 1 -- start start 2024-05-27T07:41:52.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.459+0000 7fe498106b80 1 Processor -- start 2024-05-27T07:41:52.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.460+0000 7fe498106b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d6b0df0 0x55ff2d854370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.460+0000 7fe498106b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d8548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.460+0000 7fe498106b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d84d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.460+0000 7fe498106b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d84ad70 con 0x55ff2d79c9a0 2024-05-27T07:41:52.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.460+0000 7fe498106b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d84a6a0 con 0x55ff2d6b0df0 2024-05-27T07:41:52.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.460+0000 7fe498106b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d84b440 con 0x55ff2d82fe20 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.460+0000 7fe498106b80 10 client.0 _ll_register_callbacks cb 0x55ff2d6f7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.460+0000 7fe482000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d8548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.460+0000 7fe482000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d8548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49942/0 (socket says 172.21.15.90:49942) 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.460+0000 7fe482000700 1 -- 172.21.15.90:0/1215257438 learned_addr learned my addr 172.21.15.90:0/1215257438 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.461+0000 7fe482a00700 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d6b0df0 0x55ff2d854370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.461+0000 7fe483400700 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d84d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.461+0000 7fe482000700 1 -- 172.21.15.90:0/1215257438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 msgr2=0x55ff2d84d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.461+0000 7fe482000700 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d84d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.461+0000 7fe482000700 1 -- 172.21.15.90:0/1215257438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d6b0df0 msgr2=0x55ff2d854370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.461+0000 7fe482000700 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d6b0df0 0x55ff2d854370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.461+0000 7fe482000700 1 -- 172.21.15.90:0/1215257438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff2d854ec0 con 0x55ff2d79c9a0 2024-05-27T07:41:52.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.461+0000 7fe482000700 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d8548b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55ff2d84b5f0 tx=0x7fe470012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a4423d9161451c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:52.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.461+0000 7fe46cc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:41:52.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.461+0000 7fe46cc00700 1 -- 172.21.15.90:0/1215257438 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470004230 con 0x55ff2d79c9a0 2024-05-27T07:41:52.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.462+0000 7fe46cc00700 1 -- 172.21.15.90:0/1215257438 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fe4700043d0 con 0x55ff2d79c9a0 2024-05-27T07:41:52.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.462+0000 7fe498106b80 1 -- 172.21.15.90:0/1215257438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4740c4070 con 0x55ff2d79c9a0 2024-05-27T07:41:52.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.462+0000 7fe46cc00700 1 -- 172.21.15.90:0/1215257438 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fe47002a050 con 0x55ff2d79c9a0 2024-05-27T07:41:52.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.462+0000 7fe498106b80 10 client.14670 fetch_fsmap learned FSMap version 17 2024-05-27T07:41:52.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.462+0000 7fe498106b80 1 -- 172.21.15.90:0/1215257438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55ff2d8b57e0 con 0x55ff2d79c9a0 2024-05-27T07:41:52.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.462+0000 7fe46cc00700 1 -- 172.21.15.90:0/1215257438 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4740c4070 con 0x55ff2d79c9a0 2024-05-27T07:41:52.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.462+0000 7fe46cc00700 1 -- 172.21.15.90:0/1215257438 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fe47001b590 con 0x55ff2d79c9a0 2024-05-27T07:41:52.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.463+0000 7fe498106b80 10 client.14670 fetch_fsmap finished waiting for FSMap version 17 2024-05-27T07:41:52.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.463+0000 7fe498106b80 10 client.14670 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:41:52.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.463+0000 7fe498106b80 20 client.14670 populate_metadata read hostname 'smithi090' 2024-05-27T07:41:52.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.463+0000 7fe498106b80 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ff2d8b7ce0 0x55ff2d8b9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:52.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.463+0000 7fe498106b80 4 client.14670 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:41:52.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.463+0000 7fe498106b80 1 -- 172.21.15.90:0/1215257438 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "status"}) -- 0x55ff2d759520 con 0x55ff2d8b7ce0 2024-05-27T07:41:52.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.463+0000 7fe482a00700 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ff2d8b7ce0 0x55ff2d8b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:52.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.464+0000 7fe482a00700 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ff2d8b7ce0 0x55ff2d8b9150 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1e281a193d284f6 server_cookie=c3c8c393d4205c73 in_seq=0 out_seq=0 2024-05-27T07:41:52.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.464+0000 7fe46cc00700 10 client.14670 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:41:52.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.465+0000 7fe46cc00700 1 -- 172.21.15.90:0/1215257438 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ff2d759520 con 0x55ff2d8b7ce0 2024-05-27T07:41:52.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.465+0000 7fe46cc00700 10 client.14670 handle_command_reply: tid=0 2024-05-27T07:41:52.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.465+0000 7fe498106b80 1 client.14670 shutdown 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.465+0000 7fe498106b80 1 -- 172.21.15.90:0/1215257438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 msgr2=0x55ff2d8548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.465+0000 7fe498106b80 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d8548b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55ff2d84b5f0 tx=0x7fe470012730 comp rx=0 tx=0).stop 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.465+0000 7fe498106b80 1 -- 172.21.15.90:0/1215257438 shutdown_connections 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.465+0000 7fe498106b80 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ff2d8b7ce0 0x55ff2d8b9150 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.466+0000 7fe498106b80 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2d82fe20 0x55ff2d84d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.466+0000 7fe498106b80 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ff2d6b0df0 0x55ff2d854370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.466+0000 7fe498106b80 1 --2- 172.21.15.90:0/1215257438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ff2d79c9a0 0x55ff2d8548b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.466+0000 7fe46cc00700 0 client.14670 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.466+0000 7fe498106b80 1 -- 172.21.15.90:0/1215257438 >> 172.21.15.90:0/1215257438 conn(0x55ff2d74a340 msgr2=0x55ff2d831610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.466+0000 7fe498106b80 1 -- 172.21.15.90:0/1215257438 shutdown_connections 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.466+0000 7fe498106b80 1 -- 172.21.15.90:0/1215257438 wait complete. 2024-05-27T07:41:52.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.466+0000 7fe498106b80 20 client.14670 trim_cache size 0 max 16384 2024-05-27T07:41:52.466 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:41:52.466 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:41:52.466 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-05-27T07:41:52.466 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34472, 2024-05-27T07:41:52.466 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:41:52.466 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:41:52.467 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:41:52.467 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 40.615516821999996, 2024-05-27T07:41:52.467 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 17, 2024-05-27T07:41:52.467 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 87, 2024-05-27T07:41:52.467 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-05-27T07:41:52.467 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 58.738517473999998 2024-05-27T07:41:52.467 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 -- 172.21.15.90:0/153411703 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe4680b2880 msgr2=0x7fe4680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe4680b2880 0x7fe4680b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe47c016930 tx=0x7fe47c00a000 comp rx=0 tx=0).stop 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 -- 172.21.15.90:0/153411703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe484075d20 msgr2=0x7fe48419cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe484075d20 0x7fe48419cfb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe474000c00 tx=0x7fe474005ef0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 -- 172.21.15.90:0/153411703 shutdown_connections 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe4680b2880 0x7fe4680b4d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe484110b90 0x7fe4841a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe484075d20 0x7fe48419cfb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 --2- 172.21.15.90:0/153411703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe484076670 0x7fe48419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 -- 172.21.15.90:0/153411703 >> 172.21.15.90:0/153411703 conn(0x7fe484100030 msgr2=0x7fe484100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 -- 172.21.15.90:0/153411703 shutdown_connections 2024-05-27T07:41:52.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:52.467+0000 7fe489600700 1 -- 172.21.15.90:0/153411703 wait complete. 2024-05-27T07:41:53.270 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T07:41:53.270 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:41:53.271 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-27T07:41:53.684 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:41:53.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:53 smithi090 ceph-mon[31429]: pgmap v404: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:53.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:53 smithi136 ceph-mon[35324]: pgmap v404: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:54.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:53 smithi129 ceph-mon[35254]: pgmap v404: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:55.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.783+0000 7f7a72800700 1 -- 172.21.15.90:0/3535315338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a6c108d20 msgr2=0x7f7a6c109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:55.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.783+0000 7f7a72800700 1 --2- 172.21.15.90:0/3535315338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a6c108d20 0x7f7a6c109130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7a580099c0 tx=0x7f7a58009cd0 comp rx=0 tx=0).stop 2024-05-27T07:41:55.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.783+0000 7f7a72800700 1 -- 172.21.15.90:0/3535315338 shutdown_connections 2024-05-27T07:41:55.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.783+0000 7f7a72800700 1 --2- 172.21.15.90:0/3535315338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a6c109670 0x7f7a6c110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:55.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.783+0000 7f7a72800700 1 --2- 172.21.15.90:0/3535315338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6c1043f0 0x7f7a6c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:55.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.783+0000 7f7a72800700 1 --2- 172.21.15.90:0/3535315338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a6c108d20 0x7f7a6c109130 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:55.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.783+0000 7f7a72800700 1 -- 172.21.15.90:0/3535315338 >> 172.21.15.90:0/3535315338 conn(0x7f7a6c079d00 msgr2=0x7f7a6c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:55.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.783+0000 7f7a72800700 1 -- 172.21.15.90:0/3535315338 shutdown_connections 2024-05-27T07:41:55.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.783+0000 7f7a72800700 1 -- 172.21.15.90:0/3535315338 wait complete. 2024-05-27T07:41:55.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.784+0000 7f7a72800700 1 Processor -- start 2024-05-27T07:41:55.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.784+0000 7f7a72800700 1 -- start start 2024-05-27T07:41:55.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.785+0000 7f7a72800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a6c1043f0 0x7f7a6c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:55.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.785+0000 7f7a72800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6c108d20 0x7f7a6c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:55.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.785+0000 7f7a72800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a6c109670 0x7f7a6c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:55.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.785+0000 7f7a72800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c114150 con 0x7f7a6c1043f0 2024-05-27T07:41:55.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.785+0000 7f7a72800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c113fd0 con 0x7f7a6c109670 2024-05-27T07:41:55.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.785+0000 7f7a72800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1142d0 con 0x7f7a6c108d20 2024-05-27T07:41:55.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.785+0000 7f7a71400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a6c1043f0 0x7f7a6c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:55.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.785+0000 7f7a71400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a6c1043f0 0x7f7a6c1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49952/0 (socket says 172.21.15.90:49952) 2024-05-27T07:41:55.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.785+0000 7f7a71400700 1 -- 172.21.15.90:0/3787553059 learned_addr learned my addr 172.21.15.90:0/3787553059 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:41:55.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.785+0000 7f7a70a00700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6c108d20 0x7f7a6c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:55.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a71400700 1 -- 172.21.15.90:0/3787553059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6c108d20 msgr2=0x7f7a6c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:55.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a71400700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6c108d20 0x7f7a6c1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:55.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a71400700 1 -- 172.21.15.90:0/3787553059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a6c109670 msgr2=0x7f7a6c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:55.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a71400700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a6c109670 0x7f7a6c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:55.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a71400700 1 -- 172.21.15.90:0/3787553059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c19b0c0 con 0x7f7a6c1043f0 2024-05-27T07:41:55.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a71400700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a6c1043f0 0x7f7a6c1a07f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7a60007f10 tx=0x7f7a6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:55.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a6a000700 1 -- 172.21.15.90:0/3787553059 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000dc90 con 0x7f7a6c1043f0 2024-05-27T07:41:55.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a6a000700 1 -- 172.21.15.90:0/3787553059 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a6000de30 con 0x7f7a6c1043f0 2024-05-27T07:41:55.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a72800700 1 -- 172.21.15.90:0/3787553059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c19b3b0 con 0x7f7a6c1043f0 2024-05-27T07:41:55.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a6a000700 1 -- 172.21.15.90:0/3787553059 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60005a50 con 0x7f7a6c1043f0 2024-05-27T07:41:55.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.786+0000 7f7a72800700 1 -- 172.21.15.90:0/3787553059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c1a7680 con 0x7f7a6c1043f0 2024-05-27T07:41:55.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.787+0000 7f7a72800700 1 -- 172.21.15.90:0/3787553059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6c1058a0 con 0x7f7a6c1043f0 2024-05-27T07:41:55.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.790+0000 7f7a6a000700 1 -- 172.21.15.90:0/3787553059 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7a6001a020 con 0x7f7a6c1043f0 2024-05-27T07:41:55.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:55 smithi090 ceph-mon[31429]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:55.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.792+0000 7f7a6a000700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7a540b2aa0 0x7f7a540b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:41:55.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.792+0000 7f7a6a000700 1 -- 172.21.15.90:0/3787553059 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f7a600e22a0 con 0x7f7a6c1043f0 2024-05-27T07:41:55.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.792+0000 7f7a70a00700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7a540b2aa0 0x7f7a540b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:41:55.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.793+0000 7f7a70a00700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7a540b2aa0 0x7f7a540b4fb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009990 tx=0x7f7a5801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:41:55.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:55.795+0000 7f7a6a000700 1 -- 172.21.15.90:0/3787553059 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a60010070 con 0x7f7a6c1043f0 2024-05-27T07:41:55.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:55 smithi136 ceph-mon[35324]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:56.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:55 smithi129 ceph-mon[35254]: pgmap v405: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:56.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.069+0000 7f7a72800700 1 -- 172.21.15.90:0/3787553059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7a6c02e5e0 con 0x7f7a6c1043f0 2024-05-27T07:41:56.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.070+0000 7f7a6a000700 1 -- 172.21.15.90:0/3787553059 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f7a600a9d80 con 0x7f7a6c1043f0 2024-05-27T07:41:56.068 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T07:41:56.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.073+0000 7f7a72800700 1 -- 172.21.15.90:0/3787553059 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7a540b2aa0 msgr2=0x7f7a540b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.073+0000 7f7a72800700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7a540b2aa0 0x7f7a540b4fb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009990 tx=0x7f7a5801a040 comp rx=0 tx=0).stop 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.073+0000 7f7a72800700 1 -- 172.21.15.90:0/3787553059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a6c1043f0 msgr2=0x7f7a6c1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.073+0000 7f7a72800700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a6c1043f0 0x7f7a6c1a07f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7a60007f10 tx=0x7f7a6000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.074+0000 7f7a72800700 1 -- 172.21.15.90:0/3787553059 shutdown_connections 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.074+0000 7f7a72800700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7a540b2aa0 0x7f7a540b4fb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.074+0000 7f7a72800700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a6c109670 0x7f7a6c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.074+0000 7f7a72800700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a6c1043f0 0x7f7a6c1a07f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.074+0000 7f7a72800700 1 --2- 172.21.15.90:0/3787553059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6c108d20 0x7f7a6c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.074+0000 7f7a72800700 1 -- 172.21.15.90:0/3787553059 >> 172.21.15.90:0/3787553059 conn(0x7f7a6c079d00 msgr2=0x7f7a6c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.074+0000 7f7a72800700 1 -- 172.21.15.90:0/3787553059 shutdown_connections 2024-05-27T07:41:56.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:56.074+0000 7f7a72800700 1 -- 172.21.15.90:0/3787553059 wait complete. 2024-05-27T07:41:56.901 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-27T07:41:56.901 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T07:41:56.901 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-05-27T07:41:56.901 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-27T07:41:56.901 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:41:56.901 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-27T07:41:56.901 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T07:41:56.901 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T07:41:56.901 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-27T07:41:56.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:41:56.902 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-05-27T07:41:56.955 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T07:41:56.955 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T07:41:56.955 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T07:41:56.955 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:41:56.955 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-05-27T07:41:56.955 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T07:41:56.956 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 41846sec preferred_lft 41846sec 2024-05-27T07:41:56.957 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-05-27T07:41:56.957 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:41:56.958 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T07:41:56.958 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:41:56.958 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-05-27T07:41:56.958 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-05-27T07:41:56.958 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-05-27T07:41:56.958 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-05-27T07:41:56.959 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T07:41:56.959 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-05-27T07:41:56.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:56 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3787553059' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:41:57.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:41:57.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:56 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3787553059' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:41:57.085 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:56 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3787553059' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:41:57.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:41:57.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:41:57.475 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T07:41:57.542 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-05-27T07:41:57.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:41:57.546 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-05-27T07:41:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-05-27T07:41:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T07:41:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T07:41:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T07:41:57.604 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T07:41:57.605 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:41:57.605 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-05-27T07:41:57.605 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T07:41:57.605 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T07:41:57.605 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T07:41:57.605 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-05-27T07:41:57.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:41:57.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:57 smithi136 ceph-mon[35324]: pgmap v406: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:58.007 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:57 smithi090 ceph-mon[31429]: pgmap v406: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:58.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:57 smithi129 ceph-mon[35254]: pgmap v406: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:58.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:41:58.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:41:58.444 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-05-27T07:41:58.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:41:58.498 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-05-27T07:41:58.547 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:41:58.548 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-05-27T07:41:58.548 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:41:58.548 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-27T07:41:58.548 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-05-27T07:41:58.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:41:58.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:41:58.731 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-27T07:41:58.731 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:41:58.731 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-05-27T07:41:58.731 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-27T07:41:58.731 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T07:41:58.731 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-27T07:41:58.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-27T07:41:58.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-27T07:41:58.732 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-05-27T07:41:58.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:41:58.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-27T07:41:58.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-27T07:41:58.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-27T07:41:58.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [7, 11]}]: dispatch 2024-05-27T07:41:58.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-05-27T07:41:58.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-27T07:41:58.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-05-27T07:41:58.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-27T07:41:58.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-27T07:41:58.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-05-27T07:41:59.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:41:59.031 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:41:59.031 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-05-27T07:41:59.031 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-05-27T07:41:59.031 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T07:41:59.031 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-05-27T07:41:59.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-27T07:41:59.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-27T07:41:59.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-27T07:41:59.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [7, 11]}]: dispatch 2024-05-27T07:41:59.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-05-27T07:41:59.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-27T07:41:59.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-05-27T07:41:59.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-27T07:41:59.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-27T07:41:59.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-05-27T07:41:59.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:41:59.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:41:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:41:59.229 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-27T07:41:59.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T07:41:59.230 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:41:59.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-27T07:41:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-27T07:41:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-27T07:41:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [7, 11]}]: dispatch 2024-05-27T07:41:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-05-27T07:41:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-27T07:41:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-05-27T07:41:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-27T07:41:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-27T07:41:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]: dispatch 2024-05-27T07:41:59.272 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-27T07:41:59.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T07:41:59.272 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:41:59.369 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-05-27T07:41:59.451 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-05-27T07:41:59.503 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-05-27T07:41:59.504 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/9e1bcc8f262b04006b2646c82924c60582162fe1749f0a10dc84210cf998766f/merged 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/da074e182c088100c0366956aa071553c14b7d365bcfe7ba452de0a09a3e6cdb/merged 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/74183af9942d0e0b14483025c64637abb9123624a14b778b4cc4ad7a66c834d6/merged 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c7f3c7639caab68be3492caf2029c5c0dd8c399e09fd109f58a8afd7d9d82fcf/merged 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/dbd87aa827d3ec73230a009e0e92ea3edaea081f5ff915ae8e52bceb8ee4093b/merged 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/965856f8e1e17ab9ed7fd5db92818a04751564d3a836793e9712ae733fbcd15b/merged 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f94a5ad33b8640fbbaf471ef9e89a72c75e3fa026ba1cc36be0c985a34172a14/merged 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/31385d1d7f286038f0ca0a5f821a9012ffd5aad96c380bb1eb781def5c2ec6d0/merged 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1d6e9600c151645e62fd289e9f10091935bc56ae566ab5a42dcc8d3214014af1/merged 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d6f64b9c9965724fb80d0b70b07d7eb0163f836be68aaeba2347967da586f14e/merged 2024-05-27T07:41:59.505 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-05-27T07:41:59.506 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:41:59.506 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:41:59.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:41:59.507 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-05-27T07:41:59.554 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-27T07:41:59.555 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-27T07:41:59.559 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-05-27T07:41:59.624 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:41:59.664 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-05-27T07:41:59.665 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:41:59.666 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/9e1bcc8f262b04006b2646c82924c60582162fe1749f0a10dc84210cf998766f/merged 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/da074e182c088100c0366956aa071553c14b7d365bcfe7ba452de0a09a3e6cdb/merged 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/74183af9942d0e0b14483025c64637abb9123624a14b778b4cc4ad7a66c834d6/merged 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c7f3c7639caab68be3492caf2029c5c0dd8c399e09fd109f58a8afd7d9d82fcf/merged 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/dbd87aa827d3ec73230a009e0e92ea3edaea081f5ff915ae8e52bceb8ee4093b/merged 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/965856f8e1e17ab9ed7fd5db92818a04751564d3a836793e9712ae733fbcd15b/merged 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f94a5ad33b8640fbbaf471ef9e89a72c75e3fa026ba1cc36be0c985a34172a14/merged 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/31385d1d7f286038f0ca0a5f821a9012ffd5aad96c380bb1eb781def5c2ec6d0/merged 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1d6e9600c151645e62fd289e9f10091935bc56ae566ab5a42dcc8d3214014af1/merged 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d6f64b9c9965724fb80d0b70b07d7eb0163f836be68aaeba2347967da586f14e/merged 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-05-27T07:41:59.667 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:41:59.668 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:41:59.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:41:59.669 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-05-27T07:41:59.827 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:2024-05-27T07:41:59.829+0000 7fc00f909480 -1 init, newargv = 0x7fbff8004f20 newargc=15 2024-05-27T07:41:59.827 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[76796]: starting ceph client 2024-05-27T07:41:59.843 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[76796]: starting fuse 2024-05-27T07:41:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: pgmap v407: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:41:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-27T07:41:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-27T07:41:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-27T07:41:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [7, 11]}]': finished 2024-05-27T07:41:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-05-27T07:41:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-27T07:41:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-05-27T07:41:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-27T07:41:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-27T07:41:59.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-05-27T07:41:59.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:41:59 smithi090 ceph-mon[31429]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T07:42:00.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: pgmap v407: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:42:00.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-27T07:42:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-27T07:42:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-27T07:42:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [7, 11]}]': finished 2024-05-27T07:42:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-05-27T07:42:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-27T07:42:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-05-27T07:42:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-27T07:42:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-27T07:42:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-05-27T07:42:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:41:59 smithi129 ceph-mon[35254]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T07:42:00.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: pgmap v407: 129 pgs: 129 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:42:00.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-27T07:42:00.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-27T07:42:00.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-27T07:42:00.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [7, 11]}]': finished 2024-05-27T07:42:00.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-05-27T07:42:00.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-27T07:42:00.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-05-27T07:42:00.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-27T07:42:00.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-27T07:42:00.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [5, 8]}]': finished 2024-05-27T07:42:00.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:41:59 smithi136 ceph-mon[35324]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T07:42:00.857 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-05-27T07:42:00.951 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:42:01.005 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-05-27T07:42:01.005 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-05-27T07:42:01.005 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-05-27T07:42:01.005 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-05-27T07:42:01.005 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-05-27T07:42:01.005 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-05-27T07:42:01.005 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-05-27T07:42:01.005 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-05-27T07:42:01.005 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:42:01.006 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/9e1bcc8f262b04006b2646c82924c60582162fe1749f0a10dc84210cf998766f/merged 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/da074e182c088100c0366956aa071553c14b7d365bcfe7ba452de0a09a3e6cdb/merged 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/74183af9942d0e0b14483025c64637abb9123624a14b778b4cc4ad7a66c834d6/merged 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c7f3c7639caab68be3492caf2029c5c0dd8c399e09fd109f58a8afd7d9d82fcf/merged 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/dbd87aa827d3ec73230a009e0e92ea3edaea081f5ff915ae8e52bceb8ee4093b/merged 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/965856f8e1e17ab9ed7fd5db92818a04751564d3a836793e9712ae733fbcd15b/merged 2024-05-27T07:42:01.007 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f94a5ad33b8640fbbaf471ef9e89a72c75e3fa026ba1cc36be0c985a34172a14/merged 2024-05-27T07:42:01.008 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/31385d1d7f286038f0ca0a5f821a9012ffd5aad96c380bb1eb781def5c2ec6d0/merged 2024-05-27T07:42:01.008 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1d6e9600c151645e62fd289e9f10091935bc56ae566ab5a42dcc8d3214014af1/merged 2024-05-27T07:42:01.008 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d6f64b9c9965724fb80d0b70b07d7eb0163f836be68aaeba2347967da586f14e/merged 2024-05-27T07:42:01.008 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-05-27T07:42:01.008 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:42:01.008 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:42:01.008 INFO:teuthology.orchestra.run.smithi090.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-27T07:42:01.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:42:01.009 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-05-27T07:42:01.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:00 smithi129 ceph-mon[35254]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T07:42:01.063 INFO:teuthology.orchestra.run.smithi090.stdout:107 2024-05-27T07:42:01.063 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-05-27T07:42:01.063 DEBUG:teuthology.orchestra.run.smithi090:> sudo stdin-killer -- python3 -c ' 2024-05-27T07:42:01.063 DEBUG:teuthology.orchestra.run.smithi090:> import glob 2024-05-27T07:42:01.063 DEBUG:teuthology.orchestra.run.smithi090:> import re 2024-05-27T07:42:01.063 DEBUG:teuthology.orchestra.run.smithi090:> import os 2024-05-27T07:42:01.063 DEBUG:teuthology.orchestra.run.smithi090:> import subprocess 2024-05-27T07:42:01.063 DEBUG:teuthology.orchestra.run.smithi090:> 2024-05-27T07:42:01.063 DEBUG:teuthology.orchestra.run.smithi090:> def _find_admin_socket(client_name): 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> files = glob.glob(asok_path) 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> # Given a non-glob path, it better be there 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> if "*" not in asok_path: 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> assert(len(files) == 1) 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> return files[0] 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> for f in files: 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-27T07:42:01.064 DEBUG:teuthology.orchestra.run.smithi090:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-27T07:42:01.065 DEBUG:teuthology.orchestra.run.smithi090:> contents = proc_f.read() 2024-05-27T07:42:01.065 DEBUG:teuthology.orchestra.run.smithi090:> if mountpoint in contents: 2024-05-27T07:42:01.065 DEBUG:teuthology.orchestra.run.smithi090:> return f 2024-05-27T07:42:01.065 DEBUG:teuthology.orchestra.run.smithi090:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-27T07:42:01.065 DEBUG:teuthology.orchestra.run.smithi090:> 2024-05-27T07:42:01.065 DEBUG:teuthology.orchestra.run.smithi090:> print(_find_admin_socket("client.admin")) 2024-05-27T07:42:01.065 DEBUG:teuthology.orchestra.run.smithi090:> ' 2024-05-27T07:42:01.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:00 smithi090 ceph-mon[31429]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T07:42:01.229 INFO:teuthology.orchestra.run.smithi090.stdout:/var/run/ceph/ceph-client.admin.76796.asok 2024-05-27T07:42:01.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:42:01.247 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76796.asok 2024-05-27T07:42:01.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:42:01.247 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76796.asok status 2024-05-27T07:42:01.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:00 smithi136 ceph-mon[35324]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout: "metadata": { 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_sha1": "38f9d900c83c63799fbdbe61acc9a11b0d3554a6", 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout: "entity_id": "admin", 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": "smithi090", 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout: "pid": "76796", 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout: "root": "/" 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T07:42:01.545 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_count": 0, 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_pinned_count": 0, 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 14682, 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "inst": { 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "name": { 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "client", 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "num": 14682 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 44577785 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T07:42:01.546 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 44577785 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "inst_str": "client.14682 192.168.0.1:0/44577785", 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "addr_str": "192.168.0.1:0/44577785", 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "inode_count": 1, 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_epoch": 17, 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch": 89, 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch_barrier": 0, 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "blocklisted": false, 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs" 2024-05-27T07:42:01.547 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:42:01.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:42:01.566 DEBUG:teuthology.orchestra.run.smithi090:> stat --file-system '--printf=%T 2024-05-27T07:42:01.566 DEBUG:teuthology.orchestra.run.smithi090:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:42:01.617 INFO:teuthology.orchestra.run.smithi090.stdout:fuseblk 2024-05-27T07:42:01.617 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:42:01.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:42:01.617 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:42:01.700 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-27T07:42:01.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T07:42:01.854 INFO:teuthology.orchestra.run.smithi090.stdout:# file: . 2024-05-27T07:42:01.854 INFO:teuthology.orchestra.run.smithi090.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-27T07:42:01.854 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:42:01.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:42:01.864 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:42:01.864 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T07:42:01.915 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T07:42:01.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:01 smithi090 ceph-mon[31429]: pgmap v410: 129 pgs: 6 peering, 123 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:42:02.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:01 smithi129 ceph-mon[35254]: pgmap v410: 129 pgs: 6 peering, 123 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:42:02.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:01 smithi136 ceph-mon[35324]: pgmap v410: 129 pgs: 6 peering, 123 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:42:02.354 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:03.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:03 smithi090 ceph-mon[31429]: pgmap v411: 129 pgs: 6 peering, 123 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:42:04.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:03 smithi129 ceph-mon[35254]: pgmap v411: 129 pgs: 6 peering, 123 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:42:04.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:03 smithi136 ceph-mon[35324]: pgmap v411: 129 pgs: 6 peering, 123 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:42:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.617+0000 7fb05f800700 1 -- 172.21.15.90:0/2428778194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0581034b0 msgr2=0x7fb0581038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.617+0000 7fb05f800700 1 --2- 172.21.15.90:0/2428778194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0581034b0 0x7fb0581038c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0099c0 tx=0x7fb04c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:42:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.617+0000 7fb05f800700 1 -- 172.21.15.90:0/2428778194 shutdown_connections 2024-05-27T07:42:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.617+0000 7fb05f800700 1 --2- 172.21.15.90:0/2428778194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb058108730 0x7fb058110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.617+0000 7fb05f800700 1 --2- 172.21.15.90:0/2428778194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0581034b0 0x7fb0581038c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.617+0000 7fb05f800700 1 --2- 172.21.15.90:0/2428778194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb058107de0 0x7fb0581081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.617+0000 7fb05f800700 1 -- 172.21.15.90:0/2428778194 >> 172.21.15.90:0/2428778194 conn(0x7fb0580fede0 msgr2=0x7fb058101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:04.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.618+0000 7fb05f800700 1 -- 172.21.15.90:0/2428778194 shutdown_connections 2024-05-27T07:42:04.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.618+0000 7fb05f800700 1 -- 172.21.15.90:0/2428778194 wait complete. 2024-05-27T07:42:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.619+0000 7fb05f800700 1 Processor -- start 2024-05-27T07:42:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.619+0000 7fb05f800700 1 -- start start 2024-05-27T07:42:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.619+0000 7fb05f800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0581034b0 0x7fb05819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.619+0000 7fb05f800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb058107de0 0x7fb05819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.619+0000 7fb05f800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb058108730 0x7fb05819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.619+0000 7fb05f800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb058113260 con 0x7fb058108730 2024-05-27T07:42:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.619+0000 7fb05f800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb0581130e0 con 0x7fb0581034b0 2024-05-27T07:42:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.619+0000 7fb05f800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0581133e0 con 0x7fb058107de0 2024-05-27T07:42:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.620+0000 7fb05d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb058108730 0x7fb05819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.620+0000 7fb05d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb058108730 0x7fb05819f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44270/0 (socket says 172.21.15.90:44270) 2024-05-27T07:42:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.620+0000 7fb05d800700 1 -- 172.21.15.90:0/2661089597 learned_addr learned my addr 172.21.15.90:0/2661089597 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.620+0000 7fb05ce00700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0581034b0 0x7fb05819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:04.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.620+0000 7fb057e00700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb058107de0 0x7fb05819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:04.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.620+0000 7fb05d800700 1 -- 172.21.15.90:0/2661089597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb058107de0 msgr2=0x7fb05819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:04.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.620+0000 7fb05d800700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb058107de0 0x7fb05819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:04.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.620+0000 7fb05d800700 1 -- 172.21.15.90:0/2661089597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0581034b0 msgr2=0x7fb05819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:04.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.620+0000 7fb05d800700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0581034b0 0x7fb05819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:04.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.620+0000 7fb05d800700 1 -- 172.21.15.90:0/2661089597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05819fc90 con 0x7fb058108730 2024-05-27T07:42:04.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.621+0000 7fb05d800700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb058108730 0x7fb05819f5f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb05000ef30 tx=0x7fb05000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.621+0000 7fb055600700 1 -- 172.21.15.90:0/2661089597 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0500092e0 con 0x7fb058108730 2024-05-27T07:42:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.621+0000 7fb05f800700 1 -- 172.21.15.90:0/2661089597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05819ff20 con 0x7fb058108730 2024-05-27T07:42:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.621+0000 7fb055600700 1 -- 172.21.15.90:0/2661089597 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb050004bb0 con 0x7fb058108730 2024-05-27T07:42:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.621+0000 7fb055600700 1 -- 172.21.15.90:0/2661089597 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050004220 con 0x7fb058108730 2024-05-27T07:42:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.621+0000 7fb05f800700 1 -- 172.21.15.90:0/2661089597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0581a78a0 con 0x7fb058108730 2024-05-27T07:42:04.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.622+0000 7fb05f800700 1 -- 172.21.15.90:0/2661089597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0580512b0 con 0x7fb058108730 2024-05-27T07:42:04.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.625+0000 7fb055600700 1 -- 172.21.15.90:0/2661089597 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb05001b030 con 0x7fb058108730 2024-05-27T07:42:04.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.626+0000 7fb055600700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb0400b2880 0x7fb0400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.627+0000 7fb055600700 1 -- 172.21.15.90:0/2661089597 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb0500e2ae0 con 0x7fb058108730 2024-05-27T07:42:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.627+0000 7fb05ce00700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb0400b2880 0x7fb0400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.628+0000 7fb05ce00700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb0400b2880 0x7fb0400b4d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb04c006010 tx=0x7fb04c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:04.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.629+0000 7fb055600700 1 -- 172.21.15.90:0/2661089597 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb050022020 con 0x7fb058108730 2024-05-27T07:42:04.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.881+0000 7fb05f800700 1 -- 172.21.15.90:0/2661089597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb05802e5e0 con 0x7fb058108730 2024-05-27T07:42:04.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.882+0000 7fb055600700 1 -- 172.21.15.90:0/2661089597 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7fb050010040 con 0x7fb058108730 2024-05-27T07:42:04.880 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1476836587 2024-05-28T07:30:58.154487+0000 2024-05-27T07:42:04.880 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3178069601 2024-05-28T07:30:58.154487+0000 2024-05-27T07:42:04.881 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/2742579272 2024-05-28T07:30:58.154487+0000 2024-05-27T07:42:04.881 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/2742579272 2024-05-28T07:30:58.154487+0000 2024-05-27T07:42:04.881 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/410158065 2024-05-28T07:30:23.548910+0000 2024-05-27T07:42:04.881 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2593454412 2024-05-28T07:30:23.548910+0000 2024-05-27T07:42:04.881 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3534244243 2024-05-28T07:30:23.548910+0000 2024-05-27T07:42:04.881 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2575484267 2024-05-28T07:30:23.548910+0000 2024-05-27T07:42:04.881 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/410158065 2024-05-28T07:30:23.548910+0000 2024-05-27T07:42:04.881 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2626688812 2024-05-28T07:30:58.154487+0000 2024-05-27T07:42:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.885+0000 7fb05f800700 1 -- 172.21.15.90:0/2661089597 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb0400b2880 msgr2=0x7fb0400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.885+0000 7fb05f800700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb0400b2880 0x7fb0400b4d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb04c006010 tx=0x7fb04c01a040 comp rx=0 tx=0).stop 2024-05-27T07:42:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.885+0000 7fb05f800700 1 -- 172.21.15.90:0/2661089597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb058108730 msgr2=0x7fb05819f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.885+0000 7fb05f800700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb058108730 0x7fb05819f5f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb05000ef30 tx=0x7fb05000c450 comp rx=0 tx=0).stop 2024-05-27T07:42:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.885+0000 7fb05f800700 1 -- 172.21.15.90:0/2661089597 shutdown_connections 2024-05-27T07:42:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.885+0000 7fb05f800700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb0400b2880 0x7fb0400b4d90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.885+0000 7fb05f800700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb058108730 0x7fb05819f5f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.885+0000 7fb05f800700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0581034b0 0x7fb05819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.885+0000 7fb05f800700 1 --2- 172.21.15.90:0/2661089597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb058107de0 0x7fb05819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.885+0000 7fb05f800700 1 -- 172.21.15.90:0/2661089597 >> 172.21.15.90:0/2661089597 conn(0x7fb0580fede0 msgr2=0x7fb058075e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.886+0000 7fb05f800700 1 -- 172.21.15.90:0/2661089597 shutdown_connections 2024-05-27T07:42:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:04.886+0000 7fb05f800700 1 -- 172.21.15.90:0/2661089597 wait complete. 2024-05-27T07:42:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:listed 10 entries 2024-05-27T07:42:05.662 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:42:05.662 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T07:42:05.760 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T07:42:05.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:05 smithi090 ceph-mon[31429]: pgmap v412: 129 pgs: 6 peering, 123 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:42:05.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:05 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2661089597' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:42:06.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:05 smithi129 ceph-mon[35254]: pgmap v412: 129 pgs: 6 peering, 123 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:42:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:05 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2661089597' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:42:06.216 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:06.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:05 smithi136 ceph-mon[35324]: pgmap v412: 129 pgs: 6 peering, 123 active+clean; 579 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-27T07:42:06.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:05 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2661089597' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:42:07.061 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:06 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:42:07.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:06 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:42:07.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:06 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:42:08.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:07 smithi090 ceph-mon[31429]: pgmap v413: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-05-27T07:42:08.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:07 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.222 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:07 smithi136 ceph-mon[35324]: pgmap v413: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-05-27T07:42:08.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.223 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:07 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.254 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:07 smithi129 ceph-mon[35254]: pgmap v413: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 201 B/s, 0 objects/s recovering 2024-05-27T07:42:08.254 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:07 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.255 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:07 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.255 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:07 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.255 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:07 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:08.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.591+0000 7f8403000700 1 -- 172.21.15.90:0/2012217871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc076670 msgr2=0x7f83fc110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:08.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.591+0000 7f8403000700 1 --2- 172.21.15.90:0/2012217871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc076670 0x7f83fc110650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009990 tx=0x7f83e4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:42:08.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.591+0000 7f8403000700 1 -- 172.21.15.90:0/2012217871 shutdown_connections 2024-05-27T07:42:08.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.591+0000 7f8403000700 1 --2- 172.21.15.90:0/2012217871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83fc110b90 0x7f83fc112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:08.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.591+0000 7f8403000700 1 --2- 172.21.15.90:0/2012217871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83fc075d20 0x7f83fc076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:08.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.591+0000 7f8403000700 1 --2- 172.21.15.90:0/2012217871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc076670 0x7f83fc110650 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:08.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.591+0000 7f8403000700 1 -- 172.21.15.90:0/2012217871 >> 172.21.15.90:0/2012217871 conn(0x7f83fc100030 msgr2=0x7f83fc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:08.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.591+0000 7f8403000700 1 -- 172.21.15.90:0/2012217871 shutdown_connections 2024-05-27T07:42:08.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.591+0000 7f8403000700 1 -- 172.21.15.90:0/2012217871 wait complete. 2024-05-27T07:42:08.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.593+0000 7f8403000700 1 Processor -- start 2024-05-27T07:42:08.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.593+0000 7f8403000700 1 -- start start 2024-05-27T07:42:08.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f8403000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83fc075d20 0x7f83fc19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:08.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f8403000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83fc076670 0x7f83fc19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:08.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f8403000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc110b90 0x7f83fc1a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:08.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f8403000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc116590 con 0x7f83fc110b90 2024-05-27T07:42:08.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f8403000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc116410 con 0x7f83fc075d20 2024-05-27T07:42:08.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f8403000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc116710 con 0x7f83fc076670 2024-05-27T07:42:08.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f8401000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc110b90 0x7f83fc1a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:08.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f8401000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc110b90 0x7f83fc1a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44300/0 (socket says 172.21.15.90:44300) 2024-05-27T07:42:08.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f8401000700 1 -- 172.21.15.90:0/1023797457 learned_addr learned my addr 172.21.15.90:0/1023797457 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:08.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f83f3400700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83fc076670 0x7f83fc19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:08.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.594+0000 7f83fbe00700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83fc075d20 0x7f83fc19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:08.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f8401000700 1 -- 172.21.15.90:0/1023797457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83fc076670 msgr2=0x7f83fc19d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:08.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f8401000700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83fc076670 0x7f83fc19d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:08.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f8401000700 1 -- 172.21.15.90:0/1023797457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83fc075d20 msgr2=0x7f83fc19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:08.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f8401000700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83fc075d20 0x7f83fc19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:08.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f8401000700 1 -- 172.21.15.90:0/1023797457 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc1a1e00 con 0x7f83fc110b90 2024-05-27T07:42:08.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f8401000700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc110b90 0x7f83fc1a1810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83f400d950 tx=0x7f83f400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:08.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f83f9600700 1 -- 172.21.15.90:0/1023797457 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4017070 con 0x7f83fc110b90 2024-05-27T07:42:08.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f83f9600700 1 -- 172.21.15.90:0/1023797457 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f83f4009dd0 con 0x7f83fc110b90 2024-05-27T07:42:08.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f8403000700 1 -- 172.21.15.90:0/1023797457 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc1a20f0 con 0x7f83fc110b90 2024-05-27T07:42:08.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f83f9600700 1 -- 172.21.15.90:0/1023797457 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4005280 con 0x7f83fc110b90 2024-05-27T07:42:08.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.595+0000 7f8403000700 1 -- 172.21.15.90:0/1023797457 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc1a25e0 con 0x7f83fc110b90 2024-05-27T07:42:08.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.597+0000 7f8403000700 1 -- 172.21.15.90:0/1023797457 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83fc0512b0 con 0x7f83fc110b90 2024-05-27T07:42:08.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.599+0000 7f83f9600700 1 -- 172.21.15.90:0/1023797457 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f83f4005420 con 0x7f83fc110b90 2024-05-27T07:42:08.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.601+0000 7f83f9600700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f83e00b2880 0x7f83e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:08.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.601+0000 7f83f9600700 1 -- 172.21.15.90:0/1023797457 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f83f40e22d0 con 0x7f83fc110b90 2024-05-27T07:42:08.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.601+0000 7f83fbe00700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f83e00b2880 0x7f83e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:08.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.602+0000 7f83fbe00700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f83e00b2880 0x7f83e00b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e270 tx=0x7f83ec009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:08.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.604+0000 7f83f9600700 1 -- 172.21.15.90:0/1023797457 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83f40a9c30 con 0x7f83fc110b90 2024-05-27T07:42:08.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.853+0000 7f8403000700 1 -- 172.21.15.90:0/1023797457 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f83fc02e5e0 con 0x7f83fc110b90 2024-05-27T07:42:08.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.854+0000 7f83f9600700 1 -- 172.21.15.90:0/1023797457 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7f83f40a9530 con 0x7f83fc110b90 2024-05-27T07:42:08.853 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1476836587 2024-05-28T07:30:58.154487+0000 2024-05-27T07:42:08.853 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3178069601 2024-05-28T07:30:58.154487+0000 2024-05-27T07:42:08.853 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/2742579272 2024-05-28T07:30:58.154487+0000 2024-05-27T07:42:08.853 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/2742579272 2024-05-28T07:30:58.154487+0000 2024-05-27T07:42:08.853 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/410158065 2024-05-28T07:30:23.548910+0000 2024-05-27T07:42:08.853 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2593454412 2024-05-28T07:30:23.548910+0000 2024-05-27T07:42:08.853 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3534244243 2024-05-28T07:30:23.548910+0000 2024-05-27T07:42:08.853 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2575484267 2024-05-28T07:30:23.548910+0000 2024-05-27T07:42:08.853 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/410158065 2024-05-28T07:30:23.548910+0000 2024-05-27T07:42:08.853 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2626688812 2024-05-28T07:30:58.154487+0000 2024-05-27T07:42:08.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 -- 172.21.15.90:0/1023797457 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f83e00b2880 msgr2=0x7f83e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:08.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f83e00b2880 0x7f83e00b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00e270 tx=0x7f83ec009040 comp rx=0 tx=0).stop 2024-05-27T07:42:08.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 -- 172.21.15.90:0/1023797457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc110b90 msgr2=0x7f83fc1a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:08.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc110b90 0x7f83fc1a1810 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83f400d950 tx=0x7f83f400dd10 comp rx=0 tx=0).stop 2024-05-27T07:42:08.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 -- 172.21.15.90:0/1023797457 shutdown_connections 2024-05-27T07:42:08.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f83e00b2880 0x7f83e00b4d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:08.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83fc110b90 0x7f83fc1a1810 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:08.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f83fc075d20 0x7f83fc19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:08.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 --2- 172.21.15.90:0/1023797457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83fc076670 0x7f83fc19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:08.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 -- 172.21.15.90:0/1023797457 >> 172.21.15.90:0/1023797457 conn(0x7f83fc100030 msgr2=0x7f83fc100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:08.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 -- 172.21.15.90:0/1023797457 shutdown_connections 2024-05-27T07:42:08.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:08.857+0000 7f8403000700 1 -- 172.21.15.90:0/1023797457 wait complete. 2024-05-27T07:42:08.857 INFO:teuthology.orchestra.run.smithi090.stderr:listed 10 entries 2024-05-27T07:42:09.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:08 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1023797457' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:42:09.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:08 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1023797457' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:42:09.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:08 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1023797457' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T07:42:09.666 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi090.front.sepia.ceph.com... 2024-05-27T07:42:09.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:42:09.666 DEBUG:teuthology.orchestra.run.smithi090:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-27T07:42:09.736 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[76796]: fuse finished with error 0 and tester_r 0 2024-05-27T07:42:09.740 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T07:42:10.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:09 smithi090 ceph-mon[31429]: pgmap v414: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s; 167 B/s, 0 objects/s recovering 2024-05-27T07:42:10.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:09 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:09 smithi136 ceph-mon[35324]: pgmap v414: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s; 167 B/s, 0 objects/s recovering 2024-05-27T07:42:10.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:09 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:09 smithi129 ceph-mon[35254]: pgmap v414: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 425 B/s wr, 0 op/s; 167 B/s, 0 objects/s recovering 2024-05-27T07:42:10.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:10.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:09 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:11.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:11 smithi090 ceph-mon[31429]: pgmap v415: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 385 B/s wr, 0 op/s; 151 B/s, 0 objects/s recovering 2024-05-27T07:42:11.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:11.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:11 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:11.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:11 smithi136 ceph-mon[35324]: pgmap v415: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 385 B/s wr, 0 op/s; 151 B/s, 0 objects/s recovering 2024-05-27T07:42:11.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:11.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:11 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:11.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:11 smithi129 ceph-mon[35254]: pgmap v415: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 385 B/s wr, 0 op/s; 151 B/s, 0 objects/s recovering 2024-05-27T07:42:11.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:11.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:11 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:12.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:42:12.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:42:12.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:12.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:42:12.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:42:12.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:12.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:42:12.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:42:12.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:42:13.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:13 smithi136 ceph-mon[35324]: pgmap v416: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-05-27T07:42:14.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:13 smithi129 ceph-mon[35254]: pgmap v416: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-05-27T07:42:14.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:13 smithi090 ceph-mon[31429]: pgmap v416: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-05-27T07:42:15.842 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T07:42:15.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:42:15.843 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-27T07:42:15.915 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:42:15.992 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:15 smithi090 ceph-mon[31429]: pgmap v417: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-05-27T07:42:15.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:15 smithi136 ceph-mon[35324]: pgmap v417: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-05-27T07:42:16.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:15 smithi129 ceph-mon[35254]: pgmap v417: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-05-27T07:42:16.366 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:17.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:17 smithi090 ceph-mon[31429]: pgmap v418: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-05-27T07:42:17.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:17 smithi136 ceph-mon[35324]: pgmap v418: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-05-27T07:42:18.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:17 smithi129 ceph-mon[35254]: pgmap v418: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 134 B/s, 0 objects/s recovering 2024-05-27T07:42:18.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.575+0000 7f4306c00700 1 -- 172.21.15.90:0/1013774637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4300105470 msgr2=0x7f4300105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:18.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.575+0000 7f4306c00700 1 --2- 172.21.15.90:0/1013774637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4300105470 0x7f4300105880 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009990 tx=0x7f42f0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.575+0000 7f4306c00700 1 -- 172.21.15.90:0/1013774637 shutdown_connections 2024-05-27T07:42:18.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.575+0000 7f4306c00700 1 --2- 172.21.15.90:0/1013774637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f430010e8e0 0x7f4300110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.575+0000 7f4306c00700 1 --2- 172.21.15.90:0/1013774637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4300105470 0x7f4300105880 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.575+0000 7f4306c00700 1 --2- 172.21.15.90:0/1013774637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4300105dc0 0x7f430010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.575+0000 7f4306c00700 1 -- 172.21.15.90:0/1013774637 >> 172.21.15.90:0/1013774637 conn(0x7f43000fef20 msgr2=0x7f4300101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:18.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.575+0000 7f4306c00700 1 -- 172.21.15.90:0/1013774637 shutdown_connections 2024-05-27T07:42:18.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.575+0000 7f4306c00700 1 -- 172.21.15.90:0/1013774637 wait complete. 2024-05-27T07:42:18.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.576+0000 7f4306c00700 1 Processor -- start 2024-05-27T07:42:18.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.576+0000 7f4306c00700 1 -- start start 2024-05-27T07:42:18.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.577+0000 7f4306c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4300105470 0x7f430019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:18.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.577+0000 7f4306c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4300105dc0 0x7f430019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:18.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.577+0000 7f4306c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010e8e0 0x7f430019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:18.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.577+0000 7f4306c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4300114330 con 0x7f430010e8e0 2024-05-27T07:42:18.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.577+0000 7f4306c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43001141b0 con 0x7f4300105470 2024-05-27T07:42:18.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.577+0000 7f4306c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43001144b0 con 0x7f4300105dc0 2024-05-27T07:42:18.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.577+0000 7f4304c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010e8e0 0x7f430019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:18.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.577+0000 7f4304c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010e8e0 0x7f430019f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40330/0 (socket says 172.21.15.90:40330) 2024-05-27T07:42:18.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.577+0000 7f4304c00700 1 -- 172.21.15.90:0/3532746640 learned_addr learned my addr 172.21.15.90:0/3532746640 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:18.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.577+0000 7f42f7400700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4300105dc0 0x7f430019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:18.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.578+0000 7f42ffe00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4300105470 0x7f430019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:18.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.578+0000 7f4304c00700 1 -- 172.21.15.90:0/3532746640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4300105dc0 msgr2=0x7f430019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:18.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.578+0000 7f4304c00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4300105dc0 0x7f430019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.578+0000 7f4304c00700 1 -- 172.21.15.90:0/3532746640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4300105470 msgr2=0x7f430019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:18.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.578+0000 7f4304c00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4300105470 0x7f430019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.578+0000 7f4304c00700 1 -- 172.21.15.90:0/3532746640 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430019fc10 con 0x7f430010e8e0 2024-05-27T07:42:18.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.578+0000 7f4304c00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010e8e0 0x7f430019f570 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42f800ada0 tx=0x7f42f800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:18.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.578+0000 7f42fd600700 1 -- 172.21.15.90:0/3532746640 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8017020 con 0x7f430010e8e0 2024-05-27T07:42:18.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.578+0000 7f42fd600700 1 -- 172.21.15.90:0/3532746640 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42f800ce10 con 0x7f430010e8e0 2024-05-27T07:42:18.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.578+0000 7f42fd600700 1 -- 172.21.15.90:0/3532746640 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f8005440 con 0x7f430010e8e0 2024-05-27T07:42:18.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.579+0000 7f4306c00700 1 -- 172.21.15.90:0/3532746640 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430019ff00 con 0x7f430010e8e0 2024-05-27T07:42:18.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.579+0000 7f4306c00700 1 -- 172.21.15.90:0/3532746640 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43001a7820 con 0x7f430010e8e0 2024-05-27T07:42:18.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.580+0000 7f4306c00700 1 -- 172.21.15.90:0/3532746640 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43000512b0 con 0x7f430010e8e0 2024-05-27T07:42:18.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.582+0000 7f42fd600700 1 -- 172.21.15.90:0/3532746640 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f42f8004ad0 con 0x7f430010e8e0 2024-05-27T07:42:18.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.584+0000 7f42fd600700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42e40b2900 0x7f42e40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:18.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.584+0000 7f42ffe00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42e40b2900 0x7f42e40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:18.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.584+0000 7f42fd600700 1 -- 172.21.15.90:0/3532746640 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f42f80e22c0 con 0x7f430010e8e0 2024-05-27T07:42:18.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.585+0000 7f42ffe00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42e40b2900 0x7f42e40b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f42f000b5c0 tx=0x7f42f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:18.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.587+0000 7f42fd600700 1 -- 172.21.15.90:0/3532746640 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42f8021020 con 0x7f430010e8e0 2024-05-27T07:42:18.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.820+0000 7f4306c00700 1 -- 172.21.15.90:0/3532746640 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f430002e5e0 con 0x7f430010e8e0 2024-05-27T07:42:18.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.822+0000 7f42fd600700 1 -- 172.21.15.90:0/3532746640 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24632 (secure 0 0 0) 0x7f42f80a9cb0 con 0x7f430010e8e0 2024-05-27T07:42:18.821 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:42:18.822 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":89,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:41:59.747494+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:42:18.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 -- 172.21.15.90:0/3532746640 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42e40b2900 msgr2=0x7f42e40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:18.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42e40b2900 0x7f42e40b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f42f000b5c0 tx=0x7f42f0005fb0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 -- 172.21.15.90:0/3532746640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010e8e0 msgr2=0x7f430019f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:18.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010e8e0 0x7f430019f570 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42f800ada0 tx=0x7f42f800c330 comp rx=0 tx=0).stop 2024-05-27T07:42:18.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 -- 172.21.15.90:0/3532746640 shutdown_connections 2024-05-27T07:42:18.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f42e40b2900 0x7f42e40b4e10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f430010e8e0 0x7f430019f570 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4300105470 0x7f430019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 --2- 172.21.15.90:0/3532746640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4300105dc0 0x7f430019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:18.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 -- 172.21.15.90:0/3532746640 >> 172.21.15.90:0/3532746640 conn(0x7f43000fef20 msgr2=0x7f43000ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:18.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 -- 172.21.15.90:0/3532746640 shutdown_connections 2024-05-27T07:42:18.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:18.826+0000 7f4306c00700 1 -- 172.21.15.90:0/3532746640 wait complete. 2024-05-27T07:42:19.620 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:42:19.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:19 smithi090 ceph-mon[31429]: pgmap v419: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:19.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:19 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3532746640' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:42:20.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:19 smithi129 ceph-mon[35254]: pgmap v419: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:20.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:19 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3532746640' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:42:20.070 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:20.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:19 smithi136 ceph-mon[35324]: pgmap v419: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:20.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:19 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3532746640' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:42:21.953 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:21 smithi090 ceph-mon[31429]: pgmap v420: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:22.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:21 smithi129 ceph-mon[35254]: pgmap v420: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:22.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:21 smithi136 ceph-mon[35324]: pgmap v420: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:22.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.288+0000 7ff9f1800700 1 -- 172.21.15.90:0/726969554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9ec0764a0 msgr2=0x7ff9ec114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:22.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.288+0000 7ff9f1800700 1 --2- 172.21.15.90:0/726969554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9ec0764a0 0x7ff9ec114e20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009990 tx=0x7ff9d4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.288+0000 7ff9f1800700 1 -- 172.21.15.90:0/726969554 shutdown_connections 2024-05-27T07:42:22.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.288+0000 7ff9f1800700 1 --2- 172.21.15.90:0/726969554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9ec115360 0x7ff9ec117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.288+0000 7ff9f1800700 1 --2- 172.21.15.90:0/726969554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9ec075b50 0x7ff9ec075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.288+0000 7ff9f1800700 1 --2- 172.21.15.90:0/726969554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9ec0764a0 0x7ff9ec114e20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.288+0000 7ff9f1800700 1 -- 172.21.15.90:0/726969554 >> 172.21.15.90:0/726969554 conn(0x7ff9ec100030 msgr2=0x7ff9ec102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:22.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.288+0000 7ff9f1800700 1 -- 172.21.15.90:0/726969554 shutdown_connections 2024-05-27T07:42:22.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.288+0000 7ff9f1800700 1 -- 172.21.15.90:0/726969554 wait complete. 2024-05-27T07:42:22.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.289+0000 7ff9f1800700 1 Processor -- start 2024-05-27T07:42:22.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.289+0000 7ff9f1800700 1 -- start start 2024-05-27T07:42:22.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9f1800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9ec075b50 0x7ff9ec1a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:22.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9f1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9ec0764a0 0x7ff9ec1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:22.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9f1800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9ec115360 0x7ff9ec1a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:22.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9f1800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec11a910 con 0x7ff9ec0764a0 2024-05-27T07:42:22.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9f1800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec11a790 con 0x7ff9ec115360 2024-05-27T07:42:22.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9f1800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec11aa90 con 0x7ff9ec075b50 2024-05-27T07:42:22.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9e2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9ec0764a0 0x7ff9ec1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:22.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9e2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9ec0764a0 0x7ff9ec1a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42958/0 (socket says 172.21.15.90:42958) 2024-05-27T07:42:22.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9e2000700 1 -- 172.21.15.90:0/2787697907 learned_addr learned my addr 172.21.15.90:0/2787697907 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:22.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9eaa00700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9ec075b50 0x7ff9ec1a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:22.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9eb400700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9ec115360 0x7ff9ec1a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:22.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9e2000700 1 -- 172.21.15.90:0/2787697907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9ec075b50 msgr2=0x7ff9ec1a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:22.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9e2000700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9ec075b50 0x7ff9ec1a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.290+0000 7ff9e2000700 1 -- 172.21.15.90:0/2787697907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9ec115360 msgr2=0x7ff9ec1a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:22.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.291+0000 7ff9e2000700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9ec115360 0x7ff9ec1a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.291+0000 7ff9e2000700 1 -- 172.21.15.90:0/2787697907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec1a6210 con 0x7ff9ec0764a0 2024-05-27T07:42:22.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.291+0000 7ff9eb400700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9ec115360 0x7ff9ec1a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:42:22.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.291+0000 7ff9e2000700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9ec0764a0 0x7ff9ec1a18e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4000c00 tx=0x7ff9d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:22.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.291+0000 7ff9e3e00700 1 -- 172.21.15.90:0/2787697907 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4027070 con 0x7ff9ec0764a0 2024-05-27T07:42:22.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.291+0000 7ff9e3e00700 1 -- 172.21.15.90:0/2787697907 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9d400fc70 con 0x7ff9ec0764a0 2024-05-27T07:42:22.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.291+0000 7ff9f1800700 1 -- 172.21.15.90:0/2787697907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ec1a64a0 con 0x7ff9ec0764a0 2024-05-27T07:42:22.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.291+0000 7ff9e3e00700 1 -- 172.21.15.90:0/2787697907 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4022d40 con 0x7ff9ec0764a0 2024-05-27T07:42:22.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.291+0000 7ff9f1800700 1 -- 172.21.15.90:0/2787697907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ec1a6980 con 0x7ff9ec0764a0 2024-05-27T07:42:22.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.292+0000 7ff9f1800700 1 -- 172.21.15.90:0/2787697907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ec0512b0 con 0x7ff9ec0764a0 2024-05-27T07:42:22.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.295+0000 7ff9e3e00700 1 -- 172.21.15.90:0/2787697907 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff9d402a070 con 0x7ff9ec0764a0 2024-05-27T07:42:22.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.296+0000 7ff9e3e00700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff9d00b2880 0x7ff9d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:22.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.297+0000 7ff9e3e00700 1 -- 172.21.15.90:0/2787697907 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7ff9d40e38c0 con 0x7ff9ec0764a0 2024-05-27T07:42:22.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.297+0000 7ff9eaa00700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff9d00b2880 0x7ff9d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:22.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.298+0000 7ff9eaa00700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff9d00b2880 0x7ff9d00b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc005950 tx=0x7ff9dc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:22.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.301+0000 7ff9e3e00700 1 -- 172.21.15.90:0/2787697907 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9d4013070 con 0x7ff9ec0764a0 2024-05-27T07:42:22.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.536+0000 7ff9f1800700 1 -- 172.21.15.90:0/2787697907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9ec02e5e0 con 0x7ff9ec0764a0 2024-05-27T07:42:22.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.538+0000 7ff9e3e00700 1 -- 172.21.15.90:0/2787697907 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24632 (secure 0 0 0) 0x7ff9d400fe20 con 0x7ff9ec0764a0 2024-05-27T07:42:22.538 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:42:22.538 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":89,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:41:59.747494+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:42:22.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 -- 172.21.15.90:0/2787697907 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff9d00b2880 msgr2=0x7ff9d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:22.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff9d00b2880 0x7ff9d00b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc005950 tx=0x7ff9dc0058e0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 -- 172.21.15.90:0/2787697907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9ec0764a0 msgr2=0x7ff9ec1a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:22.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9ec0764a0 0x7ff9ec1a18e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4000c00 tx=0x7ff9d4009f90 comp rx=0 tx=0).stop 2024-05-27T07:42:22.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 -- 172.21.15.90:0/2787697907 shutdown_connections 2024-05-27T07:42:22.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff9d00b2880 0x7ff9d00b4d90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9ec115360 0x7ff9ec1a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9ec075b50 0x7ff9ec1a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 --2- 172.21.15.90:0/2787697907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9ec0764a0 0x7ff9ec1a18e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:22.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 -- 172.21.15.90:0/2787697907 >> 172.21.15.90:0/2787697907 conn(0x7ff9ec100030 msgr2=0x7ff9ec101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:22.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 -- 172.21.15.90:0/2787697907 shutdown_connections 2024-05-27T07:42:22.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:22.542+0000 7ff9f1800700 1 -- 172.21.15.90:0/2787697907 wait complete. 2024-05-27T07:42:22.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:22 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2787697907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:42:23.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:22 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2787697907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:42:23.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:22 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2787697907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:42:23.361 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-27T07:42:23.778 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:24.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:23 smithi129 ceph-mon[35254]: pgmap v421: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:24.071 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:23 smithi090 ceph-mon[31429]: pgmap v421: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:24.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:23 smithi136 ceph-mon[35324]: pgmap v421: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:26.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:25 smithi129 ceph-mon[35254]: pgmap v422: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.065+0000 7f3e03200700 1 -- 172.21.15.90:0/1533818427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1043f0 msgr2=0x7f3dfc104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.065+0000 7f3e03200700 1 --2- 172.21.15.90:0/1533818427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1043f0 0x7f3dfc104800 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3df00099c0 tx=0x7f3df0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:42:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.065+0000 7f3e03200700 1 -- 172.21.15.90:0/1533818427 shutdown_connections 2024-05-27T07:42:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.065+0000 7f3e03200700 1 --2- 172.21.15.90:0/1533818427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dfc109670 0x7f3dfc110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.065+0000 7f3e03200700 1 --2- 172.21.15.90:0/1533818427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1043f0 0x7f3dfc104800 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.065+0000 7f3e03200700 1 --2- 172.21.15.90:0/1533818427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc108d20 0x7f3dfc109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.065+0000 7f3e03200700 1 -- 172.21.15.90:0/1533818427 >> 172.21.15.90:0/1533818427 conn(0x7f3dfc079d00 msgr2=0x7f3dfc07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.065+0000 7f3e03200700 1 -- 172.21.15.90:0/1533818427 shutdown_connections 2024-05-27T07:42:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.065+0000 7f3e03200700 1 -- 172.21.15.90:0/1533818427 wait complete. 2024-05-27T07:42:26.065 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:25 smithi090 ceph-mon[31429]: pgmap v422: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.067+0000 7f3e03200700 1 Processor -- start 2024-05-27T07:42:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.067+0000 7f3e03200700 1 -- start start 2024-05-27T07:42:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.067+0000 7f3e03200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1043f0 0x7f3dfc1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.067+0000 7f3e03200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dfc108d20 0x7f3dfc1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.067+0000 7f3e03200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc109670 0x7f3dfc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.067+0000 7f3e03200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1141a0 con 0x7f3dfc1043f0 2024-05-27T07:42:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.067+0000 7f3e03200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc114020 con 0x7f3dfc109670 2024-05-27T07:42:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.067+0000 7f3e03200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc114320 con 0x7f3dfc108d20 2024-05-27T07:42:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e01e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1043f0 0x7f3dfc1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e01e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1043f0 0x7f3dfc1a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42972/0 (socket says 172.21.15.90:42972) 2024-05-27T07:42:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e01e00700 1 -- 172.21.15.90:0/2980566041 learned_addr learned my addr 172.21.15.90:0/2980566041 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:26.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e01400700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dfc108d20 0x7f3dfc1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:26.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e02800700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc109670 0x7f3dfc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:26.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e01e00700 1 -- 172.21.15.90:0/2980566041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dfc108d20 msgr2=0x7f3dfc1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:26.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e01e00700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dfc108d20 0x7f3dfc1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:26.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e01e00700 1 -- 172.21.15.90:0/2980566041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc109670 msgr2=0x7f3dfc19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:26.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e01e00700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc109670 0x7f3dfc19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:26.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e01e00700 1 -- 172.21.15.90:0/2980566041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc19b130 con 0x7f3dfc1043f0 2024-05-27T07:42:26.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.068+0000 7f3e01e00700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1043f0 0x7f3dfc1a0860 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3df000b580 tx=0x7f3df000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:26.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.069+0000 7f3df6a00700 1 -- 172.21.15.90:0/2980566041 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0026070 con 0x7f3dfc1043f0 2024-05-27T07:42:26.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.069+0000 7f3e03200700 1 -- 172.21.15.90:0/2980566041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dfc19b3c0 con 0x7f3dfc1043f0 2024-05-27T07:42:26.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.069+0000 7f3df6a00700 1 -- 172.21.15.90:0/2980566041 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3df00054c0 con 0x7f3dfc1043f0 2024-05-27T07:42:26.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.069+0000 7f3e03200700 1 -- 172.21.15.90:0/2980566041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dfc19b8a0 con 0x7f3dfc1043f0 2024-05-27T07:42:26.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.069+0000 7f3df6a00700 1 -- 172.21.15.90:0/2980566041 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0004be0 con 0x7f3dfc1043f0 2024-05-27T07:42:26.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.070+0000 7f3e03200700 1 -- 172.21.15.90:0/2980566041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dfc0512b0 con 0x7f3dfc1043f0 2024-05-27T07:42:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.073+0000 7f3df6a00700 1 -- 172.21.15.90:0/2980566041 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3df0029070 con 0x7f3dfc1043f0 2024-05-27T07:42:26.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.074+0000 7f3df6a00700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3de40b2880 0x7f3de40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:26.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.074+0000 7f3df6a00700 1 -- 172.21.15.90:0/2980566041 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f3df00e2ff0 con 0x7f3dfc1043f0 2024-05-27T07:42:26.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.074+0000 7f3e01400700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3de40b2880 0x7f3de40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:26.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.075+0000 7f3e01400700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3de40b2880 0x7f3de40b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3dec005fd0 tx=0x7f3dec00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:26.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.079+0000 7f3df6a00700 1 -- 172.21.15.90:0/2980566041 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3df0013070 con 0x7f3dfc1043f0 2024-05-27T07:42:26.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:25 smithi136 ceph-mon[35324]: pgmap v422: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:26.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.363+0000 7f3e03200700 1 -- 172.21.15.90:0/2980566041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3dfc02e5e0 con 0x7f3dfc1043f0 2024-05-27T07:42:26.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.802+0000 7f3df6a00700 1 -- 172.21.15.90:0/2980566041 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f3df00aaa20 con 0x7f3dfc1043f0 2024-05-27T07:42:26.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.806+0000 7f3e03200700 1 -- 172.21.15.90:0/2980566041 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3de40b2880 msgr2=0x7f3de40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:26.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.806+0000 7f3e03200700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3de40b2880 0x7f3de40b4d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3dec005fd0 tx=0x7f3dec00a560 comp rx=0 tx=0).stop 2024-05-27T07:42:26.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.806+0000 7f3e03200700 1 -- 172.21.15.90:0/2980566041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1043f0 msgr2=0x7f3dfc1a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:26.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.806+0000 7f3e03200700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1043f0 0x7f3dfc1a0860 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3df000b580 tx=0x7f3df000f740 comp rx=0 tx=0).stop 2024-05-27T07:42:26.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.807+0000 7f3e03200700 1 -- 172.21.15.90:0/2980566041 shutdown_connections 2024-05-27T07:42:26.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.807+0000 7f3e03200700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3de40b2880 0x7f3de40b4d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:26.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.807+0000 7f3e03200700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dfc109670 0x7f3dfc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:26.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.807+0000 7f3e03200700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3dfc1043f0 0x7f3dfc1a0860 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:26.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.807+0000 7f3e03200700 1 --2- 172.21.15.90:0/2980566041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3dfc108d20 0x7f3dfc1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:26.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.807+0000 7f3e03200700 1 -- 172.21.15.90:0/2980566041 >> 172.21.15.90:0/2980566041 conn(0x7f3dfc079d00 msgr2=0x7f3dfc1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.807+0000 7f3e03200700 1 -- 172.21.15.90:0/2980566041 shutdown_connections 2024-05-27T07:42:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:26.807+0000 7f3e03200700 1 -- 172.21.15.90:0/2980566041 wait complete. 2024-05-27T07:42:26.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:26 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2980566041' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T07:42:27.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:26 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2980566041' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T07:42:27.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:26 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2980566041' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T07:42:27.631 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-27T07:42:27.631 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-27T07:42:27.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:27 smithi090 ceph-mon[31429]: pgmap v423: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:27.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:27 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2980566041' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T07:42:27.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:27 smithi090 ceph-mon[31429]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:42:28.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:27 smithi129 ceph-mon[35254]: pgmap v423: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:28.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:27 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2980566041' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T07:42:28.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:27 smithi129 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:42:28.085 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:28.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:27 smithi136 ceph-mon[35324]: pgmap v423: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-27T07:42:28.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:27 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2980566041' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T07:42:28.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:27 smithi136 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:42:29.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:29 smithi090 ceph-mon[31429]: pgmap v424: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:42:30.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:29 smithi136 ceph-mon[35324]: pgmap v424: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:42:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.254+0000 7f09bf000700 1 -- 172.21.15.90:0/3963891291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b8106570 msgr2=0x7f09b8106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.254+0000 7f09bf000700 1 --2- 172.21.15.90:0/3963891291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b8106570 0x7f09b8106980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009990 tx=0x7f09a4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.254+0000 7f09bf000700 1 -- 172.21.15.90:0/3963891291 shutdown_connections 2024-05-27T07:42:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.254+0000 7f09bf000700 1 --2- 172.21.15.90:0/3963891291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b8106ec0 0x7f09b8113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.254+0000 7f09bf000700 1 --2- 172.21.15.90:0/3963891291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b810c5b0 0x7f09b810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.254+0000 7f09bf000700 1 --2- 172.21.15.90:0/3963891291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b8106570 0x7f09b8106980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.254+0000 7f09bf000700 1 -- 172.21.15.90:0/3963891291 >> 172.21.15.90:0/3963891291 conn(0x7f09b8102070 msgr2=0x7f09b81044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.254+0000 7f09bf000700 1 -- 172.21.15.90:0/3963891291 shutdown_connections 2024-05-27T07:42:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.254+0000 7f09bf000700 1 -- 172.21.15.90:0/3963891291 wait complete. 2024-05-27T07:42:30.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.255+0000 7f09bf000700 1 Processor -- start 2024-05-27T07:42:30.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.255+0000 7f09bf000700 1 -- start start 2024-05-27T07:42:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09bf000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b8106570 0x7f09b819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09bf000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b8106ec0 0x7f09b819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09bf000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b810c5b0 0x7f09b81a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09bf000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f09b8116540 con 0x7f09b8106570 2024-05-27T07:42:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09bf000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f09b81163c0 con 0x7f09b8106ec0 2024-05-27T07:42:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09bf000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f09b81166c0 con 0x7f09b810c5b0 2024-05-27T07:42:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09b7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b8106570 0x7f09b819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09b7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b8106570 0x7f09b819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59726/0 (socket says 172.21.15.90:59726) 2024-05-27T07:42:30.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09b7e00700 1 -- 172.21.15.90:0/2227581867 learned_addr learned my addr 172.21.15.90:0/2227581867 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09b7400700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b8106ec0 0x7f09b819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:30.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09bd000700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b810c5b0 0x7f09b81a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09b7e00700 1 -- 172.21.15.90:0/2227581867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b810c5b0 msgr2=0x7f09b81a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09b7e00700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b810c5b0 0x7f09b81a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09b7e00700 1 -- 172.21.15.90:0/2227581867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b8106ec0 msgr2=0x7f09b819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09b7e00700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b8106ec0 0x7f09b819d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.256+0000 7f09b7e00700 1 -- 172.21.15.90:0/2227581867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b81a1de0 con 0x7f09b8106570 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.257+0000 7f09b7400700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b8106ec0 0x7f09b819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.257+0000 7f09b7e00700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b8106570 0x7f09b819cf10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c8f0 tx=0x7f09a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.257+0000 7f09b4c00700 1 -- 172.21.15.90:0/2227581867 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8007b00 con 0x7f09b8106570 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.257+0000 7f09bf000700 1 -- 172.21.15.90:0/2227581867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b81a20d0 con 0x7f09b8106570 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.257+0000 7f09b4c00700 1 -- 172.21.15.90:0/2227581867 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09a8007ca0 con 0x7f09b8106570 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.257+0000 7f09b4c00700 1 -- 172.21.15.90:0/2227581867 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8018660 con 0x7f09b8106570 2024-05-27T07:42:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.257+0000 7f09bf000700 1 -- 172.21.15.90:0/2227581867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b81a99f0 con 0x7f09b8106570 2024-05-27T07:42:30.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.258+0000 7f09bf000700 1 -- 172.21.15.90:0/2227581867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b80512b0 con 0x7f09b8106570 2024-05-27T07:42:30.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.261+0000 7f09b4c00700 1 -- 172.21.15.90:0/2227581867 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f09a800f450 con 0x7f09b8106570 2024-05-27T07:42:30.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.262+0000 7f09b4c00700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f099c0b2880 0x7f099c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:30.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.263+0000 7f09b4c00700 1 -- 172.21.15.90:0/2227581867 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f09a80e2670 con 0x7f09b8106570 2024-05-27T07:42:30.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.263+0000 7f09b7400700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f099c0b2880 0x7f099c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:30.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.264+0000 7f09b7400700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f099c0b2880 0x7f099c0b4d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f09a4000c00 tx=0x7f09a4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:30.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.266+0000 7f09b4c00700 1 -- 172.21.15.90:0/2227581867 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09a8014030 con 0x7f09b8106570 2024-05-27T07:42:30.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:29 smithi129 ceph-mon[35254]: pgmap v424: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:42:30.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.549+0000 7f09bf000700 1 -- 172.21.15.90:0/2227581867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f09b819e270 con 0x7f09b8106570 2024-05-27T07:42:30.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.827+0000 7f09b4c00700 1 -- 172.21.15.90:0/2227581867 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f09a80aa060 con 0x7f09b8106570 2024-05-27T07:42:30.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.832+0000 7f09bf000700 1 -- 172.21.15.90:0/2227581867 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f099c0b2880 msgr2=0x7f099c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:30.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.832+0000 7f09bf000700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f099c0b2880 0x7f099c0b4d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f09a4000c00 tx=0x7f09a4005fb0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.832+0000 7f09bf000700 1 -- 172.21.15.90:0/2227581867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b8106570 msgr2=0x7f09b819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:30.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.832+0000 7f09bf000700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b8106570 0x7f09b819cf10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c8f0 tx=0x7f09a800ccb0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.832+0000 7f09bf000700 1 -- 172.21.15.90:0/2227581867 shutdown_connections 2024-05-27T07:42:30.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.832+0000 7f09bf000700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f099c0b2880 0x7f099c0b4d90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.832+0000 7f09bf000700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b810c5b0 0x7f09b81a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.832+0000 7f09bf000700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09b8106570 0x7f09b819cf10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.832+0000 7f09bf000700 1 --2- 172.21.15.90:0/2227581867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f09b8106ec0 0x7f09b819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:30.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.832+0000 7f09bf000700 1 -- 172.21.15.90:0/2227581867 >> 172.21.15.90:0/2227581867 conn(0x7f09b8102070 msgr2=0x7f09b8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:30.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.833+0000 7f09bf000700 1 -- 172.21.15.90:0/2227581867 shutdown_connections 2024-05-27T07:42:30.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:30.833+0000 7f09bf000700 1 -- 172.21.15.90:0/2227581867 wait complete. 2024-05-27T07:42:31.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:30 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2227581867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-27T07:42:31.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:30 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2227581867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-27T07:42:31.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:30 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2227581867' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-27T07:42:31.638 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-27T07:42:31.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:31 smithi090 ceph-mon[31429]: pgmap v425: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:42:31.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:31 smithi090 ceph-mon[31429]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T07:42:31.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:31 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2227581867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-27T07:42:31.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:31 smithi090 ceph-mon[31429]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T07:42:31.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:31 smithi090 ceph-mon[31429]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:42:31.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:31 smithi090 ceph-mon[31429]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T07:42:31.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:31 smithi090 ceph-mon[31429]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T07:42:32.085 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:32.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:31 smithi136 ceph-mon[35324]: pgmap v425: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:42:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:31 smithi136 ceph-mon[35324]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T07:42:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:31 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2227581867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-27T07:42:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:31 smithi136 ceph-mon[35324]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T07:42:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:31 smithi136 ceph-mon[35324]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:42:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:31 smithi136 ceph-mon[35324]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T07:42:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:31 smithi136 ceph-mon[35324]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T07:42:32.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:31 smithi129 ceph-mon[35254]: pgmap v425: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:42:32.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:31 smithi129 ceph-mon[35254]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T07:42:32.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:31 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2227581867' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-27T07:42:32.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:31 smithi129 ceph-mon[35254]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T07:42:32.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:31 smithi129 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T07:42:32.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:31 smithi129 ceph-mon[35254]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-27T07:42:32.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:31 smithi129 ceph-mon[35254]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-27T07:42:33.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:32 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] up:active 2024-05-27T07:42:33.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:32 smithi090 ceph-mon[31429]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T07:42:33.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:32 smithi090 ceph-mon[31429]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T07:42:33.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:32 smithi090 ceph-mon[31429]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T07:42:33.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:32 smithi090 ceph-mon[31429]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T07:42:33.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:32 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] up:active 2024-05-27T07:42:33.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:32 smithi136 ceph-mon[35324]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T07:42:33.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:32 smithi136 ceph-mon[35324]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T07:42:33.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:32 smithi136 ceph-mon[35324]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T07:42:33.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:32 smithi136 ceph-mon[35324]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T07:42:33.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:32 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] up:active 2024-05-27T07:42:33.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:32 smithi129 ceph-mon[35254]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T07:42:33.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:32 smithi129 ceph-mon[35254]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-27T07:42:33.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:32 smithi129 ceph-mon[35254]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-05-27T07:42:33.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:32 smithi129 ceph-mon[35254]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-05-27T07:42:33.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:33 smithi090 ceph-mon[31429]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-05-27T07:42:33.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:33 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] up:active 2024-05-27T07:42:33.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:33 smithi090 ceph-mon[31429]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-27T07:42:33.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:33 smithi090 ceph-mon[31429]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T07:42:33.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:33 smithi090 ceph-mon[31429]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-05-27T07:42:33.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:33 smithi090 ceph-mon[31429]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-27T07:42:34.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:33 smithi136 ceph-mon[35324]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-05-27T07:42:34.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:33 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] up:active 2024-05-27T07:42:34.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:33 smithi136 ceph-mon[35324]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-27T07:42:34.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:33 smithi136 ceph-mon[35324]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T07:42:34.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:33 smithi136 ceph-mon[35324]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-05-27T07:42:34.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:33 smithi136 ceph-mon[35324]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-27T07:42:34.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:33 smithi129 ceph-mon[35254]: pgmap v426: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 1 op/s 2024-05-27T07:42:34.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:33 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] up:active 2024-05-27T07:42:34.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:33 smithi129 ceph-mon[35254]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-27T07:42:34.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:33 smithi129 ceph-mon[35254]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-05-27T07:42:34.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:33 smithi129 ceph-mon[35254]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-05-27T07:42:34.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:33 smithi129 ceph-mon[35254]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-05-27T07:42:34.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.592+0000 7f63ff800700 1 -- 172.21.15.90:0/1214253741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63f8108f10 msgr2=0x7f63f8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:34.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.592+0000 7f63ff800700 1 --2- 172.21.15.90:0/1214253741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63f8108f10 0x7f63f8109320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f63e40099c0 tx=0x7f63e4009cd0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.592+0000 7f63ff800700 1 -- 172.21.15.90:0/1214253741 shutdown_connections 2024-05-27T07:42:34.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.592+0000 7f63ff800700 1 --2- 172.21.15.90:0/1214253741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f8109860 0x7f63f81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.592+0000 7f63ff800700 1 --2- 172.21.15.90:0/1214253741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63f81045e0 0x7f63f81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.592+0000 7f63ff800700 1 --2- 172.21.15.90:0/1214253741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63f8108f10 0x7f63f8109320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.592+0000 7f63ff800700 1 -- 172.21.15.90:0/1214253741 >> 172.21.15.90:0/1214253741 conn(0x7f63f8100030 msgr2=0x7f63f8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:34.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.592+0000 7f63ff800700 1 -- 172.21.15.90:0/1214253741 shutdown_connections 2024-05-27T07:42:34.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.592+0000 7f63ff800700 1 -- 172.21.15.90:0/1214253741 wait complete. 2024-05-27T07:42:34.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.594+0000 7f63ff800700 1 Processor -- start 2024-05-27T07:42:34.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.594+0000 7f63ff800700 1 -- start start 2024-05-27T07:42:34.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.594+0000 7f63ff800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63f81045e0 0x7f63f819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:34.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63ff800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63f8108f10 0x7f63f819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:34.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63ff800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f8109860 0x7f63f81a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:34.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63ff800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f63f8114370 con 0x7f63f8108f10 2024-05-27T07:42:34.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63ff800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81141f0 con 0x7f63f8109860 2024-05-27T07:42:34.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63ff800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63f81144f0 con 0x7f63f81045e0 2024-05-27T07:42:34.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63f8108f10 0x7f63f819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:34.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63f8108f10 0x7f63f819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59746/0 (socket says 172.21.15.90:59746) 2024-05-27T07:42:34.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63f7e00700 1 -- 172.21.15.90:0/2994411544 learned_addr learned my addr 172.21.15.90:0/2994411544 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63fce00700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63f81045e0 0x7f63f819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63fd800700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f8109860 0x7f63f81a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63f7e00700 1 -- 172.21.15.90:0/2994411544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63f81045e0 msgr2=0x7f63f819d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.595+0000 7f63f7e00700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63f81045e0 0x7f63f819d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.596+0000 7f63f7e00700 1 -- 172.21.15.90:0/2994411544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f8109860 msgr2=0x7f63f81a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.596+0000 7f63f7e00700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f8109860 0x7f63f81a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.596+0000 7f63f7e00700 1 -- 172.21.15.90:0/2994411544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f81a1f10 con 0x7f63f8108f10 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.596+0000 7f63f7e00700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63f8108f10 0x7f63f819d5e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f63e40055f0 tx=0x7f63e40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.596+0000 7f63f5600700 1 -- 172.21.15.90:0/2994411544 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4026070 con 0x7f63f8108f10 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.596+0000 7f63f5600700 1 -- 172.21.15.90:0/2994411544 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63e400f460 con 0x7f63f8108f10 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.596+0000 7f63ff800700 1 -- 172.21.15.90:0/2994411544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f81a21a0 con 0x7f63f8108f10 2024-05-27T07:42:34.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.596+0000 7f63f5600700 1 -- 172.21.15.90:0/2994411544 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4009e60 con 0x7f63f8108f10 2024-05-27T07:42:34.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.596+0000 7f63ff800700 1 -- 172.21.15.90:0/2994411544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f81a2680 con 0x7f63f8108f10 2024-05-27T07:42:34.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.598+0000 7f63ff800700 1 -- 172.21.15.90:0/2994411544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f80512b0 con 0x7f63f8108f10 2024-05-27T07:42:34.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.600+0000 7f63f5600700 1 -- 172.21.15.90:0/2994411544 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f63e4029070 con 0x7f63f8108f10 2024-05-27T07:42:34.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.602+0000 7f63f5600700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f63e00b2880 0x7f63e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:34.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.603+0000 7f63f5600700 1 -- 172.21.15.90:0/2994411544 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f63e40e3390 con 0x7f63f8108f10 2024-05-27T07:42:34.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.603+0000 7f63fce00700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f63e00b2880 0x7f63e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:34.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.603+0000 7f63fce00700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f63e00b2880 0x7f63e00b4d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009d10 tx=0x7f63ec009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:34.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.605+0000 7f63f5600700 1 -- 172.21.15.90:0/2994411544 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f63e4030080 con 0x7f63f8108f10 2024-05-27T07:42:34.878 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 07:42:34 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b[65300]: 2024-05-27T07:42:34.699+0000 7f8484000700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-27T07:42:34.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.888+0000 7f63ff800700 1 -- 172.21.15.90:0/2994411544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f63f81a9b20 con 0x7f63f8108f10 2024-05-27T07:42:34.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.901+0000 7f63f5600700 1 -- 172.21.15.90:0/2994411544 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7f63e4013070 con 0x7f63f8108f10 2024-05-27T07:42:34.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 -- 172.21.15.90:0/2994411544 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f63e00b2880 msgr2=0x7f63e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:34.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f63e00b2880 0x7f63e00b4d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009d10 tx=0x7f63ec009040 comp rx=0 tx=0).stop 2024-05-27T07:42:34.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 -- 172.21.15.90:0/2994411544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63f8108f10 msgr2=0x7f63f819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:34.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63f8108f10 0x7f63f819d5e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f63e40055f0 tx=0x7f63e40056d0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 -- 172.21.15.90:0/2994411544 shutdown_connections 2024-05-27T07:42:34.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f63e00b2880 0x7f63e00b4d90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63f8109860 0x7f63f81a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63f81045e0 0x7f63f819d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 --2- 172.21.15.90:0/2994411544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63f8108f10 0x7f63f819d5e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:34.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 -- 172.21.15.90:0/2994411544 >> 172.21.15.90:0/2994411544 conn(0x7f63f8100030 msgr2=0x7f63f810a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:34.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 -- 172.21.15.90:0/2994411544 shutdown_connections 2024-05-27T07:42:34.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:34.906+0000 7f63ff800700 1 -- 172.21.15.90:0/2994411544 wait complete. 2024-05-27T07:42:35.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:34 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] up:active 2024-05-27T07:42:35.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:34 smithi090 ceph-mon[31429]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-27T07:42:35.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:34 smithi090 ceph-mon[31429]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T07:42:35.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:34 smithi090 ceph-mon[31429]: Cluster is now healthy 2024-05-27T07:42:35.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:34 smithi090 ceph-mon[31429]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-05-27T07:42:35.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:34 smithi090 ceph-mon[31429]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-27T07:42:35.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:34 smithi090 ceph-mon[31429]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-05-27T07:42:35.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:34 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] up:active 2024-05-27T07:42:35.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:34 smithi136 ceph-mon[35324]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-27T07:42:35.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:34 smithi136 ceph-mon[35324]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T07:42:35.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:34 smithi136 ceph-mon[35324]: Cluster is now healthy 2024-05-27T07:42:35.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:34 smithi136 ceph-mon[35324]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-05-27T07:42:35.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:34 smithi136 ceph-mon[35324]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-27T07:42:35.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:34 smithi136 ceph-mon[35324]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-05-27T07:42:35.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:34 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] up:active 2024-05-27T07:42:35.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:34 smithi129 ceph-mon[35254]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-27T07:42:35.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:34 smithi129 ceph-mon[35254]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T07:42:35.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:34 smithi129 ceph-mon[35254]: Cluster is now healthy 2024-05-27T07:42:35.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:34 smithi129 ceph-mon[35254]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-05-27T07:42:35.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:34 smithi129 ceph-mon[35254]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-27T07:42:35.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:34 smithi129 ceph-mon[35254]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-05-27T07:42:35.706 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-27T07:42:35.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:35 smithi090 ceph-mon[31429]: pgmap v427: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 3 op/s 2024-05-27T07:42:35.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:35 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2994411544' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T07:42:35.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:35 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] up:active 2024-05-27T07:42:35.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:35 smithi090 ceph-mon[31429]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T07:42:35.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:35 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2994411544' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T07:42:35.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:35 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:42:36.150 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:36.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:35 smithi136 ceph-mon[35324]: pgmap v427: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 3 op/s 2024-05-27T07:42:36.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:35 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2994411544' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T07:42:36.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:35 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] up:active 2024-05-27T07:42:36.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:35 smithi136 ceph-mon[35324]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T07:42:36.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:35 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2994411544' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T07:42:36.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:35 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:42:36.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:35 smithi129 ceph-mon[35254]: pgmap v427: 129 pgs: 129 active+clean; 581 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 3 op/s 2024-05-27T07:42:36.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:35 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2994411544' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T07:42:36.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:35 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] up:active 2024-05-27T07:42:36.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:35 smithi129 ceph-mon[35254]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T07:42:36.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:35 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2994411544' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T07:42:36.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:35 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:42:38.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:37 smithi090 ceph-mon[31429]: pgmap v428: 129 pgs: 129 active+clean; 585 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.0 KiB/s wr, 11 op/s 2024-05-27T07:42:38.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:37 smithi136 ceph-mon[35324]: pgmap v428: 129 pgs: 129 active+clean; 585 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.0 KiB/s wr, 11 op/s 2024-05-27T07:42:38.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:37 smithi129 ceph-mon[35254]: pgmap v428: 129 pgs: 129 active+clean; 585 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 3.0 KiB/s wr, 11 op/s 2024-05-27T07:42:38.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.491+0000 7f85bc800700 1 -- 172.21.15.90:0/1226974432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b4109860 msgr2=0x7f85b41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.491+0000 7f85bc800700 1 --2- 172.21.15.90:0/1226974432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b4109860 0x7f85b41111c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f85b000b0d0 tx=0x7f85b000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.492+0000 7f85bc800700 1 -- 172.21.15.90:0/1226974432 shutdown_connections 2024-05-27T07:42:38.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.492+0000 7f85bc800700 1 --2- 172.21.15.90:0/1226974432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b4109860 0x7f85b41111c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.492+0000 7f85bc800700 1 --2- 172.21.15.90:0/1226974432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b41045e0 0x7f85b41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.492+0000 7f85bc800700 1 --2- 172.21.15.90:0/1226974432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b4108f10 0x7f85b4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.492+0000 7f85bc800700 1 -- 172.21.15.90:0/1226974432 >> 172.21.15.90:0/1226974432 conn(0x7f85b4100030 msgr2=0x7f85b4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:38.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.492+0000 7f85bc800700 1 -- 172.21.15.90:0/1226974432 shutdown_connections 2024-05-27T07:42:38.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.492+0000 7f85bc800700 1 -- 172.21.15.90:0/1226974432 wait complete. 2024-05-27T07:42:38.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.493+0000 7f85bc800700 1 Processor -- start 2024-05-27T07:42:38.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.493+0000 7f85bc800700 1 -- start start 2024-05-27T07:42:38.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.493+0000 7f85bc800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b41045e0 0x7f85b419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.493+0000 7f85bc800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b4108f10 0x7f85b419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.493+0000 7f85bc800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b4109860 0x7f85b419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.493+0000 7f85bc800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f85b4114390 con 0x7f85b4108f10 2024-05-27T07:42:38.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.493+0000 7f85bc800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f85b4114210 con 0x7f85b4109860 2024-05-27T07:42:38.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.493+0000 7f85bc800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85b4114510 con 0x7f85b41045e0 2024-05-27T07:42:38.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.494+0000 7f85b9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b4108f10 0x7f85b419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.494+0000 7f85b9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b4108f10 0x7f85b419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59764/0 (socket says 172.21.15.90:59764) 2024-05-27T07:42:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.494+0000 7f85b9400700 1 -- 172.21.15.90:0/1821429696 learned_addr learned my addr 172.21.15.90:0/1821429696 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.494+0000 7f85ba800700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b4109860 0x7f85b419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.494+0000 7f85b9e00700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b41045e0 0x7f85b419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.495+0000 7f85b9400700 1 -- 172.21.15.90:0/1821429696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b41045e0 msgr2=0x7f85b419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.495+0000 7f85b9400700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b41045e0 0x7f85b419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.495+0000 7f85b9400700 1 -- 172.21.15.90:0/1821429696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b4109860 msgr2=0x7f85b419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.495+0000 7f85b9400700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b4109860 0x7f85b419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.495+0000 7f85b9400700 1 -- 172.21.15.90:0/1821429696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b419fc30 con 0x7f85b4108f10 2024-05-27T07:42:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.495+0000 7f85b9400700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b4108f10 0x7f85b419b300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c8f0 tx=0x7f85a400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.495+0000 7f85aea00700 1 -- 172.21.15.90:0/1821429696 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a40078d0 con 0x7f85b4108f10 2024-05-27T07:42:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.495+0000 7f85aea00700 1 -- 172.21.15.90:0/1821429696 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f85a400f450 con 0x7f85b4108f10 2024-05-27T07:42:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.495+0000 7f85aea00700 1 -- 172.21.15.90:0/1821429696 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4018660 con 0x7f85b4108f10 2024-05-27T07:42:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.495+0000 7f85bc800700 1 -- 172.21.15.90:0/1821429696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b419ff20 con 0x7f85b4108f10 2024-05-27T07:42:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.496+0000 7f85bc800700 1 -- 172.21.15.90:0/1821429696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b41a7840 con 0x7f85b4108f10 2024-05-27T07:42:38.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.497+0000 7f85bc800700 1 -- 172.21.15.90:0/1821429696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b40512b0 con 0x7f85b4108f10 2024-05-27T07:42:38.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.500+0000 7f85aea00700 1 -- 172.21.15.90:0/1821429696 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f85a400f900 con 0x7f85b4108f10 2024-05-27T07:42:38.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.502+0000 7f85aea00700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f859c0b2900 0x7f859c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:38.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.502+0000 7f85aea00700 1 -- 172.21.15.90:0/1821429696 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f85a40e23a0 con 0x7f85b4108f10 2024-05-27T07:42:38.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.502+0000 7f85b9e00700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f859c0b2900 0x7f859c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:38.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.503+0000 7f85b9e00700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f859c0b2900 0x7f859c0b4e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f85a8005950 tx=0x7f85a800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:38.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.504+0000 7f85aea00700 1 -- 172.21.15.90:0/1821429696 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85a4014030 con 0x7f85b4108f10 2024-05-27T07:42:38.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.786+0000 7f85bc800700 1 -- 172.21.15.90:0/1821429696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f85b402e5e0 con 0x7f85b4108f10 2024-05-27T07:42:38.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.902+0000 7f85aea00700 1 -- 172.21.15.90:0/1821429696 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f85a40a9e10 con 0x7f85b4108f10 2024-05-27T07:42:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.906+0000 7f85bc800700 1 -- 172.21.15.90:0/1821429696 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f859c0b2900 msgr2=0x7f859c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.906+0000 7f85bc800700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f859c0b2900 0x7f859c0b4e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f85a8005950 tx=0x7f85a800a300 comp rx=0 tx=0).stop 2024-05-27T07:42:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.906+0000 7f85bc800700 1 -- 172.21.15.90:0/1821429696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b4108f10 msgr2=0x7f85b419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.906+0000 7f85bc800700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b4108f10 0x7f85b419b300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f85a400c8f0 tx=0x7f85a400cc00 comp rx=0 tx=0).stop 2024-05-27T07:42:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.907+0000 7f85bc800700 1 -- 172.21.15.90:0/1821429696 shutdown_connections 2024-05-27T07:42:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.907+0000 7f85bc800700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f859c0b2900 0x7f859c0b4e10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.907+0000 7f85bc800700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85b4109860 0x7f85b419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.907+0000 7f85bc800700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b41045e0 0x7f85b419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.907+0000 7f85bc800700 1 --2- 172.21.15.90:0/1821429696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85b4108f10 0x7f85b419b300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.907+0000 7f85bc800700 1 -- 172.21.15.90:0/1821429696 >> 172.21.15.90:0/1821429696 conn(0x7f85b4100030 msgr2=0x7f85b4108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:38.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.907+0000 7f85bc800700 1 -- 172.21.15.90:0/1821429696 shutdown_connections 2024-05-27T07:42:38.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:38.907+0000 7f85bc800700 1 -- 172.21.15.90:0/1821429696 wait complete. 2024-05-27T07:42:39.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:38 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1821429696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T07:42:39.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:38 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1821429696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T07:42:39.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:38 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1821429696' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T07:42:39.720 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-27T07:42:39.721 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-27T07:42:40.168 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:40.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:39 smithi090 ceph-mon[31429]: pgmap v429: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-27T07:42:40.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:39 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1821429696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T07:42:40.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:39 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:42:40.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:39 smithi136 ceph-mon[35324]: pgmap v429: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-27T07:42:40.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:39 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1821429696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T07:42:40.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:39 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:42:40.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:39 smithi129 ceph-mon[35254]: pgmap v429: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-05-27T07:42:40.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:39 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1821429696' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T07:42:40.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:39 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:42:42.055 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:41 smithi090 ceph-mon[31429]: pgmap v430: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-27T07:42:42.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:41 smithi136 ceph-mon[35324]: pgmap v430: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-27T07:42:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.300+0000 7f407f200700 1 -- 172.21.15.90:0/3550475952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4080108d20 msgr2=0x7f4080109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.300+0000 7f407f200700 1 --2- 172.21.15.90:0/3550475952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4080108d20 0x7f4080109130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f406c0099c0 tx=0x7f406c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:42:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.300+0000 7f407f200700 1 -- 172.21.15.90:0/3550475952 shutdown_connections 2024-05-27T07:42:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.300+0000 7f407f200700 1 --2- 172.21.15.90:0/3550475952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4080109670 0x7f4080110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.300+0000 7f407f200700 1 --2- 172.21.15.90:0/3550475952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40801043f0 0x7f4080104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.300+0000 7f407f200700 1 --2- 172.21.15.90:0/3550475952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4080108d20 0x7f4080109130 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.300+0000 7f407f200700 1 -- 172.21.15.90:0/3550475952 >> 172.21.15.90:0/3550475952 conn(0x7f4080079d00 msgr2=0x7f408007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.300+0000 7f407f200700 1 -- 172.21.15.90:0/3550475952 shutdown_connections 2024-05-27T07:42:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.300+0000 7f407f200700 1 -- 172.21.15.90:0/3550475952 wait complete. 2024-05-27T07:42:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.301+0000 7f407f200700 1 Processor -- start 2024-05-27T07:42:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.301+0000 7f407f200700 1 -- start start 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.302+0000 7f407f200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40801043f0 0x7f40801a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.302+0000 7f407f200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4080108d20 0x7f40801a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.302+0000 7f407f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4080109670 0x7f408019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.302+0000 7f407f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4080114130 con 0x7f4080109670 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.302+0000 7f407f200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4080113fb0 con 0x7f40801043f0 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.302+0000 7f407f200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f40801142b0 con 0x7f4080108d20 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.302+0000 7f407e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4080109670 0x7f408019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.302+0000 7f407e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4080109670 0x7f408019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60626/0 (socket says 172.21.15.90:60626) 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.302+0000 7f407e800700 1 -- 172.21.15.90:0/1793005024 learned_addr learned my addr 172.21.15.90:0/1793005024 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.302+0000 7f407de00700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40801043f0 0x7f40801a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f407d400700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4080108d20 0x7f40801a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:42.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f407e800700 1 -- 172.21.15.90:0/1793005024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4080108d20 msgr2=0x7f40801a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:42.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f407e800700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4080108d20 0x7f40801a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:42.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f407e800700 1 -- 172.21.15.90:0/1793005024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40801043f0 msgr2=0x7f40801a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:42.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f407e800700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40801043f0 0x7f40801a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:42.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f407e800700 1 -- 172.21.15.90:0/1793005024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408019b0c0 con 0x7f4080109670 2024-05-27T07:42:42.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f407e800700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4080109670 0x7f408019aa10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f407800c970 tx=0x7f407800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:42.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f4076a00700 1 -- 172.21.15.90:0/1793005024 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078007b00 con 0x7f4080109670 2024-05-27T07:42:42.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f4076a00700 1 -- 172.21.15.90:0/1793005024 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4078004cc0 con 0x7f4080109670 2024-05-27T07:42:42.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f4076a00700 1 -- 172.21.15.90:0/1793005024 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40780054c0 con 0x7f4080109670 2024-05-27T07:42:42.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.303+0000 7f407f200700 1 -- 172.21.15.90:0/1793005024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408019b3b0 con 0x7f4080109670 2024-05-27T07:42:42.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.304+0000 7f407f200700 1 -- 172.21.15.90:0/1793005024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40801a7680 con 0x7f4080109670 2024-05-27T07:42:42.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.305+0000 7f407f200700 1 -- 172.21.15.90:0/1793005024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40800512b0 con 0x7f4080109670 2024-05-27T07:42:42.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.307+0000 7f4076a00700 1 -- 172.21.15.90:0/1793005024 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4078004750 con 0x7f4080109670 2024-05-27T07:42:42.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.309+0000 7f4076a00700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f40640b2880 0x7f40640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:42.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.309+0000 7f4076a00700 1 -- 172.21.15.90:0/1793005024 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f40780e2130 con 0x7f4080109670 2024-05-27T07:42:42.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.309+0000 7f407de00700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f40640b2880 0x7f40640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:42.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.310+0000 7f407de00700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f40640b2880 0x7f40640b4d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f408019bb50 tx=0x7f4070009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:42.311 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:41 smithi129 ceph-mon[35254]: pgmap v430: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-27T07:42:42.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.316+0000 7f4076a00700 1 -- 172.21.15.90:0/1793005024 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4078014030 con 0x7f4080109670 2024-05-27T07:42:42.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.577+0000 7f407f200700 1 -- 172.21.15.90:0/1793005024 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f408019bf40 con 0x7f40640b2880 2024-05-27T07:42:42.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.659+0000 7f4076a00700 1 -- 172.21.15.90:0/1793005024 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f408019bf40 con 0x7f40640b2880 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.662+0000 7f407f200700 1 -- 172.21.15.90:0/1793005024 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f40640b2880 msgr2=0x7f40640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.662+0000 7f407f200700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f40640b2880 0x7f40640b4d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f408019bb50 tx=0x7f4070009040 comp rx=0 tx=0).stop 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.662+0000 7f407f200700 1 -- 172.21.15.90:0/1793005024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4080109670 msgr2=0x7f408019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.662+0000 7f407f200700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4080109670 0x7f408019aa10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f407800c970 tx=0x7f407800cd30 comp rx=0 tx=0).stop 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.663+0000 7f407f200700 1 -- 172.21.15.90:0/1793005024 shutdown_connections 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.663+0000 7f407f200700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f40640b2880 0x7f40640b4d90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.663+0000 7f407f200700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4080109670 0x7f408019aa10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.663+0000 7f407f200700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f40801043f0 0x7f40801a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.663+0000 7f407f200700 1 --2- 172.21.15.90:0/1793005024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4080108d20 0x7f40801a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.663+0000 7f407f200700 1 -- 172.21.15.90:0/1793005024 >> 172.21.15.90:0/1793005024 conn(0x7f4080079d00 msgr2=0x7f4080101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:42.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.663+0000 7f407f200700 1 -- 172.21.15.90:0/1793005024 shutdown_connections 2024-05-27T07:42:42.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:42.663+0000 7f407f200700 1 -- 172.21.15.90:0/1793005024 wait complete. 2024-05-27T07:42:42.962 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:42:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:42:42.592+0000 7f1c20e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:42:42.962 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:42:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:42:42.592+0000 7f1c20e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:42:42.962 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:42:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:42:42.592+0000 7f1c20e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:42:42.962 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:42:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:42:42.592+0000 7f1c20e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:42:42.962 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:42:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:42:42.592+0000 7f1c20e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:42:43.496 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-27T07:42:43.964 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:44.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:43 smithi090 ceph-mon[31429]: pgmap v431: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-05-27T07:42:44.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:43 smithi090 ceph-mon[31429]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:44.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:43 smithi136 ceph-mon[35324]: pgmap v431: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-05-27T07:42:44.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:43 smithi136 ceph-mon[35324]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:44.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:43 smithi129 ceph-mon[35254]: pgmap v431: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-05-27T07:42:44.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:43 smithi129 ceph-mon[35254]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.125+0000 7fee77400700 1 -- 172.21.15.90:0/3639035645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee7010e8e0 msgr2=0x7fee70110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.125+0000 7fee77400700 1 --2- 172.21.15.90:0/3639035645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee7010e8e0 0x7fee70110dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fee6800b0a0 tx=0x7fee6800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.125+0000 7fee77400700 1 -- 172.21.15.90:0/3639035645 shutdown_connections 2024-05-27T07:42:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.125+0000 7fee77400700 1 --2- 172.21.15.90:0/3639035645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee7010e8e0 0x7fee70110dc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.125+0000 7fee77400700 1 --2- 172.21.15.90:0/3639035645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee70105470 0x7fee70105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.125+0000 7fee77400700 1 --2- 172.21.15.90:0/3639035645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee70105dc0 0x7fee7010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.125+0000 7fee77400700 1 -- 172.21.15.90:0/3639035645 >> 172.21.15.90:0/3639035645 conn(0x7fee700fef20 msgr2=0x7fee70101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.125+0000 7fee77400700 1 -- 172.21.15.90:0/3639035645 shutdown_connections 2024-05-27T07:42:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.126+0000 7fee77400700 1 -- 172.21.15.90:0/3639035645 wait complete. 2024-05-27T07:42:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:45 smithi090 ceph-mon[31429]: pgmap v432: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-27T07:42:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:45 smithi090 ceph-mon[31429]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.127+0000 7fee77400700 1 Processor -- start 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.127+0000 7fee77400700 1 -- start start 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.128+0000 7fee77400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee70105470 0x7fee7019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.128+0000 7fee77400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee70105dc0 0x7fee7019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.128+0000 7fee77400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee7010e8e0 0x7fee701a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.128+0000 7fee77400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fee701142f0 con 0x7fee70105dc0 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.128+0000 7fee77400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fee70114170 con 0x7fee7010e8e0 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.128+0000 7fee77400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee70114470 con 0x7fee70105470 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.128+0000 7fee6fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee70105dc0 0x7fee7019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.128+0000 7fee6fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee70105dc0 0x7fee7019d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60672/0 (socket says 172.21.15.90:60672) 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.128+0000 7fee6fe00700 1 -- 172.21.15.90:0/3763721910 learned_addr learned my addr 172.21.15.90:0/3763721910 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:46.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.128+0000 7fee74a00700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee70105470 0x7fee7019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:46.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.129+0000 7fee75400700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee7010e8e0 0x7fee701a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:46.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.129+0000 7fee6fe00700 1 -- 172.21.15.90:0/3763721910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee70105470 msgr2=0x7fee7019cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:46.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.129+0000 7fee6fe00700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee70105470 0x7fee7019cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.129+0000 7fee6fe00700 1 -- 172.21.15.90:0/3763721910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee7010e8e0 msgr2=0x7fee701a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:46.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.129+0000 7fee6fe00700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee7010e8e0 0x7fee701a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.129+0000 7fee6fe00700 1 -- 172.21.15.90:0/3763721910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee701a1db0 con 0x7fee70105dc0 2024-05-27T07:42:46.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.129+0000 7fee74a00700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee70105470 0x7fee7019cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.129+0000 7fee6fe00700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee70105dc0 0x7fee7019d480 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00d950 tx=0x7fee5c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.129+0000 7fee6d600700 1 -- 172.21.15.90:0/3763721910 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c016070 con 0x7fee70105dc0 2024-05-27T07:42:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.130+0000 7fee6d600700 1 -- 172.21.15.90:0/3763721910 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee5c010460 con 0x7fee70105dc0 2024-05-27T07:42:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.130+0000 7fee77400700 1 -- 172.21.15.90:0/3763721910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee701a20a0 con 0x7fee70105dc0 2024-05-27T07:42:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.130+0000 7fee6d600700 1 -- 172.21.15.90:0/3763721910 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c00f620 con 0x7fee70105dc0 2024-05-27T07:42:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.130+0000 7fee77400700 1 -- 172.21.15.90:0/3763721910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee701a99c0 con 0x7fee70105dc0 2024-05-27T07:42:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.131+0000 7fee77400700 1 -- 172.21.15.90:0/3763721910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee700512b0 con 0x7fee70105dc0 2024-05-27T07:42:46.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.134+0000 7fee6d600700 1 -- 172.21.15.90:0/3763721910 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fee5c019070 con 0x7fee70105dc0 2024-05-27T07:42:46.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.135+0000 7fee6d600700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fee580b2990 0x7fee580b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:46.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.135+0000 7fee6d600700 1 -- 172.21.15.90:0/3763721910 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fee5c0e2290 con 0x7fee70105dc0 2024-05-27T07:42:46.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.135+0000 7fee74a00700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fee580b2990 0x7fee580b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:46.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.136+0000 7fee74a00700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fee580b2990 0x7fee580b4ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee64005950 tx=0x7fee640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:46.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.138+0000 7fee6d600700 1 -- 172.21.15.90:0/3763721910 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee5c0a9a90 con 0x7fee70105dc0 2024-05-27T07:42:46.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:45 smithi136 ceph-mon[35324]: pgmap v432: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-27T07:42:46.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:45 smithi136 ceph-mon[35324]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-27T07:42:46.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:45 smithi129 ceph-mon[35254]: pgmap v432: 129 pgs: 129 active+clean; 587 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-27T07:42:46.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:45 smithi129 ceph-mon[35254]: mgrmap e17: x(active, since 11m), standbys: z, y 2024-05-27T07:42:46.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.401+0000 7fee77400700 1 -- 172.21.15.90:0/3763721910 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fee7019e240 con 0x7fee580b2990 2024-05-27T07:42:46.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.478+0000 7fee6d600700 1 -- 172.21.15.90:0/3763721910 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fee7019e240 con 0x7fee580b2990 2024-05-27T07:42:46.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 -- 172.21.15.90:0/3763721910 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fee580b2990 msgr2=0x7fee580b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:46.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fee580b2990 0x7fee580b4ea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee64005950 tx=0x7fee640058e0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 -- 172.21.15.90:0/3763721910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee70105dc0 msgr2=0x7fee7019d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:46.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee70105dc0 0x7fee7019d480 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00d950 tx=0x7fee5c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:42:46.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 -- 172.21.15.90:0/3763721910 shutdown_connections 2024-05-27T07:42:46.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fee580b2990 0x7fee580b4ea0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee7010e8e0 0x7fee701a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee70105470 0x7fee7019cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 --2- 172.21.15.90:0/3763721910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee70105dc0 0x7fee7019d480 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:46.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 -- 172.21.15.90:0/3763721910 >> 172.21.15.90:0/3763721910 conn(0x7fee700fef20 msgr2=0x7fee70103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:46.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.481+0000 7fee77400700 1 -- 172.21.15.90:0/3763721910 shutdown_connections 2024-05-27T07:42:46.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:46.482+0000 7fee77400700 1 -- 172.21.15.90:0/3763721910 wait complete. 2024-05-27T07:42:47.273 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-27T07:42:47.721 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:48.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:47 smithi090 ceph-mon[31429]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:48.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:47 smithi090 ceph-mon[31429]: pgmap v433: 129 pgs: 129 active+clean; 594 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-05-27T07:42:48.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:47 smithi136 ceph-mon[35324]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:48.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:47 smithi136 ceph-mon[35324]: pgmap v433: 129 pgs: 129 active+clean; 594 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-05-27T07:42:48.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:47 smithi129 ceph-mon[35254]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:48.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:47 smithi129 ceph-mon[35254]: pgmap v433: 129 pgs: 129 active+clean; 594 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-05-27T07:42:50.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.181+0000 7f8629200700 1 -- 172.21.15.90:0/1538377247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f862410b100 msgr2=0x7f8624117610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:50.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.181+0000 7f8629200700 1 --2- 172.21.15.90:0/1538377247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f862410b100 0x7f8624117610 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f861c00b0a0 tx=0x7f861c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:42:50.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.181+0000 7f8629200700 1 -- 172.21.15.90:0/1538377247 shutdown_connections 2024-05-27T07:42:50.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.181+0000 7f8629200700 1 --2- 172.21.15.90:0/1538377247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f862410b100 0x7f8624117610 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:50.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.181+0000 7f8629200700 1 --2- 172.21.15.90:0/1538377247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f862410f6b0 0x7f862410fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:50.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.181+0000 7f8629200700 1 --2- 172.21.15.90:0/1538377247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f862410a7b0 0x7f862410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:50.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.181+0000 7f8629200700 1 -- 172.21.15.90:0/1538377247 >> 172.21.15.90:0/1538377247 conn(0x7f8624076af0 msgr2=0x7f8624076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:50.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.181+0000 7f8629200700 1 -- 172.21.15.90:0/1538377247 shutdown_connections 2024-05-27T07:42:50.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.181+0000 7f8629200700 1 -- 172.21.15.90:0/1538377247 wait complete. 2024-05-27T07:42:50.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:49 smithi090 ceph-mon[31429]: pgmap v434: 129 pgs: 129 active+clean; 594 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-05-27T07:42:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.182+0000 7f8629200700 1 Processor -- start 2024-05-27T07:42:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.183+0000 7f8629200700 1 -- start start 2024-05-27T07:42:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.183+0000 7f8629200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f862410a7b0 0x7f86241a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.183+0000 7f8629200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f862410b100 0x7f86241a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.183+0000 7f8629200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f862410f6b0 0x7f86241a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.183+0000 7f8629200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f862411a790 con 0x7f862410f6b0 2024-05-27T07:42:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.183+0000 7f8629200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f862411a610 con 0x7f862410b100 2024-05-27T07:42:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.183+0000 7f8629200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f862411a910 con 0x7f862410a7b0 2024-05-27T07:42:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f862410f6b0 0x7f86241a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f862410f6b0 0x7f86241a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39066/0 (socket says 172.21.15.90:39066) 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623e00700 1 -- 172.21.15.90:0/3910331490 learned_addr learned my addr 172.21.15.90:0/3910331490 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623400700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f862410a7b0 0x7f86241a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8622a00700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f862410b100 0x7f86241a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623e00700 1 -- 172.21.15.90:0/3910331490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f862410a7b0 msgr2=0x7f86241a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623e00700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f862410a7b0 0x7f86241a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623e00700 1 -- 172.21.15.90:0/3910331490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f862410b100 msgr2=0x7f86241a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623e00700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f862410b100 0x7f86241a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623e00700 1 -- 172.21.15.90:0/3910331490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86241a5f60 con 0x7f862410f6b0 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623400700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f862410a7b0 0x7f86241a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.184+0000 7f8623e00700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f862410f6b0 0x7f86241a5970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f861c00b070 tx=0x7f861c004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.185+0000 7f861be00700 1 -- 172.21.15.90:0/3910331490 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c021070 con 0x7f862410f6b0 2024-05-27T07:42:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.185+0000 7f8629200700 1 -- 172.21.15.90:0/3910331490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86241a61f0 con 0x7f862410f6b0 2024-05-27T07:42:50.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.185+0000 7f861be00700 1 -- 172.21.15.90:0/3910331490 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f861c007c00 con 0x7f862410f6b0 2024-05-27T07:42:50.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.185+0000 7f861be00700 1 -- 172.21.15.90:0/3910331490 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c004860 con 0x7f862410f6b0 2024-05-27T07:42:50.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.185+0000 7f8629200700 1 -- 172.21.15.90:0/3910331490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86241a6720 con 0x7f862410f6b0 2024-05-27T07:42:50.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.186+0000 7f8629200700 1 -- 172.21.15.90:0/3910331490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86240512b0 con 0x7f862410f6b0 2024-05-27T07:42:50.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.189+0000 7f861be00700 1 -- 172.21.15.90:0/3910331490 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f861c004000 con 0x7f862410f6b0 2024-05-27T07:42:50.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.192+0000 7f861be00700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f86080b2910 0x7f86080b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:50.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.192+0000 7f861be00700 1 -- 172.21.15.90:0/3910331490 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f861c00f020 con 0x7f862410f6b0 2024-05-27T07:42:50.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.192+0000 7f8623400700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f86080b2910 0x7f86080b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:50.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.193+0000 7f8623400700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f86080b2910 0x7f86080b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8614005e10 tx=0x7f8614005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:50.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.195+0000 7f861be00700 1 -- 172.21.15.90:0/3910331490 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f861c02a020 con 0x7f862410f6b0 2024-05-27T07:42:50.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:49 smithi136 ceph-mon[35324]: pgmap v434: 129 pgs: 129 active+clean; 594 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-05-27T07:42:50.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:49 smithi129 ceph-mon[35254]: pgmap v434: 129 pgs: 129 active+clean; 594 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-05-27T07:42:50.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.461+0000 7f8629200700 1 -- 172.21.15.90:0/3910331490 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f86241a24a0 con 0x7f86080b2910 2024-05-27T07:42:50.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.474+0000 7f861be00700 1 -- 172.21.15.90:0/3910331490 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f86241a24a0 con 0x7f86080b2910 2024-05-27T07:42:50.472 INFO:teuthology.orchestra.run.smithi090.stdout:/volumes/qa/sv_0/a9b7f4ae-1094-46c7-b823-ec4601e37cd2 2024-05-27T07:42:50.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.477+0000 7f8629200700 1 -- 172.21.15.90:0/3910331490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f86080b2910 msgr2=0x7f86080b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:50.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.477+0000 7f8629200700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f86080b2910 0x7f86080b4e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8614005e10 tx=0x7f8614005d60 comp rx=0 tx=0).stop 2024-05-27T07:42:50.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.477+0000 7f8629200700 1 -- 172.21.15.90:0/3910331490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f862410f6b0 msgr2=0x7f86241a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:50.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.477+0000 7f8629200700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f862410f6b0 0x7f86241a5970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f861c00b070 tx=0x7f861c004e00 comp rx=0 tx=0).stop 2024-05-27T07:42:50.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.478+0000 7f8629200700 1 -- 172.21.15.90:0/3910331490 shutdown_connections 2024-05-27T07:42:50.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.478+0000 7f8629200700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f86080b2910 0x7f86080b4e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:50.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.478+0000 7f8629200700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f862410f6b0 0x7f86241a5970 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:50.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.478+0000 7f8629200700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f862410a7b0 0x7f86241a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:50.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.478+0000 7f8629200700 1 --2- 172.21.15.90:0/3910331490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f862410b100 0x7f86241a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:50.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.478+0000 7f8629200700 1 -- 172.21.15.90:0/3910331490 >> 172.21.15.90:0/3910331490 conn(0x7f8624076af0 msgr2=0x7f86241088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:50.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.478+0000 7f8629200700 1 -- 172.21.15.90:0/3910331490 shutdown_connections 2024-05-27T07:42:50.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:50.478+0000 7f8629200700 1 -- 172.21.15.90:0/3910331490 wait complete. 2024-05-27T07:42:51.395 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-27T07:42:51.833 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:52.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:51 smithi090 ceph-mon[31429]: pgmap v435: 129 pgs: 129 active+clean; 613 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-05-27T07:42:52.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:51 smithi090 ceph-mon[31429]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:52.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:51 smithi136 ceph-mon[35324]: pgmap v435: 129 pgs: 129 active+clean; 613 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-05-27T07:42:52.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:51 smithi136 ceph-mon[35324]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:52.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:51 smithi129 ceph-mon[35254]: pgmap v435: 129 pgs: 129 active+clean; 613 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.0 KiB/s wr, 14 op/s 2024-05-27T07:42:52.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:51 smithi129 ceph-mon[35254]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:54.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.198+0000 7f7c52c00700 1 -- 172.21.15.90:0/1672347908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c4c10a980 msgr2=0x7f7c4c10ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:54.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.198+0000 7f7c52c00700 1 --2- 172.21.15.90:0/1672347908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c4c10a980 0x7f7c4c10ad90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009990 tx=0x7f7c34009ca0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.198+0000 7f7c52c00700 1 -- 172.21.15.90:0/1672347908 shutdown_connections 2024-05-27T07:42:54.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.198+0000 7f7c52c00700 1 --2- 172.21.15.90:0/1672347908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c4c10b2d0 0x7f7c4c117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.198+0000 7f7c52c00700 1 --2- 172.21.15.90:0/1672347908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4c110960 0x7f7c4c110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.198+0000 7f7c52c00700 1 --2- 172.21.15.90:0/1672347908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c4c10a980 0x7f7c4c10ad90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.198+0000 7f7c52c00700 1 -- 172.21.15.90:0/1672347908 >> 172.21.15.90:0/1672347908 conn(0x7f7c4c077c70 msgr2=0x7f7c4c0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:54.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.199+0000 7f7c52c00700 1 -- 172.21.15.90:0/1672347908 shutdown_connections 2024-05-27T07:42:54.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.199+0000 7f7c52c00700 1 -- 172.21.15.90:0/1672347908 wait complete. 2024-05-27T07:42:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.200+0000 7f7c52c00700 1 Processor -- start 2024-05-27T07:42:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.200+0000 7f7c52c00700 1 -- start start 2024-05-27T07:42:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.200+0000 7f7c52c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4c10a980 0x7f7c4c1a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.200+0000 7f7c52c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c4c10b2d0 0x7f7c4c1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.200+0000 7f7c52c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c4c110960 0x7f7c4c1a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.200+0000 7f7c52c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c11a980 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.200+0000 7f7c52c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c11a800 con 0x7f7c4c110960 2024-05-27T07:42:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.200+0000 7f7c52c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c11ab00 con 0x7f7c4c10a980 2024-05-27T07:42:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.201+0000 7f7c4b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c4c10b2d0 0x7f7c4c1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:54.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.201+0000 7f7c4b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c4c10b2d0 0x7f7c4c1a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39084/0 (socket says 172.21.15.90:39084) 2024-05-27T07:42:54.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.201+0000 7f7c4b400700 1 -- 172.21.15.90:0/1148627259 learned_addr learned my addr 172.21.15.90:0/1148627259 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:54.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.201+0000 7f7c4be00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4c10a980 0x7f7c4c1a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:54.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.201+0000 7f7c50c00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c4c110960 0x7f7c4c1a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.201+0000 7f7c4b400700 1 -- 172.21.15.90:0/1148627259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4c10a980 msgr2=0x7f7c4c1a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.201+0000 7f7c4b400700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4c10a980 0x7f7c4c1a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.201+0000 7f7c4b400700 1 -- 172.21.15.90:0/1148627259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c4c110960 msgr2=0x7f7c4c1a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.201+0000 7f7c4b400700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c4c110960 0x7f7c4c1a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.201+0000 7f7c4b400700 1 -- 172.21.15.90:0/1148627259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c1a61d0 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.202+0000 7f7c4b400700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c4c10b2d0 0x7f7c4c1a1950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7c34006010 tx=0x7f7c340057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.202+0000 7f7c48c00700 1 -- 172.21.15.90:0/1148627259 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34027070 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.202+0000 7f7c48c00700 1 -- 172.21.15.90:0/1148627259 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7c3400fb00 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.202+0000 7f7c48c00700 1 -- 172.21.15.90:0/1148627259 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34022b90 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.202+0000 7f7c52c00700 1 -- 172.21.15.90:0/1148627259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c1a6460 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.203+0000 7f7c52c00700 1 -- 172.21.15.90:0/1148627259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c1a6990 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.204+0000 7f7c52c00700 1 -- 172.21.15.90:0/1148627259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4c0512b0 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.206+0000 7f7c48c00700 1 -- 172.21.15.90:0/1148627259 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7c3402a070 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.204 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:53 smithi090 ceph-mon[31429]: pgmap v436: 129 pgs: 129 active+clean; 613 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-05-27T07:42:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.207+0000 7f7c48c00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7c300b2990 0x7f7c300b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.208+0000 7f7c48c00700 1 -- 172.21.15.90:0/1148627259 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7c340e3730 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.208+0000 7f7c4be00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7c300b2990 0x7f7c300b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.209+0000 7f7c4be00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7c300b2990 0x7f7c300b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c07d0e0 tx=0x7f7c3c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.215+0000 7f7c48c00700 1 -- 172.21.15.90:0/1148627259 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c34013070 con 0x7f7c4c10b2d0 2024-05-27T07:42:54.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:53 smithi136 ceph-mon[35324]: pgmap v436: 129 pgs: 129 active+clean; 613 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-05-27T07:42:54.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:53 smithi129 ceph-mon[35254]: pgmap v436: 129 pgs: 129 active+clean; 613 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-05-27T07:42:54.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.478+0000 7f7c52c00700 1 -- 172.21.15.90:0/1148627259 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7c4c1a2710 con 0x7f7c300b2990 2024-05-27T07:42:54.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.550+0000 7f7c48c00700 1 -- 172.21.15.90:0/1148627259 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7c4c1a2710 con 0x7f7c300b2990 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.553+0000 7f7c52c00700 1 -- 172.21.15.90:0/1148627259 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7c300b2990 msgr2=0x7f7c300b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.553+0000 7f7c52c00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7c300b2990 0x7f7c300b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c07d0e0 tx=0x7f7c3c00a380 comp rx=0 tx=0).stop 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.553+0000 7f7c52c00700 1 -- 172.21.15.90:0/1148627259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c4c10b2d0 msgr2=0x7f7c4c1a1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.553+0000 7f7c52c00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c4c10b2d0 0x7f7c4c1a1950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7c34006010 tx=0x7f7c340057a0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.553+0000 7f7c52c00700 1 -- 172.21.15.90:0/1148627259 shutdown_connections 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.553+0000 7f7c52c00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7c300b2990 0x7f7c300b4ea0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.553+0000 7f7c52c00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7c4c110960 0x7f7c4c1a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.553+0000 7f7c52c00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4c10a980 0x7f7c4c1a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.553+0000 7f7c52c00700 1 --2- 172.21.15.90:0/1148627259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c4c10b2d0 0x7f7c4c1a1950 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.553+0000 7f7c52c00700 1 -- 172.21.15.90:0/1148627259 >> 172.21.15.90:0/1148627259 conn(0x7f7c4c077c70 msgr2=0x7f7c4c07b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:54.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.554+0000 7f7c52c00700 1 -- 172.21.15.90:0/1148627259 shutdown_connections 2024-05-27T07:42:54.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:54.554+0000 7f7c52c00700 1 -- 172.21.15.90:0/1148627259 wait complete. 2024-05-27T07:42:55.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-27T07:42:55.962 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:42:56.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:55 smithi136 ceph-mon[35324]: pgmap v437: 129 pgs: 129 active+clean; 613 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-27T07:42:56.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:55 smithi136 ceph-mon[35324]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:56.269 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:55 smithi090 ceph-mon[31429]: pgmap v437: 129 pgs: 129 active+clean; 613 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-27T07:42:56.269 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:55 smithi090 ceph-mon[31429]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:56.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:55 smithi129 ceph-mon[35254]: pgmap v437: 129 pgs: 129 active+clean; 613 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-05-27T07:42:56.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:55 smithi129 ceph-mon[35254]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:42:58.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:57 smithi090 ceph-mon[31429]: pgmap v438: 129 pgs: 129 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-05-27T07:42:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.097+0000 7f1dfe000700 1 -- 172.21.15.90:0/2924741340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1df810b100 msgr2=0x7f1df8117610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:58.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.097+0000 7f1dfe000700 1 --2- 172.21.15.90:0/2924741340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1df810b100 0x7f1df8117610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b0a0 tx=0x7f1df000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:42:58.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.097+0000 7f1dfe000700 1 -- 172.21.15.90:0/2924741340 shutdown_connections 2024-05-27T07:42:58.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.097+0000 7f1dfe000700 1 --2- 172.21.15.90:0/2924741340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1df810b100 0x7f1df8117610 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:58.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.097+0000 7f1dfe000700 1 --2- 172.21.15.90:0/2924741340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df810f6b0 0x7f1df810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:58.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.097+0000 7f1dfe000700 1 --2- 172.21.15.90:0/2924741340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df810a7b0 0x7f1df810abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:58.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.097+0000 7f1dfe000700 1 -- 172.21.15.90:0/2924741340 >> 172.21.15.90:0/2924741340 conn(0x7f1df8076af0 msgr2=0x7f1df8076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:58.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.097+0000 7f1dfe000700 1 -- 172.21.15.90:0/2924741340 shutdown_connections 2024-05-27T07:42:58.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.097+0000 7f1dfe000700 1 -- 172.21.15.90:0/2924741340 wait complete. 2024-05-27T07:42:58.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.098+0000 7f1dfe000700 1 Processor -- start 2024-05-27T07:42:58.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.099+0000 7f1dfe000700 1 -- start start 2024-05-27T07:42:58.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.099+0000 7f1dfe000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df810a7b0 0x7f1df81a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:58.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.099+0000 7f1dfe000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df810b100 0x7f1df81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:58.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.099+0000 7f1dfe000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1df810f6b0 0x7f1df81a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.099+0000 7f1dfe000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1df811a790 con 0x7f1df810f6b0 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.099+0000 7f1dfe000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1df811a610 con 0x7f1df810a7b0 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.099+0000 7f1dfe000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1df811a910 con 0x7f1df810b100 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1df810f6b0 0x7f1df81a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1df810f6b0 0x7f1df81a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39102/0 (socket says 172.21.15.90:39102) 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfd600700 1 -- 172.21.15.90:0/30497049 learned_addr learned my addr 172.21.15.90:0/30497049 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfcc00700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df810a7b0 0x7f1df81a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1defe00700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df810b100 0x7f1df81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfd600700 1 -- 172.21.15.90:0/30497049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df810b100 msgr2=0x7f1df81a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfd600700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df810b100 0x7f1df81a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfd600700 1 -- 172.21.15.90:0/30497049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df810a7b0 msgr2=0x7f1df81a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:58.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfd600700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df810a7b0 0x7f1df81a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:58.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfd600700 1 -- 172.21.15.90:0/30497049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df81a5f90 con 0x7f1df810f6b0 2024-05-27T07:42:58.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfcc00700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df810a7b0 0x7f1df81a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:42:58.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.100+0000 7f1dfd600700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1df810f6b0 0x7f1df81a58f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b070 tx=0x7f1df0004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:58.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.101+0000 7f1df6000700 1 -- 172.21.15.90:0/30497049 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0021070 con 0x7f1df810f6b0 2024-05-27T07:42:58.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.101+0000 7f1dfe000700 1 -- 172.21.15.90:0/30497049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df81a6220 con 0x7f1df810f6b0 2024-05-27T07:42:58.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.101+0000 7f1df6000700 1 -- 172.21.15.90:0/30497049 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1df0007c00 con 0x7f1df810f6b0 2024-05-27T07:42:58.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.101+0000 7f1df6000700 1 -- 172.21.15.90:0/30497049 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0004860 con 0x7f1df810f6b0 2024-05-27T07:42:58.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.101+0000 7f1dfe000700 1 -- 172.21.15.90:0/30497049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df81adba0 con 0x7f1df810f6b0 2024-05-27T07:42:58.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.102+0000 7f1dfe000700 1 -- 172.21.15.90:0/30497049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df810bea0 con 0x7f1df810f6b0 2024-05-27T07:42:58.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.105+0000 7f1df6000700 1 -- 172.21.15.90:0/30497049 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1df0003fb0 con 0x7f1df810f6b0 2024-05-27T07:42:58.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.106+0000 7f1df6000700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1ddc0b2910 0x7f1ddc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:42:58.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.106+0000 7f1df6000700 1 -- 172.21.15.90:0/30497049 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f1df000f020 con 0x7f1df810f6b0 2024-05-27T07:42:58.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.106+0000 7f1dfcc00700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1ddc0b2910 0x7f1ddc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:42:58.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.107+0000 7f1dfcc00700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1ddc0b2910 0x7f1ddc0b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005e10 tx=0x7f1de8005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:42:58.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.109+0000 7f1df6000700 1 -- 172.21.15.90:0/30497049 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1df002a020 con 0x7f1df810f6b0 2024-05-27T07:42:58.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:57 smithi136 ceph-mon[35324]: pgmap v438: 129 pgs: 129 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-05-27T07:42:58.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:57 smithi129 ceph-mon[35254]: pgmap v438: 129 pgs: 129 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-05-27T07:42:58.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.379+0000 7f1dfe000700 1 -- 172.21.15.90:0/30497049 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1df81a2420 con 0x7f1ddc0b2910 2024-05-27T07:42:58.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.386+0000 7f1df6000700 1 -- 172.21.15.90:0/30497049 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1df81a2420 con 0x7f1ddc0b2910 2024-05-27T07:42:58.384 INFO:teuthology.orchestra.run.smithi090.stdout:/volumes/qa/sv_1/0c7b9e54-beb7-4bb2-8303-96d14db9b08f 2024-05-27T07:42:58.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.389+0000 7f1dfe000700 1 -- 172.21.15.90:0/30497049 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1ddc0b2910 msgr2=0x7f1ddc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:58.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.389+0000 7f1dfe000700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1ddc0b2910 0x7f1ddc0b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005e10 tx=0x7f1de8005d60 comp rx=0 tx=0).stop 2024-05-27T07:42:58.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.389+0000 7f1dfe000700 1 -- 172.21.15.90:0/30497049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1df810f6b0 msgr2=0x7f1df81a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:42:58.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.389+0000 7f1dfe000700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1df810f6b0 0x7f1df81a58f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b070 tx=0x7f1df0004e00 comp rx=0 tx=0).stop 2024-05-27T07:42:58.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.389+0000 7f1dfe000700 1 -- 172.21.15.90:0/30497049 shutdown_connections 2024-05-27T07:42:58.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.389+0000 7f1dfe000700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f1ddc0b2910 0x7f1ddc0b4e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:58.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.389+0000 7f1dfe000700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1df810f6b0 0x7f1df81a58f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:58.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.389+0000 7f1dfe000700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df810a7b0 0x7f1df81a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:58.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.389+0000 7f1dfe000700 1 --2- 172.21.15.90:0/30497049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df810b100 0x7f1df81a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:42:58.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.389+0000 7f1dfe000700 1 -- 172.21.15.90:0/30497049 >> 172.21.15.90:0/30497049 conn(0x7f1df8076af0 msgr2=0x7f1df8108870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:42:58.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.390+0000 7f1dfe000700 1 -- 172.21.15.90:0/30497049 shutdown_connections 2024-05-27T07:42:58.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:42:58.390+0000 7f1dfe000700 1 -- 172.21.15.90:0/30497049 wait complete. 2024-05-27T07:42:59.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-27T07:42:59.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:42:58 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-27T07:42:59.247 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-27T07:42:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:42:58 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-27T07:42:59.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-05-27T07:42:59.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:42:58 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-27T07:42:59.364 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:42:59.819 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:00.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:00 smithi090 ceph-mon[31429]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:43:00.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:00 smithi090 ceph-mon[31429]: pgmap v439: 129 pgs: 129 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-05-27T07:43:00.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-27T07:43:00.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:00 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-27T07:43:00.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:00 smithi090 ceph-mon[31429]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T07:43:00.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:00 smithi129 ceph-mon[35254]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:43:00.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:00 smithi129 ceph-mon[35254]: pgmap v439: 129 pgs: 129 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-05-27T07:43:00.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-27T07:43:00.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:00 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-27T07:43:00.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:00 smithi129 ceph-mon[35254]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T07:43:00.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:00 smithi136 ceph-mon[35324]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:43:00.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:00 smithi136 ceph-mon[35324]: pgmap v439: 129 pgs: 129 active+clean; 630 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-05-27T07:43:00.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-05-27T07:43:00.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:00 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-27T07:43:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:00 smithi136 ceph-mon[35324]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T07:43:01.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:01 smithi090 ceph-mon[31429]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T07:43:01.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:01 smithi090 ceph-mon[31429]: pgmap v442: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-05-27T07:43:01.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:01 smithi129 ceph-mon[35254]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T07:43:01.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:01 smithi129 ceph-mon[35254]: pgmap v442: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-05-27T07:43:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:01 smithi136 ceph-mon[35324]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T07:43:01.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:01 smithi136 ceph-mon[35324]: pgmap v442: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-05-27T07:43:02.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.148+0000 7fbf72000700 1 -- 172.21.15.90:0/597713192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf6c106ec0 msgr2=0x7fbf6c113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:02.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.148+0000 7fbf72000700 1 --2- 172.21.15.90:0/597713192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf6c106ec0 0x7fbf6c113350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400b0a0 tx=0x7fbf6400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.148+0000 7fbf72000700 1 -- 172.21.15.90:0/597713192 shutdown_connections 2024-05-27T07:43:02.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.148+0000 7fbf72000700 1 --2- 172.21.15.90:0/597713192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf6c106ec0 0x7fbf6c113350 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.148+0000 7fbf72000700 1 --2- 172.21.15.90:0/597713192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf6c10c5b0 0x7fbf6c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.148+0000 7fbf72000700 1 --2- 172.21.15.90:0/597713192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf6c106570 0x7fbf6c106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.148+0000 7fbf72000700 1 -- 172.21.15.90:0/597713192 >> 172.21.15.90:0/597713192 conn(0x7fbf6c102070 msgr2=0x7fbf6c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:02.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.148+0000 7fbf72000700 1 -- 172.21.15.90:0/597713192 shutdown_connections 2024-05-27T07:43:02.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.148+0000 7fbf72000700 1 -- 172.21.15.90:0/597713192 wait complete. 2024-05-27T07:43:02.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.149+0000 7fbf72000700 1 Processor -- start 2024-05-27T07:43:02.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.150+0000 7fbf72000700 1 -- start start 2024-05-27T07:43:02.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.150+0000 7fbf72000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf6c106570 0x7fbf6c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:02.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.150+0000 7fbf72000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf6c106ec0 0x7fbf6c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:02.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.150+0000 7fbf72000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf6c10c5b0 0x7fbf6c1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:02.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.150+0000 7fbf72000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c1165e0 con 0x7fbf6c106570 2024-05-27T07:43:02.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.150+0000 7fbf72000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c116460 con 0x7fbf6c10c5b0 2024-05-27T07:43:02.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.150+0000 7fbf72000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c116760 con 0x7fbf6c106ec0 2024-05-27T07:43:02.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.150+0000 7fbf6b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf6c106570 0x7fbf6c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:02.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.151+0000 7fbf6b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf6c106570 0x7fbf6c19cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49088/0 (socket says 172.21.15.90:49088) 2024-05-27T07:43:02.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.151+0000 7fbf6b400700 1 -- 172.21.15.90:0/2621935502 learned_addr learned my addr 172.21.15.90:0/2621935502 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:02.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.151+0000 7fbf63e00700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf6c106ec0 0x7fbf6c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:02.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.151+0000 7fbf6be00700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf6c10c5b0 0x7fbf6c1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:02.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.151+0000 7fbf6b400700 1 -- 172.21.15.90:0/2621935502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf6c106ec0 msgr2=0x7fbf6c19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:02.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.151+0000 7fbf6b400700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf6c106ec0 0x7fbf6c19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.151+0000 7fbf6b400700 1 -- 172.21.15.90:0/2621935502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf6c10c5b0 msgr2=0x7fbf6c1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:02.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.151+0000 7fbf6b400700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf6c10c5b0 0x7fbf6c1a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.151+0000 7fbf6b400700 1 -- 172.21.15.90:0/2621935502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c1a1ea0 con 0x7fbf6c106570 2024-05-27T07:43:02.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.151+0000 7fbf6b400700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf6c106570 0x7fbf6c19cfd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00d980 tx=0x7fbf5c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:02.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.152+0000 7fbf68c00700 1 -- 172.21.15.90:0/2621935502 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c017070 con 0x7fbf6c106570 2024-05-27T07:43:02.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.152+0000 7fbf68c00700 1 -- 172.21.15.90:0/2621935502 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbf5c010460 con 0x7fbf6c106570 2024-05-27T07:43:02.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.152+0000 7fbf72000700 1 -- 172.21.15.90:0/2621935502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c1a2190 con 0x7fbf6c106570 2024-05-27T07:43:02.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.152+0000 7fbf68c00700 1 -- 172.21.15.90:0/2621935502 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00de60 con 0x7fbf6c106570 2024-05-27T07:43:02.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.152+0000 7fbf72000700 1 -- 172.21.15.90:0/2621935502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c1a9ab0 con 0x7fbf6c106570 2024-05-27T07:43:02.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.153+0000 7fbf72000700 1 -- 172.21.15.90:0/2621935502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c0512b0 con 0x7fbf6c106570 2024-05-27T07:43:02.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.156+0000 7fbf68c00700 1 -- 172.21.15.90:0/2621935502 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbf5c010610 con 0x7fbf6c106570 2024-05-27T07:43:02.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.157+0000 7fbf68c00700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbf500b2910 0x7fbf500b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:02.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.158+0000 7fbf68c00700 1 -- 172.21.15.90:0/2621935502 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fbf5c0e2220 con 0x7fbf6c106570 2024-05-27T07:43:02.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.158+0000 7fbf63e00700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbf500b2910 0x7fbf500b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:02.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.158+0000 7fbf63e00700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbf500b2910 0x7fbf500b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbf54005950 tx=0x7fbf540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:02.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.160+0000 7fbf68c00700 1 -- 172.21.15.90:0/2621935502 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf5c0a99f0 con 0x7fbf6c106570 2024-05-27T07:43:02.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.438+0000 7fbf72000700 1 -- 172.21.15.90:0/2621935502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf6c02e5e0 con 0x7fbf6c106570 2024-05-27T07:43:02.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.440+0000 7fbf68c00700 1 -- 172.21.15.90:0/2621935502 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11941 (secure 0 0 0) 0x7fbf5c0a92f0 con 0x7fbf6c106570 2024-05-27T07:43:02.438 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:43:02.439 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:42:38.788447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24478,34406,34424,34442,34472]},"id":1}]} 2024-05-27T07:43:02.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 -- 172.21.15.90:0/2621935502 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbf500b2910 msgr2=0x7fbf500b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:02.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbf500b2910 0x7fbf500b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbf54005950 tx=0x7fbf540058e0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 -- 172.21.15.90:0/2621935502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf6c106570 msgr2=0x7fbf6c19cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:02.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf6c106570 0x7fbf6c19cfd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00d980 tx=0x7fbf5c00dd40 comp rx=0 tx=0).stop 2024-05-27T07:43:02.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 -- 172.21.15.90:0/2621935502 shutdown_connections 2024-05-27T07:43:02.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbf500b2910 0x7fbf500b4e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf6c10c5b0 0x7fbf6c1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf6c106570 0x7fbf6c19cfd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 --2- 172.21.15.90:0/2621935502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf6c106ec0 0x7fbf6c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:02.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 -- 172.21.15.90:0/2621935502 >> 172.21.15.90:0/2621935502 conn(0x7fbf6c102070 msgr2=0x7fbf6c104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:02.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 -- 172.21.15.90:0/2621935502 shutdown_connections 2024-05-27T07:43:02.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:02.443+0000 7fbf72000700 1 -- 172.21.15.90:0/2621935502 wait complete. 2024-05-27T07:43:02.444 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-05-27T07:43:02.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:02 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2621935502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:43:02.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:02 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2621935502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:43:02.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:02 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2621935502' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:43:03.267 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:43:03.722 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:03.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:03 smithi136 ceph-mon[35324]: pgmap v443: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-05-27T07:43:03.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:03 smithi129 ceph-mon[35254]: pgmap v443: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-05-27T07:43:03.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:03 smithi090 ceph-mon[31429]: pgmap v443: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-05-27T07:43:05.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:05 smithi129 ceph-mon[35254]: pgmap v444: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-27T07:43:05.833 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:05 smithi090 ceph-mon[31429]: pgmap v444: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-27T07:43:05.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:05 smithi136 ceph-mon[35324]: pgmap v444: 129 pgs: 129 active+clean; 639 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-27T07:43:06.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.080+0000 7f776a400700 1 -- 172.21.15.90:0/2017857124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f776410c5b0 msgr2=0x7f776410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:06.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.080+0000 7f776a400700 1 --2- 172.21.15.90:0/2017857124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f776410c5b0 0x7f776410c9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7754009990 tx=0x7f7754009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:06.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.080+0000 7f776a400700 1 -- 172.21.15.90:0/2017857124 shutdown_connections 2024-05-27T07:43:06.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.080+0000 7f776a400700 1 --2- 172.21.15.90:0/2017857124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7764106ec0 0x7f7764113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:06.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.080+0000 7f776a400700 1 --2- 172.21.15.90:0/2017857124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f776410c5b0 0x7f776410c9c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:06.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.080+0000 7f776a400700 1 --2- 172.21.15.90:0/2017857124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7764106570 0x7f7764106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:06.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.080+0000 7f776a400700 1 -- 172.21.15.90:0/2017857124 >> 172.21.15.90:0/2017857124 conn(0x7f7764102070 msgr2=0x7f77641044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:06.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.080+0000 7f776a400700 1 -- 172.21.15.90:0/2017857124 shutdown_connections 2024-05-27T07:43:06.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.080+0000 7f776a400700 1 -- 172.21.15.90:0/2017857124 wait complete. 2024-05-27T07:43:06.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.081+0000 7f776a400700 1 Processor -- start 2024-05-27T07:43:06.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.082+0000 7f776a400700 1 -- start start 2024-05-27T07:43:06.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.082+0000 7f776a400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7764106570 0x7f776419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:06.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.082+0000 7f776a400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7764106ec0 0x7f776419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.082+0000 7f776a400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f776410c5b0 0x7f77641a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.082+0000 7f776a400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7764116570 con 0x7f7764106ec0 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.082+0000 7f776a400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77641163f0 con 0x7f776410c5b0 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.082+0000 7f776a400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77641166f0 con 0x7f7764106570 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7762a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7764106ec0 0x7f776419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7762a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7764106ec0 0x7f776419d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49094/0 (socket says 172.21.15.90:49094) 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7762a00700 1 -- 172.21.15.90:0/3891997297 learned_addr learned my addr 172.21.15.90:0/3891997297 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7763400700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7764106570 0x7f776419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7763e00700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f776410c5b0 0x7f77641a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7762a00700 1 -- 172.21.15.90:0/3891997297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7764106570 msgr2=0x7f776419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:06.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7762a00700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7764106570 0x7f776419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:06.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7762a00700 1 -- 172.21.15.90:0/3891997297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f776410c5b0 msgr2=0x7f77641a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:06.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7762a00700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f776410c5b0 0x7f77641a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:06.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7762a00700 1 -- 172.21.15.90:0/3891997297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77641a1e00 con 0x7f7764106ec0 2024-05-27T07:43:06.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.083+0000 7f7763e00700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f776410c5b0 0x7f77641a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:43:06.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.084+0000 7f7762a00700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7764106ec0 0x7f776419d520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f775000d950 tx=0x7f775000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:06.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.084+0000 7f775be00700 1 -- 172.21.15.90:0/3891997297 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750017070 con 0x7f7764106ec0 2024-05-27T07:43:06.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.084+0000 7f775be00700 1 -- 172.21.15.90:0/3891997297 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7750009dc0 con 0x7f7764106ec0 2024-05-27T07:43:06.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.084+0000 7f776a400700 1 -- 172.21.15.90:0/3891997297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77641a20f0 con 0x7f7764106ec0 2024-05-27T07:43:06.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.084+0000 7f775be00700 1 -- 172.21.15.90:0/3891997297 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775000f3c0 con 0x7f7764106ec0 2024-05-27T07:43:06.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.084+0000 7f776a400700 1 -- 172.21.15.90:0/3891997297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77641a9a10 con 0x7f7764106ec0 2024-05-27T07:43:06.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.085+0000 7f776a400700 1 -- 172.21.15.90:0/3891997297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77640512b0 con 0x7f7764106ec0 2024-05-27T07:43:06.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.090+0000 7f775be00700 1 -- 172.21.15.90:0/3891997297 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f775001a070 con 0x7f7764106ec0 2024-05-27T07:43:06.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.091+0000 7f775be00700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77480b2990 0x7f77480b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:06.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.091+0000 7f775be00700 1 -- 172.21.15.90:0/3891997297 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f77500e1f70 con 0x7f7764106ec0 2024-05-27T07:43:06.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.092+0000 7f7763400700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77480b2990 0x7f77480b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:06.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.092+0000 7f775be00700 1 -- 172.21.15.90:0/3891997297 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f77500a97c0 con 0x7f7764106ec0 2024-05-27T07:43:06.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.092+0000 7f7763400700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77480b2990 0x7f77480b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7754006010 tx=0x7f775400b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:06.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.325+0000 7f776a400700 1 -- 172.21.15.90:0/3891997297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f776402e5e0 con 0x7f7764106ec0 2024-05-27T07:43:06.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.327+0000 7f775be00700 1 -- 172.21.15.90:0/3891997297 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24725 (secure 0 0 0) 0x7f77500a90c0 con 0x7f7764106ec0 2024-05-27T07:43:06.326 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:43:06.326 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":91,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:43:00.000806+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:43:06.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 -- 172.21.15.90:0/3891997297 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77480b2990 msgr2=0x7f77480b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:06.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77480b2990 0x7f77480b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7754006010 tx=0x7f775400b560 comp rx=0 tx=0).stop 2024-05-27T07:43:06.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 -- 172.21.15.90:0/3891997297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7764106ec0 msgr2=0x7f776419d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:06.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7764106ec0 0x7f776419d520 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f775000d950 tx=0x7f775000dc60 comp rx=0 tx=0).stop 2024-05-27T07:43:06.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 -- 172.21.15.90:0/3891997297 shutdown_connections 2024-05-27T07:43:06.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77480b2990 0x7f77480b4ea0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:06.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f776410c5b0 0x7f77641a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:06.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7764106570 0x7f776419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:06.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 --2- 172.21.15.90:0/3891997297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7764106ec0 0x7f776419d520 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:06.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 -- 172.21.15.90:0/3891997297 >> 172.21.15.90:0/3891997297 conn(0x7f7764102070 msgr2=0x7f77641049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:06.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.330+0000 7f776a400700 1 -- 172.21.15.90:0/3891997297 shutdown_connections 2024-05-27T07:43:06.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:06.331+0000 7f776a400700 1 -- 172.21.15.90:0/3891997297 wait complete. 2024-05-27T07:43:06.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:06 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3891997297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:43:06.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:06 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3891997297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:43:06.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:06 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3891997297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:43:07.134 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T07:43:07.134 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T07:43:07.576 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:07.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:07 smithi129 ceph-mon[35254]: pgmap v445: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-27T07:43:07.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:07 smithi090 ceph-mon[31429]: pgmap v445: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-27T07:43:07.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:07 smithi136 ceph-mon[35324]: pgmap v445: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-27T07:43:09.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.779+0000 7fefff200700 1 -- 172.21.15.90:0/88307493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff000109670 msgr2=0x7ff000110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:09.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.779+0000 7fefff200700 1 --2- 172.21.15.90:0/88307493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff000109670 0x7ff000110fd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feff800b0d0 tx=0x7feff800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:43:09.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:09 smithi090 ceph-mon[31429]: pgmap v446: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-05-27T07:43:09.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.779+0000 7fefff200700 1 -- 172.21.15.90:0/88307493 shutdown_connections 2024-05-27T07:43:09.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.779+0000 7fefff200700 1 --2- 172.21.15.90:0/88307493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff000109670 0x7ff000110fd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.779+0000 7fefff200700 1 --2- 172.21.15.90:0/88307493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0001043f0 0x7ff000104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.779+0000 7fefff200700 1 --2- 172.21.15.90:0/88307493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff000108d20 0x7ff000109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.779+0000 7fefff200700 1 -- 172.21.15.90:0/88307493 >> 172.21.15.90:0/88307493 conn(0x7ff000079d00 msgr2=0x7ff00007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.779+0000 7fefff200700 1 -- 172.21.15.90:0/88307493 shutdown_connections 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.780+0000 7fefff200700 1 -- 172.21.15.90:0/88307493 wait complete. 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.781+0000 7fefff200700 1 Processor -- start 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.781+0000 7fefff200700 1 -- start start 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.781+0000 7fefff200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0001043f0 0x7ff0001a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.781+0000 7fefff200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff000108d20 0x7ff0001a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.781+0000 7fefff200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff000109670 0x7ff00019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.781+0000 7fefff200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001141c0 con 0x7ff000109670 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.781+0000 7fefff200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff000114040 con 0x7ff000108d20 2024-05-27T07:43:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.781+0000 7fefff200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff000114340 con 0x7ff0001043f0 2024-05-27T07:43:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.782+0000 7feffe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff000109670 0x7ff00019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.782+0000 7feffe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff000109670 0x7ff00019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44064/0 (socket says 172.21.15.90:44064) 2024-05-27T07:43:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.782+0000 7feffe800700 1 -- 172.21.15.90:0/1884699208 learned_addr learned my addr 172.21.15.90:0/1884699208 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.782+0000 7feffde00700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0001043f0 0x7ff0001a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.782+0000 7feffd400700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff000108d20 0x7ff0001a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.782+0000 7feffe800700 1 -- 172.21.15.90:0/1884699208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0001043f0 msgr2=0x7ff0001a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.782+0000 7feffe800700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0001043f0 0x7ff0001a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:09.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.782+0000 7feffe800700 1 -- 172.21.15.90:0/1884699208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff000108d20 msgr2=0x7ff0001a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:09.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.782+0000 7feffe800700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff000108d20 0x7ff0001a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:09.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.782+0000 7feffe800700 1 -- 172.21.15.90:0/1884699208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00019b000 con 0x7ff000109670 2024-05-27T07:43:09.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.783+0000 7feffe800700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff000109670 0x7ff00019a9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feff8007d90 tx=0x7feff8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:09.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.783+0000 7feff6a00700 1 -- 172.21.15.90:0/1884699208 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff801f030 con 0x7ff000109670 2024-05-27T07:43:09.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.783+0000 7feff6a00700 1 -- 172.21.15.90:0/1884699208 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feff801bb90 con 0x7ff000109670 2024-05-27T07:43:09.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.783+0000 7feff6a00700 1 -- 172.21.15.90:0/1884699208 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8012bd0 con 0x7ff000109670 2024-05-27T07:43:09.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.783+0000 7fefff200700 1 -- 172.21.15.90:0/1884699208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00019b290 con 0x7ff000109670 2024-05-27T07:43:09.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.783+0000 7fefff200700 1 -- 172.21.15.90:0/1884699208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00019b780 con 0x7ff000109670 2024-05-27T07:43:09.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.784+0000 7fefff200700 1 -- 172.21.15.90:0/1884699208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0000512b0 con 0x7ff000109670 2024-05-27T07:43:09.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.787+0000 7feff6a00700 1 -- 172.21.15.90:0/1884699208 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7feff8022030 con 0x7ff000109670 2024-05-27T07:43:09.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.788+0000 7feff6a00700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fefe40b2990 0x7fefe40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:09.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.789+0000 7feff6a00700 1 -- 172.21.15.90:0/1884699208 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7feff800f020 con 0x7ff000109670 2024-05-27T07:43:09.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.789+0000 7feffde00700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fefe40b2990 0x7fefe40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:09.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.790+0000 7feffde00700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fefe40b2990 0x7fefe40b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff00019bb00 tx=0x7feff000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:09.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:09.792+0000 7feff6a00700 1 -- 172.21.15.90:0/1884699208 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feff8029020 con 0x7ff000109670 2024-05-27T07:43:09.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:09 smithi129 ceph-mon[35254]: pgmap v446: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-05-27T07:43:09.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:09 smithi136 ceph-mon[35324]: pgmap v446: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-05-27T07:43:10.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.017+0000 7fefff200700 1 -- 172.21.15.90:0/1884699208 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff000063bf0 con 0x7fefe40b2990 2024-05-27T07:43:10.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.052+0000 7feff6a00700 1 -- 172.21.15.90:0/1884699208 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340477 (secure 0 0 0) 0x7ff000063bf0 con 0x7fefe40b2990 2024-05-27T07:43:10.055 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:43:10.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 -- 172.21.15.90:0/1884699208 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fefe40b2990 msgr2=0x7fefe40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:10.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fefe40b2990 0x7fefe40b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff00019bb00 tx=0x7feff000a380 comp rx=0 tx=0).stop 2024-05-27T07:43:10.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 -- 172.21.15.90:0/1884699208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff000109670 msgr2=0x7ff00019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:10.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff000109670 0x7ff00019a9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feff8007d90 tx=0x7feff8007dc0 comp rx=0 tx=0).stop 2024-05-27T07:43:10.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 -- 172.21.15.90:0/1884699208 shutdown_connections 2024-05-27T07:43:10.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fefe40b2990 0x7fefe40b4ea0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:10.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff000109670 0x7ff00019a9c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:10.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0001043f0 0x7ff0001a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:10.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 --2- 172.21.15.90:0/1884699208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff000108d20 0x7ff0001a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:10.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 -- 172.21.15.90:0/1884699208 >> 172.21.15.90:0/1884699208 conn(0x7ff000079d00 msgr2=0x7ff000101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:10.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.060+0000 7fefff200700 1 -- 172.21.15.90:0/1884699208 shutdown_connections 2024-05-27T07:43:10.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:10.061+0000 7fefff200700 1 -- 172.21.15.90:0/1884699208 wait complete. 2024-05-27T07:43:10.061 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-05-27T07:43:10.993 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-05-27T07:43:08.418385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663188,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":429,"num_read_kb":324,"num_write":376,"num_write_kb":2298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359468,"kb_used_data":14432,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582356,"statfs":{"total":1151940427776,"available":1151572332544,"internally_reserved":0,"allocated":14778368,"data_stored":8225546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19138,"internal_metadata":353154366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17593,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":81,"num_write":12,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.415452"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196184+0000","last_change":"2024-05-27T07:41:08.839265+0000","last_active":"2024-05-27T07:43:00.196184+0000","last_peered":"2024-05-27T07:43:00.196184+0000","last_clean":"2024-05-27T07:43:00.196184+0000","last_became_active":"2024-05-27T07:41:08.838814+0000","last_became_peered":"2024-05-27T07:41:08.838814+0000","last_unstale":"2024-05-27T07:43:00.196184+0000","last_undegraded":"2024-05-27T07:43:00.196184+0000","last_fullsized":"2024-05-27T07:43:00.196184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:32.901520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193489+0000","last_change":"2024-05-27T07:41:08.838313+0000","last_active":"2024-05-27T07:43:00.193489+0000","last_peered":"2024-05-27T07:43:00.193489+0000","last_clean":"2024-05-27T07:43:00.193489+0000","last_became_active":"2024-05-27T07:41:08.836657+0000","last_became_peered":"2024-05-27T07:41:08.836657+0000","last_unstale":"2024-05-27T07:43:00.193489+0000","last_undegraded":"2024-05-27T07:43:00.193489+0000","last_fullsized":"2024-05-27T07:43:00.193489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:33.556505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193231+0000","last_change":"2024-05-27T07:41:08.841513+0000","last_active":"2024-05-27T07:43:00.193231+0000","last_peered":"2024-05-27T07:43:00.193231+0000","last_clean":"2024-05-27T07:43:00.193231+0000","last_became_active":"2024-05-27T07:41:08.838624+0000","last_became_peered":"2024-05-27T07:41:08.838624+0000","last_unstale":"2024-05-27T07:43:00.193231+0000","last_undegraded":"2024-05-27T07:43:00.193231+0000","last_fullsized":"2024-05-27T07:43:00.193231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:01:37.320418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011839+0000","last_change":"2024-05-27T07:41:08.839138+0000","last_active":"2024-05-27T07:43:00.011839+0000","last_peered":"2024-05-27T07:43:00.011839+0000","last_clean":"2024-05-27T07:43:00.011839+0000","last_became_active":"2024-05-27T07:41:08.838766+0000","last_became_peered":"2024-05-27T07:41:08.838766+0000","last_unstale":"2024-05-27T07:43:00.011839+0000","last_undegraded":"2024-05-27T07:43:00.011839+0000","last_fullsized":"2024-05-27T07:43:00.011839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:09.129257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195829+0000","last_change":"2024-05-27T07:41:08.823006+0000","last_active":"2024-05-27T07:43:00.195829+0000","last_peered":"2024-05-27T07:43:00.195829+0000","last_clean":"2024-05-27T07:43:00.195829+0000","last_became_active":"2024-05-27T07:41:08.822551+0000","last_became_peered":"2024-05-27T07:41:08.822551+0000","last_unstale":"2024-05-27T07:43:00.195829+0000","last_undegraded":"2024-05-27T07:43:00.195829+0000","last_fullsized":"2024-05-27T07:43:00.195829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:49.255763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.034743+0000","last_change":"2024-05-27T07:41:08.846072+0000","last_active":"2024-05-27T07:43:00.034743+0000","last_peered":"2024-05-27T07:43:00.034743+0000","last_clean":"2024-05-27T07:43:00.034743+0000","last_became_active":"2024-05-27T07:41:08.845604+0000","last_became_peered":"2024-05-27T07:41:08.845604+0000","last_unstale":"2024-05-27T07:43:00.034743+0000","last_undegraded":"2024-05-27T07:43:00.034743+0000","last_fullsized":"2024-05-27T07:43:00.034743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:23:56.389135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194969+0000","last_change":"2024-05-27T07:41:08.841046+0000","last_active":"2024-05-27T07:43:00.194969+0000","last_peered":"2024-05-27T07:43:00.194969+0000","last_clean":"2024-05-27T07:43:00.194969+0000","last_became_active":"2024-05-27T07:41:08.840028+0000","last_became_peered":"2024-05-27T07:41:08.840028+0000","last_unstale":"2024-05-27T07:43:00.194969+0000","last_undegraded":"2024-05-27T07:43:00.194969+0000","last_fullsized":"2024-05-27T07:43:00.194969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:35.239919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.116821+0000","last_change":"2024-05-27T07:41:08.841182+0000","last_active":"2024-05-27T07:43:00.116821+0000","last_peered":"2024-05-27T07:43:00.116821+0000","last_clean":"2024-05-27T07:43:00.116821+0000","last_became_active":"2024-05-27T07:41:08.840645+0000","last_became_peered":"2024-05-27T07:41:08.840645+0000","last_unstale":"2024-05-27T07:43:00.116821+0000","last_undegraded":"2024-05-27T07:43:00.116821+0000","last_fullsized":"2024-05-27T07:43:00.116821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:10:59.744505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196915+0000","last_change":"2024-05-27T07:41:08.825874+0000","last_active":"2024-05-27T07:43:00.196915+0000","last_peered":"2024-05-27T07:43:00.196915+0000","last_clean":"2024-05-27T07:43:00.196915+0000","last_became_active":"2024-05-27T07:41:08.825386+0000","last_became_peered":"2024-05-27T07:41:08.825386+0000","last_unstale":"2024-05-27T07:43:00.196915+0000","last_undegraded":"2024-05-27T07:43:00.196915+0000","last_fullsized":"2024-05-27T07:43:00.196915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:13.304612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.035827+0000","last_change":"2024-05-27T07:41:08.847445+0000","last_active":"2024-05-27T07:43:00.035827+0000","last_peered":"2024-05-27T07:43:00.035827+0000","last_clean":"2024-05-27T07:43:00.035827+0000","last_became_active":"2024-05-27T07:41:08.846512+0000","last_became_peered":"2024-05-27T07:41:08.846512+0000","last_unstale":"2024-05-27T07:43:00.035827+0000","last_undegraded":"2024-05-27T07:43:00.035827+0000","last_fullsized":"2024-05-27T07:43:00.035827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:57.235218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196395+0000","last_change":"2024-05-27T07:41:08.843531+0000","last_active":"2024-05-27T07:43:00.196395+0000","last_peered":"2024-05-27T07:43:00.196395+0000","last_clean":"2024-05-27T07:43:00.196395+0000","last_became_active":"2024-05-27T07:41:08.843214+0000","last_became_peered":"2024-05-27T07:41:08.843214+0000","last_unstale":"2024-05-27T07:43:00.196395+0000","last_undegraded":"2024-05-27T07:43:00.196395+0000","last_fullsized":"2024-05-27T07:43:00.196395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:00.884178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011969+0000","last_change":"2024-05-27T07:41:08.839487+0000","last_active":"2024-05-27T07:43:00.011969+0000","last_peered":"2024-05-27T07:43:00.011969+0000","last_clean":"2024-05-27T07:43:00.011969+0000","last_became_active":"2024-05-27T07:41:08.839097+0000","last_became_peered":"2024-05-27T07:41:08.839097+0000","last_unstale":"2024-05-27T07:43:00.011969+0000","last_undegraded":"2024-05-27T07:43:00.011969+0000","last_fullsized":"2024-05-27T07:43:00.011969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:51.055482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.017645+0000","last_change":"2024-05-27T07:41:08.836843+0000","last_active":"2024-05-27T07:43:00.017645+0000","last_peered":"2024-05-27T07:43:00.017645+0000","last_clean":"2024-05-27T07:43:00.017645+0000","last_became_active":"2024-05-27T07:41:08.836362+0000","last_became_peered":"2024-05-27T07:41:08.836362+0000","last_unstale":"2024-05-27T07:43:00.017645+0000","last_undegraded":"2024-05-27T07:43:00.017645+0000","last_fullsized":"2024-05-27T07:43:00.017645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:47.844301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.192332+0000","last_change":"2024-05-27T07:41:08.839046+0000","last_active":"2024-05-27T07:43:00.192332+0000","last_peered":"2024-05-27T07:43:00.192332+0000","last_clean":"2024-05-27T07:43:00.192332+0000","last_became_active":"2024-05-27T07:41:08.838601+0000","last_became_peered":"2024-05-27T07:41:08.838601+0000","last_unstale":"2024-05-27T07:43:00.192332+0000","last_undegraded":"2024-05-27T07:43:00.192332+0000","last_fullsized":"2024-05-27T07:43:00.192332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:59.563490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.192386+0000","last_change":"2024-05-27T07:41:08.832510+0000","last_active":"2024-05-27T07:43:00.192386+0000","last_peered":"2024-05-27T07:43:00.192386+0000","last_clean":"2024-05-27T07:43:00.192386+0000","last_became_active":"2024-05-27T07:41:08.832082+0000","last_became_peered":"2024-05-27T07:41:08.832082+0000","last_unstale":"2024-05-27T07:43:00.192386+0000","last_undegraded":"2024-05-27T07:43:00.192386+0000","last_fullsized":"2024-05-27T07:43:00.192386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:59.951860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193432+0000","last_change":"2024-05-27T07:41:08.827953+0000","last_active":"2024-05-27T07:43:00.193432+0000","last_peered":"2024-05-27T07:43:00.193432+0000","last_clean":"2024-05-27T07:43:00.193432+0000","last_became_active":"2024-05-27T07:41:08.827293+0000","last_became_peered":"2024-05-27T07:41:08.827293+0000","last_unstale":"2024-05-27T07:43:00.193432+0000","last_undegraded":"2024-05-27T07:43:00.193432+0000","last_fullsized":"2024-05-27T07:43:00.193432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:08.810514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195477+0000","last_change":"2024-05-27T07:41:08.839591+0000","last_active":"2024-05-27T07:43:00.195477+0000","last_peered":"2024-05-27T07:43:00.195477+0000","last_clean":"2024-05-27T07:43:00.195477+0000","last_became_active":"2024-05-27T07:41:08.839217+0000","last_became_peered":"2024-05-27T07:41:08.839217+0000","last_unstale":"2024-05-27T07:43:00.195477+0000","last_undegraded":"2024-05-27T07:43:00.195477+0000","last_fullsized":"2024-05-27T07:43:00.195477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:11:41.636163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.010756+0000","last_change":"2024-05-27T07:41:08.839544+0000","last_active":"2024-05-27T07:43:00.010756+0000","last_peered":"2024-05-27T07:43:00.010756+0000","last_clean":"2024-05-27T07:43:00.010756+0000","last_became_active":"2024-05-27T07:41:08.839177+0000","last_became_peered":"2024-05-27T07:41:08.839177+0000","last_unstale":"2024-05-27T07:43:00.010756+0000","last_undegraded":"2024-05-27T07:43:00.010756+0000","last_fullsized":"2024-05-27T07:43:00.010756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:41.921759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013774+0000","last_change":"2024-05-27T07:41:08.847911+0000","last_active":"2024-05-27T07:43:00.013774+0000","last_peered":"2024-05-27T07:43:00.013774+0000","last_clean":"2024-05-27T07:43:00.013774+0000","last_became_active":"2024-05-27T07:41:08.847427+0000","last_became_peered":"2024-05-27T07:41:08.847427+0000","last_unstale":"2024-05-27T07:43:00.013774+0000","last_undegraded":"2024-05-27T07:43:00.013774+0000","last_fullsized":"2024-05-27T07:43:00.013774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:13.692744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012454+0000","last_change":"2024-05-27T07:41:21.999614+0000","last_active":"2024-05-27T07:43:00.012454+0000","last_peered":"2024-05-27T07:43:00.012454+0000","last_clean":"2024-05-27T07:43:00.012454+0000","last_became_active":"2024-05-27T07:41:21.999086+0000","last_became_peered":"2024-05-27T07:41:21.999086+0000","last_unstale":"2024-05-27T07:43:00.012454+0000","last_undegraded":"2024-05-27T07:43:00.012454+0000","last_fullsized":"2024-05-27T07:43:00.012454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:20.229318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":91,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:06.012523+0000","last_change":"2024-05-27T07:41:04.795079+0000","last_active":"2024-05-27T07:43:06.012523+0000","last_peered":"2024-05-27T07:43:06.012523+0000","last_clean":"2024-05-27T07:43:06.012523+0000","last_became_active":"2024-05-27T07:41:04.794367+0000","last_became_peered":"2024-05-27T07:41:04.794367+0000","last_unstale":"2024-05-27T07:43:06.012523+0000","last_undegraded":"2024-05-27T07:43:06.012523+0000","last_fullsized":"2024-05-27T07:43:06.012523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:27:52.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.118644+0000","last_change":"2024-05-27T07:41:08.846712+0000","last_active":"2024-05-27T07:43:00.118644+0000","last_peered":"2024-05-27T07:43:00.118644+0000","last_clean":"2024-05-27T07:43:00.118644+0000","last_became_active":"2024-05-27T07:41:08.839999+0000","last_became_peered":"2024-05-27T07:41:08.839999+0000","last_unstale":"2024-05-27T07:43:00.118644+0000","last_undegraded":"2024-05-27T07:43:00.118644+0000","last_fullsized":"2024-05-27T07:43:00.118644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:31:23.924074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011296+0000","last_change":"2024-05-27T07:41:21.996675+0000","last_active":"2024-05-27T07:43:00.011296+0000","last_peered":"2024-05-27T07:43:00.011296+0000","last_clean":"2024-05-27T07:43:00.011296+0000","last_became_active":"2024-05-27T07:41:21.996152+0000","last_became_peered":"2024-05-27T07:41:21.996152+0000","last_unstale":"2024-05-27T07:43:00.011296+0000","last_undegraded":"2024-05-27T07:43:00.011296+0000","last_fullsized":"2024-05-27T07:43:00.011296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:57.415146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.119419+0000","last_change":"2024-05-27T07:41:04.890045+0000","last_active":"2024-05-27T07:43:00.119419+0000","last_peered":"2024-05-27T07:43:00.119419+0000","last_clean":"2024-05-27T07:43:00.119419+0000","last_became_active":"2024-05-27T07:41:04.888993+0000","last_became_peered":"2024-05-27T07:41:04.888993+0000","last_unstale":"2024-05-27T07:43:00.119419+0000","last_undegraded":"2024-05-27T07:43:00.119419+0000","last_fullsized":"2024-05-27T07:43:00.119419+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:19.866699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193523+0000","last_change":"2024-05-27T07:41:08.847472+0000","last_active":"2024-05-27T07:43:00.193523+0000","last_peered":"2024-05-27T07:43:00.193523+0000","last_clean":"2024-05-27T07:43:00.193523+0000","last_became_active":"2024-05-27T07:41:08.846070+0000","last_became_peered":"2024-05-27T07:41:08.846070+0000","last_unstale":"2024-05-27T07:43:00.193523+0000","last_undegraded":"2024-05-27T07:43:00.193523+0000","last_fullsized":"2024-05-27T07:43:00.193523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:29:08.497007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011836+0000","last_change":"2024-05-27T07:41:59.842473+0000","last_active":"2024-05-27T07:43:00.011836+0000","last_peered":"2024-05-27T07:43:00.011836+0000","last_clean":"2024-05-27T07:43:00.011836+0000","last_became_active":"2024-05-27T07:41:59.841035+0000","last_became_peered":"2024-05-27T07:41:59.841035+0000","last_unstale":"2024-05-27T07:43:00.011836+0000","last_undegraded":"2024-05-27T07:43:00.011836+0000","last_fullsized":"2024-05-27T07:43:00.011836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:52.959589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.197143+0000","last_change":"2024-05-27T07:41:04.804046+0000","last_active":"2024-05-27T07:43:00.197143+0000","last_peered":"2024-05-27T07:43:00.197143+0000","last_clean":"2024-05-27T07:43:00.197143+0000","last_became_active":"2024-05-27T07:41:04.803360+0000","last_became_peered":"2024-05-27T07:41:04.803360+0000","last_unstale":"2024-05-27T07:43:00.197143+0000","last_undegraded":"2024-05-27T07:43:00.197143+0000","last_fullsized":"2024-05-27T07:43:00.197143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:17.325221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195789+0000","last_change":"2024-05-27T07:41:08.840842+0000","last_active":"2024-05-27T07:43:00.195789+0000","last_peered":"2024-05-27T07:43:00.195789+0000","last_clean":"2024-05-27T07:43:00.195789+0000","last_became_active":"2024-05-27T07:41:08.837817+0000","last_became_peered":"2024-05-27T07:41:08.837817+0000","last_unstale":"2024-05-27T07:43:00.195789+0000","last_undegraded":"2024-05-27T07:43:00.195789+0000","last_fullsized":"2024-05-27T07:43:00.195789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:53:38.130608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011303+0000","last_change":"2024-05-27T07:41:22.002988+0000","last_active":"2024-05-27T07:43:00.011303+0000","last_peered":"2024-05-27T07:43:00.011303+0000","last_clean":"2024-05-27T07:43:00.011303+0000","last_became_active":"2024-05-27T07:41:22.001141+0000","last_became_peered":"2024-05-27T07:41:22.001141+0000","last_unstale":"2024-05-27T07:43:00.011303+0000","last_undegraded":"2024-05-27T07:43:00.011303+0000","last_fullsized":"2024-05-27T07:43:00.011303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:29:17.380439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196935+0000","last_change":"2024-05-27T07:41:04.794424+0000","last_active":"2024-05-27T07:43:00.196935+0000","last_peered":"2024-05-27T07:43:00.196935+0000","last_clean":"2024-05-27T07:43:00.196935+0000","last_became_active":"2024-05-27T07:41:04.793912+0000","last_became_peered":"2024-05-27T07:41:04.793912+0000","last_unstale":"2024-05-27T07:43:00.196935+0000","last_undegraded":"2024-05-27T07:43:00.196935+0000","last_fullsized":"2024-05-27T07:43:00.196935+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:50.876891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042495+0000","last_change":"2024-05-27T07:41:08.832356+0000","last_active":"2024-05-27T07:43:00.042495+0000","last_peered":"2024-05-27T07:43:00.042495+0000","last_clean":"2024-05-27T07:43:00.042495+0000","last_became_active":"2024-05-27T07:41:08.831979+0000","last_became_peered":"2024-05-27T07:41:08.831979+0000","last_unstale":"2024-05-27T07:43:00.042495+0000","last_undegraded":"2024-05-27T07:43:00.042495+0000","last_fullsized":"2024-05-27T07:43:00.042495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:37:47.584177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012955+0000","last_change":"2024-05-27T07:41:22.004233+0000","last_active":"2024-05-27T07:43:00.012955+0000","last_peered":"2024-05-27T07:43:00.012955+0000","last_clean":"2024-05-27T07:43:00.012955+0000","last_became_active":"2024-05-27T07:41:22.003781+0000","last_became_peered":"2024-05-27T07:41:22.003781+0000","last_unstale":"2024-05-27T07:43:00.012955+0000","last_undegraded":"2024-05-27T07:43:00.012955+0000","last_fullsized":"2024-05-27T07:43:00.012955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:02:13.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":83,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:03.011393+0000","last_change":"2024-05-27T07:41:04.800373+0000","last_active":"2024-05-27T07:43:03.011393+0000","last_peered":"2024-05-27T07:43:03.011393+0000","last_clean":"2024-05-27T07:43:03.011393+0000","last_became_active":"2024-05-27T07:41:04.799930+0000","last_became_peered":"2024-05-27T07:41:04.799930+0000","last_unstale":"2024-05-27T07:43:03.011393+0000","last_undegraded":"2024-05-27T07:43:03.011393+0000","last_fullsized":"2024-05-27T07:43:03.011393+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:24.642479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196134+0000","last_change":"2024-05-27T07:41:08.830387+0000","last_active":"2024-05-27T07:43:00.196134+0000","last_peered":"2024-05-27T07:43:00.196134+0000","last_clean":"2024-05-27T07:43:00.196134+0000","last_became_active":"2024-05-27T07:41:08.829720+0000","last_became_peered":"2024-05-27T07:41:08.829720+0000","last_unstale":"2024-05-27T07:43:00.196134+0000","last_undegraded":"2024-05-27T07:43:00.196134+0000","last_fullsized":"2024-05-27T07:43:00.196134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:42.014383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013364+0000","last_change":"2024-05-27T07:41:22.005700+0000","last_active":"2024-05-27T07:43:00.013364+0000","last_peered":"2024-05-27T07:43:00.013364+0000","last_clean":"2024-05-27T07:43:00.013364+0000","last_became_active":"2024-05-27T07:41:22.003643+0000","last_became_peered":"2024-05-27T07:41:22.003643+0000","last_unstale":"2024-05-27T07:43:00.013364+0000","last_undegraded":"2024-05-27T07:43:00.013364+0000","last_fullsized":"2024-05-27T07:43:00.013364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:48.332767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.034535+0000","last_change":"2024-05-27T07:41:59.790579+0000","last_active":"2024-05-27T07:43:00.034535+0000","last_peered":"2024-05-27T07:43:00.034535+0000","last_clean":"2024-05-27T07:43:00.034535+0000","last_became_active":"2024-05-27T07:41:59.789849+0000","last_became_peered":"2024-05-27T07:41:59.789849+0000","last_unstale":"2024-05-27T07:43:00.034535+0000","last_undegraded":"2024-05-27T07:43:00.034535+0000","last_fullsized":"2024-05-27T07:43:00.034535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:24:30.564074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.121892+0000","last_change":"2024-05-27T07:43:00.121063+0000","last_active":"2024-05-27T07:43:00.121892+0000","last_peered":"2024-05-27T07:43:00.121892+0000","last_clean":"2024-05-27T07:43:00.121892+0000","last_became_active":"2024-05-27T07:43:00.119977+0000","last_became_peered":"2024-05-27T07:43:00.119977+0000","last_unstale":"2024-05-27T07:43:00.121892+0000","last_undegraded":"2024-05-27T07:43:00.121892+0000","last_fullsized":"2024-05-27T07:43:00.121892+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:04:43.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195101+0000","last_change":"2024-05-27T07:41:22.013820+0000","last_active":"2024-05-27T07:43:00.195101+0000","last_peered":"2024-05-27T07:43:00.195101+0000","last_clean":"2024-05-27T07:43:00.195101+0000","last_became_active":"2024-05-27T07:41:22.013110+0000","last_became_peered":"2024-05-27T07:41:22.013110+0000","last_unstale":"2024-05-27T07:43:00.195101+0000","last_undegraded":"2024-05-27T07:43:00.195101+0000","last_fullsized":"2024-05-27T07:43:00.195101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:45:07.317872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018203+0000","last_change":"2024-05-27T07:41:04.789197+0000","last_active":"2024-05-27T07:43:00.018203+0000","last_peered":"2024-05-27T07:43:00.018203+0000","last_clean":"2024-05-27T07:43:00.018203+0000","last_became_active":"2024-05-27T07:41:04.788755+0000","last_became_peered":"2024-05-27T07:41:04.788755+0000","last_unstale":"2024-05-27T07:43:00.018203+0000","last_undegraded":"2024-05-27T07:43:00.018203+0000","last_fullsized":"2024-05-27T07:43:00.018203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:27:54.526910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.116780+0000","last_change":"2024-05-27T07:41:08.841769+0000","last_active":"2024-05-27T07:43:00.116780+0000","last_peered":"2024-05-27T07:43:00.116780+0000","last_clean":"2024-05-27T07:43:00.116780+0000","last_became_active":"2024-05-27T07:41:08.840834+0000","last_became_peered":"2024-05-27T07:41:08.840834+0000","last_unstale":"2024-05-27T07:43:00.116780+0000","last_undegraded":"2024-05-27T07:43:00.116780+0000","last_fullsized":"2024-05-27T07:43:00.116780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:15.573739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193413+0000","last_change":"2024-05-27T07:41:22.005119+0000","last_active":"2024-05-27T07:43:00.193413+0000","last_peered":"2024-05-27T07:43:00.193413+0000","last_clean":"2024-05-27T07:43:00.193413+0000","last_became_active":"2024-05-27T07:41:22.002638+0000","last_became_peered":"2024-05-27T07:41:22.002638+0000","last_unstale":"2024-05-27T07:43:00.193413+0000","last_undegraded":"2024-05-27T07:43:00.193413+0000","last_fullsized":"2024-05-27T07:43:00.193413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:08:49.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041462+0000","last_change":"2024-05-27T07:41:04.799864+0000","last_active":"2024-05-27T07:43:00.041462+0000","last_peered":"2024-05-27T07:43:00.041462+0000","last_clean":"2024-05-27T07:43:00.041462+0000","last_became_active":"2024-05-27T07:41:04.799218+0000","last_became_peered":"2024-05-27T07:41:04.799218+0000","last_unstale":"2024-05-27T07:43:00.041462+0000","last_undegraded":"2024-05-27T07:43:00.041462+0000","last_fullsized":"2024-05-27T07:43:00.041462+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:15.355720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.008861+0000","last_change":"2024-05-27T07:41:08.823387+0000","last_active":"2024-05-27T07:43:00.008861+0000","last_peered":"2024-05-27T07:43:00.008861+0000","last_clean":"2024-05-27T07:43:00.008861+0000","last_became_active":"2024-05-27T07:41:08.822943+0000","last_became_peered":"2024-05-27T07:41:08.822943+0000","last_unstale":"2024-05-27T07:43:00.008861+0000","last_undegraded":"2024-05-27T07:43:00.008861+0000","last_fullsized":"2024-05-27T07:43:00.008861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:46.365284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117228+0000","last_change":"2024-05-27T07:41:22.011848+0000","last_active":"2024-05-27T07:43:00.117228+0000","last_peered":"2024-05-27T07:43:00.117228+0000","last_clean":"2024-05-27T07:43:00.117228+0000","last_became_active":"2024-05-27T07:41:22.011335+0000","last_became_peered":"2024-05-27T07:41:22.011335+0000","last_unstale":"2024-05-27T07:43:00.117228+0000","last_undegraded":"2024-05-27T07:43:00.117228+0000","last_fullsized":"2024-05-27T07:43:00.117228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:29:22.528270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:04.013056+0000","last_change":"2024-05-27T07:41:04.793877+0000","last_active":"2024-05-27T07:43:04.013056+0000","last_peered":"2024-05-27T07:43:04.013056+0000","last_clean":"2024-05-27T07:43:04.013056+0000","last_became_active":"2024-05-27T07:41:04.793389+0000","last_became_peered":"2024-05-27T07:41:04.793389+0000","last_unstale":"2024-05-27T07:43:04.013056+0000","last_undegraded":"2024-05-27T07:43:04.013056+0000","last_fullsized":"2024-05-27T07:43:04.013056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:48.670572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":60,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196685+0000","last_change":"2024-05-27T07:41:08.836682+0000","last_active":"2024-05-27T07:43:00.196685+0000","last_peered":"2024-05-27T07:43:00.196685+0000","last_clean":"2024-05-27T07:43:00.196685+0000","last_became_active":"2024-05-27T07:41:08.836229+0000","last_became_peered":"2024-05-27T07:41:08.836229+0000","last_unstale":"2024-05-27T07:43:00.196685+0000","last_undegraded":"2024-05-27T07:43:00.196685+0000","last_fullsized":"2024-05-27T07:43:00.196685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:33.249584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.008774+0000","last_change":"2024-05-27T07:41:22.004968+0000","last_active":"2024-05-27T07:43:00.008774+0000","last_peered":"2024-05-27T07:43:00.008774+0000","last_clean":"2024-05-27T07:43:00.008774+0000","last_became_active":"2024-05-27T07:41:22.001407+0000","last_became_peered":"2024-05-27T07:41:22.001407+0000","last_unstale":"2024-05-27T07:43:00.008774+0000","last_undegraded":"2024-05-27T07:43:00.008774+0000","last_fullsized":"2024-05-27T07:43:00.008774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:26.872497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195646+0000","last_change":"2024-05-27T07:41:59.787247+0000","last_active":"2024-05-27T07:43:00.195646+0000","last_peered":"2024-05-27T07:43:00.195646+0000","last_clean":"2024-05-27T07:43:00.195646+0000","last_became_active":"2024-05-27T07:41:59.786392+0000","last_became_peered":"2024-05-27T07:41:59.786392+0000","last_unstale":"2024-05-27T07:43:00.195646+0000","last_undegraded":"2024-05-27T07:43:00.195646+0000","last_fullsized":"2024-05-27T07:43:00.195646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:12.579707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2],"acting":[11,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193565+0000","last_change":"2024-05-27T07:41:08.833731+0000","last_active":"2024-05-27T07:43:00.193565+0000","last_peered":"2024-05-27T07:43:00.193565+0000","last_clean":"2024-05-27T07:43:00.193565+0000","last_became_active":"2024-05-27T07:41:08.833318+0000","last_became_peered":"2024-05-27T07:41:08.833318+0000","last_unstale":"2024-05-27T07:43:00.193565+0000","last_undegraded":"2024-05-27T07:43:00.193565+0000","last_fullsized":"2024-05-27T07:43:00.193565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:50.085113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012266+0000","last_change":"2024-05-27T07:41:59.789784+0000","last_active":"2024-05-27T07:43:00.012266+0000","last_peered":"2024-05-27T07:43:00.012266+0000","last_clean":"2024-05-27T07:43:00.012266+0000","last_became_active":"2024-05-27T07:41:59.788589+0000","last_became_peered":"2024-05-27T07:41:59.788589+0000","last_unstale":"2024-05-27T07:43:00.012266+0000","last_undegraded":"2024-05-27T07:43:00.012266+0000","last_fullsized":"2024-05-27T07:43:00.012266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:37.026907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042336+0000","last_change":"2024-05-27T07:41:04.892603+0000","last_active":"2024-05-27T07:43:00.042336+0000","last_peered":"2024-05-27T07:43:00.042336+0000","last_clean":"2024-05-27T07:43:00.042336+0000","last_became_active":"2024-05-27T07:41:04.890786+0000","last_became_peered":"2024-05-27T07:41:04.890786+0000","last_unstale":"2024-05-27T07:43:00.042336+0000","last_undegraded":"2024-05-27T07:43:00.042336+0000","last_fullsized":"2024-05-27T07:43:00.042336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:34.129227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196274+0000","last_change":"2024-05-27T07:41:08.830802+0000","last_active":"2024-05-27T07:43:00.196274+0000","last_peered":"2024-05-27T07:43:00.196274+0000","last_clean":"2024-05-27T07:43:00.196274+0000","last_became_active":"2024-05-27T07:41:08.829661+0000","last_became_peered":"2024-05-27T07:41:08.829661+0000","last_unstale":"2024-05-27T07:43:00.196274+0000","last_undegraded":"2024-05-27T07:43:00.196274+0000","last_fullsized":"2024-05-27T07:43:00.196274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:20:15.041554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009055+0000","last_change":"2024-05-27T07:41:22.003997+0000","last_active":"2024-05-27T07:43:00.009055+0000","last_peered":"2024-05-27T07:43:00.009055+0000","last_clean":"2024-05-27T07:43:00.009055+0000","last_became_active":"2024-05-27T07:41:22.003527+0000","last_became_peered":"2024-05-27T07:41:22.003527+0000","last_unstale":"2024-05-27T07:43:00.009055+0000","last_undegraded":"2024-05-27T07:43:00.009055+0000","last_fullsized":"2024-05-27T07:43:00.009055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:17.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":165,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:04.055838+0000","last_change":"2024-05-27T07:41:59.814227+0000","last_active":"2024-05-27T07:43:04.055838+0000","last_peered":"2024-05-27T07:43:04.055838+0000","last_clean":"2024-05-27T07:43:04.055838+0000","last_became_active":"2024-05-27T07:41:59.788510+0000","last_became_peered":"2024-05-27T07:41:59.788510+0000","last_unstale":"2024-05-27T07:43:04.055838+0000","last_undegraded":"2024-05-27T07:43:04.055838+0000","last_fullsized":"2024-05-27T07:43:04.055838+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:15.868348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65382,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":98,"num_write":38,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196932+0000","last_change":"2024-05-27T07:41:08.831639+0000","last_active":"2024-05-27T07:43:00.196932+0000","last_peered":"2024-05-27T07:43:00.196932+0000","last_clean":"2024-05-27T07:43:00.196932+0000","last_became_active":"2024-05-27T07:41:08.830541+0000","last_became_peered":"2024-05-27T07:41:08.830541+0000","last_unstale":"2024-05-27T07:43:00.196932+0000","last_undegraded":"2024-05-27T07:43:00.196932+0000","last_fullsized":"2024-05-27T07:43:00.196932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:45.635145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.043148+0000","last_change":"2024-05-27T07:41:21.999587+0000","last_active":"2024-05-27T07:43:00.043148+0000","last_peered":"2024-05-27T07:43:00.043148+0000","last_clean":"2024-05-27T07:43:00.043148+0000","last_became_active":"2024-05-27T07:41:21.999017+0000","last_became_peered":"2024-05-27T07:41:21.999017+0000","last_unstale":"2024-05-27T07:43:00.043148+0000","last_undegraded":"2024-05-27T07:43:00.043148+0000","last_fullsized":"2024-05-27T07:43:00.043148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:31:14.915229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013594+0000","last_change":"2024-05-27T07:41:04.799847+0000","last_active":"2024-05-27T07:43:00.013594+0000","last_peered":"2024-05-27T07:43:00.013594+0000","last_clean":"2024-05-27T07:43:00.013594+0000","last_became_active":"2024-05-27T07:41:04.798605+0000","last_became_peered":"2024-05-27T07:41:04.798605+0000","last_unstale":"2024-05-27T07:43:00.013594+0000","last_undegraded":"2024-05-27T07:43:00.013594+0000","last_fullsized":"2024-05-27T07:43:00.013594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:01.957707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.016975+0000","last_change":"2024-05-27T07:41:08.831138+0000","last_active":"2024-05-27T07:43:00.016975+0000","last_peered":"2024-05-27T07:43:00.016975+0000","last_clean":"2024-05-27T07:43:00.016975+0000","last_became_active":"2024-05-27T07:41:08.830735+0000","last_became_peered":"2024-05-27T07:41:08.830735+0000","last_unstale":"2024-05-27T07:43:00.016975+0000","last_undegraded":"2024-05-27T07:43:00.016975+0000","last_fullsized":"2024-05-27T07:43:00.016975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:12.569687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194454+0000","last_change":"2024-05-27T07:41:59.779039+0000","last_active":"2024-05-27T07:43:00.194454+0000","last_peered":"2024-05-27T07:43:00.194454+0000","last_clean":"2024-05-27T07:43:00.194454+0000","last_became_active":"2024-05-27T07:41:59.777514+0000","last_became_peered":"2024-05-27T07:41:59.777514+0000","last_unstale":"2024-05-27T07:43:00.194454+0000","last_undegraded":"2024-05-27T07:43:00.194454+0000","last_fullsized":"2024-05-27T07:43:00.194454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:12:19.223874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041902+0000","last_change":"2024-05-27T07:41:04.796467+0000","last_active":"2024-05-27T07:43:00.041902+0000","last_peered":"2024-05-27T07:43:00.041902+0000","last_clean":"2024-05-27T07:43:00.041902+0000","last_became_active":"2024-05-27T07:41:04.795519+0000","last_became_peered":"2024-05-27T07:41:04.795519+0000","last_unstale":"2024-05-27T07:43:00.041902+0000","last_undegraded":"2024-05-27T07:43:00.041902+0000","last_fullsized":"2024-05-27T07:43:00.041902+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:42.744076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195110+0000","last_change":"2024-05-27T07:41:08.832135+0000","last_active":"2024-05-27T07:43:00.195110+0000","last_peered":"2024-05-27T07:43:00.195110+0000","last_clean":"2024-05-27T07:43:00.195110+0000","last_became_active":"2024-05-27T07:41:08.830032+0000","last_became_peered":"2024-05-27T07:41:08.830032+0000","last_unstale":"2024-05-27T07:43:00.195110+0000","last_undegraded":"2024-05-27T07:43:00.195110+0000","last_fullsized":"2024-05-27T07:43:00.195110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:44:59.367026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009771+0000","last_change":"2024-05-27T07:41:22.008868+0000","last_active":"2024-05-27T07:43:00.009771+0000","last_peered":"2024-05-27T07:43:00.009771+0000","last_clean":"2024-05-27T07:43:00.009771+0000","last_became_active":"2024-05-27T07:41:22.002071+0000","last_became_peered":"2024-05-27T07:41:22.002071+0000","last_unstale":"2024-05-27T07:43:00.009771+0000","last_undegraded":"2024-05-27T07:43:00.009771+0000","last_fullsized":"2024-05-27T07:43:00.009771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:20.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":83,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:03.031103+0000","last_change":"2024-05-27T07:41:04.796198+0000","last_active":"2024-05-27T07:43:03.031103+0000","last_peered":"2024-05-27T07:43:03.031103+0000","last_clean":"2024-05-27T07:43:03.031103+0000","last_became_active":"2024-05-27T07:41:04.795646+0000","last_became_peered":"2024-05-27T07:41:04.795646+0000","last_unstale":"2024-05-27T07:43:03.031103+0000","last_undegraded":"2024-05-27T07:43:03.031103+0000","last_fullsized":"2024-05-27T07:43:03.031103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:30.936596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195103+0000","last_change":"2024-05-27T07:41:08.840518+0000","last_active":"2024-05-27T07:43:00.195103+0000","last_peered":"2024-05-27T07:43:00.195103+0000","last_clean":"2024-05-27T07:43:00.195103+0000","last_became_active":"2024-05-27T07:41:08.840162+0000","last_became_peered":"2024-05-27T07:41:08.840162+0000","last_unstale":"2024-05-27T07:43:00.195103+0000","last_undegraded":"2024-05-27T07:43:00.195103+0000","last_fullsized":"2024-05-27T07:43:00.195103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:04:14.657496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018855+0000","last_change":"2024-05-27T07:41:22.000243+0000","last_active":"2024-05-27T07:43:00.018855+0000","last_peered":"2024-05-27T07:43:00.018855+0000","last_clean":"2024-05-27T07:43:00.018855+0000","last_became_active":"2024-05-27T07:41:21.998898+0000","last_became_peered":"2024-05-27T07:41:21.998898+0000","last_unstale":"2024-05-27T07:43:00.018855+0000","last_undegraded":"2024-05-27T07:43:00.018855+0000","last_fullsized":"2024-05-27T07:43:00.018855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:35.696593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195302+0000","last_change":"2024-05-27T07:41:59.799662+0000","last_active":"2024-05-27T07:43:00.195302+0000","last_peered":"2024-05-27T07:43:00.195302+0000","last_clean":"2024-05-27T07:43:00.195302+0000","last_became_active":"2024-05-27T07:41:59.779733+0000","last_became_peered":"2024-05-27T07:41:59.779733+0000","last_unstale":"2024-05-27T07:43:00.195302+0000","last_undegraded":"2024-05-27T07:43:00.195302+0000","last_fullsized":"2024-05-27T07:43:00.195302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:23.665014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041865+0000","last_change":"2024-05-27T07:41:08.834931+0000","last_active":"2024-05-27T07:43:00.041865+0000","last_peered":"2024-05-27T07:43:00.041865+0000","last_clean":"2024-05-27T07:43:00.041865+0000","last_became_active":"2024-05-27T07:41:08.833298+0000","last_became_peered":"2024-05-27T07:41:08.833298+0000","last_unstale":"2024-05-27T07:43:00.041865+0000","last_undegraded":"2024-05-27T07:43:00.041865+0000","last_fullsized":"2024-05-27T07:43:00.041865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:03:48.507461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041766+0000","last_change":"2024-05-27T07:41:22.009407+0000","last_active":"2024-05-27T07:43:00.041766+0000","last_peered":"2024-05-27T07:43:00.041766+0000","last_clean":"2024-05-27T07:43:00.041766+0000","last_became_active":"2024-05-27T07:41:22.006921+0000","last_became_peered":"2024-05-27T07:41:22.006921+0000","last_unstale":"2024-05-27T07:43:00.041766+0000","last_undegraded":"2024-05-27T07:43:00.041766+0000","last_fullsized":"2024-05-27T07:43:00.041766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:24.034502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196436+0000","last_change":"2024-05-27T07:41:04.795641+0000","last_active":"2024-05-27T07:43:00.196436+0000","last_peered":"2024-05-27T07:43:00.196436+0000","last_clean":"2024-05-27T07:43:00.196436+0000","last_became_active":"2024-05-27T07:41:04.794935+0000","last_became_peered":"2024-05-27T07:41:04.794935+0000","last_unstale":"2024-05-27T07:43:00.196436+0000","last_undegraded":"2024-05-27T07:43:00.196436+0000","last_fullsized":"2024-05-27T07:43:00.196436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:14.850282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195110+0000","last_change":"2024-05-27T07:41:08.837277+0000","last_active":"2024-05-27T07:43:00.195110+0000","last_peered":"2024-05-27T07:43:00.195110+0000","last_clean":"2024-05-27T07:43:00.195110+0000","last_became_active":"2024-05-27T07:41:08.835935+0000","last_became_peered":"2024-05-27T07:41:08.835935+0000","last_unstale":"2024-05-27T07:43:00.195110+0000","last_undegraded":"2024-05-27T07:43:00.195110+0000","last_fullsized":"2024-05-27T07:43:00.195110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:46.436243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196061+0000","last_change":"2024-05-27T07:41:22.014700+0000","last_active":"2024-05-27T07:43:00.196061+0000","last_peered":"2024-05-27T07:43:00.196061+0000","last_clean":"2024-05-27T07:43:00.196061+0000","last_became_active":"2024-05-27T07:41:22.014136+0000","last_became_peered":"2024-05-27T07:41:22.014136+0000","last_unstale":"2024-05-27T07:43:00.196061+0000","last_undegraded":"2024-05-27T07:43:00.196061+0000","last_fullsized":"2024-05-27T07:43:00.196061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:51:08.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009324+0000","last_change":"2024-05-27T07:41:04.891269+0000","last_active":"2024-05-27T07:43:00.009324+0000","last_peered":"2024-05-27T07:43:00.009324+0000","last_clean":"2024-05-27T07:43:00.009324+0000","last_became_active":"2024-05-27T07:41:04.890843+0000","last_became_peered":"2024-05-27T07:41:04.890843+0000","last_unstale":"2024-05-27T07:43:00.009324+0000","last_undegraded":"2024-05-27T07:43:00.009324+0000","last_fullsized":"2024-05-27T07:43:00.009324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:57.065300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.035249+0000","last_change":"2024-05-27T07:41:08.841977+0000","last_active":"2024-05-27T07:43:00.035249+0000","last_peered":"2024-05-27T07:43:00.035249+0000","last_clean":"2024-05-27T07:43:00.035249+0000","last_became_active":"2024-05-27T07:41:08.841551+0000","last_became_peered":"2024-05-27T07:41:08.841551+0000","last_unstale":"2024-05-27T07:43:00.035249+0000","last_undegraded":"2024-05-27T07:43:00.035249+0000","last_fullsized":"2024-05-27T07:43:00.035249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:35:07.766193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.020134+0000","last_change":"2024-05-27T07:41:22.006310+0000","last_active":"2024-05-27T07:43:00.020134+0000","last_peered":"2024-05-27T07:43:00.020134+0000","last_clean":"2024-05-27T07:43:00.020134+0000","last_became_active":"2024-05-27T07:41:22.004135+0000","last_became_peered":"2024-05-27T07:41:22.004135+0000","last_unstale":"2024-05-27T07:43:00.020134+0000","last_undegraded":"2024-05-27T07:43:00.020134+0000","last_fullsized":"2024-05-27T07:43:00.020134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:48.564894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":80,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:03.004783+0000","last_change":"2024-05-27T07:41:04.793284+0000","last_active":"2024-05-27T07:43:03.004783+0000","last_peered":"2024-05-27T07:43:03.004783+0000","last_clean":"2024-05-27T07:43:03.004783+0000","last_became_active":"2024-05-27T07:41:04.792853+0000","last_became_peered":"2024-05-27T07:41:04.792853+0000","last_unstale":"2024-05-27T07:43:03.004783+0000","last_undegraded":"2024-05-27T07:43:03.004783+0000","last_fullsized":"2024-05-27T07:43:03.004783+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:22.979341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.034751+0000","last_change":"2024-05-27T07:41:59.776207+0000","last_active":"2024-05-27T07:43:00.034751+0000","last_peered":"2024-05-27T07:43:00.034751+0000","last_clean":"2024-05-27T07:43:00.034751+0000","last_became_active":"2024-05-27T07:41:59.775226+0000","last_became_peered":"2024-05-27T07:41:59.775226+0000","last_unstale":"2024-05-27T07:43:00.034751+0000","last_undegraded":"2024-05-27T07:43:00.034751+0000","last_fullsized":"2024-05-27T07:43:00.034751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:50.469144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.020637+0000","last_change":"2024-05-27T07:41:21.992013+0000","last_active":"2024-05-27T07:43:00.020637+0000","last_peered":"2024-05-27T07:43:00.020637+0000","last_clean":"2024-05-27T07:43:00.020637+0000","last_became_active":"2024-05-27T07:41:21.991550+0000","last_became_peered":"2024-05-27T07:41:21.991550+0000","last_unstale":"2024-05-27T07:43:00.020637+0000","last_undegraded":"2024-05-27T07:43:00.020637+0000","last_fullsized":"2024-05-27T07:43:00.020637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:34:53.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042195+0000","last_change":"2024-05-27T07:41:08.832512+0000","last_active":"2024-05-27T07:43:00.042195+0000","last_peered":"2024-05-27T07:43:00.042195+0000","last_clean":"2024-05-27T07:43:00.042195+0000","last_became_active":"2024-05-27T07:41:08.832113+0000","last_became_peered":"2024-05-27T07:41:08.832113+0000","last_unstale":"2024-05-27T07:43:00.042195+0000","last_undegraded":"2024-05-27T07:43:00.042195+0000","last_fullsized":"2024-05-27T07:43:00.042195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:00.530785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":82,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:05.036177+0000","last_change":"2024-05-27T07:41:04.891004+0000","last_active":"2024-05-27T07:43:05.036177+0000","last_peered":"2024-05-27T07:43:05.036177+0000","last_clean":"2024-05-27T07:43:05.036177+0000","last_became_active":"2024-05-27T07:41:04.890567+0000","last_became_peered":"2024-05-27T07:41:04.890567+0000","last_unstale":"2024-05-27T07:43:05.036177+0000","last_undegraded":"2024-05-27T07:43:05.036177+0000","last_fullsized":"2024-05-27T07:43:05.036177+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:43.991204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196027+0000","last_change":"2024-05-27T07:41:22.003446+0000","last_active":"2024-05-27T07:43:00.196027+0000","last_peered":"2024-05-27T07:43:00.196027+0000","last_clean":"2024-05-27T07:43:00.196027+0000","last_became_active":"2024-05-27T07:41:22.002989+0000","last_became_peered":"2024-05-27T07:41:22.002989+0000","last_unstale":"2024-05-27T07:43:00.196027+0000","last_undegraded":"2024-05-27T07:43:00.196027+0000","last_fullsized":"2024-05-27T07:43:00.196027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:12.996286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117982+0000","last_change":"2024-05-27T07:41:08.842685+0000","last_active":"2024-05-27T07:43:00.117982+0000","last_peered":"2024-05-27T07:43:00.117982+0000","last_clean":"2024-05-27T07:43:00.117982+0000","last_became_active":"2024-05-27T07:41:08.841954+0000","last_became_peered":"2024-05-27T07:41:08.841954+0000","last_unstale":"2024-05-27T07:43:00.117982+0000","last_undegraded":"2024-05-27T07:43:00.117982+0000","last_fullsized":"2024-05-27T07:43:00.117982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:18.087129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013075+0000","last_change":"2024-05-27T07:41:04.790330+0000","last_active":"2024-05-27T07:43:00.013075+0000","last_peered":"2024-05-27T07:43:00.013075+0000","last_clean":"2024-05-27T07:43:00.013075+0000","last_became_active":"2024-05-27T07:41:04.789508+0000","last_became_peered":"2024-05-27T07:41:04.789508+0000","last_unstale":"2024-05-27T07:43:00.013075+0000","last_undegraded":"2024-05-27T07:43:00.013075+0000","last_fullsized":"2024-05-27T07:43:00.013075+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:27.537662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194381+0000","last_change":"2024-05-27T07:39:11.797324+0000","last_active":"2024-05-27T07:43:00.194381+0000","last_peered":"2024-05-27T07:43:00.194381+0000","last_clean":"2024-05-27T07:43:00.194381+0000","last_became_active":"2024-05-27T07:39:11.786294+0000","last_became_peered":"2024-05-27T07:39:11.786294+0000","last_unstale":"2024-05-27T07:43:00.194381+0000","last_undegraded":"2024-05-27T07:43:00.194381+0000","last_fullsized":"2024-05-27T07:43:00.194381+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_clean_scrub_stamp":"2024-05-27T07:34:18.613413+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:23.947615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042589+0000","last_change":"2024-05-27T07:41:21.997453+0000","last_active":"2024-05-27T07:43:00.042589+0000","last_peered":"2024-05-27T07:43:00.042589+0000","last_clean":"2024-05-27T07:43:00.042589+0000","last_became_active":"2024-05-27T07:41:21.997010+0000","last_became_peered":"2024-05-27T07:41:21.997010+0000","last_unstale":"2024-05-27T07:43:00.042589+0000","last_undegraded":"2024-05-27T07:43:00.042589+0000","last_fullsized":"2024-05-27T07:43:00.042589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:54:54.913274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.024988+0000","last_change":"2024-05-27T07:43:00.024988+0000","last_active":"2024-05-27T07:43:00.024988+0000","last_peered":"2024-05-27T07:43:00.024988+0000","last_clean":"2024-05-27T07:43:00.024988+0000","last_became_active":"2024-05-27T07:43:00.023816+0000","last_became_peered":"2024-05-27T07:43:00.023816+0000","last_unstale":"2024-05-27T07:43:00.024988+0000","last_undegraded":"2024-05-27T07:43:00.024988+0000","last_fullsized":"2024-05-27T07:43:00.024988+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:58.531985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194923+0000","last_change":"2024-05-27T07:41:04.795262+0000","last_active":"2024-05-27T07:43:00.194923+0000","last_peered":"2024-05-27T07:43:00.194923+0000","last_clean":"2024-05-27T07:43:00.194923+0000","last_became_active":"2024-05-27T07:41:04.794866+0000","last_became_peered":"2024-05-27T07:41:04.794866+0000","last_unstale":"2024-05-27T07:43:00.194923+0000","last_undegraded":"2024-05-27T07:43:00.194923+0000","last_fullsized":"2024-05-27T07:43:00.194923+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:07:44.241444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194359+0000","last_change":"2024-05-27T07:41:22.006194+0000","last_active":"2024-05-27T07:43:00.194359+0000","last_peered":"2024-05-27T07:43:00.194359+0000","last_clean":"2024-05-27T07:43:00.194359+0000","last_became_active":"2024-05-27T07:41:22.004079+0000","last_became_peered":"2024-05-27T07:41:22.004079+0000","last_unstale":"2024-05-27T07:43:00.194359+0000","last_undegraded":"2024-05-27T07:43:00.194359+0000","last_fullsized":"2024-05-27T07:43:00.194359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:03.496722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194277+0000","last_change":"2024-05-27T07:41:59.776734+0000","last_active":"2024-05-27T07:43:00.194277+0000","last_peered":"2024-05-27T07:43:00.194277+0000","last_clean":"2024-05-27T07:43:00.194277+0000","last_became_active":"2024-05-27T07:41:59.775678+0000","last_became_peered":"2024-05-27T07:41:59.775678+0000","last_unstale":"2024-05-27T07:43:00.194277+0000","last_undegraded":"2024-05-27T07:43:00.194277+0000","last_fullsized":"2024-05-27T07:43:00.194277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:56.007178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.020635+0000","last_change":"2024-05-27T07:41:08.831070+0000","last_active":"2024-05-27T07:43:00.020635+0000","last_peered":"2024-05-27T07:43:00.020635+0000","last_clean":"2024-05-27T07:43:00.020635+0000","last_became_active":"2024-05-27T07:41:08.830617+0000","last_became_peered":"2024-05-27T07:41:08.830617+0000","last_unstale":"2024-05-27T07:43:00.020635+0000","last_undegraded":"2024-05-27T07:43:00.020635+0000","last_fullsized":"2024-05-27T07:43:00.020635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:13.046162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.116649+0000","last_change":"2024-05-27T07:41:22.010129+0000","last_active":"2024-05-27T07:43:00.116649+0000","last_peered":"2024-05-27T07:43:00.116649+0000","last_clean":"2024-05-27T07:43:00.116649+0000","last_became_active":"2024-05-27T07:41:22.009608+0000","last_became_peered":"2024-05-27T07:41:22.009608+0000","last_unstale":"2024-05-27T07:43:00.116649+0000","last_undegraded":"2024-05-27T07:43:00.116649+0000","last_fullsized":"2024-05-27T07:43:00.116649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.225004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'16","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.010571+0000","last_change":"2024-05-27T07:41:04.794744+0000","last_active":"2024-05-27T07:43:00.010571+0000","last_peered":"2024-05-27T07:43:00.010571+0000","last_clean":"2024-05-27T07:43:00.010571+0000","last_became_active":"2024-05-27T07:41:04.794340+0000","last_became_peered":"2024-05-27T07:41:04.794340+0000","last_unstale":"2024-05-27T07:43:00.010571+0000","last_undegraded":"2024-05-27T07:43:00.010571+0000","last_fullsized":"2024-05-27T07:43:00.010571+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:43:20.335520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.119096+0000","last_change":"2024-05-27T07:41:08.841834+0000","last_active":"2024-05-27T07:43:00.119096+0000","last_peered":"2024-05-27T07:43:00.119096+0000","last_clean":"2024-05-27T07:43:00.119096+0000","last_became_active":"2024-05-27T07:41:08.840261+0000","last_became_peered":"2024-05-27T07:41:08.840261+0000","last_unstale":"2024-05-27T07:43:00.119096+0000","last_undegraded":"2024-05-27T07:43:00.119096+0000","last_fullsized":"2024-05-27T07:43:00.119096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:33.738835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011925+0000","last_change":"2024-05-27T07:41:22.005861+0000","last_active":"2024-05-27T07:43:00.011925+0000","last_peered":"2024-05-27T07:43:00.011925+0000","last_clean":"2024-05-27T07:43:00.011925+0000","last_became_active":"2024-05-27T07:41:22.005295+0000","last_became_peered":"2024-05-27T07:41:22.005295+0000","last_unstale":"2024-05-27T07:43:00.011925+0000","last_undegraded":"2024-05-27T07:43:00.011925+0000","last_fullsized":"2024-05-27T07:43:00.011925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:46.582656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013441+0000","last_change":"2024-05-27T07:41:04.796562+0000","last_active":"2024-05-27T07:43:00.013441+0000","last_peered":"2024-05-27T07:43:00.013441+0000","last_clean":"2024-05-27T07:43:00.013441+0000","last_became_active":"2024-05-27T07:41:04.795538+0000","last_became_peered":"2024-05-27T07:41:04.795538+0000","last_unstale":"2024-05-27T07:43:00.013441+0000","last_undegraded":"2024-05-27T07:43:00.013441+0000","last_fullsized":"2024-05-27T07:43:00.013441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:12.131999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193665+0000","last_change":"2024-05-27T07:41:08.832824+0000","last_active":"2024-05-27T07:43:00.193665+0000","last_peered":"2024-05-27T07:43:00.193665+0000","last_clean":"2024-05-27T07:43:00.193665+0000","last_became_active":"2024-05-27T07:41:08.832332+0000","last_became_peered":"2024-05-27T07:41:08.832332+0000","last_unstale":"2024-05-27T07:43:00.193665+0000","last_undegraded":"2024-05-27T07:43:00.193665+0000","last_fullsized":"2024-05-27T07:43:00.193665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:34.598773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011686+0000","last_change":"2024-05-27T07:41:22.003358+0000","last_active":"2024-05-27T07:43:00.011686+0000","last_peered":"2024-05-27T07:43:00.011686+0000","last_clean":"2024-05-27T07:43:00.011686+0000","last_became_active":"2024-05-27T07:41:22.002655+0000","last_became_peered":"2024-05-27T07:41:22.002655+0000","last_unstale":"2024-05-27T07:43:00.011686+0000","last_undegraded":"2024-05-27T07:43:00.011686+0000","last_fullsized":"2024-05-27T07:43:00.011686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:41.177138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193606+0000","last_change":"2024-05-27T07:41:04.888681+0000","last_active":"2024-05-27T07:43:00.193606+0000","last_peered":"2024-05-27T07:43:00.193606+0000","last_clean":"2024-05-27T07:43:00.193606+0000","last_became_active":"2024-05-27T07:41:04.887966+0000","last_became_peered":"2024-05-27T07:41:04.887966+0000","last_unstale":"2024-05-27T07:43:00.193606+0000","last_undegraded":"2024-05-27T07:43:00.193606+0000","last_fullsized":"2024-05-27T07:43:00.193606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:01.178737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195221+0000","last_change":"2024-05-27T07:41:08.827839+0000","last_active":"2024-05-27T07:43:00.195221+0000","last_peered":"2024-05-27T07:43:00.195221+0000","last_clean":"2024-05-27T07:43:00.195221+0000","last_became_active":"2024-05-27T07:41:08.827403+0000","last_became_peered":"2024-05-27T07:41:08.827403+0000","last_unstale":"2024-05-27T07:43:00.195221+0000","last_undegraded":"2024-05-27T07:43:00.195221+0000","last_fullsized":"2024-05-27T07:43:00.195221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:01:42.075980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009996+0000","last_change":"2024-05-27T07:41:22.006161+0000","last_active":"2024-05-27T07:43:00.009996+0000","last_peered":"2024-05-27T07:43:00.009996+0000","last_clean":"2024-05-27T07:43:00.009996+0000","last_became_active":"2024-05-27T07:41:22.005762+0000","last_became_peered":"2024-05-27T07:41:22.005762+0000","last_unstale":"2024-05-27T07:43:00.009996+0000","last_undegraded":"2024-05-27T07:43:00.009996+0000","last_fullsized":"2024-05-27T07:43:00.009996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:53.536444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117166+0000","last_change":"2024-05-27T07:41:04.774211+0000","last_active":"2024-05-27T07:43:00.117166+0000","last_peered":"2024-05-27T07:43:00.117166+0000","last_clean":"2024-05-27T07:43:00.117166+0000","last_became_active":"2024-05-27T07:41:04.773215+0000","last_became_peered":"2024-05-27T07:41:04.773215+0000","last_unstale":"2024-05-27T07:43:00.117166+0000","last_undegraded":"2024-05-27T07:43:00.117166+0000","last_fullsized":"2024-05-27T07:43:00.117166+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:37.273735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193866+0000","last_change":"2024-05-27T07:41:08.830434+0000","last_active":"2024-05-27T07:43:00.193866+0000","last_peered":"2024-05-27T07:43:00.193866+0000","last_clean":"2024-05-27T07:43:00.193866+0000","last_became_active":"2024-05-27T07:41:08.830039+0000","last_became_peered":"2024-05-27T07:41:08.830039+0000","last_unstale":"2024-05-27T07:43:00.193866+0000","last_undegraded":"2024-05-27T07:43:00.193866+0000","last_fullsized":"2024-05-27T07:43:00.193866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:27.877666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012958+0000","last_change":"2024-05-27T07:41:59.783413+0000","last_active":"2024-05-27T07:43:00.012958+0000","last_peered":"2024-05-27T07:43:00.012958+0000","last_clean":"2024-05-27T07:43:00.012958+0000","last_became_active":"2024-05-27T07:41:59.782300+0000","last_became_peered":"2024-05-27T07:41:59.782300+0000","last_unstale":"2024-05-27T07:43:00.012958+0000","last_undegraded":"2024-05-27T07:43:00.012958+0000","last_fullsized":"2024-05-27T07:43:00.012958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:59.348928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012459+0000","last_change":"2024-05-27T07:41:04.798969+0000","last_active":"2024-05-27T07:43:00.012459+0000","last_peered":"2024-05-27T07:43:00.012459+0000","last_clean":"2024-05-27T07:43:00.012459+0000","last_became_active":"2024-05-27T07:41:04.795996+0000","last_became_peered":"2024-05-27T07:41:04.795996+0000","last_unstale":"2024-05-27T07:43:00.012459+0000","last_undegraded":"2024-05-27T07:43:00.012459+0000","last_fullsized":"2024-05-27T07:43:00.012459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:47.141063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117509+0000","last_change":"2024-05-27T07:41:08.835142+0000","last_active":"2024-05-27T07:43:00.117509+0000","last_peered":"2024-05-27T07:43:00.117509+0000","last_clean":"2024-05-27T07:43:00.117509+0000","last_became_active":"2024-05-27T07:41:08.834762+0000","last_became_peered":"2024-05-27T07:41:08.834762+0000","last_unstale":"2024-05-27T07:43:00.117509+0000","last_undegraded":"2024-05-27T07:43:00.117509+0000","last_fullsized":"2024-05-27T07:43:00.117509+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:52.957554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012271+0000","last_change":"2024-05-27T07:41:22.000649+0000","last_active":"2024-05-27T07:43:00.012271+0000","last_peered":"2024-05-27T07:43:00.012271+0000","last_clean":"2024-05-27T07:43:00.012271+0000","last_became_active":"2024-05-27T07:41:22.000161+0000","last_became_peered":"2024-05-27T07:41:22.000161+0000","last_unstale":"2024-05-27T07:43:00.012271+0000","last_undegraded":"2024-05-27T07:43:00.012271+0000","last_fullsized":"2024-05-27T07:43:00.012271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:10.788027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195761+0000","last_change":"2024-05-27T07:41:04.800680+0000","last_active":"2024-05-27T07:43:00.195761+0000","last_peered":"2024-05-27T07:43:00.195761+0000","last_clean":"2024-05-27T07:43:00.195761+0000","last_became_active":"2024-05-27T07:41:04.800249+0000","last_became_peered":"2024-05-27T07:41:04.800249+0000","last_unstale":"2024-05-27T07:43:00.195761+0000","last_undegraded":"2024-05-27T07:43:00.195761+0000","last_fullsized":"2024-05-27T07:43:00.195761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:23:27.797216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195250+0000","last_change":"2024-05-27T07:41:08.844380+0000","last_active":"2024-05-27T07:43:00.195250+0000","last_peered":"2024-05-27T07:43:00.195250+0000","last_clean":"2024-05-27T07:43:00.195250+0000","last_became_active":"2024-05-27T07:41:08.843464+0000","last_became_peered":"2024-05-27T07:41:08.843464+0000","last_unstale":"2024-05-27T07:43:00.195250+0000","last_undegraded":"2024-05-27T07:43:00.195250+0000","last_fullsized":"2024-05-27T07:43:00.195250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:23.290234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194261+0000","last_change":"2024-05-27T07:41:22.010968+0000","last_active":"2024-05-27T07:43:00.194261+0000","last_peered":"2024-05-27T07:43:00.194261+0000","last_clean":"2024-05-27T07:43:00.194261+0000","last_became_active":"2024-05-27T07:41:22.010529+0000","last_became_peered":"2024-05-27T07:41:22.010529+0000","last_unstale":"2024-05-27T07:43:00.194261+0000","last_undegraded":"2024-05-27T07:43:00.194261+0000","last_fullsized":"2024-05-27T07:43:00.194261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:08.636434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":83,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:05.018220+0000","last_change":"2024-05-27T07:41:04.891536+0000","last_active":"2024-05-27T07:43:05.018220+0000","last_peered":"2024-05-27T07:43:05.018220+0000","last_clean":"2024-05-27T07:43:05.018220+0000","last_became_active":"2024-05-27T07:41:04.890724+0000","last_became_peered":"2024-05-27T07:41:04.890724+0000","last_unstale":"2024-05-27T07:43:05.018220+0000","last_undegraded":"2024-05-27T07:43:05.018220+0000","last_fullsized":"2024-05-27T07:43:05.018220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:32.027624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196699+0000","last_change":"2024-05-27T07:41:08.834362+0000","last_active":"2024-05-27T07:43:00.196699+0000","last_peered":"2024-05-27T07:43:00.196699+0000","last_clean":"2024-05-27T07:43:00.196699+0000","last_became_active":"2024-05-27T07:41:08.833979+0000","last_became_peered":"2024-05-27T07:41:08.833979+0000","last_unstale":"2024-05-27T07:43:00.196699+0000","last_undegraded":"2024-05-27T07:43:00.196699+0000","last_fullsized":"2024-05-27T07:43:00.196699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:35.098352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.192512+0000","last_change":"2024-05-27T07:41:21.997587+0000","last_active":"2024-05-27T07:43:00.192512+0000","last_peered":"2024-05-27T07:43:00.192512+0000","last_clean":"2024-05-27T07:43:00.192512+0000","last_became_active":"2024-05-27T07:41:21.996413+0000","last_became_peered":"2024-05-27T07:41:21.996413+0000","last_unstale":"2024-05-27T07:43:00.192512+0000","last_undegraded":"2024-05-27T07:43:00.192512+0000","last_fullsized":"2024-05-27T07:43:00.192512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:21:46.728047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042883+0000","last_change":"2024-05-27T07:41:04.793580+0000","last_active":"2024-05-27T07:43:00.042883+0000","last_peered":"2024-05-27T07:43:00.042883+0000","last_clean":"2024-05-27T07:43:00.042883+0000","last_became_active":"2024-05-27T07:41:04.791135+0000","last_became_peered":"2024-05-27T07:41:04.791135+0000","last_unstale":"2024-05-27T07:43:00.042883+0000","last_undegraded":"2024-05-27T07:43:00.042883+0000","last_fullsized":"2024-05-27T07:43:00.042883+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:15.484187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195882+0000","last_change":"2024-05-27T07:41:08.839187+0000","last_active":"2024-05-27T07:43:00.195882+0000","last_peered":"2024-05-27T07:43:00.195882+0000","last_clean":"2024-05-27T07:43:00.195882+0000","last_became_active":"2024-05-27T07:41:08.838821+0000","last_became_peered":"2024-05-27T07:41:08.838821+0000","last_unstale":"2024-05-27T07:43:00.195882+0000","last_undegraded":"2024-05-27T07:43:00.195882+0000","last_fullsized":"2024-05-27T07:43:00.195882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:30.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.040945+0000","last_change":"2024-05-27T07:41:22.005406+0000","last_active":"2024-05-27T07:43:00.040945+0000","last_peered":"2024-05-27T07:43:00.040945+0000","last_clean":"2024-05-27T07:43:00.040945+0000","last_became_active":"2024-05-27T07:41:22.003415+0000","last_became_peered":"2024-05-27T07:41:22.003415+0000","last_unstale":"2024-05-27T07:43:00.040945+0000","last_undegraded":"2024-05-27T07:43:00.040945+0000","last_fullsized":"2024-05-27T07:43:00.040945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:03.300452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018465+0000","last_change":"2024-05-27T07:41:04.798489+0000","last_active":"2024-05-27T07:43:00.018465+0000","last_peered":"2024-05-27T07:43:00.018465+0000","last_clean":"2024-05-27T07:43:00.018465+0000","last_became_active":"2024-05-27T07:41:04.797467+0000","last_became_peered":"2024-05-27T07:41:04.797467+0000","last_unstale":"2024-05-27T07:43:00.018465+0000","last_undegraded":"2024-05-27T07:43:00.018465+0000","last_fullsized":"2024-05-27T07:43:00.018465+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:04.018577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195995+0000","last_change":"2024-05-27T07:41:08.832215+0000","last_active":"2024-05-27T07:43:00.195995+0000","last_peered":"2024-05-27T07:43:00.195995+0000","last_clean":"2024-05-27T07:43:00.195995+0000","last_became_active":"2024-05-27T07:41:08.831491+0000","last_became_peered":"2024-05-27T07:41:08.831491+0000","last_unstale":"2024-05-27T07:43:00.195995+0000","last_undegraded":"2024-05-27T07:43:00.195995+0000","last_fullsized":"2024-05-27T07:43:00.195995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:32:03.084324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194151+0000","last_change":"2024-05-27T07:41:08.837301+0000","last_active":"2024-05-27T07:43:00.194151+0000","last_peered":"2024-05-27T07:43:00.194151+0000","last_clean":"2024-05-27T07:43:00.194151+0000","last_became_active":"2024-05-27T07:41:08.835533+0000","last_became_peered":"2024-05-27T07:41:08.835533+0000","last_unstale":"2024-05-27T07:43:00.194151+0000","last_undegraded":"2024-05-27T07:43:00.194151+0000","last_fullsized":"2024-05-27T07:43:00.194151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:45.807613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013039+0000","last_change":"2024-05-27T07:41:08.846740+0000","last_active":"2024-05-27T07:43:00.013039+0000","last_peered":"2024-05-27T07:43:00.013039+0000","last_clean":"2024-05-27T07:43:00.013039+0000","last_became_active":"2024-05-27T07:41:08.845576+0000","last_became_peered":"2024-05-27T07:41:08.845576+0000","last_unstale":"2024-05-27T07:43:00.013039+0000","last_undegraded":"2024-05-27T07:43:00.013039+0000","last_fullsized":"2024-05-27T07:43:00.013039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:41:36.944923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012116+0000","last_change":"2024-05-27T07:41:08.840121+0000","last_active":"2024-05-27T07:43:00.012116+0000","last_peered":"2024-05-27T07:43:00.012116+0000","last_clean":"2024-05-27T07:43:00.012116+0000","last_became_active":"2024-05-27T07:41:08.839090+0000","last_became_peered":"2024-05-27T07:41:08.839090+0000","last_unstale":"2024-05-27T07:43:00.012116+0000","last_undegraded":"2024-05-27T07:43:00.012116+0000","last_fullsized":"2024-05-27T07:43:00.012116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:44.836953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193149+0000","last_change":"2024-05-27T07:41:08.835565+0000","last_active":"2024-05-27T07:43:00.193149+0000","last_peered":"2024-05-27T07:43:00.193149+0000","last_clean":"2024-05-27T07:43:00.193149+0000","last_became_active":"2024-05-27T07:41:08.831424+0000","last_became_peered":"2024-05-27T07:41:08.831424+0000","last_unstale":"2024-05-27T07:43:00.193149+0000","last_undegraded":"2024-05-27T07:43:00.193149+0000","last_fullsized":"2024-05-27T07:43:00.193149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:17.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.021085+0000","last_change":"2024-05-27T07:41:08.830104+0000","last_active":"2024-05-27T07:43:00.021085+0000","last_peered":"2024-05-27T07:43:00.021085+0000","last_clean":"2024-05-27T07:43:00.021085+0000","last_became_active":"2024-05-27T07:41:08.829690+0000","last_became_peered":"2024-05-27T07:41:08.829690+0000","last_unstale":"2024-05-27T07:43:00.021085+0000","last_undegraded":"2024-05-27T07:43:00.021085+0000","last_fullsized":"2024-05-27T07:43:00.021085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194789+0000","last_change":"2024-05-27T07:41:08.838654+0000","last_active":"2024-05-27T07:43:00.194789+0000","last_peered":"2024-05-27T07:43:00.194789+0000","last_clean":"2024-05-27T07:43:00.194789+0000","last_became_active":"2024-05-27T07:41:08.835033+0000","last_became_peered":"2024-05-27T07:41:08.835033+0000","last_unstale":"2024-05-27T07:43:00.194789+0000","last_undegraded":"2024-05-27T07:43:00.194789+0000","last_fullsized":"2024-05-27T07:43:00.194789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:04:24.007585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013026+0000","last_change":"2024-05-27T07:41:08.844557+0000","last_active":"2024-05-27T07:43:00.013026+0000","last_peered":"2024-05-27T07:43:00.013026+0000","last_clean":"2024-05-27T07:43:00.013026+0000","last_became_active":"2024-05-27T07:41:08.843418+0000","last_became_peered":"2024-05-27T07:41:08.843418+0000","last_unstale":"2024-05-27T07:43:00.013026+0000","last_undegraded":"2024-05-27T07:43:00.013026+0000","last_fullsized":"2024-05-27T07:43:00.013026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:39.587778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.119172+0000","last_change":"2024-05-27T07:41:08.830857+0000","last_active":"2024-05-27T07:43:00.119172+0000","last_peered":"2024-05-27T07:43:00.119172+0000","last_clean":"2024-05-27T07:43:00.119172+0000","last_became_active":"2024-05-27T07:41:08.830480+0000","last_became_peered":"2024-05-27T07:41:08.830480+0000","last_unstale":"2024-05-27T07:43:00.119172+0000","last_undegraded":"2024-05-27T07:43:00.119172+0000","last_fullsized":"2024-05-27T07:43:00.119172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:07.307528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041750+0000","last_change":"2024-05-27T07:41:08.824571+0000","last_active":"2024-05-27T07:43:00.041750+0000","last_peered":"2024-05-27T07:43:00.041750+0000","last_clean":"2024-05-27T07:43:00.041750+0000","last_became_active":"2024-05-27T07:41:08.820732+0000","last_became_peered":"2024-05-27T07:41:08.820732+0000","last_unstale":"2024-05-27T07:43:00.041750+0000","last_undegraded":"2024-05-27T07:43:00.041750+0000","last_fullsized":"2024-05-27T07:43:00.041750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:55:14.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117363+0000","last_change":"2024-05-27T07:41:08.842284+0000","last_active":"2024-05-27T07:43:00.117363+0000","last_peered":"2024-05-27T07:43:00.117363+0000","last_clean":"2024-05-27T07:43:00.117363+0000","last_became_active":"2024-05-27T07:41:08.841597+0000","last_became_peered":"2024-05-27T07:41:08.841597+0000","last_unstale":"2024-05-27T07:43:00.117363+0000","last_undegraded":"2024-05-27T07:43:00.117363+0000","last_fullsized":"2024-05-27T07:43:00.117363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:25:31.066181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195588+0000","last_change":"2024-05-27T07:41:08.828935+0000","last_active":"2024-05-27T07:43:00.195588+0000","last_peered":"2024-05-27T07:43:00.195588+0000","last_clean":"2024-05-27T07:43:00.195588+0000","last_became_active":"2024-05-27T07:41:08.828542+0000","last_became_peered":"2024-05-27T07:41:08.828542+0000","last_unstale":"2024-05-27T07:43:00.195588+0000","last_undegraded":"2024-05-27T07:43:00.195588+0000","last_fullsized":"2024-05-27T07:43:00.195588+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:35.270666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.017962+0000","last_change":"2024-05-27T07:41:08.817051+0000","last_active":"2024-05-27T07:43:00.017962+0000","last_peered":"2024-05-27T07:43:00.017962+0000","last_clean":"2024-05-27T07:43:00.017962+0000","last_became_active":"2024-05-27T07:41:08.816418+0000","last_became_peered":"2024-05-27T07:41:08.816418+0000","last_unstale":"2024-05-27T07:43:00.017962+0000","last_undegraded":"2024-05-27T07:43:00.017962+0000","last_fullsized":"2024-05-27T07:43:00.017962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:02.760707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194966+0000","last_change":"2024-05-27T07:41:08.834081+0000","last_active":"2024-05-27T07:43:00.194966+0000","last_peered":"2024-05-27T07:43:00.194966+0000","last_clean":"2024-05-27T07:43:00.194966+0000","last_became_active":"2024-05-27T07:41:08.833679+0000","last_became_peered":"2024-05-27T07:41:08.833679+0000","last_unstale":"2024-05-27T07:43:00.194966+0000","last_undegraded":"2024-05-27T07:43:00.194966+0000","last_fullsized":"2024-05-27T07:43:00.194966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:41:14.426593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72586,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":348,"num_read_kb":255,"num_write":251,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":217758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645354,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":522393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841585,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1178966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005113,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":522211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201343,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":522477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397575,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1119477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561102,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":517715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":584127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":519244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117094,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":517423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920862,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":587674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1675264,"data_stored":1114668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:43:11.000 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T07:43:11.488 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:11.807 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:11 smithi090 ceph-mon[31429]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:43:11.807 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:11 smithi090 ceph-mon[31429]: pgmap v447: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-27T07:43:11.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:11 smithi129 ceph-mon[35254]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:43:11.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:11 smithi129 ceph-mon[35254]: pgmap v447: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-27T07:43:11.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:11 smithi136 ceph-mon[35324]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:43:11.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:11 smithi136 ceph-mon[35324]: pgmap v447: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-27T07:43:12.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:12 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:43:12.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:12 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:43:12.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:12 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:43:13.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.685+0000 7f703d400700 1 -- 172.21.15.90:0/398621906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7038076440 msgr2=0x7f70381106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:13.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.685+0000 7f703d400700 1 --2- 172.21.15.90:0/398621906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7038076440 0x7f70381106b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7024009990 tx=0x7f7024009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.686+0000 7f703d400700 1 -- 172.21.15.90:0/398621906 shutdown_connections 2024-05-27T07:43:13.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.686+0000 7f703d400700 1 --2- 172.21.15.90:0/398621906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7038110bf0 0x7f7038113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.686+0000 7f703d400700 1 --2- 172.21.15.90:0/398621906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7038075af0 0x7f7038075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.686+0000 7f703d400700 1 --2- 172.21.15.90:0/398621906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7038076440 0x7f70381106b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.686+0000 7f703d400700 1 -- 172.21.15.90:0/398621906 >> 172.21.15.90:0/398621906 conn(0x7f7038100030 msgr2=0x7f7038102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:13.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.686+0000 7f703d400700 1 -- 172.21.15.90:0/398621906 shutdown_connections 2024-05-27T07:43:13.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.687+0000 7f703d400700 1 -- 172.21.15.90:0/398621906 wait complete. 2024-05-27T07:43:13.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.688+0000 7f703d400700 1 Processor -- start 2024-05-27T07:43:13.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.688+0000 7f703d400700 1 -- start start 2024-05-27T07:43:13.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.688+0000 7f703d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7038075af0 0x7f703819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:13.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.688+0000 7f703d400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7038076440 0x7f703819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:13.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.688+0000 7f703d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7038110bf0 0x7f70381a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:13.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.688+0000 7f703d400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f70381165d0 con 0x7f7038110bf0 2024-05-27T07:43:13.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.688+0000 7f703d400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7038116450 con 0x7f7038076440 2024-05-27T07:43:13.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.688+0000 7f703d400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7038116750 con 0x7f7038075af0 2024-05-27T07:43:13.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.689+0000 7f7037200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7038110bf0 0x7f70381a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:13.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.689+0000 7f7037200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7038110bf0 0x7f70381a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44090/0 (socket says 172.21.15.90:44090) 2024-05-27T07:43:13.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.689+0000 7f7037200700 1 -- 172.21.15.90:0/4116039718 learned_addr learned my addr 172.21.15.90:0/4116039718 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:13.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.689+0000 7f7036800700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7038075af0 0x7f703819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:13.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.689+0000 7f7035e00700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7038076440 0x7f703819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:13.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:13 smithi090 ceph-mon[31429]: pgmap v448: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-27T07:43:13.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.689+0000 7f7037200700 1 -- 172.21.15.90:0/4116039718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7038075af0 msgr2=0x7f703819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:13.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.689+0000 7f7037200700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7038075af0 0x7f703819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.689+0000 7f7037200700 1 -- 172.21.15.90:0/4116039718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7038076440 msgr2=0x7f703819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:13.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.689+0000 7f7037200700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7038076440 0x7f703819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.689+0000 7f7037200700 1 -- 172.21.15.90:0/4116039718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70381a1e90 con 0x7f7038110bf0 2024-05-27T07:43:13.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.690+0000 7f7037200700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7038110bf0 0x7f70381a17f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70300094f0 tx=0x7f703000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:13.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.690+0000 7f702f400700 1 -- 172.21.15.90:0/4116039718 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030004db0 con 0x7f7038110bf0 2024-05-27T07:43:13.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.690+0000 7f702f400700 1 -- 172.21.15.90:0/4116039718 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7030009e20 con 0x7f7038110bf0 2024-05-27T07:43:13.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.690+0000 7f703d400700 1 -- 172.21.15.90:0/4116039718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70381a2180 con 0x7f7038110bf0 2024-05-27T07:43:13.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.690+0000 7f703d400700 1 -- 172.21.15.90:0/4116039718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70381a9aa0 con 0x7f7038110bf0 2024-05-27T07:43:13.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.691+0000 7f702f400700 1 -- 172.21.15.90:0/4116039718 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030015400 con 0x7f7038110bf0 2024-05-27T07:43:13.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.692+0000 7f703d400700 1 -- 172.21.15.90:0/4116039718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7018005330 con 0x7f7038110bf0 2024-05-27T07:43:13.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.694+0000 7f702f400700 1 -- 172.21.15.90:0/4116039718 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f703000db20 con 0x7f7038110bf0 2024-05-27T07:43:13.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.697+0000 7f702f400700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f701c0b2990 0x7f701c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:13.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.697+0000 7f702f400700 1 -- 172.21.15.90:0/4116039718 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f70300e1ff0 con 0x7f7038110bf0 2024-05-27T07:43:13.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.697+0000 7f7036800700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f701c0b2990 0x7f701c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:13.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.698+0000 7f7036800700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f701c0b2990 0x7f701c0b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7028009a10 tx=0x7f7028009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:13.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.701+0000 7f702f400700 1 -- 172.21.15.90:0/4116039718 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7030010070 con 0x7f7038110bf0 2024-05-27T07:43:13.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:13 smithi129 ceph-mon[35254]: pgmap v448: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-27T07:43:13.846 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:13 smithi136 ceph-mon[35324]: pgmap v448: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-27T07:43:13.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.932+0000 7f703d400700 1 -- 172.21.15.90:0/4116039718 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7018000bc0 con 0x7f701c0b2990 2024-05-27T07:43:13.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.961+0000 7f702f400700 1 -- 172.21.15.90:0/4116039718 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340406 (secure 0 0 0) 0x7f7018000bc0 con 0x7f701c0b2990 2024-05-27T07:43:13.962 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:43:13.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.967+0000 7f703d400700 1 -- 172.21.15.90:0/4116039718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f701c0b2990 msgr2=0x7f701c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:13.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.968+0000 7f703d400700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f701c0b2990 0x7f701c0b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7028009a10 tx=0x7f7028009190 comp rx=0 tx=0).stop 2024-05-27T07:43:13.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.968+0000 7f703d400700 1 -- 172.21.15.90:0/4116039718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7038110bf0 msgr2=0x7f70381a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:13.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.968+0000 7f703d400700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7038110bf0 0x7f70381a17f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70300094f0 tx=0x7f703000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.968+0000 7f703d400700 1 -- 172.21.15.90:0/4116039718 shutdown_connections 2024-05-27T07:43:13.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.968+0000 7f703d400700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f701c0b2990 0x7f701c0b4ea0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.968+0000 7f703d400700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7038110bf0 0x7f70381a17f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.968+0000 7f703d400700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7038075af0 0x7f703819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.968+0000 7f703d400700 1 --2- 172.21.15.90:0/4116039718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7038076440 0x7f703819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:13.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.968+0000 7f703d400700 1 -- 172.21.15.90:0/4116039718 >> 172.21.15.90:0/4116039718 conn(0x7f7038100030 msgr2=0x7f70381006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:13.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.968+0000 7f703d400700 1 -- 172.21.15.90:0/4116039718 shutdown_connections 2024-05-27T07:43:13.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:13.969+0000 7f703d400700 1 -- 172.21.15.90:0/4116039718 wait complete. 2024-05-27T07:43:13.971 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-05-27T07:43:14.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:14 smithi090 ceph-mon[31429]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:43:14.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:14 smithi129 ceph-mon[35254]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:43:14.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:14 smithi136 ceph-mon[35324]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:43:15.372 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-05-27T07:43:12.420819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663188,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":474,"num_read_kb":344,"num_write":376,"num_write_kb":2298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359468,"kb_used_data":14432,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582356,"statfs":{"total":1151940427776,"available":1151572332544,"internally_reserved":0,"allocated":14778368,"data_stored":8225546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19138,"internal_metadata":353154366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8713,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":63,"num_write":10,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006247"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196184+0000","last_change":"2024-05-27T07:41:08.839265+0000","last_active":"2024-05-27T07:43:00.196184+0000","last_peered":"2024-05-27T07:43:00.196184+0000","last_clean":"2024-05-27T07:43:00.196184+0000","last_became_active":"2024-05-27T07:41:08.838814+0000","last_became_peered":"2024-05-27T07:41:08.838814+0000","last_unstale":"2024-05-27T07:43:00.196184+0000","last_undegraded":"2024-05-27T07:43:00.196184+0000","last_fullsized":"2024-05-27T07:43:00.196184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:32.901520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193489+0000","last_change":"2024-05-27T07:41:08.838313+0000","last_active":"2024-05-27T07:43:00.193489+0000","last_peered":"2024-05-27T07:43:00.193489+0000","last_clean":"2024-05-27T07:43:00.193489+0000","last_became_active":"2024-05-27T07:41:08.836657+0000","last_became_peered":"2024-05-27T07:41:08.836657+0000","last_unstale":"2024-05-27T07:43:00.193489+0000","last_undegraded":"2024-05-27T07:43:00.193489+0000","last_fullsized":"2024-05-27T07:43:00.193489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:33.556505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193231+0000","last_change":"2024-05-27T07:41:08.841513+0000","last_active":"2024-05-27T07:43:00.193231+0000","last_peered":"2024-05-27T07:43:00.193231+0000","last_clean":"2024-05-27T07:43:00.193231+0000","last_became_active":"2024-05-27T07:41:08.838624+0000","last_became_peered":"2024-05-27T07:41:08.838624+0000","last_unstale":"2024-05-27T07:43:00.193231+0000","last_undegraded":"2024-05-27T07:43:00.193231+0000","last_fullsized":"2024-05-27T07:43:00.193231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:01:37.320418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012829+0000","last_change":"2024-05-27T07:41:08.839138+0000","last_active":"2024-05-27T07:43:00.012829+0000","last_peered":"2024-05-27T07:43:00.012829+0000","last_clean":"2024-05-27T07:43:00.012829+0000","last_became_active":"2024-05-27T07:41:08.838766+0000","last_became_peered":"2024-05-27T07:41:08.838766+0000","last_unstale":"2024-05-27T07:43:00.012829+0000","last_undegraded":"2024-05-27T07:43:00.012829+0000","last_fullsized":"2024-05-27T07:43:00.012829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:09.129257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195829+0000","last_change":"2024-05-27T07:41:08.823006+0000","last_active":"2024-05-27T07:43:00.195829+0000","last_peered":"2024-05-27T07:43:00.195829+0000","last_clean":"2024-05-27T07:43:00.195829+0000","last_became_active":"2024-05-27T07:41:08.822551+0000","last_became_peered":"2024-05-27T07:41:08.822551+0000","last_unstale":"2024-05-27T07:43:00.195829+0000","last_undegraded":"2024-05-27T07:43:00.195829+0000","last_fullsized":"2024-05-27T07:43:00.195829+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:49.255763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.034743+0000","last_change":"2024-05-27T07:41:08.846072+0000","last_active":"2024-05-27T07:43:00.034743+0000","last_peered":"2024-05-27T07:43:00.034743+0000","last_clean":"2024-05-27T07:43:00.034743+0000","last_became_active":"2024-05-27T07:41:08.845604+0000","last_became_peered":"2024-05-27T07:41:08.845604+0000","last_unstale":"2024-05-27T07:43:00.034743+0000","last_undegraded":"2024-05-27T07:43:00.034743+0000","last_fullsized":"2024-05-27T07:43:00.034743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:23:56.389135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194969+0000","last_change":"2024-05-27T07:41:08.841046+0000","last_active":"2024-05-27T07:43:00.194969+0000","last_peered":"2024-05-27T07:43:00.194969+0000","last_clean":"2024-05-27T07:43:00.194969+0000","last_became_active":"2024-05-27T07:41:08.840028+0000","last_became_peered":"2024-05-27T07:41:08.840028+0000","last_unstale":"2024-05-27T07:43:00.194969+0000","last_undegraded":"2024-05-27T07:43:00.194969+0000","last_fullsized":"2024-05-27T07:43:00.194969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:35.239919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.116821+0000","last_change":"2024-05-27T07:41:08.841182+0000","last_active":"2024-05-27T07:43:00.116821+0000","last_peered":"2024-05-27T07:43:00.116821+0000","last_clean":"2024-05-27T07:43:00.116821+0000","last_became_active":"2024-05-27T07:41:08.840645+0000","last_became_peered":"2024-05-27T07:41:08.840645+0000","last_unstale":"2024-05-27T07:43:00.116821+0000","last_undegraded":"2024-05-27T07:43:00.116821+0000","last_fullsized":"2024-05-27T07:43:00.116821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:10:59.744505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196915+0000","last_change":"2024-05-27T07:41:08.825874+0000","last_active":"2024-05-27T07:43:00.196915+0000","last_peered":"2024-05-27T07:43:00.196915+0000","last_clean":"2024-05-27T07:43:00.196915+0000","last_became_active":"2024-05-27T07:41:08.825386+0000","last_became_peered":"2024-05-27T07:41:08.825386+0000","last_unstale":"2024-05-27T07:43:00.196915+0000","last_undegraded":"2024-05-27T07:43:00.196915+0000","last_fullsized":"2024-05-27T07:43:00.196915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:13.304612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.035827+0000","last_change":"2024-05-27T07:41:08.847445+0000","last_active":"2024-05-27T07:43:00.035827+0000","last_peered":"2024-05-27T07:43:00.035827+0000","last_clean":"2024-05-27T07:43:00.035827+0000","last_became_active":"2024-05-27T07:41:08.846512+0000","last_became_peered":"2024-05-27T07:41:08.846512+0000","last_unstale":"2024-05-27T07:43:00.035827+0000","last_undegraded":"2024-05-27T07:43:00.035827+0000","last_fullsized":"2024-05-27T07:43:00.035827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:57.235218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196395+0000","last_change":"2024-05-27T07:41:08.843531+0000","last_active":"2024-05-27T07:43:00.196395+0000","last_peered":"2024-05-27T07:43:00.196395+0000","last_clean":"2024-05-27T07:43:00.196395+0000","last_became_active":"2024-05-27T07:41:08.843214+0000","last_became_peered":"2024-05-27T07:41:08.843214+0000","last_unstale":"2024-05-27T07:43:00.196395+0000","last_undegraded":"2024-05-27T07:43:00.196395+0000","last_fullsized":"2024-05-27T07:43:00.196395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:00.884178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011969+0000","last_change":"2024-05-27T07:41:08.839487+0000","last_active":"2024-05-27T07:43:00.011969+0000","last_peered":"2024-05-27T07:43:00.011969+0000","last_clean":"2024-05-27T07:43:00.011969+0000","last_became_active":"2024-05-27T07:41:08.839097+0000","last_became_peered":"2024-05-27T07:41:08.839097+0000","last_unstale":"2024-05-27T07:43:00.011969+0000","last_undegraded":"2024-05-27T07:43:00.011969+0000","last_fullsized":"2024-05-27T07:43:00.011969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:51.055482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.017645+0000","last_change":"2024-05-27T07:41:08.836843+0000","last_active":"2024-05-27T07:43:00.017645+0000","last_peered":"2024-05-27T07:43:00.017645+0000","last_clean":"2024-05-27T07:43:00.017645+0000","last_became_active":"2024-05-27T07:41:08.836362+0000","last_became_peered":"2024-05-27T07:41:08.836362+0000","last_unstale":"2024-05-27T07:43:00.017645+0000","last_undegraded":"2024-05-27T07:43:00.017645+0000","last_fullsized":"2024-05-27T07:43:00.017645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:47.844301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.192332+0000","last_change":"2024-05-27T07:41:08.839046+0000","last_active":"2024-05-27T07:43:00.192332+0000","last_peered":"2024-05-27T07:43:00.192332+0000","last_clean":"2024-05-27T07:43:00.192332+0000","last_became_active":"2024-05-27T07:41:08.838601+0000","last_became_peered":"2024-05-27T07:41:08.838601+0000","last_unstale":"2024-05-27T07:43:00.192332+0000","last_undegraded":"2024-05-27T07:43:00.192332+0000","last_fullsized":"2024-05-27T07:43:00.192332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:59.563490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.192386+0000","last_change":"2024-05-27T07:41:08.832510+0000","last_active":"2024-05-27T07:43:00.192386+0000","last_peered":"2024-05-27T07:43:00.192386+0000","last_clean":"2024-05-27T07:43:00.192386+0000","last_became_active":"2024-05-27T07:41:08.832082+0000","last_became_peered":"2024-05-27T07:41:08.832082+0000","last_unstale":"2024-05-27T07:43:00.192386+0000","last_undegraded":"2024-05-27T07:43:00.192386+0000","last_fullsized":"2024-05-27T07:43:00.192386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:59.951860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193432+0000","last_change":"2024-05-27T07:41:08.827953+0000","last_active":"2024-05-27T07:43:00.193432+0000","last_peered":"2024-05-27T07:43:00.193432+0000","last_clean":"2024-05-27T07:43:00.193432+0000","last_became_active":"2024-05-27T07:41:08.827293+0000","last_became_peered":"2024-05-27T07:41:08.827293+0000","last_unstale":"2024-05-27T07:43:00.193432+0000","last_undegraded":"2024-05-27T07:43:00.193432+0000","last_fullsized":"2024-05-27T07:43:00.193432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:08.810514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195477+0000","last_change":"2024-05-27T07:41:08.839591+0000","last_active":"2024-05-27T07:43:00.195477+0000","last_peered":"2024-05-27T07:43:00.195477+0000","last_clean":"2024-05-27T07:43:00.195477+0000","last_became_active":"2024-05-27T07:41:08.839217+0000","last_became_peered":"2024-05-27T07:41:08.839217+0000","last_unstale":"2024-05-27T07:43:00.195477+0000","last_undegraded":"2024-05-27T07:43:00.195477+0000","last_fullsized":"2024-05-27T07:43:00.195477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:11:41.636163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.010756+0000","last_change":"2024-05-27T07:41:08.839544+0000","last_active":"2024-05-27T07:43:00.010756+0000","last_peered":"2024-05-27T07:43:00.010756+0000","last_clean":"2024-05-27T07:43:00.010756+0000","last_became_active":"2024-05-27T07:41:08.839177+0000","last_became_peered":"2024-05-27T07:41:08.839177+0000","last_unstale":"2024-05-27T07:43:00.010756+0000","last_undegraded":"2024-05-27T07:43:00.010756+0000","last_fullsized":"2024-05-27T07:43:00.010756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:41.921759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013774+0000","last_change":"2024-05-27T07:41:08.847911+0000","last_active":"2024-05-27T07:43:00.013774+0000","last_peered":"2024-05-27T07:43:00.013774+0000","last_clean":"2024-05-27T07:43:00.013774+0000","last_became_active":"2024-05-27T07:41:08.847427+0000","last_became_peered":"2024-05-27T07:41:08.847427+0000","last_unstale":"2024-05-27T07:43:00.013774+0000","last_undegraded":"2024-05-27T07:43:00.013774+0000","last_fullsized":"2024-05-27T07:43:00.013774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:13.692744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012454+0000","last_change":"2024-05-27T07:41:21.999614+0000","last_active":"2024-05-27T07:43:00.012454+0000","last_peered":"2024-05-27T07:43:00.012454+0000","last_clean":"2024-05-27T07:43:00.012454+0000","last_became_active":"2024-05-27T07:41:21.999086+0000","last_became_peered":"2024-05-27T07:41:21.999086+0000","last_unstale":"2024-05-27T07:43:00.012454+0000","last_undegraded":"2024-05-27T07:43:00.012454+0000","last_fullsized":"2024-05-27T07:43:00.012454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:20.229318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":91,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:06.012523+0000","last_change":"2024-05-27T07:41:04.795079+0000","last_active":"2024-05-27T07:43:06.012523+0000","last_peered":"2024-05-27T07:43:06.012523+0000","last_clean":"2024-05-27T07:43:06.012523+0000","last_became_active":"2024-05-27T07:41:04.794367+0000","last_became_peered":"2024-05-27T07:41:04.794367+0000","last_unstale":"2024-05-27T07:43:06.012523+0000","last_undegraded":"2024-05-27T07:43:06.012523+0000","last_fullsized":"2024-05-27T07:43:06.012523+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:27:52.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.118644+0000","last_change":"2024-05-27T07:41:08.846712+0000","last_active":"2024-05-27T07:43:00.118644+0000","last_peered":"2024-05-27T07:43:00.118644+0000","last_clean":"2024-05-27T07:43:00.118644+0000","last_became_active":"2024-05-27T07:41:08.839999+0000","last_became_peered":"2024-05-27T07:41:08.839999+0000","last_unstale":"2024-05-27T07:43:00.118644+0000","last_undegraded":"2024-05-27T07:43:00.118644+0000","last_fullsized":"2024-05-27T07:43:00.118644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:31:23.924074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011296+0000","last_change":"2024-05-27T07:41:21.996675+0000","last_active":"2024-05-27T07:43:00.011296+0000","last_peered":"2024-05-27T07:43:00.011296+0000","last_clean":"2024-05-27T07:43:00.011296+0000","last_became_active":"2024-05-27T07:41:21.996152+0000","last_became_peered":"2024-05-27T07:41:21.996152+0000","last_unstale":"2024-05-27T07:43:00.011296+0000","last_undegraded":"2024-05-27T07:43:00.011296+0000","last_fullsized":"2024-05-27T07:43:00.011296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:57.415146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.119419+0000","last_change":"2024-05-27T07:41:04.890045+0000","last_active":"2024-05-27T07:43:00.119419+0000","last_peered":"2024-05-27T07:43:00.119419+0000","last_clean":"2024-05-27T07:43:00.119419+0000","last_became_active":"2024-05-27T07:41:04.888993+0000","last_became_peered":"2024-05-27T07:41:04.888993+0000","last_unstale":"2024-05-27T07:43:00.119419+0000","last_undegraded":"2024-05-27T07:43:00.119419+0000","last_fullsized":"2024-05-27T07:43:00.119419+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:19.866699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193523+0000","last_change":"2024-05-27T07:41:08.847472+0000","last_active":"2024-05-27T07:43:00.193523+0000","last_peered":"2024-05-27T07:43:00.193523+0000","last_clean":"2024-05-27T07:43:00.193523+0000","last_became_active":"2024-05-27T07:41:08.846070+0000","last_became_peered":"2024-05-27T07:41:08.846070+0000","last_unstale":"2024-05-27T07:43:00.193523+0000","last_undegraded":"2024-05-27T07:43:00.193523+0000","last_fullsized":"2024-05-27T07:43:00.193523+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:29:08.497007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011836+0000","last_change":"2024-05-27T07:41:59.842473+0000","last_active":"2024-05-27T07:43:00.011836+0000","last_peered":"2024-05-27T07:43:00.011836+0000","last_clean":"2024-05-27T07:43:00.011836+0000","last_became_active":"2024-05-27T07:41:59.841035+0000","last_became_peered":"2024-05-27T07:41:59.841035+0000","last_unstale":"2024-05-27T07:43:00.011836+0000","last_undegraded":"2024-05-27T07:43:00.011836+0000","last_fullsized":"2024-05-27T07:43:00.011836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:52.959589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.197143+0000","last_change":"2024-05-27T07:41:04.804046+0000","last_active":"2024-05-27T07:43:00.197143+0000","last_peered":"2024-05-27T07:43:00.197143+0000","last_clean":"2024-05-27T07:43:00.197143+0000","last_became_active":"2024-05-27T07:41:04.803360+0000","last_became_peered":"2024-05-27T07:41:04.803360+0000","last_unstale":"2024-05-27T07:43:00.197143+0000","last_undegraded":"2024-05-27T07:43:00.197143+0000","last_fullsized":"2024-05-27T07:43:00.197143+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:17.325221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195789+0000","last_change":"2024-05-27T07:41:08.840842+0000","last_active":"2024-05-27T07:43:00.195789+0000","last_peered":"2024-05-27T07:43:00.195789+0000","last_clean":"2024-05-27T07:43:00.195789+0000","last_became_active":"2024-05-27T07:41:08.837817+0000","last_became_peered":"2024-05-27T07:41:08.837817+0000","last_unstale":"2024-05-27T07:43:00.195789+0000","last_undegraded":"2024-05-27T07:43:00.195789+0000","last_fullsized":"2024-05-27T07:43:00.195789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:53:38.130608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011303+0000","last_change":"2024-05-27T07:41:22.002988+0000","last_active":"2024-05-27T07:43:00.011303+0000","last_peered":"2024-05-27T07:43:00.011303+0000","last_clean":"2024-05-27T07:43:00.011303+0000","last_became_active":"2024-05-27T07:41:22.001141+0000","last_became_peered":"2024-05-27T07:41:22.001141+0000","last_unstale":"2024-05-27T07:43:00.011303+0000","last_undegraded":"2024-05-27T07:43:00.011303+0000","last_fullsized":"2024-05-27T07:43:00.011303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:29:17.380439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196935+0000","last_change":"2024-05-27T07:41:04.794424+0000","last_active":"2024-05-27T07:43:00.196935+0000","last_peered":"2024-05-27T07:43:00.196935+0000","last_clean":"2024-05-27T07:43:00.196935+0000","last_became_active":"2024-05-27T07:41:04.793912+0000","last_became_peered":"2024-05-27T07:41:04.793912+0000","last_unstale":"2024-05-27T07:43:00.196935+0000","last_undegraded":"2024-05-27T07:43:00.196935+0000","last_fullsized":"2024-05-27T07:43:00.196935+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:50.876891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042495+0000","last_change":"2024-05-27T07:41:08.832356+0000","last_active":"2024-05-27T07:43:00.042495+0000","last_peered":"2024-05-27T07:43:00.042495+0000","last_clean":"2024-05-27T07:43:00.042495+0000","last_became_active":"2024-05-27T07:41:08.831979+0000","last_became_peered":"2024-05-27T07:41:08.831979+0000","last_unstale":"2024-05-27T07:43:00.042495+0000","last_undegraded":"2024-05-27T07:43:00.042495+0000","last_fullsized":"2024-05-27T07:43:00.042495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:37:47.584177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012955+0000","last_change":"2024-05-27T07:41:22.004233+0000","last_active":"2024-05-27T07:43:00.012955+0000","last_peered":"2024-05-27T07:43:00.012955+0000","last_clean":"2024-05-27T07:43:00.012955+0000","last_became_active":"2024-05-27T07:41:22.003781+0000","last_became_peered":"2024-05-27T07:41:22.003781+0000","last_unstale":"2024-05-27T07:43:00.012955+0000","last_undegraded":"2024-05-27T07:43:00.012955+0000","last_fullsized":"2024-05-27T07:43:00.012955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:02:13.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":88,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:08.026346+0000","last_change":"2024-05-27T07:41:04.800373+0000","last_active":"2024-05-27T07:43:08.026346+0000","last_peered":"2024-05-27T07:43:08.026346+0000","last_clean":"2024-05-27T07:43:08.026346+0000","last_became_active":"2024-05-27T07:41:04.799930+0000","last_became_peered":"2024-05-27T07:41:04.799930+0000","last_unstale":"2024-05-27T07:43:08.026346+0000","last_undegraded":"2024-05-27T07:43:08.026346+0000","last_fullsized":"2024-05-27T07:43:08.026346+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:24.642479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196134+0000","last_change":"2024-05-27T07:41:08.830387+0000","last_active":"2024-05-27T07:43:00.196134+0000","last_peered":"2024-05-27T07:43:00.196134+0000","last_clean":"2024-05-27T07:43:00.196134+0000","last_became_active":"2024-05-27T07:41:08.829720+0000","last_became_peered":"2024-05-27T07:41:08.829720+0000","last_unstale":"2024-05-27T07:43:00.196134+0000","last_undegraded":"2024-05-27T07:43:00.196134+0000","last_fullsized":"2024-05-27T07:43:00.196134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:42.014383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013364+0000","last_change":"2024-05-27T07:41:22.005700+0000","last_active":"2024-05-27T07:43:00.013364+0000","last_peered":"2024-05-27T07:43:00.013364+0000","last_clean":"2024-05-27T07:43:00.013364+0000","last_became_active":"2024-05-27T07:41:22.003643+0000","last_became_peered":"2024-05-27T07:41:22.003643+0000","last_unstale":"2024-05-27T07:43:00.013364+0000","last_undegraded":"2024-05-27T07:43:00.013364+0000","last_fullsized":"2024-05-27T07:43:00.013364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:48.332767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.034535+0000","last_change":"2024-05-27T07:41:59.790579+0000","last_active":"2024-05-27T07:43:00.034535+0000","last_peered":"2024-05-27T07:43:00.034535+0000","last_clean":"2024-05-27T07:43:00.034535+0000","last_became_active":"2024-05-27T07:41:59.789849+0000","last_became_peered":"2024-05-27T07:41:59.789849+0000","last_unstale":"2024-05-27T07:43:00.034535+0000","last_undegraded":"2024-05-27T07:43:00.034535+0000","last_fullsized":"2024-05-27T07:43:00.034535+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:24:30.564074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.121892+0000","last_change":"2024-05-27T07:43:00.121063+0000","last_active":"2024-05-27T07:43:00.121892+0000","last_peered":"2024-05-27T07:43:00.121892+0000","last_clean":"2024-05-27T07:43:00.121892+0000","last_became_active":"2024-05-27T07:43:00.119977+0000","last_became_peered":"2024-05-27T07:43:00.119977+0000","last_unstale":"2024-05-27T07:43:00.121892+0000","last_undegraded":"2024-05-27T07:43:00.121892+0000","last_fullsized":"2024-05-27T07:43:00.121892+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:04:43.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195101+0000","last_change":"2024-05-27T07:41:22.013820+0000","last_active":"2024-05-27T07:43:00.195101+0000","last_peered":"2024-05-27T07:43:00.195101+0000","last_clean":"2024-05-27T07:43:00.195101+0000","last_became_active":"2024-05-27T07:41:22.013110+0000","last_became_peered":"2024-05-27T07:41:22.013110+0000","last_unstale":"2024-05-27T07:43:00.195101+0000","last_undegraded":"2024-05-27T07:43:00.195101+0000","last_fullsized":"2024-05-27T07:43:00.195101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:45:07.317872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018203+0000","last_change":"2024-05-27T07:41:04.789197+0000","last_active":"2024-05-27T07:43:00.018203+0000","last_peered":"2024-05-27T07:43:00.018203+0000","last_clean":"2024-05-27T07:43:00.018203+0000","last_became_active":"2024-05-27T07:41:04.788755+0000","last_became_peered":"2024-05-27T07:41:04.788755+0000","last_unstale":"2024-05-27T07:43:00.018203+0000","last_undegraded":"2024-05-27T07:43:00.018203+0000","last_fullsized":"2024-05-27T07:43:00.018203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:27:54.526910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.116780+0000","last_change":"2024-05-27T07:41:08.841769+0000","last_active":"2024-05-27T07:43:00.116780+0000","last_peered":"2024-05-27T07:43:00.116780+0000","last_clean":"2024-05-27T07:43:00.116780+0000","last_became_active":"2024-05-27T07:41:08.840834+0000","last_became_peered":"2024-05-27T07:41:08.840834+0000","last_unstale":"2024-05-27T07:43:00.116780+0000","last_undegraded":"2024-05-27T07:43:00.116780+0000","last_fullsized":"2024-05-27T07:43:00.116780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:15.573739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193413+0000","last_change":"2024-05-27T07:41:22.005119+0000","last_active":"2024-05-27T07:43:00.193413+0000","last_peered":"2024-05-27T07:43:00.193413+0000","last_clean":"2024-05-27T07:43:00.193413+0000","last_became_active":"2024-05-27T07:41:22.002638+0000","last_became_peered":"2024-05-27T07:41:22.002638+0000","last_unstale":"2024-05-27T07:43:00.193413+0000","last_undegraded":"2024-05-27T07:43:00.193413+0000","last_fullsized":"2024-05-27T07:43:00.193413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:08:49.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041462+0000","last_change":"2024-05-27T07:41:04.799864+0000","last_active":"2024-05-27T07:43:00.041462+0000","last_peered":"2024-05-27T07:43:00.041462+0000","last_clean":"2024-05-27T07:43:00.041462+0000","last_became_active":"2024-05-27T07:41:04.799218+0000","last_became_peered":"2024-05-27T07:41:04.799218+0000","last_unstale":"2024-05-27T07:43:00.041462+0000","last_undegraded":"2024-05-27T07:43:00.041462+0000","last_fullsized":"2024-05-27T07:43:00.041462+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:15.355720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.008861+0000","last_change":"2024-05-27T07:41:08.823387+0000","last_active":"2024-05-27T07:43:00.008861+0000","last_peered":"2024-05-27T07:43:00.008861+0000","last_clean":"2024-05-27T07:43:00.008861+0000","last_became_active":"2024-05-27T07:41:08.822943+0000","last_became_peered":"2024-05-27T07:41:08.822943+0000","last_unstale":"2024-05-27T07:43:00.008861+0000","last_undegraded":"2024-05-27T07:43:00.008861+0000","last_fullsized":"2024-05-27T07:43:00.008861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:46.365284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117228+0000","last_change":"2024-05-27T07:41:22.011848+0000","last_active":"2024-05-27T07:43:00.117228+0000","last_peered":"2024-05-27T07:43:00.117228+0000","last_clean":"2024-05-27T07:43:00.117228+0000","last_became_active":"2024-05-27T07:41:22.011335+0000","last_became_peered":"2024-05-27T07:41:22.011335+0000","last_unstale":"2024-05-27T07:43:00.117228+0000","last_undegraded":"2024-05-27T07:43:00.117228+0000","last_fullsized":"2024-05-27T07:43:00.117228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:29:22.528270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:09.027824+0000","last_change":"2024-05-27T07:41:04.793877+0000","last_active":"2024-05-27T07:43:09.027824+0000","last_peered":"2024-05-27T07:43:09.027824+0000","last_clean":"2024-05-27T07:43:09.027824+0000","last_became_active":"2024-05-27T07:41:04.793389+0000","last_became_peered":"2024-05-27T07:41:04.793389+0000","last_unstale":"2024-05-27T07:43:09.027824+0000","last_undegraded":"2024-05-27T07:43:09.027824+0000","last_fullsized":"2024-05-27T07:43:09.027824+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:48.670572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196685+0000","last_change":"2024-05-27T07:41:08.836682+0000","last_active":"2024-05-27T07:43:00.196685+0000","last_peered":"2024-05-27T07:43:00.196685+0000","last_clean":"2024-05-27T07:43:00.196685+0000","last_became_active":"2024-05-27T07:41:08.836229+0000","last_became_peered":"2024-05-27T07:41:08.836229+0000","last_unstale":"2024-05-27T07:43:00.196685+0000","last_undegraded":"2024-05-27T07:43:00.196685+0000","last_fullsized":"2024-05-27T07:43:00.196685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:33.249584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.008774+0000","last_change":"2024-05-27T07:41:22.004968+0000","last_active":"2024-05-27T07:43:00.008774+0000","last_peered":"2024-05-27T07:43:00.008774+0000","last_clean":"2024-05-27T07:43:00.008774+0000","last_became_active":"2024-05-27T07:41:22.001407+0000","last_became_peered":"2024-05-27T07:41:22.001407+0000","last_unstale":"2024-05-27T07:43:00.008774+0000","last_undegraded":"2024-05-27T07:43:00.008774+0000","last_fullsized":"2024-05-27T07:43:00.008774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:26.872497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195646+0000","last_change":"2024-05-27T07:41:59.787247+0000","last_active":"2024-05-27T07:43:00.195646+0000","last_peered":"2024-05-27T07:43:00.195646+0000","last_clean":"2024-05-27T07:43:00.195646+0000","last_became_active":"2024-05-27T07:41:59.786392+0000","last_became_peered":"2024-05-27T07:41:59.786392+0000","last_unstale":"2024-05-27T07:43:00.195646+0000","last_undegraded":"2024-05-27T07:43:00.195646+0000","last_fullsized":"2024-05-27T07:43:00.195646+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:12.579707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2],"acting":[11,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193565+0000","last_change":"2024-05-27T07:41:08.833731+0000","last_active":"2024-05-27T07:43:00.193565+0000","last_peered":"2024-05-27T07:43:00.193565+0000","last_clean":"2024-05-27T07:43:00.193565+0000","last_became_active":"2024-05-27T07:41:08.833318+0000","last_became_peered":"2024-05-27T07:41:08.833318+0000","last_unstale":"2024-05-27T07:43:00.193565+0000","last_undegraded":"2024-05-27T07:43:00.193565+0000","last_fullsized":"2024-05-27T07:43:00.193565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:50.085113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012266+0000","last_change":"2024-05-27T07:41:59.789784+0000","last_active":"2024-05-27T07:43:00.012266+0000","last_peered":"2024-05-27T07:43:00.012266+0000","last_clean":"2024-05-27T07:43:00.012266+0000","last_became_active":"2024-05-27T07:41:59.788589+0000","last_became_peered":"2024-05-27T07:41:59.788589+0000","last_unstale":"2024-05-27T07:43:00.012266+0000","last_undegraded":"2024-05-27T07:43:00.012266+0000","last_fullsized":"2024-05-27T07:43:00.012266+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:37.026907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042336+0000","last_change":"2024-05-27T07:41:04.892603+0000","last_active":"2024-05-27T07:43:00.042336+0000","last_peered":"2024-05-27T07:43:00.042336+0000","last_clean":"2024-05-27T07:43:00.042336+0000","last_became_active":"2024-05-27T07:41:04.890786+0000","last_became_peered":"2024-05-27T07:41:04.890786+0000","last_unstale":"2024-05-27T07:43:00.042336+0000","last_undegraded":"2024-05-27T07:43:00.042336+0000","last_fullsized":"2024-05-27T07:43:00.042336+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:34.129227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196274+0000","last_change":"2024-05-27T07:41:08.830802+0000","last_active":"2024-05-27T07:43:00.196274+0000","last_peered":"2024-05-27T07:43:00.196274+0000","last_clean":"2024-05-27T07:43:00.196274+0000","last_became_active":"2024-05-27T07:41:08.829661+0000","last_became_peered":"2024-05-27T07:41:08.829661+0000","last_unstale":"2024-05-27T07:43:00.196274+0000","last_undegraded":"2024-05-27T07:43:00.196274+0000","last_fullsized":"2024-05-27T07:43:00.196274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:20:15.041554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009055+0000","last_change":"2024-05-27T07:41:22.003997+0000","last_active":"2024-05-27T07:43:00.009055+0000","last_peered":"2024-05-27T07:43:00.009055+0000","last_clean":"2024-05-27T07:43:00.009055+0000","last_became_active":"2024-05-27T07:41:22.003527+0000","last_became_peered":"2024-05-27T07:41:22.003527+0000","last_unstale":"2024-05-27T07:43:00.009055+0000","last_undegraded":"2024-05-27T07:43:00.009055+0000","last_fullsized":"2024-05-27T07:43:00.009055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:17.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":175,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:09.073065+0000","last_change":"2024-05-27T07:41:59.814227+0000","last_active":"2024-05-27T07:43:09.073065+0000","last_peered":"2024-05-27T07:43:09.073065+0000","last_clean":"2024-05-27T07:43:09.073065+0000","last_became_active":"2024-05-27T07:41:59.788510+0000","last_became_peered":"2024-05-27T07:41:59.788510+0000","last_unstale":"2024-05-27T07:43:09.073065+0000","last_undegraded":"2024-05-27T07:43:09.073065+0000","last_fullsized":"2024-05-27T07:43:09.073065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:15.868348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65382,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":79,"num_read_kb":103,"num_write":38,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196932+0000","last_change":"2024-05-27T07:41:08.831639+0000","last_active":"2024-05-27T07:43:00.196932+0000","last_peered":"2024-05-27T07:43:00.196932+0000","last_clean":"2024-05-27T07:43:00.196932+0000","last_became_active":"2024-05-27T07:41:08.830541+0000","last_became_peered":"2024-05-27T07:41:08.830541+0000","last_unstale":"2024-05-27T07:43:00.196932+0000","last_undegraded":"2024-05-27T07:43:00.196932+0000","last_fullsized":"2024-05-27T07:43:00.196932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:45.635145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.043148+0000","last_change":"2024-05-27T07:41:21.999587+0000","last_active":"2024-05-27T07:43:00.043148+0000","last_peered":"2024-05-27T07:43:00.043148+0000","last_clean":"2024-05-27T07:43:00.043148+0000","last_became_active":"2024-05-27T07:41:21.999017+0000","last_became_peered":"2024-05-27T07:41:21.999017+0000","last_unstale":"2024-05-27T07:43:00.043148+0000","last_undegraded":"2024-05-27T07:43:00.043148+0000","last_fullsized":"2024-05-27T07:43:00.043148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:31:14.915229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013594+0000","last_change":"2024-05-27T07:41:04.799847+0000","last_active":"2024-05-27T07:43:00.013594+0000","last_peered":"2024-05-27T07:43:00.013594+0000","last_clean":"2024-05-27T07:43:00.013594+0000","last_became_active":"2024-05-27T07:41:04.798605+0000","last_became_peered":"2024-05-27T07:41:04.798605+0000","last_unstale":"2024-05-27T07:43:00.013594+0000","last_undegraded":"2024-05-27T07:43:00.013594+0000","last_fullsized":"2024-05-27T07:43:00.013594+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:01.957707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.016975+0000","last_change":"2024-05-27T07:41:08.831138+0000","last_active":"2024-05-27T07:43:00.016975+0000","last_peered":"2024-05-27T07:43:00.016975+0000","last_clean":"2024-05-27T07:43:00.016975+0000","last_became_active":"2024-05-27T07:41:08.830735+0000","last_became_peered":"2024-05-27T07:41:08.830735+0000","last_unstale":"2024-05-27T07:43:00.016975+0000","last_undegraded":"2024-05-27T07:43:00.016975+0000","last_fullsized":"2024-05-27T07:43:00.016975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:12.569687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194454+0000","last_change":"2024-05-27T07:41:59.779039+0000","last_active":"2024-05-27T07:43:00.194454+0000","last_peered":"2024-05-27T07:43:00.194454+0000","last_clean":"2024-05-27T07:43:00.194454+0000","last_became_active":"2024-05-27T07:41:59.777514+0000","last_became_peered":"2024-05-27T07:41:59.777514+0000","last_unstale":"2024-05-27T07:43:00.194454+0000","last_undegraded":"2024-05-27T07:43:00.194454+0000","last_fullsized":"2024-05-27T07:43:00.194454+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:12:19.223874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041902+0000","last_change":"2024-05-27T07:41:04.796467+0000","last_active":"2024-05-27T07:43:00.041902+0000","last_peered":"2024-05-27T07:43:00.041902+0000","last_clean":"2024-05-27T07:43:00.041902+0000","last_became_active":"2024-05-27T07:41:04.795519+0000","last_became_peered":"2024-05-27T07:41:04.795519+0000","last_unstale":"2024-05-27T07:43:00.041902+0000","last_undegraded":"2024-05-27T07:43:00.041902+0000","last_fullsized":"2024-05-27T07:43:00.041902+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:42.744076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195110+0000","last_change":"2024-05-27T07:41:08.832135+0000","last_active":"2024-05-27T07:43:00.195110+0000","last_peered":"2024-05-27T07:43:00.195110+0000","last_clean":"2024-05-27T07:43:00.195110+0000","last_became_active":"2024-05-27T07:41:08.830032+0000","last_became_peered":"2024-05-27T07:41:08.830032+0000","last_unstale":"2024-05-27T07:43:00.195110+0000","last_undegraded":"2024-05-27T07:43:00.195110+0000","last_fullsized":"2024-05-27T07:43:00.195110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:44:59.367026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009771+0000","last_change":"2024-05-27T07:41:22.008868+0000","last_active":"2024-05-27T07:43:00.009771+0000","last_peered":"2024-05-27T07:43:00.009771+0000","last_clean":"2024-05-27T07:43:00.009771+0000","last_became_active":"2024-05-27T07:41:22.002071+0000","last_became_peered":"2024-05-27T07:41:22.002071+0000","last_unstale":"2024-05-27T07:43:00.009771+0000","last_undegraded":"2024-05-27T07:43:00.009771+0000","last_fullsized":"2024-05-27T07:43:00.009771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:20.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":88,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:08.046921+0000","last_change":"2024-05-27T07:41:04.796198+0000","last_active":"2024-05-27T07:43:08.046921+0000","last_peered":"2024-05-27T07:43:08.046921+0000","last_clean":"2024-05-27T07:43:08.046921+0000","last_became_active":"2024-05-27T07:41:04.795646+0000","last_became_peered":"2024-05-27T07:41:04.795646+0000","last_unstale":"2024-05-27T07:43:08.046921+0000","last_undegraded":"2024-05-27T07:43:08.046921+0000","last_fullsized":"2024-05-27T07:43:08.046921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:30.936596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195103+0000","last_change":"2024-05-27T07:41:08.840518+0000","last_active":"2024-05-27T07:43:00.195103+0000","last_peered":"2024-05-27T07:43:00.195103+0000","last_clean":"2024-05-27T07:43:00.195103+0000","last_became_active":"2024-05-27T07:41:08.840162+0000","last_became_peered":"2024-05-27T07:41:08.840162+0000","last_unstale":"2024-05-27T07:43:00.195103+0000","last_undegraded":"2024-05-27T07:43:00.195103+0000","last_fullsized":"2024-05-27T07:43:00.195103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:04:14.657496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018855+0000","last_change":"2024-05-27T07:41:22.000243+0000","last_active":"2024-05-27T07:43:00.018855+0000","last_peered":"2024-05-27T07:43:00.018855+0000","last_clean":"2024-05-27T07:43:00.018855+0000","last_became_active":"2024-05-27T07:41:21.998898+0000","last_became_peered":"2024-05-27T07:41:21.998898+0000","last_unstale":"2024-05-27T07:43:00.018855+0000","last_undegraded":"2024-05-27T07:43:00.018855+0000","last_fullsized":"2024-05-27T07:43:00.018855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:35.696593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195302+0000","last_change":"2024-05-27T07:41:59.799662+0000","last_active":"2024-05-27T07:43:00.195302+0000","last_peered":"2024-05-27T07:43:00.195302+0000","last_clean":"2024-05-27T07:43:00.195302+0000","last_became_active":"2024-05-27T07:41:59.779733+0000","last_became_peered":"2024-05-27T07:41:59.779733+0000","last_unstale":"2024-05-27T07:43:00.195302+0000","last_undegraded":"2024-05-27T07:43:00.195302+0000","last_fullsized":"2024-05-27T07:43:00.195302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:23.665014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041865+0000","last_change":"2024-05-27T07:41:08.834931+0000","last_active":"2024-05-27T07:43:00.041865+0000","last_peered":"2024-05-27T07:43:00.041865+0000","last_clean":"2024-05-27T07:43:00.041865+0000","last_became_active":"2024-05-27T07:41:08.833298+0000","last_became_peered":"2024-05-27T07:41:08.833298+0000","last_unstale":"2024-05-27T07:43:00.041865+0000","last_undegraded":"2024-05-27T07:43:00.041865+0000","last_fullsized":"2024-05-27T07:43:00.041865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:03:48.507461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041766+0000","last_change":"2024-05-27T07:41:22.009407+0000","last_active":"2024-05-27T07:43:00.041766+0000","last_peered":"2024-05-27T07:43:00.041766+0000","last_clean":"2024-05-27T07:43:00.041766+0000","last_became_active":"2024-05-27T07:41:22.006921+0000","last_became_peered":"2024-05-27T07:41:22.006921+0000","last_unstale":"2024-05-27T07:43:00.041766+0000","last_undegraded":"2024-05-27T07:43:00.041766+0000","last_fullsized":"2024-05-27T07:43:00.041766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:24.034502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196436+0000","last_change":"2024-05-27T07:41:04.795641+0000","last_active":"2024-05-27T07:43:00.196436+0000","last_peered":"2024-05-27T07:43:00.196436+0000","last_clean":"2024-05-27T07:43:00.196436+0000","last_became_active":"2024-05-27T07:41:04.794935+0000","last_became_peered":"2024-05-27T07:41:04.794935+0000","last_unstale":"2024-05-27T07:43:00.196436+0000","last_undegraded":"2024-05-27T07:43:00.196436+0000","last_fullsized":"2024-05-27T07:43:00.196436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:14.850282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195110+0000","last_change":"2024-05-27T07:41:08.837277+0000","last_active":"2024-05-27T07:43:00.195110+0000","last_peered":"2024-05-27T07:43:00.195110+0000","last_clean":"2024-05-27T07:43:00.195110+0000","last_became_active":"2024-05-27T07:41:08.835935+0000","last_became_peered":"2024-05-27T07:41:08.835935+0000","last_unstale":"2024-05-27T07:43:00.195110+0000","last_undegraded":"2024-05-27T07:43:00.195110+0000","last_fullsized":"2024-05-27T07:43:00.195110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:46.436243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196061+0000","last_change":"2024-05-27T07:41:22.014700+0000","last_active":"2024-05-27T07:43:00.196061+0000","last_peered":"2024-05-27T07:43:00.196061+0000","last_clean":"2024-05-27T07:43:00.196061+0000","last_became_active":"2024-05-27T07:41:22.014136+0000","last_became_peered":"2024-05-27T07:41:22.014136+0000","last_unstale":"2024-05-27T07:43:00.196061+0000","last_undegraded":"2024-05-27T07:43:00.196061+0000","last_fullsized":"2024-05-27T07:43:00.196061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:51:08.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009324+0000","last_change":"2024-05-27T07:41:04.891269+0000","last_active":"2024-05-27T07:43:00.009324+0000","last_peered":"2024-05-27T07:43:00.009324+0000","last_clean":"2024-05-27T07:43:00.009324+0000","last_became_active":"2024-05-27T07:41:04.890843+0000","last_became_peered":"2024-05-27T07:41:04.890843+0000","last_unstale":"2024-05-27T07:43:00.009324+0000","last_undegraded":"2024-05-27T07:43:00.009324+0000","last_fullsized":"2024-05-27T07:43:00.009324+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:57.065300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.035249+0000","last_change":"2024-05-27T07:41:08.841977+0000","last_active":"2024-05-27T07:43:00.035249+0000","last_peered":"2024-05-27T07:43:00.035249+0000","last_clean":"2024-05-27T07:43:00.035249+0000","last_became_active":"2024-05-27T07:41:08.841551+0000","last_became_peered":"2024-05-27T07:41:08.841551+0000","last_unstale":"2024-05-27T07:43:00.035249+0000","last_undegraded":"2024-05-27T07:43:00.035249+0000","last_fullsized":"2024-05-27T07:43:00.035249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:35:07.766193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.020134+0000","last_change":"2024-05-27T07:41:22.006310+0000","last_active":"2024-05-27T07:43:00.020134+0000","last_peered":"2024-05-27T07:43:00.020134+0000","last_clean":"2024-05-27T07:43:00.020134+0000","last_became_active":"2024-05-27T07:41:22.004135+0000","last_became_peered":"2024-05-27T07:41:22.004135+0000","last_unstale":"2024-05-27T07:43:00.020134+0000","last_undegraded":"2024-05-27T07:43:00.020134+0000","last_fullsized":"2024-05-27T07:43:00.020134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:48.564894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":86,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:09.023526+0000","last_change":"2024-05-27T07:41:04.793284+0000","last_active":"2024-05-27T07:43:09.023526+0000","last_peered":"2024-05-27T07:43:09.023526+0000","last_clean":"2024-05-27T07:43:09.023526+0000","last_became_active":"2024-05-27T07:41:04.792853+0000","last_became_peered":"2024-05-27T07:41:04.792853+0000","last_unstale":"2024-05-27T07:43:09.023526+0000","last_undegraded":"2024-05-27T07:43:09.023526+0000","last_fullsized":"2024-05-27T07:43:09.023526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:22.979341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.034751+0000","last_change":"2024-05-27T07:41:59.776207+0000","last_active":"2024-05-27T07:43:00.034751+0000","last_peered":"2024-05-27T07:43:00.034751+0000","last_clean":"2024-05-27T07:43:00.034751+0000","last_became_active":"2024-05-27T07:41:59.775226+0000","last_became_peered":"2024-05-27T07:41:59.775226+0000","last_unstale":"2024-05-27T07:43:00.034751+0000","last_undegraded":"2024-05-27T07:43:00.034751+0000","last_fullsized":"2024-05-27T07:43:00.034751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:50.469144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.020637+0000","last_change":"2024-05-27T07:41:21.992013+0000","last_active":"2024-05-27T07:43:00.020637+0000","last_peered":"2024-05-27T07:43:00.020637+0000","last_clean":"2024-05-27T07:43:00.020637+0000","last_became_active":"2024-05-27T07:41:21.991550+0000","last_became_peered":"2024-05-27T07:41:21.991550+0000","last_unstale":"2024-05-27T07:43:00.020637+0000","last_undegraded":"2024-05-27T07:43:00.020637+0000","last_fullsized":"2024-05-27T07:43:00.020637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:34:53.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042356+0000","last_change":"2024-05-27T07:41:08.832512+0000","last_active":"2024-05-27T07:43:00.042356+0000","last_peered":"2024-05-27T07:43:00.042356+0000","last_clean":"2024-05-27T07:43:00.042356+0000","last_became_active":"2024-05-27T07:41:08.832113+0000","last_became_peered":"2024-05-27T07:41:08.832113+0000","last_unstale":"2024-05-27T07:43:00.042356+0000","last_undegraded":"2024-05-27T07:43:00.042356+0000","last_fullsized":"2024-05-27T07:43:00.042356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:00.530785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":88,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:11.054644+0000","last_change":"2024-05-27T07:41:04.891004+0000","last_active":"2024-05-27T07:43:11.054644+0000","last_peered":"2024-05-27T07:43:11.054644+0000","last_clean":"2024-05-27T07:43:11.054644+0000","last_became_active":"2024-05-27T07:41:04.890567+0000","last_became_peered":"2024-05-27T07:41:04.890567+0000","last_unstale":"2024-05-27T07:43:11.054644+0000","last_undegraded":"2024-05-27T07:43:11.054644+0000","last_fullsized":"2024-05-27T07:43:11.054644+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:43.991204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196027+0000","last_change":"2024-05-27T07:41:22.003446+0000","last_active":"2024-05-27T07:43:00.196027+0000","last_peered":"2024-05-27T07:43:00.196027+0000","last_clean":"2024-05-27T07:43:00.196027+0000","last_became_active":"2024-05-27T07:41:22.002989+0000","last_became_peered":"2024-05-27T07:41:22.002989+0000","last_unstale":"2024-05-27T07:43:00.196027+0000","last_undegraded":"2024-05-27T07:43:00.196027+0000","last_fullsized":"2024-05-27T07:43:00.196027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:12.996286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117982+0000","last_change":"2024-05-27T07:41:08.842685+0000","last_active":"2024-05-27T07:43:00.117982+0000","last_peered":"2024-05-27T07:43:00.117982+0000","last_clean":"2024-05-27T07:43:00.117982+0000","last_became_active":"2024-05-27T07:41:08.841954+0000","last_became_peered":"2024-05-27T07:41:08.841954+0000","last_unstale":"2024-05-27T07:43:00.117982+0000","last_undegraded":"2024-05-27T07:43:00.117982+0000","last_fullsized":"2024-05-27T07:43:00.117982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:18.087129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013268+0000","last_change":"2024-05-27T07:41:04.790330+0000","last_active":"2024-05-27T07:43:00.013268+0000","last_peered":"2024-05-27T07:43:00.013268+0000","last_clean":"2024-05-27T07:43:00.013268+0000","last_became_active":"2024-05-27T07:41:04.789508+0000","last_became_peered":"2024-05-27T07:41:04.789508+0000","last_unstale":"2024-05-27T07:43:00.013268+0000","last_undegraded":"2024-05-27T07:43:00.013268+0000","last_fullsized":"2024-05-27T07:43:00.013268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:27.537662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194381+0000","last_change":"2024-05-27T07:39:11.797324+0000","last_active":"2024-05-27T07:43:00.194381+0000","last_peered":"2024-05-27T07:43:00.194381+0000","last_clean":"2024-05-27T07:43:00.194381+0000","last_became_active":"2024-05-27T07:39:11.786294+0000","last_became_peered":"2024-05-27T07:39:11.786294+0000","last_unstale":"2024-05-27T07:43:00.194381+0000","last_undegraded":"2024-05-27T07:43:00.194381+0000","last_fullsized":"2024-05-27T07:43:00.194381+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_clean_scrub_stamp":"2024-05-27T07:34:18.613413+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:23.947615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042589+0000","last_change":"2024-05-27T07:41:21.997453+0000","last_active":"2024-05-27T07:43:00.042589+0000","last_peered":"2024-05-27T07:43:00.042589+0000","last_clean":"2024-05-27T07:43:00.042589+0000","last_became_active":"2024-05-27T07:41:21.997010+0000","last_became_peered":"2024-05-27T07:41:21.997010+0000","last_unstale":"2024-05-27T07:43:00.042589+0000","last_undegraded":"2024-05-27T07:43:00.042589+0000","last_fullsized":"2024-05-27T07:43:00.042589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:54:54.913274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.024988+0000","last_change":"2024-05-27T07:43:00.024988+0000","last_active":"2024-05-27T07:43:00.024988+0000","last_peered":"2024-05-27T07:43:00.024988+0000","last_clean":"2024-05-27T07:43:00.024988+0000","last_became_active":"2024-05-27T07:43:00.023816+0000","last_became_peered":"2024-05-27T07:43:00.023816+0000","last_unstale":"2024-05-27T07:43:00.024988+0000","last_undegraded":"2024-05-27T07:43:00.024988+0000","last_fullsized":"2024-05-27T07:43:00.024988+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:58.531985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194923+0000","last_change":"2024-05-27T07:41:04.795262+0000","last_active":"2024-05-27T07:43:00.194923+0000","last_peered":"2024-05-27T07:43:00.194923+0000","last_clean":"2024-05-27T07:43:00.194923+0000","last_became_active":"2024-05-27T07:41:04.794866+0000","last_became_peered":"2024-05-27T07:41:04.794866+0000","last_unstale":"2024-05-27T07:43:00.194923+0000","last_undegraded":"2024-05-27T07:43:00.194923+0000","last_fullsized":"2024-05-27T07:43:00.194923+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:07:44.241444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194359+0000","last_change":"2024-05-27T07:41:22.006194+0000","last_active":"2024-05-27T07:43:00.194359+0000","last_peered":"2024-05-27T07:43:00.194359+0000","last_clean":"2024-05-27T07:43:00.194359+0000","last_became_active":"2024-05-27T07:41:22.004079+0000","last_became_peered":"2024-05-27T07:41:22.004079+0000","last_unstale":"2024-05-27T07:43:00.194359+0000","last_undegraded":"2024-05-27T07:43:00.194359+0000","last_fullsized":"2024-05-27T07:43:00.194359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:03.496722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194277+0000","last_change":"2024-05-27T07:41:59.776734+0000","last_active":"2024-05-27T07:43:00.194277+0000","last_peered":"2024-05-27T07:43:00.194277+0000","last_clean":"2024-05-27T07:43:00.194277+0000","last_became_active":"2024-05-27T07:41:59.775678+0000","last_became_peered":"2024-05-27T07:41:59.775678+0000","last_unstale":"2024-05-27T07:43:00.194277+0000","last_undegraded":"2024-05-27T07:43:00.194277+0000","last_fullsized":"2024-05-27T07:43:00.194277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:56.007178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.020635+0000","last_change":"2024-05-27T07:41:08.831070+0000","last_active":"2024-05-27T07:43:00.020635+0000","last_peered":"2024-05-27T07:43:00.020635+0000","last_clean":"2024-05-27T07:43:00.020635+0000","last_became_active":"2024-05-27T07:41:08.830617+0000","last_became_peered":"2024-05-27T07:41:08.830617+0000","last_unstale":"2024-05-27T07:43:00.020635+0000","last_undegraded":"2024-05-27T07:43:00.020635+0000","last_fullsized":"2024-05-27T07:43:00.020635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:13.046162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.116649+0000","last_change":"2024-05-27T07:41:22.010129+0000","last_active":"2024-05-27T07:43:00.116649+0000","last_peered":"2024-05-27T07:43:00.116649+0000","last_clean":"2024-05-27T07:43:00.116649+0000","last_became_active":"2024-05-27T07:41:22.009608+0000","last_became_peered":"2024-05-27T07:41:22.009608+0000","last_unstale":"2024-05-27T07:43:00.116649+0000","last_undegraded":"2024-05-27T07:43:00.116649+0000","last_fullsized":"2024-05-27T07:43:00.116649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.225004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'16","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.010571+0000","last_change":"2024-05-27T07:41:04.794744+0000","last_active":"2024-05-27T07:43:00.010571+0000","last_peered":"2024-05-27T07:43:00.010571+0000","last_clean":"2024-05-27T07:43:00.010571+0000","last_became_active":"2024-05-27T07:41:04.794340+0000","last_became_peered":"2024-05-27T07:41:04.794340+0000","last_unstale":"2024-05-27T07:43:00.010571+0000","last_undegraded":"2024-05-27T07:43:00.010571+0000","last_fullsized":"2024-05-27T07:43:00.010571+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:43:20.335520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.119096+0000","last_change":"2024-05-27T07:41:08.841834+0000","last_active":"2024-05-27T07:43:00.119096+0000","last_peered":"2024-05-27T07:43:00.119096+0000","last_clean":"2024-05-27T07:43:00.119096+0000","last_became_active":"2024-05-27T07:41:08.840261+0000","last_became_peered":"2024-05-27T07:41:08.840261+0000","last_unstale":"2024-05-27T07:43:00.119096+0000","last_undegraded":"2024-05-27T07:43:00.119096+0000","last_fullsized":"2024-05-27T07:43:00.119096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:33.738835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011925+0000","last_change":"2024-05-27T07:41:22.005861+0000","last_active":"2024-05-27T07:43:00.011925+0000","last_peered":"2024-05-27T07:43:00.011925+0000","last_clean":"2024-05-27T07:43:00.011925+0000","last_became_active":"2024-05-27T07:41:22.005295+0000","last_became_peered":"2024-05-27T07:41:22.005295+0000","last_unstale":"2024-05-27T07:43:00.011925+0000","last_undegraded":"2024-05-27T07:43:00.011925+0000","last_fullsized":"2024-05-27T07:43:00.011925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:46.582656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013441+0000","last_change":"2024-05-27T07:41:04.796562+0000","last_active":"2024-05-27T07:43:00.013441+0000","last_peered":"2024-05-27T07:43:00.013441+0000","last_clean":"2024-05-27T07:43:00.013441+0000","last_became_active":"2024-05-27T07:41:04.795538+0000","last_became_peered":"2024-05-27T07:41:04.795538+0000","last_unstale":"2024-05-27T07:43:00.013441+0000","last_undegraded":"2024-05-27T07:43:00.013441+0000","last_fullsized":"2024-05-27T07:43:00.013441+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:12.131999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193665+0000","last_change":"2024-05-27T07:41:08.832824+0000","last_active":"2024-05-27T07:43:00.193665+0000","last_peered":"2024-05-27T07:43:00.193665+0000","last_clean":"2024-05-27T07:43:00.193665+0000","last_became_active":"2024-05-27T07:41:08.832332+0000","last_became_peered":"2024-05-27T07:41:08.832332+0000","last_unstale":"2024-05-27T07:43:00.193665+0000","last_undegraded":"2024-05-27T07:43:00.193665+0000","last_fullsized":"2024-05-27T07:43:00.193665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:34.598773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011686+0000","last_change":"2024-05-27T07:41:22.003358+0000","last_active":"2024-05-27T07:43:00.011686+0000","last_peered":"2024-05-27T07:43:00.011686+0000","last_clean":"2024-05-27T07:43:00.011686+0000","last_became_active":"2024-05-27T07:41:22.002655+0000","last_became_peered":"2024-05-27T07:41:22.002655+0000","last_unstale":"2024-05-27T07:43:00.011686+0000","last_undegraded":"2024-05-27T07:43:00.011686+0000","last_fullsized":"2024-05-27T07:43:00.011686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:41.177138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193606+0000","last_change":"2024-05-27T07:41:04.888681+0000","last_active":"2024-05-27T07:43:00.193606+0000","last_peered":"2024-05-27T07:43:00.193606+0000","last_clean":"2024-05-27T07:43:00.193606+0000","last_became_active":"2024-05-27T07:41:04.887966+0000","last_became_peered":"2024-05-27T07:41:04.887966+0000","last_unstale":"2024-05-27T07:43:00.193606+0000","last_undegraded":"2024-05-27T07:43:00.193606+0000","last_fullsized":"2024-05-27T07:43:00.193606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:01.178737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195221+0000","last_change":"2024-05-27T07:41:08.827839+0000","last_active":"2024-05-27T07:43:00.195221+0000","last_peered":"2024-05-27T07:43:00.195221+0000","last_clean":"2024-05-27T07:43:00.195221+0000","last_became_active":"2024-05-27T07:41:08.827403+0000","last_became_peered":"2024-05-27T07:41:08.827403+0000","last_unstale":"2024-05-27T07:43:00.195221+0000","last_undegraded":"2024-05-27T07:43:00.195221+0000","last_fullsized":"2024-05-27T07:43:00.195221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:01:42.075980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009996+0000","last_change":"2024-05-27T07:41:22.006161+0000","last_active":"2024-05-27T07:43:00.009996+0000","last_peered":"2024-05-27T07:43:00.009996+0000","last_clean":"2024-05-27T07:43:00.009996+0000","last_became_active":"2024-05-27T07:41:22.005762+0000","last_became_peered":"2024-05-27T07:41:22.005762+0000","last_unstale":"2024-05-27T07:43:00.009996+0000","last_undegraded":"2024-05-27T07:43:00.009996+0000","last_fullsized":"2024-05-27T07:43:00.009996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:53.536444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117166+0000","last_change":"2024-05-27T07:41:04.774211+0000","last_active":"2024-05-27T07:43:00.117166+0000","last_peered":"2024-05-27T07:43:00.117166+0000","last_clean":"2024-05-27T07:43:00.117166+0000","last_became_active":"2024-05-27T07:41:04.773215+0000","last_became_peered":"2024-05-27T07:41:04.773215+0000","last_unstale":"2024-05-27T07:43:00.117166+0000","last_undegraded":"2024-05-27T07:43:00.117166+0000","last_fullsized":"2024-05-27T07:43:00.117166+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:37.273735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193866+0000","last_change":"2024-05-27T07:41:08.830434+0000","last_active":"2024-05-27T07:43:00.193866+0000","last_peered":"2024-05-27T07:43:00.193866+0000","last_clean":"2024-05-27T07:43:00.193866+0000","last_became_active":"2024-05-27T07:41:08.830039+0000","last_became_peered":"2024-05-27T07:41:08.830039+0000","last_unstale":"2024-05-27T07:43:00.193866+0000","last_undegraded":"2024-05-27T07:43:00.193866+0000","last_fullsized":"2024-05-27T07:43:00.193866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:27.877666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012958+0000","last_change":"2024-05-27T07:41:59.783413+0000","last_active":"2024-05-27T07:43:00.012958+0000","last_peered":"2024-05-27T07:43:00.012958+0000","last_clean":"2024-05-27T07:43:00.012958+0000","last_became_active":"2024-05-27T07:41:59.782300+0000","last_became_peered":"2024-05-27T07:41:59.782300+0000","last_unstale":"2024-05-27T07:43:00.012958+0000","last_undegraded":"2024-05-27T07:43:00.012958+0000","last_fullsized":"2024-05-27T07:43:00.012958+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:59.348928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012459+0000","last_change":"2024-05-27T07:41:04.798969+0000","last_active":"2024-05-27T07:43:00.012459+0000","last_peered":"2024-05-27T07:43:00.012459+0000","last_clean":"2024-05-27T07:43:00.012459+0000","last_became_active":"2024-05-27T07:41:04.795996+0000","last_became_peered":"2024-05-27T07:41:04.795996+0000","last_unstale":"2024-05-27T07:43:00.012459+0000","last_undegraded":"2024-05-27T07:43:00.012459+0000","last_fullsized":"2024-05-27T07:43:00.012459+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:47.141063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117509+0000","last_change":"2024-05-27T07:41:08.835142+0000","last_active":"2024-05-27T07:43:00.117509+0000","last_peered":"2024-05-27T07:43:00.117509+0000","last_clean":"2024-05-27T07:43:00.117509+0000","last_became_active":"2024-05-27T07:41:08.834762+0000","last_became_peered":"2024-05-27T07:41:08.834762+0000","last_unstale":"2024-05-27T07:43:00.117509+0000","last_undegraded":"2024-05-27T07:43:00.117509+0000","last_fullsized":"2024-05-27T07:43:00.117509+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:52.957554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012271+0000","last_change":"2024-05-27T07:41:22.000649+0000","last_active":"2024-05-27T07:43:00.012271+0000","last_peered":"2024-05-27T07:43:00.012271+0000","last_clean":"2024-05-27T07:43:00.012271+0000","last_became_active":"2024-05-27T07:41:22.000161+0000","last_became_peered":"2024-05-27T07:41:22.000161+0000","last_unstale":"2024-05-27T07:43:00.012271+0000","last_undegraded":"2024-05-27T07:43:00.012271+0000","last_fullsized":"2024-05-27T07:43:00.012271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:10.788027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195761+0000","last_change":"2024-05-27T07:41:04.800680+0000","last_active":"2024-05-27T07:43:00.195761+0000","last_peered":"2024-05-27T07:43:00.195761+0000","last_clean":"2024-05-27T07:43:00.195761+0000","last_became_active":"2024-05-27T07:41:04.800249+0000","last_became_peered":"2024-05-27T07:41:04.800249+0000","last_unstale":"2024-05-27T07:43:00.195761+0000","last_undegraded":"2024-05-27T07:43:00.195761+0000","last_fullsized":"2024-05-27T07:43:00.195761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:23:27.797216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195250+0000","last_change":"2024-05-27T07:41:08.844380+0000","last_active":"2024-05-27T07:43:00.195250+0000","last_peered":"2024-05-27T07:43:00.195250+0000","last_clean":"2024-05-27T07:43:00.195250+0000","last_became_active":"2024-05-27T07:41:08.843464+0000","last_became_peered":"2024-05-27T07:41:08.843464+0000","last_unstale":"2024-05-27T07:43:00.195250+0000","last_undegraded":"2024-05-27T07:43:00.195250+0000","last_fullsized":"2024-05-27T07:43:00.195250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:23.290234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194261+0000","last_change":"2024-05-27T07:41:22.010968+0000","last_active":"2024-05-27T07:43:00.194261+0000","last_peered":"2024-05-27T07:43:00.194261+0000","last_clean":"2024-05-27T07:43:00.194261+0000","last_became_active":"2024-05-27T07:41:22.010529+0000","last_became_peered":"2024-05-27T07:41:22.010529+0000","last_unstale":"2024-05-27T07:43:00.194261+0000","last_undegraded":"2024-05-27T07:43:00.194261+0000","last_fullsized":"2024-05-27T07:43:00.194261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:08.636434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":88,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:10.033236+0000","last_change":"2024-05-27T07:41:04.891536+0000","last_active":"2024-05-27T07:43:10.033236+0000","last_peered":"2024-05-27T07:43:10.033236+0000","last_clean":"2024-05-27T07:43:10.033236+0000","last_became_active":"2024-05-27T07:41:04.890724+0000","last_became_peered":"2024-05-27T07:41:04.890724+0000","last_unstale":"2024-05-27T07:43:10.033236+0000","last_undegraded":"2024-05-27T07:43:10.033236+0000","last_fullsized":"2024-05-27T07:43:10.033236+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:32.027624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196699+0000","last_change":"2024-05-27T07:41:08.834362+0000","last_active":"2024-05-27T07:43:00.196699+0000","last_peered":"2024-05-27T07:43:00.196699+0000","last_clean":"2024-05-27T07:43:00.196699+0000","last_became_active":"2024-05-27T07:41:08.833979+0000","last_became_peered":"2024-05-27T07:41:08.833979+0000","last_unstale":"2024-05-27T07:43:00.196699+0000","last_undegraded":"2024-05-27T07:43:00.196699+0000","last_fullsized":"2024-05-27T07:43:00.196699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:35.098352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.192512+0000","last_change":"2024-05-27T07:41:21.997587+0000","last_active":"2024-05-27T07:43:00.192512+0000","last_peered":"2024-05-27T07:43:00.192512+0000","last_clean":"2024-05-27T07:43:00.192512+0000","last_became_active":"2024-05-27T07:41:21.996413+0000","last_became_peered":"2024-05-27T07:41:21.996413+0000","last_unstale":"2024-05-27T07:43:00.192512+0000","last_undegraded":"2024-05-27T07:43:00.192512+0000","last_fullsized":"2024-05-27T07:43:00.192512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:21:46.728047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042883+0000","last_change":"2024-05-27T07:41:04.793580+0000","last_active":"2024-05-27T07:43:00.042883+0000","last_peered":"2024-05-27T07:43:00.042883+0000","last_clean":"2024-05-27T07:43:00.042883+0000","last_became_active":"2024-05-27T07:41:04.791135+0000","last_became_peered":"2024-05-27T07:41:04.791135+0000","last_unstale":"2024-05-27T07:43:00.042883+0000","last_undegraded":"2024-05-27T07:43:00.042883+0000","last_fullsized":"2024-05-27T07:43:00.042883+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:15.484187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195882+0000","last_change":"2024-05-27T07:41:08.839187+0000","last_active":"2024-05-27T07:43:00.195882+0000","last_peered":"2024-05-27T07:43:00.195882+0000","last_clean":"2024-05-27T07:43:00.195882+0000","last_became_active":"2024-05-27T07:41:08.838821+0000","last_became_peered":"2024-05-27T07:41:08.838821+0000","last_unstale":"2024-05-27T07:43:00.195882+0000","last_undegraded":"2024-05-27T07:43:00.195882+0000","last_fullsized":"2024-05-27T07:43:00.195882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:30.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.040945+0000","last_change":"2024-05-27T07:41:22.005406+0000","last_active":"2024-05-27T07:43:00.040945+0000","last_peered":"2024-05-27T07:43:00.040945+0000","last_clean":"2024-05-27T07:43:00.040945+0000","last_became_active":"2024-05-27T07:41:22.003415+0000","last_became_peered":"2024-05-27T07:41:22.003415+0000","last_unstale":"2024-05-27T07:43:00.040945+0000","last_undegraded":"2024-05-27T07:43:00.040945+0000","last_fullsized":"2024-05-27T07:43:00.040945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:03.300452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018465+0000","last_change":"2024-05-27T07:41:04.798489+0000","last_active":"2024-05-27T07:43:00.018465+0000","last_peered":"2024-05-27T07:43:00.018465+0000","last_clean":"2024-05-27T07:43:00.018465+0000","last_became_active":"2024-05-27T07:41:04.797467+0000","last_became_peered":"2024-05-27T07:41:04.797467+0000","last_unstale":"2024-05-27T07:43:00.018465+0000","last_undegraded":"2024-05-27T07:43:00.018465+0000","last_fullsized":"2024-05-27T07:43:00.018465+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:04.018577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195995+0000","last_change":"2024-05-27T07:41:08.832215+0000","last_active":"2024-05-27T07:43:00.195995+0000","last_peered":"2024-05-27T07:43:00.195995+0000","last_clean":"2024-05-27T07:43:00.195995+0000","last_became_active":"2024-05-27T07:41:08.831491+0000","last_became_peered":"2024-05-27T07:41:08.831491+0000","last_unstale":"2024-05-27T07:43:00.195995+0000","last_undegraded":"2024-05-27T07:43:00.195995+0000","last_fullsized":"2024-05-27T07:43:00.195995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:32:03.084324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194151+0000","last_change":"2024-05-27T07:41:08.837301+0000","last_active":"2024-05-27T07:43:00.194151+0000","last_peered":"2024-05-27T07:43:00.194151+0000","last_clean":"2024-05-27T07:43:00.194151+0000","last_became_active":"2024-05-27T07:41:08.835533+0000","last_became_peered":"2024-05-27T07:41:08.835533+0000","last_unstale":"2024-05-27T07:43:00.194151+0000","last_undegraded":"2024-05-27T07:43:00.194151+0000","last_fullsized":"2024-05-27T07:43:00.194151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:45.807613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013039+0000","last_change":"2024-05-27T07:41:08.846740+0000","last_active":"2024-05-27T07:43:00.013039+0000","last_peered":"2024-05-27T07:43:00.013039+0000","last_clean":"2024-05-27T07:43:00.013039+0000","last_became_active":"2024-05-27T07:41:08.845576+0000","last_became_peered":"2024-05-27T07:41:08.845576+0000","last_unstale":"2024-05-27T07:43:00.013039+0000","last_undegraded":"2024-05-27T07:43:00.013039+0000","last_fullsized":"2024-05-27T07:43:00.013039+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:41:36.944923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012116+0000","last_change":"2024-05-27T07:41:08.840121+0000","last_active":"2024-05-27T07:43:00.012116+0000","last_peered":"2024-05-27T07:43:00.012116+0000","last_clean":"2024-05-27T07:43:00.012116+0000","last_became_active":"2024-05-27T07:41:08.839090+0000","last_became_peered":"2024-05-27T07:41:08.839090+0000","last_unstale":"2024-05-27T07:43:00.012116+0000","last_undegraded":"2024-05-27T07:43:00.012116+0000","last_fullsized":"2024-05-27T07:43:00.012116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:44.836953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193149+0000","last_change":"2024-05-27T07:41:08.835565+0000","last_active":"2024-05-27T07:43:00.193149+0000","last_peered":"2024-05-27T07:43:00.193149+0000","last_clean":"2024-05-27T07:43:00.193149+0000","last_became_active":"2024-05-27T07:41:08.831424+0000","last_became_peered":"2024-05-27T07:41:08.831424+0000","last_unstale":"2024-05-27T07:43:00.193149+0000","last_undegraded":"2024-05-27T07:43:00.193149+0000","last_fullsized":"2024-05-27T07:43:00.193149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:17.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.021085+0000","last_change":"2024-05-27T07:41:08.830104+0000","last_active":"2024-05-27T07:43:00.021085+0000","last_peered":"2024-05-27T07:43:00.021085+0000","last_clean":"2024-05-27T07:43:00.021085+0000","last_became_active":"2024-05-27T07:41:08.829690+0000","last_became_peered":"2024-05-27T07:41:08.829690+0000","last_unstale":"2024-05-27T07:43:00.021085+0000","last_undegraded":"2024-05-27T07:43:00.021085+0000","last_fullsized":"2024-05-27T07:43:00.021085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194789+0000","last_change":"2024-05-27T07:41:08.838654+0000","last_active":"2024-05-27T07:43:00.194789+0000","last_peered":"2024-05-27T07:43:00.194789+0000","last_clean":"2024-05-27T07:43:00.194789+0000","last_became_active":"2024-05-27T07:41:08.835033+0000","last_became_peered":"2024-05-27T07:41:08.835033+0000","last_unstale":"2024-05-27T07:43:00.194789+0000","last_undegraded":"2024-05-27T07:43:00.194789+0000","last_fullsized":"2024-05-27T07:43:00.194789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:04:24.007585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013026+0000","last_change":"2024-05-27T07:41:08.844557+0000","last_active":"2024-05-27T07:43:00.013026+0000","last_peered":"2024-05-27T07:43:00.013026+0000","last_clean":"2024-05-27T07:43:00.013026+0000","last_became_active":"2024-05-27T07:41:08.843418+0000","last_became_peered":"2024-05-27T07:41:08.843418+0000","last_unstale":"2024-05-27T07:43:00.013026+0000","last_undegraded":"2024-05-27T07:43:00.013026+0000","last_fullsized":"2024-05-27T07:43:00.013026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:39.587778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.119172+0000","last_change":"2024-05-27T07:41:08.830857+0000","last_active":"2024-05-27T07:43:00.119172+0000","last_peered":"2024-05-27T07:43:00.119172+0000","last_clean":"2024-05-27T07:43:00.119172+0000","last_became_active":"2024-05-27T07:41:08.830480+0000","last_became_peered":"2024-05-27T07:41:08.830480+0000","last_unstale":"2024-05-27T07:43:00.119172+0000","last_undegraded":"2024-05-27T07:43:00.119172+0000","last_fullsized":"2024-05-27T07:43:00.119172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:07.307528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041750+0000","last_change":"2024-05-27T07:41:08.824571+0000","last_active":"2024-05-27T07:43:00.041750+0000","last_peered":"2024-05-27T07:43:00.041750+0000","last_clean":"2024-05-27T07:43:00.041750+0000","last_became_active":"2024-05-27T07:41:08.820732+0000","last_became_peered":"2024-05-27T07:41:08.820732+0000","last_unstale":"2024-05-27T07:43:00.041750+0000","last_undegraded":"2024-05-27T07:43:00.041750+0000","last_fullsized":"2024-05-27T07:43:00.041750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:55:14.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117363+0000","last_change":"2024-05-27T07:41:08.842284+0000","last_active":"2024-05-27T07:43:00.117363+0000","last_peered":"2024-05-27T07:43:00.117363+0000","last_clean":"2024-05-27T07:43:00.117363+0000","last_became_active":"2024-05-27T07:41:08.841597+0000","last_became_peered":"2024-05-27T07:41:08.841597+0000","last_unstale":"2024-05-27T07:43:00.117363+0000","last_undegraded":"2024-05-27T07:43:00.117363+0000","last_fullsized":"2024-05-27T07:43:00.117363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:25:31.066181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195588+0000","last_change":"2024-05-27T07:41:08.828935+0000","last_active":"2024-05-27T07:43:00.195588+0000","last_peered":"2024-05-27T07:43:00.195588+0000","last_clean":"2024-05-27T07:43:00.195588+0000","last_became_active":"2024-05-27T07:41:08.828542+0000","last_became_peered":"2024-05-27T07:41:08.828542+0000","last_unstale":"2024-05-27T07:43:00.195588+0000","last_undegraded":"2024-05-27T07:43:00.195588+0000","last_fullsized":"2024-05-27T07:43:00.195588+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:35.270666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.017962+0000","last_change":"2024-05-27T07:41:08.817051+0000","last_active":"2024-05-27T07:43:00.017962+0000","last_peered":"2024-05-27T07:43:00.017962+0000","last_clean":"2024-05-27T07:43:00.017962+0000","last_became_active":"2024-05-27T07:41:08.816418+0000","last_became_peered":"2024-05-27T07:41:08.816418+0000","last_unstale":"2024-05-27T07:43:00.017962+0000","last_undegraded":"2024-05-27T07:43:00.017962+0000","last_fullsized":"2024-05-27T07:43:00.017962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:02.760707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194966+0000","last_change":"2024-05-27T07:41:08.834081+0000","last_active":"2024-05-27T07:43:00.194966+0000","last_peered":"2024-05-27T07:43:00.194966+0000","last_clean":"2024-05-27T07:43:00.194966+0000","last_became_active":"2024-05-27T07:41:08.833679+0000","last_became_peered":"2024-05-27T07:41:08.833679+0000","last_unstale":"2024-05-27T07:43:00.194966+0000","last_undegraded":"2024-05-27T07:43:00.194966+0000","last_fullsized":"2024-05-27T07:43:00.194966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:41:14.426593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72586,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":393,"num_read_kb":275,"num_write":251,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":217758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645354,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":522393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841586,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1178966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005113,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":522211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201344,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":522477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397576,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1119477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561103,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":517715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":584127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":519244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117094,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":517423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920863,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":587674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757335,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1675264,"data_stored":1114668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:43:15.378 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T07:43:15.378 INFO:tasks.cephadm:Setting up client nodes... 2024-05-27T07:43:15.379 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-27T07:43:15.853 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:15.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:15 smithi090 ceph-mon[31429]: pgmap v449: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-27T07:43:15.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:15.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:15.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:15.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:15 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:15.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:15 smithi136 ceph-mon[35324]: pgmap v449: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-27T07:43:15.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:15.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:15.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:15.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:15 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:16.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:15 smithi129 ceph-mon[35254]: pgmap v449: 129 pgs: 129 active+clean; 648 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1023 B/s wr, 9 op/s 2024-05-27T07:43:16.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:16.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:16.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:16.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:15 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:17.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:17 smithi090 ceph-mon[31429]: pgmap v450: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-27T07:43:17.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:17.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:17 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:18.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:17 smithi129 ceph-mon[35254]: pgmap v450: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-27T07:43:18.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:17 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:18.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:17 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:18.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.110+0000 7f897a000700 1 -- 172.21.15.90:0/2756176768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8974104e30 msgr2=0x7f8974107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.110+0000 7f897a000700 1 --2- 172.21.15.90:0/2756176768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8974104e30 0x7f8974107250 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8964009990 tx=0x7f8964009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.110+0000 7f897a000700 1 -- 172.21.15.90:0/2756176768 shutdown_connections 2024-05-27T07:43:18.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.110+0000 7f897a000700 1 --2- 172.21.15.90:0/2756176768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f897410a0f0 0x7f897410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.110+0000 7f897a000700 1 --2- 172.21.15.90:0/2756176768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8974104e30 0x7f8974107250 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.110+0000 7f897a000700 1 --2- 172.21.15.90:0/2756176768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8974107790 0x7f8974109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.110+0000 7f897a000700 1 -- 172.21.15.90:0/2756176768 >> 172.21.15.90:0/2756176768 conn(0x7f89740fe870 msgr2=0x7f8974100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:18.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.110+0000 7f897a000700 1 -- 172.21.15.90:0/2756176768 shutdown_connections 2024-05-27T07:43:18.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.110+0000 7f897a000700 1 -- 172.21.15.90:0/2756176768 wait complete. 2024-05-27T07:43:18.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.111+0000 7f897a000700 1 Processor -- start 2024-05-27T07:43:18.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.112+0000 7f897a000700 1 -- start start 2024-05-27T07:43:18.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.112+0000 7f897a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8974104e30 0x7f897419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.112+0000 7f897a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8974107790 0x7f897419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.112+0000 7f897a000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897410a0f0 0x7f89741a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.112+0000 7f897a000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f897410fc30 con 0x7f8974104e30 2024-05-27T07:43:18.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.112+0000 7f897a000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f897410fab0 con 0x7f897410a0f0 2024-05-27T07:43:18.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.112+0000 7f897a000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f897410fdb0 con 0x7f8974107790 2024-05-27T07:43:18.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.112+0000 7f8973400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8974104e30 0x7f897419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.112+0000 7f8973400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8974104e30 0x7f897419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44106/0 (socket says 172.21.15.90:44106) 2024-05-27T07:43:18.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.112+0000 7f8973400700 1 -- 172.21.15.90:0/644018980 learned_addr learned my addr 172.21.15.90:0/644018980 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:18.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f8973400700 1 -- 172.21.15.90:0/644018980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8974107790 msgr2=0x7f897419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f8973400700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8974107790 0x7f897419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f8973400700 1 -- 172.21.15.90:0/644018980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897410a0f0 msgr2=0x7f89741a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f8973e00700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897410a0f0 0x7f89741a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f8973400700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897410a0f0 0x7f89741a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f8973400700 1 -- 172.21.15.90:0/644018980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89741a45a0 con 0x7f8974104e30 2024-05-27T07:43:18.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f8973400700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8974104e30 0x7f897419ca40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8964000c00 tx=0x7f8964009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:18.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f8970c00700 1 -- 172.21.15.90:0/644018980 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964027070 con 0x7f8974104e30 2024-05-27T07:43:18.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f8970c00700 1 -- 172.21.15.90:0/644018980 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89640040a0 con 0x7f8974104e30 2024-05-27T07:43:18.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f897a000700 1 -- 172.21.15.90:0/644018980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89741a4830 con 0x7f8974104e30 2024-05-27T07:43:18.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.113+0000 7f8970c00700 1 -- 172.21.15.90:0/644018980 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964022d00 con 0x7f8974104e30 2024-05-27T07:43:18.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.114+0000 7f897a000700 1 -- 172.21.15.90:0/644018980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89741a4ce0 con 0x7f8974104e30 2024-05-27T07:43:18.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.115+0000 7f897a000700 1 -- 172.21.15.90:0/644018980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89740512b0 con 0x7f8974104e30 2024-05-27T07:43:18.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.119+0000 7f8970c00700 1 -- 172.21.15.90:0/644018980 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f896402a070 con 0x7f8974104e30 2024-05-27T07:43:18.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.120+0000 7f8970c00700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89580b2990 0x7f89580b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:18.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.121+0000 7f8970c00700 1 -- 172.21.15.90:0/644018980 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f89640e3f60 con 0x7f8974104e30 2024-05-27T07:43:18.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.121+0000 7f896aa00700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89580b2990 0x7f89580b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:18.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.122+0000 7f896aa00700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89580b2990 0x7f89580b4ea0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f895c0060b0 tx=0x7f895c006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:18.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.123+0000 7f8970c00700 1 -- 172.21.15.90:0/644018980 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8964013070 con 0x7f8974104e30 2024-05-27T07:43:18.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:17 smithi136 ceph-mon[35324]: pgmap v450: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-27T07:43:18.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:18.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:17 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.448+0000 7f897a000700 1 -- 172.21.15.90:0/644018980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f89741a5300 con 0x7f8974104e30 2024-05-27T07:43:18.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.454+0000 7f8970c00700 1 -- 172.21.15.90:0/644018980 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v34) ==== 170+0+59 (secure 0 0 0) 0x7f89741a5300 con 0x7f8974104e30 2024-05-27T07:43:18.453 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-05-27T07:43:18.453 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCWOVRmY8PiGhAADtPxzHeoNmpXsb5L/VDV9w== 2024-05-27T07:43:18.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.456+0000 7f897a000700 1 -- 172.21.15.90:0/644018980 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89580b2990 msgr2=0x7f89580b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.456+0000 7f897a000700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89580b2990 0x7f89580b4ea0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f895c0060b0 tx=0x7f895c006040 comp rx=0 tx=0).stop 2024-05-27T07:43:18.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.457+0000 7f897a000700 1 -- 172.21.15.90:0/644018980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8974104e30 msgr2=0x7f897419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:18.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.457+0000 7f897a000700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8974104e30 0x7f897419ca40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8964000c00 tx=0x7f8964009510 comp rx=0 tx=0).stop 2024-05-27T07:43:18.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.457+0000 7f897a000700 1 -- 172.21.15.90:0/644018980 shutdown_connections 2024-05-27T07:43:18.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.457+0000 7f897a000700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f89580b2990 0x7f89580b4ea0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.457+0000 7f897a000700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f897410a0f0 0x7f89741a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.457+0000 7f897a000700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8974104e30 0x7f897419ca40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.457+0000 7f897a000700 1 --2- 172.21.15.90:0/644018980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8974107790 0x7f897419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:18.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.457+0000 7f897a000700 1 -- 172.21.15.90:0/644018980 >> 172.21.15.90:0/644018980 conn(0x7f89740fe870 msgr2=0x7f8974102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:18.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.457+0000 7f897a000700 1 -- 172.21.15.90:0/644018980 shutdown_connections 2024-05-27T07:43:18.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:18.457+0000 7f897a000700 1 -- 172.21.15.90:0/644018980 wait complete. 2024-05-27T07:43:19.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:18 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:43:19.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:18 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:43:19.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:18 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:19.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:18 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/644018980' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T07:43:19.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:18 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/644018980' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T07:43:19.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:18 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:43:19.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:18 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:43:19.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:18 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:19.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:18 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/644018980' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T07:43:19.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:18 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/644018980' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T07:43:19.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:18 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:43:19.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:18 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:43:19.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:18 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:43:19.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:18 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/644018980' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T07:43:19.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:18 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/644018980' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T07:43:19.326 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T07:43:19.326 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-27T07:43:19.327 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-27T07:43:19.445 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-27T07:43:19.445 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-27T07:43:19.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-05-27T07:43:19.931 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:20.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:20 smithi090 ceph-mon[31429]: pgmap v451: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:43:20.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:20 smithi129 ceph-mon[35254]: pgmap v451: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:43:20.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:20 smithi136 ceph-mon[35324]: pgmap v451: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:43:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.269+0000 7f0e54600700 1 -- 172.21.15.90:0/2728030699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e4c106eb0 msgr2=0x7f0e4c113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.269+0000 7f0e54600700 1 --2- 172.21.15.90:0/2728030699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e4c106eb0 0x7f0e4c113340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400b0a0 tx=0x7f0e4400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:43:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.269+0000 7f0e54600700 1 -- 172.21.15.90:0/2728030699 shutdown_connections 2024-05-27T07:43:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.269+0000 7f0e54600700 1 --2- 172.21.15.90:0/2728030699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e4c106eb0 0x7f0e4c113340 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.269+0000 7f0e54600700 1 --2- 172.21.15.90:0/2728030699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e4c10c5a0 0x7f0e4c10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.269+0000 7f0e54600700 1 --2- 172.21.15.90:0/2728030699 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e4c106560 0x7f0e4c106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.269+0000 7f0e54600700 1 -- 172.21.15.90:0/2728030699 >> 172.21.15.90:0/2728030699 conn(0x7f0e4c102040 msgr2=0x7f0e4c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:22.270 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:22 smithi090 ceph-mon[31429]: pgmap v452: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-27T07:43:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.269+0000 7f0e54600700 1 -- 172.21.15.90:0/2728030699 shutdown_connections 2024-05-27T07:43:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.269+0000 7f0e54600700 1 -- 172.21.15.90:0/2728030699 wait complete. 2024-05-27T07:43:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.271+0000 7f0e54600700 1 Processor -- start 2024-05-27T07:43:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.271+0000 7f0e54600700 1 -- start start 2024-05-27T07:43:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.271+0000 7f0e54600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e4c106560 0x7f0e4c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.271+0000 7f0e54600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e4c106eb0 0x7f0e4c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.271+0000 7f0e54600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e4c10c5a0 0x7f0e4c1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.271+0000 7f0e54600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c1165d0 con 0x7f0e4c106eb0 2024-05-27T07:43:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.271+0000 7f0e54600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c116450 con 0x7f0e4c10c5a0 2024-05-27T07:43:22.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.271+0000 7f0e54600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e4c116750 con 0x7f0e4c106560 2024-05-27T07:43:22.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e51200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e4c106eb0 0x7f0e4c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:22.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e51200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e4c106eb0 0x7f0e4c19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44892/0 (socket says 172.21.15.90:44892) 2024-05-27T07:43:22.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e51200700 1 -- 172.21.15.90:0/3767001102 learned_addr learned my addr 172.21.15.90:0/3767001102 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:22.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e51c00700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e4c106560 0x7f0e4c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e52600700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e4c10c5a0 0x7f0e4c1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e51200700 1 -- 172.21.15.90:0/3767001102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e4c106560 msgr2=0x7f0e4c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e51200700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e4c106560 0x7f0e4c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e51200700 1 -- 172.21.15.90:0/3767001102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e4c10c5a0 msgr2=0x7f0e4c1a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e51200700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e4c10c5a0 0x7f0e4c1a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e51200700 1 -- 172.21.15.90:0/3767001102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4c1a1df0 con 0x7f0e4c106eb0 2024-05-27T07:43:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.272+0000 7f0e52600700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e4c10c5a0 0x7f0e4c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:43:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.273+0000 7f0e51200700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e4c106eb0 0x7f0e4c19d4c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800d950 tx=0x7f0e3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.273+0000 7f0e4a000700 1 -- 172.21.15.90:0/3767001102 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38017070 con 0x7f0e4c106eb0 2024-05-27T07:43:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.273+0000 7f0e4a000700 1 -- 172.21.15.90:0/3767001102 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e38009dc0 con 0x7f0e4c106eb0 2024-05-27T07:43:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.273+0000 7f0e54600700 1 -- 172.21.15.90:0/3767001102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4c1a2080 con 0x7f0e4c106eb0 2024-05-27T07:43:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.273+0000 7f0e4a000700 1 -- 172.21.15.90:0/3767001102 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800f3c0 con 0x7f0e4c106eb0 2024-05-27T07:43:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.273+0000 7f0e54600700 1 -- 172.21.15.90:0/3767001102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4c1a9a00 con 0x7f0e4c106eb0 2024-05-27T07:43:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.274+0000 7f0e54600700 1 -- 172.21.15.90:0/3767001102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e4c0512b0 con 0x7f0e4c106eb0 2024-05-27T07:43:22.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.277+0000 7f0e4a000700 1 -- 172.21.15.90:0/3767001102 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0e3801a070 con 0x7f0e4c106eb0 2024-05-27T07:43:22.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.279+0000 7f0e4a000700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e340b2990 0x7f0e340b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:22.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.279+0000 7f0e4a000700 1 -- 172.21.15.90:0/3767001102 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f0e380e1f90 con 0x7f0e4c106eb0 2024-05-27T07:43:22.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.279+0000 7f0e51c00700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e340b2990 0x7f0e340b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:22.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.280+0000 7f0e51c00700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e340b2990 0x7f0e340b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005950 tx=0x7f0e4000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:22.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.282+0000 7f0e4a000700 1 -- 172.21.15.90:0/3767001102 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e380a97e0 con 0x7f0e4c106eb0 2024-05-27T07:43:22.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:22 smithi129 ceph-mon[35254]: pgmap v452: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-27T07:43:22.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:22 smithi136 ceph-mon[35324]: pgmap v452: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-27T07:43:22.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.574+0000 7f0e54600700 1 -- 172.21.15.90:0/3767001102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0e4c02e5e0 con 0x7f0e4c106eb0 2024-05-27T07:43:22.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.582+0000 7f0e4a000700 1 -- 172.21.15.90:0/3767001102 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288646 (secure 0 0 0) 0x7f0e380a90e0 con 0x7f0e4c106eb0 2024-05-27T07:43:22.583 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:43:22.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.590+0000 7f0e54600700 1 -- 172.21.15.90:0/3767001102 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e340b2990 msgr2=0x7f0e340b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:22.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.590+0000 7f0e54600700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e340b2990 0x7f0e340b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0e40005950 tx=0x7f0e4000b410 comp rx=0 tx=0).stop 2024-05-27T07:43:22.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.591+0000 7f0e54600700 1 -- 172.21.15.90:0/3767001102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e4c106eb0 msgr2=0x7f0e4c19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:22.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.591+0000 7f0e54600700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e4c106eb0 0x7f0e4c19d4c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800d950 tx=0x7f0e3800dc60 comp rx=0 tx=0).stop 2024-05-27T07:43:22.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.591+0000 7f0e54600700 1 -- 172.21.15.90:0/3767001102 shutdown_connections 2024-05-27T07:43:22.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.591+0000 7f0e54600700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e340b2990 0x7f0e340b4ea0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:22.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.591+0000 7f0e54600700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e4c10c5a0 0x7f0e4c1a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:22.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.591+0000 7f0e54600700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e4c106560 0x7f0e4c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:22.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.591+0000 7f0e54600700 1 --2- 172.21.15.90:0/3767001102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e4c106eb0 0x7f0e4c19d4c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:22.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.591+0000 7f0e54600700 1 -- 172.21.15.90:0/3767001102 >> 172.21.15.90:0/3767001102 conn(0x7f0e4c102040 msgr2=0x7f0e4c1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:22.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.591+0000 7f0e54600700 1 -- 172.21.15.90:0/3767001102 shutdown_connections 2024-05-27T07:43:22.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:22.591+0000 7f0e54600700 1 -- 172.21.15.90:0/3767001102 wait complete. 2024-05-27T07:43:23.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:23 smithi090 ceph-mon[31429]: pgmap v453: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:43:23.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:23 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3767001102' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T07:43:23.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:23 smithi129 ceph-mon[35254]: pgmap v453: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:43:23.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:23 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3767001102' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T07:43:23.456 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":2854405112},{"type":"v1","addr":"172.21.15.90:6801","nonce":2854405112}]},"active_addr":"172.21.15.90:6801/2854405112","active_change":"2024-05-27T07:30:58.154769+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34112,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.90:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2692415447}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2932841075}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":3526000120}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":836242224}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2929565286}]}]} 2024-05-27T07:43:23.462 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-27T07:43:23.462 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-27T07:43:23.462 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:43:23.497 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:23 smithi136 ceph-mon[35324]: pgmap v453: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-27T07:43:23.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:23 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3767001102' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T07:43:23.913 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:25.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:25 smithi136 ceph-mon[35324]: pgmap v454: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T07:43:25.793 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:25 smithi090 ceph-mon[31429]: pgmap v454: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T07:43:25.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:25 smithi129 ceph-mon[35254]: pgmap v454: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-27T07:43:26.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.182+0000 7f4dc5c00700 1 -- 172.21.15.90:0/276215734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dc010b2d0 msgr2=0x7f4dc0117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.182+0000 7f4dc5c00700 1 --2- 172.21.15.90:0/276215734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dc010b2d0 0x7f4dc0117800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4db800b0a0 tx=0x7f4db800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.182+0000 7f4dc5c00700 1 -- 172.21.15.90:0/276215734 shutdown_connections 2024-05-27T07:43:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.182+0000 7f4dc5c00700 1 --2- 172.21.15.90:0/276215734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dc010b2d0 0x7f4dc0117800 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.182+0000 7f4dc5c00700 1 --2- 172.21.15.90:0/276215734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dc0110960 0x7f4dc0110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.182+0000 7f4dc5c00700 1 --2- 172.21.15.90:0/276215734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dc010a980 0x7f4dc010ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.182+0000 7f4dc5c00700 1 -- 172.21.15.90:0/276215734 >> 172.21.15.90:0/276215734 conn(0x7f4dc0077c70 msgr2=0x7f4dc00780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.182+0000 7f4dc5c00700 1 -- 172.21.15.90:0/276215734 shutdown_connections 2024-05-27T07:43:26.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.182+0000 7f4dc5c00700 1 -- 172.21.15.90:0/276215734 wait complete. 2024-05-27T07:43:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.183+0000 7f4dc5c00700 1 Processor -- start 2024-05-27T07:43:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.183+0000 7f4dc5c00700 1 -- start start 2024-05-27T07:43:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.183+0000 7f4dc5c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dc010a980 0x7f4dc01a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.183+0000 7f4dc5c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dc010b2d0 0x7f4dc01a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4dc5c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dc0110960 0x7f4dc01a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4dc5c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc011a980 con 0x7f4dc010b2d0 2024-05-27T07:43:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4dc5c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc011a800 con 0x7f4dc010a980 2024-05-27T07:43:26.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4dc5c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc011ab00 con 0x7f4dc0110960 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4db7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dc010b2d0 0x7f4dc01a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4db7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dc010b2d0 0x7f4dc01a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44908/0 (socket says 172.21.15.90:44908) 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4db7e00700 1 -- 172.21.15.90:0/2374973218 learned_addr learned my addr 172.21.15.90:0/2374973218 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4dbea00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dc010a980 0x7f4dc01a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4dbf400700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dc0110960 0x7f4dc01a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4db7e00700 1 -- 172.21.15.90:0/2374973218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dc0110960 msgr2=0x7f4dc01a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4db7e00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dc0110960 0x7f4dc01a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4db7e00700 1 -- 172.21.15.90:0/2374973218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dc010a980 msgr2=0x7f4dc01a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4db7e00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dc010a980 0x7f4dc01a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.184+0000 7f4db7e00700 1 -- 172.21.15.90:0/2374973218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc01a60e0 con 0x7f4dc010b2d0 2024-05-27T07:43:26.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.185+0000 7f4dbf400700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dc0110960 0x7f4dc01a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:43:26.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.185+0000 7f4db7e00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dc010b2d0 0x7f4dc01a18b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4da800c9e0 tx=0x7f4da800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:26.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.185+0000 7f4db7400700 1 -- 172.21.15.90:0/2374973218 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8007aa0 con 0x7f4dc010b2d0 2024-05-27T07:43:26.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.185+0000 7f4dc5c00700 1 -- 172.21.15.90:0/2374973218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc01a6370 con 0x7f4dc010b2d0 2024-05-27T07:43:26.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.185+0000 7f4db7400700 1 -- 172.21.15.90:0/2374973218 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4da8007c40 con 0x7f4dc010b2d0 2024-05-27T07:43:26.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.185+0000 7f4db7400700 1 -- 172.21.15.90:0/2374973218 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8018660 con 0x7f4dc010b2d0 2024-05-27T07:43:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.185+0000 7f4dc5c00700 1 -- 172.21.15.90:0/2374973218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc01a68b0 con 0x7f4dc010b2d0 2024-05-27T07:43:26.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.187+0000 7f4dc5c00700 1 -- 172.21.15.90:0/2374973218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc00512b0 con 0x7f4dc010b2d0 2024-05-27T07:43:26.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.189+0000 7f4db7400700 1 -- 172.21.15.90:0/2374973218 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4da8018800 con 0x7f4dc010b2d0 2024-05-27T07:43:26.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.191+0000 7f4db7400700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4da40b2910 0x7f4da40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:26.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.191+0000 7f4db7400700 1 -- 172.21.15.90:0/2374973218 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f4da80e2950 con 0x7f4dc010b2d0 2024-05-27T07:43:26.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.191+0000 7f4dbea00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4da40b2910 0x7f4da40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:26.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.192+0000 7f4dbea00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4da40b2910 0x7f4da40b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4db0005fd0 tx=0x7f4db000af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:26.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.193+0000 7f4db7400700 1 -- 172.21.15.90:0/2374973218 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4da8054df0 con 0x7f4dc010b2d0 2024-05-27T07:43:26.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.422+0000 7f4dc5c00700 1 -- 172.21.15.90:0/2374973218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4dc002e5e0 con 0x7f4dc010b2d0 2024-05-27T07:43:26.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.425+0000 7f4db7400700 1 -- 172.21.15.90:0/2374973218 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24725 (secure 0 0 0) 0x7f4da8014030 con 0x7f4dc010b2d0 2024-05-27T07:43:26.425 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:43:26.425 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":91,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:43:00.000806+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.428+0000 7f4dc5c00700 1 -- 172.21.15.90:0/2374973218 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4da40b2910 msgr2=0x7f4da40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.428+0000 7f4dc5c00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4da40b2910 0x7f4da40b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4db0005fd0 tx=0x7f4db000af20 comp rx=0 tx=0).stop 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.428+0000 7f4dc5c00700 1 -- 172.21.15.90:0/2374973218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dc010b2d0 msgr2=0x7f4dc01a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.428+0000 7f4dc5c00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dc010b2d0 0x7f4dc01a18b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4da800c9e0 tx=0x7f4da800cda0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.428+0000 7f4dc5c00700 1 -- 172.21.15.90:0/2374973218 shutdown_connections 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.428+0000 7f4dc5c00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4da40b2910 0x7f4da40b4e20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.428+0000 7f4dc5c00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4dc0110960 0x7f4dc01a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.428+0000 7f4dc5c00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dc010a980 0x7f4dc01a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.428+0000 7f4dc5c00700 1 --2- 172.21.15.90:0/2374973218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4dc010b2d0 0x7f4dc01a18b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.428+0000 7f4dc5c00700 1 -- 172.21.15.90:0/2374973218 >> 172.21.15.90:0/2374973218 conn(0x7f4dc0077c70 msgr2=0x7f4dc007b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.429+0000 7f4dc5c00700 1 -- 172.21.15.90:0/2374973218 shutdown_connections 2024-05-27T07:43:26.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:26.429+0000 7f4dc5c00700 1 -- 172.21.15.90:0/2374973218 wait complete. 2024-05-27T07:43:26.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:26 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2374973218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:43:26.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:26 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2374973218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:43:26.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:26 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2374973218' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:43:27.317 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-27T07:43:27.317 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:43:27.763 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:27.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:27 smithi129 ceph-mon[35254]: pgmap v455: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-27T07:43:27.911 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:27 smithi090 ceph-mon[31429]: pgmap v455: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-27T07:43:27.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:27 smithi136 ceph-mon[35324]: pgmap v455: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-05-27T07:43:29.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:29 smithi090 ceph-mon[31429]: pgmap v456: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:43:29.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:29 smithi129 ceph-mon[35254]: pgmap v456: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:43:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.877+0000 7f443d600700 1 -- 172.21.15.90:0/2081326196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44381043f0 msgr2=0x7f4438104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.877+0000 7f443d600700 1 --2- 172.21.15.90:0/2081326196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44381043f0 0x7f4438104800 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f442800b0d0 tx=0x7f442800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:43:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.877+0000 7f443d600700 1 -- 172.21.15.90:0/2081326196 shutdown_connections 2024-05-27T07:43:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.877+0000 7f443d600700 1 --2- 172.21.15.90:0/2081326196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438109670 0x7f4438110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.877+0000 7f443d600700 1 --2- 172.21.15.90:0/2081326196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44381043f0 0x7f4438104800 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.877+0000 7f443d600700 1 --2- 172.21.15.90:0/2081326196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4438108d20 0x7f4438109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.877+0000 7f443d600700 1 -- 172.21.15.90:0/2081326196 >> 172.21.15.90:0/2081326196 conn(0x7f4438079d00 msgr2=0x7f443807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.877+0000 7f443d600700 1 -- 172.21.15.90:0/2081326196 shutdown_connections 2024-05-27T07:43:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.878+0000 7f443d600700 1 -- 172.21.15.90:0/2081326196 wait complete. 2024-05-27T07:43:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.879+0000 7f443d600700 1 Processor -- start 2024-05-27T07:43:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.880+0000 7f443d600700 1 -- start start 2024-05-27T07:43:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.880+0000 7f443d600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44381043f0 0x7f44381a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.880+0000 7f443d600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4438108d20 0x7f44381a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.880+0000 7f443d600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438109670 0x7f443819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.880+0000 7f443d600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4438114150 con 0x7f44381043f0 2024-05-27T07:43:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.880+0000 7f443d600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4438113fd0 con 0x7f4438108d20 2024-05-27T07:43:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.880+0000 7f443d600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f44381142d0 con 0x7f4438109670 2024-05-27T07:43:29.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f4437e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44381043f0 0x7f44381a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f4437e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44381043f0 0x7f44381a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34434/0 (socket says 172.21.15.90:34434) 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f4437e00700 1 -- 172.21.15.90:0/1309716641 learned_addr learned my addr 172.21.15.90:0/1309716641 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f442f400700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4438108d20 0x7f44381a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f443cc00700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438109670 0x7f443819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f4437e00700 1 -- 172.21.15.90:0/1309716641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438109670 msgr2=0x7f443819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f4437e00700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438109670 0x7f443819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f4437e00700 1 -- 172.21.15.90:0/1309716641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4438108d20 msgr2=0x7f44381a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f4437e00700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4438108d20 0x7f44381a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f4437e00700 1 -- 172.21.15.90:0/1309716641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443819b060 con 0x7f44381043f0 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.881+0000 7f442f400700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4438108d20 0x7f44381a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.882+0000 7f4437e00700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44381043f0 0x7f44381a0790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4428007d90 tx=0x7f4428007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.882+0000 7f4435600700 1 -- 172.21.15.90:0/1309716641 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442801f030 con 0x7f44381043f0 2024-05-27T07:43:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.882+0000 7f4435600700 1 -- 172.21.15.90:0/1309716641 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f442801bad0 con 0x7f44381043f0 2024-05-27T07:43:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.882+0000 7f443d600700 1 -- 172.21.15.90:0/1309716641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443819b2f0 con 0x7f44381043f0 2024-05-27T07:43:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.882+0000 7f4435600700 1 -- 172.21.15.90:0/1309716641 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428012b50 con 0x7f44381043f0 2024-05-27T07:43:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.882+0000 7f443d600700 1 -- 172.21.15.90:0/1309716641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44381a7620 con 0x7f44381043f0 2024-05-27T07:43:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.884+0000 7f443d600700 1 -- 172.21.15.90:0/1309716641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44381058a0 con 0x7f44381043f0 2024-05-27T07:43:29.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.887+0000 7f4435600700 1 -- 172.21.15.90:0/1309716641 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4428022030 con 0x7f44381043f0 2024-05-27T07:43:29.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.888+0000 7f4435600700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f441c0b2910 0x7f441c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:29.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.888+0000 7f4435600700 1 -- 172.21.15.90:0/1309716641 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f442800f020 con 0x7f44381043f0 2024-05-27T07:43:29.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.888+0000 7f442f400700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f441c0b2910 0x7f441c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:29.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.889+0000 7f442f400700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f441c0b2910 0x7f441c0b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f443807a1c0 tx=0x7f4420005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:29.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:29.891+0000 7f4435600700 1 -- 172.21.15.90:0/1309716641 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4428113df0 con 0x7f44381043f0 2024-05-27T07:43:29.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:29 smithi136 ceph-mon[35324]: pgmap v456: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:43:30.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.131+0000 7f443d600700 1 -- 172.21.15.90:0/1309716641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f443819bee0 con 0x7f44381043f0 2024-05-27T07:43:30.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.136+0000 7f4435600700 1 -- 172.21.15.90:0/1309716641 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24725 (secure 0 0 0) 0x7f4428029020 con 0x7f44381043f0 2024-05-27T07:43:30.136 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:43:30.136 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":91,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:43:00.000806+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:43:30.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.139+0000 7f443d600700 1 -- 172.21.15.90:0/1309716641 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f441c0b2910 msgr2=0x7f441c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.139+0000 7f443d600700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f441c0b2910 0x7f441c0b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f443807a1c0 tx=0x7f4420005f50 comp rx=0 tx=0).stop 2024-05-27T07:43:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.139+0000 7f443d600700 1 -- 172.21.15.90:0/1309716641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44381043f0 msgr2=0x7f44381a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.139+0000 7f443d600700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44381043f0 0x7f44381a0790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4428007d90 tx=0x7f4428007e70 comp rx=0 tx=0).stop 2024-05-27T07:43:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.140+0000 7f443d600700 1 -- 172.21.15.90:0/1309716641 shutdown_connections 2024-05-27T07:43:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.140+0000 7f443d600700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f441c0b2910 0x7f441c0b4e20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.140+0000 7f443d600700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438109670 0x7f443819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.140+0000 7f443d600700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44381043f0 0x7f44381a0790 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.140+0000 7f443d600700 1 --2- 172.21.15.90:0/1309716641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4438108d20 0x7f44381a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.140+0000 7f443d600700 1 -- 172.21.15.90:0/1309716641 >> 172.21.15.90:0/1309716641 conn(0x7f4438079d00 msgr2=0x7f4438101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.140+0000 7f443d600700 1 -- 172.21.15.90:0/1309716641 shutdown_connections 2024-05-27T07:43:30.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:30.140+0000 7f443d600700 1 -- 172.21.15.90:0/1309716641 wait complete. 2024-05-27T07:43:30.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:30 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1309716641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:43:30.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:30 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1309716641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:43:30.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:30 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1309716641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:43:31.046 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-27T07:43:31.047 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-27T07:43:31.047 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-27T07:43:31.048 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-27T07:43:31.048 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-27T07:43:31.048 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-27T07:43:31.049 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-27T07:43:31.049 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-27T07:43:31.050 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-27T07:43:31.050 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-27T07:43:31.051 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-27T07:43:31.051 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-27T07:43:31.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:31 smithi090 ceph-mon[31429]: pgmap v457: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:31.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:31 smithi129 ceph-mon[35254]: pgmap v457: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:31.992 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:31.997 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:31 smithi136 ceph-mon[35324]: pgmap v457: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:32.428 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:32.429 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:32.471 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:32.478 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:33.032 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:33.043 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:33.043 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:33.043 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:33.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:33 smithi129 ceph-mon[35254]: pgmap v458: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:43:33.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:33 smithi090 ceph-mon[31429]: pgmap v458: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:43:34.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:33 smithi136 ceph-mon[35324]: pgmap v458: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:43:34.735 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:34.743 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:35.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:35 smithi090 ceph-mon[31429]: pgmap v459: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:43:35.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:35 smithi129 ceph-mon[35254]: pgmap v459: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:43:35.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:35 smithi136 ceph-mon[35324]: pgmap v459: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:43:36.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.364+0000 7fc99da00700 1 -- 172.21.15.90:0/2129275184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9981030d0 msgr2=0x7fc9981034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:36.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.364+0000 7fc99da00700 1 --2- 172.21.15.90:0/2129275184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9981030d0 0x7fc9981034e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc988009990 tx=0x7fc988009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:36.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.364+0000 7fc99da00700 1 -- 172.21.15.90:0/2129275184 shutdown_connections 2024-05-27T07:43:36.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.364+0000 7fc99da00700 1 --2- 172.21.15.90:0/2129275184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc998110c80 0x7fc9981130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:36.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.364+0000 7fc99da00700 1 --2- 172.21.15.90:0/2129275184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9981030d0 0x7fc9981034e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.364+0000 7fc99da00700 1 --2- 172.21.15.90:0/2129275184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998103a20 0x7fc998110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.364+0000 7fc99da00700 1 -- 172.21.15.90:0/2129275184 >> 172.21.15.90:0/2129275184 conn(0x7fc9980fede0 msgr2=0x7fc998101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.365+0000 7fc99da00700 1 -- 172.21.15.90:0/2129275184 shutdown_connections 2024-05-27T07:43:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.365+0000 7fc99da00700 1 -- 172.21.15.90:0/2129275184 wait complete. 2024-05-27T07:43:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.366+0000 7fc99da00700 1 Processor -- start 2024-05-27T07:43:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.366+0000 7fc99da00700 1 -- start start 2024-05-27T07:43:36.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.367+0000 7fc99da00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9981030d0 0x7fc99819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:36.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.367+0000 7fc99da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998103a20 0x7fc99819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:36.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.367+0000 7fc99da00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc998110c80 0x7fc9981a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:36.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.367+0000 7fc99da00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc998116510 con 0x7fc998103a20 2024-05-27T07:43:36.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.367+0000 7fc99da00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc998116390 con 0x7fc998110c80 2024-05-27T07:43:36.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.367+0000 7fc99da00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc998116690 con 0x7fc9981030d0 2024-05-27T07:43:36.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.367+0000 7fc996000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998103a20 0x7fc99819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:36.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.367+0000 7fc996000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998103a20 0x7fc99819d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34440/0 (socket says 172.21.15.90:34440) 2024-05-27T07:43:36.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.367+0000 7fc996000700 1 -- 172.21.15.90:0/3737904970 learned_addr learned my addr 172.21.15.90:0/3737904970 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:36.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.367+0000 7fc996a00700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9981030d0 0x7fc99819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.368+0000 7fc997400700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc998110c80 0x7fc9981a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.368+0000 7fc996000700 1 -- 172.21.15.90:0/3737904970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9981030d0 msgr2=0x7fc99819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.368+0000 7fc996000700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9981030d0 0x7fc99819cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.368+0000 7fc996000700 1 -- 172.21.15.90:0/3737904970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc998110c80 msgr2=0x7fc9981a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.368+0000 7fc996000700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc998110c80 0x7fc9981a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.368+0000 7fc996000700 1 -- 172.21.15.90:0/3737904970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9981a1e30 con 0x7fc998103a20 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.369+0000 7fc996000700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998103a20 0x7fc99819d500 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc98400c970 tx=0x7fc98400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.369+0000 7fc98f400700 1 -- 172.21.15.90:0/3737904970 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984022070 con 0x7fc998103a20 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.369+0000 7fc98f400700 1 -- 172.21.15.90:0/3737904970 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc984007b40 con 0x7fc998103a20 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.369+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9981a2120 con 0x7fc998103a20 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.369+0000 7fc98f400700 1 -- 172.21.15.90:0/3737904970 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98401daa0 con 0x7fc998103a20 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.369+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981052e0 con 0x7fc998103a20 2024-05-27T07:43:36.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.371+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc998063f00 con 0x7fc998103a20 2024-05-27T07:43:36.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.376+0000 7fc98f400700 1 -- 172.21.15.90:0/3737904970 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc9840076b0 con 0x7fc998103a20 2024-05-27T07:43:36.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.377+0000 7fc98f400700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc97c0c3920 0x7fc97c0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:36.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.377+0000 7fc98f400700 1 -- 172.21.15.90:0/3737904970 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc9840e2960 con 0x7fc998103a20 2024-05-27T07:43:36.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.378+0000 7fc996a00700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc97c0c3920 0x7fc97c0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:36.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.378+0000 7fc98f400700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] conn(0x7fc97c0cebd0 0x7fc97c0d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:36.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.378+0000 7fc98f400700 1 -- 172.21.15.90:0/3737904970 --> [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc97c0d1790 con 0x7fc97c0cebd0 2024-05-27T07:43:36.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.378+0000 7fc98f400700 1 -- 172.21.15.90:0/3737904970 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fc9840e2d10 con 0x7fc998103a20 2024-05-27T07:43:36.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.378+0000 7fc997400700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] conn(0x7fc97c0cebd0 0x7fc97c0d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:36.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.379+0000 7fc996a00700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc97c0c3920 0x7fc97c0c5e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc988003760 tx=0x7fc988005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:36.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.379+0000 7fc997400700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] conn(0x7fc97c0cebd0 0x7fc97c0d10c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:36.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.380+0000 7fc98f400700 1 -- 172.21.15.90:0/3737904970 <== osd.3 v2:172.21.15.90:6810/1546698486 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc97c0d1790 con 0x7fc97c0cebd0 2024-05-27T07:43:36.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.398+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 --> [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc998064380 con 0x7fc97c0cebd0 2024-05-27T07:43:36.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.399+0000 7fc98f400700 1 -- 172.21.15.90:0/3737904970 <== osd.3 v2:172.21.15.90:6810/1546698486 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc998064380 con 0x7fc97c0cebd0 2024-05-27T07:43:36.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.399+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] conn(0x7fc97c0cebd0 msgr2=0x7fc97c0d10c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:36.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.399+0000 7fc99da00700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] conn(0x7fc97c0cebd0 0x7fc97c0d10c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:36.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.399+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc97c0c3920 msgr2=0x7fc97c0c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:36.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.399+0000 7fc99da00700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc97c0c3920 0x7fc97c0c5e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc988003760 tx=0x7fc988005fb0 comp rx=0 tx=0).stop 2024-05-27T07:43:36.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.399+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998103a20 msgr2=0x7fc99819d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:36.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.399+0000 7fc99da00700 1 --2- 172.21.15.90:0/3737904970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998103a20 0x7fc99819d500 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc98400c970 tx=0x7fc98400cd30 comp rx=0 tx=0).stop 2024-05-27T07:43:36.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.399+0000 7fc997400700 1 -- 172.21.15.90:0/3737904970 reap_dead start 2024-05-27T07:43:36.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.400+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 shutdown_connections 2024-05-27T07:43:36.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.400+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 >> 172.21.15.90:0/3737904970 conn(0x7fc9980fede0 msgr2=0x7fc998101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:36.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.400+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 shutdown_connections 2024-05-27T07:43:36.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:36.400+0000 7fc99da00700 1 -- 172.21.15.90:0/3737904970 wait complete. 2024-05-27T07:43:37.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:37 smithi129 ceph-mon[35254]: pgmap v460: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:37.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:37 smithi090 ceph-mon[31429]: pgmap v460: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:37.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:37 smithi136 ceph-mon[35324]: pgmap v460: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:39.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:39 smithi090 ceph-mon[31429]: pgmap v461: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:43:39.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:39 smithi129 ceph-mon[35254]: pgmap v461: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:43:39.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:39 smithi136 ceph-mon[35324]: pgmap v461: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:43:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:115964117100 2024-05-27T07:43:40.883 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-27T07:43:41.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:41 smithi129 ceph-mon[35254]: pgmap v462: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:41.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:41 smithi090 ceph-mon[31429]: pgmap v462: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:41.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:41 smithi136 ceph-mon[35324]: pgmap v462: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:43.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:43 smithi090 ceph-mon[31429]: pgmap v463: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:43:43.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:43 smithi136 ceph-mon[35324]: pgmap v463: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:43:44.063 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:43:44.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:43 smithi129 ceph-mon[35254]: pgmap v463: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:43:45.757 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:45 smithi090 ceph-mon[31429]: pgmap v464: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:43:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.859+0000 7f81a1800700 1 -- 172.21.15.90:0/141656140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f819c108d00 msgr2=0x7f819c109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.859+0000 7f81a1800700 1 --2- 172.21.15.90:0/141656140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f819c108d00 0x7f819c109110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f81880099c0 tx=0x7f8188009cd0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.859+0000 7f81a1800700 1 -- 172.21.15.90:0/141656140 shutdown_connections 2024-05-27T07:43:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.859+0000 7f81a1800700 1 --2- 172.21.15.90:0/141656140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c109650 0x7f819c110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.859+0000 7f81a1800700 1 --2- 172.21.15.90:0/141656140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f819c1028e0 0x7f819c102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.859+0000 7f81a1800700 1 --2- 172.21.15.90:0/141656140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f819c108d00 0x7f819c109110 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.859+0000 7f81a1800700 1 -- 172.21.15.90:0/141656140 >> 172.21.15.90:0/141656140 conn(0x7f819c0783a0 msgr2=0x7f819c0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.859+0000 7f81a1800700 1 -- 172.21.15.90:0/141656140 shutdown_connections 2024-05-27T07:43:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.859+0000 7f81a1800700 1 -- 172.21.15.90:0/141656140 wait complete. 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.860+0000 7f81a1800700 1 Processor -- start 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.861+0000 7f81a1800700 1 -- start start 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.861+0000 7f81a1800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f819c1028e0 0x7f819c103830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.861+0000 7f81a1800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c108d00 0x7f819c103d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.861+0000 7f81a1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f819c109650 0x7f819c1042b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.861+0000 7f81a1800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f819c114160 con 0x7f819c109650 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.861+0000 7f81a1800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f819c113fe0 con 0x7f819c1028e0 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.861+0000 7f81a1800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f819c1142e0 con 0x7f819c108d00 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.862+0000 7f81a0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f819c109650 0x7f819c1042b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.862+0000 7f81a0e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f819c109650 0x7f819c1042b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52142/0 (socket says 172.21.15.90:52142) 2024-05-27T07:43:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.862+0000 7f81a0e00700 1 -- 172.21.15.90:0/1561500270 learned_addr learned my addr 172.21.15.90:0/1561500270 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.862+0000 7f819be00700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f819c1028e0 0x7f819c103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.862+0000 7f819b400700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c108d00 0x7f819c103d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.862+0000 7f81a0e00700 1 -- 172.21.15.90:0/1561500270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c108d00 msgr2=0x7f819c103d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.862+0000 7f81a0e00700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f819c108d00 0x7f819c103d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.862+0000 7f81a0e00700 1 -- 172.21.15.90:0/1561500270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f819c1028e0 msgr2=0x7f819c103830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.862+0000 7f81a0e00700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f819c1028e0 0x7f819c103830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.863+0000 7f81a0e00700 1 -- 172.21.15.90:0/1561500270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c1a8f10 con 0x7f819c109650 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.863+0000 7f81a0e00700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f819c109650 0x7f819c1042b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8194007ee0 tx=0x7f819400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.863+0000 7f8198c00700 1 -- 172.21.15.90:0/1561500270 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194004bb0 con 0x7f819c109650 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.863+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c1a9200 con 0x7f819c109650 2024-05-27T07:43:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.863+0000 7f8198c00700 1 -- 172.21.15.90:0/1561500270 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8194004d50 con 0x7f819c109650 2024-05-27T07:43:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.863+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c1a96c0 con 0x7f819c109650 2024-05-27T07:43:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.863+0000 7f8198c00700 1 -- 172.21.15.90:0/1561500270 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194005850 con 0x7f819c109650 2024-05-27T07:43:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.864+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f819c063eb0 con 0x7f819c109650 2024-05-27T07:43:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.869+0000 7f8198c00700 1 -- 172.21.15.90:0/1561500270 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f819401a020 con 0x7f819c109650 2024-05-27T07:43:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.870+0000 7f8198c00700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81800b2910 0x7f81800b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.870+0000 7f8198c00700 1 -- 172.21.15.90:0/1561500270 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f81940e27f0 con 0x7f819c109650 2024-05-27T07:43:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.871+0000 7f819be00700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81800b2910 0x7f81800b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.871+0000 7f8198c00700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] conn(0x7f81800bdbc0 0x7f81800c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.871+0000 7f8198c00700 1 -- 172.21.15.90:0/1561500270 --> [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81800c0780 con 0x7f81800bdbc0 2024-05-27T07:43:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.871+0000 7f8198c00700 1 -- 172.21.15.90:0/1561500270 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f81940e2ba0 con 0x7f819c109650 2024-05-27T07:43:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.871+0000 7f819b400700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] conn(0x7f81800bdbc0 0x7f81800c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.871+0000 7f819be00700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81800b2910 0x7f81800b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f818c005950 tx=0x7f818c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.872+0000 7f819b400700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] conn(0x7f81800bdbc0 0x7f81800c00b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.873+0000 7f8198c00700 1 -- 172.21.15.90:0/1561500270 <== osd.9 v2:172.21.15.90:6826/4197259682 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81800c0780 con 0x7f81800bdbc0 2024-05-27T07:43:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.891+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 --> [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f819c0642b0 con 0x7f81800bdbc0 2024-05-27T07:43:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.891+0000 7f8198c00700 1 -- 172.21.15.90:0/1561500270 <== osd.9 v2:172.21.15.90:6826/4197259682 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f819c0642b0 con 0x7f81800bdbc0 2024-05-27T07:43:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] conn(0x7f81800bdbc0 msgr2=0x7f81800c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a1800700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] conn(0x7f81800bdbc0 0x7f81800c00b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81800b2910 msgr2=0x7f81800b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a1800700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f81800b2910 0x7f81800b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f818c005950 tx=0x7f818c00b410 comp rx=0 tx=0).stop 2024-05-27T07:43:45.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f819c109650 msgr2=0x7f819c1042b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:45.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a1800700 1 --2- 172.21.15.90:0/1561500270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f819c109650 0x7f819c1042b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8194007ee0 tx=0x7f819400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:43:45.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a0e00700 1 -- 172.21.15.90:0/1561500270 reap_dead start 2024-05-27T07:43:45.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 shutdown_connections 2024-05-27T07:43:45.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 >> 172.21.15.90:0/1561500270 conn(0x7f819c0783a0 msgr2=0x7f819c0fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:45.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 shutdown_connections 2024-05-27T07:43:45.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:44.892+0000 7f81a1800700 1 -- 172.21.15.90:0/1561500270 wait complete. 2024-05-27T07:43:45.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:45 smithi136 ceph-mon[35324]: pgmap v464: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:43:46.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:45 smithi129 ceph-mon[35254]: pgmap v464: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:43:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:47 smithi090 ceph-mon[31429]: pgmap v465: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-27T07:43:47.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.874+0000 7f6102200700 1 -- 172.21.15.90:0/621381545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60fc110c10 msgr2=0x7f60fc113030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:47.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.874+0000 7f6102200700 1 --2- 172.21.15.90:0/621381545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60fc110c10 0x7f60fc113030 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f60f400b0a0 tx=0x7f60f400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:43:47.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.875+0000 7f6102200700 1 -- 172.21.15.90:0/621381545 shutdown_connections 2024-05-27T07:43:47.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.875+0000 7f6102200700 1 --2- 172.21.15.90:0/621381545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60fc110c10 0x7f60fc113030 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:47.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.875+0000 7f6102200700 1 --2- 172.21.15.90:0/621381545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60fc075af0 0x7f60fc075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:47.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.875+0000 7f6102200700 1 --2- 172.21.15.90:0/621381545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc076440 0x7f60fc1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:47.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.875+0000 7f6102200700 1 -- 172.21.15.90:0/621381545 >> 172.21.15.90:0/621381545 conn(0x7f60fc07af60 msgr2=0x7f60fc07b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:47.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.875+0000 7f6102200700 1 -- 172.21.15.90:0/621381545 shutdown_connections 2024-05-27T07:43:47.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.875+0000 7f6102200700 1 -- 172.21.15.90:0/621381545 wait complete. 2024-05-27T07:43:47.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.876+0000 7f6102200700 1 Processor -- start 2024-05-27T07:43:47.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.876+0000 7f6102200700 1 -- start start 2024-05-27T07:43:47.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f6102200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc075af0 0x7f60fc19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:47.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f6102200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60fc076440 0x7f60fc19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:47.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f6102200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60fc110c10 0x7f60fc1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:47.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f6102200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc116530 con 0x7f60fc110c10 2024-05-27T07:43:47.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f6102200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1163b0 con 0x7f60fc075af0 2024-05-27T07:43:47.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f6102200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc1166b0 con 0x7f60fc076440 2024-05-27T07:43:47.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f60fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60fc110c10 0x7f60fc1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:47.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f60fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60fc110c10 0x7f60fc1a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52162/0 (socket says 172.21.15.90:52162) 2024-05-27T07:43:47.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f60fbe00700 1 -- 172.21.15.90:0/3683631624 learned_addr learned my addr 172.21.15.90:0/3683631624 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:47.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f60fb400700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc075af0 0x7f60fc19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:47.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.877+0000 7f60faa00700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60fc076440 0x7f60fc19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:47.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.878+0000 7f60fbe00700 1 -- 172.21.15.90:0/3683631624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60fc076440 msgr2=0x7f60fc19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:47.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.878+0000 7f60fbe00700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60fc076440 0x7f60fc19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:47.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.878+0000 7f60fbe00700 1 -- 172.21.15.90:0/3683631624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc075af0 msgr2=0x7f60fc19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:47.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.878+0000 7f60fbe00700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc075af0 0x7f60fc19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:47.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.878+0000 7f60fbe00700 1 -- 172.21.15.90:0/3683631624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc1a1d90 con 0x7f60fc110c10 2024-05-27T07:43:47.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.878+0000 7f60fb400700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f60fc075af0 0x7f60fc19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:47.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.878+0000 7f60fbe00700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60fc110c10 0x7f60fc1a16f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f60f400b780 tx=0x7f60f4004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:47.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.879+0000 7f60f3e00700 1 -- 172.21.15.90:0/3683631624 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4021070 con 0x7f60fc110c10 2024-05-27T07:43:47.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.879+0000 7f60f3e00700 1 -- 172.21.15.90:0/3683631624 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f60f4007b80 con 0x7f60fc110c10 2024-05-27T07:43:47.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.879+0000 7f60f3e00700 1 -- 172.21.15.90:0/3683631624 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f40052a0 con 0x7f60fc110c10 2024-05-27T07:43:47.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.879+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc1a2020 con 0x7f60fc110c10 2024-05-27T07:43:47.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.879+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1a99a0 con 0x7f60fc110c10 2024-05-27T07:43:47.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.880+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60fc063f40 con 0x7f60fc110c10 2024-05-27T07:43:47.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.883+0000 7f60f3e00700 1 -- 172.21.15.90:0/3683631624 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f60f4024070 con 0x7f60fc110c10 2024-05-27T07:43:47.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.884+0000 7f60f3e00700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f60e00b2990 0x7f60e00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:47.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.884+0000 7f60f3e00700 1 -- 172.21.15.90:0/3683631624 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f60f400f020 con 0x7f60fc110c10 2024-05-27T07:43:47.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.884+0000 7f60fb400700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f60e00b2990 0x7f60e00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:47.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.885+0000 7f60f3e00700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] conn(0x7f60e00bdc40 0x7f60e00c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:47.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.885+0000 7f60f3e00700 1 -- 172.21.15.90:0/3683631624 --> [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60f4013e70 con 0x7f60e00bdc40 2024-05-27T07:43:47.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.885+0000 7f60f3e00700 1 -- 172.21.15.90:0/3683631624 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f60f40e82c0 con 0x7f60fc110c10 2024-05-27T07:43:47.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.885+0000 7f60fb400700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f60e00b2990 0x7f60e00b4ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009cb0 tx=0x7f60ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:47.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.886+0000 7f60faa00700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] conn(0x7f60e00bdc40 0x7f60e00c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:47.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.887+0000 7f60faa00700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] conn(0x7f60e00bdc40 0x7f60e00c0130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:47.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.890+0000 7f60f3e00700 1 -- 172.21.15.90:0/3683631624 <== osd.7 v2:172.21.15.129:6816/1361444791 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60f4013e70 con 0x7f60e00bdc40 2024-05-27T07:43:47.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.907+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 --> [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60fc064340 con 0x7f60e00bdc40 2024-05-27T07:43:47.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.908+0000 7f60f3e00700 1 -- 172.21.15.90:0/3683631624 <== osd.7 v2:172.21.15.129:6816/1361444791 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f60fc064340 con 0x7f60e00bdc40 2024-05-27T07:43:47.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.909+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 >> [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] conn(0x7f60e00bdc40 msgr2=0x7f60e00c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.909+0000 7f6102200700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] conn(0x7f60e00bdc40 0x7f60e00c0130 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.909+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f60e00b2990 msgr2=0x7f60e00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.909+0000 7f6102200700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f60e00b2990 0x7f60e00b4ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009cb0 tx=0x7f60ec009260 comp rx=0 tx=0).stop 2024-05-27T07:43:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.909+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60fc110c10 msgr2=0x7f60fc1a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.909+0000 7f6102200700 1 --2- 172.21.15.90:0/3683631624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60fc110c10 0x7f60fc1a16f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f60f400b780 tx=0x7f60f4004db0 comp rx=0 tx=0).stop 2024-05-27T07:43:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.909+0000 7f60fbe00700 1 -- 172.21.15.90:0/3683631624 reap_dead start 2024-05-27T07:43:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.909+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 shutdown_connections 2024-05-27T07:43:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.909+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 >> 172.21.15.90:0/3683631624 conn(0x7f60fc07af60 msgr2=0x7f60fc101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:47.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.910+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 shutdown_connections 2024-05-27T07:43:47.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:47.910+0000 7f6102200700 1 -- 172.21.15.90:0/3683631624 wait complete. 2024-05-27T07:43:47.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:47 smithi136 ceph-mon[35324]: pgmap v465: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-27T07:43:48.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:47 smithi129 ceph-mon[35254]: pgmap v465: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-27T07:43:48.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.602+0000 7f0361000700 1 -- 172.21.15.90:0/1504060264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c1045e0 msgr2=0x7f035c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.602+0000 7f0361000700 1 --2- 172.21.15.90:0/1504060264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c1045e0 0x7f035c1049f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f034c0099c0 tx=0x7f034c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.602+0000 7f0361000700 1 -- 172.21.15.90:0/1504060264 shutdown_connections 2024-05-27T07:43:48.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.602+0000 7f0361000700 1 --2- 172.21.15.90:0/1504060264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f035c109860 0x7f035c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.602+0000 7f0361000700 1 --2- 172.21.15.90:0/1504060264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c1045e0 0x7f035c1049f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.602+0000 7f0361000700 1 --2- 172.21.15.90:0/1504060264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f035c108f10 0x7f035c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.602+0000 7f0361000700 1 -- 172.21.15.90:0/1504060264 >> 172.21.15.90:0/1504060264 conn(0x7f035c0fffd0 msgr2=0x7f035c1023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.602+0000 7f0361000700 1 -- 172.21.15.90:0/1504060264 shutdown_connections 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.602+0000 7f0361000700 1 -- 172.21.15.90:0/1504060264 wait complete. 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.603+0000 7f0361000700 1 Processor -- start 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.603+0000 7f0361000700 1 -- start start 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f0361000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f035c1045e0 0x7f035c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f0361000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f035c108f10 0x7f035c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f0361000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c109860 0x7f035c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f0361000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f035c114340 con 0x7f035c109860 2024-05-27T07:43:48.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f0361000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f035c1141c0 con 0x7f035c108f10 2024-05-27T07:43:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f0361000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f035c1144c0 con 0x7f035c1045e0 2024-05-27T07:43:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f035a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c109860 0x7f035c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f035a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c109860 0x7f035c19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52154/0 (socket says 172.21.15.90:52154) 2024-05-27T07:43:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f035a800700 1 -- 172.21.15.90:0/99615128 learned_addr learned my addr 172.21.15.90:0/99615128 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f0359e00700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f035c1045e0 0x7f035c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.604+0000 7f0359400700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f035c108f10 0x7f035c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f035a800700 1 -- 172.21.15.90:0/99615128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f035c1045e0 msgr2=0x7f035c19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f035a800700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f035c1045e0 0x7f035c19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f035a800700 1 -- 172.21.15.90:0/99615128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f035c108f10 msgr2=0x7f035c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f035a800700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f035c108f10 0x7f035c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f035a800700 1 -- 172.21.15.90:0/99615128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c19fc10 con 0x7f035c109860 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f035a800700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c109860 0x7f035c19f570 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0354007ee0 tx=0x7f035400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f0352a00700 1 -- 172.21.15.90:0/99615128 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035400dbd0 con 0x7f035c109860 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f0352a00700 1 -- 172.21.15.90:0/99615128 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f035400dd70 con 0x7f035c109860 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f0352a00700 1 -- 172.21.15.90:0/99615128 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03540058d0 con 0x7f035c109860 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035c19ff00 con 0x7f035c109860 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.605+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035c1a7820 con 0x7f035c109860 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.607+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f035c063f40 con 0x7f035c109860 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.609+0000 7f0352a00700 1 -- 172.21.15.90:0/99615128 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f035401a020 con 0x7f035c109860 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.611+0000 7f0352a00700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f03400b2910 0x7f03400b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.611+0000 7f0352a00700 1 -- 172.21.15.90:0/99615128 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f03540e2660 con 0x7f035c109860 2024-05-27T07:43:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.611+0000 7f0352a00700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] conn(0x7f03400bdbc0 0x7f03400c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.611+0000 7f0352a00700 1 -- 172.21.15.90:0/99615128 --> [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03400c0780 con 0x7f03400bdbc0 2024-05-27T07:43:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.611+0000 7f0352a00700 1 -- 172.21.15.90:0/99615128 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f03540e2a10 con 0x7f035c109860 2024-05-27T07:43:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.612+0000 7f0359e00700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f03400b2910 0x7f03400b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.612+0000 7f0359400700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] conn(0x7f03400bdbc0 0x7f03400c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.613+0000 7f0359e00700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f03400b2910 0x7f03400b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f034c000c00 tx=0x7f034c005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.613+0000 7f0359400700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] conn(0x7f03400bdbc0 0x7f03400c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.616+0000 7f0352a00700 1 -- 172.21.15.90:0/99615128 <== osd.1 v2:172.21.15.129:6800/3393795676 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03400c0780 con 0x7f03400bdbc0 2024-05-27T07:43:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.634+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 --> [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f035c0643c0 con 0x7f03400bdbc0 2024-05-27T07:43:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.635+0000 7f0352a00700 1 -- 172.21.15.90:0/99615128 <== osd.1 v2:172.21.15.129:6800/3393795676 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f035c0643c0 con 0x7f03400bdbc0 2024-05-27T07:43:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 >> [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] conn(0x7f03400bdbc0 msgr2=0x7f03400c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f0361000700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] conn(0x7f03400bdbc0 0x7f03400c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f03400b2910 msgr2=0x7f03400b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f0361000700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f03400b2910 0x7f03400b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f034c000c00 tx=0x7f034c005940 comp rx=0 tx=0).stop 2024-05-27T07:43:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c109860 msgr2=0x7f035c19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f0361000700 1 --2- 172.21.15.90:0/99615128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f035c109860 0x7f035c19f570 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0354007ee0 tx=0x7f035400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:43:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f035a800700 1 -- 172.21.15.90:0/99615128 reap_dead start 2024-05-27T07:43:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 shutdown_connections 2024-05-27T07:43:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 >> 172.21.15.90:0/99615128 conn(0x7f035c0fffd0 msgr2=0x7f035c107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 shutdown_connections 2024-05-27T07:43:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:46.636+0000 7f0361000700 1 -- 172.21.15.90:0/99615128 wait complete. 2024-05-27T07:43:49.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.872+0000 7f172b200700 1 -- 172.21.15.90:0/1296233005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f172c10a7b0 msgr2=0x7f172c10abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:49.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.872+0000 7f172b200700 1 --2- 172.21.15.90:0/1296233005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f172c10a7b0 0x7f172c10abc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1718009990 tx=0x7f1718009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:49.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.872+0000 7f172b200700 1 -- 172.21.15.90:0/1296233005 shutdown_connections 2024-05-27T07:43:49.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.872+0000 7f172b200700 1 --2- 172.21.15.90:0/1296233005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f172c10b100 0x7f172c117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:49.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.872+0000 7f172b200700 1 --2- 172.21.15.90:0/1296233005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c10f6b0 0x7f172c10fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:49.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.872+0000 7f172b200700 1 --2- 172.21.15.90:0/1296233005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f172c10a7b0 0x7f172c10abc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:49.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.872+0000 7f172b200700 1 -- 172.21.15.90:0/1296233005 >> 172.21.15.90:0/1296233005 conn(0x7f172c076af0 msgr2=0x7f172c076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:49.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:49 smithi090 ceph-mon[31429]: pgmap v466: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:43:49.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.873+0000 7f172b200700 1 -- 172.21.15.90:0/1296233005 shutdown_connections 2024-05-27T07:43:49.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.873+0000 7f172b200700 1 -- 172.21.15.90:0/1296233005 wait complete. 2024-05-27T07:43:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.874+0000 7f172b200700 1 Processor -- start 2024-05-27T07:43:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.874+0000 7f172b200700 1 -- start start 2024-05-27T07:43:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.874+0000 7f172b200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c10a7b0 0x7f172c1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.874+0000 7f172b200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f172c10b100 0x7f172c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.874+0000 7f172b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f172c10f6b0 0x7f172c1a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.874+0000 7f172b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f172c11a740 con 0x7f172c10f6b0 2024-05-27T07:43:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.874+0000 7f172b200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f172c11a5c0 con 0x7f172c10b100 2024-05-27T07:43:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.874+0000 7f172b200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f172c11a8c0 con 0x7f172c10a7b0 2024-05-27T07:43:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.875+0000 7f172a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f172c10f6b0 0x7f172c1a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.875+0000 7f172a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f172c10f6b0 0x7f172c1a59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54394/0 (socket says 172.21.15.90:54394) 2024-05-27T07:43:49.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.875+0000 7f172a800700 1 -- 172.21.15.90:0/3971189434 learned_addr learned my addr 172.21.15.90:0/3971189434 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:49.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.875+0000 7f1729e00700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c10a7b0 0x7f172c1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:49.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.875+0000 7f1729400700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f172c10b100 0x7f172c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.875+0000 7f172a800700 1 -- 172.21.15.90:0/3971189434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c10a7b0 msgr2=0x7f172c1a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.875+0000 7f172a800700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f172c10a7b0 0x7f172c1a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.876+0000 7f172a800700 1 -- 172.21.15.90:0/3971189434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f172c10b100 msgr2=0x7f172c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.876+0000 7f172a800700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f172c10b100 0x7f172c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.876+0000 7f172a800700 1 -- 172.21.15.90:0/3971189434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c1a6060 con 0x7f172c10f6b0 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.876+0000 7f172a800700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f172c10f6b0 0x7f172c1a59c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1724007ee0 tx=0x7f172400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.876+0000 7f1722a00700 1 -- 172.21.15.90:0/3971189434 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724004bb0 con 0x7f172c10f6b0 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.876+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172c1a6350 con 0x7f172c10f6b0 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.876+0000 7f1722a00700 1 -- 172.21.15.90:0/3971189434 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1724004d50 con 0x7f172c10f6b0 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.876+0000 7f1722a00700 1 -- 172.21.15.90:0/3971189434 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17240058b0 con 0x7f172c10f6b0 2024-05-27T07:43:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.876+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f172c1adc70 con 0x7f172c10f6b0 2024-05-27T07:43:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.877+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f172c063ec0 con 0x7f172c10f6b0 2024-05-27T07:43:49.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.880+0000 7f1722a00700 1 -- 172.21.15.90:0/3971189434 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f172401a020 con 0x7f172c10f6b0 2024-05-27T07:43:49.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.882+0000 7f1722a00700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f17100b2910 0x7f17100b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:49.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.882+0000 7f1722a00700 1 -- 172.21.15.90:0/3971189434 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f17240e2500 con 0x7f172c10f6b0 2024-05-27T07:43:49.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.883+0000 7f1729e00700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f17100b2910 0x7f17100b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:49.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.883+0000 7f1722a00700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] conn(0x7f17100bdbc0 0x7f17100c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:49.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.883+0000 7f1722a00700 1 -- 172.21.15.90:0/3971189434 --> [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17240020b0 con 0x7f17100bdbc0 2024-05-27T07:43:49.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.883+0000 7f1722a00700 1 -- 172.21.15.90:0/3971189434 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f17240e28b0 con 0x7f172c10f6b0 2024-05-27T07:43:49.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.884+0000 7f1729400700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] conn(0x7f17100bdbc0 0x7f17100c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:49.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.884+0000 7f1729e00700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f17100b2910 0x7f17100b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f171c005950 tx=0x7f171c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:49.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.885+0000 7f1729400700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] conn(0x7f17100bdbc0 0x7f17100c00b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:49.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.887+0000 7f1722a00700 1 -- 172.21.15.90:0/3971189434 <== osd.2 v2:172.21.15.136:6800/1421934201 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17240020b0 con 0x7f17100bdbc0 2024-05-27T07:43:49.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.906+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 --> [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f172c064370 con 0x7f17100bdbc0 2024-05-27T07:43:49.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.907+0000 7f1722a00700 1 -- 172.21.15.90:0/3971189434 <== osd.2 v2:172.21.15.136:6800/1421934201 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f172c064370 con 0x7f17100bdbc0 2024-05-27T07:43:49.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.907+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 >> [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] conn(0x7f17100bdbc0 msgr2=0x7f17100c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:49.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.907+0000 7f172b200700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] conn(0x7f17100bdbc0 0x7f17100c00b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:49.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.907+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f17100b2910 msgr2=0x7f17100b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:49.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.907+0000 7f172b200700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f17100b2910 0x7f17100b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f171c005950 tx=0x7f171c00b410 comp rx=0 tx=0).stop 2024-05-27T07:43:49.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.907+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f172c10f6b0 msgr2=0x7f172c1a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:49.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.907+0000 7f172b200700 1 --2- 172.21.15.90:0/3971189434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f172c10f6b0 0x7f172c1a59c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1724007ee0 tx=0x7f172400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:43:49.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.907+0000 7f172a800700 1 -- 172.21.15.90:0/3971189434 reap_dead start 2024-05-27T07:43:49.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.908+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 shutdown_connections 2024-05-27T07:43:49.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.908+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 >> 172.21.15.90:0/3971189434 conn(0x7f172c076af0 msgr2=0x7f172c108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:49.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.908+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 shutdown_connections 2024-05-27T07:43:49.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.908+0000 7f172b200700 1 -- 172.21.15.90:0/3971189434 wait complete. 2024-05-27T07:43:49.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:49 smithi136 ceph-mon[35324]: pgmap v466: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:43:50.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:49 smithi129 ceph-mon[35254]: pgmap v466: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:43:50.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.225+0000 7f5455a00700 1 -- 172.21.15.90:0/1981612037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5450105450 msgr2=0x7f5450105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:50.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.225+0000 7f5455a00700 1 --2- 172.21.15.90:0/1981612037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5450105450 0x7f5450105860 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5440009990 tx=0x7f5440009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:50.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.225+0000 7f5455a00700 1 -- 172.21.15.90:0/1981612037 shutdown_connections 2024-05-27T07:43:50.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.225+0000 7f5455a00700 1 --2- 172.21.15.90:0/1981612037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f545010e8c0 0x7f5450110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:50.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.225+0000 7f5455a00700 1 --2- 172.21.15.90:0/1981612037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5450105450 0x7f5450105860 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.225+0000 7f5455a00700 1 --2- 172.21.15.90:0/1981612037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5450105da0 0x7f545010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.225+0000 7f5455a00700 1 -- 172.21.15.90:0/1981612037 >> 172.21.15.90:0/1981612037 conn(0x7f54500fede0 msgr2=0x7f5450101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.225+0000 7f5455a00700 1 -- 172.21.15.90:0/1981612037 shutdown_connections 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.225+0000 7f5455a00700 1 -- 172.21.15.90:0/1981612037 wait complete. 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.226+0000 7f5455a00700 1 Processor -- start 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.226+0000 7f5455a00700 1 -- start start 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.227+0000 7f5455a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5450105450 0x7f545019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.227+0000 7f5455a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5450105da0 0x7f545019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.227+0000 7f5455a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f545010e8c0 0x7f54501a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.227+0000 7f5455a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5450114320 con 0x7f5450105450 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.227+0000 7f5455a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54501141a0 con 0x7f545010e8c0 2024-05-27T07:43:50.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.227+0000 7f5455a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f54501144a0 con 0x7f5450105da0 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.227+0000 7f544ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5450105450 0x7f545019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.227+0000 7f544ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5450105450 0x7f545019cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54422/0 (socket says 172.21.15.90:54422) 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.227+0000 7f544ea00700 1 -- 172.21.15.90:0/224340316 learned_addr learned my addr 172.21.15.90:0/224340316 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.227+0000 7f544e000700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5450105da0 0x7f545019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f544ea00700 1 -- 172.21.15.90:0/224340316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5450105da0 msgr2=0x7f545019d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f544ea00700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5450105da0 0x7f545019d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f544ea00700 1 -- 172.21.15.90:0/224340316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f545010e8c0 msgr2=0x7f54501a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f544ea00700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f545010e8c0 0x7f54501a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f544ea00700 1 -- 172.21.15.90:0/224340316 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54501a1e80 con 0x7f5450105450 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f544e000700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5450105da0 0x7f545019d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f544ea00700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5450105450 0x7f545019cfb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5440000c00 tx=0x7f5440009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:50.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f5447400700 1 -- 172.21.15.90:0/224340316 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440027070 con 0x7f5450105450 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f5447400700 1 -- 172.21.15.90:0/224340316 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f544000fc70 con 0x7f5450105450 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54501a2110 con 0x7f5450105450 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f5447400700 1 -- 172.21.15.90:0/224340316 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440022d30 con 0x7f5450105450 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.228+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54501a25f0 con 0x7f5450105450 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.229+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5450103af0 con 0x7f5450105450 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.232+0000 7f5447400700 1 -- 172.21.15.90:0/224340316 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f544002a070 con 0x7f5450105450 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.233+0000 7f5447400700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f54340b2910 0x7f54340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.234+0000 7f5447400700 1 -- 172.21.15.90:0/224340316 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f54400e3f20 con 0x7f5450105450 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.234+0000 7f5447400700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] conn(0x7f54340bdbc0 0x7f54340c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.234+0000 7f5447400700 1 -- 172.21.15.90:0/224340316 --> [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f544000eea0 con 0x7f54340bdbc0 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.234+0000 7f5447400700 1 -- 172.21.15.90:0/224340316 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f54400e4370 con 0x7f5450105450 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.234+0000 7f544e000700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f54340b2910 0x7f54340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.234+0000 7f544f400700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] conn(0x7f54340bdbc0 0x7f54340c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:50.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.235+0000 7f544e000700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f54340b2910 0x7f54340b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f543c005fd0 tx=0x7f543c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:50.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.235+0000 7f544f400700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] conn(0x7f54340bdbc0 0x7f54340c00b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:50.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.236+0000 7f5447400700 1 -- 172.21.15.90:0/224340316 <== osd.0 v2:172.21.15.90:6802/70197447 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f544000eea0 con 0x7f54340bdbc0 2024-05-27T07:43:50.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.254+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 --> [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f54500762c0 con 0x7f54340bdbc0 2024-05-27T07:43:50.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.255+0000 7f5447400700 1 -- 172.21.15.90:0/224340316 <== osd.0 v2:172.21.15.90:6802/70197447 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f54500762c0 con 0x7f54340bdbc0 2024-05-27T07:43:50.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.255+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] conn(0x7f54340bdbc0 msgr2=0x7f54340c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:50.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.256+0000 7f5455a00700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] conn(0x7f54340bdbc0 0x7f54340c00b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:50.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.256+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f54340b2910 msgr2=0x7f54340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:50.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.256+0000 7f5455a00700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f54340b2910 0x7f54340b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f543c005fd0 tx=0x7f543c005ee0 comp rx=0 tx=0).stop 2024-05-27T07:43:50.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.256+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5450105450 msgr2=0x7f545019cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:50.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.256+0000 7f5455a00700 1 --2- 172.21.15.90:0/224340316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5450105450 0x7f545019cfb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5440000c00 tx=0x7f5440009f90 comp rx=0 tx=0).stop 2024-05-27T07:43:50.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.256+0000 7f544f400700 1 -- 172.21.15.90:0/224340316 reap_dead start 2024-05-27T07:43:50.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.256+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 shutdown_connections 2024-05-27T07:43:50.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.256+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 >> 172.21.15.90:0/224340316 conn(0x7f54500fede0 msgr2=0x7f5450103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:50.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.256+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 shutdown_connections 2024-05-27T07:43:50.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:50.256+0000 7f5455a00700 1 -- 172.21.15.90:0/224340316 wait complete. 2024-05-27T07:43:51.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:51 smithi090 ceph-mon[31429]: pgmap v467: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:51.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:51 smithi136 ceph-mon[35324]: pgmap v467: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:52.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:51 smithi129 ceph-mon[35254]: pgmap v467: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:43:52.327 INFO:teuthology.orchestra.run.smithi090.stdout:77309411446 2024-05-27T07:43:52.327 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-27T07:43:52.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.537+0000 7f0e8fe00700 1 -- 172.21.15.90:0/160165167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e90105300 msgr2=0x7f0e90105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:52.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.537+0000 7f0e8fe00700 1 --2- 172.21.15.90:0/160165167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e90105300 0x7f0e90105710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009990 tx=0x7f0e80009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.537+0000 7f0e8fe00700 1 -- 172.21.15.90:0/160165167 shutdown_connections 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.537+0000 7f0e8fe00700 1 --2- 172.21.15.90:0/160165167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e9010e730 0x7f0e90110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.537+0000 7f0e8fe00700 1 --2- 172.21.15.90:0/160165167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e90105300 0x7f0e90105710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.537+0000 7f0e8fe00700 1 --2- 172.21.15.90:0/160165167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e90105c50 0x7f0e9010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.537+0000 7f0e8fe00700 1 -- 172.21.15.90:0/160165167 >> 172.21.15.90:0/160165167 conn(0x7f0e90078ab0 msgr2=0x7f0e90078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.537+0000 7f0e8fe00700 1 -- 172.21.15.90:0/160165167 shutdown_connections 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.537+0000 7f0e8fe00700 1 -- 172.21.15.90:0/160165167 wait complete. 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.538+0000 7f0e8fe00700 1 Processor -- start 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.538+0000 7f0e8fe00700 1 -- start start 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e90105300 0x7f0e9019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e90105c50 0x7f0e9019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e9010e730 0x7f0e901a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8fe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90114140 con 0x7f0e90105300 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8fe00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90113fc0 con 0x7f0e90105c50 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8fe00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901142c0 con 0x7f0e9010e730 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e90105300 0x7f0e9019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e90105300 0x7f0e9019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54376/0 (socket says 172.21.15.90:54376) 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8ea00700 1 -- 172.21.15.90:0/2275050803 learned_addr learned my addr 172.21.15.90:0/2275050803 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8e000700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e90105c50 0x7f0e9019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.539+0000 7f0e8f400700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e9010e730 0x7f0e901a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e8ea00700 1 -- 172.21.15.90:0/2275050803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e9010e730 msgr2=0x7f0e901a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e8ea00700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e9010e730 0x7f0e901a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e8ea00700 1 -- 172.21.15.90:0/2275050803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e90105c50 msgr2=0x7f0e9019d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e8ea00700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0e90105c50 0x7f0e9019d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e8ea00700 1 -- 172.21.15.90:0/2275050803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e901a1c40 con 0x7f0e90105300 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e8ea00700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e90105300 0x7f0e9019cd70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0e80005910 tx=0x7f0e800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e87400700 1 -- 172.21.15.90:0/2275050803 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e80027070 con 0x7f0e90105300 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e87400700 1 -- 172.21.15.90:0/2275050803 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e8000fc00 con 0x7f0e90105300 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e901a1ed0 con 0x7f0e90105300 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e87400700 1 -- 172.21.15.90:0/2275050803 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e80022d80 con 0x7f0e90105300 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.540+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e901a23b0 con 0x7f0e90105300 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.541+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e90103a40 con 0x7f0e90105300 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.544+0000 7f0e87400700 1 -- 172.21.15.90:0/2275050803 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0e8002a070 con 0x7f0e90105300 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.546+0000 7f0e87400700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e740b2910 0x7f0e740b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.546+0000 7f0e87400700 1 -- 172.21.15.90:0/2275050803 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f0e800e3d60 con 0x7f0e90105300 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.546+0000 7f0e8e000700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e740b2910 0x7f0e740b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.546+0000 7f0e87400700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] conn(0x7f0e740bdbc0 0x7f0e740c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:52.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.546+0000 7f0e87400700 1 -- 172.21.15.90:0/2275050803 --> [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e740c0780 con 0x7f0e740bdbc0 2024-05-27T07:43:52.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.546+0000 7f0e87400700 1 -- 172.21.15.90:0/2275050803 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f0e800e4110 con 0x7f0e90105300 2024-05-27T07:43:52.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.547+0000 7f0e8e000700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e740b2910 0x7f0e740b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005950 tx=0x7f0e7c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:52.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.547+0000 7f0e8f400700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] conn(0x7f0e740bdbc0 0x7f0e740c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:52.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.548+0000 7f0e8f400700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] conn(0x7f0e740bdbc0 0x7f0e740c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:52.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.551+0000 7f0e87400700 1 -- 172.21.15.90:0/2275050803 <== osd.11 v2:172.21.15.136:6824/2683318661 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e740c0780 con 0x7f0e740bdbc0 2024-05-27T07:43:52.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.569+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 --> [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0e9006be50 con 0x7f0e740bdbc0 2024-05-27T07:43:52.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.570+0000 7f0e87400700 1 -- 172.21.15.90:0/2275050803 <== osd.11 v2:172.21.15.136:6824/2683318661 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0e9006be50 con 0x7f0e740bdbc0 2024-05-27T07:43:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.570+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 >> [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] conn(0x7f0e740bdbc0 msgr2=0x7f0e740c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.570+0000 7f0e8fe00700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] conn(0x7f0e740bdbc0 0x7f0e740c00b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.570+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e740b2910 msgr2=0x7f0e740b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.570+0000 7f0e8fe00700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f0e740b2910 0x7f0e740b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005950 tx=0x7f0e7c0058e0 comp rx=0 tx=0).stop 2024-05-27T07:43:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.570+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e90105300 msgr2=0x7f0e9019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.570+0000 7f0e8fe00700 1 --2- 172.21.15.90:0/2275050803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e90105300 0x7f0e9019cd70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0e80005910 tx=0x7f0e800047b0 comp rx=0 tx=0).stop 2024-05-27T07:43:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.571+0000 7f0e8f400700 1 -- 172.21.15.90:0/2275050803 reap_dead start 2024-05-27T07:43:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.571+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 shutdown_connections 2024-05-27T07:43:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.571+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 >> 172.21.15.90:0/2275050803 conn(0x7f0e90078ab0 msgr2=0x7f0e90102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.571+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 shutdown_connections 2024-05-27T07:43:52.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:49.571+0000 7f0e8fe00700 1 -- 172.21.15.90:0/2275050803 wait complete. 2024-05-27T07:43:53.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:53 smithi090 ceph-mon[31429]: pgmap v468: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:43:53.703 INFO:teuthology.orchestra.run.smithi090.stdout:34359738500 2024-05-27T07:43:53.704 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-27T07:43:53.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.675+0000 7f5087800700 1 -- 172.21.15.90:0/2502206873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50800764a0 msgr2=0x7f5080114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.675+0000 7f5087800700 1 --2- 172.21.15.90:0/2502206873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50800764a0 0x7f5080114e20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f506c009990 tx=0x7f506c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.676+0000 7f5087800700 1 -- 172.21.15.90:0/2502206873 shutdown_connections 2024-05-27T07:43:53.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.676+0000 7f5087800700 1 --2- 172.21.15.90:0/2502206873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5080115360 0x7f5080117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.676+0000 7f5087800700 1 --2- 172.21.15.90:0/2502206873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5080075b50 0x7f5080075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.676+0000 7f5087800700 1 --2- 172.21.15.90:0/2502206873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50800764a0 0x7f5080114e20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.676+0000 7f5087800700 1 -- 172.21.15.90:0/2502206873 >> 172.21.15.90:0/2502206873 conn(0x7f5080100030 msgr2=0x7f5080102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:53.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.676+0000 7f5087800700 1 -- 172.21.15.90:0/2502206873 shutdown_connections 2024-05-27T07:43:53.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.676+0000 7f5087800700 1 -- 172.21.15.90:0/2502206873 wait complete. 2024-05-27T07:43:53.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.678+0000 7f5087800700 1 Processor -- start 2024-05-27T07:43:53.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.678+0000 7f5087800700 1 -- start start 2024-05-27T07:43:53.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.679+0000 7f5087800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5080075b50 0x7f50801a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.679+0000 7f5087800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50800764a0 0x7f50801a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.679+0000 7f5087800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5080115360 0x7f50801a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.679+0000 7f5087800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f508011a980 con 0x7f5080075b50 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.679+0000 7f5087800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f508011a800 con 0x7f5080115360 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.679+0000 7f5087800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f508011ab00 con 0x7f50800764a0 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.679+0000 7f5084e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5080075b50 0x7f50801a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.679+0000 7f5084e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5080075b50 0x7f50801a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54444/0 (socket says 172.21.15.90:54444) 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.679+0000 7f5084e00700 1 -- 172.21.15.90:0/440181795 learned_addr learned my addr 172.21.15.90:0/440181795 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.680+0000 7f5085800700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5080115360 0x7f50801a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.680+0000 7f507fe00700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50800764a0 0x7f50801a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.680+0000 7f5084e00700 1 -- 172.21.15.90:0/440181795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50800764a0 msgr2=0x7f50801a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.680+0000 7f5084e00700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50800764a0 0x7f50801a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.680+0000 7f5084e00700 1 -- 172.21.15.90:0/440181795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5080115360 msgr2=0x7f50801a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.680+0000 7f5084e00700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5080115360 0x7f50801a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.680+0000 7f5084e00700 1 -- 172.21.15.90:0/440181795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50801a62a0 con 0x7f5080075b50 2024-05-27T07:43:53.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.680+0000 7f507fe00700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50800764a0 0x7f50801a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:43:53.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.680+0000 7f5084e00700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5080075b50 0x7f50801a13d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f507400d980 tx=0x7f507400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:53.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.681+0000 7f507d600700 1 -- 172.21.15.90:0/440181795 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074017070 con 0x7f5080075b50 2024-05-27T07:43:53.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.681+0000 7f507d600700 1 -- 172.21.15.90:0/440181795 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5074004830 con 0x7f5080075b50 2024-05-27T07:43:53.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.681+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50801a6590 con 0x7f5080075b50 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.681+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50801adeb0 con 0x7f5080075b50 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.681+0000 7f507d600700 1 -- 172.21.15.90:0/440181795 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50740053a0 con 0x7f5080075b50 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.682+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5060001030 con 0x7f5080075b50 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.686+0000 7f507d600700 1 -- 172.21.15.90:0/440181795 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5074005540 con 0x7f5080075b50 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.688+0000 7f507d600700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50680b2910 0x7f50680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.688+0000 7f507d600700 1 -- 172.21.15.90:0/440181795 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f50740e2c40 con 0x7f5080075b50 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.688+0000 7f507fe00700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50680b2910 0x7f50680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.688+0000 7f507d600700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] conn(0x7f50680bdbc0 0x7f50680c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.688+0000 7f507d600700 1 -- 172.21.15.90:0/440181795 --> [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5074024e70 con 0x7f50680bdbc0 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.688+0000 7f507d600700 1 -- 172.21.15.90:0/440181795 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f50740e3080 con 0x7f5080075b50 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.689+0000 7f5085800700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] conn(0x7f50680bdbc0 0x7f50680c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:53.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.689+0000 7f507fe00700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50680b2910 0x7f50680b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f506c00b5c0 tx=0x7f506c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.689+0000 7f5085800700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] conn(0x7f50680bdbc0 0x7f50680c00b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.690+0000 7f507d600700 1 -- 172.21.15.90:0/440181795 <== osd.6 v2:172.21.15.90:6818/476520683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5074024e70 con 0x7f50680bdbc0 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.708+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 --> [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5060002e60 con 0x7f50680bdbc0 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.709+0000 7f507d600700 1 -- 172.21.15.90:0/440181795 <== osd.6 v2:172.21.15.90:6818/476520683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5060002e60 con 0x7f50680bdbc0 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.709+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] conn(0x7f50680bdbc0 msgr2=0x7f50680c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.709+0000 7f5087800700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] conn(0x7f50680bdbc0 0x7f50680c00b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.709+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50680b2910 msgr2=0x7f50680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.709+0000 7f5087800700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f50680b2910 0x7f50680b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f506c00b5c0 tx=0x7f506c005fd0 comp rx=0 tx=0).stop 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.709+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5080075b50 msgr2=0x7f50801a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.709+0000 7f5087800700 1 --2- 172.21.15.90:0/440181795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5080075b50 0x7f50801a13d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f507400d980 tx=0x7f507400dd40 comp rx=0 tx=0).stop 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.709+0000 7f5085800700 1 -- 172.21.15.90:0/440181795 reap_dead start 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.709+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 shutdown_connections 2024-05-27T07:43:53.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.709+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 >> 172.21.15.90:0/440181795 conn(0x7f5080100030 msgr2=0x7f5080100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:53.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.710+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 shutdown_connections 2024-05-27T07:43:53.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:52.710+0000 7f5087800700 1 -- 172.21.15.90:0/440181795 wait complete. 2024-05-27T07:43:53.865 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:53 smithi129 ceph-mon[35254]: pgmap v468: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:43:53.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:53 smithi136 ceph-mon[35324]: pgmap v468: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:43:54.154 INFO:teuthology.orchestra.run.smithi090.stdout:261993005121 2024-05-27T07:43:54.155 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-27T07:43:54.448 INFO:teuthology.orchestra.run.smithi090.stdout:309237645363 2024-05-27T07:43:54.448 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-27T07:43:55.100 INFO:teuthology.orchestra.run.smithi090.stdout:55834574973 2024-05-27T07:43:55.100 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-27T07:43:55.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.770+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3918779597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc8105e40 msgr2=0x7f7fc810e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.770+0000 7f7fcfc00700 1 --2- 172.21.15.90:0/3918779597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc8105e40 0x7f7fc810e390 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009990 tx=0x7f7fb4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.770+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3918779597 shutdown_connections 2024-05-27T07:43:55.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.770+0000 7f7fcfc00700 1 --2- 172.21.15.90:0/3918779597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc810e8d0 0x7f7fc8110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.770+0000 7f7fcfc00700 1 --2- 172.21.15.90:0/3918779597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc81054f0 0x7f7fc8105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.770+0000 7f7fcfc00700 1 --2- 172.21.15.90:0/3918779597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc8105e40 0x7f7fc810e390 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.770+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3918779597 >> 172.21.15.90:0/3918779597 conn(0x7f7fc80fedb0 msgr2=0x7f7fc81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:55.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.770+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3918779597 shutdown_connections 2024-05-27T07:43:55.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.770+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3918779597 wait complete. 2024-05-27T07:43:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.771+0000 7f7fcfc00700 1 Processor -- start 2024-05-27T07:43:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.771+0000 7f7fcfc00700 1 -- start start 2024-05-27T07:43:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fcfc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc81054f0 0x7f7fc81a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fcfc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc8105e40 0x7f7fc81a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fcfc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc810e8d0 0x7f7fc819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fcfc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc81142e0 con 0x7f7fc8105e40 2024-05-27T07:43:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fcfc00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8114160 con 0x7f7fc810e8d0 2024-05-27T07:43:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fcfc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc8114460 con 0x7f7fc81054f0 2024-05-27T07:43:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fc7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc8105e40 0x7f7fc81a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fc7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc8105e40 0x7f7fc81a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54486/0 (socket says 172.21.15.90:54486) 2024-05-27T07:43:55.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fc7e00700 1 -- 172.21.15.90:0/3727369122 learned_addr learned my addr 172.21.15.90:0/3727369122 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:55.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fcd200700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc81054f0 0x7f7fc81a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:55.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.772+0000 7f7fcdc00700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc810e8d0 0x7f7fc819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:55.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.773+0000 7f7fc7e00700 1 -- 172.21.15.90:0/3727369122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc81054f0 msgr2=0x7f7fc81a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.773+0000 7f7fc7e00700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc81054f0 0x7f7fc81a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.773+0000 7f7fc7e00700 1 -- 172.21.15.90:0/3727369122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc810e8d0 msgr2=0x7f7fc819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.773+0000 7f7fc7e00700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fc810e8d0 0x7f7fc819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.773+0000 7f7fc7e00700 1 -- 172.21.15.90:0/3727369122 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc819b1e0 con 0x7f7fc8105e40 2024-05-27T07:43:55.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.773+0000 7f7fc7e00700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc8105e40 0x7f7fc81a0e20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40057c0 tx=0x7f7fb4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:55.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.773+0000 7f7fc5600700 1 -- 172.21.15.90:0/3727369122 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4027070 con 0x7f7fc8105e40 2024-05-27T07:43:55.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.773+0000 7f7fc5600700 1 -- 172.21.15.90:0/3727369122 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fb400fda0 con 0x7f7fc8105e40 2024-05-27T07:43:55.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.773+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc819b470 con 0x7f7fc8105e40 2024-05-27T07:43:55.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.773+0000 7f7fc5600700 1 -- 172.21.15.90:0/3727369122 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4022c70 con 0x7f7fc8105e40 2024-05-27T07:43:55.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.774+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc81a7770 con 0x7f7fc8105e40 2024-05-27T07:43:55.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.775+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7fc8063f40 con 0x7f7fc8105e40 2024-05-27T07:43:55.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.777+0000 7f7fc5600700 1 -- 172.21.15.90:0/3727369122 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7fb402a070 con 0x7f7fc8105e40 2024-05-27T07:43:55.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.779+0000 7f7fc5600700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7fb00b2910 0x7f7fb00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:55.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.779+0000 7f7fc5600700 1 -- 172.21.15.90:0/3727369122 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f7fb40e3b60 con 0x7f7fc8105e40 2024-05-27T07:43:55.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.779+0000 7f7fcd200700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7fb00b2910 0x7f7fb00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:55.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.779+0000 7f7fc5600700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] conn(0x7f7fb00bdbc0 0x7f7fb00c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:55.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.779+0000 7f7fc5600700 1 -- 172.21.15.90:0/3727369122 --> [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fb4012ec0 con 0x7f7fb00bdbc0 2024-05-27T07:43:55.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.779+0000 7f7fc5600700 1 -- 172.21.15.90:0/3727369122 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f7fb40e3f10 con 0x7f7fc8105e40 2024-05-27T07:43:55.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.780+0000 7f7fcd200700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7fb00b2910 0x7f7fb00b4e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc005950 tx=0x7f7fbc005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:55.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.780+0000 7f7fcdc00700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] conn(0x7f7fb00bdbc0 0x7f7fb00c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:55.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.781+0000 7f7fcdc00700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] conn(0x7f7fb00bdbc0 0x7f7fb00c00b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:55.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.784+0000 7f7fc5600700 1 -- 172.21.15.90:0/3727369122 <== osd.8 v2:172.21.15.136:6816/2341317946 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fb4012ec0 con 0x7f7fb00bdbc0 2024-05-27T07:43:55.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.802+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 --> [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7fc80643c0 con 0x7f7fb00bdbc0 2024-05-27T07:43:55.801 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:55 smithi090 ceph-mon[31429]: pgmap v469: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-05-27T07:43:55.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.803+0000 7f7fc5600700 1 -- 172.21.15.90:0/3727369122 <== osd.8 v2:172.21.15.136:6816/2341317946 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7fc80643c0 con 0x7f7fb00bdbc0 2024-05-27T07:43:55.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.803+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 >> [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] conn(0x7f7fb00bdbc0 msgr2=0x7f7fb00c00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.803+0000 7f7fcfc00700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] conn(0x7f7fb00bdbc0 0x7f7fb00c00b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.803+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7fb00b2910 msgr2=0x7f7fb00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.803+0000 7f7fcfc00700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f7fb00b2910 0x7f7fb00b4e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc005950 tx=0x7f7fbc005ed0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.803+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc8105e40 msgr2=0x7f7fc81a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.803+0000 7f7fcfc00700 1 --2- 172.21.15.90:0/3727369122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fc8105e40 0x7f7fc81a0e20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40057c0 tx=0x7f7fb4009f90 comp rx=0 tx=0).stop 2024-05-27T07:43:55.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.803+0000 7f7fcdc00700 1 -- 172.21.15.90:0/3727369122 reap_dead start 2024-05-27T07:43:55.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.803+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 shutdown_connections 2024-05-27T07:43:55.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.803+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 >> 172.21.15.90:0/3727369122 conn(0x7f7fc80fedb0 msgr2=0x7f7fc80ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:55.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.804+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 shutdown_connections 2024-05-27T07:43:55.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.804+0000 7f7fcfc00700 1 -- 172.21.15.90:0/3727369122 wait complete. 2024-05-27T07:43:55.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.970+0000 7fe9b1200700 1 -- 172.21.15.90:0/1230239536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ac108d20 msgr2=0x7fe9ac109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.970+0000 7fe9b1200700 1 --2- 172.21.15.90:0/1230239536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ac108d20 0x7fe9ac109130 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe9980099c0 tx=0x7fe998009cd0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.970+0000 7fe9b1200700 1 -- 172.21.15.90:0/1230239536 shutdown_connections 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.970+0000 7fe9b1200700 1 --2- 172.21.15.90:0/1230239536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9ac109670 0x7fe9ac110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.970+0000 7fe9b1200700 1 --2- 172.21.15.90:0/1230239536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac1043f0 0x7fe9ac104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.970+0000 7fe9b1200700 1 --2- 172.21.15.90:0/1230239536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ac108d20 0x7fe9ac109130 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.970+0000 7fe9b1200700 1 -- 172.21.15.90:0/1230239536 >> 172.21.15.90:0/1230239536 conn(0x7fe9ac079d00 msgr2=0x7fe9ac07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.970+0000 7fe9b1200700 1 -- 172.21.15.90:0/1230239536 shutdown_connections 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.970+0000 7fe9b1200700 1 -- 172.21.15.90:0/1230239536 wait complete. 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.971+0000 7fe9b1200700 1 Processor -- start 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.971+0000 7fe9b1200700 1 -- start start 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.971+0000 7fe9b1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ac1043f0 0x7fe9ac075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:55.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9b1200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac108d20 0x7fe9ac075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9b1200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9ac109670 0x7fe9ac076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9b1200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac114180 con 0x7fe9ac1043f0 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9b1200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac114000 con 0x7fe9ac108d20 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9b1200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac114300 con 0x7fe9ac109670 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9ab400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ac1043f0 0x7fe9ac075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9ab400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ac1043f0 0x7fe9ac075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54460/0 (socket says 172.21.15.90:54460) 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9ab400700 1 -- 172.21.15.90:0/998875260 learned_addr learned my addr 172.21.15.90:0/998875260 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9aaa00700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac108d20 0x7fe9ac075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9abe00700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9ac109670 0x7fe9ac076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9ab400700 1 -- 172.21.15.90:0/998875260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9ac109670 msgr2=0x7fe9ac076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9ab400700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9ac109670 0x7fe9ac076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9ab400700 1 -- 172.21.15.90:0/998875260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac108d20 msgr2=0x7fe9ac075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9ab400700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac108d20 0x7fe9ac075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9ab400700 1 -- 172.21.15.90:0/998875260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac1a8f30 con 0x7fe9ac1043f0 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.972+0000 7fe9aaa00700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac108d20 0x7fe9ac075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.973+0000 7fe9ab400700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ac1043f0 0x7fe9ac075ab0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00d950 tx=0x7fe99c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.973+0000 7fe9a3e00700 1 -- 172.21.15.90:0/998875260 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c016070 con 0x7fe9ac1043f0 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.973+0000 7fe9a3e00700 1 -- 172.21.15.90:0/998875260 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe99c010460 con 0x7fe9ac1043f0 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.973+0000 7fe9a3e00700 1 -- 172.21.15.90:0/998875260 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c00f620 con 0x7fe9ac1043f0 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.973+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac1a9220 con 0x7fe9ac1043f0 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.973+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac1a9710 con 0x7fe9ac1043f0 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.974+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9ac063e80 con 0x7fe9ac1043f0 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.977+0000 7fe9a3e00700 1 -- 172.21.15.90:0/998875260 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe99c019070 con 0x7fe9ac1043f0 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.978+0000 7fe9a3e00700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe9900b2990 0x7fe9900b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:55.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.978+0000 7fe9a3e00700 1 -- 172.21.15.90:0/998875260 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fe99c0e2800 con 0x7fe9ac1043f0 2024-05-27T07:43:55.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.978+0000 7fe9aaa00700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe9900b2990 0x7fe9900b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:55.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.978+0000 7fe9a3e00700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] conn(0x7fe9900bdc40 0x7fe9900c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:55.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.978+0000 7fe9a3e00700 1 -- 172.21.15.90:0/998875260 --> [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9900c0800 con 0x7fe9900bdc40 2024-05-27T07:43:55.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.978+0000 7fe9a3e00700 1 -- 172.21.15.90:0/998875260 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe99c0e2c50 con 0x7fe9ac1043f0 2024-05-27T07:43:55.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.979+0000 7fe9aaa00700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe9900b2990 0x7fe9900b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe998006010 tx=0x7fe998009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:55.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.979+0000 7fe9abe00700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] conn(0x7fe9900bdc40 0x7fe9900c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:55.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.980+0000 7fe9abe00700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] conn(0x7fe9900bdc40 0x7fe9900c0130 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:55.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.983+0000 7fe9a3e00700 1 -- 172.21.15.90:0/998875260 <== osd.5 v2:172.21.15.136:6808/3317770816 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9900c0800 con 0x7fe9900bdc40 2024-05-27T07:43:55.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:54.999+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 --> [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9ac064350 con 0x7fe9900bdc40 2024-05-27T07:43:55.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.001+0000 7fe9a3e00700 1 -- 172.21.15.90:0/998875260 <== osd.5 v2:172.21.15.136:6808/3317770816 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe9ac064350 con 0x7fe9900bdc40 2024-05-27T07:43:55.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.001+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 >> [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] conn(0x7fe9900bdc40 msgr2=0x7fe9900c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.001+0000 7fe9b1200700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] conn(0x7fe9900bdc40 0x7fe9900c0130 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:55.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.001+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe9900b2990 msgr2=0x7fe9900b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.001+0000 7fe9b1200700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe9900b2990 0x7fe9900b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe998006010 tx=0x7fe998009670 comp rx=0 tx=0).stop 2024-05-27T07:43:55.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.001+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ac1043f0 msgr2=0x7fe9ac075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:55.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.001+0000 7fe9b1200700 1 --2- 172.21.15.90:0/998875260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ac1043f0 0x7fe9ac075ab0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00d950 tx=0x7fe99c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:43:55.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.001+0000 7fe9abe00700 1 -- 172.21.15.90:0/998875260 reap_dead start 2024-05-27T07:43:55.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.001+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 shutdown_connections 2024-05-27T07:43:55.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.001+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 >> 172.21.15.90:0/998875260 conn(0x7fe9ac079d00 msgr2=0x7fe9ac101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:55.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.002+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 shutdown_connections 2024-05-27T07:43:55.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:55.002+0000 7fe9b1200700 1 -- 172.21.15.90:0/998875260 wait complete. 2024-05-27T07:43:55.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:55 smithi136 ceph-mon[35324]: pgmap v469: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-05-27T07:43:56.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:55 smithi129 ceph-mon[35254]: pgmap v469: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 8 op/s 2024-05-27T07:43:56.946 INFO:teuthology.orchestra.run.smithi090.stdout:236223201354 2024-05-27T07:43:56.947 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-27T07:43:57.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.243+0000 7f0613a00700 1 -- 172.21.15.90:0/1855563848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c076be0 msgr2=0x7f060c076ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.243+0000 7f0613a00700 1 --2- 172.21.15.90:0/1855563848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c076be0 0x7f060c076ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0600009990 tx=0x7f0600009ca0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.244+0000 7f0613a00700 1 -- 172.21.15.90:0/1855563848 shutdown_connections 2024-05-27T07:43:57.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.244+0000 7f0613a00700 1 --2- 172.21.15.90:0/1855563848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f060c110bf0 0x7f060c113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.244+0000 7f0613a00700 1 --2- 172.21.15.90:0/1855563848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c076be0 0x7f060c076ff0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.244+0000 7f0613a00700 1 --2- 172.21.15.90:0/1855563848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c077530 0x7f060c1106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.244+0000 7f0613a00700 1 -- 172.21.15.90:0/1855563848 >> 172.21.15.90:0/1855563848 conn(0x7f060c0ffe90 msgr2=0x7f060c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:57.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.244+0000 7f0613a00700 1 -- 172.21.15.90:0/1855563848 shutdown_connections 2024-05-27T07:43:57.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.244+0000 7f0613a00700 1 -- 172.21.15.90:0/1855563848 wait complete. 2024-05-27T07:43:57.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.245+0000 7f0613a00700 1 Processor -- start 2024-05-27T07:43:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.245+0000 7f0613a00700 1 -- start start 2024-05-27T07:43:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f0613a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f060c076be0 0x7f060c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f0613a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c077530 0x7f060c19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f0613a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c110bf0 0x7f060c1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f0613a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f060c1165f0 con 0x7f060c077530 2024-05-27T07:43:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f0613a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f060c116470 con 0x7f060c110bf0 2024-05-27T07:43:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f0613a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f060c116770 con 0x7f060c076be0 2024-05-27T07:43:57.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f060be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c077530 0x7f060c19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f060be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c077530 0x7f060c19d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54506/0 (socket says 172.21.15.90:54506) 2024-05-27T07:43:57.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f060be00700 1 -- 172.21.15.90:0/3528361064 learned_addr learned my addr 172.21.15.90:0/3528361064 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:57.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f0611000700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f060c076be0 0x7f060c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.246+0000 7f0611a00700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c110bf0 0x7f060c1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.247+0000 7f060be00700 1 -- 172.21.15.90:0/3528361064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f060c076be0 msgr2=0x7f060c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.247+0000 7f060be00700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f060c076be0 0x7f060c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.247+0000 7f060be00700 1 -- 172.21.15.90:0/3528361064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c110bf0 msgr2=0x7f060c1a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.247+0000 7f060be00700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f060c110bf0 0x7f060c1a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.247+0000 7f060be00700 1 -- 172.21.15.90:0/3528361064 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c1a1e50 con 0x7f060c077530 2024-05-27T07:43:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.247+0000 7f060be00700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c077530 0x7f060c19d520 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f05f800c8f0 tx=0x7f05f800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.248+0000 7f0609600700 1 -- 172.21.15.90:0/3528361064 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f80078d0 con 0x7f060c077530 2024-05-27T07:43:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.248+0000 7f0609600700 1 -- 172.21.15.90:0/3528361064 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05f800f450 con 0x7f060c077530 2024-05-27T07:43:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.248+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060c1a2140 con 0x7f060c077530 2024-05-27T07:43:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.248+0000 7f0609600700 1 -- 172.21.15.90:0/3528361064 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8018660 con 0x7f060c077530 2024-05-27T07:43:57.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.248+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060c1a9a60 con 0x7f060c077530 2024-05-27T07:43:57.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.250+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f060c063f40 con 0x7f060c077530 2024-05-27T07:43:57.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.253+0000 7f0609600700 1 -- 172.21.15.90:0/3528361064 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f05f800f900 con 0x7f060c077530 2024-05-27T07:43:57.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.254+0000 7f0609600700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05f40b2990 0x7f05f40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.254+0000 7f0609600700 1 -- 172.21.15.90:0/3528361064 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f05f80e2750 con 0x7f060c077530 2024-05-27T07:43:57.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.255+0000 7f0611000700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05f40b2990 0x7f05f40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.255+0000 7f0609600700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] conn(0x7f05f40bdc40 0x7f05f40c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.255+0000 7f0609600700 1 -- 172.21.15.90:0/3528361064 --> [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05f8019ec0 con 0x7f05f40bdc40 2024-05-27T07:43:57.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.255+0000 7f0609600700 1 -- 172.21.15.90:0/3528361064 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f05f80e2ba0 con 0x7f060c077530 2024-05-27T07:43:57.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.255+0000 7f0611a00700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] conn(0x7f05f40bdc40 0x7f05f40c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.256+0000 7f0611000700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05f40b2990 0x7f05f40b4ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0600006010 tx=0x7f060000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:57.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.257+0000 7f0611a00700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] conn(0x7f05f40bdc40 0x7f05f40c0130 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:57.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.259+0000 7f0609600700 1 -- 172.21.15.90:0/3528361064 <== osd.10 v2:172.21.15.129:6824/95466853 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05f8019ec0 con 0x7f05f40bdc40 2024-05-27T07:43:57.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.278+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 --> [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f060c064340 con 0x7f05f40bdc40 2024-05-27T07:43:57.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.279+0000 7f0609600700 1 -- 172.21.15.90:0/3528361064 <== osd.10 v2:172.21.15.129:6824/95466853 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f060c064340 con 0x7f05f40bdc40 2024-05-27T07:43:57.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.279+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 >> [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] conn(0x7f05f40bdc40 msgr2=0x7f05f40c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.279+0000 7f0613a00700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] conn(0x7f05f40bdc40 0x7f05f40c0130 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.279+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05f40b2990 msgr2=0x7f05f40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.279+0000 7f0613a00700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f05f40b2990 0x7f05f40b4ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0600006010 tx=0x7f060000b540 comp rx=0 tx=0).stop 2024-05-27T07:43:57.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.279+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c077530 msgr2=0x7f060c19d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.279+0000 7f0613a00700 1 --2- 172.21.15.90:0/3528361064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c077530 0x7f060c19d520 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f05f800c8f0 tx=0x7f05f800cc00 comp rx=0 tx=0).stop 2024-05-27T07:43:57.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.280+0000 7f0611a00700 1 -- 172.21.15.90:0/3528361064 reap_dead start 2024-05-27T07:43:57.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.280+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 shutdown_connections 2024-05-27T07:43:57.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.280+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 >> 172.21.15.90:0/3528361064 conn(0x7f060c0ffe90 msgr2=0x7f060c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:57.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.280+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 shutdown_connections 2024-05-27T07:43:57.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.280+0000 7f0613a00700 1 -- 172.21.15.90:0/3528361064 wait complete. 2024-05-27T07:43:57.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:57 smithi090 ceph-mon[31429]: pgmap v470: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-27T07:43:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.991+0000 7f77db000700 1 -- 172.21.15.90:0/3487734635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77d4109860 msgr2=0x7f77d41111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.991+0000 7f77db000700 1 --2- 172.21.15.90:0/3487734635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77d4109860 0x7f77d41111c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00b0d0 tx=0x7f77cc00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.991+0000 7f77db000700 1 -- 172.21.15.90:0/3487734635 shutdown_connections 2024-05-27T07:43:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.991+0000 7f77db000700 1 --2- 172.21.15.90:0/3487734635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77d4109860 0x7f77d41111c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.991+0000 7f77db000700 1 --2- 172.21.15.90:0/3487734635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77d41045e0 0x7f77d41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.991+0000 7f77db000700 1 --2- 172.21.15.90:0/3487734635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77d4108f10 0x7f77d4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.991+0000 7f77db000700 1 -- 172.21.15.90:0/3487734635 >> 172.21.15.90:0/3487734635 conn(0x7f77d4100030 msgr2=0x7f77d4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.991+0000 7f77db000700 1 -- 172.21.15.90:0/3487734635 shutdown_connections 2024-05-27T07:43:57.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.991+0000 7f77db000700 1 -- 172.21.15.90:0/3487734635 wait complete. 2024-05-27T07:43:57.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.993+0000 7f77db000700 1 Processor -- start 2024-05-27T07:43:57.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.993+0000 7f77db000700 1 -- start start 2024-05-27T07:43:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.993+0000 7f77db000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77d41045e0 0x7f77d419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.993+0000 7f77db000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77d4108f10 0x7f77d419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.993+0000 7f77db000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77d4109860 0x7f77d41a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.993+0000 7f77db000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f77d4114340 con 0x7f77d4108f10 2024-05-27T07:43:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.993+0000 7f77db000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41141c0 con 0x7f77d4109860 2024-05-27T07:43:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.993+0000 7f77db000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41144c0 con 0x7f77d41045e0 2024-05-27T07:43:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.994+0000 7f77cbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77d4108f10 0x7f77d419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.994+0000 7f77cbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77d4108f10 0x7f77d419d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54524/0 (socket says 172.21.15.90:54524) 2024-05-27T07:43:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.994+0000 7f77cbe00700 1 -- 172.21.15.90:0/115745924 learned_addr learned my addr 172.21.15.90:0/115745924 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:43:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.994+0000 7f77d9000700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77d4109860 0x7f77d41a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.994+0000 7f77cbe00700 1 -- 172.21.15.90:0/115745924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77d41045e0 msgr2=0x7f77d419cf30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.994+0000 7f77cbe00700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77d41045e0 0x7f77d419cf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.994+0000 7f77cbe00700 1 -- 172.21.15.90:0/115745924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77d4109860 msgr2=0x7f77d41a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.994+0000 7f77cbe00700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77d4109860 0x7f77d41a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.994+0000 7f77cbe00700 1 -- 172.21.15.90:0/115745924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d41a1e00 con 0x7f77d4108f10 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.994+0000 7f77d9000700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77d4109860 0x7f77d41a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.995+0000 7f77cbe00700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77d4108f10 0x7f77d419d4d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00c8f0 tx=0x7f77bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.995+0000 7f77d1600700 1 -- 172.21.15.90:0/115745924 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc007b00 con 0x7f77d4108f10 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.995+0000 7f77d1600700 1 -- 172.21.15.90:0/115745924 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77bc007ca0 con 0x7f77d4108f10 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.995+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d41a20f0 con 0x7f77d4108f10 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.995+0000 7f77d1600700 1 -- 172.21.15.90:0/115745924 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc018660 con 0x7f77d4108f10 2024-05-27T07:43:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.995+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d41a9a10 con 0x7f77d4108f10 2024-05-27T07:43:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:57.996+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f77b0001030 con 0x7f77d4108f10 2024-05-27T07:43:57.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:57 smithi136 ceph-mon[35324]: pgmap v470: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-27T07:43:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.000+0000 7f77d1600700 1 -- 172.21.15.90:0/115745924 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f77bc00f450 con 0x7f77d4108f10 2024-05-27T07:43:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.001+0000 7f77d1600700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77b80b2910 0x7f77b80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.002+0000 7f77d1600700 1 -- 172.21.15.90:0/115745924 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f77bc0e2870 con 0x7f77d4108f10 2024-05-27T07:43:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.002+0000 7f77d3e00700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77b80b2910 0x7f77b80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.002+0000 7f77d1600700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] conn(0x7f77b80bdbb0 0x7f77b80c00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:43:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.002+0000 7f77d1600700 1 -- 172.21.15.90:0/115745924 --> [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77bc019ec0 con 0x7f77b80bdbb0 2024-05-27T07:43:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.002+0000 7f77d1600700 1 -- 172.21.15.90:0/115745924 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f77bc0e2cb0 con 0x7f77d4108f10 2024-05-27T07:43:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.003+0000 7f77d3e00700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77b80b2910 0x7f77b80b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f77c400a910 tx=0x7f77c4005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:58.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.011+0000 7f77d9000700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] conn(0x7f77b80bdbb0 0x7f77b80c00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:43:58.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.013+0000 7f77d9000700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] conn(0x7f77b80bdbb0 0x7f77b80c00a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:43:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.015+0000 7f77d1600700 1 -- 172.21.15.90:0/115745924 <== osd.4 v2:172.21.15.129:6808/2737784840 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77bc019ec0 con 0x7f77b80bdbb0 2024-05-27T07:43:58.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.033+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 --> [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f77b0002e60 con 0x7f77b80bdbb0 2024-05-27T07:43:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.034+0000 7f77d1600700 1 -- 172.21.15.90:0/115745924 <== osd.4 v2:172.21.15.129:6808/2737784840 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f77b0002e60 con 0x7f77b80bdbb0 2024-05-27T07:43:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 >> [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] conn(0x7f77b80bdbb0 msgr2=0x7f77b80c00a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77db000700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] conn(0x7f77b80bdbb0 0x7f77b80c00a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77b80b2910 msgr2=0x7f77b80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77db000700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f77b80b2910 0x7f77b80b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f77c400a910 tx=0x7f77c4005c10 comp rx=0 tx=0).stop 2024-05-27T07:43:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77d4108f10 msgr2=0x7f77d419d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:43:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77db000700 1 --2- 172.21.15.90:0/115745924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77d4108f10 0x7f77d419d4d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f77bc00c8f0 tx=0x7f77bc00ccb0 comp rx=0 tx=0).stop 2024-05-27T07:43:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77d9000700 1 -- 172.21.15.90:0/115745924 reap_dead start 2024-05-27T07:43:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 shutdown_connections 2024-05-27T07:43:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 >> 172.21.15.90:0/115745924 conn(0x7f77d4100030 msgr2=0x7f77d4110e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:43:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 shutdown_connections 2024-05-27T07:43:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:43:58.035+0000 7f77db000700 1 -- 172.21.15.90:0/115745924 wait complete. 2024-05-27T07:43:58.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:57 smithi129 ceph-mon[35254]: pgmap v470: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-27T07:43:58.676 INFO:teuthology.orchestra.run.smithi090.stdout:163208757345 2024-05-27T07:43:58.677 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-27T07:43:58.989 INFO:teuthology.orchestra.run.smithi090.stdout:210453397584 2024-05-27T07:43:58.989 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-27T07:43:59.610 INFO:teuthology.orchestra.run.smithi090.stdout:141733920873 2024-05-27T07:43:59.610 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-27T07:43:59.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:43:59 smithi090 ceph-mon[31429]: pgmap v471: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-27T07:43:59.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:43:59 smithi136 ceph-mon[35324]: pgmap v471: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-27T07:44:00.062 INFO:teuthology.orchestra.run.smithi090.stdout:283467841596 2024-05-27T07:44:00.062 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-27T07:44:00.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:43:59 smithi129 ceph-mon[35254]: pgmap v471: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-27T07:44:00.362 INFO:teuthology.orchestra.run.smithi090.stdout:188978561112 2024-05-27T07:44:00.362 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-27T07:44:00.555 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:00.772 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:00.778 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:00.782 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:01.020 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:01.031 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:01.031 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:01.035 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:01.037 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:01.037 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:01 smithi090 ceph-mon[31429]: pgmap v472: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T07:44:01.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:01 smithi136 ceph-mon[35324]: pgmap v472: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T07:44:02.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:01 smithi129 ceph-mon[35254]: pgmap v472: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T07:44:03.931 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:03 smithi090 ceph-mon[31429]: pgmap v473: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:04.138 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:03 smithi129 ceph-mon[35254]: pgmap v473: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:04.139 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:03 smithi136 ceph-mon[35324]: pgmap v473: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:05.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:05 smithi090 ceph-mon[31429]: pgmap v474: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:05.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:05 smithi136 ceph-mon[35324]: pgmap v474: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:06.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:05 smithi129 ceph-mon[35254]: pgmap v474: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:07.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:07 smithi090 ceph-mon[31429]: pgmap v475: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-27T07:44:07.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:07 smithi136 ceph-mon[35324]: pgmap v475: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-27T07:44:08.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:07 smithi129 ceph-mon[35254]: pgmap v475: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 11 op/s 2024-05-27T07:44:09.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:09 smithi090 ceph-mon[31429]: pgmap v476: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-27T07:44:09.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:09 smithi136 ceph-mon[35324]: pgmap v476: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-27T07:44:10.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:09 smithi129 ceph-mon[35254]: pgmap v476: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-27T07:44:10.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:10 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/116883195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T07:44:10.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:10 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/116883195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T07:44:11.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:10 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/116883195' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T07:44:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:11 smithi090 ceph-mon[31429]: pgmap v477: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-27T07:44:11.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:11 smithi136 ceph-mon[35324]: pgmap v477: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-27T07:44:12.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:11 smithi129 ceph-mon[35254]: pgmap v477: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-27T07:44:12.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.903+0000 7fa67b200700 1 -- 172.21.15.90:0/3133400274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67c108550 msgr2=0x7fa67c10feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.903+0000 7fa67b200700 1 --2- 172.21.15.90:0/3133400274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67c108550 0x7fa67c10feb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa67400b0d0 tx=0x7fa67400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.903+0000 7fa67b200700 1 -- 172.21.15.90:0/3133400274 shutdown_connections 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.903+0000 7fa67b200700 1 --2- 172.21.15.90:0/3133400274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67c108550 0x7fa67c10feb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.903+0000 7fa67b200700 1 --2- 172.21.15.90:0/3133400274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa67c1032d0 0x7fa67c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.903+0000 7fa67b200700 1 --2- 172.21.15.90:0/3133400274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c107c00 0x7fa67c108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.903+0000 7fa67b200700 1 -- 172.21.15.90:0/3133400274 >> 172.21.15.90:0/3133400274 conn(0x7fa67c078ab0 msgr2=0x7fa67c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.903+0000 7fa67b200700 1 -- 172.21.15.90:0/3133400274 shutdown_connections 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.904+0000 7fa67b200700 1 -- 172.21.15.90:0/3133400274 wait complete. 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.905+0000 7fa67b200700 1 Processor -- start 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.905+0000 7fa67b200700 1 -- start start 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa67b200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa67c1032d0 0x7fa67c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa67b200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c107c00 0x7fa67c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa67b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67c108550 0x7fa67c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa67b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c113170 con 0x7fa67c108550 2024-05-27T07:44:12.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa67b200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c112ff0 con 0x7fa67c1032d0 2024-05-27T07:44:12.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa67b200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c1132f0 con 0x7fa67c107c00 2024-05-27T07:44:12.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa67a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67c108550 0x7fa67c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa67a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67c108550 0x7fa67c1a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47726/0 (socket says 172.21.15.90:47726) 2024-05-27T07:44:12.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa67a800700 1 -- 172.21.15.90:0/116883195 learned_addr learned my addr 172.21.15.90:0/116883195 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:12.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa679e00700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa67c1032d0 0x7fa67c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.906+0000 7fa679400700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c107c00 0x7fa67c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa67a800700 1 -- 172.21.15.90:0/116883195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c107c00 msgr2=0x7fa67c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa67a800700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c107c00 0x7fa67c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa67a800700 1 -- 172.21.15.90:0/116883195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa67c1032d0 msgr2=0x7fa67c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa67a800700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa67c1032d0 0x7fa67c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa67a800700 1 -- 172.21.15.90:0/116883195 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67c1a1da0 con 0x7fa67c108550 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa67a800700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67c108550 0x7fa67c1a1700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa674007d10 tx=0x7fa674007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa672a00700 1 -- 172.21.15.90:0/116883195 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67401f020 con 0x7fa67c108550 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa672a00700 1 -- 172.21.15.90:0/116883195 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa674003e10 con 0x7fa67c108550 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa67b200700 1 -- 172.21.15.90:0/116883195 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67c1a2030 con 0x7fa67c108550 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa672a00700 1 -- 172.21.15.90:0/116883195 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67401bab0 con 0x7fa67c108550 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.907+0000 7fa67b200700 1 -- 172.21.15.90:0/116883195 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa67c1a99b0 con 0x7fa67c108550 2024-05-27T07:44:12.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.909+0000 7fa67b200700 1 -- 172.21.15.90:0/116883195 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa67c104780 con 0x7fa67c108550 2024-05-27T07:44:12.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.911+0000 7fa672a00700 1 -- 172.21.15.90:0/116883195 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa674022020 con 0x7fa67c108550 2024-05-27T07:44:12.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.913+0000 7fa672a00700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa6600b2990 0x7fa6600b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:12.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.914+0000 7fa672a00700 1 -- 172.21.15.90:0/116883195 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fa67400f020 con 0x7fa67c108550 2024-05-27T07:44:12.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.914+0000 7fa679e00700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa6600b2990 0x7fa6600b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:12.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.915+0000 7fa679e00700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa6600b2990 0x7fa6600b4ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa66c006fd0 tx=0x7fa66c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:12.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:09.918+0000 7fa672a00700 1 -- 172.21.15.90:0/116883195 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6740ab1e0 con 0x7fa67c108550 2024-05-27T07:44:12.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.160+0000 7fa67b200700 1 -- 172.21.15.90:0/116883195 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa67c02e5e0 con 0x7fa67c108550 2024-05-27T07:44:12.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.161+0000 7fa672a00700 1 -- 172.21.15.90:0/116883195 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa6740aaae0 con 0x7fa67c108550 2024-05-27T07:44:12.352 INFO:teuthology.orchestra.run.smithi090.stdout:115964117107 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.164+0000 7fa67b200700 1 -- 172.21.15.90:0/116883195 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa6600b2990 msgr2=0x7fa6600b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.164+0000 7fa67b200700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa6600b2990 0x7fa6600b4ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa66c006fd0 tx=0x7fa66c008040 comp rx=0 tx=0).stop 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.164+0000 7fa67b200700 1 -- 172.21.15.90:0/116883195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67c108550 msgr2=0x7fa67c1a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.164+0000 7fa67b200700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67c108550 0x7fa67c1a1700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa674007d10 tx=0x7fa674007df0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.164+0000 7fa67b200700 1 -- 172.21.15.90:0/116883195 shutdown_connections 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.164+0000 7fa67b200700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fa6600b2990 0x7fa6600b4ea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.164+0000 7fa67b200700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67c108550 0x7fa67c1a1700 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.164+0000 7fa67b200700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa67c1032d0 0x7fa67c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.164+0000 7fa67b200700 1 --2- 172.21.15.90:0/116883195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa67c107c00 0x7fa67c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.164+0000 7fa67b200700 1 -- 172.21.15.90:0/116883195 >> 172.21.15.90:0/116883195 conn(0x7fa67c078ab0 msgr2=0x7fa67c10eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:12.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.165+0000 7fa67b200700 1 -- 172.21.15.90:0/116883195 shutdown_connections 2024-05-27T07:44:12.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:10.165+0000 7fa67b200700 1 -- 172.21.15.90:0/116883195 wait complete. 2024-05-27T07:44:13.942 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:13 smithi090 ceph-mon[31429]: pgmap v478: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:44:13.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:13 smithi136 ceph-mon[35324]: pgmap v478: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:44:14.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:13 smithi129 ceph-mon[35254]: pgmap v478: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:44:14.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.410+0000 7fd541e00700 1 -- 172.21.15.90:0/3615979874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c115170 msgr2=0x7fd53c117590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:14.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.410+0000 7fd541e00700 1 --2- 172.21.15.90:0/3615979874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c115170 0x7fd53c117590 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd53400b0a0 tx=0x7fd53400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.410+0000 7fd541e00700 1 -- 172.21.15.90:0/3615979874 shutdown_connections 2024-05-27T07:44:14.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.410+0000 7fd541e00700 1 --2- 172.21.15.90:0/3615979874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c115170 0x7fd53c117590 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.410+0000 7fd541e00700 1 --2- 172.21.15.90:0/3615979874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd53c075b50 0x7fd53c075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.410+0000 7fd541e00700 1 --2- 172.21.15.90:0/3615979874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53c0764a0 0x7fd53c114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.410+0000 7fd541e00700 1 -- 172.21.15.90:0/3615979874 >> 172.21.15.90:0/3615979874 conn(0x7fd53c079d00 msgr2=0x7fd53c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:14.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.411+0000 7fd541e00700 1 -- 172.21.15.90:0/3615979874 shutdown_connections 2024-05-27T07:44:14.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.411+0000 7fd541e00700 1 -- 172.21.15.90:0/3615979874 wait complete. 2024-05-27T07:44:14.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.413+0000 7fd541e00700 1 Processor -- start 2024-05-27T07:44:14.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.413+0000 7fd541e00700 1 -- start start 2024-05-27T07:44:14.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.414+0000 7fd541e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd53c075b50 0x7fd53c1a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:14.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.414+0000 7fd541e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53c0764a0 0x7fd53c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:14.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.414+0000 7fd541e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c115170 0x7fd53c1a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:14.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.414+0000 7fd541e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c11a7b0 con 0x7fd53c115170 2024-05-27T07:44:14.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.414+0000 7fd541e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c11a630 con 0x7fd53c075b50 2024-05-27T07:44:14.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.414+0000 7fd541e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c11a930 con 0x7fd53c0764a0 2024-05-27T07:44:14.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.414+0000 7fd541400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c115170 0x7fd53c1a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:14.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.414+0000 7fd541400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c115170 0x7fd53c1a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47778/0 (socket says 172.21.15.90:47778) 2024-05-27T07:44:14.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.414+0000 7fd541400700 1 -- 172.21.15.90:0/3077058698 learned_addr learned my addr 172.21.15.90:0/3077058698 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:14.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.415+0000 7fd541400700 1 -- 172.21.15.90:0/3077058698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53c0764a0 msgr2=0x7fd53c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:14.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.415+0000 7fd540a00700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd53c075b50 0x7fd53c1a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:14.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.415+0000 7fd541400700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53c0764a0 0x7fd53c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.415+0000 7fd541400700 1 -- 172.21.15.90:0/3077058698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd53c075b50 msgr2=0x7fd53c1a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:14.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.415+0000 7fd541400700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd53c075b50 0x7fd53c1a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.415+0000 7fd541400700 1 -- 172.21.15.90:0/3077058698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c1a6010 con 0x7fd53c115170 2024-05-27T07:44:14.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.415+0000 7fd540a00700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd53c075b50 0x7fd53c1a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:14.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.416+0000 7fd541400700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c115170 0x7fd53c1a5970 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd53400b070 tx=0x7fd534004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:14.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.416+0000 7fd539600700 1 -- 172.21.15.90:0/3077058698 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534021070 con 0x7fd53c115170 2024-05-27T07:44:14.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.416+0000 7fd539600700 1 -- 172.21.15.90:0/3077058698 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd534007c00 con 0x7fd53c115170 2024-05-27T07:44:14.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.416+0000 7fd541e00700 1 -- 172.21.15.90:0/3077058698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53c1a62a0 con 0x7fd53c115170 2024-05-27T07:44:14.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.416+0000 7fd539600700 1 -- 172.21.15.90:0/3077058698 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd534004850 con 0x7fd53c115170 2024-05-27T07:44:14.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.416+0000 7fd541e00700 1 -- 172.21.15.90:0/3077058698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53c1adc20 con 0x7fd53c115170 2024-05-27T07:44:14.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.417+0000 7fd541e00700 1 -- 172.21.15.90:0/3077058698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd53c077290 con 0x7fd53c115170 2024-05-27T07:44:14.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.420+0000 7fd539600700 1 -- 172.21.15.90:0/3077058698 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd534003fb0 con 0x7fd53c115170 2024-05-27T07:44:14.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.422+0000 7fd539600700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd5240b2990 0x7fd5240b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:14.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.422+0000 7fd539600700 1 -- 172.21.15.90:0/3077058698 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fd53400f020 con 0x7fd53c115170 2024-05-27T07:44:14.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.422+0000 7fd540a00700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd5240b2990 0x7fd5240b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:14.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.423+0000 7fd540a00700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd5240b2990 0x7fd5240b4ea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd530005d90 tx=0x7fd530005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:14.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.427+0000 7fd539600700 1 -- 172.21.15.90:0/3077058698 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd53402a020 con 0x7fd53c115170 2024-05-27T07:44:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.323+0000 7fb761a00700 1 -- 172.21.15.90:0/1466665765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c108f10 msgr2=0x7fb75c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.323+0000 7fb761a00700 1 --2- 172.21.15.90:0/1466665765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c108f10 0x7fb75c109320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb7480099c0 tx=0x7fb748009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.323+0000 7fb761a00700 1 -- 172.21.15.90:0/1466665765 shutdown_connections 2024-05-27T07:44:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.323+0000 7fb761a00700 1 --2- 172.21.15.90:0/1466665765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb75c109860 0x7fb75c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.323+0000 7fb761a00700 1 --2- 172.21.15.90:0/1466665765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb75c1045e0 0x7fb75c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.323+0000 7fb761a00700 1 --2- 172.21.15.90:0/1466665765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c108f10 0x7fb75c109320 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.323+0000 7fb761a00700 1 -- 172.21.15.90:0/1466665765 >> 172.21.15.90:0/1466665765 conn(0x7fb75c100030 msgr2=0x7fb75c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.324+0000 7fb761a00700 1 -- 172.21.15.90:0/1466665765 shutdown_connections 2024-05-27T07:44:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.324+0000 7fb761a00700 1 -- 172.21.15.90:0/1466665765 wait complete. 2024-05-27T07:44:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.325+0000 7fb761a00700 1 Processor -- start 2024-05-27T07:44:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.325+0000 7fb761a00700 1 -- start start 2024-05-27T07:44:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.325+0000 7fb761a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb75c1045e0 0x7fb75c19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.325+0000 7fb761a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c108f10 0x7fb75c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.325+0000 7fb761a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb75c109860 0x7fb75c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.325+0000 7fb761a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c1143e0 con 0x7fb75c108f10 2024-05-27T07:44:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.325+0000 7fb761a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c114260 con 0x7fb75c109860 2024-05-27T07:44:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.325+0000 7fb761a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c114560 con 0x7fb75c1045e0 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c108f10 0x7fb75c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c108f10 0x7fb75c19d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47754/0 (socket says 172.21.15.90:47754) 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75a000700 1 -- 172.21.15.90:0/723219624 learned_addr learned my addr 172.21.15.90:0/723219624 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75aa00700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb75c1045e0 0x7fb75c19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75b400700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb75c109860 0x7fb75c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75a000700 1 -- 172.21.15.90:0/723219624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb75c1045e0 msgr2=0x7fb75c19d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75a000700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb75c1045e0 0x7fb75c19d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75a000700 1 -- 172.21.15.90:0/723219624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb75c109860 msgr2=0x7fb75c1a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75a000700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb75c109860 0x7fb75c1a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75a000700 1 -- 172.21.15.90:0/723219624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75c1a1f00 con 0x7fb75c108f10 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.326+0000 7fb75b400700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb75c109860 0x7fb75c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.327+0000 7fb75a000700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c108f10 0x7fb75c19d5d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb74800b580 tx=0x7fb74800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.327+0000 7fb753400700 1 -- 172.21.15.90:0/723219624 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748027070 con 0x7fb75c108f10 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.327+0000 7fb753400700 1 -- 172.21.15.90:0/723219624 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7480056d0 con 0x7fb75c108f10 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.327+0000 7fb761a00700 1 -- 172.21.15.90:0/723219624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75c1a2190 con 0x7fb75c108f10 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.327+0000 7fb753400700 1 -- 172.21.15.90:0/723219624 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7480173f0 con 0x7fb75c108f10 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.327+0000 7fb761a00700 1 -- 172.21.15.90:0/723219624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75c1a2670 con 0x7fb75c108f10 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.328+0000 7fb761a00700 1 -- 172.21.15.90:0/723219624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb75c0512b0 con 0x7fb75c108f10 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.331+0000 7fb753400700 1 -- 172.21.15.90:0/723219624 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb74802a070 con 0x7fb75c108f10 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.333+0000 7fb753400700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7400b2910 0x7fb7400b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.334+0000 7fb753400700 1 -- 172.21.15.90:0/723219624 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fb7480e39c0 con 0x7fb75c108f10 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.334+0000 7fb75aa00700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7400b2910 0x7fb7400b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.334+0000 7fb75aa00700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7400b2910 0x7fb7400b4e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb74c005fd0 tx=0x7fb74c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.337+0000 7fb753400700 1 -- 172.21.15.90:0/723219624 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb74805fdf0 con 0x7fb75c108f10 2024-05-27T07:44:14.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.582+0000 7fb761a00700 1 -- 172.21.15.90:0/723219624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb75c1a9bd0 con 0x7fb75c108f10 2024-05-27T07:44:14.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.583+0000 7fb753400700 1 -- 172.21.15.90:0/723219624 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb748013070 con 0x7fb75c108f10 2024-05-27T07:44:14.582 INFO:teuthology.orchestra.run.smithi090.stdout:261993005126 2024-05-27T07:44:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.586+0000 7fb761a00700 1 -- 172.21.15.90:0/723219624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7400b2910 msgr2=0x7fb7400b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.586+0000 7fb761a00700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7400b2910 0x7fb7400b4e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb74c005fd0 tx=0x7fb74c00a560 comp rx=0 tx=0).stop 2024-05-27T07:44:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.586+0000 7fb761a00700 1 -- 172.21.15.90:0/723219624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c108f10 msgr2=0x7fb75c19d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.586+0000 7fb761a00700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c108f10 0x7fb75c19d5d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb74800b580 tx=0x7fb74800f740 comp rx=0 tx=0).stop 2024-05-27T07:44:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.586+0000 7fb761a00700 1 -- 172.21.15.90:0/723219624 shutdown_connections 2024-05-27T07:44:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.586+0000 7fb761a00700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7400b2910 0x7fb7400b4e20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.586+0000 7fb761a00700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb75c109860 0x7fb75c1a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.586+0000 7fb761a00700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb75c1045e0 0x7fb75c19d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.586+0000 7fb761a00700 1 --2- 172.21.15.90:0/723219624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c108f10 0x7fb75c19d5d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.587+0000 7fb761a00700 1 -- 172.21.15.90:0/723219624 >> 172.21.15.90:0/723219624 conn(0x7fb75c100030 msgr2=0x7fb75c10a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.587+0000 7fb761a00700 1 -- 172.21.15.90:0/723219624 shutdown_connections 2024-05-27T07:44:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.587+0000 7fb761a00700 1 -- 172.21.15.90:0/723219624 wait complete. 2024-05-27T07:44:14.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.671+0000 7fd541e00700 1 -- 172.21.15.90:0/3077058698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd53c02e5e0 con 0x7fd53c115170 2024-05-27T07:44:14.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.674+0000 7fd539600700 1 -- 172.21.15.90:0/3077058698 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd5340ab840 con 0x7fd53c115170 2024-05-27T07:44:14.673 INFO:teuthology.orchestra.run.smithi090.stdout:163208757348 2024-05-27T07:44:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.677+0000 7fd541e00700 1 -- 172.21.15.90:0/3077058698 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd5240b2990 msgr2=0x7fd5240b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.677+0000 7fd541e00700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd5240b2990 0x7fd5240b4ea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd530005d90 tx=0x7fd530005d00 comp rx=0 tx=0).stop 2024-05-27T07:44:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.677+0000 7fd541e00700 1 -- 172.21.15.90:0/3077058698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c115170 msgr2=0x7fd53c1a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.677+0000 7fd541e00700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c115170 0x7fd53c1a5970 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd53400b070 tx=0x7fd534004d30 comp rx=0 tx=0).stop 2024-05-27T07:44:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.677+0000 7fd541e00700 1 -- 172.21.15.90:0/3077058698 shutdown_connections 2024-05-27T07:44:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.677+0000 7fd541e00700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd5240b2990 0x7fd5240b4ea0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.677+0000 7fd541e00700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd53c115170 0x7fd53c1a5970 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.677+0000 7fd541e00700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd53c075b50 0x7fd53c1a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.677+0000 7fd541e00700 1 --2- 172.21.15.90:0/3077058698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53c0764a0 0x7fd53c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.677+0000 7fd541e00700 1 -- 172.21.15.90:0/3077058698 >> 172.21.15.90:0/3077058698 conn(0x7fd53c079d00 msgr2=0x7fd53c100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.678+0000 7fd541e00700 1 -- 172.21.15.90:0/3077058698 shutdown_connections 2024-05-27T07:44:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:14.678+0000 7fd541e00700 1 -- 172.21.15.90:0/3077058698 wait complete. 2024-05-27T07:44:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:14 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/723219624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T07:44:14.946 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117100 got 115964117107 for osd.3 2024-05-27T07:44:14.946 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:14.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:14 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/723219624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T07:44:15.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:14 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/723219624' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T07:44:15.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:15 smithi090 ceph-mon[31429]: pgmap v479: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:15.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:15 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3077058698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T07:44:15.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:15 smithi136 ceph-mon[35324]: pgmap v479: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:15.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:15 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3077058698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T07:44:16.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:15 smithi129 ceph-mon[35254]: pgmap v479: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:16.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:15 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3077058698' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T07:44:17.923 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:17.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:17 smithi090 ceph-mon[31429]: pgmap v480: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:17.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:17 smithi136 ceph-mon[35324]: pgmap v480: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:18.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.050+0000 7f3705600700 1 -- 172.21.15.90:0/1828254326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3700108f10 msgr2=0x7f3700109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.050+0000 7f3705600700 1 --2- 172.21.15.90:0/1828254326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3700108f10 0x7f3700109320 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0099c0 tx=0x7f36ec009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.051+0000 7f3705600700 1 -- 172.21.15.90:0/1828254326 shutdown_connections 2024-05-27T07:44:18.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.051+0000 7f3705600700 1 --2- 172.21.15.90:0/1828254326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3700109860 0x7f37001111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.051+0000 7f3705600700 1 --2- 172.21.15.90:0/1828254326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001045e0 0x7f37001049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.051+0000 7f3705600700 1 --2- 172.21.15.90:0/1828254326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3700108f10 0x7f3700109320 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.051+0000 7f3705600700 1 -- 172.21.15.90:0/1828254326 >> 172.21.15.90:0/1828254326 conn(0x7f3700100030 msgr2=0x7f3700102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:18.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.051+0000 7f3705600700 1 -- 172.21.15.90:0/1828254326 shutdown_connections 2024-05-27T07:44:18.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.051+0000 7f3705600700 1 -- 172.21.15.90:0/1828254326 wait complete. 2024-05-27T07:44:18.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.052+0000 7f3705600700 1 Processor -- start 2024-05-27T07:44:18.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.052+0000 7f3705600700 1 -- start start 2024-05-27T07:44:18.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f3705600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001045e0 0x7f370019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f3705600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3700108f10 0x7f370019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f3705600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3700109860 0x7f37001a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f3705600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3700114370 con 0x7f3700109860 2024-05-27T07:44:18.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f3705600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f37001141f0 con 0x7f37001045e0 2024-05-27T07:44:18.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f3705600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f37001144f0 con 0x7f3700108f10 2024-05-27T07:44:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f36ff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3700109860 0x7f37001a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f36ff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3700109860 0x7f37001a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47826/0 (socket says 172.21.15.90:47826) 2024-05-27T07:44:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f36ff400700 1 -- 172.21.15.90:0/1685511281 learned_addr learned my addr 172.21.15.90:0/1685511281 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f36fea00700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001045e0 0x7f370019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.053+0000 7f36fe000700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3700108f10 0x7f370019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f36ff400700 1 -- 172.21.15.90:0/1685511281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3700108f10 msgr2=0x7f370019d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f36ff400700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3700108f10 0x7f370019d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f36ff400700 1 -- 172.21.15.90:0/1685511281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001045e0 msgr2=0x7f370019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f36ff400700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001045e0 0x7f370019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f36ff400700 1 -- 172.21.15.90:0/1685511281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37001a1e50 con 0x7f3700109860 2024-05-27T07:44:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f36fe000700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3700108f10 0x7f370019d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f36fea00700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001045e0 0x7f370019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f36ff400700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3700109860 0x7f37001a17b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f36f8007ee0 tx=0x7f36f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f36f7400700 1 -- 172.21.15.90:0/1685511281 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f800dbd0 con 0x7f3700109860 2024-05-27T07:44:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f36f7400700 1 -- 172.21.15.90:0/1685511281 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f36f800dd70 con 0x7f3700109860 2024-05-27T07:44:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.054+0000 7f3705600700 1 -- 172.21.15.90:0/1685511281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37001a2140 con 0x7f3700109860 2024-05-27T07:44:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.055+0000 7f36f7400700 1 -- 172.21.15.90:0/1685511281 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f80058d0 con 0x7f3700109860 2024-05-27T07:44:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.055+0000 7f3705600700 1 -- 172.21.15.90:0/1685511281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37001a9a60 con 0x7f3700109860 2024-05-27T07:44:18.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.728+0000 7ff8bb200700 1 -- 172.21.15.90:0/3863141770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8bc109670 msgr2=0x7ff8bc110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.728+0000 7ff8bb200700 1 --2- 172.21.15.90:0/3863141770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8bc109670 0x7ff8bc110fd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400b0d0 tx=0x7ff8b400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.728+0000 7ff8bb200700 1 -- 172.21.15.90:0/3863141770 shutdown_connections 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.728+0000 7ff8bb200700 1 --2- 172.21.15.90:0/3863141770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8bc109670 0x7ff8bc110fd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.728+0000 7ff8bb200700 1 --2- 172.21.15.90:0/3863141770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8bc1043f0 0x7ff8bc104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.728+0000 7ff8bb200700 1 --2- 172.21.15.90:0/3863141770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8bc108d20 0x7ff8bc109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.728+0000 7ff8bb200700 1 -- 172.21.15.90:0/3863141770 >> 172.21.15.90:0/3863141770 conn(0x7ff8bc079d00 msgr2=0x7ff8bc07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.728+0000 7ff8bb200700 1 -- 172.21.15.90:0/3863141770 shutdown_connections 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.728+0000 7ff8bb200700 1 -- 172.21.15.90:0/3863141770 wait complete. 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.729+0000 7ff8bb200700 1 Processor -- start 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.730+0000 7ff8bb200700 1 -- start start 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.730+0000 7ff8bb200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8bc1043f0 0x7ff8bc075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.730+0000 7ff8bb200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8bc108d20 0x7ff8bc075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.730+0000 7ff8bb200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8bc109670 0x7ff8bc076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.730+0000 7ff8bb200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc1141a0 con 0x7ff8bc109670 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.730+0000 7ff8bb200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc114020 con 0x7ff8bc108d20 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.730+0000 7ff8bb200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc114320 con 0x7ff8bc1043f0 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.731+0000 7ff8ba800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8bc109670 0x7ff8bc076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.731+0000 7ff8b9e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8bc1043f0 0x7ff8bc075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.731+0000 7ff8ba800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8bc109670 0x7ff8bc076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47802/0 (socket says 172.21.15.90:47802) 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.731+0000 7ff8ba800700 1 -- 172.21.15.90:0/3781929952 learned_addr learned my addr 172.21.15.90:0/3781929952 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.731+0000 7ff8b9400700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8bc108d20 0x7ff8bc075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.731+0000 7ff8ba800700 1 -- 172.21.15.90:0/3781929952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8bc1043f0 msgr2=0x7ff8bc075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.731+0000 7ff8ba800700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8bc1043f0 0x7ff8bc075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.731+0000 7ff8ba800700 1 -- 172.21.15.90:0/3781929952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8bc108d20 msgr2=0x7ff8bc075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.731+0000 7ff8ba800700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8bc108d20 0x7ff8bc075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.731+0000 7ff8ba800700 1 -- 172.21.15.90:0/3781929952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc1a8f20 con 0x7ff8bc109670 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.732+0000 7ff8ba800700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8bc109670 0x7ff8bc076530 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4007d10 tx=0x7ff8b4007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.732+0000 7ff8b2a00700 1 -- 172.21.15.90:0/3781929952 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b401f070 con 0x7ff8bc109670 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.732+0000 7ff8b2a00700 1 -- 172.21.15.90:0/3781929952 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8b4003e10 con 0x7ff8bc109670 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.732+0000 7ff8bb200700 1 -- 172.21.15.90:0/3781929952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8bc1a91b0 con 0x7ff8bc109670 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.732+0000 7ff8b2a00700 1 -- 172.21.15.90:0/3781929952 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b401b720 con 0x7ff8bc109670 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.732+0000 7ff8bb200700 1 -- 172.21.15.90:0/3781929952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8bc1a9660 con 0x7ff8bc109670 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.733+0000 7ff8bb200700 1 -- 172.21.15.90:0/3781929952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8bc0512b0 con 0x7ff8bc109670 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.736+0000 7ff8b2a00700 1 -- 172.21.15.90:0/3781929952 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff8b4022070 con 0x7ff8bc109670 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.738+0000 7ff8b2a00700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8a00b2990 0x7ff8a00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.739+0000 7ff8b2a00700 1 -- 172.21.15.90:0/3781929952 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7ff8b400f020 con 0x7ff8bc109670 2024-05-27T07:44:18.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.739+0000 7ff8b9e00700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8a00b2990 0x7ff8a00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.739+0000 7ff8b9e00700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8a00b2990 0x7ff8a00b4ea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005950 tx=0x7ff8ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:18.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.743+0000 7ff8b2a00700 1 -- 172.21.15.90:0/3781929952 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8b4029020 con 0x7ff8bc109670 2024-05-27T07:44:18.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.995+0000 7ff8bb200700 1 -- 172.21.15.90:0/3781929952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff8bc02e7a0 con 0x7ff8bc109670 2024-05-27T07:44:18.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.996+0000 7ff8b2a00700 1 -- 172.21.15.90:0/3781929952 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff8b40ab350 con 0x7ff8bc109670 2024-05-27T07:44:18.059 INFO:teuthology.orchestra.run.smithi090.stdout:309237645368 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 -- 172.21.15.90:0/3781929952 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8a00b2990 msgr2=0x7ff8a00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8a00b2990 0x7ff8a00b4ea0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac005950 tx=0x7ff8ac00b410 comp rx=0 tx=0).stop 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 -- 172.21.15.90:0/3781929952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8bc109670 msgr2=0x7ff8bc076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8bc109670 0x7ff8bc076530 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4007d10 tx=0x7ff8b4007d40 comp rx=0 tx=0).stop 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 -- 172.21.15.90:0/3781929952 shutdown_connections 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8a00b2990 0x7ff8a00b4ea0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8bc109670 0x7ff8bc076530 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8bc1043f0 0x7ff8bc075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 --2- 172.21.15.90:0/3781929952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8bc108d20 0x7ff8bc075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 -- 172.21.15.90:0/3781929952 >> 172.21.15.90:0/3781929952 conn(0x7ff8bc079d00 msgr2=0x7ff8bc101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 -- 172.21.15.90:0/3781929952 shutdown_connections 2024-05-27T07:44:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:17.999+0000 7ff8bb200700 1 -- 172.21.15.90:0/3781929952 wait complete. 2024-05-27T07:44:18.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.056+0000 7f3705600700 1 -- 172.21.15.90:0/1685511281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3700105a90 con 0x7f3700109860 2024-05-27T07:44:18.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.059+0000 7f36f7400700 1 -- 172.21.15.90:0/1685511281 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f36f801a020 con 0x7f3700109860 2024-05-27T07:44:18.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.061+0000 7f36f7400700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f36e40b2910 0x7f36e40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.062+0000 7f36f7400700 1 -- 172.21.15.90:0/1685511281 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f36f80e2550 con 0x7f3700109860 2024-05-27T07:44:18.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.062+0000 7f36fea00700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f36e40b2910 0x7f36e40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.063+0000 7f36fea00700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f36e40b2910 0x7f36e40b4e20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f36f0005950 tx=0x7f36f00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:18.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:17 smithi129 ceph-mon[35254]: pgmap v480: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:18.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.063+0000 7f36f7400700 1 -- 172.21.15.90:0/1685511281 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36f8010070 con 0x7f3700109860 2024-05-27T07:44:18.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.303+0000 7f3705600700 1 -- 172.21.15.90:0/1685511281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f37001a9c30 con 0x7f3700109860 2024-05-27T07:44:18.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.303+0000 7f36f7400700 1 -- 172.21.15.90:0/1685511281 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f36f80a9d80 con 0x7f3700109860 2024-05-27T07:44:18.302 INFO:teuthology.orchestra.run.smithi090.stdout:55834574978 2024-05-27T07:44:18.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.307+0000 7f3705600700 1 -- 172.21.15.90:0/1685511281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f36e40b2910 msgr2=0x7f36e40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.307+0000 7f3705600700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f36e40b2910 0x7f36e40b4e20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f36f0005950 tx=0x7f36f00058e0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.307+0000 7f3705600700 1 -- 172.21.15.90:0/1685511281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3700109860 msgr2=0x7f37001a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.307+0000 7f3705600700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3700109860 0x7f37001a17b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f36f8007ee0 tx=0x7f36f800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.308+0000 7f3705600700 1 -- 172.21.15.90:0/1685511281 shutdown_connections 2024-05-27T07:44:18.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.308+0000 7f3705600700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f36e40b2910 0x7f36e40b4e20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.308+0000 7f3705600700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3700109860 0x7f37001a17b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.308+0000 7f3705600700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37001045e0 0x7f370019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.308+0000 7f3705600700 1 --2- 172.21.15.90:0/1685511281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3700108f10 0x7f370019d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.308+0000 7f3705600700 1 -- 172.21.15.90:0/1685511281 >> 172.21.15.90:0/1685511281 conn(0x7f3700100030 msgr2=0x7f370010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:18.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.308+0000 7f3705600700 1 -- 172.21.15.90:0/1685511281 shutdown_connections 2024-05-27T07:44:18.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.308+0000 7f3705600700 1 -- 172.21.15.90:0/1685511281 wait complete. 2024-05-27T07:44:18.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.657+0000 7fc09b400700 1 -- 172.21.15.90:0/558511582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0941045e0 msgr2=0x7fc0941049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.657+0000 7fc09b400700 1 --2- 172.21.15.90:0/558511582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0941045e0 0x7fc0941049f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc088009920 tx=0x7fc088009c30 comp rx=0 tx=0).stop 2024-05-27T07:44:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.657+0000 7fc09b400700 1 -- 172.21.15.90:0/558511582 shutdown_connections 2024-05-27T07:44:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.657+0000 7fc09b400700 1 --2- 172.21.15.90:0/558511582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc094109860 0x7fc0941111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.657+0000 7fc09b400700 1 --2- 172.21.15.90:0/558511582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0941045e0 0x7fc0941049f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.657+0000 7fc09b400700 1 --2- 172.21.15.90:0/558511582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc094108f10 0x7fc094109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.657+0000 7fc09b400700 1 -- 172.21.15.90:0/558511582 >> 172.21.15.90:0/558511582 conn(0x7fc094100030 msgr2=0x7fc094102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.657+0000 7fc09b400700 1 -- 172.21.15.90:0/558511582 shutdown_connections 2024-05-27T07:44:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.657+0000 7fc09b400700 1 -- 172.21.15.90:0/558511582 wait complete. 2024-05-27T07:44:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.658+0000 7fc09b400700 1 Processor -- start 2024-05-27T07:44:18.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.658+0000 7fc09b400700 1 -- start start 2024-05-27T07:44:18.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.659+0000 7fc09b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0941045e0 0x7fc0941a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.659+0000 7fc09b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc094108f10 0x7fc0941a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.659+0000 7fc09b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc094109860 0x7fc0941ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.659+0000 7fc09b400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc094114340 con 0x7fc094109860 2024-05-27T07:44:18.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.659+0000 7fc09b400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc0941141c0 con 0x7fc094108f10 2024-05-27T07:44:18.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.659+0000 7fc09b400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc0941144c0 con 0x7fc0941045e0 2024-05-27T07:44:18.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.659+0000 7fc093e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc094108f10 0x7fc0941a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.659+0000 7fc098a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0941045e0 0x7fc0941a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.659+0000 7fc093e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc094108f10 0x7fc0941a2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:57404/0 (socket says 172.21.15.90:57404) 2024-05-27T07:44:18.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.659+0000 7fc093e00700 1 -- 172.21.15.90:0/2241755029 learned_addr learned my addr 172.21.15.90:0/2241755029 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.660+0000 7fc099400700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc094109860 0x7fc0941ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.660+0000 7fc093e00700 1 -- 172.21.15.90:0/2241755029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0941045e0 msgr2=0x7fc0941a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.660+0000 7fc093e00700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0941045e0 0x7fc0941a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.660+0000 7fc093e00700 1 -- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc094109860 msgr2=0x7fc0941ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.660+0000 7fc093e00700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc094109860 0x7fc0941ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.660+0000 7fc093e00700 1 -- 172.21.15.90:0/2241755029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0941ad5e0 con 0x7fc094108f10 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.660+0000 7fc098a00700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0941045e0 0x7fc0941a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.660+0000 7fc099400700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc094109860 0x7fc0941ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.660+0000 7fc093e00700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc094108f10 0x7fc0941a2730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc08000d980 tx=0x7fc08000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.661+0000 7fc091600700 1 -- 172.21.15.90:0/2241755029 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080017070 con 0x7fc094108f10 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.661+0000 7fc091600700 1 -- 172.21.15.90:0/2241755029 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc080009dc0 con 0x7fc094108f10 2024-05-27T07:44:18.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.661+0000 7fc09b400700 1 -- 172.21.15.90:0/2241755029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0941ad870 con 0x7fc094108f10 2024-05-27T07:44:18.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.661+0000 7fc091600700 1 -- 172.21.15.90:0/2241755029 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08000f3c0 con 0x7fc094108f10 2024-05-27T07:44:18.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.661+0000 7fc09b400700 1 -- 172.21.15.90:0/2241755029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0941addb0 con 0x7fc094108f10 2024-05-27T07:44:18.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.662+0000 7fc09b400700 1 -- 172.21.15.90:0/2241755029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0940512b0 con 0x7fc094108f10 2024-05-27T07:44:18.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.665+0000 7fc091600700 1 -- 172.21.15.90:0/2241755029 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc080009930 con 0x7fc094108f10 2024-05-27T07:44:18.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.667+0000 7fc091600700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc07c0b2910 0x7fc07c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:18.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.667+0000 7fc091600700 1 -- 172.21.15.90:0/2241755029 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc0800e22a0 con 0x7fc094108f10 2024-05-27T07:44:18.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.667+0000 7fc098a00700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc07c0b2910 0x7fc07c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:18.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.668+0000 7fc098a00700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc07c0b2910 0x7fc07c0b4e20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc088005f40 tx=0x7fc088019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:18.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.672+0000 7fc091600700 1 -- 172.21.15.90:0/2241755029 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0800a9a70 con 0x7fc094108f10 2024-05-27T07:44:18.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.916+0000 7fc09b400700 1 -- 172.21.15.90:0/2241755029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc0941ae050 con 0x7fc094108f10 2024-05-27T07:44:18.917 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:18 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3781929952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T07:44:18.917 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:18 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1685511281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T07:44:18.917 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:18 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:44:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:34359738506 2024-05-27T07:44:18.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.917+0000 7fc091600700 1 -- 172.21.15.90:0/2241755029 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc0800a9370 con 0x7fc094108f10 2024-05-27T07:44:18.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 -- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc07c0b2910 msgr2=0x7fc07c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc07c0b2910 0x7fc07c0b4e20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc088005f40 tx=0x7fc088019040 comp rx=0 tx=0).stop 2024-05-27T07:44:18.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 -- 172.21.15.90:0/2241755029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc094108f10 msgr2=0x7fc0941a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:18.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc094108f10 0x7fc0941a2730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc08000d980 tx=0x7fc08000dc90 comp rx=0 tx=0).stop 2024-05-27T07:44:18.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 -- 172.21.15.90:0/2241755029 shutdown_connections 2024-05-27T07:44:18.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc07c0b2910 0x7fc07c0b4e20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc094109860 0x7fc0941ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0941045e0 0x7fc0941a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 --2- 172.21.15.90:0/2241755029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc094108f10 0x7fc0941a2730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:18.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 -- 172.21.15.90:0/2241755029 >> 172.21.15.90:0/2241755029 conn(0x7fc094100030 msgr2=0x7fc0941087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:18.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 -- 172.21.15.90:0/2241755029 shutdown_connections 2024-05-27T07:44:18.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:18.921+0000 7fc09b400700 1 -- 172.21.15.90:0/2241755029 wait complete. 2024-05-27T07:44:18.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:18 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3781929952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T07:44:18.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:18 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1685511281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T07:44:18.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:18 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:44:19.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:18 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3781929952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T07:44:19.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:18 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1685511281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T07:44:19.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:18 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:44:19.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.256+0000 7f1163600700 1 -- 172.21.15.90:0/726337142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075ad0 msgr2=0x7f115c075ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.256+0000 7f1163600700 1 --2- 172.21.15.90:0/726337142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075ad0 0x7f115c075ee0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f114c00b0a0 tx=0x7f114c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.256+0000 7f1163600700 1 -- 172.21.15.90:0/726337142 shutdown_connections 2024-05-27T07:44:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.256+0000 7f1163600700 1 --2- 172.21.15.90:0/726337142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f115c110c10 0x7f115c113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.256+0000 7f1163600700 1 --2- 172.21.15.90:0/726337142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c075ad0 0x7f115c075ee0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.256+0000 7f1163600700 1 --2- 172.21.15.90:0/726337142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c076420 0x7f115c1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.256+0000 7f1163600700 1 -- 172.21.15.90:0/726337142 >> 172.21.15.90:0/726337142 conn(0x7f115c100030 msgr2=0x7f115c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.256+0000 7f1163600700 1 -- 172.21.15.90:0/726337142 shutdown_connections 2024-05-27T07:44:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.256+0000 7f1163600700 1 -- 172.21.15.90:0/726337142 wait complete. 2024-05-27T07:44:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.257+0000 7f1163600700 1 Processor -- start 2024-05-27T07:44:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.257+0000 7f1163600700 1 -- start start 2024-05-27T07:44:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1163600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c075ad0 0x7f115c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1163600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f115c076420 0x7f115c19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1163600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c110c10 0x7f115c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1163600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f115c116540 con 0x7f115c110c10 2024-05-27T07:44:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1163600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1163c0 con 0x7f115c075ad0 2024-05-27T07:44:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1163600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1166c0 con 0x7f115c076420 2024-05-27T07:44:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1161600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c110c10 0x7f115c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1161600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c110c10 0x7f115c1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57850/0 (socket says 172.21.15.90:57850) 2024-05-27T07:44:19.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1161600700 1 -- 172.21.15.90:0/1485884655 learned_addr learned my addr 172.21.15.90:0/1485884655 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:19.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1160c00700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c075ad0 0x7f115c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:19.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.258+0000 7f1153e00700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f115c076420 0x7f115c19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:19.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f1161600700 1 -- 172.21.15.90:0/1485884655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f115c076420 msgr2=0x7f115c19d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:19.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f1161600700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f115c076420 0x7f115c19d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f1161600700 1 -- 172.21.15.90:0/1485884655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c075ad0 msgr2=0x7f115c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:19.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f1161600700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c075ad0 0x7f115c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f1161600700 1 -- 172.21.15.90:0/1485884655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115c1a1e20 con 0x7f115c110c10 2024-05-27T07:44:19.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f1161600700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c110c10 0x7f115c1a1780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f115400d980 tx=0x7f115400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:19.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f115a000700 1 -- 172.21.15.90:0/1485884655 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154017070 con 0x7f115c110c10 2024-05-27T07:44:19.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f115a000700 1 -- 172.21.15.90:0/1485884655 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1154005020 con 0x7f115c110c10 2024-05-27T07:44:19.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f1163600700 1 -- 172.21.15.90:0/1485884655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115c1a20b0 con 0x7f115c110c10 2024-05-27T07:44:19.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f115a000700 1 -- 172.21.15.90:0/1485884655 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115400f980 con 0x7f115c110c10 2024-05-27T07:44:19.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.259+0000 7f1163600700 1 -- 172.21.15.90:0/1485884655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115c1a25a0 con 0x7f115c110c10 2024-05-27T07:44:19.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.261+0000 7f1163600700 1 -- 172.21.15.90:0/1485884655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f115c0512b0 con 0x7f115c110c10 2024-05-27T07:44:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.264+0000 7f115a000700 1 -- 172.21.15.90:0/1485884655 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1154004a30 con 0x7f115c110c10 2024-05-27T07:44:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.266+0000 7f115a000700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f11400b2910 0x7f11400b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.267+0000 7f115a000700 1 -- 172.21.15.90:0/1485884655 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f11540e2910 con 0x7f115c110c10 2024-05-27T07:44:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.267+0000 7f1160c00700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f11400b2910 0x7f11400b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.267+0000 7f1160c00700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f11400b2910 0x7f11400b4e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f114c009900 tx=0x7f114c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:19.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.268+0000 7f115a000700 1 -- 172.21.15.90:0/1485884655 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1154020020 con 0x7f115c110c10 2024-05-27T07:44:19.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.512+0000 7f1163600700 1 -- 172.21.15.90:0/1485884655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f115c02e5e0 con 0x7f115c110c10 2024-05-27T07:44:19.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.513+0000 7f115a000700 1 -- 172.21.15.90:0/1485884655 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f11540aa0e0 con 0x7f115c110c10 2024-05-27T07:44:19.512 INFO:teuthology.orchestra.run.smithi090.stdout:236223201358 2024-05-27T07:44:19.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.515+0000 7f1163600700 1 -- 172.21.15.90:0/1485884655 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f11400b2910 msgr2=0x7f11400b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:19.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.515+0000 7f1163600700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f11400b2910 0x7f11400b4e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f114c009900 tx=0x7f114c00b7b0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.515+0000 7f1163600700 1 -- 172.21.15.90:0/1485884655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c110c10 msgr2=0x7f115c1a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:19.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.515+0000 7f1163600700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c110c10 0x7f115c1a1780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f115400d980 tx=0x7f115400dc90 comp rx=0 tx=0).stop 2024-05-27T07:44:19.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.515+0000 7f1163600700 1 -- 172.21.15.90:0/1485884655 shutdown_connections 2024-05-27T07:44:19.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.515+0000 7f1163600700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f11400b2910 0x7f11400b4e20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.515+0000 7f1163600700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f115c110c10 0x7f115c1a1780 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.515+0000 7f1163600700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c075ad0 0x7f115c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.515+0000 7f1163600700 1 --2- 172.21.15.90:0/1485884655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f115c076420 0x7f115c19d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:19.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.516+0000 7f1163600700 1 -- 172.21.15.90:0/1485884655 >> 172.21.15.90:0/1485884655 conn(0x7f115c100030 msgr2=0x7f115c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:19.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.516+0000 7f1163600700 1 -- 172.21.15.90:0/1485884655 shutdown_connections 2024-05-27T07:44:19.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:19.516+0000 7f1163600700 1 -- 172.21.15.90:0/1485884655 wait complete. 2024-05-27T07:44:19.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:19 smithi090 ceph-mon[31429]: pgmap v481: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:44:19.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:19 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2241755029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T07:44:19.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:19 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1485884655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T07:44:19.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:19 smithi136 ceph-mon[35324]: pgmap v481: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:44:19.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:19 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2241755029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T07:44:19.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:19 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1485884655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T07:44:20.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:19 smithi129 ceph-mon[35254]: pgmap v481: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:44:20.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:19 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2241755029' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T07:44:20.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:19 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1485884655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T07:44:21.053 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738500 got 34359738506 for osd.0 2024-05-27T07:44:21.053 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:21.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.146+0000 7fdd5a800700 1 -- 172.21.15.90:0/3124471933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd54108f10 msgr2=0x7fdd54109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:21.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.146+0000 7fdd5a800700 1 --2- 172.21.15.90:0/3124471933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd54108f10 0x7fdd54109320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdd400099c0 tx=0x7fdd40009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.146+0000 7fdd5a800700 1 -- 172.21.15.90:0/3124471933 shutdown_connections 2024-05-27T07:44:21.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.146+0000 7fdd5a800700 1 --2- 172.21.15.90:0/3124471933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd54109860 0x7fdd541111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.146+0000 7fdd5a800700 1 --2- 172.21.15.90:0/3124471933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd541045e0 0x7fdd541049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.146+0000 7fdd5a800700 1 --2- 172.21.15.90:0/3124471933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd54108f10 0x7fdd54109320 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.146+0000 7fdd5a800700 1 -- 172.21.15.90:0/3124471933 >> 172.21.15.90:0/3124471933 conn(0x7fdd54100030 msgr2=0x7fdd54102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:21.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.146+0000 7fdd5a800700 1 -- 172.21.15.90:0/3124471933 shutdown_connections 2024-05-27T07:44:21.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.146+0000 7fdd5a800700 1 -- 172.21.15.90:0/3124471933 wait complete. 2024-05-27T07:44:21.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.147+0000 7fdd5a800700 1 Processor -- start 2024-05-27T07:44:21.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.147+0000 7fdd5a800700 1 -- start start 2024-05-27T07:44:21.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.148+0000 7fdd5a800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd541045e0 0x7fdd5419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:21.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.148+0000 7fdd5a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd54108f10 0x7fdd5419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:21.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.148+0000 7fdd5a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd54109860 0x7fdd541a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:21.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.148+0000 7fdd5a800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54114370 con 0x7fdd54109860 2024-05-27T07:44:21.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.148+0000 7fdd5a800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541141f0 con 0x7fdd541045e0 2024-05-27T07:44:21.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.148+0000 7fdd5a800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541144f0 con 0x7fdd54108f10 2024-05-27T07:44:21.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.148+0000 7fdd53e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd54109860 0x7fdd541a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:21.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.148+0000 7fdd53e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd54109860 0x7fdd541a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57868/0 (socket says 172.21.15.90:57868) 2024-05-27T07:44:21.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.148+0000 7fdd53e00700 1 -- 172.21.15.90:0/3342642920 learned_addr learned my addr 172.21.15.90:0/3342642920 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:21.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.149+0000 7fdd53400700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd541045e0 0x7fdd5419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.149+0000 7fdd52a00700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd54108f10 0x7fdd5419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.149+0000 7fdd53e00700 1 -- 172.21.15.90:0/3342642920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd54108f10 msgr2=0x7fdd5419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.149+0000 7fdd53e00700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd54108f10 0x7fdd5419d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.149+0000 7fdd53e00700 1 -- 172.21.15.90:0/3342642920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd541045e0 msgr2=0x7fdd5419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.149+0000 7fdd53e00700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd541045e0 0x7fdd5419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.149+0000 7fdd53e00700 1 -- 172.21.15.90:0/3342642920 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd541a1e70 con 0x7fdd54109860 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.149+0000 7fdd53400700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd541045e0 0x7fdd5419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.149+0000 7fdd53e00700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd54109860 0x7fdd541a17d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c007ee0 tx=0x7fdd4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.150+0000 7fdd4be00700 1 -- 172.21.15.90:0/3342642920 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00dbd0 con 0x7fdd54109860 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.150+0000 7fdd4be00700 1 -- 172.21.15.90:0/3342642920 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd4c00dd70 con 0x7fdd54109860 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.150+0000 7fdd5a800700 1 -- 172.21.15.90:0/3342642920 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd541a2160 con 0x7fdd54109860 2024-05-27T07:44:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.150+0000 7fdd4be00700 1 -- 172.21.15.90:0/3342642920 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0058d0 con 0x7fdd54109860 2024-05-27T07:44:21.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.150+0000 7fdd5a800700 1 -- 172.21.15.90:0/3342642920 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd541a9a80 con 0x7fdd54109860 2024-05-27T07:44:21.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.151+0000 7fdd5a800700 1 -- 172.21.15.90:0/3342642920 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd34005330 con 0x7fdd54109860 2024-05-27T07:44:21.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.153+0000 7fdd4be00700 1 -- 172.21.15.90:0/3342642920 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdd4c01a020 con 0x7fdd54109860 2024-05-27T07:44:21.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.155+0000 7fdd4be00700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdd380b2910 0x7fdd380b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:21.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.155+0000 7fdd4be00700 1 -- 172.21.15.90:0/3342642920 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fdd4c0e1eb0 con 0x7fdd54109860 2024-05-27T07:44:21.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.155+0000 7fdd53400700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdd380b2910 0x7fdd380b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:21.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.156+0000 7fdd53400700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdd380b2910 0x7fdd380b4e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd44005950 tx=0x7fdd440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:21.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.159+0000 7fdd4be00700 1 -- 172.21.15.90:0/3342642920 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd4c010070 con 0x7fdd54109860 2024-05-27T07:44:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.406+0000 7fdd5a800700 1 -- 172.21.15.90:0/3342642920 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdd340058f0 con 0x7fdd54109860 2024-05-27T07:44:21.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.407+0000 7fdd4be00700 1 -- 172.21.15.90:0/3342642920 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdd4c0a9680 con 0x7fdd54109860 2024-05-27T07:44:21.406 INFO:teuthology.orchestra.run.smithi090.stdout:77309411452 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 -- 172.21.15.90:0/3342642920 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdd380b2910 msgr2=0x7fdd380b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdd380b2910 0x7fdd380b4e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd44005950 tx=0x7fdd440058e0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 -- 172.21.15.90:0/3342642920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd54109860 msgr2=0x7fdd541a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd54109860 0x7fdd541a17d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c007ee0 tx=0x7fdd4c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 -- 172.21.15.90:0/3342642920 shutdown_connections 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdd380b2910 0x7fdd380b4e20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd54109860 0x7fdd541a17d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd541045e0 0x7fdd5419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 --2- 172.21.15.90:0/3342642920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd54108f10 0x7fdd5419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 -- 172.21.15.90:0/3342642920 >> 172.21.15.90:0/3342642920 conn(0x7fdd54100030 msgr2=0x7fdd5410a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 -- 172.21.15.90:0/3342642920 shutdown_connections 2024-05-27T07:44:21.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:21.411+0000 7fdd5a800700 1 -- 172.21.15.90:0/3342642920 wait complete. 2024-05-27T07:44:21.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:21 smithi090 ceph-mon[31429]: pgmap v482: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:21.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:21 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3342642920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T07:44:21.970 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005121 got 261993005126 for osd.9 2024-05-27T07:44:21.971 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:21.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:21 smithi136 ceph-mon[35324]: pgmap v482: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:21.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:21 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3342642920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T07:44:22.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:21 smithi129 ceph-mon[35254]: pgmap v482: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:22.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:21 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3342642920' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T07:44:22.182 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757345 got 163208757348 for osd.5 2024-05-27T07:44:22.182 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.952+0000 7f3adfe00700 1 -- 172.21.15.90:0/3772739972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae01043f0 msgr2=0x7f3ae0104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.952+0000 7f3adfe00700 1 --2- 172.21.15.90:0/3772739972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae01043f0 0x7f3ae0104800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00099c0 tx=0x7f3ad0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.952+0000 7f3adfe00700 1 -- 172.21.15.90:0/3772739972 shutdown_connections 2024-05-27T07:44:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.952+0000 7f3adfe00700 1 --2- 172.21.15.90:0/3772739972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae0109670 0x7f3ae0110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.952+0000 7f3adfe00700 1 --2- 172.21.15.90:0/3772739972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae01043f0 0x7f3ae0104800 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.952+0000 7f3adfe00700 1 --2- 172.21.15.90:0/3772739972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0108d20 0x7f3ae0109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.952+0000 7f3adfe00700 1 -- 172.21.15.90:0/3772739972 >> 172.21.15.90:0/3772739972 conn(0x7f3ae0079d00 msgr2=0x7f3ae007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.952+0000 7f3adfe00700 1 -- 172.21.15.90:0/3772739972 shutdown_connections 2024-05-27T07:44:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.952+0000 7f3adfe00700 1 -- 172.21.15.90:0/3772739972 wait complete. 2024-05-27T07:44:22.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.953+0000 7f3adfe00700 1 Processor -- start 2024-05-27T07:44:22.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.953+0000 7f3adfe00700 1 -- start start 2024-05-27T07:44:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.954+0000 7f3adfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae01043f0 0x7f3ae01a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.954+0000 7f3adfe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0108d20 0x7f3ae01a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.954+0000 7f3adfe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae0109670 0x7f3ae019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.954+0000 7f3adfe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae01141c0 con 0x7f3ae01043f0 2024-05-27T07:44:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.954+0000 7f3adfe00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0114040 con 0x7f3ae0109670 2024-05-27T07:44:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.954+0000 7f3adfe00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0114340 con 0x7f3ae0108d20 2024-05-27T07:44:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.954+0000 7f3adea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae01043f0 0x7f3ae01a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:22.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.954+0000 7f3adea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae01043f0 0x7f3ae01a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57890/0 (socket says 172.21.15.90:57890) 2024-05-27T07:44:22.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.954+0000 7f3adea00700 1 -- 172.21.15.90:0/1743549240 learned_addr learned my addr 172.21.15.90:0/1743549240 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:22.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.954+0000 7f3ade000700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0108d20 0x7f3ae01a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:22.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3adf400700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae0109670 0x7f3ae019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:22.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3adea00700 1 -- 172.21.15.90:0/1743549240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0108d20 msgr2=0x7f3ae01a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:22.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3adea00700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0108d20 0x7f3ae01a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:22.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3adea00700 1 -- 172.21.15.90:0/1743549240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae0109670 msgr2=0x7f3ae019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:22.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3adea00700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae0109670 0x7f3ae019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:22.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3adea00700 1 -- 172.21.15.90:0/1743549240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae019b080 con 0x7f3ae01043f0 2024-05-27T07:44:22.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3adf400700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae0109670 0x7f3ae019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:22.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3adea00700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae01043f0 0x7f3ae01a0820 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000b580 tx=0x7f3ad000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:22.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3ae4a00700 1 -- 172.21.15.90:0/1743549240 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0026070 con 0x7f3ae01043f0 2024-05-27T07:44:22.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3ae4a00700 1 -- 172.21.15.90:0/1743549240 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ad00054c0 con 0x7f3ae01043f0 2024-05-27T07:44:22.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3ae4a00700 1 -- 172.21.15.90:0/1743549240 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0004c00 con 0x7f3ae01043f0 2024-05-27T07:44:22.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3adfe00700 1 -- 172.21.15.90:0/1743549240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae019b310 con 0x7f3ae01043f0 2024-05-27T07:44:22.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.955+0000 7f3adfe00700 1 -- 172.21.15.90:0/1743549240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae019b840 con 0x7f3ae01043f0 2024-05-27T07:44:22.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.957+0000 7f3adfe00700 1 -- 172.21.15.90:0/1743549240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae01058a0 con 0x7f3ae01043f0 2024-05-27T07:44:22.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.959+0000 7f3ae4a00700 1 -- 172.21.15.90:0/1743549240 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3ad0029070 con 0x7f3ae01043f0 2024-05-27T07:44:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.960+0000 7f3ae4a00700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3ac40b2990 0x7f3ac40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.961+0000 7f3ae4a00700 1 -- 172.21.15.90:0/1743549240 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f3ad00e30b0 con 0x7f3ae01043f0 2024-05-27T07:44:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.961+0000 7f3ade000700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3ac40b2990 0x7f3ac40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.962+0000 7f3ade000700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3ac40b2990 0x7f3ac40b4ea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3ae007a230 tx=0x7f3acc005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:22.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:22.967+0000 7f3ae4a00700 1 -- 172.21.15.90:0/1743549240 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ad0013070 con 0x7f3ae01043f0 2024-05-27T07:44:23.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.205+0000 7f3adfe00700 1 -- 172.21.15.90:0/1743549240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f3ae002e5e0 con 0x7f3ae01043f0 2024-05-27T07:44:23.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.206+0000 7f3ae4a00700 1 -- 172.21.15.90:0/1743549240 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3ad00aa970 con 0x7f3ae01043f0 2024-05-27T07:44:23.206 INFO:teuthology.orchestra.run.smithi090.stdout:283467841601 2024-05-27T07:44:23.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.209+0000 7f3adfe00700 1 -- 172.21.15.90:0/1743549240 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3ac40b2990 msgr2=0x7f3ac40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:23.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.209+0000 7f3adfe00700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3ac40b2990 0x7f3ac40b4ea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3ae007a230 tx=0x7f3acc005eb0 comp rx=0 tx=0).stop 2024-05-27T07:44:23.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.209+0000 7f3adfe00700 1 -- 172.21.15.90:0/1743549240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae01043f0 msgr2=0x7f3ae01a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:23.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.209+0000 7f3adfe00700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae01043f0 0x7f3ae01a0820 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000b580 tx=0x7f3ad000f740 comp rx=0 tx=0).stop 2024-05-27T07:44:23.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.209+0000 7f3adfe00700 1 -- 172.21.15.90:0/1743549240 shutdown_connections 2024-05-27T07:44:23.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.209+0000 7f3adfe00700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f3ac40b2990 0x7f3ac40b4ea0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:23.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.209+0000 7f3adfe00700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ae0109670 0x7f3ae019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:23.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.209+0000 7f3adfe00700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ae01043f0 0x7f3ae01a0820 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:23.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.209+0000 7f3adfe00700 1 --2- 172.21.15.90:0/1743549240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ae0108d20 0x7f3ae01a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:23.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.209+0000 7f3adfe00700 1 -- 172.21.15.90:0/1743549240 >> 172.21.15.90:0/1743549240 conn(0x7f3ae0079d00 msgr2=0x7f3ae0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:23.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.210+0000 7f3adfe00700 1 -- 172.21.15.90:0/1743549240 shutdown_connections 2024-05-27T07:44:23.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:23.210+0000 7f3adfe00700 1 -- 172.21.15.90:0/1743549240 wait complete. 2024-05-27T07:44:23.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:23 smithi090 ceph-mon[31429]: pgmap v483: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:44:23.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:23 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1743549240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T07:44:23.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:23 smithi136 ceph-mon[35324]: pgmap v483: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:44:23.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:23 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1743549240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T07:44:24.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:23 smithi129 ceph-mon[35254]: pgmap v483: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:44:24.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:23 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1743549240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T07:44:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.092+0000 7fe5eda00700 1 -- 172.21.15.90:0/3035340477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5e8075d20 msgr2=0x7fe5e8076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.092+0000 7fe5eda00700 1 --2- 172.21.15.90:0/3035340477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5e8075d20 0x7fe5e8076130 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009990 tx=0x7fe5d8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.092+0000 7fe5eda00700 1 -- 172.21.15.90:0/3035340477 shutdown_connections 2024-05-27T07:44:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.092+0000 7fe5eda00700 1 --2- 172.21.15.90:0/3035340477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e8110b90 0x7fe5e8112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.092+0000 7fe5eda00700 1 --2- 172.21.15.90:0/3035340477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5e8075d20 0x7fe5e8076130 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.092+0000 7fe5eda00700 1 --2- 172.21.15.90:0/3035340477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5e8076670 0x7fe5e8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.092+0000 7fe5eda00700 1 -- 172.21.15.90:0/3035340477 >> 172.21.15.90:0/3035340477 conn(0x7fe5e8100030 msgr2=0x7fe5e8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.092+0000 7fe5eda00700 1 -- 172.21.15.90:0/3035340477 shutdown_connections 2024-05-27T07:44:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.093+0000 7fe5eda00700 1 -- 172.21.15.90:0/3035340477 wait complete. 2024-05-27T07:44:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.094+0000 7fe5eda00700 1 Processor -- start 2024-05-27T07:44:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.094+0000 7fe5eda00700 1 -- start start 2024-05-27T07:44:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.094+0000 7fe5eda00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5e8075d20 0x7fe5e819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.094+0000 7fe5eda00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5e8076670 0x7fe5e819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.094+0000 7fe5eda00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e8110b90 0x7fe5e81a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.094+0000 7fe5eda00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e8116580 con 0x7fe5e8076670 2024-05-27T07:44:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.094+0000 7fe5eda00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e8116400 con 0x7fe5e8110b90 2024-05-27T07:44:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.094+0000 7fe5eda00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e8116700 con 0x7fe5e8075d20 2024-05-27T07:44:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.095+0000 7fe5e6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5e8076670 0x7fe5e819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.095+0000 7fe5e6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5e8076670 0x7fe5e819d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57896/0 (socket says 172.21.15.90:57896) 2024-05-27T07:44:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.095+0000 7fe5e6000700 1 -- 172.21.15.90:0/2596208075 learned_addr learned my addr 172.21.15.90:0/2596208075 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.095+0000 7fe5e6a00700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5e8075d20 0x7fe5e819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.095+0000 7fe5e7400700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e8110b90 0x7fe5e81a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.095+0000 7fe5e6000700 1 -- 172.21.15.90:0/2596208075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5e8075d20 msgr2=0x7fe5e819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.095+0000 7fe5e6000700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5e8075d20 0x7fe5e819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.095+0000 7fe5e6000700 1 -- 172.21.15.90:0/2596208075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e8110b90 msgr2=0x7fe5e81a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.095+0000 7fe5e6000700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e8110b90 0x7fe5e81a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.095+0000 7fe5e6000700 1 -- 172.21.15.90:0/2596208075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e81a1e20 con 0x7fe5e8076670 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.096+0000 7fe5e6000700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5e8076670 0x7fe5e819d4f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400d950 tx=0x7fe5d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.096+0000 7fe5df400700 1 -- 172.21.15.90:0/2596208075 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4016070 con 0x7fe5e8076670 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.096+0000 7fe5df400700 1 -- 172.21.15.90:0/2596208075 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5d4009dd0 con 0x7fe5e8076670 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.096+0000 7fe5eda00700 1 -- 172.21.15.90:0/2596208075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e81a2110 con 0x7fe5e8076670 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.096+0000 7fe5eda00700 1 -- 172.21.15.90:0/2596208075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e81a9a30 con 0x7fe5e8076670 2024-05-27T07:44:24.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.096+0000 7fe5df400700 1 -- 172.21.15.90:0/2596208075 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d400f3c0 con 0x7fe5e8076670 2024-05-27T07:44:24.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.098+0000 7fe5eda00700 1 -- 172.21.15.90:0/2596208075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c8005360 con 0x7fe5e8076670 2024-05-27T07:44:24.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.101+0000 7fe5df400700 1 -- 172.21.15.90:0/2596208075 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe5d4019070 con 0x7fe5e8076670 2024-05-27T07:44:24.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.102+0000 7fe5df400700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5cc0b2910 0x7fe5cc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:24.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.102+0000 7fe5df400700 1 -- 172.21.15.90:0/2596208075 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fe5d40e2610 con 0x7fe5e8076670 2024-05-27T07:44:24.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.103+0000 7fe5e6a00700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5cc0b2910 0x7fe5cc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:24.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.103+0000 7fe5e6a00700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5cc0b2910 0x7fe5cc0b4e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800b5c0 tx=0x7fe5d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:24.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.107+0000 7fe5df400700 1 -- 172.21.15.90:0/2596208075 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5d40a9de0 con 0x7fe5e8076670 2024-05-27T07:44:24.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.368+0000 7fe5eda00700 1 -- 172.21.15.90:0/2596208075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe5c8005920 con 0x7fe5e8076670 2024-05-27T07:44:24.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.371+0000 7fe5df400700 1 -- 172.21.15.90:0/2596208075 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe5d40a96e0 con 0x7fe5e8076670 2024-05-27T07:44:24.370 INFO:teuthology.orchestra.run.smithi090.stdout:141733920878 2024-05-27T07:44:24.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 -- 172.21.15.90:0/2596208075 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5cc0b2910 msgr2=0x7fe5cc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:24.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5cc0b2910 0x7fe5cc0b4e20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800b5c0 tx=0x7fe5d8005fd0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 -- 172.21.15.90:0/2596208075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5e8076670 msgr2=0x7fe5e819d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:24.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5e8076670 0x7fe5e819d4f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400d950 tx=0x7fe5d400dc60 comp rx=0 tx=0).stop 2024-05-27T07:44:24.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 -- 172.21.15.90:0/2596208075 shutdown_connections 2024-05-27T07:44:24.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5cc0b2910 0x7fe5cc0b4e20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5e8110b90 0x7fe5e81a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5e8075d20 0x7fe5e819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 --2- 172.21.15.90:0/2596208075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5e8076670 0x7fe5e819d4f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:24.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 -- 172.21.15.90:0/2596208075 >> 172.21.15.90:0/2596208075 conn(0x7fe5e8100030 msgr2=0x7fe5e8100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:24.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 -- 172.21.15.90:0/2596208075 shutdown_connections 2024-05-27T07:44:24.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:24.374+0000 7fe5eda00700 1 -- 172.21.15.90:0/2596208075 wait complete. 2024-05-27T07:44:24.631 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574973 got 55834574978 for osd.1 2024-05-27T07:44:24.631 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:24.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:24 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:44:24.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:24 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:44:24.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:24 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:44:24.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:24 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2596208075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T07:44:24.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:24 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:44:24.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:24 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:44:24.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:24 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:44:24.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:24 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2596208075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T07:44:25.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:24 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:44:25.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:24 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:44:25.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:24 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:44:25.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:24 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2596208075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T07:44:25.219 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201354 got 236223201358 for osd.8 2024-05-27T07:44:25.219 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:25.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:25 smithi136 ceph-mon[35324]: pgmap v484: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:26.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:25 smithi129 ceph-mon[35254]: pgmap v484: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:26.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:25 smithi090 ceph-mon[31429]: pgmap v484: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:26.249 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841596 got 283467841601 for osd.10 2024-05-27T07:44:26.249 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:26.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.327+0000 7fbde6000700 1 -- 172.21.15.90:0/3931982217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde0106570 msgr2=0x7fbde0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:26.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.327+0000 7fbde6000700 1 --2- 172.21.15.90:0/3931982217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde0106570 0x7fbde0106980 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009990 tx=0x7fbdcc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.327+0000 7fbde6000700 1 -- 172.21.15.90:0/3931982217 shutdown_connections 2024-05-27T07:44:26.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.327+0000 7fbde6000700 1 --2- 172.21.15.90:0/3931982217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbde0106ec0 0x7fbde0113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.327+0000 7fbde6000700 1 --2- 172.21.15.90:0/3931982217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde010c5b0 0x7fbde010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.327+0000 7fbde6000700 1 --2- 172.21.15.90:0/3931982217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde0106570 0x7fbde0106980 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.327+0000 7fbde6000700 1 -- 172.21.15.90:0/3931982217 >> 172.21.15.90:0/3931982217 conn(0x7fbde0102070 msgr2=0x7fbde01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:26.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.327+0000 7fbde6000700 1 -- 172.21.15.90:0/3931982217 shutdown_connections 2024-05-27T07:44:26.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.327+0000 7fbde6000700 1 -- 172.21.15.90:0/3931982217 wait complete. 2024-05-27T07:44:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.328+0000 7fbde6000700 1 Processor -- start 2024-05-27T07:44:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.328+0000 7fbde6000700 1 -- start start 2024-05-27T07:44:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.329+0000 7fbde6000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde0106570 0x7fbde019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.329+0000 7fbde6000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbde0106ec0 0x7fbde019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:26.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.329+0000 7fbde6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde010c5b0 0x7fbde01a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:26.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.329+0000 7fbde6000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0116540 con 0x7fbde010c5b0 2024-05-27T07:44:26.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.329+0000 7fbde6000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbde01163c0 con 0x7fbde0106ec0 2024-05-27T07:44:26.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.329+0000 7fbde6000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbde01166c0 con 0x7fbde0106570 2024-05-27T07:44:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.329+0000 7fbddfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde010c5b0 0x7fbde01a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.329+0000 7fbddfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde010c5b0 0x7fbde01a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57914/0 (socket says 172.21.15.90:57914) 2024-05-27T07:44:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.329+0000 7fbddfe00700 1 -- 172.21.15.90:0/672930998 learned_addr learned my addr 172.21.15.90:0/672930998 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:26.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.329+0000 7fbddf400700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde0106570 0x7fbde019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:26.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.330+0000 7fbddea00700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbde0106ec0 0x7fbde019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:26.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.330+0000 7fbddfe00700 1 -- 172.21.15.90:0/672930998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde0106570 msgr2=0x7fbde019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:26.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.330+0000 7fbddfe00700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde0106570 0x7fbde019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.330+0000 7fbddfe00700 1 -- 172.21.15.90:0/672930998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbde0106ec0 msgr2=0x7fbde019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:26.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.330+0000 7fbddfe00700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbde0106ec0 0x7fbde019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.330+0000 7fbddfe00700 1 -- 172.21.15.90:0/672930998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde01a1de0 con 0x7fbde010c5b0 2024-05-27T07:44:26.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.330+0000 7fbddf400700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde0106570 0x7fbde019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:26.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.330+0000 7fbddfe00700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde010c5b0 0x7fbde01a1740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8007ee0 tx=0x7fbdd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:26.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.330+0000 7fbdd7e00700 1 -- 172.21.15.90:0/672930998 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd800dbd0 con 0x7fbde010c5b0 2024-05-27T07:44:26.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.331+0000 7fbde6000700 1 -- 172.21.15.90:0/672930998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde01a20d0 con 0x7fbde010c5b0 2024-05-27T07:44:26.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.330+0000 7fbdd7e00700 1 -- 172.21.15.90:0/672930998 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbdd800dd70 con 0x7fbde010c5b0 2024-05-27T07:44:26.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.331+0000 7fbdd7e00700 1 -- 172.21.15.90:0/672930998 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80058d0 con 0x7fbde010c5b0 2024-05-27T07:44:26.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.331+0000 7fbde6000700 1 -- 172.21.15.90:0/672930998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde01a99f0 con 0x7fbde010c5b0 2024-05-27T07:44:26.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.332+0000 7fbde6000700 1 -- 172.21.15.90:0/672930998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde00512b0 con 0x7fbde010c5b0 2024-05-27T07:44:26.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.335+0000 7fbdd7e00700 1 -- 172.21.15.90:0/672930998 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbdd801a020 con 0x7fbde010c5b0 2024-05-27T07:44:26.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.337+0000 7fbdd7e00700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbdc40b2910 0x7fbdc40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:26.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.337+0000 7fbdd7e00700 1 -- 172.21.15.90:0/672930998 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fbdd80e2630 con 0x7fbde010c5b0 2024-05-27T07:44:26.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.338+0000 7fbddf400700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbdc40b2910 0x7fbdc40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:26.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.338+0000 7fbddf400700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbdc40b2910 0x7fbdc40b4e20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0005950 tx=0x7fbdd00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:26.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.339+0000 7fbdd7e00700 1 -- 172.21.15.90:0/672930998 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbdd8010070 con 0x7fbde010c5b0 2024-05-27T07:44:26.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.573+0000 7fbde6000700 1 -- 172.21.15.90:0/672930998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbde002e5e0 con 0x7fbde010c5b0 2024-05-27T07:44:26.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.573+0000 7fbdd7e00700 1 -- 172.21.15.90:0/672930998 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbdd80a9e60 con 0x7fbde010c5b0 2024-05-27T07:44:26.573 INFO:teuthology.orchestra.run.smithi090.stdout:188978561118 2024-05-27T07:44:26.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 -- 172.21.15.90:0/672930998 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbdc40b2910 msgr2=0x7fbdc40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:26.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbdc40b2910 0x7fbdc40b4e20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0005950 tx=0x7fbdd00058e0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 -- 172.21.15.90:0/672930998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde010c5b0 msgr2=0x7fbde01a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:26.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde010c5b0 0x7fbde01a1740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8007ee0 tx=0x7fbdd800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 -- 172.21.15.90:0/672930998 shutdown_connections 2024-05-27T07:44:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fbdc40b2910 0x7fbdc40b4e20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbde010c5b0 0x7fbde01a1740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde0106570 0x7fbde019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 --2- 172.21.15.90:0/672930998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbde0106ec0 0x7fbde019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 -- 172.21.15.90:0/672930998 >> 172.21.15.90:0/672930998 conn(0x7fbde0102070 msgr2=0x7fbde0104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 -- 172.21.15.90:0/672930998 shutdown_connections 2024-05-27T07:44:26.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:26.577+0000 7fbde6000700 1 -- 172.21.15.90:0/672930998 wait complete. 2024-05-27T07:44:26.745 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645363 got 309237645368 for osd.11 2024-05-27T07:44:26.745 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:26.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:26 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/672930998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T07:44:26.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:26 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/672930998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T07:44:27.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:26 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/672930998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T07:44:27.641 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561112 got 188978561118 for osd.6 2024-05-27T07:44:27.642 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:27.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:27 smithi090 ceph-mon[31429]: pgmap v485: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:27.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.984+0000 7f5ed6e00700 1 -- 172.21.15.90:0/3194437617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed01034d0 msgr2=0x7f5ed01038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:27.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.984+0000 7f5ed6e00700 1 --2- 172.21.15.90:0/3194437617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed01034d0 0x7f5ed01038e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00099c0 tx=0x7f5ec0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:27.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.985+0000 7f5ed6e00700 1 -- 172.21.15.90:0/3194437617 shutdown_connections 2024-05-27T07:44:27.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.985+0000 7f5ed6e00700 1 --2- 172.21.15.90:0/3194437617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed0108750 0x7f5ed01100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:27.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.985+0000 7f5ed6e00700 1 --2- 172.21.15.90:0/3194437617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed01034d0 0x7f5ed01038e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:27.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.985+0000 7f5ed6e00700 1 --2- 172.21.15.90:0/3194437617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed0107e00 0x7f5ed0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:27.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.985+0000 7f5ed6e00700 1 -- 172.21.15.90:0/3194437617 >> 172.21.15.90:0/3194437617 conn(0x7f5ed00fef20 msgr2=0x7f5ed0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:27.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.985+0000 7f5ed6e00700 1 -- 172.21.15.90:0/3194437617 shutdown_connections 2024-05-27T07:44:27.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.985+0000 7f5ed6e00700 1 -- 172.21.15.90:0/3194437617 wait complete. 2024-05-27T07:44:27.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.986+0000 7f5ed6e00700 1 Processor -- start 2024-05-27T07:44:27.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.986+0000 7f5ed6e00700 1 -- start start 2024-05-27T07:44:27.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.986+0000 7f5ed6e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed01034d0 0x7f5ed00ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:27.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.986+0000 7f5ed6e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed0107e00 0x7f5ed00ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:27.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.986+0000 7f5ed6e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed0108750 0x7f5ed00ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:27.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.986+0000 7f5ed6e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed0113280 con 0x7f5ed0108750 2024-05-27T07:44:27.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.986+0000 7f5ed6e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed0113100 con 0x7f5ed01034d0 2024-05-27T07:44:27.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.986+0000 7f5ed6e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed0113400 con 0x7f5ed0107e00 2024-05-27T07:44:27.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.987+0000 7f5ed4e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed0108750 0x7f5ed00ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:27.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.987+0000 7f5ed4e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed0108750 0x7f5ed00ffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57920/0 (socket says 172.21.15.90:57920) 2024-05-27T07:44:27.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.987+0000 7f5ed4e00700 1 -- 172.21.15.90:0/1151012367 learned_addr learned my addr 172.21.15.90:0/1151012367 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:27.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.987+0000 7f5ecfe00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed01034d0 0x7f5ed00ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:27.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.987+0000 7f5ecf400700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed0107e00 0x7f5ed00ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:27.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.988+0000 7f5ed4e00700 1 -- 172.21.15.90:0/1151012367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed0107e00 msgr2=0x7f5ed00ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:27.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.988+0000 7f5ed4e00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed0107e00 0x7f5ed00ff940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:27.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.988+0000 7f5ed4e00700 1 -- 172.21.15.90:0/1151012367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed01034d0 msgr2=0x7f5ed00ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:27.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.988+0000 7f5ed4e00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed01034d0 0x7f5ed00ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:27.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.988+0000 7f5ed4e00700 1 -- 172.21.15.90:0/1151012367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed01a91e0 con 0x7f5ed0108750 2024-05-27T07:44:27.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.988+0000 7f5ed4e00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed0108750 0x7f5ed00ffe80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800ada0 tx=0x7f5ec800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:27.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.988+0000 7f5eccc00700 1 -- 172.21.15.90:0/1151012367 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8017020 con 0x7f5ed0108750 2024-05-27T07:44:27.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.988+0000 7f5eccc00700 1 -- 172.21.15.90:0/1151012367 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ec800cd60 con 0x7f5ed0108750 2024-05-27T07:44:27.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.988+0000 7f5eccc00700 1 -- 172.21.15.90:0/1151012367 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8005280 con 0x7f5ed0108750 2024-05-27T07:44:27.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.988+0000 7f5ed6e00700 1 -- 172.21.15.90:0/1151012367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed01a94d0 con 0x7f5ed0108750 2024-05-27T07:44:27.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.989+0000 7f5ed6e00700 1 -- 172.21.15.90:0/1151012367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed01a9a10 con 0x7f5ed0108750 2024-05-27T07:44:27.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.990+0000 7f5ed6e00700 1 -- 172.21.15.90:0/1151012367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed00512b0 con 0x7f5ed0108750 2024-05-27T07:44:27.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.993+0000 7f5eccc00700 1 -- 172.21.15.90:0/1151012367 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5ec800c8d0 con 0x7f5ed0108750 2024-05-27T07:44:27.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.994+0000 7f5eccc00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5eb40b2990 0x7f5eb40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.995+0000 7f5eccc00700 1 -- 172.21.15.90:0/1151012367 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f5ec80e23c0 con 0x7f5ed0108750 2024-05-27T07:44:27.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.995+0000 7f5ecfe00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5eb40b2990 0x7f5eb40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:27.995 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411446 got 77309411452 for osd.2 2024-05-27T07:44:27.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.996+0000 7f5ecfe00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5eb40b2990 0x7f5eb40b4ea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0000c00 tx=0x7f5ec0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:27.996 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:27.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:27.998+0000 7f5eccc00700 1 -- 172.21.15.90:0/1151012367 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ec8021020 con 0x7f5ed0108750 2024-05-27T07:44:27.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:27 smithi136 ceph-mon[35324]: pgmap v485: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:28.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:27 smithi129 ceph-mon[35254]: pgmap v485: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:28.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.230+0000 7f5ed6e00700 1 -- 172.21.15.90:0/1151012367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5ed0100930 con 0x7f5ed0108750 2024-05-27T07:44:28.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.230+0000 7f5eccc00700 1 -- 172.21.15.90:0/1151012367 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5ec80a9c20 con 0x7f5ed0108750 2024-05-27T07:44:28.231 INFO:teuthology.orchestra.run.smithi090.stdout:210453397592 2024-05-27T07:44:28.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.233+0000 7f5ed6e00700 1 -- 172.21.15.90:0/1151012367 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5eb40b2990 msgr2=0x7f5eb40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:28.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.233+0000 7f5ed6e00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5eb40b2990 0x7f5eb40b4ea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0000c00 tx=0x7f5ec0005fb0 comp rx=0 tx=0).stop 2024-05-27T07:44:28.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.233+0000 7f5ed6e00700 1 -- 172.21.15.90:0/1151012367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed0108750 msgr2=0x7f5ed00ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:28.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.233+0000 7f5ed6e00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed0108750 0x7f5ed00ffe80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800ada0 tx=0x7f5ec800c330 comp rx=0 tx=0).stop 2024-05-27T07:44:28.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.233+0000 7f5ed6e00700 1 -- 172.21.15.90:0/1151012367 shutdown_connections 2024-05-27T07:44:28.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.233+0000 7f5ed6e00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5eb40b2990 0x7f5eb40b4ea0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:28.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.233+0000 7f5ed6e00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ed0108750 0x7f5ed00ffe80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:28.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.233+0000 7f5ed6e00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ed01034d0 0x7f5ed00ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:28.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.233+0000 7f5ed6e00700 1 --2- 172.21.15.90:0/1151012367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed0107e00 0x7f5ed00ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:28.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.233+0000 7f5ed6e00700 1 -- 172.21.15.90:0/1151012367 >> 172.21.15.90:0/1151012367 conn(0x7f5ed00fef20 msgr2=0x7f5ed006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:28.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.234+0000 7f5ed6e00700 1 -- 172.21.15.90:0/1151012367 shutdown_connections 2024-05-27T07:44:28.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:28.234+0000 7f5ed6e00700 1 -- 172.21.15.90:0/1151012367 wait complete. 2024-05-27T07:44:28.466 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920873 got 141733920878 for osd.4 2024-05-27T07:44:28.466 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:29.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:28 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1151012367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T07:44:29.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:28 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1151012367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T07:44:29.196 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397584 got 210453397592 for osd.7 2024-05-27T07:44:29.196 DEBUG:teuthology.parallel:result is None 2024-05-27T07:44:29.196 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T07:44:29.197 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T07:44:29.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:28 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1151012367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T07:44:29.642 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:30.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:29 smithi090 ceph-mon[31429]: pgmap v486: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:44:30.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:29 smithi090 ceph-mon[31429]: mgrmap e18: x(active, since 13m), standbys: z, y 2024-05-27T07:44:30.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:29 smithi129 ceph-mon[35254]: pgmap v486: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:44:30.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:29 smithi129 ceph-mon[35254]: mgrmap e18: x(active, since 13m), standbys: z, y 2024-05-27T07:44:30.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:29 smithi136 ceph-mon[35324]: pgmap v486: 129 pgs: 129 active+clean; 649 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:44:30.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:29 smithi136 ceph-mon[35324]: mgrmap e18: x(active, since 13m), standbys: z, y 2024-05-27T07:44:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.819+0000 7fc189c00700 1 -- 172.21.15.90:0/1752060096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1841045e0 msgr2=0x7fc1841049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.819+0000 7fc189c00700 1 --2- 172.21.15.90:0/1752060096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1841045e0 0x7fc1841049f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc17400b0d0 tx=0x7fc17400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.819+0000 7fc189c00700 1 -- 172.21.15.90:0/1752060096 shutdown_connections 2024-05-27T07:44:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.819+0000 7fc189c00700 1 --2- 172.21.15.90:0/1752060096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184109860 0x7fc1841111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.819+0000 7fc189c00700 1 --2- 172.21.15.90:0/1752060096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1841045e0 0x7fc1841049f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.819+0000 7fc189c00700 1 --2- 172.21.15.90:0/1752060096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc184108f10 0x7fc184109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.819+0000 7fc189c00700 1 -- 172.21.15.90:0/1752060096 >> 172.21.15.90:0/1752060096 conn(0x7fc184100030 msgr2=0x7fc184102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.819+0000 7fc189c00700 1 -- 172.21.15.90:0/1752060096 shutdown_connections 2024-05-27T07:44:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.820+0000 7fc189c00700 1 -- 172.21.15.90:0/1752060096 wait complete. 2024-05-27T07:44:31.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.821+0000 7fc189c00700 1 Processor -- start 2024-05-27T07:44:31.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.821+0000 7fc189c00700 1 -- start start 2024-05-27T07:44:31.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.822+0000 7fc189c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1841045e0 0x7fc18419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.822+0000 7fc189c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc184108f10 0x7fc18419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.822+0000 7fc189c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184109860 0x7fc1841a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.822+0000 7fc189c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc184114340 con 0x7fc1841045e0 2024-05-27T07:44:31.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.822+0000 7fc189c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841141c0 con 0x7fc184108f10 2024-05-27T07:44:31.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.822+0000 7fc189c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841144c0 con 0x7fc184109860 2024-05-27T07:44:31.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.822+0000 7fc182a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1841045e0 0x7fc18419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.822+0000 7fc182a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1841045e0 0x7fc18419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38374/0 (socket says 172.21.15.90:38374) 2024-05-27T07:44:31.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.822+0000 7fc182a00700 1 -- 172.21.15.90:0/2311390090 learned_addr learned my addr 172.21.15.90:0/2311390090 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:31.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.822+0000 7fc17a000700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc184108f10 0x7fc18419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc183400700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184109860 0x7fc1841a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc182a00700 1 -- 172.21.15.90:0/2311390090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184109860 msgr2=0x7fc1841a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc182a00700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184109860 0x7fc1841a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc182a00700 1 -- 172.21.15.90:0/2311390090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc184108f10 msgr2=0x7fc18419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc182a00700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc184108f10 0x7fc18419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc182a00700 1 -- 172.21.15.90:0/2311390090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1841a1e90 con 0x7fc1841045e0 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc183400700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184109860 0x7fc1841a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc182a00700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1841045e0 0x7fc18419d010 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc174007d90 tx=0x7fc174007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc17be00700 1 -- 172.21.15.90:0/2311390090 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17401f030 con 0x7fc1841045e0 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc17be00700 1 -- 172.21.15.90:0/2311390090 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc17401bad0 con 0x7fc1841045e0 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc189c00700 1 -- 172.21.15.90:0/2311390090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1841a2120 con 0x7fc1841045e0 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.823+0000 7fc17be00700 1 -- 172.21.15.90:0/2311390090 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174012b50 con 0x7fc1841045e0 2024-05-27T07:44:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.824+0000 7fc189c00700 1 -- 172.21.15.90:0/2311390090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1841a9aa0 con 0x7fc1841045e0 2024-05-27T07:44:31.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.825+0000 7fc189c00700 1 -- 172.21.15.90:0/2311390090 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc184105a90 con 0x7fc1841045e0 2024-05-27T07:44:31.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.827+0000 7fc17be00700 1 -- 172.21.15.90:0/2311390090 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc174022030 con 0x7fc1841045e0 2024-05-27T07:44:31.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.829+0000 7fc17be00700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc1680b2900 0x7fc1680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:31.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:31 smithi090 ceph-mon[31429]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-27T07:44:31.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.829+0000 7fc17be00700 1 -- 172.21.15.90:0/2311390090 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc17400f020 con 0x7fc1841045e0 2024-05-27T07:44:31.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.829+0000 7fc17a000700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc1680b2900 0x7fc1680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:31.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.830+0000 7fc17a000700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc1680b2900 0x7fc1680b4e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc16c005950 tx=0x7fc16c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:31.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:31.835+0000 7fc17be00700 1 -- 172.21.15.90:0/2311390090 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc174029020 con 0x7fc1841045e0 2024-05-27T07:44:32.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.060+0000 7fc189c00700 1 -- 172.21.15.90:0/2311390090 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1840769f0 con 0x7fc1680b2900 2024-05-27T07:44:32.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:31 smithi129 ceph-mon[35254]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-27T07:44:32.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.086+0000 7fc17be00700 1 -- 172.21.15.90:0/2311390090 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340538 (secure 0 0 0) 0x7fc1840769f0 con 0x7fc1680b2900 2024-05-27T07:44:32.092 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 -- 172.21.15.90:0/2311390090 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc1680b2900 msgr2=0x7fc1680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc1680b2900 0x7fc1680b4e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc16c005950 tx=0x7fc16c0058e0 comp rx=0 tx=0).stop 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 -- 172.21.15.90:0/2311390090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1841045e0 msgr2=0x7fc18419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1841045e0 0x7fc18419d010 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc174007d90 tx=0x7fc174007e70 comp rx=0 tx=0).stop 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 -- 172.21.15.90:0/2311390090 shutdown_connections 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc1680b2900 0x7fc1680b4e10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184109860 0x7fc1841a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1841045e0 0x7fc18419d010 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 --2- 172.21.15.90:0/2311390090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc184108f10 0x7fc18419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 -- 172.21.15.90:0/2311390090 >> 172.21.15.90:0/2311390090 conn(0x7fc184100030 msgr2=0x7fc18410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:32.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 -- 172.21.15.90:0/2311390090 shutdown_connections 2024-05-27T07:44:32.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:32.094+0000 7fc189c00700 1 -- 172.21.15.90:0/2311390090 wait complete. 2024-05-27T07:44:32.096 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-05-27T07:44:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:31 smithi136 ceph-mon[35324]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-27T07:44:32.922 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-05-27T07:44:30.464922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665371,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1269,"num_read_kb":745,"num_write":384,"num_write_kb":2307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":479,"ondisk_log_size":479,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359468,"kb_used_data":14444,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582356,"statfs":{"total":1151940427776,"available":1151572332544,"internally_reserved":0,"allocated":14790656,"data_stored":8231705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19138,"internal_metadata":353154366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":69,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005497"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196314+0000","last_change":"2024-05-27T07:41:08.839265+0000","last_active":"2024-05-27T07:43:00.196314+0000","last_peered":"2024-05-27T07:43:00.196314+0000","last_clean":"2024-05-27T07:43:00.196314+0000","last_became_active":"2024-05-27T07:41:08.838814+0000","last_became_peered":"2024-05-27T07:41:08.838814+0000","last_unstale":"2024-05-27T07:43:00.196314+0000","last_undegraded":"2024-05-27T07:43:00.196314+0000","last_fullsized":"2024-05-27T07:43:00.196314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:32.901520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193574+0000","last_change":"2024-05-27T07:41:08.838313+0000","last_active":"2024-05-27T07:43:00.193574+0000","last_peered":"2024-05-27T07:43:00.193574+0000","last_clean":"2024-05-27T07:43:00.193574+0000","last_became_active":"2024-05-27T07:41:08.836657+0000","last_became_peered":"2024-05-27T07:41:08.836657+0000","last_unstale":"2024-05-27T07:43:00.193574+0000","last_undegraded":"2024-05-27T07:43:00.193574+0000","last_fullsized":"2024-05-27T07:43:00.193574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:33.556505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193301+0000","last_change":"2024-05-27T07:41:08.841513+0000","last_active":"2024-05-27T07:43:00.193301+0000","last_peered":"2024-05-27T07:43:00.193301+0000","last_clean":"2024-05-27T07:43:00.193301+0000","last_became_active":"2024-05-27T07:41:08.838624+0000","last_became_peered":"2024-05-27T07:41:08.838624+0000","last_unstale":"2024-05-27T07:43:00.193301+0000","last_undegraded":"2024-05-27T07:43:00.193301+0000","last_fullsized":"2024-05-27T07:43:00.193301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:01:37.320418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347168+0000","last_change":"2024-05-27T07:41:08.839138+0000","last_active":"2024-05-27T07:44:28.347168+0000","last_peered":"2024-05-27T07:44:28.347168+0000","last_clean":"2024-05-27T07:44:28.347168+0000","last_became_active":"2024-05-27T07:41:08.838766+0000","last_became_peered":"2024-05-27T07:41:08.838766+0000","last_unstale":"2024-05-27T07:44:28.347168+0000","last_undegraded":"2024-05-27T07:44:28.347168+0000","last_fullsized":"2024-05-27T07:44:28.347168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:09.129257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.346785+0000","last_change":"2024-05-27T07:41:08.823006+0000","last_active":"2024-05-27T07:44:28.346785+0000","last_peered":"2024-05-27T07:44:28.346785+0000","last_clean":"2024-05-27T07:44:28.346785+0000","last_became_active":"2024-05-27T07:41:08.822551+0000","last_became_peered":"2024-05-27T07:41:08.822551+0000","last_unstale":"2024-05-27T07:44:28.346785+0000","last_undegraded":"2024-05-27T07:44:28.346785+0000","last_fullsized":"2024-05-27T07:44:28.346785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:49.255763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.346127+0000","last_change":"2024-05-27T07:41:08.846072+0000","last_active":"2024-05-27T07:44:28.346127+0000","last_peered":"2024-05-27T07:44:28.346127+0000","last_clean":"2024-05-27T07:44:28.346127+0000","last_became_active":"2024-05-27T07:41:08.845604+0000","last_became_peered":"2024-05-27T07:41:08.845604+0000","last_unstale":"2024-05-27T07:44:28.346127+0000","last_undegraded":"2024-05-27T07:44:28.346127+0000","last_fullsized":"2024-05-27T07:44:28.346127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:23:56.389135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348891+0000","last_change":"2024-05-27T07:41:08.841046+0000","last_active":"2024-05-27T07:44:28.348891+0000","last_peered":"2024-05-27T07:44:28.348891+0000","last_clean":"2024-05-27T07:44:28.348891+0000","last_became_active":"2024-05-27T07:41:08.840028+0000","last_became_peered":"2024-05-27T07:41:08.840028+0000","last_unstale":"2024-05-27T07:44:28.348891+0000","last_undegraded":"2024-05-27T07:44:28.348891+0000","last_fullsized":"2024-05-27T07:44:28.348891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:35.239919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.116899+0000","last_change":"2024-05-27T07:41:08.841182+0000","last_active":"2024-05-27T07:43:00.116899+0000","last_peered":"2024-05-27T07:43:00.116899+0000","last_clean":"2024-05-27T07:43:00.116899+0000","last_became_active":"2024-05-27T07:41:08.840645+0000","last_became_peered":"2024-05-27T07:41:08.840645+0000","last_unstale":"2024-05-27T07:43:00.116899+0000","last_undegraded":"2024-05-27T07:43:00.116899+0000","last_fullsized":"2024-05-27T07:43:00.116899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:10:59.744505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.197003+0000","last_change":"2024-05-27T07:41:08.825874+0000","last_active":"2024-05-27T07:43:00.197003+0000","last_peered":"2024-05-27T07:43:00.197003+0000","last_clean":"2024-05-27T07:43:00.197003+0000","last_became_active":"2024-05-27T07:41:08.825386+0000","last_became_peered":"2024-05-27T07:41:08.825386+0000","last_unstale":"2024-05-27T07:43:00.197003+0000","last_undegraded":"2024-05-27T07:43:00.197003+0000","last_fullsized":"2024-05-27T07:43:00.197003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:13.304612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347910+0000","last_change":"2024-05-27T07:41:08.847445+0000","last_active":"2024-05-27T07:44:28.347910+0000","last_peered":"2024-05-27T07:44:28.347910+0000","last_clean":"2024-05-27T07:44:28.347910+0000","last_became_active":"2024-05-27T07:41:08.846512+0000","last_became_peered":"2024-05-27T07:41:08.846512+0000","last_unstale":"2024-05-27T07:44:28.347910+0000","last_undegraded":"2024-05-27T07:44:28.347910+0000","last_fullsized":"2024-05-27T07:44:28.347910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:57.235218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.362506+0000","last_change":"2024-05-27T07:41:08.843531+0000","last_active":"2024-05-27T07:44:28.362506+0000","last_peered":"2024-05-27T07:44:28.362506+0000","last_clean":"2024-05-27T07:44:28.362506+0000","last_became_active":"2024-05-27T07:41:08.843214+0000","last_became_peered":"2024-05-27T07:41:08.843214+0000","last_unstale":"2024-05-27T07:44:28.362506+0000","last_undegraded":"2024-05-27T07:44:28.362506+0000","last_fullsized":"2024-05-27T07:44:28.362506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:00.884178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.346497+0000","last_change":"2024-05-27T07:41:08.839487+0000","last_active":"2024-05-27T07:44:28.346497+0000","last_peered":"2024-05-27T07:44:28.346497+0000","last_clean":"2024-05-27T07:44:28.346497+0000","last_became_active":"2024-05-27T07:41:08.839097+0000","last_became_peered":"2024-05-27T07:41:08.839097+0000","last_unstale":"2024-05-27T07:44:28.346497+0000","last_undegraded":"2024-05-27T07:44:28.346497+0000","last_fullsized":"2024-05-27T07:44:28.346497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:51.055482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.017794+0000","last_change":"2024-05-27T07:41:08.836843+0000","last_active":"2024-05-27T07:43:00.017794+0000","last_peered":"2024-05-27T07:43:00.017794+0000","last_clean":"2024-05-27T07:43:00.017794+0000","last_became_active":"2024-05-27T07:41:08.836362+0000","last_became_peered":"2024-05-27T07:41:08.836362+0000","last_unstale":"2024-05-27T07:43:00.017794+0000","last_undegraded":"2024-05-27T07:43:00.017794+0000","last_fullsized":"2024-05-27T07:43:00.017794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:47.844301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.192414+0000","last_change":"2024-05-27T07:41:08.839046+0000","last_active":"2024-05-27T07:43:00.192414+0000","last_peered":"2024-05-27T07:43:00.192414+0000","last_clean":"2024-05-27T07:43:00.192414+0000","last_became_active":"2024-05-27T07:41:08.838601+0000","last_became_peered":"2024-05-27T07:41:08.838601+0000","last_unstale":"2024-05-27T07:43:00.192414+0000","last_undegraded":"2024-05-27T07:43:00.192414+0000","last_fullsized":"2024-05-27T07:43:00.192414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:59.563490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.192454+0000","last_change":"2024-05-27T07:41:08.832510+0000","last_active":"2024-05-27T07:43:00.192454+0000","last_peered":"2024-05-27T07:43:00.192454+0000","last_clean":"2024-05-27T07:43:00.192454+0000","last_became_active":"2024-05-27T07:41:08.832082+0000","last_became_peered":"2024-05-27T07:41:08.832082+0000","last_unstale":"2024-05-27T07:43:00.192454+0000","last_undegraded":"2024-05-27T07:43:00.192454+0000","last_fullsized":"2024-05-27T07:43:00.192454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:59.951860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193599+0000","last_change":"2024-05-27T07:41:08.827953+0000","last_active":"2024-05-27T07:43:00.193599+0000","last_peered":"2024-05-27T07:43:00.193599+0000","last_clean":"2024-05-27T07:43:00.193599+0000","last_became_active":"2024-05-27T07:41:08.827293+0000","last_became_peered":"2024-05-27T07:41:08.827293+0000","last_unstale":"2024-05-27T07:43:00.193599+0000","last_undegraded":"2024-05-27T07:43:00.193599+0000","last_fullsized":"2024-05-27T07:43:00.193599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:08.810514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195609+0000","last_change":"2024-05-27T07:41:08.839591+0000","last_active":"2024-05-27T07:43:00.195609+0000","last_peered":"2024-05-27T07:43:00.195609+0000","last_clean":"2024-05-27T07:43:00.195609+0000","last_became_active":"2024-05-27T07:41:08.839217+0000","last_became_peered":"2024-05-27T07:41:08.839217+0000","last_unstale":"2024-05-27T07:43:00.195609+0000","last_undegraded":"2024-05-27T07:43:00.195609+0000","last_fullsized":"2024-05-27T07:43:00.195609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:11:41.636163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.010889+0000","last_change":"2024-05-27T07:41:08.839544+0000","last_active":"2024-05-27T07:43:00.010889+0000","last_peered":"2024-05-27T07:43:00.010889+0000","last_clean":"2024-05-27T07:43:00.010889+0000","last_became_active":"2024-05-27T07:41:08.839177+0000","last_became_peered":"2024-05-27T07:41:08.839177+0000","last_unstale":"2024-05-27T07:43:00.010889+0000","last_undegraded":"2024-05-27T07:43:00.010889+0000","last_fullsized":"2024-05-27T07:43:00.010889+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:41.921759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013888+0000","last_change":"2024-05-27T07:41:08.847911+0000","last_active":"2024-05-27T07:43:00.013888+0000","last_peered":"2024-05-27T07:43:00.013888+0000","last_clean":"2024-05-27T07:43:00.013888+0000","last_became_active":"2024-05-27T07:41:08.847427+0000","last_became_peered":"2024-05-27T07:41:08.847427+0000","last_unstale":"2024-05-27T07:43:00.013888+0000","last_undegraded":"2024-05-27T07:43:00.013888+0000","last_fullsized":"2024-05-27T07:43:00.013888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:13.692744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012606+0000","last_change":"2024-05-27T07:41:21.999614+0000","last_active":"2024-05-27T07:43:00.012606+0000","last_peered":"2024-05-27T07:43:00.012606+0000","last_clean":"2024-05-27T07:43:00.012606+0000","last_became_active":"2024-05-27T07:41:21.999086+0000","last_became_peered":"2024-05-27T07:41:21.999086+0000","last_unstale":"2024-05-27T07:43:00.012606+0000","last_undegraded":"2024-05-27T07:43:00.012606+0000","last_fullsized":"2024-05-27T07:43:00.012606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:20.229318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":172,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:26.239859+0000","last_change":"2024-05-27T07:41:04.795079+0000","last_active":"2024-05-27T07:44:26.239859+0000","last_peered":"2024-05-27T07:44:26.239859+0000","last_clean":"2024-05-27T07:44:26.239859+0000","last_became_active":"2024-05-27T07:41:04.794367+0000","last_became_peered":"2024-05-27T07:41:04.794367+0000","last_unstale":"2024-05-27T07:44:26.239859+0000","last_undegraded":"2024-05-27T07:44:26.239859+0000","last_fullsized":"2024-05-27T07:44:26.239859+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:27:52.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.118644+0000","last_change":"2024-05-27T07:41:08.846712+0000","last_active":"2024-05-27T07:43:00.118644+0000","last_peered":"2024-05-27T07:43:00.118644+0000","last_clean":"2024-05-27T07:43:00.118644+0000","last_became_active":"2024-05-27T07:41:08.839999+0000","last_became_peered":"2024-05-27T07:41:08.839999+0000","last_unstale":"2024-05-27T07:43:00.118644+0000","last_undegraded":"2024-05-27T07:43:00.118644+0000","last_fullsized":"2024-05-27T07:43:00.118644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:31:23.924074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012118+0000","last_change":"2024-05-27T07:41:21.996675+0000","last_active":"2024-05-27T07:43:00.012118+0000","last_peered":"2024-05-27T07:43:00.012118+0000","last_clean":"2024-05-27T07:43:00.012118+0000","last_became_active":"2024-05-27T07:41:21.996152+0000","last_became_peered":"2024-05-27T07:41:21.996152+0000","last_unstale":"2024-05-27T07:43:00.012118+0000","last_undegraded":"2024-05-27T07:43:00.012118+0000","last_fullsized":"2024-05-27T07:43:00.012118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:57.415146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.119419+0000","last_change":"2024-05-27T07:41:04.890045+0000","last_active":"2024-05-27T07:43:00.119419+0000","last_peered":"2024-05-27T07:43:00.119419+0000","last_clean":"2024-05-27T07:43:00.119419+0000","last_became_active":"2024-05-27T07:41:04.888993+0000","last_became_peered":"2024-05-27T07:41:04.888993+0000","last_unstale":"2024-05-27T07:43:00.119419+0000","last_undegraded":"2024-05-27T07:43:00.119419+0000","last_fullsized":"2024-05-27T07:43:00.119419+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:19.866699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193609+0000","last_change":"2024-05-27T07:41:08.847472+0000","last_active":"2024-05-27T07:43:00.193609+0000","last_peered":"2024-05-27T07:43:00.193609+0000","last_clean":"2024-05-27T07:43:00.193609+0000","last_became_active":"2024-05-27T07:41:08.846070+0000","last_became_peered":"2024-05-27T07:41:08.846070+0000","last_unstale":"2024-05-27T07:43:00.193609+0000","last_undegraded":"2024-05-27T07:43:00.193609+0000","last_fullsized":"2024-05-27T07:43:00.193609+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:29:08.497007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012033+0000","last_change":"2024-05-27T07:41:59.842473+0000","last_active":"2024-05-27T07:43:00.012033+0000","last_peered":"2024-05-27T07:43:00.012033+0000","last_clean":"2024-05-27T07:43:00.012033+0000","last_became_active":"2024-05-27T07:41:59.841035+0000","last_became_peered":"2024-05-27T07:41:59.841035+0000","last_unstale":"2024-05-27T07:43:00.012033+0000","last_undegraded":"2024-05-27T07:43:00.012033+0000","last_fullsized":"2024-05-27T07:43:00.012033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:52.959589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.197322+0000","last_change":"2024-05-27T07:41:04.804046+0000","last_active":"2024-05-27T07:43:00.197322+0000","last_peered":"2024-05-27T07:43:00.197322+0000","last_clean":"2024-05-27T07:43:00.197322+0000","last_became_active":"2024-05-27T07:41:04.803360+0000","last_became_peered":"2024-05-27T07:41:04.803360+0000","last_unstale":"2024-05-27T07:43:00.197322+0000","last_undegraded":"2024-05-27T07:43:00.197322+0000","last_fullsized":"2024-05-27T07:43:00.197322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:17.325221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195942+0000","last_change":"2024-05-27T07:41:08.840842+0000","last_active":"2024-05-27T07:43:00.195942+0000","last_peered":"2024-05-27T07:43:00.195942+0000","last_clean":"2024-05-27T07:43:00.195942+0000","last_became_active":"2024-05-27T07:41:08.837817+0000","last_became_peered":"2024-05-27T07:41:08.837817+0000","last_unstale":"2024-05-27T07:43:00.195942+0000","last_undegraded":"2024-05-27T07:43:00.195942+0000","last_fullsized":"2024-05-27T07:43:00.195942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:53:38.130608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011841+0000","last_change":"2024-05-27T07:41:22.002988+0000","last_active":"2024-05-27T07:43:00.011841+0000","last_peered":"2024-05-27T07:43:00.011841+0000","last_clean":"2024-05-27T07:43:00.011841+0000","last_became_active":"2024-05-27T07:41:22.001141+0000","last_became_peered":"2024-05-27T07:41:22.001141+0000","last_unstale":"2024-05-27T07:43:00.011841+0000","last_undegraded":"2024-05-27T07:43:00.011841+0000","last_fullsized":"2024-05-27T07:43:00.011841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:29:17.380439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.197053+0000","last_change":"2024-05-27T07:41:04.794424+0000","last_active":"2024-05-27T07:43:00.197053+0000","last_peered":"2024-05-27T07:43:00.197053+0000","last_clean":"2024-05-27T07:43:00.197053+0000","last_became_active":"2024-05-27T07:41:04.793912+0000","last_became_peered":"2024-05-27T07:41:04.793912+0000","last_unstale":"2024-05-27T07:43:00.197053+0000","last_undegraded":"2024-05-27T07:43:00.197053+0000","last_fullsized":"2024-05-27T07:43:00.197053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:50.876891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.360086+0000","last_change":"2024-05-27T07:41:08.832356+0000","last_active":"2024-05-27T07:44:28.360086+0000","last_peered":"2024-05-27T07:44:28.360086+0000","last_clean":"2024-05-27T07:44:28.360086+0000","last_became_active":"2024-05-27T07:41:08.831979+0000","last_became_peered":"2024-05-27T07:41:08.831979+0000","last_unstale":"2024-05-27T07:44:28.360086+0000","last_undegraded":"2024-05-27T07:44:28.360086+0000","last_fullsized":"2024-05-27T07:44:28.360086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:37:47.584177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013124+0000","last_change":"2024-05-27T07:41:22.004233+0000","last_active":"2024-05-27T07:43:00.013124+0000","last_peered":"2024-05-27T07:43:00.013124+0000","last_clean":"2024-05-27T07:43:00.013124+0000","last_became_active":"2024-05-27T07:41:22.003781+0000","last_became_peered":"2024-05-27T07:41:22.003781+0000","last_unstale":"2024-05-27T07:43:00.013124+0000","last_undegraded":"2024-05-27T07:43:00.013124+0000","last_fullsized":"2024-05-27T07:43:00.013124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:02:13.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":171,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:29.271871+0000","last_change":"2024-05-27T07:41:04.800373+0000","last_active":"2024-05-27T07:44:29.271871+0000","last_peered":"2024-05-27T07:44:29.271871+0000","last_clean":"2024-05-27T07:44:29.271871+0000","last_became_active":"2024-05-27T07:41:04.799930+0000","last_became_peered":"2024-05-27T07:41:04.799930+0000","last_unstale":"2024-05-27T07:44:29.271871+0000","last_undegraded":"2024-05-27T07:44:29.271871+0000","last_fullsized":"2024-05-27T07:44:29.271871+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:24.642479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.346946+0000","last_change":"2024-05-27T07:41:08.830387+0000","last_active":"2024-05-27T07:44:28.346946+0000","last_peered":"2024-05-27T07:44:28.346946+0000","last_clean":"2024-05-27T07:44:28.346946+0000","last_became_active":"2024-05-27T07:41:08.829720+0000","last_became_peered":"2024-05-27T07:41:08.829720+0000","last_unstale":"2024-05-27T07:44:28.346946+0000","last_undegraded":"2024-05-27T07:44:28.346946+0000","last_fullsized":"2024-05-27T07:44:28.346946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:42.014383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013538+0000","last_change":"2024-05-27T07:41:22.005700+0000","last_active":"2024-05-27T07:43:00.013538+0000","last_peered":"2024-05-27T07:43:00.013538+0000","last_clean":"2024-05-27T07:43:00.013538+0000","last_became_active":"2024-05-27T07:41:22.003643+0000","last_became_peered":"2024-05-27T07:41:22.003643+0000","last_unstale":"2024-05-27T07:43:00.013538+0000","last_undegraded":"2024-05-27T07:43:00.013538+0000","last_fullsized":"2024-05-27T07:43:00.013538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:48.332767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.034703+0000","last_change":"2024-05-27T07:41:59.790579+0000","last_active":"2024-05-27T07:43:00.034703+0000","last_peered":"2024-05-27T07:43:00.034703+0000","last_clean":"2024-05-27T07:43:00.034703+0000","last_became_active":"2024-05-27T07:41:59.789849+0000","last_became_peered":"2024-05-27T07:41:59.789849+0000","last_unstale":"2024-05-27T07:43:00.034703+0000","last_undegraded":"2024-05-27T07:43:00.034703+0000","last_fullsized":"2024-05-27T07:43:00.034703+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:24:30.564074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.121892+0000","last_change":"2024-05-27T07:43:00.121063+0000","last_active":"2024-05-27T07:43:00.121892+0000","last_peered":"2024-05-27T07:43:00.121892+0000","last_clean":"2024-05-27T07:43:00.121892+0000","last_became_active":"2024-05-27T07:43:00.119977+0000","last_became_peered":"2024-05-27T07:43:00.119977+0000","last_unstale":"2024-05-27T07:43:00.121892+0000","last_undegraded":"2024-05-27T07:43:00.121892+0000","last_fullsized":"2024-05-27T07:43:00.121892+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:04:43.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195294+0000","last_change":"2024-05-27T07:41:22.013820+0000","last_active":"2024-05-27T07:43:00.195294+0000","last_peered":"2024-05-27T07:43:00.195294+0000","last_clean":"2024-05-27T07:43:00.195294+0000","last_became_active":"2024-05-27T07:41:22.013110+0000","last_became_peered":"2024-05-27T07:41:22.013110+0000","last_unstale":"2024-05-27T07:43:00.195294+0000","last_undegraded":"2024-05-27T07:43:00.195294+0000","last_fullsized":"2024-05-27T07:43:00.195294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:45:07.317872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018376+0000","last_change":"2024-05-27T07:41:04.789197+0000","last_active":"2024-05-27T07:43:00.018376+0000","last_peered":"2024-05-27T07:43:00.018376+0000","last_clean":"2024-05-27T07:43:00.018376+0000","last_became_active":"2024-05-27T07:41:04.788755+0000","last_became_peered":"2024-05-27T07:41:04.788755+0000","last_unstale":"2024-05-27T07:43:00.018376+0000","last_undegraded":"2024-05-27T07:43:00.018376+0000","last_fullsized":"2024-05-27T07:43:00.018376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:27:54.526910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.116931+0000","last_change":"2024-05-27T07:41:08.841769+0000","last_active":"2024-05-27T07:43:00.116931+0000","last_peered":"2024-05-27T07:43:00.116931+0000","last_clean":"2024-05-27T07:43:00.116931+0000","last_became_active":"2024-05-27T07:41:08.840834+0000","last_became_peered":"2024-05-27T07:41:08.840834+0000","last_unstale":"2024-05-27T07:43:00.116931+0000","last_undegraded":"2024-05-27T07:43:00.116931+0000","last_fullsized":"2024-05-27T07:43:00.116931+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:15.573739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193500+0000","last_change":"2024-05-27T07:41:22.005119+0000","last_active":"2024-05-27T07:43:00.193500+0000","last_peered":"2024-05-27T07:43:00.193500+0000","last_clean":"2024-05-27T07:43:00.193500+0000","last_became_active":"2024-05-27T07:41:22.002638+0000","last_became_peered":"2024-05-27T07:41:22.002638+0000","last_unstale":"2024-05-27T07:43:00.193500+0000","last_undegraded":"2024-05-27T07:43:00.193500+0000","last_fullsized":"2024-05-27T07:43:00.193500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:08:49.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041632+0000","last_change":"2024-05-27T07:41:04.799864+0000","last_active":"2024-05-27T07:43:00.041632+0000","last_peered":"2024-05-27T07:43:00.041632+0000","last_clean":"2024-05-27T07:43:00.041632+0000","last_became_active":"2024-05-27T07:41:04.799218+0000","last_became_peered":"2024-05-27T07:41:04.799218+0000","last_unstale":"2024-05-27T07:43:00.041632+0000","last_undegraded":"2024-05-27T07:43:00.041632+0000","last_fullsized":"2024-05-27T07:43:00.041632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:15.355720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.008905+0000","last_change":"2024-05-27T07:41:08.823387+0000","last_active":"2024-05-27T07:43:00.008905+0000","last_peered":"2024-05-27T07:43:00.008905+0000","last_clean":"2024-05-27T07:43:00.008905+0000","last_became_active":"2024-05-27T07:41:08.822943+0000","last_became_peered":"2024-05-27T07:41:08.822943+0000","last_unstale":"2024-05-27T07:43:00.008905+0000","last_undegraded":"2024-05-27T07:43:00.008905+0000","last_fullsized":"2024-05-27T07:43:00.008905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:46.365284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117402+0000","last_change":"2024-05-27T07:41:22.011848+0000","last_active":"2024-05-27T07:43:00.117402+0000","last_peered":"2024-05-27T07:43:00.117402+0000","last_clean":"2024-05-27T07:43:00.117402+0000","last_became_active":"2024-05-27T07:41:22.011335+0000","last_became_peered":"2024-05-27T07:41:22.011335+0000","last_unstale":"2024-05-27T07:43:00.117402+0000","last_undegraded":"2024-05-27T07:43:00.117402+0000","last_fullsized":"2024-05-27T07:43:00.117402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:29:22.528270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":280,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:29.270022+0000","last_change":"2024-05-27T07:41:04.793877+0000","last_active":"2024-05-27T07:44:29.270022+0000","last_peered":"2024-05-27T07:44:29.270022+0000","last_clean":"2024-05-27T07:44:29.270022+0000","last_became_active":"2024-05-27T07:41:04.793389+0000","last_became_peered":"2024-05-27T07:41:04.793389+0000","last_unstale":"2024-05-27T07:44:29.270022+0000","last_undegraded":"2024-05-27T07:44:29.270022+0000","last_fullsized":"2024-05-27T07:44:29.270022+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:48.670572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196815+0000","last_change":"2024-05-27T07:41:08.836682+0000","last_active":"2024-05-27T07:43:00.196815+0000","last_peered":"2024-05-27T07:43:00.196815+0000","last_clean":"2024-05-27T07:43:00.196815+0000","last_became_active":"2024-05-27T07:41:08.836229+0000","last_became_peered":"2024-05-27T07:41:08.836229+0000","last_unstale":"2024-05-27T07:43:00.196815+0000","last_undegraded":"2024-05-27T07:43:00.196815+0000","last_fullsized":"2024-05-27T07:43:00.196815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:33.249584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.008846+0000","last_change":"2024-05-27T07:41:22.004968+0000","last_active":"2024-05-27T07:43:00.008846+0000","last_peered":"2024-05-27T07:43:00.008846+0000","last_clean":"2024-05-27T07:43:00.008846+0000","last_became_active":"2024-05-27T07:41:22.001407+0000","last_became_peered":"2024-05-27T07:41:22.001407+0000","last_unstale":"2024-05-27T07:43:00.008846+0000","last_undegraded":"2024-05-27T07:43:00.008846+0000","last_fullsized":"2024-05-27T07:43:00.008846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:26.872497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195809+0000","last_change":"2024-05-27T07:41:59.787247+0000","last_active":"2024-05-27T07:43:00.195809+0000","last_peered":"2024-05-27T07:43:00.195809+0000","last_clean":"2024-05-27T07:43:00.195809+0000","last_became_active":"2024-05-27T07:41:59.786392+0000","last_became_peered":"2024-05-27T07:41:59.786392+0000","last_unstale":"2024-05-27T07:43:00.195809+0000","last_undegraded":"2024-05-27T07:43:00.195809+0000","last_fullsized":"2024-05-27T07:43:00.195809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:12.579707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2],"acting":[11,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193643+0000","last_change":"2024-05-27T07:41:08.833731+0000","last_active":"2024-05-27T07:43:00.193643+0000","last_peered":"2024-05-27T07:43:00.193643+0000","last_clean":"2024-05-27T07:43:00.193643+0000","last_became_active":"2024-05-27T07:41:08.833318+0000","last_became_peered":"2024-05-27T07:41:08.833318+0000","last_unstale":"2024-05-27T07:43:00.193643+0000","last_undegraded":"2024-05-27T07:43:00.193643+0000","last_fullsized":"2024-05-27T07:43:00.193643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:50.085113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012443+0000","last_change":"2024-05-27T07:41:59.789784+0000","last_active":"2024-05-27T07:43:00.012443+0000","last_peered":"2024-05-27T07:43:00.012443+0000","last_clean":"2024-05-27T07:43:00.012443+0000","last_became_active":"2024-05-27T07:41:59.788589+0000","last_became_peered":"2024-05-27T07:41:59.788589+0000","last_unstale":"2024-05-27T07:43:00.012443+0000","last_undegraded":"2024-05-27T07:43:00.012443+0000","last_fullsized":"2024-05-27T07:43:00.012443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:37.026907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042497+0000","last_change":"2024-05-27T07:41:04.892603+0000","last_active":"2024-05-27T07:43:00.042497+0000","last_peered":"2024-05-27T07:43:00.042497+0000","last_clean":"2024-05-27T07:43:00.042497+0000","last_became_active":"2024-05-27T07:41:04.890786+0000","last_became_peered":"2024-05-27T07:41:04.890786+0000","last_unstale":"2024-05-27T07:43:00.042497+0000","last_undegraded":"2024-05-27T07:43:00.042497+0000","last_fullsized":"2024-05-27T07:43:00.042497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:34.129227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196418+0000","last_change":"2024-05-27T07:41:08.830802+0000","last_active":"2024-05-27T07:43:00.196418+0000","last_peered":"2024-05-27T07:43:00.196418+0000","last_clean":"2024-05-27T07:43:00.196418+0000","last_became_active":"2024-05-27T07:41:08.829661+0000","last_became_peered":"2024-05-27T07:41:08.829661+0000","last_unstale":"2024-05-27T07:43:00.196418+0000","last_undegraded":"2024-05-27T07:43:00.196418+0000","last_fullsized":"2024-05-27T07:43:00.196418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:20:15.041554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009108+0000","last_change":"2024-05-27T07:41:22.003997+0000","last_active":"2024-05-27T07:43:00.009108+0000","last_peered":"2024-05-27T07:43:00.009108+0000","last_clean":"2024-05-27T07:43:00.009108+0000","last_became_active":"2024-05-27T07:41:22.003527+0000","last_became_peered":"2024-05-27T07:41:22.003527+0000","last_unstale":"2024-05-27T07:43:00.009108+0000","last_undegraded":"2024-05-27T07:43:00.009108+0000","last_fullsized":"2024-05-27T07:43:00.009108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:17.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":344,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:29.333985+0000","last_change":"2024-05-27T07:41:59.814227+0000","last_active":"2024-05-27T07:44:29.333985+0000","last_peered":"2024-05-27T07:44:29.333985+0000","last_clean":"2024-05-27T07:44:29.333985+0000","last_became_active":"2024-05-27T07:41:59.788510+0000","last_became_peered":"2024-05-27T07:41:59.788510+0000","last_unstale":"2024-05-27T07:44:29.333985+0000","last_undegraded":"2024-05-27T07:44:29.333985+0000","last_fullsized":"2024-05-27T07:44:29.333985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:15.868348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67283,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":187,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347183+0000","last_change":"2024-05-27T07:41:08.831639+0000","last_active":"2024-05-27T07:44:28.347183+0000","last_peered":"2024-05-27T07:44:28.347183+0000","last_clean":"2024-05-27T07:44:28.347183+0000","last_became_active":"2024-05-27T07:41:08.830541+0000","last_became_peered":"2024-05-27T07:41:08.830541+0000","last_unstale":"2024-05-27T07:44:28.347183+0000","last_undegraded":"2024-05-27T07:44:28.347183+0000","last_fullsized":"2024-05-27T07:44:28.347183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:45.635145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.043291+0000","last_change":"2024-05-27T07:41:21.999587+0000","last_active":"2024-05-27T07:43:00.043291+0000","last_peered":"2024-05-27T07:43:00.043291+0000","last_clean":"2024-05-27T07:43:00.043291+0000","last_became_active":"2024-05-27T07:41:21.999017+0000","last_became_peered":"2024-05-27T07:41:21.999017+0000","last_unstale":"2024-05-27T07:43:00.043291+0000","last_undegraded":"2024-05-27T07:43:00.043291+0000","last_fullsized":"2024-05-27T07:43:00.043291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:31:14.915229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013744+0000","last_change":"2024-05-27T07:41:04.799847+0000","last_active":"2024-05-27T07:43:00.013744+0000","last_peered":"2024-05-27T07:43:00.013744+0000","last_clean":"2024-05-27T07:43:00.013744+0000","last_became_active":"2024-05-27T07:41:04.798605+0000","last_became_peered":"2024-05-27T07:41:04.798605+0000","last_unstale":"2024-05-27T07:43:00.013744+0000","last_undegraded":"2024-05-27T07:43:00.013744+0000","last_fullsized":"2024-05-27T07:43:00.013744+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:01.957707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.017142+0000","last_change":"2024-05-27T07:41:08.831138+0000","last_active":"2024-05-27T07:43:00.017142+0000","last_peered":"2024-05-27T07:43:00.017142+0000","last_clean":"2024-05-27T07:43:00.017142+0000","last_became_active":"2024-05-27T07:41:08.830735+0000","last_became_peered":"2024-05-27T07:41:08.830735+0000","last_unstale":"2024-05-27T07:43:00.017142+0000","last_undegraded":"2024-05-27T07:43:00.017142+0000","last_fullsized":"2024-05-27T07:43:00.017142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:12.569687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194622+0000","last_change":"2024-05-27T07:41:59.779039+0000","last_active":"2024-05-27T07:43:00.194622+0000","last_peered":"2024-05-27T07:43:00.194622+0000","last_clean":"2024-05-27T07:43:00.194622+0000","last_became_active":"2024-05-27T07:41:59.777514+0000","last_became_peered":"2024-05-27T07:41:59.777514+0000","last_unstale":"2024-05-27T07:43:00.194622+0000","last_undegraded":"2024-05-27T07:43:00.194622+0000","last_fullsized":"2024-05-27T07:43:00.194622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:12:19.223874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042492+0000","last_change":"2024-05-27T07:41:04.796467+0000","last_active":"2024-05-27T07:43:00.042492+0000","last_peered":"2024-05-27T07:43:00.042492+0000","last_clean":"2024-05-27T07:43:00.042492+0000","last_became_active":"2024-05-27T07:41:04.795519+0000","last_became_peered":"2024-05-27T07:41:04.795519+0000","last_unstale":"2024-05-27T07:43:00.042492+0000","last_undegraded":"2024-05-27T07:43:00.042492+0000","last_fullsized":"2024-05-27T07:43:00.042492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:42.744076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.353662+0000","last_change":"2024-05-27T07:41:08.832135+0000","last_active":"2024-05-27T07:44:28.353662+0000","last_peered":"2024-05-27T07:44:28.353662+0000","last_clean":"2024-05-27T07:44:28.353662+0000","last_became_active":"2024-05-27T07:41:08.830032+0000","last_became_peered":"2024-05-27T07:41:08.830032+0000","last_unstale":"2024-05-27T07:44:28.353662+0000","last_undegraded":"2024-05-27T07:44:28.353662+0000","last_fullsized":"2024-05-27T07:44:28.353662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:44:59.367026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009932+0000","last_change":"2024-05-27T07:41:22.008868+0000","last_active":"2024-05-27T07:43:00.009932+0000","last_peered":"2024-05-27T07:43:00.009932+0000","last_clean":"2024-05-27T07:43:00.009932+0000","last_became_active":"2024-05-27T07:41:22.002071+0000","last_became_peered":"2024-05-27T07:41:22.002071+0000","last_unstale":"2024-05-27T07:43:00.009932+0000","last_undegraded":"2024-05-27T07:43:00.009932+0000","last_fullsized":"2024-05-27T07:43:00.009932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:20.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":171,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.323191+0000","last_change":"2024-05-27T07:41:04.796198+0000","last_active":"2024-05-27T07:44:28.323191+0000","last_peered":"2024-05-27T07:44:28.323191+0000","last_clean":"2024-05-27T07:44:28.323191+0000","last_became_active":"2024-05-27T07:41:04.795646+0000","last_became_peered":"2024-05-27T07:41:04.795646+0000","last_unstale":"2024-05-27T07:44:28.323191+0000","last_undegraded":"2024-05-27T07:44:28.323191+0000","last_fullsized":"2024-05-27T07:44:28.323191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:30.936596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348243+0000","last_change":"2024-05-27T07:41:08.840518+0000","last_active":"2024-05-27T07:44:28.348243+0000","last_peered":"2024-05-27T07:44:28.348243+0000","last_clean":"2024-05-27T07:44:28.348243+0000","last_became_active":"2024-05-27T07:41:08.840162+0000","last_became_peered":"2024-05-27T07:41:08.840162+0000","last_unstale":"2024-05-27T07:44:28.348243+0000","last_undegraded":"2024-05-27T07:44:28.348243+0000","last_fullsized":"2024-05-27T07:44:28.348243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:04:14.657496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018999+0000","last_change":"2024-05-27T07:41:22.000243+0000","last_active":"2024-05-27T07:43:00.018999+0000","last_peered":"2024-05-27T07:43:00.018999+0000","last_clean":"2024-05-27T07:43:00.018999+0000","last_became_active":"2024-05-27T07:41:21.998898+0000","last_became_peered":"2024-05-27T07:41:21.998898+0000","last_unstale":"2024-05-27T07:43:00.018999+0000","last_undegraded":"2024-05-27T07:43:00.018999+0000","last_fullsized":"2024-05-27T07:43:00.018999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:35.696593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":79,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195497+0000","last_change":"2024-05-27T07:41:59.799662+0000","last_active":"2024-05-27T07:43:00.195497+0000","last_peered":"2024-05-27T07:43:00.195497+0000","last_clean":"2024-05-27T07:43:00.195497+0000","last_became_active":"2024-05-27T07:41:59.779733+0000","last_became_peered":"2024-05-27T07:41:59.779733+0000","last_unstale":"2024-05-27T07:43:00.195497+0000","last_undegraded":"2024-05-27T07:43:00.195497+0000","last_fullsized":"2024-05-27T07:43:00.195497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:23.665014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.353774+0000","last_change":"2024-05-27T07:41:08.834931+0000","last_active":"2024-05-27T07:44:28.353774+0000","last_peered":"2024-05-27T07:44:28.353774+0000","last_clean":"2024-05-27T07:44:28.353774+0000","last_became_active":"2024-05-27T07:41:08.833298+0000","last_became_peered":"2024-05-27T07:41:08.833298+0000","last_unstale":"2024-05-27T07:44:28.353774+0000","last_undegraded":"2024-05-27T07:44:28.353774+0000","last_fullsized":"2024-05-27T07:44:28.353774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:03:48.507461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042243+0000","last_change":"2024-05-27T07:41:22.009407+0000","last_active":"2024-05-27T07:43:00.042243+0000","last_peered":"2024-05-27T07:43:00.042243+0000","last_clean":"2024-05-27T07:43:00.042243+0000","last_became_active":"2024-05-27T07:41:22.006921+0000","last_became_peered":"2024-05-27T07:41:22.006921+0000","last_unstale":"2024-05-27T07:43:00.042243+0000","last_undegraded":"2024-05-27T07:43:00.042243+0000","last_fullsized":"2024-05-27T07:43:00.042243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:24.034502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196609+0000","last_change":"2024-05-27T07:41:04.795641+0000","last_active":"2024-05-27T07:43:00.196609+0000","last_peered":"2024-05-27T07:43:00.196609+0000","last_clean":"2024-05-27T07:43:00.196609+0000","last_became_active":"2024-05-27T07:41:04.794935+0000","last_became_peered":"2024-05-27T07:41:04.794935+0000","last_unstale":"2024-05-27T07:43:00.196609+0000","last_undegraded":"2024-05-27T07:43:00.196609+0000","last_fullsized":"2024-05-27T07:43:00.196609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:14.850282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195301+0000","last_change":"2024-05-27T07:41:08.837277+0000","last_active":"2024-05-27T07:43:00.195301+0000","last_peered":"2024-05-27T07:43:00.195301+0000","last_clean":"2024-05-27T07:43:00.195301+0000","last_became_active":"2024-05-27T07:41:08.835935+0000","last_became_peered":"2024-05-27T07:41:08.835935+0000","last_unstale":"2024-05-27T07:43:00.195301+0000","last_undegraded":"2024-05-27T07:43:00.195301+0000","last_fullsized":"2024-05-27T07:43:00.195301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:46.436243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196250+0000","last_change":"2024-05-27T07:41:22.014700+0000","last_active":"2024-05-27T07:43:00.196250+0000","last_peered":"2024-05-27T07:43:00.196250+0000","last_clean":"2024-05-27T07:43:00.196250+0000","last_became_active":"2024-05-27T07:41:22.014136+0000","last_became_peered":"2024-05-27T07:41:22.014136+0000","last_unstale":"2024-05-27T07:43:00.196250+0000","last_undegraded":"2024-05-27T07:43:00.196250+0000","last_fullsized":"2024-05-27T07:43:00.196250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:51:08.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009402+0000","last_change":"2024-05-27T07:41:04.891269+0000","last_active":"2024-05-27T07:43:00.009402+0000","last_peered":"2024-05-27T07:43:00.009402+0000","last_clean":"2024-05-27T07:43:00.009402+0000","last_became_active":"2024-05-27T07:41:04.890843+0000","last_became_peered":"2024-05-27T07:41:04.890843+0000","last_unstale":"2024-05-27T07:43:00.009402+0000","last_undegraded":"2024-05-27T07:43:00.009402+0000","last_fullsized":"2024-05-27T07:43:00.009402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:57.065300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347426+0000","last_change":"2024-05-27T07:41:08.841977+0000","last_active":"2024-05-27T07:44:28.347426+0000","last_peered":"2024-05-27T07:44:28.347426+0000","last_clean":"2024-05-27T07:44:28.347426+0000","last_became_active":"2024-05-27T07:41:08.841551+0000","last_became_peered":"2024-05-27T07:41:08.841551+0000","last_unstale":"2024-05-27T07:44:28.347426+0000","last_undegraded":"2024-05-27T07:44:28.347426+0000","last_fullsized":"2024-05-27T07:44:28.347426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:35:07.766193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.020285+0000","last_change":"2024-05-27T07:41:22.006310+0000","last_active":"2024-05-27T07:43:00.020285+0000","last_peered":"2024-05-27T07:43:00.020285+0000","last_clean":"2024-05-27T07:43:00.020285+0000","last_became_active":"2024-05-27T07:41:22.004135+0000","last_became_peered":"2024-05-27T07:41:22.004135+0000","last_unstale":"2024-05-27T07:43:00.020285+0000","last_undegraded":"2024-05-27T07:43:00.020285+0000","last_fullsized":"2024-05-27T07:43:00.020285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:48.564894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.247040+0000","last_change":"2024-05-27T07:41:04.793284+0000","last_active":"2024-05-27T07:44:28.247040+0000","last_peered":"2024-05-27T07:44:28.247040+0000","last_clean":"2024-05-27T07:44:28.247040+0000","last_became_active":"2024-05-27T07:41:04.792853+0000","last_became_peered":"2024-05-27T07:41:04.792853+0000","last_unstale":"2024-05-27T07:44:28.247040+0000","last_undegraded":"2024-05-27T07:44:28.247040+0000","last_fullsized":"2024-05-27T07:44:28.247040+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:22.979341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.336734+0000","last_change":"2024-05-27T07:41:59.776207+0000","last_active":"2024-05-27T07:44:28.336734+0000","last_peered":"2024-05-27T07:44:28.336734+0000","last_clean":"2024-05-27T07:44:28.336734+0000","last_became_active":"2024-05-27T07:41:59.775226+0000","last_became_peered":"2024-05-27T07:41:59.775226+0000","last_unstale":"2024-05-27T07:44:28.336734+0000","last_undegraded":"2024-05-27T07:44:28.336734+0000","last_fullsized":"2024-05-27T07:44:28.336734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:50.469144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.021403+0000","last_change":"2024-05-27T07:41:21.992013+0000","last_active":"2024-05-27T07:43:00.021403+0000","last_peered":"2024-05-27T07:43:00.021403+0000","last_clean":"2024-05-27T07:43:00.021403+0000","last_became_active":"2024-05-27T07:41:21.991550+0000","last_became_peered":"2024-05-27T07:41:21.991550+0000","last_unstale":"2024-05-27T07:43:00.021403+0000","last_undegraded":"2024-05-27T07:43:00.021403+0000","last_fullsized":"2024-05-27T07:43:00.021403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:34:53.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347945+0000","last_change":"2024-05-27T07:41:08.832512+0000","last_active":"2024-05-27T07:44:28.347945+0000","last_peered":"2024-05-27T07:44:28.347945+0000","last_clean":"2024-05-27T07:44:28.347945+0000","last_became_active":"2024-05-27T07:41:08.832113+0000","last_became_peered":"2024-05-27T07:41:08.832113+0000","last_unstale":"2024-05-27T07:44:28.347945+0000","last_undegraded":"2024-05-27T07:44:28.347945+0000","last_fullsized":"2024-05-27T07:44:28.347945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:00.530785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":163,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:25.288408+0000","last_change":"2024-05-27T07:41:04.891004+0000","last_active":"2024-05-27T07:44:25.288408+0000","last_peered":"2024-05-27T07:44:25.288408+0000","last_clean":"2024-05-27T07:44:25.288408+0000","last_became_active":"2024-05-27T07:41:04.890567+0000","last_became_peered":"2024-05-27T07:41:04.890567+0000","last_unstale":"2024-05-27T07:44:25.288408+0000","last_undegraded":"2024-05-27T07:44:25.288408+0000","last_fullsized":"2024-05-27T07:44:25.288408+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:43.991204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196216+0000","last_change":"2024-05-27T07:41:22.003446+0000","last_active":"2024-05-27T07:43:00.196216+0000","last_peered":"2024-05-27T07:43:00.196216+0000","last_clean":"2024-05-27T07:43:00.196216+0000","last_became_active":"2024-05-27T07:41:22.002989+0000","last_became_peered":"2024-05-27T07:41:22.002989+0000","last_unstale":"2024-05-27T07:43:00.196216+0000","last_undegraded":"2024-05-27T07:43:00.196216+0000","last_fullsized":"2024-05-27T07:43:00.196216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:12.996286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.118137+0000","last_change":"2024-05-27T07:41:08.842685+0000","last_active":"2024-05-27T07:43:00.118137+0000","last_peered":"2024-05-27T07:43:00.118137+0000","last_clean":"2024-05-27T07:43:00.118137+0000","last_became_active":"2024-05-27T07:41:08.841954+0000","last_became_peered":"2024-05-27T07:41:08.841954+0000","last_unstale":"2024-05-27T07:43:00.118137+0000","last_undegraded":"2024-05-27T07:43:00.118137+0000","last_fullsized":"2024-05-27T07:43:00.118137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:18.087129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013268+0000","last_change":"2024-05-27T07:41:04.790330+0000","last_active":"2024-05-27T07:43:00.013268+0000","last_peered":"2024-05-27T07:43:00.013268+0000","last_clean":"2024-05-27T07:43:00.013268+0000","last_became_active":"2024-05-27T07:41:04.789508+0000","last_became_peered":"2024-05-27T07:41:04.789508+0000","last_unstale":"2024-05-27T07:43:00.013268+0000","last_undegraded":"2024-05-27T07:43:00.013268+0000","last_fullsized":"2024-05-27T07:43:00.013268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:27.537662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194532+0000","last_change":"2024-05-27T07:39:11.797324+0000","last_active":"2024-05-27T07:43:00.194532+0000","last_peered":"2024-05-27T07:43:00.194532+0000","last_clean":"2024-05-27T07:43:00.194532+0000","last_became_active":"2024-05-27T07:39:11.786294+0000","last_became_peered":"2024-05-27T07:39:11.786294+0000","last_unstale":"2024-05-27T07:43:00.194532+0000","last_undegraded":"2024-05-27T07:43:00.194532+0000","last_fullsized":"2024-05-27T07:43:00.194532+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_clean_scrub_stamp":"2024-05-27T07:34:18.613413+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:23.947615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042739+0000","last_change":"2024-05-27T07:41:21.997453+0000","last_active":"2024-05-27T07:43:00.042739+0000","last_peered":"2024-05-27T07:43:00.042739+0000","last_clean":"2024-05-27T07:43:00.042739+0000","last_became_active":"2024-05-27T07:41:21.997010+0000","last_became_peered":"2024-05-27T07:41:21.997010+0000","last_unstale":"2024-05-27T07:43:00.042739+0000","last_undegraded":"2024-05-27T07:43:00.042739+0000","last_fullsized":"2024-05-27T07:43:00.042739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:54:54.913274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.026342+0000","last_change":"2024-05-27T07:43:00.024988+0000","last_active":"2024-05-27T07:43:00.026342+0000","last_peered":"2024-05-27T07:43:00.026342+0000","last_clean":"2024-05-27T07:43:00.026342+0000","last_became_active":"2024-05-27T07:43:00.023816+0000","last_became_peered":"2024-05-27T07:43:00.023816+0000","last_unstale":"2024-05-27T07:43:00.026342+0000","last_undegraded":"2024-05-27T07:43:00.026342+0000","last_fullsized":"2024-05-27T07:43:00.026342+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:58.531985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194923+0000","last_change":"2024-05-27T07:41:04.795262+0000","last_active":"2024-05-27T07:43:00.194923+0000","last_peered":"2024-05-27T07:43:00.194923+0000","last_clean":"2024-05-27T07:43:00.194923+0000","last_became_active":"2024-05-27T07:41:04.794866+0000","last_became_peered":"2024-05-27T07:41:04.794866+0000","last_unstale":"2024-05-27T07:43:00.194923+0000","last_undegraded":"2024-05-27T07:43:00.194923+0000","last_fullsized":"2024-05-27T07:43:00.194923+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:07:44.241444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194536+0000","last_change":"2024-05-27T07:41:22.006194+0000","last_active":"2024-05-27T07:43:00.194536+0000","last_peered":"2024-05-27T07:43:00.194536+0000","last_clean":"2024-05-27T07:43:00.194536+0000","last_became_active":"2024-05-27T07:41:22.004079+0000","last_became_peered":"2024-05-27T07:41:22.004079+0000","last_unstale":"2024-05-27T07:43:00.194536+0000","last_undegraded":"2024-05-27T07:43:00.194536+0000","last_fullsized":"2024-05-27T07:43:00.194536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:03.496722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194439+0000","last_change":"2024-05-27T07:41:59.776734+0000","last_active":"2024-05-27T07:43:00.194439+0000","last_peered":"2024-05-27T07:43:00.194439+0000","last_clean":"2024-05-27T07:43:00.194439+0000","last_became_active":"2024-05-27T07:41:59.775678+0000","last_became_peered":"2024-05-27T07:41:59.775678+0000","last_unstale":"2024-05-27T07:43:00.194439+0000","last_undegraded":"2024-05-27T07:43:00.194439+0000","last_fullsized":"2024-05-27T07:43:00.194439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:56.007178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.021656+0000","last_change":"2024-05-27T07:41:08.831070+0000","last_active":"2024-05-27T07:43:00.021656+0000","last_peered":"2024-05-27T07:43:00.021656+0000","last_clean":"2024-05-27T07:43:00.021656+0000","last_became_active":"2024-05-27T07:41:08.830617+0000","last_became_peered":"2024-05-27T07:41:08.830617+0000","last_unstale":"2024-05-27T07:43:00.021656+0000","last_undegraded":"2024-05-27T07:43:00.021656+0000","last_fullsized":"2024-05-27T07:43:00.021656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:13.046162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.116733+0000","last_change":"2024-05-27T07:41:22.010129+0000","last_active":"2024-05-27T07:43:00.116733+0000","last_peered":"2024-05-27T07:43:00.116733+0000","last_clean":"2024-05-27T07:43:00.116733+0000","last_became_active":"2024-05-27T07:41:22.009608+0000","last_became_peered":"2024-05-27T07:41:22.009608+0000","last_unstale":"2024-05-27T07:43:00.116733+0000","last_undegraded":"2024-05-27T07:43:00.116733+0000","last_fullsized":"2024-05-27T07:43:00.116733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.225004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'16","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:03.892001+0000","last_change":"2024-05-27T07:41:04.794744+0000","last_active":"2024-05-27T07:43:03.892001+0000","last_peered":"2024-05-27T07:43:03.892001+0000","last_clean":"2024-05-27T07:43:03.892001+0000","last_became_active":"2024-05-27T07:41:04.794340+0000","last_became_peered":"2024-05-27T07:41:04.794340+0000","last_unstale":"2024-05-27T07:43:03.892001+0000","last_undegraded":"2024-05-27T07:43:03.892001+0000","last_fullsized":"2024-05-27T07:43:03.892001+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:43:20.335520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.119215+0000","last_change":"2024-05-27T07:41:08.841834+0000","last_active":"2024-05-27T07:43:00.119215+0000","last_peered":"2024-05-27T07:43:00.119215+0000","last_clean":"2024-05-27T07:43:00.119215+0000","last_became_active":"2024-05-27T07:41:08.840261+0000","last_became_peered":"2024-05-27T07:41:08.840261+0000","last_unstale":"2024-05-27T07:43:00.119215+0000","last_undegraded":"2024-05-27T07:43:00.119215+0000","last_fullsized":"2024-05-27T07:43:00.119215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:33.738835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012235+0000","last_change":"2024-05-27T07:41:22.005861+0000","last_active":"2024-05-27T07:43:00.012235+0000","last_peered":"2024-05-27T07:43:00.012235+0000","last_clean":"2024-05-27T07:43:00.012235+0000","last_became_active":"2024-05-27T07:41:22.005295+0000","last_became_peered":"2024-05-27T07:41:22.005295+0000","last_unstale":"2024-05-27T07:43:00.012235+0000","last_undegraded":"2024-05-27T07:43:00.012235+0000","last_fullsized":"2024-05-27T07:43:00.012235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:46.582656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013605+0000","last_change":"2024-05-27T07:41:04.796562+0000","last_active":"2024-05-27T07:43:00.013605+0000","last_peered":"2024-05-27T07:43:00.013605+0000","last_clean":"2024-05-27T07:43:00.013605+0000","last_became_active":"2024-05-27T07:41:04.795538+0000","last_became_peered":"2024-05-27T07:41:04.795538+0000","last_unstale":"2024-05-27T07:43:00.013605+0000","last_undegraded":"2024-05-27T07:43:00.013605+0000","last_fullsized":"2024-05-27T07:43:00.013605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:12.131999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193725+0000","last_change":"2024-05-27T07:41:08.832824+0000","last_active":"2024-05-27T07:43:00.193725+0000","last_peered":"2024-05-27T07:43:00.193725+0000","last_clean":"2024-05-27T07:43:00.193725+0000","last_became_active":"2024-05-27T07:41:08.832332+0000","last_became_peered":"2024-05-27T07:41:08.832332+0000","last_unstale":"2024-05-27T07:43:00.193725+0000","last_undegraded":"2024-05-27T07:43:00.193725+0000","last_fullsized":"2024-05-27T07:43:00.193725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:34.598773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011831+0000","last_change":"2024-05-27T07:41:22.003358+0000","last_active":"2024-05-27T07:43:00.011831+0000","last_peered":"2024-05-27T07:43:00.011831+0000","last_clean":"2024-05-27T07:43:00.011831+0000","last_became_active":"2024-05-27T07:41:22.002655+0000","last_became_peered":"2024-05-27T07:41:22.002655+0000","last_unstale":"2024-05-27T07:43:00.011831+0000","last_undegraded":"2024-05-27T07:43:00.011831+0000","last_fullsized":"2024-05-27T07:43:00.011831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:41.177138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193770+0000","last_change":"2024-05-27T07:41:04.888681+0000","last_active":"2024-05-27T07:43:00.193770+0000","last_peered":"2024-05-27T07:43:00.193770+0000","last_clean":"2024-05-27T07:43:00.193770+0000","last_became_active":"2024-05-27T07:41:04.887966+0000","last_became_peered":"2024-05-27T07:41:04.887966+0000","last_unstale":"2024-05-27T07:43:00.193770+0000","last_undegraded":"2024-05-27T07:43:00.193770+0000","last_fullsized":"2024-05-27T07:43:00.193770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:01.178737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.296531+0000","last_change":"2024-05-27T07:41:08.827839+0000","last_active":"2024-05-27T07:44:28.296531+0000","last_peered":"2024-05-27T07:44:28.296531+0000","last_clean":"2024-05-27T07:44:28.296531+0000","last_became_active":"2024-05-27T07:41:08.827403+0000","last_became_peered":"2024-05-27T07:41:08.827403+0000","last_unstale":"2024-05-27T07:44:28.296531+0000","last_undegraded":"2024-05-27T07:44:28.296531+0000","last_fullsized":"2024-05-27T07:44:28.296531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:01:42.075980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.010155+0000","last_change":"2024-05-27T07:41:22.006161+0000","last_active":"2024-05-27T07:43:00.010155+0000","last_peered":"2024-05-27T07:43:00.010155+0000","last_clean":"2024-05-27T07:43:00.010155+0000","last_became_active":"2024-05-27T07:41:22.005762+0000","last_became_peered":"2024-05-27T07:41:22.005762+0000","last_unstale":"2024-05-27T07:43:00.010155+0000","last_undegraded":"2024-05-27T07:43:00.010155+0000","last_fullsized":"2024-05-27T07:43:00.010155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:53.536444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117239+0000","last_change":"2024-05-27T07:41:04.774211+0000","last_active":"2024-05-27T07:43:00.117239+0000","last_peered":"2024-05-27T07:43:00.117239+0000","last_clean":"2024-05-27T07:43:00.117239+0000","last_became_active":"2024-05-27T07:41:04.773215+0000","last_became_peered":"2024-05-27T07:41:04.773215+0000","last_unstale":"2024-05-27T07:43:00.117239+0000","last_undegraded":"2024-05-27T07:43:00.117239+0000","last_fullsized":"2024-05-27T07:43:00.117239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:37.273735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194028+0000","last_change":"2024-05-27T07:41:08.830434+0000","last_active":"2024-05-27T07:43:00.194028+0000","last_peered":"2024-05-27T07:43:00.194028+0000","last_clean":"2024-05-27T07:43:00.194028+0000","last_became_active":"2024-05-27T07:41:08.830039+0000","last_became_peered":"2024-05-27T07:41:08.830039+0000","last_unstale":"2024-05-27T07:43:00.194028+0000","last_undegraded":"2024-05-27T07:43:00.194028+0000","last_fullsized":"2024-05-27T07:43:00.194028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:27.877666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013140+0000","last_change":"2024-05-27T07:41:59.783413+0000","last_active":"2024-05-27T07:43:00.013140+0000","last_peered":"2024-05-27T07:43:00.013140+0000","last_clean":"2024-05-27T07:43:00.013140+0000","last_became_active":"2024-05-27T07:41:59.782300+0000","last_became_peered":"2024-05-27T07:41:59.782300+0000","last_unstale":"2024-05-27T07:43:00.013140+0000","last_undegraded":"2024-05-27T07:43:00.013140+0000","last_fullsized":"2024-05-27T07:43:00.013140+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:59.348928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012623+0000","last_change":"2024-05-27T07:41:04.798969+0000","last_active":"2024-05-27T07:43:00.012623+0000","last_peered":"2024-05-27T07:43:00.012623+0000","last_clean":"2024-05-27T07:43:00.012623+0000","last_became_active":"2024-05-27T07:41:04.795996+0000","last_became_peered":"2024-05-27T07:41:04.795996+0000","last_unstale":"2024-05-27T07:43:00.012623+0000","last_undegraded":"2024-05-27T07:43:00.012623+0000","last_fullsized":"2024-05-27T07:43:00.012623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:47.141063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117657+0000","last_change":"2024-05-27T07:41:08.835142+0000","last_active":"2024-05-27T07:43:00.117657+0000","last_peered":"2024-05-27T07:43:00.117657+0000","last_clean":"2024-05-27T07:43:00.117657+0000","last_became_active":"2024-05-27T07:41:08.834762+0000","last_became_peered":"2024-05-27T07:41:08.834762+0000","last_unstale":"2024-05-27T07:43:00.117657+0000","last_undegraded":"2024-05-27T07:43:00.117657+0000","last_fullsized":"2024-05-27T07:43:00.117657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:52.957554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012439+0000","last_change":"2024-05-27T07:41:22.000649+0000","last_active":"2024-05-27T07:43:00.012439+0000","last_peered":"2024-05-27T07:43:00.012439+0000","last_clean":"2024-05-27T07:43:00.012439+0000","last_became_active":"2024-05-27T07:41:22.000161+0000","last_became_peered":"2024-05-27T07:41:22.000161+0000","last_unstale":"2024-05-27T07:43:00.012439+0000","last_undegraded":"2024-05-27T07:43:00.012439+0000","last_fullsized":"2024-05-27T07:43:00.012439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:10.788027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195920+0000","last_change":"2024-05-27T07:41:04.800680+0000","last_active":"2024-05-27T07:43:00.195920+0000","last_peered":"2024-05-27T07:43:00.195920+0000","last_clean":"2024-05-27T07:43:00.195920+0000","last_became_active":"2024-05-27T07:41:04.800249+0000","last_became_peered":"2024-05-27T07:41:04.800249+0000","last_unstale":"2024-05-27T07:43:00.195920+0000","last_undegraded":"2024-05-27T07:43:00.195920+0000","last_fullsized":"2024-05-27T07:43:00.195920+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:23:27.797216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.365257+0000","last_change":"2024-05-27T07:41:08.844380+0000","last_active":"2024-05-27T07:44:28.365257+0000","last_peered":"2024-05-27T07:44:28.365257+0000","last_clean":"2024-05-27T07:44:28.365257+0000","last_became_active":"2024-05-27T07:41:08.843464+0000","last_became_peered":"2024-05-27T07:41:08.843464+0000","last_unstale":"2024-05-27T07:44:28.365257+0000","last_undegraded":"2024-05-27T07:44:28.365257+0000","last_fullsized":"2024-05-27T07:44:28.365257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:23.290234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194841+0000","last_change":"2024-05-27T07:41:22.010968+0000","last_active":"2024-05-27T07:43:00.194841+0000","last_peered":"2024-05-27T07:43:00.194841+0000","last_clean":"2024-05-27T07:43:00.194841+0000","last_became_active":"2024-05-27T07:41:22.010529+0000","last_became_peered":"2024-05-27T07:41:22.010529+0000","last_unstale":"2024-05-27T07:43:00.194841+0000","last_undegraded":"2024-05-27T07:43:00.194841+0000","last_fullsized":"2024-05-27T07:43:00.194841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:08.636434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":164,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:25.258933+0000","last_change":"2024-05-27T07:41:04.891536+0000","last_active":"2024-05-27T07:44:25.258933+0000","last_peered":"2024-05-27T07:44:25.258933+0000","last_clean":"2024-05-27T07:44:25.258933+0000","last_became_active":"2024-05-27T07:41:04.890724+0000","last_became_peered":"2024-05-27T07:41:04.890724+0000","last_unstale":"2024-05-27T07:44:25.258933+0000","last_undegraded":"2024-05-27T07:44:25.258933+0000","last_fullsized":"2024-05-27T07:44:25.258933+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:32.027624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.362943+0000","last_change":"2024-05-27T07:41:08.834362+0000","last_active":"2024-05-27T07:44:28.362943+0000","last_peered":"2024-05-27T07:44:28.362943+0000","last_clean":"2024-05-27T07:44:28.362943+0000","last_became_active":"2024-05-27T07:41:08.833979+0000","last_became_peered":"2024-05-27T07:41:08.833979+0000","last_unstale":"2024-05-27T07:44:28.362943+0000","last_undegraded":"2024-05-27T07:44:28.362943+0000","last_fullsized":"2024-05-27T07:44:28.362943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:35.098352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.192704+0000","last_change":"2024-05-27T07:41:21.997587+0000","last_active":"2024-05-27T07:43:00.192704+0000","last_peered":"2024-05-27T07:43:00.192704+0000","last_clean":"2024-05-27T07:43:00.192704+0000","last_became_active":"2024-05-27T07:41:21.996413+0000","last_became_peered":"2024-05-27T07:41:21.996413+0000","last_unstale":"2024-05-27T07:43:00.192704+0000","last_undegraded":"2024-05-27T07:43:00.192704+0000","last_fullsized":"2024-05-27T07:43:00.192704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:21:46.728047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.043021+0000","last_change":"2024-05-27T07:41:04.793580+0000","last_active":"2024-05-27T07:43:00.043021+0000","last_peered":"2024-05-27T07:43:00.043021+0000","last_clean":"2024-05-27T07:43:00.043021+0000","last_became_active":"2024-05-27T07:41:04.791135+0000","last_became_peered":"2024-05-27T07:41:04.791135+0000","last_unstale":"2024-05-27T07:43:00.043021+0000","last_undegraded":"2024-05-27T07:43:00.043021+0000","last_fullsized":"2024-05-27T07:43:00.043021+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:15.484187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348081+0000","last_change":"2024-05-27T07:41:08.839187+0000","last_active":"2024-05-27T07:44:28.348081+0000","last_peered":"2024-05-27T07:44:28.348081+0000","last_clean":"2024-05-27T07:44:28.348081+0000","last_became_active":"2024-05-27T07:41:08.838821+0000","last_became_peered":"2024-05-27T07:41:08.838821+0000","last_unstale":"2024-05-27T07:44:28.348081+0000","last_undegraded":"2024-05-27T07:44:28.348081+0000","last_fullsized":"2024-05-27T07:44:28.348081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:30.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041135+0000","last_change":"2024-05-27T07:41:22.005406+0000","last_active":"2024-05-27T07:43:00.041135+0000","last_peered":"2024-05-27T07:43:00.041135+0000","last_clean":"2024-05-27T07:43:00.041135+0000","last_became_active":"2024-05-27T07:41:22.003415+0000","last_became_peered":"2024-05-27T07:41:22.003415+0000","last_unstale":"2024-05-27T07:43:00.041135+0000","last_undegraded":"2024-05-27T07:43:00.041135+0000","last_fullsized":"2024-05-27T07:43:00.041135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:03.300452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018670+0000","last_change":"2024-05-27T07:41:04.798489+0000","last_active":"2024-05-27T07:43:00.018670+0000","last_peered":"2024-05-27T07:43:00.018670+0000","last_clean":"2024-05-27T07:43:00.018670+0000","last_became_active":"2024-05-27T07:41:04.797467+0000","last_became_peered":"2024-05-27T07:41:04.797467+0000","last_unstale":"2024-05-27T07:43:00.018670+0000","last_undegraded":"2024-05-27T07:43:00.018670+0000","last_fullsized":"2024-05-27T07:43:00.018670+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:04.018577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.349083+0000","last_change":"2024-05-27T07:41:08.832215+0000","last_active":"2024-05-27T07:44:28.349083+0000","last_peered":"2024-05-27T07:44:28.349083+0000","last_clean":"2024-05-27T07:44:28.349083+0000","last_became_active":"2024-05-27T07:41:08.831491+0000","last_became_peered":"2024-05-27T07:41:08.831491+0000","last_unstale":"2024-05-27T07:44:28.349083+0000","last_undegraded":"2024-05-27T07:44:28.349083+0000","last_fullsized":"2024-05-27T07:44:28.349083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:32:03.084324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194326+0000","last_change":"2024-05-27T07:41:08.837301+0000","last_active":"2024-05-27T07:43:00.194326+0000","last_peered":"2024-05-27T07:43:00.194326+0000","last_clean":"2024-05-27T07:43:00.194326+0000","last_became_active":"2024-05-27T07:41:08.835533+0000","last_became_peered":"2024-05-27T07:41:08.835533+0000","last_unstale":"2024-05-27T07:43:00.194326+0000","last_undegraded":"2024-05-27T07:43:00.194326+0000","last_fullsized":"2024-05-27T07:43:00.194326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:45.807613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013212+0000","last_change":"2024-05-27T07:41:08.846740+0000","last_active":"2024-05-27T07:43:00.013212+0000","last_peered":"2024-05-27T07:43:00.013212+0000","last_clean":"2024-05-27T07:43:00.013212+0000","last_became_active":"2024-05-27T07:41:08.845576+0000","last_became_peered":"2024-05-27T07:41:08.845576+0000","last_unstale":"2024-05-27T07:43:00.013212+0000","last_undegraded":"2024-05-27T07:43:00.013212+0000","last_fullsized":"2024-05-27T07:43:00.013212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:41:36.944923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012307+0000","last_change":"2024-05-27T07:41:08.840121+0000","last_active":"2024-05-27T07:43:00.012307+0000","last_peered":"2024-05-27T07:43:00.012307+0000","last_clean":"2024-05-27T07:43:00.012307+0000","last_became_active":"2024-05-27T07:41:08.839090+0000","last_became_peered":"2024-05-27T07:41:08.839090+0000","last_unstale":"2024-05-27T07:43:00.012307+0000","last_undegraded":"2024-05-27T07:43:00.012307+0000","last_fullsized":"2024-05-27T07:43:00.012307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:44.836953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193279+0000","last_change":"2024-05-27T07:41:08.835565+0000","last_active":"2024-05-27T07:43:00.193279+0000","last_peered":"2024-05-27T07:43:00.193279+0000","last_clean":"2024-05-27T07:43:00.193279+0000","last_became_active":"2024-05-27T07:41:08.831424+0000","last_became_peered":"2024-05-27T07:41:08.831424+0000","last_unstale":"2024-05-27T07:43:00.193279+0000","last_undegraded":"2024-05-27T07:43:00.193279+0000","last_fullsized":"2024-05-27T07:43:00.193279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:17.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.021222+0000","last_change":"2024-05-27T07:41:08.830104+0000","last_active":"2024-05-27T07:43:00.021222+0000","last_peered":"2024-05-27T07:43:00.021222+0000","last_clean":"2024-05-27T07:43:00.021222+0000","last_became_active":"2024-05-27T07:41:08.829690+0000","last_became_peered":"2024-05-27T07:41:08.829690+0000","last_unstale":"2024-05-27T07:43:00.021222+0000","last_undegraded":"2024-05-27T07:43:00.021222+0000","last_fullsized":"2024-05-27T07:43:00.021222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194912+0000","last_change":"2024-05-27T07:41:08.838654+0000","last_active":"2024-05-27T07:43:00.194912+0000","last_peered":"2024-05-27T07:43:00.194912+0000","last_clean":"2024-05-27T07:43:00.194912+0000","last_became_active":"2024-05-27T07:41:08.835033+0000","last_became_peered":"2024-05-27T07:41:08.835033+0000","last_unstale":"2024-05-27T07:43:00.194912+0000","last_undegraded":"2024-05-27T07:43:00.194912+0000","last_fullsized":"2024-05-27T07:43:00.194912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:04:24.007585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013198+0000","last_change":"2024-05-27T07:41:08.844557+0000","last_active":"2024-05-27T07:43:00.013198+0000","last_peered":"2024-05-27T07:43:00.013198+0000","last_clean":"2024-05-27T07:43:00.013198+0000","last_became_active":"2024-05-27T07:41:08.843418+0000","last_became_peered":"2024-05-27T07:41:08.843418+0000","last_unstale":"2024-05-27T07:43:00.013198+0000","last_undegraded":"2024-05-27T07:43:00.013198+0000","last_fullsized":"2024-05-27T07:43:00.013198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:39.587778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.119238+0000","last_change":"2024-05-27T07:41:08.830857+0000","last_active":"2024-05-27T07:43:00.119238+0000","last_peered":"2024-05-27T07:43:00.119238+0000","last_clean":"2024-05-27T07:43:00.119238+0000","last_became_active":"2024-05-27T07:41:08.830480+0000","last_became_peered":"2024-05-27T07:41:08.830480+0000","last_unstale":"2024-05-27T07:43:00.119238+0000","last_undegraded":"2024-05-27T07:43:00.119238+0000","last_fullsized":"2024-05-27T07:43:00.119238+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:07.307528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348940+0000","last_change":"2024-05-27T07:41:08.824571+0000","last_active":"2024-05-27T07:44:28.348940+0000","last_peered":"2024-05-27T07:44:28.348940+0000","last_clean":"2024-05-27T07:44:28.348940+0000","last_became_active":"2024-05-27T07:41:08.820732+0000","last_became_peered":"2024-05-27T07:41:08.820732+0000","last_unstale":"2024-05-27T07:44:28.348940+0000","last_undegraded":"2024-05-27T07:44:28.348940+0000","last_fullsized":"2024-05-27T07:44:28.348940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:55:14.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117433+0000","last_change":"2024-05-27T07:41:08.842284+0000","last_active":"2024-05-27T07:43:00.117433+0000","last_peered":"2024-05-27T07:43:00.117433+0000","last_clean":"2024-05-27T07:43:00.117433+0000","last_became_active":"2024-05-27T07:41:08.841597+0000","last_became_peered":"2024-05-27T07:41:08.841597+0000","last_unstale":"2024-05-27T07:43:00.117433+0000","last_undegraded":"2024-05-27T07:43:00.117433+0000","last_fullsized":"2024-05-27T07:43:00.117433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:25:31.066181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195736+0000","last_change":"2024-05-27T07:41:08.828935+0000","last_active":"2024-05-27T07:43:00.195736+0000","last_peered":"2024-05-27T07:43:00.195736+0000","last_clean":"2024-05-27T07:43:00.195736+0000","last_became_active":"2024-05-27T07:41:08.828542+0000","last_became_peered":"2024-05-27T07:41:08.828542+0000","last_unstale":"2024-05-27T07:43:00.195736+0000","last_undegraded":"2024-05-27T07:43:00.195736+0000","last_fullsized":"2024-05-27T07:43:00.195736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:35.270666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018065+0000","last_change":"2024-05-27T07:41:08.817051+0000","last_active":"2024-05-27T07:43:00.018065+0000","last_peered":"2024-05-27T07:43:00.018065+0000","last_clean":"2024-05-27T07:43:00.018065+0000","last_became_active":"2024-05-27T07:41:08.816418+0000","last_became_peered":"2024-05-27T07:41:08.816418+0000","last_unstale":"2024-05-27T07:43:00.018065+0000","last_undegraded":"2024-05-27T07:43:00.018065+0000","last_fullsized":"2024-05-27T07:43:00.018065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:02.760707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195099+0000","last_change":"2024-05-27T07:41:08.834081+0000","last_active":"2024-05-27T07:43:00.195099+0000","last_peered":"2024-05-27T07:43:00.195099+0000","last_clean":"2024-05-27T07:43:00.195099+0000","last_became_active":"2024-05-27T07:41:08.833679+0000","last_became_peered":"2024-05-27T07:41:08.833679+0000","last_unstale":"2024-05-27T07:43:00.195099+0000","last_undegraded":"2024-05-27T07:43:00.195099+0000","last_fullsized":"2024-05-27T07:43:00.195099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:41:14.426593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74769,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1188,"num_read_kb":676,"num_write":259,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":223917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645371,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":522393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841603,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1753088,"data_stored":1181008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005130,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":522211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201361,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":522477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397593,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1119759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561119,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":517715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":585779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738508,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":519244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117111,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":517423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920880,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":589716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757352,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1114809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:44:32.928 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T07:44:33.409 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:33.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:33 smithi090 ceph-mon[31429]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:44:33.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:33 smithi090 ceph-mon[31429]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T07:44:34.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:33 smithi129 ceph-mon[35254]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:44:34.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:33 smithi129 ceph-mon[35254]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T07:44:34.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:33 smithi136 ceph-mon[35324]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:44:34.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:33 smithi136 ceph-mon[35324]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T07:44:35.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.697+0000 7fd8ea200700 1 -- 172.21.15.90:0/1387142900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8e4075d20 msgr2=0x7fd8e4076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:35.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.697+0000 7fd8ea200700 1 --2- 172.21.15.90:0/1387142900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8e4075d20 0x7fd8e4076130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009990 tx=0x7fd8d4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.697+0000 7fd8ea200700 1 -- 172.21.15.90:0/1387142900 shutdown_connections 2024-05-27T07:44:35.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.697+0000 7fd8ea200700 1 --2- 172.21.15.90:0/1387142900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8e41109a0 0x7fd8e4112e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.697+0000 7fd8ea200700 1 --2- 172.21.15.90:0/1387142900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8e4075d20 0x7fd8e4076130 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.697+0000 7fd8ea200700 1 --2- 172.21.15.90:0/1387142900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8e4076670 0x7fd8e4110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.697+0000 7fd8ea200700 1 -- 172.21.15.90:0/1387142900 >> 172.21.15.90:0/1387142900 conn(0x7fd8e4079d00 msgr2=0x7fd8e407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:35.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.698+0000 7fd8ea200700 1 -- 172.21.15.90:0/1387142900 shutdown_connections 2024-05-27T07:44:35.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.698+0000 7fd8ea200700 1 -- 172.21.15.90:0/1387142900 wait complete. 2024-05-27T07:44:35.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.699+0000 7fd8ea200700 1 Processor -- start 2024-05-27T07:44:35.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.699+0000 7fd8ea200700 1 -- start start 2024-05-27T07:44:35.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.700+0000 7fd8ea200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8e4075d20 0x7fd8e419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:35.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.700+0000 7fd8ea200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8e4076670 0x7fd8e419d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:35.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.700+0000 7fd8ea200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8e41109a0 0x7fd8e41a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:35.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.700+0000 7fd8ea200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e4116340 con 0x7fd8e4076670 2024-05-27T07:44:35.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.700+0000 7fd8ea200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41161c0 con 0x7fd8e4075d20 2024-05-27T07:44:35.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.700+0000 7fd8ea200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e41164c0 con 0x7fd8e41109a0 2024-05-27T07:44:35.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.700+0000 7fd8dbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8e4076670 0x7fd8e419d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.700+0000 7fd8dbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8e4076670 0x7fd8e419d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38388/0 (socket says 172.21.15.90:38388) 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.700+0000 7fd8dbe00700 1 -- 172.21.15.90:0/2805855426 learned_addr learned my addr 172.21.15.90:0/2805855426 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8e8e00700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8e4075d20 0x7fd8e419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8e9800700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8e41109a0 0x7fd8e41a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8dbe00700 1 -- 172.21.15.90:0/2805855426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8e41109a0 msgr2=0x7fd8e41a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8dbe00700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8e41109a0 0x7fd8e41a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8dbe00700 1 -- 172.21.15.90:0/2805855426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8e4075d20 msgr2=0x7fd8e419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8dbe00700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8e4075d20 0x7fd8e419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8dbe00700 1 -- 172.21.15.90:0/2805855426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e41a1c60 con 0x7fd8e4076670 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8e8e00700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8e4075d20 0x7fd8e419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:35.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8dbe00700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8e4076670 0x7fd8e419d330 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000c920 tx=0x7fd8d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:35.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8e2000700 1 -- 172.21.15.90:0/2805855426 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0007b00 con 0x7fd8e4076670 2024-05-27T07:44:35.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.701+0000 7fd8e2000700 1 -- 172.21.15.90:0/2805855426 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8d0004cc0 con 0x7fd8e4076670 2024-05-27T07:44:35.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.702+0000 7fd8ea200700 1 -- 172.21.15.90:0/2805855426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e41a1f50 con 0x7fd8e4076670 2024-05-27T07:44:35.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.702+0000 7fd8e2000700 1 -- 172.21.15.90:0/2805855426 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00054f0 con 0x7fd8e4076670 2024-05-27T07:44:35.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.702+0000 7fd8ea200700 1 -- 172.21.15.90:0/2805855426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e41a9870 con 0x7fd8e4076670 2024-05-27T07:44:35.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.703+0000 7fd8ea200700 1 -- 172.21.15.90:0/2805855426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e4077290 con 0x7fd8e4076670 2024-05-27T07:44:35.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.705+0000 7fd8e2000700 1 -- 172.21.15.90:0/2805855426 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd8d0007ca0 con 0x7fd8e4076670 2024-05-27T07:44:35.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.707+0000 7fd8e2000700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8c80b2880 0x7fd8c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:35.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.707+0000 7fd8e2000700 1 -- 172.21.15.90:0/2805855426 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fd8d00e2240 con 0x7fd8e4076670 2024-05-27T07:44:35.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.707+0000 7fd8e8e00700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8c80b2880 0x7fd8c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:35.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.708+0000 7fd8e8e00700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8c80b2880 0x7fd8c80b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009750 tx=0x7fd8d400b580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:35.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.712+0000 7fd8e2000700 1 -- 172.21.15.90:0/2805855426 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8d0014030 con 0x7fd8e4076670 2024-05-27T07:44:35.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.939+0000 7fd8ea200700 1 -- 172.21.15.90:0/2805855426 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8e4063bf0 con 0x7fd8c80b2880 2024-05-27T07:44:35.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:35 smithi090 ceph-mon[31429]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-27T07:44:35.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.968+0000 7fd8e2000700 1 -- 172.21.15.90:0/2805855426 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340698 (secure 0 0 0) 0x7fd8e4063bf0 con 0x7fd8c80b2880 2024-05-27T07:44:35.970 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:44:35.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 -- 172.21.15.90:0/2805855426 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8c80b2880 msgr2=0x7fd8c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8c80b2880 0x7fd8c80b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009750 tx=0x7fd8d400b580 comp rx=0 tx=0).stop 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 -- 172.21.15.90:0/2805855426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8e4076670 msgr2=0x7fd8e419d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8e4076670 0x7fd8e419d330 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000c920 tx=0x7fd8d000cce0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 -- 172.21.15.90:0/2805855426 shutdown_connections 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fd8c80b2880 0x7fd8c80b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8e41109a0 0x7fd8e41a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8e4075d20 0x7fd8e419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 --2- 172.21.15.90:0/2805855426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8e4076670 0x7fd8e419d330 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 -- 172.21.15.90:0/2805855426 >> 172.21.15.90:0/2805855426 conn(0x7fd8e4079d00 msgr2=0x7fd8e4101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.977+0000 7fd8ea200700 1 -- 172.21.15.90:0/2805855426 shutdown_connections 2024-05-27T07:44:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:35.978+0000 7fd8ea200700 1 -- 172.21.15.90:0/2805855426 wait complete. 2024-05-27T07:44:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-05-27T07:44:36.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:35 smithi129 ceph-mon[35254]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-27T07:44:36.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:35 smithi136 ceph-mon[35324]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-27T07:44:36.786 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-05-27T07:44:34.466961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665371,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1284,"num_read_kb":755,"num_write":387,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":820,"ondisk_log_size":820,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359436,"kb_used_data":14444,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124582388,"statfs":{"total":1151940427776,"available":1151572365312,"internally_reserved":0,"allocated":14790656,"data_stored":8232095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19138,"internal_metadata":353154366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":77,"apply_latency_ms":77,"commit_latency_ns":77000000,"apply_latency_ns":77000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":59,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005314"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.357627+0000","last_change":"2024-05-27T07:41:08.839265+0000","last_active":"2024-05-27T07:44:28.357627+0000","last_peered":"2024-05-27T07:44:28.357627+0000","last_clean":"2024-05-27T07:44:28.357627+0000","last_became_active":"2024-05-27T07:41:08.838814+0000","last_became_peered":"2024-05-27T07:41:08.838814+0000","last_unstale":"2024-05-27T07:44:28.357627+0000","last_undegraded":"2024-05-27T07:44:28.357627+0000","last_fullsized":"2024-05-27T07:44:28.357627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:32.901520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.339677+0000","last_change":"2024-05-27T07:41:08.838313+0000","last_active":"2024-05-27T07:44:28.339677+0000","last_peered":"2024-05-27T07:44:28.339677+0000","last_clean":"2024-05-27T07:44:28.339677+0000","last_became_active":"2024-05-27T07:41:08.836657+0000","last_became_peered":"2024-05-27T07:41:08.836657+0000","last_unstale":"2024-05-27T07:44:28.339677+0000","last_undegraded":"2024-05-27T07:44:28.339677+0000","last_fullsized":"2024-05-27T07:44:28.339677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:33.556505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.357497+0000","last_change":"2024-05-27T07:41:08.841513+0000","last_active":"2024-05-27T07:44:28.357497+0000","last_peered":"2024-05-27T07:44:28.357497+0000","last_clean":"2024-05-27T07:44:28.357497+0000","last_became_active":"2024-05-27T07:41:08.838624+0000","last_became_peered":"2024-05-27T07:41:08.838624+0000","last_unstale":"2024-05-27T07:44:28.357497+0000","last_undegraded":"2024-05-27T07:44:28.357497+0000","last_fullsized":"2024-05-27T07:44:28.357497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:01:37.320418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347168+0000","last_change":"2024-05-27T07:41:08.839138+0000","last_active":"2024-05-27T07:44:28.347168+0000","last_peered":"2024-05-27T07:44:28.347168+0000","last_clean":"2024-05-27T07:44:28.347168+0000","last_became_active":"2024-05-27T07:41:08.838766+0000","last_became_peered":"2024-05-27T07:41:08.838766+0000","last_unstale":"2024-05-27T07:44:28.347168+0000","last_undegraded":"2024-05-27T07:44:28.347168+0000","last_fullsized":"2024-05-27T07:44:28.347168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:09.129257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.346785+0000","last_change":"2024-05-27T07:41:08.823006+0000","last_active":"2024-05-27T07:44:28.346785+0000","last_peered":"2024-05-27T07:44:28.346785+0000","last_clean":"2024-05-27T07:44:28.346785+0000","last_became_active":"2024-05-27T07:41:08.822551+0000","last_became_peered":"2024-05-27T07:41:08.822551+0000","last_unstale":"2024-05-27T07:44:28.346785+0000","last_undegraded":"2024-05-27T07:44:28.346785+0000","last_fullsized":"2024-05-27T07:44:28.346785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:49.255763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.346127+0000","last_change":"2024-05-27T07:41:08.846072+0000","last_active":"2024-05-27T07:44:28.346127+0000","last_peered":"2024-05-27T07:44:28.346127+0000","last_clean":"2024-05-27T07:44:28.346127+0000","last_became_active":"2024-05-27T07:41:08.845604+0000","last_became_peered":"2024-05-27T07:41:08.845604+0000","last_unstale":"2024-05-27T07:44:28.346127+0000","last_undegraded":"2024-05-27T07:44:28.346127+0000","last_fullsized":"2024-05-27T07:44:28.346127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:23:56.389135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348891+0000","last_change":"2024-05-27T07:41:08.841046+0000","last_active":"2024-05-27T07:44:28.348891+0000","last_peered":"2024-05-27T07:44:28.348891+0000","last_clean":"2024-05-27T07:44:28.348891+0000","last_became_active":"2024-05-27T07:41:08.840028+0000","last_became_peered":"2024-05-27T07:41:08.840028+0000","last_unstale":"2024-05-27T07:44:28.348891+0000","last_undegraded":"2024-05-27T07:44:28.348891+0000","last_fullsized":"2024-05-27T07:44:28.348891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:35.239919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348199+0000","last_change":"2024-05-27T07:41:08.841182+0000","last_active":"2024-05-27T07:44:28.348199+0000","last_peered":"2024-05-27T07:44:28.348199+0000","last_clean":"2024-05-27T07:44:28.348199+0000","last_became_active":"2024-05-27T07:41:08.840645+0000","last_became_peered":"2024-05-27T07:41:08.840645+0000","last_unstale":"2024-05-27T07:44:28.348199+0000","last_undegraded":"2024-05-27T07:44:28.348199+0000","last_fullsized":"2024-05-27T07:44:28.348199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:10:59.744505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.361818+0000","last_change":"2024-05-27T07:41:08.825874+0000","last_active":"2024-05-27T07:44:28.361818+0000","last_peered":"2024-05-27T07:44:28.361818+0000","last_clean":"2024-05-27T07:44:28.361818+0000","last_became_active":"2024-05-27T07:41:08.825386+0000","last_became_peered":"2024-05-27T07:41:08.825386+0000","last_unstale":"2024-05-27T07:44:28.361818+0000","last_undegraded":"2024-05-27T07:44:28.361818+0000","last_fullsized":"2024-05-27T07:44:28.361818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:13.304612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347910+0000","last_change":"2024-05-27T07:41:08.847445+0000","last_active":"2024-05-27T07:44:28.347910+0000","last_peered":"2024-05-27T07:44:28.347910+0000","last_clean":"2024-05-27T07:44:28.347910+0000","last_became_active":"2024-05-27T07:41:08.846512+0000","last_became_peered":"2024-05-27T07:41:08.846512+0000","last_unstale":"2024-05-27T07:44:28.347910+0000","last_undegraded":"2024-05-27T07:44:28.347910+0000","last_fullsized":"2024-05-27T07:44:28.347910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:57.235218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.362506+0000","last_change":"2024-05-27T07:41:08.843531+0000","last_active":"2024-05-27T07:44:28.362506+0000","last_peered":"2024-05-27T07:44:28.362506+0000","last_clean":"2024-05-27T07:44:28.362506+0000","last_became_active":"2024-05-27T07:41:08.843214+0000","last_became_peered":"2024-05-27T07:41:08.843214+0000","last_unstale":"2024-05-27T07:44:28.362506+0000","last_undegraded":"2024-05-27T07:44:28.362506+0000","last_fullsized":"2024-05-27T07:44:28.362506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:00.884178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.346497+0000","last_change":"2024-05-27T07:41:08.839487+0000","last_active":"2024-05-27T07:44:28.346497+0000","last_peered":"2024-05-27T07:44:28.346497+0000","last_clean":"2024-05-27T07:44:28.346497+0000","last_became_active":"2024-05-27T07:41:08.839097+0000","last_became_peered":"2024-05-27T07:41:08.839097+0000","last_unstale":"2024-05-27T07:44:28.346497+0000","last_undegraded":"2024-05-27T07:44:28.346497+0000","last_fullsized":"2024-05-27T07:44:28.346497+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:51.055482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348935+0000","last_change":"2024-05-27T07:41:08.836843+0000","last_active":"2024-05-27T07:44:28.348935+0000","last_peered":"2024-05-27T07:44:28.348935+0000","last_clean":"2024-05-27T07:44:28.348935+0000","last_became_active":"2024-05-27T07:41:08.836362+0000","last_became_peered":"2024-05-27T07:41:08.836362+0000","last_unstale":"2024-05-27T07:44:28.348935+0000","last_undegraded":"2024-05-27T07:44:28.348935+0000","last_fullsized":"2024-05-27T07:44:28.348935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:47.844301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.337769+0000","last_change":"2024-05-27T07:41:08.839046+0000","last_active":"2024-05-27T07:44:28.337769+0000","last_peered":"2024-05-27T07:44:28.337769+0000","last_clean":"2024-05-27T07:44:28.337769+0000","last_became_active":"2024-05-27T07:41:08.838601+0000","last_became_peered":"2024-05-27T07:41:08.838601+0000","last_unstale":"2024-05-27T07:44:28.337769+0000","last_undegraded":"2024-05-27T07:44:28.337769+0000","last_fullsized":"2024-05-27T07:44:28.337769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:59.563490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.344784+0000","last_change":"2024-05-27T07:41:08.832510+0000","last_active":"2024-05-27T07:44:28.344784+0000","last_peered":"2024-05-27T07:44:28.344784+0000","last_clean":"2024-05-27T07:44:28.344784+0000","last_became_active":"2024-05-27T07:41:08.832082+0000","last_became_peered":"2024-05-27T07:41:08.832082+0000","last_unstale":"2024-05-27T07:44:28.344784+0000","last_undegraded":"2024-05-27T07:44:28.344784+0000","last_fullsized":"2024-05-27T07:44:28.344784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:59.951860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.361104+0000","last_change":"2024-05-27T07:41:08.827953+0000","last_active":"2024-05-27T07:44:28.361104+0000","last_peered":"2024-05-27T07:44:28.361104+0000","last_clean":"2024-05-27T07:44:28.361104+0000","last_became_active":"2024-05-27T07:41:08.827293+0000","last_became_peered":"2024-05-27T07:41:08.827293+0000","last_unstale":"2024-05-27T07:44:28.361104+0000","last_undegraded":"2024-05-27T07:44:28.361104+0000","last_fullsized":"2024-05-27T07:44:28.361104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:08.810514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.357506+0000","last_change":"2024-05-27T07:41:08.839591+0000","last_active":"2024-05-27T07:44:28.357506+0000","last_peered":"2024-05-27T07:44:28.357506+0000","last_clean":"2024-05-27T07:44:28.357506+0000","last_became_active":"2024-05-27T07:41:08.839217+0000","last_became_peered":"2024-05-27T07:41:08.839217+0000","last_unstale":"2024-05-27T07:44:28.357506+0000","last_undegraded":"2024-05-27T07:44:28.357506+0000","last_fullsized":"2024-05-27T07:44:28.357506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:11:41.636163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.365037+0000","last_change":"2024-05-27T07:41:08.839544+0000","last_active":"2024-05-27T07:44:28.365037+0000","last_peered":"2024-05-27T07:44:28.365037+0000","last_clean":"2024-05-27T07:44:28.365037+0000","last_became_active":"2024-05-27T07:41:08.839177+0000","last_became_peered":"2024-05-27T07:41:08.839177+0000","last_unstale":"2024-05-27T07:44:28.365037+0000","last_undegraded":"2024-05-27T07:44:28.365037+0000","last_fullsized":"2024-05-27T07:44:28.365037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:41.921759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348067+0000","last_change":"2024-05-27T07:41:08.847911+0000","last_active":"2024-05-27T07:44:28.348067+0000","last_peered":"2024-05-27T07:44:28.348067+0000","last_clean":"2024-05-27T07:44:28.348067+0000","last_became_active":"2024-05-27T07:41:08.847427+0000","last_became_peered":"2024-05-27T07:41:08.847427+0000","last_unstale":"2024-05-27T07:44:28.348067+0000","last_undegraded":"2024-05-27T07:44:28.348067+0000","last_fullsized":"2024-05-27T07:44:28.348067+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:13.692744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012606+0000","last_change":"2024-05-27T07:41:21.999614+0000","last_active":"2024-05-27T07:43:00.012606+0000","last_peered":"2024-05-27T07:43:00.012606+0000","last_clean":"2024-05-27T07:43:00.012606+0000","last_became_active":"2024-05-27T07:41:21.999086+0000","last_became_peered":"2024-05-27T07:41:21.999086+0000","last_unstale":"2024-05-27T07:43:00.012606+0000","last_undegraded":"2024-05-27T07:43:00.012606+0000","last_fullsized":"2024-05-27T07:43:00.012606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:20.229318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":177,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:31.253511+0000","last_change":"2024-05-27T07:41:04.795079+0000","last_active":"2024-05-27T07:44:31.253511+0000","last_peered":"2024-05-27T07:44:31.253511+0000","last_clean":"2024-05-27T07:44:31.253511+0000","last_became_active":"2024-05-27T07:41:04.794367+0000","last_became_peered":"2024-05-27T07:41:04.794367+0000","last_unstale":"2024-05-27T07:44:31.253511+0000","last_undegraded":"2024-05-27T07:44:31.253511+0000","last_fullsized":"2024-05-27T07:44:31.253511+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:27:52.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.346797+0000","last_change":"2024-05-27T07:41:08.846712+0000","last_active":"2024-05-27T07:44:28.346797+0000","last_peered":"2024-05-27T07:44:28.346797+0000","last_clean":"2024-05-27T07:44:28.346797+0000","last_became_active":"2024-05-27T07:41:08.839999+0000","last_became_peered":"2024-05-27T07:41:08.839999+0000","last_unstale":"2024-05-27T07:44:28.346797+0000","last_undegraded":"2024-05-27T07:44:28.346797+0000","last_fullsized":"2024-05-27T07:44:28.346797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:31:23.924074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012118+0000","last_change":"2024-05-27T07:41:21.996675+0000","last_active":"2024-05-27T07:43:00.012118+0000","last_peered":"2024-05-27T07:43:00.012118+0000","last_clean":"2024-05-27T07:43:00.012118+0000","last_became_active":"2024-05-27T07:41:21.996152+0000","last_became_peered":"2024-05-27T07:41:21.996152+0000","last_unstale":"2024-05-27T07:43:00.012118+0000","last_undegraded":"2024-05-27T07:43:00.012118+0000","last_fullsized":"2024-05-27T07:43:00.012118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:57.415146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.119419+0000","last_change":"2024-05-27T07:41:04.890045+0000","last_active":"2024-05-27T07:43:00.119419+0000","last_peered":"2024-05-27T07:43:00.119419+0000","last_clean":"2024-05-27T07:43:00.119419+0000","last_became_active":"2024-05-27T07:41:04.888993+0000","last_became_peered":"2024-05-27T07:41:04.888993+0000","last_unstale":"2024-05-27T07:43:00.119419+0000","last_undegraded":"2024-05-27T07:43:00.119419+0000","last_fullsized":"2024-05-27T07:43:00.119419+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:19.866699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.357386+0000","last_change":"2024-05-27T07:41:08.847472+0000","last_active":"2024-05-27T07:44:28.357386+0000","last_peered":"2024-05-27T07:44:28.357386+0000","last_clean":"2024-05-27T07:44:28.357386+0000","last_became_active":"2024-05-27T07:41:08.846070+0000","last_became_peered":"2024-05-27T07:41:08.846070+0000","last_unstale":"2024-05-27T07:44:28.357386+0000","last_undegraded":"2024-05-27T07:44:28.357386+0000","last_fullsized":"2024-05-27T07:44:28.357386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:29:08.497007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012033+0000","last_change":"2024-05-27T07:41:59.842473+0000","last_active":"2024-05-27T07:43:00.012033+0000","last_peered":"2024-05-27T07:43:00.012033+0000","last_clean":"2024-05-27T07:43:00.012033+0000","last_became_active":"2024-05-27T07:41:59.841035+0000","last_became_peered":"2024-05-27T07:41:59.841035+0000","last_unstale":"2024-05-27T07:43:00.012033+0000","last_undegraded":"2024-05-27T07:43:00.012033+0000","last_fullsized":"2024-05-27T07:43:00.012033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:52.959589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.197322+0000","last_change":"2024-05-27T07:41:04.804046+0000","last_active":"2024-05-27T07:43:00.197322+0000","last_peered":"2024-05-27T07:43:00.197322+0000","last_clean":"2024-05-27T07:43:00.197322+0000","last_became_active":"2024-05-27T07:41:04.803360+0000","last_became_peered":"2024-05-27T07:41:04.803360+0000","last_unstale":"2024-05-27T07:43:00.197322+0000","last_undegraded":"2024-05-27T07:43:00.197322+0000","last_fullsized":"2024-05-27T07:43:00.197322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:17.325221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.365572+0000","last_change":"2024-05-27T07:41:08.840842+0000","last_active":"2024-05-27T07:44:28.365572+0000","last_peered":"2024-05-27T07:44:28.365572+0000","last_clean":"2024-05-27T07:44:28.365572+0000","last_became_active":"2024-05-27T07:41:08.837817+0000","last_became_peered":"2024-05-27T07:41:08.837817+0000","last_unstale":"2024-05-27T07:44:28.365572+0000","last_undegraded":"2024-05-27T07:44:28.365572+0000","last_fullsized":"2024-05-27T07:44:28.365572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:53:38.130608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011841+0000","last_change":"2024-05-27T07:41:22.002988+0000","last_active":"2024-05-27T07:43:00.011841+0000","last_peered":"2024-05-27T07:43:00.011841+0000","last_clean":"2024-05-27T07:43:00.011841+0000","last_became_active":"2024-05-27T07:41:22.001141+0000","last_became_peered":"2024-05-27T07:41:22.001141+0000","last_unstale":"2024-05-27T07:43:00.011841+0000","last_undegraded":"2024-05-27T07:43:00.011841+0000","last_fullsized":"2024-05-27T07:43:00.011841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:29:17.380439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.197053+0000","last_change":"2024-05-27T07:41:04.794424+0000","last_active":"2024-05-27T07:43:00.197053+0000","last_peered":"2024-05-27T07:43:00.197053+0000","last_clean":"2024-05-27T07:43:00.197053+0000","last_became_active":"2024-05-27T07:41:04.793912+0000","last_became_peered":"2024-05-27T07:41:04.793912+0000","last_unstale":"2024-05-27T07:43:00.197053+0000","last_undegraded":"2024-05-27T07:43:00.197053+0000","last_fullsized":"2024-05-27T07:43:00.197053+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:50.876891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.360086+0000","last_change":"2024-05-27T07:41:08.832356+0000","last_active":"2024-05-27T07:44:28.360086+0000","last_peered":"2024-05-27T07:44:28.360086+0000","last_clean":"2024-05-27T07:44:28.360086+0000","last_became_active":"2024-05-27T07:41:08.831979+0000","last_became_peered":"2024-05-27T07:41:08.831979+0000","last_unstale":"2024-05-27T07:44:28.360086+0000","last_undegraded":"2024-05-27T07:44:28.360086+0000","last_fullsized":"2024-05-27T07:44:28.360086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:37:47.584177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013124+0000","last_change":"2024-05-27T07:41:22.004233+0000","last_active":"2024-05-27T07:43:00.013124+0000","last_peered":"2024-05-27T07:43:00.013124+0000","last_clean":"2024-05-27T07:43:00.013124+0000","last_became_active":"2024-05-27T07:41:22.003781+0000","last_became_peered":"2024-05-27T07:41:22.003781+0000","last_unstale":"2024-05-27T07:43:00.013124+0000","last_undegraded":"2024-05-27T07:43:00.013124+0000","last_fullsized":"2024-05-27T07:43:00.013124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:02:13.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":171,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:29.271871+0000","last_change":"2024-05-27T07:41:04.800373+0000","last_active":"2024-05-27T07:44:29.271871+0000","last_peered":"2024-05-27T07:44:29.271871+0000","last_clean":"2024-05-27T07:44:29.271871+0000","last_became_active":"2024-05-27T07:41:04.799930+0000","last_became_peered":"2024-05-27T07:41:04.799930+0000","last_unstale":"2024-05-27T07:44:29.271871+0000","last_undegraded":"2024-05-27T07:44:29.271871+0000","last_fullsized":"2024-05-27T07:44:29.271871+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:24.642479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.346946+0000","last_change":"2024-05-27T07:41:08.830387+0000","last_active":"2024-05-27T07:44:28.346946+0000","last_peered":"2024-05-27T07:44:28.346946+0000","last_clean":"2024-05-27T07:44:28.346946+0000","last_became_active":"2024-05-27T07:41:08.829720+0000","last_became_peered":"2024-05-27T07:41:08.829720+0000","last_unstale":"2024-05-27T07:44:28.346946+0000","last_undegraded":"2024-05-27T07:44:28.346946+0000","last_fullsized":"2024-05-27T07:44:28.346946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:42.014383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013538+0000","last_change":"2024-05-27T07:41:22.005700+0000","last_active":"2024-05-27T07:43:00.013538+0000","last_peered":"2024-05-27T07:43:00.013538+0000","last_clean":"2024-05-27T07:43:00.013538+0000","last_became_active":"2024-05-27T07:41:22.003643+0000","last_became_peered":"2024-05-27T07:41:22.003643+0000","last_unstale":"2024-05-27T07:43:00.013538+0000","last_undegraded":"2024-05-27T07:43:00.013538+0000","last_fullsized":"2024-05-27T07:43:00.013538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:48.332767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.034703+0000","last_change":"2024-05-27T07:41:59.790579+0000","last_active":"2024-05-27T07:43:00.034703+0000","last_peered":"2024-05-27T07:43:00.034703+0000","last_clean":"2024-05-27T07:43:00.034703+0000","last_became_active":"2024-05-27T07:41:59.789849+0000","last_became_peered":"2024-05-27T07:41:59.789849+0000","last_unstale":"2024-05-27T07:43:00.034703+0000","last_undegraded":"2024-05-27T07:43:00.034703+0000","last_fullsized":"2024-05-27T07:43:00.034703+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:24:30.564074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.344468+0000","last_change":"2024-05-27T07:43:00.121063+0000","last_active":"2024-05-27T07:44:28.344468+0000","last_peered":"2024-05-27T07:44:28.344468+0000","last_clean":"2024-05-27T07:44:28.344468+0000","last_became_active":"2024-05-27T07:43:00.119977+0000","last_became_peered":"2024-05-27T07:43:00.119977+0000","last_unstale":"2024-05-27T07:44:28.344468+0000","last_undegraded":"2024-05-27T07:44:28.344468+0000","last_fullsized":"2024-05-27T07:44:28.344468+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:04:43.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195294+0000","last_change":"2024-05-27T07:41:22.013820+0000","last_active":"2024-05-27T07:43:00.195294+0000","last_peered":"2024-05-27T07:43:00.195294+0000","last_clean":"2024-05-27T07:43:00.195294+0000","last_became_active":"2024-05-27T07:41:22.013110+0000","last_became_peered":"2024-05-27T07:41:22.013110+0000","last_unstale":"2024-05-27T07:43:00.195294+0000","last_undegraded":"2024-05-27T07:43:00.195294+0000","last_fullsized":"2024-05-27T07:43:00.195294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:45:07.317872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018376+0000","last_change":"2024-05-27T07:41:04.789197+0000","last_active":"2024-05-27T07:43:00.018376+0000","last_peered":"2024-05-27T07:43:00.018376+0000","last_clean":"2024-05-27T07:43:00.018376+0000","last_became_active":"2024-05-27T07:41:04.788755+0000","last_became_peered":"2024-05-27T07:41:04.788755+0000","last_unstale":"2024-05-27T07:43:00.018376+0000","last_undegraded":"2024-05-27T07:43:00.018376+0000","last_fullsized":"2024-05-27T07:43:00.018376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:27:54.526910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.349800+0000","last_change":"2024-05-27T07:41:08.841769+0000","last_active":"2024-05-27T07:44:28.349800+0000","last_peered":"2024-05-27T07:44:28.349800+0000","last_clean":"2024-05-27T07:44:28.349800+0000","last_became_active":"2024-05-27T07:41:08.840834+0000","last_became_peered":"2024-05-27T07:41:08.840834+0000","last_unstale":"2024-05-27T07:44:28.349800+0000","last_undegraded":"2024-05-27T07:44:28.349800+0000","last_fullsized":"2024-05-27T07:44:28.349800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:15.573739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193500+0000","last_change":"2024-05-27T07:41:22.005119+0000","last_active":"2024-05-27T07:43:00.193500+0000","last_peered":"2024-05-27T07:43:00.193500+0000","last_clean":"2024-05-27T07:43:00.193500+0000","last_became_active":"2024-05-27T07:41:22.002638+0000","last_became_peered":"2024-05-27T07:41:22.002638+0000","last_unstale":"2024-05-27T07:43:00.193500+0000","last_undegraded":"2024-05-27T07:43:00.193500+0000","last_fullsized":"2024-05-27T07:43:00.193500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:08:49.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041632+0000","last_change":"2024-05-27T07:41:04.799864+0000","last_active":"2024-05-27T07:43:00.041632+0000","last_peered":"2024-05-27T07:43:00.041632+0000","last_clean":"2024-05-27T07:43:00.041632+0000","last_became_active":"2024-05-27T07:41:04.799218+0000","last_became_peered":"2024-05-27T07:41:04.799218+0000","last_unstale":"2024-05-27T07:43:00.041632+0000","last_undegraded":"2024-05-27T07:43:00.041632+0000","last_fullsized":"2024-05-27T07:43:00.041632+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:15.355720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.345926+0000","last_change":"2024-05-27T07:41:08.823387+0000","last_active":"2024-05-27T07:44:28.345926+0000","last_peered":"2024-05-27T07:44:28.345926+0000","last_clean":"2024-05-27T07:44:28.345926+0000","last_became_active":"2024-05-27T07:41:08.822943+0000","last_became_peered":"2024-05-27T07:41:08.822943+0000","last_unstale":"2024-05-27T07:44:28.345926+0000","last_undegraded":"2024-05-27T07:44:28.345926+0000","last_fullsized":"2024-05-27T07:44:28.345926+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:46.365284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117402+0000","last_change":"2024-05-27T07:41:22.011848+0000","last_active":"2024-05-27T07:43:00.117402+0000","last_peered":"2024-05-27T07:43:00.117402+0000","last_clean":"2024-05-27T07:43:00.117402+0000","last_became_active":"2024-05-27T07:41:22.011335+0000","last_became_peered":"2024-05-27T07:41:22.011335+0000","last_unstale":"2024-05-27T07:43:00.117402+0000","last_undegraded":"2024-05-27T07:43:00.117402+0000","last_fullsized":"2024-05-27T07:43:00.117402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:29:22.528270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":280,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:29.270022+0000","last_change":"2024-05-27T07:41:04.793877+0000","last_active":"2024-05-27T07:44:29.270022+0000","last_peered":"2024-05-27T07:44:29.270022+0000","last_clean":"2024-05-27T07:44:29.270022+0000","last_became_active":"2024-05-27T07:41:04.793389+0000","last_became_peered":"2024-05-27T07:41:04.793389+0000","last_unstale":"2024-05-27T07:44:29.270022+0000","last_undegraded":"2024-05-27T07:44:29.270022+0000","last_fullsized":"2024-05-27T07:44:29.270022+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:48.670572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.365831+0000","last_change":"2024-05-27T07:41:08.836682+0000","last_active":"2024-05-27T07:44:28.365831+0000","last_peered":"2024-05-27T07:44:28.365831+0000","last_clean":"2024-05-27T07:44:28.365831+0000","last_became_active":"2024-05-27T07:41:08.836229+0000","last_became_peered":"2024-05-27T07:41:08.836229+0000","last_unstale":"2024-05-27T07:44:28.365831+0000","last_undegraded":"2024-05-27T07:44:28.365831+0000","last_fullsized":"2024-05-27T07:44:28.365831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:33.249584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.008846+0000","last_change":"2024-05-27T07:41:22.004968+0000","last_active":"2024-05-27T07:43:00.008846+0000","last_peered":"2024-05-27T07:43:00.008846+0000","last_clean":"2024-05-27T07:43:00.008846+0000","last_became_active":"2024-05-27T07:41:22.001407+0000","last_became_peered":"2024-05-27T07:41:22.001407+0000","last_unstale":"2024-05-27T07:43:00.008846+0000","last_undegraded":"2024-05-27T07:43:00.008846+0000","last_fullsized":"2024-05-27T07:43:00.008846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:26.872497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195809+0000","last_change":"2024-05-27T07:41:59.787247+0000","last_active":"2024-05-27T07:43:00.195809+0000","last_peered":"2024-05-27T07:43:00.195809+0000","last_clean":"2024-05-27T07:43:00.195809+0000","last_became_active":"2024-05-27T07:41:59.786392+0000","last_became_peered":"2024-05-27T07:41:59.786392+0000","last_unstale":"2024-05-27T07:43:00.195809+0000","last_undegraded":"2024-05-27T07:43:00.195809+0000","last_fullsized":"2024-05-27T07:43:00.195809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:12.579707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2],"acting":[11,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.350932+0000","last_change":"2024-05-27T07:41:08.833731+0000","last_active":"2024-05-27T07:44:28.350932+0000","last_peered":"2024-05-27T07:44:28.350932+0000","last_clean":"2024-05-27T07:44:28.350932+0000","last_became_active":"2024-05-27T07:41:08.833318+0000","last_became_peered":"2024-05-27T07:41:08.833318+0000","last_unstale":"2024-05-27T07:44:28.350932+0000","last_undegraded":"2024-05-27T07:44:28.350932+0000","last_fullsized":"2024-05-27T07:44:28.350932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:50.085113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012443+0000","last_change":"2024-05-27T07:41:59.789784+0000","last_active":"2024-05-27T07:43:00.012443+0000","last_peered":"2024-05-27T07:43:00.012443+0000","last_clean":"2024-05-27T07:43:00.012443+0000","last_became_active":"2024-05-27T07:41:59.788589+0000","last_became_peered":"2024-05-27T07:41:59.788589+0000","last_unstale":"2024-05-27T07:43:00.012443+0000","last_undegraded":"2024-05-27T07:43:00.012443+0000","last_fullsized":"2024-05-27T07:43:00.012443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:37.026907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042497+0000","last_change":"2024-05-27T07:41:04.892603+0000","last_active":"2024-05-27T07:43:00.042497+0000","last_peered":"2024-05-27T07:43:00.042497+0000","last_clean":"2024-05-27T07:43:00.042497+0000","last_became_active":"2024-05-27T07:41:04.890786+0000","last_became_peered":"2024-05-27T07:41:04.890786+0000","last_unstale":"2024-05-27T07:43:00.042497+0000","last_undegraded":"2024-05-27T07:43:00.042497+0000","last_fullsized":"2024-05-27T07:43:00.042497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:34.129227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.366090+0000","last_change":"2024-05-27T07:41:08.830802+0000","last_active":"2024-05-27T07:44:28.366090+0000","last_peered":"2024-05-27T07:44:28.366090+0000","last_clean":"2024-05-27T07:44:28.366090+0000","last_became_active":"2024-05-27T07:41:08.829661+0000","last_became_peered":"2024-05-27T07:41:08.829661+0000","last_unstale":"2024-05-27T07:44:28.366090+0000","last_undegraded":"2024-05-27T07:44:28.366090+0000","last_fullsized":"2024-05-27T07:44:28.366090+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:20:15.041554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009108+0000","last_change":"2024-05-27T07:41:22.003997+0000","last_active":"2024-05-27T07:43:00.009108+0000","last_peered":"2024-05-27T07:43:00.009108+0000","last_clean":"2024-05-27T07:43:00.009108+0000","last_became_active":"2024-05-27T07:41:22.003527+0000","last_became_peered":"2024-05-27T07:41:22.003527+0000","last_unstale":"2024-05-27T07:43:00.009108+0000","last_undegraded":"2024-05-27T07:43:00.009108+0000","last_fullsized":"2024-05-27T07:43:00.009108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:17.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":344,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:29.333985+0000","last_change":"2024-05-27T07:41:59.814227+0000","last_active":"2024-05-27T07:44:29.333985+0000","last_peered":"2024-05-27T07:44:29.333985+0000","last_clean":"2024-05-27T07:44:29.333985+0000","last_became_active":"2024-05-27T07:41:59.788510+0000","last_became_peered":"2024-05-27T07:41:59.788510+0000","last_unstale":"2024-05-27T07:44:29.333985+0000","last_undegraded":"2024-05-27T07:44:29.333985+0000","last_fullsized":"2024-05-27T07:44:29.333985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:15.868348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67283,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":242,"num_read_kb":187,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347183+0000","last_change":"2024-05-27T07:41:08.831639+0000","last_active":"2024-05-27T07:44:28.347183+0000","last_peered":"2024-05-27T07:44:28.347183+0000","last_clean":"2024-05-27T07:44:28.347183+0000","last_became_active":"2024-05-27T07:41:08.830541+0000","last_became_peered":"2024-05-27T07:41:08.830541+0000","last_unstale":"2024-05-27T07:44:28.347183+0000","last_undegraded":"2024-05-27T07:44:28.347183+0000","last_fullsized":"2024-05-27T07:44:28.347183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:45.635145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.043291+0000","last_change":"2024-05-27T07:41:21.999587+0000","last_active":"2024-05-27T07:43:00.043291+0000","last_peered":"2024-05-27T07:43:00.043291+0000","last_clean":"2024-05-27T07:43:00.043291+0000","last_became_active":"2024-05-27T07:41:21.999017+0000","last_became_peered":"2024-05-27T07:41:21.999017+0000","last_unstale":"2024-05-27T07:43:00.043291+0000","last_undegraded":"2024-05-27T07:43:00.043291+0000","last_fullsized":"2024-05-27T07:43:00.043291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:31:14.915229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013744+0000","last_change":"2024-05-27T07:41:04.799847+0000","last_active":"2024-05-27T07:43:00.013744+0000","last_peered":"2024-05-27T07:43:00.013744+0000","last_clean":"2024-05-27T07:43:00.013744+0000","last_became_active":"2024-05-27T07:41:04.798605+0000","last_became_peered":"2024-05-27T07:41:04.798605+0000","last_unstale":"2024-05-27T07:43:00.013744+0000","last_undegraded":"2024-05-27T07:43:00.013744+0000","last_fullsized":"2024-05-27T07:43:00.013744+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:01.957707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.352892+0000","last_change":"2024-05-27T07:41:08.831138+0000","last_active":"2024-05-27T07:44:28.352892+0000","last_peered":"2024-05-27T07:44:28.352892+0000","last_clean":"2024-05-27T07:44:28.352892+0000","last_became_active":"2024-05-27T07:41:08.830735+0000","last_became_peered":"2024-05-27T07:41:08.830735+0000","last_unstale":"2024-05-27T07:44:28.352892+0000","last_undegraded":"2024-05-27T07:44:28.352892+0000","last_fullsized":"2024-05-27T07:44:28.352892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:12.569687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194622+0000","last_change":"2024-05-27T07:41:59.779039+0000","last_active":"2024-05-27T07:43:00.194622+0000","last_peered":"2024-05-27T07:43:00.194622+0000","last_clean":"2024-05-27T07:43:00.194622+0000","last_became_active":"2024-05-27T07:41:59.777514+0000","last_became_peered":"2024-05-27T07:41:59.777514+0000","last_unstale":"2024-05-27T07:43:00.194622+0000","last_undegraded":"2024-05-27T07:43:00.194622+0000","last_fullsized":"2024-05-27T07:43:00.194622+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:12:19.223874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042492+0000","last_change":"2024-05-27T07:41:04.796467+0000","last_active":"2024-05-27T07:43:00.042492+0000","last_peered":"2024-05-27T07:43:00.042492+0000","last_clean":"2024-05-27T07:43:00.042492+0000","last_became_active":"2024-05-27T07:41:04.795519+0000","last_became_peered":"2024-05-27T07:41:04.795519+0000","last_unstale":"2024-05-27T07:43:00.042492+0000","last_undegraded":"2024-05-27T07:43:00.042492+0000","last_fullsized":"2024-05-27T07:43:00.042492+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:42.744076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.353662+0000","last_change":"2024-05-27T07:41:08.832135+0000","last_active":"2024-05-27T07:44:28.353662+0000","last_peered":"2024-05-27T07:44:28.353662+0000","last_clean":"2024-05-27T07:44:28.353662+0000","last_became_active":"2024-05-27T07:41:08.830032+0000","last_became_peered":"2024-05-27T07:41:08.830032+0000","last_unstale":"2024-05-27T07:44:28.353662+0000","last_undegraded":"2024-05-27T07:44:28.353662+0000","last_fullsized":"2024-05-27T07:44:28.353662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:44:59.367026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009932+0000","last_change":"2024-05-27T07:41:22.008868+0000","last_active":"2024-05-27T07:43:00.009932+0000","last_peered":"2024-05-27T07:43:00.009932+0000","last_clean":"2024-05-27T07:43:00.009932+0000","last_became_active":"2024-05-27T07:41:22.002071+0000","last_became_peered":"2024-05-27T07:41:22.002071+0000","last_unstale":"2024-05-27T07:43:00.009932+0000","last_undegraded":"2024-05-27T07:43:00.009932+0000","last_fullsized":"2024-05-27T07:43:00.009932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:20.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":171,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.323191+0000","last_change":"2024-05-27T07:41:04.796198+0000","last_active":"2024-05-27T07:44:28.323191+0000","last_peered":"2024-05-27T07:44:28.323191+0000","last_clean":"2024-05-27T07:44:28.323191+0000","last_became_active":"2024-05-27T07:41:04.795646+0000","last_became_peered":"2024-05-27T07:41:04.795646+0000","last_unstale":"2024-05-27T07:44:28.323191+0000","last_undegraded":"2024-05-27T07:44:28.323191+0000","last_fullsized":"2024-05-27T07:44:28.323191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:30.936596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348243+0000","last_change":"2024-05-27T07:41:08.840518+0000","last_active":"2024-05-27T07:44:28.348243+0000","last_peered":"2024-05-27T07:44:28.348243+0000","last_clean":"2024-05-27T07:44:28.348243+0000","last_became_active":"2024-05-27T07:41:08.840162+0000","last_became_peered":"2024-05-27T07:41:08.840162+0000","last_unstale":"2024-05-27T07:44:28.348243+0000","last_undegraded":"2024-05-27T07:44:28.348243+0000","last_fullsized":"2024-05-27T07:44:28.348243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:04:14.657496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018999+0000","last_change":"2024-05-27T07:41:22.000243+0000","last_active":"2024-05-27T07:43:00.018999+0000","last_peered":"2024-05-27T07:43:00.018999+0000","last_clean":"2024-05-27T07:43:00.018999+0000","last_became_active":"2024-05-27T07:41:21.998898+0000","last_became_peered":"2024-05-27T07:41:21.998898+0000","last_unstale":"2024-05-27T07:43:00.018999+0000","last_undegraded":"2024-05-27T07:43:00.018999+0000","last_fullsized":"2024-05-27T07:43:00.018999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:35.696593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":79,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195497+0000","last_change":"2024-05-27T07:41:59.799662+0000","last_active":"2024-05-27T07:43:00.195497+0000","last_peered":"2024-05-27T07:43:00.195497+0000","last_clean":"2024-05-27T07:43:00.195497+0000","last_became_active":"2024-05-27T07:41:59.779733+0000","last_became_peered":"2024-05-27T07:41:59.779733+0000","last_unstale":"2024-05-27T07:43:00.195497+0000","last_undegraded":"2024-05-27T07:43:00.195497+0000","last_fullsized":"2024-05-27T07:43:00.195497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:23.665014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.353774+0000","last_change":"2024-05-27T07:41:08.834931+0000","last_active":"2024-05-27T07:44:28.353774+0000","last_peered":"2024-05-27T07:44:28.353774+0000","last_clean":"2024-05-27T07:44:28.353774+0000","last_became_active":"2024-05-27T07:41:08.833298+0000","last_became_peered":"2024-05-27T07:41:08.833298+0000","last_unstale":"2024-05-27T07:44:28.353774+0000","last_undegraded":"2024-05-27T07:44:28.353774+0000","last_fullsized":"2024-05-27T07:44:28.353774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:03:48.507461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042243+0000","last_change":"2024-05-27T07:41:22.009407+0000","last_active":"2024-05-27T07:43:00.042243+0000","last_peered":"2024-05-27T07:43:00.042243+0000","last_clean":"2024-05-27T07:43:00.042243+0000","last_became_active":"2024-05-27T07:41:22.006921+0000","last_became_peered":"2024-05-27T07:41:22.006921+0000","last_unstale":"2024-05-27T07:43:00.042243+0000","last_undegraded":"2024-05-27T07:43:00.042243+0000","last_fullsized":"2024-05-27T07:43:00.042243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:24.034502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196609+0000","last_change":"2024-05-27T07:41:04.795641+0000","last_active":"2024-05-27T07:43:00.196609+0000","last_peered":"2024-05-27T07:43:00.196609+0000","last_clean":"2024-05-27T07:43:00.196609+0000","last_became_active":"2024-05-27T07:41:04.794935+0000","last_became_peered":"2024-05-27T07:41:04.794935+0000","last_unstale":"2024-05-27T07:43:00.196609+0000","last_undegraded":"2024-05-27T07:43:00.196609+0000","last_fullsized":"2024-05-27T07:43:00.196609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:14.850282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348046+0000","last_change":"2024-05-27T07:41:08.837277+0000","last_active":"2024-05-27T07:44:28.348046+0000","last_peered":"2024-05-27T07:44:28.348046+0000","last_clean":"2024-05-27T07:44:28.348046+0000","last_became_active":"2024-05-27T07:41:08.835935+0000","last_became_peered":"2024-05-27T07:41:08.835935+0000","last_unstale":"2024-05-27T07:44:28.348046+0000","last_undegraded":"2024-05-27T07:44:28.348046+0000","last_fullsized":"2024-05-27T07:44:28.348046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:46.436243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196250+0000","last_change":"2024-05-27T07:41:22.014700+0000","last_active":"2024-05-27T07:43:00.196250+0000","last_peered":"2024-05-27T07:43:00.196250+0000","last_clean":"2024-05-27T07:43:00.196250+0000","last_became_active":"2024-05-27T07:41:22.014136+0000","last_became_peered":"2024-05-27T07:41:22.014136+0000","last_unstale":"2024-05-27T07:43:00.196250+0000","last_undegraded":"2024-05-27T07:43:00.196250+0000","last_fullsized":"2024-05-27T07:43:00.196250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:51:08.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.009402+0000","last_change":"2024-05-27T07:41:04.891269+0000","last_active":"2024-05-27T07:43:00.009402+0000","last_peered":"2024-05-27T07:43:00.009402+0000","last_clean":"2024-05-27T07:43:00.009402+0000","last_became_active":"2024-05-27T07:41:04.890843+0000","last_became_peered":"2024-05-27T07:41:04.890843+0000","last_unstale":"2024-05-27T07:43:00.009402+0000","last_undegraded":"2024-05-27T07:43:00.009402+0000","last_fullsized":"2024-05-27T07:43:00.009402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:57.065300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347426+0000","last_change":"2024-05-27T07:41:08.841977+0000","last_active":"2024-05-27T07:44:28.347426+0000","last_peered":"2024-05-27T07:44:28.347426+0000","last_clean":"2024-05-27T07:44:28.347426+0000","last_became_active":"2024-05-27T07:41:08.841551+0000","last_became_peered":"2024-05-27T07:41:08.841551+0000","last_unstale":"2024-05-27T07:44:28.347426+0000","last_undegraded":"2024-05-27T07:44:28.347426+0000","last_fullsized":"2024-05-27T07:44:28.347426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:35:07.766193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.020285+0000","last_change":"2024-05-27T07:41:22.006310+0000","last_active":"2024-05-27T07:43:00.020285+0000","last_peered":"2024-05-27T07:43:00.020285+0000","last_clean":"2024-05-27T07:43:00.020285+0000","last_became_active":"2024-05-27T07:41:22.004135+0000","last_became_peered":"2024-05-27T07:41:22.004135+0000","last_unstale":"2024-05-27T07:43:00.020285+0000","last_undegraded":"2024-05-27T07:43:00.020285+0000","last_fullsized":"2024-05-27T07:43:00.020285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:48.564894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.247040+0000","last_change":"2024-05-27T07:41:04.793284+0000","last_active":"2024-05-27T07:44:28.247040+0000","last_peered":"2024-05-27T07:44:28.247040+0000","last_clean":"2024-05-27T07:44:28.247040+0000","last_became_active":"2024-05-27T07:41:04.792853+0000","last_became_peered":"2024-05-27T07:41:04.792853+0000","last_unstale":"2024-05-27T07:44:28.247040+0000","last_undegraded":"2024-05-27T07:44:28.247040+0000","last_fullsized":"2024-05-27T07:44:28.247040+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:22.979341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.336734+0000","last_change":"2024-05-27T07:41:59.776207+0000","last_active":"2024-05-27T07:44:28.336734+0000","last_peered":"2024-05-27T07:44:28.336734+0000","last_clean":"2024-05-27T07:44:28.336734+0000","last_became_active":"2024-05-27T07:41:59.775226+0000","last_became_peered":"2024-05-27T07:41:59.775226+0000","last_unstale":"2024-05-27T07:44:28.336734+0000","last_undegraded":"2024-05-27T07:44:28.336734+0000","last_fullsized":"2024-05-27T07:44:28.336734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:50.469144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.021403+0000","last_change":"2024-05-27T07:41:21.992013+0000","last_active":"2024-05-27T07:43:00.021403+0000","last_peered":"2024-05-27T07:43:00.021403+0000","last_clean":"2024-05-27T07:43:00.021403+0000","last_became_active":"2024-05-27T07:41:21.991550+0000","last_became_peered":"2024-05-27T07:41:21.991550+0000","last_unstale":"2024-05-27T07:43:00.021403+0000","last_undegraded":"2024-05-27T07:43:00.021403+0000","last_fullsized":"2024-05-27T07:43:00.021403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:34:53.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347945+0000","last_change":"2024-05-27T07:41:08.832512+0000","last_active":"2024-05-27T07:44:28.347945+0000","last_peered":"2024-05-27T07:44:28.347945+0000","last_clean":"2024-05-27T07:44:28.347945+0000","last_became_active":"2024-05-27T07:41:08.832113+0000","last_became_peered":"2024-05-27T07:41:08.832113+0000","last_unstale":"2024-05-27T07:44:28.347945+0000","last_undegraded":"2024-05-27T07:44:28.347945+0000","last_fullsized":"2024-05-27T07:44:28.347945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:00.530785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":169,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:30.335711+0000","last_change":"2024-05-27T07:41:04.891004+0000","last_active":"2024-05-27T07:44:30.335711+0000","last_peered":"2024-05-27T07:44:30.335711+0000","last_clean":"2024-05-27T07:44:30.335711+0000","last_became_active":"2024-05-27T07:41:04.890567+0000","last_became_peered":"2024-05-27T07:41:04.890567+0000","last_unstale":"2024-05-27T07:44:30.335711+0000","last_undegraded":"2024-05-27T07:44:30.335711+0000","last_fullsized":"2024-05-27T07:44:30.335711+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:43.991204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.196216+0000","last_change":"2024-05-27T07:41:22.003446+0000","last_active":"2024-05-27T07:43:00.196216+0000","last_peered":"2024-05-27T07:43:00.196216+0000","last_clean":"2024-05-27T07:43:00.196216+0000","last_became_active":"2024-05-27T07:41:22.002989+0000","last_became_peered":"2024-05-27T07:41:22.002989+0000","last_unstale":"2024-05-27T07:43:00.196216+0000","last_undegraded":"2024-05-27T07:43:00.196216+0000","last_fullsized":"2024-05-27T07:43:00.196216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:12.996286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.351063+0000","last_change":"2024-05-27T07:41:08.842685+0000","last_active":"2024-05-27T07:44:28.351063+0000","last_peered":"2024-05-27T07:44:28.351063+0000","last_clean":"2024-05-27T07:44:28.351063+0000","last_became_active":"2024-05-27T07:41:08.841954+0000","last_became_peered":"2024-05-27T07:41:08.841954+0000","last_unstale":"2024-05-27T07:44:28.351063+0000","last_undegraded":"2024-05-27T07:44:28.351063+0000","last_fullsized":"2024-05-27T07:44:28.351063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:18.087129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013268+0000","last_change":"2024-05-27T07:41:04.790330+0000","last_active":"2024-05-27T07:43:00.013268+0000","last_peered":"2024-05-27T07:43:00.013268+0000","last_clean":"2024-05-27T07:43:00.013268+0000","last_became_active":"2024-05-27T07:41:04.789508+0000","last_became_peered":"2024-05-27T07:41:04.789508+0000","last_unstale":"2024-05-27T07:43:00.013268+0000","last_undegraded":"2024-05-27T07:43:00.013268+0000","last_fullsized":"2024-05-27T07:43:00.013268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:27.537662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194532+0000","last_change":"2024-05-27T07:39:11.797324+0000","last_active":"2024-05-27T07:43:00.194532+0000","last_peered":"2024-05-27T07:43:00.194532+0000","last_clean":"2024-05-27T07:43:00.194532+0000","last_became_active":"2024-05-27T07:39:11.786294+0000","last_became_peered":"2024-05-27T07:39:11.786294+0000","last_unstale":"2024-05-27T07:43:00.194532+0000","last_undegraded":"2024-05-27T07:43:00.194532+0000","last_fullsized":"2024-05-27T07:43:00.194532+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_clean_scrub_stamp":"2024-05-27T07:34:18.613413+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:23.947615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.042739+0000","last_change":"2024-05-27T07:41:21.997453+0000","last_active":"2024-05-27T07:43:00.042739+0000","last_peered":"2024-05-27T07:43:00.042739+0000","last_clean":"2024-05-27T07:43:00.042739+0000","last_became_active":"2024-05-27T07:41:21.997010+0000","last_became_peered":"2024-05-27T07:41:21.997010+0000","last_unstale":"2024-05-27T07:43:00.042739+0000","last_undegraded":"2024-05-27T07:43:00.042739+0000","last_fullsized":"2024-05-27T07:43:00.042739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:54:54.913274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.344451+0000","last_change":"2024-05-27T07:43:00.024988+0000","last_active":"2024-05-27T07:44:28.344451+0000","last_peered":"2024-05-27T07:44:28.344451+0000","last_clean":"2024-05-27T07:44:28.344451+0000","last_became_active":"2024-05-27T07:43:00.023816+0000","last_became_peered":"2024-05-27T07:43:00.023816+0000","last_unstale":"2024-05-27T07:44:28.344451+0000","last_undegraded":"2024-05-27T07:44:28.344451+0000","last_fullsized":"2024-05-27T07:44:28.344451+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:58.531985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194923+0000","last_change":"2024-05-27T07:41:04.795262+0000","last_active":"2024-05-27T07:43:00.194923+0000","last_peered":"2024-05-27T07:43:00.194923+0000","last_clean":"2024-05-27T07:43:00.194923+0000","last_became_active":"2024-05-27T07:41:04.794866+0000","last_became_peered":"2024-05-27T07:41:04.794866+0000","last_unstale":"2024-05-27T07:43:00.194923+0000","last_undegraded":"2024-05-27T07:43:00.194923+0000","last_fullsized":"2024-05-27T07:43:00.194923+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:07:44.241444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194536+0000","last_change":"2024-05-27T07:41:22.006194+0000","last_active":"2024-05-27T07:43:00.194536+0000","last_peered":"2024-05-27T07:43:00.194536+0000","last_clean":"2024-05-27T07:43:00.194536+0000","last_became_active":"2024-05-27T07:41:22.004079+0000","last_became_peered":"2024-05-27T07:41:22.004079+0000","last_unstale":"2024-05-27T07:43:00.194536+0000","last_undegraded":"2024-05-27T07:43:00.194536+0000","last_fullsized":"2024-05-27T07:43:00.194536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:03.496722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194439+0000","last_change":"2024-05-27T07:41:59.776734+0000","last_active":"2024-05-27T07:43:00.194439+0000","last_peered":"2024-05-27T07:43:00.194439+0000","last_clean":"2024-05-27T07:43:00.194439+0000","last_became_active":"2024-05-27T07:41:59.775678+0000","last_became_peered":"2024-05-27T07:41:59.775678+0000","last_unstale":"2024-05-27T07:43:00.194439+0000","last_undegraded":"2024-05-27T07:43:00.194439+0000","last_fullsized":"2024-05-27T07:43:00.194439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:56.007178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.345064+0000","last_change":"2024-05-27T07:41:08.831070+0000","last_active":"2024-05-27T07:44:28.345064+0000","last_peered":"2024-05-27T07:44:28.345064+0000","last_clean":"2024-05-27T07:44:28.345064+0000","last_became_active":"2024-05-27T07:41:08.830617+0000","last_became_peered":"2024-05-27T07:41:08.830617+0000","last_unstale":"2024-05-27T07:44:28.345064+0000","last_undegraded":"2024-05-27T07:44:28.345064+0000","last_fullsized":"2024-05-27T07:44:28.345064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:13.046162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.116733+0000","last_change":"2024-05-27T07:41:22.010129+0000","last_active":"2024-05-27T07:43:00.116733+0000","last_peered":"2024-05-27T07:43:00.116733+0000","last_clean":"2024-05-27T07:43:00.116733+0000","last_became_active":"2024-05-27T07:41:22.009608+0000","last_became_peered":"2024-05-27T07:41:22.009608+0000","last_unstale":"2024-05-27T07:43:00.116733+0000","last_undegraded":"2024-05-27T07:43:00.116733+0000","last_fullsized":"2024-05-27T07:43:00.116733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.225004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:17.942153+0000","last_change":"2024-05-27T07:41:04.794744+0000","last_active":"2024-05-27T07:43:17.942153+0000","last_peered":"2024-05-27T07:43:17.942153+0000","last_clean":"2024-05-27T07:43:17.942153+0000","last_became_active":"2024-05-27T07:41:04.794340+0000","last_became_peered":"2024-05-27T07:41:04.794340+0000","last_unstale":"2024-05-27T07:43:17.942153+0000","last_undegraded":"2024-05-27T07:43:17.942153+0000","last_fullsized":"2024-05-27T07:43:17.942153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:43:20.335520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.346880+0000","last_change":"2024-05-27T07:41:08.841834+0000","last_active":"2024-05-27T07:44:28.346880+0000","last_peered":"2024-05-27T07:44:28.346880+0000","last_clean":"2024-05-27T07:44:28.346880+0000","last_became_active":"2024-05-27T07:41:08.840261+0000","last_became_peered":"2024-05-27T07:41:08.840261+0000","last_unstale":"2024-05-27T07:44:28.346880+0000","last_undegraded":"2024-05-27T07:44:28.346880+0000","last_fullsized":"2024-05-27T07:44:28.346880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:33.738835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012235+0000","last_change":"2024-05-27T07:41:22.005861+0000","last_active":"2024-05-27T07:43:00.012235+0000","last_peered":"2024-05-27T07:43:00.012235+0000","last_clean":"2024-05-27T07:43:00.012235+0000","last_became_active":"2024-05-27T07:41:22.005295+0000","last_became_peered":"2024-05-27T07:41:22.005295+0000","last_unstale":"2024-05-27T07:43:00.012235+0000","last_undegraded":"2024-05-27T07:43:00.012235+0000","last_fullsized":"2024-05-27T07:43:00.012235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:46.582656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013605+0000","last_change":"2024-05-27T07:41:04.796562+0000","last_active":"2024-05-27T07:43:00.013605+0000","last_peered":"2024-05-27T07:43:00.013605+0000","last_clean":"2024-05-27T07:43:00.013605+0000","last_became_active":"2024-05-27T07:41:04.795538+0000","last_became_peered":"2024-05-27T07:41:04.795538+0000","last_unstale":"2024-05-27T07:43:00.013605+0000","last_undegraded":"2024-05-27T07:43:00.013605+0000","last_fullsized":"2024-05-27T07:43:00.013605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:12.131999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.349624+0000","last_change":"2024-05-27T07:41:08.832824+0000","last_active":"2024-05-27T07:44:28.349624+0000","last_peered":"2024-05-27T07:44:28.349624+0000","last_clean":"2024-05-27T07:44:28.349624+0000","last_became_active":"2024-05-27T07:41:08.832332+0000","last_became_peered":"2024-05-27T07:41:08.832332+0000","last_unstale":"2024-05-27T07:44:28.349624+0000","last_undegraded":"2024-05-27T07:44:28.349624+0000","last_fullsized":"2024-05-27T07:44:28.349624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:34.598773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.011831+0000","last_change":"2024-05-27T07:41:22.003358+0000","last_active":"2024-05-27T07:43:00.011831+0000","last_peered":"2024-05-27T07:43:00.011831+0000","last_clean":"2024-05-27T07:43:00.011831+0000","last_became_active":"2024-05-27T07:41:22.002655+0000","last_became_peered":"2024-05-27T07:41:22.002655+0000","last_unstale":"2024-05-27T07:43:00.011831+0000","last_undegraded":"2024-05-27T07:43:00.011831+0000","last_fullsized":"2024-05-27T07:43:00.011831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:41.177138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.193770+0000","last_change":"2024-05-27T07:41:04.888681+0000","last_active":"2024-05-27T07:43:00.193770+0000","last_peered":"2024-05-27T07:43:00.193770+0000","last_clean":"2024-05-27T07:43:00.193770+0000","last_became_active":"2024-05-27T07:41:04.887966+0000","last_became_peered":"2024-05-27T07:41:04.887966+0000","last_unstale":"2024-05-27T07:43:00.193770+0000","last_undegraded":"2024-05-27T07:43:00.193770+0000","last_fullsized":"2024-05-27T07:43:00.193770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:01.178737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.296531+0000","last_change":"2024-05-27T07:41:08.827839+0000","last_active":"2024-05-27T07:44:28.296531+0000","last_peered":"2024-05-27T07:44:28.296531+0000","last_clean":"2024-05-27T07:44:28.296531+0000","last_became_active":"2024-05-27T07:41:08.827403+0000","last_became_peered":"2024-05-27T07:41:08.827403+0000","last_unstale":"2024-05-27T07:44:28.296531+0000","last_undegraded":"2024-05-27T07:44:28.296531+0000","last_fullsized":"2024-05-27T07:44:28.296531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:01:42.075980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.010155+0000","last_change":"2024-05-27T07:41:22.006161+0000","last_active":"2024-05-27T07:43:00.010155+0000","last_peered":"2024-05-27T07:43:00.010155+0000","last_clean":"2024-05-27T07:43:00.010155+0000","last_became_active":"2024-05-27T07:41:22.005762+0000","last_became_peered":"2024-05-27T07:41:22.005762+0000","last_unstale":"2024-05-27T07:43:00.010155+0000","last_undegraded":"2024-05-27T07:43:00.010155+0000","last_fullsized":"2024-05-27T07:43:00.010155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:53.536444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.117239+0000","last_change":"2024-05-27T07:41:04.774211+0000","last_active":"2024-05-27T07:43:00.117239+0000","last_peered":"2024-05-27T07:43:00.117239+0000","last_clean":"2024-05-27T07:43:00.117239+0000","last_became_active":"2024-05-27T07:41:04.773215+0000","last_became_peered":"2024-05-27T07:41:04.773215+0000","last_unstale":"2024-05-27T07:43:00.117239+0000","last_undegraded":"2024-05-27T07:43:00.117239+0000","last_fullsized":"2024-05-27T07:43:00.117239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:37.273735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.344884+0000","last_change":"2024-05-27T07:41:08.830434+0000","last_active":"2024-05-27T07:44:28.344884+0000","last_peered":"2024-05-27T07:44:28.344884+0000","last_clean":"2024-05-27T07:44:28.344884+0000","last_became_active":"2024-05-27T07:41:08.830039+0000","last_became_peered":"2024-05-27T07:41:08.830039+0000","last_unstale":"2024-05-27T07:44:28.344884+0000","last_undegraded":"2024-05-27T07:44:28.344884+0000","last_fullsized":"2024-05-27T07:44:28.344884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:27.877666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.013140+0000","last_change":"2024-05-27T07:41:59.783413+0000","last_active":"2024-05-27T07:43:00.013140+0000","last_peered":"2024-05-27T07:43:00.013140+0000","last_clean":"2024-05-27T07:43:00.013140+0000","last_became_active":"2024-05-27T07:41:59.782300+0000","last_became_peered":"2024-05-27T07:41:59.782300+0000","last_unstale":"2024-05-27T07:43:00.013140+0000","last_undegraded":"2024-05-27T07:43:00.013140+0000","last_fullsized":"2024-05-27T07:43:00.013140+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:59.348928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012623+0000","last_change":"2024-05-27T07:41:04.798969+0000","last_active":"2024-05-27T07:43:00.012623+0000","last_peered":"2024-05-27T07:43:00.012623+0000","last_clean":"2024-05-27T07:43:00.012623+0000","last_became_active":"2024-05-27T07:41:04.795996+0000","last_became_peered":"2024-05-27T07:41:04.795996+0000","last_unstale":"2024-05-27T07:43:00.012623+0000","last_undegraded":"2024-05-27T07:43:00.012623+0000","last_fullsized":"2024-05-27T07:43:00.012623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:47.141063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.339062+0000","last_change":"2024-05-27T07:41:08.835142+0000","last_active":"2024-05-27T07:44:28.339062+0000","last_peered":"2024-05-27T07:44:28.339062+0000","last_clean":"2024-05-27T07:44:28.339062+0000","last_became_active":"2024-05-27T07:41:08.834762+0000","last_became_peered":"2024-05-27T07:41:08.834762+0000","last_unstale":"2024-05-27T07:44:28.339062+0000","last_undegraded":"2024-05-27T07:44:28.339062+0000","last_fullsized":"2024-05-27T07:44:28.339062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:52.957554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.012439+0000","last_change":"2024-05-27T07:41:22.000649+0000","last_active":"2024-05-27T07:43:00.012439+0000","last_peered":"2024-05-27T07:43:00.012439+0000","last_clean":"2024-05-27T07:43:00.012439+0000","last_became_active":"2024-05-27T07:41:22.000161+0000","last_became_peered":"2024-05-27T07:41:22.000161+0000","last_unstale":"2024-05-27T07:43:00.012439+0000","last_undegraded":"2024-05-27T07:43:00.012439+0000","last_fullsized":"2024-05-27T07:43:00.012439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:10.788027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.195920+0000","last_change":"2024-05-27T07:41:04.800680+0000","last_active":"2024-05-27T07:43:00.195920+0000","last_peered":"2024-05-27T07:43:00.195920+0000","last_clean":"2024-05-27T07:43:00.195920+0000","last_became_active":"2024-05-27T07:41:04.800249+0000","last_became_peered":"2024-05-27T07:41:04.800249+0000","last_unstale":"2024-05-27T07:43:00.195920+0000","last_undegraded":"2024-05-27T07:43:00.195920+0000","last_fullsized":"2024-05-27T07:43:00.195920+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:23:27.797216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.365257+0000","last_change":"2024-05-27T07:41:08.844380+0000","last_active":"2024-05-27T07:44:28.365257+0000","last_peered":"2024-05-27T07:44:28.365257+0000","last_clean":"2024-05-27T07:44:28.365257+0000","last_became_active":"2024-05-27T07:41:08.843464+0000","last_became_peered":"2024-05-27T07:41:08.843464+0000","last_unstale":"2024-05-27T07:44:28.365257+0000","last_undegraded":"2024-05-27T07:44:28.365257+0000","last_fullsized":"2024-05-27T07:44:28.365257+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:23.290234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.194841+0000","last_change":"2024-05-27T07:41:22.010968+0000","last_active":"2024-05-27T07:43:00.194841+0000","last_peered":"2024-05-27T07:43:00.194841+0000","last_clean":"2024-05-27T07:43:00.194841+0000","last_became_active":"2024-05-27T07:41:22.010529+0000","last_became_peered":"2024-05-27T07:41:22.010529+0000","last_unstale":"2024-05-27T07:43:00.194841+0000","last_undegraded":"2024-05-27T07:43:00.194841+0000","last_fullsized":"2024-05-27T07:43:00.194841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:08.636434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":169,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:30.275152+0000","last_change":"2024-05-27T07:41:04.891536+0000","last_active":"2024-05-27T07:44:30.275152+0000","last_peered":"2024-05-27T07:44:30.275152+0000","last_clean":"2024-05-27T07:44:30.275152+0000","last_became_active":"2024-05-27T07:41:04.890724+0000","last_became_peered":"2024-05-27T07:41:04.890724+0000","last_unstale":"2024-05-27T07:44:30.275152+0000","last_undegraded":"2024-05-27T07:44:30.275152+0000","last_fullsized":"2024-05-27T07:44:30.275152+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:32.027624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.362943+0000","last_change":"2024-05-27T07:41:08.834362+0000","last_active":"2024-05-27T07:44:28.362943+0000","last_peered":"2024-05-27T07:44:28.362943+0000","last_clean":"2024-05-27T07:44:28.362943+0000","last_became_active":"2024-05-27T07:41:08.833979+0000","last_became_peered":"2024-05-27T07:41:08.833979+0000","last_unstale":"2024-05-27T07:44:28.362943+0000","last_undegraded":"2024-05-27T07:44:28.362943+0000","last_fullsized":"2024-05-27T07:44:28.362943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:35.098352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.192704+0000","last_change":"2024-05-27T07:41:21.997587+0000","last_active":"2024-05-27T07:43:00.192704+0000","last_peered":"2024-05-27T07:43:00.192704+0000","last_clean":"2024-05-27T07:43:00.192704+0000","last_became_active":"2024-05-27T07:41:21.996413+0000","last_became_peered":"2024-05-27T07:41:21.996413+0000","last_unstale":"2024-05-27T07:43:00.192704+0000","last_undegraded":"2024-05-27T07:43:00.192704+0000","last_fullsized":"2024-05-27T07:43:00.192704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:21:46.728047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.043021+0000","last_change":"2024-05-27T07:41:04.793580+0000","last_active":"2024-05-27T07:43:00.043021+0000","last_peered":"2024-05-27T07:43:00.043021+0000","last_clean":"2024-05-27T07:43:00.043021+0000","last_became_active":"2024-05-27T07:41:04.791135+0000","last_became_peered":"2024-05-27T07:41:04.791135+0000","last_unstale":"2024-05-27T07:43:00.043021+0000","last_undegraded":"2024-05-27T07:43:00.043021+0000","last_fullsized":"2024-05-27T07:43:00.043021+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:15.484187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348081+0000","last_change":"2024-05-27T07:41:08.839187+0000","last_active":"2024-05-27T07:44:28.348081+0000","last_peered":"2024-05-27T07:44:28.348081+0000","last_clean":"2024-05-27T07:44:28.348081+0000","last_became_active":"2024-05-27T07:41:08.838821+0000","last_became_peered":"2024-05-27T07:41:08.838821+0000","last_unstale":"2024-05-27T07:44:28.348081+0000","last_undegraded":"2024-05-27T07:44:28.348081+0000","last_fullsized":"2024-05-27T07:44:28.348081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:30.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.041135+0000","last_change":"2024-05-27T07:41:22.005406+0000","last_active":"2024-05-27T07:43:00.041135+0000","last_peered":"2024-05-27T07:43:00.041135+0000","last_clean":"2024-05-27T07:43:00.041135+0000","last_became_active":"2024-05-27T07:41:22.003415+0000","last_became_peered":"2024-05-27T07:41:22.003415+0000","last_unstale":"2024-05-27T07:43:00.041135+0000","last_undegraded":"2024-05-27T07:43:00.041135+0000","last_fullsized":"2024-05-27T07:43:00.041135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:03.300452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:43:00.018670+0000","last_change":"2024-05-27T07:41:04.798489+0000","last_active":"2024-05-27T07:43:00.018670+0000","last_peered":"2024-05-27T07:43:00.018670+0000","last_clean":"2024-05-27T07:43:00.018670+0000","last_became_active":"2024-05-27T07:41:04.797467+0000","last_became_peered":"2024-05-27T07:41:04.797467+0000","last_unstale":"2024-05-27T07:43:00.018670+0000","last_undegraded":"2024-05-27T07:43:00.018670+0000","last_fullsized":"2024-05-27T07:43:00.018670+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:04.018577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.349083+0000","last_change":"2024-05-27T07:41:08.832215+0000","last_active":"2024-05-27T07:44:28.349083+0000","last_peered":"2024-05-27T07:44:28.349083+0000","last_clean":"2024-05-27T07:44:28.349083+0000","last_became_active":"2024-05-27T07:41:08.831491+0000","last_became_peered":"2024-05-27T07:41:08.831491+0000","last_unstale":"2024-05-27T07:44:28.349083+0000","last_undegraded":"2024-05-27T07:44:28.349083+0000","last_fullsized":"2024-05-27T07:44:28.349083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:32:03.084324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.349005+0000","last_change":"2024-05-27T07:41:08.837301+0000","last_active":"2024-05-27T07:44:28.349005+0000","last_peered":"2024-05-27T07:44:28.349005+0000","last_clean":"2024-05-27T07:44:28.349005+0000","last_became_active":"2024-05-27T07:41:08.835533+0000","last_became_peered":"2024-05-27T07:41:08.835533+0000","last_unstale":"2024-05-27T07:44:28.349005+0000","last_undegraded":"2024-05-27T07:44:28.349005+0000","last_fullsized":"2024-05-27T07:44:28.349005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:45.807613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.344958+0000","last_change":"2024-05-27T07:41:08.846740+0000","last_active":"2024-05-27T07:44:28.344958+0000","last_peered":"2024-05-27T07:44:28.344958+0000","last_clean":"2024-05-27T07:44:28.344958+0000","last_became_active":"2024-05-27T07:41:08.845576+0000","last_became_peered":"2024-05-27T07:41:08.845576+0000","last_unstale":"2024-05-27T07:44:28.344958+0000","last_undegraded":"2024-05-27T07:44:28.344958+0000","last_fullsized":"2024-05-27T07:44:28.344958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:41:36.944923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.338370+0000","last_change":"2024-05-27T07:41:08.840121+0000","last_active":"2024-05-27T07:44:28.338370+0000","last_peered":"2024-05-27T07:44:28.338370+0000","last_clean":"2024-05-27T07:44:28.338370+0000","last_became_active":"2024-05-27T07:41:08.839090+0000","last_became_peered":"2024-05-27T07:41:08.839090+0000","last_unstale":"2024-05-27T07:44:28.338370+0000","last_undegraded":"2024-05-27T07:44:28.338370+0000","last_fullsized":"2024-05-27T07:44:28.338370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:44.836953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347398+0000","last_change":"2024-05-27T07:41:08.835565+0000","last_active":"2024-05-27T07:44:28.347398+0000","last_peered":"2024-05-27T07:44:28.347398+0000","last_clean":"2024-05-27T07:44:28.347398+0000","last_became_active":"2024-05-27T07:41:08.831424+0000","last_became_peered":"2024-05-27T07:41:08.831424+0000","last_unstale":"2024-05-27T07:44:28.347398+0000","last_undegraded":"2024-05-27T07:44:28.347398+0000","last_fullsized":"2024-05-27T07:44:28.347398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:17.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.349943+0000","last_change":"2024-05-27T07:41:08.830104+0000","last_active":"2024-05-27T07:44:28.349943+0000","last_peered":"2024-05-27T07:44:28.349943+0000","last_clean":"2024-05-27T07:44:28.349943+0000","last_became_active":"2024-05-27T07:41:08.829690+0000","last_became_peered":"2024-05-27T07:41:08.829690+0000","last_unstale":"2024-05-27T07:44:28.349943+0000","last_undegraded":"2024-05-27T07:44:28.349943+0000","last_fullsized":"2024-05-27T07:44:28.349943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.347948+0000","last_change":"2024-05-27T07:41:08.838654+0000","last_active":"2024-05-27T07:44:28.347948+0000","last_peered":"2024-05-27T07:44:28.347948+0000","last_clean":"2024-05-27T07:44:28.347948+0000","last_became_active":"2024-05-27T07:41:08.835033+0000","last_became_peered":"2024-05-27T07:41:08.835033+0000","last_unstale":"2024-05-27T07:44:28.347948+0000","last_undegraded":"2024-05-27T07:44:28.347948+0000","last_fullsized":"2024-05-27T07:44:28.347948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:04:24.007585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.355954+0000","last_change":"2024-05-27T07:41:08.844557+0000","last_active":"2024-05-27T07:44:28.355954+0000","last_peered":"2024-05-27T07:44:28.355954+0000","last_clean":"2024-05-27T07:44:28.355954+0000","last_became_active":"2024-05-27T07:41:08.843418+0000","last_became_peered":"2024-05-27T07:41:08.843418+0000","last_unstale":"2024-05-27T07:44:28.355954+0000","last_undegraded":"2024-05-27T07:44:28.355954+0000","last_fullsized":"2024-05-27T07:44:28.355954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:39.587778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.326639+0000","last_change":"2024-05-27T07:41:08.830857+0000","last_active":"2024-05-27T07:44:28.326639+0000","last_peered":"2024-05-27T07:44:28.326639+0000","last_clean":"2024-05-27T07:44:28.326639+0000","last_became_active":"2024-05-27T07:41:08.830480+0000","last_became_peered":"2024-05-27T07:41:08.830480+0000","last_unstale":"2024-05-27T07:44:28.326639+0000","last_undegraded":"2024-05-27T07:44:28.326639+0000","last_fullsized":"2024-05-27T07:44:28.326639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:07.307528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.348940+0000","last_change":"2024-05-27T07:41:08.824571+0000","last_active":"2024-05-27T07:44:28.348940+0000","last_peered":"2024-05-27T07:44:28.348940+0000","last_clean":"2024-05-27T07:44:28.348940+0000","last_became_active":"2024-05-27T07:41:08.820732+0000","last_became_peered":"2024-05-27T07:41:08.820732+0000","last_unstale":"2024-05-27T07:44:28.348940+0000","last_undegraded":"2024-05-27T07:44:28.348940+0000","last_fullsized":"2024-05-27T07:44:28.348940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:55:14.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.341251+0000","last_change":"2024-05-27T07:41:08.842284+0000","last_active":"2024-05-27T07:44:28.341251+0000","last_peered":"2024-05-27T07:44:28.341251+0000","last_clean":"2024-05-27T07:44:28.341251+0000","last_became_active":"2024-05-27T07:41:08.841597+0000","last_became_peered":"2024-05-27T07:41:08.841597+0000","last_unstale":"2024-05-27T07:44:28.341251+0000","last_undegraded":"2024-05-27T07:44:28.341251+0000","last_fullsized":"2024-05-27T07:44:28.341251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:25:31.066181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.366916+0000","last_change":"2024-05-27T07:41:08.828935+0000","last_active":"2024-05-27T07:44:28.366916+0000","last_peered":"2024-05-27T07:44:28.366916+0000","last_clean":"2024-05-27T07:44:28.366916+0000","last_became_active":"2024-05-27T07:41:08.828542+0000","last_became_peered":"2024-05-27T07:41:08.828542+0000","last_unstale":"2024-05-27T07:44:28.366916+0000","last_undegraded":"2024-05-27T07:44:28.366916+0000","last_fullsized":"2024-05-27T07:44:28.366916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:35.270666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.351714+0000","last_change":"2024-05-27T07:41:08.817051+0000","last_active":"2024-05-27T07:44:28.351714+0000","last_peered":"2024-05-27T07:44:28.351714+0000","last_clean":"2024-05-27T07:44:28.351714+0000","last_became_active":"2024-05-27T07:41:08.816418+0000","last_became_peered":"2024-05-27T07:41:08.816418+0000","last_unstale":"2024-05-27T07:44:28.351714+0000","last_undegraded":"2024-05-27T07:44:28.351714+0000","last_fullsized":"2024-05-27T07:44:28.351714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:02.760707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-27T07:44:28.345816+0000","last_change":"2024-05-27T07:41:08.834081+0000","last_active":"2024-05-27T07:44:28.345816+0000","last_peered":"2024-05-27T07:44:28.345816+0000","last_clean":"2024-05-27T07:44:28.345816+0000","last_became_active":"2024-05-27T07:41:08.833679+0000","last_became_peered":"2024-05-27T07:41:08.833679+0000","last_unstale":"2024-05-27T07:44:28.345816+0000","last_undegraded":"2024-05-27T07:44:28.345816+0000","last_fullsized":"2024-05-27T07:44:28.345816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:41:14.426593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74769,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1203,"num_read_kb":686,"num_write":262,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":224307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645372,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":522393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841603,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1753088,"data_stored":1181008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005131,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":522211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201361,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":522477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397593,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1119759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561120,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":517715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":586028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738509,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":519171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":519385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117112,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":517423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920880,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":589716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757352,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1114809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:44:36.792 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T07:44:36.792 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-27T07:44:36.792 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-27T07:44:36.792 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-05-27T07:44:37.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:36 smithi129 ceph-mon[35254]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:44:37.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:36 smithi090 ceph-mon[31429]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:44:37.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:36 smithi136 ceph-mon[35324]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:44:37.277 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:38.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:37 smithi129 ceph-mon[35254]: pgmap v490: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T07:44:38.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:37 smithi090 ceph-mon[31429]: pgmap v490: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T07:44:38.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:37 smithi136 ceph-mon[35324]: pgmap v490: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T07:44:39.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.417+0000 7f16a2600700 1 -- 172.21.15.90:0/1393471131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f169c10b100 msgr2=0x7f169c117610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.417+0000 7f16a2600700 1 --2- 172.21.15.90:0/1393471131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f169c10b100 0x7f169c117610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f169400b0a0 tx=0x7f169400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.417+0000 7f16a2600700 1 -- 172.21.15.90:0/1393471131 shutdown_connections 2024-05-27T07:44:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.417+0000 7f16a2600700 1 --2- 172.21.15.90:0/1393471131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f169c10b100 0x7f169c117610 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.417+0000 7f16a2600700 1 --2- 172.21.15.90:0/1393471131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f169c10f6b0 0x7f169c10fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.417+0000 7f16a2600700 1 --2- 172.21.15.90:0/1393471131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f169c10a7b0 0x7f169c10abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.417+0000 7f16a2600700 1 -- 172.21.15.90:0/1393471131 >> 172.21.15.90:0/1393471131 conn(0x7f169c076af0 msgr2=0x7f169c076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.417+0000 7f16a2600700 1 -- 172.21.15.90:0/1393471131 shutdown_connections 2024-05-27T07:44:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.417+0000 7f16a2600700 1 -- 172.21.15.90:0/1393471131 wait complete. 2024-05-27T07:44:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.418+0000 7f16a2600700 1 Processor -- start 2024-05-27T07:44:39.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.418+0000 7f16a2600700 1 -- start start 2024-05-27T07:44:39.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f16a2600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f169c10a7b0 0x7f169c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:39.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f16a2600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f169c10b100 0x7f169c1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:39.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f16a2600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f169c10f6b0 0x7f169c1a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f16a2600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f169c11a750 con 0x7f169c10f6b0 2024-05-27T07:44:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f16a2600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f169c11a5d0 con 0x7f169c10a7b0 2024-05-27T07:44:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f16a2600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f169c11a8d0 con 0x7f169c10b100 2024-05-27T07:44:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f16a1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f169c10f6b0 0x7f169c1a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f16a1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f169c10f6b0 0x7f169c1a5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34694/0 (socket says 172.21.15.90:34694) 2024-05-27T07:44:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f16a1c00700 1 -- 172.21.15.90:0/757372670 learned_addr learned my addr 172.21.15.90:0/757372670 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f16a1200700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f169c10a7b0 0x7f169c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.419+0000 7f169be00700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f169c10b100 0x7f169c1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.420+0000 7f16a1c00700 1 -- 172.21.15.90:0/757372670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f169c10b100 msgr2=0x7f169c1a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.420+0000 7f16a1c00700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f169c10b100 0x7f169c1a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.420+0000 7f16a1c00700 1 -- 172.21.15.90:0/757372670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f169c10a7b0 msgr2=0x7f169c1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.420+0000 7f16a1c00700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f169c10a7b0 0x7f169c1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.420+0000 7f16a1c00700 1 -- 172.21.15.90:0/757372670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c1a5f10 con 0x7f169c10f6b0 2024-05-27T07:44:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.420+0000 7f16a1c00700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f169c10f6b0 0x7f169c1a5870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f169400b830 tx=0x7f1694004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:39.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.421+0000 7f1699600700 1 -- 172.21.15.90:0/757372670 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694021070 con 0x7f169c10f6b0 2024-05-27T07:44:39.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.421+0000 7f1699600700 1 -- 172.21.15.90:0/757372670 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1694007b80 con 0x7f169c10f6b0 2024-05-27T07:44:39.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.421+0000 7f16a2600700 1 -- 172.21.15.90:0/757372670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c1a61a0 con 0x7f169c10f6b0 2024-05-27T07:44:39.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.421+0000 7f1699600700 1 -- 172.21.15.90:0/757372670 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16940052a0 con 0x7f169c10f6b0 2024-05-27T07:44:39.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.421+0000 7f16a2600700 1 -- 172.21.15.90:0/757372670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c1a6680 con 0x7f169c10f6b0 2024-05-27T07:44:39.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.422+0000 7f16a2600700 1 -- 172.21.15.90:0/757372670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f169c0512b0 con 0x7f169c10f6b0 2024-05-27T07:44:39.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.424+0000 7f1699600700 1 -- 172.21.15.90:0/757372670 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1694024070 con 0x7f169c10f6b0 2024-05-27T07:44:39.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.427+0000 7f1699600700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f16840b2880 0x7f16840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:39.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.427+0000 7f1699600700 1 -- 172.21.15.90:0/757372670 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f169400f020 con 0x7f169c10f6b0 2024-05-27T07:44:39.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.427+0000 7f16a1200700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f16840b2880 0x7f16840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:39.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.428+0000 7f16a1200700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f16840b2880 0x7f16840b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1690005950 tx=0x7f169000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:39.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.430+0000 7f1699600700 1 -- 172.21.15.90:0/757372670 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f169402b020 con 0x7f169c10f6b0 2024-05-27T07:44:39.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.712+0000 7f16a2600700 1 -- 172.21.15.90:0/757372670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f169c02e5e0 con 0x7f169c10f6b0 2024-05-27T07:44:39.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.713+0000 7f1699600700 1 -- 172.21.15.90:0/757372670 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f16940aba40 con 0x7f169c10f6b0 2024-05-27T07:44:39.712 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:44:39.712 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-27T07:44:39.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 -- 172.21.15.90:0/757372670 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f16840b2880 msgr2=0x7f16840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:39.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f16840b2880 0x7f16840b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1690005950 tx=0x7f169000b410 comp rx=0 tx=0).stop 2024-05-27T07:44:39.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 -- 172.21.15.90:0/757372670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f169c10f6b0 msgr2=0x7f169c1a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:39.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f169c10f6b0 0x7f169c1a5870 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f169400b830 tx=0x7f1694004d00 comp rx=0 tx=0).stop 2024-05-27T07:44:39.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 -- 172.21.15.90:0/757372670 shutdown_connections 2024-05-27T07:44:39.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f16840b2880 0x7f16840b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:39.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f169c10f6b0 0x7f169c1a5870 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:39.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f169c10a7b0 0x7f169c1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:39.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 --2- 172.21.15.90:0/757372670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f169c10b100 0x7f169c1a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:39.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 -- 172.21.15.90:0/757372670 >> 172.21.15.90:0/757372670 conn(0x7f169c076af0 msgr2=0x7f169c108810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:39.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 -- 172.21.15.90:0/757372670 shutdown_connections 2024-05-27T07:44:39.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:39.716+0000 7f16a2600700 1 -- 172.21.15.90:0/757372670 wait complete. 2024-05-27T07:44:40.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:39 smithi129 ceph-mon[35254]: pgmap v491: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T07:44:40.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:39 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/757372670' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T07:44:40.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:39 smithi090 ceph-mon[31429]: pgmap v491: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T07:44:40.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:39 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/757372670' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T07:44:40.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:39 smithi136 ceph-mon[35324]: pgmap v491: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T07:44:40.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:39 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/757372670' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T07:44:40.510 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-27T07:44:40.511 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:44:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:42.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:41 smithi129 ceph-mon[35254]: pgmap v492: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T07:44:42.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:41 smithi090 ceph-mon[31429]: pgmap v492: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T07:44:42.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:41 smithi136 ceph-mon[35324]: pgmap v492: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T07:44:43.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.167+0000 7fe57f200700 1 -- 172.21.15.90:0/1560370804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5801043f0 msgr2=0x7fe580104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:43.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.167+0000 7fe57f200700 1 --2- 172.21.15.90:0/1560370804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5801043f0 0x7fe580104800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe5700099c0 tx=0x7fe570009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:43.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.167+0000 7fe57f200700 1 -- 172.21.15.90:0/1560370804 shutdown_connections 2024-05-27T07:44:43.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.167+0000 7fe57f200700 1 --2- 172.21.15.90:0/1560370804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580109670 0x7fe580110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:43.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.167+0000 7fe57f200700 1 --2- 172.21.15.90:0/1560370804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5801043f0 0x7fe580104800 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:43.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.167+0000 7fe57f200700 1 --2- 172.21.15.90:0/1560370804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe580108d20 0x7fe580109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:43.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.167+0000 7fe57f200700 1 -- 172.21.15.90:0/1560370804 >> 172.21.15.90:0/1560370804 conn(0x7fe580079d00 msgr2=0x7fe58007a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:43.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.167+0000 7fe57f200700 1 -- 172.21.15.90:0/1560370804 shutdown_connections 2024-05-27T07:44:43.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.168+0000 7fe57f200700 1 -- 172.21.15.90:0/1560370804 wait complete. 2024-05-27T07:44:43.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.169+0000 7fe57f200700 1 Processor -- start 2024-05-27T07:44:43.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.169+0000 7fe57f200700 1 -- start start 2024-05-27T07:44:43.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57f200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5801043f0 0x7fe580075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:43.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57f200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580108d20 0x7fe580075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:43.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580109670 0x7fe580076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:43.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe580114150 con 0x7fe580109670 2024-05-27T07:44:43.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57f200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe580113fd0 con 0x7fe580108d20 2024-05-27T07:44:43.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57f200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801142d0 con 0x7fe5801043f0 2024-05-27T07:44:43.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580109670 0x7fe580076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:43.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580109670 0x7fe580076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34708/0 (socket says 172.21.15.90:34708) 2024-05-27T07:44:43.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57e800700 1 -- 172.21.15.90:0/1168424001 learned_addr learned my addr 172.21.15.90:0/1168424001 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:43.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57de00700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5801043f0 0x7fe580075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:43.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.170+0000 7fe57d400700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580108d20 0x7fe580075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:43.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.171+0000 7fe57e800700 1 -- 172.21.15.90:0/1168424001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5801043f0 msgr2=0x7fe580075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:43.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.171+0000 7fe57e800700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5801043f0 0x7fe580075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:43.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.171+0000 7fe57e800700 1 -- 172.21.15.90:0/1168424001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580108d20 msgr2=0x7fe580075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:43.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.171+0000 7fe57e800700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580108d20 0x7fe580075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:43.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.171+0000 7fe57e800700 1 -- 172.21.15.90:0/1168424001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5801a8ec0 con 0x7fe580109670 2024-05-27T07:44:43.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.171+0000 7fe57e800700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580109670 0x7fe580076530 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe57800ef30 tx=0x7fe57800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:43.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.171+0000 7fe576a00700 1 -- 172.21.15.90:0/1168424001 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5780092e0 con 0x7fe580109670 2024-05-27T07:44:43.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.171+0000 7fe576a00700 1 -- 172.21.15.90:0/1168424001 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe578004bb0 con 0x7fe580109670 2024-05-27T07:44:43.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.171+0000 7fe576a00700 1 -- 172.21.15.90:0/1168424001 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578004080 con 0x7fe580109670 2024-05-27T07:44:43.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.171+0000 7fe57f200700 1 -- 172.21.15.90:0/1168424001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5801a9150 con 0x7fe580109670 2024-05-27T07:44:43.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.172+0000 7fe57f200700 1 -- 172.21.15.90:0/1168424001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5801a9640 con 0x7fe580109670 2024-05-27T07:44:43.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.173+0000 7fe57f200700 1 -- 172.21.15.90:0/1168424001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5800512b0 con 0x7fe580109670 2024-05-27T07:44:43.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.177+0000 7fe576a00700 1 -- 172.21.15.90:0/1168424001 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe57801b030 con 0x7fe580109670 2024-05-27T07:44:43.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.179+0000 7fe576a00700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5640b2880 0x7fe5640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:43.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.179+0000 7fe576a00700 1 -- 172.21.15.90:0/1168424001 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fe5780e2b60 con 0x7fe580109670 2024-05-27T07:44:43.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.179+0000 7fe57de00700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5640b2880 0x7fe5640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:43.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.180+0000 7fe576a00700 1 -- 172.21.15.90:0/1168424001 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe578022020 con 0x7fe580109670 2024-05-27T07:44:43.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.180+0000 7fe57de00700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5640b2880 0x7fe5640b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe570003760 tx=0x7fe57001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:43.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.459+0000 7fe57f200700 1 -- 172.21.15.90:0/1168424001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe58002e7a0 con 0x7fe580109670 2024-05-27T07:44:43.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.460+0000 7fe576a00700 1 -- 172.21.15.90:0/1168424001 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11941 (secure 0 0 0) 0x7fe578010040 con 0x7fe580109670 2024-05-27T07:44:43.460 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:44:43.460 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:42:38.788447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24478,34406,34424,34442,34472]},"id":1}]} 2024-05-27T07:44:43.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.463+0000 7fe57f200700 1 -- 172.21.15.90:0/1168424001 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5640b2880 msgr2=0x7fe5640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:43.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.463+0000 7fe57f200700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5640b2880 0x7fe5640b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe570003760 tx=0x7fe57001a040 comp rx=0 tx=0).stop 2024-05-27T07:44:43.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.463+0000 7fe57f200700 1 -- 172.21.15.90:0/1168424001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580109670 msgr2=0x7fe580076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:43.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.463+0000 7fe57f200700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580109670 0x7fe580076530 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe57800ef30 tx=0x7fe57800c450 comp rx=0 tx=0).stop 2024-05-27T07:44:43.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.464+0000 7fe57f200700 1 -- 172.21.15.90:0/1168424001 shutdown_connections 2024-05-27T07:44:43.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.464+0000 7fe57f200700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5640b2880 0x7fe5640b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:43.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.464+0000 7fe57f200700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580109670 0x7fe580076530 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:43.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.464+0000 7fe57f200700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5801043f0 0x7fe580075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:43.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.464+0000 7fe57f200700 1 --2- 172.21.15.90:0/1168424001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580108d20 0x7fe580075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:43.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.464+0000 7fe57f200700 1 -- 172.21.15.90:0/1168424001 >> 172.21.15.90:0/1168424001 conn(0x7fe580079d00 msgr2=0x7fe580102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:43.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.464+0000 7fe57f200700 1 -- 172.21.15.90:0/1168424001 shutdown_connections 2024-05-27T07:44:43.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:43.464+0000 7fe57f200700 1 -- 172.21.15.90:0/1168424001 wait complete. 2024-05-27T07:44:43.465 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-05-27T07:44:44.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:43 smithi090 ceph-mon[31429]: pgmap v493: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-27T07:44:44.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:43 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1168424001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:44.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:43 smithi136 ceph-mon[35324]: pgmap v493: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-27T07:44:44.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:43 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1168424001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:44.308 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:44:44.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:43 smithi129 ceph-mon[35254]: pgmap v493: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-05-27T07:44:44.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:43 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1168424001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:44.767 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:45.947 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:45 smithi090 ceph-mon[31429]: pgmap v494: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-27T07:44:46.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:45 smithi136 ceph-mon[35324]: pgmap v494: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-27T07:44:46.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:45 smithi129 ceph-mon[35254]: pgmap v494: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-27T07:44:46.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.924+0000 7f4ebec00700 1 -- 172.21.15.90:0/955257904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4eb8108f10 msgr2=0x7f4eb8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.924+0000 7f4ebec00700 1 --2- 172.21.15.90:0/955257904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4eb8108f10 0x7f4eb8109320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40099c0 tx=0x7f4ea4009cd0 comp rx=0 tx=0).stop 2024-05-27T07:44:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.925+0000 7f4ebec00700 1 -- 172.21.15.90:0/955257904 shutdown_connections 2024-05-27T07:44:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.925+0000 7f4ebec00700 1 --2- 172.21.15.90:0/955257904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eb8109860 0x7f4eb81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.925+0000 7f4ebec00700 1 --2- 172.21.15.90:0/955257904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eb81045e0 0x7f4eb81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.925+0000 7f4ebec00700 1 --2- 172.21.15.90:0/955257904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4eb8108f10 0x7f4eb8109320 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.925+0000 7f4ebec00700 1 -- 172.21.15.90:0/955257904 >> 172.21.15.90:0/955257904 conn(0x7f4eb80fffd0 msgr2=0x7f4eb81023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.925+0000 7f4ebec00700 1 -- 172.21.15.90:0/955257904 shutdown_connections 2024-05-27T07:44:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.925+0000 7f4ebec00700 1 -- 172.21.15.90:0/955257904 wait complete. 2024-05-27T07:44:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.926+0000 7f4ebec00700 1 Processor -- start 2024-05-27T07:44:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.927+0000 7f4ebec00700 1 -- start start 2024-05-27T07:44:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.927+0000 7f4ebec00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eb81045e0 0x7f4eb819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:46.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.927+0000 7f4ebec00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4eb8108f10 0x7f4eb819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:46.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.927+0000 7f4ebec00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eb8109860 0x7f4eb81a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:46.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.927+0000 7f4ebec00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8114320 con 0x7f4eb8108f10 2024-05-27T07:44:46.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.927+0000 7f4ebec00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81141a0 con 0x7f4eb81045e0 2024-05-27T07:44:46.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.927+0000 7f4ebec00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81144a0 con 0x7f4eb8109860 2024-05-27T07:44:46.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.928+0000 7f4eb7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4eb8108f10 0x7f4eb819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:46.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.928+0000 7f4eb7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4eb8108f10 0x7f4eb819d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34738/0 (socket says 172.21.15.90:34738) 2024-05-27T07:44:46.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.928+0000 7f4eb7400700 1 -- 172.21.15.90:0/2237752837 learned_addr learned my addr 172.21.15.90:0/2237752837 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:46.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.928+0000 7f4eb7e00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eb81045e0 0x7f4eb819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:46.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.928+0000 7f4ebcc00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eb8109860 0x7f4eb81a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:46.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.928+0000 7f4eb7400700 1 -- 172.21.15.90:0/2237752837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eb8109860 msgr2=0x7f4eb81a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:46.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.928+0000 7f4eb7400700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eb8109860 0x7f4eb81a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:46.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.928+0000 7f4eb7400700 1 -- 172.21.15.90:0/2237752837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eb81045e0 msgr2=0x7f4eb819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:46.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.928+0000 7f4eb7400700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eb81045e0 0x7f4eb819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:46.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.928+0000 7f4eb7400700 1 -- 172.21.15.90:0/2237752837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81a1e40 con 0x7f4eb8108f10 2024-05-27T07:44:46.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.929+0000 7f4eb7400700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4eb8108f10 0x7f4eb819d510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400b580 tx=0x7f4ea40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:46.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.929+0000 7f4eb4c00700 1 -- 172.21.15.90:0/2237752837 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4027070 con 0x7f4eb8108f10 2024-05-27T07:44:46.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.929+0000 7f4eb4c00700 1 -- 172.21.15.90:0/2237752837 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ea400fd30 con 0x7f4eb8108f10 2024-05-27T07:44:46.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.929+0000 7f4ebec00700 1 -- 172.21.15.90:0/2237752837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb81a20d0 con 0x7f4eb8108f10 2024-05-27T07:44:46.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.929+0000 7f4eb4c00700 1 -- 172.21.15.90:0/2237752837 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40173f0 con 0x7f4eb8108f10 2024-05-27T07:44:46.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.929+0000 7f4ebec00700 1 -- 172.21.15.90:0/2237752837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb81a9a50 con 0x7f4eb8108f10 2024-05-27T07:44:46.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.930+0000 7f4ebec00700 1 -- 172.21.15.90:0/2237752837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb80512b0 con 0x7f4eb8108f10 2024-05-27T07:44:46.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.933+0000 7f4eb4c00700 1 -- 172.21.15.90:0/2237752837 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4ea402a070 con 0x7f4eb8108f10 2024-05-27T07:44:46.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.935+0000 7f4eb4c00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e9c0b2900 0x7f4e9c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:46.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.935+0000 7f4eb4c00700 1 -- 172.21.15.90:0/2237752837 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f4ea40e33b0 con 0x7f4eb8108f10 2024-05-27T07:44:46.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.935+0000 7f4eb7e00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e9c0b2900 0x7f4e9c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:46.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.936+0000 7f4eb7e00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e9c0b2900 0x7f4e9c0b4e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005950 tx=0x7f4ea800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:46.942+0000 7f4eb4c00700 1 -- 172.21.15.90:0/2237752837 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4ea4013070 con 0x7f4eb8108f10 2024-05-27T07:44:47.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.238+0000 7f4ebec00700 1 -- 172.21.15.90:0/2237752837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4eb819e2d0 con 0x7f4eb8108f10 2024-05-27T07:44:47.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.240+0000 7f4eb4c00700 1 -- 172.21.15.90:0/2237752837 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11941 (secure 0 0 0) 0x7f4ea40aad70 con 0x7f4eb8108f10 2024-05-27T07:44:47.240 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:44:47.240 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:42:38.788447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24478,34406,34424,34442,34472]},"id":1}]} 2024-05-27T07:44:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.243+0000 7f4ebec00700 1 -- 172.21.15.90:0/2237752837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e9c0b2900 msgr2=0x7f4e9c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.243+0000 7f4ebec00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e9c0b2900 0x7f4e9c0b4e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8005950 tx=0x7f4ea800b410 comp rx=0 tx=0).stop 2024-05-27T07:44:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.243+0000 7f4ebec00700 1 -- 172.21.15.90:0/2237752837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4eb8108f10 msgr2=0x7f4eb819d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.243+0000 7f4ebec00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4eb8108f10 0x7f4eb819d510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4ea400b580 tx=0x7f4ea40057c0 comp rx=0 tx=0).stop 2024-05-27T07:44:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.243+0000 7f4ebec00700 1 -- 172.21.15.90:0/2237752837 shutdown_connections 2024-05-27T07:44:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.243+0000 7f4ebec00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e9c0b2900 0x7f4e9c0b4e10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.243+0000 7f4ebec00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4eb8109860 0x7f4eb81a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.243+0000 7f4ebec00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4eb81045e0 0x7f4eb819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.243+0000 7f4ebec00700 1 --2- 172.21.15.90:0/2237752837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4eb8108f10 0x7f4eb819d510 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:47.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.243+0000 7f4ebec00700 1 -- 172.21.15.90:0/2237752837 >> 172.21.15.90:0/2237752837 conn(0x7f4eb80fffd0 msgr2=0x7f4eb8110ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:47.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.244+0000 7f4ebec00700 1 -- 172.21.15.90:0/2237752837 shutdown_connections 2024-05-27T07:44:47.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:47.244+0000 7f4ebec00700 1 -- 172.21.15.90:0/2237752837 wait complete. 2024-05-27T07:44:47.245 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-05-27T07:44:48.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:47 smithi090 ceph-mon[31429]: pgmap v495: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:48.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:47 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2237752837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:48.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:47 smithi136 ceph-mon[35324]: pgmap v495: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:48.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:47 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2237752837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:48.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:47 smithi129 ceph-mon[35254]: pgmap v495: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:48.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:47 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2237752837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:48.395 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:44:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:50.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:49 smithi090 ceph-mon[31429]: pgmap v496: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:44:50.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:49 smithi136 ceph-mon[35324]: pgmap v496: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:44:50.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:49 smithi129 ceph-mon[35254]: pgmap v496: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:44:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.315+0000 7fc52fc00700 1 -- 172.21.15.90:0/2913842462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5281045e0 msgr2=0x7fc5281049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.315+0000 7fc52fc00700 1 --2- 172.21.15.90:0/2913842462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5281045e0 0x7fc5281049f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00b0d0 tx=0x7fc51c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:44:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.315+0000 7fc52fc00700 1 -- 172.21.15.90:0/2913842462 shutdown_connections 2024-05-27T07:44:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.315+0000 7fc52fc00700 1 --2- 172.21.15.90:0/2913842462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc528109860 0x7fc5281111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.315+0000 7fc52fc00700 1 --2- 172.21.15.90:0/2913842462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5281045e0 0x7fc5281049f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:51.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.315+0000 7fc52fc00700 1 --2- 172.21.15.90:0/2913842462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc528108f10 0x7fc528109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:51.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.315+0000 7fc52fc00700 1 -- 172.21.15.90:0/2913842462 >> 172.21.15.90:0/2913842462 conn(0x7fc528100030 msgr2=0x7fc528102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:51.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.315+0000 7fc52fc00700 1 -- 172.21.15.90:0/2913842462 shutdown_connections 2024-05-27T07:44:51.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.315+0000 7fc52fc00700 1 -- 172.21.15.90:0/2913842462 wait complete. 2024-05-27T07:44:51.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.316+0000 7fc52fc00700 1 Processor -- start 2024-05-27T07:44:51.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.317+0000 7fc52fc00700 1 -- start start 2024-05-27T07:44:51.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.317+0000 7fc52fc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5281045e0 0x7fc52819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:51.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.317+0000 7fc52fc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc528108f10 0x7fc52819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:51.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.317+0000 7fc52fc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc528109860 0x7fc5281a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:51.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.317+0000 7fc52fc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc5281143b0 con 0x7fc528109860 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.317+0000 7fc52fc00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc528114230 con 0x7fc528108f10 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.317+0000 7fc52fc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc528114530 con 0x7fc5281045e0 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.317+0000 7fc52dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc528109860 0x7fc5281a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.317+0000 7fc52dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc528109860 0x7fc5281a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52482/0 (socket says 172.21.15.90:52482) 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.317+0000 7fc52dc00700 1 -- 172.21.15.90:0/2749286490 learned_addr learned my addr 172.21.15.90:0/2749286490 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.318+0000 7fc527e00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc528108f10 0x7fc52819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.318+0000 7fc52d200700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5281045e0 0x7fc52819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.318+0000 7fc52dc00700 1 -- 172.21.15.90:0/2749286490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5281045e0 msgr2=0x7fc52819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.318+0000 7fc52dc00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5281045e0 0x7fc52819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.318+0000 7fc52dc00700 1 -- 172.21.15.90:0/2749286490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc528108f10 msgr2=0x7fc52819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:51.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.318+0000 7fc52dc00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc528108f10 0x7fc52819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:51.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.318+0000 7fc52dc00700 1 -- 172.21.15.90:0/2749286490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5281a1ef0 con 0x7fc528109860 2024-05-27T07:44:51.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.318+0000 7fc527e00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc528108f10 0x7fc52819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:44:51.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.318+0000 7fc52dc00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc528109860 0x7fc5281a1850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc52000d980 tx=0x7fc52000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:51.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.319+0000 7fc525600700 1 -- 172.21.15.90:0/2749286490 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520017070 con 0x7fc528109860 2024-05-27T07:44:51.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.319+0000 7fc525600700 1 -- 172.21.15.90:0/2749286490 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc520009dd0 con 0x7fc528109860 2024-05-27T07:44:51.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.319+0000 7fc525600700 1 -- 172.21.15.90:0/2749286490 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520005260 con 0x7fc528109860 2024-05-27T07:44:51.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.319+0000 7fc52fc00700 1 -- 172.21.15.90:0/2749286490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5281a21e0 con 0x7fc528109860 2024-05-27T07:44:51.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.319+0000 7fc52fc00700 1 -- 172.21.15.90:0/2749286490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5281a9b00 con 0x7fc528109860 2024-05-27T07:44:51.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.320+0000 7fc52fc00700 1 -- 172.21.15.90:0/2749286490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5280512b0 con 0x7fc528109860 2024-05-27T07:44:51.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.324+0000 7fc525600700 1 -- 172.21.15.90:0/2749286490 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc520009940 con 0x7fc528109860 2024-05-27T07:44:51.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.326+0000 7fc525600700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5100b2880 0x7fc5100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:51.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.326+0000 7fc525600700 1 -- 172.21.15.90:0/2749286490 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc5200e2990 con 0x7fc528109860 2024-05-27T07:44:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.326+0000 7fc52d200700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5100b2880 0x7fc5100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.327+0000 7fc52d200700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5100b2880 0x7fc5100b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009900 tx=0x7fc51c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:51.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.329+0000 7fc525600700 1 -- 172.21.15.90:0/2749286490 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc520020020 con 0x7fc528109860 2024-05-27T07:44:51.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.567+0000 7fc52fc00700 1 -- 172.21.15.90:0/2749286490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5281a9cd0 con 0x7fc528109860 2024-05-27T07:44:51.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.570+0000 7fc525600700 1 -- 172.21.15.90:0/2749286490 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24725 (secure 0 0 0) 0x7fc5200aa2c0 con 0x7fc528109860 2024-05-27T07:44:51.570 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:44:51.570 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":91,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:43:00.000806+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 -- 172.21.15.90:0/2749286490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5100b2880 msgr2=0x7fc5100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5100b2880 0x7fc5100b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009900 tx=0x7fc51c00b040 comp rx=0 tx=0).stop 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 -- 172.21.15.90:0/2749286490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc528109860 msgr2=0x7fc5281a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc528109860 0x7fc5281a1850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc52000d980 tx=0x7fc52000dd40 comp rx=0 tx=0).stop 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 -- 172.21.15.90:0/2749286490 shutdown_connections 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5100b2880 0x7fc5100b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc528109860 0x7fc5281a1850 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5281045e0 0x7fc52819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 --2- 172.21.15.90:0/2749286490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc528108f10 0x7fc52819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 -- 172.21.15.90:0/2749286490 >> 172.21.15.90:0/2749286490 conn(0x7fc528100030 msgr2=0x7fc52810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 -- 172.21.15.90:0/2749286490 shutdown_connections 2024-05-27T07:44:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:51.574+0000 7fc52fc00700 1 -- 172.21.15.90:0/2749286490 wait complete. 2024-05-27T07:44:52.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:51 smithi090 ceph-mon[31429]: pgmap v497: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:52.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:51 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2749286490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:44:52.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:51 smithi136 ceph-mon[35324]: pgmap v497: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:52.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:51 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2749286490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:44:52.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:51 smithi129 ceph-mon[35254]: pgmap v497: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:52.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:51 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2749286490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:44:52.763 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:44:53.217 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:53.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:53 smithi090 ceph-mon[31429]: pgmap v498: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:44:54.115 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:53 smithi129 ceph-mon[35254]: pgmap v498: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:44:54.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:53 smithi136 ceph-mon[35324]: pgmap v498: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:44:55.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.963+0000 7f8104e00700 1 -- 172.21.15.90:0/2098263729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810010e710 msgr2=0x7f8100110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:55.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.963+0000 7f8104e00700 1 --2- 172.21.15.90:0/2098263729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810010e710 0x7f8100110bf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f80f800b0a0 tx=0x7f80f800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:44:55.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.963+0000 7f8104e00700 1 -- 172.21.15.90:0/2098263729 shutdown_connections 2024-05-27T07:44:55.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.963+0000 7f8104e00700 1 --2- 172.21.15.90:0/2098263729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f810010e710 0x7f8100110bf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:55.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.963+0000 7f8104e00700 1 --2- 172.21.15.90:0/2098263729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81001052a0 0x7f81001056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:55.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.963+0000 7f8104e00700 1 --2- 172.21.15.90:0/2098263729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8100105bf0 0x7f810010e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:55.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.963+0000 7f8104e00700 1 -- 172.21.15.90:0/2098263729 >> 172.21.15.90:0/2098263729 conn(0x7f8100078c30 msgr2=0x7f8100079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:55.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.963+0000 7f8104e00700 1 -- 172.21.15.90:0/2098263729 shutdown_connections 2024-05-27T07:44:55.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.963+0000 7f8104e00700 1 -- 172.21.15.90:0/2098263729 wait complete. 2024-05-27T07:44:55.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.964+0000 7f8104e00700 1 Processor -- start 2024-05-27T07:44:55.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.965+0000 7f8104e00700 1 -- start start 2024-05-27T07:44:55.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.965+0000 7f8104e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81001052a0 0x7f8100102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:55.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.965+0000 7f8104e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8100105bf0 0x7f81001034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:55.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.965+0000 7f8104e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f810010e710 0x7f8100103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:55.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.965+0000 7f8104e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8100114170 con 0x7f81001052a0 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.965+0000 7f8104e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8100113ff0 con 0x7f810010e710 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.965+0000 7f8104e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f81001142f0 con 0x7f8100105bf0 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.965+0000 7f80ff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81001052a0 0x7f8100102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.966+0000 7f80ff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81001052a0 0x7f8100102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52500/0 (socket says 172.21.15.90:52500) 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.966+0000 7f80ff400700 1 -- 172.21.15.90:0/955938743 learned_addr learned my addr 172.21.15.90:0/955938743 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.966+0000 7f80f6a00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8100105bf0 0x7f81001034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.966+0000 7f80ffe00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f810010e710 0x7f8100103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.966+0000 7f80ff400700 1 -- 172.21.15.90:0/955938743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8100105bf0 msgr2=0x7f81001034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.966+0000 7f80ff400700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8100105bf0 0x7f81001034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.966+0000 7f80ff400700 1 -- 172.21.15.90:0/955938743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f810010e710 msgr2=0x7f8100103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.966+0000 7f80ff400700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f810010e710 0x7f8100103a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.966+0000 7f80ff400700 1 -- 172.21.15.90:0/955938743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81001a8e80 con 0x7f81001052a0 2024-05-27T07:44:55.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.966+0000 7f80ff400700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81001052a0 0x7f8100102f90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f80f000d950 tx=0x7f80f000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:55.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.967+0000 7f80fcc00700 1 -- 172.21.15.90:0/955938743 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0016070 con 0x7f81001052a0 2024-05-27T07:44:55.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.967+0000 7f8104e00700 1 -- 172.21.15.90:0/955938743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81001a9170 con 0x7f81001052a0 2024-05-27T07:44:55.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.967+0000 7f80fcc00700 1 -- 172.21.15.90:0/955938743 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f80f0010460 con 0x7f81001052a0 2024-05-27T07:44:55.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.967+0000 7f80fcc00700 1 -- 172.21.15.90:0/955938743 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f000f620 con 0x7f81001052a0 2024-05-27T07:44:55.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.967+0000 7f8104e00700 1 -- 172.21.15.90:0/955938743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81001a9700 con 0x7f81001052a0 2024-05-27T07:44:55.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.968+0000 7f8104e00700 1 -- 172.21.15.90:0/955938743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81000512b0 con 0x7f81001052a0 2024-05-27T07:44:55.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.972+0000 7f80fcc00700 1 -- 172.21.15.90:0/955938743 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f80f0019070 con 0x7f81001052a0 2024-05-27T07:44:55.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.974+0000 7f80fcc00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f80e40b2900 0x7f80e40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:44:55.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.974+0000 7f80fcc00700 1 -- 172.21.15.90:0/955938743 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f80f00e2540 con 0x7f81001052a0 2024-05-27T07:44:55.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.974+0000 7f80f6a00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f80e40b2900 0x7f80e40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:44:55.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.975+0000 7f80f6a00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f80e40b2900 0x7f80e40b4e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f81001026f0 tx=0x7f80e800d4b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:44:55.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:55.977+0000 7f80fcc00700 1 -- 172.21.15.90:0/955938743 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80f00a9e70 con 0x7f81001052a0 2024-05-27T07:44:56.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:55 smithi090 ceph-mon[31429]: pgmap v499: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:56.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:55 smithi136 ceph-mon[35324]: pgmap v499: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:56.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.257+0000 7f8104e00700 1 -- 172.21.15.90:0/955938743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f810002e5e0 con 0x7f81001052a0 2024-05-27T07:44:56.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.259+0000 7f80fcc00700 1 -- 172.21.15.90:0/955938743 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11941 (secure 0 0 0) 0x7f80f00a9770 con 0x7f81001052a0 2024-05-27T07:44:56.258 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:44:56.258 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:42:38.788447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24478,34406,34424,34442,34472]},"id":1}]} 2024-05-27T07:44:56.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.262+0000 7f8104e00700 1 -- 172.21.15.90:0/955938743 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f80e40b2900 msgr2=0x7f80e40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:56.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.262+0000 7f8104e00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f80e40b2900 0x7f80e40b4e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f81001026f0 tx=0x7f80e800d4b0 comp rx=0 tx=0).stop 2024-05-27T07:44:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.262+0000 7f8104e00700 1 -- 172.21.15.90:0/955938743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81001052a0 msgr2=0x7f8100102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:44:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.262+0000 7f8104e00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81001052a0 0x7f8100102f90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f80f000d950 tx=0x7f80f000dd10 comp rx=0 tx=0).stop 2024-05-27T07:44:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.262+0000 7f8104e00700 1 -- 172.21.15.90:0/955938743 shutdown_connections 2024-05-27T07:44:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.262+0000 7f8104e00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f80e40b2900 0x7f80e40b4e10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.262+0000 7f8104e00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f810010e710 0x7f8100103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.262+0000 7f8104e00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81001052a0 0x7f8100102f90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.262+0000 7f8104e00700 1 --2- 172.21.15.90:0/955938743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8100105bf0 0x7f81001034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:44:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.263+0000 7f8104e00700 1 -- 172.21.15.90:0/955938743 >> 172.21.15.90:0/955938743 conn(0x7f8100078c30 msgr2=0x7f81000ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:44:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.263+0000 7f8104e00700 1 -- 172.21.15.90:0/955938743 shutdown_connections 2024-05-27T07:44:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:44:56.263+0000 7f8104e00700 1 -- 172.21.15.90:0/955938743 wait complete. 2024-05-27T07:44:56.263 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-05-27T07:44:56.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:55 smithi129 ceph-mon[35254]: pgmap v499: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:44:57.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:56 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/955938743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:57.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:56 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/955938743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:57.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:56 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/955938743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:44:57.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T07:41:11.710115+0000', 'modified': '2024-05-27T07:42:38.788447+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34472, 'mds_1': 34442, 'mds_2': 34424, 'mds_3': 34406, 'mds_4': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14556': {'gid': 14556, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6839/767662982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 767662982}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 767662982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14562': {'gid': 14562, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6841/24895501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6840', 'nonce': 24895501}, {'type': 'v1', 'addr': '172.21.15.90:6841', 'nonce': 24895501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/1595668128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 1595668128}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 1595668128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/1419480441', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 1419480441}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 1419480441}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6837/3523286296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 3523286296}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 3523286296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.136:6839/2113925564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 2113925564}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 2113925564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34406': {'gid': 34406, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 37, 'addr': '172.21.15.129:6833/3131589000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6832', 'nonce': 3131589000}, {'type': 'v1', 'addr': '172.21.15.129:6833', 'nonce': 3131589000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34424': {'gid': 34424, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 34, 'addr': '172.21.15.129:6835/1894603892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 1894603892}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 1894603892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34442': {'gid': 34442, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.129:6837/2346696366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 2346696366}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 2346696366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34472': {'gid': 34472, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.129:6839/2893900972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 2893900972}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 2893900972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34472, 'qdb_cluster': [24478, 34406, 34424, 34442, 34472]} 2024-05-27T07:44:57.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-27T07:44:57.608 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-05-27T07:44:58.056 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:44:58.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:57 smithi090 ceph-mon[31429]: pgmap v500: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:58.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:57 smithi136 ceph-mon[35324]: pgmap v500: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:58.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:57 smithi129 ceph-mon[35254]: pgmap v500: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:44:59.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:44:59 smithi090 ceph-mon[31429]: pgmap v501: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:00.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:44:59 smithi136 ceph-mon[35324]: pgmap v501: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:00.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:44:59 smithi129 ceph-mon[35254]: pgmap v501: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:00.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.471+0000 7fe546200700 1 -- 172.21.15.90:0/2613383584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54010a980 msgr2=0x7fe54010ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:00.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.471+0000 7fe546200700 1 --2- 172.21.15.90:0/2613383584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54010a980 0x7fe54010ad90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009990 tx=0x7fe52c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.471+0000 7fe546200700 1 -- 172.21.15.90:0/2613383584 shutdown_connections 2024-05-27T07:45:00.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.471+0000 7fe546200700 1 --2- 172.21.15.90:0/2613383584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe54010b2d0 0x7fe540117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.471+0000 7fe546200700 1 --2- 172.21.15.90:0/2613383584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe540110960 0x7fe540110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.471+0000 7fe546200700 1 --2- 172.21.15.90:0/2613383584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54010a980 0x7fe54010ad90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.471+0000 7fe546200700 1 -- 172.21.15.90:0/2613383584 >> 172.21.15.90:0/2613383584 conn(0x7fe540077c70 msgr2=0x7fe5400780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:00.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.471+0000 7fe546200700 1 -- 172.21.15.90:0/2613383584 shutdown_connections 2024-05-27T07:45:00.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.471+0000 7fe546200700 1 -- 172.21.15.90:0/2613383584 wait complete. 2024-05-27T07:45:00.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.472+0000 7fe546200700 1 Processor -- start 2024-05-27T07:45:00.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.473+0000 7fe546200700 1 -- start start 2024-05-27T07:45:00.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.473+0000 7fe546200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe54010a980 0x7fe5401a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.473+0000 7fe546200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54010b2d0 0x7fe5401a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.473+0000 7fe546200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe540110960 0x7fe5401a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.473+0000 7fe546200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe54011a960 con 0x7fe54010b2d0 2024-05-27T07:45:00.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.473+0000 7fe546200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe54011a7e0 con 0x7fe540110960 2024-05-27T07:45:00.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.473+0000 7fe546200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe54011aae0 con 0x7fe54010a980 2024-05-27T07:45:00.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54010b2d0 0x7fe5401a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54010b2d0 0x7fe5401a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59450/0 (socket says 172.21.15.90:59450) 2024-05-27T07:45:00.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53ea00700 1 -- 172.21.15.90:0/3165173464 learned_addr learned my addr 172.21.15.90:0/3165173464 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:00.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53f400700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe54010a980 0x7fe5401a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53fe00700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe540110960 0x7fe5401a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53ea00700 1 -- 172.21.15.90:0/3165173464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe54010a980 msgr2=0x7fe5401a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:00.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53ea00700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe54010a980 0x7fe5401a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53ea00700 1 -- 172.21.15.90:0/3165173464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe540110960 msgr2=0x7fe5401a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:00.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53ea00700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe540110960 0x7fe5401a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53ea00700 1 -- 172.21.15.90:0/3165173464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5401a6260 con 0x7fe54010b2d0 2024-05-27T07:45:00.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.474+0000 7fe53ea00700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54010b2d0 0x7fe5401a1930 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000c00 tx=0x7fe52c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:00.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.475+0000 7fe537e00700 1 -- 172.21.15.90:0/3165173464 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c027070 con 0x7fe54010b2d0 2024-05-27T07:45:00.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.475+0000 7fe537e00700 1 -- 172.21.15.90:0/3165173464 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe52c00fc80 con 0x7fe54010b2d0 2024-05-27T07:45:00.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.475+0000 7fe546200700 1 -- 172.21.15.90:0/3165173464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5401a64f0 con 0x7fe54010b2d0 2024-05-27T07:45:00.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.475+0000 7fe537e00700 1 -- 172.21.15.90:0/3165173464 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c022d30 con 0x7fe54010b2d0 2024-05-27T07:45:00.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.475+0000 7fe546200700 1 -- 172.21.15.90:0/3165173464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5401ade70 con 0x7fe54010b2d0 2024-05-27T07:45:00.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.479+0000 7fe537e00700 1 -- 172.21.15.90:0/3165173464 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe52c02a070 con 0x7fe54010b2d0 2024-05-27T07:45:00.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.480+0000 7fe554d52b80 1 Processor -- start 2024-05-27T07:45:00.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.480+0000 7fe554d52b80 1 -- start start 2024-05-27T07:45:00.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfb4f9a0 0x562ddfbe22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe537e00700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5240b2900 0x7fe5240b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfa63df0 0x562ddfbe2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 0x562ddfbef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe554d52b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ddfb10df0 con 0x562ddfb4f9a0 2024-05-27T07:45:00.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe554d52b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ddfb00990 con 0x562ddfbe2e20 2024-05-27T07:45:00.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe554d52b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562ddfb049d0 con 0x562ddfa63df0 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe537e00700 1 -- 172.21.15.90:0/3165173464 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fe52c0e3600 con 0x7fe54010b2d0 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe53fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfb4f9a0 0x562ddfbe22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe53f400700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5240b2900 0x7fe5240b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe53fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfb4f9a0 0x562ddfbe22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59458/0 (socket says 172.21.15.90:59458) 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe53fe00700 1 -- 172.21.15.90:0/1051789403 learned_addr learned my addr 172.21.15.90:0/1051789403 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.481+0000 7fe53ea00700 1 --2- 172.21.15.90:0/1051789403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfa63df0 0x562ddfbe2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.482+0000 7fe53fe00700 1 --2- 172.21.15.90:0/1051789403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 0x562ddfbef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.482+0000 7fe53fe00700 1 -- 172.21.15.90:0/1051789403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfa63df0 msgr2=0x562ddfbe2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.482+0000 7fe53fe00700 1 --2- 172.21.15.90:0/1051789403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfa63df0 0x562ddfbe2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.482+0000 7fe53fe00700 1 -- 172.21.15.90:0/1051789403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 msgr2=0x562ddfbef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.482+0000 7fe53fe00700 1 --2- 172.21.15.90:0/1051789403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 0x562ddfbef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.482+0000 7fe53fe00700 1 -- 172.21.15.90:0/1051789403 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ddfac93d0 con 0x562ddfb4f9a0 2024-05-27T07:45:00.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.482+0000 7fe53fe00700 1 --2- 172.21.15.90:0/1051789403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfb4f9a0 0x562ddfbe22e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe538007ee0 tx=0x7fe538007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa8674b538beda3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:00.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.482+0000 7fe53f400700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5240b2900 0x7fe5240b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe530005fd0 tx=0x7fe530005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:00.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.482+0000 7fe534c00700 1 -- 172.21.15.90:0/1051789403 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538013040 con 0x562ddfb4f9a0 2024-05-27T07:45:00.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.483+0000 7fe534c00700 1 -- 172.21.15.90:0/1051789403 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe538003c00 con 0x562ddfb4f9a0 2024-05-27T07:45:00.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.483+0000 7fe534c00700 1 -- 172.21.15.90:0/1051789403 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380127b0 con 0x562ddfb4f9a0 2024-05-27T07:45:00.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.484+0000 7fe554d52b80 1 -- 172.21.15.90:0/1051789403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfb4f9a0 msgr2=0x562ddfbe22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:00.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.484+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1051789403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfb4f9a0 0x562ddfbe22e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe538007ee0 tx=0x7fe538007de0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.484+0000 7fe554d52b80 1 -- 172.21.15.90:0/1051789403 shutdown_connections 2024-05-27T07:45:00.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.484+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1051789403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 0x562ddfbef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.484+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1051789403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfb4f9a0 0x562ddfbe22e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.484+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1051789403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfa63df0 0x562ddfbe2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.484+0000 7fe554d52b80 1 -- 172.21.15.90:0/1051789403 >> 172.21.15.90:0/1051789403 conn(0x562ddfafd340 msgr2=0x562ddfbde470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:00.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.484+0000 7fe554d52b80 1 -- 172.21.15.90:0/1051789403 shutdown_connections 2024-05-27T07:45:00.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.484+0000 7fe554d52b80 1 -- 172.21.15.90:0/1051789403 wait complete. 2024-05-27T07:45:00.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.485+0000 7fe554d52b80 1 -- start start 2024-05-27T07:45:00.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.485+0000 7fe554d52b80 1 Processor -- start 2024-05-27T07:45:00.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbec650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfb4f9a0 0x562ddfbecb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfbe2e20 0x562ddfbed0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe554d52b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ddfa06f50 con 0x562ddfa63df0 2024-05-27T07:45:00.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe554d52b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ddfaa1710 con 0x562ddfb4f9a0 2024-05-27T07:45:00.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe554d52b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562ddfbefb30 con 0x562ddfbe2e20 2024-05-27T07:45:00.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe554d52b80 10 client.0 _ll_register_callbacks cb 0x562ddfaaac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:00.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe53f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe53f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbec650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59472/0 (socket says 172.21.15.90:59472) 2024-05-27T07:45:00.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe53f400700 1 -- 172.21.15.90:0/1095582285 learned_addr learned my addr 172.21.15.90:0/1095582285 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:00.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe53ea00700 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfb4f9a0 0x562ddfbecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe53f400700 1 -- 172.21.15.90:0/1095582285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfbe2e20 msgr2=0x562ddfbed0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:00.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe53f400700 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfbe2e20 0x562ddfbed0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe53f400700 1 -- 172.21.15.90:0/1095582285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfb4f9a0 msgr2=0x562ddfbecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:00.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.486+0000 7fe53f400700 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfb4f9a0 0x562ddfbecb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.487+0000 7fe53f400700 1 -- 172.21.15.90:0/1095582285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ddfa50880 con 0x562ddfa63df0 2024-05-27T07:45:00.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.487+0000 7fe53f400700 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbec650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe5300106b0 tx=0x7fe530004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f267267eda87e87e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:00.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.487+0000 7fe529600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:00.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.487+0000 7fe529600700 1 -- 172.21.15.90:0/1095582285 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530021070 con 0x562ddfa63df0 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.487+0000 7fe529600700 1 -- 172.21.15.90:0/1095582285 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fe53001d860 con 0x562ddfa63df0 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.487+0000 7fe554d52b80 1 -- 172.21.15.90:0/1095582285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe538003c00 con 0x562ddfa63df0 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.488+0000 7fe529600700 1 -- 172.21.15.90:0/1095582285 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe53002d050 con 0x562ddfa63df0 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.488+0000 7fe554d52b80 10 client.15009 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.488+0000 7fe554d52b80 1 -- 172.21.15.90:0/1095582285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ddfad1010 con 0x562ddfa63df0 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.488+0000 7fe529600700 1 -- 172.21.15.90:0/1095582285 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe538003c00 con 0x562ddfa63df0 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.489+0000 7fe529600700 1 -- 172.21.15.90:0/1095582285 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe53000fa20 con 0x562ddfa63df0 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.489+0000 7fe554d52b80 10 client.15009 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.489+0000 7fe554d52b80 10 client.15009 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.489+0000 7fe554d52b80 20 client.15009 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.489+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562ddfbfcbc0 0x562ddfbfe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.489+0000 7fe554d52b80 4 client.15009 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:00.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.489+0000 7fe554d52b80 1 -- 172.21.15.90:0/1095582285 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ddfb12d40 con 0x562ddfbfcbc0 2024-05-27T07:45:00.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.490+0000 7fe53ea00700 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562ddfbfcbc0 0x562ddfbfe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.491+0000 7fe53ea00700 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562ddfbfcbc0 0x562ddfbfe030 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9affc871b184b17d server_cookie=4d7c3baa7872d4dd in_seq=0 out_seq=0 2024-05-27T07:45:00.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.491+0000 7fe529600700 10 client.15009 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:45:00.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe529600700 1 -- 172.21.15.90:0/1095582285 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562ddfb12d40 con 0x562ddfbfcbc0 2024-05-27T07:45:00.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe529600700 10 client.15009 handle_command_reply: tid=0 2024-05-27T07:45:00.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe554d52b80 1 client.15009 shutdown 2024-05-27T07:45:00.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe554d52b80 1 -- 172.21.15.90:0/1095582285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 msgr2=0x562ddfbec650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:00.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbec650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe5300106b0 tx=0x7fe530004750 comp rx=0 tx=0).stop 2024-05-27T07:45:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe554d52b80 1 -- 172.21.15.90:0/1095582285 shutdown_connections 2024-05-27T07:45:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562ddfbfcbc0 0x562ddfbfe030 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfbe2e20 0x562ddfbed0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbec650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1095582285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfb4f9a0 0x562ddfbecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe529600700 0 client.15009 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:45:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.493+0000 7fe554d52b80 1 -- 172.21.15.90:0/1095582285 >> 172.21.15.90:0/1095582285 conn(0x562ddfafd340 msgr2=0x562ddfbe22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.494+0000 7fe554d52b80 1 -- 172.21.15.90:0/1095582285 shutdown_connections 2024-05-27T07:45:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.494+0000 7fe554d52b80 1 -- 172.21.15.90:0/1095582285 wait complete. 2024-05-27T07:45:00.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.494+0000 7fe554d52b80 20 client.15009 trim_cache size 0 max 16384 2024-05-27T07:45:00.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe554d52b80 1 Processor -- start 2024-05-27T07:45:00.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe554d52b80 1 -- start start 2024-05-27T07:45:00.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbe7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfb4f9a0 0x562ddfbdde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 0x562ddfbde370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe554d52b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ddfbfc370 con 0x562ddfa63df0 2024-05-27T07:45:00.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe554d52b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ddfbfca40 con 0x562ddfbe2e20 2024-05-27T07:45:00.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe554d52b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562ddfbfd110 con 0x562ddfb4f9a0 2024-05-27T07:45:00.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe53f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbe7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe53f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbe7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59478/0 (socket says 172.21.15.90:59478) 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.517+0000 7fe53f400700 1 -- 172.21.15.90:0/1964869778 learned_addr learned my addr 172.21.15.90:0/1964869778 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.518+0000 7fe53fe00700 1 --2- 172.21.15.90:0/1964869778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 0x562ddfbde370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.518+0000 7fe53f400700 1 -- 172.21.15.90:0/1964869778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfb4f9a0 msgr2=0x562ddfbdde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.518+0000 7fe53ea00700 1 --2- 172.21.15.90:0/1964869778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfb4f9a0 0x562ddfbdde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.518+0000 7fe53f400700 1 --2- 172.21.15.90:0/1964869778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfb4f9a0 0x562ddfbdde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.518+0000 7fe53f400700 1 -- 172.21.15.90:0/1964869778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 msgr2=0x562ddfbde370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.518+0000 7fe53f400700 1 --2- 172.21.15.90:0/1964869778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 0x562ddfbde370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.518+0000 7fe53f400700 1 -- 172.21.15.90:0/1964869778 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ddfbe3a00 con 0x562ddfa63df0 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.518+0000 7fe53fe00700 1 --2- 172.21.15.90:0/1964869778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 0x562ddfbde370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:00.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.518+0000 7fe53f400700 1 --2- 172.21.15.90:0/1964869778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbe7e40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe530005950 tx=0x7fe53000a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19178130481d5f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:00.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.518+0000 7fe52be00700 1 -- 172.21.15.90:0/1964869778 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530021070 con 0x562ddfa63df0 2024-05-27T07:45:00.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.519+0000 7fe52be00700 1 -- 172.21.15.90:0/1964869778 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe530025cc0 con 0x562ddfa63df0 2024-05-27T07:45:00.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.519+0000 7fe52be00700 1 -- 172.21.15.90:0/1964869778 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53001de60 con 0x562ddfa63df0 2024-05-27T07:45:00.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.519+0000 7fe554d52b80 1 -- 172.21.15.90:0/1964869778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 msgr2=0x562ddfbe7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:00.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.519+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1964869778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbe7e40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe530005950 tx=0x7fe53000a3c0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.519+0000 7fe554d52b80 1 -- 172.21.15.90:0/1964869778 shutdown_connections 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.519+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1964869778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfbe2e20 0x562ddfbde370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.519+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1964869778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfa63df0 0x562ddfbe7e40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.519+0000 7fe554d52b80 1 --2- 172.21.15.90:0/1964869778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfb4f9a0 0x562ddfbdde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.519+0000 7fe554d52b80 1 -- 172.21.15.90:0/1964869778 >> 172.21.15.90:0/1964869778 conn(0x562ddfafd340 msgr2=0x562ddfbeaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.520+0000 7fe554d52b80 1 -- 172.21.15.90:0/1964869778 shutdown_connections 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.520+0000 7fe554d52b80 1 -- 172.21.15.90:0/1964869778 wait complete. 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.520+0000 7fe554d52b80 1 -- start start 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.520+0000 7fe554d52b80 1 Processor -- start 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfa63df0 0x562ddfc07b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfb4f9a0 0x562ddfc047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe554d52b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfbe2e20 0x562ddfc04d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe554d52b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ddfbfdeb0 con 0x562ddfbe2e20 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe554d52b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ddfbfd7e0 con 0x562ddfa63df0 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe554d52b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562ddfbfe580 con 0x562ddfb4f9a0 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe554d52b80 10 client.0 _ll_register_callbacks cb 0x562ddfaaac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe53fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfbe2e20 0x562ddfc04d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe53fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfbe2e20 0x562ddfc04d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59480/0 (socket says 172.21.15.90:59480) 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe53fe00700 1 -- 172.21.15.90:0/784507702 learned_addr learned my addr 172.21.15.90:0/784507702 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.521+0000 7fe53f400700 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfa63df0 0x562ddfc07b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe53ea00700 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfb4f9a0 0x562ddfc047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe53fe00700 1 -- 172.21.15.90:0/784507702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfb4f9a0 msgr2=0x562ddfc047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:00.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe53fe00700 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfb4f9a0 0x562ddfc047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe53fe00700 1 -- 172.21.15.90:0/784507702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfa63df0 msgr2=0x562ddfc07b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:00.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe53fe00700 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfa63df0 0x562ddfc07b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe53fe00700 1 -- 172.21.15.90:0/784507702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ddfc05400 con 0x562ddfbe2e20 2024-05-27T07:45:00.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe53fe00700 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfbe2e20 0x562ddfc04d00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe53800b040 tx=0x7fe538007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16595f19b1802418 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:00.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe529600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:00.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe529600700 1 -- 172.21.15.90:0/784507702 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538003a00 con 0x562ddfbe2e20 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe529600700 1 -- 172.21.15.90:0/784507702 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fe538003ba0 con 0x562ddfbe2e20 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.522+0000 7fe554d52b80 1 -- 172.21.15.90:0/784507702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe530025cc0 con 0x562ddfbe2e20 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe529600700 1 -- 172.21.15.90:0/784507702 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe538003da0 con 0x562ddfbe2e20 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe554d52b80 10 client.15015 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe554d52b80 1 -- 172.21.15.90:0/784507702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ddfbe7b30 con 0x562ddfbe2e20 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe529600700 1 -- 172.21.15.90:0/784507702 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe530025cc0 con 0x562ddfbe2e20 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe529600700 1 -- 172.21.15.90:0/784507702 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe538012600 con 0x562ddfbe2e20 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe554d52b80 10 client.15015 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe554d52b80 10 client.15015 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe554d52b80 20 client.15015 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:00.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe554d52b80 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562ddfc6a360 0x562ddfc6b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe554d52b80 4 client.15015 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.523+0000 7fe554d52b80 1 -- 172.21.15.90:0/784507702 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "status"}) -- 0x562ddfb0c520 con 0x562ddfc6a360 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.524+0000 7fe53f400700 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562ddfc6a360 0x562ddfc6b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.525+0000 7fe53f400700 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562ddfc6a360 0x562ddfc6b7d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f4f6d2191cdf53a2 server_cookie=39425838805327dd in_seq=0 out_seq=0 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.525+0000 7fe529600700 10 client.15015 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.526+0000 7fe529600700 1 -- 172.21.15.90:0/784507702 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x562ddfb0c520 con 0x562ddfc6a360 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.526+0000 7fe529600700 10 client.15015 handle_command_reply: tid=0 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.526+0000 7fe554d52b80 1 client.15015 shutdown 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.526+0000 7fe554d52b80 1 -- 172.21.15.90:0/784507702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfbe2e20 msgr2=0x562ddfc04d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.526+0000 7fe554d52b80 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfbe2e20 0x562ddfc04d00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe53800b040 tx=0x7fe538007940 comp rx=0 tx=0).stop 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.526+0000 7fe554d52b80 1 -- 172.21.15.90:0/784507702 shutdown_connections 2024-05-27T07:45:00.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.526+0000 7fe554d52b80 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562ddfc6a360 0x562ddfc6b7d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.526+0000 7fe554d52b80 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ddfbe2e20 0x562ddfc04d00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.527+0000 7fe554d52b80 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ddfa63df0 0x562ddfc07b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.527+0000 7fe529600700 0 client.15015 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:45:00.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.527+0000 7fe554d52b80 1 --2- 172.21.15.90:0/784507702 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ddfb4f9a0 0x562ddfc047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.527+0000 7fe554d52b80 1 -- 172.21.15.90:0/784507702 >> 172.21.15.90:0/784507702 conn(0x562ddfafd340 msgr2=0x562ddfbdde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:00.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.527+0000 7fe554d52b80 1 -- 172.21.15.90:0/784507702 shutdown_connections 2024-05-27T07:45:00.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.527+0000 7fe554d52b80 1 -- 172.21.15.90:0/784507702 wait complete. 2024-05-27T07:45:00.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.527+0000 7fe554d52b80 20 client.15015 trim_cache size 0 max 16384 2024-05-27T07:45:00.528 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:45:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 4, 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24478, 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 146.63375287, 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 91, 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 260.537692338 2024-05-27T07:45:00.529 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:45:00.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 -- 172.21.15.90:0/3165173464 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5240b2900 msgr2=0x7fe5240b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:00.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5240b2900 0x7fe5240b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe530005fd0 tx=0x7fe530005db0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 -- 172.21.15.90:0/3165173464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54010b2d0 msgr2=0x7fe5401a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:00.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54010b2d0 0x7fe5401a1930 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000c00 tx=0x7fe52c0036f0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 -- 172.21.15.90:0/3165173464 shutdown_connections 2024-05-27T07:45:00.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fe5240b2900 0x7fe5240b4e10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe540110960 0x7fe5401a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe54010a980 0x7fe5401a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 --2- 172.21.15.90:0/3165173464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54010b2d0 0x7fe5401a1930 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:00.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 -- 172.21.15.90:0/3165173464 >> 172.21.15.90:0/3165173464 conn(0x7fe540077c70 msgr2=0x7fe54007b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:00.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 -- 172.21.15.90:0/3165173464 shutdown_connections 2024-05-27T07:45:00.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:00.528+0000 7fe546200700 1 -- 172.21.15.90:0/3165173464 wait complete. 2024-05-27T07:45:01.349 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.k status 2024-05-27T07:45:01.792 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:02.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:01 smithi090 ceph-mon[31429]: pgmap v502: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:02.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:01 smithi136 ceph-mon[35324]: pgmap v502: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:02.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:01 smithi129 ceph-mon[35254]: pgmap v502: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:03.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.871+0000 7fac55200700 1 -- 172.21.15.90:0/4172887997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac5010c5b0 msgr2=0x7fac5010c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:03.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.871+0000 7fac55200700 1 --2- 172.21.15.90:0/4172887997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac5010c5b0 0x7fac5010c9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fac40009990 tx=0x7fac40009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.871+0000 7fac55200700 1 -- 172.21.15.90:0/4172887997 shutdown_connections 2024-05-27T07:45:03.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.871+0000 7fac55200700 1 --2- 172.21.15.90:0/4172887997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac50106ec0 0x7fac50113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.871+0000 7fac55200700 1 --2- 172.21.15.90:0/4172887997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac5010c5b0 0x7fac5010c9c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.871+0000 7fac55200700 1 --2- 172.21.15.90:0/4172887997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac50106570 0x7fac50106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.871+0000 7fac55200700 1 -- 172.21.15.90:0/4172887997 >> 172.21.15.90:0/4172887997 conn(0x7fac50102070 msgr2=0x7fac501044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:03.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.871+0000 7fac55200700 1 -- 172.21.15.90:0/4172887997 shutdown_connections 2024-05-27T07:45:03.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.871+0000 7fac55200700 1 -- 172.21.15.90:0/4172887997 wait complete. 2024-05-27T07:45:03.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.872+0000 7fac55200700 1 Processor -- start 2024-05-27T07:45:03.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.872+0000 7fac55200700 1 -- start start 2024-05-27T07:45:03.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.873+0000 7fac55200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac50106570 0x7fac50078dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.873+0000 7fac55200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac50106ec0 0x7fac50079310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.873+0000 7fac55200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac5010c5b0 0x7fac50079850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.873+0000 7fac55200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac50116570 con 0x7fac5010c5b0 2024-05-27T07:45:03.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.873+0000 7fac55200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac501163f0 con 0x7fac50106570 2024-05-27T07:45:03.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.873+0000 7fac55200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac501166f0 con 0x7fac50106ec0 2024-05-27T07:45:03.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.873+0000 7fac4e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac5010c5b0 0x7fac50079850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.873+0000 7fac4e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac5010c5b0 0x7fac50079850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59504/0 (socket says 172.21.15.90:59504) 2024-05-27T07:45:03.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.873+0000 7fac4e800700 1 -- 172.21.15.90:0/312057947 learned_addr learned my addr 172.21.15.90:0/312057947 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:03.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.874+0000 7fac4d400700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac50106ec0 0x7fac50079310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.874+0000 7fac4de00700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac50106570 0x7fac50078dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.874+0000 7fac4e800700 1 -- 172.21.15.90:0/312057947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac50106ec0 msgr2=0x7fac50079310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.874+0000 7fac4e800700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac50106ec0 0x7fac50079310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.874+0000 7fac4e800700 1 -- 172.21.15.90:0/312057947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac50106570 msgr2=0x7fac50078dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.874+0000 7fac4e800700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac50106570 0x7fac50078dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.874+0000 7fac4e800700 1 -- 172.21.15.90:0/312057947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac501ad500 con 0x7fac5010c5b0 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.874+0000 7fac4e800700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac5010c5b0 0x7fac50079850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fac48007ee0 tx=0x7fac4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.874+0000 7fac46a00700 1 -- 172.21.15.90:0/312057947 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4800dbd0 con 0x7fac5010c5b0 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.875+0000 7fac46a00700 1 -- 172.21.15.90:0/312057947 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac4800dd70 con 0x7fac5010c5b0 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.875+0000 7fac46a00700 1 -- 172.21.15.90:0/312057947 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac480058d0 con 0x7fac5010c5b0 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.875+0000 7fac55200700 1 -- 172.21.15.90:0/312057947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac501ad790 con 0x7fac5010c5b0 2024-05-27T07:45:03.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.875+0000 7fac55200700 1 -- 172.21.15.90:0/312057947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac501adc80 con 0x7fac5010c5b0 2024-05-27T07:45:03.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.878+0000 7fac46a00700 1 -- 172.21.15.90:0/312057947 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fac4801a020 con 0x7fac5010c5b0 2024-05-27T07:45:03.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.879+0000 7fac63de3b80 1 Processor -- start 2024-05-27T07:45:03.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.880+0000 7fac63de3b80 1 -- start start 2024-05-27T07:45:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.880+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 0x5585df76e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.880+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df76ece0 0x5585df768820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.880+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 0x5585df769170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.880+0000 7fac63de3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5585df696df0 con 0x5585df76ece0 2024-05-27T07:45:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.880+0000 7fac63de3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5585df686990 con 0x5585df5e9df0 2024-05-27T07:45:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.880+0000 7fac63de3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5585df68a9d0 con 0x5585df768d60 2024-05-27T07:45:03.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.880+0000 7fac46a00700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac340b2900 0x7fac340b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.881+0000 7fac4d400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 0x5585df76e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.881+0000 7fac46a00700 1 -- 172.21.15.90:0/312057947 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fac480e1e30 con 0x7fac5010c5b0 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.881+0000 7fac4e800700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac340b2900 0x7fac340b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.881+0000 7fac4d400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 0x5585df76e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:36916/0 (socket says 172.21.15.90:36916) 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.881+0000 7fac4d400700 1 -- 172.21.15.90:0/370625685 learned_addr learned my addr 172.21.15.90:0/370625685 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.881+0000 7fac4d400700 1 --2- 172.21.15.90:0/370625685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 0x5585df769170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.881+0000 7fac4de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df76ece0 0x5585df768820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.882+0000 7fac4d400700 1 -- 172.21.15.90:0/370625685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 msgr2=0x5585df76e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.882+0000 7fac4d400700 1 --2- 172.21.15.90:0/370625685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 0x5585df76e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.882+0000 7fac4d400700 1 -- 172.21.15.90:0/370625685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df76ece0 msgr2=0x5585df768820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.882+0000 7fac4d400700 1 --2- 172.21.15.90:0/370625685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df76ece0 0x5585df768820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.882+0000 7fac4d400700 1 -- 172.21.15.90:0/370625685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585df64f3d0 con 0x5585df768d60 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.882+0000 7fac4de00700 1 --2- 172.21.15.90:0/370625685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df76ece0 0x5585df768820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:45:03.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.882+0000 7fac4e800700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac340b2900 0x7fac340b4e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fac4800dfd0 tx=0x7fac4800df20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.882+0000 7fac4d400700 1 --2- 172.21.15.90:0/370625685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 0x5585df769170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac3c005c50 tx=0x7fac3c00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0deaf503417f373 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.883+0000 7fac3b400700 1 -- 172.21.15.90:0/370625685 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c004030 con 0x5585df768d60 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.883+0000 7fac3b400700 1 -- 172.21.15.90:0/370625685 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac3c003680 con 0x5585df768d60 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.883+0000 7fac3b400700 1 -- 172.21.15.90:0/370625685 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c011700 con 0x5585df768d60 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.884+0000 7fac63de3b80 1 -- 172.21.15.90:0/370625685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 msgr2=0x5585df769170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.884+0000 7fac63de3b80 1 --2- 172.21.15.90:0/370625685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 0x5585df769170 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fac3c005c50 tx=0x7fac3c00b260 comp rx=0 tx=0).stop 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.884+0000 7fac63de3b80 1 -- 172.21.15.90:0/370625685 shutdown_connections 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.884+0000 7fac63de3b80 1 --2- 172.21.15.90:0/370625685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 0x5585df769170 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.884+0000 7fac63de3b80 1 --2- 172.21.15.90:0/370625685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 0x5585df76e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.884+0000 7fac63de3b80 1 --2- 172.21.15.90:0/370625685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df76ece0 0x5585df768820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.884+0000 7fac63de3b80 1 -- 172.21.15.90:0/370625685 >> 172.21.15.90:0/370625685 conn(0x5585df649f70 msgr2=0x5585df763880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:03.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.884+0000 7fac63de3b80 1 -- 172.21.15.90:0/370625685 shutdown_connections 2024-05-27T07:45:03.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.884+0000 7fac63de3b80 1 -- 172.21.15.90:0/370625685 wait complete. 2024-05-27T07:45:03.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.885+0000 7fac63de3b80 1 -- start start 2024-05-27T07:45:03.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.885+0000 7fac63de3b80 1 Processor -- start 2024-05-27T07:45:03.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 0x5585df7724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df768d60 0x5585df772a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df76ece0 0x5585df772f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac63de3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5585df58cf50 con 0x5585df768d60 2024-05-27T07:45:03.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac63de3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5585df627710 con 0x5585df5e9df0 2024-05-27T07:45:03.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac63de3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5585df7757b0 con 0x5585df76ece0 2024-05-27T07:45:03.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac63de3b80 10 client.0 _ll_register_callbacks cb 0x5585df630c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:03.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac4de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df768d60 0x5585df772a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac4de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df768d60 0x5585df772a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59522/0 (socket says 172.21.15.90:59522) 2024-05-27T07:45:03.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac4de00700 1 -- 172.21.15.90:0/2851282747 learned_addr learned my addr 172.21.15.90:0/2851282747 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:03.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.886+0000 7fac4d400700 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df76ece0 0x5585df772f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.887+0000 7fac4d400700 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 0x5585df7724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.887+0000 7fac4de00700 1 -- 172.21.15.90:0/2851282747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df76ece0 msgr2=0x5585df772f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.887+0000 7fac4de00700 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df76ece0 0x5585df772f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.887+0000 7fac4de00700 1 -- 172.21.15.90:0/2851282747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 msgr2=0x5585df7724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.887+0000 7fac4de00700 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 0x5585df7724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.887+0000 7fac4de00700 1 -- 172.21.15.90:0/2851282747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585df5d6880 con 0x5585df768d60 2024-05-27T07:45:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.887+0000 7fac4de00700 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df768d60 0x5585df772a00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac40009990 tx=0x7fac40003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a8468c54c00a05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.887+0000 7fac2fe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:03.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.892+0000 7fac2fe00700 1 -- 172.21.15.90:0/2851282747 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40005450 con 0x5585df768d60 2024-05-27T07:45:03.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.892+0000 7fac2fe00700 1 -- 172.21.15.90:0/2851282747 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fac400055f0 con 0x5585df768d60 2024-05-27T07:45:03.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.892+0000 7fac63de3b80 1 -- 172.21.15.90:0/2851282747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac3c003680 con 0x5585df768d60 2024-05-27T07:45:03.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.892+0000 7fac2fe00700 1 -- 172.21.15.90:0/2851282747 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac40030050 con 0x5585df768d60 2024-05-27T07:45:03.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.892+0000 7fac63de3b80 10 client.15027 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:03.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.892+0000 7fac63de3b80 1 -- 172.21.15.90:0/2851282747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5585df657010 con 0x5585df768d60 2024-05-27T07:45:03.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.893+0000 7fac2fe00700 1 -- 172.21.15.90:0/2851282747 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac3c003680 con 0x5585df768d60 2024-05-27T07:45:03.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.893+0000 7fac2fe00700 1 -- 172.21.15.90:0/2851282747 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac4000f540 con 0x5585df768d60 2024-05-27T07:45:03.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.893+0000 7fac63de3b80 10 client.15027 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:03.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.893+0000 7fac63de3b80 10 client.15027 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-27T07:45:03.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.893+0000 7fac63de3b80 20 client.15027 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:03.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.893+0000 7fac63de3b80 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5585df782680 0x5585df783af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.893+0000 7fac63de3b80 4 client.15027 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:03.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.893+0000 7fac63de3b80 1 -- 172.21.15.90:0/2851282747 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585df698d40 con 0x5585df782680 2024-05-27T07:45:03.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.894+0000 7fac4d400700 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5585df782680 0x5585df783af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.895+0000 7fac4d400700 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5585df782680 0x5585df783af0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=87723b5a9b66ce23 server_cookie=af43f7e9f185289c in_seq=0 out_seq=0 2024-05-27T07:45:03.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.895+0000 7fac2fe00700 10 client.15027 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:45:03.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.898+0000 7fac2fe00700 1 -- 172.21.15.90:0/2851282747 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5585df698d40 con 0x5585df782680 2024-05-27T07:45:03.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.898+0000 7fac2fe00700 10 client.15027 handle_command_reply: tid=0 2024-05-27T07:45:03.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 client.15027 shutdown 2024-05-27T07:45:03.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 -- 172.21.15.90:0/2851282747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df768d60 msgr2=0x5585df772a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:03.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df768d60 0x5585df772a00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac40009990 tx=0x7fac40003f10 comp rx=0 tx=0).stop 2024-05-27T07:45:03.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 -- 172.21.15.90:0/2851282747 shutdown_connections 2024-05-27T07:45:03.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5585df782680 0x5585df783af0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df76ece0 0x5585df772f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df5e9df0 0x5585df7724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac2fe00700 0 client.15027 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:45:03.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 --2- 172.21.15.90:0/2851282747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df768d60 0x5585df772a00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 -- 172.21.15.90:0/2851282747 >> 172.21.15.90:0/2851282747 conn(0x5585df649f70 msgr2=0x5585df763530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:03.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 -- 172.21.15.90:0/2851282747 shutdown_connections 2024-05-27T07:45:03.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 1 -- 172.21.15.90:0/2851282747 wait complete. 2024-05-27T07:45:03.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.899+0000 7fac63de3b80 20 client.15027 trim_cache size 0 max 16384 2024-05-27T07:45:03.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.922+0000 7fac63de3b80 1 Processor -- start 2024-05-27T07:45:03.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.923+0000 7fac63de3b80 1 -- start start 2024-05-27T07:45:03.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.923+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df768820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.923+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df768d60 0x5585df77eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.923+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df76ece0 0x5585df76c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.923+0000 7fac63de3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5585df781e30 con 0x5585df5e9df0 2024-05-27T07:45:03.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.923+0000 7fac63de3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5585df782500 con 0x5585df768d60 2024-05-27T07:45:03.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.923+0000 7fac63de3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5585df782bd0 con 0x5585df76ece0 2024-05-27T07:45:03.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.923+0000 7fac4d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df768820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.923+0000 7fac4d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df768820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59536/0 (socket says 172.21.15.90:59536) 2024-05-27T07:45:03.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.923+0000 7fac4d400700 1 -- 172.21.15.90:0/3112258810 learned_addr learned my addr 172.21.15.90:0/3112258810 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:03.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.924+0000 7fac4de00700 1 --2- 172.21.15.90:0/3112258810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df768d60 0x5585df77eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.924+0000 7fac4d400700 1 -- 172.21.15.90:0/3112258810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df76ece0 msgr2=0x5585df76c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:03.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.924+0000 7fac4d400700 1 --2- 172.21.15.90:0/3112258810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df76ece0 0x5585df76c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.924+0000 7fac4d400700 1 -- 172.21.15.90:0/3112258810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df768d60 msgr2=0x5585df77eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:03.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.924+0000 7fac4d400700 1 --2- 172.21.15.90:0/3112258810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df768d60 0x5585df77eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.924+0000 7fac4d400700 1 -- 172.21.15.90:0/3112258810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585df76dc90 con 0x5585df5e9df0 2024-05-27T07:45:03.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.924+0000 7fac4d400700 1 --2- 172.21.15.90:0/3112258810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df768820 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fac3c014630 tx=0x7fac3c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32765217a236785a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:03.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.924+0000 7fac3aa00700 1 -- 172.21.15.90:0/3112258810 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c003880 con 0x5585df5e9df0 2024-05-27T07:45:03.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.924+0000 7fac3aa00700 1 -- 172.21.15.90:0/3112258810 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac3c0142a0 con 0x5585df5e9df0 2024-05-27T07:45:03.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.924+0000 7fac3aa00700 1 -- 172.21.15.90:0/3112258810 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c011700 con 0x5585df5e9df0 2024-05-27T07:45:03.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.925+0000 7fac63de3b80 1 -- 172.21.15.90:0/3112258810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 msgr2=0x5585df768820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:03.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.925+0000 7fac63de3b80 1 --2- 172.21.15.90:0/3112258810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df768820 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fac3c014630 tx=0x7fac3c005ce0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.925+0000 7fac63de3b80 1 -- 172.21.15.90:0/3112258810 shutdown_connections 2024-05-27T07:45:03.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.926+0000 7fac63de3b80 1 --2- 172.21.15.90:0/3112258810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df76ece0 0x5585df76c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.926+0000 7fac63de3b80 1 --2- 172.21.15.90:0/3112258810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df768820 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.926+0000 7fac63de3b80 1 --2- 172.21.15.90:0/3112258810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df768d60 0x5585df77eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.926+0000 7fac63de3b80 1 -- 172.21.15.90:0/3112258810 >> 172.21.15.90:0/3112258810 conn(0x5585df649f70 msgr2=0x5585df76c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:03.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.926+0000 7fac63de3b80 1 -- 172.21.15.90:0/3112258810 shutdown_connections 2024-05-27T07:45:03.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.926+0000 7fac63de3b80 1 -- 172.21.15.90:0/3112258810 wait complete. 2024-05-27T07:45:03.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.926+0000 7fac63de3b80 1 -- start start 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.926+0000 7fac63de3b80 1 Processor -- start 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.927+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df78d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.927+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 0x5585df77daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.927+0000 7fac63de3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df76ece0 0x5585df77dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.927+0000 7fac63de3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5585df783970 con 0x5585df5e9df0 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.927+0000 7fac63de3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5585df7832a0 con 0x5585df76ece0 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.927+0000 7fac63de3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5585df784040 con 0x5585df768d60 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.927+0000 7fac63de3b80 10 client.0 _ll_register_callbacks cb 0x5585df76e1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.927+0000 7fac4d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df78d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.927+0000 7fac4d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df78d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59552/0 (socket says 172.21.15.90:59552) 2024-05-27T07:45:03.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.927+0000 7fac4d400700 1 -- 172.21.15.90:0/3473693823 learned_addr learned my addr 172.21.15.90:0/3473693823 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:03.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac4de00700 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 0x5585df77daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac4d400700 1 -- 172.21.15.90:0/3473693823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 msgr2=0x5585df77daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:03.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac4d400700 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 0x5585df77daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac4d400700 1 -- 172.21.15.90:0/3473693823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df76ece0 msgr2=0x5585df77dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:03.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac4d400700 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df76ece0 0x5585df77dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac4d400700 1 -- 172.21.15.90:0/3473693823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585df77e520 con 0x5585df5e9df0 2024-05-27T07:45:03.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac4d400700 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df78d440 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fac3c001eb0 tx=0x7fac3c003f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87749b92d3534655 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac2fe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac2fe00700 1 -- 172.21.15.90:0/3473693823 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c03a060 con 0x5585df5e9df0 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac2fe00700 1 -- 172.21.15.90:0/3473693823 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fac3c003680 con 0x5585df5e9df0 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.928+0000 7fac63de3b80 1 -- 172.21.15.90:0/3473693823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac3c0142a0 con 0x5585df5e9df0 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac2fe00700 1 -- 172.21.15.90:0/3473693823 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac3c003880 con 0x5585df5e9df0 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac63de3b80 10 client.15033 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac63de3b80 1 -- 172.21.15.90:0/3473693823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5585df78bbe0 con 0x5585df5e9df0 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac2fe00700 1 -- 172.21.15.90:0/3473693823 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac3c0142a0 con 0x5585df5e9df0 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac2fe00700 1 -- 172.21.15.90:0/3473693823 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fac3c012450 con 0x5585df5e9df0 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac63de3b80 10 client.15033 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac63de3b80 10 client.15033 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac63de3b80 20 client.15033 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:03.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac63de3b80 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5585df7ef8f0 0x5585df7f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:03.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac63de3b80 4 client.15033 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:45:03.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.929+0000 7fac63de3b80 1 -- 172.21.15.90:0/3473693823 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "status"}) -- 0x5585df692520 con 0x5585df7ef8f0 2024-05-27T07:45:03.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.930+0000 7fac4de00700 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5585df7ef8f0 0x5585df7f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:03.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.931+0000 7fac4de00700 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5585df7ef8f0 0x5585df7f0d60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da6a7e38a1631802 server_cookie=bac18a8505d6e633 in_seq=0 out_seq=0 2024-05-27T07:45:03.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.931+0000 7fac2fe00700 10 client.15033 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:45:03.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac2fe00700 1 -- 172.21.15.90:0/3473693823 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5585df692520 con 0x5585df7ef8f0 2024-05-27T07:45:03.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac2fe00700 10 client.15033 handle_command_reply: tid=0 2024-05-27T07:45:03.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 client.15033 shutdown 2024-05-27T07:45:03.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 -- 172.21.15.90:0/3473693823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 msgr2=0x5585df78d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:03.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df78d440 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fac3c001eb0 tx=0x7fac3c003f20 comp rx=0 tx=0).stop 2024-05-27T07:45:03.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 -- 172.21.15.90:0/3473693823 shutdown_connections 2024-05-27T07:45:03.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5585df7ef8f0 0x5585df7f0d60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585df76ece0 0x5585df77dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585df5e9df0 0x5585df78d440 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 --2- 172.21.15.90:0/3473693823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585df768d60 0x5585df77daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 -- 172.21.15.90:0/3473693823 >> 172.21.15.90:0/3473693823 conn(0x5585df649f70 msgr2=0x5585df76c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:03.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac2fe00700 0 client.15033 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:45:03.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 -- 172.21.15.90:0/3473693823 shutdown_connections 2024-05-27T07:45:03.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.932+0000 7fac63de3b80 1 -- 172.21.15.90:0/3473693823 wait complete. 2024-05-27T07:45:03.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac63de3b80 20 client.15033 trim_cache size 0 max 16384 2024-05-27T07:45:03.933 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:45:03.933 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 3, 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34406, 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 151.054927416, 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 91, 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 293.96694073499998 2024-05-27T07:45:03.934 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:45:03.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac55200700 1 -- 172.21.15.90:0/312057947 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac340b2900 msgr2=0x7fac340b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:03.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac55200700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac340b2900 0x7fac340b4e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fac4800dfd0 tx=0x7fac4800df20 comp rx=0 tx=0).stop 2024-05-27T07:45:03.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac55200700 1 -- 172.21.15.90:0/312057947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac5010c5b0 msgr2=0x7fac50079850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:03.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac55200700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac5010c5b0 0x7fac50079850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fac48007ee0 tx=0x7fac4800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac55200700 1 -- 172.21.15.90:0/312057947 shutdown_connections 2024-05-27T07:45:03.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac55200700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fac340b2900 0x7fac340b4e10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac55200700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac5010c5b0 0x7fac50079850 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac55200700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac50106570 0x7fac50078dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac55200700 1 --2- 172.21.15.90:0/312057947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac50106ec0 0x7fac50079310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:03.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.933+0000 7fac55200700 1 -- 172.21.15.90:0/312057947 >> 172.21.15.90:0/312057947 conn(0x7fac50102070 msgr2=0x7fac50104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:03.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.934+0000 7fac55200700 1 -- 172.21.15.90:0/312057947 shutdown_connections 2024-05-27T07:45:03.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:03.934+0000 7fac55200700 1 -- 172.21.15.90:0/312057947 wait complete. 2024-05-27T07:45:04.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:03 smithi090 ceph-mon[31429]: pgmap v503: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:04.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:03 smithi136 ceph-mon[35324]: pgmap v503: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:04.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:03 smithi129 ceph-mon[35254]: pgmap v503: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:04.908 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-05-27T07:45:05.353 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:05.944 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:05 smithi090 ceph-mon[31429]: pgmap v504: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:06.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:05 smithi136 ceph-mon[35324]: pgmap v504: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:06.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:05 smithi129 ceph-mon[35254]: pgmap v504: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.791+0000 7f6367200700 1 -- 172.21.15.90:0/3291719599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368078630 msgr2=0x7f6368114af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.791+0000 7f6367200700 1 --2- 172.21.15.90:0/3291719599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368078630 0x7f6368114af0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6354009990 tx=0x7f6354009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.791+0000 7f6367200700 1 -- 172.21.15.90:0/3291719599 shutdown_connections 2024-05-27T07:45:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.791+0000 7f6367200700 1 --2- 172.21.15.90:0/3291719599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6368115030 0x7f6368117450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.791+0000 7f6367200700 1 --2- 172.21.15.90:0/3291719599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6368077ce0 0x7f63680780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.791+0000 7f6367200700 1 --2- 172.21.15.90:0/3291719599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368078630 0x7f6368114af0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.791+0000 7f6367200700 1 -- 172.21.15.90:0/3291719599 >> 172.21.15.90:0/3291719599 conn(0x7f6368101ec0 msgr2=0x7f63681042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.792+0000 7f6367200700 1 -- 172.21.15.90:0/3291719599 shutdown_connections 2024-05-27T07:45:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.792+0000 7f6367200700 1 -- 172.21.15.90:0/3291719599 wait complete. 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.793+0000 7f6367200700 1 Processor -- start 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.793+0000 7f6367200700 1 -- start start 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.794+0000 7f6367200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6368077ce0 0x7f63681a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.794+0000 7f6367200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6368078630 0x7f63681a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.794+0000 7f6367200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368115030 0x7f63681a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.794+0000 7f6367200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f636811a7b0 con 0x7f6368115030 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.794+0000 7f6367200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f636811a630 con 0x7f6368078630 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.794+0000 7f6367200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f636811a930 con 0x7f6368077ce0 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.794+0000 7f6366800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368115030 0x7f63681a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.794+0000 7f6366800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368115030 0x7f63681a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59572/0 (socket says 172.21.15.90:59572) 2024-05-27T07:45:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.794+0000 7f6366800700 1 -- 172.21.15.90:0/2481802369 learned_addr learned my addr 172.21.15.90:0/2481802369 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f6365e00700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6368077ce0 0x7f63681a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f6365400700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6368078630 0x7f63681a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f6366800700 1 -- 172.21.15.90:0/2481802369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6368077ce0 msgr2=0x7f63681a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f6366800700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6368077ce0 0x7f63681a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f6366800700 1 -- 172.21.15.90:0/2481802369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6368078630 msgr2=0x7f63681a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f6366800700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6368078630 0x7f63681a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f6366800700 1 -- 172.21.15.90:0/2481802369 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63681a5fe0 con 0x7f6368115030 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f6366800700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368115030 0x7f63681a59f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6360007d70 tx=0x7f636000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f635ea00700 1 -- 172.21.15.90:0/2481802369 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360016020 con 0x7f6368115030 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f6367200700 1 -- 172.21.15.90:0/2481802369 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63681a62d0 con 0x7f6368115030 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.795+0000 7f635ea00700 1 -- 172.21.15.90:0/2481802369 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f636000ad40 con 0x7f6368115030 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.796+0000 7f635ea00700 1 -- 172.21.15.90:0/2481802369 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63600115e0 con 0x7f6368115030 2024-05-27T07:45:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.796+0000 7f6367200700 1 -- 172.21.15.90:0/2481802369 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63681a6810 con 0x7f6368115030 2024-05-27T07:45:07.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.800+0000 7f635ea00700 1 -- 172.21.15.90:0/2481802369 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6360019020 con 0x7f6368115030 2024-05-27T07:45:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.801+0000 7f637b4e6b80 1 Processor -- start 2024-05-27T07:45:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.801+0000 7f637b4e6b80 1 -- start start 2024-05-27T07:45:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.801+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d43819a0 0x5564d4415700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.801+0000 7f635ea00700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f634c0b2880 0x7f634c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.801+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d4415c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.801+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 0x5564d4421690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.801+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5564d4342df0 con 0x5564d4295df0 2024-05-27T07:45:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.801+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5564d4332990 con 0x5564d441f270 2024-05-27T07:45:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.801+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5564d43369d0 con 0x5564d43819a0 2024-05-27T07:45:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.802+0000 7f635ea00700 1 -- 172.21.15.90:0/2481802369 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f63600e25d0 con 0x7f6368115030 2024-05-27T07:45:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.802+0000 7f6365e00700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f634c0b2880 0x7f634c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.802+0000 7f6365400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d4415c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.802+0000 7f6365400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d4415c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59582/0 (socket says 172.21.15.90:59582) 2024-05-27T07:45:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.802+0000 7f6365400700 1 -- 172.21.15.90:0/689478306 learned_addr learned my addr 172.21.15.90:0/689478306 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.802+0000 7f6366800700 1 --2- 172.21.15.90:0/689478306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 0x5564d4421690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.803+0000 7f6365e00700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f634c0b2880 0x7f634c0b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6358005950 tx=0x7f63580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:07.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.803+0000 7f6365400700 1 -- 172.21.15.90:0/689478306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d43819a0 msgr2=0x5564d4415700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:45:07.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.803+0000 7f6365400700 1 --2- 172.21.15.90:0/689478306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d43819a0 0x5564d4415700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.803+0000 7f6365400700 1 -- 172.21.15.90:0/689478306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 msgr2=0x5564d4421690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:07.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.803+0000 7f6365400700 1 --2- 172.21.15.90:0/689478306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 0x5564d4421690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.803+0000 7f6365400700 1 -- 172.21.15.90:0/689478306 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d42fb3d0 con 0x5564d4295df0 2024-05-27T07:45:07.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.803+0000 7f6366800700 1 --2- 172.21.15.90:0/689478306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 0x5564d4421690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:07.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.803+0000 7f6365400700 1 --2- 172.21.15.90:0/689478306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d4415c40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f635400b5c0 tx=0x7f6354003720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa10592ca669a0f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:07.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.804+0000 7f6353400700 1 -- 172.21.15.90:0/689478306 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354026070 con 0x5564d4295df0 2024-05-27T07:45:07.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.804+0000 7f6353400700 1 -- 172.21.15.90:0/689478306 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6354004da0 con 0x5564d4295df0 2024-05-27T07:45:07.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.804+0000 7f6353400700 1 -- 172.21.15.90:0/689478306 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63540056e0 con 0x5564d4295df0 2024-05-27T07:45:07.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.805+0000 7f637b4e6b80 1 -- 172.21.15.90:0/689478306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 msgr2=0x5564d4415c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:07.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.805+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/689478306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d4415c40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f635400b5c0 tx=0x7f6354003720 comp rx=0 tx=0).stop 2024-05-27T07:45:07.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.805+0000 7f637b4e6b80 1 -- 172.21.15.90:0/689478306 shutdown_connections 2024-05-27T07:45:07.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.805+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/689478306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 0x5564d4421690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.805+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/689478306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d43819a0 0x5564d4415700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.805+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/689478306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d4415c40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.805+0000 7f637b4e6b80 1 -- 172.21.15.90:0/689478306 >> 172.21.15.90:0/689478306 conn(0x5564d432f340 msgr2=0x5564d44106c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:07.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.805+0000 7f637b4e6b80 1 -- 172.21.15.90:0/689478306 shutdown_connections 2024-05-27T07:45:07.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.805+0000 7f637b4e6b80 1 -- 172.21.15.90:0/689478306 wait complete. 2024-05-27T07:45:07.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.806+0000 7f637b4e6b80 1 -- start start 2024-05-27T07:45:07.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.806+0000 7f637b4e6b80 1 Processor -- start 2024-05-27T07:45:07.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d441c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d43819a0 0x5564d441c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 0x5564d441cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5564d4238f50 con 0x5564d4295df0 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5564d42d3710 con 0x5564d441f270 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5564d4421d50 con 0x5564d43819a0 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f637b4e6b80 10 client.0 _ll_register_callbacks cb 0x5564d42dcc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f6365e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d441c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f6365e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d441c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59590/0 (socket says 172.21.15.90:59590) 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f6365e00700 1 -- 172.21.15.90:0/1709419295 learned_addr learned my addr 172.21.15.90:0/1709419295 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f6365400700 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d43819a0 0x5564d441c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.807+0000 7f6366800700 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 0x5564d441cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.808+0000 7f6365e00700 1 -- 172.21.15.90:0/1709419295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d43819a0 msgr2=0x5564d441c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.808+0000 7f6365e00700 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d43819a0 0x5564d441c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.808+0000 7f6365e00700 1 -- 172.21.15.90:0/1709419295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 msgr2=0x5564d441cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.808+0000 7f6365e00700 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 0x5564d441cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.808+0000 7f6365e00700 1 -- 172.21.15.90:0/1709419295 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d4282880 con 0x5564d4295df0 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.808+0000 7f6365e00700 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d441c1f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f635800fee0 tx=0x7f6358004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4f732920e8f9ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.808+0000 7f6347e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.808+0000 7f6347e00700 1 -- 172.21.15.90:0/1709419295 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358022070 con 0x5564d4295df0 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.808+0000 7f6347e00700 1 -- 172.21.15.90:0/1709419295 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f635801e650 con 0x5564d4295df0 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.809+0000 7f637b4e6b80 1 -- 172.21.15.90:0/1709419295 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6354004da0 con 0x5564d4295df0 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.809+0000 7f6347e00700 1 -- 172.21.15.90:0/1709419295 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f635802e050 con 0x5564d4295df0 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.809+0000 7f637b4e6b80 10 client.15045 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:07.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.809+0000 7f637b4e6b80 1 -- 172.21.15.90:0/1709419295 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564d4303010 con 0x5564d4295df0 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.809+0000 7f6347e00700 1 -- 172.21.15.90:0/1709419295 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6354004da0 con 0x5564d4295df0 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.809+0000 7f6347e00700 1 -- 172.21.15.90:0/1709419295 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f63580266b0 con 0x5564d4295df0 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.810+0000 7f637b4e6b80 10 client.15045 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.810+0000 7f637b4e6b80 10 client.15045 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.810+0000 7f637b4e6b80 20 client.15045 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.810+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564d442ecd0 0x5564d4430140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.810+0000 7f637b4e6b80 4 client.15045 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.810+0000 7f637b4e6b80 1 -- 172.21.15.90:0/1709419295 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564d4344d40 con 0x5564d442ecd0 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.810+0000 7f6365400700 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564d442ecd0 0x5564d4430140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.812+0000 7f6347e00700 10 client.15045 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.812+0000 7f6365400700 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564d442ecd0 0x5564d4430140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c1d3cc2f68cae0a server_cookie=5af8903ffdabd669 in_seq=0 out_seq=0 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.814+0000 7f6347e00700 1 -- 172.21.15.90:0/1709419295 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5564d4344d40 con 0x5564d442ecd0 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.814+0000 7f6347e00700 10 client.15045 handle_command_reply: tid=0 2024-05-27T07:45:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.814+0000 7f637b4e6b80 1 client.15045 shutdown 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.814+0000 7f637b4e6b80 1 -- 172.21.15.90:0/1709419295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 msgr2=0x5564d441c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.814+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d441c1f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f635800fee0 tx=0x7f6358004750 comp rx=0 tx=0).stop 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.815+0000 7f637b4e6b80 1 -- 172.21.15.90:0/1709419295 shutdown_connections 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.815+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564d442ecd0 0x5564d4430140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.815+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d441f270 0x5564d441cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.815+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d441c1f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.815+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/1709419295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d43819a0 0x5564d441c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.815+0000 7f637b4e6b80 1 -- 172.21.15.90:0/1709419295 >> 172.21.15.90:0/1709419295 conn(0x5564d432f340 msgr2=0x5564d44102b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.815+0000 7f6347e00700 0 client.15045 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.815+0000 7f637b4e6b80 1 -- 172.21.15.90:0/1709419295 shutdown_connections 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.815+0000 7f637b4e6b80 1 -- 172.21.15.90:0/1709419295 wait complete. 2024-05-27T07:45:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.815+0000 7f637b4e6b80 20 client.15045 trim_cache size 0 max 16384 2024-05-27T07:45:07.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.838+0000 7f637b4e6b80 1 Processor -- start 2024-05-27T07:45:07.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.838+0000 7f637b4e6b80 1 -- start start 2024-05-27T07:45:07.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.838+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d44106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.838+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d43819a0 0x5564d4417a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.838+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 0x5564d442b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.838+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5564d442e480 con 0x5564d4295df0 2024-05-27T07:45:07.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.838+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5564d442eb50 con 0x5564d43819a0 2024-05-27T07:45:07.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.838+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5564d442f220 con 0x5564d441f270 2024-05-27T07:45:07.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.839+0000 7f6365e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d44106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.839+0000 7f6365e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d44106f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59604/0 (socket says 172.21.15.90:59604) 2024-05-27T07:45:07.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.839+0000 7f6365e00700 1 -- 172.21.15.90:0/3551231233 learned_addr learned my addr 172.21.15.90:0/3551231233 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:07.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.839+0000 7f6365400700 1 --2- 172.21.15.90:0/3551231233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d43819a0 0x5564d4417a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.839+0000 7f6365e00700 1 -- 172.21.15.90:0/3551231233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 msgr2=0x5564d442b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:07.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.839+0000 7f6366800700 1 --2- 172.21.15.90:0/3551231233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 0x5564d442b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.839+0000 7f6365e00700 1 --2- 172.21.15.90:0/3551231233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 0x5564d442b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.839+0000 7f6365e00700 1 -- 172.21.15.90:0/3551231233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d43819a0 msgr2=0x5564d4417a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:07.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.839+0000 7f6365e00700 1 --2- 172.21.15.90:0/3551231233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d43819a0 0x5564d4417a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.839+0000 7f6365e00700 1 -- 172.21.15.90:0/3551231233 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d4410fb0 con 0x5564d4295df0 2024-05-27T07:45:07.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.840+0000 7f6365e00700 1 --2- 172.21.15.90:0/3551231233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d44106f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f635800f9d0 tx=0x7f635800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3d084bdae5d5a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:07.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.840+0000 7f6352a00700 1 -- 172.21.15.90:0/3551231233 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358022050 con 0x5564d4295df0 2024-05-27T07:45:07.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.840+0000 7f6352a00700 1 -- 172.21.15.90:0/3551231233 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6358026840 con 0x5564d4295df0 2024-05-27T07:45:07.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.840+0000 7f6352a00700 1 -- 172.21.15.90:0/3551231233 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358004a20 con 0x5564d4295df0 2024-05-27T07:45:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.841+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3551231233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 msgr2=0x5564d44106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:07.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.841+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/3551231233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d44106f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f635800f9d0 tx=0x7f635800f730 comp rx=0 tx=0).stop 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.841+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3551231233 shutdown_connections 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.841+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/3551231233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 0x5564d442b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.841+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/3551231233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d4295df0 0x5564d44106f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.841+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/3551231233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d43819a0 0x5564d4417a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.841+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3551231233 >> 172.21.15.90:0/3551231233 conn(0x5564d432f340 msgr2=0x5564d44140d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.841+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3551231233 shutdown_connections 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.841+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3551231233 wait complete. 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.842+0000 7f637b4e6b80 1 -- start start 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.842+0000 7f637b4e6b80 1 Processor -- start 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.842+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d4295df0 0x5564d4433260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.842+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d43819a0 0x5564d44337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.842+0000 7f637b4e6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 0x5564d4433ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5564d442ffc0 con 0x5564d43819a0 2024-05-27T07:45:07.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5564d442f8f0 con 0x5564d4295df0 2024-05-27T07:45:07.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f637b4e6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5564d4430690 con 0x5564d441f270 2024-05-27T07:45:07.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f637b4e6b80 10 client.0 _ll_register_callbacks cb 0x5564d42dcc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:07.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6365400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d43819a0 0x5564d44337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6365400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d43819a0 0x5564d44337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59610/0 (socket says 172.21.15.90:59610) 2024-05-27T07:45:07.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6365400700 1 -- 172.21.15.90:0/3197215479 learned_addr learned my addr 172.21.15.90:0/3197215479 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6365e00700 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d4295df0 0x5564d4433260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6366800700 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 0x5564d4433ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6365400700 1 -- 172.21.15.90:0/3197215479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 msgr2=0x5564d4433ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6365400700 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 0x5564d4433ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6365400700 1 -- 172.21.15.90:0/3197215479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d4295df0 msgr2=0x5564d4433260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6365400700 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d4295df0 0x5564d4433260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6365400700 1 -- 172.21.15.90:0/3197215479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d4437a20 con 0x5564d43819a0 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.843+0000 7f6366800700 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 0x5564d4433ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.844+0000 7f6365400700 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d43819a0 0x5564d44337a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6354009990 tx=0x7f6354009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5dfb4a36c1d9fb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.844+0000 7f6347e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.844+0000 7f6347e00700 1 -- 172.21.15.90:0/3197215479 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354026040 con 0x5564d43819a0 2024-05-27T07:45:07.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.844+0000 7f6347e00700 1 -- 172.21.15.90:0/3197215479 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f6354004d50 con 0x5564d43819a0 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.844+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3197215479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6358026840 con 0x5564d43819a0 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.844+0000 7f6347e00700 1 -- 172.21.15.90:0/3197215479 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6354005140 con 0x5564d43819a0 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.844+0000 7f637b4e6b80 10 client.15051 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.844+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3197215479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5564d449a670 con 0x5564d43819a0 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.845+0000 7f6347e00700 1 -- 172.21.15.90:0/3197215479 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6358026840 con 0x5564d43819a0 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.845+0000 7f6347e00700 1 -- 172.21.15.90:0/3197215479 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f635400f460 con 0x5564d43819a0 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.845+0000 7f637b4e6b80 10 client.15051 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.845+0000 7f637b4e6b80 10 client.15051 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.845+0000 7f637b4e6b80 20 client.15051 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.845+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564d449c550 0x5564d449d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.845+0000 7f637b4e6b80 4 client.15051 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.845+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3197215479 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "status"}) -- 0x5564d433e520 con 0x5564d449c550 2024-05-27T07:45:07.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.846+0000 7f6365e00700 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564d449c550 0x5564d449d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.847+0000 7f6365e00700 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564d449c550 0x5564d449d9c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c6a0696d8078839 server_cookie=f671d1b2fb3905ab in_seq=0 out_seq=0 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.847+0000 7f6347e00700 10 client.15051 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.847+0000 7f6347e00700 1 -- 172.21.15.90:0/3197215479 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5564d433e520 con 0x5564d449c550 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.847+0000 7f6347e00700 10 client.15051 handle_command_reply: tid=0 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.847+0000 7f637b4e6b80 1 client.15051 shutdown 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3197215479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d43819a0 msgr2=0x5564d44337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d43819a0 0x5564d44337a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6354009990 tx=0x7f6354009290 comp rx=0 tx=0).stop 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3197215479 shutdown_connections 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564d449c550 0x5564d449d9c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564d441f270 0x5564d4433ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564d4295df0 0x5564d4433260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 1 --2- 172.21.15.90:0/3197215479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564d43819a0 0x5564d44337a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3197215479 >> 172.21.15.90:0/3197215479 conn(0x5564d432f340 msgr2=0x5564d44177b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:07.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f6347e00700 0 client.15051 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:45:07.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3197215479 shutdown_connections 2024-05-27T07:45:07.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 1 -- 172.21.15.90:0/3197215479 wait complete. 2024-05-27T07:45:07.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.848+0000 7f637b4e6b80 20 client.15051 trim_cache size 0 max 16384 2024-05-27T07:45:07.849 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:45:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:45:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-05-27T07:45:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34424, 2024-05-27T07:45:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:45:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:45:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:45:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 155.98892198600001, 2024-05-27T07:45:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-05-27T07:45:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 91, 2024-05-27T07:45:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:45:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 284.60957669999999 2024-05-27T07:45:07.850 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:45:07.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 -- 172.21.15.90:0/2481802369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f634c0b2880 msgr2=0x7f634c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:07.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f634c0b2880 0x7f634c0b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6358005950 tx=0x7f63580058e0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 -- 172.21.15.90:0/2481802369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368115030 msgr2=0x7f63681a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:07.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368115030 0x7f63681a59f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6360007d70 tx=0x7f636000a340 comp rx=0 tx=0).stop 2024-05-27T07:45:07.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 -- 172.21.15.90:0/2481802369 shutdown_connections 2024-05-27T07:45:07.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f634c0b2880 0x7f634c0b4d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6368115030 0x7f63681a59f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6368077ce0 0x7f63681a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 --2- 172.21.15.90:0/2481802369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6368078630 0x7f63681a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:07.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 -- 172.21.15.90:0/2481802369 >> 172.21.15.90:0/2481802369 conn(0x7f6368101ec0 msgr2=0x7f63681042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:07.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 -- 172.21.15.90:0/2481802369 shutdown_connections 2024-05-27T07:45:07.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:07.849+0000 7f6367200700 1 -- 172.21.15.90:0/2481802369 wait complete. 2024-05-27T07:45:08.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:07 smithi090 ceph-mon[31429]: pgmap v505: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:08.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:07 smithi136 ceph-mon[35324]: pgmap v505: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:08.373 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:07 smithi129 ceph-mon[35254]: pgmap v505: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:08.672 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T07:45:09.134 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:10.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:09 smithi090 ceph-mon[31429]: pgmap v506: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:10.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:09 smithi136 ceph-mon[35324]: pgmap v506: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:10.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:09 smithi129 ceph-mon[35254]: pgmap v506: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:11.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.384+0000 7ff880400700 1 -- 172.21.15.90:0/2363709526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878079740 msgr2=0x7ff878114cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:11.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.384+0000 7ff880400700 1 --2- 172.21.15.90:0/2363709526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878079740 0x7ff878114cd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff864009990 tx=0x7ff864009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.384+0000 7ff880400700 1 -- 172.21.15.90:0/2363709526 shutdown_connections 2024-05-27T07:45:11.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.384+0000 7ff880400700 1 --2- 172.21.15.90:0/2363709526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff878115210 0x7ff878117630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.384+0000 7ff880400700 1 --2- 172.21.15.90:0/2363709526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff878078df0 0x7ff878079200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.384+0000 7ff880400700 1 --2- 172.21.15.90:0/2363709526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878079740 0x7ff878114cd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.384+0000 7ff880400700 1 -- 172.21.15.90:0/2363709526 >> 172.21.15.90:0/2363709526 conn(0x7ff878102070 msgr2=0x7ff8781044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:11.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.384+0000 7ff880400700 1 -- 172.21.15.90:0/2363709526 shutdown_connections 2024-05-27T07:45:11.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.384+0000 7ff880400700 1 -- 172.21.15.90:0/2363709526 wait complete. 2024-05-27T07:45:11.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.385+0000 7ff880400700 1 Processor -- start 2024-05-27T07:45:11.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.386+0000 7ff880400700 1 -- start start 2024-05-27T07:45:11.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.386+0000 7ff880400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff878078df0 0x7ff8781a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.386+0000 7ff880400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff878079740 0x7ff8781a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.386+0000 7ff880400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878115210 0x7ff8781a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.386+0000 7ff880400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff87811aa00 con 0x7ff878115210 2024-05-27T07:45:11.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.386+0000 7ff880400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff87811a880 con 0x7ff878079740 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.386+0000 7ff880400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff87811ab80 con 0x7ff878078df0 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.386+0000 7ff87e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878115210 0x7ff8781a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878115210 0x7ff8781a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54636/0 (socket says 172.21.15.90:54636) 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87e400700 1 -- 172.21.15.90:0/1459133577 learned_addr learned my addr 172.21.15.90:0/1459133577 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87da00700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff878078df0 0x7ff8781a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87d000700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff878079740 0x7ff8781a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87e400700 1 -- 172.21.15.90:0/1459133577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff878078df0 msgr2=0x7ff8781a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87e400700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff878078df0 0x7ff8781a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87e400700 1 -- 172.21.15.90:0/1459133577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff878079740 msgr2=0x7ff8781a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87e400700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff878079740 0x7ff8781a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87e400700 1 -- 172.21.15.90:0/1459133577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8781a6230 con 0x7ff878115210 2024-05-27T07:45:11.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87da00700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff878078df0 0x7ff8781a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:11.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87d000700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff878079740 0x7ff8781a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:11.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.387+0000 7ff87e400700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878115210 0x7ff8781a5c40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff87000b970 tx=0x7ff87000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:11.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.388+0000 7ff876000700 1 -- 172.21.15.90:0/1459133577 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870019070 con 0x7ff878115210 2024-05-27T07:45:11.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.388+0000 7ff876000700 1 -- 172.21.15.90:0/1459133577 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff87000ce40 con 0x7ff878115210 2024-05-27T07:45:11.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.388+0000 7ff880400700 1 -- 172.21.15.90:0/1459133577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8781a6520 con 0x7ff878115210 2024-05-27T07:45:11.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.388+0000 7ff876000700 1 -- 172.21.15.90:0/1459133577 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870007370 con 0x7ff878115210 2024-05-27T07:45:11.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.388+0000 7ff880400700 1 -- 172.21.15.90:0/1459133577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8781a6a10 con 0x7ff878115210 2024-05-27T07:45:11.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.392+0000 7ff876000700 1 -- 172.21.15.90:0/1459133577 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff87001c070 con 0x7ff878115210 2024-05-27T07:45:11.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.393+0000 7ff88eea8b80 1 Processor -- start 2024-05-27T07:45:11.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.393+0000 7ff88eea8b80 1 -- start start 2024-05-27T07:45:11.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.393+0000 7ff876000700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8600b2880 0x7ff8600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.393+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349af3edf0 0x56349b0c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.393+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.393+0000 7ff876000700 1 -- 172.21.15.90:0/1459133577 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7ff8700e1bf0 con 0x7ff878115210 2024-05-27T07:45:11.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.393+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0bdd60 0x56349b0be170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.393+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56349afebdf0 con 0x56349b0c3ce0 2024-05-27T07:45:11.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.393+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56349afdb990 con 0x56349b0bdd60 2024-05-27T07:45:11.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.393+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56349afdf9d0 con 0x56349af3edf0 2024-05-27T07:45:11.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.394+0000 7ff87d000700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8600b2880 0x7ff8600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.394+0000 7ff87d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.394+0000 7ff87d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0bd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54652/0 (socket says 172.21.15.90:54652) 2024-05-27T07:45:11.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.394+0000 7ff87d000700 1 -- 172.21.15.90:0/3123599719 learned_addr learned my addr 172.21.15.90:0/3123599719 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:11.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.394+0000 7ff87da00700 1 --2- 172.21.15.90:0/3123599719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349af3edf0 0x56349b0c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.394+0000 7ff87e400700 1 --2- 172.21.15.90:0/3123599719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0bdd60 0x56349b0be170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.395+0000 7ff87d000700 1 -- 172.21.15.90:0/3123599719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349af3edf0 msgr2=0x56349b0c37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:11.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.395+0000 7ff87d000700 1 --2- 172.21.15.90:0/3123599719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349af3edf0 0x56349b0c37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.395+0000 7ff87d000700 1 -- 172.21.15.90:0/3123599719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0bdd60 msgr2=0x56349b0be170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:11.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.395+0000 7ff87d000700 1 --2- 172.21.15.90:0/3123599719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0bdd60 0x56349b0be170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.395+0000 7ff87d000700 1 -- 172.21.15.90:0/3123599719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349afa43d0 con 0x56349b0c3ce0 2024-05-27T07:45:11.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.395+0000 7ff87d000700 1 --2- 172.21.15.90:0/3123599719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0bd820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff864015000 tx=0x7ff86400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa164a0d325a2311 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:11.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.396+0000 7ff87d000700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8600b2880 0x7ff8600b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff86400bdc0 tx=0x7ff86401c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:11.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.396+0000 7ff86aa00700 1 -- 172.21.15.90:0/3123599719 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86400fd80 con 0x56349b0c3ce0 2024-05-27T07:45:11.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.396+0000 7ff86aa00700 1 -- 172.21.15.90:0/3123599719 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff864004cc0 con 0x56349b0c3ce0 2024-05-27T07:45:11.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.396+0000 7ff86aa00700 1 -- 172.21.15.90:0/3123599719 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864005500 con 0x56349b0c3ce0 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.397+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3123599719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 msgr2=0x56349b0bd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.397+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3123599719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0bd820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff864015000 tx=0x7ff86400bd50 comp rx=0 tx=0).stop 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.397+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3123599719 shutdown_connections 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.397+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3123599719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0bdd60 0x56349b0be170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.397+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3123599719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349af3edf0 0x56349b0c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.397+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3123599719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0bd820 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.397+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3123599719 >> 172.21.15.90:0/3123599719 conn(0x56349af9ef70 msgr2=0x56349b0b8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.397+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3123599719 shutdown_connections 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.397+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3123599719 wait complete. 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.398+0000 7ff88eea8b80 1 -- start start 2024-05-27T07:45:11.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.398+0000 7ff88eea8b80 1 Processor -- start 2024-05-27T07:45:11.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 0x56349b0c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0c3ce0 0x56349b0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56349aee1f50 con 0x56349af3edf0 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56349af7c710 con 0x56349b0c3ce0 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56349b0ca750 con 0x56349b0bdd60 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff88eea8b80 10 client.0 _ll_register_callbacks cb 0x56349af85c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff87da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff87da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54668/0 (socket says 172.21.15.90:54668) 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff87da00700 1 -- 172.21.15.90:0/3167015162 learned_addr learned my addr 172.21.15.90:0/3167015162 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.399+0000 7ff87d000700 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 0x56349b0c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff87e400700 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0c3ce0 0x56349b0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff87da00700 1 -- 172.21.15.90:0/3167015162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 msgr2=0x56349b0c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:11.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff87da00700 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 0x56349b0c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff87da00700 1 -- 172.21.15.90:0/3167015162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0c3ce0 msgr2=0x56349b0c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:11.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff87da00700 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0c3ce0 0x56349b0c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff87da00700 1 -- 172.21.15.90:0/3167015162 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349af2b880 con 0x56349af3edf0 2024-05-27T07:45:11.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff87e400700 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0c3ce0 0x56349b0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:11.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff87d000700 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 0x56349b0c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:11.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff87da00700 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0c7410 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff86c011a60 tx=0x7ff86c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721e1d7265c7d7e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:11.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff85f400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:11.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.400+0000 7ff85f400700 1 -- 172.21.15.90:0/3167015162 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c004460 con 0x56349af3edf0 2024-05-27T07:45:11.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.401+0000 7ff85f400700 1 -- 172.21.15.90:0/3167015162 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7ff86c00f8b0 con 0x56349af3edf0 2024-05-27T07:45:11.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.401+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3167015162 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff864004cc0 con 0x56349af3edf0 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.401+0000 7ff85f400700 1 -- 172.21.15.90:0/3167015162 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff86c00fab0 con 0x56349af3edf0 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.401+0000 7ff88eea8b80 10 client.15063 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.401+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3167015162 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56349afac010 con 0x56349af3edf0 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.401+0000 7ff85f400700 1 -- 172.21.15.90:0/3167015162 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff864004cc0 con 0x56349af3edf0 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.402+0000 7ff85f400700 1 -- 172.21.15.90:0/3167015162 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff86c018030 con 0x56349af3edf0 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.402+0000 7ff88eea8b80 10 client.15063 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.402+0000 7ff88eea8b80 10 client.15063 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.402+0000 7ff88eea8b80 20 client.15063 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.402+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56349b0d77a0 0x56349b0d8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.402+0000 7ff88eea8b80 4 client.15063 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:11.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.402+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3167015162 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56349afedd40 con 0x56349b0d77a0 2024-05-27T07:45:11.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.402+0000 7ff87d000700 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56349b0d77a0 0x56349b0d8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.403+0000 7ff87d000700 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56349b0d77a0 0x56349b0d8c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9007f17232db803c server_cookie=fe4f3c708ed922a9 in_seq=0 out_seq=0 2024-05-27T07:45:11.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.403+0000 7ff85f400700 10 client.15063 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:45:11.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff85f400700 1 -- 172.21.15.90:0/3167015162 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56349afedd40 con 0x56349b0d77a0 2024-05-27T07:45:11.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff85f400700 10 client.15063 handle_command_reply: tid=0 2024-05-27T07:45:11.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 client.15063 shutdown 2024-05-27T07:45:11.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3167015162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 msgr2=0x56349b0c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:11.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0c7410 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff86c011a60 tx=0x7ff86c011e20 comp rx=0 tx=0).stop 2024-05-27T07:45:11.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3167015162 shutdown_connections 2024-05-27T07:45:11.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56349b0d77a0 0x56349b0d8c10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0c3ce0 0x56349b0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0c7410 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3167015162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 0x56349b0c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff85f400700 0 client.15063 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:45:11.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3167015162 >> 172.21.15.90:0/3167015162 conn(0x56349af9ef70 msgr2=0x56349b0b8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:11.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3167015162 shutdown_connections 2024-05-27T07:45:11.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.406+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3167015162 wait complete. 2024-05-27T07:45:11.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.407+0000 7ff88eea8b80 20 client.15063 trim_cache size 0 max 16384 2024-05-27T07:45:11.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.429+0000 7ff88eea8b80 1 Processor -- start 2024-05-27T07:45:11.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.429+0000 7ff88eea8b80 1 -- start start 2024-05-27T07:45:11.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.430+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.430+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0bdd60 0x56349b0d3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.430+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0c3ce0 0x56349b0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.430+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56349b0d6f50 con 0x56349af3edf0 2024-05-27T07:45:11.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.430+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56349b0d7620 con 0x56349b0bdd60 2024-05-27T07:45:11.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.430+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56349b0d7cf0 con 0x56349b0c3ce0 2024-05-27T07:45:11.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.430+0000 7ff87da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.430+0000 7ff87da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0bd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54676/0 (socket says 172.21.15.90:54676) 2024-05-27T07:45:11.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.430+0000 7ff87da00700 1 -- 172.21.15.90:0/3891887815 learned_addr learned my addr 172.21.15.90:0/3891887815 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:11.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.431+0000 7ff87da00700 1 -- 172.21.15.90:0/3891887815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0c3ce0 msgr2=0x56349b0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:11.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.431+0000 7ff87da00700 1 --2- 172.21.15.90:0/3891887815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0c3ce0 0x56349b0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.431+0000 7ff87d000700 1 --2- 172.21.15.90:0/3891887815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0bdd60 0x56349b0d3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.431+0000 7ff87da00700 1 -- 172.21.15.90:0/3891887815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0bdd60 msgr2=0x56349b0d3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:11.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.431+0000 7ff87da00700 1 --2- 172.21.15.90:0/3891887815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0bdd60 0x56349b0d3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.431+0000 7ff87da00700 1 -- 172.21.15.90:0/3891887815 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349b0b94e0 con 0x56349af3edf0 2024-05-27T07:45:11.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.431+0000 7ff87da00700 1 --2- 172.21.15.90:0/3891887815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0bd820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff86c000c00 tx=0x7ff86c00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0bc5a6f58d60cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:11.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.431+0000 7ff86a000700 1 -- 172.21.15.90:0/3891887815 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c02a070 con 0x56349af3edf0 2024-05-27T07:45:11.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.431+0000 7ff86a000700 1 -- 172.21.15.90:0/3891887815 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff86c00fe50 con 0x56349af3edf0 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.431+0000 7ff86a000700 1 -- 172.21.15.90:0/3891887815 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c019810 con 0x56349af3edf0 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.432+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3891887815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 msgr2=0x56349b0bd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.432+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3891887815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0bd820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff86c000c00 tx=0x7ff86c00f6a0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.432+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3891887815 shutdown_connections 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.432+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3891887815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0c3ce0 0x56349b0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.432+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3891887815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349af3edf0 0x56349b0bd820 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.432+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/3891887815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349b0bdd60 0x56349b0d3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.432+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3891887815 >> 172.21.15.90:0/3891887815 conn(0x56349af9ef70 msgr2=0x56349b0c25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.432+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3891887815 shutdown_connections 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.432+0000 7ff88eea8b80 1 -- 172.21.15.90:0/3891887815 wait complete. 2024-05-27T07:45:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.433+0000 7ff88eea8b80 1 -- start start 2024-05-27T07:45:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.433+0000 7ff88eea8b80 1 Processor -- start 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349af3edf0 0x56349b0e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 0x56349b0e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff88eea8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0dd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56349b0d8a90 con 0x56349b0c3ce0 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56349b0d83c0 con 0x56349af3edf0 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff88eea8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56349b0d9160 con 0x56349b0bdd60 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff88eea8b80 10 client.0 _ll_register_callbacks cb 0x56349af85c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff87e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0dd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff87e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0dd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54680/0 (socket says 172.21.15.90:54680) 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff87e400700 1 -- 172.21.15.90:0/2862026638 learned_addr learned my addr 172.21.15.90:0/2862026638 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff87da00700 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349af3edf0 0x56349b0e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff87d000700 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 0x56349b0e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff87e400700 1 -- 172.21.15.90:0/2862026638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 msgr2=0x56349b0e25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff87e400700 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 0x56349b0e25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff87e400700 1 -- 172.21.15.90:0/2862026638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349af3edf0 msgr2=0x56349b0e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff87e400700 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349af3edf0 0x56349b0e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.434+0000 7ff87e400700 1 -- 172.21.15.90:0/2862026638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349b0c8200 con 0x56349b0c3ce0 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.435+0000 7ff87d000700 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 0x56349b0e25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.435+0000 7ff87e400700 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0dd040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff8700df300 tx=0x7ff870014890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effe387f76695baa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.435+0000 7ff85f400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.435+0000 7ff85f400700 1 -- 172.21.15.90:0/2862026638 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8700a8eb0 con 0x56349b0c3ce0 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.435+0000 7ff85f400700 1 -- 172.21.15.90:0/2862026638 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7ff870014440 con 0x56349b0c3ce0 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.435+0000 7ff88eea8b80 1 -- 172.21.15.90:0/2862026638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff86c00fe50 con 0x56349b0c3ce0 2024-05-27T07:45:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.435+0000 7ff85f400700 1 -- 172.21.15.90:0/2862026638 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff8700ad260 con 0x56349b0c3ce0 2024-05-27T07:45:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.435+0000 7ff88eea8b80 10 client.15069 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.435+0000 7ff88eea8b80 1 -- 172.21.15.90:0/2862026638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56349b0ba170 con 0x56349b0c3ce0 2024-05-27T07:45:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.436+0000 7ff85f400700 1 -- 172.21.15.90:0/2862026638 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff86c00fe50 con 0x56349b0c3ce0 2024-05-27T07:45:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.436+0000 7ff85f400700 1 -- 172.21.15.90:0/2862026638 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff8700a4940 con 0x56349b0c3ce0 2024-05-27T07:45:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.436+0000 7ff88eea8b80 10 client.15069 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.436+0000 7ff88eea8b80 10 client.15069 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T07:45:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.436+0000 7ff88eea8b80 20 client.15069 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.436+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56349b145460 0x56349b1468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.436+0000 7ff88eea8b80 4 client.15069 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:45:11.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.436+0000 7ff88eea8b80 1 -- 172.21.15.90:0/2862026638 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "status"}) -- 0x56349afe7520 con 0x56349b145460 2024-05-27T07:45:11.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.437+0000 7ff87da00700 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56349b145460 0x56349b1468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:11.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.438+0000 7ff87da00700 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56349b145460 0x56349b1468d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8adf40c762b12a29 server_cookie=f6a7b8779fe4482f in_seq=0 out_seq=0 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.438+0000 7ff85f400700 10 client.15069 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.438+0000 7ff85f400700 1 -- 172.21.15.90:0/2862026638 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56349afe7520 con 0x56349b145460 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff85f400700 10 client.15069 handle_command_reply: tid=0 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 client.15069 shutdown 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 -- 172.21.15.90:0/2862026638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 msgr2=0x56349b0dd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0dd040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff8700df300 tx=0x7ff870014890 comp rx=0 tx=0).stop 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 -- 172.21.15.90:0/2862026638 shutdown_connections 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56349b0c3ce0 0x56349b0dd040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56349b145460 0x56349b1468d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349af3edf0 0x56349b0e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 --2- 172.21.15.90:0/2862026638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56349b0bdd60 0x56349b0e25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 -- 172.21.15.90:0/2862026638 >> 172.21.15.90:0/2862026638 conn(0x56349af9ef70 msgr2=0x56349b0c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff85f400700 0 client.15069 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:45:11.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 -- 172.21.15.90:0/2862026638 shutdown_connections 2024-05-27T07:45:11.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 1 -- 172.21.15.90:0/2862026638 wait complete. 2024-05-27T07:45:11.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.439+0000 7ff88eea8b80 20 client.15069 trim_cache size 0 max 16384 2024-05-27T07:45:11.440 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:45:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:45:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-05-27T07:45:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34442, 2024-05-27T07:45:11.440 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:45:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:45:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:45:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 160.59386862100001, 2024-05-27T07:45:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-05-27T07:45:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 91, 2024-05-27T07:45:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:45:11.441 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 275.59482494999997 2024-05-27T07:45:11.441 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:45:11.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 -- 172.21.15.90:0/1459133577 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8600b2880 msgr2=0x7ff8600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:11.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8600b2880 0x7ff8600b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff86400bdc0 tx=0x7ff86401c000 comp rx=0 tx=0).stop 2024-05-27T07:45:11.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 -- 172.21.15.90:0/1459133577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878115210 msgr2=0x7ff8781a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:11.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878115210 0x7ff8781a5c40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff87000b970 tx=0x7ff87000bd30 comp rx=0 tx=0).stop 2024-05-27T07:45:11.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 -- 172.21.15.90:0/1459133577 shutdown_connections 2024-05-27T07:45:11.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7ff8600b2880 0x7ff8600b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff878115210 0x7ff8781a5c40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff878078df0 0x7ff8781a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 --2- 172.21.15.90:0/1459133577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff878079740 0x7ff8781a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:11.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 -- 172.21.15.90:0/1459133577 >> 172.21.15.90:0/1459133577 conn(0x7ff878102070 msgr2=0x7ff878104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:11.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 -- 172.21.15.90:0/1459133577 shutdown_connections 2024-05-27T07:45:11.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:11.440+0000 7ff880400700 1 -- 172.21.15.90:0/1459133577 wait complete. 2024-05-27T07:45:12.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:11 smithi090 ceph-mon[31429]: pgmap v507: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:12.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:11 smithi136 ceph-mon[35324]: pgmap v507: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:12.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:11 smithi129 ceph-mon[35254]: pgmap v507: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:12.380 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T07:45:12.821 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:14.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:13 smithi090 ceph-mon[31429]: pgmap v508: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:14.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:13 smithi136 ceph-mon[35324]: pgmap v508: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:14.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:13 smithi129 ceph-mon[35254]: pgmap v508: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:14.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.960+0000 7fcd78800700 1 -- 172.21.15.90:0/332162372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd70077780 msgr2=0x7fcd70110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:14.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.960+0000 7fcd78800700 1 --2- 172.21.15.90:0/332162372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd70077780 0x7fcd70110630 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009990 tx=0x7fcd60009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.960+0000 7fcd78800700 1 -- 172.21.15.90:0/332162372 shutdown_connections 2024-05-27T07:45:14.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.960+0000 7fcd78800700 1 --2- 172.21.15.90:0/332162372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd70110b70 0x7fcd70112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.961+0000 7fcd78800700 1 --2- 172.21.15.90:0/332162372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd70076e30 0x7fcd70077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.961+0000 7fcd78800700 1 --2- 172.21.15.90:0/332162372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd70077780 0x7fcd70110630 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.961+0000 7fcd78800700 1 -- 172.21.15.90:0/332162372 >> 172.21.15.90:0/332162372 conn(0x7fcd700ffe90 msgr2=0x7fcd701022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:14.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.961+0000 7fcd78800700 1 -- 172.21.15.90:0/332162372 shutdown_connections 2024-05-27T07:45:14.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.961+0000 7fcd78800700 1 -- 172.21.15.90:0/332162372 wait complete. 2024-05-27T07:45:14.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.962+0000 7fcd78800700 1 Processor -- start 2024-05-27T07:45:14.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.962+0000 7fcd78800700 1 -- start start 2024-05-27T07:45:14.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.963+0000 7fcd78800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd70076e30 0x7fcd7019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.963+0000 7fcd78800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd70077780 0x7fcd7019d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.963+0000 7fcd78800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd70110b70 0x7fcd701a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.963+0000 7fcd78800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcd70116520 con 0x7fcd70077780 2024-05-27T07:45:14.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.963+0000 7fcd78800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701163a0 con 0x7fcd70076e30 2024-05-27T07:45:14.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.963+0000 7fcd78800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcd701166a0 con 0x7fcd70110b70 2024-05-27T07:45:14.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.963+0000 7fcd75400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd70077780 0x7fcd7019d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.963+0000 7fcd75400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd70077780 0x7fcd7019d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54696/0 (socket says 172.21.15.90:54696) 2024-05-27T07:45:14.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.963+0000 7fcd75400700 1 -- 172.21.15.90:0/2777248902 learned_addr learned my addr 172.21.15.90:0/2777248902 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:14.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.964+0000 7fcd75e00700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd70076e30 0x7fcd7019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.964+0000 7fcd76800700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd70110b70 0x7fcd701a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.964+0000 7fcd75400700 1 -- 172.21.15.90:0/2777248902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd70110b70 msgr2=0x7fcd701a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:14.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.964+0000 7fcd75400700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd70110b70 0x7fcd701a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.964+0000 7fcd75400700 1 -- 172.21.15.90:0/2777248902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd70076e30 msgr2=0x7fcd7019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:14.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.964+0000 7fcd75400700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd70076e30 0x7fcd7019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.964+0000 7fcd75400700 1 -- 172.21.15.90:0/2777248902 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd701a1d70 con 0x7fcd70077780 2024-05-27T07:45:14.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.964+0000 7fcd75e00700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd70076e30 0x7fcd7019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:14.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.964+0000 7fcd75400700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd70077780 0x7fcd7019d4f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd600057c0 tx=0x7fcd60009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:14.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.965+0000 7fcd6aa00700 1 -- 172.21.15.90:0/2777248902 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60027070 con 0x7fcd70077780 2024-05-27T07:45:14.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.965+0000 7fcd6aa00700 1 -- 172.21.15.90:0/2777248902 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd6000fda0 con 0x7fcd70077780 2024-05-27T07:45:14.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.965+0000 7fcd6aa00700 1 -- 172.21.15.90:0/2777248902 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60022c70 con 0x7fcd70077780 2024-05-27T07:45:14.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.965+0000 7fcd78800700 1 -- 172.21.15.90:0/2777248902 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd701a2000 con 0x7fcd70077780 2024-05-27T07:45:14.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.965+0000 7fcd78800700 1 -- 172.21.15.90:0/2777248902 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd701a2530 con 0x7fcd70077780 2024-05-27T07:45:14.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.969+0000 7fcd6aa00700 1 -- 172.21.15.90:0/2777248902 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcd6002a070 con 0x7fcd70077780 2024-05-27T07:45:14.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.970+0000 7fcd873d2b80 1 Processor -- start 2024-05-27T07:45:14.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd873d2b80 1 -- start start 2024-05-27T07:45:14.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd6aa00700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd580b2900 0x7fcd580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd6aa00700 1 -- 172.21.15.90:0/2777248902 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fcd600e3540 con 0x7fcd70077780 2024-05-27T07:45:14.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 0x560144c312e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c31820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c3e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560144b5fdf0 con 0x560144ab2df0 2024-05-27T07:45:14.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560144b4f990 con 0x560144c31e20 2024-05-27T07:45:14.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560144b539d0 con 0x560144b9e9a0 2024-05-27T07:45:14.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd76800700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd580b2900 0x7fcd580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd75400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c31820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd75400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c31820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54712/0 (socket says 172.21.15.90:54712) 2024-05-27T07:45:14.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.971+0000 7fcd75400700 1 -- 172.21.15.90:0/427032149 learned_addr learned my addr 172.21.15.90:0/427032149 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:14.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.972+0000 7fcd75e00700 1 --2- 172.21.15.90:0/427032149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 0x560144c312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.972+0000 7fcd76800700 1 --2- 172.21.15.90:0/427032149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c3e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.972+0000 7fcd75400700 1 -- 172.21.15.90:0/427032149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 msgr2=0x560144c312e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.972+0000 7fcd75400700 1 --2- 172.21.15.90:0/427032149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 0x560144c312e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.972+0000 7fcd75400700 1 -- 172.21.15.90:0/427032149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 msgr2=0x560144c3e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.972+0000 7fcd75400700 1 --2- 172.21.15.90:0/427032149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c3e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.972+0000 7fcd75400700 1 -- 172.21.15.90:0/427032149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560144b183d0 con 0x560144ab2df0 2024-05-27T07:45:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.972+0000 7fcd75e00700 1 --2- 172.21.15.90:0/427032149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 0x560144c312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.972+0000 7fcd76800700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd580b2900 0x7fcd580b4e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcd70078cd0 tx=0x7fcd6c00e4f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.972+0000 7fcd75400700 1 --2- 172.21.15.90:0/427032149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c31820 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd60005f50 tx=0x7fcd600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410cb5144e57e3c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.973+0000 7fcd5f400700 1 -- 172.21.15.90:0/427032149 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd600aebb0 con 0x560144ab2df0 2024-05-27T07:45:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.973+0000 7fcd5f400700 1 -- 172.21.15.90:0/427032149 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd600b35d0 con 0x560144ab2df0 2024-05-27T07:45:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.973+0000 7fcd5f400700 1 -- 172.21.15.90:0/427032149 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd600aaa60 con 0x560144ab2df0 2024-05-27T07:45:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.974+0000 7fcd873d2b80 1 -- 172.21.15.90:0/427032149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 msgr2=0x560144c31820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.974+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/427032149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c31820 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd60005f50 tx=0x7fcd600057f0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.974+0000 7fcd873d2b80 1 -- 172.21.15.90:0/427032149 shutdown_connections 2024-05-27T07:45:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.974+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/427032149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c3e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.974+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/427032149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 0x560144c312e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.974+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/427032149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c31820 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.974+0000 7fcd873d2b80 1 -- 172.21.15.90:0/427032149 >> 172.21.15.90:0/427032149 conn(0x560144b4c340 msgr2=0x560144c2d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.974+0000 7fcd873d2b80 1 -- 172.21.15.90:0/427032149 shutdown_connections 2024-05-27T07:45:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.974+0000 7fcd873d2b80 1 -- 172.21.15.90:0/427032149 wait complete. 2024-05-27T07:45:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.975+0000 7fcd873d2b80 1 -- start start 2024-05-27T07:45:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.975+0000 7fcd873d2b80 1 Processor -- start 2024-05-27T07:45:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c3b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 0x560144c3bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c3c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560144a55f50 con 0x560144ab2df0 2024-05-27T07:45:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560144af0710 con 0x560144c31e20 2024-05-27T07:45:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560144c3e8f0 con 0x560144b9e9a0 2024-05-27T07:45:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd873d2b80 10 client.0 _ll_register_callbacks cb 0x560144af9c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd75e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c3b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd75e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c3b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54714/0 (socket says 172.21.15.90:54714) 2024-05-27T07:45:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd75e00700 1 -- 172.21.15.90:0/1666757802 learned_addr learned my addr 172.21.15.90:0/1666757802 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd75400700 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 0x560144c3bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd76800700 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c3c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd75e00700 1 -- 172.21.15.90:0/1666757802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 msgr2=0x560144c3bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd75e00700 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 0x560144c3bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd75e00700 1 -- 172.21.15.90:0/1666757802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 msgr2=0x560144c3c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd75e00700 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c3c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.976+0000 7fcd75e00700 1 -- 172.21.15.90:0/1666757802 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560144a9f880 con 0x560144ab2df0 2024-05-27T07:45:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.977+0000 7fcd76800700 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c3c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.977+0000 7fcd75e00700 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c3b650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcd64011490 tx=0x7fcd6400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e019023d938739d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.977+0000 7fcd53e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.977+0000 7fcd53e00700 1 -- 172.21.15.90:0/1666757802 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64012860 con 0x560144ab2df0 2024-05-27T07:45:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.977+0000 7fcd53e00700 1 -- 172.21.15.90:0/1666757802 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fcd64012a00 con 0x560144ab2df0 2024-05-27T07:45:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.977+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1666757802 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd600b35d0 con 0x560144ab2df0 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd53e00700 1 -- 172.21.15.90:0/1666757802 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd64012c00 con 0x560144ab2df0 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd873d2b80 10 client.15081 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1666757802 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560144b20010 con 0x560144ab2df0 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd53e00700 1 -- 172.21.15.90:0/1666757802 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd600b35d0 con 0x560144ab2df0 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd53e00700 1 -- 172.21.15.90:0/1666757802 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd6400f3d0 con 0x560144ab2df0 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd873d2b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd873d2b80 10 client.15081 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd873d2b80 20 client.15081 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560144c4b920 0x560144c4cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd873d2b80 4 client.15081 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.978+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1666757802 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560144b61d40 con 0x560144c4b920 2024-05-27T07:45:14.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.979+0000 7fcd75400700 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560144c4b920 0x560144c4cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:14.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.980+0000 7fcd75400700 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560144c4b920 0x560144c4cd90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52c080bc6dbf308 server_cookie=e32e0be2bb681223 in_seq=0 out_seq=0 2024-05-27T07:45:14.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.980+0000 7fcd53e00700 10 client.15081 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:45:14.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.982+0000 7fcd53e00700 1 -- 172.21.15.90:0/1666757802 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560144b61d40 con 0x560144c4b920 2024-05-27T07:45:14.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.982+0000 7fcd53e00700 10 client.15081 handle_command_reply: tid=0 2024-05-27T07:45:14.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.982+0000 7fcd873d2b80 1 client.15081 shutdown 2024-05-27T07:45:14.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1666757802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 msgr2=0x560144c3b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:14.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c3b650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcd64011490 tx=0x7fcd6400bf90 comp rx=0 tx=0).stop 2024-05-27T07:45:14.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1666757802 shutdown_connections 2024-05-27T07:45:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560144c4b920 0x560144c4cd90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c3c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144ab2df0 0x560144c3b650 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1666757802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144b9e9a0 0x560144c3bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1666757802 >> 172.21.15.90:0/1666757802 conn(0x560144b4c340 msgr2=0x560144c312c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd53e00700 0 client.15081 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:45:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1666757802 shutdown_connections 2024-05-27T07:45:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1666757802 wait complete. 2024-05-27T07:45:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:14.983+0000 7fcd873d2b80 20 client.15081 trim_cache size 0 max 16384 2024-05-27T07:45:15.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.006+0000 7fcd873d2b80 1 Processor -- start 2024-05-27T07:45:15.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.006+0000 7fcd873d2b80 1 -- start start 2024-05-27T07:45:15.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.006+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144ab2df0 0x560144c4b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:15.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.006+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c2d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:15.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.006+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c36e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:15.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.006+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560144c4b0d0 con 0x560144b9e9a0 2024-05-27T07:45:15.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.006+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560144c4b7a0 con 0x560144c31e20 2024-05-27T07:45:15.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.006+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560144c4be70 con 0x560144ab2df0 2024-05-27T07:45:15.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd75400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c2d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:15.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd75400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c2d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54720/0 (socket says 172.21.15.90:54720) 2024-05-27T07:45:15.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd75400700 1 -- 172.21.15.90:0/527615432 learned_addr learned my addr 172.21.15.90:0/527615432 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:15.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd76800700 1 --2- 172.21.15.90:0/527615432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c36e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd75e00700 1 --2- 172.21.15.90:0/527615432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144ab2df0 0x560144c4b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd75400700 1 -- 172.21.15.90:0/527615432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144ab2df0 msgr2=0x560144c4b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd75400700 1 --2- 172.21.15.90:0/527615432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144ab2df0 0x560144c4b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd75400700 1 -- 172.21.15.90:0/527615432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 msgr2=0x560144c36e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd75400700 1 --2- 172.21.15.90:0/527615432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c36e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd75400700 1 -- 172.21.15.90:0/527615432 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560144c3a970 con 0x560144b9e9a0 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.007+0000 7fcd76800700 1 --2- 172.21.15.90:0/527615432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c36e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.008+0000 7fcd75400700 1 --2- 172.21.15.90:0/527615432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c2d610 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcd600a7a70 tx=0x7fcd60009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d8aecd2617c525 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.008+0000 7fcd5ea00700 1 -- 172.21.15.90:0/527615432 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd600b35d0 con 0x560144b9e9a0 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.008+0000 7fcd5ea00700 1 -- 172.21.15.90:0/527615432 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd600094b0 con 0x560144b9e9a0 2024-05-27T07:45:15.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.008+0000 7fcd5ea00700 1 -- 172.21.15.90:0/527615432 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd600b3140 con 0x560144b9e9a0 2024-05-27T07:45:15.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.009+0000 7fcd873d2b80 1 -- 172.21.15.90:0/527615432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 msgr2=0x560144c2d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:15.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.009+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/527615432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c2d610 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcd600a7a70 tx=0x7fcd60009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.009+0000 7fcd873d2b80 1 -- 172.21.15.90:0/527615432 shutdown_connections 2024-05-27T07:45:15.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.009+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/527615432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144c31e20 0x560144c36e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.009+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/527615432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144ab2df0 0x560144c4b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.009+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/527615432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c2d610 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.009+0000 7fcd873d2b80 1 -- 172.21.15.90:0/527615432 >> 172.21.15.90:0/527615432 conn(0x560144b4c340 msgr2=0x560144c2cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:15.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.009+0000 7fcd873d2b80 1 -- 172.21.15.90:0/527615432 shutdown_connections 2024-05-27T07:45:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.009+0000 7fcd873d2b80 1 -- 172.21.15.90:0/527615432 wait complete. 2024-05-27T07:45:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.010+0000 7fcd873d2b80 1 -- start start 2024-05-27T07:45:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.010+0000 7fcd873d2b80 1 Processor -- start 2024-05-27T07:45:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.010+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144ab2df0 0x560144c4f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c4fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd873d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144c31e20 0x560144cb50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560144c4cc10 con 0x560144b9e9a0 2024-05-27T07:45:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560144c4c540 con 0x560144ab2df0 2024-05-27T07:45:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd873d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560144c4d2e0 con 0x560144c31e20 2024-05-27T07:45:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd873d2b80 10 client.0 _ll_register_callbacks cb 0x560144af9c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd75400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c4fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd75400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c4fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54730/0 (socket says 172.21.15.90:54730) 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd75400700 1 -- 172.21.15.90:0/1960560529 learned_addr learned my addr 172.21.15.90:0/1960560529 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd75e00700 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144ab2df0 0x560144c4f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd76800700 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144c31e20 0x560144cb50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd75400700 1 -- 172.21.15.90:0/1960560529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144c31e20 msgr2=0x560144cb50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd75400700 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144c31e20 0x560144cb50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd75400700 1 -- 172.21.15.90:0/1960560529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144ab2df0 msgr2=0x560144c4f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd75400700 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144ab2df0 0x560144c4f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd75400700 1 -- 172.21.15.90:0/1960560529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560144cb55e0 con 0x560144b9e9a0 2024-05-27T07:45:15.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.011+0000 7fcd76800700 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144c31e20 0x560144cb50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:15.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.012+0000 7fcd75400700 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c4fcf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcd600ad040 tx=0x7fcd600aa980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeddf1121e86a5c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:15.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.012+0000 7fcd53e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:15.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.012+0000 7fcd53e00700 1 -- 172.21.15.90:0/1960560529 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd600b10e0 con 0x560144b9e9a0 2024-05-27T07:45:15.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.012+0000 7fcd53e00700 1 -- 172.21.15.90:0/1960560529 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fcd600b3140 con 0x560144b9e9a0 2024-05-27T07:45:15.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.012+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1960560529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd600094b0 con 0x560144b9e9a0 2024-05-27T07:45:15.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.012+0000 7fcd53e00700 1 -- 172.21.15.90:0/1960560529 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcd600b3340 con 0x560144b9e9a0 2024-05-27T07:45:15.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.012+0000 7fcd873d2b80 10 client.15087 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:15.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.012+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1960560529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560144cb6840 con 0x560144b9e9a0 2024-05-27T07:45:15.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.012+0000 7fcd53e00700 1 -- 172.21.15.90:0/1960560529 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd600094b0 con 0x560144b9e9a0 2024-05-27T07:45:15.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.013+0000 7fcd53e00700 1 -- 172.21.15.90:0/1960560529 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcd600abb30 con 0x560144b9e9a0 2024-05-27T07:45:15.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.013+0000 7fcd873d2b80 10 client.15087 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:15.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.013+0000 7fcd873d2b80 10 client.15087 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:45:15.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.013+0000 7fcd873d2b80 20 client.15087 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:15.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.013+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560144cb8870 0x560144cb9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:15.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.013+0000 7fcd873d2b80 4 client.15087 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:45:15.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.013+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1960560529 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "status"}) -- 0x560144b5b520 con 0x560144cb8870 2024-05-27T07:45:15.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.013+0000 7fcd75e00700 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560144cb8870 0x560144cb9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:15.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.014+0000 7fcd75e00700 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560144cb8870 0x560144cb9ce0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b72d95af486ca7 server_cookie=b0887f2de08c0f91 in_seq=0 out_seq=0 2024-05-27T07:45:15.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.014+0000 7fcd53e00700 10 client.15087 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:45:15.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd53e00700 1 -- 172.21.15.90:0/1960560529 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560144b5b520 con 0x560144cb8870 2024-05-27T07:45:15.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd53e00700 10 client.15087 handle_command_reply: tid=0 2024-05-27T07:45:15.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd873d2b80 1 client.15087 shutdown 2024-05-27T07:45:15.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1960560529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 msgr2=0x560144c4fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:15.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c4fcf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcd600ad040 tx=0x7fcd600aa980 comp rx=0 tx=0).stop 2024-05-27T07:45:15.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1960560529 shutdown_connections 2024-05-27T07:45:15.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560144cb8870 0x560144cb9ce0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560144c31e20 0x560144cb50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560144ab2df0 0x560144c4f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd873d2b80 1 --2- 172.21.15.90:0/1960560529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560144b9e9a0 0x560144c4fcf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd53e00700 0 client.15087 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:45:15.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.015+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1960560529 >> 172.21.15.90:0/1960560529 conn(0x560144b4c340 msgr2=0x560144c31060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:15.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.016+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1960560529 shutdown_connections 2024-05-27T07:45:15.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.016+0000 7fcd873d2b80 1 -- 172.21.15.90:0/1960560529 wait complete. 2024-05-27T07:45:15.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.016+0000 7fcd873d2b80 20 client.15087 trim_cache size 0 max 16384 2024-05-27T07:45:15.020 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:45:15.020 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:45:15.020 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-05-27T07:45:15.020 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34472, 2024-05-27T07:45:15.020 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:45:15.020 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:45:15.020 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:45:15.020 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 243.165323349, 2024-05-27T07:45:15.020 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-05-27T07:45:15.020 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 91, 2024-05-27T07:45:15.021 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:45:15.021 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 261.28832293099998 2024-05-27T07:45:15.021 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:45:15.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.016+0000 7fcd78800700 1 -- 172.21.15.90:0/2777248902 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd580b2900 msgr2=0x7fcd580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:15.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.016+0000 7fcd78800700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd580b2900 0x7fcd580b4e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcd70078cd0 tx=0x7fcd6c00e4f0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.016+0000 7fcd78800700 1 -- 172.21.15.90:0/2777248902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd70077780 msgr2=0x7fcd7019d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:15.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.016+0000 7fcd78800700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd70077780 0x7fcd7019d4f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd600057c0 tx=0x7fcd60009f90 comp rx=0 tx=0).stop 2024-05-27T07:45:15.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.016+0000 7fcd78800700 1 -- 172.21.15.90:0/2777248902 shutdown_connections 2024-05-27T07:45:15.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.017+0000 7fcd78800700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd580b2900 0x7fcd580b4e10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.017+0000 7fcd78800700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd70110b70 0x7fcd701a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.017+0000 7fcd78800700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd70076e30 0x7fcd7019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.017+0000 7fcd78800700 1 --2- 172.21.15.90:0/2777248902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd70077780 0x7fcd7019d4f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:15.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.017+0000 7fcd78800700 1 -- 172.21.15.90:0/2777248902 >> 172.21.15.90:0/2777248902 conn(0x7fcd700ffe90 msgr2=0x7fcd70100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:15.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.017+0000 7fcd78800700 1 -- 172.21.15.90:0/2777248902 shutdown_connections 2024-05-27T07:45:15.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:15.017+0000 7fcd78800700 1 -- 172.21.15.90:0/2777248902 wait complete. 2024-05-27T07:45:15.976 INFO:tasks.cephadm:Setup complete, yielding 2024-05-27T07:45:15.977 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T07:45:15.986 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T07:45:15.986 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-05-27T07:45:16.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:15 smithi090 ceph-mon[31429]: pgmap v509: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:16.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:15 smithi136 ceph-mon[35324]: pgmap v509: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:16.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:15 smithi129 ceph-mon[35254]: pgmap v509: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:18.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:17 smithi090 ceph-mon[31429]: pgmap v510: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:18.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:17 smithi136 ceph-mon[35324]: pgmap v510: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:18.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:17 smithi129 ceph-mon[35254]: pgmap v510: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:18.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.901+0000 7f4e95400700 1 -- 172.21.15.90:0/372298837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e9010c000 msgr2=0x7f4e9010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:18.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.901+0000 7f4e95400700 1 --2- 172.21.15.90:0/372298837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e9010c000 0x7f4e9010e510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009990 tx=0x7f4e88009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.901+0000 7f4e95400700 1 -- 172.21.15.90:0/372298837 shutdown_connections 2024-05-27T07:45:18.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.901+0000 7f4e95400700 1 --2- 172.21.15.90:0/372298837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e9010c000 0x7f4e9010e510 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.901+0000 7f4e95400700 1 --2- 172.21.15.90:0/372298837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e90077670 0x7f4e90075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.901+0000 7f4e95400700 1 --2- 172.21.15.90:0/372298837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e90076080 0x7f4e900764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.901+0000 7f4e95400700 1 -- 172.21.15.90:0/372298837 >> 172.21.15.90:0/372298837 conn(0x7f4e900ff7a0 msgr2=0x7f4e90101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:18.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.902+0000 7f4e95400700 1 -- 172.21.15.90:0/372298837 shutdown_connections 2024-05-27T07:45:18.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.902+0000 7f4e95400700 1 -- 172.21.15.90:0/372298837 wait complete. 2024-05-27T07:45:18.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.903+0000 7f4e95400700 1 Processor -- start 2024-05-27T07:45:18.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.903+0000 7f4e95400700 1 -- start start 2024-05-27T07:45:18.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.904+0000 7f4e95400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e90076080 0x7f4e9019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.904+0000 7f4e95400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e90077670 0x7f4e9019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.904+0000 7f4e95400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e9010c000 0x7f4e901a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.904+0000 7f4e95400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90111c50 con 0x7f4e90077670 2024-05-27T07:45:18.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.904+0000 7f4e95400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90111ad0 con 0x7f4e9010c000 2024-05-27T07:45:18.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.904+0000 7f4e95400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90111dd0 con 0x7f4e90076080 2024-05-27T07:45:18.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.904+0000 7f4e8f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e90077670 0x7f4e9019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.904+0000 7f4e8f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e90077670 0x7f4e9019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49256/0 (socket says 172.21.15.90:49256) 2024-05-27T07:45:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.904+0000 7f4e8f400700 1 -- 172.21.15.90:0/227089197 learned_addr learned my addr 172.21.15.90:0/227089197 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.904+0000 7f4e8fe00700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e90076080 0x7f4e9019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.905+0000 7f4e94a00700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e9010c000 0x7f4e901a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.905+0000 7f4e8f400700 1 -- 172.21.15.90:0/227089197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e90076080 msgr2=0x7f4e9019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.905+0000 7f4e8f400700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e90076080 0x7f4e9019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.905+0000 7f4e8f400700 1 -- 172.21.15.90:0/227089197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e9010c000 msgr2=0x7f4e901a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.905+0000 7f4e8f400700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e9010c000 0x7f4e901a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.905+0000 7f4e8f400700 1 -- 172.21.15.90:0/227089197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e901a4290 con 0x7f4e90077670 2024-05-27T07:45:18.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.905+0000 7f4e94a00700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e9010c000 0x7f4e901a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:18.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.905+0000 7f4e8f400700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e90077670 0x7f4e9019cd00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c920 tx=0x7f4e7c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:18.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.906+0000 7f4e8cc00700 1 -- 172.21.15.90:0/227089197 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c007b00 con 0x7f4e90077670 2024-05-27T07:45:18.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.906+0000 7f4e8cc00700 1 -- 172.21.15.90:0/227089197 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e7c004cc0 con 0x7f4e90077670 2024-05-27T07:45:18.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.906+0000 7f4e95400700 1 -- 172.21.15.90:0/227089197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e901a4580 con 0x7f4e90077670 2024-05-27T07:45:18.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.906+0000 7f4e95400700 1 -- 172.21.15.90:0/227089197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e901a4b10 con 0x7f4e90077670 2024-05-27T07:45:18.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.906+0000 7f4e8cc00700 1 -- 172.21.15.90:0/227089197 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c005440 con 0x7f4e90077670 2024-05-27T07:45:18.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.907+0000 7f4e95400700 1 -- 172.21.15.90:0/227089197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e70005360 con 0x7f4e90077670 2024-05-27T07:45:18.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.910+0000 7f4e8cc00700 1 -- 172.21.15.90:0/227089197 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4e7c004750 con 0x7f4e90077670 2024-05-27T07:45:18.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.911+0000 7f4e8cc00700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e740b2900 0x7f4e740b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:18.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.911+0000 7f4e8cc00700 1 -- 172.21.15.90:0/227089197 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f4e7c0e1fe0 con 0x7f4e90077670 2024-05-27T07:45:18.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.911+0000 7f4e8fe00700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e740b2900 0x7f4e740b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:18.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.912+0000 7f4e8fe00700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e740b2900 0x7f4e740b4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4e90077370 tx=0x7f4e800079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:18.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:18.916+0000 7f4e8cc00700 1 -- 172.21.15.90:0/227089197 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e7c014030 con 0x7f4e90077670 2024-05-27T07:45:19.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.168+0000 7f4e95400700 1 -- 172.21.15.90:0/227089197 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f4e70000bf0 con 0x7f4e740b2900 2024-05-27T07:45:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.170+0000 7f4e8cc00700 1 -- 172.21.15.90:0/227089197 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f4e70000bf0 con 0x7f4e740b2900 2024-05-27T07:45:19.169 INFO:teuthology.orchestra.run.smithi090.stdout:Backend: cephadm 2024-05-27T07:45:19.169 INFO:teuthology.orchestra.run.smithi090.stdout:Available: Yes 2024-05-27T07:45:19.169 INFO:teuthology.orchestra.run.smithi090.stdout:Paused: No 2024-05-27T07:45:19.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 -- 172.21.15.90:0/227089197 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e740b2900 msgr2=0x7f4e740b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:19.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e740b2900 0x7f4e740b4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4e90077370 tx=0x7f4e800079e0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 -- 172.21.15.90:0/227089197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e90077670 msgr2=0x7f4e9019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:19.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e90077670 0x7f4e9019cd00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c920 tx=0x7f4e7c00cc30 comp rx=0 tx=0).stop 2024-05-27T07:45:19.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 -- 172.21.15.90:0/227089197 shutdown_connections 2024-05-27T07:45:19.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f4e740b2900 0x7f4e740b4e10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e9010c000 0x7f4e901a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e90076080 0x7f4e9019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 --2- 172.21.15.90:0/227089197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e90077670 0x7f4e9019cd00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:19.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 -- 172.21.15.90:0/227089197 >> 172.21.15.90:0/227089197 conn(0x7f4e900ff7a0 msgr2=0x7f4e90107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:19.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 -- 172.21.15.90:0/227089197 shutdown_connections 2024-05-27T07:45:19.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:19.173+0000 7f4e95400700 1 -- 172.21.15.90:0/227089197 wait complete. 2024-05-27T07:45:20.097 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-27T07:45:20.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:19 smithi090 ceph-mon[31429]: pgmap v511: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:20.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:19 smithi136 ceph-mon[35324]: pgmap v511: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:20.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:19 smithi129 ceph-mon[35254]: pgmap v511: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:20.548 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:21.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:20 smithi090 ceph-mon[31429]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:21.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:20 smithi136 ceph-mon[35324]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:21.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:20 smithi129 ceph-mon[35254]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:22.196 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:21 smithi090 ceph-mon[31429]: pgmap v512: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:22.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:21 smithi136 ceph-mon[35324]: pgmap v512: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:22.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:21 smithi129 ceph-mon[35254]: pgmap v512: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.802+0000 7fdafe800700 1 -- 172.21.15.90:0/2225319523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdaf810c240 msgr2=0x7fdaf810e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.802+0000 7fdafe800700 1 --2- 172.21.15.90:0/2225319523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdaf810c240 0x7fdaf810e720 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b0a0 tx=0x7fdaf000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:45:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.802+0000 7fdafe800700 1 -- 172.21.15.90:0/2225319523 shutdown_connections 2024-05-27T07:45:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.802+0000 7fdafe800700 1 --2- 172.21.15.90:0/2225319523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdaf810c240 0x7fdaf810e720 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.802+0000 7fdafe800700 1 --2- 172.21.15.90:0/2225319523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaf8078760 0x7fdaf8076c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:22.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.802+0000 7fdafe800700 1 --2- 172.21.15.90:0/2225319523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaf8077170 0x7fdaf8077580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:22.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.802+0000 7fdafe800700 1 -- 172.21.15.90:0/2225319523 >> 172.21.15.90:0/2225319523 conn(0x7fdaf80ff9c0 msgr2=0x7fdaf8101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:22.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.802+0000 7fdafe800700 1 -- 172.21.15.90:0/2225319523 shutdown_connections 2024-05-27T07:45:22.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.802+0000 7fdafe800700 1 -- 172.21.15.90:0/2225319523 wait complete. 2024-05-27T07:45:22.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.803+0000 7fdafe800700 1 Processor -- start 2024-05-27T07:45:22.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.803+0000 7fdafe800700 1 -- start start 2024-05-27T07:45:22.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.803+0000 7fdafe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdaf8077170 0x7fdaf819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:22.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.803+0000 7fdafe800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaf8078760 0x7fdaf819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:22.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdafe800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaf810c240 0x7fdaf81a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:22.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdafe800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8111ec0 con 0x7fdaf8077170 2024-05-27T07:45:22.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdafe800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8111d40 con 0x7fdaf8078760 2024-05-27T07:45:22.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdafe800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8112040 con 0x7fdaf810c240 2024-05-27T07:45:22.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdaf7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdaf8077170 0x7fdaf819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:22.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdaf7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdaf8077170 0x7fdaf819c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49282/0 (socket says 172.21.15.90:49282) 2024-05-27T07:45:22.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdaf7400700 1 -- 172.21.15.90:0/363643803 learned_addr learned my addr 172.21.15.90:0/363643803 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:22.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdaefe00700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaf8078760 0x7fdaf819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:22.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdaf7400700 1 -- 172.21.15.90:0/363643803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaf810c240 msgr2=0x7fdaf81a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:22.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdaf7400700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaf810c240 0x7fdaf81a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:22.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdaf7400700 1 -- 172.21.15.90:0/363643803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaf8078760 msgr2=0x7fdaf819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:22.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdaf7400700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaf8078760 0x7fdaf819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:22.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.804+0000 7fdaf7400700 1 -- 172.21.15.90:0/363643803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf81a4540 con 0x7fdaf8077170 2024-05-27T07:45:22.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.805+0000 7fdaf7400700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdaf8077170 0x7fdaf819c9e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d980 tx=0x7fdae800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:22.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.805+0000 7fdaf4c00700 1 -- 172.21.15.90:0/363643803 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8017070 con 0x7fdaf8077170 2024-05-27T07:45:22.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.805+0000 7fdaf4c00700 1 -- 172.21.15.90:0/363643803 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdae8010460 con 0x7fdaf8077170 2024-05-27T07:45:22.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.805+0000 7fdaf4c00700 1 -- 172.21.15.90:0/363643803 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800de60 con 0x7fdaf8077170 2024-05-27T07:45:22.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.805+0000 7fdafe800700 1 -- 172.21.15.90:0/363643803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf81a4830 con 0x7fdaf8077170 2024-05-27T07:45:22.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.805+0000 7fdafe800700 1 -- 172.21.15.90:0/363643803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf81a4d20 con 0x7fdaf8077170 2024-05-27T07:45:22.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.806+0000 7fdafe800700 1 -- 172.21.15.90:0/363643803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf80512b0 con 0x7fdaf8077170 2024-05-27T07:45:22.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.809+0000 7fdaf4c00700 1 -- 172.21.15.90:0/363643803 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdae8010610 con 0x7fdaf8077170 2024-05-27T07:45:22.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.812+0000 7fdaf4c00700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdadc0b2900 0x7fdadc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:22.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.812+0000 7fdaf4c00700 1 -- 172.21.15.90:0/363643803 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fdae80e20a0 con 0x7fdaf8077170 2024-05-27T07:45:22.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.812+0000 7fdaefe00700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdadc0b2900 0x7fdadc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:22.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.813+0000 7fdaefe00700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdadc0b2900 0x7fdadc0b4e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdae0005950 tx=0x7fdae00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:22.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:22.816+0000 7fdaf4c00700 1 -- 172.21.15.90:0/363643803 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdae80a9a50 con 0x7fdaf8077170 2024-05-27T07:45:23.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.058+0000 7fdafe800700 1 -- 172.21.15.90:0/363643803 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdaf8063cb0 con 0x7fdadc0b2900 2024-05-27T07:45:23.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.097+0000 7fdaf4c00700 1 -- 172.21.15.90:0/363643803 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fdaf8063cb0 con 0x7fdadc0b2900 2024-05-27T07:45:23.097 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.a smithi090 running (4m) 2m ago 4m 82.5M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 f2ede0e85200 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.b smithi129 running (4m) 2m ago 4m 92.5M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 d2c1c48c587b 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.c smithi136 running (4m) 2m ago 4m 83.8M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 5faa54b04d55 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.d smithi090 running (5m) 2m ago 5m 80.5M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 54c7cd984db6 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.e smithi129 running (5m) 2m ago 4m 84.2M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 7f2d3fbd6f30 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.f smithi136 running (4m) 2m ago 4m 82.7M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 1f254d421592 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.g smithi090 running (4m) 2m ago 4m 82.6M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 37b82db357a2 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.h smithi129 running (4m) 2m ago 4m 84.2M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 37f5f08697cf 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.i smithi136 running (4m) 2m ago 4m 83.9M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 8a9624ce7aa2 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.j smithi090 running (5m) 2m ago 5m 80.5M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 164daf34dda4 2024-05-27T07:45:23.098 INFO:teuthology.orchestra.run.smithi090.stdout:mds.k smithi129 running (5m) 2m ago 5m 84.1M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 446c1375720a 2024-05-27T07:45:23.099 INFO:teuthology.orchestra.run.smithi090.stdout:mds.l smithi136 running (5m) 2m ago 5m 82.8M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 d8a33d5999d1 2024-05-27T07:45:23.099 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.x smithi090 *:9283,8765 running (15m) 2m ago 15m 599M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 da1e5558b99a 2024-05-27T07:45:23.099 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.y smithi129 *:8443,8765 running (12m) 2m ago 12m 534M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 d65cd30f1ca7 2024-05-27T07:45:23.099 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.z smithi136 *:8443,8765 running (12m) 2m ago 12m 534M - 19.0.0-3872-g38f9d900 e36ea4fc8a58 550d9598331b 2024-05-27T07:45:23.099 INFO:teuthology.orchestra.run.smithi090.stdout:mon.a smithi090 running (15m) 2m ago 15m 118M 2048M 19.0.0-3872-g38f9d900 e36ea4fc8a58 217e3dbbb313 2024-05-27T07:45:23.099 INFO:teuthology.orchestra.run.smithi090.stdout:mon.b smithi129 running (13m) 2m ago 13m 107M 2048M 19.0.0-3872-g38f9d900 e36ea4fc8a58 714573e46108 2024-05-27T07:45:23.099 INFO:teuthology.orchestra.run.smithi090.stdout:mon.c smithi136 running (13m) 2m ago 13m 108M 2048M 19.0.0-3872-g38f9d900 e36ea4fc8a58 8986a2754a27 2024-05-27T07:45:23.099 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (12m) 2m ago 12m 112M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 cd049f999030 2024-05-27T07:45:23.099 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi129 running (11m) 2m ago 11m 120M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 fa599bc94cdc 2024-05-27T07:45:23.100 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi136 running (11m) 2m ago 11m 119M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 37b3dd9d139e 2024-05-27T07:45:23.100 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi090 running (10m) 2m ago 10m 119M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 e569ebc33c4f 2024-05-27T07:45:23.100 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi129 running (10m) 2m ago 10m 111M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 220487bdf3ed 2024-05-27T07:45:23.100 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi136 running (9m) 2m ago 9m 111M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 755538b44e88 2024-05-27T07:45:23.100 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 smithi090 running (8m) 2m ago 8m 117M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 47393c4e2cb4 2024-05-27T07:45:23.100 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 smithi129 running (8m) 2m ago 8m 111M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 097c27e12a30 2024-05-27T07:45:23.100 INFO:teuthology.orchestra.run.smithi090.stdout:osd.8 smithi136 running (7m) 2m ago 7m 112M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 610c11351dfa 2024-05-27T07:45:23.100 INFO:teuthology.orchestra.run.smithi090.stdout:osd.9 smithi090 running (6m) 2m ago 6m 109M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 b1ecf289ef21 2024-05-27T07:45:23.100 INFO:teuthology.orchestra.run.smithi090.stdout:osd.10 smithi129 running (6m) 2m ago 6m 110M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 bcead1a9ea6e 2024-05-27T07:45:23.100 INFO:teuthology.orchestra.run.smithi090.stdout:osd.11 smithi136 running (5m) 2m ago 5m 114M 4272M 19.0.0-3872-g38f9d900 e36ea4fc8a58 46575a5aaf85 2024-05-27T07:45:23.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.101+0000 7fdafe800700 1 -- 172.21.15.90:0/363643803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdadc0b2900 msgr2=0x7fdadc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:23.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdadc0b2900 0x7fdadc0b4e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdae0005950 tx=0x7fdae00058e0 comp rx=0 tx=0).stop 2024-05-27T07:45:23.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 -- 172.21.15.90:0/363643803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdaf8077170 msgr2=0x7fdaf819c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:23.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdaf8077170 0x7fdaf819c9e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdae800d980 tx=0x7fdae800dd40 comp rx=0 tx=0).stop 2024-05-27T07:45:23.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 -- 172.21.15.90:0/363643803 shutdown_connections 2024-05-27T07:45:23.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdadc0b2900 0x7fdadc0b4e10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:23.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaf810c240 0x7fdaf81a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:23.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdaf8077170 0x7fdaf819c9e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:23.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 --2- 172.21.15.90:0/363643803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdaf8078760 0x7fdaf819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:23.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 -- 172.21.15.90:0/363643803 >> 172.21.15.90:0/363643803 conn(0x7fdaf80ff9c0 msgr2=0x7fdaf8108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:23.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 -- 172.21.15.90:0/363643803 shutdown_connections 2024-05-27T07:45:23.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:23.102+0000 7fdafe800700 1 -- 172.21.15.90:0/363643803 wait complete. 2024-05-27T07:45:23.994 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-27T07:45:24.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:23 smithi090 ceph-mon[31429]: pgmap v513: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:24.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:23 smithi136 ceph-mon[35324]: pgmap v513: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:24.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:23 smithi129 ceph-mon[35254]: pgmap v513: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:24.443 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:25.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:24 smithi136 ceph-mon[35324]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:25.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:24 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:45:25.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:24 smithi129 ceph-mon[35254]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:25.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:24 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:45:25.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:24 smithi090 ceph-mon[31429]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:25.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:24 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:45:26.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:25 smithi136 ceph-mon[35324]: pgmap v514: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:26.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:25 smithi129 ceph-mon[35254]: pgmap v514: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:26.347 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:25 smithi090 ceph-mon[31429]: pgmap v514: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:26.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.715+0000 7f71a1200700 1 -- 172.21.15.90:0/2714545669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f719c105580 msgr2=0x7f719c1079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:26.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.715+0000 7f71a1200700 1 --2- 172.21.15.90:0/2714545669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f719c105580 0x7f719c1079a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7188009990 tx=0x7f7188009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.716+0000 7f71a1200700 1 -- 172.21.15.90:0/2714545669 shutdown_connections 2024-05-27T07:45:26.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.716+0000 7f71a1200700 1 --2- 172.21.15.90:0/2714545669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f719c107ee0 0x7f719c1082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.716+0000 7f71a1200700 1 --2- 172.21.15.90:0/2714545669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f719c102c20 0x7f719c105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.716+0000 7f71a1200700 1 --2- 172.21.15.90:0/2714545669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f719c105580 0x7f719c1079a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.716+0000 7f71a1200700 1 -- 172.21.15.90:0/2714545669 >> 172.21.15.90:0/2714545669 conn(0x7f719c0fe870 msgr2=0x7f719c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:26.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.716+0000 7f71a1200700 1 -- 172.21.15.90:0/2714545669 shutdown_connections 2024-05-27T07:45:26.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.716+0000 7f71a1200700 1 -- 172.21.15.90:0/2714545669 wait complete. 2024-05-27T07:45:26.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.717+0000 7f71a1200700 1 Processor -- start 2024-05-27T07:45:26.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.717+0000 7f71a1200700 1 -- start start 2024-05-27T07:45:26.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.717+0000 7f71a1200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f719c102c20 0x7f719c1a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:26.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.717+0000 7f71a1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f719c105580 0x7f719c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:26.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.717+0000 7f71a1200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f719c107ee0 0x7f719c1a8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:26.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.717+0000 7f71a1200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f719c116240 con 0x7f719c105580 2024-05-27T07:45:26.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.717+0000 7f71a1200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1160c0 con 0x7f719c107ee0 2024-05-27T07:45:26.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.717+0000 7f71a1200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f719c1163c0 con 0x7f719c102c20 2024-05-27T07:45:26.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f7199400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f719c105580 0x7f719c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:26.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f7199400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f719c105580 0x7f719c1a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49292/0 (socket says 172.21.15.90:49292) 2024-05-27T07:45:26.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f7199400700 1 -- 172.21.15.90:0/1703028504 learned_addr learned my addr 172.21.15.90:0/1703028504 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:26.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f7199e00700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f719c102c20 0x7f719c1a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:26.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f719a800700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f719c107ee0 0x7f719c1a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:26.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f7199400700 1 -- 172.21.15.90:0/1703028504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f719c102c20 msgr2=0x7f719c1a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:26.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f7199400700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f719c102c20 0x7f719c1a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f7199400700 1 -- 172.21.15.90:0/1703028504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f719c107ee0 msgr2=0x7f719c1a8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:26.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f7199400700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f719c107ee0 0x7f719c1a8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f7199400700 1 -- 172.21.15.90:0/1703028504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c1a8a00 con 0x7f719c105580 2024-05-27T07:45:26.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.718+0000 7f7199e00700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f719c102c20 0x7f719c1a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:26.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.719+0000 7f7199400700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f719c105580 0x7f719c1a13e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f71880098e0 tx=0x7f71880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:26.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.719+0000 7f7192a00700 1 -- 172.21.15.90:0/1703028504 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188003c10 con 0x7f719c105580 2024-05-27T07:45:26.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.719+0000 7f7192a00700 1 -- 172.21.15.90:0/1703028504 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7188004830 con 0x7f719c105580 2024-05-27T07:45:26.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.719+0000 7f7192a00700 1 -- 172.21.15.90:0/1703028504 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188005350 con 0x7f719c105580 2024-05-27T07:45:26.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.719+0000 7f71a1200700 1 -- 172.21.15.90:0/1703028504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c1a8c90 con 0x7f719c105580 2024-05-27T07:45:26.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.719+0000 7f71a1200700 1 -- 172.21.15.90:0/1703028504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c1a9030 con 0x7f719c105580 2024-05-27T07:45:26.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.720+0000 7f71a1200700 1 -- 172.21.15.90:0/1703028504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719c0512b0 con 0x7f719c105580 2024-05-27T07:45:26.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.722+0000 7f7192a00700 1 -- 172.21.15.90:0/1703028504 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f71880054f0 con 0x7f719c105580 2024-05-27T07:45:26.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.724+0000 7f7192a00700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f71800b2900 0x7f71800b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:26.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.724+0000 7f7192a00700 1 -- 172.21.15.90:0/1703028504 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f71880e3ad0 con 0x7f719c105580 2024-05-27T07:45:26.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.724+0000 7f7199e00700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f71800b2900 0x7f71800b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:26.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.725+0000 7f7199e00700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f71800b2900 0x7f71800b4e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f719c06c020 tx=0x7f718c00b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:26.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.726+0000 7f7192a00700 1 -- 172.21.15.90:0/1703028504 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7188013070 con 0x7f719c105580 2024-05-27T07:45:26.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.971+0000 7f71a1200700 1 -- 172.21.15.90:0/1703028504 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f719c063bf0 con 0x7f71800b2900 2024-05-27T07:45:26.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.990+0000 7f7192a00700 1 -- 172.21.15.90:0/1703028504 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f719c063bf0 con 0x7f71800b2900 2024-05-27T07:45:26.990 INFO:teuthology.orchestra.run.smithi090.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-27T07:45:26.991 INFO:teuthology.orchestra.run.smithi090.stdout:mds.all 12/12 2m ago 5m smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-05-27T07:45:26.991 INFO:teuthology.orchestra.run.smithi090.stdout:mgr 3/3 2m ago 13m smithi090=x;smithi129=y;smithi136=z;count:3 2024-05-27T07:45:26.991 INFO:teuthology.orchestra.run.smithi090.stdout:mon 3/3 2m ago 13m smithi090:172.21.15.90=a;smithi129:172.21.15.129=b;smithi136:172.21.15.136=c;count:3 2024-05-27T07:45:26.991 INFO:teuthology.orchestra.run.smithi090.stdout:osd 12 2m ago - 2024-05-27T07:45:26.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 -- 172.21.15.90:0/1703028504 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f71800b2900 msgr2=0x7f71800b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f71800b2900 0x7f71800b4e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f719c06c020 tx=0x7f718c00b5f0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 -- 172.21.15.90:0/1703028504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f719c105580 msgr2=0x7f719c1a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f719c105580 0x7f719c1a13e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f71880098e0 tx=0x7f71880047b0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 -- 172.21.15.90:0/1703028504 shutdown_connections 2024-05-27T07:45:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f71800b2900 0x7f71800b4e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f719c107ee0 0x7f719c1a8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f719c102c20 0x7f719c1a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 --2- 172.21.15.90:0/1703028504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f719c105580 0x7f719c1a13e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:26.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 -- 172.21.15.90:0/1703028504 >> 172.21.15.90:0/1703028504 conn(0x7f719c0fe870 msgr2=0x7f719c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:26.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 -- 172.21.15.90:0/1703028504 shutdown_connections 2024-05-27T07:45:26.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:26.994+0000 7f71a1200700 1 -- 172.21.15.90:0/1703028504 wait complete. 2024-05-27T07:45:27.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:26 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:45:27.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:26 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:45:27.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:26 smithi136 ceph-mon[35324]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:45:27.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:26 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:45:27.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:26 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:45:27.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:26 smithi129 ceph-mon[35254]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:45:27.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:26 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:45:27.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:26 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:45:27.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:26 smithi090 ceph-mon[31429]: from='mgr.14152 172.21.15.90:0/65769219' entity='mgr.x' 2024-05-27T07:45:27.780 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-27T07:45:28.224 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:28.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:27 smithi136 ceph-mon[35324]: pgmap v515: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:28.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:27 smithi136 ceph-mon[35324]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:28.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:27 smithi129 ceph-mon[35254]: pgmap v515: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:28.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:27 smithi129 ceph-mon[35254]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:27 smithi090 ceph-mon[31429]: pgmap v515: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:27 smithi090 ceph-mon[31429]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:29 smithi090 ceph-mon[31429]: pgmap v516: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:30.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:29 smithi136 ceph-mon[35324]: pgmap v516: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:30.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:29 smithi129 ceph-mon[35254]: pgmap v516: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.472+0000 7efebca00700 1 -- 172.21.15.90:0/1699477438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8107670 msgr2=0x7efeb8107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:30.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.472+0000 7efebca00700 1 --2- 172.21.15.90:0/1699477438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8107670 0x7efeb8107a80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efea4009990 tx=0x7efea4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.472+0000 7efebca00700 1 -- 172.21.15.90:0/1699477438 shutdown_connections 2024-05-27T07:45:30.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.472+0000 7efebca00700 1 --2- 172.21.15.90:0/1699477438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeb8108010 0x7efeb810e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.472+0000 7efebca00700 1 --2- 172.21.15.90:0/1699477438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb8106540 0x7efeb8106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.472+0000 7efebca00700 1 --2- 172.21.15.90:0/1699477438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8107670 0x7efeb8107a80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.472+0000 7efebca00700 1 -- 172.21.15.90:0/1699477438 >> 172.21.15.90:0/1699477438 conn(0x7efeb8101b80 msgr2=0x7efeb8104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:30.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.472+0000 7efebca00700 1 -- 172.21.15.90:0/1699477438 shutdown_connections 2024-05-27T07:45:30.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.472+0000 7efebca00700 1 -- 172.21.15.90:0/1699477438 wait complete. 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.473+0000 7efebca00700 1 Processor -- start 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.474+0000 7efebca00700 1 -- start start 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.474+0000 7efebca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb8106540 0x7efeb819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.474+0000 7efebca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8107670 0x7efeb819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.474+0000 7efebca00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeb8108010 0x7efeb81a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.474+0000 7efebca00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8111eb0 con 0x7efeb8107670 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.474+0000 7efebca00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8111d30 con 0x7efeb8106540 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.474+0000 7efebca00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8112030 con 0x7efeb8108010 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.474+0000 7efeb5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8107670 0x7efeb819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8107670 0x7efeb819cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41054/0 (socket says 172.21.15.90:41054) 2024-05-27T07:45:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb5400700 1 -- 172.21.15.90:0/1593001038 learned_addr learned my addr 172.21.15.90:0/1593001038 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb5e00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb8106540 0x7efeb819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb6800700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeb8108010 0x7efeb81a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb5400700 1 -- 172.21.15.90:0/1593001038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeb8108010 msgr2=0x7efeb81a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb5400700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeb8108010 0x7efeb81a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb5400700 1 -- 172.21.15.90:0/1593001038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb8106540 msgr2=0x7efeb819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb5400700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb8106540 0x7efeb819c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb5400700 1 -- 172.21.15.90:0/1593001038 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb81a4550 con 0x7efeb8107670 2024-05-27T07:45:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb5e00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb8106540 0x7efeb819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:30.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.475+0000 7efeb5400700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8107670 0x7efeb819cf30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efea4000c00 tx=0x7efea4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:30.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.476+0000 7efeaea00700 1 -- 172.21.15.90:0/1593001038 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4027070 con 0x7efeb8107670 2024-05-27T07:45:30.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.476+0000 7efeaea00700 1 -- 172.21.15.90:0/1593001038 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efea4004020 con 0x7efeb8107670 2024-05-27T07:45:30.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.476+0000 7efeaea00700 1 -- 172.21.15.90:0/1593001038 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4022ca0 con 0x7efeb8107670 2024-05-27T07:45:30.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.476+0000 7efebca00700 1 -- 172.21.15.90:0/1593001038 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb81a47e0 con 0x7efeb8107670 2024-05-27T07:45:30.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.476+0000 7efebca00700 1 -- 172.21.15.90:0/1593001038 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb81a4be0 con 0x7efeb8107670 2024-05-27T07:45:30.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.477+0000 7efebca00700 1 -- 172.21.15.90:0/1593001038 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb80512b0 con 0x7efeb8107670 2024-05-27T07:45:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.480+0000 7efeaea00700 1 -- 172.21.15.90:0/1593001038 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efea402a070 con 0x7efeb8107670 2024-05-27T07:45:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.482+0000 7efeaea00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efe9c0b2880 0x7efe9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.482+0000 7efeaea00700 1 -- 172.21.15.90:0/1593001038 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7efea40e3e70 con 0x7efeb8107670 2024-05-27T07:45:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.482+0000 7efeb5e00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efe9c0b2880 0x7efe9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:30.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.483+0000 7efeb5e00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efe9c0b2880 0x7efe9c0b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efea8005950 tx=0x7efea80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:30.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.486+0000 7efeaea00700 1 -- 172.21.15.90:0/1593001038 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efea4031080 con 0x7efeb8107670 2024-05-27T07:45:30.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.724+0000 7efebca00700 1 -- 172.21.15.90:0/1593001038 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7efeb8063bf0 con 0x7efe9c0b2880 2024-05-27T07:45:30.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.727+0000 7efeaea00700 1 -- 172.21.15.90:0/1593001038 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7efeb8063bf0 con 0x7efe9c0b2880 2024-05-27T07:45:30.726 INFO:teuthology.orchestra.run.smithi090.stdout:HOST ADDR LABELS STATUS 2024-05-27T07:45:30.727 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 172.21.15.90 2024-05-27T07:45:30.727 INFO:teuthology.orchestra.run.smithi090.stdout:smithi129 172.21.15.129 2024-05-27T07:45:30.727 INFO:teuthology.orchestra.run.smithi090.stdout:smithi136 172.21.15.136 2024-05-27T07:45:30.727 INFO:teuthology.orchestra.run.smithi090.stdout:3 hosts in cluster 2024-05-27T07:45:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 -- 172.21.15.90:0/1593001038 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efe9c0b2880 msgr2=0x7efe9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efe9c0b2880 0x7efe9c0b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efea8005950 tx=0x7efea80058e0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 -- 172.21.15.90:0/1593001038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8107670 msgr2=0x7efeb819cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8107670 0x7efeb819cf30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efea4000c00 tx=0x7efea4009580 comp rx=0 tx=0).stop 2024-05-27T07:45:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 -- 172.21.15.90:0/1593001038 shutdown_connections 2024-05-27T07:45:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efe9c0b2880 0x7efe9c0b4d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeb8108010 0x7efeb81a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb8106540 0x7efeb819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 --2- 172.21.15.90:0/1593001038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efeb8107670 0x7efeb819cf30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 -- 172.21.15.90:0/1593001038 >> 172.21.15.90:0/1593001038 conn(0x7efeb8101b80 msgr2=0x7efeb810a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 -- 172.21.15.90:0/1593001038 shutdown_connections 2024-05-27T07:45:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:30.730+0000 7efebca00700 1 -- 172.21.15.90:0/1593001038 wait complete. 2024-05-27T07:45:31.606 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-27T07:45:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:31 smithi136 ceph-mon[35324]: pgmap v517: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:31 smithi136 ceph-mon[35324]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:32.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:31 smithi129 ceph-mon[35254]: pgmap v517: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:32.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:31 smithi129 ceph-mon[35254]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:32.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:31 smithi090 ceph-mon[31429]: pgmap v517: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:32.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:31 smithi090 ceph-mon[31429]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:34.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.201+0000 7f5590800700 1 -- 172.21.15.90:0/1796342631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5588105e30 msgr2=0x7f558810c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:34.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.201+0000 7f5590800700 1 --2- 172.21.15.90:0/1796342631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5588105e30 0x7f558810c6e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f558400b0a0 tx=0x7f558400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:45:34.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.201+0000 7f5590800700 1 -- 172.21.15.90:0/1796342631 shutdown_connections 2024-05-27T07:45:34.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.201+0000 7f5590800700 1 --2- 172.21.15.90:0/1796342631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5588105e30 0x7f558810c6e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:34.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.201+0000 7f5590800700 1 --2- 172.21.15.90:0/1796342631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5588104360 0x7f5588104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:34.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.201+0000 7f5590800700 1 --2- 172.21.15.90:0/1796342631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588105490 0x7f55881058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:34.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.201+0000 7f5590800700 1 -- 172.21.15.90:0/1796342631 >> 172.21.15.90:0/1796342631 conn(0x7f55880ff9c0 msgr2=0x7f5588101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:34.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.201+0000 7f5590800700 1 -- 172.21.15.90:0/1796342631 shutdown_connections 2024-05-27T07:45:34.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.201+0000 7f5590800700 1 -- 172.21.15.90:0/1796342631 wait complete. 2024-05-27T07:45:34.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.202+0000 7f5590800700 1 Processor -- start 2024-05-27T07:45:34.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.203+0000 7f5590800700 1 -- start start 2024-05-27T07:45:34.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.203+0000 7f5590800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5588104360 0x7f558819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:34.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.203+0000 7f5590800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588105490 0x7f558819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:34.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.203+0000 7f5590800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5588105e30 0x7f55881a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:34.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.203+0000 7f5590800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f558810fce0 con 0x7f5588104360 2024-05-27T07:45:34.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.203+0000 7f5590800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f558810fb60 con 0x7f5588105490 2024-05-27T07:45:34.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.203+0000 7f5590800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f558810fe60 con 0x7f5588105e30 2024-05-27T07:45:34.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.203+0000 7f558de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5588104360 0x7f558819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:34.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.203+0000 7f558de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5588104360 0x7f558819ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41072/0 (socket says 172.21.15.90:41072) 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.203+0000 7f558de00700 1 -- 172.21.15.90:0/784424201 learned_addr learned my addr 172.21.15.90:0/784424201 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.204+0000 7f558e800700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5588105e30 0x7f55881a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.204+0000 7f558d400700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588105490 0x7f558819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.204+0000 7f558de00700 1 -- 172.21.15.90:0/784424201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5588105e30 msgr2=0x7f55881a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.204+0000 7f558de00700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5588105e30 0x7f55881a3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.204+0000 7f558de00700 1 -- 172.21.15.90:0/784424201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588105490 msgr2=0x7f558819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.204+0000 7f558de00700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588105490 0x7f558819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.204+0000 7f558de00700 1 -- 172.21.15.90:0/784424201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55881a44e0 con 0x7f5588104360 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.204+0000 7f558de00700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5588104360 0x7f558819ca10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f557c00d950 tx=0x7f557c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.205+0000 7f5582a00700 1 -- 172.21.15.90:0/784424201 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c017070 con 0x7f5588104360 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.205+0000 7f5582a00700 1 -- 172.21.15.90:0/784424201 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f557c00de30 con 0x7f5588104360 2024-05-27T07:45:34.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.205+0000 7f5582a00700 1 -- 172.21.15.90:0/784424201 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c00f3c0 con 0x7f5588104360 2024-05-27T07:45:34.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.205+0000 7f5590800700 1 -- 172.21.15.90:0/784424201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55881a47d0 con 0x7f5588104360 2024-05-27T07:45:34.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.205+0000 7f5590800700 1 -- 172.21.15.90:0/784424201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55881a4cc0 con 0x7f5588104360 2024-05-27T07:45:34.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.206+0000 7f5590800700 1 -- 172.21.15.90:0/784424201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55880512b0 con 0x7f5588104360 2024-05-27T07:45:34.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.209+0000 7f5582a00700 1 -- 172.21.15.90:0/784424201 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f557c01a070 con 0x7f5588104360 2024-05-27T07:45:34.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:34 smithi090 ceph-mon[31429]: pgmap v518: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:34.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.210+0000 7f5582a00700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55700b2880 0x7f55700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:34.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.211+0000 7f5582a00700 1 -- 172.21.15.90:0/784424201 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f557c0e1fa0 con 0x7f5588104360 2024-05-27T07:45:34.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.211+0000 7f558d400700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55700b2880 0x7f55700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:34.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.212+0000 7f558d400700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55700b2880 0x7f55700b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5578005ea0 tx=0x7f5578005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:34.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.213+0000 7f5582a00700 1 -- 172.21.15.90:0/784424201 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f557c010460 con 0x7f5588104360 2024-05-27T07:45:34.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:34 smithi129 ceph-mon[35254]: pgmap v518: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:34.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.455+0000 7f5590800700 1 -- 172.21.15.90:0/784424201 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f55881a4fe0 con 0x7f55700b2880 2024-05-27T07:45:34.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.459+0000 7f5582a00700 1 -- 172.21.15.90:0/784424201 <== mgr.14152 v2:172.21.15.90:6800/2854405112 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f55881a4fe0 con 0x7f55700b2880 2024-05-27T07:45:34.459 INFO:teuthology.orchestra.run.smithi090.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-27T07:45:34.459 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UK400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-27T07:45:34.459 INFO:teuthology.orchestra.run.smithi090.stdout:smithi129 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300HY400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-27T07:45:34.460 INFO:teuthology.orchestra.run.smithi090.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 5m ago Has a FileSystem, LVM detected 2024-05-27T07:45:34.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.462+0000 7f5590800700 1 -- 172.21.15.90:0/784424201 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55700b2880 msgr2=0x7f55700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.462+0000 7f5590800700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55700b2880 0x7f55700b4d90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5578005ea0 tx=0x7f5578005e30 comp rx=0 tx=0).stop 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.462+0000 7f5590800700 1 -- 172.21.15.90:0/784424201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5588104360 msgr2=0x7f558819ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.462+0000 7f5590800700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5588104360 0x7f558819ca10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f557c00d950 tx=0x7f557c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.462+0000 7f5590800700 1 -- 172.21.15.90:0/784424201 shutdown_connections 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.462+0000 7f5590800700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f55700b2880 0x7f55700b4d90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.462+0000 7f5590800700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5588105e30 0x7f55881a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.462+0000 7f5590800700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5588104360 0x7f558819ca10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.462+0000 7f5590800700 1 --2- 172.21.15.90:0/784424201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5588105490 0x7f558819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.463+0000 7f5590800700 1 -- 172.21.15.90:0/784424201 >> 172.21.15.90:0/784424201 conn(0x7f55880ff9c0 msgr2=0x7f5588076af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.463+0000 7f5590800700 1 -- 172.21.15.90:0/784424201 shutdown_connections 2024-05-27T07:45:34.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:34.463+0000 7f5590800700 1 -- 172.21.15.90:0/784424201 wait complete. 2024-05-27T07:45:34.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:34 smithi136 ceph-mon[35324]: pgmap v518: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:35.351 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T07:45:35.361 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T07:45:35.361 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-27T07:45:35.811 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:36.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:36 smithi090 ceph-mon[31429]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:36.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:36 smithi090 ceph-mon[31429]: pgmap v519: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:36.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:36 smithi129 ceph-mon[35254]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:36.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:36 smithi129 ceph-mon[35254]: pgmap v519: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:36.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:36 smithi136 ceph-mon[35324]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:45:36.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:36 smithi136 ceph-mon[35324]: pgmap v519: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:38.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.210+0000 7f43e3e00700 1 -- 172.21.15.90:0/1970597375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43e4105c10 msgr2=0x7f43e410c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:38.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.210+0000 7f43e3e00700 1 --2- 172.21.15.90:0/1970597375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43e4105c10 0x7f43e410c4c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009990 tx=0x7f43dc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:38.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.210+0000 7f43e3e00700 1 -- 172.21.15.90:0/1970597375 shutdown_connections 2024-05-27T07:45:38.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.210+0000 7f43e3e00700 1 --2- 172.21.15.90:0/1970597375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43e4105c10 0x7f43e410c4c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:38.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.210+0000 7f43e3e00700 1 --2- 172.21.15.90:0/1970597375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4104140 0x7f43e4104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:38.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.210+0000 7f43e3e00700 1 --2- 172.21.15.90:0/1970597375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e4105270 0x7f43e4105680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:38.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.210+0000 7f43e3e00700 1 -- 172.21.15.90:0/1970597375 >> 172.21.15.90:0/1970597375 conn(0x7f43e40ff7a0 msgr2=0x7f43e4101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:38.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.210+0000 7f43e3e00700 1 -- 172.21.15.90:0/1970597375 shutdown_connections 2024-05-27T07:45:38.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.210+0000 7f43e3e00700 1 -- 172.21.15.90:0/1970597375 wait complete. 2024-05-27T07:45:38.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:38 smithi090 ceph-mon[31429]: pgmap v520: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.211+0000 7f43e3e00700 1 Processor -- start 2024-05-27T07:45:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.211+0000 7f43e3e00700 1 -- start start 2024-05-27T07:45:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.212+0000 7f43e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43e4104140 0x7f43e419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.212+0000 7f43e3e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e4105270 0x7f43e419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.212+0000 7f43e3e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4105c10 0x7f43e41a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:38.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.212+0000 7f43e3e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f43e410fa60 con 0x7f43e4104140 2024-05-27T07:45:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.212+0000 7f43e3e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43e410f8e0 con 0x7f43e4105270 2024-05-27T07:45:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.212+0000 7f43e3e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43e410fbe0 con 0x7f43e4105c10 2024-05-27T07:45:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.212+0000 7f43e2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43e4104140 0x7f43e419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.212+0000 7f43e2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43e4104140 0x7f43e419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41102/0 (socket says 172.21.15.90:41102) 2024-05-27T07:45:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.212+0000 7f43e2a00700 1 -- 172.21.15.90:0/1720245649 learned_addr learned my addr 172.21.15.90:0/1720245649 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.212+0000 7f43da000700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e4105270 0x7f43e419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.213+0000 7f43e2a00700 1 -- 172.21.15.90:0/1720245649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4105c10 msgr2=0x7f43e41a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.213+0000 7f43e2a00700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4105c10 0x7f43e41a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:38.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.213+0000 7f43e2a00700 1 -- 172.21.15.90:0/1720245649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e4105270 msgr2=0x7f43e419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:38.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.213+0000 7f43e2a00700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e4105270 0x7f43e419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:38.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.213+0000 7f43e2a00700 1 -- 172.21.15.90:0/1720245649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e41a4340 con 0x7f43e4104140 2024-05-27T07:45:38.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.213+0000 7f43e2a00700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43e4104140 0x7f43e419c7e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f43d400ef70 tx=0x7f43d400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:38.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.213+0000 7f43dbe00700 1 -- 172.21.15.90:0/1720245649 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d40092e0 con 0x7f43e4104140 2024-05-27T07:45:38.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.213+0000 7f43dbe00700 1 -- 172.21.15.90:0/1720245649 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43d4003a00 con 0x7f43e4104140 2024-05-27T07:45:38.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.213+0000 7f43e3e00700 1 -- 172.21.15.90:0/1720245649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e41a45d0 con 0x7f43e4104140 2024-05-27T07:45:38.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.213+0000 7f43dbe00700 1 -- 172.21.15.90:0/1720245649 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d40045b0 con 0x7f43e4104140 2024-05-27T07:45:38.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.214+0000 7f43e3e00700 1 -- 172.21.15.90:0/1720245649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e41a4b10 con 0x7f43e4104140 2024-05-27T07:45:38.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.215+0000 7f43e3e00700 1 -- 172.21.15.90:0/1720245649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e40512b0 con 0x7f43e4104140 2024-05-27T07:45:38.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.217+0000 7f43dbe00700 1 -- 172.21.15.90:0/1720245649 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f43d401b030 con 0x7f43e4104140 2024-05-27T07:45:38.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.219+0000 7f43dbe00700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f43c80b2880 0x7f43c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:38.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.219+0000 7f43dbe00700 1 -- 172.21.15.90:0/1720245649 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f43d40e2570 con 0x7f43e4104140 2024-05-27T07:45:38.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.219+0000 7f43da000700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f43c80b2880 0x7f43c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:38.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.220+0000 7f43da000700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f43c80b2880 0x7f43c80b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43cc005fd0 tx=0x7f43cc005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:38.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.224+0000 7f43dbe00700 1 -- 172.21.15.90:0/1720245649 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f43d4022020 con 0x7f43e4104140 2024-05-27T07:45:38.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:38 smithi129 ceph-mon[35254]: pgmap v520: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:38.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:38 smithi136 ceph-mon[35324]: pgmap v520: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:38.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.505+0000 7f43e3e00700 1 -- 172.21.15.90:0/1720245649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f43e41a54b0 con 0x7f43e4104140 2024-05-27T07:45:38.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.506+0000 7f43dbe00700 1 -- 172.21.15.90:0/1720245649 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3098 (secure 0 0 0) 0x7f43d4010040 con 0x7f43e4104140 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout:e29 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 29 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-05-27T07:41:11.710115+0000 2024-05-27T07:45:38.508 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-05-27T07:42:38.788447+0000 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 300 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 0 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 5 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1,2,3,4 2024-05-27T07:45:38.509 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=34472,1=34442,2=34424,3=34406,4=24478} 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3,4] 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 0 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 34472 members: 24478,34406,34424,34442,34472 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b{0:34472} state up:active seq 6 addr [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.510 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.136:6836/3523286296,v1:172.21.15.136:6837/3523286296] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.h{1:34442} state up:active seq 30 addr [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.f{1:24451} state up:standby-replay seq 1 addr [v2:172.21.15.136:6834/1419480441,v1:172.21.15.136:6835/1419480441] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.e{2:34424} state up:active seq 34 addr [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.l{2:24415} state up:standby-replay seq 1 addr [v2:172.21.15.136:6832/1595668128,v1:172.21.15.136:6833/1595668128] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.k{3:34406} state up:active seq 37 addr [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a{3:14562} state up:standby-replay seq 1 addr [v2:172.21.15.90:6840/24895501,v1:172.21.15.90:6841/24895501] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.i{4:24478} state up:active seq 30 addr [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.g{4:14556} state up:standby-replay seq 1 addr [v2:172.21.15.90:6838/767662982,v1:172.21.15.90:6839/767662982] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.j{-1:14502} state up:standby seq 1 addr [v2:172.21.15.90:6834/2091177441,v1:172.21.15.90:6835/2091177441] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.511 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d{-1:14514} state up:standby seq 1 addr [v2:172.21.15.90:6836/3818984519,v1:172.21.15.90:6837/3818984519] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T07:45:38.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.509+0000 7f43e3e00700 1 -- 172.21.15.90:0/1720245649 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f43c80b2880 msgr2=0x7f43c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:38.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.509+0000 7f43e3e00700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f43c80b2880 0x7f43c80b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f43cc005fd0 tx=0x7f43cc005f60 comp rx=0 tx=0).stop 2024-05-27T07:45:38.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.509+0000 7f43e3e00700 1 -- 172.21.15.90:0/1720245649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43e4104140 msgr2=0x7f43e419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:38.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.509+0000 7f43e3e00700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43e4104140 0x7f43e419c7e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f43d400ef70 tx=0x7f43d400c450 comp rx=0 tx=0).stop 2024-05-27T07:45:38.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.510+0000 7f43e3e00700 1 -- 172.21.15.90:0/1720245649 shutdown_connections 2024-05-27T07:45:38.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.510+0000 7f43e3e00700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f43c80b2880 0x7f43c80b4d90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:38.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.510+0000 7f43e3e00700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e4105c10 0x7f43e41a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:38.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.510+0000 7f43e3e00700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43e4104140 0x7f43e419c7e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:38.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.510+0000 7f43e3e00700 1 --2- 172.21.15.90:0/1720245649 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43e4105270 0x7f43e419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:38.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.510+0000 7f43e3e00700 1 -- 172.21.15.90:0/1720245649 >> 172.21.15.90:0/1720245649 conn(0x7f43e40ff7a0 msgr2=0x7f43e40759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:38.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.510+0000 7f43e3e00700 1 -- 172.21.15.90:0/1720245649 shutdown_connections 2024-05-27T07:45:38.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:38.510+0000 7f43e3e00700 1 -- 172.21.15.90:0/1720245649 wait complete. 2024-05-27T07:45:38.513 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-05-27T07:45:39.204 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:39 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1720245649' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T07:45:39.505 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:39 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1720245649' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T07:45:39.505 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:39 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1720245649' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T07:45:39.506 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-27T07:45:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:40.308 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:40 smithi090 ceph-mon[31429]: pgmap v521: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:40.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:40 smithi129 ceph-mon[35254]: pgmap v521: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:40.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:40 smithi136 ceph-mon[35324]: pgmap v521: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:41.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:41 smithi090 ceph-mon[31429]: pgmap v522: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:41.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:41 smithi129 ceph-mon[35254]: pgmap v522: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:41.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:41 smithi136 ceph-mon[35324]: pgmap v522: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.247+0000 7f7520800700 1 -- 172.21.15.90:0/3954335087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7518106540 msgr2=0x7f7518106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.247+0000 7f7520800700 1 --2- 172.21.15.90:0/3954335087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7518106540 0x7f7518106950 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f750c009990 tx=0x7f750c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.247+0000 7f7520800700 1 -- 172.21.15.90:0/3954335087 shutdown_connections 2024-05-27T07:45:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.247+0000 7f7520800700 1 --2- 172.21.15.90:0/3954335087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7518108010 0x7f751810e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.247+0000 7f7520800700 1 --2- 172.21.15.90:0/3954335087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7518106540 0x7f7518106950 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.247+0000 7f7520800700 1 --2- 172.21.15.90:0/3954335087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7518107670 0x7f7518107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.247+0000 7f7520800700 1 -- 172.21.15.90:0/3954335087 >> 172.21.15.90:0/3954335087 conn(0x7f7518101b80 msgr2=0x7f7518104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:42.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.247+0000 7f7520800700 1 -- 172.21.15.90:0/3954335087 shutdown_connections 2024-05-27T07:45:42.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.247+0000 7f7520800700 1 -- 172.21.15.90:0/3954335087 wait complete. 2024-05-27T07:45:42.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.249+0000 7f7520800700 1 Processor -- start 2024-05-27T07:45:42.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.249+0000 7f7520800700 1 -- start start 2024-05-27T07:45:42.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.249+0000 7f7520800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7518106540 0x7f751819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:42.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.249+0000 7f7520800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7518107670 0x7f751819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:42.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f7520800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7518108010 0x7f75181a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:42.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f7520800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7518111e80 con 0x7f7518106540 2024-05-27T07:45:42.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f7520800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7518111d00 con 0x7f7518108010 2024-05-27T07:45:42.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f7520800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7518112000 con 0x7f7518107670 2024-05-27T07:45:42.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f751de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7518106540 0x7f751819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:42.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f751de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7518106540 0x7f751819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35936/0 (socket says 172.21.15.90:35936) 2024-05-27T07:45:42.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f751de00700 1 -- 172.21.15.90:0/2087205674 learned_addr learned my addr 172.21.15.90:0/2087205674 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:42.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f751d400700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7518107670 0x7f751819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:42.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f751e800700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7518108010 0x7f75181a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:42.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f751de00700 1 -- 172.21.15.90:0/2087205674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7518107670 msgr2=0x7f751819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:42.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.250+0000 7f751de00700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7518107670 0x7f751819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.251+0000 7f751de00700 1 -- 172.21.15.90:0/2087205674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7518108010 msgr2=0x7f75181a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:42.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.251+0000 7f751de00700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7518108010 0x7f75181a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.251+0000 7f751de00700 1 -- 172.21.15.90:0/2087205674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75181a45f0 con 0x7f7518106540 2024-05-27T07:45:42.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.251+0000 7f751d400700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7518107670 0x7f751819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:42.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.251+0000 7f751de00700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7518106540 0x7f751819ca90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f750c003820 tx=0x7f750c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:42.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.251+0000 7f7512a00700 1 -- 172.21.15.90:0/2087205674 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c003c10 con 0x7f7518106540 2024-05-27T07:45:42.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.251+0000 7f7520800700 1 -- 172.21.15.90:0/2087205674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75181a4880 con 0x7f7518106540 2024-05-27T07:45:42.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.251+0000 7f7512a00700 1 -- 172.21.15.90:0/2087205674 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f750c004830 con 0x7f7518106540 2024-05-27T07:45:42.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.251+0000 7f7512a00700 1 -- 172.21.15.90:0/2087205674 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c005260 con 0x7f7518106540 2024-05-27T07:45:42.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.252+0000 7f7520800700 1 -- 172.21.15.90:0/2087205674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75181a4c70 con 0x7f7518106540 2024-05-27T07:45:42.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.253+0000 7f7520800700 1 -- 172.21.15.90:0/2087205674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75180512b0 con 0x7f7518106540 2024-05-27T07:45:42.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.255+0000 7f7512a00700 1 -- 172.21.15.90:0/2087205674 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f750c0054c0 con 0x7f7518106540 2024-05-27T07:45:42.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.257+0000 7f7512a00700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f75000b2880 0x7f75000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.257+0000 7f7512a00700 1 -- 172.21.15.90:0/2087205674 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f750c0e3760 con 0x7f7518106540 2024-05-27T07:45:42.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.257+0000 7f751d400700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f75000b2880 0x7f75000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:42.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.258+0000 7f751d400700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f75000b2880 0x7f75000b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7508005950 tx=0x7f75080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:42.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.262+0000 7f7512a00700 1 -- 172.21.15.90:0/2087205674 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f750c013070 con 0x7f7518106540 2024-05-27T07:45:42.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.493+0000 7f7520800700 1 -- 172.21.15.90:0/2087205674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f751802e5e0 con 0x7f7518106540 2024-05-27T07:45:42.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.495+0000 7f7512a00700 1 -- 172.21.15.90:0/2087205674 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5646 (secure 0 0 0) 0x7f750c0ab120 con 0x7f7518106540 2024-05-27T07:45:42.496 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 91 2024-05-27T07:45:42.496 INFO:teuthology.orchestra.run.smithi090.stdout:fsid b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T07:45:42.496 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-05-27T07:29:49.415763+0000 2024-05-27T07:45:42.496 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-05-27T07:43:00.000806+0000 2024-05-27T07:45:42.496 INFO:teuthology.orchestra.run.smithi090.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-27T07:45:42.496 INFO:teuthology.orchestra.run.smithi090.stdout:crush_version 27 2024-05-27T07:45:42.497 INFO:teuthology.orchestra.run.smithi090.stdout:full_ratio 0.95 2024-05-27T07:45:42.497 INFO:teuthology.orchestra.run.smithi090.stdout:backfillfull_ratio 0.9 2024-05-27T07:45:42.497 INFO:teuthology.orchestra.run.smithi090.stdout:nearfull_ratio 0.85 2024-05-27T07:45:42.497 INFO:teuthology.orchestra.run.smithi090.stdout:require_min_compat_client luminous 2024-05-27T07:45:42.497 INFO:teuthology.orchestra.run.smithi090.stdout:min_compat_client luminous 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:require_osd_release squid 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:stretch_mode_enabled false 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:max_osd 12 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6802/70197447,v1:172.21.15.90:6803/70197447] [v2:172.21.15.90:6804/70197447,v1:172.21.15.90:6805/70197447] exists,up 81e03104-5d6d-4892-a8c1-6a186048c90c 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/3393795676,v1:172.21.15.129:6801/3393795676] [v2:172.21.15.129:6802/3393795676,v1:172.21.15.129:6803/3393795676] exists,up 0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/1421934201,v1:172.21.15.136:6801/1421934201] [v2:172.21.15.136:6802/1421934201,v1:172.21.15.136:6803/1421934201] exists,up 4b859166-8b57-42a1-852f-93a38ab9102c 2024-05-27T07:45:42.498 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 up in weight 1 up_from 27 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6810/1546698486,v1:172.21.15.90:6811/1546698486] [v2:172.21.15.90:6812/1546698486,v1:172.21.15.90:6813/1546698486] exists,up a92e6906-1cfb-4f68-9ab7-2bc35221478f 2024-05-27T07:45:42.499 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 up in weight 1 up_from 33 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/2737784840,v1:172.21.15.129:6809/2737784840] [v2:172.21.15.129:6810/2737784840,v1:172.21.15.129:6811/2737784840] exists,up a730ea3e-47fa-46e2-b3af-dba7d3ac49c4 2024-05-27T07:45:42.499 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/3317770816,v1:172.21.15.136:6809/3317770816] [v2:172.21.15.136:6810/3317770816,v1:172.21.15.136:6811/3317770816] exists,up 8d6ab904-e641-4ebf-98f8-39396f44f205 2024-05-27T07:45:42.499 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 up in weight 1 up_from 44 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6818/476520683,v1:172.21.15.90:6819/476520683] [v2:172.21.15.90:6820/476520683,v1:172.21.15.90:6821/476520683] exists,up f675d248-884d-4bbb-88ab-6edab1eafcda 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 up in weight 1 up_from 49 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6816/1361444791,v1:172.21.15.129:6817/1361444791] [v2:172.21.15.129:6818/1361444791,v1:172.21.15.129:6819/1361444791] exists,up 22a66106-a43b-4eb3-adf2-5291094db025 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:osd.8 up in weight 1 up_from 55 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/2341317946,v1:172.21.15.136:6817/2341317946] [v2:172.21.15.136:6818/2341317946,v1:172.21.15.136:6819/2341317946] exists,up 4148e784-16f8-48b6-8521-ed7556044566 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:osd.9 up in weight 1 up_from 61 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6826/4197259682,v1:172.21.15.90:6827/4197259682] [v2:172.21.15.90:6828/4197259682,v1:172.21.15.90:6829/4197259682] exists,up 14e386d1-ab82-4e8d-baa4-14d02137a192 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:osd.10 up in weight 1 up_from 66 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6824/95466853,v1:172.21.15.129:6825/95466853] [v2:172.21.15.129:6826/95466853,v1:172.21.15.129:6827/95466853] exists,up 5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:osd.11 up in weight 1 up_from 72 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/2683318661,v1:172.21.15.136:6825/2683318661] [v2:172.21.15.136:6826/2683318661,v1:172.21.15.136:6827/2683318661] exists,up d748f6bf-b26a-4a17-9b1b-51e78f7d5708 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.5 [8,11] 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.10 [2,4] 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.14 [2,4] 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.16 [7,11] 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.1a [8,1] 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.1 [4,9] 2024-05-27T07:45:42.500 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.3 [4,9] 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.1b [5,9] 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.c [9,10] 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.13 [9,8] 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.14 [9,8] 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.1b [5,8] 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/1476836587 expires 2024-05-28T07:30:58.154487+0000 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/3178069601 expires 2024-05-28T07:30:58.154487+0000 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/2742579272 expires 2024-05-28T07:30:58.154487+0000 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/2742579272 expires 2024-05-28T07:30:58.154487+0000 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/410158065 expires 2024-05-28T07:30:23.548910+0000 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2593454412 expires 2024-05-28T07:30:23.548910+0000 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/3534244243 expires 2024-05-28T07:30:23.548910+0000 2024-05-27T07:45:42.501 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2575484267 expires 2024-05-28T07:30:23.548910+0000 2024-05-27T07:45:42.502 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/410158065 expires 2024-05-28T07:30:23.548910+0000 2024-05-27T07:45:42.502 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2626688812 expires 2024-05-28T07:30:58.154487+0000 2024-05-27T07:45:42.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 -- 172.21.15.90:0/2087205674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f75000b2880 msgr2=0x7f75000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:42.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f75000b2880 0x7f75000b4d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7508005950 tx=0x7f75080058e0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 -- 172.21.15.90:0/2087205674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7518106540 msgr2=0x7f751819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7518106540 0x7f751819ca90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f750c003820 tx=0x7f750c0047b0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 -- 172.21.15.90:0/2087205674 shutdown_connections 2024-05-27T07:45:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f75000b2880 0x7f75000b4d90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7518108010 0x7f75181a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7518106540 0x7f751819ca90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 --2- 172.21.15.90:0/2087205674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7518107670 0x7f751819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 -- 172.21.15.90:0/2087205674 >> 172.21.15.90:0/2087205674 conn(0x7f7518101b80 msgr2=0x7f751810a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.498+0000 7f7520800700 1 -- 172.21.15.90:0/2087205674 shutdown_connections 2024-05-27T07:45:42.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:42.499+0000 7f7520800700 1 -- 172.21.15.90:0/2087205674 wait complete. 2024-05-27T07:45:42.813 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:42 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2087205674' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T07:45:42.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:42 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2087205674' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T07:45:42.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:42 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2087205674' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T07:45:43.377 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-27T07:45:43.389 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:45:43.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:43 smithi090 ceph-mon[31429]: pgmap v523: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:43.835 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:43.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:43 smithi136 ceph-mon[35324]: pgmap v523: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:44.062 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:43 smithi129 ceph-mon[35254]: pgmap v523: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:45.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:45 smithi090 ceph-mon[31429]: pgmap v524: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:45.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:45 smithi136 ceph-mon[35324]: pgmap v524: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:46.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:45 smithi129 ceph-mon[35254]: pgmap v524: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:46.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.096+0000 7fb6f5e00700 1 -- 172.21.15.90:0/130272706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f0106ec0 msgr2=0x7fb6f0113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.096+0000 7fb6f5e00700 1 --2- 172.21.15.90:0/130272706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f0106ec0 0x7fb6f0113350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800b0a0 tx=0x7fb6e800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.096+0000 7fb6f5e00700 1 -- 172.21.15.90:0/130272706 shutdown_connections 2024-05-27T07:45:46.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.096+0000 7fb6f5e00700 1 --2- 172.21.15.90:0/130272706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f0106ec0 0x7fb6f0113350 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.096+0000 7fb6f5e00700 1 --2- 172.21.15.90:0/130272706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f010c5b0 0x7fb6f010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.096+0000 7fb6f5e00700 1 --2- 172.21.15.90:0/130272706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f0106570 0x7fb6f0106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.096+0000 7fb6f5e00700 1 -- 172.21.15.90:0/130272706 >> 172.21.15.90:0/130272706 conn(0x7fb6f0102070 msgr2=0x7fb6f01044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:46.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.096+0000 7fb6f5e00700 1 -- 172.21.15.90:0/130272706 shutdown_connections 2024-05-27T07:45:46.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.096+0000 7fb6f5e00700 1 -- 172.21.15.90:0/130272706 wait complete. 2024-05-27T07:45:46.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.097+0000 7fb6f5e00700 1 Processor -- start 2024-05-27T07:45:46.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.097+0000 7fb6f5e00700 1 -- start start 2024-05-27T07:45:46.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.097+0000 7fb6f5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f0106570 0x7fb6f019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.097+0000 7fb6f5e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f0106ec0 0x7fb6f019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6f5e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f010c5b0 0x7fb6f01a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6f5e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01165e0 con 0x7fb6f0106570 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6f5e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0116460 con 0x7fb6f0106ec0 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6f5e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0116760 con 0x7fb6f010c5b0 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6eea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f0106570 0x7fb6f019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6eea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f0106570 0x7fb6f019cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35960/0 (socket says 172.21.15.90:35960) 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6eea00700 1 -- 172.21.15.90:0/3080126396 learned_addr learned my addr 172.21.15.90:0/3080126396 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6e6000700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f0106ec0 0x7fb6f019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6ef400700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f010c5b0 0x7fb6f01a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6eea00700 1 -- 172.21.15.90:0/3080126396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f010c5b0 msgr2=0x7fb6f01a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6eea00700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f010c5b0 0x7fb6f01a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6eea00700 1 -- 172.21.15.90:0/3080126396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f0106ec0 msgr2=0x7fb6f019d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6eea00700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f0106ec0 0x7fb6f019d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.098+0000 7fb6eea00700 1 -- 172.21.15.90:0/3080126396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f01a1e80 con 0x7fb6f0106570 2024-05-27T07:45:46.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.099+0000 7fb6eea00700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f0106570 0x7fb6f019cfb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000d980 tx=0x7fb6e000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:46.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.099+0000 7fb6f4a00700 1 -- 172.21.15.90:0/3080126396 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0017070 con 0x7fb6f0106570 2024-05-27T07:45:46.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.099+0000 7fb6f4a00700 1 -- 172.21.15.90:0/3080126396 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6e0010460 con 0x7fb6f0106570 2024-05-27T07:45:46.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.099+0000 7fb6f5e00700 1 -- 172.21.15.90:0/3080126396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f01a2110 con 0x7fb6f0106570 2024-05-27T07:45:46.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.099+0000 7fb6f4a00700 1 -- 172.21.15.90:0/3080126396 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e000de60 con 0x7fb6f0106570 2024-05-27T07:45:46.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.099+0000 7fb6f5e00700 1 -- 172.21.15.90:0/3080126396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01a9a90 con 0x7fb6f0106570 2024-05-27T07:45:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.100+0000 7fb6f5e00700 1 -- 172.21.15.90:0/3080126396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f00512b0 con 0x7fb6f0106570 2024-05-27T07:45:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.103+0000 7fb6f4a00700 1 -- 172.21.15.90:0/3080126396 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb6e0010610 con 0x7fb6f0106570 2024-05-27T07:45:46.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.104+0000 7fb6f4a00700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb6d40b2880 0x7fb6d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:46.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.104+0000 7fb6f4a00700 1 -- 172.21.15.90:0/3080126396 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fb6e00e2090 con 0x7fb6f0106570 2024-05-27T07:45:46.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.105+0000 7fb6e6000700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb6d40b2880 0x7fb6d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.106+0000 7fb6e6000700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb6d40b2880 0x7fb6d40b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005950 tx=0x7fb6d80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:46.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.107+0000 7fb6f4a00700 1 -- 172.21.15.90:0/3080126396 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb6e00a99c0 con 0x7fb6f0106570 2024-05-27T07:45:46.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.391+0000 7fb6f5e00700 1 -- 172.21.15.90:0/3080126396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6f002e5e0 con 0x7fb6f0106570 2024-05-27T07:45:46.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.392+0000 7fb6f4a00700 1 -- 172.21.15.90:0/3080126396 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11941 (secure 0 0 0) 0x7fb6e00a92c0 con 0x7fb6f0106570 2024-05-27T07:45:46.393 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:45:46.393 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:42:38.788447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24478,34406,34424,34442,34472]},"id":1}]} 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.395+0000 7fb6f5e00700 1 -- 172.21.15.90:0/3080126396 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb6d40b2880 msgr2=0x7fb6d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.395+0000 7fb6f5e00700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb6d40b2880 0x7fb6d40b4d90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8005950 tx=0x7fb6d80058e0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.395+0000 7fb6f5e00700 1 -- 172.21.15.90:0/3080126396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f0106570 msgr2=0x7fb6f019cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.395+0000 7fb6f5e00700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f0106570 0x7fb6f019cfb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000d980 tx=0x7fb6e000dd40 comp rx=0 tx=0).stop 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.395+0000 7fb6f5e00700 1 -- 172.21.15.90:0/3080126396 shutdown_connections 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.395+0000 7fb6f5e00700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb6d40b2880 0x7fb6d40b4d90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.395+0000 7fb6f5e00700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f010c5b0 0x7fb6f01a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.395+0000 7fb6f5e00700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f0106570 0x7fb6f019cfb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.395+0000 7fb6f5e00700 1 --2- 172.21.15.90:0/3080126396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f0106ec0 0x7fb6f019d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.396+0000 7fb6f5e00700 1 -- 172.21.15.90:0/3080126396 >> 172.21.15.90:0/3080126396 conn(0x7fb6f0102070 msgr2=0x7fb6f01047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:46.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.396+0000 7fb6f5e00700 1 -- 172.21.15.90:0/3080126396 shutdown_connections 2024-05-27T07:45:46.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:46.396+0000 7fb6f5e00700 1 -- 172.21.15.90:0/3080126396 wait complete. 2024-05-27T07:45:46.397 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-05-27T07:45:46.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:46 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3080126396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:45:46.900 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:46 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3080126396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:45:47.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:46 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3080126396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:45:47.303 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:45:47.766 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:47.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:47 smithi090 ceph-mon[31429]: pgmap v525: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:47.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:47 smithi136 ceph-mon[35324]: pgmap v525: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:48.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:47 smithi129 ceph-mon[35254]: pgmap v525: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:49.916 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:49 smithi090 ceph-mon[31429]: pgmap v526: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:49.999 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:49 smithi136 ceph-mon[35324]: pgmap v526: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:50.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:49 smithi129 ceph-mon[35254]: pgmap v526: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:45:50.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.158+0000 7f5f6c800700 1 -- 172.21.15.90:0/2991753292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f64103a70 msgr2=0x7f5f64110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.158+0000 7f5f6c800700 1 --2- 172.21.15.90:0/2991753292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f64103a70 0x7f5f64110700 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009990 tx=0x7f5f54009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.158+0000 7f5f6c800700 1 -- 172.21.15.90:0/2991753292 shutdown_connections 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.158+0000 7f5f6c800700 1 --2- 172.21.15.90:0/2991753292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f64110c40 0x7f5f64113060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.158+0000 7f5f6c800700 1 --2- 172.21.15.90:0/2991753292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f64103090 0x7f5f641034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.158+0000 7f5f6c800700 1 --2- 172.21.15.90:0/2991753292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f64103a70 0x7f5f64110700 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.158+0000 7f5f6c800700 1 -- 172.21.15.90:0/2991753292 >> 172.21.15.90:0/2991753292 conn(0x7f5f640fef20 msgr2=0x7f5f64101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:50.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.158+0000 7f5f6c800700 1 -- 172.21.15.90:0/2991753292 shutdown_connections 2024-05-27T07:45:50.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.158+0000 7f5f6c800700 1 -- 172.21.15.90:0/2991753292 wait complete. 2024-05-27T07:45:50.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.159+0000 7f5f6c800700 1 Processor -- start 2024-05-27T07:45:50.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.159+0000 7f5f6c800700 1 -- start start 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f6c800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f64103090 0x7f5f6419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f6c800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f64103a70 0x7f5f6419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f6c800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f64110c40 0x7f5f641a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f6c800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641165a0 con 0x7f5f64103090 2024-05-27T07:45:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f6c800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f64116420 con 0x7f5f64103a70 2024-05-27T07:45:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f6c800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5f64116720 con 0x7f5f64110c40 2024-05-27T07:45:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f69e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f64103090 0x7f5f6419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f69e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f64103090 0x7f5f6419cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56654/0 (socket says 172.21.15.90:56654) 2024-05-27T07:45:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f69e00700 1 -- 172.21.15.90:0/1708605621 learned_addr learned my addr 172.21.15.90:0/1708605621 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f69400700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f64103a70 0x7f5f6419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f69e00700 1 -- 172.21.15.90:0/1708605621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f64110c40 msgr2=0x7f5f641a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f69e00700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f64110c40 0x7f5f641a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f69e00700 1 -- 172.21.15.90:0/1708605621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f64103a70 msgr2=0x7f5f6419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f69e00700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f64103a70 0x7f5f6419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.160+0000 7f5f69e00700 1 -- 172.21.15.90:0/1708605621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f641a1ec0 con 0x7f5f64103090 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.161+0000 7f5f69400700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f64103a70 0x7f5f6419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.161+0000 7f5f69e00700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f64103090 0x7f5f6419cff0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800c920 tx=0x7f5f5800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:50.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.161+0000 7f5f5ea00700 1 -- 172.21.15.90:0/1708605621 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f58007b00 con 0x7f5f64103090 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.161+0000 7f5f5ea00700 1 -- 172.21.15.90:0/1708605621 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5f58004cc0 con 0x7f5f64103090 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.161+0000 7f5f6c800700 1 -- 172.21.15.90:0/1708605621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f641a21b0 con 0x7f5f64103090 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.161+0000 7f5f5ea00700 1 -- 172.21.15.90:0/1708605621 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f580054f0 con 0x7f5f64103090 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.161+0000 7f5f6c800700 1 -- 172.21.15.90:0/1708605621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f641a9ad0 con 0x7f5f64103090 2024-05-27T07:45:50.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.162+0000 7f5f6c800700 1 -- 172.21.15.90:0/1708605621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f640512b0 con 0x7f5f64103090 2024-05-27T07:45:50.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.165+0000 7f5f5ea00700 1 -- 172.21.15.90:0/1708605621 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5f58007ca0 con 0x7f5f64103090 2024-05-27T07:45:50.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.167+0000 7f5f5ea00700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5f4c0b2900 0x7f5f4c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:50.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.167+0000 7f5f5ea00700 1 -- 172.21.15.90:0/1708605621 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f5f580e25b0 con 0x7f5f64103090 2024-05-27T07:45:50.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.167+0000 7f5f69400700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5f4c0b2900 0x7f5f4c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:50.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.168+0000 7f5f69400700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5f4c0b2900 0x7f5f4c0b4e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5f54006010 tx=0x7f5f54005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.169+0000 7f5f5ea00700 1 -- 172.21.15.90:0/1708605621 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5f58014030 con 0x7f5f64103090 2024-05-27T07:45:50.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.467+0000 7f5f6c800700 1 -- 172.21.15.90:0/1708605621 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f6402e5e0 con 0x7f5f64103090 2024-05-27T07:45:50.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.468+0000 7f5f5ea00700 1 -- 172.21.15.90:0/1708605621 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11941 (secure 0 0 0) 0x7f5f580a9f60 con 0x7f5f64103090 2024-05-27T07:45:50.469 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:45:50.469 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:42:38.788447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24478,34406,34424,34442,34472]},"id":1}]} 2024-05-27T07:45:50.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.471+0000 7f5f6c800700 1 -- 172.21.15.90:0/1708605621 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5f4c0b2900 msgr2=0x7f5f4c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.471+0000 7f5f6c800700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5f4c0b2900 0x7f5f4c0b4e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5f54006010 tx=0x7f5f54005960 comp rx=0 tx=0).stop 2024-05-27T07:45:50.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.471+0000 7f5f6c800700 1 -- 172.21.15.90:0/1708605621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f64103090 msgr2=0x7f5f6419cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:50.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.471+0000 7f5f6c800700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f64103090 0x7f5f6419cff0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800c920 tx=0x7f5f5800cce0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.472+0000 7f5f6c800700 1 -- 172.21.15.90:0/1708605621 shutdown_connections 2024-05-27T07:45:50.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.472+0000 7f5f6c800700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f5f4c0b2900 0x7f5f4c0b4e10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.472+0000 7f5f6c800700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f64110c40 0x7f5f641a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.472+0000 7f5f6c800700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f64103090 0x7f5f6419cff0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.472+0000 7f5f6c800700 1 --2- 172.21.15.90:0/1708605621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f64103a70 0x7f5f6419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:50.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.472+0000 7f5f6c800700 1 -- 172.21.15.90:0/1708605621 >> 172.21.15.90:0/1708605621 conn(0x7f5f640fef20 msgr2=0x7f5f64112920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:50.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.472+0000 7f5f6c800700 1 -- 172.21.15.90:0/1708605621 shutdown_connections 2024-05-27T07:45:50.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:50.472+0000 7f5f6c800700 1 -- 172.21.15.90:0/1708605621 wait complete. 2024-05-27T07:45:50.473 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-05-27T07:45:50.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:50 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1708605621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:45:50.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:50 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1708605621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:45:51.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:50 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1708605621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:45:51.425 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:45:51.893 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:51.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:51 smithi090 ceph-mon[31429]: pgmap v527: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:51.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:51 smithi136 ceph-mon[35324]: pgmap v527: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:52.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:51 smithi129 ceph-mon[35254]: pgmap v527: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:53.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:53 smithi090 ceph-mon[31429]: pgmap v528: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.215+0000 7fc244800700 1 -- 172.21.15.90:0/970995902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc23c10e8d0 msgr2=0x7fc23c110db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.215+0000 7fc244800700 1 --2- 172.21.15.90:0/970995902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc23c10e8d0 0x7fc23c110db0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc23800b0a0 tx=0x7fc23800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.215+0000 7fc244800700 1 -- 172.21.15.90:0/970995902 shutdown_connections 2024-05-27T07:45:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.215+0000 7fc244800700 1 --2- 172.21.15.90:0/970995902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc23c10e8d0 0x7fc23c110db0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.215+0000 7fc244800700 1 --2- 172.21.15.90:0/970995902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc23c1054f0 0x7fc23c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.215+0000 7fc244800700 1 --2- 172.21.15.90:0/970995902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c105e40 0x7fc23c10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.215+0000 7fc244800700 1 -- 172.21.15.90:0/970995902 >> 172.21.15.90:0/970995902 conn(0x7fc23c0fedb0 msgr2=0x7fc23c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.216+0000 7fc244800700 1 -- 172.21.15.90:0/970995902 shutdown_connections 2024-05-27T07:45:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.216+0000 7fc244800700 1 -- 172.21.15.90:0/970995902 wait complete. 2024-05-27T07:45:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.217+0000 7fc244800700 1 Processor -- start 2024-05-27T07:45:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.217+0000 7fc244800700 1 -- start start 2024-05-27T07:45:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.217+0000 7fc244800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c1054f0 0x7fc23c1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.217+0000 7fc244800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc23c105e40 0x7fc23c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.217+0000 7fc244800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc23c10e8d0 0x7fc23c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.217+0000 7fc244800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c1142e0 con 0x7fc23c105e40 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.217+0000 7fc244800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c114160 con 0x7fc23c10e8d0 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.217+0000 7fc244800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c114460 con 0x7fc23c1054f0 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc241400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc23c105e40 0x7fc23c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc241400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc23c105e40 0x7fc23c1a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56668/0 (socket says 172.21.15.90:56668) 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc241400700 1 -- 172.21.15.90:0/1974407260 learned_addr learned my addr 172.21.15.90:0/1974407260 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc241e00700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c1054f0 0x7fc23c1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc242800700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc23c10e8d0 0x7fc23c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc241400700 1 -- 172.21.15.90:0/1974407260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c1054f0 msgr2=0x7fc23c1a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc241400700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c1054f0 0x7fc23c1a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc241400700 1 -- 172.21.15.90:0/1974407260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc23c10e8d0 msgr2=0x7fc23c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:54.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc241400700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc23c10e8d0 0x7fc23c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc241400700 1 -- 172.21.15.90:0/1974407260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c19b130 con 0x7fc23c105e40 2024-05-27T07:45:54.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.218+0000 7fc242800700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc23c10e8d0 0x7fc23c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:45:54.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.219+0000 7fc241400700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc23c105e40 0x7fc23c1a0da0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d950 tx=0x7fc22c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:54.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.219+0000 7fc236a00700 1 -- 172.21.15.90:0/1974407260 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c016070 con 0x7fc23c105e40 2024-05-27T07:45:54.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.219+0000 7fc236a00700 1 -- 172.21.15.90:0/1974407260 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc22c010460 con 0x7fc23c105e40 2024-05-27T07:45:54.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.219+0000 7fc244800700 1 -- 172.21.15.90:0/1974407260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c19b3c0 con 0x7fc23c105e40 2024-05-27T07:45:54.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.219+0000 7fc236a00700 1 -- 172.21.15.90:0/1974407260 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c00f620 con 0x7fc23c105e40 2024-05-27T07:45:54.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.219+0000 7fc244800700 1 -- 172.21.15.90:0/1974407260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c1a76f0 con 0x7fc23c105e40 2024-05-27T07:45:54.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.221+0000 7fc244800700 1 -- 172.21.15.90:0/1974407260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc23c0512b0 con 0x7fc23c105e40 2024-05-27T07:45:54.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.224+0000 7fc236a00700 1 -- 172.21.15.90:0/1974407260 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc22c019070 con 0x7fc23c105e40 2024-05-27T07:45:54.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.226+0000 7fc236a00700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc2240b2880 0x7fc2240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:54.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.226+0000 7fc236a00700 1 -- 172.21.15.90:0/1974407260 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc22c0e21a0 con 0x7fc23c105e40 2024-05-27T07:45:54.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.226+0000 7fc241e00700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc2240b2880 0x7fc2240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.227+0000 7fc241e00700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc2240b2880 0x7fc2240b4d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc230005950 tx=0x7fc2300058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.227+0000 7fc236a00700 1 -- 172.21.15.90:0/1974407260 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc22c0a9ad0 con 0x7fc23c105e40 2024-05-27T07:45:54.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:53 smithi136 ceph-mon[35324]: pgmap v528: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:54.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:53 smithi129 ceph-mon[35254]: pgmap v528: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:45:54.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.463+0000 7fc244800700 1 -- 172.21.15.90:0/1974407260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc23c02e5e0 con 0x7fc23c105e40 2024-05-27T07:45:54.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.468+0000 7fc236a00700 1 -- 172.21.15.90:0/1974407260 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24725 (secure 0 0 0) 0x7fc22c0a93d0 con 0x7fc23c105e40 2024-05-27T07:45:54.467 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:45:54.467 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":91,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:43:00.000806+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:45:54.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 -- 172.21.15.90:0/1974407260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc2240b2880 msgr2=0x7fc2240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:54.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc2240b2880 0x7fc2240b4d90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc230005950 tx=0x7fc2300058e0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 -- 172.21.15.90:0/1974407260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc23c105e40 msgr2=0x7fc23c1a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:54.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc23c105e40 0x7fc23c1a0da0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d950 tx=0x7fc22c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:45:54.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 -- 172.21.15.90:0/1974407260 shutdown_connections 2024-05-27T07:45:54.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc2240b2880 0x7fc2240b4d90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc23c10e8d0 0x7fc23c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc23c1054f0 0x7fc23c1a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 --2- 172.21.15.90:0/1974407260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc23c105e40 0x7fc23c1a0da0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:54.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 -- 172.21.15.90:0/1974407260 >> 172.21.15.90:0/1974407260 conn(0x7fc23c0fedb0 msgr2=0x7fc23c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:54.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.471+0000 7fc244800700 1 -- 172.21.15.90:0/1974407260 shutdown_connections 2024-05-27T07:45:54.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:54.472+0000 7fc244800700 1 -- 172.21.15.90:0/1974407260 wait complete. 2024-05-27T07:45:55.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:54 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1974407260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:45:55.215 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:54 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1974407260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:45:55.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:54 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1974407260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:45:55.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T07:41:11.710115+0000', 'modified': '2024-05-27T07:42:38.788447+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34472, 'mds_1': 34442, 'mds_2': 34424, 'mds_3': 34406, 'mds_4': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14556': {'gid': 14556, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6839/767662982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 767662982}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 767662982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14562': {'gid': 14562, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6841/24895501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6840', 'nonce': 24895501}, {'type': 'v1', 'addr': '172.21.15.90:6841', 'nonce': 24895501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/1595668128', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 1595668128}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 1595668128}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/1419480441', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 1419480441}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 1419480441}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6837/3523286296', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 3523286296}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 3523286296}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.136:6839/2113925564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 2113925564}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 2113925564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34406': {'gid': 34406, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 37, 'addr': '172.21.15.129:6833/3131589000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6832', 'nonce': 3131589000}, {'type': 'v1', 'addr': '172.21.15.129:6833', 'nonce': 3131589000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34424': {'gid': 34424, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 34, 'addr': '172.21.15.129:6835/1894603892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 1894603892}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 1894603892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34442': {'gid': 34442, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 30, 'addr': '172.21.15.129:6837/2346696366', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 2346696366}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 2346696366}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34472': {'gid': 34472, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.129:6839/2893900972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 2893900972}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 2893900972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34472, 'qdb_cluster': [24478, 34406, 34424, 34442, 34472]} 2024-05-27T07:45:55.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-27T07:45:55.250 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-05-27T07:45:55.683 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:45:56.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:55 smithi090 ceph-mon[31429]: pgmap v529: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:56.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:55 smithi136 ceph-mon[35324]: pgmap v529: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:56.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:55 smithi129 ceph-mon[35254]: pgmap v529: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:45:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.016+0000 7efdd8a00700 1 -- 172.21.15.90:0/1802331287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd40775b0 msgr2=0x7efdd4114e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.016+0000 7efdd8a00700 1 --2- 172.21.15.90:0/1802331287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd40775b0 0x7efdd4114e00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009990 tx=0x7efdc0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.016+0000 7efdd8a00700 1 -- 172.21.15.90:0/1802331287 shutdown_connections 2024-05-27T07:45:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.016+0000 7efdd8a00700 1 --2- 172.21.15.90:0/1802331287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdd4115340 0x7efdd4117760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.016+0000 7efdd8a00700 1 --2- 172.21.15.90:0/1802331287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4076c60 0x7efdd4077070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.016+0000 7efdd8a00700 1 --2- 172.21.15.90:0/1802331287 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd40775b0 0x7efdd4114e00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.017+0000 7efdd8a00700 1 -- 172.21.15.90:0/1802331287 >> 172.21.15.90:0/1802331287 conn(0x7efdd40ffe90 msgr2=0x7efdd41022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.017+0000 7efdd8a00700 1 -- 172.21.15.90:0/1802331287 shutdown_connections 2024-05-27T07:45:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.017+0000 7efdd8a00700 1 -- 172.21.15.90:0/1802331287 wait complete. 2024-05-27T07:45:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.018+0000 7efdd8a00700 1 Processor -- start 2024-05-27T07:45:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.018+0000 7efdd8a00700 1 -- start start 2024-05-27T07:45:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd4076c60 0x7efdd41a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd8a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdd40775b0 0x7efdd41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd8a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4115340 0x7efdd41a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd8a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efdd411a940 con 0x7efdd4076c60 2024-05-27T07:45:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd8a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efdd411a7c0 con 0x7efdd4115340 2024-05-27T07:45:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd8a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efdd411aac0 con 0x7efdd40775b0 2024-05-27T07:45:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd4076c60 0x7efdd41a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd4076c60 0x7efdd41a1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56682/0 (socket says 172.21.15.90:56682) 2024-05-27T07:45:58.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd1e00700 1 -- 172.21.15.90:0/1082686870 learned_addr learned my addr 172.21.15.90:0/1082686870 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:58.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd1400700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdd40775b0 0x7efdd41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.019+0000 7efdd2800700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4115340 0x7efdd41a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.020+0000 7efdd1e00700 1 -- 172.21.15.90:0/1082686870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdd40775b0 msgr2=0x7efdd41a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.020+0000 7efdd1e00700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdd40775b0 0x7efdd41a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.020+0000 7efdd1e00700 1 -- 172.21.15.90:0/1082686870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4115340 msgr2=0x7efdd41a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.020+0000 7efdd1e00700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4115340 0x7efdd41a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.020+0000 7efdd1e00700 1 -- 172.21.15.90:0/1082686870 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd41a6190 con 0x7efdd4076c60 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.020+0000 7efdd1e00700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd4076c60 0x7efdd41a1370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efdc400c9b0 tx=0x7efdc400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.020+0000 7efdcaa00700 1 -- 172.21.15.90:0/1082686870 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4007b00 con 0x7efdd4076c60 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.021+0000 7efdcaa00700 1 -- 172.21.15.90:0/1082686870 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdc4007ca0 con 0x7efdd4076c60 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.021+0000 7efdd8a00700 1 -- 172.21.15.90:0/1082686870 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd41a6480 con 0x7efdd4076c60 2024-05-27T07:45:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.021+0000 7efdd8a00700 1 -- 172.21.15.90:0/1082686870 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd41ade50 con 0x7efdd4076c60 2024-05-27T07:45:58.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.021+0000 7efdcaa00700 1 -- 172.21.15.90:0/1082686870 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4018660 con 0x7efdd4076c60 2024-05-27T07:45:58.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.025+0000 7efdcaa00700 1 -- 172.21.15.90:0/1082686870 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efdc40188c0 con 0x7efdd4076c60 2024-05-27T07:45:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:57 smithi090 ceph-mon[31429]: pgmap v530: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:58.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.026+0000 7efde75a8b80 1 Processor -- start 2024-05-27T07:45:58.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.027+0000 7efde75a8b80 1 -- start start 2024-05-27T07:45:58.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.027+0000 7efdcaa00700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efdb80b2900 0x7efdb80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.027+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.027+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.027+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a0d60 0x5565759a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.027+0000 7efde75a8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565758cedf0 con 0x5565759a6ce0 2024-05-27T07:45:58.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.027+0000 7efde75a8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5565758be990 con 0x556575821df0 2024-05-27T07:45:58.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.027+0000 7efde75a8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5565758c29d0 con 0x5565759a0d60 2024-05-27T07:45:58.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdcaa00700 1 -- 172.21.15.90:0/1082686870 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7efdc40e2550 con 0x7efdd4076c60 2024-05-27T07:45:58.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd2800700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efdb80b2900 0x7efdb80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56690/0 (socket says 172.21.15.90:56690) 2024-05-27T07:45:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd1400700 1 -- 172.21.15.90:0/4091459673 learned_addr learned my addr 172.21.15.90:0/4091459673 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd1e00700 1 --2- 172.21.15.90:0/4091459673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd1400700 1 -- 172.21.15.90:0/4091459673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a0d60 msgr2=0x5565759a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd1400700 1 --2- 172.21.15.90:0/4091459673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a0d60 0x5565759a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd1400700 1 -- 172.21.15.90:0/4091459673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 msgr2=0x5565759a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd1400700 1 --2- 172.21.15.90:0/4091459673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd1400700 1 -- 172.21.15.90:0/4091459673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565758873d0 con 0x5565759a6ce0 2024-05-27T07:45:58.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.028+0000 7efdd1e00700 1 --2- 172.21.15.90:0/4091459673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:58.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.029+0000 7efdd2800700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efdb80b2900 0x7efdb80b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efdcc006130 tx=0x7efdcc0088c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:58.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.029+0000 7efdd1400700 1 --2- 172.21.15.90:0/4091459673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a0820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efdc0015000 tx=0x7efdc0003720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350958e39d30d5f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:58.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.029+0000 7efdbf400700 1 -- 172.21.15.90:0/4091459673 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0027070 con 0x5565759a6ce0 2024-05-27T07:45:58.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.029+0000 7efdbf400700 1 -- 172.21.15.90:0/4091459673 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdc0005040 con 0x5565759a6ce0 2024-05-27T07:45:58.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.029+0000 7efdbf400700 1 -- 172.21.15.90:0/4091459673 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc000ba30 con 0x5565759a6ce0 2024-05-27T07:45:58.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.030+0000 7efde75a8b80 1 -- 172.21.15.90:0/4091459673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 msgr2=0x5565759a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.030+0000 7efde75a8b80 1 --2- 172.21.15.90:0/4091459673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a0820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efdc0015000 tx=0x7efdc0003720 comp rx=0 tx=0).stop 2024-05-27T07:45:58.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.031+0000 7efde75a8b80 1 -- 172.21.15.90:0/4091459673 shutdown_connections 2024-05-27T07:45:58.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.031+0000 7efde75a8b80 1 --2- 172.21.15.90:0/4091459673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a0d60 0x5565759a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.031+0000 7efde75a8b80 1 --2- 172.21.15.90:0/4091459673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.031+0000 7efde75a8b80 1 --2- 172.21.15.90:0/4091459673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a0820 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.031+0000 7efde75a8b80 1 -- 172.21.15.90:0/4091459673 >> 172.21.15.90:0/4091459673 conn(0x556575881f70 msgr2=0x55657599b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:58.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.031+0000 7efde75a8b80 1 -- 172.21.15.90:0/4091459673 shutdown_connections 2024-05-27T07:45:58.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.031+0000 7efde75a8b80 1 -- 172.21.15.90:0/4091459673 wait complete. 2024-05-27T07:45:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.031+0000 7efde75a8b80 1 -- start start 2024-05-27T07:45:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.032+0000 7efde75a8b80 1 Processor -- start 2024-05-27T07:45:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.032+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759aa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.032+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759aa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.033+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a6ce0 0x5565759aae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.033+0000 7efde75a8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565757c4f50 con 0x5565759a0d60 2024-05-27T07:45:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.033+0000 7efde75a8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55657585f710 con 0x556575821df0 2024-05-27T07:45:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.033+0000 7efde75a8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5565759ad750 con 0x5565759a6ce0 2024-05-27T07:45:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.033+0000 7efde75a8b80 10 client.0 _ll_register_callbacks cb 0x556575868c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.033+0000 7efdd1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.033+0000 7efdd1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759aa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56706/0 (socket says 172.21.15.90:56706) 2024-05-27T07:45:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.033+0000 7efdd1400700 1 -- 172.21.15.90:0/3050087602 learned_addr learned my addr 172.21.15.90:0/3050087602 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.033+0000 7efdd1e00700 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.033+0000 7efdd2800700 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a6ce0 0x5565759aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.034+0000 7efdd1400700 1 -- 172.21.15.90:0/3050087602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a6ce0 msgr2=0x5565759aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.034+0000 7efdd1400700 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a6ce0 0x5565759aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.034+0000 7efdd1400700 1 -- 172.21.15.90:0/3050087602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 msgr2=0x5565759aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.034+0000 7efdd1400700 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759aa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.034+0000 7efdd1400700 1 -- 172.21.15.90:0/3050087602 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55657580e880 con 0x5565759a0d60 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.034+0000 7efdd1400700 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759aa940 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efdc000b5c0 tx=0x7efdc0005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f7ea06826e52e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.034+0000 7efdb3e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.034+0000 7efdb3e00700 1 -- 172.21.15.90:0/3050087602 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0027040 con 0x5565759a0d60 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.035+0000 7efde75a8b80 1 -- 172.21.15.90:0/3050087602 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc0005040 con 0x5565759a0d60 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.035+0000 7efdb3e00700 1 -- 172.21.15.90:0/3050087602 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7efdc000b7d0 con 0x5565759a0d60 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.035+0000 7efdb3e00700 1 -- 172.21.15.90:0/3050087602 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efdc0039050 con 0x5565759a0d60 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.035+0000 7efde75a8b80 10 client.15159 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.035+0000 7efde75a8b80 1 -- 172.21.15.90:0/3050087602 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55657588f010 con 0x5565759a0d60 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.035+0000 7efdb3e00700 1 -- 172.21.15.90:0/3050087602 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdc0005040 con 0x5565759a0d60 2024-05-27T07:45:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.036+0000 7efdb3e00700 1 -- 172.21.15.90:0/3050087602 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efdc000f800 con 0x5565759a0d60 2024-05-27T07:45:58.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.036+0000 7efde75a8b80 10 client.15159 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:58.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.036+0000 7efde75a8b80 10 client.15159 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T07:45:58.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.036+0000 7efde75a8b80 20 client.15159 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:58.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.036+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5565759ba080 0x5565759bb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.036+0000 7efde75a8b80 4 client.15159 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:45:58.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.036+0000 7efde75a8b80 1 -- 172.21.15.90:0/3050087602 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565758d0d40 con 0x5565759ba080 2024-05-27T07:45:58.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.037+0000 7efdd1e00700 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5565759ba080 0x5565759bb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.038+0000 7efdd1e00700 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5565759ba080 0x5565759bb4f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=154dff6b92402f14 server_cookie=68e1d711e2581508 in_seq=0 out_seq=0 2024-05-27T07:45:58.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.038+0000 7efdb3e00700 10 client.15159 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:45:58.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.040+0000 7efdb3e00700 1 -- 172.21.15.90:0/3050087602 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5565758d0d40 con 0x5565759ba080 2024-05-27T07:45:58.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.040+0000 7efdb3e00700 10 client.15159 handle_command_reply: tid=0 2024-05-27T07:45:58.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.040+0000 7efde75a8b80 1 client.15159 shutdown 2024-05-27T07:45:58.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 1 -- 172.21.15.90:0/3050087602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 msgr2=0x5565759aa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759aa940 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efdc000b5c0 tx=0x7efdc0005420 comp rx=0 tx=0).stop 2024-05-27T07:45:58.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 1 -- 172.21.15.90:0/3050087602 shutdown_connections 2024-05-27T07:45:58.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5565759ba080 0x5565759bb4f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a6ce0 0x5565759aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3050087602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759aa940 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 1 -- 172.21.15.90:0/3050087602 >> 172.21.15.90:0/3050087602 conn(0x556575881f70 msgr2=0x55657599b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:58.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efdb3e00700 0 client.15159 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:45:58.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 1 -- 172.21.15.90:0/3050087602 shutdown_connections 2024-05-27T07:45:58.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 1 -- 172.21.15.90:0/3050087602 wait complete. 2024-05-27T07:45:58.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.041+0000 7efde75a8b80 20 client.15159 trim_cache size 0 max 16384 2024-05-27T07:45:58.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.064+0000 7efde75a8b80 1 Processor -- start 2024-05-27T07:45:58.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.064+0000 7efde75a8b80 1 -- start start 2024-05-27T07:45:58.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759a0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a0d60 0x5565759b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efde75a8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565759b9830 con 0x5565759a6ce0 2024-05-27T07:45:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efde75a8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5565759b9f00 con 0x556575821df0 2024-05-27T07:45:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efde75a8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5565759ba5d0 con 0x5565759a0d60 2024-05-27T07:45:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efdd2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efdd2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56710/0 (socket says 172.21.15.90:56710) 2024-05-27T07:45:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efdd2800700 1 -- 172.21.15.90:0/125353154 learned_addr learned my addr 172.21.15.90:0/125353154 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efdd1e00700 1 --2- 172.21.15.90:0/125353154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.065+0000 7efdd1400700 1 --2- 172.21.15.90:0/125353154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a0d60 0x5565759b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.066+0000 7efdd2800700 1 -- 172.21.15.90:0/125353154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a0d60 msgr2=0x5565759b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.066+0000 7efdd2800700 1 --2- 172.21.15.90:0/125353154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a0d60 0x5565759b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.066+0000 7efdd2800700 1 -- 172.21.15.90:0/125353154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 msgr2=0x5565759a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.066+0000 7efdd2800700 1 --2- 172.21.15.90:0/125353154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.066+0000 7efdd2800700 1 -- 172.21.15.90:0/125353154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55657599cfe0 con 0x5565759a6ce0 2024-05-27T07:45:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.066+0000 7efdd1e00700 1 --2- 172.21.15.90:0/125353154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:45:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.066+0000 7efdd2800700 1 --2- 172.21.15.90:0/125353154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a9360 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdcc01c6a0 tx=0x7efdcc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a41e5c20fa075ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.066+0000 7efdbea00700 1 -- 172.21.15.90:0/125353154 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc01c320 con 0x5565759a6ce0 2024-05-27T07:45:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.066+0000 7efdbea00700 1 -- 172.21.15.90:0/125353154 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdcc0258d0 con 0x5565759a6ce0 2024-05-27T07:45:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.066+0000 7efdbea00700 1 -- 172.21.15.90:0/125353154 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc01b890 con 0x5565759a6ce0 2024-05-27T07:45:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.067+0000 7efde75a8b80 1 -- 172.21.15.90:0/125353154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 msgr2=0x5565759a9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.067+0000 7efde75a8b80 1 --2- 172.21.15.90:0/125353154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a9360 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdcc01c6a0 tx=0x7efdcc013000 comp rx=0 tx=0).stop 2024-05-27T07:45:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.067+0000 7efde75a8b80 1 -- 172.21.15.90:0/125353154 shutdown_connections 2024-05-27T07:45:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.067+0000 7efde75a8b80 1 --2- 172.21.15.90:0/125353154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a6ce0 0x5565759a9360 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.067+0000 7efde75a8b80 1 --2- 172.21.15.90:0/125353154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556575821df0 0x5565759a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.067+0000 7efde75a8b80 1 --2- 172.21.15.90:0/125353154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5565759a0d60 0x5565759b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.067+0000 7efde75a8b80 1 -- 172.21.15.90:0/125353154 >> 172.21.15.90:0/125353154 conn(0x556575881f70 msgr2=0x5565759aa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.067+0000 7efde75a8b80 1 -- 172.21.15.90:0/125353154 shutdown_connections 2024-05-27T07:45:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.067+0000 7efde75a8b80 1 -- 172.21.15.90:0/125353154 wait complete. 2024-05-27T07:45:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.068+0000 7efde75a8b80 1 -- start start 2024-05-27T07:45:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.068+0000 7efde75a8b80 1 Processor -- start 2024-05-27T07:45:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556575821df0 0x5565759a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efde75a8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5565759a6ce0 0x5565759b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efde75a8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565759bb370 con 0x5565759a0d60 2024-05-27T07:45:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efde75a8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5565759baca0 con 0x5565759a6ce0 2024-05-27T07:45:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efde75a8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5565759bba40 con 0x556575821df0 2024-05-27T07:45:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efde75a8b80 10 client.0 _ll_register_callbacks cb 0x556575868c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:45:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efdd1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efdd1400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759a7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56718/0 (socket says 172.21.15.90:56718) 2024-05-27T07:45:58.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efdd1400700 1 -- 172.21.15.90:0/3207190442 learned_addr learned my addr 172.21.15.90:0/3207190442 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:45:58.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efdd1e00700 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556575821df0 0x5565759a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efdd1400700 1 -- 172.21.15.90:0/3207190442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556575821df0 msgr2=0x5565759a76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efdd1400700 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556575821df0 0x5565759a76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efdd1400700 1 -- 172.21.15.90:0/3207190442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5565759a6ce0 msgr2=0x5565759b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:45:58.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.069+0000 7efdd1400700 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5565759a6ce0 0x5565759b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.070+0000 7efdd1400700 1 -- 172.21.15.90:0/3207190442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565759a8120 con 0x5565759a0d60 2024-05-27T07:45:58.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.070+0000 7efdd1400700 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759a7be0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efdc000fc50 tx=0x7efdc000b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50432b44928fe83a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:45:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.070+0000 7efdb3e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:45:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.070+0000 7efdb3e00700 1 -- 172.21.15.90:0/3207190442 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0027040 con 0x5565759a0d60 2024-05-27T07:45:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.070+0000 7efdb3e00700 1 -- 172.21.15.90:0/3207190442 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7efdc000f460 con 0x5565759a0d60 2024-05-27T07:45:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.070+0000 7efde75a8b80 1 -- 172.21.15.90:0/3207190442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdcc0258d0 con 0x5565759a0d60 2024-05-27T07:45:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.070+0000 7efdb3e00700 1 -- 172.21.15.90:0/3207190442 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efdc000f660 con 0x5565759a0d60 2024-05-27T07:45:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.070+0000 7efde75a8b80 10 client.15165 fetch_fsmap learned FSMap version 29 2024-05-27T07:45:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.070+0000 7efde75a8b80 1 -- 172.21.15.90:0/3207190442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556575a259d0 con 0x5565759a0d60 2024-05-27T07:45:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.071+0000 7efdb3e00700 1 -- 172.21.15.90:0/3207190442 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdcc0258d0 con 0x5565759a0d60 2024-05-27T07:45:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.071+0000 7efdb3e00700 1 -- 172.21.15.90:0/3207190442 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efdc000b860 con 0x5565759a0d60 2024-05-27T07:45:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.071+0000 7efde75a8b80 10 client.15165 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:45:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.071+0000 7efde75a8b80 10 client.15165 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T07:45:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.071+0000 7efde75a8b80 20 client.15165 populate_metadata read hostname 'smithi090' 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.071+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556575a27e80 0x556575a292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.071+0000 7efde75a8b80 4 client.15165 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.071+0000 7efde75a8b80 1 -- 172.21.15.90:0/3207190442 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "status"}) -- 0x5565758ca520 con 0x556575a27e80 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.072+0000 7efdd1e00700 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556575a27e80 0x556575a292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.073+0000 7efdd1e00700 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556575a27e80 0x556575a292f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d0724648bb051fa server_cookie=22d878325814fb7e in_seq=0 out_seq=0 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.073+0000 7efdb3e00700 10 client.15165 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.073+0000 7efdb3e00700 1 -- 172.21.15.90:0/3207190442 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5565758ca520 con 0x556575a27e80 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.073+0000 7efdb3e00700 10 client.15165 handle_command_reply: tid=0 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.073+0000 7efde75a8b80 1 client.15165 shutdown 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.073+0000 7efde75a8b80 1 -- 172.21.15.90:0/3207190442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 msgr2=0x5565759a7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.073+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759a7be0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efdc000fc50 tx=0x7efdc000b680 comp rx=0 tx=0).stop 2024-05-27T07:45:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.074+0000 7efde75a8b80 1 -- 172.21.15.90:0/3207190442 shutdown_connections 2024-05-27T07:45:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.074+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556575a27e80 0x556575a292f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.074+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5565759a6ce0 0x5565759b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.074+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556575821df0 0x5565759a76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.074+0000 7efde75a8b80 1 --2- 172.21.15.90:0/3207190442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565759a0d60 0x5565759a7be0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.074+0000 7efde75a8b80 1 -- 172.21.15.90:0/3207190442 >> 172.21.15.90:0/3207190442 conn(0x556575881f70 msgr2=0x5565759a5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.074+0000 7efdb3e00700 0 client.15165 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:45:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.074+0000 7efde75a8b80 1 -- 172.21.15.90:0/3207190442 shutdown_connections 2024-05-27T07:45:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.074+0000 7efde75a8b80 1 -- 172.21.15.90:0/3207190442 wait complete. 2024-05-27T07:45:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.074+0000 7efde75a8b80 20 client.15165 trim_cache size 0 max 16384 2024-05-27T07:45:58.081 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:45:58.081 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:45:58.081 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 4, 2024-05-27T07:45:58.081 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24478, 2024-05-27T07:45:58.081 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:45:58.081 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:45:58.082 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:45:58.082 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 204.18109397200001, 2024-05-27T07:45:58.082 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-05-27T07:45:58.082 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 91, 2024-05-27T07:45:58.082 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:45:58.082 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 318.08503364000001 2024-05-27T07:45:58.082 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:45:58.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 -- 172.21.15.90:0/1082686870 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efdb80b2900 msgr2=0x7efdb80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efdb80b2900 0x7efdb80b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efdcc006130 tx=0x7efdcc0088c0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 -- 172.21.15.90:0/1082686870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd4076c60 msgr2=0x7efdd41a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:45:58.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd4076c60 0x7efdd41a1370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efdc400c9b0 tx=0x7efdc400cd70 comp rx=0 tx=0).stop 2024-05-27T07:45:58.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 -- 172.21.15.90:0/1082686870 shutdown_connections 2024-05-27T07:45:58.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7efdb80b2900 0x7efdb80b4e10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4115340 0x7efdd41a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd4076c60 0x7efdd41a1370 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 --2- 172.21.15.90:0/1082686870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efdd40775b0 0x7efdd41a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:45:58.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 -- 172.21.15.90:0/1082686870 >> 172.21.15.90:0/1082686870 conn(0x7efdd40ffe90 msgr2=0x7efdd41007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:45:58.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 -- 172.21.15.90:0/1082686870 shutdown_connections 2024-05-27T07:45:58.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:45:58.075+0000 7efdd8a00700 1 -- 172.21.15.90:0/1082686870 wait complete. 2024-05-27T07:45:58.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:57 smithi136 ceph-mon[35324]: pgmap v530: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:58.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:57 smithi129 ceph-mon[35254]: pgmap v530: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:45:59.221 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.k status 2024-05-27T07:45:59.674 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:00.081 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:45:59 smithi090 ceph-mon[31429]: pgmap v531: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:46:00.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:45:59 smithi136 ceph-mon[35324]: pgmap v531: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:46:00.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:45:59 smithi129 ceph-mon[35254]: pgmap v531: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:46:01.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:01 smithi090 ceph-mon[31429]: pgmap v532: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:46:02.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:01 smithi136 ceph-mon[35324]: pgmap v532: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:46:02.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:01 smithi129 ceph-mon[35254]: pgmap v532: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:46:02.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.373+0000 7fc5eac00700 1 -- 172.21.15.90:0/3019190873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5e4110c00 msgr2=0x7fc5e4113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:02.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.373+0000 7fc5eac00700 1 --2- 172.21.15.90:0/3019190873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5e4110c00 0x7fc5e4113020 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00b0a0 tx=0x7fc5dc00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.373+0000 7fc5eac00700 1 -- 172.21.15.90:0/3019190873 shutdown_connections 2024-05-27T07:46:02.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.373+0000 7fc5eac00700 1 --2- 172.21.15.90:0/3019190873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5e4110c00 0x7fc5e4113020 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.373+0000 7fc5eac00700 1 --2- 172.21.15.90:0/3019190873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e4076be0 0x7fc5e4076ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.373+0000 7fc5eac00700 1 --2- 172.21.15.90:0/3019190873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5e4077530 0x7fc5e41106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.373+0000 7fc5eac00700 1 -- 172.21.15.90:0/3019190873 >> 172.21.15.90:0/3019190873 conn(0x7fc5e40ffec0 msgr2=0x7fc5e4102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:02.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.373+0000 7fc5eac00700 1 -- 172.21.15.90:0/3019190873 shutdown_connections 2024-05-27T07:46:02.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.373+0000 7fc5eac00700 1 -- 172.21.15.90:0/3019190873 wait complete. 2024-05-27T07:46:02.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.374+0000 7fc5eac00700 1 Processor -- start 2024-05-27T07:46:02.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.374+0000 7fc5eac00700 1 -- start start 2024-05-27T07:46:02.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.375+0000 7fc5eac00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5e4076be0 0x7fc5e419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.375+0000 7fc5eac00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e4077530 0x7fc5e419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.375+0000 7fc5eac00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5e4110c00 0x7fc5e41a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.375+0000 7fc5eac00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4116590 con 0x7fc5e4110c00 2024-05-27T07:46:02.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.375+0000 7fc5eac00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4116410 con 0x7fc5e4077530 2024-05-27T07:46:02.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.375+0000 7fc5eac00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e4116710 con 0x7fc5e4076be0 2024-05-27T07:46:02.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.375+0000 7fc5e8c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5e4110c00 0x7fc5e41a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.375+0000 7fc5e8c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5e4110c00 0x7fc5e41a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49070/0 (socket says 172.21.15.90:49070) 2024-05-27T07:46:02.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.375+0000 7fc5e8c00700 1 -- 172.21.15.90:0/1825915168 learned_addr learned my addr 172.21.15.90:0/1825915168 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:02.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.376+0000 7fc5e3e00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5e4076be0 0x7fc5e419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.376+0000 7fc5e3400700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e4077530 0x7fc5e419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.376+0000 7fc5e8c00700 1 -- 172.21.15.90:0/1825915168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5e4076be0 msgr2=0x7fc5e419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:02.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.376+0000 7fc5e8c00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5e4076be0 0x7fc5e419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.376+0000 7fc5e8c00700 1 -- 172.21.15.90:0/1825915168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e4077530 msgr2=0x7fc5e419d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:02.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.376+0000 7fc5e8c00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e4077530 0x7fc5e419d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.376+0000 7fc5e8c00700 1 -- 172.21.15.90:0/1825915168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e41a1e30 con 0x7fc5e4110c00 2024-05-27T07:46:02.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.376+0000 7fc5e8c00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5e4110c00 0x7fc5e41a1790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00b070 tx=0x7fc5dc004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:02.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.377+0000 7fc5e0c00700 1 -- 172.21.15.90:0/1825915168 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc021070 con 0x7fc5e4110c00 2024-05-27T07:46:02.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.377+0000 7fc5e0c00700 1 -- 172.21.15.90:0/1825915168 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5dc007c00 con 0x7fc5e4110c00 2024-05-27T07:46:02.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.377+0000 7fc5eac00700 1 -- 172.21.15.90:0/1825915168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e41a20c0 con 0x7fc5e4110c00 2024-05-27T07:46:02.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.377+0000 7fc5e0c00700 1 -- 172.21.15.90:0/1825915168 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc004860 con 0x7fc5e4110c00 2024-05-27T07:46:02.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.377+0000 7fc5eac00700 1 -- 172.21.15.90:0/1825915168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e41a2550 con 0x7fc5e4110c00 2024-05-27T07:46:02.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.381+0000 7fc5e0c00700 1 -- 172.21.15.90:0/1825915168 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc5dc004020 con 0x7fc5e4110c00 2024-05-27T07:46:02.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.382+0000 7fc5f96aeb80 1 Processor -- start 2024-05-27T07:46:02.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.382+0000 7fc5f96aeb80 1 -- start start 2024-05-27T07:46:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.382+0000 7fc5e0c00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5c80b2880 0x7fc5c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.382+0000 7fc5e0c00700 1 -- 172.21.15.90:0/1825915168 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc5dc00f020 con 0x7fc5e4110c00 2024-05-27T07:46:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5e3400700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5c80b2880 0x7fc5c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f040f9a0 0x55e0f04a22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f0323df0 0x55e0f04a2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 0x55e0f04af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e0f03d0df0 con 0x55e0f0323df0 2024-05-27T07:46:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e0f03c0990 con 0x55e0f040f9a0 2024-05-27T07:46:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e0f03c49d0 con 0x55e0f04a2e20 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5e3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f0323df0 0x55e0f04a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5e3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f0323df0 0x55e0f04a2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49086/0 (socket says 172.21.15.90:49086) 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5e3400700 1 -- 172.21.15.90:0/1926844793 learned_addr learned my addr 172.21.15.90:0/1926844793 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5e3e00700 1 --2- 172.21.15.90:0/1926844793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f040f9a0 0x55e0f04a22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.383+0000 7fc5e8c00700 1 --2- 172.21.15.90:0/1926844793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 0x55e0f04af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.384+0000 7fc5e3400700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5c80b2880 0x7fc5c80b4d90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0013510 tx=0x7fc5d000bf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.384+0000 7fc5e3400700 1 -- 172.21.15.90:0/1926844793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 msgr2=0x55e0f04af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.384+0000 7fc5e3400700 1 --2- 172.21.15.90:0/1926844793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 0x55e0f04af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.384+0000 7fc5e3400700 1 -- 172.21.15.90:0/1926844793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f040f9a0 msgr2=0x55e0f04a22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.384+0000 7fc5e3400700 1 --2- 172.21.15.90:0/1926844793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f040f9a0 0x55e0f04a22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.384+0000 7fc5e3400700 1 -- 172.21.15.90:0/1926844793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f03893d0 con 0x55e0f0323df0 2024-05-27T07:46:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.384+0000 7fc5e3400700 1 --2- 172.21.15.90:0/1926844793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f0323df0 0x55e0f04a2820 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000ba80 tx=0x7fc5d001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aed3a11ad4d705 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.384+0000 7fc5d9600700 1 -- 172.21.15.90:0/1926844793 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d001f440 con 0x55e0f0323df0 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.385+0000 7fc5d9600700 1 -- 172.21.15.90:0/1926844793 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5d001fa70 con 0x55e0f0323df0 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.385+0000 7fc5d9600700 1 -- 172.21.15.90:0/1926844793 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d001e870 con 0x55e0f0323df0 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.385+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/1926844793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f0323df0 msgr2=0x55e0f04a2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.385+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/1926844793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f0323df0 0x55e0f04a2820 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000ba80 tx=0x7fc5d001d000 comp rx=0 tx=0).stop 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.386+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/1926844793 shutdown_connections 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.386+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/1926844793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 0x55e0f04af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.386+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/1926844793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f040f9a0 0x55e0f04a22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.386+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/1926844793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f0323df0 0x55e0f04a2820 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.386+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/1926844793 >> 172.21.15.90:0/1926844793 conn(0x55e0f03bd340 msgr2=0x55e0f049e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.386+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/1926844793 shutdown_connections 2024-05-27T07:46:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.386+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/1926844793 wait complete. 2024-05-27T07:46:02.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.386+0000 7fc5f96aeb80 1 -- start start 2024-05-27T07:46:02.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.387+0000 7fc5f96aeb80 1 Processor -- start 2024-05-27T07:46:02.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.387+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 0x55e0f04ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.387+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f04acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.387+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f04a2e20 0x55e0f04ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.387+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e0f02c6f50 con 0x55e0f040f9a0 2024-05-27T07:46:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.387+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e0f0361710 con 0x55e0f04a2e20 2024-05-27T07:46:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.387+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e0f04afad0 con 0x55e0f0323df0 2024-05-27T07:46:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.387+0000 7fc5f96aeb80 10 client.0 _ll_register_callbacks cb 0x55e0f036ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.387+0000 7fc5e3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f04acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5e3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f04acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49098/0 (socket says 172.21.15.90:49098) 2024-05-27T07:46:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5e3400700 1 -- 172.21.15.90:0/2267984531 learned_addr learned my addr 172.21.15.90:0/2267984531 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5e3e00700 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 0x55e0f04ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5e8c00700 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f04a2e20 0x55e0f04ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5e3400700 1 -- 172.21.15.90:0/2267984531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 msgr2=0x55e0f04ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5e3400700 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 0x55e0f04ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5e3400700 1 -- 172.21.15.90:0/2267984531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f04a2e20 msgr2=0x55e0f04ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5e3400700 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f04a2e20 0x55e0f04ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5e3400700 1 -- 172.21.15.90:0/2267984531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f0310880 con 0x55e0f040f9a0 2024-05-27T07:46:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5e3400700 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f04acb10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0000c00 tx=0x7fc5d0011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd39c91f04a2735f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.388+0000 7fc5ce000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.389+0000 7fc5ce000700 1 -- 172.21.15.90:0/2267984531 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0036060 con 0x55e0f040f9a0 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.389+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2267984531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d001fa70 con 0x55e0f040f9a0 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.389+0000 7fc5ce000700 1 -- 172.21.15.90:0/2267984531 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc5d001e510 con 0x55e0f040f9a0 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.389+0000 7fc5ce000700 1 -- 172.21.15.90:0/2267984531 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc5d003d050 con 0x55e0f040f9a0 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.389+0000 7fc5f96aeb80 10 client.15177 fetch_fsmap learned FSMap version 29 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.389+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2267984531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0f0391010 con 0x55e0f040f9a0 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.389+0000 7fc5ce000700 1 -- 172.21.15.90:0/2267984531 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5d001fa70 con 0x55e0f040f9a0 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.390+0000 7fc5ce000700 1 -- 172.21.15.90:0/2267984531 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc5d00274f0 con 0x55e0f040f9a0 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.390+0000 7fc5f96aeb80 10 client.15177 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.390+0000 7fc5f96aeb80 10 client.15177 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.390+0000 7fc5f96aeb80 20 client.15177 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.390+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e0f04bc9e0 0x55e0f04bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.390+0000 7fc5f96aeb80 4 client.15177 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.390+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2267984531 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0f03d2d40 con 0x55e0f04bc9e0 2024-05-27T07:46:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.391+0000 7fc5e3e00700 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e0f04bc9e0 0x55e0f04bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.392+0000 7fc5e3e00700 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e0f04bc9e0 0x55e0f04bde50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4f7b53cb158871ce server_cookie=8c2e9ee8e0cd85b0 in_seq=0 out_seq=0 2024-05-27T07:46:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.392+0000 7fc5ce000700 10 client.15177 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:46:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5ce000700 1 -- 172.21.15.90:0/2267984531 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e0f03d2d40 con 0x55e0f04bc9e0 2024-05-27T07:46:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5ce000700 10 client.15177 handle_command_reply: tid=0 2024-05-27T07:46:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5f96aeb80 1 client.15177 shutdown 2024-05-27T07:46:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2267984531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 msgr2=0x55e0f04acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f04acb10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0000c00 tx=0x7fc5d0011f40 comp rx=0 tx=0).stop 2024-05-27T07:46:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2267984531 shutdown_connections 2024-05-27T07:46:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e0f04bc9e0 0x55e0f04bde50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f04a2e20 0x55e0f04ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 0x55e0f04ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2267984531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f04acb10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5ce000700 0 client.15177 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:46:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.394+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2267984531 >> 172.21.15.90:0/2267984531 conn(0x55e0f03bd340 msgr2=0x55e0f04a22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.395+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2267984531 shutdown_connections 2024-05-27T07:46:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.395+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2267984531 wait complete. 2024-05-27T07:46:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.395+0000 7fc5f96aeb80 20 client.15177 trim_cache size 0 max 16384 2024-05-27T07:46:02.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.419+0000 7fc5f96aeb80 1 Processor -- start 2024-05-27T07:46:02.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.419+0000 7fc5f96aeb80 1 -- start start 2024-05-27T07:46:02.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.419+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f0323df0 0x55e0f04b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.419+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f049e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.419+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 0x55e0f04a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.419+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e0f04bc190 con 0x55e0f040f9a0 2024-05-27T07:46:02.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.419+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e0f04bc860 con 0x55e0f0323df0 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.419+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e0f04bcf30 con 0x55e0f04a2e20 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f049e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f049e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49110/0 (socket says 172.21.15.90:49110) 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e3400700 1 -- 172.21.15.90:0/4274285039 learned_addr learned my addr 172.21.15.90:0/4274285039 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e8c00700 1 --2- 172.21.15.90:0/4274285039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 0x55e0f04a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e3400700 1 -- 172.21.15.90:0/4274285039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 msgr2=0x55e0f04a5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e3400700 1 --2- 172.21.15.90:0/4274285039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 0x55e0f04a5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e3400700 1 -- 172.21.15.90:0/4274285039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f0323df0 msgr2=0x55e0f04b8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e3e00700 1 --2- 172.21.15.90:0/4274285039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f0323df0 0x55e0f04b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e3400700 1 --2- 172.21.15.90:0/4274285039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f0323df0 0x55e0f04b8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e3400700 1 -- 172.21.15.90:0/4274285039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f04ad3c0 con 0x55e0f040f9a0 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.420+0000 7fc5e8c00700 1 --2- 172.21.15.90:0/4274285039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 0x55e0f04a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.421+0000 7fc5e3400700 1 --2- 172.21.15.90:0/4274285039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f049e520 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00278c0 tx=0x7fc5d0005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b648ee8606da1a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:02.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.421+0000 7fc5d8c00700 1 -- 172.21.15.90:0/4274285039 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0046070 con 0x55e0f040f9a0 2024-05-27T07:46:02.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.421+0000 7fc5d8c00700 1 -- 172.21.15.90:0/4274285039 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5d0027d30 con 0x55e0f040f9a0 2024-05-27T07:46:02.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.421+0000 7fc5d8c00700 1 -- 172.21.15.90:0/4274285039 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0036040 con 0x55e0f040f9a0 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.422+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/4274285039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 msgr2=0x55e0f049e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.422+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/4274285039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f049e520 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00278c0 tx=0x7fc5d0005d00 comp rx=0 tx=0).stop 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.422+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/4274285039 shutdown_connections 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.422+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/4274285039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f04a2e20 0x55e0f04a5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.422+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/4274285039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f0323df0 0x55e0f04b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.422+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/4274285039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f040f9a0 0x55e0f049e520 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.422+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/4274285039 >> 172.21.15.90:0/4274285039 conn(0x55e0f03bd340 msgr2=0x55e0f049de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.422+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/4274285039 shutdown_connections 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.422+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/4274285039 wait complete. 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.423+0000 7fc5f96aeb80 1 -- start start 2024-05-27T07:46:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.423+0000 7fc5f96aeb80 1 Processor -- start 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.423+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 0x55e0f04c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.423+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f040f9a0 0x55e0f04a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5f96aeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f04a2e20 0x55e0f04a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e0f04bdcd0 con 0x55e0f04a2e20 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e0f04bd600 con 0x55e0f040f9a0 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5f96aeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e0f04be3a0 con 0x55e0f0323df0 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5f96aeb80 10 client.0 _ll_register_callbacks cb 0x55e0f036ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e8c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f04a2e20 0x55e0f04a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e8c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f04a2e20 0x55e0f04a6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49124/0 (socket says 172.21.15.90:49124) 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e8c00700 1 -- 172.21.15.90:0/2103978420 learned_addr learned my addr 172.21.15.90:0/2103978420 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e3400700 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f040f9a0 0x55e0f04a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e3e00700 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 0x55e0f04c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e8c00700 1 -- 172.21.15.90:0/2103978420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 msgr2=0x55e0f04c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e8c00700 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 0x55e0f04c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e8c00700 1 -- 172.21.15.90:0/2103978420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f040f9a0 msgr2=0x55e0f04a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e8c00700 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f040f9a0 0x55e0f04a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e8c00700 1 -- 172.21.15.90:0/2103978420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f04a7610 con 0x55e0f04a2e20 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.424+0000 7fc5e3e00700 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 0x55e0f04c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.425+0000 7fc5e8c00700 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f04a2e20 0x55e0f04a6f10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0e1350 tx=0x7fc5dc0ab440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd91c1b9a268f7e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.425+0000 7fc5ce000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.425+0000 7fc5ce000700 1 -- 172.21.15.90:0/2103978420 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc0aa670 con 0x55e0f04a2e20 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.425+0000 7fc5ce000700 1 -- 172.21.15.90:0/2103978420 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc5dc0aa370 con 0x55e0f04a2e20 2024-05-27T07:46:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.425+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2103978420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d0027d30 con 0x55e0f04a2e20 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.425+0000 7fc5ce000700 1 -- 172.21.15.90:0/2103978420 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc5dc0aaa10 con 0x55e0f04a2e20 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.425+0000 7fc5f96aeb80 10 client.15183 fetch_fsmap learned FSMap version 29 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.425+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2103978420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0f04c4270 con 0x55e0f04a2e20 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.426+0000 7fc5ce000700 1 -- 172.21.15.90:0/2103978420 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5d0027d30 con 0x55e0f04a2e20 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.426+0000 7fc5ce000700 1 -- 172.21.15.90:0/2103978420 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc5dc011070 con 0x55e0f04a2e20 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.426+0000 7fc5f96aeb80 10 client.15183 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.426+0000 7fc5f96aeb80 10 client.15183 resolve_mds: resolved name 'k' to daemon mds.k 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.426+0000 7fc5f96aeb80 20 client.15183 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.426+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e0f052a3f0 0x55e0f052b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.426+0000 7fc5f96aeb80 4 client.15183 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:46:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.426+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2103978420 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "status"}) -- 0x55e0f03cc520 con 0x55e0f052a3f0 2024-05-27T07:46:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.427+0000 7fc5e3e00700 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e0f052a3f0 0x55e0f052b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.428+0000 7fc5e3e00700 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e0f052a3f0 0x55e0f052b860 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff71513dd3580e14 server_cookie=8b3b7ad4af0e9949 in_seq=0 out_seq=0 2024-05-27T07:46:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.428+0000 7fc5ce000700 10 client.15183 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:46:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.428+0000 7fc5ce000700 1 -- 172.21.15.90:0/2103978420 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e0f03cc520 con 0x55e0f052a3f0 2024-05-27T07:46:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.428+0000 7fc5ce000700 10 client.15183 handle_command_reply: tid=0 2024-05-27T07:46:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.428+0000 7fc5f96aeb80 1 client.15183 shutdown 2024-05-27T07:46:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.428+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2103978420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f04a2e20 msgr2=0x55e0f04a6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.428+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f04a2e20 0x55e0f04a6f10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0e1350 tx=0x7fc5dc0ab440 comp rx=0 tx=0).stop 2024-05-27T07:46:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.429+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2103978420 shutdown_connections 2024-05-27T07:46:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.429+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e0f052a3f0 0x55e0f052b860 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.429+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e0f04a2e20 0x55e0f04a6f10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.429+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0f0323df0 0x55e0f04c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.429+0000 7fc5f96aeb80 1 --2- 172.21.15.90:0/2103978420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0f040f9a0 0x55e0f04a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.429+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2103978420 >> 172.21.15.90:0/2103978420 conn(0x55e0f03bd340 msgr2=0x55e0f049db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.429+0000 7fc5ce000700 0 client.15183 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:46:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.429+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2103978420 shutdown_connections 2024-05-27T07:46:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.429+0000 7fc5f96aeb80 1 -- 172.21.15.90:0/2103978420 wait complete. 2024-05-27T07:46:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.429+0000 7fc5f96aeb80 20 client.15183 trim_cache size 0 max 16384 2024-05-27T07:46:02.433 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:46:02.433 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:46:02.433 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 3, 2024-05-27T07:46:02.433 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34406, 2024-05-27T07:46:02.433 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:46:02.433 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:46:02.433 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:46:02.433 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 209.55120393300001, 2024-05-27T07:46:02.434 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-05-27T07:46:02.434 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 91, 2024-05-27T07:46:02.434 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:46:02.434 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 352.46321697799999 2024-05-27T07:46:02.434 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:46:02.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 -- 172.21.15.90:0/1825915168 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5c80b2880 msgr2=0x7fc5c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:02.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5c80b2880 0x7fc5c80b4d90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0013510 tx=0x7fc5d000bf90 comp rx=0 tx=0).stop 2024-05-27T07:46:02.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 -- 172.21.15.90:0/1825915168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5e4110c00 msgr2=0x7fc5e41a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:02.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5e4110c00 0x7fc5e41a1790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00b070 tx=0x7fc5dc004e00 comp rx=0 tx=0).stop 2024-05-27T07:46:02.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 -- 172.21.15.90:0/1825915168 shutdown_connections 2024-05-27T07:46:02.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fc5c80b2880 0x7fc5c80b4d90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5e4110c00 0x7fc5e41a1790 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5e4076be0 0x7fc5e419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 --2- 172.21.15.90:0/1825915168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5e4077530 0x7fc5e419d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:02.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 -- 172.21.15.90:0/1825915168 >> 172.21.15.90:0/1825915168 conn(0x7fc5e40ffec0 msgr2=0x7fc5e4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:02.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 -- 172.21.15.90:0/1825915168 shutdown_connections 2024-05-27T07:46:02.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:02.430+0000 7fc5eac00700 1 -- 172.21.15.90:0/1825915168 wait complete. 2024-05-27T07:46:03.726 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-05-27T07:46:03.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:03 smithi090 ceph-mon[31429]: pgmap v533: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:46:04.168 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:04.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:03 smithi136 ceph-mon[35324]: pgmap v533: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:46:04.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:03 smithi129 ceph-mon[35254]: pgmap v533: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:46:06.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:05 smithi090 ceph-mon[31429]: pgmap v534: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:46:06.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:05 smithi136 ceph-mon[35324]: pgmap v534: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:46:06.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:05 smithi129 ceph-mon[35254]: pgmap v534: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:46:06.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.961+0000 7f9a03800700 1 -- 172.21.15.90:0/2672927095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99fc110bf0 msgr2=0x7f99fc113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:06.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.961+0000 7f9a03800700 1 --2- 172.21.15.90:0/2672927095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99fc110bf0 0x7f99fc113010 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f99f400b0a0 tx=0x7f99f400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.961+0000 7f9a03800700 1 -- 172.21.15.90:0/2672927095 shutdown_connections 2024-05-27T07:46:06.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.961+0000 7f9a03800700 1 --2- 172.21.15.90:0/2672927095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99fc110bf0 0x7f99fc113010 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.961+0000 7f9a03800700 1 --2- 172.21.15.90:0/2672927095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99fc075af0 0x7f99fc075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.961+0000 7f9a03800700 1 --2- 172.21.15.90:0/2672927095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc076440 0x7f99fc1106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.961+0000 7f9a03800700 1 -- 172.21.15.90:0/2672927095 >> 172.21.15.90:0/2672927095 conn(0x7f99fc100030 msgr2=0x7f99fc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:06.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.961+0000 7f9a03800700 1 -- 172.21.15.90:0/2672927095 shutdown_connections 2024-05-27T07:46:06.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.961+0000 7f9a03800700 1 -- 172.21.15.90:0/2672927095 wait complete. 2024-05-27T07:46:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.963+0000 7f9a03800700 1 Processor -- start 2024-05-27T07:46:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.963+0000 7f9a03800700 1 -- start start 2024-05-27T07:46:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.963+0000 7f9a03800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99fc075af0 0x7f99fc19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.963+0000 7f9a03800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99fc076440 0x7f99fc19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.963+0000 7f9a03800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc110bf0 0x7f99fc1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.963+0000 7f9a03800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc116640 con 0x7f99fc076440 2024-05-27T07:46:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.963+0000 7f9a03800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1164c0 con 0x7f99fc110bf0 2024-05-27T07:46:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.963+0000 7f9a03800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1167c0 con 0x7f99fc075af0 2024-05-27T07:46:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f99fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99fc076440 0x7f99fc19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f99fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99fc076440 0x7f99fc19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49144/0 (socket says 172.21.15.90:49144) 2024-05-27T07:46:06.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f99fbe00700 1 -- 172.21.15.90:0/1142615774 learned_addr learned my addr 172.21.15.90:0/1142615774 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:06.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f9a00e00700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99fc075af0 0x7f99fc19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f9a01800700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc110bf0 0x7f99fc1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f99fbe00700 1 -- 172.21.15.90:0/1142615774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99fc075af0 msgr2=0x7f99fc19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:06.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f99fbe00700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99fc075af0 0x7f99fc19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f99fbe00700 1 -- 172.21.15.90:0/1142615774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc110bf0 msgr2=0x7f99fc1a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:06.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f99fbe00700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc110bf0 0x7f99fc1a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f99fbe00700 1 -- 172.21.15.90:0/1142615774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc1a1ec0 con 0x7f99fc076440 2024-05-27T07:46:06.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.964+0000 7f9a01800700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc110bf0 0x7f99fc1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:06.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.965+0000 7f99fbe00700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99fc076440 0x7f99fc19d590 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f99e800c970 tx=0x7f99e800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:06.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.965+0000 7f99f9600700 1 -- 172.21.15.90:0/1142615774 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e800ce60 con 0x7f99fc076440 2024-05-27T07:46:06.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.965+0000 7f99f9600700 1 -- 172.21.15.90:0/1142615774 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99e8004830 con 0x7f99fc076440 2024-05-27T07:46:06.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.965+0000 7f9a03800700 1 -- 172.21.15.90:0/1142615774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99fc1a21b0 con 0x7f99fc076440 2024-05-27T07:46:06.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.965+0000 7f99f9600700 1 -- 172.21.15.90:0/1142615774 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8005420 con 0x7f99fc076440 2024-05-27T07:46:06.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.965+0000 7f9a03800700 1 -- 172.21.15.90:0/1142615774 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99fc1a9ad0 con 0x7f99fc076440 2024-05-27T07:46:06.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.969+0000 7f99f9600700 1 -- 172.21.15.90:0/1142615774 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f99e8004cb0 con 0x7f99fc076440 2024-05-27T07:46:06.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.971+0000 7f9a122e9b80 1 Processor -- start 2024-05-27T07:46:06.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.971+0000 7f9a122e9b80 1 -- start start 2024-05-27T07:46:06.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f99f9600700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f99e40b2900 0x7f99e40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 0x55799ba757a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 0x55799ba6f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba70170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55799b99ddf0 con 0x55799ba6fd60 2024-05-27T07:46:06.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55799b98d990 con 0x55799ba75ce0 2024-05-27T07:46:06.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55799b9919d0 con 0x55799b8f0df0 2024-05-27T07:46:06.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f99f9600700 1 -- 172.21.15.90:0/1142615774 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f99e80e2120 con 0x7f99fc076440 2024-05-27T07:46:06.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f9a01800700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f99e40b2900 0x7f99e40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f9a00e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 0x55799ba757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.972+0000 7f9a00e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 0x55799ba757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:54318/0 (socket says 172.21.15.90:54318) 2024-05-27T07:46:06.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.973+0000 7f9a00e00700 1 -- 172.21.15.90:0/557204664 learned_addr learned my addr 172.21.15.90:0/557204664 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:06.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.973+0000 7f9a01800700 1 --2- 172.21.15.90:0/557204664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba70170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.973+0000 7f99fbe00700 1 --2- 172.21.15.90:0/557204664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 0x55799ba6f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.973+0000 7f9a00e00700 1 -- 172.21.15.90:0/557204664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 msgr2=0x55799ba6f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:06.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.973+0000 7f9a00e00700 1 --2- 172.21.15.90:0/557204664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 0x55799ba6f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.973+0000 7f9a00e00700 1 -- 172.21.15.90:0/557204664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 msgr2=0x55799ba70170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:06.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.973+0000 7f9a00e00700 1 --2- 172.21.15.90:0/557204664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba70170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.973+0000 7f9a00e00700 1 -- 172.21.15.90:0/557204664 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799b9563d0 con 0x55799b8f0df0 2024-05-27T07:46:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.973+0000 7f9a01800700 1 --2- 172.21.15.90:0/557204664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba70170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:46:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.974+0000 7f9a01800700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f99e40b2900 0x7f99e40b4e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f99f400b830 tx=0x7f99f4009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.974+0000 7f9a00e00700 1 --2- 172.21.15.90:0/557204664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 0x55799ba757a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009af0 tx=0x7f99f0009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb70374c7ada90b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.974+0000 7f99ee000700 1 -- 172.21.15.90:0/557204664 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0017070 con 0x55799b8f0df0 2024-05-27T07:46:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.974+0000 7f99ee000700 1 -- 172.21.15.90:0/557204664 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f000ed60 con 0x55799b8f0df0 2024-05-27T07:46:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.974+0000 7f99ee000700 1 -- 172.21.15.90:0/557204664 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0013d10 con 0x55799b8f0df0 2024-05-27T07:46:06.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.975+0000 7f9a122e9b80 1 -- 172.21.15.90:0/557204664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 msgr2=0x55799ba757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:06.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.975+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/557204664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 0x55799ba757a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009af0 tx=0x7f99f0009e00 comp rx=0 tx=0).stop 2024-05-27T07:46:06.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.976+0000 7f9a122e9b80 1 -- 172.21.15.90:0/557204664 shutdown_connections 2024-05-27T07:46:06.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.976+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/557204664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba70170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.976+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/557204664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 0x55799ba757a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.976+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/557204664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 0x55799ba6f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.976+0000 7f9a122e9b80 1 -- 172.21.15.90:0/557204664 >> 172.21.15.90:0/557204664 conn(0x55799b950f70 msgr2=0x55799ba6a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:06.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.976+0000 7f9a122e9b80 1 -- 172.21.15.90:0/557204664 shutdown_connections 2024-05-27T07:46:06.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.976+0000 7f9a122e9b80 1 -- 172.21.15.90:0/557204664 wait complete. 2024-05-27T07:46:06.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.976+0000 7f9a122e9b80 1 -- start start 2024-05-27T07:46:06.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.977+0000 7f9a122e9b80 1 Processor -- start 2024-05-27T07:46:06.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.977+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799b8f0df0 0x55799ba794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.977+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799ba6fd60 0x55799ba79a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.977+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba75ce0 0x55799ba79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.977+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55799b893f50 con 0x55799ba75ce0 2024-05-27T07:46:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.977+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55799b92e710 con 0x55799b8f0df0 2024-05-27T07:46:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.977+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55799ba7c7b0 con 0x55799ba6fd60 2024-05-27T07:46:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.977+0000 7f9a122e9b80 10 client.0 _ll_register_callbacks cb 0x55799b937c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f9a01800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba75ce0 0x55799ba79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f9a01800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba75ce0 0x55799ba79f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49148/0 (socket says 172.21.15.90:49148) 2024-05-27T07:46:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f9a01800700 1 -- 172.21.15.90:0/2262834995 learned_addr learned my addr 172.21.15.90:0/2262834995 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f9a00e00700 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799b8f0df0 0x55799ba794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f99fbe00700 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799ba6fd60 0x55799ba79a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f9a01800700 1 -- 172.21.15.90:0/2262834995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799ba6fd60 msgr2=0x55799ba79a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f9a01800700 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799ba6fd60 0x55799ba79a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f9a01800700 1 -- 172.21.15.90:0/2262834995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799b8f0df0 msgr2=0x55799ba794e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:06.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f9a01800700 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799b8f0df0 0x55799ba794e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f9a01800700 1 -- 172.21.15.90:0/2262834995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799b8dd880 con 0x55799ba75ce0 2024-05-27T07:46:06.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.978+0000 7f9a00e00700 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799b8f0df0 0x55799ba794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:06.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.979+0000 7f9a01800700 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba75ce0 0x55799ba79f60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f99f4015000 tx=0x7f99f4009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd46b62825aad4ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:06.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.979+0000 7f99e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:06.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.979+0000 7f99e2a00700 1 -- 172.21.15.90:0/2262834995 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f402f070 con 0x55799ba75ce0 2024-05-27T07:46:06.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.979+0000 7f9a122e9b80 1 -- 172.21.15.90:0/2262834995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f000ed60 con 0x55799ba75ce0 2024-05-27T07:46:06.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.979+0000 7f99e2a00700 1 -- 172.21.15.90:0/2262834995 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f99f4009c40 con 0x55799ba75ce0 2024-05-27T07:46:06.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.979+0000 7f99e2a00700 1 -- 172.21.15.90:0/2262834995 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99f4038050 con 0x55799ba75ce0 2024-05-27T07:46:06.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.979+0000 7f9a122e9b80 10 client.15195 fetch_fsmap learned FSMap version 29 2024-05-27T07:46:06.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.979+0000 7f9a122e9b80 1 -- 172.21.15.90:0/2262834995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55799b95e010 con 0x55799ba75ce0 2024-05-27T07:46:06.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.980+0000 7f99e2a00700 1 -- 172.21.15.90:0/2262834995 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99f000ed60 con 0x55799ba75ce0 2024-05-27T07:46:06.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.980+0000 7f99e2a00700 1 -- 172.21.15.90:0/2262834995 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f99f4032020 con 0x55799ba75ce0 2024-05-27T07:46:06.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.980+0000 7f9a122e9b80 10 client.15195 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:46:06.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.980+0000 7f9a122e9b80 10 client.15195 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T07:46:06.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.980+0000 7f9a122e9b80 20 client.15195 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:06.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.980+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55799ba896b0 0x55799ba8ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:06.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.980+0000 7f9a122e9b80 4 client.15195 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:06.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.980+0000 7f9a122e9b80 1 -- 172.21.15.90:0/2262834995 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55799b99fd40 con 0x55799ba896b0 2024-05-27T07:46:06.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.981+0000 7f9a00e00700 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55799ba896b0 0x55799ba8ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:06.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.982+0000 7f9a00e00700 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55799ba896b0 0x55799ba8ab20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9a1d1a31710616e server_cookie=347d3c3342027a87 in_seq=0 out_seq=0 2024-05-27T07:46:06.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.982+0000 7f99e2a00700 10 client.15195 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:46:06.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.984+0000 7f99e2a00700 1 -- 172.21.15.90:0/2262834995 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55799b99fd40 con 0x55799ba896b0 2024-05-27T07:46:06.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.984+0000 7f99e2a00700 10 client.15195 handle_command_reply: tid=0 2024-05-27T07:46:06.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.984+0000 7f9a122e9b80 1 client.15195 shutdown 2024-05-27T07:46:06.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.984+0000 7f9a122e9b80 1 -- 172.21.15.90:0/2262834995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba75ce0 msgr2=0x55799ba79f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:06.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.984+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba75ce0 0x55799ba79f60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f99f4015000 tx=0x7f99f4009880 comp rx=0 tx=0).stop 2024-05-27T07:46:06.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.985+0000 7f9a122e9b80 1 -- 172.21.15.90:0/2262834995 shutdown_connections 2024-05-27T07:46:06.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.985+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55799ba896b0 0x55799ba8ab20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.985+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba75ce0 0x55799ba79f60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.985+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799b8f0df0 0x55799ba794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.985+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/2262834995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799ba6fd60 0x55799ba79a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:06.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.985+0000 7f9a122e9b80 1 -- 172.21.15.90:0/2262834995 >> 172.21.15.90:0/2262834995 conn(0x55799b950f70 msgr2=0x55799ba6a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:06.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.985+0000 7f99e2a00700 0 client.15195 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:46:06.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.985+0000 7f9a122e9b80 1 -- 172.21.15.90:0/2262834995 shutdown_connections 2024-05-27T07:46:06.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.985+0000 7f9a122e9b80 1 -- 172.21.15.90:0/2262834995 wait complete. 2024-05-27T07:46:06.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:06.985+0000 7f9a122e9b80 20 client.15195 trim_cache size 0 max 16384 2024-05-27T07:46:07.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.008+0000 7f9a122e9b80 1 Processor -- start 2024-05-27T07:46:07.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.008+0000 7f9a122e9b80 1 -- start start 2024-05-27T07:46:07.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.008+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 0x55799ba6f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:07.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.008+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba63c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:07.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.008+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 0x55799ba74c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:07.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.008+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55799ba88e60 con 0x55799ba6fd60 2024-05-27T07:46:07.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.008+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55799ba89530 con 0x55799ba75ce0 2024-05-27T07:46:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.008+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55799ba89c00 con 0x55799b8f0df0 2024-05-27T07:46:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f99fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba63c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f99fbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba63c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49160/0 (socket says 172.21.15.90:49160) 2024-05-27T07:46:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f99fbe00700 1 -- 172.21.15.90:0/4208983678 learned_addr learned my addr 172.21.15.90:0/4208983678 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f9a00e00700 1 --2- 172.21.15.90:0/4208983678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 0x55799ba6f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f9a01800700 1 --2- 172.21.15.90:0/4208983678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 0x55799ba74c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f99fbe00700 1 -- 172.21.15.90:0/4208983678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 msgr2=0x55799ba6f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f99fbe00700 1 --2- 172.21.15.90:0/4208983678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 0x55799ba6f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f99fbe00700 1 -- 172.21.15.90:0/4208983678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 msgr2=0x55799ba74c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f99fbe00700 1 --2- 172.21.15.90:0/4208983678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 0x55799ba74c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f99fbe00700 1 -- 172.21.15.90:0/4208983678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799ba77a50 con 0x55799ba6fd60 2024-05-27T07:46:07.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.009+0000 7f9a01800700 1 --2- 172.21.15.90:0/4208983678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 0x55799ba74c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:07.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.010+0000 7f99fbe00700 1 --2- 172.21.15.90:0/4208983678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba63c40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f99e80a92e0 tx=0x7f99e80a9b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7168d0756cb74d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:07.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.010+0000 7f99ed600700 1 -- 172.21.15.90:0/4208983678 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e801d040 con 0x55799ba6fd60 2024-05-27T07:46:07.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.010+0000 7f99ed600700 1 -- 172.21.15.90:0/4208983678 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99e80b21b0 con 0x55799ba6fd60 2024-05-27T07:46:07.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.010+0000 7f99ed600700 1 -- 172.21.15.90:0/4208983678 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e809cd80 con 0x55799ba6fd60 2024-05-27T07:46:07.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.011+0000 7f9a122e9b80 1 -- 172.21.15.90:0/4208983678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 msgr2=0x55799ba63c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:07.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.011+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/4208983678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba63c40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f99e80a92e0 tx=0x7f99e80a9b00 comp rx=0 tx=0).stop 2024-05-27T07:46:07.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.011+0000 7f9a122e9b80 1 -- 172.21.15.90:0/4208983678 shutdown_connections 2024-05-27T07:46:07.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.011+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/4208983678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba75ce0 0x55799ba74c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.011+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/4208983678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799b8f0df0 0x55799ba6f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.011+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/4208983678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799ba6fd60 0x55799ba63c40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.011+0000 7f9a122e9b80 1 -- 172.21.15.90:0/4208983678 >> 172.21.15.90:0/4208983678 conn(0x55799b950f70 msgr2=0x55799ba74580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.011+0000 7f9a122e9b80 1 -- 172.21.15.90:0/4208983678 shutdown_connections 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.011+0000 7f9a122e9b80 1 -- 172.21.15.90:0/4208983678 wait complete. 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.012+0000 7f9a122e9b80 1 -- start start 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.012+0000 7f9a122e9b80 1 Processor -- start 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.012+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799b8f0df0 0x55799ba94750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.012+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba6fd60 0x55799ba84e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.012+0000 7f9a122e9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799ba75ce0 0x55799ba85470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55799ba8a9a0 con 0x55799b8f0df0 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55799ba8a2d0 con 0x55799ba6fd60 2024-05-27T07:46:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a122e9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55799ba8b070 con 0x55799ba75ce0 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a122e9b80 10 client.0 _ll_register_callbacks cb 0x55799b937c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a00e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799b8f0df0 0x55799ba94750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a00e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799b8f0df0 0x55799ba94750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49162/0 (socket says 172.21.15.90:49162) 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a00e00700 1 -- 172.21.15.90:0/3957102624 learned_addr learned my addr 172.21.15.90:0/3957102624 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f99fbe00700 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba6fd60 0x55799ba84e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a00e00700 1 -- 172.21.15.90:0/3957102624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799ba75ce0 msgr2=0x55799ba85470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a00e00700 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799ba75ce0 0x55799ba85470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a00e00700 1 -- 172.21.15.90:0/3957102624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba6fd60 msgr2=0x55799ba84e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a00e00700 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba6fd60 0x55799ba84e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f9a00e00700 1 -- 172.21.15.90:0/3957102624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799ba85c20 con 0x55799b8f0df0 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.013+0000 7f99fbe00700 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba6fd60 0x55799ba84e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.014+0000 7f9a00e00700 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799b8f0df0 0x55799ba94750 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f99f000a010 tx=0x7f99f0009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca44b27b9f33e89 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.014+0000 7f99e2a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:07.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.014+0000 7f99e2a00700 1 -- 172.21.15.90:0/3957102624 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0017040 con 0x55799b8f0df0 2024-05-27T07:46:07.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.014+0000 7f9a122e9b80 1 -- 172.21.15.90:0/3957102624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99e80b21b0 con 0x55799b8f0df0 2024-05-27T07:46:07.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.014+0000 7f99e2a00700 1 -- 172.21.15.90:0/3957102624 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f99f00136a0 con 0x55799b8f0df0 2024-05-27T07:46:07.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.014+0000 7f99e2a00700 1 -- 172.21.15.90:0/3957102624 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99f0005a50 con 0x55799b8f0df0 2024-05-27T07:46:07.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.014+0000 7f9a122e9b80 10 client.15201 fetch_fsmap learned FSMap version 29 2024-05-27T07:46:07.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.014+0000 7f9a122e9b80 1 -- 172.21.15.90:0/3957102624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55799baf4ea0 con 0x55799b8f0df0 2024-05-27T07:46:07.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.015+0000 7f99e2a00700 1 -- 172.21.15.90:0/3957102624 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99e80b21b0 con 0x55799b8f0df0 2024-05-27T07:46:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.015+0000 7f99e2a00700 1 -- 172.21.15.90:0/3957102624 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f99f0006020 con 0x55799b8f0df0 2024-05-27T07:46:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.015+0000 7f9a122e9b80 10 client.15201 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:46:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.015+0000 7f9a122e9b80 10 client.15201 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-27T07:46:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.015+0000 7f9a122e9b80 20 client.15201 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.015+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55799baf7000 0x55799baf8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.015+0000 7f9a122e9b80 4 client.15201 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:46:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.015+0000 7f9a122e9b80 1 -- 172.21.15.90:0/3957102624 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "status"}) -- 0x55799b999520 con 0x55799baf7000 2024-05-27T07:46:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.016+0000 7f99fbe00700 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55799baf7000 0x55799baf8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.017+0000 7f99fbe00700 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55799baf7000 0x55799baf8470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32d828bfcf56bef4 server_cookie=c3cc0d8f03ef71aa in_seq=0 out_seq=0 2024-05-27T07:46:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.017+0000 7f99e2a00700 10 client.15201 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.017+0000 7f99e2a00700 1 -- 172.21.15.90:0/3957102624 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55799b999520 con 0x55799baf7000 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.017+0000 7f99e2a00700 10 client.15201 handle_command_reply: tid=0 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.017+0000 7f9a122e9b80 1 client.15201 shutdown 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 1 -- 172.21.15.90:0/3957102624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799b8f0df0 msgr2=0x55799ba94750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799b8f0df0 0x55799ba94750 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f99f000a010 tx=0x7f99f0009200 comp rx=0 tx=0).stop 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 1 -- 172.21.15.90:0/3957102624 shutdown_connections 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55799baf7000 0x55799baf8470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799ba75ce0 0x55799ba85470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55799b8f0df0 0x55799ba94750 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 1 --2- 172.21.15.90:0/3957102624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55799ba6fd60 0x55799ba84e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 1 -- 172.21.15.90:0/3957102624 >> 172.21.15.90:0/3957102624 conn(0x55799b950f70 msgr2=0x55799ba731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f99e2a00700 0 client.15201 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:46:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 1 -- 172.21.15.90:0/3957102624 shutdown_connections 2024-05-27T07:46:07.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 1 -- 172.21.15.90:0/3957102624 wait complete. 2024-05-27T07:46:07.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.018+0000 7f9a122e9b80 20 client.15201 trim_cache size 0 max 16384 2024-05-27T07:46:07.021 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:46:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:46:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-05-27T07:46:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34424, 2024-05-27T07:46:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:46:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:46:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:46:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 215.158634976, 2024-05-27T07:46:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-05-27T07:46:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 91, 2024-05-27T07:46:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:46:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 343.779288614 2024-05-27T07:46:07.022 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 -- 172.21.15.90:0/1142615774 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f99e40b2900 msgr2=0x7f99e40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f99e40b2900 0x7f99e40b4e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f99f400b830 tx=0x7f99f4009270 comp rx=0 tx=0).stop 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 -- 172.21.15.90:0/1142615774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99fc076440 msgr2=0x7f99fc19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99fc076440 0x7f99fc19d590 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f99e800c970 tx=0x7f99e800cc80 comp rx=0 tx=0).stop 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 -- 172.21.15.90:0/1142615774 shutdown_connections 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f99e40b2900 0x7f99e40b4e10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc110bf0 0x7f99fc1a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99fc075af0 0x7f99fc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 --2- 172.21.15.90:0/1142615774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99fc076440 0x7f99fc19d590 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 -- 172.21.15.90:0/1142615774 >> 172.21.15.90:0/1142615774 conn(0x7f99fc100030 msgr2=0x7f99fc1006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 -- 172.21.15.90:0/1142615774 shutdown_connections 2024-05-27T07:46:07.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:07.019+0000 7f9a03800700 1 -- 172.21.15.90:0/1142615774 wait complete. 2024-05-27T07:46:08.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:07 smithi090 ceph-mon[31429]: pgmap v535: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:46:08.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:07 smithi136 ceph-mon[35324]: pgmap v535: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:46:08.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T07:46:08.312 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:07 smithi129 ceph-mon[35254]: pgmap v535: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:46:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:10.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:09 smithi090 ceph-mon[31429]: pgmap v536: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:46:10.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:09 smithi136 ceph-mon[35324]: pgmap v536: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:46:10.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:09 smithi129 ceph-mon[35254]: pgmap v536: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T07:46:11.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.300+0000 7fb7f8600700 1 -- 172.21.15.90:0/2360989397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f01151e0 msgr2=0x7fb7f0117600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:11.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.300+0000 7fb7f8600700 1 --2- 172.21.15.90:0/2360989397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f01151e0 0x7fb7f0117600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800b0a0 tx=0x7fb7e800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.300+0000 7fb7f8600700 1 -- 172.21.15.90:0/2360989397 shutdown_connections 2024-05-27T07:46:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.300+0000 7fb7f8600700 1 --2- 172.21.15.90:0/2360989397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f01151e0 0x7fb7f0117600 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.300+0000 7fb7f8600700 1 --2- 172.21.15.90:0/2360989397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7f0077ce0 0x7fb7f00780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.301+0000 7fb7f8600700 1 --2- 172.21.15.90:0/2360989397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7f0078630 0x7fb7f0114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.301+0000 7fb7f8600700 1 -- 172.21.15.90:0/2360989397 >> 172.21.15.90:0/2360989397 conn(0x7fb7f0102080 msgr2=0x7fb7f01044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.301+0000 7fb7f8600700 1 -- 172.21.15.90:0/2360989397 shutdown_connections 2024-05-27T07:46:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.301+0000 7fb7f8600700 1 -- 172.21.15.90:0/2360989397 wait complete. 2024-05-27T07:46:11.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.302+0000 7fb7f8600700 1 Processor -- start 2024-05-27T07:46:11.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.302+0000 7fb7f8600700 1 -- start start 2024-05-27T07:46:11.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.303+0000 7fb7f8600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7f0077ce0 0x7fb7f01a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.303+0000 7fb7f8600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f0078630 0x7fb7f01a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.303+0000 7fb7f8600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7f01151e0 0x7fb7f01a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.303+0000 7fb7f8600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f011a920 con 0x7fb7f0078630 2024-05-27T07:46:11.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.303+0000 7fb7f8600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f011a7a0 con 0x7fb7f0077ce0 2024-05-27T07:46:11.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.303+0000 7fb7f8600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f011aaa0 con 0x7fb7f01151e0 2024-05-27T07:46:11.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.303+0000 7fb7f5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f0078630 0x7fb7f01a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.303+0000 7fb7f5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f0078630 0x7fb7f01a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57818/0 (socket says 172.21.15.90:57818) 2024-05-27T07:46:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.303+0000 7fb7f5200700 1 -- 172.21.15.90:0/1145851701 learned_addr learned my addr 172.21.15.90:0/1145851701 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.304+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7f0077ce0 0x7fb7f01a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.304+0000 7fb7f6600700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7f01151e0 0x7fb7f01a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.304+0000 7fb7f5200700 1 -- 172.21.15.90:0/1145851701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7f01151e0 msgr2=0x7fb7f01a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.304+0000 7fb7f5200700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7f01151e0 0x7fb7f01a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.304+0000 7fb7f5200700 1 -- 172.21.15.90:0/1145851701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7f0077ce0 msgr2=0x7fb7f01a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.304+0000 7fb7f5200700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7f0077ce0 0x7fb7f01a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.304+0000 7fb7f5200700 1 -- 172.21.15.90:0/1145851701 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f01a60f0 con 0x7fb7f0078630 2024-05-27T07:46:11.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.304+0000 7fb7f6600700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7f01151e0 0x7fb7f01a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:11.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.304+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7f0077ce0 0x7fb7f01a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:11.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.304+0000 7fb7f5200700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f0078630 0x7fb7f01a1870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00c9e0 tx=0x7fb7dc00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:11.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.305+0000 7fb7ee000700 1 -- 172.21.15.90:0/1145851701 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc007aa0 con 0x7fb7f0078630 2024-05-27T07:46:11.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.305+0000 7fb7ee000700 1 -- 172.21.15.90:0/1145851701 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7dc007c40 con 0x7fb7f0078630 2024-05-27T07:46:11.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.305+0000 7fb7ee000700 1 -- 172.21.15.90:0/1145851701 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc018660 con 0x7fb7f0078630 2024-05-27T07:46:11.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.305+0000 7fb7f8600700 1 -- 172.21.15.90:0/1145851701 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f01a63e0 con 0x7fb7f0078630 2024-05-27T07:46:11.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.305+0000 7fb7f8600700 1 -- 172.21.15.90:0/1145851701 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f01a68d0 con 0x7fb7f0078630 2024-05-27T07:46:11.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.309+0000 7fb7ee000700 1 -- 172.21.15.90:0/1145851701 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb7dc018800 con 0x7fb7f0078630 2024-05-27T07:46:11.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.310+0000 7fb8070bcb80 1 Processor -- start 2024-05-27T07:46:11.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.310+0000 7fb8070bcb80 1 -- start start 2024-05-27T07:46:11.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.310+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 0x55aaa1d057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.310+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1d05ce0 0x55aaa1cff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.311+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1d00170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.311+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1c2ddf0 con 0x55aaa1cffd60 2024-05-27T07:46:11.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.311+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1c1d990 con 0x55aaa1d05ce0 2024-05-27T07:46:11.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.311+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1c219d0 con 0x55aaa1b80df0 2024-05-27T07:46:11.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.311+0000 7fb7f5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1d00170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.311+0000 7fb7f5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1d00170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57830/0 (socket says 172.21.15.90:57830) 2024-05-27T07:46:11.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.311+0000 7fb7f5200700 1 -- 172.21.15.90:0/898540283 learned_addr learned my addr 172.21.15.90:0/898540283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:11.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.311+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1d05ce0 0x55aaa1cff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.311+0000 7fb7f6600700 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 0x55aaa1d057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7f5200700 1 -- 172.21.15.90:0/898540283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 msgr2=0x55aaa1d057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7f5200700 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 0x55aaa1d057a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7f5200700 1 -- 172.21.15.90:0/898540283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1d05ce0 msgr2=0x55aaa1cff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7f5200700 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1d05ce0 0x55aaa1cff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7f5200700 1 -- 172.21.15.90:0/898540283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa1be63d0 con 0x55aaa1cffd60 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7f6600700 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 0x55aaa1d057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1d05ce0 0x55aaa1cff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7ee000700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7d80b2900 0x7fb7d80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7f5200700 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1d00170 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0ea5a0 tx=0x7fb7dc0ea5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c5ffcc9b255784 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7ee000700 1 -- 172.21.15.90:0/1145851701 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fb7dc0e24a0 con 0x7fb7f0078630 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7e2a00700 1 -- 172.21.15.90:0/898540283 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc0a5590 con 0x55aaa1cffd60 2024-05-27T07:46:11.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7f6600700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7d80b2900 0x7fb7d80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.312+0000 7fb7e2a00700 1 -- 172.21.15.90:0/898540283 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7dc0a91e0 con 0x55aaa1cffd60 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.313+0000 7fb7e2a00700 1 -- 172.21.15.90:0/898540283 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc0a8e40 con 0x55aaa1cffd60 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.314+0000 7fb8070bcb80 1 -- 172.21.15.90:0/898540283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 msgr2=0x55aaa1d00170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.314+0000 7fb7f6600700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7d80b2900 0x7fb7d80b4e10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8006130 tx=0x7fb7e80060a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.314+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1d00170 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0ea5a0 tx=0x7fb7dc0ea5d0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.314+0000 7fb8070bcb80 1 -- 172.21.15.90:0/898540283 shutdown_connections 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.314+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1d00170 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.314+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 0x55aaa1d057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.314+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/898540283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1d05ce0 0x55aaa1cff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.314+0000 7fb8070bcb80 1 -- 172.21.15.90:0/898540283 >> 172.21.15.90:0/898540283 conn(0x55aaa1be0f70 msgr2=0x55aaa1cfa880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.314+0000 7fb8070bcb80 1 -- 172.21.15.90:0/898540283 shutdown_connections 2024-05-27T07:46:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.314+0000 7fb8070bcb80 1 -- 172.21.15.90:0/898540283 wait complete. 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.315+0000 7fb8070bcb80 1 -- start start 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.315+0000 7fb8070bcb80 1 Processor -- start 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.315+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 0x55aaa1d09510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.315+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1cffd60 0x55aaa1d09a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d09f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1b23f50 con 0x55aaa1d05ce0 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1bbe710 con 0x55aaa1cffd60 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1d0c7b0 con 0x55aaa1b80df0 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb8070bcb80 10 client.0 _ll_register_callbacks cb 0x55aaa1bc7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb7f6600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d09f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb7f6600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d09f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57834/0 (socket says 172.21.15.90:57834) 2024-05-27T07:46:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb7f6600700 1 -- 172.21.15.90:0/1519025798 learned_addr learned my addr 172.21.15.90:0/1519025798 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:11.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb7f5200700 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1cffd60 0x55aaa1d09a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 0x55aaa1d09510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb7f6600700 1 -- 172.21.15.90:0/1519025798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 msgr2=0x55aaa1d09510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:11.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb7f6600700 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 0x55aaa1d09510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb7f6600700 1 -- 172.21.15.90:0/1519025798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1cffd60 msgr2=0x55aaa1d09a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:11.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb7f6600700 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1cffd60 0x55aaa1d09a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.316+0000 7fb7f6600700 1 -- 172.21.15.90:0/1519025798 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa1b6d880 con 0x55aaa1d05ce0 2024-05-27T07:46:11.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.317+0000 7fb7f6600700 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d09f90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800b0a0 tx=0x7fb7e80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c8503e4ef83705 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:11.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.317+0000 7fb7d7400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:11.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.317+0000 7fb7d7400700 1 -- 172.21.15.90:0/1519025798 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e802b070 con 0x55aaa1d05ce0 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.317+0000 7fb8070bcb80 1 -- 172.21.15.90:0/1519025798 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7dc0a91e0 con 0x55aaa1d05ce0 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.317+0000 7fb7d7400700 1 -- 172.21.15.90:0/1519025798 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fb7e8005040 con 0x55aaa1d05ce0 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb7d7400700 1 -- 172.21.15.90:0/1519025798 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7e8022070 con 0x55aaa1d05ce0 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb8070bcb80 10 client.15213 fetch_fsmap learned FSMap version 29 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb8070bcb80 1 -- 172.21.15.90:0/1519025798 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaa1bee010 con 0x55aaa1d05ce0 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb7d7400700 1 -- 172.21.15.90:0/1519025798 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7dc0a91e0 con 0x55aaa1d05ce0 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb7d7400700 1 -- 172.21.15.90:0/1519025798 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7e8011070 con 0x55aaa1d05ce0 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb8070bcb80 10 client.15213 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb8070bcb80 10 client.15213 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb8070bcb80 20 client.15213 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55aaa1d196a0 0x55aaa1d1ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb8070bcb80 4 client.15213 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.318+0000 7fb8070bcb80 1 -- 172.21.15.90:0/1519025798 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaa1c2fd40 con 0x55aaa1d196a0 2024-05-27T07:46:11.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.319+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55aaa1d196a0 0x55aaa1d1ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.320+0000 7fb7d7400700 10 client.15213 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:46:11.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.320+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55aaa1d196a0 0x55aaa1d1ab10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=398faf1d2a263a6 server_cookie=bfda192b8fef9cb4 in_seq=0 out_seq=0 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb7d7400700 1 -- 172.21.15.90:0/1519025798 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55aaa1c2fd40 con 0x55aaa1d196a0 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb7d7400700 10 client.15213 handle_command_reply: tid=0 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 client.15213 shutdown 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 -- 172.21.15.90:0/1519025798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 msgr2=0x55aaa1d09f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d09f90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800b0a0 tx=0x7fb7e80099e0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 -- 172.21.15.90:0/1519025798 shutdown_connections 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d09f90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1b80df0 0x55aaa1d09510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55aaa1d196a0 0x55aaa1d1ab10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/1519025798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1cffd60 0x55aaa1d09a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 -- 172.21.15.90:0/1519025798 >> 172.21.15.90:0/1519025798 conn(0x55aaa1be0f70 msgr2=0x55aaa1cfa530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb7d7400700 0 client.15213 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:46:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 -- 172.21.15.90:0/1519025798 shutdown_connections 2024-05-27T07:46:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.323+0000 7fb8070bcb80 1 -- 172.21.15.90:0/1519025798 wait complete. 2024-05-27T07:46:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.324+0000 7fb8070bcb80 20 client.15213 trim_cache size 0 max 16384 2024-05-27T07:46:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.347+0000 7fb8070bcb80 1 Processor -- start 2024-05-27T07:46:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.347+0000 7fb8070bcb80 1 -- start start 2024-05-27T07:46:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.347+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 0x55aaa1cff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.347+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1cf6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.347+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d0b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.347+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1d18e50 con 0x55aaa1cffd60 2024-05-27T07:46:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.347+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1d19520 con 0x55aaa1b80df0 2024-05-27T07:46:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.347+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1d19bf0 con 0x55aaa1d05ce0 2024-05-27T07:46:11.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1cf6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1cf6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57842/0 (socket says 172.21.15.90:57842) 2024-05-27T07:46:11.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f5200700 1 -- 172.21.15.90:0/4218311232 learned_addr learned my addr 172.21.15.90:0/4218311232 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:11.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/4218311232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 0x55aaa1cff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f6600700 1 --2- 172.21.15.90:0/4218311232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d0b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f5200700 1 -- 172.21.15.90:0/4218311232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1d05ce0 msgr2=0x55aaa1d0b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f5200700 1 --2- 172.21.15.90:0/4218311232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d0b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f5200700 1 -- 172.21.15.90:0/4218311232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 msgr2=0x55aaa1cff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f5200700 1 --2- 172.21.15.90:0/4218311232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 0x55aaa1cff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f5200700 1 -- 172.21.15.90:0/4218311232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa1d08830 con 0x55aaa1cffd60 2024-05-27T07:46:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.348+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/4218311232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 0x55aaa1cff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.349+0000 7fb7f5200700 1 --2- 172.21.15.90:0/4218311232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1cf6930 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0a93d0 tx=0x7fb7dc0e5a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb51c09e5b368cd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.349+0000 7fb7e2000700 1 -- 172.21.15.90:0/4218311232 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc0a9020 con 0x55aaa1cffd60 2024-05-27T07:46:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.349+0000 7fb7e2000700 1 -- 172.21.15.90:0/4218311232 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7dc0a91e0 con 0x55aaa1cffd60 2024-05-27T07:46:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.349+0000 7fb7e2000700 1 -- 172.21.15.90:0/4218311232 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc0a94a0 con 0x55aaa1cffd60 2024-05-27T07:46:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.350+0000 7fb8070bcb80 1 -- 172.21.15.90:0/4218311232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 msgr2=0x55aaa1cf6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.350+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/4218311232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1cf6930 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0a93d0 tx=0x7fb7dc0e5a10 comp rx=0 tx=0).stop 2024-05-27T07:46:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.350+0000 7fb8070bcb80 1 -- 172.21.15.90:0/4218311232 shutdown_connections 2024-05-27T07:46:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.350+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/4218311232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d0b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.350+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/4218311232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 0x55aaa1cff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.350+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/4218311232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1cffd60 0x55aaa1cf6930 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.350+0000 7fb8070bcb80 1 -- 172.21.15.90:0/4218311232 >> 172.21.15.90:0/4218311232 conn(0x55aaa1be0f70 msgr2=0x55aaa1d04600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:11.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.350+0000 7fb8070bcb80 1 -- 172.21.15.90:0/4218311232 shutdown_connections 2024-05-27T07:46:11.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.350+0000 7fb8070bcb80 1 -- 172.21.15.90:0/4218311232 wait complete. 2024-05-27T07:46:11.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.351+0000 7fb8070bcb80 1 -- start start 2024-05-27T07:46:11.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.351+0000 7fb8070bcb80 1 Processor -- start 2024-05-27T07:46:11.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 0x55aaa1d0a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1cffd60 0x55aaa1d0aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb8070bcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d0b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1d1a990 con 0x55aaa1d05ce0 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1d1a2c0 con 0x55aaa1b80df0 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb8070bcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aaa1d1b060 con 0x55aaa1cffd60 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb8070bcb80 10 client.0 _ll_register_callbacks cb 0x55aaa1bc7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f6600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d0b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f6600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d0b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57846/0 (socket says 172.21.15.90:57846) 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f6600700 1 -- 172.21.15.90:0/3332607447 learned_addr learned my addr 172.21.15.90:0/3332607447 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 0x55aaa1d0a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f5200700 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1cffd60 0x55aaa1d0aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f6600700 1 -- 172.21.15.90:0/3332607447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1cffd60 msgr2=0x55aaa1d0aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f6600700 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1cffd60 0x55aaa1d0aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f6600700 1 -- 172.21.15.90:0/3332607447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 msgr2=0x55aaa1d0a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:11.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f6600700 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 0x55aaa1d0a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f6600700 1 -- 172.21.15.90:0/3332607447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa1d22fb0 con 0x55aaa1d05ce0 2024-05-27T07:46:11.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.352+0000 7fb7f5200700 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1cffd60 0x55aaa1d0aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:11.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.353+0000 7fb7f6600700 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d0b3f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009e60 tx=0x7fb7e801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f49711d004a3e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:11.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.353+0000 7fb7d7400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:11.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.353+0000 7fb7d7400700 1 -- 172.21.15.90:0/3332607447 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8041060 con 0x55aaa1d05ce0 2024-05-27T07:46:11.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.353+0000 7fb7d7400700 1 -- 172.21.15.90:0/3332607447 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fb7e8005040 con 0x55aaa1d05ce0 2024-05-27T07:46:11.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.353+0000 7fb8070bcb80 1 -- 172.21.15.90:0/3332607447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7dc0a91e0 con 0x55aaa1d05ce0 2024-05-27T07:46:11.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.353+0000 7fb7d7400700 1 -- 172.21.15.90:0/3332607447 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7e8048050 con 0x55aaa1d05ce0 2024-05-27T07:46:11.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.353+0000 7fb8070bcb80 10 client.15219 fetch_fsmap learned FSMap version 29 2024-05-27T07:46:11.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.353+0000 7fb8070bcb80 1 -- 172.21.15.90:0/3332607447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaa1d24230 con 0x55aaa1d05ce0 2024-05-27T07:46:11.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.354+0000 7fb7d7400700 1 -- 172.21.15.90:0/3332607447 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7dc0a91e0 con 0x55aaa1d05ce0 2024-05-27T07:46:11.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.354+0000 7fb7d7400700 1 -- 172.21.15.90:0/3332607447 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7e8011070 con 0x55aaa1d05ce0 2024-05-27T07:46:11.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.354+0000 7fb8070bcb80 10 client.15219 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.354+0000 7fb8070bcb80 10 client.15219 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.354+0000 7fb8070bcb80 20 client.15219 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.354+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55aaa1d86760 0x55aaa1d87bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.354+0000 7fb8070bcb80 4 client.15219 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.354+0000 7fb8070bcb80 1 -- 172.21.15.90:0/3332607447 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "status"}) -- 0x55aaa1c29520 con 0x55aaa1d86760 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.355+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55aaa1d86760 0x55aaa1d87bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.356+0000 7fb7f5c00700 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55aaa1d86760 0x55aaa1d87bd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fafd34e21f450490 server_cookie=fb127970077f3e0c in_seq=0 out_seq=0 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.356+0000 7fb7d7400700 10 client.15219 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.356+0000 7fb7d7400700 1 -- 172.21.15.90:0/3332607447 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55aaa1c29520 con 0x55aaa1d86760 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.356+0000 7fb7d7400700 10 client.15219 handle_command_reply: tid=0 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.356+0000 7fb8070bcb80 1 client.15219 shutdown 2024-05-27T07:46:11.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.356+0000 7fb8070bcb80 1 -- 172.21.15.90:0/3332607447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 msgr2=0x55aaa1d0b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.356+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d0b3f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009e60 tx=0x7fb7e801ca30 comp rx=0 tx=0).stop 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.357+0000 7fb8070bcb80 1 -- 172.21.15.90:0/3332607447 shutdown_connections 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.357+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aaa1d05ce0 0x55aaa1d0b3f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.357+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55aaa1d86760 0x55aaa1d87bd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.357+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaa1b80df0 0x55aaa1d0a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.357+0000 7fb8070bcb80 1 --2- 172.21.15.90:0/3332607447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa1cffd60 0x55aaa1d0aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.357+0000 7fb7d7400700 0 client.15219 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.357+0000 7fb8070bcb80 1 -- 172.21.15.90:0/3332607447 >> 172.21.15.90:0/3332607447 conn(0x55aaa1be0f70 msgr2=0x55aaa1d030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.357+0000 7fb8070bcb80 1 -- 172.21.15.90:0/3332607447 shutdown_connections 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.357+0000 7fb8070bcb80 1 -- 172.21.15.90:0/3332607447 wait complete. 2024-05-27T07:46:11.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.357+0000 7fb8070bcb80 20 client.15219 trim_cache size 0 max 16384 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34442, 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 220.51167107099999, 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 91, 2024-05-27T07:46:11.359 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:46:11.360 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 335.51258148099998 2024-05-27T07:46:11.360 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:46:11.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 -- 172.21.15.90:0/1145851701 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7d80b2900 msgr2=0x7fb7d80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:11.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7d80b2900 0x7fb7d80b4e10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8006130 tx=0x7fb7e80060a0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 -- 172.21.15.90:0/1145851701 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f0078630 msgr2=0x7fb7f01a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:11.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f0078630 0x7fb7f01a1870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00c9e0 tx=0x7fb7dc00cda0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 -- 172.21.15.90:0/1145851701 shutdown_connections 2024-05-27T07:46:11.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fb7d80b2900 0x7fb7d80b4e10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7f01151e0 0x7fb7f01a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7f0077ce0 0x7fb7f01a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 --2- 172.21.15.90:0/1145851701 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7f0078630 0x7fb7f01a1870 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:11.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 -- 172.21.15.90:0/1145851701 >> 172.21.15.90:0/1145851701 conn(0x7fb7f0102080 msgr2=0x7fb7f0115bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:11.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 -- 172.21.15.90:0/1145851701 shutdown_connections 2024-05-27T07:46:11.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:11.358+0000 7fb7f8600700 1 -- 172.21.15.90:0/1145851701 wait complete. 2024-05-27T07:46:12.203 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:11 smithi090 ceph-mon[31429]: pgmap v537: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:46:12.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:11 smithi136 ceph-mon[35324]: pgmap v537: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:46:12.270 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T07:46:12.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:11 smithi129 ceph-mon[35254]: pgmap v537: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-27T07:46:12.714 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:13.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:13 smithi090 ceph-mon[31429]: pgmap v538: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:46:14.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:13 smithi136 ceph-mon[35324]: pgmap v538: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:46:14.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:13 smithi129 ceph-mon[35254]: pgmap v538: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T07:46:15.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.037+0000 7fcf0a800700 1 -- 172.21.15.90:0/2990324262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf040764a0 msgr2=0x7fcf04114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:15.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.037+0000 7fcf0a800700 1 --2- 172.21.15.90:0/2990324262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf040764a0 0x7fcf04114c30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009990 tx=0x7fcef0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.037+0000 7fcf0a800700 1 -- 172.21.15.90:0/2990324262 shutdown_connections 2024-05-27T07:46:15.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.038+0000 7fcf0a800700 1 --2- 172.21.15.90:0/2990324262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf04115170 0x7fcf04117590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.038+0000 7fcf0a800700 1 --2- 172.21.15.90:0/2990324262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf04075b50 0x7fcf04075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.038+0000 7fcf0a800700 1 --2- 172.21.15.90:0/2990324262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf040764a0 0x7fcf04114c30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.038+0000 7fcf0a800700 1 -- 172.21.15.90:0/2990324262 >> 172.21.15.90:0/2990324262 conn(0x7fcf04079d00 msgr2=0x7fcf0407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:15.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.038+0000 7fcf0a800700 1 -- 172.21.15.90:0/2990324262 shutdown_connections 2024-05-27T07:46:15.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.038+0000 7fcf0a800700 1 -- 172.21.15.90:0/2990324262 wait complete. 2024-05-27T07:46:15.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.039+0000 7fcf0a800700 1 Processor -- start 2024-05-27T07:46:15.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.039+0000 7fcf0a800700 1 -- start start 2024-05-27T07:46:15.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf0a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf04075b50 0x7fcf041a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf0a800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf040764a0 0x7fcf041a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf0a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04115170 0x7fcf041a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf0a800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0411a780 con 0x7fcf04115170 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf0a800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0411a600 con 0x7fcf040764a0 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf0a800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0411a900 con 0x7fcf04075b50 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04115170 0x7fcf041a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04115170 0x7fcf041a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57856/0 (socket says 172.21.15.90:57856) 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf09e00700 1 -- 172.21.15.90:0/4080394465 learned_addr learned my addr 172.21.15.90:0/4080394465 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf09400700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf04075b50 0x7fcf041a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.040+0000 7fcf08a00700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf040764a0 0x7fcf041a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.041+0000 7fcf09e00700 1 -- 172.21.15.90:0/4080394465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf04075b50 msgr2=0x7fcf041a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.041+0000 7fcf09e00700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf04075b50 0x7fcf041a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.041+0000 7fcf09e00700 1 -- 172.21.15.90:0/4080394465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf040764a0 msgr2=0x7fcf041a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:15.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.041+0000 7fcf09e00700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf040764a0 0x7fcf041a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.041+0000 7fcf09e00700 1 -- 172.21.15.90:0/4080394465 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf041a6100 con 0x7fcf04115170 2024-05-27T07:46:15.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.041+0000 7fcf09e00700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04115170 0x7fcf041a5a60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00ada0 tx=0x7fcefc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:15.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.041+0000 7fcf02000700 1 -- 172.21.15.90:0/4080394465 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc017020 con 0x7fcf04115170 2024-05-27T07:46:15.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.041+0000 7fcf02000700 1 -- 172.21.15.90:0/4080394465 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcefc00cd60 con 0x7fcf04115170 2024-05-27T07:46:15.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.041+0000 7fcf0a800700 1 -- 172.21.15.90:0/4080394465 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf041a63f0 con 0x7fcf04115170 2024-05-27T07:46:15.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.041+0000 7fcf02000700 1 -- 172.21.15.90:0/4080394465 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc005280 con 0x7fcf04115170 2024-05-27T07:46:15.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.042+0000 7fcf0a800700 1 -- 172.21.15.90:0/4080394465 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf041add10 con 0x7fcf04115170 2024-05-27T07:46:15.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.046+0000 7fcf02000700 1 -- 172.21.15.90:0/4080394465 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcefc00c8d0 con 0x7fcf04115170 2024-05-27T07:46:15.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.047+0000 7fcf1a94eb80 1 Processor -- start 2024-05-27T07:46:15.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.047+0000 7fcf1a94eb80 1 -- start start 2024-05-27T07:46:15.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.047+0000 7fcf02000700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fceec0b2900 0x7fceec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d734e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7353c0 0x55b02d73fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7357d0 0x55b02d741fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b02d663df0 con 0x55b02d7353c0 2024-05-27T07:46:15.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf02000700 1 -- 172.21.15.90:0/4080394465 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fcefc0e1d00 con 0x7fcf04115170 2024-05-27T07:46:15.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b02d653990 con 0x55b02d5b6df0 2024-05-27T07:46:15.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b02d6579d0 con 0x55b02d7357d0 2024-05-27T07:46:15.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf08a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7353c0 0x55b02d73fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf08a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7353c0 0x55b02d73fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57858/0 (socket says 172.21.15.90:57858) 2024-05-27T07:46:15.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf08a00700 1 -- 172.21.15.90:0/2617107663 learned_addr learned my addr 172.21.15.90:0/2617107663 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:15.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf08a00700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fceec0b2900 0x7fceec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf09e00700 1 --2- 172.21.15.90:0/2617107663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7357d0 0x55b02d741fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.048+0000 7fcf09400700 1 --2- 172.21.15.90:0/2617107663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d734e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.049+0000 7fcf08a00700 1 -- 172.21.15.90:0/2617107663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7357d0 msgr2=0x55b02d741fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:15.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.049+0000 7fcf08a00700 1 --2- 172.21.15.90:0/2617107663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7357d0 0x55b02d741fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.049+0000 7fcf08a00700 1 -- 172.21.15.90:0/2617107663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 msgr2=0x55b02d734e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:15.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.049+0000 7fcf08a00700 1 --2- 172.21.15.90:0/2617107663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d734e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.049+0000 7fcf08a00700 1 -- 172.21.15.90:0/2617107663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02d61c3d0 con 0x55b02d7353c0 2024-05-27T07:46:15.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.049+0000 7fcf08a00700 1 --2- 172.21.15.90:0/2617107663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7353c0 0x55b02d73fa80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcef0003760 tx=0x7fcef000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42f3f02a2e933d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:15.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.050+0000 7fcf08a00700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fceec0b2900 0x7fceec0b4e10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcef0005f50 tx=0x7fcef0022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:15.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.050+0000 7fcef6a00700 1 -- 172.21.15.90:0/2617107663 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef000fc30 con 0x55b02d7353c0 2024-05-27T07:46:15.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.050+0000 7fcef6a00700 1 -- 172.21.15.90:0/2617107663 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcef000fdd0 con 0x55b02d7353c0 2024-05-27T07:46:15.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.050+0000 7fcef6a00700 1 -- 172.21.15.90:0/2617107663 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef0003ab0 con 0x55b02d7353c0 2024-05-27T07:46:15.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.051+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/2617107663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7353c0 msgr2=0x55b02d73fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:15.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.051+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/2617107663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7353c0 0x55b02d73fa80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcef0003760 tx=0x7fcef000bf20 comp rx=0 tx=0).stop 2024-05-27T07:46:15.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.051+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/2617107663 shutdown_connections 2024-05-27T07:46:15.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.051+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/2617107663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7357d0 0x55b02d741fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.051+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/2617107663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d734e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.051+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/2617107663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7353c0 0x55b02d73fa80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.051+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/2617107663 >> 172.21.15.90:0/2617107663 conn(0x55b02d616f70 msgr2=0x55b02d730ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:15.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.051+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/2617107663 shutdown_connections 2024-05-27T07:46:15.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.051+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/2617107663 wait complete. 2024-05-27T07:46:15.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.052+0000 7fcf1a94eb80 1 -- start start 2024-05-27T07:46:15.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.052+0000 7fcf1a94eb80 1 Processor -- start 2024-05-27T07:46:15.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.053+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d5b6df0 0x55b02d73f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.053+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d7353c0 0x55b02d73fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.053+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d740070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.053+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b02d559f50 con 0x55b02d7357d0 2024-05-27T07:46:15.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.053+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b02d5f4710 con 0x55b02d7353c0 2024-05-27T07:46:15.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.053+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b02d7428e0 con 0x55b02d5b6df0 2024-05-27T07:46:15.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.053+0000 7fcf1a94eb80 10 client.0 _ll_register_callbacks cb 0x55b02d5fdc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:15.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.053+0000 7fcf09400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d5b6df0 0x55b02d73f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.053+0000 7fcf08a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d7353c0 0x55b02d73fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.053+0000 7fcf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d740070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.054+0000 7fcf08a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d7353c0 0x55b02d73fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:54926/0 (socket says 172.21.15.90:54926) 2024-05-27T07:46:15.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.054+0000 7fcf08a00700 1 -- 172.21.15.90:0/3406178805 learned_addr learned my addr 172.21.15.90:0/3406178805 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:15.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.054+0000 7fcf09e00700 1 -- 172.21.15.90:0/3406178805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d5b6df0 msgr2=0x55b02d73f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:15.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.054+0000 7fcf09e00700 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d5b6df0 0x55b02d73f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.054+0000 7fcf09e00700 1 -- 172.21.15.90:0/3406178805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d7353c0 msgr2=0x55b02d73fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:15.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.054+0000 7fcf09e00700 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d7353c0 0x55b02d73fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.054+0000 7fcf09e00700 1 -- 172.21.15.90:0/3406178805 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02d5a3880 con 0x55b02d7357d0 2024-05-27T07:46:15.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.054+0000 7fcf09e00700 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d740070 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0df3e0 tx=0x7fcefc004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148cc52c79971b18 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:15.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.054+0000 7fceeb400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:15.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.055+0000 7fceeb400700 1 -- 172.21.15.90:0/3406178805 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0a96c0 con 0x55b02d7357d0 2024-05-27T07:46:15.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.055+0000 7fceeb400700 1 -- 172.21.15.90:0/3406178805 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fcefc00a160 con 0x55b02d7357d0 2024-05-27T07:46:15.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.055+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3406178805 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef000fdd0 con 0x55b02d7357d0 2024-05-27T07:46:15.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.055+0000 7fceeb400700 1 -- 172.21.15.90:0/3406178805 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcefc0a8fc0 con 0x55b02d7357d0 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.055+0000 7fcf1a94eb80 10 client.15231 fetch_fsmap learned FSMap version 29 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.055+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3406178805 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b02d624010 con 0x55b02d7357d0 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.055+0000 7fceeb400700 1 -- 172.21.15.90:0/3406178805 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcef000fdd0 con 0x55b02d7357d0 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.056+0000 7fceeb400700 1 -- 172.21.15.90:0/3406178805 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcefc0ab070 con 0x55b02d7357d0 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.056+0000 7fcf1a94eb80 10 client.15231 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.056+0000 7fcf1a94eb80 10 client.15231 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.056+0000 7fcf1a94eb80 20 client.15231 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.056+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b02d74f950 0x55b02d750dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.056+0000 7fcf1a94eb80 4 client.15231 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.056+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3406178805 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b02d665d40 con 0x55b02d74f950 2024-05-27T07:46:15.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.057+0000 7fcf09400700 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b02d74f950 0x55b02d750dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.058+0000 7fcf09400700 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b02d74f950 0x55b02d750dc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522b8c3809588b12 server_cookie=cb4b1a4ab22d1a0d in_seq=0 out_seq=0 2024-05-27T07:46:15.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.058+0000 7fceeb400700 10 client.15231 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:15.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fceeb400700 1 -- 172.21.15.90:0/3406178805 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b02d665d40 con 0x55b02d74f950 2024-05-27T07:46:15.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fceeb400700 10 client.15231 handle_command_reply: tid=0 2024-05-27T07:46:15.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 client.15231 shutdown 2024-05-27T07:46:15.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3406178805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 msgr2=0x55b02d740070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:15.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d740070 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0df3e0 tx=0x7fcefc004c10 comp rx=0 tx=0).stop 2024-05-27T07:46:15.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3406178805 shutdown_connections 2024-05-27T07:46:15.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b02d74f950 0x55b02d750dc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d740070 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d5b6df0 0x55b02d73f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3406178805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d7353c0 0x55b02d73fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fceeb400700 0 client.15231 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:15.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3406178805 >> 172.21.15.90:0/3406178805 conn(0x55b02d616f70 msgr2=0x55b02d72ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:15.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3406178805 shutdown_connections 2024-05-27T07:46:15.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3406178805 wait complete. 2024-05-27T07:46:15.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.060+0000 7fcf1a94eb80 20 client.15231 trim_cache size 0 max 16384 2024-05-27T07:46:15.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.083+0000 7fcf1a94eb80 1 Processor -- start 2024-05-27T07:46:15.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.083+0000 7fcf1a94eb80 1 -- start start 2024-05-27T07:46:15.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.083+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d73ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.083+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7353c0 0x55b02d730e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.083+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d74bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.083+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b02d74f100 con 0x55b02d7357d0 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.083+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b02d74f7d0 con 0x55b02d5b6df0 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.083+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b02d74fea0 con 0x55b02d7353c0 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.084+0000 7fcf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d74bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.084+0000 7fcf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d74bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57882/0 (socket says 172.21.15.90:57882) 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.084+0000 7fcf09e00700 1 -- 172.21.15.90:0/1016687771 learned_addr learned my addr 172.21.15.90:0/1016687771 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.084+0000 7fcf08a00700 1 --2- 172.21.15.90:0/1016687771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7353c0 0x55b02d730e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.084+0000 7fcf09400700 1 --2- 172.21.15.90:0/1016687771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d73ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.084+0000 7fcf09e00700 1 -- 172.21.15.90:0/1016687771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7353c0 msgr2=0x55b02d730e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.084+0000 7fcf09e00700 1 --2- 172.21.15.90:0/1016687771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7353c0 0x55b02d730e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.084+0000 7fcf09e00700 1 -- 172.21.15.90:0/1016687771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 msgr2=0x55b02d73ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.084+0000 7fcf09e00700 1 --2- 172.21.15.90:0/1016687771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d73ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.084+0000 7fcf09e00700 1 -- 172.21.15.90:0/1016687771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02d73e910 con 0x55b02d7357d0 2024-05-27T07:46:15.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.085+0000 7fcf09e00700 1 --2- 172.21.15.90:0/1016687771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d74bb80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00ad70 tx=0x7fcefc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7041ff343a9b45 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:15.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.085+0000 7fcef6000700 1 -- 172.21.15.90:0/1016687771 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0a96c0 con 0x55b02d7357d0 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.085+0000 7fcef6000700 1 -- 172.21.15.90:0/1016687771 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcefc0ad370 con 0x55b02d7357d0 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.085+0000 7fcef6000700 1 -- 172.21.15.90:0/1016687771 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0a7c90 con 0x55b02d7357d0 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.086+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/1016687771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 msgr2=0x55b02d74bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.086+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/1016687771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d74bb80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00ad70 tx=0x7fcefc00c7f0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.086+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/1016687771 shutdown_connections 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.086+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/1016687771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d74bb80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.086+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/1016687771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d73ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.086+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/1016687771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7353c0 0x55b02d730e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.086+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/1016687771 >> 172.21.15.90:0/1016687771 conn(0x55b02d616f70 msgr2=0x55b02d730770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.086+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/1016687771 shutdown_connections 2024-05-27T07:46:15.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.086+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/1016687771 wait complete. 2024-05-27T07:46:15.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.086+0000 7fcf1a94eb80 1 -- start start 2024-05-27T07:46:15.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.087+0000 7fcf1a94eb80 1 Processor -- start 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.087+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d754180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.087+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7353c0 0x55b02d7546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.087+0000 7fcf1a94eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d754c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.087+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b02d750c40 con 0x55b02d7357d0 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.087+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b02d750570 con 0x55b02d5b6df0 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.087+0000 7fcf1a94eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b02d751310 con 0x55b02d7353c0 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.087+0000 7fcf1a94eb80 10 client.0 _ll_register_callbacks cb 0x55b02d5fdc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.088+0000 7fcf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d754c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.088+0000 7fcf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d754c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57894/0 (socket says 172.21.15.90:57894) 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.088+0000 7fcf09e00700 1 -- 172.21.15.90:0/3293464464 learned_addr learned my addr 172.21.15.90:0/3293464464 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:15.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.088+0000 7fcf09400700 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d754180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.088+0000 7fcf08a00700 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7353c0 0x55b02d7546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.088+0000 7fcf09e00700 1 -- 172.21.15.90:0/3293464464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7353c0 msgr2=0x55b02d7546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.088+0000 7fcf09e00700 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7353c0 0x55b02d7546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.088+0000 7fcf09e00700 1 -- 172.21.15.90:0/3293464464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 msgr2=0x55b02d754180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.088+0000 7fcf09e00700 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d754180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.088+0000 7fcf09e00700 1 -- 172.21.15.90:0/3293464464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02d758a60 con 0x55b02d7357d0 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.089+0000 7fcf09e00700 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d754c00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0a49f0 tx=0x7fcefc00a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76c8ba590d8d7ad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.089+0000 7fceeb400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.089+0000 7fceeb400700 1 -- 172.21.15.90:0/3293464464 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0a8220 con 0x55b02d7357d0 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.089+0000 7fceeb400700 1 -- 172.21.15.90:0/3293464464 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fcefc0a83c0 con 0x55b02d7357d0 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.089+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3293464464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcefc0ad370 con 0x55b02d7357d0 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.089+0000 7fceeb400700 1 -- 172.21.15.90:0/3293464464 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcefc0a7c90 con 0x55b02d7357d0 2024-05-27T07:46:15.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.089+0000 7fcf1a94eb80 10 client.15237 fetch_fsmap learned FSMap version 29 2024-05-27T07:46:15.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.089+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3293464464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b02d758a60 con 0x55b02d7357d0 2024-05-27T07:46:15.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.089+0000 7fceeb400700 1 -- 172.21.15.90:0/3293464464 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcefc0ad370 con 0x55b02d7357d0 2024-05-27T07:46:15.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.090+0000 7fceeb400700 1 -- 172.21.15.90:0/3293464464 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcefc0ab070 con 0x55b02d7357d0 2024-05-27T07:46:15.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.090+0000 7fcf1a94eb80 10 client.15237 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T07:46:15.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.090+0000 7fcf1a94eb80 10 client.15237 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T07:46:15.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.090+0000 7fcf1a94eb80 20 client.15237 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:15.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.090+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b02d7bd390 0x55b02d7be800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:15.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.090+0000 7fcf1a94eb80 4 client.15237 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T07:46:15.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.090+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3293464464 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "status"}) -- 0x55b02d65f520 con 0x55b02d7bd390 2024-05-27T07:46:15.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.090+0000 7fcf09400700 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b02d7bd390 0x55b02d7be800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:15.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fcf09400700 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b02d7bd390 0x55b02d7be800 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac8de7847e8b7ee server_cookie=bf788d080906ef8d in_seq=0 out_seq=0 2024-05-27T07:46:15.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fceeb400700 10 client.15237 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:15.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fceeb400700 1 -- 172.21.15.90:0/3293464464 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b02d65f520 con 0x55b02d7bd390 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fceeb400700 10 client.15237 handle_command_reply: tid=0 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fcf1a94eb80 1 client.15237 shutdown 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3293464464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 msgr2=0x55b02d754c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d754c00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0a49f0 tx=0x7fcefc00a0c0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3293464464 shutdown_connections 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b02d7bd390 0x55b02d7be800 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b02d7357d0 0x55b02d754c00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b02d5b6df0 0x55b02d754180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fcf1a94eb80 1 --2- 172.21.15.90:0/3293464464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b02d7353c0 0x55b02d7546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fceeb400700 0 client.15237 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.092+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3293464464 >> 172.21.15.90:0/3293464464 conn(0x55b02d616f70 msgr2=0x55b02d7368a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.093+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3293464464 shutdown_connections 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.093+0000 7fcf1a94eb80 1 -- 172.21.15.90:0/3293464464 wait complete. 2024-05-27T07:46:15.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.093+0000 7fcf1a94eb80 20 client.15237 trim_cache size 0 max 16384 2024-05-27T07:46:15.094 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:46:15.094 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b0214938-1bfa-11ef-bc9b-c7b262605968", 2024-05-27T07:46:15.094 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-05-27T07:46:15.094 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34472, 2024-05-27T07:46:15.094 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-05-27T07:46:15.094 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-05-27T07:46:15.094 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-05-27T07:46:15.094 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 303.24252624299999, 2024-05-27T07:46:15.094 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-05-27T07:46:15.095 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 91, 2024-05-27T07:46:15.095 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 89, 2024-05-27T07:46:15.095 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 321.36552618399998 2024-05-27T07:46:15.095 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:46:15.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.093+0000 7fcf0a800700 1 -- 172.21.15.90:0/4080394465 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fceec0b2900 msgr2=0x7fceec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:15.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.093+0000 7fcf0a800700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fceec0b2900 0x7fceec0b4e10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcef0005f50 tx=0x7fcef0022000 comp rx=0 tx=0).stop 2024-05-27T07:46:15.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.093+0000 7fcf0a800700 1 -- 172.21.15.90:0/4080394465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04115170 msgr2=0x7fcf041a5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.093+0000 7fcf0a800700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04115170 0x7fcf041a5a60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00ada0 tx=0x7fcefc00c330 comp rx=0 tx=0).stop 2024-05-27T07:46:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.094+0000 7fcf0a800700 1 -- 172.21.15.90:0/4080394465 shutdown_connections 2024-05-27T07:46:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.094+0000 7fcf0a800700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fceec0b2900 0x7fceec0b4e10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.094+0000 7fcf0a800700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf04115170 0x7fcf041a5a60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.094+0000 7fcf0a800700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf04075b50 0x7fcf041a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.094+0000 7fcf0a800700 1 --2- 172.21.15.90:0/4080394465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf040764a0 0x7fcf041a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.094+0000 7fcf0a800700 1 -- 172.21.15.90:0/4080394465 >> 172.21.15.90:0/4080394465 conn(0x7fcf04079d00 msgr2=0x7fcf04101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.094+0000 7fcf0a800700 1 -- 172.21.15.90:0/4080394465 shutdown_connections 2024-05-27T07:46:15.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:15.094+0000 7fcf0a800700 1 -- 172.21.15.90:0/4080394465 wait complete. 2024-05-27T07:46:15.989 INFO:teuthology.run_tasks:Running task sequential... 2024-05-27T07:46:16.000 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-27T07:46:16.000 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-27T07:46:16.001 INFO:teuthology.task.print:Enabling mgr modules 2024-05-27T07:46:16.001 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-27T07:46:16.002 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:46:16.002 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T07:46:16.002 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-27T07:46:16.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:15 smithi090 ceph-mon[31429]: pgmap v539: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:46:16.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:15 smithi136 ceph-mon[35324]: pgmap v539: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:46:16.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.283+0000 7fdc95a00640 1 -- 172.21.15.90:0/2511005236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc90105cb0 msgr2=0x7fdc901069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:16.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.283+0000 7fdc95a00640 1 --2- 172.21.15.90:0/2511005236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc90105cb0 0x7fdc901069b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009900 tx=0x7fdc8002a410 comp rx=0 tx=0).stop 2024-05-27T07:46:16.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.283+0000 7fdc95a00640 1 -- 172.21.15.90:0/2511005236 shutdown_connections 2024-05-27T07:46:16.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.283+0000 7fdc95a00640 1 --2- 172.21.15.90:0/2511005236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc90106ef0 0x7fdc90107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:16.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.283+0000 7fdc95a00640 1 --2- 172.21.15.90:0/2511005236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc9010bbf0 0x7fdc9010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:16.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.283+0000 7fdc95a00640 1 --2- 172.21.15.90:0/2511005236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc90105cb0 0x7fdc901069b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:16.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.283+0000 7fdc95a00640 1 -- 172.21.15.90:0/2511005236 >> 172.21.15.90:0/2511005236 conn(0x7fdc90100ec0 msgr2=0x7fdc90103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:16.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.283+0000 7fdc95a00640 1 -- 172.21.15.90:0/2511005236 shutdown_connections 2024-05-27T07:46:16.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.283+0000 7fdc95a00640 1 -- 172.21.15.90:0/2511005236 wait complete. 2024-05-27T07:46:16.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.284+0000 7fdc95a00640 1 Processor -- start 2024-05-27T07:46:16.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.285+0000 7fdc95a00640 1 -- start start 2024-05-27T07:46:16.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.285+0000 7fdc95a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc90105cb0 0x7fdc9019c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:16.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.285+0000 7fdc95a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc90106ef0 0x7fdc9019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:16.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.285+0000 7fdc95a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc9010bbf0 0x7fdc901a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:16.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.285+0000 7fdc95a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90111660 con 0x7fdc90106ef0 2024-05-27T07:46:16.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.285+0000 7fdc95a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901114e0 con 0x7fdc9010bbf0 2024-05-27T07:46:16.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.285+0000 7fdc95a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc901117e0 con 0x7fdc90105cb0 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.285+0000 7fdc87400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc90106ef0 0x7fdc9019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.285+0000 7fdc87400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc90106ef0 0x7fdc9019c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57922/0 (socket says 172.21.15.90:57922) 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.285+0000 7fdc87400640 1 -- 172.21.15.90:0/101180023 learned_addr learned my addr 172.21.15.90:0/101180023 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.286+0000 7fdc95000640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc9010bbf0 0x7fdc901a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.286+0000 7fdc8fe00640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc90105cb0 0x7fdc9019c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.286+0000 7fdc87400640 1 -- 172.21.15.90:0/101180023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc90105cb0 msgr2=0x7fdc9019c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.286+0000 7fdc87400640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc90105cb0 0x7fdc9019c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.286+0000 7fdc87400640 1 -- 172.21.15.90:0/101180023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc9010bbf0 msgr2=0x7fdc901a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.286+0000 7fdc87400640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc9010bbf0 0x7fdc901a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.286+0000 7fdc87400640 1 -- 172.21.15.90:0/101180023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc901a3e30 con 0x7fdc90106ef0 2024-05-27T07:46:16.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.286+0000 7fdc87400640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc90106ef0 0x7fdc9019c830 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800d950 tx=0x7fdc7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:16.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.287+0000 7fdc8d600640 1 -- 172.21.15.90:0/101180023 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78017030 con 0x7fdc90106ef0 2024-05-27T07:46:16.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.287+0000 7fdc95a00640 1 -- 172.21.15.90:0/101180023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc901a4120 con 0x7fdc90106ef0 2024-05-27T07:46:16.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.287+0000 7fdc8d600640 1 -- 172.21.15.90:0/101180023 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc78004020 con 0x7fdc90106ef0 2024-05-27T07:46:16.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.287+0000 7fdc8d600640 1 -- 172.21.15.90:0/101180023 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc780055d0 con 0x7fdc90106ef0 2024-05-27T07:46:16.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.287+0000 7fdc95a00640 1 -- 172.21.15.90:0/101180023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc901a4670 con 0x7fdc90106ef0 2024-05-27T07:46:16.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.288+0000 7fdc95a00640 1 -- 172.21.15.90:0/101180023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc9010aa00 con 0x7fdc90106ef0 2024-05-27T07:46:16.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.291+0000 7fdc8d600640 1 -- 172.21.15.90:0/101180023 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdc78009a30 con 0x7fdc90106ef0 2024-05-27T07:46:16.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.292+0000 7fdc8d600640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdc740b27e0 0x7fdc740b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:16.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.292+0000 7fdc8d600640 1 -- 172.21.15.90:0/101180023 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fdc780e2770 con 0x7fdc90106ef0 2024-05-27T07:46:16.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.292+0000 7fdc8fe00640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdc740b27e0 0x7fdc740b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:16.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.293+0000 7fdc8fe00640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdc740b27e0 0x7fdc740b4cf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc800057c0 tx=0x7fdc80005720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:16.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.295+0000 7fdc8d600640 1 -- 172.21.15.90:0/101180023 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc780aa100 con 0x7fdc90106ef0 2024-05-27T07:46:16.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:15 smithi129 ceph-mon[35254]: pgmap v539: 129 pgs: 129 active+clean; 650 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-27T07:46:16.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.557+0000 7fdc95a00640 1 -- 172.21.15.90:0/101180023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fdc900638c0 con 0x7fdc90106ef0 2024-05-27T07:46:16.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.927+0000 7fdc8d600640 1 -- 172.21.15.90:0/101180023 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fdc780a9a00 con 0x7fdc90106ef0 2024-05-27T07:46:16.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.931+0000 7fdc95a00640 1 -- 172.21.15.90:0/101180023 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdc740b27e0 msgr2=0x7fdc740b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:16.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.931+0000 7fdc95a00640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdc740b27e0 0x7fdc740b4cf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc800057c0 tx=0x7fdc80005720 comp rx=0 tx=0).stop 2024-05-27T07:46:16.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.931+0000 7fdc95a00640 1 -- 172.21.15.90:0/101180023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc90106ef0 msgr2=0x7fdc9019c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:16.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.931+0000 7fdc95a00640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc90106ef0 0x7fdc9019c830 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800d950 tx=0x7fdc7800de10 comp rx=0 tx=0).stop 2024-05-27T07:46:16.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.931+0000 7fdc95a00640 1 -- 172.21.15.90:0/101180023 shutdown_connections 2024-05-27T07:46:16.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.931+0000 7fdc95a00640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fdc740b27e0 0x7fdc740b4cf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:16.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.931+0000 7fdc95a00640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdc9010bbf0 0x7fdc901a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:16.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.931+0000 7fdc95a00640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc90106ef0 0x7fdc9019c830 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:16.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.931+0000 7fdc95a00640 1 --2- 172.21.15.90:0/101180023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc90105cb0 0x7fdc9019c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:16.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.931+0000 7fdc95a00640 1 -- 172.21.15.90:0/101180023 >> 172.21.15.90:0/101180023 conn(0x7fdc90100ec0 msgr2=0x7fdc90107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:16.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.932+0000 7fdc95a00640 1 -- 172.21.15.90:0/101180023 shutdown_connections 2024-05-27T07:46:16.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:16.932+0000 7fdc95a00640 1 -- 172.21.15.90:0/101180023 wait complete. 2024-05-27T07:46:16.965 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-27T07:46:16.976 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-27T07:46:16.976 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-27T07:46:16.976 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-27T07:46:16.976 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T07:46:16.977 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-05-27T07:46:16.977 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-05-27T07:46:16.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:16.977 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-05-27T07:46:17.019 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-27T07:46:17.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:17.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-27T07:46:17.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:17.134 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-05-27T07:46:17.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:16 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/101180023' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T07:46:17.159 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: ignoring --setuser ceph since I am not root 2024-05-27T07:46:17.159 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: ignoring --setgroup ceph since I am not root 2024-05-27T07:46:17.161 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:16 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/101180023' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T07:46:17.246 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-27T07:46:17.246 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-27T07:46:17.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:16 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/101180023' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T07:46:17.313 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: ignoring --setuser ceph since I am not root 2024-05-27T07:46:17.313 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: ignoring --setgroup ceph since I am not root 2024-05-27T07:46:17.313 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:17 smithi129 ceph-mgr[36634]: -- 172.21.15.129:0/378655669 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a966ec5a0 con 0x557a9b476900 2024-05-27T07:46:17.462 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:17.339+0000 7f4f1baae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:46:17.499 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: ignoring --setuser ceph since I am not root 2024-05-27T07:46:17.499 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: ignoring --setgroup ceph since I am not root 2024-05-27T07:46:17.499 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:17 smithi136 ceph-mgr[37053]: -- 172.21.15.136:0/4004302629 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9ca44e5a0 con 0x55e9cf1d8480 2024-05-27T07:46:17.499 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:17.345+0000 7f8dd75ea200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:46:17.590 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:17.330+0000 7f3757997200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T07:46:17.694 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:17.841 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:17.590+0000 7f4f1baae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:46:17.841 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:17.745+0000 7f4f1baae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:46:17.930 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:17.589+0000 7f3757997200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:46:17.930 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:17.723+0000 7f3757997200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:46:17.930 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:17.600+0000 7f8dd75ea200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:46:17.930 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:17.734+0000 7f8dd75ea200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T07:46:18.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:17 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/101180023' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T07:46:18.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:17 smithi090 ceph-mon[31429]: mgrmap e19: x(active, since 15m), standbys: z, y 2024-05-27T07:46:18.236 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:18.121+0000 7f3757997200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:46:18.237 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:17 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/101180023' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T07:46:18.237 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:17 smithi129 ceph-mon[35254]: mgrmap e19: x(active, since 15m), standbys: z, y 2024-05-27T07:46:18.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:17 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/101180023' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T07:46:18.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:17 smithi136 ceph-mon[35324]: mgrmap e19: x(active, since 15m), standbys: z, y 2024-05-27T07:46:18.248 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:18.133+0000 7f8dd75ea200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:46:18.462 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:18.158+0000 7f4f1baae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T07:46:18.462 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:18.282+0000 7f4f1baae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:46:18.563 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:18.236+0000 7f3757997200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:46:18.748 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:18.249+0000 7f8dd75ea200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T07:46:20.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.033+0000 7fcd31800700 1 -- 172.21.15.90:0/1152464292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd2c0764a0 msgr2=0x7fcd2c114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:20.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.033+0000 7fcd31800700 1 --2- 172.21.15.90:0/1152464292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd2c0764a0 0x7fcd2c114e20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009990 tx=0x7fcd14009ca0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.034+0000 7fcd31800700 1 -- 172.21.15.90:0/1152464292 shutdown_connections 2024-05-27T07:46:20.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.034+0000 7fcd31800700 1 --2- 172.21.15.90:0/1152464292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c115360 0x7fcd2c117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.034+0000 7fcd31800700 1 --2- 172.21.15.90:0/1152464292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd2c075b50 0x7fcd2c075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.034+0000 7fcd31800700 1 --2- 172.21.15.90:0/1152464292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd2c0764a0 0x7fcd2c114e20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.034+0000 7fcd31800700 1 -- 172.21.15.90:0/1152464292 >> 172.21.15.90:0/1152464292 conn(0x7fcd2c100030 msgr2=0x7fcd2c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:20.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.034+0000 7fcd31800700 1 -- 172.21.15.90:0/1152464292 shutdown_connections 2024-05-27T07:46:20.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.034+0000 7fcd31800700 1 -- 172.21.15.90:0/1152464292 wait complete. 2024-05-27T07:46:20.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.035+0000 7fcd31800700 1 Processor -- start 2024-05-27T07:46:20.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.035+0000 7fcd31800700 1 -- start start 2024-05-27T07:46:20.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.035+0000 7fcd31800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd2c075b50 0x7fcd2c1a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:20.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd31800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd2c0764a0 0x7fcd2c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:20.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd31800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c115360 0x7fcd2c1a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:20.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd31800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c11a960 con 0x7fcd2c075b50 2024-05-27T07:46:20.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd31800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c11a7e0 con 0x7fcd2c115360 2024-05-27T07:46:20.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd31800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c11aae0 con 0x7fcd2c0764a0 2024-05-27T07:46:20.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd2aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd2c075b50 0x7fcd2c1a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:20.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd2aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd2c075b50 0x7fcd2c1a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46158/0 (socket says 172.21.15.90:46158) 2024-05-27T07:46:20.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd2aa00700 1 -- 172.21.15.90:0/4270759062 learned_addr learned my addr 172.21.15.90:0/4270759062 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:20.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd23e00700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd2c0764a0 0x7fcd2c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd2aa00700 1 -- 172.21.15.90:0/4270759062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd2c0764a0 msgr2=0x7fcd2c1a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd2aa00700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd2c0764a0 0x7fcd2c1a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd2aa00700 1 -- 172.21.15.90:0/4270759062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c115360 msgr2=0x7fcd2c1a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd2aa00700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c115360 0x7fcd2c1a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.036+0000 7fcd2aa00700 1 -- 172.21.15.90:0/4270759062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c1a61f0 con 0x7fcd2c075b50 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.037+0000 7fcd2aa00700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd2c075b50 0x7fcd2c1a13d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00d950 tx=0x7fcd1c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.037+0000 7fcd23400700 1 -- 172.21.15.90:0/4270759062 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c016070 con 0x7fcd2c075b50 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.037+0000 7fcd31800700 1 -- 172.21.15.90:0/4270759062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2c1a64e0 con 0x7fcd2c075b50 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.037+0000 7fcd23400700 1 -- 172.21.15.90:0/4270759062 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcd1c010460 con 0x7fcd2c075b50 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.037+0000 7fcd23400700 1 -- 172.21.15.90:0/4270759062 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c00f620 con 0x7fcd2c075b50 2024-05-27T07:46:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.037+0000 7fcd31800700 1 -- 172.21.15.90:0/4270759062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2c1adeb0 con 0x7fcd2c075b50 2024-05-27T07:46:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.038+0000 7fcd31800700 1 -- 172.21.15.90:0/4270759062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2c077290 con 0x7fcd2c075b50 2024-05-27T07:46:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.043+0000 7fcd23400700 1 -- 172.21.15.90:0/4270759062 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fcd1c019070 con 0x7fcd2c075b50 2024-05-27T07:46:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.044+0000 7fcd23400700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd100b2950 0x7fcd100b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:20.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.045+0000 7fcd23e00700 1 -- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd100b2950 msgr2=0x7fcd100b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2854405112 2024-05-27T07:46:20.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.045+0000 7fcd23400700 1 -- 172.21.15.90:0/4270759062 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fcd1c0e2570 con 0x7fcd2c075b50 2024-05-27T07:46:20.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.045+0000 7fcd23e00700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd100b2950 0x7fcd100b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:46:20.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.047+0000 7fcd23400700 1 -- 172.21.15.90:0/4270759062 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd1c0a9ea0 con 0x7fcd2c075b50 2024-05-27T07:46:20.063 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:19.770+0000 7f3757997200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:46:20.063 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:19.998+0000 7f3757997200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:46:20.178 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:19.915+0000 7f4f1baae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:46:20.233 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:19.786+0000 7f8dd75ea200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T07:46:20.234 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:20.015+0000 7f8dd75ea200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:46:20.234 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:20.233+0000 7f8dd75ea200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:46:20.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.246+0000 7fcd23e00700 1 -- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd100b2950 msgr2=0x7fcd100b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2854405112 2024-05-27T07:46:20.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.246+0000 7fcd23e00700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd100b2950 0x7fcd100b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:46:20.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.343+0000 7fcd31800700 1 -- 172.21.15.90:0/4270759062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcd2c02e5e0 con 0x7fcd2c075b50 2024-05-27T07:46:20.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.344+0000 7fcd23400700 1 -- 172.21.15.90:0/4270759062 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7fcd1c0a97a0 con 0x7fcd2c075b50 2024-05-27T07:46:20.343 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-27T07:46:20.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 -- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd100b2950 msgr2=0x7fcd100b4e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:46:20.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd100b2950 0x7fcd100b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 -- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd2c075b50 msgr2=0x7fcd2c1a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:20.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd2c075b50 0x7fcd2c1a13d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c00d950 tx=0x7fcd1c00dd10 comp rx=0 tx=0).stop 2024-05-27T07:46:20.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 -- 172.21.15.90:0/4270759062 shutdown_connections 2024-05-27T07:46:20.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7fcd100b2950 0x7fcd100b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd2c115360 0x7fcd2c1a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd2c075b50 0x7fcd2c1a13d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 --2- 172.21.15.90:0/4270759062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd2c0764a0 0x7fcd2c1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:20.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 -- 172.21.15.90:0/4270759062 >> 172.21.15.90:0/4270759062 conn(0x7fcd2c100030 msgr2=0x7fcd2c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:20.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 -- 172.21.15.90:0/4270759062 shutdown_connections 2024-05-27T07:46:20.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:20.347+0000 7fcd31800700 1 -- 172.21.15.90:0/4270759062 wait complete. 2024-05-27T07:46:20.462 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:20.178+0000 7f4f1baae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T07:46:20.462 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:20.402+0000 7f4f1baae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:46:20.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:20 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/4270759062' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:46:20.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:20 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/4270759062' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:46:20.562 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:20 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/4270759062' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T07:46:20.563 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:20.217+0000 7f3757997200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T07:46:20.990 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:20.602+0000 7f3757997200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:46:20.990 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:20.726+0000 7f3757997200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:46:20.999 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:20.619+0000 7f8dd75ea200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:46:20.999 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:20.745+0000 7f8dd75ea200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:46:21.195 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:20.800+0000 7f4f1baae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T07:46:21.196 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:20.935+0000 7f4f1baae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T07:46:21.248 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:20.989+0000 7f3757997200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:46:21.256 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T07:46:21.257 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T07:46:21.257 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-05-27T07:46:21.257 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T07:46:21.257 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-05-27T07:46:21.257 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T07:46:21.257 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T07:46:21.257 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T07:46:21.257 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-05-27T07:46:21.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:21.258 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-05-27T07:46:21.267 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:21.007+0000 7f8dd75ea200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:46:21.312 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T07:46:21.312 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T07:46:21.312 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T07:46:21.312 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:46:21.312 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-05-27T07:46:21.312 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 41582sec preferred_lft 41582sec 2024-05-27T07:46:21.313 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-05-27T07:46:21.314 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-05-27T07:46:21.314 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T07:46:21.314 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:46:21.314 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-05-27T07:46:21.314 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-05-27T07:46:21.314 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-05-27T07:46:21.314 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-05-27T07:46:21.314 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T07:46:21.315 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-05-27T07:46:21.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:46:21.445 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:21.196+0000 7f4f1baae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T07:46:21.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:46:21.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:21.540 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T07:46:21.563 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:21.248+0000 7f3757997200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:46:21.599 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-05-27T07:46:21.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:21.603 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-05-27T07:46:21.649 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-05-27T07:46:21.649 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T07:46:21.650 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T07:46:21.650 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T07:46:21.650 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T07:46:21.651 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:46:21.651 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-05-27T07:46:21.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T07:46:21.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T07:46:21.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T07:46:21.651 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-05-27T07:46:21.712 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:21.475+0000 7f4f1baae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:46:21.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:46:21.748 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:21.266+0000 7f8dd75ea200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T07:46:21.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:46:21.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:21.844 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-05-27T07:46:21.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:21.889 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-05-27T07:46:21.978 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:46:21.978 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-05-27T07:46:21.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--tmp-cephfs 2024-05-27T07:46:21.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-05-27T07:46:21.978 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-05-27T07:46:22.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:46:22.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:46:22.144 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-05-27T07:46:22.144 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:46:22.144 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-05-27T07:46:22.144 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-05-27T07:46:22.144 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T07:46:22.145 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-05-27T07:46:22.145 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-05-27T07:46:22.145 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-05-27T07:46:22.145 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-05-27T07:46:22.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:46:22.313 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:22.040+0000 7f3757997200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:46:22.352 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:22 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:22.332+0000 7f4f1baae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:46:22.354 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:22.061+0000 7f8dd75ea200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T07:46:22.355 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:22.354+0000 7f8dd75ea200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:46:22.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:46:22.400 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T07:46:22.401 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-05-27T07:46:22.401 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-05-27T07:46:22.401 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T07:46:22.401 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-05-27T07:46:22.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T07:46:22.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:46:22.591 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-05-27T07:46:22.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T07:46:22.592 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /tmp/cephfs 2024-05-27T07:46:22.632 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/tmp/cephfs' 2024-05-27T07:46:22.632 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T07:46:22.633 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /tmp/cephfs 2024-05-27T07:46:22.723 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-05-27T07:46:22.797 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:46:22.812 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:22.332+0000 7f3757997200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-05-27T07:46:22.843 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:46:22.844 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/9e1bcc8f262b04006b2646c82924c60582162fe1749f0a10dc84210cf998766f/merged 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/da074e182c088100c0366956aa071553c14b7d365bcfe7ba452de0a09a3e6cdb/merged 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/74183af9942d0e0b14483025c64637abb9123624a14b778b4cc4ad7a66c834d6/merged 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c7f3c7639caab68be3492caf2029c5c0dd8c399e09fd109f58a8afd7d9d82fcf/merged 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/dbd87aa827d3ec73230a009e0e92ea3edaea081f5ff915ae8e52bceb8ee4093b/merged 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/965856f8e1e17ab9ed7fd5db92818a04751564d3a836793e9712ae733fbcd15b/merged 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f94a5ad33b8640fbbaf471ef9e89a72c75e3fa026ba1cc36be0c985a34172a14/merged 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/31385d1d7f286038f0ca0a5f821a9012ffd5aad96c380bb1eb781def5c2ec6d0/merged 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1d6e9600c151645e62fd289e9f10091935bc56ae566ab5a42dcc8d3214014af1/merged 2024-05-27T07:46:22.845 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d6f64b9c9965724fb80d0b70b07d7eb0163f836be68aaeba2347967da586f14e/merged 2024-05-27T07:46:22.846 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-05-27T07:46:22.846 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-27T07:46:22.846 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-27T07:46:22.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:22.847 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-05-27T07:46:22.936 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-27T07:46:22.936 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/qa/sv_0/a9b7f4ae-1094-46c7-b823-ec4601e37cd2 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-05-27T07:46:22.962 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:22 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:22.646+0000 7f4f1baae200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T07:46:22.967 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-05-27T07:46:23.033 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:46:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-05-27T07:46:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-05-27T07:46:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-05-27T07:46:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-05-27T07:46:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-05-27T07:46:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-05-27T07:46:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-05-27T07:46:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-05-27T07:46:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-05-27T07:46:23.073 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:46:23.074 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:46:23.075 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:46:23.075 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:46:23.075 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-05-27T07:46:23.075 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-05-27T07:46:23.075 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:46:23.075 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/9e1bcc8f262b04006b2646c82924c60582162fe1749f0a10dc84210cf998766f/merged 2024-05-27T07:46:23.075 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/da074e182c088100c0366956aa071553c14b7d365bcfe7ba452de0a09a3e6cdb/merged 2024-05-27T07:46:23.075 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-05-27T07:46:23.075 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:46:23.075 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/74183af9942d0e0b14483025c64637abb9123624a14b778b4cc4ad7a66c834d6/merged 2024-05-27T07:46:23.076 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c7f3c7639caab68be3492caf2029c5c0dd8c399e09fd109f58a8afd7d9d82fcf/merged 2024-05-27T07:46:23.076 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/dbd87aa827d3ec73230a009e0e92ea3edaea081f5ff915ae8e52bceb8ee4093b/merged 2024-05-27T07:46:23.076 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/965856f8e1e17ab9ed7fd5db92818a04751564d3a836793e9712ae733fbcd15b/merged 2024-05-27T07:46:23.076 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f94a5ad33b8640fbbaf471ef9e89a72c75e3fa026ba1cc36be0c985a34172a14/merged 2024-05-27T07:46:23.076 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/31385d1d7f286038f0ca0a5f821a9012ffd5aad96c380bb1eb781def5c2ec6d0/merged 2024-05-27T07:46:23.076 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1d6e9600c151645e62fd289e9f10091935bc56ae566ab5a42dcc8d3214014af1/merged 2024-05-27T07:46:23.076 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d6f64b9c9965724fb80d0b70b07d7eb0163f836be68aaeba2347967da586f14e/merged 2024-05-27T07:46:23.076 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-05-27T07:46:23.076 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-27T07:46:23.076 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-27T07:46:23.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:23.077 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-05-27T07:46:23.204 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:2024-05-27T07:46:23.205+0000 7f74d111f480 -1 init, newargv = 0x7f74b40047e0 newargc=17 2024-05-27T07:46:23.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[90454]: starting ceph client 2024-05-27T07:46:23.249 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:22.915+0000 7f8dd75ea200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:46:23.249 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:23.044+0000 7f8dd75ea200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:46:23.255 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[90454]: starting fuse 2024-05-27T07:46:23.313 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:22.889+0000 7f3757997200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:46:23.313 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:23 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:23.023+0000 7f3757997200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:46:23.381 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:23 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:23.250+0000 7f4f1baae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T07:46:23.712 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:23 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:23.382+0000 7f4f1baae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T07:46:23.812 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:23 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:23.501+0000 7f3757997200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:46:23.979 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:23.519+0000 7f8dd75ea200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:46:24.212 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:23 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:23.861+0000 7f4f1baae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T07:46:24.248 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:23.978+0000 7f8dd75ea200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:46:24.249 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:24.105+0000 7f8dd75ea200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:46:24.263 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-05-27T07:46:24.313 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:23 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:23.963+0000 7f3757997200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:46:24.313 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:24.090+0000 7f3757997200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:46:24.348 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-05-27T07:46:24.398 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-05-27T07:46:24.398 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-05-27T07:46:24.398 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:46:24.399 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-05-27T07:46:24.400 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-05-27T07:46:24.400 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-05-27T07:46:24.400 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-05-27T07:46:24.400 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-05-27T07:46:24.400 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-05-27T07:46:24.400 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-05-27T07:46:24.400 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-27T07:46:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-27T07:46:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-05-27T07:46:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-05-27T07:46:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-05-27T07:46:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-05-27T07:46:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/9e1bcc8f262b04006b2646c82924c60582162fe1749f0a10dc84210cf998766f/merged 2024-05-27T07:46:24.401 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/da074e182c088100c0366956aa071553c14b7d365bcfe7ba452de0a09a3e6cdb/merged 2024-05-27T07:46:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-05-27T07:46:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-05-27T07:46:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/74183af9942d0e0b14483025c64637abb9123624a14b778b4cc4ad7a66c834d6/merged 2024-05-27T07:46:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c7f3c7639caab68be3492caf2029c5c0dd8c399e09fd109f58a8afd7d9d82fcf/merged 2024-05-27T07:46:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/dbd87aa827d3ec73230a009e0e92ea3edaea081f5ff915ae8e52bceb8ee4093b/merged 2024-05-27T07:46:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/965856f8e1e17ab9ed7fd5db92818a04751564d3a836793e9712ae733fbcd15b/merged 2024-05-27T07:46:24.402 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f94a5ad33b8640fbbaf471ef9e89a72c75e3fa026ba1cc36be0c985a34172a14/merged 2024-05-27T07:46:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/31385d1d7f286038f0ca0a5f821a9012ffd5aad96c380bb1eb781def5c2ec6d0/merged 2024-05-27T07:46:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1d6e9600c151645e62fd289e9f10091935bc56ae566ab5a42dcc8d3214014af1/merged 2024-05-27T07:46:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d6f64b9c9965724fb80d0b70b07d7eb0163f836be68aaeba2347967da586f14e/merged 2024-05-27T07:46:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-05-27T07:46:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-27T07:46:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-05-27T07:46:24.403 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs 2024-05-27T07:46:24.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:24.405 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-05-27T07:46:24.448 INFO:teuthology.orchestra.run.smithi090.stdout:107 2024-05-27T07:46:24.449 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [107] 2024-05-27T07:46:24.449 DEBUG:teuthology.orchestra.run.smithi090:> sudo stdin-killer -- python3 -c ' 2024-05-27T07:46:24.449 DEBUG:teuthology.orchestra.run.smithi090:> import glob 2024-05-27T07:46:24.449 DEBUG:teuthology.orchestra.run.smithi090:> import re 2024-05-27T07:46:24.449 DEBUG:teuthology.orchestra.run.smithi090:> import os 2024-05-27T07:46:24.449 DEBUG:teuthology.orchestra.run.smithi090:> import subprocess 2024-05-27T07:46:24.450 DEBUG:teuthology.orchestra.run.smithi090:> 2024-05-27T07:46:24.450 DEBUG:teuthology.orchestra.run.smithi090:> def _find_admin_socket(client_name): 2024-05-27T07:46:24.450 DEBUG:teuthology.orchestra.run.smithi090:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-27T07:46:24.450 DEBUG:teuthology.orchestra.run.smithi090:> files = glob.glob(asok_path) 2024-05-27T07:46:24.450 DEBUG:teuthology.orchestra.run.smithi090:> mountpoint = "/tmp/cephfs" 2024-05-27T07:46:24.450 DEBUG:teuthology.orchestra.run.smithi090:> 2024-05-27T07:46:24.450 DEBUG:teuthology.orchestra.run.smithi090:> # Given a non-glob path, it better be there 2024-05-27T07:46:24.450 DEBUG:teuthology.orchestra.run.smithi090:> if "*" not in asok_path: 2024-05-27T07:46:24.450 DEBUG:teuthology.orchestra.run.smithi090:> assert(len(files) == 1) 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> return files[0] 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> for f in files: 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> contents = proc_f.read() 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> if mountpoint in contents: 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> return f 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> print(_find_admin_socket("client.0")) 2024-05-27T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi090:> ' 2024-05-27T07:46:24.644 INFO:teuthology.orchestra.run.smithi090.stdout:/var/run/ceph/ceph-client.0.90454.asok 2024-05-27T07:46:24.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T07:46:24.658 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:24 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:24.325+0000 7f4f1baae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T07:46:24.658 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:24 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:24.459+0000 7f4f1baae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T07:46:24.661 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.90454.asok 2024-05-27T07:46:24.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:24.662 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.90454.asok status 2024-05-27T07:46:24.723 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:24.415+0000 7f3757997200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:46:24.723 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:24.594+0000 7f3757997200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:46:24.740 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:24.427+0000 7f8dd75ea200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:46:24.741 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:24.608+0000 7f8dd75ea200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:46:24.741 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:24.740+0000 7f8dd75ea200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:46:24.955 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "metadata": { 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_sha1": "38f9d900c83c63799fbdbe61acc9a11b0d3554a6", 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev)", 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "entity_id": "0", 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": "smithi090", 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "mount_point": "/tmp/cephfs", 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "pid": "90454", 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "root": "/volumes/qa/sv_0/a9b7f4ae-1094-46c7-b823-ec4601e37cd2" 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_count": 0, 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_pinned_count": 0, 2024-05-27T07:46:24.956 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 15273, 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: "inst": { 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: "name": { 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "client", 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: "num": 15273 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 1107445678 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T07:46:24.957 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 1107445678 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "inst_str": "client.15273 192.168.0.1:0/1107445678", 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "addr_str": "192.168.0.1:0/1107445678", 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "inode_count": 5, 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_epoch": 29, 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch": 91, 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch_barrier": 0, 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "blocklisted": false, 2024-05-27T07:46:24.958 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs" 2024-05-27T07:46:24.959 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:46:24.962 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:24 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:24.804+0000 7f4f1baae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T07:46:24.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:24.976 DEBUG:teuthology.orchestra.run.smithi090:> stat --file-system '--printf=%T 2024-05-27T07:46:24.976 DEBUG:teuthology.orchestra.run.smithi090:> ' -- /tmp/cephfs 2024-05-27T07:46:25.027 INFO:teuthology.orchestra.run.smithi090.stdout:fuseblk 2024-05-27T07:46:25.027 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-05-27T07:46:25.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T07:46:25.028 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /tmp/cephfs 2024-05-27T07:46:25.062 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:24.723+0000 7f3757997200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:46:25.141 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T07:46:25.150 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:46:25.150 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T07:46:25.150 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-27T07:46:25.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.419+0000 7f290f200640 1 -- 172.21.15.90:0/517388076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2910104cc0 msgr2=0x7f29101050d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.419+0000 7f290f200640 1 --2- 172.21.15.90:0/517388076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2910104cc0 0x7f29101050d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009900 tx=0x7f28f802a400 comp rx=0 tx=0).stop 2024-05-27T07:46:25.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.419+0000 7f290f200640 1 -- 172.21.15.90:0/517388076 shutdown_connections 2024-05-27T07:46:25.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.419+0000 7f290f200640 1 --2- 172.21.15.90:0/517388076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29101099c0 0x7f291010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.419+0000 7f290f200640 1 --2- 172.21.15.90:0/517388076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2910104cc0 0x7f29101050d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.419+0000 7f290f200640 1 --2- 172.21.15.90:0/517388076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2910103a80 0x7f2910104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.419+0000 7f290f200640 1 -- 172.21.15.90:0/517388076 >> 172.21.15.90:0/517388076 conn(0x7f29100fec70 msgr2=0x7f2910101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:25.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.419+0000 7f290f200640 1 -- 172.21.15.90:0/517388076 shutdown_connections 2024-05-27T07:46:25.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.419+0000 7f290f200640 1 -- 172.21.15.90:0/517388076 wait complete. 2024-05-27T07:46:25.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.420+0000 7f290f200640 1 Processor -- start 2024-05-27T07:46:25.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.420+0000 7f290f200640 1 -- start start 2024-05-27T07:46:25.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f290f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2910103a80 0x7f291019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f290f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2910104cc0 0x7f291019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f290f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29101099c0 0x7f29101a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f290f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f291010f430 con 0x7f2910104cc0 2024-05-27T07:46:25.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f290f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f291010f2b0 con 0x7f2910103a80 2024-05-27T07:46:25.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f290f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f291010f5b0 con 0x7f29101099c0 2024-05-27T07:46:25.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f2907e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2910104cc0 0x7f291019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f2907e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2910104cc0 0x7f291019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46188/0 (socket says 172.21.15.90:46188) 2024-05-27T07:46:25.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f2907e00640 1 -- 172.21.15.90:0/2287930033 learned_addr learned my addr 172.21.15.90:0/2287930033 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:25.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f290de00640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2910103a80 0x7f291019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.421+0000 7f290e800640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29101099c0 0x7f29101a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:25.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.422+0000 7f2907e00640 1 -- 172.21.15.90:0/2287930033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29101099c0 msgr2=0x7f29101a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.422+0000 7f2907e00640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29101099c0 0x7f29101a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.422+0000 7f2907e00640 1 -- 172.21.15.90:0/2287930033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2910103a80 msgr2=0x7f291019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.422+0000 7f2907e00640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2910103a80 0x7f291019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.422+0000 7f2907e00640 1 -- 172.21.15.90:0/2287930033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29101a1aa0 con 0x7f2910104cc0 2024-05-27T07:46:25.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.422+0000 7f2907e00640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2910104cc0 0x7f291019a550 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009230 tx=0x7f28f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:25.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.422+0000 7f2906a00640 1 -- 172.21.15.90:0/2287930033 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f803f030 con 0x7f2910104cc0 2024-05-27T07:46:25.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.422+0000 7f290f200640 1 -- 172.21.15.90:0/2287930033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29101a1d30 con 0x7f2910104cc0 2024-05-27T07:46:25.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.422+0000 7f2906a00640 1 -- 172.21.15.90:0/2287930033 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28f8003c90 con 0x7f2910104cc0 2024-05-27T07:46:25.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.422+0000 7f2906a00640 1 -- 172.21.15.90:0/2287930033 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f8035070 con 0x7f2910104cc0 2024-05-27T07:46:25.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.423+0000 7f290f200640 1 -- 172.21.15.90:0/2287930033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29101a21d0 con 0x7f2910104cc0 2024-05-27T07:46:25.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.424+0000 7f290f200640 1 -- 172.21.15.90:0/2287930033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29101087d0 con 0x7f2910104cc0 2024-05-27T07:46:25.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.426+0000 7f2906a00640 1 -- 172.21.15.90:0/2287930033 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f28f8042030 con 0x7f2910104cc0 2024-05-27T07:46:25.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.428+0000 7f2906a00640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f28f40b2830 0x7f28f40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:25.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.428+0000 7f290de00640 1 -- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f28f40b2830 msgr2=0x7f28f40b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2854405112 2024-05-27T07:46:25.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.428+0000 7f2906a00640 1 -- 172.21.15.90:0/2287930033 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f28f8103280 con 0x7f2910104cc0 2024-05-27T07:46:25.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.428+0000 7f290de00640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f28f40b2830 0x7f28f40b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:46:25.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.431+0000 7f2906a00640 1 -- 172.21.15.90:0/2287930033 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28f80cabb0 con 0x7f2910104cc0 2024-05-27T07:46:25.462 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:25 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:25.003+0000 7f4f1baae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T07:46:25.462 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:25 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:25.142+0000 7f4f1baae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T07:46:25.498 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:25.146+0000 7f8dd75ea200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:46:25.498 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:25.271+0000 7f8dd75ea200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:46:25.563 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:25.132+0000 7f3757997200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:46:25.563 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:25.260+0000 7f3757997200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:46:25.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.629+0000 7f290de00640 1 -- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f28f40b2830 msgr2=0x7f28f40b4d40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2854405112 2024-05-27T07:46:25.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.629+0000 7f290de00640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f28f40b2830 0x7f28f40b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:46:25.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.687+0000 7f290f200640 1 -- 172.21.15.90:0/2287930033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f291002dc20 con 0x7f2910104cc0 2024-05-27T07:46:25.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.742+0000 7f2906a00640 1 -- 172.21.15.90:0/2287930033 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7f28f80ca4b0 con 0x7f2910104cc0 2024-05-27T07:46:25.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 -- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f28f40b2830 msgr2=0x7f28f40b4d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:46:25.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f28f40b2830 0x7f28f40b4d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 -- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2910104cc0 msgr2=0x7f291019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:25.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2910104cc0 0x7f291019a550 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009230 tx=0x7f28f80042f0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 -- 172.21.15.90:0/2287930033 shutdown_connections 2024-05-27T07:46:25.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f28f40b2830 0x7f28f40b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29101099c0 0x7f29101a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2910104cc0 0x7f291019a550 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 --2- 172.21.15.90:0/2287930033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2910103a80 0x7f291019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:25.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 -- 172.21.15.90:0/2287930033 >> 172.21.15.90:0/2287930033 conn(0x7f29100fec70 msgr2=0x7f29101059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:25.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 -- 172.21.15.90:0/2287930033 shutdown_connections 2024-05-27T07:46:25.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:25.746+0000 7f290f200640 1 -- 172.21.15.90:0/2287930033 wait complete. 2024-05-27T07:46:25.752 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:25 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:25.566+0000 7f4f1baae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T07:46:25.752 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:25 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:25.697+0000 7f4f1baae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T07:46:25.770 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-27T07:46:25.930 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:25 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2287930033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T07:46:26.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.018+0000 7f6c73200640 1 -- 172.21.15.90:0/3778192268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c74103a70 msgr2=0x7f6c74104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:26.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.018+0000 7f6c73200640 1 --2- 172.21.15.90:0/3778192268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c74103a70 0x7f6c74104770 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009900 tx=0x7f6c6402a410 comp rx=0 tx=0).stop 2024-05-27T07:46:26.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.018+0000 7f6c73200640 1 -- 172.21.15.90:0/3778192268 shutdown_connections 2024-05-27T07:46:26.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.018+0000 7f6c73200640 1 --2- 172.21.15.90:0/3778192268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c741099b0 0x7f6c7410bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:26.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.019+0000 7f6c73200640 1 --2- 172.21.15.90:0/3778192268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c74104cb0 0x7f6c741050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:26.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.019+0000 7f6c73200640 1 --2- 172.21.15.90:0/3778192268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c74103a70 0x7f6c74104770 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:26.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.019+0000 7f6c73200640 1 -- 172.21.15.90:0/3778192268 >> 172.21.15.90:0/3778192268 conn(0x7f6c740fec60 msgr2=0x7f6c74101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:26.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.019+0000 7f6c73200640 1 -- 172.21.15.90:0/3778192268 shutdown_connections 2024-05-27T07:46:26.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.019+0000 7f6c73200640 1 -- 172.21.15.90:0/3778192268 wait complete. 2024-05-27T07:46:26.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.020+0000 7f6c73200640 1 Processor -- start 2024-05-27T07:46:26.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.020+0000 7f6c73200640 1 -- start start 2024-05-27T07:46:26.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c73200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c74103a70 0x7f6c7419a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:26.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c73200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c74104cb0 0x7f6c7419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:26.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c73200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c741099b0 0x7f6c741a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:26.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c73200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410f470 con 0x7f6c74104cb0 2024-05-27T07:46:26.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c73200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410f2f0 con 0x7f6c74103a70 2024-05-27T07:46:26.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c73200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410f5f0 con 0x7f6c741099b0 2024-05-27T07:46:26.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c71e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c74103a70 0x7f6c7419a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:26.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c71400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c74104cb0 0x7f6c7419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:26.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c71e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c74103a70 0x7f6c7419a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52438/0 (socket says 172.21.15.90:52438) 2024-05-27T07:46:26.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c71e00640 1 -- 172.21.15.90:0/3672427595 learned_addr learned my addr 172.21.15.90:0/3672427595 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:26.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.021+0000 7f6c72800640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c741099b0 0x7f6c741a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:26.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.022+0000 7f6c71400640 1 -- 172.21.15.90:0/3672427595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c741099b0 msgr2=0x7f6c741a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:26.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.022+0000 7f6c71400640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c741099b0 0x7f6c741a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:26.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.022+0000 7f6c71400640 1 -- 172.21.15.90:0/3672427595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c74103a70 msgr2=0x7f6c7419a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:26.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.022+0000 7f6c71400640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c74103a70 0x7f6c7419a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:26.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.022+0000 7f6c71400640 1 -- 172.21.15.90:0/3672427595 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c741a1c20 con 0x7f6c74104cb0 2024-05-27T07:46:26.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.022+0000 7f6c71400640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c74104cb0 0x7f6c7419a620 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009b50 tx=0x7f6c6000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:26.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.023+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60009e50 con 0x7f6c74104cb0 2024-05-27T07:46:26.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.023+0000 7f6c73200640 1 -- 172.21.15.90:0/3672427595 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c74076060 con 0x7f6c74104cb0 2024-05-27T07:46:26.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.023+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c600054b0 con 0x7f6c74104cb0 2024-05-27T07:46:26.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.023+0000 7f6c73200640 1 -- 172.21.15.90:0/3672427595 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c740765b0 con 0x7f6c74104cb0 2024-05-27T07:46:26.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.023+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60019410 con 0x7f6c74104cb0 2024-05-27T07:46:26.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.025+0000 7f6c73200640 1 -- 172.21.15.90:0/3672427595 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c540052c0 con 0x7f6c74104cb0 2024-05-27T07:46:26.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.029+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f6c60020020 con 0x7f6c74104cb0 2024-05-27T07:46:26.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.031+0000 7f6c6aa00640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6c580b2a50 0x7f6c580b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:26.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.031+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f6c600e64e0 con 0x7f6c74104cb0 2024-05-27T07:46:26.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.031+0000 7f6c71e00640 1 -- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6c580b2a50 msgr2=0x7f6c580b4f60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2854405112 2024-05-27T07:46:26.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.031+0000 7f6c71e00640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6c580b2a50 0x7f6c580b4f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T07:46:26.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.033+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c60027070 con 0x7f6c74104cb0 2024-05-27T07:46:26.042 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:25 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2287930033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T07:46:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:25 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2287930033' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-27T07:46:26.229 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:26.053+0000 7f3757997200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:46:26.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.232+0000 7f6c71e00640 1 -- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6c580b2a50 msgr2=0x7f6c580b4f60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2854405112 2024-05-27T07:46:26.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.232+0000 7f6c71e00640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6c580b2a50 0x7f6c580b4f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T07:46:26.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.263+0000 7f6c73200640 1 -- 172.21.15.90:0/3672427595 --> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f6c54000c90 con 0x7f6c580b2a50 2024-05-27T07:46:26.330 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:26.042+0000 7f8dd75ea200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:46:26.563 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 07:46:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y[36630]: 2024-05-27T07:46:26.355+0000 7f3757997200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:46:26.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.633+0000 7f6c71e00640 1 -- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6c580b2a50 msgr2=0x7f6c580b4f60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2854405112 2024-05-27T07:46:26.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.633+0000 7f6c71e00640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6c580b2a50 0x7f6c580b4f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T07:46:26.712 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:26 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:26.526+0000 7f4f1baae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T07:46:26.751 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 07:46:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z[37049]: 2024-05-27T07:46:26.329+0000 7f8dd75ea200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:46:26.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.787+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f6c600a99a0 con 0x7f6c74104cb0 2024-05-27T07:46:26.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.866+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7f6c600df530 con 0x7f6c74104cb0 2024-05-27T07:46:26.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.867+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6c580b2a50 msgr2=0x7f6c580b4f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:46:26.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:26.867+0000 7f6c6aa00640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/2854405112,v1:172.21.15.90:6801/2854405112] conn(0x7f6c580b2a50 0x7f6c580b4f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:27.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2287930033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T07:46:27.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: Standby manager daemon z restarted 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: Standby manager daemon z started 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: Standby manager daemon y restarted 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: Standby manager daemon y started 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:46:27.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:26 smithi129 ceph-mon[35254]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2287930033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: Standby manager daemon z restarted 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: Standby manager daemon z started 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: Standby manager daemon y restarted 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: Standby manager daemon y started 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:46:27.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:46:27.074 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:46:27.074 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:26 smithi090 ceph-mon[31429]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:46:27.074 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:26 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:26.833+0000 7f4f1baae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T07:46:27.074 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2287930033' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-27T07:46:27.074 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:46:27.074 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: Standby manager daemon z restarted 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: Standby manager daemon z started 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.136:0/2377524310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: Standby manager daemon y restarted 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: Standby manager daemon y started 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T07:46:27.075 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:26 smithi136 ceph-mon[35324]: from='mgr.? 172.21.15.129:0/2619216308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T07:46:27.712 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.327+0000 7f4e64200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:27.713 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.327+0000 7f4e64200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:27.713 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.327+0000 7f4e64200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:27.713 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.327+0000 7f4e64200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:27.713 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.327+0000 7f4e64200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:27.713 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.327+0000 7f4e69200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:27.713 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.327+0000 7f4e69200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:27.713 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.327+0000 7f4e69200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:27.713 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.327+0000 7f4e69200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:27.713 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.327+0000 7f4e69200700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:27.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.886+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 <== mon.0 v2:172.21.15.90:3300/0 9 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f6c60089cc0 con 0x7f6c74104cb0 2024-05-27T07:46:27.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.887+0000 7f6c6aa00640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c58039de0 0x7f6c5803a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:27.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.887+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 --> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f6c600d0060 con 0x7f6c58039de0 2024-05-27T07:46:27.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.895+0000 7f6c71e00640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c58039de0 0x7f6c5803a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:27.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.905+0000 7f6c71e00640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c58039de0 0x7f6c5803a1f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c64006010 tx=0x7f6c64031400 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:27.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.957+0000 7f6c6aa00640 1 -- 172.21.15.90:0/3672427595 <== mgr.34967 v2:172.21.15.90:6800/4256436582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f6c600d0060 con 0x7f6c58039de0 2024-05-27T07:46:27.956 INFO:teuthology.orchestra.run.smithi090.stdout:{} 2024-05-27T07:46:27.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.959+0000 7f6c73200640 1 -- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c58039de0 msgr2=0x7f6c5803a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.959+0000 7f6c73200640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c58039de0 0x7f6c5803a1f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6c64006010 tx=0x7f6c64031400 comp rx=0 tx=0).stop 2024-05-27T07:46:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.959+0000 7f6c73200640 1 -- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c74104cb0 msgr2=0x7f6c7419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.959+0000 7f6c73200640 1 --2- 172.21.15.90:0/3672427595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c74104cb0 0x7f6c7419a620 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009b50 tx=0x7f6c6000b690 comp rx=0 tx=0).stop 2024-05-27T07:46:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.959+0000 7f6c72800640 1 -- 172.21.15.90:0/3672427595 reap_dead start 2024-05-27T07:46:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.959+0000 7f6c73200640 1 -- 172.21.15.90:0/3672427595 shutdown_connections 2024-05-27T07:46:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.959+0000 7f6c73200640 1 -- 172.21.15.90:0/3672427595 >> 172.21.15.90:0/3672427595 conn(0x7f6c740fec60 msgr2=0x7f6c74105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.960+0000 7f6c73200640 1 -- 172.21.15.90:0/3672427595 shutdown_connections 2024-05-27T07:46:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:27.960+0000 7f6c73200640 1 -- 172.21.15.90:0/3672427595 wait complete. 2024-05-27T07:46:27.984 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-27T07:46:27.997 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-27T07:46:28.045 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T07:46:28.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-27T07:46:28.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: Active manager daemon x restarted 2024-05-27T07:46:28.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: Activating manager daemon x 2024-05-27T07:46:28.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T07:46:28.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: mgrmap e21: x(active, starting, since 0.0204822s), standbys: z, y 2024-05-27T07:46:28.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:46:28.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:46:28.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:46:28.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:46:28.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:46:28.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:46:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:46:28.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:46:28.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:46:28.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:46:28.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: Manager daemon x is now available 2024-05-27T07:46:28.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:28.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:46:28.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:46:28.097 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.916+0000 7f4e62400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:28.097 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.916+0000 7f4e62400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:28.097 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.916+0000 7f4e62400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:28.098 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.916+0000 7f4e62400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:28.098 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:46:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:46:27.916+0000 7f4e62400700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:46:28.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-27T07:46:28.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: Active manager daemon x restarted 2024-05-27T07:46:28.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: Activating manager daemon x 2024-05-27T07:46:28.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T07:46:28.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: mgrmap e21: x(active, starting, since 0.0204822s), standbys: z, y 2024-05-27T07:46:28.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:46:28.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:46:28.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:46:28.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:46:28.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:46:28.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:46:28.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:46:28.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:46:28.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: Manager daemon x is now available 2024-05-27T07:46:28.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:28.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:46:28.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:46:28.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: mgrmap e20: x(active, since 15m), standbys: z, y 2024-05-27T07:46:28.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: Active manager daemon x restarted 2024-05-27T07:46:28.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: Activating manager daemon x 2024-05-27T07:46:28.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T07:46:28.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: mgrmap e21: x(active, starting, since 0.0204822s), standbys: z, y 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T07:46:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T07:46:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T07:46:28.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T07:46:28.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T07:46:28.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T07:46:28.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T07:46:28.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T07:46:28.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T07:46:28.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T07:46:28.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T07:46:28.253 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: Manager daemon x is now available 2024-05-27T07:46:28.253 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:28.253 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T07:46:28.253 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T07:46:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.306+0000 7fd199600640 1 -- 172.21.15.90:0/3548789958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19410b470 msgr2=0x7fd19410b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.306+0000 7fd199600640 1 --2- 172.21.15.90:0/3548789958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19410b470 0x7fd19410b880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd180009900 tx=0x7fd18002a3f0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.306+0000 7fd199600640 1 -- 172.21.15.90:0/3548789958 shutdown_connections 2024-05-27T07:46:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.306+0000 7fd199600640 1 --2- 172.21.15.90:0/3548789958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd194110170 0x7fd194112590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.306+0000 7fd199600640 1 --2- 172.21.15.90:0/3548789958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19410b470 0x7fd19410b880 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.306+0000 7fd199600640 1 --2- 172.21.15.90:0/3548789958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd19410a230 0x7fd19410af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.306+0000 7fd199600640 1 -- 172.21.15.90:0/3548789958 >> 172.21.15.90:0/3548789958 conn(0x7fd194078f00 msgr2=0x7fd194107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:28.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.306+0000 7fd199600640 1 -- 172.21.15.90:0/3548789958 shutdown_connections 2024-05-27T07:46:28.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.306+0000 7fd199600640 1 -- 172.21.15.90:0/3548789958 wait complete. 2024-05-27T07:46:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.307+0000 7fd199600640 1 Processor -- start 2024-05-27T07:46:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.307+0000 7fd199600640 1 -- start start 2024-05-27T07:46:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.307+0000 7fd199600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19410a230 0x7fd1941a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.307+0000 7fd199600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd19410b470 0x7fd1941a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd199600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd194110170 0x7fd1941a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd199600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd194115c50 con 0x7fd19410b470 2024-05-27T07:46:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd199600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd194115ad0 con 0x7fd19410a230 2024-05-27T07:46:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd199600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd194115dd0 con 0x7fd194110170 2024-05-27T07:46:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd192000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd19410b470 0x7fd1941a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd192000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd19410b470 0x7fd1941a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46338/0 (socket says 172.21.15.90:46338) 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd192000640 1 -- 172.21.15.90:0/2618359012 learned_addr learned my addr 172.21.15.90:0/2618359012 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd192a00640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19410a230 0x7fd1941a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd193400640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd194110170 0x7fd1941a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd192000640 1 -- 172.21.15.90:0/2618359012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd194110170 msgr2=0x7fd1941a7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd192000640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd194110170 0x7fd1941a7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd192000640 1 -- 172.21.15.90:0/2618359012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19410a230 msgr2=0x7fd1941a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd192000640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19410a230 0x7fd1941a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd192000640 1 -- 172.21.15.90:0/2618359012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1941a8370 con 0x7fd19410b470 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.308+0000 7fd192a00640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19410a230 0x7fd1941a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.309+0000 7fd192000640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd19410b470 0x7fd1941a0d70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd180005760 tx=0x7fd180003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.309+0000 7fd18b400640 1 -- 172.21.15.90:0/2618359012 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18003f070 con 0x7fd19410b470 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.309+0000 7fd18b400640 1 -- 172.21.15.90:0/2618359012 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd180031ba0 con 0x7fd19410b470 2024-05-27T07:46:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.309+0000 7fd199600640 1 -- 172.21.15.90:0/2618359012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1941a8600 con 0x7fd19410b470 2024-05-27T07:46:28.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.309+0000 7fd18b400640 1 -- 172.21.15.90:0/2618359012 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180035030 con 0x7fd19410b470 2024-05-27T07:46:28.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.309+0000 7fd199600640 1 -- 172.21.15.90:0/2618359012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1941a8af0 con 0x7fd19410b470 2024-05-27T07:46:28.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.310+0000 7fd199600640 1 -- 172.21.15.90:0/2618359012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19410ef80 con 0x7fd19410b470 2024-05-27T07:46:28.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.313+0000 7fd18b400640 1 -- 172.21.15.90:0/2618359012 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fd180042070 con 0x7fd19410b470 2024-05-27T07:46:28.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.316+0000 7fd18b400640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1780b25e0 0x7fd1780b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:28.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.316+0000 7fd18b400640 1 -- 172.21.15.90:0/2618359012 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fd180103030 con 0x7fd19410b470 2024-05-27T07:46:28.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.316+0000 7fd192a00640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1780b25e0 0x7fd1780b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:28.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.317+0000 7fd192a00640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1780b25e0 0x7fd1780b4af0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd19410ae30 tx=0x7fd184005f90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:28.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.317+0000 7fd18b400640 1 -- 172.21.15.90:0/2618359012 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1800cab90 con 0x7fd19410b470 2024-05-27T07:46:28.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.523+0000 7fd199600640 1 -- 172.21.15.90:0/2618359012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1940638c0 con 0x7fd19410b470 2024-05-27T07:46:28.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.528+0000 7fd18b400640 1 -- 172.21.15.90:0/2618359012 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+25102 (secure 0 0 0) 0x7fd1800ca490 con 0x7fd19410b470 2024-05-27T07:46:28.528 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:46:28.528 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":92,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:46:26.839777+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6801/2854405112":"2024-05-28T07:46:26.839737+0000","172.21.15.90:6800/2854405112":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/3526000120":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/836242224":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/2932841075":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2692415447":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:46:28.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.530+0000 7fd199600640 1 -- 172.21.15.90:0/2618359012 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1780b25e0 msgr2=0x7fd1780b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:28.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1780b25e0 0x7fd1780b4af0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd19410ae30 tx=0x7fd184005f90 comp rx=0 tx=0).stop 2024-05-27T07:46:28.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 -- 172.21.15.90:0/2618359012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd19410b470 msgr2=0x7fd1941a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:28.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd19410b470 0x7fd1941a0d70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd180005760 tx=0x7fd180003c10 comp rx=0 tx=0).stop 2024-05-27T07:46:28.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 -- 172.21.15.90:0/2618359012 shutdown_connections 2024-05-27T07:46:28.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1780b25e0 0x7fd1780b4af0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd194110170 0x7fd1941a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd19410b470 0x7fd1941a0d70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 --2- 172.21.15.90:0/2618359012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19410a230 0x7fd1941a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 -- 172.21.15.90:0/2618359012 >> 172.21.15.90:0/2618359012 conn(0x7fd194078f00 msgr2=0x7fd19410c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:28.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 -- 172.21.15.90:0/2618359012 shutdown_connections 2024-05-27T07:46:28.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.531+0000 7fd199600640 1 -- 172.21.15.90:0/2618359012 wait complete. 2024-05-27T07:46:28.555 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T07:34:18.340712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T07:41:03.393870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T07:41:07.610662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T07:41:19.963468+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T07:46:28.555 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T07:46:28.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.824+0000 7fec0ac00640 1 -- 172.21.15.90:0/161706263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec04107060 msgr2=0x7fec04107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:28.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.824+0000 7fec0ac00640 1 --2- 172.21.15.90:0/161706263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec04107060 0x7fec04107470 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7febf0009900 tx=0x7febf002a3f0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.824+0000 7fec0ac00640 1 -- 172.21.15.90:0/161706263 shutdown_connections 2024-05-27T07:46:28.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.824+0000 7fec0ac00640 1 --2- 172.21.15.90:0/161706263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec0410bd60 0x7fec0410e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.824+0000 7fec0ac00640 1 --2- 172.21.15.90:0/161706263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec04107060 0x7fec04107470 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.824+0000 7fec0ac00640 1 --2- 172.21.15.90:0/161706263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec04105e20 0x7fec04106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.824+0000 7fec0ac00640 1 -- 172.21.15.90:0/161706263 >> 172.21.15.90:0/161706263 conn(0x7fec04101010 msgr2=0x7fec041034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:28.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.824+0000 7fec0ac00640 1 -- 172.21.15.90:0/161706263 shutdown_connections 2024-05-27T07:46:28.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.824+0000 7fec0ac00640 1 -- 172.21.15.90:0/161706263 wait complete. 2024-05-27T07:46:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.825+0000 7fec0ac00640 1 Processor -- start 2024-05-27T07:46:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec0ac00640 1 -- start start 2024-05-27T07:46:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec0ac00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec04105e20 0x7fec0419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec0ac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec04107060 0x7fec0419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec0ac00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec0410bd60 0x7fec041a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec0ac00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fec041117f0 con 0x7fec04105e20 2024-05-27T07:46:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec0ac00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec04111670 con 0x7fec0410bd60 2024-05-27T07:46:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec0ac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fec04111970 con 0x7fec04107060 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec03e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec04105e20 0x7fec0419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec03e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec04105e20 0x7fec0419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46360/0 (socket says 172.21.15.90:46360) 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec03e00640 1 -- 172.21.15.90:0/1748378647 learned_addr learned my addr 172.21.15.90:0/1748378647 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec08c00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec0410bd60 0x7fec041a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.826+0000 7fec03400640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec04107060 0x7fec0419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.827+0000 7fec03e00640 1 -- 172.21.15.90:0/1748378647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec04107060 msgr2=0x7fec0419c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.827+0000 7fec03e00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec04107060 0x7fec0419c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.827+0000 7fec03e00640 1 -- 172.21.15.90:0/1748378647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec0410bd60 msgr2=0x7fec041a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.827+0000 7fec03e00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec0410bd60 0x7fec041a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.827+0000 7fec03e00640 1 -- 172.21.15.90:0/1748378647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec041a3f20 con 0x7fec04105e20 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.827+0000 7fec03e00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec04105e20 0x7fec0419c490 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7febf4005d00 tx=0x7febf400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.827+0000 7fec00c00640 1 -- 172.21.15.90:0/1748378647 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf4017070 con 0x7fec04105e20 2024-05-27T07:46:28.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.827+0000 7fec00c00640 1 -- 172.21.15.90:0/1748378647 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7febf4009c30 con 0x7fec04105e20 2024-05-27T07:46:28.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.827+0000 7fec0ac00640 1 -- 172.21.15.90:0/1748378647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec041a4210 con 0x7fec04105e20 2024-05-27T07:46:28.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.828+0000 7fec00c00640 1 -- 172.21.15.90:0/1748378647 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf400f620 con 0x7fec04105e20 2024-05-27T07:46:28.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.828+0000 7fec0ac00640 1 -- 172.21.15.90:0/1748378647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041a4710 con 0x7fec04105e20 2024-05-27T07:46:28.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.829+0000 7fec0ac00640 1 -- 172.21.15.90:0/1748378647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0410ab70 con 0x7fec04105e20 2024-05-27T07:46:28.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.833+0000 7fec00c00640 1 -- 172.21.15.90:0/1748378647 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7febf401a070 con 0x7fec04105e20 2024-05-27T07:46:28.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.835+0000 7fec00c00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7febe80b25e0 0x7febe80b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:28.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.836+0000 7fec00c00640 1 -- 172.21.15.90:0/1748378647 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7febf40e1d90 con 0x7fec04105e20 2024-05-27T07:46:28.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.836+0000 7fec03400640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7febe80b25e0 0x7febe80b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:28.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.836+0000 7fec00c00640 1 -- 172.21.15.90:0/1748378647 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7febf4009de0 con 0x7fec04105e20 2024-05-27T07:46:28.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.837+0000 7fec03400640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7febe80b25e0 0x7febe80b4af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7febf0006010 tx=0x7febf0000bc0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:28.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:28.889+0000 7fec00c00640 1 -- 172.21.15.90:0/1748378647 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7febf40a54a0 con 0x7fec04105e20 2024-05-27T07:46:29.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:28 smithi129 ceph-mon[35254]: [27/May/2024:07:46:27] ENGINE Bus STARTING 2024-05-27T07:46:29.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:28 smithi129 ceph-mon[35254]: [27/May/2024:07:46:27] ENGINE Serving on https://172.21.15.90:7150 2024-05-27T07:46:29.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:28 smithi129 ceph-mon[35254]: [27/May/2024:07:46:27] ENGINE Serving on http://172.21.15.90:8765 2024-05-27T07:46:29.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:28 smithi129 ceph-mon[35254]: [27/May/2024:07:46:27] ENGINE Bus STARTED 2024-05-27T07:46:29.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:28 smithi129 ceph-mon[35254]: mgrmap e22: x(active, since 1.05284s), standbys: z, y 2024-05-27T07:46:29.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:28 smithi129 ceph-mon[35254]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:29.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:28 smithi129 ceph-mon[35254]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:29.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:28 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2618359012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:46:29.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.075+0000 7fec0ac00640 1 -- 172.21.15.90:0/1748378647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fec040638c0 con 0x7fec04105e20 2024-05-27T07:46:29.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.076+0000 7fec00c00640 1 -- 172.21.15.90:0/1748378647 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7febf40a9910 con 0x7fec04105e20 2024-05-27T07:46:29.076 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-05-27T07:46:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.078+0000 7fec0ac00640 1 -- 172.21.15.90:0/1748378647 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7febe80b25e0 msgr2=0x7febe80b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.078+0000 7fec0ac00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7febe80b25e0 0x7febe80b4af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7febf0006010 tx=0x7febf0000bc0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.078+0000 7fec0ac00640 1 -- 172.21.15.90:0/1748378647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec04105e20 msgr2=0x7fec0419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.078+0000 7fec0ac00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec04105e20 0x7fec0419c490 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7febf4005d00 tx=0x7febf400dde0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.078+0000 7fec0ac00640 1 -- 172.21.15.90:0/1748378647 shutdown_connections 2024-05-27T07:46:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.078+0000 7fec0ac00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7febe80b25e0 0x7febe80b4af0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.078+0000 7fec0ac00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec0410bd60 0x7fec041a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.079+0000 7fec0ac00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec04107060 0x7fec0419c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.079+0000 7fec0ac00640 1 --2- 172.21.15.90:0/1748378647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec04105e20 0x7fec0419c490 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.079+0000 7fec0ac00640 1 -- 172.21.15.90:0/1748378647 >> 172.21.15.90:0/1748378647 conn(0x7fec04101010 msgr2=0x7fec04107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:29.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.079+0000 7fec0ac00640 1 -- 172.21.15.90:0/1748378647 shutdown_connections 2024-05-27T07:46:29.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.079+0000 7fec0ac00640 1 -- 172.21.15.90:0/1748378647 wait complete. 2024-05-27T07:46:29.105 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T07:46:29.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:28 smithi090 ceph-mon[31429]: [27/May/2024:07:46:27] ENGINE Bus STARTING 2024-05-27T07:46:29.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:28 smithi090 ceph-mon[31429]: [27/May/2024:07:46:27] ENGINE Serving on https://172.21.15.90:7150 2024-05-27T07:46:29.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:28 smithi090 ceph-mon[31429]: [27/May/2024:07:46:27] ENGINE Serving on http://172.21.15.90:8765 2024-05-27T07:46:29.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:28 smithi090 ceph-mon[31429]: [27/May/2024:07:46:27] ENGINE Bus STARTED 2024-05-27T07:46:29.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:28 smithi090 ceph-mon[31429]: mgrmap e22: x(active, since 1.05284s), standbys: z, y 2024-05-27T07:46:29.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:28 smithi090 ceph-mon[31429]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:29.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:28 smithi090 ceph-mon[31429]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:29.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:28 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2618359012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:46:29.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:28 smithi136 ceph-mon[35324]: [27/May/2024:07:46:27] ENGINE Bus STARTING 2024-05-27T07:46:29.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:28 smithi136 ceph-mon[35324]: [27/May/2024:07:46:27] ENGINE Serving on https://172.21.15.90:7150 2024-05-27T07:46:29.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:28 smithi136 ceph-mon[35324]: [27/May/2024:07:46:27] ENGINE Serving on http://172.21.15.90:8765 2024-05-27T07:46:29.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:28 smithi136 ceph-mon[35324]: [27/May/2024:07:46:27] ENGINE Bus STARTED 2024-05-27T07:46:29.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:28 smithi136 ceph-mon[35324]: mgrmap e22: x(active, since 1.05284s), standbys: z, y 2024-05-27T07:46:29.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:28 smithi136 ceph-mon[35324]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:29.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:28 smithi136 ceph-mon[35324]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:29.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:28 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2618359012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:46:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.385+0000 7fb2dd200640 1 -- 172.21.15.90:0/1611045251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d8104e50 msgr2=0x7fb2d8105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.385+0000 7fb2dd200640 1 --2- 172.21.15.90:0/1611045251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d8104e50 0x7fb2d8105260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009860 tx=0x7fb2c002a340 comp rx=0 tx=0).stop 2024-05-27T07:46:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.386+0000 7fb2dd200640 1 -- 172.21.15.90:0/1611045251 shutdown_connections 2024-05-27T07:46:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.386+0000 7fb2dd200640 1 --2- 172.21.15.90:0/1611045251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d8109b50 0x7fb2d810bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.386+0000 7fb2dd200640 1 --2- 172.21.15.90:0/1611045251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d8104e50 0x7fb2d8105260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.386+0000 7fb2dd200640 1 --2- 172.21.15.90:0/1611045251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2d8103c10 0x7fb2d8104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.386+0000 7fb2dd200640 1 -- 172.21.15.90:0/1611045251 >> 172.21.15.90:0/1611045251 conn(0x7fb2d80fee00 msgr2=0x7fb2d81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:29.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.386+0000 7fb2dd200640 1 -- 172.21.15.90:0/1611045251 shutdown_connections 2024-05-27T07:46:29.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.386+0000 7fb2dd200640 1 -- 172.21.15.90:0/1611045251 wait complete. 2024-05-27T07:46:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.387+0000 7fb2dd200640 1 Processor -- start 2024-05-27T07:46:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.387+0000 7fb2dd200640 1 -- start start 2024-05-27T07:46:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2dd200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d8103c10 0x7fb2d819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2dd200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2d8104e50 0x7fb2d819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2dd200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d8109b50 0x7fb2d81a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2dd200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d810f5e0 con 0x7fb2d8109b50 2024-05-27T07:46:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2dd200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d810f460 con 0x7fb2d8104e50 2024-05-27T07:46:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2dd200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d810f760 con 0x7fb2d8103c10 2024-05-27T07:46:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2d6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d8109b50 0x7fb2d81a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:29.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2d6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d8109b50 0x7fb2d81a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49652/0 (socket says 172.21.15.90:49652) 2024-05-27T07:46:29.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2d6800640 1 -- 172.21.15.90:0/3475816847 learned_addr learned my addr 172.21.15.90:0/3475816847 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:29.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2d5e00640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d8103c10 0x7fb2d819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:29.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2cfe00640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2d8104e50 0x7fb2d819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:29.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2d6800640 1 -- 172.21.15.90:0/3475816847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d8103c10 msgr2=0x7fb2d819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:29.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2d6800640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d8103c10 0x7fb2d819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2d6800640 1 -- 172.21.15.90:0/3475816847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2d8104e50 msgr2=0x7fb2d819c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:29.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2d6800640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2d8104e50 0x7fb2d819c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.388+0000 7fb2d6800640 1 -- 172.21.15.90:0/3475816847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d81a3fd0 con 0x7fb2d8109b50 2024-05-27T07:46:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.389+0000 7fb2d6800640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d8109b50 0x7fb2d81a3960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009fd0 tx=0x7fb2d000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.389+0000 7fb2cea00640 1 -- 172.21.15.90:0/3475816847 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0004b20 con 0x7fb2d8109b50 2024-05-27T07:46:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.389+0000 7fb2cea00640 1 -- 172.21.15.90:0/3475816847 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb2d0003e00 con 0x7fb2d8109b50 2024-05-27T07:46:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.389+0000 7fb2dd200640 1 -- 172.21.15.90:0/3475816847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d81a42c0 con 0x7fb2d8109b50 2024-05-27T07:46:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.389+0000 7fb2cea00640 1 -- 172.21.15.90:0/3475816847 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00047b0 con 0x7fb2d8109b50 2024-05-27T07:46:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.389+0000 7fb2dd200640 1 -- 172.21.15.90:0/3475816847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d81a4790 con 0x7fb2d8109b50 2024-05-27T07:46:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.390+0000 7fb2dd200640 1 -- 172.21.15.90:0/3475816847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d81a49f0 con 0x7fb2d8109b50 2024-05-27T07:46:29.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.394+0000 7fb2cea00640 1 -- 172.21.15.90:0/3475816847 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7fb2d001d020 con 0x7fb2d8109b50 2024-05-27T07:46:29.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.396+0000 7fb2cea00640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb2bc0b29f0 0x7fb2bc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:29.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.396+0000 7fb2cea00640 1 -- 172.21.15.90:0/3475816847 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fb2d00e3500 con 0x7fb2d8109b50 2024-05-27T07:46:29.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.396+0000 7fb2d5e00640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb2bc0b29f0 0x7fb2bc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:29.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.397+0000 7fb2d5e00640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb2bc0b29f0 0x7fb2bc0b4f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8005fd0 tx=0x7fb2c800b390 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:29.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.398+0000 7fb2cea00640 1 -- 172.21.15.90:0/3475816847 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2d0024030 con 0x7fb2d8109b50 2024-05-27T07:46:29.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.610+0000 7fb2dd200640 1 -- 172.21.15.90:0/3475816847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb2d81a4bc0 con 0x7fb2d8109b50 2024-05-27T07:46:29.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.611+0000 7fb2cea00640 1 -- 172.21.15.90:0/3475816847 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fb2d0004de0 con 0x7fb2d8109b50 2024-05-27T07:46:29.611 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-05-27T07:46:29.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.613+0000 7fb2dd200640 1 -- 172.21.15.90:0/3475816847 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb2bc0b29f0 msgr2=0x7fb2bc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:29.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.613+0000 7fb2dd200640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb2bc0b29f0 0x7fb2bc0b4f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8005fd0 tx=0x7fb2c800b390 comp rx=0 tx=0).stop 2024-05-27T07:46:29.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.613+0000 7fb2dd200640 1 -- 172.21.15.90:0/3475816847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d8109b50 msgr2=0x7fb2d81a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:29.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.613+0000 7fb2dd200640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d8109b50 0x7fb2d81a3960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009fd0 tx=0x7fb2d000c800 comp rx=0 tx=0).stop 2024-05-27T07:46:29.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.614+0000 7fb2dd200640 1 -- 172.21.15.90:0/3475816847 shutdown_connections 2024-05-27T07:46:29.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.614+0000 7fb2dd200640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb2bc0b29f0 0x7fb2bc0b4f00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.614+0000 7fb2dd200640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2d8104e50 0x7fb2d819c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.614+0000 7fb2dd200640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2d8109b50 0x7fb2d81a3960 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.614+0000 7fb2dd200640 1 --2- 172.21.15.90:0/3475816847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d8103c10 0x7fb2d819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.614+0000 7fb2dd200640 1 -- 172.21.15.90:0/3475816847 >> 172.21.15.90:0/3475816847 conn(0x7fb2d80fee00 msgr2=0x7fb2d80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:29.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.614+0000 7fb2dd200640 1 -- 172.21.15.90:0/3475816847 shutdown_connections 2024-05-27T07:46:29.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.614+0000 7fb2dd200640 1 -- 172.21.15.90:0/3475816847 wait complete. 2024-05-27T07:46:29.640 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T07:46:29.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:29 smithi090 ceph-mon[31429]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:29.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:29 smithi090 ceph-mon[31429]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T07:46:29.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:29 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1748378647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:46:29.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:29 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3475816847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:46:29.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.907+0000 7f12bc000640 1 -- 172.21.15.90:0/133387223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12b40750b0 msgr2=0x7f12b40754c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:29.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.907+0000 7f12bc000640 1 --2- 172.21.15.90:0/133387223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12b40750b0 0x7f12b40754c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009900 tx=0x7f12a802a410 comp rx=0 tx=0).stop 2024-05-27T07:46:29.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.907+0000 7f12bc000640 1 -- 172.21.15.90:0/133387223 shutdown_connections 2024-05-27T07:46:29.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.907+0000 7f12bc000640 1 --2- 172.21.15.90:0/133387223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b4110360 0x7f12b4112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.907+0000 7f12bc000640 1 --2- 172.21.15.90:0/133387223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12b4075a00 0x7f12b410fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.907+0000 7f12bc000640 1 --2- 172.21.15.90:0/133387223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12b40750b0 0x7f12b40754c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.907+0000 7f12bc000640 1 -- 172.21.15.90:0/133387223 >> 172.21.15.90:0/133387223 conn(0x7f12b40fedf0 msgr2=0x7f12b41012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:29.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.907+0000 7f12bc000640 1 -- 172.21.15.90:0/133387223 shutdown_connections 2024-05-27T07:46:29.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.907+0000 7f12bc000640 1 -- 172.21.15.90:0/133387223 wait complete. 2024-05-27T07:46:29.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.908+0000 7f12bc000640 1 Processor -- start 2024-05-27T07:46:29.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.908+0000 7f12bc000640 1 -- start start 2024-05-27T07:46:29.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.908+0000 7f12bc000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b40750b0 0x7f12b41a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:29.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.908+0000 7f12bc000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12b4075a00 0x7f12b41a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:29.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.908+0000 7f12bc000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12b4110360 0x7f12b41a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:29.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.908+0000 7f12bc000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4115c40 con 0x7f12b4110360 2024-05-27T07:46:29.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.908+0000 7f12bc000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4115ac0 con 0x7f12b40750b0 2024-05-27T07:46:29.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.908+0000 7f12bc000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f12b4115dc0 con 0x7f12b4075a00 2024-05-27T07:46:29.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.909+0000 7f12ba000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12b4110360 0x7f12b41a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:29.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.909+0000 7f12b9600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b40750b0 0x7f12b41a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:29.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.909+0000 7f12ba000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12b4110360 0x7f12b41a7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49682/0 (socket says 172.21.15.90:49682) 2024-05-27T07:46:29.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.909+0000 7f12ba000640 1 -- 172.21.15.90:0/3521776064 learned_addr learned my addr 172.21.15.90:0/3521776064 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:29.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.909+0000 7f12b8c00640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12b4075a00 0x7f12b41a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:29.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.909+0000 7f12ba000640 1 -- 172.21.15.90:0/3521776064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12b4075a00 msgr2=0x7f12b41a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:29.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.909+0000 7f12ba000640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12b4075a00 0x7f12b41a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.909+0000 7f12ba000640 1 -- 172.21.15.90:0/3521776064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b40750b0 msgr2=0x7f12b41a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:29.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.909+0000 7f12ba000640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b40750b0 0x7f12b41a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:29.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.909+0000 7f12ba000640 1 -- 172.21.15.90:0/3521776064 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b41a83d0 con 0x7f12b4110360 2024-05-27T07:46:29.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.910+0000 7f12ba000640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12b4110360 0x7f12b41a7d60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00dac0 tx=0x7f12ac00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:29.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.910+0000 7f12b2000640 1 -- 172.21.15.90:0/3521776064 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac017070 con 0x7f12b4110360 2024-05-27T07:46:29.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.910+0000 7f12b2000640 1 -- 172.21.15.90:0/3521776064 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12ac00be30 con 0x7f12b4110360 2024-05-27T07:46:29.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.910+0000 7f12b2000640 1 -- 172.21.15.90:0/3521776064 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac010460 con 0x7f12b4110360 2024-05-27T07:46:29.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.910+0000 7f12bc000640 1 -- 172.21.15.90:0/3521776064 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b41a86c0 con 0x7f12b4110360 2024-05-27T07:46:29.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.910+0000 7f12bc000640 1 -- 172.21.15.90:0/3521776064 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b41a8bc0 con 0x7f12b4110360 2024-05-27T07:46:29.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.911+0000 7f12bc000640 1 -- 172.21.15.90:0/3521776064 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b4105ff0 con 0x7f12b4110360 2024-05-27T07:46:29.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.914+0000 7f12b2000640 1 -- 172.21.15.90:0/3521776064 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7f12ac004a40 con 0x7f12b4110360 2024-05-27T07:46:29.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.915+0000 7f12b2000640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f129c0b29f0 0x7f129c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:29.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.916+0000 7f12b2000640 1 -- 172.21.15.90:0/3521776064 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f12ac0e2750 con 0x7f12b4110360 2024-05-27T07:46:29.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.916+0000 7f12b9600640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f129c0b29f0 0x7f129c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:29.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.916+0000 7f12b9600640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f129c0b29f0 0x7f129c0b4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f12a80057c0 tx=0x7f12a8031400 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:29.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:29.917+0000 7f12b2000640 1 -- 172.21.15.90:0/3521776064 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f12ac0a95c0 con 0x7f12b4110360 2024-05-27T07:46:30.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.138+0000 7f12bc000640 1 -- 172.21.15.90:0/3521776064 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f12b40638c0 con 0x7f12b4110360 2024-05-27T07:46:30.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.139+0000 7f12b2000640 1 -- 172.21.15.90:0/3521776064 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f12ac0ad8c0 con 0x7f12b4110360 2024-05-27T07:46:30.139 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-05-27T07:46:30.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 -- 172.21.15.90:0/3521776064 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f129c0b29f0 msgr2=0x7f129c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:30.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f129c0b29f0 0x7f129c0b4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f12a80057c0 tx=0x7f12a8031400 comp rx=0 tx=0).stop 2024-05-27T07:46:30.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 -- 172.21.15.90:0/3521776064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12b4110360 msgr2=0x7f12b41a7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:30.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12b4110360 0x7f12b41a7d60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00dac0 tx=0x7f12ac00df80 comp rx=0 tx=0).stop 2024-05-27T07:46:30.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 -- 172.21.15.90:0/3521776064 shutdown_connections 2024-05-27T07:46:30.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f129c0b29f0 0x7f129c0b4f00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12b4110360 0x7f12b41a7d60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12b4075a00 0x7f12b41a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 --2- 172.21.15.90:0/3521776064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12b40750b0 0x7f12b41a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 -- 172.21.15.90:0/3521776064 >> 172.21.15.90:0/3521776064 conn(0x7f12b40fedf0 msgr2=0x7f12b41031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:30.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 -- 172.21.15.90:0/3521776064 shutdown_connections 2024-05-27T07:46:30.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.142+0000 7f12bc000640 1 -- 172.21.15.90:0/3521776064 wait complete. 2024-05-27T07:46:30.169 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T07:46:30.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:29 smithi136 ceph-mon[35324]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:30.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:29 smithi136 ceph-mon[35324]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T07:46:30.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:29 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1748378647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:46:30.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:29 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3475816847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:46:30.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:29 smithi129 ceph-mon[35254]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:30.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:29 smithi129 ceph-mon[35254]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T07:46:30.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:29 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1748378647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T07:46:30.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:29 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3475816847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T07:46:30.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.432+0000 7fdad1000640 1 -- 172.21.15.90:0/4017634005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdacc108860 msgr2=0x7fdacc10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:30.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.432+0000 7fdad1000640 1 --2- 172.21.15.90:0/4017634005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdacc108860 0x7fdacc10ac80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009ca0 tx=0x7fdac402a8c0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.432+0000 7fdad1000640 1 -- 172.21.15.90:0/4017634005 shutdown_connections 2024-05-27T07:46:30.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.432+0000 7fdad1000640 1 --2- 172.21.15.90:0/4017634005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdacc103b60 0x7fdacc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.432+0000 7fdad1000640 1 --2- 172.21.15.90:0/4017634005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdacc108860 0x7fdacc10ac80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.432+0000 7fdad1000640 1 --2- 172.21.15.90:0/4017634005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc102920 0x7fdacc103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.432+0000 7fdad1000640 1 -- 172.21.15.90:0/4017634005 >> 172.21.15.90:0/4017634005 conn(0x7fdacc0fdb10 msgr2=0x7fdacc0fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:30.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.432+0000 7fdad1000640 1 -- 172.21.15.90:0/4017634005 shutdown_connections 2024-05-27T07:46:30.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.432+0000 7fdad1000640 1 -- 172.21.15.90:0/4017634005 wait complete. 2024-05-27T07:46:30.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.434+0000 7fdad1000640 1 Processor -- start 2024-05-27T07:46:30.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.434+0000 7fdad1000640 1 -- start start 2024-05-27T07:46:30.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.434+0000 7fdad1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc102920 0x7fdacc19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:30.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.434+0000 7fdad1000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdacc103b60 0x7fdacc19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:30.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.434+0000 7fdad1000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdacc108860 0x7fdacc1a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:30.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.434+0000 7fdad1000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc10f460 con 0x7fdacc108860 2024-05-27T07:46:30.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.434+0000 7fdad1000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc10f2e0 con 0x7fdacc103b60 2024-05-27T07:46:30.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.434+0000 7fdad1000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdacc10f5e0 con 0x7fdacc102920 2024-05-27T07:46:30.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.434+0000 7fdacbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdacc108860 0x7fdacc1a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:30.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacb400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc102920 0x7fdacc19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:30.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdacc108860 0x7fdacc1a3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49692/0 (socket says 172.21.15.90:49692) 2024-05-27T07:46:30.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacbe00640 1 -- 172.21.15.90:0/3313418603 learned_addr learned my addr 172.21.15.90:0/3313418603 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:30.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacaa00640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdacc103b60 0x7fdacc19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:30.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacb400640 1 -- 172.21.15.90:0/3313418603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdacc103b60 msgr2=0x7fdacc19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:30.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacb400640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdacc103b60 0x7fdacc19c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacb400640 1 -- 172.21.15.90:0/3313418603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdacc108860 msgr2=0x7fdacc1a3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:30.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacb400640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdacc108860 0x7fdacc1a3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacb400640 1 -- 172.21.15.90:0/3313418603 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdacc1a3d90 con 0x7fdacc102920 2024-05-27T07:46:30.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacaa00640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdacc103b60 0x7fdacc19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:30.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.435+0000 7fdacbe00640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdacc108860 0x7fdacc1a3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:46:30.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.436+0000 7fdacb400640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc102920 0x7fdacc19c250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdacc1021f0 tx=0x7fdabc00db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:30.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.440+0000 7fdac3e00640 1 -- 172.21.15.90:0/3313418603 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc017070 con 0x7fdacc102920 2024-05-27T07:46:30.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.440+0000 7fdac3e00640 1 -- 172.21.15.90:0/3313418603 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdabc009cb0 con 0x7fdacc102920 2024-05-27T07:46:30.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.440+0000 7fdac3e00640 1 -- 172.21.15.90:0/3313418603 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc003a60 con 0x7fdacc102920 2024-05-27T07:46:30.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.441+0000 7fdad1000640 1 -- 172.21.15.90:0/3313418603 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdacc1a4080 con 0x7fdacc102920 2024-05-27T07:46:30.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.441+0000 7fdad1000640 1 -- 172.21.15.90:0/3313418603 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdacc1a4580 con 0x7fdacc102920 2024-05-27T07:46:30.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.442+0000 7fdad1000640 1 -- 172.21.15.90:0/3313418603 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdacc1a47e0 con 0x7fdacc102920 2024-05-27T07:46:30.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.445+0000 7fdac3e00640 1 -- 172.21.15.90:0/3313418603 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7fdabc004220 con 0x7fdacc102920 2024-05-27T07:46:30.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.446+0000 7fdac3e00640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdab00b29e0 0x7fdab00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:30.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.446+0000 7fdac3e00640 1 -- 172.21.15.90:0/3313418603 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fdabc0e2930 con 0x7fdacc102920 2024-05-27T07:46:30.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.447+0000 7fdacaa00640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdab00b29e0 0x7fdab00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:30.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.447+0000 7fdacaa00640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdab00b29e0 0x7fdab00b4ef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005fd0 tx=0x7fdab8005850 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.460+0000 7fdac3e00640 1 -- 172.21.15.90:0/3313418603 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdabc0a9df0 con 0x7fdacc102920 2024-05-27T07:46:30.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.667+0000 7fdad1000640 1 -- 172.21.15.90:0/3313418603 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdacc1a49b0 con 0x7fdacc102920 2024-05-27T07:46:30.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.668+0000 7fdac3e00640 1 -- 172.21.15.90:0/3313418603 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fdabc0b0e20 con 0x7fdacc102920 2024-05-27T07:46:30.668 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-05-27T07:46:30.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 -- 172.21.15.90:0/3313418603 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdab00b29e0 msgr2=0x7fdab00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:30.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdab00b29e0 0x7fdab00b4ef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005fd0 tx=0x7fdab8005850 comp rx=0 tx=0).stop 2024-05-27T07:46:30.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 -- 172.21.15.90:0/3313418603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc102920 msgr2=0x7fdacc19c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:30.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc102920 0x7fdacc19c250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdacc1021f0 tx=0x7fdabc00db30 comp rx=0 tx=0).stop 2024-05-27T07:46:30.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 -- 172.21.15.90:0/3313418603 shutdown_connections 2024-05-27T07:46:30.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdab00b29e0 0x7fdab00b4ef0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdacc103b60 0x7fdacc19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdacc108860 0x7fdacc1a3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 --2- 172.21.15.90:0/3313418603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdacc102920 0x7fdacc19c250 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:30.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 -- 172.21.15.90:0/3313418603 >> 172.21.15.90:0/3313418603 conn(0x7fdacc0fdb10 msgr2=0x7fdacc0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:30.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 -- 172.21.15.90:0/3313418603 shutdown_connections 2024-05-27T07:46:30.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:30.671+0000 7fdad1000640 1 -- 172.21.15.90:0/3313418603 wait complete. 2024-05-27T07:46:30.694 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-27T07:46:30.694 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:46:30.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:30 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3521776064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:46:30.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:30 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:30.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:30 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:30.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:30 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:30.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:30 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:30.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:30 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:30.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:30 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:30.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:30 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3313418603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:46:31.128 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:31.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:30 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3521776064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:46:31.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:30 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:30 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:30 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:30 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:30 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:30 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:30 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3313418603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:46:31.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:30 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3521776064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T07:46:31.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:30 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:30 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:30 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:30 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:30 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:30 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:31.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:30 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3313418603' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T07:46:32.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:31 smithi090 ceph-mon[31429]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:32.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:31 smithi090 ceph-mon[31429]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T07:46:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:31 smithi136 ceph-mon[35324]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:32.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:31 smithi136 ceph-mon[35324]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T07:46:32.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:31 smithi129 ceph-mon[35254]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:32.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:31 smithi129 ceph-mon[35254]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T07:46:33.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.368+0000 7fb6f7e00700 1 -- 172.21.15.90:0/2395604660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f8108d20 msgr2=0x7fb6f8109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.368+0000 7fb6f7e00700 1 --2- 172.21.15.90:0/2395604660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f8108d20 0x7fb6f8109130 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00099c0 tx=0x7fb6e0009cd0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.368+0000 7fb6f7e00700 1 -- 172.21.15.90:0/2395604660 shutdown_connections 2024-05-27T07:46:33.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.368+0000 7fb6f7e00700 1 --2- 172.21.15.90:0/2395604660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8109670 0x7fb6f8110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.368+0000 7fb6f7e00700 1 --2- 172.21.15.90:0/2395604660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f81043f0 0x7fb6f8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.368+0000 7fb6f7e00700 1 --2- 172.21.15.90:0/2395604660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f8108d20 0x7fb6f8109130 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.368+0000 7fb6f7e00700 1 -- 172.21.15.90:0/2395604660 >> 172.21.15.90:0/2395604660 conn(0x7fb6f8079d00 msgr2=0x7fb6f807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:33.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.368+0000 7fb6f7e00700 1 -- 172.21.15.90:0/2395604660 shutdown_connections 2024-05-27T07:46:33.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.368+0000 7fb6f7e00700 1 -- 172.21.15.90:0/2395604660 wait complete. 2024-05-27T07:46:33.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi136 to 176.4M 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: Unable to set osd_memory_target on smithi136 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi129 to 176.4M 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: Unable to set osd_memory_target on smithi129 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-27T07:46:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: Updating smithi129:/etc/ceph/ceph.conf 2024-05-27T07:46:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: Updating smithi136:/etc/ceph/ceph.conf 2024-05-27T07:46:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:33.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:33 smithi090 ceph-mon[31429]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:46:33.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.370+0000 7fb6f7e00700 1 Processor -- start 2024-05-27T07:46:33.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.370+0000 7fb6f7e00700 1 -- start start 2024-05-27T07:46:33.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.371+0000 7fb6f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f81043f0 0x7fb6f8075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.371+0000 7fb6f7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f8108d20 0x7fb6f8075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.371+0000 7fb6f7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8109670 0x7fb6f8076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.371+0000 7fb6f7e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8114180 con 0x7fb6f81043f0 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.371+0000 7fb6f7e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8114000 con 0x7fb6f8109670 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.371+0000 7fb6f7e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f8114300 con 0x7fb6f8108d20 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.371+0000 7fb6f6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f81043f0 0x7fb6f8075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.371+0000 7fb6f6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f81043f0 0x7fb6f8075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49710/0 (socket says 172.21.15.90:49710) 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.371+0000 7fb6f6a00700 1 -- 172.21.15.90:0/4102871161 learned_addr learned my addr 172.21.15.90:0/4102871161 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.372+0000 7fb6f7400700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8109670 0x7fb6f8076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.372+0000 7fb6ee000700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f8108d20 0x7fb6f8075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.372+0000 7fb6f6a00700 1 -- 172.21.15.90:0/4102871161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f8108d20 msgr2=0x7fb6f8075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.372+0000 7fb6f6a00700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f8108d20 0x7fb6f8075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.372+0000 7fb6f6a00700 1 -- 172.21.15.90:0/4102871161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8109670 msgr2=0x7fb6f8076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.372+0000 7fb6f6a00700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8109670 0x7fb6f8076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.372+0000 7fb6f6a00700 1 -- 172.21.15.90:0/4102871161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f81a8f70 con 0x7fb6f81043f0 2024-05-27T07:46:33.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.373+0000 7fb6f6a00700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f81043f0 0x7fb6f8075ab0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800d950 tx=0x7fb6e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:33.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.374+0000 7fb6efe00700 1 -- 172.21.15.90:0/4102871161 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8016070 con 0x7fb6f81043f0 2024-05-27T07:46:33.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.374+0000 7fb6f7e00700 1 -- 172.21.15.90:0/4102871161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f81a9260 con 0x7fb6f81043f0 2024-05-27T07:46:33.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.374+0000 7fb6f7e00700 1 -- 172.21.15.90:0/4102871161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f81a97c0 con 0x7fb6f81043f0 2024-05-27T07:46:33.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.374+0000 7fb6efe00700 1 -- 172.21.15.90:0/4102871161 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6e800fd20 con 0x7fb6f81043f0 2024-05-27T07:46:33.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.374+0000 7fb6efe00700 1 -- 172.21.15.90:0/4102871161 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800e3b0 con 0x7fb6f81043f0 2024-05-27T07:46:33.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.377+0000 7fb6f7e00700 1 -- 172.21.15.90:0/4102871161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d4005330 con 0x7fb6f81043f0 2024-05-27T07:46:33.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.379+0000 7fb6efe00700 1 -- 172.21.15.90:0/4102871161 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fb6e8019070 con 0x7fb6f81043f0 2024-05-27T07:46:33.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.381+0000 7fb6efe00700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6dc0b2b90 0x7fb6dc0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:33.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.381+0000 7fb6efe00700 1 -- 172.21.15.90:0/4102871161 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fb6e80e29a0 con 0x7fb6f81043f0 2024-05-27T07:46:33.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.381+0000 7fb6ee000700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6dc0b2b90 0x7fb6dc0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:33.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.382+0000 7fb6ee000700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6dc0b2b90 0x7fb6dc0b50a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0000c00 tx=0x7fb6e001a040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:33.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.386+0000 7fb6efe00700 1 -- 172.21.15.90:0/4102871161 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb6e80a9dc0 con 0x7fb6f81043f0 2024-05-27T07:46:33.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi136 to 176.4M 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: Unable to set osd_memory_target on smithi136 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi129 to 176.4M 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: Unable to set osd_memory_target on smithi129 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: Updating smithi129:/etc/ceph/ceph.conf 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: Updating smithi136:/etc/ceph/ceph.conf 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:33.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:33 smithi136 ceph-mon[35324]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:46:33.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:33.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi136 to 176.4M 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: Unable to set osd_memory_target on smithi136 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi129 to 176.4M 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: Unable to set osd_memory_target on smithi129 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: Updating smithi090:/etc/ceph/ceph.conf 2024-05-27T07:46:33.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: Updating smithi129:/etc/ceph/ceph.conf 2024-05-27T07:46:33.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: Updating smithi136:/etc/ceph/ceph.conf 2024-05-27T07:46:33.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T07:46:33.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:33 smithi129 ceph-mon[35254]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:46:33.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.731+0000 7fb6f7e00700 1 -- 172.21.15.90:0/4102871161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6d40058f0 con 0x7fb6f81043f0 2024-05-27T07:46:33.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.732+0000 7fb6efe00700 1 -- 172.21.15.90:0/4102871161 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11941 (secure 0 0 0) 0x7fb6e80a96c0 con 0x7fb6f81043f0 2024-05-27T07:46:33.732 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:46:33.732 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:46:25.689163+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34442,34424,24478,34406]},"id":1}]} 2024-05-27T07:46:33.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 -- 172.21.15.90:0/4102871161 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6dc0b2b90 msgr2=0x7fb6dc0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6dc0b2b90 0x7fb6dc0b50a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0000c00 tx=0x7fb6e001a040 comp rx=0 tx=0).stop 2024-05-27T07:46:33.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 -- 172.21.15.90:0/4102871161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f81043f0 msgr2=0x7fb6f8075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:33.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f81043f0 0x7fb6f8075ab0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800d950 tx=0x7fb6e800dc60 comp rx=0 tx=0).stop 2024-05-27T07:46:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 -- 172.21.15.90:0/4102871161 shutdown_connections 2024-05-27T07:46:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6dc0b2b90 0x7fb6dc0b50a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8109670 0x7fb6f8076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f81043f0 0x7fb6f8075ab0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 --2- 172.21.15.90:0/4102871161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f8108d20 0x7fb6f8075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 -- 172.21.15.90:0/4102871161 >> 172.21.15.90:0/4102871161 conn(0x7fb6f8079d00 msgr2=0x7fb6f8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.735+0000 7fb6f7e00700 1 -- 172.21.15.90:0/4102871161 shutdown_connections 2024-05-27T07:46:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:33.736+0000 7fb6f7e00700 1 -- 172.21.15.90:0/4102871161 wait complete. 2024-05-27T07:46:33.737 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 30 2024-05-27T07:46:34.532 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-27T07:46:34.532 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-27T07:46:34.532 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T07:46:34.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:34 smithi090 ceph-mon[31429]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:46:34.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:34 smithi090 ceph-mon[31429]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:46:34.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:34 smithi090 ceph-mon[31429]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:46:34.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:34 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/4102871161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:46:34.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:34 smithi090 ceph-mon[31429]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:46:34.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:34 smithi090 ceph-mon[31429]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:46:34.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:34 smithi136 ceph-mon[35324]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:46:34.748 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:34 smithi136 ceph-mon[35324]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:46:34.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:34 smithi136 ceph-mon[35324]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:46:34.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:34 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/4102871161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:46:34.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:34 smithi136 ceph-mon[35324]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:46:34.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:34 smithi136 ceph-mon[35324]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:46:34.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.799+0000 7fc4dfe00640 1 -- 172.21.15.90:0/1732466939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d810b7e0 msgr2=0x7fc4d810dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:34.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.799+0000 7fc4dfe00640 1 --2- 172.21.15.90:0/1732466939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d810b7e0 0x7fc4d810dc00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009860 tx=0x7fc4c402a340 comp rx=0 tx=0).stop 2024-05-27T07:46:34.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.799+0000 7fc4dfe00640 1 -- 172.21.15.90:0/1732466939 shutdown_connections 2024-05-27T07:46:34.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.799+0000 7fc4dfe00640 1 --2- 172.21.15.90:0/1732466939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d810b7e0 0x7fc4d810dc00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:34.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.799+0000 7fc4dfe00640 1 --2- 172.21.15.90:0/1732466939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4d810e190 0x7fc4d8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:34.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.799+0000 7fc4dfe00640 1 --2- 172.21.15.90:0/1732466939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d8104d80 0x7fc4d8105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:34.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.799+0000 7fc4dfe00640 1 -- 172.21.15.90:0/1732466939 >> 172.21.15.90:0/1732466939 conn(0x7fc4d8078260 msgr2=0x7fc4d80786a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:34.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.799+0000 7fc4dfe00640 1 -- 172.21.15.90:0/1732466939 shutdown_connections 2024-05-27T07:46:34.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.799+0000 7fc4dfe00640 1 -- 172.21.15.90:0/1732466939 wait complete. 2024-05-27T07:46:34.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.800+0000 7fc4dfe00640 1 Processor -- start 2024-05-27T07:46:34.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dfe00640 1 -- start start 2024-05-27T07:46:34.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d8104d80 0x7fc4d8102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:34.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d810b7e0 0x7fc4d8102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:34.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4d810e190 0x7fc4d81030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:34.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dfe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8113c70 con 0x7fc4d810e190 2024-05-27T07:46:34.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dfe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8113af0 con 0x7fc4d810b7e0 2024-05-27T07:46:34.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dfe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8113df0 con 0x7fc4d8104d80 2024-05-27T07:46:34.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dde00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4d810e190 0x7fc4d81030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:34.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d810b7e0 0x7fc4d8102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:34.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d810b7e0 0x7fc4d8102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47168/0 (socket says 172.21.15.90:47168) 2024-05-27T07:46:34.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d8104d80 0x7fc4d8102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:34.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.801+0000 7fc4dca00640 1 -- 172.21.15.90:0/2317746374 learned_addr learned my addr 172.21.15.90:0/2317746374 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:34.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.802+0000 7fc4dca00640 1 -- 172.21.15.90:0/2317746374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d8104d80 msgr2=0x7fc4d8102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:34.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.802+0000 7fc4dca00640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d8104d80 0x7fc4d8102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:34.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.802+0000 7fc4dca00640 1 -- 172.21.15.90:0/2317746374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4d810e190 msgr2=0x7fc4d81030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:34.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.802+0000 7fc4dca00640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4d810e190 0x7fc4d81030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:34.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.802+0000 7fc4dca00640 1 -- 172.21.15.90:0/2317746374 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d81a8ad0 con 0x7fc4d810b7e0 2024-05-27T07:46:34.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.802+0000 7fc4dca00640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d810b7e0 0x7fc4d8102b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40057c0 tx=0x7fc4c402ae60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:34.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.803+0000 7fc4d6000640 1 -- 172.21.15.90:0/2317746374 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4040020 con 0x7fc4d810b7e0 2024-05-27T07:46:34.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.803+0000 7fc4dfe00640 1 -- 172.21.15.90:0/2317746374 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d81a8d60 con 0x7fc4d810b7e0 2024-05-27T07:46:34.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.803+0000 7fc4d6000640 1 -- 172.21.15.90:0/2317746374 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc4c40044d0 con 0x7fc4d810b7e0 2024-05-27T07:46:34.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.803+0000 7fc4d6000640 1 -- 172.21.15.90:0/2317746374 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4037040 con 0x7fc4d810b7e0 2024-05-27T07:46:34.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.803+0000 7fc4dfe00640 1 -- 172.21.15.90:0/2317746374 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d81a91e0 con 0x7fc4d810b7e0 2024-05-27T07:46:34.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.804+0000 7fc4dfe00640 1 -- 172.21.15.90:0/2317746374 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d806a230 con 0x7fc4d810b7e0 2024-05-27T07:46:34.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.807+0000 7fc4d6000640 1 -- 172.21.15.90:0/2317746374 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fc4c4043020 con 0x7fc4d810b7e0 2024-05-27T07:46:34.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.809+0000 7fc4d6000640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc4c00b2a80 0x7fc4c00b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:34.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.810+0000 7fc4d6000640 1 -- 172.21.15.90:0/2317746374 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fc4c4102f60 con 0x7fc4d810b7e0 2024-05-27T07:46:34.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.810+0000 7fc4dd400640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc4c00b2a80 0x7fc4c00b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:34.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.810+0000 7fc4dd400640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc4c00b2a80 0x7fc4c00b4f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009bd0 tx=0x7fc4cc005f90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:34.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:34 smithi129 ceph-mon[35254]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:46:34.812 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:34 smithi129 ceph-mon[35254]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T07:46:34.813 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:34 smithi129 ceph-mon[35254]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:46:34.813 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:34 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/4102871161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:46:34.813 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:34 smithi129 ceph-mon[35254]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:46:34.813 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:34 smithi129 ceph-mon[35254]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-05-27T07:46:34.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:34.824+0000 7fc4d6000640 1 -- 172.21.15.90:0/2317746374 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc4c404a020 con 0x7fc4d810b7e0 2024-05-27T07:46:35.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.068+0000 7fc4dfe00640 1 -- 172.21.15.90:0/2317746374 --> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4d81018f0 con 0x7fc4c00b2a80 2024-05-27T07:46:35.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.089+0000 7fc4d6000640 1 -- 172.21.15.90:0/2317746374 <== mgr.34967 v2:172.21.15.90:6800/4256436582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339499 (secure 0 0 0) 0x7fc4d81018f0 con 0x7fc4c00b2a80 2024-05-27T07:46:35.093 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:46:35.097 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-05-27T07:46:35.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 -- 172.21.15.90:0/2317746374 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc4c00b2a80 msgr2=0x7fc4c00b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:35.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc4c00b2a80 0x7fc4c00b4f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009bd0 tx=0x7fc4cc005f90 comp rx=0 tx=0).stop 2024-05-27T07:46:35.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 -- 172.21.15.90:0/2317746374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d810b7e0 msgr2=0x7fc4d8102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:35.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d810b7e0 0x7fc4d8102b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40057c0 tx=0x7fc4c402ae60 comp rx=0 tx=0).stop 2024-05-27T07:46:35.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 -- 172.21.15.90:0/2317746374 shutdown_connections 2024-05-27T07:46:35.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc4c00b2a80 0x7fc4c00b4f90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4d810b7e0 0x7fc4d8102b80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4d810e190 0x7fc4d81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 --2- 172.21.15.90:0/2317746374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4d8104d80 0x7fc4d8102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 -- 172.21.15.90:0/2317746374 >> 172.21.15.90:0/2317746374 conn(0x7fc4d8078260 msgr2=0x7fc4d81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:35.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 -- 172.21.15.90:0/2317746374 shutdown_connections 2024-05-27T07:46:35.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.097+0000 7fc4dfe00640 1 -- 172.21.15.90:0/2317746374 wait complete. 2024-05-27T07:46:35.126 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T07:46:34.889987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1202925,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2676,"num_read_kb":1643,"num_write":541,"num_write_kb":3647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":926,"ondisk_log_size":926,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361224,"kb_used_data":16208,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124580600,"statfs":{"total":1151940427776,"available":1151570534400,"internally_reserved":0,"allocated":16596992,"data_stored":9963149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19138,"internal_metadata":353154366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4084,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":211,"num_write":25,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.972610"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091553+0000","last_change":"2024-05-27T07:41:08.839265+0000","last_active":"2024-05-27T07:46:27.091553+0000","last_peered":"2024-05-27T07:46:27.091553+0000","last_clean":"2024-05-27T07:46:27.091553+0000","last_became_active":"2024-05-27T07:41:08.838814+0000","last_became_peered":"2024-05-27T07:41:08.838814+0000","last_unstale":"2024-05-27T07:46:27.091553+0000","last_undegraded":"2024-05-27T07:46:27.091553+0000","last_fullsized":"2024-05-27T07:46:27.091553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:32.901520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342694+0000","last_change":"2024-05-27T07:41:08.838313+0000","last_active":"2024-05-27T07:46:27.342694+0000","last_peered":"2024-05-27T07:46:27.342694+0000","last_clean":"2024-05-27T07:46:27.342694+0000","last_became_active":"2024-05-27T07:41:08.836657+0000","last_became_peered":"2024-05-27T07:41:08.836657+0000","last_unstale":"2024-05-27T07:46:27.342694+0000","last_undegraded":"2024-05-27T07:46:27.342694+0000","last_fullsized":"2024-05-27T07:46:27.342694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:33.556505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342426+0000","last_change":"2024-05-27T07:41:08.841513+0000","last_active":"2024-05-27T07:46:27.342426+0000","last_peered":"2024-05-27T07:46:27.342426+0000","last_clean":"2024-05-27T07:46:27.342426+0000","last_became_active":"2024-05-27T07:41:08.838624+0000","last_became_peered":"2024-05-27T07:41:08.838624+0000","last_unstale":"2024-05-27T07:46:27.342426+0000","last_undegraded":"2024-05-27T07:46:27.342426+0000","last_fullsized":"2024-05-27T07:46:27.342426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:01:37.320418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343623+0000","last_change":"2024-05-27T07:41:08.839138+0000","last_active":"2024-05-27T07:46:27.343623+0000","last_peered":"2024-05-27T07:46:27.343623+0000","last_clean":"2024-05-27T07:46:27.343623+0000","last_became_active":"2024-05-27T07:41:08.838766+0000","last_became_peered":"2024-05-27T07:41:08.838766+0000","last_unstale":"2024-05-27T07:46:27.343623+0000","last_undegraded":"2024-05-27T07:46:27.343623+0000","last_fullsized":"2024-05-27T07:46:27.343623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:09.129257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.858624+0000","last_change":"2024-05-27T07:41:08.823006+0000","last_active":"2024-05-27T07:46:26.858624+0000","last_peered":"2024-05-27T07:46:26.858624+0000","last_clean":"2024-05-27T07:46:26.858624+0000","last_became_active":"2024-05-27T07:41:08.822551+0000","last_became_peered":"2024-05-27T07:41:08.822551+0000","last_unstale":"2024-05-27T07:46:26.858624+0000","last_undegraded":"2024-05-27T07:46:26.858624+0000","last_fullsized":"2024-05-27T07:46:26.858624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:49.255763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856113+0000","last_change":"2024-05-27T07:41:08.846072+0000","last_active":"2024-05-27T07:46:26.856113+0000","last_peered":"2024-05-27T07:46:26.856113+0000","last_clean":"2024-05-27T07:46:26.856113+0000","last_became_active":"2024-05-27T07:41:08.845604+0000","last_became_peered":"2024-05-27T07:41:08.845604+0000","last_unstale":"2024-05-27T07:46:26.856113+0000","last_undegraded":"2024-05-27T07:46:26.856113+0000","last_fullsized":"2024-05-27T07:46:26.856113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:23:56.389135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350459+0000","last_change":"2024-05-27T07:41:08.841046+0000","last_active":"2024-05-27T07:46:27.350459+0000","last_peered":"2024-05-27T07:46:27.350459+0000","last_clean":"2024-05-27T07:46:27.350459+0000","last_became_active":"2024-05-27T07:41:08.840028+0000","last_became_peered":"2024-05-27T07:41:08.840028+0000","last_unstale":"2024-05-27T07:46:27.350459+0000","last_undegraded":"2024-05-27T07:46:27.350459+0000","last_fullsized":"2024-05-27T07:46:27.350459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:35.239919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013947+0000","last_change":"2024-05-27T07:41:08.841182+0000","last_active":"2024-05-27T07:46:27.013947+0000","last_peered":"2024-05-27T07:46:27.013947+0000","last_clean":"2024-05-27T07:46:27.013947+0000","last_became_active":"2024-05-27T07:41:08.840645+0000","last_became_peered":"2024-05-27T07:41:08.840645+0000","last_unstale":"2024-05-27T07:46:27.013947+0000","last_undegraded":"2024-05-27T07:46:27.013947+0000","last_fullsized":"2024-05-27T07:46:27.013947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:10:59.744505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342789+0000","last_change":"2024-05-27T07:41:08.825874+0000","last_active":"2024-05-27T07:46:27.342789+0000","last_peered":"2024-05-27T07:46:27.342789+0000","last_clean":"2024-05-27T07:46:27.342789+0000","last_became_active":"2024-05-27T07:41:08.825386+0000","last_became_peered":"2024-05-27T07:41:08.825386+0000","last_unstale":"2024-05-27T07:46:27.342789+0000","last_undegraded":"2024-05-27T07:46:27.342789+0000","last_fullsized":"2024-05-27T07:46:27.342789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:13.304612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.857247+0000","last_change":"2024-05-27T07:41:08.847445+0000","last_active":"2024-05-27T07:46:26.857247+0000","last_peered":"2024-05-27T07:46:26.857247+0000","last_clean":"2024-05-27T07:46:26.857247+0000","last_became_active":"2024-05-27T07:41:08.846512+0000","last_became_peered":"2024-05-27T07:41:08.846512+0000","last_unstale":"2024-05-27T07:46:26.857247+0000","last_undegraded":"2024-05-27T07:46:26.857247+0000","last_fullsized":"2024-05-27T07:46:26.857247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:57.235218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350533+0000","last_change":"2024-05-27T07:41:08.843531+0000","last_active":"2024-05-27T07:46:27.350533+0000","last_peered":"2024-05-27T07:46:27.350533+0000","last_clean":"2024-05-27T07:46:27.350533+0000","last_became_active":"2024-05-27T07:41:08.843214+0000","last_became_peered":"2024-05-27T07:41:08.843214+0000","last_unstale":"2024-05-27T07:46:27.350533+0000","last_undegraded":"2024-05-27T07:46:27.350533+0000","last_fullsized":"2024-05-27T07:46:27.350533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:00.884178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343952+0000","last_change":"2024-05-27T07:41:08.839487+0000","last_active":"2024-05-27T07:46:27.343952+0000","last_peered":"2024-05-27T07:46:27.343952+0000","last_clean":"2024-05-27T07:46:27.343952+0000","last_became_active":"2024-05-27T07:41:08.839097+0000","last_became_peered":"2024-05-27T07:41:08.839097+0000","last_unstale":"2024-05-27T07:46:27.343952+0000","last_undegraded":"2024-05-27T07:46:27.343952+0000","last_fullsized":"2024-05-27T07:46:27.343952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:51.055482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517759+0000","last_change":"2024-05-27T07:41:08.836843+0000","last_active":"2024-05-27T07:46:27.517759+0000","last_peered":"2024-05-27T07:46:27.517759+0000","last_clean":"2024-05-27T07:46:27.517759+0000","last_became_active":"2024-05-27T07:41:08.836362+0000","last_became_peered":"2024-05-27T07:41:08.836362+0000","last_unstale":"2024-05-27T07:46:27.517759+0000","last_undegraded":"2024-05-27T07:46:27.517759+0000","last_fullsized":"2024-05-27T07:46:27.517759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:47.844301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341794+0000","last_change":"2024-05-27T07:41:08.839046+0000","last_active":"2024-05-27T07:46:27.341794+0000","last_peered":"2024-05-27T07:46:27.341794+0000","last_clean":"2024-05-27T07:46:27.341794+0000","last_became_active":"2024-05-27T07:41:08.838601+0000","last_became_peered":"2024-05-27T07:41:08.838601+0000","last_unstale":"2024-05-27T07:46:27.341794+0000","last_undegraded":"2024-05-27T07:46:27.341794+0000","last_fullsized":"2024-05-27T07:46:27.341794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:59.563490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341873+0000","last_change":"2024-05-27T07:41:08.832510+0000","last_active":"2024-05-27T07:46:27.341873+0000","last_peered":"2024-05-27T07:46:27.341873+0000","last_clean":"2024-05-27T07:46:27.341873+0000","last_became_active":"2024-05-27T07:41:08.832082+0000","last_became_peered":"2024-05-27T07:41:08.832082+0000","last_unstale":"2024-05-27T07:46:27.341873+0000","last_undegraded":"2024-05-27T07:46:27.341873+0000","last_fullsized":"2024-05-27T07:46:27.341873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:59.951860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339613+0000","last_change":"2024-05-27T07:41:08.827953+0000","last_active":"2024-05-27T07:46:27.339613+0000","last_peered":"2024-05-27T07:46:27.339613+0000","last_clean":"2024-05-27T07:46:27.339613+0000","last_became_active":"2024-05-27T07:41:08.827293+0000","last_became_peered":"2024-05-27T07:41:08.827293+0000","last_unstale":"2024-05-27T07:46:27.339613+0000","last_undegraded":"2024-05-27T07:46:27.339613+0000","last_fullsized":"2024-05-27T07:46:27.339613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:08.810514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091279+0000","last_change":"2024-05-27T07:41:08.839591+0000","last_active":"2024-05-27T07:46:27.091279+0000","last_peered":"2024-05-27T07:46:27.091279+0000","last_clean":"2024-05-27T07:46:27.091279+0000","last_became_active":"2024-05-27T07:41:08.839217+0000","last_became_peered":"2024-05-27T07:41:08.839217+0000","last_unstale":"2024-05-27T07:46:27.091279+0000","last_undegraded":"2024-05-27T07:46:27.091279+0000","last_fullsized":"2024-05-27T07:46:27.091279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:11:41.636163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856457+0000","last_change":"2024-05-27T07:41:08.839544+0000","last_active":"2024-05-27T07:46:26.856457+0000","last_peered":"2024-05-27T07:46:26.856457+0000","last_clean":"2024-05-27T07:46:26.856457+0000","last_became_active":"2024-05-27T07:41:08.839177+0000","last_became_peered":"2024-05-27T07:41:08.839177+0000","last_unstale":"2024-05-27T07:46:26.856457+0000","last_undegraded":"2024-05-27T07:46:26.856457+0000","last_fullsized":"2024-05-27T07:46:26.856457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:41.921759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339195+0000","last_change":"2024-05-27T07:41:08.847911+0000","last_active":"2024-05-27T07:46:27.339195+0000","last_peered":"2024-05-27T07:46:27.339195+0000","last_clean":"2024-05-27T07:46:27.339195+0000","last_became_active":"2024-05-27T07:41:08.847427+0000","last_became_peered":"2024-05-27T07:41:08.847427+0000","last_unstale":"2024-05-27T07:46:27.339195+0000","last_undegraded":"2024-05-27T07:46:27.339195+0000","last_fullsized":"2024-05-27T07:46:27.339195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:13.692744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338331+0000","last_change":"2024-05-27T07:41:21.999614+0000","last_active":"2024-05-27T07:46:27.338331+0000","last_peered":"2024-05-27T07:46:27.338331+0000","last_clean":"2024-05-27T07:46:27.338331+0000","last_became_active":"2024-05-27T07:41:21.999086+0000","last_became_peered":"2024-05-27T07:41:21.999086+0000","last_unstale":"2024-05-27T07:46:27.338331+0000","last_undegraded":"2024-05-27T07:46:27.338331+0000","last_fullsized":"2024-05-27T07:46:27.338331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:20.229318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":301,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.592306+0000","last_change":"2024-05-27T07:41:04.795079+0000","last_active":"2024-05-27T07:46:31.592306+0000","last_peered":"2024-05-27T07:46:31.592306+0000","last_clean":"2024-05-27T07:46:31.592306+0000","last_became_active":"2024-05-27T07:41:04.794367+0000","last_became_peered":"2024-05-27T07:41:04.794367+0000","last_unstale":"2024-05-27T07:46:31.592306+0000","last_undegraded":"2024-05-27T07:46:31.592306+0000","last_fullsized":"2024-05-27T07:46:31.592306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:27:52.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014433+0000","last_change":"2024-05-27T07:41:08.846712+0000","last_active":"2024-05-27T07:46:27.014433+0000","last_peered":"2024-05-27T07:46:27.014433+0000","last_clean":"2024-05-27T07:46:27.014433+0000","last_became_active":"2024-05-27T07:41:08.839999+0000","last_became_peered":"2024-05-27T07:41:08.839999+0000","last_unstale":"2024-05-27T07:46:27.014433+0000","last_undegraded":"2024-05-27T07:46:27.014433+0000","last_fullsized":"2024-05-27T07:46:27.014433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:31:23.924074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.340435+0000","last_change":"2024-05-27T07:41:21.996675+0000","last_active":"2024-05-27T07:46:27.340435+0000","last_peered":"2024-05-27T07:46:27.340435+0000","last_clean":"2024-05-27T07:46:27.340435+0000","last_became_active":"2024-05-27T07:41:21.996152+0000","last_became_peered":"2024-05-27T07:41:21.996152+0000","last_unstale":"2024-05-27T07:46:27.340435+0000","last_undegraded":"2024-05-27T07:46:27.340435+0000","last_fullsized":"2024-05-27T07:46:27.340435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:57.415146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014696+0000","last_change":"2024-05-27T07:41:04.890045+0000","last_active":"2024-05-27T07:46:27.014696+0000","last_peered":"2024-05-27T07:46:27.014696+0000","last_clean":"2024-05-27T07:46:27.014696+0000","last_became_active":"2024-05-27T07:41:04.888993+0000","last_became_peered":"2024-05-27T07:41:04.888993+0000","last_unstale":"2024-05-27T07:46:27.014696+0000","last_undegraded":"2024-05-27T07:46:27.014696+0000","last_fullsized":"2024-05-27T07:46:27.014696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:19.866699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342407+0000","last_change":"2024-05-27T07:41:08.847472+0000","last_active":"2024-05-27T07:46:27.342407+0000","last_peered":"2024-05-27T07:46:27.342407+0000","last_clean":"2024-05-27T07:46:27.342407+0000","last_became_active":"2024-05-27T07:41:08.846070+0000","last_became_peered":"2024-05-27T07:41:08.846070+0000","last_unstale":"2024-05-27T07:46:27.342407+0000","last_undegraded":"2024-05-27T07:46:27.342407+0000","last_fullsized":"2024-05-27T07:46:27.342407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:29:08.497007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343782+0000","last_change":"2024-05-27T07:41:59.842473+0000","last_active":"2024-05-27T07:46:27.343782+0000","last_peered":"2024-05-27T07:46:27.343782+0000","last_clean":"2024-05-27T07:46:27.343782+0000","last_became_active":"2024-05-27T07:41:59.841035+0000","last_became_peered":"2024-05-27T07:41:59.841035+0000","last_unstale":"2024-05-27T07:46:27.343782+0000","last_undegraded":"2024-05-27T07:46:27.343782+0000","last_fullsized":"2024-05-27T07:46:27.343782+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:52.959589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'13","reported_seq":62,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.860220+0000","last_change":"2024-05-27T07:41:04.804046+0000","last_active":"2024-05-27T07:46:26.860220+0000","last_peered":"2024-05-27T07:46:26.860220+0000","last_clean":"2024-05-27T07:46:26.860220+0000","last_became_active":"2024-05-27T07:41:04.803360+0000","last_became_peered":"2024-05-27T07:41:04.803360+0000","last_unstale":"2024-05-27T07:46:26.860220+0000","last_undegraded":"2024-05-27T07:46:26.860220+0000","last_fullsized":"2024-05-27T07:46:26.860220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:17.325221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341836+0000","last_change":"2024-05-27T07:41:08.840842+0000","last_active":"2024-05-27T07:46:27.341836+0000","last_peered":"2024-05-27T07:46:27.341836+0000","last_clean":"2024-05-27T07:46:27.341836+0000","last_became_active":"2024-05-27T07:41:08.837817+0000","last_became_peered":"2024-05-27T07:41:08.837817+0000","last_unstale":"2024-05-27T07:46:27.341836+0000","last_undegraded":"2024-05-27T07:46:27.341836+0000","last_fullsized":"2024-05-27T07:46:27.341836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:53:38.130608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.340944+0000","last_change":"2024-05-27T07:41:22.002988+0000","last_active":"2024-05-27T07:46:27.340944+0000","last_peered":"2024-05-27T07:46:27.340944+0000","last_clean":"2024-05-27T07:46:27.340944+0000","last_became_active":"2024-05-27T07:41:22.001141+0000","last_became_peered":"2024-05-27T07:41:22.001141+0000","last_unstale":"2024-05-27T07:46:27.340944+0000","last_undegraded":"2024-05-27T07:46:27.340944+0000","last_fullsized":"2024-05-27T07:46:27.340944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:29:17.380439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.859451+0000","last_change":"2024-05-27T07:41:04.794424+0000","last_active":"2024-05-27T07:46:26.859451+0000","last_peered":"2024-05-27T07:46:26.859451+0000","last_clean":"2024-05-27T07:46:26.859451+0000","last_became_active":"2024-05-27T07:41:04.793912+0000","last_became_peered":"2024-05-27T07:41:04.793912+0000","last_unstale":"2024-05-27T07:46:26.859451+0000","last_undegraded":"2024-05-27T07:46:26.859451+0000","last_fullsized":"2024-05-27T07:46:26.859451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:50.876891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.031011+0000","last_change":"2024-05-27T07:41:08.832356+0000","last_active":"2024-05-27T07:46:27.031011+0000","last_peered":"2024-05-27T07:46:27.031011+0000","last_clean":"2024-05-27T07:46:27.031011+0000","last_became_active":"2024-05-27T07:41:08.831979+0000","last_became_peered":"2024-05-27T07:41:08.831979+0000","last_unstale":"2024-05-27T07:46:27.031011+0000","last_undegraded":"2024-05-27T07:46:27.031011+0000","last_fullsized":"2024-05-27T07:46:27.031011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:37:47.584177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338988+0000","last_change":"2024-05-27T07:41:22.004233+0000","last_active":"2024-05-27T07:46:27.338988+0000","last_peered":"2024-05-27T07:46:27.338988+0000","last_clean":"2024-05-27T07:46:27.338988+0000","last_became_active":"2024-05-27T07:41:22.003781+0000","last_became_peered":"2024-05-27T07:41:22.003781+0000","last_unstale":"2024-05-27T07:46:27.338988+0000","last_undegraded":"2024-05-27T07:46:27.338988+0000","last_fullsized":"2024-05-27T07:46:27.338988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:02:13.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'10","reported_seq":303,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.671516+0000","last_change":"2024-05-27T07:41:04.800373+0000","last_active":"2024-05-27T07:46:31.671516+0000","last_peered":"2024-05-27T07:46:31.671516+0000","last_clean":"2024-05-27T07:46:31.671516+0000","last_became_active":"2024-05-27T07:41:04.799930+0000","last_became_peered":"2024-05-27T07:41:04.799930+0000","last_unstale":"2024-05-27T07:46:31.671516+0000","last_undegraded":"2024-05-27T07:46:31.671516+0000","last_fullsized":"2024-05-27T07:46:31.671516+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:24.642479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2413,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":245,"num_read_kb":7,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.857099+0000","last_change":"2024-05-27T07:41:08.830387+0000","last_active":"2024-05-27T07:46:26.857099+0000","last_peered":"2024-05-27T07:46:26.857099+0000","last_clean":"2024-05-27T07:46:26.857099+0000","last_became_active":"2024-05-27T07:41:08.829720+0000","last_became_peered":"2024-05-27T07:41:08.829720+0000","last_unstale":"2024-05-27T07:46:26.857099+0000","last_undegraded":"2024-05-27T07:46:26.857099+0000","last_fullsized":"2024-05-27T07:46:26.857099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:42.014383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338425+0000","last_change":"2024-05-27T07:41:22.005700+0000","last_active":"2024-05-27T07:46:27.338425+0000","last_peered":"2024-05-27T07:46:27.338425+0000","last_clean":"2024-05-27T07:46:27.338425+0000","last_became_active":"2024-05-27T07:41:22.003643+0000","last_became_peered":"2024-05-27T07:41:22.003643+0000","last_unstale":"2024-05-27T07:46:27.338425+0000","last_undegraded":"2024-05-27T07:46:27.338425+0000","last_fullsized":"2024-05-27T07:46:27.338425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:48.332767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.855954+0000","last_change":"2024-05-27T07:41:59.790579+0000","last_active":"2024-05-27T07:46:26.855954+0000","last_peered":"2024-05-27T07:46:26.855954+0000","last_clean":"2024-05-27T07:46:26.855954+0000","last_became_active":"2024-05-27T07:41:59.789849+0000","last_became_peered":"2024-05-27T07:41:59.789849+0000","last_unstale":"2024-05-27T07:46:26.855954+0000","last_undegraded":"2024-05-27T07:46:26.855954+0000","last_fullsized":"2024-05-27T07:46:26.855954+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:24:30.564074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.851628+0000","last_change":"2024-05-27T07:43:00.121063+0000","last_active":"2024-05-27T07:46:26.851628+0000","last_peered":"2024-05-27T07:46:26.851628+0000","last_clean":"2024-05-27T07:46:26.851628+0000","last_became_active":"2024-05-27T07:43:00.119977+0000","last_became_peered":"2024-05-27T07:43:00.119977+0000","last_unstale":"2024-05-27T07:46:26.851628+0000","last_undegraded":"2024-05-27T07:46:26.851628+0000","last_fullsized":"2024-05-27T07:46:26.851628+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:04:43.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091113+0000","last_change":"2024-05-27T07:41:22.013820+0000","last_active":"2024-05-27T07:46:27.091113+0000","last_peered":"2024-05-27T07:46:27.091113+0000","last_clean":"2024-05-27T07:46:27.091113+0000","last_became_active":"2024-05-27T07:41:22.013110+0000","last_became_peered":"2024-05-27T07:41:22.013110+0000","last_unstale":"2024-05-27T07:46:27.091113+0000","last_undegraded":"2024-05-27T07:46:27.091113+0000","last_fullsized":"2024-05-27T07:46:27.091113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:45:07.317872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517879+0000","last_change":"2024-05-27T07:41:04.789197+0000","last_active":"2024-05-27T07:46:27.517879+0000","last_peered":"2024-05-27T07:46:27.517879+0000","last_clean":"2024-05-27T07:46:27.517879+0000","last_became_active":"2024-05-27T07:41:04.788755+0000","last_became_peered":"2024-05-27T07:41:04.788755+0000","last_unstale":"2024-05-27T07:46:27.517879+0000","last_undegraded":"2024-05-27T07:46:27.517879+0000","last_fullsized":"2024-05-27T07:46:27.517879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:27:54.526910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013815+0000","last_change":"2024-05-27T07:41:08.841769+0000","last_active":"2024-05-27T07:46:27.013815+0000","last_peered":"2024-05-27T07:46:27.013815+0000","last_clean":"2024-05-27T07:46:27.013815+0000","last_became_active":"2024-05-27T07:41:08.840834+0000","last_became_peered":"2024-05-27T07:41:08.840834+0000","last_unstale":"2024-05-27T07:46:27.013815+0000","last_undegraded":"2024-05-27T07:46:27.013815+0000","last_fullsized":"2024-05-27T07:46:27.013815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:15.573739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342674+0000","last_change":"2024-05-27T07:41:22.005119+0000","last_active":"2024-05-27T07:46:27.342674+0000","last_peered":"2024-05-27T07:46:27.342674+0000","last_clean":"2024-05-27T07:46:27.342674+0000","last_became_active":"2024-05-27T07:41:22.002638+0000","last_became_peered":"2024-05-27T07:41:22.002638+0000","last_unstale":"2024-05-27T07:46:27.342674+0000","last_undegraded":"2024-05-27T07:46:27.342674+0000","last_fullsized":"2024-05-27T07:46:27.342674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:08:49.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029700+0000","last_change":"2024-05-27T07:41:04.799864+0000","last_active":"2024-05-27T07:46:27.029700+0000","last_peered":"2024-05-27T07:46:27.029700+0000","last_clean":"2024-05-27T07:46:27.029700+0000","last_became_active":"2024-05-27T07:41:04.799218+0000","last_became_peered":"2024-05-27T07:41:04.799218+0000","last_unstale":"2024-05-27T07:46:27.029700+0000","last_undegraded":"2024-05-27T07:46:27.029700+0000","last_fullsized":"2024-05-27T07:46:27.029700+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:15.355720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.851911+0000","last_change":"2024-05-27T07:41:08.823387+0000","last_active":"2024-05-27T07:46:26.851911+0000","last_peered":"2024-05-27T07:46:26.851911+0000","last_clean":"2024-05-27T07:46:26.851911+0000","last_became_active":"2024-05-27T07:41:08.822943+0000","last_became_peered":"2024-05-27T07:41:08.822943+0000","last_unstale":"2024-05-27T07:46:26.851911+0000","last_undegraded":"2024-05-27T07:46:26.851911+0000","last_fullsized":"2024-05-27T07:46:26.851911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:46.365284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013958+0000","last_change":"2024-05-27T07:41:22.011848+0000","last_active":"2024-05-27T07:46:27.013958+0000","last_peered":"2024-05-27T07:46:27.013958+0000","last_clean":"2024-05-27T07:46:27.013958+0000","last_became_active":"2024-05-27T07:41:22.011335+0000","last_became_peered":"2024-05-27T07:41:22.011335+0000","last_unstale":"2024-05-27T07:46:27.013958+0000","last_undegraded":"2024-05-27T07:46:27.013958+0000","last_fullsized":"2024-05-27T07:46:27.013958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:29:22.528270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":531,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:32.651357+0000","last_change":"2024-05-27T07:41:04.793877+0000","last_active":"2024-05-27T07:46:32.651357+0000","last_peered":"2024-05-27T07:46:32.651357+0000","last_clean":"2024-05-27T07:46:32.651357+0000","last_became_active":"2024-05-27T07:41:04.793389+0000","last_became_peered":"2024-05-27T07:41:04.793389+0000","last_unstale":"2024-05-27T07:46:32.651357+0000","last_undegraded":"2024-05-27T07:46:32.651357+0000","last_fullsized":"2024-05-27T07:46:32.651357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:48.670572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342653+0000","last_change":"2024-05-27T07:41:08.836682+0000","last_active":"2024-05-27T07:46:27.342653+0000","last_peered":"2024-05-27T07:46:27.342653+0000","last_clean":"2024-05-27T07:46:27.342653+0000","last_became_active":"2024-05-27T07:41:08.836229+0000","last_became_peered":"2024-05-27T07:41:08.836229+0000","last_unstale":"2024-05-27T07:46:27.342653+0000","last_undegraded":"2024-05-27T07:46:27.342653+0000","last_fullsized":"2024-05-27T07:46:27.342653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:33.249584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.850945+0000","last_change":"2024-05-27T07:41:22.004968+0000","last_active":"2024-05-27T07:46:26.850945+0000","last_peered":"2024-05-27T07:46:26.850945+0000","last_clean":"2024-05-27T07:46:26.850945+0000","last_became_active":"2024-05-27T07:41:22.001407+0000","last_became_peered":"2024-05-27T07:41:22.001407+0000","last_unstale":"2024-05-27T07:46:26.850945+0000","last_undegraded":"2024-05-27T07:46:26.850945+0000","last_fullsized":"2024-05-27T07:46:26.850945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:26.872497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091231+0000","last_change":"2024-05-27T07:41:59.787247+0000","last_active":"2024-05-27T07:46:27.091231+0000","last_peered":"2024-05-27T07:46:27.091231+0000","last_clean":"2024-05-27T07:46:27.091231+0000","last_became_active":"2024-05-27T07:41:59.786392+0000","last_became_peered":"2024-05-27T07:41:59.786392+0000","last_unstale":"2024-05-27T07:46:27.091231+0000","last_undegraded":"2024-05-27T07:46:27.091231+0000","last_fullsized":"2024-05-27T07:46:27.091231+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:12.579707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2],"acting":[11,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342877+0000","last_change":"2024-05-27T07:41:08.833731+0000","last_active":"2024-05-27T07:46:27.342877+0000","last_peered":"2024-05-27T07:46:27.342877+0000","last_clean":"2024-05-27T07:46:27.342877+0000","last_became_active":"2024-05-27T07:41:08.833318+0000","last_became_peered":"2024-05-27T07:41:08.833318+0000","last_unstale":"2024-05-27T07:46:27.342877+0000","last_undegraded":"2024-05-27T07:46:27.342877+0000","last_fullsized":"2024-05-27T07:46:27.342877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:50.085113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343580+0000","last_change":"2024-05-27T07:41:59.789784+0000","last_active":"2024-05-27T07:46:27.343580+0000","last_peered":"2024-05-27T07:46:27.343580+0000","last_clean":"2024-05-27T07:46:27.343580+0000","last_became_active":"2024-05-27T07:41:59.788589+0000","last_became_peered":"2024-05-27T07:41:59.788589+0000","last_unstale":"2024-05-27T07:46:27.343580+0000","last_undegraded":"2024-05-27T07:46:27.343580+0000","last_fullsized":"2024-05-27T07:46:27.343580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:37.026907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030707+0000","last_change":"2024-05-27T07:41:04.892603+0000","last_active":"2024-05-27T07:46:27.030707+0000","last_peered":"2024-05-27T07:46:27.030707+0000","last_clean":"2024-05-27T07:46:27.030707+0000","last_became_active":"2024-05-27T07:41:04.890786+0000","last_became_peered":"2024-05-27T07:41:04.890786+0000","last_unstale":"2024-05-27T07:46:27.030707+0000","last_undegraded":"2024-05-27T07:46:27.030707+0000","last_fullsized":"2024-05-27T07:46:27.030707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:34.129227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342154+0000","last_change":"2024-05-27T07:41:08.830802+0000","last_active":"2024-05-27T07:46:27.342154+0000","last_peered":"2024-05-27T07:46:27.342154+0000","last_clean":"2024-05-27T07:46:27.342154+0000","last_became_active":"2024-05-27T07:41:08.829661+0000","last_became_peered":"2024-05-27T07:41:08.829661+0000","last_unstale":"2024-05-27T07:46:27.342154+0000","last_undegraded":"2024-05-27T07:46:27.342154+0000","last_fullsized":"2024-05-27T07:46:27.342154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:20:15.041554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.852318+0000","last_change":"2024-05-27T07:41:22.003997+0000","last_active":"2024-05-27T07:46:26.852318+0000","last_peered":"2024-05-27T07:46:26.852318+0000","last_clean":"2024-05-27T07:46:26.852318+0000","last_became_active":"2024-05-27T07:41:22.003527+0000","last_became_peered":"2024-05-27T07:41:22.003527+0000","last_unstale":"2024-05-27T07:46:26.852318+0000","last_undegraded":"2024-05-27T07:46:26.852318+0000","last_fullsized":"2024-05-27T07:46:26.852318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:17.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'47","reported_seq":603,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:32.704711+0000","last_change":"2024-05-27T07:41:59.814227+0000","last_active":"2024-05-27T07:46:32.704711+0000","last_peered":"2024-05-27T07:46:32.704711+0000","last_clean":"2024-05-27T07:46:32.704711+0000","last_became_active":"2024-05-27T07:41:59.788510+0000","last_became_peered":"2024-05-27T07:41:59.788510+0000","last_unstale":"2024-05-27T07:46:32.704711+0000","last_undegraded":"2024-05-27T07:46:32.704711+0000","last_fullsized":"2024-05-27T07:46:32.704711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:15.868348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75767,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":320,"num_write":49,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.859696+0000","last_change":"2024-05-27T07:41:08.831639+0000","last_active":"2024-05-27T07:46:26.859696+0000","last_peered":"2024-05-27T07:46:26.859696+0000","last_clean":"2024-05-27T07:46:26.859696+0000","last_became_active":"2024-05-27T07:41:08.830541+0000","last_became_peered":"2024-05-27T07:41:08.830541+0000","last_unstale":"2024-05-27T07:46:26.859696+0000","last_undegraded":"2024-05-27T07:46:26.859696+0000","last_fullsized":"2024-05-27T07:46:26.859696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:45.635145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.031603+0000","last_change":"2024-05-27T07:41:21.999587+0000","last_active":"2024-05-27T07:46:27.031603+0000","last_peered":"2024-05-27T07:46:27.031603+0000","last_clean":"2024-05-27T07:46:27.031603+0000","last_became_active":"2024-05-27T07:41:21.999017+0000","last_became_peered":"2024-05-27T07:41:21.999017+0000","last_unstale":"2024-05-27T07:46:27.031603+0000","last_undegraded":"2024-05-27T07:46:27.031603+0000","last_fullsized":"2024-05-27T07:46:27.031603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:31:14.915229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338645+0000","last_change":"2024-05-27T07:41:04.799847+0000","last_active":"2024-05-27T07:46:27.338645+0000","last_peered":"2024-05-27T07:46:27.338645+0000","last_clean":"2024-05-27T07:46:27.338645+0000","last_became_active":"2024-05-27T07:41:04.798605+0000","last_became_peered":"2024-05-27T07:41:04.798605+0000","last_unstale":"2024-05-27T07:46:27.338645+0000","last_undegraded":"2024-05-27T07:46:27.338645+0000","last_fullsized":"2024-05-27T07:46:27.338645+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:01.957707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517549+0000","last_change":"2024-05-27T07:41:08.831138+0000","last_active":"2024-05-27T07:46:27.517549+0000","last_peered":"2024-05-27T07:46:27.517549+0000","last_clean":"2024-05-27T07:46:27.517549+0000","last_became_active":"2024-05-27T07:41:08.830735+0000","last_became_peered":"2024-05-27T07:41:08.830735+0000","last_unstale":"2024-05-27T07:46:27.517549+0000","last_undegraded":"2024-05-27T07:46:27.517549+0000","last_fullsized":"2024-05-27T07:46:27.517549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:12.569687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090504+0000","last_change":"2024-05-27T07:41:59.779039+0000","last_active":"2024-05-27T07:46:27.090504+0000","last_peered":"2024-05-27T07:46:27.090504+0000","last_clean":"2024-05-27T07:46:27.090504+0000","last_became_active":"2024-05-27T07:41:59.777514+0000","last_became_peered":"2024-05-27T07:41:59.777514+0000","last_unstale":"2024-05-27T07:46:27.090504+0000","last_undegraded":"2024-05-27T07:46:27.090504+0000","last_fullsized":"2024-05-27T07:46:27.090504+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:12:19.223874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030345+0000","last_change":"2024-05-27T07:41:04.796467+0000","last_active":"2024-05-27T07:46:27.030345+0000","last_peered":"2024-05-27T07:46:27.030345+0000","last_clean":"2024-05-27T07:46:27.030345+0000","last_became_active":"2024-05-27T07:41:04.795519+0000","last_became_peered":"2024-05-27T07:41:04.795519+0000","last_unstale":"2024-05-27T07:46:27.030345+0000","last_undegraded":"2024-05-27T07:46:27.030345+0000","last_fullsized":"2024-05-27T07:46:27.030345+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:42.744076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350149+0000","last_change":"2024-05-27T07:41:08.832135+0000","last_active":"2024-05-27T07:46:27.350149+0000","last_peered":"2024-05-27T07:46:27.350149+0000","last_clean":"2024-05-27T07:46:27.350149+0000","last_became_active":"2024-05-27T07:41:08.830032+0000","last_became_peered":"2024-05-27T07:41:08.830032+0000","last_unstale":"2024-05-27T07:46:27.350149+0000","last_undegraded":"2024-05-27T07:46:27.350149+0000","last_fullsized":"2024-05-27T07:46:27.350149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:44:59.367026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856691+0000","last_change":"2024-05-27T07:41:22.008868+0000","last_active":"2024-05-27T07:46:26.856691+0000","last_peered":"2024-05-27T07:46:26.856691+0000","last_clean":"2024-05-27T07:46:26.856691+0000","last_became_active":"2024-05-27T07:41:22.002071+0000","last_became_peered":"2024-05-27T07:41:22.002071+0000","last_unstale":"2024-05-27T07:46:26.856691+0000","last_undegraded":"2024-05-27T07:46:26.856691+0000","last_fullsized":"2024-05-27T07:46:26.856691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:20.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'9","reported_seq":301,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.706139+0000","last_change":"2024-05-27T07:41:04.796198+0000","last_active":"2024-05-27T07:46:31.706139+0000","last_peered":"2024-05-27T07:46:31.706139+0000","last_clean":"2024-05-27T07:46:31.706139+0000","last_became_active":"2024-05-27T07:41:04.795646+0000","last_became_peered":"2024-05-27T07:41:04.795646+0000","last_unstale":"2024-05-27T07:46:31.706139+0000","last_undegraded":"2024-05-27T07:46:31.706139+0000","last_fullsized":"2024-05-27T07:46:31.706139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:30.936596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1988,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":245,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350240+0000","last_change":"2024-05-27T07:41:08.840518+0000","last_active":"2024-05-27T07:46:27.350240+0000","last_peered":"2024-05-27T07:46:27.350240+0000","last_clean":"2024-05-27T07:46:27.350240+0000","last_became_active":"2024-05-27T07:41:08.840162+0000","last_became_peered":"2024-05-27T07:41:08.840162+0000","last_unstale":"2024-05-27T07:46:27.350240+0000","last_undegraded":"2024-05-27T07:46:27.350240+0000","last_fullsized":"2024-05-27T07:46:27.350240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:04:14.657496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.518023+0000","last_change":"2024-05-27T07:41:22.000243+0000","last_active":"2024-05-27T07:46:27.518023+0000","last_peered":"2024-05-27T07:46:27.518023+0000","last_clean":"2024-05-27T07:46:27.518023+0000","last_became_active":"2024-05-27T07:41:21.998898+0000","last_became_peered":"2024-05-27T07:41:21.998898+0000","last_unstale":"2024-05-27T07:46:27.518023+0000","last_undegraded":"2024-05-27T07:46:27.518023+0000","last_fullsized":"2024-05-27T07:46:27.518023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:35.696593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":82,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856741+0000","last_change":"2024-05-27T07:41:59.799662+0000","last_active":"2024-05-27T07:46:26.856741+0000","last_peered":"2024-05-27T07:46:26.856741+0000","last_clean":"2024-05-27T07:46:26.856741+0000","last_became_active":"2024-05-27T07:41:59.779733+0000","last_became_peered":"2024-05-27T07:41:59.779733+0000","last_unstale":"2024-05-27T07:46:26.856741+0000","last_undegraded":"2024-05-27T07:46:26.856741+0000","last_fullsized":"2024-05-27T07:46:26.856741+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:23.665014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030069+0000","last_change":"2024-05-27T07:41:08.834931+0000","last_active":"2024-05-27T07:46:27.030069+0000","last_peered":"2024-05-27T07:46:27.030069+0000","last_clean":"2024-05-27T07:46:27.030069+0000","last_became_active":"2024-05-27T07:41:08.833298+0000","last_became_peered":"2024-05-27T07:41:08.833298+0000","last_unstale":"2024-05-27T07:46:27.030069+0000","last_undegraded":"2024-05-27T07:46:27.030069+0000","last_fullsized":"2024-05-27T07:46:27.030069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:03:48.507461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029888+0000","last_change":"2024-05-27T07:41:22.009407+0000","last_active":"2024-05-27T07:46:27.029888+0000","last_peered":"2024-05-27T07:46:27.029888+0000","last_clean":"2024-05-27T07:46:27.029888+0000","last_became_active":"2024-05-27T07:41:22.006921+0000","last_became_peered":"2024-05-27T07:41:22.006921+0000","last_unstale":"2024-05-27T07:46:27.029888+0000","last_undegraded":"2024-05-27T07:46:27.029888+0000","last_fullsized":"2024-05-27T07:46:27.029888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:24.034502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.860585+0000","last_change":"2024-05-27T07:41:04.795641+0000","last_active":"2024-05-27T07:46:26.860585+0000","last_peered":"2024-05-27T07:46:26.860585+0000","last_clean":"2024-05-27T07:46:26.860585+0000","last_became_active":"2024-05-27T07:41:04.794935+0000","last_became_peered":"2024-05-27T07:41:04.794935+0000","last_unstale":"2024-05-27T07:46:26.860585+0000","last_undegraded":"2024-05-27T07:46:26.860585+0000","last_fullsized":"2024-05-27T07:46:26.860585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:14.850282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091428+0000","last_change":"2024-05-27T07:41:08.837277+0000","last_active":"2024-05-27T07:46:27.091428+0000","last_peered":"2024-05-27T07:46:27.091428+0000","last_clean":"2024-05-27T07:46:27.091428+0000","last_became_active":"2024-05-27T07:41:08.835935+0000","last_became_peered":"2024-05-27T07:41:08.835935+0000","last_unstale":"2024-05-27T07:46:27.091428+0000","last_undegraded":"2024-05-27T07:46:27.091428+0000","last_fullsized":"2024-05-27T07:46:27.091428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:46.436243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856802+0000","last_change":"2024-05-27T07:41:22.014700+0000","last_active":"2024-05-27T07:46:26.856802+0000","last_peered":"2024-05-27T07:46:26.856802+0000","last_clean":"2024-05-27T07:46:26.856802+0000","last_became_active":"2024-05-27T07:41:22.014136+0000","last_became_peered":"2024-05-27T07:41:22.014136+0000","last_unstale":"2024-05-27T07:46:26.856802+0000","last_undegraded":"2024-05-27T07:46:26.856802+0000","last_fullsized":"2024-05-27T07:46:26.856802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:51:08.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.855048+0000","last_change":"2024-05-27T07:41:04.891269+0000","last_active":"2024-05-27T07:46:26.855048+0000","last_peered":"2024-05-27T07:46:26.855048+0000","last_clean":"2024-05-27T07:46:26.855048+0000","last_became_active":"2024-05-27T07:41:04.890843+0000","last_became_peered":"2024-05-27T07:41:04.890843+0000","last_unstale":"2024-05-27T07:46:26.855048+0000","last_undegraded":"2024-05-27T07:46:26.855048+0000","last_fullsized":"2024-05-27T07:46:26.855048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:57.065300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856313+0000","last_change":"2024-05-27T07:41:08.841977+0000","last_active":"2024-05-27T07:46:26.856313+0000","last_peered":"2024-05-27T07:46:26.856313+0000","last_clean":"2024-05-27T07:46:26.856313+0000","last_became_active":"2024-05-27T07:41:08.841551+0000","last_became_peered":"2024-05-27T07:41:08.841551+0000","last_unstale":"2024-05-27T07:46:26.856313+0000","last_undegraded":"2024-05-27T07:46:26.856313+0000","last_fullsized":"2024-05-27T07:46:26.856313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:35:07.766193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517708+0000","last_change":"2024-05-27T07:41:22.006310+0000","last_active":"2024-05-27T07:46:27.517708+0000","last_peered":"2024-05-27T07:46:27.517708+0000","last_clean":"2024-05-27T07:46:27.517708+0000","last_became_active":"2024-05-27T07:41:22.004135+0000","last_became_peered":"2024-05-27T07:41:22.004135+0000","last_unstale":"2024-05-27T07:46:27.517708+0000","last_undegraded":"2024-05-27T07:46:27.517708+0000","last_fullsized":"2024-05-27T07:46:27.517708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:48.564894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":296,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.593482+0000","last_change":"2024-05-27T07:41:04.793284+0000","last_active":"2024-05-27T07:46:31.593482+0000","last_peered":"2024-05-27T07:46:31.593482+0000","last_clean":"2024-05-27T07:46:31.593482+0000","last_became_active":"2024-05-27T07:41:04.792853+0000","last_became_peered":"2024-05-27T07:41:04.792853+0000","last_unstale":"2024-05-27T07:46:31.593482+0000","last_undegraded":"2024-05-27T07:46:31.593482+0000","last_fullsized":"2024-05-27T07:46:31.593482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:22.979341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1456,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":241,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856334+0000","last_change":"2024-05-27T07:41:59.776207+0000","last_active":"2024-05-27T07:46:26.856334+0000","last_peered":"2024-05-27T07:46:26.856334+0000","last_clean":"2024-05-27T07:46:26.856334+0000","last_became_active":"2024-05-27T07:41:59.775226+0000","last_became_peered":"2024-05-27T07:41:59.775226+0000","last_unstale":"2024-05-27T07:46:26.856334+0000","last_undegraded":"2024-05-27T07:46:26.856334+0000","last_fullsized":"2024-05-27T07:46:26.856334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:50.469144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517906+0000","last_change":"2024-05-27T07:41:21.992013+0000","last_active":"2024-05-27T07:46:27.517906+0000","last_peered":"2024-05-27T07:46:27.517906+0000","last_clean":"2024-05-27T07:46:27.517906+0000","last_became_active":"2024-05-27T07:41:21.991550+0000","last_became_peered":"2024-05-27T07:41:21.991550+0000","last_unstale":"2024-05-27T07:46:27.517906+0000","last_undegraded":"2024-05-27T07:46:27.517906+0000","last_fullsized":"2024-05-27T07:46:27.517906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:34:53.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.031130+0000","last_change":"2024-05-27T07:41:08.832512+0000","last_active":"2024-05-27T07:46:27.031130+0000","last_peered":"2024-05-27T07:46:27.031130+0000","last_clean":"2024-05-27T07:46:27.031130+0000","last_became_active":"2024-05-27T07:41:08.832113+0000","last_became_peered":"2024-05-27T07:41:08.832113+0000","last_unstale":"2024-05-27T07:46:27.031130+0000","last_undegraded":"2024-05-27T07:46:27.031130+0000","last_fullsized":"2024-05-27T07:46:27.031130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:00.530785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'7","reported_seq":294,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.705393+0000","last_change":"2024-05-27T07:41:04.891004+0000","last_active":"2024-05-27T07:46:31.705393+0000","last_peered":"2024-05-27T07:46:31.705393+0000","last_clean":"2024-05-27T07:46:31.705393+0000","last_became_active":"2024-05-27T07:41:04.890567+0000","last_became_peered":"2024-05-27T07:41:04.890567+0000","last_unstale":"2024-05-27T07:46:31.705393+0000","last_undegraded":"2024-05-27T07:46:31.705393+0000","last_fullsized":"2024-05-27T07:46:31.705393+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:43.991204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350297+0000","last_change":"2024-05-27T07:41:22.003446+0000","last_active":"2024-05-27T07:46:27.350297+0000","last_peered":"2024-05-27T07:46:27.350297+0000","last_clean":"2024-05-27T07:46:27.350297+0000","last_became_active":"2024-05-27T07:41:22.002989+0000","last_became_peered":"2024-05-27T07:41:22.002989+0000","last_unstale":"2024-05-27T07:46:27.350297+0000","last_undegraded":"2024-05-27T07:46:27.350297+0000","last_fullsized":"2024-05-27T07:46:27.350297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:12.996286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013957+0000","last_change":"2024-05-27T07:41:08.842685+0000","last_active":"2024-05-27T07:46:27.013957+0000","last_peered":"2024-05-27T07:46:27.013957+0000","last_clean":"2024-05-27T07:46:27.013957+0000","last_became_active":"2024-05-27T07:41:08.841954+0000","last_became_peered":"2024-05-27T07:41:08.841954+0000","last_unstale":"2024-05-27T07:46:27.013957+0000","last_undegraded":"2024-05-27T07:46:27.013957+0000","last_fullsized":"2024-05-27T07:46:27.013957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:18.087129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342500+0000","last_change":"2024-05-27T07:41:04.790330+0000","last_active":"2024-05-27T07:46:27.342500+0000","last_peered":"2024-05-27T07:46:27.342500+0000","last_clean":"2024-05-27T07:46:27.342500+0000","last_became_active":"2024-05-27T07:41:04.789508+0000","last_became_peered":"2024-05-27T07:41:04.789508+0000","last_unstale":"2024-05-27T07:46:27.342500+0000","last_undegraded":"2024-05-27T07:46:27.342500+0000","last_fullsized":"2024-05-27T07:46:27.342500+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:27.537662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'99","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:29.032290+0000","last_change":"2024-05-27T07:39:11.797324+0000","last_active":"2024-05-27T07:46:29.032290+0000","last_peered":"2024-05-27T07:46:29.032290+0000","last_clean":"2024-05-27T07:46:29.032290+0000","last_became_active":"2024-05-27T07:39:11.786294+0000","last_became_peered":"2024-05-27T07:39:11.786294+0000","last_unstale":"2024-05-27T07:46:29.032290+0000","last_undegraded":"2024-05-27T07:46:29.032290+0000","last_fullsized":"2024-05-27T07:46:29.032290+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_clean_scrub_stamp":"2024-05-27T07:34:18.613413+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:23.947615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030810+0000","last_change":"2024-05-27T07:41:21.997453+0000","last_active":"2024-05-27T07:46:27.030810+0000","last_peered":"2024-05-27T07:46:27.030810+0000","last_clean":"2024-05-27T07:46:27.030810+0000","last_became_active":"2024-05-27T07:41:21.997010+0000","last_became_peered":"2024-05-27T07:41:21.997010+0000","last_unstale":"2024-05-27T07:46:27.030810+0000","last_undegraded":"2024-05-27T07:46:27.030810+0000","last_fullsized":"2024-05-27T07:46:27.030810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:54:54.913274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517351+0000","last_change":"2024-05-27T07:43:00.024988+0000","last_active":"2024-05-27T07:46:27.517351+0000","last_peered":"2024-05-27T07:46:27.517351+0000","last_clean":"2024-05-27T07:46:27.517351+0000","last_became_active":"2024-05-27T07:43:00.023816+0000","last_became_peered":"2024-05-27T07:43:00.023816+0000","last_unstale":"2024-05-27T07:46:27.517351+0000","last_undegraded":"2024-05-27T07:46:27.517351+0000","last_fullsized":"2024-05-27T07:46:27.517351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:58.531985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.349949+0000","last_change":"2024-05-27T07:41:04.795262+0000","last_active":"2024-05-27T07:46:27.349949+0000","last_peered":"2024-05-27T07:46:27.349949+0000","last_clean":"2024-05-27T07:46:27.349949+0000","last_became_active":"2024-05-27T07:41:04.794866+0000","last_became_peered":"2024-05-27T07:41:04.794866+0000","last_unstale":"2024-05-27T07:46:27.349949+0000","last_undegraded":"2024-05-27T07:46:27.349949+0000","last_fullsized":"2024-05-27T07:46:27.349949+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:07:44.241444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090350+0000","last_change":"2024-05-27T07:41:22.006194+0000","last_active":"2024-05-27T07:46:27.090350+0000","last_peered":"2024-05-27T07:46:27.090350+0000","last_clean":"2024-05-27T07:46:27.090350+0000","last_became_active":"2024-05-27T07:41:22.004079+0000","last_became_peered":"2024-05-27T07:41:22.004079+0000","last_unstale":"2024-05-27T07:46:27.090350+0000","last_undegraded":"2024-05-27T07:46:27.090350+0000","last_fullsized":"2024-05-27T07:46:27.090350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:03.496722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091032+0000","last_change":"2024-05-27T07:41:59.776734+0000","last_active":"2024-05-27T07:46:27.091032+0000","last_peered":"2024-05-27T07:46:27.091032+0000","last_clean":"2024-05-27T07:46:27.091032+0000","last_became_active":"2024-05-27T07:41:59.775678+0000","last_became_peered":"2024-05-27T07:41:59.775678+0000","last_unstale":"2024-05-27T07:46:27.091032+0000","last_undegraded":"2024-05-27T07:46:27.091032+0000","last_fullsized":"2024-05-27T07:46:27.091032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:56.007178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517918+0000","last_change":"2024-05-27T07:41:08.831070+0000","last_active":"2024-05-27T07:46:27.517918+0000","last_peered":"2024-05-27T07:46:27.517918+0000","last_clean":"2024-05-27T07:46:27.517918+0000","last_became_active":"2024-05-27T07:41:08.830617+0000","last_became_peered":"2024-05-27T07:41:08.830617+0000","last_unstale":"2024-05-27T07:46:27.517918+0000","last_undegraded":"2024-05-27T07:46:27.517918+0000","last_fullsized":"2024-05-27T07:46:27.517918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:13.046162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013806+0000","last_change":"2024-05-27T07:41:22.010129+0000","last_active":"2024-05-27T07:46:27.013806+0000","last_peered":"2024-05-27T07:46:27.013806+0000","last_clean":"2024-05-27T07:46:27.013806+0000","last_became_active":"2024-05-27T07:41:22.009608+0000","last_became_peered":"2024-05-27T07:41:22.009608+0000","last_unstale":"2024-05-27T07:46:27.013806+0000","last_undegraded":"2024-05-27T07:46:27.013806+0000","last_fullsized":"2024-05-27T07:46:27.013806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.225004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'54","reported_seq":125,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.926903+0000","last_change":"2024-05-27T07:41:04.794744+0000","last_active":"2024-05-27T07:46:27.926903+0000","last_peered":"2024-05-27T07:46:27.926903+0000","last_clean":"2024-05-27T07:46:27.926903+0000","last_became_active":"2024-05-27T07:41:04.794340+0000","last_became_peered":"2024-05-27T07:41:04.794340+0000","last_unstale":"2024-05-27T07:46:27.926903+0000","last_undegraded":"2024-05-27T07:46:27.926903+0000","last_fullsized":"2024-05-27T07:46:27.926903+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:43:20.335520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":96,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014574+0000","last_change":"2024-05-27T07:41:08.841834+0000","last_active":"2024-05-27T07:46:27.014574+0000","last_peered":"2024-05-27T07:46:27.014574+0000","last_clean":"2024-05-27T07:46:27.014574+0000","last_became_active":"2024-05-27T07:41:08.840261+0000","last_became_peered":"2024-05-27T07:41:08.840261+0000","last_unstale":"2024-05-27T07:46:27.014574+0000","last_undegraded":"2024-05-27T07:46:27.014574+0000","last_fullsized":"2024-05-27T07:46:27.014574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:33.738835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338031+0000","last_change":"2024-05-27T07:41:22.005861+0000","last_active":"2024-05-27T07:46:27.338031+0000","last_peered":"2024-05-27T07:46:27.338031+0000","last_clean":"2024-05-27T07:46:27.338031+0000","last_became_active":"2024-05-27T07:41:22.005295+0000","last_became_peered":"2024-05-27T07:41:22.005295+0000","last_unstale":"2024-05-27T07:46:27.338031+0000","last_undegraded":"2024-05-27T07:46:27.338031+0000","last_fullsized":"2024-05-27T07:46:27.338031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:46.582656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343151+0000","last_change":"2024-05-27T07:41:04.796562+0000","last_active":"2024-05-27T07:46:27.343151+0000","last_peered":"2024-05-27T07:46:27.343151+0000","last_clean":"2024-05-27T07:46:27.343151+0000","last_became_active":"2024-05-27T07:41:04.795538+0000","last_became_peered":"2024-05-27T07:41:04.795538+0000","last_unstale":"2024-05-27T07:46:27.343151+0000","last_undegraded":"2024-05-27T07:46:27.343151+0000","last_fullsized":"2024-05-27T07:46:27.343151+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:12.131999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342791+0000","last_change":"2024-05-27T07:41:08.832824+0000","last_active":"2024-05-27T07:46:27.342791+0000","last_peered":"2024-05-27T07:46:27.342791+0000","last_clean":"2024-05-27T07:46:27.342791+0000","last_became_active":"2024-05-27T07:41:08.832332+0000","last_became_peered":"2024-05-27T07:41:08.832332+0000","last_unstale":"2024-05-27T07:46:27.342791+0000","last_undegraded":"2024-05-27T07:46:27.342791+0000","last_fullsized":"2024-05-27T07:46:27.342791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:34.598773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342420+0000","last_change":"2024-05-27T07:41:22.003358+0000","last_active":"2024-05-27T07:46:27.342420+0000","last_peered":"2024-05-27T07:46:27.342420+0000","last_clean":"2024-05-27T07:46:27.342420+0000","last_became_active":"2024-05-27T07:41:22.002655+0000","last_became_peered":"2024-05-27T07:41:22.002655+0000","last_unstale":"2024-05-27T07:46:27.342420+0000","last_undegraded":"2024-05-27T07:46:27.342420+0000","last_fullsized":"2024-05-27T07:46:27.342420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:41.177138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339729+0000","last_change":"2024-05-27T07:41:04.888681+0000","last_active":"2024-05-27T07:46:27.339729+0000","last_peered":"2024-05-27T07:46:27.339729+0000","last_clean":"2024-05-27T07:46:27.339729+0000","last_became_active":"2024-05-27T07:41:04.887966+0000","last_became_peered":"2024-05-27T07:41:04.887966+0000","last_unstale":"2024-05-27T07:46:27.339729+0000","last_undegraded":"2024-05-27T07:46:27.339729+0000","last_fullsized":"2024-05-27T07:46:27.339729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:01.178737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856720+0000","last_change":"2024-05-27T07:41:08.827839+0000","last_active":"2024-05-27T07:46:26.856720+0000","last_peered":"2024-05-27T07:46:26.856720+0000","last_clean":"2024-05-27T07:46:26.856720+0000","last_became_active":"2024-05-27T07:41:08.827403+0000","last_became_peered":"2024-05-27T07:41:08.827403+0000","last_unstale":"2024-05-27T07:46:26.856720+0000","last_undegraded":"2024-05-27T07:46:26.856720+0000","last_fullsized":"2024-05-27T07:46:26.856720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:01:42.075980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856869+0000","last_change":"2024-05-27T07:41:22.006161+0000","last_active":"2024-05-27T07:46:26.856869+0000","last_peered":"2024-05-27T07:46:26.856869+0000","last_clean":"2024-05-27T07:46:26.856869+0000","last_became_active":"2024-05-27T07:41:22.005762+0000","last_became_peered":"2024-05-27T07:41:22.005762+0000","last_unstale":"2024-05-27T07:46:26.856869+0000","last_undegraded":"2024-05-27T07:46:26.856869+0000","last_fullsized":"2024-05-27T07:46:26.856869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:53.536444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014004+0000","last_change":"2024-05-27T07:41:04.774211+0000","last_active":"2024-05-27T07:46:27.014004+0000","last_peered":"2024-05-27T07:46:27.014004+0000","last_clean":"2024-05-27T07:46:27.014004+0000","last_became_active":"2024-05-27T07:41:04.773215+0000","last_became_peered":"2024-05-27T07:41:04.773215+0000","last_unstale":"2024-05-27T07:46:27.014004+0000","last_undegraded":"2024-05-27T07:46:27.014004+0000","last_fullsized":"2024-05-27T07:46:27.014004+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:37.273735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090283+0000","last_change":"2024-05-27T07:41:08.830434+0000","last_active":"2024-05-27T07:46:27.090283+0000","last_peered":"2024-05-27T07:46:27.090283+0000","last_clean":"2024-05-27T07:46:27.090283+0000","last_became_active":"2024-05-27T07:41:08.830039+0000","last_became_peered":"2024-05-27T07:41:08.830039+0000","last_unstale":"2024-05-27T07:46:27.090283+0000","last_undegraded":"2024-05-27T07:46:27.090283+0000","last_fullsized":"2024-05-27T07:46:27.090283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:27.877666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.345102+0000","last_change":"2024-05-27T07:41:59.783413+0000","last_active":"2024-05-27T07:46:27.345102+0000","last_peered":"2024-05-27T07:46:27.345102+0000","last_clean":"2024-05-27T07:46:27.345102+0000","last_became_active":"2024-05-27T07:41:59.782300+0000","last_became_peered":"2024-05-27T07:41:59.782300+0000","last_unstale":"2024-05-27T07:46:27.345102+0000","last_undegraded":"2024-05-27T07:46:27.345102+0000","last_fullsized":"2024-05-27T07:46:27.345102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:59.348928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338570+0000","last_change":"2024-05-27T07:41:04.798969+0000","last_active":"2024-05-27T07:46:27.338570+0000","last_peered":"2024-05-27T07:46:27.338570+0000","last_clean":"2024-05-27T07:46:27.338570+0000","last_became_active":"2024-05-27T07:41:04.795996+0000","last_became_peered":"2024-05-27T07:41:04.795996+0000","last_unstale":"2024-05-27T07:46:27.338570+0000","last_undegraded":"2024-05-27T07:46:27.338570+0000","last_fullsized":"2024-05-27T07:46:27.338570+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:47.141063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014169+0000","last_change":"2024-05-27T07:41:08.835142+0000","last_active":"2024-05-27T07:46:27.014169+0000","last_peered":"2024-05-27T07:46:27.014169+0000","last_clean":"2024-05-27T07:46:27.014169+0000","last_became_active":"2024-05-27T07:41:08.834762+0000","last_became_peered":"2024-05-27T07:41:08.834762+0000","last_unstale":"2024-05-27T07:46:27.014169+0000","last_undegraded":"2024-05-27T07:46:27.014169+0000","last_fullsized":"2024-05-27T07:46:27.014169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:52.957554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342357+0000","last_change":"2024-05-27T07:41:22.000649+0000","last_active":"2024-05-27T07:46:27.342357+0000","last_peered":"2024-05-27T07:46:27.342357+0000","last_clean":"2024-05-27T07:46:27.342357+0000","last_became_active":"2024-05-27T07:41:22.000161+0000","last_became_peered":"2024-05-27T07:41:22.000161+0000","last_unstale":"2024-05-27T07:46:27.342357+0000","last_undegraded":"2024-05-27T07:46:27.342357+0000","last_fullsized":"2024-05-27T07:46:27.342357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:10.788027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.858452+0000","last_change":"2024-05-27T07:41:04.800680+0000","last_active":"2024-05-27T07:46:26.858452+0000","last_peered":"2024-05-27T07:46:26.858452+0000","last_clean":"2024-05-27T07:46:26.858452+0000","last_became_active":"2024-05-27T07:41:04.800249+0000","last_became_peered":"2024-05-27T07:41:04.800249+0000","last_unstale":"2024-05-27T07:46:26.858452+0000","last_undegraded":"2024-05-27T07:46:26.858452+0000","last_fullsized":"2024-05-27T07:46:26.858452+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:23:27.797216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350163+0000","last_change":"2024-05-27T07:41:08.844380+0000","last_active":"2024-05-27T07:46:27.350163+0000","last_peered":"2024-05-27T07:46:27.350163+0000","last_clean":"2024-05-27T07:46:27.350163+0000","last_became_active":"2024-05-27T07:41:08.843464+0000","last_became_peered":"2024-05-27T07:41:08.843464+0000","last_unstale":"2024-05-27T07:46:27.350163+0000","last_undegraded":"2024-05-27T07:46:27.350163+0000","last_fullsized":"2024-05-27T07:46:27.350163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:23.290234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341019+0000","last_change":"2024-05-27T07:41:22.010968+0000","last_active":"2024-05-27T07:46:27.341019+0000","last_peered":"2024-05-27T07:46:27.341019+0000","last_clean":"2024-05-27T07:46:27.341019+0000","last_became_active":"2024-05-27T07:41:22.010529+0000","last_became_peered":"2024-05-27T07:41:22.010529+0000","last_unstale":"2024-05-27T07:46:27.341019+0000","last_undegraded":"2024-05-27T07:46:27.341019+0000","last_fullsized":"2024-05-27T07:46:27.341019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:08.636434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":299,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.650113+0000","last_change":"2024-05-27T07:41:04.891536+0000","last_active":"2024-05-27T07:46:31.650113+0000","last_peered":"2024-05-27T07:46:31.650113+0000","last_clean":"2024-05-27T07:46:31.650113+0000","last_became_active":"2024-05-27T07:41:04.890724+0000","last_became_peered":"2024-05-27T07:41:04.890724+0000","last_unstale":"2024-05-27T07:46:31.650113+0000","last_undegraded":"2024-05-27T07:46:31.650113+0000","last_fullsized":"2024-05-27T07:46:31.650113+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:32.027624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2429,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350778+0000","last_change":"2024-05-27T07:41:08.834362+0000","last_active":"2024-05-27T07:46:27.350778+0000","last_peered":"2024-05-27T07:46:27.350778+0000","last_clean":"2024-05-27T07:46:27.350778+0000","last_became_active":"2024-05-27T07:41:08.833979+0000","last_became_peered":"2024-05-27T07:41:08.833979+0000","last_unstale":"2024-05-27T07:46:27.350778+0000","last_undegraded":"2024-05-27T07:46:27.350778+0000","last_fullsized":"2024-05-27T07:46:27.350778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:35.098352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341771+0000","last_change":"2024-05-27T07:41:21.997587+0000","last_active":"2024-05-27T07:46:27.341771+0000","last_peered":"2024-05-27T07:46:27.341771+0000","last_clean":"2024-05-27T07:46:27.341771+0000","last_became_active":"2024-05-27T07:41:21.996413+0000","last_became_peered":"2024-05-27T07:41:21.996413+0000","last_unstale":"2024-05-27T07:46:27.341771+0000","last_undegraded":"2024-05-27T07:46:27.341771+0000","last_fullsized":"2024-05-27T07:46:27.341771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:21:46.728047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"92'41","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.228573+0000","last_change":"2024-05-27T07:41:04.793580+0000","last_active":"2024-05-27T07:46:27.228573+0000","last_peered":"2024-05-27T07:46:27.228573+0000","last_clean":"2024-05-27T07:46:27.228573+0000","last_became_active":"2024-05-27T07:41:04.791135+0000","last_became_peered":"2024-05-27T07:41:04.791135+0000","last_unstale":"2024-05-27T07:46:27.228573+0000","last_undegraded":"2024-05-27T07:46:27.228573+0000","last_fullsized":"2024-05-27T07:46:27.228573+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:15.484187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350999+0000","last_change":"2024-05-27T07:41:08.839187+0000","last_active":"2024-05-27T07:46:27.350999+0000","last_peered":"2024-05-27T07:46:27.350999+0000","last_clean":"2024-05-27T07:46:27.350999+0000","last_became_active":"2024-05-27T07:41:08.838821+0000","last_became_peered":"2024-05-27T07:41:08.838821+0000","last_unstale":"2024-05-27T07:46:27.350999+0000","last_undegraded":"2024-05-27T07:46:27.350999+0000","last_fullsized":"2024-05-27T07:46:27.350999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:30.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029214+0000","last_change":"2024-05-27T07:41:22.005406+0000","last_active":"2024-05-27T07:46:27.029214+0000","last_peered":"2024-05-27T07:46:27.029214+0000","last_clean":"2024-05-27T07:46:27.029214+0000","last_became_active":"2024-05-27T07:41:22.003415+0000","last_became_peered":"2024-05-27T07:41:22.003415+0000","last_unstale":"2024-05-27T07:46:27.029214+0000","last_undegraded":"2024-05-27T07:46:27.029214+0000","last_fullsized":"2024-05-27T07:46:27.029214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:03.300452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517988+0000","last_change":"2024-05-27T07:41:04.798489+0000","last_active":"2024-05-27T07:46:27.517988+0000","last_peered":"2024-05-27T07:46:27.517988+0000","last_clean":"2024-05-27T07:46:27.517988+0000","last_became_active":"2024-05-27T07:41:04.797467+0000","last_became_peered":"2024-05-27T07:41:04.797467+0000","last_unstale":"2024-05-27T07:46:27.517988+0000","last_undegraded":"2024-05-27T07:46:27.517988+0000","last_fullsized":"2024-05-27T07:46:27.517988+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:04.018577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.351043+0000","last_change":"2024-05-27T07:41:08.832215+0000","last_active":"2024-05-27T07:46:27.351043+0000","last_peered":"2024-05-27T07:46:27.351043+0000","last_clean":"2024-05-27T07:46:27.351043+0000","last_became_active":"2024-05-27T07:41:08.831491+0000","last_became_peered":"2024-05-27T07:41:08.831491+0000","last_unstale":"2024-05-27T07:46:27.351043+0000","last_undegraded":"2024-05-27T07:46:27.351043+0000","last_fullsized":"2024-05-27T07:46:27.351043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:32:03.084324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090501+0000","last_change":"2024-05-27T07:41:08.837301+0000","last_active":"2024-05-27T07:46:27.090501+0000","last_peered":"2024-05-27T07:46:27.090501+0000","last_clean":"2024-05-27T07:46:27.090501+0000","last_became_active":"2024-05-27T07:41:08.835533+0000","last_became_peered":"2024-05-27T07:41:08.835533+0000","last_unstale":"2024-05-27T07:46:27.090501+0000","last_undegraded":"2024-05-27T07:46:27.090501+0000","last_fullsized":"2024-05-27T07:46:27.090501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:45.807613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338530+0000","last_change":"2024-05-27T07:41:08.846740+0000","last_active":"2024-05-27T07:46:27.338530+0000","last_peered":"2024-05-27T07:46:27.338530+0000","last_clean":"2024-05-27T07:46:27.338530+0000","last_became_active":"2024-05-27T07:41:08.845576+0000","last_became_peered":"2024-05-27T07:41:08.845576+0000","last_unstale":"2024-05-27T07:46:27.338530+0000","last_undegraded":"2024-05-27T07:46:27.338530+0000","last_fullsized":"2024-05-27T07:46:27.338530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:41:36.944923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338096+0000","last_change":"2024-05-27T07:41:08.840121+0000","last_active":"2024-05-27T07:46:27.338096+0000","last_peered":"2024-05-27T07:46:27.338096+0000","last_clean":"2024-05-27T07:46:27.338096+0000","last_became_active":"2024-05-27T07:41:08.839090+0000","last_became_peered":"2024-05-27T07:41:08.839090+0000","last_unstale":"2024-05-27T07:46:27.338096+0000","last_undegraded":"2024-05-27T07:46:27.338096+0000","last_fullsized":"2024-05-27T07:46:27.338096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:44.836953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342097+0000","last_change":"2024-05-27T07:41:08.835565+0000","last_active":"2024-05-27T07:46:27.342097+0000","last_peered":"2024-05-27T07:46:27.342097+0000","last_clean":"2024-05-27T07:46:27.342097+0000","last_became_active":"2024-05-27T07:41:08.831424+0000","last_became_peered":"2024-05-27T07:41:08.831424+0000","last_unstale":"2024-05-27T07:46:27.342097+0000","last_undegraded":"2024-05-27T07:46:27.342097+0000","last_fullsized":"2024-05-27T07:46:27.342097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:17.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517940+0000","last_change":"2024-05-27T07:41:08.830104+0000","last_active":"2024-05-27T07:46:27.517940+0000","last_peered":"2024-05-27T07:46:27.517940+0000","last_clean":"2024-05-27T07:46:27.517940+0000","last_became_active":"2024-05-27T07:41:08.829690+0000","last_became_peered":"2024-05-27T07:41:08.829690+0000","last_unstale":"2024-05-27T07:46:27.517940+0000","last_undegraded":"2024-05-27T07:46:27.517940+0000","last_fullsized":"2024-05-27T07:46:27.517940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091546+0000","last_change":"2024-05-27T07:41:08.838654+0000","last_active":"2024-05-27T07:46:27.091546+0000","last_peered":"2024-05-27T07:46:27.091546+0000","last_clean":"2024-05-27T07:46:27.091546+0000","last_became_active":"2024-05-27T07:41:08.835033+0000","last_became_peered":"2024-05-27T07:41:08.835033+0000","last_unstale":"2024-05-27T07:46:27.091546+0000","last_undegraded":"2024-05-27T07:46:27.091546+0000","last_fullsized":"2024-05-27T07:46:27.091546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:04:24.007585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339099+0000","last_change":"2024-05-27T07:41:08.844557+0000","last_active":"2024-05-27T07:46:27.339099+0000","last_peered":"2024-05-27T07:46:27.339099+0000","last_clean":"2024-05-27T07:46:27.339099+0000","last_became_active":"2024-05-27T07:41:08.843418+0000","last_became_peered":"2024-05-27T07:41:08.843418+0000","last_unstale":"2024-05-27T07:46:27.339099+0000","last_undegraded":"2024-05-27T07:46:27.339099+0000","last_fullsized":"2024-05-27T07:46:27.339099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:39.587778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014764+0000","last_change":"2024-05-27T07:41:08.830857+0000","last_active":"2024-05-27T07:46:27.014764+0000","last_peered":"2024-05-27T07:46:27.014764+0000","last_clean":"2024-05-27T07:46:27.014764+0000","last_became_active":"2024-05-27T07:41:08.830480+0000","last_became_peered":"2024-05-27T07:41:08.830480+0000","last_unstale":"2024-05-27T07:46:27.014764+0000","last_undegraded":"2024-05-27T07:46:27.014764+0000","last_fullsized":"2024-05-27T07:46:27.014764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:07.307528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029828+0000","last_change":"2024-05-27T07:41:08.824571+0000","last_active":"2024-05-27T07:46:27.029828+0000","last_peered":"2024-05-27T07:46:27.029828+0000","last_clean":"2024-05-27T07:46:27.029828+0000","last_became_active":"2024-05-27T07:41:08.820732+0000","last_became_peered":"2024-05-27T07:41:08.820732+0000","last_unstale":"2024-05-27T07:46:27.029828+0000","last_undegraded":"2024-05-27T07:46:27.029828+0000","last_fullsized":"2024-05-27T07:46:27.029828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:55:14.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014133+0000","last_change":"2024-05-27T07:41:08.842284+0000","last_active":"2024-05-27T07:46:27.014133+0000","last_peered":"2024-05-27T07:46:27.014133+0000","last_clean":"2024-05-27T07:46:27.014133+0000","last_became_active":"2024-05-27T07:41:08.841597+0000","last_became_peered":"2024-05-27T07:41:08.841597+0000","last_unstale":"2024-05-27T07:46:27.014133+0000","last_undegraded":"2024-05-27T07:46:27.014133+0000","last_fullsized":"2024-05-27T07:46:27.014133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:25:31.066181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342151+0000","last_change":"2024-05-27T07:41:08.828935+0000","last_active":"2024-05-27T07:46:27.342151+0000","last_peered":"2024-05-27T07:46:27.342151+0000","last_clean":"2024-05-27T07:46:27.342151+0000","last_became_active":"2024-05-27T07:41:08.828542+0000","last_became_peered":"2024-05-27T07:41:08.828542+0000","last_unstale":"2024-05-27T07:46:27.342151+0000","last_undegraded":"2024-05-27T07:46:27.342151+0000","last_fullsized":"2024-05-27T07:46:27.342151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:35.270666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517888+0000","last_change":"2024-05-27T07:41:08.817051+0000","last_active":"2024-05-27T07:46:27.517888+0000","last_peered":"2024-05-27T07:46:27.517888+0000","last_clean":"2024-05-27T07:46:27.517888+0000","last_became_active":"2024-05-27T07:41:08.816418+0000","last_became_peered":"2024-05-27T07:41:08.816418+0000","last_unstale":"2024-05-27T07:46:27.517888+0000","last_undegraded":"2024-05-27T07:46:27.517888+0000","last_fullsized":"2024-05-27T07:46:27.517888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:02.760707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090720+0000","last_change":"2024-05-27T07:41:08.834081+0000","last_active":"2024-05-27T07:46:27.090720+0000","last_peered":"2024-05-27T07:46:27.090720+0000","last_clean":"2024-05-27T07:46:27.090720+0000","last_became_active":"2024-05-27T07:41:08.833679+0000","last_became_peered":"2024-05-27T07:41:08.833679+0000","last_unstale":"2024-05-27T07:46:27.090720+0000","last_undegraded":"2024-05-27T07:46:27.090720+0000","last_fullsized":"2024-05-27T07:46:27.090720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:41:14.426593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":612323,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2532,"num_read_kb":1395,"num_write":404,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2199552,"data_stored":1836969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":325,"ondisk_log_size":325,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":533518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841626,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28884,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716268,"statfs":{"total":95995035648,"available":95965458432,"internally_reserved":0,"allocated":1912832,"data_stored":1331826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005153,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1482752,"data_stored":925293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201384,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":925559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397616,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1683456,"data_stored":1132620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561142,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1064960,"data_stored":528449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1146880,"data_stored":605246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738532,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":530296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":531769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117134,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32744,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712408,"statfs":{"total":95995035648,"available":95961505792,"internally_reserved":0,"allocated":1605632,"data_stored":1052105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920903,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":608934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757375,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28800,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716352,"statfs":{"total":95995035648,"available":95965544448,"internally_reserved":0,"allocated":1826816,"data_stored":1257534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":78387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":77979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":136211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":210503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:46:35.133 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T07:46:35.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.445+0000 7f028a000640 1 -- 172.21.15.90:0/2034124458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02841063e0 msgr2=0x7f02841127b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.445+0000 7f028a000640 1 --2- 172.21.15.90:0/2034124458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02841063e0 0x7f02841127b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f027c009ca0 tx=0x7f027c02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.445+0000 7f028a000640 1 -- 172.21.15.90:0/2034124458 shutdown_connections 2024-05-27T07:46:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.445+0000 7f028a000640 1 --2- 172.21.15.90:0/2034124458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028410c130 0x7f0284105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.445+0000 7f028a000640 1 --2- 172.21.15.90:0/2034124458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02841063e0 0x7f02841127b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.445+0000 7f028a000640 1 --2- 172.21.15.90:0/2034124458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f028410bc90 0x7f0284105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.445+0000 7f028a000640 1 -- 172.21.15.90:0/2034124458 >> 172.21.15.90:0/2034124458 conn(0x7f02841013d0 msgr2=0x7f02841037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.445+0000 7f028a000640 1 -- 172.21.15.90:0/2034124458 shutdown_connections 2024-05-27T07:46:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.445+0000 7f028a000640 1 -- 172.21.15.90:0/2034124458 wait complete. 2024-05-27T07:46:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.447+0000 7f028a000640 1 Processor -- start 2024-05-27T07:46:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.447+0000 7f028a000640 1 -- start start 2024-05-27T07:46:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.447+0000 7f028a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02841063e0 0x7f028419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f028a000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028410bc90 0x7f028419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f028a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f028410c130 0x7f02841a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f028a000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0284115da0 con 0x7f028410c130 2024-05-27T07:46:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f028a000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0284115c20 con 0x7f028410bc90 2024-05-27T07:46:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f028a000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0284115f20 con 0x7f02841063e0 2024-05-27T07:46:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f0289600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f028410c130 0x7f02841a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f0288c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02841063e0 0x7f028419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f0289600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f028410c130 0x7f02841a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49750/0 (socket says 172.21.15.90:49750) 2024-05-27T07:46:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f0289600640 1 -- 172.21.15.90:0/4114222436 learned_addr learned my addr 172.21.15.90:0/4114222436 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f027be00640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028410bc90 0x7f028419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.448+0000 7f0289600640 1 -- 172.21.15.90:0/4114222436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02841063e0 msgr2=0x7f028419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f0289600640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02841063e0 0x7f028419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f0289600640 1 -- 172.21.15.90:0/4114222436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028410bc90 msgr2=0x7f028419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f0289600640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028410bc90 0x7f028419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f0289600640 1 -- 172.21.15.90:0/4114222436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02841a1620 con 0x7f028410c130 2024-05-27T07:46:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f0288c00640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02841063e0 0x7f028419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:46:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f0289600640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f028410c130 0x7f02841a0fe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f027c005cc0 tx=0x7f027c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f0282000640 1 -- 172.21.15.90:0/4114222436 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c032c10 con 0x7f028410c130 2024-05-27T07:46:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f0282000640 1 -- 172.21.15.90:0/4114222436 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f027c032db0 con 0x7f028410c130 2024-05-27T07:46:35.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f028a000640 1 -- 172.21.15.90:0/4114222436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02841a18b0 con 0x7f028410c130 2024-05-27T07:46:35.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f0282000640 1 -- 172.21.15.90:0/4114222436 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c030a60 con 0x7f028410c130 2024-05-27T07:46:35.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.449+0000 7f028a000640 1 -- 172.21.15.90:0/4114222436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02841a1e00 con 0x7f028410c130 2024-05-27T07:46:35.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.451+0000 7f028a000640 1 -- 172.21.15.90:0/4114222436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028406a150 con 0x7f028410c130 2024-05-27T07:46:35.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.453+0000 7f0282000640 1 -- 172.21.15.90:0/4114222436 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f027c030c00 con 0x7f028410c130 2024-05-27T07:46:35.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.455+0000 7f0282000640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f02680b2a70 0x7f02680b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:35.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.455+0000 7f0282000640 1 -- 172.21.15.90:0/4114222436 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f027c103ff0 con 0x7f028410c130 2024-05-27T07:46:35.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.455+0000 7f0288c00640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f02680b2a70 0x7f02680b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:35.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.456+0000 7f0288c00640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f02680b2a70 0x7f02680b4f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0284106970 tx=0x7f0274005f90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:35.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.458+0000 7f0282000640 1 -- 172.21.15.90:0/4114222436 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f027c0cb570 con 0x7f028410c130 2024-05-27T07:46:35.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.668+0000 7f028a000640 1 -- 172.21.15.90:0/4114222436 --> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0284063100 con 0x7f02680b2a70 2024-05-27T07:46:35.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.699+0000 7f0282000640 1 -- 172.21.15.90:0/4114222436 <== mgr.34967 v2:172.21.15.90:6800/4256436582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339499 (secure 0 0 0) 0x7f0284063100 con 0x7f02680b2a70 2024-05-27T07:46:35.704 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:46:35.706 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 -- 172.21.15.90:0/4114222436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f02680b2a70 msgr2=0x7f02680b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f02680b2a70 0x7f02680b4f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0284106970 tx=0x7f0274005f90 comp rx=0 tx=0).stop 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 -- 172.21.15.90:0/4114222436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f028410c130 msgr2=0x7f02841a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f028410c130 0x7f02841a0fe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f027c005cc0 tx=0x7f027c003f90 comp rx=0 tx=0).stop 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 -- 172.21.15.90:0/4114222436 shutdown_connections 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f02680b2a70 0x7f02680b4f80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f028410bc90 0x7f028419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f028410c130 0x7f02841a0fe0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 --2- 172.21.15.90:0/4114222436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02841063e0 0x7f028419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 -- 172.21.15.90:0/4114222436 >> 172.21.15.90:0/4114222436 conn(0x7f02841013d0 msgr2=0x7f0284108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 -- 172.21.15.90:0/4114222436 shutdown_connections 2024-05-27T07:46:35.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:35.707+0000 7f028a000640 1 -- 172.21.15.90:0/4114222436 wait complete. 2024-05-27T07:46:35.736 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T07:46:34.889987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1202925,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2676,"num_read_kb":1643,"num_write":541,"num_write_kb":3647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":926,"ondisk_log_size":926,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361224,"kb_used_data":16208,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124580600,"statfs":{"total":1151940427776,"available":1151570534400,"internally_reserved":0,"allocated":16596992,"data_stored":9963149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19138,"internal_metadata":353154366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4084,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":211,"num_write":25,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.972610"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091553+0000","last_change":"2024-05-27T07:41:08.839265+0000","last_active":"2024-05-27T07:46:27.091553+0000","last_peered":"2024-05-27T07:46:27.091553+0000","last_clean":"2024-05-27T07:46:27.091553+0000","last_became_active":"2024-05-27T07:41:08.838814+0000","last_became_peered":"2024-05-27T07:41:08.838814+0000","last_unstale":"2024-05-27T07:46:27.091553+0000","last_undegraded":"2024-05-27T07:46:27.091553+0000","last_fullsized":"2024-05-27T07:46:27.091553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:32.901520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342694+0000","last_change":"2024-05-27T07:41:08.838313+0000","last_active":"2024-05-27T07:46:27.342694+0000","last_peered":"2024-05-27T07:46:27.342694+0000","last_clean":"2024-05-27T07:46:27.342694+0000","last_became_active":"2024-05-27T07:41:08.836657+0000","last_became_peered":"2024-05-27T07:41:08.836657+0000","last_unstale":"2024-05-27T07:46:27.342694+0000","last_undegraded":"2024-05-27T07:46:27.342694+0000","last_fullsized":"2024-05-27T07:46:27.342694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:33.556505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342426+0000","last_change":"2024-05-27T07:41:08.841513+0000","last_active":"2024-05-27T07:46:27.342426+0000","last_peered":"2024-05-27T07:46:27.342426+0000","last_clean":"2024-05-27T07:46:27.342426+0000","last_became_active":"2024-05-27T07:41:08.838624+0000","last_became_peered":"2024-05-27T07:41:08.838624+0000","last_unstale":"2024-05-27T07:46:27.342426+0000","last_undegraded":"2024-05-27T07:46:27.342426+0000","last_fullsized":"2024-05-27T07:46:27.342426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:01:37.320418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343623+0000","last_change":"2024-05-27T07:41:08.839138+0000","last_active":"2024-05-27T07:46:27.343623+0000","last_peered":"2024-05-27T07:46:27.343623+0000","last_clean":"2024-05-27T07:46:27.343623+0000","last_became_active":"2024-05-27T07:41:08.838766+0000","last_became_peered":"2024-05-27T07:41:08.838766+0000","last_unstale":"2024-05-27T07:46:27.343623+0000","last_undegraded":"2024-05-27T07:46:27.343623+0000","last_fullsized":"2024-05-27T07:46:27.343623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:09.129257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.858624+0000","last_change":"2024-05-27T07:41:08.823006+0000","last_active":"2024-05-27T07:46:26.858624+0000","last_peered":"2024-05-27T07:46:26.858624+0000","last_clean":"2024-05-27T07:46:26.858624+0000","last_became_active":"2024-05-27T07:41:08.822551+0000","last_became_peered":"2024-05-27T07:41:08.822551+0000","last_unstale":"2024-05-27T07:46:26.858624+0000","last_undegraded":"2024-05-27T07:46:26.858624+0000","last_fullsized":"2024-05-27T07:46:26.858624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:49.255763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856113+0000","last_change":"2024-05-27T07:41:08.846072+0000","last_active":"2024-05-27T07:46:26.856113+0000","last_peered":"2024-05-27T07:46:26.856113+0000","last_clean":"2024-05-27T07:46:26.856113+0000","last_became_active":"2024-05-27T07:41:08.845604+0000","last_became_peered":"2024-05-27T07:41:08.845604+0000","last_unstale":"2024-05-27T07:46:26.856113+0000","last_undegraded":"2024-05-27T07:46:26.856113+0000","last_fullsized":"2024-05-27T07:46:26.856113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:23:56.389135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350459+0000","last_change":"2024-05-27T07:41:08.841046+0000","last_active":"2024-05-27T07:46:27.350459+0000","last_peered":"2024-05-27T07:46:27.350459+0000","last_clean":"2024-05-27T07:46:27.350459+0000","last_became_active":"2024-05-27T07:41:08.840028+0000","last_became_peered":"2024-05-27T07:41:08.840028+0000","last_unstale":"2024-05-27T07:46:27.350459+0000","last_undegraded":"2024-05-27T07:46:27.350459+0000","last_fullsized":"2024-05-27T07:46:27.350459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:35.239919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013947+0000","last_change":"2024-05-27T07:41:08.841182+0000","last_active":"2024-05-27T07:46:27.013947+0000","last_peered":"2024-05-27T07:46:27.013947+0000","last_clean":"2024-05-27T07:46:27.013947+0000","last_became_active":"2024-05-27T07:41:08.840645+0000","last_became_peered":"2024-05-27T07:41:08.840645+0000","last_unstale":"2024-05-27T07:46:27.013947+0000","last_undegraded":"2024-05-27T07:46:27.013947+0000","last_fullsized":"2024-05-27T07:46:27.013947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:10:59.744505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342789+0000","last_change":"2024-05-27T07:41:08.825874+0000","last_active":"2024-05-27T07:46:27.342789+0000","last_peered":"2024-05-27T07:46:27.342789+0000","last_clean":"2024-05-27T07:46:27.342789+0000","last_became_active":"2024-05-27T07:41:08.825386+0000","last_became_peered":"2024-05-27T07:41:08.825386+0000","last_unstale":"2024-05-27T07:46:27.342789+0000","last_undegraded":"2024-05-27T07:46:27.342789+0000","last_fullsized":"2024-05-27T07:46:27.342789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:13.304612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.857247+0000","last_change":"2024-05-27T07:41:08.847445+0000","last_active":"2024-05-27T07:46:26.857247+0000","last_peered":"2024-05-27T07:46:26.857247+0000","last_clean":"2024-05-27T07:46:26.857247+0000","last_became_active":"2024-05-27T07:41:08.846512+0000","last_became_peered":"2024-05-27T07:41:08.846512+0000","last_unstale":"2024-05-27T07:46:26.857247+0000","last_undegraded":"2024-05-27T07:46:26.857247+0000","last_fullsized":"2024-05-27T07:46:26.857247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:57.235218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350533+0000","last_change":"2024-05-27T07:41:08.843531+0000","last_active":"2024-05-27T07:46:27.350533+0000","last_peered":"2024-05-27T07:46:27.350533+0000","last_clean":"2024-05-27T07:46:27.350533+0000","last_became_active":"2024-05-27T07:41:08.843214+0000","last_became_peered":"2024-05-27T07:41:08.843214+0000","last_unstale":"2024-05-27T07:46:27.350533+0000","last_undegraded":"2024-05-27T07:46:27.350533+0000","last_fullsized":"2024-05-27T07:46:27.350533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:00.884178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343952+0000","last_change":"2024-05-27T07:41:08.839487+0000","last_active":"2024-05-27T07:46:27.343952+0000","last_peered":"2024-05-27T07:46:27.343952+0000","last_clean":"2024-05-27T07:46:27.343952+0000","last_became_active":"2024-05-27T07:41:08.839097+0000","last_became_peered":"2024-05-27T07:41:08.839097+0000","last_unstale":"2024-05-27T07:46:27.343952+0000","last_undegraded":"2024-05-27T07:46:27.343952+0000","last_fullsized":"2024-05-27T07:46:27.343952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:51.055482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517759+0000","last_change":"2024-05-27T07:41:08.836843+0000","last_active":"2024-05-27T07:46:27.517759+0000","last_peered":"2024-05-27T07:46:27.517759+0000","last_clean":"2024-05-27T07:46:27.517759+0000","last_became_active":"2024-05-27T07:41:08.836362+0000","last_became_peered":"2024-05-27T07:41:08.836362+0000","last_unstale":"2024-05-27T07:46:27.517759+0000","last_undegraded":"2024-05-27T07:46:27.517759+0000","last_fullsized":"2024-05-27T07:46:27.517759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:47.844301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341794+0000","last_change":"2024-05-27T07:41:08.839046+0000","last_active":"2024-05-27T07:46:27.341794+0000","last_peered":"2024-05-27T07:46:27.341794+0000","last_clean":"2024-05-27T07:46:27.341794+0000","last_became_active":"2024-05-27T07:41:08.838601+0000","last_became_peered":"2024-05-27T07:41:08.838601+0000","last_unstale":"2024-05-27T07:46:27.341794+0000","last_undegraded":"2024-05-27T07:46:27.341794+0000","last_fullsized":"2024-05-27T07:46:27.341794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:59.563490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341873+0000","last_change":"2024-05-27T07:41:08.832510+0000","last_active":"2024-05-27T07:46:27.341873+0000","last_peered":"2024-05-27T07:46:27.341873+0000","last_clean":"2024-05-27T07:46:27.341873+0000","last_became_active":"2024-05-27T07:41:08.832082+0000","last_became_peered":"2024-05-27T07:41:08.832082+0000","last_unstale":"2024-05-27T07:46:27.341873+0000","last_undegraded":"2024-05-27T07:46:27.341873+0000","last_fullsized":"2024-05-27T07:46:27.341873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:59.951860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339613+0000","last_change":"2024-05-27T07:41:08.827953+0000","last_active":"2024-05-27T07:46:27.339613+0000","last_peered":"2024-05-27T07:46:27.339613+0000","last_clean":"2024-05-27T07:46:27.339613+0000","last_became_active":"2024-05-27T07:41:08.827293+0000","last_became_peered":"2024-05-27T07:41:08.827293+0000","last_unstale":"2024-05-27T07:46:27.339613+0000","last_undegraded":"2024-05-27T07:46:27.339613+0000","last_fullsized":"2024-05-27T07:46:27.339613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:08.810514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091279+0000","last_change":"2024-05-27T07:41:08.839591+0000","last_active":"2024-05-27T07:46:27.091279+0000","last_peered":"2024-05-27T07:46:27.091279+0000","last_clean":"2024-05-27T07:46:27.091279+0000","last_became_active":"2024-05-27T07:41:08.839217+0000","last_became_peered":"2024-05-27T07:41:08.839217+0000","last_unstale":"2024-05-27T07:46:27.091279+0000","last_undegraded":"2024-05-27T07:46:27.091279+0000","last_fullsized":"2024-05-27T07:46:27.091279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:11:41.636163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856457+0000","last_change":"2024-05-27T07:41:08.839544+0000","last_active":"2024-05-27T07:46:26.856457+0000","last_peered":"2024-05-27T07:46:26.856457+0000","last_clean":"2024-05-27T07:46:26.856457+0000","last_became_active":"2024-05-27T07:41:08.839177+0000","last_became_peered":"2024-05-27T07:41:08.839177+0000","last_unstale":"2024-05-27T07:46:26.856457+0000","last_undegraded":"2024-05-27T07:46:26.856457+0000","last_fullsized":"2024-05-27T07:46:26.856457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:41.921759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339195+0000","last_change":"2024-05-27T07:41:08.847911+0000","last_active":"2024-05-27T07:46:27.339195+0000","last_peered":"2024-05-27T07:46:27.339195+0000","last_clean":"2024-05-27T07:46:27.339195+0000","last_became_active":"2024-05-27T07:41:08.847427+0000","last_became_peered":"2024-05-27T07:41:08.847427+0000","last_unstale":"2024-05-27T07:46:27.339195+0000","last_undegraded":"2024-05-27T07:46:27.339195+0000","last_fullsized":"2024-05-27T07:46:27.339195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:13.692744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338331+0000","last_change":"2024-05-27T07:41:21.999614+0000","last_active":"2024-05-27T07:46:27.338331+0000","last_peered":"2024-05-27T07:46:27.338331+0000","last_clean":"2024-05-27T07:46:27.338331+0000","last_became_active":"2024-05-27T07:41:21.999086+0000","last_became_peered":"2024-05-27T07:41:21.999086+0000","last_unstale":"2024-05-27T07:46:27.338331+0000","last_undegraded":"2024-05-27T07:46:27.338331+0000","last_fullsized":"2024-05-27T07:46:27.338331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:20.229318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":301,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.592306+0000","last_change":"2024-05-27T07:41:04.795079+0000","last_active":"2024-05-27T07:46:31.592306+0000","last_peered":"2024-05-27T07:46:31.592306+0000","last_clean":"2024-05-27T07:46:31.592306+0000","last_became_active":"2024-05-27T07:41:04.794367+0000","last_became_peered":"2024-05-27T07:41:04.794367+0000","last_unstale":"2024-05-27T07:46:31.592306+0000","last_undegraded":"2024-05-27T07:46:31.592306+0000","last_fullsized":"2024-05-27T07:46:31.592306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:27:52.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014433+0000","last_change":"2024-05-27T07:41:08.846712+0000","last_active":"2024-05-27T07:46:27.014433+0000","last_peered":"2024-05-27T07:46:27.014433+0000","last_clean":"2024-05-27T07:46:27.014433+0000","last_became_active":"2024-05-27T07:41:08.839999+0000","last_became_peered":"2024-05-27T07:41:08.839999+0000","last_unstale":"2024-05-27T07:46:27.014433+0000","last_undegraded":"2024-05-27T07:46:27.014433+0000","last_fullsized":"2024-05-27T07:46:27.014433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:31:23.924074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.340435+0000","last_change":"2024-05-27T07:41:21.996675+0000","last_active":"2024-05-27T07:46:27.340435+0000","last_peered":"2024-05-27T07:46:27.340435+0000","last_clean":"2024-05-27T07:46:27.340435+0000","last_became_active":"2024-05-27T07:41:21.996152+0000","last_became_peered":"2024-05-27T07:41:21.996152+0000","last_unstale":"2024-05-27T07:46:27.340435+0000","last_undegraded":"2024-05-27T07:46:27.340435+0000","last_fullsized":"2024-05-27T07:46:27.340435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:57.415146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014696+0000","last_change":"2024-05-27T07:41:04.890045+0000","last_active":"2024-05-27T07:46:27.014696+0000","last_peered":"2024-05-27T07:46:27.014696+0000","last_clean":"2024-05-27T07:46:27.014696+0000","last_became_active":"2024-05-27T07:41:04.888993+0000","last_became_peered":"2024-05-27T07:41:04.888993+0000","last_unstale":"2024-05-27T07:46:27.014696+0000","last_undegraded":"2024-05-27T07:46:27.014696+0000","last_fullsized":"2024-05-27T07:46:27.014696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:19.866699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342407+0000","last_change":"2024-05-27T07:41:08.847472+0000","last_active":"2024-05-27T07:46:27.342407+0000","last_peered":"2024-05-27T07:46:27.342407+0000","last_clean":"2024-05-27T07:46:27.342407+0000","last_became_active":"2024-05-27T07:41:08.846070+0000","last_became_peered":"2024-05-27T07:41:08.846070+0000","last_unstale":"2024-05-27T07:46:27.342407+0000","last_undegraded":"2024-05-27T07:46:27.342407+0000","last_fullsized":"2024-05-27T07:46:27.342407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:29:08.497007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343782+0000","last_change":"2024-05-27T07:41:59.842473+0000","last_active":"2024-05-27T07:46:27.343782+0000","last_peered":"2024-05-27T07:46:27.343782+0000","last_clean":"2024-05-27T07:46:27.343782+0000","last_became_active":"2024-05-27T07:41:59.841035+0000","last_became_peered":"2024-05-27T07:41:59.841035+0000","last_unstale":"2024-05-27T07:46:27.343782+0000","last_undegraded":"2024-05-27T07:46:27.343782+0000","last_fullsized":"2024-05-27T07:46:27.343782+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:52.959589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'13","reported_seq":62,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.860220+0000","last_change":"2024-05-27T07:41:04.804046+0000","last_active":"2024-05-27T07:46:26.860220+0000","last_peered":"2024-05-27T07:46:26.860220+0000","last_clean":"2024-05-27T07:46:26.860220+0000","last_became_active":"2024-05-27T07:41:04.803360+0000","last_became_peered":"2024-05-27T07:41:04.803360+0000","last_unstale":"2024-05-27T07:46:26.860220+0000","last_undegraded":"2024-05-27T07:46:26.860220+0000","last_fullsized":"2024-05-27T07:46:26.860220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:17.325221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341836+0000","last_change":"2024-05-27T07:41:08.840842+0000","last_active":"2024-05-27T07:46:27.341836+0000","last_peered":"2024-05-27T07:46:27.341836+0000","last_clean":"2024-05-27T07:46:27.341836+0000","last_became_active":"2024-05-27T07:41:08.837817+0000","last_became_peered":"2024-05-27T07:41:08.837817+0000","last_unstale":"2024-05-27T07:46:27.341836+0000","last_undegraded":"2024-05-27T07:46:27.341836+0000","last_fullsized":"2024-05-27T07:46:27.341836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:53:38.130608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.340944+0000","last_change":"2024-05-27T07:41:22.002988+0000","last_active":"2024-05-27T07:46:27.340944+0000","last_peered":"2024-05-27T07:46:27.340944+0000","last_clean":"2024-05-27T07:46:27.340944+0000","last_became_active":"2024-05-27T07:41:22.001141+0000","last_became_peered":"2024-05-27T07:41:22.001141+0000","last_unstale":"2024-05-27T07:46:27.340944+0000","last_undegraded":"2024-05-27T07:46:27.340944+0000","last_fullsized":"2024-05-27T07:46:27.340944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:29:17.380439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.859451+0000","last_change":"2024-05-27T07:41:04.794424+0000","last_active":"2024-05-27T07:46:26.859451+0000","last_peered":"2024-05-27T07:46:26.859451+0000","last_clean":"2024-05-27T07:46:26.859451+0000","last_became_active":"2024-05-27T07:41:04.793912+0000","last_became_peered":"2024-05-27T07:41:04.793912+0000","last_unstale":"2024-05-27T07:46:26.859451+0000","last_undegraded":"2024-05-27T07:46:26.859451+0000","last_fullsized":"2024-05-27T07:46:26.859451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:50.876891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.031011+0000","last_change":"2024-05-27T07:41:08.832356+0000","last_active":"2024-05-27T07:46:27.031011+0000","last_peered":"2024-05-27T07:46:27.031011+0000","last_clean":"2024-05-27T07:46:27.031011+0000","last_became_active":"2024-05-27T07:41:08.831979+0000","last_became_peered":"2024-05-27T07:41:08.831979+0000","last_unstale":"2024-05-27T07:46:27.031011+0000","last_undegraded":"2024-05-27T07:46:27.031011+0000","last_fullsized":"2024-05-27T07:46:27.031011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:37:47.584177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338988+0000","last_change":"2024-05-27T07:41:22.004233+0000","last_active":"2024-05-27T07:46:27.338988+0000","last_peered":"2024-05-27T07:46:27.338988+0000","last_clean":"2024-05-27T07:46:27.338988+0000","last_became_active":"2024-05-27T07:41:22.003781+0000","last_became_peered":"2024-05-27T07:41:22.003781+0000","last_unstale":"2024-05-27T07:46:27.338988+0000","last_undegraded":"2024-05-27T07:46:27.338988+0000","last_fullsized":"2024-05-27T07:46:27.338988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:02:13.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'10","reported_seq":303,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.671516+0000","last_change":"2024-05-27T07:41:04.800373+0000","last_active":"2024-05-27T07:46:31.671516+0000","last_peered":"2024-05-27T07:46:31.671516+0000","last_clean":"2024-05-27T07:46:31.671516+0000","last_became_active":"2024-05-27T07:41:04.799930+0000","last_became_peered":"2024-05-27T07:41:04.799930+0000","last_unstale":"2024-05-27T07:46:31.671516+0000","last_undegraded":"2024-05-27T07:46:31.671516+0000","last_fullsized":"2024-05-27T07:46:31.671516+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:24.642479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2413,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":245,"num_read_kb":7,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.857099+0000","last_change":"2024-05-27T07:41:08.830387+0000","last_active":"2024-05-27T07:46:26.857099+0000","last_peered":"2024-05-27T07:46:26.857099+0000","last_clean":"2024-05-27T07:46:26.857099+0000","last_became_active":"2024-05-27T07:41:08.829720+0000","last_became_peered":"2024-05-27T07:41:08.829720+0000","last_unstale":"2024-05-27T07:46:26.857099+0000","last_undegraded":"2024-05-27T07:46:26.857099+0000","last_fullsized":"2024-05-27T07:46:26.857099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:42.014383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338425+0000","last_change":"2024-05-27T07:41:22.005700+0000","last_active":"2024-05-27T07:46:27.338425+0000","last_peered":"2024-05-27T07:46:27.338425+0000","last_clean":"2024-05-27T07:46:27.338425+0000","last_became_active":"2024-05-27T07:41:22.003643+0000","last_became_peered":"2024-05-27T07:41:22.003643+0000","last_unstale":"2024-05-27T07:46:27.338425+0000","last_undegraded":"2024-05-27T07:46:27.338425+0000","last_fullsized":"2024-05-27T07:46:27.338425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:48.332767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.855954+0000","last_change":"2024-05-27T07:41:59.790579+0000","last_active":"2024-05-27T07:46:26.855954+0000","last_peered":"2024-05-27T07:46:26.855954+0000","last_clean":"2024-05-27T07:46:26.855954+0000","last_became_active":"2024-05-27T07:41:59.789849+0000","last_became_peered":"2024-05-27T07:41:59.789849+0000","last_unstale":"2024-05-27T07:46:26.855954+0000","last_undegraded":"2024-05-27T07:46:26.855954+0000","last_fullsized":"2024-05-27T07:46:26.855954+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:24:30.564074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.851628+0000","last_change":"2024-05-27T07:43:00.121063+0000","last_active":"2024-05-27T07:46:26.851628+0000","last_peered":"2024-05-27T07:46:26.851628+0000","last_clean":"2024-05-27T07:46:26.851628+0000","last_became_active":"2024-05-27T07:43:00.119977+0000","last_became_peered":"2024-05-27T07:43:00.119977+0000","last_unstale":"2024-05-27T07:46:26.851628+0000","last_undegraded":"2024-05-27T07:46:26.851628+0000","last_fullsized":"2024-05-27T07:46:26.851628+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:04:43.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091113+0000","last_change":"2024-05-27T07:41:22.013820+0000","last_active":"2024-05-27T07:46:27.091113+0000","last_peered":"2024-05-27T07:46:27.091113+0000","last_clean":"2024-05-27T07:46:27.091113+0000","last_became_active":"2024-05-27T07:41:22.013110+0000","last_became_peered":"2024-05-27T07:41:22.013110+0000","last_unstale":"2024-05-27T07:46:27.091113+0000","last_undegraded":"2024-05-27T07:46:27.091113+0000","last_fullsized":"2024-05-27T07:46:27.091113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:45:07.317872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517879+0000","last_change":"2024-05-27T07:41:04.789197+0000","last_active":"2024-05-27T07:46:27.517879+0000","last_peered":"2024-05-27T07:46:27.517879+0000","last_clean":"2024-05-27T07:46:27.517879+0000","last_became_active":"2024-05-27T07:41:04.788755+0000","last_became_peered":"2024-05-27T07:41:04.788755+0000","last_unstale":"2024-05-27T07:46:27.517879+0000","last_undegraded":"2024-05-27T07:46:27.517879+0000","last_fullsized":"2024-05-27T07:46:27.517879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:27:54.526910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013815+0000","last_change":"2024-05-27T07:41:08.841769+0000","last_active":"2024-05-27T07:46:27.013815+0000","last_peered":"2024-05-27T07:46:27.013815+0000","last_clean":"2024-05-27T07:46:27.013815+0000","last_became_active":"2024-05-27T07:41:08.840834+0000","last_became_peered":"2024-05-27T07:41:08.840834+0000","last_unstale":"2024-05-27T07:46:27.013815+0000","last_undegraded":"2024-05-27T07:46:27.013815+0000","last_fullsized":"2024-05-27T07:46:27.013815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:15.573739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342674+0000","last_change":"2024-05-27T07:41:22.005119+0000","last_active":"2024-05-27T07:46:27.342674+0000","last_peered":"2024-05-27T07:46:27.342674+0000","last_clean":"2024-05-27T07:46:27.342674+0000","last_became_active":"2024-05-27T07:41:22.002638+0000","last_became_peered":"2024-05-27T07:41:22.002638+0000","last_unstale":"2024-05-27T07:46:27.342674+0000","last_undegraded":"2024-05-27T07:46:27.342674+0000","last_fullsized":"2024-05-27T07:46:27.342674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:08:49.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029700+0000","last_change":"2024-05-27T07:41:04.799864+0000","last_active":"2024-05-27T07:46:27.029700+0000","last_peered":"2024-05-27T07:46:27.029700+0000","last_clean":"2024-05-27T07:46:27.029700+0000","last_became_active":"2024-05-27T07:41:04.799218+0000","last_became_peered":"2024-05-27T07:41:04.799218+0000","last_unstale":"2024-05-27T07:46:27.029700+0000","last_undegraded":"2024-05-27T07:46:27.029700+0000","last_fullsized":"2024-05-27T07:46:27.029700+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:15.355720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.851911+0000","last_change":"2024-05-27T07:41:08.823387+0000","last_active":"2024-05-27T07:46:26.851911+0000","last_peered":"2024-05-27T07:46:26.851911+0000","last_clean":"2024-05-27T07:46:26.851911+0000","last_became_active":"2024-05-27T07:41:08.822943+0000","last_became_peered":"2024-05-27T07:41:08.822943+0000","last_unstale":"2024-05-27T07:46:26.851911+0000","last_undegraded":"2024-05-27T07:46:26.851911+0000","last_fullsized":"2024-05-27T07:46:26.851911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:46.365284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013958+0000","last_change":"2024-05-27T07:41:22.011848+0000","last_active":"2024-05-27T07:46:27.013958+0000","last_peered":"2024-05-27T07:46:27.013958+0000","last_clean":"2024-05-27T07:46:27.013958+0000","last_became_active":"2024-05-27T07:41:22.011335+0000","last_became_peered":"2024-05-27T07:41:22.011335+0000","last_unstale":"2024-05-27T07:46:27.013958+0000","last_undegraded":"2024-05-27T07:46:27.013958+0000","last_fullsized":"2024-05-27T07:46:27.013958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:29:22.528270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":531,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:32.651357+0000","last_change":"2024-05-27T07:41:04.793877+0000","last_active":"2024-05-27T07:46:32.651357+0000","last_peered":"2024-05-27T07:46:32.651357+0000","last_clean":"2024-05-27T07:46:32.651357+0000","last_became_active":"2024-05-27T07:41:04.793389+0000","last_became_peered":"2024-05-27T07:41:04.793389+0000","last_unstale":"2024-05-27T07:46:32.651357+0000","last_undegraded":"2024-05-27T07:46:32.651357+0000","last_fullsized":"2024-05-27T07:46:32.651357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:48.670572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342653+0000","last_change":"2024-05-27T07:41:08.836682+0000","last_active":"2024-05-27T07:46:27.342653+0000","last_peered":"2024-05-27T07:46:27.342653+0000","last_clean":"2024-05-27T07:46:27.342653+0000","last_became_active":"2024-05-27T07:41:08.836229+0000","last_became_peered":"2024-05-27T07:41:08.836229+0000","last_unstale":"2024-05-27T07:46:27.342653+0000","last_undegraded":"2024-05-27T07:46:27.342653+0000","last_fullsized":"2024-05-27T07:46:27.342653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:33.249584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.850945+0000","last_change":"2024-05-27T07:41:22.004968+0000","last_active":"2024-05-27T07:46:26.850945+0000","last_peered":"2024-05-27T07:46:26.850945+0000","last_clean":"2024-05-27T07:46:26.850945+0000","last_became_active":"2024-05-27T07:41:22.001407+0000","last_became_peered":"2024-05-27T07:41:22.001407+0000","last_unstale":"2024-05-27T07:46:26.850945+0000","last_undegraded":"2024-05-27T07:46:26.850945+0000","last_fullsized":"2024-05-27T07:46:26.850945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:26.872497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091231+0000","last_change":"2024-05-27T07:41:59.787247+0000","last_active":"2024-05-27T07:46:27.091231+0000","last_peered":"2024-05-27T07:46:27.091231+0000","last_clean":"2024-05-27T07:46:27.091231+0000","last_became_active":"2024-05-27T07:41:59.786392+0000","last_became_peered":"2024-05-27T07:41:59.786392+0000","last_unstale":"2024-05-27T07:46:27.091231+0000","last_undegraded":"2024-05-27T07:46:27.091231+0000","last_fullsized":"2024-05-27T07:46:27.091231+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:12.579707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2],"acting":[11,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342877+0000","last_change":"2024-05-27T07:41:08.833731+0000","last_active":"2024-05-27T07:46:27.342877+0000","last_peered":"2024-05-27T07:46:27.342877+0000","last_clean":"2024-05-27T07:46:27.342877+0000","last_became_active":"2024-05-27T07:41:08.833318+0000","last_became_peered":"2024-05-27T07:41:08.833318+0000","last_unstale":"2024-05-27T07:46:27.342877+0000","last_undegraded":"2024-05-27T07:46:27.342877+0000","last_fullsized":"2024-05-27T07:46:27.342877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:50.085113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343580+0000","last_change":"2024-05-27T07:41:59.789784+0000","last_active":"2024-05-27T07:46:27.343580+0000","last_peered":"2024-05-27T07:46:27.343580+0000","last_clean":"2024-05-27T07:46:27.343580+0000","last_became_active":"2024-05-27T07:41:59.788589+0000","last_became_peered":"2024-05-27T07:41:59.788589+0000","last_unstale":"2024-05-27T07:46:27.343580+0000","last_undegraded":"2024-05-27T07:46:27.343580+0000","last_fullsized":"2024-05-27T07:46:27.343580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:37.026907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030707+0000","last_change":"2024-05-27T07:41:04.892603+0000","last_active":"2024-05-27T07:46:27.030707+0000","last_peered":"2024-05-27T07:46:27.030707+0000","last_clean":"2024-05-27T07:46:27.030707+0000","last_became_active":"2024-05-27T07:41:04.890786+0000","last_became_peered":"2024-05-27T07:41:04.890786+0000","last_unstale":"2024-05-27T07:46:27.030707+0000","last_undegraded":"2024-05-27T07:46:27.030707+0000","last_fullsized":"2024-05-27T07:46:27.030707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:34.129227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342154+0000","last_change":"2024-05-27T07:41:08.830802+0000","last_active":"2024-05-27T07:46:27.342154+0000","last_peered":"2024-05-27T07:46:27.342154+0000","last_clean":"2024-05-27T07:46:27.342154+0000","last_became_active":"2024-05-27T07:41:08.829661+0000","last_became_peered":"2024-05-27T07:41:08.829661+0000","last_unstale":"2024-05-27T07:46:27.342154+0000","last_undegraded":"2024-05-27T07:46:27.342154+0000","last_fullsized":"2024-05-27T07:46:27.342154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:20:15.041554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.852318+0000","last_change":"2024-05-27T07:41:22.003997+0000","last_active":"2024-05-27T07:46:26.852318+0000","last_peered":"2024-05-27T07:46:26.852318+0000","last_clean":"2024-05-27T07:46:26.852318+0000","last_became_active":"2024-05-27T07:41:22.003527+0000","last_became_peered":"2024-05-27T07:41:22.003527+0000","last_unstale":"2024-05-27T07:46:26.852318+0000","last_undegraded":"2024-05-27T07:46:26.852318+0000","last_fullsized":"2024-05-27T07:46:26.852318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:17.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'47","reported_seq":603,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:32.704711+0000","last_change":"2024-05-27T07:41:59.814227+0000","last_active":"2024-05-27T07:46:32.704711+0000","last_peered":"2024-05-27T07:46:32.704711+0000","last_clean":"2024-05-27T07:46:32.704711+0000","last_became_active":"2024-05-27T07:41:59.788510+0000","last_became_peered":"2024-05-27T07:41:59.788510+0000","last_unstale":"2024-05-27T07:46:32.704711+0000","last_undegraded":"2024-05-27T07:46:32.704711+0000","last_fullsized":"2024-05-27T07:46:32.704711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:15.868348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75767,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":320,"num_write":49,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.859696+0000","last_change":"2024-05-27T07:41:08.831639+0000","last_active":"2024-05-27T07:46:26.859696+0000","last_peered":"2024-05-27T07:46:26.859696+0000","last_clean":"2024-05-27T07:46:26.859696+0000","last_became_active":"2024-05-27T07:41:08.830541+0000","last_became_peered":"2024-05-27T07:41:08.830541+0000","last_unstale":"2024-05-27T07:46:26.859696+0000","last_undegraded":"2024-05-27T07:46:26.859696+0000","last_fullsized":"2024-05-27T07:46:26.859696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:45.635145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.031603+0000","last_change":"2024-05-27T07:41:21.999587+0000","last_active":"2024-05-27T07:46:27.031603+0000","last_peered":"2024-05-27T07:46:27.031603+0000","last_clean":"2024-05-27T07:46:27.031603+0000","last_became_active":"2024-05-27T07:41:21.999017+0000","last_became_peered":"2024-05-27T07:41:21.999017+0000","last_unstale":"2024-05-27T07:46:27.031603+0000","last_undegraded":"2024-05-27T07:46:27.031603+0000","last_fullsized":"2024-05-27T07:46:27.031603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:31:14.915229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338645+0000","last_change":"2024-05-27T07:41:04.799847+0000","last_active":"2024-05-27T07:46:27.338645+0000","last_peered":"2024-05-27T07:46:27.338645+0000","last_clean":"2024-05-27T07:46:27.338645+0000","last_became_active":"2024-05-27T07:41:04.798605+0000","last_became_peered":"2024-05-27T07:41:04.798605+0000","last_unstale":"2024-05-27T07:46:27.338645+0000","last_undegraded":"2024-05-27T07:46:27.338645+0000","last_fullsized":"2024-05-27T07:46:27.338645+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:01.957707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517549+0000","last_change":"2024-05-27T07:41:08.831138+0000","last_active":"2024-05-27T07:46:27.517549+0000","last_peered":"2024-05-27T07:46:27.517549+0000","last_clean":"2024-05-27T07:46:27.517549+0000","last_became_active":"2024-05-27T07:41:08.830735+0000","last_became_peered":"2024-05-27T07:41:08.830735+0000","last_unstale":"2024-05-27T07:46:27.517549+0000","last_undegraded":"2024-05-27T07:46:27.517549+0000","last_fullsized":"2024-05-27T07:46:27.517549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:12.569687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090504+0000","last_change":"2024-05-27T07:41:59.779039+0000","last_active":"2024-05-27T07:46:27.090504+0000","last_peered":"2024-05-27T07:46:27.090504+0000","last_clean":"2024-05-27T07:46:27.090504+0000","last_became_active":"2024-05-27T07:41:59.777514+0000","last_became_peered":"2024-05-27T07:41:59.777514+0000","last_unstale":"2024-05-27T07:46:27.090504+0000","last_undegraded":"2024-05-27T07:46:27.090504+0000","last_fullsized":"2024-05-27T07:46:27.090504+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:12:19.223874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030345+0000","last_change":"2024-05-27T07:41:04.796467+0000","last_active":"2024-05-27T07:46:27.030345+0000","last_peered":"2024-05-27T07:46:27.030345+0000","last_clean":"2024-05-27T07:46:27.030345+0000","last_became_active":"2024-05-27T07:41:04.795519+0000","last_became_peered":"2024-05-27T07:41:04.795519+0000","last_unstale":"2024-05-27T07:46:27.030345+0000","last_undegraded":"2024-05-27T07:46:27.030345+0000","last_fullsized":"2024-05-27T07:46:27.030345+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:42.744076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350149+0000","last_change":"2024-05-27T07:41:08.832135+0000","last_active":"2024-05-27T07:46:27.350149+0000","last_peered":"2024-05-27T07:46:27.350149+0000","last_clean":"2024-05-27T07:46:27.350149+0000","last_became_active":"2024-05-27T07:41:08.830032+0000","last_became_peered":"2024-05-27T07:41:08.830032+0000","last_unstale":"2024-05-27T07:46:27.350149+0000","last_undegraded":"2024-05-27T07:46:27.350149+0000","last_fullsized":"2024-05-27T07:46:27.350149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:44:59.367026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856691+0000","last_change":"2024-05-27T07:41:22.008868+0000","last_active":"2024-05-27T07:46:26.856691+0000","last_peered":"2024-05-27T07:46:26.856691+0000","last_clean":"2024-05-27T07:46:26.856691+0000","last_became_active":"2024-05-27T07:41:22.002071+0000","last_became_peered":"2024-05-27T07:41:22.002071+0000","last_unstale":"2024-05-27T07:46:26.856691+0000","last_undegraded":"2024-05-27T07:46:26.856691+0000","last_fullsized":"2024-05-27T07:46:26.856691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:20.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'9","reported_seq":301,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.706139+0000","last_change":"2024-05-27T07:41:04.796198+0000","last_active":"2024-05-27T07:46:31.706139+0000","last_peered":"2024-05-27T07:46:31.706139+0000","last_clean":"2024-05-27T07:46:31.706139+0000","last_became_active":"2024-05-27T07:41:04.795646+0000","last_became_peered":"2024-05-27T07:41:04.795646+0000","last_unstale":"2024-05-27T07:46:31.706139+0000","last_undegraded":"2024-05-27T07:46:31.706139+0000","last_fullsized":"2024-05-27T07:46:31.706139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:30.936596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1988,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":245,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350240+0000","last_change":"2024-05-27T07:41:08.840518+0000","last_active":"2024-05-27T07:46:27.350240+0000","last_peered":"2024-05-27T07:46:27.350240+0000","last_clean":"2024-05-27T07:46:27.350240+0000","last_became_active":"2024-05-27T07:41:08.840162+0000","last_became_peered":"2024-05-27T07:41:08.840162+0000","last_unstale":"2024-05-27T07:46:27.350240+0000","last_undegraded":"2024-05-27T07:46:27.350240+0000","last_fullsized":"2024-05-27T07:46:27.350240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:04:14.657496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.518023+0000","last_change":"2024-05-27T07:41:22.000243+0000","last_active":"2024-05-27T07:46:27.518023+0000","last_peered":"2024-05-27T07:46:27.518023+0000","last_clean":"2024-05-27T07:46:27.518023+0000","last_became_active":"2024-05-27T07:41:21.998898+0000","last_became_peered":"2024-05-27T07:41:21.998898+0000","last_unstale":"2024-05-27T07:46:27.518023+0000","last_undegraded":"2024-05-27T07:46:27.518023+0000","last_fullsized":"2024-05-27T07:46:27.518023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:35.696593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":82,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856741+0000","last_change":"2024-05-27T07:41:59.799662+0000","last_active":"2024-05-27T07:46:26.856741+0000","last_peered":"2024-05-27T07:46:26.856741+0000","last_clean":"2024-05-27T07:46:26.856741+0000","last_became_active":"2024-05-27T07:41:59.779733+0000","last_became_peered":"2024-05-27T07:41:59.779733+0000","last_unstale":"2024-05-27T07:46:26.856741+0000","last_undegraded":"2024-05-27T07:46:26.856741+0000","last_fullsized":"2024-05-27T07:46:26.856741+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:23.665014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030069+0000","last_change":"2024-05-27T07:41:08.834931+0000","last_active":"2024-05-27T07:46:27.030069+0000","last_peered":"2024-05-27T07:46:27.030069+0000","last_clean":"2024-05-27T07:46:27.030069+0000","last_became_active":"2024-05-27T07:41:08.833298+0000","last_became_peered":"2024-05-27T07:41:08.833298+0000","last_unstale":"2024-05-27T07:46:27.030069+0000","last_undegraded":"2024-05-27T07:46:27.030069+0000","last_fullsized":"2024-05-27T07:46:27.030069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:03:48.507461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029888+0000","last_change":"2024-05-27T07:41:22.009407+0000","last_active":"2024-05-27T07:46:27.029888+0000","last_peered":"2024-05-27T07:46:27.029888+0000","last_clean":"2024-05-27T07:46:27.029888+0000","last_became_active":"2024-05-27T07:41:22.006921+0000","last_became_peered":"2024-05-27T07:41:22.006921+0000","last_unstale":"2024-05-27T07:46:27.029888+0000","last_undegraded":"2024-05-27T07:46:27.029888+0000","last_fullsized":"2024-05-27T07:46:27.029888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:24.034502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.860585+0000","last_change":"2024-05-27T07:41:04.795641+0000","last_active":"2024-05-27T07:46:26.860585+0000","last_peered":"2024-05-27T07:46:26.860585+0000","last_clean":"2024-05-27T07:46:26.860585+0000","last_became_active":"2024-05-27T07:41:04.794935+0000","last_became_peered":"2024-05-27T07:41:04.794935+0000","last_unstale":"2024-05-27T07:46:26.860585+0000","last_undegraded":"2024-05-27T07:46:26.860585+0000","last_fullsized":"2024-05-27T07:46:26.860585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:14.850282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091428+0000","last_change":"2024-05-27T07:41:08.837277+0000","last_active":"2024-05-27T07:46:27.091428+0000","last_peered":"2024-05-27T07:46:27.091428+0000","last_clean":"2024-05-27T07:46:27.091428+0000","last_became_active":"2024-05-27T07:41:08.835935+0000","last_became_peered":"2024-05-27T07:41:08.835935+0000","last_unstale":"2024-05-27T07:46:27.091428+0000","last_undegraded":"2024-05-27T07:46:27.091428+0000","last_fullsized":"2024-05-27T07:46:27.091428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:46.436243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856802+0000","last_change":"2024-05-27T07:41:22.014700+0000","last_active":"2024-05-27T07:46:26.856802+0000","last_peered":"2024-05-27T07:46:26.856802+0000","last_clean":"2024-05-27T07:46:26.856802+0000","last_became_active":"2024-05-27T07:41:22.014136+0000","last_became_peered":"2024-05-27T07:41:22.014136+0000","last_unstale":"2024-05-27T07:46:26.856802+0000","last_undegraded":"2024-05-27T07:46:26.856802+0000","last_fullsized":"2024-05-27T07:46:26.856802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:51:08.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.855048+0000","last_change":"2024-05-27T07:41:04.891269+0000","last_active":"2024-05-27T07:46:26.855048+0000","last_peered":"2024-05-27T07:46:26.855048+0000","last_clean":"2024-05-27T07:46:26.855048+0000","last_became_active":"2024-05-27T07:41:04.890843+0000","last_became_peered":"2024-05-27T07:41:04.890843+0000","last_unstale":"2024-05-27T07:46:26.855048+0000","last_undegraded":"2024-05-27T07:46:26.855048+0000","last_fullsized":"2024-05-27T07:46:26.855048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:57.065300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856313+0000","last_change":"2024-05-27T07:41:08.841977+0000","last_active":"2024-05-27T07:46:26.856313+0000","last_peered":"2024-05-27T07:46:26.856313+0000","last_clean":"2024-05-27T07:46:26.856313+0000","last_became_active":"2024-05-27T07:41:08.841551+0000","last_became_peered":"2024-05-27T07:41:08.841551+0000","last_unstale":"2024-05-27T07:46:26.856313+0000","last_undegraded":"2024-05-27T07:46:26.856313+0000","last_fullsized":"2024-05-27T07:46:26.856313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:35:07.766193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517708+0000","last_change":"2024-05-27T07:41:22.006310+0000","last_active":"2024-05-27T07:46:27.517708+0000","last_peered":"2024-05-27T07:46:27.517708+0000","last_clean":"2024-05-27T07:46:27.517708+0000","last_became_active":"2024-05-27T07:41:22.004135+0000","last_became_peered":"2024-05-27T07:41:22.004135+0000","last_unstale":"2024-05-27T07:46:27.517708+0000","last_undegraded":"2024-05-27T07:46:27.517708+0000","last_fullsized":"2024-05-27T07:46:27.517708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:48.564894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":296,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.593482+0000","last_change":"2024-05-27T07:41:04.793284+0000","last_active":"2024-05-27T07:46:31.593482+0000","last_peered":"2024-05-27T07:46:31.593482+0000","last_clean":"2024-05-27T07:46:31.593482+0000","last_became_active":"2024-05-27T07:41:04.792853+0000","last_became_peered":"2024-05-27T07:41:04.792853+0000","last_unstale":"2024-05-27T07:46:31.593482+0000","last_undegraded":"2024-05-27T07:46:31.593482+0000","last_fullsized":"2024-05-27T07:46:31.593482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:22.979341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1456,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":241,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856334+0000","last_change":"2024-05-27T07:41:59.776207+0000","last_active":"2024-05-27T07:46:26.856334+0000","last_peered":"2024-05-27T07:46:26.856334+0000","last_clean":"2024-05-27T07:46:26.856334+0000","last_became_active":"2024-05-27T07:41:59.775226+0000","last_became_peered":"2024-05-27T07:41:59.775226+0000","last_unstale":"2024-05-27T07:46:26.856334+0000","last_undegraded":"2024-05-27T07:46:26.856334+0000","last_fullsized":"2024-05-27T07:46:26.856334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:50.469144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517906+0000","last_change":"2024-05-27T07:41:21.992013+0000","last_active":"2024-05-27T07:46:27.517906+0000","last_peered":"2024-05-27T07:46:27.517906+0000","last_clean":"2024-05-27T07:46:27.517906+0000","last_became_active":"2024-05-27T07:41:21.991550+0000","last_became_peered":"2024-05-27T07:41:21.991550+0000","last_unstale":"2024-05-27T07:46:27.517906+0000","last_undegraded":"2024-05-27T07:46:27.517906+0000","last_fullsized":"2024-05-27T07:46:27.517906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:34:53.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.031130+0000","last_change":"2024-05-27T07:41:08.832512+0000","last_active":"2024-05-27T07:46:27.031130+0000","last_peered":"2024-05-27T07:46:27.031130+0000","last_clean":"2024-05-27T07:46:27.031130+0000","last_became_active":"2024-05-27T07:41:08.832113+0000","last_became_peered":"2024-05-27T07:41:08.832113+0000","last_unstale":"2024-05-27T07:46:27.031130+0000","last_undegraded":"2024-05-27T07:46:27.031130+0000","last_fullsized":"2024-05-27T07:46:27.031130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:00.530785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'7","reported_seq":294,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.705393+0000","last_change":"2024-05-27T07:41:04.891004+0000","last_active":"2024-05-27T07:46:31.705393+0000","last_peered":"2024-05-27T07:46:31.705393+0000","last_clean":"2024-05-27T07:46:31.705393+0000","last_became_active":"2024-05-27T07:41:04.890567+0000","last_became_peered":"2024-05-27T07:41:04.890567+0000","last_unstale":"2024-05-27T07:46:31.705393+0000","last_undegraded":"2024-05-27T07:46:31.705393+0000","last_fullsized":"2024-05-27T07:46:31.705393+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:43.991204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350297+0000","last_change":"2024-05-27T07:41:22.003446+0000","last_active":"2024-05-27T07:46:27.350297+0000","last_peered":"2024-05-27T07:46:27.350297+0000","last_clean":"2024-05-27T07:46:27.350297+0000","last_became_active":"2024-05-27T07:41:22.002989+0000","last_became_peered":"2024-05-27T07:41:22.002989+0000","last_unstale":"2024-05-27T07:46:27.350297+0000","last_undegraded":"2024-05-27T07:46:27.350297+0000","last_fullsized":"2024-05-27T07:46:27.350297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:12.996286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013957+0000","last_change":"2024-05-27T07:41:08.842685+0000","last_active":"2024-05-27T07:46:27.013957+0000","last_peered":"2024-05-27T07:46:27.013957+0000","last_clean":"2024-05-27T07:46:27.013957+0000","last_became_active":"2024-05-27T07:41:08.841954+0000","last_became_peered":"2024-05-27T07:41:08.841954+0000","last_unstale":"2024-05-27T07:46:27.013957+0000","last_undegraded":"2024-05-27T07:46:27.013957+0000","last_fullsized":"2024-05-27T07:46:27.013957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:18.087129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342500+0000","last_change":"2024-05-27T07:41:04.790330+0000","last_active":"2024-05-27T07:46:27.342500+0000","last_peered":"2024-05-27T07:46:27.342500+0000","last_clean":"2024-05-27T07:46:27.342500+0000","last_became_active":"2024-05-27T07:41:04.789508+0000","last_became_peered":"2024-05-27T07:41:04.789508+0000","last_unstale":"2024-05-27T07:46:27.342500+0000","last_undegraded":"2024-05-27T07:46:27.342500+0000","last_fullsized":"2024-05-27T07:46:27.342500+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:27.537662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'99","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:29.032290+0000","last_change":"2024-05-27T07:39:11.797324+0000","last_active":"2024-05-27T07:46:29.032290+0000","last_peered":"2024-05-27T07:46:29.032290+0000","last_clean":"2024-05-27T07:46:29.032290+0000","last_became_active":"2024-05-27T07:39:11.786294+0000","last_became_peered":"2024-05-27T07:39:11.786294+0000","last_unstale":"2024-05-27T07:46:29.032290+0000","last_undegraded":"2024-05-27T07:46:29.032290+0000","last_fullsized":"2024-05-27T07:46:29.032290+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_clean_scrub_stamp":"2024-05-27T07:34:18.613413+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:23.947615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030810+0000","last_change":"2024-05-27T07:41:21.997453+0000","last_active":"2024-05-27T07:46:27.030810+0000","last_peered":"2024-05-27T07:46:27.030810+0000","last_clean":"2024-05-27T07:46:27.030810+0000","last_became_active":"2024-05-27T07:41:21.997010+0000","last_became_peered":"2024-05-27T07:41:21.997010+0000","last_unstale":"2024-05-27T07:46:27.030810+0000","last_undegraded":"2024-05-27T07:46:27.030810+0000","last_fullsized":"2024-05-27T07:46:27.030810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:54:54.913274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517351+0000","last_change":"2024-05-27T07:43:00.024988+0000","last_active":"2024-05-27T07:46:27.517351+0000","last_peered":"2024-05-27T07:46:27.517351+0000","last_clean":"2024-05-27T07:46:27.517351+0000","last_became_active":"2024-05-27T07:43:00.023816+0000","last_became_peered":"2024-05-27T07:43:00.023816+0000","last_unstale":"2024-05-27T07:46:27.517351+0000","last_undegraded":"2024-05-27T07:46:27.517351+0000","last_fullsized":"2024-05-27T07:46:27.517351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:58.531985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.349949+0000","last_change":"2024-05-27T07:41:04.795262+0000","last_active":"2024-05-27T07:46:27.349949+0000","last_peered":"2024-05-27T07:46:27.349949+0000","last_clean":"2024-05-27T07:46:27.349949+0000","last_became_active":"2024-05-27T07:41:04.794866+0000","last_became_peered":"2024-05-27T07:41:04.794866+0000","last_unstale":"2024-05-27T07:46:27.349949+0000","last_undegraded":"2024-05-27T07:46:27.349949+0000","last_fullsized":"2024-05-27T07:46:27.349949+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:07:44.241444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090350+0000","last_change":"2024-05-27T07:41:22.006194+0000","last_active":"2024-05-27T07:46:27.090350+0000","last_peered":"2024-05-27T07:46:27.090350+0000","last_clean":"2024-05-27T07:46:27.090350+0000","last_became_active":"2024-05-27T07:41:22.004079+0000","last_became_peered":"2024-05-27T07:41:22.004079+0000","last_unstale":"2024-05-27T07:46:27.090350+0000","last_undegraded":"2024-05-27T07:46:27.090350+0000","last_fullsized":"2024-05-27T07:46:27.090350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:03.496722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091032+0000","last_change":"2024-05-27T07:41:59.776734+0000","last_active":"2024-05-27T07:46:27.091032+0000","last_peered":"2024-05-27T07:46:27.091032+0000","last_clean":"2024-05-27T07:46:27.091032+0000","last_became_active":"2024-05-27T07:41:59.775678+0000","last_became_peered":"2024-05-27T07:41:59.775678+0000","last_unstale":"2024-05-27T07:46:27.091032+0000","last_undegraded":"2024-05-27T07:46:27.091032+0000","last_fullsized":"2024-05-27T07:46:27.091032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:56.007178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517918+0000","last_change":"2024-05-27T07:41:08.831070+0000","last_active":"2024-05-27T07:46:27.517918+0000","last_peered":"2024-05-27T07:46:27.517918+0000","last_clean":"2024-05-27T07:46:27.517918+0000","last_became_active":"2024-05-27T07:41:08.830617+0000","last_became_peered":"2024-05-27T07:41:08.830617+0000","last_unstale":"2024-05-27T07:46:27.517918+0000","last_undegraded":"2024-05-27T07:46:27.517918+0000","last_fullsized":"2024-05-27T07:46:27.517918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:13.046162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013806+0000","last_change":"2024-05-27T07:41:22.010129+0000","last_active":"2024-05-27T07:46:27.013806+0000","last_peered":"2024-05-27T07:46:27.013806+0000","last_clean":"2024-05-27T07:46:27.013806+0000","last_became_active":"2024-05-27T07:41:22.009608+0000","last_became_peered":"2024-05-27T07:41:22.009608+0000","last_unstale":"2024-05-27T07:46:27.013806+0000","last_undegraded":"2024-05-27T07:46:27.013806+0000","last_fullsized":"2024-05-27T07:46:27.013806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.225004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'54","reported_seq":125,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.926903+0000","last_change":"2024-05-27T07:41:04.794744+0000","last_active":"2024-05-27T07:46:27.926903+0000","last_peered":"2024-05-27T07:46:27.926903+0000","last_clean":"2024-05-27T07:46:27.926903+0000","last_became_active":"2024-05-27T07:41:04.794340+0000","last_became_peered":"2024-05-27T07:41:04.794340+0000","last_unstale":"2024-05-27T07:46:27.926903+0000","last_undegraded":"2024-05-27T07:46:27.926903+0000","last_fullsized":"2024-05-27T07:46:27.926903+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:43:20.335520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":96,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014574+0000","last_change":"2024-05-27T07:41:08.841834+0000","last_active":"2024-05-27T07:46:27.014574+0000","last_peered":"2024-05-27T07:46:27.014574+0000","last_clean":"2024-05-27T07:46:27.014574+0000","last_became_active":"2024-05-27T07:41:08.840261+0000","last_became_peered":"2024-05-27T07:41:08.840261+0000","last_unstale":"2024-05-27T07:46:27.014574+0000","last_undegraded":"2024-05-27T07:46:27.014574+0000","last_fullsized":"2024-05-27T07:46:27.014574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:33.738835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338031+0000","last_change":"2024-05-27T07:41:22.005861+0000","last_active":"2024-05-27T07:46:27.338031+0000","last_peered":"2024-05-27T07:46:27.338031+0000","last_clean":"2024-05-27T07:46:27.338031+0000","last_became_active":"2024-05-27T07:41:22.005295+0000","last_became_peered":"2024-05-27T07:41:22.005295+0000","last_unstale":"2024-05-27T07:46:27.338031+0000","last_undegraded":"2024-05-27T07:46:27.338031+0000","last_fullsized":"2024-05-27T07:46:27.338031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:46.582656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343151+0000","last_change":"2024-05-27T07:41:04.796562+0000","last_active":"2024-05-27T07:46:27.343151+0000","last_peered":"2024-05-27T07:46:27.343151+0000","last_clean":"2024-05-27T07:46:27.343151+0000","last_became_active":"2024-05-27T07:41:04.795538+0000","last_became_peered":"2024-05-27T07:41:04.795538+0000","last_unstale":"2024-05-27T07:46:27.343151+0000","last_undegraded":"2024-05-27T07:46:27.343151+0000","last_fullsized":"2024-05-27T07:46:27.343151+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:12.131999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342791+0000","last_change":"2024-05-27T07:41:08.832824+0000","last_active":"2024-05-27T07:46:27.342791+0000","last_peered":"2024-05-27T07:46:27.342791+0000","last_clean":"2024-05-27T07:46:27.342791+0000","last_became_active":"2024-05-27T07:41:08.832332+0000","last_became_peered":"2024-05-27T07:41:08.832332+0000","last_unstale":"2024-05-27T07:46:27.342791+0000","last_undegraded":"2024-05-27T07:46:27.342791+0000","last_fullsized":"2024-05-27T07:46:27.342791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:34.598773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342420+0000","last_change":"2024-05-27T07:41:22.003358+0000","last_active":"2024-05-27T07:46:27.342420+0000","last_peered":"2024-05-27T07:46:27.342420+0000","last_clean":"2024-05-27T07:46:27.342420+0000","last_became_active":"2024-05-27T07:41:22.002655+0000","last_became_peered":"2024-05-27T07:41:22.002655+0000","last_unstale":"2024-05-27T07:46:27.342420+0000","last_undegraded":"2024-05-27T07:46:27.342420+0000","last_fullsized":"2024-05-27T07:46:27.342420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:41.177138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339729+0000","last_change":"2024-05-27T07:41:04.888681+0000","last_active":"2024-05-27T07:46:27.339729+0000","last_peered":"2024-05-27T07:46:27.339729+0000","last_clean":"2024-05-27T07:46:27.339729+0000","last_became_active":"2024-05-27T07:41:04.887966+0000","last_became_peered":"2024-05-27T07:41:04.887966+0000","last_unstale":"2024-05-27T07:46:27.339729+0000","last_undegraded":"2024-05-27T07:46:27.339729+0000","last_fullsized":"2024-05-27T07:46:27.339729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:01.178737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856720+0000","last_change":"2024-05-27T07:41:08.827839+0000","last_active":"2024-05-27T07:46:26.856720+0000","last_peered":"2024-05-27T07:46:26.856720+0000","last_clean":"2024-05-27T07:46:26.856720+0000","last_became_active":"2024-05-27T07:41:08.827403+0000","last_became_peered":"2024-05-27T07:41:08.827403+0000","last_unstale":"2024-05-27T07:46:26.856720+0000","last_undegraded":"2024-05-27T07:46:26.856720+0000","last_fullsized":"2024-05-27T07:46:26.856720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:01:42.075980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856869+0000","last_change":"2024-05-27T07:41:22.006161+0000","last_active":"2024-05-27T07:46:26.856869+0000","last_peered":"2024-05-27T07:46:26.856869+0000","last_clean":"2024-05-27T07:46:26.856869+0000","last_became_active":"2024-05-27T07:41:22.005762+0000","last_became_peered":"2024-05-27T07:41:22.005762+0000","last_unstale":"2024-05-27T07:46:26.856869+0000","last_undegraded":"2024-05-27T07:46:26.856869+0000","last_fullsized":"2024-05-27T07:46:26.856869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:53.536444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014004+0000","last_change":"2024-05-27T07:41:04.774211+0000","last_active":"2024-05-27T07:46:27.014004+0000","last_peered":"2024-05-27T07:46:27.014004+0000","last_clean":"2024-05-27T07:46:27.014004+0000","last_became_active":"2024-05-27T07:41:04.773215+0000","last_became_peered":"2024-05-27T07:41:04.773215+0000","last_unstale":"2024-05-27T07:46:27.014004+0000","last_undegraded":"2024-05-27T07:46:27.014004+0000","last_fullsized":"2024-05-27T07:46:27.014004+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:37.273735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090283+0000","last_change":"2024-05-27T07:41:08.830434+0000","last_active":"2024-05-27T07:46:27.090283+0000","last_peered":"2024-05-27T07:46:27.090283+0000","last_clean":"2024-05-27T07:46:27.090283+0000","last_became_active":"2024-05-27T07:41:08.830039+0000","last_became_peered":"2024-05-27T07:41:08.830039+0000","last_unstale":"2024-05-27T07:46:27.090283+0000","last_undegraded":"2024-05-27T07:46:27.090283+0000","last_fullsized":"2024-05-27T07:46:27.090283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:27.877666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.345102+0000","last_change":"2024-05-27T07:41:59.783413+0000","last_active":"2024-05-27T07:46:27.345102+0000","last_peered":"2024-05-27T07:46:27.345102+0000","last_clean":"2024-05-27T07:46:27.345102+0000","last_became_active":"2024-05-27T07:41:59.782300+0000","last_became_peered":"2024-05-27T07:41:59.782300+0000","last_unstale":"2024-05-27T07:46:27.345102+0000","last_undegraded":"2024-05-27T07:46:27.345102+0000","last_fullsized":"2024-05-27T07:46:27.345102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:59.348928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338570+0000","last_change":"2024-05-27T07:41:04.798969+0000","last_active":"2024-05-27T07:46:27.338570+0000","last_peered":"2024-05-27T07:46:27.338570+0000","last_clean":"2024-05-27T07:46:27.338570+0000","last_became_active":"2024-05-27T07:41:04.795996+0000","last_became_peered":"2024-05-27T07:41:04.795996+0000","last_unstale":"2024-05-27T07:46:27.338570+0000","last_undegraded":"2024-05-27T07:46:27.338570+0000","last_fullsized":"2024-05-27T07:46:27.338570+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:47.141063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014169+0000","last_change":"2024-05-27T07:41:08.835142+0000","last_active":"2024-05-27T07:46:27.014169+0000","last_peered":"2024-05-27T07:46:27.014169+0000","last_clean":"2024-05-27T07:46:27.014169+0000","last_became_active":"2024-05-27T07:41:08.834762+0000","last_became_peered":"2024-05-27T07:41:08.834762+0000","last_unstale":"2024-05-27T07:46:27.014169+0000","last_undegraded":"2024-05-27T07:46:27.014169+0000","last_fullsized":"2024-05-27T07:46:27.014169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:52.957554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342357+0000","last_change":"2024-05-27T07:41:22.000649+0000","last_active":"2024-05-27T07:46:27.342357+0000","last_peered":"2024-05-27T07:46:27.342357+0000","last_clean":"2024-05-27T07:46:27.342357+0000","last_became_active":"2024-05-27T07:41:22.000161+0000","last_became_peered":"2024-05-27T07:41:22.000161+0000","last_unstale":"2024-05-27T07:46:27.342357+0000","last_undegraded":"2024-05-27T07:46:27.342357+0000","last_fullsized":"2024-05-27T07:46:27.342357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:10.788027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.858452+0000","last_change":"2024-05-27T07:41:04.800680+0000","last_active":"2024-05-27T07:46:26.858452+0000","last_peered":"2024-05-27T07:46:26.858452+0000","last_clean":"2024-05-27T07:46:26.858452+0000","last_became_active":"2024-05-27T07:41:04.800249+0000","last_became_peered":"2024-05-27T07:41:04.800249+0000","last_unstale":"2024-05-27T07:46:26.858452+0000","last_undegraded":"2024-05-27T07:46:26.858452+0000","last_fullsized":"2024-05-27T07:46:26.858452+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:23:27.797216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350163+0000","last_change":"2024-05-27T07:41:08.844380+0000","last_active":"2024-05-27T07:46:27.350163+0000","last_peered":"2024-05-27T07:46:27.350163+0000","last_clean":"2024-05-27T07:46:27.350163+0000","last_became_active":"2024-05-27T07:41:08.843464+0000","last_became_peered":"2024-05-27T07:41:08.843464+0000","last_unstale":"2024-05-27T07:46:27.350163+0000","last_undegraded":"2024-05-27T07:46:27.350163+0000","last_fullsized":"2024-05-27T07:46:27.350163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:23.290234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341019+0000","last_change":"2024-05-27T07:41:22.010968+0000","last_active":"2024-05-27T07:46:27.341019+0000","last_peered":"2024-05-27T07:46:27.341019+0000","last_clean":"2024-05-27T07:46:27.341019+0000","last_became_active":"2024-05-27T07:41:22.010529+0000","last_became_peered":"2024-05-27T07:41:22.010529+0000","last_unstale":"2024-05-27T07:46:27.341019+0000","last_undegraded":"2024-05-27T07:46:27.341019+0000","last_fullsized":"2024-05-27T07:46:27.341019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:08.636434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":299,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.650113+0000","last_change":"2024-05-27T07:41:04.891536+0000","last_active":"2024-05-27T07:46:31.650113+0000","last_peered":"2024-05-27T07:46:31.650113+0000","last_clean":"2024-05-27T07:46:31.650113+0000","last_became_active":"2024-05-27T07:41:04.890724+0000","last_became_peered":"2024-05-27T07:41:04.890724+0000","last_unstale":"2024-05-27T07:46:31.650113+0000","last_undegraded":"2024-05-27T07:46:31.650113+0000","last_fullsized":"2024-05-27T07:46:31.650113+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:32.027624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2429,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350778+0000","last_change":"2024-05-27T07:41:08.834362+0000","last_active":"2024-05-27T07:46:27.350778+0000","last_peered":"2024-05-27T07:46:27.350778+0000","last_clean":"2024-05-27T07:46:27.350778+0000","last_became_active":"2024-05-27T07:41:08.833979+0000","last_became_peered":"2024-05-27T07:41:08.833979+0000","last_unstale":"2024-05-27T07:46:27.350778+0000","last_undegraded":"2024-05-27T07:46:27.350778+0000","last_fullsized":"2024-05-27T07:46:27.350778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:35.098352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341771+0000","last_change":"2024-05-27T07:41:21.997587+0000","last_active":"2024-05-27T07:46:27.341771+0000","last_peered":"2024-05-27T07:46:27.341771+0000","last_clean":"2024-05-27T07:46:27.341771+0000","last_became_active":"2024-05-27T07:41:21.996413+0000","last_became_peered":"2024-05-27T07:41:21.996413+0000","last_unstale":"2024-05-27T07:46:27.341771+0000","last_undegraded":"2024-05-27T07:46:27.341771+0000","last_fullsized":"2024-05-27T07:46:27.341771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:21:46.728047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"92'41","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.228573+0000","last_change":"2024-05-27T07:41:04.793580+0000","last_active":"2024-05-27T07:46:27.228573+0000","last_peered":"2024-05-27T07:46:27.228573+0000","last_clean":"2024-05-27T07:46:27.228573+0000","last_became_active":"2024-05-27T07:41:04.791135+0000","last_became_peered":"2024-05-27T07:41:04.791135+0000","last_unstale":"2024-05-27T07:46:27.228573+0000","last_undegraded":"2024-05-27T07:46:27.228573+0000","last_fullsized":"2024-05-27T07:46:27.228573+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:15.484187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350999+0000","last_change":"2024-05-27T07:41:08.839187+0000","last_active":"2024-05-27T07:46:27.350999+0000","last_peered":"2024-05-27T07:46:27.350999+0000","last_clean":"2024-05-27T07:46:27.350999+0000","last_became_active":"2024-05-27T07:41:08.838821+0000","last_became_peered":"2024-05-27T07:41:08.838821+0000","last_unstale":"2024-05-27T07:46:27.350999+0000","last_undegraded":"2024-05-27T07:46:27.350999+0000","last_fullsized":"2024-05-27T07:46:27.350999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:30.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029214+0000","last_change":"2024-05-27T07:41:22.005406+0000","last_active":"2024-05-27T07:46:27.029214+0000","last_peered":"2024-05-27T07:46:27.029214+0000","last_clean":"2024-05-27T07:46:27.029214+0000","last_became_active":"2024-05-27T07:41:22.003415+0000","last_became_peered":"2024-05-27T07:41:22.003415+0000","last_unstale":"2024-05-27T07:46:27.029214+0000","last_undegraded":"2024-05-27T07:46:27.029214+0000","last_fullsized":"2024-05-27T07:46:27.029214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:03.300452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517988+0000","last_change":"2024-05-27T07:41:04.798489+0000","last_active":"2024-05-27T07:46:27.517988+0000","last_peered":"2024-05-27T07:46:27.517988+0000","last_clean":"2024-05-27T07:46:27.517988+0000","last_became_active":"2024-05-27T07:41:04.797467+0000","last_became_peered":"2024-05-27T07:41:04.797467+0000","last_unstale":"2024-05-27T07:46:27.517988+0000","last_undegraded":"2024-05-27T07:46:27.517988+0000","last_fullsized":"2024-05-27T07:46:27.517988+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:04.018577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.351043+0000","last_change":"2024-05-27T07:41:08.832215+0000","last_active":"2024-05-27T07:46:27.351043+0000","last_peered":"2024-05-27T07:46:27.351043+0000","last_clean":"2024-05-27T07:46:27.351043+0000","last_became_active":"2024-05-27T07:41:08.831491+0000","last_became_peered":"2024-05-27T07:41:08.831491+0000","last_unstale":"2024-05-27T07:46:27.351043+0000","last_undegraded":"2024-05-27T07:46:27.351043+0000","last_fullsized":"2024-05-27T07:46:27.351043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:32:03.084324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090501+0000","last_change":"2024-05-27T07:41:08.837301+0000","last_active":"2024-05-27T07:46:27.090501+0000","last_peered":"2024-05-27T07:46:27.090501+0000","last_clean":"2024-05-27T07:46:27.090501+0000","last_became_active":"2024-05-27T07:41:08.835533+0000","last_became_peered":"2024-05-27T07:41:08.835533+0000","last_unstale":"2024-05-27T07:46:27.090501+0000","last_undegraded":"2024-05-27T07:46:27.090501+0000","last_fullsized":"2024-05-27T07:46:27.090501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:45.807613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338530+0000","last_change":"2024-05-27T07:41:08.846740+0000","last_active":"2024-05-27T07:46:27.338530+0000","last_peered":"2024-05-27T07:46:27.338530+0000","last_clean":"2024-05-27T07:46:27.338530+0000","last_became_active":"2024-05-27T07:41:08.845576+0000","last_became_peered":"2024-05-27T07:41:08.845576+0000","last_unstale":"2024-05-27T07:46:27.338530+0000","last_undegraded":"2024-05-27T07:46:27.338530+0000","last_fullsized":"2024-05-27T07:46:27.338530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:41:36.944923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338096+0000","last_change":"2024-05-27T07:41:08.840121+0000","last_active":"2024-05-27T07:46:27.338096+0000","last_peered":"2024-05-27T07:46:27.338096+0000","last_clean":"2024-05-27T07:46:27.338096+0000","last_became_active":"2024-05-27T07:41:08.839090+0000","last_became_peered":"2024-05-27T07:41:08.839090+0000","last_unstale":"2024-05-27T07:46:27.338096+0000","last_undegraded":"2024-05-27T07:46:27.338096+0000","last_fullsized":"2024-05-27T07:46:27.338096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:44.836953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342097+0000","last_change":"2024-05-27T07:41:08.835565+0000","last_active":"2024-05-27T07:46:27.342097+0000","last_peered":"2024-05-27T07:46:27.342097+0000","last_clean":"2024-05-27T07:46:27.342097+0000","last_became_active":"2024-05-27T07:41:08.831424+0000","last_became_peered":"2024-05-27T07:41:08.831424+0000","last_unstale":"2024-05-27T07:46:27.342097+0000","last_undegraded":"2024-05-27T07:46:27.342097+0000","last_fullsized":"2024-05-27T07:46:27.342097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:17.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517940+0000","last_change":"2024-05-27T07:41:08.830104+0000","last_active":"2024-05-27T07:46:27.517940+0000","last_peered":"2024-05-27T07:46:27.517940+0000","last_clean":"2024-05-27T07:46:27.517940+0000","last_became_active":"2024-05-27T07:41:08.829690+0000","last_became_peered":"2024-05-27T07:41:08.829690+0000","last_unstale":"2024-05-27T07:46:27.517940+0000","last_undegraded":"2024-05-27T07:46:27.517940+0000","last_fullsized":"2024-05-27T07:46:27.517940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091546+0000","last_change":"2024-05-27T07:41:08.838654+0000","last_active":"2024-05-27T07:46:27.091546+0000","last_peered":"2024-05-27T07:46:27.091546+0000","last_clean":"2024-05-27T07:46:27.091546+0000","last_became_active":"2024-05-27T07:41:08.835033+0000","last_became_peered":"2024-05-27T07:41:08.835033+0000","last_unstale":"2024-05-27T07:46:27.091546+0000","last_undegraded":"2024-05-27T07:46:27.091546+0000","last_fullsized":"2024-05-27T07:46:27.091546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:04:24.007585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339099+0000","last_change":"2024-05-27T07:41:08.844557+0000","last_active":"2024-05-27T07:46:27.339099+0000","last_peered":"2024-05-27T07:46:27.339099+0000","last_clean":"2024-05-27T07:46:27.339099+0000","last_became_active":"2024-05-27T07:41:08.843418+0000","last_became_peered":"2024-05-27T07:41:08.843418+0000","last_unstale":"2024-05-27T07:46:27.339099+0000","last_undegraded":"2024-05-27T07:46:27.339099+0000","last_fullsized":"2024-05-27T07:46:27.339099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:39.587778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014764+0000","last_change":"2024-05-27T07:41:08.830857+0000","last_active":"2024-05-27T07:46:27.014764+0000","last_peered":"2024-05-27T07:46:27.014764+0000","last_clean":"2024-05-27T07:46:27.014764+0000","last_became_active":"2024-05-27T07:41:08.830480+0000","last_became_peered":"2024-05-27T07:41:08.830480+0000","last_unstale":"2024-05-27T07:46:27.014764+0000","last_undegraded":"2024-05-27T07:46:27.014764+0000","last_fullsized":"2024-05-27T07:46:27.014764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:07.307528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029828+0000","last_change":"2024-05-27T07:41:08.824571+0000","last_active":"2024-05-27T07:46:27.029828+0000","last_peered":"2024-05-27T07:46:27.029828+0000","last_clean":"2024-05-27T07:46:27.029828+0000","last_became_active":"2024-05-27T07:41:08.820732+0000","last_became_peered":"2024-05-27T07:41:08.820732+0000","last_unstale":"2024-05-27T07:46:27.029828+0000","last_undegraded":"2024-05-27T07:46:27.029828+0000","last_fullsized":"2024-05-27T07:46:27.029828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:55:14.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014133+0000","last_change":"2024-05-27T07:41:08.842284+0000","last_active":"2024-05-27T07:46:27.014133+0000","last_peered":"2024-05-27T07:46:27.014133+0000","last_clean":"2024-05-27T07:46:27.014133+0000","last_became_active":"2024-05-27T07:41:08.841597+0000","last_became_peered":"2024-05-27T07:41:08.841597+0000","last_unstale":"2024-05-27T07:46:27.014133+0000","last_undegraded":"2024-05-27T07:46:27.014133+0000","last_fullsized":"2024-05-27T07:46:27.014133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:25:31.066181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342151+0000","last_change":"2024-05-27T07:41:08.828935+0000","last_active":"2024-05-27T07:46:27.342151+0000","last_peered":"2024-05-27T07:46:27.342151+0000","last_clean":"2024-05-27T07:46:27.342151+0000","last_became_active":"2024-05-27T07:41:08.828542+0000","last_became_peered":"2024-05-27T07:41:08.828542+0000","last_unstale":"2024-05-27T07:46:27.342151+0000","last_undegraded":"2024-05-27T07:46:27.342151+0000","last_fullsized":"2024-05-27T07:46:27.342151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:35.270666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517888+0000","last_change":"2024-05-27T07:41:08.817051+0000","last_active":"2024-05-27T07:46:27.517888+0000","last_peered":"2024-05-27T07:46:27.517888+0000","last_clean":"2024-05-27T07:46:27.517888+0000","last_became_active":"2024-05-27T07:41:08.816418+0000","last_became_peered":"2024-05-27T07:41:08.816418+0000","last_unstale":"2024-05-27T07:46:27.517888+0000","last_undegraded":"2024-05-27T07:46:27.517888+0000","last_fullsized":"2024-05-27T07:46:27.517888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:02.760707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090720+0000","last_change":"2024-05-27T07:41:08.834081+0000","last_active":"2024-05-27T07:46:27.090720+0000","last_peered":"2024-05-27T07:46:27.090720+0000","last_clean":"2024-05-27T07:46:27.090720+0000","last_became_active":"2024-05-27T07:41:08.833679+0000","last_became_peered":"2024-05-27T07:41:08.833679+0000","last_unstale":"2024-05-27T07:46:27.090720+0000","last_undegraded":"2024-05-27T07:46:27.090720+0000","last_fullsized":"2024-05-27T07:46:27.090720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:41:14.426593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":612323,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2532,"num_read_kb":1395,"num_write":404,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2199552,"data_stored":1836969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":325,"ondisk_log_size":325,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":533518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841626,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28884,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716268,"statfs":{"total":95995035648,"available":95965458432,"internally_reserved":0,"allocated":1912832,"data_stored":1331826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005153,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1482752,"data_stored":925293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201384,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":925559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397616,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1683456,"data_stored":1132620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561142,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1064960,"data_stored":528449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1146880,"data_stored":605246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738532,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":530296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":531769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117134,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32744,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712408,"statfs":{"total":95995035648,"available":95961505792,"internally_reserved":0,"allocated":1605632,"data_stored":1052105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920903,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":608934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757375,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28800,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716352,"statfs":{"total":95995035648,"available":95965544448,"internally_reserved":0,"allocated":1826816,"data_stored":1257534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":78387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":77979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":136211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":210503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:46:35.741 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-27T07:46:35.741 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-27T07:46:36.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.068+0000 7f6783800640 1 -- 172.21.15.90:0/3810296736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f677c1065c0 msgr2=0x7f677c112990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:36.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.068+0000 7f6783800640 1 --2- 172.21.15.90:0/3810296736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f677c1065c0 0x7f677c112990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6774009ca0 tx=0x7f677402a8c0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.068+0000 7f6783800640 1 -- 172.21.15.90:0/3810296736 shutdown_connections 2024-05-27T07:46:36.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.068+0000 7f6783800640 1 --2- 172.21.15.90:0/3810296736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f677c1065c0 0x7f677c112990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.068+0000 7f6783800640 1 --2- 172.21.15.90:0/3810296736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c10c310 0x7f677c105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.068+0000 7f6783800640 1 --2- 172.21.15.90:0/3810296736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f677c10be70 0x7f677c105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.068+0000 7f6783800640 1 -- 172.21.15.90:0/3810296736 >> 172.21.15.90:0/3810296736 conn(0x7f677c1014f0 msgr2=0x7f677c103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:36.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.069+0000 7f6783800640 1 -- 172.21.15.90:0/3810296736 shutdown_connections 2024-05-27T07:46:36.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.069+0000 7f6783800640 1 -- 172.21.15.90:0/3810296736 wait complete. 2024-05-27T07:46:36.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.070+0000 7f6783800640 1 Processor -- start 2024-05-27T07:46:36.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.070+0000 7f6783800640 1 -- start start 2024-05-27T07:46:36.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.070+0000 7f6783800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1065c0 0x7f677c1a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:36.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.070+0000 7f6783800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f677c10be70 0x7f677c1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:36.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.070+0000 7f6783800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f677c10c310 0x7f677c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:36.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.070+0000 7f6783800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f677c115f80 con 0x7f677c10be70 2024-05-27T07:46:36.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.070+0000 7f6783800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f677c115e00 con 0x7f677c10c310 2024-05-27T07:46:36.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.070+0000 7f6783800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f677c116100 con 0x7f677c1065c0 2024-05-27T07:46:36.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.071+0000 7f6773e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f677c10be70 0x7f677c1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:36.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.071+0000 7f6773e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f677c10be70 0x7f677c1a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49770/0 (socket says 172.21.15.90:49770) 2024-05-27T07:46:36.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.071+0000 7f6773e00640 1 -- 172.21.15.90:0/661232217 learned_addr learned my addr 172.21.15.90:0/661232217 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:36.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.071+0000 7f6780e00640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1065c0 0x7f677c1a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:36.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.071+0000 7f6781800640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f677c10c310 0x7f677c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:36.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.071+0000 7f6773e00640 1 -- 172.21.15.90:0/661232217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1065c0 msgr2=0x7f677c1a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:36.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.071+0000 7f6773e00640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1065c0 0x7f677c1a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.071+0000 7f6773e00640 1 -- 172.21.15.90:0/661232217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f677c10c310 msgr2=0x7f677c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:36.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.071+0000 7f6773e00640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f677c10c310 0x7f677c19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.071+0000 7f6773e00640 1 -- 172.21.15.90:0/661232217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c19cea0 con 0x7f677c10be70 2024-05-27T07:46:36.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.072+0000 7f6773e00640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f677c10be70 0x7f677c1a2ad0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f676c00e9f0 tx=0x7f676c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:36.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.072+0000 7f677a000640 1 -- 172.21.15.90:0/661232217 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c00ccc0 con 0x7f677c10be70 2024-05-27T07:46:36.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.072+0000 7f677a000640 1 -- 172.21.15.90:0/661232217 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f676c0035f0 con 0x7f677c10be70 2024-05-27T07:46:36.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.072+0000 7f6783800640 1 -- 172.21.15.90:0/661232217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c19d190 con 0x7f677c10be70 2024-05-27T07:46:36.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.072+0000 7f677a000640 1 -- 172.21.15.90:0/661232217 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c00ce60 con 0x7f677c10be70 2024-05-27T07:46:36.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.072+0000 7f6783800640 1 -- 172.21.15.90:0/661232217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c1a93e0 con 0x7f677c10be70 2024-05-27T07:46:36.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.074+0000 7f6783800640 1 -- 172.21.15.90:0/661232217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c107150 con 0x7f677c10be70 2024-05-27T07:46:36.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.077+0000 7f677a000640 1 -- 172.21.15.90:0/661232217 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f676c004000 con 0x7f677c10be70 2024-05-27T07:46:36.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.078+0000 7f677a000640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f675c0b2a80 0x7f675c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:36.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.079+0000 7f677a000640 1 -- 172.21.15.90:0/661232217 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f676c0e2fe0 con 0x7f677c10be70 2024-05-27T07:46:36.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.079+0000 7f6780e00640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f675c0b2a80 0x7f675c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:36.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.079+0000 7f6780e00640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f675c0b2a80 0x7f675c0b4f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f677c19daf0 tx=0x7f6764005f90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:36.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.083+0000 7f677a000640 1 -- 172.21.15.90:0/661232217 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f676c0aa400 con 0x7f677c10be70 2024-05-27T07:46:36.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.6 KiB/s wr, 20 op/s 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: from='client.35060 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:35 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:46:36.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:46:36.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:46:36.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.6 KiB/s wr, 20 op/s 2024-05-27T07:46:36.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: from='client.35060 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:46:36.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:35 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.302+0000 7f6783800640 1 -- 172.21.15.90:0/661232217 --> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f677c063100 con 0x7f675c0b2a80 2024-05-27T07:46:36.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: Updating smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:46:36.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: Updating smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:46:36.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: Updating smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T07:46:36.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.6 KiB/s wr, 20 op/s 2024-05-27T07:46:36.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: from='client.35060 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:46:36.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:35 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:36.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.314+0000 7f677a000640 1 -- 172.21.15.90:0/661232217 <== mgr.34967 v2:172.21.15.90:6800/4256436582 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339499 (secure 0 0 0) 0x7f677c063100 con 0x7f675c0b2a80 2024-05-27T07:46:36.319 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:46:36.321 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-05-27T07:46:36.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.321+0000 7f6783800640 1 -- 172.21.15.90:0/661232217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f675c0b2a80 msgr2=0x7f675c0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:36.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.321+0000 7f6783800640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f675c0b2a80 0x7f675c0b4f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f677c19daf0 tx=0x7f6764005f90 comp rx=0 tx=0).stop 2024-05-27T07:46:36.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.321+0000 7f6783800640 1 -- 172.21.15.90:0/661232217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f677c10be70 msgr2=0x7f677c1a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:36.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.321+0000 7f6783800640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f677c10be70 0x7f677c1a2ad0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f676c00e9f0 tx=0x7f676c00eeb0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.322+0000 7f6783800640 1 -- 172.21.15.90:0/661232217 shutdown_connections 2024-05-27T07:46:36.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.322+0000 7f6783800640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f675c0b2a80 0x7f675c0b4f90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.322+0000 7f6783800640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f677c10c310 0x7f677c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.322+0000 7f6783800640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f677c10be70 0x7f677c1a2ad0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.322+0000 7f6783800640 1 --2- 172.21.15.90:0/661232217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f677c1065c0 0x7f677c1a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:36.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.322+0000 7f6783800640 1 -- 172.21.15.90:0/661232217 >> 172.21.15.90:0/661232217 conn(0x7f677c1014f0 msgr2=0x7f677c102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:36.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.322+0000 7f6783800640 1 -- 172.21.15.90:0/661232217 shutdown_connections 2024-05-27T07:46:36.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:36.322+0000 7f6783800640 1 -- 172.21.15.90:0/661232217 wait complete. 2024-05-27T07:46:36.348 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-27T07:46:34.889987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1202925,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2676,"num_read_kb":1643,"num_write":541,"num_write_kb":3647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2953452,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":926,"ondisk_log_size":926,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361224,"kb_used_data":16208,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124580600,"statfs":{"total":1151940427776,"available":1151570534400,"internally_reserved":0,"allocated":16596992,"data_stored":9963149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19138,"internal_metadata":353154366},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4084,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":211,"num_write":25,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.972610"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091553+0000","last_change":"2024-05-27T07:41:08.839265+0000","last_active":"2024-05-27T07:46:27.091553+0000","last_peered":"2024-05-27T07:46:27.091553+0000","last_clean":"2024-05-27T07:46:27.091553+0000","last_became_active":"2024-05-27T07:41:08.838814+0000","last_became_peered":"2024-05-27T07:41:08.838814+0000","last_unstale":"2024-05-27T07:46:27.091553+0000","last_undegraded":"2024-05-27T07:46:27.091553+0000","last_fullsized":"2024-05-27T07:46:27.091553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:22:32.901520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342694+0000","last_change":"2024-05-27T07:41:08.838313+0000","last_active":"2024-05-27T07:46:27.342694+0000","last_peered":"2024-05-27T07:46:27.342694+0000","last_clean":"2024-05-27T07:46:27.342694+0000","last_became_active":"2024-05-27T07:41:08.836657+0000","last_became_peered":"2024-05-27T07:41:08.836657+0000","last_unstale":"2024-05-27T07:46:27.342694+0000","last_undegraded":"2024-05-27T07:46:27.342694+0000","last_fullsized":"2024-05-27T07:46:27.342694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:33.556505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342426+0000","last_change":"2024-05-27T07:41:08.841513+0000","last_active":"2024-05-27T07:46:27.342426+0000","last_peered":"2024-05-27T07:46:27.342426+0000","last_clean":"2024-05-27T07:46:27.342426+0000","last_became_active":"2024-05-27T07:41:08.838624+0000","last_became_peered":"2024-05-27T07:41:08.838624+0000","last_unstale":"2024-05-27T07:46:27.342426+0000","last_undegraded":"2024-05-27T07:46:27.342426+0000","last_fullsized":"2024-05-27T07:46:27.342426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:01:37.320418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343623+0000","last_change":"2024-05-27T07:41:08.839138+0000","last_active":"2024-05-27T07:46:27.343623+0000","last_peered":"2024-05-27T07:46:27.343623+0000","last_clean":"2024-05-27T07:46:27.343623+0000","last_became_active":"2024-05-27T07:41:08.838766+0000","last_became_peered":"2024-05-27T07:41:08.838766+0000","last_unstale":"2024-05-27T07:46:27.343623+0000","last_undegraded":"2024-05-27T07:46:27.343623+0000","last_fullsized":"2024-05-27T07:46:27.343623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:09.129257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.858624+0000","last_change":"2024-05-27T07:41:08.823006+0000","last_active":"2024-05-27T07:46:26.858624+0000","last_peered":"2024-05-27T07:46:26.858624+0000","last_clean":"2024-05-27T07:46:26.858624+0000","last_became_active":"2024-05-27T07:41:08.822551+0000","last_became_peered":"2024-05-27T07:41:08.822551+0000","last_unstale":"2024-05-27T07:46:26.858624+0000","last_undegraded":"2024-05-27T07:46:26.858624+0000","last_fullsized":"2024-05-27T07:46:26.858624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:40:49.255763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856113+0000","last_change":"2024-05-27T07:41:08.846072+0000","last_active":"2024-05-27T07:46:26.856113+0000","last_peered":"2024-05-27T07:46:26.856113+0000","last_clean":"2024-05-27T07:46:26.856113+0000","last_became_active":"2024-05-27T07:41:08.845604+0000","last_became_peered":"2024-05-27T07:41:08.845604+0000","last_unstale":"2024-05-27T07:46:26.856113+0000","last_undegraded":"2024-05-27T07:46:26.856113+0000","last_fullsized":"2024-05-27T07:46:26.856113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:23:56.389135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350459+0000","last_change":"2024-05-27T07:41:08.841046+0000","last_active":"2024-05-27T07:46:27.350459+0000","last_peered":"2024-05-27T07:46:27.350459+0000","last_clean":"2024-05-27T07:46:27.350459+0000","last_became_active":"2024-05-27T07:41:08.840028+0000","last_became_peered":"2024-05-27T07:41:08.840028+0000","last_unstale":"2024-05-27T07:46:27.350459+0000","last_undegraded":"2024-05-27T07:46:27.350459+0000","last_fullsized":"2024-05-27T07:46:27.350459+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:52:35.239919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013947+0000","last_change":"2024-05-27T07:41:08.841182+0000","last_active":"2024-05-27T07:46:27.013947+0000","last_peered":"2024-05-27T07:46:27.013947+0000","last_clean":"2024-05-27T07:46:27.013947+0000","last_became_active":"2024-05-27T07:41:08.840645+0000","last_became_peered":"2024-05-27T07:41:08.840645+0000","last_unstale":"2024-05-27T07:46:27.013947+0000","last_undegraded":"2024-05-27T07:46:27.013947+0000","last_fullsized":"2024-05-27T07:46:27.013947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:10:59.744505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342789+0000","last_change":"2024-05-27T07:41:08.825874+0000","last_active":"2024-05-27T07:46:27.342789+0000","last_peered":"2024-05-27T07:46:27.342789+0000","last_clean":"2024-05-27T07:46:27.342789+0000","last_became_active":"2024-05-27T07:41:08.825386+0000","last_became_peered":"2024-05-27T07:41:08.825386+0000","last_unstale":"2024-05-27T07:46:27.342789+0000","last_undegraded":"2024-05-27T07:46:27.342789+0000","last_fullsized":"2024-05-27T07:46:27.342789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:48:13.304612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.857247+0000","last_change":"2024-05-27T07:41:08.847445+0000","last_active":"2024-05-27T07:46:26.857247+0000","last_peered":"2024-05-27T07:46:26.857247+0000","last_clean":"2024-05-27T07:46:26.857247+0000","last_became_active":"2024-05-27T07:41:08.846512+0000","last_became_peered":"2024-05-27T07:41:08.846512+0000","last_unstale":"2024-05-27T07:46:26.857247+0000","last_undegraded":"2024-05-27T07:46:26.857247+0000","last_fullsized":"2024-05-27T07:46:26.857247+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:22:57.235218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350533+0000","last_change":"2024-05-27T07:41:08.843531+0000","last_active":"2024-05-27T07:46:27.350533+0000","last_peered":"2024-05-27T07:46:27.350533+0000","last_clean":"2024-05-27T07:46:27.350533+0000","last_became_active":"2024-05-27T07:41:08.843214+0000","last_became_peered":"2024-05-27T07:41:08.843214+0000","last_unstale":"2024-05-27T07:46:27.350533+0000","last_undegraded":"2024-05-27T07:46:27.350533+0000","last_fullsized":"2024-05-27T07:46:27.350533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:17:00.884178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343952+0000","last_change":"2024-05-27T07:41:08.839487+0000","last_active":"2024-05-27T07:46:27.343952+0000","last_peered":"2024-05-27T07:46:27.343952+0000","last_clean":"2024-05-27T07:46:27.343952+0000","last_became_active":"2024-05-27T07:41:08.839097+0000","last_became_peered":"2024-05-27T07:41:08.839097+0000","last_unstale":"2024-05-27T07:46:27.343952+0000","last_undegraded":"2024-05-27T07:46:27.343952+0000","last_fullsized":"2024-05-27T07:46:27.343952+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:51.055482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517759+0000","last_change":"2024-05-27T07:41:08.836843+0000","last_active":"2024-05-27T07:46:27.517759+0000","last_peered":"2024-05-27T07:46:27.517759+0000","last_clean":"2024-05-27T07:46:27.517759+0000","last_became_active":"2024-05-27T07:41:08.836362+0000","last_became_peered":"2024-05-27T07:41:08.836362+0000","last_unstale":"2024-05-27T07:46:27.517759+0000","last_undegraded":"2024-05-27T07:46:27.517759+0000","last_fullsized":"2024-05-27T07:46:27.517759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:15:47.844301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341794+0000","last_change":"2024-05-27T07:41:08.839046+0000","last_active":"2024-05-27T07:46:27.341794+0000","last_peered":"2024-05-27T07:46:27.341794+0000","last_clean":"2024-05-27T07:46:27.341794+0000","last_became_active":"2024-05-27T07:41:08.838601+0000","last_became_peered":"2024-05-27T07:41:08.838601+0000","last_unstale":"2024-05-27T07:46:27.341794+0000","last_undegraded":"2024-05-27T07:46:27.341794+0000","last_fullsized":"2024-05-27T07:46:27.341794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:40:59.563490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341873+0000","last_change":"2024-05-27T07:41:08.832510+0000","last_active":"2024-05-27T07:46:27.341873+0000","last_peered":"2024-05-27T07:46:27.341873+0000","last_clean":"2024-05-27T07:46:27.341873+0000","last_became_active":"2024-05-27T07:41:08.832082+0000","last_became_peered":"2024-05-27T07:41:08.832082+0000","last_unstale":"2024-05-27T07:46:27.341873+0000","last_undegraded":"2024-05-27T07:46:27.341873+0000","last_fullsized":"2024-05-27T07:46:27.341873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:01:59.951860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339613+0000","last_change":"2024-05-27T07:41:08.827953+0000","last_active":"2024-05-27T07:46:27.339613+0000","last_peered":"2024-05-27T07:46:27.339613+0000","last_clean":"2024-05-27T07:46:27.339613+0000","last_became_active":"2024-05-27T07:41:08.827293+0000","last_became_peered":"2024-05-27T07:41:08.827293+0000","last_unstale":"2024-05-27T07:46:27.339613+0000","last_undegraded":"2024-05-27T07:46:27.339613+0000","last_fullsized":"2024-05-27T07:46:27.339613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:14:08.810514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091279+0000","last_change":"2024-05-27T07:41:08.839591+0000","last_active":"2024-05-27T07:46:27.091279+0000","last_peered":"2024-05-27T07:46:27.091279+0000","last_clean":"2024-05-27T07:46:27.091279+0000","last_became_active":"2024-05-27T07:41:08.839217+0000","last_became_peered":"2024-05-27T07:41:08.839217+0000","last_unstale":"2024-05-27T07:46:27.091279+0000","last_undegraded":"2024-05-27T07:46:27.091279+0000","last_fullsized":"2024-05-27T07:46:27.091279+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:11:41.636163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856457+0000","last_change":"2024-05-27T07:41:08.839544+0000","last_active":"2024-05-27T07:46:26.856457+0000","last_peered":"2024-05-27T07:46:26.856457+0000","last_clean":"2024-05-27T07:46:26.856457+0000","last_became_active":"2024-05-27T07:41:08.839177+0000","last_became_peered":"2024-05-27T07:41:08.839177+0000","last_unstale":"2024-05-27T07:46:26.856457+0000","last_undegraded":"2024-05-27T07:46:26.856457+0000","last_fullsized":"2024-05-27T07:46:26.856457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:22:41.921759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339195+0000","last_change":"2024-05-27T07:41:08.847911+0000","last_active":"2024-05-27T07:46:27.339195+0000","last_peered":"2024-05-27T07:46:27.339195+0000","last_clean":"2024-05-27T07:46:27.339195+0000","last_became_active":"2024-05-27T07:41:08.847427+0000","last_became_peered":"2024-05-27T07:41:08.847427+0000","last_unstale":"2024-05-27T07:46:27.339195+0000","last_undegraded":"2024-05-27T07:46:27.339195+0000","last_fullsized":"2024-05-27T07:46:27.339195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:06:13.692744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338331+0000","last_change":"2024-05-27T07:41:21.999614+0000","last_active":"2024-05-27T07:46:27.338331+0000","last_peered":"2024-05-27T07:46:27.338331+0000","last_clean":"2024-05-27T07:46:27.338331+0000","last_became_active":"2024-05-27T07:41:21.999086+0000","last_became_peered":"2024-05-27T07:41:21.999086+0000","last_unstale":"2024-05-27T07:46:27.338331+0000","last_undegraded":"2024-05-27T07:46:27.338331+0000","last_fullsized":"2024-05-27T07:46:27.338331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:20.229318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":301,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.592306+0000","last_change":"2024-05-27T07:41:04.795079+0000","last_active":"2024-05-27T07:46:31.592306+0000","last_peered":"2024-05-27T07:46:31.592306+0000","last_clean":"2024-05-27T07:46:31.592306+0000","last_became_active":"2024-05-27T07:41:04.794367+0000","last_became_peered":"2024-05-27T07:41:04.794367+0000","last_unstale":"2024-05-27T07:46:31.592306+0000","last_undegraded":"2024-05-27T07:46:31.592306+0000","last_fullsized":"2024-05-27T07:46:31.592306+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:27:52.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014433+0000","last_change":"2024-05-27T07:41:08.846712+0000","last_active":"2024-05-27T07:46:27.014433+0000","last_peered":"2024-05-27T07:46:27.014433+0000","last_clean":"2024-05-27T07:46:27.014433+0000","last_became_active":"2024-05-27T07:41:08.839999+0000","last_became_peered":"2024-05-27T07:41:08.839999+0000","last_unstale":"2024-05-27T07:46:27.014433+0000","last_undegraded":"2024-05-27T07:46:27.014433+0000","last_fullsized":"2024-05-27T07:46:27.014433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:31:23.924074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.340435+0000","last_change":"2024-05-27T07:41:21.996675+0000","last_active":"2024-05-27T07:46:27.340435+0000","last_peered":"2024-05-27T07:46:27.340435+0000","last_clean":"2024-05-27T07:46:27.340435+0000","last_became_active":"2024-05-27T07:41:21.996152+0000","last_became_peered":"2024-05-27T07:41:21.996152+0000","last_unstale":"2024-05-27T07:46:27.340435+0000","last_undegraded":"2024-05-27T07:46:27.340435+0000","last_fullsized":"2024-05-27T07:46:27.340435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:13:57.415146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014696+0000","last_change":"2024-05-27T07:41:04.890045+0000","last_active":"2024-05-27T07:46:27.014696+0000","last_peered":"2024-05-27T07:46:27.014696+0000","last_clean":"2024-05-27T07:46:27.014696+0000","last_became_active":"2024-05-27T07:41:04.888993+0000","last_became_peered":"2024-05-27T07:41:04.888993+0000","last_unstale":"2024-05-27T07:46:27.014696+0000","last_undegraded":"2024-05-27T07:46:27.014696+0000","last_fullsized":"2024-05-27T07:46:27.014696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:19.866699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342407+0000","last_change":"2024-05-27T07:41:08.847472+0000","last_active":"2024-05-27T07:46:27.342407+0000","last_peered":"2024-05-27T07:46:27.342407+0000","last_clean":"2024-05-27T07:46:27.342407+0000","last_became_active":"2024-05-27T07:41:08.846070+0000","last_became_peered":"2024-05-27T07:41:08.846070+0000","last_unstale":"2024-05-27T07:46:27.342407+0000","last_undegraded":"2024-05-27T07:46:27.342407+0000","last_fullsized":"2024-05-27T07:46:27.342407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:29:08.497007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343782+0000","last_change":"2024-05-27T07:41:59.842473+0000","last_active":"2024-05-27T07:46:27.343782+0000","last_peered":"2024-05-27T07:46:27.343782+0000","last_clean":"2024-05-27T07:46:27.343782+0000","last_became_active":"2024-05-27T07:41:59.841035+0000","last_became_peered":"2024-05-27T07:41:59.841035+0000","last_unstale":"2024-05-27T07:46:27.343782+0000","last_undegraded":"2024-05-27T07:46:27.343782+0000","last_fullsized":"2024-05-27T07:46:27.343782+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:06:52.959589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,8],"acting":[4,3,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'13","reported_seq":62,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.860220+0000","last_change":"2024-05-27T07:41:04.804046+0000","last_active":"2024-05-27T07:46:26.860220+0000","last_peered":"2024-05-27T07:46:26.860220+0000","last_clean":"2024-05-27T07:46:26.860220+0000","last_became_active":"2024-05-27T07:41:04.803360+0000","last_became_peered":"2024-05-27T07:41:04.803360+0000","last_unstale":"2024-05-27T07:46:26.860220+0000","last_undegraded":"2024-05-27T07:46:26.860220+0000","last_fullsized":"2024-05-27T07:46:26.860220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:17.325221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341836+0000","last_change":"2024-05-27T07:41:08.840842+0000","last_active":"2024-05-27T07:46:27.341836+0000","last_peered":"2024-05-27T07:46:27.341836+0000","last_clean":"2024-05-27T07:46:27.341836+0000","last_became_active":"2024-05-27T07:41:08.837817+0000","last_became_peered":"2024-05-27T07:41:08.837817+0000","last_unstale":"2024-05-27T07:46:27.341836+0000","last_undegraded":"2024-05-27T07:46:27.341836+0000","last_fullsized":"2024-05-27T07:46:27.341836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:53:38.130608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.340944+0000","last_change":"2024-05-27T07:41:22.002988+0000","last_active":"2024-05-27T07:46:27.340944+0000","last_peered":"2024-05-27T07:46:27.340944+0000","last_clean":"2024-05-27T07:46:27.340944+0000","last_became_active":"2024-05-27T07:41:22.001141+0000","last_became_peered":"2024-05-27T07:41:22.001141+0000","last_unstale":"2024-05-27T07:46:27.340944+0000","last_undegraded":"2024-05-27T07:46:27.340944+0000","last_fullsized":"2024-05-27T07:46:27.340944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:29:17.380439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.859451+0000","last_change":"2024-05-27T07:41:04.794424+0000","last_active":"2024-05-27T07:46:26.859451+0000","last_peered":"2024-05-27T07:46:26.859451+0000","last_clean":"2024-05-27T07:46:26.859451+0000","last_became_active":"2024-05-27T07:41:04.793912+0000","last_became_peered":"2024-05-27T07:41:04.793912+0000","last_unstale":"2024-05-27T07:46:26.859451+0000","last_undegraded":"2024-05-27T07:46:26.859451+0000","last_fullsized":"2024-05-27T07:46:26.859451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:32:50.876891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.031011+0000","last_change":"2024-05-27T07:41:08.832356+0000","last_active":"2024-05-27T07:46:27.031011+0000","last_peered":"2024-05-27T07:46:27.031011+0000","last_clean":"2024-05-27T07:46:27.031011+0000","last_became_active":"2024-05-27T07:41:08.831979+0000","last_became_peered":"2024-05-27T07:41:08.831979+0000","last_unstale":"2024-05-27T07:46:27.031011+0000","last_undegraded":"2024-05-27T07:46:27.031011+0000","last_fullsized":"2024-05-27T07:46:27.031011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:37:47.584177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338988+0000","last_change":"2024-05-27T07:41:22.004233+0000","last_active":"2024-05-27T07:46:27.338988+0000","last_peered":"2024-05-27T07:46:27.338988+0000","last_clean":"2024-05-27T07:46:27.338988+0000","last_became_active":"2024-05-27T07:41:22.003781+0000","last_became_peered":"2024-05-27T07:41:22.003781+0000","last_unstale":"2024-05-27T07:46:27.338988+0000","last_undegraded":"2024-05-27T07:46:27.338988+0000","last_fullsized":"2024-05-27T07:46:27.338988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:02:13.634460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'10","reported_seq":303,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.671516+0000","last_change":"2024-05-27T07:41:04.800373+0000","last_active":"2024-05-27T07:46:31.671516+0000","last_peered":"2024-05-27T07:46:31.671516+0000","last_clean":"2024-05-27T07:46:31.671516+0000","last_became_active":"2024-05-27T07:41:04.799930+0000","last_became_peered":"2024-05-27T07:41:04.799930+0000","last_unstale":"2024-05-27T07:46:31.671516+0000","last_undegraded":"2024-05-27T07:46:31.671516+0000","last_fullsized":"2024-05-27T07:46:31.671516+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:56:24.642479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2413,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":245,"num_read_kb":7,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.857099+0000","last_change":"2024-05-27T07:41:08.830387+0000","last_active":"2024-05-27T07:46:26.857099+0000","last_peered":"2024-05-27T07:46:26.857099+0000","last_clean":"2024-05-27T07:46:26.857099+0000","last_became_active":"2024-05-27T07:41:08.829720+0000","last_became_peered":"2024-05-27T07:41:08.829720+0000","last_unstale":"2024-05-27T07:46:26.857099+0000","last_undegraded":"2024-05-27T07:46:26.857099+0000","last_fullsized":"2024-05-27T07:46:26.857099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:42.014383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338425+0000","last_change":"2024-05-27T07:41:22.005700+0000","last_active":"2024-05-27T07:46:27.338425+0000","last_peered":"2024-05-27T07:46:27.338425+0000","last_clean":"2024-05-27T07:46:27.338425+0000","last_became_active":"2024-05-27T07:41:22.003643+0000","last_became_peered":"2024-05-27T07:41:22.003643+0000","last_unstale":"2024-05-27T07:46:27.338425+0000","last_undegraded":"2024-05-27T07:46:27.338425+0000","last_fullsized":"2024-05-27T07:46:27.338425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:40:48.332767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.855954+0000","last_change":"2024-05-27T07:41:59.790579+0000","last_active":"2024-05-27T07:46:26.855954+0000","last_peered":"2024-05-27T07:46:26.855954+0000","last_clean":"2024-05-27T07:46:26.855954+0000","last_became_active":"2024-05-27T07:41:59.789849+0000","last_became_peered":"2024-05-27T07:41:59.789849+0000","last_unstale":"2024-05-27T07:46:26.855954+0000","last_undegraded":"2024-05-27T07:46:26.855954+0000","last_fullsized":"2024-05-27T07:46:26.855954+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:24:30.564074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.851628+0000","last_change":"2024-05-27T07:43:00.121063+0000","last_active":"2024-05-27T07:46:26.851628+0000","last_peered":"2024-05-27T07:46:26.851628+0000","last_clean":"2024-05-27T07:46:26.851628+0000","last_became_active":"2024-05-27T07:43:00.119977+0000","last_became_peered":"2024-05-27T07:43:00.119977+0000","last_unstale":"2024-05-27T07:46:26.851628+0000","last_undegraded":"2024-05-27T07:46:26.851628+0000","last_fullsized":"2024-05-27T07:46:26.851628+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:04:43.139899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091113+0000","last_change":"2024-05-27T07:41:22.013820+0000","last_active":"2024-05-27T07:46:27.091113+0000","last_peered":"2024-05-27T07:46:27.091113+0000","last_clean":"2024-05-27T07:46:27.091113+0000","last_became_active":"2024-05-27T07:41:22.013110+0000","last_became_peered":"2024-05-27T07:41:22.013110+0000","last_unstale":"2024-05-27T07:46:27.091113+0000","last_undegraded":"2024-05-27T07:46:27.091113+0000","last_fullsized":"2024-05-27T07:46:27.091113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:45:07.317872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517879+0000","last_change":"2024-05-27T07:41:04.789197+0000","last_active":"2024-05-27T07:46:27.517879+0000","last_peered":"2024-05-27T07:46:27.517879+0000","last_clean":"2024-05-27T07:46:27.517879+0000","last_became_active":"2024-05-27T07:41:04.788755+0000","last_became_peered":"2024-05-27T07:41:04.788755+0000","last_unstale":"2024-05-27T07:46:27.517879+0000","last_undegraded":"2024-05-27T07:46:27.517879+0000","last_fullsized":"2024-05-27T07:46:27.517879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:27:54.526910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013815+0000","last_change":"2024-05-27T07:41:08.841769+0000","last_active":"2024-05-27T07:46:27.013815+0000","last_peered":"2024-05-27T07:46:27.013815+0000","last_clean":"2024-05-27T07:46:27.013815+0000","last_became_active":"2024-05-27T07:41:08.840834+0000","last_became_peered":"2024-05-27T07:41:08.840834+0000","last_unstale":"2024-05-27T07:46:27.013815+0000","last_undegraded":"2024-05-27T07:46:27.013815+0000","last_fullsized":"2024-05-27T07:46:27.013815+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:15.573739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342674+0000","last_change":"2024-05-27T07:41:22.005119+0000","last_active":"2024-05-27T07:46:27.342674+0000","last_peered":"2024-05-27T07:46:27.342674+0000","last_clean":"2024-05-27T07:46:27.342674+0000","last_became_active":"2024-05-27T07:41:22.002638+0000","last_became_peered":"2024-05-27T07:41:22.002638+0000","last_unstale":"2024-05-27T07:46:27.342674+0000","last_undegraded":"2024-05-27T07:46:27.342674+0000","last_fullsized":"2024-05-27T07:46:27.342674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:08:49.877955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029700+0000","last_change":"2024-05-27T07:41:04.799864+0000","last_active":"2024-05-27T07:46:27.029700+0000","last_peered":"2024-05-27T07:46:27.029700+0000","last_clean":"2024-05-27T07:46:27.029700+0000","last_became_active":"2024-05-27T07:41:04.799218+0000","last_became_peered":"2024-05-27T07:41:04.799218+0000","last_unstale":"2024-05-27T07:46:27.029700+0000","last_undegraded":"2024-05-27T07:46:27.029700+0000","last_fullsized":"2024-05-27T07:46:27.029700+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:15.355720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.851911+0000","last_change":"2024-05-27T07:41:08.823387+0000","last_active":"2024-05-27T07:46:26.851911+0000","last_peered":"2024-05-27T07:46:26.851911+0000","last_clean":"2024-05-27T07:46:26.851911+0000","last_became_active":"2024-05-27T07:41:08.822943+0000","last_became_peered":"2024-05-27T07:41:08.822943+0000","last_unstale":"2024-05-27T07:46:26.851911+0000","last_undegraded":"2024-05-27T07:46:26.851911+0000","last_fullsized":"2024-05-27T07:46:26.851911+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:37:46.365284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013958+0000","last_change":"2024-05-27T07:41:22.011848+0000","last_active":"2024-05-27T07:46:27.013958+0000","last_peered":"2024-05-27T07:46:27.013958+0000","last_clean":"2024-05-27T07:46:27.013958+0000","last_became_active":"2024-05-27T07:41:22.011335+0000","last_became_peered":"2024-05-27T07:41:22.011335+0000","last_unstale":"2024-05-27T07:46:27.013958+0000","last_undegraded":"2024-05-27T07:46:27.013958+0000","last_fullsized":"2024-05-27T07:46:27.013958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:29:22.528270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'7","reported_seq":531,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:32.651357+0000","last_change":"2024-05-27T07:41:04.793877+0000","last_active":"2024-05-27T07:46:32.651357+0000","last_peered":"2024-05-27T07:46:32.651357+0000","last_clean":"2024-05-27T07:46:32.651357+0000","last_became_active":"2024-05-27T07:41:04.793389+0000","last_became_peered":"2024-05-27T07:41:04.793389+0000","last_unstale":"2024-05-27T07:46:32.651357+0000","last_undegraded":"2024-05-27T07:46:32.651357+0000","last_fullsized":"2024-05-27T07:46:32.651357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:41:48.670572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":476,"num_read_kb":476,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342653+0000","last_change":"2024-05-27T07:41:08.836682+0000","last_active":"2024-05-27T07:46:27.342653+0000","last_peered":"2024-05-27T07:46:27.342653+0000","last_clean":"2024-05-27T07:46:27.342653+0000","last_became_active":"2024-05-27T07:41:08.836229+0000","last_became_peered":"2024-05-27T07:41:08.836229+0000","last_unstale":"2024-05-27T07:46:27.342653+0000","last_undegraded":"2024-05-27T07:46:27.342653+0000","last_fullsized":"2024-05-27T07:46:27.342653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:11:33.249584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.850945+0000","last_change":"2024-05-27T07:41:22.004968+0000","last_active":"2024-05-27T07:46:26.850945+0000","last_peered":"2024-05-27T07:46:26.850945+0000","last_clean":"2024-05-27T07:46:26.850945+0000","last_became_active":"2024-05-27T07:41:22.001407+0000","last_became_peered":"2024-05-27T07:41:22.001407+0000","last_unstale":"2024-05-27T07:46:26.850945+0000","last_undegraded":"2024-05-27T07:46:26.850945+0000","last_fullsized":"2024-05-27T07:46:26.850945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:02:26.872497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091231+0000","last_change":"2024-05-27T07:41:59.787247+0000","last_active":"2024-05-27T07:46:27.091231+0000","last_peered":"2024-05-27T07:46:27.091231+0000","last_clean":"2024-05-27T07:46:27.091231+0000","last_became_active":"2024-05-27T07:41:59.786392+0000","last_became_peered":"2024-05-27T07:41:59.786392+0000","last_unstale":"2024-05-27T07:46:27.091231+0000","last_undegraded":"2024-05-27T07:46:27.091231+0000","last_fullsized":"2024-05-27T07:46:27.091231+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:12.579707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2],"acting":[11,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342877+0000","last_change":"2024-05-27T07:41:08.833731+0000","last_active":"2024-05-27T07:46:27.342877+0000","last_peered":"2024-05-27T07:46:27.342877+0000","last_clean":"2024-05-27T07:46:27.342877+0000","last_became_active":"2024-05-27T07:41:08.833318+0000","last_became_peered":"2024-05-27T07:41:08.833318+0000","last_unstale":"2024-05-27T07:46:27.342877+0000","last_undegraded":"2024-05-27T07:46:27.342877+0000","last_fullsized":"2024-05-27T07:46:27.342877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:51:50.085113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343580+0000","last_change":"2024-05-27T07:41:59.789784+0000","last_active":"2024-05-27T07:46:27.343580+0000","last_peered":"2024-05-27T07:46:27.343580+0000","last_clean":"2024-05-27T07:46:27.343580+0000","last_became_active":"2024-05-27T07:41:59.788589+0000","last_became_peered":"2024-05-27T07:41:59.788589+0000","last_unstale":"2024-05-27T07:46:27.343580+0000","last_undegraded":"2024-05-27T07:46:27.343580+0000","last_fullsized":"2024-05-27T07:46:27.343580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:34:37.026907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030707+0000","last_change":"2024-05-27T07:41:04.892603+0000","last_active":"2024-05-27T07:46:27.030707+0000","last_peered":"2024-05-27T07:46:27.030707+0000","last_clean":"2024-05-27T07:46:27.030707+0000","last_became_active":"2024-05-27T07:41:04.890786+0000","last_became_peered":"2024-05-27T07:41:04.890786+0000","last_unstale":"2024-05-27T07:46:27.030707+0000","last_undegraded":"2024-05-27T07:46:27.030707+0000","last_fullsized":"2024-05-27T07:46:27.030707+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:12:34.129227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342154+0000","last_change":"2024-05-27T07:41:08.830802+0000","last_active":"2024-05-27T07:46:27.342154+0000","last_peered":"2024-05-27T07:46:27.342154+0000","last_clean":"2024-05-27T07:46:27.342154+0000","last_became_active":"2024-05-27T07:41:08.829661+0000","last_became_peered":"2024-05-27T07:41:08.829661+0000","last_unstale":"2024-05-27T07:46:27.342154+0000","last_undegraded":"2024-05-27T07:46:27.342154+0000","last_fullsized":"2024-05-27T07:46:27.342154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:20:15.041554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.852318+0000","last_change":"2024-05-27T07:41:22.003997+0000","last_active":"2024-05-27T07:46:26.852318+0000","last_peered":"2024-05-27T07:46:26.852318+0000","last_clean":"2024-05-27T07:46:26.852318+0000","last_became_active":"2024-05-27T07:41:22.003527+0000","last_became_peered":"2024-05-27T07:41:22.003527+0000","last_unstale":"2024-05-27T07:46:26.852318+0000","last_undegraded":"2024-05-27T07:46:26.852318+0000","last_fullsized":"2024-05-27T07:46:26.852318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:05:17.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'47","reported_seq":603,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:32.704711+0000","last_change":"2024-05-27T07:41:59.814227+0000","last_active":"2024-05-27T07:46:32.704711+0000","last_peered":"2024-05-27T07:46:32.704711+0000","last_clean":"2024-05-27T07:46:32.704711+0000","last_became_active":"2024-05-27T07:41:59.788510+0000","last_became_peered":"2024-05-27T07:41:59.788510+0000","last_unstale":"2024-05-27T07:46:32.704711+0000","last_undegraded":"2024-05-27T07:46:32.704711+0000","last_fullsized":"2024-05-27T07:46:32.704711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:09:15.868348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75767,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":320,"num_write":49,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.859696+0000","last_change":"2024-05-27T07:41:08.831639+0000","last_active":"2024-05-27T07:46:26.859696+0000","last_peered":"2024-05-27T07:46:26.859696+0000","last_clean":"2024-05-27T07:46:26.859696+0000","last_became_active":"2024-05-27T07:41:08.830541+0000","last_became_peered":"2024-05-27T07:41:08.830541+0000","last_unstale":"2024-05-27T07:46:26.859696+0000","last_undegraded":"2024-05-27T07:46:26.859696+0000","last_fullsized":"2024-05-27T07:46:26.859696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:45.635145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.031603+0000","last_change":"2024-05-27T07:41:21.999587+0000","last_active":"2024-05-27T07:46:27.031603+0000","last_peered":"2024-05-27T07:46:27.031603+0000","last_clean":"2024-05-27T07:46:27.031603+0000","last_became_active":"2024-05-27T07:41:21.999017+0000","last_became_peered":"2024-05-27T07:41:21.999017+0000","last_unstale":"2024-05-27T07:46:27.031603+0000","last_undegraded":"2024-05-27T07:46:27.031603+0000","last_fullsized":"2024-05-27T07:46:27.031603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:31:14.915229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338645+0000","last_change":"2024-05-27T07:41:04.799847+0000","last_active":"2024-05-27T07:46:27.338645+0000","last_peered":"2024-05-27T07:46:27.338645+0000","last_clean":"2024-05-27T07:46:27.338645+0000","last_became_active":"2024-05-27T07:41:04.798605+0000","last_became_peered":"2024-05-27T07:41:04.798605+0000","last_unstale":"2024-05-27T07:46:27.338645+0000","last_undegraded":"2024-05-27T07:46:27.338645+0000","last_fullsized":"2024-05-27T07:46:27.338645+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:01.957707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517549+0000","last_change":"2024-05-27T07:41:08.831138+0000","last_active":"2024-05-27T07:46:27.517549+0000","last_peered":"2024-05-27T07:46:27.517549+0000","last_clean":"2024-05-27T07:46:27.517549+0000","last_became_active":"2024-05-27T07:41:08.830735+0000","last_became_peered":"2024-05-27T07:41:08.830735+0000","last_unstale":"2024-05-27T07:46:27.517549+0000","last_undegraded":"2024-05-27T07:46:27.517549+0000","last_fullsized":"2024-05-27T07:46:27.517549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:12.569687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090504+0000","last_change":"2024-05-27T07:41:59.779039+0000","last_active":"2024-05-27T07:46:27.090504+0000","last_peered":"2024-05-27T07:46:27.090504+0000","last_clean":"2024-05-27T07:46:27.090504+0000","last_became_active":"2024-05-27T07:41:59.777514+0000","last_became_peered":"2024-05-27T07:41:59.777514+0000","last_unstale":"2024-05-27T07:46:27.090504+0000","last_undegraded":"2024-05-27T07:46:27.090504+0000","last_fullsized":"2024-05-27T07:46:27.090504+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:12:19.223874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030345+0000","last_change":"2024-05-27T07:41:04.796467+0000","last_active":"2024-05-27T07:46:27.030345+0000","last_peered":"2024-05-27T07:46:27.030345+0000","last_clean":"2024-05-27T07:46:27.030345+0000","last_became_active":"2024-05-27T07:41:04.795519+0000","last_became_peered":"2024-05-27T07:41:04.795519+0000","last_unstale":"2024-05-27T07:46:27.030345+0000","last_undegraded":"2024-05-27T07:46:27.030345+0000","last_fullsized":"2024-05-27T07:46:27.030345+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:55:42.744076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350149+0000","last_change":"2024-05-27T07:41:08.832135+0000","last_active":"2024-05-27T07:46:27.350149+0000","last_peered":"2024-05-27T07:46:27.350149+0000","last_clean":"2024-05-27T07:46:27.350149+0000","last_became_active":"2024-05-27T07:41:08.830032+0000","last_became_peered":"2024-05-27T07:41:08.830032+0000","last_unstale":"2024-05-27T07:46:27.350149+0000","last_undegraded":"2024-05-27T07:46:27.350149+0000","last_fullsized":"2024-05-27T07:46:27.350149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:44:59.367026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856691+0000","last_change":"2024-05-27T07:41:22.008868+0000","last_active":"2024-05-27T07:46:26.856691+0000","last_peered":"2024-05-27T07:46:26.856691+0000","last_clean":"2024-05-27T07:46:26.856691+0000","last_became_active":"2024-05-27T07:41:22.002071+0000","last_became_peered":"2024-05-27T07:41:22.002071+0000","last_unstale":"2024-05-27T07:46:26.856691+0000","last_undegraded":"2024-05-27T07:46:26.856691+0000","last_fullsized":"2024-05-27T07:46:26.856691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:57:20.488695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'9","reported_seq":301,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.706139+0000","last_change":"2024-05-27T07:41:04.796198+0000","last_active":"2024-05-27T07:46:31.706139+0000","last_peered":"2024-05-27T07:46:31.706139+0000","last_clean":"2024-05-27T07:46:31.706139+0000","last_became_active":"2024-05-27T07:41:04.795646+0000","last_became_peered":"2024-05-27T07:41:04.795646+0000","last_unstale":"2024-05-27T07:46:31.706139+0000","last_undegraded":"2024-05-27T07:46:31.706139+0000","last_fullsized":"2024-05-27T07:46:31.706139+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:05:30.936596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1988,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":245,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350240+0000","last_change":"2024-05-27T07:41:08.840518+0000","last_active":"2024-05-27T07:46:27.350240+0000","last_peered":"2024-05-27T07:46:27.350240+0000","last_clean":"2024-05-27T07:46:27.350240+0000","last_became_active":"2024-05-27T07:41:08.840162+0000","last_became_peered":"2024-05-27T07:41:08.840162+0000","last_unstale":"2024-05-27T07:46:27.350240+0000","last_undegraded":"2024-05-27T07:46:27.350240+0000","last_fullsized":"2024-05-27T07:46:27.350240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:04:14.657496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.518023+0000","last_change":"2024-05-27T07:41:22.000243+0000","last_active":"2024-05-27T07:46:27.518023+0000","last_peered":"2024-05-27T07:46:27.518023+0000","last_clean":"2024-05-27T07:46:27.518023+0000","last_became_active":"2024-05-27T07:41:21.998898+0000","last_became_peered":"2024-05-27T07:41:21.998898+0000","last_unstale":"2024-05-27T07:46:27.518023+0000","last_undegraded":"2024-05-27T07:46:27.518023+0000","last_fullsized":"2024-05-27T07:46:27.518023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:02:35.696593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":82,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856741+0000","last_change":"2024-05-27T07:41:59.799662+0000","last_active":"2024-05-27T07:46:26.856741+0000","last_peered":"2024-05-27T07:46:26.856741+0000","last_clean":"2024-05-27T07:46:26.856741+0000","last_became_active":"2024-05-27T07:41:59.779733+0000","last_became_peered":"2024-05-27T07:41:59.779733+0000","last_unstale":"2024-05-27T07:46:26.856741+0000","last_undegraded":"2024-05-27T07:46:26.856741+0000","last_fullsized":"2024-05-27T07:46:26.856741+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:39:23.665014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030069+0000","last_change":"2024-05-27T07:41:08.834931+0000","last_active":"2024-05-27T07:46:27.030069+0000","last_peered":"2024-05-27T07:46:27.030069+0000","last_clean":"2024-05-27T07:46:27.030069+0000","last_became_active":"2024-05-27T07:41:08.833298+0000","last_became_peered":"2024-05-27T07:41:08.833298+0000","last_unstale":"2024-05-27T07:46:27.030069+0000","last_undegraded":"2024-05-27T07:46:27.030069+0000","last_fullsized":"2024-05-27T07:46:27.030069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:03:48.507461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029888+0000","last_change":"2024-05-27T07:41:22.009407+0000","last_active":"2024-05-27T07:46:27.029888+0000","last_peered":"2024-05-27T07:46:27.029888+0000","last_clean":"2024-05-27T07:46:27.029888+0000","last_became_active":"2024-05-27T07:41:22.006921+0000","last_became_peered":"2024-05-27T07:41:22.006921+0000","last_unstale":"2024-05-27T07:46:27.029888+0000","last_undegraded":"2024-05-27T07:46:27.029888+0000","last_fullsized":"2024-05-27T07:46:27.029888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:04:24.034502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.860585+0000","last_change":"2024-05-27T07:41:04.795641+0000","last_active":"2024-05-27T07:46:26.860585+0000","last_peered":"2024-05-27T07:46:26.860585+0000","last_clean":"2024-05-27T07:46:26.860585+0000","last_became_active":"2024-05-27T07:41:04.794935+0000","last_became_peered":"2024-05-27T07:41:04.794935+0000","last_unstale":"2024-05-27T07:46:26.860585+0000","last_undegraded":"2024-05-27T07:46:26.860585+0000","last_fullsized":"2024-05-27T07:46:26.860585+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:15:14.850282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091428+0000","last_change":"2024-05-27T07:41:08.837277+0000","last_active":"2024-05-27T07:46:27.091428+0000","last_peered":"2024-05-27T07:46:27.091428+0000","last_clean":"2024-05-27T07:46:27.091428+0000","last_became_active":"2024-05-27T07:41:08.835935+0000","last_became_peered":"2024-05-27T07:41:08.835935+0000","last_unstale":"2024-05-27T07:46:27.091428+0000","last_undegraded":"2024-05-27T07:46:27.091428+0000","last_fullsized":"2024-05-27T07:46:27.091428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:43:46.436243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856802+0000","last_change":"2024-05-27T07:41:22.014700+0000","last_active":"2024-05-27T07:46:26.856802+0000","last_peered":"2024-05-27T07:46:26.856802+0000","last_clean":"2024-05-27T07:46:26.856802+0000","last_became_active":"2024-05-27T07:41:22.014136+0000","last_became_peered":"2024-05-27T07:41:22.014136+0000","last_unstale":"2024-05-27T07:46:26.856802+0000","last_undegraded":"2024-05-27T07:46:26.856802+0000","last_fullsized":"2024-05-27T07:46:26.856802+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:51:08.619344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.855048+0000","last_change":"2024-05-27T07:41:04.891269+0000","last_active":"2024-05-27T07:46:26.855048+0000","last_peered":"2024-05-27T07:46:26.855048+0000","last_clean":"2024-05-27T07:46:26.855048+0000","last_became_active":"2024-05-27T07:41:04.890843+0000","last_became_peered":"2024-05-27T07:41:04.890843+0000","last_unstale":"2024-05-27T07:46:26.855048+0000","last_undegraded":"2024-05-27T07:46:26.855048+0000","last_fullsized":"2024-05-27T07:46:26.855048+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:05:57.065300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856313+0000","last_change":"2024-05-27T07:41:08.841977+0000","last_active":"2024-05-27T07:46:26.856313+0000","last_peered":"2024-05-27T07:46:26.856313+0000","last_clean":"2024-05-27T07:46:26.856313+0000","last_became_active":"2024-05-27T07:41:08.841551+0000","last_became_peered":"2024-05-27T07:41:08.841551+0000","last_unstale":"2024-05-27T07:46:26.856313+0000","last_undegraded":"2024-05-27T07:46:26.856313+0000","last_fullsized":"2024-05-27T07:46:26.856313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:35:07.766193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517708+0000","last_change":"2024-05-27T07:41:22.006310+0000","last_active":"2024-05-27T07:46:27.517708+0000","last_peered":"2024-05-27T07:46:27.517708+0000","last_clean":"2024-05-27T07:46:27.517708+0000","last_became_active":"2024-05-27T07:41:22.004135+0000","last_became_peered":"2024-05-27T07:41:22.004135+0000","last_unstale":"2024-05-27T07:46:27.517708+0000","last_undegraded":"2024-05-27T07:46:27.517708+0000","last_fullsized":"2024-05-27T07:46:27.517708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:45:48.564894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'7","reported_seq":296,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.593482+0000","last_change":"2024-05-27T07:41:04.793284+0000","last_active":"2024-05-27T07:46:31.593482+0000","last_peered":"2024-05-27T07:46:31.593482+0000","last_clean":"2024-05-27T07:46:31.593482+0000","last_became_active":"2024-05-27T07:41:04.792853+0000","last_became_peered":"2024-05-27T07:41:04.792853+0000","last_unstale":"2024-05-27T07:46:31.593482+0000","last_undegraded":"2024-05-27T07:46:31.593482+0000","last_fullsized":"2024-05-27T07:46:31.593482+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:52:22.979341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1456,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":241,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856334+0000","last_change":"2024-05-27T07:41:59.776207+0000","last_active":"2024-05-27T07:46:26.856334+0000","last_peered":"2024-05-27T07:46:26.856334+0000","last_clean":"2024-05-27T07:46:26.856334+0000","last_became_active":"2024-05-27T07:41:59.775226+0000","last_became_peered":"2024-05-27T07:41:59.775226+0000","last_unstale":"2024-05-27T07:46:26.856334+0000","last_undegraded":"2024-05-27T07:46:26.856334+0000","last_fullsized":"2024-05-27T07:46:26.856334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:33:50.469144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517906+0000","last_change":"2024-05-27T07:41:21.992013+0000","last_active":"2024-05-27T07:46:27.517906+0000","last_peered":"2024-05-27T07:46:27.517906+0000","last_clean":"2024-05-27T07:46:27.517906+0000","last_became_active":"2024-05-27T07:41:21.991550+0000","last_became_peered":"2024-05-27T07:41:21.991550+0000","last_unstale":"2024-05-27T07:46:27.517906+0000","last_undegraded":"2024-05-27T07:46:27.517906+0000","last_fullsized":"2024-05-27T07:46:27.517906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:34:53.554107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.031130+0000","last_change":"2024-05-27T07:41:08.832512+0000","last_active":"2024-05-27T07:46:27.031130+0000","last_peered":"2024-05-27T07:46:27.031130+0000","last_clean":"2024-05-27T07:46:27.031130+0000","last_became_active":"2024-05-27T07:41:08.832113+0000","last_became_peered":"2024-05-27T07:41:08.832113+0000","last_unstale":"2024-05-27T07:46:27.031130+0000","last_undegraded":"2024-05-27T07:46:27.031130+0000","last_fullsized":"2024-05-27T07:46:27.031130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:50:00.530785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'7","reported_seq":294,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.705393+0000","last_change":"2024-05-27T07:41:04.891004+0000","last_active":"2024-05-27T07:46:31.705393+0000","last_peered":"2024-05-27T07:46:31.705393+0000","last_clean":"2024-05-27T07:46:31.705393+0000","last_became_active":"2024-05-27T07:41:04.890567+0000","last_became_peered":"2024-05-27T07:41:04.890567+0000","last_unstale":"2024-05-27T07:46:31.705393+0000","last_undegraded":"2024-05-27T07:46:31.705393+0000","last_fullsized":"2024-05-27T07:46:31.705393+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:04:43.991204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350297+0000","last_change":"2024-05-27T07:41:22.003446+0000","last_active":"2024-05-27T07:46:27.350297+0000","last_peered":"2024-05-27T07:46:27.350297+0000","last_clean":"2024-05-27T07:46:27.350297+0000","last_became_active":"2024-05-27T07:41:22.002989+0000","last_became_peered":"2024-05-27T07:41:22.002989+0000","last_unstale":"2024-05-27T07:46:27.350297+0000","last_undegraded":"2024-05-27T07:46:27.350297+0000","last_fullsized":"2024-05-27T07:46:27.350297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:12.996286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013957+0000","last_change":"2024-05-27T07:41:08.842685+0000","last_active":"2024-05-27T07:46:27.013957+0000","last_peered":"2024-05-27T07:46:27.013957+0000","last_clean":"2024-05-27T07:46:27.013957+0000","last_became_active":"2024-05-27T07:41:08.841954+0000","last_became_peered":"2024-05-27T07:41:08.841954+0000","last_unstale":"2024-05-27T07:46:27.013957+0000","last_undegraded":"2024-05-27T07:46:27.013957+0000","last_fullsized":"2024-05-27T07:46:27.013957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:18.087129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342500+0000","last_change":"2024-05-27T07:41:04.790330+0000","last_active":"2024-05-27T07:46:27.342500+0000","last_peered":"2024-05-27T07:46:27.342500+0000","last_clean":"2024-05-27T07:46:27.342500+0000","last_became_active":"2024-05-27T07:41:04.789508+0000","last_became_peered":"2024-05-27T07:41:04.789508+0000","last_unstale":"2024-05-27T07:46:27.342500+0000","last_undegraded":"2024-05-27T07:46:27.342500+0000","last_fullsized":"2024-05-27T07:46:27.342500+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:27.537662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'99","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:29.032290+0000","last_change":"2024-05-27T07:39:11.797324+0000","last_active":"2024-05-27T07:46:29.032290+0000","last_peered":"2024-05-27T07:46:29.032290+0000","last_clean":"2024-05-27T07:46:29.032290+0000","last_became_active":"2024-05-27T07:39:11.786294+0000","last_became_peered":"2024-05-27T07:39:11.786294+0000","last_unstale":"2024-05-27T07:46:29.032290+0000","last_undegraded":"2024-05-27T07:46:29.032290+0000","last_fullsized":"2024-05-27T07:46:29.032290+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:34:18.613413+0000","last_clean_scrub_stamp":"2024-05-27T07:34:18.613413+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:09:23.947615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.030810+0000","last_change":"2024-05-27T07:41:21.997453+0000","last_active":"2024-05-27T07:46:27.030810+0000","last_peered":"2024-05-27T07:46:27.030810+0000","last_clean":"2024-05-27T07:46:27.030810+0000","last_became_active":"2024-05-27T07:41:21.997010+0000","last_became_peered":"2024-05-27T07:41:21.997010+0000","last_unstale":"2024-05-27T07:46:27.030810+0000","last_undegraded":"2024-05-27T07:46:27.030810+0000","last_fullsized":"2024-05-27T07:46:27.030810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:54:54.913274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517351+0000","last_change":"2024-05-27T07:43:00.024988+0000","last_active":"2024-05-27T07:46:27.517351+0000","last_peered":"2024-05-27T07:46:27.517351+0000","last_clean":"2024-05-27T07:46:27.517351+0000","last_became_active":"2024-05-27T07:43:00.023816+0000","last_became_peered":"2024-05-27T07:43:00.023816+0000","last_unstale":"2024-05-27T07:46:27.517351+0000","last_undegraded":"2024-05-27T07:46:27.517351+0000","last_fullsized":"2024-05-27T07:46:27.517351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:14:58.531985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.349949+0000","last_change":"2024-05-27T07:41:04.795262+0000","last_active":"2024-05-27T07:46:27.349949+0000","last_peered":"2024-05-27T07:46:27.349949+0000","last_clean":"2024-05-27T07:46:27.349949+0000","last_became_active":"2024-05-27T07:41:04.794866+0000","last_became_peered":"2024-05-27T07:41:04.794866+0000","last_unstale":"2024-05-27T07:46:27.349949+0000","last_undegraded":"2024-05-27T07:46:27.349949+0000","last_fullsized":"2024-05-27T07:46:27.349949+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:07:44.241444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090350+0000","last_change":"2024-05-27T07:41:22.006194+0000","last_active":"2024-05-27T07:46:27.090350+0000","last_peered":"2024-05-27T07:46:27.090350+0000","last_clean":"2024-05-27T07:46:27.090350+0000","last_became_active":"2024-05-27T07:41:22.004079+0000","last_became_peered":"2024-05-27T07:41:22.004079+0000","last_unstale":"2024-05-27T07:46:27.090350+0000","last_undegraded":"2024-05-27T07:46:27.090350+0000","last_fullsized":"2024-05-27T07:46:27.090350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:09:03.496722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091032+0000","last_change":"2024-05-27T07:41:59.776734+0000","last_active":"2024-05-27T07:46:27.091032+0000","last_peered":"2024-05-27T07:46:27.091032+0000","last_clean":"2024-05-27T07:46:27.091032+0000","last_became_active":"2024-05-27T07:41:59.775678+0000","last_became_peered":"2024-05-27T07:41:59.775678+0000","last_unstale":"2024-05-27T07:46:27.091032+0000","last_undegraded":"2024-05-27T07:46:27.091032+0000","last_fullsized":"2024-05-27T07:46:27.091032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:41:56.007178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517918+0000","last_change":"2024-05-27T07:41:08.831070+0000","last_active":"2024-05-27T07:46:27.517918+0000","last_peered":"2024-05-27T07:46:27.517918+0000","last_clean":"2024-05-27T07:46:27.517918+0000","last_became_active":"2024-05-27T07:41:08.830617+0000","last_became_peered":"2024-05-27T07:41:08.830617+0000","last_unstale":"2024-05-27T07:46:27.517918+0000","last_undegraded":"2024-05-27T07:46:27.517918+0000","last_fullsized":"2024-05-27T07:46:27.517918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:09:13.046162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.013806+0000","last_change":"2024-05-27T07:41:22.010129+0000","last_active":"2024-05-27T07:46:27.013806+0000","last_peered":"2024-05-27T07:46:27.013806+0000","last_clean":"2024-05-27T07:46:27.013806+0000","last_became_active":"2024-05-27T07:41:22.009608+0000","last_became_peered":"2024-05-27T07:41:22.009608+0000","last_unstale":"2024-05-27T07:46:27.013806+0000","last_undegraded":"2024-05-27T07:46:27.013806+0000","last_fullsized":"2024-05-27T07:46:27.013806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:50:27.225004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'54","reported_seq":125,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.926903+0000","last_change":"2024-05-27T07:41:04.794744+0000","last_active":"2024-05-27T07:46:27.926903+0000","last_peered":"2024-05-27T07:46:27.926903+0000","last_clean":"2024-05-27T07:46:27.926903+0000","last_became_active":"2024-05-27T07:41:04.794340+0000","last_became_peered":"2024-05-27T07:41:04.794340+0000","last_unstale":"2024-05-27T07:46:27.926903+0000","last_undegraded":"2024-05-27T07:46:27.926903+0000","last_fullsized":"2024-05-27T07:46:27.926903+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:43:20.335520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":96,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014574+0000","last_change":"2024-05-27T07:41:08.841834+0000","last_active":"2024-05-27T07:46:27.014574+0000","last_peered":"2024-05-27T07:46:27.014574+0000","last_clean":"2024-05-27T07:46:27.014574+0000","last_became_active":"2024-05-27T07:41:08.840261+0000","last_became_peered":"2024-05-27T07:41:08.840261+0000","last_unstale":"2024-05-27T07:46:27.014574+0000","last_undegraded":"2024-05-27T07:46:27.014574+0000","last_fullsized":"2024-05-27T07:46:27.014574+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:33.738835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338031+0000","last_change":"2024-05-27T07:41:22.005861+0000","last_active":"2024-05-27T07:46:27.338031+0000","last_peered":"2024-05-27T07:46:27.338031+0000","last_clean":"2024-05-27T07:46:27.338031+0000","last_became_active":"2024-05-27T07:41:22.005295+0000","last_became_peered":"2024-05-27T07:41:22.005295+0000","last_unstale":"2024-05-27T07:46:27.338031+0000","last_undegraded":"2024-05-27T07:46:27.338031+0000","last_fullsized":"2024-05-27T07:46:27.338031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:10:46.582656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.343151+0000","last_change":"2024-05-27T07:41:04.796562+0000","last_active":"2024-05-27T07:46:27.343151+0000","last_peered":"2024-05-27T07:46:27.343151+0000","last_clean":"2024-05-27T07:46:27.343151+0000","last_became_active":"2024-05-27T07:41:04.795538+0000","last_became_peered":"2024-05-27T07:41:04.795538+0000","last_unstale":"2024-05-27T07:46:27.343151+0000","last_undegraded":"2024-05-27T07:46:27.343151+0000","last_fullsized":"2024-05-27T07:46:27.343151+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:49:12.131999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342791+0000","last_change":"2024-05-27T07:41:08.832824+0000","last_active":"2024-05-27T07:46:27.342791+0000","last_peered":"2024-05-27T07:46:27.342791+0000","last_clean":"2024-05-27T07:46:27.342791+0000","last_became_active":"2024-05-27T07:41:08.832332+0000","last_became_peered":"2024-05-27T07:41:08.832332+0000","last_unstale":"2024-05-27T07:46:27.342791+0000","last_undegraded":"2024-05-27T07:46:27.342791+0000","last_fullsized":"2024-05-27T07:46:27.342791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:34.598773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342420+0000","last_change":"2024-05-27T07:41:22.003358+0000","last_active":"2024-05-27T07:46:27.342420+0000","last_peered":"2024-05-27T07:46:27.342420+0000","last_clean":"2024-05-27T07:46:27.342420+0000","last_became_active":"2024-05-27T07:41:22.002655+0000","last_became_peered":"2024-05-27T07:41:22.002655+0000","last_unstale":"2024-05-27T07:46:27.342420+0000","last_undegraded":"2024-05-27T07:46:27.342420+0000","last_fullsized":"2024-05-27T07:46:27.342420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:55:41.177138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339729+0000","last_change":"2024-05-27T07:41:04.888681+0000","last_active":"2024-05-27T07:46:27.339729+0000","last_peered":"2024-05-27T07:46:27.339729+0000","last_clean":"2024-05-27T07:46:27.339729+0000","last_became_active":"2024-05-27T07:41:04.887966+0000","last_became_peered":"2024-05-27T07:41:04.887966+0000","last_unstale":"2024-05-27T07:46:27.339729+0000","last_undegraded":"2024-05-27T07:46:27.339729+0000","last_fullsized":"2024-05-27T07:46:27.339729+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:37:01.178737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856720+0000","last_change":"2024-05-27T07:41:08.827839+0000","last_active":"2024-05-27T07:46:26.856720+0000","last_peered":"2024-05-27T07:46:26.856720+0000","last_clean":"2024-05-27T07:46:26.856720+0000","last_became_active":"2024-05-27T07:41:08.827403+0000","last_became_peered":"2024-05-27T07:41:08.827403+0000","last_unstale":"2024-05-27T07:46:26.856720+0000","last_undegraded":"2024-05-27T07:46:26.856720+0000","last_fullsized":"2024-05-27T07:46:26.856720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:01:42.075980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.856869+0000","last_change":"2024-05-27T07:41:22.006161+0000","last_active":"2024-05-27T07:46:26.856869+0000","last_peered":"2024-05-27T07:46:26.856869+0000","last_clean":"2024-05-27T07:46:26.856869+0000","last_became_active":"2024-05-27T07:41:22.005762+0000","last_became_peered":"2024-05-27T07:41:22.005762+0000","last_unstale":"2024-05-27T07:46:26.856869+0000","last_undegraded":"2024-05-27T07:46:26.856869+0000","last_fullsized":"2024-05-27T07:46:26.856869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:36:53.536444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014004+0000","last_change":"2024-05-27T07:41:04.774211+0000","last_active":"2024-05-27T07:46:27.014004+0000","last_peered":"2024-05-27T07:46:27.014004+0000","last_clean":"2024-05-27T07:46:27.014004+0000","last_became_active":"2024-05-27T07:41:04.773215+0000","last_became_peered":"2024-05-27T07:41:04.773215+0000","last_unstale":"2024-05-27T07:46:27.014004+0000","last_undegraded":"2024-05-27T07:46:27.014004+0000","last_fullsized":"2024-05-27T07:46:27.014004+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T07:46:37.273735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090283+0000","last_change":"2024-05-27T07:41:08.830434+0000","last_active":"2024-05-27T07:46:27.090283+0000","last_peered":"2024-05-27T07:46:27.090283+0000","last_clean":"2024-05-27T07:46:27.090283+0000","last_became_active":"2024-05-27T07:41:08.830039+0000","last_became_peered":"2024-05-27T07:41:08.830039+0000","last_unstale":"2024-05-27T07:46:27.090283+0000","last_undegraded":"2024-05-27T07:46:27.090283+0000","last_fullsized":"2024-05-27T07:46:27.090283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:42:27.877666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.345102+0000","last_change":"2024-05-27T07:41:59.783413+0000","last_active":"2024-05-27T07:46:27.345102+0000","last_peered":"2024-05-27T07:46:27.345102+0000","last_clean":"2024-05-27T07:46:27.345102+0000","last_became_active":"2024-05-27T07:41:59.782300+0000","last_became_peered":"2024-05-27T07:41:59.782300+0000","last_unstale":"2024-05-27T07:46:27.345102+0000","last_undegraded":"2024-05-27T07:46:27.345102+0000","last_fullsized":"2024-05-27T07:46:27.345102+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:57:59.348928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'5","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338570+0000","last_change":"2024-05-27T07:41:04.798969+0000","last_active":"2024-05-27T07:46:27.338570+0000","last_peered":"2024-05-27T07:46:27.338570+0000","last_clean":"2024-05-27T07:46:27.338570+0000","last_became_active":"2024-05-27T07:41:04.795996+0000","last_became_peered":"2024-05-27T07:41:04.795996+0000","last_unstale":"2024-05-27T07:46:27.338570+0000","last_undegraded":"2024-05-27T07:46:27.338570+0000","last_fullsized":"2024-05-27T07:46:27.338570+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:56:47.141063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014169+0000","last_change":"2024-05-27T07:41:08.835142+0000","last_active":"2024-05-27T07:46:27.014169+0000","last_peered":"2024-05-27T07:46:27.014169+0000","last_clean":"2024-05-27T07:46:27.014169+0000","last_became_active":"2024-05-27T07:41:08.834762+0000","last_became_peered":"2024-05-27T07:41:08.834762+0000","last_unstale":"2024-05-27T07:46:27.014169+0000","last_undegraded":"2024-05-27T07:46:27.014169+0000","last_fullsized":"2024-05-27T07:46:27.014169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:26:52.957554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342357+0000","last_change":"2024-05-27T07:41:22.000649+0000","last_active":"2024-05-27T07:46:27.342357+0000","last_peered":"2024-05-27T07:46:27.342357+0000","last_clean":"2024-05-27T07:46:27.342357+0000","last_became_active":"2024-05-27T07:41:22.000161+0000","last_became_peered":"2024-05-27T07:41:22.000161+0000","last_unstale":"2024-05-27T07:46:27.342357+0000","last_undegraded":"2024-05-27T07:46:27.342357+0000","last_fullsized":"2024-05-27T07:46:27.342357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:10.788027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:26.858452+0000","last_change":"2024-05-27T07:41:04.800680+0000","last_active":"2024-05-27T07:46:26.858452+0000","last_peered":"2024-05-27T07:46:26.858452+0000","last_clean":"2024-05-27T07:46:26.858452+0000","last_became_active":"2024-05-27T07:41:04.800249+0000","last_became_peered":"2024-05-27T07:41:04.800249+0000","last_unstale":"2024-05-27T07:46:26.858452+0000","last_undegraded":"2024-05-27T07:46:26.858452+0000","last_fullsized":"2024-05-27T07:46:26.858452+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:23:27.797216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350163+0000","last_change":"2024-05-27T07:41:08.844380+0000","last_active":"2024-05-27T07:46:27.350163+0000","last_peered":"2024-05-27T07:46:27.350163+0000","last_clean":"2024-05-27T07:46:27.350163+0000","last_became_active":"2024-05-27T07:41:08.843464+0000","last_became_peered":"2024-05-27T07:41:08.843464+0000","last_unstale":"2024-05-27T07:46:27.350163+0000","last_undegraded":"2024-05-27T07:46:27.350163+0000","last_fullsized":"2024-05-27T07:46:27.350163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:02:23.290234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341019+0000","last_change":"2024-05-27T07:41:22.010968+0000","last_active":"2024-05-27T07:46:27.341019+0000","last_peered":"2024-05-27T07:46:27.341019+0000","last_clean":"2024-05-27T07:46:27.341019+0000","last_became_active":"2024-05-27T07:41:22.010529+0000","last_became_peered":"2024-05-27T07:41:22.010529+0000","last_unstale":"2024-05-27T07:46:27.341019+0000","last_undegraded":"2024-05-27T07:46:27.341019+0000","last_fullsized":"2024-05-27T07:46:27.341019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:47:08.636434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'8","reported_seq":299,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:31.650113+0000","last_change":"2024-05-27T07:41:04.891536+0000","last_active":"2024-05-27T07:46:31.650113+0000","last_peered":"2024-05-27T07:46:31.650113+0000","last_clean":"2024-05-27T07:46:31.650113+0000","last_became_active":"2024-05-27T07:41:04.890724+0000","last_became_peered":"2024-05-27T07:41:04.890724+0000","last_unstale":"2024-05-27T07:46:31.650113+0000","last_undegraded":"2024-05-27T07:46:31.650113+0000","last_fullsized":"2024-05-27T07:46:31.650113+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:08:32.027624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2429,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350778+0000","last_change":"2024-05-27T07:41:08.834362+0000","last_active":"2024-05-27T07:46:27.350778+0000","last_peered":"2024-05-27T07:46:27.350778+0000","last_clean":"2024-05-27T07:46:27.350778+0000","last_became_active":"2024-05-27T07:41:08.833979+0000","last_became_peered":"2024-05-27T07:41:08.833979+0000","last_unstale":"2024-05-27T07:46:27.350778+0000","last_undegraded":"2024-05-27T07:46:27.350778+0000","last_fullsized":"2024-05-27T07:46:27.350778+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:35.098352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.341771+0000","last_change":"2024-05-27T07:41:21.997587+0000","last_active":"2024-05-27T07:46:27.341771+0000","last_peered":"2024-05-27T07:46:27.341771+0000","last_clean":"2024-05-27T07:46:27.341771+0000","last_became_active":"2024-05-27T07:41:21.996413+0000","last_became_peered":"2024-05-27T07:41:21.996413+0000","last_unstale":"2024-05-27T07:46:27.341771+0000","last_undegraded":"2024-05-27T07:46:27.341771+0000","last_fullsized":"2024-05-27T07:46:27.341771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:21:46.728047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"92'41","reported_seq":97,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.228573+0000","last_change":"2024-05-27T07:41:04.793580+0000","last_active":"2024-05-27T07:46:27.228573+0000","last_peered":"2024-05-27T07:46:27.228573+0000","last_clean":"2024-05-27T07:46:27.228573+0000","last_became_active":"2024-05-27T07:41:04.791135+0000","last_became_peered":"2024-05-27T07:41:04.791135+0000","last_unstale":"2024-05-27T07:46:27.228573+0000","last_undegraded":"2024-05-27T07:46:27.228573+0000","last_fullsized":"2024-05-27T07:46:27.228573+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:00:15.484187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.350999+0000","last_change":"2024-05-27T07:41:08.839187+0000","last_active":"2024-05-27T07:46:27.350999+0000","last_peered":"2024-05-27T07:46:27.350999+0000","last_clean":"2024-05-27T07:46:27.350999+0000","last_became_active":"2024-05-27T07:41:08.838821+0000","last_became_peered":"2024-05-27T07:41:08.838821+0000","last_unstale":"2024-05-27T07:46:27.350999+0000","last_undegraded":"2024-05-27T07:46:27.350999+0000","last_fullsized":"2024-05-27T07:46:27.350999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:41:30.806257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029214+0000","last_change":"2024-05-27T07:41:22.005406+0000","last_active":"2024-05-27T07:46:27.029214+0000","last_peered":"2024-05-27T07:46:27.029214+0000","last_clean":"2024-05-27T07:46:27.029214+0000","last_became_active":"2024-05-27T07:41:22.003415+0000","last_became_peered":"2024-05-27T07:41:22.003415+0000","last_unstale":"2024-05-27T07:46:27.029214+0000","last_undegraded":"2024-05-27T07:46:27.029214+0000","last_fullsized":"2024-05-27T07:46:27.029214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:20.934570+0000","last_clean_scrub_stamp":"2024-05-27T07:41:20.934570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:16:03.300452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517988+0000","last_change":"2024-05-27T07:41:04.798489+0000","last_active":"2024-05-27T07:46:27.517988+0000","last_peered":"2024-05-27T07:46:27.517988+0000","last_clean":"2024-05-27T07:46:27.517988+0000","last_became_active":"2024-05-27T07:41:04.797467+0000","last_became_peered":"2024-05-27T07:41:04.797467+0000","last_unstale":"2024-05-27T07:46:27.517988+0000","last_undegraded":"2024-05-27T07:46:27.517988+0000","last_fullsized":"2024-05-27T07:46:27.517988+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:03.740943+0000","last_clean_scrub_stamp":"2024-05-27T07:41:03.740943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:04.018577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.351043+0000","last_change":"2024-05-27T07:41:08.832215+0000","last_active":"2024-05-27T07:46:27.351043+0000","last_peered":"2024-05-27T07:46:27.351043+0000","last_clean":"2024-05-27T07:46:27.351043+0000","last_became_active":"2024-05-27T07:41:08.831491+0000","last_became_peered":"2024-05-27T07:41:08.831491+0000","last_unstale":"2024-05-27T07:46:27.351043+0000","last_undegraded":"2024-05-27T07:46:27.351043+0000","last_fullsized":"2024-05-27T07:46:27.351043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:32:03.084324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090501+0000","last_change":"2024-05-27T07:41:08.837301+0000","last_active":"2024-05-27T07:46:27.090501+0000","last_peered":"2024-05-27T07:46:27.090501+0000","last_clean":"2024-05-27T07:46:27.090501+0000","last_became_active":"2024-05-27T07:41:08.835533+0000","last_became_peered":"2024-05-27T07:41:08.835533+0000","last_unstale":"2024-05-27T07:46:27.090501+0000","last_undegraded":"2024-05-27T07:46:27.090501+0000","last_fullsized":"2024-05-27T07:46:27.090501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:03:45.807613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338530+0000","last_change":"2024-05-27T07:41:08.846740+0000","last_active":"2024-05-27T07:46:27.338530+0000","last_peered":"2024-05-27T07:46:27.338530+0000","last_clean":"2024-05-27T07:46:27.338530+0000","last_became_active":"2024-05-27T07:41:08.845576+0000","last_became_peered":"2024-05-27T07:41:08.845576+0000","last_unstale":"2024-05-27T07:46:27.338530+0000","last_undegraded":"2024-05-27T07:46:27.338530+0000","last_fullsized":"2024-05-27T07:46:27.338530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:41:36.944923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.338096+0000","last_change":"2024-05-27T07:41:08.840121+0000","last_active":"2024-05-27T07:46:27.338096+0000","last_peered":"2024-05-27T07:46:27.338096+0000","last_clean":"2024-05-27T07:46:27.338096+0000","last_became_active":"2024-05-27T07:41:08.839090+0000","last_became_peered":"2024-05-27T07:41:08.839090+0000","last_unstale":"2024-05-27T07:46:27.338096+0000","last_undegraded":"2024-05-27T07:46:27.338096+0000","last_fullsized":"2024-05-27T07:46:27.338096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:44.836953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342097+0000","last_change":"2024-05-27T07:41:08.835565+0000","last_active":"2024-05-27T07:46:27.342097+0000","last_peered":"2024-05-27T07:46:27.342097+0000","last_clean":"2024-05-27T07:46:27.342097+0000","last_became_active":"2024-05-27T07:41:08.831424+0000","last_became_peered":"2024-05-27T07:41:08.831424+0000","last_unstale":"2024-05-27T07:46:27.342097+0000","last_undegraded":"2024-05-27T07:46:27.342097+0000","last_fullsized":"2024-05-27T07:46:27.342097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:14:17.882368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517940+0000","last_change":"2024-05-27T07:41:08.830104+0000","last_active":"2024-05-27T07:46:27.517940+0000","last_peered":"2024-05-27T07:46:27.517940+0000","last_clean":"2024-05-27T07:46:27.517940+0000","last_became_active":"2024-05-27T07:41:08.829690+0000","last_became_peered":"2024-05-27T07:41:08.829690+0000","last_unstale":"2024-05-27T07:46:27.517940+0000","last_undegraded":"2024-05-27T07:46:27.517940+0000","last_fullsized":"2024-05-27T07:46:27.517940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:54:34.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.091546+0000","last_change":"2024-05-27T07:41:08.838654+0000","last_active":"2024-05-27T07:46:27.091546+0000","last_peered":"2024-05-27T07:46:27.091546+0000","last_clean":"2024-05-27T07:46:27.091546+0000","last_became_active":"2024-05-27T07:41:08.835033+0000","last_became_peered":"2024-05-27T07:41:08.835033+0000","last_unstale":"2024-05-27T07:46:27.091546+0000","last_undegraded":"2024-05-27T07:46:27.091546+0000","last_fullsized":"2024-05-27T07:46:27.091546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:04:24.007585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.339099+0000","last_change":"2024-05-27T07:41:08.844557+0000","last_active":"2024-05-27T07:46:27.339099+0000","last_peered":"2024-05-27T07:46:27.339099+0000","last_clean":"2024-05-27T07:46:27.339099+0000","last_became_active":"2024-05-27T07:41:08.843418+0000","last_became_peered":"2024-05-27T07:41:08.843418+0000","last_unstale":"2024-05-27T07:46:27.339099+0000","last_undegraded":"2024-05-27T07:46:27.339099+0000","last_fullsized":"2024-05-27T07:46:27.339099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:10:39.587778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014764+0000","last_change":"2024-05-27T07:41:08.830857+0000","last_active":"2024-05-27T07:46:27.014764+0000","last_peered":"2024-05-27T07:46:27.014764+0000","last_clean":"2024-05-27T07:46:27.014764+0000","last_became_active":"2024-05-27T07:41:08.830480+0000","last_became_peered":"2024-05-27T07:41:08.830480+0000","last_unstale":"2024-05-27T07:46:27.014764+0000","last_undegraded":"2024-05-27T07:46:27.014764+0000","last_fullsized":"2024-05-27T07:46:27.014764+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:49:07.307528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":48,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.029828+0000","last_change":"2024-05-27T07:41:08.824571+0000","last_active":"2024-05-27T07:46:27.029828+0000","last_peered":"2024-05-27T07:46:27.029828+0000","last_clean":"2024-05-27T07:46:27.029828+0000","last_became_active":"2024-05-27T07:41:08.820732+0000","last_became_peered":"2024-05-27T07:41:08.820732+0000","last_unstale":"2024-05-27T07:46:27.029828+0000","last_undegraded":"2024-05-27T07:46:27.029828+0000","last_fullsized":"2024-05-27T07:46:27.029828+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:55:14.299447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.014133+0000","last_change":"2024-05-27T07:41:08.842284+0000","last_active":"2024-05-27T07:46:27.014133+0000","last_peered":"2024-05-27T07:46:27.014133+0000","last_clean":"2024-05-27T07:46:27.014133+0000","last_became_active":"2024-05-27T07:41:08.841597+0000","last_became_peered":"2024-05-27T07:41:08.841597+0000","last_unstale":"2024-05-27T07:46:27.014133+0000","last_undegraded":"2024-05-27T07:46:27.014133+0000","last_fullsized":"2024-05-27T07:46:27.014133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:25:31.066181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.342151+0000","last_change":"2024-05-27T07:41:08.828935+0000","last_active":"2024-05-27T07:46:27.342151+0000","last_peered":"2024-05-27T07:46:27.342151+0000","last_clean":"2024-05-27T07:46:27.342151+0000","last_became_active":"2024-05-27T07:41:08.828542+0000","last_became_peered":"2024-05-27T07:41:08.828542+0000","last_unstale":"2024-05-27T07:46:27.342151+0000","last_undegraded":"2024-05-27T07:46:27.342151+0000","last_fullsized":"2024-05-27T07:46:27.342151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:07:35.270666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.517888+0000","last_change":"2024-05-27T07:41:08.817051+0000","last_active":"2024-05-27T07:46:27.517888+0000","last_peered":"2024-05-27T07:46:27.517888+0000","last_clean":"2024-05-27T07:46:27.517888+0000","last_became_active":"2024-05-27T07:41:08.816418+0000","last_became_peered":"2024-05-27T07:41:08.816418+0000","last_unstale":"2024-05-27T07:46:27.517888+0000","last_undegraded":"2024-05-27T07:46:27.517888+0000","last_fullsized":"2024-05-27T07:46:27.517888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:02.760707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-05-27T07:46:27.090720+0000","last_change":"2024-05-27T07:41:08.834081+0000","last_active":"2024-05-27T07:46:27.090720+0000","last_peered":"2024-05-27T07:46:27.090720+0000","last_clean":"2024-05-27T07:46:27.090720+0000","last_became_active":"2024-05-27T07:41:08.833679+0000","last_became_peered":"2024-05-27T07:41:08.833679+0000","last_unstale":"2024-05-27T07:46:27.090720+0000","last_undegraded":"2024-05-27T07:46:27.090720+0000","last_fullsized":"2024-05-27T07:46:27.090720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T07:41:07.781511+0000","last_clean_scrub_stamp":"2024-05-27T07:41:07.781511+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:41:14.426593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":612323,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2532,"num_read_kb":1395,"num_write":404,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1612,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2199552,"data_stored":1836969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":325,"ondisk_log_size":325,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645394,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":533518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841626,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28884,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716268,"statfs":{"total":95995035648,"available":95965458432,"internally_reserved":0,"allocated":1912832,"data_stored":1331826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005153,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28464,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716688,"statfs":{"total":95995035648,"available":95965888512,"internally_reserved":0,"allocated":1482752,"data_stored":925293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201384,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":925559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397616,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1683456,"data_stored":1132620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561142,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1064960,"data_stored":528449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1146880,"data_stored":605246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738532,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":530296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411478,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":531769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117134,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32744,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712408,"statfs":{"total":95995035648,"available":95961505792,"internally_reserved":0,"allocated":1605632,"data_stored":1052105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920903,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":608934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757375,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28800,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716352,"statfs":{"total":95995035648,"available":95965544448,"internally_reserved":0,"allocated":1826816,"data_stored":1257534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":78387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":77979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":136211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":210503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T07:46:36.353 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:46:36.830 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:37.234 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:36 smithi129 ceph-mon[35254]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:46:37.235 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:36 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:37.235 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:36 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:37.235 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:36 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:37.235 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:36 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:37.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:36 smithi136 ceph-mon[35324]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:46:37.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:36 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:37.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:36 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:37.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:36 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:37.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:36 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:37.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:36 smithi090 ceph-mon[31429]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:46:37.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:36 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:37.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:36 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:37.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:36 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:37.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:36 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:38.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:38 smithi090 ceph-mon[31429]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:46:38.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:38 smithi090 ceph-mon[31429]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-05-27T07:46:38.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:38 smithi129 ceph-mon[35254]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:46:38.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:38 smithi129 ceph-mon[35254]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-05-27T07:46:38.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:38 smithi136 ceph-mon[35324]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T07:46:38.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:38 smithi136 ceph-mon[35324]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-05-27T07:46:39.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.095+0000 7f18bd000700 1 -- 172.21.15.90:0/961022546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b810c5b0 msgr2=0x7f18b810c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:39.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.095+0000 7f18bd000700 1 --2- 172.21.15.90:0/961022546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b810c5b0 0x7f18b810c9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009990 tx=0x7f18a8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.095+0000 7f18bd000700 1 -- 172.21.15.90:0/961022546 shutdown_connections 2024-05-27T07:46:39.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.095+0000 7f18bd000700 1 --2- 172.21.15.90:0/961022546 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b8106ec0 0x7f18b8113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.095+0000 7f18bd000700 1 --2- 172.21.15.90:0/961022546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b810c5b0 0x7f18b810c9c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.095+0000 7f18bd000700 1 --2- 172.21.15.90:0/961022546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b8106570 0x7f18b8106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.095+0000 7f18bd000700 1 -- 172.21.15.90:0/961022546 >> 172.21.15.90:0/961022546 conn(0x7f18b8102070 msgr2=0x7f18b81044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:39.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.095+0000 7f18bd000700 1 -- 172.21.15.90:0/961022546 shutdown_connections 2024-05-27T07:46:39.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.095+0000 7f18bd000700 1 -- 172.21.15.90:0/961022546 wait complete. 2024-05-27T07:46:39.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.096+0000 7f18bd000700 1 Processor -- start 2024-05-27T07:46:39.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.096+0000 7f18bd000700 1 -- start start 2024-05-27T07:46:39.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.097+0000 7f18bd000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b8106570 0x7f18b819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.097+0000 7f18bd000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b8106ec0 0x7f18b819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.097+0000 7f18bd000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b810c5b0 0x7f18b81a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.097+0000 7f18bd000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81165c0 con 0x7f18b810c5b0 2024-05-27T07:46:39.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.097+0000 7f18bd000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8116440 con 0x7f18b8106570 2024-05-27T07:46:39.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.097+0000 7f18bd000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8116740 con 0x7f18b8106ec0 2024-05-27T07:46:39.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.097+0000 7f18b6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b810c5b0 0x7f18b81a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.097+0000 7f18b6800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b810c5b0 0x7f18b81a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56118/0 (socket says 172.21.15.90:56118) 2024-05-27T07:46:39.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.097+0000 7f18b6800700 1 -- 172.21.15.90:0/4148338735 learned_addr learned my addr 172.21.15.90:0/4148338735 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:39.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.097+0000 7f18b5400700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b8106ec0 0x7f18b819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.098+0000 7f18b5e00700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b8106570 0x7f18b819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.098+0000 7f18b6800700 1 -- 172.21.15.90:0/4148338735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b8106ec0 msgr2=0x7f18b819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:39.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.098+0000 7f18b6800700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b8106ec0 0x7f18b819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.098+0000 7f18b6800700 1 -- 172.21.15.90:0/4148338735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b8106570 msgr2=0x7f18b819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:39.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.098+0000 7f18b6800700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b8106570 0x7f18b819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.098+0000 7f18b6800700 1 -- 172.21.15.90:0/4148338735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b81a1ec0 con 0x7f18b810c5b0 2024-05-27T07:46:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.098+0000 7f18b6800700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b810c5b0 0x7f18b81a1820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f18b0007ee0 tx=0x7f18b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.099+0000 7f18aea00700 1 -- 172.21.15.90:0/4148338735 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0004bb0 con 0x7f18b810c5b0 2024-05-27T07:46:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.099+0000 7f18aea00700 1 -- 172.21.15.90:0/4148338735 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18b0004d50 con 0x7f18b810c5b0 2024-05-27T07:46:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.099+0000 7f18bd000700 1 -- 172.21.15.90:0/4148338735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b81a21b0 con 0x7f18b810c5b0 2024-05-27T07:46:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.099+0000 7f18aea00700 1 -- 172.21.15.90:0/4148338735 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b00058b0 con 0x7f18b810c5b0 2024-05-27T07:46:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.099+0000 7f18bd000700 1 -- 172.21.15.90:0/4148338735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b81a9ad0 con 0x7f18b810c5b0 2024-05-27T07:46:39.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.100+0000 7f18bd000700 1 -- 172.21.15.90:0/4148338735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b80512b0 con 0x7f18b810c5b0 2024-05-27T07:46:39.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.102+0000 7f18aea00700 1 -- 172.21.15.90:0/4148338735 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f18b001a020 con 0x7f18b810c5b0 2024-05-27T07:46:39.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.104+0000 7f18aea00700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f189c0b2b90 0x7f189c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:39.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.104+0000 7f18aea00700 1 -- 172.21.15.90:0/4148338735 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f18b00e2220 con 0x7f18b810c5b0 2024-05-27T07:46:39.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.104+0000 7f18b5e00700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f189c0b2b90 0x7f189c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:39.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.105+0000 7f18b5e00700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f189c0b2b90 0x7f189c0b50a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f18a8006010 tx=0x7f18a800b540 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:39.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.109+0000 7f18aea00700 1 -- 172.21.15.90:0/4148338735 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18b0010070 con 0x7f18b810c5b0 2024-05-27T07:46:39.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.386+0000 7f18bd000700 1 -- 172.21.15.90:0/4148338735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18b802e5e0 con 0x7f18b810c5b0 2024-05-27T07:46:39.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.387+0000 7f18aea00700 1 -- 172.21.15.90:0/4148338735 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11941 (secure 0 0 0) 0x7f18b00cb070 con 0x7f18b810c5b0 2024-05-27T07:46:39.388 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:46:39.388 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:46:25.689163+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/3523286296","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3523286296},{"type":"v1","addr":"172.21.15.136:6837","nonce":3523286296}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34442,34424,24478,34406]},"id":1}]} 2024-05-27T07:46:39.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.390+0000 7f18bd000700 1 -- 172.21.15.90:0/4148338735 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f189c0b2b90 msgr2=0x7f189c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:39.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.390+0000 7f18bd000700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f189c0b2b90 0x7f189c0b50a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f18a8006010 tx=0x7f18a800b540 comp rx=0 tx=0).stop 2024-05-27T07:46:39.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.390+0000 7f18bd000700 1 -- 172.21.15.90:0/4148338735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b810c5b0 msgr2=0x7f18b81a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:39.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.390+0000 7f18bd000700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b810c5b0 0x7f18b81a1820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f18b0007ee0 tx=0x7f18b000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.390+0000 7f18bd000700 1 -- 172.21.15.90:0/4148338735 shutdown_connections 2024-05-27T07:46:39.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.390+0000 7f18bd000700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b810c5b0 0x7f18b81a1820 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.390+0000 7f18bd000700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f189c0b2b90 0x7f189c0b50a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.390+0000 7f18bd000700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b8106570 0x7f18b819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.390+0000 7f18bd000700 1 --2- 172.21.15.90:0/4148338735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b8106ec0 0x7f18b819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:39.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.390+0000 7f18bd000700 1 -- 172.21.15.90:0/4148338735 >> 172.21.15.90:0/4148338735 conn(0x7f18b8102070 msgr2=0x7f18b8104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:39.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.391+0000 7f18bd000700 1 -- 172.21.15.90:0/4148338735 shutdown_connections 2024-05-27T07:46:39.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:39.391+0000 7f18bd000700 1 -- 172.21.15.90:0/4148338735 wait complete. 2024-05-27T07:46:39.392 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 30 2024-05-27T07:46:40.252 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T07:46:40.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:40 smithi129 ceph-mon[35254]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.0 KiB/s wr, 17 op/s 2024-05-27T07:46:40.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:40 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/4148338735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:46:40.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:40 smithi090 ceph-mon[31429]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.0 KiB/s wr, 17 op/s 2024-05-27T07:46:40.322 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:40 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/4148338735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:46:40.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:40 smithi136 ceph-mon[35324]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.0 KiB/s wr, 17 op/s 2024-05-27T07:46:40.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:40 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/4148338735' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:46:40.689 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:42.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:42 smithi129 ceph-mon[35254]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 938 B/s wr, 16 op/s 2024-05-27T07:46:42.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:42 smithi090 ceph-mon[31429]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 938 B/s wr, 16 op/s 2024-05-27T07:46:42.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:42 smithi136 ceph-mon[35324]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 938 B/s wr, 16 op/s 2024-05-27T07:46:43.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.004+0000 7f284b000700 1 -- 172.21.15.90:0/1039497307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2844108d20 msgr2=0x7f2844109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:43.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.004+0000 7f284b000700 1 --2- 172.21.15.90:0/1039497307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2844108d20 0x7f2844109130 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f28300099c0 tx=0x7f2830009cd0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.004+0000 7f284b000700 1 -- 172.21.15.90:0/1039497307 shutdown_connections 2024-05-27T07:46:43.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.004+0000 7f284b000700 1 --2- 172.21.15.90:0/1039497307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844109670 0x7f2844110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.004+0000 7f284b000700 1 --2- 172.21.15.90:0/1039497307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28441043f0 0x7f2844104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.004+0000 7f284b000700 1 --2- 172.21.15.90:0/1039497307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2844108d20 0x7f2844109130 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.004+0000 7f284b000700 1 -- 172.21.15.90:0/1039497307 >> 172.21.15.90:0/1039497307 conn(0x7f2844079d00 msgr2=0x7f284407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:43.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.004+0000 7f284b000700 1 -- 172.21.15.90:0/1039497307 shutdown_connections 2024-05-27T07:46:43.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.004+0000 7f284b000700 1 -- 172.21.15.90:0/1039497307 wait complete. 2024-05-27T07:46:43.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.005+0000 7f284b000700 1 Processor -- start 2024-05-27T07:46:43.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.005+0000 7f284b000700 1 -- start start 2024-05-27T07:46:43.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f284b000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28441043f0 0x7f28441a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:43.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f284b000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844108d20 0x7f28441a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:43.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f284b000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2844109670 0x7f284419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:43.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f284b000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2844114180 con 0x7f2844109670 2024-05-27T07:46:43.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f284b000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2844114000 con 0x7f28441043f0 2024-05-27T07:46:43.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f284b000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2844114300 con 0x7f2844108d20 2024-05-27T07:46:43.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f284a600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2844109670 0x7f284419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f284a600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2844109670 0x7f284419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56134/0 (socket says 172.21.15.90:56134) 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f284a600700 1 -- 172.21.15.90:0/92282607 learned_addr learned my addr 172.21.15.90:0/92282607 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f2849c00700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28441043f0 0x7f28441a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.006+0000 7f2849200700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844108d20 0x7f28441a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f284a600700 1 -- 172.21.15.90:0/92282607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844108d20 msgr2=0x7f28441a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f284a600700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844108d20 0x7f28441a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f284a600700 1 -- 172.21.15.90:0/92282607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28441043f0 msgr2=0x7f28441a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f284a600700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28441043f0 0x7f28441a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f284a600700 1 -- 172.21.15.90:0/92282607 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284419b0d0 con 0x7f2844109670 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f284a600700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2844109670 0x7f284419aa20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f283c007ee0 tx=0x7f283c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f2842000700 1 -- 172.21.15.90:0/92282607 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c00dbd0 con 0x7f2844109670 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f284b000700 1 -- 172.21.15.90:0/92282607 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284419b3c0 con 0x7f2844109670 2024-05-27T07:46:43.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f2842000700 1 -- 172.21.15.90:0/92282607 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f283c00dd70 con 0x7f2844109670 2024-05-27T07:46:43.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f2842000700 1 -- 172.21.15.90:0/92282607 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c0058d0 con 0x7f2844109670 2024-05-27T07:46:43.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.007+0000 7f284b000700 1 -- 172.21.15.90:0/92282607 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28441a7690 con 0x7f2844109670 2024-05-27T07:46:43.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.009+0000 7f284b000700 1 -- 172.21.15.90:0/92282607 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28441058a0 con 0x7f2844109670 2024-05-27T07:46:43.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.013+0000 7f2842000700 1 -- 172.21.15.90:0/92282607 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f283c01a020 con 0x7f2844109670 2024-05-27T07:46:43.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.014+0000 7f2842000700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f28280b2b10 0x7f28280b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:43.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.014+0000 7f2842000700 1 -- 172.21.15.90:0/92282607 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f283c0e2b60 con 0x7f2844109670 2024-05-27T07:46:43.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.014+0000 7f2849c00700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f28280b2b10 0x7f28280b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:43.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.015+0000 7f2849c00700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f28280b2b10 0x7f28280b5020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2834005950 tx=0x7f28340058e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:43.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.017+0000 7f2842000700 1 -- 172.21.15.90:0/92282607 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f283c010070 con 0x7f2844109670 2024-05-27T07:46:43.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.259+0000 7f284b000700 1 -- 172.21.15.90:0/92282607 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f284402e5e0 con 0x7f2844109670 2024-05-27T07:46:43.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.264+0000 7f2842000700 1 -- 172.21.15.90:0/92282607 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+25102 (secure 0 0 0) 0x7f283c0a9f40 con 0x7f2844109670 2024-05-27T07:46:43.265 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:46:43.266 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":92,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T07:46:26.839777+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6801/2854405112":"2024-05-28T07:46:26.839737+0000","172.21.15.90:6800/2854405112":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/3526000120":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/836242224":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/2932841075":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2692415447":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T07:46:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 -- 172.21.15.90:0/92282607 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f28280b2b10 msgr2=0x7f28280b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f28280b2b10 0x7f28280b5020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2834005950 tx=0x7f28340058e0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 -- 172.21.15.90:0/92282607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2844109670 msgr2=0x7f284419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2844109670 0x7f284419aa20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f283c007ee0 tx=0x7f283c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 -- 172.21.15.90:0/92282607 shutdown_connections 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2844109670 0x7f284419aa20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f28280b2b10 0x7f28280b5020 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28441043f0 0x7f28441a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 --2- 172.21.15.90:0/92282607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2844108d20 0x7f28441a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 -- 172.21.15.90:0/92282607 >> 172.21.15.90:0/92282607 conn(0x7f2844079d00 msgr2=0x7f2844101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.268+0000 7f284b000700 1 -- 172.21.15.90:0/92282607 shutdown_connections 2024-05-27T07:46:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:43.269+0000 7f284b000700 1 -- 172.21.15.90:0/92282607 wait complete. 2024-05-27T07:46:44.136 DEBUG:tasks.fwd_scrub:Yielding 2024-05-27T07:46:44.136 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T07:46:44.145 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:46:44.145 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T07:46:44.145 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-27T07:46:44.145 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-27T07:46:44.145 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:46:44.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:46:44.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:44 smithi129 ceph-mon[35254]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 938 B/s wr, 16 op/s 2024-05-27T07:46:44.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:44 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/92282607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:46:44.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.411+0000 7f7b09e00640 1 -- 172.21.15.90:0/1017564277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0410a110 msgr2=0x7f7b04103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:44.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.411+0000 7f7b09e00640 1 --2- 172.21.15.90:0/1017564277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0410a110 0x7f7b04103d50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7af0009900 tx=0x7f7af002a410 comp rx=0 tx=0).stop 2024-05-27T07:46:44.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.412+0000 7f7b09e00640 1 -- 172.21.15.90:0/1017564277 shutdown_connections 2024-05-27T07:46:44.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.412+0000 7f7b09e00640 1 --2- 172.21.15.90:0/1017564277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b041043c0 0x7f7b04110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.412+0000 7f7b09e00640 1 --2- 172.21.15.90:0/1017564277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b0410a110 0x7f7b04103d50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.412+0000 7f7b09e00640 1 --2- 172.21.15.90:0/1017564277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b04109c70 0x7f7b04103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.412+0000 7f7b09e00640 1 -- 172.21.15.90:0/1017564277 >> 172.21.15.90:0/1017564277 conn(0x7f7b040ff350 msgr2=0x7f7b041017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:44.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.412+0000 7f7b09e00640 1 -- 172.21.15.90:0/1017564277 shutdown_connections 2024-05-27T07:46:44.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.412+0000 7f7b09e00640 1 -- 172.21.15.90:0/1017564277 wait complete. 2024-05-27T07:46:44.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.413+0000 7f7b09e00640 1 Processor -- start 2024-05-27T07:46:44.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.413+0000 7f7b09e00640 1 -- start start 2024-05-27T07:46:44.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b09e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b041043c0 0x7f7b0419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:44.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b09e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04109c70 0x7f7b0419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:44.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b09e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0410a110 0x7f7b0419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:44.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b09e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04113d10 con 0x7f7b04109c70 2024-05-27T07:46:44.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b09e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04113b90 con 0x7f7b041043c0 2024-05-27T07:46:44.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b09e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04113e90 con 0x7f7b0410a110 2024-05-27T07:46:44.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b02000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04109c70 0x7f7b0419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:44.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b02000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04109c70 0x7f7b0419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56144/0 (socket says 172.21.15.90:56144) 2024-05-27T07:46:44.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b02000640 1 -- 172.21.15.90:0/4006196031 learned_addr learned my addr 172.21.15.90:0/4006196031 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:44.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b02a00640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b041043c0 0x7f7b0419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:44.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.414+0000 7f7b03400640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0410a110 0x7f7b0419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:44.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.415+0000 7f7b02000640 1 -- 172.21.15.90:0/4006196031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0410a110 msgr2=0x7f7b0419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:44.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.415+0000 7f7b02000640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0410a110 0x7f7b0419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.415+0000 7f7b02000640 1 -- 172.21.15.90:0/4006196031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b041043c0 msgr2=0x7f7b0419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:44.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.415+0000 7f7b02000640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b041043c0 0x7f7b0419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.415+0000 7f7b02000640 1 -- 172.21.15.90:0/4006196031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0419f630 con 0x7f7b04109c70 2024-05-27T07:46:44.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.415+0000 7f7b02000640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04109c70 0x7f7b0419ad30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7af0005940 tx=0x7f7af00040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:44.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.415+0000 7f7b08a00640 1 -- 172.21.15.90:0/4006196031 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af003f030 con 0x7f7b04109c70 2024-05-27T07:46:44.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.415+0000 7f7b08a00640 1 -- 172.21.15.90:0/4006196031 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7af0003a80 con 0x7f7b04109c70 2024-05-27T07:46:44.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.415+0000 7f7b09e00640 1 -- 172.21.15.90:0/4006196031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0419f8c0 con 0x7f7b04109c70 2024-05-27T07:46:44.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.415+0000 7f7b08a00640 1 -- 172.21.15.90:0/4006196031 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af0035070 con 0x7f7b04109c70 2024-05-27T07:46:44.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.416+0000 7f7b09e00640 1 -- 172.21.15.90:0/4006196031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0419fdb0 con 0x7f7b04109c70 2024-05-27T07:46:44.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.417+0000 7f7b09e00640 1 -- 172.21.15.90:0/4006196031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b04050860 con 0x7f7b04109c70 2024-05-27T07:46:44.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.419+0000 7f7b08a00640 1 -- 172.21.15.90:0/4006196031 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f7af0042030 con 0x7f7b04109c70 2024-05-27T07:46:44.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.421+0000 7f7b08a00640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ae80b2a70 0x7f7ae80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:44.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.421+0000 7f7b08a00640 1 -- 172.21.15.90:0/4006196031 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f7af0103600 con 0x7f7b04109c70 2024-05-27T07:46:44.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.421+0000 7f7b02a00640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ae80b2a70 0x7f7ae80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:44.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.422+0000 7f7b02a00640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ae80b2a70 0x7f7ae80b4f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7af4005fd0 tx=0x7f7af4005850 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:44.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.425+0000 7f7b08a00640 1 -- 172.21.15.90:0/4006196031 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7af00caab0 con 0x7f7b04109c70 2024-05-27T07:46:44.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:44 smithi090 ceph-mon[31429]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 938 B/s wr, 16 op/s 2024-05-27T07:46:44.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:44 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/92282607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:46:44.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:44 smithi136 ceph-mon[35324]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 938 B/s wr, 16 op/s 2024-05-27T07:46:44.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:44 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/92282607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T07:46:44.599 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:44.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.648+0000 7f7b09e00640 1 -- 172.21.15.90:0/4006196031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f7b041a73c0 con 0x7f7b04109c70 2024-05-27T07:46:44.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.653+0000 7f7b08a00640 1 -- 172.21.15.90:0/4006196031 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f7af00ca3b0 con 0x7f7b04109c70 2024-05-27T07:46:44.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.656+0000 7f7b09e00640 1 -- 172.21.15.90:0/4006196031 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ae80b2a70 msgr2=0x7f7ae80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.656+0000 7f7b09e00640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ae80b2a70 0x7f7ae80b4f80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7af4005fd0 tx=0x7f7af4005850 comp rx=0 tx=0).stop 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.656+0000 7f7b09e00640 1 -- 172.21.15.90:0/4006196031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04109c70 msgr2=0x7f7b0419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.656+0000 7f7b09e00640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04109c70 0x7f7b0419ad30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7af0005940 tx=0x7f7af00040e0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.656+0000 7f7b09e00640 1 -- 172.21.15.90:0/4006196031 shutdown_connections 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.656+0000 7f7b09e00640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ae80b2a70 0x7f7ae80b4f80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.656+0000 7f7b09e00640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b0410a110 0x7f7b0419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.656+0000 7f7b09e00640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b04109c70 0x7f7b0419ad30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.656+0000 7f7b09e00640 1 --2- 172.21.15.90:0/4006196031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b041043c0 0x7f7b0419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.656+0000 7f7b09e00640 1 -- 172.21.15.90:0/4006196031 >> 172.21.15.90:0/4006196031 conn(0x7f7b040ff350 msgr2=0x7f7b0410f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.657+0000 7f7b09e00640 1 -- 172.21.15.90:0/4006196031 shutdown_connections 2024-05-27T07:46:44.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.657+0000 7f7b09e00640 1 -- 172.21.15.90:0/4006196031 wait complete. 2024-05-27T07:46:44.689 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-27T07:46:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.961+0000 7ff421600640 1 -- 172.21.15.90:0/3603699096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c10d990 msgr2=0x7ff41c10fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.961+0000 7ff421600640 1 --2- 172.21.15.90:0/3603699096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c10d990 0x7ff41c10fdb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff408009900 tx=0x7ff40802a480 comp rx=0 tx=0).stop 2024-05-27T07:46:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.961+0000 7ff421600640 1 -- 172.21.15.90:0/3603699096 shutdown_connections 2024-05-27T07:46:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.961+0000 7ff421600640 1 --2- 172.21.15.90:0/3603699096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff41c1102f0 0x7ff41c112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.961+0000 7ff421600640 1 --2- 172.21.15.90:0/3603699096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c10d990 0x7ff41c10fdb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.961+0000 7ff421600640 1 --2- 172.21.15.90:0/3603699096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c075480 0x7ff41c075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.961+0000 7ff421600640 1 -- 172.21.15.90:0/3603699096 >> 172.21.15.90:0/3603699096 conn(0x7ff41c079240 msgr2=0x7ff41c079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.961+0000 7ff421600640 1 -- 172.21.15.90:0/3603699096 shutdown_connections 2024-05-27T07:46:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.961+0000 7ff421600640 1 -- 172.21.15.90:0/3603699096 wait complete. 2024-05-27T07:46:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.963+0000 7ff421600640 1 Processor -- start 2024-05-27T07:46:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.963+0000 7ff421600640 1 -- start start 2024-05-27T07:46:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.964+0000 7ff421600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c075480 0x7ff41c1a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.964+0000 7ff421600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c10d990 0x7ff41c1a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.964+0000 7ff421600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff41c1102f0 0x7ff41c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.964+0000 7ff421600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c115d70 con 0x7ff41c10d990 2024-05-27T07:46:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.964+0000 7ff421600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c115bf0 con 0x7ff41c075480 2024-05-27T07:46:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.964+0000 7ff421600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c115ef0 con 0x7ff41c1102f0 2024-05-27T07:46:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.964+0000 7ff41b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c10d990 0x7ff41c1a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.964+0000 7ff41b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c10d990 0x7ff41c1a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56162/0 (socket says 172.21.15.90:56162) 2024-05-27T07:46:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.964+0000 7ff41b400640 1 -- 172.21.15.90:0/2133306059 learned_addr learned my addr 172.21.15.90:0/2133306059 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.964+0000 7ff41be00640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c075480 0x7ff41c1a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.965+0000 7ff420c00640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff41c1102f0 0x7ff41c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.965+0000 7ff41b400640 1 -- 172.21.15.90:0/2133306059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff41c1102f0 msgr2=0x7ff41c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.965+0000 7ff41b400640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff41c1102f0 0x7ff41c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.965+0000 7ff41b400640 1 -- 172.21.15.90:0/2133306059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c075480 msgr2=0x7ff41c1a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.965+0000 7ff41b400640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c075480 0x7ff41c1a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.965+0000 7ff41b400640 1 -- 172.21.15.90:0/2133306059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c19ce00 con 0x7ff41c10d990 2024-05-27T07:46:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.965+0000 7ff41b400640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c10d990 0x7ff41c1a2a90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff408005940 tx=0x7ff408004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.965+0000 7ff418c00640 1 -- 172.21.15.90:0/2133306059 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40803f030 con 0x7ff41c10d990 2024-05-27T07:46:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.966+0000 7ff421600640 1 -- 172.21.15.90:0/2133306059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c19d030 con 0x7ff41c10d990 2024-05-27T07:46:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.966+0000 7ff418c00640 1 -- 172.21.15.90:0/2133306059 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff408004440 con 0x7ff41c10d990 2024-05-27T07:46:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.966+0000 7ff418c00640 1 -- 172.21.15.90:0/2133306059 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408035070 con 0x7ff41c10d990 2024-05-27T07:46:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.966+0000 7ff421600640 1 -- 172.21.15.90:0/2133306059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c1a92f0 con 0x7ff41c10d990 2024-05-27T07:46:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.967+0000 7ff421600640 1 -- 172.21.15.90:0/2133306059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41c076750 con 0x7ff41c10d990 2024-05-27T07:46:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.970+0000 7ff418c00640 1 -- 172.21.15.90:0/2133306059 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7ff408042030 con 0x7ff41c10d990 2024-05-27T07:46:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.972+0000 7ff418c00640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4000b2a70 0x7ff4000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.972+0000 7ff418c00640 1 -- 172.21.15.90:0/2133306059 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7ff408103440 con 0x7ff41c10d990 2024-05-27T07:46:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.972+0000 7ff41be00640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4000b2a70 0x7ff4000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.973+0000 7ff41be00640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4000b2a70 0x7ff4000b4f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0770d0 tx=0x7ff40c005f90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:44.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:44.977+0000 7ff418c00640 1 -- 172.21.15.90:0/2133306059 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff4080ca8f0 con 0x7ff41c10d990 2024-05-27T07:46:45.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.192+0000 7ff421600640 1 -- 172.21.15.90:0/2133306059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7ff41c1a94c0 con 0x7ff41c10d990 2024-05-27T07:46:45.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.202+0000 7ff418c00640 1 -- 172.21.15.90:0/2133306059 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7ff4080ca1f0 con 0x7ff41c10d990 2024-05-27T07:46:45.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 -- 172.21.15.90:0/2133306059 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4000b2a70 msgr2=0x7ff4000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4000b2a70 0x7ff4000b4f80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff41c0770d0 tx=0x7ff40c005f90 comp rx=0 tx=0).stop 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 -- 172.21.15.90:0/2133306059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c10d990 msgr2=0x7ff41c1a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c10d990 0x7ff41c1a2a90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff408005940 tx=0x7ff408004040 comp rx=0 tx=0).stop 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 -- 172.21.15.90:0/2133306059 shutdown_connections 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4000b2a70 0x7ff4000b4f80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff41c1102f0 0x7ff41c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c10d990 0x7ff41c1a2a90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 --2- 172.21.15.90:0/2133306059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff41c075480 0x7ff41c1a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 -- 172.21.15.90:0/2133306059 >> 172.21.15.90:0/2133306059 conn(0x7ff41c079240 msgr2=0x7ff41c102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.205+0000 7ff421600640 1 -- 172.21.15.90:0/2133306059 shutdown_connections 2024-05-27T07:46:45.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.206+0000 7ff421600640 1 -- 172.21.15.90:0/2133306059 wait complete. 2024-05-27T07:46:45.234 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-27T07:46:45.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.501+0000 7fdfbe600640 1 -- 172.21.15.90:0/2370628936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfb81043e0 msgr2=0x7fdfb81107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.501+0000 7fdfbe600640 1 --2- 172.21.15.90:0/2370628936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfb81043e0 0x7fdfb81107b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009900 tx=0x7fdfb002a410 comp rx=0 tx=0).stop 2024-05-27T07:46:45.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.501+0000 7fdfbe600640 1 -- 172.21.15.90:0/2370628936 shutdown_connections 2024-05-27T07:46:45.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.501+0000 7fdfbe600640 1 --2- 172.21.15.90:0/2370628936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfb81043e0 0x7fdfb81107b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.501+0000 7fdfbe600640 1 --2- 172.21.15.90:0/2370628936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfb810a130 0x7fdfb8103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.501+0000 7fdfbe600640 1 --2- 172.21.15.90:0/2370628936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfb8109c90 0x7fdfb8103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.501+0000 7fdfbe600640 1 -- 172.21.15.90:0/2370628936 >> 172.21.15.90:0/2370628936 conn(0x7fdfb8079390 msgr2=0x7fdfb80797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:45.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.501+0000 7fdfbe600640 1 -- 172.21.15.90:0/2370628936 shutdown_connections 2024-05-27T07:46:45.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.501+0000 7fdfbe600640 1 -- 172.21.15.90:0/2370628936 wait complete. 2024-05-27T07:46:45.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.503+0000 7fdfbe600640 1 Processor -- start 2024-05-27T07:46:45.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.503+0000 7fdfbe600640 1 -- start start 2024-05-27T07:46:45.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.503+0000 7fdfbe600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfb81043e0 0x7fdfb819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.503+0000 7fdfbe600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfb8109c90 0x7fdfb819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.503+0000 7fdfbe600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfb810a130 0x7fdfb819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.503+0000 7fdfbe600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb8113d80 con 0x7fdfb810a130 2024-05-27T07:46:45.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.503+0000 7fdfbe600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb8113c00 con 0x7fdfb8109c90 2024-05-27T07:46:45.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.503+0000 7fdfbe600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb8113f00 con 0x7fdfb81043e0 2024-05-27T07:46:45.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.503+0000 7fdfb7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfb810a130 0x7fdfb819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.504+0000 7fdfb7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfb810a130 0x7fdfb819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56186/0 (socket says 172.21.15.90:56186) 2024-05-27T07:46:45.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.504+0000 7fdfb7e00640 1 -- 172.21.15.90:0/3129993660 learned_addr learned my addr 172.21.15.90:0/3129993660 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:45.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.504+0000 7fdfaea00640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfb8109c90 0x7fdfb819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.504+0000 7fdfb7400640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfb81043e0 0x7fdfb819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.504+0000 7fdfb7e00640 1 -- 172.21.15.90:0/3129993660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfb81043e0 msgr2=0x7fdfb819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.504+0000 7fdfb7e00640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfb81043e0 0x7fdfb819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.504+0000 7fdfb7e00640 1 -- 172.21.15.90:0/3129993660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfb8109c90 msgr2=0x7fdfb819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.504+0000 7fdfb7e00640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfb8109c90 0x7fdfb819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.504+0000 7fdfb7e00640 1 -- 172.21.15.90:0/3129993660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb819f660 con 0x7fdfb810a130 2024-05-27T07:46:45.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.504+0000 7fdfb7e00640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfb810a130 0x7fdfb819f020 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0004770 tx=0x7fdfb0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:45.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.505+0000 7fdfb4c00640 1 -- 172.21.15.90:0/3129993660 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb003f070 con 0x7fdfb810a130 2024-05-27T07:46:45.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.505+0000 7fdfb4c00640 1 -- 172.21.15.90:0/3129993660 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfb003baf0 con 0x7fdfb810a130 2024-05-27T07:46:45.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.505+0000 7fdfb4c00640 1 -- 172.21.15.90:0/3129993660 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0035030 con 0x7fdfb810a130 2024-05-27T07:46:45.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.505+0000 7fdfbe600640 1 -- 172.21.15.90:0/3129993660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb819f8f0 con 0x7fdfb810a130 2024-05-27T07:46:45.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.505+0000 7fdfbe600640 1 -- 172.21.15.90:0/3129993660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb819fde0 con 0x7fdfb810a130 2024-05-27T07:46:45.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.506+0000 7fdfbe600640 1 -- 172.21.15.90:0/3129993660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb8104f70 con 0x7fdfb810a130 2024-05-27T07:46:45.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.508+0000 7fdfb4c00640 1 -- 172.21.15.90:0/3129993660 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fdfb0042070 con 0x7fdfb810a130 2024-05-27T07:46:45.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.510+0000 7fdfb4c00640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdf9c0b2a70 0x7fdf9c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:45.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.510+0000 7fdfb4c00640 1 -- 172.21.15.90:0/3129993660 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fdfb0103720 con 0x7fdfb810a130 2024-05-27T07:46:45.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.510+0000 7fdfb7400640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdf9c0b2a70 0x7fdf9c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:45.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.511+0000 7fdfb7400640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdf9c0b2a70 0x7fdf9c0b4f80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8104970 tx=0x7fdfa8009500 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:45.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.513+0000 7fdfb4c00640 1 -- 172.21.15.90:0/3129993660 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdfb00cabd0 con 0x7fdfb810a130 2024-05-27T07:46:45.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.732+0000 7fdfbe600640 1 -- 172.21.15.90:0/3129993660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fdfb810bda0 con 0x7fdfb810a130 2024-05-27T07:46:45.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.744+0000 7fdfb4c00640 1 -- 172.21.15.90:0/3129993660 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7fdfb00ca4d0 con 0x7fdfb810a130 2024-05-27T07:46:45.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.746+0000 7fdfbe600640 1 -- 172.21.15.90:0/3129993660 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdf9c0b2a70 msgr2=0x7fdf9c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.746+0000 7fdfbe600640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdf9c0b2a70 0x7fdf9c0b4f80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8104970 tx=0x7fdfa8009500 comp rx=0 tx=0).stop 2024-05-27T07:46:45.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.746+0000 7fdfbe600640 1 -- 172.21.15.90:0/3129993660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfb810a130 msgr2=0x7fdfb819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:45.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.746+0000 7fdfbe600640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfb810a130 0x7fdfb819f020 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0004770 tx=0x7fdfb0003b90 comp rx=0 tx=0).stop 2024-05-27T07:46:45.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.746+0000 7fdfbe600640 1 -- 172.21.15.90:0/3129993660 shutdown_connections 2024-05-27T07:46:45.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.746+0000 7fdfbe600640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdf9c0b2a70 0x7fdf9c0b4f80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.746+0000 7fdfbe600640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfb8109c90 0x7fdfb819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.747+0000 7fdfbe600640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfb810a130 0x7fdfb819f020 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.747+0000 7fdfbe600640 1 --2- 172.21.15.90:0/3129993660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfb81043e0 0x7fdfb819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:45.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.747+0000 7fdfbe600640 1 -- 172.21.15.90:0/3129993660 >> 172.21.15.90:0/3129993660 conn(0x7fdfb8079390 msgr2=0x7fdfb8102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:45.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.747+0000 7fdfbe600640 1 -- 172.21.15.90:0/3129993660 shutdown_connections 2024-05-27T07:46:45.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:45.747+0000 7fdfbe600640 1 -- 172.21.15.90:0/3129993660 wait complete. 2024-05-27T07:46:45.776 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/4006196031' entity='client.admin' 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 938 B/s wr, 20 op/s 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2133306059' entity='client.admin' 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:45.838 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:45 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:45.999 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/4006196031' entity='client.admin' 2024-05-27T07:46:45.999 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:45.999 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:46.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:46.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:46.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 938 B/s wr, 20 op/s 2024-05-27T07:46:46.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2133306059' entity='client.admin' 2024-05-27T07:46:46.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:46.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:46.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:46.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:45 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:46.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.042+0000 7fbc56000640 1 -- 172.21.15.90:0/252995922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5010b800 msgr2=0x7fbc5010dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.042+0000 7fbc56000640 1 --2- 172.21.15.90:0/252995922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5010b800 0x7fbc5010dc20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009900 tx=0x7fbc3c02a410 comp rx=0 tx=0).stop 2024-05-27T07:46:46.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.042+0000 7fbc56000640 1 -- 172.21.15.90:0/252995922 shutdown_connections 2024-05-27T07:46:46.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.042+0000 7fbc56000640 1 --2- 172.21.15.90:0/252995922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5010e1f0 0x7fbc501106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.042+0000 7fbc56000640 1 --2- 172.21.15.90:0/252995922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5010b800 0x7fbc5010dc20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.042+0000 7fbc56000640 1 --2- 172.21.15.90:0/252995922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc50104da0 0x7fbc501051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.042+0000 7fbc56000640 1 -- 172.21.15.90:0/252995922 >> 172.21.15.90:0/252995922 conn(0x7fbc500782d0 msgr2=0x7fbc50078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:46.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.042+0000 7fbc56000640 1 -- 172.21.15.90:0/252995922 shutdown_connections 2024-05-27T07:46:46.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.042+0000 7fbc56000640 1 -- 172.21.15.90:0/252995922 wait complete. 2024-05-27T07:46:46.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.043+0000 7fbc56000640 1 Processor -- start 2024-05-27T07:46:46.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.043+0000 7fbc56000640 1 -- start start 2024-05-27T07:46:46.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.044+0000 7fbc56000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc50104da0 0x7fbc501a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.044+0000 7fbc56000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5010b800 0x7fbc501a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.044+0000 7fbc56000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5010e1f0 0x7fbc5019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.044+0000 7fbc56000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50113d50 con 0x7fbc5010b800 2024-05-27T07:46:46.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.044+0000 7fbc56000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50113bd0 con 0x7fbc50104da0 2024-05-27T07:46:46.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.044+0000 7fbc56000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50113ed0 con 0x7fbc5010e1f0 2024-05-27T07:46:46.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.044+0000 7fbc4ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5010b800 0x7fbc501a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.044+0000 7fbc4ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5010b800 0x7fbc501a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56200/0 (socket says 172.21.15.90:56200) 2024-05-27T07:46:46.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.044+0000 7fbc4ea00640 1 -- 172.21.15.90:0/1290178698 learned_addr learned my addr 172.21.15.90:0/1290178698 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:46.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.044+0000 7fbc4f400640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc50104da0 0x7fbc501a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.045+0000 7fbc4fe00640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5010e1f0 0x7fbc5019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.045+0000 7fbc4ea00640 1 -- 172.21.15.90:0/1290178698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5010e1f0 msgr2=0x7fbc5019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.045+0000 7fbc4ea00640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5010e1f0 0x7fbc5019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.045+0000 7fbc4ea00640 1 -- 172.21.15.90:0/1290178698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc50104da0 msgr2=0x7fbc501a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.045+0000 7fbc4ea00640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc50104da0 0x7fbc501a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.045+0000 7fbc4ea00640 1 -- 172.21.15.90:0/1290178698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5019cef0 con 0x7fbc5010b800 2024-05-27T07:46:46.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.045+0000 7fbc4ea00640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5010b800 0x7fbc501a2b20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c02a930 tx=0x7fbc3c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:46.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.045+0000 7fbc47e00640 1 -- 172.21.15.90:0/1290178698 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c03f070 con 0x7fbc5010b800 2024-05-27T07:46:46.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.046+0000 7fbc56000640 1 -- 172.21.15.90:0/1290178698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5019d180 con 0x7fbc5010b800 2024-05-27T07:46:46.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.046+0000 7fbc47e00640 1 -- 172.21.15.90:0/1290178698 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbc3c003e40 con 0x7fbc5010b800 2024-05-27T07:46:46.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.046+0000 7fbc47e00640 1 -- 172.21.15.90:0/1290178698 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c035030 con 0x7fbc5010b800 2024-05-27T07:46:46.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.046+0000 7fbc56000640 1 -- 172.21.15.90:0/1290178698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc501a9430 con 0x7fbc5010b800 2024-05-27T07:46:46.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.047+0000 7fbc56000640 1 -- 172.21.15.90:0/1290178698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5006bb90 con 0x7fbc5010b800 2024-05-27T07:46:46.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.051+0000 7fbc47e00640 1 -- 172.21.15.90:0/1290178698 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fbc3c042070 con 0x7fbc5010b800 2024-05-27T07:46:46.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.053+0000 7fbc47e00640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbc340b2a70 0x7fbc340b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.053+0000 7fbc47e00640 1 -- 172.21.15.90:0/1290178698 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fbc3c1034c0 con 0x7fbc5010b800 2024-05-27T07:46:46.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.053+0000 7fbc4f400640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbc340b2a70 0x7fbc340b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.054+0000 7fbc4f400640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbc340b2a70 0x7fbc340b4f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc40005fd0 tx=0x7fbc40005850 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:46.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.058+0000 7fbc47e00640 1 -- 172.21.15.90:0/1290178698 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc3c0ca9b0 con 0x7fbc5010b800 2024-05-27T07:46:46.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/4006196031' entity='client.admin' 2024-05-27T07:46:46.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:46.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:46.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:46.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:46.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 938 B/s wr, 20 op/s 2024-05-27T07:46:46.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2133306059' entity='client.admin' 2024-05-27T07:46:46.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:46.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:46.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:46.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:45 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.311+0000 7fbc56000640 1 -- 172.21.15.90:0/1290178698 --> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fbc5010f6e0 con 0x7fbc340b2a70 2024-05-27T07:46:46.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.482+0000 7fbc47e00640 1 -- 172.21.15.90:0/1290178698 <== mgr.34967 v2:172.21.15.90:6800/4256436582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbc5010f6e0 con 0x7fbc340b2a70 2024-05-27T07:46:46.482 INFO:teuthology.orchestra.run.smithi090.stdout:Schedule set for path / 2024-05-27T07:46:46.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 -- 172.21.15.90:0/1290178698 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbc340b2a70 msgr2=0x7fbc340b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbc340b2a70 0x7fbc340b4f80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc40005fd0 tx=0x7fbc40005850 comp rx=0 tx=0).stop 2024-05-27T07:46:46.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 -- 172.21.15.90:0/1290178698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5010b800 msgr2=0x7fbc501a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5010b800 0x7fbc501a2b20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c02a930 tx=0x7fbc3c003c20 comp rx=0 tx=0).stop 2024-05-27T07:46:46.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 -- 172.21.15.90:0/1290178698 shutdown_connections 2024-05-27T07:46:46.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbc340b2a70 0x7fbc340b4f80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc5010e1f0 0x7fbc5019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc5010b800 0x7fbc501a2b20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 --2- 172.21.15.90:0/1290178698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbc50104da0 0x7fbc501a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 -- 172.21.15.90:0/1290178698 >> 172.21.15.90:0/1290178698 conn(0x7fbc500782d0 msgr2=0x7fbc5010a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:46.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 -- 172.21.15.90:0/1290178698 shutdown_connections 2024-05-27T07:46:46.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.486+0000 7fbc56000640 1 -- 172.21.15.90:0/1290178698 wait complete. 2024-05-27T07:46:46.517 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-27T07:46:46.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.781+0000 7fb8e3a00640 1 -- 172.21.15.90:0/3834750124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8dc1029a0 msgr2=0x7fb8dc102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.781+0000 7fb8e3a00640 1 --2- 172.21.15.90:0/3834750124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8dc1029a0 0x7fb8dc102db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009ca0 tx=0x7fb8d002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.782+0000 7fb8e3a00640 1 -- 172.21.15.90:0/3834750124 shutdown_connections 2024-05-27T07:46:46.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.782+0000 7fb8e3a00640 1 --2- 172.21.15.90:0/3834750124 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8dc110540 0x7fb8dc112960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.782+0000 7fb8e3a00640 1 --2- 172.21.15.90:0/3834750124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8dc10dbe0 0x7fb8dc110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.782+0000 7fb8e3a00640 1 --2- 172.21.15.90:0/3834750124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8dc1029a0 0x7fb8dc102db0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.782+0000 7fb8e3a00640 1 -- 172.21.15.90:0/3834750124 >> 172.21.15.90:0/3834750124 conn(0x7fb8dc0782d0 msgr2=0x7fb8dc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:46.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.782+0000 7fb8e3a00640 1 -- 172.21.15.90:0/3834750124 shutdown_connections 2024-05-27T07:46:46.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.782+0000 7fb8e3a00640 1 -- 172.21.15.90:0/3834750124 wait complete. 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.783+0000 7fb8e3a00640 1 Processor -- start 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.783+0000 7fb8e3a00640 1 -- start start 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.783+0000 7fb8e3a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8dc1029a0 0x7fb8dc1a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.783+0000 7fb8e3a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8dc10dbe0 0x7fb8dc1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.783+0000 7fb8e3a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8dc110540 0x7fb8dc19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.783+0000 7fb8e3a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc115f20 con 0x7fb8dc1029a0 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.783+0000 7fb8e3a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc115da0 con 0x7fb8dc110540 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.783+0000 7fb8e3a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc1160a0 con 0x7fb8dc10dbe0 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8e1000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8dc1029a0 0x7fb8dc1a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8e1000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8dc1029a0 0x7fb8dc1a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56228/0 (socket says 172.21.15.90:56228) 2024-05-27T07:46:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8e1000640 1 -- 172.21.15.90:0/1392393582 learned_addr learned my addr 172.21.15.90:0/1392393582 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8dbe00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8dc10dbe0 0x7fb8dc1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8e1a00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8dc110540 0x7fb8dc19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8e1000640 1 -- 172.21.15.90:0/1392393582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8dc10dbe0 msgr2=0x7fb8dc1a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8e1000640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8dc10dbe0 0x7fb8dc1a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8e1000640 1 -- 172.21.15.90:0/1392393582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8dc110540 msgr2=0x7fb8dc19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8e1000640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8dc110540 0x7fb8dc19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8e1000640 1 -- 172.21.15.90:0/1392393582 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc19cea0 con 0x7fb8dc1029a0 2024-05-27T07:46:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.784+0000 7fb8dbe00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8dc10dbe0 0x7fb8dc1a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.785+0000 7fb8e1000640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8dc1029a0 0x7fb8dc1a2590 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009dc0 tx=0x7fb8d0032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.785+0000 7fb8d9600640 1 -- 172.21.15.90:0/1392393582 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0004040 con 0x7fb8dc1029a0 2024-05-27T07:46:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.785+0000 7fb8d9600640 1 -- 172.21.15.90:0/1392393582 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8d0004670 con 0x7fb8dc1029a0 2024-05-27T07:46:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.785+0000 7fb8e3a00640 1 -- 172.21.15.90:0/1392393582 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc19d130 con 0x7fb8dc1029a0 2024-05-27T07:46:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.785+0000 7fb8d9600640 1 -- 172.21.15.90:0/1392393582 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0030c80 con 0x7fb8dc1029a0 2024-05-27T07:46:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.785+0000 7fb8e3a00640 1 -- 172.21.15.90:0/1392393582 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc1a93e0 con 0x7fb8dc1029a0 2024-05-27T07:46:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.787+0000 7fb8e3a00640 1 -- 172.21.15.90:0/1392393582 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8dc06adc0 con 0x7fb8dc1029a0 2024-05-27T07:46:46.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.791+0000 7fb8d9600640 1 -- 172.21.15.90:0/1392393582 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fb8d00041e0 con 0x7fb8dc1029a0 2024-05-27T07:46:46.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.792+0000 7fb8d9600640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8c40b2a70 0x7fb8c40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.792+0000 7fb8d9600640 1 -- 172.21.15.90:0/1392393582 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fb8d01042b0 con 0x7fb8dc1029a0 2024-05-27T07:46:46.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.793+0000 7fb8dbe00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8c40b2a70 0x7fb8c40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.793+0000 7fb8d9600640 1 -- 172.21.15.90:0/1392393582 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb8d00cb7a0 con 0x7fb8dc1029a0 2024-05-27T07:46:46.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.793+0000 7fb8dbe00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8c40b2a70 0x7fb8c40b4f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80058c0 tx=0x7fb8c8005850 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.940+0000 7f093e400700 1 -- 172.21.15.90:0/220891434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f093810a980 msgr2=0x7f093810ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.940+0000 7f093e400700 1 --2- 172.21.15.90:0/220891434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f093810a980 0x7f093810ad90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0924009990 tx=0x7f0924009ca0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.940+0000 7f093e400700 1 -- 172.21.15.90:0/220891434 shutdown_connections 2024-05-27T07:46:46.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.940+0000 7f093e400700 1 --2- 172.21.15.90:0/220891434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f093810b2d0 0x7f0938117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.940+0000 7f093e400700 1 --2- 172.21.15.90:0/220891434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0938110960 0x7f0938110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.940+0000 7f093e400700 1 --2- 172.21.15.90:0/220891434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f093810a980 0x7f093810ad90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.940+0000 7f093e400700 1 -- 172.21.15.90:0/220891434 >> 172.21.15.90:0/220891434 conn(0x7f0938077c70 msgr2=0x7f09380780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:46.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.941+0000 7f093e400700 1 -- 172.21.15.90:0/220891434 shutdown_connections 2024-05-27T07:46:46.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.941+0000 7f093e400700 1 -- 172.21.15.90:0/220891434 wait complete. 2024-05-27T07:46:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.942+0000 7f093e400700 1 Processor -- start 2024-05-27T07:46:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.942+0000 7f093e400700 1 -- start start 2024-05-27T07:46:46.943 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:46 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3129993660' entity='client.admin' 2024-05-27T07:46:46.943 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:46 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:46.943 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:46 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:46.943 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:46 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:46.943 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:46 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:46.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.943+0000 7f093e400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f093810a980 0x7f09381a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.943+0000 7f093e400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f093810b2d0 0x7f09381a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.943+0000 7f093e400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0938110960 0x7f09381a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.943+0000 7f093e400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f093811a980 con 0x7f093810a980 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.943+0000 7f093e400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f093811a800 con 0x7f093810b2d0 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.943+0000 7f093e400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f093811ab00 con 0x7f0938110960 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.943+0000 7f0937400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f093810a980 0x7f09381a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.943+0000 7f0937400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f093810a980 0x7f09381a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56244/0 (socket says 172.21.15.90:56244) 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.943+0000 7f0937400700 1 -- 172.21.15.90:0/4002027089 learned_addr learned my addr 172.21.15.90:0/4002027089 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.944+0000 7f0936a00700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f093810b2d0 0x7f09381a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.944+0000 7f0937e00700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0938110960 0x7f09381a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.944+0000 7f0937400700 1 -- 172.21.15.90:0/4002027089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0938110960 msgr2=0x7f09381a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.944+0000 7f0937400700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0938110960 0x7f09381a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.944+0000 7f0937400700 1 -- 172.21.15.90:0/4002027089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f093810b2d0 msgr2=0x7f09381a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.944+0000 7f0937400700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f093810b2d0 0x7f09381a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.944+0000 7f0937400700 1 -- 172.21.15.90:0/4002027089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09381a6280 con 0x7f093810a980 2024-05-27T07:46:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.944+0000 7f0937400700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f093810a980 0x7f09381a13b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f092800d980 tx=0x7f092800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.945+0000 7f092fe00700 1 -- 172.21.15.90:0/4002027089 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928017070 con 0x7f093810a980 2024-05-27T07:46:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.945+0000 7f093e400700 1 -- 172.21.15.90:0/4002027089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09381a6570 con 0x7f093810a980 2024-05-27T07:46:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.945+0000 7f092fe00700 1 -- 172.21.15.90:0/4002027089 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0928010460 con 0x7f093810a980 2024-05-27T07:46:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.945+0000 7f093e400700 1 -- 172.21.15.90:0/4002027089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09381ade90 con 0x7f093810a980 2024-05-27T07:46:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.945+0000 7f092fe00700 1 -- 172.21.15.90:0/4002027089 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0928020060 con 0x7f093810a980 2024-05-27T07:46:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.949+0000 7f092fe00700 1 -- 172.21.15.90:0/4002027089 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f092800f620 con 0x7f093810a980 2024-05-27T07:46:46.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.950+0000 7f094ce82b80 1 Processor -- start 2024-05-27T07:46:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.950+0000 7f094ce82b80 1 -- start start 2024-05-27T07:46:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.950+0000 7f092fe00700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f091c0b2b90 0x7f091c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f092fe00700 1 -- 172.21.15.90:0/4002027089 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f09280e25e0 con 0x7f093810a980 2024-05-27T07:46:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f0937e00700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f091c0b2b90 0x7f091c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2ea3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2ea8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de2eaef0 0x5636de2f7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f094ce82b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5636de218ec0 con 0x5636de16bec0 2024-05-27T07:46:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f094ce82b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5636de208a60 con 0x5636de2eaef0 2024-05-27T07:46:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f094ce82b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5636de20caa0 con 0x5636de257a70 2024-05-27T07:46:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f0936a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2ea8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f0936a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2ea8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56258/0 (socket says 172.21.15.90:56258) 2024-05-27T07:46:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.951+0000 7f0936a00700 1 -- 172.21.15.90:0/904748157 learned_addr learned my addr 172.21.15.90:0/904748157 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.952+0000 7f0937400700 1 --2- 172.21.15.90:0/904748157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2ea3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.952+0000 7f0937e00700 1 --2- 172.21.15.90:0/904748157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de2eaef0 0x5636de2f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.952+0000 7f0937e00700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f091c0b2b90 0x7f091c0b50a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0930009cb0 tx=0x7f09300077e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.952+0000 7f0936a00700 1 -- 172.21.15.90:0/904748157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 msgr2=0x5636de2ea3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.952+0000 7f0936a00700 1 --2- 172.21.15.90:0/904748157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2ea3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.952+0000 7f0936a00700 1 -- 172.21.15.90:0/904748157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de2eaef0 msgr2=0x5636de2f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.952+0000 7f0936a00700 1 --2- 172.21.15.90:0/904748157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de2eaef0 0x5636de2f7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.952+0000 7f0936a00700 1 -- 172.21.15.90:0/904748157 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636de1d14a0 con 0x5636de16bec0 2024-05-27T07:46:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.952+0000 7f0937400700 1 --2- 172.21.15.90:0/904748157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2ea3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.952+0000 7f0936a00700 1 --2- 172.21.15.90:0/904748157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2ea8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0924009990 tx=0x7f09240096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bff05b637a47909 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.953+0000 7f092cc00700 1 -- 172.21.15.90:0/904748157 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924026070 con 0x5636de16bec0 2024-05-27T07:46:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.953+0000 7f092cc00700 1 -- 172.21.15.90:0/904748157 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0924003c50 con 0x5636de16bec0 2024-05-27T07:46:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.953+0000 7f092cc00700 1 -- 172.21.15.90:0/904748157 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09240056d0 con 0x5636de16bec0 2024-05-27T07:46:46.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.954+0000 7f094ce82b80 1 -- 172.21.15.90:0/904748157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 msgr2=0x5636de2ea8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.954+0000 7f094ce82b80 1 --2- 172.21.15.90:0/904748157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2ea8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0924009990 tx=0x7f09240096a0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.954+0000 7f094ce82b80 1 -- 172.21.15.90:0/904748157 shutdown_connections 2024-05-27T07:46:46.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.954+0000 7f094ce82b80 1 --2- 172.21.15.90:0/904748157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de2eaef0 0x5636de2f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.954+0000 7f094ce82b80 1 --2- 172.21.15.90:0/904748157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2ea3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.954+0000 7f094ce82b80 1 --2- 172.21.15.90:0/904748157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2ea8f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.954+0000 7f094ce82b80 1 -- 172.21.15.90:0/904748157 >> 172.21.15.90:0/904748157 conn(0x5636de205410 msgr2=0x5636de2e6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:46.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.954+0000 7f094ce82b80 1 -- 172.21.15.90:0/904748157 shutdown_connections 2024-05-27T07:46:46.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.954+0000 7f094ce82b80 1 -- 172.21.15.90:0/904748157 wait complete. 2024-05-27T07:46:46.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.955+0000 7f094ce82b80 1 -- start start 2024-05-27T07:46:46.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.955+0000 7f094ce82b80 1 Processor -- start 2024-05-27T07:46:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.956+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2f4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.956+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de257a70 0x5636de2f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.956+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de2eaef0 0x5636de2f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.956+0000 7f094ce82b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5636de10f020 con 0x5636de16bec0 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.956+0000 7f094ce82b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5636de1a97e0 con 0x5636de257a70 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.956+0000 7f094ce82b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5636de2f7ba0 con 0x5636de2eaef0 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.956+0000 7f094ce82b80 10 client.0 _ll_register_callbacks cb 0x5636de1b2cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.956+0000 7f0937400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.956+0000 7f0937400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56268/0 (socket says 172.21.15.90:56268) 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.956+0000 7f0937400700 1 -- 172.21.15.90:0/304331887 learned_addr learned my addr 172.21.15.90:0/304331887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0936a00700 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de257a70 0x5636de2f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0937e00700 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de2eaef0 0x5636de2f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0937400700 1 -- 172.21.15.90:0/304331887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de2eaef0 msgr2=0x5636de2f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0937400700 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de2eaef0 0x5636de2f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0937400700 1 -- 172.21.15.90:0/304331887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de257a70 msgr2=0x5636de2f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0937400700 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de257a70 0x5636de2f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0937400700 1 -- 172.21.15.90:0/304331887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636de158950 con 0x5636de16bec0 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0937400700 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2f4740 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f09280a5340 tx=0x7f09280b1f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b2707ecb26df30 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0921600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0921600700 1 -- 172.21.15.90:0/304331887 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09280c3060 con 0x5636de16bec0 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.957+0000 7f0921600700 1 -- 172.21.15.90:0/304331887 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f09280e23c0 con 0x5636de16bec0 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f094ce82b80 1 -- 172.21.15.90:0/304331887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0924003c50 con 0x5636de16bec0 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f0921600700 1 -- 172.21.15.90:0/304331887 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f09280c8050 con 0x5636de16bec0 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f094ce82b80 10 client.15435 fetch_fsmap learned FSMap version 30 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f094ce82b80 1 -- 172.21.15.90:0/304331887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636de1d90e0 con 0x5636de16bec0 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f0921600700 1 -- 172.21.15.90:0/304331887 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0924003c50 con 0x5636de16bec0 2024-05-27T07:46:46.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f0921600700 1 -- 172.21.15.90:0/304331887 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f09280c3bb0 con 0x5636de16bec0 2024-05-27T07:46:46.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f094ce82b80 10 client.15435 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:46:46.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f094ce82b80 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:46.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f094ce82b80 20 client.15435 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:46.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f094ce82b80 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5636de304b10 0x5636de306f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f094ce82b80 4 client.15435 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:46.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.958+0000 7f094ce82b80 1 -- 172.21.15.90:0/304331887 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636de21ae10 con 0x5636de304b10 2024-05-27T07:46:46.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.959+0000 7f0936a00700 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5636de304b10 0x5636de306f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.960+0000 7f0936a00700 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5636de304b10 0x5636de306f90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa123264190fd133 server_cookie=cd53d99f429ca810 in_seq=0 out_seq=0 2024-05-27T07:46:46.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.960+0000 7f0921600700 10 client.15435 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:46.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f0921600700 1 -- 172.21.15.90:0/304331887 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5636de21ae10 con 0x5636de304b10 2024-05-27T07:46:46.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f0921600700 10 client.15435 handle_command_reply: tid=0 2024-05-27T07:46:46.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f094ce82b80 1 client.15435 shutdown 2024-05-27T07:46:46.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f094ce82b80 1 -- 172.21.15.90:0/304331887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 msgr2=0x5636de2f4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f094ce82b80 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2f4740 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f09280a5340 tx=0x7f09280b1f70 comp rx=0 tx=0).stop 2024-05-27T07:46:46.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f094ce82b80 1 -- 172.21.15.90:0/304331887 shutdown_connections 2024-05-27T07:46:46.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f094ce82b80 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5636de304b10 0x5636de306f90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f094ce82b80 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de2eaef0 0x5636de2f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f094ce82b80 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2f4740 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f0921600700 0 client.15435 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:46.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f094ce82b80 1 --2- 172.21.15.90:0/304331887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de257a70 0x5636de2f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.962+0000 7f094ce82b80 1 -- 172.21.15.90:0/304331887 >> 172.21.15.90:0/304331887 conn(0x5636de205410 msgr2=0x5636de2ea390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:46.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.963+0000 7f094ce82b80 1 -- 172.21.15.90:0/304331887 shutdown_connections 2024-05-27T07:46:46.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.963+0000 7f094ce82b80 1 -- 172.21.15.90:0/304331887 wait complete. 2024-05-27T07:46:46.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.963+0000 7f094ce82b80 20 client.15435 trim_cache size 0 max 16384 2024-05-27T07:46:46.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.985+0000 7f094ce82b80 1 Processor -- start 2024-05-27T07:46:46.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.985+0000 7f094ce82b80 1 -- start start 2024-05-27T07:46:46.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de16bec0 0x5636de2f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2eff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de2eaef0 0x5636de2ee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f094ce82b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f091800ecc0 con 0x5636de2eaef0 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f094ce82b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0918010800 con 0x5636de16bec0 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f094ce82b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f091800f390 con 0x5636de257a70 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f0937e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de2eaef0 0x5636de2ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f0937e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de2eaef0 0x5636de2ee670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56280/0 (socket says 172.21.15.90:56280) 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f0937e00700 1 -- 172.21.15.90:0/2932925972 learned_addr learned my addr 172.21.15.90:0/2932925972 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f0936a00700 1 --2- 172.21.15.90:0/2932925972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f0937400700 1 --2- 172.21.15.90:0/2932925972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de16bec0 0x5636de2f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f0937e00700 1 -- 172.21.15.90:0/2932925972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 msgr2=0x5636de2eff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:46.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f0937e00700 1 --2- 172.21.15.90:0/2932925972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2eff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f0937e00700 1 -- 172.21.15.90:0/2932925972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de16bec0 msgr2=0x5636de2f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:46.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f0937e00700 1 --2- 172.21.15.90:0/2932925972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de16bec0 0x5636de2f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.986+0000 7f0937e00700 1 -- 172.21.15.90:0/2932925972 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636de2e22c0 con 0x5636de2eaef0 2024-05-27T07:46:46.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.987+0000 7f0936a00700 1 --2- 172.21.15.90:0/2932925972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:46:46.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.987+0000 7f0937e00700 1 --2- 172.21.15.90:0/2932925972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de2eaef0 0x5636de2ee670 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0930018930 tx=0x7f0930017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2bb9d304be2252 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:46.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.987+0000 7f0923e00700 1 -- 172.21.15.90:0/2932925972 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930016570 con 0x5636de2eaef0 2024-05-27T07:46:46.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.987+0000 7f0923e00700 1 -- 172.21.15.90:0/2932925972 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0930027070 con 0x5636de2eaef0 2024-05-27T07:46:46.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.987+0000 7f0923e00700 1 -- 172.21.15.90:0/2932925972 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930022b10 con 0x5636de2eaef0 2024-05-27T07:46:46.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.988+0000 7f094ce82b80 1 -- 172.21.15.90:0/2932925972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de2eaef0 msgr2=0x5636de2ee670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.988+0000 7f094ce82b80 1 --2- 172.21.15.90:0/2932925972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de2eaef0 0x5636de2ee670 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0930018930 tx=0x7f0930017000 comp rx=0 tx=0).stop 2024-05-27T07:46:46.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.988+0000 7f094ce82b80 1 -- 172.21.15.90:0/2932925972 shutdown_connections 2024-05-27T07:46:46.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.988+0000 7f094ce82b80 1 --2- 172.21.15.90:0/2932925972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de2eaef0 0x5636de2ee670 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.988+0000 7f094ce82b80 1 --2- 172.21.15.90:0/2932925972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de16bec0 0x5636de2f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.988+0000 7f094ce82b80 1 --2- 172.21.15.90:0/2932925972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.988+0000 7f094ce82b80 1 -- 172.21.15.90:0/2932925972 >> 172.21.15.90:0/2932925972 conn(0x5636de205410 msgr2=0x5636de2edf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:46.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.989+0000 7f094ce82b80 1 -- 172.21.15.90:0/2932925972 shutdown_connections 2024-05-27T07:46:46.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.989+0000 7f094ce82b80 1 -- 172.21.15.90:0/2932925972 wait complete. 2024-05-27T07:46:46.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.989+0000 7f094ce82b80 1 -- start start 2024-05-27T07:46:46.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.989+0000 7f094ce82b80 1 Processor -- start 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2fd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2f8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f094ce82b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de2eaef0 0x5636de2f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f094ce82b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0918010130 con 0x5636de16bec0 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f094ce82b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f091800fa60 con 0x5636de2eaef0 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f094ce82b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f091800cc40 con 0x5636de257a70 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f094ce82b80 10 client.0 _ll_register_callbacks cb 0x5636de1b2cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f0937400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2fd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f0937400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2fd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56290/0 (socket says 172.21.15.90:56290) 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f0937400700 1 -- 172.21.15.90:0/3250450730 learned_addr learned my addr 172.21.15.90:0/3250450730 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.990+0000 7f0936a00700 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f0937e00700 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de2eaef0 0x5636de2f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f0937400700 1 -- 172.21.15.90:0/3250450730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 msgr2=0x5636de2f8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:46.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f0937400700 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2f8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f0937400700 1 -- 172.21.15.90:0/3250450730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de2eaef0 msgr2=0x5636de2f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:46.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f0937400700 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de2eaef0 0x5636de2f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f0937400700 1 -- 172.21.15.90:0/3250450730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636de2f96b0 con 0x5636de16bec0 2024-05-27T07:46:46.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f0937400700 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2fd610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f09280c3f50 tx=0x7f09280dfd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c7508497d1c922 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:46.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f0921600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:46.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f0921600700 1 -- 172.21.15.90:0/3250450730 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09280c3040 con 0x5636de16bec0 2024-05-27T07:46:46.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f0921600700 1 -- 172.21.15.90:0/3250450730 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f09280e23c0 con 0x5636de16bec0 2024-05-27T07:46:46.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.991+0000 7f094ce82b80 1 -- 172.21.15.90:0/3250450730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0930027070 con 0x5636de16bec0 2024-05-27T07:46:46.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f0921600700 1 -- 172.21.15.90:0/3250450730 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f09280a5180 con 0x5636de16bec0 2024-05-27T07:46:46.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f094ce82b80 10 client.15441 fetch_fsmap learned FSMap version 30 2024-05-27T07:46:46.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f094ce82b80 1 -- 172.21.15.90:0/3250450730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636de36f560 con 0x5636de16bec0 2024-05-27T07:46:46.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f0921600700 1 -- 172.21.15.90:0/3250450730 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0930027070 con 0x5636de16bec0 2024-05-27T07:46:46.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f0921600700 1 -- 172.21.15.90:0/3250450730 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f09280a8520 con 0x5636de16bec0 2024-05-27T07:46:46.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f094ce82b80 10 client.15441 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:46:46.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f094ce82b80 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:46.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f094ce82b80 20 client.15441 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:46.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f094ce82b80 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5636de36fc60 0x5636de372080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:46.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f094ce82b80 4 client.15441 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:46:46.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.992+0000 7f094ce82b80 1 -- 172.21.15.90:0/3250450730 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5636de2145f0 con 0x5636de36fc60 2024-05-27T07:46:46.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.993+0000 7f0936a00700 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5636de36fc60 0x5636de372080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:46.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.994+0000 7f0936a00700 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5636de36fc60 0x5636de372080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa06715811e09e09 server_cookie=51af3263436ed5a6 in_seq=0 out_seq=0 2024-05-27T07:46:46.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.994+0000 7f0921600700 10 client.15441 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:46.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f0921600700 1 -- 172.21.15.90:0/3250450730 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5636de2145f0 con 0x5636de36fc60 2024-05-27T07:46:46.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f0921600700 10 client.15441 handle_command_reply: tid=0 2024-05-27T07:46:46.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f094ce82b80 1 client.15441 shutdown 2024-05-27T07:46:46.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f094ce82b80 1 -- 172.21.15.90:0/3250450730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 msgr2=0x5636de2fd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:46.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f094ce82b80 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2fd610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f09280c3f50 tx=0x7f09280dfd10 comp rx=0 tx=0).stop 2024-05-27T07:46:46.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f094ce82b80 1 -- 172.21.15.90:0/3250450730 shutdown_connections 2024-05-27T07:46:46.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f094ce82b80 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5636de36fc60 0x5636de372080 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f094ce82b80 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5636de2eaef0 0x5636de2f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f094ce82b80 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5636de16bec0 0x5636de2fd610 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f0921600700 0 client.15441 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:46.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f094ce82b80 1 --2- 172.21.15.90:0/3250450730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636de257a70 0x5636de2f8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:46.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.998+0000 7f094ce82b80 1 -- 172.21.15.90:0/3250450730 >> 172.21.15.90:0/3250450730 conn(0x5636de205410 msgr2=0x5636de2f37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:47.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.999+0000 7f094ce82b80 1 -- 172.21.15.90:0/3250450730 shutdown_connections 2024-05-27T07:46:47.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.999+0000 7f094ce82b80 1 -- 172.21.15.90:0/3250450730 wait complete. 2024-05-27T07:46:47.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.999+0000 7f094ce82b80 20 client.15441 trim_cache size 0 max 16384 2024-05-27T07:46:47.000 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:46:47.000 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:46:47.000 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "ed33bed2-a406-446b-9c32-2a124d3a7255", 2024-05-27T07:46:47.000 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:46:47.000 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:46:47.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.999+0000 7f093e400700 1 -- 172.21.15.90:0/4002027089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f091c0b2b90 msgr2=0x7f091c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:47.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.999+0000 7f093e400700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f091c0b2b90 0x7f091c0b50a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0930009cb0 tx=0x7f09300077e0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:46.999+0000 7f093e400700 1 -- 172.21.15.90:0/4002027089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f093810a980 msgr2=0x7f09381a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:47.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.000+0000 7f093e400700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f093810a980 0x7f09381a13b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f092800d980 tx=0x7f092800dd40 comp rx=0 tx=0).stop 2024-05-27T07:46:47.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.000+0000 7f093e400700 1 -- 172.21.15.90:0/4002027089 shutdown_connections 2024-05-27T07:46:47.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.000+0000 7f093e400700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0938110960 0x7f09381a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.000+0000 7f093e400700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f093810a980 0x7f09381a13b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.000+0000 7f093e400700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f091c0b2b90 0x7f091c0b50a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.000+0000 7f093e400700 1 --2- 172.21.15.90:0/4002027089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f093810b2d0 0x7f09381a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.000+0000 7f093e400700 1 -- 172.21.15.90:0/4002027089 >> 172.21.15.90:0/4002027089 conn(0x7f0938077c70 msgr2=0x7f093807b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:47.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.000+0000 7f093e400700 1 -- 172.21.15.90:0/4002027089 shutdown_connections 2024-05-27T07:46:47.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.000+0000 7f093e400700 1 -- 172.21.15.90:0/4002027089 wait complete. 2024-05-27T07:46:47.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:46 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3129993660' entity='client.admin' 2024-05-27T07:46:47.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:46 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:47.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:46 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:47.003 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:46 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:47.003 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:46 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:47.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.043+0000 7fb8e3a00640 1 -- 172.21.15.90:0/1392393582 --> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fb8dc111750 con 0x7fb8c40b2a70 2024-05-27T07:46:47.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:46 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3129993660' entity='client.admin' 2024-05-27T07:46:47.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:46 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:46:47.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:46 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:46:47.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:46 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:46:47.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:46 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:46:47.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.190+0000 7fb8d9600640 1 -- 172.21.15.90:0/1392393582 <== mgr.34967 v2:172.21.15.90:6800/4256436582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fb8dc111750 con 0x7fb8c40b2a70 2024-05-27T07:46:47.190 INFO:teuthology.orchestra.run.smithi090.stdout:Retention added to path / 2024-05-27T07:46:47.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 -- 172.21.15.90:0/1392393582 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8c40b2a70 msgr2=0x7fb8c40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:47.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8c40b2a70 0x7fb8c40b4f80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80058c0 tx=0x7fb8c8005850 comp rx=0 tx=0).stop 2024-05-27T07:46:47.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 -- 172.21.15.90:0/1392393582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8dc1029a0 msgr2=0x7fb8dc1a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:47.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8dc1029a0 0x7fb8dc1a2590 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009dc0 tx=0x7fb8d0032bc0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 -- 172.21.15.90:0/1392393582 shutdown_connections 2024-05-27T07:46:47.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8c40b2a70 0x7fb8c40b4f80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8dc110540 0x7fb8dc19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8dc10dbe0 0x7fb8dc1a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 --2- 172.21.15.90:0/1392393582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8dc1029a0 0x7fb8dc1a2590 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 -- 172.21.15.90:0/1392393582 >> 172.21.15.90:0/1392393582 conn(0x7fb8dc0782d0 msgr2=0x7fb8dc111040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:47.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 -- 172.21.15.90:0/1392393582 shutdown_connections 2024-05-27T07:46:47.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.194+0000 7fb8e3a00640 1 -- 172.21.15.90:0/1392393582 wait complete. 2024-05-27T07:46:47.222 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-27T07:46:47.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.481+0000 7fea59c00640 1 -- 172.21.15.90:0/2397917836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea54108b80 msgr2=0x7fea54102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:47.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.481+0000 7fea59c00640 1 --2- 172.21.15.90:0/2397917836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea54108b80 0x7fea54102720 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fea44009900 tx=0x7fea4402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.481+0000 7fea59c00640 1 -- 172.21.15.90:0/2397917836 shutdown_connections 2024-05-27T07:46:47.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.481+0000 7fea59c00640 1 --2- 172.21.15.90:0/2397917836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea54109020 0x7fea54102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.481+0000 7fea59c00640 1 --2- 172.21.15.90:0/2397917836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541032d0 0x7fea5410f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.481+0000 7fea59c00640 1 --2- 172.21.15.90:0/2397917836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea54108b80 0x7fea54102720 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.481+0000 7fea59c00640 1 -- 172.21.15.90:0/2397917836 >> 172.21.15.90:0/2397917836 conn(0x7fea540fe380 msgr2=0x7fea541007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:47.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.481+0000 7fea59c00640 1 -- 172.21.15.90:0/2397917836 shutdown_connections 2024-05-27T07:46:47.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.481+0000 7fea59c00640 1 -- 172.21.15.90:0/2397917836 wait complete. 2024-05-27T07:46:47.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.483+0000 7fea59c00640 1 Processor -- start 2024-05-27T07:46:47.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.483+0000 7fea59c00640 1 -- start start 2024-05-27T07:46:47.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea59c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541032d0 0x7fea540fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:47.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea59c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea54108b80 0x7fea540feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:47.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea59c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea54109020 0x7fea540ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:47.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea59c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fea54112c20 con 0x7fea54108b80 2024-05-27T07:46:47.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea59c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fea54112aa0 con 0x7fea54109020 2024-05-27T07:46:47.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea59c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea54112da0 con 0x7fea541032d0 2024-05-27T07:46:47.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea52000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea54108b80 0x7fea540feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:47.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea52000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea54108b80 0x7fea540feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56302/0 (socket says 172.21.15.90:56302) 2024-05-27T07:46:47.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea52000640 1 -- 172.21.15.90:0/76404244 learned_addr learned my addr 172.21.15.90:0/76404244 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:47.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea52a00640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541032d0 0x7fea540fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:47.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea53400640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea54109020 0x7fea540ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:47.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea52000640 1 -- 172.21.15.90:0/76404244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541032d0 msgr2=0x7fea540fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:47.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea52000640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541032d0 0x7fea540fe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.484+0000 7fea52000640 1 -- 172.21.15.90:0/76404244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea54109020 msgr2=0x7fea540ff2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:47.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.485+0000 7fea52000640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea54109020 0x7fea540ff2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.485+0000 7fea52000640 1 -- 172.21.15.90:0/76404244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea541a8c00 con 0x7fea54108b80 2024-05-27T07:46:47.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.485+0000 7fea52000640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea54108b80 0x7fea540feda0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c970 tx=0x7fea4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:47.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.485+0000 7fea4b400640 1 -- 172.21.15.90:0/76404244 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40004220 con 0x7fea54108b80 2024-05-27T07:46:47.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.485+0000 7fea4b400640 1 -- 172.21.15.90:0/76404244 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea40007e40 con 0x7fea54108b80 2024-05-27T07:46:47.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.485+0000 7fea4b400640 1 -- 172.21.15.90:0/76404244 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4000f450 con 0x7fea54108b80 2024-05-27T07:46:47.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.485+0000 7fea59c00640 1 -- 172.21.15.90:0/76404244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea541a8ef0 con 0x7fea54108b80 2024-05-27T07:46:47.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.485+0000 7fea59c00640 1 -- 172.21.15.90:0/76404244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea541a9360 con 0x7fea54108b80 2024-05-27T07:46:47.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.487+0000 7fea59c00640 1 -- 172.21.15.90:0/76404244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea54050860 con 0x7fea54108b80 2024-05-27T07:46:47.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.492+0000 7fea4b400640 1 -- 172.21.15.90:0/76404244 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fea400079b0 con 0x7fea54108b80 2024-05-27T07:46:47.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.494+0000 7fea4b400640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fea380b2a70 0x7fea380b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:47.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.494+0000 7fea4b400640 1 -- 172.21.15.90:0/76404244 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fea400e2d40 con 0x7fea54108b80 2024-05-27T07:46:47.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.494+0000 7fea52a00640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fea380b2a70 0x7fea380b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:47.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.495+0000 7fea52a00640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fea380b2a70 0x7fea380b4f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea44006010 tx=0x7fea44031c70 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:47.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.496+0000 7fea4b400640 1 -- 172.21.15.90:0/76404244 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea40014030 con 0x7fea54108b80 2024-05-27T07:46:47.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.729+0000 7fea59c00640 1 -- 172.21.15.90:0/76404244 --> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fea54101450 con 0x7fea380b2a70 2024-05-27T07:46:47.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.746+0000 7fea4b400640 1 -- 172.21.15.90:0/76404244 <== mgr.34967 v2:172.21.15.90:6800/4256436582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fea54101450 con 0x7fea380b2a70 2024-05-27T07:46:47.747 INFO:teuthology.orchestra.run.smithi090.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-27T00:00:00", "created": "2024-05-27T07:46:46", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-27T07:46:47.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 -- 172.21.15.90:0/76404244 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fea380b2a70 msgr2=0x7fea380b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fea380b2a70 0x7fea380b4f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea44006010 tx=0x7fea44031c70 comp rx=0 tx=0).stop 2024-05-27T07:46:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 -- 172.21.15.90:0/76404244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea54108b80 msgr2=0x7fea540feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea54108b80 0x7fea540feda0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c970 tx=0x7fea4000ce30 comp rx=0 tx=0).stop 2024-05-27T07:46:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 -- 172.21.15.90:0/76404244 shutdown_connections 2024-05-27T07:46:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fea380b2a70 0x7fea380b4f80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea54109020 0x7fea540ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea54108b80 0x7fea540feda0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 --2- 172.21.15.90:0/76404244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea541032d0 0x7fea540fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 -- 172.21.15.90:0/76404244 >> 172.21.15.90:0/76404244 conn(0x7fea540fe380 msgr2=0x7fea54100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:47.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.749+0000 7fea59c00640 1 -- 172.21.15.90:0/76404244 shutdown_connections 2024-05-27T07:46:47.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:47.750+0000 7fea59c00640 1 -- 172.21.15.90:0/76404244 wait complete. 2024-05-27T07:46:47.777 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-27T07:46:47.953 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:46:47.998 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:47 smithi136 ceph-mon[35324]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:47.999 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:47 smithi136 ceph-mon[35324]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:46:47.999 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:47 smithi136 ceph-mon[35324]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:48.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:47 smithi090 ceph-mon[31429]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:48.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:47 smithi090 ceph-mon[31429]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:46:48.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:47 smithi090 ceph-mon[31429]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:48.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.040+0000 7f784d400640 1 -- 172.21.15.90:0/3728056920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f784810beb0 msgr2=0x7f7848105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.040+0000 7f784d400640 1 --2- 172.21.15.90:0/3728056920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f784810beb0 0x7f7848105a50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7838009ca0 tx=0x7f783802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.040+0000 7f784d400640 1 -- 172.21.15.90:0/3728056920 shutdown_connections 2024-05-27T07:46:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.040+0000 7f784d400640 1 --2- 172.21.15.90:0/3728056920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7848106600 0x7f78481129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.040+0000 7f784d400640 1 --2- 172.21.15.90:0/3728056920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f784810c350 0x7f7848105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.040+0000 7f784d400640 1 --2- 172.21.15.90:0/3728056920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f784810beb0 0x7f7848105a50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.040+0000 7f784d400640 1 -- 172.21.15.90:0/3728056920 >> 172.21.15.90:0/3728056920 conn(0x7f7848101530 msgr2=0x7f7848103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.040+0000 7f784d400640 1 -- 172.21.15.90:0/3728056920 shutdown_connections 2024-05-27T07:46:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.040+0000 7f784d400640 1 -- 172.21.15.90:0/3728056920 wait complete. 2024-05-27T07:46:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.041+0000 7f784d400640 1 Processor -- start 2024-05-27T07:46:48.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.041+0000 7f784d400640 1 -- start start 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.042+0000 7f784d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7848106600 0x7f78481a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.042+0000 7f784d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f784810beb0 0x7f78481a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.042+0000 7f784d400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f784810c350 0x7f784819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.042+0000 7f784d400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7848115fb0 con 0x7f784810beb0 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.042+0000 7f784d400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7848115e30 con 0x7f784810c350 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.042+0000 7f784d400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7848116130 con 0x7f7848106600 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.042+0000 7f783de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f784810beb0 0x7f78481a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.042+0000 7f783de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f784810beb0 0x7f78481a2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56332/0 (socket says 172.21.15.90:56332) 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.042+0000 7f783de00640 1 -- 172.21.15.90:0/2428287210 learned_addr learned my addr 172.21.15.90:0/2428287210 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f7846800640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7848106600 0x7f78481a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f7847200640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f784810c350 0x7f784819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:48.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f783de00640 1 -- 172.21.15.90:0/2428287210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7848106600 msgr2=0x7f78481a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f783de00640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7848106600 0x7f78481a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f783de00640 1 -- 172.21.15.90:0/2428287210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f784810c350 msgr2=0x7f784819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f783de00640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f784810c350 0x7f784819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f783de00640 1 -- 172.21.15.90:0/2428287210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784819cec0 con 0x7f784810beb0 2024-05-27T07:46:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f783de00640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f784810beb0 0x7f78481a2c00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7830005d00 tx=0x7f783000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f783fe00640 1 -- 172.21.15.90:0/2428287210 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830017070 con 0x7f784810beb0 2024-05-27T07:46:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f783fe00640 1 -- 172.21.15.90:0/2428287210 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7830009c30 con 0x7f784810beb0 2024-05-27T07:46:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f784d400640 1 -- 172.21.15.90:0/2428287210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784819d1b0 con 0x7f784810beb0 2024-05-27T07:46:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.043+0000 7f783fe00640 1 -- 172.21.15.90:0/2428287210 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783000f620 con 0x7f784810beb0 2024-05-27T07:46:48.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.044+0000 7f784d400640 1 -- 172.21.15.90:0/2428287210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784819d6b0 con 0x7f784810beb0 2024-05-27T07:46:48.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.045+0000 7f784d400640 1 -- 172.21.15.90:0/2428287210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7848050860 con 0x7f784810beb0 2024-05-27T07:46:48.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.047+0000 7f783fe00640 1 -- 172.21.15.90:0/2428287210 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f783001a070 con 0x7f784810beb0 2024-05-27T07:46:48.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.049+0000 7f783fe00640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f782c0b2a70 0x7f782c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:48.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.049+0000 7f783fe00640 1 -- 172.21.15.90:0/2428287210 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f78300e2670 con 0x7f784810beb0 2024-05-27T07:46:48.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.049+0000 7f7846800640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f782c0b2a70 0x7f782c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:48.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.050+0000 7f7846800640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f782c0b2a70 0x7f782c0b4f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7838006010 tx=0x7f7838000f40 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.052+0000 7f783fe00640 1 -- 172.21.15.90:0/2428287210 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7830009de0 con 0x7f784810beb0 2024-05-27T07:46:48.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:47 smithi129 ceph-mon[35254]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:48.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:47 smithi129 ceph-mon[35254]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T07:46:48.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:47 smithi129 ceph-mon[35254]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.290+0000 7f784d400640 1 -- 172.21.15.90:0/2428287210 --> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7848112120 con 0x7f782c0b2a70 2024-05-27T07:46:48.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.302+0000 7f783fe00640 1 -- 172.21.15.90:0/2428287210 <== mgr.34967 v2:172.21.15.90:6800/4256436582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7848112120 con 0x7f782c0b2a70 2024-05-27T07:46:48.302 INFO:teuthology.orchestra.run.smithi090.stdout:/ 1m 6m3h 2024-05-27T07:46:48.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 -- 172.21.15.90:0/2428287210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f782c0b2a70 msgr2=0x7f782c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:48.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f782c0b2a70 0x7f782c0b4f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7838006010 tx=0x7f7838000f40 comp rx=0 tx=0).stop 2024-05-27T07:46:48.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 -- 172.21.15.90:0/2428287210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f784810beb0 msgr2=0x7f78481a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:48.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f784810beb0 0x7f78481a2c00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7830005d00 tx=0x7f783000dde0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 -- 172.21.15.90:0/2428287210 shutdown_connections 2024-05-27T07:46:48.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f782c0b2a70 0x7f782c0b4f80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f784810c350 0x7f784819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f784810beb0 0x7f78481a2c00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 --2- 172.21.15.90:0/2428287210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7848106600 0x7f78481a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:48.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 -- 172.21.15.90:0/2428287210 >> 172.21.15.90:0/2428287210 conn(0x7f7848101530 msgr2=0x7f7848111a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:48.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 -- 172.21.15.90:0/2428287210 shutdown_connections 2024-05-27T07:46:48.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:48.305+0000 7f784d400640 1 -- 172.21.15.90:0/2428287210 wait complete. 2024-05-27T07:46:48.331 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-27T07:46:48.378 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:46:48.402 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-27T07:46:48.413 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-27T07:46:48.484 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T07:46:48.484 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-27T07:46:48.491 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T07:46:48.541 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T07:46:48.541 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T07:46:48.541 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-05-27T07:46:48.639 INFO:teuthology.orchestra.run.smithi090.stdout:total 0 2024-05-27T07:46:48.680 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem Size Used Avail Use% Mounted on 2024-05-27T07:46:48.680 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-fuse 1.1T 356M 1.1T 1% /tmp/cephfs 2024-05-27T07:46:48.686 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-05-27T07:46:48.840 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-05-27T07:46:48.899 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:48 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:46:48.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:48 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:48.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:48 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:46:48.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:48 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:48.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:48 smithi090 ceph-mon[31429]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:48.948 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-05-27T07:46:49.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:48 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:46:49.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:48 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:49.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:48 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:46:49.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:48 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:49.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:48 smithi129 ceph-mon[35254]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:49.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:48 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:46:49.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:48 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:49.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:48 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:46:49.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:48 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:46:49.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:48 smithi136 ceph-mon[35324]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:50.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:49 smithi129 ceph-mon[35254]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:50.063 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:49 smithi129 ceph-mon[35254]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-05-27T07:46:50.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:49 smithi090 ceph-mon[31429]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:50.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:49 smithi090 ceph-mon[31429]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-05-27T07:46:50.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:49 smithi136 ceph-mon[35324]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T07:46:50.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:49 smithi136 ceph-mon[35324]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-05-27T07:46:51.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.532+0000 7fddaa800700 1 -- 172.21.15.90:0/224088517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdda4076420 msgr2=0x7fdda41106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.532+0000 7fddaa800700 1 --2- 172.21.15.90:0/224088517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdda4076420 0x7fdda41106c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009990 tx=0x7fdd8c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.532+0000 7fddaa800700 1 -- 172.21.15.90:0/224088517 shutdown_connections 2024-05-27T07:46:51.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.532+0000 7fddaa800700 1 --2- 172.21.15.90:0/224088517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4110c00 0x7fdda4113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.532+0000 7fddaa800700 1 --2- 172.21.15.90:0/224088517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdda4075ad0 0x7fdda4075ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.532+0000 7fddaa800700 1 --2- 172.21.15.90:0/224088517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdda4076420 0x7fdda41106c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.532+0000 7fddaa800700 1 -- 172.21.15.90:0/224088517 >> 172.21.15.90:0/224088517 conn(0x7fdda40ffe60 msgr2=0x7fdda41022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.532+0000 7fddaa800700 1 -- 172.21.15.90:0/224088517 shutdown_connections 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.532+0000 7fddaa800700 1 -- 172.21.15.90:0/224088517 wait complete. 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.533+0000 7fddaa800700 1 Processor -- start 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.533+0000 7fddaa800700 1 -- start start 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.534+0000 7fddaa800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdda4075ad0 0x7fdda419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.534+0000 7fddaa800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdda4076420 0x7fdda419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.534+0000 7fddaa800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4110c00 0x7fdda41a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.534+0000 7fddaa800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4116560 con 0x7fdda4075ad0 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.534+0000 7fddaa800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41163e0 con 0x7fdda4110c00 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.534+0000 7fddaa800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41166e0 con 0x7fdda4076420 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.534+0000 7fdda3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdda4075ad0 0x7fdda419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.534+0000 7fdda3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdda4075ad0 0x7fdda419cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44796/0 (socket says 172.21.15.90:44796) 2024-05-27T07:46:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.534+0000 7fdda3400700 1 -- 172.21.15.90:0/569347292 learned_addr learned my addr 172.21.15.90:0/569347292 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.535+0000 7fdd9be00700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdda4076420 0x7fdda419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.535+0000 7fdda3e00700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4110c00 0x7fdda41a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.535+0000 7fdda3400700 1 -- 172.21.15.90:0/569347292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdda4076420 msgr2=0x7fdda419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.535+0000 7fdda3400700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdda4076420 0x7fdda419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.535+0000 7fdda3400700 1 -- 172.21.15.90:0/569347292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4110c00 msgr2=0x7fdda41a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.535+0000 7fdda3400700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4110c00 0x7fdda41a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.535+0000 7fdda3400700 1 -- 172.21.15.90:0/569347292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda41a1ea0 con 0x7fdda4075ad0 2024-05-27T07:46:51.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.535+0000 7fdda3400700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdda4075ad0 0x7fdda419cfd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400e9a0 tx=0x7fdd9400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.536+0000 7fdda0c00700 1 -- 172.21.15.90:0/569347292 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400cbd0 con 0x7fdda4075ad0 2024-05-27T07:46:51.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.536+0000 7fdda0c00700 1 -- 172.21.15.90:0/569347292 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd9400cd70 con 0x7fdda4075ad0 2024-05-27T07:46:51.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.536+0000 7fdda0c00700 1 -- 172.21.15.90:0/569347292 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94018680 con 0x7fdda4075ad0 2024-05-27T07:46:51.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.536+0000 7fddaa800700 1 -- 172.21.15.90:0/569347292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda41a2190 con 0x7fdda4075ad0 2024-05-27T07:46:51.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.536+0000 7fddaa800700 1 -- 172.21.15.90:0/569347292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda41a9ab0 con 0x7fdda4075ad0 2024-05-27T07:46:51.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.540+0000 7fdda0c00700 1 -- 172.21.15.90:0/569347292 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fdd94010950 con 0x7fdda4075ad0 2024-05-27T07:46:51.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.541+0000 7fddb9360b80 1 Processor -- start 2024-05-27T07:46:51.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.541+0000 7fddb9360b80 1 -- start start 2024-05-27T07:46:51.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 0x564449b8d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449a0ee40 0x564449b8d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 0x564449b9a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fddb9360b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564449abbe40 con 0x564449a0ee40 2024-05-27T07:46:51.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fddb9360b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564449aab9e0 con 0x564449afa9f0 2024-05-27T07:46:51.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fddb9360b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564449aafa20 con 0x564449b8de70 2024-05-27T07:46:51.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fdda0c00700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdd880b2b10 0x7fdd880b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fdda3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449a0ee40 0x564449b8d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fdda3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449a0ee40 0x564449b8d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44806/0 (socket says 172.21.15.90:44806) 2024-05-27T07:46:51.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fdda3400700 1 -- 172.21.15.90:0/3135410097 learned_addr learned my addr 172.21.15.90:0/3135410097 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:51.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fdda3e00700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdd880b2b10 0x7fdd880b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.542+0000 7fdda0c00700 1 -- 172.21.15.90:0/569347292 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fdd940e2fb0 con 0x7fdda4075ad0 2024-05-27T07:46:51.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.543+0000 7fdd9be00700 1 --2- 172.21.15.90:0/3135410097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 0x564449b9a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.543+0000 7fdda3e00700 1 --2- 172.21.15.90:0/3135410097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 0x564449b8d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.543+0000 7fdda3400700 1 -- 172.21.15.90:0/3135410097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 msgr2=0x564449b9a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.543+0000 7fdda3400700 1 --2- 172.21.15.90:0/3135410097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 0x564449b9a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.543+0000 7fdda3400700 1 -- 172.21.15.90:0/3135410097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 msgr2=0x564449b8d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.543+0000 7fdda3400700 1 --2- 172.21.15.90:0/3135410097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 0x564449b8d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.543+0000 7fdda3400700 1 -- 172.21.15.90:0/3135410097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564449a74420 con 0x564449a0ee40 2024-05-27T07:46:51.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.543+0000 7fdda3e00700 1 --2- 172.21.15.90:0/3135410097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 0x564449b8d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:51.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.543+0000 7fdda3400700 1 --2- 172.21.15.90:0/3135410097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449a0ee40 0x564449b8d870 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdd940a5d10 tx=0x7fdd94077860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53d83033dd62525 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.544+0000 7fdda3e00700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdd880b2b10 0x7fdd880b5020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x564449b8fff0 tx=0x7fdd9c009920 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.544+0000 7fdd98c00700 1 -- 172.21.15.90:0/3135410097 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94026040 con 0x564449a0ee40 2024-05-27T07:46:51.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.544+0000 7fdd98c00700 1 -- 172.21.15.90:0/3135410097 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd940b2430 con 0x564449a0ee40 2024-05-27T07:46:51.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.544+0000 7fdd98c00700 1 -- 172.21.15.90:0/3135410097 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd940aa150 con 0x564449a0ee40 2024-05-27T07:46:51.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.545+0000 7fddb9360b80 1 -- 172.21.15.90:0/3135410097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449a0ee40 msgr2=0x564449b8d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.545+0000 7fddb9360b80 1 --2- 172.21.15.90:0/3135410097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449a0ee40 0x564449b8d870 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdd940a5d10 tx=0x7fdd94077860 comp rx=0 tx=0).stop 2024-05-27T07:46:51.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.545+0000 7fddb9360b80 1 -- 172.21.15.90:0/3135410097 shutdown_connections 2024-05-27T07:46:51.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.545+0000 7fddb9360b80 1 --2- 172.21.15.90:0/3135410097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 0x564449b9a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.545+0000 7fddb9360b80 1 --2- 172.21.15.90:0/3135410097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 0x564449b8d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.545+0000 7fddb9360b80 1 --2- 172.21.15.90:0/3135410097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449a0ee40 0x564449b8d870 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.545+0000 7fddb9360b80 1 -- 172.21.15.90:0/3135410097 >> 172.21.15.90:0/3135410097 conn(0x564449aa8390 msgr2=0x564449b894c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.545+0000 7fddb9360b80 1 -- 172.21.15.90:0/3135410097 shutdown_connections 2024-05-27T07:46:51.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.545+0000 7fddb9360b80 1 -- 172.21.15.90:0/3135410097 wait complete. 2024-05-27T07:46:51.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.546+0000 7fddb9360b80 1 -- start start 2024-05-27T07:46:51.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.546+0000 7fddb9360b80 1 Processor -- start 2024-05-27T07:46:51.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 0x564449b97790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449afa9f0 0x564449b97cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 0x564449b98210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fddb9360b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5644499b1fa0 con 0x564449afa9f0 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fddb9360b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564449a4c760 con 0x564449a0ee40 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fddb9360b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564449b9aac0 con 0x564449b8de70 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fddb9360b80 10 client.0 _ll_register_callbacks cb 0x564449a55c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fdd9be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449afa9f0 0x564449b97cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fdd9be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449afa9f0 0x564449b97cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44808/0 (socket says 172.21.15.90:44808) 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fdd9be00700 1 -- 172.21.15.90:0/1444122450 learned_addr learned my addr 172.21.15.90:0/1444122450 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.547+0000 7fdda3400700 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 0x564449b97790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.548+0000 7fdda3e00700 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 0x564449b98210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.548+0000 7fdd9be00700 1 -- 172.21.15.90:0/1444122450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 msgr2=0x564449b98210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.548+0000 7fdd9be00700 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 0x564449b98210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.548+0000 7fdd9be00700 1 -- 172.21.15.90:0/1444122450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 msgr2=0x564449b97790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.548+0000 7fdd9be00700 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 0x564449b97790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.548+0000 7fdd9be00700 1 -- 172.21.15.90:0/1444122450 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644499fb8d0 con 0x564449afa9f0 2024-05-27T07:46:51.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.548+0000 7fdda3e00700 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 0x564449b98210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:51.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.548+0000 7fdd9be00700 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449afa9f0 0x564449b97cd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009990 tx=0x7fdd8c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6eb51a28af5631 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.548+0000 7fdd91600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:51.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.549+0000 7fdd91600700 1 -- 172.21.15.90:0/1444122450 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c0095d0 con 0x564449afa9f0 2024-05-27T07:46:51.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.549+0000 7fddb9360b80 1 -- 172.21.15.90:0/1444122450 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564449abdd90 con 0x564449afa9f0 2024-05-27T07:46:51.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.549+0000 7fdd91600700 1 -- 172.21.15.90:0/1444122450 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fdd8c00b6b0 con 0x564449afa9f0 2024-05-27T07:46:51.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.549+0000 7fdd91600700 1 -- 172.21.15.90:0/1444122450 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdd8c030050 con 0x564449afa9f0 2024-05-27T07:46:51.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.549+0000 7fddb9360b80 10 client.15465 fetch_fsmap learned FSMap version 30 2024-05-27T07:46:51.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.549+0000 7fddb9360b80 1 -- 172.21.15.90:0/1444122450 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564449a7c060 con 0x564449afa9f0 2024-05-27T07:46:51.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.549+0000 7fdd91600700 1 -- 172.21.15.90:0/1444122450 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x564449abdd90 con 0x564449afa9f0 2024-05-27T07:46:51.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.550+0000 7fdd91600700 1 -- 172.21.15.90:0/1444122450 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fdd8c00f750 con 0x564449afa9f0 2024-05-27T07:46:51.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.550+0000 7fddb9360b80 10 client.15465 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:46:51.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.550+0000 7fddb9360b80 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:51.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.550+0000 7fddb9360b80 20 client.15465 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:51.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.550+0000 7fddb9360b80 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564449ba7a40 0x564449ba9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.550+0000 7fddb9360b80 4 client.15465 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:46:51.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.550+0000 7fddb9360b80 1 -- 172.21.15.90:0/1444122450 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564449ab7570 con 0x564449ba7a40 2024-05-27T07:46:51.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.551+0000 7fdda3400700 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564449ba7a40 0x564449ba9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.552+0000 7fdd91600700 10 client.15465 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:51.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.552+0000 7fdda3400700 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564449ba7a40 0x564449ba9e60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec0856fbfeefdd17 server_cookie=87a50699a816cb8d in_seq=0 out_seq=0 2024-05-27T07:46:51.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fdd91600700 1 -- 172.21.15.90:0/1444122450 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564449ab7570 con 0x564449ba7a40 2024-05-27T07:46:51.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fdd91600700 10 client.15465 handle_command_reply: tid=0 2024-05-27T07:46:51.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 client.15465 shutdown 2024-05-27T07:46:51.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 -- 172.21.15.90:0/1444122450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449afa9f0 msgr2=0x564449b97cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449afa9f0 0x564449b97cd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009990 tx=0x7fdd8c005750 comp rx=0 tx=0).stop 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 -- 172.21.15.90:0/1444122450 shutdown_connections 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564449ba7a40 0x564449ba9e60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449b8de70 0x564449b98210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 0x564449b97790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 --2- 172.21.15.90:0/1444122450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449afa9f0 0x564449b97cd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 -- 172.21.15.90:0/1444122450 >> 172.21.15.90:0/1444122450 conn(0x564449aa8390 msgr2=0x564449b8d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fdd91600700 0 client.15465 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 -- 172.21.15.90:0/1444122450 shutdown_connections 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.554+0000 7fddb9360b80 1 -- 172.21.15.90:0/1444122450 wait complete. 2024-05-27T07:46:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.555+0000 7fddb9360b80 20 client.15465 trim_cache size 0 max 16384 2024-05-27T07:46:51.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.577+0000 7fddb9360b80 1 Processor -- start 2024-05-27T07:46:51.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.577+0000 7fddb9360b80 1 -- start start 2024-05-27T07:46:51.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 0x564449b966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449afa9f0 0x564449b92f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449ba3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fddb9360b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b26f0 con 0x564449b8de70 2024-05-27T07:46:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fddb9360b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b4230 con 0x564449a0ee40 2024-05-27T07:46:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fddb9360b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b2dc0 con 0x564449afa9f0 2024-05-27T07:46:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fdda3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449ba3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fdda3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449ba3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44824/0 (socket says 172.21.15.90:44824) 2024-05-27T07:46:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fdda3e00700 1 -- 172.21.15.90:0/2788425010 learned_addr learned my addr 172.21.15.90:0/2788425010 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fdd9be00700 1 --2- 172.21.15.90:0/2788425010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449afa9f0 0x564449b92f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.578+0000 7fdda3400700 1 --2- 172.21.15.90:0/2788425010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 0x564449b966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.579+0000 7fdda3e00700 1 -- 172.21.15.90:0/2788425010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449afa9f0 msgr2=0x564449b92f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.579+0000 7fdda3e00700 1 --2- 172.21.15.90:0/2788425010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449afa9f0 0x564449b92f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.579+0000 7fdda3e00700 1 -- 172.21.15.90:0/2788425010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 msgr2=0x564449b966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.579+0000 7fdda3e00700 1 --2- 172.21.15.90:0/2788425010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 0x564449b966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.579+0000 7fdda3e00700 1 -- 172.21.15.90:0/2788425010 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564449b85240 con 0x564449b8de70 2024-05-27T07:46:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.579+0000 7fdda3400700 1 --2- 172.21.15.90:0/2788425010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 0x564449b966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:46:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.579+0000 7fdda3e00700 1 --2- 172.21.15.90:0/2788425010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449ba3b60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009410 tx=0x7fdd9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edcc243633a19ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.580+0000 7fdd93e00700 1 -- 172.21.15.90:0/2788425010 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c007500 con 0x564449b8de70 2024-05-27T07:46:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.580+0000 7fdd93e00700 1 -- 172.21.15.90:0/2788425010 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd9c01d070 con 0x564449b8de70 2024-05-27T07:46:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.580+0000 7fdd93e00700 1 -- 172.21.15.90:0/2788425010 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c0197e0 con 0x564449b8de70 2024-05-27T07:46:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.581+0000 7fddb9360b80 1 -- 172.21.15.90:0/2788425010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 msgr2=0x564449ba3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.581+0000 7fddb9360b80 1 --2- 172.21.15.90:0/2788425010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449ba3b60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009410 tx=0x7fdd9c00b040 comp rx=0 tx=0).stop 2024-05-27T07:46:51.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.581+0000 7fddb9360b80 1 -- 172.21.15.90:0/2788425010 shutdown_connections 2024-05-27T07:46:51.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.581+0000 7fddb9360b80 1 --2- 172.21.15.90:0/2788425010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449ba3b60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.581+0000 7fddb9360b80 1 --2- 172.21.15.90:0/2788425010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449a0ee40 0x564449b966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.581+0000 7fddb9360b80 1 --2- 172.21.15.90:0/2788425010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449afa9f0 0x564449b92f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.581+0000 7fddb9360b80 1 -- 172.21.15.90:0/2788425010 >> 172.21.15.90:0/2788425010 conn(0x564449aa8390 msgr2=0x564449b8eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.581+0000 7fddb9360b80 1 -- 172.21.15.90:0/2788425010 shutdown_connections 2024-05-27T07:46:51.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.581+0000 7fddb9360b80 1 -- 172.21.15.90:0/2788425010 wait complete. 2024-05-27T07:46:51.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.582+0000 7fddb9360b80 1 -- start start 2024-05-27T07:46:51.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.582+0000 7fddb9360b80 1 Processor -- start 2024-05-27T07:46:51.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.582+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449a0ee40 0x564449ba0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.582+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 0x564449ba0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.582+0000 7fddb9360b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449b91c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fddb9360b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b3b60 con 0x564449b8de70 2024-05-27T07:46:51.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fddb9360b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b3490 con 0x564449afa9f0 2024-05-27T07:46:51.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fddb9360b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdda41b0670 con 0x564449a0ee40 2024-05-27T07:46:51.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fddb9360b80 10 client.0 _ll_register_callbacks cb 0x564449b88220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:46:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fdda3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449b91c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fdda3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449b91c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44838/0 (socket says 172.21.15.90:44838) 2024-05-27T07:46:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fdda3e00700 1 -- 172.21.15.90:0/3787428683 learned_addr learned my addr 172.21.15.90:0/3787428683 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:46:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fdd9be00700 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 0x564449ba0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fdda3e00700 1 -- 172.21.15.90:0/3787428683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449a0ee40 msgr2=0x564449ba0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fdda3e00700 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449a0ee40 0x564449ba0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fdda3e00700 1 -- 172.21.15.90:0/3787428683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 msgr2=0x564449ba0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fdda3e00700 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 0x564449ba0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.583+0000 7fdda3e00700 1 -- 172.21.15.90:0/3787428683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564449b98580 con 0x564449b8de70 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.584+0000 7fdda3e00700 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449b91c60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009040 tx=0x7fdd9c007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564d5446047b8d5b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.584+0000 7fdd91600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.584+0000 7fdd91600700 1 -- 172.21.15.90:0/3787428683 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c007700 con 0x564449b8de70 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.584+0000 7fdd91600700 1 -- 172.21.15.90:0/3787428683 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7fdd9c0078a0 con 0x564449b8de70 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.584+0000 7fddb9360b80 1 -- 172.21.15.90:0/3787428683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd9c01d070 con 0x564449b8de70 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.584+0000 7fdd91600700 1 -- 172.21.15.90:0/3787428683 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdd9c007aa0 con 0x564449b8de70 2024-05-27T07:46:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.585+0000 7fddb9360b80 10 client.15471 fetch_fsmap learned FSMap version 30 2024-05-27T07:46:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.585+0000 7fddb9360b80 1 -- 172.21.15.90:0/3787428683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564449b9f730 con 0x564449b8de70 2024-05-27T07:46:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.585+0000 7fdd91600700 1 -- 172.21.15.90:0/3787428683 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd9c01d070 con 0x564449b8de70 2024-05-27T07:46:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.585+0000 7fdd91600700 1 -- 172.21.15.90:0/3787428683 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fdd9c022070 con 0x564449b8de70 2024-05-27T07:46:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.585+0000 7fddb9360b80 10 client.15471 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:46:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.585+0000 7fddb9360b80 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:46:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.585+0000 7fddb9360b80 20 client.15471 populate_metadata read hostname 'smithi090' 2024-05-27T07:46:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.585+0000 7fddb9360b80 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564449c12530 0x564449c14950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:46:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.585+0000 7fddb9360b80 4 client.15471 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:46:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.585+0000 7fddb9360b80 1 -- 172.21.15.90:0/3787428683 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56444992e250 con 0x564449c12530 2024-05-27T07:46:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.586+0000 7fdda3400700 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564449c12530 0x564449c14950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:46:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.587+0000 7fdda3400700 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564449c12530 0x564449c14950 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f2362ec1be1b0f server_cookie=d25b4a9305902fc1 in_seq=0 out_seq=0 2024-05-27T07:46:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.587+0000 7fdd91600700 10 client.15471 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fdd91600700 1 -- 172.21.15.90:0/3787428683 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56444992e250 con 0x564449c12530 2024-05-27T07:46:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fdd91600700 10 client.15471 handle_command_reply: tid=0 2024-05-27T07:46:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 client.15471 shutdown 2024-05-27T07:46:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 -- 172.21.15.90:0/3787428683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 msgr2=0x564449b91c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449b91c60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009040 tx=0x7fdd9c007e70 comp rx=0 tx=0).stop 2024-05-27T07:46:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 -- 172.21.15.90:0/3787428683 shutdown_connections 2024-05-27T07:46:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564449c12530 0x564449c14950 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564449b8de70 0x564449b91c60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564449a0ee40 0x564449ba0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 --2- 172.21.15.90:0/3787428683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564449afa9f0 0x564449ba0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fdd91600700 0 client.15471 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:46:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 -- 172.21.15.90:0/3787428683 >> 172.21.15.90:0/3787428683 conn(0x564449aa8390 msgr2=0x564449b8eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 -- 172.21.15.90:0/3787428683 shutdown_connections 2024-05-27T07:46:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 1 -- 172.21.15.90:0/3787428683 wait complete. 2024-05-27T07:46:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.588+0000 7fddb9360b80 20 client.15471 trim_cache size 0 max 16384 2024-05-27T07:46:51.590 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:46:51.590 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:46:51.590 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": { 2024-05-27T07:46:51.590 INFO:teuthology.orchestra.run.smithi090.stdout: "ed33bed2-a406-446b-9c32-2a124d3a7255": { 2024-05-27T07:46:51.590 INFO:teuthology.orchestra.run.smithi090.stdout: "path": "/", 2024-05-27T07:46:51.590 INFO:teuthology.orchestra.run.smithi090.stdout: "tag": "ed33bed2-a406-446b-9c32-2a124d3a7255", 2024-05-27T07:46:51.590 INFO:teuthology.orchestra.run.smithi090.stdout: "options": "recursive,force" 2024-05-27T07:46:51.590 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T07:46:51.590 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T07:46:51.590 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:46:51.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.589+0000 7fddaa800700 1 -- 172.21.15.90:0/569347292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdd880b2b10 msgr2=0x7fdd880b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.589+0000 7fddaa800700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdd880b2b10 0x7fdd880b5020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x564449b8fff0 tx=0x7fdd9c009920 comp rx=0 tx=0).stop 2024-05-27T07:46:51.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.589+0000 7fddaa800700 1 -- 172.21.15.90:0/569347292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdda4075ad0 msgr2=0x7fdda419cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:46:51.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.589+0000 7fddaa800700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdda4075ad0 0x7fdda419cfd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400e9a0 tx=0x7fdd9400ed60 comp rx=0 tx=0).stop 2024-05-27T07:46:51.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.590+0000 7fddaa800700 1 -- 172.21.15.90:0/569347292 shutdown_connections 2024-05-27T07:46:51.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.590+0000 7fddaa800700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdd880b2b10 0x7fdd880b5020 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.590+0000 7fddaa800700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdda4110c00 0x7fdda41a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.590+0000 7fddaa800700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdda4075ad0 0x7fdda419cfd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.590+0000 7fddaa800700 1 --2- 172.21.15.90:0/569347292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdda4076420 0x7fdda419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:46:51.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.590+0000 7fddaa800700 1 -- 172.21.15.90:0/569347292 >> 172.21.15.90:0/569347292 conn(0x7fdda40ffe60 msgr2=0x7fdda4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:46:51.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.590+0000 7fddaa800700 1 -- 172.21.15.90:0/569347292 shutdown_connections 2024-05-27T07:46:51.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:46:51.590+0000 7fddaa800700 1 -- 172.21.15.90:0/569347292 wait complete. 2024-05-27T07:46:51.631 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-05-27T07:46:51.722 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-05-27T07:46:51.731 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-05-27T07:46:51.837 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-27T07:46:51.967 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/: 2024-05-27T07:46:51.967 INFO:teuthology.orchestra.run.smithi090.stdout:total 2 2024-05-27T07:46:51.967 INFO:teuthology.orchestra.run.smithi090.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:46:51.967 INFO:teuthology.orchestra.run.smithi090.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 .. 2024-05-27T07:46:51.967 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 data 2024-05-27T07:46:51.968 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:46:51.968 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data: 2024-05-27T07:46:51.968 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:46:51.968 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:46:51.968 INFO:teuthology.orchestra.run.smithi090.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 .. 2024-05-27T07:46:51.974 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-05-27T07:46:52.111 INFO:teuthology.orchestra.run.smithi090.stdout:[Service] 2024-05-27T07:46:52.111 INFO:teuthology.orchestra.run.smithi090.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-05-27T07:46:52.111 INFO:teuthology.orchestra.run.smithi090.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-05-27T07:46:52.119 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-05-27T07:46:52.207 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:51 smithi090 ceph-mon[31429]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-05-27T07:46:52.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:51 smithi136 ceph-mon[35324]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-05-27T07:46:52.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:51 smithi129 ceph-mon[35254]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-05-27T07:46:52.346 INFO:teuthology.orchestra.run.smithi090.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-05-27T07:46:52.855 INFO:tasks.cephfs.filesystem:scrub status for tag:ed33bed2-a406-446b-9c32-2a124d3a7255 - {'path': '/', 'tag': 'ed33bed2-a406-446b-9c32-2a124d3a7255', 'options': 'recursive,force'} 2024-05-27T07:46:54.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:53 smithi090 ceph-mon[31429]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-05-27T07:46:54.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:53 smithi136 ceph-mon[35324]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-05-27T07:46:54.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:53 smithi129 ceph-mon[35254]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 38 op/s 2024-05-27T07:46:56.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:55 smithi136 ceph-mon[35324]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-05-27T07:46:56.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:55 smithi129 ceph-mon[35254]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-05-27T07:46:56.424 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:55 smithi090 ceph-mon[31429]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 78 KiB/s wr, 45 op/s 2024-05-27T07:46:57.999 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:57 smithi136 ceph-mon[35324]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 KiB/s wr, 41 op/s 2024-05-27T07:46:58.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:57 smithi090 ceph-mon[31429]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 KiB/s wr, 41 op/s 2024-05-27T07:46:58.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:57 smithi129 ceph-mon[35254]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 78 KiB/s wr, 41 op/s 2024-05-27T07:46:59.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:58 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:46:59.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:58 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:46:59.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:58 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:47:00.248 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:46:59 smithi136 ceph-mon[35324]: pgmap v19: 129 pgs: 129 active+clean; 19 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 52 op/s 2024-05-27T07:47:00.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:46:59 smithi129 ceph-mon[35254]: pgmap v19: 129 pgs: 129 active+clean; 19 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 52 op/s 2024-05-27T07:47:00.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:46:59 smithi090 ceph-mon[31429]: pgmap v19: 129 pgs: 129 active+clean; 19 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.6 MiB/s wr, 52 op/s 2024-05-27T07:47:00.463 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:47:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:47:00.219+0000 7f4e3b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:47:00.463 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:47:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:47:00.219+0000 7f4e3b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:47:00.463 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:47:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:47:00.219+0000 7f4e3b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:47:00.463 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:47:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:47:00.219+0000 7f4e3b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:47:00.463 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 07:47:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T07:47:00.219+0000 7f4e3b800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T07:47:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.834+0000 7f0cf1000640 1 -- 172.21.15.90:0/1121282615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cec110500 msgr2=0x7f0cec112920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.834+0000 7f0cf1000640 1 --2- 172.21.15.90:0/1121282615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cec110500 0x7f0cec112920 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009ca0 tx=0x7f0ce402a940 comp rx=0 tx=0).stop 2024-05-27T07:47:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.834+0000 7f0cf1000640 1 -- 172.21.15.90:0/1121282615 shutdown_connections 2024-05-27T07:47:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.834+0000 7f0cf1000640 1 --2- 172.21.15.90:0/1121282615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cec110500 0x7f0cec112920 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.834+0000 7f0cf1000640 1 --2- 172.21.15.90:0/1121282615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cec10dba0 0x7f0cec10ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.834+0000 7f0cf1000640 1 --2- 172.21.15.90:0/1121282615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec076510 0x7f0cec076920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.834+0000 7f0cf1000640 1 -- 172.21.15.90:0/1121282615 >> 172.21.15.90:0/1121282615 conn(0x7f0cec0ff330 msgr2=0x7f0cec101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.835+0000 7f0cf1000640 1 -- 172.21.15.90:0/1121282615 shutdown_connections 2024-05-27T07:47:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.835+0000 7f0cf1000640 1 -- 172.21.15.90:0/1121282615 wait complete. 2024-05-27T07:47:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.836+0000 7f0cf1000640 1 Processor -- start 2024-05-27T07:47:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.836+0000 7f0cf1000640 1 -- start start 2024-05-27T07:47:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0cf1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cec076510 0x7f0cec1a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0cf1000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cec10dba0 0x7f0cec1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0cf1000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec110500 0x7f0cec19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0cf1000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec115f40 con 0x7f0cec10dba0 2024-05-27T07:47:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0cf1000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec115dc0 con 0x7f0cec110500 2024-05-27T07:47:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0cf1000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec1160c0 con 0x7f0cec076510 2024-05-27T07:47:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0ce9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cec10dba0 0x7f0cec1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0ce9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cec10dba0 0x7f0cec1a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53256/0 (socket says 172.21.15.90:53256) 2024-05-27T07:47:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0ce9400640 1 -- 172.21.15.90:0/2292554228 learned_addr learned my addr 172.21.15.90:0/2292554228 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cec076510 0x7f0cec1a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.837+0000 7f0cea800640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec110500 0x7f0cec19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.838+0000 7f0ce9400640 1 -- 172.21.15.90:0/2292554228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cec076510 msgr2=0x7f0cec1a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.838+0000 7f0ce9400640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cec076510 0x7f0cec1a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.838+0000 7f0ce9400640 1 -- 172.21.15.90:0/2292554228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec110500 msgr2=0x7f0cec19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.838+0000 7f0ce9400640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec110500 0x7f0cec19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.838+0000 7f0ce9400640 1 -- 172.21.15.90:0/2292554228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec19cea0 con 0x7f0cec10dba0 2024-05-27T07:47:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.838+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cec076510 0x7f0cec1a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.838+0000 7f0ce9400640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cec10dba0 0x7f0cec1a2ad0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8005fd0 tx=0x7f0cd800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.838+0000 7f0ce2a00640 1 -- 172.21.15.90:0/2292554228 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8017070 con 0x7f0cec10dba0 2024-05-27T07:47:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.838+0000 7f0ce2a00640 1 -- 172.21.15.90:0/2292554228 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cd8009c30 con 0x7f0cec10dba0 2024-05-27T07:47:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.838+0000 7f0cf1000640 1 -- 172.21.15.90:0/2292554228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec19d190 con 0x7f0cec10dba0 2024-05-27T07:47:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.839+0000 7f0ce2a00640 1 -- 172.21.15.90:0/2292554228 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8003c70 con 0x7f0cec10dba0 2024-05-27T07:47:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.839+0000 7f0cf1000640 1 -- 172.21.15.90:0/2292554228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec1a93e0 con 0x7f0cec10dba0 2024-05-27T07:47:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.842+0000 7f0cf8004740 1 Processor -- start 2024-05-27T07:47:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.842+0000 7f0cf8004740 1 -- start start 2024-05-27T07:47:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.843+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cafd1b0 0x55899ce9f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.843+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ce9fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.843+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cadc4d0 0x55899cea02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.843+0000 7f0cf8004740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55899ce10b90 con 0x55899cad42f0 2024-05-27T07:47:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.843+0000 7f0cf8004740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55899ce00730 con 0x55899cafd1b0 2024-05-27T07:47:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.843+0000 7f0cf8004740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55899ce04770 con 0x55899cadc4d0 2024-05-27T07:47:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.843+0000 7f0ce9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ce9fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.843+0000 7f0ce9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ce9fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53264/0 (socket says 172.21.15.90:53264) 2024-05-27T07:47:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.843+0000 7f0ce9e00640 1 -- 172.21.15.90:0/496277965 learned_addr learned my addr 172.21.15.90:0/496277965 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.843+0000 7f0cea800640 1 --2- 172.21.15.90:0/496277965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cafd1b0 0x55899ce9f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.844+0000 7f0ce9e00640 1 -- 172.21.15.90:0/496277965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cadc4d0 msgr2=0x55899cea02e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:47:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.844+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/496277965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cadc4d0 0x55899cea02e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.844+0000 7f0ce9e00640 1 -- 172.21.15.90:0/496277965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cafd1b0 msgr2=0x55899ce9f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.844+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/496277965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cafd1b0 0x55899ce9f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.844+0000 7f0ce9e00640 1 -- 172.21.15.90:0/496277965 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899cd998b0 con 0x55899cad42f0 2024-05-27T07:47:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.844+0000 7f0ce2a00640 1 -- 172.21.15.90:0/2292554228 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f0cd8003e10 con 0x7f0cec10dba0 2024-05-27T07:47:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.844+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/496277965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ce9fda0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00bcc0 tx=0x7f0cdc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964dcb66c194e67e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.844+0000 7f0cd7400640 1 -- 172.21.15.90:0/496277965 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc005020 con 0x55899cad42f0 2024-05-27T07:47:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.844+0000 7f0cd7400640 1 -- 172.21.15.90:0/496277965 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cdc0047a0 con 0x55899cad42f0 2024-05-27T07:47:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.844+0000 7f0cd7400640 1 -- 172.21.15.90:0/496277965 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc005650 con 0x55899cad42f0 2024-05-27T07:47:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.845+0000 7f0cf8004740 1 -- 172.21.15.90:0/496277965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 msgr2=0x55899ce9fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.845+0000 7f0cf8004740 1 --2- 172.21.15.90:0/496277965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ce9fda0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00bcc0 tx=0x7f0cdc00f730 comp rx=0 tx=0).stop 2024-05-27T07:47:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0cf8004740 1 -- 172.21.15.90:0/496277965 shutdown_connections 2024-05-27T07:47:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0cf8004740 1 --2- 172.21.15.90:0/496277965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cadc4d0 0x55899cea02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0cf8004740 1 --2- 172.21.15.90:0/496277965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ce9fda0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0cf8004740 1 --2- 172.21.15.90:0/496277965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cafd1b0 0x55899ce9f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0cf8004740 1 -- 172.21.15.90:0/496277965 >> 172.21.15.90:0/496277965 conn(0x55899caf1300 msgr2=0x55899ce9c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0cf8004740 1 -- 172.21.15.90:0/496277965 shutdown_connections 2024-05-27T07:47:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0cf8004740 1 -- 172.21.15.90:0/496277965 wait complete. 2024-05-27T07:47:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0ce2a00640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0cd00b2a10 0x7f0cd00b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0ce2a00640 1 -- 172.21.15.90:0/2292554228 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f0cd80e28f0 con 0x7f0cec10dba0 2024-05-27T07:47:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0cd00b2a10 0x7f0cd00b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.846+0000 7f0cf8004740 1 -- start start 2024-05-27T07:47:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.847+0000 7f0cf8004740 1 Processor -- start 2024-05-27T07:47:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.847+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0cd00b2a10 0x7f0cd00b4f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00ba80 tx=0x7f0cdc000bc0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.847+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899cea9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.847+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cadc4d0 0x55899cea9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.847+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 0x55899ceb5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.847+0000 7f0cf8004740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55899cae6790 con 0x55899cad42f0 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.847+0000 7f0cf8004740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55899cda8bc0 con 0x55899cadc4d0 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.847+0000 7f0cf8004740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55899ceac990 con 0x55899cafd1b0 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0cf8004740 10 client.0 _ll_register_callbacks cb 0x55899cd86400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0ce9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899cea9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0ce9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899cea9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53278/0 (socket says 172.21.15.90:53278) 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0ce9e00640 1 -- 172.21.15.90:0/281126611 learned_addr learned my addr 172.21.15.90:0/281126611 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0ce9400640 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cadc4d0 0x55899cea9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0cea800640 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 0x55899ceb5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0ce9e00640 1 -- 172.21.15.90:0/281126611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 msgr2=0x55899ceb5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 0x55899ceb5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0ce9e00640 1 -- 172.21.15.90:0/281126611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cadc4d0 msgr2=0x55899cea9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cadc4d0 0x55899cea9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0ce9e00640 1 -- 172.21.15.90:0/281126611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899cb15210 con 0x55899cad42f0 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.848+0000 7f0cea800640 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 0x55899ceb5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.849+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899cea9a10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc019000 tx=0x7f0cdc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acac4db615b93dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.849+0000 7f0ccbe00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.849+0000 7f0ccbe00640 1 -- 172.21.15.90:0/281126611 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0044d0 con 0x55899cad42f0 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.849+0000 7f0ccbe00640 1 -- 172.21.15.90:0/281126611 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f0cdc0035f0 con 0x55899cad42f0 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.850+0000 7f0cf8004740 1 -- 172.21.15.90:0/281126611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cdc0047a0 con 0x55899cad42f0 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.850+0000 7f0ccbe00640 1 -- 172.21.15.90:0/281126611 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0cdc0037f0 con 0x55899cad42f0 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.850+0000 7f0cf8004740 10 client.15489 fetch_fsmap learned FSMap version 30 2024-05-27T07:47:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.850+0000 7f0cf8004740 1 -- 172.21.15.90:0/281126611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55899cbe1620 con 0x55899cad42f0 2024-05-27T07:47:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.850+0000 7f0ccbe00640 1 -- 172.21.15.90:0/281126611 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cdc0047a0 con 0x55899cad42f0 2024-05-27T07:47:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.851+0000 7f0ccbe00640 1 -- 172.21.15.90:0/281126611 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0cdc031070 con 0x55899cad42f0 2024-05-27T07:47:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.851+0000 7f0cf8004740 10 client.15489 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:47:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.851+0000 7f0cf8004740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.851+0000 7f0cf8004740 20 client.15489 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.851+0000 7f0cf8004740 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55899cabbd80 0x55899cebbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.851+0000 7f0cf8004740 4 client.15489 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.851+0000 7f0cf8004740 1 -- 172.21.15.90:0/281126611 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55899ce12ae0 con 0x55899cabbd80 2024-05-27T07:47:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.851+0000 7f0ce9400640 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55899cabbd80 0x55899cebbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.853+0000 7f0ce9400640 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55899cabbd80 0x55899cebbbc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea43ca7168345bbe server_cookie=94827685305ef6f8 in_seq=0 out_seq=0 2024-05-27T07:47:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.853+0000 7f0ccbe00640 10 client.15489 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0ccbe00640 1 -- 172.21.15.90:0/281126611 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55899ce12ae0 con 0x55899cabbd80 2024-05-27T07:47:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0ccbe00640 10 client.15489 handle_command_reply: tid=0 2024-05-27T07:47:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 client.15489 shutdown 2024-05-27T07:47:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 -- 172.21.15.90:0/281126611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 msgr2=0x55899cea9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899cea9a10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc019000 tx=0x7f0cdc00f730 comp rx=0 tx=0).stop 2024-05-27T07:47:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 -- 172.21.15.90:0/281126611 shutdown_connections 2024-05-27T07:47:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55899cabbd80 0x55899cebbbc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cadc4d0 0x55899cea9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 0x55899ceb5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0ccbe00640 0 client.15489 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 --2- 172.21.15.90:0/281126611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899cea9a10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 -- 172.21.15.90:0/281126611 >> 172.21.15.90:0/281126611 conn(0x55899caf1300 msgr2=0x55899cea1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 -- 172.21.15.90:0/281126611 shutdown_connections 2024-05-27T07:47:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 1 -- 172.21.15.90:0/281126611 wait complete. 2024-05-27T07:47:01.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.855+0000 7f0cf8004740 20 client.15489 trim_cache size 0 max 16384 2024-05-27T07:47:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.876+0000 7f0cf8004740 1 Processor -- start 2024-05-27T07:47:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.876+0000 7f0cf8004740 1 -- start start 2024-05-27T07:47:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cad42f0 0x55899cea5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cadc4d0 0x55899ceab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 0x55899ceb5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0cf8004740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec1b3460 con 0x55899cadc4d0 2024-05-27T07:47:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0cf8004740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec1b4fa0 con 0x55899cad42f0 2024-05-27T07:47:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0cf8004740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec1b3b30 con 0x55899cafd1b0 2024-05-27T07:47:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0ce9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cadc4d0 0x55899ceab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0ce9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cadc4d0 0x55899ceab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53286/0 (socket says 172.21.15.90:53286) 2024-05-27T07:47:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0ce9400640 1 -- 172.21.15.90:0/2085297100 learned_addr learned my addr 172.21.15.90:0/2085297100 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0cea800640 1 --2- 172.21.15.90:0/2085297100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 0x55899ceb5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.877+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/2085297100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cad42f0 0x55899cea5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.878+0000 7f0ce9400640 1 -- 172.21.15.90:0/2085297100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 msgr2=0x55899ceb5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.878+0000 7f0ce9400640 1 --2- 172.21.15.90:0/2085297100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 0x55899ceb5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.878+0000 7f0ce9400640 1 -- 172.21.15.90:0/2085297100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cad42f0 msgr2=0x55899cea5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.878+0000 7f0ce9400640 1 --2- 172.21.15.90:0/2085297100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cad42f0 0x55899cea5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.878+0000 7f0ce9400640 1 -- 172.21.15.90:0/2085297100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899ceabfe0 con 0x55899cadc4d0 2024-05-27T07:47:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.878+0000 7f0ce9400640 1 --2- 172.21.15.90:0/2085297100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cadc4d0 0x55899ceab130 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80a6890 tx=0x7f0cd8091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46dcccaa788fb812 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.878+0000 7f0cd6a00640 1 -- 172.21.15.90:0/2085297100 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80a9620 con 0x55899cadc4d0 2024-05-27T07:47:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.879+0000 7f0cd6a00640 1 -- 172.21.15.90:0/2085297100 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cd80ad9d0 con 0x55899cadc4d0 2024-05-27T07:47:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.879+0000 7f0cd6a00640 1 -- 172.21.15.90:0/2085297100 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80a8880 con 0x55899cadc4d0 2024-05-27T07:47:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.880+0000 7f0cf8004740 1 -- 172.21.15.90:0/2085297100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cadc4d0 msgr2=0x55899ceab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.880+0000 7f0cf8004740 1 --2- 172.21.15.90:0/2085297100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cadc4d0 0x55899ceab130 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80a6890 tx=0x7f0cd8091040 comp rx=0 tx=0).stop 2024-05-27T07:47:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.880+0000 7f0cf8004740 1 -- 172.21.15.90:0/2085297100 shutdown_connections 2024-05-27T07:47:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.880+0000 7f0cf8004740 1 --2- 172.21.15.90:0/2085297100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cafd1b0 0x55899ceb5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.880+0000 7f0cf8004740 1 --2- 172.21.15.90:0/2085297100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cadc4d0 0x55899ceab130 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.880+0000 7f0cf8004740 1 --2- 172.21.15.90:0/2085297100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cad42f0 0x55899cea5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.880+0000 7f0cf8004740 1 -- 172.21.15.90:0/2085297100 >> 172.21.15.90:0/2085297100 conn(0x55899caf1300 msgr2=0x55899cea9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.880+0000 7f0cf8004740 1 -- 172.21.15.90:0/2085297100 shutdown_connections 2024-05-27T07:47:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.880+0000 7f0cf8004740 1 -- 172.21.15.90:0/2085297100 wait complete. 2024-05-27T07:47:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.881+0000 7f0cf8004740 1 -- start start 2024-05-27T07:47:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.881+0000 7f0cf8004740 1 Processor -- start 2024-05-27T07:47:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ceb23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cadc4d0 0x55899ceacbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0cf8004740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cafd1b0 0x55899cead1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0cf8004740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec1b48d0 con 0x55899cad42f0 2024-05-27T07:47:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0cf8004740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec1b4200 con 0x55899cafd1b0 2024-05-27T07:47:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0cf8004740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec1b1420 con 0x55899cadc4d0 2024-05-27T07:47:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0cf8004740 10 client.0 _ll_register_callbacks cb 0x7f0cdc015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0ce9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ceb23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0ce9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ceb23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53296/0 (socket says 172.21.15.90:53296) 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0ce9e00640 1 -- 172.21.15.90:0/1206488968 learned_addr learned my addr 172.21.15.90:0/1206488968 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0cea800640 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cafd1b0 0x55899cead1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.882+0000 7f0ce9400640 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cadc4d0 0x55899ceacbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.883+0000 7f0ce9e00640 1 -- 172.21.15.90:0/1206488968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cadc4d0 msgr2=0x55899ceacbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.883+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cadc4d0 0x55899ceacbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.883+0000 7f0ce9e00640 1 -- 172.21.15.90:0/1206488968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cafd1b0 msgr2=0x55899cead1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.883+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cafd1b0 0x55899cead1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.883+0000 7f0ce9e00640 1 -- 172.21.15.90:0/1206488968 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899cead940 con 0x55899cad42f0 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.883+0000 7f0ce9e00640 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ceb23b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc019000 tx=0x7f0cdc004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84dc5f792ce3ad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.883+0000 7f0ccbe00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.883+0000 7f0ccbe00640 1 -- 172.21.15.90:0/1206488968 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc044060 con 0x55899cad42f0 2024-05-27T07:47:01.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.883+0000 7f0ccbe00640 1 -- 172.21.15.90:0/1206488968 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f0cdc0035f0 con 0x55899cad42f0 2024-05-27T07:47:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.883+0000 7f0cf8004740 1 -- 172.21.15.90:0/1206488968 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd80ad9d0 con 0x55899cad42f0 2024-05-27T07:47:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0ccbe00640 1 -- 172.21.15.90:0/1206488968 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0cdc04b050 con 0x55899cad42f0 2024-05-27T07:47:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0cf8004740 10 client.15495 fetch_fsmap learned FSMap version 30 2024-05-27T07:47:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0cf8004740 1 -- 172.21.15.90:0/1206488968 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55899cf23160 con 0x55899cad42f0 2024-05-27T07:47:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0ccbe00640 1 -- 172.21.15.90:0/1206488968 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0cd80ad9d0 con 0x55899cad42f0 2024-05-27T07:47:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0ccbe00640 1 -- 172.21.15.90:0/1206488968 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0cdc031040 con 0x55899cad42f0 2024-05-27T07:47:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0cf8004740 10 client.15495 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:47:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0cf8004740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0cf8004740 20 client.15495 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0cf8004740 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55899ca911c0 0x55899cf258c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:01.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0cf8004740 4 client.15495 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:47:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.884+0000 7f0cf8004740 1 -- 172.21.15.90:0/1206488968 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55899ce0c2c0 con 0x55899ca911c0 2024-05-27T07:47:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.885+0000 7f0ce9400640 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55899ca911c0 0x55899cf258c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.886+0000 7f0ce9400640 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55899ca911c0 0x55899cf258c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f882d12cc702fad server_cookie=56607a16565e2d34 in_seq=0 out_seq=0 2024-05-27T07:47:01.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:01.886+0000 7f0ccbe00640 10 client.15495 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0ccbe00640 1 -- 172.21.15.90:0/1206488968 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55899ce0c2c0 con 0x55899ca911c0 2024-05-27T07:47:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0ccbe00640 10 client.15495 handle_command_reply: tid=0 2024-05-27T07:47:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0cf8004740 1 client.15495 shutdown 2024-05-27T07:47:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0cf8004740 1 -- 172.21.15.90:0/1206488968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 msgr2=0x55899ceb23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:02.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0cf8004740 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ceb23b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc019000 tx=0x7f0cdc004da0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0cf8004740 1 -- 172.21.15.90:0/1206488968 shutdown_connections 2024-05-27T07:47:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0cf8004740 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55899ca911c0 0x55899cf258c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0cf8004740 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55899cafd1b0 0x55899cead1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0ccbe00640 0 client.15495 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0cf8004740 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55899cadc4d0 0x55899ceacbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0cf8004740 1 --2- 172.21.15.90:0/1206488968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899cad42f0 0x55899ceb23b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.095+0000 7f0cf8004740 1 -- 172.21.15.90:0/1206488968 >> 172.21.15.90:0/1206488968 conn(0x55899caf1300 msgr2=0x55899ce9f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.096+0000 7f0cf8004740 1 -- 172.21.15.90:0/1206488968 shutdown_connections 2024-05-27T07:47:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.096+0000 7f0cf8004740 1 -- 172.21.15.90:0/1206488968 wait complete. 2024-05-27T07:47:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.096+0000 7f0cf8004740 20 client.15495 trim_cache size 0 max 16384 2024-05-27T07:47:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:47:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:47:02.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:47:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 -- 172.21.15.90:0/2292554228 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0cd00b2a10 msgr2=0x7f0cd00b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0cd00b2a10 0x7f0cd00b4f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00ba80 tx=0x7f0cdc000bc0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 -- 172.21.15.90:0/2292554228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cec10dba0 msgr2=0x7f0cec1a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cec10dba0 0x7f0cec1a2ad0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8005fd0 tx=0x7f0cd800dde0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 -- 172.21.15.90:0/2292554228 shutdown_connections 2024-05-27T07:47:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0cd00b2a10 0x7f0cd00b4f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0cec110500 0x7f0cec19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cec10dba0 0x7f0cec1a2ad0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 --2- 172.21.15.90:0/2292554228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cec076510 0x7f0cec1a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 -- 172.21.15.90:0/2292554228 >> 172.21.15.90:0/2292554228 conn(0x7f0cec0ff330 msgr2=0x7f0cec110910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 -- 172.21.15.90:0/2292554228 shutdown_connections 2024-05-27T07:47:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:02.097+0000 7f0cf1000640 1 -- 172.21.15.90:0/2292554228 wait complete. 2024-05-27T07:47:02.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:02 smithi129 ceph-mon[35254]: pgmap v20: 129 pgs: 129 active+clean; 19 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.5 MiB/s wr, 18 op/s 2024-05-27T07:47:02.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:02 smithi129 ceph-mon[35254]: mgrmap e25: x(active, since 34s), standbys: z, y 2024-05-27T07:47:02.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:02 smithi090 ceph-mon[31429]: pgmap v20: 129 pgs: 129 active+clean; 19 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.5 MiB/s wr, 18 op/s 2024-05-27T07:47:02.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:02 smithi090 ceph-mon[31429]: mgrmap e25: x(active, since 34s), standbys: z, y 2024-05-27T07:47:02.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:02 smithi136 ceph-mon[35324]: pgmap v20: 129 pgs: 129 active+clean; 19 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.5 MiB/s wr, 18 op/s 2024-05-27T07:47:02.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:02 smithi136 ceph-mon[35324]: mgrmap e25: x(active, since 34s), standbys: z, y 2024-05-27T07:47:04.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:04 smithi129 ceph-mon[35254]: pgmap v21: 129 pgs: 129 active+clean; 19 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.5 MiB/s wr, 31 op/s 2024-05-27T07:47:04.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:04 smithi090 ceph-mon[31429]: pgmap v21: 129 pgs: 129 active+clean; 19 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.5 MiB/s wr, 31 op/s 2024-05-27T07:47:04.498 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-mon[35324]: pgmap v21: 129 pgs: 129 active+clean; 19 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.5 MiB/s wr, 31 op/s 2024-05-27T07:47:05.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fbf85c00700 time 2024-05-27T07:47:04.932293+0000 2024-05-27T07:47:05.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:47:05.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:47:05.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbf9b7dcba6] 2024-05-27T07:47:05.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fbf9b7dcdd1] 2024-05-27T07:47:05.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55666bea223d] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55666be8d971] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 5: (SnapRealm::check_cache() const+0x13e) [0x55666be8bfa0] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55666be8c651] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55666be21347] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55666bb1d87e] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55666bb22e94] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55666bb2d969] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 11: (MDLog::_replay_thread()+0x1f38) [0x55666bec648c] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55666becf9cb] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 13: (Thread::entry_wrapper()+0x43) [0x7fbf9b7ab72d] 2024-05-27T07:47:05.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 14: (Thread::_entry_func(void*)+0xd) [0x7fbf9b7ab749] 2024-05-27T07:47:05.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbf9a34f1ca] 2024-05-27T07:47:05.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 16: clone() 2024-05-27T07:47:05.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: *** Caught signal (Aborted) ** 2024-05-27T07:47:05.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: in thread 7fbf85c00700 thread_name:md_log_replay 2024-05-27T07:47:05.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:04.934+0000 7fbf85c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fbf85c00700 time 2024-05-27T07:47:04.932293+0000 2024-05-27T07:47:05.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:47:05.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:47:05.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbf9b7dcba6] 2024-05-27T07:47:05.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fbf9b7dcdd1] 2024-05-27T07:47:05.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55666bea223d] 2024-05-27T07:47:05.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55666be8d971] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 5: (SnapRealm::check_cache() const+0x13e) [0x55666be8bfa0] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55666be8c651] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55666be21347] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55666bb1d87e] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55666bb22e94] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55666bb2d969] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 11: (MDLog::_replay_thread()+0x1f38) [0x55666bec648c] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55666becf9cb] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 13: (Thread::entry_wrapper()+0x43) [0x7fbf9b7ab72d] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 14: (Thread::_entry_func(void*)+0xd) [0x7fbf9b7ab749] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbf9a34f1ca] 2024-05-27T07:47:05.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 16: clone() 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbf9a359d20] 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2: gsignal() 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 3: abort() 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fbf9b7dcced] 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fbf9b7dcdd1] 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55666bea223d] 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55666be8d971] 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 8: (SnapRealm::check_cache() const+0x13e) [0x55666be8bfa0] 2024-05-27T07:47:05.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55666be8c651] 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55666be21347] 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55666bb1d87e] 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55666bb22e94] 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55666bb2d969] 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 14: (MDLog::_replay_thread()+0x1f38) [0x55666bec648c] 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55666becf9cb] 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 16: (Thread::entry_wrapper()+0x43) [0x7fbf9b7ab72d] 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 17: (Thread::_entry_func(void*)+0xd) [0x7fbf9b7ab749] 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbf9a34f1ca] 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 19: clone() 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:04.938+0000 7fbf85c00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:47:05.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: in thread 7fbf85c00700 thread_name:md_log_replay 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbf9a359d20] 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2: gsignal() 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 3: abort() 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fbf9b7dcced] 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fbf9b7dcdd1] 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55666bea223d] 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55666be8d971] 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 8: (SnapRealm::check_cache() const+0x13e) [0x55666be8bfa0] 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55666be8c651] 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55666be21347] 2024-05-27T07:47:05.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55666bb1d87e] 2024-05-27T07:47:05.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55666bb22e94] 2024-05-27T07:47:05.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55666bb2d969] 2024-05-27T07:47:05.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 14: (MDLog::_replay_thread()+0x1f38) [0x55666bec648c] 2024-05-27T07:47:05.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55666becf9cb] 2024-05-27T07:47:05.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 16: (Thread::entry_wrapper()+0x43) [0x7fbf9b7ab72d] 2024-05-27T07:47:05.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 17: (Thread::_entry_func(void*)+0xd) [0x7fbf9b7ab749] 2024-05-27T07:47:05.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbf9a34f1ca] 2024-05-27T07:47:05.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 19: clone() 2024-05-27T07:47:05.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: -1> 2024-05-27T07:47:04.934+0000 7fbf85c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fbf85c00700 time 2024-05-27T07:47:04.932293+0000 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbf9b7dcba6] 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fbf9b7dcdd1] 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55666bea223d] 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55666be8d971] 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 5: (SnapRealm::check_cache() const+0x13e) [0x55666be8bfa0] 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55666be8c651] 2024-05-27T07:47:05.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55666be21347] 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55666bb1d87e] 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55666bb22e94] 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55666bb2d969] 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 11: (MDLog::_replay_thread()+0x1f38) [0x55666bec648c] 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55666becf9cb] 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 13: (Thread::entry_wrapper()+0x43) [0x7fbf9b7ab72d] 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 14: (Thread::_entry_func(void*)+0xd) [0x7fbf9b7ab749] 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbf9a34f1ca] 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 16: clone() 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 0> 2024-05-27T07:47:04.938+0000 7fbf85c00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: in thread 7fbf85c00700 thread_name:md_log_replay 2024-05-27T07:47:05.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbf9a359d20] 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2: gsignal() 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 3: abort() 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fbf9b7dcced] 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fbf9b7dcdd1] 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55666bea223d] 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55666be8d971] 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 8: (SnapRealm::check_cache() const+0x13e) [0x55666be8bfa0] 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55666be8c651] 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55666be21347] 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55666bb1d87e] 2024-05-27T07:47:05.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55666bb22e94] 2024-05-27T07:47:05.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55666bb2d969] 2024-05-27T07:47:05.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 14: (MDLog::_replay_thread()+0x1f38) [0x55666bec648c] 2024-05-27T07:47:05.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55666becf9cb] 2024-05-27T07:47:05.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 16: (Thread::entry_wrapper()+0x43) [0x7fbf9b7ab72d] 2024-05-27T07:47:05.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 17: (Thread::_entry_func(void*)+0xd) [0x7fbf9b7ab749] 2024-05-27T07:47:05.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbf9a34f1ca] 2024-05-27T07:47:05.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 19: clone() 2024-05-27T07:47:05.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:47:05.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: -1> 2024-05-27T07:47:04.934+0000 7fbf85c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fbf85c00700 time 2024-05-27T07:47:04.932293+0000 2024-05-27T07:47:05.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:47:05.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:47:05.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fbf9b7dcba6] 2024-05-27T07:47:05.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fbf9b7dcdd1] 2024-05-27T07:47:05.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55666bea223d] 2024-05-27T07:47:05.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55666be8d971] 2024-05-27T07:47:05.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 5: (SnapRealm::check_cache() const+0x13e) [0x55666be8bfa0] 2024-05-27T07:47:05.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55666be8c651] 2024-05-27T07:47:05.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55666be21347] 2024-05-27T07:47:05.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55666bb1d87e] 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55666bb22e94] 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55666bb2d969] 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 11: (MDLog::_replay_thread()+0x1f38) [0x55666bec648c] 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55666becf9cb] 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 13: (Thread::entry_wrapper()+0x43) [0x7fbf9b7ab72d] 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 14: (Thread::_entry_func(void*)+0xd) [0x7fbf9b7ab749] 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbf9a34f1ca] 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 16: clone() 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 0> 2024-05-27T07:47:04.938+0000 7fbf85c00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:47:05.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: in thread 7fbf85c00700 thread_name:md_log_replay 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbf9a359d20] 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2: gsignal() 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 3: abort() 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fbf9b7dcced] 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fbf9b7dcdd1] 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55666bea223d] 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55666be8d971] 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 8: (SnapRealm::check_cache() const+0x13e) [0x55666be8bfa0] 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55666be8c651] 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55666be21347] 2024-05-27T07:47:05.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55666bb1d87e] 2024-05-27T07:47:05.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55666bb22e94] 2024-05-27T07:47:05.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55666bb2d969] 2024-05-27T07:47:05.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 14: (MDLog::_replay_thread()+0x1f38) [0x55666bec648c] 2024-05-27T07:47:05.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55666becf9cb] 2024-05-27T07:47:05.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 16: (Thread::entry_wrapper()+0x43) [0x7fbf9b7ab72d] 2024-05-27T07:47:05.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 17: (Thread::_entry_func(void*)+0xd) [0x7fbf9b7ab749] 2024-05-27T07:47:05.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fbf9a34f1ca] 2024-05-27T07:47:05.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 19: clone() 2024-05-27T07:47:05.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:47:05.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[66031]: 2024-05-27T07:47:05.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:05 smithi136 podman[70247]: 2024-05-27 07:47:05.077888821 +0000 UTC m=+0.041376500 container died 5faa54b04d554c51f7adc1353246546154ace0d76017d9cba59763a1fcc84cc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-27T07:47:06.034 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:05 smithi136 podman[70247]: 2024-05-27 07:47:05.918060636 +0000 UTC m=+0.881548290 container remove 5faa54b04d554c51f7adc1353246546154ace0d76017d9cba59763a1fcc84cc9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-27T07:47:06.034 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:05 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:47:06.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:06 smithi129 ceph-mon[35254]: pgmap v22: 129 pgs: 129 active+clean; 21 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-05-27T07:47:06.380 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:06 smithi136 ceph-mon[35324]: pgmap v22: 129 pgs: 129 active+clean; 21 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-05-27T07:47:06.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:06 smithi090 ceph-mon[31429]: pgmap v22: 129 pgs: 129 active+clean; 21 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-05-27T07:47:06.749 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:06 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:47:06.749 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:06 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 3.098s CPU time. 2024-05-27T07:47:08.117 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:08 smithi136 ceph-mon[35324]: pgmap v23: 129 pgs: 129 active+clean; 21 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-05-27T07:47:08.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:08 smithi090 ceph-mon[31429]: pgmap v23: 129 pgs: 129 active+clean; 21 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-05-27T07:47:08.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:08 smithi129 ceph-mon[35254]: pgmap v23: 129 pgs: 129 active+clean; 21 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 1.7 MiB/s wr, 126 op/s 2024-05-27T07:47:10.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:10 smithi129 ceph-mon[35254]: pgmap v24: 129 pgs: 129 active+clean; 28 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-05-27T07:47:10.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:10 smithi090 ceph-mon[31429]: pgmap v24: 129 pgs: 129 active+clean; 28 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-05-27T07:47:10.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:10 smithi136 ceph-mon[35324]: pgmap v24: 129 pgs: 129 active+clean; 28 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-05-27T07:47:12.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:12 smithi129 ceph-mon[35254]: pgmap v25: 129 pgs: 129 active+clean; 28 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 772 KiB/s wr, 141 op/s 2024-05-27T07:47:12.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:12 smithi090 ceph-mon[31429]: pgmap v25: 129 pgs: 129 active+clean; 28 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 772 KiB/s wr, 141 op/s 2024-05-27T07:47:12.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:12 smithi136 ceph-mon[35324]: pgmap v25: 129 pgs: 129 active+clean; 28 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 772 KiB/s wr, 141 op/s 2024-05-27T07:47:14.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:14 smithi090 ceph-mon[31429]: pgmap v26: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-27T07:47:14.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:14 smithi136 ceph-mon[35324]: pgmap v26: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-27T07:47:14.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:14 smithi129 ceph-mon[35254]: pgmap v26: 129 pgs: 129 active+clean; 32 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-27T07:47:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.336+0000 7f5dafe00640 1 -- 172.21.15.90:0/1464264622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db010c180 msgr2=0x7f5db0105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.336+0000 7f5dafe00640 1 --2- 172.21.15.90:0/1464264622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db010c180 0x7f5db0105dc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009900 tx=0x7f5d9c02a410 comp rx=0 tx=0).stop 2024-05-27T07:47:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.336+0000 7f5dafe00640 1 -- 172.21.15.90:0/1464264622 shutdown_connections 2024-05-27T07:47:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.336+0000 7f5dafe00640 1 --2- 172.21.15.90:0/1464264622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db0106430 0x7f5db0112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.336+0000 7f5dafe00640 1 --2- 172.21.15.90:0/1464264622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db010c180 0x7f5db0105dc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.336+0000 7f5dafe00640 1 --2- 172.21.15.90:0/1464264622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db010bce0 0x7f5db0105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.336+0000 7f5dafe00640 1 -- 172.21.15.90:0/1464264622 >> 172.21.15.90:0/1464264622 conn(0x7f5db0101380 msgr2=0x7f5db01037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.336+0000 7f5dafe00640 1 -- 172.21.15.90:0/1464264622 shutdown_connections 2024-05-27T07:47:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.336+0000 7f5dafe00640 1 -- 172.21.15.90:0/1464264622 wait complete. 2024-05-27T07:47:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.337+0000 7f5dafe00640 1 Processor -- start 2024-05-27T07:47:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.337+0000 7f5dafe00640 1 -- start start 2024-05-27T07:47:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.338+0000 7f5dafe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db0106430 0x7f5db01a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.338+0000 7f5dafe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db010bce0 0x7f5db01a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.338+0000 7f5dafe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db010c180 0x7f5db019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.338+0000 7f5dafe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5db0115e30 con 0x7f5db010c180 2024-05-27T07:47:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.338+0000 7f5dafe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5db0115cb0 con 0x7f5db010bce0 2024-05-27T07:47:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.338+0000 7f5dafe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5db0115fb0 con 0x7f5db0106430 2024-05-27T07:47:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.338+0000 7f5daf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db010c180 0x7f5db019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.338+0000 7f5daf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db010c180 0x7f5db019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34804/0 (socket says 172.21.15.90:34804) 2024-05-27T07:47:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.338+0000 7f5daf400640 1 -- 172.21.15.90:0/1566574768 learned_addr learned my addr 172.21.15.90:0/1566574768 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.338+0000 7f5daea00640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db0106430 0x7f5db01a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5daf400640 1 -- 172.21.15.90:0/1566574768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db0106430 msgr2=0x7f5db01a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5daf400640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db0106430 0x7f5db01a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5daf400640 1 -- 172.21.15.90:0/1566574768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db010bce0 msgr2=0x7f5db01a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5daf400640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db010bce0 0x7f5db01a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5daf400640 1 -- 172.21.15.90:0/1566574768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db019ce40 con 0x7f5db010c180 2024-05-27T07:47:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5daf400640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db010c180 0x7f5db019c710 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5da800b970 tx=0x7f5da800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5da7400640 1 -- 172.21.15.90:0/1566574768 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8019070 con 0x7f5db010c180 2024-05-27T07:47:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5da7400640 1 -- 172.21.15.90:0/1566574768 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5da8014440 con 0x7f5db010c180 2024-05-27T07:47:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5da7400640 1 -- 172.21.15.90:0/1566574768 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da80075d0 con 0x7f5db010c180 2024-05-27T07:47:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5dafe00640 1 -- 172.21.15.90:0/1566574768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db019d0d0 con 0x7f5db010c180 2024-05-27T07:47:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.339+0000 7f5dafe00640 1 -- 172.21.15.90:0/1566574768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db01a9330 con 0x7f5db010c180 2024-05-27T07:47:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.344+0000 7f5dbc973740 1 Processor -- start 2024-05-27T07:47:15.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.344+0000 7f5da7400640 1 -- 172.21.15.90:0/1566574768 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f5da801c070 con 0x7f5db010c180 2024-05-27T07:47:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.345+0000 7f5dbc973740 1 -- start start 2024-05-27T07:47:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.345+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac26947a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.345+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac26947fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.345+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac269484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.345+0000 7f5dbc973740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ac268b8b90 con 0x55ac265844d0 2024-05-27T07:47:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.345+0000 7f5dbc973740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ac268a8730 con 0x55ac265a51b0 2024-05-27T07:47:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.345+0000 7f5dbc973740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ac268ac770 con 0x55ac2657c2f0 2024-05-27T07:47:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5dae000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac269484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5dae000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac269484e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34812/0 (socket says 172.21.15.90:34812) 2024-05-27T07:47:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5dae000640 1 -- 172.21.15.90:0/917668791 learned_addr learned my addr 172.21.15.90:0/917668791 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5daea00640 1 --2- 172.21.15.90:0/917668791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac26947fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5dae000640 1 --2- 172.21.15.90:0/917668791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac26947a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5dae000640 1 -- 172.21.15.90:0/917668791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 msgr2=0x55ac26947fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5dae000640 1 --2- 172.21.15.90:0/917668791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac26947fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5dae000640 1 -- 172.21.15.90:0/917668791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 msgr2=0x55ac26947a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5dae000640 1 --2- 172.21.15.90:0/917668791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac26947a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5dae000640 1 -- 172.21.15.90:0/917668791 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac268418b0 con 0x55ac265844d0 2024-05-27T07:47:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.346+0000 7f5da7400640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d940b2b00 0x7f5d940b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.347+0000 7f5da7400640 1 -- 172.21.15.90:0/1566574768 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f5da80e2160 con 0x7f5db010c180 2024-05-27T07:47:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.347+0000 7f5dae000640 1 --2- 172.21.15.90:0/917668791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac269484e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c005760 tx=0x7f5d9c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e43b583259d86e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.347+0000 7f5daea00640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d940b2b00 0x7f5d940b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.347+0000 7f5d9be00640 1 -- 172.21.15.90:0/917668791 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c041070 con 0x55ac265844d0 2024-05-27T07:47:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.347+0000 7f5d9be00640 1 -- 172.21.15.90:0/917668791 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d9c0044b0 con 0x55ac265844d0 2024-05-27T07:47:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.347+0000 7f5d9be00640 1 -- 172.21.15.90:0/917668791 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c035030 con 0x55ac265844d0 2024-05-27T07:47:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.348+0000 7f5dbc973740 1 -- 172.21.15.90:0/917668791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 msgr2=0x55ac269484e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.348+0000 7f5daea00640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d940b2b00 0x7f5d940b5010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55ac26948e10 tx=0x7f5da000c430 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.348+0000 7f5dbc973740 1 --2- 172.21.15.90:0/917668791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac269484e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c005760 tx=0x7f5d9c005a20 comp rx=0 tx=0).stop 2024-05-27T07:47:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.349+0000 7f5dbc973740 1 -- 172.21.15.90:0/917668791 shutdown_connections 2024-05-27T07:47:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.349+0000 7f5dbc973740 1 --2- 172.21.15.90:0/917668791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac269484e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.349+0000 7f5dbc973740 1 --2- 172.21.15.90:0/917668791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac26947fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.349+0000 7f5dbc973740 1 --2- 172.21.15.90:0/917668791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac26947a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.349+0000 7f5dbc973740 1 -- 172.21.15.90:0/917668791 >> 172.21.15.90:0/917668791 conn(0x55ac26599300 msgr2=0x55ac26944210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.349+0000 7f5dbc973740 1 -- 172.21.15.90:0/917668791 shutdown_connections 2024-05-27T07:47:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.349+0000 7f5dbc973740 1 -- 172.21.15.90:0/917668791 wait complete. 2024-05-27T07:47:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.350+0000 7f5dbc973740 1 -- start start 2024-05-27T07:47:15.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.350+0000 7f5dbc973740 1 Processor -- start 2024-05-27T07:47:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac2657c2f0 0x55ac26951ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac265844d0 0x55ac26952410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac2695de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5dbc973740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ac2658e790 con 0x55ac2657c2f0 2024-05-27T07:47:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5dbc973740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ac26850bc0 con 0x55ac265a51b0 2024-05-27T07:47:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5dbc973740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ac26954b30 con 0x55ac265844d0 2024-05-27T07:47:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5dbc973740 10 client.0 _ll_register_callbacks cb 0x55ac2682e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5daea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac2657c2f0 0x55ac26951ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5daea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac2657c2f0 0x55ac26951ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34822/0 (socket says 172.21.15.90:34822) 2024-05-27T07:47:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5daea00640 1 -- 172.21.15.90:0/1961526819 learned_addr learned my addr 172.21.15.90:0/1961526819 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.351+0000 7f5daf400640 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac2695de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.352+0000 7f5dae000640 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac265844d0 0x55ac26952410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.352+0000 7f5daea00640 1 -- 172.21.15.90:0/1961526819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac265844d0 msgr2=0x55ac26952410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.352+0000 7f5daea00640 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac265844d0 0x55ac26952410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.352+0000 7f5daea00640 1 -- 172.21.15.90:0/1961526819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 msgr2=0x55ac2695de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:15.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.352+0000 7f5daea00640 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac2695de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.352+0000 7f5daea00640 1 -- 172.21.15.90:0/1961526819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac265bd210 con 0x55ac2657c2f0 2024-05-27T07:47:15.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.352+0000 7f5dae000640 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac265844d0 0x55ac26952410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.352+0000 7f5daea00640 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac2657c2f0 0x55ac26951ed0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5da001caf0 tx=0x7f5da00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadac48195ce7280 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.352+0000 7f5d98c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.353+0000 7f5d98c00640 1 -- 172.21.15.90:0/1961526819 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da0029060 con 0x55ac2657c2f0 2024-05-27T07:47:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.353+0000 7f5d98c00640 1 -- 172.21.15.90:0/1961526819 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f5da000eb80 con 0x55ac2657c2f0 2024-05-27T07:47:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.353+0000 7f5dbc973740 1 -- 172.21.15.90:0/1961526819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9c0044b0 con 0x55ac2657c2f0 2024-05-27T07:47:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.354+0000 7f5d98c00640 1 -- 172.21.15.90:0/1961526819 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5da000ed80 con 0x55ac2657c2f0 2024-05-27T07:47:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.354+0000 7f5dbc973740 10 client.15507 fetch_fsmap learned FSMap version 30 2024-05-27T07:47:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.354+0000 7f5dbc973740 1 -- 172.21.15.90:0/1961526819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac26689620 con 0x55ac2657c2f0 2024-05-27T07:47:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.354+0000 7f5d98c00640 1 -- 172.21.15.90:0/1961526819 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d9c0044b0 con 0x55ac2657c2f0 2024-05-27T07:47:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.354+0000 7f5d98c00640 1 -- 172.21.15.90:0/1961526819 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5da0018040 con 0x55ac2657c2f0 2024-05-27T07:47:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.354+0000 7f5dbc973740 10 client.15507 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:47:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.354+0000 7f5dbc973740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.355+0000 7f5dbc973740 20 client.15507 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.355+0000 7f5dbc973740 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ac26563d80 0x55ac2696c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.355+0000 7f5dbc973740 4 client.15507 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.355+0000 7f5dbc973740 1 -- 172.21.15.90:0/1961526819 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac268baae0 con 0x55ac26563d80 2024-05-27T07:47:15.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.355+0000 7f5dae000640 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ac26563d80 0x55ac2696c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.356+0000 7f5d98c00640 10 client.15507 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:15.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.356+0000 7f5dae000640 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ac26563d80 0x55ac2696c7b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e142e5805b92b2 server_cookie=4297c2e7e89c1852 in_seq=0 out_seq=0 2024-05-27T07:47:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5d98c00640 1 -- 172.21.15.90:0/1961526819 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ac268baae0 con 0x55ac26563d80 2024-05-27T07:47:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5d98c00640 10 client.15507 handle_command_reply: tid=0 2024-05-27T07:47:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5dbc973740 1 client.15507 shutdown 2024-05-27T07:47:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5dbc973740 1 -- 172.21.15.90:0/1961526819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac2657c2f0 msgr2=0x55ac26951ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5dbc973740 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac2657c2f0 0x55ac26951ed0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5da001caf0 tx=0x7f5da00046c0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5dbc973740 1 -- 172.21.15.90:0/1961526819 shutdown_connections 2024-05-27T07:47:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5dbc973740 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ac26563d80 0x55ac2696c7b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5dbc973740 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac2695de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5dbc973740 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac265844d0 0x55ac26952410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5d98c00640 0 client.15507 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5dbc973740 1 --2- 172.21.15.90:0/1961526819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac2657c2f0 0x55ac26951ed0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5dbc973740 1 -- 172.21.15.90:0/1961526819 >> 172.21.15.90:0/1961526819 conn(0x55ac26599300 msgr2=0x55ac26949780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.364+0000 7f5dbc973740 1 -- 172.21.15.90:0/1961526819 shutdown_connections 2024-05-27T07:47:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.365+0000 7f5dbc973740 1 -- 172.21.15.90:0/1961526819 wait complete. 2024-05-27T07:47:15.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.365+0000 7f5dbc973740 20 client.15507 trim_cache size 0 max 16384 2024-05-27T07:47:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.384+0000 7f5dbc973740 1 Processor -- start 2024-05-27T07:47:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.384+0000 7f5dbc973740 1 -- start start 2024-05-27T07:47:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.384+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac2695dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.384+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac2694b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.384+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac2694bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.384+0000 7f5dbc973740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01aee30 con 0x55ac265844d0 2024-05-27T07:47:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.384+0000 7f5dbc973740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01b0970 con 0x55ac265a51b0 2024-05-27T07:47:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.384+0000 7f5dbc973740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01af500 con 0x55ac2657c2f0 2024-05-27T07:47:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5dae000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac2694b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5dae000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac2694b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34832/0 (socket says 172.21.15.90:34832) 2024-05-27T07:47:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5dae000640 1 -- 172.21.15.90:0/3716464825 learned_addr learned my addr 172.21.15.90:0/3716464825 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5daf400640 1 --2- 172.21.15.90:0/3716464825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac2694bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5dae000640 1 -- 172.21.15.90:0/3716464825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 msgr2=0x55ac2695dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5daea00640 1 --2- 172.21.15.90:0/3716464825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac2695dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5dae000640 1 --2- 172.21.15.90:0/3716464825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac2695dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5dae000640 1 -- 172.21.15.90:0/3716464825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 msgr2=0x55ac2694bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5dae000640 1 --2- 172.21.15.90:0/3716464825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac2694bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5dae000640 1 -- 172.21.15.90:0/3716464825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac26950210 con 0x55ac265844d0 2024-05-27T07:47:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.385+0000 7f5dae000640 1 --2- 172.21.15.90:0/3716464825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac2694b710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c02a930 tx=0x7f5d9c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f23584ff66ad57e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.386+0000 7f5d9b400640 1 -- 172.21.15.90:0/3716464825 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c035030 con 0x55ac265844d0 2024-05-27T07:47:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.386+0000 7f5d9b400640 1 -- 172.21.15.90:0/3716464825 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d9c032cd0 con 0x55ac265844d0 2024-05-27T07:47:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.386+0000 7f5d9b400640 1 -- 172.21.15.90:0/3716464825 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c041040 con 0x55ac265844d0 2024-05-27T07:47:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.387+0000 7f5dbc973740 1 -- 172.21.15.90:0/3716464825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 msgr2=0x55ac2694b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.387+0000 7f5dbc973740 1 --2- 172.21.15.90:0/3716464825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac2694b710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c02a930 tx=0x7f5d9c005fd0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.387+0000 7f5dbc973740 1 -- 172.21.15.90:0/3716464825 shutdown_connections 2024-05-27T07:47:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.387+0000 7f5dbc973740 1 --2- 172.21.15.90:0/3716464825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265a51b0 0x55ac2694bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.387+0000 7f5dbc973740 1 --2- 172.21.15.90:0/3716464825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265844d0 0x55ac2694b710 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.387+0000 7f5dbc973740 1 --2- 172.21.15.90:0/3716464825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac2695dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.387+0000 7f5dbc973740 1 -- 172.21.15.90:0/3716464825 >> 172.21.15.90:0/3716464825 conn(0x55ac26599300 msgr2=0x55ac26951550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.387+0000 7f5dbc973740 1 -- 172.21.15.90:0/3716464825 shutdown_connections 2024-05-27T07:47:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.387+0000 7f5dbc973740 1 -- 172.21.15.90:0/3716464825 wait complete. 2024-05-27T07:47:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.387+0000 7f5dbc973740 1 -- start start 2024-05-27T07:47:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.388+0000 7f5dbc973740 1 Processor -- start 2024-05-27T07:47:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.388+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac269553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.388+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265844d0 0x55ac269558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.388+0000 7f5dbc973740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265a51b0 0x55ac269d2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.388+0000 7f5dbc973740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01b02a0 con 0x55ac265a51b0 2024-05-27T07:47:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.388+0000 7f5dbc973740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01afbd0 con 0x55ac265844d0 2024-05-27T07:47:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.388+0000 7f5dbc973740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5db01ace30 con 0x55ac2657c2f0 2024-05-27T07:47:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.389+0000 7f5dbc973740 10 client.0 _ll_register_callbacks cb 0x7f5da001c770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.389+0000 7f5daf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265a51b0 0x55ac269d2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.389+0000 7f5daf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265a51b0 0x55ac269d2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34844/0 (socket says 172.21.15.90:34844) 2024-05-27T07:47:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.389+0000 7f5daf400640 1 -- 172.21.15.90:0/4218902475 learned_addr learned my addr 172.21.15.90:0/4218902475 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.389+0000 7f5dae000640 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265844d0 0x55ac269558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.389+0000 7f5daea00640 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac269553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.390+0000 7f5daf400640 1 -- 172.21.15.90:0/4218902475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 msgr2=0x55ac269553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.390+0000 7f5daf400640 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac269553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.390+0000 7f5daf400640 1 -- 172.21.15.90:0/4218902475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265844d0 msgr2=0x55ac269558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.390+0000 7f5daf400640 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265844d0 0x55ac269558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.390+0000 7f5daf400640 1 -- 172.21.15.90:0/4218902475 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac269d3370 con 0x55ac265a51b0 2024-05-27T07:47:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.390+0000 7f5daf400640 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265a51b0 0x55ac269d2e30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5da80df500 tx=0x7f5da80a9760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cb795363e8378c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.390+0000 7f5d98c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.390+0000 7f5d98c00640 1 -- 172.21.15.90:0/4218902475 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da80c3060 con 0x55ac265a51b0 2024-05-27T07:47:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.390+0000 7f5d98c00640 1 -- 172.21.15.90:0/4218902475 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9408+0+0 (secure 0 0 0) 0x7f5da80b1b80 con 0x55ac265a51b0 2024-05-27T07:47:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.390+0000 7f5dbc973740 1 -- 172.21.15.90:0/4218902475 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9c032cd0 con 0x55ac265a51b0 2024-05-27T07:47:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5d98c00640 1 -- 172.21.15.90:0/4218902475 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5da80b1e00 con 0x55ac265a51b0 2024-05-27T07:47:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5dbc973740 10 client.15513 fetch_fsmap learned FSMap version 30 2024-05-27T07:47:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5dbc973740 1 -- 172.21.15.90:0/4218902475 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac269d45d0 con 0x55ac265a51b0 2024-05-27T07:47:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5d98c00640 1 -- 172.21.15.90:0/4218902475 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5da80ca020 con 0x55ac265a51b0 2024-05-27T07:47:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5d98c00640 1 -- 172.21.15.90:0/4218902475 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5da80a4910 con 0x55ac265a51b0 2024-05-27T07:47:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5dbc973740 10 client.15513 fetch_fsmap finished waiting for FSMap version 30 2024-05-27T07:47:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5dbc973740 10 client.15513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5dbc973740 20 client.15513 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5dbc973740 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ac265391c0 0x55ac269d6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5dbc973740 4 client.15513 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:47:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.391+0000 7f5dbc973740 1 -- 172.21.15.90:0/4218902475 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac268b42c0 con 0x55ac265391c0 2024-05-27T07:47:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.392+0000 7f5daea00640 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ac265391c0 0x55ac269d6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.393+0000 7f5d98c00640 10 client.15513 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.393+0000 7f5daea00640 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ac265391c0 0x55ac269d6d60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e79e17d0dbbb19 server_cookie=80d4fafe3a02d71d in_seq=0 out_seq=0 2024-05-27T07:47:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.970+0000 7f5d98c00640 1 -- 172.21.15.90:0/4218902475 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ac268b42c0 con 0x55ac265391c0 2024-05-27T07:47:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.970+0000 7f5d98c00640 10 client.15513 handle_command_reply: tid=0 2024-05-27T07:47:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.970+0000 7f5dbc973740 1 client.15513 shutdown 2024-05-27T07:47:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.971+0000 7f5dbc973740 1 -- 172.21.15.90:0/4218902475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265a51b0 msgr2=0x55ac269d2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.971+0000 7f5dbc973740 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265a51b0 0x55ac269d2e30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5da80df500 tx=0x7f5da80a9760 comp rx=0 tx=0).stop 2024-05-27T07:47:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.971+0000 7f5dbc973740 1 -- 172.21.15.90:0/4218902475 shutdown_connections 2024-05-27T07:47:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.971+0000 7f5dbc973740 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ac265391c0 0x55ac269d6d60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.971+0000 7f5dbc973740 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ac265844d0 0x55ac269558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.971+0000 7f5dbc973740 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ac265a51b0 0x55ac269d2e30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.971+0000 7f5dbc973740 1 --2- 172.21.15.90:0/4218902475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ac2657c2f0 0x55ac269553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.971+0000 7f5dbc973740 1 -- 172.21.15.90:0/4218902475 >> 172.21.15.90:0/4218902475 conn(0x55ac26599300 msgr2=0x55ac26948330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.971+0000 7f5d98c00640 0 client.15513 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.971+0000 7f5dbc973740 1 -- 172.21.15.90:0/4218902475 shutdown_connections 2024-05-27T07:47:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.972+0000 7f5dbc973740 1 -- 172.21.15.90:0/4218902475 wait complete. 2024-05-27T07:47:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.972+0000 7f5dbc973740 20 client.15513 trim_cache size 0 max 16384 2024-05-27T07:47:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:47:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:47:15.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 -- 172.21.15.90:0/1566574768 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d940b2b00 msgr2=0x7f5d940b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d940b2b00 0x7f5d940b5010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55ac26948e10 tx=0x7f5da000c430 comp rx=0 tx=0).stop 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 -- 172.21.15.90:0/1566574768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db010c180 msgr2=0x7f5db019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db010c180 0x7f5db019c710 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5da800b970 tx=0x7f5da800be30 comp rx=0 tx=0).stop 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 -- 172.21.15.90:0/1566574768 shutdown_connections 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d940b2b00 0x7f5d940b5010 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db010bce0 0x7f5db01a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db010c180 0x7f5db019c710 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 --2- 172.21.15.90:0/1566574768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db0106430 0x7f5db01a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 -- 172.21.15.90:0/1566574768 >> 172.21.15.90:0/1566574768 conn(0x7f5db0101380 msgr2=0x7f5db0111880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 -- 172.21.15.90:0/1566574768 shutdown_connections 2024-05-27T07:47:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:15.973+0000 7f5dafe00640 1 -- 172.21.15.90:0/1566574768 wait complete. 2024-05-27T07:47:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:16 smithi090 ceph-mon[31429]: pgmap v27: 129 pgs: 129 active+clean; 35 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-05-27T07:47:16.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:16 smithi136 ceph-mon[35324]: pgmap v27: 129 pgs: 129 active+clean; 35 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-05-27T07:47:16.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:16 smithi129 ceph-mon[35254]: pgmap v27: 129 pgs: 129 active+clean; 35 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-05-27T07:47:16.908 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:16 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-05-27T07:47:16.908 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:16 smithi136 systemd[1]: Stopped Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:47:16.908 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:16 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 3.098s CPU time. 2024-05-27T07:47:16.908 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:16 smithi136 systemd[1]: Starting Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:47:17.234 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:16 smithi136 podman[70347]: 2024-05-27 07:47:16.813839498 +0000 UTC m=+0.034303890 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:47:17.235 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:16 smithi136 podman[70347]: 2024-05-27 07:47:16.937005617 +0000 UTC m=+0.157470065 container create 1a38beffc215fc713acdf281c518e8113f882c927ba680cda57773656a282a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:47:17.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:17 smithi136 podman[70347]: 2024-05-27 07:47:17.235285686 +0000 UTC m=+0.455750151 container init 1a38beffc215fc713acdf281c518e8113f882c927ba680cda57773656a282a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521) 2024-05-27T07:47:17.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:17 smithi136 podman[70347]: 2024-05-27 07:47:17.245526836 +0000 UTC m=+0.465991284 container start 1a38beffc215fc713acdf281c518e8113f882c927ba680cda57773656a282a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, ceph=True) 2024-05-27T07:47:17.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:17 smithi136 bash[70347]: 1a38beffc215fc713acdf281c518e8113f882c927ba680cda57773656a282a85 2024-05-27T07:47:17.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:17 smithi136 systemd[1]: Started Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:47:17.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:17 smithi136 ceph-mds[70365]: -- 172.21.15.136:0/1945069420 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e33c317a40 con 0x55e33c35fa80 2024-05-27T07:47:17.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:47:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: starting mds.c at 2024-05-27T07:47:18.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:18 smithi090 ceph-mon[31429]: pgmap v28: 129 pgs: 129 active+clean; 35 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 66 op/s 2024-05-27T07:47:18.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:18 smithi090 ceph-mon[31429]: daemon mds.c restarted 2024-05-27T07:47:18.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:18 smithi136 ceph-mon[35324]: pgmap v28: 129 pgs: 129 active+clean; 35 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 66 op/s 2024-05-27T07:47:18.496 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:18 smithi136 ceph-mon[35324]: daemon mds.c restarted 2024-05-27T07:47:18.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:18 smithi129 ceph-mon[35254]: pgmap v28: 129 pgs: 129 active+clean; 35 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 66 op/s 2024-05-27T07:47:18.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:18 smithi129 ceph-mon[35254]: daemon mds.c restarted 2024-05-27T07:47:19.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:19 smithi090 ceph-mon[31429]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T07:47:19.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:19 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.136:6836/504128634,v1:172.21.15.136:6837/504128634] up:boot 2024-05-27T07:47:19.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:19 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:47:19.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:19 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:47:19.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:19 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:47:19.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:19 smithi136 ceph-mon[35324]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T07:47:19.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:19 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.136:6836/504128634,v1:172.21.15.136:6837/504128634] up:boot 2024-05-27T07:47:19.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:19 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:47:19.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:19 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:47:19.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:19 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:47:19.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:19 smithi129 ceph-mon[35254]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T07:47:19.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:19 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.136:6836/504128634,v1:172.21.15.136:6837/504128634] up:boot 2024-05-27T07:47:19.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:19 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:47:19.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:19 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:47:19.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:19 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:47:20.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:20 smithi090 ceph-mon[31429]: pgmap v30: 129 pgs: 129 active+clean; 44 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-05-27T07:47:20.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:20 smithi136 ceph-mon[35324]: pgmap v30: 129 pgs: 129 active+clean; 44 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-05-27T07:47:20.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:20 smithi129 ceph-mon[35254]: pgmap v30: 129 pgs: 129 active+clean; 44 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-05-27T07:47:22.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:22 smithi090 ceph-mon[31429]: pgmap v31: 129 pgs: 129 active+clean; 44 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-05-27T07:47:22.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:22 smithi136 ceph-mon[35324]: pgmap v31: 129 pgs: 129 active+clean; 44 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-05-27T07:47:22.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:22 smithi129 ceph-mon[35254]: pgmap v31: 129 pgs: 129 active+clean; 44 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-05-27T07:47:22.857 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:47:23.335 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:24.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:24 smithi090 ceph-mon[31429]: pgmap v32: 129 pgs: 129 active+clean; 52 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-05-27T07:47:24.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:24 smithi136 ceph-mon[35324]: pgmap v32: 129 pgs: 129 active+clean; 52 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-05-27T07:47:24.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:24 smithi129 ceph-mon[35254]: pgmap v32: 129 pgs: 129 active+clean; 52 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-05-27T07:47:25.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.705+0000 7f3e33400700 1 -- 172.21.15.90:0/2643017033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2c10c5a0 msgr2=0x7f3e2c10c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:25.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.705+0000 7f3e33400700 1 --2- 172.21.15.90:0/2643017033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2c10c5a0 0x7f3e2c10c9b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009990 tx=0x7f3e20009ca0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.705+0000 7f3e33400700 1 -- 172.21.15.90:0/2643017033 shutdown_connections 2024-05-27T07:47:25.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.705+0000 7f3e33400700 1 --2- 172.21.15.90:0/2643017033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e2c106eb0 0x7f3e2c113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.705+0000 7f3e33400700 1 --2- 172.21.15.90:0/2643017033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2c10c5a0 0x7f3e2c10c9b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.705+0000 7f3e33400700 1 --2- 172.21.15.90:0/2643017033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e2c106560 0x7f3e2c106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.705+0000 7f3e33400700 1 -- 172.21.15.90:0/2643017033 >> 172.21.15.90:0/2643017033 conn(0x7f3e2c102040 msgr2=0x7f3e2c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:25.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.706+0000 7f3e33400700 1 -- 172.21.15.90:0/2643017033 shutdown_connections 2024-05-27T07:47:25.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.706+0000 7f3e33400700 1 -- 172.21.15.90:0/2643017033 wait complete. 2024-05-27T07:47:25.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.707+0000 7f3e33400700 1 Processor -- start 2024-05-27T07:47:25.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.707+0000 7f3e33400700 1 -- start start 2024-05-27T07:47:25.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.708+0000 7f3e33400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2c106560 0x7f3e2c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.708+0000 7f3e33400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e2c106eb0 0x7f3e2c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.708+0000 7f3e33400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e2c10c5a0 0x7f3e2c1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.708+0000 7f3e33400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1165b0 con 0x7f3e2c106560 2024-05-27T07:47:25.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.708+0000 7f3e33400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c116430 con 0x7f3e2c106eb0 2024-05-27T07:47:25.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.708+0000 7f3e33400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c116730 con 0x7f3e2c10c5a0 2024-05-27T07:47:25.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.708+0000 7f3e30a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2c106560 0x7f3e2c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.708+0000 7f3e30a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2c106560 0x7f3e2c19cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41764/0 (socket says 172.21.15.90:41764) 2024-05-27T07:47:25.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.708+0000 7f3e30a00700 1 -- 172.21.15.90:0/495795022 learned_addr learned my addr 172.21.15.90:0/495795022 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:25.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.708+0000 7f3e31400700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e2c10c5a0 0x7f3e2c1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.709+0000 7f3e2be00700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e2c106eb0 0x7f3e2c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.709+0000 7f3e30a00700 1 -- 172.21.15.90:0/495795022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e2c10c5a0 msgr2=0x7f3e2c1a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:25.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.709+0000 7f3e30a00700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e2c10c5a0 0x7f3e2c1a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.709+0000 7f3e30a00700 1 -- 172.21.15.90:0/495795022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e2c106eb0 msgr2=0x7f3e2c19d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:25.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.709+0000 7f3e30a00700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e2c106eb0 0x7f3e2c19d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.709+0000 7f3e30a00700 1 -- 172.21.15.90:0/495795022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c1a1e60 con 0x7f3e2c106560 2024-05-27T07:47:25.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.709+0000 7f3e2be00700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e2c106eb0 0x7f3e2c19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:25.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.709+0000 7f3e31400700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e2c10c5a0 0x7f3e2c1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:25.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.709+0000 7f3e30a00700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2c106560 0x7f3e2c19cfe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e20000c00 tx=0x7f3e20005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:25.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.710+0000 7f3e29600700 1 -- 172.21.15.90:0/495795022 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20027070 con 0x7f3e2c106560 2024-05-27T07:47:25.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.710+0000 7f3e29600700 1 -- 172.21.15.90:0/495795022 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e2000fb80 con 0x7f3e2c106560 2024-05-27T07:47:25.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.710+0000 7f3e33400700 1 -- 172.21.15.90:0/495795022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2c1a20f0 con 0x7f3e2c106560 2024-05-27T07:47:25.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.710+0000 7f3e33400700 1 -- 172.21.15.90:0/495795022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2c1a25d0 con 0x7f3e2c106560 2024-05-27T07:47:25.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.710+0000 7f3e29600700 1 -- 172.21.15.90:0/495795022 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20022c00 con 0x7f3e2c106560 2024-05-27T07:47:25.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.715+0000 7f3e29600700 1 -- 172.21.15.90:0/495795022 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f3e200050d0 con 0x7f3e2c106560 2024-05-27T07:47:25.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.716+0000 7f3e41ec1b80 1 Processor -- start 2024-05-27T07:47:25.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.716+0000 7f3e41ec1b80 1 -- start start 2024-05-27T07:47:25.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.717+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac49719f0 0x55bac4a04330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.717+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 0x55bac4a04870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.717+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a11110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.717+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bac4932e40 con 0x55bac4a04e70 2024-05-27T07:47:25.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.717+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bac49229e0 con 0x55bac4885e40 2024-05-27T07:47:25.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.717+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bac4926a20 con 0x55bac49719f0 2024-05-27T07:47:25.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e2be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a11110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e2be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a11110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41778/0 (socket says 172.21.15.90:41778) 2024-05-27T07:47:25.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e2be00700 1 -- 172.21.15.90:0/790182492 learned_addr learned my addr 172.21.15.90:0/790182492 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:25.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e31400700 1 --2- 172.21.15.90:0/790182492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac49719f0 0x55bac4a04330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e30a00700 1 --2- 172.21.15.90:0/790182492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 0x55bac4a04870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e2be00700 1 -- 172.21.15.90:0/790182492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac49719f0 msgr2=0x55bac4a04330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:25.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e2be00700 1 --2- 172.21.15.90:0/790182492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac49719f0 0x55bac4a04330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e2be00700 1 -- 172.21.15.90:0/790182492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 msgr2=0x55bac4a04870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:25.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e2be00700 1 --2- 172.21.15.90:0/790182492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 0x55bac4a04870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e2be00700 1 -- 172.21.15.90:0/790182492 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac48eb420 con 0x55bac4a04e70 2024-05-27T07:47:25.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.718+0000 7f3e29600700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e140b2ba0 0x7f3e140b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.719+0000 7f3e31400700 1 --2- 172.21.15.90:0/790182492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac49719f0 0x55bac4a04330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:25.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.719+0000 7f3e29600700 1 -- 172.21.15.90:0/495795022 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f3e200e37f0 con 0x7f3e2c106560 2024-05-27T07:47:25.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.719+0000 7f3e31400700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e140b2ba0 0x7f3e140b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.719+0000 7f3e2be00700 1 --2- 172.21.15.90:0/790182492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a11110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800bd00 tx=0x7f3e1800f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc04766ba1e753dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:25.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.719+0000 7f3e1e000700 1 -- 172.21.15.90:0/790182492 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1800fda0 con 0x55bac4a04e70 2024-05-27T07:47:25.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.720+0000 7f3e1e000700 1 -- 172.21.15.90:0/790182492 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e1800be50 con 0x55bac4a04e70 2024-05-27T07:47:25.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.720+0000 7f3e1e000700 1 -- 172.21.15.90:0/790182492 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1801b650 con 0x55bac4a04e70 2024-05-27T07:47:25.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.721+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/790182492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 msgr2=0x55bac4a11110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:25.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.721+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/790182492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a11110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800bd00 tx=0x7f3e1800f640 comp rx=0 tx=0).stop 2024-05-27T07:47:25.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.721+0000 7f3e31400700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e140b2ba0 0x7f3e140b50b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55bac4a06ff0 tx=0x7f3e2400b040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:25.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.721+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/790182492 shutdown_connections 2024-05-27T07:47:25.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.721+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/790182492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a11110 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.721+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/790182492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac49719f0 0x55bac4a04330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.721+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/790182492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 0x55bac4a04870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.721+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/790182492 >> 172.21.15.90:0/790182492 conn(0x55bac491f390 msgr2=0x55bac4a004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:25.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.721+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/790182492 shutdown_connections 2024-05-27T07:47:25.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.721+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/790182492 wait complete. 2024-05-27T07:47:25.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.722+0000 7f3e41ec1b80 1 -- start start 2024-05-27T07:47:25.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.722+0000 7f3e41ec1b80 1 Processor -- start 2024-05-27T07:47:25.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.723+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 0x55bac4a0e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.723+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac49719f0 0x55bac4a0ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.723+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a0f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.723+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bac4828fa0 con 0x55bac4a04e70 2024-05-27T07:47:25.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.723+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bac48c3760 con 0x55bac4885e40 2024-05-27T07:47:25.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.723+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bac4a11b80 con 0x55bac49719f0 2024-05-27T07:47:25.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.723+0000 7f3e41ec1b80 10 client.0 _ll_register_callbacks cb 0x55bac48ccc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:25.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.723+0000 7f3e31400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a0f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.723+0000 7f3e31400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a0f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41780/0 (socket says 172.21.15.90:41780) 2024-05-27T07:47:25.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.723+0000 7f3e31400700 1 -- 172.21.15.90:0/2981414727 learned_addr learned my addr 172.21.15.90:0/2981414727 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.724+0000 7f3e30a00700 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 0x55bac4a0e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.724+0000 7f3e31400700 1 -- 172.21.15.90:0/2981414727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac49719f0 msgr2=0x55bac4a0ebc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:47:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.724+0000 7f3e31400700 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac49719f0 0x55bac4a0ebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.724+0000 7f3e31400700 1 -- 172.21.15.90:0/2981414727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 msgr2=0x55bac4a0e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.724+0000 7f3e31400700 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 0x55bac4a0e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.724+0000 7f3e31400700 1 -- 172.21.15.90:0/2981414727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac48728d0 con 0x55bac4a04e70 2024-05-27T07:47:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.724+0000 7f3e30a00700 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 0x55bac4a0e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.725+0000 7f3e31400700 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a0f100 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3e240179b0 tx=0x7f3e24004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e030ce5dc7b937a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.725+0000 7f3e12a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.725+0000 7f3e12a00700 1 -- 172.21.15.90:0/2981414727 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24027060 con 0x55bac4a04e70 2024-05-27T07:47:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.725+0000 7f3e12a00700 1 -- 172.21.15.90:0/2981414727 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f3e24015a80 con 0x55bac4a04e70 2024-05-27T07:47:25.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.725+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2981414727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e1800be50 con 0x55bac4a04e70 2024-05-27T07:47:25.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.726+0000 7f3e12a00700 1 -- 172.21.15.90:0/2981414727 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3e24017e20 con 0x55bac4a04e70 2024-05-27T07:47:25.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.726+0000 7f3e41ec1b80 10 client.15525 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:25.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.726+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2981414727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bac48f3060 con 0x55bac4a04e70 2024-05-27T07:47:25.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.726+0000 7f3e12a00700 1 -- 172.21.15.90:0/2981414727 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e1800be50 con 0x55bac4a04e70 2024-05-27T07:47:25.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.727+0000 7f3e12a00700 1 -- 172.21.15.90:0/2981414727 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3e24016070 con 0x55bac4a04e70 2024-05-27T07:47:25.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.727+0000 7f3e41ec1b80 10 client.15525 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:25.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.727+0000 7f3e41ec1b80 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:25.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.727+0000 7f3e41ec1b80 20 client.15525 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:25.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.727+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac4a27350 0x55bac4a29820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.727+0000 7f3e41ec1b80 4 client.15525 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:25.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.727+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2981414727 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bac4934d90 con 0x55bac4a27350 2024-05-27T07:47:25.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.728+0000 7f3e30a00700 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac4a27350 0x55bac4a29820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.729+0000 7f3e30a00700 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac4a27350 0x55bac4a29820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b1625f4ca0c720 server_cookie=2c98e0fd9f5372d2 in_seq=0 out_seq=0 2024-05-27T07:47:25.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.729+0000 7f3e12a00700 10 client.15525 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:25.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.731+0000 7f3e12a00700 1 -- 172.21.15.90:0/2981414727 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bac4934d90 con 0x55bac4a27350 2024-05-27T07:47:25.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.731+0000 7f3e12a00700 10 client.15525 handle_command_reply: tid=0 2024-05-27T07:47:25.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.731+0000 7f3e41ec1b80 1 client.15525 shutdown 2024-05-27T07:47:25.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.732+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2981414727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 msgr2=0x55bac4a0f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:25.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.732+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a0f100 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3e240179b0 tx=0x7f3e24004b40 comp rx=0 tx=0).stop 2024-05-27T07:47:25.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.732+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2981414727 shutdown_connections 2024-05-27T07:47:25.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.732+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac4a27350 0x55bac4a29820 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.732+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4a04e70 0x55bac4a0f100 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.732+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4885e40 0x55bac4a0e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.732+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2981414727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac49719f0 0x55bac4a0ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.732+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2981414727 >> 172.21.15.90:0/2981414727 conn(0x55bac491f390 msgr2=0x55bac4a04310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:25.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.732+0000 7f3e12a00700 0 client.15525 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:25.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.733+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2981414727 shutdown_connections 2024-05-27T07:47:25.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.733+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2981414727 wait complete. 2024-05-27T07:47:25.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.733+0000 7f3e41ec1b80 20 client.15525 trim_cache size 0 max 16384 2024-05-27T07:47:25.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.763+0000 7f3e41ec1b80 1 Processor -- start 2024-05-27T07:47:25.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.763+0000 7f3e41ec1b80 1 -- start start 2024-05-27T07:47:25.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.764+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4885e40 0x55bac4a09e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.764+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac49719f0 0x55bac4a2f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.764+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4a04e70 0x55bac4a2f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.764+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c00ecc0 con 0x55bac49719f0 2024-05-27T07:47:25.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.764+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c010800 con 0x55bac4a04e70 2024-05-27T07:47:25.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.764+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c00f390 con 0x55bac4885e40 2024-05-27T07:47:25.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.764+0000 7f3e2be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac49719f0 0x55bac4a2f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.764+0000 7f3e2be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac49719f0 0x55bac4a2f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41786/0 (socket says 172.21.15.90:41786) 2024-05-27T07:47:25.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.764+0000 7f3e2be00700 1 -- 172.21.15.90:0/1891162984 learned_addr learned my addr 172.21.15.90:0/1891162984 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:25.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.764+0000 7f3e30a00700 1 --2- 172.21.15.90:0/1891162984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4885e40 0x55bac4a09e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.765+0000 7f3e2be00700 1 -- 172.21.15.90:0/1891162984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4885e40 msgr2=0x55bac4a09e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:25.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.765+0000 7f3e2be00700 1 --2- 172.21.15.90:0/1891162984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4885e40 0x55bac4a09e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.765+0000 7f3e2be00700 1 -- 172.21.15.90:0/1891162984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4a04e70 msgr2=0x55bac4a2f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:25.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.765+0000 7f3e31400700 1 --2- 172.21.15.90:0/1891162984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4a04e70 0x55bac4a2f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.765+0000 7f3e2be00700 1 --2- 172.21.15.90:0/1891162984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4a04e70 0x55bac4a2f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.765+0000 7f3e2be00700 1 -- 172.21.15.90:0/1891162984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac49fc240 con 0x55bac49719f0 2024-05-27T07:47:25.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.765+0000 7f3e2be00700 1 --2- 172.21.15.90:0/1891162984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac49719f0 0x55bac4a2f020 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800bfe0 tx=0x7f3e1800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df67a70e9e7ea1c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:25.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.766+0000 7f3e1d600700 1 -- 172.21.15.90:0/1891162984 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1802a070 con 0x55bac49719f0 2024-05-27T07:47:25.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.766+0000 7f3e1d600700 1 -- 172.21.15.90:0/1891162984 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e18005040 con 0x55bac49719f0 2024-05-27T07:47:25.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.766+0000 7f3e1d600700 1 -- 172.21.15.90:0/1891162984 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18004c10 con 0x55bac49719f0 2024-05-27T07:47:25.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.767+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/1891162984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac49719f0 msgr2=0x55bac4a2f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:25.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.767+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/1891162984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac49719f0 0x55bac4a2f020 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800bfe0 tx=0x7f3e1800b460 comp rx=0 tx=0).stop 2024-05-27T07:47:25.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.767+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/1891162984 shutdown_connections 2024-05-27T07:47:25.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.767+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/1891162984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac4a04e70 0x55bac4a2f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.767+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/1891162984 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4885e40 0x55bac4a09e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.767+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/1891162984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac49719f0 0x55bac4a2f020 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.767+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/1891162984 >> 172.21.15.90:0/1891162984 conn(0x55bac491f390 msgr2=0x55bac4a296d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:25.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.768+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/1891162984 shutdown_connections 2024-05-27T07:47:25.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.768+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/1891162984 wait complete. 2024-05-27T07:47:25.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.768+0000 7f3e41ec1b80 1 -- start start 2024-05-27T07:47:25.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.769+0000 7f3e41ec1b80 1 Processor -- start 2024-05-27T07:47:25.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.769+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4885e40 0x55bac4a175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.769+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac49719f0 0x55bac4a08ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.769+0000 7f3e41ec1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4a04e70 0x55bac4a09420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.769+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c010130 con 0x55bac4885e40 2024-05-27T07:47:25.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.769+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c00fa60 con 0x55bac49719f0 2024-05-27T07:47:25.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.769+0000 7f3e41ec1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c00cc40 con 0x55bac4a04e70 2024-05-27T07:47:25.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.769+0000 7f3e41ec1b80 10 client.0 _ll_register_callbacks cb 0x55bac48ccc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:25.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.770+0000 7f3e30a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4885e40 0x55bac4a175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.770+0000 7f3e30a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4885e40 0x55bac4a175d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41792/0 (socket says 172.21.15.90:41792) 2024-05-27T07:47:25.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.770+0000 7f3e30a00700 1 -- 172.21.15.90:0/2478544039 learned_addr learned my addr 172.21.15.90:0/2478544039 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:25.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.770+0000 7f3e2be00700 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac49719f0 0x55bac4a08ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.770+0000 7f3e30a00700 1 -- 172.21.15.90:0/2478544039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4a04e70 msgr2=0x55bac4a09420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:25.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.770+0000 7f3e31400700 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4a04e70 0x55bac4a09420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.770+0000 7f3e30a00700 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4a04e70 0x55bac4a09420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.770+0000 7f3e30a00700 1 -- 172.21.15.90:0/2478544039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac49719f0 msgr2=0x55bac4a08ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:25.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.770+0000 7f3e30a00700 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac49719f0 0x55bac4a08ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.770+0000 7f3e30a00700 1 -- 172.21.15.90:0/2478544039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac4a09b40 con 0x55bac4885e40 2024-05-27T07:47:25.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.771+0000 7f3e30a00700 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4885e40 0x55bac4a175d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3e200a6630 tx=0x7f3e200e9530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cf1b28d0c5ec1b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:25.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.771+0000 7f3e12a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:25.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.771+0000 7f3e31400700 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4a04e70 0x55bac4a09420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:25.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.772+0000 7f3e12a00700 1 -- 172.21.15.90:0/2478544039 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e200a6470 con 0x55bac4885e40 2024-05-27T07:47:25.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.772+0000 7f3e12a00700 1 -- 172.21.15.90:0/2478544039 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f3e200e3420 con 0x55bac4885e40 2024-05-27T07:47:25.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.772+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2478544039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e18005040 con 0x55bac4885e40 2024-05-27T07:47:25.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.772+0000 7f3e12a00700 1 -- 172.21.15.90:0/2478544039 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3e200053c0 con 0x55bac4885e40 2024-05-27T07:47:25.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.772+0000 7f3e41ec1b80 10 client.15531 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:25.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.772+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2478544039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bac4a91ef0 con 0x55bac4885e40 2024-05-27T07:47:25.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.773+0000 7f3e12a00700 1 -- 172.21.15.90:0/2478544039 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e18005040 con 0x55bac4885e40 2024-05-27T07:47:25.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.773+0000 7f3e12a00700 1 -- 172.21.15.90:0/2478544039 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f3e2002a030 con 0x55bac4885e40 2024-05-27T07:47:25.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.773+0000 7f3e41ec1b80 10 client.15531 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:25.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.773+0000 7f3e41ec1b80 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:25.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.773+0000 7f3e41ec1b80 20 client.15531 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:25.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.773+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac4a924a0 0x55bac4a948c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:25.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.773+0000 7f3e41ec1b80 4 client.15531 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:47:25.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.773+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2478544039 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bac492e570 con 0x55bac4a924a0 2024-05-27T07:47:25.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.774+0000 7f3e2be00700 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac4a924a0 0x55bac4a948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:25.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.775+0000 7f3e12a00700 10 client.15531 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:25.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.775+0000 7f3e2be00700 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac4a924a0 0x55bac4a948c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba6b32e63e0b768a server_cookie=ac61dd3ee883ef39 in_seq=0 out_seq=0 2024-05-27T07:47:25.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.781+0000 7f3e12a00700 1 -- 172.21.15.90:0/2478544039 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bac492e570 con 0x55bac4a924a0 2024-05-27T07:47:25.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.781+0000 7f3e12a00700 10 client.15531 handle_command_reply: tid=0 2024-05-27T07:47:25.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.781+0000 7f3e41ec1b80 1 client.15531 shutdown 2024-05-27T07:47:25.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.781+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2478544039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4885e40 msgr2=0x55bac4a175d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.781+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4885e40 0x55bac4a175d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3e200a6630 tx=0x7f3e200e9530 comp rx=0 tx=0).stop 2024-05-27T07:47:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.782+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2478544039 shutdown_connections 2024-05-27T07:47:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.782+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac4a924a0 0x55bac4a948c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.782+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4a04e70 0x55bac4a09420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.782+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4885e40 0x55bac4a175d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.782+0000 7f3e41ec1b80 1 --2- 172.21.15.90:0/2478544039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac49719f0 0x55bac4a08ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.782+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2478544039 >> 172.21.15.90:0/2478544039 conn(0x55bac491f390 msgr2=0x55bac4a05ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.782+0000 7f3e12a00700 0 client.15531 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.782+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2478544039 shutdown_connections 2024-05-27T07:47:25.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.783+0000 7f3e41ec1b80 1 -- 172.21.15.90:0/2478544039 wait complete. 2024-05-27T07:47:25.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.783+0000 7f3e41ec1b80 20 client.15531 trim_cache size 0 max 16384 2024-05-27T07:47:25.783 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:47:25.783 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:47:25.783 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:47:25.783 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:47:25.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 -- 172.21.15.90:0/495795022 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e140b2ba0 msgr2=0x7f3e140b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:25.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e140b2ba0 0x7f3e140b50b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55bac4a06ff0 tx=0x7f3e2400b040 comp rx=0 tx=0).stop 2024-05-27T07:47:25.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 -- 172.21.15.90:0/495795022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2c106560 msgr2=0x7f3e2c19cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:25.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2c106560 0x7f3e2c19cfe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e20000c00 tx=0x7f3e20005ef0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 -- 172.21.15.90:0/495795022 shutdown_connections 2024-05-27T07:47:25.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e2c10c5a0 0x7f3e2c1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2c106560 0x7f3e2c19cfe0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e140b2ba0 0x7f3e140b50b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 --2- 172.21.15.90:0/495795022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e2c106eb0 0x7f3e2c19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:25.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 -- 172.21.15.90:0/495795022 >> 172.21.15.90:0/495795022 conn(0x7f3e2c102040 msgr2=0x7f3e2c103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:25.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 -- 172.21.15.90:0/495795022 shutdown_connections 2024-05-27T07:47:25.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:25.784+0000 7f3e33400700 1 -- 172.21.15.90:0/495795022 wait complete. 2024-05-27T07:47:26.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:26 smithi090 ceph-mon[31429]: pgmap v33: 129 pgs: 129 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-05-27T07:47:26.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:26 smithi136 ceph-mon[35324]: pgmap v33: 129 pgs: 129 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-05-27T07:47:26.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:26 smithi129 ceph-mon[35254]: pgmap v33: 129 pgs: 129 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-05-27T07:47:26.848 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:47:26.848 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:47:27.329 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:28.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:28 smithi090 ceph-mon[31429]: pgmap v34: 129 pgs: 129 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-05-27T07:47:28.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:28 smithi136 ceph-mon[35324]: pgmap v34: 129 pgs: 129 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-05-27T07:47:28.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:28 smithi129 ceph-mon[35254]: pgmap v34: 129 pgs: 129 active+clean; 54 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.1 MiB/s wr, 156 op/s 2024-05-27T07:47:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.215+0000 7f1549600640 1 -- 172.21.15.90:0/2371905688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15441043e0 msgr2=0x7f15441107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.215+0000 7f1549600640 1 --2- 172.21.15.90:0/2371905688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15441043e0 0x7f15441107b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f153c009ca0 tx=0x7f153c02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.215+0000 7f1549600640 1 -- 172.21.15.90:0/2371905688 shutdown_connections 2024-05-27T07:47:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.215+0000 7f1549600640 1 --2- 172.21.15.90:0/2371905688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15441043e0 0x7f15441107b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.215+0000 7f1549600640 1 --2- 172.21.15.90:0/2371905688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f154410a130 0x7f1544103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.215+0000 7f1549600640 1 --2- 172.21.15.90:0/2371905688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1544109c90 0x7f1544103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.215+0000 7f1549600640 1 -- 172.21.15.90:0/2371905688 >> 172.21.15.90:0/2371905688 conn(0x7f1544079390 msgr2=0x7f15440797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.215+0000 7f1549600640 1 -- 172.21.15.90:0/2371905688 shutdown_connections 2024-05-27T07:47:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.215+0000 7f1549600640 1 -- 172.21.15.90:0/2371905688 wait complete. 2024-05-27T07:47:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.216+0000 7f1549600640 1 Processor -- start 2024-05-27T07:47:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.216+0000 7f1549600640 1 -- start start 2024-05-27T07:47:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.217+0000 7f1549600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15441043e0 0x7f154419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.217+0000 7f1549600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544109c90 0x7f154419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.217+0000 7f1549600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154410a130 0x7f154419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.217+0000 7f1549600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1544113d80 con 0x7f1544109c90 2024-05-27T07:47:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.217+0000 7f1549600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1544113c00 con 0x7f154410a130 2024-05-27T07:47:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.217+0000 7f1549600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1544113f00 con 0x7f15441043e0 2024-05-27T07:47:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.217+0000 7f1542000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544109c90 0x7f154419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.217+0000 7f1542000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544109c90 0x7f154419ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35612/0 (socket says 172.21.15.90:35612) 2024-05-27T07:47:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.217+0000 7f1542000640 1 -- 172.21.15.90:0/710078334 learned_addr learned my addr 172.21.15.90:0/710078334 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f1543400640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154410a130 0x7f154419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f1542a00640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15441043e0 0x7f154419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f1542000640 1 -- 172.21.15.90:0/710078334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15441043e0 msgr2=0x7f154419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f1542000640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15441043e0 0x7f154419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f1542000640 1 -- 172.21.15.90:0/710078334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154410a130 msgr2=0x7f154419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f1542000640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154410a130 0x7f154419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f1542000640 1 -- 172.21.15.90:0/710078334 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154419f600 con 0x7f1544109c90 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f1543400640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154410a130 0x7f154419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f1542000640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544109c90 0x7f154419ad00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f153000afa0 tx=0x7f153000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f153b400640 1 -- 172.21.15.90:0/710078334 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530004220 con 0x7f1544109c90 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f153b400640 1 -- 172.21.15.90:0/710078334 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1530007e10 con 0x7f1544109c90 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f1549600640 1 -- 172.21.15.90:0/710078334 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154419f8f0 con 0x7f1544109c90 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.218+0000 7f153b400640 1 -- 172.21.15.90:0/710078334 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530003610 con 0x7f1544109c90 2024-05-27T07:47:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.219+0000 7f1549600640 1 -- 172.21.15.90:0/710078334 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1544074ef0 con 0x7f1544109c90 2024-05-27T07:47:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.223+0000 7f155063a740 1 Processor -- start 2024-05-27T07:47:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.223+0000 7f155063a740 1 -- start start 2024-05-27T07:47:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.223+0000 7f155063a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x556308377860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.223+0000 7f155063a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308377da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.223+0000 7f155063a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fb44d0 0x5563083782e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.223+0000 7f155063a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5563082e8b90 con 0x556307fd51b0 2024-05-27T07:47:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.223+0000 7f155063a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5563082d8730 con 0x556307fac2f0 2024-05-27T07:47:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.223+0000 7f155063a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5563082dc770 con 0x556307fb44d0 2024-05-27T07:47:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f1543400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x556308377860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f1543400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x556308377860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35620/0 (socket says 172.21.15.90:35620) 2024-05-27T07:47:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f1543400640 1 -- 172.21.15.90:0/2675238031 learned_addr learned my addr 172.21.15.90:0/2675238031 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f153b400640 1 -- 172.21.15.90:0/710078334 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f1530007980 con 0x7f1544109c90 2024-05-27T07:47:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f1542a00640 1 --2- 172.21.15.90:0/2675238031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308377da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f1543400640 1 -- 172.21.15.90:0/2675238031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fb44d0 msgr2=0x5563083782e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f1543400640 1 --2- 172.21.15.90:0/2675238031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fb44d0 0x5563083782e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f1543400640 1 -- 172.21.15.90:0/2675238031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 msgr2=0x556308377da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f1543400640 1 --2- 172.21.15.90:0/2675238031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308377da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f1543400640 1 -- 172.21.15.90:0/2675238031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563082718b0 con 0x556307fd51b0 2024-05-27T07:47:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.224+0000 7f1542a00640 1 --2- 172.21.15.90:0/2675238031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308377da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.225+0000 7f1543400640 1 --2- 172.21.15.90:0/2675238031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x556308377860 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f153c009a60 tx=0x7f153c031610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be57cc4672ebf050 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.225+0000 7f152f400640 1 -- 172.21.15.90:0/2675238031 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c031b80 con 0x556307fd51b0 2024-05-27T07:47:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.225+0000 7f152f400640 1 -- 172.21.15.90:0/2675238031 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f153c036040 con 0x556307fd51b0 2024-05-27T07:47:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.226+0000 7f153b400640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f15280b2b00 0x7f15280b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.226+0000 7f153b400640 1 -- 172.21.15.90:0/710078334 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f15300e6910 con 0x7f1544109c90 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.226+0000 7f1542a00640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f15280b2b00 0x7f15280b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.226+0000 7f155063a740 1 -- 172.21.15.90:0/2675238031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 msgr2=0x556308377860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.226+0000 7f155063a740 1 --2- 172.21.15.90:0/2675238031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x556308377860 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f153c009a60 tx=0x7f153c031610 comp rx=0 tx=0).stop 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.226+0000 7f155063a740 1 -- 172.21.15.90:0/2675238031 shutdown_connections 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.226+0000 7f155063a740 1 --2- 172.21.15.90:0/2675238031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308377da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.226+0000 7f155063a740 1 --2- 172.21.15.90:0/2675238031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fb44d0 0x5563083782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.226+0000 7f155063a740 1 --2- 172.21.15.90:0/2675238031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x556308377860 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.227+0000 7f155063a740 1 -- 172.21.15.90:0/2675238031 >> 172.21.15.90:0/2675238031 conn(0x556307fc9300 msgr2=0x556308374010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.227+0000 7f155063a740 1 -- 172.21.15.90:0/2675238031 shutdown_connections 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.227+0000 7f155063a740 1 -- 172.21.15.90:0/2675238031 wait complete. 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.227+0000 7f1542a00640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f15280b2b00 0x7f15280b5010 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x556308378c10 tx=0x7f153400b710 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:29.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.227+0000 7f155063a740 1 -- start start 2024-05-27T07:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.228+0000 7f155063a740 1 Processor -- start 2024-05-27T07:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.228+0000 7f155063a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fac2f0 0x556308381990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.228+0000 7f155063a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fb44d0 0x556308381ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.228+0000 7f155063a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x55630838dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.228+0000 7f155063a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556307fbe790 con 0x556307fd51b0 2024-05-27T07:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.228+0000 7f155063a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556308280bc0 con 0x556307fb44d0 2024-05-27T07:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.228+0000 7f155063a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556308384980 con 0x556307fac2f0 2024-05-27T07:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.229+0000 7f155063a740 10 client.0 _ll_register_callbacks cb 0x55630825e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.229+0000 7f1543400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x55630838dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.229+0000 7f1542a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fac2f0 0x556308381990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.229+0000 7f1543400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x55630838dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35626/0 (socket says 172.21.15.90:35626) 2024-05-27T07:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.229+0000 7f1543400640 1 -- 172.21.15.90:0/108453831 learned_addr learned my addr 172.21.15.90:0/108453831 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.229+0000 7f1542000640 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fb44d0 0x556308381ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.230+0000 7f1543400640 1 -- 172.21.15.90:0/108453831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fac2f0 msgr2=0x556308381990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.230+0000 7f1543400640 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fac2f0 0x556308381990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.230+0000 7f1543400640 1 -- 172.21.15.90:0/108453831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fb44d0 msgr2=0x556308381ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.230+0000 7f1543400640 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fb44d0 0x556308381ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.230+0000 7f1543400640 1 -- 172.21.15.90:0/108453831 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556307fed210 con 0x556307fd51b0 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.230+0000 7f1543400640 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x55630838dd30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f153c0094b0 tx=0x7f153c0099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78276599e462f002 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.230+0000 7f1523e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.230+0000 7f1523e00640 1 -- 172.21.15.90:0/108453831 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c04a040 con 0x556307fd51b0 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.231+0000 7f1523e00640 1 -- 172.21.15.90:0/108453831 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f153c052bf0 con 0x556307fd51b0 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.231+0000 7f155063a740 1 -- 172.21.15.90:0/108453831 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f153c036040 con 0x556307fd51b0 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.231+0000 7f1523e00640 1 -- 172.21.15.90:0/108453831 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f153c052df0 con 0x556307fd51b0 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.231+0000 7f155063a740 10 client.15543 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.231+0000 7f155063a740 1 -- 172.21.15.90:0/108453831 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5563080b9620 con 0x556307fd51b0 2024-05-27T07:47:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.231+0000 7f1523e00640 1 -- 172.21.15.90:0/108453831 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f153c036040 con 0x556307fd51b0 2024-05-27T07:47:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.232+0000 7f1523e00640 1 -- 172.21.15.90:0/108453831 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f153c0305c0 con 0x556307fd51b0 2024-05-27T07:47:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.232+0000 7f155063a740 10 client.15543 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.232+0000 7f155063a740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.232+0000 7f155063a740 20 client.15543 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.232+0000 7f155063a740 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556307f93d80 0x5563083a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.232+0000 7f155063a740 4 client.15543 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.232+0000 7f155063a740 1 -- 172.21.15.90:0/108453831 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563082eaae0 con 0x556307f93d80 2024-05-27T07:47:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.233+0000 7f1542a00640 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556307f93d80 0x5563083a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.234+0000 7f1523e00640 10 client.15543 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.234+0000 7f1542a00640 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556307f93d80 0x5563083a4ab0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=246b4d9eefbd759e server_cookie=ecc35468e4abe659 in_seq=0 out_seq=0 2024-05-27T07:47:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f1523e00640 1 -- 172.21.15.90:0/108453831 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5563082eaae0 con 0x556307f93d80 2024-05-27T07:47:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f1523e00640 10 client.15543 handle_command_reply: tid=0 2024-05-27T07:47:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f155063a740 1 client.15543 shutdown 2024-05-27T07:47:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f155063a740 1 -- 172.21.15.90:0/108453831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 msgr2=0x55630838dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f155063a740 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x55630838dd30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f153c0094b0 tx=0x7f153c0099f0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f155063a740 1 -- 172.21.15.90:0/108453831 shutdown_connections 2024-05-27T07:47:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f155063a740 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556307f93d80 0x5563083a4ab0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f155063a740 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fb44d0 0x556308381ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f155063a740 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x55630838dd30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f155063a740 1 --2- 172.21.15.90:0/108453831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fac2f0 0x556308381990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f155063a740 1 -- 172.21.15.90:0/108453831 >> 172.21.15.90:0/108453831 conn(0x556307fc9300 msgr2=0x556308377690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.240+0000 7f1523e00640 0 client.15543 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.241+0000 7f155063a740 1 -- 172.21.15.90:0/108453831 shutdown_connections 2024-05-27T07:47:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.241+0000 7f155063a740 1 -- 172.21.15.90:0/108453831 wait complete. 2024-05-27T07:47:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.241+0000 7f155063a740 20 client.15543 trim_cache size 0 max 16384 2024-05-27T07:47:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.265+0000 7f155063a740 1 Processor -- start 2024-05-27T07:47:29.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.265+0000 7f155063a740 1 -- start start 2024-05-27T07:47:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.265+0000 7f155063a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308383b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.265+0000 7f155063a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fb44d0 0x55630838dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.265+0000 7f155063a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x5563083a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.265+0000 7f155063a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1544074450 con 0x556307fd51b0 2024-05-27T07:47:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.265+0000 7f155063a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15441a8310 con 0x556307fac2f0 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.265+0000 7f155063a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1544074b20 con 0x556307fb44d0 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.265+0000 7f1543400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x5563083a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.266+0000 7f1542a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308383b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.266+0000 7f1543400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x5563083a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35636/0 (socket says 172.21.15.90:35636) 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.266+0000 7f1543400640 1 -- 172.21.15.90:0/200084274 learned_addr learned my addr 172.21.15.90:0/200084274 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.266+0000 7f1542000640 1 --2- 172.21.15.90:0/200084274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fb44d0 0x55630838dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.266+0000 7f1543400640 1 -- 172.21.15.90:0/200084274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fb44d0 msgr2=0x55630838dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.266+0000 7f1543400640 1 --2- 172.21.15.90:0/200084274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fb44d0 0x55630838dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.266+0000 7f1543400640 1 -- 172.21.15.90:0/200084274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 msgr2=0x556308383b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.266+0000 7f1543400640 1 --2- 172.21.15.90:0/200084274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308383b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.266+0000 7f1543400640 1 -- 172.21.15.90:0/200084274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556308378c40 con 0x556307fd51b0 2024-05-27T07:47:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.266+0000 7f1543400640 1 --2- 172.21.15.90:0/200084274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x5563083a7ae0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f153c000f80 tx=0x7f153c052f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c548b4afbb7d6415 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.267+0000 7f152ea00640 1 -- 172.21.15.90:0/200084274 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c0046d0 con 0x556307fd51b0 2024-05-27T07:47:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.267+0000 7f152ea00640 1 -- 172.21.15.90:0/200084274 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f153c036070 con 0x556307fd51b0 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.267+0000 7f152ea00640 1 -- 172.21.15.90:0/200084274 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c0529f0 con 0x556307fd51b0 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.268+0000 7f155063a740 1 -- 172.21.15.90:0/200084274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 msgr2=0x5563083a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.268+0000 7f155063a740 1 --2- 172.21.15.90:0/200084274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x5563083a7ae0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f153c000f80 tx=0x7f153c052f90 comp rx=0 tx=0).stop 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.268+0000 7f155063a740 1 -- 172.21.15.90:0/200084274 shutdown_connections 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.268+0000 7f155063a740 1 --2- 172.21.15.90:0/200084274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fd51b0 0x5563083a7ae0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.268+0000 7f155063a740 1 --2- 172.21.15.90:0/200084274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fb44d0 0x55630838dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.268+0000 7f155063a740 1 --2- 172.21.15.90:0/200084274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308383b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.268+0000 7f155063a740 1 -- 172.21.15.90:0/200084274 >> 172.21.15.90:0/200084274 conn(0x556307fc9300 msgr2=0x55630837cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.268+0000 7f155063a740 1 -- 172.21.15.90:0/200084274 shutdown_connections 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.268+0000 7f155063a740 1 -- 172.21.15.90:0/200084274 wait complete. 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.268+0000 7f155063a740 1 -- start start 2024-05-27T07:47:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.269+0000 7f155063a740 1 Processor -- start 2024-05-27T07:47:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.269+0000 7f155063a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308405110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.269+0000 7f155063a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fb44d0 0x556308405650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.269+0000 7f155063a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fd51b0 0x5563083ff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.269+0000 7f155063a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f15441a7c40 con 0x556307fb44d0 2024-05-27T07:47:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.269+0000 7f155063a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15441a7570 con 0x556307fac2f0 2024-05-27T07:47:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.269+0000 7f155063a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1544072410 con 0x556307fd51b0 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.269+0000 7f155063a740 10 client.0 _ll_register_callbacks cb 0x7f153c0314e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.269+0000 7f1542000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fb44d0 0x556308405650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1542000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fb44d0 0x556308405650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35644/0 (socket says 172.21.15.90:35644) 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1542000640 1 -- 172.21.15.90:0/1596557034 learned_addr learned my addr 172.21.15.90:0/1596557034 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1542a00640 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308405110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1543400640 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fd51b0 0x5563083ff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1542000640 1 -- 172.21.15.90:0/1596557034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fd51b0 msgr2=0x5563083ff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1542000640 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fd51b0 0x5563083ff500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1542000640 1 -- 172.21.15.90:0/1596557034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 msgr2=0x556308405110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1542000640 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308405110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1542000640 1 -- 172.21.15.90:0/1596557034 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563083ffb10 con 0x556307fb44d0 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1542000640 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fb44d0 0x556308405650 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f153000a5f0 tx=0x7f15300adc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33830dc0f3ec74ed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.270+0000 7f1523e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f1523e00640 1 -- 172.21.15.90:0/1596557034 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530007c70 con 0x556307fb44d0 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f1523e00640 1 -- 172.21.15.90:0/1596557034 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f15300acbc0 con 0x556307fb44d0 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f155063a740 1 -- 172.21.15.90:0/1596557034 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f153c036070 con 0x556307fb44d0 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f1523e00640 1 -- 172.21.15.90:0/1596557034 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f15300d00b0 con 0x556307fb44d0 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f155063a740 10 client.15549 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f155063a740 1 -- 172.21.15.90:0/1596557034 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55630840c7c0 con 0x556307fb44d0 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f1523e00640 1 -- 172.21.15.90:0/1596557034 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f153c036070 con 0x556307fb44d0 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f1523e00640 1 -- 172.21.15.90:0/1596557034 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f15300c2ba0 con 0x556307fb44d0 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f155063a740 10 client.15549 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f155063a740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f155063a740 20 client.15549 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f155063a740 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556307f691c0 0x55630840efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f155063a740 4 client.15549 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:47:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.271+0000 7f155063a740 1 -- 172.21.15.90:0/1596557034 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563082e42c0 con 0x556307f691c0 2024-05-27T07:47:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.272+0000 7f1542a00640 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556307f691c0 0x55630840efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.273+0000 7f1542a00640 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556307f691c0 0x55630840efd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2f2fa293042a34f server_cookie=2c308df81eafa250 in_seq=0 out_seq=0 2024-05-27T07:47:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.273+0000 7f1523e00640 10 client.15549 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f1523e00640 1 -- 172.21.15.90:0/1596557034 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5563082e42c0 con 0x556307f691c0 2024-05-27T07:47:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f1523e00640 10 client.15549 handle_command_reply: tid=0 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f155063a740 1 client.15549 shutdown 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f155063a740 1 -- 172.21.15.90:0/1596557034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fb44d0 msgr2=0x556308405650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f155063a740 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fb44d0 0x556308405650 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f153000a5f0 tx=0x7f15300adc60 comp rx=0 tx=0).stop 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f155063a740 1 -- 172.21.15.90:0/1596557034 shutdown_connections 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f155063a740 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556307f691c0 0x55630840efd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f155063a740 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556307fd51b0 0x5563083ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f155063a740 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556307fb44d0 0x556308405650 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f155063a740 1 --2- 172.21.15.90:0/1596557034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556307fac2f0 0x556308405110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f1523e00640 0 client.15549 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.638+0000 7f155063a740 1 -- 172.21.15.90:0/1596557034 >> 172.21.15.90:0/1596557034 conn(0x556307fc9300 msgr2=0x55630837c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.639+0000 7f155063a740 1 -- 172.21.15.90:0/1596557034 shutdown_connections 2024-05-27T07:47:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.639+0000 7f155063a740 1 -- 172.21.15.90:0/1596557034 wait complete. 2024-05-27T07:47:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.639+0000 7f155063a740 20 client.15549 trim_cache size 0 max 16384 2024-05-27T07:47:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:47:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:47:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:47:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 -- 172.21.15.90:0/710078334 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f15280b2b00 msgr2=0x7f15280b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f15280b2b00 0x7f15280b5010 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x556308378c10 tx=0x7f153400b710 comp rx=0 tx=0).stop 2024-05-27T07:47:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 -- 172.21.15.90:0/710078334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544109c90 msgr2=0x7f154419ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544109c90 0x7f154419ad00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f153000afa0 tx=0x7f153000cd50 comp rx=0 tx=0).stop 2024-05-27T07:47:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 -- 172.21.15.90:0/710078334 shutdown_connections 2024-05-27T07:47:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f15280b2b00 0x7f15280b5010 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f154410a130 0x7f154419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1544109c90 0x7f154419ad00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 --2- 172.21.15.90:0/710078334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15441043e0 0x7f154419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 -- 172.21.15.90:0/710078334 >> 172.21.15.90:0/710078334 conn(0x7f1544079390 msgr2=0x7f1544101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 -- 172.21.15.90:0/710078334 shutdown_connections 2024-05-27T07:47:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:29.640+0000 7f1549600640 1 -- 172.21.15.90:0/710078334 wait complete. 2024-05-27T07:47:29.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.715+0000 7f7feee00700 1 -- 172.21.15.90:0/2007727722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe8108730 msgr2=0x7f7fe8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.715+0000 7f7feee00700 1 --2- 172.21.15.90:0/2007727722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe8108730 0x7f7fe8110090 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000b0d0 tx=0x7f7fe000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.715+0000 7f7feee00700 1 -- 172.21.15.90:0/2007727722 shutdown_connections 2024-05-27T07:47:29.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.715+0000 7f7feee00700 1 --2- 172.21.15.90:0/2007727722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe8108730 0x7f7fe8110090 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.715+0000 7f7feee00700 1 --2- 172.21.15.90:0/2007727722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fe81034b0 0x7f7fe81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.715+0000 7f7feee00700 1 --2- 172.21.15.90:0/2007727722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe8107de0 0x7f7fe81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.715+0000 7f7feee00700 1 -- 172.21.15.90:0/2007727722 >> 172.21.15.90:0/2007727722 conn(0x7f7fe80fedb0 msgr2=0x7f7fe81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:29.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.716+0000 7f7feee00700 1 -- 172.21.15.90:0/2007727722 shutdown_connections 2024-05-27T07:47:29.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.716+0000 7f7feee00700 1 -- 172.21.15.90:0/2007727722 wait complete. 2024-05-27T07:47:29.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.717+0000 7f7feee00700 1 Processor -- start 2024-05-27T07:47:29.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.718+0000 7f7feee00700 1 -- start start 2024-05-27T07:47:29.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.718+0000 7f7feee00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe81034b0 0x7f7fe81a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.718+0000 7f7feee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe8107de0 0x7f7fe81a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.718+0000 7f7feee00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fe8108730 0x7f7fe819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.718+0000 7f7feee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe8113260 con 0x7f7fe8107de0 2024-05-27T07:47:29.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.718+0000 7f7feee00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81130e0 con 0x7f7fe8108730 2024-05-27T07:47:29.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.718+0000 7f7feee00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81133e0 con 0x7f7fe81034b0 2024-05-27T07:47:29.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.719+0000 7f7fe7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe81034b0 0x7f7fe81a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.719+0000 7f7fe7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe81034b0 0x7f7fe81a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39286/0 (socket says 172.21.15.90:39286) 2024-05-27T07:47:29.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.719+0000 7f7fe7e00700 1 -- 172.21.15.90:0/1611685320 learned_addr learned my addr 172.21.15.90:0/1611685320 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:29.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.719+0000 7f7fece00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fe8108730 0x7f7fe819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.719+0000 7f7fe7400700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe8107de0 0x7f7fe81a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.719+0000 7f7fe7e00700 1 -- 172.21.15.90:0/1611685320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fe8108730 msgr2=0x7f7fe819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.719+0000 7f7fe7e00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fe8108730 0x7f7fe819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.719+0000 7f7fe7e00700 1 -- 172.21.15.90:0/1611685320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe8107de0 msgr2=0x7f7fe81a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:29.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.719+0000 7f7fe7e00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe8107de0 0x7f7fe81a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:29.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.719+0000 7f7fe7e00700 1 -- 172.21.15.90:0/1611685320 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe819b280 con 0x7f7fe81034b0 2024-05-27T07:47:29.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.720+0000 7f7fe7e00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe81034b0 0x7f7fe81a09c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800e9a0 tx=0x7f7fd800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:29.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.720+0000 7f7fe4c00700 1 -- 172.21.15.90:0/1611685320 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800ca10 con 0x7f7fe81034b0 2024-05-27T07:47:29.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.720+0000 7f7fe4c00700 1 -- 172.21.15.90:0/1611685320 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7fd800cbb0 con 0x7f7fe81034b0 2024-05-27T07:47:29.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.720+0000 7f7fe4c00700 1 -- 172.21.15.90:0/1611685320 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8018680 con 0x7f7fe81034b0 2024-05-27T07:47:29.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.720+0000 7f7feee00700 1 -- 172.21.15.90:0/1611685320 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe819b570 con 0x7f7fe81034b0 2024-05-27T07:47:29.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.720+0000 7f7feee00700 1 -- 172.21.15.90:0/1611685320 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe81a7770 con 0x7f7fe81034b0 2024-05-27T07:47:29.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.721+0000 7f7feee00700 1 -- 172.21.15.90:0/1611685320 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe8104960 con 0x7f7fe81034b0 2024-05-27T07:47:29.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.724+0000 7f7fe4c00700 1 -- 172.21.15.90:0/1611685320 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f7fd8010430 con 0x7f7fe81034b0 2024-05-27T07:47:29.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.726+0000 7f7fe4c00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7fcc0b2ba0 0x7f7fcc0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:29.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.726+0000 7f7fe4c00700 1 -- 172.21.15.90:0/1611685320 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f7fd80e31d0 con 0x7f7fe81034b0 2024-05-27T07:47:29.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.726+0000 7f7fe7400700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7fcc0b2ba0 0x7f7fcc0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:29.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.727+0000 7f7fe7400700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7fcc0b2ba0 0x7f7fcc0b50b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7fe819c280 tx=0x7f7fd4009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:29.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:29.734+0000 7f7fe4c00700 1 -- 172.21.15.90:0/1611685320 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7fd80aa600 con 0x7f7fe81034b0 2024-05-27T07:47:30.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.040+0000 7f7feee00700 1 -- 172.21.15.90:0/1611685320 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fe819bff0 con 0x7f7fe81034b0 2024-05-27T07:47:30.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.042+0000 7f7fe4c00700 1 -- 172.21.15.90:0/1611685320 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11938 (secure 0 0 0) 0x7f7fd80cc020 con 0x7f7fe81034b0 2024-05-27T07:47:30.042 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:30.042 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:47:18.105261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25213":{"gid":25213,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/504128634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":504128634},{"type":"v1","addr":"172.21.15.136:6837","nonce":504128634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T07:47:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.045+0000 7f7feee00700 1 -- 172.21.15.90:0/1611685320 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7fcc0b2ba0 msgr2=0x7f7fcc0b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.045+0000 7f7feee00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7fcc0b2ba0 0x7f7fcc0b50b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7fe819c280 tx=0x7f7fd4009260 comp rx=0 tx=0).stop 2024-05-27T07:47:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.045+0000 7f7feee00700 1 -- 172.21.15.90:0/1611685320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe81034b0 msgr2=0x7f7fe81a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.045+0000 7f7feee00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe81034b0 0x7f7fe81a09c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800e9a0 tx=0x7f7fd800ecb0 comp rx=0 tx=0).stop 2024-05-27T07:47:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.045+0000 7f7feee00700 1 -- 172.21.15.90:0/1611685320 shutdown_connections 2024-05-27T07:47:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.046+0000 7f7feee00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7fcc0b2ba0 0x7f7fcc0b50b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.046+0000 7f7feee00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7fe8108730 0x7f7fe819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.046+0000 7f7feee00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe81034b0 0x7f7fe81a09c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.046+0000 7f7feee00700 1 --2- 172.21.15.90:0/1611685320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fe8107de0 0x7f7fe81a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.046+0000 7f7feee00700 1 -- 172.21.15.90:0/1611685320 >> 172.21.15.90:0/1611685320 conn(0x7f7fe80fedb0 msgr2=0x7f7fe806b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.046+0000 7f7feee00700 1 -- 172.21.15.90:0/1611685320 shutdown_connections 2024-05-27T07:47:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:30.046+0000 7f7feee00700 1 -- 172.21.15.90:0/1611685320 wait complete. 2024-05-27T07:47:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 32 2024-05-27T07:47:30.433 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:30 smithi090 ceph-mon[31429]: pgmap v35: 129 pgs: 129 active+clean; 55 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.0 MiB/s wr, 42 op/s 2024-05-27T07:47:30.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:30 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1611685320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:47:30.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:30 smithi136 ceph-mon[35324]: pgmap v35: 129 pgs: 129 active+clean; 55 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.0 MiB/s wr, 42 op/s 2024-05-27T07:47:30.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:30 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1611685320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:47:30.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:30 smithi129 ceph-mon[35254]: pgmap v35: 129 pgs: 129 active+clean; 55 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.0 MiB/s wr, 42 op/s 2024-05-27T07:47:30.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:30 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1611685320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:47:31.012 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:47:31.474 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:32.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:32 smithi090 ceph-mon[31429]: pgmap v36: 129 pgs: 129 active+clean; 55 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 962 KiB/s wr, 38 op/s 2024-05-27T07:47:32.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:32 smithi136 ceph-mon[35324]: pgmap v36: 129 pgs: 129 active+clean; 55 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 962 KiB/s wr, 38 op/s 2024-05-27T07:47:32.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:32 smithi129 ceph-mon[35254]: pgmap v36: 129 pgs: 129 active+clean; 55 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 962 KiB/s wr, 38 op/s 2024-05-27T07:47:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.046+0000 7fbaff200700 1 -- 172.21.15.90:0/617695764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb00105bd0 msgr2=0x7fbb0010e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.046+0000 7fbaff200700 1 --2- 172.21.15.90:0/617695764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb00105bd0 0x7fbb0010e120 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009990 tx=0x7fbaec009ca0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.047+0000 7fbaff200700 1 -- 172.21.15.90:0/617695764 shutdown_connections 2024-05-27T07:47:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.047+0000 7fbaff200700 1 --2- 172.21.15.90:0/617695764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb0010e6f0 0x7fbb00110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.047+0000 7fbaff200700 1 --2- 172.21.15.90:0/617695764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb00105280 0x7fbb00105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.047+0000 7fbaff200700 1 --2- 172.21.15.90:0/617695764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb00105bd0 0x7fbb0010e120 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.047+0000 7fbaff200700 1 -- 172.21.15.90:0/617695764 >> 172.21.15.90:0/617695764 conn(0x7fbb00078ab0 msgr2=0x7fbb00078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.047+0000 7fbaff200700 1 -- 172.21.15.90:0/617695764 shutdown_connections 2024-05-27T07:47:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.047+0000 7fbaff200700 1 -- 172.21.15.90:0/617695764 wait complete. 2024-05-27T07:47:34.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.048+0000 7fbaff200700 1 Processor -- start 2024-05-27T07:47:34.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.048+0000 7fbaff200700 1 -- start start 2024-05-27T07:47:34.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbaff200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb00105280 0x7fbb0019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbaff200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb00105bd0 0x7fbb0019d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbaff200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb0010e6f0 0x7fbb001a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbaff200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbb00114100 con 0x7fbb0010e6f0 2024-05-27T07:47:34.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbaff200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb00113f80 con 0x7fbb00105280 2024-05-27T07:47:34.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbaff200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb00114280 con 0x7fbb00105bd0 2024-05-27T07:47:34.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbafe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb0010e6f0 0x7fbb001a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbafe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb0010e6f0 0x7fbb001a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35690/0 (socket says 172.21.15.90:35690) 2024-05-27T07:47:34.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbafe800700 1 -- 172.21.15.90:0/519895653 learned_addr learned my addr 172.21.15.90:0/519895653 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:34.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbafde00700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb00105280 0x7fbb0019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.049+0000 7fbafd400700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb00105bd0 0x7fbb0019d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.050+0000 7fbafe800700 1 -- 172.21.15.90:0/519895653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb00105bd0 msgr2=0x7fbb0019d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.050+0000 7fbafe800700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb00105bd0 0x7fbb0019d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.050+0000 7fbafe800700 1 -- 172.21.15.90:0/519895653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb00105280 msgr2=0x7fbb0019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.050+0000 7fbafe800700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb00105280 0x7fbb0019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.050+0000 7fbafe800700 1 -- 172.21.15.90:0/519895653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb001a1b70 con 0x7fbb0010e6f0 2024-05-27T07:47:34.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.050+0000 7fbafd400700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb00105bd0 0x7fbb0019d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:34.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.050+0000 7fbafe800700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb0010e6f0 0x7fbb001a1580 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8007d70 tx=0x7fbaf800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.051+0000 7fbaf6a00700 1 -- 172.21.15.90:0/519895653 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8016020 con 0x7fbb0010e6f0 2024-05-27T07:47:34.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.051+0000 7fbaf6a00700 1 -- 172.21.15.90:0/519895653 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaf800ad40 con 0x7fbb0010e6f0 2024-05-27T07:47:34.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.051+0000 7fbaff200700 1 -- 172.21.15.90:0/519895653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb001a1e60 con 0x7fbb0010e6f0 2024-05-27T07:47:34.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.051+0000 7fbaff200700 1 -- 172.21.15.90:0/519895653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb001a23a0 con 0x7fbb0010e6f0 2024-05-27T07:47:34.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.051+0000 7fbaf6a00700 1 -- 172.21.15.90:0/519895653 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80115e0 con 0x7fbb0010e6f0 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.055+0000 7fbaf6a00700 1 -- 172.21.15.90:0/519895653 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7fbaf8019020 con 0x7fbb0010e6f0 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.056+0000 7fbaf6a00700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbae40b2c20 0x7fbae40b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.057+0000 7fbaf6a00700 1 -- 172.21.15.90:0/519895653 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fbaf80e2990 con 0x7fbb0010e6f0 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.057+0000 7fbb138c9b80 1 Processor -- start 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.057+0000 7fbafde00700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbae40b2c20 0x7fbae40b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.057+0000 7fbb138c9b80 1 -- start start 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.057+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0109f0 0x56232e0a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.057+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232df24e40 0x56232e0a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.057+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.058+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56232dfd1e40 con 0x56232e0ae2c0 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.058+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56232dfc19e0 con 0x56232e0109f0 2024-05-27T07:47:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.058+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56232dfc5a20 con 0x56232df24e40 2024-05-27T07:47:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.058+0000 7fbafde00700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbae40b2c20 0x7fbae40b5130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0005950 tx=0x7fbaf00058e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.058+0000 7fbafe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.058+0000 7fbafe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35698/0 (socket says 172.21.15.90:35698) 2024-05-27T07:47:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.058+0000 7fbafe800700 1 -- 172.21.15.90:0/318424552 learned_addr learned my addr 172.21.15.90:0/318424552 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.058+0000 7fbafde00700 1 --2- 172.21.15.90:0/318424552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0109f0 0x56232e0a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.058+0000 7fbafd400700 1 --2- 172.21.15.90:0/318424552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232df24e40 0x56232e0a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.059+0000 7fbafe800700 1 -- 172.21.15.90:0/318424552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232df24e40 msgr2=0x56232e0a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.059+0000 7fbafe800700 1 --2- 172.21.15.90:0/318424552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232df24e40 0x56232e0a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.059+0000 7fbafe800700 1 -- 172.21.15.90:0/318424552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0109f0 msgr2=0x56232e0a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.059+0000 7fbafe800700 1 --2- 172.21.15.90:0/318424552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0109f0 0x56232e0a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.059+0000 7fbafe800700 1 -- 172.21.15.90:0/318424552 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56232df8a420 con 0x56232e0ae2c0 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.059+0000 7fbafe800700 1 --2- 172.21.15.90:0/318424552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0b06e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80a94d0 tx=0x7fbaf8007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292295831681ee5c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.059+0000 7fbaeb400700 1 -- 172.21.15.90:0/318424552 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80a9ed0 con 0x56232e0ae2c0 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.059+0000 7fbaeb400700 1 -- 172.21.15.90:0/318424552 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaf80a97d0 con 0x56232e0ae2c0 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.059+0000 7fbaeb400700 1 -- 172.21.15.90:0/318424552 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80910d0 con 0x56232e0ae2c0 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.060+0000 7fbb138c9b80 1 -- 172.21.15.90:0/318424552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 msgr2=0x56232e0b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.060+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/318424552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0b06e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80a94d0 tx=0x7fbaf8007b20 comp rx=0 tx=0).stop 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.061+0000 7fbb138c9b80 1 -- 172.21.15.90:0/318424552 shutdown_connections 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.061+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/318424552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0b06e0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.061+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/318424552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0109f0 0x56232e0a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.061+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/318424552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232df24e40 0x56232e0a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.061+0000 7fbb138c9b80 1 -- 172.21.15.90:0/318424552 >> 172.21.15.90:0/318424552 conn(0x56232dfbe390 msgr2=0x56232e09f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.061+0000 7fbb138c9b80 1 -- 172.21.15.90:0/318424552 shutdown_connections 2024-05-27T07:47:34.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.061+0000 7fbb138c9b80 1 -- 172.21.15.90:0/318424552 wait complete. 2024-05-27T07:47:34.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.061+0000 7fbb138c9b80 1 -- start start 2024-05-27T07:47:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.062+0000 7fbb138c9b80 1 Processor -- start 2024-05-27T07:47:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.062+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232df24e40 0x56232e0ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.062+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0109f0 0x56232e0ab780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.062+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0ae2c0 0x56232e0abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.062+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56232dec7fa0 con 0x56232e0109f0 2024-05-27T07:47:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.062+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56232df62760 con 0x56232df24e40 2024-05-27T07:47:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.062+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56232e0b0e00 con 0x56232e0ae2c0 2024-05-27T07:47:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.062+0000 7fbb138c9b80 10 client.0 _ll_register_callbacks cb 0x56232df6bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0109f0 0x56232e0ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0109f0 0x56232e0ab780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35708/0 (socket says 172.21.15.90:35708) 2024-05-27T07:47:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafd400700 1 -- 172.21.15.90:0/3072496130 learned_addr learned my addr 172.21.15.90:0/3072496130 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafde00700 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232df24e40 0x56232e0ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafe800700 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0ae2c0 0x56232e0abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafd400700 1 -- 172.21.15.90:0/3072496130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0ae2c0 msgr2=0x56232e0abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafd400700 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0ae2c0 0x56232e0abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafd400700 1 -- 172.21.15.90:0/3072496130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232df24e40 msgr2=0x56232e0ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafd400700 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232df24e40 0x56232e0ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafd400700 1 -- 172.21.15.90:0/3072496130 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232df118d0 con 0x56232e0109f0 2024-05-27T07:47:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafde00700 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232df24e40 0x56232e0ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.063+0000 7fbafe800700 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0ae2c0 0x56232e0abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.064+0000 7fbafd400700 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0109f0 0x56232e0ab780 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009990 tx=0x7fbaec009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8610b6c550e3f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.064+0000 7fbadfe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.064+0000 7fbadfe00700 1 -- 172.21.15.90:0/3072496130 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec027060 con 0x56232e0109f0 2024-05-27T07:47:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.064+0000 7fbadfe00700 1 -- 172.21.15.90:0/3072496130 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fbaec005450 con 0x56232e0109f0 2024-05-27T07:47:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.064+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3072496130 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf80a97d0 con 0x56232e0109f0 2024-05-27T07:47:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.065+0000 7fbadfe00700 1 -- 172.21.15.90:0/3072496130 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbaec02e050 con 0x56232e0109f0 2024-05-27T07:47:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.065+0000 7fbb138c9b80 10 client.15567 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.065+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3072496130 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56232df92060 con 0x56232e0109f0 2024-05-27T07:47:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.065+0000 7fbadfe00700 1 -- 172.21.15.90:0/3072496130 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaf80a97d0 con 0x56232e0109f0 2024-05-27T07:47:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.065+0000 7fbadfe00700 1 -- 172.21.15.90:0/3072496130 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbaec021070 con 0x56232e0109f0 2024-05-27T07:47:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.065+0000 7fbb138c9b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.065+0000 7fbb138c9b80 10 client.15567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:47:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.065+0000 7fbb138c9b80 20 client.15567 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:34.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.065+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x56232e0c6540 0x56232e0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.066+0000 7fbb138c9b80 4 client.15567 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:34.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.066+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3072496130 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56232dfd3d90 con 0x56232e0c6540 2024-05-27T07:47:34.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.066+0000 7fbafde00700 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x56232e0c6540 0x56232e0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.067+0000 7fbafde00700 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x56232e0c6540 0x56232e0c8960 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b0a902750d1f5738 server_cookie=7c4c8dd41ab684ac in_seq=0 out_seq=0 2024-05-27T07:47:34.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.067+0000 7fbadfe00700 10 client.15567 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:47:34.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.069+0000 7fbadfe00700 1 -- 172.21.15.90:0/3072496130 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56232dfd3d90 con 0x56232e0c6540 2024-05-27T07:47:34.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.069+0000 7fbadfe00700 10 client.15567 handle_command_reply: tid=0 2024-05-27T07:47:34.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.069+0000 7fbb138c9b80 1 client.15567 shutdown 2024-05-27T07:47:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.069+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3072496130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0109f0 msgr2=0x56232e0ab780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.069+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0109f0 0x56232e0ab780 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009990 tx=0x7fbaec009f40 comp rx=0 tx=0).stop 2024-05-27T07:47:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.070+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3072496130 shutdown_connections 2024-05-27T07:47:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.070+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x56232e0c6540 0x56232e0c8960 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.070+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0ae2c0 0x56232e0abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.070+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232df24e40 0x56232e0ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.070+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/3072496130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0109f0 0x56232e0ab780 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.070+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3072496130 >> 172.21.15.90:0/3072496130 conn(0x56232dfbe390 msgr2=0x56232e09f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.070+0000 7fbadfe00700 0 client.15567 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:47:34.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.070+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3072496130 shutdown_connections 2024-05-27T07:47:34.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.070+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3072496130 wait complete. 2024-05-27T07:47:34.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.070+0000 7fbb138c9b80 20 client.15567 trim_cache size 0 max 16384 2024-05-27T07:47:34.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.088+0000 7fbb138c9b80 1 Processor -- start 2024-05-27T07:47:34.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.089+0000 7fbb138c9b80 1 -- start start 2024-05-27T07:47:34.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.089+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232df24e40 0x56232e0a6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.089+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0109f0 0x56232e0c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.089+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.089+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbae000ecc0 con 0x56232e0ae2c0 2024-05-27T07:47:34.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.089+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0010800 con 0x56232e0109f0 2024-05-27T07:47:34.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.089+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbae000f390 con 0x56232df24e40 2024-05-27T07:47:34.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.089+0000 7fbafe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.089+0000 7fbafe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35712/0 (socket says 172.21.15.90:35712) 2024-05-27T07:47:34.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.089+0000 7fbafe800700 1 -- 172.21.15.90:0/3689400029 learned_addr learned my addr 172.21.15.90:0/3689400029 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:34.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.090+0000 7fbafde00700 1 --2- 172.21.15.90:0/3689400029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232df24e40 0x56232e0a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.090+0000 7fbafd400700 1 --2- 172.21.15.90:0/3689400029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0109f0 0x56232e0c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.090+0000 7fbafe800700 1 -- 172.21.15.90:0/3689400029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232df24e40 msgr2=0x56232e0a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.090+0000 7fbafe800700 1 --2- 172.21.15.90:0/3689400029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232df24e40 0x56232e0a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.090+0000 7fbafe800700 1 -- 172.21.15.90:0/3689400029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0109f0 msgr2=0x56232e0c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.090+0000 7fbafe800700 1 --2- 172.21.15.90:0/3689400029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0109f0 0x56232e0c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.090+0000 7fbafe800700 1 -- 172.21.15.90:0/3689400029 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56232e09fc50 con 0x56232e0ae2c0 2024-05-27T07:47:34.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.090+0000 7fbafe800700 1 --2- 172.21.15.90:0/3689400029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0ce160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80b27f0 tx=0x7fbaf80a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9070fbbe917da045 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.091+0000 7fbaeaa00700 1 -- 172.21.15.90:0/3689400029 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80a9ed0 con 0x56232e0ae2c0 2024-05-27T07:47:34.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.091+0000 7fbaeaa00700 1 -- 172.21.15.90:0/3689400029 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaf80c3070 con 0x56232e0ae2c0 2024-05-27T07:47:34.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.091+0000 7fbaeaa00700 1 -- 172.21.15.90:0/3689400029 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80910d0 con 0x56232e0ae2c0 2024-05-27T07:47:34.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.092+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3689400029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 msgr2=0x56232e0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.092+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/3689400029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0ce160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80b27f0 tx=0x7fbaf80a57c0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.092+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3689400029 shutdown_connections 2024-05-27T07:47:34.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.092+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/3689400029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232e0ae2c0 0x56232e0ce160 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.092+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/3689400029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232df24e40 0x56232e0a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.092+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/3689400029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0109f0 0x56232e0c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.092+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3689400029 >> 172.21.15.90:0/3689400029 conn(0x56232dfbe390 msgr2=0x56232e0a6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.092+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3689400029 shutdown_connections 2024-05-27T07:47:34.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.092+0000 7fbb138c9b80 1 -- 172.21.15.90:0/3689400029 wait complete. 2024-05-27T07:47:34.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.093+0000 7fbb138c9b80 1 -- start start 2024-05-27T07:47:34.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.093+0000 7fbb138c9b80 1 Processor -- start 2024-05-27T07:47:34.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.093+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232df24e40 0x56232e0b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.093+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0109f0 0x56232e0ba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.093+0000 7fbb138c9b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0ae2c0 0x56232e12f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.093+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0010130 con 0x56232df24e40 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.093+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbae000fa60 con 0x56232e0ae2c0 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.093+0000 7fbb138c9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbae000cc40 con 0x56232e0109f0 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbb138c9b80 10 client.0 _ll_register_callbacks cb 0x56232df6bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbafde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232df24e40 0x56232e0b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbafde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232df24e40 0x56232e0b9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35716/0 (socket says 172.21.15.90:35716) 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbafde00700 1 -- 172.21.15.90:0/659817337 learned_addr learned my addr 172.21.15.90:0/659817337 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbafd400700 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0109f0 0x56232e0ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbafe800700 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0ae2c0 0x56232e12f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbafde00700 1 -- 172.21.15.90:0/659817337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0109f0 msgr2=0x56232e0ba450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbafde00700 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0109f0 0x56232e0ba450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbafde00700 1 -- 172.21.15.90:0/659817337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0ae2c0 msgr2=0x56232e12f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbafde00700 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0ae2c0 0x56232e12f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.094+0000 7fbafde00700 1 -- 172.21.15.90:0/659817337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232e0ac030 con 0x56232df24e40 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.095+0000 7fbafde00700 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232df24e40 0x56232e0b9f10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x56232e0a0a90 tx=0x7fbaf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a966f1eaa5fc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.095+0000 7fbadfe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.095+0000 7fbadfe00700 1 -- 172.21.15.90:0/659817337 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0029500 con 0x56232df24e40 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.095+0000 7fbadfe00700 1 -- 172.21.15.90:0/659817337 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fbaf00296a0 con 0x56232df24e40 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.095+0000 7fbb138c9b80 1 -- 172.21.15.90:0/659817337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf80c3070 con 0x56232df24e40 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.095+0000 7fbadfe00700 1 -- 172.21.15.90:0/659817337 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbaf003a050 con 0x56232df24e40 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.095+0000 7fbb138c9b80 10 client.15573 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:34.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.095+0000 7fbb138c9b80 1 -- 172.21.15.90:0/659817337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56232e130570 con 0x56232df24e40 2024-05-27T07:47:34.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.096+0000 7fbadfe00700 1 -- 172.21.15.90:0/659817337 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaf80c3070 con 0x56232df24e40 2024-05-27T07:47:34.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.096+0000 7fbadfe00700 1 -- 172.21.15.90:0/659817337 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fbaf00119b0 con 0x56232df24e40 2024-05-27T07:47:34.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.096+0000 7fbb138c9b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:34.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.096+0000 7fbb138c9b80 10 client.15573 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:47:34.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.096+0000 7fbb138c9b80 20 client.15573 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:34.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.096+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x56232e130c90 0x56232e1330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:34.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.096+0000 7fbb138c9b80 4 client.15573 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:47:34.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.096+0000 7fbb138c9b80 1 -- 172.21.15.90:0/659817337 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56232dfcd570 con 0x56232e130c90 2024-05-27T07:47:34.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.097+0000 7fbafd400700 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x56232e130c90 0x56232e1330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:34.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.098+0000 7fbadfe00700 10 client.15573 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:47:34.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.098+0000 7fbafd400700 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x56232e130c90 0x56232e1330b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=68f544f659318503 server_cookie=eecd76e62b3fd792 in_seq=0 out_seq=0 2024-05-27T07:47:34.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbadfe00700 1 -- 172.21.15.90:0/659817337 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56232dfcd570 con 0x56232e130c90 2024-05-27T07:47:34.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbadfe00700 10 client.15573 handle_command_reply: tid=0 2024-05-27T07:47:34.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 client.15573 shutdown 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 -- 172.21.15.90:0/659817337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232df24e40 msgr2=0x56232e0b9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232df24e40 0x56232e0b9f10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x56232e0a0a90 tx=0x7fbaf00047c0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 -- 172.21.15.90:0/659817337 shutdown_connections 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x56232e130c90 0x56232e1330b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56232e0ae2c0 0x56232e12f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbadfe00700 0 client.15573 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56232df24e40 0x56232e0b9f10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 --2- 172.21.15.90:0/659817337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56232e0109f0 0x56232e0ba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 -- 172.21.15.90:0/659817337 >> 172.21.15.90:0/659817337 conn(0x56232dfbe390 msgr2=0x56232e0a3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 -- 172.21.15.90:0/659817337 shutdown_connections 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 1 -- 172.21.15.90:0/659817337 wait complete. 2024-05-27T07:47:34.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.099+0000 7fbb138c9b80 20 client.15573 trim_cache size 0 max 16384 2024-05-27T07:47:34.102 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:47:34.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.100+0000 7fbaff200700 1 -- 172.21.15.90:0/519895653 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbae40b2c20 msgr2=0x7fbae40b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.100+0000 7fbaff200700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbae40b2c20 0x7fbae40b5130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0005950 tx=0x7fbaf00058e0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.100+0000 7fbaff200700 1 -- 172.21.15.90:0/519895653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb0010e6f0 msgr2=0x7fbb001a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:34.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.100+0000 7fbaff200700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb0010e6f0 0x7fbb001a1580 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8007d70 tx=0x7fbaf800a340 comp rx=0 tx=0).stop 2024-05-27T07:47:34.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.100+0000 7fbaff200700 1 -- 172.21.15.90:0/519895653 shutdown_connections 2024-05-27T07:47:34.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.100+0000 7fbaff200700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb0010e6f0 0x7fbb001a1580 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.100+0000 7fbaff200700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbae40b2c20 0x7fbae40b5130 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.100+0000 7fbaff200700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb00105280 0x7fbb0019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.100+0000 7fbaff200700 1 --2- 172.21.15.90:0/519895653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb00105bd0 0x7fbb0019d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:34.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.100+0000 7fbaff200700 1 -- 172.21.15.90:0/519895653 >> 172.21.15.90:0/519895653 conn(0x7fbb00078ab0 msgr2=0x7fbb000ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:34.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.101+0000 7fbaff200700 1 -- 172.21.15.90:0/519895653 shutdown_connections 2024-05-27T07:47:34.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:34.101+0000 7fbaff200700 1 -- 172.21.15.90:0/519895653 wait complete. 2024-05-27T07:47:34.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:34 smithi090 ceph-mon[31429]: pgmap v37: 129 pgs: 129 active+clean; 58 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-05-27T07:47:34.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:34 smithi136 ceph-mon[35324]: pgmap v37: 129 pgs: 129 active+clean; 58 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-05-27T07:47:34.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:34 smithi129 ceph-mon[35254]: pgmap v37: 129 pgs: 129 active+clean; 58 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-05-27T07:47:35.048 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:47:35.515 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:36.478 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:36 smithi090 ceph-mon[31429]: pgmap v38: 129 pgs: 129 active+clean; 54 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 594 KiB/s wr, 182 op/s 2024-05-27T07:47:36.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:36 smithi136 ceph-mon[35324]: pgmap v38: 129 pgs: 129 active+clean; 54 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 594 KiB/s wr, 182 op/s 2024-05-27T07:47:36.575 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:36 smithi129 ceph-mon[35254]: pgmap v38: 129 pgs: 129 active+clean; 54 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 594 KiB/s wr, 182 op/s 2024-05-27T07:47:37.169 INFO:teuthology.orchestra.run.smithi090.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-05-27T07:47:37.177 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-27T07:47:38.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:38 smithi090 ceph-mon[31429]: pgmap v39: 129 pgs: 129 active+clean; 54 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 395 KiB/s wr, 171 op/s 2024-05-27T07:47:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.304+0000 7f7a36200700 1 -- 172.21.15.90:0/1415665076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a30107de0 msgr2=0x7f7a301081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.304+0000 7f7a36200700 1 --2- 172.21.15.90:0/1415665076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a30107de0 0x7f7a301081f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0099c0 tx=0x7f7a1c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.304+0000 7f7a36200700 1 -- 172.21.15.90:0/1415665076 shutdown_connections 2024-05-27T07:47:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.304+0000 7f7a36200700 1 --2- 172.21.15.90:0/1415665076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a30108730 0x7f7a30110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.304+0000 7f7a36200700 1 --2- 172.21.15.90:0/1415665076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a301034b0 0x7f7a301038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.304+0000 7f7a36200700 1 --2- 172.21.15.90:0/1415665076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a30107de0 0x7f7a301081f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.304+0000 7f7a36200700 1 -- 172.21.15.90:0/1415665076 >> 172.21.15.90:0/1415665076 conn(0x7f7a300fedb0 msgr2=0x7f7a301011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.304+0000 7f7a36200700 1 -- 172.21.15.90:0/1415665076 shutdown_connections 2024-05-27T07:47:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.304+0000 7f7a36200700 1 -- 172.21.15.90:0/1415665076 wait complete. 2024-05-27T07:47:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.305+0000 7f7a36200700 1 Processor -- start 2024-05-27T07:47:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.306+0000 7f7a36200700 1 -- start start 2024-05-27T07:47:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.306+0000 7f7a36200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301034b0 0x7f7a3019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.306+0000 7f7a36200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a30107de0 0x7f7a3019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.306+0000 7f7a36200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a30108730 0x7f7a301a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.306+0000 7f7a36200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7a30113240 con 0x7f7a30108730 2024-05-27T07:47:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.306+0000 7f7a36200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301130c0 con 0x7f7a30107de0 2024-05-27T07:47:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.306+0000 7f7a36200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301133c0 con 0x7f7a301034b0 2024-05-27T07:47:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.306+0000 7f7a2fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a30108730 0x7f7a301a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.307+0000 7f7a2fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a30108730 0x7f7a301a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35734/0 (socket says 172.21.15.90:35734) 2024-05-27T07:47:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.307+0000 7f7a2fe00700 1 -- 172.21.15.90:0/2872483302 learned_addr learned my addr 172.21.15.90:0/2872483302 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.307+0000 7f7a2f400700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301034b0 0x7f7a3019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.307+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a30107de0 0x7f7a3019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.307+0000 7f7a2fe00700 1 -- 172.21.15.90:0/2872483302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301034b0 msgr2=0x7f7a3019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.307+0000 7f7a2fe00700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301034b0 0x7f7a3019cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.307+0000 7f7a2fe00700 1 -- 172.21.15.90:0/2872483302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a30107de0 msgr2=0x7f7a3019d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.307+0000 7f7a2fe00700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a30107de0 0x7f7a3019d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.307+0000 7f7a2fe00700 1 -- 172.21.15.90:0/2872483302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a301a1dd0 con 0x7f7a30108730 2024-05-27T07:47:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.308+0000 7f7a2fe00700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a30108730 0x7f7a301a1730 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7a28007ee0 tx=0x7f7a2800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.308+0000 7f7a27e00700 1 -- 172.21.15.90:0/2872483302 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28004bb0 con 0x7f7a30108730 2024-05-27T07:47:38.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.308+0000 7f7a27e00700 1 -- 172.21.15.90:0/2872483302 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a28004d50 con 0x7f7a30108730 2024-05-27T07:47:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.308+0000 7f7a36200700 1 -- 172.21.15.90:0/2872483302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a301a20c0 con 0x7f7a30108730 2024-05-27T07:47:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.308+0000 7f7a27e00700 1 -- 172.21.15.90:0/2872483302 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a280058b0 con 0x7f7a30108730 2024-05-27T07:47:38.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.308+0000 7f7a36200700 1 -- 172.21.15.90:0/2872483302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a301a99e0 con 0x7f7a30108730 2024-05-27T07:47:38.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.313+0000 7f7a27e00700 1 -- 172.21.15.90:0/2872483302 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f7a2801a020 con 0x7f7a30108730 2024-05-27T07:47:38.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.314+0000 7f7a44e27b80 1 Processor -- start 2024-05-27T07:47:38.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.314+0000 7f7a44e27b80 1 -- start start 2024-05-27T07:47:38.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.314+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 0x56549d7e5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.314+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d666e40 0x56549d7e5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7e5e70 0x56549d7f2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56549d713e40 con 0x56549d666e40 2024-05-27T07:47:38.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56549d7039e0 con 0x56549d7e5e70 2024-05-27T07:47:38.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56549d707a20 con 0x56549d7529f0 2024-05-27T07:47:38.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a27e00700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7a140b2ba0 0x7f7a140b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a2f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d666e40 0x56549d7e5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a2f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d666e40 0x56549d7e5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35740/0 (socket says 172.21.15.90:35740) 2024-05-27T07:47:38.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a2f400700 1 -- 172.21.15.90:0/3878892264 learned_addr learned my addr 172.21.15.90:0/3878892264 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:38.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a27e00700 1 -- 172.21.15.90:0/2872483302 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f7a280e1cb0 con 0x7f7a30108730 2024-05-27T07:47:38.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a2fe00700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7a140b2ba0 0x7f7a140b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.315+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/3878892264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 0x56549d7e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.316+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/3878892264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7e5e70 0x56549d7f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.316+0000 7f7a2f400700 1 -- 172.21.15.90:0/3878892264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 msgr2=0x56549d7e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:38.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.316+0000 7f7a2f400700 1 --2- 172.21.15.90:0/3878892264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 0x56549d7e5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.316+0000 7f7a2f400700 1 -- 172.21.15.90:0/3878892264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7e5e70 msgr2=0x56549d7f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:38.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.316+0000 7f7a2f400700 1 --2- 172.21.15.90:0/3878892264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7e5e70 0x56549d7f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.316+0000 7f7a2f400700 1 -- 172.21.15.90:0/3878892264 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549d6cc420 con 0x56549d666e40 2024-05-27T07:47:38.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.316+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/3878892264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 0x56549d7e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:38.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.317+0000 7f7a2f400700 1 --2- 172.21.15.90:0/3878892264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d666e40 0x56549d7e5870 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000bd70 tx=0x7f7a200106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3965760582e68cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:38.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.317+0000 7f7a2fe00700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7a140b2ba0 0x7f7a140b50b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a280deeb0 tx=0x7f7a28004ad0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:38.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.317+0000 7f7a24c00700 1 -- 172.21.15.90:0/3878892264 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20010e10 con 0x56549d666e40 2024-05-27T07:47:38.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.317+0000 7f7a24c00700 1 -- 172.21.15.90:0/3878892264 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a2001c910 con 0x56549d666e40 2024-05-27T07:47:38.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.317+0000 7f7a24c00700 1 -- 172.21.15.90:0/3878892264 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20011770 con 0x56549d666e40 2024-05-27T07:47:38.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.318+0000 7f7a44e27b80 1 -- 172.21.15.90:0/3878892264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d666e40 msgr2=0x56549d7e5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.318+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/3878892264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d666e40 0x56549d7e5870 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000bd70 tx=0x7f7a200106b0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.318+0000 7f7a44e27b80 1 -- 172.21.15.90:0/3878892264 shutdown_connections 2024-05-27T07:47:38.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.318+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/3878892264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7e5e70 0x56549d7f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.318+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/3878892264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 0x56549d7e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.318+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/3878892264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d666e40 0x56549d7e5870 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.318+0000 7f7a44e27b80 1 -- 172.21.15.90:0/3878892264 >> 172.21.15.90:0/3878892264 conn(0x56549d700390 msgr2=0x56549d7e14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.319+0000 7f7a44e27b80 1 -- 172.21.15.90:0/3878892264 shutdown_connections 2024-05-27T07:47:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.319+0000 7f7a44e27b80 1 -- 172.21.15.90:0/3878892264 wait complete. 2024-05-27T07:47:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.319+0000 7f7a44e27b80 1 -- start start 2024-05-27T07:47:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.319+0000 7f7a44e27b80 1 Processor -- start 2024-05-27T07:47:38.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.320+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d666e40 0x56549d7fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.320+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 0x56549d7fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.320+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d7f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.320+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56549d609fa0 con 0x56549d7e5e70 2024-05-27T07:47:38.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.320+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56549d6a4760 con 0x56549d666e40 2024-05-27T07:47:38.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.320+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56549d7f2b80 con 0x56549d7529f0 2024-05-27T07:47:38.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.320+0000 7f7a44e27b80 10 client.0 _ll_register_callbacks cb 0x56549d6adc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:38.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.321+0000 7f7a2ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d7f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.321+0000 7f7a2ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d7f86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35746/0 (socket says 172.21.15.90:35746) 2024-05-27T07:47:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.321+0000 7f7a2ea00700 1 -- 172.21.15.90:0/785785642 learned_addr learned my addr 172.21.15.90:0/785785642 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.321+0000 7f7a2f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 0x56549d7fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.321+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d666e40 0x56549d7fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.321+0000 7f7a2ea00700 1 -- 172.21.15.90:0/785785642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 msgr2=0x56549d7fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.321+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 0x56549d7fe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.321+0000 7f7a2ea00700 1 -- 172.21.15.90:0/785785642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d666e40 msgr2=0x56549d7fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.321+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d666e40 0x56549d7fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.321+0000 7f7a2ea00700 1 -- 172.21.15.90:0/785785642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549d6538d0 con 0x56549d7e5e70 2024-05-27T07:47:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.322+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d7f86b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c005850 tx=0x7f7a1c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baef9490a249f1db server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.322+0000 7f7a19600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.322+0000 7f7a19600700 1 -- 172.21.15.90:0/785785642 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0050d0 con 0x56549d7e5e70 2024-05-27T07:47:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.322+0000 7f7a19600700 1 -- 172.21.15.90:0/785785642 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f7a1c005270 con 0x56549d7e5e70 2024-05-27T07:47:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.322+0000 7f7a44e27b80 1 -- 172.21.15.90:0/785785642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a2001c910 con 0x56549d7e5e70 2024-05-27T07:47:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.322+0000 7f7a19600700 1 -- 172.21.15.90:0/785785642 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c005470 con 0x56549d7e5e70 2024-05-27T07:47:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.322+0000 7f7a44e27b80 10 client.15585 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.323+0000 7f7a44e27b80 1 -- 172.21.15.90:0/785785642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56549d6d4060 con 0x56549d7e5e70 2024-05-27T07:47:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.323+0000 7f7a19600700 1 -- 172.21.15.90:0/785785642 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a2001c910 con 0x56549d7e5e70 2024-05-27T07:47:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.323+0000 7f7a19600700 1 -- 172.21.15.90:0/785785642 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7a1c003a90 con 0x56549d7e5e70 2024-05-27T07:47:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.323+0000 7f7a44e27b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:38.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.323+0000 7f7a44e27b80 10 client.15585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:47:38.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.323+0000 7f7a44e27b80 20 client.15585 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:38.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.323+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56549d8082e0 0x56549d80a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.324+0000 7f7a44e27b80 4 client.15585 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:38.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.324+0000 7f7a44e27b80 1 -- 172.21.15.90:0/785785642 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56549d715d90 con 0x56549d8082e0 2024-05-27T07:47:38.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.324+0000 7f7a2f400700 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56549d8082e0 0x56549d80a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.325+0000 7f7a2f400700 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56549d8082e0 0x56549d80a700 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=220e19eb8f57a6ce server_cookie=59baae13f70bf06b in_seq=0 out_seq=0 2024-05-27T07:47:38.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.325+0000 7f7a19600700 10 client.15585 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:47:38.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.327+0000 7f7a19600700 1 -- 172.21.15.90:0/785785642 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56549d715d90 con 0x56549d8082e0 2024-05-27T07:47:38.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.327+0000 7f7a19600700 10 client.15585 handle_command_reply: tid=0 2024-05-27T07:47:38.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.327+0000 7f7a44e27b80 1 client.15585 shutdown 2024-05-27T07:47:38.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.328+0000 7f7a44e27b80 1 -- 172.21.15.90:0/785785642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 msgr2=0x56549d7f86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.328+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d7f86b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c005850 tx=0x7f7a1c009310 comp rx=0 tx=0).stop 2024-05-27T07:47:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.328+0000 7f7a44e27b80 1 -- 172.21.15.90:0/785785642 shutdown_connections 2024-05-27T07:47:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.328+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56549d8082e0 0x56549d80a700 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.328+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d7f86b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.328+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d666e40 0x56549d7fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.328+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/785785642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d7529f0 0x56549d7fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.328+0000 7f7a44e27b80 1 -- 172.21.15.90:0/785785642 >> 172.21.15.90:0/785785642 conn(0x56549d700390 msgr2=0x56549d7e5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:38.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.328+0000 7f7a19600700 0 client.15585 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:47:38.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.328+0000 7f7a44e27b80 1 -- 172.21.15.90:0/785785642 shutdown_connections 2024-05-27T07:47:38.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.329+0000 7f7a44e27b80 1 -- 172.21.15.90:0/785785642 wait complete. 2024-05-27T07:47:38.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.329+0000 7f7a44e27b80 20 client.15585 trim_cache size 0 max 16384 2024-05-27T07:47:38.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.347+0000 7f7a44e27b80 1 Processor -- start 2024-05-27T07:47:38.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.347+0000 7f7a44e27b80 1 -- start start 2024-05-27T07:47:38.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.347+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d666e40 0x56549d7f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.347+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7529f0 0x56549d7e0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.347+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7e5e70 0x56549d7e1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.347+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301ae140 con 0x56549d7529f0 2024-05-27T07:47:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.347+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301afc80 con 0x56549d7e5e70 2024-05-27T07:47:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.347+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301ae810 con 0x56549d666e40 2024-05-27T07:47:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7529f0 0x56549d7e0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7529f0 0x56549d7e0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35750/0 (socket says 172.21.15.90:35750) 2024-05-27T07:47:38.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2f400700 1 -- 172.21.15.90:0/3199502359 learned_addr learned my addr 172.21.15.90:0/3199502359 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/3199502359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d666e40 0x56549d7f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/3199502359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7e5e70 0x56549d7e1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2f400700 1 -- 172.21.15.90:0/3199502359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d666e40 msgr2=0x56549d7f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2f400700 1 --2- 172.21.15.90:0/3199502359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d666e40 0x56549d7f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2f400700 1 -- 172.21.15.90:0/3199502359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7e5e70 msgr2=0x56549d7e1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2f400700 1 --2- 172.21.15.90:0/3199502359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7e5e70 0x56549d7e1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2f400700 1 -- 172.21.15.90:0/3199502359 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549d7e0b10 con 0x56549d7529f0 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.348+0000 7f7a2f400700 1 --2- 172.21.15.90:0/3199502359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7529f0 0x56549d7e0f50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x56549d7e2a20 tx=0x7f7a2000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17c3d6dc15b7cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.349+0000 7f7a1be00700 1 -- 172.21.15.90:0/3199502359 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2002d070 con 0x56549d7529f0 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.349+0000 7f7a1be00700 1 -- 172.21.15.90:0/3199502359 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a20005310 con 0x56549d7529f0 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.349+0000 7f7a1be00700 1 -- 172.21.15.90:0/3199502359 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2001c480 con 0x56549d7529f0 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.350+0000 7f7a44e27b80 1 -- 172.21.15.90:0/3199502359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7529f0 msgr2=0x56549d7e0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.350+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/3199502359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7529f0 0x56549d7e0f50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x56549d7e2a20 tx=0x7f7a2000b410 comp rx=0 tx=0).stop 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.350+0000 7f7a44e27b80 1 -- 172.21.15.90:0/3199502359 shutdown_connections 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.350+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/3199502359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7e5e70 0x56549d7e1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.350+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/3199502359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d666e40 0x56549d7f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.350+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/3199502359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7529f0 0x56549d7e0f50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.350+0000 7f7a44e27b80 1 -- 172.21.15.90:0/3199502359 >> 172.21.15.90:0/3199502359 conn(0x56549d700390 msgr2=0x56549d7f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:38.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.350+0000 7f7a44e27b80 1 -- 172.21.15.90:0/3199502359 shutdown_connections 2024-05-27T07:47:38.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.350+0000 7f7a44e27b80 1 -- 172.21.15.90:0/3199502359 wait complete. 2024-05-27T07:47:38.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.351+0000 7f7a44e27b80 1 -- start start 2024-05-27T07:47:38.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.351+0000 7f7a44e27b80 1 Processor -- start 2024-05-27T07:47:38.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d666e40 0x56549d8016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7529f0 0x56549d801bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a44e27b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d870d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301af5b0 con 0x56549d7e5e70 2024-05-27T07:47:38.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301aeee0 con 0x56549d7529f0 2024-05-27T07:47:38.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a44e27b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301ac0f0 con 0x56549d666e40 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a44e27b80 10 client.0 _ll_register_callbacks cb 0x56549d6adc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a2ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d870d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a2ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d870d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35760/0 (socket says 172.21.15.90:35760) 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a2ea00700 1 -- 172.21.15.90:0/2796736537 learned_addr learned my addr 172.21.15.90:0/2796736537 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.352+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d666e40 0x56549d8016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.353+0000 7f7a2f400700 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7529f0 0x56549d801bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.353+0000 7f7a2ea00700 1 -- 172.21.15.90:0/2796736537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d666e40 msgr2=0x56549d8016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.353+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d666e40 0x56549d8016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.353+0000 7f7a2ea00700 1 -- 172.21.15.90:0/2796736537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7529f0 msgr2=0x56549d801bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.353+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7529f0 0x56549d801bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.353+0000 7f7a2ea00700 1 -- 172.21.15.90:0/2796736537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549d8024d0 con 0x56549d7e5e70 2024-05-27T07:47:38.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.353+0000 7f7a2ea00700 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d870d80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0345c0 tx=0x7f7a1c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf665da835871dba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.353+0000 7f7a19600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.354+0000 7f7a19600700 1 -- 172.21.15.90:0/2796736537 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c034a50 con 0x56549d7e5e70 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.354+0000 7f7a19600700 1 -- 172.21.15.90:0/2796736537 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f7a1c034bf0 con 0x56549d7e5e70 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.354+0000 7f7a44e27b80 1 -- 172.21.15.90:0/2796736537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a20005310 con 0x56549d7e5e70 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.354+0000 7f7a19600700 1 -- 172.21.15.90:0/2796736537 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7a1c049050 con 0x56549d7e5e70 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.354+0000 7f7a44e27b80 10 client.15591 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.354+0000 7f7a44e27b80 1 -- 172.21.15.90:0/2796736537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56549d872330 con 0x56549d7e5e70 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.354+0000 7f7a19600700 1 -- 172.21.15.90:0/2796736537 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a20005310 con 0x56549d7e5e70 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.355+0000 7f7a19600700 1 -- 172.21.15.90:0/2796736537 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f7a1c040070 con 0x56549d7e5e70 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.355+0000 7f7a44e27b80 10 client.15591 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.355+0000 7f7a44e27b80 10 client.15591 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:47:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.355+0000 7f7a44e27b80 20 client.15591 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.355+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56549d8728d0 0x56549d874cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.355+0000 7f7a44e27b80 4 client.15591 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.355+0000 7f7a44e27b80 1 -- 172.21.15.90:0/2796736537 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56549d70f570 con 0x56549d8728d0 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.356+0000 7f7a2f400700 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56549d8728d0 0x56549d874cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.357+0000 7f7a2f400700 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56549d8728d0 0x56549d874cf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c95e244761ddade server_cookie=b0b9bc2120f80701 in_seq=0 out_seq=0 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.357+0000 7f7a19600700 10 client.15591 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.357+0000 7f7a19600700 1 -- 172.21.15.90:0/2796736537 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56549d70f570 con 0x56549d8728d0 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.357+0000 7f7a19600700 10 client.15591 handle_command_reply: tid=0 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.357+0000 7f7a44e27b80 1 client.15591 shutdown 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.357+0000 7f7a44e27b80 1 -- 172.21.15.90:0/2796736537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 msgr2=0x56549d870d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.357+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d870d80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0345c0 tx=0x7f7a1c009310 comp rx=0 tx=0).stop 2024-05-27T07:47:38.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.357+0000 7f7a44e27b80 1 -- 172.21.15.90:0/2796736537 shutdown_connections 2024-05-27T07:47:38.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.357+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56549d8728d0 0x56549d874cf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.357+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56549d7e5e70 0x56549d870d80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.358+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56549d666e40 0x56549d8016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.358+0000 7f7a44e27b80 1 --2- 172.21.15.90:0/2796736537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56549d7529f0 0x56549d801bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.358+0000 7f7a19600700 0 client.15591 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:47:38.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.358+0000 7f7a44e27b80 1 -- 172.21.15.90:0/2796736537 >> 172.21.15.90:0/2796736537 conn(0x56549d700390 msgr2=0x56549d7e0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:38.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.358+0000 7f7a44e27b80 1 -- 172.21.15.90:0/2796736537 shutdown_connections 2024-05-27T07:47:38.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.358+0000 7f7a44e27b80 1 -- 172.21.15.90:0/2796736537 wait complete. 2024-05-27T07:47:38.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.358+0000 7f7a44e27b80 20 client.15591 trim_cache size 0 max 16384 2024-05-27T07:47:38.361 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:47:38.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 -- 172.21.15.90:0/2872483302 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7a140b2ba0 msgr2=0x7f7a140b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7a140b2ba0 0x7f7a140b50b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a280deeb0 tx=0x7f7a28004ad0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 -- 172.21.15.90:0/2872483302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a30108730 msgr2=0x7f7a301a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:38.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a30108730 0x7f7a301a1730 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7a28007ee0 tx=0x7f7a2800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 -- 172.21.15.90:0/2872483302 shutdown_connections 2024-05-27T07:47:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a30108730 0x7f7a301a1730 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a301034b0 0x7f7a3019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7a140b2ba0 0x7f7a140b50b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 --2- 172.21.15.90:0/2872483302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a30107de0 0x7f7a3019d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 -- 172.21.15.90:0/2872483302 >> 172.21.15.90:0/2872483302 conn(0x7f7a300fedb0 msgr2=0x7f7a3006b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 -- 172.21.15.90:0/2872483302 shutdown_connections 2024-05-27T07:47:38.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:38.359+0000 7f7a36200700 1 -- 172.21.15.90:0/2872483302 wait complete. 2024-05-27T07:47:38.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:38 smithi136 ceph-mon[35324]: pgmap v39: 129 pgs: 129 active+clean; 54 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 395 KiB/s wr, 171 op/s 2024-05-27T07:47:38.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:38 smithi129 ceph-mon[35254]: pgmap v39: 129 pgs: 129 active+clean; 54 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 371 KiB/s rd, 395 KiB/s wr, 171 op/s 2024-05-27T07:47:38.613 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/: 2024-05-27T07:47:38.613 INFO:teuthology.orchestra.run.smithi090.stdout:total 2 2024-05-27T07:47:38.613 INFO:teuthology.orchestra.run.smithi090.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:46 . 2024-05-27T07:47:38.613 INFO:teuthology.orchestra.run.smithi090.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:46 .. 2024-05-27T07:47:38.613 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 data 2024-05-27T07:47:38.613 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:38.613 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data: 2024-05-27T07:47:38.613 INFO:teuthology.orchestra.run.smithi090.stdout:total 45 2024-05-27T07:47:38.613 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 . 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:46 .. 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 27 07:47 base 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 27 07:47 global 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 log 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_commit_ts 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_dynshmem 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 27 07:46 pg_hba.conf 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 27 07:46 pg_ident.conf 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 27 07:47 pg_logical 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:46 pg_multixact 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_notify 2024-05-27T07:47:38.614 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_replslot 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_serial 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_snapshots 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_stat 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_stat_tmp 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 pg_subtrans 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_tblspc 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_twophase 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 27 07:46 PG_VERSION 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:46 pg_wal 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 pg_xact 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 27 07:46 postgresql.auto.conf 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 27 07:46 postgresql.conf 2024-05-27T07:47:38.615 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/base: 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:total 3 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 27 07:47 . 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 27 07:47 1 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 27 07:47 13493 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 27 07:47 13494 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:total 7778 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 27 07:47 . 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 27 07:47 .. 2024-05-27T07:47:38.616 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 112 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 113 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:47 1247 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1247_fsm 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1247_vm 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 27 07:47 1249 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1249_fsm 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1249_vm 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 27 07:47 1255 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1255_fsm 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1255_vm 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 27 07:47 1259 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1259_fsm 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1259_vm 2024-05-27T07:47:38.617 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 27 07:47 13344 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13344_fsm 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13344_vm 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13346 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13348 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13349_fsm 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349_vm 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13351 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13353 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13354_fsm 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354_vm 2024-05-27T07:47:38.618 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13356 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13358 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13359_fsm 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359_vm 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13361 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13363 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 1417 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 1418 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 174 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 175 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2187 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2224 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2328 2024-05-27T07:47:38.619 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2336 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2337 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2579 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2600 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2600_fsm 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2600_vm 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2601_fsm 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601_vm 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2602 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2602_fsm 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2602_vm 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2603 2024-05-27T07:47:38.620 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2603_fsm 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2603_vm 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2604 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2605 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2605_fsm 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2605_vm 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2606 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2606_fsm 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2606_vm 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2607 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2607_fsm 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2607_vm 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 27 07:47 2608 2024-05-27T07:47:38.621 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2608_fsm 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2608_vm 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 27 07:47 2609 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2609_fsm 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2609_vm 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2610 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2610_fsm 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2610_vm 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2611 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2612_fsm 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612_vm 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2613 2024-05-27T07:47:38.622 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615 2024-05-27T07:47:38.623 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2615_fsm 2024-05-27T07:47:38.623 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615_vm 2024-05-27T07:47:38.623 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2616 2024-05-27T07:47:38.623 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2616_fsm 2024-05-27T07:47:38.623 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2616_vm 2024-05-27T07:47:38.623 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 27 07:47 2617 2024-05-27T07:47:38.623 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2617_fsm 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2617_vm 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 27 07:47 2618 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2618_fsm 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2618_vm 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 27 07:47 2619 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2619_fsm 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2619_vm 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2620 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2650 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2651 2024-05-27T07:47:38.624 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2652 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2653 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2654 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2655 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2656 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2657 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 27 07:47 2658 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:47 2659 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2660 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2661 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2662 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2663 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2664 2024-05-27T07:47:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2665 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2666 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2667 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2668 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2669 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2670 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 27 07:47 2673 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 27 07:47 2674 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 27 07:47 2675 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2678 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2679 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2680 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2681 2024-05-27T07:47:38.626 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2682 2024-05-27T07:47:38.627 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2683 2024-05-27T07:47:38.627 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2684 2024-05-27T07:47:38.627 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2685 2024-05-27T07:47:38.627 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2686 2024-05-27T07:47:38.627 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2687 2024-05-27T07:47:38.627 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2688 2024-05-27T07:47:38.627 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2689 2024-05-27T07:47:38.627 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 27 07:47 2690 2024-05-27T07:47:38.627 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 27 07:47 2691 2024-05-27T07:47:38.628 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2692 2024-05-27T07:47:38.628 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2693 2024-05-27T07:47:38.628 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2696 2024-05-27T07:47:38.628 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2699 2024-05-27T07:47:38.628 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2701 2024-05-27T07:47:38.628 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2702 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2703 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2704 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2753 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2753_fsm 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2753_vm 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2754 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2755 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2756 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2757 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2830 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2831 2024-05-27T07:47:38.629 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2832 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2833 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2834 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2835 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2836 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2837 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 27 07:47 2838 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2838_fsm 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2838_vm 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2839 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840_fsm 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2840_vm 2024-05-27T07:47:38.630 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2841 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2995 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2996 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3079_fsm 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079_vm 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3080 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3081 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3085 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3118 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3119 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 27 07:47 3164 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3256 2024-05-27T07:47:38.631 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3257 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3258 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3350 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3351 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3379 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3380 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3381 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3394 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3394_fsm 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3394_vm 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3395 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3429 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3430 2024-05-27T07:47:38.632 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3431 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3433 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3439 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3440 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3455 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 27 07:47 3456 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3456_fsm 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3456_vm 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3466 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3467 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3468 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3501 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3502 2024-05-27T07:47:38.633 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3503 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3534 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3541_fsm 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541_vm 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3542 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3574 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3575 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3576 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3596 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3597 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3598 2024-05-27T07:47:38.634 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3599 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3600_fsm 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600_vm 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3601_fsm 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601_vm 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3602_fsm 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602_vm 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603_fsm 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3603_vm 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3604 2024-05-27T07:47:38.635 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3605 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3606 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3607 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3608 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 3609 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3712 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3764_fsm 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764_vm 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3766 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3767 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3997 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4143 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4144 2024-05-27T07:47:38.636 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4145 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4146 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4147 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4148 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4149 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4150 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4151 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4152 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4153 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4154 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4155 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4156 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4157 2024-05-27T07:47:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4158 2024-05-27T07:47:39.356 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:47:39.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:39 smithi136 ceph-mon[35324]: pgmap v40: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 616 KiB/s wr, 214 op/s 2024-05-27T07:47:39.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:39 smithi129 ceph-mon[35254]: pgmap v40: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 616 KiB/s wr, 214 op/s 2024-05-27T07:47:39.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:39 smithi090 ceph-mon[31429]: pgmap v40: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 616 KiB/s wr, 214 op/s 2024-05-27T07:47:39.829 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:39.888 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4159 2024-05-27T07:47:39.888 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4160 2024-05-27T07:47:39.888 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4161 2024-05-27T07:47:39.888 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4162 2024-05-27T07:47:39.888 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4163 2024-05-27T07:47:39.888 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4164 2024-05-27T07:47:39.888 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4165 2024-05-27T07:47:39.888 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4166 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4167 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4168 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4169 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4170 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4171 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4172 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4173 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4174 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 5002 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 548 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 549 2024-05-27T07:47:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 6102 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 6104 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 6106 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6110 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6111 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6112 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6113 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6117 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 826 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 827 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 828 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 27 07:47 pg_filenode.map 2024-05-27T07:47:39.890 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 27 07:47 PG_VERSION 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:total 7778 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 27 07:47 . 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 27 07:47 .. 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 112 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 113 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:47 1247 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1247_fsm 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1247_vm 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 27 07:47 1249 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1249_fsm 2024-05-27T07:47:39.891 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1249_vm 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 27 07:47 1255 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1255_fsm 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1255_vm 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 27 07:47 1259 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1259_fsm 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1259_vm 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 27 07:47 13344 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13344_fsm 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13344_vm 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13346 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13348 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349 2024-05-27T07:47:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13349_fsm 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349_vm 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13351 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13353 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13354_fsm 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354_vm 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13356 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13358 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13359_fsm 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359_vm 2024-05-27T07:47:39.893 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13361 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13363 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 1417 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 1418 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 174 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 175 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2187 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2224 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2328 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2336 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2337 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2579 2024-05-27T07:47:39.894 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2600 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2600_fsm 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2600_vm 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2601_fsm 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601_vm 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2602 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2602_fsm 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2602_vm 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2603 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2603_fsm 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2603_vm 2024-05-27T07:47:39.895 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2604 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2605 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2605_fsm 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2605_vm 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2606 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2606_fsm 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2606_vm 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2607 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2607_fsm 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2607_vm 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 27 07:47 2608 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2608_fsm 2024-05-27T07:47:39.896 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2608_vm 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 27 07:47 2609 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2609_fsm 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2609_vm 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2610 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2610_fsm 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2610_vm 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2611 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2612_fsm 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612_vm 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2613 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615 2024-05-27T07:47:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2615_fsm 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615_vm 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2616 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2616_fsm 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2616_vm 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 27 07:47 2617 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2617_fsm 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2617_vm 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 27 07:47 2618 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2618_fsm 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2618_vm 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 27 07:47 2619 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2619_fsm 2024-05-27T07:47:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2619_vm 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2620 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2650 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2651 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2652 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2653 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2654 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2655 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2656 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2657 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 27 07:47 2658 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:47 2659 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2660 2024-05-27T07:47:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2661 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2662 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2663 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2664 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2665 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2666 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2667 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2668 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2669 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2670 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 27 07:47 2673 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 27 07:47 2674 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 27 07:47 2675 2024-05-27T07:47:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2678 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2679 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2680 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2681 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2682 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2683 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2684 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2685 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2686 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2687 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2688 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2689 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 27 07:47 2690 2024-05-27T07:47:39.901 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 27 07:47 2691 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2692 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2693 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2696 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2699 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2701 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2702 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2703 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2704 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2753 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2753_fsm 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2753_vm 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2754 2024-05-27T07:47:39.902 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2755 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2756 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2757 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2830 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2831 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2832 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2833 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2834 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2835 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2836 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2837 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 27 07:47 2838 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2838_fsm 2024-05-27T07:47:39.903 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2838_vm 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2839 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840_fsm 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2840_vm 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2841 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2995 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2996 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3079_fsm 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079_vm 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3080 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3081 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3085 2024-05-27T07:47:39.904 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3118 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3119 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 27 07:47 3164 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3256 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3257 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3258 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3350 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3351 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3379 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3380 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3381 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3394 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3394_fsm 2024-05-27T07:47:39.905 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3394_vm 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3395 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3429 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3430 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3431 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3433 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3439 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3440 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3455 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 27 07:47 3456 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3456_fsm 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3456_vm 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3466 2024-05-27T07:47:39.906 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3467 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3468 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3501 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3502 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3503 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3534 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3541_fsm 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541_vm 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3542 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3574 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3575 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3576 2024-05-27T07:47:39.907 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3596 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3597 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3598 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3599 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3600_fsm 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600_vm 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3601_fsm 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601_vm 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3602_fsm 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602_vm 2024-05-27T07:47:39.908 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603_fsm 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3603_vm 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3604 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3605 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3606 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3607 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3608 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 3609 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3712 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3764_fsm 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764_vm 2024-05-27T07:47:39.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3766 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3767 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3997 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4143 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4144 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4145 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4146 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4147 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4148 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4149 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4150 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4151 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4152 2024-05-27T07:47:39.910 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4153 2024-05-27T07:47:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4154 2024-05-27T07:47:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4155 2024-05-27T07:47:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4156 2024-05-27T07:47:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4157 2024-05-27T07:47:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4158 2024-05-27T07:47:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4159 2024-05-27T07:47:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4160 2024-05-27T07:47:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4161 2024-05-27T07:47:39.911 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4162 2024-05-27T07:47:41.360 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4163 2024-05-27T07:47:41.360 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4164 2024-05-27T07:47:41.360 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4165 2024-05-27T07:47:41.360 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4166 2024-05-27T07:47:41.360 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4167 2024-05-27T07:47:41.360 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4168 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4169 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4170 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4171 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4172 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4173 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4174 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 5002 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 548 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 549 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6102 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6104 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6106 2024-05-27T07:47:41.361 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6110 2024-05-27T07:47:41.362 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6111 2024-05-27T07:47:41.362 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6112 2024-05-27T07:47:41.362 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6113 2024-05-27T07:47:41.362 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6117 2024-05-27T07:47:41.362 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 826 2024-05-27T07:47:41.362 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 827 2024-05-27T07:47:41.362 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 828 2024-05-27T07:47:41.363 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 27 07:47 pg_filenode.map 2024-05-27T07:47:41.363 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 27 07:47 PG_VERSION 2024-05-27T07:47:41.363 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.363 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-27T07:47:41.363 INFO:teuthology.orchestra.run.smithi090.stdout:total 7778 2024-05-27T07:47:41.363 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 27 07:47 . 2024-05-27T07:47:41.363 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 May 27 07:47 .. 2024-05-27T07:47:41.363 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 112 2024-05-27T07:47:41.363 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 113 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:47 1247 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1247_fsm 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1247_vm 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 27 07:47 1249 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1249_fsm 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1249_vm 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 27 07:47 1255 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1255_fsm 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1255_vm 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 27 07:47 1259 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1259_fsm 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1259_vm 2024-05-27T07:47:41.364 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 27 07:47 13344 2024-05-27T07:47:41.365 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13344_fsm 2024-05-27T07:47:41.365 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13344_vm 2024-05-27T07:47:41.365 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13346 2024-05-27T07:47:41.365 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13348 2024-05-27T07:47:41.365 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349 2024-05-27T07:47:41.365 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13349_fsm 2024-05-27T07:47:41.365 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349_vm 2024-05-27T07:47:41.365 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13351 2024-05-27T07:47:41.365 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13353 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13354_fsm 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354_vm 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13356 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13358 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13359_fsm 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359_vm 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13361 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13363 2024-05-27T07:47:41.366 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 1417 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 1418 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 174 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 175 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2187 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2224 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2328 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2336 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2337 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2579 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2600 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2600_fsm 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2600_vm 2024-05-27T07:47:41.367 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2601_fsm 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601_vm 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2602 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2602_fsm 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2602_vm 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2603 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2603_fsm 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2603_vm 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2604 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2605 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2605_fsm 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2605_vm 2024-05-27T07:47:41.368 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2606 2024-05-27T07:47:41.369 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2606_fsm 2024-05-27T07:47:41.369 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2606_vm 2024-05-27T07:47:41.369 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2607 2024-05-27T07:47:41.369 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2607_fsm 2024-05-27T07:47:41.369 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2607_vm 2024-05-27T07:47:41.369 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 27 07:47 2608 2024-05-27T07:47:41.369 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2608_fsm 2024-05-27T07:47:41.369 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2608_vm 2024-05-27T07:47:41.369 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 27 07:47 2609 2024-05-27T07:47:41.369 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2609_fsm 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2609_vm 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2610 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2610_fsm 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2610_vm 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2611 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2612_fsm 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612_vm 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2613 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615 2024-05-27T07:47:41.370 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2615_fsm 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615_vm 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2616 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2616_fsm 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2616_vm 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 27 07:47 2617 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2617_fsm 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2617_vm 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 27 07:47 2618 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2618_fsm 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2618_vm 2024-05-27T07:47:41.371 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 27 07:47 2619 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2619_fsm 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2619_vm 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2620 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2650 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2651 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2652 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2653 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2654 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2655 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2656 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2657 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 27 07:47 2658 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:47 2659 2024-05-27T07:47:41.372 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2660 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2661 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2662 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2663 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2664 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2665 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2666 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2667 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2668 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2669 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2670 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 27 07:47 2673 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 27 07:47 2674 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 27 07:47 2675 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2678 2024-05-27T07:47:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2679 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2680 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2681 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2682 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2683 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2684 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2685 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2686 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2687 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2688 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2689 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 27 07:47 2690 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 27 07:47 2691 2024-05-27T07:47:41.374 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2692 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2693 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2696 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2699 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2701 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2702 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2703 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2704 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2753 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2753_fsm 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2753_vm 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2754 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2755 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2756 2024-05-27T07:47:41.375 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2757 2024-05-27T07:47:41.376 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2830 2024-05-27T07:47:41.376 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2831 2024-05-27T07:47:41.376 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2832 2024-05-27T07:47:41.376 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2833 2024-05-27T07:47:41.376 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2834 2024-05-27T07:47:41.376 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2835 2024-05-27T07:47:41.377 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2836 2024-05-27T07:47:41.377 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2837 2024-05-27T07:47:41.377 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 27 07:47 2838 2024-05-27T07:47:41.377 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2838_fsm 2024-05-27T07:47:41.377 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2838_vm 2024-05-27T07:47:41.378 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2839 2024-05-27T07:47:41.378 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840 2024-05-27T07:47:41.378 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840_fsm 2024-05-27T07:47:41.378 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2840_vm 2024-05-27T07:47:41.378 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2841 2024-05-27T07:47:41.378 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2995 2024-05-27T07:47:41.378 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2996 2024-05-27T07:47:41.379 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079 2024-05-27T07:47:41.379 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3079_fsm 2024-05-27T07:47:41.379 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079_vm 2024-05-27T07:47:41.379 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3080 2024-05-27T07:47:41.379 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3081 2024-05-27T07:47:41.379 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3085 2024-05-27T07:47:41.379 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3118 2024-05-27T07:47:41.379 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3119 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 27 07:47 3164 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3256 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3257 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3258 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3350 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3351 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3379 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3380 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3381 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3394 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3394_fsm 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3394_vm 2024-05-27T07:47:41.380 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3395 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3429 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3430 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3431 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3433 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3439 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3440 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3455 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 27 07:47 3456 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3456_fsm 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3456_vm 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3466 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3467 2024-05-27T07:47:41.381 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3468 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3501 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3502 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3503 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3534 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3541_fsm 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541_vm 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3542 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3574 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3575 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3576 2024-05-27T07:47:41.382 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3596 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3597 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3598 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3599 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3600_fsm 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600_vm 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3601_fsm 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601_vm 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602 2024-05-27T07:47:41.383 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3602_fsm 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602_vm 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603_fsm 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3603_vm 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3604 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3605 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3606 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3607 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3608 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 3609 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3712 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3764_fsm 2024-05-27T07:47:41.384 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764_vm 2024-05-27T07:47:41.385 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3766 2024-05-27T07:47:41.385 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3767 2024-05-27T07:47:41.385 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3997 2024-05-27T07:47:41.385 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4143 2024-05-27T07:47:41.385 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4144 2024-05-27T07:47:41.385 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4145 2024-05-27T07:47:41.385 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4146 2024-05-27T07:47:41.386 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4147 2024-05-27T07:47:41.386 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4148 2024-05-27T07:47:41.386 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4149 2024-05-27T07:47:41.386 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4150 2024-05-27T07:47:41.386 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4151 2024-05-27T07:47:41.386 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4152 2024-05-27T07:47:41.387 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4153 2024-05-27T07:47:41.387 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4154 2024-05-27T07:47:41.387 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4155 2024-05-27T07:47:41.387 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4156 2024-05-27T07:47:41.387 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4157 2024-05-27T07:47:41.387 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4158 2024-05-27T07:47:41.387 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4159 2024-05-27T07:47:41.387 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4160 2024-05-27T07:47:41.387 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4161 2024-05-27T07:47:41.387 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4162 2024-05-27T07:47:41.388 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4163 2024-05-27T07:47:41.388 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4164 2024-05-27T07:47:41.388 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4165 2024-05-27T07:47:41.388 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4166 2024-05-27T07:47:41.388 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4167 2024-05-27T07:47:41.537 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4168 2024-05-27T07:47:41.537 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4169 2024-05-27T07:47:41.537 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4170 2024-05-27T07:47:41.537 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4171 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4172 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4173 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4174 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 5002 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 548 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 549 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6102 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6104 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6106 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6110 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6111 2024-05-27T07:47:41.538 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6112 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6113 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6117 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 826 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 827 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 828 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 27 07:47 pg_filenode.map 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 27 07:47 PG_VERSION 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/global: 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:total 538 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 May 27 07:47 . 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.539 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1213 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1213_fsm 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1213_vm 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1214 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1214_fsm 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1214_vm 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 1232 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 1233 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1260 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1260_fsm 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1260_vm 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1261 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1261_fsm 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1261_vm 2024-05-27T07:47:41.540 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1262 2024-05-27T07:47:41.541 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1262_fsm 2024-05-27T07:47:41.545 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1262_vm 2024-05-27T07:47:41.545 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2396 2024-05-27T07:47:41.545 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2396_fsm 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2396_vm 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2397 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2671 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2672 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2676 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2677 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2694 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2695 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2697 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2698 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2846 2024-05-27T07:47:41.546 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2847 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2964 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2965 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2966 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2967 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3592 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3593 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4060 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4061 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4175 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4176 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4177 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4178 2024-05-27T07:47:41.547 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4181 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4182 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4183 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4184 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4185 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4186 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 6000 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6001 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6002 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 6100 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6114 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6115 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 pg_control 2024-05-27T07:47:41.548 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 27 07:47 pg_filenode.map 2024-05-27T07:47:41.549 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.549 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/log: 2024-05-27T07:47:41.549 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.549 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 . 2024-05-27T07:47:41.816 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-27T07:47:41.817 INFO:teuthology.orchestra.run.smithi090.stdout:total 3 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 27 07:47 . 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 mappings 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 27 07:47 replorigin_checkpoint 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 snapshots 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 27 07:47 .. 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.818 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 27 07:47 .. 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout:total 2 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:46 . 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 members 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 offsets 2024-05-27T07:47:41.819 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.820 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-27T07:47:41.820 INFO:teuthology.orchestra.run.smithi090.stdout:total 9 2024-05-27T07:47:41.820 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 . 2024-05-27T07:47:41.820 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:46 .. 2024-05-27T07:47:41.820 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 0000 2024-05-27T07:47:41.820 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.820 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-27T07:47:41.820 INFO:teuthology.orchestra.run.smithi090.stdout:total 9 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 . 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:46 .. 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 0000 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-27T07:47:41.821 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.822 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.822 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.822 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.822 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-27T07:47:41.822 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.822 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.823 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.824 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-27T07:47:41.824 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.825 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.825 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.825 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.826 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-27T07:47:41.909 INFO:teuthology.orchestra.run.smithi090.stdout:total 9 2024-05-27T07:47:41.909 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 . 2024-05-27T07:47:41.909 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.909 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 0000 2024-05-27T07:47:41.909 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.909 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-27T07:47:41.909 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.909 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.909 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.910 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.910 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-27T07:47:41.910 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.910 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.910 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.910 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.910 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-27T07:47:41.910 INFO:teuthology.orchestra.run.smithi090.stdout:total 16386 2024-05-27T07:47:41.910 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:46 . 2024-05-27T07:47:41.911 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.911 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:47 000000010000000000000001 2024-05-27T07:47:41.911 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 archive_status 2024-05-27T07:47:41.911 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.911 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-27T07:47:41.911 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T07:47:41.912 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T07:47:41.912 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:46 .. 2024-05-27T07:47:41.912 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:47:41.912 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-27T07:47:41.912 INFO:teuthology.orchestra.run.smithi090.stdout:total 9 2024-05-27T07:47:41.913 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 . 2024-05-27T07:47:41.913 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283110 May 27 07:47 .. 2024-05-27T07:47:41.913 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 0000 2024-05-27T07:47:41.919 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-05-27T07:47:42.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:41 smithi090 ceph-mon[31429]: pgmap v41: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 555 KiB/s wr, 204 op/s 2024-05-27T07:47:42.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:41 smithi136 ceph-mon[35324]: pgmap v41: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 555 KiB/s wr, 204 op/s 2024-05-27T07:47:42.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:41 smithi129 ceph-mon[35254]: pgmap v41: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 555 KiB/s wr, 204 op/s 2024-05-27T07:47:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.864+0000 7fdcd8c00640 1 -- 172.21.15.90:0/1521811043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcd410bce0 msgr2=0x7fdcd4105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.864+0000 7fdcd8c00640 1 --2- 172.21.15.90:0/1521811043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcd410bce0 0x7fdcd4105880 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009900 tx=0x7fdcbc02a410 comp rx=0 tx=0).stop 2024-05-27T07:47:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.864+0000 7fdcd8c00640 1 -- 172.21.15.90:0/1521811043 shutdown_connections 2024-05-27T07:47:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.864+0000 7fdcd8c00640 1 --2- 172.21.15.90:0/1521811043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd410c180 0x7fdcd4105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.864+0000 7fdcd8c00640 1 --2- 172.21.15.90:0/1521811043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcd4106430 0x7fdcd4112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.864+0000 7fdcd8c00640 1 --2- 172.21.15.90:0/1521811043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcd410bce0 0x7fdcd4105880 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.864+0000 7fdcd8c00640 1 -- 172.21.15.90:0/1521811043 >> 172.21.15.90:0/1521811043 conn(0x7fdcd4101380 msgr2=0x7fdcd41037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.864+0000 7fdcd8c00640 1 -- 172.21.15.90:0/1521811043 shutdown_connections 2024-05-27T07:47:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.864+0000 7fdcd8c00640 1 -- 172.21.15.90:0/1521811043 wait complete. 2024-05-27T07:47:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.865+0000 7fdcd8c00640 1 Processor -- start 2024-05-27T07:47:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.866+0000 7fdcd8c00640 1 -- start start 2024-05-27T07:47:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.866+0000 7fdcd8c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcd4106430 0x7fdcd41a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.866+0000 7fdcd8c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcd410bce0 0x7fdcd41a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.866+0000 7fdcd8c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd410c180 0x7fdcd419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.866+0000 7fdcd8c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd4115d80 con 0x7fdcd410bce0 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.866+0000 7fdcd8c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd4115c00 con 0x7fdcd410c180 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.866+0000 7fdcd8c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd4115f00 con 0x7fdcd4106430 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.866+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcd410bce0 0x7fdcd41a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.866+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcd410bce0 0x7fdcd41a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34512/0 (socket says 172.21.15.90:34512) 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.866+0000 7fdcd2a00640 1 -- 172.21.15.90:0/961077803 learned_addr learned my addr 172.21.15.90:0/961077803 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.867+0000 7fdcd3400640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcd4106430 0x7fdcd41a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.867+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd410c180 0x7fdcd419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.867+0000 7fdcd2a00640 1 -- 172.21.15.90:0/961077803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcd4106430 msgr2=0x7fdcd41a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.867+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcd4106430 0x7fdcd41a24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.867+0000 7fdcd2a00640 1 -- 172.21.15.90:0/961077803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd410c180 msgr2=0x7fdcd419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.867+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd410c180 0x7fdcd419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.867+0000 7fdcd2a00640 1 -- 172.21.15.90:0/961077803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd419cd90 con 0x7fdcd410bce0 2024-05-27T07:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.867+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcd410bce0 0x7fdcd41a2a20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400e9f0 tx=0x7fdcc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.867+0000 7fdccbe00640 1 -- 172.21.15.90:0/961077803 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc400ccc0 con 0x7fdcd410bce0 2024-05-27T07:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.868+0000 7fdcd8c00640 1 -- 172.21.15.90:0/961077803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd419d020 con 0x7fdcd410bce0 2024-05-27T07:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.868+0000 7fdccbe00640 1 -- 172.21.15.90:0/961077803 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcc40044b0 con 0x7fdcd410bce0 2024-05-27T07:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.868+0000 7fdccbe00640 1 -- 172.21.15.90:0/961077803 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4010430 con 0x7fdcd410bce0 2024-05-27T07:47:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.868+0000 7fdcd8c00640 1 -- 172.21.15.90:0/961077803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd41a9280 con 0x7fdcd410bce0 2024-05-27T07:47:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.871+0000 7fdce11db740 1 Processor -- start 2024-05-27T07:47:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.871+0000 7fdce11db740 1 -- start start 2024-05-27T07:47:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.871+0000 7fdce11db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957231b0 0x560b95ac5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.871+0000 7fdce11db740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b956fa2f0 0x560b95ac5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.871+0000 7fdccbe00640 1 -- 172.21.15.90:0/961077803 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7fdcc4003f90 con 0x7fdcd410bce0 2024-05-27T07:47:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.871+0000 7fdce11db740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957024d0 0x560b95ac64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.871+0000 7fdce11db740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560b95a36b90 con 0x560b957024d0 2024-05-27T07:47:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.871+0000 7fdce11db740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560b95a26730 con 0x560b956fa2f0 2024-05-27T07:47:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.871+0000 7fdce11db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b95a2a770 con 0x560b957231b0 2024-05-27T07:47:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.872+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957024d0 0x560b95ac64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.872+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957024d0 0x560b95ac64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34522/0 (socket says 172.21.15.90:34522) 2024-05-27T07:47:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.872+0000 7fdcd2a00640 1 -- 172.21.15.90:0/725286504 learned_addr learned my addr 172.21.15.90:0/725286504 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.872+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/725286504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957231b0 0x560b95ac5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.872+0000 7fdcd3400640 1 --2- 172.21.15.90:0/725286504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b956fa2f0 0x560b95ac5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.872+0000 7fdcd2a00640 1 -- 172.21.15.90:0/725286504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957231b0 msgr2=0x560b95ac5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.872+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/725286504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957231b0 0x560b95ac5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.872+0000 7fdcd2a00640 1 -- 172.21.15.90:0/725286504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b956fa2f0 msgr2=0x560b95ac5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.872+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/725286504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b956fa2f0 0x560b95ac5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.872+0000 7fdcd2a00640 1 -- 172.21.15.90:0/725286504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b959bf8b0 con 0x560b957024d0 2024-05-27T07:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.873+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/725286504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957024d0 0x560b95ac64e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40e00f0 tx=0x7fdcc40eb4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf17ca7469b416d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.873+0000 7fdcc3e00640 1 -- 172.21.15.90:0/725286504 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40f6070 con 0x560b957024d0 2024-05-27T07:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.873+0000 7fdcc3e00640 1 -- 172.21.15.90:0/725286504 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcc40f1a90 con 0x560b957024d0 2024-05-27T07:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.873+0000 7fdccbe00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcb80b2b00 0x7fdcb80b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.873+0000 7fdcc3e00640 1 -- 172.21.15.90:0/725286504 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40e39a0 con 0x560b957024d0 2024-05-27T07:47:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.873+0000 7fdccbe00640 1 -- 172.21.15.90:0/961077803 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fdcc40e2fe0 con 0x7fdcd410bce0 2024-05-27T07:47:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.873+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcb80b2b00 0x7fdcb80b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.874+0000 7fdce11db740 1 -- 172.21.15.90:0/725286504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957024d0 msgr2=0x560b95ac64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.874+0000 7fdce11db740 1 --2- 172.21.15.90:0/725286504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957024d0 0x560b95ac64e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdcc40e00f0 tx=0x7fdcc40eb4b0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.874+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcb80b2b00 0x7fdcb80b5010 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x560b95ac2bd0 tx=0x7fdccc009500 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.874+0000 7fdce11db740 1 -- 172.21.15.90:0/725286504 shutdown_connections 2024-05-27T07:47:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.874+0000 7fdce11db740 1 --2- 172.21.15.90:0/725286504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b956fa2f0 0x560b95ac5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.874+0000 7fdce11db740 1 --2- 172.21.15.90:0/725286504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957024d0 0x560b95ac64e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.874+0000 7fdce11db740 1 --2- 172.21.15.90:0/725286504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957231b0 0x560b95ac5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.874+0000 7fdce11db740 1 -- 172.21.15.90:0/725286504 >> 172.21.15.90:0/725286504 conn(0x560b95717300 msgr2=0x560b95ac2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.874+0000 7fdce11db740 1 -- 172.21.15.90:0/725286504 shutdown_connections 2024-05-27T07:47:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.874+0000 7fdce11db740 1 -- 172.21.15.90:0/725286504 wait complete. 2024-05-27T07:47:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.875+0000 7fdce11db740 1 -- start start 2024-05-27T07:47:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.875+0000 7fdce11db740 1 Processor -- start 2024-05-27T07:47:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdce11db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b956fa2f0 0x560b95acfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdce11db740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 0x560b95ad0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdce11db740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95adbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdce11db740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560b9570c790 con 0x560b957231b0 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdce11db740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560b959cebc0 con 0x560b957024d0 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdce11db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b95ad2b90 con 0x560b956fa2f0 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdce11db740 10 client.0 _ll_register_callbacks cb 0x560b959ac400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdcd3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95adbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdcd3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95adbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34536/0 (socket says 172.21.15.90:34536) 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdcd3e00640 1 -- 172.21.15.90:0/3829014742 learned_addr learned my addr 172.21.15.90:0/3829014742 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdcd3400640 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b956fa2f0 0x560b95acfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 0x560b95ad0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdcd3e00640 1 -- 172.21.15.90:0/3829014742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b956fa2f0 msgr2=0x560b95acfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b956fa2f0 0x560b95acfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdcd3e00640 1 -- 172.21.15.90:0/3829014742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 msgr2=0x560b95ad0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 0x560b95ad0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.876+0000 7fdcd3e00640 1 -- 172.21.15.90:0/3829014742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b9573b210 con 0x560b957231b0 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.877+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95adbec0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009e90 tx=0x7fdccc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb6bce4ed7fa1a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.877+0000 7fdcc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.877+0000 7fdcc0c00640 1 -- 172.21.15.90:0/3829014742 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc010df0 con 0x560b957231b0 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.877+0000 7fdcc0c00640 1 -- 172.21.15.90:0/3829014742 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fdccc01b460 con 0x560b957231b0 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.877+0000 7fdce11db740 1 -- 172.21.15.90:0/3829014742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc40f1a90 con 0x560b957231b0 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.877+0000 7fdcc0c00640 1 -- 172.21.15.90:0/3829014742 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdccc01b660 con 0x560b957231b0 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.877+0000 7fdce11db740 10 client.15603 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.877+0000 7fdce11db740 1 -- 172.21.15.90:0/3829014742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560b95807620 con 0x560b957231b0 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.878+0000 7fdcc0c00640 1 -- 172.21.15.90:0/3829014742 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcc40f1a90 con 0x560b957231b0 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.878+0000 7fdcc0c00640 1 -- 172.21.15.90:0/3829014742 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdccc024070 con 0x560b957231b0 2024-05-27T07:47:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.878+0000 7fdce11db740 10 client.15603 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.878+0000 7fdce11db740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.878+0000 7fdce11db740 20 client.15603 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.878+0000 7fdce11db740 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b956e1d80 0x560b95ae1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.878+0000 7fdce11db740 4 client.15603 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.878+0000 7fdce11db740 1 -- 172.21.15.90:0/3829014742 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b95a38ae0 con 0x560b956e1d80 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.879+0000 7fdcd3400640 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b956e1d80 0x560b95ae1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.880+0000 7fdcd3400640 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b956e1d80 0x560b95ae1fa0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c301affcbb8e6d01 server_cookie=92ee779e80271193 in_seq=0 out_seq=0 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.880+0000 7fdcc0c00640 10 client.15603 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdcc0c00640 1 -- 172.21.15.90:0/3829014742 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560b95a38ae0 con 0x560b956e1d80 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdcc0c00640 10 client.15603 handle_command_reply: tid=0 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 client.15603 shutdown 2024-05-27T07:47:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 -- 172.21.15.90:0/3829014742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 msgr2=0x560b95adbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95adbec0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009e90 tx=0x7fdccc004a40 comp rx=0 tx=0).stop 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 -- 172.21.15.90:0/3829014742 shutdown_connections 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b956e1d80 0x560b95ae1fa0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 0x560b95ad0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95adbec0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 --2- 172.21.15.90:0/3829014742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b956fa2f0 0x560b95acfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdcc0c00640 0 client.15603 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 -- 172.21.15.90:0/3829014742 >> 172.21.15.90:0/3829014742 conn(0x560b95717300 msgr2=0x560b95ac7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 -- 172.21.15.90:0/3829014742 shutdown_connections 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 1 -- 172.21.15.90:0/3829014742 wait complete. 2024-05-27T07:47:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.882+0000 7fdce11db740 20 client.15603 trim_cache size 0 max 16384 2024-05-27T07:47:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.899+0000 7fdce11db740 1 Processor -- start 2024-05-27T07:47:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.899+0000 7fdce11db740 1 -- start start 2024-05-27T07:47:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.899+0000 7fdce11db740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b956fa2f0 0x560b95acb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.900+0000 7fdce11db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957024d0 0x560b95ac6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.900+0000 7fdce11db740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95ad1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.900+0000 7fdce11db740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b3280 con 0x560b957231b0 2024-05-27T07:47:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.900+0000 7fdce11db740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b4dc0 con 0x560b956fa2f0 2024-05-27T07:47:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.900+0000 7fdce11db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b3950 con 0x560b957024d0 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.900+0000 7fdcd3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95ad1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.900+0000 7fdcd3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95ad1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34544/0 (socket says 172.21.15.90:34544) 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.900+0000 7fdcd3e00640 1 -- 172.21.15.90:0/1880907410 learned_addr learned my addr 172.21.15.90:0/1880907410 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.900+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/1880907410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957024d0 0x560b95ac6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.900+0000 7fdcd3400640 1 --2- 172.21.15.90:0/1880907410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b956fa2f0 0x560b95acb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.901+0000 7fdcd3e00640 1 -- 172.21.15.90:0/1880907410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957024d0 msgr2=0x560b95ac6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.901+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/1880907410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957024d0 0x560b95ac6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.901+0000 7fdcd3e00640 1 -- 172.21.15.90:0/1880907410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b956fa2f0 msgr2=0x560b95acb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.901+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/1880907410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b956fa2f0 0x560b95acb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.901+0000 7fdcd3e00640 1 -- 172.21.15.90:0/1880907410 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b95ac0610 con 0x560b957231b0 2024-05-27T07:47:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.901+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/1880907410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957024d0 0x560b95ac6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.901+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/1880907410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95ad1af0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdccc004010 tx=0x7fdccc004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965086f9285b165a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.901+0000 7fdcc3400640 1 -- 172.21.15.90:0/1880907410 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc01cdc0 con 0x560b957231b0 2024-05-27T07:47:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.901+0000 7fdcc3400640 1 -- 172.21.15.90:0/1880907410 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdccc012e50 con 0x560b957231b0 2024-05-27T07:47:42.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.901+0000 7fdcc3400640 1 -- 172.21.15.90:0/1880907410 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc01b800 con 0x560b957231b0 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.902+0000 7fdce11db740 1 -- 172.21.15.90:0/1880907410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 msgr2=0x560b95ad1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.902+0000 7fdce11db740 1 --2- 172.21.15.90:0/1880907410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95ad1af0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdccc004010 tx=0x7fdccc004980 comp rx=0 tx=0).stop 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.902+0000 7fdce11db740 1 -- 172.21.15.90:0/1880907410 shutdown_connections 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.902+0000 7fdce11db740 1 --2- 172.21.15.90:0/1880907410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b957231b0 0x560b95ad1af0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.902+0000 7fdce11db740 1 --2- 172.21.15.90:0/1880907410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957024d0 0x560b95ac6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.902+0000 7fdce11db740 1 --2- 172.21.15.90:0/1880907410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b956fa2f0 0x560b95acb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.902+0000 7fdce11db740 1 -- 172.21.15.90:0/1880907410 >> 172.21.15.90:0/1880907410 conn(0x560b95717300 msgr2=0x560b95ad13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.902+0000 7fdce11db740 1 -- 172.21.15.90:0/1880907410 shutdown_connections 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.902+0000 7fdce11db740 1 -- 172.21.15.90:0/1880907410 wait complete. 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.903+0000 7fdce11db740 1 -- start start 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.903+0000 7fdce11db740 1 Processor -- start 2024-05-27T07:47:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdce11db740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b956fa2f0 0x560b95ad8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdce11db740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 0x560b95ad37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdce11db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957231b0 0x560b95ad3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdce11db740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b46f0 con 0x560b956fa2f0 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdce11db740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b4020 con 0x560b957024d0 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdce11db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd41b1240 con 0x560b957231b0 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdce11db740 10 client.0 _ll_register_callbacks cb 0x7fdccc009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdcd3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b956fa2f0 0x560b95ad8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdcd3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b956fa2f0 0x560b95ad8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34548/0 (socket says 172.21.15.90:34548) 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdcd3400640 1 -- 172.21.15.90:0/2138941484 learned_addr learned my addr 172.21.15.90:0/2138941484 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdcd3e00640 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957231b0 0x560b95ad3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 0x560b95ad37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdcd3400640 1 -- 172.21.15.90:0/2138941484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957231b0 msgr2=0x560b95ad3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdcd3400640 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957231b0 0x560b95ad3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdcd3400640 1 -- 172.21.15.90:0/2138941484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 msgr2=0x560b95ad37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.904+0000 7fdcd3400640 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 0x560b95ad37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.905+0000 7fdcd3400640 1 -- 172.21.15.90:0/2138941484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b95ac57f0 con 0x560b956fa2f0 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.905+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 0x560b95ad37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.905+0000 7fdcd3400640 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b956fa2f0 0x560b95ad8470 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc000c00 tx=0x7fdcbc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fd68c69be14aeb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.905+0000 7fdcc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.905+0000 7fdcc0c00640 1 -- 172.21.15.90:0/2138941484 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc035070 con 0x560b956fa2f0 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.905+0000 7fdcc0c00640 1 -- 172.21.15.90:0/2138941484 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fdcbc005040 con 0x560b956fa2f0 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.905+0000 7fdce11db740 1 -- 172.21.15.90:0/2138941484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdccc012e50 con 0x560b956fa2f0 2024-05-27T07:47:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.905+0000 7fdcc0c00640 1 -- 172.21.15.90:0/2138941484 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc051050 con 0x560b956fa2f0 2024-05-27T07:47:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.905+0000 7fdce11db740 10 client.15609 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.905+0000 7fdce11db740 1 -- 172.21.15.90:0/2138941484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560b95b4a350 con 0x560b956fa2f0 2024-05-27T07:47:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.906+0000 7fdcc0c00640 1 -- 172.21.15.90:0/2138941484 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdccc012e50 con 0x560b956fa2f0 2024-05-27T07:47:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.906+0000 7fdcc0c00640 1 -- 172.21.15.90:0/2138941484 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdcbc004c60 con 0x560b956fa2f0 2024-05-27T07:47:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.906+0000 7fdce11db740 10 client.15609 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.906+0000 7fdce11db740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.906+0000 7fdce11db740 20 client.15609 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.906+0000 7fdce11db740 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b956b71c0 0x560b95b4cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.906+0000 7fdce11db740 4 client.15609 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:47:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.906+0000 7fdce11db740 1 -- 172.21.15.90:0/2138941484 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b95a322c0 con 0x560b956b71c0 2024-05-27T07:47:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.907+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b956b71c0 0x560b95b4cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.908+0000 7fdcd2a00640 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b956b71c0 0x560b95b4cb40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd790e03772d4b2d server_cookie=7857f16149f26e5f in_seq=0 out_seq=0 2024-05-27T07:47:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:42.908+0000 7fdcc0c00640 10 client.15609 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdcc0c00640 1 -- 172.21.15.90:0/2138941484 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560b95a322c0 con 0x560b956b71c0 2024-05-27T07:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdcc0c00640 10 client.15609 handle_command_reply: tid=0 2024-05-27T07:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdce11db740 1 client.15609 shutdown 2024-05-27T07:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdce11db740 1 -- 172.21.15.90:0/2138941484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b956fa2f0 msgr2=0x560b95ad8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdce11db740 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b956fa2f0 0x560b95ad8470 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc000c00 tx=0x7fdcbc005a20 comp rx=0 tx=0).stop 2024-05-27T07:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdce11db740 1 -- 172.21.15.90:0/2138941484 shutdown_connections 2024-05-27T07:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdce11db740 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b956b71c0 0x560b95b4cb40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdce11db740 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b957024d0 0x560b95ad37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdce11db740 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b957231b0 0x560b95ad3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdce11db740 1 --2- 172.21.15.90:0/2138941484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b956fa2f0 0x560b95ad8470 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdcc0c00640 0 client.15609 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.003+0000 7fdce11db740 1 -- 172.21.15.90:0/2138941484 >> 172.21.15.90:0/2138941484 conn(0x560b95717300 msgr2=0x560b95ac6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.004+0000 7fdce11db740 1 -- 172.21.15.90:0/2138941484 shutdown_connections 2024-05-27T07:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.004+0000 7fdce11db740 1 -- 172.21.15.90:0/2138941484 wait complete. 2024-05-27T07:47:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.004+0000 7fdce11db740 20 client.15609 trim_cache size 0 max 16384 2024-05-27T07:47:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:47:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:47:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:47:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 -- 172.21.15.90:0/961077803 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcb80b2b00 msgr2=0x7fdcb80b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcb80b2b00 0x7fdcb80b5010 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x560b95ac2bd0 tx=0x7fdccc009500 comp rx=0 tx=0).stop 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 -- 172.21.15.90:0/961077803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcd410bce0 msgr2=0x7fdcd41a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcd410bce0 0x7fdcd41a2a20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400e9f0 tx=0x7fdcc400eeb0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 -- 172.21.15.90:0/961077803 shutdown_connections 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcb80b2b00 0x7fdcb80b5010 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcd410c180 0x7fdcd419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcd410bce0 0x7fdcd41a2a20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 --2- 172.21.15.90:0/961077803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcd4106430 0x7fdcd41a24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 -- 172.21.15.90:0/961077803 >> 172.21.15.90:0/961077803 conn(0x7fdcd4101380 msgr2=0x7fdcd4101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 -- 172.21.15.90:0/961077803 shutdown_connections 2024-05-27T07:47:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:43.005+0000 7fdcd8c00640 1 -- 172.21.15.90:0/961077803 wait complete. 2024-05-27T07:47:43.420 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-05-27T07:47:43.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.506+0000 7f56abe00700 1 -- 172.21.15.90:0/985505970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a4105e20 msgr2=0x7f56a410e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.506+0000 7f56abe00700 1 --2- 172.21.15.90:0/985505970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a4105e20 0x7f56a410e370 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5690009990 tx=0x7f5690009ca0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.506+0000 7f56abe00700 1 -- 172.21.15.90:0/985505970 shutdown_connections 2024-05-27T07:47:43.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.506+0000 7f56abe00700 1 --2- 172.21.15.90:0/985505970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a410e900 0x7f56a4110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.506+0000 7f56abe00700 1 --2- 172.21.15.90:0/985505970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a41054d0 0x7f56a41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.506+0000 7f56abe00700 1 --2- 172.21.15.90:0/985505970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a4105e20 0x7f56a410e370 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.506+0000 7f56abe00700 1 -- 172.21.15.90:0/985505970 >> 172.21.15.90:0/985505970 conn(0x7f56a40fedb0 msgr2=0x7f56a41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.506+0000 7f56abe00700 1 -- 172.21.15.90:0/985505970 shutdown_connections 2024-05-27T07:47:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.506+0000 7f56abe00700 1 -- 172.21.15.90:0/985505970 wait complete. 2024-05-27T07:47:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.508+0000 7f56abe00700 1 Processor -- start 2024-05-27T07:47:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.508+0000 7f56abe00700 1 -- start start 2024-05-27T07:47:43.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56abe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a41054d0 0x7f56a41031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56abe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a4105e20 0x7f56a4103700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56abe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a410e900 0x7f56a4103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56abe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41142e0 con 0x7f56a410e900 2024-05-27T07:47:43.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56abe00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4114160 con 0x7f56a4105e20 2024-05-27T07:47:43.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56abe00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4114460 con 0x7f56a41054d0 2024-05-27T07:47:43.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a410e900 0x7f56a4103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56a9400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a41054d0 0x7f56a41031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a410e900 0x7f56a4103c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34570/0 (socket says 172.21.15.90:34570) 2024-05-27T07:47:43.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56a9e00700 1 -- 172.21.15.90:0/3645872102 learned_addr learned my addr 172.21.15.90:0/3645872102 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:43.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.509+0000 7f56a8a00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a4105e20 0x7f56a4103700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.510+0000 7f56a9e00700 1 -- 172.21.15.90:0/3645872102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a41054d0 msgr2=0x7f56a41031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.510+0000 7f56a9e00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a41054d0 0x7f56a41031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.510+0000 7f56a9e00700 1 -- 172.21.15.90:0/3645872102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a4105e20 msgr2=0x7f56a4103700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.510+0000 7f56a9e00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a4105e20 0x7f56a4103700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.510+0000 7f56a9e00700 1 -- 172.21.15.90:0/3645872102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a41a90d0 con 0x7f56a410e900 2024-05-27T07:47:43.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.510+0000 7f56a9e00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a410e900 0x7f56a4103c40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f569c007ee0 tx=0x7f569c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:43.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.511+0000 7f56a2000700 1 -- 172.21.15.90:0/3645872102 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c00dbd0 con 0x7f56a410e900 2024-05-27T07:47:43.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.511+0000 7f56a2000700 1 -- 172.21.15.90:0/3645872102 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f569c00dd70 con 0x7f56a410e900 2024-05-27T07:47:43.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.511+0000 7f56abe00700 1 -- 172.21.15.90:0/3645872102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a41a93c0 con 0x7f56a410e900 2024-05-27T07:47:43.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.511+0000 7f56a2000700 1 -- 172.21.15.90:0/3645872102 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c0058d0 con 0x7f56a410e900 2024-05-27T07:47:43.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.511+0000 7f56abe00700 1 -- 172.21.15.90:0/3645872102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a41a98d0 con 0x7f56a410e900 2024-05-27T07:47:43.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.515+0000 7f56a2000700 1 -- 172.21.15.90:0/3645872102 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f569c01a020 con 0x7f56a410e900 2024-05-27T07:47:43.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.517+0000 7f56ba86db80 1 Processor -- start 2024-05-27T07:47:43.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.518+0000 7f56ba86db80 1 -- start start 2024-05-27T07:47:43.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.518+0000 7f56a2000700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f568c0b2c20 0x7f568c0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.518+0000 7f56a2000700 1 -- 172.21.15.90:0/3645872102 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f569c0e2310 con 0x7f56a410e900 2024-05-27T07:47:43.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.518+0000 7f56a8a00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f568c0b2c20 0x7f568c0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.518+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.518+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7aed30 0x55e2bb7a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.518+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.518+0000 7f56ba86db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e2bb6d6e40 con 0x55e2bb7a8db0 2024-05-27T07:47:43.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.518+0000 7f56ba86db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2bb6c69e0 con 0x55e2bb629e40 2024-05-27T07:47:43.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.518+0000 7f56ba86db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e2bb6caa20 con 0x55e2bb7aed30 2024-05-27T07:47:43.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7a91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34576/0 (socket says 172.21.15.90:34576) 2024-05-27T07:47:43.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a9e00700 1 -- 172.21.15.90:0/2265918473 learned_addr learned my addr 172.21.15.90:0/2265918473 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:43.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a8a00700 1 --2- 172.21.15.90:0/2265918473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7aed30 0x55e2bb7a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a9400700 1 --2- 172.21.15.90:0/2265918473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a8a00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f568c0b2c20 0x7f568c0b5130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f569000b5c0 tx=0x7f5690009630 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:43.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a9e00700 1 -- 172.21.15.90:0/2265918473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7aed30 msgr2=0x55e2bb7a8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:43.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a9e00700 1 --2- 172.21.15.90:0/2265918473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7aed30 0x55e2bb7a8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a9e00700 1 -- 172.21.15.90:0/2265918473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 msgr2=0x55e2bb7ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:43.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a9e00700 1 --2- 172.21.15.90:0/2265918473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.519+0000 7f56a9e00700 1 -- 172.21.15.90:0/2265918473 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2bb68f420 con 0x55e2bb7a8db0 2024-05-27T07:47:43.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.520+0000 7f56a9e00700 1 --2- 172.21.15.90:0/2265918473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7a91c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f569c0a8fd0 tx=0x7f569c0a87b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b56921673e992e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:43.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.520+0000 7f5696a00700 1 -- 172.21.15.90:0/2265918473 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c0a9040 con 0x55e2bb7a8db0 2024-05-27T07:47:43.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.520+0000 7f5696a00700 1 -- 172.21.15.90:0/2265918473 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f569c0ad3f0 con 0x55e2bb7a8db0 2024-05-27T07:47:43.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.520+0000 7f5696a00700 1 -- 172.21.15.90:0/2265918473 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c0a9530 con 0x55e2bb7a8db0 2024-05-27T07:47:43.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.522+0000 7f56ba86db80 1 -- 172.21.15.90:0/2265918473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7a8db0 msgr2=0x55e2bb7a91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.522+0000 7f56ba86db80 1 --2- 172.21.15.90:0/2265918473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7a91c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f569c0a8fd0 tx=0x7f569c0a87b0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.522+0000 7f56ba86db80 1 -- 172.21.15.90:0/2265918473 shutdown_connections 2024-05-27T07:47:43.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.522+0000 7f56ba86db80 1 --2- 172.21.15.90:0/2265918473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7a91c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.522+0000 7f56ba86db80 1 --2- 172.21.15.90:0/2265918473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.522+0000 7f56ba86db80 1 --2- 172.21.15.90:0/2265918473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7aed30 0x55e2bb7a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.522+0000 7f56ba86db80 1 -- 172.21.15.90:0/2265918473 >> 172.21.15.90:0/2265918473 conn(0x55e2bb689fc0 msgr2=0x55e2bb7a38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:43.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.522+0000 7f56ba86db80 1 -- 172.21.15.90:0/2265918473 shutdown_connections 2024-05-27T07:47:43.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.522+0000 7f56ba86db80 1 -- 172.21.15.90:0/2265918473 wait complete. 2024-05-27T07:47:43.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.523+0000 7f56ba86db80 1 -- start start 2024-05-27T07:47:43.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.523+0000 7f56ba86db80 1 Processor -- start 2024-05-27T07:47:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.524+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.524+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.524+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.524+0000 7f56ba86db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e2bb5ccfa0 con 0x55e2bb7aed30 2024-05-27T07:47:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.524+0000 7f56ba86db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2bb667760 con 0x55e2bb629e40 2024-05-27T07:47:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.524+0000 7f56ba86db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e2bb7b5800 con 0x55e2bb7a8db0 2024-05-27T07:47:43.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.524+0000 7f56ba86db80 10 client.0 _ll_register_callbacks cb 0x55e2bb670c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:43.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.525+0000 7f56a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.525+0000 7f56a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34590/0 (socket says 172.21.15.90:34590) 2024-05-27T07:47:43.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.525+0000 7f56a9e00700 1 -- 172.21.15.90:0/4286112108 learned_addr learned my addr 172.21.15.90:0/4286112108 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:43.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.525+0000 7f56a9400700 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.525+0000 7f56a9e00700 1 -- 172.21.15.90:0/4286112108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7a8db0 msgr2=0x55e2bb7b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.525+0000 7f56a9e00700 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.525+0000 7f56a9e00700 1 -- 172.21.15.90:0/4286112108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 msgr2=0x55e2bb7b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.525+0000 7f56a9e00700 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.525+0000 7f56a9e00700 1 -- 172.21.15.90:0/4286112108 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2bb6168d0 con 0x55e2bb7aed30 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.526+0000 7f56a9e00700 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7b2fb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f569c0a8fd0 tx=0x7f569c0b1cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f4f61c94c38516 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.526+0000 7f568b400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.526+0000 7f568b400700 1 -- 172.21.15.90:0/4286112108 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c005a70 con 0x55e2bb7aed30 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.526+0000 7f568b400700 1 -- 172.21.15.90:0/4286112108 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f569c0a84b0 con 0x55e2bb7aed30 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.526+0000 7f56ba86db80 1 -- 172.21.15.90:0/4286112108 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569c0ad3f0 con 0x55e2bb7aed30 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.526+0000 7f568b400700 1 -- 172.21.15.90:0/4286112108 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f569c08b050 con 0x55e2bb7aed30 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.526+0000 7f56ba86db80 10 client.15621 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:43.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.526+0000 7f56ba86db80 1 -- 172.21.15.90:0/4286112108 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e2bb697060 con 0x55e2bb7aed30 2024-05-27T07:47:43.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.527+0000 7f568b400700 1 -- 172.21.15.90:0/4286112108 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f569c0ad3f0 con 0x55e2bb7aed30 2024-05-27T07:47:43.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.527+0000 7f568b400700 1 -- 172.21.15.90:0/4286112108 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f569c0cb030 con 0x55e2bb7aed30 2024-05-27T07:47:43.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.527+0000 7f56ba86db80 10 client.15621 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:43.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.527+0000 7f56ba86db80 10 client.15621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:47:43.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.527+0000 7f56ba86db80 20 client.15621 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:43.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.527+0000 7f56ba86db80 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e2bb7cadc0 0x55e2bb7cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.527+0000 7f56ba86db80 4 client.15621 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:43.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.527+0000 7f56ba86db80 1 -- 172.21.15.90:0/4286112108 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2bb6d8d90 con 0x55e2bb7cadc0 2024-05-27T07:47:43.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.528+0000 7f56a9400700 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e2bb7cadc0 0x55e2bb7cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.529+0000 7f56a9400700 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e2bb7cadc0 0x55e2bb7cd1e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d35c885072840210 server_cookie=a751c2cdb45430b8 in_seq=0 out_seq=0 2024-05-27T07:47:43.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.529+0000 7f568b400700 10 client.15621 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:47:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.531+0000 7f568b400700 1 -- 172.21.15.90:0/4286112108 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e2bb6d8d90 con 0x55e2bb7cadc0 2024-05-27T07:47:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.531+0000 7f568b400700 10 client.15621 handle_command_reply: tid=0 2024-05-27T07:47:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.531+0000 7f56ba86db80 1 client.15621 shutdown 2024-05-27T07:47:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.531+0000 7f56ba86db80 1 -- 172.21.15.90:0/4286112108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 msgr2=0x55e2bb7b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.531+0000 7f56ba86db80 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7b2fb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f569c0a8fd0 tx=0x7f569c0b1cf0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.532+0000 7f56ba86db80 1 -- 172.21.15.90:0/4286112108 shutdown_connections 2024-05-27T07:47:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.532+0000 7f56ba86db80 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e2bb7cadc0 0x55e2bb7cd1e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.532+0000 7f56ba86db80 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7b2fb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.532+0000 7f56ba86db80 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.532+0000 7f56ba86db80 1 --2- 172.21.15.90:0/4286112108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.532+0000 7f56ba86db80 1 -- 172.21.15.90:0/4286112108 >> 172.21.15.90:0/4286112108 conn(0x55e2bb689fc0 msgr2=0x55e2bb7a3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.532+0000 7f568b400700 0 client.15621 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:47:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.532+0000 7f56ba86db80 1 -- 172.21.15.90:0/4286112108 shutdown_connections 2024-05-27T07:47:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.532+0000 7f56ba86db80 1 -- 172.21.15.90:0/4286112108 wait complete. 2024-05-27T07:47:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.532+0000 7f56ba86db80 20 client.15621 trim_cache size 0 max 16384 2024-05-27T07:47:43.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.551+0000 7f56ba86db80 1 Processor -- start 2024-05-27T07:47:43.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.551+0000 7f56ba86db80 1 -- start start 2024-05-27T07:47:43.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.551+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb629e40 0x55e2bb7a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.551+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7acc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.551+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7aed30 0x55e2bb7c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.551+0000 7f56ba86db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41ae260 con 0x55e2bb629e40 2024-05-27T07:47:43.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.551+0000 7f56ba86db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41afda0 con 0x55e2bb7a8db0 2024-05-27T07:47:43.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.551+0000 7f56ba86db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41ae930 con 0x55e2bb7aed30 2024-05-27T07:47:43.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.552+0000 7f56a9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb629e40 0x55e2bb7a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.552+0000 7f56a9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb629e40 0x55e2bb7a8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34606/0 (socket says 172.21.15.90:34606) 2024-05-27T07:47:43.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.552+0000 7f56a9400700 1 -- 172.21.15.90:0/562875202 learned_addr learned my addr 172.21.15.90:0/562875202 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:43.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.552+0000 7f56a8a00700 1 --2- 172.21.15.90:0/562875202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.552+0000 7f56a9e00700 1 --2- 172.21.15.90:0/562875202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7aed30 0x55e2bb7c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.552+0000 7f56a9400700 1 -- 172.21.15.90:0/562875202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7aed30 msgr2=0x55e2bb7c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:43.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.552+0000 7f56a9400700 1 --2- 172.21.15.90:0/562875202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7aed30 0x55e2bb7c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.552+0000 7f56a9400700 1 -- 172.21.15.90:0/562875202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb7a8db0 msgr2=0x55e2bb7acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.552+0000 7f56a9400700 1 --2- 172.21.15.90:0/562875202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7acc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.552+0000 7f56a9400700 1 -- 172.21.15.90:0/562875202 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2bb79fcf0 con 0x55e2bb629e40 2024-05-27T07:47:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.553+0000 7f56a9400700 1 --2- 172.21.15.90:0/562875202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb629e40 0x55e2bb7a8870 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f569800ffc0 tx=0x7f569800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39225f631aea07e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.553+0000 7f5696000700 1 -- 172.21.15.90:0/562875202 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698014190 con 0x55e2bb629e40 2024-05-27T07:47:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.553+0000 7f5696000700 1 -- 172.21.15.90:0/562875202 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f569801f020 con 0x55e2bb629e40 2024-05-27T07:47:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.553+0000 7f5696000700 1 -- 172.21.15.90:0/562875202 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698003aa0 con 0x55e2bb629e40 2024-05-27T07:47:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.554+0000 7f56ba86db80 1 -- 172.21.15.90:0/562875202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb629e40 msgr2=0x55e2bb7a8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.554+0000 7f56ba86db80 1 --2- 172.21.15.90:0/562875202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb629e40 0x55e2bb7a8870 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f569800ffc0 tx=0x7f569800f3c0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.555+0000 7f56ba86db80 1 -- 172.21.15.90:0/562875202 shutdown_connections 2024-05-27T07:47:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.555+0000 7f56ba86db80 1 --2- 172.21.15.90:0/562875202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7aed30 0x55e2bb7c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.555+0000 7f56ba86db80 1 --2- 172.21.15.90:0/562875202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb629e40 0x55e2bb7a8870 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.555+0000 7f56ba86db80 1 --2- 172.21.15.90:0/562875202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.555+0000 7f56ba86db80 1 -- 172.21.15.90:0/562875202 >> 172.21.15.90:0/562875202 conn(0x55e2bb689fc0 msgr2=0x55e2bb7b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.555+0000 7f56ba86db80 1 -- 172.21.15.90:0/562875202 shutdown_connections 2024-05-27T07:47:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.555+0000 7f56ba86db80 1 -- 172.21.15.90:0/562875202 wait complete. 2024-05-27T07:47:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.556+0000 7f56ba86db80 1 -- start start 2024-05-27T07:47:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.556+0000 7f56ba86db80 1 Processor -- start 2024-05-27T07:47:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.557+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.557+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.557+0000 7f56ba86db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.557+0000 7f56ba86db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41af6d0 con 0x55e2bb7aed30 2024-05-27T07:47:43.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.557+0000 7f56ba86db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41af000 con 0x55e2bb629e40 2024-05-27T07:47:43.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.557+0000 7f56ba86db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41ac1e0 con 0x55e2bb7a8db0 2024-05-27T07:47:43.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.557+0000 7f56ba86db80 10 client.0 _ll_register_callbacks cb 0x55e2bb670c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:43.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.557+0000 7f56a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.557+0000 7f56a9e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34612/0 (socket says 172.21.15.90:34612) 2024-05-27T07:47:43.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.557+0000 7f56a9e00700 1 -- 172.21.15.90:0/2257318574 learned_addr learned my addr 172.21.15.90:0/2257318574 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.558+0000 7f56a9400700 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.558+0000 7f56a8a00700 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.558+0000 7f56a9e00700 1 -- 172.21.15.90:0/2257318574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7a8db0 msgr2=0x55e2bb7ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.558+0000 7f56a9e00700 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.558+0000 7f56a9e00700 1 -- 172.21.15.90:0/2257318574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 msgr2=0x55e2bb7d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.558+0000 7f56a9e00700 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.558+0000 7f56a9e00700 1 -- 172.21.15.90:0/2257318574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2bb7cdb70 con 0x55e2bb7aed30 2024-05-27T07:47:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.559+0000 7f56a9e00700 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7cd3d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f569c0b1e10 tx=0x7f569c0a87b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290cba526106c0c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.559+0000 7f568b400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:43.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.559+0000 7f568b400700 1 -- 172.21.15.90:0/2257318574 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c005a70 con 0x55e2bb7aed30 2024-05-27T07:47:43.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.559+0000 7f568b400700 1 -- 172.21.15.90:0/2257318574 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f569c0df9d0 con 0x55e2bb7aed30 2024-05-27T07:47:43.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.559+0000 7f56ba86db80 1 -- 172.21.15.90:0/2257318574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569801f020 con 0x55e2bb7aed30 2024-05-27T07:47:43.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.560+0000 7f568b400700 1 -- 172.21.15.90:0/2257318574 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f569c0a8b50 con 0x55e2bb7aed30 2024-05-27T07:47:43.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.560+0000 7f56ba86db80 10 client.15627 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:43.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.560+0000 7f56ba86db80 1 -- 172.21.15.90:0/2257318574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e2bb834610 con 0x55e2bb7aed30 2024-05-27T07:47:43.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.560+0000 7f568b400700 1 -- 172.21.15.90:0/2257318574 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f569801f020 con 0x55e2bb7aed30 2024-05-27T07:47:43.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.561+0000 7f568b400700 1 -- 172.21.15.90:0/2257318574 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f569c0cb040 con 0x55e2bb7aed30 2024-05-27T07:47:43.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.561+0000 7f56ba86db80 10 client.15627 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:43.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.561+0000 7f56ba86db80 10 client.15627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:47:43.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.561+0000 7f56ba86db80 20 client.15627 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:43.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.561+0000 7f56ba86db80 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e2bb834ad0 0x55e2bb836ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:43.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.561+0000 7f56ba86db80 4 client.15627 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:47:43.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.561+0000 7f56ba86db80 1 -- 172.21.15.90:0/2257318574 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2bb6d2570 con 0x55e2bb834ad0 2024-05-27T07:47:43.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.562+0000 7f56a9400700 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e2bb834ad0 0x55e2bb836ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:43.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.563+0000 7f56a9400700 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e2bb834ad0 0x55e2bb836ef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d8d4cc719241133 server_cookie=5562d086bf46ba35 in_seq=0 out_seq=0 2024-05-27T07:47:43.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.563+0000 7f568b400700 10 client.15627 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:47:43.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f568b400700 1 -- 172.21.15.90:0/2257318574 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2bb6d2570 con 0x55e2bb834ad0 2024-05-27T07:47:43.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f568b400700 10 client.15627 handle_command_reply: tid=0 2024-05-27T07:47:43.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f56ba86db80 1 client.15627 shutdown 2024-05-27T07:47:43.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f56ba86db80 1 -- 172.21.15.90:0/2257318574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 msgr2=0x55e2bb7cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f56ba86db80 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7cd3d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f569c0b1e10 tx=0x7f569c0a87b0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f56ba86db80 1 -- 172.21.15.90:0/2257318574 shutdown_connections 2024-05-27T07:47:43.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f56ba86db80 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e2bb834ad0 0x55e2bb836ef0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f56ba86db80 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2bb7aed30 0x55e2bb7cd3d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f56ba86db80 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2bb629e40 0x55e2bb7d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f56ba86db80 1 --2- 172.21.15.90:0/2257318574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2bb7a8db0 0x55e2bb7ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f56ba86db80 1 -- 172.21.15.90:0/2257318574 >> 172.21.15.90:0/2257318574 conn(0x55e2bb689fc0 msgr2=0x55e2bb7adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:43.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.564+0000 7f568b400700 0 client.15627 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:47:43.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.565+0000 7f56ba86db80 1 -- 172.21.15.90:0/2257318574 shutdown_connections 2024-05-27T07:47:43.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.565+0000 7f56ba86db80 1 -- 172.21.15.90:0/2257318574 wait complete. 2024-05-27T07:47:43.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.565+0000 7f56ba86db80 20 client.15627 trim_cache size 0 max 16384 2024-05-27T07:47:43.566 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:47:43.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.566+0000 7f56abe00700 1 -- 172.21.15.90:0/3645872102 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f568c0b2c20 msgr2=0x7f568c0b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.566+0000 7f56abe00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f568c0b2c20 0x7f568c0b5130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f569000b5c0 tx=0x7f5690009630 comp rx=0 tx=0).stop 2024-05-27T07:47:43.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.566+0000 7f56abe00700 1 -- 172.21.15.90:0/3645872102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a410e900 msgr2=0x7f56a4103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:43.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.566+0000 7f56abe00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a410e900 0x7f56a4103c40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f569c007ee0 tx=0x7f569c00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.567+0000 7f56abe00700 1 -- 172.21.15.90:0/3645872102 shutdown_connections 2024-05-27T07:47:43.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.567+0000 7f56abe00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56a410e900 0x7f56a4103c40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.567+0000 7f56abe00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a41054d0 0x7f56a41031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.567+0000 7f56abe00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f568c0b2c20 0x7f568c0b5130 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.567+0000 7f56abe00700 1 --2- 172.21.15.90:0/3645872102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a4105e20 0x7f56a4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:43.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.567+0000 7f56abe00700 1 -- 172.21.15.90:0/3645872102 >> 172.21.15.90:0/3645872102 conn(0x7f56a40fedb0 msgr2=0x7f56a40ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:43.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.567+0000 7f56abe00700 1 -- 172.21.15.90:0/3645872102 shutdown_connections 2024-05-27T07:47:43.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:43.567+0000 7f56abe00700 1 -- 172.21.15.90:0/3645872102 wait complete. 2024-05-27T07:47:43.629 INFO:teuthology.orchestra.run.smithi090.stderr:dropping old tables... 2024-05-27T07:47:43.646 INFO:teuthology.orchestra.run.smithi090.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-05-27T07:47:43.647 INFO:teuthology.orchestra.run.smithi090.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-05-27T07:47:43.647 INFO:teuthology.orchestra.run.smithi090.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-05-27T07:47:43.647 INFO:teuthology.orchestra.run.smithi090.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-05-27T07:47:43.647 INFO:teuthology.orchestra.run.smithi090.stderr:creating tables... 2024-05-27T07:47:43.836 INFO:teuthology.orchestra.run.smithi090.stderr:generating data (client-side)... 2024-05-27T07:47:44.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:43 smithi136 ceph-mon[35324]: pgmap v42: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 560 KiB/s wr, 210 op/s 2024-05-27T07:47:44.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:43 smithi129 ceph-mon[35254]: pgmap v42: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 560 KiB/s wr, 210 op/s 2024-05-27T07:47:44.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:43 smithi090 ceph-mon[31429]: pgmap v42: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 560 KiB/s wr, 210 op/s 2024-05-27T07:47:44.610 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:47:45.098 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:45.647 INFO:teuthology.orchestra.run.smithi090.stderr:100000 of 3200000 tuples (3%) done (elapsed 1.70 s, remaining 52.68 s) 2024-05-27T07:47:46.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:45 smithi136 ceph-mon[35324]: pgmap v43: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 239 KiB/s wr, 77 op/s 2024-05-27T07:47:46.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:45 smithi090 ceph-mon[31429]: pgmap v43: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 239 KiB/s wr, 77 op/s 2024-05-27T07:47:46.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:45 smithi129 ceph-mon[35254]: pgmap v43: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 239 KiB/s wr, 77 op/s 2024-05-27T07:47:47.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:47 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:47:47.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:47 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:47:47.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:47 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:47:47.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.499+0000 7f8dc7a00700 1 -- 172.21.15.90:0/2606703323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dc0108730 msgr2=0x7f8dc0110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:47.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.499+0000 7f8dc7a00700 1 --2- 172.21.15.90:0/2606703323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dc0108730 0x7f8dc0110090 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8db800b0d0 tx=0x7f8db800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.499+0000 7f8dc7a00700 1 -- 172.21.15.90:0/2606703323 shutdown_connections 2024-05-27T07:47:47.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.499+0000 7f8dc7a00700 1 --2- 172.21.15.90:0/2606703323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dc0108730 0x7f8dc0110090 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.499+0000 7f8dc7a00700 1 --2- 172.21.15.90:0/2606703323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dc01034b0 0x7f8dc01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.499+0000 7f8dc7a00700 1 --2- 172.21.15.90:0/2606703323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dc0107de0 0x7f8dc01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.499+0000 7f8dc7a00700 1 -- 172.21.15.90:0/2606703323 >> 172.21.15.90:0/2606703323 conn(0x7f8dc00fedb0 msgr2=0x7f8dc01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:47.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.499+0000 7f8dc7a00700 1 -- 172.21.15.90:0/2606703323 shutdown_connections 2024-05-27T07:47:47.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.500+0000 7f8dc7a00700 1 -- 172.21.15.90:0/2606703323 wait complete. 2024-05-27T07:47:47.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.502+0000 7f8dc7a00700 1 Processor -- start 2024-05-27T07:47:47.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.502+0000 7f8dc7a00700 1 -- start start 2024-05-27T07:47:47.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.502+0000 7f8dc7a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dc01034b0 0x7f8dc00ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.502+0000 7f8dc7a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dc0107de0 0x7f8dc00ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.502+0000 7f8dc7a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dc0108730 0x7f8dc00ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.502+0000 7f8dc7a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0113210 con 0x7f8dc01034b0 2024-05-27T07:47:47.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.502+0000 7f8dc7a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0113090 con 0x7f8dc0107de0 2024-05-27T07:47:47.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.503+0000 7f8dc7a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0113390 con 0x7f8dc0108730 2024-05-27T07:47:47.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.503+0000 7f8dc5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dc01034b0 0x7f8dc00ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.503+0000 7f8dc5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dc01034b0 0x7f8dc00ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34624/0 (socket says 172.21.15.90:34624) 2024-05-27T07:47:47.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.503+0000 7f8dc5000700 1 -- 172.21.15.90:0/3621141787 learned_addr learned my addr 172.21.15.90:0/3621141787 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:47.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.503+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dc0107de0 0x7f8dc00ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.503+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dc0108730 0x7f8dc00ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.504+0000 7f8dc5000700 1 -- 172.21.15.90:0/3621141787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dc0108730 msgr2=0x7f8dc00ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:47.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.504+0000 7f8dc5000700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dc0108730 0x7f8dc00ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.504+0000 7f8dc5000700 1 -- 172.21.15.90:0/3621141787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dc0107de0 msgr2=0x7f8dc00ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:47.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.504+0000 7f8dc5000700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dc0107de0 0x7f8dc00ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.504+0000 7f8dc5000700 1 -- 172.21.15.90:0/3621141787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc01a9010 con 0x7f8dc01034b0 2024-05-27T07:47:47.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.504+0000 7f8dc5000700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dc01034b0 0x7f8dc00ff290 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8db400d950 tx=0x7f8db400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:47.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.504+0000 7f8dbd600700 1 -- 172.21.15.90:0/3621141787 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4016070 con 0x7f8dc01034b0 2024-05-27T07:47:47.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.504+0000 7f8dbd600700 1 -- 172.21.15.90:0/3621141787 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8db4010460 con 0x7f8dc01034b0 2024-05-27T07:47:47.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.504+0000 7f8dbd600700 1 -- 172.21.15.90:0/3621141787 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db400f620 con 0x7f8dc01034b0 2024-05-27T07:47:47.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.504+0000 7f8dc7a00700 1 -- 172.21.15.90:0/3621141787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc01a9300 con 0x7f8dc01034b0 2024-05-27T07:47:47.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.505+0000 7f8dc7a00700 1 -- 172.21.15.90:0/3621141787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc01a97f0 con 0x7f8dc01034b0 2024-05-27T07:47:47.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.510+0000 7f8dbd600700 1 -- 172.21.15.90:0/3621141787 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f8db4019070 con 0x7f8dc01034b0 2024-05-27T07:47:47.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.511+0000 7f8dd6512b80 1 Processor -- start 2024-05-27T07:47:47.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.511+0000 7f8dd6512b80 1 -- start start 2024-05-27T07:47:47.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.512+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096c17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.512+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096c1d30 0x557e096bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.512+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096bbdb0 0x557e096bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.512+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557e095e9e40 con 0x557e096c1d30 2024-05-27T07:47:47.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.512+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557e095d99e0 con 0x557e096bbdb0 2024-05-27T07:47:47.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.512+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557e095dda20 con 0x557e0953ce40 2024-05-27T07:47:47.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.512+0000 7f8dc5000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096c1d30 0x557e096bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.513+0000 7f8dc5a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096c17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.513+0000 7f8dc5a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096c17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:54168/0 (socket says 172.21.15.90:54168) 2024-05-27T07:47:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.513+0000 7f8dc5a00700 1 -- 172.21.15.90:0/2517743656 learned_addr learned my addr 172.21.15.90:0/2517743656 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.513+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/2517743656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096bbdb0 0x557e096bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.513+0000 7f8dc5a00700 1 -- 172.21.15.90:0/2517743656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096bbdb0 msgr2=0x557e096bc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.513+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/2517743656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096bbdb0 0x557e096bc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.513+0000 7f8dc5a00700 1 -- 172.21.15.90:0/2517743656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096c1d30 msgr2=0x557e096bb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.513+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/2517743656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096c1d30 0x557e096bb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.513+0000 7f8dc5a00700 1 -- 172.21.15.90:0/2517743656 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e095a2420 con 0x557e0953ce40 2024-05-27T07:47:47.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.513+0000 7f8dc5000700 1 --2- 172.21.15.90:0/2517743656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096c1d30 0x557e096bb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:47.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.514+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/2517743656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096c17f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009bb0 tx=0x7f8db8009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d5846abc7e22ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:47.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.514+0000 7f8dbd600700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8da80b2ba0 0x7f8da80b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.514+0000 7f8db2000700 1 -- 172.21.15.90:0/2517743656 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8020030 con 0x557e0953ce40 2024-05-27T07:47:47.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.515+0000 7f8dc5000700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8da80b2ba0 0x7f8da80b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.515+0000 7f8dbd600700 1 -- 172.21.15.90:0/3621141787 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f8db40e2870 con 0x7f8dc01034b0 2024-05-27T07:47:47.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.515+0000 7f8db2000700 1 -- 172.21.15.90:0/2517743656 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8db8004a50 con 0x557e0953ce40 2024-05-27T07:47:47.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.515+0000 7f8db2000700 1 -- 172.21.15.90:0/2517743656 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8007c80 con 0x557e0953ce40 2024-05-27T07:47:47.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.516+0000 7f8dd6512b80 1 -- 172.21.15.90:0/2517743656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 msgr2=0x557e096c17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:47.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.516+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/2517743656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096c17f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009bb0 tx=0x7f8db8009be0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.516+0000 7f8dc5000700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8da80b2ba0 0x7f8da80b50b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8db40dfa70 tx=0x7f8db4005910 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:47.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.516+0000 7f8dd6512b80 1 -- 172.21.15.90:0/2517743656 shutdown_connections 2024-05-27T07:47:47.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.516+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/2517743656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096bbdb0 0x557e096bc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.516+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/2517743656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096c17f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.516+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/2517743656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096c1d30 0x557e096bb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.516+0000 7f8dd6512b80 1 -- 172.21.15.90:0/2517743656 >> 172.21.15.90:0/2517743656 conn(0x557e0959cfc0 msgr2=0x557e096b68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:47.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.516+0000 7f8dd6512b80 1 -- 172.21.15.90:0/2517743656 shutdown_connections 2024-05-27T07:47:47.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.516+0000 7f8dd6512b80 1 -- 172.21.15.90:0/2517743656 wait complete. 2024-05-27T07:47:47.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.517+0000 7f8dd6512b80 1 -- start start 2024-05-27T07:47:47.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.518+0000 7f8dd6512b80 1 Processor -- start 2024-05-27T07:47:47.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.518+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e0953ce40 0x557e096c5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.519+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e096bbdb0 0x557e096c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.519+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.519+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557e094dffa0 con 0x557e0953ce40 2024-05-27T07:47:47.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.519+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557e0957a760 con 0x557e096c1d30 2024-05-27T07:47:47.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.519+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557e096c87a0 con 0x557e096bbdb0 2024-05-27T07:47:47.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.519+0000 7f8dc5a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e0953ce40 0x557e096c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.519+0000 7f8dd6512b80 10 client.0 _ll_register_callbacks cb 0x557e09583c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:47.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.520+0000 7f8dc5a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e0953ce40 0x557e096c5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34650/0 (socket says 172.21.15.90:34650) 2024-05-27T07:47:47.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.520+0000 7f8dc5a00700 1 -- 172.21.15.90:0/278238586 learned_addr learned my addr 172.21.15.90:0/278238586 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:47.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.520+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e096bbdb0 0x557e096c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.520+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.520+0000 7f8dc5a00700 1 -- 172.21.15.90:0/278238586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e096bbdb0 msgr2=0x557e096c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:47.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.520+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e096bbdb0 0x557e096c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.520+0000 7f8dc5a00700 1 -- 172.21.15.90:0/278238586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 msgr2=0x557e096c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:47.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.520+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.520+0000 7f8dc5a00700 1 -- 172.21.15.90:0/278238586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e095298d0 con 0x557e0953ce40 2024-05-27T07:47:47.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.521+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e0953ce40 0x557e096c5530 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8db800b820 tx=0x7f8db80076e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3a1099e94b7066 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:47.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.521+0000 7f8da6a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:47.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.521+0000 7f8da6a00700 1 -- 172.21.15.90:0/278238586 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8020050 con 0x557e0953ce40 2024-05-27T07:47:47.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.521+0000 7f8da6a00700 1 -- 172.21.15.90:0/278238586 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f8db800f030 con 0x557e0953ce40 2024-05-27T07:47:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.521+0000 7f8dd6512b80 1 -- 172.21.15.90:0/278238586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8db8004a50 con 0x557e0953ce40 2024-05-27T07:47:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.522+0000 7f8da6a00700 1 -- 172.21.15.90:0/278238586 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8db8038050 con 0x557e0953ce40 2024-05-27T07:47:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.522+0000 7f8dd6512b80 10 client.15639 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.522+0000 7f8dd6512b80 1 -- 172.21.15.90:0/278238586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557e095aa060 con 0x557e0953ce40 2024-05-27T07:47:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.522+0000 7f8da6a00700 1 -- 172.21.15.90:0/278238586 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8db8004a50 con 0x557e0953ce40 2024-05-27T07:47:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.523+0000 7f8da6a00700 1 -- 172.21.15.90:0/278238586 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8db8011070 con 0x557e0953ce40 2024-05-27T07:47:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.523+0000 7f8dd6512b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.523+0000 7f8dd6512b80 10 client.15639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:47:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.523+0000 7f8dd6512b80 20 client.15639 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.523+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x557e096d5550 0x557e096d7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.523+0000 7f8dd6512b80 4 client.15639 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.523+0000 7f8dd6512b80 1 -- 172.21.15.90:0/278238586 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e095ebd90 con 0x557e096d5550 2024-05-27T07:47:47.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.524+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x557e096d5550 0x557e096d7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.525+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x557e096d5550 0x557e096d7970 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d8883b344dc00de server_cookie=3a5b1c13fdb518f0 in_seq=0 out_seq=0 2024-05-27T07:47:47.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.525+0000 7f8da6a00700 10 client.15639 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:47:47.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.527+0000 7f8da6a00700 1 -- 172.21.15.90:0/278238586 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557e095ebd90 con 0x557e096d5550 2024-05-27T07:47:47.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.527+0000 7f8da6a00700 10 client.15639 handle_command_reply: tid=0 2024-05-27T07:47:47.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.527+0000 7f8dd6512b80 1 client.15639 shutdown 2024-05-27T07:47:47.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.527+0000 7f8dd6512b80 1 -- 172.21.15.90:0/278238586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e0953ce40 msgr2=0x557e096c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:47.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.527+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e0953ce40 0x557e096c5530 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8db800b820 tx=0x7f8db80076e0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.528+0000 7f8dd6512b80 1 -- 172.21.15.90:0/278238586 shutdown_connections 2024-05-27T07:47:47.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.528+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x557e096d5550 0x557e096d7970 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.528+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.528+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e0953ce40 0x557e096c5530 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.528+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/278238586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e096bbdb0 0x557e096c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.528+0000 7f8dd6512b80 1 -- 172.21.15.90:0/278238586 >> 172.21.15.90:0/278238586 conn(0x557e0959cfc0 msgr2=0x557e096b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:47.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.528+0000 7f8da6a00700 0 client.15639 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:47:47.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.528+0000 7f8dd6512b80 1 -- 172.21.15.90:0/278238586 shutdown_connections 2024-05-27T07:47:47.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.528+0000 7f8dd6512b80 1 -- 172.21.15.90:0/278238586 wait complete. 2024-05-27T07:47:47.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.528+0000 7f8dd6512b80 20 client.15639 trim_cache size 0 max 16384 2024-05-27T07:47:47.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.550+0000 7f8dd6512b80 1 Processor -- start 2024-05-27T07:47:47.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.550+0000 7f8dd6512b80 1 -- start start 2024-05-27T07:47:47.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01b2560 con 0x557e096bbdb0 2024-05-27T07:47:47.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01b40a0 con 0x557e096c1d30 2024-05-27T07:47:47.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01b2c30 con 0x557e0953ce40 2024-05-27T07:47:47.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dbfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dbfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096c5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34658/0 (socket says 172.21.15.90:34658) 2024-05-27T07:47:47.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dbfe00700 1 -- 172.21.15.90:0/302477540 learned_addr learned my addr 172.21.15.90:0/302477540 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:47.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/302477540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.551+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/302477540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.552+0000 7f8dbfe00700 1 -- 172.21.15.90:0/302477540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 msgr2=0x557e096bb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:47.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.552+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/302477540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096bb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.552+0000 7f8dbfe00700 1 -- 172.21.15.90:0/302477540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 msgr2=0x557e096c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:47.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.552+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/302477540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.552+0000 7f8dbfe00700 1 -- 172.21.15.90:0/302477540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e096c1110 con 0x557e096bbdb0 2024-05-27T07:47:47.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.552+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/302477540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096c5f20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0109a70 tx=0x7f8dac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6070b1d58bc68ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:47.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.553+0000 7f8db1600700 1 -- 172.21.15.90:0/302477540 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac003ae0 con 0x557e096bbdb0 2024-05-27T07:47:47.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.553+0000 7f8db1600700 1 -- 172.21.15.90:0/302477540 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dac003c80 con 0x557e096bbdb0 2024-05-27T07:47:47.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.553+0000 7f8db1600700 1 -- 172.21.15.90:0/302477540 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0117c0 con 0x557e096bbdb0 2024-05-27T07:47:47.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.554+0000 7f8dd6512b80 1 -- 172.21.15.90:0/302477540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 msgr2=0x557e096c5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:47.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.554+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/302477540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096c5f20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0109a70 tx=0x7f8dac00dc60 comp rx=0 tx=0).stop 2024-05-27T07:47:47.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.554+0000 7f8dd6512b80 1 -- 172.21.15.90:0/302477540 shutdown_connections 2024-05-27T07:47:47.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.554+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/302477540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.554+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/302477540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096bb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.554+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/302477540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096c5f20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.554+0000 7f8dd6512b80 1 -- 172.21.15.90:0/302477540 >> 172.21.15.90:0/302477540 conn(0x557e0959cfc0 msgr2=0x557e096bf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:47.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.555+0000 7f8dd6512b80 1 -- 172.21.15.90:0/302477540 shutdown_connections 2024-05-27T07:47:47.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.555+0000 7f8dd6512b80 1 -- 172.21.15.90:0/302477540 wait complete. 2024-05-27T07:47:47.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.555+0000 7f8dd6512b80 1 -- start start 2024-05-27T07:47:47.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.556+0000 7f8dd6512b80 1 Processor -- start 2024-05-27T07:47:47.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.556+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096d8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096d86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dd6512b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01b39d0 con 0x557e096bbdb0 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01b3300 con 0x557e096c1d30 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dd6512b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01b04e0 con 0x557e0953ce40 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dd6512b80 10 client.0 _ll_register_callbacks cb 0x557e09583c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dbfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096d86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dc5a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096d8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dc5a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096d8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:54196/0 (socket says 172.21.15.90:54196) 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dc5a00700 1 -- 172.21.15.90:0/1100387447 learned_addr learned my addr 172.21.15.90:0/1100387447 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.557+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.558+0000 7f8dc5a00700 1 -- 172.21.15.90:0/1100387447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 msgr2=0x557e096da6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:47.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.558+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096da6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.558+0000 7f8dc5a00700 1 -- 172.21.15.90:0/1100387447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 msgr2=0x557e096d86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:47.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.558+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096d86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.558+0000 7f8dc5a00700 1 -- 172.21.15.90:0/1100387447 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e096dacb0 con 0x557e0953ce40 2024-05-27T07:47:47.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.558+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:47.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.559+0000 7f8dc5a00700 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096d8190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8db801c740 tx=0x7f8db80076e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dff69f1d70b98d3f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:47.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.559+0000 7f8da6a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:47:47.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.559+0000 7f8da6a00700 1 -- 172.21.15.90:0/1100387447 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8020040 con 0x557e0953ce40 2024-05-27T07:47:47.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.560+0000 7f8dd6512b80 1 -- 172.21.15.90:0/1100387447 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dac003c80 con 0x557e0953ce40 2024-05-27T07:47:47.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.560+0000 7f8da6a00700 1 -- 172.21.15.90:0/1100387447 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f8db8002030 con 0x557e0953ce40 2024-05-27T07:47:47.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.560+0000 7f8da6a00700 1 -- 172.21.15.90:0/1100387447 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8db8004a50 con 0x557e0953ce40 2024-05-27T07:47:47.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.560+0000 7f8dd6512b80 10 client.25318 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:47.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.560+0000 7f8dd6512b80 1 -- 172.21.15.90:0/1100387447 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557e0973f770 con 0x557e0953ce40 2024-05-27T07:47:47.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.561+0000 7f8da6a00700 1 -- 172.21.15.90:0/1100387447 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8dac003c80 con 0x557e0953ce40 2024-05-27T07:47:47.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.562+0000 7f8da6a00700 1 -- 172.21.15.90:0/1100387447 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f8db8011070 con 0x557e0953ce40 2024-05-27T07:47:47.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.562+0000 7f8dd6512b80 10 client.25318 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:47.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.562+0000 7f8dd6512b80 10 client.25318 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:47:47.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.562+0000 7f8dd6512b80 20 client.25318 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:47.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.562+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x557e0973fc30 0x557e09742050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:47.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.562+0000 7f8dd6512b80 4 client.25318 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:47:47.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.562+0000 7f8dd6512b80 1 -- 172.21.15.90:0/1100387447 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e095e5570 con 0x557e0973fc30 2024-05-27T07:47:47.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.563+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x557e0973fc30 0x557e09742050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:47.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.564+0000 7f8da6a00700 10 client.25318 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:47:47.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.564+0000 7f8dbfe00700 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x557e0973fc30 0x557e09742050 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db23cbc6a9a32dcc server_cookie=8d33fbe7eaf0e572 in_seq=0 out_seq=0 2024-05-27T07:47:47.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.564+0000 7f8da6a00700 1 -- 172.21.15.90:0/1100387447 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e095e5570 con 0x557e0973fc30 2024-05-27T07:47:47.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.564+0000 7f8da6a00700 10 client.25318 handle_command_reply: tid=0 2024-05-27T07:47:47.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.564+0000 7f8dd6512b80 1 client.25318 shutdown 2024-05-27T07:47:47.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8dd6512b80 1 -- 172.21.15.90:0/1100387447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 msgr2=0x557e096d8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:47.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096d8190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8db801c740 tx=0x7f8db80076e0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8dd6512b80 1 -- 172.21.15.90:0/1100387447 shutdown_connections 2024-05-27T07:47:47.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x557e0973fc30 0x557e09742050 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e096c1d30 0x557e096da6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557e0953ce40 0x557e096d8190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8da6a00700 0 client.25318 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:47:47.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8dd6512b80 1 --2- 172.21.15.90:0/1100387447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557e096bbdb0 0x557e096d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8dd6512b80 1 -- 172.21.15.90:0/1100387447 >> 172.21.15.90:0/1100387447 conn(0x557e0959cfc0 msgr2=0x557e096c3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:47.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8dd6512b80 1 -- 172.21.15.90:0/1100387447 shutdown_connections 2024-05-27T07:47:47.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.565+0000 7f8dd6512b80 1 -- 172.21.15.90:0/1100387447 wait complete. 2024-05-27T07:47:47.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.566+0000 7f8dd6512b80 20 client.25318 trim_cache size 0 max 16384 2024-05-27T07:47:47.568 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:47:47.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.567+0000 7f8dc7a00700 1 -- 172.21.15.90:0/3621141787 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8da80b2ba0 msgr2=0x7f8da80b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:47.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.567+0000 7f8dc7a00700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8da80b2ba0 0x7f8da80b50b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8db40dfa70 tx=0x7f8db4005910 comp rx=0 tx=0).stop 2024-05-27T07:47:47.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.567+0000 7f8dc7a00700 1 -- 172.21.15.90:0/3621141787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dc01034b0 msgr2=0x7f8dc00ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:47.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.567+0000 7f8dc7a00700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dc01034b0 0x7f8dc00ff290 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8db400d950 tx=0x7f8db400dd10 comp rx=0 tx=0).stop 2024-05-27T07:47:47.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.567+0000 7f8dc7a00700 1 -- 172.21.15.90:0/3621141787 shutdown_connections 2024-05-27T07:47:47.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.567+0000 7f8dc7a00700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dc0108730 0x7f8dc00ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.567+0000 7f8dc7a00700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dc01034b0 0x7f8dc00ff290 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.567+0000 7f8dc7a00700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8da80b2ba0 0x7f8da80b50b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.567+0000 7f8dc7a00700 1 --2- 172.21.15.90:0/3621141787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8dc0107de0 0x7f8dc00ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:47.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.567+0000 7f8dc7a00700 1 -- 172.21.15.90:0/3621141787 >> 172.21.15.90:0/3621141787 conn(0x7f8dc00fedb0 msgr2=0x7f8dc006b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:47.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.568+0000 7f8dc7a00700 1 -- 172.21.15.90:0/3621141787 shutdown_connections 2024-05-27T07:47:47.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:47.568+0000 7f8dc7a00700 1 -- 172.21.15.90:0/3621141787 wait complete. 2024-05-27T07:47:48.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:48 smithi129 ceph-mon[35254]: pgmap v44: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 226 KiB/s wr, 49 op/s 2024-05-27T07:47:48.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:48 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:47:48.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:48 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:47:48.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:48 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:47:48.382 INFO:teuthology.orchestra.run.smithi090.stderr:200000 of 3200000 tuples (6%) done (elapsed 4.43 s, remaining 66.51 s) 2024-05-27T07:47:48.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:48 smithi090 ceph-mon[31429]: pgmap v44: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 226 KiB/s wr, 49 op/s 2024-05-27T07:47:48.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:48 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:47:48.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:48 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:47:48.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:48 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:47:48.497 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:47:48.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:48 smithi136 ceph-mon[35324]: pgmap v44: 129 pgs: 129 active+clean; 56 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 226 KiB/s wr, 49 op/s 2024-05-27T07:47:48.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:48 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:47:48.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:48 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:47:48.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:48 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:47:48.986 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:50.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:50 smithi129 ceph-mon[35254]: pgmap v45: 129 pgs: 129 active+clean; 95 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.1 MiB/s wr, 75 op/s 2024-05-27T07:47:50.436 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:50 smithi090 ceph-mon[31429]: pgmap v45: 129 pgs: 129 active+clean; 95 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.1 MiB/s wr, 75 op/s 2024-05-27T07:47:50.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:50 smithi136 ceph-mon[35324]: pgmap v45: 129 pgs: 129 active+clean; 95 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.1 MiB/s wr, 75 op/s 2024-05-27T07:47:51.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.320+0000 7f764e200700 1 -- 172.21.15.90:0/2563010444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7648109670 msgr2=0x7f7648110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.320+0000 7f764e200700 1 --2- 172.21.15.90:0/2563010444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7648109670 0x7f7648110fd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f764000b0d0 tx=0x7f764000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.320+0000 7f764e200700 1 -- 172.21.15.90:0/2563010444 shutdown_connections 2024-05-27T07:47:51.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.320+0000 7f764e200700 1 --2- 172.21.15.90:0/2563010444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7648109670 0x7f7648110fd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.320+0000 7f764e200700 1 --2- 172.21.15.90:0/2563010444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76481043f0 0x7f7648104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.320+0000 7f764e200700 1 --2- 172.21.15.90:0/2563010444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7648108d20 0x7f7648109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.320+0000 7f764e200700 1 -- 172.21.15.90:0/2563010444 >> 172.21.15.90:0/2563010444 conn(0x7f7648079d00 msgr2=0x7f764807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:51.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.321+0000 7f764e200700 1 -- 172.21.15.90:0/2563010444 shutdown_connections 2024-05-27T07:47:51.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.321+0000 7f764e200700 1 -- 172.21.15.90:0/2563010444 wait complete. 2024-05-27T07:47:51.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.322+0000 7f764e200700 1 Processor -- start 2024-05-27T07:47:51.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.322+0000 7f764e200700 1 -- start start 2024-05-27T07:47:51.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.323+0000 7f764e200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76481043f0 0x7f764819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.323+0000 7f764e200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7648108d20 0x7f764819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.323+0000 7f764e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7648109670 0x7f76481a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.323+0000 7f764e200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f76481141a0 con 0x7f7648109670 2024-05-27T07:47:51.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.323+0000 7f764e200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7648114020 con 0x7f7648108d20 2024-05-27T07:47:51.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.323+0000 7f764e200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7648114320 con 0x7f76481043f0 2024-05-27T07:47:51.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.323+0000 7f764d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7648109670 0x7f76481a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.323+0000 7f764d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7648109670 0x7f76481a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41098/0 (socket says 172.21.15.90:41098) 2024-05-27T07:47:51.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.323+0000 7f764d800700 1 -- 172.21.15.90:0/3853065067 learned_addr learned my addr 172.21.15.90:0/3853065067 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.324+0000 7f7647e00700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7648108d20 0x7f764819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.324+0000 7f764ce00700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76481043f0 0x7f764819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.324+0000 7f764d800700 1 -- 172.21.15.90:0/3853065067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76481043f0 msgr2=0x7f764819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.324+0000 7f764d800700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76481043f0 0x7f764819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.324+0000 7f764d800700 1 -- 172.21.15.90:0/3853065067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7648108d20 msgr2=0x7f764819d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.324+0000 7f764d800700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7648108d20 0x7f764819d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.324+0000 7f764d800700 1 -- 172.21.15.90:0/3853065067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76481a1ca0 con 0x7f7648109670 2024-05-27T07:47:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.324+0000 7f764d800700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7648109670 0x7f76481a1600 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f76400079a0 tx=0x7f7640007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:51.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.325+0000 7f7645600700 1 -- 172.21.15.90:0/3853065067 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764001f070 con 0x7f7648109670 2024-05-27T07:47:51.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.325+0000 7f764e200700 1 -- 172.21.15.90:0/3853065067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76481a1f30 con 0x7f7648109670 2024-05-27T07:47:51.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.325+0000 7f7645600700 1 -- 172.21.15.90:0/3853065067 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7640007d90 con 0x7f7648109670 2024-05-27T07:47:51.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.325+0000 7f7645600700 1 -- 172.21.15.90:0/3853065067 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764001b690 con 0x7f7648109670 2024-05-27T07:47:51.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.325+0000 7f764e200700 1 -- 172.21.15.90:0/3853065067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481a98b0 con 0x7f7648109670 2024-05-27T07:47:51.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.331+0000 7f765e3c2b80 1 Processor -- start 2024-05-27T07:47:51.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.331+0000 7f7645600700 1 -- 172.21.15.90:0/3853065067 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f7640022070 con 0x7f7648109670 2024-05-27T07:47:51.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.331+0000 7f765e3c2b80 1 -- start start 2024-05-27T07:47:51.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.332+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2b7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.332+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.332+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.332+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c40e1e4e40 con 0x55c40e2239f0 2024-05-27T07:47:51.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.332+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c40e1d49e0 con 0x55c40e2c12c0 2024-05-27T07:47:51.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.332+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c40e1d8a20 con 0x55c40e137e40 2024-05-27T07:47:51.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.332+0000 7f7647e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.332+0000 7f7647e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2b7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41102/0 (socket says 172.21.15.90:41102) 2024-05-27T07:47:51.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.332+0000 7f7647e00700 1 -- 172.21.15.90:0/397139519 learned_addr learned my addr 172.21.15.90:0/397139519 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:51.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.333+0000 7f764ce00700 1 --2- 172.21.15.90:0/397139519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.333+0000 7f7647e00700 1 --2- 172.21.15.90:0/397139519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.333+0000 7f7647e00700 1 -- 172.21.15.90:0/397139519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 msgr2=0x55c40e2b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.333+0000 7f7647e00700 1 --2- 172.21.15.90:0/397139519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.333+0000 7f7647e00700 1 -- 172.21.15.90:0/397139519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 msgr2=0x55c40e2c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.333+0000 7f7647e00700 1 --2- 172.21.15.90:0/397139519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.333+0000 7f7647e00700 1 -- 172.21.15.90:0/397139519 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c40e19d420 con 0x55c40e2239f0 2024-05-27T07:47:51.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.333+0000 7f7647e00700 1 --2- 172.21.15.90:0/397139519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2b7750 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7634005950 tx=0x7f763400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f263c7b2200b9a38 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:51.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.333+0000 7f7645600700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f76300b2c20 0x7f76300b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.333+0000 7f763a000700 1 -- 172.21.15.90:0/397139519 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634019020 con 0x55c40e2239f0 2024-05-27T07:47:51.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.334+0000 7f763a000700 1 -- 172.21.15.90:0/397139519 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76340108f0 con 0x55c40e2239f0 2024-05-27T07:47:51.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.334+0000 7f763a000700 1 -- 172.21.15.90:0/397139519 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763400f690 con 0x55c40e2239f0 2024-05-27T07:47:51.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.334+0000 7f7645600700 1 -- 172.21.15.90:0/3853065067 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f764000f020 con 0x7f7648109670 2024-05-27T07:47:51.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.334+0000 7f764ce00700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f76300b2c20 0x7f76300b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.335+0000 7f765e3c2b80 1 -- 172.21.15.90:0/397139519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 msgr2=0x55c40e2b7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.335+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/397139519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2b7750 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7634005950 tx=0x7f763400b690 comp rx=0 tx=0).stop 2024-05-27T07:47:51.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.335+0000 7f764ce00700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f76300b2c20 0x7f76300b5130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55c40e2b3970 tx=0x7f763c009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:51.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.335+0000 7f765e3c2b80 1 -- 172.21.15.90:0/397139519 shutdown_connections 2024-05-27T07:47:51.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.335+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/397139519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.335+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/397139519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2b7750 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.335+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/397139519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.335+0000 7f765e3c2b80 1 -- 172.21.15.90:0/397139519 >> 172.21.15.90:0/397139519 conn(0x55c40e1d1390 msgr2=0x55c40e2b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:51.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.335+0000 7f765e3c2b80 1 -- 172.21.15.90:0/397139519 shutdown_connections 2024-05-27T07:47:51.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.335+0000 7f765e3c2b80 1 -- 172.21.15.90:0/397139519 wait complete. 2024-05-27T07:47:51.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.336+0000 7f765e3c2b80 1 -- start start 2024-05-27T07:47:51.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.336+0000 7f765e3c2b80 1 Processor -- start 2024-05-27T07:47:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.337+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e137e40 0x55c40e2be250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.337+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e2239f0 0x55c40e2be790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.337+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.337+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c40e0dafa0 con 0x55c40e137e40 2024-05-27T07:47:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.337+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c40e175760 con 0x55c40e2c12c0 2024-05-27T07:47:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.337+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c40e2c3e00 con 0x55c40e2239f0 2024-05-27T07:47:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.337+0000 7f765e3c2b80 10 client.0 _ll_register_callbacks cb 0x55c40e17ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.337+0000 7f764ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e137e40 0x55c40e2be250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.337+0000 7f764ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e137e40 0x55c40e2be250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41116/0 (socket says 172.21.15.90:41116) 2024-05-27T07:47:51.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.337+0000 7f764ce00700 1 -- 172.21.15.90:0/2979647159 learned_addr learned my addr 172.21.15.90:0/2979647159 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:51.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.338+0000 7f764d800700 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.338+0000 7f7647e00700 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e2239f0 0x55c40e2be790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.338+0000 7f764ce00700 1 -- 172.21.15.90:0/2979647159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e2239f0 msgr2=0x55c40e2be790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.338+0000 7f764ce00700 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e2239f0 0x55c40e2be790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.338+0000 7f764ce00700 1 -- 172.21.15.90:0/2979647159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 msgr2=0x55c40e2becd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.338+0000 7f764ce00700 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2becd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.338+0000 7f764ce00700 1 -- 172.21.15.90:0/2979647159 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c40e1248d0 con 0x55c40e137e40 2024-05-27T07:47:51.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.338+0000 7f764ce00700 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e137e40 0x55c40e2be250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f763c009670 tx=0x7f763c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d761c0bf4b1a77 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:51.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.338+0000 7f762ea00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:51.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.339+0000 7f762ea00700 1 -- 172.21.15.90:0/2979647159 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c011880 con 0x55c40e137e40 2024-05-27T07:47:51.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.339+0000 7f762ea00700 1 -- 172.21.15.90:0/2979647159 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f763c011a20 con 0x55c40e137e40 2024-05-27T07:47:51.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.339+0000 7f765e3c2b80 1 -- 172.21.15.90:0/2979647159 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76340108f0 con 0x55c40e137e40 2024-05-27T07:47:51.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.339+0000 7f762ea00700 1 -- 172.21.15.90:0/2979647159 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f763c030050 con 0x55c40e137e40 2024-05-27T07:47:51.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.339+0000 7f765e3c2b80 10 client.15657 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:51.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.339+0000 7f765e3c2b80 1 -- 172.21.15.90:0/2979647159 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c40e1a5060 con 0x55c40e137e40 2024-05-27T07:47:51.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.340+0000 7f762ea00700 1 -- 172.21.15.90:0/2979647159 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76340108f0 con 0x55c40e137e40 2024-05-27T07:47:51.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.340+0000 7f762ea00700 1 -- 172.21.15.90:0/2979647159 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f763c013ce0 con 0x55c40e137e40 2024-05-27T07:47:51.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.340+0000 7f765e3c2b80 10 client.15657 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:51.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.340+0000 7f765e3c2b80 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:51.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.340+0000 7f765e3c2b80 20 client.15657 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:51.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.340+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c40e2d0d20 0x55c40e2d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.340+0000 7f765e3c2b80 4 client.15657 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:51.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.340+0000 7f765e3c2b80 1 -- 172.21.15.90:0/2979647159 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c40e1e6d90 con 0x55c40e2d0d20 2024-05-27T07:47:51.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.341+0000 7f7647e00700 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c40e2d0d20 0x55c40e2d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.342+0000 7f7647e00700 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c40e2d0d20 0x55c40e2d31a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b6619020a519b1c server_cookie=4225730ac8b2eafb in_seq=0 out_seq=0 2024-05-27T07:47:51.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.342+0000 7f762ea00700 10 client.15657 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:51.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f762ea00700 1 -- 172.21.15.90:0/2979647159 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c40e1e6d90 con 0x55c40e2d0d20 2024-05-27T07:47:51.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f762ea00700 10 client.15657 handle_command_reply: tid=0 2024-05-27T07:47:51.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f765e3c2b80 1 client.15657 shutdown 2024-05-27T07:47:51.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f765e3c2b80 1 -- 172.21.15.90:0/2979647159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e137e40 msgr2=0x55c40e2be250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e137e40 0x55c40e2be250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f763c009670 tx=0x7f763c004750 comp rx=0 tx=0).stop 2024-05-27T07:47:51.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f765e3c2b80 1 -- 172.21.15.90:0/2979647159 shutdown_connections 2024-05-27T07:47:51.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c40e2d0d20 0x55c40e2d31a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2becd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e137e40 0x55c40e2be250 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/2979647159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e2239f0 0x55c40e2be790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.344+0000 7f765e3c2b80 1 -- 172.21.15.90:0/2979647159 >> 172.21.15.90:0/2979647159 conn(0x55c40e1d1390 msgr2=0x55c40e2b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:51.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.345+0000 7f762ea00700 0 client.15657 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:51.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.345+0000 7f765e3c2b80 1 -- 172.21.15.90:0/2979647159 shutdown_connections 2024-05-27T07:47:51.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.345+0000 7f765e3c2b80 1 -- 172.21.15.90:0/2979647159 wait complete. 2024-05-27T07:47:51.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.345+0000 7f765e3c2b80 20 client.15657 trim_cache size 0 max 16384 2024-05-27T07:47:51.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.363+0000 7f765e3c2b80 1 Processor -- start 2024-05-27T07:47:51.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.363+0000 7f765e3c2b80 1 -- start start 2024-05-27T07:47:51.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.363+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2b6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f76481ae010 con 0x55c40e2239f0 2024-05-27T07:47:51.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76481afb50 con 0x55c40e2c12c0 2024-05-27T07:47:51.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76481ae6e0 con 0x55c40e137e40 2024-05-27T07:47:51.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f7647e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f7647e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41120/0 (socket says 172.21.15.90:41120) 2024-05-27T07:47:51.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f7647e00700 1 -- 172.21.15.90:0/198641721 learned_addr learned my addr 172.21.15.90:0/198641721 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:51.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f764ce00700 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f764d800700 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.364+0000 7f7647e00700 1 -- 172.21.15.90:0/198641721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 msgr2=0x55c40e2b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.365+0000 7f7647e00700 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.365+0000 7f7647e00700 1 -- 172.21.15.90:0/198641721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 msgr2=0x55c40e2b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.365+0000 7f7647e00700 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.365+0000 7f7647e00700 1 -- 172.21.15.90:0/198641721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c40e2b3180 con 0x55c40e2239f0 2024-05-27T07:47:51.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.365+0000 7f764ce00700 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:51.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.365+0000 7f764d800700 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:51.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.365+0000 7f7647e00700 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2cced0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f763400b940 tx=0x7f7634005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b907cf2a7ce179 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:51.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.365+0000 7f7639600700 1 -- 172.21.15.90:0/198641721 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634019040 con 0x55c40e2239f0 2024-05-27T07:47:51.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.365+0000 7f7639600700 1 -- 172.21.15.90:0/198641721 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7634004300 con 0x55c40e2239f0 2024-05-27T07:47:51.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.365+0000 7f7639600700 1 -- 172.21.15.90:0/198641721 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634003e60 con 0x55c40e2239f0 2024-05-27T07:47:51.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.366+0000 7f765e3c2b80 1 -- 172.21.15.90:0/198641721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 msgr2=0x55c40e2cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.366+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2cced0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f763400b940 tx=0x7f7634005740 comp rx=0 tx=0).stop 2024-05-27T07:47:51.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.367+0000 7f765e3c2b80 1 -- 172.21.15.90:0/198641721 shutdown_connections 2024-05-27T07:47:51.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.367+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.367+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.367+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/198641721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2cced0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.367+0000 7f765e3c2b80 1 -- 172.21.15.90:0/198641721 >> 172.21.15.90:0/198641721 conn(0x55c40e1d1390 msgr2=0x55c40e2b6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:51.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.367+0000 7f765e3c2b80 1 -- 172.21.15.90:0/198641721 shutdown_connections 2024-05-27T07:47:51.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.367+0000 7f765e3c2b80 1 -- 172.21.15.90:0/198641721 wait complete. 2024-05-27T07:47:51.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.367+0000 7f765e3c2b80 1 -- start start 2024-05-27T07:47:51.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.368+0000 7f765e3c2b80 1 Processor -- start 2024-05-27T07:47:51.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.368+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.368+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2c4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.368+0000 7f765e3c2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.368+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f76481af480 con 0x55c40e2239f0 2024-05-27T07:47:51.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.368+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76481aedb0 con 0x55c40e2c12c0 2024-05-27T07:47:51.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.368+0000 7f765e3c2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76481abfb0 con 0x55c40e137e40 2024-05-27T07:47:51.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.368+0000 7f765e3c2b80 10 client.0 _ll_register_callbacks cb 0x55c40e17ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:51.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.368+0000 7f7647e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f7647e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2c4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41136/0 (socket says 172.21.15.90:41136) 2024-05-27T07:47:51.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f7647e00700 1 -- 172.21.15.90:0/1297716876 learned_addr learned my addr 172.21.15.90:0/1297716876 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:51.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f764ce00700 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f764d800700 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f7647e00700 1 -- 172.21.15.90:0/1297716876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 msgr2=0x55c40e2c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f7647e00700 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f7647e00700 1 -- 172.21.15.90:0/1297716876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 msgr2=0x55c40e2c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:51.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f7647e00700 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f7647e00700 1 -- 172.21.15.90:0/1297716876 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c40e2bf040 con 0x55c40e2239f0 2024-05-27T07:47:51.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f764ce00700 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:51.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f7647e00700 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2c4080 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7634013fd0 tx=0x7f763400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6f8dd04dd4225f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:51.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.369+0000 7f762ea00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:51.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.370+0000 7f762ea00700 1 -- 172.21.15.90:0/1297716876 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634039060 con 0x55c40e2239f0 2024-05-27T07:47:51.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.370+0000 7f762ea00700 1 -- 172.21.15.90:0/1297716876 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f7634003a90 con 0x55c40e2239f0 2024-05-27T07:47:51.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.370+0000 7f765e3c2b80 1 -- 172.21.15.90:0/1297716876 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7634004300 con 0x55c40e2239f0 2024-05-27T07:47:51.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.370+0000 7f762ea00700 1 -- 172.21.15.90:0/1297716876 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7634040050 con 0x55c40e2239f0 2024-05-27T07:47:51.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.370+0000 7f765e3c2b80 10 client.15663 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:51.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.370+0000 7f765e3c2b80 1 -- 172.21.15.90:0/1297716876 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c40e33a630 con 0x55c40e2239f0 2024-05-27T07:47:51.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.371+0000 7f762ea00700 1 -- 172.21.15.90:0/1297716876 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7634004300 con 0x55c40e2239f0 2024-05-27T07:47:51.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.371+0000 7f762ea00700 1 -- 172.21.15.90:0/1297716876 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f76340105a0 con 0x55c40e2239f0 2024-05-27T07:47:51.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.371+0000 7f765e3c2b80 10 client.15663 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.371+0000 7f765e3c2b80 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.371+0000 7f765e3c2b80 20 client.15663 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.371+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c40e33aaa0 0x55c40e33cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.371+0000 7f765e3c2b80 4 client.15663 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.371+0000 7f765e3c2b80 1 -- 172.21.15.90:0/1297716876 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c40e1e0570 con 0x55c40e33aaa0 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.372+0000 7f764ce00700 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c40e33aaa0 0x55c40e33cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.373+0000 7f764ce00700 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c40e33aaa0 0x55c40e33cec0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba570b1167d5f3e3 server_cookie=d6e4721c0470bd02 in_seq=0 out_seq=0 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.373+0000 7f762ea00700 10 client.15663 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.373+0000 7f762ea00700 1 -- 172.21.15.90:0/1297716876 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c40e1e0570 con 0x55c40e33aaa0 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.373+0000 7f762ea00700 10 client.15663 handle_command_reply: tid=0 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.373+0000 7f765e3c2b80 1 client.15663 shutdown 2024-05-27T07:47:51.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.373+0000 7f765e3c2b80 1 -- 172.21.15.90:0/1297716876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 msgr2=0x55c40e2c4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2c4080 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7634013fd0 tx=0x7f763400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f765e3c2b80 1 -- 172.21.15.90:0/1297716876 shutdown_connections 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c40e33aaa0 0x55c40e33cec0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40e2c12c0 0x55c40e2c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c40e137e40 0x55c40e2c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f765e3c2b80 1 --2- 172.21.15.90:0/1297716876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40e2239f0 0x55c40e2c4080 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f765e3c2b80 1 -- 172.21.15.90:0/1297716876 >> 172.21.15.90:0/1297716876 conn(0x55c40e1d1390 msgr2=0x55c40e2b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f762ea00700 0 client.15663 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f765e3c2b80 1 -- 172.21.15.90:0/1297716876 shutdown_connections 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f765e3c2b80 1 -- 172.21.15.90:0/1297716876 wait complete. 2024-05-27T07:47:51.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.374+0000 7f765e3c2b80 20 client.15663 trim_cache size 0 max 16384 2024-05-27T07:47:51.384 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:47:51.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 -- 172.21.15.90:0/3853065067 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f76300b2c20 msgr2=0x7f76300b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f76300b2c20 0x7f76300b5130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55c40e2b3970 tx=0x7f763c009260 comp rx=0 tx=0).stop 2024-05-27T07:47:51.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 -- 172.21.15.90:0/3853065067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7648109670 msgr2=0x7f76481a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:51.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7648109670 0x7f76481a1600 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f76400079a0 tx=0x7f7640007a80 comp rx=0 tx=0).stop 2024-05-27T07:47:51.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 -- 172.21.15.90:0/3853065067 shutdown_connections 2024-05-27T07:47:51.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7648109670 0x7f76481a1600 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76481043f0 0x7f764819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f76300b2c20 0x7f76300b5130 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 --2- 172.21.15.90:0/3853065067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7648108d20 0x7f764819d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 -- 172.21.15.90:0/3853065067 >> 172.21.15.90:0/3853065067 conn(0x7f7648079d00 msgr2=0x7f764810f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 -- 172.21.15.90:0/3853065067 shutdown_connections 2024-05-27T07:47:51.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:51.375+0000 7f764e200700 1 -- 172.21.15.90:0/3853065067 wait complete. 2024-05-27T07:47:51.597 INFO:teuthology.orchestra.run.smithi090.stderr:300000 of 3200000 tuples (9%) done (elapsed 7.65 s, remaining 73.95 s) 2024-05-27T07:47:52.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:52 smithi090 ceph-mon[31429]: pgmap v46: 129 pgs: 129 active+clean; 95 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.9 MiB/s wr, 31 op/s 2024-05-27T07:47:52.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:52 smithi129 ceph-mon[35254]: pgmap v46: 129 pgs: 129 active+clean; 95 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.9 MiB/s wr, 31 op/s 2024-05-27T07:47:52.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:52 smithi136 ceph-mon[35324]: pgmap v46: 129 pgs: 129 active+clean; 95 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.9 MiB/s wr, 31 op/s 2024-05-27T07:47:53.241 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:47:53.241 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:47:53.704 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:54.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:54 smithi090 ceph-mon[31429]: pgmap v47: 129 pgs: 129 active+clean; 125 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 40 op/s 2024-05-27T07:47:54.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:54 smithi129 ceph-mon[35254]: pgmap v47: 129 pgs: 129 active+clean; 125 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 40 op/s 2024-05-27T07:47:54.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:54 smithi136 ceph-mon[35324]: pgmap v47: 129 pgs: 129 active+clean; 125 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 10 MiB/s wr, 40 op/s 2024-05-27T07:47:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:400000 of 3200000 tuples (12%) done (elapsed 10.64 s, remaining 74.51 s) 2024-05-27T07:47:56.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.101+0000 7f4fa5c00700 1 -- 172.21.15.90:0/1558119594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fa0103260 msgr2=0x7f4fa0103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:56 smithi090 ceph-mon[31429]: pgmap v48: 129 pgs: 129 active+clean; 129 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 35 op/s 2024-05-27T07:47:56.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.101+0000 7f4fa5c00700 1 --2- 172.21.15.90:0/1558119594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fa0103260 0x7f4fa0103670 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009990 tx=0x7f4f88009ca0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.102+0000 7f4fa5c00700 1 -- 172.21.15.90:0/1558119594 shutdown_connections 2024-05-27T07:47:56.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.102+0000 7f4fa5c00700 1 --2- 172.21.15.90:0/1558119594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa01086b0 0x7f4fa0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.102+0000 7f4fa5c00700 1 --2- 172.21.15.90:0/1558119594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fa0103260 0x7f4fa0103670 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.102+0000 7f4fa5c00700 1 --2- 172.21.15.90:0/1558119594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fa0103bb0 0x7f4fa01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.102+0000 7f4fa5c00700 1 -- 172.21.15.90:0/1558119594 >> 172.21.15.90:0/1558119594 conn(0x7f4fa00fef20 msgr2=0x7f4fa0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.102+0000 7f4fa5c00700 1 -- 172.21.15.90:0/1558119594 shutdown_connections 2024-05-27T07:47:56.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.102+0000 7f4fa5c00700 1 -- 172.21.15.90:0/1558119594 wait complete. 2024-05-27T07:47:56.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.104+0000 7f4fa5c00700 1 Processor -- start 2024-05-27T07:47:56.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.104+0000 7f4fa5c00700 1 -- start start 2024-05-27T07:47:56.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.104+0000 7f4fa5c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fa0103260 0x7f4fa01a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.104+0000 7f4fa5c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa0103bb0 0x7f4fa01a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.104+0000 7f4fa5c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fa01086b0 0x7f4fa01a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.105+0000 7f4fa5c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa011a980 con 0x7f4fa0103260 2024-05-27T07:47:56.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.105+0000 7f4fa5c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa011a800 con 0x7f4fa0103bb0 2024-05-27T07:47:56.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.105+0000 7f4fa5c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa011ab00 con 0x7f4fa01086b0 2024-05-27T07:47:56.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.105+0000 7f4f9ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fa0103260 0x7f4fa01a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.105+0000 7f4f9e000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa0103bb0 0x7f4fa01a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.105+0000 7f4f9f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fa01086b0 0x7f4fa01a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.105+0000 7f4f9e000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa0103bb0 0x7f4fa01a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:35498/0 (socket says 172.21.15.90:35498) 2024-05-27T07:47:56.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.105+0000 7f4f9e000700 1 -- 172.21.15.90:0/615707495 learned_addr learned my addr 172.21.15.90:0/615707495 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:56.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.106+0000 7f4f9ea00700 1 -- 172.21.15.90:0/615707495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fa01086b0 msgr2=0x7f4fa01a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.106+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fa01086b0 0x7f4fa01a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.106+0000 7f4f9ea00700 1 -- 172.21.15.90:0/615707495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa0103bb0 msgr2=0x7f4fa01a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.107+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa0103bb0 0x7f4fa01a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.107+0000 7f4f9ea00700 1 -- 172.21.15.90:0/615707495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa01a6220 con 0x7f4fa0103260 2024-05-27T07:47:56.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.107+0000 7f4f9f400700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fa01086b0 0x7f4fa01a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:56.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.107+0000 7f4f9e000700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa0103bb0 0x7f4fa01a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:56.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.108+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fa0103260 0x7f4fa01a1450 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000c00 tx=0x7f4f880057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.110+0000 7f4f97400700 1 -- 172.21.15.90:0/615707495 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88026070 con 0x7f4fa0103260 2024-05-27T07:47:56.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.110+0000 7f4f97400700 1 -- 172.21.15.90:0/615707495 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f8800fbc0 con 0x7f4fa0103260 2024-05-27T07:47:56.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.110+0000 7f4f97400700 1 -- 172.21.15.90:0/615707495 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88021aa0 con 0x7f4fa0103260 2024-05-27T07:47:56.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.110+0000 7f4fa5c00700 1 -- 172.21.15.90:0/615707495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa01a64b0 con 0x7f4fa0103260 2024-05-27T07:47:56.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.110+0000 7f4fa5c00700 1 -- 172.21.15.90:0/615707495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa01a6940 con 0x7f4fa0103260 2024-05-27T07:47:56.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.116+0000 7f4f97400700 1 -- 172.21.15.90:0/615707495 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f4f8800f730 con 0x7f4fa0103260 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.117+0000 7f4fb4838b80 1 Processor -- start 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.117+0000 7f4fb4838b80 1 -- start start 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.118+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf1e7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.118+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf1e7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.118+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf203190 0x55aebf2055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.118+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aebf115ec0 con 0x55aebf154a70 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.118+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aebf105a60 con 0x55aebf203190 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.118+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aebf109aa0 con 0x55aebf068ec0 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.119+0000 7f4f9f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf1e7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.119+0000 7f4f9f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf1e7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41150/0 (socket says 172.21.15.90:41150) 2024-05-27T07:47:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.119+0000 7f4f9f400700 1 -- 172.21.15.90:0/3358565076 learned_addr learned my addr 172.21.15.90:0/3358565076 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.119+0000 7f4f9e000700 1 --2- 172.21.15.90:0/3358565076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf203190 0x55aebf2055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.120+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/3358565076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf1e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.120+0000 7f4f9f400700 1 -- 172.21.15.90:0/3358565076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 msgr2=0x55aebf1e7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.120+0000 7f4f9f400700 1 --2- 172.21.15.90:0/3358565076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf1e7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.120+0000 7f4f9f400700 1 -- 172.21.15.90:0/3358565076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf203190 msgr2=0x55aebf2055b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.120+0000 7f4f97400700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f840b2cd0 0x7f4f840b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.120+0000 7f4f9f400700 1 --2- 172.21.15.90:0/3358565076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf203190 0x55aebf2055b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.120+0000 7f4f9f400700 1 -- 172.21.15.90:0/3358565076 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aebf0ce4a0 con 0x55aebf154a70 2024-05-27T07:47:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.120+0000 7f4f97400700 1 -- 172.21.15.90:0/615707495 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f4f880e3980 con 0x7f4fa0103260 2024-05-27T07:47:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.121+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/3358565076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf1e7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.121+0000 7f4f9f400700 1 --2- 172.21.15.90:0/3358565076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf1e7450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800a730 tx=0x7f4f980078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9019e170e5da5d05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.121+0000 7f4f9f400700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f840b2cd0 0x7f4f840b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.122+0000 7f4f8fe00700 1 -- 172.21.15.90:0/3358565076 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98004dd0 con 0x55aebf154a70 2024-05-27T07:47:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.122+0000 7f4f8fe00700 1 -- 172.21.15.90:0/3358565076 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f98012e50 con 0x55aebf154a70 2024-05-27T07:47:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.122+0000 7f4f8fe00700 1 -- 172.21.15.90:0/3358565076 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98014460 con 0x55aebf154a70 2024-05-27T07:47:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.122+0000 7f4f9f400700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f840b2cd0 0x7f4f840b51e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4f9801e000 tx=0x7f4f98021000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.123+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3358565076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 msgr2=0x55aebf1e7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.123+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/3358565076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf1e7450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800a730 tx=0x7f4f980078d0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.124+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3358565076 shutdown_connections 2024-05-27T07:47:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.124+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/3358565076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf203190 0x55aebf2055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.124+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/3358565076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf1e7450 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.124+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/3358565076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf1e7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.124+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3358565076 >> 172.21.15.90:0/3358565076 conn(0x55aebf102410 msgr2=0x55aebf1e3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.124+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3358565076 shutdown_connections 2024-05-27T07:47:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.124+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3358565076 wait complete. 2024-05-27T07:47:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.125+0000 7f4fb4838b80 1 -- start start 2024-05-27T07:47:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.125+0000 7f4fb4838b80 1 Processor -- start 2024-05-27T07:47:56.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.126+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf200200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.126+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf200740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.126+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf203190 0x55aebf200c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.126+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aebf00c020 con 0x55aebf154a70 2024-05-27T07:47:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.126+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aebf0a67e0 con 0x55aebf203190 2024-05-27T07:47:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.126+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aebf205c40 con 0x55aebf068ec0 2024-05-27T07:47:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.126+0000 7f4fb4838b80 10 client.0 _ll_register_callbacks cb 0x55aebf0afcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:56.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.127+0000 7f4f9ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf200200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.127+0000 7f4f9ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf200200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33722/0 (socket says 172.21.15.90:33722) 2024-05-27T07:47:56.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.127+0000 7f4f9ea00700 1 -- 172.21.15.90:0/3978762384 learned_addr learned my addr 172.21.15.90:0/3978762384 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:56.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.127+0000 7f4f9e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf200740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.127+0000 7f4f9f400700 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf203190 0x55aebf200c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.127+0000 7f4f9ea00700 1 -- 172.21.15.90:0/3978762384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf203190 msgr2=0x55aebf200c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.127+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf203190 0x55aebf200c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.127+0000 7f4f9ea00700 1 -- 172.21.15.90:0/3978762384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 msgr2=0x55aebf200740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.128+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf200740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.128+0000 7f4f9ea00700 1 -- 172.21.15.90:0/3978762384 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aebf055950 con 0x55aebf068ec0 2024-05-27T07:47:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.128+0000 7f4f9e000700 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf200740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.128+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf200200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4f880aa6d0 tx=0x7f4f880b3360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3525c47fb07906d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.128+0000 7f4f8cc00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:47:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.129+0000 7f4f8cc00700 1 -- 172.21.15.90:0/3978762384 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f880a4e60 con 0x55aebf068ec0 2024-05-27T07:47:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.129+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3978762384 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f98012e50 con 0x55aebf068ec0 2024-05-27T07:47:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.129+0000 7f4f8cc00700 1 -- 172.21.15.90:0/3978762384 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f4f88005110 con 0x55aebf068ec0 2024-05-27T07:47:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.129+0000 7f4f8cc00700 1 -- 172.21.15.90:0/3978762384 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4f880a9eb0 con 0x55aebf068ec0 2024-05-27T07:47:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.129+0000 7f4fb4838b80 10 client.25342 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.129+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3978762384 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aebf0d60e0 con 0x55aebf068ec0 2024-05-27T07:47:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.130+0000 7f4f8cc00700 1 -- 172.21.15.90:0/3978762384 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f98012e50 con 0x55aebf068ec0 2024-05-27T07:47:56.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.130+0000 7f4f8cc00700 1 -- 172.21.15.90:0/3978762384 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4f88029030 con 0x55aebf068ec0 2024-05-27T07:47:56.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.130+0000 7f4fb4838b80 10 client.25342 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:56.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.130+0000 7f4fb4838b80 10 client.25342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:56.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.130+0000 7f4fb4838b80 20 client.25342 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:56.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.130+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aebf212af0 0x55aebf214f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.130+0000 7f4fb4838b80 4 client.25342 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:56.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.130+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3978762384 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aebf117e10 con 0x55aebf212af0 2024-05-27T07:47:56.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.131+0000 7f4f9e000700 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aebf212af0 0x55aebf214f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.132+0000 7f4f9e000700 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aebf212af0 0x55aebf214f70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e973a7edacc8cd server_cookie=bc499c57984b07c4 in_seq=0 out_seq=0 2024-05-27T07:47:56.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.132+0000 7f4f8cc00700 10 client.25342 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.135+0000 7f4f8cc00700 1 -- 172.21.15.90:0/3978762384 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55aebf117e10 con 0x55aebf212af0 2024-05-27T07:47:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.135+0000 7f4f8cc00700 10 client.25342 handle_command_reply: tid=0 2024-05-27T07:47:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.135+0000 7f4fb4838b80 1 client.25342 shutdown 2024-05-27T07:47:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.135+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3978762384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 msgr2=0x55aebf200200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.135+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf200200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4f880aa6d0 tx=0x7f4f880b3360 comp rx=0 tx=0).stop 2024-05-27T07:47:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.136+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3978762384 shutdown_connections 2024-05-27T07:47:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.136+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aebf212af0 0x55aebf214f70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.136+0000 7f4f8cc00700 0 client.25342 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.136+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf203190 0x55aebf200c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.136+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf200200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.136+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/3978762384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf200740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.136+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3978762384 >> 172.21.15.90:0/3978762384 conn(0x55aebf102410 msgr2=0x55aebf1e7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.136+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3978762384 shutdown_connections 2024-05-27T07:47:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.136+0000 7f4fb4838b80 1 -- 172.21.15.90:0/3978762384 wait complete. 2024-05-27T07:47:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.136+0000 7f4fb4838b80 20 client.25342 trim_cache size 0 max 16384 2024-05-27T07:47:56.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.168+0000 7f4fb4838b80 1 Processor -- start 2024-05-27T07:47:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.168+0000 7f4fb4838b80 1 -- start start 2024-05-27T07:47:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.168+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf1e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.168+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf154a70 0x55aebf1fb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.168+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf203190 0x55aebf2045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.168+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa01ae0f0 con 0x55aebf203190 2024-05-27T07:47:56.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.168+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa01afc30 con 0x55aebf154a70 2024-05-27T07:47:56.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.168+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa01ae7c0 con 0x55aebf068ec0 2024-05-27T07:47:56.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.169+0000 7f4f9f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf203190 0x55aebf2045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.169+0000 7f4f9f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf203190 0x55aebf2045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41166/0 (socket says 172.21.15.90:41166) 2024-05-27T07:47:56.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.169+0000 7f4f9f400700 1 -- 172.21.15.90:0/1788816409 learned_addr learned my addr 172.21.15.90:0/1788816409 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:56.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.169+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/1788816409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf1e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f9e000700 1 --2- 172.21.15.90:0/1788816409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf154a70 0x55aebf1fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f9ea00700 1 -- 172.21.15.90:0/1788816409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf154a70 msgr2=0x55aebf1fb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/1788816409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf154a70 0x55aebf1fb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f9ea00700 1 -- 172.21.15.90:0/1788816409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf203190 msgr2=0x55aebf2045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/1788816409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf203190 0x55aebf2045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f9ea00700 1 -- 172.21.15.90:0/1788816409 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aebf1df2c0 con 0x55aebf068ec0 2024-05-27T07:47:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f9f400700 1 --2- 172.21.15.90:0/1788816409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf203190 0x55aebf2045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/1788816409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf1e7aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4f880b3530 tx=0x7f4f88009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0dafab5ed9b0155 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f8f400700 1 -- 172.21.15.90:0/1788816409 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f880a4e60 con 0x55aebf068ec0 2024-05-27T07:47:56.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f8f400700 1 -- 172.21.15.90:0/1788816409 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f880aac20 con 0x55aebf068ec0 2024-05-27T07:47:56.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.171+0000 7f4f8f400700 1 -- 172.21.15.90:0/1788816409 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8800fa20 con 0x55aebf068ec0 2024-05-27T07:47:56.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.172+0000 7f4fb4838b80 1 -- 172.21.15.90:0/1788816409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 msgr2=0x55aebf1e7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.172+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/1788816409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf1e7aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4f880b3530 tx=0x7f4f88009ca0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.173+0000 7f4fb4838b80 1 -- 172.21.15.90:0/1788816409 shutdown_connections 2024-05-27T07:47:56.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.173+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/1788816409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf203190 0x55aebf2045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.173+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/1788816409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf068ec0 0x55aebf1e7aa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.173+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/1788816409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf154a70 0x55aebf1fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.173+0000 7f4fb4838b80 1 -- 172.21.15.90:0/1788816409 >> 172.21.15.90:0/1788816409 conn(0x55aebf102410 msgr2=0x55aebf1fb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.173+0000 7f4fb4838b80 1 -- 172.21.15.90:0/1788816409 shutdown_connections 2024-05-27T07:47:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.173+0000 7f4fb4838b80 1 -- 172.21.15.90:0/1788816409 wait complete. 2024-05-27T07:47:56.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.174+0000 7f4fb4838b80 1 -- start start 2024-05-27T07:47:56.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.174+0000 7f4fb4838b80 1 Processor -- start 2024-05-27T07:47:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.175+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf068ec0 0x55aebf20b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.175+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf20b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.175+0000 7f4fb4838b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf203190 0x55aebf205eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.175+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa01af560 con 0x55aebf154a70 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.175+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa01aee90 con 0x55aebf068ec0 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.175+0000 7f4fb4838b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa0073b60 con 0x55aebf203190 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.175+0000 7f4fb4838b80 10 client.0 _ll_register_callbacks cb 0x55aebf0afcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.175+0000 7f4f9e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf20b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.175+0000 7f4f9e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf20b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41170/0 (socket says 172.21.15.90:41170) 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.175+0000 7f4f9e000700 1 -- 172.21.15.90:0/566685660 learned_addr learned my addr 172.21.15.90:0/566685660 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.176+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf068ec0 0x55aebf20b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.176+0000 7f4f9f400700 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf203190 0x55aebf205eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.176+0000 7f4f9e000700 1 -- 172.21.15.90:0/566685660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf203190 msgr2=0x55aebf205eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.176+0000 7f4f9e000700 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf203190 0x55aebf205eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.176+0000 7f4f9e000700 1 -- 172.21.15.90:0/566685660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf068ec0 msgr2=0x55aebf20b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.176+0000 7f4f9e000700 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf068ec0 0x55aebf20b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.176+0000 7f4f9e000700 1 -- 172.21.15.90:0/566685660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aebf200ff0 con 0x55aebf154a70 2024-05-27T07:47:56.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.176+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf068ec0 0x55aebf20b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:47:56.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.176+0000 7f4f9e000700 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf20b690 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55aebf1e47e0 tx=0x7f4f9000f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d70680707c5911 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.176+0000 7f4f8cc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.177+0000 7f4f8cc00700 1 -- 172.21.15.90:0/566685660 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90028430 con 0x55aebf154a70 2024-05-27T07:47:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.177+0000 7f4f8cc00700 1 -- 172.21.15.90:0/566685660 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f4f9001e400 con 0x55aebf154a70 2024-05-27T07:47:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.177+0000 7f4fb4838b80 1 -- 172.21.15.90:0/566685660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f880aac20 con 0x55aebf154a70 2024-05-27T07:47:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.177+0000 7f4f8cc00700 1 -- 172.21.15.90:0/566685660 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4f9001e600 con 0x55aebf154a70 2024-05-27T07:47:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.177+0000 7f4fb4838b80 10 client.15681 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.177+0000 7f4fb4838b80 1 -- 172.21.15.90:0/566685660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aebf27df20 con 0x55aebf154a70 2024-05-27T07:47:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.178+0000 7f4f8cc00700 1 -- 172.21.15.90:0/566685660 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f880aac20 con 0x55aebf154a70 2024-05-27T07:47:56.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.178+0000 7f4f8cc00700 1 -- 172.21.15.90:0/566685660 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f4f90028960 con 0x55aebf154a70 2024-05-27T07:47:56.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.178+0000 7f4fb4838b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:56.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.178+0000 7f4fb4838b80 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:56.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.178+0000 7f4fb4838b80 20 client.15681 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:56.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.178+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aebf212af0 0x55aebf2806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.178+0000 7f4fb4838b80 4 client.15681 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:47:56.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.178+0000 7f4fb4838b80 1 -- 172.21.15.90:0/566685660 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aebf1115f0 con 0x55aebf212af0 2024-05-27T07:47:56.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.178+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aebf212af0 0x55aebf2806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.179+0000 7f4f9ea00700 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aebf212af0 0x55aebf2806a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3911ba6091aa4f4 server_cookie=7e1367e45fc7bf1e in_seq=0 out_seq=0 2024-05-27T07:47:56.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.179+0000 7f4f8cc00700 10 client.15681 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:56.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.182+0000 7f4f8cc00700 1 -- 172.21.15.90:0/566685660 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aebf1115f0 con 0x55aebf212af0 2024-05-27T07:47:56.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.182+0000 7f4f8cc00700 10 client.15681 handle_command_reply: tid=0 2024-05-27T07:47:56.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.182+0000 7f4fb4838b80 1 client.15681 shutdown 2024-05-27T07:47:56.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.182+0000 7f4fb4838b80 1 -- 172.21.15.90:0/566685660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 msgr2=0x55aebf20b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.182+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf20b690 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55aebf1e47e0 tx=0x7f4f9000f700 comp rx=0 tx=0).stop 2024-05-27T07:47:56.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.183+0000 7f4fb4838b80 1 -- 172.21.15.90:0/566685660 shutdown_connections 2024-05-27T07:47:56.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.183+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aebf212af0 0x55aebf2806a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.183+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aebf203190 0x55aebf205eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.183+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aebf068ec0 0x55aebf20b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.183+0000 7f4fb4838b80 1 --2- 172.21.15.90:0/566685660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aebf154a70 0x55aebf20b690 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.183+0000 7f4f8cc00700 0 client.15681 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:56.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.183+0000 7f4fb4838b80 1 -- 172.21.15.90:0/566685660 >> 172.21.15.90:0/566685660 conn(0x55aebf102410 msgr2=0x55aebf1e8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.183+0000 7f4fb4838b80 1 -- 172.21.15.90:0/566685660 shutdown_connections 2024-05-27T07:47:56.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.183+0000 7f4fb4838b80 1 -- 172.21.15.90:0/566685660 wait complete. 2024-05-27T07:47:56.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.184+0000 7f4fb4838b80 20 client.15681 trim_cache size 0 max 16384 2024-05-27T07:47:56.185 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:47:56.185 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:47:56.185 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "320faed9-091d-4550-978d-9a27c0388ee0", 2024-05-27T07:47:56.185 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:47:56.185 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:47:56.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.184+0000 7f4fa5c00700 1 -- 172.21.15.90:0/615707495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f840b2cd0 msgr2=0x7f4f840b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.185+0000 7f4fa5c00700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f840b2cd0 0x7f4f840b51e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4f9801e000 tx=0x7f4f98021000 comp rx=0 tx=0).stop 2024-05-27T07:47:56.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.185+0000 7f4fa5c00700 1 -- 172.21.15.90:0/615707495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fa0103260 msgr2=0x7f4fa01a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.185+0000 7f4fa5c00700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fa0103260 0x7f4fa01a1450 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000c00 tx=0x7f4f880057a0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.185+0000 7f4fa5c00700 1 -- 172.21.15.90:0/615707495 shutdown_connections 2024-05-27T07:47:56.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.185+0000 7f4fa5c00700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fa01086b0 0x7f4fa01a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.185+0000 7f4fa5c00700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fa0103260 0x7f4fa01a1450 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.185+0000 7f4fa5c00700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f840b2cd0 0x7f4f840b51e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.185+0000 7f4fa5c00700 1 --2- 172.21.15.90:0/615707495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa0103bb0 0x7f4fa01a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.185+0000 7f4fa5c00700 1 -- 172.21.15.90:0/615707495 >> 172.21.15.90:0/615707495 conn(0x7f4fa00fef20 msgr2=0x7f4fa0106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.185+0000 7f4fa5c00700 1 -- 172.21.15.90:0/615707495 shutdown_connections 2024-05-27T07:47:56.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:47:56.186+0000 7f4fa5c00700 1 -- 172.21.15.90:0/615707495 wait complete. 2024-05-27T07:47:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.304+0000 7fb06ea00640 1 -- 172.21.15.90:0/3589623206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb068104400 msgr2=0x7fb0681107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.304+0000 7fb06ea00640 1 --2- 172.21.15.90:0/3589623206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb068104400 0x7fb0681107d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb060009be0 tx=0x7fb06002a6c0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.304+0000 7fb06ea00640 1 -- 172.21.15.90:0/3589623206 shutdown_connections 2024-05-27T07:47:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.304+0000 7fb06ea00640 1 --2- 172.21.15.90:0/3589623206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb06810a150 0x7fb068103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.304+0000 7fb06ea00640 1 --2- 172.21.15.90:0/3589623206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb068104400 0x7fb0681107d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.305+0000 7fb06ea00640 1 --2- 172.21.15.90:0/3589623206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068109cb0 0x7fb068103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.305+0000 7fb06ea00640 1 -- 172.21.15.90:0/3589623206 >> 172.21.15.90:0/3589623206 conn(0x7fb0680ff480 msgr2=0x7fb0681018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.305+0000 7fb06ea00640 1 -- 172.21.15.90:0/3589623206 shutdown_connections 2024-05-27T07:47:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.305+0000 7fb06ea00640 1 -- 172.21.15.90:0/3589623206 wait complete. 2024-05-27T07:47:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.307+0000 7fb06ea00640 1 Processor -- start 2024-05-27T07:47:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.307+0000 7fb06ea00640 1 -- start start 2024-05-27T07:47:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.308+0000 7fb06ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068104400 0x7fb0681a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.308+0000 7fb06ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb068109cb0 0x7fb0681a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.308+0000 7fb06ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb06810a150 0x7fb06819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.308+0000 7fb06ea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb068113d50 con 0x7fb068109cb0 2024-05-27T07:47:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.308+0000 7fb06ea00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb068113bd0 con 0x7fb06810a150 2024-05-27T07:47:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.308+0000 7fb06ea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb068113ed0 con 0x7fb068104400 2024-05-27T07:47:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.309+0000 7fb067e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068104400 0x7fb0681a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.309+0000 7fb067e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068104400 0x7fb0681a2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33758/0 (socket says 172.21.15.90:33758) 2024-05-27T07:47:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.309+0000 7fb067e00640 1 -- 172.21.15.90:0/1760347035 learned_addr learned my addr 172.21.15.90:0/1760347035 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.309+0000 7fb067400640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb068109cb0 0x7fb0681a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.309+0000 7fb06ca00640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb06810a150 0x7fb06819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.309+0000 7fb067400640 1 -- 172.21.15.90:0/1760347035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068104400 msgr2=0x7fb0681a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.309+0000 7fb067400640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068104400 0x7fb0681a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.309+0000 7fb067400640 1 -- 172.21.15.90:0/1760347035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb06810a150 msgr2=0x7fb06819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.309+0000 7fb067400640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb06810a150 0x7fb06819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.309+0000 7fb067400640 1 -- 172.21.15.90:0/1760347035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06819ce20 con 0x7fb068109cb0 2024-05-27T07:47:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.310+0000 7fb067400640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb068109cb0 0x7fb0681a2ab0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb06806c530 tx=0x7fb05400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.310+0000 7fb064c00640 1 -- 172.21.15.90:0/1760347035 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05400cc90 con 0x7fb068109cb0 2024-05-27T07:47:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.310+0000 7fb06ea00640 1 -- 172.21.15.90:0/1760347035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06819d0b0 con 0x7fb068109cb0 2024-05-27T07:47:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.310+0000 7fb06ea00640 1 -- 172.21.15.90:0/1760347035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0681a9310 con 0x7fb068109cb0 2024-05-27T07:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.311+0000 7fb064c00640 1 -- 172.21.15.90:0/1760347035 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0540044b0 con 0x7fb068109cb0 2024-05-27T07:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.311+0000 7fb064c00640 1 -- 172.21.15.90:0/1760347035 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054003da0 con 0x7fb068109cb0 2024-05-27T07:47:56.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:56 smithi129 ceph-mon[35254]: pgmap v48: 129 pgs: 129 active+clean; 129 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 35 op/s 2024-05-27T07:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.316+0000 7fb075967740 1 Processor -- start 2024-05-27T07:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.316+0000 7fb075967740 1 -- start start 2024-05-27T07:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.316+0000 7fb075967740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 0x5578807d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.317+0000 7fb075967740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 0x5578807d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.317+0000 7fb075967740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807d52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.317+0000 7fb075967740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557880745b90 con 0x5578804321b0 2024-05-27T07:47:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.317+0000 7fb075967740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557880735730 con 0x5578804114d0 2024-05-27T07:47:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.317+0000 7fb075967740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557880739770 con 0x5578804092f0 2024-05-27T07:47:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.317+0000 7fb06ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 0x5578807d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.317+0000 7fb06ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 0x5578807d4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41182/0 (socket says 172.21.15.90:41182) 2024-05-27T07:47:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.317+0000 7fb06ca00640 1 -- 172.21.15.90:0/2478163891 learned_addr learned my addr 172.21.15.90:0/2478163891 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:56.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.318+0000 7fb067e00640 1 --2- 172.21.15.90:0/2478163891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 0x5578807d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.318+0000 7fb06ca00640 1 -- 172.21.15.90:0/2478163891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 msgr2=0x5578807d4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.318+0000 7fb064c00640 1 -- 172.21.15.90:0/1760347035 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7fb054018430 con 0x7fb068109cb0 2024-05-27T07:47:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.318+0000 7fb06ca00640 1 --2- 172.21.15.90:0/2478163891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 0x5578807d4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.319+0000 7fb06ca00640 1 -- 172.21.15.90:0/2478163891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 msgr2=0x5578807d52e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:47:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.319+0000 7fb06ca00640 1 --2- 172.21.15.90:0/2478163891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807d52e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.319+0000 7fb06ca00640 1 -- 172.21.15.90:0/2478163891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578806ce8b0 con 0x5578804321b0 2024-05-27T07:47:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.319+0000 7fb06ca00640 1 --2- 172.21.15.90:0/2478163891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 0x5578807d4860 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb060033000 tx=0x7fb060009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e96ffadfc7d499 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.320+0000 7fb05d200640 1 -- 172.21.15.90:0/2478163891 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060040030 con 0x5578804321b0 2024-05-27T07:47:56.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.320+0000 7fb05d200640 1 -- 172.21.15.90:0/2478163891 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb060037040 con 0x5578804321b0 2024-05-27T07:47:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.320+0000 7fb05d200640 1 -- 172.21.15.90:0/2478163891 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060036070 con 0x5578804321b0 2024-05-27T07:47:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.321+0000 7fb064c00640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb04c0b2b00 0x7fb04c0b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.321+0000 7fb064c00640 1 -- 172.21.15.90:0/1760347035 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fb0540e2e10 con 0x7fb068109cb0 2024-05-27T07:47:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.322+0000 7fb067e00640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb04c0b2b00 0x7fb04c0b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.322+0000 7fb075967740 1 -- 172.21.15.90:0/2478163891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 msgr2=0x5578807d4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.322+0000 7fb075967740 1 --2- 172.21.15.90:0/2478163891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 0x5578807d4860 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb060033000 tx=0x7fb060009310 comp rx=0 tx=0).stop 2024-05-27T07:47:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.323+0000 7fb075967740 1 -- 172.21.15.90:0/2478163891 shutdown_connections 2024-05-27T07:47:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.323+0000 7fb075967740 1 --2- 172.21.15.90:0/2478163891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807d52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.323+0000 7fb075967740 1 --2- 172.21.15.90:0/2478163891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 0x5578807d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.323+0000 7fb075967740 1 --2- 172.21.15.90:0/2478163891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 0x5578807d4860 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.323+0000 7fb075967740 1 -- 172.21.15.90:0/2478163891 >> 172.21.15.90:0/2478163891 conn(0x557880426300 msgr2=0x5578807d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.323+0000 7fb067e00640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb04c0b2b00 0x7fb04c0b5010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5578807d5c10 tx=0x7fb058005f90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.323+0000 7fb075967740 1 -- 172.21.15.90:0/2478163891 shutdown_connections 2024-05-27T07:47:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.323+0000 7fb075967740 1 -- 172.21.15.90:0/2478163891 wait complete. 2024-05-27T07:47:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.324+0000 7fb075967740 1 -- start start 2024-05-27T07:47:56.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.324+0000 7fb075967740 1 Processor -- start 2024-05-27T07:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.325+0000 7fb075967740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 0x5578807dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.325+0000 7fb075967740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807def80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.325+0000 7fb075967740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 0x5578807eacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.325+0000 7fb075967740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55788041b790 con 0x5578804321b0 2024-05-27T07:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.325+0000 7fb075967740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5578806ddbc0 con 0x5578804114d0 2024-05-27T07:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.325+0000 7fb075967740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5578807e1930 con 0x5578804092f0 2024-05-27T07:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.325+0000 7fb075967740 10 client.0 _ll_register_callbacks cb 0x5578806bb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.326+0000 7fb06ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 0x5578807eacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.326+0000 7fb067e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 0x5578807dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.326+0000 7fb067400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807def80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.326+0000 7fb067e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 0x5578807dea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33780/0 (socket says 172.21.15.90:33780) 2024-05-27T07:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.326+0000 7fb067e00640 1 -- 172.21.15.90:0/2098246951 learned_addr learned my addr 172.21.15.90:0/2098246951 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.326+0000 7fb067400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807def80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:35580/0 (socket says 172.21.15.90:35580) 2024-05-27T07:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.327+0000 7fb067400640 1 -- 172.21.15.90:0/2098246951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 msgr2=0x5578807dea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.327+0000 7fb067400640 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 0x5578807dea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.327+0000 7fb067400640 1 -- 172.21.15.90:0/2098246951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 msgr2=0x5578807eacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.327+0000 7fb067400640 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 0x5578807eacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.327+0000 7fb067400640 1 -- 172.21.15.90:0/2098246951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55788044a210 con 0x5578804114d0 2024-05-27T07:47:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.327+0000 7fb067400640 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807def80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb0540dffb0 tx=0x7fb0540a9370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e893e8bceae7bea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.327+0000 7fb051600640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:47:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.328+0000 7fb051600640 1 -- 172.21.15.90:0/2098246951 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540aa240 con 0x5578804114d0 2024-05-27T07:47:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.328+0000 7fb075967740 1 -- 172.21.15.90:0/2098246951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb060037040 con 0x5578804114d0 2024-05-27T07:47:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.328+0000 7fb051600640 1 -- 172.21.15.90:0/2098246951 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fb0540a8da0 con 0x5578804114d0 2024-05-27T07:47:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.329+0000 7fb051600640 1 -- 172.21.15.90:0/2098246951 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb0540a9b40 con 0x5578804114d0 2024-05-27T07:47:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.329+0000 7fb075967740 10 client.35342 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.329+0000 7fb075967740 1 -- 172.21.15.90:0/2098246951 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557880516620 con 0x5578804114d0 2024-05-27T07:47:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.329+0000 7fb051600640 1 -- 172.21.15.90:0/2098246951 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb060037040 con 0x5578804114d0 2024-05-27T07:47:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.330+0000 7fb051600640 1 -- 172.21.15.90:0/2098246951 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb0540a8810 con 0x5578804114d0 2024-05-27T07:47:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.330+0000 7fb075967740 10 client.35342 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.330+0000 7fb075967740 10 client.35342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.330+0000 7fb075967740 20 client.35342 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.330+0000 7fb075967740 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5578803f0d80 0x5578807f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.330+0000 7fb075967740 4 client.35342 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:47:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.330+0000 7fb075967740 1 -- 172.21.15.90:0/2098246951 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557880747ae0 con 0x5578803f0d80 2024-05-27T07:47:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.331+0000 7fb067e00640 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5578803f0d80 0x5578807f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.331+0000 7fb067e00640 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5578803f0d80 0x5578807f0d80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc7e30884f8bfff server_cookie=eda6beb405cd01f0 in_seq=0 out_seq=0 2024-05-27T07:47:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.331+0000 7fb051600640 10 client.35342 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb051600640 1 -- 172.21.15.90:0/2098246951 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557880747ae0 con 0x5578803f0d80 2024-05-27T07:47:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb051600640 10 client.35342 handle_command_reply: tid=0 2024-05-27T07:47:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb075967740 1 client.35342 shutdown 2024-05-27T07:47:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb075967740 1 -- 172.21.15.90:0/2098246951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 msgr2=0x5578807def80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb075967740 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807def80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb0540dffb0 tx=0x7fb0540a9370 comp rx=0 tx=0).stop 2024-05-27T07:47:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb075967740 1 -- 172.21.15.90:0/2098246951 shutdown_connections 2024-05-27T07:47:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb075967740 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5578803f0d80 0x5578807f0d80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb075967740 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807def80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb075967740 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804321b0 0x5578807eacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb075967740 1 --2- 172.21.15.90:0/2098246951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804092f0 0x5578807dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.333+0000 7fb075967740 1 -- 172.21.15.90:0/2098246951 >> 172.21.15.90:0/2098246951 conn(0x557880426300 msgr2=0x5578807d4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.334+0000 7fb051600640 0 client.35342 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.334+0000 7fb075967740 1 -- 172.21.15.90:0/2098246951 shutdown_connections 2024-05-27T07:47:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.334+0000 7fb075967740 1 -- 172.21.15.90:0/2098246951 wait complete. 2024-05-27T07:47:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.334+0000 7fb075967740 20 client.35342 trim_cache size 0 max 16384 2024-05-27T07:47:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.356+0000 7fb075967740 1 Processor -- start 2024-05-27T07:47:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.356+0000 7fb075967740 1 -- start start 2024-05-27T07:47:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.357+0000 7fb075967740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807def90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.357+0000 7fb075967740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804114d0 0x5578807d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.357+0000 7fb075967740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804321b0 0x5578807f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.357+0000 7fb075967740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb04800fda0 con 0x5578804092f0 2024-05-27T07:47:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.357+0000 7fb075967740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb0480118e0 con 0x5578804321b0 2024-05-27T07:47:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.357+0000 7fb075967740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb048010470 con 0x5578804114d0 2024-05-27T07:47:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.357+0000 7fb06ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804321b0 0x5578807f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.357+0000 7fb06ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804321b0 0x5578807f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:35592/0 (socket says 172.21.15.90:35592) 2024-05-27T07:47:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.357+0000 7fb06ca00640 1 -- 172.21.15.90:0/4030024919 learned_addr learned my addr 172.21.15.90:0/4030024919 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.357+0000 7fb067e00640 1 --2- 172.21.15.90:0/4030024919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807def90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.358+0000 7fb067400640 1 --2- 172.21.15.90:0/4030024919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804114d0 0x5578807d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.358+0000 7fb06ca00640 1 -- 172.21.15.90:0/4030024919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804114d0 msgr2=0x5578807d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.358+0000 7fb06ca00640 1 --2- 172.21.15.90:0/4030024919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804114d0 0x5578807d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.358+0000 7fb06ca00640 1 -- 172.21.15.90:0/4030024919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 msgr2=0x5578807def90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.358+0000 7fb06ca00640 1 --2- 172.21.15.90:0/4030024919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807def90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.358+0000 7fb06ca00640 1 -- 172.21.15.90:0/4030024919 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578807d5c40 con 0x5578804321b0 2024-05-27T07:47:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.358+0000 7fb067e00640 1 --2- 172.21.15.90:0/4030024919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807def90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:47:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.359+0000 7fb06ca00640 1 --2- 172.21.15.90:0/4030024919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804321b0 0x5578807f6580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb060009be0 tx=0x7fb060000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e59ebdba9afd052 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.359+0000 7fb053e00640 1 -- 172.21.15.90:0/4030024919 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060049070 con 0x5578804321b0 2024-05-27T07:47:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.359+0000 7fb053e00640 1 -- 172.21.15.90:0/4030024919 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb060003e00 con 0x5578804321b0 2024-05-27T07:47:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.359+0000 7fb053e00640 1 -- 172.21.15.90:0/4030024919 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060036070 con 0x5578804321b0 2024-05-27T07:47:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.360+0000 7fb075967740 1 -- 172.21.15.90:0/4030024919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804321b0 msgr2=0x5578807f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.360+0000 7fb075967740 1 --2- 172.21.15.90:0/4030024919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804321b0 0x5578807f6580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb060009be0 tx=0x7fb060000f40 comp rx=0 tx=0).stop 2024-05-27T07:47:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.361+0000 7fb075967740 1 -- 172.21.15.90:0/4030024919 shutdown_connections 2024-05-27T07:47:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.361+0000 7fb075967740 1 --2- 172.21.15.90:0/4030024919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804321b0 0x5578807f6580 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.361+0000 7fb075967740 1 --2- 172.21.15.90:0/4030024919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804114d0 0x5578807d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.361+0000 7fb075967740 1 --2- 172.21.15.90:0/4030024919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807def90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.361+0000 7fb075967740 1 -- 172.21.15.90:0/4030024919 >> 172.21.15.90:0/4030024919 conn(0x557880426300 msgr2=0x5578807e0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.362+0000 7fb075967740 1 -- 172.21.15.90:0/4030024919 shutdown_connections 2024-05-27T07:47:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.362+0000 7fb075967740 1 -- 172.21.15.90:0/4030024919 wait complete. 2024-05-27T07:47:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.362+0000 7fb075967740 1 -- start start 2024-05-27T07:47:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.363+0000 7fb075967740 1 Processor -- start 2024-05-27T07:47:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.363+0000 7fb075967740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807e7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.363+0000 7fb075967740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807e3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.364+0000 7fb067e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.364+0000 7fb075967740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804321b0 0x5578807e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.364+0000 7fb067e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807e7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41206/0 (socket says 172.21.15.90:41206) 2024-05-27T07:47:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.364+0000 7fb067e00640 1 -- 172.21.15.90:0/2235375539 learned_addr learned my addr 172.21.15.90:0/2235375539 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:47:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.364+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb048011210 con 0x5578804092f0 2024-05-27T07:47:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.364+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb048010b40 con 0x5578804114d0 2024-05-27T07:47:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.364+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb04800dd60 con 0x5578804321b0 2024-05-27T07:47:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.364+0000 7fb075967740 10 client.0 _ll_register_callbacks cb 0x7fb0540046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:47:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.364+0000 7fb067400640 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.365+0000 7fb067e00640 1 -- 172.21.15.90:0/2235375539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804321b0 msgr2=0x5578807e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.365+0000 7fb067e00640 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804321b0 0x5578807e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.365+0000 7fb067e00640 1 -- 172.21.15.90:0/2235375539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 msgr2=0x5578807e3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:47:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.365+0000 7fb067e00640 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807e3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.365+0000 7fb067e00640 1 -- 172.21.15.90:0/2235375539 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578807e44a0 con 0x5578804092f0 2024-05-27T07:47:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.365+0000 7fb067400640 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:47:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.365+0000 7fb067e00640 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807e7410 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x5578807f1520 tx=0x7fb058004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d00c794c87dee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:47:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.365+0000 7fb051600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:47:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.366+0000 7fb051600640 1 -- 172.21.15.90:0/2235375539 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058003b40 con 0x5578804092f0 2024-05-27T07:47:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.366+0000 7fb051600640 1 -- 172.21.15.90:0/2235375539 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fb058003ce0 con 0x5578804092f0 2024-05-27T07:47:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.366+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb060003e00 con 0x5578804092f0 2024-05-27T07:47:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.366+0000 7fb051600640 1 -- 172.21.15.90:0/2235375539 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb058037050 con 0x5578804092f0 2024-05-27T07:47:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.367+0000 7fb075967740 10 client.15696 fetch_fsmap learned FSMap version 32 2024-05-27T07:47:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.367+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557880858450 con 0x5578804092f0 2024-05-27T07:47:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.367+0000 7fb051600640 1 -- 172.21.15.90:0/2235375539 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb060003e00 con 0x5578804092f0 2024-05-27T07:47:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.367+0000 7fb051600640 1 -- 172.21.15.90:0/2235375539 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb058004b30 con 0x5578804092f0 2024-05-27T07:47:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.367+0000 7fb075967740 10 client.15696 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:47:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.367+0000 7fb075967740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:47:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.367+0000 7fb075967740 20 client.15696 populate_metadata read hostname 'smithi090' 2024-05-27T07:47:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.368+0000 7fb075967740 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5578803c61c0 0x55788085acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:47:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.368+0000 7fb075967740 4 client.15696 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:47:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.368+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578807412c0 con 0x5578803c61c0 2024-05-27T07:47:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.368+0000 7fb067400640 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5578803c61c0 0x55788085acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:47:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.369+0000 7fb067400640 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5578803c61c0 0x55788085acc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e686d157a03529f server_cookie=d4a8e192e442d81e in_seq=0 out_seq=0 2024-05-27T07:47:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.369+0000 7fb051600640 10 client.15696 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.429+0000 7fb051600640 1 -- 172.21.15.90:0/2235375539 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5578807412c0 con 0x5578803c61c0 2024-05-27T07:47:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.429+0000 7fb051600640 10 client.15696 handle_command_reply: tid=0 2024-05-27T07:47:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.429+0000 7fb075967740 1 client.15696 shutdown 2024-05-27T07:47:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.429+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 msgr2=0x5578807e7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.429+0000 7fb075967740 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807e7410 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x5578807f1520 tx=0x7fb058004730 comp rx=0 tx=0).stop 2024-05-27T07:47:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.430+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 shutdown_connections 2024-05-27T07:47:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.430+0000 7fb075967740 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5578803c61c0 0x55788085acc0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.430+0000 7fb075967740 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5578804114d0 0x5578807e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.430+0000 7fb075967740 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578804321b0 0x5578807e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.430+0000 7fb075967740 1 --2- 172.21.15.90:0/2235375539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578804092f0 0x5578807e7410 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.430+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 >> 172.21.15.90:0/2235375539 conn(0x557880426300 msgr2=0x5578807cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.430+0000 7fb051600640 0 client.15696 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:47:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.430+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 shutdown_connections 2024-05-27T07:47:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.430+0000 7fb075967740 1 -- 172.21.15.90:0/2235375539 wait complete. 2024-05-27T07:47:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.431+0000 7fb075967740 20 client.15696 trim_cache size 0 max 16384 2024-05-27T07:47:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:47:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:47:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:47:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 -- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb04c0b2b00 msgr2=0x7fb04c0b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb04c0b2b00 0x7fb04c0b5010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5578807d5c10 tx=0x7fb058005f90 comp rx=0 tx=0).stop 2024-05-27T07:47:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 -- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb068109cb0 msgr2=0x7fb0681a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:47:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb068109cb0 0x7fb0681a2ab0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb06806c530 tx=0x7fb05400ec20 comp rx=0 tx=0).stop 2024-05-27T07:47:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 -- 172.21.15.90:0/1760347035 shutdown_connections 2024-05-27T07:47:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb04c0b2b00 0x7fb04c0b5010 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb06810a150 0x7fb06819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb068109cb0 0x7fb0681a2ab0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 --2- 172.21.15.90:0/1760347035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068104400 0x7fb0681a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:47:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 -- 172.21.15.90:0/1760347035 >> 172.21.15.90:0/1760347035 conn(0x7fb0680ff480 msgr2=0x7fb0680753d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:47:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.432+0000 7fb06ea00640 1 -- 172.21.15.90:0/1760347035 shutdown_connections 2024-05-27T07:47:56.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:47:56.433+0000 7fb06ea00640 1 -- 172.21.15.90:0/1760347035 wait complete. 2024-05-27T07:47:56.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:56 smithi136 ceph-mon[35324]: pgmap v48: 129 pgs: 129 active+clean; 129 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 35 op/s 2024-05-27T07:47:57.261 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:47:57.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:57 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:47:57.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:57 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:57.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:57 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:47:57.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:57 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:57.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:57 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:47:57.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:57 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:57.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:57 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:47:57.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:57 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:57.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:57 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:47:57.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:57 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:57.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:57 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:47:57.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:57 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:47:57.766 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:47:57.804 INFO:teuthology.orchestra.run.smithi090.stderr:500000 of 3200000 tuples (15%) done (elapsed 13.86 s, remaining 74.83 s) 2024-05-27T07:47:58.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:58 smithi090 ceph-mon[31429]: pgmap v49: 129 pgs: 129 active+clean; 129 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 35 op/s 2024-05-27T07:47:58.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:58 smithi129 ceph-mon[35254]: pgmap v49: 129 pgs: 129 active+clean; 129 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 35 op/s 2024-05-27T07:47:58.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:58 smithi136 ceph-mon[35324]: pgmap v49: 129 pgs: 129 active+clean; 129 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 35 op/s 2024-05-27T07:47:59.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:47:59 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:47:59.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:47:59 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:47:59.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:47:59 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:48:00.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.032+0000 7fb529e00700 1 -- 172.21.15.90:0/1692179234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb52410b8b0 msgr2=0x7fb524113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:00.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.032+0000 7fb529e00700 1 --2- 172.21.15.90:0/1692179234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb52410b8b0 0x7fb524113210 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00b0d0 tx=0x7fb51c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.033+0000 7fb529e00700 1 -- 172.21.15.90:0/1692179234 shutdown_connections 2024-05-27T07:48:00.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.033+0000 7fb529e00700 1 --2- 172.21.15.90:0/1692179234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb52410b8b0 0x7fb524113210 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.033+0000 7fb529e00700 1 --2- 172.21.15.90:0/1692179234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb524106630 0x7fb524106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.033+0000 7fb529e00700 1 --2- 172.21.15.90:0/1692179234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb52410af60 0x7fb52410b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.033+0000 7fb529e00700 1 -- 172.21.15.90:0/1692179234 >> 172.21.15.90:0/1692179234 conn(0x7fb524101ec0 msgr2=0x7fb524104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:00.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.033+0000 7fb529e00700 1 -- 172.21.15.90:0/1692179234 shutdown_connections 2024-05-27T07:48:00.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.033+0000 7fb529e00700 1 -- 172.21.15.90:0/1692179234 wait complete. 2024-05-27T07:48:00.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.034+0000 7fb529e00700 1 Processor -- start 2024-05-27T07:48:00.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.034+0000 7fb529e00700 1 -- start start 2024-05-27T07:48:00.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb529e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb524106630 0x7fb5241a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb529e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb52410af60 0x7fb5241a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb529e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52410b8b0 0x7fb52419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb529e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5241163b0 con 0x7fb52410af60 2024-05-27T07:48:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb529e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb524116230 con 0x7fb52410b8b0 2024-05-27T07:48:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb529e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb524116530 con 0x7fb524106630 2024-05-27T07:48:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb523e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb52410af60 0x7fb5241a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb523e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb52410af60 0x7fb5241a2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55330/0 (socket says 172.21.15.90:55330) 2024-05-27T07:48:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb523e00700 1 -- 172.21.15.90:0/14293624 learned_addr learned my addr 172.21.15.90:0/14293624 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb528a00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb524106630 0x7fb5241a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.035+0000 7fb529400700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52410b8b0 0x7fb52419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.036+0000 7fb523e00700 1 -- 172.21.15.90:0/14293624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb524106630 msgr2=0x7fb5241a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.036+0000 7fb523e00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb524106630 0x7fb5241a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.036+0000 7fb523e00700 1 -- 172.21.15.90:0/14293624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52410b8b0 msgr2=0x7fb52419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.036+0000 7fb523e00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52410b8b0 0x7fb52419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.036+0000 7fb523e00700 1 -- 172.21.15.90:0/14293624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52419d160 con 0x7fb52410af60 2024-05-27T07:48:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.036+0000 7fb528a00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb524106630 0x7fb5241a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.036+0000 7fb529400700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52410b8b0 0x7fb52419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.036+0000 7fb523e00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb52410af60 0x7fb5241a2e10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c970 tx=0x7fb51000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.037+0000 7fb521600700 1 -- 172.21.15.90:0/14293624 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510007b00 con 0x7fb52410af60 2024-05-27T07:48:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.037+0000 7fb521600700 1 -- 172.21.15.90:0/14293624 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb510004cc0 con 0x7fb52410af60 2024-05-27T07:48:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.037+0000 7fb529e00700 1 -- 172.21.15.90:0/14293624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52419d450 con 0x7fb52410af60 2024-05-27T07:48:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.038+0000 7fb521600700 1 -- 172.21.15.90:0/14293624 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510005580 con 0x7fb52410af60 2024-05-27T07:48:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.038+0000 7fb529e00700 1 -- 172.21.15.90:0/14293624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5241a9760 con 0x7fb52410af60 2024-05-27T07:48:00.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.041+0000 7fb521600700 1 -- 172.21.15.90:0/14293624 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb510007ca0 con 0x7fb52410af60 2024-05-27T07:48:00.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.044+0000 7fb539fbcb80 1 Processor -- start 2024-05-27T07:48:00.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.044+0000 7fb521600700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 0x7fb50c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.044+0000 7fb539fbcb80 1 -- start start 2024-05-27T07:48:00.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.044+0000 7fb521600700 1 -- 172.21.15.90:0/14293624 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fb5100e2500 con 0x7fb52410af60 2024-05-27T07:48:00.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d7d4e40 0x55cf5d952ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953410 0x55cf5d95dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953820 0x55cf5d960020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d881e40 con 0x55cf5d953820 2024-05-27T07:48:00.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d8719e0 con 0x55cf5d7d4e40 2024-05-27T07:48:00.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d875a20 con 0x55cf5d953410 2024-05-27T07:48:00.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb529400700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 0x7fb50c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb529400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953820 0x55cf5d960020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb529400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953820 0x55cf5d960020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55336/0 (socket says 172.21.15.90:55336) 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb529400700 1 -- 172.21.15.90:0/923013764 learned_addr learned my addr 172.21.15.90:0/923013764 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb528a00700 1 --2- 172.21.15.90:0/923013764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d7d4e40 0x55cf5d952ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.045+0000 7fb523e00700 1 --2- 172.21.15.90:0/923013764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953410 0x55cf5d95dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.046+0000 7fb529400700 1 -- 172.21.15.90:0/923013764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953410 msgr2=0x55cf5d95dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.046+0000 7fb529400700 1 --2- 172.21.15.90:0/923013764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953410 0x55cf5d95dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.046+0000 7fb529400700 1 -- 172.21.15.90:0/923013764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d7d4e40 msgr2=0x55cf5d952ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.046+0000 7fb529400700 1 --2- 172.21.15.90:0/923013764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d7d4e40 0x55cf5d952ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.046+0000 7fb529400700 1 -- 172.21.15.90:0/923013764 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5d83a420 con 0x55cf5d953820 2024-05-27T07:48:00.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.046+0000 7fb529400700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 0x7fb50c0b4f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009900 tx=0x7fb51c009e00 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:00.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.046+0000 7fb529400700 1 --2- 172.21.15.90:0/923013764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953820 0x55cf5d960020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb51c007b30 tx=0x7fb51c026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856665f175c660b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:00.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.047+0000 7fb516000700 1 -- 172.21.15.90:0/923013764 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c009330 con 0x55cf5d953820 2024-05-27T07:48:00.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.047+0000 7fb516000700 1 -- 172.21.15.90:0/923013764 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb51c021770 con 0x55cf5d953820 2024-05-27T07:48:00.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.047+0000 7fb516000700 1 -- 172.21.15.90:0/923013764 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c01e6c0 con 0x55cf5d953820 2024-05-27T07:48:00.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.048+0000 7fb539fbcb80 1 -- 172.21.15.90:0/923013764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953820 msgr2=0x55cf5d960020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:00.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.048+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/923013764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953820 0x55cf5d960020 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb51c007b30 tx=0x7fb51c026000 comp rx=0 tx=0).stop 2024-05-27T07:48:00.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.048+0000 7fb539fbcb80 1 -- 172.21.15.90:0/923013764 shutdown_connections 2024-05-27T07:48:00.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.048+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/923013764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953820 0x55cf5d960020 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.048+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/923013764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d7d4e40 0x55cf5d952ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.048+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/923013764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953410 0x55cf5d95dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.048+0000 7fb539fbcb80 1 -- 172.21.15.90:0/923013764 >> 172.21.15.90:0/923013764 conn(0x55cf5d834fc0 msgr2=0x55cf5d94ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:00.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.048+0000 7fb539fbcb80 1 -- 172.21.15.90:0/923013764 shutdown_connections 2024-05-27T07:48:00.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.049+0000 7fb539fbcb80 1 -- 172.21.15.90:0/923013764 wait complete. 2024-05-27T07:48:00.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.049+0000 7fb539fbcb80 1 -- start start 2024-05-27T07:48:00.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.049+0000 7fb539fbcb80 1 Processor -- start 2024-05-27T07:48:00.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.050+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d95d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.050+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953410 0x55cf5d95da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.050+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953820 0x55cf5d95dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.050+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d777fa0 con 0x55cf5d7d4e40 2024-05-27T07:48:00.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.050+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d812760 con 0x55cf5d953820 2024-05-27T07:48:00.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.050+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cf5d960930 con 0x55cf5d953410 2024-05-27T07:48:00.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.050+0000 7fb539fbcb80 10 client.0 _ll_register_callbacks cb 0x55cf5d81bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.050+0000 7fb528a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d95d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.050+0000 7fb528a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d95d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55352/0 (socket says 172.21.15.90:55352) 2024-05-27T07:48:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.050+0000 7fb528a00700 1 -- 172.21.15.90:0/8672464 learned_addr learned my addr 172.21.15.90:0/8672464 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.051+0000 7fb529400700 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953820 0x55cf5d95dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.051+0000 7fb523e00700 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953410 0x55cf5d95da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.051+0000 7fb528a00700 1 -- 172.21.15.90:0/8672464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953410 msgr2=0x55cf5d95da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.051+0000 7fb528a00700 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953410 0x55cf5d95da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.051+0000 7fb528a00700 1 -- 172.21.15.90:0/8672464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953820 msgr2=0x55cf5d95dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.051+0000 7fb528a00700 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953820 0x55cf5d95dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.051+0000 7fb528a00700 1 -- 172.21.15.90:0/8672464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5d7c18d0 con 0x55cf5d7d4e40 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.051+0000 7fb529400700 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953820 0x55cf5d95dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.051+0000 7fb528a00700 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d95d550 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb518011a60 tx=0x7fb518011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fed61d3b3d6ebd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.051+0000 7fb50aa00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.052+0000 7fb50aa00700 1 -- 172.21.15.90:0/8672464 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518004460 con 0x55cf5d7d4e40 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.052+0000 7fb50aa00700 1 -- 172.21.15.90:0/8672464 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fb51800f8b0 con 0x55cf5d7d4e40 2024-05-27T07:48:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.052+0000 7fb539fbcb80 1 -- 172.21.15.90:0/8672464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb51c021770 con 0x55cf5d7d4e40 2024-05-27T07:48:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.052+0000 7fb50aa00700 1 -- 172.21.15.90:0/8672464 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb518023050 con 0x55cf5d7d4e40 2024-05-27T07:48:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.052+0000 7fb539fbcb80 10 client.15708 fetch_fsmap learned FSMap version 32 2024-05-27T07:48:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.052+0000 7fb539fbcb80 1 -- 172.21.15.90:0/8672464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cf5d842060 con 0x55cf5d7d4e40 2024-05-27T07:48:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.053+0000 7fb50aa00700 1 -- 172.21.15.90:0/8672464 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb51c021770 con 0x55cf5d7d4e40 2024-05-27T07:48:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.053+0000 7fb50aa00700 1 -- 172.21.15.90:0/8672464 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb518018020 con 0x55cf5d7d4e40 2024-05-27T07:48:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.053+0000 7fb539fbcb80 10 client.15708 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:48:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.053+0000 7fb539fbcb80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.053+0000 7fb539fbcb80 20 client.15708 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.053+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cf5d96d8d0 0x55cf5d96fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.053+0000 7fb539fbcb80 4 client.15708 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.053+0000 7fb539fbcb80 1 -- 172.21.15.90:0/8672464 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf5d883d90 con 0x55cf5d96d8d0 2024-05-27T07:48:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.054+0000 7fb523e00700 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cf5d96d8d0 0x55cf5d96fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.055+0000 7fb50aa00700 10 client.15708 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:00.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.055+0000 7fb523e00700 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cf5d96d8d0 0x55cf5d96fd10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b1885c401b2241 server_cookie=e276f2de54ad889a in_seq=0 out_seq=0 2024-05-27T07:48:00.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb50aa00700 1 -- 172.21.15.90:0/8672464 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cf5d883d90 con 0x55cf5d96d8d0 2024-05-27T07:48:00.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb50aa00700 10 client.15708 handle_command_reply: tid=0 2024-05-27T07:48:00.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb539fbcb80 1 client.15708 shutdown 2024-05-27T07:48:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb539fbcb80 1 -- 172.21.15.90:0/8672464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 msgr2=0x55cf5d95d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d95d550 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb518011a60 tx=0x7fb518011e20 comp rx=0 tx=0).stop 2024-05-27T07:48:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb539fbcb80 1 -- 172.21.15.90:0/8672464 shutdown_connections 2024-05-27T07:48:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cf5d96d8d0 0x55cf5d96fd10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953820 0x55cf5d95dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d95d550 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/8672464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953410 0x55cf5d95da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb50aa00700 0 client.15708 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.057+0000 7fb539fbcb80 1 -- 172.21.15.90:0/8672464 >> 172.21.15.90:0/8672464 conn(0x55cf5d834fc0 msgr2=0x55cf5d94dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.058+0000 7fb539fbcb80 1 -- 172.21.15.90:0/8672464 shutdown_connections 2024-05-27T07:48:00.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.058+0000 7fb539fbcb80 1 -- 172.21.15.90:0/8672464 wait complete. 2024-05-27T07:48:00.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.058+0000 7fb539fbcb80 20 client.15708 trim_cache size 0 max 16384 2024-05-27T07:48:00.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.084+0000 7fb539fbcb80 1 Processor -- start 2024-05-27T07:48:00.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.084+0000 7fb539fbcb80 1 -- start start 2024-05-27T07:48:00.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.085+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d7d4e40 0x55cf5d95f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.085+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953410 0x55cf5d9699d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.085+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953820 0x55cf5d9754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.085+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5241b23f0 con 0x55cf5d953410 2024-05-27T07:48:00.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.085+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5241b3f30 con 0x55cf5d7d4e40 2024-05-27T07:48:00.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.085+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb5241b2ac0 con 0x55cf5d953820 2024-05-27T07:48:00.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.085+0000 7fb523e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953410 0x55cf5d9699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.085+0000 7fb523e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953410 0x55cf5d9699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55360/0 (socket says 172.21.15.90:55360) 2024-05-27T07:48:00.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.085+0000 7fb523e00700 1 -- 172.21.15.90:0/3052246211 learned_addr learned my addr 172.21.15.90:0/3052246211 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.085+0000 7fb528a00700 1 --2- 172.21.15.90:0/3052246211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d7d4e40 0x55cf5d95f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.086+0000 7fb529400700 1 --2- 172.21.15.90:0/3052246211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953820 0x55cf5d9754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.086+0000 7fb523e00700 1 -- 172.21.15.90:0/3052246211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953820 msgr2=0x55cf5d9754d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.086+0000 7fb523e00700 1 --2- 172.21.15.90:0/3052246211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953820 0x55cf5d9754d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.086+0000 7fb523e00700 1 -- 172.21.15.90:0/3052246211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d7d4e40 msgr2=0x55cf5d95f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.086+0000 7fb523e00700 1 --2- 172.21.15.90:0/3052246211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d7d4e40 0x55cf5d95f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.086+0000 7fb523e00700 1 -- 172.21.15.90:0/3052246211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5d9491d0 con 0x55cf5d953410 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.086+0000 7fb523e00700 1 --2- 172.21.15.90:0/3052246211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953410 0x55cf5d9699d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb5100df900 tx=0x7fb510091050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b32fbe073a2148a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.087+0000 7fb515600700 1 -- 172.21.15.90:0/3052246211 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100c2070 con 0x55cf5d953410 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.087+0000 7fb515600700 1 -- 172.21.15.90:0/3052246211 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5100a9430 con 0x55cf5d953410 2024-05-27T07:48:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.087+0000 7fb515600700 1 -- 172.21.15.90:0/3052246211 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51001d040 con 0x55cf5d953410 2024-05-27T07:48:00.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.088+0000 7fb539fbcb80 1 -- 172.21.15.90:0/3052246211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953410 msgr2=0x55cf5d9699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:00.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.088+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/3052246211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953410 0x55cf5d9699d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb5100df900 tx=0x7fb510091050 comp rx=0 tx=0).stop 2024-05-27T07:48:00.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.088+0000 7fb539fbcb80 1 -- 172.21.15.90:0/3052246211 shutdown_connections 2024-05-27T07:48:00.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.088+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/3052246211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953820 0x55cf5d9754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.088+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/3052246211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d7d4e40 0x55cf5d95f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.088+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/3052246211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d953410 0x55cf5d9699d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.088+0000 7fb539fbcb80 1 -- 172.21.15.90:0/3052246211 >> 172.21.15.90:0/3052246211 conn(0x55cf5d834fc0 msgr2=0x55cf5d95dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:00.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.088+0000 7fb539fbcb80 1 -- 172.21.15.90:0/3052246211 shutdown_connections 2024-05-27T07:48:00.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.088+0000 7fb539fbcb80 1 -- 172.21.15.90:0/3052246211 wait complete. 2024-05-27T07:48:00.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.089+0000 7fb539fbcb80 1 -- start start 2024-05-27T07:48:00.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.089+0000 7fb539fbcb80 1 Processor -- start 2024-05-27T07:48:00.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.089+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d961d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.089+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953410 0x55cf5d962240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.089+0000 7fb539fbcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953820 0x55cf5d9d73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.089+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5241b3860 con 0x55cf5d7d4e40 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.089+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5241b3190 con 0x55cf5d953410 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.089+0000 7fb539fbcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb5241b0370 con 0x55cf5d953820 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.089+0000 7fb539fbcb80 10 client.0 _ll_register_callbacks cb 0x55cf5d81bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.090+0000 7fb528a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d961d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.090+0000 7fb528a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d961d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55374/0 (socket says 172.21.15.90:55374) 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.090+0000 7fb528a00700 1 -- 172.21.15.90:0/2892970881 learned_addr learned my addr 172.21.15.90:0/2892970881 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.090+0000 7fb523e00700 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953410 0x55cf5d962240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.090+0000 7fb529400700 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953820 0x55cf5d9d73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.090+0000 7fb528a00700 1 -- 172.21.15.90:0/2892970881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953820 msgr2=0x55cf5d9d73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.090+0000 7fb528a00700 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953820 0x55cf5d9d73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.090+0000 7fb528a00700 1 -- 172.21.15.90:0/2892970881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953410 msgr2=0x55cf5d962240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.090+0000 7fb528a00700 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953410 0x55cf5d962240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.090+0000 7fb528a00700 1 -- 172.21.15.90:0/2892970881 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5d950350 con 0x55cf5d7d4e40 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.091+0000 7fb528a00700 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d961d00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb518008e80 tx=0x7fb518003a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15adcf73b566ab6c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.091+0000 7fb50aa00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.091+0000 7fb50aa00700 1 -- 172.21.15.90:0/2892970881 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51802b070 con 0x55cf5d7d4e40 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.091+0000 7fb50aa00700 1 -- 172.21.15.90:0/2892970881 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fb518005510 con 0x55cf5d7d4e40 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.091+0000 7fb539fbcb80 1 -- 172.21.15.90:0/2892970881 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5100a9430 con 0x55cf5d7d4e40 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.091+0000 7fb50aa00700 1 -- 172.21.15.90:0/2892970881 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb518038050 con 0x55cf5d7d4e40 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.091+0000 7fb539fbcb80 10 client.15714 fetch_fsmap learned FSMap version 32 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.091+0000 7fb539fbcb80 1 -- 172.21.15.90:0/2892970881 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cf5d9d8840 con 0x55cf5d7d4e40 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.092+0000 7fb50aa00700 1 -- 172.21.15.90:0/2892970881 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5100a9430 con 0x55cf5d7d4e40 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.092+0000 7fb50aa00700 1 -- 172.21.15.90:0/2892970881 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fb518018040 con 0x55cf5d7d4e40 2024-05-27T07:48:00.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.092+0000 7fb539fbcb80 10 client.15714 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:48:00.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.092+0000 7fb539fbcb80 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:00.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.092+0000 7fb539fbcb80 20 client.15714 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:00.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.092+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cf5d9d8f70 0x55cf5d9db390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:00.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.092+0000 7fb539fbcb80 4 client.15714 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:48:00.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.092+0000 7fb539fbcb80 1 -- 172.21.15.90:0/2892970881 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf5d87d570 con 0x55cf5d9d8f70 2024-05-27T07:48:00.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.093+0000 7fb523e00700 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cf5d9d8f70 0x55cf5d9db390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:00.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.094+0000 7fb523e00700 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cf5d9d8f70 0x55cf5d9db390 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3464bed8b2fe45b server_cookie=3dc36ae0da2ecab1 in_seq=0 out_seq=0 2024-05-27T07:48:00.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.094+0000 7fb50aa00700 10 client.15714 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:00.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.094+0000 7fb50aa00700 1 -- 172.21.15.90:0/2892970881 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cf5d87d570 con 0x55cf5d9d8f70 2024-05-27T07:48:00.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.094+0000 7fb50aa00700 10 client.15714 handle_command_reply: tid=0 2024-05-27T07:48:00.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 client.15714 shutdown 2024-05-27T07:48:00.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 -- 172.21.15.90:0/2892970881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 msgr2=0x55cf5d961d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:00.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d961d00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb518008e80 tx=0x7fb518003a60 comp rx=0 tx=0).stop 2024-05-27T07:48:00.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 -- 172.21.15.90:0/2892970881 shutdown_connections 2024-05-27T07:48:00.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cf5d9d8f70 0x55cf5d9db390 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf5d953820 0x55cf5d9d73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cf5d7d4e40 0x55cf5d961d00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 --2- 172.21.15.90:0/2892970881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf5d953410 0x55cf5d962240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 -- 172.21.15.90:0/2892970881 >> 172.21.15.90:0/2892970881 conn(0x55cf5d834fc0 msgr2=0x55cf5d95dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:00.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb50aa00700 0 client.15714 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:00.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 -- 172.21.15.90:0/2892970881 shutdown_connections 2024-05-27T07:48:00.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.095+0000 7fb539fbcb80 1 -- 172.21.15.90:0/2892970881 wait complete. 2024-05-27T07:48:00.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.096+0000 7fb539fbcb80 20 client.15714 trim_cache size 0 max 16384 2024-05-27T07:48:00.097 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:48:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:48:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:48:00.098 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:48:00.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.096+0000 7fb529e00700 1 -- 172.21.15.90:0/14293624 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 msgr2=0x7fb50c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:00.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.096+0000 7fb529e00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 0x7fb50c0b4f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009900 tx=0x7fb51c009e00 comp rx=0 tx=0).stop 2024-05-27T07:48:00.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.096+0000 7fb529e00700 1 -- 172.21.15.90:0/14293624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb52410af60 msgr2=0x7fb5241a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:00.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.096+0000 7fb529e00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb52410af60 0x7fb5241a2e10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c970 tx=0x7fb51000cd30 comp rx=0 tx=0).stop 2024-05-27T07:48:00.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.097+0000 7fb529e00700 1 -- 172.21.15.90:0/14293624 shutdown_connections 2024-05-27T07:48:00.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.097+0000 7fb529e00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 0x7fb50c0b4f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.097+0000 7fb529e00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb52410b8b0 0x7fb52419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.097+0000 7fb529e00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb524106630 0x7fb5241a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.097+0000 7fb529e00700 1 --2- 172.21.15.90:0/14293624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb52410af60 0x7fb5241a2e10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:00.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.097+0000 7fb529e00700 1 -- 172.21.15.90:0/14293624 >> 172.21.15.90:0/14293624 conn(0x7fb524101ec0 msgr2=0x7fb52410a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:00.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.097+0000 7fb529e00700 1 -- 172.21.15.90:0/14293624 shutdown_connections 2024-05-27T07:48:00.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:00.097+0000 7fb529e00700 1 -- 172.21.15.90:0/14293624 wait complete. 2024-05-27T07:48:00.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:00 smithi090 ceph-mon[31429]: pgmap v50: 129 pgs: 129 active+clean; 167 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 17 MiB/s wr, 138 op/s 2024-05-27T07:48:00.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:00 smithi090 ceph-mon[31429]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T07:48:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:00 smithi136 ceph-mon[35324]: pgmap v50: 129 pgs: 129 active+clean; 167 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 17 MiB/s wr, 138 op/s 2024-05-27T07:48:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:00 smithi136 ceph-mon[35324]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T07:48:00.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:00 smithi129 ceph-mon[35254]: pgmap v50: 129 pgs: 129 active+clean; 167 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 17 MiB/s wr, 138 op/s 2024-05-27T07:48:00.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:00 smithi129 ceph-mon[35254]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T07:48:00.692 INFO:teuthology.orchestra.run.smithi090.stderr:600000 of 3200000 tuples (18%) done (elapsed 16.74 s, remaining 72.56 s) 2024-05-27T07:48:01.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:48:01.052 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:48:01.526 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:02.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:02 smithi136 ceph-mon[35324]: pgmap v51: 129 pgs: 129 active+clean; 167 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 12 MiB/s wr, 112 op/s 2024-05-27T07:48:02.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fe845200700 time 2024-05-27T07:48:01.856041+0000 2024-05-27T07:48:02.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:02.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:02.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fe85ac7bba6] 2024-05-27T07:48:02.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe85ac7bdd1] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e328a1523d] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55e328a00971] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 5: (SnapRealm::check_cache() const+0x13e) [0x55e3289fefa0] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e3289ff651] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e328994347] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e32869087e] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e328695e94] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55e3286a0969] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 11: (MDLog::_replay_thread()+0x1f38) [0x55e328a3948c] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55e328a429cb] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 13: (Thread::entry_wrapper()+0x43) [0x7fe85ac4a72d] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 14: (Thread::_entry_func(void*)+0xd) [0x7fe85ac4a749] 2024-05-27T07:48:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe8597ee1ca] 2024-05-27T07:48:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 16: clone() 2024-05-27T07:48:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: *** Caught signal (Aborted) ** 2024-05-27T07:48:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: in thread 7fe845200700 thread_name:md_log_replay 2024-05-27T07:48:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:01.858+0000 7fe845200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fe845200700 time 2024-05-27T07:48:01.856041+0000 2024-05-27T07:48:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fe85ac7bba6] 2024-05-27T07:48:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe85ac7bdd1] 2024-05-27T07:48:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e328a1523d] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55e328a00971] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 5: (SnapRealm::check_cache() const+0x13e) [0x55e3289fefa0] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e3289ff651] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e328994347] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e32869087e] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e328695e94] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55e3286a0969] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 11: (MDLog::_replay_thread()+0x1f38) [0x55e328a3948c] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55e328a429cb] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 13: (Thread::entry_wrapper()+0x43) [0x7fe85ac4a72d] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 14: (Thread::_entry_func(void*)+0xd) [0x7fe85ac4a749] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe8597ee1ca] 2024-05-27T07:48:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 16: clone() 2024-05-27T07:48:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe8597f8d20] 2024-05-27T07:48:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2: gsignal() 2024-05-27T07:48:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 3: abort() 2024-05-27T07:48:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fe85ac7bced] 2024-05-27T07:48:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe85ac7bdd1] 2024-05-27T07:48:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e328a1523d] 2024-05-27T07:48:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55e328a00971] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 8: (SnapRealm::check_cache() const+0x13e) [0x55e3289fefa0] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e3289ff651] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e328994347] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e32869087e] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e328695e94] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55e3286a0969] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 14: (MDLog::_replay_thread()+0x1f38) [0x55e328a3948c] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55e328a429cb] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 16: (Thread::entry_wrapper()+0x43) [0x7fe85ac4a72d] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 17: (Thread::_entry_func(void*)+0xd) [0x7fe85ac4a749] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe8597ee1ca] 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 19: clone() 2024-05-27T07:48:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:01.862+0000 7fe845200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: in thread 7fe845200700 thread_name:md_log_replay 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe8597f8d20] 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2: gsignal() 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 3: abort() 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fe85ac7bced] 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe85ac7bdd1] 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e328a1523d] 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55e328a00971] 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 8: (SnapRealm::check_cache() const+0x13e) [0x55e3289fefa0] 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e3289ff651] 2024-05-27T07:48:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e328994347] 2024-05-27T07:48:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e32869087e] 2024-05-27T07:48:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e328695e94] 2024-05-27T07:48:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55e3286a0969] 2024-05-27T07:48:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 14: (MDLog::_replay_thread()+0x1f38) [0x55e328a3948c] 2024-05-27T07:48:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55e328a429cb] 2024-05-27T07:48:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 16: (Thread::entry_wrapper()+0x43) [0x7fe85ac4a72d] 2024-05-27T07:48:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 17: (Thread::_entry_func(void*)+0xd) [0x7fe85ac4a749] 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe8597ee1ca] 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 19: clone() 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: -1> 2024-05-27T07:48:01.858+0000 7fe845200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fe845200700 time 2024-05-27T07:48:01.856041+0000 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fe85ac7bba6] 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe85ac7bdd1] 2024-05-27T07:48:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e328a1523d] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55e328a00971] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 5: (SnapRealm::check_cache() const+0x13e) [0x55e3289fefa0] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e3289ff651] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e328994347] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e32869087e] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e328695e94] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55e3286a0969] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 11: (MDLog::_replay_thread()+0x1f38) [0x55e328a3948c] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55e328a429cb] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 13: (Thread::entry_wrapper()+0x43) [0x7fe85ac4a72d] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 14: (Thread::_entry_func(void*)+0xd) [0x7fe85ac4a749] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe8597ee1ca] 2024-05-27T07:48:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 16: clone() 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 0> 2024-05-27T07:48:01.862+0000 7fe845200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: in thread 7fe845200700 thread_name:md_log_replay 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe8597f8d20] 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2: gsignal() 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 3: abort() 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fe85ac7bced] 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe85ac7bdd1] 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e328a1523d] 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55e328a00971] 2024-05-27T07:48:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 8: (SnapRealm::check_cache() const+0x13e) [0x55e3289fefa0] 2024-05-27T07:48:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e3289ff651] 2024-05-27T07:48:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e328994347] 2024-05-27T07:48:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e32869087e] 2024-05-27T07:48:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e328695e94] 2024-05-27T07:48:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55e3286a0969] 2024-05-27T07:48:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 14: (MDLog::_replay_thread()+0x1f38) [0x55e328a3948c] 2024-05-27T07:48:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55e328a429cb] 2024-05-27T07:48:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 16: (Thread::entry_wrapper()+0x43) [0x7fe85ac4a72d] 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 17: (Thread::_entry_func(void*)+0xd) [0x7fe85ac4a749] 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe8597ee1ca] 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 19: clone() 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: -1> 2024-05-27T07:48:01.858+0000 7fe845200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fe845200700 time 2024-05-27T07:48:01.856041+0000 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fe85ac7bba6] 2024-05-27T07:48:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe85ac7bdd1] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e328a1523d] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55e328a00971] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 5: (SnapRealm::check_cache() const+0x13e) [0x55e3289fefa0] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e3289ff651] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e328994347] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e32869087e] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e328695e94] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55e3286a0969] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 11: (MDLog::_replay_thread()+0x1f38) [0x55e328a3948c] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55e328a429cb] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 13: (Thread::entry_wrapper()+0x43) [0x7fe85ac4a72d] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 14: (Thread::_entry_func(void*)+0xd) [0x7fe85ac4a749] 2024-05-27T07:48:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe8597ee1ca] 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 16: clone() 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 0> 2024-05-27T07:48:01.862+0000 7fe845200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: in thread 7fe845200700 thread_name:md_log_replay 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe8597f8d20] 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2: gsignal() 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 3: abort() 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fe85ac7bced] 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fe85ac7bdd1] 2024-05-27T07:48:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55e328a1523d] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55e328a00971] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 8: (SnapRealm::check_cache() const+0x13e) [0x55e3289fefa0] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55e3289ff651] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55e328994347] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55e32869087e] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55e328695e94] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55e3286a0969] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 14: (MDLog::_replay_thread()+0x1f38) [0x55e328a3948c] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55e328a429cb] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 16: (Thread::entry_wrapper()+0x43) [0x7fe85ac4a72d] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 17: (Thread::_entry_func(void*)+0xd) [0x7fe85ac4a749] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fe8597ee1ca] 2024-05-27T07:48:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 19: clone() 2024-05-27T07:48:02.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:48:02.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70361]: 2024-05-27T07:48:02.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:01 smithi136 podman[70561]: 2024-05-27 07:48:01.995212368 +0000 UTC m=+0.034268023 container died 1a38beffc215fc713acdf281c518e8113f882c927ba680cda57773656a282a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:48:02.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:02 smithi090 ceph-mon[31429]: pgmap v51: 129 pgs: 129 active+clean; 167 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 12 MiB/s wr, 112 op/s 2024-05-27T07:48:02.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:02 smithi129 ceph-mon[35254]: pgmap v51: 129 pgs: 129 active+clean; 167 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 12 MiB/s wr, 112 op/s 2024-05-27T07:48:02.999 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:02 smithi136 podman[70561]: 2024-05-27 07:48:02.692258246 +0000 UTC m=+0.731313920 container remove 1a38beffc215fc713acdf281c518e8113f882c927ba680cda57773656a282a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T07:48:02.999 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:02 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:48:03.749 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:03 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:48:03.870 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:03 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 2.753s CPU time. 2024-05-27T07:48:03.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.968+0000 7fa8b3a00700 1 -- 172.21.15.90:0/836634727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8ac1045e0 msgr2=0x7fa8ac1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:03.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.968+0000 7fa8b3a00700 1 --2- 172.21.15.90:0/836634727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8ac1045e0 0x7fa8ac1049f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00b0d0 tx=0x7fa89c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:48:03.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.968+0000 7fa8b3a00700 1 -- 172.21.15.90:0/836634727 shutdown_connections 2024-05-27T07:48:03.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.968+0000 7fa8b3a00700 1 --2- 172.21.15.90:0/836634727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8ac109860 0x7fa8ac1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:03.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.968+0000 7fa8b3a00700 1 --2- 172.21.15.90:0/836634727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8ac1045e0 0x7fa8ac1049f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:03.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.968+0000 7fa8b3a00700 1 --2- 172.21.15.90:0/836634727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac108f10 0x7fa8ac109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:03.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.968+0000 7fa8b3a00700 1 -- 172.21.15.90:0/836634727 >> 172.21.15.90:0/836634727 conn(0x7fa8ac100030 msgr2=0x7fa8ac102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:03.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.968+0000 7fa8b3a00700 1 -- 172.21.15.90:0/836634727 shutdown_connections 2024-05-27T07:48:03.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.968+0000 7fa8b3a00700 1 -- 172.21.15.90:0/836634727 wait complete. 2024-05-27T07:48:03.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.969+0000 7fa8b3a00700 1 Processor -- start 2024-05-27T07:48:03.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.970+0000 7fa8b3a00700 1 -- start start 2024-05-27T07:48:03.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.970+0000 7fa8b3a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac1045e0 0x7fa8ac075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:03.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.970+0000 7fa8b3a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8ac108f10 0x7fa8ac075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:03.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.970+0000 7fa8b3a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8ac109860 0x7fa8ac076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:03.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.970+0000 7fa8b3a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac114340 con 0x7fa8ac109860 2024-05-27T07:48:03.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.970+0000 7fa8b3a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1141c0 con 0x7fa8ac1045e0 2024-05-27T07:48:03.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.970+0000 7fa8b3a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1144c0 con 0x7fa8ac108f10 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.971+0000 7fa8b1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8ac109860 0x7fa8ac076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.971+0000 7fa8b1a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8ac109860 0x7fa8ac076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55404/0 (socket says 172.21.15.90:55404) 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.971+0000 7fa8b1a00700 1 -- 172.21.15.90:0/2417928289 learned_addr learned my addr 172.21.15.90:0/2417928289 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.971+0000 7fa8b1000700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac1045e0 0x7fa8ac075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.971+0000 7fa8a3e00700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8ac108f10 0x7fa8ac075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.971+0000 7fa8b1a00700 1 -- 172.21.15.90:0/2417928289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8ac108f10 msgr2=0x7fa8ac075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.971+0000 7fa8b1a00700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8ac108f10 0x7fa8ac075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.971+0000 7fa8b1a00700 1 -- 172.21.15.90:0/2417928289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac1045e0 msgr2=0x7fa8ac075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.971+0000 7fa8b1a00700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac1045e0 0x7fa8ac075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.971+0000 7fa8b1a00700 1 -- 172.21.15.90:0/2417928289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ac1a90f0 con 0x7fa8ac109860 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.972+0000 7fa8b1a00700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8ac109860 0x7fa8ac076530 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400d980 tx=0x7fa8a400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:03.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.972+0000 7fa8aa000700 1 -- 172.21.15.90:0/2417928289 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4017070 con 0x7fa8ac109860 2024-05-27T07:48:03.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.972+0000 7fa8aa000700 1 -- 172.21.15.90:0/2417928289 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8a4009dd0 con 0x7fa8ac109860 2024-05-27T07:48:03.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.972+0000 7fa8b3a00700 1 -- 172.21.15.90:0/2417928289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ac1a93e0 con 0x7fa8ac109860 2024-05-27T07:48:03.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.972+0000 7fa8aa000700 1 -- 172.21.15.90:0/2417928289 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4005260 con 0x7fa8ac109860 2024-05-27T07:48:03.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.972+0000 7fa8b3a00700 1 -- 172.21.15.90:0/2417928289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ac1a9920 con 0x7fa8ac109860 2024-05-27T07:48:03.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.974+0000 7fa8b3a00700 1 -- 172.21.15.90:0/2417928289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ac0512b0 con 0x7fa8ac109860 2024-05-27T07:48:03.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.976+0000 7fa8aa000700 1 -- 172.21.15.90:0/2417928289 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa8a40054e0 con 0x7fa8ac109860 2024-05-27T07:48:03.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.978+0000 7fa8aa000700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8900b2a70 0x7fa8900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:03.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.978+0000 7fa8aa000700 1 -- 172.21.15.90:0/2417928289 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fa8a40e28d0 con 0x7fa8ac109860 2024-05-27T07:48:03.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.979+0000 7fa8b1000700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8900b2a70 0x7fa8900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:03.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.979+0000 7fa8b1000700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8900b2a70 0x7fa8900b4f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009fd0 tx=0x7fa89c00b040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:03.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:03.984+0000 7fa8aa000700 1 -- 172.21.15.90:0/2417928289 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8a4020020 con 0x7fa8ac109860 2024-05-27T07:48:04.087 INFO:teuthology.orchestra.run.smithi090.stderr:700000 of 3200000 tuples (21%) done (elapsed 20.14 s, remaining 71.92 s) 2024-05-27T07:48:04.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.305+0000 7fa8b3a00700 1 -- 172.21.15.90:0/2417928289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8ac02e5e0 con 0x7fa8ac109860 2024-05-27T07:48:04.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:04 smithi090 ceph-mon[31429]: pgmap v52: 129 pgs: 129 active+clean; 207 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 18 MiB/s wr, 136 op/s 2024-05-27T07:48:04.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.307+0000 7fa8aa000700 1 -- 172.21.15.90:0/2417928289 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11938 (secure 0 0 0) 0x7fa8a40a9f40 con 0x7fa8ac109860 2024-05-27T07:48:04.307 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:48:04.307 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:47:18.105261+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25213":{"gid":25213,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/504128634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":504128634},{"type":"v1","addr":"172.21.15.136:6837","nonce":504128634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.310+0000 7fa8b3a00700 1 -- 172.21.15.90:0/2417928289 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8900b2a70 msgr2=0x7fa8900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.310+0000 7fa8b3a00700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8900b2a70 0x7fa8900b4f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009fd0 tx=0x7fa89c00b040 comp rx=0 tx=0).stop 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.310+0000 7fa8b3a00700 1 -- 172.21.15.90:0/2417928289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8ac109860 msgr2=0x7fa8ac076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.310+0000 7fa8b3a00700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8ac109860 0x7fa8ac076530 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400d980 tx=0x7fa8a400dd40 comp rx=0 tx=0).stop 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.311+0000 7fa8b3a00700 1 -- 172.21.15.90:0/2417928289 shutdown_connections 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.311+0000 7fa8b3a00700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8ac109860 0x7fa8ac076530 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.311+0000 7fa8b3a00700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8900b2a70 0x7fa8900b4f80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.311+0000 7fa8b3a00700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8ac1045e0 0x7fa8ac075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.311+0000 7fa8b3a00700 1 --2- 172.21.15.90:0/2417928289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8ac108f10 0x7fa8ac075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.311+0000 7fa8b3a00700 1 -- 172.21.15.90:0/2417928289 >> 172.21.15.90:0/2417928289 conn(0x7fa8ac100030 msgr2=0x7fa8ac102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:04.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.311+0000 7fa8b3a00700 1 -- 172.21.15.90:0/2417928289 shutdown_connections 2024-05-27T07:48:04.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:04.312+0000 7fa8b3a00700 1 -- 172.21.15.90:0/2417928289 wait complete. 2024-05-27T07:48:04.316 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 32 2024-05-27T07:48:04.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:04 smithi136 ceph-mon[35324]: pgmap v52: 129 pgs: 129 active+clean; 207 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 18 MiB/s wr, 136 op/s 2024-05-27T07:48:04.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:04 smithi129 ceph-mon[35254]: pgmap v52: 129 pgs: 129 active+clean; 207 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 18 MiB/s wr, 136 op/s 2024-05-27T07:48:05.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:48:05.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:05 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2417928289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:48:05.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:05 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2417928289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:48:05.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:05 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2417928289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:48:05.671 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:06.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:06 smithi136 ceph-mon[35324]: pgmap v53: 129 pgs: 129 active+clean; 207 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 13 MiB/s wr, 127 op/s 2024-05-27T07:48:06.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:06 smithi090 ceph-mon[31429]: pgmap v53: 129 pgs: 129 active+clean; 207 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 13 MiB/s wr, 127 op/s 2024-05-27T07:48:06.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:06 smithi129 ceph-mon[35254]: pgmap v53: 129 pgs: 129 active+clean; 207 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 13 MiB/s wr, 127 op/s 2024-05-27T07:48:07.666 INFO:teuthology.orchestra.run.smithi090.stderr:800000 of 3200000 tuples (25%) done (elapsed 23.72 s, remaining 71.15 s) 2024-05-27T07:48:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.059+0000 7f753d800700 1 -- 172.21.15.90:0/2676146564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538076440 msgr2=0x7f75381106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.059+0000 7f753d800700 1 --2- 172.21.15.90:0/2676146564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538076440 0x7f75381106b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7524009990 tx=0x7f7524009ca0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.059+0000 7f753d800700 1 -- 172.21.15.90:0/2676146564 shutdown_connections 2024-05-27T07:48:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.059+0000 7f753d800700 1 --2- 172.21.15.90:0/2676146564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538110bf0 0x7f7538113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.059+0000 7f753d800700 1 --2- 172.21.15.90:0/2676146564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538075af0 0x7f7538075f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.059+0000 7f753d800700 1 --2- 172.21.15.90:0/2676146564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538076440 0x7f75381106b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.059+0000 7f753d800700 1 -- 172.21.15.90:0/2676146564 >> 172.21.15.90:0/2676146564 conn(0x7f7538100030 msgr2=0x7f7538102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.059+0000 7f753d800700 1 -- 172.21.15.90:0/2676146564 shutdown_connections 2024-05-27T07:48:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.059+0000 7f753d800700 1 -- 172.21.15.90:0/2676146564 wait complete. 2024-05-27T07:48:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.061+0000 7f753d800700 1 Processor -- start 2024-05-27T07:48:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.061+0000 7f753d800700 1 -- start start 2024-05-27T07:48:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.061+0000 7f753d800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538075af0 0x7f753819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.061+0000 7f753d800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538076440 0x7f753819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.061+0000 7f753d800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538110bf0 0x7f75381a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.061+0000 7f753d800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75381165b0 con 0x7f7538075af0 2024-05-27T07:48:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.061+0000 7f753d800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7538116430 con 0x7f7538076440 2024-05-27T07:48:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.061+0000 7f753d800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7538116730 con 0x7f7538110bf0 2024-05-27T07:48:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7536a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538075af0 0x7f753819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7536a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538075af0 0x7f753819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55418/0 (socket says 172.21.15.90:55418) 2024-05-27T07:48:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7536a00700 1 -- 172.21.15.90:0/378596834 learned_addr learned my addr 172.21.15.90:0/378596834 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7536000700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538076440 0x7f753819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7537400700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538110bf0 0x7f75381a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7536a00700 1 -- 172.21.15.90:0/378596834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538110bf0 msgr2=0x7f75381a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7536a00700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538110bf0 0x7f75381a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7536a00700 1 -- 172.21.15.90:0/378596834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538076440 msgr2=0x7f753819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7536a00700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538076440 0x7f753819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7536a00700 1 -- 172.21.15.90:0/378596834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75381a1de0 con 0x7f7538075af0 2024-05-27T07:48:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.062+0000 7f7537400700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538110bf0 0x7f75381a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.063+0000 7f7536a00700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538075af0 0x7f753819cfc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f752800c9b0 tx=0x7f752800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.063+0000 7f752f400700 1 -- 172.21.15.90:0/378596834 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528007b00 con 0x7f7538075af0 2024-05-27T07:48:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.063+0000 7f752f400700 1 -- 172.21.15.90:0/378596834 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7528007ca0 con 0x7f7538075af0 2024-05-27T07:48:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.063+0000 7f752f400700 1 -- 172.21.15.90:0/378596834 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528018660 con 0x7f7538075af0 2024-05-27T07:48:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.063+0000 7f753d800700 1 -- 172.21.15.90:0/378596834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75381a20d0 con 0x7f7538075af0 2024-05-27T07:48:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.063+0000 7f753d800700 1 -- 172.21.15.90:0/378596834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75381a25c0 con 0x7f7538075af0 2024-05-27T07:48:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.067+0000 7f752f400700 1 -- 172.21.15.90:0/378596834 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f752800f450 con 0x7f7538075af0 2024-05-27T07:48:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.069+0000 7f754c2c5b80 1 Processor -- start 2024-05-27T07:48:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.069+0000 7f752f400700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f751c0b2a70 0x7f751c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.069+0000 7f754c2c5b80 1 -- start start 2024-05-27T07:48:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.069+0000 7f752f400700 1 -- 172.21.15.90:0/378596834 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f75280e29b0 con 0x7f7538075af0 2024-05-27T07:48:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.070+0000 7f7537400700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f751c0b2a70 0x7f751c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.070+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e362e40 0x55972e4e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.070+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 0x55972e4e1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.070+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e1db0 0x55972e4e21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.070+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55972e40fe40 con 0x55972e362e40 2024-05-27T07:48:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.070+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55972e3ff9e0 con 0x55972e4e7d30 2024-05-27T07:48:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.070+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55972e403a20 con 0x55972e4e1db0 2024-05-27T07:48:08.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7536000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 0x55972e4e1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7537400700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f751c0b2a70 0x7f751c0b4f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7538076ad0 tx=0x7f7530007560 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:08.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7536000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 0x55972e4e1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:42178/0 (socket says 172.21.15.90:42178) 2024-05-27T07:48:08.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7537400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e1db0 0x55972e4e21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7536000700 1 -- 172.21.15.90:0/561502910 learned_addr learned my addr 172.21.15.90:0/561502910 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:08.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7536a00700 1 --2- 172.21.15.90:0/561502910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e362e40 0x55972e4e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7536000700 1 -- 172.21.15.90:0/561502910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e1db0 msgr2=0x55972e4e21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:08.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7536000700 1 --2- 172.21.15.90:0/561502910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e1db0 0x55972e4e21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7536000700 1 -- 172.21.15.90:0/561502910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e362e40 msgr2=0x55972e4e77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:08.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7536000700 1 --2- 172.21.15.90:0/561502910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e362e40 0x55972e4e77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7536000700 1 -- 172.21.15.90:0/561502910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55972e3c8420 con 0x55972e4e7d30 2024-05-27T07:48:08.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7536a00700 1 --2- 172.21.15.90:0/561502910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e362e40 0x55972e4e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:08.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.071+0000 7f7537400700 1 --2- 172.21.15.90:0/561502910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e1db0 0x55972e4e21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:48:08.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.072+0000 7f7536000700 1 --2- 172.21.15.90:0/561502910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 0x55972e4e1870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7524009990 tx=0x7f7524003720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fed18106ddbc572 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.072+0000 7f7523e00700 1 -- 172.21.15.90:0/561502910 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524027070 con 0x55972e4e7d30 2024-05-27T07:48:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.073+0000 7f7523e00700 1 -- 172.21.15.90:0/561502910 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7524004da0 con 0x55972e4e7d30 2024-05-27T07:48:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.073+0000 7f7523e00700 1 -- 172.21.15.90:0/561502910 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752400b9f0 con 0x55972e4e7d30 2024-05-27T07:48:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.074+0000 7f754c2c5b80 1 -- 172.21.15.90:0/561502910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 msgr2=0x55972e4e1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.074+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/561502910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 0x55972e4e1870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7524009990 tx=0x7f7524003720 comp rx=0 tx=0).stop 2024-05-27T07:48:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.074+0000 7f754c2c5b80 1 -- 172.21.15.90:0/561502910 shutdown_connections 2024-05-27T07:48:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.074+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/561502910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e1db0 0x55972e4e21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.074+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/561502910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e362e40 0x55972e4e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.074+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/561502910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 0x55972e4e1870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.074+0000 7f754c2c5b80 1 -- 172.21.15.90:0/561502910 >> 172.21.15.90:0/561502910 conn(0x55972e3c2fc0 msgr2=0x55972e4dc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.074+0000 7f754c2c5b80 1 -- 172.21.15.90:0/561502910 shutdown_connections 2024-05-27T07:48:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.074+0000 7f754c2c5b80 1 -- 172.21.15.90:0/561502910 wait complete. 2024-05-27T07:48:08.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.075+0000 7f754c2c5b80 1 -- start start 2024-05-27T07:48:08.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.075+0000 7f754c2c5b80 1 Processor -- start 2024-05-27T07:48:08.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e362e40 0x55972e4eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 0x55972e4ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55972e305fa0 con 0x55972e4e1db0 2024-05-27T07:48:08.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55972e3a0760 con 0x55972e4e7d30 2024-05-27T07:48:08.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55972e4ff7b0 con 0x55972e362e40 2024-05-27T07:48:08.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f754c2c5b80 10 client.0 _ll_register_callbacks cb 0x55972e3a9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:08.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f7536000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f7536000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4eb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55428/0 (socket says 172.21.15.90:55428) 2024-05-27T07:48:08.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f7536000700 1 -- 172.21.15.90:0/3678184447 learned_addr learned my addr 172.21.15.90:0/3678184447 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:08.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.076+0000 7f7536a00700 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e362e40 0x55972e4eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.077+0000 7f7536000700 1 -- 172.21.15.90:0/3678184447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e362e40 msgr2=0x55972e4eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:08.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.077+0000 7f7536000700 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e362e40 0x55972e4eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.077+0000 7f7536000700 1 -- 172.21.15.90:0/3678184447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 msgr2=0x55972e4ebf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:08.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.077+0000 7f7536000700 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 0x55972e4ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.077+0000 7f7536000700 1 -- 172.21.15.90:0/3678184447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55972e34f8d0 con 0x55972e4e1db0 2024-05-27T07:48:08.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.077+0000 7f7536000700 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4eb9f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f75240098e0 tx=0x7f752400ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b76a00f643e3bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:08.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.077+0000 7f7520c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:08.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.078+0000 7f7520c00700 1 -- 172.21.15.90:0/3678184447 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524027070 con 0x55972e4e1db0 2024-05-27T07:48:08.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.078+0000 7f7520c00700 1 -- 172.21.15.90:0/3678184447 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f7524005210 con 0x55972e4e1db0 2024-05-27T07:48:08.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.078+0000 7f754c2c5b80 1 -- 172.21.15.90:0/3678184447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7524004da0 con 0x55972e4e1db0 2024-05-27T07:48:08.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.078+0000 7f7520c00700 1 -- 172.21.15.90:0/3678184447 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f752400be60 con 0x55972e4e1db0 2024-05-27T07:48:08.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.078+0000 7f754c2c5b80 10 client.15732 fetch_fsmap learned FSMap version 32 2024-05-27T07:48:08.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.078+0000 7f754c2c5b80 1 -- 172.21.15.90:0/3678184447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55972e3d0060 con 0x55972e4e1db0 2024-05-27T07:48:08.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.079+0000 7f7520c00700 1 -- 172.21.15.90:0/3678184447 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7524004da0 con 0x55972e4e1db0 2024-05-27T07:48:08.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.079+0000 7f7520c00700 1 -- 172.21.15.90:0/3678184447 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f752400faf0 con 0x55972e4e1db0 2024-05-27T07:48:08.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.079+0000 7f754c2c5b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:48:08.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.079+0000 7f754c2c5b80 10 client.15732 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:48:08.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.079+0000 7f754c2c5b80 20 client.15732 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:08.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.079+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55972e50c4e0 0x55972e50e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.079+0000 7f754c2c5b80 4 client.15732 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:08.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.079+0000 7f754c2c5b80 1 -- 172.21.15.90:0/3678184447 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55972e411d90 con 0x55972e50c4e0 2024-05-27T07:48:08.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.080+0000 7f7536a00700 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55972e50c4e0 0x55972e50e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.081+0000 7f7536a00700 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55972e50c4e0 0x55972e50e900 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de10b22fbc058626 server_cookie=d9a39e3c234822b9 in_seq=0 out_seq=0 2024-05-27T07:48:08.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.081+0000 7f7520c00700 10 client.15732 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:48:08.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.083+0000 7f7520c00700 1 -- 172.21.15.90:0/3678184447 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55972e411d90 con 0x55972e50c4e0 2024-05-27T07:48:08.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.083+0000 7f7520c00700 10 client.15732 handle_command_reply: tid=0 2024-05-27T07:48:08.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.083+0000 7f754c2c5b80 1 client.15732 shutdown 2024-05-27T07:48:08.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.083+0000 7f754c2c5b80 1 -- 172.21.15.90:0/3678184447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 msgr2=0x55972e4eb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:08.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.083+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4eb9f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f75240098e0 tx=0x7f752400ba80 comp rx=0 tx=0).stop 2024-05-27T07:48:08.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.083+0000 7f754c2c5b80 1 -- 172.21.15.90:0/3678184447 shutdown_connections 2024-05-27T07:48:08.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.083+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55972e50c4e0 0x55972e50e900 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.083+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e4e7d30 0x55972e4ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.083+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e362e40 0x55972e4eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.083+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/3678184447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4eb9f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.084+0000 7f754c2c5b80 1 -- 172.21.15.90:0/3678184447 >> 172.21.15.90:0/3678184447 conn(0x55972e3c2fc0 msgr2=0x55972e4dc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:08.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.084+0000 7f7520c00700 0 client.15732 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:48:08.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.084+0000 7f754c2c5b80 1 -- 172.21.15.90:0/3678184447 shutdown_connections 2024-05-27T07:48:08.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.084+0000 7f754c2c5b80 1 -- 172.21.15.90:0/3678184447 wait complete. 2024-05-27T07:48:08.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.084+0000 7f754c2c5b80 20 client.15732 trim_cache size 0 max 16384 2024-05-27T07:48:08.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.109+0000 7f754c2c5b80 1 Processor -- start 2024-05-27T07:48:08.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.109+0000 7f754c2c5b80 1 -- start start 2024-05-27T07:48:08.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.110+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 0x55972e4e1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.110+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4ed950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.110+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e7d30 0x55972e508a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.110+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75381a9df0 con 0x55972e4e1db0 2024-05-27T07:48:08.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.110+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75381ab930 con 0x55972e362e40 2024-05-27T07:48:08.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.110+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75381aa4c0 con 0x55972e4e7d30 2024-05-27T07:48:08.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.110+0000 7f7536000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.110+0000 7f7536000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4ed950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55434/0 (socket says 172.21.15.90:55434) 2024-05-27T07:48:08.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.110+0000 7f7536000700 1 -- 172.21.15.90:0/1839398293 learned_addr learned my addr 172.21.15.90:0/1839398293 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:08.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.110+0000 7f7536a00700 1 --2- 172.21.15.90:0/1839398293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 0x55972e4e1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.111+0000 7f7537400700 1 --2- 172.21.15.90:0/1839398293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e7d30 0x55972e508a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.111+0000 7f7536000700 1 -- 172.21.15.90:0/1839398293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e7d30 msgr2=0x55972e508a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:08.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.111+0000 7f7536000700 1 --2- 172.21.15.90:0/1839398293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e7d30 0x55972e508a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.111+0000 7f7536000700 1 -- 172.21.15.90:0/1839398293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 msgr2=0x55972e4e1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:08.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.111+0000 7f7536000700 1 --2- 172.21.15.90:0/1839398293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 0x55972e4e1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.111+0000 7f7536000700 1 -- 172.21.15.90:0/1839398293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55972e4e9a40 con 0x55972e4e1db0 2024-05-27T07:48:08.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.111+0000 7f7536000700 1 --2- 172.21.15.90:0/1839398293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4ed950 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f752402d460 tx=0x7f752400b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cfa0916130a1b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:08.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.112+0000 7f7523400700 1 -- 172.21.15.90:0/1839398293 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524027070 con 0x55972e4e1db0 2024-05-27T07:48:08.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.112+0000 7f7523400700 1 -- 172.21.15.90:0/1839398293 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75240056a0 con 0x55972e4e1db0 2024-05-27T07:48:08.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.112+0000 7f7523400700 1 -- 172.21.15.90:0/1839398293 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752402ddc0 con 0x55972e4e1db0 2024-05-27T07:48:08.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.113+0000 7f754c2c5b80 1 -- 172.21.15.90:0/1839398293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 msgr2=0x55972e4ed950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.113+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/1839398293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4ed950 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f752402d460 tx=0x7f752400b580 comp rx=0 tx=0).stop 2024-05-27T07:48:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.113+0000 7f754c2c5b80 1 -- 172.21.15.90:0/1839398293 shutdown_connections 2024-05-27T07:48:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.113+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/1839398293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e7d30 0x55972e508a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.113+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/1839398293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 0x55972e4e1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.113+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/1839398293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e4ed950 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.113+0000 7f754c2c5b80 1 -- 172.21.15.90:0/1839398293 >> 172.21.15.90:0/1839398293 conn(0x55972e3c2fc0 msgr2=0x55972e4ebcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.113+0000 7f754c2c5b80 1 -- 172.21.15.90:0/1839398293 shutdown_connections 2024-05-27T07:48:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.113+0000 7f754c2c5b80 1 -- 172.21.15.90:0/1839398293 wait complete. 2024-05-27T07:48:08.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.114+0000 7f754c2c5b80 1 -- start start 2024-05-27T07:48:08.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.114+0000 7f754c2c5b80 1 Processor -- start 2024-05-27T07:48:08.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 0x55972e511980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e50ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f754c2c5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e7d30 0x55972e50f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75381ab260 con 0x55972e4e1db0 2024-05-27T07:48:08.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75381aab90 con 0x55972e362e40 2024-05-27T07:48:08.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f754c2c5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7538073bf0 con 0x55972e4e7d30 2024-05-27T07:48:08.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f754c2c5b80 10 client.0 _ll_register_callbacks cb 0x55972e3a9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:08.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f7536000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e50ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f7536000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e50ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55444/0 (socket says 172.21.15.90:55444) 2024-05-27T07:48:08.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f7536000700 1 -- 172.21.15.90:0/2729893913 learned_addr learned my addr 172.21.15.90:0/2729893913 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:08.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f7536a00700 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 0x55972e511980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f7536000700 1 -- 172.21.15.90:0/2729893913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e7d30 msgr2=0x55972e50f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f7536000700 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e7d30 0x55972e50f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f7536000700 1 -- 172.21.15.90:0/2729893913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 msgr2=0x55972e511980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f7536000700 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 0x55972e511980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.115+0000 7f7536000700 1 -- 172.21.15.90:0/2729893913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55972e50f780 con 0x55972e4e1db0 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.116+0000 7f7536a00700 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 0x55972e511980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.116+0000 7f7536000700 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e50ed00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f752400fec0 tx=0x7f752400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a751416af3c363 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.116+0000 7f7520c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.116+0000 7f7520c00700 1 -- 172.21.15.90:0/2729893913 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524027040 con 0x55972e4e1db0 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.116+0000 7f7520c00700 1 -- 172.21.15.90:0/2729893913 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f752402d960 con 0x55972e4e1db0 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.116+0000 7f754c2c5b80 1 -- 172.21.15.90:0/2729893913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75240056a0 con 0x55972e4e1db0 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f7520c00700 1 -- 172.21.15.90:0/2729893913 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f752402db60 con 0x55972e4e1db0 2024-05-27T07:48:08.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f754c2c5b80 10 client.15738 fetch_fsmap learned FSMap version 32 2024-05-27T07:48:08.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f754c2c5b80 1 -- 172.21.15.90:0/2729893913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55972e5769c0 con 0x55972e4e1db0 2024-05-27T07:48:08.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f7520c00700 1 -- 172.21.15.90:0/2729893913 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75240056a0 con 0x55972e4e1db0 2024-05-27T07:48:08.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f7520c00700 1 -- 172.21.15.90:0/2729893913 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f752403a020 con 0x55972e4e1db0 2024-05-27T07:48:08.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f754c2c5b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:48:08.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f754c2c5b80 10 client.15738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:48:08.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f754c2c5b80 20 client.15738 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:08.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55972e576e50 0x55972e579270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:08.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f754c2c5b80 4 client.15738 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:08.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.117+0000 7f754c2c5b80 1 -- 172.21.15.90:0/2729893913 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55972e40b570 con 0x55972e576e50 2024-05-27T07:48:08.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.118+0000 7f7536a00700 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55972e576e50 0x55972e579270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:08.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.119+0000 7f7536a00700 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55972e576e50 0x55972e579270 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b8fe40821483f686 server_cookie=ce5a7aed7312754f in_seq=0 out_seq=0 2024-05-27T07:48:08.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.119+0000 7f7520c00700 10 client.15738 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:48:08.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f7520c00700 1 -- 172.21.15.90:0/2729893913 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55972e40b570 con 0x55972e576e50 2024-05-27T07:48:08.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f7520c00700 10 client.15738 handle_command_reply: tid=0 2024-05-27T07:48:08.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f754c2c5b80 1 client.15738 shutdown 2024-05-27T07:48:08.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f754c2c5b80 1 -- 172.21.15.90:0/2729893913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 msgr2=0x55972e50ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:08.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e50ed00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f752400fec0 tx=0x7f752400bc10 comp rx=0 tx=0).stop 2024-05-27T07:48:08.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f754c2c5b80 1 -- 172.21.15.90:0/2729893913 shutdown_connections 2024-05-27T07:48:08.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55972e576e50 0x55972e579270 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55972e4e7d30 0x55972e50f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55972e362e40 0x55972e511980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f754c2c5b80 1 --2- 172.21.15.90:0/2729893913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55972e4e1db0 0x55972e50ed00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.120+0000 7f754c2c5b80 1 -- 172.21.15.90:0/2729893913 >> 172.21.15.90:0/2729893913 conn(0x55972e3c2fc0 msgr2=0x55972e4ebcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:08.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.121+0000 7f7520c00700 0 client.15738 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:48:08.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.121+0000 7f754c2c5b80 1 -- 172.21.15.90:0/2729893913 shutdown_connections 2024-05-27T07:48:08.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.121+0000 7f754c2c5b80 1 -- 172.21.15.90:0/2729893913 wait complete. 2024-05-27T07:48:08.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.121+0000 7f754c2c5b80 20 client.15738 trim_cache size 0 max 16384 2024-05-27T07:48:08.122 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:48:08.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 -- 172.21.15.90:0/378596834 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f751c0b2a70 msgr2=0x7f751c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:08.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f751c0b2a70 0x7f751c0b4f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7538076ad0 tx=0x7f7530007560 comp rx=0 tx=0).stop 2024-05-27T07:48:08.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 -- 172.21.15.90:0/378596834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538075af0 msgr2=0x7f753819cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:08.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538075af0 0x7f753819cfc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f752800c9b0 tx=0x7f752800cd70 comp rx=0 tx=0).stop 2024-05-27T07:48:08.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 -- 172.21.15.90:0/378596834 shutdown_connections 2024-05-27T07:48:08.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7538110bf0 0x7f75381a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538075af0 0x7f753819cfc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f751c0b2a70 0x7f751c0b4f80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 --2- 172.21.15.90:0/378596834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7538076440 0x7f753819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:08.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 -- 172.21.15.90:0/378596834 >> 172.21.15.90:0/378596834 conn(0x7f7538100030 msgr2=0x7f7538101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:08.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 -- 172.21.15.90:0/378596834 shutdown_connections 2024-05-27T07:48:08.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:08.122+0000 7f753d800700 1 -- 172.21.15.90:0/378596834 wait complete. 2024-05-27T07:48:08.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:08 smithi090 ceph-mon[31429]: pgmap v54: 129 pgs: 129 active+clean; 207 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 12 MiB/s wr, 127 op/s 2024-05-27T07:48:08.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:08 smithi136 ceph-mon[35324]: pgmap v54: 129 pgs: 129 active+clean; 207 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 12 MiB/s wr, 127 op/s 2024-05-27T07:48:08.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:08 smithi129 ceph-mon[35254]: pgmap v54: 129 pgs: 129 active+clean; 207 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 12 MiB/s wr, 127 op/s 2024-05-27T07:48:09.362 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:48:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.689+0000 7fe49b200640 1 -- 172.21.15.90:0/1526697061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4941105a0 msgr2=0x7fe4941129c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.689+0000 7fe49b200640 1 --2- 172.21.15.90:0/1526697061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4941105a0 0x7fe4941129c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009ca0 tx=0x7fe48c02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.690+0000 7fe49b200640 1 -- 172.21.15.90:0/1526697061 shutdown_connections 2024-05-27T07:48:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.690+0000 7fe49b200640 1 --2- 172.21.15.90:0/1526697061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe49410dc40 0x7fe494110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.690+0000 7fe49b200640 1 --2- 172.21.15.90:0/1526697061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4941105a0 0x7fe4941129c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.690+0000 7fe49b200640 1 --2- 172.21.15.90:0/1526697061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe494102a00 0x7fe494102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.690+0000 7fe49b200640 1 -- 172.21.15.90:0/1526697061 >> 172.21.15.90:0/1526697061 conn(0x7fe4940782d0 msgr2=0x7fe494078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.690+0000 7fe49b200640 1 -- 172.21.15.90:0/1526697061 shutdown_connections 2024-05-27T07:48:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.690+0000 7fe49b200640 1 -- 172.21.15.90:0/1526697061 wait complete. 2024-05-27T07:48:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.691+0000 7fe49b200640 1 Processor -- start 2024-05-27T07:48:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.691+0000 7fe49b200640 1 -- start start 2024-05-27T07:48:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe49b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe494102a00 0x7fe4941a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe49b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe49410dc40 0x7fe4941a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe49b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4941105a0 0x7fe49419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe49b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe494115f70 con 0x7fe4941105a0 2024-05-27T07:48:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe49b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe494115df0 con 0x7fe49410dc40 2024-05-27T07:48:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe49b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941160f0 con 0x7fe494102a00 2024-05-27T07:48:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe499200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4941105a0 0x7fe49419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe499200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4941105a0 0x7fe49419c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39376/0 (socket says 172.21.15.90:39376) 2024-05-27T07:48:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe499200640 1 -- 172.21.15.90:0/198944586 learned_addr learned my addr 172.21.15.90:0/198944586 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe493400640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe49410dc40 0x7fe4941a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.692+0000 7fe493e00640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe494102a00 0x7fe4941a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.693+0000 7fe499200640 1 -- 172.21.15.90:0/198944586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe494102a00 msgr2=0x7fe4941a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.693+0000 7fe499200640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe494102a00 0x7fe4941a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.693+0000 7fe499200640 1 -- 172.21.15.90:0/198944586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe49410dc40 msgr2=0x7fe4941a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.693+0000 7fe499200640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe49410dc40 0x7fe4941a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.693+0000 7fe499200640 1 -- 172.21.15.90:0/198944586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49419cea0 con 0x7fe4941105a0 2024-05-27T07:48:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.693+0000 7fe499200640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4941105a0 0x7fe49419c770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe48c031040 tx=0x7fe48c032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.694+0000 7fe490c00640 1 -- 172.21.15.90:0/198944586 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c004040 con 0x7fe4941105a0 2024-05-27T07:48:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.694+0000 7fe490c00640 1 -- 172.21.15.90:0/198944586 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe48c004670 con 0x7fe4941105a0 2024-05-27T07:48:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.694+0000 7fe49b200640 1 -- 172.21.15.90:0/198944586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49419d0d0 con 0x7fe4941105a0 2024-05-27T07:48:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.694+0000 7fe490c00640 1 -- 172.21.15.90:0/198944586 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c030bb0 con 0x7fe4941105a0 2024-05-27T07:48:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.694+0000 7fe49b200640 1 -- 172.21.15.90:0/198944586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4941a9390 con 0x7fe4941105a0 2024-05-27T07:48:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.699+0000 7fe4a216e740 1 Processor -- start 2024-05-27T07:48:09.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.699+0000 7fe490c00640 1 -- 172.21.15.90:0/198944586 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe48c0041e0 con 0x7fe4941105a0 2024-05-27T07:48:09.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.699+0000 7fe4a216e740 1 -- start start 2024-05-27T07:48:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.700+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd118860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.700+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd118da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.700+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd554d0 0x55e8fd1192e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.700+0000 7fe4a216e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e8fd089b90 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.700+0000 7fe4a216e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8fd079730 con 0x55e8fcd554d0 2024-05-27T07:48:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.700+0000 7fe4a216e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e8fd07d770 con 0x55e8fcd761b0 2024-05-27T07:48:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.700+0000 7fe493e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd118da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.700+0000 7fe493e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd118da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39384/0 (socket says 172.21.15.90:39384) 2024-05-27T07:48:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.700+0000 7fe493e00640 1 -- 172.21.15.90:0/1682307238 learned_addr learned my addr 172.21.15.90:0/1682307238 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.701+0000 7fe493400640 1 --2- 172.21.15.90:0/1682307238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd118860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.701+0000 7fe493400640 1 --2- 172.21.15.90:0/1682307238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd554d0 0x55e8fd1192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.701+0000 7fe493e00640 1 -- 172.21.15.90:0/1682307238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 msgr2=0x55e8fd118860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.701+0000 7fe493e00640 1 --2- 172.21.15.90:0/1682307238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd118860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.701+0000 7fe493e00640 1 -- 172.21.15.90:0/1682307238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd554d0 msgr2=0x55e8fd1192e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.701+0000 7fe493e00640 1 --2- 172.21.15.90:0/1682307238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd554d0 0x55e8fd1192e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.701+0000 7fe493e00640 1 -- 172.21.15.90:0/1682307238 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8fd0128b0 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.702+0000 7fe490c00640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe4780b2950 0x7fe4780b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.702+0000 7fe493e00640 1 --2- 172.21.15.90:0/1682307238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd118da0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe48400b760 tx=0x7fe48400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6e0c663c287a68 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.702+0000 7fe490c00640 1 -- 172.21.15.90:0/198944586 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fe48c103c70 con 0x7fe4941105a0 2024-05-27T07:48:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.702+0000 7fe493400640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe4780b2950 0x7fe4780b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.703+0000 7fe489600640 1 -- 172.21.15.90:0/1682307238 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484003dd0 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.703+0000 7fe489600640 1 -- 172.21.15.90:0/1682307238 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe484015070 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.703+0000 7fe489600640 1 -- 172.21.15.90:0/1682307238 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484004c30 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.704+0000 7fe4a216e740 1 -- 172.21.15.90:0/1682307238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 msgr2=0x55e8fd118da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.704+0000 7fe4a216e740 1 --2- 172.21.15.90:0/1682307238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd118da0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe48400b760 tx=0x7fe48400bc20 comp rx=0 tx=0).stop 2024-05-27T07:48:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.704+0000 7fe493400640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe4780b2950 0x7fe4780b4e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55e8fd1159d0 tx=0x7fe48000b390 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.704+0000 7fe4a216e740 1 -- 172.21.15.90:0/1682307238 shutdown_connections 2024-05-27T07:48:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.704+0000 7fe4a216e740 1 --2- 172.21.15.90:0/1682307238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd554d0 0x55e8fd1192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.704+0000 7fe4a216e740 1 --2- 172.21.15.90:0/1682307238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd118da0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.704+0000 7fe4a216e740 1 --2- 172.21.15.90:0/1682307238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd118860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.704+0000 7fe4a216e740 1 -- 172.21.15.90:0/1682307238 >> 172.21.15.90:0/1682307238 conn(0x55e8fcd6a300 msgr2=0x55e8fd115010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:09.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.704+0000 7fe4a216e740 1 -- 172.21.15.90:0/1682307238 shutdown_connections 2024-05-27T07:48:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.704+0000 7fe4a216e740 1 -- 172.21.15.90:0/1682307238 wait complete. 2024-05-27T07:48:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.705+0000 7fe4a216e740 1 -- start start 2024-05-27T07:48:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.705+0000 7fe4a216e740 1 Processor -- start 2024-05-27T07:48:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.706+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd122a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.706+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd554d0 0x55e8fd122fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.706+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd12ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.706+0000 7fe4a216e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e8fcd5f790 con 0x55e8fcd554d0 2024-05-27T07:48:09.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.706+0000 7fe4a216e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8fd021bc0 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.706+0000 7fe4a216e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e8fd125990 con 0x55e8fcd761b0 2024-05-27T07:48:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.706+0000 7fe4a216e740 10 client.0 _ll_register_callbacks cb 0x55e8fcfff400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.706+0000 7fe493e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd122a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.706+0000 7fe493e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd122a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43064/0 (socket says 172.21.15.90:43064) 2024-05-27T07:48:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.706+0000 7fe493e00640 1 -- 172.21.15.90:0/2863047705 learned_addr learned my addr 172.21.15.90:0/2863047705 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.707+0000 7fe499200640 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd12ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.707+0000 7fe493e00640 1 -- 172.21.15.90:0/2863047705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 msgr2=0x55e8fd12ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.707+0000 7fe493e00640 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd12ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.707+0000 7fe493e00640 1 -- 172.21.15.90:0/2863047705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd554d0 msgr2=0x55e8fd122fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.707+0000 7fe493e00640 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd554d0 0x55e8fd122fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.707+0000 7fe493e00640 1 -- 172.21.15.90:0/2863047705 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8fcd8e210 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.708+0000 7fe493e00640 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd122a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe484018000 tx=0x7fe484003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adad250cce051c19 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.708+0000 7fe47e000640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:48:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.708+0000 7fe47e000640 1 -- 172.21.15.90:0/2863047705 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48400be70 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.708+0000 7fe4a216e740 1 -- 172.21.15.90:0/2863047705 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe484015070 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.709+0000 7fe47e000640 1 -- 172.21.15.90:0/2863047705 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fe484011650 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.709+0000 7fe47e000640 1 -- 172.21.15.90:0/2863047705 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe484030050 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.709+0000 7fe4a216e740 10 client.35387 fetch_fsmap learned FSMap version 32 2024-05-27T07:48:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.709+0000 7fe4a216e740 1 -- 172.21.15.90:0/2863047705 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e8fce5a620 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.710+0000 7fe47e000640 1 -- 172.21.15.90:0/2863047705 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe484015070 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.710+0000 7fe47e000640 1 -- 172.21.15.90:0/2863047705 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe484025760 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.710+0000 7fe4a216e740 10 client.35387 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.710+0000 7fe4a216e740 10 client.35387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.710+0000 7fe4a216e740 20 client.35387 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.710+0000 7fe4a216e740 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e8fcd34d80 0x55e8fd134b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.710+0000 7fe4a216e740 4 client.35387 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.710+0000 7fe4a216e740 1 -- 172.21.15.90:0/2863047705 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8fd08bae0 con 0x55e8fcd34d80 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.711+0000 7fe493400640 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e8fcd34d80 0x55e8fd134b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.712+0000 7fe493400640 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e8fcd34d80 0x55e8fd134b70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3953d9407a9f24d server_cookie=fcaea27e67a58964 in_seq=0 out_seq=0 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.712+0000 7fe47e000640 10 client.35387 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe47e000640 1 -- 172.21.15.90:0/2863047705 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e8fd08bae0 con 0x55e8fcd34d80 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe47e000640 10 client.35387 handle_command_reply: tid=0 2024-05-27T07:48:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe4a216e740 1 client.35387 shutdown 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe4a216e740 1 -- 172.21.15.90:0/2863047705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd4d2f0 msgr2=0x55e8fd122a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe4a216e740 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd122a70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe484018000 tx=0x7fe484003e00 comp rx=0 tx=0).stop 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe4a216e740 1 -- 172.21.15.90:0/2863047705 shutdown_connections 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe4a216e740 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e8fcd34d80 0x55e8fd134b70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe4a216e740 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd12ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe4a216e740 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd554d0 0x55e8fd122fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe47e000640 0 client.35387 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe4a216e740 1 --2- 172.21.15.90:0/2863047705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd122a70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.714+0000 7fe4a216e740 1 -- 172.21.15.90:0/2863047705 >> 172.21.15.90:0/2863047705 conn(0x55e8fcd6a300 msgr2=0x55e8fd118690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.715+0000 7fe4a216e740 1 -- 172.21.15.90:0/2863047705 shutdown_connections 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.715+0000 7fe4a216e740 1 -- 172.21.15.90:0/2863047705 wait complete. 2024-05-27T07:48:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.715+0000 7fe4a216e740 20 client.35387 trim_cache size 0 max 16384 2024-05-27T07:48:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.735+0000 7fe4a216e740 1 Processor -- start 2024-05-27T07:48:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.736+0000 7fe4a216e740 1 -- start start 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.736+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd11e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.736+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd554d0 0x55e8fd124bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.736+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd13a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.736+0000 7fe4a216e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941b3300 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.736+0000 7fe4a216e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941b4e40 con 0x55e8fcd554d0 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.736+0000 7fe4a216e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941b39d0 con 0x55e8fcd761b0 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.737+0000 7fe493e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd11e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.737+0000 7fe493e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd11e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39400/0 (socket says 172.21.15.90:39400) 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.737+0000 7fe493e00640 1 -- 172.21.15.90:0/2854992501 learned_addr learned my addr 172.21.15.90:0/2854992501 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.737+0000 7fe499200640 1 --2- 172.21.15.90:0/2854992501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd13a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.737+0000 7fe493400640 1 --2- 172.21.15.90:0/2854992501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd554d0 0x55e8fd124bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.737+0000 7fe493e00640 1 -- 172.21.15.90:0/2854992501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 msgr2=0x55e8fd13a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.737+0000 7fe493e00640 1 --2- 172.21.15.90:0/2854992501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd13a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.737+0000 7fe493e00640 1 -- 172.21.15.90:0/2854992501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd554d0 msgr2=0x55e8fd124bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.737+0000 7fe493e00640 1 --2- 172.21.15.90:0/2854992501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd554d0 0x55e8fd124bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.737+0000 7fe493e00640 1 -- 172.21.15.90:0/2854992501 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8fd115c00 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.738+0000 7fe493e00640 1 --2- 172.21.15.90:0/2854992501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd11e280 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe48400bc60 tx=0x7fe4840041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1429c55f34447f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.738+0000 7fe488c00640 1 -- 172.21.15.90:0/2854992501 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484025630 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.738+0000 7fe488c00640 1 -- 172.21.15.90:0/2854992501 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe484015070 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.738+0000 7fe488c00640 1 -- 172.21.15.90:0/2854992501 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484011ae0 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.739+0000 7fe4a216e740 1 -- 172.21.15.90:0/2854992501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 msgr2=0x55e8fd11e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.739+0000 7fe4a216e740 1 --2- 172.21.15.90:0/2854992501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd11e280 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe48400bc60 tx=0x7fe4840041a0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.739+0000 7fe4a216e740 1 -- 172.21.15.90:0/2854992501 shutdown_connections 2024-05-27T07:48:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.739+0000 7fe4a216e740 1 --2- 172.21.15.90:0/2854992501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd554d0 0x55e8fd124bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.739+0000 7fe4a216e740 1 --2- 172.21.15.90:0/2854992501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd761b0 0x55e8fd13a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.739+0000 7fe4a216e740 1 --2- 172.21.15.90:0/2854992501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd11e280 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.739+0000 7fe4a216e740 1 -- 172.21.15.90:0/2854992501 >> 172.21.15.90:0/2854992501 conn(0x55e8fcd6a300 msgr2=0x55e8fd122dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.740+0000 7fe4a216e740 1 -- 172.21.15.90:0/2854992501 shutdown_connections 2024-05-27T07:48:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.740+0000 7fe4a216e740 1 -- 172.21.15.90:0/2854992501 wait complete. 2024-05-27T07:48:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.740+0000 7fe4a216e740 1 -- start start 2024-05-27T07:48:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.740+0000 7fe4a216e740 1 Processor -- start 2024-05-27T07:48:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.741+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd12b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.741+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd554d0 0x55e8fd128480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.741+0000 7fe4a216e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd761b0 0x55e8fd1289c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.741+0000 7fe4a216e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941b4770 con 0x55e8fcd4d2f0 2024-05-27T07:48:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.741+0000 7fe4a216e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941b40a0 con 0x55e8fcd761b0 2024-05-27T07:48:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.741+0000 7fe4a216e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe4941b12c0 con 0x55e8fcd554d0 2024-05-27T07:48:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.741+0000 7fe4a216e740 10 client.0 _ll_register_callbacks cb 0x7fe48000bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe499200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd761b0 0x55e8fd1289c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe493400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd554d0 0x55e8fd128480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe499200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd761b0 0x55e8fd1289c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43078/0 (socket says 172.21.15.90:43078) 2024-05-27T07:48:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe499200640 1 -- 172.21.15.90:0/3433227669 learned_addr learned my addr 172.21.15.90:0/3433227669 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe493e00640 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd12b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe499200640 1 -- 172.21.15.90:0/3433227669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd554d0 msgr2=0x55e8fd128480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe499200640 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd554d0 0x55e8fd128480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe499200640 1 -- 172.21.15.90:0/3433227669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 msgr2=0x55e8fd12b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe499200640 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd12b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe499200640 1 -- 172.21.15.90:0/3433227669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8fd128f00 con 0x55e8fcd761b0 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.742+0000 7fe493e00640 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd12b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.743+0000 7fe499200640 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd761b0 0x55e8fd1289c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0cac80 tx=0x7fe48c0ca700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc7298605d506b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.743+0000 7fe47e000640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.743+0000 7fe47e000640 1 -- 172.21.15.90:0/3433227669 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0ca800 con 0x55e8fcd761b0 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.743+0000 7fe4a216e740 1 -- 172.21.15.90:0/3433227669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe484015070 con 0x55e8fcd761b0 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.743+0000 7fe47e000640 1 -- 172.21.15.90:0/3433227669 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fe48c0c6f80 con 0x55e8fcd761b0 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.744+0000 7fe47e000640 1 -- 172.21.15.90:0/3433227669 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe48c0cb3f0 con 0x55e8fcd761b0 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.744+0000 7fe4a216e740 10 client.35393 fetch_fsmap learned FSMap version 32 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.744+0000 7fe4a216e740 1 -- 172.21.15.90:0/3433227669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e8fd193e50 con 0x55e8fcd761b0 2024-05-27T07:48:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.744+0000 7fe47e000640 1 -- 172.21.15.90:0/3433227669 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe484015070 con 0x55e8fcd761b0 2024-05-27T07:48:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.744+0000 7fe47e000640 1 -- 172.21.15.90:0/3433227669 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fe48c0caf50 con 0x55e8fcd761b0 2024-05-27T07:48:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.744+0000 7fe4a216e740 10 client.35393 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:48:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.744+0000 7fe4a216e740 10 client.35393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.744+0000 7fe4a216e740 20 client.35393 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.745+0000 7fe4a216e740 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e8fcd0a1c0 0x55e8fd1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.745+0000 7fe4a216e740 4 client.35393 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:48:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.745+0000 7fe4a216e740 1 -- 172.21.15.90:0/3433227669 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8fd0852c0 con 0x55e8fcd0a1c0 2024-05-27T07:48:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.745+0000 7fe493e00640 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e8fcd0a1c0 0x55e8fd1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.746+0000 7fe493e00640 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e8fcd0a1c0 0x55e8fd1965e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8ca22eeda52376b server_cookie=f08202a48cf2b9e3 in_seq=0 out_seq=0 2024-05-27T07:48:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.746+0000 7fe47e000640 10 client.35393 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.790+0000 7fe47e000640 1 -- 172.21.15.90:0/3433227669 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e8fd0852c0 con 0x55e8fcd0a1c0 2024-05-27T07:48:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.790+0000 7fe47e000640 10 client.35393 handle_command_reply: tid=0 2024-05-27T07:48:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.790+0000 7fe4a216e740 1 client.35393 shutdown 2024-05-27T07:48:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.790+0000 7fe4a216e740 1 -- 172.21.15.90:0/3433227669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd761b0 msgr2=0x55e8fd1289c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.790+0000 7fe4a216e740 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd761b0 0x55e8fd1289c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0cac80 tx=0x7fe48c0ca700 comp rx=0 tx=0).stop 2024-05-27T07:48:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.791+0000 7fe4a216e740 1 -- 172.21.15.90:0/3433227669 shutdown_connections 2024-05-27T07:48:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.791+0000 7fe4a216e740 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e8fcd0a1c0 0x55e8fd1965e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.791+0000 7fe4a216e740 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8fcd761b0 0x55e8fd1289c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.791+0000 7fe4a216e740 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8fcd554d0 0x55e8fd128480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.791+0000 7fe47e000640 0 client.35393 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.791+0000 7fe4a216e740 1 --2- 172.21.15.90:0/3433227669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8fcd4d2f0 0x55e8fd12b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.791+0000 7fe4a216e740 1 -- 172.21.15.90:0/3433227669 >> 172.21.15.90:0/3433227669 conn(0x55e8fcd6a300 msgr2=0x55e8fd124b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.791+0000 7fe4a216e740 1 -- 172.21.15.90:0/3433227669 shutdown_connections 2024-05-27T07:48:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.791+0000 7fe4a216e740 1 -- 172.21.15.90:0/3433227669 wait complete. 2024-05-27T07:48:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.791+0000 7fe4a216e740 20 client.35393 trim_cache size 0 max 16384 2024-05-27T07:48:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:48:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:48:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:48:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 -- 172.21.15.90:0/198944586 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe4780b2950 msgr2=0x7fe4780b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe4780b2950 0x7fe4780b4e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55e8fd1159d0 tx=0x7fe48000b390 comp rx=0 tx=0).stop 2024-05-27T07:48:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 -- 172.21.15.90:0/198944586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4941105a0 msgr2=0x7fe49419c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4941105a0 0x7fe49419c770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe48c031040 tx=0x7fe48c032bc0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 -- 172.21.15.90:0/198944586 shutdown_connections 2024-05-27T07:48:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe4780b2950 0x7fe4780b4e60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe49410dc40 0x7fe4941a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4941105a0 0x7fe49419c770 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 --2- 172.21.15.90:0/198944586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe494102a00 0x7fe4941a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 -- 172.21.15.90:0/198944586 >> 172.21.15.90:0/198944586 conn(0x7fe4940782d0 msgr2=0x7fe494110f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 -- 172.21.15.90:0/198944586 shutdown_connections 2024-05-27T07:48:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:09.792+0000 7fe49b200640 1 -- 172.21.15.90:0/198944586 wait complete. 2024-05-27T07:48:09.850 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:10.329 INFO:teuthology.orchestra.run.smithi090.stderr:900000 of 3200000 tuples (28%) done (elapsed 26.38 s, remaining 67.42 s) 2024-05-27T07:48:10.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:10 smithi090 ceph-mon[31429]: pgmap v55: 129 pgs: 129 active+clean; 255 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 18 MiB/s wr, 135 op/s 2024-05-27T07:48:10.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:10 smithi136 ceph-mon[35324]: pgmap v55: 129 pgs: 129 active+clean; 255 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 18 MiB/s wr, 135 op/s 2024-05-27T07:48:10.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:10 smithi129 ceph-mon[35254]: pgmap v55: 129 pgs: 129 active+clean; 255 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 18 MiB/s wr, 135 op/s 2024-05-27T07:48:12.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:12 smithi090 ceph-mon[31429]: pgmap v56: 129 pgs: 129 active+clean; 255 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 31 op/s 2024-05-27T07:48:12.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.207+0000 7fc3ff600700 1 -- 172.21.15.90:0/1988234082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f8108f10 msgr2=0x7fc3f8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.207+0000 7fc3ff600700 1 --2- 172.21.15.90:0/1988234082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f8108f10 0x7fc3f8109320 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40099c0 tx=0x7fc3e4009cd0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.208+0000 7fc3ff600700 1 -- 172.21.15.90:0/1988234082 shutdown_connections 2024-05-27T07:48:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.208+0000 7fc3ff600700 1 --2- 172.21.15.90:0/1988234082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3f8109860 0x7fc3f81111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.208+0000 7fc3ff600700 1 --2- 172.21.15.90:0/1988234082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f81045e0 0x7fc3f81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.208+0000 7fc3ff600700 1 --2- 172.21.15.90:0/1988234082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f8108f10 0x7fc3f8109320 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.208+0000 7fc3ff600700 1 -- 172.21.15.90:0/1988234082 >> 172.21.15.90:0/1988234082 conn(0x7fc3f8100030 msgr2=0x7fc3f8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.208+0000 7fc3ff600700 1 -- 172.21.15.90:0/1988234082 shutdown_connections 2024-05-27T07:48:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.208+0000 7fc3ff600700 1 -- 172.21.15.90:0/1988234082 wait complete. 2024-05-27T07:48:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.209+0000 7fc3ff600700 1 Processor -- start 2024-05-27T07:48:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.209+0000 7fc3ff600700 1 -- start start 2024-05-27T07:48:12.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.210+0000 7fc3ff600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3f81045e0 0x7fc3f819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.210+0000 7fc3ff600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f8108f10 0x7fc3f819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.210+0000 7fc3ff600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f8109860 0x7fc3f819f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.210+0000 7fc3ff600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8114370 con 0x7fc3f8108f10 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.210+0000 7fc3ff600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81141f0 con 0x7fc3f8109860 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.210+0000 7fc3ff600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81144f0 con 0x7fc3f81045e0 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.210+0000 7fc3f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f8108f10 0x7fc3f819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.210+0000 7fc3f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f8108f10 0x7fc3f819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39438/0 (socket says 172.21.15.90:39438) 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.210+0000 7fc3f7e00700 1 -- 172.21.15.90:0/2427356292 learned_addr learned my addr 172.21.15.90:0/2427356292 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.211+0000 7fc3fd600700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f8109860 0x7fc3f819f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.211+0000 7fc3fcc00700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3f81045e0 0x7fc3f819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.211+0000 7fc3f7e00700 1 -- 172.21.15.90:0/2427356292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3f81045e0 msgr2=0x7fc3f819adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.211+0000 7fc3f7e00700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3f81045e0 0x7fc3f819adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.211+0000 7fc3f7e00700 1 -- 172.21.15.90:0/2427356292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f8109860 msgr2=0x7fc3f819f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.211+0000 7fc3f7e00700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f8109860 0x7fc3f819f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.211+0000 7fc3f7e00700 1 -- 172.21.15.90:0/2427356292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f819fb90 con 0x7fc3f8108f10 2024-05-27T07:48:12.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.211+0000 7fc3f7e00700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f8108f10 0x7fc3f819b360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4006010 tx=0x7fc3e4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.212+0000 7fc3f5600700 1 -- 172.21.15.90:0/2427356292 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4027070 con 0x7fc3f8108f10 2024-05-27T07:48:12.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.212+0000 7fc3f5600700 1 -- 172.21.15.90:0/2427356292 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3e4003ef0 con 0x7fc3f8108f10 2024-05-27T07:48:12.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.212+0000 7fc3f5600700 1 -- 172.21.15.90:0/2427356292 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4022ad0 con 0x7fc3f8108f10 2024-05-27T07:48:12.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.212+0000 7fc3ff600700 1 -- 172.21.15.90:0/2427356292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f819fe20 con 0x7fc3f8108f10 2024-05-27T07:48:12.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.212+0000 7fc3ff600700 1 -- 172.21.15.90:0/2427356292 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f81a02b0 con 0x7fc3f8108f10 2024-05-27T07:48:12.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.218+0000 7fc3f5600700 1 -- 172.21.15.90:0/2427356292 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc3e402a070 con 0x7fc3f8108f10 2024-05-27T07:48:12.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.218+0000 7fc40e049b80 1 Processor -- start 2024-05-27T07:48:12.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.219+0000 7fc40e049b80 1 -- start start 2024-05-27T07:48:12.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.219+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x560040181330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.219+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x560040181870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.219+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040181e70 0x56004018e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.219+0000 7fc40e049b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5600400afe40 con 0x5600400ee9f0 2024-05-27T07:48:12.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.219+0000 7fc40e049b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56004009f9e0 con 0x560040002e40 2024-05-27T07:48:12.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.219+0000 7fc40e049b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5600400a3a20 con 0x560040181e70 2024-05-27T07:48:12.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.220+0000 7fc3fd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x560040181330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.220+0000 7fc3fd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x560040181330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39448/0 (socket says 172.21.15.90:39448) 2024-05-27T07:48:12.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.220+0000 7fc3fd600700 1 -- 172.21.15.90:0/294312455 learned_addr learned my addr 172.21.15.90:0/294312455 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:12.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.220+0000 7fc3f5600700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc3e00b2a70 0x7fc3e00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.220+0000 7fc3f7e00700 1 --2- 172.21.15.90:0/294312455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040181e70 0x56004018e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.220+0000 7fc3fcc00700 1 --2- 172.21.15.90:0/294312455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x560040181870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.220+0000 7fc3f5600700 1 -- 172.21.15.90:0/2427356292 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fc3e40e3610 con 0x7fc3f8108f10 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.220+0000 7fc3fd600700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc3e00b2a70 0x7fc3e00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.221+0000 7fc3fd600700 1 -- 172.21.15.90:0/294312455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040181e70 msgr2=0x56004018e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.221+0000 7fc3fd600700 1 --2- 172.21.15.90:0/294312455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040181e70 0x56004018e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.221+0000 7fc3fd600700 1 -- 172.21.15.90:0/294312455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 msgr2=0x560040181870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.221+0000 7fc3fd600700 1 --2- 172.21.15.90:0/294312455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x560040181870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.221+0000 7fc3fd600700 1 -- 172.21.15.90:0/294312455 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560040068420 con 0x5600400ee9f0 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.221+0000 7fc3fd600700 1 --2- 172.21.15.90:0/294312455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x560040181330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009e30 tx=0x7fc3f0004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7959c69304926436 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.221+0000 7fc3ea000700 1 -- 172.21.15.90:0/294312455 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0022030 con 0x5600400ee9f0 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.222+0000 7fc3ea000700 1 -- 172.21.15.90:0/294312455 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3f00099d0 con 0x5600400ee9f0 2024-05-27T07:48:12.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.222+0000 7fc3ea000700 1 -- 172.21.15.90:0/294312455 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f000e8b0 con 0x5600400ee9f0 2024-05-27T07:48:12.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.223+0000 7fc40e049b80 1 -- 172.21.15.90:0/294312455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 msgr2=0x560040181330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.223+0000 7fc3fd600700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc3e00b2a70 0x7fc3e00b4f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0025000 tx=0x7fc3f0021000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.223+0000 7fc40e049b80 1 --2- 172.21.15.90:0/294312455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x560040181330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009e30 tx=0x7fc3f0004320 comp rx=0 tx=0).stop 2024-05-27T07:48:12.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.223+0000 7fc40e049b80 1 -- 172.21.15.90:0/294312455 shutdown_connections 2024-05-27T07:48:12.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.223+0000 7fc40e049b80 1 --2- 172.21.15.90:0/294312455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040181e70 0x56004018e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.223+0000 7fc40e049b80 1 --2- 172.21.15.90:0/294312455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x560040181330 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.223+0000 7fc40e049b80 1 --2- 172.21.15.90:0/294312455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x560040181870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.223+0000 7fc40e049b80 1 -- 172.21.15.90:0/294312455 >> 172.21.15.90:0/294312455 conn(0x56004009c390 msgr2=0x56004017d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.223+0000 7fc40e049b80 1 -- 172.21.15.90:0/294312455 shutdown_connections 2024-05-27T07:48:12.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.223+0000 7fc40e049b80 1 -- 172.21.15.90:0/294312455 wait complete. 2024-05-27T07:48:12.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.224+0000 7fc40e049b80 1 -- start start 2024-05-27T07:48:12.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.224+0000 7fc40e049b80 1 Processor -- start 2024-05-27T07:48:12.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.225+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x56004019a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.225+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600400ee9f0 0x56004019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.225+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x560040194680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.226+0000 7fc40e049b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56003ffa5fa0 con 0x560040181e70 2024-05-27T07:48:12.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.226+0000 7fc40e049b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560040040760 con 0x560040002e40 2024-05-27T07:48:12.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.226+0000 7fc40e049b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56004018eb80 con 0x5600400ee9f0 2024-05-27T07:48:12.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.226+0000 7fc40e049b80 10 client.0 _ll_register_callbacks cb 0x560040049c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:12.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.226+0000 7fc3fd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x560040194680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.226+0000 7fc3fd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x560040194680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39458/0 (socket says 172.21.15.90:39458) 2024-05-27T07:48:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.226+0000 7fc3fd600700 1 -- 172.21.15.90:0/3501860560 learned_addr learned my addr 172.21.15.90:0/3501860560 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.226+0000 7fc3fcc00700 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x56004019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.226+0000 7fc3f7e00700 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600400ee9f0 0x56004019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.227+0000 7fc3fd600700 1 -- 172.21.15.90:0/3501860560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600400ee9f0 msgr2=0x56004019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.227+0000 7fc3fd600700 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600400ee9f0 0x56004019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.227+0000 7fc3fd600700 1 -- 172.21.15.90:0/3501860560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 msgr2=0x56004019a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.227+0000 7fc3fd600700 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x56004019a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.227+0000 7fc3fd600700 1 -- 172.21.15.90:0/3501860560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56003ffef8d0 con 0x560040181e70 2024-05-27T07:48:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.227+0000 7fc3fd600700 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x560040194680 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0003da0 tx=0x7fc3f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2b3f5499ea973c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.227+0000 7fc3dea00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.228+0000 7fc3dea00700 1 -- 172.21.15.90:0/3501860560 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0022070 con 0x560040181e70 2024-05-27T07:48:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.228+0000 7fc3dea00700 1 -- 172.21.15.90:0/3501860560 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fc3f0015070 con 0x560040181e70 2024-05-27T07:48:12.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.228+0000 7fc40e049b80 1 -- 172.21.15.90:0/3501860560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f00099d0 con 0x560040181e70 2024-05-27T07:48:12.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.228+0000 7fc3dea00700 1 -- 172.21.15.90:0/3501860560 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc3f0007d80 con 0x560040181e70 2024-05-27T07:48:12.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.228+0000 7fc40e049b80 10 client.15765 fetch_fsmap learned FSMap version 32 2024-05-27T07:48:12.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.228+0000 7fc40e049b80 1 -- 172.21.15.90:0/3501860560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560040070060 con 0x560040181e70 2024-05-27T07:48:12.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.229+0000 7fc3dea00700 1 -- 172.21.15.90:0/3501860560 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3f00099d0 con 0x560040181e70 2024-05-27T07:48:12.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.230+0000 7fc3dea00700 1 -- 172.21.15.90:0/3501860560 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc3f0039080 con 0x560040181e70 2024-05-27T07:48:12.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.230+0000 7fc40e049b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:48:12.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.230+0000 7fc40e049b80 10 client.15765 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:48:12.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.230+0000 7fc40e049b80 20 client.15765 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:12.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.230+0000 7fc40e049b80 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5600401a4260 0x5600401a6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.230+0000 7fc40e049b80 4 client.15765 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:12.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.230+0000 7fc40e049b80 1 -- 172.21.15.90:0/3501860560 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600400b1d90 con 0x5600401a4260 2024-05-27T07:48:12.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.231+0000 7fc3fcc00700 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5600401a4260 0x5600401a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.233+0000 7fc3fcc00700 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5600401a4260 0x5600401a6680 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=334eefb4b97ba140 server_cookie=4a5e00091931f07b in_seq=0 out_seq=0 2024-05-27T07:48:12.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.233+0000 7fc3dea00700 10 client.15765 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:48:12.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc3dea00700 1 -- 172.21.15.90:0/3501860560 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5600400b1d90 con 0x5600401a4260 2024-05-27T07:48:12.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc3dea00700 10 client.15765 handle_command_reply: tid=0 2024-05-27T07:48:12.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc40e049b80 1 client.15765 shutdown 2024-05-27T07:48:12.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc40e049b80 1 -- 172.21.15.90:0/3501860560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 msgr2=0x560040194680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc40e049b80 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x560040194680 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0003da0 tx=0x7fc3f0014000 comp rx=0 tx=0).stop 2024-05-27T07:48:12.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc40e049b80 1 -- 172.21.15.90:0/3501860560 shutdown_connections 2024-05-27T07:48:12.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc40e049b80 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5600401a4260 0x5600401a6680 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc40e049b80 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x560040194680 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc40e049b80 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x56004019a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc40e049b80 1 --2- 172.21.15.90:0/3501860560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600400ee9f0 0x56004019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.235+0000 7fc40e049b80 1 -- 172.21.15.90:0/3501860560 >> 172.21.15.90:0/3501860560 conn(0x56004009c390 msgr2=0x560040181310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.236+0000 7fc3dea00700 0 client.15765 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:48:12.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.236+0000 7fc40e049b80 1 -- 172.21.15.90:0/3501860560 shutdown_connections 2024-05-27T07:48:12.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.236+0000 7fc40e049b80 1 -- 172.21.15.90:0/3501860560 wait complete. 2024-05-27T07:48:12.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.236+0000 7fc40e049b80 20 client.15765 trim_cache size 0 max 16384 2024-05-27T07:48:12.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.256+0000 7fc40e049b80 1 Processor -- start 2024-05-27T07:48:12.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.256+0000 7fc40e049b80 1 -- start start 2024-05-27T07:48:12.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.257+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x56004018fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.257+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x56004017d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.257+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040181e70 0x5600401abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.257+0000 7fc40e049b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81ae1d0 con 0x5600400ee9f0 2024-05-27T07:48:12.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.257+0000 7fc40e049b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81afd10 con 0x560040002e40 2024-05-27T07:48:12.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.257+0000 7fc40e049b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81ae8a0 con 0x560040181e70 2024-05-27T07:48:12.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.257+0000 7fc3f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x56004017d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.257+0000 7fc3f7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x56004017d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39462/0 (socket says 172.21.15.90:39462) 2024-05-27T07:48:12.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.257+0000 7fc3f7e00700 1 -- 172.21.15.90:0/975093994 learned_addr learned my addr 172.21.15.90:0/975093994 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:12.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.258+0000 7fc3fcc00700 1 --2- 172.21.15.90:0/975093994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x56004018fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.258+0000 7fc3fd600700 1 --2- 172.21.15.90:0/975093994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040181e70 0x5600401abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.258+0000 7fc3f7e00700 1 -- 172.21.15.90:0/975093994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040181e70 msgr2=0x5600401abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.258+0000 7fc3f7e00700 1 --2- 172.21.15.90:0/975093994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040181e70 0x5600401abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.258+0000 7fc3f7e00700 1 -- 172.21.15.90:0/975093994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 msgr2=0x56004018fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.258+0000 7fc3f7e00700 1 --2- 172.21.15.90:0/975093994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x56004018fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.258+0000 7fc3f7e00700 1 -- 172.21.15.90:0/975093994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560040179240 con 0x5600400ee9f0 2024-05-27T07:48:12.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.258+0000 7fc3fcc00700 1 --2- 172.21.15.90:0/975093994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x56004018fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:12.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.258+0000 7fc3f7e00700 1 --2- 172.21.15.90:0/975093994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x56004017d590 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40aa560 tx=0x7fc3e40133f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10085f496b2e79d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.259+0000 7fc3e9600700 1 -- 172.21.15.90:0/975093994 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40e3040 con 0x5600400ee9f0 2024-05-27T07:48:12.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.259+0000 7fc3e9600700 1 -- 172.21.15.90:0/975093994 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3e40aa0e0 con 0x5600400ee9f0 2024-05-27T07:48:12.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.259+0000 7fc3e9600700 1 -- 172.21.15.90:0/975093994 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40b33a0 con 0x5600400ee9f0 2024-05-27T07:48:12.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.260+0000 7fc40e049b80 1 -- 172.21.15.90:0/975093994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 msgr2=0x56004017d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.260+0000 7fc40e049b80 1 --2- 172.21.15.90:0/975093994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x56004017d590 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40aa560 tx=0x7fc3e40133f0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.260+0000 7fc40e049b80 1 -- 172.21.15.90:0/975093994 shutdown_connections 2024-05-27T07:48:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.260+0000 7fc40e049b80 1 --2- 172.21.15.90:0/975093994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040181e70 0x5600401abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.260+0000 7fc40e049b80 1 --2- 172.21.15.90:0/975093994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560040002e40 0x56004018fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.260+0000 7fc40e049b80 1 --2- 172.21.15.90:0/975093994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600400ee9f0 0x56004017d590 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.260+0000 7fc40e049b80 1 -- 172.21.15.90:0/975093994 >> 172.21.15.90:0/975093994 conn(0x56004009c390 msgr2=0x56004017ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.260+0000 7fc40e049b80 1 -- 172.21.15.90:0/975093994 shutdown_connections 2024-05-27T07:48:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.260+0000 7fc40e049b80 1 -- 172.21.15.90:0/975093994 wait complete. 2024-05-27T07:48:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.261+0000 7fc40e049b80 1 -- start start 2024-05-27T07:48:12.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.261+0000 7fc40e049b80 1 Processor -- start 2024-05-27T07:48:12.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.261+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040002e40 0x56004019b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.261+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600400ee9f0 0x56004019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.261+0000 7fc40e049b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x56004019be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.261+0000 7fc40e049b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81af640 con 0x560040181e70 2024-05-27T07:48:12.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.261+0000 7fc40e049b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81aef70 con 0x5600400ee9f0 2024-05-27T07:48:12.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.261+0000 7fc40e049b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81ac150 con 0x560040002e40 2024-05-27T07:48:12.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc40e049b80 10 client.0 _ll_register_callbacks cb 0x560040049c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:12.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc3fd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x56004019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc3fd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x56004019be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39464/0 (socket says 172.21.15.90:39464) 2024-05-27T07:48:12.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc3fd600700 1 -- 172.21.15.90:0/419088420 learned_addr learned my addr 172.21.15.90:0/419088420 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc3fcc00700 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040002e40 0x56004019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc3f7e00700 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600400ee9f0 0x56004019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc3fd600700 1 -- 172.21.15.90:0/419088420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040002e40 msgr2=0x56004019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc3fd600700 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040002e40 0x56004019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc3fd600700 1 -- 172.21.15.90:0/419088420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600400ee9f0 msgr2=0x56004019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc3fd600700 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600400ee9f0 0x56004019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.262+0000 7fc3fd600700 1 -- 172.21.15.90:0/419088420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56004020cba0 con 0x560040181e70 2024-05-27T07:48:12.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.263+0000 7fc3fd600700 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x56004019be00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00049a0 tx=0x7fc3f0004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6736e044ed1d09 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.263+0000 7fc3dea00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.263+0000 7fc3dea00700 1 -- 172.21.15.90:0/419088420 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0022070 con 0x560040181e70 2024-05-27T07:48:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.263+0000 7fc3dea00700 1 -- 172.21.15.90:0/419088420 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fc3f002d040 con 0x560040181e70 2024-05-27T07:48:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.263+0000 7fc40e049b80 1 -- 172.21.15.90:0/419088420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e40aa0e0 con 0x560040181e70 2024-05-27T07:48:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.263+0000 7fc3dea00700 1 -- 172.21.15.90:0/419088420 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc3f0048050 con 0x560040181e70 2024-05-27T07:48:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.264+0000 7fc40e049b80 10 client.15771 fetch_fsmap learned FSMap version 32 2024-05-27T07:48:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.264+0000 7fc40e049b80 1 -- 172.21.15.90:0/419088420 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56004020de00 con 0x560040181e70 2024-05-27T07:48:12.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.264+0000 7fc3dea00700 1 -- 172.21.15.90:0/419088420 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3e40aa0e0 con 0x560040181e70 2024-05-27T07:48:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.264+0000 7fc3dea00700 1 -- 172.21.15.90:0/419088420 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc3f0027ab0 con 0x560040181e70 2024-05-27T07:48:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.264+0000 7fc40e049b80 10 client.15771 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T07:48:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.264+0000 7fc40e049b80 10 client.15771 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:48:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.264+0000 7fc40e049b80 20 client.15771 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.264+0000 7fc40e049b80 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56004020e420 0x560040210840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.264+0000 7fc40e049b80 4 client.15771 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.264+0000 7fc40e049b80 1 -- 172.21.15.90:0/419088420 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600400ab570 con 0x56004020e420 2024-05-27T07:48:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.265+0000 7fc3fcc00700 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56004020e420 0x560040210840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:12.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.266+0000 7fc3fcc00700 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56004020e420 0x560040210840 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=236214dbeb07018a server_cookie=51c3a4293f900af7 in_seq=0 out_seq=0 2024-05-27T07:48:12.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.266+0000 7fc3dea00700 10 client.15771 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:48:12.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc3dea00700 1 -- 172.21.15.90:0/419088420 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600400ab570 con 0x56004020e420 2024-05-27T07:48:12.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc3dea00700 10 client.15771 handle_command_reply: tid=0 2024-05-27T07:48:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc40e049b80 1 client.15771 shutdown 2024-05-27T07:48:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc40e049b80 1 -- 172.21.15.90:0/419088420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 msgr2=0x56004019be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc40e049b80 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x56004019be00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00049a0 tx=0x7fc3f0004d60 comp rx=0 tx=0).stop 2024-05-27T07:48:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc40e049b80 1 -- 172.21.15.90:0/419088420 shutdown_connections 2024-05-27T07:48:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc40e049b80 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56004020e420 0x560040210840 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc40e049b80 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560040181e70 0x56004019be00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc40e049b80 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560040002e40 0x56004019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc40e049b80 1 --2- 172.21.15.90:0/419088420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600400ee9f0 0x56004019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc3dea00700 0 client.15771 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:48:12.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc40e049b80 1 -- 172.21.15.90:0/419088420 >> 172.21.15.90:0/419088420 conn(0x56004009c390 msgr2=0x560040182ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.267+0000 7fc40e049b80 1 -- 172.21.15.90:0/419088420 shutdown_connections 2024-05-27T07:48:12.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.268+0000 7fc40e049b80 1 -- 172.21.15.90:0/419088420 wait complete. 2024-05-27T07:48:12.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.268+0000 7fc40e049b80 20 client.15771 trim_cache size 0 max 16384 2024-05-27T07:48:12.268 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:48:12.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.268+0000 7fc3ff600700 1 -- 172.21.15.90:0/2427356292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc3e00b2a70 msgr2=0x7fc3e00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.268+0000 7fc3ff600700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc3e00b2a70 0x7fc3e00b4f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0025000 tx=0x7fc3f0021000 comp rx=0 tx=0).stop 2024-05-27T07:48:12.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.268+0000 7fc3ff600700 1 -- 172.21.15.90:0/2427356292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f8108f10 msgr2=0x7fc3f819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:12.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.268+0000 7fc3ff600700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f8108f10 0x7fc3f819b360 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4006010 tx=0x7fc3e4004790 comp rx=0 tx=0).stop 2024-05-27T07:48:12.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.269+0000 7fc3ff600700 1 -- 172.21.15.90:0/2427356292 shutdown_connections 2024-05-27T07:48:12.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.269+0000 7fc3ff600700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc3e00b2a70 0x7fc3e00b4f80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.269+0000 7fc3ff600700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3f8109860 0x7fc3f819f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.269+0000 7fc3ff600700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3f81045e0 0x7fc3f819adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.269+0000 7fc3ff600700 1 --2- 172.21.15.90:0/2427356292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f8108f10 0x7fc3f819b360 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:12.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.269+0000 7fc3ff600700 1 -- 172.21.15.90:0/2427356292 >> 172.21.15.90:0/2427356292 conn(0x7fc3f8100030 msgr2=0x7fc3f8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:12.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.269+0000 7fc3ff600700 1 -- 172.21.15.90:0/2427356292 shutdown_connections 2024-05-27T07:48:12.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:12.269+0000 7fc3ff600700 1 -- 172.21.15.90:0/2427356292 wait complete. 2024-05-27T07:48:12.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:12 smithi136 ceph-mon[35324]: pgmap v56: 129 pgs: 129 active+clean; 255 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 31 op/s 2024-05-27T07:48:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:12 smithi129 ceph-mon[35254]: pgmap v56: 129 pgs: 129 active+clean; 255 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 12 MiB/s wr, 31 op/s 2024-05-27T07:48:13.406 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:48:13.903 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:13.907 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:13 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 2. 2024-05-27T07:48:13.908 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:13 smithi136 systemd[1]: Stopped Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:48:13.908 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:13 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 2.753s CPU time. 2024-05-27T07:48:13.908 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:13 smithi136 systemd[1]: Starting Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:48:13.922 INFO:teuthology.orchestra.run.smithi090.stderr:1000000 of 3200000 tuples (31%) done (elapsed 29.97 s, remaining 65.94 s) 2024-05-27T07:48:14.205 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:14 smithi136 ceph-mon[35324]: pgmap v57: 129 pgs: 129 active+clean; 291 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 44 op/s 2024-05-27T07:48:14.205 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:13 smithi136 podman[70663]: 2024-05-27 07:48:13.813194248 +0000 UTC m=+0.035345331 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:48:14.205 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:13 smithi136 podman[70663]: 2024-05-27 07:48:13.950871631 +0000 UTC m=+0.173022736 container create 28952448acc411f26d5d4bd7bd23c6eeaaf00d153e24ed1eb947d46cac35f260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T07:48:14.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:14 smithi090 ceph-mon[31429]: pgmap v57: 129 pgs: 129 active+clean; 291 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 44 op/s 2024-05-27T07:48:14.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:14 smithi136 podman[70663]: 2024-05-27 07:48:14.204686953 +0000 UTC m=+0.426838076 container init 28952448acc411f26d5d4bd7bd23c6eeaaf00d153e24ed1eb947d46cac35f260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:48:14.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:14 smithi136 podman[70663]: 2024-05-27 07:48:14.217514812 +0000 UTC m=+0.439665947 container start 28952448acc411f26d5d4bd7bd23c6eeaaf00d153e24ed1eb947d46cac35f260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:48:14.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:14 smithi136 bash[70663]: 28952448acc411f26d5d4bd7bd23c6eeaaf00d153e24ed1eb947d46cac35f260 2024-05-27T07:48:14.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:14 smithi136 systemd[1]: Started Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:48:14.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:14 smithi136 ceph-mds[70682]: -- 172.21.15.136:0/841724134 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a2f89da40 con 0x561a2f8e5180 2024-05-27T07:48:14.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: starting mds.c at 2024-05-27T07:48:14.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:14 smithi129 ceph-mon[35254]: pgmap v57: 129 pgs: 129 active+clean; 291 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 44 op/s 2024-05-27T07:48:15.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:15 smithi090 ceph-mon[31429]: daemon mds.c restarted 2024-05-27T07:48:15.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-mon[35324]: daemon mds.c restarted 2024-05-27T07:48:15.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f3865c00700 time 2024-05-27T07:48:15.213120+0000 2024-05-27T07:48:15.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:15.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:15.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f387b6c1ba6] 2024-05-27T07:48:15.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f387b6c1dd1] 2024-05-27T07:48:15.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561a2e02a23d] 2024-05-27T07:48:15.500 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561a2e015971] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 5: (SnapRealm::check_cache() const+0x13e) [0x561a2e013fa0] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561a2e014651] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561a2dfa9347] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561a2dca587e] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561a2dcaaf2f] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561a2dcb5969] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 11: (MDLog::_replay_thread()+0x1f38) [0x561a2e04e48c] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561a2e0579cb] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 13: (Thread::entry_wrapper()+0x43) [0x7f387b69072d] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 14: (Thread::_entry_func(void*)+0xd) [0x7f387b690749] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f387a2341ca] 2024-05-27T07:48:15.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 16: clone() 2024-05-27T07:48:15.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.214+0000 7f3865c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f3865c00700 time 2024-05-27T07:48:15.213120+0000 2024-05-27T07:48:15.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:15.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:15.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f387b6c1ba6] 2024-05-27T07:48:15.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f387b6c1dd1] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561a2e02a23d] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561a2e015971] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 5: (SnapRealm::check_cache() const+0x13e) [0x561a2e013fa0] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561a2e014651] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561a2dfa9347] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561a2dca587e] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561a2dcaaf2f] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561a2dcb5969] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 11: (MDLog::_replay_thread()+0x1f38) [0x561a2e04e48c] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561a2e0579cb] 2024-05-27T07:48:15.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 13: (Thread::entry_wrapper()+0x43) [0x7f387b69072d] 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 14: (Thread::_entry_func(void*)+0xd) [0x7f387b690749] 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f387a2341ca] 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 16: clone() 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: *** Caught signal (Aborted) ** 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: in thread 7f3865c00700 thread_name:md_log_replay 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f387a23ed20] 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2: gsignal() 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 3: abort() 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f387b6c1ced] 2024-05-27T07:48:15.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f387b6c1dd1] 2024-05-27T07:48:15.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561a2e02a23d] 2024-05-27T07:48:15.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561a2e015971] 2024-05-27T07:48:15.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 8: (SnapRealm::check_cache() const+0x13e) [0x561a2e013fa0] 2024-05-27T07:48:15.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561a2e014651] 2024-05-27T07:48:15.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561a2dfa9347] 2024-05-27T07:48:15.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561a2dca587e] 2024-05-27T07:48:15.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561a2dcaaf2f] 2024-05-27T07:48:15.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561a2dcb5969] 2024-05-27T07:48:15.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 14: (MDLog::_replay_thread()+0x1f38) [0x561a2e04e48c] 2024-05-27T07:48:15.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561a2e0579cb] 2024-05-27T07:48:15.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 16: (Thread::entry_wrapper()+0x43) [0x7f387b69072d] 2024-05-27T07:48:15.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 17: (Thread::_entry_func(void*)+0xd) [0x7f387b690749] 2024-05-27T07:48:15.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f387a2341ca] 2024-05-27T07:48:15.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 19: clone() 2024-05-27T07:48:15.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.217+0000 7f3865c00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:48:15.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: in thread 7f3865c00700 thread_name:md_log_replay 2024-05-27T07:48:15.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f387a23ed20] 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2: gsignal() 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 3: abort() 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f387b6c1ced] 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f387b6c1dd1] 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561a2e02a23d] 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561a2e015971] 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 8: (SnapRealm::check_cache() const+0x13e) [0x561a2e013fa0] 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561a2e014651] 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561a2dfa9347] 2024-05-27T07:48:15.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561a2dca587e] 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561a2dcaaf2f] 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561a2dcb5969] 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 14: (MDLog::_replay_thread()+0x1f38) [0x561a2e04e48c] 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561a2e0579cb] 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 16: (Thread::entry_wrapper()+0x43) [0x7f387b69072d] 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 17: (Thread::_entry_func(void*)+0xd) [0x7f387b690749] 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f387a2341ca] 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 19: clone() 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: -1> 2024-05-27T07:48:15.214+0000 7f3865c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f3865c00700 time 2024-05-27T07:48:15.213120+0000 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:15.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:15.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f387b6c1ba6] 2024-05-27T07:48:15.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f387b6c1dd1] 2024-05-27T07:48:15.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561a2e02a23d] 2024-05-27T07:48:15.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561a2e015971] 2024-05-27T07:48:15.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 5: (SnapRealm::check_cache() const+0x13e) [0x561a2e013fa0] 2024-05-27T07:48:15.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561a2e014651] 2024-05-27T07:48:15.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561a2dfa9347] 2024-05-27T07:48:15.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561a2dca587e] 2024-05-27T07:48:15.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561a2dcaaf2f] 2024-05-27T07:48:15.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561a2dcb5969] 2024-05-27T07:48:15.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 11: (MDLog::_replay_thread()+0x1f38) [0x561a2e04e48c] 2024-05-27T07:48:15.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561a2e0579cb] 2024-05-27T07:48:15.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 13: (Thread::entry_wrapper()+0x43) [0x7f387b69072d] 2024-05-27T07:48:15.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 14: (Thread::_entry_func(void*)+0xd) [0x7f387b690749] 2024-05-27T07:48:15.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f387a2341ca] 2024-05-27T07:48:15.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 16: clone() 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 0> 2024-05-27T07:48:15.217+0000 7f3865c00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: in thread 7f3865c00700 thread_name:md_log_replay 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f387a23ed20] 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2: gsignal() 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 3: abort() 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f387b6c1ced] 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f387b6c1dd1] 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561a2e02a23d] 2024-05-27T07:48:15.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561a2e015971] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 8: (SnapRealm::check_cache() const+0x13e) [0x561a2e013fa0] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561a2e014651] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561a2dfa9347] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561a2dca587e] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561a2dcaaf2f] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561a2dcb5969] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 14: (MDLog::_replay_thread()+0x1f38) [0x561a2e04e48c] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561a2e0579cb] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 16: (Thread::entry_wrapper()+0x43) [0x7f387b69072d] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 17: (Thread::_entry_func(void*)+0xd) [0x7f387b690749] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f387a2341ca] 2024-05-27T07:48:15.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 19: clone() 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: -1> 2024-05-27T07:48:15.214+0000 7f3865c00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f3865c00700 time 2024-05-27T07:48:15.213120+0000 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f387b6c1ba6] 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f387b6c1dd1] 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561a2e02a23d] 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561a2e015971] 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 5: (SnapRealm::check_cache() const+0x13e) [0x561a2e013fa0] 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561a2e014651] 2024-05-27T07:48:15.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561a2dfa9347] 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561a2dca587e] 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561a2dcaaf2f] 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561a2dcb5969] 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 11: (MDLog::_replay_thread()+0x1f38) [0x561a2e04e48c] 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561a2e0579cb] 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 13: (Thread::entry_wrapper()+0x43) [0x7f387b69072d] 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 14: (Thread::_entry_func(void*)+0xd) [0x7f387b690749] 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f387a2341ca] 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 16: clone() 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 0> 2024-05-27T07:48:15.217+0000 7f3865c00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: in thread 7f3865c00700 thread_name:md_log_replay 2024-05-27T07:48:15.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f387a23ed20] 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2: gsignal() 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 3: abort() 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f387b6c1ced] 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f387b6c1dd1] 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561a2e02a23d] 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561a2e015971] 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 8: (SnapRealm::check_cache() const+0x13e) [0x561a2e013fa0] 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561a2e014651] 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561a2dfa9347] 2024-05-27T07:48:15.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561a2dca587e] 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561a2dcaaf2f] 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561a2dcb5969] 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 14: (MDLog::_replay_thread()+0x1f38) [0x561a2e04e48c] 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561a2e0579cb] 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 16: (Thread::entry_wrapper()+0x43) [0x7f387b69072d] 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 17: (Thread::_entry_func(void*)+0xd) [0x7f387b690749] 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f387a2341ca] 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 19: clone() 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70678]: 2024-05-27T07:48:15.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:15 smithi136 podman[70713]: 2024-05-27 07:48:15.300653451 +0000 UTC m=+0.034126441 container died 28952448acc411f26d5d4bd7bd23c6eeaaf00d153e24ed1eb947d46cac35f260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-27T07:48:15.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:15 smithi129 ceph-mon[35254]: daemon mds.c restarted 2024-05-27T07:48:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:16 smithi090 ceph-mon[31429]: pgmap v58: 129 pgs: 129 active+clean; 291 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-05-27T07:48:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:16 smithi090 ceph-mon[31429]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T07:48:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:16 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.136:6836/1486871974,v1:172.21.15.136:6837/1486871974] up:boot 2024-05-27T07:48:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:16 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:48:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:16 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:48:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:16 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:48:16.499 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:16 smithi136 podman[70713]: 2024-05-27 07:48:16.051899312 +0000 UTC m=+0.785372288 container remove 28952448acc411f26d5d4bd7bd23c6eeaaf00d153e24ed1eb947d46cac35f260 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-27T07:48:16.499 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:16 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:48:16.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:16 smithi136 ceph-mon[35324]: pgmap v58: 129 pgs: 129 active+clean; 291 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-05-27T07:48:16.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:16 smithi136 ceph-mon[35324]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T07:48:16.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:16 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.136:6836/1486871974,v1:172.21.15.136:6837/1486871974] up:boot 2024-05-27T07:48:16.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:16 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:48:16.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:16 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:48:16.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:16 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:48:16.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:16 smithi129 ceph-mon[35254]: pgmap v58: 129 pgs: 129 active+clean; 291 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-05-27T07:48:16.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:16 smithi129 ceph-mon[35254]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T07:48:16.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:16 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.136:6836/1486871974,v1:172.21.15.136:6837/1486871974] up:boot 2024-05-27T07:48:16.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:16 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:48:16.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:16 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:48:16.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:16 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:48:16.999 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:16 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:48:16.999 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:16 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.418s CPU time. 2024-05-27T07:48:17.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.836+0000 7fe7ea200700 1 -- 172.21.15.90:0/1247975106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e4075af0 msgr2=0x7fe7e4075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:17.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.836+0000 7fe7ea200700 1 --2- 172.21.15.90:0/1247975106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e4075af0 0x7fe7e4075f00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009990 tx=0x7fe7d4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.836+0000 7fe7ea200700 1 -- 172.21.15.90:0/1247975106 shutdown_connections 2024-05-27T07:48:17.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.836+0000 7fe7ea200700 1 --2- 172.21.15.90:0/1247975106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4110bf0 0x7fe7e4113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.836+0000 7fe7ea200700 1 --2- 172.21.15.90:0/1247975106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e4075af0 0x7fe7e4075f00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.836+0000 7fe7ea200700 1 --2- 172.21.15.90:0/1247975106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e4076440 0x7fe7e41106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.836+0000 7fe7ea200700 1 -- 172.21.15.90:0/1247975106 >> 172.21.15.90:0/1247975106 conn(0x7fe7e4100030 msgr2=0x7fe7e4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:17.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.836+0000 7fe7ea200700 1 -- 172.21.15.90:0/1247975106 shutdown_connections 2024-05-27T07:48:17.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.837+0000 7fe7ea200700 1 -- 172.21.15.90:0/1247975106 wait complete. 2024-05-27T07:48:17.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.838+0000 7fe7ea200700 1 Processor -- start 2024-05-27T07:48:17.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.838+0000 7fe7ea200700 1 -- start start 2024-05-27T07:48:17.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.839+0000 7fe7ea200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e4075af0 0x7fe7e419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.839+0000 7fe7ea200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4076440 0x7fe7e419d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.839+0000 7fe7ea200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e4110bf0 0x7fe7e41a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.839+0000 7fe7ea200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4116580 con 0x7fe7e4110bf0 2024-05-27T07:48:17.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.839+0000 7fe7ea200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4116400 con 0x7fe7e4075af0 2024-05-27T07:48:17.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.839+0000 7fe7ea200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4116700 con 0x7fe7e4076440 2024-05-27T07:48:17.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e4110bf0 0x7fe7e41a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e4110bf0 0x7fe7e41a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39480/0 (socket says 172.21.15.90:39480) 2024-05-27T07:48:17.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e3e00700 1 -- 172.21.15.90:0/1083314286 learned_addr learned my addr 172.21.15.90:0/1083314286 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:17.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e3400700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e4075af0 0x7fe7e419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e2a00700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4076440 0x7fe7e419d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e3e00700 1 -- 172.21.15.90:0/1083314286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4076440 msgr2=0x7fe7e419d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:17.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e3e00700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4076440 0x7fe7e419d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e3e00700 1 -- 172.21.15.90:0/1083314286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e4075af0 msgr2=0x7fe7e419cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:17.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e3e00700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e4075af0 0x7fe7e419cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e3e00700 1 -- 172.21.15.90:0/1083314286 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e41a1e20 con 0x7fe7e4110bf0 2024-05-27T07:48:17.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.840+0000 7fe7e2a00700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4076440 0x7fe7e419d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:17.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.841+0000 7fe7e3e00700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e4110bf0 0x7fe7e41a1780 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0094f0 tx=0x7fe7dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:17.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.841+0000 7fe7dbe00700 1 -- 172.21.15.90:0/1083314286 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc00ddd0 con 0x7fe7e4110bf0 2024-05-27T07:48:17.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.841+0000 7fe7ea200700 1 -- 172.21.15.90:0/1083314286 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e41a2110 con 0x7fe7e4110bf0 2024-05-27T07:48:17.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.841+0000 7fe7ea200700 1 -- 172.21.15.90:0/1083314286 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e41a9a30 con 0x7fe7e4110bf0 2024-05-27T07:48:17.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.841+0000 7fe7dbe00700 1 -- 172.21.15.90:0/1083314286 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7dc009e20 con 0x7fe7e4110bf0 2024-05-27T07:48:17.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.841+0000 7fe7dbe00700 1 -- 172.21.15.90:0/1083314286 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc015400 con 0x7fe7e4110bf0 2024-05-27T07:48:17.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.848+0000 7fe7dbe00700 1 -- 172.21.15.90:0/1083314286 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe7dc0155a0 con 0x7fe7e4110bf0 2024-05-27T07:48:17.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.848+0000 7fe7f8c9db80 1 Processor -- start 2024-05-27T07:48:17.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.848+0000 7fe7f8c9db80 1 -- start start 2024-05-27T07:48:17.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.849+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c2d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.849+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c2d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.849+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4c2ddb0 0x558cc4c42940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.849+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558cc4b5be40 con 0x558cc4b9a9f0 2024-05-27T07:48:17.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.849+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558cc4b4b9e0 con 0x558cc4aaee40 2024-05-27T07:48:17.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.849+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558cc4b4fa20 con 0x558cc4c2ddb0 2024-05-27T07:48:17.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.849+0000 7fe7e2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c2d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.849+0000 7fe7e2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c2d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39486/0 (socket says 172.21.15.90:39486) 2024-05-27T07:48:17.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.849+0000 7fe7e2a00700 1 -- 172.21.15.90:0/553534300 learned_addr learned my addr 172.21.15.90:0/553534300 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:17.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.850+0000 7fe7e3400700 1 --2- 172.21.15.90:0/553534300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c2d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.850+0000 7fe7e2a00700 1 -- 172.21.15.90:0/553534300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4c2ddb0 msgr2=0x558cc4c42940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:17.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.850+0000 7fe7e2a00700 1 --2- 172.21.15.90:0/553534300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4c2ddb0 0x558cc4c42940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.850+0000 7fe7e2a00700 1 -- 172.21.15.90:0/553534300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 msgr2=0x558cc4c2d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:17.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.850+0000 7fe7e2a00700 1 --2- 172.21.15.90:0/553534300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c2d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.850+0000 7fe7e2a00700 1 -- 172.21.15.90:0/553534300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc4b14420 con 0x558cc4b9a9f0 2024-05-27T07:48:17.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.850+0000 7fe7dbe00700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7c80b2a70 0x7fe7c80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.850+0000 7fe7e2a00700 1 --2- 172.21.15.90:0/553534300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c2d250 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000bde0 tx=0x7fe7d0011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b72dfb077adc164 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:17.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.851+0000 7fe7dbe00700 1 -- 172.21.15.90:0/1083314286 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe7dc0e2330 con 0x7fe7e4110bf0 2024-05-27T07:48:17.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.851+0000 7fe7e3400700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7c80b2a70 0x7fe7c80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.851+0000 7fe7d8c00700 1 -- 172.21.15.90:0/553534300 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0016070 con 0x558cc4b9a9f0 2024-05-27T07:48:17.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.851+0000 7fe7d8c00700 1 -- 172.21.15.90:0/553534300 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7d00044c0 con 0x558cc4b9a9f0 2024-05-27T07:48:17.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.851+0000 7fe7d8c00700 1 -- 172.21.15.90:0/553534300 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0003e00 con 0x558cc4b9a9f0 2024-05-27T07:48:17.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.852+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/553534300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 msgr2=0x558cc4c2d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:17.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.852+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/553534300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c2d250 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000bde0 tx=0x7fe7d0011660 comp rx=0 tx=0).stop 2024-05-27T07:48:17.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.852+0000 7fe7e3400700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7c80b2a70 0x7fe7c80b4f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40098e0 tx=0x7fe7d4005040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:17.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.852+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/553534300 shutdown_connections 2024-05-27T07:48:17.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.852+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/553534300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4c2ddb0 0x558cc4c42940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.852+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/553534300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c2d250 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.852+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/553534300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c2d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.852+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/553534300 >> 172.21.15.90:0/553534300 conn(0x558cc4b48390 msgr2=0x558cc4c294c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:17.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.852+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/553534300 shutdown_connections 2024-05-27T07:48:17.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.852+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/553534300 wait complete. 2024-05-27T07:48:17.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.853+0000 7fe7f8c9db80 1 -- start start 2024-05-27T07:48:17.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.853+0000 7fe7f8c9db80 1 Processor -- start 2024-05-27T07:48:17.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.854+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c3d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.854+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4b9a9f0 0x558cc4c3de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.854+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c3e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.854+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558cc4a51fa0 con 0x558cc4c2ddb0 2024-05-27T07:48:17.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.854+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558cc4aec760 con 0x558cc4aaee40 2024-05-27T07:48:17.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.854+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558cc4c43430 con 0x558cc4b9a9f0 2024-05-27T07:48:17.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.854+0000 7fe7f8c9db80 10 client.0 _ll_register_callbacks cb 0x558cc4af5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:17.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.855+0000 7fe7e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c3e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.855+0000 7fe7e3400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c3d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.855+0000 7fe7e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c3e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39492/0 (socket says 172.21.15.90:39492) 2024-05-27T07:48:17.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.855+0000 7fe7e3e00700 1 -- 172.21.15.90:0/2931610629 learned_addr learned my addr 172.21.15.90:0/2931610629 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:17.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.855+0000 7fe7e2a00700 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4b9a9f0 0x558cc4c3de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.856+0000 7fe7e3e00700 1 -- 172.21.15.90:0/2931610629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4b9a9f0 msgr2=0x558cc4c3de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:17.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.856+0000 7fe7e3e00700 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4b9a9f0 0x558cc4c3de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.856+0000 7fe7e3e00700 1 -- 172.21.15.90:0/2931610629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 msgr2=0x558cc4c3d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:17.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.856+0000 7fe7e3e00700 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c3d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.856+0000 7fe7e3e00700 1 -- 172.21.15.90:0/2931610629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc4a9b8d0 con 0x558cc4c2ddb0 2024-05-27T07:48:17.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.856+0000 7fe7e3e00700 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c3e380 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009710 tx=0x7fe7dc0a9ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f136a4e0b035822 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:17.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.856+0000 7fe7cd600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:17.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.856+0000 7fe7cd600700 1 -- 172.21.15.90:0/2931610629 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc0c3060 con 0x558cc4c2ddb0 2024-05-27T07:48:17.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.856+0000 7fe7cd600700 1 -- 172.21.15.90:0/2931610629 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe7dc0b1c70 con 0x558cc4c2ddb0 2024-05-27T07:48:17.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.856+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2931610629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d00044c0 con 0x558cc4c2ddb0 2024-05-27T07:48:17.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7cd600700 1 -- 172.21.15.90:0/2931610629 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe7dc0c8050 con 0x558cc4c2ddb0 2024-05-27T07:48:17.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7f8c9db80 10 client.15786 fetch_fsmap learned FSMap version 34 2024-05-27T07:48:17.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2931610629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558cc4b1c060 con 0x558cc4c2ddb0 2024-05-27T07:48:17.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7cd600700 1 -- 172.21.15.90:0/2931610629 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7d00044c0 con 0x558cc4c2ddb0 2024-05-27T07:48:17.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7cd600700 1 -- 172.21.15.90:0/2931610629 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe7dc012070 con 0x558cc4c2ddb0 2024-05-27T07:48:17.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7f8c9db80 10 client.15786 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T07:48:17.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7f8c9db80 10 client.15786 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:48:17.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7f8c9db80 20 client.15786 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:17.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558cc4c506f0 0x558cc4c52b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7f8c9db80 4 client.15786 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:17.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.857+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2931610629 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cc4b5dd90 con 0x558cc4c506f0 2024-05-27T07:48:17.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.858+0000 7fe7e3400700 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558cc4c506f0 0x558cc4c52b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.859+0000 7fe7e3400700 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558cc4c506f0 0x558cc4c52b10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ef45bfb9db4fca9 server_cookie=b534dbfc6ddc938d in_seq=0 out_seq=0 2024-05-27T07:48:17.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.859+0000 7fe7cd600700 10 client.15786 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:48:17.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.861+0000 7fe7cd600700 1 -- 172.21.15.90:0/2931610629 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558cc4b5dd90 con 0x558cc4c506f0 2024-05-27T07:48:17.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.861+0000 7fe7cd600700 10 client.15786 handle_command_reply: tid=0 2024-05-27T07:48:17.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.861+0000 7fe7f8c9db80 1 client.15786 shutdown 2024-05-27T07:48:17.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2931610629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 msgr2=0x558cc4c3e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c3e380 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009710 tx=0x7fe7dc0a9ae0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2931610629 shutdown_connections 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558cc4c506f0 0x558cc4c52b10 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c3e380 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c3d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/2931610629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4b9a9f0 0x558cc4c3de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2931610629 >> 172.21.15.90:0/2931610629 conn(0x558cc4b48390 msgr2=0x558cc4c29290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7cd600700 0 client.15786 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2931610629 shutdown_connections 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2931610629 wait complete. 2024-05-27T07:48:17.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.862+0000 7fe7f8c9db80 20 client.15786 trim_cache size 0 max 16384 2024-05-27T07:48:17.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.880+0000 7fe7f8c9db80 1 Processor -- start 2024-05-27T07:48:17.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.880+0000 7fe7f8c9db80 1 -- start start 2024-05-27T07:48:17.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4aaee40 0x558cc4c42800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4b9a9f0 0x558cc4c29590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c4c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41ae1b0 con 0x558cc4c2ddb0 2024-05-27T07:48:17.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41afcf0 con 0x558cc4b9a9f0 2024-05-27T07:48:17.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41ae880 con 0x558cc4aaee40 2024-05-27T07:48:17.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c4c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7e3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c4c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39506/0 (socket says 172.21.15.90:39506) 2024-05-27T07:48:17.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7e3e00700 1 -- 172.21.15.90:0/2882561058 learned_addr learned my addr 172.21.15.90:0/2882561058 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:17.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7e3400700 1 --2- 172.21.15.90:0/2882561058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4aaee40 0x558cc4c42800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.881+0000 7fe7e2a00700 1 --2- 172.21.15.90:0/2882561058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4b9a9f0 0x558cc4c29590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.882+0000 7fe7e3e00700 1 -- 172.21.15.90:0/2882561058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4aaee40 msgr2=0x558cc4c42800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:17.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.882+0000 7fe7e3e00700 1 --2- 172.21.15.90:0/2882561058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4aaee40 0x558cc4c42800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.882+0000 7fe7e3e00700 1 -- 172.21.15.90:0/2882561058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4b9a9f0 msgr2=0x558cc4c29590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:17.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.882+0000 7fe7e3e00700 1 --2- 172.21.15.90:0/2882561058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4b9a9f0 0x558cc4c29590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.882+0000 7fe7e3e00700 1 -- 172.21.15.90:0/2882561058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc4c2e1c0 con 0x558cc4c2ddb0 2024-05-27T07:48:17.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.882+0000 7fe7e3e00700 1 --2- 172.21.15.90:0/2882561058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c4c720 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0a9b50 tx=0x7fe7dc004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a08931509f79dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:17.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.882+0000 7fe7cfe00700 1 -- 172.21.15.90:0/2882561058 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc087070 con 0x558cc4c2ddb0 2024-05-27T07:48:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.882+0000 7fe7cfe00700 1 -- 172.21.15.90:0/2882561058 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7dc0b2100 con 0x558cc4c2ddb0 2024-05-27T07:48:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.882+0000 7fe7cfe00700 1 -- 172.21.15.90:0/2882561058 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc0ad1c0 con 0x558cc4c2ddb0 2024-05-27T07:48:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.883+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2882561058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 msgr2=0x558cc4c4c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.883+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/2882561058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c4c720 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0a9b50 tx=0x7fe7dc004ce0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.883+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2882561058 shutdown_connections 2024-05-27T07:48:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.883+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/2882561058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4c2ddb0 0x558cc4c4c720 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.883+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/2882561058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4aaee40 0x558cc4c42800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.883+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/2882561058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4b9a9f0 0x558cc4c29590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.883+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2882561058 >> 172.21.15.90:0/2882561058 conn(0x558cc4b48390 msgr2=0x558cc4c28e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.884+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2882561058 shutdown_connections 2024-05-27T07:48:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.884+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/2882561058 wait complete. 2024-05-27T07:48:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.884+0000 7fe7f8c9db80 1 -- start start 2024-05-27T07:48:17.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.885+0000 7fe7f8c9db80 1 Processor -- start 2024-05-27T07:48:17.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.885+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.885+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c55de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.885+0000 7fe7f8c9db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4c2ddb0 0x558cc4c50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.885+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41af620 con 0x558cc4b9a9f0 2024-05-27T07:48:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.885+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41aef50 con 0x558cc4aaee40 2024-05-27T07:48:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.885+0000 7fe7f8c9db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41ac130 con 0x558cc4c2ddb0 2024-05-27T07:48:17.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.886+0000 7fe7f8c9db80 10 client.0 _ll_register_callbacks cb 0x558cc4af5c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.886+0000 7fe7e2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c55de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.886+0000 7fe7e2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c55de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39512/0 (socket says 172.21.15.90:39512) 2024-05-27T07:48:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.886+0000 7fe7e2a00700 1 -- 172.21.15.90:0/3528027945 learned_addr learned my addr 172.21.15.90:0/3528027945 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.886+0000 7fe7e3e00700 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4c2ddb0 0x558cc4c50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.886+0000 7fe7e3400700 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.887+0000 7fe7e2a00700 1 -- 172.21.15.90:0/3528027945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4c2ddb0 msgr2=0x558cc4c50d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.887+0000 7fe7e2a00700 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4c2ddb0 0x558cc4c50d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.887+0000 7fe7e2a00700 1 -- 172.21.15.90:0/3528027945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 msgr2=0x558cc4c558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.887+0000 7fe7e2a00700 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.887+0000 7fe7e2a00700 1 -- 172.21.15.90:0/3528027945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc4c25240 con 0x558cc4b9a9f0 2024-05-27T07:48:17.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.887+0000 7fe7e2a00700 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c55de0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00039e0 tx=0x7fe7d0003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41b101dc9157d24 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:17.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.887+0000 7fe7cd600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:17.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.887+0000 7fe7cd600700 1 -- 172.21.15.90:0/3528027945 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d002a060 con 0x558cc4b9a9f0 2024-05-27T07:48:17.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.887+0000 7fe7cd600700 1 -- 172.21.15.90:0/3528027945 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe7d0005240 con 0x558cc4b9a9f0 2024-05-27T07:48:17.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.887+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/3528027945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7dc0b2100 con 0x558cc4b9a9f0 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.888+0000 7fe7cd600700 1 -- 172.21.15.90:0/3528027945 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe7d0031050 con 0x558cc4b9a9f0 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.888+0000 7fe7f8c9db80 10 client.15792 fetch_fsmap learned FSMap version 34 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.888+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/3528027945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558cc4c4de90 con 0x558cc4b9a9f0 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.888+0000 7fe7cd600700 1 -- 172.21.15.90:0/3528027945 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7dc0b2100 con 0x558cc4b9a9f0 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.888+0000 7fe7cd600700 1 -- 172.21.15.90:0/3528027945 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe7d0025410 con 0x558cc4b9a9f0 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.888+0000 7fe7f8c9db80 10 client.15792 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.889+0000 7fe7f8c9db80 10 client.15792 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.889+0000 7fe7f8c9db80 20 client.15792 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.889+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558cc4cbaa30 0x558cc4cbce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.889+0000 7fe7f8c9db80 4 client.15792 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:17.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.889+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/3528027945 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cc4c42e30 con 0x558cc4cbaa30 2024-05-27T07:48:17.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.889+0000 7fe7e3400700 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558cc4cbaa30 0x558cc4cbce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:17.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.890+0000 7fe7e3400700 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558cc4cbaa30 0x558cc4cbce50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=156fda186eefec39 server_cookie=5ae5fdaff74dc8b6 in_seq=0 out_seq=0 2024-05-27T07:48:17.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.890+0000 7fe7cd600700 10 client.15792 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:48:17.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7cd600700 1 -- 172.21.15.90:0/3528027945 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cc4c42e30 con 0x558cc4cbaa30 2024-05-27T07:48:17.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7cd600700 10 client.15792 handle_command_reply: tid=0 2024-05-27T07:48:17.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7f8c9db80 1 client.15792 shutdown 2024-05-27T07:48:17.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/3528027945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 msgr2=0x558cc4c55de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:17.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c55de0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00039e0 tx=0x7fe7d0003900 comp rx=0 tx=0).stop 2024-05-27T07:48:17.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/3528027945 shutdown_connections 2024-05-27T07:48:17.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558cc4cbaa30 0x558cc4cbce50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558cc4c2ddb0 0x558cc4c50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558cc4aaee40 0x558cc4c558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7f8c9db80 1 --2- 172.21.15.90:0/3528027945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558cc4b9a9f0 0x558cc4c55de0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/3528027945 >> 172.21.15.90:0/3528027945 conn(0x558cc4b48390 msgr2=0x558cc4c388b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:17.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.891+0000 7fe7cd600700 0 client.15792 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:48:17.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.892+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/3528027945 shutdown_connections 2024-05-27T07:48:17.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.892+0000 7fe7f8c9db80 1 -- 172.21.15.90:0/3528027945 wait complete. 2024-05-27T07:48:17.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.892+0000 7fe7f8c9db80 20 client.15792 trim_cache size 0 max 16384 2024-05-27T07:48:17.892 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:48:17.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 -- 172.21.15.90:0/1083314286 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7c80b2a70 msgr2=0x7fe7c80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:17.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7c80b2a70 0x7fe7c80b4f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40098e0 tx=0x7fe7d4005040 comp rx=0 tx=0).stop 2024-05-27T07:48:17.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 -- 172.21.15.90:0/1083314286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e4110bf0 msgr2=0x7fe7e41a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:17.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e4110bf0 0x7fe7e41a1780 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0094f0 tx=0x7fe7dc00d3b0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 -- 172.21.15.90:0/1083314286 shutdown_connections 2024-05-27T07:48:17.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e4110bf0 0x7fe7e41a1780 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7c80b2a70 0x7fe7c80b4f80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7e4075af0 0x7fe7e419cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 --2- 172.21.15.90:0/1083314286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4076440 0x7fe7e419d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:17.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 -- 172.21.15.90:0/1083314286 >> 172.21.15.90:0/1083314286 conn(0x7fe7e4100030 msgr2=0x7fe7e4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:17.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 -- 172.21.15.90:0/1083314286 shutdown_connections 2024-05-27T07:48:17.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:17.893+0000 7fe7ea200700 1 -- 172.21.15.90:0/1083314286 wait complete. 2024-05-27T07:48:17.942 INFO:teuthology.orchestra.run.smithi090.stderr:1100000 of 3200000 tuples (34%) done (elapsed 33.99 s, remaining 64.90 s) 2024-05-27T07:48:18.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:18 smithi090 ceph-mon[31429]: pgmap v60: 129 pgs: 129 active+clean; 291 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:48:18.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:18 smithi136 ceph-mon[35324]: pgmap v60: 129 pgs: 129 active+clean; 291 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:48:18.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:18 smithi129 ceph-mon[35254]: pgmap v60: 129 pgs: 129 active+clean; 291 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:48:18.986 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:48:19.472 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:20.457 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:20 smithi090 ceph-mon[31429]: pgmap v61: 129 pgs: 129 active+clean; 327 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:48:20.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:20 smithi136 ceph-mon[35324]: pgmap v61: 129 pgs: 129 active+clean; 327 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:48:20.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:20 smithi129 ceph-mon[35254]: pgmap v61: 129 pgs: 129 active+clean; 327 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:48:20.787 INFO:teuthology.orchestra.run.smithi090.stderr:1200000 of 3200000 tuples (37%) done (elapsed 36.84 s, remaining 61.40 s) 2024-05-27T07:48:21.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.797+0000 7fe341000700 1 -- 172.21.15.90:0/3757726100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe33c105300 msgr2=0x7fe33c105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:21.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.797+0000 7fe341000700 1 --2- 172.21.15.90:0/3757726100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe33c105300 0x7fe33c105710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe32c0098f0 tx=0x7fe32c009c00 comp rx=0 tx=0).stop 2024-05-27T07:48:21.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.797+0000 7fe341000700 1 -- 172.21.15.90:0/3757726100 shutdown_connections 2024-05-27T07:48:21.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.797+0000 7fe341000700 1 --2- 172.21.15.90:0/3757726100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe33c10e730 0x7fe33c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.797+0000 7fe341000700 1 --2- 172.21.15.90:0/3757726100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe33c105300 0x7fe33c105710 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.797+0000 7fe341000700 1 --2- 172.21.15.90:0/3757726100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe33c105c50 0x7fe33c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.797+0000 7fe341000700 1 -- 172.21.15.90:0/3757726100 >> 172.21.15.90:0/3757726100 conn(0x7fe33c078ab0 msgr2=0x7fe33c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:21.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.797+0000 7fe341000700 1 -- 172.21.15.90:0/3757726100 shutdown_connections 2024-05-27T07:48:21.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.797+0000 7fe341000700 1 -- 172.21.15.90:0/3757726100 wait complete. 2024-05-27T07:48:21.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.799+0000 7fe341000700 1 Processor -- start 2024-05-27T07:48:21.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.800+0000 7fe341000700 1 -- start start 2024-05-27T07:48:21.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.800+0000 7fe341000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe33c105300 0x7fe33c10df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.800+0000 7fe341000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe33c105c50 0x7fe33c106140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.800+0000 7fe341000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe33c10e730 0x7fe33c106680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.800+0000 7fe341000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c06d930 con 0x7fe33c10e730 2024-05-27T07:48:21.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.800+0000 7fe341000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c06d7b0 con 0x7fe33c105c50 2024-05-27T07:48:21.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.800+0000 7fe341000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c06dab0 con 0x7fe33c105300 2024-05-27T07:48:21.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.801+0000 7fe33be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe33c10e730 0x7fe33c106680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.801+0000 7fe33b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe33c105300 0x7fe33c10df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.801+0000 7fe33be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe33c10e730 0x7fe33c106680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50656/0 (socket says 172.21.15.90:50656) 2024-05-27T07:48:21.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.801+0000 7fe33be00700 1 -- 172.21.15.90:0/1836952980 learned_addr learned my addr 172.21.15.90:0/1836952980 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:21.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.801+0000 7fe33aa00700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe33c105c50 0x7fe33c106140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.801+0000 7fe33be00700 1 -- 172.21.15.90:0/1836952980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe33c105300 msgr2=0x7fe33c10df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:21.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.801+0000 7fe33be00700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe33c105300 0x7fe33c10df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.801+0000 7fe33be00700 1 -- 172.21.15.90:0/1836952980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe33c105c50 msgr2=0x7fe33c106140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:21.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.801+0000 7fe33be00700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe33c105c50 0x7fe33c106140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.801+0000 7fe33be00700 1 -- 172.21.15.90:0/1836952980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c106db0 con 0x7fe33c10e730 2024-05-27T07:48:21.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.802+0000 7fe33be00700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe33c10e730 0x7fe33c106680 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c110 tx=0x7fe33400c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:21.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.802+0000 7fe333e00700 1 -- 172.21.15.90:0/1836952980 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3340092e0 con 0x7fe33c10e730 2024-05-27T07:48:21.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.802+0000 7fe333e00700 1 -- 172.21.15.90:0/1836952980 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe334004bb0 con 0x7fe33c10e730 2024-05-27T07:48:21.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.802+0000 7fe333e00700 1 -- 172.21.15.90:0/1836952980 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334007b20 con 0x7fe33c10e730 2024-05-27T07:48:21.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.802+0000 7fe341000700 1 -- 172.21.15.90:0/1836952980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c1b5ee0 con 0x7fe33c10e730 2024-05-27T07:48:21.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.802+0000 7fe341000700 1 -- 172.21.15.90:0/1836952980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c1b6280 con 0x7fe33c10e730 2024-05-27T07:48:21.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.807+0000 7fe333e00700 1 -- 172.21.15.90:0/1836952980 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe33401a020 con 0x7fe33c10e730 2024-05-27T07:48:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.808+0000 7fe351145b80 1 Processor -- start 2024-05-27T07:48:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.808+0000 7fe351145b80 1 -- start start 2024-05-27T07:48:21.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.808+0000 7fe333e00700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3200b29f0 0x7fe3200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.808+0000 7fe333e00700 1 -- 172.21.15.90:0/1836952980 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe3340e23f0 con 0x7fe33c10e730 2024-05-27T07:48:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe351145b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84081e40 0x560b841ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe351145b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b8420aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe351145b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84200820 0x560b8420d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe351145b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560b8412ee40 con 0x560b84200410 2024-05-27T07:48:21.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe351145b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560b8411e9e0 con 0x560b84200820 2024-05-27T07:48:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe351145b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b84122a20 con 0x560b84081e40 2024-05-27T07:48:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe33aa00700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3200b29f0 0x7fe3200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe33aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b8420aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe33aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b8420aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50666/0 (socket says 172.21.15.90:50666) 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe33aa00700 1 -- 172.21.15.90:0/2153333968 learned_addr learned my addr 172.21.15.90:0/2153333968 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.809+0000 7fe33be00700 1 --2- 172.21.15.90:0/2153333968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84200820 0x560b8420d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.810+0000 7fe33b400700 1 --2- 172.21.15.90:0/2153333968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84081e40 0x560b841ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.810+0000 7fe33aa00700 1 -- 172.21.15.90:0/2153333968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84081e40 msgr2=0x560b841ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.810+0000 7fe33aa00700 1 --2- 172.21.15.90:0/2153333968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84081e40 0x560b841ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.810+0000 7fe33aa00700 1 -- 172.21.15.90:0/2153333968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84200820 msgr2=0x560b8420d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.810+0000 7fe33aa00700 1 --2- 172.21.15.90:0/2153333968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84200820 0x560b8420d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.810+0000 7fe33aa00700 1 -- 172.21.15.90:0/2153333968 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b840e7420 con 0x560b84200410 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.810+0000 7fe33aa00700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3200b29f0 0x7fe3200b4f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe328012520 tx=0x7fe3280128e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:21.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.810+0000 7fe33aa00700 1 --2- 172.21.15.90:0/2153333968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b8420aad0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe328012ce0 tx=0x7fe32801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d717edc36e796f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:21.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.811+0000 7fe330c00700 1 -- 172.21.15.90:0/2153333968 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32801ede0 con 0x560b84200410 2024-05-27T07:48:21.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.811+0000 7fe330c00700 1 -- 172.21.15.90:0/2153333968 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe32801f8e0 con 0x560b84200410 2024-05-27T07:48:21.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.811+0000 7fe330c00700 1 -- 172.21.15.90:0/2153333968 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32800f7a0 con 0x560b84200410 2024-05-27T07:48:21.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.812+0000 7fe351145b80 1 -- 172.21.15.90:0/2153333968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 msgr2=0x560b8420aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:21.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.812+0000 7fe351145b80 1 --2- 172.21.15.90:0/2153333968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b8420aad0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe328012ce0 tx=0x7fe32801e000 comp rx=0 tx=0).stop 2024-05-27T07:48:21.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.812+0000 7fe351145b80 1 -- 172.21.15.90:0/2153333968 shutdown_connections 2024-05-27T07:48:21.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.812+0000 7fe351145b80 1 --2- 172.21.15.90:0/2153333968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84200820 0x560b8420d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.812+0000 7fe351145b80 1 --2- 172.21.15.90:0/2153333968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84081e40 0x560b841ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.812+0000 7fe351145b80 1 --2- 172.21.15.90:0/2153333968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b8420aad0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.812+0000 7fe351145b80 1 -- 172.21.15.90:0/2153333968 >> 172.21.15.90:0/2153333968 conn(0x560b840e1fc0 msgr2=0x560b841fbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:21.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.813+0000 7fe351145b80 1 -- 172.21.15.90:0/2153333968 shutdown_connections 2024-05-27T07:48:21.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.813+0000 7fe351145b80 1 -- 172.21.15.90:0/2153333968 wait complete. 2024-05-27T07:48:21.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.813+0000 7fe351145b80 1 -- start start 2024-05-27T07:48:21.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.814+0000 7fe351145b80 1 Processor -- start 2024-05-27T07:48:21.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.814+0000 7fe351145b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 0x560b8420a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.814+0000 7fe351145b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 0x560b8420aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.814+0000 7fe351145b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.814+0000 7fe351145b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560b84024fa0 con 0x560b84200820 2024-05-27T07:48:21.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.814+0000 7fe351145b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560b840bf760 con 0x560b84081e40 2024-05-27T07:48:21.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.814+0000 7fe351145b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b8420d930 con 0x560b84200410 2024-05-27T07:48:21.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.814+0000 7fe351145b80 10 client.0 _ll_register_callbacks cb 0x560b840c8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:21.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.815+0000 7fe33be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.815+0000 7fe33be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50676/0 (socket says 172.21.15.90:50676) 2024-05-27T07:48:21.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.815+0000 7fe33b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 0x560b8420a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.815+0000 7fe33be00700 1 -- 172.21.15.90:0/323233734 learned_addr learned my addr 172.21.15.90:0/323233734 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:21.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.815+0000 7fe33aa00700 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 0x560b8420aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.816+0000 7fe33be00700 1 -- 172.21.15.90:0/323233734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 msgr2=0x560b8420aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:21.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.816+0000 7fe33be00700 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 0x560b8420aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.816+0000 7fe33be00700 1 -- 172.21.15.90:0/323233734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 msgr2=0x560b8420a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:21.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.816+0000 7fe33be00700 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 0x560b8420a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.816+0000 7fe33be00700 1 -- 172.21.15.90:0/323233734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b8406e8d0 con 0x560b84200820 2024-05-27T07:48:21.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.816+0000 7fe33be00700 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420b020 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe3340df7f0 tx=0x7fe3340a9aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef66f6fd642ae57b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:21.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.816+0000 7fe325600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:21.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.817+0000 7fe325600700 1 -- 172.21.15.90:0/323233734 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3340b2180 con 0x560b84200820 2024-05-27T07:48:21.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.817+0000 7fe325600700 1 -- 172.21.15.90:0/323233734 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe3340a9580 con 0x560b84200820 2024-05-27T07:48:21.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.817+0000 7fe351145b80 1 -- 172.21.15.90:0/323233734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe32801f8e0 con 0x560b84200820 2024-05-27T07:48:21.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.817+0000 7fe325600700 1 -- 172.21.15.90:0/323233734 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe3340a9780 con 0x560b84200820 2024-05-27T07:48:21.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.817+0000 7fe351145b80 10 client.15804 fetch_fsmap learned FSMap version 34 2024-05-27T07:48:21.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.817+0000 7fe351145b80 1 -- 172.21.15.90:0/323233734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560b840ef060 con 0x560b84200820 2024-05-27T07:48:21.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.817+0000 7fe325600700 1 -- 172.21.15.90:0/323233734 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe32801f8e0 con 0x560b84200820 2024-05-27T07:48:21.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.818+0000 7fe325600700 1 -- 172.21.15.90:0/323233734 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe3340ad200 con 0x560b84200820 2024-05-27T07:48:21.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.819+0000 7fe351145b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T07:48:21.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.819+0000 7fe351145b80 10 client.15804 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:48:21.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.819+0000 7fe351145b80 20 client.15804 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:21.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.819+0000 7fe351145b80 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x560b8421a870 0x560b8421cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.819+0000 7fe351145b80 4 client.15804 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:21.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.819+0000 7fe351145b80 1 -- 172.21.15.90:0/323233734 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b84130d90 con 0x560b8421a870 2024-05-27T07:48:21.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.819+0000 7fe33b400700 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x560b8421a870 0x560b8421cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.821+0000 7fe33b400700 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x560b8421a870 0x560b8421cc90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8da13ac5f1b0ac0e server_cookie=53ed14df9437c3ca in_seq=0 out_seq=0 2024-05-27T07:48:21.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.821+0000 7fe325600700 10 client.15804 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:48:21.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.823+0000 7fe325600700 1 -- 172.21.15.90:0/323233734 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560b84130d90 con 0x560b8421a870 2024-05-27T07:48:21.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.823+0000 7fe325600700 10 client.15804 handle_command_reply: tid=0 2024-05-27T07:48:21.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.823+0000 7fe351145b80 1 client.15804 shutdown 2024-05-27T07:48:21.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.823+0000 7fe351145b80 1 -- 172.21.15.90:0/323233734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 msgr2=0x560b8420b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:21.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.823+0000 7fe351145b80 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420b020 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe3340df7f0 tx=0x7fe3340a9aa0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.824+0000 7fe351145b80 1 -- 172.21.15.90:0/323233734 shutdown_connections 2024-05-27T07:48:21.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.824+0000 7fe351145b80 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420b020 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.824+0000 7fe351145b80 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x560b8421a870 0x560b8421cc90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.824+0000 7fe351145b80 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 0x560b8420a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.824+0000 7fe351145b80 1 --2- 172.21.15.90:0/323233734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 0x560b8420aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.824+0000 7fe351145b80 1 -- 172.21.15.90:0/323233734 >> 172.21.15.90:0/323233734 conn(0x560b840e1fc0 msgr2=0x560b841faff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:21.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.824+0000 7fe325600700 0 client.15804 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:48:21.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.824+0000 7fe351145b80 1 -- 172.21.15.90:0/323233734 shutdown_connections 2024-05-27T07:48:21.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.824+0000 7fe351145b80 1 -- 172.21.15.90:0/323233734 wait complete. 2024-05-27T07:48:21.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.825+0000 7fe351145b80 20 client.15804 trim_cache size 0 max 16384 2024-05-27T07:48:21.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.847+0000 7fe351145b80 1 Processor -- start 2024-05-27T07:48:21.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.847+0000 7fe351145b80 1 -- start start 2024-05-27T07:48:21.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.847+0000 7fe351145b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84081e40 0x560b8420abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.847+0000 7fe351145b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b84205d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.847+0000 7fe351145b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84200820 0x560b842169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.847+0000 7fe351145b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1bf110 con 0x560b84200410 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.847+0000 7fe351145b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1c0c50 con 0x560b84200820 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.847+0000 7fe351145b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1bf7e0 con 0x560b84081e40 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.848+0000 7fe33aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b84205d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.848+0000 7fe33aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b84205d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50690/0 (socket says 172.21.15.90:50690) 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.848+0000 7fe33aa00700 1 -- 172.21.15.90:0/455616629 learned_addr learned my addr 172.21.15.90:0/455616629 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.848+0000 7fe33be00700 1 --2- 172.21.15.90:0/455616629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84200820 0x560b842169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.848+0000 7fe33b400700 1 --2- 172.21.15.90:0/455616629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84081e40 0x560b8420abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.848+0000 7fe33aa00700 1 -- 172.21.15.90:0/455616629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84081e40 msgr2=0x560b8420abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.848+0000 7fe33aa00700 1 --2- 172.21.15.90:0/455616629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84081e40 0x560b8420abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.848+0000 7fe33aa00700 1 -- 172.21.15.90:0/455616629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84200820 msgr2=0x560b842169d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:21.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.848+0000 7fe33aa00700 1 --2- 172.21.15.90:0/455616629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84200820 0x560b842169d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.848+0000 7fe33aa00700 1 -- 172.21.15.90:0/455616629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b841fc620 con 0x560b84200410 2024-05-27T07:48:21.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.849+0000 7fe33aa00700 1 --2- 172.21.15.90:0/455616629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b84205d90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe328023000 tx=0x7fe32800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe2fcc716fa4826 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:21.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.849+0000 7fe327e00700 1 -- 172.21.15.90:0/455616629 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328037070 con 0x560b84200410 2024-05-27T07:48:21.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.849+0000 7fe327e00700 1 -- 172.21.15.90:0/455616629 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe32801eca0 con 0x560b84200410 2024-05-27T07:48:21.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.849+0000 7fe327e00700 1 -- 172.21.15.90:0/455616629 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3280052c0 con 0x560b84200410 2024-05-27T07:48:21.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.850+0000 7fe351145b80 1 -- 172.21.15.90:0/455616629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 msgr2=0x560b84205d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:21.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.850+0000 7fe351145b80 1 --2- 172.21.15.90:0/455616629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b84205d90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe328023000 tx=0x7fe32800be60 comp rx=0 tx=0).stop 2024-05-27T07:48:21.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.850+0000 7fe351145b80 1 -- 172.21.15.90:0/455616629 shutdown_connections 2024-05-27T07:48:21.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.851+0000 7fe351145b80 1 --2- 172.21.15.90:0/455616629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84200820 0x560b842169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.851+0000 7fe351145b80 1 --2- 172.21.15.90:0/455616629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84081e40 0x560b8420abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.851+0000 7fe351145b80 1 --2- 172.21.15.90:0/455616629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200410 0x560b84205d90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.851+0000 7fe351145b80 1 -- 172.21.15.90:0/455616629 >> 172.21.15.90:0/455616629 conn(0x560b840e1fc0 msgr2=0x560b84209790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:21.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.851+0000 7fe351145b80 1 -- 172.21.15.90:0/455616629 shutdown_connections 2024-05-27T07:48:21.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.851+0000 7fe351145b80 1 -- 172.21.15.90:0/455616629 wait complete. 2024-05-27T07:48:21.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.851+0000 7fe351145b80 1 -- start start 2024-05-27T07:48:21.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.852+0000 7fe351145b80 1 Processor -- start 2024-05-27T07:48:21.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.852+0000 7fe351145b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 0x560b842131f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.852+0000 7fe351145b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 0x560b8420e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.852+0000 7fe351145b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.852+0000 7fe351145b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1c0580 con 0x560b84200820 2024-05-27T07:48:21.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.852+0000 7fe351145b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1bfeb0 con 0x560b84081e40 2024-05-27T07:48:21.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.852+0000 7fe351145b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c1bd090 con 0x560b84200410 2024-05-27T07:48:21.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.852+0000 7fe351145b80 10 client.0 _ll_register_callbacks cb 0x560b840c8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:21.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.853+0000 7fe33b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 0x560b842131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.853+0000 7fe33aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 0x560b8420e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.853+0000 7fe33b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 0x560b842131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38442/0 (socket says 172.21.15.90:38442) 2024-05-27T07:48:21.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.853+0000 7fe33b400700 1 -- 172.21.15.90:0/2045503997 learned_addr learned my addr 172.21.15.90:0/2045503997 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:21.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.853+0000 7fe33be00700 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe33aa00700 1 -- 172.21.15.90:0/2045503997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 msgr2=0x560b842131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:21.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe33aa00700 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 0x560b842131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe33aa00700 1 -- 172.21.15.90:0/2045503997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 msgr2=0x560b8420ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:21.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe33aa00700 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe33aa00700 1 -- 172.21.15.90:0/2045503997 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b841fd1f0 con 0x560b84200410 2024-05-27T07:48:21.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe33b400700 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 0x560b842131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:48:21.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe33be00700 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:21.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe33aa00700 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 0x560b8420e550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe328023000 tx=0x7fe32800f800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e07c9cc6a84bd8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:21.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe325600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:48:21.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe325600700 1 -- 172.21.15.90:0/2045503997 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328037070 con 0x560b84200410 2024-05-27T07:48:21.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.854+0000 7fe351145b80 1 -- 172.21.15.90:0/2045503997 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe32801eca0 con 0x560b84200410 2024-05-27T07:48:21.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.855+0000 7fe325600700 1 -- 172.21.15.90:0/2045503997 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe32800faa0 con 0x560b84200410 2024-05-27T07:48:21.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.855+0000 7fe325600700 1 -- 172.21.15.90:0/2045503997 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe32800fca0 con 0x560b84200410 2024-05-27T07:48:21.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.855+0000 7fe351145b80 10 client.25465 fetch_fsmap learned FSMap version 34 2024-05-27T07:48:21.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.855+0000 7fe351145b80 1 -- 172.21.15.90:0/2045503997 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560b84284f50 con 0x560b84200410 2024-05-27T07:48:21.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.855+0000 7fe325600700 1 -- 172.21.15.90:0/2045503997 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe32801eca0 con 0x560b84200410 2024-05-27T07:48:21.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.856+0000 7fe325600700 1 -- 172.21.15.90:0/2045503997 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe328029420 con 0x560b84200410 2024-05-27T07:48:21.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.856+0000 7fe351145b80 10 client.25465 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T07:48:21.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.856+0000 7fe351145b80 10 client.25465 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:48:21.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.856+0000 7fe351145b80 20 client.25465 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:21.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.856+0000 7fe351145b80 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x560b84285300 0x560b84287720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:21.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.856+0000 7fe351145b80 4 client.25465 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:21.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.856+0000 7fe351145b80 1 -- 172.21.15.90:0/2045503997 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b8412a570 con 0x560b84285300 2024-05-27T07:48:21.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.857+0000 7fe33b400700 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x560b84285300 0x560b84287720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:21.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.858+0000 7fe325600700 10 client.25465 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:48:21.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.858+0000 7fe33b400700 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x560b84285300 0x560b84287720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=586eaa6bb133e887 server_cookie=6a9af3149b2f7705 in_seq=0 out_seq=0 2024-05-27T07:48:21.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.858+0000 7fe325600700 1 -- 172.21.15.90:0/2045503997 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b8412a570 con 0x560b84285300 2024-05-27T07:48:21.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.858+0000 7fe325600700 10 client.25465 handle_command_reply: tid=0 2024-05-27T07:48:21.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.858+0000 7fe351145b80 1 client.25465 shutdown 2024-05-27T07:48:21.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe351145b80 1 -- 172.21.15.90:0/2045503997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 msgr2=0x560b8420e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:21.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe351145b80 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 0x560b8420e550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe328023000 tx=0x7fe32800f800 comp rx=0 tx=0).stop 2024-05-27T07:48:21.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe351145b80 1 -- 172.21.15.90:0/2045503997 shutdown_connections 2024-05-27T07:48:21.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe351145b80 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b84200820 0x560b8420ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe351145b80 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x560b84285300 0x560b84287720 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe351145b80 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b84081e40 0x560b842131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe351145b80 1 --2- 172.21.15.90:0/2045503997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b84200410 0x560b8420e550 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe328023000 tx=0x7fe32800f800 comp rx=0 tx=0).stop 2024-05-27T07:48:21.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe351145b80 1 -- 172.21.15.90:0/2045503997 >> 172.21.15.90:0/2045503997 conn(0x560b840e1fc0 msgr2=0x560b8420b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:21.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe325600700 0 client.25465 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:48:21.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe351145b80 1 -- 172.21.15.90:0/2045503997 shutdown_connections 2024-05-27T07:48:21.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.859+0000 7fe351145b80 1 -- 172.21.15.90:0/2045503997 wait complete. 2024-05-27T07:48:21.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe351145b80 20 client.25465 trim_cache size 0 max 16384 2024-05-27T07:48:21.861 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:48:21.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe341000700 1 -- 172.21.15.90:0/1836952980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3200b29f0 msgr2=0x7fe3200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:21.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe341000700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3200b29f0 0x7fe3200b4f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe328012520 tx=0x7fe3280128e0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe341000700 1 -- 172.21.15.90:0/1836952980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe33c10e730 msgr2=0x7fe33c106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:21.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe341000700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe33c10e730 0x7fe33c106680 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c110 tx=0x7fe33400c420 comp rx=0 tx=0).stop 2024-05-27T07:48:21.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe341000700 1 -- 172.21.15.90:0/1836952980 shutdown_connections 2024-05-27T07:48:21.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe341000700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe33c10e730 0x7fe33c106680 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe341000700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe33c105300 0x7fe33c10df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe341000700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3200b29f0 0x7fe3200b4f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe341000700 1 --2- 172.21.15.90:0/1836952980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe33c105c50 0x7fe33c106140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:21.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.860+0000 7fe341000700 1 -- 172.21.15.90:0/1836952980 >> 172.21.15.90:0/1836952980 conn(0x7fe33c078ab0 msgr2=0x7fe33c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:21.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.861+0000 7fe341000700 1 -- 172.21.15.90:0/1836952980 shutdown_connections 2024-05-27T07:48:21.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:21.861+0000 7fe341000700 1 -- 172.21.15.90:0/1836952980 wait complete. 2024-05-27T07:48:22.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:22 smithi090 ceph-mon[31429]: pgmap v62: 129 pgs: 129 active+clean; 327 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:48:22.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:22 smithi136 ceph-mon[35324]: pgmap v62: 129 pgs: 129 active+clean; 327 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:48:22.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:22 smithi129 ceph-mon[35254]: pgmap v62: 129 pgs: 129 active+clean; 327 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:48:22.918 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:48:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.056+0000 7ffab2400640 1 -- 172.21.15.90:0/971294006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaac10b7f0 msgr2=0x7ffaac10dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.056+0000 7ffab2400640 1 --2- 172.21.15.90:0/971294006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaac10b7f0 0x7ffaac10dc10 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009900 tx=0x7ffa9802a410 comp rx=0 tx=0).stop 2024-05-27T07:48:23.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.057+0000 7ffab2400640 1 -- 172.21.15.90:0/971294006 shutdown_connections 2024-05-27T07:48:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.057+0000 7ffab2400640 1 --2- 172.21.15.90:0/971294006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaac10e1e0 0x7ffaac1106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.057+0000 7ffab2400640 1 --2- 172.21.15.90:0/971294006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaac10b7f0 0x7ffaac10dc10 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.057+0000 7ffab2400640 1 --2- 172.21.15.90:0/971294006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac104d90 0x7ffaac1051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.057+0000 7ffab2400640 1 -- 172.21.15.90:0/971294006 >> 172.21.15.90:0/971294006 conn(0x7ffaac0782d0 msgr2=0x7ffaac078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.057+0000 7ffab2400640 1 -- 172.21.15.90:0/971294006 shutdown_connections 2024-05-27T07:48:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.057+0000 7ffab2400640 1 -- 172.21.15.90:0/971294006 wait complete. 2024-05-27T07:48:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.059+0000 7ffab2400640 1 Processor -- start 2024-05-27T07:48:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.059+0000 7ffab2400640 1 -- start start 2024-05-27T07:48:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.060+0000 7ffab2400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaac104d90 0x7ffaac1a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.060+0000 7ffab2400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac10b7f0 0x7ffaac1a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.060+0000 7ffab2400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaac10e1e0 0x7ffaac19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.060+0000 7ffab2400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac113cf0 con 0x7ffaac10e1e0 2024-05-27T07:48:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.060+0000 7ffab2400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac113b70 con 0x7ffaac104d90 2024-05-27T07:48:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.060+0000 7ffab2400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac113e70 con 0x7ffaac10b7f0 2024-05-27T07:48:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.061+0000 7ffaaaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac10b7f0 0x7ffaac1a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.061+0000 7ffaaaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac10b7f0 0x7ffaac1a2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:55742/0 (socket says 172.21.15.90:55742) 2024-05-27T07:48:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.061+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1629621572 learned_addr learned my addr 172.21.15.90:0/1629621572 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.061+0000 7ffaabe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaac10e1e0 0x7ffaac19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.061+0000 7ffaab400640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaac104d90 0x7ffaac1a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.062+0000 7ffaabe00640 1 -- 172.21.15.90:0/1629621572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac10b7f0 msgr2=0x7ffaac1a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.062+0000 7ffaabe00640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac10b7f0 0x7ffaac1a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.062+0000 7ffaabe00640 1 -- 172.21.15.90:0/1629621572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaac104d90 msgr2=0x7ffaac1a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.062+0000 7ffaabe00640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaac104d90 0x7ffaac1a2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.062+0000 7ffaabe00640 1 -- 172.21.15.90:0/1629621572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac19cf30 con 0x7ffaac10e1e0 2024-05-27T07:48:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.062+0000 7ffaabe00640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaac10e1e0 0x7ffaac19c890 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400e940 tx=0x7ffaa400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.063+0000 7ffaa3e00640 1 -- 172.21.15.90:0/1629621572 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4004d20 con 0x7ffaac10e1e0 2024-05-27T07:48:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.063+0000 7ffaa3e00640 1 -- 172.21.15.90:0/1629621572 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffaa4005020 con 0x7ffaac10e1e0 2024-05-27T07:48:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.063+0000 7ffaa3e00640 1 -- 172.21.15.90:0/1629621572 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa40116d0 con 0x7ffaac10e1e0 2024-05-27T07:48:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.063+0000 7ffab2400640 1 -- 172.21.15.90:0/1629621572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaac19d220 con 0x7ffaac10e1e0 2024-05-27T07:48:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.063+0000 7ffab2400640 1 -- 172.21.15.90:0/1629621572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaac1a9500 con 0x7ffaac10e1e0 2024-05-27T07:48:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.067+0000 7ffab9386740 1 Processor -- start 2024-05-27T07:48:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.069+0000 7ffaa3e00640 1 -- 172.21.15.90:0/1629621572 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffaa400cb20 con 0x7ffaac10e1e0 2024-05-27T07:48:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.069+0000 7ffab9386740 1 -- start start 2024-05-27T07:48:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.069+0000 7ffab9386740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f38c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.070+0000 7ffab9386740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc12f0 0x559e2f38cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.070+0000 7ffab9386740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc94d0 0x559e2f38d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.070+0000 7ffab9386740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559e2f2fdb90 con 0x559e2efea1b0 2024-05-27T07:48:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.070+0000 7ffab9386740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559e2f2ed730 con 0x559e2efc94d0 2024-05-27T07:48:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.070+0000 7ffab9386740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559e2f2f1770 con 0x559e2efc12f0 2024-05-27T07:48:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.070+0000 7ffaaaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f38c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.070+0000 7ffaaaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f38c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50730/0 (socket says 172.21.15.90:50730) 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.070+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1505805131 learned_addr learned my addr 172.21.15.90:0/1505805131 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.070+0000 7ffaab400640 1 --2- 172.21.15.90:0/1505805131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc12f0 0x559e2f38cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.070+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1505805131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc94d0 0x559e2f38d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.071+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1505805131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc12f0 msgr2=0x559e2f38cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.071+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1505805131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc12f0 0x559e2f38cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.071+0000 7ffaa3e00640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ffa900b29d0 0x7ffa900b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.071+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1505805131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc94d0 msgr2=0x559e2f38d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.071+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1505805131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc94d0 0x559e2f38d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.071+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1505805131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e2f2868b0 con 0x559e2efea1b0 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.071+0000 7ffaabe00640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ffa900b29d0 0x7ffa900b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.071+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1505805131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f38c860 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ffa98005f70 tx=0x7ffa98005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215ec81425c8b74f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.071+0000 7ffaa3e00640 1 -- 172.21.15.90:0/1629621572 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ffaa40e25b0 con 0x7ffaac10e1e0 2024-05-27T07:48:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.072+0000 7ffaa0c00640 1 -- 172.21.15.90:0/1505805131 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98035040 con 0x559e2efea1b0 2024-05-27T07:48:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.072+0000 7ffaa0c00640 1 -- 172.21.15.90:0/1505805131 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa980044c0 con 0x559e2efea1b0 2024-05-27T07:48:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.072+0000 7ffaa0c00640 1 -- 172.21.15.90:0/1505805131 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98003c40 con 0x559e2efea1b0 2024-05-27T07:48:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.073+0000 7ffab9386740 1 -- 172.21.15.90:0/1505805131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 msgr2=0x559e2f38c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.073+0000 7ffab9386740 1 --2- 172.21.15.90:0/1505805131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f38c860 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ffa98005f70 tx=0x7ffa98005ee0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.073+0000 7ffaabe00640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ffa900b29d0 0x7ffa900b4ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4005fd0 tx=0x7ffaa4004a40 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.073+0000 7ffab9386740 1 -- 172.21.15.90:0/1505805131 shutdown_connections 2024-05-27T07:48:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.073+0000 7ffab9386740 1 --2- 172.21.15.90:0/1505805131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc94d0 0x559e2f38d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.073+0000 7ffab9386740 1 --2- 172.21.15.90:0/1505805131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc12f0 0x559e2f38cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.073+0000 7ffab9386740 1 --2- 172.21.15.90:0/1505805131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f38c860 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.073+0000 7ffab9386740 1 -- 172.21.15.90:0/1505805131 >> 172.21.15.90:0/1505805131 conn(0x559e2efde300 msgr2=0x559e2f389010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.073+0000 7ffab9386740 1 -- 172.21.15.90:0/1505805131 shutdown_connections 2024-05-27T07:48:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.073+0000 7ffab9386740 1 -- 172.21.15.90:0/1505805131 wait complete. 2024-05-27T07:48:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.074+0000 7ffab9386740 1 -- start start 2024-05-27T07:48:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.074+0000 7ffab9386740 1 Processor -- start 2024-05-27T07:48:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.075+0000 7ffab9386740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc12f0 0x559e2f396a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.075+0000 7ffab9386740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efc94d0 0x559e2f396f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.075+0000 7ffab9386740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.075+0000 7ffab9386740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559e2efd3790 con 0x559e2efc94d0 2024-05-27T07:48:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.075+0000 7ffab9386740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559e2f295bc0 con 0x559e2efc12f0 2024-05-27T07:48:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.075+0000 7ffab9386740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559e2f399930 con 0x559e2efea1b0 2024-05-27T07:48:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.075+0000 7ffab9386740 10 client.0 _ll_register_callbacks cb 0x559e2f273400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.076+0000 7ffaaaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.076+0000 7ffaab400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efc94d0 0x559e2f396f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.076+0000 7ffaaaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:55758/0 (socket says 172.21.15.90:55758) 2024-05-27T07:48:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.076+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1419440688 learned_addr learned my addr 172.21.15.90:0/1419440688 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.077+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1419440688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc12f0 msgr2=0x559e2f396a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.077+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc12f0 0x559e2f396a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.077+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1419440688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efc94d0 msgr2=0x559e2f396f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.077+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efc94d0 0x559e2f396f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.077+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1419440688 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e2f002210 con 0x559e2efea1b0 2024-05-27T07:48:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.077+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009330 tx=0x7ffa98003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9083a66ca43a00a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.077+0000 7ffa95600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:48:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.078+0000 7ffa95600640 1 -- 172.21.15.90:0/1419440688 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98031750 con 0x559e2efea1b0 2024-05-27T07:48:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.078+0000 7ffa95600640 1 -- 172.21.15.90:0/1419440688 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ffa98005050 con 0x559e2efea1b0 2024-05-27T07:48:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.079+0000 7ffab9386740 1 -- 172.21.15.90:0/1419440688 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e2f2ffae0 con 0x559e2efea1b0 2024-05-27T07:48:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.080+0000 7ffa95600640 1 -- 172.21.15.90:0/1419440688 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffa98058050 con 0x559e2efea1b0 2024-05-27T07:48:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.081+0000 7ffab9386740 10 client.25477 fetch_fsmap learned FSMap version 34 2024-05-27T07:48:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.081+0000 7ffab9386740 1 -- 172.21.15.90:0/1419440688 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559e2f0ce620 con 0x559e2efea1b0 2024-05-27T07:48:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.082+0000 7ffa95600640 1 -- 172.21.15.90:0/1419440688 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559e2f2ffae0 con 0x559e2efea1b0 2024-05-27T07:48:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.082+0000 7ffa95600640 1 -- 172.21.15.90:0/1419440688 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7ffa980046c0 con 0x559e2efea1b0 2024-05-27T07:48:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.082+0000 7ffab9386740 10 client.25477 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T07:48:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.082+0000 7ffab9386740 10 client.25477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.082+0000 7ffab9386740 20 client.25477 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.082+0000 7ffab9386740 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559e2efa8d80 0x559e2f3a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.082+0000 7ffab9386740 4 client.25477 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.082+0000 7ffab9386740 1 -- 172.21.15.90:0/1419440688 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e2f2f92c0 con 0x559e2efa8d80 2024-05-27T07:48:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.083+0000 7ffaab400640 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559e2efa8d80 0x559e2f3a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.084+0000 7ffaab400640 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559e2efa8d80 0x559e2f3a8a60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365ab2c81fb8580 server_cookie=553f1abe6eee949b in_seq=0 out_seq=0 2024-05-27T07:48:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.084+0000 7ffa95600640 10 client.25477 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.087+0000 7ffa95600640 1 -- 172.21.15.90:0/1419440688 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559e2f2f92c0 con 0x559e2efa8d80 2024-05-27T07:48:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.087+0000 7ffa95600640 10 client.25477 handle_command_reply: tid=0 2024-05-27T07:48:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.087+0000 7ffab9386740 1 client.25477 shutdown 2024-05-27T07:48:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.088+0000 7ffab9386740 1 -- 172.21.15.90:0/1419440688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efea1b0 msgr2=0x559e2f3a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.088+0000 7ffab9386740 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009330 tx=0x7ffa98003820 comp rx=0 tx=0).stop 2024-05-27T07:48:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.088+0000 7ffab9386740 1 -- 172.21.15.90:0/1419440688 shutdown_connections 2024-05-27T07:48:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.088+0000 7ffab9386740 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559e2efa8d80 0x559e2f3a8a60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.088+0000 7ffab9386740 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.088+0000 7ffab9386740 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efc94d0 0x559e2f396f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.088+0000 7ffab9386740 1 --2- 172.21.15.90:0/1419440688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc12f0 0x559e2f396a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.088+0000 7ffab9386740 1 -- 172.21.15.90:0/1419440688 >> 172.21.15.90:0/1419440688 conn(0x559e2efde300 msgr2=0x559e2f38c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.088+0000 7ffa95600640 0 client.25477 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.088+0000 7ffab9386740 1 -- 172.21.15.90:0/1419440688 shutdown_connections 2024-05-27T07:48:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.089+0000 7ffab9386740 1 -- 172.21.15.90:0/1419440688 wait complete. 2024-05-27T07:48:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.089+0000 7ffab9386740 20 client.25477 trim_cache size 0 max 16384 2024-05-27T07:48:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.109+0000 7ffab9386740 1 Processor -- start 2024-05-27T07:48:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.109+0000 7ffab9386740 1 -- start start 2024-05-27T07:48:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.109+0000 7ffab9386740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc12f0 0x559e2f392210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.110+0000 7ffab9386740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc94d0 0x559e2f388330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.110+0000 7ffab9386740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.110+0000 7ffab9386740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1b3370 con 0x559e2efea1b0 2024-05-27T07:48:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.110+0000 7ffab9386740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1b4eb0 con 0x559e2efc12f0 2024-05-27T07:48:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.110+0000 7ffab9386740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1b3a40 con 0x559e2efc94d0 2024-05-27T07:48:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.110+0000 7ffaaaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.110+0000 7ffaaaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50754/0 (socket says 172.21.15.90:50754) 2024-05-27T07:48:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.110+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1749136448 learned_addr learned my addr 172.21.15.90:0/1749136448 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.110+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1749136448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc12f0 0x559e2f392210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.110+0000 7ffaab400640 1 --2- 172.21.15.90:0/1749136448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc94d0 0x559e2f388330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.111+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1749136448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc94d0 msgr2=0x559e2f388330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.111+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1749136448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc94d0 0x559e2f388330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.111+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1749136448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc12f0 msgr2=0x559e2f392210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.111+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1749136448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc12f0 0x559e2f392210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.111+0000 7ffaaaa00640 1 -- 172.21.15.90:0/1749136448 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e2f38dc40 con 0x559e2efea1b0 2024-05-27T07:48:23.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.111+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/1749136448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffa98001eb0 tx=0x7ffa98032ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d9f110155e3d23 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.111+0000 7ffa97e00640 1 -- 172.21.15.90:0/1749136448 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98035020 con 0x559e2efea1b0 2024-05-27T07:48:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.112+0000 7ffa97e00640 1 -- 172.21.15.90:0/1749136448 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa9804d030 con 0x559e2efea1b0 2024-05-27T07:48:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.112+0000 7ffa97e00640 1 -- 172.21.15.90:0/1749136448 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98005260 con 0x559e2efea1b0 2024-05-27T07:48:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.113+0000 7ffab9386740 1 -- 172.21.15.90:0/1749136448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 msgr2=0x559e2f3a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.113+0000 7ffab9386740 1 --2- 172.21.15.90:0/1749136448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ffa98001eb0 tx=0x7ffa98032ef0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.113+0000 7ffab9386740 1 -- 172.21.15.90:0/1749136448 shutdown_connections 2024-05-27T07:48:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.113+0000 7ffab9386740 1 --2- 172.21.15.90:0/1749136448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a2c00 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.113+0000 7ffab9386740 1 --2- 172.21.15.90:0/1749136448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc94d0 0x559e2f388330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.113+0000 7ffab9386740 1 --2- 172.21.15.90:0/1749136448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc12f0 0x559e2f392210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.113+0000 7ffab9386740 1 -- 172.21.15.90:0/1749136448 >> 172.21.15.90:0/1749136448 conn(0x559e2efde300 msgr2=0x559e2f398120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.113+0000 7ffab9386740 1 -- 172.21.15.90:0/1749136448 shutdown_connections 2024-05-27T07:48:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.113+0000 7ffab9386740 1 -- 172.21.15.90:0/1749136448 wait complete. 2024-05-27T07:48:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.114+0000 7ffab9386740 1 -- start start 2024-05-27T07:48:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.114+0000 7ffab9386740 1 Processor -- start 2024-05-27T07:48:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffab9386740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc12f0 0x559e2f3a8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffab9386740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc94d0 0x559e2f3a8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffab9386740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffab9386740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1b47e0 con 0x559e2efea1b0 2024-05-27T07:48:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffab9386740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1b4110 con 0x559e2efc94d0 2024-05-27T07:48:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffab9386740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1b1330 con 0x559e2efc12f0 2024-05-27T07:48:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffab9386740 10 client.0 _ll_register_callbacks cb 0x7ffa9c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffaaaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffaaaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50768/0 (socket says 172.21.15.90:50768) 2024-05-27T07:48:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffaaaa00640 1 -- 172.21.15.90:0/2190510965 learned_addr learned my addr 172.21.15.90:0/2190510965 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.115+0000 7ffaab400640 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc94d0 0x559e2f3a8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.116+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc12f0 0x559e2f3a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.116+0000 7ffaaaa00640 1 -- 172.21.15.90:0/2190510965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc12f0 msgr2=0x559e2f3a8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.116+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc12f0 0x559e2f3a8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.116+0000 7ffaaaa00640 1 -- 172.21.15.90:0/2190510965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc94d0 msgr2=0x559e2f3a8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.116+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc94d0 0x559e2f3a8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.116+0000 7ffaaaa00640 1 -- 172.21.15.90:0/2190510965 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e2f40eea0 con 0x559e2efea1b0 2024-05-27T07:48:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.116+0000 7ffaaaa00640 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a92c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffa98004bd0 tx=0x7ffa9803c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e008a8a230cf5cc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.116+0000 7ffa95600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.116+0000 7ffa95600640 1 -- 172.21.15.90:0/2190510965 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98064020 con 0x559e2efea1b0 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.116+0000 7ffa95600640 1 -- 172.21.15.90:0/2190510965 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ffa98004dd0 con 0x559e2efea1b0 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.117+0000 7ffab9386740 1 -- 172.21.15.90:0/2190510965 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa9804d030 con 0x559e2efea1b0 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.117+0000 7ffa95600640 1 -- 172.21.15.90:0/2190510965 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffa98035070 con 0x559e2efea1b0 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.117+0000 7ffab9386740 10 client.15828 fetch_fsmap learned FSMap version 34 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.117+0000 7ffab9386740 1 -- 172.21.15.90:0/2190510965 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559e2f4100e0 con 0x559e2efea1b0 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.117+0000 7ffa95600640 1 -- 172.21.15.90:0/2190510965 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa9804d030 con 0x559e2efea1b0 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.118+0000 7ffa95600640 1 -- 172.21.15.90:0/2190510965 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7ffa980321b0 con 0x559e2efea1b0 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.118+0000 7ffab9386740 10 client.15828 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.118+0000 7ffab9386740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.118+0000 7ffab9386740 20 client.15828 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.118+0000 7ffab9386740 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559e2ef7e1c0 0x559e2f412730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.118+0000 7ffab9386740 4 client.15828 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:48:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.118+0000 7ffab9386740 1 -- 172.21.15.90:0/2190510965 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e2f2734b0 con 0x559e2ef7e1c0 2024-05-27T07:48:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.119+0000 7ffaab400640 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559e2ef7e1c0 0x559e2f412730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.120+0000 7ffa95600640 10 client.15828 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.120+0000 7ffaab400640 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559e2ef7e1c0 0x559e2f412730 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aafe79272590b6dc server_cookie=946e53d1dccb1480 in_seq=0 out_seq=0 2024-05-27T07:48:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.165+0000 7ffa95600640 1 -- 172.21.15.90:0/2190510965 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559e2f2734b0 con 0x559e2ef7e1c0 2024-05-27T07:48:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.165+0000 7ffa95600640 10 client.15828 handle_command_reply: tid=0 2024-05-27T07:48:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffab9386740 1 client.15828 shutdown 2024-05-27T07:48:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffab9386740 1 -- 172.21.15.90:0/2190510965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 msgr2=0x559e2f3a92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffab9386740 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a92c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ffa98004bd0 tx=0x7ffa9803c040 comp rx=0 tx=0).stop 2024-05-27T07:48:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffab9386740 1 -- 172.21.15.90:0/2190510965 shutdown_connections 2024-05-27T07:48:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffab9386740 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559e2ef7e1c0 0x559e2f412730 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffab9386740 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e2efc94d0 0x559e2f3a8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffab9386740 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e2efea1b0 0x559e2f3a92c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffab9386740 1 --2- 172.21.15.90:0/2190510965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e2efc12f0 0x559e2f3a8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffa95600640 0 client.15828 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffab9386740 1 -- 172.21.15.90:0/2190510965 >> 172.21.15.90:0/2190510965 conn(0x559e2efde300 msgr2=0x559e2f387d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.166+0000 7ffab9386740 1 -- 172.21.15.90:0/2190510965 shutdown_connections 2024-05-27T07:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.167+0000 7ffab9386740 1 -- 172.21.15.90:0/2190510965 wait complete. 2024-05-27T07:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.167+0000 7ffab9386740 20 client.15828 trim_cache size 0 max 16384 2024-05-27T07:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:48:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.167+0000 7ffab2400640 1 -- 172.21.15.90:0/1629621572 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ffa900b29d0 msgr2=0x7ffa900b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.167+0000 7ffab2400640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ffa900b29d0 0x7ffa900b4ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4005fd0 tx=0x7ffaa4004a40 comp rx=0 tx=0).stop 2024-05-27T07:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.168+0000 7ffab2400640 1 -- 172.21.15.90:0/1629621572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaac10e1e0 msgr2=0x7ffaac19c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.168+0000 7ffab2400640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaac10e1e0 0x7ffaac19c890 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400e940 tx=0x7ffaa400ee00 comp rx=0 tx=0).stop 2024-05-27T07:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.168+0000 7ffab2400640 1 -- 172.21.15.90:0/1629621572 shutdown_connections 2024-05-27T07:48:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.168+0000 7ffab2400640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ffa900b29d0 0x7ffa900b4ee0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.168+0000 7ffab2400640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffaac10e1e0 0x7ffaac19c890 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.168+0000 7ffab2400640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac10b7f0 0x7ffaac1a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.168+0000 7ffab2400640 1 --2- 172.21.15.90:0/1629621572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffaac104d90 0x7ffaac1a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.168+0000 7ffab2400640 1 -- 172.21.15.90:0/1629621572 >> 172.21.15.90:0/1629621572 conn(0x7ffaac0782d0 msgr2=0x7ffaac109c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.168+0000 7ffab2400640 1 -- 172.21.15.90:0/1629621572 shutdown_connections 2024-05-27T07:48:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:23.168+0000 7ffab2400640 1 -- 172.21.15.90:0/1629621572 wait complete. 2024-05-27T07:48:23.417 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:23.521 INFO:teuthology.orchestra.run.smithi090.stderr:1300000 of 3200000 tuples (40%) done (elapsed 39.57 s, remaining 57.84 s) 2024-05-27T07:48:24.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:24 smithi090 ceph-mon[31429]: pgmap v63: 129 pgs: 129 active+clean; 363 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-05-27T07:48:24.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:24 smithi136 ceph-mon[35324]: pgmap v63: 129 pgs: 129 active+clean; 363 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-05-27T07:48:24.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:24 smithi129 ceph-mon[35254]: pgmap v63: 129 pgs: 129 active+clean; 363 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-05-27T07:48:25.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.811+0000 7f5a7a800700 1 -- 172.21.15.90:0/1558579969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a74075af0 msgr2=0x7f5a74075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.811+0000 7f5a7a800700 1 --2- 172.21.15.90:0/1558579969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a74075af0 0x7f5a74075f00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009990 tx=0x7f5a64009ca0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.811+0000 7f5a7a800700 1 -- 172.21.15.90:0/1558579969 shutdown_connections 2024-05-27T07:48:25.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.811+0000 7f5a7a800700 1 --2- 172.21.15.90:0/1558579969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a74110c10 0x7f5a74113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.811+0000 7f5a7a800700 1 --2- 172.21.15.90:0/1558579969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a74075af0 0x7f5a74075f00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.811+0000 7f5a7a800700 1 --2- 172.21.15.90:0/1558579969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a74076440 0x7f5a741106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.811+0000 7f5a7a800700 1 -- 172.21.15.90:0/1558579969 >> 172.21.15.90:0/1558579969 conn(0x7f5a7407af60 msgr2=0x7f5a7407b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.812+0000 7f5a7a800700 1 -- 172.21.15.90:0/1558579969 shutdown_connections 2024-05-27T07:48:25.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.812+0000 7f5a7a800700 1 -- 172.21.15.90:0/1558579969 wait complete. 2024-05-27T07:48:25.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.813+0000 7f5a7a800700 1 Processor -- start 2024-05-27T07:48:25.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.814+0000 7f5a7a800700 1 -- start start 2024-05-27T07:48:25.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.814+0000 7f5a7a800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a74075af0 0x7f5a7419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.814+0000 7f5a7a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a74076440 0x7f5a7419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.815+0000 7f5a7a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a74110c10 0x7f5a741a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.815+0000 7f5a7a800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74116530 con 0x7f5a74110c10 2024-05-27T07:48:25.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.815+0000 7f5a7a800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741163b0 con 0x7f5a74075af0 2024-05-27T07:48:25.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.815+0000 7f5a7a800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741166b0 con 0x7f5a74076440 2024-05-27T07:48:25.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.815+0000 7f5a73400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a74075af0 0x7f5a7419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.815+0000 7f5a73400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a74075af0 0x7f5a7419cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38490/0 (socket says 172.21.15.90:38490) 2024-05-27T07:48:25.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.815+0000 7f5a73400700 1 -- 172.21.15.90:0/2907118722 learned_addr learned my addr 172.21.15.90:0/2907118722 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:25.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.815+0000 7f5a73e00700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a74110c10 0x7f5a741a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.815+0000 7f5a72a00700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a74076440 0x7f5a7419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.816+0000 7f5a73400700 1 -- 172.21.15.90:0/2907118722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a74076440 msgr2=0x7f5a7419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.816+0000 7f5a73400700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a74076440 0x7f5a7419d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.816+0000 7f5a73400700 1 -- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a74110c10 msgr2=0x7f5a741a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.816+0000 7f5a73400700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a74110c10 0x7f5a741a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.816+0000 7f5a73400700 1 -- 172.21.15.90:0/2907118722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a741a1e90 con 0x7f5a74075af0 2024-05-27T07:48:25.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.816+0000 7f5a72a00700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a74076440 0x7f5a7419d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:25.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.816+0000 7f5a73e00700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a74110c10 0x7f5a741a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:25.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.816+0000 7f5a73400700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a74075af0 0x7f5a7419cfc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a64006010 tx=0x7f5a6400bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.817+0000 7f5a6be00700 1 -- 172.21.15.90:0/2907118722 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64027070 con 0x7f5a74075af0 2024-05-27T07:48:25.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.817+0000 7f5a7a800700 1 -- 172.21.15.90:0/2907118722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a741a2120 con 0x7f5a74075af0 2024-05-27T07:48:25.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.817+0000 7f5a7a800700 1 -- 172.21.15.90:0/2907118722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a741a9aa0 con 0x7f5a74075af0 2024-05-27T07:48:25.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.817+0000 7f5a6be00700 1 -- 172.21.15.90:0/2907118722 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a6400fb90 con 0x7f5a74075af0 2024-05-27T07:48:25.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.817+0000 7f5a6be00700 1 -- 172.21.15.90:0/2907118722 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a640173f0 con 0x7f5a74075af0 2024-05-27T07:48:25.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.824+0000 7f5a6be00700 1 -- 172.21.15.90:0/2907118722 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5a6402a070 con 0x7f5a74075af0 2024-05-27T07:48:25.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.824+0000 7f5a8930db80 1 Processor -- start 2024-05-27T07:48:25.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.824+0000 7f5a8930db80 1 -- start start 2024-05-27T07:48:25.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.825+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b991250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.825+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b812e40 0x55ad0b9917b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.825+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9a6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.825+0000 7f5a8930db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ad0b8bfe40 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.825+0000 7f5a8930db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ad0b8af9e0 con 0x55ad0b991db0 2024-05-27T07:48:25.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.825+0000 7f5a8930db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ad0b8b3a20 con 0x55ad0b812e40 2024-05-27T07:48:25.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.825+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b991250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.825+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b991250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50800/0 (socket says 172.21.15.90:50800) 2024-05-27T07:48:25.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.825+0000 7f5a73e00700 1 -- 172.21.15.90:0/991055411 learned_addr learned my addr 172.21.15.90:0/991055411 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:25.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.826+0000 7f5a73e00700 1 -- 172.21.15.90:0/991055411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b812e40 msgr2=0x55ad0b9917b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:48:25.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.826+0000 7f5a73e00700 1 --2- 172.21.15.90:0/991055411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b812e40 0x55ad0b9917b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.826+0000 7f5a73e00700 1 -- 172.21.15.90:0/991055411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 msgr2=0x55ad0b9a6940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:48:25.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.827+0000 7f5a72a00700 1 --2- 172.21.15.90:0/991055411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.827+0000 7f5a6be00700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5a580b2a70 0x7f5a580b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.827+0000 7f5a73e00700 1 --2- 172.21.15.90:0/991055411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.827+0000 7f5a73e00700 1 -- 172.21.15.90:0/991055411 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad0b878420 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.828+0000 7f5a72a00700 1 --2- 172.21.15.90:0/991055411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:25.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.828+0000 7f5a6be00700 1 -- 172.21.15.90:0/2907118722 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f5a640e3850 con 0x7f5a74075af0 2024-05-27T07:48:25.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.829+0000 7f5a73e00700 1 --2- 172.21.15.90:0/991055411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b991250 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009dd0 tx=0x7f5a6c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f246bc89baf0e88b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.829+0000 7f5a73400700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5a580b2a70 0x7f5a580b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.829+0000 7f5a68c00700 1 -- 172.21.15.90:0/991055411 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c014610 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.829+0000 7f5a68c00700 1 -- 172.21.15.90:0/991055411 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a6c019070 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.829+0000 7f5a68c00700 1 -- 172.21.15.90:0/991055411 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c013910 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.831+0000 7f5a8930db80 1 -- 172.21.15.90:0/991055411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b8fe9f0 msgr2=0x55ad0b991250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.831+0000 7f5a8930db80 1 --2- 172.21.15.90:0/991055411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b991250 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009dd0 tx=0x7f5a6c0077e0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.831+0000 7f5a8930db80 1 -- 172.21.15.90:0/991055411 shutdown_connections 2024-05-27T07:48:25.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.831+0000 7f5a8930db80 1 --2- 172.21.15.90:0/991055411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.831+0000 7f5a8930db80 1 --2- 172.21.15.90:0/991055411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b991250 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.831+0000 7f5a8930db80 1 --2- 172.21.15.90:0/991055411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b812e40 0x55ad0b9917b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.831+0000 7f5a8930db80 1 -- 172.21.15.90:0/991055411 >> 172.21.15.90:0/991055411 conn(0x55ad0b8ac390 msgr2=0x55ad0b98d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.831+0000 7f5a73400700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5a580b2a70 0x7f5a580b4f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55ad0b98ea20 tx=0x7f5a64009710 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.832+0000 7f5a8930db80 1 -- 172.21.15.90:0/991055411 shutdown_connections 2024-05-27T07:48:25.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.832+0000 7f5a8930db80 1 -- 172.21.15.90:0/991055411 wait complete. 2024-05-27T07:48:25.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.833+0000 7f5a8930db80 1 -- start start 2024-05-27T07:48:25.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.833+0000 7f5a8930db80 1 Processor -- start 2024-05-27T07:48:25.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.834+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.834+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b9a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.834+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.834+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.834+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50806/0 (socket says 172.21.15.90:50806) 2024-05-27T07:48:25.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.835+0000 7f5a8930db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ad0b7b5fa0 con 0x55ad0b812e40 2024-05-27T07:48:25.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.835+0000 7f5a8930db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ad0b850760 con 0x55ad0b991db0 2024-05-27T07:48:25.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.835+0000 7f5a8930db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ad0b9a73e0 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.835+0000 7f5a8930db80 10 client.0 _ll_register_callbacks cb 0x55ad0b859c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:25.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.835+0000 7f5a72a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b9a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.835+0000 7f5a73e00700 1 -- 172.21.15.90:0/684868817 learned_addr learned my addr 172.21.15.90:0/684868817 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:25.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.835+0000 7f5a73e00700 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.835+0000 7f5a73e00700 1 -- 172.21.15.90:0/684868817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 msgr2=0x55ad0b9a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.835+0000 7f5a73e00700 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b9a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.836+0000 7f5a73e00700 1 -- 172.21.15.90:0/684868817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 msgr2=0x55ad0b9a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.836+0000 7f5a73e00700 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.836+0000 7f5a73e00700 1 -- 172.21.15.90:0/684868817 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad0b7ff8d0 con 0x55ad0b812e40 2024-05-27T07:48:25.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.836+0000 7f5a73e00700 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9a18b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009500 tx=0x7f5a6c0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563ef5c525f83902 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.836+0000 7f5a5d600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:25.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.836+0000 7f5a5d600700 1 -- 172.21.15.90:0/684868817 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0144c0 con 0x55ad0b812e40 2024-05-27T07:48:25.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.837+0000 7f5a5d600700 1 -- 172.21.15.90:0/684868817 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f5a6c014660 con 0x55ad0b812e40 2024-05-27T07:48:25.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.837+0000 7f5a8930db80 1 -- 172.21.15.90:0/684868817 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a6c019070 con 0x55ad0b812e40 2024-05-27T07:48:25.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.837+0000 7f5a5d600700 1 -- 172.21.15.90:0/684868817 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5a6c032050 con 0x55ad0b812e40 2024-05-27T07:48:25.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.837+0000 7f5a8930db80 10 client.15840 fetch_fsmap learned FSMap version 34 2024-05-27T07:48:25.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.837+0000 7f5a8930db80 1 -- 172.21.15.90:0/684868817 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ad0b880060 con 0x55ad0b812e40 2024-05-27T07:48:25.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.838+0000 7f5a5d600700 1 -- 172.21.15.90:0/684868817 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a6c019070 con 0x55ad0b812e40 2024-05-27T07:48:25.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.839+0000 7f5a5d600700 1 -- 172.21.15.90:0/684868817 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f5a6c005450 con 0x55ad0b812e40 2024-05-27T07:48:25.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.839+0000 7f5a8930db80 10 client.15840 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T07:48:25.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.839+0000 7f5a8930db80 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:25.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.839+0000 7f5a8930db80 20 client.15840 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:25.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.839+0000 7f5a8930db80 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ad0b9b4210 0x55ad0b9b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.839+0000 7f5a8930db80 4 client.15840 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:25.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.839+0000 7f5a8930db80 1 -- 172.21.15.90:0/684868817 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad0b8c1d90 con 0x55ad0b9b4210 2024-05-27T07:48:25.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.840+0000 7f5a72a00700 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ad0b9b4210 0x55ad0b9b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.841+0000 7f5a72a00700 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ad0b9b4210 0x55ad0b9b6630 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d625b5203299690c server_cookie=cee5f476fa94f41d in_seq=0 out_seq=0 2024-05-27T07:48:25.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.841+0000 7f5a5d600700 10 client.15840 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:25.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.843+0000 7f5a5d600700 1 -- 172.21.15.90:0/684868817 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ad0b8c1d90 con 0x55ad0b9b4210 2024-05-27T07:48:25.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.843+0000 7f5a5d600700 10 client.15840 handle_command_reply: tid=0 2024-05-27T07:48:25.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.843+0000 7f5a8930db80 1 client.15840 shutdown 2024-05-27T07:48:25.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.843+0000 7f5a8930db80 1 -- 172.21.15.90:0/684868817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 msgr2=0x55ad0b9a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.843+0000 7f5a8930db80 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9a18b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009500 tx=0x7f5a6c0092f0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.844+0000 7f5a8930db80 1 -- 172.21.15.90:0/684868817 shutdown_connections 2024-05-27T07:48:25.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.844+0000 7f5a8930db80 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ad0b9b4210 0x55ad0b9b6630 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.844+0000 7f5a8930db80 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.844+0000 7f5a8930db80 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9a18b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.844+0000 7f5a8930db80 1 --2- 172.21.15.90:0/684868817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b9a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.844+0000 7f5a8930db80 1 -- 172.21.15.90:0/684868817 >> 172.21.15.90:0/684868817 conn(0x55ad0b8ac390 msgr2=0x55ad0b98d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.844+0000 7f5a5d600700 0 client.15840 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:25.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.844+0000 7f5a8930db80 1 -- 172.21.15.90:0/684868817 shutdown_connections 2024-05-27T07:48:25.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.844+0000 7f5a8930db80 1 -- 172.21.15.90:0/684868817 wait complete. 2024-05-27T07:48:25.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.844+0000 7f5a8930db80 20 client.15840 trim_cache size 0 max 16384 2024-05-27T07:48:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.869+0000 7f5a8930db80 1 Processor -- start 2024-05-27T07:48:25.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.869+0000 7f5a8930db80 1 -- start start 2024-05-27T07:48:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.870+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b812e40 0x55ad0b9b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.870+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b99cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.870+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b991db0 0x55ad0b9b9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.870+0000 7f5a8930db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5400ecc0 con 0x55ad0b991db0 2024-05-27T07:48:25.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.870+0000 7f5a8930db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54010800 con 0x55ad0b812e40 2024-05-27T07:48:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.870+0000 7f5a8930db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5400f390 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.871+0000 7f5a72a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b99cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.871+0000 7f5a72a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b99cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:55812/0 (socket says 172.21.15.90:55812) 2024-05-27T07:48:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.871+0000 7f5a72a00700 1 -- 172.21.15.90:0/3333199010 learned_addr learned my addr 172.21.15.90:0/3333199010 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.872+0000 7f5a73e00700 1 --2- 172.21.15.90:0/3333199010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b991db0 0x55ad0b9b9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.872+0000 7f5a73e00700 1 --2- 172.21.15.90:0/3333199010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b812e40 0x55ad0b9b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.872+0000 7f5a72a00700 1 -- 172.21.15.90:0/3333199010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b812e40 msgr2=0x55ad0b9b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.872+0000 7f5a72a00700 1 --2- 172.21.15.90:0/3333199010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b812e40 0x55ad0b9b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.872+0000 7f5a72a00700 1 -- 172.21.15.90:0/3333199010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b991db0 msgr2=0x55ad0b9b9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.872+0000 7f5a72a00700 1 --2- 172.21.15.90:0/3333199010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b991db0 0x55ad0b9b9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.872+0000 7f5a72a00700 1 -- 172.21.15.90:0/3333199010 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad0b99fe20 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.873+0000 7f5a72a00700 1 --2- 172.21.15.90:0/3333199010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b99cfc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55ad0b7b60f0 tx=0x7f5a60009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=317c2bb65843bc50 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.873+0000 7f5a5fe00700 1 -- 172.21.15.90:0/3333199010 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60003cc0 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.873+0000 7f5a5fe00700 1 -- 172.21.15.90:0/3333199010 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a60003e60 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.873+0000 7f5a5fe00700 1 -- 172.21.15.90:0/3333199010 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60010820 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.875+0000 7f5a8930db80 1 -- 172.21.15.90:0/3333199010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 msgr2=0x55ad0b99cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.875+0000 7f5a8930db80 1 --2- 172.21.15.90:0/3333199010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b99cfc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55ad0b7b60f0 tx=0x7f5a60009d30 comp rx=0 tx=0).stop 2024-05-27T07:48:25.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.875+0000 7f5a8930db80 1 -- 172.21.15.90:0/3333199010 shutdown_connections 2024-05-27T07:48:25.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.875+0000 7f5a8930db80 1 --2- 172.21.15.90:0/3333199010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b991db0 0x55ad0b9b9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.875+0000 7f5a8930db80 1 --2- 172.21.15.90:0/3333199010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b812e40 0x55ad0b9b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.875+0000 7f5a8930db80 1 --2- 172.21.15.90:0/3333199010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b99cfc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.875+0000 7f5a8930db80 1 -- 172.21.15.90:0/3333199010 >> 172.21.15.90:0/3333199010 conn(0x55ad0b8ac390 msgr2=0x55ad0b99c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.875+0000 7f5a8930db80 1 -- 172.21.15.90:0/3333199010 shutdown_connections 2024-05-27T07:48:25.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.875+0000 7f5a8930db80 1 -- 172.21.15.90:0/3333199010 wait complete. 2024-05-27T07:48:25.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.875+0000 7f5a8930db80 1 -- start start 2024-05-27T07:48:25.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.876+0000 7f5a8930db80 1 Processor -- start 2024-05-27T07:48:25.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.876+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.876+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b9b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.876+0000 7f5a8930db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.876+0000 7f5a8930db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54010130 con 0x55ad0b812e40 2024-05-27T07:48:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.876+0000 7f5a8930db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5400fa60 con 0x55ad0b991db0 2024-05-27T07:48:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.876+0000 7f5a8930db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5400cc40 con 0x55ad0b8fe9f0 2024-05-27T07:48:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.876+0000 7f5a8930db80 10 client.0 _ll_register_callbacks cb 0x55ad0b859c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.877+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.877+0000 7f5a73e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50810/0 (socket says 172.21.15.90:50810) 2024-05-27T07:48:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.877+0000 7f5a73e00700 1 -- 172.21.15.90:0/3898967392 learned_addr learned my addr 172.21.15.90:0/3898967392 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.877+0000 7f5a72a00700 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b9b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.878+0000 7f5a73e00700 1 -- 172.21.15.90:0/3898967392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 msgr2=0x55ad0b9b6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.878+0000 7f5a73e00700 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b9b6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.878+0000 7f5a73e00700 1 -- 172.21.15.90:0/3898967392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 msgr2=0x55ad0b9b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:25.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.878+0000 7f5a73e00700 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.878+0000 7f5a73e00700 1 -- 172.21.15.90:0/3898967392 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad0ba1cb70 con 0x55ad0b812e40 2024-05-27T07:48:25.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.878+0000 7f5a73e00700 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9b6510 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0058a0 tx=0x7f5a6c020680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9807dd9478ab71ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:25.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.878+0000 7f5a5d600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:25.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.878+0000 7f5a5d600700 1 -- 172.21.15.90:0/3898967392 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c020c00 con 0x55ad0b812e40 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.879+0000 7f5a5d600700 1 -- 172.21.15.90:0/3898967392 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f5a6c020da0 con 0x55ad0b812e40 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.879+0000 7f5a8930db80 1 -- 172.21.15.90:0/3898967392 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a60003e60 con 0x55ad0b812e40 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.879+0000 7f5a5d600700 1 -- 172.21.15.90:0/3898967392 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5a6c03f050 con 0x55ad0b812e40 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.879+0000 7f5a8930db80 10 client.15846 fetch_fsmap learned FSMap version 34 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.879+0000 7f5a8930db80 1 -- 172.21.15.90:0/3898967392 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ad0ba1ddb0 con 0x55ad0b812e40 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.880+0000 7f5a5d600700 1 -- 172.21.15.90:0/3898967392 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5a60003e60 con 0x55ad0b812e40 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.880+0000 7f5a5d600700 1 -- 172.21.15.90:0/3898967392 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f5a6c035030 con 0x55ad0b812e40 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.880+0000 7f5a8930db80 10 client.15846 fetch_fsmap finished waiting for FSMap version 34 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.880+0000 7f5a8930db80 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.880+0000 7f5a8930db80 20 client.15846 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.880+0000 7f5a8930db80 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ad0ba1e3d0 0x55ad0ba207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:25.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.880+0000 7f5a8930db80 4 client.15846 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:25.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.880+0000 7f5a8930db80 1 -- 172.21.15.90:0/3898967392 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad0b8bb570 con 0x55ad0ba1e3d0 2024-05-27T07:48:25.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.881+0000 7f5a72a00700 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ad0ba1e3d0 0x55ad0ba207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:25.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.882+0000 7f5a5d600700 10 client.15846 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:25.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.882+0000 7f5a72a00700 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ad0ba1e3d0 0x55ad0ba207f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=895f3a4e705f5ea5 server_cookie=8228e89d0edde570 in_seq=0 out_seq=0 2024-05-27T07:48:25.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.882+0000 7f5a5d600700 1 -- 172.21.15.90:0/3898967392 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad0b8bb570 con 0x55ad0ba1e3d0 2024-05-27T07:48:25.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.882+0000 7f5a5d600700 10 client.15846 handle_command_reply: tid=0 2024-05-27T07:48:25.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.882+0000 7f5a8930db80 1 client.15846 shutdown 2024-05-27T07:48:25.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.883+0000 7f5a8930db80 1 -- 172.21.15.90:0/3898967392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 msgr2=0x55ad0b9b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.883+0000 7f5a8930db80 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9b6510 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0058a0 tx=0x7f5a6c020680 comp rx=0 tx=0).stop 2024-05-27T07:48:25.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.883+0000 7f5a8930db80 1 -- 172.21.15.90:0/3898967392 shutdown_connections 2024-05-27T07:48:25.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.883+0000 7f5a8930db80 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ad0ba1e3d0 0x55ad0ba207f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.883+0000 7f5a8930db80 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad0b991db0 0x55ad0b9b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.883+0000 7f5a8930db80 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad0b812e40 0x55ad0b9b6510 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.883+0000 7f5a8930db80 1 --2- 172.21.15.90:0/3898967392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad0b8fe9f0 0x55ad0b9b6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.883+0000 7f5a8930db80 1 -- 172.21.15.90:0/3898967392 >> 172.21.15.90:0/3898967392 conn(0x55ad0b8ac390 msgr2=0x55ad0b9a01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.883+0000 7f5a5d600700 0 client.15846 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:25.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.884+0000 7f5a8930db80 1 -- 172.21.15.90:0/3898967392 shutdown_connections 2024-05-27T07:48:25.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.884+0000 7f5a8930db80 1 -- 172.21.15.90:0/3898967392 wait complete. 2024-05-27T07:48:25.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.884+0000 7f5a8930db80 20 client.15846 trim_cache size 0 max 16384 2024-05-27T07:48:25.884 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:48:25.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.885+0000 7f5a7a800700 1 -- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5a580b2a70 msgr2=0x7f5a580b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.885+0000 7f5a7a800700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5a580b2a70 0x7f5a580b4f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55ad0b98ea20 tx=0x7f5a64009710 comp rx=0 tx=0).stop 2024-05-27T07:48:25.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.885+0000 7f5a7a800700 1 -- 172.21.15.90:0/2907118722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a74075af0 msgr2=0x7f5a7419cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:25.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.885+0000 7f5a7a800700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a74075af0 0x7f5a7419cfc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a64006010 tx=0x7f5a6400bfa0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.885+0000 7f5a7a800700 1 -- 172.21.15.90:0/2907118722 shutdown_connections 2024-05-27T07:48:25.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.885+0000 7f5a7a800700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a74110c10 0x7f5a741a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.885+0000 7f5a7a800700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5a580b2a70 0x7f5a580b4f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.885+0000 7f5a7a800700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a74075af0 0x7f5a7419cfc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.885+0000 7f5a7a800700 1 --2- 172.21.15.90:0/2907118722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a74076440 0x7f5a7419d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:25.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.885+0000 7f5a7a800700 1 -- 172.21.15.90:0/2907118722 >> 172.21.15.90:0/2907118722 conn(0x7f5a7407af60 msgr2=0x7f5a74102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:25.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.886+0000 7f5a7a800700 1 -- 172.21.15.90:0/2907118722 shutdown_connections 2024-05-27T07:48:25.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:25.886+0000 7f5a7a800700 1 -- 172.21.15.90:0/2907118722 wait complete. 2024-05-27T07:48:26.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:26 smithi090 ceph-mon[31429]: pgmap v64: 129 pgs: 129 active+clean; 363 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-05-27T07:48:26.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:26 smithi136 ceph-mon[35324]: pgmap v64: 129 pgs: 129 active+clean; 363 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-05-27T07:48:26.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:26 smithi129 ceph-mon[35254]: pgmap v64: 129 pgs: 129 active+clean; 363 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-05-27T07:48:26.777 INFO:teuthology.orchestra.run.smithi090.stderr:1400000 of 3200000 tuples (43%) done (elapsed 42.83 s, remaining 55.07 s) 2024-05-27T07:48:27.150 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:26 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 3. 2024-05-27T07:48:27.150 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:26 smithi136 systemd[1]: Stopped Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:48:27.150 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:26 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.418s CPU time. 2024-05-27T07:48:27.151 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:26 smithi136 systemd[1]: Starting Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:48:27.473 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:27 smithi136 podman[70806]: 2024-05-27 07:48:27.055287578 +0000 UTC m=+0.032825858 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:48:27.474 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:27 smithi136 podman[70806]: 2024-05-27 07:48:27.182803379 +0000 UTC m=+0.160341716 container create 5e5edd60f637a362ec3eb7c9aef61550625f1a55bf48dece28b3eac94b96725a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-27T07:48:27.749 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:27 smithi136 podman[70806]: 2024-05-27 07:48:27.474204114 +0000 UTC m=+0.451742494 container init 5e5edd60f637a362ec3eb7c9aef61550625f1a55bf48dece28b3eac94b96725a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:48:27.750 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:27 smithi136 podman[70806]: 2024-05-27 07:48:27.485268902 +0000 UTC m=+0.462807226 container start 5e5edd60f637a362ec3eb7c9aef61550625f1a55bf48dece28b3eac94b96725a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-27T07:48:27.750 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:27 smithi136 bash[70806]: 5e5edd60f637a362ec3eb7c9aef61550625f1a55bf48dece28b3eac94b96725a 2024-05-27T07:48:27.750 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:27 smithi136 systemd[1]: Started Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:48:27.750 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:27 smithi136 ceph-mds[70825]: -- 172.21.15.136:0/2820557003 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4b87c7a40 con 0x55f4b880f180 2024-05-27T07:48:27.750 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: starting mds.c at 2024-05-27T07:48:27.873 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:48:27.873 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:48:28.364 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:28.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:28 smithi090 ceph-mon[31429]: pgmap v65: 129 pgs: 129 active+clean; 363 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 16 op/s 2024-05-27T07:48:28.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:28 smithi090 ceph-mon[31429]: daemon mds.c restarted 2024-05-27T07:48:28.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:28 smithi136 ceph-mon[35324]: pgmap v65: 129 pgs: 129 active+clean; 363 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 16 op/s 2024-05-27T07:48:28.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:28 smithi136 ceph-mon[35324]: daemon mds.c restarted 2024-05-27T07:48:28.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:28 smithi129 ceph-mon[35254]: pgmap v65: 129 pgs: 129 active+clean; 363 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 16 op/s 2024-05-27T07:48:28.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:28 smithi129 ceph-mon[35254]: daemon mds.c restarted 2024-05-27T07:48:29.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:29 smithi090 ceph-mon[31429]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T07:48:29.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:29 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.136:6836/508081175,v1:172.21.15.136:6837/508081175] up:boot 2024-05-27T07:48:29.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:29 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:48:29.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:29 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:48:29.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:29 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:48:29.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:29 smithi136 ceph-mon[35324]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T07:48:29.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:29 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.136:6836/508081175,v1:172.21.15.136:6837/508081175] up:boot 2024-05-27T07:48:29.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:29 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:48:29.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:29 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:48:29.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:29 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:48:29.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:29 smithi129 ceph-mon[35254]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T07:48:29.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:29 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.136:6836/508081175,v1:172.21.15.136:6837/508081175] up:boot 2024-05-27T07:48:29.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:29 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:48:29.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:29 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:48:29.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:29 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:48:29.834 INFO:teuthology.orchestra.run.smithi090.stderr:1500000 of 3200000 tuples (46%) done (elapsed 45.89 s, remaining 52.00 s) 2024-05-27T07:48:30.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:30 smithi090 ceph-mon[31429]: pgmap v67: 129 pgs: 129 active+clean; 399 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 15 MiB/s wr, 25 op/s 2024-05-27T07:48:30.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:30 smithi136 ceph-mon[35324]: pgmap v67: 129 pgs: 129 active+clean; 399 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 15 MiB/s wr, 25 op/s 2024-05-27T07:48:30.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:30 smithi129 ceph-mon[35254]: pgmap v67: 129 pgs: 129 active+clean; 399 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 15 MiB/s wr, 25 op/s 2024-05-27T07:48:30.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.713+0000 7f0dc2a00700 1 -- 172.21.15.90:0/252022142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dbc108d20 msgr2=0x7f0dbc109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:30.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.713+0000 7f0dc2a00700 1 --2- 172.21.15.90:0/252022142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dbc108d20 0x7f0dbc109130 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0da80099c0 tx=0x7f0da8009cd0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.713+0000 7f0dc2a00700 1 -- 172.21.15.90:0/252022142 shutdown_connections 2024-05-27T07:48:30.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.713+0000 7f0dc2a00700 1 --2- 172.21.15.90:0/252022142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dbc109670 0x7f0dbc110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.713+0000 7f0dc2a00700 1 --2- 172.21.15.90:0/252022142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dbc1043f0 0x7f0dbc104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.713+0000 7f0dc2a00700 1 --2- 172.21.15.90:0/252022142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dbc108d20 0x7f0dbc109130 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.713+0000 7f0dc2a00700 1 -- 172.21.15.90:0/252022142 >> 172.21.15.90:0/252022142 conn(0x7f0dbc079d00 msgr2=0x7f0dbc07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:30.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.714+0000 7f0dc2a00700 1 -- 172.21.15.90:0/252022142 shutdown_connections 2024-05-27T07:48:30.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.714+0000 7f0dc2a00700 1 -- 172.21.15.90:0/252022142 wait complete. 2024-05-27T07:48:30.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.715+0000 7f0dc2a00700 1 Processor -- start 2024-05-27T07:48:30.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.715+0000 7f0dc2a00700 1 -- start start 2024-05-27T07:48:30.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.716+0000 7f0dc2a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dbc1043f0 0x7f0dbc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.716+0000 7f0dc2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dbc108d20 0x7f0dbc19d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.716+0000 7f0dc2a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dbc109670 0x7f0dbc1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.716+0000 7f0dc2a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc114180 con 0x7f0dbc108d20 2024-05-27T07:48:30.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.716+0000 7f0dc2a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc114000 con 0x7f0dbc1043f0 2024-05-27T07:48:30.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.716+0000 7f0dc2a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc114300 con 0x7f0dbc109670 2024-05-27T07:48:30.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.716+0000 7f0dc0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dbc108d20 0x7f0dbc19d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.716+0000 7f0dc0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dbc108d20 0x7f0dbc19d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56532/0 (socket says 172.21.15.90:56532) 2024-05-27T07:48:30.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.716+0000 7f0dc0c00700 1 -- 172.21.15.90:0/2636504134 learned_addr learned my addr 172.21.15.90:0/2636504134 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:30.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.716+0000 7f0dc1600700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dbc1043f0 0x7f0dbc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.717+0000 7f0dc2000700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dbc109670 0x7f0dbc1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.717+0000 7f0dc0c00700 1 -- 172.21.15.90:0/2636504134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dbc109670 msgr2=0x7f0dbc1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:30.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.717+0000 7f0dc0c00700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dbc109670 0x7f0dbc1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.717+0000 7f0dc0c00700 1 -- 172.21.15.90:0/2636504134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dbc1043f0 msgr2=0x7f0dbc19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:30.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.717+0000 7f0dc0c00700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dbc1043f0 0x7f0dbc19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.717+0000 7f0dc0c00700 1 -- 172.21.15.90:0/2636504134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc1a1bb0 con 0x7f0dbc108d20 2024-05-27T07:48:30.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.718+0000 7f0dc0c00700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dbc108d20 0x7f0dbc19d330 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0da800b580 tx=0x7f0da80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:30.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.718+0000 7f0dba000700 1 -- 172.21.15.90:0/2636504134 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8027070 con 0x7f0dbc108d20 2024-05-27T07:48:30.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.718+0000 7f0dba000700 1 -- 172.21.15.90:0/2636504134 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0da800fc80 con 0x7f0dbc108d20 2024-05-27T07:48:30.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.718+0000 7f0dba000700 1 -- 172.21.15.90:0/2636504134 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8022d70 con 0x7f0dbc108d20 2024-05-27T07:48:30.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.718+0000 7f0dc2a00700 1 -- 172.21.15.90:0/2636504134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dbc1a1e40 con 0x7f0dbc108d20 2024-05-27T07:48:30.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.718+0000 7f0dc2a00700 1 -- 172.21.15.90:0/2636504134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dbc1a22d0 con 0x7f0dbc108d20 2024-05-27T07:48:30.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.724+0000 7f0dd2aeab80 1 Processor -- start 2024-05-27T07:48:30.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.724+0000 7f0dd2aeab80 1 -- start start 2024-05-27T07:48:30.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.724+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c613124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.724+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c61312a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.724+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61326b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.724+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c6123fec0 con 0x557c6127ea70 2024-05-27T07:48:30.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.724+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c6122fa60 con 0x557c61192ec0 2024-05-27T07:48:30.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.724+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557c61233aa0 con 0x557c61313010 2024-05-27T07:48:30.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.725+0000 7f0dba000700 1 -- 172.21.15.90:0/2636504134 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0da802a070 con 0x7f0dbc108d20 2024-05-27T07:48:30.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.725+0000 7f0dc1600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c61312a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.725+0000 7f0dc1600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c61312a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50884/0 (socket says 172.21.15.90:50884) 2024-05-27T07:48:30.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.725+0000 7f0dc1600700 1 -- 172.21.15.90:0/3216306860 learned_addr learned my addr 172.21.15.90:0/3216306860 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:30.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.726+0000 7f0dc1600700 1 -- 172.21.15.90:0/3216306860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 msgr2=0x557c61326b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:48:30.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.726+0000 7f0dc1600700 1 --2- 172.21.15.90:0/3216306860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61326b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.726+0000 7f0dc1600700 1 -- 172.21.15.90:0/3216306860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 msgr2=0x557c613124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:30.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.726+0000 7f0dc1600700 1 --2- 172.21.15.90:0/3216306860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c613124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.726+0000 7f0dc1600700 1 -- 172.21.15.90:0/3216306860 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c611f84a0 con 0x557c61192ec0 2024-05-27T07:48:30.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.727+0000 7f0dc1600700 1 --2- 172.21.15.90:0/3216306860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c61312a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0db000ad70 tx=0x7f0db000f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e73e05719672f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:30.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.727+0000 7f0dba000700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0da40b2a70 0x7f0da40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.727+0000 7f0daea00700 1 -- 172.21.15.90:0/3216306860 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0004220 con 0x557c61192ec0 2024-05-27T07:48:30.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.727+0000 7f0dba000700 1 -- 172.21.15.90:0/2636504134 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0da80e36a0 con 0x7f0dbc108d20 2024-05-27T07:48:30.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.727+0000 7f0dc2000700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0da40b2a70 0x7f0da40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.727+0000 7f0daea00700 1 -- 172.21.15.90:0/3216306860 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0db00043c0 con 0x557c61192ec0 2024-05-27T07:48:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.729+0000 7f0daea00700 1 -- 172.21.15.90:0/3216306860 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00186a0 con 0x557c61192ec0 2024-05-27T07:48:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.729+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/3216306860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 msgr2=0x557c61312a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.729+0000 7f0dc2000700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0da40b2a70 0x7f0da40b4f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0db40093f0 tx=0x7f0db4009380 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.729+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/3216306860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c61312a30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0db000ad70 tx=0x7f0db000f6a0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.729+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/3216306860 shutdown_connections 2024-05-27T07:48:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.729+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/3216306860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61326b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.729+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/3216306860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c613124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.729+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/3216306860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c61312a30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.730+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/3216306860 >> 172.21.15.90:0/3216306860 conn(0x557c6122c410 msgr2=0x557c6130d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.730+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/3216306860 shutdown_connections 2024-05-27T07:48:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.730+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/3216306860 wait complete. 2024-05-27T07:48:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.731+0000 7f0dd2aeab80 1 -- start start 2024-05-27T07:48:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.731+0000 7f0dd2aeab80 1 Processor -- start 2024-05-27T07:48:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.732+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61192ec0 0x557c61324060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.732+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6127ea70 0x557c613245a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.732+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c61313010 0x557c61324ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.732+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c61136020 con 0x557c61313010 2024-05-27T07:48:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.732+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c611d07e0 con 0x557c6127ea70 2024-05-27T07:48:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.732+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557c61327600 con 0x557c61192ec0 2024-05-27T07:48:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.732+0000 7f0dd2aeab80 10 client.0 _ll_register_callbacks cb 0x557c611d9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.732+0000 7f0dc1600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61192ec0 0x557c61324060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.732+0000 7f0dc2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c61313010 0x557c61324ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.733+0000 7f0dc0c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6127ea70 0x557c613245a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.733+0000 7f0dc1600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61192ec0 0x557c61324060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:35394/0 (socket says 172.21.15.90:35394) 2024-05-27T07:48:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.733+0000 7f0dc2000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c61313010 0x557c61324ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56548/0 (socket says 172.21.15.90:56548) 2024-05-27T07:48:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.733+0000 7f0dc1600700 1 -- 172.21.15.90:0/2016827259 learned_addr learned my addr 172.21.15.90:0/2016827259 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.733+0000 7f0dc2000700 1 -- 172.21.15.90:0/2016827259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61192ec0 msgr2=0x557c61324060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:30.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.733+0000 7f0dc2000700 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61192ec0 0x557c61324060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.733+0000 7f0dc2000700 1 -- 172.21.15.90:0/2016827259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6127ea70 msgr2=0x557c613245a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:30.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.733+0000 7f0dc2000700 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6127ea70 0x557c613245a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.733+0000 7f0dc2000700 1 -- 172.21.15.90:0/2016827259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6117f950 con 0x557c61313010 2024-05-27T07:48:30.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.734+0000 7f0dc2000700 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c61313010 0x557c61324ae0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009530 tx=0x7f0db4009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ce4029e076630a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:30.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.734+0000 7f0da3400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:30.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.734+0000 7f0da3400700 1 -- 172.21.15.90:0/2016827259 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db401c050 con 0x557c61313010 2024-05-27T07:48:30.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.734+0000 7f0da3400700 1 -- 172.21.15.90:0/2016827259 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0db4015850 con 0x557c61313010 2024-05-27T07:48:30.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.734+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2016827259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0db00043c0 con 0x557c61313010 2024-05-27T07:48:30.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.735+0000 7f0da3400700 1 -- 172.21.15.90:0/2016827259 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0db40048f0 con 0x557c61313010 2024-05-27T07:48:30.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.735+0000 7f0dd2aeab80 10 client.15861 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:30.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.735+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2016827259 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557c612000e0 con 0x557c61313010 2024-05-27T07:48:30.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.735+0000 7f0da3400700 1 -- 172.21.15.90:0/2016827259 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0db00043c0 con 0x557c61313010 2024-05-27T07:48:30.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.735+0000 7f0da3400700 1 -- 172.21.15.90:0/2016827259 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f0db4021a10 con 0x557c61313010 2024-05-27T07:48:30.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.736+0000 7f0dd2aeab80 10 client.15861 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:30.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.736+0000 7f0dd2aeab80 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:30.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.736+0000 7f0dd2aeab80 20 client.15861 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.736+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557c61334540 0x557c613369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.736+0000 7f0dd2aeab80 4 client.15861 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.736+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2016827259 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c61241e10 con 0x557c61334540 2024-05-27T07:48:30.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.737+0000 7f0dc1600700 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557c61334540 0x557c613369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.738+0000 7f0da3400700 10 client.15861 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:30.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.738+0000 7f0dc1600700 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557c61334540 0x557c613369f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29aab195e437c566 server_cookie=ae0dfac0775dbcdf in_seq=0 out_seq=0 2024-05-27T07:48:30.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.740+0000 7f0da3400700 1 -- 172.21.15.90:0/2016827259 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557c61241e10 con 0x557c61334540 2024-05-27T07:48:30.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.740+0000 7f0da3400700 10 client.15861 handle_command_reply: tid=0 2024-05-27T07:48:30.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.740+0000 7f0dd2aeab80 1 client.15861 shutdown 2024-05-27T07:48:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.740+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2016827259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c61313010 msgr2=0x557c61324ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.740+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c61313010 0x557c61324ae0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009530 tx=0x7f0db4009100 comp rx=0 tx=0).stop 2024-05-27T07:48:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.741+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2016827259 shutdown_connections 2024-05-27T07:48:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.741+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557c61334540 0x557c613369f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.741+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c61313010 0x557c61324ae0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.741+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61192ec0 0x557c61324060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.741+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2016827259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6127ea70 0x557c613245a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.741+0000 7f0da3400700 0 client.15861 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.741+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2016827259 >> 172.21.15.90:0/2016827259 conn(0x557c6122c410 msgr2=0x557c6130d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.741+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2016827259 shutdown_connections 2024-05-27T07:48:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.741+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2016827259 wait complete. 2024-05-27T07:48:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.741+0000 7f0dd2aeab80 20 client.15861 trim_cache size 0 max 16384 2024-05-27T07:48:30.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.775+0000 7f0dd2aeab80 1 Processor -- start 2024-05-27T07:48:30.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.775+0000 7f0dd2aeab80 1 -- start start 2024-05-27T07:48:30.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.775+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c61326140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.775+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6130d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.775+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61330780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.775+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c00ede0 con 0x557c6127ea70 2024-05-27T07:48:30.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.775+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c010920 con 0x557c61192ec0 2024-05-27T07:48:30.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.775+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c00f4b0 con 0x557c61313010 2024-05-27T07:48:30.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.776+0000 7f0dc0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6130d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.776+0000 7f0dc0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6130d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56560/0 (socket says 172.21.15.90:56560) 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.776+0000 7f0dc0c00700 1 -- 172.21.15.90:0/3252022310 learned_addr learned my addr 172.21.15.90:0/3252022310 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.776+0000 7f0dc2000700 1 --2- 172.21.15.90:0/3252022310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61330780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.776+0000 7f0dc0c00700 1 -- 172.21.15.90:0/3252022310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 msgr2=0x557c61330780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.776+0000 7f0dc0c00700 1 --2- 172.21.15.90:0/3252022310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61330780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.776+0000 7f0dc0c00700 1 -- 172.21.15.90:0/3252022310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 msgr2=0x557c61326140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.776+0000 7f0dc0c00700 1 --2- 172.21.15.90:0/3252022310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c61326140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.776+0000 7f0dc0c00700 1 -- 172.21.15.90:0/3252022310 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c61313580 con 0x557c6127ea70 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.777+0000 7f0dc0c00700 1 --2- 172.21.15.90:0/3252022310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6130d7c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0da80aa5f0 tx=0x7f0da800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cacf21b2a009fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.777+0000 7f0dae000700 1 -- 172.21.15.90:0/3252022310 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da80a9dd0 con 0x557c6127ea70 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.777+0000 7f0dae000700 1 -- 172.21.15.90:0/3252022310 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0da80aa170 con 0x557c6127ea70 2024-05-27T07:48:30.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.777+0000 7f0dae000700 1 -- 172.21.15.90:0/3252022310 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8022470 con 0x557c6127ea70 2024-05-27T07:48:30.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.778+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/3252022310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 msgr2=0x557c6130d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:30.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.778+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/3252022310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6130d7c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0da80aa5f0 tx=0x7f0da800bf40 comp rx=0 tx=0).stop 2024-05-27T07:48:30.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.778+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/3252022310 shutdown_connections 2024-05-27T07:48:30.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.778+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/3252022310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61330780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.778+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/3252022310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c61326140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.778+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/3252022310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6130d7c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.778+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/3252022310 >> 172.21.15.90:0/3252022310 conn(0x557c6122c410 msgr2=0x557c61311810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:30.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.779+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/3252022310 shutdown_connections 2024-05-27T07:48:30.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.779+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/3252022310 wait complete. 2024-05-27T07:48:30.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.779+0000 7f0dd2aeab80 1 -- start start 2024-05-27T07:48:30.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.780+0000 7f0dd2aeab80 1 Processor -- start 2024-05-27T07:48:30.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.781+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c6132c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.781+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6132cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.781+0000 7f0dd2aeab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61327930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.781+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c010250 con 0x557c6127ea70 2024-05-27T07:48:30.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.781+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c00fb80 con 0x557c61192ec0 2024-05-27T07:48:30.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.781+0000 7f0dd2aeab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c00cd20 con 0x557c61313010 2024-05-27T07:48:30.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.781+0000 7f0dc0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6132cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.781+0000 7f0dd2aeab80 10 client.0 _ll_register_callbacks cb 0x557c611d9cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:30.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.782+0000 7f0dc1600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c6132c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.782+0000 7f0dc1600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c6132c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50922/0 (socket says 172.21.15.90:50922) 2024-05-27T07:48:30.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.782+0000 7f0dc1600700 1 -- 172.21.15.90:0/2814416781 learned_addr learned my addr 172.21.15.90:0/2814416781 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:30.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.782+0000 7f0dc2000700 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61327930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.782+0000 7f0dc1600700 1 -- 172.21.15.90:0/2814416781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 msgr2=0x557c61327930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:30.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.782+0000 7f0dc1600700 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61327930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.783+0000 7f0dc1600700 1 -- 172.21.15.90:0/2814416781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 msgr2=0x557c6132cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:30.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.783+0000 7f0dc1600700 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6132cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.783+0000 7f0dc1600700 1 -- 172.21.15.90:0/2814416781 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6132d4d0 con 0x557c61192ec0 2024-05-27T07:48:30.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.783+0000 7f0dc0c00700 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6132cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:48:30.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.783+0000 7f0dc1600700 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c6132c980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0db001a040 tx=0x7f0db000af50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3ee60437e539b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:30.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.783+0000 7f0da3400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:48:30.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.784+0000 7f0da3400700 1 -- 172.21.15.90:0/2814416781 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0030060 con 0x557c61192ec0 2024-05-27T07:48:30.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.784+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2814416781 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da80aa170 con 0x557c61192ec0 2024-05-27T07:48:30.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.784+0000 7f0da3400700 1 -- 172.21.15.90:0/2814416781 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f0db0004310 con 0x557c61192ec0 2024-05-27T07:48:30.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.785+0000 7f0da3400700 1 -- 172.21.15.90:0/2814416781 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0db0037050 con 0x557c61192ec0 2024-05-27T07:48:30.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.785+0000 7f0dd2aeab80 10 client.35489 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:30.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.785+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2814416781 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557c6139f650 con 0x557c61192ec0 2024-05-27T07:48:30.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.785+0000 7f0da3400700 1 -- 172.21.15.90:0/2814416781 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0da80aa170 con 0x557c61192ec0 2024-05-27T07:48:30.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.786+0000 7f0da3400700 1 -- 172.21.15.90:0/2814416781 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f0db00054c0 con 0x557c61192ec0 2024-05-27T07:48:30.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.786+0000 7f0dd2aeab80 10 client.35489 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:30.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.786+0000 7f0dd2aeab80 10 client.35489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:30.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.786+0000 7f0dd2aeab80 20 client.35489 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:30.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.786+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557c6139fa50 0x557c613a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:30.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.786+0000 7f0dd2aeab80 4 client.35489 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:48:30.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.786+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2814416781 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557c6123b5f0 con 0x557c6139fa50 2024-05-27T07:48:30.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.787+0000 7f0dc0c00700 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557c6139fa50 0x557c613a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:30.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.788+0000 7f0dc0c00700 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557c6139fa50 0x557c613a1e70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c624bec594419153 server_cookie=245155aa1e5969b1 in_seq=0 out_seq=0 2024-05-27T07:48:30.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.788+0000 7f0da3400700 10 client.35489 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:30.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.791+0000 7f0da3400700 1 -- 172.21.15.90:0/2814416781 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557c6123b5f0 con 0x557c6139fa50 2024-05-27T07:48:30.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.791+0000 7f0da3400700 10 client.35489 handle_command_reply: tid=0 2024-05-27T07:48:30.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.791+0000 7f0dd2aeab80 1 client.35489 shutdown 2024-05-27T07:48:30.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.791+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2814416781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 msgr2=0x557c6132c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:30.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.791+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c6132c980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0db001a040 tx=0x7f0db000af50 comp rx=0 tx=0).stop 2024-05-27T07:48:30.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.791+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2814416781 shutdown_connections 2024-05-27T07:48:30.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.791+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557c6139fa50 0x557c613a1e70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.791+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c61313010 0x557c61327930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.791+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c61192ec0 0x557c6132c980 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.791+0000 7f0dd2aeab80 1 --2- 172.21.15.90:0/2814416781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c6127ea70 0x557c6132cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.792+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2814416781 >> 172.21.15.90:0/2814416781 conn(0x557c6122c410 msgr2=0x557c61323e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:30.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.792+0000 7f0da3400700 0 client.35489 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:30.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.792+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2814416781 shutdown_connections 2024-05-27T07:48:30.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.792+0000 7f0dd2aeab80 1 -- 172.21.15.90:0/2814416781 wait complete. 2024-05-27T07:48:30.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.792+0000 7f0dd2aeab80 20 client.35489 trim_cache size 0 max 16384 2024-05-27T07:48:30.793 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:48:30.793 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:48:30.793 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "d0cf478e-4faa-4fdf-9040-1de4b5de0104", 2024-05-27T07:48:30.793 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:48:30.793 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:48:30.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.793+0000 7f0dc2a00700 1 -- 172.21.15.90:0/2636504134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0da40b2a70 msgr2=0x7f0da40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.793+0000 7f0dc2a00700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0da40b2a70 0x7f0da40b4f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0db40093f0 tx=0x7f0db4009380 comp rx=0 tx=0).stop 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.793+0000 7f0dc2a00700 1 -- 172.21.15.90:0/2636504134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dbc108d20 msgr2=0x7f0dbc19d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.793+0000 7f0dc2a00700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dbc108d20 0x7f0dbc19d330 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0da800b580 tx=0x7f0da80057c0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.793+0000 7f0dc2a00700 1 -- 172.21.15.90:0/2636504134 shutdown_connections 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.793+0000 7f0dc2a00700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dbc109670 0x7f0dbc1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.793+0000 7f0dc2a00700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0da40b2a70 0x7f0da40b4f80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.793+0000 7f0dc2a00700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0dbc1043f0 0x7f0dbc19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.793+0000 7f0dc2a00700 1 --2- 172.21.15.90:0/2636504134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0dbc108d20 0x7f0dbc19d330 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.793+0000 7f0dc2a00700 1 -- 172.21.15.90:0/2636504134 >> 172.21.15.90:0/2636504134 conn(0x7f0dbc079d00 msgr2=0x7f0dbc101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.794+0000 7f0dc2a00700 1 -- 172.21.15.90:0/2636504134 shutdown_connections 2024-05-27T07:48:30.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:30.794+0000 7f0dc2a00700 1 -- 172.21.15.90:0/2636504134 wait complete. 2024-05-27T07:48:31.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:31 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:48:31.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:31 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:31.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:31 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:48:31.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:31 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:48:31.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:31 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:31.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:31 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:48:31.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:31 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:48:31.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:31 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:31.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:31 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:48:31.664 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:48:32.124 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:32.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:32 smithi090 ceph-mon[31429]: pgmap v68: 129 pgs: 129 active+clean; 399 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 15 MiB/s wr, 25 op/s 2024-05-27T07:48:32.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:32 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:32.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:32 smithi136 ceph-mon[35324]: pgmap v68: 129 pgs: 129 active+clean; 399 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 15 MiB/s wr, 25 op/s 2024-05-27T07:48:32.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:32 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:32.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:32 smithi129 ceph-mon[35254]: pgmap v68: 129 pgs: 129 active+clean; 399 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 15 MiB/s wr, 25 op/s 2024-05-27T07:48:32.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:32 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:48:32.928 INFO:teuthology.orchestra.run.smithi090.stderr:1600000 of 3200000 tuples (50%) done (elapsed 48.98 s, remaining 48.98 s) 2024-05-27T07:48:33.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:33 smithi090 ceph-mon[31429]: pgmap v69: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 127 op/s 2024-05-27T07:48:33.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:33 smithi136 ceph-mon[35324]: pgmap v69: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 127 op/s 2024-05-27T07:48:33.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:33 smithi129 ceph-mon[35254]: pgmap v69: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 127 op/s 2024-05-27T07:48:34.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:34 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:48:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.479+0000 7f1e20e00700 1 -- 172.21.15.90:0/4119776639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e1c105280 msgr2=0x7f1e1c105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.479+0000 7f1e20e00700 1 --2- 172.21.15.90:0/4119776639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e1c105280 0x7f1e1c105690 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009990 tx=0x7f1e0c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.479+0000 7f1e20e00700 1 -- 172.21.15.90:0/4119776639 shutdown_connections 2024-05-27T07:48:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.479+0000 7f1e20e00700 1 --2- 172.21.15.90:0/4119776639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c10e6f0 0x7f1e1c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.479+0000 7f1e20e00700 1 --2- 172.21.15.90:0/4119776639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e1c105280 0x7f1e1c105690 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.479+0000 7f1e20e00700 1 --2- 172.21.15.90:0/4119776639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c105bd0 0x7f1e1c10e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.479+0000 7f1e20e00700 1 -- 172.21.15.90:0/4119776639 >> 172.21.15.90:0/4119776639 conn(0x7f1e1c078af0 msgr2=0x7f1e1c078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:34.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.480+0000 7f1e20e00700 1 -- 172.21.15.90:0/4119776639 shutdown_connections 2024-05-27T07:48:34.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.480+0000 7f1e20e00700 1 -- 172.21.15.90:0/4119776639 wait complete. 2024-05-27T07:48:34.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.481+0000 7f1e20e00700 1 Processor -- start 2024-05-27T07:48:34.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.481+0000 7f1e20e00700 1 -- start start 2024-05-27T07:48:34.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.482+0000 7f1e20e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c105280 0x7f1e1c102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.482+0000 7f1e20e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e1c105bd0 0x7f1e1c1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.482+0000 7f1e20e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c10e6f0 0x7f1e1c1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.482+0000 7f1e20e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1140f0 con 0x7f1e1c105bd0 2024-05-27T07:48:34.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.482+0000 7f1e20e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c113f70 con 0x7f1e1c105280 2024-05-27T07:48:34.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.482+0000 7f1e20e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c114270 con 0x7f1e1c10e6f0 2024-05-27T07:48:34.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.482+0000 7f1e1b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c105280 0x7f1e1c102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.482+0000 7f1e1b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c105280 0x7f1e1c102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50938/0 (socket says 172.21.15.90:50938) 2024-05-27T07:48:34.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.482+0000 7f1e1b400700 1 -- 172.21.15.90:0/749657042 learned_addr learned my addr 172.21.15.90:0/749657042 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:34.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.482+0000 7f1e1be00700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c10e6f0 0x7f1e1c1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.483+0000 7f1e1b400700 1 -- 172.21.15.90:0/749657042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c10e6f0 msgr2=0x7f1e1c1039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:34.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.483+0000 7f1e1b400700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c10e6f0 0x7f1e1c1039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.483+0000 7f1e1b400700 1 -- 172.21.15.90:0/749657042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e1c105bd0 msgr2=0x7f1e1c1034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:34.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.483+0000 7f1e1b400700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e1c105bd0 0x7f1e1c1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.483+0000 7f1e1b400700 1 -- 172.21.15.90:0/749657042 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c1a8ef0 con 0x7f1e1c105280 2024-05-27T07:48:34.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.483+0000 7f1e1be00700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c10e6f0 0x7f1e1c1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:34.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.484+0000 7f1e1b400700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c105280 0x7f1e1c102f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0057f0 tx=0x7f1e0c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:34.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.484+0000 7f1e13e00700 1 -- 172.21.15.90:0/749657042 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c027070 con 0x7f1e1c105280 2024-05-27T07:48:34.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.484+0000 7f1e20e00700 1 -- 172.21.15.90:0/749657042 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1c1a9180 con 0x7f1e1c105280 2024-05-27T07:48:34.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.484+0000 7f1e13e00700 1 -- 172.21.15.90:0/749657042 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e0c00fd60 con 0x7f1e1c105280 2024-05-27T07:48:34.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.484+0000 7f1e13e00700 1 -- 172.21.15.90:0/749657042 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c022d50 con 0x7f1e1c105280 2024-05-27T07:48:34.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.484+0000 7f1e20e00700 1 -- 172.21.15.90:0/749657042 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e1c1a96b0 con 0x7f1e1c105280 2024-05-27T07:48:34.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.489+0000 7f1e30e4eb80 1 Processor -- start 2024-05-27T07:48:34.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.490+0000 7f1e30e4eb80 1 -- start start 2024-05-27T07:48:34.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.490+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f847192750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.490+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847192c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.490+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 0x55f84719e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.490+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f8470bfe40 con 0x55f84719c2c0 2024-05-27T07:48:34.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.490+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f8470af9e0 con 0x55f847012e40 2024-05-27T07:48:34.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.490+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f8470b3a20 con 0x55f8470fe9f0 2024-05-27T07:48:34.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.491+0000 7f1e13e00700 1 -- 172.21.15.90:0/749657042 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1e0c02a070 con 0x7f1e1c105280 2024-05-27T07:48:34.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.491+0000 7f1e1be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f847192750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.491+0000 7f1e1be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f847192750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:35440/0 (socket says 172.21.15.90:35440) 2024-05-27T07:48:34.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.491+0000 7f1e1be00700 1 -- 172.21.15.90:0/2704715477 learned_addr learned my addr 172.21.15.90:0/2704715477 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:34.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.491+0000 7f1e1b400700 1 --2- 172.21.15.90:0/2704715477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847192c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.491+0000 7f1e1aa00700 1 --2- 172.21.15.90:0/2704715477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 0x55f84719e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.492+0000 7f1e1be00700 1 -- 172.21.15.90:0/2704715477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 msgr2=0x55f847192c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:34.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.492+0000 7f1e1be00700 1 --2- 172.21.15.90:0/2704715477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847192c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.492+0000 7f1e1be00700 1 -- 172.21.15.90:0/2704715477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 msgr2=0x55f84719e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:34.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.492+0000 7f1e1be00700 1 --2- 172.21.15.90:0/2704715477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 0x55f84719e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.492+0000 7f1e1be00700 1 -- 172.21.15.90:0/2704715477 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f847078420 con 0x55f8470fe9f0 2024-05-27T07:48:34.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.492+0000 7f1e1be00700 1 --2- 172.21.15.90:0/2704715477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f847192750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400e160 tx=0x7f1e1400e470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6baea4277ea8759 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.493+0000 7f1e10c00700 1 -- 172.21.15.90:0/2704715477 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400edb0 con 0x55f8470fe9f0 2024-05-27T07:48:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.493+0000 7f1e13e00700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1e000b2af0 0x7f1e000b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.493+0000 7f1e10c00700 1 -- 172.21.15.90:0/2704715477 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e14004020 con 0x55f8470fe9f0 2024-05-27T07:48:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.493+0000 7f1e10c00700 1 -- 172.21.15.90:0/2704715477 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14014480 con 0x55f8470fe9f0 2024-05-27T07:48:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.493+0000 7f1e13e00700 1 -- 172.21.15.90:0/749657042 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1e0c0e3730 con 0x7f1e1c105280 2024-05-27T07:48:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.493+0000 7f1e1b400700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1e000b2af0 0x7f1e000b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.494+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/2704715477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 msgr2=0x55f847192750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.494+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/2704715477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f847192750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400e160 tx=0x7f1e1400e470 comp rx=0 tx=0).stop 2024-05-27T07:48:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.494+0000 7f1e1b400700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1e000b2af0 0x7f1e000b5000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55f84718e970 tx=0x7f1e0c005a00 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.494+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/2704715477 shutdown_connections 2024-05-27T07:48:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.494+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/2704715477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 0x55f84719e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.494+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/2704715477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f847192750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.494+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/2704715477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847192c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.494+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/2704715477 >> 172.21.15.90:0/2704715477 conn(0x55f8470ac390 msgr2=0x55f84718d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.494+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/2704715477 shutdown_connections 2024-05-27T07:48:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.494+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/2704715477 wait complete. 2024-05-27T07:48:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.495+0000 7f1e30e4eb80 1 -- start start 2024-05-27T07:48:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.495+0000 7f1e30e4eb80 1 Processor -- start 2024-05-27T07:48:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847199250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8470fe9f0 0x55f847199790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f84719c2c0 0x55f847199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f846fb5fa0 con 0x55f8470fe9f0 2024-05-27T07:48:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f847050760 con 0x55f847012e40 2024-05-27T07:48:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f84719ee00 con 0x55f84719c2c0 2024-05-27T07:48:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e30e4eb80 10 client.0 _ll_register_callbacks cb 0x55f847059c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e1aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8470fe9f0 0x55f847199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e1be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e1aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8470fe9f0 0x55f847199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56612/0 (socket says 172.21.15.90:56612) 2024-05-27T07:48:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.496+0000 7f1e1aa00700 1 -- 172.21.15.90:0/285227581 learned_addr learned my addr 172.21.15.90:0/285227581 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.497+0000 7f1e1be00700 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f84719c2c0 0x55f847199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.497+0000 7f1e1aa00700 1 -- 172.21.15.90:0/285227581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f84719c2c0 msgr2=0x55f847199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.497+0000 7f1e1aa00700 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f84719c2c0 0x55f847199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.497+0000 7f1e1aa00700 1 -- 172.21.15.90:0/285227581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 msgr2=0x55f847199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.497+0000 7f1e1aa00700 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.497+0000 7f1e1aa00700 1 -- 172.21.15.90:0/285227581 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f846fff8d0 con 0x55f8470fe9f0 2024-05-27T07:48:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.497+0000 7f1e1aa00700 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8470fe9f0 0x55f847199790 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800e680 tx=0x7f1e0800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38efa83d185e9655 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.497+0000 7f1e05600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.498+0000 7f1e05600700 1 -- 172.21.15.90:0/285227581 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e08004220 con 0x55f8470fe9f0 2024-05-27T07:48:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.498+0000 7f1e05600700 1 -- 172.21.15.90:0/285227581 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1e080043c0 con 0x55f8470fe9f0 2024-05-27T07:48:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.498+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/285227581 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e14004020 con 0x55f8470fe9f0 2024-05-27T07:48:34.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.498+0000 7f1e05600700 1 -- 172.21.15.90:0/285227581 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1e08022050 con 0x55f8470fe9f0 2024-05-27T07:48:34.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.498+0000 7f1e30e4eb80 10 client.15876 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:34.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.498+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/285227581 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f847080060 con 0x55f8470fe9f0 2024-05-27T07:48:34.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.499+0000 7f1e05600700 1 -- 172.21.15.90:0/285227581 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e14004020 con 0x55f8470fe9f0 2024-05-27T07:48:34.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.499+0000 7f1e05600700 1 -- 172.21.15.90:0/285227581 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1e08010660 con 0x55f8470fe9f0 2024-05-27T07:48:34.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.499+0000 7f1e30e4eb80 10 client.15876 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:34.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.499+0000 7f1e30e4eb80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:34.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.499+0000 7f1e30e4eb80 20 client.15876 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:34.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.499+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8471abde0 0x55f8471ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.499+0000 7f1e30e4eb80 4 client.15876 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:34.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.499+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/285227581 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8470c1d90 con 0x55f8471abde0 2024-05-27T07:48:34.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:34 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:48:34.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.500+0000 7f1e1be00700 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8471abde0 0x55f8471ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.501+0000 7f1e05600700 10 client.15876 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:34.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.501+0000 7f1e1be00700 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8471abde0 0x55f8471ae240 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0b7c57b7f3100d7 server_cookie=e72e9f7faf1c18a0 in_seq=0 out_seq=0 2024-05-27T07:48:34.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.503+0000 7f1e05600700 1 -- 172.21.15.90:0/285227581 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f8470c1d90 con 0x55f8471abde0 2024-05-27T07:48:34.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.503+0000 7f1e05600700 10 client.15876 handle_command_reply: tid=0 2024-05-27T07:48:34.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.503+0000 7f1e30e4eb80 1 client.15876 shutdown 2024-05-27T07:48:34.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.503+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/285227581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8470fe9f0 msgr2=0x55f847199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:34.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.503+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8470fe9f0 0x55f847199790 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800e680 tx=0x7f1e0800e990 comp rx=0 tx=0).stop 2024-05-27T07:48:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.504+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/285227581 shutdown_connections 2024-05-27T07:48:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.504+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8471abde0 0x55f8471ae240 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.504+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f84719c2c0 0x55f847199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.504+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.504+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/285227581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8470fe9f0 0x55f847199790 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.504+0000 7f1e05600700 0 client.15876 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.504+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/285227581 >> 172.21.15.90:0/285227581 conn(0x55f8470ac390 msgr2=0x55f84718d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.504+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/285227581 shutdown_connections 2024-05-27T07:48:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.504+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/285227581 wait complete. 2024-05-27T07:48:34.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.504+0000 7f1e30e4eb80 20 client.15876 trim_cache size 0 max 16384 2024-05-27T07:48:34.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.531+0000 7f1e30e4eb80 1 Processor -- start 2024-05-27T07:48:34.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.531+0000 7f1e30e4eb80 1 -- start start 2024-05-27T07:48:34.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.531+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.531+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f8471a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.531+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 0x55f847191830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.531+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1b2470 con 0x55f84719c2c0 2024-05-27T07:48:34.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.531+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1b3fb0 con 0x55f847012e40 2024-05-27T07:48:34.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.531+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1b2b40 con 0x55f8470fe9f0 2024-05-27T07:48:34.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 0x55f847191830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 0x55f847191830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56622/0 (socket says 172.21.15.90:56622) 2024-05-27T07:48:34.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1be00700 1 -- 172.21.15.90:0/1469655607 learned_addr learned my addr 172.21.15.90:0/1469655607 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:34.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1be00700 1 --2- 172.21.15.90:0/1469655607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1aa00700 1 --2- 172.21.15.90:0/1469655607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f8471a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1be00700 1 -- 172.21.15.90:0/1469655607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 msgr2=0x55f8471a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:34.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1be00700 1 --2- 172.21.15.90:0/1469655607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f8471a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1be00700 1 -- 172.21.15.90:0/1469655607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 msgr2=0x55f847194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1be00700 1 --2- 172.21.15.90:0/1469655607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847194aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1be00700 1 -- 172.21.15.90:0/1469655607 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f847187930 con 0x55f84719c2c0 2024-05-27T07:48:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.532+0000 7f1e1be00700 1 --2- 172.21.15.90:0/1469655607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 0x55f847191830 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1e14006130 tx=0x7f1e14009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1083ad9ea7e89de8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.533+0000 7f1e07e00700 1 -- 172.21.15.90:0/1469655607 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1402f020 con 0x55f84719c2c0 2024-05-27T07:48:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.533+0000 7f1e07e00700 1 -- 172.21.15.90:0/1469655607 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e14003a00 con 0x55f84719c2c0 2024-05-27T07:48:34.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.533+0000 7f1e07e00700 1 -- 172.21.15.90:0/1469655607 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14005020 con 0x55f84719c2c0 2024-05-27T07:48:34.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.534+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1469655607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 msgr2=0x55f847191830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:34.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.534+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/1469655607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 0x55f847191830 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1e14006130 tx=0x7f1e14009040 comp rx=0 tx=0).stop 2024-05-27T07:48:34.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.534+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1469655607 shutdown_connections 2024-05-27T07:48:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.534+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/1469655607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f84719c2c0 0x55f847191830 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.534+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/1469655607 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f847012e40 0x55f847194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.534+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/1469655607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f8471a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.534+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1469655607 >> 172.21.15.90:0/1469655607 conn(0x55f8470ac390 msgr2=0x55f847191120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.534+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1469655607 shutdown_connections 2024-05-27T07:48:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.534+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1469655607 wait complete. 2024-05-27T07:48:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.535+0000 7f1e30e4eb80 1 -- start start 2024-05-27T07:48:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.535+0000 7f1e30e4eb80 1 Processor -- start 2024-05-27T07:48:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.536+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f847012e40 0x55f8471a4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.536+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f84719fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.536+0000 7f1e30e4eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f84719c2c0 0x55f8471a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.536+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1b38e0 con 0x55f847012e40 2024-05-27T07:48:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.536+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1b3210 con 0x55f84719c2c0 2024-05-27T07:48:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.536+0000 7f1e30e4eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c1b03f0 con 0x55f8470fe9f0 2024-05-27T07:48:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.536+0000 7f1e30e4eb80 10 client.0 _ll_register_callbacks cb 0x55f847059c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.536+0000 7f1e1be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f847012e40 0x55f8471a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.536+0000 7f1e1be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f847012e40 0x55f8471a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56630/0 (socket says 172.21.15.90:56630) 2024-05-27T07:48:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.536+0000 7f1e1be00700 1 -- 172.21.15.90:0/1856494514 learned_addr learned my addr 172.21.15.90:0/1856494514 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.537+0000 7f1e1aa00700 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f84719fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.537+0000 7f1e1be00700 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f84719c2c0 0x55f8471a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.537+0000 7f1e1be00700 1 -- 172.21.15.90:0/1856494514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 msgr2=0x55f84719fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.537+0000 7f1e1be00700 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f84719fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.537+0000 7f1e1be00700 1 -- 172.21.15.90:0/1856494514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f84719c2c0 msgr2=0x55f8471a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.537+0000 7f1e1be00700 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f84719c2c0 0x55f8471a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.537+0000 7f1e1be00700 1 -- 172.21.15.90:0/1856494514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8471a0a10 con 0x55f847012e40 2024-05-27T07:48:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.537+0000 7f1e1be00700 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f847012e40 0x55f8471a4910 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400ef60 tx=0x7f1e14007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4831e044e42f7399 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.537+0000 7f1e05600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.538+0000 7f1e05600700 1 -- 172.21.15.90:0/1856494514 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1403a060 con 0x55f847012e40 2024-05-27T07:48:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.538+0000 7f1e05600700 1 -- 172.21.15.90:0/1856494514 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f1e14010040 con 0x55f847012e40 2024-05-27T07:48:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.538+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1856494514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e14003a00 con 0x55f847012e40 2024-05-27T07:48:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.538+0000 7f1e05600700 1 -- 172.21.15.90:0/1856494514 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1e1402f030 con 0x55f847012e40 2024-05-27T07:48:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.538+0000 7f1e30e4eb80 10 client.15882 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.538+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1856494514 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f8472168d0 con 0x55f847012e40 2024-05-27T07:48:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.539+0000 7f1e05600700 1 -- 172.21.15.90:0/1856494514 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e14003a00 con 0x55f847012e40 2024-05-27T07:48:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.539+0000 7f1e05600700 1 -- 172.21.15.90:0/1856494514 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1e14034030 con 0x55f847012e40 2024-05-27T07:48:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.539+0000 7f1e30e4eb80 10 client.15882 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.539+0000 7f1e30e4eb80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.539+0000 7f1e30e4eb80 20 client.15882 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.539+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f847216ce0 0x55f847219100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.539+0000 7f1e30e4eb80 4 client.15882 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:48:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.539+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1856494514 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8470bb570 con 0x55f847216ce0 2024-05-27T07:48:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.539+0000 7f1e1aa00700 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f847216ce0 0x55f847219100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:34.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.540+0000 7f1e1aa00700 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f847216ce0 0x55f847219100 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f786b63a7c346c72 server_cookie=d154a48b52ae5a09 in_seq=0 out_seq=0 2024-05-27T07:48:34.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.540+0000 7f1e05600700 10 client.15882 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:34.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.541+0000 7f1e05600700 1 -- 172.21.15.90:0/1856494514 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f8470bb570 con 0x55f847216ce0 2024-05-27T07:48:34.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.541+0000 7f1e05600700 10 client.15882 handle_command_reply: tid=0 2024-05-27T07:48:34.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.541+0000 7f1e30e4eb80 1 client.15882 shutdown 2024-05-27T07:48:34.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.542+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1856494514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f847012e40 msgr2=0x55f8471a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:34.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.542+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f847012e40 0x55f8471a4910 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400ef60 tx=0x7f1e14007c50 comp rx=0 tx=0).stop 2024-05-27T07:48:34.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.543+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1856494514 shutdown_connections 2024-05-27T07:48:34.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.543+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f847216ce0 0x55f847219100 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.543+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f84719c2c0 0x55f8471a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.543+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f847012e40 0x55f8471a4910 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.543+0000 7f1e30e4eb80 1 --2- 172.21.15.90:0/1856494514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8470fe9f0 0x55f84719fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.543+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1856494514 >> 172.21.15.90:0/1856494514 conn(0x55f8470ac390 msgr2=0x55f847194800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:34.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.543+0000 7f1e05600700 0 client.15882 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:34.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.543+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1856494514 shutdown_connections 2024-05-27T07:48:34.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.543+0000 7f1e30e4eb80 1 -- 172.21.15.90:0/1856494514 wait complete. 2024-05-27T07:48:34.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.544+0000 7f1e30e4eb80 20 client.15882 trim_cache size 0 max 16384 2024-05-27T07:48:34.544 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:48:34.544 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:48:34.544 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:48:34.544 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:48:34.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.545+0000 7f1e20e00700 1 -- 172.21.15.90:0/749657042 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1e000b2af0 msgr2=0x7f1e000b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:34.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.545+0000 7f1e20e00700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1e000b2af0 0x7f1e000b5000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55f84718e970 tx=0x7f1e0c005a00 comp rx=0 tx=0).stop 2024-05-27T07:48:34.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.545+0000 7f1e20e00700 1 -- 172.21.15.90:0/749657042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c105280 msgr2=0x7f1e1c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:34.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.545+0000 7f1e20e00700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c105280 0x7f1e1c102f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0057f0 tx=0x7f1e0c009f90 comp rx=0 tx=0).stop 2024-05-27T07:48:34.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.545+0000 7f1e20e00700 1 -- 172.21.15.90:0/749657042 shutdown_connections 2024-05-27T07:48:34.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.545+0000 7f1e20e00700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e1c10e6f0 0x7f1e1c1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.545+0000 7f1e20e00700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1e000b2af0 0x7f1e000b5000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.545+0000 7f1e20e00700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c105280 0x7f1e1c102f70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.545+0000 7f1e20e00700 1 --2- 172.21.15.90:0/749657042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e1c105bd0 0x7f1e1c1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:34.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.545+0000 7f1e20e00700 1 -- 172.21.15.90:0/749657042 >> 172.21.15.90:0/749657042 conn(0x7f1e1c078af0 msgr2=0x7f1e1c101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:34.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.546+0000 7f1e20e00700 1 -- 172.21.15.90:0/749657042 shutdown_connections 2024-05-27T07:48:34.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:34.546+0000 7f1e20e00700 1 -- 172.21.15.90:0/749657042 wait complete. 2024-05-27T07:48:34.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:34 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:48:35.419 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:48:35.419 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:48:35.495 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:35 smithi090 ceph-mon[31429]: pgmap v70: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 127 op/s 2024-05-27T07:48:35.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:35 smithi136 ceph-mon[35324]: pgmap v70: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 127 op/s 2024-05-27T07:48:35.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:35 smithi129 ceph-mon[35254]: pgmap v70: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 127 op/s 2024-05-27T07:48:35.677 INFO:teuthology.orchestra.run.smithi090.stderr:1700000 of 3200000 tuples (53%) done (elapsed 51.73 s, remaining 45.64 s) 2024-05-27T07:48:35.926 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:36.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.427+0000 7fd1d3e00640 1 -- 172.21.15.90:0/552714831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1cc096af0 msgr2=0x7fd1cc096f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.427+0000 7fd1d3e00640 1 --2- 172.21.15.90:0/552714831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1cc096af0 0x7fd1cc096f00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009900 tx=0x7fd1c002a410 comp rx=0 tx=0).stop 2024-05-27T07:48:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.427+0000 7fd1d3e00640 1 -- 172.21.15.90:0/552714831 shutdown_connections 2024-05-27T07:48:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.427+0000 7fd1d3e00640 1 --2- 172.21.15.90:0/552714831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1cc0a1380 0x7fd1cc0a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.427+0000 7fd1d3e00640 1 --2- 172.21.15.90:0/552714831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1cc096af0 0x7fd1cc096f00 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.427+0000 7fd1d3e00640 1 --2- 172.21.15.90:0/552714831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1cc09ca20 0x7fd1cc0965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.427+0000 7fd1d3e00640 1 -- 172.21.15.90:0/552714831 >> 172.21.15.90:0/552714831 conn(0x7fd1cc00d3c0 msgr2=0x7fd1cc00d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:36.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.427+0000 7fd1d3e00640 1 -- 172.21.15.90:0/552714831 shutdown_connections 2024-05-27T07:48:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.427+0000 7fd1d3e00640 1 -- 172.21.15.90:0/552714831 wait complete. 2024-05-27T07:48:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.429+0000 7fd1d3e00640 1 Processor -- start 2024-05-27T07:48:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.429+0000 7fd1d3e00640 1 -- start start 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.430+0000 7fd1d3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1cc096af0 0x7fd1cc12bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.430+0000 7fd1d3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1cc09ca20 0x7fd1cc12c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.430+0000 7fd1d3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1cc0a1380 0x7fd1cc12c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.430+0000 7fd1d3e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc0a6a90 con 0x7fd1cc09ca20 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.430+0000 7fd1d3e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc0a6910 con 0x7fd1cc0a1380 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.430+0000 7fd1d3e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc0a6c10 con 0x7fd1cc096af0 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.430+0000 7fd1d2000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1cc09ca20 0x7fd1cc12c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.430+0000 7fd1d2000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1cc09ca20 0x7fd1cc12c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56638/0 (socket says 172.21.15.90:56638) 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.430+0000 7fd1d2000640 1 -- 172.21.15.90:0/1123283236 learned_addr learned my addr 172.21.15.90:0/1123283236 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.431+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1cc096af0 0x7fd1cc12bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.431+0000 7fd1d3400640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1cc0a1380 0x7fd1cc12c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.431+0000 7fd1d2000640 1 -- 172.21.15.90:0/1123283236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1cc096af0 msgr2=0x7fd1cc12bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:36.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.431+0000 7fd1d2000640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1cc096af0 0x7fd1cc12bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.431+0000 7fd1d2000640 1 -- 172.21.15.90:0/1123283236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1cc0a1380 msgr2=0x7fd1cc12c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.431+0000 7fd1d2000640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1cc0a1380 0x7fd1cc12c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.431+0000 7fd1d2000640 1 -- 172.21.15.90:0/1123283236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc13fba0 con 0x7fd1cc09ca20 2024-05-27T07:48:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.431+0000 7fd1d2000640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1cc09ca20 0x7fd1cc12c480 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0005940 tx=0x7fd1c00040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.432+0000 7fd1cb400640 1 -- 172.21.15.90:0/1123283236 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c003f030 con 0x7fd1cc09ca20 2024-05-27T07:48:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.432+0000 7fd1d3e00640 1 -- 172.21.15.90:0/1123283236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc13fe30 con 0x7fd1cc09ca20 2024-05-27T07:48:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.432+0000 7fd1cb400640 1 -- 172.21.15.90:0/1123283236 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1c0003a80 con 0x7fd1cc09ca20 2024-05-27T07:48:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.432+0000 7fd1cb400640 1 -- 172.21.15.90:0/1123283236 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0035070 con 0x7fd1cc09ca20 2024-05-27T07:48:36.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.432+0000 7fd1d3e00640 1 -- 172.21.15.90:0/1123283236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc140290 con 0x7fd1cc09ca20 2024-05-27T07:48:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.437+0000 7fd1e0996740 1 Processor -- start 2024-05-27T07:48:36.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.437+0000 7fd1e0996740 1 -- start start 2024-05-27T07:48:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.437+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 0x557a4bf45a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.437+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf45fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.437+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb824d0 0x557a4bf46510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.437+0000 7fd1e0996740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557a4beb6b90 con 0x557a4bb824d0 2024-05-27T07:48:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.437+0000 7fd1e0996740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557a4bea6730 con 0x557a4bb7a2f0 2024-05-27T07:48:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.437+0000 7fd1e0996740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557a4beaa770 con 0x557a4bba31b0 2024-05-27T07:48:36.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.437+0000 7fd1cb400640 1 -- 172.21.15.90:0/1123283236 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd1c0042030 con 0x7fd1cc09ca20 2024-05-27T07:48:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.438+0000 7fd1d3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 0x557a4bf45a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.438+0000 7fd1d3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 0x557a4bf45a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:35504/0 (socket says 172.21.15.90:35504) 2024-05-27T07:48:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.438+0000 7fd1d3400640 1 -- 172.21.15.90:0/2887607776 learned_addr learned my addr 172.21.15.90:0/2887607776 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.438+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf45fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.438+0000 7fd1d2000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb824d0 0x557a4bf46510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.439+0000 7fd1d3400640 1 -- 172.21.15.90:0/2887607776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 msgr2=0x557a4bf45fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.439+0000 7fd1d3400640 1 --2- 172.21.15.90:0/2887607776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf45fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.439+0000 7fd1d3400640 1 -- 172.21.15.90:0/2887607776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb824d0 msgr2=0x557a4bf46510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.439+0000 7fd1d3400640 1 --2- 172.21.15.90:0/2887607776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb824d0 0x557a4bf46510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.439+0000 7fd1d3400640 1 -- 172.21.15.90:0/2887607776 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4be3f8b0 con 0x557a4bba31b0 2024-05-27T07:48:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.439+0000 7fd1d2000640 1 --2- 172.21.15.90:0/2887607776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb824d0 0x557a4bf46510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.439+0000 7fd1d3400640 1 --2- 172.21.15.90:0/2887607776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 0x557a4bf45a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4068be0 tx=0x7fd1d406c550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d65a837e4789e2d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.440+0000 7fd1bfe00640 1 -- 172.21.15.90:0/2887607776 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d407f630 con 0x557a4bba31b0 2024-05-27T07:48:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.440+0000 7fd1bfe00640 1 -- 172.21.15.90:0/2887607776 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1d407fc60 con 0x557a4bba31b0 2024-05-27T07:48:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.440+0000 7fd1bfe00640 1 -- 172.21.15.90:0/2887607776 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d407eaf0 con 0x557a4bba31b0 2024-05-27T07:48:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.440+0000 7fd1cb400640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1b80b2960 0x7fd1b80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.440+0000 7fd1cb400640 1 -- 172.21.15.90:0/1123283236 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd1c0102e30 con 0x7fd1cc09ca20 2024-05-27T07:48:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.441+0000 7fd1e0996740 1 -- 172.21.15.90:0/2887607776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 msgr2=0x557a4bf45a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.441+0000 7fd1e0996740 1 --2- 172.21.15.90:0/2887607776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 0x557a4bf45a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4068be0 tx=0x7fd1d406c550 comp rx=0 tx=0).stop 2024-05-27T07:48:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.441+0000 7fd1e0996740 1 -- 172.21.15.90:0/2887607776 shutdown_connections 2024-05-27T07:48:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.441+0000 7fd1e0996740 1 --2- 172.21.15.90:0/2887607776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf45fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.441+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1b80b2960 0x7fd1b80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.441+0000 7fd1e0996740 1 --2- 172.21.15.90:0/2887607776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb824d0 0x557a4bf46510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.441+0000 7fd1e0996740 1 --2- 172.21.15.90:0/2887607776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 0x557a4bf45a90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.441+0000 7fd1e0996740 1 -- 172.21.15.90:0/2887607776 >> 172.21.15.90:0/2887607776 conn(0x557a4bb97300 msgr2=0x557a4bf42240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.441+0000 7fd1e0996740 1 -- 172.21.15.90:0/2887607776 shutdown_connections 2024-05-27T07:48:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.442+0000 7fd1e0996740 1 -- 172.21.15.90:0/2887607776 wait complete. 2024-05-27T07:48:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.442+0000 7fd1e0996740 1 -- start start 2024-05-27T07:48:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.442+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1b80b2960 0x7fd1b80b4e70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x557a4bf46e40 tx=0x7fd1c400a2d0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.442+0000 7fd1e0996740 1 Processor -- start 2024-05-27T07:48:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.443+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf4fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.443+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 0x557a4bf50160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.443+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf5bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.443+0000 7fd1e0996740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557a4bb8c790 con 0x557a4bba31b0 2024-05-27T07:48:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.443+0000 7fd1e0996740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557a4be4ebc0 con 0x557a4bb7a2f0 2024-05-27T07:48:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.443+0000 7fd1e0996740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557a4bf52bc0 con 0x557a4bb824d0 2024-05-27T07:48:36.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.443+0000 7fd1e0996740 10 client.0 _ll_register_callbacks cb 0x557a4be2c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.444+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf4fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.444+0000 7fd1d2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 0x557a4bf50160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.444+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf4fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:51012/0 (socket says 172.21.15.90:51012) 2024-05-27T07:48:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.444+0000 7fd1d2a00640 1 -- 172.21.15.90:0/3630422840 learned_addr learned my addr 172.21.15.90:0/3630422840 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.444+0000 7fd1d3400640 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf5bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.445+0000 7fd1d2000640 1 -- 172.21.15.90:0/3630422840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 msgr2=0x557a4bf4fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:36.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.445+0000 7fd1d2000640 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf4fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.445+0000 7fd1d2000640 1 -- 172.21.15.90:0/3630422840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 msgr2=0x557a4bf5bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.445+0000 7fd1d2000640 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf5bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.445+0000 7fd1d2000640 1 -- 172.21.15.90:0/3630422840 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4bbbb210 con 0x557a4bb824d0 2024-05-27T07:48:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.445+0000 7fd1d3400640 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf5bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.445+0000 7fd1d2000640 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 0x557a4bf50160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd1c01001d0 tx=0x7fd1c0009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3dd73ac503881a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:36.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.445+0000 7fd1bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:48:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.446+0000 7fd1bcc00640 1 -- 172.21.15.90:0/3630422840 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0049040 con 0x557a4bb824d0 2024-05-27T07:48:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.446+0000 7fd1e0996740 1 -- 172.21.15.90:0/3630422840 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d407fc60 con 0x557a4bb824d0 2024-05-27T07:48:36.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.446+0000 7fd1bcc00640 1 -- 172.21.15.90:0/3630422840 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd1c00c9fc0 con 0x557a4bb824d0 2024-05-27T07:48:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.447+0000 7fd1bcc00640 1 -- 172.21.15.90:0/3630422840 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd1c00e9050 con 0x557a4bb824d0 2024-05-27T07:48:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.447+0000 7fd1e0996740 10 client.25546 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.447+0000 7fd1e0996740 1 -- 172.21.15.90:0/3630422840 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557a4bc87620 con 0x557a4bb824d0 2024-05-27T07:48:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.448+0000 7fd1bcc00640 1 -- 172.21.15.90:0/3630422840 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1d407fc60 con 0x557a4bb824d0 2024-05-27T07:48:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.448+0000 7fd1bcc00640 1 -- 172.21.15.90:0/3630422840 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd1c00d2730 con 0x557a4bb824d0 2024-05-27T07:48:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.449+0000 7fd1e0996740 10 client.25546 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.449+0000 7fd1e0996740 10 client.25546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.449+0000 7fd1e0996740 20 client.25546 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.449+0000 7fd1e0996740 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a4bb61d80 0x557a4bf61f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.449+0000 7fd1e0996740 4 client.25546 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.449+0000 7fd1e0996740 1 -- 172.21.15.90:0/3630422840 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a4beb8ae0 con 0x557a4bb61d80 2024-05-27T07:48:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.450+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a4bb61d80 0x557a4bf61f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.451+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a4bb61d80 0x557a4bf61f10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc9b130e7f97500e server_cookie=c0c7f1938a6da55f in_seq=0 out_seq=0 2024-05-27T07:48:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.451+0000 7fd1bcc00640 10 client.25546 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1bcc00640 1 -- 172.21.15.90:0/3630422840 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557a4beb8ae0 con 0x557a4bb61d80 2024-05-27T07:48:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1bcc00640 10 client.25546 handle_command_reply: tid=0 2024-05-27T07:48:36.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1e0996740 1 client.25546 shutdown 2024-05-27T07:48:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1e0996740 1 -- 172.21.15.90:0/3630422840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 msgr2=0x557a4bf50160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1e0996740 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 0x557a4bf50160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd1c01001d0 tx=0x7fd1c0009620 comp rx=0 tx=0).stop 2024-05-27T07:48:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1e0996740 1 -- 172.21.15.90:0/3630422840 shutdown_connections 2024-05-27T07:48:36.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1e0996740 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a4bb61d80 0x557a4bf61f10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1e0996740 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf5bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1e0996740 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 0x557a4bf50160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1bcc00640 0 client.25546 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1e0996740 1 --2- 172.21.15.90:0/3630422840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf4fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.453+0000 7fd1e0996740 1 -- 172.21.15.90:0/3630422840 >> 172.21.15.90:0/3630422840 conn(0x557a4bb97300 msgr2=0x557a4bf477b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.454+0000 7fd1e0996740 1 -- 172.21.15.90:0/3630422840 shutdown_connections 2024-05-27T07:48:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.454+0000 7fd1e0996740 1 -- 172.21.15.90:0/3630422840 wait complete. 2024-05-27T07:48:36.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.454+0000 7fd1e0996740 20 client.25546 trim_cache size 0 max 16384 2024-05-27T07:48:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.474+0000 7fd1e0996740 1 Processor -- start 2024-05-27T07:48:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.474+0000 7fd1e0996740 1 -- start start 2024-05-27T07:48:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.474+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb7a2f0 0x557a4bf52350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.474+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb824d0 0x557a4bf45a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.474+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 0x557a4bf49810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.474+0000 7fd1e0996740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc14a470 con 0x557a4bb7a2f0 2024-05-27T07:48:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.474+0000 7fd1e0996740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc14bfb0 con 0x557a4bb824d0 2024-05-27T07:48:36.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.474+0000 7fd1e0996740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc14ab40 con 0x557a4bba31b0 2024-05-27T07:48:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb7a2f0 0x557a4bf52350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb7a2f0 0x557a4bf52350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56664/0 (socket says 172.21.15.90:56664) 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d2a00640 1 -- 172.21.15.90:0/3029560459 learned_addr learned my addr 172.21.15.90:0/3029560459 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d2000640 1 --2- 172.21.15.90:0/3029560459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb824d0 0x557a4bf45a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d3400640 1 --2- 172.21.15.90:0/3029560459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 0x557a4bf49810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d2a00640 1 -- 172.21.15.90:0/3029560459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 msgr2=0x557a4bf49810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/3029560459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 0x557a4bf49810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d2a00640 1 -- 172.21.15.90:0/3029560459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb824d0 msgr2=0x557a4bf45a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/3029560459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb824d0 0x557a4bf45a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d2a00640 1 -- 172.21.15.90:0/3029560459 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4bf52aa0 con 0x557a4bb7a2f0 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.475+0000 7fd1d2000640 1 --2- 172.21.15.90:0/3029560459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb824d0 0x557a4bf45a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.476+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/3029560459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb7a2f0 0x557a4bf52350 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400a7b0 tx=0x7fd1c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a76363e83f93e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.476+0000 7fd1bf400640 1 -- 172.21.15.90:0/3029560459 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4029450 con 0x557a4bb7a2f0 2024-05-27T07:48:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.476+0000 7fd1bf400640 1 -- 172.21.15.90:0/3029560459 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1c4029a80 con 0x557a4bb7a2f0 2024-05-27T07:48:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.476+0000 7fd1bf400640 1 -- 172.21.15.90:0/3029560459 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4027d20 con 0x557a4bb7a2f0 2024-05-27T07:48:36.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.477+0000 7fd1e0996740 1 -- 172.21.15.90:0/3029560459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb7a2f0 msgr2=0x557a4bf52350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.477+0000 7fd1e0996740 1 --2- 172.21.15.90:0/3029560459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb7a2f0 0x557a4bf52350 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400a7b0 tx=0x7fd1c4019000 comp rx=0 tx=0).stop 2024-05-27T07:48:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.477+0000 7fd1e0996740 1 -- 172.21.15.90:0/3029560459 shutdown_connections 2024-05-27T07:48:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.477+0000 7fd1e0996740 1 --2- 172.21.15.90:0/3029560459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb824d0 0x557a4bf45a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.477+0000 7fd1e0996740 1 --2- 172.21.15.90:0/3029560459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bba31b0 0x557a4bf49810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.477+0000 7fd1e0996740 1 --2- 172.21.15.90:0/3029560459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bb7a2f0 0x557a4bf52350 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.477+0000 7fd1e0996740 1 -- 172.21.15.90:0/3029560459 >> 172.21.15.90:0/3029560459 conn(0x557a4bb97300 msgr2=0x557a4bf4fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.477+0000 7fd1e0996740 1 -- 172.21.15.90:0/3029560459 shutdown_connections 2024-05-27T07:48:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.477+0000 7fd1e0996740 1 -- 172.21.15.90:0/3029560459 wait complete. 2024-05-27T07:48:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.478+0000 7fd1e0996740 1 -- start start 2024-05-27T07:48:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.478+0000 7fd1e0996740 1 Processor -- start 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 0x557a4bf4a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1e0996740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf4a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1e0996740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc14b8e0 con 0x557a4bba31b0 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1e0996740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc14b210 con 0x557a4bb7a2f0 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1e0996740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc148430 con 0x557a4bb824d0 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1e0996740 10 client.0 _ll_register_callbacks cb 0x7fd1c0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1d3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf4a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1d3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf4a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56672/0 (socket says 172.21.15.90:56672) 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1d3400640 1 -- 172.21.15.90:0/1427569579 learned_addr learned my addr 172.21.15.90:0/1427569579 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.479+0000 7fd1d2000640 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 0x557a4bf4a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.480+0000 7fd1d3400640 1 -- 172.21.15.90:0/1427569579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 msgr2=0x557a4bf4a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.480+0000 7fd1d3400640 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 0x557a4bf4a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.480+0000 7fd1d3400640 1 -- 172.21.15.90:0/1427569579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 msgr2=0x557a4bf584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.480+0000 7fd1d3400640 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.480+0000 7fd1d3400640 1 -- 172.21.15.90:0/1427569579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a4bf4ae20 con 0x557a4bba31b0 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.480+0000 7fd1d3400640 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf4a830 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4000b60 tx=0x7fd1d407fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a33be786d6f11a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.480+0000 7fd1bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.480+0000 7fd1bcc00640 1 -- 172.21.15.90:0/1427569579 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4096460 con 0x557a4bba31b0 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.480+0000 7fd1e0996740 1 -- 172.21.15.90:0/1427569579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c4029a80 con 0x557a4bba31b0 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.480+0000 7fd1bcc00640 1 -- 172.21.15.90:0/1427569579 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fd1d4096600 con 0x557a4bba31b0 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1bcc00640 1 -- 172.21.15.90:0/1427569579 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd1d409f050 con 0x557a4bba31b0 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1e0996740 10 client.15900 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:36.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1e0996740 1 -- 172.21.15.90:0/1427569579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557a4bf56cd0 con 0x557a4bba31b0 2024-05-27T07:48:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1bcc00640 1 -- 172.21.15.90:0/1427569579 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1c4029a80 con 0x557a4bba31b0 2024-05-27T07:48:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1bcc00640 1 -- 172.21.15.90:0/1427569579 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd1d40894e0 con 0x557a4bba31b0 2024-05-27T07:48:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1e0996740 10 client.15900 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1e0996740 10 client.15900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1e0996740 20 client.15900 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1e0996740 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a4bb371c0 0x557a4bfcc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1e0996740 4 client.15900 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:48:36.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.481+0000 7fd1e0996740 1 -- 172.21.15.90:0/1427569579 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a4bf48630 con 0x557a4bb371c0 2024-05-27T07:48:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.482+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a4bb371c0 0x557a4bfcc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.483+0000 7fd1d2a00640 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a4bb371c0 0x557a4bfcc8f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b895fd599c02d2e6 server_cookie=b57c57a196cbc088 in_seq=0 out_seq=0 2024-05-27T07:48:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.483+0000 7fd1bcc00640 10 client.15900 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1bcc00640 1 -- 172.21.15.90:0/1427569579 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557a4bf48630 con 0x557a4bb371c0 2024-05-27T07:48:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1bcc00640 10 client.15900 handle_command_reply: tid=0 2024-05-27T07:48:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1e0996740 1 client.15900 shutdown 2024-05-27T07:48:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1e0996740 1 -- 172.21.15.90:0/1427569579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 msgr2=0x557a4bf4a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1e0996740 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf4a830 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4000b60 tx=0x7fd1d407fb70 comp rx=0 tx=0).stop 2024-05-27T07:48:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1e0996740 1 -- 172.21.15.90:0/1427569579 shutdown_connections 2024-05-27T07:48:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1e0996740 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a4bb371c0 0x557a4bfcc8f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1e0996740 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a4bba31b0 0x557a4bf4a830 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1e0996740 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a4bb824d0 0x557a4bf4a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1e0996740 1 --2- 172.21.15.90:0/1427569579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a4bb7a2f0 0x557a4bf584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1bcc00640 0 client.15900 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1e0996740 1 -- 172.21.15.90:0/1427569579 >> 172.21.15.90:0/1427569579 conn(0x557a4bb97300 msgr2=0x557a4bf459a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.529+0000 7fd1e0996740 1 -- 172.21.15.90:0/1427569579 shutdown_connections 2024-05-27T07:48:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.530+0000 7fd1e0996740 1 -- 172.21.15.90:0/1427569579 wait complete. 2024-05-27T07:48:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.530+0000 7fd1e0996740 20 client.15900 trim_cache size 0 max 16384 2024-05-27T07:48:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:48:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:48:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:48:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 -- 172.21.15.90:0/1123283236 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1b80b2960 msgr2=0x7fd1b80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1b80b2960 0x7fd1b80b4e70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x557a4bf46e40 tx=0x7fd1c400a2d0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 -- 172.21.15.90:0/1123283236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1cc09ca20 msgr2=0x7fd1cc12c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1cc09ca20 0x7fd1cc12c480 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0005940 tx=0x7fd1c00040e0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 -- 172.21.15.90:0/1123283236 shutdown_connections 2024-05-27T07:48:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1b80b2960 0x7fd1b80b4e70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1cc0a1380 0x7fd1cc12c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1cc09ca20 0x7fd1cc12c480 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 --2- 172.21.15.90:0/1123283236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1cc096af0 0x7fd1cc12bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 -- 172.21.15.90:0/1123283236 >> 172.21.15.90:0/1123283236 conn(0x7fd1cc00d3c0 msgr2=0x7fd1cc007240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 -- 172.21.15.90:0/1123283236 shutdown_connections 2024-05-27T07:48:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:36.531+0000 7fd1d3e00640 1 -- 172.21.15.90:0/1123283236 wait complete. 2024-05-27T07:48:38.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:38 smithi090 ceph-mon[31429]: pgmap v71: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 127 op/s 2024-05-27T07:48:38.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:38 smithi129 ceph-mon[35254]: pgmap v71: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 127 op/s 2024-05-27T07:48:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.392+0000 7fe152a00700 1 -- 172.21.15.90:0/4174324520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe14c109670 msgr2=0x7fe14c110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.392+0000 7fe152a00700 1 --2- 172.21.15.90:0/4174324520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe14c109670 0x7fe14c110fd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe14400b0d0 tx=0x7fe14400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:48:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.393+0000 7fe152a00700 1 -- 172.21.15.90:0/4174324520 shutdown_connections 2024-05-27T07:48:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.393+0000 7fe152a00700 1 --2- 172.21.15.90:0/4174324520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe14c109670 0x7fe14c110fd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.393+0000 7fe152a00700 1 --2- 172.21.15.90:0/4174324520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe14c1043f0 0x7fe14c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:38.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.393+0000 7fe152a00700 1 --2- 172.21.15.90:0/4174324520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe14c108d20 0x7fe14c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:38.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.393+0000 7fe152a00700 1 -- 172.21.15.90:0/4174324520 >> 172.21.15.90:0/4174324520 conn(0x7fe14c079d00 msgr2=0x7fe14c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:38.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.393+0000 7fe152a00700 1 -- 172.21.15.90:0/4174324520 shutdown_connections 2024-05-27T07:48:38.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.393+0000 7fe152a00700 1 -- 172.21.15.90:0/4174324520 wait complete. 2024-05-27T07:48:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.394+0000 7fe152a00700 1 Processor -- start 2024-05-27T07:48:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.394+0000 7fe152a00700 1 -- start start 2024-05-27T07:48:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.395+0000 7fe152a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe14c1043f0 0x7fe14c1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.395+0000 7fe152a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe14c108d20 0x7fe14c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.395+0000 7fe152a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe14c109670 0x7fe14c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.395+0000 7fe152a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1141a0 con 0x7fe14c109670 2024-05-27T07:48:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.395+0000 7fe152a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c114020 con 0x7fe14c108d20 2024-05-27T07:48:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.395+0000 7fe152a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c114320 con 0x7fe14c1043f0 2024-05-27T07:48:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.395+0000 7fe152000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe14c109670 0x7fe14c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:38.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.395+0000 7fe152000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe14c109670 0x7fe14c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56678/0 (socket says 172.21.15.90:56678) 2024-05-27T07:48:38.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.396+0000 7fe152000700 1 -- 172.21.15.90:0/3418772858 learned_addr learned my addr 172.21.15.90:0/3418772858 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:38.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.396+0000 7fe151600700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe14c1043f0 0x7fe14c1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.396+0000 7fe150c00700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe14c108d20 0x7fe14c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.396+0000 7fe152000700 1 -- 172.21.15.90:0/3418772858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe14c1043f0 msgr2=0x7fe14c1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.396+0000 7fe152000700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe14c1043f0 0x7fe14c1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.396+0000 7fe152000700 1 -- 172.21.15.90:0/3418772858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe14c108d20 msgr2=0x7fe14c1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.396+0000 7fe152000700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe14c108d20 0x7fe14c1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.396+0000 7fe152000700 1 -- 172.21.15.90:0/3418772858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c19b0d0 con 0x7fe14c109670 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.397+0000 7fe152000700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe14c109670 0x7fe14c19aa20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe144007d10 tx=0x7fe144007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.397+0000 7fe14a000700 1 -- 172.21.15.90:0/3418772858 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14401f070 con 0x7fe14c109670 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.397+0000 7fe14a000700 1 -- 172.21.15.90:0/3418772858 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe144003e10 con 0x7fe14c109670 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.397+0000 7fe152a00700 1 -- 172.21.15.90:0/3418772858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c19b360 con 0x7fe14c109670 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.397+0000 7fe152a00700 1 -- 172.21.15.90:0/3418772858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c19b840 con 0x7fe14c109670 2024-05-27T07:48:38.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.397+0000 7fe14a000700 1 -- 172.21.15.90:0/3418772858 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14401b740 con 0x7fe14c109670 2024-05-27T07:48:38.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.399+0000 7fe152a00700 1 -- 172.21.15.90:0/3418772858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12c005330 con 0x7fe14c109670 2024-05-27T07:48:38.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.402+0000 7fe14a000700 1 -- 172.21.15.90:0/3418772858 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe144022070 con 0x7fe14c109670 2024-05-27T07:48:38.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.405+0000 7fe14a000700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1340b29f0 0x7fe1340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:38.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.405+0000 7fe14a000700 1 -- 172.21.15.90:0/3418772858 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe14400f020 con 0x7fe14c109670 2024-05-27T07:48:38.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.405+0000 7fe151600700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1340b29f0 0x7fe1340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:38.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.406+0000 7fe151600700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1340b29f0 0x7fe1340b4f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe14c19bb60 tx=0x7fe14000b410 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:38.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.409+0000 7fe14a000700 1 -- 172.21.15.90:0/3418772858 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe144029020 con 0x7fe14c109670 2024-05-27T07:48:38.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:38 smithi136 ceph-mon[35324]: pgmap v71: 129 pgs: 129 active+clean; 437 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 16 MiB/s wr, 127 op/s 2024-05-27T07:48:38.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.728+0000 7fe152a00700 1 -- 172.21.15.90:0/3418772858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe12c005620 con 0x7fe14c109670 2024-05-27T07:48:38.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.730+0000 7fe14a000700 1 -- 172.21.15.90:0/3418772858 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11938 (secure 0 0 0) 0x7fe1440ab8d0 con 0x7fe14c109670 2024-05-27T07:48:38.730 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:48:38.730 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:48:28.203182+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25504":{"gid":25504,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/508081175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":508081175},{"type":"v1","addr":"172.21.15.136:6837","nonce":508081175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T07:48:38.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.733+0000 7fe152a00700 1 -- 172.21.15.90:0/3418772858 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1340b29f0 msgr2=0x7fe1340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:38.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.733+0000 7fe152a00700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1340b29f0 0x7fe1340b4f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe14c19bb60 tx=0x7fe14000b410 comp rx=0 tx=0).stop 2024-05-27T07:48:38.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.733+0000 7fe152a00700 1 -- 172.21.15.90:0/3418772858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe14c109670 msgr2=0x7fe14c19aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:38.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.733+0000 7fe152a00700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe14c109670 0x7fe14c19aa20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe144007d10 tx=0x7fe144007d40 comp rx=0 tx=0).stop 2024-05-27T07:48:38.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.733+0000 7fe152a00700 1 -- 172.21.15.90:0/3418772858 shutdown_connections 2024-05-27T07:48:38.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.733+0000 7fe152a00700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe14c109670 0x7fe14c19aa20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:38.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.733+0000 7fe152a00700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe14c1043f0 0x7fe14c1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:38.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.733+0000 7fe152a00700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1340b29f0 0x7fe1340b4f00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:38.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.733+0000 7fe152a00700 1 --2- 172.21.15.90:0/3418772858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe14c108d20 0x7fe14c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:38.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.733+0000 7fe152a00700 1 -- 172.21.15.90:0/3418772858 >> 172.21.15.90:0/3418772858 conn(0x7fe14c079d00 msgr2=0x7fe14c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:38.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.734+0000 7fe152a00700 1 -- 172.21.15.90:0/3418772858 shutdown_connections 2024-05-27T07:48:38.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:38.734+0000 7fe152a00700 1 -- 172.21.15.90:0/3418772858 wait complete. 2024-05-27T07:48:38.735 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 36 2024-05-27T07:48:39.009 INFO:teuthology.orchestra.run.smithi090.stderr:1800000 of 3200000 tuples (56%) done (elapsed 55.06 s, remaining 42.83 s) 2024-05-27T07:48:39.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:39 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3418772858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:48:39.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:39 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3418772858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:48:39.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:39 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3418772858' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:48:39.703 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:48:40.173 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:40.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:40 smithi129 ceph-mon[35254]: pgmap v72: 129 pgs: 129 active+clean; 473 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 14 MiB/s wr, 119 op/s 2024-05-27T07:48:40.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:40 smithi090 ceph-mon[31429]: pgmap v72: 129 pgs: 129 active+clean; 473 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 14 MiB/s wr, 119 op/s 2024-05-27T07:48:40.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:40 smithi136 ceph-mon[35324]: pgmap v72: 129 pgs: 129 active+clean; 473 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 14 MiB/s wr, 119 op/s 2024-05-27T07:48:42.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:42 smithi090 ceph-mon[31429]: pgmap v73: 129 pgs: 129 active+clean; 473 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 107 op/s 2024-05-27T07:48:42.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:42 smithi129 ceph-mon[35254]: pgmap v73: 129 pgs: 129 active+clean; 473 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 107 op/s 2024-05-27T07:48:42.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:42 smithi136 ceph-mon[35324]: pgmap v73: 129 pgs: 129 active+clean; 473 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 13 MiB/s wr, 107 op/s 2024-05-27T07:48:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.779+0000 7f4f48e00700 1 -- 172.21.15.90:0/3931325308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f4410e900 msgr2=0x7f4f44110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.779+0000 7f4f48e00700 1 --2- 172.21.15.90:0/3931325308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f4410e900 0x7f4f44110dc0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00b0a0 tx=0x7f4f3c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.779+0000 7f4f48e00700 1 -- 172.21.15.90:0/3931325308 shutdown_connections 2024-05-27T07:48:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.779+0000 7f4f48e00700 1 --2- 172.21.15.90:0/3931325308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f4410e900 0x7f4f44110dc0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.779+0000 7f4f48e00700 1 --2- 172.21.15.90:0/3931325308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f44105490 0x7f4f441058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.779+0000 7f4f48e00700 1 --2- 172.21.15.90:0/3931325308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f44105de0 0x7f4f4410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.779+0000 7f4f48e00700 1 -- 172.21.15.90:0/3931325308 >> 172.21.15.90:0/3931325308 conn(0x7f4f440fedb0 msgr2=0x7f4f441011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.779+0000 7f4f48e00700 1 -- 172.21.15.90:0/3931325308 shutdown_connections 2024-05-27T07:48:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.779+0000 7f4f48e00700 1 -- 172.21.15.90:0/3931325308 wait complete. 2024-05-27T07:48:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.781+0000 7f4f48e00700 1 Processor -- start 2024-05-27T07:48:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.781+0000 7f4f48e00700 1 -- start start 2024-05-27T07:48:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.782+0000 7f4f48e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f44105490 0x7f4f44103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.782+0000 7f4f48e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f44105de0 0x7f4f441036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.782+0000 7f4f48e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f4410e900 0x7f4f44103c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.782+0000 7f4f48e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44114380 con 0x7f4f44105de0 2024-05-27T07:48:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.782+0000 7f4f48e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44114200 con 0x7f4f44105490 2024-05-27T07:48:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.782+0000 7f4f48e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44114500 con 0x7f4f4410e900 2024-05-27T07:48:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.782+0000 7f4f41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f44105de0 0x7f4f441036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.782+0000 7f4f41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f44105de0 0x7f4f441036c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37612/0 (socket says 172.21.15.90:37612) 2024-05-27T07:48:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.782+0000 7f4f41400700 1 -- 172.21.15.90:0/4160887811 learned_addr learned my addr 172.21.15.90:0/4160887811 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.782+0000 7f4f41e00700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f44105490 0x7f4f44103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.783+0000 7f4f42800700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f4410e900 0x7f4f44103c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.783+0000 7f4f41400700 1 -- 172.21.15.90:0/4160887811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f4410e900 msgr2=0x7f4f44103c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.783+0000 7f4f41400700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f4410e900 0x7f4f44103c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.783+0000 7f4f41400700 1 -- 172.21.15.90:0/4160887811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f44105490 msgr2=0x7f4f44103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.783+0000 7f4f41400700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f44105490 0x7f4f44103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.783+0000 7f4f41400700 1 -- 172.21.15.90:0/4160887811 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f441a9110 con 0x7f4f44105de0 2024-05-27T07:48:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.783+0000 7f4f41400700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f44105de0 0x7f4f441036c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d980 tx=0x7f4f3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.784+0000 7f4f3aa00700 1 -- 172.21.15.90:0/4160887811 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30017070 con 0x7f4f44105de0 2024-05-27T07:48:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.784+0000 7f4f3aa00700 1 -- 172.21.15.90:0/4160887811 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f3000fd20 con 0x7f4f44105de0 2024-05-27T07:48:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.784+0000 7f4f3aa00700 1 -- 172.21.15.90:0/4160887811 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000e3b0 con 0x7f4f44105de0 2024-05-27T07:48:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.784+0000 7f4f48e00700 1 -- 172.21.15.90:0/4160887811 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f441a93a0 con 0x7f4f44105de0 2024-05-27T07:48:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.784+0000 7f4f48e00700 1 -- 172.21.15.90:0/4160887811 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f441a9890 con 0x7f4f44105de0 2024-05-27T07:48:42.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.789+0000 7f4f3aa00700 1 -- 172.21.15.90:0/4160887811 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4f3000f890 con 0x7f4f44105de0 2024-05-27T07:48:42.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.789+0000 7f4f578dcb80 1 Processor -- start 2024-05-27T07:48:42.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.789+0000 7f4f578dcb80 1 -- start start 2024-05-27T07:48:42.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.790+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e80ee40 0x563c2e9937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.790+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e993d30 0x563c2e98d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.790+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e98ddb0 0x563c2e98e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.790+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563c2e8bbe40 con 0x563c2e80ee40 2024-05-27T07:48:42.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.790+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563c2e8ab9e0 con 0x563c2e993d30 2024-05-27T07:48:42.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.790+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563c2e8afa20 con 0x563c2e98ddb0 2024-05-27T07:48:42.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.790+0000 7f4f42800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e80ee40 0x563c2e9937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.791+0000 7f4f42800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e80ee40 0x563c2e9937f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37624/0 (socket says 172.21.15.90:37624) 2024-05-27T07:48:42.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.791+0000 7f4f42800700 1 -- 172.21.15.90:0/1069158216 learned_addr learned my addr 172.21.15.90:0/1069158216 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:42.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.791+0000 7f4f41400700 1 --2- 172.21.15.90:0/1069158216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e98ddb0 0x563c2e98e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.791+0000 7f4f41e00700 1 --2- 172.21.15.90:0/1069158216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e993d30 0x563c2e98d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.791+0000 7f4f42800700 1 -- 172.21.15.90:0/1069158216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e98ddb0 msgr2=0x563c2e98e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.791+0000 7f4f42800700 1 --2- 172.21.15.90:0/1069158216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e98ddb0 0x563c2e98e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.791+0000 7f4f42800700 1 -- 172.21.15.90:0/1069158216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e993d30 msgr2=0x563c2e98d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.791+0000 7f4f42800700 1 --2- 172.21.15.90:0/1069158216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e993d30 0x563c2e98d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.791+0000 7f4f42800700 1 -- 172.21.15.90:0/1069158216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2e874420 con 0x563c2e80ee40 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.791+0000 7f4f3aa00700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f280b2a70 0x7f4f280b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.792+0000 7f4f42800700 1 --2- 172.21.15.90:0/1069158216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e80ee40 0x563c2e9937f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c015000 tx=0x7f4f3c0090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3fc00f47b93a87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.792+0000 7f4f3aa00700 1 -- 172.21.15.90:0/4160887811 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4f300e2340 con 0x7f4f44105de0 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.792+0000 7f4f2f400700 1 -- 172.21.15.90:0/1069158216 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c020070 con 0x563c2e80ee40 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.792+0000 7f4f2f400700 1 -- 172.21.15.90:0/1069158216 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f3c0075c0 con 0x563c2e80ee40 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.792+0000 7f4f2f400700 1 -- 172.21.15.90:0/1069158216 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0052a0 con 0x563c2e80ee40 2024-05-27T07:48:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.792+0000 7f4f41e00700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f280b2a70 0x7f4f280b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.793+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1069158216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e80ee40 msgr2=0x563c2e9937f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:42.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.793+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/1069158216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e80ee40 0x563c2e9937f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c015000 tx=0x7f4f3c0090d0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.793+0000 7f4f41e00700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f280b2a70 0x7f4f280b4f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400b700 tx=0x7f4f3400b3c0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:42.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.794+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1069158216 shutdown_connections 2024-05-27T07:48:42.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.794+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/1069158216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e98ddb0 0x563c2e98e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.794+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/1069158216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e80ee40 0x563c2e9937f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.794+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/1069158216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e993d30 0x563c2e98d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.794+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1069158216 >> 172.21.15.90:0/1069158216 conn(0x563c2e86efc0 msgr2=0x563c2e9888d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:42.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.794+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1069158216 shutdown_connections 2024-05-27T07:48:42.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.794+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1069158216 wait complete. 2024-05-27T07:48:42.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.794+0000 7f4f578dcb80 1 -- start start 2024-05-27T07:48:42.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.795+0000 7f4f578dcb80 1 Processor -- start 2024-05-27T07:48:42.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.795+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e9975c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.795+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e98ddb0 0x563c2e997b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.795+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e993d30 0x563c2e998040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.795+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563c2e7b1fa0 con 0x563c2e993d30 2024-05-27T07:48:42.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.795+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563c2e84c760 con 0x563c2e80ee40 2024-05-27T07:48:42.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.795+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563c2e99a7a0 con 0x563c2e98ddb0 2024-05-27T07:48:42.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.795+0000 7f4f578dcb80 10 client.0 _ll_register_callbacks cb 0x563c2e855c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:42.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f42800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e993d30 0x563c2e998040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f42800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e993d30 0x563c2e998040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37636/0 (socket says 172.21.15.90:37636) 2024-05-27T07:48:42.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f42800700 1 -- 172.21.15.90:0/1025850859 learned_addr learned my addr 172.21.15.90:0/1025850859 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:42.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f41e00700 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e9975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f41400700 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e98ddb0 0x563c2e997b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f42800700 1 -- 172.21.15.90:0/1025850859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e98ddb0 msgr2=0x563c2e997b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f42800700 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e98ddb0 0x563c2e997b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f42800700 1 -- 172.21.15.90:0/1025850859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 msgr2=0x563c2e9975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f42800700 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e9975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f42800700 1 -- 172.21.15.90:0/1025850859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2e7fb8d0 con 0x563c2e993d30 2024-05-27T07:48:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.796+0000 7f4f41e00700 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e9975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.797+0000 7f4f42800700 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e993d30 0x563c2e998040 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0062a0 tx=0x7f4f3c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f440b450b6fd5f18 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.797+0000 7f4f23e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.797+0000 7f4f23e00700 1 -- 172.21.15.90:0/1025850859 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c020040 con 0x563c2e993d30 2024-05-27T07:48:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.797+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1025850859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f3c0075c0 con 0x563c2e993d30 2024-05-27T07:48:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.797+0000 7f4f23e00700 1 -- 172.21.15.90:0/1025850859 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4f3c005040 con 0x563c2e993d30 2024-05-27T07:48:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.798+0000 7f4f23e00700 1 -- 172.21.15.90:0/1025850859 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4f3c005240 con 0x563c2e993d30 2024-05-27T07:48:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.798+0000 7f4f578dcb80 10 client.15918 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:42.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.798+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1025850859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563c2e87c060 con 0x563c2e993d30 2024-05-27T07:48:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.798+0000 7f4f23e00700 1 -- 172.21.15.90:0/1025850859 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f3c0075c0 con 0x563c2e993d30 2024-05-27T07:48:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.798+0000 7f4f23e00700 1 -- 172.21.15.90:0/1025850859 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4f3c011070 con 0x563c2e993d30 2024-05-27T07:48:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.798+0000 7f4f578dcb80 10 client.15918 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.798+0000 7f4f578dcb80 10 client.15918 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:48:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.799+0000 7f4f578dcb80 20 client.15918 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.799+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x563c2e9a75f0 0x563c2e9a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.799+0000 7f4f578dcb80 4 client.15918 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.799+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1025850859 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c2e8bdd90 con 0x563c2e9a75f0 2024-05-27T07:48:42.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.799+0000 7f4f41e00700 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x563c2e9a75f0 0x563c2e9a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.800+0000 7f4f41e00700 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x563c2e9a75f0 0x563c2e9a9a10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5edbba4adbd9a294 server_cookie=64929dc5991f5aff in_seq=0 out_seq=0 2024-05-27T07:48:42.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.800+0000 7f4f23e00700 10 client.15918 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:48:42.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.802+0000 7f4f23e00700 1 -- 172.21.15.90:0/1025850859 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563c2e8bdd90 con 0x563c2e9a75f0 2024-05-27T07:48:42.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.802+0000 7f4f23e00700 10 client.15918 handle_command_reply: tid=0 2024-05-27T07:48:42.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 client.15918 shutdown 2024-05-27T07:48:42.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1025850859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e993d30 msgr2=0x563c2e998040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:42.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e993d30 0x563c2e998040 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0062a0 tx=0x7f4f3c0097a0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1025850859 shutdown_connections 2024-05-27T07:48:42.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x563c2e9a75f0 0x563c2e9a9a10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e993d30 0x563c2e998040 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e9975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/1025850859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e98ddb0 0x563c2e997b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1025850859 >> 172.21.15.90:0/1025850859 conn(0x563c2e86efc0 msgr2=0x563c2e988580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:42.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f23e00700 0 client.15918 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:48:42.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1025850859 shutdown_connections 2024-05-27T07:48:42.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.803+0000 7f4f578dcb80 1 -- 172.21.15.90:0/1025850859 wait complete. 2024-05-27T07:48:42.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.804+0000 7f4f578dcb80 20 client.15918 trim_cache size 0 max 16384 2024-05-27T07:48:42.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.823+0000 7f4f578dcb80 1 Processor -- start 2024-05-27T07:48:42.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.824+0000 7f4f578dcb80 1 -- start start 2024-05-27T07:48:42.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.824+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e98d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.824+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e992d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.824+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e993d30 0x563c2e999a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.824+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441b25e0 con 0x563c2e98ddb0 2024-05-27T07:48:42.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.824+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441b4120 con 0x563c2e80ee40 2024-05-27T07:48:42.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.824+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441b2cb0 con 0x563c2e993d30 2024-05-27T07:48:42.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.824+0000 7f4f41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e992d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.825+0000 7f4f41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e992d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37640/0 (socket says 172.21.15.90:37640) 2024-05-27T07:48:42.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.825+0000 7f4f41400700 1 -- 172.21.15.90:0/2277878849 learned_addr learned my addr 172.21.15.90:0/2277878849 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:42.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.825+0000 7f4f41e00700 1 --2- 172.21.15.90:0/2277878849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e98d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.825+0000 7f4f42800700 1 --2- 172.21.15.90:0/2277878849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e993d30 0x563c2e999a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.825+0000 7f4f41400700 1 -- 172.21.15.90:0/2277878849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e993d30 msgr2=0x563c2e999a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:42.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.825+0000 7f4f41400700 1 --2- 172.21.15.90:0/2277878849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e993d30 0x563c2e999a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.825+0000 7f4f41400700 1 -- 172.21.15.90:0/2277878849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 msgr2=0x563c2e98d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:42.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.825+0000 7f4f41400700 1 --2- 172.21.15.90:0/2277878849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e98d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.825+0000 7f4f41400700 1 -- 172.21.15.90:0/2277878849 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2e984d20 con 0x563c2e98ddb0 2024-05-27T07:48:42.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.825+0000 7f4f41400700 1 --2- 172.21.15.90:0/2277878849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e992d80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4f300a8f70 tx=0x7f4f300a8bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f73880e1606dcbc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:42.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.826+0000 7f4f2ea00700 1 -- 172.21.15.90:0/2277878849 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30020040 con 0x563c2e98ddb0 2024-05-27T07:48:42.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.826+0000 7f4f2ea00700 1 -- 172.21.15.90:0/2277878849 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f300e2170 con 0x563c2e98ddb0 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.826+0000 7f4f2ea00700 1 -- 172.21.15.90:0/2277878849 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f300a9730 con 0x563c2e98ddb0 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.827+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2277878849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 msgr2=0x563c2e992d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.827+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/2277878849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e992d80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4f300a8f70 tx=0x7f4f300a8bc0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.827+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2277878849 shutdown_connections 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.827+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/2277878849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e993d30 0x563c2e999a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.827+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/2277878849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e98d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.827+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/2277878849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e992d80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.827+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2277878849 >> 172.21.15.90:0/2277878849 conn(0x563c2e86efc0 msgr2=0x563c2e997580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.827+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2277878849 shutdown_connections 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.827+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2277878849 wait complete. 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.828+0000 7f4f578dcb80 1 -- start start 2024-05-27T07:48:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.828+0000 7f4f578dcb80 1 Processor -- start 2024-05-27T07:48:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e9acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e9a9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f578dcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e993d30 0x563c2e9a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441b3a50 con 0x563c2e98ddb0 2024-05-27T07:48:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441b3380 con 0x563c2e80ee40 2024-05-27T07:48:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f578dcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441b0560 con 0x563c2e993d30 2024-05-27T07:48:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f578dcb80 10 client.0 _ll_register_callbacks cb 0x563c2e855c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e9a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e9a9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37652/0 (socket says 172.21.15.90:37652) 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f41400700 1 -- 172.21.15.90:0/2761270155 learned_addr learned my addr 172.21.15.90:0/2761270155 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.829+0000 7f4f41e00700 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e9acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.830+0000 7f4f41400700 1 -- 172.21.15.90:0/2761270155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e993d30 msgr2=0x563c2e9a9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.830+0000 7f4f41400700 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e993d30 0x563c2e9a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.830+0000 7f4f41400700 1 -- 172.21.15.90:0/2761270155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 msgr2=0x563c2e9acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.830+0000 7f4f41400700 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e9acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.830+0000 7f4f41400700 1 -- 172.21.15.90:0/2761270155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2e9aa340 con 0x563c2e98ddb0 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.830+0000 7f4f41400700 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e9a9600 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d950 tx=0x7f4f300a97b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf32904364aad5e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.830+0000 7f4f23e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.830+0000 7f4f23e00700 1 -- 172.21.15.90:0/2761270155 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30020050 con 0x563c2e98ddb0 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.830+0000 7f4f23e00700 1 -- 172.21.15.90:0/2761270155 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f4f300b1e50 con 0x563c2e98ddb0 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.830+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2761270155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f300e2170 con 0x563c2e98ddb0 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f23e00700 1 -- 172.21.15.90:0/2761270155 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4f300a9270 con 0x563c2e98ddb0 2024-05-27T07:48:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f578dcb80 10 client.15924 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:42.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2761270155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563c2ea10e00 con 0x563c2e98ddb0 2024-05-27T07:48:42.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f23e00700 1 -- 172.21.15.90:0/2761270155 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f300e2170 con 0x563c2e98ddb0 2024-05-27T07:48:42.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f23e00700 1 -- 172.21.15.90:0/2761270155 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4f300a84d0 con 0x563c2e98ddb0 2024-05-27T07:48:42.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f578dcb80 10 client.15924 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:42.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f578dcb80 10 client.15924 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:48:42.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f578dcb80 20 client.15924 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:42.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x563c2ea11400 0x563c2ea13820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:42.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f578dcb80 4 client.15924 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:42.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.831+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2761270155 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c2e8b7570 con 0x563c2ea11400 2024-05-27T07:48:42.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.832+0000 7f4f41e00700 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x563c2ea11400 0x563c2ea13820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:42.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.833+0000 7f4f41e00700 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x563c2ea11400 0x563c2ea13820 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de249134ece38cac server_cookie=58406648f6f64d39 in_seq=0 out_seq=0 2024-05-27T07:48:42.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.833+0000 7f4f23e00700 10 client.15924 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:48:42.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f23e00700 1 -- 172.21.15.90:0/2761270155 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c2e8b7570 con 0x563c2ea11400 2024-05-27T07:48:42.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f23e00700 10 client.15924 handle_command_reply: tid=0 2024-05-27T07:48:42.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 client.15924 shutdown 2024-05-27T07:48:42.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2761270155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 msgr2=0x563c2e9a9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:42.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e9a9600 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d950 tx=0x7f4f300a97b0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2761270155 shutdown_connections 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x563c2ea11400 0x563c2ea13820 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c2e993d30 0x563c2e9a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c2e80ee40 0x563c2e9acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 --2- 172.21.15.90:0/2761270155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c2e98ddb0 0x563c2e9a9600 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f23e00700 0 client.15924 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2761270155 >> 172.21.15.90:0/2761270155 conn(0x563c2e86efc0 msgr2=0x563c2e996510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2761270155 shutdown_connections 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.834+0000 7f4f578dcb80 1 -- 172.21.15.90:0/2761270155 wait complete. 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.835+0000 7f4f578dcb80 20 client.15924 trim_cache size 0 max 16384 2024-05-27T07:48:42.836 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:48:42.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.835+0000 7f4f48e00700 1 -- 172.21.15.90:0/4160887811 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f280b2a70 msgr2=0x7f4f280b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:42.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.835+0000 7f4f48e00700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f280b2a70 0x7f4f280b4f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400b700 tx=0x7f4f3400b3c0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.835+0000 7f4f48e00700 1 -- 172.21.15.90:0/4160887811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f44105de0 msgr2=0x7f4f441036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:42.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.835+0000 7f4f48e00700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f44105de0 0x7f4f441036c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d980 tx=0x7f4f3000dc90 comp rx=0 tx=0).stop 2024-05-27T07:48:42.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.836+0000 7f4f48e00700 1 -- 172.21.15.90:0/4160887811 shutdown_connections 2024-05-27T07:48:42.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.836+0000 7f4f48e00700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f4410e900 0x7f4f44103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.836+0000 7f4f48e00700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4f280b2a70 0x7f4f280b4f80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.836+0000 7f4f48e00700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f44105490 0x7f4f44103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.836+0000 7f4f48e00700 1 --2- 172.21.15.90:0/4160887811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f44105de0 0x7f4f441036c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:42.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.836+0000 7f4f48e00700 1 -- 172.21.15.90:0/4160887811 >> 172.21.15.90:0/4160887811 conn(0x7f4f440fedb0 msgr2=0x7f4f440ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:42.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.836+0000 7f4f48e00700 1 -- 172.21.15.90:0/4160887811 shutdown_connections 2024-05-27T07:48:42.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:42.836+0000 7f4f48e00700 1 -- 172.21.15.90:0/4160887811 wait complete. 2024-05-27T07:48:42.912 INFO:teuthology.orchestra.run.smithi090.stderr:1900000 of 3200000 tuples (59%) done (elapsed 58.96 s, remaining 40.34 s) 2024-05-27T07:48:44.030 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:48:44.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:44 smithi129 ceph-mon[35254]: pgmap v74: 129 pgs: 129 active+clean; 507 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 115 op/s 2024-05-27T07:48:44.351 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-mon[35324]: pgmap v74: 129 pgs: 129 active+clean; 507 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 115 op/s 2024-05-27T07:48:44.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:44 smithi090 ceph-mon[31429]: pgmap v74: 129 pgs: 129 active+clean; 507 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 115 op/s 2024-05-27T07:48:44.497 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:44.750 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f578a200700 time 2024-05-27T07:48:44.351735+0000 2024-05-27T07:48:44.751 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:44.751 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:44.751 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f579fd54ba6] 2024-05-27T07:48:44.751 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f579fd54dd1] 2024-05-27T07:48:44.751 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f4a31e323d] 2024-05-27T07:48:44.751 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f4a31ce971] 2024-05-27T07:48:44.751 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f4a31ccfa0] 2024-05-27T07:48:44.751 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f4a31cd651] 2024-05-27T07:48:44.751 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f4a3162347] 2024-05-27T07:48:44.751 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f4a2e5e87e] 2024-05-27T07:48:44.752 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f4a2e63e94] 2024-05-27T07:48:44.752 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f4a2e6e969] 2024-05-27T07:48:44.752 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f4a320748c] 2024-05-27T07:48:44.752 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f4a32109cb] 2024-05-27T07:48:44.752 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 13: (Thread::entry_wrapper()+0x43) [0x7f579fd2372d] 2024-05-27T07:48:44.752 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 14: (Thread::_entry_func(void*)+0xd) [0x7f579fd23749] 2024-05-27T07:48:44.752 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f579e8c71ca] 2024-05-27T07:48:44.752 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 16: clone() 2024-05-27T07:48:44.752 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: *** Caught signal (Aborted) ** 2024-05-27T07:48:44.752 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: in thread 7f578a200700 thread_name:md_log_replay 2024-05-27T07:48:44.753 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.354+0000 7f578a200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f578a200700 time 2024-05-27T07:48:44.351735+0000 2024-05-27T07:48:44.753 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:44.753 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.753 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:44.753 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f579fd54ba6] 2024-05-27T07:48:44.753 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f579fd54dd1] 2024-05-27T07:48:44.753 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f4a31e323d] 2024-05-27T07:48:44.753 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f4a31ce971] 2024-05-27T07:48:44.753 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f4a31ccfa0] 2024-05-27T07:48:44.753 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f4a31cd651] 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f4a3162347] 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f4a2e5e87e] 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f4a2e63e94] 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f4a2e6e969] 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f4a320748c] 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f4a32109cb] 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 13: (Thread::entry_wrapper()+0x43) [0x7f579fd2372d] 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 14: (Thread::_entry_func(void*)+0xd) [0x7f579fd23749] 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f579e8c71ca] 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 16: clone() 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.754 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:44.755 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f579e8d1d20] 2024-05-27T07:48:44.755 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2: gsignal() 2024-05-27T07:48:44.755 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 3: abort() 2024-05-27T07:48:44.755 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f579fd54ced] 2024-05-27T07:48:44.755 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f579fd54dd1] 2024-05-27T07:48:44.755 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f4a31e323d] 2024-05-27T07:48:44.755 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f4a31ce971] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f4a31ccfa0] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f4a31cd651] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f4a3162347] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f4a2e5e87e] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f4a2e63e94] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f4a2e6e969] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f4a320748c] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f4a32109cb] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 16: (Thread::entry_wrapper()+0x43) [0x7f579fd2372d] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 17: (Thread::_entry_func(void*)+0xd) [0x7f579fd23749] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f579e8c71ca] 2024-05-27T07:48:44.756 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 19: clone() 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.358+0000 7f578a200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: in thread 7f578a200700 thread_name:md_log_replay 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f579e8d1d20] 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2: gsignal() 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 3: abort() 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f579fd54ced] 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f579fd54dd1] 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f4a31e323d] 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f4a31ce971] 2024-05-27T07:48:44.757 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f4a31ccfa0] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f4a31cd651] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f4a3162347] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f4a2e5e87e] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f4a2e63e94] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f4a2e6e969] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f4a320748c] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f4a32109cb] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 16: (Thread::entry_wrapper()+0x43) [0x7f579fd2372d] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 17: (Thread::_entry_func(void*)+0xd) [0x7f579fd23749] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f579e8c71ca] 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 19: clone() 2024-05-27T07:48:44.758 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: -1> 2024-05-27T07:48:44.354+0000 7f578a200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f578a200700 time 2024-05-27T07:48:44.351735+0000 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f579fd54ba6] 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f579fd54dd1] 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f4a31e323d] 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f4a31ce971] 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f4a31ccfa0] 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f4a31cd651] 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f4a3162347] 2024-05-27T07:48:44.759 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f4a2e5e87e] 2024-05-27T07:48:44.760 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f4a2e63e94] 2024-05-27T07:48:44.760 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f4a2e6e969] 2024-05-27T07:48:44.760 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f4a320748c] 2024-05-27T07:48:44.760 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f4a32109cb] 2024-05-27T07:48:44.760 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 13: (Thread::entry_wrapper()+0x43) [0x7f579fd2372d] 2024-05-27T07:48:44.760 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 14: (Thread::_entry_func(void*)+0xd) [0x7f579fd23749] 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f579e8c71ca] 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 16: clone() 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 0> 2024-05-27T07:48:44.358+0000 7f578a200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: in thread 7f578a200700 thread_name:md_log_replay 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f579e8d1d20] 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2: gsignal() 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 3: abort() 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f579fd54ced] 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f579fd54dd1] 2024-05-27T07:48:44.761 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f4a31e323d] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f4a31ce971] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f4a31ccfa0] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f4a31cd651] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f4a3162347] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f4a2e5e87e] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f4a2e63e94] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f4a2e6e969] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f4a320748c] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f4a32109cb] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 16: (Thread::entry_wrapper()+0x43) [0x7f579fd2372d] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 17: (Thread::_entry_func(void*)+0xd) [0x7f579fd23749] 2024-05-27T07:48:44.762 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f579e8c71ca] 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 19: clone() 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: -1> 2024-05-27T07:48:44.354+0000 7f578a200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f578a200700 time 2024-05-27T07:48:44.351735+0000 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f579fd54ba6] 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f579fd54dd1] 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f4a31e323d] 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f4a31ce971] 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f4a31ccfa0] 2024-05-27T07:48:44.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f4a31cd651] 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f4a3162347] 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f4a2e5e87e] 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f4a2e63e94] 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f4a2e6e969] 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f4a320748c] 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f4a32109cb] 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 13: (Thread::entry_wrapper()+0x43) [0x7f579fd2372d] 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 14: (Thread::_entry_func(void*)+0xd) [0x7f579fd23749] 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f579e8c71ca] 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 16: clone() 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 0> 2024-05-27T07:48:44.358+0000 7f578a200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: in thread 7f578a200700 thread_name:md_log_replay 2024-05-27T07:48:44.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:48:44.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f579e8d1d20] 2024-05-27T07:48:44.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2: gsignal() 2024-05-27T07:48:44.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 3: abort() 2024-05-27T07:48:44.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f579fd54ced] 2024-05-27T07:48:44.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f579fd54dd1] 2024-05-27T07:48:44.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f4a31e323d] 2024-05-27T07:48:44.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f4a31ce971] 2024-05-27T07:48:44.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f4a31ccfa0] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f4a31cd651] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f4a3162347] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f4a2e5e87e] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55f4a2e63e94] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f4a2e6e969] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f4a320748c] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f4a32109cb] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 16: (Thread::entry_wrapper()+0x43) [0x7f579fd2372d] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 17: (Thread::_entry_func(void*)+0xd) [0x7f579fd23749] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f579e8c71ca] 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 19: clone() 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:48:44.766 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[70821]: 2024-05-27T07:48:44.767 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:44 smithi136 podman[70866]: 2024-05-27 07:48:44.460378389 +0000 UTC m=+0.032477374 container died 5e5edd60f637a362ec3eb7c9aef61550625f1a55bf48dece28b3eac94b96725a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-05-27T07:48:45.499 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:45 smithi136 podman[70866]: 2024-05-27 07:48:45.151093621 +0000 UTC m=+0.723192632 container remove 5e5edd60f637a362ec3eb7c9aef61550625f1a55bf48dece28b3eac94b96725a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T07:48:45.499 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:45 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:48:45.984 INFO:teuthology.orchestra.run.smithi090.stderr:2000000 of 3200000 tuples (62%) done (elapsed 62.04 s, remaining 37.22 s) 2024-05-27T07:48:45.999 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:45 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:48:45.999 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:45 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.538s CPU time. 2024-05-27T07:48:46.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:46 smithi090 ceph-mon[31429]: pgmap v75: 129 pgs: 129 active+clean; 507 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-05-27T07:48:46.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:46 smithi129 ceph-mon[35254]: pgmap v75: 129 pgs: 129 active+clean; 507 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-05-27T07:48:46.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:46 smithi136 ceph-mon[35324]: pgmap v75: 129 pgs: 129 active+clean; 507 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-05-27T07:48:48.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:48 smithi090 ceph-mon[31429]: pgmap v76: 129 pgs: 129 active+clean; 507 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-05-27T07:48:48.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:48 smithi129 ceph-mon[35254]: pgmap v76: 129 pgs: 129 active+clean; 507 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-05-27T07:48:48.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:48 smithi136 ceph-mon[35324]: pgmap v76: 129 pgs: 129 active+clean; 507 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 21 op/s 2024-05-27T07:48:48.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.530+0000 7f3f0c000700 1 -- 172.21.15.90:0/3904728798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f04103120 msgr2=0x7f3f04111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.530+0000 7f3f0c000700 1 --2- 172.21.15.90:0/3904728798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f04103120 0x7f3f04111110 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00b0a0 tx=0x7f3efc00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.530+0000 7f3f0c000700 1 -- 172.21.15.90:0/3904728798 shutdown_connections 2024-05-27T07:48:48.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.530+0000 7f3f0c000700 1 --2- 172.21.15.90:0/3904728798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f04103120 0x7f3f04111110 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.530+0000 7f3f0c000700 1 --2- 172.21.15.90:0/3904728798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f0410a3d0 0x7f3f0410a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.530+0000 7f3f0c000700 1 --2- 172.21.15.90:0/3904728798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f041027d0 0x7f3f04102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.530+0000 7f3f0c000700 1 -- 172.21.15.90:0/3904728798 >> 172.21.15.90:0/3904728798 conn(0x7f3f040fe6e0 msgr2=0x7f3f04100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.530+0000 7f3f0c000700 1 -- 172.21.15.90:0/3904728798 shutdown_connections 2024-05-27T07:48:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.530+0000 7f3f0c000700 1 -- 172.21.15.90:0/3904728798 wait complete. 2024-05-27T07:48:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.532+0000 7f3f0c000700 1 Processor -- start 2024-05-27T07:48:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.532+0000 7f3f0c000700 1 -- start start 2024-05-27T07:48:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.532+0000 7f3f0c000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f041027d0 0x7f3f0419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.532+0000 7f3f0c000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f04103120 0x7f3f0419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.532+0000 7f3f0c000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f0410a3d0 0x7f3f041a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.532+0000 7f3f0c000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3f04114370 con 0x7f3f04103120 2024-05-27T07:48:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.532+0000 7f3f0c000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041141f0 con 0x7f3f0410a3d0 2024-05-27T07:48:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.532+0000 7f3f0c000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041144f0 con 0x7f3f041027d0 2024-05-27T07:48:48.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f08c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f04103120 0x7f3f0419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f08c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f04103120 0x7f3f0419d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37670/0 (socket says 172.21.15.90:37670) 2024-05-27T07:48:48.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f08c00700 1 -- 172.21.15.90:0/1250754344 learned_addr learned my addr 172.21.15.90:0/1250754344 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:48.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f09600700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f041027d0 0x7f3f0419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f0a000700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f0410a3d0 0x7f3f041a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f08c00700 1 -- 172.21.15.90:0/1250754344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f041027d0 msgr2=0x7f3f0419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f08c00700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f041027d0 0x7f3f0419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f08c00700 1 -- 172.21.15.90:0/1250754344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f0410a3d0 msgr2=0x7f3f041a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f08c00700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f0410a3d0 0x7f3f041a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f08c00700 1 -- 172.21.15.90:0/1250754344 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f041a1e30 con 0x7f3f04103120 2024-05-27T07:48:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.533+0000 7f3f09600700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f041027d0 0x7f3f0419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.534+0000 7f3f08c00700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f04103120 0x7f3f0419d500 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000e970 tx=0x7f3ef000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.534+0000 7f3f02000700 1 -- 172.21.15.90:0/1250754344 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef000cbd0 con 0x7f3f04103120 2024-05-27T07:48:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.534+0000 7f3f0c000700 1 -- 172.21.15.90:0/1250754344 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f041a2120 con 0x7f3f04103120 2024-05-27T07:48:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.534+0000 7f3f02000700 1 -- 172.21.15.90:0/1250754344 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ef000cd70 con 0x7f3f04103120 2024-05-27T07:48:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.534+0000 7f3f02000700 1 -- 172.21.15.90:0/1250754344 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00055c0 con 0x7f3f04103120 2024-05-27T07:48:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.534+0000 7f3f0c000700 1 -- 172.21.15.90:0/1250754344 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f0410b0c0 con 0x7f3f04103120 2024-05-27T07:48:48.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.538+0000 7f3f02000700 1 -- 172.21.15.90:0/1250754344 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3ef0004750 con 0x7f3f04103120 2024-05-27T07:48:48.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.540+0000 7f3f1ac0db80 1 Processor -- start 2024-05-27T07:48:48.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.541+0000 7f3f1ac0db80 1 -- start start 2024-05-27T07:48:48.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.541+0000 7f3f02000700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3eec0bb200 0x7f3eec0bd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.541+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 0x55cc9c500330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.541+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 0x55cc9c500870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.541+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c500e70 0x55cc9c50d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.541+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c42ee40 con 0x55cc9c381e40 2024-05-27T07:48:48.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.541+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c41e9e0 con 0x55cc9c500e70 2024-05-27T07:48:48.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.541+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c422a20 con 0x55cc9c46d9f0 2024-05-27T07:48:48.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f02000700 1 -- 172.21.15.90:0/1250754344 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3ef00e2cf0 con 0x7f3f04103120 2024-05-27T07:48:48.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f09600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 0x55cc9c500330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f09600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 0x55cc9c500330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50996/0 (socket says 172.21.15.90:50996) 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f09600700 1 -- 172.21.15.90:0/2500065567 learned_addr learned my addr 172.21.15.90:0/2500065567 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f0a000700 1 --2- 172.21.15.90:0/2500065567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c500e70 0x55cc9c50d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f0a000700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3eec0bb200 0x7f3eec0bd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f09600700 1 -- 172.21.15.90:0/2500065567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c500e70 msgr2=0x55cc9c50d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f09600700 1 --2- 172.21.15.90:0/2500065567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c500e70 0x55cc9c50d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f09600700 1 -- 172.21.15.90:0/2500065567 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 msgr2=0x55cc9c500870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f09600700 1 --2- 172.21.15.90:0/2500065567 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 0x55cc9c500870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.542+0000 7f3f09600700 1 -- 172.21.15.90:0/2500065567 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9c3e7420 con 0x55cc9c46d9f0 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.543+0000 7f3f09600700 1 --2- 172.21.15.90:0/2500065567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 0x55cc9c500330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800bd10 tx=0x7f3ef800f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb40d3aacd11844a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.543+0000 7f3f0a000700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3eec0bb200 0x7f3eec0bd710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00b780 tx=0x7f3efc00b3b0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.544+0000 7f3ef6a00700 1 -- 172.21.15.90:0/2500065567 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef800be60 con 0x55cc9c46d9f0 2024-05-27T07:48:48.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.544+0000 7f3ef6a00700 1 -- 172.21.15.90:0/2500065567 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ef800fe50 con 0x55cc9c46d9f0 2024-05-27T07:48:48.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.544+0000 7f3ef6a00700 1 -- 172.21.15.90:0/2500065567 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8010730 con 0x55cc9c46d9f0 2024-05-27T07:48:48.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.545+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2500065567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 msgr2=0x55cc9c500330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.545+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/2500065567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 0x55cc9c500330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800bd10 tx=0x7f3ef800f640 comp rx=0 tx=0).stop 2024-05-27T07:48:48.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.545+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2500065567 shutdown_connections 2024-05-27T07:48:48.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.545+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/2500065567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c500e70 0x55cc9c50d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.545+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/2500065567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 0x55cc9c500330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.545+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/2500065567 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 0x55cc9c500870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.545+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2500065567 >> 172.21.15.90:0/2500065567 conn(0x55cc9c41b390 msgr2=0x55cc9c4fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.545+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2500065567 shutdown_connections 2024-05-27T07:48:48.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.545+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2500065567 wait complete. 2024-05-27T07:48:48.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.546+0000 7f3f1ac0db80 1 -- start start 2024-05-27T07:48:48.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.546+0000 7f3f1ac0db80 1 Processor -- start 2024-05-27T07:48:48.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c381e40 0x55cc9c50a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c46d9f0 0x55cc9c50acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c50b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c324fa0 con 0x55cc9c500e70 2024-05-27T07:48:48.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c3bf760 con 0x55cc9c46d9f0 2024-05-27T07:48:48.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c50db80 con 0x55cc9c381e40 2024-05-27T07:48:48.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f1ac0db80 10 client.0 _ll_register_callbacks cb 0x55cc9c3c8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:48.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f0a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c50b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f0a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c50b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37700/0 (socket says 172.21.15.90:37700) 2024-05-27T07:48:48.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f0a000700 1 -- 172.21.15.90:0/2040291758 learned_addr learned my addr 172.21.15.90:0/2040291758 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:48.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.547+0000 7f3f09600700 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c381e40 0x55cc9c50a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.548+0000 7f3f08c00700 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c46d9f0 0x55cc9c50acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.548+0000 7f3f0a000700 1 -- 172.21.15.90:0/2040291758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c381e40 msgr2=0x55cc9c50a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.548+0000 7f3f0a000700 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c381e40 0x55cc9c50a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.548+0000 7f3f0a000700 1 -- 172.21.15.90:0/2040291758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c46d9f0 msgr2=0x55cc9c50acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.548+0000 7f3f0a000700 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c46d9f0 0x55cc9c50acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.548+0000 7f3f0a000700 1 -- 172.21.15.90:0/2040291758 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc9c36e8d0 con 0x55cc9c500e70 2024-05-27T07:48:48.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.548+0000 7f3f0a000700 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c50b200 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3efc004cd0 tx=0x7f3efc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b2c2627f25525a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.549+0000 7f3eeb400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:48.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.549+0000 7f3eeb400700 1 -- 172.21.15.90:0/2040291758 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc030060 con 0x55cc9c500e70 2024-05-27T07:48:48.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.549+0000 7f3eeb400700 1 -- 172.21.15.90:0/2040291758 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3efc007c40 con 0x55cc9c500e70 2024-05-27T07:48:48.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.549+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2040291758 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef800fe50 con 0x55cc9c500e70 2024-05-27T07:48:48.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.549+0000 7f3eeb400700 1 -- 172.21.15.90:0/2040291758 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3efc024050 con 0x55cc9c500e70 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.549+0000 7f3f1ac0db80 10 client.15933 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.549+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2040291758 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cc9c3ef060 con 0x55cc9c500e70 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.550+0000 7f3eeb400700 1 -- 172.21.15.90:0/2040291758 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ef800fe50 con 0x55cc9c500e70 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.550+0000 7f3eeb400700 1 -- 172.21.15.90:0/2040291758 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f3efc011070 con 0x55cc9c500e70 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.550+0000 7f3f1ac0db80 10 client.15933 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.550+0000 7f3f1ac0db80 10 client.15933 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.550+0000 7f3f1ac0db80 20 client.15933 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.550+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55cc9c51ab50 0x55cc9c51cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.550+0000 7f3f1ac0db80 4 client.15933 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.550+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2040291758 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc9c430d90 con 0x55cc9c51ab50 2024-05-27T07:48:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.551+0000 7f3f09600700 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55cc9c51ab50 0x55cc9c51cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.552+0000 7f3f09600700 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55cc9c51ab50 0x55cc9c51cf90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba1051aa2284a6d5 server_cookie=2a03f21daa09c36c in_seq=0 out_seq=0 2024-05-27T07:48:48.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.552+0000 7f3eeb400700 10 client.15933 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:48:48.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3eeb400700 1 -- 172.21.15.90:0/2040291758 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cc9c430d90 con 0x55cc9c51ab50 2024-05-27T07:48:48.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3eeb400700 10 client.15933 handle_command_reply: tid=0 2024-05-27T07:48:48.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3f1ac0db80 1 client.15933 shutdown 2024-05-27T07:48:48.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2040291758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 msgr2=0x55cc9c50b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c50b200 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3efc004cd0 tx=0x7f3efc0099e0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2040291758 shutdown_connections 2024-05-27T07:48:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55cc9c51ab50 0x55cc9c51cf90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c50b200 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3eeb400700 0 client.15933 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:48:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c381e40 0x55cc9c50a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/2040291758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c46d9f0 0x55cc9c50acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.554+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2040291758 >> 172.21.15.90:0/2040291758 conn(0x55cc9c41b390 msgr2=0x55cc9c500450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.555+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2040291758 shutdown_connections 2024-05-27T07:48:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.555+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/2040291758 wait complete. 2024-05-27T07:48:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.555+0000 7f3f1ac0db80 20 client.15933 trim_cache size 0 max 16384 2024-05-27T07:48:48.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.577+0000 7f3f1ac0db80 1 Processor -- start 2024-05-27T07:48:48.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.577+0000 7f3f1ac0db80 1 -- start start 2024-05-27T07:48:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.577+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c381e40 0x55cc9c4fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.577+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c46d9f0 0x55cc9c505fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.577+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c516c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.577+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041ae310 con 0x55cc9c500e70 2024-05-27T07:48:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.577+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041afe50 con 0x55cc9c46d9f0 2024-05-27T07:48:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.577+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041ae9e0 con 0x55cc9c381e40 2024-05-27T07:48:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.578+0000 7f3f0a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c516c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.578+0000 7f3f0a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c516c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37702/0 (socket says 172.21.15.90:37702) 2024-05-27T07:48:48.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.578+0000 7f3f0a000700 1 -- 172.21.15.90:0/4098783498 learned_addr learned my addr 172.21.15.90:0/4098783498 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:48.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.578+0000 7f3f0a000700 1 -- 172.21.15.90:0/4098783498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c381e40 msgr2=0x55cc9c4fc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.578+0000 7f3f0a000700 1 --2- 172.21.15.90:0/4098783498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c381e40 0x55cc9c4fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.579+0000 7f3f0a000700 1 -- 172.21.15.90:0/4098783498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c46d9f0 msgr2=0x55cc9c505fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.579+0000 7f3f0a000700 1 --2- 172.21.15.90:0/4098783498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c46d9f0 0x55cc9c505fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.579+0000 7f3f0a000700 1 -- 172.21.15.90:0/4098783498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9c50b570 con 0x55cc9c500e70 2024-05-27T07:48:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.579+0000 7f3f0a000700 1 --2- 172.21.15.90:0/4098783498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c516c20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009d20 tx=0x7f3efc006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694793680f20884c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.579+0000 7f3ef6000700 1 -- 172.21.15.90:0/4098783498 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc030040 con 0x55cc9c500e70 2024-05-27T07:48:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.579+0000 7f3ef6000700 1 -- 172.21.15.90:0/4098783498 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3efc025040 con 0x55cc9c500e70 2024-05-27T07:48:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.579+0000 7f3ef6000700 1 -- 172.21.15.90:0/4098783498 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc020450 con 0x55cc9c500e70 2024-05-27T07:48:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.581+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/4098783498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 msgr2=0x55cc9c516c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.581+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/4098783498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c516c20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009d20 tx=0x7f3efc006040 comp rx=0 tx=0).stop 2024-05-27T07:48:48.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.581+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/4098783498 shutdown_connections 2024-05-27T07:48:48.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.581+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/4098783498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c500e70 0x55cc9c516c20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.581+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/4098783498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c381e40 0x55cc9c4fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.581+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/4098783498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c46d9f0 0x55cc9c505fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.581+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/4098783498 >> 172.21.15.90:0/4098783498 conn(0x55cc9c41b390 msgr2=0x55cc9c4fbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.581+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/4098783498 shutdown_connections 2024-05-27T07:48:48.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.581+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/4098783498 wait complete. 2024-05-27T07:48:48.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.581+0000 7f3f1ac0db80 1 -- start start 2024-05-27T07:48:48.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.582+0000 7f3f1ac0db80 1 Processor -- start 2024-05-27T07:48:48.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.582+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 0x55cc9c504840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.582+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 0x55cc9c504d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.582+0000 7f3f1ac0db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c500e70 0x55cc9c511e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.582+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041af780 con 0x55cc9c381e40 2024-05-27T07:48:48.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.582+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041af0b0 con 0x55cc9c500e70 2024-05-27T07:48:48.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.582+0000 7f3f1ac0db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041ac290 con 0x55cc9c46d9f0 2024-05-27T07:48:48.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.582+0000 7f3f1ac0db80 10 client.0 _ll_register_callbacks cb 0x55cc9c3c8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.583+0000 7f3f09600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 0x55cc9c504840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.583+0000 7f3f09600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 0x55cc9c504840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37710/0 (socket says 172.21.15.90:37710) 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.583+0000 7f3f08c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 0x55cc9c504d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.583+0000 7f3f09600700 1 -- 172.21.15.90:0/1174053939 learned_addr learned my addr 172.21.15.90:0/1174053939 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.583+0000 7f3f0a000700 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c500e70 0x55cc9c511e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.583+0000 7f3f09600700 1 -- 172.21.15.90:0/1174053939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 msgr2=0x55cc9c504d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.583+0000 7f3f09600700 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 0x55cc9c504d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.583+0000 7f3f09600700 1 -- 172.21.15.90:0/1174053939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c500e70 msgr2=0x55cc9c511e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.583+0000 7f3f09600700 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c500e70 0x55cc9c511e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.583+0000 7f3f09600700 1 -- 172.21.15.90:0/1174053939 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc9c512350 con 0x55cc9c381e40 2024-05-27T07:48:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.584+0000 7f3f09600700 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 0x55cc9c504840 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800bd10 tx=0x7f3ef800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b22f1fb59ee54bf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.584+0000 7f3eeb400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.584+0000 7f3eeb400700 1 -- 172.21.15.90:0/1174053939 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8004030 con 0x55cc9c381e40 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.584+0000 7f3eeb400700 1 -- 172.21.15.90:0/1174053939 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f3ef80041d0 con 0x55cc9c381e40 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.584+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/1174053939 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3efc025040 con 0x55cc9c381e40 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.584+0000 7f3eeb400700 1 -- 172.21.15.90:0/1174053939 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3ef8038050 con 0x55cc9c381e40 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.584+0000 7f3f1ac0db80 10 client.15939 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.584+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/1174053939 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cc9c5135b0 con 0x55cc9c381e40 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.585+0000 7f3eeb400700 1 -- 172.21.15.90:0/1174053939 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3efc025040 con 0x55cc9c381e40 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.585+0000 7f3eeb400700 1 -- 172.21.15.90:0/1174053939 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f3ef80049a0 con 0x55cc9c381e40 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.585+0000 7f3f1ac0db80 10 client.15939 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.585+0000 7f3f1ac0db80 10 client.15939 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.585+0000 7f3f1ac0db80 20 client.15939 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:48.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.585+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55cc9c585620 0x55cc9c587a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:48.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.585+0000 7f3f1ac0db80 4 client.15939 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:48.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.585+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/1174053939 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc9c42a570 con 0x55cc9c585620 2024-05-27T07:48:48.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.586+0000 7f3f08c00700 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55cc9c585620 0x55cc9c587a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:48.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.587+0000 7f3eeb400700 10 client.15939 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:48:48.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.587+0000 7f3f08c00700 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55cc9c585620 0x55cc9c587a40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c77c6bc855246d7 server_cookie=279179d16163d54c in_seq=0 out_seq=0 2024-05-27T07:48:48.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3eeb400700 1 -- 172.21.15.90:0/1174053939 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc9c42a570 con 0x55cc9c585620 2024-05-27T07:48:48.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3eeb400700 10 client.15939 handle_command_reply: tid=0 2024-05-27T07:48:48.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3f1ac0db80 1 client.15939 shutdown 2024-05-27T07:48:48.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/1174053939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 msgr2=0x55cc9c504840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 0x55cc9c504840 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800bd10 tx=0x7f3ef800be60 comp rx=0 tx=0).stop 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/1174053939 shutdown_connections 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55cc9c585620 0x55cc9c587a40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cc9c500e70 0x55cc9c511e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cc9c381e40 0x55cc9c504840 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3f1ac0db80 1 --2- 172.21.15.90:0/1174053939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cc9c46d9f0 0x55cc9c504d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/1174053939 >> 172.21.15.90:0/1174053939 conn(0x55cc9c41b390 msgr2=0x55cc9c501ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.588+0000 7f3eeb400700 0 client.15939 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.589+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/1174053939 shutdown_connections 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.589+0000 7f3f1ac0db80 1 -- 172.21.15.90:0/1174053939 wait complete. 2024-05-27T07:48:48.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.589+0000 7f3f1ac0db80 20 client.15939 trim_cache size 0 max 16384 2024-05-27T07:48:48.591 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:48:48.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 -- 172.21.15.90:0/1250754344 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3eec0bb200 msgr2=0x7f3eec0bd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3eec0bb200 0x7f3eec0bd710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00b780 tx=0x7f3efc00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 -- 172.21.15.90:0/1250754344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f04103120 msgr2=0x7f3f0419d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:48.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f04103120 0x7f3f0419d500 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000e970 tx=0x7f3ef000ed30 comp rx=0 tx=0).stop 2024-05-27T07:48:48.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 -- 172.21.15.90:0/1250754344 shutdown_connections 2024-05-27T07:48:48.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3eec0bb200 0x7f3eec0bd710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f0410a3d0 0x7f3f041a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f041027d0 0x7f3f0419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 --2- 172.21.15.90:0/1250754344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f04103120 0x7f3f0419d500 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:48.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 -- 172.21.15.90:0/1250754344 >> 172.21.15.90:0/1250754344 conn(0x7f3f040fe6e0 msgr2=0x7f3f04100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:48.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 -- 172.21.15.90:0/1250754344 shutdown_connections 2024-05-27T07:48:48.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:48.590+0000 7f3f0c000700 1 -- 172.21.15.90:0/1250754344 wait complete. 2024-05-27T07:48:49.151 INFO:teuthology.orchestra.run.smithi090.stderr:2100000 of 3200000 tuples (65%) done (elapsed 65.20 s, remaining 34.15 s) 2024-05-27T07:48:49.430 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:49 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:48:49.481 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:48:49.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:49 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:48:49.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:49 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:48:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.792+0000 7f6d65a00640 1 -- 172.21.15.90:0/2698157174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d60104bb0 msgr2=0x7f6d60104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.792+0000 7f6d65a00640 1 --2- 172.21.15.90:0/2698157174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d60104bb0 0x7f6d60104fc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009900 tx=0x7f6d5002a410 comp rx=0 tx=0).stop 2024-05-27T07:48:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.792+0000 7f6d65a00640 1 -- 172.21.15.90:0/2698157174 shutdown_connections 2024-05-27T07:48:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.792+0000 7f6d65a00640 1 --2- 172.21.15.90:0/2698157174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6010b610 0x7f6d6010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.792+0000 7f6d65a00640 1 --2- 172.21.15.90:0/2698157174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d6010e000 0x7f6d601104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.792+0000 7f6d65a00640 1 --2- 172.21.15.90:0/2698157174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d60104bb0 0x7f6d60104fc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.792+0000 7f6d65a00640 1 -- 172.21.15.90:0/2698157174 >> 172.21.15.90:0/2698157174 conn(0x7f6d600fe210 msgr2=0x7f6d60100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.792+0000 7f6d65a00640 1 -- 172.21.15.90:0/2698157174 shutdown_connections 2024-05-27T07:48:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.792+0000 7f6d65a00640 1 -- 172.21.15.90:0/2698157174 wait complete. 2024-05-27T07:48:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.793+0000 7f6d65a00640 1 Processor -- start 2024-05-27T07:48:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.794+0000 7f6d65a00640 1 -- start start 2024-05-27T07:48:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.794+0000 7f6d65a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d60104bb0 0x7f6d60104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.794+0000 7f6d65a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d6010b610 0x7f6d60102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.794+0000 7f6d65a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6010e000 0x7f6d60102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.794+0000 7f6d65a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60113b60 con 0x7f6d6010b610 2024-05-27T07:48:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.794+0000 7f6d65a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601139e0 con 0x7f6d6010e000 2024-05-27T07:48:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.794+0000 7f6d65a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60113ce0 con 0x7f6d60104bb0 2024-05-27T07:48:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d6010b610 0x7f6d60102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d6010b610 0x7f6d60102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41366/0 (socket says 172.21.15.90:41366) 2024-05-27T07:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d5f400640 1 -- 172.21.15.90:0/1082059197 learned_addr learned my addr 172.21.15.90:0/1082059197 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d65000640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6010e000 0x7f6d60102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d60104bb0 0x7f6d60104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d5f400640 1 -- 172.21.15.90:0/1082059197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d60104bb0 msgr2=0x7f6d60104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d5f400640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d60104bb0 0x7f6d60104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d5f400640 1 -- 172.21.15.90:0/1082059197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6010e000 msgr2=0x7f6d60102a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d5f400640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6010e000 0x7f6d60102a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d5f400640 1 -- 172.21.15.90:0/1082059197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d60103260 con 0x7f6d6010b610 2024-05-27T07:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.795+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d60104bb0 0x7f6d60104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.796+0000 7f6d5f400640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d6010b610 0x7f6d60102470 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00d950 tx=0x7f6d4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.796+0000 7f6d5cc00640 1 -- 172.21.15.90:0/1082059197 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c017070 con 0x7f6d6010b610 2024-05-27T07:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.796+0000 7f6d5cc00640 1 -- 172.21.15.90:0/1082059197 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d4c009c30 con 0x7f6d6010b610 2024-05-27T07:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.796+0000 7f6d65a00640 1 -- 172.21.15.90:0/1082059197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d601ad080 con 0x7f6d6010b610 2024-05-27T07:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.796+0000 7f6d5cc00640 1 -- 172.21.15.90:0/1082059197 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c00f620 con 0x7f6d6010b610 2024-05-27T07:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.797+0000 7f6d65a00640 1 -- 172.21.15.90:0/1082059197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d601ad5d0 con 0x7f6d6010b610 2024-05-27T07:48:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.801+0000 7f6d6e07c740 1 Processor -- start 2024-05-27T07:48:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.801+0000 7f6d6e07c740 1 -- start start 2024-05-27T07:48:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.801+0000 7f6d5cc00640 1 -- 172.21.15.90:0/1082059197 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6d4c010460 con 0x7f6d6010b610 2024-05-27T07:48:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.801+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8254f1b0 0x558c828f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.801+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 0x558c828f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.801+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8252e4d0 0x558c828f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.802+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558c82862b90 con 0x558c8252e4d0 2024-05-27T07:48:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.802+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558c82852730 con 0x558c825262f0 2024-05-27T07:48:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.802+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558c82856770 con 0x558c8254f1b0 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.802+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8252e4d0 0x558c828f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.802+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8252e4d0 0x558c828f24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41370/0 (socket says 172.21.15.90:41370) 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.802+0000 7f6d5f400640 1 -- 172.21.15.90:0/3768943796 learned_addr learned my addr 172.21.15.90:0/3768943796 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.802+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/3768943796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 0x558c828f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d5f400640 1 -- 172.21.15.90:0/3768943796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8254f1b0 msgr2=0x558c828f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d5f400640 1 --2- 172.21.15.90:0/3768943796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8254f1b0 0x558c828f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d5f400640 1 -- 172.21.15.90:0/3768943796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 msgr2=0x558c828f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d5f400640 1 --2- 172.21.15.90:0/3768943796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 0x558c828f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d5f400640 1 -- 172.21.15.90:0/3768943796 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c827eb8b0 con 0x558c8252e4d0 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/3768943796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 0x558c828f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d5f400640 1 --2- 172.21.15.90:0/3768943796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8252e4d0 0x558c828f24e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0df890 tx=0x7f6d4c0e8b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16edc1b04ad7a7b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d55600640 1 -- 172.21.15.90:0/3768943796 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0f5070 con 0x558c8252e4d0 2024-05-27T07:48:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d55600640 1 -- 172.21.15.90:0/3768943796 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d4c0e4070 con 0x558c8252e4d0 2024-05-27T07:48:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d55600640 1 -- 172.21.15.90:0/3768943796 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0ea9d0 con 0x558c8252e4d0 2024-05-27T07:48:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.803+0000 7f6d5cc00640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6d440b29d0 0x7f6d440b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.804+0000 7f6d5cc00640 1 -- 172.21.15.90:0/1082059197 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6d4c0e2580 con 0x7f6d6010b610 2024-05-27T07:48:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.804+0000 7f6d65000640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6d440b29d0 0x7f6d440b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.805+0000 7f6d6e07c740 1 -- 172.21.15.90:0/3768943796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8252e4d0 msgr2=0x558c828f24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.805+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/3768943796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8252e4d0 0x558c828f24e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0df890 tx=0x7f6d4c0e8b60 comp rx=0 tx=0).stop 2024-05-27T07:48:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.805+0000 7f6d65000640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6d440b29d0 0x7f6d440b4ee0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x558c828eebd0 tx=0x7f6d58009a60 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.805+0000 7f6d6e07c740 1 -- 172.21.15.90:0/3768943796 shutdown_connections 2024-05-27T07:48:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.805+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/3768943796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 0x558c828f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.805+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/3768943796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8252e4d0 0x558c828f24e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.805+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/3768943796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8254f1b0 0x558c828f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.805+0000 7f6d6e07c740 1 -- 172.21.15.90:0/3768943796 >> 172.21.15.90:0/3768943796 conn(0x558c82543300 msgr2=0x558c828ee210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.805+0000 7f6d6e07c740 1 -- 172.21.15.90:0/3768943796 shutdown_connections 2024-05-27T07:48:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.805+0000 7f6d6e07c740 1 -- 172.21.15.90:0/3768943796 wait complete. 2024-05-27T07:48:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.806+0000 7f6d6e07c740 1 -- start start 2024-05-27T07:48:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.806+0000 7f6d6e07c740 1 Processor -- start 2024-05-27T07:48:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.807+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 0x558c828fbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.807+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c828fc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.807+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8254f1b0 0x558c82907ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.807+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558c82538790 con 0x558c8254f1b0 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.807+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558c827fabc0 con 0x558c825262f0 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.807+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558c828feb90 con 0x558c8252e4d0 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.807+0000 7f6d6e07c740 10 client.0 _ll_register_callbacks cb 0x558c827d8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.807+0000 7f6d65000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8254f1b0 0x558c82907ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.807+0000 7f6d65000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8254f1b0 0x558c82907ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41376/0 (socket says 172.21.15.90:41376) 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.807+0000 7f6d65000640 1 -- 172.21.15.90:0/47919660 learned_addr learned my addr 172.21.15.90:0/47919660 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.808+0000 7f6d5f400640 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c828fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.808+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 0x558c828fbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.808+0000 7f6d65000640 1 -- 172.21.15.90:0/47919660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 msgr2=0x558c828fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.808+0000 7f6d65000640 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c828fc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.808+0000 7f6d65000640 1 -- 172.21.15.90:0/47919660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 msgr2=0x558c828fbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.808+0000 7f6d65000640 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 0x558c828fbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.808+0000 7f6d65000640 1 -- 172.21.15.90:0/47919660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c82567210 con 0x558c8254f1b0 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.808+0000 7f6d65000640 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8254f1b0 0x558c82907ec0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6d58012c40 tx=0x7f6d58004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238e934d3c97a307 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.808+0000 7f6d4a000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.809+0000 7f6d4a000640 1 -- 172.21.15.90:0/47919660 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58023070 con 0x558c8254f1b0 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.809+0000 7f6d6e07c740 1 -- 172.21.15.90:0/47919660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d4c0e4070 con 0x558c8254f1b0 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.809+0000 7f6d4a000640 1 -- 172.21.15.90:0/47919660 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6d5801f450 con 0x558c8254f1b0 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.809+0000 7f6d4a000640 1 -- 172.21.15.90:0/47919660 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6d5802f050 con 0x558c8254f1b0 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.809+0000 7f6d6e07c740 10 client.15951 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.809+0000 7f6d6e07c740 1 -- 172.21.15.90:0/47919660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558c82633620 con 0x558c8254f1b0 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.809+0000 7f6d4a000640 1 -- 172.21.15.90:0/47919660 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d4c0e4070 con 0x558c8254f1b0 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.810+0000 7f6d4a000640 1 -- 172.21.15.90:0/47919660 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f6d5800f040 con 0x558c8254f1b0 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.810+0000 7f6d6e07c740 10 client.15951 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.810+0000 7f6d6e07c740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.810+0000 7f6d6e07c740 20 client.15951 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.810+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558c8250dd80 0x558c8290df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.810+0000 7f6d6e07c740 4 client.15951 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.810+0000 7f6d6e07c740 1 -- 172.21.15.90:0/47919660 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c82864ae0 con 0x558c8250dd80 2024-05-27T07:48:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.810+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558c8250dd80 0x558c8290df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.812+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558c8250dd80 0x558c8290df10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2a4aa7c86ff84b server_cookie=a929dce052233375 in_seq=0 out_seq=0 2024-05-27T07:48:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.812+0000 7f6d4a000640 10 client.15951 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d4a000640 1 -- 172.21.15.90:0/47919660 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558c82864ae0 con 0x558c8250dd80 2024-05-27T07:48:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d4a000640 10 client.15951 handle_command_reply: tid=0 2024-05-27T07:48:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d6e07c740 1 client.15951 shutdown 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d6e07c740 1 -- 172.21.15.90:0/47919660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8254f1b0 msgr2=0x558c82907ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8254f1b0 0x558c82907ec0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6d58012c40 tx=0x7f6d58004ab0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d6e07c740 1 -- 172.21.15.90:0/47919660 shutdown_connections 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558c8250dd80 0x558c8290df10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c8254f1b0 0x558c82907ec0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c828fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/47919660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c825262f0 0x558c828fbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d4a000640 0 client.15951 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.813+0000 7f6d6e07c740 1 -- 172.21.15.90:0/47919660 >> 172.21.15.90:0/47919660 conn(0x558c82543300 msgr2=0x558c828f3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.814+0000 7f6d6e07c740 1 -- 172.21.15.90:0/47919660 shutdown_connections 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.814+0000 7f6d6e07c740 1 -- 172.21.15.90:0/47919660 wait complete. 2024-05-27T07:48:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.814+0000 7f6d6e07c740 20 client.15951 trim_cache size 0 max 16384 2024-05-27T07:48:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.839+0000 7f6d6e07c740 1 Processor -- start 2024-05-27T07:48:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.839+0000 7f6d6e07c740 1 -- start start 2024-05-27T07:48:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.839+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c828fe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.839+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c828f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.839+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c8254f1b0 0x558c828f5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.839+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601b76a0 con 0x558c825262f0 2024-05-27T07:48:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.839+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601b91e0 con 0x558c8254f1b0 2024-05-27T07:48:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.839+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601b7d70 con 0x558c8252e4d0 2024-05-27T07:48:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.840+0000 7f6d5fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c828fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.840+0000 7f6d5fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c828fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41380/0 (socket says 172.21.15.90:41380) 2024-05-27T07:48:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.840+0000 7f6d5fe00640 1 -- 172.21.15.90:0/2911450608 learned_addr learned my addr 172.21.15.90:0/2911450608 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.840+0000 7f6d65000640 1 --2- 172.21.15.90:0/2911450608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c8254f1b0 0x558c828f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.840+0000 7f6d5f400640 1 --2- 172.21.15.90:0/2911450608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c828f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.840+0000 7f6d5fe00640 1 -- 172.21.15.90:0/2911450608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 msgr2=0x558c828f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.840+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/2911450608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c828f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.840+0000 7f6d5fe00640 1 -- 172.21.15.90:0/2911450608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c8254f1b0 msgr2=0x558c828f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.840+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/2911450608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c8254f1b0 0x558c828f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.840+0000 7f6d5fe00640 1 -- 172.21.15.90:0/2911450608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c828f5c50 con 0x558c825262f0 2024-05-27T07:48:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.841+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/2911450608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c828fe380 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6d5002a9d0 tx=0x7f6d50009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2389070964395f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.841+0000 7f6d54c00640 1 -- 172.21.15.90:0/2911450608 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50035040 con 0x558c825262f0 2024-05-27T07:48:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.841+0000 7f6d54c00640 1 -- 172.21.15.90:0/2911450608 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d50048070 con 0x558c825262f0 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.841+0000 7f6d54c00640 1 -- 172.21.15.90:0/2911450608 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50003850 con 0x558c825262f0 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.842+0000 7f6d6e07c740 1 -- 172.21.15.90:0/2911450608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 msgr2=0x558c828fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.842+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/2911450608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c828fe380 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6d5002a9d0 tx=0x7f6d50009680 comp rx=0 tx=0).stop 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.842+0000 7f6d6e07c740 1 -- 172.21.15.90:0/2911450608 shutdown_connections 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.842+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/2911450608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c8254f1b0 0x558c828f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.842+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/2911450608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c828f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.842+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/2911450608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c828fe380 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.842+0000 7f6d6e07c740 1 -- 172.21.15.90:0/2911450608 >> 172.21.15.90:0/2911450608 conn(0x558c82543300 msgr2=0x558c828fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.843+0000 7f6d6e07c740 1 -- 172.21.15.90:0/2911450608 shutdown_connections 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.843+0000 7f6d6e07c740 1 -- 172.21.15.90:0/2911450608 wait complete. 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.843+0000 7f6d6e07c740 1 -- start start 2024-05-27T07:48:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.843+0000 7f6d6e07c740 1 Processor -- start 2024-05-27T07:48:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c82903f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c82904490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d6e07c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c8254f1b0 0x558c828ff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601b8b10 con 0x558c825262f0 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601b8440 con 0x558c8254f1b0 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d6e07c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601b5660 con 0x558c8252e4d0 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d6e07c740 10 client.0 _ll_register_callbacks cb 0x7f6d58016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d5fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c82903f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d5fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c82903f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41394/0 (socket says 172.21.15.90:41394) 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d5fe00640 1 -- 172.21.15.90:0/1434353313 learned_addr learned my addr 172.21.15.90:0/1434353313 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.844+0000 7f6d65000640 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c8254f1b0 0x558c828ff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.845+0000 7f6d5f400640 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c82904490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.845+0000 7f6d5fe00640 1 -- 172.21.15.90:0/1434353313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 msgr2=0x558c82904490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.845+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c82904490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.845+0000 7f6d5fe00640 1 -- 172.21.15.90:0/1434353313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c8254f1b0 msgr2=0x558c828ff3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.845+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c8254f1b0 0x558c828ff3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.845+0000 7f6d5fe00640 1 -- 172.21.15.90:0/1434353313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c828ffaa0 con 0x558c825262f0 2024-05-27T07:48:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.845+0000 7f6d5fe00640 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c82903f50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6d50006010 tx=0x7f6d500052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebca4d16adf3a3d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.845+0000 7f6d4a000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.846+0000 7f6d4a000640 1 -- 172.21.15.90:0/1434353313 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50050070 con 0x558c825262f0 2024-05-27T07:48:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.846+0000 7f6d6e07c740 1 -- 172.21.15.90:0/1434353313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d50048070 con 0x558c825262f0 2024-05-27T07:48:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.846+0000 7f6d4a000640 1 -- 172.21.15.90:0/1434353313 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f6d500048c0 con 0x558c825262f0 2024-05-27T07:48:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.846+0000 7f6d4a000640 1 -- 172.21.15.90:0/1434353313 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6d50035070 con 0x558c825262f0 2024-05-27T07:48:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.846+0000 7f6d6e07c740 10 client.15957 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.846+0000 7f6d6e07c740 1 -- 172.21.15.90:0/1434353313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558c82975ff0 con 0x558c825262f0 2024-05-27T07:48:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.846+0000 7f6d4a000640 1 -- 172.21.15.90:0/1434353313 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d50048070 con 0x558c825262f0 2024-05-27T07:48:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.847+0000 7f6d4a000640 1 -- 172.21.15.90:0/1434353313 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f6d50043380 con 0x558c825262f0 2024-05-27T07:48:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.847+0000 7f6d6e07c740 10 client.15957 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.847+0000 7f6d6e07c740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:48:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.847+0000 7f6d6e07c740 20 client.15957 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.847+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558c824e31c0 0x558c829787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.847+0000 7f6d6e07c740 4 client.15957 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:48:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.847+0000 7f6d6e07c740 1 -- 172.21.15.90:0/1434353313 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c8285e2c0 con 0x558c824e31c0 2024-05-27T07:48:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.848+0000 7f6d5f400640 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558c824e31c0 0x558c829787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.849+0000 7f6d5f400640 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558c824e31c0 0x558c829787b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4865328ab652db1 server_cookie=85ed19a98aad7c6a in_seq=0 out_seq=0 2024-05-27T07:48:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.849+0000 7f6d4a000640 10 client.15957 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d4a000640 1 -- 172.21.15.90:0/1434353313 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558c8285e2c0 con 0x558c824e31c0 2024-05-27T07:48:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d4a000640 10 client.15957 handle_command_reply: tid=0 2024-05-27T07:48:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 client.15957 shutdown 2024-05-27T07:48:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 -- 172.21.15.90:0/1434353313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 msgr2=0x558c82903f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c82903f50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6d50006010 tx=0x7f6d500052b0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 -- 172.21.15.90:0/1434353313 shutdown_connections 2024-05-27T07:48:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558c824e31c0 0x558c829787b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558c8254f1b0 0x558c828ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d4a000640 0 client.15957 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:48:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558c8252e4d0 0x558c82904490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 --2- 172.21.15.90:0/1434353313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558c825262f0 0x558c82903f50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 -- 172.21.15.90:0/1434353313 >> 172.21.15.90:0/1434353313 conn(0x558c82543300 msgr2=0x558c828fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 -- 172.21.15.90:0/1434353313 shutdown_connections 2024-05-27T07:48:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.882+0000 7f6d6e07c740 1 -- 172.21.15.90:0/1434353313 wait complete. 2024-05-27T07:48:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.883+0000 7f6d6e07c740 20 client.15957 trim_cache size 0 max 16384 2024-05-27T07:48:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:48:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:48:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:48:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.883+0000 7f6d65a00640 1 -- 172.21.15.90:0/1082059197 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6d440b29d0 msgr2=0x7f6d440b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.883+0000 7f6d65a00640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6d440b29d0 0x7f6d440b4ee0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x558c828eebd0 tx=0x7f6d58009a60 comp rx=0 tx=0).stop 2024-05-27T07:48:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.883+0000 7f6d65a00640 1 -- 172.21.15.90:0/1082059197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d6010b610 msgr2=0x7f6d60102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.883+0000 7f6d65a00640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d6010b610 0x7f6d60102470 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00d950 tx=0x7f6d4c00de10 comp rx=0 tx=0).stop 2024-05-27T07:48:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.884+0000 7f6d65a00640 1 -- 172.21.15.90:0/1082059197 shutdown_connections 2024-05-27T07:48:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.884+0000 7f6d65a00640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6d440b29d0 0x7f6d440b4ee0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.884+0000 7f6d65a00640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d6010e000 0x7f6d60102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.884+0000 7f6d65a00640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d6010b610 0x7f6d60102470 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.884+0000 7f6d65a00640 1 --2- 172.21.15.90:0/1082059197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d60104bb0 0x7f6d60104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.884+0000 7f6d65a00640 1 -- 172.21.15.90:0/1082059197 >> 172.21.15.90:0/1082059197 conn(0x7f6d600fe210 msgr2=0x7f6d6010e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.884+0000 7f6d65a00640 1 -- 172.21.15.90:0/1082059197 shutdown_connections 2024-05-27T07:48:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:48:49.884+0000 7f6d65a00640 1 -- 172.21.15.90:0/1082059197 wait complete. 2024-05-27T07:48:49.995 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:50.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:50 smithi090 ceph-mon[31429]: pgmap v77: 129 pgs: 129 active+clean; 544 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 31 op/s 2024-05-27T07:48:50.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:50 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:48:50.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:50 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:48:50.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:50 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:48:50.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:50 smithi136 ceph-mon[35324]: pgmap v77: 129 pgs: 129 active+clean; 544 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 31 op/s 2024-05-27T07:48:50.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:50 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:48:50.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:50 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:48:50.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:50 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:48:50.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:50 smithi129 ceph-mon[35254]: pgmap v77: 129 pgs: 129 active+clean; 544 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 31 op/s 2024-05-27T07:48:50.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:50 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:48:50.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:50 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:48:50.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:50 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:48:52.257 INFO:teuthology.orchestra.run.smithi090.stderr:2200000 of 3200000 tuples (68%) done (elapsed 68.31 s, remaining 31.05 s) 2024-05-27T07:48:52.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.298+0000 7fe2e3800700 1 -- 172.21.15.90:0/2595173705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2dc109860 msgr2=0x7fe2dc1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.298+0000 7fe2e3800700 1 --2- 172.21.15.90:0/2595173705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2dc109860 0x7fe2dc1111c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00b0d0 tx=0x7fe2cc00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.298+0000 7fe2e3800700 1 -- 172.21.15.90:0/2595173705 shutdown_connections 2024-05-27T07:48:52.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.298+0000 7fe2e3800700 1 --2- 172.21.15.90:0/2595173705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2dc109860 0x7fe2dc1111c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.298+0000 7fe2e3800700 1 --2- 172.21.15.90:0/2595173705 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2dc1045e0 0x7fe2dc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.298+0000 7fe2e3800700 1 --2- 172.21.15.90:0/2595173705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2dc108f10 0x7fe2dc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.298+0000 7fe2e3800700 1 -- 172.21.15.90:0/2595173705 >> 172.21.15.90:0/2595173705 conn(0x7fe2dc100030 msgr2=0x7fe2dc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:52 smithi090 ceph-mon[31429]: pgmap v78: 129 pgs: 129 active+clean; 544 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 17 op/s 2024-05-27T07:48:52.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.299+0000 7fe2e3800700 1 -- 172.21.15.90:0/2595173705 shutdown_connections 2024-05-27T07:48:52.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.299+0000 7fe2e3800700 1 -- 172.21.15.90:0/2595173705 wait complete. 2024-05-27T07:48:52.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.301+0000 7fe2e3800700 1 Processor -- start 2024-05-27T07:48:52.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.301+0000 7fe2e3800700 1 -- start start 2024-05-27T07:48:52.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.301+0000 7fe2e3800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2dc1045e0 0x7fe2dc19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.301+0000 7fe2e3800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2dc108f10 0x7fe2dc19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.302+0000 7fe2e3800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2dc109860 0x7fe2dc1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.302+0000 7fe2e3800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc114390 con 0x7fe2dc109860 2024-05-27T07:48:52.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.302+0000 7fe2e3800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc114210 con 0x7fe2dc108f10 2024-05-27T07:48:52.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.302+0000 7fe2e3800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc114510 con 0x7fe2dc1045e0 2024-05-27T07:48:52.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.302+0000 7fe2dbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2dc108f10 0x7fe2dc19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.302+0000 7fe2dbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2dc108f10 0x7fe2dc19d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45644/0 (socket says 172.21.15.90:45644) 2024-05-27T07:48:52.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.302+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1312109977 learned_addr learned my addr 172.21.15.90:0/1312109977 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:52.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.303+0000 7fe2e0e00700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2dc1045e0 0x7fe2dc19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.303+0000 7fe2e1800700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2dc109860 0x7fe2dc1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.303+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1312109977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2dc1045e0 msgr2=0x7fe2dc19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.303+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2dc1045e0 0x7fe2dc19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.303+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2dc109860 msgr2=0x7fe2dc1a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.303+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2dc109860 0x7fe2dc1a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.303+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1312109977 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc1a1e30 con 0x7fe2dc108f10 2024-05-27T07:48:52.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.303+0000 7fe2e1800700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2dc109860 0x7fe2dc1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:48:52.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.304+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2dc108f10 0x7fe2dc19d500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400d950 tx=0x7fe2c400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.304+0000 7fe2d9600700 1 -- 172.21.15.90:0/1312109977 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4016070 con 0x7fe2dc108f10 2024-05-27T07:48:52.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.304+0000 7fe2e3800700 1 -- 172.21.15.90:0/1312109977 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2dc1a2120 con 0x7fe2dc108f10 2024-05-27T07:48:52.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.304+0000 7fe2d9600700 1 -- 172.21.15.90:0/1312109977 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2c4004830 con 0x7fe2dc108f10 2024-05-27T07:48:52.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.304+0000 7fe2d9600700 1 -- 172.21.15.90:0/1312109977 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40053c0 con 0x7fe2dc108f10 2024-05-27T07:48:52.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.304+0000 7fe2e3800700 1 -- 172.21.15.90:0/1312109977 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2dc1a9a40 con 0x7fe2dc108f10 2024-05-27T07:48:52.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.310+0000 7fe2f2402b80 1 Processor -- start 2024-05-27T07:48:52.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.311+0000 7fe2f2402b80 1 -- start start 2024-05-27T07:48:52.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.311+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a6346577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.311+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a634651870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.311+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a6346521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.311+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a63457fe40 con 0x55a634651db0 2024-05-27T07:48:52.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.311+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a63456f9e0 con 0x55a6344d2e40 2024-05-27T07:48:52.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.311+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a634573a20 con 0x55a634657d30 2024-05-27T07:48:52.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.311+0000 7fe2d9600700 1 -- 172.21.15.90:0/1312109977 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe2c4004220 con 0x7fe2dc108f10 2024-05-27T07:48:52.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.312+0000 7fe2e1800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a6346577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.312+0000 7fe2e0e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a634651870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.312+0000 7fe2e1800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a6346577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45646/0 (socket says 172.21.15.90:45646) 2024-05-27T07:48:52.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.312+0000 7fe2e1800700 1 -- 172.21.15.90:0/4170741766 learned_addr learned my addr 172.21.15.90:0/4170741766 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:52.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.312+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/4170741766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a6346521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.313+0000 7fe2e1800700 1 -- 172.21.15.90:0/4170741766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 msgr2=0x55a634651870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.313+0000 7fe2e1800700 1 --2- 172.21.15.90:0/4170741766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a634651870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.313+0000 7fe2e1800700 1 -- 172.21.15.90:0/4170741766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 msgr2=0x55a6346521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.313+0000 7fe2e1800700 1 --2- 172.21.15.90:0/4170741766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a6346521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.313+0000 7fe2e1800700 1 -- 172.21.15.90:0/4170741766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a634538420 con 0x55a6344d2e40 2024-05-27T07:48:52.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.313+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/4170741766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a6346521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:48:52.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.313+0000 7fe2d9600700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2c00b2a70 0x7fe2c00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.313+0000 7fe2d9600700 1 -- 172.21.15.90:0/1312109977 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe2c40e2590 con 0x7fe2dc108f10 2024-05-27T07:48:52.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.313+0000 7fe2e1800700 1 --2- 172.21.15.90:0/4170741766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a6346577f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009c30 tx=0x7fe2cc009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c07719d094a67dae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.314+0000 7fe2e0e00700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2c00b2a70 0x7fe2c00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.314+0000 7fe2d2000700 1 -- 172.21.15.90:0/4170741766 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc003c00 con 0x55a6344d2e40 2024-05-27T07:48:52.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.314+0000 7fe2d2000700 1 -- 172.21.15.90:0/4170741766 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2cc003da0 con 0x55a6344d2e40 2024-05-27T07:48:52.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.314+0000 7fe2d2000700 1 -- 172.21.15.90:0/4170741766 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc0054a0 con 0x55a6344d2e40 2024-05-27T07:48:52.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.315+0000 7fe2f2402b80 1 -- 172.21.15.90:0/4170741766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 msgr2=0x55a6346577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.315+0000 7fe2e0e00700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2c00b2a70 0x7fe2c00b4f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005950 tx=0x7fe2d400b410 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.315+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/4170741766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a6346577f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009c30 tx=0x7fe2cc009c60 comp rx=0 tx=0).stop 2024-05-27T07:48:52.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.315+0000 7fe2f2402b80 1 -- 172.21.15.90:0/4170741766 shutdown_connections 2024-05-27T07:48:52.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.315+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/4170741766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a6346521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.315+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/4170741766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a6346577f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.315+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/4170741766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a634651870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.315+0000 7fe2f2402b80 1 -- 172.21.15.90:0/4170741766 >> 172.21.15.90:0/4170741766 conn(0x55a634532fc0 msgr2=0x55a63464c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.316+0000 7fe2f2402b80 1 -- 172.21.15.90:0/4170741766 shutdown_connections 2024-05-27T07:48:52.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.316+0000 7fe2f2402b80 1 -- 172.21.15.90:0/4170741766 wait complete. 2024-05-27T07:48:52.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.316+0000 7fe2f2402b80 1 -- start start 2024-05-27T07:48:52.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.317+0000 7fe2f2402b80 1 Processor -- start 2024-05-27T07:48:52.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.317+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a63465b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.317+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.317+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a63465bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.317+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a634475fa0 con 0x55a634651db0 2024-05-27T07:48:52.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.317+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a634510760 con 0x55a6344d2e40 2024-05-27T07:48:52.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.317+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a63465e7a0 con 0x55a634657d30 2024-05-27T07:48:52.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.317+0000 7fe2f2402b80 10 client.0 _ll_register_callbacks cb 0x55a634519c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:52.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.318+0000 7fe2dbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.318+0000 7fe2dbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41432/0 (socket says 172.21.15.90:41432) 2024-05-27T07:48:52.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.318+0000 7fe2dbe00700 1 -- 172.21.15.90:0/2940407682 learned_addr learned my addr 172.21.15.90:0/2940407682 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:52.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.318+0000 7fe2e0e00700 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a63465b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.318+0000 7fe2dbe00700 1 -- 172.21.15.90:0/2940407682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 msgr2=0x55a63465bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.318+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a63465bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.318+0000 7fe2dbe00700 1 -- 172.21.15.90:0/2940407682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 msgr2=0x55a63465b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.318+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a63465b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.318+0000 7fe2dbe00700 1 -- 172.21.15.90:0/2940407682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6344bf8d0 con 0x55a634651db0 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.319+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465baa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4007660 tx=0x7fe2c40055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1554968473b2955 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.319+0000 7fe2caa00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.319+0000 7fe2caa00700 1 -- 172.21.15.90:0/2940407682 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40a8c70 con 0x55a634651db0 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.319+0000 7fe2caa00700 1 -- 172.21.15.90:0/2940407682 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe2c40049e0 con 0x55a634651db0 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.319+0000 7fe2f2402b80 1 -- 172.21.15.90:0/2940407682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2cc003da0 con 0x55a634651db0 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.319+0000 7fe2caa00700 1 -- 172.21.15.90:0/2940407682 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe2c4004510 con 0x55a634651db0 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.319+0000 7fe2f2402b80 10 client.15969 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.319+0000 7fe2f2402b80 1 -- 172.21.15.90:0/2940407682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a634540060 con 0x55a634651db0 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.320+0000 7fe2caa00700 1 -- 172.21.15.90:0/2940407682 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2cc003da0 con 0x55a634651db0 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.320+0000 7fe2caa00700 1 -- 172.21.15.90:0/2940407682 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fe2c40e21c0 con 0x55a634651db0 2024-05-27T07:48:52.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.320+0000 7fe2f2402b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:52.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.320+0000 7fe2f2402b80 10 client.15969 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:48:52.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.320+0000 7fe2f2402b80 20 client.15969 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:52.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.320+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a634673f50 0x55a6346763b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.320+0000 7fe2f2402b80 4 client.15969 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:52.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.320+0000 7fe2f2402b80 1 -- 172.21.15.90:0/2940407682 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a634581d90 con 0x55a634673f50 2024-05-27T07:48:52.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.321+0000 7fe2e0e00700 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a634673f50 0x55a6346763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.322+0000 7fe2e0e00700 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a634673f50 0x55a6346763b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b233821cde7b375d server_cookie=f25491b05303393a in_seq=0 out_seq=0 2024-05-27T07:48:52.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.322+0000 7fe2caa00700 10 client.15969 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:48:52.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.324+0000 7fe2caa00700 1 -- 172.21.15.90:0/2940407682 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a634581d90 con 0x55a634673f50 2024-05-27T07:48:52.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.324+0000 7fe2caa00700 10 client.15969 handle_command_reply: tid=0 2024-05-27T07:48:52.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.324+0000 7fe2f2402b80 1 client.15969 shutdown 2024-05-27T07:48:52.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.324+0000 7fe2f2402b80 1 -- 172.21.15.90:0/2940407682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 msgr2=0x55a63465baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.324+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465baa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4007660 tx=0x7fe2c40055e0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.324+0000 7fe2f2402b80 1 -- 172.21.15.90:0/2940407682 shutdown_connections 2024-05-27T07:48:52.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.324+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a634673f50 0x55a6346763b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.325+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a63465bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.325+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a63465b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.325+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/2940407682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465baa0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.325+0000 7fe2caa00700 0 client.15969 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:48:52.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.325+0000 7fe2f2402b80 1 -- 172.21.15.90:0/2940407682 >> 172.21.15.90:0/2940407682 conn(0x55a634532fc0 msgr2=0x55a63464c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.325+0000 7fe2f2402b80 1 -- 172.21.15.90:0/2940407682 shutdown_connections 2024-05-27T07:48:52.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.325+0000 7fe2f2402b80 1 -- 172.21.15.90:0/2940407682 wait complete. 2024-05-27T07:48:52.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.325+0000 7fe2f2402b80 20 client.15969 trim_cache size 0 max 16384 2024-05-27T07:48:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.347+0000 7fe2f2402b80 1 Processor -- start 2024-05-27T07:48:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.347+0000 7fe2f2402b80 1 -- start start 2024-05-27T07:48:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.347+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a634651870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.348+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.348+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a634655920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.348+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1ae140 con 0x55a634651db0 2024-05-27T07:48:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.348+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1afc80 con 0x55a6344d2e40 2024-05-27T07:48:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.348+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1ae810 con 0x55a634657d30 2024-05-27T07:48:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.348+0000 7fe2dbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.348+0000 7fe2e0e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a634651870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.348+0000 7fe2dbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41436/0 (socket says 172.21.15.90:41436) 2024-05-27T07:48:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.348+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1450721500 learned_addr learned my addr 172.21.15.90:0/1450721500 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.348+0000 7fe2e1800700 1 --2- 172.21.15.90:0/1450721500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a634655920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.349+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1450721500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 msgr2=0x55a634655920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.349+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/1450721500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a634655920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.349+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1450721500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 msgr2=0x55a634651870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.349+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/1450721500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a634651870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.349+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1450721500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63465c350 con 0x55a634651db0 2024-05-27T07:48:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.349+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/1450721500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465da00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40b24a0 tx=0x7fe2c40a92d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1055c0b69a9c3f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.349+0000 7fe2d1600700 1 -- 172.21.15.90:0/1450721500 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40a8c70 con 0x55a634651db0 2024-05-27T07:48:52.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.350+0000 7fe2d1600700 1 -- 172.21.15.90:0/1450721500 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2c40a9010 con 0x55a634651db0 2024-05-27T07:48:52.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.350+0000 7fe2d1600700 1 -- 172.21.15.90:0/1450721500 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40a9760 con 0x55a634651db0 2024-05-27T07:48:52.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.351+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1450721500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 msgr2=0x55a63465da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.351+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/1450721500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465da00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40b24a0 tx=0x7fe2c40a92d0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.351+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1450721500 shutdown_connections 2024-05-27T07:48:52.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.351+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/1450721500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a634655920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.351+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/1450721500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a634651870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.351+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/1450721500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63465da00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.351+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1450721500 >> 172.21.15.90:0/1450721500 conn(0x55a634532fc0 msgr2=0x55a634655210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.351+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1450721500 shutdown_connections 2024-05-27T07:48:52.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.351+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1450721500 wait complete. 2024-05-27T07:48:52.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.352+0000 7fe2f2402b80 1 -- start start 2024-05-27T07:48:52.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.352+0000 7fe2f2402b80 1 Processor -- start 2024-05-27T07:48:52.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.352+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a63466c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.352+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63467b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.352+0000 7fe2f2402b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a63467b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.352+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1af5b0 con 0x55a634651db0 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.352+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1aeee0 con 0x55a6344d2e40 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.352+0000 7fe2f2402b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1ac0c0 con 0x55a634657d30 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.353+0000 7fe2f2402b80 10 client.0 _ll_register_callbacks cb 0x55a634519c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.353+0000 7fe2dbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63467b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.353+0000 7fe2dbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63467b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41452/0 (socket says 172.21.15.90:41452) 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.353+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1570148726 learned_addr learned my addr 172.21.15.90:0/1570148726 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.353+0000 7fe2e0e00700 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a63466c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.353+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1570148726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 msgr2=0x55a63467b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.353+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a63467b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.353+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1570148726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 msgr2=0x55a63466c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:52.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.353+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a63466c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.353+0000 7fe2dbe00700 1 -- 172.21.15.90:0/1570148726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63467bbc0 con 0x55a634651db0 2024-05-27T07:48:52.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.354+0000 7fe2dbe00700 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63467b140 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40e5e20 tx=0x7fe2c40a8da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bba0d3338c06b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:52.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.354+0000 7fe2caa00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:52.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.354+0000 7fe2caa00700 1 -- 172.21.15.90:0/1570148726 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40049e0 con 0x55a634651db0 2024-05-27T07:48:52.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.354+0000 7fe2caa00700 1 -- 172.21.15.90:0/1570148726 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7fe2c40a9760 con 0x55a634651db0 2024-05-27T07:48:52.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.354+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1570148726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c40a9010 con 0x55a634651db0 2024-05-27T07:48:52.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.354+0000 7fe2caa00700 1 -- 172.21.15.90:0/1570148726 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe2c40a99a0 con 0x55a634651db0 2024-05-27T07:48:52.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.354+0000 7fe2f2402b80 10 client.15975 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:52.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.354+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1570148726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a6346de680 con 0x55a634651db0 2024-05-27T07:48:52.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.355+0000 7fe2caa00700 1 -- 172.21.15.90:0/1570148726 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2c40a9010 con 0x55a634651db0 2024-05-27T07:48:52.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.355+0000 7fe2caa00700 1 -- 172.21.15.90:0/1570148726 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fe2c40ca070 con 0x55a634651db0 2024-05-27T07:48:52.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.355+0000 7fe2f2402b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:52.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.355+0000 7fe2f2402b80 10 client.15975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:48:52.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.355+0000 7fe2f2402b80 20 client.15975 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:52.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.355+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a6346dea90 0x55a6346e0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:52.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.355+0000 7fe2f2402b80 4 client.15975 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:52.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.355+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1570148726 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a63457b570 con 0x55a6346dea90 2024-05-27T07:48:52.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.356+0000 7fe2e0e00700 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a6346dea90 0x55a6346e0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:52.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.357+0000 7fe2e0e00700 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a6346dea90 0x55a6346e0eb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4d8933a79f434b1 server_cookie=5bcaf4b1e3e33635 in_seq=0 out_seq=0 2024-05-27T07:48:52.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.357+0000 7fe2caa00700 10 client.15975 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:48:52.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.357+0000 7fe2caa00700 1 -- 172.21.15.90:0/1570148726 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a63457b570 con 0x55a6346dea90 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.357+0000 7fe2caa00700 10 client.15975 handle_command_reply: tid=0 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.357+0000 7fe2f2402b80 1 client.15975 shutdown 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1570148726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 msgr2=0x55a63467b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63467b140 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40e5e20 tx=0x7fe2c40a8da0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1570148726 shutdown_connections 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a6346dea90 0x55a6346e0eb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a634657d30 0x55a63467b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a6344d2e40 0x55a63466c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 1 --2- 172.21.15.90:0/1570148726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a634651db0 0x55a63467b140 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1570148726 >> 172.21.15.90:0/1570148726 conn(0x55a634532fc0 msgr2=0x55a63465b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2caa00700 0 client.15975 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1570148726 shutdown_connections 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 1 -- 172.21.15.90:0/1570148726 wait complete. 2024-05-27T07:48:52.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.358+0000 7fe2f2402b80 20 client.15975 trim_cache size 0 max 16384 2024-05-27T07:48:52.360 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:48:52.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 -- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2c00b2a70 msgr2=0x7fe2c00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2c00b2a70 0x7fe2c00b4f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005950 tx=0x7fe2d400b410 comp rx=0 tx=0).stop 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 -- 172.21.15.90:0/1312109977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2dc108f10 msgr2=0x7fe2dc19d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2dc108f10 0x7fe2dc19d500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400d950 tx=0x7fe2c400dc60 comp rx=0 tx=0).stop 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 -- 172.21.15.90:0/1312109977 shutdown_connections 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2dc109860 0x7fe2dc1a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2dc1045e0 0x7fe2dc19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2c00b2a70 0x7fe2c00b4f80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 --2- 172.21.15.90:0/1312109977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2dc108f10 0x7fe2dc19d500 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 -- 172.21.15.90:0/1312109977 >> 172.21.15.90:0/1312109977 conn(0x7fe2dc100030 msgr2=0x7fe2dc110e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 -- 172.21.15.90:0/1312109977 shutdown_connections 2024-05-27T07:48:52.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:52.359+0000 7fe2e3800700 1 -- 172.21.15.90:0/1312109977 wait complete. 2024-05-27T07:48:52.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:52 smithi136 ceph-mon[35324]: pgmap v78: 129 pgs: 129 active+clean; 544 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 17 op/s 2024-05-27T07:48:52.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:52 smithi129 ceph-mon[35254]: pgmap v78: 129 pgs: 129 active+clean; 544 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 17 op/s 2024-05-27T07:48:53.318 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:48:53.792 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:54.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:54 smithi090 ceph-mon[31429]: pgmap v79: 129 pgs: 129 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 31 op/s 2024-05-27T07:48:54.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:54 smithi136 ceph-mon[35324]: pgmap v79: 129 pgs: 129 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 31 op/s 2024-05-27T07:48:54.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:54 smithi129 ceph-mon[35254]: pgmap v79: 129 pgs: 129 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 MiB/s wr, 31 op/s 2024-05-27T07:48:55.255 INFO:teuthology.orchestra.run.smithi090.stderr:2300000 of 3200000 tuples (71%) done (elapsed 71.31 s, remaining 27.90 s) 2024-05-27T07:48:56.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:56 smithi090 ceph-mon[31429]: pgmap v80: 129 pgs: 129 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 23 op/s 2024-05-27T07:48:56.104 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:55 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 4. 2024-05-27T07:48:56.104 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:55 smithi136 systemd[1]: Stopped Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:48:56.104 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:55 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.538s CPU time. 2024-05-27T07:48:56.104 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:55 smithi136 systemd[1]: Starting Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:48:56.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.110+0000 7f9b6d200700 1 -- 172.21.15.90:0/4230945947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b68108f10 msgr2=0x7f9b68109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:56.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.110+0000 7f9b6d200700 1 --2- 172.21.15.90:0/4230945947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b68108f10 0x7f9b68109320 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9b540099c0 tx=0x7f9b54009cd0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.110+0000 7f9b6d200700 1 -- 172.21.15.90:0/4230945947 shutdown_connections 2024-05-27T07:48:56.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.110+0000 7f9b6d200700 1 --2- 172.21.15.90:0/4230945947 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b68109860 0x7f9b681111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.110+0000 7f9b6d200700 1 --2- 172.21.15.90:0/4230945947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b681045e0 0x7f9b681049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.110+0000 7f9b6d200700 1 --2- 172.21.15.90:0/4230945947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b68108f10 0x7f9b68109320 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.110+0000 7f9b6d200700 1 -- 172.21.15.90:0/4230945947 >> 172.21.15.90:0/4230945947 conn(0x7f9b68100030 msgr2=0x7f9b68102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:56.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.110+0000 7f9b6d200700 1 -- 172.21.15.90:0/4230945947 shutdown_connections 2024-05-27T07:48:56.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.110+0000 7f9b6d200700 1 -- 172.21.15.90:0/4230945947 wait complete. 2024-05-27T07:48:56.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.111+0000 7f9b6d200700 1 Processor -- start 2024-05-27T07:48:56.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.111+0000 7f9b6d200700 1 -- start start 2024-05-27T07:48:56.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b6d200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b681045e0 0x7f9b681a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b6d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b68108f10 0x7f9b681a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b6d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b68109860 0x7f9b6819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b6d200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9b680762e0 con 0x7f9b68108f10 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b6d200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b68076160 con 0x7f9b681045e0 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b6d200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9b68076460 con 0x7f9b68109860 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b65400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b68108f10 0x7f9b681a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b65400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b68108f10 0x7f9b681a3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41476/0 (socket says 172.21.15.90:41476) 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b65400700 1 -- 172.21.15.90:0/2349418990 learned_addr learned my addr 172.21.15.90:0/2349418990 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b66800700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b68109860 0x7f9b6819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.112+0000 7f9b65e00700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b681045e0 0x7f9b681a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.113+0000 7f9b65400700 1 -- 172.21.15.90:0/2349418990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b68109860 msgr2=0x7f9b6819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.113+0000 7f9b65400700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b68109860 0x7f9b6819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.113+0000 7f9b65400700 1 -- 172.21.15.90:0/2349418990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b681045e0 msgr2=0x7f9b681a2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:56.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.113+0000 7f9b65400700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b681045e0 0x7f9b681a2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.113+0000 7f9b65400700 1 -- 172.21.15.90:0/2349418990 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6819d530 con 0x7f9b68108f10 2024-05-27T07:48:56.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.113+0000 7f9b65400700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b68108f10 0x7f9b681a3150 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9b54006010 tx=0x7f9b540045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:56.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.114+0000 7f9b5ea00700 1 -- 172.21.15.90:0/2349418990 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54009e60 con 0x7f9b68108f10 2024-05-27T07:48:56.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.114+0000 7f9b6d200700 1 -- 172.21.15.90:0/2349418990 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6819d7c0 con 0x7f9b68108f10 2024-05-27T07:48:56.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.114+0000 7f9b5ea00700 1 -- 172.21.15.90:0/2349418990 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b54003d20 con 0x7f9b68108f10 2024-05-27T07:48:56.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.114+0000 7f9b5ea00700 1 -- 172.21.15.90:0/2349418990 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54005290 con 0x7f9b68108f10 2024-05-27T07:48:56.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.114+0000 7f9b6d200700 1 -- 172.21.15.90:0/2349418990 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b681a99c0 con 0x7f9b68108f10 2024-05-27T07:48:56.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.118+0000 7f9b5ea00700 1 -- 172.21.15.90:0/2349418990 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9b54004800 con 0x7f9b68108f10 2024-05-27T07:48:56.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.119+0000 7f9b7bdaab80 1 Processor -- start 2024-05-27T07:48:56.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.120+0000 7f9b5ea00700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9b4c0b29f0 0x7f9b4c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.120+0000 7f9b7bdaab80 1 -- start start 2024-05-27T07:48:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.120+0000 7f9b5ea00700 1 -- 172.21.15.90:0/2349418990 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b540e3780 con 0x7f9b68108f10 2024-05-27T07:48:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.120+0000 7f9b66800700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9b4c0b29f0 0x7f9b4c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.120+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb596f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.120+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb5969870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.120+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb596a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.120+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564bb5897e40 con 0x564bb5969db0 2024-05-27T07:48:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.120+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564bb58879e0 con 0x564bb596fd30 2024-05-27T07:48:56.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.120+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564bb588ba20 con 0x564bb57eae40 2024-05-27T07:48:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b65e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb596f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b65e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb596f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53030/0 (socket says 172.21.15.90:53030) 2024-05-27T07:48:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b65e00700 1 -- 172.21.15.90:0/269528933 learned_addr learned my addr 172.21.15.90:0/269528933 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b66800700 1 --2- 172.21.15.90:0/269528933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb596a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b65400700 1 --2- 172.21.15.90:0/269528933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb5969870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b65e00700 1 -- 172.21.15.90:0/269528933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 msgr2=0x564bb5969870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b65e00700 1 --2- 172.21.15.90:0/269528933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb5969870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b65e00700 1 -- 172.21.15.90:0/269528933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 msgr2=0x564bb596a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b65e00700 1 --2- 172.21.15.90:0/269528933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb596a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b65e00700 1 -- 172.21.15.90:0/269528933 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb5850420 con 0x564bb57eae40 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b66800700 1 --2- 172.21.15.90:0/269528933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb596a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.121+0000 7f9b66800700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9b4c0b29f0 0x7f9b4c0b4f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b600093f0 tx=0x7f9b60009cb0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.122+0000 7f9b65e00700 1 --2- 172.21.15.90:0/269528933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb596f7f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009cb0 tx=0x7f9b580113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3adffe73a44a4a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.122+0000 7f9b53400700 1 -- 172.21.15.90:0/269528933 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58011d10 con 0x564bb57eae40 2024-05-27T07:48:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.122+0000 7f9b53400700 1 -- 172.21.15.90:0/269528933 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b580128e0 con 0x564bb57eae40 2024-05-27T07:48:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.122+0000 7f9b53400700 1 -- 172.21.15.90:0/269528933 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5800f730 con 0x564bb57eae40 2024-05-27T07:48:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.123+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/269528933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 msgr2=0x564bb596f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.123+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/269528933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb596f7f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009cb0 tx=0x7f9b580113e0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.123+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/269528933 shutdown_connections 2024-05-27T07:48:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.123+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/269528933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb596a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.123+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/269528933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb596f7f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.123+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/269528933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb5969870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.123+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/269528933 >> 172.21.15.90:0/269528933 conn(0x564bb584afc0 msgr2=0x564bb59648d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.124+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/269528933 shutdown_connections 2024-05-27T07:48:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.124+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/269528933 wait complete. 2024-05-27T07:48:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.124+0000 7f9b7bdaab80 1 -- start start 2024-05-27T07:48:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.125+0000 7f9b7bdaab80 1 Processor -- start 2024-05-27T07:48:56.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.125+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb5973530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.125+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb5969db0 0x564bb5973a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.125+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb596fd30 0x564bb5973fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.125+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564bb578dfa0 con 0x564bb596fd30 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.125+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564bb5828760 con 0x564bb5969db0 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.125+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564bb5976800 con 0x564bb57eae40 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.125+0000 7f9b7bdaab80 10 client.0 _ll_register_callbacks cb 0x564bb5831c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.126+0000 7f9b66800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb596fd30 0x564bb5973fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.126+0000 7f9b65e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb5973530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.126+0000 7f9b65e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb5973530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53038/0 (socket says 172.21.15.90:53038) 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.126+0000 7f9b65e00700 1 -- 172.21.15.90:0/3708095653 learned_addr learned my addr 172.21.15.90:0/3708095653 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.126+0000 7f9b65400700 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb5969db0 0x564bb5973a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.127+0000 7f9b65e00700 1 -- 172.21.15.90:0/3708095653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb5969db0 msgr2=0x564bb5973a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:56.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.127+0000 7f9b65e00700 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb5969db0 0x564bb5973a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.127+0000 7f9b65e00700 1 -- 172.21.15.90:0/3708095653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb596fd30 msgr2=0x564bb5973fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:56.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.127+0000 7f9b65e00700 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb596fd30 0x564bb5973fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.127+0000 7f9b65e00700 1 -- 172.21.15.90:0/3708095653 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb57d78d0 con 0x564bb57eae40 2024-05-27T07:48:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.127+0000 7f9b65e00700 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb5973530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b580127c0 tx=0x7f9b580127f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46070d81da2805c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.127+0000 7f9b47e00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:48:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.128+0000 7f9b47e00700 1 -- 172.21.15.90:0/3708095653 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58013db0 con 0x564bb57eae40 2024-05-27T07:48:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.128+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/3708095653 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b580128e0 con 0x564bb57eae40 2024-05-27T07:48:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.128+0000 7f9b47e00700 1 -- 172.21.15.90:0/3708095653 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b58004050 con 0x564bb57eae40 2024-05-27T07:48:56.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.128+0000 7f9b47e00700 1 -- 172.21.15.90:0/3708095653 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9b58031050 con 0x564bb57eae40 2024-05-27T07:48:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.129+0000 7f9b7bdaab80 10 client.25618 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.129+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/3708095653 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564bb5858060 con 0x564bb57eae40 2024-05-27T07:48:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.129+0000 7f9b47e00700 1 -- 172.21.15.90:0/3708095653 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b580128e0 con 0x564bb57eae40 2024-05-27T07:48:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.130+0000 7f9b47e00700 1 -- 172.21.15.90:0/3708095653 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f9b58004870 con 0x564bb57eae40 2024-05-27T07:48:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.130+0000 7f9b7bdaab80 10 client.25618 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:56.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.130+0000 7f9b7bdaab80 10 client.25618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:48:56.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.130+0000 7f9b7bdaab80 20 client.25618 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:56.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.130+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x564bb59835c0 0x564bb59859e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.130+0000 7f9b7bdaab80 4 client.25618 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:48:56.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.130+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/3708095653 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bb5899d90 con 0x564bb59835c0 2024-05-27T07:48:56.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.131+0000 7f9b65400700 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x564bb59835c0 0x564bb59859e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.131+0000 7f9b65400700 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x564bb59835c0 0x564bb59859e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fa4a7ebe786b054 server_cookie=eea9b245d34b212e in_seq=0 out_seq=0 2024-05-27T07:48:56.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.131+0000 7f9b47e00700 10 client.25618 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:48:56.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.133+0000 7f9b47e00700 1 -- 172.21.15.90:0/3708095653 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564bb5899d90 con 0x564bb59835c0 2024-05-27T07:48:56.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b47e00700 10 client.25618 handle_command_reply: tid=0 2024-05-27T07:48:56.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 client.25618 shutdown 2024-05-27T07:48:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/3708095653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 msgr2=0x564bb5973530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb5973530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b580127c0 tx=0x7f9b580127f0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/3708095653 shutdown_connections 2024-05-27T07:48:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb596fd30 0x564bb5973fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb5973530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x564bb59835c0 0x564bb59859e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/3708095653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb5969db0 0x564bb5973a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b47e00700 0 client.25618 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:48:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/3708095653 >> 172.21.15.90:0/3708095653 conn(0x564bb584afc0 msgr2=0x564bb5964580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/3708095653 shutdown_connections 2024-05-27T07:48:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.134+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/3708095653 wait complete. 2024-05-27T07:48:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.135+0000 7f9b7bdaab80 20 client.25618 trim_cache size 0 max 16384 2024-05-27T07:48:56.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.159+0000 7f9b7bdaab80 1 Processor -- start 2024-05-27T07:48:56.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.159+0000 7f9b7bdaab80 1 -- start start 2024-05-27T07:48:56.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.159+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb5969870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.159+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb5975980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.159+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb597fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.159+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9b681b2680 con 0x564bb5969db0 2024-05-27T07:48:56.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.159+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b681b41c0 con 0x564bb596fd30 2024-05-27T07:48:56.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.159+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9b681b2d50 con 0x564bb57eae40 2024-05-27T07:48:56.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.160+0000 7f9b65400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb5975980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.160+0000 7f9b65400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb5975980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41510/0 (socket says 172.21.15.90:41510) 2024-05-27T07:48:56.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.160+0000 7f9b65400700 1 -- 172.21.15.90:0/1176398552 learned_addr learned my addr 172.21.15.90:0/1176398552 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:56.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.160+0000 7f9b65e00700 1 --2- 172.21.15.90:0/1176398552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb5969870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.161+0000 7f9b66800700 1 --2- 172.21.15.90:0/1176398552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb597fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.161+0000 7f9b65400700 1 -- 172.21.15.90:0/1176398552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 msgr2=0x564bb5969870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:56.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.161+0000 7f9b65400700 1 --2- 172.21.15.90:0/1176398552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb5969870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.161+0000 7f9b65400700 1 -- 172.21.15.90:0/1176398552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 msgr2=0x564bb597fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:56.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.161+0000 7f9b65400700 1 --2- 172.21.15.90:0/1176398552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb597fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.161+0000 7f9b65400700 1 -- 172.21.15.90:0/1176398552 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bb5960980 con 0x564bb5969db0 2024-05-27T07:48:56.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.161+0000 7f9b66800700 1 --2- 172.21.15.90:0/1176398552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb597fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:48:56.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.161+0000 7f9b65400700 1 --2- 172.21.15.90:0/1176398552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb5975980 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9b540aa440 tx=0x7f9b54092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b4c4d27efbb1f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:56.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.162+0000 7f9b52a00700 1 -- 172.21.15.90:0/1176398552 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540c4070 con 0x564bb5969db0 2024-05-27T07:48:56.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.162+0000 7f9b52a00700 1 -- 172.21.15.90:0/1176398552 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b54004e50 con 0x564bb5969db0 2024-05-27T07:48:56.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.162+0000 7f9b52a00700 1 -- 172.21.15.90:0/1176398552 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b540a4e60 con 0x564bb5969db0 2024-05-27T07:48:56.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.163+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/1176398552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 msgr2=0x564bb5975980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:56.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.163+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/1176398552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb5975980 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9b540aa440 tx=0x7f9b54092040 comp rx=0 tx=0).stop 2024-05-27T07:48:56.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.163+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/1176398552 shutdown_connections 2024-05-27T07:48:56.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.163+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/1176398552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb597fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.163+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/1176398552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb57eae40 0x564bb5969870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.163+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/1176398552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb5969db0 0x564bb5975980 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.163+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/1176398552 >> 172.21.15.90:0/1176398552 conn(0x564bb584afc0 msgr2=0x564bb5973310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:56.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.164+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/1176398552 shutdown_connections 2024-05-27T07:48:56.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.164+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/1176398552 wait complete. 2024-05-27T07:48:56.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.165+0000 7f9b7bdaab80 1 -- start start 2024-05-27T07:48:56.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.165+0000 7f9b7bdaab80 1 Processor -- start 2024-05-27T07:48:56.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb57eae40 0x564bb5988e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb59861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b7bdaab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb59866f0 0x564bb5986b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9b681b3af0 con 0x564bb57eae40 2024-05-27T07:48:56.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b681b3420 con 0x564bb596fd30 2024-05-27T07:48:56.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b7bdaab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9b681b0600 con 0x564bb59866f0 2024-05-27T07:48:56.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b7bdaab80 10 client.0 _ll_register_callbacks cb 0x564bb5831c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:48:56.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b65e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb57eae40 0x564bb5988e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b65e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb57eae40 0x564bb5988e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41522/0 (socket says 172.21.15.90:41522) 2024-05-27T07:48:56.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b65e00700 1 -- 172.21.15.90:0/816255569 learned_addr learned my addr 172.21.15.90:0/816255569 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:48:56.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.166+0000 7f9b65400700 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb59861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.167+0000 7f9b66800700 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb59866f0 0x564bb5986b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.167+0000 7f9b65e00700 1 -- 172.21.15.90:0/816255569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb59866f0 msgr2=0x564bb5986b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.167+0000 7f9b65e00700 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb59866f0 0x564bb5986b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.167+0000 7f9b65e00700 1 -- 172.21.15.90:0/816255569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 msgr2=0x564bb59861b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:48:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.167+0000 7f9b65e00700 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb59861b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.167+0000 7f9b65e00700 1 -- 172.21.15.90:0/816255569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bb59895f0 con 0x564bb57eae40 2024-05-27T07:48:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.167+0000 7f9b65e00700 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb57eae40 0x564bb5988e30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9b58012560 tx=0x7f9b58004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adfaa2f18be91fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:48:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.167+0000 7f9b47e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:48:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.168+0000 7f9b47e00700 1 -- 172.21.15.90:0/816255569 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58012c70 con 0x564bb57eae40 2024-05-27T07:48:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.168+0000 7f9b47e00700 1 -- 172.21.15.90:0/816255569 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9408+0+0 (secure 0 0 0) 0x7f9b58004ed0 con 0x564bb57eae40 2024-05-27T07:48:56.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.168+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/816255569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b54004e50 con 0x564bb57eae40 2024-05-27T07:48:56.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.168+0000 7f9b47e00700 1 -- 172.21.15.90:0/816255569 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9b58012e10 con 0x564bb57eae40 2024-05-27T07:48:56.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.168+0000 7f9b7bdaab80 10 client.15993 fetch_fsmap learned FSMap version 36 2024-05-27T07:48:56.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.168+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/816255569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564bb59ede70 con 0x564bb57eae40 2024-05-27T07:48:56.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.169+0000 7f9b47e00700 1 -- 172.21.15.90:0/816255569 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b54004e50 con 0x564bb57eae40 2024-05-27T07:48:56.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.169+0000 7f9b47e00700 1 -- 172.21.15.90:0/816255569 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f9b58013480 con 0x564bb57eae40 2024-05-27T07:48:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.169+0000 7f9b7bdaab80 10 client.15993 fetch_fsmap finished waiting for FSMap version 36 2024-05-27T07:48:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.169+0000 7f9b7bdaab80 10 client.15993 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:48:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.169+0000 7f9b7bdaab80 20 client.15993 populate_metadata read hostname 'smithi090' 2024-05-27T07:48:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.169+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x564bb59ee480 0x564bb59f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:48:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.169+0000 7f9b7bdaab80 4 client.15993 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:48:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.169+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/816255569 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bb5893570 con 0x564bb59ee480 2024-05-27T07:48:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.170+0000 7f9b65400700 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x564bb59ee480 0x564bb59f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:48:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.171+0000 7f9b47e00700 10 client.15993 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:48:56.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.171+0000 7f9b65400700 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x564bb59ee480 0x564bb59f08a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3904cc87f00d7519 server_cookie=9f11a9ec051192a in_seq=0 out_seq=0 2024-05-27T07:48:56.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b47e00700 1 -- 172.21.15.90:0/816255569 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bb5893570 con 0x564bb59ee480 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b47e00700 10 client.15993 handle_command_reply: tid=0 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b7bdaab80 1 client.15993 shutdown 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/816255569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb57eae40 msgr2=0x564bb5988e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb57eae40 0x564bb5988e30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9b58012560 tx=0x7f9b58004ad0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/816255569 shutdown_connections 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564bb59866f0 0x564bb5986b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564bb57eae40 0x564bb5988e30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x564bb59ee480 0x564bb59f08a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b7bdaab80 1 --2- 172.21.15.90:0/816255569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564bb596fd30 0x564bb59861b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b47e00700 0 client.15993 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/816255569 >> 172.21.15.90:0/816255569 conn(0x564bb584afc0 msgr2=0x564bb596d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.172+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/816255569 shutdown_connections 2024-05-27T07:48:56.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.173+0000 7f9b7bdaab80 1 -- 172.21.15.90:0/816255569 wait complete. 2024-05-27T07:48:56.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.173+0000 7f9b7bdaab80 20 client.15993 trim_cache size 0 max 16384 2024-05-27T07:48:56.175 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:48:56.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 -- 172.21.15.90:0/2349418990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9b4c0b29f0 msgr2=0x7f9b4c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:56.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9b4c0b29f0 0x7f9b4c0b4f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b600093f0 tx=0x7f9b60009cb0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 -- 172.21.15.90:0/2349418990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b68108f10 msgr2=0x7f9b681a3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:48:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b68108f10 0x7f9b681a3150 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9b54006010 tx=0x7f9b540045e0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 -- 172.21.15.90:0/2349418990 shutdown_connections 2024-05-27T07:48:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b68109860 0x7f9b6819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9b4c0b29f0 0x7f9b4c0b4f00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b681045e0 0x7f9b681a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 --2- 172.21.15.90:0/2349418990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b68108f10 0x7f9b681a3150 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:48:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 -- 172.21.15.90:0/2349418990 >> 172.21.15.90:0/2349418990 conn(0x7f9b68100030 msgr2=0x7f9b68102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:48:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 -- 172.21.15.90:0/2349418990 shutdown_connections 2024-05-27T07:48:56.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:48:56.174+0000 7f9b6d200700 1 -- 172.21.15.90:0/2349418990 wait complete. 2024-05-27T07:48:56.366 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:56 smithi129 ceph-mon[35254]: pgmap v80: 129 pgs: 129 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 23 op/s 2024-05-27T07:48:56.463 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:56 smithi136 ceph-mon[35324]: pgmap v80: 129 pgs: 129 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 23 op/s 2024-05-27T07:48:56.464 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:56 smithi136 podman[71085]: 2024-05-27 07:48:56.057041938 +0000 UTC m=+0.033486621 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:48:56.464 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:56 smithi136 podman[71085]: 2024-05-27 07:48:56.185113631 +0000 UTC m=+0.161558347 container create 8e007cf054497624966946e1fbe8724a67ee2c3ddb9e87eef3601540c0446be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-27T07:48:56.464 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:56 smithi136 podman[71085]: 2024-05-27 07:48:56.464077989 +0000 UTC m=+0.440522756 container init 8e007cf054497624966946e1fbe8724a67ee2c3ddb9e87eef3601540c0446be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-27T07:48:56.749 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:56 smithi136 podman[71085]: 2024-05-27 07:48:56.475202116 +0000 UTC m=+0.451646802 container start 8e007cf054497624966946e1fbe8724a67ee2c3ddb9e87eef3601540c0446be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-27T07:48:56.750 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:56 smithi136 bash[71085]: 8e007cf054497624966946e1fbe8724a67ee2c3ddb9e87eef3601540c0446be8 2024-05-27T07:48:56.750 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:56 smithi136 ceph-mds[71103]: -- 172.21.15.136:0/3522291907 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5603f100da40 con 0x5603f1055600 2024-05-27T07:48:56.750 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:56 smithi136 systemd[1]: Started Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:48:56.750 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:48:56 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: starting mds.c at 2024-05-27T07:48:57.150 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:48:57.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:57 smithi090 ceph-mon[31429]: daemon mds.c restarted 2024-05-27T07:48:57.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:57 smithi090 ceph-mon[31429]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T07:48:57.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:57 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.136:6836/3741952392,v1:172.21.15.136:6837/3741952392] up:boot 2024-05-27T07:48:57.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:57 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:48:57.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:57 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:48:57.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:57 smithi136 ceph-mon[35324]: daemon mds.c restarted 2024-05-27T07:48:57.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:57 smithi136 ceph-mon[35324]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T07:48:57.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:57 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.136:6836/3741952392,v1:172.21.15.136:6837/3741952392] up:boot 2024-05-27T07:48:57.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:57 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:48:57.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:57 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:48:57.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:57 smithi129 ceph-mon[35254]: daemon mds.c restarted 2024-05-27T07:48:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:57 smithi129 ceph-mon[35254]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T07:48:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:57 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.136:6836/3741952392,v1:172.21.15.136:6837/3741952392] up:boot 2024-05-27T07:48:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:57 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:48:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:57 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:48:57.627 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:48:58.356 INFO:teuthology.orchestra.run.smithi090.stderr:2400000 of 3200000 tuples (75%) done (elapsed 74.41 s, remaining 24.80 s) 2024-05-27T07:48:58.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:48:58 smithi090 ceph-mon[31429]: pgmap v82: 129 pgs: 129 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 28 op/s 2024-05-27T07:48:58.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:48:58 smithi136 ceph-mon[35324]: pgmap v82: 129 pgs: 129 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 28 op/s 2024-05-27T07:48:58.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:48:58 smithi129 ceph-mon[35254]: pgmap v82: 129 pgs: 129 active+clean; 582 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 28 op/s 2024-05-27T07:49:00.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.002+0000 7fd48a800700 1 -- 172.21.15.90:0/4231456608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd484107de0 msgr2=0x7fd4841081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:00.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.002+0000 7fd48a800700 1 --2- 172.21.15.90:0/4231456608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd484107de0 0x7fd4841081f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd4700099c0 tx=0x7fd470009cd0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.002+0000 7fd48a800700 1 -- 172.21.15.90:0/4231456608 shutdown_connections 2024-05-27T07:49:00.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.002+0000 7fd48a800700 1 --2- 172.21.15.90:0/4231456608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd484108730 0x7fd484110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.002+0000 7fd48a800700 1 --2- 172.21.15.90:0/4231456608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4841034b0 0x7fd4841038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.002+0000 7fd48a800700 1 --2- 172.21.15.90:0/4231456608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd484107de0 0x7fd4841081f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.002+0000 7fd48a800700 1 -- 172.21.15.90:0/4231456608 >> 172.21.15.90:0/4231456608 conn(0x7fd4840fedb0 msgr2=0x7fd4841011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:00.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.002+0000 7fd48a800700 1 -- 172.21.15.90:0/4231456608 shutdown_connections 2024-05-27T07:49:00.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.002+0000 7fd48a800700 1 -- 172.21.15.90:0/4231456608 wait complete. 2024-05-27T07:49:00.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.004+0000 7fd48a800700 1 Processor -- start 2024-05-27T07:49:00.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.004+0000 7fd48a800700 1 -- start start 2024-05-27T07:49:00.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.004+0000 7fd48a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4841034b0 0x7fd4840ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.004+0000 7fd48a800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd484107de0 0x7fd4840ff7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.004+0000 7fd48a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd484108730 0x7fd4840ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.004+0000 7fd48a800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841131f0 con 0x7fd4841034b0 2024-05-27T07:49:00.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.004+0000 7fd48a800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd484113070 con 0x7fd484107de0 2024-05-27T07:49:00.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.004+0000 7fd48a800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd484113370 con 0x7fd484108730 2024-05-27T07:49:00.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd483400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4841034b0 0x7fd4840ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd483400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4841034b0 0x7fd4840ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39176/0 (socket says 172.21.15.90:39176) 2024-05-27T07:49:00.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd483400700 1 -- 172.21.15.90:0/3382025937 learned_addr learned my addr 172.21.15.90:0/3382025937 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:00.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd482a00700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd484107de0 0x7fd4840ff7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd483e00700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd484108730 0x7fd4840ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd483400700 1 -- 172.21.15.90:0/3382025937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd484108730 msgr2=0x7fd4840ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:00.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd483400700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd484108730 0x7fd4840ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd483400700 1 -- 172.21.15.90:0/3382025937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd484107de0 msgr2=0x7fd4840ff7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:00.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd483400700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd484107de0 0x7fd4840ff7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd483400700 1 -- 172.21.15.90:0/3382025937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4841a9060 con 0x7fd4841034b0 2024-05-27T07:49:00.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.005+0000 7fd483e00700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd484108730 0x7fd4840ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:00.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.006+0000 7fd482a00700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd484107de0 0x7fd4840ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:00.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.006+0000 7fd483400700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4841034b0 0x7fd4840ff290 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd4740060b0 tx=0x7fd474007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:00.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.006+0000 7fd47be00700 1 -- 172.21.15.90:0/3382025937 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47401a070 con 0x7fd4841034b0 2024-05-27T07:49:00.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.006+0000 7fd47be00700 1 -- 172.21.15.90:0/3382025937 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd47400de20 con 0x7fd4841034b0 2024-05-27T07:49:00.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.006+0000 7fd48a800700 1 -- 172.21.15.90:0/3382025937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4841a9350 con 0x7fd4841034b0 2024-05-27T07:49:00.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.006+0000 7fd47be00700 1 -- 172.21.15.90:0/3382025937 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474014690 con 0x7fd4841034b0 2024-05-27T07:49:00.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.006+0000 7fd48a800700 1 -- 172.21.15.90:0/3382025937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4841a98b0 con 0x7fd4841034b0 2024-05-27T07:49:00.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.011+0000 7fd47be00700 1 -- 172.21.15.90:0/3382025937 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd47401d070 con 0x7fd4841034b0 2024-05-27T07:49:00.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.012+0000 7fd499290b80 1 Processor -- start 2024-05-27T07:49:00.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.012+0000 7fd499290b80 1 -- start start 2024-05-27T07:49:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd499290b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5179b6e40 0x55d517b3b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd499290b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b3bd30 0x55d517b35870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd499290b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b35db0 0x55d517b361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd499290b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d517a63e40 con 0x55d517b35db0 2024-05-27T07:49:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd499290b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d517a539e0 con 0x55d517b3bd30 2024-05-27T07:49:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd499290b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d517a57a20 con 0x55d5179b6e40 2024-05-27T07:49:00.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd47be00700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd4680b2a70 0x7fd4680b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd482a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b35db0 0x55d517b361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd482a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b35db0 0x55d517b361c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39182/0 (socket says 172.21.15.90:39182) 2024-05-27T07:49:00.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd482a00700 1 -- 172.21.15.90:0/3438436566 learned_addr learned my addr 172.21.15.90:0/3438436566 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:00.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.013+0000 7fd483400700 1 --2- 172.21.15.90:0/3438436566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b3bd30 0x55d517b35870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.014+0000 7fd47be00700 1 -- 172.21.15.90:0/3382025937 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fd4740e2200 con 0x7fd4841034b0 2024-05-27T07:49:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.014+0000 7fd482a00700 1 -- 172.21.15.90:0/3438436566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5179b6e40 msgr2=0x55d517b3b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.014+0000 7fd482a00700 1 --2- 172.21.15.90:0/3438436566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5179b6e40 0x55d517b3b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.014+0000 7fd482a00700 1 -- 172.21.15.90:0/3438436566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b3bd30 msgr2=0x55d517b35870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.014+0000 7fd482a00700 1 --2- 172.21.15.90:0/3438436566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b3bd30 0x55d517b35870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.014+0000 7fd482a00700 1 -- 172.21.15.90:0/3438436566 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d517a1c420 con 0x55d517b35db0 2024-05-27T07:49:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.014+0000 7fd483400700 1 --2- 172.21.15.90:0/3438436566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b3bd30 0x55d517b35870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:49:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.014+0000 7fd483e00700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd4680b2a70 0x7fd4680b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.015+0000 7fd482a00700 1 --2- 172.21.15.90:0/3438436566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b35db0 0x55d517b361c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd484076aa0 tx=0x7fd4700051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e04c1a23c769154 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.015+0000 7fd478c00700 1 -- 172.21.15.90:0/3438436566 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470026070 con 0x55d517b35db0 2024-05-27T07:49:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.015+0000 7fd478c00700 1 -- 172.21.15.90:0/3438436566 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd470009590 con 0x55d517b35db0 2024-05-27T07:49:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.015+0000 7fd478c00700 1 -- 172.21.15.90:0/3438436566 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470003930 con 0x55d517b35db0 2024-05-27T07:49:00.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.017+0000 7fd483e00700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd4680b2a70 0x7fd4680b4f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55d517b36b30 tx=0x7fd47c005da0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:00.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.017+0000 7fd499290b80 1 -- 172.21.15.90:0/3438436566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b35db0 msgr2=0x55d517b361c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:00.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.017+0000 7fd499290b80 1 --2- 172.21.15.90:0/3438436566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b35db0 0x55d517b361c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd484076aa0 tx=0x7fd4700051d0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.018+0000 7fd499290b80 1 -- 172.21.15.90:0/3438436566 shutdown_connections 2024-05-27T07:49:00.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.018+0000 7fd499290b80 1 --2- 172.21.15.90:0/3438436566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b35db0 0x55d517b361c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.018+0000 7fd499290b80 1 --2- 172.21.15.90:0/3438436566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5179b6e40 0x55d517b3b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.018+0000 7fd499290b80 1 --2- 172.21.15.90:0/3438436566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b3bd30 0x55d517b35870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.018+0000 7fd499290b80 1 -- 172.21.15.90:0/3438436566 >> 172.21.15.90:0/3438436566 conn(0x55d517a16fc0 msgr2=0x55d517b308d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:00.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.018+0000 7fd499290b80 1 -- 172.21.15.90:0/3438436566 shutdown_connections 2024-05-27T07:49:00.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.018+0000 7fd499290b80 1 -- 172.21.15.90:0/3438436566 wait complete. 2024-05-27T07:49:00.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.019+0000 7fd499290b80 1 -- start start 2024-05-27T07:49:00.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.019+0000 7fd499290b80 1 Processor -- start 2024-05-27T07:49:00.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.020+0000 7fd499290b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 0x55d517b3f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.020+0000 7fd499290b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 0x55d517b3fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.020+0000 7fd499290b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.020+0000 7fd499290b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d517959fa0 con 0x55d517b3bd30 2024-05-27T07:49:00.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.020+0000 7fd499290b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d5179f4760 con 0x55d5179b6e40 2024-05-27T07:49:00.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.020+0000 7fd499290b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d517b4afb0 con 0x55d517b35db0 2024-05-27T07:49:00.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.020+0000 7fd499290b80 10 client.0 _ll_register_callbacks cb 0x55d5179fdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:00.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.021+0000 7fd483e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.021+0000 7fd482a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 0x55d517b3fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.021+0000 7fd483400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 0x55d517b3f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.021+0000 7fd483400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 0x55d517b3f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:51970/0 (socket says 172.21.15.90:51970) 2024-05-27T07:49:00.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.021+0000 7fd483e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39188/0 (socket says 172.21.15.90:39188) 2024-05-27T07:49:00.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.021+0000 7fd483400700 1 -- 172.21.15.90:0/2646450983 learned_addr learned my addr 172.21.15.90:0/2646450983 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:00.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.022+0000 7fd482a00700 1 -- 172.21.15.90:0/2646450983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 msgr2=0x55d517b3f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:00.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.022+0000 7fd482a00700 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 0x55d517b3f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.022+0000 7fd482a00700 1 -- 172.21.15.90:0/2646450983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 msgr2=0x55d517b400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:00.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.022+0000 7fd482a00700 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.022+0000 7fd482a00700 1 -- 172.21.15.90:0/2646450983 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5179a38d0 con 0x55d517b35db0 2024-05-27T07:49:00.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.023+0000 7fd482a00700 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 0x55d517b3fb60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd484076aa0 tx=0x7fd47000ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b9364ea8540e1e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:00.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.023+0000 7fd46d600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:49:00.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.023+0000 7fd46d600700 1 -- 172.21.15.90:0/2646450983 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47002f060 con 0x55d517b35db0 2024-05-27T07:49:00.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.023+0000 7fd499290b80 1 -- 172.21.15.90:0/2646450983 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd470009590 con 0x55d517b35db0 2024-05-27T07:49:00.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.024+0000 7fd46d600700 1 -- 172.21.15.90:0/2646450983 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fd47000f460 con 0x55d517b35db0 2024-05-27T07:49:00.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.024+0000 7fd46d600700 1 -- 172.21.15.90:0/2646450983 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd470026070 con 0x55d517b35db0 2024-05-27T07:49:00.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.024+0000 7fd499290b80 10 client.25636 fetch_fsmap learned FSMap version 37 2024-05-27T07:49:00.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.024+0000 7fd499290b80 1 -- 172.21.15.90:0/2646450983 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d517a24060 con 0x55d517b35db0 2024-05-27T07:49:00.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.025+0000 7fd46d600700 1 -- 172.21.15.90:0/2646450983 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd470009590 con 0x55d517b35db0 2024-05-27T07:49:00.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.025+0000 7fd46d600700 1 -- 172.21.15.90:0/2646450983 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fd47002a650 con 0x55d517b35db0 2024-05-27T07:49:00.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.025+0000 7fd499290b80 10 client.25636 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T07:49:00.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.025+0000 7fd499290b80 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:00.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.025+0000 7fd499290b80 20 client.25636 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:00.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.025+0000 7fd499290b80 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d517b57e50 0x55d517b5a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.025+0000 7fd499290b80 4 client.25636 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:00.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.025+0000 7fd499290b80 1 -- 172.21.15.90:0/2646450983 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d517a65d90 con 0x55d517b57e50 2024-05-27T07:49:00.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.026+0000 7fd483400700 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d517b57e50 0x55d517b5a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.027+0000 7fd483400700 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d517b57e50 0x55d517b5a270 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4664e03d77e99915 server_cookie=b31800b68d505d26 in_seq=0 out_seq=0 2024-05-27T07:49:00.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.027+0000 7fd46d600700 10 client.25636 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:00.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.029+0000 7fd46d600700 1 -- 172.21.15.90:0/2646450983 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d517a65d90 con 0x55d517b57e50 2024-05-27T07:49:00.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.029+0000 7fd46d600700 10 client.25636 handle_command_reply: tid=0 2024-05-27T07:49:00.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.029+0000 7fd499290b80 1 client.25636 shutdown 2024-05-27T07:49:00.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.030+0000 7fd499290b80 1 -- 172.21.15.90:0/2646450983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 msgr2=0x55d517b3fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:00.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.030+0000 7fd499290b80 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 0x55d517b3fb60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd484076aa0 tx=0x7fd47000ba30 comp rx=0 tx=0).stop 2024-05-27T07:49:00.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.030+0000 7fd499290b80 1 -- 172.21.15.90:0/2646450983 shutdown_connections 2024-05-27T07:49:00.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.030+0000 7fd499290b80 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d517b57e50 0x55d517b5a270 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.030+0000 7fd499290b80 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.030+0000 7fd46d600700 0 client.25636 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:00.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.030+0000 7fd499290b80 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 0x55d517b3f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.030+0000 7fd499290b80 1 --2- 172.21.15.90:0/2646450983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 0x55d517b3fb60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.030+0000 7fd499290b80 1 -- 172.21.15.90:0/2646450983 >> 172.21.15.90:0/2646450983 conn(0x55d517a16fc0 msgr2=0x55d517b30580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:00.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.030+0000 7fd499290b80 1 -- 172.21.15.90:0/2646450983 shutdown_connections 2024-05-27T07:49:00.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.031+0000 7fd499290b80 1 -- 172.21.15.90:0/2646450983 wait complete. 2024-05-27T07:49:00.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.031+0000 7fd499290b80 20 client.25636 trim_cache size 0 max 16384 2024-05-27T07:49:00.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.050+0000 7fd499290b80 1 Processor -- start 2024-05-27T07:49:00.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.050+0000 7fd499290b80 1 -- start start 2024-05-27T07:49:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.051+0000 7fd499290b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 0x55d517b35870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.051+0000 7fd499290b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 0x55d517b41ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.051+0000 7fd499290b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b39c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.051+0000 7fd499290b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841ae120 con 0x55d517b3bd30 2024-05-27T07:49:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.051+0000 7fd499290b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841afc60 con 0x55d5179b6e40 2024-05-27T07:49:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.051+0000 7fd499290b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841ae7f0 con 0x55d517b35db0 2024-05-27T07:49:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.051+0000 7fd483e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b39c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.051+0000 7fd483e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b39c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39200/0 (socket says 172.21.15.90:39200) 2024-05-27T07:49:00.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.051+0000 7fd483e00700 1 -- 172.21.15.90:0/224633008 learned_addr learned my addr 172.21.15.90:0/224633008 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.051+0000 7fd482a00700 1 --2- 172.21.15.90:0/224633008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 0x55d517b41ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.052+0000 7fd483400700 1 --2- 172.21.15.90:0/224633008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 0x55d517b35870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.052+0000 7fd483e00700 1 -- 172.21.15.90:0/224633008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 msgr2=0x55d517b41ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:00.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.052+0000 7fd483e00700 1 --2- 172.21.15.90:0/224633008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 0x55d517b41ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.052+0000 7fd483e00700 1 -- 172.21.15.90:0/224633008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 msgr2=0x55d517b35870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.052+0000 7fd483e00700 1 --2- 172.21.15.90:0/224633008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 0x55d517b35870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.052+0000 7fd483e00700 1 -- 172.21.15.90:0/224633008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d517b36500 con 0x55d517b3bd30 2024-05-27T07:49:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.052+0000 7fd483e00700 1 --2- 172.21.15.90:0/224633008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b39c40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00dd10 tx=0x7fd47c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82886e398bdb248 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.053+0000 7fd46fe00700 1 -- 172.21.15.90:0/224633008 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c01cd40 con 0x55d517b3bd30 2024-05-27T07:49:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.053+0000 7fd46fe00700 1 -- 172.21.15.90:0/224633008 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd47c0258d0 con 0x55d517b3bd30 2024-05-27T07:49:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.053+0000 7fd46fe00700 1 -- 172.21.15.90:0/224633008 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c00f7e0 con 0x55d517b3bd30 2024-05-27T07:49:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.054+0000 7fd499290b80 1 -- 172.21.15.90:0/224633008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 msgr2=0x55d517b39c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:00.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.054+0000 7fd499290b80 1 --2- 172.21.15.90:0/224633008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b39c40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00dd10 tx=0x7fd47c005f90 comp rx=0 tx=0).stop 2024-05-27T07:49:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.054+0000 7fd499290b80 1 -- 172.21.15.90:0/224633008 shutdown_connections 2024-05-27T07:49:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.054+0000 7fd499290b80 1 --2- 172.21.15.90:0/224633008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d517b3bd30 0x55d517b39c40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.054+0000 7fd499290b80 1 --2- 172.21.15.90:0/224633008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5179b6e40 0x55d517b35870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.054+0000 7fd499290b80 1 --2- 172.21.15.90:0/224633008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b35db0 0x55d517b41ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.054+0000 7fd499290b80 1 -- 172.21.15.90:0/224633008 >> 172.21.15.90:0/224633008 conn(0x55d517a16fc0 msgr2=0x55d517b3adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.054+0000 7fd499290b80 1 -- 172.21.15.90:0/224633008 shutdown_connections 2024-05-27T07:49:00.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.054+0000 7fd499290b80 1 -- 172.21.15.90:0/224633008 wait complete. 2024-05-27T07:49:00.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.055+0000 7fd499290b80 1 -- start start 2024-05-27T07:49:00.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.055+0000 7fd499290b80 1 Processor -- start 2024-05-27T07:49:00.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.056+0000 7fd499290b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5179b6e40 0x55d517b5d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.056+0000 7fd499290b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b35db0 0x55d517b5a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.056+0000 7fd499290b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b3bd30 0x55d517b5ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.056+0000 7fd499290b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841af590 con 0x55d5179b6e40 2024-05-27T07:49:00.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.056+0000 7fd499290b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841aeec0 con 0x55d517b35db0 2024-05-27T07:49:00.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.056+0000 7fd499290b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd4841ac0a0 con 0x55d517b3bd30 2024-05-27T07:49:00.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.056+0000 7fd499290b80 10 client.0 _ll_register_callbacks cb 0x55d5179fdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.057+0000 7fd483400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5179b6e40 0x55d517b5d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.057+0000 7fd483400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5179b6e40 0x55d517b5d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39204/0 (socket says 172.21.15.90:39204) 2024-05-27T07:49:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.057+0000 7fd483400700 1 -- 172.21.15.90:0/3928662977 learned_addr learned my addr 172.21.15.90:0/3928662977 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.057+0000 7fd483e00700 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b3bd30 0x55d517b5ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.057+0000 7fd482a00700 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b35db0 0x55d517b5a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.057+0000 7fd483400700 1 -- 172.21.15.90:0/3928662977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b3bd30 msgr2=0x55d517b5ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:00.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.057+0000 7fd483400700 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b3bd30 0x55d517b5ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.057+0000 7fd483400700 1 -- 172.21.15.90:0/3928662977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b35db0 msgr2=0x55d517b5a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:00.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.057+0000 7fd483400700 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b35db0 0x55d517b5a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.057+0000 7fd483400700 1 -- 172.21.15.90:0/3928662977 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d517b5b3d0 con 0x55d5179b6e40 2024-05-27T07:49:00.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.058+0000 7fd483400700 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5179b6e40 0x55d517b5d620 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd4740df580 tx=0x7fd474072a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ac1ab5536b0456 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:00.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.058+0000 7fd46d600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:00.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.058+0000 7fd46d600700 1 -- 172.21.15.90:0/3928662977 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740dfa70 con 0x55d5179b6e40 2024-05-27T07:49:00.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.059+0000 7fd46d600700 1 -- 172.21.15.90:0/3928662977 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fd4740ad150 con 0x55d5179b6e40 2024-05-27T07:49:00.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.059+0000 7fd499290b80 1 -- 172.21.15.90:0/3928662977 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd47c0258d0 con 0x55d5179b6e40 2024-05-27T07:49:00.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.059+0000 7fd46d600700 1 -- 172.21.15.90:0/3928662977 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd4740b1fd0 con 0x55d5179b6e40 2024-05-27T07:49:00.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.059+0000 7fd499290b80 10 client.16011 fetch_fsmap learned FSMap version 37 2024-05-27T07:49:00.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.059+0000 7fd499290b80 1 -- 172.21.15.90:0/3928662977 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d517bc2280 con 0x55d5179b6e40 2024-05-27T07:49:00.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.060+0000 7fd46d600700 1 -- 172.21.15.90:0/3928662977 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd47c0258d0 con 0x55d5179b6e40 2024-05-27T07:49:00.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.060+0000 7fd46d600700 1 -- 172.21.15.90:0/3928662977 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fd474075b10 con 0x55d5179b6e40 2024-05-27T07:49:00.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.060+0000 7fd499290b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T07:49:00.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.060+0000 7fd499290b80 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:00.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.060+0000 7fd499290b80 20 client.16011 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:00.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.060+0000 7fd499290b80 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d517bc2970 0x55d517bc4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:00.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.060+0000 7fd499290b80 4 client.16011 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:49:00.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.060+0000 7fd499290b80 1 -- 172.21.15.90:0/3928662977 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d517a5f570 con 0x55d517bc2970 2024-05-27T07:49:00.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.061+0000 7fd482a00700 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d517bc2970 0x55d517bc4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:00.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.062+0000 7fd482a00700 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d517bc2970 0x55d517bc4d90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96100eea9848b4c1 server_cookie=ac7a6ac7000970a2 in_seq=0 out_seq=0 2024-05-27T07:49:00.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.062+0000 7fd46d600700 10 client.16011 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:00.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.062+0000 7fd46d600700 1 -- 172.21.15.90:0/3928662977 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d517a5f570 con 0x55d517bc2970 2024-05-27T07:49:00.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.062+0000 7fd46d600700 10 client.16011 handle_command_reply: tid=0 2024-05-27T07:49:00.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.062+0000 7fd499290b80 1 client.16011 shutdown 2024-05-27T07:49:00.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 1 -- 172.21.15.90:0/3928662977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5179b6e40 msgr2=0x55d517b5d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:00.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5179b6e40 0x55d517b5d620 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd4740df580 tx=0x7fd474072a10 comp rx=0 tx=0).stop 2024-05-27T07:49:00.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 1 -- 172.21.15.90:0/3928662977 shutdown_connections 2024-05-27T07:49:00.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d517bc2970 0x55d517bc4d90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d517b3bd30 0x55d517b5ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d5179b6e40 0x55d517b5d620 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 1 --2- 172.21.15.90:0/3928662977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d517b35db0 0x55d517b5a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 1 -- 172.21.15.90:0/3928662977 >> 172.21.15.90:0/3928662977 conn(0x55d517a16fc0 msgr2=0x55d517b3adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:00.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd46d600700 0 client.16011 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:00.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 1 -- 172.21.15.90:0/3928662977 shutdown_connections 2024-05-27T07:49:00.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 1 -- 172.21.15.90:0/3928662977 wait complete. 2024-05-27T07:49:00.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.063+0000 7fd499290b80 20 client.16011 trim_cache size 0 max 16384 2024-05-27T07:49:00.065 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.064+0000 7fd48a800700 1 -- 172.21.15.90:0/3382025937 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd4680b2a70 msgr2=0x7fd4680b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.064+0000 7fd48a800700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd4680b2a70 0x7fd4680b4f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55d517b36b30 tx=0x7fd47c005da0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.064+0000 7fd48a800700 1 -- 172.21.15.90:0/3382025937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4841034b0 msgr2=0x7fd4840ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.064+0000 7fd48a800700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4841034b0 0x7fd4840ff290 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd4740060b0 tx=0x7fd474007f90 comp rx=0 tx=0).stop 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.065+0000 7fd48a800700 1 -- 172.21.15.90:0/3382025937 shutdown_connections 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.065+0000 7fd48a800700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd484108730 0x7fd4840ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.065+0000 7fd48a800700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd4841034b0 0x7fd4840ff290 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.065+0000 7fd48a800700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd4680b2a70 0x7fd4680b4f80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.065+0000 7fd48a800700 1 --2- 172.21.15.90:0/3382025937 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd484107de0 0x7fd4840ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.065+0000 7fd48a800700 1 -- 172.21.15.90:0/3382025937 >> 172.21.15.90:0/3382025937 conn(0x7fd4840fedb0 msgr2=0x7fd48406b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.065+0000 7fd48a800700 1 -- 172.21.15.90:0/3382025937 shutdown_connections 2024-05-27T07:49:00.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:00.065+0000 7fd48a800700 1 -- 172.21.15.90:0/3382025937 wait complete. 2024-05-27T07:49:00.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:00 smithi090 ceph-mon[31429]: pgmap v83: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 26 op/s 2024-05-27T07:49:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:00 smithi136 ceph-mon[35324]: pgmap v83: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 26 op/s 2024-05-27T07:49:00.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:00 smithi129 ceph-mon[35254]: pgmap v83: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 26 op/s 2024-05-27T07:49:01.853 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:49:01.853 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:49:02.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:02 smithi090 ceph-mon[31429]: pgmap v84: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 26 op/s 2024-05-27T07:49:02.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-mon[35324]: pgmap v84: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 26 op/s 2024-05-27T07:49:02.250 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7ff6de600700 time 2024-05-27T07:49:01.980704+0000 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff6f4089ba6] 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff6f4089dd1] 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5603bfd1223d] 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5603bfcfd971] 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 5: (SnapRealm::check_cache() const+0x13e) [0x5603bfcfbfa0] 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5603bfcfc651] 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5603bfc91347] 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5603bf98d87e] 2024-05-27T07:49:02.251 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5603bf992e94] 2024-05-27T07:49:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5603bf99d969] 2024-05-27T07:49:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 11: (MDLog::_replay_thread()+0x1f38) [0x5603bfd3648c] 2024-05-27T07:49:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5603bfd3f9cb] 2024-05-27T07:49:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 13: (Thread::entry_wrapper()+0x43) [0x7ff6f405872d] 2024-05-27T07:49:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 14: (Thread::_entry_func(void*)+0xd) [0x7ff6f4058749] 2024-05-27T07:49:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff6f2bfc1ca] 2024-05-27T07:49:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 16: clone() 2024-05-27T07:49:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: *** Caught signal (Aborted) ** 2024-05-27T07:49:02.252 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: in thread 7ff6de600700 thread_name:md_log_replay 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:01.982+0000 7ff6de600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7ff6de600700 time 2024-05-27T07:49:01.980704+0000 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff6f4089ba6] 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff6f4089dd1] 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5603bfd1223d] 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5603bfcfd971] 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 5: (SnapRealm::check_cache() const+0x13e) [0x5603bfcfbfa0] 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5603bfcfc651] 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5603bfc91347] 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5603bf98d87e] 2024-05-27T07:49:02.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5603bf992e94] 2024-05-27T07:49:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5603bf99d969] 2024-05-27T07:49:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 11: (MDLog::_replay_thread()+0x1f38) [0x5603bfd3648c] 2024-05-27T07:49:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5603bfd3f9cb] 2024-05-27T07:49:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 13: (Thread::entry_wrapper()+0x43) [0x7ff6f405872d] 2024-05-27T07:49:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 14: (Thread::_entry_func(void*)+0xd) [0x7ff6f4058749] 2024-05-27T07:49:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff6f2bfc1ca] 2024-05-27T07:49:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 16: clone() 2024-05-27T07:49:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.254 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff6f2c06d20] 2024-05-27T07:49:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2: gsignal() 2024-05-27T07:49:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 3: abort() 2024-05-27T07:49:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7ff6f4089ced] 2024-05-27T07:49:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff6f4089dd1] 2024-05-27T07:49:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5603bfd1223d] 2024-05-27T07:49:02.255 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5603bfcfd971] 2024-05-27T07:49:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 8: (SnapRealm::check_cache() const+0x13e) [0x5603bfcfbfa0] 2024-05-27T07:49:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5603bfcfc651] 2024-05-27T07:49:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5603bfc91347] 2024-05-27T07:49:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5603bf98d87e] 2024-05-27T07:49:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5603bf992e94] 2024-05-27T07:49:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5603bf99d969] 2024-05-27T07:49:02.256 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 14: (MDLog::_replay_thread()+0x1f38) [0x5603bfd3648c] 2024-05-27T07:49:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5603bfd3f9cb] 2024-05-27T07:49:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 16: (Thread::entry_wrapper()+0x43) [0x7ff6f405872d] 2024-05-27T07:49:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 17: (Thread::_entry_func(void*)+0xd) [0x7ff6f4058749] 2024-05-27T07:49:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff6f2bfc1ca] 2024-05-27T07:49:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 19: clone() 2024-05-27T07:49:02.257 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:01.986+0000 7ff6de600700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: in thread 7ff6de600700 thread_name:md_log_replay 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff6f2c06d20] 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2: gsignal() 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 3: abort() 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7ff6f4089ced] 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff6f4089dd1] 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5603bfd1223d] 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5603bfcfd971] 2024-05-27T07:49:02.258 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 8: (SnapRealm::check_cache() const+0x13e) [0x5603bfcfbfa0] 2024-05-27T07:49:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5603bfcfc651] 2024-05-27T07:49:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5603bfc91347] 2024-05-27T07:49:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5603bf98d87e] 2024-05-27T07:49:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5603bf992e94] 2024-05-27T07:49:02.259 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5603bf99d969] 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 14: (MDLog::_replay_thread()+0x1f38) [0x5603bfd3648c] 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5603bfd3f9cb] 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 16: (Thread::entry_wrapper()+0x43) [0x7ff6f405872d] 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 17: (Thread::_entry_func(void*)+0xd) [0x7ff6f4058749] 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff6f2bfc1ca] 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 19: clone() 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: -1> 2024-05-27T07:49:01.982+0000 7ff6de600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7ff6de600700 time 2024-05-27T07:49:01.980704+0000 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.260 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff6f4089ba6] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff6f4089dd1] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5603bfd1223d] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5603bfcfd971] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 5: (SnapRealm::check_cache() const+0x13e) [0x5603bfcfbfa0] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5603bfcfc651] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5603bfc91347] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5603bf98d87e] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5603bf992e94] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5603bf99d969] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 11: (MDLog::_replay_thread()+0x1f38) [0x5603bfd3648c] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5603bfd3f9cb] 2024-05-27T07:49:02.261 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 13: (Thread::entry_wrapper()+0x43) [0x7ff6f405872d] 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 14: (Thread::_entry_func(void*)+0xd) [0x7ff6f4058749] 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff6f2bfc1ca] 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 16: clone() 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 0> 2024-05-27T07:49:01.986+0000 7ff6de600700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: in thread 7ff6de600700 thread_name:md_log_replay 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff6f2c06d20] 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2: gsignal() 2024-05-27T07:49:02.262 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 3: abort() 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7ff6f4089ced] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff6f4089dd1] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5603bfd1223d] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5603bfcfd971] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 8: (SnapRealm::check_cache() const+0x13e) [0x5603bfcfbfa0] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5603bfcfc651] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5603bfc91347] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5603bf98d87e] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5603bf992e94] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5603bf99d969] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 14: (MDLog::_replay_thread()+0x1f38) [0x5603bfd3648c] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5603bfd3f9cb] 2024-05-27T07:49:02.263 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 16: (Thread::entry_wrapper()+0x43) [0x7ff6f405872d] 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 17: (Thread::_entry_func(void*)+0xd) [0x7ff6f4058749] 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff6f2bfc1ca] 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 19: clone() 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: -1> 2024-05-27T07:49:01.982+0000 7ff6de600700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7ff6de600700 time 2024-05-27T07:49:01.980704+0000 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7ff6f4089ba6] 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff6f4089dd1] 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5603bfd1223d] 2024-05-27T07:49:02.264 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5603bfcfd971] 2024-05-27T07:49:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 5: (SnapRealm::check_cache() const+0x13e) [0x5603bfcfbfa0] 2024-05-27T07:49:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5603bfcfc651] 2024-05-27T07:49:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5603bfc91347] 2024-05-27T07:49:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5603bf98d87e] 2024-05-27T07:49:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5603bf992e94] 2024-05-27T07:49:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5603bf99d969] 2024-05-27T07:49:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 11: (MDLog::_replay_thread()+0x1f38) [0x5603bfd3648c] 2024-05-27T07:49:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5603bfd3f9cb] 2024-05-27T07:49:02.265 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 13: (Thread::entry_wrapper()+0x43) [0x7ff6f405872d] 2024-05-27T07:49:02.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 14: (Thread::_entry_func(void*)+0xd) [0x7ff6f4058749] 2024-05-27T07:49:02.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff6f2bfc1ca] 2024-05-27T07:49:02.266 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 16: clone() 2024-05-27T07:49:02.267 INFO:teuthology.orchestra.run.smithi090.stderr:2500000 of 3200000 tuples (78%) done (elapsed 78.31 s, remaining 21.93 s) 2024-05-27T07:49:02.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 0> 2024-05-27T07:49:01.986+0000 7ff6de600700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:49:02.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: in thread 7ff6de600700 thread_name:md_log_replay 2024-05-27T07:49:02.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:02.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff6f2c06d20] 2024-05-27T07:49:02.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2: gsignal() 2024-05-27T07:49:02.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 3: abort() 2024-05-27T07:49:02.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7ff6f4089ced] 2024-05-27T07:49:02.267 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7ff6f4089dd1] 2024-05-27T07:49:02.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5603bfd1223d] 2024-05-27T07:49:02.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5603bfcfd971] 2024-05-27T07:49:02.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 8: (SnapRealm::check_cache() const+0x13e) [0x5603bfcfbfa0] 2024-05-27T07:49:02.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5603bfcfc651] 2024-05-27T07:49:02.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5603bfc91347] 2024-05-27T07:49:02.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5603bf98d87e] 2024-05-27T07:49:02.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5603bf992e94] 2024-05-27T07:49:02.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5603bf99d969] 2024-05-27T07:49:02.268 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 14: (MDLog::_replay_thread()+0x1f38) [0x5603bfd3648c] 2024-05-27T07:49:02.269 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5603bfd3f9cb] 2024-05-27T07:49:02.269 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 16: (Thread::entry_wrapper()+0x43) [0x7ff6f405872d] 2024-05-27T07:49:02.269 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 17: (Thread::_entry_func(void*)+0xd) [0x7ff6f4058749] 2024-05-27T07:49:02.269 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7ff6f2bfc1ca] 2024-05-27T07:49:02.269 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 19: clone() 2024-05-27T07:49:02.269 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:49:02.270 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71099]: 2024-05-27T07:49:02.270 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 podman[71137]: 2024-05-27 07:49:02.09697007 +0000 UTC m=+0.033951091 container died 8e007cf054497624966946e1fbe8724a67ee2c3ddb9e87eef3601540c0446be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-27T07:49:02.311 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:02.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:02 smithi129 ceph-mon[35254]: pgmap v84: 129 pgs: 129 active+clean; 618 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 MiB/s wr, 26 op/s 2024-05-27T07:49:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.128+0000 7fb767200640 1 -- 172.21.15.90:0/3993246308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb768104bb0 msgr2=0x7fb768104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.128+0000 7fb767200640 1 --2- 172.21.15.90:0/3993246308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb768104bb0 0x7fb768104fc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb758009900 tx=0x7fb75802a410 comp rx=0 tx=0).stop 2024-05-27T07:49:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.128+0000 7fb767200640 1 -- 172.21.15.90:0/3993246308 shutdown_connections 2024-05-27T07:49:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.128+0000 7fb767200640 1 --2- 172.21.15.90:0/3993246308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb76810e000 0x7fb7681104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.128+0000 7fb767200640 1 --2- 172.21.15.90:0/3993246308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb76810b610 0x7fb76810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.128+0000 7fb767200640 1 --2- 172.21.15.90:0/3993246308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb768104bb0 0x7fb768104fc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.128+0000 7fb767200640 1 -- 172.21.15.90:0/3993246308 >> 172.21.15.90:0/3993246308 conn(0x7fb7680fe210 msgr2=0x7fb768100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.128+0000 7fb767200640 1 -- 172.21.15.90:0/3993246308 shutdown_connections 2024-05-27T07:49:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.128+0000 7fb767200640 1 -- 172.21.15.90:0/3993246308 wait complete. 2024-05-27T07:49:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.129+0000 7fb767200640 1 Processor -- start 2024-05-27T07:49:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.130+0000 7fb767200640 1 -- start start 2024-05-27T07:49:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.130+0000 7fb767200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb768104bb0 0x7fb768102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.130+0000 7fb767200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb76810b610 0x7fb7681029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.130+0000 7fb767200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb76810e000 0x7fb768102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.130+0000 7fb767200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb768113b60 con 0x7fb768104bb0 2024-05-27T07:49:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.130+0000 7fb767200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681139e0 con 0x7fb76810e000 2024-05-27T07:49:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.130+0000 7fb767200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb768113ce0 con 0x7fb76810b610 2024-05-27T07:49:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.130+0000 7fb765e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb768104bb0 0x7fb768102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.131+0000 7fb765e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb768104bb0 0x7fb768102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39222/0 (socket says 172.21.15.90:39222) 2024-05-27T07:49:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.131+0000 7fb765e00640 1 -- 172.21.15.90:0/2395784971 learned_addr learned my addr 172.21.15.90:0/2395784971 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.131+0000 7fb765400640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb76810b610 0x7fb7681029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.131+0000 7fb766800640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb76810e000 0x7fb768102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.131+0000 7fb765e00640 1 -- 172.21.15.90:0/2395784971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb76810b610 msgr2=0x7fb7681029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.131+0000 7fb765e00640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb76810b610 0x7fb7681029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.131+0000 7fb765e00640 1 -- 172.21.15.90:0/2395784971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb76810e000 msgr2=0x7fb768102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.131+0000 7fb765e00640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb76810e000 0x7fb768102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.131+0000 7fb765e00640 1 -- 172.21.15.90:0/2395784971 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7681a8890 con 0x7fb768104bb0 2024-05-27T07:49:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.132+0000 7fb765e00640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb768104bb0 0x7fb768102470 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb75802a980 tx=0x7fb758003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.132+0000 7fb75ea00640 1 -- 172.21.15.90:0/2395784971 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75803f070 con 0x7fb768104bb0 2024-05-27T07:49:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.132+0000 7fb75ea00640 1 -- 172.21.15.90:0/2395784971 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb75803bd00 con 0x7fb768104bb0 2024-05-27T07:49:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.132+0000 7fb767200640 1 -- 172.21.15.90:0/2395784971 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7681a8b20 con 0x7fb768104bb0 2024-05-27T07:49:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.132+0000 7fb75ea00640 1 -- 172.21.15.90:0/2395784971 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758035030 con 0x7fb768104bb0 2024-05-27T07:49:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.132+0000 7fb767200640 1 -- 172.21.15.90:0/2395784971 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7681a9060 con 0x7fb768104bb0 2024-05-27T07:49:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.136+0000 7fb773f8a740 1 Processor -- start 2024-05-27T07:49:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.136+0000 7fb773f8a740 1 -- start start 2024-05-27T07:49:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.136+0000 7fb75ea00640 1 -- 172.21.15.90:0/2395784971 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb758042070 con 0x7fb768104bb0 2024-05-27T07:49:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.136+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510dd7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.137+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510dd7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.137+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a144d0 0x562510dd84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.137+0000 7fb773f8a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562510d48b90 con 0x562510a0c2f0 2024-05-27T07:49:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.137+0000 7fb773f8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562510d38730 con 0x562510a144d0 2024-05-27T07:49:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.137+0000 7fb773f8a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562510d3c770 con 0x562510a351b0 2024-05-27T07:49:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.137+0000 7fb765e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510dd7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.137+0000 7fb766800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510dd7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.137+0000 7fb765e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510dd7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39232/0 (socket says 172.21.15.90:39232) 2024-05-27T07:49:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.137+0000 7fb765e00640 1 -- 172.21.15.90:0/2209242001 learned_addr learned my addr 172.21.15.90:0/2209242001 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.137+0000 7fb765400640 1 --2- 172.21.15.90:0/2209242001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a144d0 0x562510dd84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.138+0000 7fb765e00640 1 -- 172.21.15.90:0/2209242001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 msgr2=0x562510dd7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.138+0000 7fb765e00640 1 --2- 172.21.15.90:0/2209242001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510dd7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.138+0000 7fb765e00640 1 -- 172.21.15.90:0/2209242001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a144d0 msgr2=0x562510dd84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.138+0000 7fb765e00640 1 --2- 172.21.15.90:0/2209242001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a144d0 0x562510dd84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.138+0000 7fb765e00640 1 -- 172.21.15.90:0/2209242001 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562510cd18b0 con 0x562510a0c2f0 2024-05-27T07:49:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.138+0000 7fb765e00640 1 --2- 172.21.15.90:0/2209242001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510dd7fa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb7581004e0 tx=0x7fb75810b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4978c82a0b81b3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.138+0000 7fb75ea00640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb74c0b2a20 0x7fb74c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.138+0000 7fb753400640 1 -- 172.21.15.90:0/2209242001 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580c6780 con 0x562510a0c2f0 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.138+0000 7fb753400640 1 -- 172.21.15.90:0/2209242001 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb758110030 con 0x562510a0c2f0 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.139+0000 7fb753400640 1 -- 172.21.15.90:0/2209242001 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580c98a0 con 0x562510a0c2f0 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.139+0000 7fb766800640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb74c0b2a20 0x7fb74c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.139+0000 7fb75ea00640 1 -- 172.21.15.90:0/2395784971 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb7581031c0 con 0x7fb768104bb0 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.140+0000 7fb773f8a740 1 -- 172.21.15.90:0/2209242001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 msgr2=0x562510dd7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.140+0000 7fb766800640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb74c0b2a20 0x7fb74c0b4f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x562510dd4bd0 tx=0x7fb760007600 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.140+0000 7fb773f8a740 1 --2- 172.21.15.90:0/2209242001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510dd7fa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb7581004e0 tx=0x7fb75810b480 comp rx=0 tx=0).stop 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.140+0000 7fb773f8a740 1 -- 172.21.15.90:0/2209242001 shutdown_connections 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.140+0000 7fb773f8a740 1 --2- 172.21.15.90:0/2209242001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a144d0 0x562510dd84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.140+0000 7fb773f8a740 1 --2- 172.21.15.90:0/2209242001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510dd7fa0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.140+0000 7fb773f8a740 1 --2- 172.21.15.90:0/2209242001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510dd7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.140+0000 7fb773f8a740 1 -- 172.21.15.90:0/2209242001 >> 172.21.15.90:0/2209242001 conn(0x562510a29300 msgr2=0x562510dd4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:03.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.140+0000 7fb773f8a740 1 -- 172.21.15.90:0/2209242001 shutdown_connections 2024-05-27T07:49:03.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.140+0000 7fb773f8a740 1 -- 172.21.15.90:0/2209242001 wait complete. 2024-05-27T07:49:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.141+0000 7fb773f8a740 1 -- start start 2024-05-27T07:49:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.141+0000 7fb773f8a740 1 Processor -- start 2024-05-27T07:49:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.141+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.141+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a144d0 0x562510de20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.141+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a351b0 0x562510dedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.141+0000 7fb773f8a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562510a1e790 con 0x562510a0c2f0 2024-05-27T07:49:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.141+0000 7fb773f8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562510ce0bc0 con 0x562510a351b0 2024-05-27T07:49:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb773f8a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562510de4950 con 0x562510a144d0 2024-05-27T07:49:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb773f8a740 10 client.0 _ll_register_callbacks cb 0x562510cbe400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:03.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb765e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb765e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39242/0 (socket says 172.21.15.90:39242) 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb765e00640 1 -- 172.21.15.90:0/990407177 learned_addr learned my addr 172.21.15.90:0/990407177 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb765400640 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a144d0 0x562510de20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb766800640 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a351b0 0x562510dedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb765e00640 1 -- 172.21.15.90:0/990407177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a144d0 msgr2=0x562510de20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb765e00640 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a144d0 0x562510de20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb765e00640 1 -- 172.21.15.90:0/990407177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a351b0 msgr2=0x562510dedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb765e00640 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a351b0 0x562510dedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.142+0000 7fb765e00640 1 -- 172.21.15.90:0/990407177 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562510a4d210 con 0x562510a0c2f0 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.143+0000 7fb765e00640 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de1bb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb7580c61b0 tx=0x7fb758009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5492ac7f80e10ab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.143+0000 7fb747e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:03.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.143+0000 7fb747e00640 1 -- 172.21.15.90:0/990407177 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758049070 con 0x562510a0c2f0 2024-05-27T07:49:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.143+0000 7fb747e00640 1 -- 172.21.15.90:0/990407177 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb758031400 con 0x562510a0c2f0 2024-05-27T07:49:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.143+0000 7fb773f8a740 1 -- 172.21.15.90:0/990407177 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb758110030 con 0x562510a0c2f0 2024-05-27T07:49:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.143+0000 7fb747e00640 1 -- 172.21.15.90:0/990407177 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb7580c9a80 con 0x562510a0c2f0 2024-05-27T07:49:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.143+0000 7fb773f8a740 10 client.16023 fetch_fsmap learned FSMap version 37 2024-05-27T07:49:03.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.144+0000 7fb773f8a740 1 -- 172.21.15.90:0/990407177 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562510b19620 con 0x562510a0c2f0 2024-05-27T07:49:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.144+0000 7fb747e00640 1 -- 172.21.15.90:0/990407177 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb758110030 con 0x562510a0c2f0 2024-05-27T07:49:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.144+0000 7fb747e00640 1 -- 172.21.15.90:0/990407177 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fb758100820 con 0x562510a0c2f0 2024-05-27T07:49:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.144+0000 7fb773f8a740 10 client.16023 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T07:49:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.144+0000 7fb773f8a740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.144+0000 7fb773f8a740 20 client.16023 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:03.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.144+0000 7fb773f8a740 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5625109f3d80 0x562510df3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.144+0000 7fb773f8a740 4 client.16023 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.144+0000 7fb773f8a740 1 -- 172.21.15.90:0/990407177 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562510d4aae0 con 0x5625109f3d80 2024-05-27T07:49:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.145+0000 7fb765400640 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5625109f3d80 0x562510df3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.146+0000 7fb765400640 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5625109f3d80 0x562510df3940 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6f8677de0c5db3c server_cookie=ecf53a127b9e060b in_seq=0 out_seq=0 2024-05-27T07:49:03.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.146+0000 7fb747e00640 10 client.16023 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb747e00640 1 -- 172.21.15.90:0/990407177 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562510d4aae0 con 0x5625109f3d80 2024-05-27T07:49:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb747e00640 10 client.16023 handle_command_reply: tid=0 2024-05-27T07:49:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb773f8a740 1 client.16023 shutdown 2024-05-27T07:49:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb773f8a740 1 -- 172.21.15.90:0/990407177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 msgr2=0x562510de1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb773f8a740 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de1bb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb7580c61b0 tx=0x7fb758009260 comp rx=0 tx=0).stop 2024-05-27T07:49:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb773f8a740 1 -- 172.21.15.90:0/990407177 shutdown_connections 2024-05-27T07:49:03.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb773f8a740 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5625109f3d80 0x562510df3940 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb773f8a740 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a351b0 0x562510dedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb773f8a740 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a144d0 0x562510de20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb773f8a740 1 --2- 172.21.15.90:0/990407177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de1bb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb773f8a740 1 -- 172.21.15.90:0/990407177 >> 172.21.15.90:0/990407177 conn(0x562510a29300 msgr2=0x562510dd9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.148+0000 7fb747e00640 0 client.16023 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.149+0000 7fb773f8a740 1 -- 172.21.15.90:0/990407177 shutdown_connections 2024-05-27T07:49:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.149+0000 7fb773f8a740 1 -- 172.21.15.90:0/990407177 wait complete. 2024-05-27T07:49:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.149+0000 7fb773f8a740 20 client.16023 trim_cache size 0 max 16384 2024-05-27T07:49:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.169+0000 7fb773f8a740 1 Processor -- start 2024-05-27T07:49:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.170+0000 7fb773f8a740 1 -- start start 2024-05-27T07:49:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.170+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.170+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a144d0 0x562510ded9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.170+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510dedef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.170+0000 7fb773f8a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681b30f0 con 0x562510a0c2f0 2024-05-27T07:49:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.170+0000 7fb773f8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681b4c30 con 0x562510a144d0 2024-05-27T07:49:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.170+0000 7fb773f8a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681b37c0 con 0x562510a351b0 2024-05-27T07:49:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.170+0000 7fb765e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.170+0000 7fb765e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39248/0 (socket says 172.21.15.90:39248) 2024-05-27T07:49:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.170+0000 7fb765e00640 1 -- 172.21.15.90:0/1079934733 learned_addr learned my addr 172.21.15.90:0/1079934733 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.171+0000 7fb766800640 1 --2- 172.21.15.90:0/1079934733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510dedef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.171+0000 7fb765400640 1 --2- 172.21.15.90:0/1079934733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a144d0 0x562510ded9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.171+0000 7fb765e00640 1 -- 172.21.15.90:0/1079934733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 msgr2=0x562510dedef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.171+0000 7fb765e00640 1 --2- 172.21.15.90:0/1079934733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510dedef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.171+0000 7fb765e00640 1 -- 172.21.15.90:0/1079934733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a144d0 msgr2=0x562510ded9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.171+0000 7fb765e00640 1 --2- 172.21.15.90:0/1079934733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a144d0 0x562510ded9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.171+0000 7fb765e00640 1 -- 172.21.15.90:0/1079934733 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562510de4680 con 0x562510a0c2f0 2024-05-27T07:49:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.171+0000 7fb765e00640 1 --2- 172.21.15.90:0/1079934733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de3da0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb758005ab0 tx=0x7fb75810b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a67e5d16ca45873 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.172+0000 7fb752a00640 1 -- 172.21.15.90:0/1079934733 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758049030 con 0x562510a0c2f0 2024-05-27T07:49:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.172+0000 7fb752a00640 1 -- 172.21.15.90:0/1079934733 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb758110070 con 0x562510a0c2f0 2024-05-27T07:49:03.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.172+0000 7fb752a00640 1 -- 172.21.15.90:0/1079934733 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580c9c70 con 0x562510a0c2f0 2024-05-27T07:49:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.173+0000 7fb773f8a740 1 -- 172.21.15.90:0/1079934733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 msgr2=0x562510de3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.173+0000 7fb773f8a740 1 --2- 172.21.15.90:0/1079934733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de3da0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb758005ab0 tx=0x7fb75810b200 comp rx=0 tx=0).stop 2024-05-27T07:49:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.173+0000 7fb773f8a740 1 -- 172.21.15.90:0/1079934733 shutdown_connections 2024-05-27T07:49:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.173+0000 7fb773f8a740 1 --2- 172.21.15.90:0/1079934733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a144d0 0x562510ded9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.173+0000 7fb773f8a740 1 --2- 172.21.15.90:0/1079934733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510dedef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.173+0000 7fb773f8a740 1 --2- 172.21.15.90:0/1079934733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a0c2f0 0x562510de3da0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.173+0000 7fb773f8a740 1 -- 172.21.15.90:0/1079934733 >> 172.21.15.90:0/1079934733 conn(0x562510a29300 msgr2=0x562510de1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.173+0000 7fb773f8a740 1 -- 172.21.15.90:0/1079934733 shutdown_connections 2024-05-27T07:49:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.173+0000 7fb773f8a740 1 -- 172.21.15.90:0/1079934733 wait complete. 2024-05-27T07:49:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.174+0000 7fb773f8a740 1 -- start start 2024-05-27T07:49:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.174+0000 7fb773f8a740 1 Processor -- start 2024-05-27T07:49:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.174+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a0c2f0 0x562510de5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a144d0 0x562510de56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb773f8a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510e59f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb773f8a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681b4560 con 0x562510a144d0 2024-05-27T07:49:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb773f8a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681b3e90 con 0x562510a0c2f0 2024-05-27T07:49:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb773f8a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7681b10b0 con 0x562510a351b0 2024-05-27T07:49:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb773f8a740 10 client.0 _ll_register_callbacks cb 0x7fb7600074a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb765400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a144d0 0x562510de56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb765400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a144d0 0x562510de56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39250/0 (socket says 172.21.15.90:39250) 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb765400640 1 -- 172.21.15.90:0/3315698333 learned_addr learned my addr 172.21.15.90:0/3315698333 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb766800640 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510e59f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb765e00640 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a0c2f0 0x562510de5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb765400640 1 -- 172.21.15.90:0/3315698333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 msgr2=0x562510e59f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb765400640 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510e59f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb765400640 1 -- 172.21.15.90:0/3315698333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a0c2f0 msgr2=0x562510de5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb765400640 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a0c2f0 0x562510de5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.175+0000 7fb765400640 1 -- 172.21.15.90:0/3315698333 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562510e5a440 con 0x562510a144d0 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.176+0000 7fb765400640 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a144d0 0x562510de56c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x562510dcf8f0 tx=0x7fb75400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cd6bd1ab916ef9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.176+0000 7fb747e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.176+0000 7fb747e00640 1 -- 172.21.15.90:0/3315698333 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754005220 con 0x562510a144d0 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.176+0000 7fb747e00640 1 -- 172.21.15.90:0/3315698333 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7fb7540053c0 con 0x562510a144d0 2024-05-27T07:49:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.176+0000 7fb773f8a740 1 -- 172.21.15.90:0/3315698333 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb758110070 con 0x562510a144d0 2024-05-27T07:49:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.176+0000 7fb747e00640 1 -- 172.21.15.90:0/3315698333 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb75402d050 con 0x562510a144d0 2024-05-27T07:49:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.177+0000 7fb773f8a740 10 client.16029 fetch_fsmap learned FSMap version 37 2024-05-27T07:49:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.177+0000 7fb773f8a740 1 -- 172.21.15.90:0/3315698333 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562510e5b6f0 con 0x562510a144d0 2024-05-27T07:49:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.177+0000 7fb747e00640 1 -- 172.21.15.90:0/3315698333 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb758110070 con 0x562510a144d0 2024-05-27T07:49:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.177+0000 7fb747e00640 1 -- 172.21.15.90:0/3315698333 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fb75401d4a0 con 0x562510a144d0 2024-05-27T07:49:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.177+0000 7fb773f8a740 10 client.16029 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T07:49:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.177+0000 7fb773f8a740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.177+0000 7fb773f8a740 20 client.16029 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.177+0000 7fb773f8a740 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5625109c91c0 0x562510e5df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.177+0000 7fb773f8a740 4 client.16029 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:49:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.177+0000 7fb773f8a740 1 -- 172.21.15.90:0/3315698333 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562510d442c0 con 0x5625109c91c0 2024-05-27T07:49:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.178+0000 7fb765e00640 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5625109c91c0 0x562510e5df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.179+0000 7fb765e00640 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5625109c91c0 0x562510e5df00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b6edd49294afeba server_cookie=591a507cb8cc9d21 in_seq=0 out_seq=0 2024-05-27T07:49:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.179+0000 7fb747e00640 10 client.16029 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.225+0000 7fb747e00640 1 -- 172.21.15.90:0/3315698333 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562510d442c0 con 0x5625109c91c0 2024-05-27T07:49:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.225+0000 7fb747e00640 10 client.16029 handle_command_reply: tid=0 2024-05-27T07:49:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.225+0000 7fb773f8a740 1 client.16029 shutdown 2024-05-27T07:49:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 1 -- 172.21.15.90:0/3315698333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a144d0 msgr2=0x562510de56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a144d0 0x562510de56c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x562510dcf8f0 tx=0x7fb75400bf00 comp rx=0 tx=0).stop 2024-05-27T07:49:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 1 -- 172.21.15.90:0/3315698333 shutdown_connections 2024-05-27T07:49:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5625109c91c0 0x562510e5df00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562510a351b0 0x562510e59f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562510a144d0 0x562510de56c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 1 --2- 172.21.15.90:0/3315698333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562510a0c2f0 0x562510de5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb747e00640 0 client.16029 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:03.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 1 -- 172.21.15.90:0/3315698333 >> 172.21.15.90:0/3315698333 conn(0x562510a29300 msgr2=0x562510dd77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:03.228 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 podman[71137]: 2024-05-27 07:49:02.801788138 +0000 UTC m=+0.738769176 container remove 8e007cf054497624966946e1fbe8724a67ee2c3ddb9e87eef3601540c0446be8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-27T07:49:03.228 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:02 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:49:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 1 -- 172.21.15.90:0/3315698333 shutdown_connections 2024-05-27T07:49:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 1 -- 172.21.15.90:0/3315698333 wait complete. 2024-05-27T07:49:03.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.226+0000 7fb773f8a740 20 client.16029 trim_cache size 0 max 16384 2024-05-27T07:49:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:49:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:49:03.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.227+0000 7fb767200640 1 -- 172.21.15.90:0/2395784971 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb74c0b2a20 msgr2=0x7fb74c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.227+0000 7fb767200640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb74c0b2a20 0x7fb74c0b4f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x562510dd4bd0 tx=0x7fb760007600 comp rx=0 tx=0).stop 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.227+0000 7fb767200640 1 -- 172.21.15.90:0/2395784971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb768104bb0 msgr2=0x7fb768102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.227+0000 7fb767200640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb768104bb0 0x7fb768102470 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb75802a980 tx=0x7fb758003cd0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.227+0000 7fb767200640 1 -- 172.21.15.90:0/2395784971 shutdown_connections 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.227+0000 7fb767200640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb74c0b2a20 0x7fb74c0b4f30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.228+0000 7fb767200640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb76810e000 0x7fb768102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.228+0000 7fb767200640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb76810b610 0x7fb7681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.228+0000 7fb767200640 1 --2- 172.21.15.90:0/2395784971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb768104bb0 0x7fb768102470 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.228+0000 7fb767200640 1 -- 172.21.15.90:0/2395784971 >> 172.21.15.90:0/2395784971 conn(0x7fb7680fe210 msgr2=0x7fb76810e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.228+0000 7fb767200640 1 -- 172.21.15.90:0/2395784971 shutdown_connections 2024-05-27T07:49:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:03.228+0000 7fb767200640 1 -- 172.21.15.90:0/2395784971 wait complete. 2024-05-27T07:49:03.499 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:03 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:49:03.499 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:03 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.438s CPU time. 2024-05-27T07:49:04.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:04 smithi090 ceph-mon[31429]: pgmap v85: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 38 op/s 2024-05-27T07:49:04.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:04 smithi136 ceph-mon[35324]: pgmap v85: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 38 op/s 2024-05-27T07:49:04.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:04 smithi129 ceph-mon[35254]: pgmap v85: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 38 op/s 2024-05-27T07:49:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2600000 of 3200000 tuples (81%) done (elapsed 80.66 s, remaining 18.61 s) 2024-05-27T07:49:04.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.868+0000 7f2f49000700 1 -- 172.21.15.90:0/1819188872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f44105470 msgr2=0x7f2f44105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:04.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.869+0000 7f2f49000700 1 --2- 172.21.15.90:0/1819188872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f44105470 0x7f2f44105880 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009990 tx=0x7f2f34009ca0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.869+0000 7f2f49000700 1 -- 172.21.15.90:0/1819188872 shutdown_connections 2024-05-27T07:49:04.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.869+0000 7f2f49000700 1 --2- 172.21.15.90:0/1819188872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f4410e8e0 0x7f2f44110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.869+0000 7f2f49000700 1 --2- 172.21.15.90:0/1819188872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f44105470 0x7f2f44105880 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.869+0000 7f2f49000700 1 --2- 172.21.15.90:0/1819188872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f44105dc0 0x7f2f4410e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.869+0000 7f2f49000700 1 -- 172.21.15.90:0/1819188872 >> 172.21.15.90:0/1819188872 conn(0x7f2f440fef20 msgr2=0x7f2f44101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:04.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.869+0000 7f2f49000700 1 -- 172.21.15.90:0/1819188872 shutdown_connections 2024-05-27T07:49:04.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.869+0000 7f2f49000700 1 -- 172.21.15.90:0/1819188872 wait complete. 2024-05-27T07:49:04.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.870+0000 7f2f49000700 1 Processor -- start 2024-05-27T07:49:04.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.870+0000 7f2f49000700 1 -- start start 2024-05-27T07:49:04.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.871+0000 7f2f49000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f44105470 0x7f2f4419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.871+0000 7f2f49000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f44105dc0 0x7f2f4419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.871+0000 7f2f49000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f4410e8e0 0x7f2f441a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.871+0000 7f2f49000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441142e0 con 0x7f2f44105470 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.871+0000 7f2f49000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44114160 con 0x7f2f4410e8e0 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.871+0000 7f2f49000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44114460 con 0x7f2f44105dc0 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.871+0000 7f2f41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f44105470 0x7f2f4419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.871+0000 7f2f41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f44105470 0x7f2f4419cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39268/0 (socket says 172.21.15.90:39268) 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.871+0000 7f2f41e00700 1 -- 172.21.15.90:0/917188617 learned_addr learned my addr 172.21.15.90:0/917188617 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.872+0000 7f2f41400700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f44105dc0 0x7f2f4419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.872+0000 7f2f41e00700 1 -- 172.21.15.90:0/917188617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f44105dc0 msgr2=0x7f2f4419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.872+0000 7f2f41e00700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f44105dc0 0x7f2f4419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.872+0000 7f2f42800700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f4410e8e0 0x7f2f441a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.872+0000 7f2f41e00700 1 -- 172.21.15.90:0/917188617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f4410e8e0 msgr2=0x7f2f441a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.872+0000 7f2f41e00700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f4410e8e0 0x7f2f441a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.872+0000 7f2f41e00700 1 -- 172.21.15.90:0/917188617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f441a1de0 con 0x7f2f44105470 2024-05-27T07:49:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.872+0000 7f2f41e00700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f44105470 0x7f2f4419cf10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2f34000c00 tx=0x7f2f34009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.873+0000 7f2f3aa00700 1 -- 172.21.15.90:0/917188617 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34027070 con 0x7f2f44105470 2024-05-27T07:49:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.873+0000 7f2f3aa00700 1 -- 172.21.15.90:0/917188617 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f3400fc70 con 0x7f2f44105470 2024-05-27T07:49:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.873+0000 7f2f3aa00700 1 -- 172.21.15.90:0/917188617 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34022d40 con 0x7f2f44105470 2024-05-27T07:49:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.873+0000 7f2f49000700 1 -- 172.21.15.90:0/917188617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f441a2070 con 0x7f2f44105470 2024-05-27T07:49:04.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.873+0000 7f2f49000700 1 -- 172.21.15.90:0/917188617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f441a2500 con 0x7f2f44105470 2024-05-27T07:49:04.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.877+0000 7f2f3aa00700 1 -- 172.21.15.90:0/917188617 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2f3402a070 con 0x7f2f44105470 2024-05-27T07:49:04.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.879+0000 7f2f57b8eb80 1 Processor -- start 2024-05-27T07:49:04.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.879+0000 7f2f3aa00700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f280b2a70 0x7f2f280b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.879+0000 7f2f57b8eb80 1 -- start start 2024-05-27T07:49:04.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.879+0000 7f2f3aa00700 1 -- 172.21.15.90:0/917188617 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2f340e36f0 con 0x7f2f44105470 2024-05-27T07:49:04.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.880+0000 7f2f42800700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f280b2a70 0x7f2f280b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.880+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9beaec0 0x558fe9d6f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.880+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d6fdb0 0x558fe9d698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.880+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d6a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.880+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558fe9c97ec0 con 0x558fe9d6fdb0 2024-05-27T07:49:04.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.880+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558fe9c87a60 con 0x558fe9beaec0 2024-05-27T07:49:04.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.880+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558fe9c8baa0 con 0x558fe9d69e30 2024-05-27T07:49:04.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.880+0000 7f2f41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d6fdb0 0x558fe9d698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.880+0000 7f2f41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d6fdb0 0x558fe9d698f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39278/0 (socket says 172.21.15.90:39278) 2024-05-27T07:49:04.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.880+0000 7f2f41400700 1 -- 172.21.15.90:0/675781763 learned_addr learned my addr 172.21.15.90:0/675781763 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:04.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.881+0000 7f2f41e00700 1 --2- 172.21.15.90:0/675781763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9beaec0 0x558fe9d6f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.881+0000 7f2f42800700 1 --2- 172.21.15.90:0/675781763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d6a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.881+0000 7f2f41400700 1 -- 172.21.15.90:0/675781763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 msgr2=0x558fe9d6a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:04.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.881+0000 7f2f41400700 1 --2- 172.21.15.90:0/675781763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d6a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.881+0000 7f2f41400700 1 -- 172.21.15.90:0/675781763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9beaec0 msgr2=0x558fe9d6f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:04.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.881+0000 7f2f41400700 1 --2- 172.21.15.90:0/675781763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9beaec0 0x558fe9d6f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.881+0000 7f2f41400700 1 -- 172.21.15.90:0/675781763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fe9c504a0 con 0x558fe9d6fdb0 2024-05-27T07:49:04.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.881+0000 7f2f42800700 1 --2- 172.21.15.90:0/675781763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d6a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:04.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.881+0000 7f2f41400700 1 --2- 172.21.15.90:0/675781763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d6fdb0 0x558fe9d698f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009bf0 tx=0x7f2f30009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32adbd020df15fdc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:04.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.882+0000 7f2f2f400700 1 -- 172.21.15.90:0/675781763 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30016070 con 0x558fe9d6fdb0 2024-05-27T07:49:04.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.882+0000 7f2f42800700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f280b2a70 0x7f2f280b4f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009b10 tx=0x7f2f3c009ed0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:04.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.882+0000 7f2f2f400700 1 -- 172.21.15.90:0/675781763 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f30004830 con 0x558fe9d6fdb0 2024-05-27T07:49:04.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.882+0000 7f2f2f400700 1 -- 172.21.15.90:0/675781763 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f300054e0 con 0x558fe9d6fdb0 2024-05-27T07:49:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.883+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/675781763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d6fdb0 msgr2=0x558fe9d698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.883+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/675781763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d6fdb0 0x558fe9d698f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009bf0 tx=0x7f2f30009f00 comp rx=0 tx=0).stop 2024-05-27T07:49:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.883+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/675781763 shutdown_connections 2024-05-27T07:49:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.883+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/675781763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d6a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.883+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/675781763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9beaec0 0x558fe9d6f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.883+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/675781763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d6fdb0 0x558fe9d698f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.883+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/675781763 >> 172.21.15.90:0/675781763 conn(0x558fe9c4b040 msgr2=0x558fe9d64950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.883+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/675781763 shutdown_connections 2024-05-27T07:49:04.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.883+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/675781763 wait complete. 2024-05-27T07:49:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.884+0000 7f2f57b8eb80 1 -- start start 2024-05-27T07:49:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.884+0000 7f2f57b8eb80 1 Processor -- start 2024-05-27T07:49:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d73520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d73a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d73fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558fe9b8e020 con 0x558fe9beaec0 2024-05-27T07:49:04.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558fe9c287e0 con 0x558fe9d6fdb0 2024-05-27T07:49:04.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558fe9d7f090 con 0x558fe9d69e30 2024-05-27T07:49:04.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f57b8eb80 10 client.0 _ll_register_callbacks cb 0x558fe9c31cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d73520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d73520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39280/0 (socket says 172.21.15.90:39280) 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f41e00700 1 -- 172.21.15.90:0/4267299072 learned_addr learned my addr 172.21.15.90:0/4267299072 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f42800700 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d73fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.885+0000 7f2f41400700 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d73a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.886+0000 7f2f41e00700 1 -- 172.21.15.90:0/4267299072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 msgr2=0x558fe9d73a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.886+0000 7f2f41e00700 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d73a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.886+0000 7f2f41e00700 1 -- 172.21.15.90:0/4267299072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 msgr2=0x558fe9d73fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.886+0000 7f2f41e00700 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d73fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.886+0000 7f2f41e00700 1 -- 172.21.15.90:0/4267299072 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fe9bd7950 con 0x558fe9beaec0 2024-05-27T07:49:04.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.886+0000 7f2f41e00700 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d73520 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2f340aa640 tx=0x7f2f340227e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d956796613451c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:04.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.886+0000 7f2f23e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:04.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.887+0000 7f2f23e00700 1 -- 172.21.15.90:0/4267299072 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f340aa6b0 con 0x558fe9beaec0 2024-05-27T07:49:04.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.887+0000 7f2f23e00700 1 -- 172.21.15.90:0/4267299072 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2f340920f0 con 0x558fe9beaec0 2024-05-27T07:49:04.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.887+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/4267299072 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f30004830 con 0x558fe9beaec0 2024-05-27T07:49:04.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.887+0000 7f2f23e00700 1 -- 172.21.15.90:0/4267299072 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2f340aea60 con 0x558fe9beaec0 2024-05-27T07:49:04.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.887+0000 7f2f57b8eb80 10 client.16041 fetch_fsmap learned FSMap version 37 2024-05-27T07:49:04.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.887+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/4267299072 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558fe9c580e0 con 0x558fe9beaec0 2024-05-27T07:49:04.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.888+0000 7f2f23e00700 1 -- 172.21.15.90:0/4267299072 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f30004830 con 0x558fe9beaec0 2024-05-27T07:49:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.888+0000 7f2f23e00700 1 -- 172.21.15.90:0/4267299072 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f2f34090bd0 con 0x558fe9beaec0 2024-05-27T07:49:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.888+0000 7f2f57b8eb80 10 client.16041 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T07:49:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.888+0000 7f2f57b8eb80 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.888+0000 7f2f57b8eb80 20 client.16041 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.888+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558fe9d8bc90 0x558fe9d8e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.888+0000 7f2f57b8eb80 4 client.16041 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:04.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.888+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/4267299072 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fe9c99e10 con 0x558fe9d8bc90 2024-05-27T07:49:04.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.889+0000 7f2f41400700 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558fe9d8bc90 0x558fe9d8e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.890+0000 7f2f23e00700 10 client.16041 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:04.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.890+0000 7f2f41400700 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558fe9d8bc90 0x558fe9d8e0f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b342206ca814c1 server_cookie=a38027f3577f9a24 in_seq=0 out_seq=0 2024-05-27T07:49:04.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.892+0000 7f2f23e00700 1 -- 172.21.15.90:0/4267299072 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558fe9c99e10 con 0x558fe9d8bc90 2024-05-27T07:49:04.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.892+0000 7f2f23e00700 10 client.16041 handle_command_reply: tid=0 2024-05-27T07:49:04.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.892+0000 7f2f57b8eb80 1 client.16041 shutdown 2024-05-27T07:49:04.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/4267299072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 msgr2=0x558fe9d73520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:04.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d73520 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2f340aa640 tx=0x7f2f340227e0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/4267299072 shutdown_connections 2024-05-27T07:49:04.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558fe9d8bc90 0x558fe9d8e0f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d73fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d73520 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/4267299072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d73a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/4267299072 >> 172.21.15.90:0/4267299072 conn(0x558fe9c4b040 msgr2=0x558fe9d64600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:04.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f23e00700 0 client.16041 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:04.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/4267299072 shutdown_connections 2024-05-27T07:49:04.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/4267299072 wait complete. 2024-05-27T07:49:04.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.893+0000 7f2f57b8eb80 20 client.16041 trim_cache size 0 max 16384 2024-05-27T07:49:04.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.919+0000 7f2f57b8eb80 1 Processor -- start 2024-05-27T07:49:04.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.919+0000 7f2f57b8eb80 1 -- start start 2024-05-27T07:49:04.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.919+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9beaec0 0x558fe9d698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.919+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d69e30 0x558fe9d759c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.919+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d880f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.919+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441ae0e0 con 0x558fe9d69e30 2024-05-27T07:49:04.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.919+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441afc20 con 0x558fe9d6fdb0 2024-05-27T07:49:04.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.919+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441ae7b0 con 0x558fe9beaec0 2024-05-27T07:49:04.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d69e30 0x558fe9d759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f41400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d69e30 0x558fe9d759c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39284/0 (socket says 172.21.15.90:39284) 2024-05-27T07:49:04.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f41400700 1 -- 172.21.15.90:0/3885753200 learned_addr learned my addr 172.21.15.90:0/3885753200 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:04.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f42800700 1 --2- 172.21.15.90:0/3885753200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d880f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f41e00700 1 --2- 172.21.15.90:0/3885753200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9beaec0 0x558fe9d698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f41400700 1 -- 172.21.15.90:0/3885753200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9beaec0 msgr2=0x558fe9d698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:04.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f41400700 1 --2- 172.21.15.90:0/3885753200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9beaec0 0x558fe9d698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f41400700 1 -- 172.21.15.90:0/3885753200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 msgr2=0x558fe9d880f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:04.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f41400700 1 --2- 172.21.15.90:0/3885753200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d880f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f41400700 1 -- 172.21.15.90:0/3885753200 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fe9d701c0 con 0x558fe9d69e30 2024-05-27T07:49:04.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.920+0000 7f2f41e00700 1 --2- 172.21.15.90:0/3885753200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9beaec0 0x558fe9d698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:04.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.921+0000 7f2f41400700 1 --2- 172.21.15.90:0/3885753200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d69e30 0x558fe9d759c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2f30005d90 tx=0x7f2f30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b772084ce138969e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:04.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.921+0000 7f2f2ea00700 1 -- 172.21.15.90:0/3885753200 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30016040 con 0x558fe9d69e30 2024-05-27T07:49:04.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.921+0000 7f2f2ea00700 1 -- 172.21.15.90:0/3885753200 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f3000ede0 con 0x558fe9d69e30 2024-05-27T07:49:04.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.921+0000 7f2f2ea00700 1 -- 172.21.15.90:0/3885753200 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30005680 con 0x558fe9d69e30 2024-05-27T07:49:04.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.922+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/3885753200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d69e30 msgr2=0x558fe9d759c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:04.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.922+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/3885753200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d69e30 0x558fe9d759c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2f30005d90 tx=0x7f2f30009b50 comp rx=0 tx=0).stop 2024-05-27T07:49:04.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.922+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/3885753200 shutdown_connections 2024-05-27T07:49:04.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.922+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/3885753200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d880f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.922+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/3885753200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9beaec0 0x558fe9d698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.922+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/3885753200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9d69e30 0x558fe9d759c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.922+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/3885753200 >> 172.21.15.90:0/3885753200 conn(0x558fe9c4b040 msgr2=0x558fe9d73d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:04.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.923+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/3885753200 shutdown_connections 2024-05-27T07:49:04.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.923+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/3885753200 wait complete. 2024-05-27T07:49:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.923+0000 7f2f57b8eb80 1 -- start start 2024-05-27T07:49:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.924+0000 7f2f57b8eb80 1 Processor -- start 2024-05-27T07:49:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.924+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.924+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d812b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.924+0000 7f2f57b8eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.924+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441af550 con 0x558fe9beaec0 2024-05-27T07:49:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.924+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441aee80 con 0x558fe9d6fdb0 2024-05-27T07:49:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.924+0000 7f2f57b8eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441ac0c0 con 0x558fe9d69e30 2024-05-27T07:49:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.924+0000 7f2f57b8eb80 10 client.0 _ll_register_callbacks cb 0x558fe9c31cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:04.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.925+0000 7f2f41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.925+0000 7f2f41e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d846a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39292/0 (socket says 172.21.15.90:39292) 2024-05-27T07:49:04.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.925+0000 7f2f41e00700 1 -- 172.21.15.90:0/263602449 learned_addr learned my addr 172.21.15.90:0/263602449 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:04.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.925+0000 7f2f41400700 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d812b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.925+0000 7f2f42800700 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.925+0000 7f2f41e00700 1 -- 172.21.15.90:0/263602449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 msgr2=0x558fe9d812b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:04.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.925+0000 7f2f41e00700 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d812b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.925+0000 7f2f41e00700 1 -- 172.21.15.90:0/263602449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 msgr2=0x558fe9d817f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:04.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.925+0000 7f2f41e00700 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d817f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.925+0000 7f2f41e00700 1 -- 172.21.15.90:0/263602449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fe9d81d30 con 0x558fe9beaec0 2024-05-27T07:49:04.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.926+0000 7f2f41e00700 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d846a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2f340aec20 tx=0x7f2f340036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48ce3cba94eb7e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:04.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.926+0000 7f2f23e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:04.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.926+0000 7f2f23e00700 1 -- 172.21.15.90:0/263602449 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f340aa000 con 0x558fe9beaec0 2024-05-27T07:49:04.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.926+0000 7f2f23e00700 1 -- 172.21.15.90:0/263602449 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f2f340a9a50 con 0x558fe9beaec0 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.926+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/263602449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f3000ede0 con 0x558fe9beaec0 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.926+0000 7f2f23e00700 1 -- 172.21.15.90:0/263602449 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2f340aa6b0 con 0x558fe9beaec0 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.926+0000 7f2f57b8eb80 10 client.16047 fetch_fsmap learned FSMap version 37 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.926+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/263602449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558fe9df72a0 con 0x558fe9beaec0 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.927+0000 7f2f23e00700 1 -- 172.21.15.90:0/263602449 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f3000ede0 con 0x558fe9beaec0 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.927+0000 7f2f23e00700 1 -- 172.21.15.90:0/263602449 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f2f340a6020 con 0x558fe9beaec0 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.927+0000 7f2f57b8eb80 10 client.16047 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.927+0000 7f2f57b8eb80 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.927+0000 7f2f57b8eb80 20 client.16047 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.927+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558fe9df7810 0x558fe9df9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:04.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.927+0000 7f2f57b8eb80 4 client.16047 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:49:04.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.927+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/263602449 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558fe9c935f0 con 0x558fe9df7810 2024-05-27T07:49:04.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.928+0000 7f2f41400700 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558fe9df7810 0x558fe9df9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:04.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.929+0000 7f2f41400700 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558fe9df7810 0x558fe9df9c30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23c3df839238be2 server_cookie=9f979056b0aead65 in_seq=0 out_seq=0 2024-05-27T07:49:04.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.929+0000 7f2f23e00700 10 client.16047 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:04.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.932+0000 7f2f23e00700 1 -- 172.21.15.90:0/263602449 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558fe9c935f0 con 0x558fe9df7810 2024-05-27T07:49:04.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.932+0000 7f2f23e00700 10 client.16047 handle_command_reply: tid=0 2024-05-27T07:49:04.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.932+0000 7f2f57b8eb80 1 client.16047 shutdown 2024-05-27T07:49:04.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/263602449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 msgr2=0x558fe9d846a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:04.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d846a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2f340aec20 tx=0x7f2f340036f0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/263602449 shutdown_connections 2024-05-27T07:49:04.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558fe9df7810 0x558fe9df9c30 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558fe9d6fdb0 0x558fe9d817f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558fe9beaec0 0x558fe9d846a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 1 --2- 172.21.15.90:0/263602449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fe9d69e30 0x558fe9d812b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f23e00700 0 client.16047 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:04.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/263602449 >> 172.21.15.90:0/263602449 conn(0x558fe9c4b040 msgr2=0x558fe9d73d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:04.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/263602449 shutdown_connections 2024-05-27T07:49:04.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 1 -- 172.21.15.90:0/263602449 wait complete. 2024-05-27T07:49:04.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.933+0000 7f2f57b8eb80 20 client.16047 trim_cache size 0 max 16384 2024-05-27T07:49:04.935 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:49:04.935 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:49:04.935 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "9149481f-94da-436e-b03f-c586f0d5367e", 2024-05-27T07:49:04.935 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:49:04.935 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:49:04.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.934+0000 7f2f49000700 1 -- 172.21.15.90:0/917188617 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f280b2a70 msgr2=0x7f2f280b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:04.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.934+0000 7f2f49000700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f280b2a70 0x7f2f280b4f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009b10 tx=0x7f2f3c009ed0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.934+0000 7f2f49000700 1 -- 172.21.15.90:0/917188617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f44105470 msgr2=0x7f2f4419cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:04.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.934+0000 7f2f49000700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f44105470 0x7f2f4419cf10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2f34000c00 tx=0x7f2f34009f90 comp rx=0 tx=0).stop 2024-05-27T07:49:04.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.934+0000 7f2f49000700 1 -- 172.21.15.90:0/917188617 shutdown_connections 2024-05-27T07:49:04.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.934+0000 7f2f49000700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f280b2a70 0x7f2f280b4f80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.934+0000 7f2f49000700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f4410e8e0 0x7f2f441a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.934+0000 7f2f49000700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f44105470 0x7f2f4419cf10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.934+0000 7f2f49000700 1 --2- 172.21.15.90:0/917188617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f44105dc0 0x7f2f4419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:04.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.934+0000 7f2f49000700 1 -- 172.21.15.90:0/917188617 >> 172.21.15.90:0/917188617 conn(0x7f2f440fef20 msgr2=0x7f2f44103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:04.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.935+0000 7f2f49000700 1 -- 172.21.15.90:0/917188617 shutdown_connections 2024-05-27T07:49:04.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:04.935+0000 7f2f49000700 1 -- 172.21.15.90:0/917188617 wait complete. 2024-05-27T07:49:05.910 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:49:06.438 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:06.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:06 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:49:06.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:06 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:06.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:06 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:49:06.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:06 smithi090 ceph-mon[31429]: pgmap v86: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 38 op/s 2024-05-27T07:49:06.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:06 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:06.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:06 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:49:06.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:06 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:06.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:06 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:49:06.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:06 smithi136 ceph-mon[35324]: pgmap v86: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 38 op/s 2024-05-27T07:49:06.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:06 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:06.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:06 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:49:06.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:06 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:06.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:06 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:49:06.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:06 smithi129 ceph-mon[35254]: pgmap v86: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 38 op/s 2024-05-27T07:49:06.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:06 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:08.236 INFO:teuthology.orchestra.run.smithi090.stderr:2700000 of 3200000 tuples (84%) done (elapsed 84.29 s, remaining 15.61 s) 2024-05-27T07:49:08.395 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:08 smithi090 ceph-mon[31429]: pgmap v87: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 38 op/s 2024-05-27T07:49:08.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:08 smithi136 ceph-mon[35324]: pgmap v87: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 38 op/s 2024-05-27T07:49:08.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:08 smithi129 ceph-mon[35254]: pgmap v87: 129 pgs: 129 active+clean; 663 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 38 op/s 2024-05-27T07:49:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.760+0000 7f575e600700 1 -- 172.21.15.90:0/2983659377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5758106610 msgr2=0x7f5758106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.760+0000 7f575e600700 1 --2- 172.21.15.90:0/2983659377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5758106610 0x7f5758106a20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f574c0099c0 tx=0x7f574c009cd0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.761+0000 7f575e600700 1 -- 172.21.15.90:0/2983659377 shutdown_connections 2024-05-27T07:49:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.761+0000 7f575e600700 1 --2- 172.21.15.90:0/2983659377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575810b890 0x7f57581131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.761+0000 7f575e600700 1 --2- 172.21.15.90:0/2983659377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5758106610 0x7f5758106a20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.761+0000 7f575e600700 1 --2- 172.21.15.90:0/2983659377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f575810af40 0x7f575810b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.761+0000 7f575e600700 1 -- 172.21.15.90:0/2983659377 >> 172.21.15.90:0/2983659377 conn(0x7f5758102020 msgr2=0x7f5758104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.761+0000 7f575e600700 1 -- 172.21.15.90:0/2983659377 shutdown_connections 2024-05-27T07:49:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.761+0000 7f575e600700 1 -- 172.21.15.90:0/2983659377 wait complete. 2024-05-27T07:49:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.762+0000 7f575e600700 1 Processor -- start 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.762+0000 7f575e600700 1 -- start start 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f575e600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5758106610 0x7f575819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f575e600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575810af40 0x7f575819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f575e600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f575810b890 0x7f57581a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f575e600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57581163c0 con 0x7f5758106610 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f575e600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5758116240 con 0x7f575810b890 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f575e600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5758116540 con 0x7f575810af40 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f575d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5758106610 0x7f575819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f575d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5758106610 0x7f575819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39310/0 (socket says 172.21.15.90:39310) 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f575d200700 1 -- 172.21.15.90:0/788355140 learned_addr learned my addr 172.21.15.90:0/788355140 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f5757e00700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575810af40 0x7f575819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.763+0000 7f575dc00700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f575810b890 0x7f57581a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.764+0000 7f575d200700 1 -- 172.21.15.90:0/788355140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575810af40 msgr2=0x7f575819d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.764+0000 7f575d200700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575810af40 0x7f575819d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.764+0000 7f575d200700 1 -- 172.21.15.90:0/788355140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f575810b890 msgr2=0x7f57581a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.764+0000 7f575d200700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f575810b890 0x7f57581a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.764+0000 7f575d200700 1 -- 172.21.15.90:0/788355140 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57581a1ca0 con 0x7f5758106610 2024-05-27T07:49:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.764+0000 7f575d200700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5758106610 0x7f575819cdd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f574c00b580 tx=0x7f574c00f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.764+0000 7f5755600700 1 -- 172.21.15.90:0/788355140 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c027070 con 0x7f5758106610 2024-05-27T07:49:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.764+0000 7f575e600700 1 -- 172.21.15.90:0/788355140 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57581a1f30 con 0x7f5758106610 2024-05-27T07:49:08.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.764+0000 7f5755600700 1 -- 172.21.15.90:0/788355140 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f574c00fdd0 con 0x7f5758106610 2024-05-27T07:49:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.765+0000 7f5755600700 1 -- 172.21.15.90:0/788355140 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c0173f0 con 0x7f5758106610 2024-05-27T07:49:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.765+0000 7f575e600700 1 -- 172.21.15.90:0/788355140 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57581a2410 con 0x7f5758106610 2024-05-27T07:49:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.769+0000 7f5755600700 1 -- 172.21.15.90:0/788355140 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f574c02a070 con 0x7f5758106610 2024-05-27T07:49:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.770+0000 7f576e718b80 1 Processor -- start 2024-05-27T07:49:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.770+0000 7f576e718b80 1 -- start start 2024-05-27T07:49:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.770+0000 7f576e718b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 0x55edf94d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.770+0000 7f576e718b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.770+0000 7f576e718b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 0x55edf94e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.770+0000 7f576e718b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55edf9402e40 con 0x55edf9355e40 2024-05-27T07:49:08.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.770+0000 7f576e718b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55edf93f29e0 con 0x55edf94419f0 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.770+0000 7f576e718b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55edf93f6a20 con 0x55edf94df2c0 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.771+0000 7f575d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.771+0000 7f575d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94d5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39318/0 (socket says 172.21.15.90:39318) 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.771+0000 7f575d200700 1 -- 172.21.15.90:0/2649173610 learned_addr learned my addr 172.21.15.90:0/2649173610 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.771+0000 7f575dc00700 1 --2- 172.21.15.90:0/2649173610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 0x55edf94d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.771+0000 7f5757e00700 1 --2- 172.21.15.90:0/2649173610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 0x55edf94e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.772+0000 7f5755600700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57400b29f0 0x7f57400b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.772+0000 7f5757e00700 1 -- 172.21.15.90:0/2649173610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 msgr2=0x55edf94d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.772+0000 7f5757e00700 1 --2- 172.21.15.90:0/2649173610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 0x55edf94d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.772+0000 7f5757e00700 1 -- 172.21.15.90:0/2649173610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 msgr2=0x55edf94d5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.772+0000 7f5757e00700 1 --2- 172.21.15.90:0/2649173610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94d5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.772+0000 7f5757e00700 1 -- 172.21.15.90:0/2649173610 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf93bb420 con 0x55edf94df2c0 2024-05-27T07:49:08.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.772+0000 7f5755600700 1 -- 172.21.15.90:0/788355140 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f574c0e38c0 con 0x7f5758106610 2024-05-27T07:49:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.772+0000 7f575dc00700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57400b29f0 0x7f57400b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.772+0000 7f5757e00700 1 --2- 172.21.15.90:0/2649173610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 0x55edf94e16e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5758109880 tx=0x7f574400ceb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cecdeab726a4e284 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.773+0000 7f575dc00700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57400b29f0 0x7f57400b4f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55edf94d1a90 tx=0x7f5750004ad0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.773+0000 7f574a000700 1 -- 172.21.15.90:0/2649173610 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574400ece0 con 0x55edf94df2c0 2024-05-27T07:49:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.773+0000 7f574a000700 1 -- 172.21.15.90:0/2649173610 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f574400ee80 con 0x55edf94df2c0 2024-05-27T07:49:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.773+0000 7f574a000700 1 -- 172.21.15.90:0/2649173610 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574401a670 con 0x55edf94df2c0 2024-05-27T07:49:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.774+0000 7f576e718b80 1 -- 172.21.15.90:0/2649173610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 msgr2=0x55edf94e16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.774+0000 7f576e718b80 1 --2- 172.21.15.90:0/2649173610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 0x55edf94e16e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5758109880 tx=0x7f574400ceb0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.774+0000 7f576e718b80 1 -- 172.21.15.90:0/2649173610 shutdown_connections 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.774+0000 7f576e718b80 1 --2- 172.21.15.90:0/2649173610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 0x55edf94e16e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.774+0000 7f576e718b80 1 --2- 172.21.15.90:0/2649173610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 0x55edf94d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.774+0000 7f576e718b80 1 --2- 172.21.15.90:0/2649173610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94d5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.774+0000 7f576e718b80 1 -- 172.21.15.90:0/2649173610 >> 172.21.15.90:0/2649173610 conn(0x55edf93ef390 msgr2=0x55edf94d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.774+0000 7f576e718b80 1 -- 172.21.15.90:0/2649173610 shutdown_connections 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.774+0000 7f576e718b80 1 -- 172.21.15.90:0/2649173610 wait complete. 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.775+0000 7f576e718b80 1 -- start start 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.775+0000 7f576e718b80 1 Processor -- start 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f576e718b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf9355e40 0x55edf94dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f576e718b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 0x55edf94dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f576e718b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94df2c0 0x55edf94dcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f576e718b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55edf92f8fa0 con 0x55edf94df2c0 2024-05-27T07:49:08.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f576e718b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55edf9393760 con 0x55edf94419f0 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f576e718b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55edf94e1e00 con 0x55edf9355e40 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f576e718b80 10 client.0 _ll_register_callbacks cb 0x55edf939cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f575dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94df2c0 0x55edf94dcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f575dc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94df2c0 0x55edf94dcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39332/0 (socket says 172.21.15.90:39332) 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f575dc00700 1 -- 172.21.15.90:0/1815465599 learned_addr learned my addr 172.21.15.90:0/1815465599 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.776+0000 7f575d200700 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf9355e40 0x55edf94dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.777+0000 7f5757e00700 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 0x55edf94dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.777+0000 7f575dc00700 1 -- 172.21.15.90:0/1815465599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf9355e40 msgr2=0x55edf94dc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.777+0000 7f575dc00700 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf9355e40 0x55edf94dc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.777+0000 7f575dc00700 1 -- 172.21.15.90:0/1815465599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 msgr2=0x55edf94dc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.777+0000 7f575dc00700 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 0x55edf94dc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.777+0000 7f575dc00700 1 -- 172.21.15.90:0/1815465599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf93428d0 con 0x55edf94df2c0 2024-05-27T07:49:08.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.777+0000 7f5757e00700 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 0x55edf94dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.777+0000 7f575dc00700 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94df2c0 0x55edf94dcce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5750017c40 tx=0x7f5750009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686d834f39db1532 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.777+0000 7f573ea00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.778+0000 7f573ea00700 1 -- 172.21.15.90:0/1815465599 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575001d070 con 0x55edf94df2c0 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.778+0000 7f576e718b80 1 -- 172.21.15.90:0/1815465599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55edf9404d90 con 0x55edf94df2c0 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.778+0000 7f573ea00700 1 -- 172.21.15.90:0/1815465599 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f5750016c80 con 0x55edf94df2c0 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.778+0000 7f573ea00700 1 -- 172.21.15.90:0/1815465599 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f575002f050 con 0x55edf94df2c0 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.778+0000 7f576e718b80 10 client.16059 fetch_fsmap learned FSMap version 37 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.778+0000 7f576e718b80 1 -- 172.21.15.90:0/1815465599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55edf93c3060 con 0x55edf94df2c0 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.778+0000 7f573ea00700 1 -- 172.21.15.90:0/1815465599 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55edf9404d90 con 0x55edf94df2c0 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.779+0000 7f573ea00700 1 -- 172.21.15.90:0/1815465599 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f575002a030 con 0x55edf94df2c0 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.779+0000 7f576e718b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.779+0000 7f576e718b80 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.779+0000 7f576e718b80 20 client.16059 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:08.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.779+0000 7f576e718b80 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55edf94eed90 0x55edf94f1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.779+0000 7f576e718b80 4 client.16059 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.779+0000 7f576e718b80 1 -- 172.21.15.90:0/1815465599 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edf93fe570 con 0x55edf94eed90 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.780+0000 7f575d200700 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55edf94eed90 0x55edf94f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.781+0000 7f575d200700 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55edf94eed90 0x55edf94f1210 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87c41ed69ab682c server_cookie=a38100f2c6c91280 in_seq=0 out_seq=0 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.781+0000 7f573ea00700 10 client.16059 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f573ea00700 1 -- 172.21.15.90:0/1815465599 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55edf93fe570 con 0x55edf94eed90 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f573ea00700 10 client.16059 handle_command_reply: tid=0 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f576e718b80 1 client.16059 shutdown 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f576e718b80 1 -- 172.21.15.90:0/1815465599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94df2c0 msgr2=0x55edf94dcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f576e718b80 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94df2c0 0x55edf94dcce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5750017c40 tx=0x7f5750009040 comp rx=0 tx=0).stop 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f576e718b80 1 -- 172.21.15.90:0/1815465599 shutdown_connections 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f576e718b80 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55edf94eed90 0x55edf94f1210 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f576e718b80 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94df2c0 0x55edf94dcce0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f576e718b80 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf9355e40 0x55edf94dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f576e718b80 1 --2- 172.21.15.90:0/1815465599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94419f0 0x55edf94dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f573ea00700 0 client.16059 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.783+0000 7f576e718b80 1 -- 172.21.15.90:0/1815465599 >> 172.21.15.90:0/1815465599 conn(0x55edf93ef390 msgr2=0x55edf94d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.784+0000 7f576e718b80 1 -- 172.21.15.90:0/1815465599 shutdown_connections 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.784+0000 7f576e718b80 1 -- 172.21.15.90:0/1815465599 wait complete. 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.784+0000 7f576e718b80 20 client.16059 trim_cache size 0 max 16384 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.808+0000 7f576e718b80 1 Processor -- start 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.808+0000 7f576e718b80 1 -- start start 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.808+0000 7f576e718b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf9355e40 0x55edf94d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.808+0000 7f576e718b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94419f0 0x55edf94d4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.808+0000 7f576e718b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 0x55edf94eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.808+0000 7f576e718b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57581b2330 con 0x55edf94419f0 2024-05-27T07:49:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.808+0000 7f576e718b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57581b3e70 con 0x55edf9355e40 2024-05-27T07:49:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.808+0000 7f576e718b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57581b2a00 con 0x55edf94df2c0 2024-05-27T07:49:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.809+0000 7f5757e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94419f0 0x55edf94d4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.809+0000 7f5757e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94419f0 0x55edf94d4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39346/0 (socket says 172.21.15.90:39346) 2024-05-27T07:49:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.809+0000 7f5757e00700 1 -- 172.21.15.90:0/2009227352 learned_addr learned my addr 172.21.15.90:0/2009227352 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.809+0000 7f575d200700 1 --2- 172.21.15.90:0/2009227352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf9355e40 0x55edf94d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.809+0000 7f575dc00700 1 --2- 172.21.15.90:0/2009227352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 0x55edf94eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.809+0000 7f5757e00700 1 -- 172.21.15.90:0/2009227352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 msgr2=0x55edf94eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.809+0000 7f5757e00700 1 --2- 172.21.15.90:0/2009227352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 0x55edf94eaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.809+0000 7f5757e00700 1 -- 172.21.15.90:0/2009227352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf9355e40 msgr2=0x55edf94d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.809+0000 7f5757e00700 1 --2- 172.21.15.90:0/2009227352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf9355e40 0x55edf94d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.809+0000 7f5757e00700 1 -- 172.21.15.90:0/2009227352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf94ec7c0 con 0x55edf94419f0 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.810+0000 7f5757e00700 1 --2- 172.21.15.90:0/2009227352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94419f0 0x55edf94d4b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5744019000 tx=0x7f574400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2873995958e38b84 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.810+0000 7f5749600700 1 -- 172.21.15.90:0/2009227352 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744005040 con 0x55edf94419f0 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.810+0000 7f5749600700 1 -- 172.21.15.90:0/2009227352 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5744005670 con 0x55edf94419f0 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.810+0000 7f5749600700 1 -- 172.21.15.90:0/2009227352 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744010690 con 0x55edf94419f0 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.811+0000 7f576e718b80 1 -- 172.21.15.90:0/2009227352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94419f0 msgr2=0x55edf94d4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.811+0000 7f576e718b80 1 --2- 172.21.15.90:0/2009227352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94419f0 0x55edf94d4b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5744019000 tx=0x7f574400ece0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.811+0000 7f576e718b80 1 -- 172.21.15.90:0/2009227352 shutdown_connections 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.811+0000 7f576e718b80 1 --2- 172.21.15.90:0/2009227352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94df2c0 0x55edf94eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.811+0000 7f576e718b80 1 --2- 172.21.15.90:0/2009227352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf9355e40 0x55edf94d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.811+0000 7f576e718b80 1 --2- 172.21.15.90:0/2009227352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf94419f0 0x55edf94d4b50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.811+0000 7f576e718b80 1 -- 172.21.15.90:0/2009227352 >> 172.21.15.90:0/2009227352 conn(0x55edf93ef390 msgr2=0x55edf94d78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.811+0000 7f576e718b80 1 -- 172.21.15.90:0/2009227352 shutdown_connections 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.811+0000 7f576e718b80 1 -- 172.21.15.90:0/2009227352 wait complete. 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.812+0000 7f576e718b80 1 -- start start 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.812+0000 7f576e718b80 1 Processor -- start 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f576e718b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94e31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f576e718b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94419f0 0x55edf94e36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f576e718b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94df2c0 0x55edf95588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f576e718b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57581b37a0 con 0x55edf9355e40 2024-05-27T07:49:08.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f576e718b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57581b30d0 con 0x55edf94df2c0 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f576e718b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57581b02b0 con 0x55edf94419f0 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f576e718b80 10 client.0 _ll_register_callbacks cb 0x55edf939cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f575d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94e31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f575d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94e31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39348/0 (socket says 172.21.15.90:39348) 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f575d200700 1 -- 172.21.15.90:0/819901073 learned_addr learned my addr 172.21.15.90:0/819901073 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.813+0000 7f5757e00700 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94419f0 0x55edf94e36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f575dc00700 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94df2c0 0x55edf95588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f575d200700 1 -- 172.21.15.90:0/819901073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94419f0 msgr2=0x55edf94e36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f575d200700 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94419f0 0x55edf94e36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f575d200700 1 -- 172.21.15.90:0/819901073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94df2c0 msgr2=0x55edf95588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:08.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f575d200700 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94df2c0 0x55edf95588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f575d200700 1 -- 172.21.15.90:0/819901073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf94dd050 con 0x55edf9355e40 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f575d200700 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94e31a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f574c0b3010 tx=0x7f574c0e5d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c756f0ca1c5c555a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f573ea00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f573ea00700 1 -- 172.21.15.90:0/819901073 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c0aa650 con 0x55edf9355e40 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f573ea00700 1 -- 172.21.15.90:0/819901073 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f574c0e0ee0 con 0x55edf9355e40 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.814+0000 7f576e718b80 1 -- 172.21.15.90:0/819901073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5744005670 con 0x55edf9355e40 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.815+0000 7f573ea00700 1 -- 172.21.15.90:0/819901073 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f574c0a9ff0 con 0x55edf9355e40 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.815+0000 7f576e718b80 10 client.16065 fetch_fsmap learned FSMap version 37 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.815+0000 7f576e718b80 1 -- 172.21.15.90:0/819901073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55edf9559bf0 con 0x55edf9355e40 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.815+0000 7f573ea00700 1 -- 172.21.15.90:0/819901073 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5744005670 con 0x55edf9355e40 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.815+0000 7f573ea00700 1 -- 172.21.15.90:0/819901073 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f574c0aaae0 con 0x55edf9355e40 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.815+0000 7f576e718b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T07:49:08.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.815+0000 7f576e718b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.815+0000 7f576e718b80 20 client.16065 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.816+0000 7f576e718b80 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55edf955a2e0 0x55edf955c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.816+0000 7f576e718b80 4 client.16065 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.816+0000 7f576e718b80 1 -- 172.21.15.90:0/819901073 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55edf9275250 con 0x55edf955a2e0 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.816+0000 7f5757e00700 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55edf955a2e0 0x55edf955c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.817+0000 7f5757e00700 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55edf955a2e0 0x55edf955c700 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ec342d96fd225f server_cookie=e0c04edf5ff6a222 in_seq=0 out_seq=0 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.817+0000 7f573ea00700 10 client.16065 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f573ea00700 1 -- 172.21.15.90:0/819901073 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55edf9275250 con 0x55edf955a2e0 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f573ea00700 10 client.16065 handle_command_reply: tid=0 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f576e718b80 1 client.16065 shutdown 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f576e718b80 1 -- 172.21.15.90:0/819901073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 msgr2=0x55edf94e31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f576e718b80 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94e31a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f574c0b3010 tx=0x7f574c0e5d10 comp rx=0 tx=0).stop 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f576e718b80 1 -- 172.21.15.90:0/819901073 shutdown_connections 2024-05-27T07:49:08.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f576e718b80 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55edf955a2e0 0x55edf955c700 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f576e718b80 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55edf94df2c0 0x55edf95588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f576e718b80 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55edf9355e40 0x55edf94e31a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f576e718b80 1 --2- 172.21.15.90:0/819901073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55edf94419f0 0x55edf94e36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.818+0000 7f576e718b80 1 -- 172.21.15.90:0/819901073 >> 172.21.15.90:0/819901073 conn(0x55edf93ef390 msgr2=0x55edf94d78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.819+0000 7f573ea00700 0 client.16065 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.819+0000 7f576e718b80 1 -- 172.21.15.90:0/819901073 shutdown_connections 2024-05-27T07:49:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.819+0000 7f576e718b80 1 -- 172.21.15.90:0/819901073 wait complete. 2024-05-27T07:49:08.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.819+0000 7f576e718b80 20 client.16065 trim_cache size 0 max 16384 2024-05-27T07:49:08.961 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:49:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:49:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:49:08.962 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:49:08.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.820+0000 7f575e600700 1 -- 172.21.15.90:0/788355140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57400b29f0 msgr2=0x7f57400b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:08.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.820+0000 7f575e600700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57400b29f0 0x7f57400b4f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55edf94d1a90 tx=0x7f5750004ad0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.820+0000 7f575e600700 1 -- 172.21.15.90:0/788355140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5758106610 msgr2=0x7f575819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:08.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.820+0000 7f575e600700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5758106610 0x7f575819cdd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f574c00b580 tx=0x7f574c00f510 comp rx=0 tx=0).stop 2024-05-27T07:49:08.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.820+0000 7f575e600700 1 -- 172.21.15.90:0/788355140 shutdown_connections 2024-05-27T07:49:08.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.820+0000 7f575e600700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57400b29f0 0x7f57400b4f00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.820+0000 7f575e600700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f575810b890 0x7f57581a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.820+0000 7f575e600700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5758106610 0x7f575819cdd0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.820+0000 7f575e600700 1 --2- 172.21.15.90:0/788355140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f575810af40 0x7f575819d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:08.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.820+0000 7f575e600700 1 -- 172.21.15.90:0/788355140 >> 172.21.15.90:0/788355140 conn(0x7f5758102020 msgr2=0x7f575810a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:08.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.821+0000 7f575e600700 1 -- 172.21.15.90:0/788355140 shutdown_connections 2024-05-27T07:49:08.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:08.821+0000 7f575e600700 1 -- 172.21.15.90:0/788355140 wait complete. 2024-05-27T07:49:09.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:09 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:49:09.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:09 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:49:09.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:09 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:49:09.770 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:49:09.771 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:49:10.241 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:10.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:10 smithi090 ceph-mon[31429]: pgmap v88: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 129 op/s 2024-05-27T07:49:10.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:10 smithi136 ceph-mon[35324]: pgmap v88: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 129 op/s 2024-05-27T07:49:10.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:10 smithi129 ceph-mon[35254]: pgmap v88: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 18 MiB/s wr, 129 op/s 2024-05-27T07:49:11.144 INFO:teuthology.orchestra.run.smithi090.stderr:2800000 of 3200000 tuples (87%) done (elapsed 87.20 s, remaining 12.46 s) 2024-05-27T07:49:12.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:12 smithi090 ceph-mon[31429]: pgmap v89: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 121 op/s 2024-05-27T07:49:12.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:12 smithi136 ceph-mon[35324]: pgmap v89: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 121 op/s 2024-05-27T07:49:12.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:12 smithi129 ceph-mon[35254]: pgmap v89: 129 pgs: 129 active+clean; 709 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 121 op/s 2024-05-27T07:49:12.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.859+0000 7f5c5c400700 1 -- 172.21.15.90:0/709154246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c54105e30 msgr2=0x7f5c5410e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:12.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.859+0000 7f5c5c400700 1 --2- 172.21.15.90:0/709154246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c54105e30 0x7f5c5410e380 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009990 tx=0x7f5c40009ca0 comp rx=0 tx=0).stop 2024-05-27T07:49:12.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.859+0000 7f5c5c400700 1 -- 172.21.15.90:0/709154246 shutdown_connections 2024-05-27T07:49:12.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.859+0000 7f5c5c400700 1 --2- 172.21.15.90:0/709154246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5410e910 0x7f5c54110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:12.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.859+0000 7f5c5c400700 1 --2- 172.21.15.90:0/709154246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c541054e0 0x7f5c541058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:12.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.859+0000 7f5c5c400700 1 --2- 172.21.15.90:0/709154246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c54105e30 0x7f5c5410e380 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:12.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.859+0000 7f5c5c400700 1 -- 172.21.15.90:0/709154246 >> 172.21.15.90:0/709154246 conn(0x7f5c54078df0 msgr2=0x7f5c54079230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:12.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.859+0000 7f5c5c400700 1 -- 172.21.15.90:0/709154246 shutdown_connections 2024-05-27T07:49:12.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.859+0000 7f5c5c400700 1 -- 172.21.15.90:0/709154246 wait complete. 2024-05-27T07:49:12.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.861+0000 7f5c5c400700 1 Processor -- start 2024-05-27T07:49:12.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.861+0000 7f5c5c400700 1 -- start start 2024-05-27T07:49:12.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.862+0000 7f5c5c400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c541054e0 0x7f5c541031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:12.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.862+0000 7f5c5c400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54105e30 0x7f5c54103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:12.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.862+0000 7f5c5c400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5410e910 0x7f5c54103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:12.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.862+0000 7f5c5c400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5c54114320 con 0x7f5c541054e0 2024-05-27T07:49:12.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.862+0000 7f5c5c400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541141a0 con 0x7f5c5410e910 2024-05-27T07:49:12.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.862+0000 7f5c5c400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541144a0 con 0x7f5c54105e30 2024-05-27T07:49:12.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.862+0000 7f5c59a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c541054e0 0x7f5c541031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:12.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.862+0000 7f5c59a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c541054e0 0x7f5c541031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53238/0 (socket says 172.21.15.90:53238) 2024-05-27T07:49:12.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.862+0000 7f5c59a00700 1 -- 172.21.15.90:0/16670100 learned_addr learned my addr 172.21.15.90:0/16670100 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:12.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.863+0000 7f5c5a400700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5410e910 0x7f5c54103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:12.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.863+0000 7f5c59000700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54105e30 0x7f5c54103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:12.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.863+0000 7f5c59a00700 1 -- 172.21.15.90:0/16670100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54105e30 msgr2=0x7f5c54103710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:12.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.863+0000 7f5c59a00700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54105e30 0x7f5c54103710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:12.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.863+0000 7f5c59a00700 1 -- 172.21.15.90:0/16670100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5410e910 msgr2=0x7f5c54103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:12.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.863+0000 7f5c59a00700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5410e910 0x7f5c54103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:12.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.863+0000 7f5c59a00700 1 -- 172.21.15.90:0/16670100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c541a9110 con 0x7f5c541054e0 2024-05-27T07:49:12.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.863+0000 7f5c5a400700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5410e910 0x7f5c54103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:12.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.864+0000 7f5c59a00700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c541054e0 0x7f5c541031d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800d950 tx=0x7f5c4800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:12.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.864+0000 7f5c52000700 1 -- 172.21.15.90:0/16670100 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c48016070 con 0x7f5c541054e0 2024-05-27T07:49:12.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.864+0000 7f5c5c400700 1 -- 172.21.15.90:0/16670100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c541a9400 con 0x7f5c541054e0 2024-05-27T07:49:12.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.864+0000 7f5c5c400700 1 -- 172.21.15.90:0/16670100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c541a9990 con 0x7f5c541054e0 2024-05-27T07:49:12.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.864+0000 7f5c52000700 1 -- 172.21.15.90:0/16670100 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c48010460 con 0x7f5c541054e0 2024-05-27T07:49:12.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.864+0000 7f5c52000700 1 -- 172.21.15.90:0/16670100 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4800f620 con 0x7f5c541054e0 2024-05-27T07:49:12.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.866+0000 7f5c5c400700 1 -- 172.21.15.90:0/16670100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c34005330 con 0x7f5c541054e0 2024-05-27T07:49:12.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.870+0000 7f5c52000700 1 -- 172.21.15.90:0/16670100 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5c48019070 con 0x7f5c541054e0 2024-05-27T07:49:12.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.872+0000 7f5c52000700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5c3c0b29f0 0x7f5c3c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.872+0000 7f5c52000700 1 -- 172.21.15.90:0/16670100 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9408+0+0 (secure 0 0 0) 0x7f5c480e2460 con 0x7f5c541054e0 2024-05-27T07:49:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.872+0000 7f5c59000700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5c3c0b29f0 0x7f5c3c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.873+0000 7f5c59000700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5c3c0b29f0 0x7f5c3c0b4f00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c40003760 tx=0x7f5c40005fb0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:12.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:12.875+0000 7f5c52000700 1 -- 172.21.15.90:0/16670100 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5c480a9a90 con 0x7f5c541054e0 2024-05-27T07:49:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.212+0000 7f5c5c400700 1 -- 172.21.15.90:0/16670100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c34005620 con 0x7f5c541054e0 2024-05-27T07:49:13.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.214+0000 7f5c52000700 1 -- 172.21.15.90:0/16670100 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11941 (secure 0 0 0) 0x7f5c480a9390 con 0x7f5c541054e0 2024-05-27T07:49:13.214 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:49:13.214 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:48:56.900321+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25630":{"gid":25630,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6837/3741952392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":3741952392},{"type":"v1","addr":"172.21.15.136:6837","nonce":3741952392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[24478,34406,34424,34442,34472]},"id":1}]} 2024-05-27T07:49:13.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.217+0000 7f5c5c400700 1 -- 172.21.15.90:0/16670100 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5c3c0b29f0 msgr2=0x7f5c3c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:13.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.217+0000 7f5c5c400700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5c3c0b29f0 0x7f5c3c0b4f00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c40003760 tx=0x7f5c40005fb0 comp rx=0 tx=0).stop 2024-05-27T07:49:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.217+0000 7f5c5c400700 1 -- 172.21.15.90:0/16670100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c541054e0 msgr2=0x7f5c541031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.217+0000 7f5c5c400700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c541054e0 0x7f5c541031d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800d950 tx=0x7f5c4800dd10 comp rx=0 tx=0).stop 2024-05-27T07:49:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.218+0000 7f5c5c400700 1 -- 172.21.15.90:0/16670100 shutdown_connections 2024-05-27T07:49:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.218+0000 7f5c5c400700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5c3c0b29f0 0x7f5c3c0b4f00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:13.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.218+0000 7f5c5c400700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c5410e910 0x7f5c54103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:13.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.218+0000 7f5c5c400700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c541054e0 0x7f5c541031d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:13.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.218+0000 7f5c5c400700 1 --2- 172.21.15.90:0/16670100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54105e30 0x7f5c54103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:13.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.218+0000 7f5c5c400700 1 -- 172.21.15.90:0/16670100 >> 172.21.15.90:0/16670100 conn(0x7f5c54078df0 msgr2=0x7f5c54100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:13.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.218+0000 7f5c5c400700 1 -- 172.21.15.90:0/16670100 shutdown_connections 2024-05-27T07:49:13.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:13.218+0000 7f5c5c400700 1 -- 172.21.15.90:0/16670100 wait complete. 2024-05-27T07:49:13.219 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 37 2024-05-27T07:49:13.650 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:13 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 5. 2024-05-27T07:49:13.650 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:13 smithi136 systemd[1]: Stopped Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:49:13.651 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:13 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.438s CPU time. 2024-05-27T07:49:13.651 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:13 smithi136 systemd[1]: Starting Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:49:13.982 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:13 smithi136 podman[71228]: 2024-05-27 07:49:13.55660636 +0000 UTC m=+0.036281990 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:49:13.982 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:13 smithi136 podman[71228]: 2024-05-27 07:49:13.685438299 +0000 UTC m=+0.165113973 container create 0c99cc6523b4793c617676417192d222d5f9bd289fabad1b71e9f1cdc4d25f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T07:49:13.982 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:13 smithi136 podman[71228]: 2024-05-27 07:49:13.981921443 +0000 UTC m=+0.461597098 container init 0c99cc6523b4793c617676417192d222d5f9bd289fabad1b71e9f1cdc4d25f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:49:14.237 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-mon[35324]: pgmap v90: 129 pgs: 129 active+clean; 746 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 17 MiB/s wr, 128 op/s 2024-05-27T07:49:14.237 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/16670100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:49:14.237 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-mon[35324]: daemon mds.c restarted 2024-05-27T07:49:14.237 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:13 smithi136 podman[71228]: 2024-05-27 07:49:13.991746401 +0000 UTC m=+0.471422019 container start 0c99cc6523b4793c617676417192d222d5f9bd289fabad1b71e9f1cdc4d25f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:49:14.237 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 bash[71228]: 0c99cc6523b4793c617676417192d222d5f9bd289fabad1b71e9f1cdc4d25f5c 2024-05-27T07:49:14.237 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 systemd[1]: Started Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:49:14.237 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-mds[71247]: -- 172.21.15.136:0/1791399087 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f2d9b8ba40 con 0x55f2d9bd3600 2024-05-27T07:49:14.237 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: starting mds.c at 2024-05-27T07:49:14.237 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fded5400700 time 2024-05-27T07:49:14.237051+0000 2024-05-27T07:49:14.238 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:49:14.383 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:49:14.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:14 smithi090 ceph-mon[31429]: pgmap v90: 129 pgs: 129 active+clean; 746 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 17 MiB/s wr, 128 op/s 2024-05-27T07:49:14.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:14 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/16670100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:49:14.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:14 smithi090 ceph-mon[31429]: daemon mds.c restarted 2024-05-27T07:49:14.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:14.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fdeeae63ba6] 2024-05-27T07:49:14.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdeeae63dd1] 2024-05-27T07:49:14.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f29db7523d] 2024-05-27T07:49:14.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f29db60971] 2024-05-27T07:49:14.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f29db5efa0] 2024-05-27T07:49:14.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f29db5f651] 2024-05-27T07:49:14.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f29daf4347] 2024-05-27T07:49:14.501 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f29d7f087e] 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55f29d7f5f2f] 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f29d800969] 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f29db9948c] 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f29dba29cb] 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 13: (Thread::entry_wrapper()+0x43) [0x7fdeeae3272d] 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 14: (Thread::_entry_func(void*)+0xd) [0x7fdeeae32749] 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fdee99d61ca] 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 16: clone() 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.239+0000 7fded5400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fded5400700 time 2024-05-27T07:49:14.237051+0000 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:14.502 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fdeeae63ba6] 2024-05-27T07:49:14.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdeeae63dd1] 2024-05-27T07:49:14.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f29db7523d] 2024-05-27T07:49:14.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f29db60971] 2024-05-27T07:49:14.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f29db5efa0] 2024-05-27T07:49:14.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f29db5f651] 2024-05-27T07:49:14.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f29daf4347] 2024-05-27T07:49:14.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f29d7f087e] 2024-05-27T07:49:14.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55f29d7f5f2f] 2024-05-27T07:49:14.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f29d800969] 2024-05-27T07:49:14.503 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f29db9948c] 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f29dba29cb] 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 13: (Thread::entry_wrapper()+0x43) [0x7fdeeae3272d] 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 14: (Thread::_entry_func(void*)+0xd) [0x7fdeeae32749] 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fdee99d61ca] 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 16: clone() 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: *** Caught signal (Aborted) ** 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: in thread 7fded5400700 thread_name:md_log_replay 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdee99e0d20] 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2: gsignal() 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 3: abort() 2024-05-27T07:49:14.504 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fdeeae63ced] 2024-05-27T07:49:14.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdeeae63dd1] 2024-05-27T07:49:14.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f29db7523d] 2024-05-27T07:49:14.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f29db60971] 2024-05-27T07:49:14.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f29db5efa0] 2024-05-27T07:49:14.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f29db5f651] 2024-05-27T07:49:14.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f29daf4347] 2024-05-27T07:49:14.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f29d7f087e] 2024-05-27T07:49:14.505 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55f29d7f5f2f] 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f29d800969] 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f29db9948c] 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f29dba29cb] 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 16: (Thread::entry_wrapper()+0x43) [0x7fdeeae3272d] 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 17: (Thread::_entry_func(void*)+0xd) [0x7fdeeae32749] 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fdee99d61ca] 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 19: clone() 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.242+0000 7fded5400700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: in thread 7fded5400700 thread_name:md_log_replay 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdee99e0d20] 2024-05-27T07:49:14.506 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2: gsignal() 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 3: abort() 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fdeeae63ced] 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdeeae63dd1] 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f29db7523d] 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f29db60971] 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f29db5efa0] 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f29db5f651] 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f29daf4347] 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f29d7f087e] 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55f29d7f5f2f] 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f29d800969] 2024-05-27T07:49:14.507 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f29db9948c] 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f29dba29cb] 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 16: (Thread::entry_wrapper()+0x43) [0x7fdeeae3272d] 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 17: (Thread::_entry_func(void*)+0xd) [0x7fdeeae32749] 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fdee99d61ca] 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 19: clone() 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: -1> 2024-05-27T07:49:14.239+0000 7fded5400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fded5400700 time 2024-05-27T07:49:14.237051+0000 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fdeeae63ba6] 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdeeae63dd1] 2024-05-27T07:49:14.508 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f29db7523d] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f29db60971] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f29db5efa0] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f29db5f651] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f29daf4347] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f29d7f087e] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55f29d7f5f2f] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f29d800969] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f29db9948c] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f29dba29cb] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 13: (Thread::entry_wrapper()+0x43) [0x7fdeeae3272d] 2024-05-27T07:49:14.509 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 14: (Thread::_entry_func(void*)+0xd) [0x7fdeeae32749] 2024-05-27T07:49:14.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fdee99d61ca] 2024-05-27T07:49:14.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 16: clone() 2024-05-27T07:49:14.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 0> 2024-05-27T07:49:14.242+0000 7fded5400700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:49:14.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: in thread 7fded5400700 thread_name:md_log_replay 2024-05-27T07:49:14.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:14.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdee99e0d20] 2024-05-27T07:49:14.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2: gsignal() 2024-05-27T07:49:14.510 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 3: abort() 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fdeeae63ced] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdeeae63dd1] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f29db7523d] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f29db60971] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f29db5efa0] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f29db5f651] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f29daf4347] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f29d7f087e] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55f29d7f5f2f] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f29d800969] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f29db9948c] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f29dba29cb] 2024-05-27T07:49:14.511 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 16: (Thread::entry_wrapper()+0x43) [0x7fdeeae3272d] 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 17: (Thread::_entry_func(void*)+0xd) [0x7fdeeae32749] 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fdee99d61ca] 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 19: clone() 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: -1> 2024-05-27T07:49:14.239+0000 7fded5400700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fded5400700 time 2024-05-27T07:49:14.237051+0000 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fdeeae63ba6] 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdeeae63dd1] 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f29db7523d] 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55f29db60971] 2024-05-27T07:49:14.512 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 5: (SnapRealm::check_cache() const+0x13e) [0x55f29db5efa0] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f29db5f651] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f29daf4347] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f29d7f087e] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55f29d7f5f2f] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55f29d800969] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 11: (MDLog::_replay_thread()+0x1f38) [0x55f29db9948c] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55f29dba29cb] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 13: (Thread::entry_wrapper()+0x43) [0x7fdeeae3272d] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 14: (Thread::_entry_func(void*)+0xd) [0x7fdeeae32749] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fdee99d61ca] 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 16: clone() 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 0> 2024-05-27T07:49:14.242+0000 7fded5400700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:49:14.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: in thread 7fded5400700 thread_name:md_log_replay 2024-05-27T07:49:14.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:49:14.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdee99e0d20] 2024-05-27T07:49:14.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2: gsignal() 2024-05-27T07:49:14.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 3: abort() 2024-05-27T07:49:14.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fdeeae63ced] 2024-05-27T07:49:14.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fdeeae63dd1] 2024-05-27T07:49:14.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55f29db7523d] 2024-05-27T07:49:14.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55f29db60971] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 8: (SnapRealm::check_cache() const+0x13e) [0x55f29db5efa0] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55f29db5f651] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55f29daf4347] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55f29d7f087e] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55f29d7f5f2f] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55f29d800969] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 14: (MDLog::_replay_thread()+0x1f38) [0x55f29db9948c] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55f29dba29cb] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 16: (Thread::entry_wrapper()+0x43) [0x7fdeeae3272d] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 17: (Thread::_entry_func(void*)+0xd) [0x7fdeeae32749] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fdee99d61ca] 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 19: clone() 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:49:14.515 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c[71243]: 2024-05-27T07:49:14.516 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:14 smithi136 podman[71278]: 2024-05-27 07:49:14.316622558 +0000 UTC m=+0.032507413 container died 0c99cc6523b4793c617676417192d222d5f9bd289fabad1b71e9f1cdc4d25f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T07:49:14.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:14 smithi129 ceph-mon[35254]: pgmap v90: 129 pgs: 129 active+clean; 746 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 17 MiB/s wr, 128 op/s 2024-05-27T07:49:14.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:14 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/16670100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:49:14.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:14 smithi129 ceph-mon[35254]: daemon mds.c restarted 2024-05-27T07:49:14.749 INFO:teuthology.orchestra.run.smithi090.stderr:2900000 of 3200000 tuples (90%) done (elapsed 90.80 s, remaining 9.39 s) 2024-05-27T07:49:14.861 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:15.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:15 smithi090 ceph-mon[31429]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T07:49:15.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:15 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.136:6836/3868289609,v1:172.21.15.136:6837/3868289609] up:boot 2024-05-27T07:49:15.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:15 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:49:15.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:15 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:49:15.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:15 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:49:15.444 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:15 smithi136 podman[71278]: 2024-05-27 07:49:15.045391083 +0000 UTC m=+0.761275942 container remove 0c99cc6523b4793c617676417192d222d5f9bd289fabad1b71e9f1cdc4d25f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-05-27T07:49:15.444 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:15 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:49:15.444 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:15 smithi136 ceph-mon[35324]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T07:49:15.444 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:15 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.136:6836/3868289609,v1:172.21.15.136:6837/3868289609] up:boot 2024-05-27T07:49:15.444 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:15 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:49:15.444 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:15 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:49:15.444 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:15 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:49:15.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:15 smithi129 ceph-mon[35254]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T07:49:15.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:15 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.136:6836/3868289609,v1:172.21.15.136:6837/3868289609] up:boot 2024-05-27T07:49:15.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:15 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-27T07:49:15.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:15 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T07:49:15.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:15 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:49:15.749 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:15 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:49:15.749 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:15 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.364s CPU time. 2024-05-27T07:49:16.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:16 smithi090 ceph-mon[31429]: pgmap v92: 129 pgs: 129 active+clean; 746 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 125 op/s 2024-05-27T07:49:16.492 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:16 smithi136 ceph-mon[35324]: pgmap v92: 129 pgs: 129 active+clean; 746 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 125 op/s 2024-05-27T07:49:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.496+0000 7f0ba8a00640 1 -- 172.21.15.90:0/1519553593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ba41043f0 msgr2=0x7f0ba41107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.496+0000 7f0ba8a00640 1 --2- 172.21.15.90:0/1519553593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ba41043f0 0x7f0ba41107c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009ca0 tx=0x7f0b9c02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.496+0000 7f0ba8a00640 1 -- 172.21.15.90:0/1519553593 shutdown_connections 2024-05-27T07:49:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.496+0000 7f0ba8a00640 1 --2- 172.21.15.90:0/1519553593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ba41043f0 0x7f0ba41107c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.496+0000 7f0ba8a00640 1 --2- 172.21.15.90:0/1519553593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ba410a140 0x7f0ba4103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.496+0000 7f0ba8a00640 1 --2- 172.21.15.90:0/1519553593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ba4109ca0 0x7f0ba4103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.496+0000 7f0ba8a00640 1 -- 172.21.15.90:0/1519553593 >> 172.21.15.90:0/1519553593 conn(0x7f0ba4079390 msgr2=0x7f0ba40797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.496+0000 7f0ba8a00640 1 -- 172.21.15.90:0/1519553593 shutdown_connections 2024-05-27T07:49:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.496+0000 7f0ba8a00640 1 -- 172.21.15.90:0/1519553593 wait complete. 2024-05-27T07:49:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.497+0000 7f0ba8a00640 1 Processor -- start 2024-05-27T07:49:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.498+0000 7f0ba8a00640 1 -- start start 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.498+0000 7f0ba8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ba41043f0 0x7f0ba419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.498+0000 7f0ba8a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ba4109ca0 0x7f0ba419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.498+0000 7f0ba8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ba410a140 0x7f0ba419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.498+0000 7f0ba8a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4113db0 con 0x7f0ba41043f0 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.498+0000 7f0ba8a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4113c30 con 0x7f0ba4109ca0 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.498+0000 7f0ba8a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4113f30 con 0x7f0ba410a140 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.498+0000 7f0ba1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ba41043f0 0x7f0ba419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ba41043f0 0x7f0ba419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53256/0 (socket says 172.21.15.90:53256) 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba1e00640 1 -- 172.21.15.90:0/3121064217 learned_addr learned my addr 172.21.15.90:0/3121064217 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba2800640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ba410a140 0x7f0ba419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba1400640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ba4109ca0 0x7f0ba419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba1e00640 1 -- 172.21.15.90:0/3121064217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ba410a140 msgr2=0x7f0ba419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ba410a140 0x7f0ba419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba1e00640 1 -- 172.21.15.90:0/3121064217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ba4109ca0 msgr2=0x7f0ba419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ba4109ca0 0x7f0ba419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba1e00640 1 -- 172.21.15.90:0/3121064217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba419f610 con 0x7f0ba41043f0 2024-05-27T07:49:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba2800640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ba410a140 0x7f0ba419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.499+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ba41043f0 0x7f0ba419a7b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400d770 tx=0x7f0b9400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.500+0000 7f0b9aa00640 1 -- 172.21.15.90:0/3121064217 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94017070 con 0x7f0ba41043f0 2024-05-27T07:49:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.500+0000 7f0b9aa00640 1 -- 172.21.15.90:0/3121064217 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b94009c30 con 0x7f0ba41043f0 2024-05-27T07:49:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.500+0000 7f0ba8a00640 1 -- 172.21.15.90:0/3121064217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba419f900 con 0x7f0ba41043f0 2024-05-27T07:49:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.500+0000 7f0b9aa00640 1 -- 172.21.15.90:0/3121064217 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400f620 con 0x7f0ba41043f0 2024-05-27T07:49:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.500+0000 7f0ba8a00640 1 -- 172.21.15.90:0/3121064217 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba41a7280 con 0x7f0ba41043f0 2024-05-27T07:49:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.504+0000 7f0baf9f6740 1 Processor -- start 2024-05-27T07:49:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.504+0000 7f0baf9f6740 1 -- start start 2024-05-27T07:49:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.504+0000 7f0b9aa00640 1 -- 172.21.15.90:0/3121064217 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0b9401a070 con 0x7f0ba41043f0 2024-05-27T07:49:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.504+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1d6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.504+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.504+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1d71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.504+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561b6c147b90 con 0x561b6be134d0 2024-05-27T07:49:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.504+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561b6c137730 con 0x561b6be341b0 2024-05-27T07:49:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.504+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561b6c13b770 con 0x561b6be0b2f0 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.505+0000 7f0ba1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1d71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.505+0000 7f0ba1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1d71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53260/0 (socket says 172.21.15.90:53260) 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.505+0000 7f0ba1400640 1 -- 172.21.15.90:0/3000763680 learned_addr learned my addr 172.21.15.90:0/3000763680 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.505+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/3000763680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.505+0000 7f0ba2800640 1 --2- 172.21.15.90:0/3000763680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.505+0000 7f0ba1400640 1 -- 172.21.15.90:0/3000763680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 msgr2=0x561b6c1d6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.505+0000 7f0ba1400640 1 --2- 172.21.15.90:0/3000763680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1d6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.505+0000 7f0ba1400640 1 -- 172.21.15.90:0/3000763680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 msgr2=0x561b6c1d6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.505+0000 7f0ba1400640 1 --2- 172.21.15.90:0/3000763680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1d6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.505+0000 7f0ba1400640 1 -- 172.21.15.90:0/3000763680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6c0d08b0 con 0x561b6be134d0 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.506+0000 7f0ba1400640 1 --2- 172.21.15.90:0/3000763680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1d71b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000bbd0 tx=0x7f0b9000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1edeee8103f0c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:16.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.506+0000 7f0b8f000640 1 -- 172.21.15.90:0/3000763680 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90015070 con 0x561b6be134d0 2024-05-27T07:49:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.506+0000 7f0b8f000640 1 -- 172.21.15.90:0/3000763680 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b9001a8f0 con 0x561b6be134d0 2024-05-27T07:49:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.506+0000 7f0b8f000640 1 -- 172.21.15.90:0/3000763680 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900117c0 con 0x561b6be134d0 2024-05-27T07:49:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.506+0000 7f0b9aa00640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0b880b29d0 0x7f0b880b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.507+0000 7f0b9aa00640 1 -- 172.21.15.90:0/3121064217 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f0b940e2470 con 0x7f0ba41043f0 2024-05-27T07:49:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.507+0000 7f0ba2800640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0b880b29d0 0x7f0b880b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.508+0000 7f0baf9f6740 1 -- 172.21.15.90:0/3000763680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 msgr2=0x561b6c1d71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.508+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/3000763680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1d71b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000bbd0 tx=0x7f0b9000f730 comp rx=0 tx=0).stop 2024-05-27T07:49:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.508+0000 7f0ba2800640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0b880b29d0 0x7f0b880b4ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c005e30 tx=0x7f0b9c005d80 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.508+0000 7f0baf9f6740 1 -- 172.21.15.90:0/3000763680 shutdown_connections 2024-05-27T07:49:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.508+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/3000763680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1d71b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.508+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/3000763680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1d6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.508+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/3000763680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1d6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.508+0000 7f0baf9f6740 1 -- 172.21.15.90:0/3000763680 >> 172.21.15.90:0/3000763680 conn(0x561b6be28300 msgr2=0x561b6c1d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.508+0000 7f0baf9f6740 1 -- 172.21.15.90:0/3000763680 shutdown_connections 2024-05-27T07:49:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.508+0000 7f0baf9f6740 1 -- 172.21.15.90:0/3000763680 wait complete. 2024-05-27T07:49:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.509+0000 7f0baf9f6740 1 -- start start 2024-05-27T07:49:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.509+0000 7f0baf9f6740 1 Processor -- start 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.509+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be0b2f0 0x561b6c1e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be134d0 0x561b6c1e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be341b0 0x561b6c1f1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561b6be1d790 con 0x561b6be0b2f0 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561b6c0dfbc0 con 0x561b6be134d0 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561b6c1e7e50 con 0x561b6be341b0 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0baf9f6740 10 client.0 _ll_register_callbacks cb 0x561b6c0bd400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0ba1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be0b2f0 0x561b6c1e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0ba1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be0b2f0 0x561b6c1e4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53266/0 (socket says 172.21.15.90:53266) 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0ba1e00640 1 -- 172.21.15.90:0/1602759260 learned_addr learned my addr 172.21.15.90:0/1602759260 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0ba1400640 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be134d0 0x561b6c1e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.510+0000 7f0ba2800640 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be341b0 0x561b6c1f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.511+0000 7f0ba1e00640 1 -- 172.21.15.90:0/1602759260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be341b0 msgr2=0x561b6c1f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.511+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be341b0 0x561b6c1f1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.511+0000 7f0ba1e00640 1 -- 172.21.15.90:0/1602759260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be134d0 msgr2=0x561b6c1e52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.511+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be134d0 0x561b6c1e52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.511+0000 7f0ba1e00640 1 -- 172.21.15.90:0/1602759260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6be4c210 con 0x561b6be0b2f0 2024-05-27T07:49:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.511+0000 7f0ba1400640 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be134d0 0x561b6c1e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:49:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.511+0000 7f0ba2800640 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be341b0 0x561b6c1f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.511+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be0b2f0 0x561b6c1e4d90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009610 tx=0x7f0b94010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9561ccc9ba12f52a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.511+0000 7f0b83400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.512+0000 7f0b83400640 1 -- 172.21.15.90:0/1602759260 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94021030 con 0x561b6be0b2f0 2024-05-27T07:49:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.512+0000 7f0b83400640 1 -- 172.21.15.90:0/1602759260 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f0b940a6870 con 0x561b6be0b2f0 2024-05-27T07:49:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.512+0000 7f0baf9f6740 1 -- 172.21.15.90:0/1602759260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b9001a8f0 con 0x561b6be0b2f0 2024-05-27T07:49:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.512+0000 7f0b83400640 1 -- 172.21.15.90:0/1602759260 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0b94021030 con 0x561b6be0b2f0 2024-05-27T07:49:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.512+0000 7f0baf9f6740 10 client.16086 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.512+0000 7f0baf9f6740 1 -- 172.21.15.90:0/1602759260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561b6bf18620 con 0x561b6be0b2f0 2024-05-27T07:49:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.513+0000 7f0b83400640 1 -- 172.21.15.90:0/1602759260 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b940e2140 con 0x561b6be0b2f0 2024-05-27T07:49:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.513+0000 7f0b83400640 1 -- 172.21.15.90:0/1602759260 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0b940ae8e0 con 0x561b6be0b2f0 2024-05-27T07:49:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.513+0000 7f0baf9f6740 10 client.16086 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.513+0000 7f0baf9f6740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.513+0000 7f0baf9f6740 20 client.16086 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.513+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561b6bdf2d80 0x561b6c1f7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.513+0000 7f0baf9f6740 4 client.16086 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.513+0000 7f0baf9f6740 1 -- 172.21.15.90:0/1602759260 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0b9001a8f0 con 0x561b6bdf2d80 2024-05-27T07:49:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.514+0000 7f0ba1400640 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561b6bdf2d80 0x561b6c1f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.515+0000 7f0ba1400640 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561b6bdf2d80 0x561b6c1f7230 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de512f14ce92cf9a server_cookie=5d875d45cddd2ec5 in_seq=0 out_seq=0 2024-05-27T07:49:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.515+0000 7f0b83400640 10 client.16086 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.517+0000 7f0b83400640 1 -- 172.21.15.90:0/1602759260 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f0b9001a8f0 con 0x561b6bdf2d80 2024-05-27T07:49:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.517+0000 7f0b83400640 10 client.16086 handle_command_reply: tid=0 2024-05-27T07:49:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.517+0000 7f0baf9f6740 1 client.16086 shutdown 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.517+0000 7f0baf9f6740 1 -- 172.21.15.90:0/1602759260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be0b2f0 msgr2=0x561b6c1e4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.517+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be0b2f0 0x561b6c1e4d90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009610 tx=0x7f0b94010f00 comp rx=0 tx=0).stop 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.518+0000 7f0baf9f6740 1 -- 172.21.15.90:0/1602759260 shutdown_connections 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.518+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561b6bdf2d80 0x561b6c1f7230 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.518+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be134d0 0x561b6c1e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.518+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be341b0 0x561b6c1f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.518+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/1602759260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be0b2f0 0x561b6c1e4d90 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.518+0000 7f0baf9f6740 1 -- 172.21.15.90:0/1602759260 >> 172.21.15.90:0/1602759260 conn(0x561b6be28300 msgr2=0x561b6c1d65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.518+0000 7f0b83400640 0 client.16086 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.518+0000 7f0baf9f6740 1 -- 172.21.15.90:0/1602759260 shutdown_connections 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.518+0000 7f0baf9f6740 1 -- 172.21.15.90:0/1602759260 wait complete. 2024-05-27T07:49:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.518+0000 7f0baf9f6740 20 client.16086 trim_cache size 0 max 16384 2024-05-27T07:49:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.539+0000 7f0baf9f6740 1 Processor -- start 2024-05-27T07:49:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.539+0000 7f0baf9f6740 1 -- start start 2024-05-27T07:49:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.539+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1e05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.539+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.539+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1deaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.539+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8400feb0 con 0x561b6be134d0 2024-05-27T07:49:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.539+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b840119f0 con 0x561b6be341b0 2024-05-27T07:49:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.539+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84010580 con 0x561b6be0b2f0 2024-05-27T07:49:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53270/0 (socket says 172.21.15.90:53270) 2024-05-27T07:49:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba1400640 1 -- 172.21.15.90:0/639451296 learned_addr learned my addr 172.21.15.90:0/639451296 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/639451296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1e05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba2800640 1 --2- 172.21.15.90:0/639451296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1deaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba1400640 1 -- 172.21.15.90:0/639451296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 msgr2=0x561b6c1e05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba1400640 1 --2- 172.21.15.90:0/639451296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1e05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba1400640 1 -- 172.21.15.90:0/639451296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 msgr2=0x561b6c1deaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba1400640 1 --2- 172.21.15.90:0/639451296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1deaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba1400640 1 -- 172.21.15.90:0/639451296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6c1d74a0 con 0x561b6be134d0 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.540+0000 7f0ba2800640 1 --2- 172.21.15.90:0/639451296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1deaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.541+0000 7f0ba1400640 1 --2- 172.21.15.90:0/639451296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e3f60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000bac0 tx=0x7f0b900051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d29a6689e4d534 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.541+0000 7f0b8e600640 1 -- 172.21.15.90:0/639451296 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90015040 con 0x561b6be134d0 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.541+0000 7f0b8e600640 1 -- 172.21.15.90:0/639451296 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b90005540 con 0x561b6be134d0 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.541+0000 7f0b8e600640 1 -- 172.21.15.90:0/639451296 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900049e0 con 0x561b6be134d0 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.542+0000 7f0baf9f6740 1 -- 172.21.15.90:0/639451296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 msgr2=0x561b6c1e3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.542+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/639451296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e3f60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000bac0 tx=0x7f0b900051d0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.542+0000 7f0baf9f6740 1 -- 172.21.15.90:0/639451296 shutdown_connections 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.542+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/639451296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1deaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.542+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/639451296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e3f60 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.542+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/639451296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1e05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.542+0000 7f0baf9f6740 1 -- 172.21.15.90:0/639451296 >> 172.21.15.90:0/639451296 conn(0x561b6be28300 msgr2=0x561b6c1e4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.542+0000 7f0baf9f6740 1 -- 172.21.15.90:0/639451296 shutdown_connections 2024-05-27T07:49:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.542+0000 7f0baf9f6740 1 -- 172.21.15.90:0/639451296 wait complete. 2024-05-27T07:49:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.543+0000 7f0baf9f6740 1 -- start start 2024-05-27T07:49:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.543+0000 7f0baf9f6740 1 Processor -- start 2024-05-27T07:49:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.544+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1ed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.544+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.544+0000 7f0baf9f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1e8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.544+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84011320 con 0x561b6be134d0 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.544+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84010c50 con 0x561b6be341b0 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.544+0000 7f0baf9f6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8400de70 con 0x561b6be0b2f0 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.544+0000 7f0baf9f6740 10 client.0 _ll_register_callbacks cb 0x7f0b940e5c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.544+0000 7f0ba1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.544+0000 7f0ba1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53280/0 (socket says 172.21.15.90:53280) 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.544+0000 7f0ba1400640 1 -- 172.21.15.90:0/2642377748 learned_addr learned my addr 172.21.15.90:0/2642377748 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.545+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1ed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.545+0000 7f0ba2800640 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.545+0000 7f0ba1400640 1 -- 172.21.15.90:0/2642377748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 msgr2=0x561b6c1ed7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.545+0000 7f0ba1400640 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1ed7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.545+0000 7f0ba1400640 1 -- 172.21.15.90:0/2642377748 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 msgr2=0x561b6c1e8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.545+0000 7f0ba1400640 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1e8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.545+0000 7f0ba1400640 1 -- 172.21.15.90:0/2642377748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6c1e8d60 con 0x561b6be134d0 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.545+0000 7f0ba1400640 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e8130 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000f770 tx=0x7f0b9000bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50105f5905877373 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.545+0000 7f0b83400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.546+0000 7f0b83400640 1 -- 172.21.15.90:0/2642377748 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90037020 con 0x561b6be134d0 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.546+0000 7f0b83400640 1 -- 172.21.15.90:0/2642377748 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f0b90004240 con 0x561b6be134d0 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.546+0000 7f0baf9f6740 1 -- 172.21.15.90:0/2642377748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b90005540 con 0x561b6be134d0 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.546+0000 7f0b83400640 1 -- 172.21.15.90:0/2642377748 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0b90042050 con 0x561b6be134d0 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.546+0000 7f0baf9f6740 10 client.16092 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.546+0000 7f0baf9f6740 1 -- 172.21.15.90:0/2642377748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561b6c25e6c0 con 0x561b6be134d0 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.547+0000 7f0b83400640 1 -- 172.21.15.90:0/2642377748 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0b90005540 con 0x561b6be134d0 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.547+0000 7f0b83400640 1 -- 172.21.15.90:0/2642377748 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f0b90004760 con 0x561b6be134d0 2024-05-27T07:49:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.547+0000 7f0baf9f6740 10 client.16092 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.547+0000 7f0baf9f6740 10 client.16092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.547+0000 7f0baf9f6740 20 client.16092 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.547+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561b6bdc81c0 0x561b6c260c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.547+0000 7f0baf9f6740 4 client.16092 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:49:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.547+0000 7f0baf9f6740 1 -- 172.21.15.90:0/2642377748 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b6c149ae0 con 0x561b6bdc81c0 2024-05-27T07:49:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.548+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561b6bdc81c0 0x561b6c260c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.549+0000 7f0ba1e00640 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561b6bdc81c0 0x561b6c260c50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a455643db2fd2808 server_cookie=af081401ba954da9 in_seq=0 out_seq=0 2024-05-27T07:49:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.549+0000 7f0b83400640 10 client.16092 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:16.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:16 smithi129 ceph-mon[35254]: pgmap v92: 129 pgs: 129 active+clean; 746 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 125 op/s 2024-05-27T07:49:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0b83400640 1 -- 172.21.15.90:0/2642377748 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561b6c149ae0 con 0x561b6bdc81c0 2024-05-27T07:49:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0b83400640 10 client.16092 handle_command_reply: tid=0 2024-05-27T07:49:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 client.16092 shutdown 2024-05-27T07:49:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 -- 172.21.15.90:0/2642377748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 msgr2=0x561b6c1e8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e8130 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000f770 tx=0x7f0b9000bf20 comp rx=0 tx=0).stop 2024-05-27T07:49:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 -- 172.21.15.90:0/2642377748 shutdown_connections 2024-05-27T07:49:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561b6bdc81c0 0x561b6c260c50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561b6be341b0 0x561b6c1e8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561b6be134d0 0x561b6c1e8130 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 --2- 172.21.15.90:0/2642377748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561b6be0b2f0 0x561b6c1ed7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0b83400640 0 client.16092 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 -- 172.21.15.90:0/2642377748 >> 172.21.15.90:0/2642377748 conn(0x561b6be28300 msgr2=0x561b6c1e3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 -- 172.21.15.90:0/2642377748 shutdown_connections 2024-05-27T07:49:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.595+0000 7f0baf9f6740 1 -- 172.21.15.90:0/2642377748 wait complete. 2024-05-27T07:49:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.596+0000 7f0baf9f6740 20 client.16092 trim_cache size 0 max 16384 2024-05-27T07:49:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:49:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:49:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:49:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.596+0000 7f0ba8a00640 1 -- 172.21.15.90:0/3121064217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0b880b29d0 msgr2=0x7f0b880b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.596+0000 7f0ba8a00640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0b880b29d0 0x7f0b880b4ee0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c005e30 tx=0x7f0b9c005d80 comp rx=0 tx=0).stop 2024-05-27T07:49:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.596+0000 7f0ba8a00640 1 -- 172.21.15.90:0/3121064217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ba41043f0 msgr2=0x7f0ba419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.596+0000 7f0ba8a00640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ba41043f0 0x7f0ba419a7b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400d770 tx=0x7f0b9400dc30 comp rx=0 tx=0).stop 2024-05-27T07:49:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.597+0000 7f0ba8a00640 1 -- 172.21.15.90:0/3121064217 shutdown_connections 2024-05-27T07:49:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.597+0000 7f0ba8a00640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0b880b29d0 0x7f0b880b4ee0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.597+0000 7f0ba8a00640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ba4109ca0 0x7f0ba419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.597+0000 7f0ba8a00640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ba410a140 0x7f0ba419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.597+0000 7f0ba8a00640 1 --2- 172.21.15.90:0/3121064217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ba41043f0 0x7f0ba419a7b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.597+0000 7f0ba8a00640 1 -- 172.21.15.90:0/3121064217 >> 172.21.15.90:0/3121064217 conn(0x7f0ba4079390 msgr2=0x7f0ba4100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.597+0000 7f0ba8a00640 1 -- 172.21.15.90:0/3121064217 shutdown_connections 2024-05-27T07:49:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:16.597+0000 7f0ba8a00640 1 -- 172.21.15.90:0/3121064217 wait complete. 2024-05-27T07:49:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.041+0000 7f827b400700 1 -- 172.21.15.90:0/2026956532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82741034b0 msgr2=0x7f82741038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.041+0000 7f827b400700 1 --2- 172.21.15.90:0/2026956532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82741034b0 0x7f82741038c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f82680099c0 tx=0x7f8268009cd0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.041+0000 7f827b400700 1 -- 172.21.15.90:0/2026956532 shutdown_connections 2024-05-27T07:49:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.041+0000 7f827b400700 1 --2- 172.21.15.90:0/2026956532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8274108730 0x7f8274110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.041+0000 7f827b400700 1 --2- 172.21.15.90:0/2026956532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82741034b0 0x7f82741038c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.041+0000 7f827b400700 1 --2- 172.21.15.90:0/2026956532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8274107de0 0x7f82741081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.041+0000 7f827b400700 1 -- 172.21.15.90:0/2026956532 >> 172.21.15.90:0/2026956532 conn(0x7f82740fedb0 msgr2=0x7f82741011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.041+0000 7f827b400700 1 -- 172.21.15.90:0/2026956532 shutdown_connections 2024-05-27T07:49:17.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.041+0000 7f827b400700 1 -- 172.21.15.90:0/2026956532 wait complete. 2024-05-27T07:49:17.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.043+0000 7f827b400700 1 Processor -- start 2024-05-27T07:49:17.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.043+0000 7f827b400700 1 -- start start 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.043+0000 7f827b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82741034b0 0x7f827419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.043+0000 7f827b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8274107de0 0x7f827419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.043+0000 7f827b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8274108730 0x7f82741a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.043+0000 7f827b400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8274113260 con 0x7f8274108730 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.043+0000 7f827b400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f82741130e0 con 0x7f8274107de0 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.043+0000 7f827b400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82741133e0 con 0x7f82741034b0 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.044+0000 7f8279400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8274108730 0x7f82741a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.044+0000 7f8278a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82741034b0 0x7f827419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.044+0000 7f8278a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82741034b0 0x7f827419d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33008/0 (socket says 172.21.15.90:33008) 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.044+0000 7f8278a00700 1 -- 172.21.15.90:0/1988519474 learned_addr learned my addr 172.21.15.90:0/1988519474 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:17.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.044+0000 7f8273e00700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8274107de0 0x7f827419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.044+0000 7f8279400700 1 -- 172.21.15.90:0/1988519474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82741034b0 msgr2=0x7f827419d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.044+0000 7f8279400700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82741034b0 0x7f827419d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.044+0000 7f8279400700 1 -- 172.21.15.90:0/1988519474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8274107de0 msgr2=0x7f827419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.045+0000 7f8279400700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8274107de0 0x7f827419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.045+0000 7f8279400700 1 -- 172.21.15.90:0/1988519474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82741a1f00 con 0x7f8274108730 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.045+0000 7f8278a00700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82741034b0 0x7f827419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.045+0000 7f8279400700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8274108730 0x7f82741a1860 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f826c00efc0 tx=0x7f826c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.045+0000 7f8271600700 1 -- 172.21.15.90:0/1988519474 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0092e0 con 0x7f8274108730 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.045+0000 7f8271600700 1 -- 172.21.15.90:0/1988519474 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f826c004bb0 con 0x7f8274108730 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.045+0000 7f827b400700 1 -- 172.21.15.90:0/1988519474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82741a2190 con 0x7f8274108730 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.046+0000 7f8271600700 1 -- 172.21.15.90:0/1988519474 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c004020 con 0x7f8274108730 2024-05-27T07:49:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.046+0000 7f827b400700 1 -- 172.21.15.90:0/1988519474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82741a9b10 con 0x7f8274108730 2024-05-27T07:49:17.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.051+0000 7f8271600700 1 -- 172.21.15.90:0/1988519474 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f826c01b030 con 0x7f8274108730 2024-05-27T07:49:17.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.051+0000 7f8289e7db80 1 Processor -- start 2024-05-27T07:49:17.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.051+0000 7f8289e7db80 1 -- start start 2024-05-27T07:49:17.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 0x556c079837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c0797d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c0797e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8289e7db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556c078abe40 con 0x556c07983d30 2024-05-27T07:49:17.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8289e7db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556c0789b9e0 con 0x556c077fee40 2024-05-27T07:49:17.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8289e7db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556c0789fa20 con 0x556c0797ddb0 2024-05-27T07:49:17.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8278a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c0797d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8278a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c0797d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53306/0 (socket says 172.21.15.90:53306) 2024-05-27T07:49:17.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8278a00700 1 -- 172.21.15.90:0/1000670220 learned_addr learned my addr 172.21.15.90:0/1000670220 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:17.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8273e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 0x556c079837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.052+0000 7f8273e00700 1 --2- 172.21.15.90:0/1000670220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c0797e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.053+0000 7f8273e00700 1 -- 172.21.15.90:0/1000670220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 msgr2=0x556c0797e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:17.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.053+0000 7f8273e00700 1 --2- 172.21.15.90:0/1000670220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c0797e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.053+0000 7f8273e00700 1 -- 172.21.15.90:0/1000670220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 msgr2=0x556c0797d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:17.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.053+0000 7f8273e00700 1 --2- 172.21.15.90:0/1000670220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c0797d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.053+0000 7f8273e00700 1 -- 172.21.15.90:0/1000670220 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c07864420 con 0x556c077fee40 2024-05-27T07:49:17.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.053+0000 7f8271600700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f825c0b2a70 0x7f825c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.053+0000 7f8278a00700 1 --2- 172.21.15.90:0/1000670220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c0797d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:49:17.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.054+0000 7f8271600700 1 -- 172.21.15.90:0/1988519474 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f826c0e2410 con 0x7f8274108730 2024-05-27T07:49:17.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.054+0000 7f8273e00700 1 --2- 172.21.15.90:0/1000670220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 0x556c079837f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82741066f0 tx=0x7f8264005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a0b4da7b0027b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:17.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.054+0000 7f8278a00700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f825c0b2a70 0x7f825c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.054+0000 7f8262000700 1 -- 172.21.15.90:0/1000670220 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264012e40 con 0x556c077fee40 2024-05-27T07:49:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.054+0000 7f8262000700 1 -- 172.21.15.90:0/1000670220 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82640044c0 con 0x556c077fee40 2024-05-27T07:49:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.054+0000 7f8262000700 1 -- 172.21.15.90:0/1000670220 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826400f3f0 con 0x556c077fee40 2024-05-27T07:49:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.056+0000 7f8278a00700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f825c0b2a70 0x7f825c0b4f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f826800b910 tx=0x7f826801a040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.056+0000 7f8289e7db80 1 -- 172.21.15.90:0/1000670220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 msgr2=0x556c079837f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:17.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.056+0000 7f8289e7db80 1 --2- 172.21.15.90:0/1000670220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 0x556c079837f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82741066f0 tx=0x7f8264005d20 comp rx=0 tx=0).stop 2024-05-27T07:49:17.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.056+0000 7f8289e7db80 1 -- 172.21.15.90:0/1000670220 shutdown_connections 2024-05-27T07:49:17.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.056+0000 7f8289e7db80 1 --2- 172.21.15.90:0/1000670220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c0797e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.056+0000 7f8289e7db80 1 --2- 172.21.15.90:0/1000670220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 0x556c079837f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.056+0000 7f8289e7db80 1 --2- 172.21.15.90:0/1000670220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c0797d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.056+0000 7f8289e7db80 1 -- 172.21.15.90:0/1000670220 >> 172.21.15.90:0/1000670220 conn(0x556c0785efc0 msgr2=0x556c079788d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:17.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.056+0000 7f8289e7db80 1 -- 172.21.15.90:0/1000670220 shutdown_connections 2024-05-27T07:49:17.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.056+0000 7f8289e7db80 1 -- 172.21.15.90:0/1000670220 wait complete. 2024-05-27T07:49:17.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.057+0000 7f8289e7db80 1 -- start start 2024-05-27T07:49:17.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.057+0000 7f8289e7db80 1 Processor -- start 2024-05-27T07:49:17.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.058+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 0x556c07987560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.058+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c07987aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.058+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07987fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.058+0000 7f8289e7db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556c077a1fa0 con 0x556c07983d30 2024-05-27T07:49:17.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.058+0000 7f8289e7db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556c0783c760 con 0x556c077fee40 2024-05-27T07:49:17.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.058+0000 7f8289e7db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556c0798a800 con 0x556c0797ddb0 2024-05-27T07:49:17.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.058+0000 7f8289e7db80 10 client.0 _ll_register_callbacks cb 0x556c07845c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:17.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8278a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 0x556c07987560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8278a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 0x556c07987560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56110/0 (socket says 172.21.15.90:56110) 2024-05-27T07:49:17.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8278a00700 1 -- 172.21.15.90:0/940644147 learned_addr learned my addr 172.21.15.90:0/940644147 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:17.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8279400700 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07987fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8273e00700 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c07987aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8279400700 1 -- 172.21.15.90:0/940644147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 msgr2=0x556c07987aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:17.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8279400700 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c07987aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8279400700 1 -- 172.21.15.90:0/940644147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 msgr2=0x556c07987560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:17.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8279400700 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 0x556c07987560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8279400700 1 -- 172.21.15.90:0/940644147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c077eb8d0 con 0x556c07983d30 2024-05-27T07:49:17.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.059+0000 7f8273e00700 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c07987aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:17.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.060+0000 7f8279400700 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07987fe0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f826c0df7f0 tx=0x7f826c0a98c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5692375bd8c6962 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:17.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.060+0000 7f8256a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:17.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.060+0000 7f8256a00700 1 -- 172.21.15.90:0/940644147 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c004d60 con 0x556c07983d30 2024-05-27T07:49:17.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.060+0000 7f8256a00700 1 -- 172.21.15.90:0/940644147 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f826c0a85e0 con 0x556c07983d30 2024-05-27T07:49:17.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.060+0000 7f8289e7db80 1 -- 172.21.15.90:0/940644147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82640044c0 con 0x556c07983d30 2024-05-27T07:49:17.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.061+0000 7f8256a00700 1 -- 172.21.15.90:0/940644147 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f826c0a8e90 con 0x556c07983d30 2024-05-27T07:49:17.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.061+0000 7f8289e7db80 10 client.16104 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:17.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.061+0000 7f8289e7db80 1 -- 172.21.15.90:0/940644147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556c0786c060 con 0x556c07983d30 2024-05-27T07:49:17.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.062+0000 7f8256a00700 1 -- 172.21.15.90:0/940644147 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f82640044c0 con 0x556c07983d30 2024-05-27T07:49:17.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.062+0000 7f8256a00700 1 -- 172.21.15.90:0/940644147 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f826c0a8050 con 0x556c07983d30 2024-05-27T07:49:17.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.062+0000 7f8289e7db80 10 client.16104 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:17.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.062+0000 7f8289e7db80 10 client.16104 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:49:17.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.062+0000 7f8289e7db80 20 client.16104 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:17.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.062+0000 7f8289e7db80 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c079977b0 0x556c07999bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.062+0000 7f8289e7db80 4 client.16104 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:17.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.062+0000 7f8289e7db80 1 -- 172.21.15.90:0/940644147 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c078add90 con 0x556c079977b0 2024-05-27T07:49:17.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.063+0000 7f8278a00700 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c079977b0 0x556c07999bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.064+0000 7f8278a00700 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c079977b0 0x556c07999bf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9f6baab72d7dae9d server_cookie=d59d09839b46761d in_seq=0 out_seq=0 2024-05-27T07:49:17.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.064+0000 7f8256a00700 10 client.16104 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:49:17.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8256a00700 1 -- 172.21.15.90:0/940644147 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556c078add90 con 0x556c079977b0 2024-05-27T07:49:17.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8256a00700 10 client.16104 handle_command_reply: tid=0 2024-05-27T07:49:17.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8289e7db80 1 client.16104 shutdown 2024-05-27T07:49:17.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8289e7db80 1 -- 172.21.15.90:0/940644147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 msgr2=0x556c07987fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:17.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8289e7db80 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07987fe0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f826c0df7f0 tx=0x7f826c0a98c0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8289e7db80 1 -- 172.21.15.90:0/940644147 shutdown_connections 2024-05-27T07:49:17.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8289e7db80 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c079977b0 0x556c07999bf0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8289e7db80 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07987fe0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8289e7db80 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c077fee40 0x556c07987560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8289e7db80 1 --2- 172.21.15.90:0/940644147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c07987aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8256a00700 0 client.16104 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:49:17.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.066+0000 7f8289e7db80 1 -- 172.21.15.90:0/940644147 >> 172.21.15.90:0/940644147 conn(0x556c0785efc0 msgr2=0x556c0797e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:17.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.067+0000 7f8289e7db80 1 -- 172.21.15.90:0/940644147 shutdown_connections 2024-05-27T07:49:17.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.067+0000 7f8289e7db80 1 -- 172.21.15.90:0/940644147 wait complete. 2024-05-27T07:49:17.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.068+0000 7f8289e7db80 20 client.16104 trim_cache size 0 max 16384 2024-05-27T07:49:17.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.086+0000 7f8289e7db80 1 Processor -- start 2024-05-27T07:49:17.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.086+0000 7f8289e7db80 1 -- start start 2024-05-27T07:49:17.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.086+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c077fee40 0x556c0797d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.086+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c0797ddb0 0x556c07986880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.086+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07981850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.086+0000 7f8289e7db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f82741b2710 con 0x556c07983d30 2024-05-27T07:49:17.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.086+0000 7f8289e7db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f82741b4250 con 0x556c0797ddb0 2024-05-27T07:49:17.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.086+0000 7f8289e7db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82741b2de0 con 0x556c077fee40 2024-05-27T07:49:17.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8279400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07981850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8279400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07981850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53326/0 (socket says 172.21.15.90:53326) 2024-05-27T07:49:17.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8279400700 1 -- 172.21.15.90:0/2070945825 learned_addr learned my addr 172.21.15.90:0/2070945825 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8278a00700 1 --2- 172.21.15.90:0/2070945825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c077fee40 0x556c0797d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8273e00700 1 --2- 172.21.15.90:0/2070945825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c0797ddb0 0x556c07986880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8279400700 1 -- 172.21.15.90:0/2070945825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c077fee40 msgr2=0x556c0797d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8279400700 1 --2- 172.21.15.90:0/2070945825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c077fee40 0x556c0797d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8279400700 1 -- 172.21.15.90:0/2070945825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c0797ddb0 msgr2=0x556c07986880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8279400700 1 --2- 172.21.15.90:0/2070945825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c0797ddb0 0x556c07986880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8279400700 1 -- 172.21.15.90:0/2070945825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c07985ad0 con 0x556c07983d30 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8278a00700 1 --2- 172.21.15.90:0/2070945825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c077fee40 0x556c0797d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.087+0000 7f8279400700 1 --2- 172.21.15.90:0/2070945825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07981850 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f826c0a9030 tx=0x7f826c0a8d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee5c775e3806ed3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.088+0000 7f8261600700 1 -- 172.21.15.90:0/2070945825 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0ca340 con 0x556c07983d30 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.088+0000 7f8261600700 1 -- 172.21.15.90:0/2070945825 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f826c0a9a80 con 0x556c07983d30 2024-05-27T07:49:17.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.088+0000 7f8261600700 1 -- 172.21.15.90:0/2070945825 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0a4cf0 con 0x556c07983d30 2024-05-27T07:49:17.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.089+0000 7f8289e7db80 1 -- 172.21.15.90:0/2070945825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 msgr2=0x556c07981850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:17.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.089+0000 7f8289e7db80 1 --2- 172.21.15.90:0/2070945825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07981850 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f826c0a9030 tx=0x7f826c0a8d80 comp rx=0 tx=0).stop 2024-05-27T07:49:17.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.089+0000 7f8289e7db80 1 -- 172.21.15.90:0/2070945825 shutdown_connections 2024-05-27T07:49:17.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.089+0000 7f8289e7db80 1 --2- 172.21.15.90:0/2070945825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c07983d30 0x556c07981850 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.089+0000 7f8289e7db80 1 --2- 172.21.15.90:0/2070945825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c077fee40 0x556c0797d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.089+0000 7f8289e7db80 1 --2- 172.21.15.90:0/2070945825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c0797ddb0 0x556c07986880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.089+0000 7f8289e7db80 1 -- 172.21.15.90:0/2070945825 >> 172.21.15.90:0/2070945825 conn(0x556c0785efc0 msgr2=0x556c07981140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:17.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.089+0000 7f8289e7db80 1 -- 172.21.15.90:0/2070945825 shutdown_connections 2024-05-27T07:49:17.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.090+0000 7f8289e7db80 1 -- 172.21.15.90:0/2070945825 wait complete. 2024-05-27T07:49:17.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.090+0000 7f8289e7db80 1 -- start start 2024-05-27T07:49:17.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.091+0000 7f8289e7db80 1 Processor -- start 2024-05-27T07:49:17.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.091+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c077fee40 0x556c0798fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.091+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c07990190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.091+0000 7f8289e7db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c07983d30 0x556c0797c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.091+0000 7f8289e7db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f82741b3b80 con 0x556c077fee40 2024-05-27T07:49:17.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.091+0000 7f8289e7db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f82741b34b0 con 0x556c07983d30 2024-05-27T07:49:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.091+0000 7f8289e7db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82741b0690 con 0x556c0797ddb0 2024-05-27T07:49:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.091+0000 7f8289e7db80 10 client.0 _ll_register_callbacks cb 0x556c07845c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8278a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c077fee40 0x556c0798fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8278a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c077fee40 0x556c0798fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53336/0 (socket says 172.21.15.90:53336) 2024-05-27T07:49:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8278a00700 1 -- 172.21.15.90:0/2320931469 learned_addr learned my addr 172.21.15.90:0/2320931469 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8273e00700 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c07990190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8279400700 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c07983d30 0x556c0797c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8278a00700 1 -- 172.21.15.90:0/2320931469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 msgr2=0x556c07990190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8278a00700 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c07990190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8278a00700 1 -- 172.21.15.90:0/2320931469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c07983d30 msgr2=0x556c0797c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8278a00700 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c07983d30 0x556c0797c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8278a00700 1 -- 172.21.15.90:0/2320931469 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0797cf90 con 0x556c077fee40 2024-05-27T07:49:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8273e00700 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c07990190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8278a00700 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c077fee40 0x556c0798fc50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f826800b580 tx=0x7f8268009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dba6478394ecc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.092+0000 7f8256a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.093+0000 7f8256a00700 1 -- 172.21.15.90:0/2320931469 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826803b070 con 0x556c077fee40 2024-05-27T07:49:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.093+0000 7f8256a00700 1 -- 172.21.15.90:0/2320931469 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f8268024470 con 0x556c077fee40 2024-05-27T07:49:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.093+0000 7f8289e7db80 1 -- 172.21.15.90:0/2320931469 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f826c0a9a80 con 0x556c077fee40 2024-05-27T07:49:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.093+0000 7f8256a00700 1 -- 172.21.15.90:0/2320931469 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8268047050 con 0x556c077fee40 2024-05-27T07:49:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.093+0000 7f8289e7db80 10 client.16110 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.093+0000 7f8289e7db80 1 -- 172.21.15.90:0/2320931469 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556c07a01c80 con 0x556c077fee40 2024-05-27T07:49:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.093+0000 7f8256a00700 1 -- 172.21.15.90:0/2320931469 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f826c0a9a80 con 0x556c077fee40 2024-05-27T07:49:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.094+0000 7f8256a00700 1 -- 172.21.15.90:0/2320931469 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f8268034720 con 0x556c077fee40 2024-05-27T07:49:17.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.094+0000 7f8289e7db80 10 client.16110 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:17.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.094+0000 7f8289e7db80 10 client.16110 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:49:17.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.094+0000 7f8289e7db80 20 client.16110 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:17.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.094+0000 7f8289e7db80 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c07a02310 0x556c07a04730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:17.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.094+0000 7f8289e7db80 4 client.16110 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:49:17.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.094+0000 7f8289e7db80 1 -- 172.21.15.90:0/2320931469 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c078a7570 con 0x556c07a02310 2024-05-27T07:49:17.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.095+0000 7f8273e00700 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c07a02310 0x556c07a04730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:17.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.096+0000 7f8256a00700 10 client.16110 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:49:17.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.096+0000 7f8273e00700 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c07a02310 0x556c07a04730 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=67dff10f9f184fc6 server_cookie=368098d38f6122d7 in_seq=0 out_seq=0 2024-05-27T07:49:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.096+0000 7f8256a00700 1 -- 172.21.15.90:0/2320931469 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c078a7570 con 0x556c07a02310 2024-05-27T07:49:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.096+0000 7f8256a00700 10 client.16110 handle_command_reply: tid=0 2024-05-27T07:49:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.096+0000 7f8289e7db80 1 client.16110 shutdown 2024-05-27T07:49:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 1 -- 172.21.15.90:0/2320931469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c077fee40 msgr2=0x556c0798fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c077fee40 0x556c0798fc50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f826800b580 tx=0x7f8268009cd0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 1 -- 172.21.15.90:0/2320931469 shutdown_connections 2024-05-27T07:49:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c07a02310 0x556c07a04730 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c07983d30 0x556c0797c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c077fee40 0x556c0798fc50 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 1 --2- 172.21.15.90:0/2320931469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c0797ddb0 0x556c07990190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 1 -- 172.21.15.90:0/2320931469 >> 172.21.15.90:0/2320931469 conn(0x556c0785efc0 msgr2=0x556c079865e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:17.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8256a00700 0 client.16110 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:49:17.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 1 -- 172.21.15.90:0/2320931469 shutdown_connections 2024-05-27T07:49:17.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 1 -- 172.21.15.90:0/2320931469 wait complete. 2024-05-27T07:49:17.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.097+0000 7f8289e7db80 20 client.16110 trim_cache size 0 max 16384 2024-05-27T07:49:17.098 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:49:17.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 -- 172.21.15.90:0/1988519474 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f825c0b2a70 msgr2=0x7f825c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:17.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f825c0b2a70 0x7f825c0b4f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f826800b910 tx=0x7f826801a040 comp rx=0 tx=0).stop 2024-05-27T07:49:17.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 -- 172.21.15.90:0/1988519474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8274108730 msgr2=0x7f82741a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:17.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8274108730 0x7f82741a1860 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f826c00efc0 tx=0x7f826c00c3a0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 -- 172.21.15.90:0/1988519474 shutdown_connections 2024-05-27T07:49:17.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8274108730 0x7f82741a1860 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82741034b0 0x7f827419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f825c0b2a70 0x7f825c0b4f80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 --2- 172.21.15.90:0/1988519474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8274107de0 0x7f827419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:17.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 -- 172.21.15.90:0/1988519474 >> 172.21.15.90:0/1988519474 conn(0x7f82740fedb0 msgr2=0x7f8274106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:17.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 -- 172.21.15.90:0/1988519474 shutdown_connections 2024-05-27T07:49:17.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:17.098+0000 7f827b400700 1 -- 172.21.15.90:0/1988519474 wait complete. 2024-05-27T07:49:17.975 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:49:18.006 INFO:teuthology.orchestra.run.smithi090.stderr:3000000 of 3200000 tuples (93%) done (elapsed 94.06 s, remaining 6.27 s) 2024-05-27T07:49:18.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:17 smithi090 ceph-mon[31429]: pgmap v93: 129 pgs: 129 active+clean; 746 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 125 op/s 2024-05-27T07:49:18.249 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:17 smithi136 ceph-mon[35324]: pgmap v93: 129 pgs: 129 active+clean; 746 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 125 op/s 2024-05-27T07:49:18.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:17 smithi129 ceph-mon[35254]: pgmap v93: 129 pgs: 129 active+clean; 746 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 14 MiB/s wr, 125 op/s 2024-05-27T07:49:18.447 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:20.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:20 smithi090 ceph-mon[31429]: pgmap v94: 129 pgs: 129 active+clean; 782 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:49:20.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:20 smithi129 ceph-mon[35254]: pgmap v94: 129 pgs: 129 active+clean; 782 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:49:20.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:20 smithi136 ceph-mon[35324]: pgmap v94: 129 pgs: 129 active+clean; 782 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:49:21.289 INFO:teuthology.orchestra.run.smithi090.stderr:3100000 of 3200000 tuples (96%) done (elapsed 97.34 s, remaining 3.14 s) 2024-05-27T07:49:22.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.209+0000 7f2d66a00700 1 -- 172.21.15.90:0/3593602723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d60106630 msgr2=0x7f2d60106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:22.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.209+0000 7f2d66a00700 1 --2- 172.21.15.90:0/3593602723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d60106630 0x7f2d60106a40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2d540099c0 tx=0x7f2d54009cd0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.209+0000 7f2d66a00700 1 -- 172.21.15.90:0/3593602723 shutdown_connections 2024-05-27T07:49:22.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.209+0000 7f2d66a00700 1 --2- 172.21.15.90:0/3593602723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6010b8b0 0x7f2d60113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.209+0000 7f2d66a00700 1 --2- 172.21.15.90:0/3593602723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d60106630 0x7f2d60106a40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.209+0000 7f2d66a00700 1 --2- 172.21.15.90:0/3593602723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d6010af60 0x7f2d6010b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.209+0000 7f2d66a00700 1 -- 172.21.15.90:0/3593602723 >> 172.21.15.90:0/3593602723 conn(0x7f2d60101ec0 msgr2=0x7f2d60104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:22.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.209+0000 7f2d66a00700 1 -- 172.21.15.90:0/3593602723 shutdown_connections 2024-05-27T07:49:22.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.210+0000 7f2d66a00700 1 -- 172.21.15.90:0/3593602723 wait complete. 2024-05-27T07:49:22.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:22 smithi090 ceph-mon[31429]: pgmap v95: 129 pgs: 129 active+clean; 782 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:49:22.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.211+0000 7f2d66a00700 1 Processor -- start 2024-05-27T07:49:22.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.211+0000 7f2d66a00700 1 -- start start 2024-05-27T07:49:22.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.212+0000 7f2d66a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d60106630 0x7f2d6019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.212+0000 7f2d66a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d6010af60 0x7f2d6019d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.212+0000 7f2d66a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6010b8b0 0x7f2d601a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.212+0000 7f2d66a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2d60116400 con 0x7f2d6010af60 2024-05-27T07:49:22.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.212+0000 7f2d66a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2d60116280 con 0x7f2d60106630 2024-05-27T07:49:22.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.212+0000 7f2d66a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2d60116580 con 0x7f2d6010b8b0 2024-05-27T07:49:22.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.212+0000 7f2d64c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d6010af60 0x7f2d6019d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.212+0000 7f2d64c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d6010af60 0x7f2d6019d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56268/0 (socket says 172.21.15.90:56268) 2024-05-27T07:49:22.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.212+0000 7f2d64c00700 1 -- 172.21.15.90:0/614394819 learned_addr learned my addr 172.21.15.90:0/614394819 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:22.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.212+0000 7f2d65600700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d60106630 0x7f2d6019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.213+0000 7f2d66000700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6010b8b0 0x7f2d601a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.213+0000 7f2d64c00700 1 -- 172.21.15.90:0/614394819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6010b8b0 msgr2=0x7f2d601a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:22.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.213+0000 7f2d64c00700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6010b8b0 0x7f2d601a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.213+0000 7f2d64c00700 1 -- 172.21.15.90:0/614394819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d60106630 msgr2=0x7f2d6019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:22.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.213+0000 7f2d64c00700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d60106630 0x7f2d6019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.213+0000 7f2d64c00700 1 -- 172.21.15.90:0/614394819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d601a1c60 con 0x7f2d6010af60 2024-05-27T07:49:22.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.213+0000 7f2d65600700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d60106630 0x7f2d6019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:22.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.213+0000 7f2d66000700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6010b8b0 0x7f2d601a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:22.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.213+0000 7f2d64c00700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d6010af60 0x7f2d6019d330 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00d980 tx=0x7f2d4c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:22.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.214+0000 7f2d5e000700 1 -- 172.21.15.90:0/614394819 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c017070 con 0x7f2d6010af60 2024-05-27T07:49:22.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.214+0000 7f2d5e000700 1 -- 172.21.15.90:0/614394819 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d4c010460 con 0x7f2d6010af60 2024-05-27T07:49:22.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.214+0000 7f2d66a00700 1 -- 172.21.15.90:0/614394819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d601a1f50 con 0x7f2d6010af60 2024-05-27T07:49:22.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.214+0000 7f2d5e000700 1 -- 172.21.15.90:0/614394819 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c00de60 con 0x7f2d6010af60 2024-05-27T07:49:22.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.214+0000 7f2d66a00700 1 -- 172.21.15.90:0/614394819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d601a9870 con 0x7f2d6010af60 2024-05-27T07:49:22.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.219+0000 7f2d5e000700 1 -- 172.21.15.90:0/614394819 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2d4c00f620 con 0x7f2d6010af60 2024-05-27T07:49:22.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.219+0000 7f2d76adbb80 1 Processor -- start 2024-05-27T07:49:22.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.219+0000 7f2d76adbb80 1 -- start start 2024-05-27T07:49:22.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.220+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e057e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.220+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e057ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.220+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 0x5562e058a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.220+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5562e04abe40 con 0x5562e03fee40 2024-05-27T07:49:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.220+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5562e049b9e0 con 0x5562e04ea9f0 2024-05-27T07:49:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.220+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5562e049fa20 con 0x5562e05882c0 2024-05-27T07:49:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.220+0000 7f2d65600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e057ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.220+0000 7f2d65600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e057ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56284/0 (socket says 172.21.15.90:56284) 2024-05-27T07:49:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.220+0000 7f2d65600700 1 -- 172.21.15.90:0/911251961 learned_addr learned my addr 172.21.15.90:0/911251961 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d64c00700 1 --2- 172.21.15.90:0/911251961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 0x5562e058a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d66000700 1 --2- 172.21.15.90:0/911251961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e057e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d65600700 1 -- 172.21.15.90:0/911251961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 msgr2=0x5562e058a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:22.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d65600700 1 --2- 172.21.15.90:0/911251961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 0x5562e058a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d65600700 1 -- 172.21.15.90:0/911251961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 msgr2=0x5562e057e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d65600700 1 --2- 172.21.15.90:0/911251961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e057e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d65600700 1 -- 172.21.15.90:0/911251961 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e0464420 con 0x5562e03fee40 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d64c00700 1 --2- 172.21.15.90:0/911251961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 0x5562e058a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d5e000700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2d480b2a70 0x7f2d480b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d5e000700 1 -- 172.21.15.90:0/614394819 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f2d4c0e2450 con 0x7f2d6010af60 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.221+0000 7f2d65600700 1 --2- 172.21.15.90:0/911251961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e057ec90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2d540099c0 tx=0x7f2d540096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1269457f33cbab server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.222+0000 7f2d66000700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2d480b2a70 0x7f2d480b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.222+0000 7f2d52a00700 1 -- 172.21.15.90:0/911251961 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54026070 con 0x5562e03fee40 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.222+0000 7f2d52a00700 1 -- 172.21.15.90:0/911251961 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d5400b790 con 0x5562e03fee40 2024-05-27T07:49:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.222+0000 7f2d52a00700 1 -- 172.21.15.90:0/911251961 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d540038e0 con 0x5562e03fee40 2024-05-27T07:49:22.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.223+0000 7f2d76adbb80 1 -- 172.21.15.90:0/911251961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 msgr2=0x5562e057ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:22.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.223+0000 7f2d66000700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2d480b2a70 0x7f2d480b4f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5562e057aa90 tx=0x7f2d58009040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:22.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.223+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/911251961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e057ec90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2d540099c0 tx=0x7f2d540096c0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.223+0000 7f2d76adbb80 1 -- 172.21.15.90:0/911251961 shutdown_connections 2024-05-27T07:49:22.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.223+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/911251961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 0x5562e058a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.223+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/911251961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e057e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.223+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/911251961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e057ec90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.223+0000 7f2d76adbb80 1 -- 172.21.15.90:0/911251961 >> 172.21.15.90:0/911251961 conn(0x5562e0498390 msgr2=0x5562e0579710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:22.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.223+0000 7f2d76adbb80 1 -- 172.21.15.90:0/911251961 shutdown_connections 2024-05-27T07:49:22.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.224+0000 7f2d76adbb80 1 -- 172.21.15.90:0/911251961 wait complete. 2024-05-27T07:49:22.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.224+0000 7f2d76adbb80 1 -- start start 2024-05-27T07:49:22.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.225+0000 7f2d76adbb80 1 Processor -- start 2024-05-27T07:49:22.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.225+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e0585350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.225+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e0585890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.225+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 0x5562e0585dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.225+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5562e03a1fa0 con 0x5562e03fee40 2024-05-27T07:49:22.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.225+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5562e043c760 con 0x5562e04ea9f0 2024-05-27T07:49:22.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.225+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5562e058ada0 con 0x5562e05882c0 2024-05-27T07:49:22.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.225+0000 7f2d76adbb80 10 client.0 _ll_register_callbacks cb 0x5562e0445c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:22.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.226+0000 7f2d65600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e0585350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.226+0000 7f2d65600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e0585350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56286/0 (socket says 172.21.15.90:56286) 2024-05-27T07:49:22.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.226+0000 7f2d65600700 1 -- 172.21.15.90:0/2273388296 learned_addr learned my addr 172.21.15.90:0/2273388296 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.226+0000 7f2d64c00700 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e0585890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.226+0000 7f2d66000700 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 0x5562e0585dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.226+0000 7f2d65600700 1 -- 172.21.15.90:0/2273388296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 msgr2=0x5562e0585dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.226+0000 7f2d65600700 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 0x5562e0585dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.226+0000 7f2d65600700 1 -- 172.21.15.90:0/2273388296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 msgr2=0x5562e0585890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.226+0000 7f2d65600700 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e0585890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.226+0000 7f2d65600700 1 -- 172.21.15.90:0/2273388296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e03eb8d0 con 0x5562e03fee40 2024-05-27T07:49:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.227+0000 7f2d65600700 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e0585350 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2d54024040 tx=0x7f2d5400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18f3faec1020af9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.227+0000 7f2d47400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:22.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.227+0000 7f2d47400700 1 -- 172.21.15.90:0/2273388296 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5402f060 con 0x5562e03fee40 2024-05-27T07:49:22.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.227+0000 7f2d76adbb80 1 -- 172.21.15.90:0/2273388296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d5400b790 con 0x5562e03fee40 2024-05-27T07:49:22.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.227+0000 7f2d47400700 1 -- 172.21.15.90:0/2273388296 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f2d5400bdc0 con 0x5562e03fee40 2024-05-27T07:49:22.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.227+0000 7f2d47400700 1 -- 172.21.15.90:0/2273388296 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2d54036050 con 0x5562e03fee40 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.227+0000 7f2d76adbb80 10 client.16122 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.227+0000 7f2d76adbb80 1 -- 172.21.15.90:0/2273388296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5562e046c060 con 0x5562e03fee40 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.228+0000 7f2d47400700 1 -- 172.21.15.90:0/2273388296 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d5400b790 con 0x5562e03fee40 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.228+0000 7f2d47400700 1 -- 172.21.15.90:0/2273388296 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f2d5400f8e0 con 0x5562e03fee40 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.228+0000 7f2d76adbb80 10 client.16122 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.228+0000 7f2d76adbb80 10 client.16122 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.228+0000 7f2d76adbb80 20 client.16122 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.228+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5562e0597c50 0x5562e059a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.228+0000 7f2d76adbb80 4 client.16122 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.228+0000 7f2d76adbb80 1 -- 172.21.15.90:0/2273388296 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562e04add90 con 0x5562e0597c50 2024-05-27T07:49:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.229+0000 7f2d64c00700 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5562e0597c50 0x5562e059a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.230+0000 7f2d64c00700 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5562e0597c50 0x5562e059a070 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f31bb164f1cbc1b3 server_cookie=fb2d3dc2c1b27726 in_seq=0 out_seq=0 2024-05-27T07:49:22.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.230+0000 7f2d47400700 10 client.16122 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:49:22.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d47400700 1 -- 172.21.15.90:0/2273388296 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5562e04add90 con 0x5562e0597c50 2024-05-27T07:49:22.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d47400700 10 client.16122 handle_command_reply: tid=0 2024-05-27T07:49:22.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d76adbb80 1 client.16122 shutdown 2024-05-27T07:49:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d76adbb80 1 -- 172.21.15.90:0/2273388296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 msgr2=0x5562e0585350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e0585350 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2d54024040 tx=0x7f2d5400bc20 comp rx=0 tx=0).stop 2024-05-27T07:49:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d76adbb80 1 -- 172.21.15.90:0/2273388296 shutdown_connections 2024-05-27T07:49:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5562e0597c50 0x5562e059a070 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e05882c0 0x5562e0585dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e03fee40 0x5562e0585350 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/2273388296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e0585890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d76adbb80 1 -- 172.21.15.90:0/2273388296 >> 172.21.15.90:0/2273388296 conn(0x5562e0498390 msgr2=0x5562e0579300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:22.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.232+0000 7f2d47400700 0 client.16122 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:49:22.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.233+0000 7f2d76adbb80 1 -- 172.21.15.90:0/2273388296 shutdown_connections 2024-05-27T07:49:22.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.233+0000 7f2d76adbb80 1 -- 172.21.15.90:0/2273388296 wait complete. 2024-05-27T07:49:22.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.233+0000 7f2d76adbb80 20 client.16122 trim_cache size 0 max 16384 2024-05-27T07:49:22.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.251+0000 7f2d76adbb80 1 Processor -- start 2024-05-27T07:49:22.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.251+0000 7f2d76adbb80 1 -- start start 2024-05-27T07:49:22.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.252+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e03fee40 0x5562e057db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.252+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e04ea9f0 0x5562e0579740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.252+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e0594070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.252+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601b24f0 con 0x5562e05882c0 2024-05-27T07:49:22.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.252+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601b4030 con 0x5562e03fee40 2024-05-27T07:49:22.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.252+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601b2bc0 con 0x5562e04ea9f0 2024-05-27T07:49:22.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d66000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e0594070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d66000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e0594070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56302/0 (socket says 172.21.15.90:56302) 2024-05-27T07:49:22.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d66000700 1 -- 172.21.15.90:0/1302838823 learned_addr learned my addr 172.21.15.90:0/1302838823 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:22.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d65600700 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e03fee40 0x5562e057db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d64c00700 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e04ea9f0 0x5562e0579740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d66000700 1 -- 172.21.15.90:0/1302838823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e04ea9f0 msgr2=0x5562e0579740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:22.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d66000700 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e04ea9f0 0x5562e0579740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d66000700 1 -- 172.21.15.90:0/1302838823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e03fee40 msgr2=0x5562e057db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:22.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d66000700 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e03fee40 0x5562e057db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d66000700 1 -- 172.21.15.90:0/1302838823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e05945b0 con 0x5562e05882c0 2024-05-27T07:49:22.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d64c00700 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e04ea9f0 0x5562e0579740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:22.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.253+0000 7f2d65600700 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e03fee40 0x5562e057db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:22.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.254+0000 7f2d66000700 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e0594070 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009360 tx=0x7f2d58014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8793674c1e7a1694 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:22.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.254+0000 7f2d52000700 1 -- 172.21.15.90:0/1302838823 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5801d040 con 0x5562e05882c0 2024-05-27T07:49:22.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.254+0000 7f2d52000700 1 -- 172.21.15.90:0/1302838823 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d58027070 con 0x5562e05882c0 2024-05-27T07:49:22.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.254+0000 7f2d52000700 1 -- 172.21.15.90:0/1302838823 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58022e00 con 0x5562e05882c0 2024-05-27T07:49:22.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.255+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1302838823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 msgr2=0x5562e0594070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:22.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.255+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e0594070 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009360 tx=0x7f2d58014000 comp rx=0 tx=0).stop 2024-05-27T07:49:22.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.255+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1302838823 shutdown_connections 2024-05-27T07:49:22.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.255+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e0594070 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.255+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e03fee40 0x5562e057db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.255+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/1302838823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e04ea9f0 0x5562e0579740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.255+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1302838823 >> 172.21.15.90:0/1302838823 conn(0x5562e0498390 msgr2=0x5562e05807c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:22.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.255+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1302838823 shutdown_connections 2024-05-27T07:49:22.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.256+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1302838823 wait complete. 2024-05-27T07:49:22.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.256+0000 7f2d76adbb80 1 -- start start 2024-05-27T07:49:22.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.256+0000 7f2d76adbb80 1 Processor -- start 2024-05-27T07:49:22.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e03fee40 0x5562e059a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e059ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d76adbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e05fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601b3960 con 0x5562e05882c0 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601b3290 con 0x5562e04ea9f0 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d76adbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2d601b0470 con 0x5562e03fee40 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d76adbb80 10 client.0 _ll_register_callbacks cb 0x5562e0445c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d66000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e05fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d66000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e05fa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56310/0 (socket says 172.21.15.90:56310) 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d66000700 1 -- 172.21.15.90:0/1810513951 learned_addr learned my addr 172.21.15.90:0/1810513951 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.257+0000 7f2d65600700 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e03fee40 0x5562e059a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.258+0000 7f2d64c00700 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e059ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.258+0000 7f2d66000700 1 -- 172.21.15.90:0/1810513951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e03fee40 msgr2=0x5562e059a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:22.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.258+0000 7f2d66000700 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e03fee40 0x5562e059a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.258+0000 7f2d66000700 1 -- 172.21.15.90:0/1810513951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 msgr2=0x5562e059ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.258+0000 7f2d66000700 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e059ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.258+0000 7f2d66000700 1 -- 172.21.15.90:0/1810513951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e05faa10 con 0x5562e05882c0 2024-05-27T07:49:22.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.258+0000 7f2d64c00700 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e059ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.258+0000 7f2d66000700 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e05fa480 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2d58022cb0 tx=0x7f2d58022ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acaeea53ec7b769a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.258+0000 7f2d47400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.259+0000 7f2d47400700 1 -- 172.21.15.90:0/1810513951 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58007d80 con 0x5562e05882c0 2024-05-27T07:49:22.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.259+0000 7f2d47400700 1 -- 172.21.15.90:0/1810513951 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f2d58004ce0 con 0x5562e05882c0 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.259+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1810513951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d58027070 con 0x5562e05882c0 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.259+0000 7f2d47400700 1 -- 172.21.15.90:0/1810513951 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2d58022410 con 0x5562e05882c0 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.259+0000 7f2d76adbb80 10 client.16128 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.259+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1810513951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5562e0601d10 con 0x5562e05882c0 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.260+0000 7f2d47400700 1 -- 172.21.15.90:0/1810513951 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d58027070 con 0x5562e05882c0 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.260+0000 7f2d47400700 1 -- 172.21.15.90:0/1810513951 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f2d58033400 con 0x5562e05882c0 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.260+0000 7f2d76adbb80 10 client.16128 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.260+0000 7f2d76adbb80 10 client.16128 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.260+0000 7f2d76adbb80 20 client.16128 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.260+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5562e06022c0 0x5562e06046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.260+0000 7f2d76adbb80 4 client.16128 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.260+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1810513951 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562e04a7570 con 0x5562e06022c0 2024-05-27T07:49:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.261+0000 7f2d65600700 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5562e06022c0 0x5562e06046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.262+0000 7f2d65600700 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5562e06022c0 0x5562e06046e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21569bc9f4c1ca3d server_cookie=26f9f1282aa7d8d7 in_seq=0 out_seq=0 2024-05-27T07:49:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.262+0000 7f2d47400700 10 client.16128 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:49:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.262+0000 7f2d47400700 1 -- 172.21.15.90:0/1810513951 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562e04a7570 con 0x5562e06022c0 2024-05-27T07:49:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.262+0000 7f2d47400700 10 client.16128 handle_command_reply: tid=0 2024-05-27T07:49:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.262+0000 7f2d76adbb80 1 client.16128 shutdown 2024-05-27T07:49:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1810513951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 msgr2=0x5562e05fa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e05fa480 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2d58022cb0 tx=0x7f2d58022ce0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1810513951 shutdown_connections 2024-05-27T07:49:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5562e06022c0 0x5562e06046e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5562e05882c0 0x5562e05fa480 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562e03fee40 0x5562e059a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 1 --2- 172.21.15.90:0/1810513951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5562e04ea9f0 0x5562e059ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d47400700 0 client.16128 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:49:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1810513951 >> 172.21.15.90:0/1810513951 conn(0x5562e0498390 msgr2=0x5562e0585130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1810513951 shutdown_connections 2024-05-27T07:49:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 1 -- 172.21.15.90:0/1810513951 wait complete. 2024-05-27T07:49:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.263+0000 7f2d76adbb80 20 client.16128 trim_cache size 0 max 16384 2024-05-27T07:49:22.264 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:49:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.264+0000 7f2d66a00700 1 -- 172.21.15.90:0/614394819 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2d480b2a70 msgr2=0x7f2d480b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.264+0000 7f2d66a00700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2d480b2a70 0x7f2d480b4f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5562e057aa90 tx=0x7f2d58009040 comp rx=0 tx=0).stop 2024-05-27T07:49:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.264+0000 7f2d66a00700 1 -- 172.21.15.90:0/614394819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d6010af60 msgr2=0x7f2d6019d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.264+0000 7f2d66a00700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d6010af60 0x7f2d6019d330 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00d980 tx=0x7f2d4c00dd40 comp rx=0 tx=0).stop 2024-05-27T07:49:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.264+0000 7f2d66a00700 1 -- 172.21.15.90:0/614394819 shutdown_connections 2024-05-27T07:49:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.264+0000 7f2d66a00700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6010b8b0 0x7f2d601a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.264+0000 7f2d66a00700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2d480b2a70 0x7f2d480b4f80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.264+0000 7f2d66a00700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d60106630 0x7f2d6019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.264+0000 7f2d66a00700 1 --2- 172.21.15.90:0/614394819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d6010af60 0x7f2d6019d330 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.264+0000 7f2d66a00700 1 -- 172.21.15.90:0/614394819 >> 172.21.15.90:0/614394819 conn(0x7f2d60101ec0 msgr2=0x7f2d6010a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.265+0000 7f2d66a00700 1 -- 172.21.15.90:0/614394819 shutdown_connections 2024-05-27T07:49:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:22.265+0000 7f2d66a00700 1 -- 172.21.15.90:0/614394819 wait complete. 2024-05-27T07:49:22.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:22 smithi129 ceph-mon[35254]: pgmap v95: 129 pgs: 129 active+clean; 782 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:49:22.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:22 smithi136 ceph-mon[35324]: pgmap v95: 129 pgs: 129 active+clean; 782 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:49:23.174 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:49:23.686 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:24.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:24 smithi090 ceph-mon[31429]: pgmap v96: 129 pgs: 129 active+clean; 818 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:49:24.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:24 smithi129 ceph-mon[35254]: pgmap v96: 129 pgs: 129 active+clean; 818 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:49:24.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:24 smithi136 ceph-mon[35324]: pgmap v96: 129 pgs: 129 active+clean; 818 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 24 op/s 2024-05-27T07:49:24.615 INFO:teuthology.orchestra.run.smithi090.stderr:3200000 of 3200000 tuples (100%) done (elapsed 100.67 s, remaining 0.00 s) 2024-05-27T07:49:24.971 INFO:teuthology.orchestra.run.smithi090.stderr:vacuuming... 2024-05-27T07:49:25.999 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:25 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 6. 2024-05-27T07:49:26.000 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:25 smithi136 systemd[1]: Stopped Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:49:26.000 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:25 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Consumed 1.364s CPU time. 2024-05-27T07:49:26.000 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:25 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Start request repeated too quickly. 2024-05-27T07:49:26.000 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:25 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-05-27T07:49:26.000 INFO:journalctl@ceph.mds.c.smithi136.stdout:May 27 07:49:25 smithi136 systemd[1]: Failed to start Ceph mds.c for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:49:26.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.038+0000 7f895e800700 1 -- 172.21.15.90:0/2449402166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f895810b100 msgr2=0x7f8958117610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:26.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.038+0000 7f895e800700 1 --2- 172.21.15.90:0/2449402166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f895810b100 0x7f8958117610 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f895000b0a0 tx=0x7f895000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.038+0000 7f895e800700 1 -- 172.21.15.90:0/2449402166 shutdown_connections 2024-05-27T07:49:26.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.038+0000 7f895e800700 1 --2- 172.21.15.90:0/2449402166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f895810b100 0x7f8958117610 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.038+0000 7f895e800700 1 --2- 172.21.15.90:0/2449402166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f895810f6b0 0x7f895810fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.038+0000 7f895e800700 1 --2- 172.21.15.90:0/2449402166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f895810a7b0 0x7f895810abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.038+0000 7f895e800700 1 -- 172.21.15.90:0/2449402166 >> 172.21.15.90:0/2449402166 conn(0x7f8958076af0 msgr2=0x7f8958076f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:26.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.038+0000 7f895e800700 1 -- 172.21.15.90:0/2449402166 shutdown_connections 2024-05-27T07:49:26.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.039+0000 7f895e800700 1 -- 172.21.15.90:0/2449402166 wait complete. 2024-05-27T07:49:26.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.040+0000 7f895e800700 1 Processor -- start 2024-05-27T07:49:26.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.040+0000 7f895e800700 1 -- start start 2024-05-27T07:49:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.040+0000 7f895e800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f895810a7b0 0x7f89581a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.040+0000 7f895e800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f895810b100 0x7f89581a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.040+0000 7f895e800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f895810f6b0 0x7f89581a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895e800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f895811a790 con 0x7f895810f6b0 2024-05-27T07:49:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895e800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f895811a610 con 0x7f895810a7b0 2024-05-27T07:49:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895e800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f895811a910 con 0x7f895810b100 2024-05-27T07:49:26.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f895810f6b0 0x7f89581a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f895810f6b0 0x7f89581a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56320/0 (socket says 172.21.15.90:56320) 2024-05-27T07:49:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895de00700 1 -- 172.21.15.90:0/3251255285 learned_addr learned my addr 172.21.15.90:0/3251255285 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895d400700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f895810a7b0 0x7f89581a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895ca00700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f895810b100 0x7f89581a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895de00700 1 -- 172.21.15.90:0/3251255285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f895810b100 msgr2=0x7f89581a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895de00700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f895810b100 0x7f89581a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895de00700 1 -- 172.21.15.90:0/3251255285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f895810a7b0 msgr2=0x7f89581a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895de00700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f895810a7b0 0x7f89581a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895de00700 1 -- 172.21.15.90:0/3251255285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89581a5fb0 con 0x7f895810f6b0 2024-05-27T07:49:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.041+0000 7f895d400700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f895810a7b0 0x7f89581a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.042+0000 7f895de00700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f895810f6b0 0x7f89581a5910 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f895000b070 tx=0x7f8950004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:26.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.042+0000 7f8956000700 1 -- 172.21.15.90:0/3251255285 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950021070 con 0x7f895810f6b0 2024-05-27T07:49:26.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.042+0000 7f8956000700 1 -- 172.21.15.90:0/3251255285 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8950007c00 con 0x7f895810f6b0 2024-05-27T07:49:26.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.042+0000 7f895e800700 1 -- 172.21.15.90:0/3251255285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89581a6240 con 0x7f895810f6b0 2024-05-27T07:49:26.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.042+0000 7f8956000700 1 -- 172.21.15.90:0/3251255285 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950004860 con 0x7f895810f6b0 2024-05-27T07:49:26.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.044+0000 7f895e800700 1 -- 172.21.15.90:0/3251255285 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89581a6720 con 0x7f895810f6b0 2024-05-27T07:49:26.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.047+0000 7f8956000700 1 -- 172.21.15.90:0/3251255285 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8950003fb0 con 0x7f895810f6b0 2024-05-27T07:49:26.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.049+0000 7f896e8b4b80 1 Processor -- start 2024-05-27T07:49:26.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.049+0000 7f896e8b4b80 1 -- start start 2024-05-27T07:49:26.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.049+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.049+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.049+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.049+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55847b4f4e40 con 0x55847b5d12c0 2024-05-27T07:49:26.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.049+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55847b4e49e0 con 0x55847b447e40 2024-05-27T07:49:26.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.049+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55847b4e8a20 con 0x55847b5339f0 2024-05-27T07:49:26.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.050+0000 7f8956000700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f89400b29f0 0x7f89400b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.050+0000 7f895ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.050+0000 7f895ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5d36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56324/0 (socket says 172.21.15.90:56324) 2024-05-27T07:49:26.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.050+0000 7f895ca00700 1 -- 172.21.15.90:0/1943643414 learned_addr learned my addr 172.21.15.90:0/1943643414 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:26.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.050+0000 7f895ca00700 1 --2- 172.21.15.90:0/1943643414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.050+0000 7f895d400700 1 --2- 172.21.15.90:0/1943643414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.051+0000 7f895de00700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f89400b29f0 0x7f89400b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.051+0000 7f895ca00700 1 -- 172.21.15.90:0/1943643414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 msgr2=0x55847b5c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:26.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.051+0000 7f895ca00700 1 --2- 172.21.15.90:0/1943643414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.051+0000 7f895ca00700 1 -- 172.21.15.90:0/1943643414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 msgr2=0x55847b5c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:26.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.051+0000 7f895ca00700 1 --2- 172.21.15.90:0/1943643414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.051+0000 7f8956000700 1 -- 172.21.15.90:0/3251255285 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f895000f020 con 0x7f895810f6b0 2024-05-27T07:49:26.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.051+0000 7f895ca00700 1 -- 172.21.15.90:0/1943643414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847b4ad420 con 0x55847b5d12c0 2024-05-27T07:49:26.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.052+0000 7f895ca00700 1 --2- 172.21.15.90:0/1943643414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5d36e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f89440046c0 tx=0x7f894400c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceac7b4c80f72e19 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:26.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.052+0000 7f895de00700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f89400b29f0 0x7f89400b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f89500a6a80 tx=0x7f895000b3b0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.055+0000 7f894aa00700 1 -- 172.21.15.90:0/1943643414 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89440142d0 con 0x55847b5d12c0 2024-05-27T07:49:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.055+0000 7f894aa00700 1 -- 172.21.15.90:0/1943643414 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8944011990 con 0x55847b5d12c0 2024-05-27T07:49:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.055+0000 7f894aa00700 1 -- 172.21.15.90:0/1943643414 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89440126b0 con 0x55847b5d12c0 2024-05-27T07:49:26.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.056+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1943643414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 msgr2=0x55847b5d36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:26.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.056+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/1943643414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5d36e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f89440046c0 tx=0x7f894400c4f0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.056+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1943643414 shutdown_connections 2024-05-27T07:49:26.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.056+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/1943643414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5d36e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.056+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/1943643414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.057+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/1943643414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.057+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1943643414 >> 172.21.15.90:0/1943643414 conn(0x55847b4e1390 msgr2=0x55847b5c2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:26.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.057+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1943643414 shutdown_connections 2024-05-27T07:49:26.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.057+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1943643414 wait complete. 2024-05-27T07:49:26.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.057+0000 7f896e8b4b80 1 -- start start 2024-05-27T07:49:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.058+0000 7f896e8b4b80 1 Processor -- start 2024-05-27T07:49:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.058+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.058+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.058+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.058+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55847b3eafa0 con 0x55847b5d12c0 2024-05-27T07:49:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.058+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55847b485760 con 0x55847b447e40 2024-05-27T07:49:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.058+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55847b5d3da0 con 0x55847b5339f0 2024-05-27T07:49:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.058+0000 7f896e8b4b80 10 client.0 _ll_register_callbacks cb 0x55847b48ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.059+0000 7f895ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.059+0000 7f895ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56330/0 (socket says 172.21.15.90:56330) 2024-05-27T07:49:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.059+0000 7f895ca00700 1 -- 172.21.15.90:0/1098805394 learned_addr learned my addr 172.21.15.90:0/1098805394 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.059+0000 7f895d400700 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.059+0000 7f895ca00700 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.060+0000 7f895ca00700 1 -- 172.21.15.90:0/1098805394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 msgr2=0x55847b5ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.060+0000 7f895ca00700 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.060+0000 7f895ca00700 1 -- 172.21.15.90:0/1098805394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 msgr2=0x55847b5ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.060+0000 7f895ca00700 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.060+0000 7f895ca00700 1 -- 172.21.15.90:0/1098805394 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847b4348d0 con 0x55847b5d12c0 2024-05-27T07:49:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.060+0000 7f895ca00700 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5cecc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f894400ce00 tx=0x7f8944016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd89476c893dfa78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.060+0000 7f893f400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.060+0000 7f893f400700 1 -- 172.21.15.90:0/1098805394 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944011500 con 0x55847b5d12c0 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.060+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1098805394 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8944011990 con 0x55847b5d12c0 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.060+0000 7f893f400700 1 -- 172.21.15.90:0/1098805394 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f89440116a0 con 0x55847b5d12c0 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.061+0000 7f893f400700 1 -- 172.21.15.90:0/1098805394 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f894402f480 con 0x55847b5d12c0 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.061+0000 7f896e8b4b80 10 client.16140 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.061+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1098805394 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55847b4b5060 con 0x55847b5d12c0 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.061+0000 7f893f400700 1 -- 172.21.15.90:0/1098805394 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8944011990 con 0x55847b5d12c0 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.062+0000 7f893f400700 1 -- 172.21.15.90:0/1098805394 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f894401f420 con 0x55847b5d12c0 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.062+0000 7f896e8b4b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.062+0000 7f896e8b4b80 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.062+0000 7f896e8b4b80 20 client.16140 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.062+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55847b5e0ca0 0x55847b5e30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.062+0000 7f896e8b4b80 4 client.16140 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.062+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1098805394 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55847b4f6d90 con 0x55847b5e0ca0 2024-05-27T07:49:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.063+0000 7f895d400700 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55847b5e0ca0 0x55847b5e30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.064+0000 7f895d400700 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55847b5e0ca0 0x55847b5e30c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ab7e9c8d5cbcc29 server_cookie=1d7a4c389a5b935e in_seq=0 out_seq=0 2024-05-27T07:49:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.064+0000 7f893f400700 10 client.16140 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:49:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f893f400700 1 -- 172.21.15.90:0/1098805394 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55847b4f6d90 con 0x55847b5e0ca0 2024-05-27T07:49:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f893f400700 10 client.16140 handle_command_reply: tid=0 2024-05-27T07:49:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f896e8b4b80 1 client.16140 shutdown 2024-05-27T07:49:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1098805394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 msgr2=0x55847b5cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5cecc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f894400ce00 tx=0x7f8944016040 comp rx=0 tx=0).stop 2024-05-27T07:49:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1098805394 shutdown_connections 2024-05-27T07:49:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55847b5e0ca0 0x55847b5e30c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5cecc0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f893f400700 0 client.16140 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:49:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/1098805394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.066+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1098805394 >> 172.21.15.90:0/1098805394 conn(0x55847b4e1390 msgr2=0x55847b5c2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.067+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1098805394 shutdown_connections 2024-05-27T07:49:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.067+0000 7f896e8b4b80 1 -- 172.21.15.90:0/1098805394 wait complete. 2024-05-27T07:49:26.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.067+0000 7f896e8b4b80 20 client.16140 trim_cache size 0 max 16384 2024-05-27T07:49:26.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.085+0000 7f896e8b4b80 1 Processor -- start 2024-05-27T07:49:26.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.085+0000 7f896e8b4b80 1 -- start start 2024-05-27T07:49:26.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.085+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.085+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5dced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.085+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5e8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.085+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f89581b6700 con 0x55847b5d12c0 2024-05-27T07:49:26.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.085+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f89581b8240 con 0x55847b447e40 2024-05-27T07:49:26.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.085+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f89581b6dd0 con 0x55847b5339f0 2024-05-27T07:49:26.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.086+0000 7f895ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.086+0000 7f895ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5e8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56340/0 (socket says 172.21.15.90:56340) 2024-05-27T07:49:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.086+0000 7f895ca00700 1 -- 172.21.15.90:0/540735923 learned_addr learned my addr 172.21.15.90:0/540735923 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.086+0000 7f895d400700 1 --2- 172.21.15.90:0/540735923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5dced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.086+0000 7f895ca00700 1 --2- 172.21.15.90:0/540735923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.087+0000 7f895ca00700 1 -- 172.21.15.90:0/540735923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 msgr2=0x55847b5dced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.087+0000 7f895ca00700 1 --2- 172.21.15.90:0/540735923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5dced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.087+0000 7f895ca00700 1 -- 172.21.15.90:0/540735923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 msgr2=0x55847b5c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.087+0000 7f895ca00700 1 --2- 172.21.15.90:0/540735923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.087+0000 7f895ca00700 1 -- 172.21.15.90:0/540735923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847b5c2c50 con 0x55847b5d12c0 2024-05-27T07:49:26.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.087+0000 7f895ca00700 1 --2- 172.21.15.90:0/540735923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5e8690 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55847b5cf480 tx=0x7f894400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a952e8825f0c0e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:26.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.087+0000 7f894a000700 1 -- 172.21.15.90:0/540735923 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89440142c0 con 0x55847b5d12c0 2024-05-27T07:49:26.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.088+0000 7f894a000700 1 -- 172.21.15.90:0/540735923 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8944011990 con 0x55847b5d12c0 2024-05-27T07:49:26.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.088+0000 7f894a000700 1 -- 172.21.15.90:0/540735923 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944005280 con 0x55847b5d12c0 2024-05-27T07:49:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.088+0000 7f896e8b4b80 1 -- 172.21.15.90:0/540735923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 msgr2=0x55847b5e8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.088+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/540735923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5e8690 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55847b5cf480 tx=0x7f894400cc30 comp rx=0 tx=0).stop 2024-05-27T07:49:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.089+0000 7f896e8b4b80 1 -- 172.21.15.90:0/540735923 shutdown_connections 2024-05-27T07:49:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.089+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/540735923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b5e8690 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.089+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/540735923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.089+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/540735923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5dced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.089+0000 7f896e8b4b80 1 -- 172.21.15.90:0/540735923 >> 172.21.15.90:0/540735923 conn(0x55847b4e1390 msgr2=0x55847b5c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.089+0000 7f896e8b4b80 1 -- 172.21.15.90:0/540735923 shutdown_connections 2024-05-27T07:49:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.089+0000 7f896e8b4b80 1 -- 172.21.15.90:0/540735923 wait complete. 2024-05-27T07:49:26.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.089+0000 7f896e8b4b80 1 -- start start 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.090+0000 7f896e8b4b80 1 Processor -- start 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.090+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.090+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5d49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.090+0000 7f896e8b4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b649550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.090+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f89581b7b70 con 0x55847b5d12c0 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.090+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f89581b74a0 con 0x55847b447e40 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.090+0000 7f896e8b4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f89581b4680 con 0x55847b5339f0 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.090+0000 7f896e8b4b80 10 client.0 _ll_register_callbacks cb 0x55847b48ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.091+0000 7f895ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b649550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.091+0000 7f895ca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b649550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56352/0 (socket says 172.21.15.90:56352) 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.091+0000 7f895ca00700 1 -- 172.21.15.90:0/716104668 learned_addr learned my addr 172.21.15.90:0/716104668 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.091+0000 7f895d400700 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5d49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.091+0000 7f895ca00700 1 -- 172.21.15.90:0/716104668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 msgr2=0x55847b5d49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.091+0000 7f895ca00700 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5d49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.091+0000 7f895ca00700 1 -- 172.21.15.90:0/716104668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 msgr2=0x55847b5d4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.091+0000 7f895ca00700 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.091+0000 7f895ca00700 1 -- 172.21.15.90:0/716104668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847b649a90 con 0x55847b5d12c0 2024-05-27T07:49:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.092+0000 7f895ca00700 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b649550 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f894401f960 tx=0x7f8944014580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c3a48629c1db79 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.092+0000 7f893f400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.092+0000 7f893f400700 1 -- 172.21.15.90:0/716104668 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894401f680 con 0x55847b5d12c0 2024-05-27T07:49:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.092+0000 7f893f400700 1 -- 172.21.15.90:0/716104668 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f894401fb90 con 0x55847b5d12c0 2024-05-27T07:49:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.092+0000 7f896e8b4b80 1 -- 172.21.15.90:0/716104668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8944011990 con 0x55847b5d12c0 2024-05-27T07:49:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.092+0000 7f893f400700 1 -- 172.21.15.90:0/716104668 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8944004030 con 0x55847b5d12c0 2024-05-27T07:49:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.092+0000 7f896e8b4b80 10 client.16146 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.092+0000 7f896e8b4b80 1 -- 172.21.15.90:0/716104668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55847b64abe0 con 0x55847b5d12c0 2024-05-27T07:49:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.093+0000 7f893f400700 1 -- 172.21.15.90:0/716104668 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8944011990 con 0x55847b5d12c0 2024-05-27T07:49:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.093+0000 7f893f400700 1 -- 172.21.15.90:0/716104668 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f8944004710 con 0x55847b5d12c0 2024-05-27T07:49:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.093+0000 7f896e8b4b80 10 client.16146 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.093+0000 7f896e8b4b80 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:49:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.093+0000 7f896e8b4b80 20 client.16146 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.093+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55847b5e0ca0 0x55847b64d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.093+0000 7f896e8b4b80 4 client.16146 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:49:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.093+0000 7f896e8b4b80 1 -- 172.21.15.90:0/716104668 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55847b4f0570 con 0x55847b5e0ca0 2024-05-27T07:49:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.094+0000 7f895d400700 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55847b5e0ca0 0x55847b64d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.095+0000 7f893f400700 10 client.16146 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:49:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.095+0000 7f895d400700 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55847b5e0ca0 0x55847b64d1a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af79024e252b55ba server_cookie=b41c48f68988d2e in_seq=0 out_seq=0 2024-05-27T07:49:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.095+0000 7f893f400700 1 -- 172.21.15.90:0/716104668 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55847b4f0570 con 0x55847b5e0ca0 2024-05-27T07:49:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.095+0000 7f893f400700 10 client.16146 handle_command_reply: tid=0 2024-05-27T07:49:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.095+0000 7f896e8b4b80 1 client.16146 shutdown 2024-05-27T07:49:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 1 -- 172.21.15.90:0/716104668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 msgr2=0x55847b649550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b649550 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f894401f960 tx=0x7f8944014580 comp rx=0 tx=0).stop 2024-05-27T07:49:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 1 -- 172.21.15.90:0/716104668 shutdown_connections 2024-05-27T07:49:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55847b5e0ca0 0x55847b64d1a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55847b5d12c0 0x55847b649550 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55847b447e40 0x55847b5d4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 1 --2- 172.21.15.90:0/716104668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55847b5339f0 0x55847b5d49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 1 -- 172.21.15.90:0/716104668 >> 172.21.15.90:0/716104668 conn(0x55847b4e1390 msgr2=0x55847b5cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f893f400700 0 client.16146 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:49:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 1 -- 172.21.15.90:0/716104668 shutdown_connections 2024-05-27T07:49:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 1 -- 172.21.15.90:0/716104668 wait complete. 2024-05-27T07:49:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.096+0000 7f896e8b4b80 20 client.16146 trim_cache size 0 max 16384 2024-05-27T07:49:26.100 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:49:26.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 -- 172.21.15.90:0/3251255285 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f89400b29f0 msgr2=0x7f89400b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:26.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f89400b29f0 0x7f89400b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f89500a6a80 tx=0x7f895000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 -- 172.21.15.90:0/3251255285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f895810f6b0 msgr2=0x7f89581a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:26.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f895810f6b0 0x7f89581a5910 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f895000b070 tx=0x7f8950004e00 comp rx=0 tx=0).stop 2024-05-27T07:49:26.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 -- 172.21.15.90:0/3251255285 shutdown_connections 2024-05-27T07:49:26.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f895810f6b0 0x7f89581a5910 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f89400b29f0 0x7f89400b4f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f895810a7b0 0x7f89581a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 --2- 172.21.15.90:0/3251255285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f895810b100 0x7f89581a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 -- 172.21.15.90:0/3251255285 >> 172.21.15.90:0/3251255285 conn(0x7f8958076af0 msgr2=0x7f8958108850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.097+0000 7f895e800700 1 -- 172.21.15.90:0/3251255285 shutdown_connections 2024-05-27T07:49:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:26.098+0000 7f895e800700 1 -- 172.21.15.90:0/3251255285 wait complete. 2024-05-27T07:49:26.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:26 smithi129 ceph-mon[35254]: pgmap v97: 129 pgs: 129 active+clean; 818 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-05-27T07:49:26.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:26 smithi090 ceph-mon[31429]: pgmap v97: 129 pgs: 129 active+clean; 818 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-05-27T07:49:26.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:26 smithi136 ceph-mon[35324]: pgmap v97: 129 pgs: 129 active+clean; 818 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 22 op/s 2024-05-27T07:49:27.018 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:49:27.515 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:28.313 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:28 smithi129 ceph-mon[35254]: pgmap v98: 129 pgs: 129 active+clean; 818 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-05-27T07:49:28.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:28 smithi090 ceph-mon[31429]: pgmap v98: 129 pgs: 129 active+clean; 818 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-05-27T07:49:28.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:28 smithi136 ceph-mon[35324]: pgmap v98: 129 pgs: 129 active+clean; 818 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 20 op/s 2024-05-27T07:49:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.842+0000 7fb8b5400640 1 -- 172.21.15.90:0/1609615733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8b0110320 msgr2=0x7fb8b0112740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.842+0000 7fb8b5400640 1 --2- 172.21.15.90:0/1609615733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8b0110320 0x7fb8b0112740 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009ca0 tx=0x7fb8a802a950 comp rx=0 tx=0).stop 2024-05-27T07:49:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.842+0000 7fb8b5400640 1 -- 172.21.15.90:0/1609615733 shutdown_connections 2024-05-27T07:49:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.842+0000 7fb8b5400640 1 --2- 172.21.15.90:0/1609615733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8b0110320 0x7fb8b0112740 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.842+0000 7fb8b5400640 1 --2- 172.21.15.90:0/1609615733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b010d9c0 0x7fb8b010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.842+0000 7fb8b5400640 1 --2- 172.21.15.90:0/1609615733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8b0075400 0x7fb8b0075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.842+0000 7fb8b5400640 1 -- 172.21.15.90:0/1609615733 >> 172.21.15.90:0/1609615733 conn(0x7fb8b0079140 msgr2=0x7fb8b0079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.843+0000 7fb8b5400640 1 -- 172.21.15.90:0/1609615733 shutdown_connections 2024-05-27T07:49:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.843+0000 7fb8b5400640 1 -- 172.21.15.90:0/1609615733 wait complete. 2024-05-27T07:49:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.844+0000 7fb8b5400640 1 Processor -- start 2024-05-27T07:49:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.844+0000 7fb8b5400640 1 -- start start 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8b5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b0075400 0x7fb8b019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8b5400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8b010d9c0 0x7fb8b019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8b5400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8b0110320 0x7fb8b01a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8b5400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b0115d40 con 0x7fb8b0110320 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8b5400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b0115bc0 con 0x7fb8b010d9c0 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8b5400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b0115ec0 con 0x7fb8b0075400 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8b4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8b0110320 0x7fb8b01a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8b4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8b0110320 0x7fb8b01a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33454/0 (socket says 172.21.15.90:33454) 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8b4a00640 1 -- 172.21.15.90:0/2950782597 learned_addr learned my addr 172.21.15.90:0/2950782597 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8afe00640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b0075400 0x7fb8b019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.845+0000 7fb8af400640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8b010d9c0 0x7fb8b019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8b4a00640 1 -- 172.21.15.90:0/2950782597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b0075400 msgr2=0x7fb8b019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8b4a00640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b0075400 0x7fb8b019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8b4a00640 1 -- 172.21.15.90:0/2950782597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8b010d9c0 msgr2=0x7fb8b019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8b4a00640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8b010d9c0 0x7fb8b019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8b4a00640 1 -- 172.21.15.90:0/2950782597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b01a1600 con 0x7fb8b0110320 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8afe00640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b0075400 0x7fb8b019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8b4a00640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8b0110320 0x7fb8b01a0fc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb8a802ae70 tx=0x7fb8a8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8acc00640 1 -- 172.21.15.90:0/2950782597 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8034040 con 0x7fb8b0110320 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8acc00640 1 -- 172.21.15.90:0/2950782597 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a8030850 con 0x7fb8b0110320 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8acc00640 1 -- 172.21.15.90:0/2950782597 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8004670 con 0x7fb8b0110320 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.846+0000 7fb8b5400640 1 -- 172.21.15.90:0/2950782597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b01a1890 con 0x7fb8b0110320 2024-05-27T07:49:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.847+0000 7fb8b5400640 1 -- 172.21.15.90:0/2950782597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b01a1dd0 con 0x7fb8b0110320 2024-05-27T07:49:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.850+0000 7fb8bd971740 1 Processor -- start 2024-05-27T07:49:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.851+0000 7fb8acc00640 1 -- 172.21.15.90:0/2950782597 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb8a8037020 con 0x7fb8b0110320 2024-05-27T07:49:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.851+0000 7fb8bd971740 1 -- start start 2024-05-27T07:49:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a221b0 0x562680dc4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680dc4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a014d0 0x562680dc54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8bd971740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562680d35b90 con 0x5626809f92f0 2024-05-27T07:49:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8bd971740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562680d25730 con 0x562680a014d0 2024-05-27T07:49:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8bd971740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562680d29770 con 0x562680a221b0 2024-05-27T07:49:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680dc4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680dc4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33466/0 (socket says 172.21.15.90:33466) 2024-05-27T07:49:29.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8afe00640 1 -- 172.21.15.90:0/2879239152 learned_addr learned my addr 172.21.15.90:0/2879239152 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8af400640 1 --2- 172.21.15.90:0/2879239152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a221b0 0x562680dc4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.852+0000 7fb8af400640 1 --2- 172.21.15.90:0/2879239152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a014d0 0x562680dc54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.853+0000 7fb8afe00640 1 -- 172.21.15.90:0/2879239152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a221b0 msgr2=0x562680dc4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.853+0000 7fb8afe00640 1 --2- 172.21.15.90:0/2879239152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a221b0 0x562680dc4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.853+0000 7fb8afe00640 1 -- 172.21.15.90:0/2879239152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a014d0 msgr2=0x562680dc54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.853+0000 7fb8afe00640 1 --2- 172.21.15.90:0/2879239152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a014d0 0x562680dc54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.853+0000 7fb8afe00640 1 -- 172.21.15.90:0/2879239152 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562680cbe8b0 con 0x5626809f92f0 2024-05-27T07:49:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.853+0000 7fb8acc00640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8940b29d0 0x7fb8940b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.853+0000 7fb8acc00640 1 -- 172.21.15.90:0/2950782597 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fb8a81038d0 con 0x7fb8b0110320 2024-05-27T07:49:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.853+0000 7fb8afe00640 1 --2- 172.21.15.90:0/2879239152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680dc4fa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000bd40 tx=0x7fb8a000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f063ede69fa380d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.853+0000 7fb8af400640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8940b29d0 0x7fb8940b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.854+0000 7fb8a5200640 1 -- 172.21.15.90:0/2879239152 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0015070 con 0x5626809f92f0 2024-05-27T07:49:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.854+0000 7fb8a5200640 1 -- 172.21.15.90:0/2879239152 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a000be30 con 0x5626809f92f0 2024-05-27T07:49:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.854+0000 7fb8a5200640 1 -- 172.21.15.90:0/2879239152 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0010630 con 0x5626809f92f0 2024-05-27T07:49:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.855+0000 7fb8bd971740 1 -- 172.21.15.90:0/2879239152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 msgr2=0x562680dc4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.855+0000 7fb8bd971740 1 --2- 172.21.15.90:0/2879239152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680dc4fa0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000bd40 tx=0x7fb8a000f730 comp rx=0 tx=0).stop 2024-05-27T07:49:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.855+0000 7fb8af400640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8940b29d0 0x7fb8940b4ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x562680dc1bd0 tx=0x7fb89c0046c0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.855+0000 7fb8bd971740 1 -- 172.21.15.90:0/2879239152 shutdown_connections 2024-05-27T07:49:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.855+0000 7fb8bd971740 1 --2- 172.21.15.90:0/2879239152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a014d0 0x562680dc54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.855+0000 7fb8bd971740 1 --2- 172.21.15.90:0/2879239152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680dc4fa0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.855+0000 7fb8bd971740 1 --2- 172.21.15.90:0/2879239152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a221b0 0x562680dc4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.855+0000 7fb8bd971740 1 -- 172.21.15.90:0/2879239152 >> 172.21.15.90:0/2879239152 conn(0x562680a16300 msgr2=0x562680dc1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.855+0000 7fb8bd971740 1 -- 172.21.15.90:0/2879239152 shutdown_connections 2024-05-27T07:49:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.855+0000 7fb8bd971740 1 -- 172.21.15.90:0/2879239152 wait complete. 2024-05-27T07:49:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.856+0000 7fb8bd971740 1 -- start start 2024-05-27T07:49:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.856+0000 7fb8bd971740 1 Processor -- start 2024-05-27T07:49:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626809f92f0 0x562680dcec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a014d0 0x562680dcf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562680a221b0 0x562680ddaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8bd971740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562680a0b790 con 0x562680a221b0 2024-05-27T07:49:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8bd971740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562680ccdbc0 con 0x5626809f92f0 2024-05-27T07:49:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8bd971740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562680dd1b90 con 0x562680a014d0 2024-05-27T07:49:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8bd971740 10 client.0 _ll_register_callbacks cb 0x562680cab400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8b4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562680a221b0 0x562680ddaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8af400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a014d0 0x562680dcf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8b4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562680a221b0 0x562680ddaec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33474/0 (socket says 172.21.15.90:33474) 2024-05-27T07:49:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8b4a00640 1 -- 172.21.15.90:0/407792633 learned_addr learned my addr 172.21.15.90:0/407792633 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.857+0000 7fb8afe00640 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626809f92f0 0x562680dcec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.858+0000 7fb8b4a00640 1 -- 172.21.15.90:0/407792633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a014d0 msgr2=0x562680dcf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.858+0000 7fb8b4a00640 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a014d0 0x562680dcf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.858+0000 7fb8b4a00640 1 -- 172.21.15.90:0/407792633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626809f92f0 msgr2=0x562680dcec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.858+0000 7fb8b4a00640 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626809f92f0 0x562680dcec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.858+0000 7fb8b4a00640 1 -- 172.21.15.90:0/407792633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562680a3a210 con 0x562680a221b0 2024-05-27T07:49:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.858+0000 7fb8b4a00640 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562680a221b0 0x562680ddaec0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb8a802f800 tx=0x7fb8a80ca4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5a82c16d169609 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.858+0000 7fb899600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.858+0000 7fb899600640 1 -- 172.21.15.90:0/407792633 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a802c040 con 0x562680a221b0 2024-05-27T07:49:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.859+0000 7fb899600640 1 -- 172.21.15.90:0/407792633 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fb8a80ed0d0 con 0x562680a221b0 2024-05-27T07:49:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.859+0000 7fb8bd971740 1 -- 172.21.15.90:0/407792633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a000be30 con 0x562680a221b0 2024-05-27T07:49:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.859+0000 7fb899600640 1 -- 172.21.15.90:0/407792633 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb8a80ca9e0 con 0x562680a221b0 2024-05-27T07:49:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.859+0000 7fb8bd971740 10 client.16158 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.859+0000 7fb8bd971740 1 -- 172.21.15.90:0/407792633 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562680b06620 con 0x562680a221b0 2024-05-27T07:49:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.860+0000 7fb899600640 1 -- 172.21.15.90:0/407792633 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a000be30 con 0x562680a221b0 2024-05-27T07:49:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.860+0000 7fb899600640 1 -- 172.21.15.90:0/407792633 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb8a804d030 con 0x562680a221b0 2024-05-27T07:49:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.860+0000 7fb8bd971740 10 client.16158 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.860+0000 7fb8bd971740 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.860+0000 7fb8bd971740 20 client.16158 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.860+0000 7fb8bd971740 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626809e0d80 0x562680de0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.860+0000 7fb8bd971740 4 client.16158 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.860+0000 7fb8bd971740 1 -- 172.21.15.90:0/407792633 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562680d37ae0 con 0x5626809e0d80 2024-05-27T07:49:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.861+0000 7fb8afe00640 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626809e0d80 0x562680de0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.862+0000 7fb8afe00640 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626809e0d80 0x562680de0f30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=101e28271743e275 server_cookie=2f7c639560eafdb9 in_seq=0 out_seq=0 2024-05-27T07:49:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.862+0000 7fb899600640 10 client.16158 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.864+0000 7fb899600640 1 -- 172.21.15.90:0/407792633 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562680d37ae0 con 0x5626809e0d80 2024-05-27T07:49:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.864+0000 7fb899600640 10 client.16158 handle_command_reply: tid=0 2024-05-27T07:49:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.864+0000 7fb8bd971740 1 client.16158 shutdown 2024-05-27T07:49:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.864+0000 7fb8bd971740 1 -- 172.21.15.90:0/407792633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562680a221b0 msgr2=0x562680ddaec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.864+0000 7fb8bd971740 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562680a221b0 0x562680ddaec0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb8a802f800 tx=0x7fb8a80ca4f0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.863+0000 7f3048800700 1 -- 172.21.15.90:0/634122993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3040105e40 msgr2=0x7f304010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.863+0000 7f3048800700 1 --2- 172.21.15.90:0/634122993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3040105e40 0x7f304010e390 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3030009990 tx=0x7f3030009ca0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.863+0000 7f3048800700 1 -- 172.21.15.90:0/634122993 shutdown_connections 2024-05-27T07:49:29.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.863+0000 7f3048800700 1 --2- 172.21.15.90:0/634122993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f304010e8d0 0x7f3040110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.863+0000 7f3048800700 1 --2- 172.21.15.90:0/634122993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30401054f0 0x7f3040105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.863+0000 7f3048800700 1 --2- 172.21.15.90:0/634122993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3040105e40 0x7f304010e390 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.863+0000 7f3048800700 1 -- 172.21.15.90:0/634122993 >> 172.21.15.90:0/634122993 conn(0x7f30400fedb0 msgr2=0x7f30401011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.863+0000 7f3048800700 1 -- 172.21.15.90:0/634122993 shutdown_connections 2024-05-27T07:49:29.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.863+0000 7f3048800700 1 -- 172.21.15.90:0/634122993 wait complete. 2024-05-27T07:49:29.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.864+0000 7f3048800700 1 Processor -- start 2024-05-27T07:49:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.865+0000 7fb8bd971740 1 -- 172.21.15.90:0/407792633 shutdown_connections 2024-05-27T07:49:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.865+0000 7fb8bd971740 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626809e0d80 0x562680de0f30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.865+0000 7fb8bd971740 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562680a221b0 0x562680ddaec0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.865+0000 7fb8bd971740 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a014d0 0x562680dcf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.865+0000 7fb899600640 0 client.16158 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.865+0000 7fb8bd971740 1 --2- 172.21.15.90:0/407792633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626809f92f0 0x562680dcec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.865+0000 7fb8bd971740 1 -- 172.21.15.90:0/407792633 >> 172.21.15.90:0/407792633 conn(0x562680a16300 msgr2=0x562680dc4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.865+0000 7fb8bd971740 1 -- 172.21.15.90:0/407792633 shutdown_connections 2024-05-27T07:49:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.865+0000 7fb8bd971740 1 -- 172.21.15.90:0/407792633 wait complete. 2024-05-27T07:49:29.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.865+0000 7f3048800700 1 -- start start 2024-05-27T07:49:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.865+0000 7fb8bd971740 20 client.16158 trim_cache size 0 max 16384 2024-05-27T07:49:29.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.865+0000 7f3048800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30401054f0 0x7f3040199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.865+0000 7f3048800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f304010e8d0 0x7f3040199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.865+0000 7f3048800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30401a08e0 0x7f30401a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.865+0000 7f3048800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f30401143b0 con 0x7f30401a08e0 2024-05-27T07:49:29.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.865+0000 7f3048800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3040114230 con 0x7f304010e8d0 2024-05-27T07:49:29.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.865+0000 7f3048800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3040114530 con 0x7f30401054f0 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.866+0000 7f3045400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f304010e8d0 0x7f3040199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.866+0000 7f3045400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f304010e8d0 0x7f3040199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:35190/0 (socket says 172.21.15.90:35190) 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.866+0000 7f3045400700 1 -- 172.21.15.90:0/2464895791 learned_addr learned my addr 172.21.15.90:0/2464895791 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.866+0000 7f3046800700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30401a08e0 0x7f30401a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.866+0000 7f3045e00700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30401054f0 0x7f3040199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.866+0000 7f3045400700 1 -- 172.21.15.90:0/2464895791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30401054f0 msgr2=0x7f3040199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.866+0000 7f3045400700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30401054f0 0x7f3040199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.866+0000 7f3045400700 1 -- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30401a08e0 msgr2=0x7f30401a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.867+0000 7f3045400700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30401a08e0 0x7f30401a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.867+0000 7f3045400700 1 -- 172.21.15.90:0/2464895791 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30401a1260 con 0x7f304010e8d0 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.867+0000 7f3046800700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30401a08e0 0x7f30401a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:29.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.868+0000 7f3045400700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f304010e8d0 0x7f3040199f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3030000c00 tx=0x7f3030005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.868+0000 7f303aa00700 1 -- 172.21.15.90:0/2464895791 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030026070 con 0x7f304010e8d0 2024-05-27T07:49:29.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.868+0000 7f3048800700 1 -- 172.21.15.90:0/2464895791 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30401a14f0 con 0x7f304010e8d0 2024-05-27T07:49:29.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.868+0000 7f3048800700 1 -- 172.21.15.90:0/2464895791 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30401a19d0 con 0x7f304010e8d0 2024-05-27T07:49:29.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.869+0000 7f303aa00700 1 -- 172.21.15.90:0/2464895791 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f303000fcf0 con 0x7f304010e8d0 2024-05-27T07:49:29.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.869+0000 7f303aa00700 1 -- 172.21.15.90:0/2464895791 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3030017bf0 con 0x7f304010e8d0 2024-05-27T07:49:29.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.874+0000 7f303aa00700 1 -- 172.21.15.90:0/2464895791 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3030029070 con 0x7f304010e8d0 2024-05-27T07:49:29.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.874+0000 7f30572b3b80 1 Processor -- start 2024-05-27T07:49:29.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.875+0000 7f30572b3b80 1 -- start start 2024-05-27T07:49:29.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.875+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb8a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.875+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb8a4d30 0x559ffb89e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.875+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb89f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.875+0000 7f30572b3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559ffb7cce40 con 0x559ffb71fe40 2024-05-27T07:49:29.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.875+0000 7f30572b3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ffb7bc9e0 con 0x559ffb89edb0 2024-05-27T07:49:29.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.875+0000 7f30572b3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559ffb7c0a20 con 0x559ffb8a4d30 2024-05-27T07:49:29.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.876+0000 7f3046800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb8a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.876+0000 7f3046800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb8a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33498/0 (socket says 172.21.15.90:33498) 2024-05-27T07:49:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.876+0000 7f3046800700 1 -- 172.21.15.90:0/3247839361 learned_addr learned my addr 172.21.15.90:0/3247839361 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.877+0000 7f3046800700 1 -- 172.21.15.90:0/3247839361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb8a4d30 msgr2=0x559ffb89e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.877+0000 7f3046800700 1 --2- 172.21.15.90:0/3247839361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb8a4d30 0x559ffb89e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.877+0000 7f3046800700 1 -- 172.21.15.90:0/3247839361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 msgr2=0x559ffb89f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.877+0000 7f3045400700 1 --2- 172.21.15.90:0/3247839361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb89f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.877+0000 7f3046800700 1 --2- 172.21.15.90:0/3247839361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb89f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.877+0000 7f3046800700 1 -- 172.21.15.90:0/3247839361 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ffb785420 con 0x559ffb71fe40 2024-05-27T07:49:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.877+0000 7f303aa00700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30280b2a70 0x7f30280b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.877+0000 7f3046800700 1 --2- 172.21.15.90:0/3247839361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb8a47f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f303c007f30 tx=0x7f303c00c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565d5c4c3d3932cd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.877+0000 7f303aa00700 1 -- 172.21.15.90:0/2464895791 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f30300e32c0 con 0x7f304010e8d0 2024-05-27T07:49:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.878+0000 7f302f400700 1 -- 172.21.15.90:0/3247839361 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c014070 con 0x559ffb71fe40 2024-05-27T07:49:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.878+0000 7f3045e00700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30280b2a70 0x7f30280b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.878+0000 7f302f400700 1 -- 172.21.15.90:0/3247839361 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f303c00fad0 con 0x559ffb71fe40 2024-05-27T07:49:29.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.878+0000 7f302f400700 1 -- 172.21.15.90:0/3247839361 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c00ea00 con 0x559ffb71fe40 2024-05-27T07:49:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.879+0000 7f30572b3b80 1 -- 172.21.15.90:0/3247839361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 msgr2=0x559ffb8a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.879+0000 7f3045e00700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30280b2a70 0x7f30280b4f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f303400abb0 tx=0x7f3034005f50 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.879+0000 7f30572b3b80 1 --2- 172.21.15.90:0/3247839361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb8a47f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f303c007f30 tx=0x7f303c00c5a0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.879+0000 7f30572b3b80 1 -- 172.21.15.90:0/3247839361 shutdown_connections 2024-05-27T07:49:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.879+0000 7f30572b3b80 1 --2- 172.21.15.90:0/3247839361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb89f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.879+0000 7f30572b3b80 1 --2- 172.21.15.90:0/3247839361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb8a47f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.879+0000 7f30572b3b80 1 --2- 172.21.15.90:0/3247839361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb8a4d30 0x559ffb89e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.879+0000 7f30572b3b80 1 -- 172.21.15.90:0/3247839361 >> 172.21.15.90:0/3247839361 conn(0x559ffb77ffc0 msgr2=0x559ffb8998d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.879+0000 7f30572b3b80 1 -- 172.21.15.90:0/3247839361 shutdown_connections 2024-05-27T07:49:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.880+0000 7f30572b3b80 1 -- 172.21.15.90:0/3247839361 wait complete. 2024-05-27T07:49:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.880+0000 7f30572b3b80 1 -- start start 2024-05-27T07:49:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.880+0000 7f30572b3b80 1 Processor -- start 2024-05-27T07:49:29.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.881+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb71fe40 0x559ffb8a84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.881+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb8a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.881+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.881+0000 7f30572b3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559ffb6c2fa0 con 0x559ffb8a4d30 2024-05-27T07:49:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.881+0000 7f30572b3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ffb75d760 con 0x559ffb89edb0 2024-05-27T07:49:29.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.881+0000 7f30572b3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559ffb8ab800 con 0x559ffb71fe40 2024-05-27T07:49:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.881+0000 7f30572b3b80 10 client.0 _ll_register_callbacks cb 0x559ffb766c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.882+0000 7f3046800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.882+0000 7f3046800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8a8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33504/0 (socket says 172.21.15.90:33504) 2024-05-27T07:49:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.882+0000 7f3046800700 1 -- 172.21.15.90:0/2272113656 learned_addr learned my addr 172.21.15.90:0/2272113656 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.882+0000 7f3045e00700 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb71fe40 0x559ffb8a84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.883+0000 7f3045400700 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb8a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.883+0000 7f3046800700 1 -- 172.21.15.90:0/2272113656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb71fe40 msgr2=0x559ffb8a84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.883+0000 7f3046800700 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb71fe40 0x559ffb8a84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.883+0000 7f3046800700 1 -- 172.21.15.90:0/2272113656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 msgr2=0x559ffb8a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.883+0000 7f3046800700 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb8a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.883+0000 7f3046800700 1 -- 172.21.15.90:0/2272113656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ffb70c8d0 con 0x559ffb8a4d30 2024-05-27T07:49:29.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.884+0000 7f3046800700 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8a8f70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f303c000f80 tx=0x7f303c007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b0282eff81121 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.884+0000 7f3023e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:29.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.884+0000 7f3023e00700 1 -- 172.21.15.90:0/2272113656 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c026060 con 0x559ffb8a4d30 2024-05-27T07:49:29.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.884+0000 7f3023e00700 1 -- 172.21.15.90:0/2272113656 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f303c00e880 con 0x559ffb8a4d30 2024-05-27T07:49:29.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.885+0000 7f30572b3b80 1 -- 172.21.15.90:0/2272113656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f303c00fad0 con 0x559ffb8a4d30 2024-05-27T07:49:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.885+0000 7fb8bd971740 1 Processor -- start 2024-05-27T07:49:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.885+0000 7fb8bd971740 1 -- start start 2024-05-27T07:49:29.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.885+0000 7f3023e00700 1 -- 172.21.15.90:0/2272113656 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f303c014040 con 0x559ffb8a4d30 2024-05-27T07:49:29.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.885+0000 7f30572b3b80 10 client.16167 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:29.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.885+0000 7f30572b3b80 1 -- 172.21.15.90:0/2272113656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559ffb78d060 con 0x559ffb8a4d30 2024-05-27T07:49:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680ddac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a014d0 0x562680ddb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a221b0 0x562680dd6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8bd971740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01b3150 con 0x5626809f92f0 2024-05-27T07:49:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8bd971740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01b4c90 con 0x562680a014d0 2024-05-27T07:49:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8bd971740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01b3820 con 0x562680a221b0 2024-05-27T07:49:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680ddac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680ddac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33512/0 (socket says 172.21.15.90:33512) 2024-05-27T07:49:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8afe00640 1 -- 172.21.15.90:0/2295014333 learned_addr learned my addr 172.21.15.90:0/2295014333 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8afe00640 1 -- 172.21.15.90:0/2295014333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a221b0 msgr2=0x562680dd6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8afe00640 1 --2- 172.21.15.90:0/2295014333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a221b0 0x562680dd6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8afe00640 1 -- 172.21.15.90:0/2295014333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a014d0 msgr2=0x562680ddb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8afe00640 1 --2- 172.21.15.90:0/2295014333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a014d0 0x562680ddb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.886+0000 7fb8afe00640 1 -- 172.21.15.90:0/2295014333 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562680ddb6b0 con 0x5626809f92f0 2024-05-27T07:49:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.887+0000 7fb8afe00640 1 --2- 172.21.15.90:0/2295014333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680ddac30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000bd40 tx=0x7fb8a0011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bd4e13afaf685 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.887+0000 7fb89be00640 1 -- 172.21.15.90:0/2295014333 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0015040 con 0x5626809f92f0 2024-05-27T07:49:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.887+0000 7fb89be00640 1 -- 172.21.15.90:0/2295014333 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a0011e50 con 0x5626809f92f0 2024-05-27T07:49:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.887+0000 7fb89be00640 1 -- 172.21.15.90:0/2295014333 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00116a0 con 0x5626809f92f0 2024-05-27T07:49:29.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.886+0000 7f3023e00700 1 -- 172.21.15.90:0/2272113656 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f303c00fad0 con 0x559ffb8a4d30 2024-05-27T07:49:29.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.887+0000 7f3023e00700 1 -- 172.21.15.90:0/2272113656 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f303c019a70 con 0x559ffb8a4d30 2024-05-27T07:49:29.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.888+0000 7f30572b3b80 10 client.16167 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:29.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.888+0000 7f30572b3b80 10 client.16167 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:49:29.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.888+0000 7f30572b3b80 20 client.16167 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:29.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.888+0000 7f30572b3b80 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559ffb8b85e0 0x559ffb8baa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.888+0000 7f30572b3b80 4 client.16167 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:29.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.888+0000 7f30572b3b80 1 -- 172.21.15.90:0/2272113656 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ffb7ced90 con 0x559ffb8b85e0 2024-05-27T07:49:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.888+0000 7fb8bd971740 1 -- 172.21.15.90:0/2295014333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 msgr2=0x562680ddac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.888+0000 7fb8bd971740 1 --2- 172.21.15.90:0/2295014333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680ddac30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000bd40 tx=0x7fb8a0011de0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.888+0000 7fb8bd971740 1 -- 172.21.15.90:0/2295014333 shutdown_connections 2024-05-27T07:49:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.888+0000 7fb8bd971740 1 --2- 172.21.15.90:0/2295014333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a014d0 0x562680ddb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.888+0000 7fb8bd971740 1 --2- 172.21.15.90:0/2295014333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a221b0 0x562680dd6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.888+0000 7fb8bd971740 1 --2- 172.21.15.90:0/2295014333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680ddac30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.889+0000 7fb8bd971740 1 -- 172.21.15.90:0/2295014333 >> 172.21.15.90:0/2295014333 conn(0x562680a16300 msgr2=0x562680dc9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.889+0000 7f3045e00700 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559ffb8b85e0 0x559ffb8baa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.889+0000 7fb8bd971740 1 -- 172.21.15.90:0/2295014333 shutdown_connections 2024-05-27T07:49:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.889+0000 7fb8bd971740 1 -- 172.21.15.90:0/2295014333 wait complete. 2024-05-27T07:49:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.889+0000 7fb8bd971740 1 -- start start 2024-05-27T07:49:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.890+0000 7fb8bd971740 1 Processor -- start 2024-05-27T07:49:29.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.890+0000 7f3023e00700 10 client.16167 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:49:29.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.890+0000 7f3045e00700 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559ffb8b85e0 0x559ffb8baa00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=133d4999ec6169e7 server_cookie=da6e7cd82dd650a1 in_seq=0 out_seq=0 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.890+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680e49180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.890+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a014d0 0x562680e496c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.890+0000 7fb8bd971740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a221b0 0x562680dd64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.890+0000 7fb8bd971740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01b45c0 con 0x5626809f92f0 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.890+0000 7fb8bd971740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01b3ef0 con 0x562680a221b0 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.890+0000 7fb8bd971740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01b1110 con 0x562680a014d0 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.890+0000 7fb8bd971740 10 client.0 _ll_register_callbacks cb 0x7fb8a80d3040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680e49180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb8afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680e49180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33518/0 (socket says 172.21.15.90:33518) 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb8afe00640 1 -- 172.21.15.90:0/37524352 learned_addr learned my addr 172.21.15.90:0/37524352 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb8b4a00640 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a221b0 0x562680dd64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb8afe00640 1 -- 172.21.15.90:0/37524352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a014d0 msgr2=0x562680e496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb8afe00640 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a014d0 0x562680e496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb8afe00640 1 -- 172.21.15.90:0/37524352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a221b0 msgr2=0x562680dd64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb8afe00640 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a221b0 0x562680dd64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb8afe00640 1 -- 172.21.15.90:0/37524352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562680dd6ad0 con 0x5626809f92f0 2024-05-27T07:49:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb8afe00640 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680e49180 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000fc30 tx=0x7fb8a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a8bfb604e4583d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.891+0000 7fb899600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.892+0000 7fb899600640 1 -- 172.21.15.90:0/37524352 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0015070 con 0x5626809f92f0 2024-05-27T07:49:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.892+0000 7fb899600640 1 -- 172.21.15.90:0/37524352 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fb8a0004310 con 0x5626809f92f0 2024-05-27T07:49:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.892+0000 7fb8bd971740 1 -- 172.21.15.90:0/37524352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a0011e50 con 0x5626809f92f0 2024-05-27T07:49:29.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.892+0000 7f3023e00700 1 -- 172.21.15.90:0/2272113656 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559ffb7ced90 con 0x559ffb8b85e0 2024-05-27T07:49:29.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.892+0000 7f3023e00700 10 client.16167 handle_command_reply: tid=0 2024-05-27T07:49:29.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.892+0000 7f30572b3b80 1 client.16167 shutdown 2024-05-27T07:49:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.892+0000 7fb899600640 1 -- 172.21.15.90:0/37524352 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb8a0005040 con 0x5626809f92f0 2024-05-27T07:49:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.892+0000 7fb8bd971740 10 client.16173 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.892+0000 7fb8bd971740 1 -- 172.21.15.90:0/37524352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562680e518c0 con 0x5626809f92f0 2024-05-27T07:49:29.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.892+0000 7f30572b3b80 1 -- 172.21.15.90:0/2272113656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 msgr2=0x559ffb8a8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.892+0000 7f30572b3b80 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8a8f70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f303c000f80 tx=0x7f303c007ac0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.893+0000 7f30572b3b80 1 -- 172.21.15.90:0/2272113656 shutdown_connections 2024-05-27T07:49:29.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.893+0000 7f30572b3b80 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8a8f70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.893+0000 7f30572b3b80 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb71fe40 0x559ffb8a84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.893+0000 7f30572b3b80 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559ffb8b85e0 0x559ffb8baa00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.893+0000 7f30572b3b80 1 --2- 172.21.15.90:0/2272113656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb8a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.893+0000 7f30572b3b80 1 -- 172.21.15.90:0/2272113656 >> 172.21.15.90:0/2272113656 conn(0x559ffb77ffc0 msgr2=0x559ffb899580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.893+0000 7f3023e00700 0 client.16167 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:49:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.893+0000 7fb899600640 1 -- 172.21.15.90:0/37524352 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb8a0011e50 con 0x5626809f92f0 2024-05-27T07:49:29.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.893+0000 7f30572b3b80 1 -- 172.21.15.90:0/2272113656 shutdown_connections 2024-05-27T07:49:29.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.893+0000 7f30572b3b80 1 -- 172.21.15.90:0/2272113656 wait complete. 2024-05-27T07:49:29.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.894+0000 7f30572b3b80 20 client.16167 trim_cache size 0 max 16384 2024-05-27T07:49:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.893+0000 7fb899600640 1 -- 172.21.15.90:0/37524352 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7fb8a0004680 con 0x5626809f92f0 2024-05-27T07:49:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.893+0000 7fb8bd971740 10 client.16173 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.893+0000 7fb8bd971740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.893+0000 7fb8bd971740 20 client.16173 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.893+0000 7fb8bd971740 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626809b61c0 0x562680e53dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.894+0000 7fb8bd971740 4 client.16173 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:49:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.894+0000 7fb8bd971740 1 -- 172.21.15.90:0/37524352 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562680d312c0 con 0x5626809b61c0 2024-05-27T07:49:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.894+0000 7fb8af400640 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626809b61c0 0x562680e53dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.895+0000 7fb8af400640 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626809b61c0 0x562680e53dd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f99cd3fd67f83c1 server_cookie=57d077370f44fb92 in_seq=0 out_seq=0 2024-05-27T07:49:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.895+0000 7fb899600640 10 client.16173 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:29.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.918+0000 7f30572b3b80 1 Processor -- start 2024-05-27T07:49:29.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.919+0000 7f30572b3b80 1 -- start start 2024-05-27T07:49:29.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.919+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb71fe40 0x559ffb89e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.919+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb89edb0 0x559ffb8a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.919+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.919+0000 7f30572b3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f302400ecc0 con 0x559ffb8a4d30 2024-05-27T07:49:29.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.919+0000 7f30572b3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3024010800 con 0x559ffb71fe40 2024-05-27T07:49:29.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.919+0000 7f30572b3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f302400f390 con 0x559ffb89edb0 2024-05-27T07:49:29.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.920+0000 7f3046800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.921+0000 7f3046800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33522/0 (socket says 172.21.15.90:33522) 2024-05-27T07:49:29.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.921+0000 7f3046800700 1 -- 172.21.15.90:0/3804342582 learned_addr learned my addr 172.21.15.90:0/3804342582 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:29.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.921+0000 7f3045e00700 1 --2- 172.21.15.90:0/3804342582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb71fe40 0x559ffb89e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.921+0000 7f3045400700 1 --2- 172.21.15.90:0/3804342582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb89edb0 0x559ffb8a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.921+0000 7f3046800700 1 -- 172.21.15.90:0/3804342582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb89edb0 msgr2=0x559ffb8a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.921+0000 7f3046800700 1 --2- 172.21.15.90:0/3804342582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb89edb0 0x559ffb8a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.921+0000 7f3046800700 1 -- 172.21.15.90:0/3804342582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb71fe40 msgr2=0x559ffb89e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.921+0000 7f3046800700 1 --2- 172.21.15.90:0/3804342582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb71fe40 0x559ffb89e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.921+0000 7f3046800700 1 -- 172.21.15.90:0/3804342582 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ffb8a5140 con 0x559ffb8a4d30 2024-05-27T07:49:29.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.921+0000 7f3045400700 1 --2- 172.21.15.90:0/3804342582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb89edb0 0x559ffb8a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:29.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.922+0000 7f3046800700 1 --2- 172.21.15.90:0/3804342582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8b4b00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f303c0216c0 tx=0x7f303c0216f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8402ce7f992e34 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.922+0000 7f302ea00700 1 -- 172.21.15.90:0/3804342582 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c026070 con 0x559ffb8a4d30 2024-05-27T07:49:29.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.923+0000 7f302ea00700 1 -- 172.21.15.90:0/3804342582 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f303c007780 con 0x559ffb8a4d30 2024-05-27T07:49:29.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.923+0000 7f302ea00700 1 -- 172.21.15.90:0/3804342582 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c026070 con 0x559ffb8a4d30 2024-05-27T07:49:29.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.924+0000 7f30572b3b80 1 -- 172.21.15.90:0/3804342582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 msgr2=0x559ffb8b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.924+0000 7f30572b3b80 1 --2- 172.21.15.90:0/3804342582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8b4b00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f303c0216c0 tx=0x7f303c0216f0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.926+0000 7f30572b3b80 1 -- 172.21.15.90:0/3804342582 shutdown_connections 2024-05-27T07:49:29.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.926+0000 7f30572b3b80 1 --2- 172.21.15.90:0/3804342582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb8a4d30 0x559ffb8b4b00 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.926+0000 7f30572b3b80 1 --2- 172.21.15.90:0/3804342582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb71fe40 0x559ffb89e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.926+0000 7f30572b3b80 1 --2- 172.21.15.90:0/3804342582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb89edb0 0x559ffb8a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.926+0000 7f30572b3b80 1 -- 172.21.15.90:0/3804342582 >> 172.21.15.90:0/3804342582 conn(0x559ffb77ffc0 msgr2=0x559ffb8a82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.926+0000 7f30572b3b80 1 -- 172.21.15.90:0/3804342582 shutdown_connections 2024-05-27T07:49:29.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.927+0000 7f30572b3b80 1 -- 172.21.15.90:0/3804342582 wait complete. 2024-05-27T07:49:29.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.928+0000 7f30572b3b80 1 -- start start 2024-05-27T07:49:29.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.928+0000 7f30572b3b80 1 Processor -- start 2024-05-27T07:49:29.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.929+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb91b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.930+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb91b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.930+0000 7f30572b3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb8a4d30 0x559ffb8bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.930+0000 7f30572b3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3024010130 con 0x559ffb71fe40 2024-05-27T07:49:29.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.930+0000 7f30572b3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f302400fa60 con 0x559ffb89edb0 2024-05-27T07:49:29.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.930+0000 7f3045e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb91b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.930+0000 7f3045e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb91b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33532/0 (socket says 172.21.15.90:33532) 2024-05-27T07:49:29.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.931+0000 7f3045e00700 1 -- 172.21.15.90:0/168179728 learned_addr learned my addr 172.21.15.90:0/168179728 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:29.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.931+0000 7f30572b3b80 1 -- 172.21.15.90:0/168179728 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f302400cc40 con 0x559ffb8a4d30 2024-05-27T07:49:29.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.931+0000 7f3046800700 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb8a4d30 0x559ffb8bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.931+0000 7f30572b3b80 10 client.0 _ll_register_callbacks cb 0x559ffb766c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:29.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.932+0000 7f3045e00700 1 -- 172.21.15.90:0/168179728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb8a4d30 msgr2=0x559ffb8bc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.932+0000 7f3045e00700 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb8a4d30 0x559ffb8bc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.932+0000 7f3045e00700 1 -- 172.21.15.90:0/168179728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 msgr2=0x559ffb91b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:29.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.932+0000 7f3045e00700 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb91b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.932+0000 7f3045e00700 1 -- 172.21.15.90:0/168179728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ffb91bff0 con 0x559ffb71fe40 2024-05-27T07:49:29.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.932+0000 7f3045e00700 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb91b200 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3034004790 tx=0x7f3034005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a990648fe7761d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:29.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.932+0000 7f3023e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:29.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.933+0000 7f3023e00700 1 -- 172.21.15.90:0/168179728 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034027330 con 0x559ffb71fe40 2024-05-27T07:49:29.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.933+0000 7f30572b3b80 1 -- 172.21.15.90:0/168179728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f303c007780 con 0x559ffb71fe40 2024-05-27T07:49:29.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.933+0000 7f3023e00700 1 -- 172.21.15.90:0/168179728 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f30340274d0 con 0x559ffb71fe40 2024-05-27T07:49:29.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.934+0000 7f3023e00700 1 -- 172.21.15.90:0/168179728 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3034035050 con 0x559ffb71fe40 2024-05-27T07:49:29.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.934+0000 7f30572b3b80 10 client.16179 fetch_fsmap learned FSMap version 39 2024-05-27T07:49:29.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.934+0000 7f30572b3b80 1 -- 172.21.15.90:0/168179728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559ffb8bdb10 con 0x559ffb71fe40 2024-05-27T07:49:29.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.935+0000 7f3023e00700 1 -- 172.21.15.90:0/168179728 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f303c007780 con 0x559ffb71fe40 2024-05-27T07:49:29.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.935+0000 7f3023e00700 1 -- 172.21.15.90:0/168179728 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8362+0+0 (secure 0 0 0) 0x7f3034038020 con 0x559ffb71fe40 2024-05-27T07:49:29.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.935+0000 7f30572b3b80 10 client.16179 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T07:49:29.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.935+0000 7f30572b3b80 10 client.16179 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:49:29.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.935+0000 7f30572b3b80 20 client.16179 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:29.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.935+0000 7f30572b3b80 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559ffb933e50 0x559ffb936270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:29.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.936+0000 7f30572b3b80 4 client.16179 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:49:29.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.936+0000 7f30572b3b80 1 -- 172.21.15.90:0/168179728 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ffb7c8570 con 0x559ffb933e50 2024-05-27T07:49:29.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.937+0000 7f3045400700 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559ffb933e50 0x559ffb936270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:29.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.941+0000 7f3023e00700 10 client.16179 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:49:29.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.941+0000 7f3045400700 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559ffb933e50 0x559ffb936270 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3738d71a8c81791a server_cookie=1aef785553cce67f in_seq=0 out_seq=0 2024-05-27T07:49:29.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f3023e00700 1 -- 172.21.15.90:0/168179728 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ffb7c8570 con 0x559ffb933e50 2024-05-27T07:49:29.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f3023e00700 10 client.16179 handle_command_reply: tid=0 2024-05-27T07:49:29.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f30572b3b80 1 client.16179 shutdown 2024-05-27T07:49:29.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f30572b3b80 1 -- 172.21.15.90:0/168179728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 msgr2=0x559ffb91b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f30572b3b80 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb91b200 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3034004790 tx=0x7f3034005e30 comp rx=0 tx=0).stop 2024-05-27T07:49:29.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f30572b3b80 1 -- 172.21.15.90:0/168179728 shutdown_connections 2024-05-27T07:49:29.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f30572b3b80 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffb8a4d30 0x559ffb8bc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f30572b3b80 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ffb71fe40 0x559ffb91b200 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f30572b3b80 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559ffb933e50 0x559ffb936270 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f30572b3b80 1 --2- 172.21.15.90:0/168179728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ffb89edb0 0x559ffb91b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f30572b3b80 1 -- 172.21.15.90:0/168179728 >> 172.21.15.90:0/168179728 conn(0x559ffb77ffc0 msgr2=0x559ffb8a3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f3023e00700 0 client.16179 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:49:29.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.942+0000 7f30572b3b80 1 -- 172.21.15.90:0/168179728 shutdown_connections 2024-05-27T07:49:29.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.943+0000 7f30572b3b80 1 -- 172.21.15.90:0/168179728 wait complete. 2024-05-27T07:49:29.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.943+0000 7f30572b3b80 20 client.16179 trim_cache size 0 max 16384 2024-05-27T07:49:29.944 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:49:29.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 -- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30280b2a70 msgr2=0x7f30280b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30280b2a70 0x7f30280b4f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f303400abb0 tx=0x7f3034005f50 comp rx=0 tx=0).stop 2024-05-27T07:49:29.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 -- 172.21.15.90:0/2464895791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f304010e8d0 msgr2=0x7f3040199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f304010e8d0 0x7f3040199f90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3030000c00 tx=0x7f3030005ad0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 -- 172.21.15.90:0/2464895791 shutdown_connections 2024-05-27T07:49:29.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30401a08e0 0x7f30401a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30401054f0 0x7f3040199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30280b2a70 0x7f30280b4f80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 --2- 172.21.15.90:0/2464895791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f304010e8d0 0x7f3040199f90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 -- 172.21.15.90:0/2464895791 >> 172.21.15.90:0/2464895791 conn(0x7f30400fedb0 msgr2=0x7f30401032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 -- 172.21.15.90:0/2464895791 shutdown_connections 2024-05-27T07:49:29.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:29.944+0000 7f3048800700 1 -- 172.21.15.90:0/2464895791 wait complete. 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb899600640 1 -- 172.21.15.90:0/37524352 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562680d312c0 con 0x5626809b61c0 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb899600640 10 client.16173 handle_command_reply: tid=0 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 client.16173 shutdown 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 -- 172.21.15.90:0/37524352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 msgr2=0x562680e49180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680e49180 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000fc30 tx=0x7fb8a000be30 comp rx=0 tx=0).stop 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 -- 172.21.15.90:0/37524352 shutdown_connections 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626809b61c0 0x562680e53dd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562680a221b0 0x562680dd64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562680a014d0 0x562680e496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 --2- 172.21.15.90:0/37524352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626809f92f0 0x562680e49180 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb899600640 0 client.16173 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 -- 172.21.15.90:0/37524352 >> 172.21.15.90:0/37524352 conn(0x562680a16300 msgr2=0x562680dd9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 -- 172.21.15.90:0/37524352 shutdown_connections 2024-05-27T07:49:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 1 -- 172.21.15.90:0/37524352 wait complete. 2024-05-27T07:49:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.948+0000 7fb8bd971740 20 client.16173 trim_cache size 0 max 16384 2024-05-27T07:49:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:49:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:49:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:49:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.949+0000 7fb8b5400640 1 -- 172.21.15.90:0/2950782597 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8940b29d0 msgr2=0x7fb8940b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.949+0000 7fb8b5400640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8940b29d0 0x7fb8940b4ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x562680dc1bd0 tx=0x7fb89c0046c0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.949+0000 7fb8b5400640 1 -- 172.21.15.90:0/2950782597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8b0110320 msgr2=0x7fb8b01a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.949+0000 7fb8b5400640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8b0110320 0x7fb8b01a0fc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb8a802ae70 tx=0x7fb8a8003f10 comp rx=0 tx=0).stop 2024-05-27T07:49:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.949+0000 7fb8b5400640 1 -- 172.21.15.90:0/2950782597 shutdown_connections 2024-05-27T07:49:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.949+0000 7fb8b5400640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb8940b29d0 0x7fb8940b4ee0 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x562680dc1bd0 tx=0x7fb89c0046c0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.949+0000 7fb8b5400640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb8b010d9c0 0x7fb8b019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.949+0000 7fb8b5400640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8b0110320 0x7fb8b01a0fc0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.949+0000 7fb8b5400640 1 --2- 172.21.15.90:0/2950782597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b0075400 0x7fb8b019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.949+0000 7fb8b5400640 1 -- 172.21.15.90:0/2950782597 >> 172.21.15.90:0/2950782597 conn(0x7fb8b0079140 msgr2=0x7fb8b0102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.950+0000 7fb8b5400640 1 -- 172.21.15.90:0/2950782597 shutdown_connections 2024-05-27T07:49:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:29.950+0000 7fb8b5400640 1 -- 172.21.15.90:0/2950782597 wait complete. 2024-05-27T07:49:30.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:30 smithi090 ceph-mon[31429]: pgmap v99: 129 pgs: 129 active+clean; 862 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 31 op/s 2024-05-27T07:49:30.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:30 smithi129 ceph-mon[35254]: pgmap v99: 129 pgs: 129 active+clean; 862 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 31 op/s 2024-05-27T07:49:30.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:30 smithi136 ceph-mon[35324]: pgmap v99: 129 pgs: 129 active+clean; 862 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 17 MiB/s wr, 31 op/s 2024-05-27T07:49:30.868 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:49:31.353 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:32.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:32 smithi090 ceph-mon[31429]: pgmap v100: 129 pgs: 129 active+clean; 862 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-05-27T07:49:32.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:32 smithi090 ceph-mon[31429]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T07:49:32.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:32 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:49:32.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:32 smithi136 ceph-mon[35324]: pgmap v100: 129 pgs: 129 active+clean; 862 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-05-27T07:49:32.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:32 smithi136 ceph-mon[35324]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T07:49:32.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:32 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:49:32.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:32 smithi129 ceph-mon[35254]: pgmap v100: 129 pgs: 129 active+clean; 862 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 11 MiB/s wr, 18 op/s 2024-05-27T07:49:32.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:32 smithi129 ceph-mon[35254]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-05-27T07:49:32.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:32 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:49:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.608+0000 7efd26a00700 1 -- 172.21.15.90:0/1779997227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd20106eb0 msgr2=0x7efd20113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.608+0000 7efd26a00700 1 --2- 172.21.15.90:0/1779997227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd20106eb0 0x7efd20113340 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efd1800b0a0 tx=0x7efd1800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.608+0000 7efd26a00700 1 -- 172.21.15.90:0/1779997227 shutdown_connections 2024-05-27T07:49:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.608+0000 7efd26a00700 1 --2- 172.21.15.90:0/1779997227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd20106eb0 0x7efd20113340 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.608+0000 7efd26a00700 1 --2- 172.21.15.90:0/1779997227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd2010c5a0 0x7efd2010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.608+0000 7efd26a00700 1 --2- 172.21.15.90:0/1779997227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd20106560 0x7efd20106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.608+0000 7efd26a00700 1 -- 172.21.15.90:0/1779997227 >> 172.21.15.90:0/1779997227 conn(0x7efd20102040 msgr2=0x7efd20104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.608+0000 7efd26a00700 1 -- 172.21.15.90:0/1779997227 shutdown_connections 2024-05-27T07:49:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.608+0000 7efd26a00700 1 -- 172.21.15.90:0/1779997227 wait complete. 2024-05-27T07:49:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.609+0000 7efd26a00700 1 Processor -- start 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.609+0000 7efd26a00700 1 -- start start 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.610+0000 7efd26a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd20106560 0x7efd201106f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.610+0000 7efd26a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd20106eb0 0x7efd20110c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.610+0000 7efd26a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd2010c5a0 0x7efd20111170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.610+0000 7efd26a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd201165d0 con 0x7efd20106560 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.610+0000 7efd26a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd20116450 con 0x7efd20106eb0 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.610+0000 7efd26a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd20116750 con 0x7efd2010c5a0 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.610+0000 7efd1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd20106560 0x7efd201106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.610+0000 7efd1fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd20106560 0x7efd201106f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33562/0 (socket says 172.21.15.90:33562) 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.610+0000 7efd1fe00700 1 -- 172.21.15.90:0/839089541 learned_addr learned my addr 172.21.15.90:0/839089541 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.611+0000 7efd1f400700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd20106eb0 0x7efd20110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.611+0000 7efd24a00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd2010c5a0 0x7efd20111170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.611+0000 7efd1fe00700 1 -- 172.21.15.90:0/839089541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd2010c5a0 msgr2=0x7efd20111170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:33.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.611+0000 7efd1fe00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd2010c5a0 0x7efd20111170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.611+0000 7efd1fe00700 1 -- 172.21.15.90:0/839089541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd20106eb0 msgr2=0x7efd20110c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:33.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.611+0000 7efd1fe00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd20106eb0 0x7efd20110c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.611+0000 7efd1fe00700 1 -- 172.21.15.90:0/839089541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd201ad440 con 0x7efd20106560 2024-05-27T07:49:33.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.611+0000 7efd24a00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd2010c5a0 0x7efd20111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:33.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.611+0000 7efd1fe00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd20106560 0x7efd201106f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efd1000d980 tx=0x7efd1000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:33.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.612+0000 7efd1cc00700 1 -- 172.21.15.90:0/839089541 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10017070 con 0x7efd20106560 2024-05-27T07:49:33.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.612+0000 7efd1cc00700 1 -- 172.21.15.90:0/839089541 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd10010460 con 0x7efd20106560 2024-05-27T07:49:33.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.612+0000 7efd1cc00700 1 -- 172.21.15.90:0/839089541 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1000de60 con 0x7efd20106560 2024-05-27T07:49:33.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.612+0000 7efd26a00700 1 -- 172.21.15.90:0/839089541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd201ad6d0 con 0x7efd20106560 2024-05-27T07:49:33.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.612+0000 7efd26a00700 1 -- 172.21.15.90:0/839089541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd201adbc0 con 0x7efd20106560 2024-05-27T07:49:33.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.617+0000 7efd1cc00700 1 -- 172.21.15.90:0/839089541 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efd10010610 con 0x7efd20106560 2024-05-27T07:49:33.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.617+0000 7efd35452b80 1 Processor -- start 2024-05-27T07:49:33.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.617+0000 7efd35452b80 1 -- start start 2024-05-27T07:49:33.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.618+0000 7efd35452b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d02330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.618+0000 7efd35452b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d02870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.618+0000 7efd35452b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.618+0000 7efd35452b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f1d4c30e40 con 0x55f1d4d02e70 2024-05-27T07:49:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.618+0000 7efd35452b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f1d4c209e0 con 0x55f1d4c6f9f0 2024-05-27T07:49:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.618+0000 7efd35452b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f1d4c24a20 con 0x55f1d4b83e40 2024-05-27T07:49:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.618+0000 7efd1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.619+0000 7efd1fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d02870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.619+0000 7efd1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33566/0 (socket says 172.21.15.90:33566) 2024-05-27T07:49:33.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.619+0000 7efd1f400700 1 -- 172.21.15.90:0/2766188902 learned_addr learned my addr 172.21.15.90:0/2766188902 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:33.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.619+0000 7efd24a00700 1 --2- 172.21.15.90:0/2766188902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d02330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.619+0000 7efd1f400700 1 -- 172.21.15.90:0/2766188902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 msgr2=0x55f1d4d02870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:33.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.619+0000 7efd1f400700 1 --2- 172.21.15.90:0/2766188902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d02870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.619+0000 7efd1f400700 1 -- 172.21.15.90:0/2766188902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 msgr2=0x55f1d4d02330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:33.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.619+0000 7efd1f400700 1 --2- 172.21.15.90:0/2766188902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d02330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.619+0000 7efd1f400700 1 -- 172.21.15.90:0/2766188902 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d4be9420 con 0x55f1d4d02e70 2024-05-27T07:49:33.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.619+0000 7efd1fe00700 1 --2- 172.21.15.90:0/2766188902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d02870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:33.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.620+0000 7efd1f400700 1 --2- 172.21.15.90:0/2766188902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0f110 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00bcb0 tx=0x7efd0c00f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44fc5aad1381884 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:33.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.620+0000 7efd1cc00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efd040b29f0 0x7efd040b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.620+0000 7efd1cc00700 1 -- 172.21.15.90:0/839089541 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7efd100e2580 con 0x7efd20106560 2024-05-27T07:49:33.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.620+0000 7efd15600700 1 -- 172.21.15.90:0/2766188902 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c015070 con 0x55f1d4d02e70 2024-05-27T07:49:33.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.620+0000 7efd15600700 1 -- 172.21.15.90:0/2766188902 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd0c00fc90 con 0x55f1d4d02e70 2024-05-27T07:49:33.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.620+0000 7efd15600700 1 -- 172.21.15.90:0/2766188902 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c0116a0 con 0x55f1d4d02e70 2024-05-27T07:49:33.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.620+0000 7efd24a00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efd040b29f0 0x7efd040b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.622+0000 7efd24a00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efd040b29f0 0x7efd040b4f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efd18009900 tx=0x7efd18009ca0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:33.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.622+0000 7efd35452b80 1 -- 172.21.15.90:0/2766188902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 msgr2=0x55f1d4d0f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:33.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.622+0000 7efd35452b80 1 --2- 172.21.15.90:0/2766188902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0f110 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00bcb0 tx=0x7efd0c00f360 comp rx=0 tx=0).stop 2024-05-27T07:49:33.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.623+0000 7efd35452b80 1 -- 172.21.15.90:0/2766188902 shutdown_connections 2024-05-27T07:49:33.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.623+0000 7efd35452b80 1 --2- 172.21.15.90:0/2766188902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0f110 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.623+0000 7efd35452b80 1 --2- 172.21.15.90:0/2766188902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d02330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.623+0000 7efd35452b80 1 --2- 172.21.15.90:0/2766188902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d02870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.623+0000 7efd35452b80 1 -- 172.21.15.90:0/2766188902 >> 172.21.15.90:0/2766188902 conn(0x55f1d4c1d390 msgr2=0x55f1d4cfe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:33.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.623+0000 7efd35452b80 1 -- 172.21.15.90:0/2766188902 shutdown_connections 2024-05-27T07:49:33.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.623+0000 7efd35452b80 1 -- 172.21.15.90:0/2766188902 wait complete. 2024-05-27T07:49:33.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.624+0000 7efd35452b80 1 -- start start 2024-05-27T07:49:33.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.624+0000 7efd35452b80 1 Processor -- start 2024-05-27T07:49:33.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.625+0000 7efd35452b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d0c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.625+0000 7efd35452b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d0cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.625+0000 7efd35452b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.625+0000 7efd35452b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f1d4b26fa0 con 0x55f1d4d02e70 2024-05-27T07:49:33.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.625+0000 7efd35452b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f1d4bc1760 con 0x55f1d4c6f9f0 2024-05-27T07:49:33.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.625+0000 7efd35452b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f1d4d0fb80 con 0x55f1d4b83e40 2024-05-27T07:49:33.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.625+0000 7efd35452b80 10 client.0 _ll_register_callbacks cb 0x55f1d4bcac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:33.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.626+0000 7efd24a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.626+0000 7efd24a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33568/0 (socket says 172.21.15.90:33568) 2024-05-27T07:49:33.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.626+0000 7efd24a00700 1 -- 172.21.15.90:0/2082115711 learned_addr learned my addr 172.21.15.90:0/2082115711 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:33.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.626+0000 7efd1fe00700 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d0c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.626+0000 7efd1f400700 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d0cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.627+0000 7efd24a00700 1 -- 172.21.15.90:0/2082115711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 msgr2=0x55f1d4d0c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:33.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.627+0000 7efd24a00700 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d0c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.627+0000 7efd24a00700 1 -- 172.21.15.90:0/2082115711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 msgr2=0x55f1d4d0cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:33.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.627+0000 7efd24a00700 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d0cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.627+0000 7efd24a00700 1 -- 172.21.15.90:0/2082115711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d4b708d0 con 0x55f1d4d02e70 2024-05-27T07:49:33.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.627+0000 7efd1f400700 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d0cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:33.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.627+0000 7efd24a00700 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0d0e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efd18015000 tx=0x7efd1800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679ea39aa1950cb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:33.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.627+0000 7efd0a000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:33.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.628+0000 7efd0a000700 1 -- 172.21.15.90:0/2082115711 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1802c030 con 0x55f1d4d02e70 2024-05-27T07:49:33.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.628+0000 7efd0a000700 1 -- 172.21.15.90:0/2082115711 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7efd18004e00 con 0x55f1d4d02e70 2024-05-27T07:49:33.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.628+0000 7efd35452b80 1 -- 172.21.15.90:0/2082115711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd0c00fc90 con 0x55f1d4d02e70 2024-05-27T07:49:33.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.628+0000 7efd0a000700 1 -- 172.21.15.90:0/2082115711 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efd18007be0 con 0x55f1d4d02e70 2024-05-27T07:49:33.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.629+0000 7efd35452b80 10 client.16191 fetch_fsmap learned FSMap version 40 2024-05-27T07:49:33.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.629+0000 7efd35452b80 1 -- 172.21.15.90:0/2082115711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f1d4bf1060 con 0x55f1d4d02e70 2024-05-27T07:49:33.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.629+0000 7efd0a000700 1 -- 172.21.15.90:0/2082115711 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd18039070 con 0x55f1d4d02e70 2024-05-27T07:49:33.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.630+0000 7efd0a000700 1 -- 172.21.15.90:0/2082115711 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7efd18011070 con 0x55f1d4d02e70 2024-05-27T07:49:33.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.630+0000 7efd35452b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:49:33.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.630+0000 7efd35452b80 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:33.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.630+0000 7efd35452b80 20 client.16191 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:33.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.630+0000 7efd35452b80 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f1d4d2db10 0x55f1d4d2ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.630+0000 7efd35452b80 4 client.16191 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:33.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.630+0000 7efd35452b80 1 -- 172.21.15.90:0/2082115711 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1d4c32d90 con 0x55f1d4d2db10 2024-05-27T07:49:33.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.630+0000 7efd1fe00700 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f1d4d2db10 0x55f1d4d2ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.631+0000 7efd1fe00700 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f1d4d2db10 0x55f1d4d2ff50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a637883676c6cd2 server_cookie=dbe8b754a718efa2 in_seq=0 out_seq=0 2024-05-27T07:49:33.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.631+0000 7efd0a000700 10 client.16191 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:33.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.633+0000 7efd0a000700 1 -- 172.21.15.90:0/2082115711 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f1d4c32d90 con 0x55f1d4d2db10 2024-05-27T07:49:33.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.633+0000 7efd0a000700 10 client.16191 handle_command_reply: tid=0 2024-05-27T07:49:33.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.634+0000 7efd35452b80 1 client.16191 shutdown 2024-05-27T07:49:33.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.634+0000 7efd35452b80 1 -- 172.21.15.90:0/2082115711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 msgr2=0x55f1d4d0d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:33.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.634+0000 7efd35452b80 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0d0e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7efd18015000 tx=0x7efd1800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.635+0000 7efd35452b80 1 -- 172.21.15.90:0/2082115711 shutdown_connections 2024-05-27T07:49:33.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.635+0000 7efd35452b80 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f1d4d2db10 0x55f1d4d2ff50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.635+0000 7efd35452b80 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4d02e70 0x55f1d4d0d0e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.635+0000 7efd35452b80 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d0c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.635+0000 7efd0a000700 0 client.16191 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:33.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.635+0000 7efd35452b80 1 --2- 172.21.15.90:0/2082115711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d0cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.635+0000 7efd35452b80 1 -- 172.21.15.90:0/2082115711 >> 172.21.15.90:0/2082115711 conn(0x55f1d4c1d390 msgr2=0x55f1d4d02310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:33.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.635+0000 7efd35452b80 1 -- 172.21.15.90:0/2082115711 shutdown_connections 2024-05-27T07:49:33.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.635+0000 7efd35452b80 1 -- 172.21.15.90:0/2082115711 wait complete. 2024-05-27T07:49:33.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.635+0000 7efd35452b80 20 client.16191 trim_cache size 0 max 16384 2024-05-27T07:49:33.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.655+0000 7efd35452b80 1 Processor -- start 2024-05-27T07:49:33.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.655+0000 7efd35452b80 1 -- start start 2024-05-27T07:49:33.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.656+0000 7efd35452b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d07e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.656+0000 7efd35452b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.656+0000 7efd35452b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 0x55f1d4d35750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.656+0000 7efd35452b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd201afcd0 con 0x55f1d4c6f9f0 2024-05-27T07:49:33.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.656+0000 7efd35452b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd20075680 con 0x55f1d4d02e70 2024-05-27T07:49:33.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.656+0000 7efd35452b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd201af600 con 0x55f1d4b83e40 2024-05-27T07:49:33.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.656+0000 7efd1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.656+0000 7efd1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33576/0 (socket says 172.21.15.90:33576) 2024-05-27T07:49:33.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.656+0000 7efd1f400700 1 -- 172.21.15.90:0/2461628402 learned_addr learned my addr 172.21.15.90:0/2461628402 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:33.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.656+0000 7efd1fe00700 1 --2- 172.21.15.90:0/2461628402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d07e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.657+0000 7efd24a00700 1 --2- 172.21.15.90:0/2461628402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 0x55f1d4d35750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.657+0000 7efd1f400700 1 -- 172.21.15.90:0/2461628402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 msgr2=0x55f1d4d07e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:33.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.657+0000 7efd1f400700 1 --2- 172.21.15.90:0/2461628402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d07e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.657+0000 7efd1f400700 1 -- 172.21.15.90:0/2461628402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 msgr2=0x55f1d4d35750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:33.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.657+0000 7efd1f400700 1 --2- 172.21.15.90:0/2461628402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 0x55f1d4d35750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.657+0000 7efd1f400700 1 -- 172.21.15.90:0/2461628402 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d4d0b980 con 0x55f1d4c6f9f0 2024-05-27T07:49:33.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.657+0000 7efd24a00700 1 --2- 172.21.15.90:0/2461628402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 0x55f1d4d35750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:33.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.657+0000 7efd1f400700 1 --2- 172.21.15.90:0/2461628402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d027c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7efd0c019000 tx=0x7efd0c00b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bd61facfe48e3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:33.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.658+0000 7efd14c00700 1 -- 172.21.15.90:0/2461628402 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c015040 con 0x55f1d4c6f9f0 2024-05-27T07:49:33.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.658+0000 7efd14c00700 1 -- 172.21.15.90:0/2461628402 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd0c0044c0 con 0x55f1d4c6f9f0 2024-05-27T07:49:33.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.658+0000 7efd14c00700 1 -- 172.21.15.90:0/2461628402 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c004970 con 0x55f1d4c6f9f0 2024-05-27T07:49:33.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.659+0000 7efd35452b80 1 -- 172.21.15.90:0/2461628402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4c6f9f0 msgr2=0x55f1d4d027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:33.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.659+0000 7efd35452b80 1 --2- 172.21.15.90:0/2461628402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d027c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7efd0c019000 tx=0x7efd0c00b3c0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.659+0000 7efd35452b80 1 -- 172.21.15.90:0/2461628402 shutdown_connections 2024-05-27T07:49:33.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.659+0000 7efd35452b80 1 --2- 172.21.15.90:0/2461628402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 0x55f1d4d35750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.659+0000 7efd35452b80 1 --2- 172.21.15.90:0/2461628402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4b83e40 0x55f1d4d07e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.659+0000 7efd35452b80 1 --2- 172.21.15.90:0/2461628402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4c6f9f0 0x55f1d4d027c0 secure :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7efd0c019000 tx=0x7efd0c00b3c0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.659+0000 7efd35452b80 1 -- 172.21.15.90:0/2461628402 >> 172.21.15.90:0/2461628402 conn(0x55f1d4c1d390 msgr2=0x55f1d4d020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:33.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.660+0000 7efd35452b80 1 -- 172.21.15.90:0/2461628402 shutdown_connections 2024-05-27T07:49:33.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.660+0000 7efd35452b80 1 -- 172.21.15.90:0/2461628402 wait complete. 2024-05-27T07:49:33.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.660+0000 7efd35452b80 1 -- start start 2024-05-27T07:49:33.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.661+0000 7efd35452b80 1 Processor -- start 2024-05-27T07:49:33.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.661+0000 7efd35452b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4b83e40 0x55f1d4d06a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.661+0000 7efd35452b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 0x55f1d4d06f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.661+0000 7efd35452b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4d13e10 0x55f1d4d14220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.661+0000 7efd35452b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd201ae860 con 0x55f1d4b83e40 2024-05-27T07:49:33.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.661+0000 7efd35452b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd201ae190 con 0x55f1d4d02e70 2024-05-27T07:49:33.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.661+0000 7efd35452b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd201aef30 con 0x55f1d4d13e10 2024-05-27T07:49:33.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.661+0000 7efd35452b80 10 client.0 _ll_register_callbacks cb 0x55f1d4bcac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:33.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.662+0000 7efd24a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4d13e10 0x55f1d4d14220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.662+0000 7efd24a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4d13e10 0x55f1d4d14220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:48588/0 (socket says 172.21.15.90:48588) 2024-05-27T07:49:33.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.662+0000 7efd24a00700 1 -- 172.21.15.90:0/1038469283 learned_addr learned my addr 172.21.15.90:0/1038469283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:33.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.662+0000 7efd1f400700 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 0x55f1d4d06f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.663+0000 7efd1fe00700 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4b83e40 0x55f1d4d06a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.663+0000 7efd24a00700 1 -- 172.21.15.90:0/1038469283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 msgr2=0x55f1d4d06f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:33.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.663+0000 7efd24a00700 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 0x55f1d4d06f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.663+0000 7efd24a00700 1 -- 172.21.15.90:0/1038469283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4b83e40 msgr2=0x55f1d4d06a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:33.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.663+0000 7efd24a00700 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4b83e40 0x55f1d4d06a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.663+0000 7efd24a00700 1 -- 172.21.15.90:0/1038469283 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d4d14760 con 0x55f1d4d13e10 2024-05-27T07:49:33.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.664+0000 7efd24a00700 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4d13e10 0x55f1d4d14220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd18005110 tx=0x7efd18005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a1c4fd14910ffb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:33.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.664+0000 7efd0a000700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:49:33.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.665+0000 7efd0a000700 1 -- 172.21.15.90:0/1038469283 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1802c040 con 0x55f1d4d13e10 2024-05-27T07:49:33.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.665+0000 7efd0a000700 1 -- 172.21.15.90:0/1038469283 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7efd18005610 con 0x55f1d4d13e10 2024-05-27T07:49:33.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.665+0000 7efd35452b80 1 -- 172.21.15.90:0/1038469283 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd0c0044c0 con 0x55f1d4d13e10 2024-05-27T07:49:33.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.666+0000 7efd0a000700 1 -- 172.21.15.90:0/1038469283 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efd18043050 con 0x55f1d4d13e10 2024-05-27T07:49:33.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.666+0000 7efd35452b80 10 client.25795 fetch_fsmap learned FSMap version 40 2024-05-27T07:49:33.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.666+0000 7efd35452b80 1 -- 172.21.15.90:0/1038469283 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f1d4d15930 con 0x55f1d4d13e10 2024-05-27T07:49:33.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.666+0000 7efd0a000700 1 -- 172.21.15.90:0/1038469283 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd0c0044c0 con 0x55f1d4d13e10 2024-05-27T07:49:33.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.667+0000 7efd0a000700 1 -- 172.21.15.90:0/1038469283 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7efd18011050 con 0x55f1d4d13e10 2024-05-27T07:49:33.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.667+0000 7efd35452b80 10 client.25795 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:49:33.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.667+0000 7efd35452b80 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:33.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.667+0000 7efd35452b80 20 client.25795 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:33.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.667+0000 7efd35452b80 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f1d4d98a20 0x55f1d4d9ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:33.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.667+0000 7efd35452b80 4 client.25795 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:49:33.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.667+0000 7efd35452b80 1 -- 172.21.15.90:0/1038469283 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1d4c2c570 con 0x55f1d4d98a20 2024-05-27T07:49:33.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.668+0000 7efd1fe00700 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f1d4d98a20 0x55f1d4d9ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.670+0000 7efd1fe00700 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f1d4d98a20 0x55f1d4d9ae40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df1d7fce96f4680 server_cookie=3af3fc78e4b9a5bf in_seq=0 out_seq=0 2024-05-27T07:49:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.670+0000 7efd0a000700 10 client.25795 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:33.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.671+0000 7efd0a000700 1 -- 172.21.15.90:0/1038469283 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1d4c2c570 con 0x55f1d4d98a20 2024-05-27T07:49:33.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.671+0000 7efd0a000700 10 client.25795 handle_command_reply: tid=0 2024-05-27T07:49:33.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.671+0000 7efd35452b80 1 client.25795 shutdown 2024-05-27T07:49:33.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd35452b80 1 -- 172.21.15.90:0/1038469283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4d13e10 msgr2=0x55f1d4d14220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:33.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd35452b80 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4d13e10 0x55f1d4d14220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd18005110 tx=0x7efd18005f10 comp rx=0 tx=0).stop 2024-05-27T07:49:33.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd35452b80 1 -- 172.21.15.90:0/1038469283 shutdown_connections 2024-05-27T07:49:33.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd35452b80 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f1d4d98a20 0x55f1d4d9ae40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd35452b80 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1d4d13e10 0x55f1d4d14220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd35452b80 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f1d4b83e40 0x55f1d4d06a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd35452b80 1 --2- 172.21.15.90:0/1038469283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1d4d02e70 0x55f1d4d06f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd35452b80 1 -- 172.21.15.90:0/1038469283 >> 172.21.15.90:0/1038469283 conn(0x55f1d4c1d390 msgr2=0x55f1d4d03ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:33.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd0a000700 0 client.25795 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:33.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd35452b80 1 -- 172.21.15.90:0/1038469283 shutdown_connections 2024-05-27T07:49:33.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.673+0000 7efd35452b80 1 -- 172.21.15.90:0/1038469283 wait complete. 2024-05-27T07:49:33.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.674+0000 7efd35452b80 20 client.25795 trim_cache size 0 max 16384 2024-05-27T07:49:33.675 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:49:33.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 -- 172.21.15.90:0/839089541 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efd040b29f0 msgr2=0x7efd040b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:33.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efd040b29f0 0x7efd040b4f00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efd18009900 tx=0x7efd18009ca0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 -- 172.21.15.90:0/839089541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd20106560 msgr2=0x7efd201106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:33.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd20106560 0x7efd201106f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7efd1000d980 tx=0x7efd1000dd40 comp rx=0 tx=0).stop 2024-05-27T07:49:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 -- 172.21.15.90:0/839089541 shutdown_connections 2024-05-27T07:49:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd2010c5a0 0x7efd20111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd20106560 0x7efd201106f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efd040b29f0 0x7efd040b4f00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 --2- 172.21.15.90:0/839089541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd20106eb0 0x7efd20110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 -- 172.21.15.90:0/839089541 >> 172.21.15.90:0/839089541 conn(0x7efd20102040 msgr2=0x7efd201035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.675+0000 7efd26a00700 1 -- 172.21.15.90:0/839089541 shutdown_connections 2024-05-27T07:49:33.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:33.676+0000 7efd26a00700 1 -- 172.21.15.90:0/839089541 wait complete. 2024-05-27T07:49:34.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:34 smithi090 ceph-mon[31429]: pgmap v101: 129 pgs: 129 active+clean; 926 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 36 op/s 2024-05-27T07:49:34.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:34 smithi136 ceph-mon[35324]: pgmap v101: 129 pgs: 129 active+clean; 926 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 36 op/s 2024-05-27T07:49:34.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:34 smithi129 ceph-mon[35254]: pgmap v101: 129 pgs: 129 active+clean; 926 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 16 MiB/s wr, 36 op/s 2024-05-27T07:49:35.769 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:49:35.769 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:49:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:36 smithi090 ceph-mon[31429]: pgmap v102: 129 pgs: 129 active+clean; 926 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 28 op/s 2024-05-27T07:49:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:36.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:36 smithi136 ceph-mon[35324]: pgmap v102: 129 pgs: 129 active+clean; 926 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 28 op/s 2024-05-27T07:49:36.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:36 smithi129 ceph-mon[35254]: pgmap v102: 129 pgs: 129 active+clean; 926 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 28 op/s 2024-05-27T07:49:38.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:38 smithi090 ceph-mon[31429]: pgmap v103: 129 pgs: 129 active+clean; 926 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 28 op/s 2024-05-27T07:49:38.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:38 smithi136 ceph-mon[35324]: pgmap v103: 129 pgs: 129 active+clean; 926 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 28 op/s 2024-05-27T07:49:38.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.560+0000 7fccb0200700 1 -- 172.21.15.90:0/2611520647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcca8109860 msgr2=0x7fcca81111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:38.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.560+0000 7fccb0200700 1 --2- 172.21.15.90:0/2611520647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcca8109860 0x7fcca81111c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcca000b0d0 tx=0x7fcca000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.561+0000 7fccb0200700 1 -- 172.21.15.90:0/2611520647 shutdown_connections 2024-05-27T07:49:38.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.561+0000 7fccb0200700 1 --2- 172.21.15.90:0/2611520647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcca8109860 0x7fcca81111c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.561+0000 7fccb0200700 1 --2- 172.21.15.90:0/2611520647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcca81045e0 0x7fcca81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.561+0000 7fccb0200700 1 --2- 172.21.15.90:0/2611520647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcca8108f10 0x7fcca8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.561+0000 7fccb0200700 1 -- 172.21.15.90:0/2611520647 >> 172.21.15.90:0/2611520647 conn(0x7fcca8100030 msgr2=0x7fcca8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:38.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.561+0000 7fccb0200700 1 -- 172.21.15.90:0/2611520647 shutdown_connections 2024-05-27T07:49:38.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.561+0000 7fccb0200700 1 -- 172.21.15.90:0/2611520647 wait complete. 2024-05-27T07:49:38.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.562+0000 7fccb0200700 1 Processor -- start 2024-05-27T07:49:38.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.562+0000 7fccb0200700 1 -- start start 2024-05-27T07:49:38.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.563+0000 7fccb0200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcca81045e0 0x7fcca8075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.563+0000 7fccb0200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcca8108f10 0x7fcca8075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.563+0000 7fccb0200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcca8109860 0x7fcca8076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.563+0000 7fccb0200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcca8114390 con 0x7fcca81045e0 2024-05-27T07:49:38.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.563+0000 7fccb0200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcca8114210 con 0x7fcca8108f10 2024-05-27T07:49:38.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.563+0000 7fccb0200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcca8114510 con 0x7fcca8109860 2024-05-27T07:49:38.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.563+0000 7fccad800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcca81045e0 0x7fcca8075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.563+0000 7fccad800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcca81045e0 0x7fcca8075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33600/0 (socket says 172.21.15.90:33600) 2024-05-27T07:49:38.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.563+0000 7fccad800700 1 -- 172.21.15.90:0/4184863792 learned_addr learned my addr 172.21.15.90:0/4184863792 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:38.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:38 smithi129 ceph-mon[35254]: pgmap v103: 129 pgs: 129 active+clean; 926 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 MiB/s wr, 28 op/s 2024-05-27T07:49:38.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.563+0000 7fccace00700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcca8108f10 0x7fcca8075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.564+0000 7fccad800700 1 -- 172.21.15.90:0/4184863792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcca8109860 msgr2=0x7fcca8076530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:49:38.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.564+0000 7fccad800700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcca8109860 0x7fcca8076530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.564+0000 7fccad800700 1 -- 172.21.15.90:0/4184863792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcca8108f10 msgr2=0x7fcca8075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:38.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.564+0000 7fccad800700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcca8108f10 0x7fcca8075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.564+0000 7fccad800700 1 -- 172.21.15.90:0/4184863792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca81a9100 con 0x7fcca81045e0 2024-05-27T07:49:38.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.564+0000 7fccad800700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcca81045e0 0x7fcca8075ab0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00c920 tx=0x7fcc9c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:38.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.565+0000 7fcca6000700 1 -- 172.21.15.90:0/4184863792 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c007b00 con 0x7fcca81045e0 2024-05-27T07:49:38.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.565+0000 7fcca6000700 1 -- 172.21.15.90:0/4184863792 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc9c004cc0 con 0x7fcca81045e0 2024-05-27T07:49:38.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.565+0000 7fccb0200700 1 -- 172.21.15.90:0/4184863792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca81a93f0 con 0x7fcca81045e0 2024-05-27T07:49:38.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.565+0000 7fcca6000700 1 -- 172.21.15.90:0/4184863792 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0054f0 con 0x7fcca81045e0 2024-05-27T07:49:38.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.565+0000 7fccb0200700 1 -- 172.21.15.90:0/4184863792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca81a98e0 con 0x7fcca81045e0 2024-05-27T07:49:38.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.569+0000 7fcca6000700 1 -- 172.21.15.90:0/4184863792 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcc9c007ca0 con 0x7fcca81045e0 2024-05-27T07:49:38.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.570+0000 7fccbecb1b80 1 Processor -- start 2024-05-27T07:49:38.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.570+0000 7fccbecb1b80 1 -- start start 2024-05-27T07:49:38.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20cb83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20b39ec0 0x55ce20cb88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ce20be6ec0 con 0x55ce20cb8ef0 2024-05-27T07:49:38.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ce20bd6a60 con 0x55ce20b39ec0 2024-05-27T07:49:38.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ce20bdaaa0 con 0x55ce20c25a70 2024-05-27T07:49:38.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccace00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccace00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33604/0 (socket says 172.21.15.90:33604) 2024-05-27T07:49:38.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccace00700 1 -- 172.21.15.90:0/3151635419 learned_addr learned my addr 172.21.15.90:0/3151635419 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:38.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fcca6000700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcc900b2b40 0x7fcc900b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccae200700 1 --2- 172.21.15.90:0/3151635419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20cb83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.571+0000 7fccad800700 1 --2- 172.21.15.90:0/3151635419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20b39ec0 0x55ce20cb88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.572+0000 7fcca6000700 1 -- 172.21.15.90:0/4184863792 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fcc9c0e24d0 con 0x7fcca81045e0 2024-05-27T07:49:38.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.572+0000 7fccace00700 1 -- 172.21.15.90:0/3151635419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 msgr2=0x55ce20cb83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.572+0000 7fccae200700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcc900b2b40 0x7fcc900b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.572+0000 7fccace00700 1 --2- 172.21.15.90:0/3151635419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20cb83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.572+0000 7fccace00700 1 -- 172.21.15.90:0/3151635419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20b39ec0 msgr2=0x55ce20cb88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.572+0000 7fccace00700 1 --2- 172.21.15.90:0/3151635419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20b39ec0 0x55ce20cb88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.572+0000 7fccace00700 1 -- 172.21.15.90:0/3151635419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce20b9f4a0 con 0x55ce20cb8ef0 2024-05-27T07:49:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.572+0000 7fccace00700 1 --2- 172.21.15.90:0/3151635419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5190 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400dc80 tx=0x7fcc9400df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4383e18280864313 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.573+0000 7fcc9aa00700 1 -- 172.21.15.90:0/3151635419 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400fb70 con 0x55ce20cb8ef0 2024-05-27T07:49:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.573+0000 7fccae200700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcc900b2b40 0x7fcc900b5050 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcca000d010 tx=0x7fcca00099e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.573+0000 7fcc9aa00700 1 -- 172.21.15.90:0/3151635419 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc9400fd10 con 0x55ce20cb8ef0 2024-05-27T07:49:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.573+0000 7fcc9aa00700 1 -- 172.21.15.90:0/3151635419 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9401b670 con 0x55ce20cb8ef0 2024-05-27T07:49:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.574+0000 7fccbecb1b80 1 -- 172.21.15.90:0/3151635419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 msgr2=0x55ce20cc5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.574+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/3151635419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5190 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400dc80 tx=0x7fcc9400df90 comp rx=0 tx=0).stop 2024-05-27T07:49:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.574+0000 7fccbecb1b80 1 -- 172.21.15.90:0/3151635419 shutdown_connections 2024-05-27T07:49:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.574+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/3151635419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5190 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.574+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/3151635419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20cb83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.574+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/3151635419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20b39ec0 0x55ce20cb88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.574+0000 7fccbecb1b80 1 -- 172.21.15.90:0/3151635419 >> 172.21.15.90:0/3151635419 conn(0x55ce20bd3410 msgr2=0x55ce20cb4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:38.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.574+0000 7fccbecb1b80 1 -- 172.21.15.90:0/3151635419 shutdown_connections 2024-05-27T07:49:38.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.574+0000 7fccbecb1b80 1 -- 172.21.15.90:0/3151635419 wait complete. 2024-05-27T07:49:38.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.575+0000 7fccbecb1b80 1 -- start start 2024-05-27T07:49:38.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.575+0000 7fccbecb1b80 1 Processor -- start 2024-05-27T07:49:38.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.576+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cc26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.576+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20cc2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.576+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.576+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ce20add020 con 0x55ce20b39ec0 2024-05-27T07:49:38.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.576+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ce20b777e0 con 0x55ce20cb8ef0 2024-05-27T07:49:38.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.576+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ce20cc5c00 con 0x55ce20c25a70 2024-05-27T07:49:38.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.576+0000 7fccbecb1b80 10 client.0 _ll_register_callbacks cb 0x55ce20b80cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:38.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.576+0000 7fccad800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cc26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.576+0000 7fccad800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cc26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33618/0 (socket says 172.21.15.90:33618) 2024-05-27T07:49:38.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.576+0000 7fccad800700 1 -- 172.21.15.90:0/364923982 learned_addr learned my addr 172.21.15.90:0/364923982 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:38.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.577+0000 7fccae200700 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.577+0000 7fccace00700 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20cc2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.577+0000 7fccad800700 1 -- 172.21.15.90:0/364923982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 msgr2=0x55ce20cc2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:38.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.577+0000 7fccad800700 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20cc2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.577+0000 7fccad800700 1 -- 172.21.15.90:0/364923982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20cb8ef0 msgr2=0x55ce20cc3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:38.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.577+0000 7fccad800700 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.577+0000 7fccad800700 1 -- 172.21.15.90:0/364923982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce20b26950 con 0x55ce20b39ec0 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.577+0000 7fccace00700 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20cc2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.578+0000 7fccad800700 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cc26f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0df910 tx=0x7fcc9c0a99d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553181daf5c3f84a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.578+0000 7fcc8f400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.578+0000 7fcc8f400700 1 -- 172.21.15.90:0/364923982 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c01d040 con 0x55ce20b39ec0 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.578+0000 7fcc8f400700 1 -- 172.21.15.90:0/364923982 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fcc9c0b0d70 con 0x55ce20b39ec0 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.578+0000 7fccbecb1b80 1 -- 172.21.15.90:0/364923982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9400fd10 con 0x55ce20b39ec0 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.578+0000 7fcc8f400700 1 -- 172.21.15.90:0/364923982 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcc9c0a8f50 con 0x55ce20b39ec0 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.578+0000 7fccbecb1b80 10 client.16209 fetch_fsmap learned FSMap version 40 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.578+0000 7fccbecb1b80 1 -- 172.21.15.90:0/364923982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ce20ba70e0 con 0x55ce20b39ec0 2024-05-27T07:49:38.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.579+0000 7fcc8f400700 1 -- 172.21.15.90:0/364923982 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc9400fd10 con 0x55ce20b39ec0 2024-05-27T07:49:38.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.579+0000 7fcc8f400700 1 -- 172.21.15.90:0/364923982 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7fcc9c0a86a0 con 0x55ce20b39ec0 2024-05-27T07:49:38.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.579+0000 7fccbecb1b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:49:38.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.579+0000 7fccbecb1b80 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:38.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.579+0000 7fccbecb1b80 20 client.16209 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:38.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.579+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ce20cd2bc0 0x55ce20cd4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.579+0000 7fccbecb1b80 4 client.16209 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:38.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.579+0000 7fccbecb1b80 1 -- 172.21.15.90:0/364923982 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce20be8e10 con 0x55ce20cd2bc0 2024-05-27T07:49:38.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.580+0000 7fccace00700 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ce20cd2bc0 0x55ce20cd4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.581+0000 7fcc8f400700 10 client.16209 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:38.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.581+0000 7fccace00700 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ce20cd2bc0 0x55ce20cd4fe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0a7a2af58c04fb server_cookie=b423effaf6339586 in_seq=0 out_seq=0 2024-05-27T07:49:38.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.583+0000 7fcc8f400700 1 -- 172.21.15.90:0/364923982 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ce20be8e10 con 0x55ce20cd2bc0 2024-05-27T07:49:38.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.583+0000 7fcc8f400700 10 client.16209 handle_command_reply: tid=0 2024-05-27T07:49:38.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fccbecb1b80 1 client.16209 shutdown 2024-05-27T07:49:38.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fccbecb1b80 1 -- 172.21.15.90:0/364923982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 msgr2=0x55ce20cc26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:38.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cc26f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0df910 tx=0x7fcc9c0a99d0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fccbecb1b80 1 -- 172.21.15.90:0/364923982 shutdown_connections 2024-05-27T07:49:38.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ce20cd2bc0 0x55ce20cd4fe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cc26f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/364923982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20cc2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fcc8f400700 0 client.16209 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:38.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fccbecb1b80 1 -- 172.21.15.90:0/364923982 >> 172.21.15.90:0/364923982 conn(0x55ce20bd3410 msgr2=0x55ce20cb8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:38.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.584+0000 7fccbecb1b80 1 -- 172.21.15.90:0/364923982 shutdown_connections 2024-05-27T07:49:38.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.585+0000 7fccbecb1b80 1 -- 172.21.15.90:0/364923982 wait complete. 2024-05-27T07:49:38.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.585+0000 7fccbecb1b80 20 client.16209 trim_cache size 0 max 16384 2024-05-27T07:49:38.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.608+0000 7fccbecb1b80 1 Processor -- start 2024-05-27T07:49:38.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.608+0000 7fccbecb1b80 1 -- start start 2024-05-27T07:49:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.609+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cbdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.609+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20c25a70 0x55ce20cc30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.609+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20cb8ef0 0x55ce20cb8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.609+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b4140 con 0x55ce20b39ec0 2024-05-27T07:49:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.609+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b1f30 con 0x55ce20c25a70 2024-05-27T07:49:38.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.609+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b3a70 con 0x55ce20cb8ef0 2024-05-27T07:49:38.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.609+0000 7fccad800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cbdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.610+0000 7fccad800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cbdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33626/0 (socket says 172.21.15.90:33626) 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.610+0000 7fccad800700 1 -- 172.21.15.90:0/2063051113 learned_addr learned my addr 172.21.15.90:0/2063051113 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.610+0000 7fccace00700 1 --2- 172.21.15.90:0/2063051113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20c25a70 0x55ce20cc30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.610+0000 7fccae200700 1 --2- 172.21.15.90:0/2063051113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20cb8ef0 0x55ce20cb8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.610+0000 7fccad800700 1 -- 172.21.15.90:0/2063051113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20cb8ef0 msgr2=0x55ce20cb8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.610+0000 7fccad800700 1 --2- 172.21.15.90:0/2063051113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20cb8ef0 0x55ce20cb8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.610+0000 7fccad800700 1 -- 172.21.15.90:0/2063051113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20c25a70 msgr2=0x55ce20cc30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.610+0000 7fccad800700 1 --2- 172.21.15.90:0/2063051113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20c25a70 0x55ce20cc30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.610+0000 7fccad800700 1 -- 172.21.15.90:0/2063051113 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce20cb5080 con 0x55ce20b39ec0 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.610+0000 7fccad800700 1 --2- 172.21.15.90:0/2063051113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cbdf10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0b22a0 tx=0x7fcc9c0a8b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81190afba1de2f2a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:38.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.611+0000 7fcc9a000700 1 -- 172.21.15.90:0/2063051113 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c01d040 con 0x55ce20b39ec0 2024-05-27T07:49:38.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.611+0000 7fcc9a000700 1 -- 172.21.15.90:0/2063051113 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc9c0ad7f0 con 0x55ce20b39ec0 2024-05-27T07:49:38.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.611+0000 7fcc9a000700 1 -- 172.21.15.90:0/2063051113 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0a3b60 con 0x55ce20b39ec0 2024-05-27T07:49:38.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.612+0000 7fccbecb1b80 1 -- 172.21.15.90:0/2063051113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 msgr2=0x55ce20cbdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:38.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.612+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/2063051113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cbdf10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0b22a0 tx=0x7fcc9c0a8b00 comp rx=0 tx=0).stop 2024-05-27T07:49:38.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.612+0000 7fccbecb1b80 1 -- 172.21.15.90:0/2063051113 shutdown_connections 2024-05-27T07:49:38.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.612+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/2063051113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20cb8ef0 0x55ce20cb8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.612+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/2063051113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20b39ec0 0x55ce20cbdf10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.612+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/2063051113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20c25a70 0x55ce20cc30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.612+0000 7fccbecb1b80 1 -- 172.21.15.90:0/2063051113 >> 172.21.15.90:0/2063051113 conn(0x55ce20bd3410 msgr2=0x55ce20cb9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:38.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.612+0000 7fccbecb1b80 1 -- 172.21.15.90:0/2063051113 shutdown_connections 2024-05-27T07:49:38.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.612+0000 7fccbecb1b80 1 -- 172.21.15.90:0/2063051113 wait complete. 2024-05-27T07:49:38.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.612+0000 7fccbecb1b80 1 -- start start 2024-05-27T07:49:38.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.613+0000 7fccbecb1b80 1 Processor -- start 2024-05-27T07:49:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.613+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20b39ec0 0x55ce20ccb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.613+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20ccb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.613+0000 7fccbecb1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.613+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b2cd0 con 0x55ce20cb8ef0 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.613+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b2600 con 0x55ce20b39ec0 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.613+0000 7fccbecb1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcca81b33a0 con 0x55ce20c25a70 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.613+0000 7fccbecb1b80 10 client.0 _ll_register_callbacks cb 0x55ce20b80cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccae200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccae200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33638/0 (socket says 172.21.15.90:33638) 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccae200700 1 -- 172.21.15.90:0/261139151 learned_addr learned my addr 172.21.15.90:0/261139151 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccad800700 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20b39ec0 0x55ce20ccb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccace00700 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20ccb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccae200700 1 -- 172.21.15.90:0/261139151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 msgr2=0x55ce20ccb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccae200700 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20ccb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccae200700 1 -- 172.21.15.90:0/261139151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20b39ec0 msgr2=0x55ce20ccb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccae200700 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20b39ec0 0x55ce20ccb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccae200700 1 -- 172.21.15.90:0/261139151 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce20cc66b0 con 0x55ce20cb8ef0 2024-05-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.614+0000 7fccace00700 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20ccb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.615+0000 7fccae200700 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5f10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcca000b0d0 tx=0x7fcca000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fee2a26fa6b7cdc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.615+0000 7fcc8f400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.615+0000 7fcc8f400700 1 -- 172.21.15.90:0/261139151 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0032060 con 0x55ce20cb8ef0 2024-05-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.615+0000 7fccbecb1b80 1 -- 172.21.15.90:0/261139151 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9c0ad7f0 con 0x55ce20cb8ef0 2024-05-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.615+0000 7fcc8f400700 1 -- 172.21.15.90:0/261139151 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fcca0007b90 con 0x55ce20cb8ef0 2024-05-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.615+0000 7fcc8f400700 1 -- 172.21.15.90:0/261139151 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcca0024040 con 0x55ce20cb8ef0 2024-05-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.615+0000 7fccbecb1b80 10 client.16215 fetch_fsmap learned FSMap version 40 2024-05-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.616+0000 7fccbecb1b80 1 -- 172.21.15.90:0/261139151 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ce20d3e0d0 con 0x55ce20cb8ef0 2024-05-27T07:49:38.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.616+0000 7fcc8f400700 1 -- 172.21.15.90:0/261139151 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcc9c0ad7f0 con 0x55ce20cb8ef0 2024-05-27T07:49:38.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.616+0000 7fcc8f400700 1 -- 172.21.15.90:0/261139151 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7fcca002a040 con 0x55ce20cb8ef0 2024-05-27T07:49:38.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.616+0000 7fccbecb1b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:49:38.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.616+0000 7fccbecb1b80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:38.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.616+0000 7fccbecb1b80 20 client.16215 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:38.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.616+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ce20d3e7e0 0x55ce20d40c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:38.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.616+0000 7fccbecb1b80 4 client.16215 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:49:38.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.616+0000 7fccbecb1b80 1 -- 172.21.15.90:0/261139151 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ce20be25f0 con 0x55ce20d3e7e0 2024-05-27T07:49:38.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.617+0000 7fccad800700 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ce20d3e7e0 0x55ce20d40c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:38.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.618+0000 7fccad800700 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ce20d3e7e0 0x55ce20d40c00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712c21714be07ba4 server_cookie=8f1c35c54dadf9ee in_seq=0 out_seq=0 2024-05-27T07:49:38.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.618+0000 7fcc8f400700 10 client.16215 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:38.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.622+0000 7fcc8f400700 1 -- 172.21.15.90:0/261139151 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ce20be25f0 con 0x55ce20d3e7e0 2024-05-27T07:49:38.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.622+0000 7fcc8f400700 10 client.16215 handle_command_reply: tid=0 2024-05-27T07:49:38.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.622+0000 7fccbecb1b80 1 client.16215 shutdown 2024-05-27T07:49:38.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fccbecb1b80 1 -- 172.21.15.90:0/261139151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 msgr2=0x55ce20cc5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:38.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5f10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcca000b0d0 tx=0x7fcca000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fccbecb1b80 1 -- 172.21.15.90:0/261139151 shutdown_connections 2024-05-27T07:49:38.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ce20d3e7e0 0x55ce20d40c00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ce20cb8ef0 0x55ce20cc5f10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ce20b39ec0 0x55ce20ccb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fccbecb1b80 1 --2- 172.21.15.90:0/261139151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ce20c25a70 0x55ce20ccb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fccbecb1b80 1 -- 172.21.15.90:0/261139151 >> 172.21.15.90:0/261139151 conn(0x55ce20bd3410 msgr2=0x55ce20cc2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:38.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fcc8f400700 0 client.16215 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:38.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fccbecb1b80 1 -- 172.21.15.90:0/261139151 shutdown_connections 2024-05-27T07:49:38.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.623+0000 7fccbecb1b80 1 -- 172.21.15.90:0/261139151 wait complete. 2024-05-27T07:49:38.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.624+0000 7fccbecb1b80 20 client.16215 trim_cache size 0 max 16384 2024-05-27T07:49:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:49:38.625 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:49:38.625 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "79657e60-e554-4cd7-b486-fc3c86a24473", 2024-05-27T07:49:38.625 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:49:38.625 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:49:38.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.624+0000 7fccb0200700 1 -- 172.21.15.90:0/4184863792 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcc900b2b40 msgr2=0x7fcc900b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:38.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.624+0000 7fccb0200700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcc900b2b40 0x7fcc900b5050 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcca000d010 tx=0x7fcca00099e0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.624+0000 7fccb0200700 1 -- 172.21.15.90:0/4184863792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcca81045e0 msgr2=0x7fcca8075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:38.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.624+0000 7fccb0200700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcca81045e0 0x7fcca8075ab0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00c920 tx=0x7fcc9c00cce0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.625+0000 7fccb0200700 1 -- 172.21.15.90:0/4184863792 shutdown_connections 2024-05-27T07:49:38.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.625+0000 7fccb0200700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcca8109860 0x7fcca8076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.625+0000 7fccb0200700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcca81045e0 0x7fcca8075ab0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.625+0000 7fccb0200700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcc900b2b40 0x7fcc900b5050 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.625+0000 7fccb0200700 1 --2- 172.21.15.90:0/4184863792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcca8108f10 0x7fcca8075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:38.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.625+0000 7fccb0200700 1 -- 172.21.15.90:0/4184863792 >> 172.21.15.90:0/4184863792 conn(0x7fcca8100030 msgr2=0x7fcca810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:38.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.625+0000 7fccb0200700 1 -- 172.21.15.90:0/4184863792 shutdown_connections 2024-05-27T07:49:38.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:38.625+0000 7fccb0200700 1 -- 172.21.15.90:0/4184863792 wait complete. 2024-05-27T07:49:39.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:39 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:49:39.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:39 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:39.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:39 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:49:39.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:39 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:49:39.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:39 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:39.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:39 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:49:39.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:39 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:49:39.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:39 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:39.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:39 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:49:39.890 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:49:40.378 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:49:40.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:40 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:40.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:40 smithi090 ceph-mon[31429]: pgmap v104: 129 pgs: 129 active+clean; 1002 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 36 op/s 2024-05-27T07:49:40.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:40 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:40.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:40 smithi136 ceph-mon[35324]: pgmap v104: 129 pgs: 129 active+clean; 1002 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 36 op/s 2024-05-27T07:49:40.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:40 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:49:40.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:40 smithi129 ceph-mon[35254]: pgmap v104: 129 pgs: 129 active+clean; 1002 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 MiB/s wr, 36 op/s 2024-05-27T07:49:42.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:42 smithi090 ceph-mon[31429]: pgmap v105: 129 pgs: 129 active+clean; 1002 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-27T07:49:42.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:42 smithi136 ceph-mon[35324]: pgmap v105: 129 pgs: 129 active+clean; 1002 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-27T07:49:42.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:42 smithi129 ceph-mon[35254]: pgmap v105: 129 pgs: 129 active+clean; 1002 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 25 op/s 2024-05-27T07:49:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.752+0000 7fa8da800700 1 -- 172.21.15.90:0/2474744413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d41045e0 msgr2=0x7fa8d41049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:42.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.752+0000 7fa8da800700 1 --2- 172.21.15.90:0/2474744413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d41045e0 0x7fa8d41049f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400b0d0 tx=0x7fa8c400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.752+0000 7fa8da800700 1 -- 172.21.15.90:0/2474744413 shutdown_connections 2024-05-27T07:49:42.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.752+0000 7fa8da800700 1 --2- 172.21.15.90:0/2474744413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8d4109860 0x7fa8d41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.752+0000 7fa8da800700 1 --2- 172.21.15.90:0/2474744413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d41045e0 0x7fa8d41049f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.752+0000 7fa8da800700 1 --2- 172.21.15.90:0/2474744413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8d4108f10 0x7fa8d4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.752+0000 7fa8da800700 1 -- 172.21.15.90:0/2474744413 >> 172.21.15.90:0/2474744413 conn(0x7fa8d4100030 msgr2=0x7fa8d4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:42.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.753+0000 7fa8da800700 1 -- 172.21.15.90:0/2474744413 shutdown_connections 2024-05-27T07:49:42.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.753+0000 7fa8da800700 1 -- 172.21.15.90:0/2474744413 wait complete. 2024-05-27T07:49:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.754+0000 7fa8da800700 1 Processor -- start 2024-05-27T07:49:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.754+0000 7fa8da800700 1 -- start start 2024-05-27T07:49:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8da800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d41045e0 0x7fa8d419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8da800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8d4108f10 0x7fa8d419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8da800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8d4109860 0x7fa8d4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8da800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d40776e0 con 0x7fa8d41045e0 2024-05-27T07:49:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8da800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4077560 con 0x7fa8d4108f10 2024-05-27T07:49:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8da800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4077860 con 0x7fa8d4109860 2024-05-27T07:49:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8d3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d41045e0 0x7fa8d419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8d3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d41045e0 0x7fa8d419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58406/0 (socket says 172.21.15.90:58406) 2024-05-27T07:49:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8d3400700 1 -- 172.21.15.90:0/1905973318 learned_addr learned my addr 172.21.15.90:0/1905973318 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8d4108f10 0x7fa8d419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.755+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8d4109860 0x7fa8d4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.756+0000 7fa8d3400700 1 -- 172.21.15.90:0/1905973318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8d4109860 msgr2=0x7fa8d4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.756+0000 7fa8d3400700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8d4109860 0x7fa8d4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.756+0000 7fa8d3400700 1 -- 172.21.15.90:0/1905973318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8d4108f10 msgr2=0x7fa8d419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.756+0000 7fa8d3400700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8d4108f10 0x7fa8d419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.756+0000 7fa8d3400700 1 -- 172.21.15.90:0/1905973318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d4199040 con 0x7fa8d41045e0 2024-05-27T07:49:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.756+0000 7fa8d3400700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d41045e0 0x7fa8d419e770 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4007d90 tx=0x7fa8c4007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.756+0000 7fa8cbe00700 1 -- 172.21.15.90:0/1905973318 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c401f030 con 0x7fa8d41045e0 2024-05-27T07:49:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.757+0000 7fa8cbe00700 1 -- 172.21.15.90:0/1905973318 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8c401bad0 con 0x7fa8d41045e0 2024-05-27T07:49:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.757+0000 7fa8cbe00700 1 -- 172.21.15.90:0/1905973318 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4012b50 con 0x7fa8d41045e0 2024-05-27T07:49:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.757+0000 7fa8da800700 1 -- 172.21.15.90:0/1905973318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d41992d0 con 0x7fa8d41045e0 2024-05-27T07:49:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.757+0000 7fa8da800700 1 -- 172.21.15.90:0/1905973318 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d41a5600 con 0x7fa8d41045e0 2024-05-27T07:49:42.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.761+0000 7fa8cbe00700 1 -- 172.21.15.90:0/1905973318 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa8c4022030 con 0x7fa8d41045e0 2024-05-27T07:49:42.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.762+0000 7fa8e9292b80 1 Processor -- start 2024-05-27T07:49:42.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.762+0000 7fa8e9292b80 1 -- start start 2024-05-27T07:49:42.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.763+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c59be40 0x561c3c7207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.763+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c71a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.763+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c71adb0 0x561c3c71b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.763+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561c3c648e40 con 0x561c3c59be40 2024-05-27T07:49:42.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.763+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561c3c6389e0 con 0x561c3c720d30 2024-05-27T07:49:42.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.763+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561c3c63ca20 con 0x561c3c71adb0 2024-05-27T07:49:42.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.763+0000 7fa8d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c59be40 0x561c3c7207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.763+0000 7fa8d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c59be40 0x561c3c7207f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58410/0 (socket says 172.21.15.90:58410) 2024-05-27T07:49:42.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.763+0000 7fa8d3e00700 1 -- 172.21.15.90:0/3011693679 learned_addr learned my addr 172.21.15.90:0/3011693679 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:42.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.763+0000 7fa8cbe00700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8b80b6e00 0x7fa8b80b9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.764+0000 7fa8cbe00700 1 -- 172.21.15.90:0/1905973318 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fa8c400f020 con 0x7fa8d41045e0 2024-05-27T07:49:42.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.764+0000 7fa8d3e00700 1 -- 172.21.15.90:0/3011693679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c71adb0 msgr2=0x561c3c71b1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:49:42.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.764+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/3011693679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c71adb0 0x561c3c71b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.764+0000 7fa8d3e00700 1 -- 172.21.15.90:0/3011693679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 msgr2=0x561c3c71a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:42.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.764+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/3011693679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c71a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.764+0000 7fa8d3e00700 1 -- 172.21.15.90:0/3011693679 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3c601420 con 0x561c3c59be40 2024-05-27T07:49:42.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.764+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8b80b6e00 0x7fa8b80b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.764+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/3011693679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c59be40 0x561c3c7207f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00dd70 tx=0x7fa8cc00f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3945aeffb6b2540c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:42.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.765+0000 7fa8c8c00700 1 -- 172.21.15.90:0/3011693679 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc01cd50 con 0x561c3c59be40 2024-05-27T07:49:42.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.765+0000 7fa8c8c00700 1 -- 172.21.15.90:0/3011693679 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8cc01b8d0 con 0x561c3c59be40 2024-05-27T07:49:42.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.765+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8b80b6e00 0x7fa8b80b9310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc020000 tx=0x7fa8cc01d000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.765+0000 7fa8c8c00700 1 -- 172.21.15.90:0/3011693679 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc01a610 con 0x561c3c59be40 2024-05-27T07:49:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.766+0000 7fa8e9292b80 1 -- 172.21.15.90:0/3011693679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c59be40 msgr2=0x561c3c7207f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.766+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/3011693679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c59be40 0x561c3c7207f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00dd70 tx=0x7fa8cc00f640 comp rx=0 tx=0).stop 2024-05-27T07:49:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.766+0000 7fa8e9292b80 1 -- 172.21.15.90:0/3011693679 shutdown_connections 2024-05-27T07:49:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.766+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/3011693679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c71adb0 0x561c3c71b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.766+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/3011693679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c59be40 0x561c3c7207f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.766+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/3011693679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c71a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.766+0000 7fa8e9292b80 1 -- 172.21.15.90:0/3011693679 >> 172.21.15.90:0/3011693679 conn(0x561c3c5fbfc0 msgr2=0x561c3c7158d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:42.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.767+0000 7fa8e9292b80 1 -- 172.21.15.90:0/3011693679 shutdown_connections 2024-05-27T07:49:42.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.767+0000 7fa8e9292b80 1 -- 172.21.15.90:0/3011693679 wait complete. 2024-05-27T07:49:42.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.767+0000 7fa8e9292b80 1 -- start start 2024-05-27T07:49:42.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.767+0000 7fa8e9292b80 1 Processor -- start 2024-05-27T07:49:42.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.768+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c59be40 0x561c3c724580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.768+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c71adb0 0x561c3c724ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.768+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c720d30 0x561c3c725000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.768+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561c3c53efa0 con 0x561c3c720d30 2024-05-27T07:49:42.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.768+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561c3c5d9760 con 0x561c3c59be40 2024-05-27T07:49:42.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.768+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561c3c727800 con 0x561c3c71adb0 2024-05-27T07:49:42.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.768+0000 7fa8e9292b80 10 client.0 _ll_register_callbacks cb 0x561c3c5e2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:42.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.769+0000 7fa8d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c720d30 0x561c3c725000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.769+0000 7fa8d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c720d30 0x561c3c725000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58420/0 (socket says 172.21.15.90:58420) 2024-05-27T07:49:42.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.769+0000 7fa8d3e00700 1 -- 172.21.15.90:0/825283809 learned_addr learned my addr 172.21.15.90:0/825283809 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:42.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.769+0000 7fa8d3400700 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c59be40 0x561c3c724580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.769+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c71adb0 0x561c3c724ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.769+0000 7fa8d2a00700 1 -- 172.21.15.90:0/825283809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c59be40 msgr2=0x561c3c724580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.769+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c59be40 0x561c3c724580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.769+0000 7fa8d2a00700 1 -- 172.21.15.90:0/825283809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c720d30 msgr2=0x561c3c725000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.769+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c720d30 0x561c3c725000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.769+0000 7fa8d2a00700 1 -- 172.21.15.90:0/825283809 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3c5888d0 con 0x561c3c71adb0 2024-05-27T07:49:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.770+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c720d30 0x561c3c725000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:49:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.770+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c71adb0 0x561c3c724ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4107b50 tx=0x7fa8bc00f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eb939b48a136306 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.770+0000 7fa8c1600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:49:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.771+0000 7fa8c1600700 1 -- 172.21.15.90:0/825283809 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00fe30 con 0x561c3c71adb0 2024-05-27T07:49:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.771+0000 7fa8c1600700 1 -- 172.21.15.90:0/825283809 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fa8bc004030 con 0x561c3c71adb0 2024-05-27T07:49:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.771+0000 7fa8e9292b80 1 -- 172.21.15.90:0/825283809 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8cc01b8d0 con 0x561c3c71adb0 2024-05-27T07:49:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.771+0000 7fa8c1600700 1 -- 172.21.15.90:0/825283809 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa8bc022050 con 0x561c3c71adb0 2024-05-27T07:49:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.771+0000 7fa8e9292b80 10 client.25819 fetch_fsmap learned FSMap version 40 2024-05-27T07:49:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.771+0000 7fa8e9292b80 1 -- 172.21.15.90:0/825283809 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561c3c609060 con 0x561c3c71adb0 2024-05-27T07:49:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.772+0000 7fa8c1600700 1 -- 172.21.15.90:0/825283809 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8cc01b8d0 con 0x561c3c71adb0 2024-05-27T07:49:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.772+0000 7fa8c1600700 1 -- 172.21.15.90:0/825283809 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7fa8bc011440 con 0x561c3c71adb0 2024-05-27T07:49:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.773+0000 7fa8e9292b80 10 client.25819 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:49:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.773+0000 7fa8e9292b80 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.773+0000 7fa8e9292b80 20 client.25819 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.773+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561c3c73cfe0 0x561c3c73f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.773+0000 7fa8e9292b80 4 client.25819 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.773+0000 7fa8e9292b80 1 -- 172.21.15.90:0/825283809 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c3c64ad90 con 0x561c3c73cfe0 2024-05-27T07:49:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.774+0000 7fa8d3400700 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561c3c73cfe0 0x561c3c73f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.775+0000 7fa8c1600700 10 client.25819 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:42.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.775+0000 7fa8d3400700 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561c3c73cfe0 0x561c3c73f400 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac06515c757897e2 server_cookie=927d919919cfc308 in_seq=0 out_seq=0 2024-05-27T07:49:42.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8c1600700 1 -- 172.21.15.90:0/825283809 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561c3c64ad90 con 0x561c3c73cfe0 2024-05-27T07:49:42.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8c1600700 10 client.25819 handle_command_reply: tid=0 2024-05-27T07:49:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8e9292b80 1 client.25819 shutdown 2024-05-27T07:49:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8e9292b80 1 -- 172.21.15.90:0/825283809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c71adb0 msgr2=0x561c3c724ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c71adb0 0x561c3c724ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4107b50 tx=0x7fa8bc00f640 comp rx=0 tx=0).stop 2024-05-27T07:49:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8e9292b80 1 -- 172.21.15.90:0/825283809 shutdown_connections 2024-05-27T07:49:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561c3c73cfe0 0x561c3c73f400 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c720d30 0x561c3c725000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c59be40 0x561c3c724580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/825283809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c71adb0 0x561c3c724ac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8e9292b80 1 -- 172.21.15.90:0/825283809 >> 172.21.15.90:0/825283809 conn(0x561c3c5fbfc0 msgr2=0x561c3c71b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.777+0000 7fa8c1600700 0 client.25819 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.778+0000 7fa8e9292b80 1 -- 172.21.15.90:0/825283809 shutdown_connections 2024-05-27T07:49:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.778+0000 7fa8e9292b80 1 -- 172.21.15.90:0/825283809 wait complete. 2024-05-27T07:49:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.778+0000 7fa8e9292b80 20 client.25819 trim_cache size 0 max 16384 2024-05-27T07:49:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.808+0000 7fa8e9292b80 1 Processor -- start 2024-05-27T07:49:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.808+0000 7fa8e9292b80 1 -- start start 2024-05-27T07:49:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.808+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c59be40 0x561c3c71a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.809+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c71ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.809+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c739130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.809+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41afce0 con 0x561c3c71adb0 2024-05-27T07:49:42.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.809+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41adad0 con 0x561c3c720d30 2024-05-27T07:49:42.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.809+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41af610 con 0x561c3c59be40 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.809+0000 7fa8d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c71ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.809+0000 7fa8d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c71ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58424/0 (socket says 172.21.15.90:58424) 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.809+0000 7fa8d2a00700 1 -- 172.21.15.90:0/2010877590 learned_addr learned my addr 172.21.15.90:0/2010877590 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.809+0000 7fa8d3400700 1 --2- 172.21.15.90:0/2010877590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c59be40 0x561c3c71a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.809+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/2010877590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c739130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.810+0000 7fa8d2a00700 1 -- 172.21.15.90:0/2010877590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c59be40 msgr2=0x561c3c71a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.810+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/2010877590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c59be40 0x561c3c71a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.810+0000 7fa8d2a00700 1 -- 172.21.15.90:0/2010877590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 msgr2=0x561c3c739130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.810+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/2010877590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c739130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.810+0000 7fa8d2a00700 1 -- 172.21.15.90:0/2010877590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3c73f8a0 con 0x561c3c71adb0 2024-05-27T07:49:42.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.810+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/2010877590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c71ec40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0117e0 tx=0x7fa8bc00b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5e3b6e35a81fd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:42.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.810+0000 7fa8c3e00700 1 -- 172.21.15.90:0/2010877590 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00be00 con 0x561c3c71adb0 2024-05-27T07:49:42.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.811+0000 7fa8c3e00700 1 -- 172.21.15.90:0/2010877590 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8bc02a070 con 0x561c3c71adb0 2024-05-27T07:49:42.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.811+0000 7fa8c3e00700 1 -- 172.21.15.90:0/2010877590 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0052a0 con 0x561c3c71adb0 2024-05-27T07:49:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.812+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2010877590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 msgr2=0x561c3c71ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.812+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/2010877590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c71ec40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0117e0 tx=0x7fa8bc00b5c0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.812+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2010877590 shutdown_connections 2024-05-27T07:49:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.812+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/2010877590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c739130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.812+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/2010877590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c59be40 0x561c3c71a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.812+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/2010877590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c71ec40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.812+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2010877590 >> 172.21.15.90:0/2010877590 conn(0x561c3c5fbfc0 msgr2=0x561c3c71f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.812+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2010877590 shutdown_connections 2024-05-27T07:49:42.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.812+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2010877590 wait complete. 2024-05-27T07:49:42.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.812+0000 7fa8e9292b80 1 -- start start 2024-05-27T07:49:42.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.813+0000 7fa8e9292b80 1 Processor -- start 2024-05-27T07:49:42.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.813+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c59be40 0x561c3c735940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.813+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c744300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.813+0000 7fa8e9292b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c744840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.813+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41ae870 con 0x561c3c71adb0 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.813+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41ae1a0 con 0x561c3c720d30 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.813+0000 7fa8e9292b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41aef40 con 0x561c3c59be40 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.813+0000 7fa8e9292b80 10 client.0 _ll_register_callbacks cb 0x561c3c5e2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c744300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c744300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58426/0 (socket says 172.21.15.90:58426) 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d2a00700 1 -- 172.21.15.90:0/2093932669 learned_addr learned my addr 172.21.15.90:0/2093932669 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d3400700 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c59be40 0x561c3c735940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c744840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d2a00700 1 -- 172.21.15.90:0/2093932669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c59be40 msgr2=0x561c3c735940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c59be40 0x561c3c735940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d2a00700 1 -- 172.21.15.90:0/2093932669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 msgr2=0x561c3c744840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c744840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d2a00700 1 -- 172.21.15.90:0/2093932669 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3c725370 con 0x561c3c71adb0 2024-05-27T07:49:42.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.814+0000 7fa8d3e00700 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c744840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:42.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.815+0000 7fa8d2a00700 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c744300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b490 tx=0x7fa8bc011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99acfea3f2792cd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:42.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.815+0000 7fa8c1600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:42.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.815+0000 7fa8c1600700 1 -- 172.21.15.90:0/2093932669 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc005240 con 0x561c3c71adb0 2024-05-27T07:49:42.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.815+0000 7fa8c1600700 1 -- 172.21.15.90:0/2093932669 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fa8bc0053e0 con 0x561c3c71adb0 2024-05-27T07:49:42.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.815+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2093932669 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8bc02a070 con 0x561c3c71adb0 2024-05-27T07:49:42.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.816+0000 7fa8c1600700 1 -- 172.21.15.90:0/2093932669 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa8bc041050 con 0x561c3c71adb0 2024-05-27T07:49:42.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.816+0000 7fa8e9292b80 10 client.16233 fetch_fsmap learned FSMap version 40 2024-05-27T07:49:42.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.816+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2093932669 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561c3c7a7680 con 0x561c3c71adb0 2024-05-27T07:49:42.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.816+0000 7fa8c1600700 1 -- 172.21.15.90:0/2093932669 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8bc02a070 con 0x561c3c71adb0 2024-05-27T07:49:42.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.816+0000 7fa8c1600700 1 -- 172.21.15.90:0/2093932669 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7fa8bc038520 con 0x561c3c71adb0 2024-05-27T07:49:42.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.816+0000 7fa8e9292b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:49:42.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.816+0000 7fa8e9292b80 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:42.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.816+0000 7fa8e9292b80 20 client.16233 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:42.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.817+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561c3c7a7c70 0x561c3c7aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:42.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.817+0000 7fa8e9292b80 4 client.16233 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:49:42.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.817+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2093932669 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561c3c644570 con 0x561c3c7a7c70 2024-05-27T07:49:42.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.817+0000 7fa8d3400700 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561c3c7a7c70 0x561c3c7aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:42.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.818+0000 7fa8d3400700 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561c3c7a7c70 0x561c3c7aa090 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e229587d883f86e6 server_cookie=ffd5762e2f31cd80 in_seq=0 out_seq=0 2024-05-27T07:49:42.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.818+0000 7fa8c1600700 10 client.16233 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:42.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.819+0000 7fa8c1600700 1 -- 172.21.15.90:0/2093932669 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561c3c644570 con 0x561c3c7a7c70 2024-05-27T07:49:42.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.819+0000 7fa8c1600700 10 client.16233 handle_command_reply: tid=0 2024-05-27T07:49:42.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.819+0000 7fa8e9292b80 1 client.16233 shutdown 2024-05-27T07:49:42.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.819+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2093932669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 msgr2=0x561c3c744300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:42.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.819+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c744300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b490 tx=0x7fa8bc011a00 comp rx=0 tx=0).stop 2024-05-27T07:49:42.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.819+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2093932669 shutdown_connections 2024-05-27T07:49:42.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.819+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561c3c7a7c70 0x561c3c7aa090 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.819+0000 7fa8c1600700 0 client.16233 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:42.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.820+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c3c720d30 0x561c3c744840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.820+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c3c59be40 0x561c3c735940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.820+0000 7fa8e9292b80 1 --2- 172.21.15.90:0/2093932669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c3c71adb0 0x561c3c744300 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.820+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2093932669 >> 172.21.15.90:0/2093932669 conn(0x561c3c5fbfc0 msgr2=0x561c3c71f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:42.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.820+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2093932669 shutdown_connections 2024-05-27T07:49:42.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.820+0000 7fa8e9292b80 1 -- 172.21.15.90:0/2093932669 wait complete. 2024-05-27T07:49:42.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.820+0000 7fa8e9292b80 20 client.16233 trim_cache size 0 max 16384 2024-05-27T07:49:42.821 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:49:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-27T07:49:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": { 2024-05-27T07:49:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "79657e60-e554-4cd7-b486-fc3c86a24473": { 2024-05-27T07:49:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "path": "/", 2024-05-27T07:49:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "tag": "79657e60-e554-4cd7-b486-fc3c86a24473", 2024-05-27T07:49:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: "options": "recursive,force" 2024-05-27T07:49:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T07:49:42.821 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T07:49:42.822 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:49:42.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.821+0000 7fa8da800700 1 -- 172.21.15.90:0/1905973318 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8b80b6e00 msgr2=0x7fa8b80b9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:42.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.821+0000 7fa8da800700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8b80b6e00 0x7fa8b80b9310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc020000 tx=0x7fa8cc01d000 comp rx=0 tx=0).stop 2024-05-27T07:49:42.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.821+0000 7fa8da800700 1 -- 172.21.15.90:0/1905973318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d41045e0 msgr2=0x7fa8d419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:42.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.821+0000 7fa8da800700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d41045e0 0x7fa8d419e770 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4007d90 tx=0x7fa8c4007e70 comp rx=0 tx=0).stop 2024-05-27T07:49:42.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.821+0000 7fa8da800700 1 -- 172.21.15.90:0/1905973318 shutdown_connections 2024-05-27T07:49:42.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.821+0000 7fa8da800700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8d4109860 0x7fa8d4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.821+0000 7fa8da800700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d41045e0 0x7fa8d419e770 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.821+0000 7fa8da800700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa8b80b6e00 0x7fa8b80b9310 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.821+0000 7fa8da800700 1 --2- 172.21.15.90:0/1905973318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8d4108f10 0x7fa8d419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:42.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.821+0000 7fa8da800700 1 -- 172.21.15.90:0/1905973318 >> 172.21.15.90:0/1905973318 conn(0x7fa8d4100030 msgr2=0x7fa8d4102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:42.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.822+0000 7fa8da800700 1 -- 172.21.15.90:0/1905973318 shutdown_connections 2024-05-27T07:49:42.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:49:42.822+0000 7fa8da800700 1 -- 172.21.15.90:0/1905973318 wait complete. 2024-05-27T07:49:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.187+0000 7fda1f200640 1 -- 172.21.15.90:0/722459390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda20104250 msgr2=0x7fda201105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.187+0000 7fda1f200640 1 --2- 172.21.15.90:0/722459390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda20104250 0x7fda201105d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fda18009ca0 tx=0x7fda1802a8c0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.187+0000 7fda1f200640 1 -- 172.21.15.90:0/722459390 shutdown_connections 2024-05-27T07:49:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.187+0000 7fda1f200640 1 --2- 172.21.15.90:0/722459390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda20109f50 0x7fda20103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.187+0000 7fda1f200640 1 --2- 172.21.15.90:0/722459390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda20104250 0x7fda201105d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.187+0000 7fda1f200640 1 --2- 172.21.15.90:0/722459390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda20109ab0 0x7fda201036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.187+0000 7fda1f200640 1 -- 172.21.15.90:0/722459390 >> 172.21.15.90:0/722459390 conn(0x7fda20079240 msgr2=0x7fda20079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.187+0000 7fda1f200640 1 -- 172.21.15.90:0/722459390 shutdown_connections 2024-05-27T07:49:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.187+0000 7fda1f200640 1 -- 172.21.15.90:0/722459390 wait complete. 2024-05-27T07:49:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.188+0000 7fda1f200640 1 Processor -- start 2024-05-27T07:49:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.189+0000 7fda1f200640 1 -- start start 2024-05-27T07:49:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.189+0000 7fda1f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda20104250 0x7fda2019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.189+0000 7fda1f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda20109ab0 0x7fda2019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.189+0000 7fda1f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda20109f50 0x7fda201a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.189+0000 7fda1f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fda20113ba0 con 0x7fda20109ab0 2024-05-27T07:49:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.189+0000 7fda1f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda20113a20 con 0x7fda20104250 2024-05-27T07:49:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.189+0000 7fda1f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fda20113d20 con 0x7fda20109f50 2024-05-27T07:49:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.189+0000 7fda1d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda20109ab0 0x7fda2019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.189+0000 7fda1d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda20109ab0 0x7fda2019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58450/0 (socket says 172.21.15.90:58450) 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.189+0000 7fda1d400640 1 -- 172.21.15.90:0/3431086047 learned_addr learned my addr 172.21.15.90:0/3431086047 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.190+0000 7fda1de00640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda20104250 0x7fda2019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.190+0000 7fda1e800640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda20109f50 0x7fda201a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.190+0000 7fda1d400640 1 -- 172.21.15.90:0/3431086047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda20109f50 msgr2=0x7fda201a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.190+0000 7fda1d400640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda20109f50 0x7fda201a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.190+0000 7fda1d400640 1 -- 172.21.15.90:0/3431086047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda20104250 msgr2=0x7fda2019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.190+0000 7fda1d400640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda20104250 0x7fda2019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.190+0000 7fda1d400640 1 -- 172.21.15.90:0/3431086047 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda201a1620 con 0x7fda20109ab0 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.190+0000 7fda1de00640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda20104250 0x7fda2019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.190+0000 7fda1e800640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda20109f50 0x7fda201a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.190+0000 7fda1d400640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda20109ab0 0x7fda2019cd20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fda1000d740 tx=0x7fda1000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.191+0000 7fda16a00640 1 -- 172.21.15.90:0/3431086047 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10017070 con 0x7fda20109ab0 2024-05-27T07:49:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.191+0000 7fda16a00640 1 -- 172.21.15.90:0/3431086047 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda1000de50 con 0x7fda20109ab0 2024-05-27T07:49:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.191+0000 7fda1f200640 1 -- 172.21.15.90:0/3431086047 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda201a1910 con 0x7fda20109ab0 2024-05-27T07:49:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.191+0000 7fda16a00640 1 -- 172.21.15.90:0/3431086047 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10003bb0 con 0x7fda20109ab0 2024-05-27T07:49:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.191+0000 7fda1f200640 1 -- 172.21.15.90:0/3431086047 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda201a9290 con 0x7fda20109ab0 2024-05-27T07:49:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.195+0000 7fda2c018740 1 Processor -- start 2024-05-27T07:49:43.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.195+0000 7fda2c018740 1 -- start start 2024-05-27T07:49:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.195+0000 7fda16a00640 1 -- 172.21.15.90:0/3431086047 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fda10003d50 con 0x7fda20109ab0 2024-05-27T07:49:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.196+0000 7fda2c018740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 0x562fe89aba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.196+0000 7fda2c018740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.196+0000 7fda2c018740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.196+0000 7fda2c018740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562fe891cb90 con 0x562fe85e02f0 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.196+0000 7fda2c018740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562fe890c730 con 0x562fe85e84d0 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.196+0000 7fda2c018740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562fe8910770 con 0x562fe86091b0 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.196+0000 7fda1de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.196+0000 7fda1de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89abfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58462/0 (socket says 172.21.15.90:58462) 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.196+0000 7fda1de00640 1 -- 172.21.15.90:0/815767759 learned_addr learned my addr 172.21.15.90:0/815767759 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.196+0000 7fda1e800640 1 --2- 172.21.15.90:0/815767759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 0x562fe89aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.197+0000 7fda1d400640 1 --2- 172.21.15.90:0/815767759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.197+0000 7fda1de00640 1 -- 172.21.15.90:0/815767759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 msgr2=0x562fe89aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.197+0000 7fda1de00640 1 --2- 172.21.15.90:0/815767759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 0x562fe89aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.197+0000 7fda1de00640 1 -- 172.21.15.90:0/815767759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 msgr2=0x562fe89ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:43.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.197+0000 7fda1de00640 1 --2- 172.21.15.90:0/815767759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89ac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.197+0000 7fda1de00640 1 -- 172.21.15.90:0/815767759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe88a58b0 con 0x562fe85e02f0 2024-05-27T07:49:43.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.197+0000 7fda1d400640 1 --2- 172.21.15.90:0/815767759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.197+0000 7fda1de00640 1 --2- 172.21.15.90:0/815767759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89abfa0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fda0800bd30 tx=0x7fda0800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5e81564f8374ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.197+0000 7fda0f400640 1 -- 172.21.15.90:0/815767759 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08015070 con 0x562fe85e02f0 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.198+0000 7fda0f400640 1 -- 172.21.15.90:0/815767759 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda0800be20 con 0x562fe85e02f0 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.198+0000 7fda16a00640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fda040b2950 0x7fda040b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.198+0000 7fda0f400640 1 -- 172.21.15.90:0/815767759 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08011900 con 0x562fe85e02f0 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.198+0000 7fda16a00640 1 -- 172.21.15.90:0/3431086047 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fda100e23d0 con 0x7fda20109ab0 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.198+0000 7fda1e800640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fda040b2950 0x7fda040b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.199+0000 7fda2c018740 1 -- 172.21.15.90:0/815767759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 msgr2=0x562fe89abfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.199+0000 7fda2c018740 1 --2- 172.21.15.90:0/815767759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89abfa0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fda0800bd30 tx=0x7fda0800f730 comp rx=0 tx=0).stop 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.199+0000 7fda1e800640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fda040b2950 0x7fda040b4e60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fda18005cc0 tx=0x7fda18005c30 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.199+0000 7fda2c018740 1 -- 172.21.15.90:0/815767759 shutdown_connections 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.199+0000 7fda2c018740 1 --2- 172.21.15.90:0/815767759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.199+0000 7fda2c018740 1 --2- 172.21.15.90:0/815767759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89abfa0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.199+0000 7fda2c018740 1 --2- 172.21.15.90:0/815767759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 0x562fe89aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.199+0000 7fda2c018740 1 -- 172.21.15.90:0/815767759 >> 172.21.15.90:0/815767759 conn(0x562fe85fd300 msgr2=0x562fe89a8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.199+0000 7fda2c018740 1 -- 172.21.15.90:0/815767759 shutdown_connections 2024-05-27T07:49:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.199+0000 7fda2c018740 1 -- 172.21.15.90:0/815767759 wait complete. 2024-05-27T07:49:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.200+0000 7fda2c018740 1 -- start start 2024-05-27T07:49:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.200+0000 7fda2c018740 1 Processor -- start 2024-05-27T07:49:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.200+0000 7fda2c018740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.200+0000 7fda2c018740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda2c018740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 0x562fe89c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda2c018740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562fe85f2790 con 0x562fe85e02f0 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda2c018740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562fe88b4bc0 con 0x562fe85e84d0 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda2c018740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562fe89b8b90 con 0x562fe86091b0 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda2c018740 10 client.0 _ll_register_callbacks cb 0x562fe8892400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58466/0 (socket says 172.21.15.90:58466) 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1de00640 1 -- 172.21.15.90:0/3907264891 learned_addr learned my addr 172.21.15.90:0/3907264891 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1d400640 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1e800640 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 0x562fe89c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1de00640 1 -- 172.21.15.90:0/3907264891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 msgr2=0x562fe89c1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1de00640 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 0x562fe89c1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1de00640 1 -- 172.21.15.90:0/3907264891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 msgr2=0x562fe89b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1de00640 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1de00640 1 -- 172.21.15.90:0/3907264891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe8621210 con 0x562fe85e02f0 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1e800640 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 0x562fe89c1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.201+0000 7fda1d400640 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.202+0000 7fda1de00640 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89b5c80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fda08019000 tx=0x7fda08005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb59e25da6d55d22 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.202+0000 7fda03e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.202+0000 7fda03e00640 1 -- 172.21.15.90:0/3907264891 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08028060 con 0x562fe85e02f0 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.202+0000 7fda2c018740 1 -- 172.21.15.90:0/3907264891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda0800be20 con 0x562fe85e02f0 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.202+0000 7fda03e00640 1 -- 172.21.15.90:0/3907264891 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fda080116e0 con 0x562fe85e02f0 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.202+0000 7fda03e00640 1 -- 172.21.15.90:0/3907264891 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fda0802f050 con 0x562fe85e02f0 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.202+0000 7fda2c018740 10 client.16245 fetch_fsmap learned FSMap version 40 2024-05-27T07:49:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.202+0000 7fda2c018740 1 -- 172.21.15.90:0/3907264891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562fe86ed620 con 0x562fe85e02f0 2024-05-27T07:49:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.203+0000 7fda03e00640 1 -- 172.21.15.90:0/3907264891 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda0800be20 con 0x562fe85e02f0 2024-05-27T07:49:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.203+0000 7fda03e00640 1 -- 172.21.15.90:0/3907264891 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7fda08004ad0 con 0x562fe85e02f0 2024-05-27T07:49:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.203+0000 7fda2c018740 10 client.16245 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:49:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.203+0000 7fda2c018740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.203+0000 7fda2c018740 20 client.16245 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.203+0000 7fda2c018740 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562fe85c7d80 0x562fe89c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.203+0000 7fda2c018740 4 client.16245 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.203+0000 7fda2c018740 1 -- 172.21.15.90:0/3907264891 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fe891eae0 con 0x562fe85c7d80 2024-05-27T07:49:43.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.204+0000 7fda1d400640 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562fe85c7d80 0x562fe89c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.205+0000 7fda1d400640 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562fe85c7d80 0x562fe89c7d60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f35fd996662a59a server_cookie=87d7575f75f776fd in_seq=0 out_seq=0 2024-05-27T07:49:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.205+0000 7fda03e00640 10 client.16245 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda03e00640 1 -- 172.21.15.90:0/3907264891 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562fe891eae0 con 0x562fe85c7d80 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda03e00640 10 client.16245 handle_command_reply: tid=0 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda2c018740 1 client.16245 shutdown 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda2c018740 1 -- 172.21.15.90:0/3907264891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 msgr2=0x562fe89b5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda2c018740 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89b5c80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fda08019000 tx=0x7fda08005270 comp rx=0 tx=0).stop 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda2c018740 1 -- 172.21.15.90:0/3907264891 shutdown_connections 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda2c018740 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562fe85c7d80 0x562fe89c7d60 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda2c018740 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda2c018740 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe86091b0 0x562fe89c1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda03e00640 0 client.16245 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda2c018740 1 --2- 172.21.15.90:0/3907264891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e02f0 0x562fe89b5c80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.207+0000 7fda2c018740 1 -- 172.21.15.90:0/3907264891 >> 172.21.15.90:0/3907264891 conn(0x562fe85fd300 msgr2=0x562fe89ab890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.208+0000 7fda2c018740 1 -- 172.21.15.90:0/3907264891 shutdown_connections 2024-05-27T07:49:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.208+0000 7fda2c018740 1 -- 172.21.15.90:0/3907264891 wait complete. 2024-05-27T07:49:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.208+0000 7fda2c018740 20 client.16245 trim_cache size 0 max 16384 2024-05-27T07:49:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.226+0000 7fda2c018740 1 Processor -- start 2024-05-27T07:49:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.226+0000 7fda2c018740 1 -- start start 2024-05-27T07:49:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.227+0000 7fda2c018740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 0x562fe89b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.227+0000 7fda2c018740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e84d0 0x562fe89cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.227+0000 7fda2c018740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe86091b0 0x562fe89cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.227+0000 7fda2c018740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fda201b4da0 con 0x562fe85e84d0 2024-05-27T07:49:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.227+0000 7fda2c018740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda201b2b90 con 0x562fe86091b0 2024-05-27T07:49:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.227+0000 7fda2c018740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fda201b46d0 con 0x562fe85e02f0 2024-05-27T07:49:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.228+0000 7fda1de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 0x562fe89b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.228+0000 7fda1de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 0x562fe89b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:40492/0 (socket says 172.21.15.90:40492) 2024-05-27T07:49:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.228+0000 7fda1de00640 1 -- 172.21.15.90:0/1189142960 learned_addr learned my addr 172.21.15.90:0/1189142960 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.228+0000 7fda1e800640 1 --2- 172.21.15.90:0/1189142960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe86091b0 0x562fe89cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.229+0000 7fda1e800640 1 -- 172.21.15.90:0/1189142960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 msgr2=0x562fe89b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.229+0000 7fda1e800640 1 --2- 172.21.15.90:0/1189142960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 0x562fe89b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.229+0000 7fda1e800640 1 -- 172.21.15.90:0/1189142960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e84d0 msgr2=0x562fe89cd390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:49:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.231+0000 7fda1e800640 1 --2- 172.21.15.90:0/1189142960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e84d0 0x562fe89cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.231+0000 7fda1e800640 1 -- 172.21.15.90:0/1189142960 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe89a8c00 con 0x562fe86091b0 2024-05-27T07:49:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.231+0000 7fda1e800640 1 --2- 172.21.15.90:0/1189142960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe86091b0 0x562fe89cd8d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda18006010 tx=0x7fda18040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29d381c8d28dcba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.232+0000 7fda0ea00640 1 -- 172.21.15.90:0/1189142960 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18031ac0 con 0x562fe86091b0 2024-05-27T07:49:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.232+0000 7fda0ea00640 1 -- 172.21.15.90:0/1189142960 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda18036040 con 0x562fe86091b0 2024-05-27T07:49:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.232+0000 7fda0ea00640 1 -- 172.21.15.90:0/1189142960 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18030820 con 0x562fe86091b0 2024-05-27T07:49:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.233+0000 7fda2c018740 1 -- 172.21.15.90:0/1189142960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe86091b0 msgr2=0x562fe89cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.233+0000 7fda2c018740 1 --2- 172.21.15.90:0/1189142960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe86091b0 0x562fe89cd8d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fda18006010 tx=0x7fda18040000 comp rx=0 tx=0).stop 2024-05-27T07:49:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.235+0000 7fda2c018740 1 -- 172.21.15.90:0/1189142960 shutdown_connections 2024-05-27T07:49:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.235+0000 7fda2c018740 1 --2- 172.21.15.90:0/1189142960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe86091b0 0x562fe89cd8d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.235+0000 7fda2c018740 1 --2- 172.21.15.90:0/1189142960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe85e84d0 0x562fe89cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.235+0000 7fda2c018740 1 --2- 172.21.15.90:0/1189142960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 0x562fe89b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.235+0000 7fda2c018740 1 -- 172.21.15.90:0/1189142960 >> 172.21.15.90:0/1189142960 conn(0x562fe85fd300 msgr2=0x562fe89b73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.235+0000 7fda2c018740 1 -- 172.21.15.90:0/1189142960 shutdown_connections 2024-05-27T07:49:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.235+0000 7fda2c018740 1 -- 172.21.15.90:0/1189142960 wait complete. 2024-05-27T07:49:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.236+0000 7fda2c018740 1 -- start start 2024-05-27T07:49:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.236+0000 7fda2c018740 1 Processor -- start 2024-05-27T07:49:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda2c018740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 0x562fe89bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda2c018740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89bb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda2c018740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe86091b0 0x562fe89bb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda2c018740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fda201b3930 con 0x562fe86091b0 2024-05-27T07:49:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda2c018740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda201b3260 con 0x562fe85e84d0 2024-05-27T07:49:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda2c018740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fda201b4000 con 0x562fe85e02f0 2024-05-27T07:49:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda2c018740 10 client.0 _ll_register_callbacks cb 0x7fda0800bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda1e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe86091b0 0x562fe89bb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda1e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe86091b0 0x562fe89bb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58474/0 (socket says 172.21.15.90:58474) 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda1e800640 1 -- 172.21.15.90:0/4065451929 learned_addr learned my addr 172.21.15.90:0/4065451929 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda1de00640 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 0x562fe89bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.237+0000 7fda1d400640 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89bb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.238+0000 7fda1e800640 1 -- 172.21.15.90:0/4065451929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 msgr2=0x562fe89bacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.238+0000 7fda1e800640 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 0x562fe89bacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.238+0000 7fda1e800640 1 -- 172.21.15.90:0/4065451929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 msgr2=0x562fe89bb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.238+0000 7fda1e800640 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89bb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.238+0000 7fda1e800640 1 -- 172.21.15.90:0/4065451929 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe8a2e2e0 con 0x562fe86091b0 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.238+0000 7fda1e800640 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe86091b0 0x562fe89bb740 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fda18031850 tx=0x7fda18031880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a47e32a2230b3f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.238+0000 7fda03e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.239+0000 7fda03e00640 1 -- 172.21.15.90:0/4065451929 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18004360 con 0x562fe86091b0 2024-05-27T07:49:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.239+0000 7fda03e00640 1 -- 172.21.15.90:0/4065451929 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fda180305c0 con 0x562fe86091b0 2024-05-27T07:49:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.239+0000 7fda2c018740 1 -- 172.21.15.90:0/4065451929 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda18036040 con 0x562fe86091b0 2024-05-27T07:49:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.239+0000 7fda03e00640 1 -- 172.21.15.90:0/4065451929 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fda18004500 con 0x562fe86091b0 2024-05-27T07:49:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.239+0000 7fda2c018740 10 client.16248 fetch_fsmap learned FSMap version 40 2024-05-27T07:49:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.239+0000 7fda2c018740 1 -- 172.21.15.90:0/4065451929 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562fe8a2f550 con 0x562fe86091b0 2024-05-27T07:49:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.239+0000 7fda03e00640 1 -- 172.21.15.90:0/4065451929 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda18036040 con 0x562fe86091b0 2024-05-27T07:49:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.240+0000 7fda03e00640 1 -- 172.21.15.90:0/4065451929 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7fda18044040 con 0x562fe86091b0 2024-05-27T07:49:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.240+0000 7fda2c018740 10 client.16248 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:49:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.240+0000 7fda2c018740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.240+0000 7fda2c018740 20 client.16248 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.240+0000 7fda2c018740 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562fe859d1c0 0x562fe8a31ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.240+0000 7fda2c018740 4 client.16248 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:49:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.240+0000 7fda2c018740 1 -- 172.21.15.90:0/4065451929 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fe89b0260 con 0x562fe859d1c0 2024-05-27T07:49:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.241+0000 7fda1de00640 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562fe859d1c0 0x562fe8a31ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.242+0000 7fda1de00640 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562fe859d1c0 0x562fe8a31ca0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d05fb81036decada server_cookie=79244edf18dcadf7 in_seq=0 out_seq=0 2024-05-27T07:49:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.242+0000 7fda03e00640 10 client.16248 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.309+0000 7fda03e00640 1 -- 172.21.15.90:0/4065451929 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562fe89b0260 con 0x562fe859d1c0 2024-05-27T07:49:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.309+0000 7fda03e00640 10 client.16248 handle_command_reply: tid=0 2024-05-27T07:49:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.309+0000 7fda2c018740 1 client.16248 shutdown 2024-05-27T07:49:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 1 -- 172.21.15.90:0/4065451929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe86091b0 msgr2=0x562fe89bb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe86091b0 0x562fe89bb740 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fda18031850 tx=0x7fda18031880 comp rx=0 tx=0).stop 2024-05-27T07:49:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 1 -- 172.21.15.90:0/4065451929 shutdown_connections 2024-05-27T07:49:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562fe859d1c0 0x562fe8a31ca0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562fe85e84d0 0x562fe89bb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562fe86091b0 0x562fe89bb740 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 1 --2- 172.21.15.90:0/4065451929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fe85e02f0 0x562fe89bacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 1 -- 172.21.15.90:0/4065451929 >> 172.21.15.90:0/4065451929 conn(0x562fe85fd300 msgr2=0x562fe89b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda03e00640 0 client.16248 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 1 -- 172.21.15.90:0/4065451929 shutdown_connections 2024-05-27T07:49:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 1 -- 172.21.15.90:0/4065451929 wait complete. 2024-05-27T07:49:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.310+0000 7fda2c018740 20 client.16248 trim_cache size 0 max 16384 2024-05-27T07:49:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:49:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:49:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:49:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 -- 172.21.15.90:0/3431086047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fda040b2950 msgr2=0x7fda040b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fda040b2950 0x7fda040b4e60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fda18005cc0 tx=0x7fda18005c30 comp rx=0 tx=0).stop 2024-05-27T07:49:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 -- 172.21.15.90:0/3431086047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda20109ab0 msgr2=0x7fda2019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda20109ab0 0x7fda2019cd20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fda1000d740 tx=0x7fda1000dc00 comp rx=0 tx=0).stop 2024-05-27T07:49:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 -- 172.21.15.90:0/3431086047 shutdown_connections 2024-05-27T07:49:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fda040b2950 0x7fda040b4e60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda20109f50 0x7fda201a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda20109ab0 0x7fda2019cd20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 --2- 172.21.15.90:0/3431086047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda20104250 0x7fda2019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 -- 172.21.15.90:0/3431086047 >> 172.21.15.90:0/3431086047 conn(0x7fda20079240 msgr2=0x7fda20076830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.311+0000 7fda1f200640 1 -- 172.21.15.90:0/3431086047 shutdown_connections 2024-05-27T07:49:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:43.312+0000 7fda1f200640 1 -- 172.21.15.90:0/3431086047 wait complete. 2024-05-27T07:49:43.735 INFO:tasks.cephfs.filesystem:scrub status for tag:79657e60-e554-4cd7-b486-fc3c86a24473 - {'path': '/', 'tag': '79657e60-e554-4cd7-b486-fc3c86a24473', 'options': 'recursive,force'} 2024-05-27T07:49:44.440 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:44 smithi090 ceph-mon[31429]: pgmap v106: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 17 MiB/s wr, 119 op/s 2024-05-27T07:49:44.440 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:44 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:49:44.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:44 smithi136 ceph-mon[35324]: pgmap v106: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 17 MiB/s wr, 119 op/s 2024-05-27T07:49:44.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:44 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:49:44.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:44 smithi129 ceph-mon[35254]: pgmap v106: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 17 MiB/s wr, 119 op/s 2024-05-27T07:49:44.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:44 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:49:46.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:46 smithi090 ceph-mon[31429]: pgmap v107: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 101 op/s 2024-05-27T07:49:46.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:46 smithi136 ceph-mon[35324]: pgmap v107: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 101 op/s 2024-05-27T07:49:46.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:46 smithi129 ceph-mon[35254]: pgmap v107: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 101 op/s 2024-05-27T07:49:48.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:48 smithi090 ceph-mon[31429]: pgmap v108: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 101 op/s 2024-05-27T07:49:48.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:48 smithi136 ceph-mon[35324]: pgmap v108: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 101 op/s 2024-05-27T07:49:48.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:48 smithi129 ceph-mon[35254]: pgmap v108: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 12 MiB/s wr, 101 op/s 2024-05-27T07:49:50.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:50 smithi090 ceph-mon[31429]: pgmap v109: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 16 MiB/s wr, 116 op/s 2024-05-27T07:49:50.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:50 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:49:50.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:50 smithi136 ceph-mon[35324]: pgmap v109: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 16 MiB/s wr, 116 op/s 2024-05-27T07:49:50.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:50 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:49:50.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:50 smithi129 ceph-mon[35254]: pgmap v109: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 16 MiB/s wr, 116 op/s 2024-05-27T07:49:50.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:50 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:49:52.165 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:52 smithi136 ceph-mon[35324]: pgmap v110: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 10 MiB/s wr, 107 op/s 2024-05-27T07:49:52.165 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:52 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:49:52.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:52 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:49:52.166 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:52 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:49:52.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:52 smithi090 ceph-mon[31429]: pgmap v110: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 10 MiB/s wr, 107 op/s 2024-05-27T07:49:52.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:52 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:49:52.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:52 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:49:52.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:52 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:49:52.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:52 smithi129 ceph-mon[35254]: pgmap v110: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 10 MiB/s wr, 107 op/s 2024-05-27T07:49:52.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:52 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:49:52.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:52 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:49:52.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:52 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:49:54.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:54 smithi090 ceph-mon[31429]: pgmap v111: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 16 MiB/s wr, 118 op/s 2024-05-27T07:49:54.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:54 smithi136 ceph-mon[35324]: pgmap v111: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 16 MiB/s wr, 118 op/s 2024-05-27T07:49:54.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:54 smithi129 ceph-mon[35254]: pgmap v111: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 16 MiB/s wr, 118 op/s 2024-05-27T07:49:56.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:56 smithi090 ceph-mon[31429]: pgmap v112: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 24 op/s 2024-05-27T07:49:56.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:56 smithi136 ceph-mon[35324]: pgmap v112: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 24 op/s 2024-05-27T07:49:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.551+0000 7f46be000640 1 -- 172.21.15.90:0/3102111382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46b810e1e0 msgr2=0x7f46b81106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.551+0000 7f46be000640 1 --2- 172.21.15.90:0/3102111382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46b810e1e0 0x7f46b81106d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009ca0 tx=0x7f46b002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.551+0000 7f46be000640 1 -- 172.21.15.90:0/3102111382 shutdown_connections 2024-05-27T07:49:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.551+0000 7f46be000640 1 --2- 172.21.15.90:0/3102111382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46b810b7f0 0x7f46b810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.551+0000 7f46be000640 1 --2- 172.21.15.90:0/3102111382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46b810e1e0 0x7f46b81106d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.551+0000 7f46be000640 1 --2- 172.21.15.90:0/3102111382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46b8104d90 0x7f46b81051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.551+0000 7f46be000640 1 -- 172.21.15.90:0/3102111382 >> 172.21.15.90:0/3102111382 conn(0x7f46b80782d0 msgr2=0x7f46b8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.551+0000 7f46be000640 1 -- 172.21.15.90:0/3102111382 shutdown_connections 2024-05-27T07:49:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.551+0000 7f46be000640 1 -- 172.21.15.90:0/3102111382 wait complete. 2024-05-27T07:49:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.553+0000 7f46be000640 1 Processor -- start 2024-05-27T07:49:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.553+0000 7f46be000640 1 -- start start 2024-05-27T07:49:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46be000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46b8104d90 0x7f46b81a2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46be000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46b810b7f0 0x7f46b81a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46be000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46b810e1e0 0x7f46b819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46be000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8113d40 con 0x7f46b810e1e0 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46be000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8113bc0 con 0x7f46b810b7f0 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46be000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8113ec0 con 0x7f46b8104d90 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46b7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46b810e1e0 0x7f46b819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46b7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46b810e1e0 0x7f46b819c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39080/0 (socket says 172.21.15.90:39080) 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46b7e00640 1 -- 172.21.15.90:0/3752422443 learned_addr learned my addr 172.21.15.90:0/3752422443 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46b7400640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46b8104d90 0x7f46b81a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.554+0000 7f46b6a00640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46b810b7f0 0x7f46b81a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.555+0000 7f46b7e00640 1 -- 172.21.15.90:0/3752422443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46b8104d90 msgr2=0x7f46b81a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.555+0000 7f46b7e00640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46b8104d90 0x7f46b81a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.555+0000 7f46b7e00640 1 -- 172.21.15.90:0/3752422443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46b810b7f0 msgr2=0x7f46b81a2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.555+0000 7f46b7e00640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46b810b7f0 0x7f46b81a2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.555+0000 7f46b7e00640 1 -- 172.21.15.90:0/3752422443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b819ce80 con 0x7f46b810e1e0 2024-05-27T07:49:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.555+0000 7f46b7400640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46b8104d90 0x7f46b81a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.555+0000 7f46b7e00640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46b810e1e0 0x7f46b819c7a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f46b0005e30 tx=0x7f46b00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.555+0000 7f46afe00640 1 -- 172.21.15.90:0/3752422443 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0032d10 con 0x7f46b810e1e0 2024-05-27T07:49:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.555+0000 7f46be000640 1 -- 172.21.15.90:0/3752422443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b819d110 con 0x7f46b810e1e0 2024-05-27T07:49:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.556+0000 7f46afe00640 1 -- 172.21.15.90:0/3752422443 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46b0032eb0 con 0x7f46b810e1e0 2024-05-27T07:49:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.556+0000 7f46afe00640 1 -- 172.21.15.90:0/3752422443 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b0030b20 con 0x7f46b810e1e0 2024-05-27T07:49:56.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.556+0000 7f46be000640 1 -- 172.21.15.90:0/3752422443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b81a93c0 con 0x7f46b810e1e0 2024-05-27T07:49:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.560+0000 7f46afe00640 1 -- 172.21.15.90:0/3752422443 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f46b0030ce0 con 0x7f46b810e1e0 2024-05-27T07:49:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.560+0000 7f46c5074740 1 Processor -- start 2024-05-27T07:49:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.561+0000 7f46c5074740 1 -- start start 2024-05-27T07:49:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.561+0000 7f46c5074740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0711b0 0x558d2d413860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.561+0000 7f46c5074740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d413da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.561+0000 7f46c5074740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d4142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.561+0000 7f46c5074740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558d2d384b90 con 0x558d2d0482f0 2024-05-27T07:49:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.561+0000 7f46c5074740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558d2d374730 con 0x558d2d0504d0 2024-05-27T07:49:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.561+0000 7f46c5074740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558d2d378770 con 0x558d2d0711b0 2024-05-27T07:49:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.562+0000 7f46afe00640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f469c0b29d0 0x7f469c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.562+0000 7f46b7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d413da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.562+0000 7f46b7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d413da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39096/0 (socket says 172.21.15.90:39096) 2024-05-27T07:49:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.562+0000 7f46b7400640 1 -- 172.21.15.90:0/3054463002 learned_addr learned my addr 172.21.15.90:0/3054463002 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.562+0000 7f46b6a00640 1 --2- 172.21.15.90:0/3054463002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0711b0 0x558d2d413860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.562+0000 7f46afe00640 1 -- 172.21.15.90:0/3752422443 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f46b0103d30 con 0x7f46b810e1e0 2024-05-27T07:49:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.562+0000 7f46b7e00640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f469c0b29d0 0x7f469c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.562+0000 7f46b6a00640 1 --2- 172.21.15.90:0/3054463002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d4142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:56 smithi129 ceph-mon[35254]: pgmap v112: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 24 op/s 2024-05-27T07:49:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.563+0000 7f46b7400640 1 -- 172.21.15.90:0/3054463002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0711b0 msgr2=0x558d2d413860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.563+0000 7f46b7400640 1 --2- 172.21.15.90:0/3054463002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0711b0 0x558d2d413860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.563+0000 7f46b7400640 1 -- 172.21.15.90:0/3054463002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 msgr2=0x558d2d4142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.563+0000 7f46b7400640 1 --2- 172.21.15.90:0/3054463002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d4142e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.563+0000 7f46b7400640 1 -- 172.21.15.90:0/3054463002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d2d30d8b0 con 0x558d2d0482f0 2024-05-27T07:49:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.563+0000 7f46b7e00640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f469c0b29d0 0x7f469c0b4ee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f46b00c6df0 tx=0x7f46b0009460 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.563+0000 7f46b7400640 1 --2- 172.21.15.90:0/3054463002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d413da0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009cd0 tx=0x7f46a80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96e6442d73ab3e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:56.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.564+0000 7f46a3e00640 1 -- 172.21.15.90:0/3054463002 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8012450 con 0x558d2d0482f0 2024-05-27T07:49:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.564+0000 7f46a3e00640 1 -- 172.21.15.90:0/3054463002 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46a8012a80 con 0x558d2d0482f0 2024-05-27T07:49:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.564+0000 7f46a3e00640 1 -- 172.21.15.90:0/3054463002 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a800fa80 con 0x558d2d0482f0 2024-05-27T07:49:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.565+0000 7f46c5074740 1 -- 172.21.15.90:0/3054463002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 msgr2=0x558d2d413da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.565+0000 7f46c5074740 1 --2- 172.21.15.90:0/3054463002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d413da0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009cd0 tx=0x7f46a80117b0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.565+0000 7f46c5074740 1 -- 172.21.15.90:0/3054463002 shutdown_connections 2024-05-27T07:49:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.565+0000 7f46c5074740 1 --2- 172.21.15.90:0/3054463002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d4142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.565+0000 7f46c5074740 1 --2- 172.21.15.90:0/3054463002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d413da0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.565+0000 7f46c5074740 1 --2- 172.21.15.90:0/3054463002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0711b0 0x558d2d413860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.565+0000 7f46c5074740 1 -- 172.21.15.90:0/3054463002 >> 172.21.15.90:0/3054463002 conn(0x558d2d065300 msgr2=0x558d2d410010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:56.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.566+0000 7f46c5074740 1 -- 172.21.15.90:0/3054463002 shutdown_connections 2024-05-27T07:49:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.566+0000 7f46c5074740 1 -- 172.21.15.90:0/3054463002 wait complete. 2024-05-27T07:49:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.567+0000 7f46c5074740 1 -- start start 2024-05-27T07:49:56.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.567+0000 7f46c5074740 1 Processor -- start 2024-05-27T07:49:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46c5074740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0482f0 0x558d2d41da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46c5074740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d41df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46c5074740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0711b0 0x558d2d4324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46c5074740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558d2d05a790 con 0x558d2d0711b0 2024-05-27T07:49:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46c5074740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558d2d31cbc0 con 0x558d2d0504d0 2024-05-27T07:49:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46c5074740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558d2d4291a0 con 0x558d2d0482f0 2024-05-27T07:49:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46c5074740 10 client.0 _ll_register_callbacks cb 0x558d2d2fa400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46b6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0711b0 0x558d2d4324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46b6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0711b0 0x558d2d4324d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39108/0 (socket says 172.21.15.90:39108) 2024-05-27T07:49:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46b6a00640 1 -- 172.21.15.90:0/1437382252 learned_addr learned my addr 172.21.15.90:0/1437382252 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46b7400640 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d41df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.568+0000 7f46b6a00640 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0482f0 0x558d2d41da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.569+0000 7f46b6a00640 1 -- 172.21.15.90:0/1437382252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0482f0 msgr2=0x558d2d41da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.569+0000 7f46b6a00640 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0482f0 0x558d2d41da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.569+0000 7f46b6a00640 1 -- 172.21.15.90:0/1437382252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 msgr2=0x558d2d41df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.569+0000 7f46b6a00640 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d41df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.569+0000 7f46b6a00640 1 -- 172.21.15.90:0/1437382252 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d2d089210 con 0x558d2d0711b0 2024-05-27T07:49:56.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.569+0000 7f46b7400640 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d41df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:49:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.569+0000 7f46b6a00640 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0711b0 0x558d2d4324d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005fd0 tx=0x7f46a400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0088c85a263a06f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:56.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.569+0000 7f46a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.569+0000 7f46a0c00640 1 -- 172.21.15.90:0/1437382252 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4014110 con 0x558d2d0711b0 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.570+0000 7f46c5074740 1 -- 172.21.15.90:0/1437382252 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a8012a80 con 0x558d2d0711b0 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.570+0000 7f46a0c00640 1 -- 172.21.15.90:0/1437382252 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f46a4004030 con 0x558d2d0711b0 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.570+0000 7f46a0c00640 1 -- 172.21.15.90:0/1437382252 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f46a4028050 con 0x558d2d0711b0 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.570+0000 7f46c5074740 10 client.16260 fetch_fsmap learned FSMap version 40 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.570+0000 7f46c5074740 1 -- 172.21.15.90:0/1437382252 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558d2d155620 con 0x558d2d0711b0 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.570+0000 7f46a0c00640 1 -- 172.21.15.90:0/1437382252 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46a8012a80 con 0x558d2d0711b0 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.570+0000 7f46a0c00640 1 -- 172.21.15.90:0/1437382252 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7f46a40047a0 con 0x558d2d0711b0 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.571+0000 7f46c5074740 10 client.16260 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.571+0000 7f46c5074740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:56.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.571+0000 7f46c5074740 20 client.16260 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.571+0000 7f46c5074740 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558d2d02fd80 0x558d2d438430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.571+0000 7f46c5074740 4 client.16260 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:49:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.571+0000 7f46c5074740 1 -- 172.21.15.90:0/1437382252 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d2d386ae0 con 0x558d2d02fd80 2024-05-27T07:49:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.571+0000 7f46b7400640 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558d2d02fd80 0x558d2d438430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.572+0000 7f46b7400640 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558d2d02fd80 0x558d2d438430 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db62dab300a1155 server_cookie=a381efc94145e002 in_seq=0 out_seq=0 2024-05-27T07:49:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.572+0000 7f46a0c00640 10 client.16260 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.574+0000 7f46a0c00640 1 -- 172.21.15.90:0/1437382252 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558d2d386ae0 con 0x558d2d02fd80 2024-05-27T07:49:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.574+0000 7f46a0c00640 10 client.16260 handle_command_reply: tid=0 2024-05-27T07:49:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 client.16260 shutdown 2024-05-27T07:49:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 -- 172.21.15.90:0/1437382252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0711b0 msgr2=0x558d2d4324d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0711b0 0x558d2d4324d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005fd0 tx=0x7f46a400fb60 comp rx=0 tx=0).stop 2024-05-27T07:49:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 -- 172.21.15.90:0/1437382252 shutdown_connections 2024-05-27T07:49:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558d2d02fd80 0x558d2d438430 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d41df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0711b0 0x558d2d4324d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 --2- 172.21.15.90:0/1437382252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0482f0 0x558d2d41da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46a0c00640 0 client.16260 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 -- 172.21.15.90:0/1437382252 >> 172.21.15.90:0/1437382252 conn(0x558d2d065300 msgr2=0x558d2d413690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 -- 172.21.15.90:0/1437382252 shutdown_connections 2024-05-27T07:49:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 1 -- 172.21.15.90:0/1437382252 wait complete. 2024-05-27T07:49:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.575+0000 7f46c5074740 20 client.16260 trim_cache size 0 max 16384 2024-05-27T07:49:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.592+0000 7f46c5074740 1 Processor -- start 2024-05-27T07:49:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.592+0000 7f46c5074740 1 -- start start 2024-05-27T07:49:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.592+0000 7f46c5074740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d420150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.592+0000 7f46c5074740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0504d0 0x558d2d432240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.592+0000 7f46c5074740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0711b0 0x558d2d432780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.592+0000 7f46c5074740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b0a40 con 0x558d2d0482f0 2024-05-27T07:49:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.592+0000 7f46c5074740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81ae830 con 0x558d2d0711b0 2024-05-27T07:49:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.592+0000 7f46c5074740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81b0370 con 0x558d2d0504d0 2024-05-27T07:49:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.593+0000 7f46b6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d420150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.593+0000 7f46b6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d420150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39114/0 (socket says 172.21.15.90:39114) 2024-05-27T07:49:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.593+0000 7f46b6a00640 1 -- 172.21.15.90:0/2524689319 learned_addr learned my addr 172.21.15.90:0/2524689319 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.593+0000 7f46b7400640 1 --2- 172.21.15.90:0/2524689319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0504d0 0x558d2d432240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.593+0000 7f46b6a00640 1 -- 172.21.15.90:0/2524689319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0504d0 msgr2=0x558d2d432240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.593+0000 7f46b6a00640 1 --2- 172.21.15.90:0/2524689319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0504d0 0x558d2d432240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.593+0000 7f46b6a00640 1 -- 172.21.15.90:0/2524689319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0711b0 msgr2=0x558d2d432780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.593+0000 7f46b6a00640 1 --2- 172.21.15.90:0/2524689319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0711b0 0x558d2d432780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.593+0000 7f46b6a00640 1 -- 172.21.15.90:0/2524689319 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d2d417de0 con 0x558d2d0482f0 2024-05-27T07:49:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.593+0000 7f46b6a00640 1 --2- 172.21.15.90:0/2524689319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d420150 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f46a4026040 tx=0x7f46a400f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d50106251898078 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.594+0000 7f46a3400640 1 -- 172.21.15.90:0/2524689319 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4014110 con 0x558d2d0482f0 2024-05-27T07:49:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.594+0000 7f46a3400640 1 -- 172.21.15.90:0/2524689319 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46a400f420 con 0x558d2d0482f0 2024-05-27T07:49:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.594+0000 7f46a3400640 1 -- 172.21.15.90:0/2524689319 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4005170 con 0x558d2d0482f0 2024-05-27T07:49:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.595+0000 7f46c5074740 1 -- 172.21.15.90:0/2524689319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 msgr2=0x558d2d420150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.595+0000 7f46c5074740 1 --2- 172.21.15.90:0/2524689319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d420150 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f46a4026040 tx=0x7f46a400f760 comp rx=0 tx=0).stop 2024-05-27T07:49:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.595+0000 7f46c5074740 1 -- 172.21.15.90:0/2524689319 shutdown_connections 2024-05-27T07:49:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.595+0000 7f46c5074740 1 --2- 172.21.15.90:0/2524689319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0711b0 0x558d2d432780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.595+0000 7f46c5074740 1 --2- 172.21.15.90:0/2524689319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0504d0 0x558d2d432240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.595+0000 7f46c5074740 1 --2- 172.21.15.90:0/2524689319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d420150 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.595+0000 7f46c5074740 1 -- 172.21.15.90:0/2524689319 >> 172.21.15.90:0/2524689319 conn(0x558d2d065300 msgr2=0x558d2d418ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.595+0000 7f46c5074740 1 -- 172.21.15.90:0/2524689319 shutdown_connections 2024-05-27T07:49:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.595+0000 7f46c5074740 1 -- 172.21.15.90:0/2524689319 wait complete. 2024-05-27T07:49:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.596+0000 7f46c5074740 1 -- start start 2024-05-27T07:49:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.596+0000 7f46c5074740 1 Processor -- start 2024-05-27T07:49:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.596+0000 7f46c5074740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d42b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.596+0000 7f46c5074740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d42bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.596+0000 7f46c5074740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0711b0 0x558d2d498a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.596+0000 7f46c5074740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81af5d0 con 0x558d2d0482f0 2024-05-27T07:49:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.596+0000 7f46c5074740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81aef00 con 0x558d2d0504d0 2024-05-27T07:49:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.596+0000 7f46c5074740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f46b81afca0 con 0x558d2d0711b0 2024-05-27T07:49:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.597+0000 7f46c5074740 10 client.0 _ll_register_callbacks cb 0x7f46a400f860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:49:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.597+0000 7f46b6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d42b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.597+0000 7f46b6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d42b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39122/0 (socket says 172.21.15.90:39122) 2024-05-27T07:49:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.597+0000 7f46b6a00640 1 -- 172.21.15.90:0/3751395202 learned_addr learned my addr 172.21.15.90:0/3751395202 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:49:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.597+0000 7f46b7400640 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d42bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.597+0000 7f46b6a00640 1 -- 172.21.15.90:0/3751395202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0711b0 msgr2=0x558d2d498a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.597+0000 7f46b6a00640 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0711b0 0x558d2d498a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.597+0000 7f46b6a00640 1 -- 172.21.15.90:0/3751395202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 msgr2=0x558d2d42bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:49:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.597+0000 7f46b6a00640 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d42bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.597+0000 7f46b6a00640 1 -- 172.21.15.90:0/3751395202 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d2d498fc0 con 0x558d2d0482f0 2024-05-27T07:49:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.598+0000 7f46b6a00640 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d42b9b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f46a4015b70 tx=0x7f46a40143a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd8c08ecb319913 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:49:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.598+0000 7f46a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:49:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.598+0000 7f46a0c00640 1 -- 172.21.15.90:0/3751395202 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4004ef0 con 0x558d2d0482f0 2024-05-27T07:49:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.598+0000 7f46a0c00640 1 -- 172.21.15.90:0/3751395202 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f46a4005090 con 0x558d2d0482f0 2024-05-27T07:49:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.598+0000 7f46c5074740 1 -- 172.21.15.90:0/3751395202 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a400f420 con 0x558d2d0482f0 2024-05-27T07:49:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.598+0000 7f46a0c00640 1 -- 172.21.15.90:0/3751395202 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f46a403e050 con 0x558d2d0482f0 2024-05-27T07:49:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.598+0000 7f46c5074740 10 client.16266 fetch_fsmap learned FSMap version 40 2024-05-27T07:49:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.598+0000 7f46c5074740 1 -- 172.21.15.90:0/3751395202 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558d2d4a02a0 con 0x558d2d0482f0 2024-05-27T07:49:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.599+0000 7f46a0c00640 1 -- 172.21.15.90:0/3751395202 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46a400f420 con 0x558d2d0482f0 2024-05-27T07:49:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.599+0000 7f46a0c00640 1 -- 172.21.15.90:0/3751395202 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7f46a4012b30 con 0x558d2d0482f0 2024-05-27T07:49:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.599+0000 7f46c5074740 10 client.16266 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:49:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.599+0000 7f46c5074740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:49:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.599+0000 7f46c5074740 20 client.16266 populate_metadata read hostname 'smithi090' 2024-05-27T07:49:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.599+0000 7f46c5074740 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558d2d0051c0 0x558d2d4a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:49:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.599+0000 7f46c5074740 4 client.16266 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:49:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.599+0000 7f46c5074740 1 -- 172.21.15.90:0/3751395202 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d2d418060 con 0x558d2d0051c0 2024-05-27T07:49:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.600+0000 7f46b7400640 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558d2d0051c0 0x558d2d4a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:49:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.601+0000 7f46a0c00640 10 client.16266 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.601+0000 7f46b7400640 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558d2d0051c0 0x558d2d4a28c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ef72fe1d1088d5 server_cookie=4d583bbe82151680 in_seq=0 out_seq=0 2024-05-27T07:49:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46a0c00640 1 -- 172.21.15.90:0/3751395202 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558d2d418060 con 0x558d2d0051c0 2024-05-27T07:49:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46a0c00640 10 client.16266 handle_command_reply: tid=0 2024-05-27T07:49:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 client.16266 shutdown 2024-05-27T07:49:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 -- 172.21.15.90:0/3751395202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 msgr2=0x558d2d42b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d42b9b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f46a4015b70 tx=0x7f46a40143a0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 -- 172.21.15.90:0/3751395202 shutdown_connections 2024-05-27T07:49:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558d2d0051c0 0x558d2d4a28c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d2d0504d0 0x558d2d42bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558d2d0711b0 0x558d2d498a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 --2- 172.21.15.90:0/3751395202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558d2d0482f0 0x558d2d42b9b0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 -- 172.21.15.90:0/3751395202 >> 172.21.15.90:0/3751395202 conn(0x558d2d065300 msgr2=0x558d2d41cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46a0c00640 0 client.16266 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:49:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 -- 172.21.15.90:0/3751395202 shutdown_connections 2024-05-27T07:49:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.657+0000 7f46c5074740 1 -- 172.21.15.90:0/3751395202 wait complete. 2024-05-27T07:49:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.658+0000 7f46c5074740 20 client.16266 trim_cache size 0 max 16384 2024-05-27T07:49:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:49:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:49:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:49:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.658+0000 7f46be000640 1 -- 172.21.15.90:0/3752422443 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f469c0b29d0 msgr2=0x7f469c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.658+0000 7f46be000640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f469c0b29d0 0x7f469c0b4ee0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f46b00c6df0 tx=0x7f46b0009460 comp rx=0 tx=0).stop 2024-05-27T07:49:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.658+0000 7f46be000640 1 -- 172.21.15.90:0/3752422443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46b810e1e0 msgr2=0x7f46b819c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:49:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.658+0000 7f46be000640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46b810e1e0 0x7f46b819c7a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f46b0005e30 tx=0x7f46b00093d0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.659+0000 7f46be000640 1 -- 172.21.15.90:0/3752422443 shutdown_connections 2024-05-27T07:49:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.659+0000 7f46be000640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f469c0b29d0 0x7f469c0b4ee0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.659+0000 7f46be000640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46b810b7f0 0x7f46b81a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.659+0000 7f46be000640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f46b810e1e0 0x7f46b819c7a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.659+0000 7f46be000640 1 --2- 172.21.15.90:0/3752422443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46b8104d90 0x7f46b81a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:49:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.659+0000 7f46be000640 1 -- 172.21.15.90:0/3752422443 >> 172.21.15.90:0/3752422443 conn(0x7f46b80782d0 msgr2=0x7f46b810a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:49:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.659+0000 7f46be000640 1 -- 172.21.15.90:0/3752422443 shutdown_connections 2024-05-27T07:49:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:49:56.659+0000 7f46be000640 1 -- 172.21.15.90:0/3752422443 wait complete. 2024-05-27T07:49:57.449 INFO:teuthology.orchestra.run.smithi090.stderr:creating primary keys... 2024-05-27T07:49:58.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:49:58 smithi090 ceph-mon[31429]: pgmap v113: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 24 op/s 2024-05-27T07:49:58.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:49:58 smithi136 ceph-mon[35324]: pgmap v113: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 24 op/s 2024-05-27T07:49:58.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:49:58 smithi129 ceph-mon[35254]: pgmap v113: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 24 op/s 2024-05-27T07:50:00.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:00 smithi090 ceph-mon[31429]: pgmap v114: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 40 op/s 2024-05-27T07:50:00.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:00 smithi090 ceph-mon[31429]: overall HEALTH_OK 2024-05-27T07:50:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:00 smithi136 ceph-mon[35324]: pgmap v114: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 40 op/s 2024-05-27T07:50:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:00 smithi136 ceph-mon[35324]: overall HEALTH_OK 2024-05-27T07:50:00.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:00 smithi129 ceph-mon[35254]: pgmap v114: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 40 op/s 2024-05-27T07:50:00.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:00 smithi129 ceph-mon[35254]: overall HEALTH_OK 2024-05-27T07:50:02.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:02 smithi090 ceph-mon[31429]: pgmap v115: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.9 MiB/s wr, 25 op/s 2024-05-27T07:50:02.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:02 smithi136 ceph-mon[35324]: pgmap v115: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.9 MiB/s wr, 25 op/s 2024-05-27T07:50:02.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:02 smithi129 ceph-mon[35254]: pgmap v115: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.9 MiB/s wr, 25 op/s 2024-05-27T07:50:04.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:04 smithi090 ceph-mon[31429]: pgmap v116: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 49 op/s 2024-05-27T07:50:04.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:04 smithi136 ceph-mon[35324]: pgmap v116: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 49 op/s 2024-05-27T07:50:04.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:04 smithi129 ceph-mon[35254]: pgmap v116: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 49 op/s 2024-05-27T07:50:06.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:06 smithi090 ceph-mon[31429]: pgmap v117: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.3 MiB/s wr, 38 op/s 2024-05-27T07:50:06.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:06 smithi136 ceph-mon[35324]: pgmap v117: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.3 MiB/s wr, 38 op/s 2024-05-27T07:50:06.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:06 smithi129 ceph-mon[35254]: pgmap v117: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.3 MiB/s wr, 38 op/s 2024-05-27T07:50:08.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:08 smithi090 ceph-mon[31429]: pgmap v118: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.3 MiB/s wr, 38 op/s 2024-05-27T07:50:08.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:08 smithi136 ceph-mon[35324]: pgmap v118: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.3 MiB/s wr, 38 op/s 2024-05-27T07:50:08.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:08 smithi129 ceph-mon[35254]: pgmap v118: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 9.3 MiB/s wr, 38 op/s 2024-05-27T07:50:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.895+0000 7f976e400640 1 -- 172.21.15.90:0/3464067505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f976810b8b0 msgr2=0x7f976810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.895+0000 7f976e400640 1 --2- 172.21.15.90:0/3464067505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f976810b8b0 0x7f976810dcd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9750009900 tx=0x7f975002a480 comp rx=0 tx=0).stop 2024-05-27T07:50:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.895+0000 7f976e400640 1 -- 172.21.15.90:0/3464067505 shutdown_connections 2024-05-27T07:50:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.895+0000 7f976e400640 1 --2- 172.21.15.90:0/3464067505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f976810e210 0x7f97681106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.895+0000 7f976e400640 1 --2- 172.21.15.90:0/3464067505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f976810b8b0 0x7f976810dcd0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.896+0000 7f976e400640 1 --2- 172.21.15.90:0/3464067505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9768104e50 0x7f9768105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.896+0000 7f976e400640 1 -- 172.21.15.90:0/3464067505 >> 172.21.15.90:0/3464067505 conn(0x7f97680782d0 msgr2=0x7f9768078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.896+0000 7f976e400640 1 -- 172.21.15.90:0/3464067505 shutdown_connections 2024-05-27T07:50:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.896+0000 7f976e400640 1 -- 172.21.15.90:0/3464067505 wait complete. 2024-05-27T07:50:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.897+0000 7f976e400640 1 Processor -- start 2024-05-27T07:50:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.897+0000 7f976e400640 1 -- start start 2024-05-27T07:50:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.898+0000 7f976e400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9768104e50 0x7f9768102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.898+0000 7f976e400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f976810b8b0 0x7f9768102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.898+0000 7f976e400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f976810e210 0x7f9768103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.898+0000 7f976e400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9768113d00 con 0x7f9768104e50 2024-05-27T07:50:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.898+0000 7f976e400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9768113b80 con 0x7f976810b8b0 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.898+0000 7f976e400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9768113e80 con 0x7f976810e210 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.898+0000 7f9767400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9768104e50 0x7f9768102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.898+0000 7f9767400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9768104e50 0x7f9768102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37816/0 (socket says 172.21.15.90:37816) 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.898+0000 7f9767400640 1 -- 172.21.15.90:0/281717049 learned_addr learned my addr 172.21.15.90:0/281717049 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.898+0000 7f9766a00640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f976810b8b0 0x7f9768102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.899+0000 7f9767e00640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f976810e210 0x7f9768103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.899+0000 7f9767400640 1 -- 172.21.15.90:0/281717049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f976810e210 msgr2=0x7f9768103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.899+0000 7f9767400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f976810e210 0x7f9768103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.899+0000 7f9767400640 1 -- 172.21.15.90:0/281717049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f976810b8b0 msgr2=0x7f9768102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.899+0000 7f9767400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f976810b8b0 0x7f9768102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.899+0000 7f9767400640 1 -- 172.21.15.90:0/281717049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97681a8a50 con 0x7f9768104e50 2024-05-27T07:50:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.899+0000 7f9767400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9768104e50 0x7f9768102710 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f975800cce0 tx=0x7f975800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.900+0000 7f975fe00640 1 -- 172.21.15.90:0/281717049 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975800f650 con 0x7f9768104e50 2024-05-27T07:50:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.900+0000 7f976e400640 1 -- 172.21.15.90:0/281717049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97681a8d40 con 0x7f9768104e50 2024-05-27T07:50:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.900+0000 7f975fe00640 1 -- 172.21.15.90:0/281717049 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f975800fc80 con 0x7f9768104e50 2024-05-27T07:50:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.900+0000 7f975fe00640 1 -- 172.21.15.90:0/281717049 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758018ae0 con 0x7f9768104e50 2024-05-27T07:50:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.900+0000 7f976e400640 1 -- 172.21.15.90:0/281717049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97681a91a0 con 0x7f9768104e50 2024-05-27T07:50:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.904+0000 7f977539f740 1 Processor -- start 2024-05-27T07:50:09.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.904+0000 7f977539f740 1 -- start start 2024-05-27T07:50:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.904+0000 7f977539f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55bd1b0 0x5629c595f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.904+0000 7f977539f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c55942f0 0x5629c595fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.904+0000 7f977539f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c559c4d0 0x5629c59602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.904+0000 7f977539f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629c58d0b90 con 0x5629c55bd1b0 2024-05-27T07:50:09.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.904+0000 7f977539f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5629c58c0730 con 0x5629c55942f0 2024-05-27T07:50:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.904+0000 7f977539f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5629c58c4770 con 0x5629c559c4d0 2024-05-27T07:50:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.905+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55bd1b0 0x5629c595f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.905+0000 7f975fe00640 1 -- 172.21.15.90:0/281717049 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9758018c80 con 0x7f9768104e50 2024-05-27T07:50:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.905+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55bd1b0 0x5629c595f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37828/0 (socket says 172.21.15.90:37828) 2024-05-27T07:50:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.905+0000 7f9767e00640 1 -- 172.21.15.90:0/1494747053 learned_addr learned my addr 172.21.15.90:0/1494747053 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.905+0000 7f9766a00640 1 --2- 172.21.15.90:0/1494747053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c559c4d0 0x5629c59602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.905+0000 7f9767e00640 1 -- 172.21.15.90:0/1494747053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c559c4d0 msgr2=0x5629c59602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.905+0000 7f9767e00640 1 --2- 172.21.15.90:0/1494747053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c559c4d0 0x5629c59602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.905+0000 7f9767e00640 1 -- 172.21.15.90:0/1494747053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c55942f0 msgr2=0x5629c595fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.905+0000 7f9767e00640 1 --2- 172.21.15.90:0/1494747053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c55942f0 0x5629c595fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.905+0000 7f9767e00640 1 -- 172.21.15.90:0/1494747053 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c58598b0 con 0x5629c55bd1b0 2024-05-27T07:50:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.906+0000 7f9767e00640 1 --2- 172.21.15.90:0/1494747053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55bd1b0 0x5629c595f860 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f976000bba0 tx=0x7f976000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538f5bfc838428cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.906+0000 7f975cc00640 1 -- 172.21.15.90:0/1494747053 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760015070 con 0x5629c55bd1b0 2024-05-27T07:50:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.906+0000 7f975cc00640 1 -- 172.21.15.90:0/1494747053 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97600054b0 con 0x5629c55bd1b0 2024-05-27T07:50:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.906+0000 7f975cc00640 1 -- 172.21.15.90:0/1494747053 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760011440 con 0x5629c55bd1b0 2024-05-27T07:50:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.907+0000 7f975fe00640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f974c0b2950 0x7f974c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.907+0000 7f977539f740 1 -- 172.21.15.90:0/1494747053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55bd1b0 msgr2=0x5629c595f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.907+0000 7f977539f740 1 --2- 172.21.15.90:0/1494747053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55bd1b0 0x5629c595f860 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f976000bba0 tx=0x7f976000f730 comp rx=0 tx=0).stop 2024-05-27T07:50:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.907+0000 7f977539f740 1 -- 172.21.15.90:0/1494747053 shutdown_connections 2024-05-27T07:50:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.907+0000 7f977539f740 1 --2- 172.21.15.90:0/1494747053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c55942f0 0x5629c595fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.907+0000 7f977539f740 1 --2- 172.21.15.90:0/1494747053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c559c4d0 0x5629c59602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.907+0000 7f977539f740 1 --2- 172.21.15.90:0/1494747053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55bd1b0 0x5629c595f860 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.907+0000 7f977539f740 1 -- 172.21.15.90:0/1494747053 >> 172.21.15.90:0/1494747053 conn(0x5629c55b1300 msgr2=0x5629c595c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.908+0000 7f9767400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f974c0b2950 0x7f974c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.908+0000 7f977539f740 1 -- 172.21.15.90:0/1494747053 shutdown_connections 2024-05-27T07:50:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.908+0000 7f975fe00640 1 -- 172.21.15.90:0/281717049 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f97580e2f10 con 0x7f9768104e50 2024-05-27T07:50:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.908+0000 7f977539f740 1 -- 172.21.15.90:0/1494747053 wait complete. 2024-05-27T07:50:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.908+0000 7f9767400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f974c0b2950 0x7f974c0b4e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x5629c595d350 tx=0x7f975800ce90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.908+0000 7f977539f740 1 -- start start 2024-05-27T07:50:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.909+0000 7f977539f740 1 Processor -- start 2024-05-27T07:50:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.909+0000 7f977539f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c59699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.909+0000 7f977539f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c559c4d0 0x5629c5969f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.909+0000 7f977539f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c55bd1b0 0x5629c5975cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.909+0000 7f977539f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629c55a6790 con 0x5629c55942f0 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.909+0000 7f977539f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5629c5868bc0 con 0x5629c55bd1b0 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.909+0000 7f977539f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5629c596c990 con 0x5629c559c4d0 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.909+0000 7f977539f740 10 client.0 _ll_register_callbacks cb 0x5629c5846400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.910+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c59699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.910+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c59699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37830/0 (socket says 172.21.15.90:37830) 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.910+0000 7f9767e00640 1 -- 172.21.15.90:0/2155358263 learned_addr learned my addr 172.21.15.90:0/2155358263 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.910+0000 7f9766a00640 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c559c4d0 0x5629c5969f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.910+0000 7f9767e00640 1 -- 172.21.15.90:0/2155358263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c559c4d0 msgr2=0x5629c5969f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.910+0000 7f9767e00640 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c559c4d0 0x5629c5969f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.910+0000 7f9767e00640 1 -- 172.21.15.90:0/2155358263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c55bd1b0 msgr2=0x5629c5975cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.910+0000 7f9767e00640 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c55bd1b0 0x5629c5975cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.910+0000 7f9767e00640 1 -- 172.21.15.90:0/2155358263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c55d5210 con 0x5629c55942f0 2024-05-27T07:50:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.911+0000 7f9767e00640 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c59699e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9760010010 tx=0x7f976000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74db8fcfa2108097 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.911+0000 7f9755600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.911+0000 7f9755600640 1 -- 172.21.15.90:0/2155358263 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760015070 con 0x5629c55942f0 2024-05-27T07:50:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.911+0000 7f9755600640 1 -- 172.21.15.90:0/2155358263 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f9760004ea0 con 0x5629c55942f0 2024-05-27T07:50:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.911+0000 7f977539f740 1 -- 172.21.15.90:0/2155358263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97600054b0 con 0x5629c55942f0 2024-05-27T07:50:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.911+0000 7f9755600640 1 -- 172.21.15.90:0/2155358263 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f97600050a0 con 0x5629c55942f0 2024-05-27T07:50:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.911+0000 7f977539f740 10 client.16278 fetch_fsmap learned FSMap version 40 2024-05-27T07:50:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.911+0000 7f977539f740 1 -- 172.21.15.90:0/2155358263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5629c56a1620 con 0x5629c55942f0 2024-05-27T07:50:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.912+0000 7f9755600640 1 -- 172.21.15.90:0/2155358263 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97600054b0 con 0x5629c55942f0 2024-05-27T07:50:09.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.912+0000 7f9755600640 1 -- 172.21.15.90:0/2155358263 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7f9760026cc0 con 0x5629c55942f0 2024-05-27T07:50:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.912+0000 7f977539f740 10 client.16278 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:50:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.912+0000 7f977539f740 10 client.16278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.912+0000 7f977539f740 20 client.16278 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.912+0000 7f977539f740 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5629c557bd80 0x5629c597bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.912+0000 7f977539f740 4 client.16278 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.912+0000 7f977539f740 1 -- 172.21.15.90:0/2155358263 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629c58d2ae0 con 0x5629c557bd80 2024-05-27T07:50:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.913+0000 7f9766a00640 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5629c557bd80 0x5629c597bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.914+0000 7f9755600640 10 client.16278 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.914+0000 7f9766a00640 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5629c557bd80 0x5629c597bb30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af040c4ba978f109 server_cookie=78fca590c1c2d343 in_seq=0 out_seq=0 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f9755600640 1 -- 172.21.15.90:0/2155358263 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5629c58d2ae0 con 0x5629c557bd80 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f9755600640 10 client.16278 handle_command_reply: tid=0 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 client.16278 shutdown 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 -- 172.21.15.90:0/2155358263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 msgr2=0x5629c59699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c59699e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9760010010 tx=0x7f976000bec0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 -- 172.21.15.90:0/2155358263 shutdown_connections 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5629c557bd80 0x5629c597bb30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c55bd1b0 0x5629c5975cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c559c4d0 0x5629c5969f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 --2- 172.21.15.90:0/2155358263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c59699e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 -- 172.21.15.90:0/2155358263 >> 172.21.15.90:0/2155358263 conn(0x5629c55b1300 msgr2=0x5629c595f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f9755600640 0 client.16278 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 -- 172.21.15.90:0/2155358263 shutdown_connections 2024-05-27T07:50:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.916+0000 7f977539f740 1 -- 172.21.15.90:0/2155358263 wait complete. 2024-05-27T07:50:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.917+0000 7f977539f740 20 client.16278 trim_cache size 0 max 16384 2024-05-27T07:50:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.935+0000 7f977539f740 1 Processor -- start 2024-05-27T07:50:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.935+0000 7f977539f740 1 -- start start 2024-05-27T07:50:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.935+0000 7f977539f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c5965230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.935+0000 7f977539f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c559c4d0 0x5629c5975a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.935+0000 7f977539f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c55bd1b0 0x5629c5975f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.935+0000 7f977539f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f97681b4d80 con 0x5629c55942f0 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.935+0000 7f977539f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97681b2b70 con 0x5629c559c4d0 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.935+0000 7f977539f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f97681b46b0 con 0x5629c55bd1b0 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.936+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c5965230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.936+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c5965230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37838/0 (socket says 172.21.15.90:37838) 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.936+0000 7f9767e00640 1 -- 172.21.15.90:0/914317648 learned_addr learned my addr 172.21.15.90:0/914317648 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.936+0000 7f9766a00640 1 --2- 172.21.15.90:0/914317648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c559c4d0 0x5629c5975a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.936+0000 7f9767e00640 1 -- 172.21.15.90:0/914317648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c55bd1b0 msgr2=0x5629c5975f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.936+0000 7f9767e00640 1 --2- 172.21.15.90:0/914317648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c55bd1b0 0x5629c5975f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.936+0000 7f9767e00640 1 -- 172.21.15.90:0/914317648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c559c4d0 msgr2=0x5629c5975a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.936+0000 7f9767e00640 1 --2- 172.21.15.90:0/914317648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c559c4d0 0x5629c5975a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.936+0000 7f9767e00640 1 -- 172.21.15.90:0/914317648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629c5955fc0 con 0x5629c55942f0 2024-05-27T07:50:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.936+0000 7f9767e00640 1 --2- 172.21.15.90:0/914317648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c5965230 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9760001eb0 tx=0x7f9760005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74837989f5347fa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.937+0000 7f9757e00640 1 -- 172.21.15.90:0/914317648 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760038070 con 0x5629c55942f0 2024-05-27T07:50:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.937+0000 7f9757e00640 1 -- 172.21.15.90:0/914317648 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9760021070 con 0x5629c55942f0 2024-05-27T07:50:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.937+0000 7f9757e00640 1 -- 172.21.15.90:0/914317648 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760015070 con 0x5629c55942f0 2024-05-27T07:50:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.938+0000 7f977539f740 1 -- 172.21.15.90:0/914317648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 msgr2=0x5629c5965230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.938+0000 7f977539f740 1 --2- 172.21.15.90:0/914317648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c5965230 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9760001eb0 tx=0x7f9760005c70 comp rx=0 tx=0).stop 2024-05-27T07:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.938+0000 7f977539f740 1 -- 172.21.15.90:0/914317648 shutdown_connections 2024-05-27T07:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.938+0000 7f977539f740 1 --2- 172.21.15.90:0/914317648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c559c4d0 0x5629c5975a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.938+0000 7f977539f740 1 --2- 172.21.15.90:0/914317648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c55bd1b0 0x5629c5975f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.938+0000 7f977539f740 1 --2- 172.21.15.90:0/914317648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c5965230 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.938+0000 7f977539f740 1 -- 172.21.15.90:0/914317648 >> 172.21.15.90:0/914317648 conn(0x5629c55b1300 msgr2=0x5629c5969ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.938+0000 7f977539f740 1 -- 172.21.15.90:0/914317648 shutdown_connections 2024-05-27T07:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.938+0000 7f977539f740 1 -- 172.21.15.90:0/914317648 wait complete. 2024-05-27T07:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.939+0000 7f977539f740 1 -- start start 2024-05-27T07:50:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.939+0000 7f977539f740 1 Processor -- start 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.939+0000 7f977539f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c596e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.939+0000 7f977539f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c559c4d0 0x5629c596edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f977539f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c55bd1b0 0x5629c596f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f977539f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f97681b3910 con 0x5629c55942f0 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f977539f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97681b3240 con 0x5629c559c4d0 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f977539f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f97681b3fe0 con 0x5629c55bd1b0 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f977539f740 10 client.0 _ll_register_callbacks cb 0x7f976000b9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c596e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c596e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37852/0 (socket says 172.21.15.90:37852) 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f9767e00640 1 -- 172.21.15.90:0/2636290377 learned_addr learned my addr 172.21.15.90:0/2636290377 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f9766a00640 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c559c4d0 0x5629c596edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f9767e00640 1 -- 172.21.15.90:0/2636290377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c55bd1b0 msgr2=0x5629c596f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f9767e00640 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c55bd1b0 0x5629c596f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f9767e00640 1 -- 172.21.15.90:0/2636290377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c559c4d0 msgr2=0x5629c596edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f9767e00640 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c559c4d0 0x5629c596edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.940+0000 7f9767e00640 1 -- 172.21.15.90:0/2636290377 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629c59e1fd0 con 0x5629c55942f0 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.941+0000 7f9767e00640 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c596e8b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f976000b660 tx=0x7f9760011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67fdbda2e7a2ba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.941+0000 7f9755600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.941+0000 7f9755600640 1 -- 172.21.15.90:0/2636290377 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760038020 con 0x5629c55942f0 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.941+0000 7f9755600640 1 -- 172.21.15.90:0/2636290377 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f9760005440 con 0x5629c55942f0 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.941+0000 7f977539f740 1 -- 172.21.15.90:0/2636290377 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9760021070 con 0x5629c55942f0 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.941+0000 7f9755600640 1 -- 172.21.15.90:0/2636290377 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9760043050 con 0x5629c55942f0 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.941+0000 7f977539f740 10 client.16284 fetch_fsmap learned FSMap version 40 2024-05-27T07:50:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.941+0000 7f977539f740 1 -- 172.21.15.90:0/2636290377 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5629c59e30e0 con 0x5629c55942f0 2024-05-27T07:50:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.942+0000 7f9755600640 1 -- 172.21.15.90:0/2636290377 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9760021070 con 0x5629c55942f0 2024-05-27T07:50:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.942+0000 7f9755600640 1 -- 172.21.15.90:0/2636290377 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7f9760026900 con 0x5629c55942f0 2024-05-27T07:50:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.942+0000 7f977539f740 10 client.16284 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:50:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.942+0000 7f977539f740 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.942+0000 7f977539f740 20 client.16284 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.942+0000 7f977539f740 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5629c55511c0 0x5629c59e5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.942+0000 7f977539f740 4 client.16284 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:50:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.942+0000 7f977539f740 1 -- 172.21.15.90:0/2636290377 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629c5960130 con 0x5629c55511c0 2024-05-27T07:50:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.943+0000 7f9766a00640 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5629c55511c0 0x5629c59e5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.944+0000 7f9766a00640 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5629c55511c0 0x5629c59e5780 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c686be8e5726f9 server_cookie=55430cc974ee2ccd in_seq=0 out_seq=0 2024-05-27T07:50:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.944+0000 7f9755600640 10 client.16284 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f9755600640 1 -- 172.21.15.90:0/2636290377 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5629c5960130 con 0x5629c55511c0 2024-05-27T07:50:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f9755600640 10 client.16284 handle_command_reply: tid=0 2024-05-27T07:50:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 client.16284 shutdown 2024-05-27T07:50:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 -- 172.21.15.90:0/2636290377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 msgr2=0x5629c596e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c596e8b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f976000b660 tx=0x7f9760011bb0 comp rx=0 tx=0).stop 2024-05-27T07:50:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 -- 172.21.15.90:0/2636290377 shutdown_connections 2024-05-27T07:50:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5629c55511c0 0x5629c59e5780 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629c559c4d0 0x5629c596edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629c55bd1b0 0x5629c596f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 --2- 172.21.15.90:0/2636290377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629c55942f0 0x5629c596e8b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 -- 172.21.15.90:0/2636290377 >> 172.21.15.90:0/2636290377 conn(0x5629c55b1300 msgr2=0x5629c59699d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f9755600640 0 client.16284 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 -- 172.21.15.90:0/2636290377 shutdown_connections 2024-05-27T07:50:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 1 -- 172.21.15.90:0/2636290377 wait complete. 2024-05-27T07:50:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.998+0000 7f977539f740 20 client.16284 trim_cache size 0 max 16384 2024-05-27T07:50:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:50:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:50:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:50:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.999+0000 7f976e400640 1 -- 172.21.15.90:0/281717049 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f974c0b2950 msgr2=0x7f974c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.999+0000 7f976e400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f974c0b2950 0x7f974c0b4e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x5629c595d350 tx=0x7f975800ce90 comp rx=0 tx=0).stop 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.999+0000 7f976e400640 1 -- 172.21.15.90:0/281717049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9768104e50 msgr2=0x7f9768102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.999+0000 7f976e400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9768104e50 0x7f9768102710 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f975800cce0 tx=0x7f975800a620 comp rx=0 tx=0).stop 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.999+0000 7f976e400640 1 -- 172.21.15.90:0/281717049 shutdown_connections 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.999+0000 7f976e400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f974c0b2950 0x7f974c0b4e60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.999+0000 7f976e400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f976810b8b0 0x7f9768102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.999+0000 7f976e400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f976810e210 0x7f9768103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.999+0000 7f976e400640 1 --2- 172.21.15.90:0/281717049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9768104e50 0x7f9768102710 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:09.999+0000 7f976e400640 1 -- 172.21.15.90:0/281717049 >> 172.21.15.90:0/281717049 conn(0x7f97680782d0 msgr2=0x7f97681012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:10.000+0000 7f976e400640 1 -- 172.21.15.90:0/281717049 shutdown_connections 2024-05-27T07:50:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:10.000+0000 7f976e400640 1 -- 172.21.15.90:0/281717049 wait complete. 2024-05-27T07:50:10.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:10 smithi090 ceph-mon[31429]: pgmap v119: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 47 op/s 2024-05-27T07:50:10.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:10 smithi136 ceph-mon[35324]: pgmap v119: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 47 op/s 2024-05-27T07:50:10.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:10 smithi129 ceph-mon[35254]: pgmap v119: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 16 MiB/s wr, 47 op/s 2024-05-27T07:50:12.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:12 smithi090 ceph-mon[31429]: pgmap v120: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 31 op/s 2024-05-27T07:50:12.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:12 smithi136 ceph-mon[35324]: pgmap v120: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 31 op/s 2024-05-27T07:50:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:12 smithi129 ceph-mon[35254]: pgmap v120: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 31 op/s 2024-05-27T07:50:13.736 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:50:14.214 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:14.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:14 smithi136 ceph-mon[35324]: pgmap v121: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 17 MiB/s wr, 50 op/s 2024-05-27T07:50:14.513 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:14 smithi090 ceph-mon[31429]: pgmap v121: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 17 MiB/s wr, 50 op/s 2024-05-27T07:50:14.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:14 smithi129 ceph-mon[35254]: pgmap v121: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 17 MiB/s wr, 50 op/s 2024-05-27T07:50:15.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:15 smithi090 ceph-mon[31429]: pgmap v122: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 26 op/s 2024-05-27T07:50:15.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:15 smithi136 ceph-mon[35324]: pgmap v122: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 26 op/s 2024-05-27T07:50:15.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:15 smithi129 ceph-mon[35254]: pgmap v122: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 26 op/s 2024-05-27T07:50:16.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.636+0000 7fbf0b200700 1 -- 172.21.15.90:0/2117420820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0c109670 msgr2=0x7fbf0c110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:16.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.636+0000 7fbf0b200700 1 --2- 172.21.15.90:0/2117420820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0c109670 0x7fbf0c110fd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b0d0 tx=0x7fbf0400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.636+0000 7fbf0b200700 1 -- 172.21.15.90:0/2117420820 shutdown_connections 2024-05-27T07:50:16.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.636+0000 7fbf0b200700 1 --2- 172.21.15.90:0/2117420820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0c109670 0x7fbf0c110fd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.636+0000 7fbf0b200700 1 --2- 172.21.15.90:0/2117420820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf0c1043f0 0x7fbf0c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.636+0000 7fbf0b200700 1 --2- 172.21.15.90:0/2117420820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf0c108d20 0x7fbf0c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.636+0000 7fbf0b200700 1 -- 172.21.15.90:0/2117420820 >> 172.21.15.90:0/2117420820 conn(0x7fbf0c079d00 msgr2=0x7fbf0c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:16.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.636+0000 7fbf0b200700 1 -- 172.21.15.90:0/2117420820 shutdown_connections 2024-05-27T07:50:16.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.636+0000 7fbf0b200700 1 -- 172.21.15.90:0/2117420820 wait complete. 2024-05-27T07:50:16.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.638+0000 7fbf0b200700 1 Processor -- start 2024-05-27T07:50:16.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.638+0000 7fbf0b200700 1 -- start start 2024-05-27T07:50:16.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.638+0000 7fbf0b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0c1043f0 0x7fbf0c1a2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.638+0000 7fbf0b200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf0c108d20 0x7fbf0c1a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.639+0000 7fbf0b200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf0c109670 0x7fbf0c1acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.639+0000 7fbf0b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1141a0 con 0x7fbf0c1043f0 2024-05-27T07:50:16.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.639+0000 7fbf0b200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c114020 con 0x7fbf0c109670 2024-05-27T07:50:16.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.639+0000 7fbf0b200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c114320 con 0x7fbf0c108d20 2024-05-27T07:50:16.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.639+0000 7fbf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0c1043f0 0x7fbf0c1a2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.639+0000 7fbf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0c1043f0 0x7fbf0c1a2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37868/0 (socket says 172.21.15.90:37868) 2024-05-27T07:50:16.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.639+0000 7fbf09e00700 1 -- 172.21.15.90:0/730352642 learned_addr learned my addr 172.21.15.90:0/730352642 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:16.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.639+0000 7fbf09400700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf0c108d20 0x7fbf0c1a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.640+0000 7fbf0a800700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf0c109670 0x7fbf0c1acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.640+0000 7fbf09e00700 1 -- 172.21.15.90:0/730352642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf0c108d20 msgr2=0x7fbf0c1a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:16.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.640+0000 7fbf09e00700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf0c108d20 0x7fbf0c1a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.640+0000 7fbf09e00700 1 -- 172.21.15.90:0/730352642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf0c109670 msgr2=0x7fbf0c1acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:16.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.640+0000 7fbf09e00700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf0c109670 0x7fbf0c1acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.640+0000 7fbf09e00700 1 -- 172.21.15.90:0/730352642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0c1ad440 con 0x7fbf0c1043f0 2024-05-27T07:50:16.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.640+0000 7fbf09e00700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0c1043f0 0x7fbf0c1a2050 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00d950 tx=0x7fbefc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:16.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.641+0000 7fbf02a00700 1 -- 172.21.15.90:0/730352642 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc016070 con 0x7fbf0c1043f0 2024-05-27T07:50:16.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.641+0000 7fbf02a00700 1 -- 172.21.15.90:0/730352642 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbefc009dd0 con 0x7fbf0c1043f0 2024-05-27T07:50:16.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.641+0000 7fbf0b200700 1 -- 172.21.15.90:0/730352642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0c1ad730 con 0x7fbf0c1043f0 2024-05-27T07:50:16.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.641+0000 7fbf02a00700 1 -- 172.21.15.90:0/730352642 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc00f3c0 con 0x7fbf0c1043f0 2024-05-27T07:50:16.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.641+0000 7fbf0b200700 1 -- 172.21.15.90:0/730352642 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0c1adcc0 con 0x7fbf0c1043f0 2024-05-27T07:50:16.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.646+0000 7fbf02a00700 1 -- 172.21.15.90:0/730352642 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbefc019070 con 0x7fbf0c1043f0 2024-05-27T07:50:16.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.647+0000 7fbf1f4c6b80 1 Processor -- start 2024-05-27T07:50:16.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.647+0000 7fbf1f4c6b80 1 -- start start 2024-05-27T07:50:16.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.647+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 0x563b02949750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02949c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b029556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf02a00700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbef00b2a70 0x7fbef00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563b02876e40 con 0x563b027c9e40 2024-05-27T07:50:16.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563b028669e0 con 0x563b029532c0 2024-05-27T07:50:16.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563b0286aa20 con 0x563b028b59f0 2024-05-27T07:50:16.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf02a00700 1 -- 172.21.15.90:0/730352642 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fbefc0e2920 con 0x7fbf0c1043f0 2024-05-27T07:50:16.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf0a800700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbef00b2a70 0x7fbef00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02949c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf0a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 0x563b02949750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf0a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 0x563b02949750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50818/0 (socket says 172.21.15.90:50818) 2024-05-27T07:50:16.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02949c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37884/0 (socket says 172.21.15.90:37884) 2024-05-27T07:50:16.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.648+0000 7fbf0a800700 1 -- 172.21.15.90:0/3082839012 learned_addr learned my addr 172.21.15.90:0/3082839012 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:16.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.649+0000 7fbf09e00700 1 -- 172.21.15.90:0/3082839012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 msgr2=0x563b02949750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:16.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.649+0000 7fbf09e00700 1 --2- 172.21.15.90:0/3082839012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 0x563b02949750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.649+0000 7fbf09e00700 1 -- 172.21.15.90:0/3082839012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 msgr2=0x563b029556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:16.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.649+0000 7fbf09e00700 1 --2- 172.21.15.90:0/3082839012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b029556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.649+0000 7fbf09e00700 1 -- 172.21.15.90:0/3082839012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0282f420 con 0x563b027c9e40 2024-05-27T07:50:16.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.649+0000 7fbf09e00700 1 --2- 172.21.15.90:0/3082839012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02949c90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0dfd00 tx=0x7fbefc0a9310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db8b83b9dae62ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:16.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.650+0000 7fbef7400700 1 -- 172.21.15.90:0/3082839012 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc020020 con 0x563b027c9e40 2024-05-27T07:50:16.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.650+0000 7fbf0a800700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbef00b2a70 0x7fbef00b4f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009900 tx=0x7fbf04009c00 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:16.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.650+0000 7fbef7400700 1 -- 172.21.15.90:0/3082839012 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbefc0a93a0 con 0x563b027c9e40 2024-05-27T07:50:16.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.650+0000 7fbef7400700 1 -- 172.21.15.90:0/3082839012 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0b26f0 con 0x563b027c9e40 2024-05-27T07:50:16.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.651+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3082839012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 msgr2=0x563b02949c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:16.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.651+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3082839012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02949c90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0dfd00 tx=0x7fbefc0a9310 comp rx=0 tx=0).stop 2024-05-27T07:50:16.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.651+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3082839012 shutdown_connections 2024-05-27T07:50:16.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.651+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3082839012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b029556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.651+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3082839012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 0x563b02949750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.651+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3082839012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02949c90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.651+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3082839012 >> 172.21.15.90:0/3082839012 conn(0x563b02863390 msgr2=0x563b02944710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:16.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.652+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3082839012 shutdown_connections 2024-05-27T07:50:16.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.652+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3082839012 wait complete. 2024-05-27T07:50:16.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.653+0000 7fbf1f4c6b80 1 -- start start 2024-05-27T07:50:16.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.653+0000 7fbf1f4c6b80 1 Processor -- start 2024-05-27T07:50:16.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.654+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b027c9e40 0x563b02950260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.654+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b029507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.654+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b02950ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.654+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563b0276cfa0 con 0x563b028b59f0 2024-05-27T07:50:16.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.654+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563b02807760 con 0x563b029532c0 2024-05-27T07:50:16.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.654+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563b02955e00 con 0x563b027c9e40 2024-05-27T07:50:16.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.654+0000 7fbf1f4c6b80 10 client.0 _ll_register_callbacks cb 0x563b02810c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:16.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.654+0000 7fbf09400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b029507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.654+0000 7fbf0a800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b02950ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.654+0000 7fbf09400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b029507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37896/0 (socket says 172.21.15.90:37896) 2024-05-27T07:50:16.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.655+0000 7fbf09400700 1 -- 172.21.15.90:0/3443700639 learned_addr learned my addr 172.21.15.90:0/3443700639 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:16.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.655+0000 7fbf0a800700 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b02950ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39284/0 (socket says 172.21.15.90:39284) 2024-05-27T07:50:16.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.655+0000 7fbf09400700 1 -- 172.21.15.90:0/3443700639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b027c9e40 msgr2=0x563b02950260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:16.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.655+0000 7fbf09e00700 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b027c9e40 0x563b02950260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.655+0000 7fbf09400700 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b027c9e40 0x563b02950260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.655+0000 7fbf09400700 1 -- 172.21.15.90:0/3443700639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 msgr2=0x563b02950ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:16.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.655+0000 7fbf09400700 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b02950ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.655+0000 7fbf09400700 1 -- 172.21.15.90:0/3443700639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b027b68d0 con 0x563b028b59f0 2024-05-27T07:50:16.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.656+0000 7fbf09400700 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b029507a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fbef800be60 tx=0x7fbef80106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b5d29cf158f625 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:16.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.656+0000 7fbeebe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:16.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.656+0000 7fbf09e00700 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b027c9e40 0x563b02950260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:16.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.657+0000 7fbeebe00700 1 -- 172.21.15.90:0/3443700639 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8004230 con 0x563b028b59f0 2024-05-27T07:50:16.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.657+0000 7fbeebe00700 1 -- 172.21.15.90:0/3443700639 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fbef80043d0 con 0x563b028b59f0 2024-05-27T07:50:16.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.657+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3443700639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbefc0a93a0 con 0x563b028b59f0 2024-05-27T07:50:16.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.657+0000 7fbeebe00700 1 -- 172.21.15.90:0/3443700639 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbef8023050 con 0x563b028b59f0 2024-05-27T07:50:16.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.657+0000 7fbf1f4c6b80 10 client.16296 fetch_fsmap learned FSMap version 40 2024-05-27T07:50:16.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.657+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3443700639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563b02837060 con 0x563b028b59f0 2024-05-27T07:50:16.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.657+0000 7fbeebe00700 1 -- 172.21.15.90:0/3443700639 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbefc0a93a0 con 0x563b028b59f0 2024-05-27T07:50:16.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.658+0000 7fbeebe00700 1 -- 172.21.15.90:0/3443700639 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7fbef8011560 con 0x563b028b59f0 2024-05-27T07:50:16.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.658+0000 7fbf1f4c6b80 10 client.16296 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:50:16.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.658+0000 7fbf1f4c6b80 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:16.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.658+0000 7fbf1f4c6b80 20 client.16296 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:16.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.658+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563b02962da0 0x563b02965220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.658+0000 7fbf1f4c6b80 4 client.16296 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:16.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.658+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3443700639 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b02878d90 con 0x563b02962da0 2024-05-27T07:50:16.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.659+0000 7fbf09e00700 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563b02962da0 0x563b02965220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.661+0000 7fbf09e00700 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563b02962da0 0x563b02965220 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f03510ca9a27e5 server_cookie=c853b5113ddd67e3 in_seq=0 out_seq=0 2024-05-27T07:50:16.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.661+0000 7fbeebe00700 10 client.16296 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:16.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbeebe00700 1 -- 172.21.15.90:0/3443700639 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563b02878d90 con 0x563b02962da0 2024-05-27T07:50:16.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbeebe00700 10 client.16296 handle_command_reply: tid=0 2024-05-27T07:50:16.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbf1f4c6b80 1 client.16296 shutdown 2024-05-27T07:50:16.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3443700639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 msgr2=0x563b029507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b029507a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fbef800be60 tx=0x7fbef80106b0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3443700639 shutdown_connections 2024-05-27T07:50:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563b02962da0 0x563b02965220 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbeebe00700 0 client.16296 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b02950ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b027c9e40 0x563b02950260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3443700639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b029507a0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.664+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3443700639 >> 172.21.15.90:0/3443700639 conn(0x563b02863390 msgr2=0x563b02949750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.665+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3443700639 shutdown_connections 2024-05-27T07:50:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.665+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3443700639 wait complete. 2024-05-27T07:50:16.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.665+0000 7fbf1f4c6b80 20 client.16296 trim_cache size 0 max 16384 2024-05-27T07:50:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.695+0000 7fbf1f4c6b80 1 Processor -- start 2024-05-27T07:50:16.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.695+0000 7fbf1f4c6b80 1 -- start start 2024-05-27T07:50:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.696+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02944820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.696+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 0x563b0295eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.696+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b0296a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.696+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b8430 con 0x563b027c9e40 2024-05-27T07:50:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.696+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b6220 con 0x563b029532c0 2024-05-27T07:50:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.696+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b7d60 con 0x563b028b59f0 2024-05-27T07:50:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.696+0000 7fbf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02944820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.696+0000 7fbf09e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02944820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37900/0 (socket says 172.21.15.90:37900) 2024-05-27T07:50:16.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.696+0000 7fbf09e00700 1 -- 172.21.15.90:0/780881241 learned_addr learned my addr 172.21.15.90:0/780881241 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.696+0000 7fbf09400700 1 --2- 172.21.15.90:0/780881241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 0x563b0295eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.697+0000 7fbf0a800700 1 --2- 172.21.15.90:0/780881241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b0296a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.697+0000 7fbf09e00700 1 -- 172.21.15.90:0/780881241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 msgr2=0x563b0295eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.697+0000 7fbf09e00700 1 --2- 172.21.15.90:0/780881241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 0x563b0295eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.697+0000 7fbf09e00700 1 -- 172.21.15.90:0/780881241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 msgr2=0x563b0296a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.697+0000 7fbf09e00700 1 --2- 172.21.15.90:0/780881241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b0296a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.697+0000 7fbf09e00700 1 -- 172.21.15.90:0/780881241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b029657b0 con 0x563b027c9e40 2024-05-27T07:50:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.697+0000 7fbf0a800700 1 --2- 172.21.15.90:0/780881241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b0296a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.697+0000 7fbf09e00700 1 --2- 172.21.15.90:0/780881241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02944820 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0a9820 tx=0x7fbefc010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159623d391bbed6e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.697+0000 7fbef6a00700 1 -- 172.21.15.90:0/780881241 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc020040 con 0x563b027c9e40 2024-05-27T07:50:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.698+0000 7fbef6a00700 1 -- 172.21.15.90:0/780881241 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbefc023350 con 0x563b027c9e40 2024-05-27T07:50:16.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.698+0000 7fbef6a00700 1 -- 172.21.15.90:0/780881241 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0106a0 con 0x563b027c9e40 2024-05-27T07:50:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.699+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/780881241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 msgr2=0x563b02944820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.699+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/780881241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02944820 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0a9820 tx=0x7fbefc010f30 comp rx=0 tx=0).stop 2024-05-27T07:50:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.699+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/780881241 shutdown_connections 2024-05-27T07:50:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.699+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/780881241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b029532c0 0x563b0296a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.699+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/780881241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b027c9e40 0x563b02944820 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.699+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/780881241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b028b59f0 0x563b0295eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.699+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/780881241 >> 172.21.15.90:0/780881241 conn(0x563b02863390 msgr2=0x563b02944110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.699+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/780881241 shutdown_connections 2024-05-27T07:50:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.699+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/780881241 wait complete. 2024-05-27T07:50:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.700+0000 7fbf1f4c6b80 1 -- start start 2024-05-27T07:50:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.700+0000 7fbf1f4c6b80 1 Processor -- start 2024-05-27T07:50:16.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.701+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b027c9e40 0x563b0295b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.701+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b02956b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.701+0000 7fbf1f4c6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b029532c0 0x563b029570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.701+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b6fc0 con 0x563b028b59f0 2024-05-27T07:50:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.701+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b68f0 con 0x563b027c9e40 2024-05-27T07:50:16.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.701+0000 7fbf1f4c6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0c1b7690 con 0x563b029532c0 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.701+0000 7fbf1f4c6b80 10 client.0 _ll_register_callbacks cb 0x563b02810c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.701+0000 7fbf09400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b02956b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.701+0000 7fbf09400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b02956b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37916/0 (socket says 172.21.15.90:37916) 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.701+0000 7fbf09400700 1 -- 172.21.15.90:0/3081738742 learned_addr learned my addr 172.21.15.90:0/3081738742 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbf0a800700 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b029532c0 0x563b029570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbf09400700 1 -- 172.21.15.90:0/3081738742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b029532c0 msgr2=0x563b029570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbf09400700 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b029532c0 0x563b029570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbf09400700 1 -- 172.21.15.90:0/3081738742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b027c9e40 msgr2=0x563b0295b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbf09400700 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b027c9e40 0x563b0295b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbf09400700 1 -- 172.21.15.90:0/3081738742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0294a500 con 0x563b028b59f0 2024-05-27T07:50:16.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbf0a800700 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b029532c0 0x563b029570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:16.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbf09400700 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b02956b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbef800bbb0 tx=0x7fbef8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7386032af85daa2c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:16.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbeebe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:16.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbeebe00700 1 -- 172.21.15.90:0/3081738742 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8005310 con 0x563b028b59f0 2024-05-27T07:50:16.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbeebe00700 1 -- 172.21.15.90:0/3081738742 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7fbef80054b0 con 0x563b028b59f0 2024-05-27T07:50:16.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.702+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3081738742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbefc023350 con 0x563b028b59f0 2024-05-27T07:50:16.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.703+0000 7fbeebe00700 1 -- 172.21.15.90:0/3081738742 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbef8038050 con 0x563b028b59f0 2024-05-27T07:50:16.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.703+0000 7fbf1f4c6b80 10 client.16302 fetch_fsmap learned FSMap version 40 2024-05-27T07:50:16.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.703+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3081738742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563b029cd5d0 con 0x563b028b59f0 2024-05-27T07:50:16.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.703+0000 7fbeebe00700 1 -- 172.21.15.90:0/3081738742 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbefc023350 con 0x563b028b59f0 2024-05-27T07:50:16.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.703+0000 7fbeebe00700 1 -- 172.21.15.90:0/3081738742 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7fbef801b420 con 0x563b028b59f0 2024-05-27T07:50:16.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.703+0000 7fbf1f4c6b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:50:16.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.703+0000 7fbf1f4c6b80 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:16.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.703+0000 7fbf1f4c6b80 20 client.16302 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:16.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.704+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563b02962da0 0x563b029cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:16.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.704+0000 7fbf1f4c6b80 4 client.16302 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:50:16.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.704+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3081738742 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b02872570 con 0x563b02962da0 2024-05-27T07:50:16.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.704+0000 7fbf09e00700 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563b02962da0 0x563b029cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.705+0000 7fbf09e00700 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563b02962da0 0x563b029cfd00 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19326360116b353f server_cookie=d6b785a1fa71f390 in_seq=0 out_seq=0 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.705+0000 7fbeebe00700 10 client.16302 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbeebe00700 1 -- 172.21.15.90:0/3081738742 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563b02872570 con 0x563b02962da0 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbeebe00700 10 client.16302 handle_command_reply: tid=0 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 client.16302 shutdown 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3081738742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 msgr2=0x563b02956b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b02956b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbef800bbb0 tx=0x7fbef8011e70 comp rx=0 tx=0).stop 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3081738742 shutdown_connections 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563b02962da0 0x563b029cfd00 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563b029532c0 0x563b029570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b027c9e40 0x563b0295b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 --2- 172.21.15.90:0/3081738742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b028b59f0 0x563b02956b50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3081738742 >> 172.21.15.90:0/3081738742 conn(0x563b02863390 msgr2=0x563b0294b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:16.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbeebe00700 0 client.16302 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:16.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3081738742 shutdown_connections 2024-05-27T07:50:16.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.706+0000 7fbf1f4c6b80 1 -- 172.21.15.90:0/3081738742 wait complete. 2024-05-27T07:50:16.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.707+0000 7fbf1f4c6b80 20 client.16302 trim_cache size 0 max 16384 2024-05-27T07:50:16.708 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:50:16.708 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:50:16.708 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:50:16.708 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:50:16.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.707+0000 7fbf0b200700 1 -- 172.21.15.90:0/730352642 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbef00b2a70 msgr2=0x7fbef00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.707+0000 7fbf0b200700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbef00b2a70 0x7fbef00b4f80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009900 tx=0x7fbf04009c00 comp rx=0 tx=0).stop 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.707+0000 7fbf0b200700 1 -- 172.21.15.90:0/730352642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0c1043f0 msgr2=0x7fbf0c1a2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.707+0000 7fbf0b200700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0c1043f0 0x7fbf0c1a2050 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00d950 tx=0x7fbefc00dc60 comp rx=0 tx=0).stop 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.708+0000 7fbf0b200700 1 -- 172.21.15.90:0/730352642 shutdown_connections 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.708+0000 7fbf0b200700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbef00b2a70 0x7fbef00b4f80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.708+0000 7fbf0b200700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf0c109670 0x7fbf0c1acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.708+0000 7fbf0b200700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0c1043f0 0x7fbf0c1a2050 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.708+0000 7fbf0b200700 1 --2- 172.21.15.90:0/730352642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf0c108d20 0x7fbf0c1a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.708+0000 7fbf0b200700 1 -- 172.21.15.90:0/730352642 >> 172.21.15.90:0/730352642 conn(0x7fbf0c079d00 msgr2=0x7fbf0c0759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.708+0000 7fbf0b200700 1 -- 172.21.15.90:0/730352642 shutdown_connections 2024-05-27T07:50:16.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:16.708+0000 7fbf0b200700 1 -- 172.21.15.90:0/730352642 wait complete. 2024-05-27T07:50:17.620 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:50:17.620 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:50:18.083 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:18.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:18 smithi129 ceph-mon[35254]: pgmap v123: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 26 op/s 2024-05-27T07:50:18.416 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:18 smithi090 ceph-mon[31429]: pgmap v123: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 26 op/s 2024-05-27T07:50:18.443 INFO:teuthology.orchestra.run.smithi090.stderr:done in 154.81 s (drop tables 0.02 s, create tables 0.19 s, client-side generate 101.13 s, vacuum 32.48 s, primary keys 20.99 s). 2024-05-27T07:50:18.452 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-05-27T07:50:18.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:18 smithi136 ceph-mon[35324]: pgmap v123: 129 pgs: 129 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 26 op/s 2024-05-27T07:50:18.826 INFO:teuthology.orchestra.run.smithi090.stderr:starting vacuum...end. 2024-05-27T07:50:19.463 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f009dc00700 time 2024-05-27T07:50:19.202064+0000 2024-05-27T07:50:19.463 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:50:19.463 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f00b376eba6] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f00b376edd1] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5567a21d523d] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5567a21c0971] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 5: (SnapRealm::check_cache() const+0x13e) [0x5567a21befa0] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5567a21bf651] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5567a2154347] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5567a1e5087e] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5567a1e55e94] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5567a1e60969] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 11: (MDLog::_replay_thread()+0x1f38) [0x5567a21f948c] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5567a22029cb] 2024-05-27T07:50:19.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 13: (Thread::entry_wrapper()+0x43) [0x7f00b373d72d] 2024-05-27T07:50:19.465 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 14: (Thread::_entry_func(void*)+0xd) [0x7f00b373d749] 2024-05-27T07:50:19.465 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f00b22e11ca] 2024-05-27T07:50:19.465 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 16: clone() 2024-05-27T07:50:19.465 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.204+0000 7f009dc00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f009dc00700 time 2024-05-27T07:50:19.202064+0000 2024-05-27T07:50:19.465 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:50:19.465 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.465 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:50:19.465 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f00b376eba6] 2024-05-27T07:50:19.465 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f00b376edd1] 2024-05-27T07:50:19.465 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5567a21d523d] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5567a21c0971] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 5: (SnapRealm::check_cache() const+0x13e) [0x5567a21befa0] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5567a21bf651] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5567a2154347] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5567a1e5087e] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5567a1e55e94] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5567a1e60969] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 11: (MDLog::_replay_thread()+0x1f38) [0x5567a21f948c] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5567a22029cb] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 13: (Thread::entry_wrapper()+0x43) [0x7f00b373d72d] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 14: (Thread::_entry_func(void*)+0xd) [0x7f00b373d749] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f00b22e11ca] 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 16: clone() 2024-05-27T07:50:19.466 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.467 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: *** Caught signal (Aborted) ** 2024-05-27T07:50:19.467 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: in thread 7f009dc00700 thread_name:md_log_replay 2024-05-27T07:50:19.467 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:50:19.467 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f00b22ebd20] 2024-05-27T07:50:19.467 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2: gsignal() 2024-05-27T07:50:19.467 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 3: abort() 2024-05-27T07:50:19.467 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f00b376eced] 2024-05-27T07:50:19.467 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f00b376edd1] 2024-05-27T07:50:19.467 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5567a21d523d] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5567a21c0971] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 8: (SnapRealm::check_cache() const+0x13e) [0x5567a21befa0] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5567a21bf651] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5567a2154347] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5567a1e5087e] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5567a1e55e94] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5567a1e60969] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 14: (MDLog::_replay_thread()+0x1f38) [0x5567a21f948c] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5567a22029cb] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 16: (Thread::entry_wrapper()+0x43) [0x7f00b373d72d] 2024-05-27T07:50:19.468 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 17: (Thread::_entry_func(void*)+0xd) [0x7f00b373d749] 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f00b22e11ca] 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 19: clone() 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.208+0000 7f009dc00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: in thread 7f009dc00700 thread_name:md_log_replay 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f00b22ebd20] 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2: gsignal() 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 3: abort() 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f00b376eced] 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f00b376edd1] 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5567a21d523d] 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5567a21c0971] 2024-05-27T07:50:19.469 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 8: (SnapRealm::check_cache() const+0x13e) [0x5567a21befa0] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5567a21bf651] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5567a2154347] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5567a1e5087e] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5567a1e55e94] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5567a1e60969] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 14: (MDLog::_replay_thread()+0x1f38) [0x5567a21f948c] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5567a22029cb] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 16: (Thread::entry_wrapper()+0x43) [0x7f00b373d72d] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 17: (Thread::_entry_func(void*)+0xd) [0x7f00b373d749] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f00b22e11ca] 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 19: clone() 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.470 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: -1> 2024-05-27T07:50:19.204+0000 7f009dc00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f009dc00700 time 2024-05-27T07:50:19.202064+0000 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f00b376eba6] 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f00b376edd1] 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5567a21d523d] 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5567a21c0971] 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 5: (SnapRealm::check_cache() const+0x13e) [0x5567a21befa0] 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5567a21bf651] 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5567a2154347] 2024-05-27T07:50:19.471 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5567a1e5087e] 2024-05-27T07:50:19.472 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5567a1e55e94] 2024-05-27T07:50:19.472 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5567a1e60969] 2024-05-27T07:50:19.472 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 11: (MDLog::_replay_thread()+0x1f38) [0x5567a21f948c] 2024-05-27T07:50:19.472 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5567a22029cb] 2024-05-27T07:50:19.472 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 13: (Thread::entry_wrapper()+0x43) [0x7f00b373d72d] 2024-05-27T07:50:19.472 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 14: (Thread::_entry_func(void*)+0xd) [0x7f00b373d749] 2024-05-27T07:50:19.472 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f00b22e11ca] 2024-05-27T07:50:19.472 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 16: clone() 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 0> 2024-05-27T07:50:19.208+0000 7f009dc00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: in thread 7f009dc00700 thread_name:md_log_replay 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f00b22ebd20] 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2: gsignal() 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 3: abort() 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f00b376eced] 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f00b376edd1] 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5567a21d523d] 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5567a21c0971] 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 8: (SnapRealm::check_cache() const+0x13e) [0x5567a21befa0] 2024-05-27T07:50:19.473 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5567a21bf651] 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5567a2154347] 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5567a1e5087e] 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5567a1e55e94] 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5567a1e60969] 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 14: (MDLog::_replay_thread()+0x1f38) [0x5567a21f948c] 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5567a22029cb] 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 16: (Thread::entry_wrapper()+0x43) [0x7f00b373d72d] 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 17: (Thread::_entry_func(void*)+0xd) [0x7f00b373d749] 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f00b22e11ca] 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 19: clone() 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.474 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: -4> 2024-05-27T07:50:19.204+0000 7f009dc00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f009dc00700 time 2024-05-27T07:50:19.202064+0000 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f00b376eba6] 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f00b376edd1] 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5567a21d523d] 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5567a21c0971] 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 5: (SnapRealm::check_cache() const+0x13e) [0x5567a21befa0] 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5567a21bf651] 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5567a2154347] 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5567a1e5087e] 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5567a1e55e94] 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5567a1e60969] 2024-05-27T07:50:19.475 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 11: (MDLog::_replay_thread()+0x1f38) [0x5567a21f948c] 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5567a22029cb] 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 13: (Thread::entry_wrapper()+0x43) [0x7f00b373d72d] 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 14: (Thread::_entry_func(void*)+0xd) [0x7f00b373d749] 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f00b22e11ca] 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 16: clone() 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: -3> 2024-05-27T07:50:19.208+0000 7f009dc00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: in thread 7f009dc00700 thread_name:md_log_replay 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f00b22ebd20] 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2: gsignal() 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 3: abort() 2024-05-27T07:50:19.476 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f00b376eced] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f00b376edd1] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5567a21d523d] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5567a21c0971] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 8: (SnapRealm::check_cache() const+0x13e) [0x5567a21befa0] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5567a21bf651] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5567a2154347] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5567a1e5087e] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5567a1e55e94] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5567a1e60969] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 14: (MDLog::_replay_thread()+0x1f38) [0x5567a21f948c] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5567a22029cb] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 16: (Thread::entry_wrapper()+0x43) [0x7f00b373d72d] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 17: (Thread::_entry_func(void*)+0xd) [0x7f00b373d749] 2024-05-27T07:50:19.477 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f00b22e11ca] 2024-05-27T07:50:19.478 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 19: clone() 2024-05-27T07:50:19.478 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:50:19.478 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[71433]: 2024-05-27T07:50:19.919 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:19 smithi090 podman[101642]: 2024-05-27 07:50:19.611542131 +0000 UTC m=+0.291094733 container died 54c7cd984db6c32eaba3cb732331877a47c71c78c58cd937fa4ac97944b25443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0) 2024-05-27T07:50:20.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:20 smithi090 ceph-mon[31429]: pgmap v124: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 19 MiB/s wr, 36 op/s 2024-05-27T07:50:20.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:20 smithi136 ceph-mon[35324]: pgmap v124: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 19 MiB/s wr, 36 op/s 2024-05-27T07:50:20.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:20 smithi129 ceph-mon[35254]: pgmap v124: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 19 MiB/s wr, 36 op/s 2024-05-27T07:50:20.584 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:20 smithi090 podman[101642]: 2024-05-27 07:50:20.242884164 +0000 UTC m=+0.922436759 container remove 54c7cd984db6c32eaba3cb732331877a47c71c78c58cd937fa4ac97944b25443 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-27T07:50:20.585 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:20 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:50:21.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.067+0000 7f24ca600700 1 -- 172.21.15.90:0/3832655319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24c4116ec0 msgr2=0x7f24c41193d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:21.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.068+0000 7f24ca600700 1 --2- 172.21.15.90:0/3832655319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24c4116ec0 0x7f24c41193d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00b0a0 tx=0x7f24bc00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.068+0000 7f24ca600700 1 -- 172.21.15.90:0/3832655319 shutdown_connections 2024-05-27T07:50:21.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.068+0000 7f24ca600700 1 --2- 172.21.15.90:0/3832655319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24c4116ec0 0x7f24c41193d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.068+0000 7f24ca600700 1 --2- 172.21.15.90:0/3832655319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c410aa40 0x7f24c410ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.068+0000 7f24ca600700 1 --2- 172.21.15.90:0/3832655319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24c410b390 0x7f24c4116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.068+0000 7f24ca600700 1 -- 172.21.15.90:0/3832655319 >> 172.21.15.90:0/3832655319 conn(0x7f24c406f280 msgr2=0x7f24c406f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:21.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.068+0000 7f24ca600700 1 -- 172.21.15.90:0/3832655319 shutdown_connections 2024-05-27T07:50:21.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.068+0000 7f24ca600700 1 -- 172.21.15.90:0/3832655319 wait complete. 2024-05-27T07:50:21.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.071+0000 7f24ca600700 1 Processor -- start 2024-05-27T07:50:21.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.071+0000 7f24ca600700 1 -- start start 2024-05-27T07:50:21.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.072+0000 7f24ca600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24c410aa40 0x7f24c41a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:21.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.072+0000 7f24ca600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c410b390 0x7f24c41a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:21.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.072+0000 7f24ca600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24c4116ec0 0x7f24c41a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.072+0000 7f24ca600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f24c411c960 con 0x7f24c4116ec0 2024-05-27T07:50:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.072+0000 7f24ca600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f24c411c7e0 con 0x7f24c410aa40 2024-05-27T07:50:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.072+0000 7f24ca600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f24c411cae0 con 0x7f24c410b390 2024-05-27T07:50:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.073+0000 7f24c9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24c4116ec0 0x7f24c41a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:21.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.073+0000 7f24c9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24c4116ec0 0x7f24c41a7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43620/0 (socket says 172.21.15.90:43620) 2024-05-27T07:50:21.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.073+0000 7f24c9c00700 1 -- 172.21.15.90:0/2331775766 learned_addr learned my addr 172.21.15.90:0/2331775766 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:21.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.073+0000 7f24c9200700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24c410aa40 0x7f24c41a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:21.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.073+0000 7f24c3e00700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c410b390 0x7f24c41a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:21.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.074+0000 7f24c3e00700 1 -- 172.21.15.90:0/2331775766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24c410aa40 msgr2=0x7f24c41a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:21.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.074+0000 7f24c3e00700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24c410aa40 0x7f24c41a32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.075+0000 7f24c3e00700 1 -- 172.21.15.90:0/2331775766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24c4116ec0 msgr2=0x7f24c41a7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:21.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.075+0000 7f24c3e00700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24c4116ec0 0x7f24c41a7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.075+0000 7f24c3e00700 1 -- 172.21.15.90:0/2331775766 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c41a81c0 con 0x7f24c410b390 2024-05-27T07:50:21.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.075+0000 7f24c9200700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24c410aa40 0x7f24c41a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:21.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.075+0000 7f24c3e00700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c410b390 0x7f24c41a3890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009ef0 tx=0x7f24b800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:21.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.078+0000 7f24c1600700 1 -- 172.21.15.90:0/2331775766 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8020070 con 0x7f24c410b390 2024-05-27T07:50:21.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.078+0000 7f24ca600700 1 -- 172.21.15.90:0/2331775766 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c41a8450 con 0x7f24c410b390 2024-05-27T07:50:21.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.078+0000 7f24ca600700 1 -- 172.21.15.90:0/2331775766 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c410c5a0 con 0x7f24c410b390 2024-05-27T07:50:21.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.078+0000 7f24c1600700 1 -- 172.21.15.90:0/2331775766 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24b800ee80 con 0x7f24c410b390 2024-05-27T07:50:21.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.078+0000 7f24c1600700 1 -- 172.21.15.90:0/2331775766 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b801b730 con 0x7f24c410b390 2024-05-27T07:50:21.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.081+0000 7f24ca600700 1 -- 172.21.15.90:0/2331775766 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a4005330 con 0x7f24c410b390 2024-05-27T07:50:21.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.084+0000 7f24c1600700 1 -- 172.21.15.90:0/2331775766 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f24b8023070 con 0x7f24c410b390 2024-05-27T07:50:21.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.086+0000 7f24c1600700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f24ac0b2b20 0x7f24ac0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:21.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.086+0000 7f24c1600700 1 -- 172.21.15.90:0/2331775766 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f24b80e66e0 con 0x7f24c410b390 2024-05-27T07:50:21.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.087+0000 7f24c9200700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f24ac0b2b20 0x7f24ac0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:21.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.088+0000 7f24c9200700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f24ac0b2b20 0x7f24ac0b5030 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24b0005950 tx=0x7f24b00058e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:21.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.105+0000 7f24c1600700 1 -- 172.21.15.90:0/2331775766 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f24b80ade20 con 0x7f24c410b390 2024-05-27T07:50:21.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:21 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-05-27T07:50:21.464 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:21 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 2.081s CPU time. 2024-05-27T07:50:21.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.568+0000 7f24ca600700 1 -- 172.21.15.90:0/2331775766 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24a40058f0 con 0x7f24c410b390 2024-05-27T07:50:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.570+0000 7f24c1600700 1 -- 172.21.15.90:0/2331775766 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11113 (secure 0 0 0) 0x7f24b80ad720 con 0x7f24c410b390 2024-05-27T07:50:21.570 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:50:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:49:31.929341+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14514":{"gid":14514,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6837/3818984519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3818984519},{"type":"v1","addr":"172.21.15.90:6837","nonce":3818984519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T07:50:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.573+0000 7f24ca600700 1 -- 172.21.15.90:0/2331775766 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f24ac0b2b20 msgr2=0x7f24ac0b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.573+0000 7f24ca600700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f24ac0b2b20 0x7f24ac0b5030 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24b0005950 tx=0x7f24b00058e0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.573+0000 7f24ca600700 1 -- 172.21.15.90:0/2331775766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c410b390 msgr2=0x7f24c41a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.573+0000 7f24ca600700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c410b390 0x7f24c41a3890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009ef0 tx=0x7f24b800eca0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.573+0000 7f24ca600700 1 -- 172.21.15.90:0/2331775766 shutdown_connections 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.573+0000 7f24ca600700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24c4116ec0 0x7f24c41a7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.573+0000 7f24ca600700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f24ac0b2b20 0x7f24ac0b5030 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.573+0000 7f24ca600700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24c410aa40 0x7f24c41a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.573+0000 7f24ca600700 1 --2- 172.21.15.90:0/2331775766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c410b390 0x7f24c41a3890 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.573+0000 7f24ca600700 1 -- 172.21.15.90:0/2331775766 >> 172.21.15.90:0/2331775766 conn(0x7f24c406f280 msgr2=0x7f24c40727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.574+0000 7f24ca600700 1 -- 172.21.15.90:0/2331775766 shutdown_connections 2024-05-27T07:50:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:21.574+0000 7f24ca600700 1 -- 172.21.15.90:0/2331775766 wait complete. 2024-05-27T07:50:21.575 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 40 2024-05-27T07:50:22.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:22 smithi090 ceph-mon[31429]: pgmap v125: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 27 op/s 2024-05-27T07:50:22.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:22 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2331775766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:50:22.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:22 smithi136 ceph-mon[35324]: pgmap v125: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 27 op/s 2024-05-27T07:50:22.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:22 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2331775766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:50:22.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:22 smithi129 ceph-mon[35254]: pgmap v125: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 27 op/s 2024-05-27T07:50:22.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:22 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2331775766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.235+0000 7f94fc200640 1 -- 172.21.15.90:0/4243612714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f410a130 msgr2=0x7f94f4103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.235+0000 7f94fc200640 1 --2- 172.21.15.90:0/4243612714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f410a130 0x7f94f4103d70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009900 tx=0x7f94e002a410 comp rx=0 tx=0).stop 2024-05-27T07:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.235+0000 7f94fc200640 1 -- 172.21.15.90:0/4243612714 shutdown_connections 2024-05-27T07:50:23.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.235+0000 7f94fc200640 1 --2- 172.21.15.90:0/4243612714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94f41043e0 0x7f94f41107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.235+0000 7f94fc200640 1 --2- 172.21.15.90:0/4243612714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f410a130 0x7f94f4103d70 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.235+0000 7f94fc200640 1 --2- 172.21.15.90:0/4243612714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f4109c90 0x7f94f4103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.235+0000 7f94fc200640 1 -- 172.21.15.90:0/4243612714 >> 172.21.15.90:0/4243612714 conn(0x7f94f40ff370 msgr2=0x7f94f41017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.235+0000 7f94fc200640 1 -- 172.21.15.90:0/4243612714 shutdown_connections 2024-05-27T07:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.235+0000 7f94fc200640 1 -- 172.21.15.90:0/4243612714 wait complete. 2024-05-27T07:50:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.236+0000 7f94fc200640 1 Processor -- start 2024-05-27T07:50:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.237+0000 7f94fc200640 1 -- start start 2024-05-27T07:50:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.237+0000 7f94fc200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f41043e0 0x7f94f40760d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.237+0000 7f94fc200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94f4109c90 0x7f94f4076610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.237+0000 7f94fc200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f410a130 0x7f94f4076b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.237+0000 7f94fc200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4113d30 con 0x7f94f41043e0 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.237+0000 7f94fc200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4113bb0 con 0x7f94f410a130 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.237+0000 7f94fc200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4113eb0 con 0x7f94f4109c90 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.237+0000 7f94f9800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f41043e0 0x7f94f40760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.237+0000 7f94f9800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f41043e0 0x7f94f40760d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43634/0 (socket says 172.21.15.90:43634) 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.237+0000 7f94f9800640 1 -- 172.21.15.90:0/48962348 learned_addr learned my addr 172.21.15.90:0/48962348 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94fa200640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f410a130 0x7f94f4076b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94f8e00640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94f4109c90 0x7f94f4076610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94f9800640 1 -- 172.21.15.90:0/48962348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94f4109c90 msgr2=0x7f94f4076610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:23.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94f9800640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94f4109c90 0x7f94f4076610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94f9800640 1 -- 172.21.15.90:0/48962348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f410a130 msgr2=0x7f94f4076b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94f9800640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f410a130 0x7f94f4076b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94f9800640 1 -- 172.21.15.90:0/48962348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f41a8b50 con 0x7f94f41043e0 2024-05-27T07:50:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94f9800640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f41043e0 0x7f94f40760d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f94e400ea80 tx=0x7f94e400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94f2000640 1 -- 172.21.15.90:0/48962348 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e400ce40 con 0x7f94f41043e0 2024-05-27T07:50:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94f2000640 1 -- 172.21.15.90:0/48962348 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94e4011900 con 0x7f94f41043e0 2024-05-27T07:50:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.238+0000 7f94f2000640 1 -- 172.21.15.90:0/48962348 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e401a9c0 con 0x7f94f41043e0 2024-05-27T07:50:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.239+0000 7f94fc200640 1 -- 172.21.15.90:0/48962348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f41a8e40 con 0x7f94f41043e0 2024-05-27T07:50:23.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.239+0000 7f94fc200640 1 -- 172.21.15.90:0/48962348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f41a92b0 con 0x7f94f41043e0 2024-05-27T07:50:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.243+0000 7f9503280740 1 Processor -- start 2024-05-27T07:50:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.243+0000 7f9503280740 1 -- start start 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f9503280740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646b01b0 0x563d64a52860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f9503280740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 0x563d64a52da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f9503280740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f9503280740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563d649c3b90 con 0x563d6468f4d0 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f9503280740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563d649b3730 con 0x563d646872f0 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f9503280740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563d649b7770 con 0x563d646b01b0 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f94f2000640 1 -- 172.21.15.90:0/48962348 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f94e4011470 con 0x7f94f41043e0 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f94f8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f94f8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43650/0 (socket says 172.21.15.90:43650) 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f94f8e00640 1 -- 172.21.15.90:0/996057475 learned_addr learned my addr 172.21.15.90:0/996057475 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.244+0000 7f94fa200640 1 --2- 172.21.15.90:0/996057475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646b01b0 0x563d64a52860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.245+0000 7f94f8e00640 1 -- 172.21.15.90:0/996057475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646b01b0 msgr2=0x563d64a52860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.245+0000 7f94f8e00640 1 --2- 172.21.15.90:0/996057475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646b01b0 0x563d64a52860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.245+0000 7f94f9800640 1 --2- 172.21.15.90:0/996057475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 0x563d64a52da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.245+0000 7f94f8e00640 1 -- 172.21.15.90:0/996057475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 msgr2=0x563d64a52da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.245+0000 7f94f8e00640 1 --2- 172.21.15.90:0/996057475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 0x563d64a52da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.245+0000 7f94f8e00640 1 -- 172.21.15.90:0/996057475 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6494c8b0 con 0x563d6468f4d0 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.245+0000 7f94f8e00640 1 --2- 172.21.15.90:0/996057475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a532e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f94e0000c00 tx=0x7f94e0032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ef8c658a17438a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.246+0000 7f94f2000640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f94d80b2950 0x7f94d80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.246+0000 7f94eaa00640 1 -- 172.21.15.90:0/996057475 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e003f070 con 0x563d6468f4d0 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.246+0000 7f94eaa00640 1 -- 172.21.15.90:0/996057475 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94e0004030 con 0x563d6468f4d0 2024-05-27T07:50:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.246+0000 7f94eaa00640 1 -- 172.21.15.90:0/996057475 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0035070 con 0x563d6468f4d0 2024-05-27T07:50:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.246+0000 7f94f2000640 1 -- 172.21.15.90:0/48962348 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f94e40e2f90 con 0x7f94f41043e0 2024-05-27T07:50:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.246+0000 7f94fa200640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f94d80b2950 0x7f94d80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.247+0000 7f9503280740 1 -- 172.21.15.90:0/996057475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 msgr2=0x563d64a532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.247+0000 7f9503280740 1 --2- 172.21.15.90:0/996057475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a532e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f94e0000c00 tx=0x7f94e0032bc0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.247+0000 7f94fa200640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f94d80b2950 0x7f94d80b4e60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005fd0 tx=0x7f94ec00b5d0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.247+0000 7f9503280740 1 -- 172.21.15.90:0/996057475 shutdown_connections 2024-05-27T07:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.247+0000 7f9503280740 1 --2- 172.21.15.90:0/996057475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 0x563d64a52da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.247+0000 7f9503280740 1 --2- 172.21.15.90:0/996057475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a532e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.247+0000 7f9503280740 1 --2- 172.21.15.90:0/996057475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646b01b0 0x563d64a52860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.247+0000 7f9503280740 1 -- 172.21.15.90:0/996057475 >> 172.21.15.90:0/996057475 conn(0x563d646a4300 msgr2=0x563d64a4f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.247+0000 7f9503280740 1 -- 172.21.15.90:0/996057475 shutdown_connections 2024-05-27T07:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.247+0000 7f9503280740 1 -- 172.21.15.90:0/996057475 wait complete. 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.248+0000 7f9503280740 1 -- start start 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.248+0000 7f9503280740 1 Processor -- start 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.248+0000 7f9503280740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646872f0 0x563d64a5ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.248+0000 7f9503280740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6468f4d0 0x563d64a5cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.248+0000 7f9503280740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d646b01b0 0x563d64a68c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.248+0000 7f9503280740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563d64699790 con 0x563d646b01b0 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.248+0000 7f9503280740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563d6495bbc0 con 0x563d6468f4d0 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.248+0000 7f9503280740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563d64a5f930 con 0x563d646872f0 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.248+0000 7f9503280740 10 client.0 _ll_register_callbacks cb 0x563d64939400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.249+0000 7f94fa200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d646b01b0 0x563d64a68c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.249+0000 7f94fa200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d646b01b0 0x563d64a68c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43654/0 (socket says 172.21.15.90:43654) 2024-05-27T07:50:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.249+0000 7f94fa200640 1 -- 172.21.15.90:0/2121856495 learned_addr learned my addr 172.21.15.90:0/2121856495 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.249+0000 7f94f9800640 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646872f0 0x563d64a5ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.249+0000 7f94f8e00640 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6468f4d0 0x563d64a5cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.249+0000 7f94fa200640 1 -- 172.21.15.90:0/2121856495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646872f0 msgr2=0x563d64a5ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.249+0000 7f94fa200640 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646872f0 0x563d64a5ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.249+0000 7f94fa200640 1 -- 172.21.15.90:0/2121856495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6468f4d0 msgr2=0x563d64a5cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.249+0000 7f94fa200640 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6468f4d0 0x563d64a5cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.249+0000 7f94fa200640 1 -- 172.21.15.90:0/2121856495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d646c8210 con 0x563d646b01b0 2024-05-27T07:50:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.250+0000 7f94fa200640 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d646b01b0 0x563d64a68c60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f94ec012700 tx=0x7f94ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc0779a0ad9c61b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.250+0000 7f94df400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.250+0000 7f94df400640 1 -- 172.21.15.90:0/2121856495 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec005220 con 0x563d646b01b0 2024-05-27T07:50:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.250+0000 7f94df400640 1 -- 172.21.15.90:0/2121856495 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f94ec0053c0 con 0x563d646b01b0 2024-05-27T07:50:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.250+0000 7f9503280740 1 -- 172.21.15.90:0/2121856495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94e0004030 con 0x563d646b01b0 2024-05-27T07:50:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.250+0000 7f94df400640 1 -- 172.21.15.90:0/2121856495 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f94ec02f050 con 0x563d646b01b0 2024-05-27T07:50:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.250+0000 7f9503280740 10 client.16320 fetch_fsmap learned FSMap version 40 2024-05-27T07:50:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.250+0000 7f9503280740 1 -- 172.21.15.90:0/2121856495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563d64794620 con 0x563d646b01b0 2024-05-27T07:50:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.250+0000 7f94df400640 1 -- 172.21.15.90:0/2121856495 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94e0004030 con 0x563d646b01b0 2024-05-27T07:50:23.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.251+0000 7f94df400640 1 -- 172.21.15.90:0/2121856495 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7f94ec01f9b0 con 0x563d646b01b0 2024-05-27T07:50:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.251+0000 7f9503280740 10 client.16320 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:50:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.251+0000 7f9503280740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.251+0000 7f9503280740 20 client.16320 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.251+0000 7f9503280740 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563d6466ed80 0x563d64a6ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.251+0000 7f9503280740 4 client.16320 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.251+0000 7f9503280740 1 -- 172.21.15.90:0/2121856495 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d649c5ae0 con 0x563d6466ed80 2024-05-27T07:50:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.252+0000 7f94f9800640 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563d6466ed80 0x563d64a6ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.252+0000 7f94f9800640 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563d6466ed80 0x563d64a6ed50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9f4ea2974a2f05 server_cookie=1d1824e443dbf70c in_seq=0 out_seq=0 2024-05-27T07:50:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.252+0000 7f94df400640 10 client.16320 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f94df400640 1 -- 172.21.15.90:0/2121856495 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563d649c5ae0 con 0x563d6466ed80 2024-05-27T07:50:23.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f94df400640 10 client.16320 handle_command_reply: tid=0 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 client.16320 shutdown 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 -- 172.21.15.90:0/2121856495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d646b01b0 msgr2=0x563d64a68c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d646b01b0 0x563d64a68c60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f94ec012700 tx=0x7f94ec0046c0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 -- 172.21.15.90:0/2121856495 shutdown_connections 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563d6466ed80 0x563d64a6ed50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6468f4d0 0x563d64a5cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d646b01b0 0x563d64a68c60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f94df400640 0 client.16320 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 --2- 172.21.15.90:0/2121856495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646872f0 0x563d64a5ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 -- 172.21.15.90:0/2121856495 >> 172.21.15.90:0/2121856495 conn(0x563d646a4300 msgr2=0x563d64a54580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 -- 172.21.15.90:0/2121856495 shutdown_connections 2024-05-27T07:50:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 1 -- 172.21.15.90:0/2121856495 wait complete. 2024-05-27T07:50:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.255+0000 7f9503280740 20 client.16320 trim_cache size 0 max 16384 2024-05-27T07:50:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.273+0000 7f9503280740 1 Processor -- start 2024-05-27T07:50:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.273+0000 7f9503280740 1 -- start start 2024-05-27T07:50:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.273+0000 7f9503280740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646872f0 0x563d64a5ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.273+0000 7f9503280740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.273+0000 7f9503280740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646b01b0 0x563d64a68f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.273+0000 7f9503280740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41b4d70 con 0x563d6468f4d0 2024-05-27T07:50:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.273+0000 7f9503280740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41b2b60 con 0x563d646b01b0 2024-05-27T07:50:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.273+0000 7f9503280740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41b46a0 con 0x563d646872f0 2024-05-27T07:50:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94f8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94f8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a689d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43658/0 (socket says 172.21.15.90:43658) 2024-05-27T07:50:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94f8e00640 1 -- 172.21.15.90:0/2099432417 learned_addr learned my addr 172.21.15.90:0/2099432417 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94fa200640 1 --2- 172.21.15.90:0/2099432417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646b01b0 0x563d64a68f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94f9800640 1 --2- 172.21.15.90:0/2099432417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646872f0 0x563d64a5ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94f8e00640 1 -- 172.21.15.90:0/2099432417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646872f0 msgr2=0x563d64a5ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94f8e00640 1 --2- 172.21.15.90:0/2099432417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646872f0 0x563d64a5ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94f8e00640 1 -- 172.21.15.90:0/2099432417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646b01b0 msgr2=0x563d64a68f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94f8e00640 1 --2- 172.21.15.90:0/2099432417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646b01b0 0x563d64a68f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94f8e00640 1 -- 172.21.15.90:0/2099432417 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d64a5af50 con 0x563d6468f4d0 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.274+0000 7f94f8e00640 1 --2- 172.21.15.90:0/2099432417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a689d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009900 tx=0x7f94e00056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85936c518d0d589e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.275+0000 7f94ea000640 1 -- 172.21.15.90:0/2099432417 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e004b020 con 0x563d6468f4d0 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.275+0000 7f94ea000640 1 -- 172.21.15.90:0/2099432417 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94e00044d0 con 0x563d6468f4d0 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.275+0000 7f94ea000640 1 -- 172.21.15.90:0/2099432417 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0035040 con 0x563d6468f4d0 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.275+0000 7f9503280740 1 -- 172.21.15.90:0/2099432417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 msgr2=0x563d64a689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.275+0000 7f9503280740 1 --2- 172.21.15.90:0/2099432417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a689d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009900 tx=0x7f94e00056f0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.276+0000 7f9503280740 1 -- 172.21.15.90:0/2099432417 shutdown_connections 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.276+0000 7f9503280740 1 --2- 172.21.15.90:0/2099432417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646b01b0 0x563d64a68f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.276+0000 7f9503280740 1 --2- 172.21.15.90:0/2099432417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a689d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.276+0000 7f9503280740 1 --2- 172.21.15.90:0/2099432417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646872f0 0x563d64a5ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.276+0000 7f9503280740 1 -- 172.21.15.90:0/2099432417 >> 172.21.15.90:0/2099432417 conn(0x563d646a4300 msgr2=0x563d64a5ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.276+0000 7f9503280740 1 -- 172.21.15.90:0/2099432417 shutdown_connections 2024-05-27T07:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.276+0000 7f9503280740 1 -- 172.21.15.90:0/2099432417 wait complete. 2024-05-27T07:50:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.276+0000 7f9503280740 1 -- start start 2024-05-27T07:50:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.276+0000 7f9503280740 1 Processor -- start 2024-05-27T07:50:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f9503280740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 0x563d64a64eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f9503280740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f9503280740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646b01b0 0x563d64a60310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f9503280740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41b3900 con 0x563d6468f4d0 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f9503280740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41b3230 con 0x563d646872f0 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f9503280740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41b3fd0 con 0x563d646b01b0 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f9503280740 10 client.0 _ll_register_callbacks cb 0x7f94ec00b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f94f8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f94f8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a653f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43666/0 (socket says 172.21.15.90:43666) 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f94f8e00640 1 -- 172.21.15.90:0/3403214199 learned_addr learned my addr 172.21.15.90:0/3403214199 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f94f9800640 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 0x563d64a64eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f94fa200640 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646b01b0 0x563d64a60310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f94f8e00640 1 -- 172.21.15.90:0/3403214199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646b01b0 msgr2=0x563d64a60310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f94f8e00640 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646b01b0 0x563d64a60310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.277+0000 7f94f8e00640 1 -- 172.21.15.90:0/3403214199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 msgr2=0x563d64a64eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f94f8e00640 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 0x563d64a64eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f94f8e00640 1 -- 172.21.15.90:0/3403214199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d64a60a50 con 0x563d6468f4d0 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f94f9800640 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 0x563d64a64eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f94f8e00640 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a653f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f94e002a930 tx=0x7f94e003b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aa6b50d62ed9f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f94df400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f94df400640 1 -- 172.21.15.90:0/3403214199 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e004b030 con 0x563d6468f4d0 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f94df400640 1 -- 172.21.15.90:0/3403214199 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f94e003bc60 con 0x563d6468f4d0 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f9503280740 1 -- 172.21.15.90:0/3403214199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94e00044d0 con 0x563d6468f4d0 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f94df400640 1 -- 172.21.15.90:0/3403214199 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f94e005e050 con 0x563d6468f4d0 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f9503280740 10 client.16326 fetch_fsmap learned FSMap version 40 2024-05-27T07:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.278+0000 7f9503280740 1 -- 172.21.15.90:0/3403214199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563d64ad6eb0 con 0x563d6468f4d0 2024-05-27T07:50:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.279+0000 7f94df400640 1 -- 172.21.15.90:0/3403214199 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94e00044d0 con 0x563d6468f4d0 2024-05-27T07:50:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.279+0000 7f94df400640 1 -- 172.21.15.90:0/3403214199 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7f94e0050c10 con 0x563d6468f4d0 2024-05-27T07:50:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.279+0000 7f9503280740 10 client.16326 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:50:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.279+0000 7f9503280740 10 client.16326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.279+0000 7f9503280740 20 client.16326 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.279+0000 7f9503280740 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563d646441c0 0x563d64ad90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.279+0000 7f9503280740 4 client.16326 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:50:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.279+0000 7f9503280740 1 -- 172.21.15.90:0/3403214199 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d649bf2c0 con 0x563d646441c0 2024-05-27T07:50:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.280+0000 7f94f9800640 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563d646441c0 0x563d64ad90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.281+0000 7f94f9800640 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563d646441c0 0x563d64ad90e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286ae53e6235ec53 server_cookie=37effd95fa23cc09 in_seq=0 out_seq=0 2024-05-27T07:50:23.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.282+0000 7f94df400640 10 client.16326 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.335+0000 7f94df400640 1 -- 172.21.15.90:0/3403214199 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563d649bf2c0 con 0x563d646441c0 2024-05-27T07:50:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.335+0000 7f94df400640 10 client.16326 handle_command_reply: tid=0 2024-05-27T07:50:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.335+0000 7f9503280740 1 client.16326 shutdown 2024-05-27T07:50:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.335+0000 7f9503280740 1 -- 172.21.15.90:0/3403214199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 msgr2=0x563d64a653f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.335+0000 7f9503280740 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a653f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f94e002a930 tx=0x7f94e003b660 comp rx=0 tx=0).stop 2024-05-27T07:50:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.336+0000 7f9503280740 1 -- 172.21.15.90:0/3403214199 shutdown_connections 2024-05-27T07:50:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.336+0000 7f9503280740 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x563d646441c0 0x563d64ad90e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.336+0000 7f9503280740 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563d646b01b0 0x563d64a60310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.336+0000 7f9503280740 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563d6468f4d0 0x563d64a653f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.336+0000 7f9503280740 1 --2- 172.21.15.90:0/3403214199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d646872f0 0x563d64a64eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.336+0000 7f9503280740 1 -- 172.21.15.90:0/3403214199 >> 172.21.15.90:0/3403214199 conn(0x563d646a4300 msgr2=0x563d64a525f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.336+0000 7f94df400640 0 client.16326 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.336+0000 7f9503280740 1 -- 172.21.15.90:0/3403214199 shutdown_connections 2024-05-27T07:50:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.336+0000 7f9503280740 1 -- 172.21.15.90:0/3403214199 wait complete. 2024-05-27T07:50:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.336+0000 7f9503280740 20 client.16326 trim_cache size 0 max 16384 2024-05-27T07:50:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:50:23.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:50:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:50:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.337+0000 7f94fc200640 1 -- 172.21.15.90:0/48962348 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f94d80b2950 msgr2=0x7f94d80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.337+0000 7f94fc200640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f94d80b2950 0x7f94d80b4e60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005fd0 tx=0x7f94ec00b5d0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.337+0000 7f94fc200640 1 -- 172.21.15.90:0/48962348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f41043e0 msgr2=0x7f94f40760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.337+0000 7f94fc200640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f41043e0 0x7f94f40760d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f94e400ea80 tx=0x7f94e400ef40 comp rx=0 tx=0).stop 2024-05-27T07:50:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.337+0000 7f94fc200640 1 -- 172.21.15.90:0/48962348 shutdown_connections 2024-05-27T07:50:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.337+0000 7f94fc200640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f94d80b2950 0x7f94d80b4e60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.337+0000 7f94fc200640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94f410a130 0x7f94f4076b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.337+0000 7f94fc200640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94f4109c90 0x7f94f4076610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.337+0000 7f94fc200640 1 --2- 172.21.15.90:0/48962348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f41043e0 0x7f94f40760d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.337+0000 7f94fc200640 1 -- 172.21.15.90:0/48962348 >> 172.21.15.90:0/48962348 conn(0x7f94f40ff370 msgr2=0x7f94f410f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.338+0000 7f94fc200640 1 -- 172.21.15.90:0/48962348 shutdown_connections 2024-05-27T07:50:23.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:23.338+0000 7f94fc200640 1 -- 172.21.15.90:0/48962348 wait complete. 2024-05-27T07:50:23.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 5.0 s, 381.6 tps, lat 229.431 ms stddev 444.951 2024-05-27T07:50:24.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:24 smithi090 ceph-mon[31429]: pgmap v126: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 59 op/s 2024-05-27T07:50:24.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:24 smithi136 ceph-mon[35324]: pgmap v126: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 59 op/s 2024-05-27T07:50:24.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:24 smithi129 ceph-mon[35254]: pgmap v126: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 15 MiB/s wr, 59 op/s 2024-05-27T07:50:24.566 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:50:25.234 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:26.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:26 smithi090 ceph-mon[31429]: pgmap v127: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.6 MiB/s wr, 40 op/s 2024-05-27T07:50:26.366 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:26 smithi129 ceph-mon[35254]: pgmap v127: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.6 MiB/s wr, 40 op/s 2024-05-27T07:50:26.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:26 smithi136 ceph-mon[35324]: pgmap v127: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.6 MiB/s wr, 40 op/s 2024-05-27T07:50:28.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.261+0000 7f518a200700 1 -- 172.21.15.90:0/1750041215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5184075d20 msgr2=0x7f5184076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:28.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.261+0000 7f518a200700 1 --2- 172.21.15.90:0/1750041215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5184075d20 0x7f5184076130 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f517400b0a0 tx=0x7f517400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.262+0000 7f518a200700 1 -- 172.21.15.90:0/1750041215 shutdown_connections 2024-05-27T07:50:28.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.262+0000 7f518a200700 1 --2- 172.21.15.90:0/1750041215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5184110b90 0x7f5184112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.262+0000 7f518a200700 1 --2- 172.21.15.90:0/1750041215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5184075d20 0x7f5184076130 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.262+0000 7f518a200700 1 --2- 172.21.15.90:0/1750041215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5184076670 0x7f5184110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.262+0000 7f518a200700 1 -- 172.21.15.90:0/1750041215 >> 172.21.15.90:0/1750041215 conn(0x7f5184100030 msgr2=0x7f5184102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:28.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.263+0000 7f518a200700 1 -- 172.21.15.90:0/1750041215 shutdown_connections 2024-05-27T07:50:28.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.264+0000 7f518a200700 1 -- 172.21.15.90:0/1750041215 wait complete. 2024-05-27T07:50:28.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.265+0000 7f518a200700 1 Processor -- start 2024-05-27T07:50:28.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.265+0000 7f518a200700 1 -- start start 2024-05-27T07:50:28.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.266+0000 7f518a200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5184075d20 0x7f51841104b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.266+0000 7f518a200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5184076670 0x7f5184107510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.266+0000 7f518a200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5184110b90 0x7f5184107a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.266+0000 7f518a200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5184105cb0 con 0x7f5184110b90 2024-05-27T07:50:28.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.266+0000 7f518a200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5184105b30 con 0x7f5184075d20 2024-05-27T07:50:28.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.266+0000 7f518a200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5184105e30 con 0x7f5184076670 2024-05-27T07:50:28.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.267+0000 7f5183e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5184110b90 0x7f5184107a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.267+0000 7f517be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5184076670 0x7f5184107510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.267+0000 7f517be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5184076670 0x7f5184107510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37418/0 (socket says 172.21.15.90:37418) 2024-05-27T07:50:28.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.267+0000 7f517be00700 1 -- 172.21.15.90:0/312058814 learned_addr learned my addr 172.21.15.90:0/312058814 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:28.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.267+0000 7f5183400700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5184075d20 0x7f51841104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.268+0000 7f517be00700 1 -- 172.21.15.90:0/312058814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5184075d20 msgr2=0x7f51841104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:28.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.268+0000 7f517be00700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5184075d20 0x7f51841104b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.268+0000 7f517be00700 1 -- 172.21.15.90:0/312058814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5184110b90 msgr2=0x7f5184107a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:28.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.268+0000 7f517be00700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5184110b90 0x7f5184107a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.268+0000 7f517be00700 1 -- 172.21.15.90:0/312058814 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51841082c0 con 0x7f5184076670 2024-05-27T07:50:28.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.268+0000 7f5183e00700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5184110b90 0x7f5184107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:50:28.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.269+0000 7f517be00700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5184076670 0x7f5184107510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f516c00d950 tx=0x7f516c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:28.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.269+0000 7f5180c00700 1 -- 172.21.15.90:0/312058814 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c016070 con 0x7f5184076670 2024-05-27T07:50:28.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.270+0000 7f518a200700 1 -- 172.21.15.90:0/312058814 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518410bfc0 con 0x7f5184076670 2024-05-27T07:50:28.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.270+0000 7f5180c00700 1 -- 172.21.15.90:0/312058814 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f516c009dd0 con 0x7f5184076670 2024-05-27T07:50:28.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.270+0000 7f518a200700 1 -- 172.21.15.90:0/312058814 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518410c520 con 0x7f5184076670 2024-05-27T07:50:28.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.270+0000 7f5180c00700 1 -- 172.21.15.90:0/312058814 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c00f3c0 con 0x7f5184076670 2024-05-27T07:50:28.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:28 smithi090 ceph-mon[31429]: pgmap v128: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.6 MiB/s wr, 40 op/s 2024-05-27T07:50:28.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.281+0000 7f5198d48b80 1 Processor -- start 2024-05-27T07:50:28.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.281+0000 7f5198d48b80 1 -- start start 2024-05-27T07:50:28.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.282+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876013d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.282+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd87601910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.282+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8761d110 0x55fd8761f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.282+0000 7f5198d48b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fd8752fe40 con 0x55fd87482e40 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.282+0000 7f5198d48b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fd8751f9e0 con 0x55fd8761d110 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.282+0000 7f5198d48b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fd87523a20 con 0x55fd8756e9f0 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.282+0000 7f5180c00700 1 -- 172.21.15.90:0/312058814 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f516c019070 con 0x7f5184076670 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.283+0000 7f5183e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.283+0000 7f5183e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876013d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37424/0 (socket says 172.21.15.90:37424) 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.283+0000 7f5183e00700 1 -- 172.21.15.90:0/3829229131 learned_addr learned my addr 172.21.15.90:0/3829229131 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.283+0000 7f517be00700 1 --2- 172.21.15.90:0/3829229131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8761d110 0x55fd8761f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.283+0000 7f5183e00700 1 -- 172.21.15.90:0/3829229131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8761d110 msgr2=0x55fd8761f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.283+0000 7f5183e00700 1 --2- 172.21.15.90:0/3829229131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8761d110 0x55fd8761f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.284+0000 7f5183e00700 1 -- 172.21.15.90:0/3829229131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 msgr2=0x55fd87601910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:28.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.284+0000 7f5183e00700 1 --2- 172.21.15.90:0/3829229131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd87601910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.284+0000 7f5183e00700 1 -- 172.21.15.90:0/3829229131 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd874e8420 con 0x55fd8756e9f0 2024-05-27T07:50:28.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.285+0000 7f5183e00700 1 --2- 172.21.15.90:0/3829229131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876013d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f517c009dc0 tx=0x7f517c00e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad71b49dd24dcec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:28.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.285+0000 7f5180c00700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f51680b2b20 0x7f51680b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.285+0000 7f5178c00700 1 -- 172.21.15.90:0/3829229131 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c017070 con 0x55fd8756e9f0 2024-05-27T07:50:28.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.286+0000 7f5178c00700 1 -- 172.21.15.90:0/3829229131 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f517c00ec20 con 0x55fd8756e9f0 2024-05-27T07:50:28.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.286+0000 7f5178c00700 1 -- 172.21.15.90:0/3829229131 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c0123d0 con 0x55fd8756e9f0 2024-05-27T07:50:28.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.286+0000 7f5180c00700 1 -- 172.21.15.90:0/312058814 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f516c0e2800 con 0x7f5184076670 2024-05-27T07:50:28.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.287+0000 7f5198d48b80 1 -- 172.21.15.90:0/3829229131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 msgr2=0x55fd876013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:28.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.287+0000 7f5198d48b80 1 --2- 172.21.15.90:0/3829229131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876013d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f517c009dc0 tx=0x7f517c00e4b0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.288+0000 7f5198d48b80 1 -- 172.21.15.90:0/3829229131 shutdown_connections 2024-05-27T07:50:28.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.288+0000 7f5198d48b80 1 --2- 172.21.15.90:0/3829229131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8761d110 0x55fd8761f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.288+0000 7f5198d48b80 1 --2- 172.21.15.90:0/3829229131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876013d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.288+0000 7f5198d48b80 1 --2- 172.21.15.90:0/3829229131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd87601910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.288+0000 7f5198d48b80 1 -- 172.21.15.90:0/3829229131 >> 172.21.15.90:0/3829229131 conn(0x55fd8751c390 msgr2=0x55fd875fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:28.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.289+0000 7f5183400700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f51680b2b20 0x7f51680b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.290+0000 7f5198d48b80 1 -- 172.21.15.90:0/3829229131 shutdown_connections 2024-05-27T07:50:28.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.290+0000 7f5198d48b80 1 -- 172.21.15.90:0/3829229131 wait complete. 2024-05-27T07:50:28.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.291+0000 7f5198d48b80 1 -- start start 2024-05-27T07:50:28.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.292+0000 7f5183400700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f51680b2b20 0x7f51680b5030 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55fd875fea20 tx=0x7f517400b7f0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:28.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.292+0000 7f5198d48b80 1 Processor -- start 2024-05-27T07:50:28.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.293+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.293+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd8761a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.293+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8761d110 0x55fd8761ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.293+0000 7f5198d48b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fd87425fa0 con 0x55fd87482e40 2024-05-27T07:50:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.293+0000 7f5198d48b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fd874c0760 con 0x55fd8761d110 2024-05-27T07:50:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.293+0000 7f5198d48b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fd8761fbc0 con 0x55fd8756e9f0 2024-05-27T07:50:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.293+0000 7f5198d48b80 10 client.0 _ll_register_callbacks cb 0x55fd874c9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.294+0000 7f5183400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.294+0000 7f5183400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43702/0 (socket says 172.21.15.90:43702) 2024-05-27T07:50:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.294+0000 7f5183400700 1 -- 172.21.15.90:0/458271003 learned_addr learned my addr 172.21.15.90:0/458271003 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.294+0000 7f517be00700 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd8761a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.294+0000 7f5183e00700 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8761d110 0x55fd8761ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.295+0000 7f5183400700 1 -- 172.21.15.90:0/458271003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 msgr2=0x55fd8761a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.295+0000 7f5183400700 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd8761a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.295+0000 7f5183400700 1 -- 172.21.15.90:0/458271003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8761d110 msgr2=0x55fd8761ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.295+0000 7f5183400700 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8761d110 0x55fd8761ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.295+0000 7f5183400700 1 -- 172.21.15.90:0/458271003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd8746f8d0 con 0x55fd87482e40 2024-05-27T07:50:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.296+0000 7f5183400700 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761a190 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5174015000 tx=0x7f517400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ab9251cf7b093e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.296+0000 7f5171600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:28.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.298+0000 7f5171600700 1 -- 172.21.15.90:0/458271003 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517402d020 con 0x55fd87482e40 2024-05-27T07:50:28.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.298+0000 7f5171600700 1 -- 172.21.15.90:0/458271003 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f517401c440 con 0x55fd87482e40 2024-05-27T07:50:28.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.298+0000 7f5198d48b80 1 -- 172.21.15.90:0/458271003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517c00ec20 con 0x55fd87482e40 2024-05-27T07:50:28.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.300+0000 7f5171600700 1 -- 172.21.15.90:0/458271003 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5174038050 con 0x55fd87482e40 2024-05-27T07:50:28.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.300+0000 7f5198d48b80 10 client.16335 fetch_fsmap learned FSMap version 40 2024-05-27T07:50:28.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.300+0000 7f5198d48b80 1 -- 172.21.15.90:0/458271003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fd874f0060 con 0x55fd87482e40 2024-05-27T07:50:28.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.302+0000 7f5171600700 1 -- 172.21.15.90:0/458271003 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f517c00ec20 con 0x55fd87482e40 2024-05-27T07:50:28.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.303+0000 7f5171600700 1 -- 172.21.15.90:0/458271003 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7f5174011070 con 0x55fd87482e40 2024-05-27T07:50:28.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.303+0000 7f5198d48b80 10 client.16335 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:50:28.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.303+0000 7f5198d48b80 10 client.16335 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:50:28.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.303+0000 7f5198d48b80 20 client.16335 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:28.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.303+0000 7f5198d48b80 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fd8762cc20 0x55fd8762f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.303+0000 7f5198d48b80 4 client.16335 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:28.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.303+0000 7f5198d48b80 1 -- 172.21.15.90:0/458271003 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd87531d90 con 0x55fd8762cc20 2024-05-27T07:50:28.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.304+0000 7f517be00700 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fd8762cc20 0x55fd8762f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.306+0000 7f5171600700 10 client.16335 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:50:28.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.306+0000 7f517be00700 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fd8762cc20 0x55fd8762f080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3ce603324e2085dd server_cookie=32c67a323de7e17b in_seq=0 out_seq=0 2024-05-27T07:50:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.308+0000 7f5171600700 1 -- 172.21.15.90:0/458271003 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fd87531d90 con 0x55fd8762cc20 2024-05-27T07:50:28.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.308+0000 7f5171600700 10 client.16335 handle_command_reply: tid=0 2024-05-27T07:50:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.308+0000 7f5198d48b80 1 client.16335 shutdown 2024-05-27T07:50:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.309+0000 7f5198d48b80 1 -- 172.21.15.90:0/458271003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 msgr2=0x55fd8761a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.309+0000 7f5198d48b80 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761a190 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5174015000 tx=0x7f517400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.309+0000 7f5198d48b80 1 -- 172.21.15.90:0/458271003 shutdown_connections 2024-05-27T07:50:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.309+0000 7f5198d48b80 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fd8762cc20 0x55fd8762f080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.309+0000 7f5198d48b80 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8761d110 0x55fd8761ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.309+0000 7f5198d48b80 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761a190 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.309+0000 7f5198d48b80 1 --2- 172.21.15.90:0/458271003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd8761a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.309+0000 7f5198d48b80 1 -- 172.21.15.90:0/458271003 >> 172.21.15.90:0/458271003 conn(0x55fd8751c390 msgr2=0x55fd876013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:28.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.310+0000 7f5171600700 0 client.16335 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:50:28.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.310+0000 7f5198d48b80 1 -- 172.21.15.90:0/458271003 shutdown_connections 2024-05-27T07:50:28.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.310+0000 7f5198d48b80 1 -- 172.21.15.90:0/458271003 wait complete. 2024-05-27T07:50:28.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.310+0000 7f5198d48b80 20 client.16335 trim_cache size 0 max 16384 2024-05-27T07:50:28.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.339+0000 7f5198d48b80 1 Processor -- start 2024-05-27T07:50:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.340+0000 7f5198d48b80 1 -- start start 2024-05-27T07:50:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.340+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd87482e40 0x55fd875fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.340+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876157c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.340+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd8761d110 0x55fd87628cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.340+0000 7f5198d48b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f51600107b0 con 0x55fd8761d110 2024-05-27T07:50:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.340+0000 7f5198d48b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f516000e5a0 con 0x55fd87482e40 2024-05-27T07:50:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.340+0000 7f5198d48b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f51600100e0 con 0x55fd8756e9f0 2024-05-27T07:50:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.341+0000 7f5183e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd8761d110 0x55fd87628cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.341+0000 7f5183400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd87482e40 0x55fd875fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.341+0000 7f5183e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd8761d110 0x55fd87628cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43708/0 (socket says 172.21.15.90:43708) 2024-05-27T07:50:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.341+0000 7f5183e00700 1 -- 172.21.15.90:0/110781922 learned_addr learned my addr 172.21.15.90:0/110781922 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:28.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.341+0000 7f517be00700 1 --2- 172.21.15.90:0/110781922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876157c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.342+0000 7f517be00700 1 -- 172.21.15.90:0/110781922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd87482e40 msgr2=0x55fd875fd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:28.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.342+0000 7f517be00700 1 --2- 172.21.15.90:0/110781922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd87482e40 0x55fd875fd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.342+0000 7f517be00700 1 -- 172.21.15.90:0/110781922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd8761d110 msgr2=0x55fd87628cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:28.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.342+0000 7f517be00700 1 --2- 172.21.15.90:0/110781922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd8761d110 0x55fd87628cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.342+0000 7f517be00700 1 -- 172.21.15.90:0/110781922 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd875f9240 con 0x55fd8756e9f0 2024-05-27T07:50:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.343+0000 7f517be00700 1 --2- 172.21.15.90:0/110781922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876157c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f516c0dfba0 tx=0x7f516c0cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f6eecd2b94a7e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.344+0000 7f5173e00700 1 -- 172.21.15.90:0/110781922 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c020040 con 0x55fd8756e9f0 2024-05-27T07:50:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.344+0000 7f5173e00700 1 -- 172.21.15.90:0/110781922 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f516c0a3e50 con 0x55fd8756e9f0 2024-05-27T07:50:28.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.344+0000 7f5173e00700 1 -- 172.21.15.90:0/110781922 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c0b0e60 con 0x55fd8756e9f0 2024-05-27T07:50:28.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.346+0000 7f5198d48b80 1 -- 172.21.15.90:0/110781922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 msgr2=0x55fd876157c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.346+0000 7f5198d48b80 1 --2- 172.21.15.90:0/110781922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876157c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f516c0dfba0 tx=0x7f516c0cc040 comp rx=0 tx=0).stop 2024-05-27T07:50:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.346+0000 7f5198d48b80 1 -- 172.21.15.90:0/110781922 shutdown_connections 2024-05-27T07:50:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.346+0000 7f5198d48b80 1 --2- 172.21.15.90:0/110781922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd8761d110 0x55fd87628cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.346+0000 7f5198d48b80 1 --2- 172.21.15.90:0/110781922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd87482e40 0x55fd875fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.346+0000 7f5198d48b80 1 --2- 172.21.15.90:0/110781922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8756e9f0 0x55fd876157c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.346+0000 7f5198d48b80 1 -- 172.21.15.90:0/110781922 >> 172.21.15.90:0/110781922 conn(0x55fd8751c390 msgr2=0x55fd876150b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:28.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.347+0000 7f5198d48b80 1 -- 172.21.15.90:0/110781922 shutdown_connections 2024-05-27T07:50:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.347+0000 7f5198d48b80 1 -- 172.21.15.90:0/110781922 wait complete. 2024-05-27T07:50:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.348+0000 7f5198d48b80 1 -- start start 2024-05-27T07:50:28.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.348+0000 7f5198d48b80 1 Processor -- start 2024-05-27T07:50:28.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.349+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.350+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8756e9f0 0x55fd8761da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.350+0000 7f5198d48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8761d110 0x55fd8761dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.350+0000 7f5183400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.350+0000 7f5183400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43712/0 (socket says 172.21.15.90:43712) 2024-05-27T07:50:28.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.351+0000 7f517be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8756e9f0 0x55fd8761da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.351+0000 7f517be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8756e9f0 0x55fd8761da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46980/0 (socket says 172.21.15.90:46980) 2024-05-27T07:50:28.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.351+0000 7f517be00700 1 -- 172.21.15.90:0/189340273 learned_addr learned my addr 172.21.15.90:0/189340273 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:28.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.351+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f516000f340 con 0x55fd87482e40 2024-05-27T07:50:28.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.351+0000 7f5183e00700 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8761d110 0x55fd8761dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.352+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f516000ec70 con 0x55fd8756e9f0 2024-05-27T07:50:28.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.352+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f516000fa10 con 0x55fd8761d110 2024-05-27T07:50:28.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.352+0000 7f5198d48b80 10 client.0 _ll_register_callbacks cb 0x55fd874c9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:28.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.353+0000 7f5183400700 1 -- 172.21.15.90:0/189340273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8761d110 msgr2=0x55fd8761dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:28.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.353+0000 7f5183400700 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8761d110 0x55fd8761dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.353+0000 7f5183400700 1 -- 172.21.15.90:0/189340273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8756e9f0 msgr2=0x55fd8761da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:28.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.353+0000 7f5183400700 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8756e9f0 0x55fd8761da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.353+0000 7f5183400700 1 -- 172.21.15.90:0/189340273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd87623f60 con 0x55fd87482e40 2024-05-27T07:50:28.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.353+0000 7f5183e00700 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8761d110 0x55fd8761dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:28.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.353+0000 7f5183400700 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761d520 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5174009fd0 tx=0x7f5174007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cf839e20574136 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:28.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.355+0000 7f5171600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:28.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.355+0000 7f5171600700 1 -- 172.21.15.90:0/189340273 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517402d070 con 0x55fd87482e40 2024-05-27T07:50:28.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.355+0000 7f5171600700 1 -- 172.21.15.90:0/189340273 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f5174005e70 con 0x55fd87482e40 2024-05-27T07:50:28.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.355+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f516c0a3e50 con 0x55fd87482e40 2024-05-27T07:50:28.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.356+0000 7f5171600700 1 -- 172.21.15.90:0/189340273 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5174005600 con 0x55fd87482e40 2024-05-27T07:50:28.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.356+0000 7f5198d48b80 10 client.16341 fetch_fsmap learned FSMap version 40 2024-05-27T07:50:28.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.356+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fd87625180 con 0x55fd87482e40 2024-05-27T07:50:28.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.357+0000 7f5171600700 1 -- 172.21.15.90:0/189340273 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f516c0a3e50 con 0x55fd87482e40 2024-05-27T07:50:28.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.358+0000 7f5171600700 1 -- 172.21.15.90:0/189340273 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7f5174011040 con 0x55fd87482e40 2024-05-27T07:50:28.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.358+0000 7f5198d48b80 10 client.16341 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:50:28.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.358+0000 7f5198d48b80 10 client.16341 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:50:28.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.358+0000 7f5198d48b80 20 client.16341 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:28.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.359+0000 7f5198d48b80 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fd876257d0 0x55fd87699760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:28.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.359+0000 7f5198d48b80 4 client.16341 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:50:28.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.359+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd8752b570 con 0x55fd876257d0 2024-05-27T07:50:28.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.360+0000 7f517be00700 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fd876257d0 0x55fd87699760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:28.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.361+0000 7f5171600700 10 client.16341 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:50:28.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.361+0000 7f517be00700 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fd876257d0 0x55fd87699760 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1a0dcb929764276 server_cookie=f586679a3fe1d025 in_seq=0 out_seq=0 2024-05-27T07:50:28.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.362+0000 7f5171600700 1 -- 172.21.15.90:0/189340273 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd8752b570 con 0x55fd876257d0 2024-05-27T07:50:28.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.362+0000 7f5171600700 10 client.16341 handle_command_reply: tid=0 2024-05-27T07:50:28.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.362+0000 7f5198d48b80 1 client.16341 shutdown 2024-05-27T07:50:28.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.362+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 msgr2=0x55fd8761d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:28.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.362+0000 7f5198d48b80 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761d520 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5174009fd0 tx=0x7f5174007a90 comp rx=0 tx=0).stop 2024-05-27T07:50:28.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.363+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 shutdown_connections 2024-05-27T07:50:28.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.363+0000 7f5198d48b80 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fd876257d0 0x55fd87699760 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.363+0000 7f5198d48b80 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd8761d110 0x55fd8761dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.363+0000 7f5198d48b80 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fd87482e40 0x55fd8761d520 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.363+0000 7f5198d48b80 1 --2- 172.21.15.90:0/189340273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd8756e9f0 0x55fd8761da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.363+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 >> 172.21.15.90:0/189340273 conn(0x55fd8751c390 msgr2=0x55fd87602e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:28.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.363+0000 7f5171600700 0 client.16341 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:50:28.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.364+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 shutdown_connections 2024-05-27T07:50:28.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.364+0000 7f5198d48b80 1 -- 172.21.15.90:0/189340273 wait complete. 2024-05-27T07:50:28.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.365+0000 7f5198d48b80 20 client.16341 trim_cache size 0 max 16384 2024-05-27T07:50:28.366 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:50:28.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.366+0000 7f518a200700 1 -- 172.21.15.90:0/312058814 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f51680b2b20 msgr2=0x7f51680b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:28.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.366+0000 7f518a200700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f51680b2b20 0x7f51680b5030 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55fd875fea20 tx=0x7f517400b7f0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.366+0000 7f518a200700 1 -- 172.21.15.90:0/312058814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5184076670 msgr2=0x7f5184107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:28.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.366+0000 7f518a200700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5184076670 0x7f5184107510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f516c00d950 tx=0x7f516c00dc60 comp rx=0 tx=0).stop 2024-05-27T07:50:28.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.367+0000 7f518a200700 1 -- 172.21.15.90:0/312058814 shutdown_connections 2024-05-27T07:50:28.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.367+0000 7f518a200700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5184110b90 0x7f5184107a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.367+0000 7f518a200700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f51680b2b20 0x7f51680b5030 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.367+0000 7f518a200700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5184075d20 0x7f51841104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.367+0000 7f518a200700 1 --2- 172.21.15.90:0/312058814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5184076670 0x7f5184107510 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:28.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.367+0000 7f518a200700 1 -- 172.21.15.90:0/312058814 >> 172.21.15.90:0/312058814 conn(0x7f5184100030 msgr2=0x7f5184100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:28.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.368+0000 7f518a200700 1 -- 172.21.15.90:0/312058814 shutdown_connections 2024-05-27T07:50:28.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:28.368+0000 7f518a200700 1 -- 172.21.15.90:0/312058814 wait complete. 2024-05-27T07:50:28.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:28 smithi136 ceph-mon[35324]: pgmap v128: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.6 MiB/s wr, 40 op/s 2024-05-27T07:50:28.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:28 smithi129 ceph-mon[35254]: pgmap v128: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 9.6 MiB/s wr, 40 op/s 2024-05-27T07:50:28.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 10.0 s, 973.8 tps, lat 102.672 ms stddev 35.766 2024-05-27T07:50:29.370 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:50:30.070 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:30.424 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:30 smithi090 ceph-mon[31429]: pgmap v129: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 106 op/s 2024-05-27T07:50:30.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:30 smithi136 ceph-mon[35324]: pgmap v129: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 106 op/s 2024-05-27T07:50:30.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:30 smithi129 ceph-mon[35254]: pgmap v129: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 11 MiB/s wr, 106 op/s 2024-05-27T07:50:31.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:31 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 1. 2024-05-27T07:50:31.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:31 smithi090 systemd[1]: Stopped Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:50:31.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:31 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 2.081s CPU time. 2024-05-27T07:50:31.579 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:31 smithi090 systemd[1]: Starting Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:50:32.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:32 smithi090 ceph-mon[31429]: pgmap v130: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.7 MiB/s wr, 97 op/s 2024-05-27T07:50:32.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:32 smithi136 ceph-mon[35324]: pgmap v130: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.7 MiB/s wr, 97 op/s 2024-05-27T07:50:32.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:32 smithi129 ceph-mon[35254]: pgmap v130: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.7 MiB/s wr, 97 op/s 2024-05-27T07:50:33.171 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:32 smithi090 podman[102516]: 2024-05-27 07:50:32.815662836 +0000 UTC m=+0.058502467 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:50:33.171 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:32 smithi090 podman[102516]: 2024-05-27 07:50:32.965900418 +0000 UTC m=+0.208740072 container create e5cd702d818caf5d7702e74bb1eb61637356cab3fdbfc03eba4cc0f5f34e5525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-27T07:50:33.463 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:33 smithi090 podman[102516]: 2024-05-27 07:50:33.408289604 +0000 UTC m=+0.651129218 container init e5cd702d818caf5d7702e74bb1eb61637356cab3fdbfc03eba4cc0f5f34e5525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:50:33.463 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:33 smithi090 podman[102516]: 2024-05-27 07:50:33.419060319 +0000 UTC m=+0.661899923 container start e5cd702d818caf5d7702e74bb1eb61637356cab3fdbfc03eba4cc0f5f34e5525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:50:33.818 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:33 smithi090 bash[102516]: e5cd702d818caf5d7702e74bb1eb61637356cab3fdbfc03eba4cc0f5f34e5525 2024-05-27T07:50:33.818 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:33 smithi090 systemd[1]: Started Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:50:33.818 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:50:33 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: starting mds.d at 2024-05-27T07:50:33.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 15.0 s, 865.2 tps, lat 109.647 ms stddev 42.582 2024-05-27T07:50:34.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.037+0000 7f7b4b200700 1 -- 172.21.15.90:0/432662660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b44110650 msgr2=0x7f7b44110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:34.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.037+0000 7f7b4b200700 1 --2- 172.21.15.90:0/432662660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b44110650 0x7f7b44110a60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7b300099c0 tx=0x7f7b30009cd0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.037+0000 7f7b4b200700 1 -- 172.21.15.90:0/432662660 shutdown_connections 2024-05-27T07:50:34.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.037+0000 7f7b4b200700 1 --2- 172.21.15.90:0/432662660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b44110fa0 0x7f7b441177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.037+0000 7f7b4b200700 1 --2- 172.21.15.90:0/432662660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b4410b890 0x7f7b4410bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.037+0000 7f7b4b200700 1 --2- 172.21.15.90:0/432662660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b44110650 0x7f7b44110a60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.037+0000 7f7b4b200700 1 -- 172.21.15.90:0/432662660 >> 172.21.15.90:0/432662660 conn(0x7f7b4407a010 msgr2=0x7f7b4407c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:34.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.038+0000 7f7b4b200700 1 -- 172.21.15.90:0/432662660 shutdown_connections 2024-05-27T07:50:34.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.038+0000 7f7b4b200700 1 -- 172.21.15.90:0/432662660 wait complete. 2024-05-27T07:50:34.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.042+0000 7f7b4b200700 1 Processor -- start 2024-05-27T07:50:34.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.042+0000 7f7b4b200700 1 -- start start 2024-05-27T07:50:34.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.044+0000 7f7b4b200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b4410b890 0x7f7b441a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.044+0000 7f7b4b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44110650 0x7f7b441a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.044+0000 7f7b4b200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b44110fa0 0x7f7b441a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.044+0000 7f7b4b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4411aa10 con 0x7f7b44110650 2024-05-27T07:50:34.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.044+0000 7f7b4b200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4411a890 con 0x7f7b44110fa0 2024-05-27T07:50:34.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.044+0000 7f7b4b200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4411ab90 con 0x7f7b4410b890 2024-05-27T07:50:34.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.045+0000 7f7b43e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b4410b890 0x7f7b441a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.045+0000 7f7b43e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b4410b890 0x7f7b441a1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:54510/0 (socket says 172.21.15.90:54510) 2024-05-27T07:50:34.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.045+0000 7f7b43e00700 1 -- 172.21.15.90:0/3338941118 learned_addr learned my addr 172.21.15.90:0/3338941118 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:34.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.045+0000 7f7b49200700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b44110fa0 0x7f7b441a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.046+0000 7f7b43400700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44110650 0x7f7b441a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.047+0000 7f7b43e00700 1 -- 172.21.15.90:0/3338941118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b44110fa0 msgr2=0x7f7b441a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:34.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.047+0000 7f7b43e00700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b44110fa0 0x7f7b441a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.047+0000 7f7b43e00700 1 -- 172.21.15.90:0/3338941118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44110650 msgr2=0x7f7b441a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:34.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.047+0000 7f7b43e00700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44110650 0x7f7b441a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.047+0000 7f7b43e00700 1 -- 172.21.15.90:0/3338941118 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b441a62f0 con 0x7f7b4410b890 2024-05-27T07:50:34.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.047+0000 7f7b49200700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b44110fa0 0x7f7b441a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:50:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.047+0000 7f7b43e00700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b4410b890 0x7f7b441a1420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800d980 tx=0x7f7b3800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.048+0000 7f7b40c00700 1 -- 172.21.15.90:0/3338941118 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38017070 con 0x7f7b4410b890 2024-05-27T07:50:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.048+0000 7f7b4b200700 1 -- 172.21.15.90:0/3338941118 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b441a65e0 con 0x7f7b4410b890 2024-05-27T07:50:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.048+0000 7f7b4b200700 1 -- 172.21.15.90:0/3338941118 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b441adf00 con 0x7f7b4410b890 2024-05-27T07:50:34.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.048+0000 7f7b40c00700 1 -- 172.21.15.90:0/3338941118 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b3800fd20 con 0x7f7b4410b890 2024-05-27T07:50:34.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.049+0000 7f7b40c00700 1 -- 172.21.15.90:0/3338941118 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800e3b0 con 0x7f7b4410b890 2024-05-27T07:50:34.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.054+0000 7f7b59d87b80 1 Processor -- start 2024-05-27T07:50:34.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.054+0000 7f7b40c00700 1 -- 172.21.15.90:0/3338941118 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7b3800e550 con 0x7f7b4410b890 2024-05-27T07:50:34.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.055+0000 7f7b59d87b80 1 -- start start 2024-05-27T07:50:34.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.056+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.056+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d649e40 0x55f09d7c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.056+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d7c8e70 0x55f09d7d5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.056+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f09d6f6e40 con 0x55f09d7359f0 2024-05-27T07:50:34.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.056+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f09d6e69e0 con 0x55f09d7c8e70 2024-05-27T07:50:34.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.056+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f09d6eaa20 con 0x55f09d649e40 2024-05-27T07:50:34.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.056+0000 7f7b49200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.057+0000 7f7b49200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43376/0 (socket says 172.21.15.90:43376) 2024-05-27T07:50:34.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.057+0000 7f7b49200700 1 -- 172.21.15.90:0/1541415560 learned_addr learned my addr 172.21.15.90:0/1541415560 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:34.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.057+0000 7f7b40c00700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7b2c0b2af0 0x7f7b2c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.058+0000 7f7b40c00700 1 -- 172.21.15.90:0/3338941118 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f7b380e2580 con 0x7f7b4410b890 2024-05-27T07:50:34.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.058+0000 7f7b43400700 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d7c8e70 0x55f09d7d5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.058+0000 7f7b49200700 1 -- 172.21.15.90:0/1541415560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d649e40 msgr2=0x55f09d7c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:34.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.058+0000 7f7b43e00700 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d649e40 0x55f09d7c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.059+0000 7f7b49200700 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d649e40 0x55f09d7c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.059+0000 7f7b49200700 1 -- 172.21.15.90:0/1541415560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d7c8e70 msgr2=0x55f09d7d5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.059+0000 7f7b49200700 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d7c8e70 0x55f09d7d5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.059+0000 7f7b49200700 1 -- 172.21.15.90:0/1541415560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09d6af420 con 0x55f09d7359f0 2024-05-27T07:50:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.059+0000 7f7b43400700 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d7c8e70 0x55f09d7d5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.059+0000 7f7b43e00700 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d649e40 0x55f09d7c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.060+0000 7f7b49200700 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7c8330 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00a910 tx=0x7f7b3c00ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e296e6c8b7d6bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:34.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.060+0000 7f7b49200700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7b2c0b2af0 0x7f7b2c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.061+0000 7f7b35600700 1 -- 172.21.15.90:0/1541415560 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c01a020 con 0x55f09d7359f0 2024-05-27T07:50:34.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.061+0000 7f7b35600700 1 -- 172.21.15.90:0/1541415560 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b3c010d00 con 0x55f09d7359f0 2024-05-27T07:50:34.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.061+0000 7f7b35600700 1 -- 172.21.15.90:0/1541415560 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c016790 con 0x55f09d7359f0 2024-05-27T07:50:34.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.063+0000 7f7b59d87b80 1 -- 172.21.15.90:0/1541415560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 msgr2=0x55f09d7c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:34.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.063+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7c8330 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00a910 tx=0x7f7b3c00ac20 comp rx=0 tx=0).stop 2024-05-27T07:50:34.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.063+0000 7f7b49200700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7b2c0b2af0 0x7f7b2c0b5000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c01b000 tx=0x7f7b3c016c00 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:34.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.063+0000 7f7b59d87b80 1 -- 172.21.15.90:0/1541415560 shutdown_connections 2024-05-27T07:50:34.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.063+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d7c8e70 0x55f09d7d5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.063+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7c8330 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.063+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/1541415560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d649e40 0x55f09d7c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.063+0000 7f7b59d87b80 1 -- 172.21.15.90:0/1541415560 >> 172.21.15.90:0/1541415560 conn(0x55f09d6e3390 msgr2=0x55f09d7c44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:34.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.064+0000 7f7b59d87b80 1 -- 172.21.15.90:0/1541415560 shutdown_connections 2024-05-27T07:50:34.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.064+0000 7f7b59d87b80 1 -- 172.21.15.90:0/1541415560 wait complete. 2024-05-27T07:50:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.065+0000 7f7b59d87b80 1 -- start start 2024-05-27T07:50:34.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.065+0000 7f7b59d87b80 1 Processor -- start 2024-05-27T07:50:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.066+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7d2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.066+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.066+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 0x55f09d7d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.066+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f09d5ecfa0 con 0x55f09d7359f0 2024-05-27T07:50:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.066+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f09d687760 con 0x55f09d649e40 2024-05-27T07:50:34.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.066+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f09d7d5b80 con 0x55f09d7c8e70 2024-05-27T07:50:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.067+0000 7f7b59d87b80 10 client.0 _ll_register_callbacks cb 0x55f09d690c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.067+0000 7f7b43e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7d2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.067+0000 7f7b43e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7d2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:42732/0 (socket says 172.21.15.90:42732) 2024-05-27T07:50:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.067+0000 7f7b43e00700 1 -- 172.21.15.90:0/2206256812 learned_addr learned my addr 172.21.15.90:0/2206256812 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.067+0000 7f7b43400700 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.068+0000 7f7b43e00700 1 -- 172.21.15.90:0/2206256812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 msgr2=0x55f09d7d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.068+0000 7f7b49200700 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 0x55f09d7d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.068+0000 7f7b43e00700 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 0x55f09d7d30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.068+0000 7f7b43e00700 1 -- 172.21.15.90:0/2206256812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 msgr2=0x55f09d7d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:34.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.068+0000 7f7b43e00700 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.068+0000 7f7b43e00700 1 -- 172.21.15.90:0/2206256812 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09d6368d0 con 0x55f09d649e40 2024-05-27T07:50:34.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.068+0000 7f7b43400700 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:34.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.070+0000 7f7b43e00700 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7d2650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7b38010af0 tx=0x7f7b3804e8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b310b4534f1d9f81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:34.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.070+0000 7f7b2a000700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:50:34.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.071+0000 7f7b2a000700 1 -- 172.21.15.90:0/2206256812 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38020030 con 0x55f09d649e40 2024-05-27T07:50:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.071+0000 7f7b2a000700 1 -- 172.21.15.90:0/2206256812 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9408+0+0 (secure 0 0 0) 0x7f7b38055370 con 0x55f09d649e40 2024-05-27T07:50:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.071+0000 7f7b59d87b80 1 -- 172.21.15.90:0/2206256812 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b3c010d00 con 0x55f09d649e40 2024-05-27T07:50:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.071+0000 7f7b2a000700 1 -- 172.21.15.90:0/2206256812 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7b380a9bb0 con 0x55f09d649e40 2024-05-27T07:50:34.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.071+0000 7f7b59d87b80 10 client.35870 fetch_fsmap learned FSMap version 40 2024-05-27T07:50:34.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.072+0000 7f7b59d87b80 1 -- 172.21.15.90:0/2206256812 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f09d6b7060 con 0x55f09d649e40 2024-05-27T07:50:34.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.072+0000 7f7b2a000700 1 -- 172.21.15.90:0/2206256812 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b3c010d00 con 0x55f09d649e40 2024-05-27T07:50:34.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.072+0000 7f7b2a000700 1 -- 172.21.15.90:0/2206256812 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7f7b380e2160 con 0x55f09d649e40 2024-05-27T07:50:34.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.072+0000 7f7b59d87b80 10 client.35870 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:50:34.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.072+0000 7f7b59d87b80 10 client.35870 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:50:34.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.072+0000 7f7b59d87b80 20 client.35870 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:34.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.072+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55f09d7e2a10 0x55f09d7e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.072+0000 7f7b59d87b80 4 client.35870 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:34.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.072+0000 7f7b59d87b80 1 -- 172.21.15.90:0/2206256812 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f09d6f8d90 con 0x55f09d7e2a10 2024-05-27T07:50:34.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.074+0000 7f7b43400700 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55f09d7e2a10 0x55f09d7e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.075+0000 7f7b2a000700 10 client.35870 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:50:34.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.075+0000 7f7b43400700 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55f09d7e2a10 0x55f09d7e4ec0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d1d6348f44d6d9e server_cookie=61159fa3a0f27b97 in_seq=0 out_seq=0 2024-05-27T07:50:34.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.077+0000 7f7b2a000700 1 -- 172.21.15.90:0/2206256812 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f09d6f8d90 con 0x55f09d7e2a10 2024-05-27T07:50:34.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.077+0000 7f7b2a000700 10 client.35870 handle_command_reply: tid=0 2024-05-27T07:50:34.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.077+0000 7f7b59d87b80 1 client.35870 shutdown 2024-05-27T07:50:34.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 1 -- 172.21.15.90:0/2206256812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 msgr2=0x55f09d7d2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:34.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7d2650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7b38010af0 tx=0x7f7b3804e8f0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 1 -- 172.21.15.90:0/2206256812 shutdown_connections 2024-05-27T07:50:34.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55f09d7e2a10 0x55f09d7e4ec0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 0x55f09d7d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7d2650 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/2206256812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 1 -- 172.21.15.90:0/2206256812 >> 172.21.15.90:0/2206256812 conn(0x55f09d6e3390 msgr2=0x55f09d7c8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:34.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b2a000700 0 client.35870 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:50:34.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 1 -- 172.21.15.90:0/2206256812 shutdown_connections 2024-05-27T07:50:34.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 1 -- 172.21.15.90:0/2206256812 wait complete. 2024-05-27T07:50:34.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.081+0000 7f7b59d87b80 20 client.35870 trim_cache size 0 max 16384 2024-05-27T07:50:34.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.114+0000 7f7b59d87b80 1 Processor -- start 2024-05-27T07:50:34.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.114+0000 7f7b59d87b80 1 -- start start 2024-05-27T07:50:34.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.114+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.114+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7359f0 0x55f09d7d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.114+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7c8e70 0x55f09d7c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.114+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24010850 con 0x55f09d7c8e70 2024-05-27T07:50:34.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.114+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2400e640 con 0x55f09d649e40 2024-05-27T07:50:34.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.114+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b24010180 con 0x55f09d7359f0 2024-05-27T07:50:34.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.115+0000 7f7b49200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7c8e70 0x55f09d7c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.116+0000 7f7b43400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7359f0 0x55f09d7d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.116+0000 7f7b43400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7359f0 0x55f09d7d15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:54534/0 (socket says 172.21.15.90:54534) 2024-05-27T07:50:34.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.116+0000 7f7b43400700 1 -- 172.21.15.90:0/4059989604 learned_addr learned my addr 172.21.15.90:0/4059989604 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:34.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.116+0000 7f7b43e00700 1 --2- 172.21.15.90:0/4059989604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.117+0000 7f7b43e00700 1 -- 172.21.15.90:0/4059989604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7359f0 msgr2=0x55f09d7d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:34.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.117+0000 7f7b43e00700 1 --2- 172.21.15.90:0/4059989604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7359f0 0x55f09d7d15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.117+0000 7f7b43e00700 1 -- 172.21.15.90:0/4059989604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7c8e70 msgr2=0x55f09d7c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:34.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.117+0000 7f7b43e00700 1 --2- 172.21.15.90:0/4059989604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7c8e70 0x55f09d7c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.117+0000 7f7b43e00700 1 -- 172.21.15.90:0/4059989604 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f09d7d0bc0 con 0x55f09d649e40 2024-05-27T07:50:34.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.118+0000 7f7b43e00700 1 --2- 172.21.15.90:0/4059989604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7cde90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b380a9440 tx=0x7f7b380a9100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e1a5a7a2feece9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:34.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.119+0000 7f7b34c00700 1 -- 172.21.15.90:0/4059989604 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b380a9bb0 con 0x55f09d649e40 2024-05-27T07:50:34.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.119+0000 7f7b34c00700 1 -- 172.21.15.90:0/4059989604 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b380e2160 con 0x55f09d649e40 2024-05-27T07:50:34.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.119+0000 7f7b34c00700 1 -- 172.21.15.90:0/4059989604 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b380a9910 con 0x55f09d649e40 2024-05-27T07:50:34.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.122+0000 7f7b59d87b80 1 -- 172.21.15.90:0/4059989604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 msgr2=0x55f09d7cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:34.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.122+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/4059989604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7cde90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b380a9440 tx=0x7f7b380a9100 comp rx=0 tx=0).stop 2024-05-27T07:50:34.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.122+0000 7f7b59d87b80 1 -- 172.21.15.90:0/4059989604 shutdown_connections 2024-05-27T07:50:34.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.122+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/4059989604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7c8e70 0x55f09d7c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.122+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/4059989604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7cde90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.122+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/4059989604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7359f0 0x55f09d7d15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.122+0000 7f7b59d87b80 1 -- 172.21.15.90:0/4059989604 >> 172.21.15.90:0/4059989604 conn(0x55f09d6e3390 msgr2=0x55f09d7c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:34.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.123+0000 7f7b59d87b80 1 -- 172.21.15.90:0/4059989604 shutdown_connections 2024-05-27T07:50:34.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.123+0000 7f7b59d87b80 1 -- 172.21.15.90:0/4059989604 wait complete. 2024-05-27T07:50:34.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.124+0000 7f7b59d87b80 1 -- start start 2024-05-27T07:50:34.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.124+0000 7f7b59d87b80 1 Processor -- start 2024-05-27T07:50:34.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.125+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7db160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.125+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7db6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.125+0000 7f7b59d87b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 0x55f09d7ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.126+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2400f3e0 con 0x55f09d7359f0 2024-05-27T07:50:34.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.126+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2400ed10 con 0x55f09d649e40 2024-05-27T07:50:34.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.126+0000 7f7b59d87b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2400fab0 con 0x55f09d7c8e70 2024-05-27T07:50:34.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.126+0000 7f7b59d87b80 10 client.0 _ll_register_callbacks cb 0x55f09d690c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.126+0000 7f7b43e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7db160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.127+0000 7f7b43400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7db6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.127+0000 7f7b43400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7db6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43420/0 (socket says 172.21.15.90:43420) 2024-05-27T07:50:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.127+0000 7f7b43400700 1 -- 172.21.15.90:0/3327886930 learned_addr learned my addr 172.21.15.90:0/3327886930 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.127+0000 7f7b49200700 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 0x55f09d7ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.128+0000 7f7b49200700 1 -- 172.21.15.90:0/3327886930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 msgr2=0x55f09d7db160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.128+0000 7f7b49200700 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7db160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.128+0000 7f7b49200700 1 -- 172.21.15.90:0/3327886930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 msgr2=0x55f09d7db6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.128+0000 7f7b49200700 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7db6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.128+0000 7f7b49200700 1 -- 172.21.15.90:0/3327886930 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09d7cd270 con 0x55f09d7c8e70 2024-05-27T07:50:34.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.128+0000 7f7b49200700 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 0x55f09d7ccb20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009440 tx=0x7f7b3c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3925301e8c91c07 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:34.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.128+0000 7f7b2a000700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:50:34.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.131+0000 7f7b2a000700 1 -- 172.21.15.90:0/3327886930 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c035060 con 0x55f09d7c8e70 2024-05-27T07:50:34.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.131+0000 7f7b2a000700 1 -- 172.21.15.90:0/3327886930 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f7b3c010c50 con 0x55f09d7c8e70 2024-05-27T07:50:34.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.131+0000 7f7b59d87b80 1 -- 172.21.15.90:0/3327886930 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b380e2160 con 0x55f09d7c8e70 2024-05-27T07:50:34.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.137+0000 7f7b2a000700 1 -- 172.21.15.90:0/3327886930 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7b3c01a040 con 0x55f09d7c8e70 2024-05-27T07:50:34.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.137+0000 7f7b59d87b80 10 client.25927 fetch_fsmap learned FSMap version 40 2024-05-27T07:50:34.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.137+0000 7f7b59d87b80 1 -- 172.21.15.90:0/3327886930 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f09d7da9c0 con 0x55f09d7c8e70 2024-05-27T07:50:34.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.138+0000 7f7b2a000700 1 -- 172.21.15.90:0/3327886930 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b380e2160 con 0x55f09d7c8e70 2024-05-27T07:50:34.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.138+0000 7f7b2a000700 1 -- 172.21.15.90:0/3327886930 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 40) ==== 7757+0+0 (secure 0 0 0) 0x7f7b3c01e940 con 0x55f09d7c8e70 2024-05-27T07:50:34.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.138+0000 7f7b59d87b80 10 client.25927 fetch_fsmap finished waiting for FSMap version 40 2024-05-27T07:50:34.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.138+0000 7f7b59d87b80 10 client.25927 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:50:34.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.138+0000 7f7b59d87b80 20 client.25927 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:34.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.138+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55f09d85e5a0 0x55f09d8609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:34.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.138+0000 7f7b59d87b80 4 client.25927 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:50:34.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.138+0000 7f7b59d87b80 1 -- 172.21.15.90:0/3327886930 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f09d6f2570 con 0x55f09d85e5a0 2024-05-27T07:50:34.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.139+0000 7f7b43e00700 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55f09d85e5a0 0x55f09d8609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:34.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.140+0000 7f7b43e00700 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55f09d85e5a0 0x55f09d8609c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2534cc9fc5f96e11 server_cookie=fb9d636f4564a0f6 in_seq=0 out_seq=0 2024-05-27T07:50:34.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.140+0000 7f7b2a000700 10 client.25927 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:50:34.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.141+0000 7f7b2a000700 1 -- 172.21.15.90:0/3327886930 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f09d6f2570 con 0x55f09d85e5a0 2024-05-27T07:50:34.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.141+0000 7f7b2a000700 10 client.25927 handle_command_reply: tid=0 2024-05-27T07:50:34.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.141+0000 7f7b59d87b80 1 client.25927 shutdown 2024-05-27T07:50:34.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.141+0000 7f7b59d87b80 1 -- 172.21.15.90:0/3327886930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 msgr2=0x55f09d7ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:34.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.142+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 0x55f09d7ccb20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009440 tx=0x7f7b3c019000 comp rx=0 tx=0).stop 2024-05-27T07:50:34.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.142+0000 7f7b59d87b80 1 -- 172.21.15.90:0/3327886930 shutdown_connections 2024-05-27T07:50:34.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.142+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55f09d85e5a0 0x55f09d8609c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.142+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f09d7c8e70 0x55f09d7ccb20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.142+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f09d649e40 0x55f09d7db160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.142+0000 7f7b59d87b80 1 --2- 172.21.15.90:0/3327886930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f09d7359f0 0x55f09d7db6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.142+0000 7f7b2a000700 0 client.25927 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:50:34.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.142+0000 7f7b59d87b80 1 -- 172.21.15.90:0/3327886930 >> 172.21.15.90:0/3327886930 conn(0x55f09d6e3390 msgr2=0x55f09d7c9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:34.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.143+0000 7f7b59d87b80 1 -- 172.21.15.90:0/3327886930 shutdown_connections 2024-05-27T07:50:34.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.143+0000 7f7b59d87b80 1 -- 172.21.15.90:0/3327886930 wait complete. 2024-05-27T07:50:34.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.143+0000 7f7b59d87b80 20 client.25927 trim_cache size 0 max 16384 2024-05-27T07:50:34.145 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:50:34.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.146+0000 7f7b4b200700 1 -- 172.21.15.90:0/3338941118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7b2c0b2af0 msgr2=0x7f7b2c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:34.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.146+0000 7f7b4b200700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7b2c0b2af0 0x7f7b2c0b5000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c01b000 tx=0x7f7b3c016c00 comp rx=0 tx=0).stop 2024-05-27T07:50:34.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.146+0000 7f7b4b200700 1 -- 172.21.15.90:0/3338941118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b4410b890 msgr2=0x7f7b441a1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:34.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.146+0000 7f7b4b200700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b4410b890 0x7f7b441a1420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800d980 tx=0x7f7b3800dc90 comp rx=0 tx=0).stop 2024-05-27T07:50:34.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.147+0000 7f7b4b200700 1 -- 172.21.15.90:0/3338941118 shutdown_connections 2024-05-27T07:50:34.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.147+0000 7f7b4b200700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7b2c0b2af0 0x7f7b2c0b5000 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.147+0000 7f7b4b200700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b44110fa0 0x7f7b441a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.147+0000 7f7b4b200700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b4410b890 0x7f7b441a1420 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.147+0000 7f7b4b200700 1 --2- 172.21.15.90:0/3338941118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b44110650 0x7f7b441a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:34.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.147+0000 7f7b4b200700 1 -- 172.21.15.90:0/3338941118 >> 172.21.15.90:0/3338941118 conn(0x7f7b4407a010 msgr2=0x7f7b4407c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:34.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.148+0000 7f7b4b200700 1 -- 172.21.15.90:0/3338941118 shutdown_connections 2024-05-27T07:50:34.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:34.148+0000 7f7b4b200700 1 -- 172.21.15.90:0/3338941118 wait complete. 2024-05-27T07:50:34.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:34 smithi090 ceph-mon[31429]: pgmap v131: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 170 op/s 2024-05-27T07:50:34.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:34 smithi090 ceph-mon[31429]: daemon mds.d restarted 2024-05-27T07:50:34.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:34 smithi136 ceph-mon[35324]: pgmap v131: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 170 op/s 2024-05-27T07:50:34.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:34 smithi136 ceph-mon[35324]: daemon mds.d restarted 2024-05-27T07:50:34.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:34 smithi129 ceph-mon[35254]: pgmap v131: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.9 MiB/s wr, 170 op/s 2024-05-27T07:50:34.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:34 smithi129 ceph-mon[35254]: daemon mds.d restarted 2024-05-27T07:50:35.076 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:50:35.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:35 smithi090 ceph-mon[31429]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T07:50:35.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:35 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6836/2432811315,v1:172.21.15.90:6837/2432811315] up:boot 2024-05-27T07:50:35.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:35 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:50:35.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:35 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:50:35.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:35 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:50:35.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:35 smithi136 ceph-mon[35324]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T07:50:35.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:35 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6836/2432811315,v1:172.21.15.90:6837/2432811315] up:boot 2024-05-27T07:50:35.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:35 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:50:35.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:35 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:50:35.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:35 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:50:35.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:35 smithi129 ceph-mon[35254]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T07:50:35.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:35 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6836/2432811315,v1:172.21.15.90:6837/2432811315] up:boot 2024-05-27T07:50:35.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:35 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:50:35.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:35 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:50:35.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:35 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:50:35.757 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:36.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:36 smithi090 ceph-mon[31429]: pgmap v133: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.9 MiB/s wr, 165 op/s 2024-05-27T07:50:36.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:36 smithi136 ceph-mon[35324]: pgmap v133: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.9 MiB/s wr, 165 op/s 2024-05-27T07:50:36.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:36 smithi129 ceph-mon[35254]: pgmap v133: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.9 MiB/s wr, 165 op/s 2024-05-27T07:50:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.665+0000 7fc93a800640 1 -- 172.21.15.90:0/3440191210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc93410dbb0 msgr2=0x7fc93410ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.665+0000 7fc93a800640 1 --2- 172.21.15.90:0/3440191210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc93410dbb0 0x7fc93410ffd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc92402a480 comp rx=0 tx=0).stop 2024-05-27T07:50:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.665+0000 7fc93a800640 1 -- 172.21.15.90:0/3440191210 shutdown_connections 2024-05-27T07:50:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.665+0000 7fc93a800640 1 --2- 172.21.15.90:0/3440191210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc934110510 0x7fc934112930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.665+0000 7fc93a800640 1 --2- 172.21.15.90:0/3440191210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc93410dbb0 0x7fc93410ffd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.665+0000 7fc93a800640 1 --2- 172.21.15.90:0/3440191210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc934075420 0x7fc934075830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.665+0000 7fc93a800640 1 -- 172.21.15.90:0/3440191210 >> 172.21.15.90:0/3440191210 conn(0x7fc9340793c0 msgr2=0x7fc934079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.665+0000 7fc93a800640 1 -- 172.21.15.90:0/3440191210 shutdown_connections 2024-05-27T07:50:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.665+0000 7fc93a800640 1 -- 172.21.15.90:0/3440191210 wait complete. 2024-05-27T07:50:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.666+0000 7fc93a800640 1 Processor -- start 2024-05-27T07:50:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc93a800640 1 -- start start 2024-05-27T07:50:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc93a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc934075420 0x7fc9341a2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc93a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc93410dbb0 0x7fc9341a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc93a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc934110510 0x7fc93419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc93a800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc934115f20 con 0x7fc934110510 2024-05-27T07:50:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc93a800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc934115da0 con 0x7fc93410dbb0 2024-05-27T07:50:36.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc93a800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341160a0 con 0x7fc934075420 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc933e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc934110510 0x7fc93419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc933e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc934110510 0x7fc93419c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43436/0 (socket says 172.21.15.90:43436) 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc933e00640 1 -- 172.21.15.90:0/2862408163 learned_addr learned my addr 172.21.15.90:0/2862408163 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc932a00640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc93410dbb0 0x7fc9341a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.667+0000 7fc933400640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc934075420 0x7fc9341a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc933e00640 1 -- 172.21.15.90:0/2862408163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc934075420 msgr2=0x7fc9341a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc933e00640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc934075420 0x7fc9341a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc933e00640 1 -- 172.21.15.90:0/2862408163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc93410dbb0 msgr2=0x7fc9341a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc933e00640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc93410dbb0 0x7fc9341a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc933e00640 1 -- 172.21.15.90:0/2862408163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93419cf40 con 0x7fc934110510 2024-05-27T07:50:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc933e00640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc934110510 0x7fc93419c860 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc92000d750 tx=0x7fc92000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc92be00640 1 -- 172.21.15.90:0/2862408163 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc920017070 con 0x7fc934110510 2024-05-27T07:50:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc92be00640 1 -- 172.21.15.90:0/2862408163 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9200054b0 con 0x7fc934110510 2024-05-27T07:50:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc93a800640 1 -- 172.21.15.90:0/2862408163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93419d230 con 0x7fc934110510 2024-05-27T07:50:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc92be00640 1 -- 172.21.15.90:0/2862408163 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc920004b90 con 0x7fc934110510 2024-05-27T07:50:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.668+0000 7fc93a800640 1 -- 172.21.15.90:0/2862408163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9341a9480 con 0x7fc934110510 2024-05-27T07:50:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.672+0000 7fc9417b7740 1 Processor -- start 2024-05-27T07:50:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.672+0000 7fc92be00640 1 -- 172.21.15.90:0/2862408163 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc920004d30 con 0x7fc934110510 2024-05-27T07:50:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.672+0000 7fc9417b7740 1 -- start start 2024-05-27T07:50:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.672+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59301b0 0x5635f5cd2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.672+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59072f0 0x5635f5cd2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f590f4d0 0x5635f5cd32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc9417b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5635f5c43b90 con 0x5635f59301b0 2024-05-27T07:50:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc9417b7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635f5c33730 con 0x5635f59072f0 2024-05-27T07:50:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc9417b7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5635f5c37770 con 0x5635f590f4d0 2024-05-27T07:50:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc932a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59301b0 0x5635f5cd2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc932a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59301b0 0x5635f5cd2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43450/0 (socket says 172.21.15.90:43450) 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc932a00640 1 -- 172.21.15.90:0/3779413592 learned_addr learned my addr 172.21.15.90:0/3779413592 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc933400640 1 --2- 172.21.15.90:0/3779413592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59072f0 0x5635f5cd2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc932a00640 1 -- 172.21.15.90:0/3779413592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f590f4d0 msgr2=0x5635f5cd32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc932a00640 1 --2- 172.21.15.90:0/3779413592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f590f4d0 0x5635f5cd32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc932a00640 1 -- 172.21.15.90:0/3779413592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59072f0 msgr2=0x5635f5cd2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc932a00640 1 --2- 172.21.15.90:0/3779413592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59072f0 0x5635f5cd2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc932a00640 1 -- 172.21.15.90:0/3779413592 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f5bcc8b0 con 0x5635f59301b0 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.673+0000 7fc932a00640 1 --2- 172.21.15.90:0/3779413592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59301b0 0x5635f5cd2860 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc924000c00 tx=0x7fc924005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1927b82098ea109 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.674+0000 7fc928c00640 1 -- 172.21.15.90:0/3779413592 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924035050 con 0x5635f59301b0 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.674+0000 7fc928c00640 1 -- 172.21.15.90:0/3779413592 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc924032e40 con 0x5635f59301b0 2024-05-27T07:50:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.674+0000 7fc928c00640 1 -- 172.21.15.90:0/3779413592 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9240055d0 con 0x5635f59301b0 2024-05-27T07:50:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.674+0000 7fc92be00640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9180b29d0 0x7fc9180b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.674+0000 7fc92be00640 1 -- 172.21.15.90:0/2862408163 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fc9200e23b0 con 0x7fc934110510 2024-05-27T07:50:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc9417b7740 1 -- 172.21.15.90:0/3779413592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59301b0 msgr2=0x5635f5cd2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc9417b7740 1 --2- 172.21.15.90:0/3779413592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59301b0 0x5635f5cd2860 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc924000c00 tx=0x7fc924005e50 comp rx=0 tx=0).stop 2024-05-27T07:50:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc933400640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9180b29d0 0x7fc9180b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc9417b7740 1 -- 172.21.15.90:0/3779413592 shutdown_connections 2024-05-27T07:50:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc9417b7740 1 --2- 172.21.15.90:0/3779413592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59072f0 0x5635f5cd2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc9417b7740 1 --2- 172.21.15.90:0/3779413592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f590f4d0 0x5635f5cd32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc9417b7740 1 --2- 172.21.15.90:0/3779413592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59301b0 0x5635f5cd2860 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc9417b7740 1 -- 172.21.15.90:0/3779413592 >> 172.21.15.90:0/3779413592 conn(0x5635f5924300 msgr2=0x5635f5ccf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc9417b7740 1 -- 172.21.15.90:0/3779413592 shutdown_connections 2024-05-27T07:50:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc9417b7740 1 -- 172.21.15.90:0/3779413592 wait complete. 2024-05-27T07:50:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc933400640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9180b29d0 0x7fc9180b4ee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc92c005c40 tx=0x7fc92c005bd0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.675+0000 7fc9417b7740 1 -- start start 2024-05-27T07:50:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.676+0000 7fc9417b7740 1 Processor -- start 2024-05-27T07:50:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.676+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59072f0 0x5635f5cdca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.676+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cdcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.676+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59301b0 0x5635f5ce8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.676+0000 7fc9417b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5635f5919790 con 0x5635f590f4d0 2024-05-27T07:50:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.676+0000 7fc9417b7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635f5bdbbc0 con 0x5635f59072f0 2024-05-27T07:50:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.676+0000 7fc9417b7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5635f5cdf930 con 0x5635f59301b0 2024-05-27T07:50:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.676+0000 7fc9417b7740 10 client.0 _ll_register_callbacks cb 0x5635f5bb9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.676+0000 7fc932a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cdcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc933400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59072f0 0x5635f5cdca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc932a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cdcfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43456/0 (socket says 172.21.15.90:43456) 2024-05-27T07:50:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc932a00640 1 -- 172.21.15.90:0/720254488 learned_addr learned my addr 172.21.15.90:0/720254488 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc933e00640 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59301b0 0x5635f5ce8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc932a00640 1 -- 172.21.15.90:0/720254488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59301b0 msgr2=0x5635f5ce8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc932a00640 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59301b0 0x5635f5ce8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc932a00640 1 -- 172.21.15.90:0/720254488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59072f0 msgr2=0x5635f5cdca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc932a00640 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59072f0 0x5635f5cdca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc932a00640 1 -- 172.21.15.90:0/720254488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635f5948210 con 0x5635f590f4d0 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc932a00640 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cdcfa0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc92402a9a0 tx=0x7fc9240316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaebb8e287d385a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc91d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.677+0000 7fc91d600640 1 -- 172.21.15.90:0/720254488 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92404d060 con 0x5635f590f4d0 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc9417b7740 1 -- 172.21.15.90:0/720254488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5635f5c45ae0 con 0x5635f590f4d0 2024-05-27T07:50:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc91d600640 1 -- 172.21.15.90:0/720254488 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fc924004440 con 0x5635f590f4d0 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc91d600640 1 -- 172.21.15.90:0/720254488 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc924054050 con 0x5635f590f4d0 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc9417b7740 10 client.16371 fetch_fsmap learned FSMap version 42 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc9417b7740 1 -- 172.21.15.90:0/720254488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5635f5a14620 con 0x5635f590f4d0 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc91d600640 1 -- 172.21.15.90:0/720254488 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5635f5c45ae0 con 0x5635f590f4d0 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc91d600640 1 -- 172.21.15.90:0/720254488 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fc924004880 con 0x5635f590f4d0 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc9417b7740 10 client.16371 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc9417b7740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc9417b7740 20 client.16371 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc9417b7740 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5635f58eed80 0x5635f5ceead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc9417b7740 4 client.16371 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.678+0000 7fc9417b7740 1 -- 172.21.15.90:0/720254488 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635f5c3f2c0 con 0x5635f58eed80 2024-05-27T07:50:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.679+0000 7fc933400640 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5635f58eed80 0x5635f5ceead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.680+0000 7fc933400640 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5635f58eed80 0x5635f5ceead0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881c19e839638c38 server_cookie=39d9f075bf0ca2c0 in_seq=0 out_seq=0 2024-05-27T07:50:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.680+0000 7fc91d600640 10 client.16371 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc91d600640 1 -- 172.21.15.90:0/720254488 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5635f5c3f2c0 con 0x5635f58eed80 2024-05-27T07:50:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc91d600640 10 client.16371 handle_command_reply: tid=0 2024-05-27T07:50:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 client.16371 shutdown 2024-05-27T07:50:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 -- 172.21.15.90:0/720254488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 msgr2=0x5635f5cdcfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cdcfa0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc92402a9a0 tx=0x7fc9240316b0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 -- 172.21.15.90:0/720254488 shutdown_connections 2024-05-27T07:50:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5635f58eed80 0x5635f5ceead0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59301b0 0x5635f5ce8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc91d600640 0 client.16371 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cdcfa0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 --2- 172.21.15.90:0/720254488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59072f0 0x5635f5cdca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 -- 172.21.15.90:0/720254488 >> 172.21.15.90:0/720254488 conn(0x5635f5924300 msgr2=0x5635f5cd2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 -- 172.21.15.90:0/720254488 shutdown_connections 2024-05-27T07:50:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 1 -- 172.21.15.90:0/720254488 wait complete. 2024-05-27T07:50:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.681+0000 7fc9417b7740 20 client.16371 trim_cache size 0 max 16384 2024-05-27T07:50:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.697+0000 7fc9417b7740 1 Processor -- start 2024-05-27T07:50:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.697+0000 7fc9417b7740 1 -- start start 2024-05-27T07:50:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.697+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59072f0 0x5635f5cdcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.697+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cf1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.697+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59301b0 0x5635f5cf2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.697+0000 7fc9417b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341b4e90 con 0x5635f590f4d0 2024-05-27T07:50:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.697+0000 7fc9417b7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341b2c80 con 0x5635f59301b0 2024-05-27T07:50:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.697+0000 7fc9417b7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341b47c0 con 0x5635f59072f0 2024-05-27T07:50:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.698+0000 7fc932a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cf1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.698+0000 7fc932a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cf1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43458/0 (socket says 172.21.15.90:43458) 2024-05-27T07:50:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.698+0000 7fc932a00640 1 -- 172.21.15.90:0/1529122981 learned_addr learned my addr 172.21.15.90:0/1529122981 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.698+0000 7fc932a00640 1 -- 172.21.15.90:0/1529122981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59072f0 msgr2=0x5635f5cdcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.698+0000 7fc932a00640 1 --2- 172.21.15.90:0/1529122981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59072f0 0x5635f5cdcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.698+0000 7fc932a00640 1 -- 172.21.15.90:0/1529122981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59301b0 msgr2=0x5635f5cf2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.698+0000 7fc932a00640 1 --2- 172.21.15.90:0/1529122981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59301b0 0x5635f5cf2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.698+0000 7fc932a00640 1 -- 172.21.15.90:0/1529122981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635f5cd3c40 con 0x5635f590f4d0 2024-05-27T07:50:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.698+0000 7fc932a00640 1 --2- 172.21.15.90:0/1529122981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cf1b00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc924032ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36987de0d07f4157 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc91fe00640 1 -- 172.21.15.90:0/1529122981 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92404d070 con 0x5635f590f4d0 2024-05-27T07:50:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc91fe00640 1 -- 172.21.15.90:0/1529122981 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc924004440 con 0x5635f590f4d0 2024-05-27T07:50:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc91fe00640 1 -- 172.21.15.90:0/1529122981 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924035030 con 0x5635f590f4d0 2024-05-27T07:50:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc9417b7740 1 -- 172.21.15.90:0/1529122981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 msgr2=0x5635f5cf1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc9417b7740 1 --2- 172.21.15.90:0/1529122981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cf1b00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc924032ca0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc9417b7740 1 -- 172.21.15.90:0/1529122981 shutdown_connections 2024-05-27T07:50:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc9417b7740 1 --2- 172.21.15.90:0/1529122981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f59301b0 0x5635f5cf2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc9417b7740 1 --2- 172.21.15.90:0/1529122981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f590f4d0 0x5635f5cf1b00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc9417b7740 1 --2- 172.21.15.90:0/1529122981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59072f0 0x5635f5cdcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc9417b7740 1 -- 172.21.15.90:0/1529122981 >> 172.21.15.90:0/1529122981 conn(0x5635f5924300 msgr2=0x5635f5cd7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc9417b7740 1 -- 172.21.15.90:0/1529122981 shutdown_connections 2024-05-27T07:50:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.699+0000 7fc9417b7740 1 -- 172.21.15.90:0/1529122981 wait complete. 2024-05-27T07:50:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.700+0000 7fc9417b7740 1 -- start start 2024-05-27T07:50:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.700+0000 7fc9417b7740 1 Processor -- start 2024-05-27T07:50:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.700+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59072f0 0x5635f5cf12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.700+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f590f4d0 0x5635f5cf1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.700+0000 7fc9417b7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59301b0 0x5635f5d4dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.700+0000 7fc9417b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341b3a20 con 0x5635f59072f0 2024-05-27T07:50:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.700+0000 7fc9417b7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341b3350 con 0x5635f590f4d0 2024-05-27T07:50:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.700+0000 7fc9417b7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9341b40f0 con 0x5635f59301b0 2024-05-27T07:50:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.700+0000 7fc9417b7740 10 client.0 _ll_register_callbacks cb 0x7fc924032d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc933400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59072f0 0x5635f5cf12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc933400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59072f0 0x5635f5cf12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43460/0 (socket says 172.21.15.90:43460) 2024-05-27T07:50:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc933400640 1 -- 172.21.15.90:0/907690082 learned_addr learned my addr 172.21.15.90:0/907690082 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc933400640 1 -- 172.21.15.90:0/907690082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59301b0 msgr2=0x5635f5d4dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc933400640 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59301b0 0x5635f5d4dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc933400640 1 -- 172.21.15.90:0/907690082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f590f4d0 msgr2=0x5635f5cf1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc932a00640 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f590f4d0 0x5635f5cf1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc933400640 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f590f4d0 0x5635f5cf1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc933400640 1 -- 172.21.15.90:0/907690082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635f5cd3520 con 0x5635f59072f0 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc933400640 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59072f0 0x5635f5cf12f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc93419db90 tx=0x7fc92c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b372027db63df7ae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.701+0000 7fc91d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc91d600640 1 -- 172.21.15.90:0/907690082 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0129d0 con 0x5635f59072f0 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc91d600640 1 -- 172.21.15.90:0/907690082 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fc92c012b70 con 0x5635f59072f0 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc9417b7740 1 -- 172.21.15.90:0/907690082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc924004440 con 0x5635f59072f0 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc91d600640 1 -- 172.21.15.90:0/907690082 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc92c039050 con 0x5635f59072f0 2024-05-27T07:50:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc9417b7740 10 client.16377 fetch_fsmap learned FSMap version 42 2024-05-27T07:50:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc9417b7740 1 -- 172.21.15.90:0/907690082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5635f5d56810 con 0x5635f59072f0 2024-05-27T07:50:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc91d600640 1 -- 172.21.15.90:0/907690082 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc924004440 con 0x5635f59072f0 2024-05-27T07:50:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc91d600640 1 -- 172.21.15.90:0/907690082 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fc92c00f050 con 0x5635f59072f0 2024-05-27T07:50:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc9417b7740 10 client.16377 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:50:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc9417b7740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc9417b7740 20 client.16377 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc9417b7740 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5635f58c41c0 0x5635f5d59000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc9417b7740 4 client.16377 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:50:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.702+0000 7fc9417b7740 1 -- 172.21.15.90:0/907690082 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635f5bb94b0 con 0x5635f58c41c0 2024-05-27T07:50:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.703+0000 7fc932a00640 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5635f58c41c0 0x5635f5d59000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.704+0000 7fc932a00640 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5635f58c41c0 0x5635f5d59000 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61a542b64aea1703 server_cookie=681440e9f31434a5 in_seq=0 out_seq=0 2024-05-27T07:50:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.704+0000 7fc91d600640 10 client.16377 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.848+0000 7fc91d600640 1 -- 172.21.15.90:0/907690082 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5635f5bb94b0 con 0x5635f58c41c0 2024-05-27T07:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.848+0000 7fc91d600640 10 client.16377 handle_command_reply: tid=0 2024-05-27T07:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.848+0000 7fc9417b7740 1 client.16377 shutdown 2024-05-27T07:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.848+0000 7fc9417b7740 1 -- 172.21.15.90:0/907690082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59072f0 msgr2=0x5635f5cf12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:36.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.848+0000 7fc9417b7740 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59072f0 0x5635f5cf12f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc93419db90 tx=0x7fc92c004ab0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.849+0000 7fc9417b7740 1 -- 172.21.15.90:0/907690082 shutdown_connections 2024-05-27T07:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.849+0000 7fc9417b7740 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5635f58c41c0 0x5635f5d59000 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.849+0000 7fc9417b7740 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635f590f4d0 0x5635f5cf1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.849+0000 7fc9417b7740 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5635f59301b0 0x5635f5d4dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.849+0000 7fc9417b7740 1 --2- 172.21.15.90:0/907690082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5635f59072f0 0x5635f5cf12f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.849+0000 7fc91d600640 0 client.16377 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:36.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.849+0000 7fc9417b7740 1 -- 172.21.15.90:0/907690082 >> 172.21.15.90:0/907690082 conn(0x5635f5924300 msgr2=0x5635f5cd7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.850+0000 7fc9417b7740 1 -- 172.21.15.90:0/907690082 shutdown_connections 2024-05-27T07:50:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.850+0000 7fc9417b7740 1 -- 172.21.15.90:0/907690082 wait complete. 2024-05-27T07:50:36.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.850+0000 7fc9417b7740 20 client.16377 trim_cache size 0 max 16384 2024-05-27T07:50:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:50:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:50:36.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:50:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.852+0000 7fc93a800640 1 -- 172.21.15.90:0/2862408163 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9180b29d0 msgr2=0x7fc9180b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.852+0000 7fc93a800640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9180b29d0 0x7fc9180b4ee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc92c005c40 tx=0x7fc92c005bd0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.852+0000 7fc93a800640 1 -- 172.21.15.90:0/2862408163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc934110510 msgr2=0x7fc93419c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.852+0000 7fc93a800640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc934110510 0x7fc93419c860 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc92000d750 tx=0x7fc92000dc10 comp rx=0 tx=0).stop 2024-05-27T07:50:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.852+0000 7fc93a800640 1 -- 172.21.15.90:0/2862408163 shutdown_connections 2024-05-27T07:50:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.852+0000 7fc93a800640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9180b29d0 0x7fc9180b4ee0 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc92c005c40 tx=0x7fc92c005bd0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.852+0000 7fc93a800640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc93410dbb0 0x7fc9341a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.852+0000 7fc93a800640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc934110510 0x7fc93419c860 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.852+0000 7fc93a800640 1 --2- 172.21.15.90:0/2862408163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc934075420 0x7fc9341a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.852+0000 7fc93a800640 1 -- 172.21.15.90:0/2862408163 >> 172.21.15.90:0/2862408163 conn(0x7fc9340793c0 msgr2=0x7fc9341023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.853+0000 7fc93a800640 1 -- 172.21.15.90:0/2862408163 shutdown_connections 2024-05-27T07:50:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:36.853+0000 7fc93a800640 1 -- 172.21.15.90:0/2862408163 wait complete. 2024-05-27T07:50:38.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:38 smithi090 ceph-mon[31429]: pgmap v134: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.9 MiB/s wr, 165 op/s 2024-05-27T07:50:38.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:38 smithi136 ceph-mon[35324]: pgmap v134: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.9 MiB/s wr, 165 op/s 2024-05-27T07:50:38.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:38 smithi129 ceph-mon[35254]: pgmap v134: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.9 MiB/s wr, 165 op/s 2024-05-27T07:50:38.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 20.0 s, 481.4 tps, lat 214.027 ms stddev 487.087 2024-05-27T07:50:39.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.331+0000 7fadb2800700 1 -- 172.21.15.90:0/246146877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadac10b500 msgr2=0x7fadac118c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.331+0000 7fadb2800700 1 --2- 172.21.15.90:0/246146877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadac10b500 0x7fadac118c20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0098f0 tx=0x7fad9c009c00 comp rx=0 tx=0).stop 2024-05-27T07:50:39.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.332+0000 7fadb2800700 1 -- 172.21.15.90:0/246146877 shutdown_connections 2024-05-27T07:50:39.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.332+0000 7fadb2800700 1 --2- 172.21.15.90:0/246146877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadac1192b0 0x7fadac11b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.332+0000 7fadb2800700 1 --2- 172.21.15.90:0/246146877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadac10abb0 0x7fadac10afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.332+0000 7fadb2800700 1 --2- 172.21.15.90:0/246146877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadac10b500 0x7fadac118c20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.332+0000 7fadb2800700 1 -- 172.21.15.90:0/246146877 >> 172.21.15.90:0/246146877 conn(0x7fadac06f4c0 msgr2=0x7fadac06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.332+0000 7fadb2800700 1 -- 172.21.15.90:0/246146877 shutdown_connections 2024-05-27T07:50:39.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.333+0000 7fadb2800700 1 -- 172.21.15.90:0/246146877 wait complete. 2024-05-27T07:50:39.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.334+0000 7fadb2800700 1 Processor -- start 2024-05-27T07:50:39.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.334+0000 7fadb2800700 1 -- start start 2024-05-27T07:50:39.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.335+0000 7fadb2800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadac10abb0 0x7fadac116200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.335+0000 7fadb2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadac1192b0 0x7fadac116740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.335+0000 7fadb2800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadac116cd0 0x7fadac1b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.335+0000 7fadb2800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fadac11ed30 con 0x7fadac1192b0 2024-05-27T07:50:39.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.335+0000 7fadb2800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fadac11ebb0 con 0x7fadac10abb0 2024-05-27T07:50:39.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.335+0000 7fadb2800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fadac11eeb0 con 0x7fadac116cd0 2024-05-27T07:50:39.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.335+0000 7fadaaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadac1192b0 0x7fadac116740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.336+0000 7fadabe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadac116cd0 0x7fadac1b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.336+0000 7fadabe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadac116cd0 0x7fadac1b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:47624/0 (socket says 172.21.15.90:47624) 2024-05-27T07:50:39.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.336+0000 7fadabe00700 1 -- 172.21.15.90:0/3226461714 learned_addr learned my addr 172.21.15.90:0/3226461714 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:39.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.337+0000 7fadab400700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadac10abb0 0x7fadac116200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.337+0000 7fadaaa00700 1 -- 172.21.15.90:0/3226461714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadac116cd0 msgr2=0x7fadac1b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.337+0000 7fadaaa00700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadac116cd0 0x7fadac1b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.337+0000 7fadaaa00700 1 -- 172.21.15.90:0/3226461714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadac10abb0 msgr2=0x7fadac116200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.337+0000 7fadaaa00700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadac10abb0 0x7fadac116200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.337+0000 7fadaaa00700 1 -- 172.21.15.90:0/3226461714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac1b6280 con 0x7fadac1192b0 2024-05-27T07:50:39.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.337+0000 7fadab400700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadac10abb0 0x7fadac116200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:39.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.338+0000 7fadaaa00700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadac1192b0 0x7fadac116740 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fadac10ba90 tx=0x7fad9c00f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.340+0000 7fada3e00700 1 -- 172.21.15.90:0/3226461714 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00b6e0 con 0x7fadac1192b0 2024-05-27T07:50:39.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.340+0000 7fada3e00700 1 -- 172.21.15.90:0/3226461714 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad9c005040 con 0x7fadac1192b0 2024-05-27T07:50:39.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.340+0000 7fadb2800700 1 -- 172.21.15.90:0/3226461714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac1b6510 con 0x7fadac1192b0 2024-05-27T07:50:39.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.341+0000 7fada3e00700 1 -- 172.21.15.90:0/3226461714 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c003900 con 0x7fadac1192b0 2024-05-27T07:50:39.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.341+0000 7fadb2800700 1 -- 172.21.15.90:0/3226461714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac1b6970 con 0x7fadac1192b0 2024-05-27T07:50:39.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.347+0000 7fada3e00700 1 -- 172.21.15.90:0/3226461714 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fad9c00b880 con 0x7fadac1192b0 2024-05-27T07:50:39.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.350+0000 7fada3e00700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad900b2bc0 0x7fad900b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.350+0000 7fada3e00700 1 -- 172.21.15.90:0/3226461714 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fad9c0e37f0 con 0x7fadac1192b0 2024-05-27T07:50:39.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.351+0000 7fadab400700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad900b2bc0 0x7fad900b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.351+0000 7fadc13cbb80 1 Processor -- start 2024-05-27T07:50:39.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.351+0000 7fadc13cbb80 1 -- start start 2024-05-27T07:50:39.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.352+0000 7fadab400700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad900b2bc0 0x7fad900b50d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fad98005fd0 tx=0x7fad98005ee0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.352+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 0x55e27abb27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 0x55e27abad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e27aadae40 con 0x55e27abb2d30 2024-05-27T07:50:39.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e27aaca9e0 con 0x55e27abacdb0 2024-05-27T07:50:39.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e27aacea20 con 0x55e27aa2de40 2024-05-27T07:50:39.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadab400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 0x55e27abb27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadabe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 0x55e27abad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadaaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadab400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 0x55e27abb27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:47630/0 (socket says 172.21.15.90:47630) 2024-05-27T07:50:39.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadaaa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32874/0 (socket says 172.21.15.90:32874) 2024-05-27T07:50:39.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.353+0000 7fadab400700 1 -- 172.21.15.90:0/813918592 learned_addr learned my addr 172.21.15.90:0/813918592 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:39.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.354+0000 7fadaaa00700 1 -- 172.21.15.90:0/813918592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 msgr2=0x55e27abb27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.354+0000 7fadaaa00700 1 --2- 172.21.15.90:0/813918592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 0x55e27abb27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.354+0000 7fadaaa00700 1 -- 172.21.15.90:0/813918592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 msgr2=0x55e27abad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.354+0000 7fadaaa00700 1 --2- 172.21.15.90:0/813918592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 0x55e27abad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.354+0000 7fadaaa00700 1 -- 172.21.15.90:0/813918592 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27aa93420 con 0x55e27abb2d30 2024-05-27T07:50:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.355+0000 7fadaaa00700 1 --2- 172.21.15.90:0/813918592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abac870 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0e0b90 tx=0x7fad9c0aa180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd2c93447e61fff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.355+0000 7fada0c00700 1 -- 172.21.15.90:0/813918592 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0b10b0 con 0x55e27abb2d30 2024-05-27T07:50:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.355+0000 7fada0c00700 1 -- 172.21.15.90:0/813918592 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad9c0b1e50 con 0x55e27abb2d30 2024-05-27T07:50:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.356+0000 7fadc13cbb80 1 -- 172.21.15.90:0/813918592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 msgr2=0x55e27abac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.356+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/813918592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abac870 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0e0b90 tx=0x7fad9c0aa180 comp rx=0 tx=0).stop 2024-05-27T07:50:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.357+0000 7fadc13cbb80 1 -- 172.21.15.90:0/813918592 shutdown_connections 2024-05-27T07:50:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.357+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/813918592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 0x55e27abad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.357+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/813918592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 0x55e27abb27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.357+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/813918592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abac870 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.357+0000 7fadc13cbb80 1 -- 172.21.15.90:0/813918592 >> 172.21.15.90:0/813918592 conn(0x55e27aa8dfc0 msgr2=0x55e27aba78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.358+0000 7fadc13cbb80 1 -- 172.21.15.90:0/813918592 shutdown_connections 2024-05-27T07:50:39.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.358+0000 7fadc13cbb80 1 -- 172.21.15.90:0/813918592 wait complete. 2024-05-27T07:50:39.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.359+0000 7fadc13cbb80 1 -- start start 2024-05-27T07:50:39.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.359+0000 7fadc13cbb80 1 Processor -- start 2024-05-27T07:50:39.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.360+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abb6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.360+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 0x55e27abb6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.360+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 0x55e27abb6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.360+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e27a9d0fa0 con 0x55e27aa2de40 2024-05-27T07:50:39.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.360+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e27aa6b760 con 0x55e27abb2d30 2024-05-27T07:50:39.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.360+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e27abb95c0 con 0x55e27abacdb0 2024-05-27T07:50:39.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.360+0000 7fadc13cbb80 10 client.0 _ll_register_callbacks cb 0x55e27aa74c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:39.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.361+0000 7fadab400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abb6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.361+0000 7fadab400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abb6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32884/0 (socket says 172.21.15.90:32884) 2024-05-27T07:50:39.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.361+0000 7fadab400700 1 -- 172.21.15.90:0/3515493550 learned_addr learned my addr 172.21.15.90:0/3515493550 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:39.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.361+0000 7fadabe00700 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 0x55e27abb6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.362+0000 7fadab400700 1 -- 172.21.15.90:0/3515493550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 msgr2=0x55e27abb6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.362+0000 7fadab400700 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 0x55e27abb6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.362+0000 7fadab400700 1 -- 172.21.15.90:0/3515493550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 msgr2=0x55e27abb6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.362+0000 7fadab400700 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 0x55e27abb6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.362+0000 7fadab400700 1 -- 172.21.15.90:0/3515493550 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27aa1a8d0 con 0x55e27aa2de40 2024-05-27T07:50:39.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.362+0000 7fadab400700 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abb6440 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fad980117c0 tx=0x7fad980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8eb3eb9551cdf96 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.362+0000 7fad95600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:39.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.363+0000 7fad95600700 1 -- 172.21.15.90:0/3515493550 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98023070 con 0x55e27aa2de40 2024-05-27T07:50:39.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.363+0000 7fad95600700 1 -- 172.21.15.90:0/3515493550 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fad98004220 con 0x55e27aa2de40 2024-05-27T07:50:39.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.363+0000 7fadc13cbb80 1 -- 172.21.15.90:0/3515493550 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9c0b1e50 con 0x55e27aa2de40 2024-05-27T07:50:39.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.363+0000 7fad95600700 1 -- 172.21.15.90:0/3515493550 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fad9802e050 con 0x55e27aa2de40 2024-05-27T07:50:39.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.363+0000 7fadc13cbb80 10 client.16389 fetch_fsmap learned FSMap version 42 2024-05-27T07:50:39.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.363+0000 7fadc13cbb80 1 -- 172.21.15.90:0/3515493550 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e27aa9b060 con 0x55e27aa2de40 2024-05-27T07:50:39.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.365+0000 7fad95600700 1 -- 172.21.15.90:0/3515493550 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad9c0b1e50 con 0x55e27aa2de40 2024-05-27T07:50:39.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.365+0000 7fad95600700 1 -- 172.21.15.90:0/3515493550 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fad9800f040 con 0x55e27aa2de40 2024-05-27T07:50:39.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.365+0000 7fadc13cbb80 10 client.16389 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:50:39.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.365+0000 7fadc13cbb80 10 client.16389 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:50:39.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.365+0000 7fadc13cbb80 20 client.16389 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:39.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.365+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e27abc6360 0x55e27abc8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.365+0000 7fadc13cbb80 4 client.16389 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:39.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.365+0000 7fadc13cbb80 1 -- 172.21.15.90:0/3515493550 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e27aadcd90 con 0x55e27abc6360 2024-05-27T07:50:39.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.366+0000 7fadaaa00700 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e27abc6360 0x55e27abc8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.367+0000 7fadaaa00700 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e27abc6360 0x55e27abc8830 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b531151c7e0076c9 server_cookie=4641d8b400ee0eb2 in_seq=0 out_seq=0 2024-05-27T07:50:39.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.367+0000 7fad95600700 10 client.16389 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:50:39.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.370+0000 7fad95600700 1 -- 172.21.15.90:0/3515493550 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e27aadcd90 con 0x55e27abc6360 2024-05-27T07:50:39.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.370+0000 7fad95600700 10 client.16389 handle_command_reply: tid=0 2024-05-27T07:50:39.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.370+0000 7fadc13cbb80 1 client.16389 shutdown 2024-05-27T07:50:39.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.371+0000 7fadc13cbb80 1 -- 172.21.15.90:0/3515493550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 msgr2=0x55e27abb6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.371+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abb6440 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fad980117c0 tx=0x7fad980047c0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.371+0000 7fadc13cbb80 1 -- 172.21.15.90:0/3515493550 shutdown_connections 2024-05-27T07:50:39.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.371+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e27abc6360 0x55e27abc8830 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.371+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 0x55e27abb6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.371+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abb6440 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.371+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/3515493550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 0x55e27abb6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.371+0000 7fadc13cbb80 1 -- 172.21.15.90:0/3515493550 >> 172.21.15.90:0/3515493550 conn(0x55e27aa8dfc0 msgr2=0x55e27aba7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.372+0000 7fad95600700 0 client.16389 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:50:39.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.372+0000 7fadc13cbb80 1 -- 172.21.15.90:0/3515493550 shutdown_connections 2024-05-27T07:50:39.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.372+0000 7fadc13cbb80 1 -- 172.21.15.90:0/3515493550 wait complete. 2024-05-27T07:50:39.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.372+0000 7fadc13cbb80 20 client.16389 trim_cache size 0 max 16384 2024-05-27T07:50:39.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.398+0000 7fadc13cbb80 1 Processor -- start 2024-05-27T07:50:39.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.398+0000 7fadc13cbb80 1 -- start start 2024-05-27T07:50:39.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.399+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 0x55e27abac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.399+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 0x55e27abb5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.399+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abc2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.399+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1c12c0 con 0x55e27abb2d30 2024-05-27T07:50:39.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.399+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1bf0b0 con 0x55e27abacdb0 2024-05-27T07:50:39.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.399+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1c0bf0 con 0x55e27aa2de40 2024-05-27T07:50:39.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.400+0000 7fadaaa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 0x55e27abb5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.400+0000 7fadaaa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 0x55e27abb5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46056/0 (socket says 172.21.15.90:46056) 2024-05-27T07:50:39.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.400+0000 7fadaaa00700 1 -- 172.21.15.90:0/2648006041 learned_addr learned my addr 172.21.15.90:0/2648006041 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:39.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.400+0000 7fadab400700 1 --2- 172.21.15.90:0/2648006041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 0x55e27abac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.400+0000 7fadabe00700 1 --2- 172.21.15.90:0/2648006041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abc2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.400+0000 7fadaaa00700 1 -- 172.21.15.90:0/2648006041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 msgr2=0x55e27abac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.400+0000 7fadaaa00700 1 --2- 172.21.15.90:0/2648006041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 0x55e27abac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.401+0000 7fadaaa00700 1 -- 172.21.15.90:0/2648006041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 msgr2=0x55e27abc2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.401+0000 7fadaaa00700 1 --2- 172.21.15.90:0/2648006041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abc2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.401+0000 7fadaaa00700 1 -- 172.21.15.90:0/2648006041 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27abb49b0 con 0x55e27abacdb0 2024-05-27T07:50:39.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.401+0000 7fadabe00700 1 --2- 172.21.15.90:0/2648006041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abc2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:39.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.401+0000 7fadaaa00700 1 --2- 172.21.15.90:0/2648006041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 0x55e27abb5760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0b1230 tx=0x7fad9c0cd1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c84cce4133d3afe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.402+0000 7fad97e00700 1 -- 172.21.15.90:0/2648006041 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0aa7b0 con 0x55e27abacdb0 2024-05-27T07:50:39.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.403+0000 7fad97e00700 1 -- 172.21.15.90:0/2648006041 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad9c0b1e50 con 0x55e27abacdb0 2024-05-27T07:50:39.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.403+0000 7fad97e00700 1 -- 172.21.15.90:0/2648006041 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0e0e10 con 0x55e27abacdb0 2024-05-27T07:50:39.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.404+0000 7fadc13cbb80 1 -- 172.21.15.90:0/2648006041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 msgr2=0x55e27abb5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.404+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/2648006041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 0x55e27abb5760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0b1230 tx=0x7fad9c0cd1b0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.404+0000 7fadc13cbb80 1 -- 172.21.15.90:0/2648006041 shutdown_connections 2024-05-27T07:50:39.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.404+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/2648006041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27abb2d30 0x55e27abc2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.404+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/2648006041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27aa2de40 0x55e27abac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.404+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/2648006041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abacdb0 0x55e27abb5760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.404+0000 7fadc13cbb80 1 -- 172.21.15.90:0/2648006041 >> 172.21.15.90:0/2648006041 conn(0x55e27aa8dfc0 msgr2=0x55e27abb1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.404+0000 7fadc13cbb80 1 -- 172.21.15.90:0/2648006041 shutdown_connections 2024-05-27T07:50:39.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.405+0000 7fadc13cbb80 1 -- 172.21.15.90:0/2648006041 wait complete. 2024-05-27T07:50:39.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.405+0000 7fadc13cbb80 1 -- start start 2024-05-27T07:50:39.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.405+0000 7fadc13cbb80 1 Processor -- start 2024-05-27T07:50:39.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.407+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abbefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.407+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 0x55e27abcd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.407+0000 7fadc13cbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 0x55e27abcddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.407+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1bfe50 con 0x55e27aa2de40 2024-05-27T07:50:39.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.407+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1bf780 con 0x55e27abb2d30 2024-05-27T07:50:39.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.407+0000 7fadc13cbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fadac1c0520 con 0x55e27abacdb0 2024-05-27T07:50:39.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.407+0000 7fadc13cbb80 10 client.0 _ll_register_callbacks cb 0x55e27aa74c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:39.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.407+0000 7fadaaa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 0x55e27abcd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.408+0000 7fadaaa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 0x55e27abcd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:47648/0 (socket says 172.21.15.90:47648) 2024-05-27T07:50:39.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.408+0000 7fadaaa00700 1 -- 172.21.15.90:0/1539607268 learned_addr learned my addr 172.21.15.90:0/1539607268 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:39.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.408+0000 7fadab400700 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abbefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.408+0000 7fadabe00700 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 0x55e27abcddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.408+0000 7fadaaa00700 1 -- 172.21.15.90:0/1539607268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 msgr2=0x55e27abcddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.408+0000 7fadaaa00700 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 0x55e27abcddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.408+0000 7fadaaa00700 1 -- 172.21.15.90:0/1539607268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 msgr2=0x55e27abbefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:39.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.408+0000 7fadaaa00700 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abbefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.408+0000 7fadaaa00700 1 -- 172.21.15.90:0/1539607268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27abce550 con 0x55e27abacdb0 2024-05-27T07:50:39.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.409+0000 7fadab400700 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abbefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:50:39.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.409+0000 7fadabe00700 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 0x55e27abcddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:39.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.409+0000 7fadaaa00700 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 0x55e27abcd870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0b3740 tx=0x7fad9c0097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24611584b213ae6a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:39.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.409+0000 7fad95600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:50:39.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.410+0000 7fad95600700 1 -- 172.21.15.90:0/1539607268 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0b9060 con 0x55e27abacdb0 2024-05-27T07:50:39.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.410+0000 7fad95600700 1 -- 172.21.15.90:0/1539607268 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fad9c0e3420 con 0x55e27abacdb0 2024-05-27T07:50:39.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.410+0000 7fadc13cbb80 1 -- 172.21.15.90:0/1539607268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad9c0b1e50 con 0x55e27abacdb0 2024-05-27T07:50:39.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.410+0000 7fad95600700 1 -- 172.21.15.90:0/1539607268 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fad9c0e3620 con 0x55e27abacdb0 2024-05-27T07:50:39.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.410+0000 7fadc13cbb80 10 client.25951 fetch_fsmap learned FSMap version 42 2024-05-27T07:50:39.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.410+0000 7fadc13cbb80 1 -- 172.21.15.90:0/1539607268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e27ac30e60 con 0x55e27abacdb0 2024-05-27T07:50:39.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.411+0000 7fad95600700 1 -- 172.21.15.90:0/1539607268 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad9c0e3620 con 0x55e27abacdb0 2024-05-27T07:50:39.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.411+0000 7fad95600700 1 -- 172.21.15.90:0/1539607268 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fad9c0aaa10 con 0x55e27abacdb0 2024-05-27T07:50:39.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.412+0000 7fadc13cbb80 10 client.25951 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:50:39.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.412+0000 7fadc13cbb80 10 client.25951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:50:39.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.412+0000 7fadc13cbb80 20 client.25951 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:39.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.412+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e27ac31090 0x55e27ac334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:39.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.412+0000 7fadc13cbb80 4 client.25951 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:50:39.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.412+0000 7fadc13cbb80 1 -- 172.21.15.90:0/1539607268 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e27aad6570 con 0x55e27ac31090 2024-05-27T07:50:39.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.413+0000 7fadab400700 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e27ac31090 0x55e27ac334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:39.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.414+0000 7fadab400700 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e27ac31090 0x55e27ac334b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f87a75a0fd9b348d server_cookie=aa9b2dcd394de072 in_seq=0 out_seq=0 2024-05-27T07:50:39.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.414+0000 7fad95600700 10 client.25951 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:50:39.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.415+0000 7fad95600700 1 -- 172.21.15.90:0/1539607268 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e27aad6570 con 0x55e27ac31090 2024-05-27T07:50:39.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.416+0000 7fad95600700 10 client.25951 handle_command_reply: tid=0 2024-05-27T07:50:39.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.416+0000 7fadc13cbb80 1 client.25951 shutdown 2024-05-27T07:50:39.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.416+0000 7fadc13cbb80 1 -- 172.21.15.90:0/1539607268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 msgr2=0x55e27abcd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.416+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 0x55e27abcd870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0b3740 tx=0x7fad9c0097f0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.416+0000 7fadc13cbb80 1 -- 172.21.15.90:0/1539607268 shutdown_connections 2024-05-27T07:50:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.416+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55e27ac31090 0x55e27ac334b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.416+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27abb2d30 0x55e27abcddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.416+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e27aa2de40 0x55e27abbefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.416+0000 7fadc13cbb80 1 --2- 172.21.15.90:0/1539607268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e27abacdb0 0x55e27abcd870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.416+0000 7fadc13cbb80 1 -- 172.21.15.90:0/1539607268 >> 172.21.15.90:0/1539607268 conn(0x55e27aa8dfc0 msgr2=0x55e27abb3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.417+0000 7fad95600700 0 client.25951 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:50:39.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.417+0000 7fadc13cbb80 1 -- 172.21.15.90:0/1539607268 shutdown_connections 2024-05-27T07:50:39.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.417+0000 7fadc13cbb80 1 -- 172.21.15.90:0/1539607268 wait complete. 2024-05-27T07:50:39.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.417+0000 7fadc13cbb80 20 client.25951 trim_cache size 0 max 16384 2024-05-27T07:50:39.418 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:50:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.418+0000 7fadb2800700 1 -- 172.21.15.90:0/3226461714 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad900b2bc0 msgr2=0x7fad900b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.418+0000 7fadb2800700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad900b2bc0 0x7fad900b50d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fad98005fd0 tx=0x7fad98005ee0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.418+0000 7fadb2800700 1 -- 172.21.15.90:0/3226461714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadac1192b0 msgr2=0x7fadac116740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.418+0000 7fadb2800700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadac1192b0 0x7fadac116740 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fadac10ba90 tx=0x7fad9c00f970 comp rx=0 tx=0).stop 2024-05-27T07:50:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.419+0000 7fadb2800700 1 -- 172.21.15.90:0/3226461714 shutdown_connections 2024-05-27T07:50:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.419+0000 7fadb2800700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadac116cd0 0x7fadac1b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.419+0000 7fadb2800700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad900b2bc0 0x7fad900b50d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.419+0000 7fadb2800700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadac10abb0 0x7fadac116200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.419+0000 7fadb2800700 1 --2- 172.21.15.90:0/3226461714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadac1192b0 0x7fadac116740 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.419+0000 7fadb2800700 1 -- 172.21.15.90:0/3226461714 >> 172.21.15.90:0/3226461714 conn(0x7fadac06f4c0 msgr2=0x7fadac119e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.419+0000 7fadb2800700 1 -- 172.21.15.90:0/3226461714 shutdown_connections 2024-05-27T07:50:39.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:39.419+0000 7fadb2800700 1 -- 172.21.15.90:0/3226461714 wait complete. 2024-05-27T07:50:40.399 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:50:40.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:40 smithi090 ceph-mon[31429]: pgmap v135: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 447 op/s 2024-05-27T07:50:40.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:40 smithi136 ceph-mon[35324]: pgmap v135: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 447 op/s 2024-05-27T07:50:40.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:40 smithi129 ceph-mon[35254]: pgmap v135: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 447 op/s 2024-05-27T07:50:41.098 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:42.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:42 smithi090 ceph-mon[31429]: pgmap v136: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 447 op/s 2024-05-27T07:50:42.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:42 smithi136 ceph-mon[35324]: pgmap v136: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 447 op/s 2024-05-27T07:50:42.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:42 smithi129 ceph-mon[35254]: pgmap v136: 129 pgs: 129 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.3 MiB/s wr, 447 op/s 2024-05-27T07:50:43.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 25.0 s, 671.4 tps, lat 151.743 ms stddev 49.590 2024-05-27T07:50:44.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:44 smithi090 ceph-mon[31429]: pgmap v137: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.8 MiB/s wr, 816 op/s 2024-05-27T07:50:44.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:44 smithi136 ceph-mon[35324]: pgmap v137: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.8 MiB/s wr, 816 op/s 2024-05-27T07:50:44.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:44 smithi129 ceph-mon[35254]: pgmap v137: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.8 MiB/s wr, 816 op/s 2024-05-27T07:50:46.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:46 smithi090 ceph-mon[31429]: pgmap v138: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.1 MiB/s wr, 749 op/s 2024-05-27T07:50:46.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:46 smithi136 ceph-mon[35324]: pgmap v138: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.1 MiB/s wr, 749 op/s 2024-05-27T07:50:46.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:46 smithi129 ceph-mon[35254]: pgmap v138: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.1 MiB/s wr, 749 op/s 2024-05-27T07:50:48.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.184+0000 7f8a1da00700 1 -- 172.21.15.90:0/3992650117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a1009c5e0 msgr2=0x7f8a1009c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:48.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.185+0000 7f8a1da00700 1 --2- 172.21.15.90:0/3992650117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a1009c5e0 0x7f8a1009c9f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009990 tx=0x7f8a0c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.188+0000 7f8a1da00700 1 -- 172.21.15.90:0/3992650117 shutdown_connections 2024-05-27T07:50:48.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.188+0000 7f8a1da00700 1 --2- 172.21.15.90:0/3992650117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a10096f10 0x7f8a100a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.188+0000 7f8a1da00700 1 --2- 172.21.15.90:0/3992650117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a1009c5e0 0x7f8a1009c9f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.188+0000 7f8a1da00700 1 --2- 172.21.15.90:0/3992650117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a100965c0 0x7f8a100969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.188+0000 7f8a1da00700 1 -- 172.21.15.90:0/3992650117 >> 172.21.15.90:0/3992650117 conn(0x7f8a1000d410 msgr2=0x7f8a1000d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:48.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.189+0000 7f8a1da00700 1 -- 172.21.15.90:0/3992650117 shutdown_connections 2024-05-27T07:50:48.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.189+0000 7f8a1da00700 1 -- 172.21.15.90:0/3992650117 wait complete. 2024-05-27T07:50:48.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.191+0000 7f8a1da00700 1 Processor -- start 2024-05-27T07:50:48.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.192+0000 7f8a1da00700 1 -- start start 2024-05-27T07:50:48.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.193+0000 7f8a1da00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a100965c0 0x7f8a1013be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.194+0000 7f8a1da00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a10096f10 0x7f8a1013c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.194+0000 7f8a1da00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a1009c5e0 0x7f8a10140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.194+0000 7f8a17400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a10096f10 0x7f8a1013c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.194+0000 7f8a17400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a10096f10 0x7f8a1013c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:47660/0 (socket says 172.21.15.90:47660) 2024-05-27T07:50:48.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.196+0000 7f8a1f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a1009c5e0 0x7f8a10140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.196+0000 7f8a1f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a1009c5e0 0x7f8a10140630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32900/0 (socket says 172.21.15.90:32900) 2024-05-27T07:50:48.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.196+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8a100a44a0 con 0x7f8a1009c5e0 2024-05-27T07:50:48.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.196+0000 7f8a17400700 1 -- 172.21.15.90:0/2148682484 learned_addr learned my addr 172.21.15.90:0/2148682484 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:48.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.196+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a100a4320 con 0x7f8a100965c0 2024-05-27T07:50:48.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.196+0000 7f8a17e00700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a100965c0 0x7f8a1013be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.196+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a100a4620 con 0x7f8a10096f10 2024-05-27T07:50:48.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.198+0000 7f8a1f000700 1 -- 172.21.15.90:0/2148682484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a10096f10 msgr2=0x7f8a1013c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:48.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.198+0000 7f8a1f000700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a10096f10 0x7f8a1013c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.198+0000 7f8a1f000700 1 -- 172.21.15.90:0/2148682484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a100965c0 msgr2=0x7f8a1013be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:48.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.198+0000 7f8a1f000700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a100965c0 0x7f8a1013be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.198+0000 7f8a1f000700 1 -- 172.21.15.90:0/2148682484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a10140cd0 con 0x7f8a1009c5e0 2024-05-27T07:50:48.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.200+0000 7f8a1f000700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a1009c5e0 0x7f8a10140630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8a1806a3b0 tx=0x7f8a1807d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:48.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.201+0000 7f8a14c00700 1 -- 172.21.15.90:0/2148682484 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18083070 con 0x7f8a1009c5e0 2024-05-27T07:50:48.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.202+0000 7f8a14c00700 1 -- 172.21.15.90:0/2148682484 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a1807ba60 con 0x7f8a1009c5e0 2024-05-27T07:50:48.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.202+0000 7f8a14c00700 1 -- 172.21.15.90:0/2148682484 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18073710 con 0x7f8a1009c5e0 2024-05-27T07:50:48.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.202+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a10140fc0 con 0x7f8a1009c5e0 2024-05-27T07:50:48.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.202+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a101488e0 con 0x7f8a1009c5e0 2024-05-27T07:50:48.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.210+0000 7f8a14c00700 1 -- 172.21.15.90:0/2148682484 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8a18074070 con 0x7f8a1009c5e0 2024-05-27T07:50:48.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.213+0000 7f8a14c00700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a000b2a70 0x7f8a000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.214+0000 7f8a14c00700 1 -- 172.21.15.90:0/2148682484 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f8a1814bc10 con 0x7f8a1009c5e0 2024-05-27T07:50:48.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.214+0000 7f8a2dbb5b80 1 Processor -- start 2024-05-27T07:50:48.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.214+0000 7f8a17e00700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a000b2a70 0x7f8a000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.214+0000 7f8a2dbb5b80 1 -- start start 2024-05-27T07:50:48.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.215+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc6949f0 0x55f9cc728470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.215+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc7289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.215+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc728f90 0x55f9cc73cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.215+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f9cc655e40 con 0x55f9cc728f90 2024-05-27T07:50:48.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.215+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f9cc6459e0 con 0x55f9cc5a8e40 2024-05-27T07:50:48.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.215+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f9cc649a20 con 0x55f9cc6949f0 2024-05-27T07:50:48.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.217+0000 7f8a17400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc7289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.217+0000 7f8a1f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc728f90 0x55f9cc73cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.217+0000 7f8a1f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc728f90 0x55f9cc73cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32902/0 (socket says 172.21.15.90:32902) 2024-05-27T07:50:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.217+0000 7f8a1f000700 1 -- 172.21.15.90:0/1448060349 learned_addr learned my addr 172.21.15.90:0/1448060349 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.217+0000 7f8a17e00700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a000b2a70 0x7f8a000b4f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00b5c0 tx=0x7f8a0c005fb0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:48.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.217+0000 7f8a17400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc7289b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46088/0 (socket says 172.21.15.90:46088) 2024-05-27T07:50:48.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.218+0000 7f8a17e00700 1 --2- 172.21.15.90:0/1448060349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc6949f0 0x55f9cc728470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.219+0000 7f8a17400700 1 -- 172.21.15.90:0/1448060349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc6949f0 msgr2=0x55f9cc728470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:48.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.219+0000 7f8a17400700 1 --2- 172.21.15.90:0/1448060349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc6949f0 0x55f9cc728470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.219+0000 7f8a17400700 1 -- 172.21.15.90:0/1448060349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc728f90 msgr2=0x55f9cc73cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:48.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.219+0000 7f8a17400700 1 --2- 172.21.15.90:0/1448060349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc728f90 0x55f9cc73cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.219+0000 7f8a17400700 1 -- 172.21.15.90:0/1448060349 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9cc60e420 con 0x55f9cc5a8e40 2024-05-27T07:50:48.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.221+0000 7f8a17400700 1 --2- 172.21.15.90:0/1448060349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc7289b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005c50 tx=0x7f8a0400df60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a70f07c8fb56d3a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:48.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.224+0000 7f8a09600700 1 -- 172.21.15.90:0/1448060349 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04004220 con 0x55f9cc5a8e40 2024-05-27T07:50:48.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.224+0000 7f8a09600700 1 -- 172.21.15.90:0/1448060349 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a0400fdd0 con 0x55f9cc5a8e40 2024-05-27T07:50:48.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.224+0000 7f8a09600700 1 -- 172.21.15.90:0/1448060349 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04003dc0 con 0x55f9cc5a8e40 2024-05-27T07:50:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.226+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1448060349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 msgr2=0x55f9cc7289b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:48.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.226+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1448060349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc7289b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005c50 tx=0x7f8a0400df60 comp rx=0 tx=0).stop 2024-05-27T07:50:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.226+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1448060349 shutdown_connections 2024-05-27T07:50:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.226+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1448060349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc728f90 0x55f9cc73cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.226+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1448060349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc6949f0 0x55f9cc728470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.226+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1448060349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc7289b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.226+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1448060349 >> 172.21.15.90:0/1448060349 conn(0x55f9cc642390 msgr2=0x55f9cc723710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.227+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1448060349 shutdown_connections 2024-05-27T07:50:48.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.227+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1448060349 wait complete. 2024-05-27T07:50:48.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.228+0000 7f8a2dbb5b80 1 -- start start 2024-05-27T07:50:48.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.228+0000 7f8a2dbb5b80 1 Processor -- start 2024-05-27T07:50:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.230+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc73a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.230+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc73a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.230+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc728f90 0x55f9cc73aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.230+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f9cc54bfa0 con 0x55f9cc6949f0 2024-05-27T07:50:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.230+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f9cc5e6760 con 0x55f9cc5a8e40 2024-05-27T07:50:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.230+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f9cc73d580 con 0x55f9cc728f90 2024-05-27T07:50:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.230+0000 7f8a2dbb5b80 10 client.0 _ll_register_callbacks cb 0x55f9cc5efc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.231+0000 7f8a17400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc73a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.231+0000 7f8a17400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc73a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32918/0 (socket says 172.21.15.90:32918) 2024-05-27T07:50:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.231+0000 7f8a17400700 1 -- 172.21.15.90:0/1687859132 learned_addr learned my addr 172.21.15.90:0/1687859132 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:48.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.231+0000 7f8a1f000700 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc728f90 0x55f9cc73aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.231+0000 7f8a17e00700 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc73a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.231+0000 7f8a17400700 1 -- 172.21.15.90:0/1687859132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc728f90 msgr2=0x55f9cc73aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.231+0000 7f8a17400700 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc728f90 0x55f9cc73aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.231+0000 7f8a17400700 1 -- 172.21.15.90:0/1687859132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 msgr2=0x55f9cc73a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:48.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.231+0000 7f8a17400700 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc73a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.232+0000 7f8a17400700 1 -- 172.21.15.90:0/1687859132 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9cc5958d0 con 0x55f9cc6949f0 2024-05-27T07:50:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.232+0000 7f8a17e00700 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc73a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.232+0000 7f8a17400700 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc73a5a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8a04000c00 tx=0x7f8a0400fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc4e61750aae84f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.232+0000 7f89fe000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.233+0000 7f89fe000700 1 -- 172.21.15.90:0/1687859132 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402a060 con 0x55f9cc6949f0 2024-05-27T07:50:48.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.233+0000 7f89fe000700 1 -- 172.21.15.90:0/1687859132 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f8a04005310 con 0x55f9cc6949f0 2024-05-27T07:50:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.234+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1687859132 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a0400fdd0 con 0x55f9cc6949f0 2024-05-27T07:50:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.234+0000 7f89fe000700 1 -- 172.21.15.90:0/1687859132 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8a04005510 con 0x55f9cc6949f0 2024-05-27T07:50:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.234+0000 7f8a2dbb5b80 10 client.16407 fetch_fsmap learned FSMap version 42 2024-05-27T07:50:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.234+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1687859132 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f9cc616060 con 0x55f9cc6949f0 2024-05-27T07:50:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.236+0000 7f89fe000700 1 -- 172.21.15.90:0/1687859132 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a0400fdd0 con 0x55f9cc6949f0 2024-05-27T07:50:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.236+0000 7f89fe000700 1 -- 172.21.15.90:0/1687859132 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f8a0401b6a0 con 0x55f9cc6949f0 2024-05-27T07:50:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.236+0000 7f8a2dbb5b80 10 client.16407 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:50:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.236+0000 7f8a2dbb5b80 10 client.16407 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:50:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.236+0000 7f8a2dbb5b80 20 client.16407 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.236+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55f9cc74a2d0 0x55f9cc74c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.236+0000 7f8a2dbb5b80 4 client.16407 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.236+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1687859132 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9cc657d90 con 0x55f9cc74a2d0 2024-05-27T07:50:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.237+0000 7f8a17e00700 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55f9cc74a2d0 0x55f9cc74c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.239+0000 7f8a17e00700 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55f9cc74a2d0 0x55f9cc74c6f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1ca58a8afdf41e3 server_cookie=53ab09e135809c1b in_seq=0 out_seq=0 2024-05-27T07:50:48.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.240+0000 7f89fe000700 10 client.16407 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:50:48.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.241+0000 7f89fe000700 1 -- 172.21.15.90:0/1687859132 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f9cc657d90 con 0x55f9cc74a2d0 2024-05-27T07:50:48.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.241+0000 7f89fe000700 10 client.16407 handle_command_reply: tid=0 2024-05-27T07:50:48.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.242+0000 7f8a2dbb5b80 1 client.16407 shutdown 2024-05-27T07:50:48.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.242+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1687859132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 msgr2=0x55f9cc73a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:48.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.242+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc73a5a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8a04000c00 tx=0x7f8a0400fb70 comp rx=0 tx=0).stop 2024-05-27T07:50:48.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.243+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1687859132 shutdown_connections 2024-05-27T07:50:48.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.243+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc728f90 0x55f9cc73aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.244+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55f9cc74a2d0 0x55f9cc74c6f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.244+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc5a8e40 0x55f9cc73a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.244+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1687859132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc73a5a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.244+0000 7f89fe000700 0 client.16407 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:50:48.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.244+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1687859132 >> 172.21.15.90:0/1687859132 conn(0x55f9cc642390 msgr2=0x55f9cc7233a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.245+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1687859132 shutdown_connections 2024-05-27T07:50:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.245+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1687859132 wait complete. 2024-05-27T07:50:48.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.245+0000 7f8a2dbb5b80 20 client.16407 trim_cache size 0 max 16384 2024-05-27T07:50:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.277+0000 7f8a2dbb5b80 1 Processor -- start 2024-05-27T07:50:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.277+0000 7f8a2dbb5b80 1 -- start start 2024-05-27T07:50:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.278+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc5a8e40 0x55f9cc738fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.278+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc727b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.278+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc728f90 0x55f9cc723740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.278+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10153160 con 0x55f9cc6949f0 2024-05-27T07:50:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.278+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10150f50 con 0x55f9cc728f90 2024-05-27T07:50:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.278+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10152a90 con 0x55f9cc5a8e40 2024-05-27T07:50:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.279+0000 7f8a17400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc727b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.279+0000 7f8a17e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc5a8e40 0x55f9cc738fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.279+0000 7f8a17e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc5a8e40 0x55f9cc738fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:47700/0 (socket says 172.21.15.90:47700) 2024-05-27T07:50:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.279+0000 7f8a17e00700 1 -- 172.21.15.90:0/1452159821 learned_addr learned my addr 172.21.15.90:0/1452159821 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.279+0000 7f8a1f000700 1 --2- 172.21.15.90:0/1452159821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc728f90 0x55f9cc723740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.280+0000 7f8a17400700 1 -- 172.21.15.90:0/1452159821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc5a8e40 msgr2=0x55f9cc738fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.280+0000 7f8a17400700 1 --2- 172.21.15.90:0/1452159821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc5a8e40 0x55f9cc738fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.280+0000 7f8a17400700 1 -- 172.21.15.90:0/1452159821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc728f90 msgr2=0x55f9cc723740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.280+0000 7f8a17400700 1 --2- 172.21.15.90:0/1452159821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc728f90 0x55f9cc723740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.280+0000 7f8a17400700 1 -- 172.21.15.90:0/1452159821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9cc73ae50 con 0x55f9cc6949f0 2024-05-27T07:50:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.281+0000 7f8a17400700 1 --2- 172.21.15.90:0/1452159821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc727b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8a0401bc10 tx=0x7f8a0400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e38863829111d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.281+0000 7f8a08c00700 1 -- 172.21.15.90:0/1452159821 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402a040 con 0x55f9cc6949f0 2024-05-27T07:50:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.281+0000 7f8a08c00700 1 -- 172.21.15.90:0/1452159821 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a04005310 con 0x55f9cc6949f0 2024-05-27T07:50:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.282+0000 7f8a08c00700 1 -- 172.21.15.90:0/1452159821 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402a040 con 0x55f9cc6949f0 2024-05-27T07:50:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.283+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1452159821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 msgr2=0x55f9cc727b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.283+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1452159821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc727b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8a0401bc10 tx=0x7f8a0400df60 comp rx=0 tx=0).stop 2024-05-27T07:50:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.284+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1452159821 shutdown_connections 2024-05-27T07:50:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.284+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1452159821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc728f90 0x55f9cc723740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.284+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1452159821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc5a8e40 0x55f9cc738fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.284+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/1452159821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc6949f0 0x55f9cc727b50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.284+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1452159821 >> 172.21.15.90:0/1452159821 conn(0x55f9cc642390 msgr2=0x55f9cc73b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.284+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1452159821 shutdown_connections 2024-05-27T07:50:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.285+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/1452159821 wait complete. 2024-05-27T07:50:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.286+0000 7f8a2dbb5b80 1 -- start start 2024-05-27T07:50:48.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.286+0000 7f8a2dbb5b80 1 Processor -- start 2024-05-27T07:50:48.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.289+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc5a8e40 0x55f9cc734040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.289+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc6949f0 0x55f9cc734580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.289+0000 7f8a2dbb5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc728f90 0x55f9cc734ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.289+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10151cf0 con 0x55f9cc5a8e40 2024-05-27T07:50:48.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.289+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10151620 con 0x55f9cc6949f0 2024-05-27T07:50:48.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.289+0000 7f8a2dbb5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a101523c0 con 0x55f9cc728f90 2024-05-27T07:50:48.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.289+0000 7f8a2dbb5b80 10 client.0 _ll_register_callbacks cb 0x55f9cc5efc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.290+0000 7f8a1f000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc728f90 0x55f9cc734ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.290+0000 7f8a17e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc5a8e40 0x55f9cc734040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.290+0000 7f8a17e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc5a8e40 0x55f9cc734040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32936/0 (socket says 172.21.15.90:32936) 2024-05-27T07:50:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.290+0000 7f8a17e00700 1 -- 172.21.15.90:0/2391713177 learned_addr learned my addr 172.21.15.90:0/2391713177 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.290+0000 7f8a17400700 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc6949f0 0x55f9cc734580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.291+0000 7f8a17e00700 1 -- 172.21.15.90:0/2391713177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc728f90 msgr2=0x55f9cc734ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:48.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.291+0000 7f8a17e00700 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc728f90 0x55f9cc734ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.291+0000 7f8a17e00700 1 -- 172.21.15.90:0/2391713177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc6949f0 msgr2=0x55f9cc734580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:48.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.291+0000 7f8a17e00700 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc6949f0 0x55f9cc734580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.291+0000 7f8a17e00700 1 -- 172.21.15.90:0/2391713177 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9cc741ae0 con 0x55f9cc5a8e40 2024-05-27T07:50:48.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.293+0000 7f8a17400700 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc6949f0 0x55f9cc734580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:48.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.293+0000 7f8a17e00700 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc5a8e40 0x55f9cc734040 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009990 tx=0x7f8a0c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19985fb65f43728 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:48.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.293+0000 7f89fe000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:48.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.294+0000 7f89fe000700 1 -- 172.21.15.90:0/2391713177 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c036020 con 0x55f9cc5a8e40 2024-05-27T07:50:48.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.294+0000 7f89fe000700 1 -- 172.21.15.90:0/2391713177 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f8a0c00b7d0 con 0x55f9cc5a8e40 2024-05-27T07:50:48.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.294+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/2391713177 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a04005310 con 0x55f9cc5a8e40 2024-05-27T07:50:48.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f89fe000700 1 -- 172.21.15.90:0/2391713177 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8a0c00b9d0 con 0x55f9cc5a8e40 2024-05-27T07:50:48.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f8a2dbb5b80 10 client.16413 fetch_fsmap learned FSMap version 42 2024-05-27T07:50:48.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/2391713177 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f9cc742da0 con 0x55f9cc5a8e40 2024-05-27T07:50:48.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f89fe000700 1 -- 172.21.15.90:0/2391713177 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a04005310 con 0x55f9cc5a8e40 2024-05-27T07:50:48.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f89fe000700 1 -- 172.21.15.90:0/2391713177 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f8a0c00f460 con 0x55f9cc5a8e40 2024-05-27T07:50:48.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f8a2dbb5b80 10 client.16413 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:50:48.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f8a2dbb5b80 10 client.16413 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:50:48.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f8a2dbb5b80 20 client.16413 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:48.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55f9cc7b4c20 0x55f9cc7b7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:48.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f8a2dbb5b80 4 client.16413 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:50:48.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.295+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/2391713177 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9cc651570 con 0x55f9cc7b4c20 2024-05-27T07:50:48.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.296+0000 7f8a17400700 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55f9cc7b4c20 0x55f9cc7b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:48.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.298+0000 7f8a17400700 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55f9cc7b4c20 0x55f9cc7b7040 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6119d8904b75c23 server_cookie=377912c4fd9af825 in_seq=0 out_seq=0 2024-05-27T07:50:48.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.298+0000 7f89fe000700 10 client.16413 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:50:48.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.299+0000 7f89fe000700 1 -- 172.21.15.90:0/2391713177 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9cc651570 con 0x55f9cc7b4c20 2024-05-27T07:50:48.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.299+0000 7f89fe000700 10 client.16413 handle_command_reply: tid=0 2024-05-27T07:50:48.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.299+0000 7f8a2dbb5b80 1 client.16413 shutdown 2024-05-27T07:50:48.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.300+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/2391713177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc5a8e40 msgr2=0x55f9cc734040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:48.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.300+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc5a8e40 0x55f9cc734040 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009990 tx=0x7f8a0c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.300+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/2391713177 shutdown_connections 2024-05-27T07:50:48.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.300+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9cc728f90 0x55f9cc734ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.300+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9cc5a8e40 0x55f9cc734040 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.300+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55f9cc7b4c20 0x55f9cc7b7040 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.300+0000 7f8a2dbb5b80 1 --2- 172.21.15.90:0/2391713177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9cc6949f0 0x55f9cc734580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.300+0000 7f89fe000700 0 client.16413 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:50:48.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.300+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/2391713177 >> 172.21.15.90:0/2391713177 conn(0x55f9cc642390 msgr2=0x55f9cc7278b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:48.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.301+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/2391713177 shutdown_connections 2024-05-27T07:50:48.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.302+0000 7f8a2dbb5b80 1 -- 172.21.15.90:0/2391713177 wait complete. 2024-05-27T07:50:48.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.302+0000 7f8a2dbb5b80 20 client.16413 trim_cache size 0 max 16384 2024-05-27T07:50:48.302 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:50:48.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.303+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a000b2a70 msgr2=0x7f8a000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:48.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.303+0000 7f8a1da00700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a000b2a70 0x7f8a000b4f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00b5c0 tx=0x7f8a0c005fb0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.303+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a1009c5e0 msgr2=0x7f8a10140630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:48.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.303+0000 7f8a1da00700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a1009c5e0 0x7f8a10140630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8a1806a3b0 tx=0x7f8a1807d8d0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.303+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 shutdown_connections 2024-05-27T07:50:48.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.303+0000 7f8a1da00700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a1009c5e0 0x7f8a10140630 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.303+0000 7f8a1da00700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a000b2a70 0x7f8a000b4f80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.303+0000 7f8a1da00700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a100965c0 0x7f8a1013be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.303+0000 7f8a1da00700 1 --2- 172.21.15.90:0/2148682484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a10096f10 0x7f8a1013c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:48.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.303+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 >> 172.21.15.90:0/2148682484 conn(0x7f8a1000d410 msgr2=0x7f8a100950d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:48.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.305+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 shutdown_connections 2024-05-27T07:50:48.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:48.305+0000 7f8a1da00700 1 -- 172.21.15.90:0/2148682484 wait complete. 2024-05-27T07:50:48.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:48 smithi090 ceph-mon[31429]: pgmap v139: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.5 MiB/s wr, 680 op/s 2024-05-27T07:50:48.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:48 smithi136 ceph-mon[35324]: pgmap v139: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.5 MiB/s wr, 680 op/s 2024-05-27T07:50:48.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:48 smithi129 ceph-mon[35254]: pgmap v139: 129 pgs: 129 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.5 MiB/s wr, 680 op/s 2024-05-27T07:50:48.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 30.0 s, 636.2 tps, lat 155.742 ms stddev 56.516 2024-05-27T07:50:49.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:49 smithi129 ceph-mon[35254]: pgmap v140: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 8.9 MiB/s wr, 969 op/s 2024-05-27T07:50:49.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:49 smithi090 ceph-mon[31429]: pgmap v140: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 8.9 MiB/s wr, 969 op/s 2024-05-27T07:50:49.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:49 smithi136 ceph-mon[35324]: pgmap v140: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 8.9 MiB/s wr, 969 op/s 2024-05-27T07:50:49.971 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:50:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.232+0000 7fbce3200640 1 -- 172.21.15.90:0/1673380013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbce4109f70 msgr2=0x7fbce4103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.232+0000 7fbce3200640 1 --2- 172.21.15.90:0/1673380013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbce4109f70 0x7fbce4103bb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009900 tx=0x7fbcd002a400 comp rx=0 tx=0).stop 2024-05-27T07:50:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.232+0000 7fbce3200640 1 -- 172.21.15.90:0/1673380013 shutdown_connections 2024-05-27T07:50:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.232+0000 7fbce3200640 1 --2- 172.21.15.90:0/1673380013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4104220 0x7fbce41105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.232+0000 7fbce3200640 1 --2- 172.21.15.90:0/1673380013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbce4109f70 0x7fbce4103bb0 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009900 tx=0x7fbcd002a400 comp rx=0 tx=0).stop 2024-05-27T07:50:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.232+0000 7fbce3200640 1 --2- 172.21.15.90:0/1673380013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbce4109ad0 0x7fbce4103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.233+0000 7fbce3200640 1 -- 172.21.15.90:0/1673380013 >> 172.21.15.90:0/1673380013 conn(0x7fbce4079100 msgr2=0x7fbce4079540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.233+0000 7fbce3200640 1 -- 172.21.15.90:0/1673380013 shutdown_connections 2024-05-27T07:50:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.233+0000 7fbce3200640 1 -- 172.21.15.90:0/1673380013 wait complete. 2024-05-27T07:50:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.237+0000 7fbce3200640 1 Processor -- start 2024-05-27T07:50:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.237+0000 7fbce3200640 1 -- start start 2024-05-27T07:50:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.238+0000 7fbce3200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbce4104220 0x7fbce410c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.239+0000 7fbce3200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4109ad0 0x7fbce410c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.239+0000 7fbce3200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbce410ce00 0x7fbce41accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.239+0000 7fbce3200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbce4111710 con 0x7fbce410ce00 2024-05-27T07:50:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.239+0000 7fbce3200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbce4111590 con 0x7fbce4109ad0 2024-05-27T07:50:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.239+0000 7fbce3200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbce4111890 con 0x7fbce4104220 2024-05-27T07:50:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.239+0000 7fbce1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbce4104220 0x7fbce410c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.239+0000 7fbce1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbce4104220 0x7fbce410c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56162/0 (socket says 172.21.15.90:56162) 2024-05-27T07:50:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.239+0000 7fbce1e00640 1 -- 172.21.15.90:0/2409452108 learned_addr learned my addr 172.21.15.90:0/2409452108 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.240+0000 7fbce1400640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4109ad0 0x7fbce410c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.240+0000 7fbce1e00640 1 -- 172.21.15.90:0/2409452108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4109ad0 msgr2=0x7fbce410c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.240+0000 7fbce1e00640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4109ad0 0x7fbce410c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.240+0000 7fbce1e00640 1 -- 172.21.15.90:0/2409452108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbce410ce00 msgr2=0x7fbce41accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.240+0000 7fbce1e00640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbce410ce00 0x7fbce41accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.240+0000 7fbce1e00640 1 -- 172.21.15.90:0/2409452108 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce41ad2e0 con 0x7fbce4104220 2024-05-27T07:50:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.241+0000 7fbce1e00640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbce4104220 0x7fbce410c380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00efd0 tx=0x7fbcdc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.242+0000 7fbcdaa00640 1 -- 172.21.15.90:0/2409452108 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc003b70 con 0x7fbce4104220 2024-05-27T07:50:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.242+0000 7fbcdaa00640 1 -- 172.21.15.90:0/2409452108 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcdc003d10 con 0x7fbce4104220 2024-05-27T07:50:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.242+0000 7fbce3200640 1 -- 172.21.15.90:0/2409452108 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce41ad5d0 con 0x7fbce4104220 2024-05-27T07:50:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.243+0000 7fbce3200640 1 -- 172.21.15.90:0/2409452108 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce41adb40 con 0x7fbce4104220 2024-05-27T07:50:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.243+0000 7fbcdaa00640 1 -- 172.21.15.90:0/2409452108 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc003b70 con 0x7fbce4104220 2024-05-27T07:50:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.247+0000 7fbcefa1b740 1 Processor -- start 2024-05-27T07:50:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.247+0000 7fbcefa1b740 1 -- start start 2024-05-27T07:50:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.248+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317fe1b0 0x55ec31ba0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.248+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317d52f0 0x55ec31ba0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.248+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317dd4d0 0x55ec31ba14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.248+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ec31b11b90 con 0x55ec317fe1b0 2024-05-27T07:50:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.248+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ec31b01730 con 0x55ec317dd4d0 2024-05-27T07:50:50.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.248+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ec31b05770 con 0x55ec317d52f0 2024-05-27T07:50:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.249+0000 7fbce2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317fe1b0 0x55ec31ba0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.249+0000 7fbce2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317fe1b0 0x55ec31ba0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39598/0 (socket says 172.21.15.90:39598) 2024-05-27T07:50:50.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.249+0000 7fbce2800640 1 -- 172.21.15.90:0/1317985577 learned_addr learned my addr 172.21.15.90:0/1317985577 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.249+0000 7fbce2800640 1 -- 172.21.15.90:0/1317985577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317d52f0 msgr2=0x55ec31ba0fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:50:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.249+0000 7fbce2800640 1 --2- 172.21.15.90:0/1317985577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317d52f0 0x55ec31ba0fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.249+0000 7fbce2800640 1 -- 172.21.15.90:0/1317985577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317dd4d0 msgr2=0x55ec31ba14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.249+0000 7fbce1400640 1 --2- 172.21.15.90:0/1317985577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317dd4d0 0x55ec31ba14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.250+0000 7fbce2800640 1 --2- 172.21.15.90:0/1317985577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317dd4d0 0x55ec31ba14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.250+0000 7fbce2800640 1 -- 172.21.15.90:0/1317985577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec31a9a8b0 con 0x55ec317fe1b0 2024-05-27T07:50:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.250+0000 7fbce1400640 1 --2- 172.21.15.90:0/1317985577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317dd4d0 0x55ec31ba14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.250+0000 7fbce2800640 1 --2- 172.21.15.90:0/1317985577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317fe1b0 0x55ec31ba0a60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400ba90 tx=0x7fbcd400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743a048949196bb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.251+0000 7fbccf000640 1 -- 172.21.15.90:0/1317985577 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4016070 con 0x55ec317fe1b0 2024-05-27T07:50:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.251+0000 7fbccf000640 1 -- 172.21.15.90:0/1317985577 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcd40054b0 con 0x55ec317fe1b0 2024-05-27T07:50:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.251+0000 7fbccf000640 1 -- 172.21.15.90:0/1317985577 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4016070 con 0x55ec317fe1b0 2024-05-27T07:50:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.253+0000 7fbcefa1b740 1 -- 172.21.15.90:0/1317985577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317fe1b0 msgr2=0x55ec31ba0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.253+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/1317985577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317fe1b0 0x55ec31ba0a60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400ba90 tx=0x7fbcd400bf50 comp rx=0 tx=0).stop 2024-05-27T07:50:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.254+0000 7fbcefa1b740 1 -- 172.21.15.90:0/1317985577 shutdown_connections 2024-05-27T07:50:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.254+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/1317985577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317dd4d0 0x55ec31ba14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.254+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/1317985577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317d52f0 0x55ec31ba0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.254+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/1317985577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317fe1b0 0x55ec31ba0a60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.254+0000 7fbcefa1b740 1 -- 172.21.15.90:0/1317985577 >> 172.21.15.90:0/1317985577 conn(0x55ec317f2300 msgr2=0x55ec31b9d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.260+0000 7fbcefa1b740 1 -- 172.21.15.90:0/1317985577 shutdown_connections 2024-05-27T07:50:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.260+0000 7fbcdaa00640 1 -- 172.21.15.90:0/2409452108 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbcdc003fe0 con 0x7fbce4104220 2024-05-27T07:50:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.261+0000 7fbcefa1b740 1 -- 172.21.15.90:0/1317985577 wait complete. 2024-05-27T07:50:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.262+0000 7fbcefa1b740 1 -- start start 2024-05-27T07:50:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.262+0000 7fbcefa1b740 1 Processor -- start 2024-05-27T07:50:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.264+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317dd4d0 0x55ec31baaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.264+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317fe1b0 0x55ec31bab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.264+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.264+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ec317e7790 con 0x55ec317dd4d0 2024-05-27T07:50:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.264+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ec31aa9bc0 con 0x55ec317bcd80 2024-05-27T07:50:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.264+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ec31badb90 con 0x55ec317fe1b0 2024-05-27T07:50:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.264+0000 7fbcefa1b740 10 client.0 _ll_register_callbacks cb 0x55ec31a87400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.265+0000 7fbce2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317dd4d0 0x55ec31baaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.265+0000 7fbce2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317dd4d0 0x55ec31baaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39606/0 (socket says 172.21.15.90:39606) 2024-05-27T07:50:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.265+0000 7fbce2800640 1 -- 172.21.15.90:0/99387975 learned_addr learned my addr 172.21.15.90:0/99387975 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.266+0000 7fbce1e00640 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317fe1b0 0x55ec31bab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.266+0000 7fbce2800640 1 -- 172.21.15.90:0/99387975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317fe1b0 msgr2=0x55ec31bab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.266+0000 7fbce2800640 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317fe1b0 0x55ec31bab410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.266+0000 7fbce2800640 1 -- 172.21.15.90:0/99387975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 msgr2=0x55ec31bb6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.266+0000 7fbce2800640 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.266+0000 7fbce2800640 1 -- 172.21.15.90:0/99387975 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec31816210 con 0x55ec317dd4d0 2024-05-27T07:50:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.266+0000 7fbcdaa00640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbcc80b2b70 0x7fbcc80b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.266+0000 7fbcefa1b740 1 -- 172.21.15.90:0/99387975 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd40054b0 con 0x55ec317dd4d0 2024-05-27T07:50:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.267+0000 7fbcdaa00640 1 -- 172.21.15.90:0/2409452108 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fbcdc0e2d80 con 0x7fbce4104220 2024-05-27T07:50:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.267+0000 7fbce1e00640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbcc80b2b70 0x7fbcc80b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.267+0000 7fbce2800640 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317dd4d0 0x55ec31baaed0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b590 tx=0x7fbcd40052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39edafbddfa3c5a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.267+0000 7fbcc3400640 10 client.16422 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.268+0000 7fbce1e00640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbcc80b2b70 0x7fbcc80b5080 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55ec31b9e550 tx=0x7fbcdc00cf60 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.269+0000 7fbcc3400640 1 -- 172.21.15.90:0/99387975 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4016070 con 0x55ec317dd4d0 2024-05-27T07:50:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.269+0000 7fbcc3400640 1 -- 172.21.15.90:0/99387975 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fbcd4004d00 con 0x55ec317dd4d0 2024-05-27T07:50:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.270+0000 7fbcc3400640 1 -- 172.21.15.90:0/99387975 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbcd400daa0 con 0x55ec317dd4d0 2024-05-27T07:50:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.270+0000 7fbcefa1b740 10 client.16422 fetch_fsmap learned FSMap version 42 2024-05-27T07:50:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.270+0000 7fbcefa1b740 1 -- 172.21.15.90:0/99387975 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55ec318e2620 con 0x55ec317dd4d0 2024-05-27T07:50:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.273+0000 7fbcc3400640 1 -- 172.21.15.90:0/99387975 <== mon.0 v2:172.21.15.90:3300/0 4 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fbcd400dc40 con 0x55ec317dd4d0 2024-05-27T07:50:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.273+0000 7fbcefa1b740 10 client.16422 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:50:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.273+0000 7fbcefa1b740 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.273+0000 7fbcefa1b740 20 client.16422 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:50.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.273+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ec317921c0 0x55ec31bbca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.273+0000 7fbcefa1b740 4 client.16422 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.273+0000 7fbcefa1b740 1 -- 172.21.15.90:0/99387975 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec31b13ae0 con 0x55ec317921c0 2024-05-27T07:50:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.274+0000 7fbce1400640 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ec317921c0 0x55ec31bbca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.275+0000 7fbcc3400640 10 client.16422 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:50.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.275+0000 7fbce1400640 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ec317921c0 0x55ec31bbca30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b44fba70eb767dcf server_cookie=fc7df8a372a95ee4 in_seq=0 out_seq=0 2024-05-27T07:50:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.278+0000 7fbcc3400640 1 -- 172.21.15.90:0/99387975 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ec31b13ae0 con 0x55ec317921c0 2024-05-27T07:50:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.278+0000 7fbcc3400640 10 client.16422 handle_command_reply: tid=0 2024-05-27T07:50:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.279+0000 7fbcefa1b740 1 client.16422 shutdown 2024-05-27T07:50:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.280+0000 7fbcefa1b740 1 -- 172.21.15.90:0/99387975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317dd4d0 msgr2=0x55ec31baaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.280+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317dd4d0 0x55ec31baaed0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b590 tx=0x7fbcd40052b0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.281+0000 7fbcefa1b740 1 -- 172.21.15.90:0/99387975 shutdown_connections 2024-05-27T07:50:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.281+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ec317921c0 0x55ec31bbca30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.281+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.281+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317fe1b0 0x55ec31bab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.281+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/99387975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317dd4d0 0x55ec31baaed0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.281+0000 7fbcc3400640 0 client.16422 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:50.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.281+0000 7fbcefa1b740 1 -- 172.21.15.90:0/99387975 >> 172.21.15.90:0/99387975 conn(0x55ec317f2300 msgr2=0x55ec31ba2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.281+0000 7fbcefa1b740 1 -- 172.21.15.90:0/99387975 shutdown_connections 2024-05-27T07:50:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.283+0000 7fbcefa1b740 1 -- 172.21.15.90:0/99387975 wait complete. 2024-05-27T07:50:50.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.283+0000 7fbcefa1b740 20 client.16422 trim_cache size 0 max 16384 2024-05-27T07:50:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.328+0000 7fbcefa1b740 1 Processor -- start 2024-05-27T07:50:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.329+0000 7fbcefa1b740 1 -- start start 2024-05-27T07:50:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.329+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317f2300 0x55ec31bb6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.329+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.329+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 0x55ec31ba4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.329+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41b96e0 con 0x55ec317f2300 2024-05-27T07:50:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.329+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41b74d0 con 0x55ec317bcd80 2024-05-27T07:50:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.329+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41b9010 con 0x55ec317dd4d0 2024-05-27T07:50:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.331+0000 7fbce1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.331+0000 7fbce1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45486/0 (socket says 172.21.15.90:45486) 2024-05-27T07:50:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.331+0000 7fbce1400640 1 -- 172.21.15.90:0/2349691350 learned_addr learned my addr 172.21.15.90:0/2349691350 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.332+0000 7fbce2800640 1 --2- 172.21.15.90:0/2349691350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317f2300 0x55ec31bb6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.333+0000 7fbce2800640 1 --2- 172.21.15.90:0/2349691350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 0x55ec31ba4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.334+0000 7fbce1400640 1 -- 172.21.15.90:0/2349691350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 msgr2=0x55ec31ba4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.335+0000 7fbce1400640 1 --2- 172.21.15.90:0/2349691350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 0x55ec31ba4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.335+0000 7fbce1400640 1 -- 172.21.15.90:0/2349691350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317f2300 msgr2=0x55ec31bb6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.335+0000 7fbce1400640 1 --2- 172.21.15.90:0/2349691350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317f2300 0x55ec31bb6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.337+0000 7fbce1400640 1 -- 172.21.15.90:0/2349691350 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec31bb76e0 con 0x55ec317bcd80 2024-05-27T07:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.337+0000 7fbce2800640 1 --2- 172.21.15.90:0/2349691350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 0x55ec31ba4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.338+0000 7fbce1400640 1 --2- 172.21.15.90:0/2349691350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb71a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005ab0 tx=0x7fbcd0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ba5314145f895c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.339+0000 7fbcce600640 1 -- 172.21.15.90:0/2349691350 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0049070 con 0x55ec317bcd80 2024-05-27T07:50:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.339+0000 7fbcce600640 1 -- 172.21.15.90:0/2349691350 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcd00054c0 con 0x55ec317bcd80 2024-05-27T07:50:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.339+0000 7fbcce600640 1 -- 172.21.15.90:0/2349691350 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0035040 con 0x55ec317bcd80 2024-05-27T07:50:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.342+0000 7fbcefa1b740 1 -- 172.21.15.90:0/2349691350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 msgr2=0x55ec31bb71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.342+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/2349691350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb71a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005ab0 tx=0x7fbcd0000bc0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.343+0000 7fbcefa1b740 1 -- 172.21.15.90:0/2349691350 shutdown_connections 2024-05-27T07:50:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.343+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/2349691350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb71a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.343+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/2349691350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 0x55ec31ba4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.343+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/2349691350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317f2300 0x55ec31bb6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.343+0000 7fbcefa1b740 1 -- 172.21.15.90:0/2349691350 >> 172.21.15.90:0/2349691350 conn(0x55ec31ba30b0 msgr2=0x55ec31bace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.344+0000 7fbcefa1b740 1 -- 172.21.15.90:0/2349691350 shutdown_connections 2024-05-27T07:50:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.344+0000 7fbcefa1b740 1 -- 172.21.15.90:0/2349691350 wait complete. 2024-05-27T07:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.345+0000 7fbcefa1b740 1 -- start start 2024-05-27T07:50:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.345+0000 7fbcefa1b740 1 Processor -- start 2024-05-27T07:50:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.346+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.347+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 0x55ec31bae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.347+0000 7fbcefa1b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317f2300 0x55ec31baebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.347+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41b8270 con 0x55ec317f2300 2024-05-27T07:50:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.347+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41b7ba0 con 0x55ec317bcd80 2024-05-27T07:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.348+0000 7fbcefa1b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbce41b8940 con 0x55ec317dd4d0 2024-05-27T07:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.348+0000 7fbcefa1b740 10 client.0 _ll_register_callbacks cb 0x7fbcd400b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.348+0000 7fbce2800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.348+0000 7fbce2800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45494/0 (socket says 172.21.15.90:45494) 2024-05-27T07:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.348+0000 7fbce2800640 1 -- 172.21.15.90:0/245171723 learned_addr learned my addr 172.21.15.90:0/245171723 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.349+0000 7fbce2800640 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317f2300 0x55ec31baebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.349+0000 7fbce1400640 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 0x55ec31bae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.350+0000 7fbce2800640 1 -- 172.21.15.90:0/245171723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 msgr2=0x55ec31bae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.351+0000 7fbce2800640 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 0x55ec31bae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.351+0000 7fbce2800640 1 -- 172.21.15.90:0/245171723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317f2300 msgr2=0x55ec31baebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.351+0000 7fbce2800640 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317f2300 0x55ec31baebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.352+0000 7fbce2800640 1 -- 172.21.15.90:0/245171723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec31baf130 con 0x55ec317bcd80 2024-05-27T07:50:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.352+0000 7fbce1400640 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 0x55ec31bae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:50:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.354+0000 7fbce2800640 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb3360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4005cd0 tx=0x7fbcd401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01fd0abb0fcd5ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:50.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.354+0000 7fbcc3400640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:50:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.355+0000 7fbcc3400640 1 -- 172.21.15.90:0/245171723 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4036040 con 0x55ec317bcd80 2024-05-27T07:50:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.355+0000 7fbcc3400640 1 -- 172.21.15.90:0/245171723 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fbcd4004020 con 0x55ec317bcd80 2024-05-27T07:50:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.356+0000 7fbcefa1b740 1 -- 172.21.15.90:0/245171723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd00054c0 con 0x55ec317bcd80 2024-05-27T07:50:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.357+0000 7fbcc3400640 1 -- 172.21.15.90:0/245171723 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbcd4004dc0 con 0x55ec317bcd80 2024-05-27T07:50:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.357+0000 7fbcefa1b740 10 client.35927 fetch_fsmap learned FSMap version 42 2024-05-27T07:50:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.357+0000 7fbcefa1b740 1 -- 172.21.15.90:0/245171723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ec31c24d00 con 0x55ec317bcd80 2024-05-27T07:50:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.358+0000 7fbcc3400640 1 -- 172.21.15.90:0/245171723 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbcd00054c0 con 0x55ec317bcd80 2024-05-27T07:50:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.358+0000 7fbcc3400640 1 -- 172.21.15.90:0/245171723 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fbcd4039070 con 0x55ec317bcd80 2024-05-27T07:50:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.359+0000 7fbcefa1b740 10 client.35927 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:50:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.359+0000 7fbcefa1b740 10 client.35927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.359+0000 7fbcefa1b740 20 client.35927 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.359+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ec317fe1b0 0x55ec31c27370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.359+0000 7fbcefa1b740 4 client.35927 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:50:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.360+0000 7fbcefa1b740 1 -- 172.21.15.90:0/245171723 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec31b0d2c0 con 0x55ec317fe1b0 2024-05-27T07:50:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.360+0000 7fbce1400640 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ec317fe1b0 0x55ec31c27370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.362+0000 7fbce1400640 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ec317fe1b0 0x55ec31c27370 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=747f6c96e16bbde server_cookie=9711ecf9d7761d1 in_seq=0 out_seq=0 2024-05-27T07:50:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.362+0000 7fbcc3400640 10 client.35927 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.444+0000 7fbcc3400640 1 -- 172.21.15.90:0/245171723 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ec31b0d2c0 con 0x55ec317fe1b0 2024-05-27T07:50:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.444+0000 7fbcc3400640 10 client.35927 handle_command_reply: tid=0 2024-05-27T07:50:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.445+0000 7fbcefa1b740 1 client.35927 shutdown 2024-05-27T07:50:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.445+0000 7fbcefa1b740 1 -- 172.21.15.90:0/245171723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 msgr2=0x55ec31bb3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.445+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb3360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4005cd0 tx=0x7fbcd401f000 comp rx=0 tx=0).stop 2024-05-27T07:50:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.446+0000 7fbcefa1b740 1 -- 172.21.15.90:0/245171723 shutdown_connections 2024-05-27T07:50:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.446+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ec317fe1b0 0x55ec31c27370 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.446+0000 7fbcc3400640 0 client.35927 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.446+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec317f2300 0x55ec31baebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.447+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec317dd4d0 0x55ec31bae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.448+0000 7fbcefa1b740 1 --2- 172.21.15.90:0/245171723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ec317bcd80 0x55ec31bb3360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.448+0000 7fbcefa1b740 1 -- 172.21.15.90:0/245171723 >> 172.21.15.90:0/245171723 conn(0x55ec31ba30b0 msgr2=0x55ec31baa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.448+0000 7fbcefa1b740 1 -- 172.21.15.90:0/245171723 shutdown_connections 2024-05-27T07:50:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.448+0000 7fbcefa1b740 1 -- 172.21.15.90:0/245171723 wait complete. 2024-05-27T07:50:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.448+0000 7fbcefa1b740 20 client.35927 trim_cache size 0 max 16384 2024-05-27T07:50:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:50:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:50:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:50:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.453+0000 7fbce3200640 1 -- 172.21.15.90:0/2409452108 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbcc80b2b70 msgr2=0x7fbcc80b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.453+0000 7fbce3200640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbcc80b2b70 0x7fbcc80b5080 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55ec31b9e550 tx=0x7fbcdc00cf60 comp rx=0 tx=0).stop 2024-05-27T07:50:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.453+0000 7fbce3200640 1 -- 172.21.15.90:0/2409452108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbce4104220 msgr2=0x7fbce410c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.453+0000 7fbce3200640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbce4104220 0x7fbce410c380 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00efd0 tx=0x7fbcdc00c6a0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.453+0000 7fbce3200640 1 -- 172.21.15.90:0/2409452108 shutdown_connections 2024-05-27T07:50:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.453+0000 7fbce3200640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbcc80b2b70 0x7fbcc80b5080 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.453+0000 7fbce3200640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbce4109ad0 0x7fbce410c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.453+0000 7fbce3200640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbce410ce00 0x7fbce41accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.453+0000 7fbce3200640 1 --2- 172.21.15.90:0/2409452108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbce4104220 0x7fbce410c380 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.453+0000 7fbce3200640 1 -- 172.21.15.90:0/2409452108 >> 172.21.15.90:0/2409452108 conn(0x7fbce4079100 msgr2=0x7fbce41022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.454+0000 7fbce3200640 1 -- 172.21.15.90:0/2409452108 shutdown_connections 2024-05-27T07:50:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:50:50.454+0000 7fbce3200640 1 -- 172.21.15.90:0/2409452108 wait complete. 2024-05-27T07:50:50.764 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:52.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:52 smithi090 ceph-mon[31429]: pgmap v141: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.6 MiB/s wr, 669 op/s 2024-05-27T07:50:52.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:52 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:50:52.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:52 smithi136 ceph-mon[35324]: pgmap v141: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.6 MiB/s wr, 669 op/s 2024-05-27T07:50:52.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:52 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:50:52.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:52 smithi129 ceph-mon[35254]: pgmap v141: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.6 MiB/s wr, 669 op/s 2024-05-27T07:50:52.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:52 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:50:53.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 35.0 s, 566.2 tps, lat 175.973 ms stddev 48.502 2024-05-27T07:50:54.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.113+0000 7f254b200700 1 -- 172.21.15.90:0/1533983799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25441092b0 msgr2=0x7f25441096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.113+0000 7f254b200700 1 --2- 172.21.15.90:0/1533983799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25441092b0 0x7f25441096c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f253c009f90 tx=0x7f253c00b330 comp rx=0 tx=0).stop 2024-05-27T07:50:54.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.114+0000 7f254b200700 1 -- 172.21.15.90:0/1533983799 shutdown_connections 2024-05-27T07:50:54.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.114+0000 7f254b200700 1 --2- 172.21.15.90:0/1533983799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2544103120 0x7f2544107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.114+0000 7f254b200700 1 --2- 172.21.15.90:0/1533983799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25441092b0 0x7f25441096c0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f253c009f90 tx=0x7f253c00b330 comp rx=0 tx=0).stop 2024-05-27T07:50:54.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.114+0000 7f254b200700 1 --2- 172.21.15.90:0/1533983799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25441027d0 0x7f2544102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.114+0000 7f254b200700 1 -- 172.21.15.90:0/1533983799 >> 172.21.15.90:0/1533983799 conn(0x7f25440fe6e0 msgr2=0x7f2544100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.114+0000 7f254b200700 1 -- 172.21.15.90:0/1533983799 shutdown_connections 2024-05-27T07:50:54.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.115+0000 7f254b200700 1 -- 172.21.15.90:0/1533983799 wait complete. 2024-05-27T07:50:54.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.116+0000 7f254b200700 1 Processor -- start 2024-05-27T07:50:54.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.117+0000 7f254b200700 1 -- start start 2024-05-27T07:50:54.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.117+0000 7f254b200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25441027d0 0x7f25441a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.117+0000 7f254b200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2544103120 0x7f25441a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.117+0000 7f254b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25441a5c90 0x7f25441a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.117+0000 7f254b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f254411a9f0 con 0x7f25441a5c90 2024-05-27T07:50:54.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.117+0000 7f254b200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f254411a870 con 0x7f2544103120 2024-05-27T07:50:54.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.117+0000 7f254b200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f254411ab70 con 0x7f25441027d0 2024-05-27T07:50:54.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.118+0000 7f2543e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25441027d0 0x7f25441a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.118+0000 7f2543e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25441027d0 0x7f25441a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56224/0 (socket says 172.21.15.90:56224) 2024-05-27T07:50:54.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.118+0000 7f2543e00700 1 -- 172.21.15.90:0/2377010098 learned_addr learned my addr 172.21.15.90:0/2377010098 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:54.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.118+0000 7f2549200700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25441a5c90 0x7f25441a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.118+0000 7f2543400700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2544103120 0x7f25441a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.119+0000 7f2543e00700 1 -- 172.21.15.90:0/2377010098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2544103120 msgr2=0x7f25441a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.119+0000 7f2543e00700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2544103120 0x7f25441a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.119+0000 7f2543e00700 1 -- 172.21.15.90:0/2377010098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25441a5c90 msgr2=0x7f25441a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.119+0000 7f2543e00700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25441a5c90 0x7f25441a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.119+0000 7f2543e00700 1 -- 172.21.15.90:0/2377010098 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25441a67e0 con 0x7f25441027d0 2024-05-27T07:50:54.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.120+0000 7f2543e00700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25441027d0 0x7f25441a1520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f253c009790 tx=0x7f253c00bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.120+0000 7f2540c00700 1 -- 172.21.15.90:0/2377010098 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c021020 con 0x7f25441027d0 2024-05-27T07:50:54.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.120+0000 7f254b200700 1 -- 172.21.15.90:0/2377010098 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25441a6a70 con 0x7f25441027d0 2024-05-27T07:50:54.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.121+0000 7f2540c00700 1 -- 172.21.15.90:0/2377010098 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f253c007ac0 con 0x7f25441027d0 2024-05-27T07:50:54.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.121+0000 7f2540c00700 1 -- 172.21.15.90:0/2377010098 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c01b430 con 0x7f25441027d0 2024-05-27T07:50:54.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.121+0000 7f254b200700 1 -- 172.21.15.90:0/2377010098 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25441ae170 con 0x7f25441027d0 2024-05-27T07:50:54.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.127+0000 7f2540c00700 1 -- 172.21.15.90:0/2377010098 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f253c024020 con 0x7f25441027d0 2024-05-27T07:50:54.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.128+0000 7f2559e39b80 1 Processor -- start 2024-05-27T07:50:54.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.130+0000 7f2559e39b80 1 -- start start 2024-05-27T07:50:54.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.131+0000 7f2540c00700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25280b2af0 0x7f25280b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.131+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 0x55d6547b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.131+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547b3d30 0x55d6547ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.131+0000 7f2540c00700 1 -- 172.21.15.90:0/2377010098 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f253c0e3b00 con 0x7f25441027d0 2024-05-27T07:50:54.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.132+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547addb0 0x55d6547ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.132+0000 7f2559e39b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d6546dbe40 con 0x55d6547b3d30 2024-05-27T07:50:54.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.132+0000 7f2559e39b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d6546cb9e0 con 0x55d6547addb0 2024-05-27T07:50:54.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.132+0000 7f2559e39b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d6546cfa20 con 0x55d65462ee40 2024-05-27T07:50:54.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.132+0000 7f2543400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547b3d30 0x55d6547ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.132+0000 7f2549200700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25280b2af0 0x7f25280b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.132+0000 7f2543400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547b3d30 0x55d6547ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39676/0 (socket says 172.21.15.90:39676) 2024-05-27T07:50:54.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.132+0000 7f2543400700 1 -- 172.21.15.90:0/2825219741 learned_addr learned my addr 172.21.15.90:0/2825219741 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:54.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.132+0000 7f2549200700 1 --2- 172.21.15.90:0/2825219741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547addb0 0x55d6547ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.132+0000 7f2543e00700 1 --2- 172.21.15.90:0/2825219741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 0x55d6547b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.133+0000 7f2543400700 1 -- 172.21.15.90:0/2825219741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 msgr2=0x55d6547b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.133+0000 7f2543400700 1 --2- 172.21.15.90:0/2825219741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 0x55d6547b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.133+0000 7f2543400700 1 -- 172.21.15.90:0/2825219741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547addb0 msgr2=0x55d6547ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.133+0000 7f2543400700 1 --2- 172.21.15.90:0/2825219741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547addb0 0x55d6547ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.133+0000 7f2543400700 1 -- 172.21.15.90:0/2825219741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d654694420 con 0x55d6547b3d30 2024-05-27T07:50:54.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.133+0000 7f2549200700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25280b2af0 0x7f25280b5000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2534009700 tx=0x7f2534009c50 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.133+0000 7f2543400700 1 --2- 172.21.15.90:0/2825219741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547b3d30 0x55d6547ad870 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f252c009c40 tx=0x7f252c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab75c790d551038 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.134+0000 7f2539600700 1 -- 172.21.15.90:0/2825219741 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c004220 con 0x55d6547b3d30 2024-05-27T07:50:54.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.134+0000 7f2539600700 1 -- 172.21.15.90:0/2825219741 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f252c011e00 con 0x55d6547b3d30 2024-05-27T07:50:54.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.134+0000 7f2539600700 1 -- 172.21.15.90:0/2825219741 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c003d90 con 0x55d6547b3d30 2024-05-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.136+0000 7f2559e39b80 1 -- 172.21.15.90:0/2825219741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547b3d30 msgr2=0x55d6547ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.136+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2825219741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547b3d30 0x55d6547ad870 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f252c009c40 tx=0x7f252c009f50 comp rx=0 tx=0).stop 2024-05-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.136+0000 7f2559e39b80 1 -- 172.21.15.90:0/2825219741 shutdown_connections 2024-05-27T07:50:54.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.136+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2825219741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547addb0 0x55d6547ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.136+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2825219741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 0x55d6547b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.136+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2825219741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547b3d30 0x55d6547ad870 secure :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f252c009c40 tx=0x7f252c009f50 comp rx=0 tx=0).stop 2024-05-27T07:50:54.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.136+0000 7f2559e39b80 1 -- 172.21.15.90:0/2825219741 >> 172.21.15.90:0/2825219741 conn(0x55d65468efc0 msgr2=0x55d6547a88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.136+0000 7f2559e39b80 1 -- 172.21.15.90:0/2825219741 shutdown_connections 2024-05-27T07:50:54.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.136+0000 7f2559e39b80 1 -- 172.21.15.90:0/2825219741 wait complete. 2024-05-27T07:50:54.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.137+0000 7f2559e39b80 1 -- start start 2024-05-27T07:50:54.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.138+0000 7f2559e39b80 1 Processor -- start 2024-05-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.138+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d65462ee40 0x55d6547b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.138+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547addb0 0x55d6547b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.138+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547b80d0 0x55d6547c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.138+0000 7f2559e39b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d6545d1fa0 con 0x55d65462ee40 2024-05-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.138+0000 7f2559e39b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d65466c760 con 0x55d6547b80d0 2024-05-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.138+0000 7f2559e39b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d6547ba800 con 0x55d6547addb0 2024-05-27T07:50:54.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.139+0000 7f2559e39b80 10 client.0 _ll_register_callbacks cb 0x55d654675c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.139+0000 7f2543e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d65462ee40 0x55d6547b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.139+0000 7f2543e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d65462ee40 0x55d6547b7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39690/0 (socket says 172.21.15.90:39690) 2024-05-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.139+0000 7f2543e00700 1 -- 172.21.15.90:0/2440489308 learned_addr learned my addr 172.21.15.90:0/2440489308 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.139+0000 7f2549200700 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547b80d0 0x55d6547c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.140+0000 7f2543e00700 1 -- 172.21.15.90:0/2440489308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547addb0 msgr2=0x55d6547b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.140+0000 7f2543e00700 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547addb0 0x55d6547b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.140+0000 7f2543e00700 1 -- 172.21.15.90:0/2440489308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547b80d0 msgr2=0x55d6547c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.140+0000 7f2543e00700 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547b80d0 0x55d6547c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.140+0000 7f2543e00700 1 -- 172.21.15.90:0/2440489308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65461b8d0 con 0x55d65462ee40 2024-05-27T07:50:54.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.140+0000 7f2543e00700 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d65462ee40 0x55d6547b7650 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f253c00b6c0 tx=0x7f253c0aa9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3716aaa205def4a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.140+0000 7f2532000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:50:54.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.141+0000 7f2532000700 1 -- 172.21.15.90:0/2440489308 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0e1330 con 0x55d65462ee40 2024-05-27T07:50:54.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.141+0000 7f2532000700 1 -- 172.21.15.90:0/2440489308 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f253c0aad20 con 0x55d65462ee40 2024-05-27T07:50:54.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.141+0000 7f2559e39b80 1 -- 172.21.15.90:0/2440489308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f252c011e00 con 0x55d65462ee40 2024-05-27T07:50:54.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.141+0000 7f2532000700 1 -- 172.21.15.90:0/2440489308 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f253c0b31d0 con 0x55d65462ee40 2024-05-27T07:50:54.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.141+0000 7f2559e39b80 10 client.16440 fetch_fsmap learned FSMap version 42 2024-05-27T07:50:54.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.141+0000 7f2559e39b80 1 -- 172.21.15.90:0/2440489308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d65469c060 con 0x55d65462ee40 2024-05-27T07:50:54.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.142+0000 7f2532000700 1 -- 172.21.15.90:0/2440489308 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f252c011e00 con 0x55d65462ee40 2024-05-27T07:50:54.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.142+0000 7f2532000700 1 -- 172.21.15.90:0/2440489308 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f253c0cbce0 con 0x55d65462ee40 2024-05-27T07:50:54.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.142+0000 7f2559e39b80 10 client.16440 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:50:54.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.142+0000 7f2559e39b80 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:54.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.142+0000 7f2559e39b80 20 client.16440 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:54.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.142+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d6547c7cf0 0x55d6547ca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.142+0000 7f2559e39b80 4 client.16440 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:50:54.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.142+0000 7f2559e39b80 1 -- 172.21.15.90:0/2440489308 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6546ddd90 con 0x55d6547c7cf0 2024-05-27T07:50:54.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.143+0000 7f2543400700 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d6547c7cf0 0x55d6547ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.144+0000 7f2543400700 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d6547c7cf0 0x55d6547ca200 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=997813f4ceee8cc9 server_cookie=f6941e55125ce4d in_seq=0 out_seq=0 2024-05-27T07:50:54.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.144+0000 7f2532000700 10 client.16440 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:54.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.146+0000 7f2532000700 1 -- 172.21.15.90:0/2440489308 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d6546ddd90 con 0x55d6547c7cf0 2024-05-27T07:50:54.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.146+0000 7f2532000700 10 client.16440 handle_command_reply: tid=0 2024-05-27T07:50:54.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.146+0000 7f2559e39b80 1 client.16440 shutdown 2024-05-27T07:50:54.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.147+0000 7f2559e39b80 1 -- 172.21.15.90:0/2440489308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d65462ee40 msgr2=0x55d6547b7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.147+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d65462ee40 0x55d6547b7650 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f253c00b6c0 tx=0x7f253c0aa9c0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.147+0000 7f2559e39b80 1 -- 172.21.15.90:0/2440489308 shutdown_connections 2024-05-27T07:50:54.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.147+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d6547c7cf0 0x55d6547ca200 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.147+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547b80d0 0x55d6547c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.147+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d65462ee40 0x55d6547b7650 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.147+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2440489308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547addb0 0x55d6547b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.147+0000 7f2559e39b80 1 -- 172.21.15.90:0/2440489308 >> 172.21.15.90:0/2440489308 conn(0x55d65468efc0 msgr2=0x55d6547a8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.147+0000 7f2532000700 0 client.16440 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:54.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.148+0000 7f2559e39b80 1 -- 172.21.15.90:0/2440489308 shutdown_connections 2024-05-27T07:50:54.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.148+0000 7f2559e39b80 1 -- 172.21.15.90:0/2440489308 wait complete. 2024-05-27T07:50:54.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.148+0000 7f2559e39b80 20 client.16440 trim_cache size 0 max 16384 2024-05-27T07:50:54.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.177+0000 7f2559e39b80 1 Processor -- start 2024-05-27T07:50:54.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.177+0000 7f2559e39b80 1 -- start start 2024-05-27T07:50:54.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.178+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 0x55d6547ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.178+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 0x55d6547b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.178+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547b80d0 0x55d6547b6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.178+0000 7f2559e39b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b89b0 con 0x55d6547addb0 2024-05-27T07:50:54.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.178+0000 7f2559e39b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b67a0 con 0x55d6547b80d0 2024-05-27T07:50:54.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.178+0000 7f2559e39b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b82e0 con 0x55d65462ee40 2024-05-27T07:50:54.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.179+0000 7f2543e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 0x55d6547ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.179+0000 7f2543e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 0x55d6547ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56248/0 (socket says 172.21.15.90:56248) 2024-05-27T07:50:54.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.179+0000 7f2543e00700 1 -- 172.21.15.90:0/2949918912 learned_addr learned my addr 172.21.15.90:0/2949918912 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:54.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.179+0000 7f2549200700 1 --2- 172.21.15.90:0/2949918912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547b80d0 0x55d6547b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.179+0000 7f2543400700 1 --2- 172.21.15.90:0/2949918912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 0x55d6547b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.179+0000 7f2543e00700 1 -- 172.21.15.90:0/2949918912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547b80d0 msgr2=0x55d6547b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.179+0000 7f2543e00700 1 --2- 172.21.15.90:0/2949918912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547b80d0 0x55d6547b6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.179+0000 7f2543e00700 1 -- 172.21.15.90:0/2949918912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 msgr2=0x55d6547b9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.180+0000 7f2543e00700 1 --2- 172.21.15.90:0/2949918912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 0x55d6547b9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.180+0000 7f2543e00700 1 -- 172.21.15.90:0/2949918912 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6547a2130 con 0x55d65462ee40 2024-05-27T07:50:54.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.180+0000 7f2543400700 1 --2- 172.21.15.90:0/2949918912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 0x55d6547b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:50:54.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.182+0000 7f2543e00700 1 --2- 172.21.15.90:0/2949918912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 0x55d6547ad870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f253c007f90 tx=0x7f253c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36e605e31381983a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.182+0000 7f2538c00700 1 -- 172.21.15.90:0/2949918912 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0e1330 con 0x55d65462ee40 2024-05-27T07:50:54.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.182+0000 7f2538c00700 1 -- 172.21.15.90:0/2949918912 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f253c0aa5d0 con 0x55d65462ee40 2024-05-27T07:50:54.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.182+0000 7f2538c00700 1 -- 172.21.15.90:0/2949918912 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c0aa230 con 0x55d65462ee40 2024-05-27T07:50:54.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.184+0000 7f2559e39b80 1 -- 172.21.15.90:0/2949918912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 msgr2=0x55d6547ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.184+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2949918912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 0x55d6547ad870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f253c007f90 tx=0x7f253c01b000 comp rx=0 tx=0).stop 2024-05-27T07:50:54.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.184+0000 7f2559e39b80 1 -- 172.21.15.90:0/2949918912 shutdown_connections 2024-05-27T07:50:54.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.184+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2949918912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6547b80d0 0x55d6547b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.184+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2949918912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d65462ee40 0x55d6547ad870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.184+0000 7f2559e39b80 1 --2- 172.21.15.90:0/2949918912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 0x55d6547b9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.184+0000 7f2559e39b80 1 -- 172.21.15.90:0/2949918912 >> 172.21.15.90:0/2949918912 conn(0x55d65468efc0 msgr2=0x55d6547b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.185+0000 7f2559e39b80 1 -- 172.21.15.90:0/2949918912 shutdown_connections 2024-05-27T07:50:54.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.185+0000 7f2559e39b80 1 -- 172.21.15.90:0/2949918912 wait complete. 2024-05-27T07:50:54.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.185+0000 7f2559e39b80 1 -- start start 2024-05-27T07:50:54.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.185+0000 7f2559e39b80 1 Processor -- start 2024-05-27T07:50:54.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.186+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d65462ee40 0x55d6547c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.186+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 0x55d6547bfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.186+0000 7f2559e39b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547b80d0 0x55d6547bff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.187+0000 7f2559e39b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b7540 con 0x55d6547addb0 2024-05-27T07:50:54.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.187+0000 7f2559e39b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b6e70 con 0x55d65462ee40 2024-05-27T07:50:54.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.187+0000 7f2559e39b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f25441b7c10 con 0x55d6547b80d0 2024-05-27T07:50:54.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.187+0000 7f2559e39b80 10 client.0 _ll_register_callbacks cb 0x55d654675c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:50:54.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.187+0000 7f2543e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d65462ee40 0x55d6547c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.187+0000 7f2549200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547b80d0 0x55d6547bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.187+0000 7f2549200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547b80d0 0x55d6547bff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56262/0 (socket says 172.21.15.90:56262) 2024-05-27T07:50:54.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.187+0000 7f2549200700 1 -- 172.21.15.90:0/3616718741 learned_addr learned my addr 172.21.15.90:0/3616718741 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:50:54.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.188+0000 7f2543400700 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 0x55d6547bfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.188+0000 7f2549200700 1 -- 172.21.15.90:0/3616718741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d65462ee40 msgr2=0x55d6547c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.188+0000 7f2549200700 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d65462ee40 0x55d6547c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.188+0000 7f2549200700 1 -- 172.21.15.90:0/3616718741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 msgr2=0x55d6547bfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:50:54.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.189+0000 7f2549200700 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 0x55d6547bfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.189+0000 7f2549200700 1 -- 172.21.15.90:0/3616718741 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6547c0500 con 0x55d6547b80d0 2024-05-27T07:50:54.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.189+0000 7f2543400700 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 0x55d6547bfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:50:54.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.189+0000 7f2549200700 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547b80d0 0x55d6547bff90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2534011f20 tx=0x7f2534009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3dc8648d678f1c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:50:54.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.189+0000 7f2532000700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:50:54.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.190+0000 7f2532000700 1 -- 172.21.15.90:0/3616718741 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253402e060 con 0x55d6547b80d0 2024-05-27T07:50:54.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.190+0000 7f2532000700 1 -- 172.21.15.90:0/3616718741 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f253400fa10 con 0x55d6547b80d0 2024-05-27T07:50:54.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.191+0000 7f2559e39b80 1 -- 172.21.15.90:0/3616718741 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f253c0aa5d0 con 0x55d6547b80d0 2024-05-27T07:50:54.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.191+0000 7f2532000700 1 -- 172.21.15.90:0/3616718741 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f253400fc10 con 0x55d6547b80d0 2024-05-27T07:50:54.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.191+0000 7f2559e39b80 10 client.25996 fetch_fsmap learned FSMap version 42 2024-05-27T07:50:54.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.191+0000 7f2559e39b80 1 -- 172.21.15.90:0/3616718741 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d6548318d0 con 0x55d6547b80d0 2024-05-27T07:50:54.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.192+0000 7f2532000700 1 -- 172.21.15.90:0/3616718741 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f253c0aa5d0 con 0x55d6547b80d0 2024-05-27T07:50:54.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.192+0000 7f2532000700 1 -- 172.21.15.90:0/3616718741 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f2534029440 con 0x55d6547b80d0 2024-05-27T07:50:54.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.192+0000 7f2559e39b80 10 client.25996 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:50:54.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.192+0000 7f2559e39b80 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:50:54.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.192+0000 7f2559e39b80 20 client.25996 populate_metadata read hostname 'smithi090' 2024-05-27T07:50:54.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.192+0000 7f2559e39b80 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d654831ea0 0x55d6548342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:50:54.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.192+0000 7f2559e39b80 4 client.25996 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:50:54.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.192+0000 7f2559e39b80 1 -- 172.21.15.90:0/3616718741 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6546d7570 con 0x55d654831ea0 2024-05-27T07:50:54.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.193+0000 7f2543e00700 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d654831ea0 0x55d6548342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:50:54.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.196+0000 7f2543e00700 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d654831ea0 0x55d6548342c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a787978ad31ecf server_cookie=6f8e72ab23dc611a in_seq=0 out_seq=0 2024-05-27T07:50:54.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.196+0000 7f2532000700 10 client.25996 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:54.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.197+0000 7f2532000700 1 -- 172.21.15.90:0/3616718741 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6546d7570 con 0x55d654831ea0 2024-05-27T07:50:54.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.197+0000 7f2532000700 10 client.25996 handle_command_reply: tid=0 2024-05-27T07:50:54.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.197+0000 7f2559e39b80 1 client.25996 shutdown 2024-05-27T07:50:54.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.197+0000 7f2559e39b80 1 -- 172.21.15.90:0/3616718741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547b80d0 msgr2=0x55d6547bff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.197+0000 7f2559e39b80 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547b80d0 0x55d6547bff90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2534011f20 tx=0x7f2534009310 comp rx=0 tx=0).stop 2024-05-27T07:50:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.198+0000 7f2559e39b80 1 -- 172.21.15.90:0/3616718741 shutdown_connections 2024-05-27T07:50:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.198+0000 7f2559e39b80 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d654831ea0 0x55d6548342c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.198+0000 7f2559e39b80 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6547b80d0 0x55d6547bff90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.198+0000 7f2559e39b80 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d65462ee40 0x55d6547c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.198+0000 7f2559e39b80 1 --2- 172.21.15.90:0/3616718741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6547addb0 0x55d6547bfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.198+0000 7f2559e39b80 1 -- 172.21.15.90:0/3616718741 >> 172.21.15.90:0/3616718741 conn(0x55d65468efc0 msgr2=0x55d6547b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.198+0000 7f2532000700 0 client.25996 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:50:54.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.198+0000 7f2559e39b80 1 -- 172.21.15.90:0/3616718741 shutdown_connections 2024-05-27T07:50:54.200 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:50:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.198+0000 7f2559e39b80 1 -- 172.21.15.90:0/3616718741 wait complete. 2024-05-27T07:50:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.199+0000 7f2559e39b80 20 client.25996 trim_cache size 0 max 16384 2024-05-27T07:50:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.200+0000 7f254b200700 1 -- 172.21.15.90:0/2377010098 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25280b2af0 msgr2=0x7f25280b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.200+0000 7f254b200700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25280b2af0 0x7f25280b5000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2534009700 tx=0x7f2534009c50 comp rx=0 tx=0).stop 2024-05-27T07:50:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.200+0000 7f254b200700 1 -- 172.21.15.90:0/2377010098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25441027d0 msgr2=0x7f25441a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:50:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.200+0000 7f254b200700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25441027d0 0x7f25441a1520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f253c009790 tx=0x7f253c00bd50 comp rx=0 tx=0).stop 2024-05-27T07:50:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.200+0000 7f254b200700 1 -- 172.21.15.90:0/2377010098 shutdown_connections 2024-05-27T07:50:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.200+0000 7f254b200700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25441a5c90 0x7f25441a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.200+0000 7f254b200700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25441027d0 0x7f25441a1520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.200+0000 7f254b200700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25280b2af0 0x7f25280b5000 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.200+0000 7f254b200700 1 --2- 172.21.15.90:0/2377010098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2544103120 0x7f25441a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:50:54.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.200+0000 7f254b200700 1 -- 172.21.15.90:0/2377010098 >> 172.21.15.90:0/2377010098 conn(0x7f25440fe6e0 msgr2=0x7f2544106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:50:54.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.201+0000 7f254b200700 1 -- 172.21.15.90:0/2377010098 shutdown_connections 2024-05-27T07:50:54.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:50:54.201+0000 7f254b200700 1 -- 172.21.15.90:0/2377010098 wait complete. 2024-05-27T07:50:54.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:54 smithi090 ceph-mon[31429]: pgmap v142: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.0 MiB/s wr, 959 op/s 2024-05-27T07:50:54.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:54 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:50:54.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:54 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:50:54.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:54 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:50:54.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:54 smithi136 ceph-mon[35324]: pgmap v142: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.0 MiB/s wr, 959 op/s 2024-05-27T07:50:54.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:54 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:50:54.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:54 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:50:54.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:54 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:50:54.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:54 smithi129 ceph-mon[35254]: pgmap v142: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.0 MiB/s wr, 959 op/s 2024-05-27T07:50:54.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:54 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:50:54.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:54 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:50:54.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:54 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:50:56.367 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:56 smithi129 ceph-mon[35254]: pgmap v143: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.9 MiB/s wr, 578 op/s 2024-05-27T07:50:56.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:56 smithi090 ceph-mon[31429]: pgmap v143: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.9 MiB/s wr, 578 op/s 2024-05-27T07:50:56.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:56 smithi136 ceph-mon[35324]: pgmap v143: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.9 MiB/s wr, 578 op/s 2024-05-27T07:50:56.740 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:50:56.741 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:50:57.418 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:50:58.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:50:58 smithi090 ceph-mon[31429]: pgmap v144: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.9 MiB/s wr, 579 op/s 2024-05-27T07:50:58.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:50:58 smithi136 ceph-mon[35324]: pgmap v144: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.9 MiB/s wr, 579 op/s 2024-05-27T07:50:58.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:50:58 smithi129 ceph-mon[35254]: pgmap v144: 129 pgs: 129 active+clean; 1.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 4.9 MiB/s wr, 579 op/s 2024-05-27T07:50:58.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 40.0 s, 600.0 tps, lat 168.355 ms stddev 44.945 2024-05-27T07:51:00.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:00 smithi090 ceph-mon[31429]: pgmap v145: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.4 MiB/s wr, 882 op/s 2024-05-27T07:51:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:00 smithi136 ceph-mon[35324]: pgmap v145: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.4 MiB/s wr, 882 op/s 2024-05-27T07:51:00.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:00 smithi129 ceph-mon[35254]: pgmap v145: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.4 MiB/s wr, 882 op/s 2024-05-27T07:51:02.199 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:02 smithi090 ceph-mon[31429]: pgmap v146: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.0 MiB/s wr, 593 op/s 2024-05-27T07:51:02.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:02 smithi136 ceph-mon[35324]: pgmap v146: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.0 MiB/s wr, 593 op/s 2024-05-27T07:51:02.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:02 smithi129 ceph-mon[35254]: pgmap v146: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.0 MiB/s wr, 593 op/s 2024-05-27T07:51:03.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:03 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:51:03.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:03 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:03.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:03 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:51:03.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:03 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:51:03.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:03 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:03.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:03 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:51:03.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.537+0000 7f4904c00700 1 -- 172.21.15.90:0/875528328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4900108ee0 msgr2=0x7f49001092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.537+0000 7f4904c00700 1 --2- 172.21.15.90:0/875528328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4900108ee0 0x7f49001092f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0099c0 tx=0x7f48ec009cd0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.538+0000 7f4904c00700 1 -- 172.21.15.90:0/875528328 shutdown_connections 2024-05-27T07:51:03.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.538+0000 7f4904c00700 1 --2- 172.21.15.90:0/875528328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4900109830 0x7f4900111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.538+0000 7f4904c00700 1 --2- 172.21.15.90:0/875528328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49001045b0 0x7f49001049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.538+0000 7f4904c00700 1 --2- 172.21.15.90:0/875528328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4900108ee0 0x7f49001092f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.538+0000 7f4904c00700 1 -- 172.21.15.90:0/875528328 >> 172.21.15.90:0/875528328 conn(0x7f49000ffe40 msgr2=0x7f4900102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.538+0000 7f4904c00700 1 -- 172.21.15.90:0/875528328 shutdown_connections 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.538+0000 7f4904c00700 1 -- 172.21.15.90:0/875528328 wait complete. 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.542+0000 7f4904c00700 1 Processor -- start 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.542+0000 7f4904c00700 1 -- start start 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.546+0000 7f4904c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49001045b0 0x7f490010e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.546+0000 7f4904c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4900108ee0 0x7f490010e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.546+0000 7f4904c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4900109830 0x7f490010eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.546+0000 7f4904c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4900078310 con 0x7f49001045b0 2024-05-27T07:51:03.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.546+0000 7f4904c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4900078190 con 0x7f4900109830 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.547+0000 7f4904c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4900078490 con 0x7f4900108ee0 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.547+0000 7f48fe800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4900109830 0x7f490010eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.549+0000 7f48fe800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4900109830 0x7f490010eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50402/0 (socket says 172.21.15.90:50402) 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.549+0000 7f48fe800700 1 -- 172.21.15.90:0/1794623740 learned_addr learned my addr 172.21.15.90:0/1794623740 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.549+0000 7f48fde00700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49001045b0 0x7f490010e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.549+0000 7f48fe800700 1 -- 172.21.15.90:0/1794623740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4900108ee0 msgr2=0x7f490010e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.548+0000 7f48fd400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4900108ee0 0x7f490010e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.550+0000 7f48fe800700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4900108ee0 0x7f490010e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.550+0000 7f48fe800700 1 -- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49001045b0 msgr2=0x7f490010e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.550+0000 7f48fe800700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49001045b0 0x7f490010e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.550+0000 7f48fe800700 1 -- 172.21.15.90:0/1794623740 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4900074270 con 0x7f4900109830 2024-05-27T07:51:03.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.550+0000 7f48fde00700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49001045b0 0x7f490010e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.551+0000 7f48fd400700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4900108ee0 0x7f490010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.551+0000 7f48fe800700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4900109830 0x7f490010eef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f48f800ef30 tx=0x7f48f800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.552+0000 7f48f6a00700 1 -- 172.21.15.90:0/1794623740 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80092e0 con 0x7f4900109830 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.552+0000 7f48f6a00700 1 -- 172.21.15.90:0/1794623740 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48f8004bb0 con 0x7f4900109830 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.552+0000 7f48f6a00700 1 -- 172.21.15.90:0/1794623740 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8004020 con 0x7f4900109830 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.552+0000 7f4904c00700 1 -- 172.21.15.90:0/1794623740 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4900074500 con 0x7f4900109830 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.552+0000 7f4904c00700 1 -- 172.21.15.90:0/1794623740 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49000749f0 con 0x7f4900109830 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.562+0000 7f48f6a00700 1 -- 172.21.15.90:0/1794623740 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f48f801b030 con 0x7f4900109830 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.563+0000 7f491372fb80 1 Processor -- start 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.566+0000 7f48f6a00700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f48e40b2b40 0x7f48e40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.566+0000 7f48f6a00700 1 -- 172.21.15.90:0/1794623740 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f48f80e2390 con 0x7f4900109830 2024-05-27T07:51:03.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.567+0000 7f48fd400700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f48e40b2b40 0x7f48e40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.567+0000 7f491372fb80 1 -- start start 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.569+0000 7f491372fb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f80501870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.569+0000 7f491372fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f80501db0 0x561f804fb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.569+0000 7f491372fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 0x561f804fc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.570+0000 7f491372fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561f80429ec0 con 0x561f804fbe30 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.570+0000 7f491372fb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561f80419a60 con 0x561f8037cec0 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.570+0000 7f48fe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 0x561f804fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.570+0000 7f491372fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561f8041daa0 con 0x561f80501db0 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.571+0000 7f48fe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 0x561f804fc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40710/0 (socket says 172.21.15.90:40710) 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.572+0000 7f48fe800700 1 -- 172.21.15.90:0/3493529781 learned_addr learned my addr 172.21.15.90:0/3493529781 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.572+0000 7f48fe800700 1 -- 172.21.15.90:0/3493529781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f80501db0 msgr2=0x561f804fb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.573+0000 7f48fe800700 1 --2- 172.21.15.90:0/3493529781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f80501db0 0x561f804fb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.573+0000 7f48fe800700 1 -- 172.21.15.90:0/3493529781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 msgr2=0x561f80501870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.573+0000 7f48fe800700 1 --2- 172.21.15.90:0/3493529781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f80501870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.573+0000 7f48fe800700 1 -- 172.21.15.90:0/3493529781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f803e24a0 con 0x561f804fbe30 2024-05-27T07:51:03.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.573+0000 7f48fe800700 1 --2- 172.21.15.90:0/3493529781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 0x561f804fc240 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f48f80a9320 tx=0x7f48f80a8f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687fffa134d0faf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.574+0000 7f48eb400700 1 -- 172.21.15.90:0/3493529781 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80ad740 con 0x561f804fbe30 2024-05-27T07:51:03.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.574+0000 7f48eb400700 1 -- 172.21.15.90:0/3493529781 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48f80b2160 con 0x561f804fbe30 2024-05-27T07:51:03.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.574+0000 7f48eb400700 1 -- 172.21.15.90:0/3493529781 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80a4d00 con 0x561f804fbe30 2024-05-27T07:51:03.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.575+0000 7f491372fb80 1 -- 172.21.15.90:0/3493529781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 msgr2=0x561f804fc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.575+0000 7f491372fb80 1 --2- 172.21.15.90:0/3493529781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 0x561f804fc240 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f48f80a9320 tx=0x7f48f80a8f20 comp rx=0 tx=0).stop 2024-05-27T07:51:03.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.576+0000 7f48fd400700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f48e40b2b40 0x7f48e40b5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f48ec000c00 tx=0x7f48ec009530 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.576+0000 7f491372fb80 1 -- 172.21.15.90:0/3493529781 shutdown_connections 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.576+0000 7f491372fb80 1 --2- 172.21.15.90:0/3493529781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 0x561f804fc240 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.576+0000 7f491372fb80 1 --2- 172.21.15.90:0/3493529781 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f80501870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.576+0000 7f491372fb80 1 --2- 172.21.15.90:0/3493529781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f80501db0 0x561f804fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.576+0000 7f491372fb80 1 -- 172.21.15.90:0/3493529781 >> 172.21.15.90:0/3493529781 conn(0x561f803dd040 msgr2=0x561f804f6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.576+0000 7f491372fb80 1 -- 172.21.15.90:0/3493529781 shutdown_connections 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.577+0000 7f491372fb80 1 -- 172.21.15.90:0/3493529781 wait complete. 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.578+0000 7f491372fb80 1 -- start start 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.578+0000 7f491372fb80 1 Processor -- start 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.580+0000 7f491372fb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f80505690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.580+0000 7f491372fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 0x561f80505bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.580+0000 7f491372fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f80501db0 0x561f80506110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.580+0000 7f491372fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561f80320020 con 0x561f804fbe30 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.580+0000 7f491372fb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561f803ba7e0 con 0x561f8037cec0 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.580+0000 7f491372fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561f80508820 con 0x561f80501db0 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.580+0000 7f491372fb80 10 client.0 _ll_register_callbacks cb 0x561f803c3cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.582+0000 7f48fde00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f80505690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.582+0000 7f48fde00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f80505690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50406/0 (socket says 172.21.15.90:50406) 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.582+0000 7f48fde00700 1 -- 172.21.15.90:0/534727695 learned_addr learned my addr 172.21.15.90:0/534727695 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.582+0000 7f48fd400700 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 0x561f80505bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.582+0000 7f48fe800700 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f80501db0 0x561f80506110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.583+0000 7f48fe800700 1 -- 172.21.15.90:0/534727695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 msgr2=0x561f80505690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.583+0000 7f48fe800700 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f80505690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.583+0000 7f48fe800700 1 -- 172.21.15.90:0/534727695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 msgr2=0x561f80505bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.583+0000 7f48fe800700 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 0x561f80505bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.583+0000 7f48fe800700 1 -- 172.21.15.90:0/534727695 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f80369950 con 0x561f80501db0 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.584+0000 7f48fe800700 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f80501db0 0x561f80506110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x561f80502320 tx=0x7f48f80b1d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a678b401e32ac12 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.584+0000 7f48dfe00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.585+0000 7f48dfe00700 1 -- 172.21.15.90:0/534727695 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80c8060 con 0x561f80501db0 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.585+0000 7f491372fb80 1 -- 172.21.15.90:0/534727695 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f80b2160 con 0x561f80501db0 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.585+0000 7f48dfe00700 1 -- 172.21.15.90:0/534727695 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f48f80a85f0 con 0x561f80501db0 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.585+0000 7f48dfe00700 1 -- 172.21.15.90:0/534727695 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f48f80ca300 con 0x561f80501db0 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.585+0000 7f491372fb80 10 client.26002 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.585+0000 7f491372fb80 1 -- 172.21.15.90:0/534727695 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561f803ea0e0 con 0x561f80501db0 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.586+0000 7f48dfe00700 1 -- 172.21.15.90:0/534727695 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48f80b2160 con 0x561f80501db0 2024-05-27T07:51:03.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.586+0000 7f48dfe00700 1 -- 172.21.15.90:0/534727695 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f48f80a3ce0 con 0x561f80501db0 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.586+0000 7f491372fb80 10 client.26002 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.586+0000 7f491372fb80 10 client.26002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.586+0000 7f491372fb80 20 client.26002 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.586+0000 7f491372fb80 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561f805156c0 0x561f80517ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.586+0000 7f491372fb80 4 client.26002 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.586+0000 7f491372fb80 1 -- 172.21.15.90:0/534727695 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f8042be10 con 0x561f805156c0 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.587+0000 7f48fde00700 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561f805156c0 0x561f80517ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.588+0000 7f48fde00700 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561f805156c0 0x561f80517ae0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6e9bf35dc75db6 server_cookie=631d315d96b4cb70 in_seq=0 out_seq=0 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.588+0000 7f48dfe00700 10 client.26002 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.589+0000 7f48dfe00700 1 -- 172.21.15.90:0/534727695 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561f8042be10 con 0x561f805156c0 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.589+0000 7f48dfe00700 10 client.26002 handle_command_reply: tid=0 2024-05-27T07:51:03.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.589+0000 7f491372fb80 1 client.26002 shutdown 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.589+0000 7f491372fb80 1 -- 172.21.15.90:0/534727695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f80501db0 msgr2=0x561f80506110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.589+0000 7f491372fb80 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f80501db0 0x561f80506110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x561f80502320 tx=0x7f48f80b1d40 comp rx=0 tx=0).stop 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.590+0000 7f491372fb80 1 -- 172.21.15.90:0/534727695 shutdown_connections 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.590+0000 7f491372fb80 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561f805156c0 0x561f80517ae0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.590+0000 7f491372fb80 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f80501db0 0x561f80506110 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.590+0000 7f491372fb80 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f80505690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.590+0000 7f491372fb80 1 --2- 172.21.15.90:0/534727695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f804fbe30 0x561f80505bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.590+0000 7f491372fb80 1 -- 172.21.15.90:0/534727695 >> 172.21.15.90:0/534727695 conn(0x561f803dd040 msgr2=0x561f804f6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.590+0000 7f48dfe00700 0 client.26002 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.590+0000 7f491372fb80 1 -- 172.21.15.90:0/534727695 shutdown_connections 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.590+0000 7f491372fb80 1 -- 172.21.15.90:0/534727695 wait complete. 2024-05-27T07:51:03.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.591+0000 7f491372fb80 20 client.26002 trim_cache size 0 max 16384 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.614+0000 7f491372fb80 1 Processor -- start 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.614+0000 7f491372fb80 1 -- start start 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.614+0000 7f491372fb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f804fb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.614+0000 7f491372fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f804fbe30 0x561f80507b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.614+0000 7f491372fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f804ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.614+0000 7f491372fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f49001c0d70 con 0x561f80501db0 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.614+0000 7f491372fb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49001beb60 con 0x561f8037cec0 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.614+0000 7f491372fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49001c06a0 con 0x561f804fbe30 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.614+0000 7f48fe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f804ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.614+0000 7f48fe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f804ffcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40726/0 (socket says 172.21.15.90:40726) 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.615+0000 7f48fe800700 1 -- 172.21.15.90:0/3149573740 learned_addr learned my addr 172.21.15.90:0/3149573740 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.615+0000 7f48fd400700 1 --2- 172.21.15.90:0/3149573740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f804fbe30 0x561f80507b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.615+0000 7f48fde00700 1 --2- 172.21.15.90:0/3149573740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f804fb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.615+0000 7f48fe800700 1 -- 172.21.15.90:0/3149573740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f804fbe30 msgr2=0x561f80507b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.615+0000 7f48fe800700 1 --2- 172.21.15.90:0/3149573740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f804fbe30 0x561f80507b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.615+0000 7f48fe800700 1 -- 172.21.15.90:0/3149573740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 msgr2=0x561f804fb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.615+0000 7f48fe800700 1 --2- 172.21.15.90:0/3149573740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f804fb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.615+0000 7f48fe800700 1 -- 172.21.15.90:0/3149573740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f80503c00 con 0x561f80501db0 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.615+0000 7f48fe800700 1 --2- 172.21.15.90:0/3149573740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f804ffcc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009510 tx=0x7f48f80a8f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b476c540f160291 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.616+0000 7f48eaa00700 1 -- 172.21.15.90:0/3149573740 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80c8070 con 0x561f80501db0 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.616+0000 7f48eaa00700 1 -- 172.21.15.90:0/3149573740 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48f80a4d00 con 0x561f80501db0 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.616+0000 7f48eaa00700 1 -- 172.21.15.90:0/3149573740 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8003c90 con 0x561f80501db0 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.617+0000 7f491372fb80 1 -- 172.21.15.90:0/3149573740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 msgr2=0x561f804ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.617+0000 7f491372fb80 1 --2- 172.21.15.90:0/3149573740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f804ffcc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009510 tx=0x7f48f80a8f20 comp rx=0 tx=0).stop 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.617+0000 7f491372fb80 1 -- 172.21.15.90:0/3149573740 shutdown_connections 2024-05-27T07:51:03.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.617+0000 7f491372fb80 1 --2- 172.21.15.90:0/3149573740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f804ffcc0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.617+0000 7f491372fb80 1 --2- 172.21.15.90:0/3149573740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f8037cec0 0x561f804fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.617+0000 7f491372fb80 1 --2- 172.21.15.90:0/3149573740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f804fbe30 0x561f80507b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.617+0000 7f491372fb80 1 -- 172.21.15.90:0/3149573740 >> 172.21.15.90:0/3149573740 conn(0x561f803dd040 msgr2=0x561f80505470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.617+0000 7f491372fb80 1 -- 172.21.15.90:0/3149573740 shutdown_connections 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.617+0000 7f491372fb80 1 -- 172.21.15.90:0/3149573740 wait complete. 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.618+0000 7f491372fb80 1 -- start start 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.618+0000 7f491372fb80 1 Processor -- start 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f491372fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f8037cec0 0x561f805184c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f491372fb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f804fbe30 0x561f80518a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f491372fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f80518f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f491372fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f49001bf900 con 0x561f80501db0 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f491372fb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49001bf230 con 0x561f804fbe30 2024-05-27T07:51:03.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f491372fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49001bffd0 con 0x561f8037cec0 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f491372fb80 10 client.0 _ll_register_callbacks cb 0x561f803c3cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f48fe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f80518f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f48fe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f80518f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40730/0 (socket says 172.21.15.90:40730) 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f48fe800700 1 -- 172.21.15.90:0/2219773508 learned_addr learned my addr 172.21.15.90:0/2219773508 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f48fde00700 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f8037cec0 0x561f805184c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.619+0000 7f48fd400700 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f804fbe30 0x561f80518a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48fe800700 1 -- 172.21.15.90:0/2219773508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f8037cec0 msgr2=0x561f805184c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48fe800700 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f8037cec0 0x561f805184c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48fe800700 1 -- 172.21.15.90:0/2219773508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f804fbe30 msgr2=0x561f80518a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48fe800700 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f804fbe30 0x561f80518a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48fe800700 1 -- 172.21.15.90:0/2219773508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f8057ef30 con 0x561f80501db0 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48fde00700 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f8037cec0 0x561f805184c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:03.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48fe800700 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f80518f40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f48f80a9040 tx=0x7f48f8009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e0fb43e99d2732 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48dfe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48dfe00700 1 -- 172.21.15.90:0/2219773508 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80c8070 con 0x561f80501db0 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48dfe00700 1 -- 172.21.15.90:0/2219773508 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f48f80a3ab0 con 0x561f80501db0 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f491372fb80 1 -- 172.21.15.90:0/2219773508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f80a4d00 con 0x561f80501db0 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f48dfe00700 1 -- 172.21.15.90:0/2219773508 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f48f80a3cb0 con 0x561f80501db0 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f491372fb80 10 client.16461 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.620+0000 7f491372fb80 1 -- 172.21.15.90:0/2219773508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561f8057ef30 con 0x561f80501db0 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.621+0000 7f48dfe00700 1 -- 172.21.15.90:0/2219773508 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48f80a4d00 con 0x561f80501db0 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.621+0000 7f48dfe00700 1 -- 172.21.15.90:0/2219773508 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f48f80ad3a0 con 0x561f80501db0 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.621+0000 7f491372fb80 10 client.16461 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.621+0000 7f491372fb80 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.621+0000 7f491372fb80 20 client.16461 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.621+0000 7f491372fb80 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561f805803f0 0x561f80582810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.621+0000 7f491372fb80 4 client.16461 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.621+0000 7f491372fb80 1 -- 172.21.15.90:0/2219773508 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561f804255f0 con 0x561f805803f0 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.621+0000 7f48fde00700 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561f805803f0 0x561f80582810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.622+0000 7f48fde00700 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561f805803f0 0x561f80582810 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d461d0a0f20c1be1 server_cookie=d8e950f617e548a in_seq=0 out_seq=0 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.622+0000 7f48dfe00700 10 client.16461 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f48dfe00700 1 -- 172.21.15.90:0/2219773508 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561f804255f0 con 0x561f805803f0 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f48dfe00700 10 client.16461 handle_command_reply: tid=0 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f491372fb80 1 client.16461 shutdown 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f491372fb80 1 -- 172.21.15.90:0/2219773508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 msgr2=0x561f80518f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f491372fb80 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f80518f40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f48f80a9040 tx=0x7f48f8009060 comp rx=0 tx=0).stop 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f491372fb80 1 -- 172.21.15.90:0/2219773508 shutdown_connections 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f491372fb80 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561f805803f0 0x561f80582810 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f491372fb80 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561f80501db0 0x561f80518f40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f491372fb80 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561f8037cec0 0x561f805184c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f491372fb80 1 --2- 172.21.15.90:0/2219773508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f804fbe30 0x561f80518a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f48dfe00700 0 client.16461 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:03.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f491372fb80 1 -- 172.21.15.90:0/2219773508 >> 172.21.15.90:0/2219773508 conn(0x561f803dd040 msgr2=0x561f804ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:03.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.624+0000 7f491372fb80 1 -- 172.21.15.90:0/2219773508 shutdown_connections 2024-05-27T07:51:03.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f491372fb80 1 -- 172.21.15.90:0/2219773508 wait complete. 2024-05-27T07:51:03.531 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:51:03.531 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "cb8d0f88-d190-4c62-980b-3d0c065032e2", 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f491372fb80 20 client.16461 trim_cache size 0 max 16384 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f4904c00700 1 -- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f48e40b2b40 msgr2=0x7f48e40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f4904c00700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f48e40b2b40 0x7f48e40b5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f48ec000c00 tx=0x7f48ec009530 comp rx=0 tx=0).stop 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f4904c00700 1 -- 172.21.15.90:0/1794623740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4900109830 msgr2=0x7f490010eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f4904c00700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4900109830 0x7f490010eef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f48f800ef30 tx=0x7f48f800c3a0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f4904c00700 1 -- 172.21.15.90:0/1794623740 shutdown_connections 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f4904c00700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f48e40b2b40 0x7f48e40b5050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f4904c00700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4900109830 0x7f490010eef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f4904c00700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49001045b0 0x7f490010e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f4904c00700 1 --2- 172.21.15.90:0/1794623740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4900108ee0 0x7f490010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.625+0000 7f4904c00700 1 -- 172.21.15.90:0/1794623740 >> 172.21.15.90:0/1794623740 conn(0x7f49000ffe40 msgr2=0x7f4900106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.626+0000 7f4904c00700 1 -- 172.21.15.90:0/1794623740 shutdown_connections 2024-05-27T07:51:03.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:02.626+0000 7f4904c00700 1 -- 172.21.15.90:0/1794623740 wait complete. 2024-05-27T07:51:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:03 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:51:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:03 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:03 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:51:03.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 45.0 s, 421.8 tps, lat 232.065 ms stddev 213.114 2024-05-27T07:51:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.866+0000 7f265fc00640 1 -- 172.21.15.90:0/3255334292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2658112e50 msgr2=0x7f2658113280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.866+0000 7f265fc00640 1 --2- 172.21.15.90:0/3255334292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2658112e50 0x7f2658113280 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2648009900 tx=0x7f264802a3f0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.866+0000 7f265fc00640 1 -- 172.21.15.90:0/3255334292 shutdown_connections 2024-05-27T07:51:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.866+0000 7f265fc00640 1 --2- 172.21.15.90:0/3255334292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2658113910 0x7f265811b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.866+0000 7f265fc00640 1 --2- 172.21.15.90:0/3255334292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2658112e50 0x7f2658113280 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.866+0000 7f265fc00640 1 --2- 172.21.15.90:0/3255334292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f265810e8c0 0x7f265810ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.866+0000 7f265fc00640 1 -- 172.21.15.90:0/3255334292 >> 172.21.15.90:0/3255334292 conn(0x7f2658109ea0 msgr2=0x7f265810c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.867+0000 7f265fc00640 1 -- 172.21.15.90:0/3255334292 shutdown_connections 2024-05-27T07:51:03.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.867+0000 7f265fc00640 1 -- 172.21.15.90:0/3255334292 wait complete. 2024-05-27T07:51:03.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.868+0000 7f265fc00640 1 Processor -- start 2024-05-27T07:51:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.869+0000 7f265fc00640 1 -- start start 2024-05-27T07:51:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.870+0000 7f265fc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f265810e8c0 0x7f265806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.870+0000 7f265fc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2658112e50 0x7f265806f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.870+0000 7f265fc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2658113910 0x7f265806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.871+0000 7f265d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f265810e8c0 0x7f265806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.871+0000 7f265fc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f265811e6f0 con 0x7f2658113910 2024-05-27T07:51:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.871+0000 7f265fc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f265811e570 con 0x7f2658112e50 2024-05-27T07:51:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.871+0000 7f265fc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f265811e870 con 0x7f265810e8c0 2024-05-27T07:51:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.871+0000 7f265d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f265810e8c0 0x7f265806ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37636/0 (socket says 172.21.15.90:37636) 2024-05-27T07:51:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.871+0000 7f265d200640 1 -- 172.21.15.90:0/531592047 learned_addr learned my addr 172.21.15.90:0/531592047 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.871+0000 7f2657e00640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2658112e50 0x7f265806f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.872+0000 7f265d200640 1 -- 172.21.15.90:0/531592047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2658112e50 msgr2=0x7f265806f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.872+0000 7f265d200640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2658112e50 0x7f265806f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.872+0000 7f265d200640 1 -- 172.21.15.90:0/531592047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2658113910 msgr2=0x7f265806f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.872+0000 7f265d200640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2658113910 0x7f265806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.872+0000 7f265d200640 1 -- 172.21.15.90:0/531592047 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26581b5860 con 0x7f265810e8c0 2024-05-27T07:51:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.872+0000 7f265d200640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f265810e8c0 0x7f265806ec50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f264400d950 tx=0x7f264400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.873+0000 7f2655600640 1 -- 172.21.15.90:0/531592047 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644017070 con 0x7f265810e8c0 2024-05-27T07:51:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.873+0000 7f2655600640 1 -- 172.21.15.90:0/531592047 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2644009c30 con 0x7f265810e8c0 2024-05-27T07:51:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.873+0000 7f265fc00640 1 -- 172.21.15.90:0/531592047 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26581b5af0 con 0x7f265810e8c0 2024-05-27T07:51:03.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.873+0000 7f2655600640 1 -- 172.21.15.90:0/531592047 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264400f620 con 0x7f265810e8c0 2024-05-27T07:51:03.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.874+0000 7f265fc00640 1 -- 172.21.15.90:0/531592047 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581b6010 con 0x7f265810e8c0 2024-05-27T07:51:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.881+0000 7f2666bf3740 1 Processor -- start 2024-05-27T07:51:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.881+0000 7f2666bf3740 1 -- start start 2024-05-27T07:51:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.882+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f69860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.882+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 0x5572b5f69da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.882+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f6a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.882+0000 7f2666bf3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5572b5edab90 con 0x5572b5ba64d0 2024-05-27T07:51:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.882+0000 7f2666bf3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5572b5eca730 con 0x5572b5bc71b0 2024-05-27T07:51:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.882+0000 7f2666bf3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5572b5ece770 con 0x5572b5b9e2f0 2024-05-27T07:51:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.882+0000 7f265dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f69860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.883+0000 7f2657e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f6a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.883+0000 7f2657e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f6a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40756/0 (socket says 172.21.15.90:40756) 2024-05-27T07:51:03.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.883+0000 7f2657e00640 1 -- 172.21.15.90:0/2516886526 learned_addr learned my addr 172.21.15.90:0/2516886526 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.884+0000 7f2657e00640 1 -- 172.21.15.90:0/2516886526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 msgr2=0x5572b5f69da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:51:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.884+0000 7f2657e00640 1 --2- 172.21.15.90:0/2516886526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 0x5572b5f69da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.884+0000 7f2657e00640 1 -- 172.21.15.90:0/2516886526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 msgr2=0x5572b5f69860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.884+0000 7f2657e00640 1 --2- 172.21.15.90:0/2516886526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f69860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.884+0000 7f2657e00640 1 -- 172.21.15.90:0/2516886526 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b5e638b0 con 0x5572b5ba64d0 2024-05-27T07:51:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.885+0000 7f2657e00640 1 --2- 172.21.15.90:0/2516886526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f6a2e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2648009900 tx=0x7f2648005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd00b8e880d8425c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.886+0000 7f2655600640 1 -- 172.21.15.90:0/531592047 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2644010460 con 0x7f265810e8c0 2024-05-27T07:51:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.886+0000 7f264e000640 1 -- 172.21.15.90:0/2516886526 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648040070 con 0x5572b5ba64d0 2024-05-27T07:51:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.886+0000 7f264e000640 1 -- 172.21.15.90:0/2516886526 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26480054b0 con 0x5572b5ba64d0 2024-05-27T07:51:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.886+0000 7f264e000640 1 -- 172.21.15.90:0/2516886526 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648035040 con 0x5572b5ba64d0 2024-05-27T07:51:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.888+0000 7f2655600640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f263c0b2aa0 0x7f263c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.888+0000 7f2655600640 1 -- 172.21.15.90:0/531592047 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f26440e23f0 con 0x7f265810e8c0 2024-05-27T07:51:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.888+0000 7f2666bf3740 1 -- 172.21.15.90:0/2516886526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 msgr2=0x5572b5f6a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.888+0000 7f2666bf3740 1 --2- 172.21.15.90:0/2516886526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f6a2e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2648009900 tx=0x7f2648005fd0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.890+0000 7f2666bf3740 1 -- 172.21.15.90:0/2516886526 shutdown_connections 2024-05-27T07:51:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.890+0000 7f265dc00640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f263c0b2aa0 0x7f263c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.891+0000 7f2666bf3740 1 --2- 172.21.15.90:0/2516886526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f6a2e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.891+0000 7f2666bf3740 1 --2- 172.21.15.90:0/2516886526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 0x5572b5f69da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.891+0000 7f2666bf3740 1 --2- 172.21.15.90:0/2516886526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f69860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.891+0000 7f2666bf3740 1 -- 172.21.15.90:0/2516886526 >> 172.21.15.90:0/2516886526 conn(0x5572b5bbb300 msgr2=0x5572b5f66010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.892+0000 7f2666bf3740 1 -- 172.21.15.90:0/2516886526 shutdown_connections 2024-05-27T07:51:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.893+0000 7f265dc00640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f263c0b2aa0 0x7f263c0b4fb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5572b5f669d0 tx=0x7f26500098e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.893+0000 7f2666bf3740 1 -- 172.21.15.90:0/2516886526 wait complete. 2024-05-27T07:51:03.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.894+0000 7f2666bf3740 1 -- start start 2024-05-27T07:51:03.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.894+0000 7f2666bf3740 1 Processor -- start 2024-05-27T07:51:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.895+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5b9e2f0 0x5572b5f739f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.895+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5ba64d0 0x5572b5f73f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.895+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f7fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.895+0000 7f2666bf3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5572b5bb0790 con 0x5572b5b9e2f0 2024-05-27T07:51:03.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.895+0000 7f2666bf3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5572b5e72bc0 con 0x5572b5bc71b0 2024-05-27T07:51:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.895+0000 7f2666bf3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5572b5f76930 con 0x5572b5ba64d0 2024-05-27T07:51:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.895+0000 7f2666bf3740 10 client.0 _ll_register_callbacks cb 0x5572b5e50400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.896+0000 7f2657e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5ba64d0 0x5572b5f73f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.896+0000 7f2657e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5ba64d0 0x5572b5f73f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37638/0 (socket says 172.21.15.90:37638) 2024-05-27T07:51:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.896+0000 7f2657e00640 1 -- 172.21.15.90:0/1778316969 learned_addr learned my addr 172.21.15.90:0/1778316969 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.897+0000 7f2657e00640 1 -- 172.21.15.90:0/1778316969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 msgr2=0x5572b5f7fc90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:51:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.897+0000 7f2657e00640 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f7fc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.897+0000 7f2657e00640 1 -- 172.21.15.90:0/1778316969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5b9e2f0 msgr2=0x5572b5f739f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.897+0000 7f2657e00640 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5b9e2f0 0x5572b5f739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.897+0000 7f2657e00640 1 -- 172.21.15.90:0/1778316969 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b5bdf210 con 0x5572b5ba64d0 2024-05-27T07:51:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.898+0000 7f2657e00640 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5ba64d0 0x5572b5f73f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2648004bd0 tx=0x7f2648004930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b04dad8cb66b848 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.898+0000 7f2642a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:51:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.898+0000 7f2642a00640 1 -- 172.21.15.90:0/1778316969 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648035070 con 0x5572b5ba64d0 2024-05-27T07:51:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.899+0000 7f2642a00640 1 -- 172.21.15.90:0/1778316969 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f2648004f40 con 0x5572b5ba64d0 2024-05-27T07:51:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.900+0000 7f2666bf3740 1 -- 172.21.15.90:0/1778316969 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26480054b0 con 0x5572b5ba64d0 2024-05-27T07:51:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.900+0000 7f2642a00640 1 -- 172.21.15.90:0/1778316969 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2648040040 con 0x5572b5ba64d0 2024-05-27T07:51:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.901+0000 7f2666bf3740 10 client.26017 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:03.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.901+0000 7f2666bf3740 1 -- 172.21.15.90:0/1778316969 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5572b5cab620 con 0x5572b5ba64d0 2024-05-27T07:51:03.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.901+0000 7f2642a00640 1 -- 172.21.15.90:0/1778316969 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26480054b0 con 0x5572b5ba64d0 2024-05-27T07:51:03.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.902+0000 7f2642a00640 1 -- 172.21.15.90:0/1778316969 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f2648003cf0 con 0x5572b5ba64d0 2024-05-27T07:51:03.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.902+0000 7f2666bf3740 10 client.26017 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.902+0000 7f2666bf3740 10 client.26017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.902+0000 7f2666bf3740 20 client.26017 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.902+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5572b5b85d80 0x5572b5f85a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.903+0000 7f2666bf3740 4 client.26017 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.903+0000 7f2666bf3740 1 -- 172.21.15.90:0/1778316969 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b5edcae0 con 0x5572b5b85d80 2024-05-27T07:51:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.904+0000 7f265d200640 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5572b5b85d80 0x5572b5f85a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.905+0000 7f265d200640 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5572b5b85d80 0x5572b5f85a60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195999b82d93cba server_cookie=e2b17e7cdb0c0027 in_seq=0 out_seq=0 2024-05-27T07:51:03.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.905+0000 7f2642a00640 10 client.26017 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.907+0000 7f2642a00640 1 -- 172.21.15.90:0/1778316969 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5572b5edcae0 con 0x5572b5b85d80 2024-05-27T07:51:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.907+0000 7f2642a00640 10 client.26017 handle_command_reply: tid=0 2024-05-27T07:51:03.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.907+0000 7f2666bf3740 1 client.26017 shutdown 2024-05-27T07:51:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.908+0000 7f2666bf3740 1 -- 172.21.15.90:0/1778316969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5ba64d0 msgr2=0x5572b5f73f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.908+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5ba64d0 0x5572b5f73f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2648004bd0 tx=0x7f2648004930 comp rx=0 tx=0).stop 2024-05-27T07:51:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.908+0000 7f2666bf3740 1 -- 172.21.15.90:0/1778316969 shutdown_connections 2024-05-27T07:51:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.908+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5572b5b85d80 0x5572b5f85a60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.909+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f7fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.909+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5ba64d0 0x5572b5f73f30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.909+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1778316969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5b9e2f0 0x5572b5f739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.909+0000 7f2666bf3740 1 -- 172.21.15.90:0/1778316969 >> 172.21.15.90:0/1778316969 conn(0x5572b5bbb300 msgr2=0x5572b5f6b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.909+0000 7f2642a00640 0 client.26017 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:03.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.909+0000 7f2666bf3740 1 -- 172.21.15.90:0/1778316969 shutdown_connections 2024-05-27T07:51:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.910+0000 7f2666bf3740 1 -- 172.21.15.90:0/1778316969 wait complete. 2024-05-27T07:51:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.910+0000 7f2666bf3740 20 client.26017 trim_cache size 0 max 16384 2024-05-27T07:51:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.944+0000 7f2666bf3740 1 Processor -- start 2024-05-27T07:51:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.944+0000 7f2666bf3740 1 -- start start 2024-05-27T07:51:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.945+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 0x5572b5f76120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.945+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f6a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.945+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f65330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.945+0000 7f2666bf3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2638011900 con 0x5572b5ba64d0 2024-05-27T07:51:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.945+0000 7f2666bf3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f263800f6f0 con 0x5572b5bc71b0 2024-05-27T07:51:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.945+0000 7f2666bf3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2638011230 con 0x5572b5b9e2f0 2024-05-27T07:51:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.946+0000 7f265dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f65330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.946+0000 7f265dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f65330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50468/0 (socket says 172.21.15.90:50468) 2024-05-27T07:51:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.946+0000 7f265dc00640 1 -- 172.21.15.90:0/1605599236 learned_addr learned my addr 172.21.15.90:0/1605599236 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.947+0000 7f2657e00640 1 --2- 172.21.15.90:0/1605599236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f6a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.947+0000 7f265dc00640 1 -- 172.21.15.90:0/1605599236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 msgr2=0x5572b5f76120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.947+0000 7f265dc00640 1 --2- 172.21.15.90:0/1605599236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 0x5572b5f76120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.947+0000 7f265dc00640 1 -- 172.21.15.90:0/1605599236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 msgr2=0x5572b5f6a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.947+0000 7f265dc00640 1 --2- 172.21.15.90:0/1605599236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f6a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.947+0000 7f265dc00640 1 -- 172.21.15.90:0/1605599236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b5f6bf70 con 0x5572b5bc71b0 2024-05-27T07:51:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.949+0000 7f265dc00640 1 --2- 172.21.15.90:0/1605599236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f65330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2650003bd0 tx=0x7f2650014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3adeeef318f095a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.949+0000 7f264d600640 1 -- 172.21.15.90:0/1605599236 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265001e420 con 0x5572b5bc71b0 2024-05-27T07:51:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.950+0000 7f264d600640 1 -- 172.21.15.90:0/1605599236 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265001ea50 con 0x5572b5bc71b0 2024-05-27T07:51:03.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.950+0000 7f264d600640 1 -- 172.21.15.90:0/1605599236 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265001e420 con 0x5572b5bc71b0 2024-05-27T07:51:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.951+0000 7f2666bf3740 1 -- 172.21.15.90:0/1605599236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 msgr2=0x5572b5f65330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.951+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1605599236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f65330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2650003bd0 tx=0x7f2650014000 comp rx=0 tx=0).stop 2024-05-27T07:51:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.951+0000 7f2666bf3740 1 -- 172.21.15.90:0/1605599236 shutdown_connections 2024-05-27T07:51:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.951+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1605599236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5f65330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.951+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1605599236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f6a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.951+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1605599236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 0x5572b5f76120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.951+0000 7f2666bf3740 1 -- 172.21.15.90:0/1605599236 >> 172.21.15.90:0/1605599236 conn(0x5572b5bbb300 msgr2=0x5572b5f73280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.951+0000 7f2666bf3740 1 -- 172.21.15.90:0/1605599236 shutdown_connections 2024-05-27T07:51:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.951+0000 7f2666bf3740 1 -- 172.21.15.90:0/1605599236 wait complete. 2024-05-27T07:51:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.952+0000 7f2666bf3740 1 -- start start 2024-05-27T07:51:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.952+0000 7f2666bf3740 1 Processor -- start 2024-05-27T07:51:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.953+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 0x5572b5f84070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.953+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.953+0000 7f2666bf3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5febe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.954+0000 7f2666bf3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2638010490 con 0x5572b5ba64d0 2024-05-27T07:51:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.954+0000 7f2666bf3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f263800fdc0 con 0x5572b5bc71b0 2024-05-27T07:51:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.954+0000 7f2666bf3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2638010b60 con 0x5572b5b9e2f0 2024-05-27T07:51:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.954+0000 7f2666bf3740 10 client.0 _ll_register_callbacks cb 0x7f26480040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.954+0000 7f2657e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.954+0000 7f265d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 0x5572b5f84070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.954+0000 7f2657e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f845b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40784/0 (socket says 172.21.15.90:40784) 2024-05-27T07:51:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.954+0000 7f2657e00640 1 -- 172.21.15.90:0/1074272823 learned_addr learned my addr 172.21.15.90:0/1074272823 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.954+0000 7f265dc00640 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5febe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.955+0000 7f2657e00640 1 -- 172.21.15.90:0/1074272823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 msgr2=0x5572b5f84070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.955+0000 7f2657e00640 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 0x5572b5f84070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.955+0000 7f2657e00640 1 -- 172.21.15.90:0/1074272823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 msgr2=0x5572b5febe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.955+0000 7f2657e00640 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5febe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.955+0000 7f2657e00640 1 -- 172.21.15.90:0/1074272823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b5fec400 con 0x5572b5ba64d0 2024-05-27T07:51:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.956+0000 7f265dc00640 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5febe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.956+0000 7f2657e00640 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f845b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f26480092a0 tx=0x7f2648004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9886d32f1547bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.956+0000 7f2642a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:03.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.958+0000 7f2642a00640 1 -- 172.21.15.90:0/1074272823 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264805f060 con 0x5572b5ba64d0 2024-05-27T07:51:03.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.958+0000 7f2642a00640 1 -- 172.21.15.90:0/1074272823 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f2648005350 con 0x5572b5ba64d0 2024-05-27T07:51:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.958+0000 7f2666bf3740 1 -- 172.21.15.90:0/1074272823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f265001ea50 con 0x5572b5ba64d0 2024-05-27T07:51:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.959+0000 7f2642a00640 1 -- 172.21.15.90:0/1074272823 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2648035040 con 0x5572b5ba64d0 2024-05-27T07:51:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.959+0000 7f2666bf3740 10 client.16470 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:03.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.959+0000 7f2666bf3740 1 -- 172.21.15.90:0/1074272823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5572b5fed650 con 0x5572b5ba64d0 2024-05-27T07:51:03.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.960+0000 7f2642a00640 1 -- 172.21.15.90:0/1074272823 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f265001ea50 con 0x5572b5ba64d0 2024-05-27T07:51:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.962+0000 7f2642a00640 1 -- 172.21.15.90:0/1074272823 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f26480035f0 con 0x5572b5ba64d0 2024-05-27T07:51:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.962+0000 7f2666bf3740 10 client.16470 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.962+0000 7f2666bf3740 10 client.16470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.962+0000 7f2666bf3740 20 client.16470 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.962+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5572b5b5b1c0 0x5572b5fefb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.963+0000 7f2666bf3740 4 client.16470 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:51:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.963+0000 7f2666bf3740 1 -- 172.21.15.90:0/1074272823 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572b5ed62c0 con 0x5572b5b5b1c0 2024-05-27T07:51:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.963+0000 7f265d200640 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5572b5b5b1c0 0x5572b5fefb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.964+0000 7f265d200640 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5572b5b5b1c0 0x5572b5fefb30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bafa930445bcc1b server_cookie=83abfef2132fda8f in_seq=0 out_seq=0 2024-05-27T07:51:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:03.964+0000 7f2642a00640 10 client.16470 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.012+0000 7f2642a00640 1 -- 172.21.15.90:0/1074272823 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5572b5ed62c0 con 0x5572b5b5b1c0 2024-05-27T07:51:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.012+0000 7f2642a00640 10 client.16470 handle_command_reply: tid=0 2024-05-27T07:51:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.012+0000 7f2666bf3740 1 client.16470 shutdown 2024-05-27T07:51:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.013+0000 7f2666bf3740 1 -- 172.21.15.90:0/1074272823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 msgr2=0x5572b5f845b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.013+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f845b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f26480092a0 tx=0x7f2648004c80 comp rx=0 tx=0).stop 2024-05-27T07:51:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.013+0000 7f2666bf3740 1 -- 172.21.15.90:0/1074272823 shutdown_connections 2024-05-27T07:51:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.013+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5572b5b5b1c0 0x5572b5fefb30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.013+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5572b5bc71b0 0x5572b5febe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.013+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5572b5ba64d0 0x5572b5f845b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.013+0000 7f2666bf3740 1 --2- 172.21.15.90:0/1074272823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572b5b9e2f0 0x5572b5f84070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.013+0000 7f2666bf3740 1 -- 172.21.15.90:0/1074272823 >> 172.21.15.90:0/1074272823 conn(0x5572b5bbb300 msgr2=0x5572b5f69f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.014+0000 7f2642a00640 0 client.16470 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.014+0000 7f2666bf3740 1 -- 172.21.15.90:0/1074272823 shutdown_connections 2024-05-27T07:51:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.014+0000 7f2666bf3740 1 -- 172.21.15.90:0/1074272823 wait complete. 2024-05-27T07:51:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.014+0000 7f2666bf3740 20 client.16470 trim_cache size 0 max 16384 2024-05-27T07:51:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:51:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:51:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:51:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.015+0000 7f265fc00640 1 -- 172.21.15.90:0/531592047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f263c0b2aa0 msgr2=0x7f263c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.015+0000 7f265fc00640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f263c0b2aa0 0x7f263c0b4fb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x5572b5f669d0 tx=0x7f26500098e0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.016+0000 7f265fc00640 1 -- 172.21.15.90:0/531592047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f265810e8c0 msgr2=0x7f265806ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.016+0000 7f265fc00640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f265810e8c0 0x7f265806ec50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f264400d950 tx=0x7f264400de10 comp rx=0 tx=0).stop 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.019+0000 7f265fc00640 1 -- 172.21.15.90:0/531592047 shutdown_connections 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.019+0000 7f265fc00640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f263c0b2aa0 0x7f263c0b4fb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.019+0000 7f265fc00640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2658112e50 0x7f265806f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.019+0000 7f265fc00640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2658113910 0x7f265806f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.019+0000 7f265fc00640 1 --2- 172.21.15.90:0/531592047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f265810e8c0 0x7f265806ec50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.019+0000 7f265fc00640 1 -- 172.21.15.90:0/531592047 >> 172.21.15.90:0/531592047 conn(0x7f2658109ea0 msgr2=0x7f265811a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.019+0000 7f265fc00640 1 -- 172.21.15.90:0/531592047 shutdown_connections 2024-05-27T07:51:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:04.019+0000 7f265fc00640 1 -- 172.21.15.90:0/531592047 wait complete. 2024-05-27T07:51:04.401 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:04 smithi090 ceph-mon[31429]: pgmap v147: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.2 MiB/s wr, 754 op/s 2024-05-27T07:51:04.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:04 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:04.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:04 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:51:04.489 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:51:04.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:04 smithi136 ceph-mon[35324]: pgmap v147: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.2 MiB/s wr, 754 op/s 2024-05-27T07:51:04.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:04 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:04.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:04 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:51:04.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:04 smithi129 ceph-mon[35254]: pgmap v147: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.2 MiB/s wr, 754 op/s 2024-05-27T07:51:04.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:04 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:51:04.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:04 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:51:05.223 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:06.443 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:06 smithi090 ceph-mon[31429]: pgmap v148: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 543 op/s 2024-05-27T07:51:06.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:06 smithi136 ceph-mon[35324]: pgmap v148: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 543 op/s 2024-05-27T07:51:06.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:06 smithi129 ceph-mon[35254]: pgmap v148: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 543 op/s 2024-05-27T07:51:08.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:08 smithi090 ceph-mon[31429]: pgmap v149: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 542 op/s 2024-05-27T07:51:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:08 smithi136 ceph-mon[35324]: pgmap v149: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 542 op/s 2024-05-27T07:51:08.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:08 smithi129 ceph-mon[35254]: pgmap v149: 129 pgs: 129 active+clean; 1.4 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.4 MiB/s wr, 542 op/s 2024-05-27T07:51:08.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 50.0 s, 607.5 tps, lat 167.641 ms stddev 78.797 2024-05-27T07:51:09.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.080+0000 7fac15600700 1 -- 172.21.15.90:0/286337605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac10107c00 msgr2=0x7fac10108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:09.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.080+0000 7fac15600700 1 --2- 172.21.15.90:0/286337605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac10107c00 0x7fac10108010 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fabfc0099c0 tx=0x7fabfc009cd0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.081+0000 7fac15600700 1 -- 172.21.15.90:0/286337605 shutdown_connections 2024-05-27T07:51:09.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.081+0000 7fac15600700 1 --2- 172.21.15.90:0/286337605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac10108550 0x7fac1010feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.081+0000 7fac15600700 1 --2- 172.21.15.90:0/286337605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac101032d0 0x7fac101036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.081+0000 7fac15600700 1 --2- 172.21.15.90:0/286337605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac10107c00 0x7fac10108010 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.081+0000 7fac15600700 1 -- 172.21.15.90:0/286337605 >> 172.21.15.90:0/286337605 conn(0x7fac10078ab0 msgr2=0x7fac10078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:09.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.081+0000 7fac15600700 1 -- 172.21.15.90:0/286337605 shutdown_connections 2024-05-27T07:51:09.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.081+0000 7fac15600700 1 -- 172.21.15.90:0/286337605 wait complete. 2024-05-27T07:51:09.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.086+0000 7fac15600700 1 Processor -- start 2024-05-27T07:51:09.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.086+0000 7fac15600700 1 -- start start 2024-05-27T07:51:09.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.088+0000 7fac15600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac101032d0 0x7fac1010ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.088+0000 7fac15600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac10107c00 0x7fac1010f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.089+0000 7fac15600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac10108550 0x7fac101a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.089+0000 7fac15600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac10111ac0 con 0x7fac101032d0 2024-05-27T07:51:09.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.089+0000 7fac15600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac10111940 con 0x7fac10108550 2024-05-27T07:51:09.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.089+0000 7fac15600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac10111c40 con 0x7fac10107c00 2024-05-27T07:51:09.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.090+0000 7fac0f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac10107c00 0x7fac1010f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.090+0000 7fac0f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac10107c00 0x7fac1010f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:58784/0 (socket says 172.21.15.90:58784) 2024-05-27T07:51:09.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.090+0000 7fac0f400700 1 -- 172.21.15.90:0/1608351926 learned_addr learned my addr 172.21.15.90:0/1608351926 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:09.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.090+0000 7fac0fe00700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac101032d0 0x7fac1010ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.090+0000 7fac14c00700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac10108550 0x7fac101a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.091+0000 7fac0f400700 1 -- 172.21.15.90:0/1608351926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac10108550 msgr2=0x7fac101a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:09.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.091+0000 7fac0f400700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac10108550 0x7fac101a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.091+0000 7fac0f400700 1 -- 172.21.15.90:0/1608351926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac101032d0 msgr2=0x7fac1010ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:09.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.091+0000 7fac0f400700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac101032d0 0x7fac1010ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.091+0000 7fac0f400700 1 -- 172.21.15.90:0/1608351926 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac101a6120 con 0x7fac10107c00 2024-05-27T07:51:09.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.091+0000 7fac0f400700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac10107c00 0x7fac1010f320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00b580 tx=0x7fabfc0055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:09.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.092+0000 7fac0cc00700 1 -- 172.21.15.90:0/1608351926 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc009e60 con 0x7fac10107c00 2024-05-27T07:51:09.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.092+0000 7fac0cc00700 1 -- 172.21.15.90:0/1608351926 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabfc00f460 con 0x7fac10107c00 2024-05-27T07:51:09.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.092+0000 7fac0cc00700 1 -- 172.21.15.90:0/1608351926 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc003e60 con 0x7fac10107c00 2024-05-27T07:51:09.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.093+0000 7fac15600700 1 -- 172.21.15.90:0/1608351926 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac101a63b0 con 0x7fac10107c00 2024-05-27T07:51:09.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.093+0000 7fac15600700 1 -- 172.21.15.90:0/1608351926 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac101a6840 con 0x7fac10107c00 2024-05-27T07:51:09.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.101+0000 7fac2565ab80 1 Processor -- start 2024-05-27T07:51:09.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.102+0000 7fac2565ab80 1 -- start start 2024-05-27T07:51:09.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.103+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.103+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de410 0x55bac94e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.103+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de820 0x55bac94eb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.103+0000 7fac2565ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bac940ce40 con 0x55bac935fe40 2024-05-27T07:51:09.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.103+0000 7fac2565ab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bac93fc9e0 con 0x55bac94de410 2024-05-27T07:51:09.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.103+0000 7fac2565ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bac9400a20 con 0x55bac94de820 2024-05-27T07:51:09.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.104+0000 7fac14c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.104+0000 7fac0fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de410 0x55bac94e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.104+0000 7fac14c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94dded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48208/0 (socket says 172.21.15.90:48208) 2024-05-27T07:51:09.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.104+0000 7fac14c00700 1 -- 172.21.15.90:0/1568496160 learned_addr learned my addr 172.21.15.90:0/1568496160 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:09.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.104+0000 7fac0cc00700 1 -- 172.21.15.90:0/1608351926 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fabfc00f910 con 0x7fac10107c00 2024-05-27T07:51:09.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.104+0000 7fac0fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de410 0x55bac94e8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48856/0 (socket says 172.21.15.90:48856) 2024-05-27T07:51:09.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.105+0000 7fac14c00700 1 -- 172.21.15.90:0/1568496160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de820 msgr2=0x55bac94eb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:09.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.105+0000 7fac14c00700 1 --2- 172.21.15.90:0/1568496160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de820 0x55bac94eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.105+0000 7fac14c00700 1 -- 172.21.15.90:0/1568496160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de410 msgr2=0x55bac94e8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:09.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.105+0000 7fac14c00700 1 --2- 172.21.15.90:0/1568496160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de410 0x55bac94e8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.105+0000 7fac14c00700 1 -- 172.21.15.90:0/1568496160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac93c5420 con 0x55bac935fe40 2024-05-27T07:51:09.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.106+0000 7fac14c00700 1 --2- 172.21.15.90:0/1568496160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94dded0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fac08009440 tx=0x7fac08009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9ab4c58f3a02a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:09.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.106+0000 7fac0cc00700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fabf40b2aa0 0x7fabf40b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.107+0000 7fac0cc00700 1 -- 172.21.15.90:0/1608351926 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fabfc0e38f0 con 0x7fac10107c00 2024-05-27T07:51:09.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.107+0000 7fac0fe00700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fabf40b2aa0 0x7fabf40b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.108+0000 7fac05600700 1 -- 172.21.15.90:0/1568496160 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08013750 con 0x55bac935fe40 2024-05-27T07:51:09.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.108+0000 7fac05600700 1 -- 172.21.15.90:0/1568496160 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac08013d80 con 0x55bac935fe40 2024-05-27T07:51:09.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.108+0000 7fac05600700 1 -- 172.21.15.90:0/1568496160 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08015e40 con 0x55bac935fe40 2024-05-27T07:51:09.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.108+0000 7fac0fe00700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fabf40b2aa0 0x7fabf40b4fb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55bac94df430 tx=0x7fac0000d040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:09.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.109+0000 7fac2565ab80 1 -- 172.21.15.90:0/1568496160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 msgr2=0x55bac94dded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:09.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.109+0000 7fac2565ab80 1 --2- 172.21.15.90:0/1568496160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94dded0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fac08009440 tx=0x7fac08009920 comp rx=0 tx=0).stop 2024-05-27T07:51:09.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.109+0000 7fac2565ab80 1 -- 172.21.15.90:0/1568496160 shutdown_connections 2024-05-27T07:51:09.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.109+0000 7fac2565ab80 1 --2- 172.21.15.90:0/1568496160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de820 0x55bac94eb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.109+0000 7fac2565ab80 1 --2- 172.21.15.90:0/1568496160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94dded0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.109+0000 7fac2565ab80 1 --2- 172.21.15.90:0/1568496160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de410 0x55bac94e8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.109+0000 7fac2565ab80 1 -- 172.21.15.90:0/1568496160 >> 172.21.15.90:0/1568496160 conn(0x55bac93bffc0 msgr2=0x55bac94d9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:09.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.110+0000 7fac2565ab80 1 -- 172.21.15.90:0/1568496160 shutdown_connections 2024-05-27T07:51:09.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.110+0000 7fac2565ab80 1 -- 172.21.15.90:0/1568496160 wait complete. 2024-05-27T07:51:09.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.111+0000 7fac2565ab80 1 -- start start 2024-05-27T07:51:09.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.112+0000 7fac2565ab80 1 Processor -- start 2024-05-27T07:51:09.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.113+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.113+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.113+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 0x55bac94e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.113+0000 7fac2565ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bac9302fa0 con 0x55bac935fe40 2024-05-27T07:51:09.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.113+0000 7fac2565ab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bac939d760 con 0x55bac94de820 2024-05-27T07:51:09.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.113+0000 7fac2565ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bac94eb990 con 0x55bac94de410 2024-05-27T07:51:09.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.113+0000 7fac2565ab80 10 client.0 _ll_register_callbacks cb 0x55bac93a6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:09.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.114+0000 7fac0f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.114+0000 7fac0fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.114+0000 7fac0fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94e8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48216/0 (socket says 172.21.15.90:48216) 2024-05-27T07:51:09.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.114+0000 7fac0fe00700 1 -- 172.21.15.90:0/2205065948 learned_addr learned my addr 172.21.15.90:0/2205065948 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:09.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.114+0000 7fac14c00700 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 0x55bac94e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.115+0000 7fac0fe00700 1 -- 172.21.15.90:0/2205065948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 msgr2=0x55bac94e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:09.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.115+0000 7fac0fe00700 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94e8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.115+0000 7fac0fe00700 1 -- 172.21.15.90:0/2205065948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 msgr2=0x55bac94e90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:09.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.115+0000 7fac0fe00700 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 0x55bac94e90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.115+0000 7fac0fe00700 1 -- 172.21.15.90:0/2205065948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac934c8d0 con 0x55bac935fe40 2024-05-27T07:51:09.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.115+0000 7fac14c00700 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 0x55bac94e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:09.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.115+0000 7fac0fe00700 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94e8640 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fac00019a10 tx=0x7fac000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a5256643e2a82c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:09.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.115+0000 7fabfa000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:09.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.116+0000 7fabfa000700 1 -- 172.21.15.90:0/2205065948 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0001f050 con 0x55bac935fe40 2024-05-27T07:51:09.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.116+0000 7fac2565ab80 1 -- 172.21.15.90:0/2205065948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac08013d80 con 0x55bac935fe40 2024-05-27T07:51:09.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.117+0000 7fabfa000700 1 -- 172.21.15.90:0/2205065948 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fac00017c30 con 0x55bac935fe40 2024-05-27T07:51:09.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.117+0000 7fabfa000700 1 -- 172.21.15.90:0/2205065948 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fac00032050 con 0x55bac935fe40 2024-05-27T07:51:09.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.117+0000 7fac2565ab80 10 client.16482 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:09.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.118+0000 7fac2565ab80 1 -- 172.21.15.90:0/2205065948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bac93cd060 con 0x55bac935fe40 2024-05-27T07:51:09.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.118+0000 7fabfa000700 1 -- 172.21.15.90:0/2205065948 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac08013d80 con 0x55bac935fe40 2024-05-27T07:51:09.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.119+0000 7fabfa000700 1 -- 172.21.15.90:0/2205065948 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fac00024cb0 con 0x55bac935fe40 2024-05-27T07:51:09.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.119+0000 7fac2565ab80 10 client.16482 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:09.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.119+0000 7fac2565ab80 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:09.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.119+0000 7fac2565ab80 20 client.16482 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:09.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.119+0000 7fac2565ab80 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac94f8a00 0x55bac94fae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.119+0000 7fac2565ab80 4 client.16482 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:09.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.119+0000 7fac2565ab80 1 -- 172.21.15.90:0/2205065948 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bac940ed90 con 0x55bac94f8a00 2024-05-27T07:51:09.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.120+0000 7fac0f400700 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac94f8a00 0x55bac94fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.123+0000 7fac0f400700 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac94f8a00 0x55bac94fae80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5216319115c8e1b6 server_cookie=19bcfb3a252fea77 in_seq=0 out_seq=0 2024-05-27T07:51:09.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.123+0000 7fabfa000700 10 client.16482 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:09.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.126+0000 7fabfa000700 1 -- 172.21.15.90:0/2205065948 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bac940ed90 con 0x55bac94f8a00 2024-05-27T07:51:09.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.126+0000 7fabfa000700 10 client.16482 handle_command_reply: tid=0 2024-05-27T07:51:09.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.126+0000 7fac2565ab80 1 client.16482 shutdown 2024-05-27T07:51:09.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.127+0000 7fac2565ab80 1 -- 172.21.15.90:0/2205065948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 msgr2=0x55bac94e8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:09.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.127+0000 7fac2565ab80 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94e8640 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fac00019a10 tx=0x7fac000047c0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.127+0000 7fac2565ab80 1 -- 172.21.15.90:0/2205065948 shutdown_connections 2024-05-27T07:51:09.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.127+0000 7fac2565ab80 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac94f8a00 0x55bac94fae80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.127+0000 7fac2565ab80 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 0x55bac94e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.127+0000 7fac2565ab80 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94e8640 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.127+0000 7fac2565ab80 1 --2- 172.21.15.90:0/2205065948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.127+0000 7fac2565ab80 1 -- 172.21.15.90:0/2205065948 >> 172.21.15.90:0/2205065948 conn(0x55bac93bffc0 msgr2=0x55bac94d8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:09.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.127+0000 7fabfa000700 0 client.16482 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:09.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.128+0000 7fac2565ab80 1 -- 172.21.15.90:0/2205065948 shutdown_connections 2024-05-27T07:51:09.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.128+0000 7fac2565ab80 1 -- 172.21.15.90:0/2205065948 wait complete. 2024-05-27T07:51:09.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.128+0000 7fac2565ab80 20 client.16482 trim_cache size 0 max 16384 2024-05-27T07:51:09.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.169+0000 7fac2565ab80 1 Processor -- start 2024-05-27T07:51:09.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.169+0000 7fac2565ab80 1 -- start start 2024-05-27T07:51:09.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.170+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.170+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94e8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.170+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 0x55bac94f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.170+0000 7fac2565ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac101b8310 con 0x55bac935fe40 2024-05-27T07:51:09.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.170+0000 7fac2565ab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac101b6100 con 0x55bac94de820 2024-05-27T07:51:09.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.170+0000 7fac2565ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac101b7c40 con 0x55bac94de410 2024-05-27T07:51:09.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.171+0000 7fac0f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94e8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.171+0000 7fac14c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 0x55bac94f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.171+0000 7fac0fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.172+0000 7fac14c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 0x55bac94f4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48876/0 (socket says 172.21.15.90:48876) 2024-05-27T07:51:09.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.172+0000 7fac14c00700 1 -- 172.21.15.90:0/3315249106 learned_addr learned my addr 172.21.15.90:0/3315249106 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:09.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.172+0000 7fac0fe00700 1 -- 172.21.15.90:0/3315249106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 msgr2=0x55bac94e8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:09.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.173+0000 7fac0fe00700 1 --2- 172.21.15.90:0/3315249106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94e8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.173+0000 7fac0fe00700 1 -- 172.21.15.90:0/3315249106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 msgr2=0x55bac94f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:09.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.173+0000 7fac0fe00700 1 --2- 172.21.15.90:0/3315249106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 0x55bac94f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.173+0000 7fac0fe00700 1 -- 172.21.15.90:0/3315249106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac94db350 con 0x55bac935fe40 2024-05-27T07:51:09.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.174+0000 7fac0fe00700 1 --2- 172.21.15.90:0/3315249106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94ea9a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fac0002d430 tx=0x7fac0002d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e421a27d27e2ab46 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.174+0000 7fac04c00700 1 -- 172.21.15.90:0/3315249106 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0001f040 con 0x55bac935fe40 2024-05-27T07:51:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.175+0000 7fac04c00700 1 -- 172.21.15.90:0/3315249106 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac00029070 con 0x55bac935fe40 2024-05-27T07:51:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.175+0000 7fac04c00700 1 -- 172.21.15.90:0/3315249106 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00004280 con 0x55bac935fe40 2024-05-27T07:51:09.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.181+0000 7fac2565ab80 1 -- 172.21.15.90:0/3315249106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 msgr2=0x55bac94ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:09.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.181+0000 7fac2565ab80 1 --2- 172.21.15.90:0/3315249106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94ea9a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fac0002d430 tx=0x7fac0002d8b0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.182+0000 7fac2565ab80 1 -- 172.21.15.90:0/3315249106 shutdown_connections 2024-05-27T07:51:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.182+0000 7fac2565ab80 1 --2- 172.21.15.90:0/3315249106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac94de820 0x55bac94f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.182+0000 7fac2565ab80 1 --2- 172.21.15.90:0/3315249106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac935fe40 0x55bac94ea9a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.182+0000 7fac2565ab80 1 --2- 172.21.15.90:0/3315249106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94e8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.182+0000 7fac2565ab80 1 -- 172.21.15.90:0/3315249106 >> 172.21.15.90:0/3315249106 conn(0x55bac93bffc0 msgr2=0x55bac94e8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.182+0000 7fac2565ab80 1 -- 172.21.15.90:0/3315249106 shutdown_connections 2024-05-27T07:51:09.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.182+0000 7fac2565ab80 1 -- 172.21.15.90:0/3315249106 wait complete. 2024-05-27T07:51:09.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.183+0000 7fac2565ab80 1 -- start start 2024-05-27T07:51:09.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.183+0000 7fac2565ab80 1 Processor -- start 2024-05-27T07:51:09.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.184+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac935fe40 0x55bac94f1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.184+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94ec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.184+0000 7fac2565ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac94de820 0x55bac94ec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.184+0000 7fac2565ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac101b6ea0 con 0x55bac94de820 2024-05-27T07:51:09.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.184+0000 7fac2565ab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac101b67d0 con 0x55bac935fe40 2024-05-27T07:51:09.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.184+0000 7fac2565ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac101b7570 con 0x55bac94de410 2024-05-27T07:51:09.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.184+0000 7fac2565ab80 10 client.0 _ll_register_callbacks cb 0x55bac93a6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:09.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.186+0000 7fac0fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac935fe40 0x55bac94f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.186+0000 7fac14c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac94de820 0x55bac94ec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.186+0000 7fac0f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94ec3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.186+0000 7fac0f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94ec3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:58812/0 (socket says 172.21.15.90:58812) 2024-05-27T07:51:09.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.186+0000 7fac0f400700 1 -- 172.21.15.90:0/1452438017 learned_addr learned my addr 172.21.15.90:0/1452438017 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:09.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.187+0000 7fac14c00700 1 -- 172.21.15.90:0/1452438017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 msgr2=0x55bac94ec3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:09.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.187+0000 7fac14c00700 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94ec3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.187+0000 7fac14c00700 1 -- 172.21.15.90:0/1452438017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac935fe40 msgr2=0x55bac94f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:09.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.187+0000 7fac14c00700 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac935fe40 0x55bac94f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.187+0000 7fac14c00700 1 -- 172.21.15.90:0/1452438017 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac94ece50 con 0x55bac94de820 2024-05-27T07:51:09.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.188+0000 7fac0f400700 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94ec3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:09.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.188+0000 7fac14c00700 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac94de820 0x55bac94ec910 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fac08007ac0 tx=0x7fac080138c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdb4ad08cae0cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:09.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.189+0000 7fabfa000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:09.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.190+0000 7fabfa000700 1 -- 172.21.15.90:0/1452438017 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac08009e60 con 0x55bac94de820 2024-05-27T07:51:09.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.190+0000 7fabfa000700 1 -- 172.21.15.90:0/1452438017 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fac08015440 con 0x55bac94de820 2024-05-27T07:51:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.190+0000 7fac2565ab80 1 -- 172.21.15.90:0/1452438017 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac00029070 con 0x55bac94de820 2024-05-27T07:51:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.190+0000 7fabfa000700 1 -- 172.21.15.90:0/1452438017 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fac08015640 con 0x55bac94de820 2024-05-27T07:51:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.190+0000 7fac2565ab80 10 client.16488 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.190+0000 7fac2565ab80 1 -- 172.21.15.90:0/1452438017 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bac9574150 con 0x55bac94de820 2024-05-27T07:51:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.191+0000 7fabfa000700 1 -- 172.21.15.90:0/1452438017 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac00029070 con 0x55bac94de820 2024-05-27T07:51:09.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.191+0000 7fabfa000700 1 -- 172.21.15.90:0/1452438017 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fac08028020 con 0x55bac94de820 2024-05-27T07:51:09.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.191+0000 7fac2565ab80 10 client.16488 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:09.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.191+0000 7fac2565ab80 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:09.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.191+0000 7fac2565ab80 20 client.16488 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:09.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.191+0000 7fac2565ab80 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac9574710 0x55bac9576b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:09.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.191+0000 7fac2565ab80 4 client.16488 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:51:09.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.191+0000 7fac2565ab80 1 -- 172.21.15.90:0/1452438017 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bac9408570 con 0x55bac9574710 2024-05-27T07:51:09.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.193+0000 7fac0fe00700 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac9574710 0x55bac9576b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:09.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.194+0000 7fabfa000700 10 client.16488 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:09.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.194+0000 7fac0fe00700 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac9574710 0x55bac9576b30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5aca9fd3ee85143 server_cookie=149c445f6c28f8e6 in_seq=0 out_seq=0 2024-05-27T07:51:09.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.195+0000 7fabfa000700 1 -- 172.21.15.90:0/1452438017 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bac9408570 con 0x55bac9574710 2024-05-27T07:51:09.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.195+0000 7fabfa000700 10 client.16488 handle_command_reply: tid=0 2024-05-27T07:51:09.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.195+0000 7fac2565ab80 1 client.16488 shutdown 2024-05-27T07:51:09.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fac2565ab80 1 -- 172.21.15.90:0/1452438017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac94de820 msgr2=0x55bac94ec910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:09.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fac2565ab80 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac94de820 0x55bac94ec910 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fac08007ac0 tx=0x7fac080138c0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fac2565ab80 1 -- 172.21.15.90:0/1452438017 shutdown_connections 2024-05-27T07:51:09.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fac2565ab80 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bac9574710 0x55bac9576b30 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fabfa000700 0 client.16488 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:09.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fac2565ab80 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac94de820 0x55bac94ec910 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fac2565ab80 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac935fe40 0x55bac94f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fac2565ab80 1 --2- 172.21.15.90:0/1452438017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac94de410 0x55bac94ec3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fac2565ab80 1 -- 172.21.15.90:0/1452438017 >> 172.21.15.90:0/1452438017 conn(0x55bac93bffc0 msgr2=0x55bac94e9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:09.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fac2565ab80 1 -- 172.21.15.90:0/1452438017 shutdown_connections 2024-05-27T07:51:09.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.196+0000 7fac2565ab80 1 -- 172.21.15.90:0/1452438017 wait complete. 2024-05-27T07:51:09.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.197+0000 7fac2565ab80 20 client.16488 trim_cache size 0 max 16384 2024-05-27T07:51:09.199 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:51:09.200 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:51:09.200 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:51:09.200 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:51:09.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.198+0000 7fac15600700 1 -- 172.21.15.90:0/1608351926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fabf40b2aa0 msgr2=0x7fabf40b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:09.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.198+0000 7fac15600700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fabf40b2aa0 0x7fabf40b4fb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55bac94df430 tx=0x7fac0000d040 comp rx=0 tx=0).stop 2024-05-27T07:51:09.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.198+0000 7fac15600700 1 -- 172.21.15.90:0/1608351926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac10107c00 msgr2=0x7fac1010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:09.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.198+0000 7fac15600700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac10107c00 0x7fac1010f320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00b580 tx=0x7fabfc0055f0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.199+0000 7fac15600700 1 -- 172.21.15.90:0/1608351926 shutdown_connections 2024-05-27T07:51:09.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.199+0000 7fac15600700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fabf40b2aa0 0x7fabf40b4fb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.199+0000 7fac15600700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac10108550 0x7fac101a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.199+0000 7fac15600700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac101032d0 0x7fac1010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.199+0000 7fac15600700 1 --2- 172.21.15.90:0/1608351926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac10107c00 0x7fac1010f320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:09.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.199+0000 7fac15600700 1 -- 172.21.15.90:0/1608351926 >> 172.21.15.90:0/1608351926 conn(0x7fac10078ab0 msgr2=0x7fac10114620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:09.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.200+0000 7fac15600700 1 -- 172.21.15.90:0/1608351926 shutdown_connections 2024-05-27T07:51:09.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:09.200+0000 7fac15600700 1 -- 172.21.15.90:0/1608351926 wait complete. 2024-05-27T07:51:10.146 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:51:10.147 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:51:10.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:10 smithi090 ceph-mon[31429]: pgmap v150: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 6.7 MiB/s wr, 907 op/s 2024-05-27T07:51:10.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:10 smithi136 ceph-mon[35324]: pgmap v150: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 6.7 MiB/s wr, 907 op/s 2024-05-27T07:51:10.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:10 smithi129 ceph-mon[35254]: pgmap v150: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 6.7 MiB/s wr, 907 op/s 2024-05-27T07:51:10.860 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:12.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:12 smithi090 ceph-mon[31429]: pgmap v151: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.2 MiB/s wr, 604 op/s 2024-05-27T07:51:12.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:12 smithi136 ceph-mon[35324]: pgmap v151: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.2 MiB/s wr, 604 op/s 2024-05-27T07:51:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:12 smithi129 ceph-mon[35254]: pgmap v151: 129 pgs: 129 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 4.2 MiB/s wr, 604 op/s 2024-05-27T07:51:13.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 55.0 s, 537.0 tps, lat 186.131 ms stddev 238.340 2024-05-27T07:51:14.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.056+0000 7fdcea800700 1 -- 172.21.15.90:0/431513477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce411d920 msgr2=0x7fdce411fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:14.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.056+0000 7fdcea800700 1 --2- 172.21.15.90:0/431513477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce411d920 0x7fdce411fd40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00b0a0 tx=0x7fdcdc00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.057+0000 7fdcea800700 1 -- 172.21.15.90:0/431513477 shutdown_connections 2024-05-27T07:51:14.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.057+0000 7fdcea800700 1 --2- 172.21.15.90:0/431513477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce411d920 0x7fdce411fd40 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.057+0000 7fdcea800700 1 --2- 172.21.15.90:0/431513477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdce410a920 0x7fdce410ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.057+0000 7fdcea800700 1 --2- 172.21.15.90:0/431513477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdce410b270 0x7fdce411d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.057+0000 7fdcea800700 1 -- 172.21.15.90:0/431513477 >> 172.21.15.90:0/431513477 conn(0x7fdce406f4c0 msgr2=0x7fdce406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:14.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.058+0000 7fdcea800700 1 -- 172.21.15.90:0/431513477 shutdown_connections 2024-05-27T07:51:14.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.059+0000 7fdcea800700 1 -- 172.21.15.90:0/431513477 wait complete. 2024-05-27T07:51:14.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.060+0000 7fdcea800700 1 Processor -- start 2024-05-27T07:51:14.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.060+0000 7fdcea800700 1 -- start start 2024-05-27T07:51:14.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.061+0000 7fdcea800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdce410a920 0x7fdce41af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:14.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.061+0000 7fdcea800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdce410b270 0x7fdce41afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:14.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.061+0000 7fdcea800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce411d920 0x7fdce41a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:14.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.061+0000 7fdcea800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4123160 con 0x7fdce411d920 2024-05-27T07:51:14.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.061+0000 7fdcea800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4122fe0 con 0x7fdce410b270 2024-05-27T07:51:14.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.062+0000 7fdce3400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdce410a920 0x7fdce41af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:14.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.061+0000 7fdcea800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41232e0 con 0x7fdce410a920 2024-05-27T07:51:14.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.062+0000 7fdce3400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdce410a920 0x7fdce41af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:58826/0 (socket says 172.21.15.90:58826) 2024-05-27T07:51:14.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.062+0000 7fdce3400700 1 -- 172.21.15.90:0/2285950697 learned_addr learned my addr 172.21.15.90:0/2285950697 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:14.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.063+0000 7fdce2a00700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdce410b270 0x7fdce41afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:14.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.063+0000 7fdce3400700 1 -- 172.21.15.90:0/2285950697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdce410b270 msgr2=0x7fdce41afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:14.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.063+0000 7fdce3400700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdce410b270 0x7fdce41afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.063+0000 7fdce3400700 1 -- 172.21.15.90:0/2285950697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce411d920 msgr2=0x7fdce41a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:14.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.063+0000 7fdce3400700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce411d920 0x7fdce41a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.063+0000 7fdce3400700 1 -- 172.21.15.90:0/2285950697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce41aa0b0 con 0x7fdce410a920 2024-05-27T07:51:14.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.064+0000 7fdce3400700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdce410a920 0x7fdce41af740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400d950 tx=0x7fdcd400dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:14.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.065+0000 7fdcdbe00700 1 -- 172.21.15.90:0/2285950697 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4016070 con 0x7fdce410a920 2024-05-27T07:51:14.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.065+0000 7fdcea800700 1 -- 172.21.15.90:0/2285950697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce41aa3a0 con 0x7fdce410a920 2024-05-27T07:51:14.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.065+0000 7fdcdbe00700 1 -- 172.21.15.90:0/2285950697 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcd4010460 con 0x7fdce410a920 2024-05-27T07:51:14.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.065+0000 7fdcdbe00700 1 -- 172.21.15.90:0/2285950697 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4020060 con 0x7fdce410a920 2024-05-27T07:51:14.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.066+0000 7fdcea800700 1 -- 172.21.15.90:0/2285950697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce41b65d0 con 0x7fdce410a920 2024-05-27T07:51:14.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.072+0000 7fdcea800700 1 -- 172.21.15.90:0/2285950697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce40512b0 con 0x7fdce410a920 2024-05-27T07:51:14.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.077+0000 7fdcdbe00700 1 -- 172.21.15.90:0/2285950697 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdcd4019070 con 0x7fdce410a920 2024-05-27T07:51:14.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.079+0000 7fdcdbe00700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcc80b2aa0 0x7fdcc80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:14.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.079+0000 7fdcdbe00700 1 -- 172.21.15.90:0/2285950697 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fdcd40e24b0 con 0x7fdce410a920 2024-05-27T07:51:14.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.081+0000 7fdce2a00700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcc80b2aa0 0x7fdcc80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:14.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.082+0000 7fdce2a00700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcc80b2aa0 0x7fdcc80b4fb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005950 tx=0x7fdcd00058e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:14.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.087+0000 7fdcdbe00700 1 -- 172.21.15.90:0/2285950697 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdcd40a9b20 con 0x7fdce410a920 2024-05-27T07:51:14.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:14 smithi090 ceph-mon[31429]: pgmap v152: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 7.0 MiB/s wr, 822 op/s 2024-05-27T07:51:14.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:14 smithi136 ceph-mon[35324]: pgmap v152: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 7.0 MiB/s wr, 822 op/s 2024-05-27T07:51:14.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:14 smithi129 ceph-mon[35254]: pgmap v152: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 7.0 MiB/s wr, 822 op/s 2024-05-27T07:51:14.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.568+0000 7fdcea800700 1 -- 172.21.15.90:0/2285950697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdce41b67a0 con 0x7fdce410a920 2024-05-27T07:51:14.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.571+0000 7fdcdbe00700 1 -- 172.21.15.90:0/2285950697 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11113 (secure 0 0 0) 0x7fdcd40a9420 con 0x7fdce410a920 2024-05-27T07:51:14.577 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:51:14.577 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2091177441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2091177441},{"type":"v1","addr":"172.21.15.90:6835","nonce":2091177441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:50:34.152852+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_16347":{"gid":16347,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6837/2432811315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":2432811315},{"type":"v1","addr":"172.21.15.90:6837","nonce":2432811315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34442,34424,24478,34406]},"id":1}]} 2024-05-27T07:51:14.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 -- 172.21.15.90:0/2285950697 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcc80b2aa0 msgr2=0x7fdcc80b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:14.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcc80b2aa0 0x7fdcc80b4fb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0005950 tx=0x7fdcd00058e0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 -- 172.21.15.90:0/2285950697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdce410a920 msgr2=0x7fdce41af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:14.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdce410a920 0x7fdce41af740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400d950 tx=0x7fdcd400dd10 comp rx=0 tx=0).stop 2024-05-27T07:51:14.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 -- 172.21.15.90:0/2285950697 shutdown_connections 2024-05-27T07:51:14.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce411d920 0x7fdce41a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdce410a920 0x7fdce41af740 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdcc80b2aa0 0x7fdcc80b4fb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 --2- 172.21.15.90:0/2285950697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdce410b270 0x7fdce41afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:14.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 -- 172.21.15.90:0/2285950697 >> 172.21.15.90:0/2285950697 conn(0x7fdce406f4c0 msgr2=0x7fdce411b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:14.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.575+0000 7fdcea800700 1 -- 172.21.15.90:0/2285950697 shutdown_connections 2024-05-27T07:51:14.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:14.576+0000 7fdcea800700 1 -- 172.21.15.90:0/2285950697 wait complete. 2024-05-27T07:51:14.584 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 42 2024-05-27T07:51:15.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:15 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2285950697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:51:15.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:15 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2285950697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:51:15.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:15 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2285950697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:51:16.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:16 smithi090 ceph-mon[31429]: pgmap v153: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 6.6 MiB/s wr, 718 op/s 2024-05-27T07:51:16.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:16 smithi136 ceph-mon[35324]: pgmap v153: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 6.6 MiB/s wr, 718 op/s 2024-05-27T07:51:16.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:16 smithi129 ceph-mon[35254]: pgmap v153: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 6.6 MiB/s wr, 718 op/s 2024-05-27T07:51:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.316+0000 7f6c6e400640 1 -- 172.21.15.90:0/4174252505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c68076530 msgr2=0x7f6c68076940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.316+0000 7f6c6e400640 1 --2- 172.21.15.90:0/4174252505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c68076530 0x7f6c68076940 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009900 tx=0x7f6c5402a4a0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.317+0000 7f6c6e400640 1 -- 172.21.15.90:0/4174252505 shutdown_connections 2024-05-27T07:51:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.317+0000 7f6c6e400640 1 --2- 172.21.15.90:0/4174252505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c6810db70 0x7f6c6810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.317+0000 7f6c6e400640 1 --2- 172.21.15.90:0/4174252505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c681104d0 0x7f6c681128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.317+0000 7f6c6e400640 1 --2- 172.21.15.90:0/4174252505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c68076530 0x7f6c68076940 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.317+0000 7f6c6e400640 1 -- 172.21.15.90:0/4174252505 >> 172.21.15.90:0/4174252505 conn(0x7f6c680ff340 msgr2=0x7f6c68101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.317+0000 7f6c6e400640 1 -- 172.21.15.90:0/4174252505 shutdown_connections 2024-05-27T07:51:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.317+0000 7f6c6e400640 1 -- 172.21.15.90:0/4174252505 wait complete. 2024-05-27T07:51:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.318+0000 7f6c6e400640 1 Processor -- start 2024-05-27T07:51:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.319+0000 7f6c6e400640 1 -- start start 2024-05-27T07:51:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.319+0000 7f6c6e400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c68076530 0x7f6c68108210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.319+0000 7f6c6e400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c6810db70 0x7f6c68108770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.319+0000 7f6c6e400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c681104d0 0x7f6c68073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.319+0000 7f6c6e400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68104010 con 0x7f6c681104d0 2024-05-27T07:51:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.319+0000 7f6c6e400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68103e90 con 0x7f6c6810db70 2024-05-27T07:51:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.319+0000 7f6c6e400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68104190 con 0x7f6c68076530 2024-05-27T07:51:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.320+0000 7f6c67400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c68076530 0x7f6c68108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.320+0000 7f6c66a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c6810db70 0x7f6c68108770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.320+0000 7f6c67400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c68076530 0x7f6c68108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:58842/0 (socket says 172.21.15.90:58842) 2024-05-27T07:51:17.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.320+0000 7f6c67400640 1 -- 172.21.15.90:0/1400784271 learned_addr learned my addr 172.21.15.90:0/1400784271 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.321+0000 7f6c67400640 1 -- 172.21.15.90:0/1400784271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c6810db70 msgr2=0x7f6c68108770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.321+0000 7f6c67400640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c6810db70 0x7f6c68108770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.321+0000 7f6c67400640 1 -- 172.21.15.90:0/1400784271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c681104d0 msgr2=0x7f6c68073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.321+0000 7f6c67400640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c681104d0 0x7f6c68073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.321+0000 7f6c67400640 1 -- 172.21.15.90:0/1400784271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c68074330 con 0x7f6c68076530 2024-05-27T07:51:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.321+0000 7f6c67400640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c68076530 0x7f6c68108210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6c5402aa10 tx=0x7f6c54003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.322+0000 7f6c5fe00640 1 -- 172.21.15.90:0/1400784271 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5403f070 con 0x7f6c68076530 2024-05-27T07:51:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.322+0000 7f6c5fe00640 1 -- 172.21.15.90:0/1400784271 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c5403bd00 con 0x7f6c68076530 2024-05-27T07:51:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.322+0000 7f6c5fe00640 1 -- 172.21.15.90:0/1400784271 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54035030 con 0x7f6c68076530 2024-05-27T07:51:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.322+0000 7f6c6e400640 1 -- 172.21.15.90:0/1400784271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c680745c0 con 0x7f6c68076530 2024-05-27T07:51:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.323+0000 7f6c6e400640 1 -- 172.21.15.90:0/1400784271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c6810bf10 con 0x7f6c68076530 2024-05-27T07:51:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.327+0000 7f6c7539e740 1 Processor -- start 2024-05-27T07:51:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.328+0000 7f6c7539e740 1 -- start start 2024-05-27T07:51:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.328+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 0x5584ff12c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.328+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed612f0 0x5584ff12cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.328+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 0x5584ff12d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.328+0000 7f6c7539e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5584ff09db90 con 0x5584fed694d0 2024-05-27T07:51:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.328+0000 7f6c7539e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5584ff08d730 con 0x5584fed612f0 2024-05-27T07:51:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.328+0000 7f6c7539e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5584ff091770 con 0x5584fed8a1b0 2024-05-27T07:51:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.329+0000 7f6c66a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 0x5584ff12d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.329+0000 7f6c66a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 0x5584ff12d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48274/0 (socket says 172.21.15.90:48274) 2024-05-27T07:51:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.329+0000 7f6c66a00640 1 -- 172.21.15.90:0/828020865 learned_addr learned my addr 172.21.15.90:0/828020865 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.329+0000 7f6c67e00640 1 --2- 172.21.15.90:0/828020865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 0x5584ff12c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.330+0000 7f6c67e00640 1 -- 172.21.15.90:0/828020865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed612f0 msgr2=0x5584ff12cda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:51:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.330+0000 7f6c67e00640 1 --2- 172.21.15.90:0/828020865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed612f0 0x5584ff12cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.330+0000 7f6c67e00640 1 -- 172.21.15.90:0/828020865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 msgr2=0x5584ff12d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.330+0000 7f6c67e00640 1 --2- 172.21.15.90:0/828020865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 0x5584ff12d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.330+0000 7f6c67e00640 1 -- 172.21.15.90:0/828020865 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ff0268b0 con 0x5584fed8a1b0 2024-05-27T07:51:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.331+0000 7f6c67e00640 1 --2- 172.21.15.90:0/828020865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 0x5584ff12c860 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6c60005fd0 tx=0x7f6c6000e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5987f1cc7208ed0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.331+0000 7f6c5cc00640 1 -- 172.21.15.90:0/828020865 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60016070 con 0x5584fed8a1b0 2024-05-27T07:51:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.331+0000 7f6c5cc00640 1 -- 172.21.15.90:0/828020865 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c60005020 con 0x5584fed8a1b0 2024-05-27T07:51:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.331+0000 7f6c5cc00640 1 -- 172.21.15.90:0/828020865 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c600116d0 con 0x5584fed8a1b0 2024-05-27T07:51:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.332+0000 7f6c7539e740 1 -- 172.21.15.90:0/828020865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 msgr2=0x5584ff12c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.332+0000 7f6c7539e740 1 --2- 172.21.15.90:0/828020865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 0x5584ff12c860 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6c60005fd0 tx=0x7f6c6000e6c0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.333+0000 7f6c7539e740 1 -- 172.21.15.90:0/828020865 shutdown_connections 2024-05-27T07:51:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.333+0000 7f6c7539e740 1 --2- 172.21.15.90:0/828020865 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed612f0 0x5584ff12cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.333+0000 7f6c7539e740 1 --2- 172.21.15.90:0/828020865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 0x5584ff12d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.333+0000 7f6c7539e740 1 --2- 172.21.15.90:0/828020865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 0x5584ff12c860 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.333+0000 7f6c7539e740 1 -- 172.21.15.90:0/828020865 >> 172.21.15.90:0/828020865 conn(0x5584fed7e300 msgr2=0x5584ff129010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.334+0000 7f6c5fe00640 1 -- 172.21.15.90:0/1400784271 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6c54042070 con 0x7f6c68076530 2024-05-27T07:51:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.335+0000 7f6c7539e740 1 -- 172.21.15.90:0/828020865 shutdown_connections 2024-05-27T07:51:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.335+0000 7f6c7539e740 1 -- 172.21.15.90:0/828020865 wait complete. 2024-05-27T07:51:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.336+0000 7f6c5fe00640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c4c0b2940 0x7f6c4c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.336+0000 7f6c7539e740 1 -- start start 2024-05-27T07:51:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.336+0000 7f6c5fe00640 1 -- 172.21.15.90:0/1400784271 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f6c541031d0 con 0x7f6c68076530 2024-05-27T07:51:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.336+0000 7f6c7539e740 1 Processor -- start 2024-05-27T07:51:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.337+0000 7f6c67e00640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c4c0b2940 0x7f6c4c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed694d0 0x5584ff136c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed8a1b0 0x5584ff1371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c7539e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5584fed73790 con 0x5584fed48d80 2024-05-27T07:51:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c7539e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5584ff035bc0 con 0x5584fed8a1b0 2024-05-27T07:51:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c7539e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5584ff139990 con 0x5584fed694d0 2024-05-27T07:51:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c7539e740 10 client.0 _ll_register_callbacks cb 0x5584ff013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c67e00640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c4c0b2940 0x7f6c4c0b4e50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000f010 tx=0x7f6c6001c040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c67400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed694d0 0x5584ff136c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c67400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed694d0 0x5584ff136c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:58864/0 (socket says 172.21.15.90:58864) 2024-05-27T07:51:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c67400640 1 -- 172.21.15.90:0/2610736994 learned_addr learned my addr 172.21.15.90:0/2610736994 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.338+0000 7f6c66a00640 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed8a1b0 0x5584ff1371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.339+0000 7f6c67e00640 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.339+0000 7f6c67400640 1 -- 172.21.15.90:0/2610736994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed8a1b0 msgr2=0x5584ff1371c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.339+0000 7f6c67400640 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed8a1b0 0x5584ff1371c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.339+0000 7f6c67400640 1 -- 172.21.15.90:0/2610736994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 msgr2=0x5584ff142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.339+0000 7f6c67400640 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff142cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.339+0000 7f6c67400640 1 -- 172.21.15.90:0/2610736994 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584feda2210 con 0x5584fed694d0 2024-05-27T07:51:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.339+0000 7f6c67e00640 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.339+0000 7f6c67400640 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed694d0 0x5584ff136c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c540c73b0 tx=0x7f6c540c9970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f1e0b365e6d896b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.339+0000 7f6c51600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:51:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.340+0000 7f6c51600640 1 -- 172.21.15.90:0/2610736994 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54049050 con 0x5584fed694d0 2024-05-27T07:51:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.340+0000 7f6c51600640 1 -- 172.21.15.90:0/2610736994 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f6c540ce120 con 0x5584fed694d0 2024-05-27T07:51:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.340+0000 7f6c7539e740 1 -- 172.21.15.90:0/2610736994 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c60005020 con 0x5584fed694d0 2024-05-27T07:51:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.341+0000 7f6c51600640 1 -- 172.21.15.90:0/2610736994 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6c540ca140 con 0x5584fed694d0 2024-05-27T07:51:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.341+0000 7f6c7539e740 10 client.26053 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.341+0000 7f6c7539e740 1 -- 172.21.15.90:0/2610736994 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5584fee6e620 con 0x5584fed694d0 2024-05-27T07:51:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.341+0000 7f6c51600640 1 -- 172.21.15.90:0/2610736994 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c60005020 con 0x5584fed694d0 2024-05-27T07:51:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.341+0000 7f6c51600640 1 -- 172.21.15.90:0/2610736994 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f6c540ca3a0 con 0x5584fed694d0 2024-05-27T07:51:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.342+0000 7f6c7539e740 10 client.26053 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.342+0000 7f6c7539e740 10 client.26053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.342+0000 7f6c7539e740 20 client.26053 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.342+0000 7f6c7539e740 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5584fed1e1c0 0x5584ff148c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.342+0000 7f6c7539e740 4 client.26053 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.342+0000 7f6c7539e740 1 -- 172.21.15.90:0/2610736994 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584ff09fae0 con 0x5584fed1e1c0 2024-05-27T07:51:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.343+0000 7f6c66a00640 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5584fed1e1c0 0x5584ff148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.344+0000 7f6c51600640 10 client.26053 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.344+0000 7f6c66a00640 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5584fed1e1c0 0x5584ff148c60 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe188eea0475839 server_cookie=4d965e6edfcb6f45 in_seq=0 out_seq=0 2024-05-27T07:51:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.345+0000 7f6c51600640 1 -- 172.21.15.90:0/2610736994 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5584ff09fae0 con 0x5584fed1e1c0 2024-05-27T07:51:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.345+0000 7f6c51600640 10 client.26053 handle_command_reply: tid=0 2024-05-27T07:51:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.345+0000 7f6c7539e740 1 client.26053 shutdown 2024-05-27T07:51:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.348+0000 7f6c7539e740 1 -- 172.21.15.90:0/2610736994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed694d0 msgr2=0x5584ff136c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.348+0000 7f6c7539e740 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed694d0 0x5584ff136c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c540c73b0 tx=0x7f6c540c9970 comp rx=0 tx=0).stop 2024-05-27T07:51:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.349+0000 7f6c7539e740 1 -- 172.21.15.90:0/2610736994 shutdown_connections 2024-05-27T07:51:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.349+0000 7f6c7539e740 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5584fed1e1c0 0x5584ff148c60 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.349+0000 7f6c7539e740 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed8a1b0 0x5584ff1371c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.349+0000 7f6c7539e740 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.349+0000 7f6c7539e740 1 --2- 172.21.15.90:0/2610736994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed694d0 0x5584ff136c80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.349+0000 7f6c7539e740 1 -- 172.21.15.90:0/2610736994 >> 172.21.15.90:0/2610736994 conn(0x5584fed7e300 msgr2=0x5584ff12c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.349+0000 7f6c51600640 0 client.26053 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.350+0000 7f6c7539e740 1 -- 172.21.15.90:0/2610736994 shutdown_connections 2024-05-27T07:51:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.350+0000 7f6c7539e740 1 -- 172.21.15.90:0/2610736994 wait complete. 2024-05-27T07:51:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.351+0000 7f6c7539e740 20 client.26053 trim_cache size 0 max 16384 2024-05-27T07:51:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.385+0000 7f6c7539e740 1 Processor -- start 2024-05-27T07:51:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.385+0000 7f6c7539e740 1 -- start start 2024-05-27T07:51:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.385+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff132280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.385+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed694d0 0x5584ff138be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.385+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 0x5584ff136a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.385+0000 7f6c7539e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681c1a00 con 0x5584fed48d80 2024-05-27T07:51:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.385+0000 7f6c7539e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681bf7f0 con 0x5584fed694d0 2024-05-27T07:51:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.385+0000 7f6c7539e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681c1330 con 0x5584fed8a1b0 2024-05-27T07:51:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.386+0000 7f6c67400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff132280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.386+0000 7f6c67400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff132280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48294/0 (socket says 172.21.15.90:48294) 2024-05-27T07:51:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.386+0000 7f6c67400640 1 -- 172.21.15.90:0/2590303340 learned_addr learned my addr 172.21.15.90:0/2590303340 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.386+0000 7f6c67e00640 1 --2- 172.21.15.90:0/2590303340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 0x5584ff136a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.387+0000 7f6c66a00640 1 --2- 172.21.15.90:0/2590303340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed694d0 0x5584ff138be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.387+0000 7f6c67e00640 1 -- 172.21.15.90:0/2590303340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed694d0 msgr2=0x5584ff138be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.387+0000 7f6c67e00640 1 --2- 172.21.15.90:0/2590303340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed694d0 0x5584ff138be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.387+0000 7f6c67e00640 1 -- 172.21.15.90:0/2590303340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 msgr2=0x5584ff132280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.387+0000 7f6c67e00640 1 --2- 172.21.15.90:0/2590303340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff132280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.387+0000 7f6c67e00640 1 -- 172.21.15.90:0/2590303340 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ff130f80 con 0x5584fed8a1b0 2024-05-27T07:51:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.387+0000 7f6c67400640 1 --2- 172.21.15.90:0/2590303340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff132280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.388+0000 7f6c67e00640 1 --2- 172.21.15.90:0/2590303340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 0x5584ff136a30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5584ff124710 tx=0x7f6c600095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2380d76136417 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.388+0000 7f6c53e00640 1 -- 172.21.15.90:0/2590303340 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60016040 con 0x5584fed8a1b0 2024-05-27T07:51:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.388+0000 7f6c53e00640 1 -- 172.21.15.90:0/2590303340 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c6001a890 con 0x5584fed8a1b0 2024-05-27T07:51:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.388+0000 7f6c53e00640 1 -- 172.21.15.90:0/2590303340 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60024660 con 0x5584fed8a1b0 2024-05-27T07:51:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.389+0000 7f6c7539e740 1 -- 172.21.15.90:0/2590303340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 msgr2=0x5584ff136a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.389+0000 7f6c7539e740 1 --2- 172.21.15.90:0/2590303340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 0x5584ff136a30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5584ff124710 tx=0x7f6c600095e0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.389+0000 7f6c7539e740 1 -- 172.21.15.90:0/2590303340 shutdown_connections 2024-05-27T07:51:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.389+0000 7f6c7539e740 1 --2- 172.21.15.90:0/2590303340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed694d0 0x5584ff138be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.389+0000 7f6c7539e740 1 --2- 172.21.15.90:0/2590303340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed8a1b0 0x5584ff136a30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.389+0000 7f6c7539e740 1 --2- 172.21.15.90:0/2590303340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed48d80 0x5584ff132280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.390+0000 7f6c7539e740 1 -- 172.21.15.90:0/2590303340 >> 172.21.15.90:0/2590303340 conn(0x5584fed7e300 msgr2=0x5584ff136320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.391+0000 7f6c7539e740 1 -- 172.21.15.90:0/2590303340 shutdown_connections 2024-05-27T07:51:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.391+0000 7f6c7539e740 1 -- 172.21.15.90:0/2590303340 wait complete. 2024-05-27T07:51:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.391+0000 7f6c7539e740 1 -- start start 2024-05-27T07:51:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.392+0000 7f6c7539e740 1 Processor -- start 2024-05-27T07:51:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.393+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed48d80 0x5584ff13c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.393+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 0x5584ff13c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.393+0000 7f6c7539e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed8a1b0 0x5584ff1af330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.393+0000 7f6c7539e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681c0590 con 0x5584fed694d0 2024-05-27T07:51:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.393+0000 7f6c7539e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681bfec0 con 0x5584fed8a1b0 2024-05-27T07:51:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.393+0000 7f6c7539e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681c0c60 con 0x5584fed48d80 2024-05-27T07:51:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.393+0000 7f6c7539e740 10 client.0 _ll_register_callbacks cb 0x7f6c54003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.393+0000 7f6c67400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed48d80 0x5584ff13c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.393+0000 7f6c67400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed48d80 0x5584ff13c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:58880/0 (socket says 172.21.15.90:58880) 2024-05-27T07:51:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.393+0000 7f6c67400640 1 -- 172.21.15.90:0/1104522667 learned_addr learned my addr 172.21.15.90:0/1104522667 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.394+0000 7f6c67e00640 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed8a1b0 0x5584ff1af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.394+0000 7f6c66a00640 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 0x5584ff13c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.394+0000 7f6c67400640 1 -- 172.21.15.90:0/1104522667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed8a1b0 msgr2=0x5584ff1af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.394+0000 7f6c67400640 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed8a1b0 0x5584ff1af330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.394+0000 7f6c67400640 1 -- 172.21.15.90:0/1104522667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 msgr2=0x5584ff13c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.394+0000 7f6c67400640 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 0x5584ff13c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.394+0000 7f6c67400640 1 -- 172.21.15.90:0/1104522667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ff1af870 con 0x5584fed48d80 2024-05-27T07:51:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.395+0000 7f6c66a00640 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 0x5584ff13c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.395+0000 7f6c67400640 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed48d80 0x5584ff13c170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c540c73b0 tx=0x7f6c5402ab20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc558060b16b8d6a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.395+0000 7f6c51600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:51:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.395+0000 7f6c51600640 1 -- 172.21.15.90:0/1104522667 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c54049070 con 0x5584fed48d80 2024-05-27T07:51:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.395+0000 7f6c51600640 1 -- 172.21.15.90:0/1104522667 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f6c540ce120 con 0x5584fed48d80 2024-05-27T07:51:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.395+0000 7f6c7539e740 1 -- 172.21.15.90:0/1104522667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6001a890 con 0x5584fed48d80 2024-05-27T07:51:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.396+0000 7f6c51600640 1 -- 172.21.15.90:0/1104522667 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6c540ac050 con 0x5584fed48d80 2024-05-27T07:51:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.396+0000 7f6c7539e740 10 client.26059 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.396+0000 7f6c7539e740 1 -- 172.21.15.90:0/1104522667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5584ff1b0ad0 con 0x5584fed48d80 2024-05-27T07:51:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.396+0000 7f6c51600640 1 -- 172.21.15.90:0/1104522667 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c6001a890 con 0x5584fed48d80 2024-05-27T07:51:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.397+0000 7f6c51600640 1 -- 172.21.15.90:0/1104522667 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f6c540c93a0 con 0x5584fed48d80 2024-05-27T07:51:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.397+0000 7f6c7539e740 10 client.26059 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.397+0000 7f6c7539e740 10 client.26059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.397+0000 7f6c7539e740 20 client.26059 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.397+0000 7f6c7539e740 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5584ff1b0fc0 0x5584ff1b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.398+0000 7f6c7539e740 4 client.26059 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:51:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.398+0000 7f6c7539e740 1 -- 172.21.15.90:0/1104522667 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584ff0992c0 con 0x5584ff1b0fc0 2024-05-27T07:51:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.398+0000 7f6c66a00640 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5584ff1b0fc0 0x5584ff1b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.400+0000 7f6c66a00640 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5584ff1b0fc0 0x5584ff1b33e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6768bdf57d7d3bc8 server_cookie=e95116576fe2fd41 in_seq=0 out_seq=0 2024-05-27T07:51:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.400+0000 7f6c51600640 10 client.26059 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.450+0000 7f6c51600640 1 -- 172.21.15.90:0/1104522667 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5584ff0992c0 con 0x5584ff1b0fc0 2024-05-27T07:51:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.450+0000 7f6c51600640 10 client.26059 handle_command_reply: tid=0 2024-05-27T07:51:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.450+0000 7f6c7539e740 1 client.26059 shutdown 2024-05-27T07:51:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.451+0000 7f6c7539e740 1 -- 172.21.15.90:0/1104522667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed48d80 msgr2=0x5584ff13c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.451+0000 7f6c7539e740 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed48d80 0x5584ff13c170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c540c73b0 tx=0x7f6c5402ab20 comp rx=0 tx=0).stop 2024-05-27T07:51:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.452+0000 7f6c7539e740 1 -- 172.21.15.90:0/1104522667 shutdown_connections 2024-05-27T07:51:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.452+0000 7f6c7539e740 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5584ff1b0fc0 0x5584ff1b33e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.452+0000 7f6c51600640 0 client.26059 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.452+0000 7f6c7539e740 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584fed8a1b0 0x5584ff1af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.452+0000 7f6c7539e740 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584fed694d0 0x5584ff13c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.452+0000 7f6c7539e740 1 --2- 172.21.15.90:0/1104522667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584fed48d80 0x5584ff13c170 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.452+0000 7f6c7539e740 1 -- 172.21.15.90:0/1104522667 >> 172.21.15.90:0/1104522667 conn(0x5584fed7e300 msgr2=0x5584ff1388d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.452+0000 7f6c7539e740 1 -- 172.21.15.90:0/1104522667 shutdown_connections 2024-05-27T07:51:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.452+0000 7f6c7539e740 1 -- 172.21.15.90:0/1104522667 wait complete. 2024-05-27T07:51:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.452+0000 7f6c7539e740 20 client.26059 trim_cache size 0 max 16384 2024-05-27T07:51:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:51:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:51:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:51:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 -- 172.21.15.90:0/1400784271 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c4c0b2940 msgr2=0x7f6c4c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c4c0b2940 0x7f6c4c0b4e50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000f010 tx=0x7f6c6001c040 comp rx=0 tx=0).stop 2024-05-27T07:51:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 -- 172.21.15.90:0/1400784271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c68076530 msgr2=0x7f6c68108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c68076530 0x7f6c68108210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6c5402aa10 tx=0x7f6c54003cd0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 -- 172.21.15.90:0/1400784271 shutdown_connections 2024-05-27T07:51:17.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c4c0b2940 0x7f6c4c0b4e50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c6810db70 0x7f6c68108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c681104d0 0x7f6c68073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 --2- 172.21.15.90:0/1400784271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c68076530 0x7f6c68108210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 -- 172.21.15.90:0/1400784271 >> 172.21.15.90:0/1400784271 conn(0x7f6c680ff340 msgr2=0x7f6c681108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 -- 172.21.15.90:0/1400784271 shutdown_connections 2024-05-27T07:51:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:17.454+0000 7f6c6e400640 1 -- 172.21.15.90:0/1400784271 wait complete. 2024-05-27T07:51:18.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:18 smithi136 ceph-mon[35324]: pgmap v154: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.0 MiB/s wr, 640 op/s 2024-05-27T07:51:18.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:18 smithi129 ceph-mon[35254]: pgmap v154: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.0 MiB/s wr, 640 op/s 2024-05-27T07:51:18.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:18 smithi090 ceph-mon[31429]: pgmap v154: 129 pgs: 129 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.0 MiB/s wr, 640 op/s 2024-05-27T07:51:18.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 60.0 s, 610.0 tps, lat 163.690 ms stddev 84.483 2024-05-27T07:51:19.304 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:51:20.022 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:20.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:20 smithi090 ceph-mon[31429]: pgmap v155: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 8.5 MiB/s wr, 938 op/s 2024-05-27T07:51:20.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:20 smithi136 ceph-mon[35324]: pgmap v155: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 8.5 MiB/s wr, 938 op/s 2024-05-27T07:51:20.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:20 smithi129 ceph-mon[35254]: pgmap v155: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 8.5 MiB/s wr, 938 op/s 2024-05-27T07:51:21.441 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:21 smithi090 ceph-mon[31429]: pgmap v156: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.2 MiB/s wr, 573 op/s 2024-05-27T07:51:21.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:21 smithi129 ceph-mon[35254]: pgmap v156: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.2 MiB/s wr, 573 op/s 2024-05-27T07:51:21.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:21 smithi136 ceph-mon[35324]: pgmap v156: 129 pgs: 129 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.2 MiB/s wr, 573 op/s 2024-05-27T07:51:23.242 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:51:23.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.509+0000 7f7f97800700 1 -- 172.21.15.90:0/906902260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f90108730 msgr2=0x7f7f9006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:23.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.509+0000 7f7f97800700 1 --2- 172.21.15.90:0/906902260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f90108730 0x7f7f9006b600 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800b0a0 tx=0x7f7f8800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.510+0000 7f7f97800700 1 -- 172.21.15.90:0/906902260 shutdown_connections 2024-05-27T07:51:23.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.510+0000 7f7f97800700 1 --2- 172.21.15.90:0/906902260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f90108730 0x7f7f9006b600 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.510+0000 7f7f97800700 1 --2- 172.21.15.90:0/906902260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f901034b0 0x7f7f901038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.510+0000 7f7f97800700 1 --2- 172.21.15.90:0/906902260 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f90107de0 0x7f7f901081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.510+0000 7f7f97800700 1 -- 172.21.15.90:0/906902260 >> 172.21.15.90:0/906902260 conn(0x7f7f900fedb0 msgr2=0x7f7f901011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:23.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.510+0000 7f7f97800700 1 -- 172.21.15.90:0/906902260 shutdown_connections 2024-05-27T07:51:23.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.510+0000 7f7f97800700 1 -- 172.21.15.90:0/906902260 wait complete. 2024-05-27T07:51:23.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.512+0000 7f7f97800700 1 Processor -- start 2024-05-27T07:51:23.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.512+0000 7f7f97800700 1 -- start start 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.513+0000 7f7f97800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f901034b0 0x7f7f9019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.513+0000 7f7f97800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f90107de0 0x7f7f9019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.513+0000 7f7f97800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f90108730 0x7f7f901a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.513+0000 7f7f97800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901166b0 con 0x7f7f901034b0 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.513+0000 7f7f97800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9010c080 con 0x7f7f90108730 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.513+0000 7f7f97800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7f90116830 con 0x7f7f90107de0 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.513+0000 7f7f94e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f901034b0 0x7f7f9019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.513+0000 7f7f94e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f901034b0 0x7f7f9019d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58156/0 (socket says 172.21.15.90:58156) 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.513+0000 7f7f94e00700 1 -- 172.21.15.90:0/1216025284 learned_addr learned my addr 172.21.15.90:0/1216025284 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.514+0000 7f7f8fe00700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f90107de0 0x7f7f9019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.514+0000 7f7f94e00700 1 -- 172.21.15.90:0/1216025284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f90107de0 msgr2=0x7f7f9019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.514+0000 7f7f94e00700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f90107de0 0x7f7f9019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.514+0000 7f7f94e00700 1 -- 172.21.15.90:0/1216025284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f90108730 msgr2=0x7f7f901a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.514+0000 7f7f95800700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f90108730 0x7f7f901a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.514+0000 7f7f94e00700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f90108730 0x7f7f901a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.514+0000 7f7f94e00700 1 -- 172.21.15.90:0/1216025284 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f901a1f60 con 0x7f7f901034b0 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.516+0000 7f7f94e00700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f901034b0 0x7f7f9019d090 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400b530 tx=0x7f7f8400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.517+0000 7f7f8d600700 1 -- 172.21.15.90:0/1216025284 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84015020 con 0x7f7f901034b0 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.517+0000 7f7f8d600700 1 -- 172.21.15.90:0/1216025284 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f84011cc0 con 0x7f7f901034b0 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.517+0000 7f7f8d600700 1 -- 172.21.15.90:0/1216025284 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84010dc0 con 0x7f7f901034b0 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.517+0000 7f7f97800700 1 -- 172.21.15.90:0/1216025284 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f901a21f0 con 0x7f7f901034b0 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.518+0000 7f7f97800700 1 -- 172.21.15.90:0/1216025284 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f901a26e0 con 0x7f7f901034b0 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.525+0000 7f7fa63d2b80 1 Processor -- start 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.526+0000 7f7fa63d2b80 1 -- start start 2024-05-27T07:51:23.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.527+0000 7f7f8d600700 1 -- 172.21.15.90:0/1216025284 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7f84018020 con 0x7f7f901034b0 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.527+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca31330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.527+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca31870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.527+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 0x560beca3e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.527+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560bec95fe40 con 0x560bec8b2e40 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.527+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560bec94f9e0 con 0x560bec99e9f0 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.527+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560bec953a20 con 0x560beca31e70 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f94e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca31870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f95800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca31330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f94e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca31870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58170/0 (socket says 172.21.15.90:58170) 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f94e00700 1 -- 172.21.15.90:0/278794226 learned_addr learned my addr 172.21.15.90:0/278794226 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f8fe00700 1 --2- 172.21.15.90:0/278794226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 0x560beca3e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f94e00700 1 -- 172.21.15.90:0/278794226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 msgr2=0x560beca3e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f94e00700 1 --2- 172.21.15.90:0/278794226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 0x560beca3e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f94e00700 1 -- 172.21.15.90:0/278794226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 msgr2=0x560beca31330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f94e00700 1 --2- 172.21.15.90:0/278794226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca31330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f94e00700 1 -- 172.21.15.90:0/278794226 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bec918420 con 0x560bec8b2e40 2024-05-27T07:51:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.528+0000 7f7f8fe00700 1 --2- 172.21.15.90:0/278794226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 0x560beca3e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.529+0000 7f7f94e00700 1 --2- 172.21.15.90:0/278794226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca31870 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7f840df430 tx=0x7f7f840fb550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9128a63042cd3bf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.529+0000 7f7f82000700 1 -- 172.21.15.90:0/278794226 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84106070 con 0x560bec8b2e40 2024-05-27T07:51:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.529+0000 7f7f82000700 1 -- 172.21.15.90:0/278794226 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f840fbbf0 con 0x560bec8b2e40 2024-05-27T07:51:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.530+0000 7f7f82000700 1 -- 172.21.15.90:0/278794226 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84101410 con 0x560bec8b2e40 2024-05-27T07:51:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.533+0000 7f7f8d600700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7f780b2b40 0x7f7f780b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.533+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/278794226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 msgr2=0x560beca31870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.533+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/278794226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca31870 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7f840df430 tx=0x7f7f840fb550 comp rx=0 tx=0).stop 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.534+0000 7f7f8d600700 1 -- 172.21.15.90:0/1216025284 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f7f840e2000 con 0x7f7f901034b0 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.534+0000 7f7f95800700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7f780b2b40 0x7f7f780b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.535+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/278794226 shutdown_connections 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.535+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/278794226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 0x560beca3e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.535+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/278794226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca31330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.535+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/278794226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca31870 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.535+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/278794226 >> 172.21.15.90:0/278794226 conn(0x560bec94c390 msgr2=0x560beca2d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.536+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/278794226 shutdown_connections 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.537+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/278794226 wait complete. 2024-05-27T07:51:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.537+0000 7f7f95800700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7f780b2b40 0x7f7f780b5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009900 tx=0x7f7f88005040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.538+0000 7f7fa63d2b80 1 -- start start 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.539+0000 7f7fa63d2b80 1 Processor -- start 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.540+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca3b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.540+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bec99e9f0 0x560beca3bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.540+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560beca31e70 0x560beca3c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.542+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560bec855fa0 con 0x560bec8b2e40 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.542+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560bec8f0760 con 0x560beca31e70 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.542+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560beca3eb80 con 0x560bec99e9f0 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.542+0000 7f7fa63d2b80 10 client.0 _ll_register_callbacks cb 0x560bec8f9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.542+0000 7f7f8fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bec99e9f0 0x560beca3bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.542+0000 7f7f94e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca3b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.542+0000 7f7f94e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca3b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58184/0 (socket says 172.21.15.90:58184) 2024-05-27T07:51:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.542+0000 7f7f94e00700 1 -- 172.21.15.90:0/396894283 learned_addr learned my addr 172.21.15.90:0/396894283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.542+0000 7f7f95800700 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560beca31e70 0x560beca3c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.542+0000 7f7f8fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bec99e9f0 0x560beca3bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53094/0 (socket says 172.21.15.90:53094) 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.543+0000 7f7f95800700 1 -- 172.21.15.90:0/396894283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bec99e9f0 msgr2=0x560beca3bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.543+0000 7f7f95800700 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bec99e9f0 0x560beca3bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.543+0000 7f7f95800700 1 -- 172.21.15.90:0/396894283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 msgr2=0x560beca3b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.543+0000 7f7f95800700 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca3b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.543+0000 7f7f95800700 1 -- 172.21.15.90:0/396894283 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bec89f8d0 con 0x560beca31e70 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.543+0000 7f7f8fe00700 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bec99e9f0 0x560beca3bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.544+0000 7f7f95800700 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560beca31e70 0x560beca3c160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800b830 tx=0x7f7f8800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa29727b0593db4b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.544+0000 7f7f76a00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.544+0000 7f7f76a00700 1 -- 172.21.15.90:0/396894283 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8802c070 con 0x560beca31e70 2024-05-27T07:51:23.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.544+0000 7f7f76a00700 1 -- 172.21.15.90:0/396894283 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f7f880096e0 con 0x560beca31e70 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.545+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/396894283 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f840fbbf0 con 0x560beca31e70 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.545+0000 7f7f76a00700 1 -- 172.21.15.90:0/396894283 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7f880050b0 con 0x560beca31e70 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.545+0000 7f7fa63d2b80 10 client.36008 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.546+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/396894283 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560bec920060 con 0x560beca31e70 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.546+0000 7f7f76a00700 1 -- 172.21.15.90:0/396894283 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f840fbbf0 con 0x560beca31e70 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.548+0000 7f7f76a00700 1 -- 172.21.15.90:0/396894283 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f7f88011070 con 0x560beca31e70 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.548+0000 7f7fa63d2b80 10 client.36008 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.548+0000 7f7fa63d2b80 10 client.36008 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.548+0000 7f7fa63d2b80 20 client.36008 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.548+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x560beca4bad0 0x560beca4def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.548+0000 7f7fa63d2b80 4 client.36008 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.548+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/396894283 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bec961d90 con 0x560beca4bad0 2024-05-27T07:51:23.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.549+0000 7f7f94e00700 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x560beca4bad0 0x560beca4def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.550+0000 7f7f94e00700 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x560beca4bad0 0x560beca4def0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa4acaf79f7f6690 server_cookie=7972760aad7f8500 in_seq=0 out_seq=0 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.550+0000 7f7f76a00700 10 client.36008 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.552+0000 7f7f76a00700 1 -- 172.21.15.90:0/396894283 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560bec961d90 con 0x560beca4bad0 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.552+0000 7f7f76a00700 10 client.36008 handle_command_reply: tid=0 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.553+0000 7f7fa63d2b80 1 client.36008 shutdown 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.553+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/396894283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560beca31e70 msgr2=0x560beca3c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.553+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560beca31e70 0x560beca3c160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800b830 tx=0x7f7f8800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.554+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/396894283 shutdown_connections 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.554+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x560beca4bad0 0x560beca4def0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.554+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560beca31e70 0x560beca3c160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.554+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca3b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.554+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/396894283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bec99e9f0 0x560beca3bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.554+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/396894283 >> 172.21.15.90:0/396894283 conn(0x560bec94c390 msgr2=0x560beca313b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.554+0000 7f7f76a00700 0 client.36008 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.554+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/396894283 shutdown_connections 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.555+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/396894283 wait complete. 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.556+0000 7f7fa63d2b80 20 client.36008 trim_cache size 0 max 16384 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.585+0000 7f7fa63d2b80 1 Processor -- start 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.585+0000 7f7fa63d2b80 1 -- start start 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.586+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca36ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.586+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca31830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.586+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 0x560beca536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.586+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901ab960 con 0x560bec8b2e40 2024-05-27T07:51:23.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.586+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f900754c0 con 0x560bec99e9f0 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.586+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901ab290 con 0x560beca31e70 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.587+0000 7f7f94e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca36ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.587+0000 7f7f94e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca36ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58186/0 (socket says 172.21.15.90:58186) 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.587+0000 7f7f8fe00700 1 --2- 172.21.15.90:0/1713814580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca31830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.587+0000 7f7f95800700 1 --2- 172.21.15.90:0/1713814580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 0x560beca536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.587+0000 7f7f94e00700 1 -- 172.21.15.90:0/1713814580 learned_addr learned my addr 172.21.15.90:0/1713814580 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.588+0000 7f7f94e00700 1 -- 172.21.15.90:0/1713814580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 msgr2=0x560beca536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.588+0000 7f7f94e00700 1 --2- 172.21.15.90:0/1713814580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 0x560beca536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.588+0000 7f7f94e00700 1 -- 172.21.15.90:0/1713814580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 msgr2=0x560beca31830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.588+0000 7f7f94e00700 1 --2- 172.21.15.90:0/1713814580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca31830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.588+0000 7f7f94e00700 1 -- 172.21.15.90:0/1713814580 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560beca2e9c0 con 0x560bec8b2e40 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.588+0000 7f7f8fe00700 1 --2- 172.21.15.90:0/1713814580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca31830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.589+0000 7f7f94e00700 1 --2- 172.21.15.90:0/1713814580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca36ef0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7f840a8d80 tx=0x7f7f840fb1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92344f4087af4ba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.589+0000 7f7f81600700 1 -- 172.21.15.90:0/1713814580 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f840a4f40 con 0x560bec8b2e40 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.590+0000 7f7f81600700 1 -- 172.21.15.90:0/1713814580 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f840a8b90 con 0x560bec8b2e40 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.590+0000 7f7f81600700 1 -- 172.21.15.90:0/1713814580 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f840e59c0 con 0x560bec8b2e40 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.591+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/1713814580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 msgr2=0x560beca36ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.591+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/1713814580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca36ef0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7f840a8d80 tx=0x7f7f840fb1e0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.591+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/1713814580 shutdown_connections 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.591+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/1713814580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca31e70 0x560beca536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.591+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/1713814580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bec8b2e40 0x560beca36ef0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.591+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/1713814580 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca31830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.591+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/1713814580 >> 172.21.15.90:0/1713814580 conn(0x560bec94c390 msgr2=0x560beca31120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.591+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/1713814580 shutdown_connections 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.591+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/1713814580 wait complete. 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.592+0000 7f7fa63d2b80 1 -- start start 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.592+0000 7f7fa63d2b80 1 Processor -- start 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.593+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca35730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.593+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560beca31e70 0x560beca35c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.593+0000 7f7fa63d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca361b0 0x560beca42e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.593+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901aa4f0 con 0x560beca31e70 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.593+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901a9e20 con 0x560bec99e9f0 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.593+0000 7f7fa63d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901aabc0 con 0x560beca361b0 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.593+0000 7f7fa63d2b80 10 client.0 _ll_register_callbacks cb 0x560bec8f9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.593+0000 7f7f8fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560beca31e70 0x560beca35c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.594+0000 7f7f8fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560beca31e70 0x560beca35c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58200/0 (socket says 172.21.15.90:58200) 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.594+0000 7f7f8fe00700 1 -- 172.21.15.90:0/2893502989 learned_addr learned my addr 172.21.15.90:0/2893502989 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.594+0000 7f7f95800700 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca361b0 0x560beca42e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.594+0000 7f7f8fe00700 1 -- 172.21.15.90:0/2893502989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca361b0 msgr2=0x560beca42e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.595+0000 7f7f8fe00700 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca361b0 0x560beca42e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.595+0000 7f7f8fe00700 1 -- 172.21.15.90:0/2893502989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 msgr2=0x560beca35730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.595+0000 7f7f8fe00700 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca35730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.595+0000 7f7f8fe00700 1 -- 172.21.15.90:0/2893502989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560beca29240 con 0x560beca31e70 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.596+0000 7f7f8fe00700 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560beca31e70 0x560beca35c70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009f80 tx=0x7f7f7c0156f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c324da04449011 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.596+0000 7f7f76a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.596+0000 7f7f76a00700 1 -- 172.21.15.90:0/2893502989 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c01b070 con 0x560beca31e70 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.597+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/2893502989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f840a8b90 con 0x560beca31e70 2024-05-27T07:51:23.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.597+0000 7f7f76a00700 1 -- 172.21.15.90:0/2893502989 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f7f7c020660 con 0x560beca31e70 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.598+0000 7f7f76a00700 1 -- 172.21.15.90:0/2893502989 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7f7c00f040 con 0x560beca31e70 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.598+0000 7f7fa63d2b80 10 client.16521 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.598+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/2893502989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560beca445c0 con 0x560beca31e70 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.599+0000 7f7f76a00700 1 -- 172.21.15.90:0/2893502989 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f840a8b90 con 0x560beca31e70 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.599+0000 7f7f76a00700 1 -- 172.21.15.90:0/2893502989 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f7f7c017640 con 0x560beca31e70 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.599+0000 7f7fa63d2b80 10 client.16521 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.599+0000 7f7fa63d2b80 10 client.16521 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.599+0000 7f7fa63d2b80 20 client.16521 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.600+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x560becabee50 0x560becac1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.600+0000 7f7fa63d2b80 4 client.16521 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.600+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/2893502989 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560bec95b570 con 0x560becabee50 2024-05-27T07:51:23.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.600+0000 7f7f94e00700 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x560becabee50 0x560becac1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.601+0000 7f7f94e00700 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x560becabee50 0x560becac1270 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6bade84c1e25c0aa server_cookie=a8c1c19b6e1743be in_seq=0 out_seq=0 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.601+0000 7f7f76a00700 10 client.16521 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7f76a00700 1 -- 172.21.15.90:0/2893502989 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560bec95b570 con 0x560becabee50 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7f76a00700 10 client.16521 handle_command_reply: tid=0 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7fa63d2b80 1 client.16521 shutdown 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/2893502989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560beca31e70 msgr2=0x560beca35c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560beca31e70 0x560beca35c70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009f80 tx=0x7f7f7c0156f0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/2893502989 shutdown_connections 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x560becabee50 0x560becac1270 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560beca361b0 0x560beca42e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7f76a00700 0 client.16521 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:51:23.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560bec99e9f0 0x560beca35730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7fa63d2b80 1 --2- 172.21.15.90:0/2893502989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560beca31e70 0x560beca35c70 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.602+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/2893502989 >> 172.21.15.90:0/2893502989 conn(0x560bec94c390 msgr2=0x560beca32ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.603+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/2893502989 shutdown_connections 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.603+0000 7f7fa63d2b80 1 -- 172.21.15.90:0/2893502989 wait complete. 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.603+0000 7f7fa63d2b80 20 client.16521 trim_cache size 0 max 16384 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.604+0000 7f7f97800700 1 -- 172.21.15.90:0/1216025284 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7f780b2b40 msgr2=0x7f7f780b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.604+0000 7f7f97800700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7f780b2b40 0x7f7f780b5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009900 tx=0x7f7f88005040 comp rx=0 tx=0).stop 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.604+0000 7f7f97800700 1 -- 172.21.15.90:0/1216025284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f901034b0 msgr2=0x7f7f9019d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.604+0000 7f7f97800700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f901034b0 0x7f7f9019d090 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400b530 tx=0x7f7f8400b840 comp rx=0 tx=0).stop 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.604+0000 7f7f97800700 1 -- 172.21.15.90:0/1216025284 shutdown_connections 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.604+0000 7f7f97800700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7f780b2b40 0x7f7f780b5050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.604+0000 7f7f97800700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f90108730 0x7f7f901a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.604+0000 7f7f97800700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f901034b0 0x7f7f9019d090 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.604+0000 7f7f97800700 1 --2- 172.21.15.90:0/1216025284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f90107de0 0x7f7f9019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:23.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.604+0000 7f7f97800700 1 -- 172.21.15.90:0/1216025284 >> 172.21.15.90:0/1216025284 conn(0x7f7f900fedb0 msgr2=0x7f7f90105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:23.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.605+0000 7f7f97800700 1 -- 172.21.15.90:0/1216025284 shutdown_connections 2024-05-27T07:51:23.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:22.605+0000 7f7f97800700 1 -- 172.21.15.90:0/1216025284 wait complete. 2024-05-27T07:51:23.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 65.0 s, 543.4 tps, lat 169.157 ms stddev 59.223 2024-05-27T07:51:24.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:24 smithi090 ceph-mon[31429]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.2 MiB/s wr, 822 op/s 2024-05-27T07:51:24.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:24 smithi136 ceph-mon[35324]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.2 MiB/s wr, 822 op/s 2024-05-27T07:51:24.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:24 smithi129 ceph-mon[35254]: pgmap v157: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 8.2 MiB/s wr, 822 op/s 2024-05-27T07:51:26.366 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:26 smithi129 ceph-mon[35254]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.0 MiB/s wr, 666 op/s 2024-05-27T07:51:26.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:26 smithi090 ceph-mon[31429]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.0 MiB/s wr, 666 op/s 2024-05-27T07:51:26.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:26 smithi136 ceph-mon[35324]: pgmap v158: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.0 MiB/s wr, 666 op/s 2024-05-27T07:51:26.985 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:51:27.719 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:28.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:28 smithi090 ceph-mon[31429]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.1 MiB/s wr, 608 op/s 2024-05-27T07:51:28.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:28 smithi136 ceph-mon[35324]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.1 MiB/s wr, 608 op/s 2024-05-27T07:51:28.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:28 smithi129 ceph-mon[35254]: pgmap v159: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.1 MiB/s wr, 608 op/s 2024-05-27T07:51:28.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 70.0 s, 541.8 tps, lat 199.240 ms stddev 127.601 2024-05-27T07:51:30.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:30 smithi090 ceph-mon[31429]: pgmap v160: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 7.2 MiB/s wr, 857 op/s 2024-05-27T07:51:30.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:30 smithi136 ceph-mon[35324]: pgmap v160: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 7.2 MiB/s wr, 857 op/s 2024-05-27T07:51:30.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:30 smithi129 ceph-mon[35254]: pgmap v160: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 7.2 MiB/s wr, 857 op/s 2024-05-27T07:51:30.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.651+0000 7fa32a200700 1 -- 172.21.15.90:0/776327238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32410abf0 msgr2=0x7fa32410b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.651+0000 7fa32a200700 1 --2- 172.21.15.90:0/776327238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32410abf0 0x7fa32410b000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa3100099c0 tx=0x7fa310009cd0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.652+0000 7fa32a200700 1 -- 172.21.15.90:0/776327238 shutdown_connections 2024-05-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.652+0000 7fa32a200700 1 --2- 172.21.15.90:0/776327238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa32410b540 0x7fa324119890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.652+0000 7fa32a200700 1 --2- 172.21.15.90:0/776327238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa324111bb0 0x7fa324111fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.652+0000 7fa32a200700 1 --2- 172.21.15.90:0/776327238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32410abf0 0x7fa32410b000 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.652+0000 7fa32a200700 1 -- 172.21.15.90:0/776327238 >> 172.21.15.90:0/776327238 conn(0x7fa32406f980 msgr2=0x7fa32406fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.653+0000 7fa32a200700 1 -- 172.21.15.90:0/776327238 shutdown_connections 2024-05-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.653+0000 7fa32a200700 1 -- 172.21.15.90:0/776327238 wait complete. 2024-05-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.654+0000 7fa32a200700 1 Processor -- start 2024-05-27T07:51:30.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.655+0000 7fa32a200700 1 -- start start 2024-05-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.655+0000 7fa32a200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32410abf0 0x7fa3241a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.655+0000 7fa32a200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa32410b540 0x7fa3241a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.655+0000 7fa32a200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa324111bb0 0x7fa3241a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.656+0000 7fa32a200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa32411eda0 con 0x7fa324111bb0 2024-05-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.656+0000 7fa32a200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa32411ec20 con 0x7fa32410b540 2024-05-27T07:51:30.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.656+0000 7fa32a200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa32411ef20 con 0x7fa32410abf0 2024-05-27T07:51:30.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.656+0000 7fa322a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa32410b540 0x7fa3241a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.656+0000 7fa322a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa32410b540 0x7fa3241a5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:58530/0 (socket says 172.21.15.90:58530) 2024-05-27T07:51:30.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.656+0000 7fa322a00700 1 -- 172.21.15.90:0/3239549923 learned_addr learned my addr 172.21.15.90:0/3239549923 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:30.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.656+0000 7fa323400700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32410abf0 0x7fa3241a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.657+0000 7fa323e00700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa324111bb0 0x7fa3241a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.657+0000 7fa322a00700 1 -- 172.21.15.90:0/3239549923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32410abf0 msgr2=0x7fa3241a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.657+0000 7fa322a00700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32410abf0 0x7fa3241a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.657+0000 7fa322a00700 1 -- 172.21.15.90:0/3239549923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa324111bb0 msgr2=0x7fa3241a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.658+0000 7fa322a00700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa324111bb0 0x7fa3241a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.658+0000 7fa322a00700 1 -- 172.21.15.90:0/3239549923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3241aa620 con 0x7fa32410b540 2024-05-27T07:51:30.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.658+0000 7fa322a00700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa32410b540 0x7fa3241a5cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa310005f50 tx=0x7fa3100057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.658+0000 7fa31be00700 1 -- 172.21.15.90:0/3239549923 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310027070 con 0x7fa32410b540 2024-05-27T07:51:30.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.658+0000 7fa32a200700 1 -- 172.21.15.90:0/3239549923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3241aa8b0 con 0x7fa32410b540 2024-05-27T07:51:30.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.659+0000 7fa32a200700 1 -- 172.21.15.90:0/3239549923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3241b2230 con 0x7fa32410b540 2024-05-27T07:51:30.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.659+0000 7fa31be00700 1 -- 172.21.15.90:0/3239549923 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa31000fe20 con 0x7fa32410b540 2024-05-27T07:51:30.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.659+0000 7fa31be00700 1 -- 172.21.15.90:0/3239549923 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310022c70 con 0x7fa32410b540 2024-05-27T07:51:30.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.669+0000 7fa31be00700 1 -- 172.21.15.90:0/3239549923 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa310022470 con 0x7fa32410b540 2024-05-27T07:51:30.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.672+0000 7fa31be00700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa3080b2af0 0x7fa3080b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.673+0000 7fa323400700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa3080b2af0 0x7fa3080b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.674+0000 7fa338e0eb80 1 Processor -- start 2024-05-27T07:51:30.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.674+0000 7fa31be00700 1 -- 172.21.15.90:0/3239549923 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fa3100e36d0 con 0x7fa32410b540 2024-05-27T07:51:30.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.674+0000 7fa338e0eb80 1 -- start start 2024-05-27T07:51:30.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.674+0000 7fa323400700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa3080b2af0 0x7fa3080b5000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa314005950 tx=0x7fa31400a300 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.675+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f1791230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.675+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 0x5629f1791770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.675+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1791d00 0x5629f17a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.676+0000 7fa323400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f1791230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.676+0000 7fa323400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f1791230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59360/0 (socket says 172.21.15.90:59360) 2024-05-27T07:51:30.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.676+0000 7fa338e0eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629f16bfe40 con 0x5629f1791d00 2024-05-27T07:51:30.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.676+0000 7fa338e0eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5629f16af9e0 con 0x5629f1612e40 2024-05-27T07:51:30.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.676+0000 7fa323e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1791d00 0x5629f17a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.676+0000 7fa338e0eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5629f16b3a20 con 0x5629f16fe9f0 2024-05-27T07:51:30.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.676+0000 7fa323400700 1 -- 172.21.15.90:0/1533309225 learned_addr learned my addr 172.21.15.90:0/1533309225 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:30.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.677+0000 7fa322a00700 1 --2- 172.21.15.90:0/1533309225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 0x5629f1791770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.677+0000 7fa323e00700 1 -- 172.21.15.90:0/1533309225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 msgr2=0x5629f1791230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.677+0000 7fa323e00700 1 --2- 172.21.15.90:0/1533309225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f1791230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.677+0000 7fa323e00700 1 -- 172.21.15.90:0/1533309225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 msgr2=0x5629f1791770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.677+0000 7fa323e00700 1 --2- 172.21.15.90:0/1533309225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 0x5629f1791770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.677+0000 7fa323e00700 1 -- 172.21.15.90:0/1533309225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f1678420 con 0x5629f1791d00 2024-05-27T07:51:30.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.678+0000 7fa323400700 1 --2- 172.21.15.90:0/1533309225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f1791230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:30.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.678+0000 7fa323e00700 1 --2- 172.21.15.90:0/1533309225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1791d00 0x5629f17a6920 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009ca0 tx=0x7fa31c007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fb5f8bf091a017 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.680+0000 7fa318c00700 1 -- 172.21.15.90:0/1533309225 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c007cd0 con 0x5629f1791d00 2024-05-27T07:51:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.680+0000 7fa318c00700 1 -- 172.21.15.90:0/1533309225 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa31c004bb0 con 0x5629f1791d00 2024-05-27T07:51:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.680+0000 7fa318c00700 1 -- 172.21.15.90:0/1533309225 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c007cd0 con 0x5629f1791d00 2024-05-27T07:51:30.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.685+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1533309225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1791d00 msgr2=0x5629f17a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.685+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/1533309225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1791d00 0x5629f17a6920 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009ca0 tx=0x7fa31c007500 comp rx=0 tx=0).stop 2024-05-27T07:51:30.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.686+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1533309225 shutdown_connections 2024-05-27T07:51:30.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.686+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/1533309225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1791d00 0x5629f17a6920 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.686+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/1533309225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f1791230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.686+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/1533309225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 0x5629f1791770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.686+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1533309225 >> 172.21.15.90:0/1533309225 conn(0x5629f16ac390 msgr2=0x5629f178d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:30.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.686+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1533309225 shutdown_connections 2024-05-27T07:51:30.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.686+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1533309225 wait complete. 2024-05-27T07:51:30.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.687+0000 7fa338e0eb80 1 -- start start 2024-05-27T07:51:30.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.687+0000 7fa338e0eb80 1 Processor -- start 2024-05-27T07:51:30.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.688+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 0x5629f17a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.689+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f17a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.689+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1791d00 0x5629f17a4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.689+0000 7fa338e0eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629f15b5fa0 con 0x5629f1791d00 2024-05-27T07:51:30.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.689+0000 7fa338e0eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5629f1650760 con 0x5629f1612e40 2024-05-27T07:51:30.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.689+0000 7fa338e0eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5629f17a73f0 con 0x5629f16fe9f0 2024-05-27T07:51:30.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.689+0000 7fa338e0eb80 10 client.0 _ll_register_callbacks cb 0x5629f1659c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:30.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.690+0000 7fa322a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f17a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.690+0000 7fa323400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 0x5629f17a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.690+0000 7fa323400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 0x5629f17a3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:58540/0 (socket says 172.21.15.90:58540) 2024-05-27T07:51:30.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.690+0000 7fa323400700 1 -- 172.21.15.90:0/863862210 learned_addr learned my addr 172.21.15.90:0/863862210 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:30.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.691+0000 7fa323400700 1 -- 172.21.15.90:0/863862210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 msgr2=0x5629f17a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.691+0000 7fa323400700 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f17a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.691+0000 7fa323400700 1 -- 172.21.15.90:0/863862210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1791d00 msgr2=0x5629f17a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.691+0000 7fa323400700 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1791d00 0x5629f17a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.691+0000 7fa323400700 1 -- 172.21.15.90:0/863862210 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f15ff8d0 con 0x5629f1612e40 2024-05-27T07:51:30.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.692+0000 7fa323400700 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 0x5629f17a3de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa31401da80 tx=0x7fa3140047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d6077873b0efe95 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.692+0000 7fa30d600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:51:30.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.692+0000 7fa30d600700 1 -- 172.21.15.90:0/863862210 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314016070 con 0x5629f1612e40 2024-05-27T07:51:30.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.692+0000 7fa30d600700 1 -- 172.21.15.90:0/863862210 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fa314010840 con 0x5629f1612e40 2024-05-27T07:51:30.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.692+0000 7fa338e0eb80 1 -- 172.21.15.90:0/863862210 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa31c004bb0 con 0x5629f1612e40 2024-05-27T07:51:30.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.693+0000 7fa30d600700 1 -- 172.21.15.90:0/863862210 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa314032050 con 0x5629f1612e40 2024-05-27T07:51:30.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.693+0000 7fa338e0eb80 10 client.36020 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:30.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.693+0000 7fa338e0eb80 1 -- 172.21.15.90:0/863862210 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5629f1680060 con 0x5629f1612e40 2024-05-27T07:51:30.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.694+0000 7fa30d600700 1 -- 172.21.15.90:0/863862210 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa31c004bb0 con 0x5629f1612e40 2024-05-27T07:51:30.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.695+0000 7fa30d600700 1 -- 172.21.15.90:0/863862210 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fa314027be0 con 0x5629f1612e40 2024-05-27T07:51:30.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.696+0000 7fa338e0eb80 10 client.36020 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:30.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.696+0000 7fa338e0eb80 10 client.36020 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:51:30.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.696+0000 7fa338e0eb80 20 client.36020 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:30.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.696+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5629f17b42b0 0x5629f17b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.696+0000 7fa338e0eb80 4 client.36020 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:30.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.696+0000 7fa338e0eb80 1 -- 172.21.15.90:0/863862210 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629f16c1d90 con 0x5629f17b42b0 2024-05-27T07:51:30.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.697+0000 7fa322a00700 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5629f17b42b0 0x5629f17b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.698+0000 7fa30d600700 10 client.36020 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:51:30.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.698+0000 7fa322a00700 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5629f17b42b0 0x5629f17b6750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8bd327a4f8553faa server_cookie=bdfe038ae73ff741 in_seq=0 out_seq=0 2024-05-27T07:51:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.699+0000 7fa30d600700 1 -- 172.21.15.90:0/863862210 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5629f16c1d90 con 0x5629f17b42b0 2024-05-27T07:51:30.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.699+0000 7fa30d600700 10 client.36020 handle_command_reply: tid=0 2024-05-27T07:51:30.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.700+0000 7fa338e0eb80 1 client.36020 shutdown 2024-05-27T07:51:30.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.700+0000 7fa338e0eb80 1 -- 172.21.15.90:0/863862210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 msgr2=0x5629f17a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.700+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 0x5629f17a3de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa31401da80 tx=0x7fa3140047c0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.701+0000 7fa338e0eb80 1 -- 172.21.15.90:0/863862210 shutdown_connections 2024-05-27T07:51:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.701+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5629f17b42b0 0x5629f17b6750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.701+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1791d00 0x5629f17a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.701+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1612e40 0x5629f17a3de0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.701+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/863862210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f17a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.701+0000 7fa338e0eb80 1 -- 172.21.15.90:0/863862210 >> 172.21.15.90:0/863862210 conn(0x5629f16ac390 msgr2=0x5629f1791190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.701+0000 7fa30d600700 0 client.36020 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:51:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.701+0000 7fa338e0eb80 1 -- 172.21.15.90:0/863862210 shutdown_connections 2024-05-27T07:51:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.702+0000 7fa338e0eb80 1 -- 172.21.15.90:0/863862210 wait complete. 2024-05-27T07:51:30.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.702+0000 7fa338e0eb80 20 client.36020 trim_cache size 0 max 16384 2024-05-27T07:51:30.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.737+0000 7fa338e0eb80 1 Processor -- start 2024-05-27T07:51:30.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.737+0000 7fa338e0eb80 1 -- start start 2024-05-27T07:51:30.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.737+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f1612e40 0x5629f179f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.737+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f16fe9f0 0x5629f17a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.737+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 0x5629f17b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.737+0000 7fa338e0eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa304010780 con 0x5629f16fe9f0 2024-05-27T07:51:30.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.737+0000 7fa338e0eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa30400e570 con 0x5629f1791d00 2024-05-27T07:51:30.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.737+0000 7fa338e0eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa3040100b0 con 0x5629f1612e40 2024-05-27T07:51:30.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.738+0000 7fa322a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f16fe9f0 0x5629f17a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.738+0000 7fa322a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f16fe9f0 0x5629f17a2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55580/0 (socket says 172.21.15.90:55580) 2024-05-27T07:51:30.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.738+0000 7fa322a00700 1 -- 172.21.15.90:0/3000250527 learned_addr learned my addr 172.21.15.90:0/3000250527 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:30.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.738+0000 7fa323e00700 1 --2- 172.21.15.90:0/3000250527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 0x5629f17b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.739+0000 7fa323400700 1 --2- 172.21.15.90:0/3000250527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f1612e40 0x5629f179f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.739+0000 7fa322a00700 1 -- 172.21.15.90:0/3000250527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f1612e40 msgr2=0x5629f179f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.739+0000 7fa322a00700 1 --2- 172.21.15.90:0/3000250527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f1612e40 0x5629f179f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.739+0000 7fa322a00700 1 -- 172.21.15.90:0/3000250527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 msgr2=0x5629f17b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.739+0000 7fa322a00700 1 --2- 172.21.15.90:0/3000250527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 0x5629f17b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.739+0000 7fa322a00700 1 -- 172.21.15.90:0/3000250527 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f17a4bd0 con 0x5629f16fe9f0 2024-05-27T07:51:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.740+0000 7fa322a00700 1 --2- 172.21.15.90:0/3000250527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f16fe9f0 0x5629f17a2d40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa3100a6710 tx=0x7fa31001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8cd6ff88bd4f74 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.740+0000 7fa30fe00700 1 -- 172.21.15.90:0/3000250527 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100aa690 con 0x5629f16fe9f0 2024-05-27T07:51:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.740+0000 7fa30fe00700 1 -- 172.21.15.90:0/3000250527 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3100aea40 con 0x5629f16fe9f0 2024-05-27T07:51:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.740+0000 7fa30fe00700 1 -- 172.21.15.90:0/3000250527 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100aa8f0 con 0x5629f16fe9f0 2024-05-27T07:51:30.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.741+0000 7fa338e0eb80 1 -- 172.21.15.90:0/3000250527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f16fe9f0 msgr2=0x5629f17a2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.741+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/3000250527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f16fe9f0 0x5629f17a2d40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa3100a6710 tx=0x7fa31001a040 comp rx=0 tx=0).stop 2024-05-27T07:51:30.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.742+0000 7fa338e0eb80 1 -- 172.21.15.90:0/3000250527 shutdown_connections 2024-05-27T07:51:30.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.742+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/3000250527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 0x5629f17b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.742+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/3000250527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f1612e40 0x5629f179f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.742+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/3000250527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f16fe9f0 0x5629f17a2d40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.742+0000 7fa338e0eb80 1 -- 172.21.15.90:0/3000250527 >> 172.21.15.90:0/3000250527 conn(0x5629f16ac390 msgr2=0x5629f1792f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:30.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.744+0000 7fa338e0eb80 1 -- 172.21.15.90:0/3000250527 shutdown_connections 2024-05-27T07:51:30.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.744+0000 7fa338e0eb80 1 -- 172.21.15.90:0/3000250527 wait complete. 2024-05-27T07:51:30.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.745+0000 7fa338e0eb80 1 -- start start 2024-05-27T07:51:30.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.745+0000 7fa338e0eb80 1 Processor -- start 2024-05-27T07:51:30.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.747+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1612e40 0x5629f179e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.747+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f179ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.747+0000 7fa338e0eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 0x5629f179f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.748+0000 7fa323400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1612e40 0x5629f179e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.748+0000 7fa322a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f179ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.748+0000 7fa323400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1612e40 0x5629f179e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55590/0 (socket says 172.21.15.90:55590) 2024-05-27T07:51:30.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.748+0000 7fa323400700 1 -- 172.21.15.90:0/1831454097 learned_addr learned my addr 172.21.15.90:0/1831454097 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:30.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.749+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa30400f310 con 0x5629f1612e40 2024-05-27T07:51:30.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.749+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa30400ec40 con 0x5629f1791d00 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.749+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa30400f9e0 con 0x5629f16fe9f0 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.749+0000 7fa338e0eb80 10 client.0 _ll_register_callbacks cb 0x5629f1659c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.749+0000 7fa323400700 1 -- 172.21.15.90:0/1831454097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 msgr2=0x5629f179ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.748+0000 7fa322a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f179ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59398/0 (socket says 172.21.15.90:59398) 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.748+0000 7fa323e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 0x5629f179f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.750+0000 7fa323400700 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f179ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.750+0000 7fa323400700 1 -- 172.21.15.90:0/1831454097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 msgr2=0x5629f179f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.750+0000 7fa323400700 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 0x5629f179f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.750+0000 7fa323400700 1 -- 172.21.15.90:0/1831454097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f17aabc0 con 0x5629f1612e40 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.750+0000 7fa323e00700 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 0x5629f179f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:30.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.750+0000 7fa322a00700 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f179ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:30.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.752+0000 7fa323400700 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1612e40 0x5629f179e910 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa314027fd0 tx=0x7fa31400f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc8c3d42c42316f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.752+0000 7fa30d600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:30.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.754+0000 7fa30d600700 1 -- 172.21.15.90:0/1831454097 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31403a020 con 0x5629f1612e40 2024-05-27T07:51:30.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.754+0000 7fa30d600700 1 -- 172.21.15.90:0/1831454097 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fa314010640 con 0x5629f1612e40 2024-05-27T07:51:30.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.755+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3100aea40 con 0x5629f1612e40 2024-05-27T07:51:30.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.756+0000 7fa30d600700 1 -- 172.21.15.90:0/1831454097 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa314010840 con 0x5629f1612e40 2024-05-27T07:51:30.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.756+0000 7fa338e0eb80 10 client.16536 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:30.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.756+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5629f181efb0 con 0x5629f1612e40 2024-05-27T07:51:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.757+0000 7fa30d600700 1 -- 172.21.15.90:0/1831454097 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa3100aea40 con 0x5629f1612e40 2024-05-27T07:51:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.757+0000 7fa30d600700 1 -- 172.21.15.90:0/1831454097 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fa314027910 con 0x5629f1612e40 2024-05-27T07:51:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.757+0000 7fa338e0eb80 10 client.16536 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.757+0000 7fa338e0eb80 10 client.16536 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:51:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.757+0000 7fa338e0eb80 20 client.16536 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.757+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5629f181f740 0x5629f1821b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.757+0000 7fa338e0eb80 4 client.16536 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:51:30.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.757+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629f16bb570 con 0x5629f181f740 2024-05-27T07:51:30.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.759+0000 7fa322a00700 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5629f181f740 0x5629f1821b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.763+0000 7fa322a00700 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5629f181f740 0x5629f1821b60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=43861c3425340ee4 server_cookie=f17d3c9464374a45 in_seq=0 out_seq=0 2024-05-27T07:51:30.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.763+0000 7fa30d600700 10 client.16536 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:51:30.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.763+0000 7fa30d600700 1 -- 172.21.15.90:0/1831454097 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629f16bb570 con 0x5629f181f740 2024-05-27T07:51:30.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.763+0000 7fa30d600700 10 client.16536 handle_command_reply: tid=0 2024-05-27T07:51:30.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.763+0000 7fa338e0eb80 1 client.16536 shutdown 2024-05-27T07:51:30.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.764+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1612e40 msgr2=0x5629f179e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.764+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1612e40 0x5629f179e910 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa314027fd0 tx=0x7fa31400f6d0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.764+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 shutdown_connections 2024-05-27T07:51:30.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.764+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5629f181f740 0x5629f1821b60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.764+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5629f1791d00 0x5629f179f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.764+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629f1612e40 0x5629f179e910 secure :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa314027fd0 tx=0x7fa31400f6d0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.764+0000 7fa30d600700 0 client.16536 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:51:30.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.764+0000 7fa338e0eb80 1 --2- 172.21.15.90:0/1831454097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5629f16fe9f0 0x5629f179ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.764+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 >> 172.21.15.90:0/1831454097 conn(0x5629f16ac390 msgr2=0x5629f1792f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:30.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.765+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 shutdown_connections 2024-05-27T07:51:30.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.766+0000 7fa338e0eb80 1 -- 172.21.15.90:0/1831454097 wait complete. 2024-05-27T07:51:30.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.766+0000 7fa338e0eb80 20 client.16536 trim_cache size 0 max 16384 2024-05-27T07:51:30.767 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:51:30.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.767+0000 7fa32a200700 1 -- 172.21.15.90:0/3239549923 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa3080b2af0 msgr2=0x7fa3080b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.767+0000 7fa32a200700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa3080b2af0 0x7fa3080b5000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa314005950 tx=0x7fa31400a300 comp rx=0 tx=0).stop 2024-05-27T07:51:30.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.767+0000 7fa32a200700 1 -- 172.21.15.90:0/3239549923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa32410b540 msgr2=0x7fa3241a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.767+0000 7fa32a200700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa32410b540 0x7fa3241a5cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa310005f50 tx=0x7fa3100057c0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.767+0000 7fa32a200700 1 -- 172.21.15.90:0/3239549923 shutdown_connections 2024-05-27T07:51:30.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.767+0000 7fa32a200700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa324111bb0 0x7fa3241a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.767+0000 7fa32a200700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32410abf0 0x7fa3241a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.767+0000 7fa32a200700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa3080b2af0 0x7fa3080b5000 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.767+0000 7fa32a200700 1 --2- 172.21.15.90:0/3239549923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa32410b540 0x7fa3241a5cf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.768+0000 7fa32a200700 1 -- 172.21.15.90:0/3239549923 >> 172.21.15.90:0/3239549923 conn(0x7fa32406f980 msgr2=0x7fa324110560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:30.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.768+0000 7fa32a200700 1 -- 172.21.15.90:0/3239549923 shutdown_connections 2024-05-27T07:51:30.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:30.768+0000 7fa32a200700 1 -- 172.21.15.90:0/3239549923 wait complete. 2024-05-27T07:51:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.850+0000 7f08e0a00640 1 -- 172.21.15.90:0/39097055 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0307c0 con 0x7f08e41065f0 2024-05-27T07:51:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.851+0000 7f08e8e00640 1 -- 172.21.15.90:0/39097055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08e41065f0 msgr2=0x7f08e41129c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.851+0000 7f08e8e00640 1 --2- 172.21.15.90:0/39097055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08e41065f0 0x7f08e41129c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009ca0 tx=0x7f08dc02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.852+0000 7f08e8e00640 1 -- 172.21.15.90:0/39097055 shutdown_connections 2024-05-27T07:51:30.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.852+0000 7f08e8e00640 1 --2- 172.21.15.90:0/39097055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08e410c340 0x7f08e4105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.852+0000 7f08e8e00640 1 --2- 172.21.15.90:0/39097055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08e41065f0 0x7f08e41129c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.852+0000 7f08e8e00640 1 --2- 172.21.15.90:0/39097055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08e410bea0 0x7f08e4105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.852+0000 7f08e8e00640 1 -- 172.21.15.90:0/39097055 >> 172.21.15.90:0/39097055 conn(0x7f08e4101520 msgr2=0x7f08e4103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.854+0000 7f08e8e00640 1 -- 172.21.15.90:0/39097055 shutdown_connections 2024-05-27T07:51:30.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.854+0000 7f08e8e00640 1 -- 172.21.15.90:0/39097055 wait complete. 2024-05-27T07:51:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.859+0000 7f08e8e00640 1 Processor -- start 2024-05-27T07:51:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.859+0000 7f08e8e00640 1 -- start start 2024-05-27T07:51:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.860+0000 7f08e8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08e41065f0 0x7f08e410feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.860+0000 7f08e8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08e410bea0 0x7f08e41103f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.860+0000 7f08e8e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08e410c340 0x7f08e4110930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.860+0000 7f08e8e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08e406c9d0 con 0x7f08e41065f0 2024-05-27T07:51:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.860+0000 7f08e8e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08e406c850 con 0x7f08e410c340 2024-05-27T07:51:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.860+0000 7f08e8e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f08e406cb50 con 0x7f08e410bea0 2024-05-27T07:51:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.861+0000 7f08e1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08e41065f0 0x7f08e410feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.861+0000 7f08e1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08e41065f0 0x7f08e410feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55610/0 (socket says 172.21.15.90:55610) 2024-05-27T07:51:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.861+0000 7f08e1e00640 1 -- 172.21.15.90:0/85263767 learned_addr learned my addr 172.21.15.90:0/85263767 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.861+0000 7f08e1400640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08e410bea0 0x7f08e41103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.861+0000 7f08e2800640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08e410c340 0x7f08e4110930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.862+0000 7f08e1e00640 1 -- 172.21.15.90:0/85263767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08e410bea0 msgr2=0x7f08e41103f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.862+0000 7f08e1e00640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08e410bea0 0x7f08e41103f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.862+0000 7f08e1e00640 1 -- 172.21.15.90:0/85263767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08e410c340 msgr2=0x7f08e4110930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.862+0000 7f08e1e00640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08e410c340 0x7f08e4110930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.862+0000 7f08e1e00640 1 -- 172.21.15.90:0/85263767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e40743a0 con 0x7f08e41065f0 2024-05-27T07:51:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.862+0000 7f08e1e00640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08e41065f0 0x7f08e410feb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f08d400e9c0 tx=0x7f08d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.863+0000 7f08daa00640 1 -- 172.21.15.90:0/85263767 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d400cd80 con 0x7f08e41065f0 2024-05-27T07:51:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.863+0000 7f08e8e00640 1 -- 172.21.15.90:0/85263767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e4074690 con 0x7f08e41065f0 2024-05-27T07:51:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.863+0000 7f08e8e00640 1 -- 172.21.15.90:0/85263767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e4074c30 con 0x7f08e41065f0 2024-05-27T07:51:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.863+0000 7f08daa00640 1 -- 172.21.15.90:0/85263767 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08d40054b0 con 0x7f08e41065f0 2024-05-27T07:51:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.863+0000 7f08daa00640 1 -- 172.21.15.90:0/85263767 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4004da0 con 0x7f08e41065f0 2024-05-27T07:51:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.868+0000 7f08daa00640 1 -- 172.21.15.90:0/85263767 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f08d4005020 con 0x7f08e41065f0 2024-05-27T07:51:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.869+0000 7f08efdd8740 1 Processor -- start 2024-05-27T07:51:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.871+0000 7f08efdd8740 1 -- start start 2024-05-27T07:51:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.872+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be70f1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.872+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be70f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.872+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d2e4d0 0x561be70f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.872+0000 7f08efdd8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561be7062b90 con 0x561be6d262f0 2024-05-27T07:51:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.872+0000 7f08efdd8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561be7052730 con 0x561be6d2e4d0 2024-05-27T07:51:30.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.872+0000 7f08efdd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561be7056770 con 0x561be6d4f1b0 2024-05-27T07:51:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.873+0000 7f08e1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be70f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.873+0000 7f08e1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be70f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55612/0 (socket says 172.21.15.90:55612) 2024-05-27T07:51:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.873+0000 7f08e1e00640 1 -- 172.21.15.90:0/4055912141 learned_addr learned my addr 172.21.15.90:0/4055912141 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.873+0000 7f08e2800640 1 --2- 172.21.15.90:0/4055912141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be70f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.874+0000 7f08e1400640 1 --2- 172.21.15.90:0/4055912141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d2e4d0 0x561be70f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.876+0000 7f08e1e00640 1 -- 172.21.15.90:0/4055912141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 msgr2=0x561be70f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.876+0000 7f08e1e00640 1 --2- 172.21.15.90:0/4055912141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be70f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.876+0000 7f08e1e00640 1 -- 172.21.15.90:0/4055912141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d2e4d0 msgr2=0x561be70f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.876+0000 7f08e1e00640 1 --2- 172.21.15.90:0/4055912141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d2e4d0 0x561be70f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.876+0000 7f08e1e00640 1 -- 172.21.15.90:0/4055912141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be6feb8b0 con 0x561be6d262f0 2024-05-27T07:51:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.876+0000 7f08daa00640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f08c80b29d0 0x7f08c80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.876+0000 7f08daa00640 1 -- 172.21.15.90:0/85263767 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f08d40e2d70 con 0x7f08e41065f0 2024-05-27T07:51:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.878+0000 7f08e2800640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f08c80b29d0 0x7f08c80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.878+0000 7f08e1e00640 1 --2- 172.21.15.90:0/4055912141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be70f1da0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f08d40e0090 tx=0x7f08d40e6dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4f0cb9205e12e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.879+0000 7f08e2800640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f08c80b29d0 0x7f08c80b4ee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x561be70ee9d0 tx=0x7f08dc005bd0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.880+0000 7f08cf400640 1 -- 172.21.15.90:0/4055912141 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d40a5ba0 con 0x561be6d262f0 2024-05-27T07:51:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.880+0000 7f08cf400640 1 -- 172.21.15.90:0/4055912141 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08d40a97f0 con 0x561be6d262f0 2024-05-27T07:51:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.880+0000 7f08cf400640 1 -- 172.21.15.90:0/4055912141 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4005310 con 0x561be6d262f0 2024-05-27T07:51:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.882+0000 7f08efdd8740 1 -- 172.21.15.90:0/4055912141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 msgr2=0x561be70f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.882+0000 7f08efdd8740 1 --2- 172.21.15.90:0/4055912141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be70f1da0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f08d40e0090 tx=0x7f08d40e6dd0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.882+0000 7f08efdd8740 1 -- 172.21.15.90:0/4055912141 shutdown_connections 2024-05-27T07:51:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.882+0000 7f08efdd8740 1 --2- 172.21.15.90:0/4055912141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d2e4d0 0x561be70f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.882+0000 7f08efdd8740 1 --2- 172.21.15.90:0/4055912141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be70f1da0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.882+0000 7f08efdd8740 1 --2- 172.21.15.90:0/4055912141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be70f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.882+0000 7f08efdd8740 1 -- 172.21.15.90:0/4055912141 >> 172.21.15.90:0/4055912141 conn(0x561be6d43300 msgr2=0x561be70ee010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.883+0000 7f08efdd8740 1 -- 172.21.15.90:0/4055912141 shutdown_connections 2024-05-27T07:51:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.883+0000 7f08efdd8740 1 -- 172.21.15.90:0/4055912141 wait complete. 2024-05-27T07:51:30.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.884+0000 7f08efdd8740 1 -- start start 2024-05-27T07:51:30.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.884+0000 7f08efdd8740 1 Processor -- start 2024-05-27T07:51:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.885+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 0x561be70fba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.885+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d2e4d0 0x561be70fbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.886+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d4f1b0 0x561be7107d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.886+0000 7f08efdd8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561be6d38790 con 0x561be6d4f1b0 2024-05-27T07:51:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.886+0000 7f08efdd8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561be6ffabc0 con 0x561be6d262f0 2024-05-27T07:51:30.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.886+0000 7f08efdd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561be70fe990 con 0x561be6d2e4d0 2024-05-27T07:51:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.886+0000 7f08e1e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 0x561be70fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.886+0000 7f08efdd8740 10 client.0 _ll_register_callbacks cb 0x561be6fd8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.887+0000 7f08e1e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 0x561be70fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:58594/0 (socket says 172.21.15.90:58594) 2024-05-27T07:51:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.887+0000 7f08e1e00640 1 -- 172.21.15.90:0/107824004 learned_addr learned my addr 172.21.15.90:0/107824004 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.888+0000 7f08e1400640 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d2e4d0 0x561be70fbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.888+0000 7f08e1e00640 1 -- 172.21.15.90:0/107824004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d2e4d0 msgr2=0x561be70fbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.888+0000 7f08e1e00640 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d2e4d0 0x561be70fbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.888+0000 7f08e1e00640 1 -- 172.21.15.90:0/107824004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d4f1b0 msgr2=0x561be7107d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.888+0000 7f08e1e00640 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d4f1b0 0x561be7107d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.888+0000 7f08e1e00640 1 -- 172.21.15.90:0/107824004 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be6d67210 con 0x561be6d262f0 2024-05-27T07:51:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.888+0000 7f08e1400640 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d2e4d0 0x561be70fbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.889+0000 7f08e1e00640 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 0x561be70fba10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08d4000c00 tx=0x7f08d40e92d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d5dde3925da194f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.889+0000 7f08c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:51:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.890+0000 7f08c3e00640 1 -- 172.21.15.90:0/107824004 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d40a9450 con 0x561be6d262f0 2024-05-27T07:51:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.890+0000 7f08c3e00640 1 -- 172.21.15.90:0/107824004 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f08d40a62d0 con 0x561be6d262f0 2024-05-27T07:51:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.890+0000 7f08efdd8740 1 -- 172.21.15.90:0/107824004 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d40a97f0 con 0x561be6d262f0 2024-05-27T07:51:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.891+0000 7f08c3e00640 1 -- 172.21.15.90:0/107824004 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f08d40a9630 con 0x561be6d262f0 2024-05-27T07:51:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.891+0000 7f08efdd8740 10 client.36032 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.891+0000 7f08efdd8740 1 -- 172.21.15.90:0/107824004 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561be6e33620 con 0x561be6d262f0 2024-05-27T07:51:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.892+0000 7f08c3e00640 1 -- 172.21.15.90:0/107824004 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08d40a97f0 con 0x561be6d262f0 2024-05-27T07:51:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.893+0000 7f08c3e00640 1 -- 172.21.15.90:0/107824004 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f08d40e03d0 con 0x561be6d262f0 2024-05-27T07:51:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.893+0000 7f08efdd8740 10 client.36032 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.893+0000 7f08efdd8740 10 client.36032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.893+0000 7f08efdd8740 20 client.36032 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.893+0000 7f08efdd8740 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561be6d0dd80 0x561be710db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.894+0000 7f08efdd8740 4 client.36032 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.894+0000 7f08efdd8740 1 -- 172.21.15.90:0/107824004 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561be7064ae0 con 0x561be6d0dd80 2024-05-27T07:51:30.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.894+0000 7f08e1400640 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561be6d0dd80 0x561be710db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.895+0000 7f08e1400640 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561be6d0dd80 0x561be710db40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9397b5294bc7df server_cookie=8d890e960c175615 in_seq=0 out_seq=0 2024-05-27T07:51:30.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.895+0000 7f08c3e00640 10 client.36032 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.897+0000 7f08c3e00640 1 -- 172.21.15.90:0/107824004 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561be7064ae0 con 0x561be6d0dd80 2024-05-27T07:51:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.897+0000 7f08c3e00640 10 client.36032 handle_command_reply: tid=0 2024-05-27T07:51:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.897+0000 7f08efdd8740 1 client.36032 shutdown 2024-05-27T07:51:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.898+0000 7f08efdd8740 1 -- 172.21.15.90:0/107824004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 msgr2=0x561be70fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.898+0000 7f08efdd8740 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 0x561be70fba10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08d4000c00 tx=0x7f08d40e92d0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.898+0000 7f08efdd8740 1 -- 172.21.15.90:0/107824004 shutdown_connections 2024-05-27T07:51:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.898+0000 7f08efdd8740 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561be6d0dd80 0x561be710db40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.898+0000 7f08efdd8740 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d4f1b0 0x561be7107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.898+0000 7f08c3e00640 0 client.36032 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.898+0000 7f08efdd8740 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d2e4d0 0x561be70fbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.898+0000 7f08efdd8740 1 --2- 172.21.15.90:0/107824004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 0x561be70fba10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.898+0000 7f08efdd8740 1 -- 172.21.15.90:0/107824004 >> 172.21.15.90:0/107824004 conn(0x561be6d43300 msgr2=0x561be70f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.899+0000 7f08efdd8740 1 -- 172.21.15.90:0/107824004 shutdown_connections 2024-05-27T07:51:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.900+0000 7f08efdd8740 1 -- 172.21.15.90:0/107824004 wait complete. 2024-05-27T07:51:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.900+0000 7f08efdd8740 20 client.36032 trim_cache size 0 max 16384 2024-05-27T07:51:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.949+0000 7f08efdd8740 1 Processor -- start 2024-05-27T07:51:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.949+0000 7f08efdd8740 1 -- start start 2024-05-27T07:51:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.949+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 0x561be70fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.949+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d2e4d0 0x561be70f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.950+0000 7f08e1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d2e4d0 0x561be70f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.950+0000 7f08e1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d2e4d0 0x561be70f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55634/0 (socket says 172.21.15.90:55634) 2024-05-27T07:51:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.950+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be7107cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.950+0000 7f08e1400640 1 -- 172.21.15.90:0/3135131498 learned_addr learned my addr 172.21.15.90:0/3135131498 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.950+0000 7f08efdd8740 1 -- 172.21.15.90:0/3135131498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4011a20 con 0x561be6d2e4d0 2024-05-27T07:51:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.950+0000 7f08efdd8740 1 -- 172.21.15.90:0/3135131498 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400f810 con 0x561be6d262f0 2024-05-27T07:51:30.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.950+0000 7f08efdd8740 1 -- 172.21.15.90:0/3135131498 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4011350 con 0x561be6d4f1b0 2024-05-27T07:51:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.951+0000 7f08e1e00640 1 --2- 172.21.15.90:0/3135131498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 0x561be70fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.951+0000 7f08e2800640 1 --2- 172.21.15.90:0/3135131498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be7107cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.951+0000 7f08e1400640 1 -- 172.21.15.90:0/3135131498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 msgr2=0x561be7107cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.951+0000 7f08e1400640 1 --2- 172.21.15.90:0/3135131498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be7107cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.951+0000 7f08e1400640 1 -- 172.21.15.90:0/3135131498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 msgr2=0x561be70fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.951+0000 7f08e1400640 1 --2- 172.21.15.90:0/3135131498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 0x561be70fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.951+0000 7f08e1400640 1 -- 172.21.15.90:0/3135131498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561be70f2020 con 0x561be6d2e4d0 2024-05-27T07:51:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.952+0000 7f08e1400640 1 --2- 172.21.15.90:0/3135131498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d2e4d0 0x561be70f5b30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f08e406c530 tx=0x7f08d0003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed810a5eaf9f2092 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.953+0000 7f08cea00640 1 -- 172.21.15.90:0/3135131498 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d000ccd0 con 0x561be6d2e4d0 2024-05-27T07:51:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.953+0000 7f08cea00640 1 -- 172.21.15.90:0/3135131498 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08d000ce70 con 0x561be6d2e4d0 2024-05-27T07:51:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.953+0000 7f08cea00640 1 -- 172.21.15.90:0/3135131498 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d001a330 con 0x561be6d2e4d0 2024-05-27T07:51:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.954+0000 7f08efdd8740 1 -- 172.21.15.90:0/3135131498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d2e4d0 msgr2=0x561be70f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.954+0000 7f08efdd8740 1 --2- 172.21.15.90:0/3135131498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d2e4d0 0x561be70f5b30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f08e406c530 tx=0x7f08d0003eb0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.955+0000 7f08efdd8740 1 -- 172.21.15.90:0/3135131498 shutdown_connections 2024-05-27T07:51:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.955+0000 7f08efdd8740 1 --2- 172.21.15.90:0/3135131498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be7107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.955+0000 7f08efdd8740 1 --2- 172.21.15.90:0/3135131498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d2e4d0 0x561be70f5b30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.955+0000 7f08efdd8740 1 --2- 172.21.15.90:0/3135131498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d262f0 0x561be70fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.955+0000 7f08efdd8740 1 -- 172.21.15.90:0/3135131498 >> 172.21.15.90:0/3135131498 conn(0x561be6d43300 msgr2=0x561be70f5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.955+0000 7f08efdd8740 1 -- 172.21.15.90:0/3135131498 shutdown_connections 2024-05-27T07:51:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.955+0000 7f08efdd8740 1 -- 172.21.15.90:0/3135131498 wait complete. 2024-05-27T07:51:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.956+0000 7f08efdd8740 1 -- start start 2024-05-27T07:51:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.956+0000 7f08efdd8740 1 Processor -- start 2024-05-27T07:51:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.957+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be710fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.957+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d2e4d0 0x561be710ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.957+0000 7f08efdd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be71104e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.957+0000 7f08efdd8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08c40105b0 con 0x561be6d262f0 2024-05-27T07:51:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.957+0000 7f08efdd8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08c400fee0 con 0x561be6d2e4d0 2024-05-27T07:51:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.957+0000 7f08efdd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f08c4010c80 con 0x561be6d4f1b0 2024-05-27T07:51:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.957+0000 7f08efdd8740 10 client.0 _ll_register_callbacks cb 0x7f08d000c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.958+0000 7f08e2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be71104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.958+0000 7f08e1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be710fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.958+0000 7f08e1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d2e4d0 0x561be710ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.958+0000 7f08e1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be710fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55648/0 (socket says 172.21.15.90:55648) 2024-05-27T07:51:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.958+0000 7f08e2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be71104e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59444/0 (socket says 172.21.15.90:59444) 2024-05-27T07:51:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.958+0000 7f08e1e00640 1 -- 172.21.15.90:0/721570274 learned_addr learned my addr 172.21.15.90:0/721570274 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.959+0000 7f08e1e00640 1 -- 172.21.15.90:0/721570274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 msgr2=0x561be71104e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.959+0000 7f08e1e00640 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be71104e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.959+0000 7f08e1e00640 1 -- 172.21.15.90:0/721570274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d2e4d0 msgr2=0x561be710ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.959+0000 7f08e1e00640 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d2e4d0 0x561be710ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.959+0000 7f08e1e00640 1 -- 172.21.15.90:0/721570274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561be7174090 con 0x561be6d262f0 2024-05-27T07:51:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.959+0000 7f08e2800640 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be71104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.960+0000 7f08e1e00640 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be710fa60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f08d40a6f50 tx=0x7f08d40b29e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01053b4bf905176 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.960+0000 7f08c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.962+0000 7f08c3e00640 1 -- 172.21.15.90:0/721570274 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d40a9ab0 con 0x561be6d262f0 2024-05-27T07:51:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.962+0000 7f08c3e00640 1 -- 172.21.15.90:0/721570274 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f08d40a62d0 con 0x561be6d262f0 2024-05-27T07:51:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.962+0000 7f08efdd8740 1 -- 172.21.15.90:0/721570274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d000ce70 con 0x561be6d262f0 2024-05-27T07:51:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.963+0000 7f08c3e00640 1 -- 172.21.15.90:0/721570274 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f08d40a9450 con 0x561be6d262f0 2024-05-27T07:51:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.963+0000 7f08efdd8740 10 client.16551 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.963+0000 7f08efdd8740 1 -- 172.21.15.90:0/721570274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561be7174090 con 0x561be6d262f0 2024-05-27T07:51:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.964+0000 7f08c3e00640 1 -- 172.21.15.90:0/721570274 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08d000ce70 con 0x561be6d262f0 2024-05-27T07:51:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.964+0000 7f08c3e00640 1 -- 172.21.15.90:0/721570274 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f08d40a5650 con 0x561be6d262f0 2024-05-27T07:51:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.964+0000 7f08efdd8740 10 client.16551 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.964+0000 7f08efdd8740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.964+0000 7f08efdd8740 20 client.16551 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:30.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.964+0000 7f08efdd8740 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561be6ce31c0 0x561be7177580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.964+0000 7f08efdd8740 4 client.16551 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:51:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.964+0000 7f08efdd8740 1 -- 172.21.15.90:0/721570274 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561be705e2c0 con 0x561be6ce31c0 2024-05-27T07:51:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.965+0000 7f08e1400640 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561be6ce31c0 0x561be7177580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.968+0000 7f08c3e00640 10 client.16551 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:30.968+0000 7f08e1400640 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561be6ce31c0 0x561be7177580 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92023860d780505a server_cookie=f9a8e61d18972d9b in_seq=0 out_seq=0 2024-05-27T07:51:31.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.016+0000 7f08c3e00640 1 -- 172.21.15.90:0/721570274 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561be705e2c0 con 0x561be6ce31c0 2024-05-27T07:51:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.016+0000 7f08c3e00640 10 client.16551 handle_command_reply: tid=0 2024-05-27T07:51:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.016+0000 7f08efdd8740 1 client.16551 shutdown 2024-05-27T07:51:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.016+0000 7f08efdd8740 1 -- 172.21.15.90:0/721570274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 msgr2=0x561be710fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.016+0000 7f08efdd8740 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be710fa60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f08d40a6f50 tx=0x7f08d40b29e0 comp rx=0 tx=0).stop 2024-05-27T07:51:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.017+0000 7f08efdd8740 1 -- 172.21.15.90:0/721570274 shutdown_connections 2024-05-27T07:51:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.017+0000 7f08efdd8740 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561be6ce31c0 0x561be7177580 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.017+0000 7f08efdd8740 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561be6d2e4d0 0x561be710ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.017+0000 7f08efdd8740 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561be6d4f1b0 0x561be71104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:31.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.017+0000 7f08efdd8740 1 --2- 172.21.15.90:0/721570274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561be6d262f0 0x561be710fa60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.017+0000 7f08efdd8740 1 -- 172.21.15.90:0/721570274 >> 172.21.15.90:0/721570274 conn(0x561be6d43300 msgr2=0x561be70fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.018+0000 7f08c3e00640 0 client.16551 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.019+0000 7f08efdd8740 1 -- 172.21.15.90:0/721570274 shutdown_connections 2024-05-27T07:51:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.019+0000 7f08efdd8740 1 -- 172.21.15.90:0/721570274 wait complete. 2024-05-27T07:51:31.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.020+0000 7f08efdd8740 20 client.16551 trim_cache size 0 max 16384 2024-05-27T07:51:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:51:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:51:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:51:31.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.022+0000 7f08e8e00640 1 -- 172.21.15.90:0/85263767 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f08c80b29d0 msgr2=0x7f08c80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.022+0000 7f08e8e00640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f08c80b29d0 0x7f08c80b4ee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x561be70ee9d0 tx=0x7f08dc005bd0 comp rx=0 tx=0).stop 2024-05-27T07:51:31.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.022+0000 7f08e8e00640 1 -- 172.21.15.90:0/85263767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08e41065f0 msgr2=0x7f08e410feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.022+0000 7f08e8e00640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08e41065f0 0x7f08e410feb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f08d400e9c0 tx=0x7f08d400ee80 comp rx=0 tx=0).stop 2024-05-27T07:51:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.023+0000 7f08e8e00640 1 -- 172.21.15.90:0/85263767 shutdown_connections 2024-05-27T07:51:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.023+0000 7f08e8e00640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f08c80b29d0 0x7f08c80b4ee0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:31.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.023+0000 7f08e8e00640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08e410c340 0x7f08e4110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.023+0000 7f08e8e00640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08e410bea0 0x7f08e41103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.023+0000 7f08e8e00640 1 --2- 172.21.15.90:0/85263767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08e41065f0 0x7f08e410feb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:31.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.023+0000 7f08e8e00640 1 -- 172.21.15.90:0/85263767 >> 172.21.15.90:0/85263767 conn(0x7f08e4101520 msgr2=0x7f08e4101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.027+0000 7f08e8e00640 1 -- 172.21.15.90:0/85263767 shutdown_connections 2024-05-27T07:51:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:31.027+0000 7f08e8e00640 1 -- 172.21.15.90:0/85263767 wait complete. 2024-05-27T07:51:31.680 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:51:32.387 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:32.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:32 smithi090 ceph-mon[31429]: pgmap v161: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 559 op/s 2024-05-27T07:51:32.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:32 smithi136 ceph-mon[35324]: pgmap v161: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 559 op/s 2024-05-27T07:51:32.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:32 smithi129 ceph-mon[35254]: pgmap v161: 129 pgs: 129 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.7 MiB/s wr, 559 op/s 2024-05-27T07:51:33.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 75.0 s, 609.8 tps, lat 164.669 ms stddev 44.132 2024-05-27T07:51:34.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:34 smithi090 ceph-mon[31429]: pgmap v162: 129 pgs: 129 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.7 MiB/s wr, 797 op/s 2024-05-27T07:51:34.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:34 smithi136 ceph-mon[35324]: pgmap v162: 129 pgs: 129 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.7 MiB/s wr, 797 op/s 2024-05-27T07:51:34.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:34 smithi129 ceph-mon[35254]: pgmap v162: 129 pgs: 129 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.7 MiB/s wr, 797 op/s 2024-05-27T07:51:36.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:36 smithi090 ceph-mon[31429]: pgmap v163: 129 pgs: 129 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.1 MiB/s wr, 608 op/s 2024-05-27T07:51:36.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.201+0000 7f112d400700 1 -- 172.21.15.90:0/4136085714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11280764a0 msgr2=0x7f1128114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.201+0000 7f112d400700 1 --2- 172.21.15.90:0/4136085714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11280764a0 0x7f1128114e20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1110009990 tx=0x7f1110009ca0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.202+0000 7f112d400700 1 -- 172.21.15.90:0/4136085714 shutdown_connections 2024-05-27T07:51:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.202+0000 7f112d400700 1 --2- 172.21.15.90:0/4136085714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128115360 0x7f1128117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.202+0000 7f112d400700 1 --2- 172.21.15.90:0/4136085714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1128075b50 0x7f1128075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.202+0000 7f112d400700 1 --2- 172.21.15.90:0/4136085714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11280764a0 0x7f1128114e20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.202+0000 7f112d400700 1 -- 172.21.15.90:0/4136085714 >> 172.21.15.90:0/4136085714 conn(0x7f1128100030 msgr2=0x7f1128102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.203+0000 7f112d400700 1 -- 172.21.15.90:0/4136085714 shutdown_connections 2024-05-27T07:51:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.203+0000 7f112d400700 1 -- 172.21.15.90:0/4136085714 wait complete. 2024-05-27T07:51:36.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.207+0000 7f112d400700 1 Processor -- start 2024-05-27T07:51:36.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.207+0000 7f112d400700 1 -- start start 2024-05-27T07:51:36.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.208+0000 7f112d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1128075b50 0x7f11281a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.208+0000 7f112d400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11280764a0 0x7f11281a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.208+0000 7f112d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128115360 0x7f11281a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.209+0000 7f1126800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1128075b50 0x7f11281a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.209+0000 7f1126800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1128075b50 0x7f11281a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55666/0 (socket says 172.21.15.90:55666) 2024-05-27T07:51:36.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.209+0000 7f1125e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11280764a0 0x7f11281a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.209+0000 7f1125e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11280764a0 0x7f11281a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:58630/0 (socket says 172.21.15.90:58630) 2024-05-27T07:51:36.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.209+0000 7f1126800700 1 -- 172.21.15.90:0/3341937436 learned_addr learned my addr 172.21.15.90:0/3341937436 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:36.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.209+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f112811a920 con 0x7f1128075b50 2024-05-27T07:51:36.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.209+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f112811a7a0 con 0x7f11280764a0 2024-05-27T07:51:36.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.209+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f112811aaa0 con 0x7f1128115360 2024-05-27T07:51:36.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.210+0000 7f1127200700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128115360 0x7f11281a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.210+0000 7f1126800700 1 -- 172.21.15.90:0/3341937436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128115360 msgr2=0x7f11281a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:36.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.211+0000 7f1126800700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128115360 0x7f11281a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.211+0000 7f1126800700 1 -- 172.21.15.90:0/3341937436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11280764a0 msgr2=0x7f11281a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:36.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.211+0000 7f1126800700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11280764a0 0x7f11281a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.211+0000 7f1126800700 1 -- 172.21.15.90:0/3341937436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11281a6050 con 0x7f1128075b50 2024-05-27T07:51:36.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.212+0000 7f1126800700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1128075b50 0x7f11281a1230 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1118007e20 tx=0x7f111800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:36.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.212+0000 7f111f400700 1 -- 172.21.15.90:0/3341937436 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118016020 con 0x7f1128075b50 2024-05-27T07:51:36.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.213+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11281a6340 con 0x7f1128075b50 2024-05-27T07:51:36.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.213+0000 7f111f400700 1 -- 172.21.15.90:0/3341937436 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f111800ac90 con 0x7f1128075b50 2024-05-27T07:51:36.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.214+0000 7f111f400700 1 -- 172.21.15.90:0/3341937436 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180115e0 con 0x7f1128075b50 2024-05-27T07:51:36.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.214+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11281add10 con 0x7f1128075b50 2024-05-27T07:51:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.222+0000 7f111f400700 1 -- 172.21.15.90:0/3341937436 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1118019020 con 0x7f1128075b50 2024-05-27T07:51:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.224+0000 7f111f400700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f110c0b2af0 0x7f110c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.225+0000 7f111f400700 1 -- 172.21.15.90:0/3341937436 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f11180e2240 con 0x7f1128075b50 2024-05-27T07:51:36.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.227+0000 7f1125e00700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f110c0b2af0 0x7f110c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.229+0000 7f1125e00700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f110c0b2af0 0x7f110c0b5000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1110009750 tx=0x7f111001a040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:36.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.229+0000 7f113bff1b80 1 Processor -- start 2024-05-27T07:51:36.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.229+0000 7f113bff1b80 1 -- start start 2024-05-27T07:51:36.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.230+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c88a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.231+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 0x557c0c884870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.231+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c884db0 0x557c0c8851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.231+0000 7f113bff1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c0c7b2e40 con 0x557c0c705e40 2024-05-27T07:51:36.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.231+0000 7f113bff1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c0c7a29e0 con 0x557c0c884db0 2024-05-27T07:51:36.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.231+0000 7f113bff1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557c0c7a6a20 con 0x557c0c88ad30 2024-05-27T07:51:36.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.231+0000 7f1125e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 0x557c0c884870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.231+0000 7f1125e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 0x557c0c884870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59482/0 (socket says 172.21.15.90:59482) 2024-05-27T07:51:36.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.231+0000 7f1125e00700 1 -- 172.21.15.90:0/450921564 learned_addr learned my addr 172.21.15.90:0/450921564 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:36.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.232+0000 7f1127200700 1 --2- 172.21.15.90:0/450921564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c884db0 0x557c0c8851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.233+0000 7f1125e00700 1 -- 172.21.15.90:0/450921564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c884db0 msgr2=0x557c0c8851c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:36.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.233+0000 7f1125e00700 1 --2- 172.21.15.90:0/450921564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c884db0 0x557c0c8851c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.233+0000 7f1125e00700 1 -- 172.21.15.90:0/450921564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 msgr2=0x557c0c88a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:36.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.233+0000 7f1125e00700 1 --2- 172.21.15.90:0/450921564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c88a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.233+0000 7f1125e00700 1 -- 172.21.15.90:0/450921564 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0c76b420 con 0x557c0c88ad30 2024-05-27T07:51:36.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.233+0000 7f1127200700 1 --2- 172.21.15.90:0/450921564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c884db0 0x557c0c8851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:36.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.234+0000 7f1125e00700 1 --2- 172.21.15.90:0/450921564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 0x557c0c884870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1110015000 tx=0x7f111001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=187eebbf07533cfd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:36.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.235+0000 7f1117e00700 1 -- 172.21.15.90:0/450921564 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110003bd0 con 0x557c0c88ad30 2024-05-27T07:51:36.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.235+0000 7f1117e00700 1 -- 172.21.15.90:0/450921564 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1110005210 con 0x557c0c88ad30 2024-05-27T07:51:36.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.235+0000 7f1117e00700 1 -- 172.21.15.90:0/450921564 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111000f760 con 0x557c0c88ad30 2024-05-27T07:51:36.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.237+0000 7f113bff1b80 1 -- 172.21.15.90:0/450921564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 msgr2=0x557c0c884870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:36.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.237+0000 7f113bff1b80 1 --2- 172.21.15.90:0/450921564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 0x557c0c884870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1110015000 tx=0x7f111001f000 comp rx=0 tx=0).stop 2024-05-27T07:51:36.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.237+0000 7f113bff1b80 1 -- 172.21.15.90:0/450921564 shutdown_connections 2024-05-27T07:51:36.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.237+0000 7f113bff1b80 1 --2- 172.21.15.90:0/450921564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c884db0 0x557c0c8851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.237+0000 7f113bff1b80 1 --2- 172.21.15.90:0/450921564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c88a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.237+0000 7f113bff1b80 1 --2- 172.21.15.90:0/450921564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 0x557c0c884870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.237+0000 7f113bff1b80 1 -- 172.21.15.90:0/450921564 >> 172.21.15.90:0/450921564 conn(0x557c0c765fc0 msgr2=0x557c0c87f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:36.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.237+0000 7f113bff1b80 1 -- 172.21.15.90:0/450921564 shutdown_connections 2024-05-27T07:51:36.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.237+0000 7f113bff1b80 1 -- 172.21.15.90:0/450921564 wait complete. 2024-05-27T07:51:36.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.238+0000 7f113bff1b80 1 -- start start 2024-05-27T07:51:36.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.239+0000 7f113bff1b80 1 Processor -- start 2024-05-27T07:51:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.240+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c88e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.240+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.240+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 0x557c0c88f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.240+0000 7f113bff1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c0c6a8fa0 con 0x557c0c705e40 2024-05-27T07:51:36.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.240+0000 7f113bff1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c0c743760 con 0x557c0c88ad30 2024-05-27T07:51:36.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.240+0000 7f113bff1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557c0c891740 con 0x557c0c884db0 2024-05-27T07:51:36.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.240+0000 7f113bff1b80 10 client.0 _ll_register_callbacks cb 0x557c0c74cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:36.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.241+0000 7f1125e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.241+0000 7f1125e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59490/0 (socket says 172.21.15.90:59490) 2024-05-27T07:51:36.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.241+0000 7f1125e00700 1 -- 172.21.15.90:0/1922648497 learned_addr learned my addr 172.21.15.90:0/1922648497 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:36.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.241+0000 7f1127200700 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 0x557c0c88f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.241+0000 7f1126800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c88e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.243+0000 7f1126800700 1 -- 172.21.15.90:0/1922648497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 msgr2=0x557c0c88eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.243+0000 7f1126800700 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.243+0000 7f1126800700 1 -- 172.21.15.90:0/1922648497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 msgr2=0x557c0c88f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.243+0000 7f1126800700 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 0x557c0c88f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.243+0000 7f1126800700 1 -- 172.21.15.90:0/1922648497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0c6f28d0 con 0x557c0c705e40 2024-05-27T07:51:36.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.243+0000 7f1127200700 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 0x557c0c88f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:36.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.244+0000 7f1126800700 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c88e650 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f11180df6f0 tx=0x7f11180dfb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a9b1ad76fb8cca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:36.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.244+0000 7f1114c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:36.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.245+0000 7f1114c00700 1 -- 172.21.15.90:0/1922648497 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180a8d90 con 0x557c0c705e40 2024-05-27T07:51:36.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.245+0000 7f1114c00700 1 -- 172.21.15.90:0/1922648497 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f11180b1e60 con 0x557c0c705e40 2024-05-27T07:51:36.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.245+0000 7f113bff1b80 1 -- 172.21.15.90:0/1922648497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1110005210 con 0x557c0c705e40 2024-05-27T07:51:36.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.245+0000 7f1114c00700 1 -- 172.21.15.90:0/1922648497 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f11180a9980 con 0x557c0c705e40 2024-05-27T07:51:36.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.245+0000 7f113bff1b80 10 client.16557 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:36.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.245+0000 7f113bff1b80 1 -- 172.21.15.90:0/1922648497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557c0c773060 con 0x557c0c705e40 2024-05-27T07:51:36.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.246+0000 7f1114c00700 1 -- 172.21.15.90:0/1922648497 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1110005210 con 0x557c0c705e40 2024-05-27T07:51:36.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.246+0000 7f1114c00700 1 -- 172.21.15.90:0/1922648497 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f11180ad150 con 0x557c0c705e40 2024-05-27T07:51:36.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.247+0000 7f113bff1b80 10 client.16557 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:36.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.247+0000 7f113bff1b80 10 client.16557 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:51:36.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.247+0000 7f113bff1b80 20 client.16557 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:36.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.247+0000 7f113bff1b80 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x557c0c89e690 0x557c0c8a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.247+0000 7f113bff1b80 4 client.16557 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.247+0000 7f113bff1b80 1 -- 172.21.15.90:0/1922648497 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c0c7b4d90 con 0x557c0c89e690 2024-05-27T07:51:36.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.248+0000 7f1125e00700 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x557c0c89e690 0x557c0c8a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.250+0000 7f1125e00700 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x557c0c89e690 0x557c0c8a0ab0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc263eda953b5c8e server_cookie=74735f74b3343684 in_seq=0 out_seq=0 2024-05-27T07:51:36.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.250+0000 7f1114c00700 10 client.16557 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:51:36.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.252+0000 7f1114c00700 1 -- 172.21.15.90:0/1922648497 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557c0c7b4d90 con 0x557c0c89e690 2024-05-27T07:51:36.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.252+0000 7f1114c00700 10 client.16557 handle_command_reply: tid=0 2024-05-27T07:51:36.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.252+0000 7f113bff1b80 1 client.16557 shutdown 2024-05-27T07:51:36.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.253+0000 7f113bff1b80 1 -- 172.21.15.90:0/1922648497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 msgr2=0x557c0c88e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:36.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.253+0000 7f113bff1b80 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c88e650 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f11180df6f0 tx=0x7f11180dfb40 comp rx=0 tx=0).stop 2024-05-27T07:51:36.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.253+0000 7f113bff1b80 1 -- 172.21.15.90:0/1922648497 shutdown_connections 2024-05-27T07:51:36.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.253+0000 7f113bff1b80 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x557c0c89e690 0x557c0c8a0ab0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.253+0000 7f113bff1b80 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 0x557c0c88f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.253+0000 7f1114c00700 0 client.16557 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:51:36.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.253+0000 7f113bff1b80 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c88e650 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.253+0000 7f113bff1b80 1 --2- 172.21.15.90:0/1922648497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.253+0000 7f113bff1b80 1 -- 172.21.15.90:0/1922648497 >> 172.21.15.90:0/1922648497 conn(0x557c0c765fc0 msgr2=0x557c0c87f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:36.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.254+0000 7f113bff1b80 1 -- 172.21.15.90:0/1922648497 shutdown_connections 2024-05-27T07:51:36.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.254+0000 7f113bff1b80 1 -- 172.21.15.90:0/1922648497 wait complete. 2024-05-27T07:51:36.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.255+0000 7f113bff1b80 20 client.16557 trim_cache size 0 max 16384 2024-05-27T07:51:36.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.283+0000 7f113bff1b80 1 Processor -- start 2024-05-27T07:51:36.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.283+0000 7f113bff1b80 1 -- start start 2024-05-27T07:51:36.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.283+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c884870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.283+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.283+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 0x557c0c88c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.283+0000 7f113bff1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b8530 con 0x557c0c705e40 2024-05-27T07:51:36.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.283+0000 7f113bff1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b6320 con 0x557c0c88ad30 2024-05-27T07:51:36.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.283+0000 7f113bff1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b7e60 con 0x557c0c884db0 2024-05-27T07:51:36.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.284+0000 7f1125e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.284+0000 7f1127200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 0x557c0c88c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.284+0000 7f1125e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59500/0 (socket says 172.21.15.90:59500) 2024-05-27T07:51:36.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.284+0000 7f1125e00700 1 -- 172.21.15.90:0/3290526343 learned_addr learned my addr 172.21.15.90:0/3290526343 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:36.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.285+0000 7f1125e00700 1 -- 172.21.15.90:0/3290526343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 msgr2=0x557c0c88c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:36.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.285+0000 7f1125e00700 1 --2- 172.21.15.90:0/3290526343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 0x557c0c88c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.285+0000 7f1125e00700 1 -- 172.21.15.90:0/3290526343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 msgr2=0x557c0c884870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:36.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.285+0000 7f1125e00700 1 --2- 172.21.15.90:0/3290526343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c884870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.285+0000 7f1125e00700 1 -- 172.21.15.90:0/3290526343 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c0c885ad0 con 0x557c0c884db0 2024-05-27T07:51:36.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.285+0000 7f1125e00700 1 --2- 172.21.15.90:0/3290526343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88f040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f11100098e0 tx=0x7f1110022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2afb8d8a26253747 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:36.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.286+0000 7f1117400700 1 -- 172.21.15.90:0/3290526343 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110043070 con 0x557c0c884db0 2024-05-27T07:51:36.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.286+0000 7f1117400700 1 -- 172.21.15.90:0/3290526343 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f111000fc30 con 0x557c0c884db0 2024-05-27T07:51:36.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.286+0000 7f1117400700 1 -- 172.21.15.90:0/3290526343 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110005460 con 0x557c0c884db0 2024-05-27T07:51:36.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.287+0000 7f113bff1b80 1 -- 172.21.15.90:0/3290526343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 msgr2=0x557c0c88f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:36.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.287+0000 7f113bff1b80 1 --2- 172.21.15.90:0/3290526343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88f040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f11100098e0 tx=0x7f1110022000 comp rx=0 tx=0).stop 2024-05-27T07:51:36.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.288+0000 7f113bff1b80 1 -- 172.21.15.90:0/3290526343 shutdown_connections 2024-05-27T07:51:36.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.288+0000 7f113bff1b80 1 --2- 172.21.15.90:0/3290526343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c88ad30 0x557c0c88c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.288+0000 7f113bff1b80 1 --2- 172.21.15.90:0/3290526343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c705e40 0x557c0c884870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.288+0000 7f113bff1b80 1 --2- 172.21.15.90:0/3290526343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c884db0 0x557c0c88f040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.288+0000 7f113bff1b80 1 -- 172.21.15.90:0/3290526343 >> 172.21.15.90:0/3290526343 conn(0x557c0c765fc0 msgr2=0x557c0c889e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:36.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.288+0000 7f113bff1b80 1 -- 172.21.15.90:0/3290526343 shutdown_connections 2024-05-27T07:51:36.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.288+0000 7f113bff1b80 1 -- 172.21.15.90:0/3290526343 wait complete. 2024-05-27T07:51:36.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.289+0000 7f113bff1b80 1 -- start start 2024-05-27T07:51:36.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.289+0000 7f113bff1b80 1 Processor -- start 2024-05-27T07:51:36.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.290+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c705e40 0x557c0c896e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.290+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c884db0 0x557c0c8a5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.290+0000 7f113bff1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 0x557c0c8a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.290+0000 7f113bff1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b70c0 con 0x557c0c884db0 2024-05-27T07:51:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.290+0000 7f113bff1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b69f0 con 0x557c0c705e40 2024-05-27T07:51:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.290+0000 7f113bff1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b7790 con 0x557c0c88ad30 2024-05-27T07:51:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.290+0000 7f113bff1b80 10 client.0 _ll_register_callbacks cb 0x557c0c74cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.291+0000 7f1126800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c705e40 0x557c0c896e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.291+0000 7f1126800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c705e40 0x557c0c896e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:58670/0 (socket says 172.21.15.90:58670) 2024-05-27T07:51:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.291+0000 7f1126800700 1 -- 172.21.15.90:0/129122804 learned_addr learned my addr 172.21.15.90:0/129122804 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.291+0000 7f1127200700 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 0x557c0c8a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.292+0000 7f1126800700 1 -- 172.21.15.90:0/129122804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 msgr2=0x557c0c8a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.292+0000 7f1126800700 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 0x557c0c8a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.292+0000 7f1126800700 1 -- 172.21.15.90:0/129122804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c884db0 msgr2=0x557c0c8a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.292+0000 7f1126800700 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c884db0 0x557c0c8a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.292+0000 7f1126800700 1 -- 172.21.15.90:0/129122804 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c0c8a63c0 con 0x557c0c705e40 2024-05-27T07:51:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.292+0000 7f1126800700 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c705e40 0x557c0c896e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f11180ad560 tx=0x7f1118012b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed76d80374e8013a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.292+0000 7f1114c00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:51:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.293+0000 7f1114c00700 1 -- 172.21.15.90:0/129122804 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180a9280 con 0x557c0c705e40 2024-05-27T07:51:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.293+0000 7f1114c00700 1 -- 172.21.15.90:0/129122804 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f1118005c60 con 0x557c0c705e40 2024-05-27T07:51:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.293+0000 7f113bff1b80 1 -- 172.21.15.90:0/129122804 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111000fc30 con 0x557c0c705e40 2024-05-27T07:51:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.294+0000 7f1114c00700 1 -- 172.21.15.90:0/129122804 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1118005e60 con 0x557c0c705e40 2024-05-27T07:51:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.294+0000 7f113bff1b80 10 client.36056 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.294+0000 7f113bff1b80 1 -- 172.21.15.90:0/129122804 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557c0c908d30 con 0x557c0c705e40 2024-05-27T07:51:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.294+0000 7f1114c00700 1 -- 172.21.15.90:0/129122804 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f111000fc30 con 0x557c0c705e40 2024-05-27T07:51:36.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.294+0000 7f1114c00700 1 -- 172.21.15.90:0/129122804 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7f11180a4bf0 con 0x557c0c705e40 2024-05-27T07:51:36.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.295+0000 7f113bff1b80 10 client.36056 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:36.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.295+0000 7f113bff1b80 10 client.36056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:51:36.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.295+0000 7f113bff1b80 20 client.36056 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:36.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.295+0000 7f113bff1b80 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x557c0c909470 0x557c0c90b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:36.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.295+0000 7f113bff1b80 4 client.36056 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:51:36.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.295+0000 7f113bff1b80 1 -- 172.21.15.90:0/129122804 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c0c7ae570 con 0x557c0c909470 2024-05-27T07:51:36.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.295+0000 7f1125e00700 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x557c0c909470 0x557c0c90b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:36.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.297+0000 7f1125e00700 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x557c0c909470 0x557c0c90b890 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=947f2fb1022dc007 server_cookie=605a5815953221cb in_seq=0 out_seq=0 2024-05-27T07:51:36.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.297+0000 7f1114c00700 10 client.36056 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:51:36.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.297+0000 7f1114c00700 1 -- 172.21.15.90:0/129122804 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c0c7ae570 con 0x557c0c909470 2024-05-27T07:51:36.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.297+0000 7f1114c00700 10 client.36056 handle_command_reply: tid=0 2024-05-27T07:51:36.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.297+0000 7f113bff1b80 1 client.36056 shutdown 2024-05-27T07:51:36.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.297+0000 7f113bff1b80 1 -- 172.21.15.90:0/129122804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c705e40 msgr2=0x557c0c896e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:36.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.297+0000 7f113bff1b80 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c705e40 0x557c0c896e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f11180ad560 tx=0x7f1118012b30 comp rx=0 tx=0).stop 2024-05-27T07:51:36.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.299+0000 7f113bff1b80 1 -- 172.21.15.90:0/129122804 shutdown_connections 2024-05-27T07:51:36.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.299+0000 7f113bff1b80 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x557c0c909470 0x557c0c90b890 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.299+0000 7f113bff1b80 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c0c88ad30 0x557c0c8a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.299+0000 7f113bff1b80 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c0c705e40 0x557c0c896e30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.299+0000 7f113bff1b80 1 --2- 172.21.15.90:0/129122804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c0c884db0 0x557c0c8a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.299+0000 7f113bff1b80 1 -- 172.21.15.90:0/129122804 >> 172.21.15.90:0/129122804 conn(0x557c0c765fc0 msgr2=0x557c0c889e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.299+0000 7f1114c00700 0 client.36056 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:51:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.300+0000 7f113bff1b80 1 -- 172.21.15.90:0/129122804 shutdown_connections 2024-05-27T07:51:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.300+0000 7f113bff1b80 1 -- 172.21.15.90:0/129122804 wait complete. 2024-05-27T07:51:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.300+0000 7f113bff1b80 20 client.36056 trim_cache size 0 max 16384 2024-05-27T07:51:36.301 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:51:36.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.301+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f110c0b2af0 msgr2=0x7f110c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:36.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.301+0000 7f112d400700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f110c0b2af0 0x7f110c0b5000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1110009750 tx=0x7f111001a040 comp rx=0 tx=0).stop 2024-05-27T07:51:36.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.302+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1128075b50 msgr2=0x7f11281a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:36.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.302+0000 7f112d400700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1128075b50 0x7f11281a1230 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1118007e20 tx=0x7f111800a340 comp rx=0 tx=0).stop 2024-05-27T07:51:36.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.302+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 shutdown_connections 2024-05-27T07:51:36.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.302+0000 7f112d400700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1128115360 0x7f11281a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.302+0000 7f112d400700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1128075b50 0x7f11281a1230 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.302+0000 7f112d400700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f110c0b2af0 0x7f110c0b5000 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.302+0000 7f112d400700 1 --2- 172.21.15.90:0/3341937436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11280764a0 0x7f11281a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:36.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.302+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 >> 172.21.15.90:0/3341937436 conn(0x7f1128100030 msgr2=0x7f1128100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:36.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.303+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 shutdown_connections 2024-05-27T07:51:36.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:36.304+0000 7f112d400700 1 -- 172.21.15.90:0/3341937436 wait complete. 2024-05-27T07:51:36.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:36 smithi136 ceph-mon[35324]: pgmap v163: 129 pgs: 129 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.1 MiB/s wr, 608 op/s 2024-05-27T07:51:36.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:36 smithi129 ceph-mon[35254]: pgmap v163: 129 pgs: 129 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.1 MiB/s wr, 608 op/s 2024-05-27T07:51:38.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:38 smithi090 ceph-mon[31429]: pgmap v164: 129 pgs: 129 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.5 MiB/s wr, 546 op/s 2024-05-27T07:51:38.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:38 smithi136 ceph-mon[35324]: pgmap v164: 129 pgs: 129 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.5 MiB/s wr, 546 op/s 2024-05-27T07:51:38.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:38 smithi129 ceph-mon[35254]: pgmap v164: 129 pgs: 129 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.5 MiB/s wr, 546 op/s 2024-05-27T07:51:38.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 80.0 s, 564.0 tps, lat 171.774 ms stddev 81.624 2024-05-27T07:51:40.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:40 smithi090 ceph-mon[31429]: pgmap v165: 129 pgs: 129 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 852 op/s 2024-05-27T07:51:40.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:40 smithi136 ceph-mon[35324]: pgmap v165: 129 pgs: 129 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 852 op/s 2024-05-27T07:51:40.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:40 smithi129 ceph-mon[35254]: pgmap v165: 129 pgs: 129 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.1 MiB/s wr, 852 op/s 2024-05-27T07:51:41.066 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:51:41.721 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:42.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:42 smithi090 ceph-mon[31429]: pgmap v166: 129 pgs: 129 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.0 MiB/s wr, 603 op/s 2024-05-27T07:51:42.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:42 smithi136 ceph-mon[35324]: pgmap v166: 129 pgs: 129 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.0 MiB/s wr, 603 op/s 2024-05-27T07:51:42.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:42 smithi129 ceph-mon[35254]: pgmap v166: 129 pgs: 129 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.0 MiB/s wr, 603 op/s 2024-05-27T07:51:43.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 85.0 s, 597.2 tps, lat 171.095 ms stddev 93.413 2024-05-27T07:51:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.386+0000 7fc2c5200640 1 -- 172.21.15.90:0/3232169846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2c0108b90 msgr2=0x7fc2c0102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:44.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.386+0000 7fc2c5200640 1 --2- 172.21.15.90:0/3232169846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2c0108b90 0x7fc2c0102730 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009ca0 tx=0x7fc2b802a8d0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.389+0000 7fc2c5200640 1 -- 172.21.15.90:0/3232169846 shutdown_connections 2024-05-27T07:51:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.389+0000 7fc2c5200640 1 --2- 172.21.15.90:0/3232169846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c01032e0 0x7fc2c010f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.389+0000 7fc2c5200640 1 --2- 172.21.15.90:0/3232169846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2c0109030 0x7fc2c0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.389+0000 7fc2c5200640 1 --2- 172.21.15.90:0/3232169846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2c0108b90 0x7fc2c0102730 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.389+0000 7fc2c5200640 1 -- 172.21.15.90:0/3232169846 >> 172.21.15.90:0/3232169846 conn(0x7fc2c00782d0 msgr2=0x7fc2c0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:44.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.389+0000 7fc2c5200640 1 -- 172.21.15.90:0/3232169846 shutdown_connections 2024-05-27T07:51:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.389+0000 7fc2c5200640 1 -- 172.21.15.90:0/3232169846 wait complete. 2024-05-27T07:51:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.391+0000 7fc2c5200640 1 Processor -- start 2024-05-27T07:51:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.391+0000 7fc2c5200640 1 -- start start 2024-05-27T07:51:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.391+0000 7fc2c5200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2c01032e0 0x7fc2c0070080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.391+0000 7fc2c5200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c0108b90 0x7fc2c00705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.392+0000 7fc2c5200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2c0109030 0x7fc2c010c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.392+0000 7fc2c5200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0111b10 con 0x7fc2c0108b90 2024-05-27T07:51:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.392+0000 7fc2c5200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0111990 con 0x7fc2c0109030 2024-05-27T07:51:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.392+0000 7fc2c5200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0111c90 con 0x7fc2c01032e0 2024-05-27T07:51:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.392+0000 7fc2bde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2c01032e0 0x7fc2c0070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.392+0000 7fc2bde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2c01032e0 0x7fc2c0070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50542/0 (socket says 172.21.15.90:50542) 2024-05-27T07:51:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.392+0000 7fc2bde00640 1 -- 172.21.15.90:0/1259850645 learned_addr learned my addr 172.21.15.90:0/1259850645 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:44.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.393+0000 7fc2bd400640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c0108b90 0x7fc2c00705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.393+0000 7fc2be800640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2c0109030 0x7fc2c010c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.393+0000 7fc2bde00640 1 -- 172.21.15.90:0/1259850645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2c0109030 msgr2=0x7fc2c010c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.393+0000 7fc2bde00640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2c0109030 0x7fc2c010c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.393+0000 7fc2bde00640 1 -- 172.21.15.90:0/1259850645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c0108b90 msgr2=0x7fc2c00705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.393+0000 7fc2bde00640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c0108b90 0x7fc2c00705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.393+0000 7fc2bde00640 1 -- 172.21.15.90:0/1259850645 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c010c970 con 0x7fc2c01032e0 2024-05-27T07:51:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.394+0000 7fc2bde00640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2c01032e0 0x7fc2c0070080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8005e30 tx=0x7fc2b8030880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.395+0000 7fc2b6a00640 1 -- 172.21.15.90:0/1259850645 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8030e80 con 0x7fc2c01032e0 2024-05-27T07:51:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.395+0000 7fc2c5200640 1 -- 172.21.15.90:0/1259850645 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c010cc00 con 0x7fc2c01032e0 2024-05-27T07:51:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.396+0000 7fc2b6a00640 1 -- 172.21.15.90:0/1259850645 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2b80458c0 con 0x7fc2c01032e0 2024-05-27T07:51:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.396+0000 7fc2b6a00640 1 -- 172.21.15.90:0/1259850645 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b803b950 con 0x7fc2c01032e0 2024-05-27T07:51:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.396+0000 7fc2c5200640 1 -- 172.21.15.90:0/1259850645 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c0073d80 con 0x7fc2c01032e0 2024-05-27T07:51:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.401+0000 7fc2cc226740 1 Processor -- start 2024-05-27T07:51:44.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.401+0000 7fc2cc226740 1 -- start start 2024-05-27T07:51:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.405+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36d11b0 0x5597d3a73860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.405+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36a82f0 0x5597d3a73da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.405+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36b04d0 0x5597d3a742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.405+0000 7fc2cc226740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5597d39e4b90 con 0x5597d36a82f0 2024-05-27T07:51:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.405+0000 7fc2cc226740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5597d39d4730 con 0x5597d36d11b0 2024-05-27T07:51:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.405+0000 7fc2cc226740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5597d39d8770 con 0x5597d36b04d0 2024-05-27T07:51:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.406+0000 7fc2be800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36d11b0 0x5597d3a73860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.406+0000 7fc2be800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36d11b0 0x5597d3a73860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47040/0 (socket says 172.21.15.90:47040) 2024-05-27T07:51:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.406+0000 7fc2be800640 1 -- 172.21.15.90:0/370276468 learned_addr learned my addr 172.21.15.90:0/370276468 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.406+0000 7fc2bd400640 1 --2- 172.21.15.90:0/370276468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36b04d0 0x5597d3a742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.407+0000 7fc2be800640 1 -- 172.21.15.90:0/370276468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36b04d0 msgr2=0x5597d3a742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.407+0000 7fc2b6a00640 1 -- 172.21.15.90:0/1259850645 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc2b8045430 con 0x7fc2c01032e0 2024-05-27T07:51:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.407+0000 7fc2be800640 1 --2- 172.21.15.90:0/370276468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36b04d0 0x5597d3a742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.408+0000 7fc2be800640 1 -- 172.21.15.90:0/370276468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36a82f0 msgr2=0x5597d3a73da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:51:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.408+0000 7fc2be800640 1 --2- 172.21.15.90:0/370276468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36a82f0 0x5597d3a73da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.408+0000 7fc2be800640 1 -- 172.21.15.90:0/370276468 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d396d8b0 con 0x5597d36d11b0 2024-05-27T07:51:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.408+0000 7fc2bd400640 1 --2- 172.21.15.90:0/370276468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36b04d0 0x5597d3a742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.409+0000 7fc2be800640 1 --2- 172.21.15.90:0/370276468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36d11b0 0x5597d3a73860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000ab40 tx=0x7fc2b000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0de184cc6b5162 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.409+0000 7fc2ab400640 1 -- 172.21.15.90:0/370276468 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0005020 con 0x5597d36d11b0 2024-05-27T07:51:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.409+0000 7fc2ab400640 1 -- 172.21.15.90:0/370276468 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2b0005650 con 0x5597d36d11b0 2024-05-27T07:51:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.411+0000 7fc2cc226740 1 -- 172.21.15.90:0/370276468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36d11b0 msgr2=0x5597d3a73860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.411+0000 7fc2cc226740 1 --2- 172.21.15.90:0/370276468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36d11b0 0x5597d3a73860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000ab40 tx=0x7fc2b000f3c0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.413+0000 7fc2b6a00640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2a40b2a90 0x7fc2a40b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.414+0000 7fc2b6a00640 1 -- 172.21.15.90:0/1259850645 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fc2b81039d0 con 0x7fc2c01032e0 2024-05-27T07:51:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.414+0000 7fc2cc226740 1 -- 172.21.15.90:0/370276468 shutdown_connections 2024-05-27T07:51:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.414+0000 7fc2cc226740 1 --2- 172.21.15.90:0/370276468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36b04d0 0x5597d3a742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.414+0000 7fc2cc226740 1 --2- 172.21.15.90:0/370276468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36a82f0 0x5597d3a73da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.414+0000 7fc2cc226740 1 --2- 172.21.15.90:0/370276468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36d11b0 0x5597d3a73860 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.414+0000 7fc2cc226740 1 -- 172.21.15.90:0/370276468 >> 172.21.15.90:0/370276468 conn(0x5597d36c5300 msgr2=0x5597d3a70010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.414+0000 7fc2be800640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2a40b2a90 0x7fc2a40b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.415+0000 7fc2cc226740 1 -- 172.21.15.90:0/370276468 shutdown_connections 2024-05-27T07:51:44.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.416+0000 7fc2be800640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2a40b2a90 0x7fc2a40b4fa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000f8c0 tx=0x7fc2b0012040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.416+0000 7fc2cc226740 1 -- 172.21.15.90:0/370276468 wait complete. 2024-05-27T07:51:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.417+0000 7fc2cc226740 1 -- start start 2024-05-27T07:51:44.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.417+0000 7fc2cc226740 1 Processor -- start 2024-05-27T07:51:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.418+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36a82f0 0x5597d3a7d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.418+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a7de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.418+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 0x5597d3a7e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.418+0000 7fc2cc226740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5597d36ba790 con 0x5597d36a82f0 2024-05-27T07:51:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.418+0000 7fc2cc226740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5597d397cbc0 con 0x5597d36b04d0 2024-05-27T07:51:44.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.418+0000 7fc2cc226740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5597d3a809e0 con 0x5597d36d11b0 2024-05-27T07:51:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.418+0000 7fc2bd400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a7de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.419+0000 7fc2bd400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a7de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47044/0 (socket says 172.21.15.90:47044) 2024-05-27T07:51:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.419+0000 7fc2bd400640 1 -- 172.21.15.90:0/2943656729 learned_addr learned my addr 172.21.15.90:0/2943656729 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.419+0000 7fc2be800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 0x5597d3a7e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.419+0000 7fc2cc226740 10 client.0 _ll_register_callbacks cb 0x5597d395a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.419+0000 7fc2bde00640 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36a82f0 0x5597d3a7d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.420+0000 7fc2bd400640 1 -- 172.21.15.90:0/2943656729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 msgr2=0x5597d3a7e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.420+0000 7fc2bd400640 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 0x5597d3a7e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.420+0000 7fc2bd400640 1 -- 172.21.15.90:0/2943656729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36a82f0 msgr2=0x5597d3a7d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.420+0000 7fc2bd400640 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36a82f0 0x5597d3a7d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.420+0000 7fc2bd400640 1 -- 172.21.15.90:0/2943656729 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d36e9210 con 0x5597d36b04d0 2024-05-27T07:51:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.420+0000 7fc2bd400640 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a7de90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00bdc0 tx=0x7fc2ac012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=486c133e3d179ace server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.420+0000 7fc29fe00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:51:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.421+0000 7fc29fe00640 1 -- 172.21.15.90:0/2943656729 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac004220 con 0x5597d36b04d0 2024-05-27T07:51:44.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.421+0000 7fc2cc226740 1 -- 172.21.15.90:0/2943656729 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2b0005650 con 0x5597d36b04d0 2024-05-27T07:51:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.421+0000 7fc29fe00640 1 -- 172.21.15.90:0/2943656729 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fc2ac0043c0 con 0x5597d36b04d0 2024-05-27T07:51:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.422+0000 7fc29fe00640 1 -- 172.21.15.90:0/2943656729 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac005020 con 0x5597d36b04d0 2024-05-27T07:51:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.422+0000 7fc2cc226740 10 client.36068 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.422+0000 7fc2cc226740 1 -- 172.21.15.90:0/2943656729 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5597d37b5620 con 0x5597d36b04d0 2024-05-27T07:51:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.422+0000 7fc29fe00640 1 -- 172.21.15.90:0/2943656729 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2b0005650 con 0x5597d36b04d0 2024-05-27T07:51:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.422+0000 7fc29fe00640 1 -- 172.21.15.90:0/2943656729 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fc2ac019070 con 0x5597d36b04d0 2024-05-27T07:51:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.422+0000 7fc2cc226740 10 client.36068 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.422+0000 7fc2cc226740 10 client.36068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.422+0000 7fc2cc226740 20 client.36068 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.423+0000 7fc2cc226740 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5597d368fd80 0x5597d3a8fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.423+0000 7fc2cc226740 4 client.36068 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.423+0000 7fc2cc226740 1 -- 172.21.15.90:0/2943656729 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597d39e6ae0 con 0x5597d368fd80 2024-05-27T07:51:44.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.424+0000 7fc2bde00640 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5597d368fd80 0x5597d3a8fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.425+0000 7fc29fe00640 10 client.36068 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:44.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.425+0000 7fc2bde00640 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5597d368fd80 0x5597d3a8fdf0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bdef1f9e445a0fa server_cookie=bfe4b9d7833a1277 in_seq=0 out_seq=0 2024-05-27T07:51:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.426+0000 7fc29fe00640 1 -- 172.21.15.90:0/2943656729 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5597d39e6ae0 con 0x5597d368fd80 2024-05-27T07:51:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.426+0000 7fc29fe00640 10 client.36068 handle_command_reply: tid=0 2024-05-27T07:51:44.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.426+0000 7fc2cc226740 1 client.36068 shutdown 2024-05-27T07:51:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.427+0000 7fc2cc226740 1 -- 172.21.15.90:0/2943656729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 msgr2=0x5597d3a7de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.427+0000 7fc2cc226740 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a7de90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00bdc0 tx=0x7fc2ac012820 comp rx=0 tx=0).stop 2024-05-27T07:51:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.427+0000 7fc2cc226740 1 -- 172.21.15.90:0/2943656729 shutdown_connections 2024-05-27T07:51:44.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.427+0000 7fc2cc226740 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5597d368fd80 0x5597d3a8fdf0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.427+0000 7fc2cc226740 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a7de90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.427+0000 7fc2cc226740 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 0x5597d3a7e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.427+0000 7fc2cc226740 1 --2- 172.21.15.90:0/2943656729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36a82f0 0x5597d3a7d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.427+0000 7fc2cc226740 1 -- 172.21.15.90:0/2943656729 >> 172.21.15.90:0/2943656729 conn(0x5597d36c5300 msgr2=0x5597d3a73690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.428+0000 7fc29fe00640 0 client.36068 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.428+0000 7fc2cc226740 1 -- 172.21.15.90:0/2943656729 shutdown_connections 2024-05-27T07:51:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.429+0000 7fc2cc226740 1 -- 172.21.15.90:0/2943656729 wait complete. 2024-05-27T07:51:44.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.429+0000 7fc2cc226740 20 client.36068 trim_cache size 0 max 16384 2024-05-27T07:51:44.452 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:44 smithi090 ceph-mon[31429]: pgmap v167: 129 pgs: 129 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.9 MiB/s wr, 828 op/s 2024-05-27T07:51:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.462+0000 7fc2cc226740 1 Processor -- start 2024-05-27T07:51:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.462+0000 7fc2cc226740 1 -- start start 2024-05-27T07:51:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.463+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36a82f0 0x5597d3a73d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.463+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a6f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.463+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36d11b0 0x5597d3a955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.463+0000 7fc2cc226740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01c1900 con 0x5597d36d11b0 2024-05-27T07:51:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.464+0000 7fc2cc226740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01bf6f0 con 0x5597d36b04d0 2024-05-27T07:51:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.464+0000 7fc2cc226740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01c1230 con 0x5597d36a82f0 2024-05-27T07:51:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.464+0000 7fc2be800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36d11b0 0x5597d3a955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.464+0000 7fc2be800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36d11b0 0x5597d3a955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52762/0 (socket says 172.21.15.90:52762) 2024-05-27T07:51:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.464+0000 7fc2be800640 1 -- 172.21.15.90:0/3942702819 learned_addr learned my addr 172.21.15.90:0/3942702819 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.464+0000 7fc2bd400640 1 --2- 172.21.15.90:0/3942702819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a6f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.464+0000 7fc2be800640 1 -- 172.21.15.90:0/3942702819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36a82f0 msgr2=0x5597d3a73d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.464+0000 7fc2be800640 1 --2- 172.21.15.90:0/3942702819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36a82f0 0x5597d3a73d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.464+0000 7fc2be800640 1 -- 172.21.15.90:0/3942702819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 msgr2=0x5597d3a6f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.464+0000 7fc2be800640 1 --2- 172.21.15.90:0/3942702819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a6f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.465+0000 7fc2be800640 1 -- 172.21.15.90:0/3942702819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d3a6a050 con 0x5597d36d11b0 2024-05-27T07:51:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.465+0000 7fc2bd400640 1 --2- 172.21.15.90:0/3942702819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a6f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:44.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.465+0000 7fc2be800640 1 --2- 172.21.15.90:0/3942702819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36d11b0 0x5597d3a955f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc2b001a040 tx=0x7fc2b0022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b123478a4379b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.466+0000 7fc2aaa00640 1 -- 172.21.15.90:0/3942702819 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b002b040 con 0x5597d36d11b0 2024-05-27T07:51:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.466+0000 7fc2aaa00640 1 -- 172.21.15.90:0/3942702819 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2b002c040 con 0x5597d36d11b0 2024-05-27T07:51:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.466+0000 7fc2aaa00640 1 -- 172.21.15.90:0/3942702819 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00053c0 con 0x5597d36d11b0 2024-05-27T07:51:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.468+0000 7fc2cc226740 1 -- 172.21.15.90:0/3942702819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36d11b0 msgr2=0x5597d3a955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:44.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.468+0000 7fc2cc226740 1 --2- 172.21.15.90:0/3942702819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36d11b0 0x5597d3a955f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc2b001a040 tx=0x7fc2b0022000 comp rx=0 tx=0).stop 2024-05-27T07:51:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.468+0000 7fc2cc226740 1 -- 172.21.15.90:0/3942702819 shutdown_connections 2024-05-27T07:51:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.468+0000 7fc2cc226740 1 --2- 172.21.15.90:0/3942702819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36b04d0 0x5597d3a6f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.468+0000 7fc2cc226740 1 --2- 172.21.15.90:0/3942702819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36d11b0 0x5597d3a955f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.468+0000 7fc2cc226740 1 --2- 172.21.15.90:0/3942702819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36a82f0 0x5597d3a73d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.468+0000 7fc2cc226740 1 -- 172.21.15.90:0/3942702819 >> 172.21.15.90:0/3942702819 conn(0x5597d36c5300 msgr2=0x5597d3a735f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.469+0000 7fc2cc226740 1 -- 172.21.15.90:0/3942702819 shutdown_connections 2024-05-27T07:51:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.469+0000 7fc2cc226740 1 -- 172.21.15.90:0/3942702819 wait complete. 2024-05-27T07:51:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.469+0000 7fc2cc226740 1 -- start start 2024-05-27T07:51:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.470+0000 7fc2cc226740 1 Processor -- start 2024-05-27T07:51:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.471+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36a82f0 0x5597d3a860c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.471+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36b04d0 0x5597d3a81440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.471+0000 7fc2cc226740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 0x5597d3a81980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.471+0000 7fc2cc226740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01c0490 con 0x5597d36b04d0 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.471+0000 7fc2cc226740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01bfdc0 con 0x5597d36a82f0 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.471+0000 7fc2cc226740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c01c0b60 con 0x5597d36d11b0 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.471+0000 7fc2cc226740 10 client.0 _ll_register_callbacks cb 0x7fc2ac00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.471+0000 7fc2bde00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36a82f0 0x5597d3a860c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.472+0000 7fc2bde00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36a82f0 0x5597d3a860c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47060/0 (socket says 172.21.15.90:47060) 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.471+0000 7fc2be800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 0x5597d3a81980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.472+0000 7fc2be800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 0x5597d3a81980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50586/0 (socket says 172.21.15.90:50586) 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.472+0000 7fc2be800640 1 -- 172.21.15.90:0/3870832646 learned_addr learned my addr 172.21.15.90:0/3870832646 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.471+0000 7fc2bd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36b04d0 0x5597d3a81440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.473+0000 7fc2bd400640 1 -- 172.21.15.90:0/3870832646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 msgr2=0x5597d3a81980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.473+0000 7fc2bd400640 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 0x5597d3a81980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.473+0000 7fc2bd400640 1 -- 172.21.15.90:0/3870832646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36a82f0 msgr2=0x5597d3a860c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.473+0000 7fc2bd400640 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36a82f0 0x5597d3a860c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.473+0000 7fc2bd400640 1 -- 172.21.15.90:0/3870832646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d3a81ec0 con 0x5597d36b04d0 2024-05-27T07:51:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.473+0000 7fc2bde00640 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36a82f0 0x5597d3a860c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.474+0000 7fc2bd400640 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36b04d0 0x5597d3a81440 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac000c00 tx=0x7fc2ac012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02f0f0b624fabad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.474+0000 7fc29fe00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.474+0000 7fc29fe00640 1 -- 172.21.15.90:0/3870832646 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0053c0 con 0x5597d36b04d0 2024-05-27T07:51:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.474+0000 7fc29fe00640 1 -- 172.21.15.90:0/3870832646 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fc2ac005560 con 0x5597d36b04d0 2024-05-27T07:51:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.474+0000 7fc2cc226740 1 -- 172.21.15.90:0/3870832646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2b002c040 con 0x5597d36b04d0 2024-05-27T07:51:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.475+0000 7fc29fe00640 1 -- 172.21.15.90:0/3870832646 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac004680 con 0x5597d36b04d0 2024-05-27T07:51:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.475+0000 7fc2cc226740 10 client.16569 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.475+0000 7fc2cc226740 1 -- 172.21.15.90:0/3870832646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5597d3af80b0 con 0x5597d36b04d0 2024-05-27T07:51:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.476+0000 7fc29fe00640 1 -- 172.21.15.90:0/3870832646 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2b002c040 con 0x5597d36b04d0 2024-05-27T07:51:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.477+0000 7fc29fe00640 1 -- 172.21.15.90:0/3870832646 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fc2ac019040 con 0x5597d36b04d0 2024-05-27T07:51:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.478+0000 7fc2cc226740 10 client.16569 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.478+0000 7fc2cc226740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.478+0000 7fc2cc226740 20 client.16569 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.478+0000 7fc2cc226740 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5597d36651c0 0x5597d3afa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.478+0000 7fc2cc226740 4 client.16569 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:51:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.478+0000 7fc2cc226740 1 -- 172.21.15.90:0/3870832646 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597d39e02c0 con 0x5597d36651c0 2024-05-27T07:51:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.478+0000 7fc2bde00640 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5597d36651c0 0x5597d3afa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:44.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.480+0000 7fc2bde00640 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5597d36651c0 0x5597d3afa810 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d3ac3cae52fd617 server_cookie=c30442fe0d86aff3 in_seq=0 out_seq=0 2024-05-27T07:51:44.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.480+0000 7fc29fe00640 10 client.16569 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:44.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:44 smithi136 ceph-mon[35324]: pgmap v167: 129 pgs: 129 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.9 MiB/s wr, 828 op/s 2024-05-27T07:51:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.530+0000 7fc29fe00640 1 -- 172.21.15.90:0/3870832646 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5597d39e02c0 con 0x5597d36651c0 2024-05-27T07:51:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.530+0000 7fc29fe00640 10 client.16569 handle_command_reply: tid=0 2024-05-27T07:51:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.530+0000 7fc2cc226740 1 client.16569 shutdown 2024-05-27T07:51:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.530+0000 7fc2cc226740 1 -- 172.21.15.90:0/3870832646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36b04d0 msgr2=0x5597d3a81440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.531+0000 7fc2cc226740 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36b04d0 0x5597d3a81440 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac000c00 tx=0x7fc2ac012e10 comp rx=0 tx=0).stop 2024-05-27T07:51:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.531+0000 7fc2cc226740 1 -- 172.21.15.90:0/3870832646 shutdown_connections 2024-05-27T07:51:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.531+0000 7fc2cc226740 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5597d36651c0 0x5597d3afa810 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.531+0000 7fc2cc226740 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5597d36d11b0 0x5597d3a81980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.531+0000 7fc2cc226740 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5597d36b04d0 0x5597d3a81440 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.531+0000 7fc2cc226740 1 --2- 172.21.15.90:0/3870832646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597d36a82f0 0x5597d3a860c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.531+0000 7fc2cc226740 1 -- 172.21.15.90:0/3870832646 >> 172.21.15.90:0/3870832646 conn(0x5597d36c5300 msgr2=0x5597d3a7efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.531+0000 7fc29fe00640 0 client.16569 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.532+0000 7fc2cc226740 1 -- 172.21.15.90:0/3870832646 shutdown_connections 2024-05-27T07:51:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.533+0000 7fc2cc226740 1 -- 172.21.15.90:0/3870832646 wait complete. 2024-05-27T07:51:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.533+0000 7fc2cc226740 20 client.16569 trim_cache size 0 max 16384 2024-05-27T07:51:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:51:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:51:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.536+0000 7fc2c5200640 1 -- 172.21.15.90:0/1259850645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2a40b2a90 msgr2=0x7fc2a40b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.536+0000 7fc2c5200640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2a40b2a90 0x7fc2a40b4fa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000f8c0 tx=0x7fc2b0012040 comp rx=0 tx=0).stop 2024-05-27T07:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.536+0000 7fc2c5200640 1 -- 172.21.15.90:0/1259850645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2c01032e0 msgr2=0x7fc2c0070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.536+0000 7fc2c5200640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2c01032e0 0x7fc2c0070080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8005e30 tx=0x7fc2b8030880 comp rx=0 tx=0).stop 2024-05-27T07:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.536+0000 7fc2c5200640 1 -- 172.21.15.90:0/1259850645 shutdown_connections 2024-05-27T07:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.536+0000 7fc2c5200640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2a40b2a90 0x7fc2a40b4fa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.537+0000 7fc2c5200640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2c0109030 0x7fc2c010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.537+0000 7fc2c5200640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2c0108b90 0x7fc2c00705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.537+0000 7fc2c5200640 1 --2- 172.21.15.90:0/1259850645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2c01032e0 0x7fc2c0070080 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8005e30 tx=0x7fc2b8030880 comp rx=0 tx=0).stop 2024-05-27T07:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.537+0000 7fc2c5200640 1 -- 172.21.15.90:0/1259850645 >> 172.21.15.90:0/1259850645 conn(0x7fc2c00782d0 msgr2=0x7fc2c01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.537+0000 7fc2c5200640 1 -- 172.21.15.90:0/1259850645 shutdown_connections 2024-05-27T07:51:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:44.537+0000 7fc2c5200640 1 -- 172.21.15.90:0/1259850645 wait complete. 2024-05-27T07:51:44.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:44 smithi129 ceph-mon[35254]: pgmap v167: 129 pgs: 129 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.9 MiB/s wr, 828 op/s 2024-05-27T07:51:44.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa640200700 time 2024-05-27T07:51:44.449228+0000 2024-05-27T07:51:44.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:51:44.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:44.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa655d3fba6] 2024-05-27T07:51:44.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa655d3fdd1] 2024-05-27T07:51:44.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ac508cd23d] 2024-05-27T07:51:44.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55ac508b8971] 2024-05-27T07:51:44.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 5: (SnapRealm::check_cache() const+0x13e) [0x55ac508b6fa0] 2024-05-27T07:51:44.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ac508b7651] 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ac5084c347] 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ac5054887e] 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55ac5054de94] 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55ac50558969] 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 11: (MDLog::_replay_thread()+0x1f38) [0x55ac508f148c] 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55ac508fa9cb] 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 13: (Thread::entry_wrapper()+0x43) [0x7fa655d0e72d] 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa655d0e749] 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa6548b21ca] 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 16: clone() 2024-05-27T07:51:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.456+0000 7fa640200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa640200700 time 2024-05-27T07:51:44.449228+0000 2024-05-27T07:51:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:51:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa655d3fba6] 2024-05-27T07:51:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa655d3fdd1] 2024-05-27T07:51:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ac508cd23d] 2024-05-27T07:51:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55ac508b8971] 2024-05-27T07:51:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 5: (SnapRealm::check_cache() const+0x13e) [0x55ac508b6fa0] 2024-05-27T07:51:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ac508b7651] 2024-05-27T07:51:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ac5084c347] 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ac5054887e] 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55ac5054de94] 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55ac50558969] 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 11: (MDLog::_replay_thread()+0x1f38) [0x55ac508f148c] 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55ac508fa9cb] 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 13: (Thread::entry_wrapper()+0x43) [0x7fa655d0e72d] 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa655d0e749] 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa6548b21ca] 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 16: clone() 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: *** Caught signal (Aborted) ** 2024-05-27T07:51:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: in thread 7fa640200700 thread_name:md_log_replay 2024-05-27T07:51:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa6548bcd20] 2024-05-27T07:51:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2: gsignal() 2024-05-27T07:51:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 3: abort() 2024-05-27T07:51:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa655d3fced] 2024-05-27T07:51:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa655d3fdd1] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ac508cd23d] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55ac508b8971] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 8: (SnapRealm::check_cache() const+0x13e) [0x55ac508b6fa0] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ac508b7651] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ac5084c347] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ac5054887e] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55ac5054de94] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55ac50558969] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 14: (MDLog::_replay_thread()+0x1f38) [0x55ac508f148c] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55ac508fa9cb] 2024-05-27T07:51:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 16: (Thread::entry_wrapper()+0x43) [0x7fa655d0e72d] 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa655d0e749] 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa6548b21ca] 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 19: clone() 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.459+0000 7fa640200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: in thread 7fa640200700 thread_name:md_log_replay 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa6548bcd20] 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2: gsignal() 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 3: abort() 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa655d3fced] 2024-05-27T07:51:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa655d3fdd1] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ac508cd23d] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55ac508b8971] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 8: (SnapRealm::check_cache() const+0x13e) [0x55ac508b6fa0] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ac508b7651] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ac5084c347] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ac5054887e] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55ac5054de94] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55ac50558969] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 14: (MDLog::_replay_thread()+0x1f38) [0x55ac508f148c] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55ac508fa9cb] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 16: (Thread::entry_wrapper()+0x43) [0x7fa655d0e72d] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa655d0e749] 2024-05-27T07:51:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa6548b21ca] 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 19: clone() 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: -1> 2024-05-27T07:51:44.456+0000 7fa640200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa640200700 time 2024-05-27T07:51:44.449228+0000 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa655d3fba6] 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa655d3fdd1] 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ac508cd23d] 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55ac508b8971] 2024-05-27T07:51:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 5: (SnapRealm::check_cache() const+0x13e) [0x55ac508b6fa0] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ac508b7651] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ac5084c347] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ac5054887e] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55ac5054de94] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55ac50558969] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 11: (MDLog::_replay_thread()+0x1f38) [0x55ac508f148c] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55ac508fa9cb] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 13: (Thread::entry_wrapper()+0x43) [0x7fa655d0e72d] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa655d0e749] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa6548b21ca] 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 16: clone() 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 0> 2024-05-27T07:51:44.459+0000 7fa640200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:51:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: in thread 7fa640200700 thread_name:md_log_replay 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa6548bcd20] 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2: gsignal() 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 3: abort() 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa655d3fced] 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa655d3fdd1] 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ac508cd23d] 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55ac508b8971] 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 8: (SnapRealm::check_cache() const+0x13e) [0x55ac508b6fa0] 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ac508b7651] 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ac5084c347] 2024-05-27T07:51:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ac5054887e] 2024-05-27T07:51:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55ac5054de94] 2024-05-27T07:51:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55ac50558969] 2024-05-27T07:51:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 14: (MDLog::_replay_thread()+0x1f38) [0x55ac508f148c] 2024-05-27T07:51:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55ac508fa9cb] 2024-05-27T07:51:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 16: (Thread::entry_wrapper()+0x43) [0x7fa655d0e72d] 2024-05-27T07:51:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa655d0e749] 2024-05-27T07:51:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa6548b21ca] 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 19: clone() 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: -1> 2024-05-27T07:51:44.456+0000 7fa640200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fa640200700 time 2024-05-27T07:51:44.449228+0000 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fa655d3fba6] 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa655d3fdd1] 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ac508cd23d] 2024-05-27T07:51:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55ac508b8971] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 5: (SnapRealm::check_cache() const+0x13e) [0x55ac508b6fa0] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ac508b7651] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ac5084c347] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ac5054887e] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55ac5054de94] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55ac50558969] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 11: (MDLog::_replay_thread()+0x1f38) [0x55ac508f148c] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55ac508fa9cb] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 13: (Thread::entry_wrapper()+0x43) [0x7fa655d0e72d] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 14: (Thread::_entry_func(void*)+0xd) [0x7fa655d0e749] 2024-05-27T07:51:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa6548b21ca] 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 16: clone() 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 0> 2024-05-27T07:51:44.459+0000 7fa640200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: in thread 7fa640200700 thread_name:md_log_replay 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa6548bcd20] 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2: gsignal() 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 3: abort() 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fa655d3fced] 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fa655d3fdd1] 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55ac508cd23d] 2024-05-27T07:51:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55ac508b8971] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 8: (SnapRealm::check_cache() const+0x13e) [0x55ac508b6fa0] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55ac508b7651] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55ac5084c347] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55ac5054887e] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x55ac5054de94] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55ac50558969] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 14: (MDLog::_replay_thread()+0x1f38) [0x55ac508f148c] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55ac508fa9cb] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 16: (Thread::entry_wrapper()+0x43) [0x7fa655d0e72d] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 17: (Thread::_entry_func(void*)+0xd) [0x7fa655d0e749] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fa6548b21ca] 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 19: clone() 2024-05-27T07:51:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:51:44.730 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[102538]: 2024-05-27T07:51:46.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:46 smithi090 ceph-mon[31429]: pgmap v168: 129 pgs: 129 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 641 op/s 2024-05-27T07:51:46.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:46 smithi136 ceph-mon[35324]: pgmap v168: 129 pgs: 129 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 641 op/s 2024-05-27T07:51:46.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:46 smithi129 ceph-mon[35254]: pgmap v168: 129 pgs: 129 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 641 op/s 2024-05-27T07:51:46.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:46 smithi090 podman[105099]: 2024-05-27 07:51:46.613518477 +0000 UTC m=+2.015758227 container died e5cd702d818caf5d7702e74bb1eb61637356cab3fdbfc03eba4cc0f5f34e5525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T07:51:46.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.905+0000 7fcadbc00700 1 -- 172.21.15.90:0/3996592502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad41034b0 msgr2=0x7fcad41038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:46.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.906+0000 7fcadbc00700 1 --2- 172.21.15.90:0/3996592502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad41034b0 0x7fcad41038c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcac80099c0 tx=0x7fcac8009cd0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.906+0000 7fcadbc00700 1 -- 172.21.15.90:0/3996592502 shutdown_connections 2024-05-27T07:51:46.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.906+0000 7fcadbc00700 1 --2- 172.21.15.90:0/3996592502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4108730 0x7fcad4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.906+0000 7fcadbc00700 1 --2- 172.21.15.90:0/3996592502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad41034b0 0x7fcad41038c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.906+0000 7fcadbc00700 1 --2- 172.21.15.90:0/3996592502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcad4107de0 0x7fcad41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.906+0000 7fcadbc00700 1 -- 172.21.15.90:0/3996592502 >> 172.21.15.90:0/3996592502 conn(0x7fcad40fede0 msgr2=0x7fcad4101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:46.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.906+0000 7fcadbc00700 1 -- 172.21.15.90:0/3996592502 shutdown_connections 2024-05-27T07:51:46.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.906+0000 7fcadbc00700 1 -- 172.21.15.90:0/3996592502 wait complete. 2024-05-27T07:51:46.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.907+0000 7fcadbc00700 1 Processor -- start 2024-05-27T07:51:46.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.907+0000 7fcadbc00700 1 -- start start 2024-05-27T07:51:46.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcadbc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad41034b0 0x7fcad40ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcadbc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcad4107de0 0x7fcad40ff800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcadbc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4108730 0x7fcad40ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcadbc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4113210 con 0x7fcad41034b0 2024-05-27T07:51:46.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcadbc00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4113090 con 0x7fcad4108730 2024-05-27T07:51:46.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcadbc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4113390 con 0x7fcad4107de0 2024-05-27T07:51:46.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad41034b0 0x7fcad40ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad41034b0 0x7fcad40ff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52792/0 (socket says 172.21.15.90:52792) 2024-05-27T07:51:46.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad9200700 1 -- 172.21.15.90:0/2337965700 learned_addr learned my addr 172.21.15.90:0/2337965700 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:46.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad9c00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4108730 0x7fcad40ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad9200700 1 -- 172.21.15.90:0/2337965700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcad4107de0 msgr2=0x7fcad40ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad3e00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcad4107de0 0x7fcad40ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad9200700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcad4107de0 0x7fcad40ff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad9200700 1 -- 172.21.15.90:0/2337965700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4108730 msgr2=0x7fcad40ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad9200700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4108730 0x7fcad40ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad9200700 1 -- 172.21.15.90:0/2337965700 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad41a8ff0 con 0x7fcad41034b0 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.908+0000 7fcad9c00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4108730 0x7fcad40ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.909+0000 7fcad9200700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad41034b0 0x7fcad40ff2c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000c00 tx=0x7fcac800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.909+0000 7fcad1600700 1 -- 172.21.15.90:0/2337965700 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8026070 con 0x7fcad41034b0 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.909+0000 7fcad1600700 1 -- 172.21.15.90:0/2337965700 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcac80054c0 con 0x7fcad41034b0 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.909+0000 7fcadbc00700 1 -- 172.21.15.90:0/2337965700 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad41a9280 con 0x7fcad41034b0 2024-05-27T07:51:46.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.909+0000 7fcad1600700 1 -- 172.21.15.90:0/2337965700 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8004c20 con 0x7fcad41034b0 2024-05-27T07:51:46.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.909+0000 7fcadbc00700 1 -- 172.21.15.90:0/2337965700 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41a96d0 con 0x7fcad41034b0 2024-05-27T07:51:46.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.913+0000 7fcad1600700 1 -- 172.21.15.90:0/2337965700 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcac8029070 con 0x7fcad41034b0 2024-05-27T07:51:46.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcaea7dab80 1 Processor -- start 2024-05-27T07:51:46.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcaea7dab80 1 -- start start 2024-05-27T07:51:46.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcad1600700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcabc0b29f0 0x7fcabc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333ccf1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333ccf1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccfe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56333cc1fe40 con 0x56333cc5e9f0 2024-05-27T07:51:46.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56333cc0f9e0 con 0x56333cb72e40 2024-05-27T07:51:46.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56333cc13a20 con 0x56333ccf1e70 2024-05-27T07:51:46.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcad1600700 1 -- 172.21.15.90:0/2337965700 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fcac80e34e0 con 0x7fcad41034b0 2024-05-27T07:51:46.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.914+0000 7fcad9c00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcabc0b29f0 0x7fcabc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333ccf1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333ccf1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52796/0 (socket says 172.21.15.90:52796) 2024-05-27T07:51:46.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9200700 1 -- 172.21.15.90:0/3290030896 learned_addr learned my addr 172.21.15.90:0/3290030896 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:46.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad3e00700 1 --2- 172.21.15.90:0/3290030896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333ccf1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9c00700 1 --2- 172.21.15.90:0/3290030896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccfe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9200700 1 -- 172.21.15.90:0/3290030896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 msgr2=0x56333ccfe110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:46.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9200700 1 --2- 172.21.15.90:0/3290030896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccfe110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9200700 1 -- 172.21.15.90:0/3290030896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 msgr2=0x56333ccf1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:46.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9200700 1 --2- 172.21.15.90:0/3290030896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333ccf1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9200700 1 -- 172.21.15.90:0/3290030896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333cbd8420 con 0x56333cc5e9f0 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9200700 1 --2- 172.21.15.90:0/3290030896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333ccf1330 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcac80aa470 tx=0x7fcac80a9cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c5766514ab3970 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.915+0000 7fcad9c00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcabc0b29f0 0x7fcabc0b4f00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcacc006130 tx=0x7fcacc011970 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.916+0000 7fcac6000700 1 -- 172.21.15.90:0/3290030896 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac80a9ff0 con 0x56333cc5e9f0 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.916+0000 7fcac6000700 1 -- 172.21.15.90:0/3290030896 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcac8005670 con 0x56333cc5e9f0 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.916+0000 7fcac6000700 1 -- 172.21.15.90:0/3290030896 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8080120 con 0x56333cc5e9f0 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.917+0000 7fcaea7dab80 1 -- 172.21.15.90:0/3290030896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 msgr2=0x56333ccf1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.917+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/3290030896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333ccf1330 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcac80aa470 tx=0x7fcac80a9cd0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.917+0000 7fcaea7dab80 1 -- 172.21.15.90:0/3290030896 shutdown_connections 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.917+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/3290030896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccfe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.917+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/3290030896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333ccf1330 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.917+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/3290030896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333ccf1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.917+0000 7fcaea7dab80 1 -- 172.21.15.90:0/3290030896 >> 172.21.15.90:0/3290030896 conn(0x56333cc0c390 msgr2=0x56333cced4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:46.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.917+0000 7fcaea7dab80 1 -- 172.21.15.90:0/3290030896 shutdown_connections 2024-05-27T07:51:46.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.917+0000 7fcaea7dab80 1 -- 172.21.15.90:0/3290030896 wait complete. 2024-05-27T07:51:46.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.917+0000 7fcaea7dab80 1 -- start start 2024-05-27T07:51:46.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcaea7dab80 1 Processor -- start 2024-05-27T07:51:46.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cb72e40 0x56333ccfb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cc5e9f0 0x56333ccfbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccfc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56333cb15fa0 con 0x56333cb72e40 2024-05-27T07:51:46.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56333cbb0760 con 0x56333cc5e9f0 2024-05-27T07:51:46.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56333ccfe940 con 0x56333ccf1e70 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcaea7dab80 10 client.0 _ll_register_callbacks cb 0x56333cbb9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcad9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cb72e40 0x56333ccfb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcad9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cb72e40 0x56333ccfb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52806/0 (socket says 172.21.15.90:52806) 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.918+0000 7fcad9200700 1 -- 172.21.15.90:0/1574606231 learned_addr learned my addr 172.21.15.90:0/1574606231 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcad3e00700 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cc5e9f0 0x56333ccfbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcad9c00700 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccfc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcad9200700 1 -- 172.21.15.90:0/1574606231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 msgr2=0x56333ccfc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcad9200700 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccfc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcad9200700 1 -- 172.21.15.90:0/1574606231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cc5e9f0 msgr2=0x56333ccfbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcad9200700 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cc5e9f0 0x56333ccfbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcad9200700 1 -- 172.21.15.90:0/1574606231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333cb5f8d0 con 0x56333cb72e40 2024-05-27T07:51:46.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcad9c00700 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccfc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcad3e00700 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cc5e9f0 0x56333ccfbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcad9200700 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cb72e40 0x56333ccfb670 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009d10 tx=0x7fcac807d690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b714e0169abaebc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcabaa00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcabaa00700 1 -- 172.21.15.90:0/1574606231 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac80a9ff0 con 0x56333cb72e40 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcabaa00700 1 -- 172.21.15.90:0/1574606231 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fcac80aa950 con 0x56333cb72e40 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.919+0000 7fcaea7dab80 1 -- 172.21.15.90:0/1574606231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac8005670 con 0x56333cb72e40 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcabaa00700 1 -- 172.21.15.90:0/1574606231 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcac80aabe0 con 0x56333cb72e40 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcaea7dab80 10 client.16581 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcaea7dab80 1 -- 172.21.15.90:0/1574606231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56333cbe0060 con 0x56333cb72e40 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcabaa00700 1 -- 172.21.15.90:0/1574606231 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcac8005670 con 0x56333cb72e40 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcabaa00700 1 -- 172.21.15.90:0/1574606231 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fcac80b1bf0 con 0x56333cb72e40 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcaea7dab80 10 client.16581 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:46.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcaea7dab80 10 client.16581 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:51:46.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcaea7dab80 20 client.16581 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:46.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56333cd0b680 0x56333cd0daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcaea7dab80 4 client.16581 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:46.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.920+0000 7fcaea7dab80 1 -- 172.21.15.90:0/1574606231 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56333cc21d90 con 0x56333cd0b680 2024-05-27T07:51:46.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.921+0000 7fcad3e00700 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56333cd0b680 0x56333cd0daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.922+0000 7fcabaa00700 10 client.16581 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:51:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.922+0000 7fcad3e00700 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56333cd0b680 0x56333cd0daa0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c3527925a9dc66f server_cookie=33b259eab591f742 in_seq=0 out_seq=0 2024-05-27T07:51:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcabaa00700 1 -- 172.21.15.90:0/1574606231 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56333cc21d90 con 0x56333cd0b680 2024-05-27T07:51:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcabaa00700 10 client.16581 handle_command_reply: tid=0 2024-05-27T07:51:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcaea7dab80 1 client.16581 shutdown 2024-05-27T07:51:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcaea7dab80 1 -- 172.21.15.90:0/1574606231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cb72e40 msgr2=0x56333ccfb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cb72e40 0x56333ccfb670 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009d10 tx=0x7fcac807d690 comp rx=0 tx=0).stop 2024-05-27T07:51:46.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcaea7dab80 1 -- 172.21.15.90:0/1574606231 shutdown_connections 2024-05-27T07:51:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccfc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cb72e40 0x56333ccfb670 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56333cd0b680 0x56333cd0daa0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/1574606231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cc5e9f0 0x56333ccfbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcabaa00700 0 client.16581 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:51:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.924+0000 7fcaea7dab80 1 -- 172.21.15.90:0/1574606231 >> 172.21.15.90:0/1574606231 conn(0x56333cc0c390 msgr2=0x56333ccf1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.925+0000 7fcaea7dab80 1 -- 172.21.15.90:0/1574606231 shutdown_connections 2024-05-27T07:51:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.925+0000 7fcaea7dab80 1 -- 172.21.15.90:0/1574606231 wait complete. 2024-05-27T07:51:46.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.925+0000 7fcaea7dab80 20 client.16581 trim_cache size 0 max 16384 2024-05-27T07:51:46.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.940+0000 7fcaea7dab80 1 Processor -- start 2024-05-27T07:51:46.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.940+0000 7fcaea7dab80 1 -- start start 2024-05-27T07:51:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333ccfa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333cd079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccf55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b3f40 con 0x56333cc5e9f0 2024-05-27T07:51:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b1d30 con 0x56333cb72e40 2024-05-27T07:51:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b3870 con 0x56333ccf1e70 2024-05-27T07:51:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcad3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333cd079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcad3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333cd079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52812/0 (socket says 172.21.15.90:52812) 2024-05-27T07:51:46.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcad3e00700 1 -- 172.21.15.90:0/4185464304 learned_addr learned my addr 172.21.15.90:0/4185464304 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:46.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcad3e00700 1 -- 172.21.15.90:0/4185464304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 msgr2=0x56333ccf55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:46.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcad3e00700 1 --2- 172.21.15.90:0/4185464304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccf55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcad3e00700 1 -- 172.21.15.90:0/4185464304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 msgr2=0x56333ccfa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcad3e00700 1 --2- 172.21.15.90:0/4185464304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333ccfa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.941+0000 7fcad3e00700 1 -- 172.21.15.90:0/4185464304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56333ccf2800 con 0x56333cc5e9f0 2024-05-27T07:51:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.942+0000 7fcad3e00700 1 --2- 172.21.15.90:0/4185464304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333cd079a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcad41024e0 tx=0x7fcac00058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d93c33b09a5a12 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.942+0000 7fcac5600700 1 -- 172.21.15.90:0/4185464304 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0012cc0 con 0x56333cc5e9f0 2024-05-27T07:51:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.942+0000 7fcac5600700 1 -- 172.21.15.90:0/4185464304 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcac0004cc0 con 0x56333cc5e9f0 2024-05-27T07:51:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.942+0000 7fcac5600700 1 -- 172.21.15.90:0/4185464304 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac001a3f0 con 0x56333cc5e9f0 2024-05-27T07:51:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.943+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4185464304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 msgr2=0x56333cd079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:46.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.943+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/4185464304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333cd079a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcad41024e0 tx=0x7fcac00058e0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.943+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4185464304 shutdown_connections 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.943+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/4185464304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333ccf1e70 0x56333ccf55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.943+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/4185464304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333ccfa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.943+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/4185464304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333cc5e9f0 0x56333cd079a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.943+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4185464304 >> 172.21.15.90:0/4185464304 conn(0x56333cc0c390 msgr2=0x56333ccf4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.943+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4185464304 shutdown_connections 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.943+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4185464304 wait complete. 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.943+0000 7fcaea7dab80 1 -- start start 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.944+0000 7fcaea7dab80 1 Processor -- start 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.944+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333cd04050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.944+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333cc5e9f0 0x56333ccff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.944+0000 7fcaea7dab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333ccf1e70 0x56333ccff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.944+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b2ad0 con 0x56333ccf1e70 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.944+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b2400 con 0x56333cb72e40 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.944+0000 7fcaea7dab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b31a0 con 0x56333cc5e9f0 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.944+0000 7fcaea7dab80 10 client.0 _ll_register_callbacks cb 0x56333cbb9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333ccf1e70 0x56333ccff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333ccf1e70 0x56333ccff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52816/0 (socket says 172.21.15.90:52816) 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad9c00700 1 -- 172.21.15.90:0/4160339855 learned_addr learned my addr 172.21.15.90:0/4160339855 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad3e00700 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333cc5e9f0 0x56333ccff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad9200700 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333cd04050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad9c00700 1 -- 172.21.15.90:0/4160339855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333cc5e9f0 msgr2=0x56333ccff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad9c00700 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333cc5e9f0 0x56333ccff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad9c00700 1 -- 172.21.15.90:0/4160339855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 msgr2=0x56333cd04050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:46.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad9c00700 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333cd04050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad9c00700 1 -- 172.21.15.90:0/4160339855 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56333ccffe30 con 0x56333ccf1e70 2024-05-27T07:51:46.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcad9c00700 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333ccf1e70 0x56333ccff8f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcacc017d50 tx=0x7fcacc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44935f9c3c87c408 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:46.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.945+0000 7fcabaa00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:51:46.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.946+0000 7fcabaa00700 1 -- 172.21.15.90:0/4160339855 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc029060 con 0x56333ccf1e70 2024-05-27T07:51:46.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.946+0000 7fcabaa00700 1 -- 172.21.15.90:0/4160339855 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fcacc016bd0 con 0x56333ccf1e70 2024-05-27T07:51:46.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.946+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4160339855 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcac0004cc0 con 0x56333ccf1e70 2024-05-27T07:51:46.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.946+0000 7fcabaa00700 1 -- 172.21.15.90:0/4160339855 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcacc030050 con 0x56333ccf1e70 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.946+0000 7fcaea7dab80 10 client.16587 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.946+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4160339855 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56333cd75df0 con 0x56333ccf1e70 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.946+0000 7fcabaa00700 1 -- 172.21.15.90:0/4160339855 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcac0004cc0 con 0x56333ccf1e70 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.947+0000 7fcabaa00700 1 -- 172.21.15.90:0/4160339855 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fcacc016040 con 0x56333ccf1e70 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.947+0000 7fcaea7dab80 10 client.16587 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.947+0000 7fcaea7dab80 10 client.16587 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.947+0000 7fcaea7dab80 20 client.16587 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.947+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56333cd76560 0x56333cd78980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.947+0000 7fcaea7dab80 4 client.16587 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.947+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4160339855 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56333cc1b570 con 0x56333cd76560 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.947+0000 7fcad9200700 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56333cd76560 0x56333cd78980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.948+0000 7fcad9200700 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56333cd76560 0x56333cd78980 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c94b33b9ab54b61 server_cookie=155cbb48a0a49405 in_seq=0 out_seq=0 2024-05-27T07:51:46.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.948+0000 7fcabaa00700 10 client.16587 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcabaa00700 1 -- 172.21.15.90:0/4160339855 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56333cc1b570 con 0x56333cd76560 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcabaa00700 10 client.16587 handle_command_reply: tid=0 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcaea7dab80 1 client.16587 shutdown 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4160339855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333ccf1e70 msgr2=0x56333ccff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333ccf1e70 0x56333ccff8f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcacc017d50 tx=0x7fcacc004ad0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4160339855 shutdown_connections 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56333ccf1e70 0x56333ccff8f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56333cd76560 0x56333cd78980 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56333cb72e40 0x56333cd04050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcaea7dab80 1 --2- 172.21.15.90:0/4160339855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56333cc5e9f0 0x56333ccff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4160339855 >> 172.21.15.90:0/4160339855 conn(0x56333cc0c390 msgr2=0x56333ccfc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.949+0000 7fcabaa00700 0 client.16587 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:51:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.950+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4160339855 shutdown_connections 2024-05-27T07:51:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.950+0000 7fcaea7dab80 1 -- 172.21.15.90:0/4160339855 wait complete. 2024-05-27T07:51:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.950+0000 7fcaea7dab80 20 client.16587 trim_cache size 0 max 16384 2024-05-27T07:51:46.952 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 -- 172.21.15.90:0/2337965700 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcabc0b29f0 msgr2=0x7fcabc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcabc0b29f0 0x7fcabc0b4f00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcacc006130 tx=0x7fcacc011970 comp rx=0 tx=0).stop 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 -- 172.21.15.90:0/2337965700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad41034b0 msgr2=0x7fcad40ff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad41034b0 0x7fcad40ff2c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000c00 tx=0x7fcac800f740 comp rx=0 tx=0).stop 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 -- 172.21.15.90:0/2337965700 shutdown_connections 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcabc0b29f0 0x7fcabc0b4f00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcad4108730 0x7fcad40ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcad41034b0 0x7fcad40ff2c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 --2- 172.21.15.90:0/2337965700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcad4107de0 0x7fcad40ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 -- 172.21.15.90:0/2337965700 >> 172.21.15.90:0/2337965700 conn(0x7fcad40fede0 msgr2=0x7fcad4109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 -- 172.21.15.90:0/2337965700 shutdown_connections 2024-05-27T07:51:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:46.951+0000 7fcadbc00700 1 -- 172.21.15.90:0/2337965700 wait complete. 2024-05-27T07:51:48.190 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:47 smithi090 podman[105099]: 2024-05-27 07:51:47.741119219 +0000 UTC m=+3.143358945 container remove e5cd702d818caf5d7702e74bb1eb61637356cab3fdbfc03eba4cc0f5f34e5525 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-27T07:51:48.190 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:47 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:51:48.428 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:51:48.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:48 smithi090 ceph-mon[31429]: pgmap v169: 129 pgs: 129 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 MiB/s wr, 581 op/s 2024-05-27T07:51:48.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:48 smithi136 ceph-mon[35324]: pgmap v169: 129 pgs: 129 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 MiB/s wr, 581 op/s 2024-05-27T07:51:48.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:48 smithi129 ceph-mon[35254]: pgmap v169: 129 pgs: 129 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 MiB/s wr, 581 op/s 2024-05-27T07:51:48.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 90.0 s, 527.4 tps, lat 189.821 ms stddev 109.101 2024-05-27T07:51:48.963 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:48 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-05-27T07:51:48.963 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:48 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.927s CPU time. 2024-05-27T07:51:49.213 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:50.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:50 smithi090 ceph-mon[31429]: pgmap v170: 129 pgs: 129 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 886 op/s 2024-05-27T07:51:50.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:50 smithi136 ceph-mon[35324]: pgmap v170: 129 pgs: 129 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 886 op/s 2024-05-27T07:51:50.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:50 smithi129 ceph-mon[35254]: pgmap v170: 129 pgs: 129 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 886 op/s 2024-05-27T07:51:52.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.055+0000 7fa622800700 1 -- 172.21.15.90:0/2682141129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c119540 msgr2=0x7fa61c11b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:52.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.055+0000 7fa622800700 1 --2- 172.21.15.90:0/2682141129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c119540 0x7fa61c11b960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098f0 tx=0x7fa614009c00 comp rx=0 tx=0).stop 2024-05-27T07:51:52.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.056+0000 7fa622800700 1 -- 172.21.15.90:0/2682141129 shutdown_connections 2024-05-27T07:51:52.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.056+0000 7fa622800700 1 --2- 172.21.15.90:0/2682141129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c119540 0x7fa61c11b960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.056+0000 7fa622800700 1 --2- 172.21.15.90:0/2682141129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa61c10ae80 0x7fa61c10b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.056+0000 7fa622800700 1 --2- 172.21.15.90:0/2682141129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c116be0 0x7fa61c119000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.056+0000 7fa622800700 1 -- 172.21.15.90:0/2682141129 >> 172.21.15.90:0/2682141129 conn(0x7fa61c06f980 msgr2=0x7fa61c06fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:52.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.056+0000 7fa622800700 1 -- 172.21.15.90:0/2682141129 shutdown_connections 2024-05-27T07:51:52.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.057+0000 7fa622800700 1 -- 172.21.15.90:0/2682141129 wait complete. 2024-05-27T07:51:52.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.058+0000 7fa622800700 1 Processor -- start 2024-05-27T07:51:52.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.058+0000 7fa622800700 1 -- start start 2024-05-27T07:51:52.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.058+0000 7fa622800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa61c10ae80 0x7fa61c115410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.058+0000 7fa622800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c116be0 0x7fa61c115950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.058+0000 7fa622800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c119540 0x7fa61c1ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.058+0000 7fa622800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10f700 con 0x7fa61c119540 2024-05-27T07:51:52.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.058+0000 7fa622800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10f580 con 0x7fa61c116be0 2024-05-27T07:51:52.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.058+0000 7fa622800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10f880 con 0x7fa61c10ae80 2024-05-27T07:51:52.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.059+0000 7fa61b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa61c10ae80 0x7fa61c115410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.059+0000 7fa61b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa61c10ae80 0x7fa61c115410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57576/0 (socket says 172.21.15.90:57576) 2024-05-27T07:51:52.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.059+0000 7fa61b400700 1 -- 172.21.15.90:0/1468404685 learned_addr learned my addr 172.21.15.90:0/1468404685 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:52.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.059+0000 7fa61be00700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c119540 0x7fa61c1ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.059+0000 7fa612a00700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c116be0 0x7fa61c115950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.060+0000 7fa61b400700 1 -- 172.21.15.90:0/1468404685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c116be0 msgr2=0x7fa61c115950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:52.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.060+0000 7fa61b400700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c116be0 0x7fa61c115950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.060+0000 7fa61b400700 1 -- 172.21.15.90:0/1468404685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c119540 msgr2=0x7fa61c1ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:52.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.060+0000 7fa61b400700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c119540 0x7fa61c1ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.060+0000 7fa61b400700 1 -- 172.21.15.90:0/1468404685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c1aeaf0 con 0x7fa61c10ae80 2024-05-27T07:51:52.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.060+0000 7fa612a00700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c116be0 0x7fa61c115950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:52.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.061+0000 7fa61b400700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa61c10ae80 0x7fa61c115410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00b530 tx=0x7fa60c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:52.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.061+0000 7fa618c00700 1 -- 172.21.15.90:0/1468404685 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c015020 con 0x7fa61c10ae80 2024-05-27T07:51:52.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.061+0000 7fa618c00700 1 -- 172.21.15.90:0/1468404685 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa60c011cc0 con 0x7fa61c10ae80 2024-05-27T07:51:52.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.061+0000 7fa618c00700 1 -- 172.21.15.90:0/1468404685 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c010d10 con 0x7fa61c10ae80 2024-05-27T07:51:52.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.061+0000 7fa622800700 1 -- 172.21.15.90:0/1468404685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c1aede0 con 0x7fa61c10ae80 2024-05-27T07:51:52.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.061+0000 7fa622800700 1 -- 172.21.15.90:0/1468404685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1af270 con 0x7fa61c10ae80 2024-05-27T07:51:52.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.067+0000 7fa618c00700 1 -- 172.21.15.90:0/1468404685 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa60c018020 con 0x7fa61c10ae80 2024-05-27T07:51:52.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.069+0000 7fa618c00700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa6000b2a70 0x7fa6000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.069+0000 7fa618c00700 1 -- 172.21.15.90:0/1468404685 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fa60c0e2040 con 0x7fa61c10ae80 2024-05-27T07:51:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.070+0000 7fa612a00700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa6000b2a70 0x7fa6000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.071+0000 7fa612a00700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa6000b2a70 0x7fa6000b4f80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa604009fd0 tx=0x7fa60400b040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:52.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.073+0000 7fa631344b80 1 Processor -- start 2024-05-27T07:51:52.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.074+0000 7fa631344b80 1 -- start start 2024-05-27T07:51:52.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.075+0000 7fa631344b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa40ae9f0 0x55cfa4141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.075+0000 7fa631344b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 0x55cfa4141870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.075+0000 7fa631344b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa4141e70 0x55cfa414e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.075+0000 7fa631344b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cfa406fe40 con 0x55cfa40ae9f0 2024-05-27T07:51:52.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.075+0000 7fa631344b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cfa405f9e0 con 0x55cfa4141e70 2024-05-27T07:51:52.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.075+0000 7fa631344b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cfa4063a20 con 0x55cfa3fc2e40 2024-05-27T07:51:52.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.076+0000 7fa612a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 0x55cfa4141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.076+0000 7fa612a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 0x55cfa4141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57582/0 (socket says 172.21.15.90:57582) 2024-05-27T07:51:52.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.076+0000 7fa612a00700 1 -- 172.21.15.90:0/1140172299 learned_addr learned my addr 172.21.15.90:0/1140172299 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:52.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.076+0000 7fa61be00700 1 --2- 172.21.15.90:0/1140172299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa4141e70 0x55cfa414e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.076+0000 7fa61be00700 1 -- 172.21.15.90:0/1140172299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 msgr2=0x55cfa4141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:52.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.076+0000 7fa61be00700 1 --2- 172.21.15.90:0/1140172299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 0x55cfa4141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.076+0000 7fa61be00700 1 -- 172.21.15.90:0/1140172299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa40ae9f0 msgr2=0x55cfa4141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:52.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.076+0000 7fa61be00700 1 --2- 172.21.15.90:0/1140172299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa40ae9f0 0x55cfa4141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.077+0000 7fa61be00700 1 -- 172.21.15.90:0/1140172299 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa4028420 con 0x55cfa4141e70 2024-05-27T07:51:52.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.077+0000 7fa61be00700 1 --2- 172.21.15.90:0/1140172299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa4141e70 0x55cfa414e110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098c0 tx=0x7fa614009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7613abdc32e04cf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:52.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.078+0000 7fa610c00700 1 -- 172.21.15.90:0/1140172299 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61400fc00 con 0x55cfa4141e70 2024-05-27T07:51:52.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.078+0000 7fa610c00700 1 -- 172.21.15.90:0/1140172299 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa61400f8f0 con 0x55cfa4141e70 2024-05-27T07:51:52.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.078+0000 7fa610c00700 1 -- 172.21.15.90:0/1140172299 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140046d0 con 0x55cfa4141e70 2024-05-27T07:51:52.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.079+0000 7fa631344b80 1 -- 172.21.15.90:0/1140172299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa4141e70 msgr2=0x55cfa414e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:52.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.079+0000 7fa631344b80 1 --2- 172.21.15.90:0/1140172299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa4141e70 0x55cfa414e110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098c0 tx=0x7fa614009610 comp rx=0 tx=0).stop 2024-05-27T07:51:52.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.079+0000 7fa631344b80 1 -- 172.21.15.90:0/1140172299 shutdown_connections 2024-05-27T07:51:52.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.079+0000 7fa631344b80 1 --2- 172.21.15.90:0/1140172299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa4141e70 0x55cfa414e110 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.079+0000 7fa631344b80 1 --2- 172.21.15.90:0/1140172299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa40ae9f0 0x55cfa4141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.079+0000 7fa631344b80 1 --2- 172.21.15.90:0/1140172299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 0x55cfa4141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.079+0000 7fa631344b80 1 -- 172.21.15.90:0/1140172299 >> 172.21.15.90:0/1140172299 conn(0x55cfa405c390 msgr2=0x55cfa413d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:52.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.080+0000 7fa631344b80 1 -- 172.21.15.90:0/1140172299 shutdown_connections 2024-05-27T07:51:52.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.081+0000 7fa631344b80 1 -- 172.21.15.90:0/1140172299 wait complete. 2024-05-27T07:51:52.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.081+0000 7fa631344b80 1 -- start start 2024-05-27T07:51:52.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.082+0000 7fa631344b80 1 Processor -- start 2024-05-27T07:51:52.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.084+0000 7fa631344b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 0x55cfa414b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.084+0000 7fa631344b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa414bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.084+0000 7fa631344b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa4141e70 0x55cfa414c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.085+0000 7fa631344b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cfa3f65fa0 con 0x55cfa4141e70 2024-05-27T07:51:52.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.085+0000 7fa631344b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cfa4000760 con 0x55cfa40ae9f0 2024-05-27T07:51:52.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.085+0000 7fa631344b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cfa414eb20 con 0x55cfa3fc2e40 2024-05-27T07:51:52.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.085+0000 7fa631344b80 10 client.0 _ll_register_callbacks cb 0x55cfa4009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:52.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.085+0000 7fa612a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa414bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.085+0000 7fa61b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 0x55cfa414b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.085+0000 7fa612a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa414bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34460/0 (socket says 172.21.15.90:34460) 2024-05-27T07:51:52.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.085+0000 7fa612a00700 1 -- 172.21.15.90:0/1643409656 learned_addr learned my addr 172.21.15.90:0/1643409656 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:52.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.086+0000 7fa61be00700 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa4141e70 0x55cfa414c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.086+0000 7fa612a00700 1 -- 172.21.15.90:0/1643409656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 msgr2=0x55cfa414b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:52.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.086+0000 7fa612a00700 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 0x55cfa414b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.086+0000 7fa612a00700 1 -- 172.21.15.90:0/1643409656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa4141e70 msgr2=0x55cfa414c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:52.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.086+0000 7fa612a00700 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa4141e70 0x55cfa414c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.086+0000 7fa612a00700 1 -- 172.21.15.90:0/1643409656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa3faf8d0 con 0x55cfa40ae9f0 2024-05-27T07:51:52.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.086+0000 7fa61be00700 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa4141e70 0x55cfa414c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:52.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.087+0000 7fa612a00700 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa414bc10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa604015df0 tx=0x7fa604004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8083000daee73bc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:52.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.087+0000 7fa609600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:51:52.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.087+0000 7fa609600700 1 -- 172.21.15.90:0/1643409656 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604026070 con 0x55cfa40ae9f0 2024-05-27T07:51:52.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.087+0000 7fa609600700 1 -- 172.21.15.90:0/1643409656 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fa6040225f0 con 0x55cfa40ae9f0 2024-05-27T07:51:52.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.088+0000 7fa631344b80 1 -- 172.21.15.90:0/1643409656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61400f8f0 con 0x55cfa40ae9f0 2024-05-27T07:51:52.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.088+0000 7fa609600700 1 -- 172.21.15.90:0/1643409656 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa60401c040 con 0x55cfa40ae9f0 2024-05-27T07:51:52.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.089+0000 7fa631344b80 10 client.36092 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:52.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.089+0000 7fa631344b80 1 -- 172.21.15.90:0/1643409656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cfa4030060 con 0x55cfa40ae9f0 2024-05-27T07:51:52.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.089+0000 7fa609600700 1 -- 172.21.15.90:0/1643409656 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa61400f8f0 con 0x55cfa40ae9f0 2024-05-27T07:51:52.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.089+0000 7fa609600700 1 -- 172.21.15.90:0/1643409656 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fa604022b40 con 0x55cfa40ae9f0 2024-05-27T07:51:52.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.089+0000 7fa631344b80 10 client.36092 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:52.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.089+0000 7fa631344b80 10 client.36092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:52.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.089+0000 7fa631344b80 20 client.36092 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:52.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.090+0000 7fa631344b80 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfa415baa0 0x55cfa415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.090+0000 7fa631344b80 4 client.36092 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:52.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.090+0000 7fa631344b80 1 -- 172.21.15.90:0/1643409656 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfa4071d90 con 0x55cfa415baa0 2024-05-27T07:51:52.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.090+0000 7fa61b400700 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfa415baa0 0x55cfa415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.091+0000 7fa61b400700 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfa415baa0 0x55cfa415dec0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74062646af8a7b6 server_cookie=8622db3a1c4461e6 in_seq=0 out_seq=0 2024-05-27T07:51:52.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.091+0000 7fa609600700 10 client.36092 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:52.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.092+0000 7fa609600700 1 -- 172.21.15.90:0/1643409656 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cfa4071d90 con 0x55cfa415baa0 2024-05-27T07:51:52.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.093+0000 7fa609600700 10 client.36092 handle_command_reply: tid=0 2024-05-27T07:51:52.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.093+0000 7fa631344b80 1 client.36092 shutdown 2024-05-27T07:51:52.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.094+0000 7fa631344b80 1 -- 172.21.15.90:0/1643409656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 msgr2=0x55cfa414bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:52.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.094+0000 7fa631344b80 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa414bc10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa604015df0 tx=0x7fa604004ad0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.094+0000 7fa631344b80 1 -- 172.21.15.90:0/1643409656 shutdown_connections 2024-05-27T07:51:52.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.094+0000 7fa631344b80 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfa415baa0 0x55cfa415dec0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.094+0000 7fa631344b80 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa4141e70 0x55cfa414c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.094+0000 7fa631344b80 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa3fc2e40 0x55cfa414b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.094+0000 7fa631344b80 1 --2- 172.21.15.90:0/1643409656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa414bc10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.094+0000 7fa631344b80 1 -- 172.21.15.90:0/1643409656 >> 172.21.15.90:0/1643409656 conn(0x55cfa405c390 msgr2=0x55cfa4141310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:52.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.094+0000 7fa609600700 0 client.36092 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:52.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.095+0000 7fa631344b80 1 -- 172.21.15.90:0/1643409656 shutdown_connections 2024-05-27T07:51:52.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.095+0000 7fa631344b80 1 -- 172.21.15.90:0/1643409656 wait complete. 2024-05-27T07:51:52.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.095+0000 7fa631344b80 20 client.36092 trim_cache size 0 max 16384 2024-05-27T07:51:52.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.126+0000 7fa631344b80 1 Processor -- start 2024-05-27T07:51:52.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.126+0000 7fa631344b80 1 -- start start 2024-05-27T07:51:52.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.126+0000 7fa631344b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa3fc2e40 0x55cfa4146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.126+0000 7fa631344b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa40ae9f0 0x55cfa414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.126+0000 7fa631344b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 0x55cfa4163680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.126+0000 7fa631344b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1c0cc0 con 0x55cfa40ae9f0 2024-05-27T07:51:52.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.126+0000 7fa631344b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1beab0 con 0x55cfa3fc2e40 2024-05-27T07:51:52.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.126+0000 7fa631344b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1c05f0 con 0x55cfa4141e70 2024-05-27T07:51:52.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.127+0000 7fa61b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa3fc2e40 0x55cfa4146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.127+0000 7fa61be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 0x55cfa4163680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.127+0000 7fa61b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa3fc2e40 0x55cfa4146ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34466/0 (socket says 172.21.15.90:34466) 2024-05-27T07:51:52.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.127+0000 7fa61b400700 1 -- 172.21.15.90:0/3859972013 learned_addr learned my addr 172.21.15.90:0/3859972013 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:52.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.127+0000 7fa612a00700 1 --2- 172.21.15.90:0/3859972013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa40ae9f0 0x55cfa414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.128+0000 7fa61be00700 1 -- 172.21.15.90:0/3859972013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa3fc2e40 msgr2=0x55cfa4146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:52.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.128+0000 7fa61be00700 1 --2- 172.21.15.90:0/3859972013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa3fc2e40 0x55cfa4146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.128+0000 7fa61be00700 1 -- 172.21.15.90:0/3859972013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa40ae9f0 msgr2=0x55cfa414a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:52.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.128+0000 7fa61be00700 1 --2- 172.21.15.90:0/3859972013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa40ae9f0 0x55cfa414a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.128+0000 7fa61be00700 1 -- 172.21.15.90:0/3859972013 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa41410b0 con 0x55cfa4141e70 2024-05-27T07:51:52.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.128+0000 7fa612a00700 1 --2- 172.21.15.90:0/3859972013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa40ae9f0 0x55cfa414a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:52.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.129+0000 7fa61be00700 1 --2- 172.21.15.90:0/3859972013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 0x55cfa4163680 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098c0 tx=0x7fa614010da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48bfe7f7bb2e0543 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:52.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.130+0000 7fa60be00700 1 -- 172.21.15.90:0/3859972013 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61402d020 con 0x55cfa4141e70 2024-05-27T07:51:52.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.130+0000 7fa60be00700 1 -- 172.21.15.90:0/3859972013 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa61400f8f0 con 0x55cfa4141e70 2024-05-27T07:51:52.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.130+0000 7fa60be00700 1 -- 172.21.15.90:0/3859972013 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6140046d0 con 0x55cfa4141e70 2024-05-27T07:51:52.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.131+0000 7fa631344b80 1 -- 172.21.15.90:0/3859972013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 msgr2=0x55cfa4163680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:52.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.131+0000 7fa631344b80 1 --2- 172.21.15.90:0/3859972013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 0x55cfa4163680 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6140098c0 tx=0x7fa614010da0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.132+0000 7fa631344b80 1 -- 172.21.15.90:0/3859972013 shutdown_connections 2024-05-27T07:51:52.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.132+0000 7fa631344b80 1 --2- 172.21.15.90:0/3859972013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 0x55cfa4163680 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.132+0000 7fa631344b80 1 --2- 172.21.15.90:0/3859972013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa3fc2e40 0x55cfa4146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.132+0000 7fa631344b80 1 --2- 172.21.15.90:0/3859972013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa40ae9f0 0x55cfa414a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.132+0000 7fa631344b80 1 -- 172.21.15.90:0/3859972013 >> 172.21.15.90:0/3859972013 conn(0x55cfa405c390 msgr2=0x55cfa4149fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:52.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.132+0000 7fa631344b80 1 -- 172.21.15.90:0/3859972013 shutdown_connections 2024-05-27T07:51:52.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.132+0000 7fa631344b80 1 -- 172.21.15.90:0/3859972013 wait complete. 2024-05-27T07:51:52.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.133+0000 7fa631344b80 1 -- start start 2024-05-27T07:51:52.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.133+0000 7fa631344b80 1 Processor -- start 2024-05-27T07:51:52.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.134+0000 7fa631344b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa3fc2e40 0x55cfa4145480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.134+0000 7fa631344b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa41459c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.134+0000 7fa631344b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 0x55cfa4145f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.134+0000 7fa631344b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1bf850 con 0x55cfa3fc2e40 2024-05-27T07:51:52.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.134+0000 7fa631344b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1bf180 con 0x55cfa40ae9f0 2024-05-27T07:51:52.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.134+0000 7fa631344b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1bff20 con 0x55cfa4141e70 2024-05-27T07:51:52.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.134+0000 7fa631344b80 10 client.0 _ll_register_callbacks cb 0x55cfa4009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:52.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.134+0000 7fa61b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa3fc2e40 0x55cfa4145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.135+0000 7fa612a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa41459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.135+0000 7fa612a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa41459c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34476/0 (socket says 172.21.15.90:34476) 2024-05-27T07:51:52.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.135+0000 7fa612a00700 1 -- 172.21.15.90:0/2652179763 learned_addr learned my addr 172.21.15.90:0/2652179763 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:52.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.135+0000 7fa61be00700 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 0x55cfa4145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.136+0000 7fa612a00700 1 -- 172.21.15.90:0/2652179763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 msgr2=0x55cfa4145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:52.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.136+0000 7fa612a00700 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 0x55cfa4145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.136+0000 7fa612a00700 1 -- 172.21.15.90:0/2652179763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa3fc2e40 msgr2=0x55cfa4145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:52.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.136+0000 7fa612a00700 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa3fc2e40 0x55cfa4145480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.136+0000 7fa612a00700 1 -- 172.21.15.90:0/2652179763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa4152db0 con 0x55cfa40ae9f0 2024-05-27T07:51:52.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.137+0000 7fa61b400700 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa3fc2e40 0x55cfa4145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:52.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.137+0000 7fa61be00700 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 0x55cfa4145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:51:52.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.138+0000 7fa612a00700 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa41459c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b010 tx=0x7fa6040043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8990f7d272bab300 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:52.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.138+0000 7fa609600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:51:52.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.139+0000 7fa609600700 1 -- 172.21.15.90:0/2652179763 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604026070 con 0x55cfa40ae9f0 2024-05-27T07:51:52.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.139+0000 7fa609600700 1 -- 172.21.15.90:0/2652179763 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fa6040046f0 con 0x55cfa40ae9f0 2024-05-27T07:51:52.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.140+0000 7fa631344b80 1 -- 172.21.15.90:0/2652179763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa61400f8f0 con 0x55cfa40ae9f0 2024-05-27T07:51:52.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.141+0000 7fa609600700 1 -- 172.21.15.90:0/2652179763 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa604044050 con 0x55cfa40ae9f0 2024-05-27T07:51:52.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.141+0000 7fa631344b80 10 client.36098 fetch_fsmap learned FSMap version 42 2024-05-27T07:51:52.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.141+0000 7fa631344b80 1 -- 172.21.15.90:0/2652179763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cfa4153f80 con 0x55cfa40ae9f0 2024-05-27T07:51:52.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.141+0000 7fa609600700 1 -- 172.21.15.90:0/2652179763 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa61400f8f0 con 0x55cfa40ae9f0 2024-05-27T07:51:52.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.144+0000 7fa609600700 1 -- 172.21.15.90:0/2652179763 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 42) ==== 7757+0+0 (secure 0 0 0) 0x7fa604022760 con 0x55cfa40ae9f0 2024-05-27T07:51:52.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.144+0000 7fa631344b80 10 client.36098 fetch_fsmap finished waiting for FSMap version 42 2024-05-27T07:51:52.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.144+0000 7fa631344b80 10 client.36098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:52.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.144+0000 7fa631344b80 20 client.36098 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:52.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.144+0000 7fa631344b80 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfa4154630 0x55cfa41c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:52.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.144+0000 7fa631344b80 4 client.36098 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:51:52.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.144+0000 7fa631344b80 1 -- 172.21.15.90:0/2652179763 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfa406b570 con 0x55cfa4154630 2024-05-27T07:51:52.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.146+0000 7fa61b400700 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfa4154630 0x55cfa41c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:52.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.147+0000 7fa61b400700 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfa4154630 0x55cfa41c8560 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a1e791e24d5435 server_cookie=8d2a488371569cdb in_seq=0 out_seq=0 2024-05-27T07:51:52.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.147+0000 7fa609600700 10 client.36098 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:52.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.147+0000 7fa609600700 1 -- 172.21.15.90:0/2652179763 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfa406b570 con 0x55cfa4154630 2024-05-27T07:51:52.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.147+0000 7fa609600700 10 client.36098 handle_command_reply: tid=0 2024-05-27T07:51:52.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.147+0000 7fa631344b80 1 client.36098 shutdown 2024-05-27T07:51:52.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.148+0000 7fa631344b80 1 -- 172.21.15.90:0/2652179763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 msgr2=0x55cfa41459c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:52.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.148+0000 7fa631344b80 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa41459c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b010 tx=0x7fa6040043f0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.148+0000 7fa631344b80 1 -- 172.21.15.90:0/2652179763 shutdown_connections 2024-05-27T07:51:52.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.148+0000 7fa631344b80 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfa4154630 0x55cfa41c8560 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.148+0000 7fa631344b80 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfa4141e70 0x55cfa4145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.148+0000 7fa631344b80 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfa3fc2e40 0x55cfa4145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.148+0000 7fa631344b80 1 --2- 172.21.15.90:0/2652179763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfa40ae9f0 0x55cfa41459c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.148+0000 7fa609600700 0 client.36098 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:52.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.148+0000 7fa631344b80 1 -- 172.21.15.90:0/2652179763 >> 172.21.15.90:0/2652179763 conn(0x55cfa405c390 msgr2=0x55cfa4142ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:52.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.149+0000 7fa631344b80 1 -- 172.21.15.90:0/2652179763 shutdown_connections 2024-05-27T07:51:52.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.149+0000 7fa631344b80 1 -- 172.21.15.90:0/2652179763 wait complete. 2024-05-27T07:51:52.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.149+0000 7fa631344b80 20 client.36098 trim_cache size 0 max 16384 2024-05-27T07:51:52.150 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:51:52.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.150+0000 7fa622800700 1 -- 172.21.15.90:0/1468404685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa6000b2a70 msgr2=0x7fa6000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:52.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.150+0000 7fa622800700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa6000b2a70 0x7fa6000b4f80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa604009fd0 tx=0x7fa60400b040 comp rx=0 tx=0).stop 2024-05-27T07:51:52.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.150+0000 7fa622800700 1 -- 172.21.15.90:0/1468404685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa61c10ae80 msgr2=0x7fa61c115410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:52.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.150+0000 7fa622800700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa61c10ae80 0x7fa61c115410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00b530 tx=0x7fa60c00b8f0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.151+0000 7fa622800700 1 -- 172.21.15.90:0/1468404685 shutdown_connections 2024-05-27T07:51:52.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.151+0000 7fa622800700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c119540 0x7fa61c1ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.151+0000 7fa622800700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa61c10ae80 0x7fa61c115410 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.151+0000 7fa622800700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa6000b2a70 0x7fa6000b4f80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.151+0000 7fa622800700 1 --2- 172.21.15.90:0/1468404685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c116be0 0x7fa61c115950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:52.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.151+0000 7fa622800700 1 -- 172.21.15.90:0/1468404685 >> 172.21.15.90:0/1468404685 conn(0x7fa61c06f980 msgr2=0x7fa61c116ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:52.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.151+0000 7fa622800700 1 -- 172.21.15.90:0/1468404685 shutdown_connections 2024-05-27T07:51:52.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:51:52.151+0000 7fa622800700 1 -- 172.21.15.90:0/1468404685 wait complete. 2024-05-27T07:51:52.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:52 smithi090 ceph-mon[31429]: pgmap v171: 129 pgs: 129 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 580 op/s 2024-05-27T07:51:52.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:52 smithi136 ceph-mon[35324]: pgmap v171: 129 pgs: 129 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 580 op/s 2024-05-27T07:51:52.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:52 smithi129 ceph-mon[35254]: pgmap v171: 129 pgs: 129 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 4.8 MiB/s wr, 580 op/s 2024-05-27T07:51:53.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 95.0 s, 618.6 tps, lat 162.551 ms stddev 126.905 2024-05-27T07:51:54.228 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:51:54.228 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:51:54.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:54 smithi090 ceph-mon[31429]: pgmap v172: 129 pgs: 129 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.2 MiB/s wr, 850 op/s 2024-05-27T07:51:54.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:54 smithi136 ceph-mon[35324]: pgmap v172: 129 pgs: 129 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.2 MiB/s wr, 850 op/s 2024-05-27T07:51:54.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:54 smithi129 ceph-mon[35254]: pgmap v172: 129 pgs: 129 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.2 MiB/s wr, 850 op/s 2024-05-27T07:51:54.998 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:51:55.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:55 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:51:55.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:55 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:51:55.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:55 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:51:56.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:56 smithi090 ceph-mon[31429]: pgmap v173: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 692 op/s 2024-05-27T07:51:56.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:56 smithi136 ceph-mon[35324]: pgmap v173: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 692 op/s 2024-05-27T07:51:56.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:56 smithi129 ceph-mon[35254]: pgmap v173: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.0 MiB/s wr, 692 op/s 2024-05-27T07:51:57.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:57 smithi090 ceph-mon[31429]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:51:57.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:57 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:51:57.365 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f2ebb800700 time 2024-05-27T07:51:57.083713+0000 2024-05-27T07:51:57.365 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:51:57.365 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:57.365 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f2ed12c3ba6] 2024-05-27T07:51:57.365 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f2ed12c3dd1] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55cc4178823d] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55cc41773971] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 5: (SnapRealm::check_cache() const+0x13e) [0x55cc41771fa0] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55cc41772651] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55cc41707347] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55cc4140387e] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55cc41408f2f] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55cc41413969] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 11: (MDLog::_replay_thread()+0x1f38) [0x55cc417ac48c] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55cc417b59cb] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 13: (Thread::entry_wrapper()+0x43) [0x7f2ed129272d] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 14: (Thread::_entry_func(void*)+0xd) [0x7f2ed1292749] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2ecfe361ca] 2024-05-27T07:51:57.366 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 16: clone() 2024-05-27T07:51:57.367 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.085+0000 7f2ebb800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f2ebb800700 time 2024-05-27T07:51:57.083713+0000 2024-05-27T07:51:57.367 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:51:57.367 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.367 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:57.367 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f2ed12c3ba6] 2024-05-27T07:51:57.367 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f2ed12c3dd1] 2024-05-27T07:51:57.367 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55cc4178823d] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55cc41773971] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 5: (SnapRealm::check_cache() const+0x13e) [0x55cc41771fa0] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55cc41772651] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55cc41707347] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55cc4140387e] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55cc41408f2f] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55cc41413969] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 11: (MDLog::_replay_thread()+0x1f38) [0x55cc417ac48c] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55cc417b59cb] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 13: (Thread::entry_wrapper()+0x43) [0x7f2ed129272d] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 14: (Thread::_entry_func(void*)+0xd) [0x7f2ed1292749] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2ecfe361ca] 2024-05-27T07:51:57.368 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 16: clone() 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: *** Caught signal (Aborted) ** 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: in thread 7f2ebb800700 thread_name:md_log_replay 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2ecfe40d20] 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2: gsignal() 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 3: abort() 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f2ed12c3ced] 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f2ed12c3dd1] 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55cc4178823d] 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55cc41773971] 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 8: (SnapRealm::check_cache() const+0x13e) [0x55cc41771fa0] 2024-05-27T07:51:57.369 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55cc41772651] 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55cc41707347] 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55cc4140387e] 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55cc41408f2f] 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55cc41413969] 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 14: (MDLog::_replay_thread()+0x1f38) [0x55cc417ac48c] 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55cc417b59cb] 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 16: (Thread::entry_wrapper()+0x43) [0x7f2ed129272d] 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 17: (Thread::_entry_func(void*)+0xd) [0x7f2ed1292749] 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f2ecfe361ca] 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 19: clone() 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.090+0000 7f2ebb800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: in thread 7f2ebb800700 thread_name:md_log_replay 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:57.370 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2ecfe40d20] 2024-05-27T07:51:57.371 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2: gsignal() 2024-05-27T07:51:57.371 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 3: abort() 2024-05-27T07:51:57.371 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f2ed12c3ced] 2024-05-27T07:51:57.371 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f2ed12c3dd1] 2024-05-27T07:51:57.371 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55cc4178823d] 2024-05-27T07:51:57.371 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55cc41773971] 2024-05-27T07:51:57.371 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 8: (SnapRealm::check_cache() const+0x13e) [0x55cc41771fa0] 2024-05-27T07:51:57.371 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55cc41772651] 2024-05-27T07:51:57.371 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55cc41707347] 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55cc4140387e] 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55cc41408f2f] 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55cc41413969] 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 14: (MDLog::_replay_thread()+0x1f38) [0x55cc417ac48c] 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55cc417b59cb] 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 16: (Thread::entry_wrapper()+0x43) [0x7f2ed129272d] 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 17: (Thread::_entry_func(void*)+0xd) [0x7f2ed1292749] 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f2ecfe361ca] 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 19: clone() 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: -1> 2024-05-27T07:51:57.085+0000 7f2ebb800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f2ebb800700 time 2024-05-27T07:51:57.083713+0000 2024-05-27T07:51:57.372 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f2ed12c3ba6] 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f2ed12c3dd1] 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55cc4178823d] 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55cc41773971] 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 5: (SnapRealm::check_cache() const+0x13e) [0x55cc41771fa0] 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55cc41772651] 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55cc41707347] 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55cc4140387e] 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55cc41408f2f] 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55cc41413969] 2024-05-27T07:51:57.373 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 11: (MDLog::_replay_thread()+0x1f38) [0x55cc417ac48c] 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55cc417b59cb] 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 13: (Thread::entry_wrapper()+0x43) [0x7f2ed129272d] 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 14: (Thread::_entry_func(void*)+0xd) [0x7f2ed1292749] 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2ecfe361ca] 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 16: clone() 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 0> 2024-05-27T07:51:57.090+0000 7f2ebb800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: in thread 7f2ebb800700 thread_name:md_log_replay 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2ecfe40d20] 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2: gsignal() 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 3: abort() 2024-05-27T07:51:57.374 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f2ed12c3ced] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f2ed12c3dd1] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55cc4178823d] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55cc41773971] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 8: (SnapRealm::check_cache() const+0x13e) [0x55cc41771fa0] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55cc41772651] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55cc41707347] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55cc4140387e] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55cc41408f2f] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55cc41413969] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 14: (MDLog::_replay_thread()+0x1f38) [0x55cc417ac48c] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55cc417b59cb] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 16: (Thread::entry_wrapper()+0x43) [0x7f2ed129272d] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 17: (Thread::_entry_func(void*)+0xd) [0x7f2ed1292749] 2024-05-27T07:51:57.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f2ecfe361ca] 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 19: clone() 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: -1> 2024-05-27T07:51:57.085+0000 7f2ebb800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f2ebb800700 time 2024-05-27T07:51:57.083713+0000 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f2ed12c3ba6] 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f2ed12c3dd1] 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55cc4178823d] 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55cc41773971] 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 5: (SnapRealm::check_cache() const+0x13e) [0x55cc41771fa0] 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55cc41772651] 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55cc41707347] 2024-05-27T07:51:57.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55cc4140387e] 2024-05-27T07:51:57.377 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55cc41408f2f] 2024-05-27T07:51:57.377 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55cc41413969] 2024-05-27T07:51:57.377 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 11: (MDLog::_replay_thread()+0x1f38) [0x55cc417ac48c] 2024-05-27T07:51:57.377 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55cc417b59cb] 2024-05-27T07:51:57.377 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 13: (Thread::entry_wrapper()+0x43) [0x7f2ed129272d] 2024-05-27T07:51:57.377 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 14: (Thread::_entry_func(void*)+0xd) [0x7f2ed1292749] 2024-05-27T07:51:57.377 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2ecfe361ca] 2024-05-27T07:51:57.377 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 16: clone() 2024-05-27T07:51:57.377 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.377 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 0> 2024-05-27T07:51:57.090+0000 7f2ebb800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: in thread 7f2ebb800700 thread_name:md_log_replay 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2ecfe40d20] 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2: gsignal() 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 3: abort() 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f2ed12c3ced] 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f2ed12c3dd1] 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55cc4178823d] 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55cc41773971] 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 8: (SnapRealm::check_cache() const+0x13e) [0x55cc41771fa0] 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55cc41772651] 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55cc41707347] 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55cc4140387e] 2024-05-27T07:51:57.378 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55cc41408f2f] 2024-05-27T07:51:57.379 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55cc41413969] 2024-05-27T07:51:57.379 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 14: (MDLog::_replay_thread()+0x1f38) [0x55cc417ac48c] 2024-05-27T07:51:57.379 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55cc417b59cb] 2024-05-27T07:51:57.379 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 16: (Thread::entry_wrapper()+0x43) [0x7f2ed129272d] 2024-05-27T07:51:57.379 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 17: (Thread::_entry_func(void*)+0xd) [0x7f2ed1292749] 2024-05-27T07:51:57.379 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f2ecfe361ca] 2024-05-27T07:51:57.379 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 19: clone() 2024-05-27T07:51:57.379 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:51:57.379 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[71015]: 2024-05-27T07:51:57.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:57 smithi136 ceph-mon[35324]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:51:57.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:57 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:51:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:57 smithi129 ceph-mon[35254]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:51:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:57 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:51:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.934+0000 7f8c72400640 1 -- 172.21.15.90:0/3415585416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c6c102720 msgr2=0x7f8c6c106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.934+0000 7f8c72400640 1 --2- 172.21.15.90:0/3415585416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c6c102720 0x7f8c6c106a70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009860 tx=0x7f8c5402a340 comp rx=0 tx=0).stop 2024-05-27T07:51:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.935+0000 7f8c72400640 1 -- 172.21.15.90:0/3415585416 shutdown_connections 2024-05-27T07:51:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.935+0000 7f8c72400640 1 --2- 172.21.15.90:0/3415585416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c6c106fb0 0x7f8c6c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.935+0000 7f8c72400640 1 --2- 172.21.15.90:0/3415585416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c6c102720 0x7f8c6c106a70 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.935+0000 7f8c72400640 1 --2- 172.21.15.90:0/3415585416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c6c06a070 0x7f8c6c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.935+0000 7f8c72400640 1 -- 172.21.15.90:0/3415585416 >> 172.21.15.90:0/3415585416 conn(0x7f8c6c0779d0 msgr2=0x7f8c6c077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.935+0000 7f8c72400640 1 -- 172.21.15.90:0/3415585416 shutdown_connections 2024-05-27T07:51:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.936+0000 7f8c72400640 1 -- 172.21.15.90:0/3415585416 wait complete. 2024-05-27T07:51:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.939+0000 7f8c72400640 1 Processor -- start 2024-05-27T07:51:57.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.940+0000 7f8c72400640 1 -- start start 2024-05-27T07:51:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.940+0000 7f8c72400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c6c06a070 0x7f8c6c1a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.940+0000 7f8c72400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c6c102720 0x7f8c6c1a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.940+0000 7f8c72400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c6c106fb0 0x7f8c6c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.940+0000 7f8c72400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10fb60 con 0x7f8c6c102720 2024-05-27T07:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.940+0000 7f8c72400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10f9e0 con 0x7f8c6c06a070 2024-05-27T07:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.940+0000 7f8c72400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c6c10fce0 con 0x7f8c6c106fb0 2024-05-27T07:51:57.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.941+0000 7f8c63e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c6c102720 0x7f8c6c1a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.941+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c6c06a070 0x7f8c6c1a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.941+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c6c06a070 0x7f8c6c1a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34496/0 (socket says 172.21.15.90:34496) 2024-05-27T07:51:57.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.941+0000 7f8c6b400640 1 -- 172.21.15.90:0/494191728 learned_addr learned my addr 172.21.15.90:0/494191728 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.941+0000 7f8c6be00640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c6c106fb0 0x7f8c6c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.942+0000 7f8c6b400640 1 -- 172.21.15.90:0/494191728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c6c106fb0 msgr2=0x7f8c6c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.942+0000 7f8c6b400640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c6c106fb0 0x7f8c6c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.942+0000 7f8c6b400640 1 -- 172.21.15.90:0/494191728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c6c102720 msgr2=0x7f8c6c1a2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.942+0000 7f8c6b400640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c6c102720 0x7f8c6c1a2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.942+0000 7f8c6b400640 1 -- 172.21.15.90:0/494191728 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6c19cf50 con 0x7f8c6c06a070 2024-05-27T07:51:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.944+0000 7f8c6b400640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c6c06a070 0x7f8c6c1a26a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00ba50 tx=0x7f8c5c00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.944+0000 7f8c68c00640 1 -- 172.21.15.90:0/494191728 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c019070 con 0x7f8c6c06a070 2024-05-27T07:51:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.945+0000 7f8c72400640 1 -- 172.21.15.90:0/494191728 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6c19d240 con 0x7f8c6c06a070 2024-05-27T07:51:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.945+0000 7f8c68c00640 1 -- 172.21.15.90:0/494191728 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c5c00cbe0 con 0x7f8c6c06a070 2024-05-27T07:51:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.945+0000 7f8c72400640 1 -- 172.21.15.90:0/494191728 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c6c112040 con 0x7f8c6c06a070 2024-05-27T07:51:57.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.945+0000 7f8c68c00640 1 -- 172.21.15.90:0/494191728 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c019070 con 0x7f8c6c06a070 2024-05-27T07:51:57.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.953+0000 7f8c68c00640 1 -- 172.21.15.90:0/494191728 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8c5c01c070 con 0x7f8c6c06a070 2024-05-27T07:51:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.955+0000 7f8c79401740 1 Processor -- start 2024-05-27T07:51:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.955+0000 7f8c79401740 1 -- start start 2024-05-27T07:51:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.956+0000 7f8c79401740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99491b0 0x55e4a9ceb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.956+0000 7f8c79401740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.956+0000 7f8c79401740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99284d0 0x55e4a9cec2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.957+0000 7f8c79401740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e4a9c5cb90 con 0x55e4a99491b0 2024-05-27T07:51:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.957+0000 7f8c79401740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e4a9c4c730 con 0x55e4a99284d0 2024-05-27T07:51:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.957+0000 7f8c79401740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e4a9c50770 con 0x55e4a99202f0 2024-05-27T07:51:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.957+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.957+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57626/0 (socket says 172.21.15.90:57626) 2024-05-27T07:51:57.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.957+0000 7f8c6b400640 1 -- 172.21.15.90:0/2117537387 learned_addr learned my addr 172.21.15.90:0/2117537387 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:57.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.957+0000 7f8c63e00640 1 --2- 172.21.15.90:0/2117537387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99284d0 0x55e4a9cec2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.958+0000 7f8c68c00640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8c500b2c90 0x7f8c500b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.958+0000 7f8c68c00640 1 -- 172.21.15.90:0/494191728 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f8c5c0e19a0 con 0x7f8c6c06a070 2024-05-27T07:51:57.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.959+0000 7f8c6be00640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8c500b2c90 0x7f8c500b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.959+0000 7f8c6be00640 1 --2- 172.21.15.90:0/2117537387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99491b0 0x55e4a9ceb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.959+0000 7f8c6b400640 1 -- 172.21.15.90:0/2117537387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99284d0 msgr2=0x55e4a9cec2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.959+0000 7f8c6b400640 1 --2- 172.21.15.90:0/2117537387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99284d0 0x55e4a9cec2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.959+0000 7f8c6b400640 1 -- 172.21.15.90:0/2117537387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99491b0 msgr2=0x55e4a9ceb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:57.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.959+0000 7f8c6b400640 1 --2- 172.21.15.90:0/2117537387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99491b0 0x55e4a9ceb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.960+0000 7f8c6b400640 1 -- 172.21.15.90:0/2117537387 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a9be58b0 con 0x55e4a99202f0 2024-05-27T07:51:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.961+0000 7f8c6b400640 1 --2- 172.21.15.90:0/2117537387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cebda0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0a9710 tx=0x7f8c5c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10822bbfe2043816 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.961+0000 7f8c6be00640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8c500b2c90 0x7f8c500b51a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55e4a9ce89d0 tx=0x7f8c64018040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.961+0000 7f8c60c00640 1 -- 172.21.15.90:0/2117537387 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0a9360 con 0x55e4a99202f0 2024-05-27T07:51:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.961+0000 7f8c60c00640 1 -- 172.21.15.90:0/2117537387 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c5c0a9520 con 0x55e4a99202f0 2024-05-27T07:51:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.961+0000 7f8c60c00640 1 -- 172.21.15.90:0/2117537387 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0a9180 con 0x55e4a99202f0 2024-05-27T07:51:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.963+0000 7f8c79401740 1 -- 172.21.15.90:0/2117537387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 msgr2=0x55e4a9cebda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:57.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.963+0000 7f8c79401740 1 --2- 172.21.15.90:0/2117537387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cebda0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0a9710 tx=0x7f8c5c000f40 comp rx=0 tx=0).stop 2024-05-27T07:51:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.963+0000 7f8c79401740 1 -- 172.21.15.90:0/2117537387 shutdown_connections 2024-05-27T07:51:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.963+0000 7f8c79401740 1 --2- 172.21.15.90:0/2117537387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99284d0 0x55e4a9cec2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.963+0000 7f8c79401740 1 --2- 172.21.15.90:0/2117537387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cebda0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.963+0000 7f8c79401740 1 --2- 172.21.15.90:0/2117537387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99491b0 0x55e4a9ceb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.963+0000 7f8c79401740 1 -- 172.21.15.90:0/2117537387 >> 172.21.15.90:0/2117537387 conn(0x55e4a993d300 msgr2=0x55e4a9ce8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.963+0000 7f8c79401740 1 -- 172.21.15.90:0/2117537387 shutdown_connections 2024-05-27T07:51:57.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.963+0000 7f8c79401740 1 -- 172.21.15.90:0/2117537387 wait complete. 2024-05-27T07:51:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.964+0000 7f8c79401740 1 -- start start 2024-05-27T07:51:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.965+0000 7f8c79401740 1 Processor -- start 2024-05-27T07:51:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.966+0000 7f8c79401740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cf5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.966+0000 7f8c79401740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9cf5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.966+0000 7f8c79401740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 0x55e4a9d01c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.967+0000 7f8c79401740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e4a9932790 con 0x55e4a99284d0 2024-05-27T07:51:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.967+0000 7f8c79401740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e4a9bf4bc0 con 0x55e4a99491b0 2024-05-27T07:51:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.967+0000 7f8c79401740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e4a9cf88d0 con 0x55e4a99202f0 2024-05-27T07:51:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.967+0000 7f8c79401740 10 client.0 _ll_register_callbacks cb 0x55e4a9bd2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.967+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cf5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.967+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cf5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57636/0 (socket says 172.21.15.90:57636) 2024-05-27T07:51:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.967+0000 7f8c6b400640 1 -- 172.21.15.90:0/2116751556 learned_addr learned my addr 172.21.15.90:0/2116751556 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.968+0000 7f8c63e00640 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9cf5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.968+0000 7f8c6be00640 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 0x55e4a9d01c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.968+0000 7f8c6b400640 1 -- 172.21.15.90:0/2116751556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 msgr2=0x55e4a9d01c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.968+0000 7f8c6b400640 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 0x55e4a9d01c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.968+0000 7f8c6b400640 1 -- 172.21.15.90:0/2116751556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 msgr2=0x55e4a9cf5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.968+0000 7f8c6b400640 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9cf5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.968+0000 7f8c6b400640 1 -- 172.21.15.90:0/2116751556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a9961210 con 0x55e4a99202f0 2024-05-27T07:51:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.968+0000 7f8c79401740 1 -- 172.21.15.90:0/2116751556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c5c0a9520 con 0x55e4a99202f0 2024-05-27T07:51:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.969+0000 7f8c63e00640 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9cf5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:57.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.969+0000 7f8c6b400640 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cf5980 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0e1c80 tx=0x7f8c5c0cb2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e18ad1033b186bc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.969+0000 7f8c59600640 10 client.26158 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:51:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.969+0000 7f8c6be00640 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 0x55e4a9d01c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.969+0000 7f8c59600640 1 -- 172.21.15.90:0/2116751556 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0e7020 con 0x55e4a99202f0 2024-05-27T07:51:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.969+0000 7f8c59600640 1 -- 172.21.15.90:0/2116751556 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f8c5c0b2540 con 0x55e4a99202f0 2024-05-27T07:51:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.969+0000 7f8c59600640 1 -- 172.21.15.90:0/2116751556 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8c5c0a9180 con 0x55e4a99202f0 2024-05-27T07:51:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.969+0000 7f8c79401740 10 client.26158 fetch_fsmap learned FSMap version 43 2024-05-27T07:51:57.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.969+0000 7f8c79401740 1 -- 172.21.15.90:0/2116751556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55e4a9a2d620 con 0x55e4a99202f0 2024-05-27T07:51:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.970+0000 7f8c59600640 1 -- 172.21.15.90:0/2116751556 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 43) ==== 7152+0+0 (secure 0 0 0) 0x7f8c5c0a8c70 con 0x55e4a99202f0 2024-05-27T07:51:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.970+0000 7f8c79401740 10 client.26158 fetch_fsmap finished waiting for FSMap version 43 2024-05-27T07:51:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.970+0000 7f8c79401740 10 client.26158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.970+0000 7f8c79401740 20 client.26158 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.970+0000 7f8c79401740 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e4a9907d80 0x55e4a9d07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.970+0000 7f8c79401740 4 client.26158 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:51:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.970+0000 7f8c79401740 1 -- 172.21.15.90:0/2116751556 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4a9c5eae0 con 0x55e4a9907d80 2024-05-27T07:51:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.971+0000 7f8c63e00640 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e4a9907d80 0x55e4a9d07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:57.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.973+0000 7f8c63e00640 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e4a9907d80 0x55e4a9d07980 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb879289352fd7dc server_cookie=b4bd75e3321a44d9 in_seq=0 out_seq=0 2024-05-27T07:51:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.974+0000 7f8c59600640 10 client.26158 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c59600640 1 -- 172.21.15.90:0/2116751556 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e4a9c5eae0 con 0x55e4a9907d80 2024-05-27T07:51:57.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c59600640 10 client.26158 handle_command_reply: tid=0 2024-05-27T07:51:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c79401740 1 client.26158 shutdown 2024-05-27T07:51:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c79401740 1 -- 172.21.15.90:0/2116751556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 msgr2=0x55e4a9cf5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c79401740 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cf5980 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0e1c80 tx=0x7f8c5c0cb2e0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c79401740 1 -- 172.21.15.90:0/2116751556 shutdown_connections 2024-05-27T07:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c79401740 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e4a9907d80 0x55e4a9d07980 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c79401740 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 0x55e4a9d01c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c79401740 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9cf5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c79401740 1 --2- 172.21.15.90:0/2116751556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9cf5980 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c79401740 1 -- 172.21.15.90:0/2116751556 >> 172.21.15.90:0/2116751556 conn(0x55e4a993d300 msgr2=0x55e4a9ceb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.975+0000 7f8c59600640 0 client.26158 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.976+0000 7f8c79401740 1 -- 172.21.15.90:0/2116751556 shutdown_connections 2024-05-27T07:51:57.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.977+0000 7f8c79401740 1 -- 172.21.15.90:0/2116751556 wait complete. 2024-05-27T07:51:57.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:57.978+0000 7f8c79401740 20 client.26158 trim_cache size 0 max 16384 2024-05-27T07:51:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.030+0000 7f8c79401740 1 Processor -- start 2024-05-27T07:51:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.030+0000 7f8c79401740 1 -- start start 2024-05-27T07:51:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.031+0000 7f8c79401740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99202f0 0x55e4a9cf4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.031+0000 7f8c79401740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9d01c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.031+0000 7f8c79401740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99491b0 0x55e4a9cf77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.031+0000 7f8c79401740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4800e910 con 0x55e4a99284d0 2024-05-27T07:51:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.031+0000 7f8c79401740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c480111f0 con 0x55e4a99202f0 2024-05-27T07:51:58.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.031+0000 7f8c79401740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4800efe0 con 0x55e4a99491b0 2024-05-27T07:51:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.032+0000 7f8c6be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99491b0 0x55e4a9cf77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.032+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99202f0 0x55e4a9cf4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.032+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99202f0 0x55e4a9cf4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34524/0 (socket says 172.21.15.90:34524) 2024-05-27T07:51:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.032+0000 7f8c6b400640 1 -- 172.21.15.90:0/453982238 learned_addr learned my addr 172.21.15.90:0/453982238 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.032+0000 7f8c63e00640 1 --2- 172.21.15.90:0/453982238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9d01c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.034+0000 7f8c63e00640 1 -- 172.21.15.90:0/453982238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99491b0 msgr2=0x55e4a9cf77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.034+0000 7f8c63e00640 1 --2- 172.21.15.90:0/453982238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99491b0 0x55e4a9cf77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.035+0000 7f8c63e00640 1 -- 172.21.15.90:0/453982238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99202f0 msgr2=0x55e4a9cf4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.035+0000 7f8c63e00640 1 --2- 172.21.15.90:0/453982238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99202f0 0x55e4a9cf4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.035+0000 7f8c63e00640 1 -- 172.21.15.90:0/453982238 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a9907fb0 con 0x55e4a99284d0 2024-05-27T07:51:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.037+0000 7f8c6be00640 1 --2- 172.21.15.90:0/453982238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99491b0 0x55e4a9cf77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:51:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.037+0000 7f8c6b400640 1 --2- 172.21.15.90:0/453982238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99202f0 0x55e4a9cf4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:51:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.037+0000 7f8c63e00640 1 --2- 172.21.15.90:0/453982238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9d01c00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009780 tx=0x7f8c54009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdbadc5576fad30 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.038+0000 7f8c5be00640 1 -- 172.21.15.90:0/453982238 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54037040 con 0x55e4a99284d0 2024-05-27T07:51:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.039+0000 7f8c5be00640 1 -- 172.21.15.90:0/453982238 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c540307d0 con 0x55e4a99284d0 2024-05-27T07:51:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.039+0000 7f8c5be00640 1 -- 172.21.15.90:0/453982238 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54005610 con 0x55e4a99284d0 2024-05-27T07:51:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.043+0000 7f8c79401740 1 -- 172.21.15.90:0/453982238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 msgr2=0x55e4a9d01c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.043+0000 7f8c79401740 1 --2- 172.21.15.90:0/453982238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9d01c00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009780 tx=0x7f8c54009520 comp rx=0 tx=0).stop 2024-05-27T07:51:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.043+0000 7f8c79401740 1 -- 172.21.15.90:0/453982238 shutdown_connections 2024-05-27T07:51:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.043+0000 7f8c79401740 1 --2- 172.21.15.90:0/453982238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99491b0 0x55e4a9cf77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.043+0000 7f8c79401740 1 --2- 172.21.15.90:0/453982238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9d01c00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.043+0000 7f8c79401740 1 --2- 172.21.15.90:0/453982238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99202f0 0x55e4a9cf4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.043+0000 7f8c79401740 1 -- 172.21.15.90:0/453982238 >> 172.21.15.90:0/453982238 conn(0x55e4a993d300 msgr2=0x55e4a9cf70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.044+0000 7f8c79401740 1 -- 172.21.15.90:0/453982238 shutdown_connections 2024-05-27T07:51:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.044+0000 7f8c79401740 1 -- 172.21.15.90:0/453982238 wait complete. 2024-05-27T07:51:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.046+0000 7f8c79401740 1 -- start start 2024-05-27T07:51:58.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.046+0000 7f8c79401740 1 Processor -- start 2024-05-27T07:51:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.050+0000 7f8c79401740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9d0adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.050+0000 7f8c79401740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9d09170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.050+0000 7f8c79401740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 0x55e4a9d096b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.050+0000 7f8c79401740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4800f6b0 con 0x55e4a99284d0 2024-05-27T07:51:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.050+0000 7f8c79401740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c48010b20 con 0x55e4a99491b0 2024-05-27T07:51:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.050+0000 7f8c79401740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4800fd80 con 0x55e4a99202f0 2024-05-27T07:51:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.051+0000 7f8c79401740 10 client.0 _ll_register_callbacks cb 0x7f8c54035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:51:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.051+0000 7f8c63e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9d09170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.051+0000 7f8c6b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9d0adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.051+0000 7f8c63e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9d09170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57184/0 (socket says 172.21.15.90:57184) 2024-05-27T07:51:58.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.051+0000 7f8c63e00640 1 -- 172.21.15.90:0/3658456614 learned_addr learned my addr 172.21.15.90:0/3658456614 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:51:58.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.052+0000 7f8c6be00640 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 0x55e4a9d096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.053+0000 7f8c6b400640 1 -- 172.21.15.90:0/3658456614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 msgr2=0x55e4a9d096b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.053+0000 7f8c6b400640 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 0x55e4a9d096b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.053+0000 7f8c6b400640 1 -- 172.21.15.90:0/3658456614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 msgr2=0x55e4a9d09170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:51:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.053+0000 7f8c6b400640 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9d09170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.053+0000 7f8c6b400640 1 -- 172.21.15.90:0/3658456614 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a9d09bf0 con 0x55e4a99202f0 2024-05-27T07:51:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.054+0000 7f8c59600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:51:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.054+0000 7f8c6b400640 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9d0adf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00ba20 tx=0x7f8c5c00e070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=662068fc9b193f7d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:51:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.056+0000 7f8c59600640 1 -- 172.21.15.90:0/3658456614 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0e7040 con 0x55e4a99202f0 2024-05-27T07:51:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.056+0000 7f8c59600640 1 -- 172.21.15.90:0/3658456614 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f8c5c0cb2e0 con 0x55e4a99202f0 2024-05-27T07:51:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.057+0000 7f8c79401740 1 -- 172.21.15.90:0/3658456614 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c540307d0 con 0x55e4a99202f0 2024-05-27T07:51:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.058+0000 7f8c59600640 1 -- 172.21.15.90:0/3658456614 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8c5c0a9360 con 0x55e4a99202f0 2024-05-27T07:51:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.058+0000 7f8c79401740 10 client.26161 fetch_fsmap learned FSMap version 43 2024-05-27T07:51:58.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.058+0000 7f8c79401740 1 -- 172.21.15.90:0/3658456614 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e4a9d09bf0 con 0x55e4a99202f0 2024-05-27T07:51:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.058+0000 7f8c59600640 1 -- 172.21.15.90:0/3658456614 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c540307d0 con 0x55e4a99202f0 2024-05-27T07:51:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.059+0000 7f8c59600640 1 -- 172.21.15.90:0/3658456614 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 43) ==== 7152+0+0 (secure 0 0 0) 0x7f8c5c0b4cb0 con 0x55e4a99202f0 2024-05-27T07:51:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.059+0000 7f8c79401740 10 client.26161 fetch_fsmap finished waiting for FSMap version 43 2024-05-27T07:51:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.059+0000 7f8c79401740 10 client.26161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:51:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.059+0000 7f8c79401740 20 client.26161 populate_metadata read hostname 'smithi090' 2024-05-27T07:51:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.059+0000 7f8c79401740 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e4a98dd1c0 0x55e4a9d70d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:51:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.059+0000 7f8c79401740 4 client.26161 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:51:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.059+0000 7f8c79401740 1 -- 172.21.15.90:0/3658456614 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4a9c582c0 con 0x55e4a98dd1c0 2024-05-27T07:51:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.060+0000 7f8c63e00640 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e4a98dd1c0 0x55e4a9d70d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:51:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.062+0000 7f8c63e00640 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e4a98dd1c0 0x55e4a9d70d90 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89286500d15ecbe0 server_cookie=e0de6e92d2e4a6a4 in_seq=0 out_seq=0 2024-05-27T07:51:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.062+0000 7f8c59600640 10 client.26161 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.123+0000 7f8c59600640 1 -- 172.21.15.90:0/3658456614 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e4a9c582c0 con 0x55e4a98dd1c0 2024-05-27T07:51:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.123+0000 7f8c59600640 10 client.26161 handle_command_reply: tid=0 2024-05-27T07:51:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.124+0000 7f8c79401740 1 client.26161 shutdown 2024-05-27T07:51:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.125+0000 7f8c79401740 1 -- 172.21.15.90:0/3658456614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 msgr2=0x55e4a9d0adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.125+0000 7f8c79401740 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9d0adf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00ba20 tx=0x7f8c5c00e070 comp rx=0 tx=0).stop 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.125+0000 7f8c79401740 1 -- 172.21.15.90:0/3658456614 shutdown_connections 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.125+0000 7f8c79401740 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e4a98dd1c0 0x55e4a9d70d90 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.125+0000 7f8c79401740 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4a99491b0 0x55e4a9d096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.125+0000 7f8c79401740 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e4a99284d0 0x55e4a9d09170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.125+0000 7f8c79401740 1 --2- 172.21.15.90:0/3658456614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e4a99202f0 0x55e4a9d0adf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.125+0000 7f8c79401740 1 -- 172.21.15.90:0/3658456614 >> 172.21.15.90:0/3658456614 conn(0x55e4a993d300 msgr2=0x55e4a9cf7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.126+0000 7f8c59600640 0 client.26161 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.127+0000 7f8c79401740 1 -- 172.21.15.90:0/3658456614 shutdown_connections 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.127+0000 7f8c79401740 1 -- 172.21.15.90:0/3658456614 wait complete. 2024-05-27T07:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.128+0000 7f8c79401740 20 client.26161 trim_cache size 0 max 16384 2024-05-27T07:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:51:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.129+0000 7f8c72400640 1 -- 172.21.15.90:0/494191728 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8c500b2c90 msgr2=0x7f8c500b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.129+0000 7f8c72400640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8c500b2c90 0x7f8c500b51a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55e4a9ce89d0 tx=0x7f8c64018040 comp rx=0 tx=0).stop 2024-05-27T07:51:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.129+0000 7f8c72400640 1 -- 172.21.15.90:0/494191728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c6c06a070 msgr2=0x7f8c6c1a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:51:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.129+0000 7f8c72400640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c6c06a070 0x7f8c6c1a26a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00ba50 tx=0x7f8c5c00bf10 comp rx=0 tx=0).stop 2024-05-27T07:51:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.129+0000 7f8c72400640 1 -- 172.21.15.90:0/494191728 shutdown_connections 2024-05-27T07:51:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.129+0000 7f8c72400640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8c500b2c90 0x7f8c500b51a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.129+0000 7f8c72400640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c6c106fb0 0x7f8c6c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.129+0000 7f8c72400640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c6c102720 0x7f8c6c1a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.129+0000 7f8c72400640 1 --2- 172.21.15.90:0/494191728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c6c06a070 0x7f8c6c1a26a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:51:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.129+0000 7f8c72400640 1 -- 172.21.15.90:0/494191728 >> 172.21.15.90:0/494191728 conn(0x7f8c6c0779d0 msgr2=0x7f8c6c100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:51:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.130+0000 7f8c72400640 1 -- 172.21.15.90:0/494191728 shutdown_connections 2024-05-27T07:51:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:51:58.130+0000 7f8c72400640 1 -- 172.21.15.90:0/494191728 wait complete. 2024-05-27T07:51:58.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:51:58 smithi136 ceph-mon[35324]: pgmap v174: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 641 op/s 2024-05-27T07:51:58.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:51:58 smithi129 ceph-mon[35254]: pgmap v174: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 641 op/s 2024-05-27T07:51:58.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:51:58 smithi090 ceph-mon[31429]: pgmap v174: 129 pgs: 129 active+clean; 1.7 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 641 op/s 2024-05-27T07:51:58.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 100.0 s, 531.4 tps, lat 187.545 ms stddev 108.196 2024-05-27T07:51:58.963 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:58 smithi090 podman[105787]: 2024-05-27 07:51:58.837377752 +0000 UTC m=+1.646889686 container died 164daf34dda43fce4b9f3a8a26cc2a57dc18bb8024218ae9224b1e98567eb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-27T07:51:59.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:58 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 2. 2024-05-27T07:51:59.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:58 smithi090 systemd[1]: Stopped Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:51:59.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:58 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.927s CPU time. 2024-05-27T07:51:59.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:51:58 smithi090 systemd[1]: Starting Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:51:59.964 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:59 smithi090 podman[105787]: 2024-05-27 07:51:59.531169576 +0000 UTC m=+2.340681509 container remove 164daf34dda43fce4b9f3a8a26cc2a57dc18bb8024218ae9224b1e98567eb554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:51:59.964 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:51:59 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:52:00.463 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:00 smithi090 podman[105951]: 2024-05-27 07:52:00.150729749 +0000 UTC m=+0.158052691 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:52:00.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:00 smithi090 ceph-mon[31429]: pgmap v175: 129 pgs: 129 active+clean; 1.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.9 MiB/s wr, 912 op/s 2024-05-27T07:52:00.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:00 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:52:00.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:00 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:52:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:00 smithi136 ceph-mon[35324]: pgmap v175: 129 pgs: 129 active+clean; 1.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.9 MiB/s wr, 912 op/s 2024-05-27T07:52:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:00 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:52:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:00 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:52:00.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:00 smithi129 ceph-mon[35254]: pgmap v175: 129 pgs: 129 active+clean; 1.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.9 MiB/s wr, 912 op/s 2024-05-27T07:52:00.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:00 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:52:00.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:00 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:52:01.461 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:01 smithi090 podman[105951]: 2024-05-27 07:52:01.288080641 +0000 UTC m=+1.295403581 container create ad2feb8f6311c193d1a808b96ccaa83d082c256676ebf9929cd4d6e2ecceca58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-27T07:52:01.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:01 smithi090 ceph-mon[31429]: pgmap v176: 129 pgs: 129 active+clean; 1.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.4 MiB/s wr, 607 op/s 2024-05-27T07:52:01.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:01 smithi129 ceph-mon[35254]: pgmap v176: 129 pgs: 129 active+clean; 1.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.4 MiB/s wr, 607 op/s 2024-05-27T07:52:01.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:01 smithi136 ceph-mon[35324]: pgmap v176: 129 pgs: 129 active+clean; 1.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.4 MiB/s wr, 607 op/s 2024-05-27T07:52:01.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:01 smithi090 podman[105951]: 2024-05-27 07:52:01.592741619 +0000 UTC m=+1.600064585 container init ad2feb8f6311c193d1a808b96ccaa83d082c256676ebf9929cd4d6e2ecceca58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:52:01.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:01 smithi090 podman[105951]: 2024-05-27 07:52:01.604300864 +0000 UTC m=+1.611623814 container start ad2feb8f6311c193d1a808b96ccaa83d082c256676ebf9929cd4d6e2ecceca58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T07:52:01.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:01 smithi090 ceph-mds[106074]: -- 172.21.15.90:0/707454673 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558381547a40 con 0x55838158fa80 2024-05-27T07:52:01.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:01 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: starting mds.d at 2024-05-27T07:52:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.390+0000 7f6bc5200700 1 -- 172.21.15.90:0/1749842036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc0106560 msgr2=0x7f6bc0106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.390+0000 7f6bc5200700 1 --2- 172.21.15.90:0/1749842036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc0106560 0x7f6bc0106970 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009990 tx=0x7f6ba8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.390+0000 7f6bc5200700 1 -- 172.21.15.90:0/1749842036 shutdown_connections 2024-05-27T07:52:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.390+0000 7f6bc5200700 1 --2- 172.21.15.90:0/1749842036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc0106eb0 0x7f6bc0113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.390+0000 7f6bc5200700 1 --2- 172.21.15.90:0/1749842036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bc010c5a0 0x7f6bc010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.390+0000 7f6bc5200700 1 --2- 172.21.15.90:0/1749842036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc0106560 0x7f6bc0106970 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.390+0000 7f6bc5200700 1 -- 172.21.15.90:0/1749842036 >> 172.21.15.90:0/1749842036 conn(0x7f6bc0102040 msgr2=0x7f6bc0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.390+0000 7f6bc5200700 1 -- 172.21.15.90:0/1749842036 shutdown_connections 2024-05-27T07:52:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.390+0000 7f6bc5200700 1 -- 172.21.15.90:0/1749842036 wait complete. 2024-05-27T07:52:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.392+0000 7f6bc5200700 1 Processor -- start 2024-05-27T07:52:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.392+0000 7f6bc5200700 1 -- start start 2024-05-27T07:52:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.393+0000 7f6bc5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc0106560 0x7f6bc0111c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.393+0000 7f6bc5200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc0106eb0 0x7f6bc01121b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.393+0000 7f6bc5200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bc010c5a0 0x7f6bc01acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.393+0000 7f6bc5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0078710 con 0x7f6bc0106560 2024-05-27T07:52:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.393+0000 7f6bc5200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0078590 con 0x7f6bc0106eb0 2024-05-27T07:52:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.393+0000 7f6bc5200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0078890 con 0x7f6bc010c5a0 2024-05-27T07:52:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.394+0000 7f6bbe800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bc010c5a0 0x7f6bc01acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.394+0000 7f6bbe800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bc010c5a0 0x7f6bc01acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38324/0 (socket says 172.21.15.90:38324) 2024-05-27T07:52:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.394+0000 7f6bbe800700 1 -- 172.21.15.90:0/194421118 learned_addr learned my addr 172.21.15.90:0/194421118 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.394+0000 7f6bbd400700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc0106eb0 0x7f6bc01121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.394+0000 7f6bbde00700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc0106560 0x7f6bc0111c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.395+0000 7f6bbe800700 1 -- 172.21.15.90:0/194421118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc0106eb0 msgr2=0x7f6bc01121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.395+0000 7f6bbe800700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc0106eb0 0x7f6bc01121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.395+0000 7f6bbe800700 1 -- 172.21.15.90:0/194421118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc0106560 msgr2=0x7f6bc0111c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.395+0000 7f6bbe800700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc0106560 0x7f6bc0111c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.395+0000 7f6bbe800700 1 -- 172.21.15.90:0/194421118 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc01ad530 con 0x7f6bc010c5a0 2024-05-27T07:52:02.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.396+0000 7f6bbe800700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bc010c5a0 0x7f6bc01acff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007d70 tx=0x7f6bb800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.397+0000 7f6bb6a00700 1 -- 172.21.15.90:0/194421118 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8016020 con 0x7f6bc010c5a0 2024-05-27T07:52:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.397+0000 7f6bb6a00700 1 -- 172.21.15.90:0/194421118 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bb800ac90 con 0x7f6bc010c5a0 2024-05-27T07:52:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.397+0000 7f6bc5200700 1 -- 172.21.15.90:0/194421118 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc01ad7c0 con 0x7f6bc010c5a0 2024-05-27T07:52:02.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:02 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:52:02.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:02 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:52:02.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:02 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:52:02.399 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:02 smithi090 bash[105951]: ad2feb8f6311c193d1a808b96ccaa83d082c256676ebf9929cd4d6e2ecceca58 2024-05-27T07:52:02.400 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:02 smithi090 systemd[1]: Started Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:52:02.400 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:02 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-05-27T07:52:02.400 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:02 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.806s CPU time. 2024-05-27T07:52:02.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.398+0000 7f6bb6a00700 1 -- 172.21.15.90:0/194421118 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8011380 con 0x7f6bc010c5a0 2024-05-27T07:52:02.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.398+0000 7f6bc5200700 1 -- 172.21.15.90:0/194421118 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc01adcd0 con 0x7f6bc010c5a0 2024-05-27T07:52:02.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.405+0000 7f6bd3d47b80 1 Processor -- start 2024-05-27T07:52:02.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.405+0000 7f6bb6a00700 1 -- 172.21.15.90:0/194421118 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6bb8019020 con 0x7f6bc010c5a0 2024-05-27T07:52:02.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.405+0000 7f6bd3d47b80 1 -- start start 2024-05-27T07:52:02.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.406+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 0x55d1d7ac33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.406+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7ac38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.406+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ad0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.406+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d1d79f1ec0 con 0x55d1d7944ec0 2024-05-27T07:52:02.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.406+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d1d79e1a60 con 0x55d1d7a30a70 2024-05-27T07:52:02.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.406+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d1d79e5aa0 con 0x55d1d7ac3ef0 2024-05-27T07:52:02.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.406+0000 7f6bbd400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 0x55d1d7ac33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.406+0000 7f6bbd400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ad0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.407+0000 7f6bbd400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 0x55d1d7ac33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:57704/0 (socket says 172.21.15.90:57704) 2024-05-27T07:52:02.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.407+0000 7f6bbd400700 1 -- 172.21.15.90:0/809880098 learned_addr learned my addr 172.21.15.90:0/809880098 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:02.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.407+0000 7f6bbde00700 1 --2- 172.21.15.90:0/809880098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7ac38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.407+0000 7f6bbd400700 1 -- 172.21.15.90:0/809880098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7ac3ef0 msgr2=0x55d1d7ad0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:02.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.407+0000 7f6bbd400700 1 --2- 172.21.15.90:0/809880098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ad0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.407+0000 7f6bbd400700 1 -- 172.21.15.90:0/809880098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 msgr2=0x55d1d7ac38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:02.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.407+0000 7f6bbd400700 1 --2- 172.21.15.90:0/809880098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7ac38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.407+0000 7f6bbd400700 1 -- 172.21.15.90:0/809880098 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d79aa4a0 con 0x55d1d7a30a70 2024-05-27T07:52:02.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.408+0000 7f6bbd400700 1 --2- 172.21.15.90:0/809880098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 0x55d1d7ac33b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009990 tx=0x7f6ba800b2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518b50325c506057 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:02.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.409+0000 7f6baf400700 1 -- 172.21.15.90:0/809880098 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8021040 con 0x55d1d7a30a70 2024-05-27T07:52:02.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.409+0000 7f6baf400700 1 -- 172.21.15.90:0/809880098 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ba80056d0 con 0x55d1d7a30a70 2024-05-27T07:52:02.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.409+0000 7f6baf400700 1 -- 172.21.15.90:0/809880098 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800f460 con 0x55d1d7a30a70 2024-05-27T07:52:02.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.409+0000 7f6bb6a00700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6ba40b2a70 0x7f6ba40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.410+0000 7f6bb6a00700 1 -- 172.21.15.90:0/194421118 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f6bb80e2700 con 0x7f6bc010c5a0 2024-05-27T07:52:02.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.410+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/809880098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 msgr2=0x55d1d7ac33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:02.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.410+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/809880098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 0x55d1d7ac33b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009990 tx=0x7f6ba800b2a0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.410+0000 7f6bbde00700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6ba40b2a70 0x7f6ba40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.411+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/809880098 shutdown_connections 2024-05-27T07:52:02.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.411+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/809880098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ad0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.411+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/809880098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 0x55d1d7ac33b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.411+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/809880098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7ac38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.411+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/809880098 >> 172.21.15.90:0/809880098 conn(0x55d1d79de410 msgr2=0x55d1d7abf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:02.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.412+0000 7f6bbde00700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6ba40b2a70 0x7f6ba40b4f80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55d1d7ac0810 tx=0x7f6bb0009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:02.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.412+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/809880098 shutdown_connections 2024-05-27T07:52:02.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.412+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/809880098 wait complete. 2024-05-27T07:52:02.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.413+0000 7f6bd3d47b80 1 -- start start 2024-05-27T07:52:02.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.413+0000 7f6bd3d47b80 1 Processor -- start 2024-05-27T07:52:02.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.414+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7acd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.414+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7a30a70 0x55d1d7acdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.414+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ace240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.414+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d1d78e8020 con 0x55d1d7944ec0 2024-05-27T07:52:02.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.414+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d1d79827e0 con 0x55d1d7ac3ef0 2024-05-27T07:52:02.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.414+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d1d7ad0ba0 con 0x55d1d7a30a70 2024-05-27T07:52:02.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.414+0000 7f6bd3d47b80 10 client.0 _ll_register_callbacks cb 0x55d1d798bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:02.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.414+0000 7f6bbde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7acd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.415+0000 7f6bbde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7acd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39752/0 (socket says 172.21.15.90:39752) 2024-05-27T07:52:02.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.415+0000 7f6bbde00700 1 -- 172.21.15.90:0/3935439586 learned_addr learned my addr 172.21.15.90:0/3935439586 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:02.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.415+0000 7f6bbd400700 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7a30a70 0x55d1d7acdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.415+0000 7f6bbe800700 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ace240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.416+0000 7f6bbd400700 1 -- 172.21.15.90:0/3935439586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 msgr2=0x55d1d7ace240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:02.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.416+0000 7f6bbd400700 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ace240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.416+0000 7f6bbd400700 1 -- 172.21.15.90:0/3935439586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 msgr2=0x55d1d7acd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:02.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.416+0000 7f6bbd400700 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7acd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.416+0000 7f6bbd400700 1 -- 172.21.15.90:0/3935439586 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d7931950 con 0x55d1d7a30a70 2024-05-27T07:52:02.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.416+0000 7f6bbde00700 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7acd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:02.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.417+0000 7f6bbd400700 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7a30a70 0x55d1d7acdd00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80098e0 tx=0x7f6ba8009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a75d7010b15d8f22 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:02.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.417+0000 7f6ba3e00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:52:02.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.418+0000 7f6ba3e00700 1 -- 172.21.15.90:0/3935439586 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8003680 con 0x55d1d7a30a70 2024-05-27T07:52:02.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.418+0000 7f6ba3e00700 1 -- 172.21.15.90:0/3935439586 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f6ba8003820 con 0x55d1d7a30a70 2024-05-27T07:52:02.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.419+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/3935439586 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba80056d0 con 0x55d1d7a30a70 2024-05-27T07:52:02.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.419+0000 7f6ba3e00700 1 -- 172.21.15.90:0/3935439586 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6ba800b990 con 0x55d1d7a30a70 2024-05-27T07:52:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.420+0000 7f6bd3d47b80 10 client.26179 fetch_fsmap learned FSMap version 44 2024-05-27T07:52:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.420+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/3935439586 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d1d79b20e0 con 0x55d1d7a30a70 2024-05-27T07:52:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.421+0000 7f6ba3e00700 1 -- 172.21.15.90:0/3935439586 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6ba80056d0 con 0x55d1d7a30a70 2024-05-27T07:52:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.422+0000 7f6ba3e00700 1 -- 172.21.15.90:0/3935439586 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 44) ==== 7757+0+0 (secure 0 0 0) 0x7f6ba8028930 con 0x55d1d7a30a70 2024-05-27T07:52:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.422+0000 7f6bd3d47b80 10 client.26179 fetch_fsmap finished waiting for FSMap version 44 2024-05-27T07:52:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.422+0000 7f6bd3d47b80 10 client.26179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.423+0000 7f6bd3d47b80 20 client.26179 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.423+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d1d7add970 0x55d1d7adfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.423+0000 7f6bd3d47b80 4 client.26179 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.423+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/3935439586 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1d79f3e10 con 0x55d1d7add970 2024-05-27T07:52:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.424+0000 7f6bbde00700 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d1d7add970 0x55d1d7adfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.425+0000 7f6ba3e00700 10 client.26179 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.425+0000 7f6bbde00700 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d1d7add970 0x55d1d7adfd90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e48b8fe57b2128f server_cookie=c76b5f6f16e2c021 in_seq=0 out_seq=0 2024-05-27T07:52:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.427+0000 7f6ba3e00700 1 -- 172.21.15.90:0/3935439586 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d1d79f3e10 con 0x55d1d7add970 2024-05-27T07:52:02.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.427+0000 7f6ba3e00700 10 client.26179 handle_command_reply: tid=0 2024-05-27T07:52:02.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.428+0000 7f6bd3d47b80 1 client.26179 shutdown 2024-05-27T07:52:02.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.428+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/3935439586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7a30a70 msgr2=0x55d1d7acdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:02.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.428+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7a30a70 0x55d1d7acdd00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80098e0 tx=0x7f6ba8009290 comp rx=0 tx=0).stop 2024-05-27T07:52:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.428+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/3935439586 shutdown_connections 2024-05-27T07:52:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.428+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d1d7add970 0x55d1d7adfd90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.428+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ace240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.428+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7acd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.428+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/3935439586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7a30a70 0x55d1d7acdd00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.428+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/3935439586 >> 172.21.15.90:0/3935439586 conn(0x55d1d79de410 msgr2=0x55d1d7ac3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.429+0000 7f6ba3e00700 0 client.26179 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.429+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/3935439586 shutdown_connections 2024-05-27T07:52:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.429+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/3935439586 wait complete. 2024-05-27T07:52:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.430+0000 7f6bd3d47b80 20 client.26179 trim_cache size 0 max 16384 2024-05-27T07:52:02.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.469+0000 7f6bd3d47b80 1 Processor -- start 2024-05-27T07:52:02.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.469+0000 7f6bd3d47b80 1 -- start start 2024-05-27T07:52:02.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.469+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7944ec0 0x55d1d7ace1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.469+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7a30a70 0x55d1d7abf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.469+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ad9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.469+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01b85d0 con 0x55d1d7a30a70 2024-05-27T07:52:02.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.469+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01b63c0 con 0x55d1d7ac3ef0 2024-05-27T07:52:02.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.470+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01b7f00 con 0x55d1d7944ec0 2024-05-27T07:52:02.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.470+0000 7f6bbe800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ad9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.471+0000 7f6bbd400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7a30a70 0x55d1d7abf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.471+0000 7f6bbe800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ad9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:57712/0 (socket says 172.21.15.90:57712) 2024-05-27T07:52:02.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.471+0000 7f6bbe800700 1 -- 172.21.15.90:0/197143742 learned_addr learned my addr 172.21.15.90:0/197143742 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:02.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.471+0000 7f6bbde00700 1 --2- 172.21.15.90:0/197143742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7944ec0 0x55d1d7ace1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.472+0000 7f6bbe800700 1 -- 172.21.15.90:0/197143742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7944ec0 msgr2=0x55d1d7ace1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:02.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.472+0000 7f6bbe800700 1 --2- 172.21.15.90:0/197143742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7944ec0 0x55d1d7ace1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.472+0000 7f6bbe800700 1 -- 172.21.15.90:0/197143742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7a30a70 msgr2=0x55d1d7abf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:02.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.472+0000 7f6bbe800700 1 --2- 172.21.15.90:0/197143742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7a30a70 0x55d1d7abf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.472+0000 7f6bbe800700 1 -- 172.21.15.90:0/197143742 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d7ac0080 con 0x55d1d7ac3ef0 2024-05-27T07:52:02.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.472+0000 7f6bbe800700 1 --2- 172.21.15.90:0/197143742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ad9e70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007d40 tx=0x7f6bb80a5040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27fbf46e7ce9dfc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:02.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.473+0000 7f6baea00700 1 -- 172.21.15.90:0/197143742 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80e2560 con 0x55d1d7ac3ef0 2024-05-27T07:52:02.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.473+0000 7f6baea00700 1 -- 172.21.15.90:0/197143742 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bb80cc0d0 con 0x55d1d7ac3ef0 2024-05-27T07:52:02.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.473+0000 7f6baea00700 1 -- 172.21.15.90:0/197143742 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80a99e0 con 0x55d1d7ac3ef0 2024-05-27T07:52:02.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.474+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/197143742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 msgr2=0x55d1d7ad9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:02.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.474+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/197143742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ad9e70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007d40 tx=0x7f6bb80a5040 comp rx=0 tx=0).stop 2024-05-27T07:52:02.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.475+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/197143742 shutdown_connections 2024-05-27T07:52:02.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.475+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/197143742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7ad9e70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.475+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/197143742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7944ec0 0x55d1d7ace1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.475+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/197143742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7a30a70 0x55d1d7abf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.475+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/197143742 >> 172.21.15.90:0/197143742 conn(0x55d1d79de410 msgr2=0x55d1d7ac4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:02.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.476+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/197143742 shutdown_connections 2024-05-27T07:52:02.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.476+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/197143742 wait complete. 2024-05-27T07:52:02.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.476+0000 7f6bd3d47b80 1 -- start start 2024-05-27T07:52:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.476+0000 7f6bd3d47b80 1 Processor -- start 2024-05-27T07:52:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.477+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7ae2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.477+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 0x55d1d7ae3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.477+0000 7f6bd3d47b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7b411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.477+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01b7160 con 0x55d1d7944ec0 2024-05-27T07:52:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.477+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01b6a90 con 0x55d1d7a30a70 2024-05-27T07:52:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.477+0000 7f6bd3d47b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01b7830 con 0x55d1d7ac3ef0 2024-05-27T07:52:02.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.477+0000 7f6bd3d47b80 10 client.0 _ll_register_callbacks cb 0x55d1d798bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:02.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.478+0000 7f6bbde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7ae2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.478+0000 7f6bbe800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7b411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.478+0000 7f6bbd400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 0x55d1d7ae3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.478+0000 7f6bbde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7ae2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39770/0 (socket says 172.21.15.90:39770) 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.478+0000 7f6bbde00700 1 -- 172.21.15.90:0/4159759974 learned_addr learned my addr 172.21.15.90:0/4159759974 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.479+0000 7f6bbde00700 1 -- 172.21.15.90:0/4159759974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7ac3ef0 msgr2=0x55d1d7b411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.479+0000 7f6bbde00700 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7b411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.479+0000 7f6bbde00700 1 -- 172.21.15.90:0/4159759974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 msgr2=0x55d1d7ae3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.479+0000 7f6bbde00700 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 0x55d1d7ae3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.479+0000 7f6bbde00700 1 -- 172.21.15.90:0/4159759974 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d7b43ea0 con 0x55d1d7944ec0 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.479+0000 7f6bbde00700 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7ae2dc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55d1d7acd040 tx=0x7f6bb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460a177de8f192dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.479+0000 7f6ba3e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:52:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.479+0000 7f6ba3e00700 1 -- 172.21.15.90:0/4159759974 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb002e070 con 0x55d1d7944ec0 2024-05-27T07:52:02.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.480+0000 7f6ba3e00700 1 -- 172.21.15.90:0/4159759974 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f6bb0004830 con 0x55d1d7944ec0 2024-05-27T07:52:02.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.480+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/4159759974 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb80cc0d0 con 0x55d1d7944ec0 2024-05-27T07:52:02.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.480+0000 7f6ba3e00700 1 -- 172.21.15.90:0/4159759974 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6bb0037050 con 0x55d1d7944ec0 2024-05-27T07:52:02.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.480+0000 7f6bd3d47b80 10 client.16629 fetch_fsmap learned FSMap version 44 2024-05-27T07:52:02.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.480+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/4159759974 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d1d7b59700 con 0x55d1d7944ec0 2024-05-27T07:52:02.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.481+0000 7f6ba3e00700 1 -- 172.21.15.90:0/4159759974 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6bb80cc0d0 con 0x55d1d7944ec0 2024-05-27T07:52:02.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.481+0000 7f6ba3e00700 1 -- 172.21.15.90:0/4159759974 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 44) ==== 7757+0+0 (secure 0 0 0) 0x7f6bb002a3b0 con 0x55d1d7944ec0 2024-05-27T07:52:02.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.481+0000 7f6bd3d47b80 10 client.16629 fetch_fsmap finished waiting for FSMap version 44 2024-05-27T07:52:02.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.481+0000 7f6bd3d47b80 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:02.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.481+0000 7f6bd3d47b80 20 client.16629 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:02.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.481+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d1d7b59e40 0x55d1d7b5c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:02.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.481+0000 7f6bd3d47b80 4 client.16629 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:52:02.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.481+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/4159759974 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d1d7abe1a0 con 0x55d1d7b59e40 2024-05-27T07:52:02.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.483+0000 7f6bbd400700 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d1d7b59e40 0x55d1d7b5c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:02.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.485+0000 7f6bbd400700 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d1d7b59e40 0x55d1d7b5c260 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329f8506c32fc040 server_cookie=8534e49a92477857 in_seq=0 out_seq=0 2024-05-27T07:52:02.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.485+0000 7f6ba3e00700 10 client.16629 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.489+0000 7f6ba3e00700 1 -- 172.21.15.90:0/4159759974 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d1d7abe1a0 con 0x55d1d7b59e40 2024-05-27T07:52:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.489+0000 7f6ba3e00700 10 client.16629 handle_command_reply: tid=0 2024-05-27T07:52:02.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.489+0000 7f6bd3d47b80 1 client.16629 shutdown 2024-05-27T07:52:02.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.490+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/4159759974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 msgr2=0x55d1d7ae2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:02.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.490+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7ae2dc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55d1d7acd040 tx=0x7f6bb0004750 comp rx=0 tx=0).stop 2024-05-27T07:52:02.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.490+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/4159759974 shutdown_connections 2024-05-27T07:52:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.490+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d1d7b59e40 0x55d1d7b5c260 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.490+0000 7f6ba3e00700 0 client.16629 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.490+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d1d7ac3ef0 0x55d1d7b411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.490+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d1d7944ec0 0x55d1d7ae2dc0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.490+0000 7f6bd3d47b80 1 --2- 172.21.15.90:0/4159759974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1d7a30a70 0x55d1d7ae3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.490+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/4159759974 >> 172.21.15.90:0/4159759974 conn(0x55d1d79de410 msgr2=0x55d1d7acdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:02.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.491+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/4159759974 shutdown_connections 2024-05-27T07:52:02.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.491+0000 7f6bd3d47b80 1 -- 172.21.15.90:0/4159759974 wait complete. 2024-05-27T07:52:02.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.491+0000 7f6bd3d47b80 20 client.16629 trim_cache size 0 max 16384 2024-05-27T07:52:02.493 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:52:02.493 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:52:02.494 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "ef6c6ab6-1c4c-4688-b2b1-0a5b790ea62e", 2024-05-27T07:52:02.494 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:52:02.494 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:52:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.493+0000 7f6bc5200700 1 -- 172.21.15.90:0/194421118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6ba40b2a70 msgr2=0x7f6ba40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.493+0000 7f6bc5200700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6ba40b2a70 0x7f6ba40b4f80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55d1d7ac0810 tx=0x7f6bb0009260 comp rx=0 tx=0).stop 2024-05-27T07:52:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.493+0000 7f6bc5200700 1 -- 172.21.15.90:0/194421118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bc010c5a0 msgr2=0x7f6bc01acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:02.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.493+0000 7f6bc5200700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bc010c5a0 0x7f6bc01acff0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007d70 tx=0x7f6bb800a340 comp rx=0 tx=0).stop 2024-05-27T07:52:02.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.494+0000 7f6bc5200700 1 -- 172.21.15.90:0/194421118 shutdown_connections 2024-05-27T07:52:02.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.494+0000 7f6bc5200700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bc010c5a0 0x7f6bc01acff0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.494+0000 7f6bc5200700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bc0106560 0x7f6bc0111c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.494+0000 7f6bc5200700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6ba40b2a70 0x7f6ba40b4f80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.494+0000 7f6bc5200700 1 --2- 172.21.15.90:0/194421118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6bc0106eb0 0x7f6bc01121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:02.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.494+0000 7f6bc5200700 1 -- 172.21.15.90:0/194421118 >> 172.21.15.90:0/194421118 conn(0x7f6bc0102040 msgr2=0x7f6bc0103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:02.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.495+0000 7f6bc5200700 1 -- 172.21.15.90:0/194421118 shutdown_connections 2024-05-27T07:52:02.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:02.495+0000 7f6bc5200700 1 -- 172.21.15.90:0/194421118 wait complete. 2024-05-27T07:52:02.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:02 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:52:02.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:02 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:52:02.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:02 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:52:02.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:02 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:52:02.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:02 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:52:02.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:02 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:52:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:03 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6834/3381199377,v1:172.21.15.90:6835/3381199377] up:boot 2024-05-27T07:52:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:03 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:52:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:03 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:52:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:03 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:52:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:03 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:03 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:52:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:03 smithi129 ceph-mon[35254]: pgmap v177: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.8 MiB/s wr, 827 op/s 2024-05-27T07:52:03.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:03 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6834/3381199377,v1:172.21.15.90:6835/3381199377] up:boot 2024-05-27T07:52:03.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:03 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:52:03.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:03 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:52:03.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:03 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:52:03.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:03 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:03 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:52:03.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:03 smithi090 ceph-mon[31429]: pgmap v177: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.8 MiB/s wr, 827 op/s 2024-05-27T07:52:03.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:03 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6834/3381199377,v1:172.21.15.90:6835/3381199377] up:boot 2024-05-27T07:52:03.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:03 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:52:03.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:03 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:52:03.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:03 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:52:03.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:03 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:03.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:03 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:52:03.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:03 smithi136 ceph-mon[35324]: pgmap v177: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.8 MiB/s wr, 827 op/s 2024-05-27T07:52:03.787 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:52:03.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 105.0 s, 579.2 tps, lat 172.144 ms stddev 83.004 2024-05-27T07:52:04.527 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:04.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:04 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:04.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:04 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:52:04.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:04 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:04.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:04 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:52:04.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:04 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:04.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:04 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:52:05.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:05 smithi090 ceph-mon[31429]: pgmap v178: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.3 MiB/s wr, 685 op/s 2024-05-27T07:52:05.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:05 smithi136 ceph-mon[35324]: pgmap v178: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.3 MiB/s wr, 685 op/s 2024-05-27T07:52:05.813 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:05 smithi129 ceph-mon[35254]: pgmap v178: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.3 MiB/s wr, 685 op/s 2024-05-27T07:52:08.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:08 smithi090 ceph-mon[31429]: pgmap v179: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 4.6 MiB/s wr, 617 op/s 2024-05-27T07:52:08.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:08 smithi136 ceph-mon[35324]: pgmap v179: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 4.6 MiB/s wr, 617 op/s 2024-05-27T07:52:08.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:08 smithi129 ceph-mon[35254]: pgmap v179: 129 pgs: 129 active+clean; 1.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 4.6 MiB/s wr, 617 op/s 2024-05-27T07:52:08.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 110.0 s, 637.2 tps, lat 157.974 ms stddev 44.796 2024-05-27T07:52:10.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:10 smithi090 ceph-mon[31429]: pgmap v180: 129 pgs: 129 active+clean; 1.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 7.0 MiB/s wr, 935 op/s 2024-05-27T07:52:10.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:10 smithi136 ceph-mon[35324]: pgmap v180: 129 pgs: 129 active+clean; 1.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 7.0 MiB/s wr, 935 op/s 2024-05-27T07:52:10.563 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:10 smithi129 ceph-mon[35254]: pgmap v180: 129 pgs: 129 active+clean; 1.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 7.0 MiB/s wr, 935 op/s 2024-05-27T07:52:11.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.188+0000 7f630f600700 1 -- 172.21.15.90:0/694887616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f630810b2d0 msgr2=0x7f6308117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.188+0000 7f630f600700 1 --2- 172.21.15.90:0/694887616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f630810b2d0 0x7f6308117800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f630000b0a0 tx=0x7f630000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.190+0000 7f630f600700 1 -- 172.21.15.90:0/694887616 shutdown_connections 2024-05-27T07:52:11.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.190+0000 7f630f600700 1 --2- 172.21.15.90:0/694887616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f630810b2d0 0x7f6308117800 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.190+0000 7f630f600700 1 --2- 172.21.15.90:0/694887616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6308110960 0x7f6308110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.190+0000 7f630f600700 1 --2- 172.21.15.90:0/694887616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630810a980 0x7f630810ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.190+0000 7f630f600700 1 -- 172.21.15.90:0/694887616 >> 172.21.15.90:0/694887616 conn(0x7f6308077c70 msgr2=0x7f63080780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:11.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.190+0000 7f630f600700 1 -- 172.21.15.90:0/694887616 shutdown_connections 2024-05-27T07:52:11.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.191+0000 7f630f600700 1 -- 172.21.15.90:0/694887616 wait complete. 2024-05-27T07:52:11.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.192+0000 7f630f600700 1 Processor -- start 2024-05-27T07:52:11.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.193+0000 7f630f600700 1 -- start start 2024-05-27T07:52:11.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.193+0000 7f630f600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630810a980 0x7f63081a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.193+0000 7f630f600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f630810b2d0 0x7f63081a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.194+0000 7f630f600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6308110960 0x7f63081a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.194+0000 7f630f600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f630811aa00 con 0x7f630810b2d0 2024-05-27T07:52:11.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.194+0000 7f630f600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f630811a880 con 0x7f6308110960 2024-05-27T07:52:11.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.194+0000 7f630f600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f630811ab80 con 0x7f630810a980 2024-05-27T07:52:11.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.194+0000 7f630cc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630810a980 0x7f63081a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.194+0000 7f6307e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f630810b2d0 0x7f63081a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.194+0000 7f6307e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f630810b2d0 0x7f63081a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35520/0 (socket says 172.21.15.90:35520) 2024-05-27T07:52:11.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.194+0000 7f6307e00700 1 -- 172.21.15.90:0/2773786205 learned_addr learned my addr 172.21.15.90:0/2773786205 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:11.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.195+0000 7f630d600700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6308110960 0x7f63081a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.195+0000 7f6307e00700 1 -- 172.21.15.90:0/2773786205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630810a980 msgr2=0x7f63081a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.195+0000 7f6307e00700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630810a980 0x7f63081a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.195+0000 7f6307e00700 1 -- 172.21.15.90:0/2773786205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6308110960 msgr2=0x7f63081a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.195+0000 7f6307e00700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6308110960 0x7f63081a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.195+0000 7f6307e00700 1 -- 172.21.15.90:0/2773786205 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63081a62c0 con 0x7f630810b2d0 2024-05-27T07:52:11.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.196+0000 7f6307e00700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f630810b2d0 0x7f63081a1990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f62f400e970 tx=0x7f62f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.197+0000 7f6305600700 1 -- 172.21.15.90:0/2773786205 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4004a30 con 0x7f630810b2d0 2024-05-27T07:52:11.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.197+0000 7f6305600700 1 -- 172.21.15.90:0/2773786205 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62f400cc70 con 0x7f630810b2d0 2024-05-27T07:52:11.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.197+0000 7f630f600700 1 -- 172.21.15.90:0/2773786205 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63081a6550 con 0x7f630810b2d0 2024-05-27T07:52:11.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.197+0000 7f6305600700 1 -- 172.21.15.90:0/2773786205 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4005520 con 0x7f630810b2d0 2024-05-27T07:52:11.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.197+0000 7f630f600700 1 -- 172.21.15.90:0/2773786205 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63081aded0 con 0x7f630810b2d0 2024-05-27T07:52:11.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.206+0000 7f6305600700 1 -- 172.21.15.90:0/2773786205 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f62f400c7e0 con 0x7f630810b2d0 2024-05-27T07:52:11.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.207+0000 7f631e201b80 1 Processor -- start 2024-05-27T07:52:11.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.207+0000 7f631e201b80 1 -- start start 2024-05-27T07:52:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.208+0000 7f631e201b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8519f0 0x56121c8e4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.209+0000 7f6305600700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62f00b2b20 0x7f62f00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.209+0000 7f631e201b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8e4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.209+0000 7f631e201b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8e4e70 0x56121c8f1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.209+0000 7f631e201b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56121c812e40 con 0x56121c765e40 2024-05-27T07:52:11.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.209+0000 7f631e201b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56121c8029e0 con 0x56121c8e4e70 2024-05-27T07:52:11.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.209+0000 7f631e201b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56121c806a20 con 0x56121c8519f0 2024-05-27T07:52:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.209+0000 7f6305600700 1 -- 172.21.15.90:0/2773786205 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f62f40e2a30 con 0x7f630810b2d0 2024-05-27T07:52:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.210+0000 7f630d600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8519f0 0x56121c8e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.210+0000 7f630d600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8519f0 0x56121c8e4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39218/0 (socket says 172.21.15.90:39218) 2024-05-27T07:52:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.210+0000 7f630d600700 1 -- 172.21.15.90:0/3071637772 learned_addr learned my addr 172.21.15.90:0/3071637772 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.210+0000 7f630cc00700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62f00b2b20 0x7f62f00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.210+0000 7f6307e00700 1 --2- 172.21.15.90:0/3071637772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8e4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.210+0000 7f630d600700 1 --2- 172.21.15.90:0/3071637772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8e4e70 0x56121c8f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.211+0000 7f630d600700 1 -- 172.21.15.90:0/3071637772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8e4e70 msgr2=0x56121c8f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.211+0000 7f630d600700 1 --2- 172.21.15.90:0/3071637772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8e4e70 0x56121c8f1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.211+0000 7f630d600700 1 -- 172.21.15.90:0/3071637772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 msgr2=0x56121c8e4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.211+0000 7f630d600700 1 --2- 172.21.15.90:0/3071637772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8e4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.211+0000 7f630d600700 1 -- 172.21.15.90:0/3071637772 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121c7cb420 con 0x56121c8519f0 2024-05-27T07:52:11.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.211+0000 7f630cc00700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62f00b2b20 0x7f62f00b5030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f630807d0e0 tx=0x7f62fc009190 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.211+0000 7f6307e00700 1 --2- 172.21.15.90:0/3071637772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:11.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.212+0000 7f630d600700 1 --2- 172.21.15.90:0/3071637772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8519f0 0x56121c8e4330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f63000095f0 tx=0x7f6300009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af8f4a47127a4d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.214+0000 7f62fa000700 1 -- 172.21.15.90:0/3071637772 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300003a00 con 0x56121c8519f0 2024-05-27T07:52:11.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.214+0000 7f62fa000700 1 -- 172.21.15.90:0/3071637772 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6300003ba0 con 0x56121c8519f0 2024-05-27T07:52:11.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.214+0000 7f62fa000700 1 -- 172.21.15.90:0/3071637772 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630001e6b0 con 0x56121c8519f0 2024-05-27T07:52:11.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.216+0000 7f631e201b80 1 -- 172.21.15.90:0/3071637772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8519f0 msgr2=0x56121c8e4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.216+0000 7f631e201b80 1 --2- 172.21.15.90:0/3071637772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8519f0 0x56121c8e4330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f63000095f0 tx=0x7f6300009d20 comp rx=0 tx=0).stop 2024-05-27T07:52:11.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.217+0000 7f631e201b80 1 -- 172.21.15.90:0/3071637772 shutdown_connections 2024-05-27T07:52:11.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.217+0000 7f631e201b80 1 --2- 172.21.15.90:0/3071637772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8e4e70 0x56121c8f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.217+0000 7f631e201b80 1 --2- 172.21.15.90:0/3071637772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8519f0 0x56121c8e4330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.217+0000 7f631e201b80 1 --2- 172.21.15.90:0/3071637772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.217+0000 7f631e201b80 1 -- 172.21.15.90:0/3071637772 >> 172.21.15.90:0/3071637772 conn(0x56121c7ff390 msgr2=0x56121c8e04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:11.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.217+0000 7f631e201b80 1 -- 172.21.15.90:0/3071637772 shutdown_connections 2024-05-27T07:52:11.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.217+0000 7f631e201b80 1 -- 172.21.15.90:0/3071637772 wait complete. 2024-05-27T07:52:11.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.218+0000 7f631e201b80 1 -- start start 2024-05-27T07:52:11.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.219+0000 7f631e201b80 1 Processor -- start 2024-05-27T07:52:11.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.221+0000 7f631e201b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c765e40 0x56121c8ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.221+0000 7f631e201b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c8519f0 0x56121c8eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.222+0000 7f631e201b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8ef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.221+0000 7f630cc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c765e40 0x56121c8ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.222+0000 7f630cc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c765e40 0x56121c8ee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:44748/0 (socket says 172.21.15.90:44748) 2024-05-27T07:52:11.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.222+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56121c708fa0 con 0x56121c8519f0 2024-05-27T07:52:11.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.222+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56121c7a3760 con 0x56121c765e40 2024-05-27T07:52:11.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.222+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56121c8f1ac0 con 0x56121c8e4e70 2024-05-27T07:52:11.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.222+0000 7f630cc00700 1 -- 172.21.15.90:0/1814327500 learned_addr learned my addr 172.21.15.90:0/1814327500 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:11.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.222+0000 7f631e201b80 10 client.0 _ll_register_callbacks cb 0x56121c7acc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:11.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.222+0000 7f630d600700 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8ef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.223+0000 7f6307e00700 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c8519f0 0x56121c8eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.223+0000 7f630cc00700 1 -- 172.21.15.90:0/1814327500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 msgr2=0x56121c8ef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.223+0000 7f630cc00700 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8ef0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.223+0000 7f630cc00700 1 -- 172.21.15.90:0/1814327500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c8519f0 msgr2=0x56121c8eeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.223+0000 7f630cc00700 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c8519f0 0x56121c8eeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.223+0000 7f630cc00700 1 -- 172.21.15.90:0/1814327500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121c7528d0 con 0x56121c765e40 2024-05-27T07:52:11.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.224+0000 7f630d600700 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8ef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:11.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.224+0000 7f6307e00700 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c8519f0 0x56121c8eeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:11.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.225+0000 7f630cc00700 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c765e40 0x56121c8ee620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62fc015700 tx=0x7f62fc004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55776b67507c60af server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.225+0000 7f62eea00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:52:11.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.226+0000 7f62eea00700 1 -- 172.21.15.90:0/1814327500 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc023020 con 0x56121c765e40 2024-05-27T07:52:11.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.226+0000 7f62eea00700 1 -- 172.21.15.90:0/1814327500 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f62fc015b40 con 0x56121c765e40 2024-05-27T07:52:11.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.226+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6300003ba0 con 0x56121c765e40 2024-05-27T07:52:11.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.227+0000 7f62eea00700 1 -- 172.21.15.90:0/1814327500 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f62fc01b040 con 0x56121c765e40 2024-05-27T07:52:11.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.227+0000 7f631e201b80 10 client.36146 fetch_fsmap learned FSMap version 44 2024-05-27T07:52:11.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.227+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56121c7d3060 con 0x56121c765e40 2024-05-27T07:52:11.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.228+0000 7f62eea00700 1 -- 172.21.15.90:0/1814327500 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6300003ba0 con 0x56121c765e40 2024-05-27T07:52:11.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.228+0000 7f62eea00700 1 -- 172.21.15.90:0/1814327500 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 44) ==== 7757+0+0 (secure 0 0 0) 0x7f62fc01e030 con 0x56121c765e40 2024-05-27T07:52:11.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.228+0000 7f631e201b80 10 client.36146 fetch_fsmap finished waiting for FSMap version 44 2024-05-27T07:52:11.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.228+0000 7f631e201b80 10 client.36146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:11.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.228+0000 7f631e201b80 20 client.36146 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:11.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.228+0000 7f631e201b80 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56121c8fe990 0x56121c900df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.228+0000 7f631e201b80 4 client.36146 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:11.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.228+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56121c814d90 con 0x56121c8fe990 2024-05-27T07:52:11.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.229+0000 7f6307e00700 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56121c8fe990 0x56121c900df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.230+0000 7f62eea00700 10 client.36146 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:11.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.231+0000 7f6307e00700 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56121c8fe990 0x56121c900df0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85ab4261c67b6956 server_cookie=c508c2de70438a6a in_seq=0 out_seq=0 2024-05-27T07:52:11.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.233+0000 7f62eea00700 1 -- 172.21.15.90:0/1814327500 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56121c814d90 con 0x56121c8fe990 2024-05-27T07:52:11.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.233+0000 7f62eea00700 10 client.36146 handle_command_reply: tid=0 2024-05-27T07:52:11.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.235+0000 7f631e201b80 1 client.36146 shutdown 2024-05-27T07:52:11.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.235+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c765e40 msgr2=0x56121c8ee620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.235+0000 7f631e201b80 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c765e40 0x56121c8ee620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62fc015700 tx=0x7f62fc004860 comp rx=0 tx=0).stop 2024-05-27T07:52:11.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.236+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 shutdown_connections 2024-05-27T07:52:11.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.236+0000 7f631e201b80 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56121c8fe990 0x56121c900df0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.236+0000 7f631e201b80 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8ef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.236+0000 7f631e201b80 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c765e40 0x56121c8ee620 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.236+0000 7f631e201b80 1 --2- 172.21.15.90:0/1814327500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c8519f0 0x56121c8eeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.236+0000 7f62eea00700 0 client.36146 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:11.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.236+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 >> 172.21.15.90:0/1814327500 conn(0x56121c7ff390 msgr2=0x56121c8e4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:11.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.237+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 shutdown_connections 2024-05-27T07:52:11.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.237+0000 7f631e201b80 1 -- 172.21.15.90:0/1814327500 wait complete. 2024-05-27T07:52:11.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.237+0000 7f631e201b80 20 client.36146 trim_cache size 0 max 16384 2024-05-27T07:52:11.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.278+0000 7f631e201b80 1 Processor -- start 2024-05-27T07:52:11.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.278+0000 7f631e201b80 1 -- start start 2024-05-27T07:52:11.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.278+0000 7f631e201b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8ed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.278+0000 7f631e201b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 0x56121c8e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.278+0000 7f631e201b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8fabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.278+0000 7f631e201b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f62e8010850 con 0x56121c765e40 2024-05-27T07:52:11.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.278+0000 7f631e201b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62e800e640 con 0x56121c8519f0 2024-05-27T07:52:11.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.278+0000 7f631e201b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f62e8010180 con 0x56121c8e4e70 2024-05-27T07:52:11.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.279+0000 7f630cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.279+0000 7f6307e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 0x56121c8e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.279+0000 7f630cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8ed580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35564/0 (socket says 172.21.15.90:35564) 2024-05-27T07:52:11.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.279+0000 7f630cc00700 1 -- 172.21.15.90:0/2014325031 learned_addr learned my addr 172.21.15.90:0/2014325031 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:11.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.279+0000 7f630d600700 1 --2- 172.21.15.90:0/2014325031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8fabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.280+0000 7f630cc00700 1 -- 172.21.15.90:0/2014325031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 msgr2=0x56121c8fabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.280+0000 7f630cc00700 1 --2- 172.21.15.90:0/2014325031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8fabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.280+0000 7f630cc00700 1 -- 172.21.15.90:0/2014325031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 msgr2=0x56121c8e9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.280+0000 7f630cc00700 1 --2- 172.21.15.90:0/2014325031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 0x56121c8e9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.280+0000 7f630cc00700 1 -- 172.21.15.90:0/2014325031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121c8e0620 con 0x56121c765e40 2024-05-27T07:52:11.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.280+0000 7f6307e00700 1 --2- 172.21.15.90:0/2014325031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 0x56121c8e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:11.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.281+0000 7f630cc00700 1 --2- 172.21.15.90:0/2014325031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8ed580 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f62fc013dc0 tx=0x7f62fc009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f37498810a17cd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.282+0000 7f62f9600700 1 -- 172.21.15.90:0/2014325031 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc023070 con 0x56121c765e40 2024-05-27T07:52:11.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.282+0000 7f62f9600700 1 -- 172.21.15.90:0/2014325031 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62fc037070 con 0x56121c765e40 2024-05-27T07:52:11.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.282+0000 7f62f9600700 1 -- 172.21.15.90:0/2014325031 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc013450 con 0x56121c765e40 2024-05-27T07:52:11.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.283+0000 7f631e201b80 1 -- 172.21.15.90:0/2014325031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 msgr2=0x56121c8ed580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.283+0000 7f631e201b80 1 --2- 172.21.15.90:0/2014325031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8ed580 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f62fc013dc0 tx=0x7f62fc009200 comp rx=0 tx=0).stop 2024-05-27T07:52:11.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.284+0000 7f631e201b80 1 -- 172.21.15.90:0/2014325031 shutdown_connections 2024-05-27T07:52:11.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.284+0000 7f631e201b80 1 --2- 172.21.15.90:0/2014325031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8fabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.284+0000 7f631e201b80 1 --2- 172.21.15.90:0/2014325031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8ed580 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.284+0000 7f631e201b80 1 --2- 172.21.15.90:0/2014325031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 0x56121c8e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.284+0000 7f631e201b80 1 -- 172.21.15.90:0/2014325031 >> 172.21.15.90:0/2014325031 conn(0x56121c7ff390 msgr2=0x56121c8ee5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:11.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.284+0000 7f631e201b80 1 -- 172.21.15.90:0/2014325031 shutdown_connections 2024-05-27T07:52:11.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.284+0000 7f631e201b80 1 -- 172.21.15.90:0/2014325031 wait complete. 2024-05-27T07:52:11.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.285+0000 7f631e201b80 1 -- start start 2024-05-27T07:52:11.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.285+0000 7f631e201b80 1 Processor -- start 2024-05-27T07:52:11.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.286+0000 7f631e201b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.286+0000 7f631e201b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 0x56121c8e8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.286+0000 7f631e201b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8e8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.286+0000 7f631e201b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f62e800f3e0 con 0x56121c765e40 2024-05-27T07:52:11.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.286+0000 7f631e201b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62e800ed10 con 0x56121c8519f0 2024-05-27T07:52:11.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.286+0000 7f631e201b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f62e800fab0 con 0x56121c8e4e70 2024-05-27T07:52:11.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.286+0000 7f631e201b80 10 client.0 _ll_register_callbacks cb 0x56121c7acc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:11.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.287+0000 7f630cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.287+0000 7f630cc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8f7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35570/0 (socket says 172.21.15.90:35570) 2024-05-27T07:52:11.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.287+0000 7f630cc00700 1 -- 172.21.15.90:0/3169258058 learned_addr learned my addr 172.21.15.90:0/3169258058 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:11.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.288+0000 7f630cc00700 1 -- 172.21.15.90:0/3169258058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 msgr2=0x56121c8e8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.288+0000 7f6307e00700 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 0x56121c8e8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.288+0000 7f630cc00700 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8e8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.288+0000 7f630cc00700 1 -- 172.21.15.90:0/3169258058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 msgr2=0x56121c8e8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.288+0000 7f630cc00700 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 0x56121c8e8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.288+0000 7f630cc00700 1 -- 172.21.15.90:0/3169258058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121c8e94d0 con 0x56121c765e40 2024-05-27T07:52:11.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.288+0000 7f6307e00700 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 0x56121c8e8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:11.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.290+0000 7f630cc00700 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8f7130 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f62fc013f30 tx=0x7f62fc015e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9737d21946d0b30 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.290+0000 7f62eea00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:52:11.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.290+0000 7f62eea00700 1 -- 172.21.15.90:0/3169258058 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc023070 con 0x56121c765e40 2024-05-27T07:52:11.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.291+0000 7f631e201b80 1 -- 172.21.15.90:0/3169258058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62fc037070 con 0x56121c765e40 2024-05-27T07:52:11.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.291+0000 7f62eea00700 1 -- 172.21.15.90:0/3169258058 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f62fc01fac0 con 0x56121c765e40 2024-05-27T07:52:11.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.292+0000 7f62eea00700 1 -- 172.21.15.90:0/3169258058 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f62fc04a050 con 0x56121c765e40 2024-05-27T07:52:11.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.293+0000 7f631e201b80 10 client.16641 fetch_fsmap learned FSMap version 44 2024-05-27T07:52:11.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.293+0000 7f631e201b80 1 -- 172.21.15.90:0/3169258058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56121c8f6a00 con 0x56121c765e40 2024-05-27T07:52:11.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.294+0000 7f62eea00700 1 -- 172.21.15.90:0/3169258058 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62fc037070 con 0x56121c765e40 2024-05-27T07:52:11.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.294+0000 7f62eea00700 1 -- 172.21.15.90:0/3169258058 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 44) ==== 7757+0+0 (secure 0 0 0) 0x7f62fc01e040 con 0x56121c765e40 2024-05-27T07:52:11.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.294+0000 7f631e201b80 10 client.16641 fetch_fsmap finished waiting for FSMap version 44 2024-05-27T07:52:11.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.294+0000 7f631e201b80 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:11.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.294+0000 7f631e201b80 20 client.16641 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:11.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.294+0000 7f631e201b80 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56121c969660 0x56121c96ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.294+0000 7f631e201b80 4 client.16641 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:52:11.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.294+0000 7f631e201b80 1 -- 172.21.15.90:0/3169258058 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56121c80e570 con 0x56121c969660 2024-05-27T07:52:11.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.295+0000 7f6307e00700 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56121c969660 0x56121c96ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.296+0000 7f6307e00700 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56121c969660 0x56121c96ba80 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf87fe87f4bc0b5 server_cookie=f648475e2d6dcb94 in_seq=0 out_seq=0 2024-05-27T07:52:11.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.296+0000 7f62eea00700 10 client.16641 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:11.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.296+0000 7f62eea00700 1 -- 172.21.15.90:0/3169258058 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56121c80e570 con 0x56121c969660 2024-05-27T07:52:11.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.297+0000 7f62eea00700 10 client.16641 handle_command_reply: tid=0 2024-05-27T07:52:11.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.297+0000 7f631e201b80 1 client.16641 shutdown 2024-05-27T07:52:11.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.298+0000 7f631e201b80 1 -- 172.21.15.90:0/3169258058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 msgr2=0x56121c8f7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.298+0000 7f631e201b80 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8f7130 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f62fc013f30 tx=0x7f62fc015e30 comp rx=0 tx=0).stop 2024-05-27T07:52:11.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.298+0000 7f631e201b80 1 -- 172.21.15.90:0/3169258058 shutdown_connections 2024-05-27T07:52:11.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.298+0000 7f631e201b80 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56121c969660 0x56121c96ba80 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.298+0000 7f631e201b80 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121c8e4e70 0x56121c8e8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.298+0000 7f631e201b80 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56121c765e40 0x56121c8f7130 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.298+0000 7f631e201b80 1 --2- 172.21.15.90:0/3169258058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56121c8519f0 0x56121c8e8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.298+0000 7f631e201b80 1 -- 172.21.15.90:0/3169258058 >> 172.21.15.90:0/3169258058 conn(0x56121c7ff390 msgr2=0x56121c8e5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.298+0000 7f62eea00700 0 client.16641 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.299+0000 7f631e201b80 1 -- 172.21.15.90:0/3169258058 shutdown_connections 2024-05-27T07:52:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.300+0000 7f631e201b80 1 -- 172.21.15.90:0/3169258058 wait complete. 2024-05-27T07:52:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.300+0000 7f631e201b80 20 client.16641 trim_cache size 0 max 16384 2024-05-27T07:52:11.302 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:52:11.302 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:52:11.302 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:52:11.302 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:52:11.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.302+0000 7f630f600700 1 -- 172.21.15.90:0/2773786205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62f00b2b20 msgr2=0x7f62f00b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.302+0000 7f630f600700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62f00b2b20 0x7f62f00b5030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f630807d0e0 tx=0x7f62fc009190 comp rx=0 tx=0).stop 2024-05-27T07:52:11.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.302+0000 7f630f600700 1 -- 172.21.15.90:0/2773786205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f630810b2d0 msgr2=0x7f63081a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.302+0000 7f630f600700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f630810b2d0 0x7f63081a1990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f62f400e970 tx=0x7f62f400ec80 comp rx=0 tx=0).stop 2024-05-27T07:52:11.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.302+0000 7f630f600700 1 -- 172.21.15.90:0/2773786205 shutdown_connections 2024-05-27T07:52:11.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.302+0000 7f630f600700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62f00b2b20 0x7f62f00b5030 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.302+0000 7f630f600700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6308110960 0x7f63081a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.302+0000 7f630f600700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f630810a980 0x7f63081a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.302+0000 7f630f600700 1 --2- 172.21.15.90:0/2773786205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f630810b2d0 0x7f63081a1990 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.302+0000 7f630f600700 1 -- 172.21.15.90:0/2773786205 >> 172.21.15.90:0/2773786205 conn(0x7f6308077c70 msgr2=0x7f630807b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.303+0000 7f630f600700 1 -- 172.21.15.90:0/2773786205 shutdown_connections 2024-05-27T07:52:11.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:11.303+0000 7f630f600700 1 -- 172.21.15.90:0/2773786205 wait complete. 2024-05-27T07:52:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.502+0000 7f2616800640 1 -- 172.21.15.90:0/1764637520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2610104240 msgr2=0x7f2610110610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.502+0000 7f2616800640 1 --2- 172.21.15.90:0/1764637520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2610104240 0x7f2610110610 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2608009ca0 tx=0x7f260802e9a0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.503+0000 7f2616800640 1 -- 172.21.15.90:0/1764637520 shutdown_connections 2024-05-27T07:52:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.503+0000 7f2616800640 1 --2- 172.21.15.90:0/1764637520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2610109f90 0x7f2610103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.503+0000 7f2616800640 1 --2- 172.21.15.90:0/1764637520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2610104240 0x7f2610110610 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.503+0000 7f2616800640 1 --2- 172.21.15.90:0/1764637520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610109af0 0x7f2610103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.503+0000 7f2616800640 1 -- 172.21.15.90:0/1764637520 >> 172.21.15.90:0/1764637520 conn(0x7f2610079250 msgr2=0x7f2610079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.503+0000 7f2616800640 1 -- 172.21.15.90:0/1764637520 shutdown_connections 2024-05-27T07:52:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.503+0000 7f2616800640 1 -- 172.21.15.90:0/1764637520 wait complete. 2024-05-27T07:52:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.505+0000 7f2616800640 1 Processor -- start 2024-05-27T07:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.506+0000 7f2616800640 1 -- start start 2024-05-27T07:52:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.507+0000 7f2616800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610104240 0x7f2610198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.507+0000 7f2616800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2610109af0 0x7f26101988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.507+0000 7f2616800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2610109f90 0x7f261019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.507+0000 7f2616800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2610111a00 con 0x7f2610109af0 2024-05-27T07:52:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.508+0000 7f2616800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2610111880 con 0x7f2610109f90 2024-05-27T07:52:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.507+0000 7f260f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610104240 0x7f2610198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.508+0000 7f260f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610104240 0x7f2610198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39264/0 (socket says 172.21.15.90:39264) 2024-05-27T07:52:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.508+0000 7f260f400640 1 -- 172.21.15.90:0/742672323 learned_addr learned my addr 172.21.15.90:0/742672323 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.508+0000 7f2616800640 1 -- 172.21.15.90:0/742672323 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2610111b80 con 0x7f2610104240 2024-05-27T07:52:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.508+0000 7f260fe00640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2610109f90 0x7f261019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.508+0000 7f260ea00640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2610109af0 0x7f26101988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.509+0000 7f260f400640 1 -- 172.21.15.90:0/742672323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2610109f90 msgr2=0x7f261019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.509+0000 7f260f400640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2610109f90 0x7f261019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.509+0000 7f260f400640 1 -- 172.21.15.90:0/742672323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2610109af0 msgr2=0x7f26101988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.509+0000 7f260f400640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2610109af0 0x7f26101988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.509+0000 7f260f400640 1 -- 172.21.15.90:0/742672323 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261019d1d0 con 0x7f2610104240 2024-05-27T07:52:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.510+0000 7f260f400640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610104240 0x7f2610198370 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f260000e9c0 tx=0x7f260000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.510+0000 7f2607e00640 1 -- 172.21.15.90:0/742672323 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260000cd80 con 0x7f2610104240 2024-05-27T07:52:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.511+0000 7f2616800640 1 -- 172.21.15.90:0/742672323 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261019d4c0 con 0x7f2610104240 2024-05-27T07:52:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.511+0000 7f2616800640 1 -- 172.21.15.90:0/742672323 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101a4e40 con 0x7f2610104240 2024-05-27T07:52:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.511+0000 7f2607e00640 1 -- 172.21.15.90:0/742672323 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f26000054b0 con 0x7f2610104240 2024-05-27T07:52:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.511+0000 7f2607e00640 1 -- 172.21.15.90:0/742672323 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600010430 con 0x7f2610104240 2024-05-27T07:52:11.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.520+0000 7f261d3fd740 1 Processor -- start 2024-05-27T07:52:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.521+0000 7f261d3fd740 1 -- start start 2024-05-27T07:52:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.522+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca3d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.522+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6692f0 0x55cfcca3d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.522+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca3dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.522+0000 7f261d3fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc9a5b90 con 0x55cfcc6714d0 2024-05-27T07:52:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.522+0000 7f261d3fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc995730 con 0x55cfcc6692f0 2024-05-27T07:52:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.522+0000 7f261d3fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc999770 con 0x55cfcc6921b0 2024-05-27T07:52:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.523+0000 7f260ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca3dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.523+0000 7f260ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca3dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35610/0 (socket says 172.21.15.90:35610) 2024-05-27T07:52:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.523+0000 7f260ea00640 1 -- 172.21.15.90:0/3453890991 learned_addr learned my addr 172.21.15.90:0/3453890991 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.523+0000 7f260ea00640 1 -- 172.21.15.90:0/3453890991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 msgr2=0x55cfcca3d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.524+0000 7f260fe00640 1 --2- 172.21.15.90:0/3453890991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca3d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.524+0000 7f2607e00640 1 -- 172.21.15.90:0/742672323 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f26000105d0 con 0x7f2610104240 2024-05-27T07:52:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.524+0000 7f260ea00640 1 --2- 172.21.15.90:0/3453890991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca3d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.525+0000 7f260ea00640 1 -- 172.21.15.90:0/3453890991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6692f0 msgr2=0x55cfcca3d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.525+0000 7f260ea00640 1 --2- 172.21.15.90:0/3453890991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6692f0 0x55cfcca3d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.525+0000 7f260ea00640 1 -- 172.21.15.90:0/3453890991 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfcc92e8b0 con 0x55cfcc6714d0 2024-05-27T07:52:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.526+0000 7f260fe00640 1 --2- 172.21.15.90:0/3453890991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca3d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.527+0000 7f260ea00640 1 --2- 172.21.15.90:0/3453890991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca3dcf0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ba0 tx=0x7f25fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d42d0a22614dcbc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.528+0000 7f2604c00640 1 -- 172.21.15.90:0/3453890991 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc004220 con 0x55cfcc6714d0 2024-05-27T07:52:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.528+0000 7f2604c00640 1 -- 172.21.15.90:0/3453890991 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25fc009e10 con 0x55cfcc6714d0 2024-05-27T07:52:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.528+0000 7f2604c00640 1 -- 172.21.15.90:0/3453890991 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc019690 con 0x55cfcc6714d0 2024-05-27T07:52:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.529+0000 7f2607e00640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25f40b2a50 0x7f25f40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.529+0000 7f2607e00640 1 -- 172.21.15.90:0/742672323 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f26000e2df0 con 0x7f2610104240 2024-05-27T07:52:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.529+0000 7f260fe00640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25f40b2a50 0x7f25f40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.530+0000 7f261d3fd740 1 -- 172.21.15.90:0/3453890991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 msgr2=0x55cfcca3dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.530+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3453890991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca3dcf0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ba0 tx=0x7f25fc011810 comp rx=0 tx=0).stop 2024-05-27T07:52:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.531+0000 7f260fe00640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25f40b2a50 0x7f25f40b4f60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55cfcca3a3e0 tx=0x7f2608009800 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.531+0000 7f261d3fd740 1 -- 172.21.15.90:0/3453890991 shutdown_connections 2024-05-27T07:52:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.531+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3453890991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6692f0 0x55cfcca3d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.531+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3453890991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca3dcf0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.531+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3453890991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca3d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.531+0000 7f261d3fd740 1 -- 172.21.15.90:0/3453890991 >> 172.21.15.90:0/3453890991 conn(0x55cfcc686300 msgr2=0x55cfcca39a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.532+0000 7f261d3fd740 1 -- 172.21.15.90:0/3453890991 shutdown_connections 2024-05-27T07:52:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.532+0000 7f261d3fd740 1 -- 172.21.15.90:0/3453890991 wait complete. 2024-05-27T07:52:11.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.532+0000 7f261d3fd740 1 -- start start 2024-05-27T07:52:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.533+0000 7f261d3fd740 1 Processor -- start 2024-05-27T07:52:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.534+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6692f0 0x55cfcca473e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.534+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6714d0 0x55cfcca47920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.534+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca53700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.534+0000 7f261d3fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc67b790 con 0x55cfcc6692f0 2024-05-27T07:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.534+0000 7f261d3fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc93dbc0 con 0x55cfcc6714d0 2024-05-27T07:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.534+0000 7f261d3fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cfcca4a3a0 con 0x55cfcc6921b0 2024-05-27T07:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6692f0 0x55cfcca473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f261d3fd740 10 client.0 _ll_register_callbacks cb 0x55cfcc91b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6692f0 0x55cfcca473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35614/0 (socket says 172.21.15.90:35614) 2024-05-27T07:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260f400640 1 -- 172.21.15.90:0/3751649157 learned_addr learned my addr 172.21.15.90:0/3751649157 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260fe00640 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca53700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260ea00640 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6714d0 0x55cfcca47920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260f400640 1 -- 172.21.15.90:0/3751649157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 msgr2=0x55cfcca53700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260f400640 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca53700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260f400640 1 -- 172.21.15.90:0/3751649157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6714d0 msgr2=0x55cfcca47920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260f400640 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6714d0 0x55cfcca47920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260f400640 1 -- 172.21.15.90:0/3751649157 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfcc6aa210 con 0x55cfcc6692f0 2024-05-27T07:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.535+0000 7f260fe00640 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca53700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.536+0000 7f260f400640 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6692f0 0x55cfcca473e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f260000c6a0 tx=0x7f26000a9550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4110b12659ac2ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.536+0000 7f25f9600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.536+0000 7f25f9600640 1 -- 172.21.15.90:0/3751649157 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26000c0070 con 0x55cfcc6692f0 2024-05-27T07:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.537+0000 7f261d3fd740 1 -- 172.21.15.90:0/3751649157 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc009e10 con 0x55cfcc6692f0 2024-05-27T07:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.537+0000 7f25f9600640 1 -- 172.21.15.90:0/3751649157 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f26000c43b0 con 0x55cfcc6692f0 2024-05-27T07:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.538+0000 7f25f9600640 1 -- 172.21.15.90:0/3751649157 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f26000e0620 con 0x55cfcc6692f0 2024-05-27T07:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.539+0000 7f261d3fd740 10 client.16653 fetch_fsmap learned FSMap version 44 2024-05-27T07:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.539+0000 7f261d3fd740 1 -- 172.21.15.90:0/3751649157 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cfcc776620 con 0x55cfcc6692f0 2024-05-27T07:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.539+0000 7f25f9600640 1 -- 172.21.15.90:0/3751649157 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25fc009e10 con 0x55cfcc6692f0 2024-05-27T07:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.540+0000 7f25f9600640 1 -- 172.21.15.90:0/3751649157 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 44) ==== 7757+0+0 (secure 0 0 0) 0x7f26000a89f0 con 0x55cfcc6692f0 2024-05-27T07:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.540+0000 7f261d3fd740 10 client.16653 fetch_fsmap finished waiting for FSMap version 44 2024-05-27T07:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.540+0000 7f261d3fd740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.540+0000 7f261d3fd740 20 client.16653 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.540+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfcc650d80 0x55cfcca59aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.540+0000 7f261d3fd740 4 client.16653 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.540+0000 7f261d3fd740 1 -- 172.21.15.90:0/3751649157 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfcc9a7ae0 con 0x55cfcc650d80 2024-05-27T07:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.541+0000 7f260ea00640 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfcc650d80 0x55cfcca59aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.543+0000 7f25f9600640 10 client.16653 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.543+0000 7f260ea00640 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfcc650d80 0x55cfcca59aa0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde0d15c0e21fc43 server_cookie=15cf5138647dbbdb in_seq=0 out_seq=0 2024-05-27T07:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.546+0000 7f25f9600640 1 -- 172.21.15.90:0/3751649157 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55cfcc9a7ae0 con 0x55cfcc650d80 2024-05-27T07:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.546+0000 7f25f9600640 10 client.16653 handle_command_reply: tid=0 2024-05-27T07:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.546+0000 7f261d3fd740 1 client.16653 shutdown 2024-05-27T07:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.548+0000 7f261d3fd740 1 -- 172.21.15.90:0/3751649157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6692f0 msgr2=0x55cfcca473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.548+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6692f0 0x55cfcca473e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f260000c6a0 tx=0x7f26000a9550 comp rx=0 tx=0).stop 2024-05-27T07:52:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.549+0000 7f261d3fd740 1 -- 172.21.15.90:0/3751649157 shutdown_connections 2024-05-27T07:52:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.549+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfcc650d80 0x55cfcca59aa0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.549+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6714d0 0x55cfcca47920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.549+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca53700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.549+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3751649157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6692f0 0x55cfcca473e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.549+0000 7f261d3fd740 1 -- 172.21.15.90:0/3751649157 >> 172.21.15.90:0/3751649157 conn(0x55cfcc686300 msgr2=0x55cfcca3ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.549+0000 7f25f9600640 0 client.16653 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.549+0000 7f261d3fd740 1 -- 172.21.15.90:0/3751649157 shutdown_connections 2024-05-27T07:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.549+0000 7f261d3fd740 1 -- 172.21.15.90:0/3751649157 wait complete. 2024-05-27T07:52:11.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.550+0000 7f261d3fd740 20 client.16653 trim_cache size 0 max 16384 2024-05-27T07:52:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.583+0000 7f261d3fd740 1 Processor -- start 2024-05-27T07:52:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.583+0000 7f261d3fd740 1 -- start start 2024-05-27T07:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.584+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca42bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.584+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6921b0 0x55cfcca49b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.584+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6261c0 0x55cfcca53470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.584+0000 7f261d3fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0011900 con 0x55cfcc6714d0 2024-05-27T07:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.584+0000 7f261d3fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25f000f6f0 con 0x55cfcc6921b0 2024-05-27T07:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.584+0000 7f261d3fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0011230 con 0x55cfcc6261c0 2024-05-27T07:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.584+0000 7f260f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca42bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.584+0000 7f260f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca42bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35630/0 (socket says 172.21.15.90:35630) 2024-05-27T07:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.584+0000 7f260f400640 1 -- 172.21.15.90:0/3802965552 learned_addr learned my addr 172.21.15.90:0/3802965552 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.584+0000 7f260ea00640 1 --2- 172.21.15.90:0/3802965552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6921b0 0x55cfcca49b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.585+0000 7f260fe00640 1 --2- 172.21.15.90:0/3802965552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6261c0 0x55cfcca53470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.585+0000 7f260ea00640 1 -- 172.21.15.90:0/3802965552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6261c0 msgr2=0x55cfcca53470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.585+0000 7f260ea00640 1 --2- 172.21.15.90:0/3802965552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6261c0 0x55cfcca53470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.585+0000 7f260ea00640 1 -- 172.21.15.90:0/3802965552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 msgr2=0x55cfcca42bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.585+0000 7f260ea00640 1 --2- 172.21.15.90:0/3802965552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca42bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.585+0000 7f260ea00640 1 -- 172.21.15.90:0/3802965552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfcca2b210 con 0x55cfcc6921b0 2024-05-27T07:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.586+0000 7f260ea00640 1 --2- 172.21.15.90:0/3802965552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6921b0 0x55cfcca49b10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009b70 tx=0x7f25fc003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b4a620233a85c57 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.586+0000 7f260f400640 1 --2- 172.21.15.90:0/3802965552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca42bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.587+0000 7f25fbe00640 1 -- 172.21.15.90:0/3802965552 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc009e10 con 0x55cfcc6921b0 2024-05-27T07:52:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.587+0000 7f25fbe00640 1 -- 172.21.15.90:0/3802965552 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25fc0035f0 con 0x55cfcc6921b0 2024-05-27T07:52:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.587+0000 7f25fbe00640 1 -- 172.21.15.90:0/3802965552 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc0052a0 con 0x55cfcc6921b0 2024-05-27T07:52:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.589+0000 7f261d3fd740 1 -- 172.21.15.90:0/3802965552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6921b0 msgr2=0x55cfcca49b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.589+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3802965552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6921b0 0x55cfcca49b10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009b70 tx=0x7f25fc003d70 comp rx=0 tx=0).stop 2024-05-27T07:52:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.589+0000 7f261d3fd740 1 -- 172.21.15.90:0/3802965552 shutdown_connections 2024-05-27T07:52:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.589+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3802965552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6921b0 0x55cfcca49b10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.589+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3802965552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6261c0 0x55cfcca53470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.589+0000 7f261d3fd740 1 --2- 172.21.15.90:0/3802965552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca42bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.589+0000 7f261d3fd740 1 -- 172.21.15.90:0/3802965552 >> 172.21.15.90:0/3802965552 conn(0x55cfcc686300 msgr2=0x55cfcca476d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.589+0000 7f261d3fd740 1 -- 172.21.15.90:0/3802965552 shutdown_connections 2024-05-27T07:52:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.590+0000 7f261d3fd740 1 -- 172.21.15.90:0/3802965552 wait complete. 2024-05-27T07:52:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.590+0000 7f261d3fd740 1 -- start start 2024-05-27T07:52:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.590+0000 7f261d3fd740 1 Processor -- start 2024-05-27T07:52:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.591+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6261c0 0x55cfcca412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.591+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca41800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.591+0000 7f261d3fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca41d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.592+0000 7f261d3fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0010490 con 0x55cfcc6714d0 2024-05-27T07:52:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.592+0000 7f261d3fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25f000fdc0 con 0x55cfcc6261c0 2024-05-27T07:52:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.592+0000 7f261d3fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0010b60 con 0x55cfcc6921b0 2024-05-27T07:52:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.592+0000 7f261d3fd740 10 client.0 _ll_register_callbacks cb 0x7f26000a9670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.592+0000 7f260f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6261c0 0x55cfcca412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.592+0000 7f260f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6261c0 0x55cfcca412c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:44812/0 (socket says 172.21.15.90:44812) 2024-05-27T07:52:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.592+0000 7f260f400640 1 -- 172.21.15.90:0/318490136 learned_addr learned my addr 172.21.15.90:0/318490136 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.593+0000 7f260ea00640 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca41800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.593+0000 7f260fe00640 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca41d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.593+0000 7f260f400640 1 -- 172.21.15.90:0/318490136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 msgr2=0x55cfcca41d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.593+0000 7f260f400640 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca41d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.593+0000 7f260f400640 1 -- 172.21.15.90:0/318490136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 msgr2=0x55cfcca41800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.593+0000 7f260f400640 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca41800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.593+0000 7f260f400640 1 -- 172.21.15.90:0/318490136 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfcca4e860 con 0x55cfcc6261c0 2024-05-27T07:52:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.594+0000 7f260f400640 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6261c0 0x55cfcca412c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2600000c00 tx=0x7f26000af530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2edf6090dafcd04b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.594+0000 7f25f9600640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:52:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.594+0000 7f25f9600640 1 -- 172.21.15.90:0/318490136 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26000c0040 con 0x55cfcc6261c0 2024-05-27T07:52:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.594+0000 7f25f9600640 1 -- 172.21.15.90:0/318490136 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f26000a6310 con 0x55cfcc6261c0 2024-05-27T07:52:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.595+0000 7f261d3fd740 1 -- 172.21.15.90:0/318490136 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc0035f0 con 0x55cfcc6261c0 2024-05-27T07:52:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.595+0000 7f25f9600640 1 -- 172.21.15.90:0/318490136 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f26000a9d20 con 0x55cfcc6261c0 2024-05-27T07:52:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.595+0000 7f261d3fd740 10 client.36161 fetch_fsmap learned FSMap version 44 2024-05-27T07:52:11.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.595+0000 7f261d3fd740 1 -- 172.21.15.90:0/318490136 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55cfcca4faf0 con 0x55cfcc6261c0 2024-05-27T07:52:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.596+0000 7f25f9600640 1 -- 172.21.15.90:0/318490136 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25fc0035f0 con 0x55cfcc6261c0 2024-05-27T07:52:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.597+0000 7f25f9600640 1 -- 172.21.15.90:0/318490136 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 44) ==== 7757+0+0 (secure 0 0 0) 0x7f2600004f20 con 0x55cfcc6261c0 2024-05-27T07:52:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.598+0000 7f261d3fd740 10 client.36161 fetch_fsmap finished waiting for FSMap version 44 2024-05-27T07:52:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.598+0000 7f261d3fd740 10 client.36161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.598+0000 7f261d3fd740 20 client.36161 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.598+0000 7f261d3fd740 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfccac2130 0x55cfccac4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.598+0000 7f261d3fd740 4 client.36161 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:52:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.598+0000 7f261d3fd740 1 -- 172.21.15.90:0/318490136 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cfcc9a12c0 con 0x55cfccac2130 2024-05-27T07:52:11.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.599+0000 7f260ea00640 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfccac2130 0x55cfccac4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.600+0000 7f260ea00640 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfccac2130 0x55cfccac4550 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8934ccdeae1b6c19 server_cookie=c1de99b3f3029fbc in_seq=0 out_seq=0 2024-05-27T07:52:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:11.600+0000 7f25f9600640 10 client.36161 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:12.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:12 smithi090 ceph-mon[31429]: pgmap v181: 129 pgs: 129 active+clean; 1.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.7 MiB/s wr, 664 op/s 2024-05-27T07:52:12.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:12 smithi090 ceph-mon[31429]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:52:12.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:12 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:52:12.431 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:12 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 1. 2024-05-27T07:52:12.432 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:12 smithi090 systemd[1]: Stopped Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:52:12.432 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:12 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.806s CPU time. 2024-05-27T07:52:12.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:12 smithi136 ceph-mon[35324]: pgmap v181: 129 pgs: 129 active+clean; 1.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.7 MiB/s wr, 664 op/s 2024-05-27T07:52:12.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:12 smithi136 ceph-mon[35324]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:52:12.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:12 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:52:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:12 smithi129 ceph-mon[35254]: pgmap v181: 129 pgs: 129 active+clean; 1.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.7 MiB/s wr, 664 op/s 2024-05-27T07:52:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:12 smithi129 ceph-mon[35254]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:52:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:12 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:52:12.707 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:12 smithi090 systemd[1]: Starting Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:52:12.775 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:52:12.776 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:52:13.464 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:13 smithi090 podman[106489]: 2024-05-27 07:52:13.074485799 +0000 UTC m=+0.067681367 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:52:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.822+0000 7f25f9600640 1 -- 172.21.15.90:0/318490136 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cfcc9a12c0 con 0x55cfccac2130 2024-05-27T07:52:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.822+0000 7f25f9600640 10 client.36161 handle_command_reply: tid=0 2024-05-27T07:52:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.822+0000 7f261d3fd740 1 client.36161 shutdown 2024-05-27T07:52:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.823+0000 7f261d3fd740 1 -- 172.21.15.90:0/318490136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6261c0 msgr2=0x55cfcca412c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.823+0000 7f261d3fd740 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6261c0 0x55cfcca412c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2600000c00 tx=0x7f26000af530 comp rx=0 tx=0).stop 2024-05-27T07:52:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.824+0000 7f261d3fd740 1 -- 172.21.15.90:0/318490136 shutdown_connections 2024-05-27T07:52:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.824+0000 7f261d3fd740 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55cfccac2130 0x55cfccac4550 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.824+0000 7f261d3fd740 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cfcc6921b0 0x55cfcca41d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.824+0000 7f261d3fd740 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cfcc6714d0 0x55cfcca41800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.824+0000 7f261d3fd740 1 --2- 172.21.15.90:0/318490136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cfcc6261c0 0x55cfcca412c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.824+0000 7f261d3fd740 1 -- 172.21.15.90:0/318490136 >> 172.21.15.90:0/318490136 conn(0x55cfcc686300 msgr2=0x55cfcca30240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.825+0000 7f25f9600640 0 client.36161 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.826+0000 7f261d3fd740 1 -- 172.21.15.90:0/318490136 shutdown_connections 2024-05-27T07:52:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.826+0000 7f261d3fd740 1 -- 172.21.15.90:0/318490136 wait complete. 2024-05-27T07:52:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.826+0000 7f261d3fd740 20 client.36161 trim_cache size 0 max 16384 2024-05-27T07:52:13.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 115.0 s, 498.9 tps, lat 199.182 ms stddev 248.187 2024-05-27T07:52:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:52:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:52:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.827+0000 7f2616800640 1 -- 172.21.15.90:0/742672323 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25f40b2a50 msgr2=0x7f25f40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.827+0000 7f2616800640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25f40b2a50 0x7f25f40b4f60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55cfcca3a3e0 tx=0x7f2608009800 comp rx=0 tx=0).stop 2024-05-27T07:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.827+0000 7f2616800640 1 -- 172.21.15.90:0/742672323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610104240 msgr2=0x7f2610198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.827+0000 7f2616800640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610104240 0x7f2610198370 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f260000e9c0 tx=0x7f260000ee80 comp rx=0 tx=0).stop 2024-05-27T07:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.828+0000 7f2616800640 1 -- 172.21.15.90:0/742672323 shutdown_connections 2024-05-27T07:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.828+0000 7f2616800640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f25f40b2a50 0x7f25f40b4f60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.828+0000 7f2616800640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2610109f90 0x7f261019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.828+0000 7f2616800640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2610109af0 0x7f26101988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.828+0000 7f2616800640 1 --2- 172.21.15.90:0/742672323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610104240 0x7f2610198370 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.828+0000 7f2616800640 1 -- 172.21.15.90:0/742672323 >> 172.21.15.90:0/742672323 conn(0x7f2610079250 msgr2=0x7f2610100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.828+0000 7f2616800640 1 -- 172.21.15.90:0/742672323 shutdown_connections 2024-05-27T07:52:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:13.828+0000 7f2616800640 1 -- 172.21.15.90:0/742672323 wait complete. 2024-05-27T07:52:13.848 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:13 smithi090 podman[106489]: 2024-05-27 07:52:13.593390885 +0000 UTC m=+0.586586455 container create 42e738ac06199e7bde57b90adcf79b8ee997c96bb159dd843b0e0a0eadc1933d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-27T07:52:14.158 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:13 smithi090 podman[106489]: 2024-05-27 07:52:13.847905122 +0000 UTC m=+0.841100709 container init 42e738ac06199e7bde57b90adcf79b8ee997c96bb159dd843b0e0a0eadc1933d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, io.buildah.version=1.33.5) 2024-05-27T07:52:14.158 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:13 smithi090 podman[106489]: 2024-05-27 07:52:13.868236975 +0000 UTC m=+0.861432543 container start 42e738ac06199e7bde57b90adcf79b8ee997c96bb159dd843b0e0a0eadc1933d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:52:14.158 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:14 smithi090 ceph-mds[106532]: -- 172.21.15.90:0/981945235 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf2fbf3a40 con 0x55bf2fc3ba80 2024-05-27T07:52:14.158 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: starting mds.j at 2024-05-27T07:52:14.421 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:14 smithi090 ceph-mon[31429]: pgmap v182: 129 pgs: 129 active+clean; 1.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.2 MiB/s wr, 841 op/s 2024-05-27T07:52:14.421 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:14 smithi090 bash[106489]: 42e738ac06199e7bde57b90adcf79b8ee997c96bb159dd843b0e0a0eadc1933d 2024-05-27T07:52:14.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:14 smithi136 ceph-mon[35324]: pgmap v182: 129 pgs: 129 active+clean; 1.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.2 MiB/s wr, 841 op/s 2024-05-27T07:52:14.542 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:14.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:14 smithi129 ceph-mon[35254]: pgmap v182: 129 pgs: 129 active+clean; 1.7 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.2 MiB/s wr, 841 op/s 2024-05-27T07:52:14.713 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:52:14 smithi090 systemd[1]: Started Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:52:15.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:15 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6836/292961192,v1:172.21.15.90:6837/292961192] up:boot 2024-05-27T07:52:15.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:15 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:52:15.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:15 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:52:15.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:15 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6836/292961192,v1:172.21.15.90:6837/292961192] up:boot 2024-05-27T07:52:15.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:15 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:52:15.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:15 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:52:15.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:15 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6836/292961192,v1:172.21.15.90:6837/292961192] up:boot 2024-05-27T07:52:15.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:15 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:52:15.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:15 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:52:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:16 smithi090 ceph-mon[31429]: pgmap v183: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-05-27T07:52:16.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:16 smithi136 ceph-mon[35324]: pgmap v183: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-05-27T07:52:16.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:16 smithi129 ceph-mon[35254]: pgmap v183: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 6.6 MiB/s wr, 685 op/s 2024-05-27T07:52:17.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.009+0000 7fad4a000700 1 -- 172.21.15.90:0/3345028913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad44109860 msgr2=0x7fad441111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:17.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.009+0000 7fad4a000700 1 --2- 172.21.15.90:0/3345028913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad44109860 0x7fad441111c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00b0d0 tx=0x7fad3c00b3e0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.009+0000 7fad4a000700 1 -- 172.21.15.90:0/3345028913 shutdown_connections 2024-05-27T07:52:17.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.009+0000 7fad4a000700 1 --2- 172.21.15.90:0/3345028913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad44109860 0x7fad441111c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.009+0000 7fad4a000700 1 --2- 172.21.15.90:0/3345028913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad441045e0 0x7fad441049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.009+0000 7fad4a000700 1 --2- 172.21.15.90:0/3345028913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad44108f10 0x7fad44109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.009+0000 7fad4a000700 1 -- 172.21.15.90:0/3345028913 >> 172.21.15.90:0/3345028913 conn(0x7fad44100030 msgr2=0x7fad44102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:17.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.010+0000 7fad4a000700 1 -- 172.21.15.90:0/3345028913 shutdown_connections 2024-05-27T07:52:17.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.010+0000 7fad4a000700 1 -- 172.21.15.90:0/3345028913 wait complete. 2024-05-27T07:52:17.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.011+0000 7fad4a000700 1 Processor -- start 2024-05-27T07:52:17.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.012+0000 7fad4a000700 1 -- start start 2024-05-27T07:52:17.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.012+0000 7fad4a000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad441045e0 0x7fad44110c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:17.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.012+0000 7fad4a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad44108f10 0x7fad44111170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:17.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.012+0000 7fad4a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad44109860 0x7fad4410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:17.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.012+0000 7fad4a000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fad44076bb0 con 0x7fad44108f10 2024-05-27T07:52:17.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.012+0000 7fad4a000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad44076a30 con 0x7fad441045e0 2024-05-27T07:52:17.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.012+0000 7fad4a000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fad44076d30 con 0x7fad44109860 2024-05-27T07:52:17.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.013+0000 7fad43400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad441045e0 0x7fad44110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:17.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.013+0000 7fad43400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad441045e0 0x7fad44110c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:44854/0 (socket says 172.21.15.90:44854) 2024-05-27T07:52:17.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.013+0000 7fad43400700 1 -- 172.21.15.90:0/627603830 learned_addr learned my addr 172.21.15.90:0/627603830 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:17.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.013+0000 7fad42a00700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad44108f10 0x7fad44111170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:17.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.014+0000 7fad43e00700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad44109860 0x7fad4410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:17.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.014+0000 7fad43400700 1 -- 172.21.15.90:0/627603830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad44109860 msgr2=0x7fad4410df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:17.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.014+0000 7fad43400700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad44109860 0x7fad4410df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.014+0000 7fad43400700 1 -- 172.21.15.90:0/627603830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad44108f10 msgr2=0x7fad44111170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:17.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.014+0000 7fad43400700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad44108f10 0x7fad44111170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.014+0000 7fad43400700 1 -- 172.21.15.90:0/627603830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4410e7a0 con 0x7fad441045e0 2024-05-27T07:52:17.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.015+0000 7fad43400700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad441045e0 0x7fad44110c30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c920 tx=0x7fad2c00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:17.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.016+0000 7fad3be00700 1 -- 172.21.15.90:0/627603830 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c007b00 con 0x7fad441045e0 2024-05-27T07:52:17.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.016+0000 7fad3be00700 1 -- 172.21.15.90:0/627603830 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad2c004cc0 con 0x7fad441045e0 2024-05-27T07:52:17.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.016+0000 7fad3be00700 1 -- 172.21.15.90:0/627603830 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0054f0 con 0x7fad441045e0 2024-05-27T07:52:17.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.016+0000 7fad4a000700 1 -- 172.21.15.90:0/627603830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4410ea90 con 0x7fad441045e0 2024-05-27T07:52:17.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.016+0000 7fad4a000700 1 -- 172.21.15.90:0/627603830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad441add90 con 0x7fad441045e0 2024-05-27T07:52:17.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.020+0000 7fad4a000700 1 -- 172.21.15.90:0/627603830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad440512b0 con 0x7fad441045e0 2024-05-27T07:52:17.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.022+0000 7fad3be00700 1 -- 172.21.15.90:0/627603830 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fad2c00f450 con 0x7fad441045e0 2024-05-27T07:52:17.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.026+0000 7fad3be00700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad280b2b80 0x7fad280b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:17.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.026+0000 7fad3be00700 1 -- 172.21.15.90:0/627603830 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fad2c0e2440 con 0x7fad441045e0 2024-05-27T07:52:17.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.027+0000 7fad42a00700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad280b2b80 0x7fad280b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:17.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.030+0000 7fad42a00700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad280b2b80 0x7fad280b5090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fad34006fd0 tx=0x7fad34008040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:17.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.045+0000 7fad3be00700 1 -- 172.21.15.90:0/627603830 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad2c014030 con 0x7fad441045e0 2024-05-27T07:52:17.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.551+0000 7fad4a000700 1 -- 172.21.15.90:0/627603830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad441adfc0 con 0x7fad441045e0 2024-05-27T07:52:17.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.554+0000 7fad3be00700 1 -- 172.21.15.90:0/627603830 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11111 (secure 0 0 0) 0x7fad2c0a9b50 con 0x7fad441045e0 2024-05-27T07:52:17.558 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:52:17.558 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":26221,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/292961192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":292961192},{"type":"v1","addr":"172.21.15.90:6837","nonce":292961192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:52:12.015377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26167":{"gid":26167,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/3381199377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":3381199377},{"type":"v1","addr":"172.21.15.90:6835","nonce":3381199377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34442,34424,24478,34406]},"id":1}]} 2024-05-27T07:52:17.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.558+0000 7fad4a000700 1 -- 172.21.15.90:0/627603830 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad280b2b80 msgr2=0x7fad280b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:17.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.558+0000 7fad4a000700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad280b2b80 0x7fad280b5090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fad34006fd0 tx=0x7fad34008040 comp rx=0 tx=0).stop 2024-05-27T07:52:17.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.558+0000 7fad4a000700 1 -- 172.21.15.90:0/627603830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad441045e0 msgr2=0x7fad44110c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:17.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.558+0000 7fad4a000700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad441045e0 0x7fad44110c30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00c920 tx=0x7fad2c00cce0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.560+0000 7fad4a000700 1 -- 172.21.15.90:0/627603830 shutdown_connections 2024-05-27T07:52:17.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.560+0000 7fad4a000700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad44109860 0x7fad4410df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.560+0000 7fad4a000700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad280b2b80 0x7fad280b5090 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.560+0000 7fad4a000700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad441045e0 0x7fad44110c30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.560+0000 7fad4a000700 1 --2- 172.21.15.90:0/627603830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad44108f10 0x7fad44111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:17.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.560+0000 7fad4a000700 1 -- 172.21.15.90:0/627603830 >> 172.21.15.90:0/627603830 conn(0x7fad44100030 msgr2=0x7fad441066a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:17.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.561+0000 7fad4a000700 1 -- 172.21.15.90:0/627603830 shutdown_connections 2024-05-27T07:52:17.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:17.561+0000 7fad4a000700 1 -- 172.21.15.90:0/627603830 wait complete. 2024-05-27T07:52:17.566 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 46 2024-05-27T07:52:18.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:18 smithi090 ceph-mon[31429]: pgmap v184: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.7 MiB/s wr, 558 op/s 2024-05-27T07:52:18.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:18 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/627603830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:52:18.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:18 smithi136 ceph-mon[35324]: pgmap v184: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.7 MiB/s wr, 558 op/s 2024-05-27T07:52:18.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:18 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/627603830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:52:18.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:18 smithi129 ceph-mon[35254]: pgmap v184: 129 pgs: 129 active+clean; 1.8 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.7 MiB/s wr, 558 op/s 2024-05-27T07:52:18.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:18 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/627603830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:52:18.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 120.0 s, 589.4 tps, lat 170.517 ms stddev 46.634 2024-05-27T07:52:20.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:20 smithi090 ceph-mon[31429]: pgmap v185: 129 pgs: 129 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 8.1 MiB/s wr, 841 op/s 2024-05-27T07:52:20.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:20 smithi136 ceph-mon[35324]: pgmap v185: 129 pgs: 129 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 8.1 MiB/s wr, 841 op/s 2024-05-27T07:52:20.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:20 smithi129 ceph-mon[35254]: pgmap v185: 129 pgs: 129 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 8.1 MiB/s wr, 841 op/s 2024-05-27T07:52:21.215 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:52:22.058 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:22.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:22 smithi136 ceph-mon[35324]: pgmap v186: 129 pgs: 129 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.8 MiB/s wr, 523 op/s 2024-05-27T07:52:22.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:22 smithi129 ceph-mon[35254]: pgmap v186: 129 pgs: 129 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.8 MiB/s wr, 523 op/s 2024-05-27T07:52:22.570 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:22 smithi090 ceph-mon[31429]: pgmap v186: 129 pgs: 129 active+clean; 1.8 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.8 MiB/s wr, 523 op/s 2024-05-27T07:52:23.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 125.0 s, 599.8 tps, lat 166.058 ms stddev 60.969 2024-05-27T07:52:24.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:24 smithi090 ceph-mon[31429]: pgmap v187: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.1 MiB/s wr, 689 op/s 2024-05-27T07:52:24.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:24 smithi136 ceph-mon[35324]: pgmap v187: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.1 MiB/s wr, 689 op/s 2024-05-27T07:52:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.561+0000 7efeea600700 1 -- 172.21.15.90:0/3279861873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee4109670 msgr2=0x7efee4110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.561+0000 7efeea600700 1 --2- 172.21.15.90:0/3279861873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee4109670 0x7efee4110fd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efed40098f0 tx=0x7efed4009c00 comp rx=0 tx=0).stop 2024-05-27T07:52:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.561+0000 7efeea600700 1 -- 172.21.15.90:0/3279861873 shutdown_connections 2024-05-27T07:52:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.561+0000 7efeea600700 1 --2- 172.21.15.90:0/3279861873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee4109670 0x7efee4110fd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.561+0000 7efeea600700 1 --2- 172.21.15.90:0/3279861873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efee41043f0 0x7efee4104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.561+0000 7efeea600700 1 --2- 172.21.15.90:0/3279861873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efee4108d20 0x7efee4109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.561+0000 7efeea600700 1 -- 172.21.15.90:0/3279861873 >> 172.21.15.90:0/3279861873 conn(0x7efee4079d00 msgr2=0x7efee407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:24.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:24 smithi129 ceph-mon[35254]: pgmap v187: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.1 MiB/s wr, 689 op/s 2024-05-27T07:52:24.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.562+0000 7efeea600700 1 -- 172.21.15.90:0/3279861873 shutdown_connections 2024-05-27T07:52:24.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.563+0000 7efeea600700 1 -- 172.21.15.90:0/3279861873 wait complete. 2024-05-27T07:52:24.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.565+0000 7efeea600700 1 Processor -- start 2024-05-27T07:52:24.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.565+0000 7efeea600700 1 -- start start 2024-05-27T07:52:24.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.567+0000 7efeea600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efee41043f0 0x7efee419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.567+0000 7efeea600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efee4108d20 0x7efee419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.567+0000 7efeea600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee4109670 0x7efee41a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.567+0000 7efeea600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efee4114210 con 0x7efee41043f0 2024-05-27T07:52:24.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.567+0000 7efeea600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efee4114090 con 0x7efee4109670 2024-05-27T07:52:24.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.567+0000 7efeea600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efee4114390 con 0x7efee4108d20 2024-05-27T07:52:24.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.568+0000 7efee9c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee4109670 0x7efee41a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.568+0000 7efee9c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee4109670 0x7efee41a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46126/0 (socket says 172.21.15.90:46126) 2024-05-27T07:52:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.569+0000 7efee9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efee41043f0 0x7efee419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.569+0000 7efee9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efee41043f0 0x7efee419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55766/0 (socket says 172.21.15.90:55766) 2024-05-27T07:52:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.569+0000 7efee9200700 1 -- 172.21.15.90:0/3537844493 learned_addr learned my addr 172.21.15.90:0/3537844493 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:24.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.570+0000 7efee3e00700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efee4108d20 0x7efee419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.571+0000 7efee9200700 1 -- 172.21.15.90:0/3537844493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efee4108d20 msgr2=0x7efee419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.571+0000 7efee9200700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efee4108d20 0x7efee419d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.571+0000 7efee9200700 1 -- 172.21.15.90:0/3537844493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee4109670 msgr2=0x7efee41a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.571+0000 7efee9200700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee4109670 0x7efee41a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.571+0000 7efee9200700 1 -- 172.21.15.90:0/3537844493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee41a1db0 con 0x7efee41043f0 2024-05-27T07:52:24.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.572+0000 7efee9200700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efee41043f0 0x7efee419cee0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efed000d980 tx=0x7efed000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:24.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.573+0000 7efee1600700 1 -- 172.21.15.90:0/3537844493 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0016070 con 0x7efee41043f0 2024-05-27T07:52:24.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.573+0000 7efee1600700 1 -- 172.21.15.90:0/3537844493 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efed000fd20 con 0x7efee41043f0 2024-05-27T07:52:24.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.574+0000 7efeea600700 1 -- 172.21.15.90:0/3537844493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee41a20a0 con 0x7efee41043f0 2024-05-27T07:52:24.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.574+0000 7efeea600700 1 -- 172.21.15.90:0/3537844493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41a99c0 con 0x7efee41043f0 2024-05-27T07:52:24.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.574+0000 7efee1600700 1 -- 172.21.15.90:0/3537844493 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00093a0 con 0x7efee41043f0 2024-05-27T07:52:24.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.579+0000 7efefa7a5b80 1 Processor -- start 2024-05-27T07:52:24.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.581+0000 7efefa7a5b80 1 -- start start 2024-05-27T07:52:24.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.583+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565108759e40 0x5651088e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.583+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651088e0c20 0x5651088eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.583+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e1030 0x5651088ed830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.583+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x565108806e40 con 0x565108759e40 2024-05-27T07:52:24.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.583+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5651087f69e0 con 0x5651088e0c20 2024-05-27T07:52:24.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.583+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5651087faa20 con 0x5651088e1030 2024-05-27T07:52:24.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.584+0000 7efee9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565108759e40 0x5651088e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.584+0000 7efee9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565108759e40 0x5651088e06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55776/0 (socket says 172.21.15.90:55776) 2024-05-27T07:52:24.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.584+0000 7efee9c00700 1 -- 172.21.15.90:0/2244085446 learned_addr learned my addr 172.21.15.90:0/2244085446 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:24.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.585+0000 7efee9c00700 1 -- 172.21.15.90:0/2244085446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e1030 msgr2=0x5651088ed830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:52:24.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.585+0000 7efee9c00700 1 --2- 172.21.15.90:0/2244085446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e1030 0x5651088ed830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.585+0000 7efee9c00700 1 -- 172.21.15.90:0/2244085446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651088e0c20 msgr2=0x5651088eb2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:52:24.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.585+0000 7efee9c00700 1 --2- 172.21.15.90:0/2244085446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651088e0c20 0x5651088eb2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.585+0000 7efee9c00700 1 -- 172.21.15.90:0/2244085446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651087bf420 con 0x565108759e40 2024-05-27T07:52:24.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.587+0000 7efee9c00700 1 --2- 172.21.15.90:0/2244085446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565108759e40 0x5651088e06e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efed40098f0 tx=0x7efed4009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473a1b24d2b0d5b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:24.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.588+0000 7efeda000700 1 -- 172.21.15.90:0/2244085446 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4005070 con 0x565108759e40 2024-05-27T07:52:24.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.588+0000 7efeda000700 1 -- 172.21.15.90:0/2244085446 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efed4026070 con 0x565108759e40 2024-05-27T07:52:24.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.588+0000 7efeda000700 1 -- 172.21.15.90:0/2244085446 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4003970 con 0x565108759e40 2024-05-27T07:52:24.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.590+0000 7efee1600700 1 -- 172.21.15.90:0/3537844493 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efed0019070 con 0x7efee41043f0 2024-05-27T07:52:24.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.590+0000 7efefa7a5b80 1 -- 172.21.15.90:0/2244085446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565108759e40 msgr2=0x5651088e06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:24.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.590+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/2244085446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565108759e40 0x5651088e06e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efed40098f0 tx=0x7efed4009400 comp rx=0 tx=0).stop 2024-05-27T07:52:24.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.591+0000 7efefa7a5b80 1 -- 172.21.15.90:0/2244085446 shutdown_connections 2024-05-27T07:52:24.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.591+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/2244085446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e1030 0x5651088ed830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.591+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/2244085446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565108759e40 0x5651088e06e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.591+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/2244085446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651088e0c20 0x5651088eb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.591+0000 7efefa7a5b80 1 -- 172.21.15.90:0/2244085446 >> 172.21.15.90:0/2244085446 conn(0x5651087b9fc0 msgr2=0x5651088d3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:24.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.591+0000 7efefa7a5b80 1 -- 172.21.15.90:0/2244085446 shutdown_connections 2024-05-27T07:52:24.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.592+0000 7efefa7a5b80 1 -- 172.21.15.90:0/2244085446 wait complete. 2024-05-27T07:52:24.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.593+0000 7efefa7a5b80 1 -- start start 2024-05-27T07:52:24.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.593+0000 7efefa7a5b80 1 Processor -- start 2024-05-27T07:52:24.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.594+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565108759e40 0x5651088eae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.594+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651088e0c20 0x5651088eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.594+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e1030 0x5651088eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.594+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5651086fcfa0 con 0x5651088e1030 2024-05-27T07:52:24.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.594+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565108797760 con 0x5651088e0c20 2024-05-27T07:52:24.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.594+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5651088ee140 con 0x565108759e40 2024-05-27T07:52:24.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.594+0000 7efefa7a5b80 10 client.0 _ll_register_callbacks cb 0x5651087a0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:24.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.595+0000 7efee9c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565108759e40 0x5651088eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.595+0000 7efee1600700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efec80b2a00 0x7efec80b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.595+0000 7efee9c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565108759e40 0x5651088eae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36840/0 (socket says 172.21.15.90:36840) 2024-05-27T07:52:24.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.595+0000 7efee9c00700 1 -- 172.21.15.90:0/410596456 learned_addr learned my addr 172.21.15.90:0/410596456 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:24.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.595+0000 7efee1600700 1 -- 172.21.15.90:0/3537844493 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7efed00e2730 con 0x7efee41043f0 2024-05-27T07:52:24.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.595+0000 7efee9200700 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651088e0c20 0x5651088eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.596+0000 7efee3e00700 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e1030 0x5651088eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.596+0000 7efee9c00700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efec80b2a00 0x7efec80b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.596+0000 7efee9200700 1 -- 172.21.15.90:0/410596456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565108759e40 msgr2=0x5651088eae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:24.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.596+0000 7efee9200700 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565108759e40 0x5651088eae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.596+0000 7efee9200700 1 -- 172.21.15.90:0/410596456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e1030 msgr2=0x5651088eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:24.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.596+0000 7efee9200700 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e1030 0x5651088eb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.596+0000 7efee9200700 1 -- 172.21.15.90:0/410596456 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651087468d0 con 0x5651088e0c20 2024-05-27T07:52:24.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.598+0000 7efecea00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:52:24.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.598+0000 7efee9200700 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651088e0c20 0x5651088eb340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efed00dfad0 tx=0x7efed00dffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f412fb75c42dde9b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:24.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.598+0000 7efecea00700 1 -- 172.21.15.90:0/410596456 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0020030 con 0x5651088e0c20 2024-05-27T07:52:24.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.598+0000 7efecea00700 1 -- 172.21.15.90:0/410596456 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7efed00106a0 con 0x5651088e0c20 2024-05-27T07:52:24.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.598+0000 7efee9c00700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efec80b2a00 0x7efec80b4f10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efed4009c40 tx=0x7efed4019040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:24.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.598+0000 7efefa7a5b80 1 -- 172.21.15.90:0/410596456 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed4026070 con 0x5651088e0c20 2024-05-27T07:52:24.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.599+0000 7efecea00700 1 -- 172.21.15.90:0/410596456 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7efed00a9660 con 0x5651088e0c20 2024-05-27T07:52:24.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.599+0000 7efefa7a5b80 10 client.36179 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:24.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.599+0000 7efefa7a5b80 1 -- 172.21.15.90:0/410596456 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5651087c7060 con 0x5651088e0c20 2024-05-27T07:52:24.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.600+0000 7efecea00700 1 -- 172.21.15.90:0/410596456 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efed4026070 con 0x5651088e0c20 2024-05-27T07:52:24.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.600+0000 7efecea00700 1 -- 172.21.15.90:0/410596456 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7efed00e23b0 con 0x5651088e0c20 2024-05-27T07:52:24.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.601+0000 7efefa7a5b80 10 client.36179 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:24.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.601+0000 7efefa7a5b80 10 client.36179 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:52:24.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.601+0000 7efefa7a5b80 20 client.36179 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:24.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.601+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5651088fb210 0x5651088fd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.601+0000 7efefa7a5b80 4 client.36179 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:24.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.601+0000 7efefa7a5b80 1 -- 172.21.15.90:0/410596456 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565108808d90 con 0x5651088fb210 2024-05-27T07:52:24.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.602+0000 7efee3e00700 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5651088fb210 0x5651088fd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.603+0000 7efee3e00700 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5651088fb210 0x5651088fd630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c1acbc236037a25 server_cookie=f2dc4449eaa166a4 in_seq=0 out_seq=0 2024-05-27T07:52:24.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.603+0000 7efecea00700 10 client.36179 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:52:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.605+0000 7efecea00700 1 -- 172.21.15.90:0/410596456 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565108808d90 con 0x5651088fb210 2024-05-27T07:52:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.605+0000 7efecea00700 10 client.36179 handle_command_reply: tid=0 2024-05-27T07:52:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.605+0000 7efefa7a5b80 1 client.36179 shutdown 2024-05-27T07:52:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.606+0000 7efefa7a5b80 1 -- 172.21.15.90:0/410596456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651088e0c20 msgr2=0x5651088eb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.606+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651088e0c20 0x5651088eb340 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efed00dfad0 tx=0x7efed00dffd0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.606+0000 7efefa7a5b80 1 -- 172.21.15.90:0/410596456 shutdown_connections 2024-05-27T07:52:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.606+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5651088fb210 0x5651088fd630 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.607+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e1030 0x5651088eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.607+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x565108759e40 0x5651088eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.607+0000 7efecea00700 0 client.36179 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:52:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.607+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/410596456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651088e0c20 0x5651088eb340 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.607+0000 7efefa7a5b80 1 -- 172.21.15.90:0/410596456 >> 172.21.15.90:0/410596456 conn(0x5651087b9fc0 msgr2=0x5651088d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.608+0000 7efefa7a5b80 1 -- 172.21.15.90:0/410596456 shutdown_connections 2024-05-27T07:52:24.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.608+0000 7efefa7a5b80 1 -- 172.21.15.90:0/410596456 wait complete. 2024-05-27T07:52:24.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.611+0000 7efefa7a5b80 20 client.36179 trim_cache size 0 max 16384 2024-05-27T07:52:24.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.645+0000 7efefa7a5b80 1 Processor -- start 2024-05-27T07:52:24.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.645+0000 7efefa7a5b80 1 -- start start 2024-05-27T07:52:24.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.645+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088e6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.645+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e0c20 0x5651088e9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.645+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e1030 0x5651088ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.646+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efee41afd10 con 0x5651088e0c20 2024-05-27T07:52:24.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.646+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efee41adb00 con 0x565108759e40 2024-05-27T07:52:24.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.646+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efee41af640 con 0x5651088e1030 2024-05-27T07:52:24.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.646+0000 7efee9200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.646+0000 7efee9200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088e6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46150/0 (socket says 172.21.15.90:46150) 2024-05-27T07:52:24.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.646+0000 7efee9200700 1 -- 172.21.15.90:0/3543331211 learned_addr learned my addr 172.21.15.90:0/3543331211 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:24.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.647+0000 7efee9c00700 1 --2- 172.21.15.90:0/3543331211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e1030 0x5651088ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.647+0000 7efee3e00700 1 --2- 172.21.15.90:0/3543331211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e0c20 0x5651088e9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.648+0000 7efee9200700 1 -- 172.21.15.90:0/3543331211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e1030 msgr2=0x5651088ed2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:24.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.648+0000 7efee9200700 1 --2- 172.21.15.90:0/3543331211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e1030 0x5651088ed2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.648+0000 7efee9200700 1 -- 172.21.15.90:0/3543331211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e0c20 msgr2=0x5651088e9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:24.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.648+0000 7efee9200700 1 --2- 172.21.15.90:0/3543331211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e0c20 0x5651088e9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.648+0000 7efee9200700 1 -- 172.21.15.90:0/3543331211 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651088e19e0 con 0x565108759e40 2024-05-27T07:52:24.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.649+0000 7efee9200700 1 --2- 172.21.15.90:0/3543331211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088e6640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efed00a5690 tx=0x7efed0010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd7da01b4f581d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.650+0000 7efed9600700 1 -- 172.21.15.90:0/3543331211 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0020070 con 0x565108759e40 2024-05-27T07:52:24.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.650+0000 7efed9600700 1 -- 172.21.15.90:0/3543331211 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efed00ada10 con 0x565108759e40 2024-05-27T07:52:24.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.651+0000 7efed9600700 1 -- 172.21.15.90:0/3543331211 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00e2320 con 0x565108759e40 2024-05-27T07:52:24.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.652+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3543331211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 msgr2=0x5651088e6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:24.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.652+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/3543331211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088e6640 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efed00a5690 tx=0x7efed0010000 comp rx=0 tx=0).stop 2024-05-27T07:52:24.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.653+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3543331211 shutdown_connections 2024-05-27T07:52:24.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.653+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/3543331211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e1030 0x5651088ed2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.653+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/3543331211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088e6640 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.653+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/3543331211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e0c20 0x5651088e9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.653+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3543331211 >> 172.21.15.90:0/3543331211 conn(0x5651087b9fc0 msgr2=0x5651088eabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:24.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.653+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3543331211 shutdown_connections 2024-05-27T07:52:24.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.653+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3543331211 wait complete. 2024-05-27T07:52:24.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.654+0000 7efefa7a5b80 1 -- start start 2024-05-27T07:52:24.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.655+0000 7efefa7a5b80 1 Processor -- start 2024-05-27T07:52:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.656+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.656+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e0c20 0x5651088eef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.656+0000 7efefa7a5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e1030 0x5651088ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.656+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efee41ae8a0 con 0x5651088e1030 2024-05-27T07:52:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.656+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efee41ae1d0 con 0x565108759e40 2024-05-27T07:52:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.656+0000 7efefa7a5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efee41aef70 con 0x5651088e0c20 2024-05-27T07:52:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.656+0000 7efefa7a5b80 10 client.0 _ll_register_callbacks cb 0x5651087a0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:24.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.657+0000 7efee3e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e0c20 0x5651088eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.657+0000 7efee3e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e0c20 0x5651088eef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36848/0 (socket says 172.21.15.90:36848) 2024-05-27T07:52:24.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.657+0000 7efee3e00700 1 -- 172.21.15.90:0/3908767314 learned_addr learned my addr 172.21.15.90:0/3908767314 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:24.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.658+0000 7efee9200700 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.659+0000 7efee9c00700 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e1030 0x5651088ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.660+0000 7efee9200700 1 -- 172.21.15.90:0/3908767314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e0c20 msgr2=0x5651088eef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:24.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.660+0000 7efee9200700 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e0c20 0x5651088eef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.660+0000 7efee9200700 1 -- 172.21.15.90:0/3908767314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e1030 msgr2=0x5651088ef5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:24.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.660+0000 7efee9200700 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e1030 0x5651088ef5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.660+0000 7efee9200700 1 -- 172.21.15.90:0/3908767314 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651088efd60 con 0x565108759e40 2024-05-27T07:52:24.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.661+0000 7efee9200700 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088f3c30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efed00a5690 tx=0x7efed00a9c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=846ed4a9fedeca87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:24.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.662+0000 7efecea00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:52:24.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.663+0000 7efecea00700 1 -- 172.21.15.90:0/3908767314 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0020070 con 0x565108759e40 2024-05-27T07:52:24.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.663+0000 7efecea00700 1 -- 172.21.15.90:0/3908767314 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7efed00106a0 con 0x565108759e40 2024-05-27T07:52:24.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.663+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3908767314 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed00ada10 con 0x565108759e40 2024-05-27T07:52:24.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.664+0000 7efecea00700 1 -- 172.21.15.90:0/3908767314 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7efed00b24c0 con 0x565108759e40 2024-05-27T07:52:24.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.664+0000 7efefa7a5b80 10 client.36185 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:24.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.664+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3908767314 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x565108976c20 con 0x565108759e40 2024-05-27T07:52:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.666+0000 7efecea00700 1 -- 172.21.15.90:0/3908767314 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efed00ada10 con 0x565108759e40 2024-05-27T07:52:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.666+0000 7efecea00700 1 -- 172.21.15.90:0/3908767314 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7efed008fc50 con 0x565108759e40 2024-05-27T07:52:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.666+0000 7efefa7a5b80 10 client.36185 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.666+0000 7efefa7a5b80 10 client.36185 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:52:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.666+0000 7efefa7a5b80 20 client.36185 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.666+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x565108977080 0x5651089794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.666+0000 7efefa7a5b80 4 client.36185 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:52:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.666+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3908767314 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565108802570 con 0x565108977080 2024-05-27T07:52:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.668+0000 7efee3e00700 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x565108977080 0x5651089794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:24.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.669+0000 7efecea00700 10 client.36185 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:52:24.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.672+0000 7efee3e00700 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x565108977080 0x5651089794a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=97f42b3f5eb86644 server_cookie=fb4a5da2316d0c5f in_seq=0 out_seq=0 2024-05-27T07:52:24.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.673+0000 7efecea00700 1 -- 172.21.15.90:0/3908767314 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565108802570 con 0x565108977080 2024-05-27T07:52:24.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.673+0000 7efecea00700 10 client.36185 handle_command_reply: tid=0 2024-05-27T07:52:24.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.673+0000 7efefa7a5b80 1 client.36185 shutdown 2024-05-27T07:52:24.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.674+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3908767314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 msgr2=0x5651088f3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:24.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.674+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088f3c30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efed00a5690 tx=0x7efed00a9c30 comp rx=0 tx=0).stop 2024-05-27T07:52:24.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.674+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3908767314 shutdown_connections 2024-05-27T07:52:24.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.674+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x565108977080 0x5651089794a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.674+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651088e1030 0x5651088ef5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.674+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565108759e40 0x5651088f3c30 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efed00a5690 tx=0x7efed00a9c30 comp rx=0 tx=0).stop 2024-05-27T07:52:24.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.674+0000 7efefa7a5b80 1 --2- 172.21.15.90:0/3908767314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651088e0c20 0x5651088eef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.674+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3908767314 >> 172.21.15.90:0/3908767314 conn(0x5651087b9fc0 msgr2=0x5651088f6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:24.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.675+0000 7efecea00700 0 client.36185 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:52:24.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.678+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3908767314 shutdown_connections 2024-05-27T07:52:24.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.678+0000 7efefa7a5b80 1 -- 172.21.15.90:0/3908767314 wait complete. 2024-05-27T07:52:24.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.679+0000 7efefa7a5b80 20 client.36185 trim_cache size 0 max 16384 2024-05-27T07:52:24.681 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:52:24.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.681+0000 7efeea600700 1 -- 172.21.15.90:0/3537844493 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efec80b2a00 msgr2=0x7efec80b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:24.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.681+0000 7efeea600700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efec80b2a00 0x7efec80b4f10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efed4009c40 tx=0x7efed4019040 comp rx=0 tx=0).stop 2024-05-27T07:52:24.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.681+0000 7efeea600700 1 -- 172.21.15.90:0/3537844493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efee41043f0 msgr2=0x7efee419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:24.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.681+0000 7efeea600700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efee41043f0 0x7efee419cee0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efed000d980 tx=0x7efed000dc90 comp rx=0 tx=0).stop 2024-05-27T07:52:24.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.681+0000 7efeea600700 1 -- 172.21.15.90:0/3537844493 shutdown_connections 2024-05-27T07:52:24.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.681+0000 7efeea600700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efec80b2a00 0x7efec80b4f10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.681+0000 7efeea600700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee4109670 0x7efee41a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.681+0000 7efeea600700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efee41043f0 0x7efee419cee0 secure :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efed000d980 tx=0x7efed000dc90 comp rx=0 tx=0).stop 2024-05-27T07:52:24.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.681+0000 7efeea600700 1 --2- 172.21.15.90:0/3537844493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efee4108d20 0x7efee419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:24.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.681+0000 7efeea600700 1 -- 172.21.15.90:0/3537844493 >> 172.21.15.90:0/3537844493 conn(0x7efee4079d00 msgr2=0x7efee410f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:24.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.684+0000 7efeea600700 1 -- 172.21.15.90:0/3537844493 shutdown_connections 2024-05-27T07:52:24.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:24.685+0000 7efeea600700 1 -- 172.21.15.90:0/3537844493 wait complete. 2024-05-27T07:52:26.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:26 smithi136 ceph-mon[35324]: pgmap v188: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.6 MiB/s wr, 624 op/s 2024-05-27T07:52:26.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:26 smithi129 ceph-mon[35254]: pgmap v188: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.6 MiB/s wr, 624 op/s 2024-05-27T07:52:26.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:26 smithi090 ceph-mon[31429]: pgmap v188: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.6 MiB/s wr, 624 op/s 2024-05-27T07:52:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.111+0000 7f5ae7e00640 1 -- 172.21.15.90:0/1928092096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8104bb0 msgr2=0x7f5ae8104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.111+0000 7f5ae7e00640 1 --2- 172.21.15.90:0/1928092096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8104bb0 0x7f5ae8104fc0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009900 tx=0x7f5ad802a410 comp rx=0 tx=0).stop 2024-05-27T07:52:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.111+0000 7f5ae7e00640 1 -- 172.21.15.90:0/1928092096 shutdown_connections 2024-05-27T07:52:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.111+0000 7f5ae7e00640 1 --2- 172.21.15.90:0/1928092096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ae810e000 0x7f5ae81104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.111+0000 7f5ae7e00640 1 --2- 172.21.15.90:0/1928092096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ae810b610 0x7f5ae810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.111+0000 7f5ae7e00640 1 --2- 172.21.15.90:0/1928092096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae8104bb0 0x7f5ae8104fc0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.111+0000 7f5ae7e00640 1 -- 172.21.15.90:0/1928092096 >> 172.21.15.90:0/1928092096 conn(0x7f5ae80fe210 msgr2=0x7f5ae8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.112+0000 7f5ae7e00640 1 -- 172.21.15.90:0/1928092096 shutdown_connections 2024-05-27T07:52:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.112+0000 7f5ae7e00640 1 -- 172.21.15.90:0/1928092096 wait complete. 2024-05-27T07:52:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7e00640 1 Processor -- start 2024-05-27T07:52:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7e00640 1 -- start start 2024-05-27T07:52:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ae8104bb0 0x7f5ae81a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ae810b610 0x7f5ae81a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae810e000 0x7f5ae819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae8113c20 con 0x7f5ae810e000 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae8113aa0 con 0x7f5ae8104bb0 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae8113da0 con 0x7f5ae810b610 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae810e000 0x7f5ae819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae810e000 0x7f5ae819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55844/0 (socket says 172.21.15.90:55844) 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.113+0000 7f5ae7400640 1 -- 172.21.15.90:0/184033129 learned_addr learned my addr 172.21.15.90:0/184033129 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ae8104bb0 0x7f5ae81a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5ae6000640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ae810b610 0x7f5ae81a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5ae7400640 1 -- 172.21.15.90:0/184033129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ae810b610 msgr2=0x7f5ae81a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5ae7400640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ae810b610 0x7f5ae81a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5ae7400640 1 -- 172.21.15.90:0/184033129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ae8104bb0 msgr2=0x7f5ae81a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5ae7400640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ae8104bb0 0x7f5ae81a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5ae7400640 1 -- 172.21.15.90:0/184033129 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae819cea0 con 0x7f5ae810e000 2024-05-27T07:52:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5ae7400640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae810e000 0x7f5ae819c7c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000b970 tx=0x7f5ae000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5adf400640 1 -- 172.21.15.90:0/184033129 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0019070 con 0x7f5ae810e000 2024-05-27T07:52:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5adf400640 1 -- 172.21.15.90:0/184033129 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ae0014440 con 0x7f5ae810e000 2024-05-27T07:52:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5adf400640 1 -- 172.21.15.90:0/184033129 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae00075d0 con 0x7f5ae810e000 2024-05-27T07:52:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5ae7e00640 1 -- 172.21.15.90:0/184033129 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae819d190 con 0x7f5ae810e000 2024-05-27T07:52:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.114+0000 7f5ae7e00640 1 -- 172.21.15.90:0/184033129 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae81a93e0 con 0x7f5ae810e000 2024-05-27T07:52:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.117+0000 7f5af45af740 1 Processor -- start 2024-05-27T07:52:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.118+0000 7f5adf400640 1 -- 172.21.15.90:0/184033129 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5ae001c070 con 0x7f5ae810e000 2024-05-27T07:52:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.118+0000 7f5af45af740 1 -- start start 2024-05-27T07:52:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5af45af740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44ed6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5af45af740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ed6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5af45af740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b134d0 0x558a44ed74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5af45af740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558a44e47b90 con 0x558a44b0b2f0 2024-05-27T07:52:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5af45af740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558a44e37730 con 0x558a44b341b0 2024-05-27T07:52:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5af45af740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558a44e3b770 con 0x558a44b134d0 2024-05-27T07:52:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5ae6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ed6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5ae6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ed6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55852/0 (socket says 172.21.15.90:55852) 2024-05-27T07:52:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5ae6a00640 1 -- 172.21.15.90:0/3700919218 learned_addr learned my addr 172.21.15.90:0/3700919218 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5ae6000640 1 --2- 172.21.15.90:0/3700919218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44ed6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5ae6a00640 1 -- 172.21.15.90:0/3700919218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b134d0 msgr2=0x558a44ed74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/3700919218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b134d0 0x558a44ed74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5ae6a00640 1 -- 172.21.15.90:0/3700919218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 msgr2=0x558a44ed6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/3700919218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44ed6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5ae6a00640 1 -- 172.21.15.90:0/3700919218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a44dd08b0 con 0x558a44b0b2f0 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.119+0000 7f5ae6000640 1 --2- 172.21.15.90:0/3700919218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44ed6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.120+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/3700919218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ed6fa0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8005ab0 tx=0x7f5ad8004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5408603c95410f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.120+0000 7f5ad3400640 1 -- 172.21.15.90:0/3700919218 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8035030 con 0x558a44b0b2f0 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.120+0000 7f5ad3400640 1 -- 172.21.15.90:0/3700919218 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad8005de0 con 0x558a44b0b2f0 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.120+0000 7f5ad3400640 1 -- 172.21.15.90:0/3700919218 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad803cb20 con 0x558a44b0b2f0 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.120+0000 7f5adf400640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5acc0b29d0 0x7f5acc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.120+0000 7f5adf400640 1 -- 172.21.15.90:0/184033129 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f5ae00e1ea0 con 0x7f5ae810e000 2024-05-27T07:52:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5ae6000640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5acc0b29d0 0x7f5acc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5af45af740 1 -- 172.21.15.90:0/3700919218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 msgr2=0x558a44ed6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5af45af740 1 --2- 172.21.15.90:0/3700919218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ed6fa0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8005ab0 tx=0x7f5ad8004390 comp rx=0 tx=0).stop 2024-05-27T07:52:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5af45af740 1 -- 172.21.15.90:0/3700919218 shutdown_connections 2024-05-27T07:52:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5af45af740 1 --2- 172.21.15.90:0/3700919218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b134d0 0x558a44ed74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5af45af740 1 --2- 172.21.15.90:0/3700919218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ed6fa0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5af45af740 1 --2- 172.21.15.90:0/3700919218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44ed6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5af45af740 1 -- 172.21.15.90:0/3700919218 >> 172.21.15.90:0/3700919218 conn(0x558a44b28300 msgr2=0x558a44ed3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5af45af740 1 -- 172.21.15.90:0/3700919218 shutdown_connections 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5ae6000640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5acc0b29d0 0x7f5acc0b4ee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x558a44ed3bd0 tx=0x7f5ad4008040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.121+0000 7f5af45af740 1 -- 172.21.15.90:0/3700919218 wait complete. 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.122+0000 7f5af45af740 1 -- start start 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.122+0000 7f5af45af740 1 Processor -- start 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.122+0000 7f5af45af740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b0b2f0 0x558a44ee0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.122+0000 7f5af45af740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.122+0000 7f5af45af740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b341b0 0x558a44eecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.122+0000 7f5af45af740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558a44b1d790 con 0x558a44b134d0 2024-05-27T07:52:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.122+0000 7f5af45af740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558a44ddfbc0 con 0x558a44b0b2f0 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.122+0000 7f5af45af740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558a44ee3b30 con 0x558a44b341b0 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.123+0000 7f5af45af740 10 client.0 _ll_register_callbacks cb 0x558a44dbd400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.123+0000 7f5ae6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.123+0000 7f5ae6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55866/0 (socket says 172.21.15.90:55866) 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.123+0000 7f5ae6000640 1 -- 172.21.15.90:0/1767426021 learned_addr learned my addr 172.21.15.90:0/1767426021 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.123+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b0b2f0 0x558a44ee0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.123+0000 7f5ae6000640 1 -- 172.21.15.90:0/1767426021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b341b0 msgr2=0x558a44eecec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.123+0000 7f5ae6000640 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b341b0 0x558a44eecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.123+0000 7f5ae6000640 1 -- 172.21.15.90:0/1767426021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b0b2f0 msgr2=0x558a44ee0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.123+0000 7f5ae6000640 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b0b2f0 0x558a44ee0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.123+0000 7f5ae6000640 1 -- 172.21.15.90:0/1767426021 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a44b4c210 con 0x558a44b134d0 2024-05-27T07:52:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.124+0000 7f5ae6000640 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee1140 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400dd60 tx=0x7f5ad4006b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0271c4d8f0241c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.124+0000 7f5ac7e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:52:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.124+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1767426021 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4024060 con 0x558a44b134d0 2024-05-27T07:52:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.124+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1767426021 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f5ad401f400 con 0x558a44b134d0 2024-05-27T07:52:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.124+0000 7f5af45af740 1 -- 172.21.15.90:0/1767426021 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558a44e49ae0 con 0x558a44b134d0 2024-05-27T07:52:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.125+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1767426021 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5ad4014e40 con 0x558a44b134d0 2024-05-27T07:52:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.125+0000 7f5af45af740 10 client.16695 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.125+0000 7f5af45af740 1 -- 172.21.15.90:0/1767426021 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558a44c18620 con 0x558a44b134d0 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.126+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1767426021 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x558a44e49ae0 con 0x558a44b134d0 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.126+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1767426021 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7f5ad4013070 con 0x558a44b134d0 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.127+0000 7f5af45af740 10 client.16695 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.127+0000 7f5af45af740 10 client.16695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.127+0000 7f5af45af740 20 client.16695 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.127+0000 7f5af45af740 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558a44af2d80 0x558a44ef2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.127+0000 7f5af45af740 4 client.16695 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.127+0000 7f5af45af740 1 -- 172.21.15.90:0/1767426021 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a44e432c0 con 0x558a44af2d80 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.128+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558a44af2d80 0x558a44ef2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.129+0000 7f5ac7e00640 10 client.16695 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.129+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558a44af2d80 0x558a44ef2de0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=959e637d4cb45fcb server_cookie=4a58e8b8fccb4356 in_seq=0 out_seq=0 2024-05-27T07:52:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.131+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1767426021 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558a44e432c0 con 0x558a44af2d80 2024-05-27T07:52:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.131+0000 7f5ac7e00640 10 client.16695 handle_command_reply: tid=0 2024-05-27T07:52:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.131+0000 7f5af45af740 1 client.16695 shutdown 2024-05-27T07:52:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.132+0000 7f5af45af740 1 -- 172.21.15.90:0/1767426021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 msgr2=0x558a44ee1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.132+0000 7f5af45af740 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee1140 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400dd60 tx=0x7f5ad4006b20 comp rx=0 tx=0).stop 2024-05-27T07:52:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.133+0000 7f5af45af740 1 -- 172.21.15.90:0/1767426021 shutdown_connections 2024-05-27T07:52:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.133+0000 7f5af45af740 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558a44af2d80 0x558a44ef2de0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.133+0000 7f5af45af740 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b341b0 0x558a44eecec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.133+0000 7f5af45af740 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee1140 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.133+0000 7f5af45af740 1 --2- 172.21.15.90:0/1767426021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b0b2f0 0x558a44ee0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.133+0000 7f5af45af740 1 -- 172.21.15.90:0/1767426021 >> 172.21.15.90:0/1767426021 conn(0x558a44b28300 msgr2=0x558a44ed6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.134+0000 7f5ac7e00640 0 client.16695 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.134+0000 7f5af45af740 1 -- 172.21.15.90:0/1767426021 shutdown_connections 2024-05-27T07:52:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.134+0000 7f5af45af740 1 -- 172.21.15.90:0/1767426021 wait complete. 2024-05-27T07:52:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.136+0000 7f5af45af740 20 client.16695 trim_cache size 0 max 16384 2024-05-27T07:52:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.165+0000 7f5af45af740 1 Processor -- start 2024-05-27T07:52:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.165+0000 7f5af45af740 1 -- start start 2024-05-27T07:52:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5af45af740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ee3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5af45af740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b134d0 0x558a44eecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5af45af740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44eed170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5af45af740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae81b4e60 con 0x558a44b0b2f0 2024-05-27T07:52:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5af45af740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae81b2c50 con 0x558a44b341b0 2024-05-27T07:52:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5af45af740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae81b4790 con 0x558a44b134d0 2024-05-27T07:52:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5ae6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ee3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5ae6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ee3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55882/0 (socket says 172.21.15.90:55882) 2024-05-27T07:52:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5ae6a00640 1 -- 172.21.15.90:0/495470384 learned_addr learned my addr 172.21.15.90:0/495470384 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5ae7400640 1 --2- 172.21.15.90:0/495470384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44eed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.166+0000 7f5ae6000640 1 --2- 172.21.15.90:0/495470384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b134d0 0x558a44eecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.167+0000 7f5ae6a00640 1 -- 172.21.15.90:0/495470384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b134d0 msgr2=0x558a44eecc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.167+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/495470384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b134d0 0x558a44eecc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.167+0000 7f5ae6a00640 1 -- 172.21.15.90:0/495470384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 msgr2=0x558a44eed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.167+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/495470384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44eed170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.167+0000 7f5ae6a00640 1 -- 172.21.15.90:0/495470384 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a44ed7e40 con 0x558a44b0b2f0 2024-05-27T07:52:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.167+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/495470384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ee3330 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5ad802a930 tx=0x7f5ad8004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56423fb1d5bfde88 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.168+0000 7f5ad2a00640 1 -- 172.21.15.90:0/495470384 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8035040 con 0x558a44b0b2f0 2024-05-27T07:52:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.169+0000 7f5ad2a00640 1 -- 172.21.15.90:0/495470384 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad803cc40 con 0x558a44b0b2f0 2024-05-27T07:52:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.169+0000 7f5ad2a00640 1 -- 172.21.15.90:0/495470384 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8031400 con 0x558a44b0b2f0 2024-05-27T07:52:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.170+0000 7f5af45af740 1 -- 172.21.15.90:0/495470384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 msgr2=0x558a44ee3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.170+0000 7f5af45af740 1 --2- 172.21.15.90:0/495470384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ee3330 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f5ad802a930 tx=0x7f5ad8004480 comp rx=0 tx=0).stop 2024-05-27T07:52:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.170+0000 7f5af45af740 1 -- 172.21.15.90:0/495470384 shutdown_connections 2024-05-27T07:52:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.170+0000 7f5af45af740 1 --2- 172.21.15.90:0/495470384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44eed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.170+0000 7f5af45af740 1 --2- 172.21.15.90:0/495470384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b134d0 0x558a44eecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.170+0000 7f5af45af740 1 --2- 172.21.15.90:0/495470384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b0b2f0 0x558a44ee3330 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.170+0000 7f5af45af740 1 -- 172.21.15.90:0/495470384 >> 172.21.15.90:0/495470384 conn(0x558a44b28300 msgr2=0x558a44ed68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.170+0000 7f5af45af740 1 -- 172.21.15.90:0/495470384 shutdown_connections 2024-05-27T07:52:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.170+0000 7f5af45af740 1 -- 172.21.15.90:0/495470384 wait complete. 2024-05-27T07:52:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.171+0000 7f5af45af740 1 -- start start 2024-05-27T07:52:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.171+0000 7f5af45af740 1 Processor -- start 2024-05-27T07:52:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.172+0000 7f5af45af740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b0b2f0 0x558a44ee3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.172+0000 7f5af45af740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.173+0000 7f5af45af740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44ee4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.173+0000 7f5af45af740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae81b39f0 con 0x558a44b134d0 2024-05-27T07:52:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.173+0000 7f5af45af740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae81b3320 con 0x558a44b341b0 2024-05-27T07:52:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.173+0000 7f5af45af740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae81b40c0 con 0x558a44b0b2f0 2024-05-27T07:52:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.173+0000 7f5af45af740 10 client.0 _ll_register_callbacks cb 0x7f5ad4009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.173+0000 7f5ae6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.173+0000 7f5ae6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55896/0 (socket says 172.21.15.90:55896) 2024-05-27T07:52:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.174+0000 7f5ae6000640 1 -- 172.21.15.90:0/1303060662 learned_addr learned my addr 172.21.15.90:0/1303060662 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.174+0000 7f5ae7400640 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44ee4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.174+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b0b2f0 0x558a44ee3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.175+0000 7f5ae6000640 1 -- 172.21.15.90:0/1303060662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b0b2f0 msgr2=0x558a44ee3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.175+0000 7f5ae6000640 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b0b2f0 0x558a44ee3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.175+0000 7f5ae6000640 1 -- 172.21.15.90:0/1303060662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 msgr2=0x558a44ee4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.175+0000 7f5ae6000640 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44ee4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.175+0000 7f5ae6000640 1 -- 172.21.15.90:0/1303060662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a44f595d0 con 0x558a44b134d0 2024-05-27T07:52:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.176+0000 7f5ae6000640 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee4320 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400dd30 tx=0x7f5ad4028dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b73ad8d216875b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.176+0000 7f5ac7e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:52:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.176+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1303060662 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4038060 con 0x558a44b134d0 2024-05-27T07:52:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.176+0000 7f5af45af740 1 -- 172.21.15.90:0/1303060662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ad803cc40 con 0x558a44b134d0 2024-05-27T07:52:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.177+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1303060662 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7f5ad4029830 con 0x558a44b134d0 2024-05-27T07:52:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.177+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1303060662 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5ad403f050 con 0x558a44b134d0 2024-05-27T07:52:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.177+0000 7f5af45af740 10 client.16701 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.177+0000 7f5af45af740 1 -- 172.21.15.90:0/1303060662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x558a44f63070 con 0x558a44b134d0 2024-05-27T07:52:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.178+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1303060662 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ad803cc40 con 0x558a44b134d0 2024-05-27T07:52:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.178+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1303060662 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7f5ad401f760 con 0x558a44b134d0 2024-05-27T07:52:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.178+0000 7f5af45af740 10 client.16701 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.178+0000 7f5af45af740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.178+0000 7f5af45af740 20 client.16701 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.178+0000 7f5af45af740 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558a44ac81c0 0x558a44f65860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.178+0000 7f5af45af740 4 client.16701 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:52:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.178+0000 7f5af45af740 1 -- 172.21.15.90:0/1303060662 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a44dbd4b0 con 0x558a44ac81c0 2024-05-27T07:52:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.179+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558a44ac81c0 0x558a44f65860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.182+0000 7f5ae6a00640 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558a44ac81c0 0x558a44f65860 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f1c2917472c3f0a server_cookie=f795488a7549a7a1 in_seq=0 out_seq=0 2024-05-27T07:52:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.182+0000 7f5ac7e00640 10 client.16701 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.249+0000 7f5ac7e00640 1 -- 172.21.15.90:0/1303060662 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558a44dbd4b0 con 0x558a44ac81c0 2024-05-27T07:52:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5ac7e00640 10 client.16701 handle_command_reply: tid=0 2024-05-27T07:52:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5af45af740 1 client.16701 shutdown 2024-05-27T07:52:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5af45af740 1 -- 172.21.15.90:0/1303060662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 msgr2=0x558a44ee4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5af45af740 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee4320 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400dd30 tx=0x7f5ad4028dc0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5af45af740 1 -- 172.21.15.90:0/1303060662 shutdown_connections 2024-05-27T07:52:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5af45af740 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558a44ac81c0 0x558a44f65860 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5af45af740 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a44b341b0 0x558a44ee4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5af45af740 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558a44b134d0 0x558a44ee4320 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5af45af740 1 --2- 172.21.15.90:0/1303060662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558a44b0b2f0 0x558a44ee3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5af45af740 1 -- 172.21.15.90:0/1303060662 >> 172.21.15.90:0/1303060662 conn(0x558a44b28300 msgr2=0x558a44ed72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.250+0000 7f5ac7e00640 0 client.16701 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.251+0000 7f5af45af740 1 -- 172.21.15.90:0/1303060662 shutdown_connections 2024-05-27T07:52:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.251+0000 7f5af45af740 1 -- 172.21.15.90:0/1303060662 wait complete. 2024-05-27T07:52:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.251+0000 7f5af45af740 20 client.16701 trim_cache size 0 max 16384 2024-05-27T07:52:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:52:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:52:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:52:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.253+0000 7f5ae7e00640 1 -- 172.21.15.90:0/184033129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5acc0b29d0 msgr2=0x7f5acc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.253+0000 7f5ae7e00640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5acc0b29d0 0x7f5acc0b4ee0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x558a44ed3bd0 tx=0x7f5ad4008040 comp rx=0 tx=0).stop 2024-05-27T07:52:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.253+0000 7f5ae7e00640 1 -- 172.21.15.90:0/184033129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae810e000 msgr2=0x7f5ae819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.253+0000 7f5ae7e00640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae810e000 0x7f5ae819c7c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000b970 tx=0x7f5ae000be30 comp rx=0 tx=0).stop 2024-05-27T07:52:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.253+0000 7f5ae7e00640 1 -- 172.21.15.90:0/184033129 shutdown_connections 2024-05-27T07:52:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.253+0000 7f5ae7e00640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5acc0b29d0 0x7f5acc0b4ee0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.253+0000 7f5ae7e00640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae810e000 0x7f5ae819c7c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.253+0000 7f5ae7e00640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ae810b610 0x7f5ae81a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.253+0000 7f5ae7e00640 1 --2- 172.21.15.90:0/184033129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ae8104bb0 0x7f5ae81a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.253+0000 7f5ae7e00640 1 -- 172.21.15.90:0/184033129 >> 172.21.15.90:0/184033129 conn(0x7f5ae80fe210 msgr2=0x7f5ae8109d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.254+0000 7f5ae7e00640 1 -- 172.21.15.90:0/184033129 shutdown_connections 2024-05-27T07:52:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:27.254+0000 7f5ae7e00640 1 -- 172.21.15.90:0/184033129 wait complete. 2024-05-27T07:52:28.213 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:52:28.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:28 smithi090 ceph-mon[31429]: pgmap v189: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.7 MiB/s wr, 559 op/s 2024-05-27T07:52:28.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:28 smithi136 ceph-mon[35324]: pgmap v189: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.7 MiB/s wr, 559 op/s 2024-05-27T07:52:28.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:28 smithi129 ceph-mon[35254]: pgmap v189: 129 pgs: 129 active+clean; 1.8 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.7 MiB/s wr, 559 op/s 2024-05-27T07:52:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 130.0 s, 539.6 tps, lat 185.342 ms stddev 128.193 2024-05-27T07:52:28.887 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:30.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:30 smithi090 ceph-mon[31429]: pgmap v190: 129 pgs: 129 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.9 MiB/s wr, 826 op/s 2024-05-27T07:52:30.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:30 smithi129 ceph-mon[35254]: pgmap v190: 129 pgs: 129 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.9 MiB/s wr, 826 op/s 2024-05-27T07:52:30.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:30 smithi136 ceph-mon[35324]: pgmap v190: 129 pgs: 129 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.9 MiB/s wr, 826 op/s 2024-05-27T07:52:31.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:31 smithi090 ceph-mon[31429]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.5 MiB/s wr, 544 op/s 2024-05-27T07:52:31.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.538+0000 7fa566800700 1 -- 172.21.15.90:0/728068637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5600764a0 msgr2=0x7fa560114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:31.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.538+0000 7fa566800700 1 --2- 172.21.15.90:0/728068637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5600764a0 0x7fa560114e20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa548009990 tx=0x7fa548009ca0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.538+0000 7fa566800700 1 -- 172.21.15.90:0/728068637 shutdown_connections 2024-05-27T07:52:31.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.538+0000 7fa566800700 1 --2- 172.21.15.90:0/728068637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa560115360 0x7fa560117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.538+0000 7fa566800700 1 --2- 172.21.15.90:0/728068637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa560075b50 0x7fa560075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.538+0000 7fa566800700 1 --2- 172.21.15.90:0/728068637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5600764a0 0x7fa560114e20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.538+0000 7fa566800700 1 -- 172.21.15.90:0/728068637 >> 172.21.15.90:0/728068637 conn(0x7fa560100030 msgr2=0x7fa560102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:31.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.538+0000 7fa566800700 1 -- 172.21.15.90:0/728068637 shutdown_connections 2024-05-27T07:52:31.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.539+0000 7fa566800700 1 -- 172.21.15.90:0/728068637 wait complete. 2024-05-27T07:52:31.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.540+0000 7fa566800700 1 Processor -- start 2024-05-27T07:52:31.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.541+0000 7fa566800700 1 -- start start 2024-05-27T07:52:31.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.541+0000 7fa566800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa560075b50 0x7fa5601a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.541+0000 7fa566800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5600764a0 0x7fa5601a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.541+0000 7fa566800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560115360 0x7fa5601a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.541+0000 7fa566800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa56011a960 con 0x7fa5600764a0 2024-05-27T07:52:31.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.541+0000 7fa566800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa56011a7e0 con 0x7fa560075b50 2024-05-27T07:52:31.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.541+0000 7fa566800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa56011aae0 con 0x7fa560115360 2024-05-27T07:52:31.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.542+0000 7fa55fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560115360 0x7fa5601a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.542+0000 7fa55f400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa560075b50 0x7fa5601a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.542+0000 7fa55f400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa560075b50 0x7fa5601a1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50376/0 (socket says 172.21.15.90:50376) 2024-05-27T07:52:31.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.542+0000 7fa55f400700 1 -- 172.21.15.90:0/563622299 learned_addr learned my addr 172.21.15.90:0/563622299 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:31.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.543+0000 7fa55fe00700 1 -- 172.21.15.90:0/563622299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa560075b50 msgr2=0x7fa5601a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:31.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.543+0000 7fa55fe00700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa560075b50 0x7fa5601a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.543+0000 7fa55fe00700 1 -- 172.21.15.90:0/563622299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5600764a0 msgr2=0x7fa5601a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:31.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.543+0000 7fa55fe00700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5600764a0 0x7fa5601a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.543+0000 7fa55fe00700 1 -- 172.21.15.90:0/563622299 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5601a6260 con 0x7fa560115360 2024-05-27T07:52:31.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.543+0000 7fa55fe00700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560115360 0x7fa5601a5bc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa558007f10 tx=0x7fa55800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:31.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.544+0000 7fa557e00700 1 -- 172.21.15.90:0/563622299 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55800ddd0 con 0x7fa560115360 2024-05-27T07:52:31.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.544+0000 7fa557e00700 1 -- 172.21.15.90:0/563622299 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa558009e20 con 0x7fa560115360 2024-05-27T07:52:31.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.544+0000 7fa566800700 1 -- 172.21.15.90:0/563622299 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5601a6550 con 0x7fa560115360 2024-05-27T07:52:31.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.544+0000 7fa557e00700 1 -- 172.21.15.90:0/563622299 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558015400 con 0x7fa560115360 2024-05-27T07:52:31.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.545+0000 7fa566800700 1 -- 172.21.15.90:0/563622299 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5601085f0 con 0x7fa560115360 2024-05-27T07:52:31.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.552+0000 7fa575417b80 1 Processor -- start 2024-05-27T07:52:31.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.553+0000 7fa557e00700 1 -- 172.21.15.90:0/563622299 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa558004ad0 con 0x7fa560115360 2024-05-27T07:52:31.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.556+0000 7fa557e00700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa5440b2a70 0x7fa5440b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.556+0000 7fa575417b80 1 -- start start 2024-05-27T07:52:31.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.556+0000 7fa557e00700 1 -- 172.21.15.90:0/563622299 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fa5580e2290 con 0x7fa560115360 2024-05-27T07:52:31.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.557+0000 7fa575417b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 0x557b1e505330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.557+0000 7fa55ea00700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa5440b2a70 0x7fa5440b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.557+0000 7fa575417b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e505870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.557+0000 7fa575417b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e512110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.558+0000 7fa575417b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557b1e433e40 con 0x557b1e386e40 2024-05-27T07:52:31.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.558+0000 7fa575417b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557b1e4239e0 con 0x557b1e505e70 2024-05-27T07:52:31.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.558+0000 7fa575417b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557b1e427a20 con 0x557b1e4729f0 2024-05-27T07:52:31.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.558+0000 7fa55ea00700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa5440b2a70 0x7fa5440b4f80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa5480098e0 tx=0x7fa5480095d0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:31.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.559+0000 7fa55fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e512110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.559+0000 7fa55fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e512110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50392/0 (socket says 172.21.15.90:50392) 2024-05-27T07:52:31.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.559+0000 7fa55fe00700 1 -- 172.21.15.90:0/344288736 learned_addr learned my addr 172.21.15.90:0/344288736 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:31.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.559+0000 7fa55ea00700 1 --2- 172.21.15.90:0/344288736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e505870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.559+0000 7fa55f400700 1 --2- 172.21.15.90:0/344288736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 0x557b1e505330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.560+0000 7fa55fe00700 1 -- 172.21.15.90:0/344288736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 msgr2=0x557b1e505330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:31.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.560+0000 7fa55fe00700 1 --2- 172.21.15.90:0/344288736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 0x557b1e505330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.560+0000 7fa55fe00700 1 -- 172.21.15.90:0/344288736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 msgr2=0x557b1e505870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:31.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.560+0000 7fa55fe00700 1 --2- 172.21.15.90:0/344288736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e505870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.560+0000 7fa55fe00700 1 -- 172.21.15.90:0/344288736 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1e3ec420 con 0x557b1e505e70 2024-05-27T07:52:31.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.560+0000 7fa55fe00700 1 --2- 172.21.15.90:0/344288736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e512110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5580a8ec0 tx=0x7fa5580a5130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3361befbdf34c84 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:31.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.561+0000 7fa554c00700 1 -- 172.21.15.90:0/344288736 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5580ad570 con 0x557b1e505e70 2024-05-27T07:52:31.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.563+0000 7fa554c00700 1 -- 172.21.15.90:0/344288736 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5580b2020 con 0x557b1e505e70 2024-05-27T07:52:31.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:31 smithi129 ceph-mon[35254]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.5 MiB/s wr, 544 op/s 2024-05-27T07:52:31.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.564+0000 7fa554c00700 1 -- 172.21.15.90:0/344288736 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5580ad570 con 0x557b1e505e70 2024-05-27T07:52:31.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.564+0000 7fa575417b80 1 -- 172.21.15.90:0/344288736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 msgr2=0x557b1e512110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:31.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.564+0000 7fa575417b80 1 --2- 172.21.15.90:0/344288736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e512110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5580a8ec0 tx=0x7fa5580a5130 comp rx=0 tx=0).stop 2024-05-27T07:52:31.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.564+0000 7fa575417b80 1 -- 172.21.15.90:0/344288736 shutdown_connections 2024-05-27T07:52:31.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.564+0000 7fa575417b80 1 --2- 172.21.15.90:0/344288736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e512110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.564+0000 7fa575417b80 1 --2- 172.21.15.90:0/344288736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 0x557b1e505330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.564+0000 7fa575417b80 1 --2- 172.21.15.90:0/344288736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e505870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.564+0000 7fa575417b80 1 -- 172.21.15.90:0/344288736 >> 172.21.15.90:0/344288736 conn(0x557b1e420390 msgr2=0x557b1e5014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.565+0000 7fa575417b80 1 -- 172.21.15.90:0/344288736 shutdown_connections 2024-05-27T07:52:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.565+0000 7fa575417b80 1 -- 172.21.15.90:0/344288736 wait complete. 2024-05-27T07:52:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.565+0000 7fa575417b80 1 -- start start 2024-05-27T07:52:31.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.566+0000 7fa575417b80 1 Processor -- start 2024-05-27T07:52:31.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.567+0000 7fa575417b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e50f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.567+0000 7fa575417b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 0x557b1e50faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.567+0000 7fa575417b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e50ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.567+0000 7fa575417b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557b1e329fa0 con 0x557b1e386e40 2024-05-27T07:52:31.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.567+0000 7fa575417b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557b1e3c4760 con 0x557b1e505e70 2024-05-27T07:52:31.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.567+0000 7fa575417b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557b1e512c20 con 0x557b1e4729f0 2024-05-27T07:52:31.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.567+0000 7fa575417b80 10 client.0 _ll_register_callbacks cb 0x557b1e3cdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:31.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.568+0000 7fa55f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e50f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.568+0000 7fa55f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e50f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52232/0 (socket says 172.21.15.90:52232) 2024-05-27T07:52:31.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.568+0000 7fa55f400700 1 -- 172.21.15.90:0/3779074740 learned_addr learned my addr 172.21.15.90:0/3779074740 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:31.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.568+0000 7fa55ea00700 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 0x557b1e50faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.568+0000 7fa55fe00700 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e50ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.568+0000 7fa55ea00700 1 -- 172.21.15.90:0/3779074740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 msgr2=0x557b1e50ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.569+0000 7fa55ea00700 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e50ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.569+0000 7fa55ea00700 1 -- 172.21.15.90:0/3779074740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 msgr2=0x557b1e50f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.569+0000 7fa55ea00700 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e50f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.569+0000 7fa55ea00700 1 -- 172.21.15.90:0/3779074740 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1e3738d0 con 0x557b1e4729f0 2024-05-27T07:52:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.569+0000 7fa55f400700 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e50f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.569+0000 7fa55fe00700 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e50ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.569+0000 7fa55ea00700 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 0x557b1e50faa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa54800b2e0 tx=0x7fa548000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=658ab42a544b428a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:31.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.570+0000 7fa54d600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:52:31.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.570+0000 7fa54d600700 1 -- 172.21.15.90:0/3779074740 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548005690 con 0x557b1e4729f0 2024-05-27T07:52:31.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.570+0000 7fa54d600700 1 -- 172.21.15.90:0/3779074740 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fa54800f460 con 0x557b1e4729f0 2024-05-27T07:52:31.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.570+0000 7fa575417b80 1 -- 172.21.15.90:0/3779074740 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5580b2020 con 0x557b1e4729f0 2024-05-27T07:52:31.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.571+0000 7fa54d600700 1 -- 172.21.15.90:0/3779074740 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa54800f660 con 0x557b1e4729f0 2024-05-27T07:52:31.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.571+0000 7fa575417b80 10 client.26257 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:31.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.571+0000 7fa575417b80 1 -- 172.21.15.90:0/3779074740 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557b1e3f4060 con 0x557b1e4729f0 2024-05-27T07:52:31.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.571+0000 7fa54d600700 1 -- 172.21.15.90:0/3779074740 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5580b2020 con 0x557b1e4729f0 2024-05-27T07:52:31.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.572+0000 7fa54d600700 1 -- 172.21.15.90:0/3779074740 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fa54802d020 con 0x557b1e4729f0 2024-05-27T07:52:31.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.572+0000 7fa575417b80 10 client.26257 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:31.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.572+0000 7fa575417b80 10 client.26257 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:52:31.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.572+0000 7fa575417b80 20 client.26257 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:31.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.572+0000 7fa575417b80 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x557b1e51fa70 0x557b1e521ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.572+0000 7fa575417b80 4 client.26257 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:31.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.572+0000 7fa575417b80 1 -- 172.21.15.90:0/3779074740 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b1e435d90 con 0x557b1e51fa70 2024-05-27T07:52:31.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.573+0000 7fa55f400700 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x557b1e51fa70 0x557b1e521ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.575+0000 7fa54d600700 10 client.26257 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:52:31.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.575+0000 7fa55f400700 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x557b1e51fa70 0x557b1e521ec0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f1a27ebdffeb65e server_cookie=a06e4b205055c900 in_seq=0 out_seq=0 2024-05-27T07:52:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.580+0000 7fa54d600700 1 -- 172.21.15.90:0/3779074740 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557b1e435d90 con 0x557b1e51fa70 2024-05-27T07:52:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.580+0000 7fa54d600700 10 client.26257 handle_command_reply: tid=0 2024-05-27T07:52:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.580+0000 7fa575417b80 1 client.26257 shutdown 2024-05-27T07:52:31.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.581+0000 7fa575417b80 1 -- 172.21.15.90:0/3779074740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 msgr2=0x557b1e50faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:31.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.581+0000 7fa575417b80 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 0x557b1e50faa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa54800b2e0 tx=0x7fa548000bc0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.581+0000 7fa575417b80 1 -- 172.21.15.90:0/3779074740 shutdown_connections 2024-05-27T07:52:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.581+0000 7fa575417b80 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x557b1e51fa70 0x557b1e521ec0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.581+0000 7fa575417b80 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e505e70 0x557b1e50ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.581+0000 7fa575417b80 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e50f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.581+0000 7fa575417b80 1 --2- 172.21.15.90:0/3779074740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e4729f0 0x557b1e50faa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.581+0000 7fa575417b80 1 -- 172.21.15.90:0/3779074740 >> 172.21.15.90:0/3779074740 conn(0x557b1e420390 msgr2=0x557b1e505310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:31.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.581+0000 7fa54d600700 0 client.26257 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:52:31.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.582+0000 7fa575417b80 1 -- 172.21.15.90:0/3779074740 shutdown_connections 2024-05-27T07:52:31.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.582+0000 7fa575417b80 1 -- 172.21.15.90:0/3779074740 wait complete. 2024-05-27T07:52:31.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.582+0000 7fa575417b80 20 client.26257 trim_cache size 0 max 16384 2024-05-27T07:52:31.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.610+0000 7fa575417b80 1 Processor -- start 2024-05-27T07:52:31.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.611+0000 7fa575417b80 1 -- start start 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.611+0000 7fa575417b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e386e40 0x557b1e50ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.611+0000 7fa575417b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 0x557b1e5014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.611+0000 7fa575417b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e505e70 0x557b1e51bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.611+0000 7fa575417b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c0108a0 con 0x557b1e505e70 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.611+0000 7fa575417b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c00e690 con 0x557b1e4729f0 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.611+0000 7fa575417b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c0101d0 con 0x557b1e386e40 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.612+0000 7fa55fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e505e70 0x557b1e51bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.612+0000 7fa55ea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 0x557b1e5014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.612+0000 7fa55fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e505e70 0x557b1e51bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52242/0 (socket says 172.21.15.90:52242) 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.612+0000 7fa55fe00700 1 -- 172.21.15.90:0/2539896750 learned_addr learned my addr 172.21.15.90:0/2539896750 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:31.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.612+0000 7fa55f400700 1 --2- 172.21.15.90:0/2539896750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e386e40 0x557b1e50ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.613+0000 7fa55f400700 1 -- 172.21.15.90:0/2539896750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 msgr2=0x557b1e5014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:31.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.613+0000 7fa55f400700 1 --2- 172.21.15.90:0/2539896750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 0x557b1e5014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.613+0000 7fa55f400700 1 -- 172.21.15.90:0/2539896750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e505e70 msgr2=0x557b1e51bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:31.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.613+0000 7fa55f400700 1 --2- 172.21.15.90:0/2539896750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e505e70 0x557b1e51bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.613+0000 7fa55f400700 1 -- 172.21.15.90:0/2539896750 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1e510350 con 0x557b1e386e40 2024-05-27T07:52:31.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.614+0000 7fa55f400700 1 --2- 172.21.15.90:0/2539896750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e386e40 0x557b1e50ad50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x557b1e51fe80 tx=0x7fa550009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ed6ae70455365e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:31.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.614+0000 7fa54fe00700 1 -- 172.21.15.90:0/2539896750 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550004830 con 0x557b1e386e40 2024-05-27T07:52:31.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.614+0000 7fa54fe00700 1 -- 172.21.15.90:0/2539896750 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa550005020 con 0x557b1e386e40 2024-05-27T07:52:31.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.614+0000 7fa54fe00700 1 -- 172.21.15.90:0/2539896750 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550004e60 con 0x557b1e386e40 2024-05-27T07:52:31.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.616+0000 7fa575417b80 1 -- 172.21.15.90:0/2539896750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e386e40 msgr2=0x557b1e50ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:31.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.616+0000 7fa575417b80 1 --2- 172.21.15.90:0/2539896750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e386e40 0x557b1e50ad50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x557b1e51fe80 tx=0x7fa550009260 comp rx=0 tx=0).stop 2024-05-27T07:52:31.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.616+0000 7fa575417b80 1 -- 172.21.15.90:0/2539896750 shutdown_connections 2024-05-27T07:52:31.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.616+0000 7fa575417b80 1 --2- 172.21.15.90:0/2539896750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e505e70 0x557b1e51bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.616+0000 7fa575417b80 1 --2- 172.21.15.90:0/2539896750 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e386e40 0x557b1e50ad50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.616+0000 7fa575417b80 1 --2- 172.21.15.90:0/2539896750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 0x557b1e5014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.616+0000 7fa575417b80 1 -- 172.21.15.90:0/2539896750 >> 172.21.15.90:0/2539896750 conn(0x557b1e420390 msgr2=0x557b1e506ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:31.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.618+0000 7fa575417b80 1 -- 172.21.15.90:0/2539896750 shutdown_connections 2024-05-27T07:52:31.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.618+0000 7fa575417b80 1 -- 172.21.15.90:0/2539896750 wait complete. 2024-05-27T07:52:31.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.620+0000 7fa575417b80 1 -- start start 2024-05-27T07:52:31.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.620+0000 7fa575417b80 1 Processor -- start 2024-05-27T07:52:31.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.621+0000 7fa575417b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e518490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.621+0000 7fa575417b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 0x557b1e50c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.621+0000 7fa575417b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e505e70 0x557b1e50c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.621+0000 7fa575417b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c00f430 con 0x557b1e386e40 2024-05-27T07:52:31.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.621+0000 7fa575417b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c00ed60 con 0x557b1e4729f0 2024-05-27T07:52:31.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.621+0000 7fa575417b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa53c00fb00 con 0x557b1e505e70 2024-05-27T07:52:31.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.621+0000 7fa575417b80 10 client.0 _ll_register_callbacks cb 0x557b1e3cdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:31.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.622+0000 7fa55ea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 0x557b1e50c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.622+0000 7fa55fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e505e70 0x557b1e50c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.622+0000 7fa55fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e505e70 0x557b1e50c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:40416/0 (socket says 172.21.15.90:40416) 2024-05-27T07:52:31.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.622+0000 7fa55fe00700 1 -- 172.21.15.90:0/1844799688 learned_addr learned my addr 172.21.15.90:0/1844799688 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:31.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.622+0000 7fa55f400700 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e518490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.623+0000 7fa55ea00700 1 -- 172.21.15.90:0/1844799688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e505e70 msgr2=0x557b1e50c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:31.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.623+0000 7fa55ea00700 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e505e70 0x557b1e50c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.623+0000 7fa55ea00700 1 -- 172.21.15.90:0/1844799688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 msgr2=0x557b1e518490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:31.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.624+0000 7fa55ea00700 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e518490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.624+0000 7fa55ea00700 1 -- 172.21.15.90:0/1844799688 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1e50cae0 con 0x557b1e4729f0 2024-05-27T07:52:31.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.624+0000 7fa55f400700 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e518490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:31.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.625+0000 7fa55ea00700 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 0x557b1e50c060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa548009990 tx=0x7fa548005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc996c4326c2f6b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:31.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.625+0000 7fa54d600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:52:31.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.625+0000 7fa54d600700 1 -- 172.21.15.90:0/1844799688 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54800f660 con 0x557b1e4729f0 2024-05-27T07:52:31.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.625+0000 7fa575417b80 1 -- 172.21.15.90:0/1844799688 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa550005020 con 0x557b1e4729f0 2024-05-27T07:52:31.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.626+0000 7fa54d600700 1 -- 172.21.15.90:0/1844799688 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fa54800fa00 con 0x557b1e4729f0 2024-05-27T07:52:31.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.626+0000 7fa54d600700 1 -- 172.21.15.90:0/1844799688 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa54804a050 con 0x557b1e4729f0 2024-05-27T07:52:31.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.626+0000 7fa575417b80 10 client.36212 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:31.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.626+0000 7fa575417b80 1 -- 172.21.15.90:0/1844799688 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557b1e516c10 con 0x557b1e4729f0 2024-05-27T07:52:31.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.627+0000 7fa54d600700 1 -- 172.21.15.90:0/1844799688 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa550005020 con 0x557b1e4729f0 2024-05-27T07:52:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.628+0000 7fa54d600700 1 -- 172.21.15.90:0/1844799688 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fa548040030 con 0x557b1e4729f0 2024-05-27T07:52:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.628+0000 7fa575417b80 10 client.36212 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.628+0000 7fa575417b80 10 client.36212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:52:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.628+0000 7fa575417b80 20 client.36212 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.628+0000 7fa575417b80 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x557b1e58a8d0 0x557b1e58ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.628+0000 7fa575417b80 4 client.36212 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:52:31.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.628+0000 7fa575417b80 1 -- 172.21.15.90:0/1844799688 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b1e500120 con 0x557b1e58a8d0 2024-05-27T07:52:31.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.629+0000 7fa55f400700 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x557b1e58a8d0 0x557b1e58ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:31.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.630+0000 7fa55f400700 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x557b1e58a8d0 0x557b1e58ccf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=34cb5edd54557dfb server_cookie=c7eed02cff929194 in_seq=0 out_seq=0 2024-05-27T07:52:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.630+0000 7fa54d600700 10 client.36212 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:52:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.630+0000 7fa54d600700 1 -- 172.21.15.90:0/1844799688 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b1e500120 con 0x557b1e58a8d0 2024-05-27T07:52:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.630+0000 7fa54d600700 10 client.36212 handle_command_reply: tid=0 2024-05-27T07:52:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.630+0000 7fa575417b80 1 client.36212 shutdown 2024-05-27T07:52:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.631+0000 7fa575417b80 1 -- 172.21.15.90:0/1844799688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 msgr2=0x557b1e50c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.631+0000 7fa575417b80 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 0x557b1e50c060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa548009990 tx=0x7fa548005620 comp rx=0 tx=0).stop 2024-05-27T07:52:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.631+0000 7fa575417b80 1 -- 172.21.15.90:0/1844799688 shutdown_connections 2024-05-27T07:52:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.631+0000 7fa575417b80 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x557b1e58a8d0 0x557b1e58ccf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.631+0000 7fa575417b80 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b1e505e70 0x557b1e50c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.631+0000 7fa575417b80 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557b1e386e40 0x557b1e518490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.631+0000 7fa575417b80 1 --2- 172.21.15.90:0/1844799688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557b1e4729f0 0x557b1e50c060 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.631+0000 7fa575417b80 1 -- 172.21.15.90:0/1844799688 >> 172.21.15.90:0/1844799688 conn(0x557b1e420390 msgr2=0x557b1e506ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:31.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.631+0000 7fa54d600700 0 client.36212 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:52:31.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.631+0000 7fa575417b80 1 -- 172.21.15.90:0/1844799688 shutdown_connections 2024-05-27T07:52:31.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.632+0000 7fa575417b80 1 -- 172.21.15.90:0/1844799688 wait complete. 2024-05-27T07:52:31.633 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:52:31.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.632+0000 7fa575417b80 20 client.36212 trim_cache size 0 max 16384 2024-05-27T07:52:31.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 -- 172.21.15.90:0/563622299 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa5440b2a70 msgr2=0x7fa5440b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:31.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa5440b2a70 0x7fa5440b4f80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa5480098e0 tx=0x7fa5480095d0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 -- 172.21.15.90:0/563622299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560115360 msgr2=0x7fa5601a5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:31.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560115360 0x7fa5601a5bc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa558007f10 tx=0x7fa55800d3b0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 -- 172.21.15.90:0/563622299 shutdown_connections 2024-05-27T07:52:31.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa560115360 0x7fa5601a5bc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa5440b2a70 0x7fa5440b4f80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa560075b50 0x7fa5601a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 --2- 172.21.15.90:0/563622299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5600764a0 0x7fa5601a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:31.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 -- 172.21.15.90:0/563622299 >> 172.21.15.90:0/563622299 conn(0x7fa560100030 msgr2=0x7fa560101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:31.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.633+0000 7fa566800700 1 -- 172.21.15.90:0/563622299 shutdown_connections 2024-05-27T07:52:31.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:31.634+0000 7fa566800700 1 -- 172.21.15.90:0/563622299 wait complete. 2024-05-27T07:52:31.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:31 smithi136 ceph-mon[35324]: pgmap v191: 129 pgs: 129 active+clean; 1.8 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.5 MiB/s wr, 544 op/s 2024-05-27T07:52:32.636 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:52:33.244 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:33.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 135.0 s, 564.2 tps, lat 177.741 ms stddev 148.964 2024-05-27T07:52:34.419 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:34 smithi090 ceph-mon[31429]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.2 MiB/s wr, 756 op/s 2024-05-27T07:52:34.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:34 smithi136 ceph-mon[35324]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.2 MiB/s wr, 756 op/s 2024-05-27T07:52:34.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:34 smithi129 ceph-mon[35254]: pgmap v192: 129 pgs: 129 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.2 MiB/s wr, 756 op/s 2024-05-27T07:52:36.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:36 smithi090 ceph-mon[31429]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.8 MiB/s wr, 696 op/s 2024-05-27T07:52:36.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:36 smithi136 ceph-mon[35324]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.8 MiB/s wr, 696 op/s 2024-05-27T07:52:36.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:36 smithi129 ceph-mon[35254]: pgmap v193: 129 pgs: 129 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.8 MiB/s wr, 696 op/s 2024-05-27T07:52:38.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:38 smithi090 ceph-mon[31429]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 585 op/s 2024-05-27T07:52:38.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:38 smithi136 ceph-mon[35324]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 585 op/s 2024-05-27T07:52:38.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:38 smithi129 ceph-mon[35254]: pgmap v194: 129 pgs: 129 active+clean; 1.8 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.8 MiB/s wr, 585 op/s 2024-05-27T07:52:38.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.694+0000 7fad87a00700 1 -- 172.21.15.90:0/3567651948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad80105e60 msgr2=0x7fad8010e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:38.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.694+0000 7fad87a00700 1 --2- 172.21.15.90:0/3567651948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad80105e60 0x7fad8010e3b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009990 tx=0x7fad6c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.694+0000 7fad87a00700 1 -- 172.21.15.90:0/3567651948 shutdown_connections 2024-05-27T07:52:38.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.694+0000 7fad87a00700 1 --2- 172.21.15.90:0/3567651948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad8010e8f0 0x7fad80110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.694+0000 7fad87a00700 1 --2- 172.21.15.90:0/3567651948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad80105510 0x7fad80105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.694+0000 7fad87a00700 1 --2- 172.21.15.90:0/3567651948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad80105e60 0x7fad8010e3b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.694+0000 7fad87a00700 1 -- 172.21.15.90:0/3567651948 >> 172.21.15.90:0/3567651948 conn(0x7fad800fedb0 msgr2=0x7fad801011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:38.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.695+0000 7fad87a00700 1 -- 172.21.15.90:0/3567651948 shutdown_connections 2024-05-27T07:52:38.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.696+0000 7fad87a00700 1 -- 172.21.15.90:0/3567651948 wait complete. 2024-05-27T07:52:38.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.698+0000 7fad87a00700 1 Processor -- start 2024-05-27T07:52:38.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.698+0000 7fad87a00700 1 -- start start 2024-05-27T07:52:38.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.701+0000 7fad87a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad80105510 0x7fad8019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.701+0000 7fad87a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad80105e60 0x7fad8019d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.701+0000 7fad87a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad8010e8f0 0x7fad801a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.702+0000 7fad87a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fad80104b60 con 0x7fad80105e60 2024-05-27T07:52:38.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.702+0000 7fad87a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad801049e0 con 0x7fad80105510 2024-05-27T07:52:38.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.702+0000 7fad87a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fad80104ce0 con 0x7fad8010e8f0 2024-05-27T07:52:38.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.702+0000 7fad85000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad80105510 0x7fad8019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.702+0000 7fad7fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad80105e60 0x7fad8019d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.702+0000 7fad85000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad80105510 0x7fad8019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50432/0 (socket says 172.21.15.90:50432) 2024-05-27T07:52:38.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.702+0000 7fad85000700 1 -- 172.21.15.90:0/3834981210 learned_addr learned my addr 172.21.15.90:0/3834981210 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:38.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.702+0000 7fad85a00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad8010e8f0 0x7fad801a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.703+0000 7fad85a00700 1 -- 172.21.15.90:0/3834981210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad80105510 msgr2=0x7fad8019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:38.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.703+0000 7fad85a00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad80105510 0x7fad8019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.703+0000 7fad85a00700 1 -- 172.21.15.90:0/3834981210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad80105e60 msgr2=0x7fad8019d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:38.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.703+0000 7fad85a00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad80105e60 0x7fad8019d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.703+0000 7fad85a00700 1 -- 172.21.15.90:0/3834981210 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad801a1e60 con 0x7fad8010e8f0 2024-05-27T07:52:38.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.703+0000 7fad85000700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad80105510 0x7fad8019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:38.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.704+0000 7fad85a00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad8010e8f0 0x7fad801a17c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fad7800efc0 tx=0x7fad7800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:38.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.705+0000 7fad7d600700 1 -- 172.21.15.90:0/3834981210 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780092e0 con 0x7fad8010e8f0 2024-05-27T07:52:38.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.705+0000 7fad7d600700 1 -- 172.21.15.90:0/3834981210 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad78004bb0 con 0x7fad8010e8f0 2024-05-27T07:52:38.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.705+0000 7fad7d600700 1 -- 172.21.15.90:0/3834981210 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad78004020 con 0x7fad8010e8f0 2024-05-27T07:52:38.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.705+0000 7fad87a00700 1 -- 172.21.15.90:0/3834981210 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad801a20f0 con 0x7fad8010e8f0 2024-05-27T07:52:38.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.705+0000 7fad87a00700 1 -- 172.21.15.90:0/3834981210 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad801a25e0 con 0x7fad8010e8f0 2024-05-27T07:52:38.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.712+0000 7fad7d600700 1 -- 172.21.15.90:0/3834981210 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fad7801b030 con 0x7fad8010e8f0 2024-05-27T07:52:38.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.713+0000 7fad9657eb80 1 Processor -- start 2024-05-27T07:52:38.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.714+0000 7fad7d600700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad680b2af0 0x7fad680b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.714+0000 7fad7d600700 1 -- 172.21.15.90:0/3834981210 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fad780e2430 con 0x7fad8010e8f0 2024-05-27T07:52:38.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.715+0000 7fad7fe00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad680b2af0 0x7fad680b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.715+0000 7fad9657eb80 1 -- start start 2024-05-27T07:52:38.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9018cb9f0 0x55d90195e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 0x55d901973920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad9657eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d90188ce40 con 0x55d9018cb9f0 2024-05-27T07:52:38.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad9657eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d90187c9e0 con 0x55d9017dfe40 2024-05-27T07:52:38.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad9657eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d901880a20 con 0x55d90195ed00 2024-05-27T07:52:38.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad7fe00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad680b2af0 0x7fad680b5000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fad80102440 tx=0x7fad6c005fb0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:38.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad85a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 0x55d901973920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad85a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 0x55d901973920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:40438/0 (socket says 172.21.15.90:40438) 2024-05-27T07:52:38.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad85a00700 1 -- 172.21.15.90:0/2362425116 learned_addr learned my addr 172.21.15.90:0/2362425116 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:38.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.716+0000 7fad85000700 1 --2- 172.21.15.90:0/2362425116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9018cb9f0 0x55d90195e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.717+0000 7fad7fe00700 1 --2- 172.21.15.90:0/2362425116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.717+0000 7fad7fe00700 1 -- 172.21.15.90:0/2362425116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 msgr2=0x55d901973920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:38.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.717+0000 7fad7fe00700 1 --2- 172.21.15.90:0/2362425116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 0x55d901973920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.717+0000 7fad7fe00700 1 -- 172.21.15.90:0/2362425116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9018cb9f0 msgr2=0x55d90195e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:38.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.717+0000 7fad7fe00700 1 --2- 172.21.15.90:0/2362425116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9018cb9f0 0x55d90195e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.717+0000 7fad7fe00700 1 -- 172.21.15.90:0/2362425116 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d901845420 con 0x55d9017dfe40 2024-05-27T07:52:38.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.718+0000 7fad7fe00700 1 --2- 172.21.15.90:0/2362425116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b650 tx=0x7fad6c01e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7c75b0ec33f709 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:38.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.718+0000 7fad72000700 1 -- 172.21.15.90:0/2362425116 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003680 con 0x55d9017dfe40 2024-05-27T07:52:38.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.718+0000 7fad72000700 1 -- 172.21.15.90:0/2362425116 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad6c003cb0 con 0x55d9017dfe40 2024-05-27T07:52:38.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.719+0000 7fad72000700 1 -- 172.21.15.90:0/2362425116 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00fb80 con 0x55d9017dfe40 2024-05-27T07:52:38.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.720+0000 7fad9657eb80 1 -- 172.21.15.90:0/2362425116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 msgr2=0x55d90195e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:38.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.720+0000 7fad9657eb80 1 --2- 172.21.15.90:0/2362425116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00b650 tx=0x7fad6c01e000 comp rx=0 tx=0).stop 2024-05-27T07:52:38.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.720+0000 7fad9657eb80 1 -- 172.21.15.90:0/2362425116 shutdown_connections 2024-05-27T07:52:38.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.720+0000 7fad9657eb80 1 --2- 172.21.15.90:0/2362425116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 0x55d901973920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.720+0000 7fad9657eb80 1 --2- 172.21.15.90:0/2362425116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9018cb9f0 0x55d90195e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.720+0000 7fad9657eb80 1 --2- 172.21.15.90:0/2362425116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.720+0000 7fad9657eb80 1 -- 172.21.15.90:0/2362425116 >> 172.21.15.90:0/2362425116 conn(0x55d901879390 msgr2=0x55d90195a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:38.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.721+0000 7fad9657eb80 1 -- 172.21.15.90:0/2362425116 shutdown_connections 2024-05-27T07:52:38.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.721+0000 7fad9657eb80 1 -- 172.21.15.90:0/2362425116 wait complete. 2024-05-27T07:52:38.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.721+0000 7fad9657eb80 1 -- start start 2024-05-27T07:52:38.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.722+0000 7fad9657eb80 1 Processor -- start 2024-05-27T07:52:38.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.723+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 0x55d90196d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.723+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9018cb9f0 0x55d90196df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.723+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 0x55d901973720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.723+0000 7fad9657eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d901782fa0 con 0x55d9017dfe40 2024-05-27T07:52:38.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.723+0000 7fad9657eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d90181d760 con 0x55d9018cb9f0 2024-05-27T07:52:38.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.723+0000 7fad9657eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d901961470 con 0x55d90195ed00 2024-05-27T07:52:38.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.723+0000 7fad9657eb80 10 client.0 _ll_register_callbacks cb 0x55d901826c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:38.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.723+0000 7fad7fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9018cb9f0 0x55d90196df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.723+0000 7fad7fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9018cb9f0 0x55d90196df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50454/0 (socket says 172.21.15.90:50454) 2024-05-27T07:52:38.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.723+0000 7fad7fe00700 1 -- 172.21.15.90:0/1294371970 learned_addr learned my addr 172.21.15.90:0/1294371970 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:38.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.724+0000 7fad85a00700 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 0x55d901973720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.724+0000 7fad85000700 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 0x55d90196d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.725+0000 7fad7fe00700 1 -- 172.21.15.90:0/1294371970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 msgr2=0x55d901973720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:38.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.725+0000 7fad7fe00700 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 0x55d901973720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.725+0000 7fad7fe00700 1 -- 172.21.15.90:0/1294371970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 msgr2=0x55d90196d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:38.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.725+0000 7fad7fe00700 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 0x55d90196d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.725+0000 7fad7fe00700 1 -- 172.21.15.90:0/1294371970 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9017cc8d0 con 0x55d9018cb9f0 2024-05-27T07:52:38.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.725+0000 7fad7fe00700 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9018cb9f0 0x55d90196df90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00f710 tx=0x7fad6c00f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efdad9e952c9422a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:38.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.725+0000 7fad66a00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:52:38.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.726+0000 7fad66a00700 1 -- 172.21.15.90:0/1294371970 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003e60 con 0x55d9018cb9f0 2024-05-27T07:52:38.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.726+0000 7fad9657eb80 1 -- 172.21.15.90:0/1294371970 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad6c003cb0 con 0x55d9018cb9f0 2024-05-27T07:52:38.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.726+0000 7fad66a00700 1 -- 172.21.15.90:0/1294371970 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fad6c003ab0 con 0x55d9018cb9f0 2024-05-27T07:52:38.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.727+0000 7fad66a00700 1 -- 172.21.15.90:0/1294371970 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fad6c03f050 con 0x55d9018cb9f0 2024-05-27T07:52:38.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.727+0000 7fad9657eb80 10 client.36221 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:38.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.727+0000 7fad9657eb80 1 -- 172.21.15.90:0/1294371970 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d90184d060 con 0x55d9018cb9f0 2024-05-27T07:52:38.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.728+0000 7fad66a00700 1 -- 172.21.15.90:0/1294371970 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad6c003cb0 con 0x55d9018cb9f0 2024-05-27T07:52:38.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.728+0000 7fad66a00700 1 -- 172.21.15.90:0/1294371970 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fad6c022070 con 0x55d9018cb9f0 2024-05-27T07:52:38.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.728+0000 7fad9657eb80 10 client.36221 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:38.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.728+0000 7fad9657eb80 10 client.36221 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:52:38.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.728+0000 7fad9657eb80 20 client.36221 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:38.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.728+0000 7fad9657eb80 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d901989990 0x55d90198bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.728+0000 7fad9657eb80 4 client.36221 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:38.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.728+0000 7fad9657eb80 1 -- 172.21.15.90:0/1294371970 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d90188ed90 con 0x55d901989990 2024-05-27T07:52:38.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.731+0000 7fad85000700 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d901989990 0x55d90198bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.733+0000 7fad66a00700 10 client.36221 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:52:38.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.733+0000 7fad85000700 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d901989990 0x55d90198bdb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e8609cce9c52a68 server_cookie=a1bad1ac594e52cd in_seq=0 out_seq=0 2024-05-27T07:52:38.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.735+0000 7fad66a00700 1 -- 172.21.15.90:0/1294371970 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d90188ed90 con 0x55d901989990 2024-05-27T07:52:38.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.735+0000 7fad66a00700 10 client.36221 handle_command_reply: tid=0 2024-05-27T07:52:38.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.735+0000 7fad9657eb80 1 client.36221 shutdown 2024-05-27T07:52:38.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.739+0000 7fad9657eb80 1 -- 172.21.15.90:0/1294371970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9018cb9f0 msgr2=0x55d90196df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:38.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.739+0000 7fad9657eb80 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9018cb9f0 0x55d90196df90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00f710 tx=0x7fad6c00f7f0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.739+0000 7fad9657eb80 1 -- 172.21.15.90:0/1294371970 shutdown_connections 2024-05-27T07:52:38.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.739+0000 7fad9657eb80 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d901989990 0x55d90198bdb0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.739+0000 7fad9657eb80 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d90195ed00 0x55d901973720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.739+0000 7fad9657eb80 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 0x55d90196d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.739+0000 7fad9657eb80 1 --2- 172.21.15.90:0/1294371970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9018cb9f0 0x55d90196df90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.739+0000 7fad9657eb80 1 -- 172.21.15.90:0/1294371970 >> 172.21.15.90:0/1294371970 conn(0x55d901879390 msgr2=0x55d90195e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:38.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.740+0000 7fad66a00700 0 client.36221 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:52:38.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.740+0000 7fad9657eb80 1 -- 172.21.15.90:0/1294371970 shutdown_connections 2024-05-27T07:52:38.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.740+0000 7fad9657eb80 1 -- 172.21.15.90:0/1294371970 wait complete. 2024-05-27T07:52:38.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.740+0000 7fad9657eb80 20 client.36221 trim_cache size 0 max 16384 2024-05-27T07:52:38.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.770+0000 7fad9657eb80 1 Processor -- start 2024-05-27T07:52:38.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.770+0000 7fad9657eb80 1 -- start start 2024-05-27T07:52:38.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.771+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.771+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 0x55d9019645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.771+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d90195ed00 0x55d901973650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.771+0000 7fad9657eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fad801afcd0 con 0x55d90195ed00 2024-05-27T07:52:38.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.771+0000 7fad9657eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad801adac0 con 0x55d9017dfe40 2024-05-27T07:52:38.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.771+0000 7fad9657eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fad801af600 con 0x55d9018cb9f0 2024-05-27T07:52:38.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.772+0000 7fad85000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.772+0000 7fad85000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50470/0 (socket says 172.21.15.90:50470) 2024-05-27T07:52:38.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.772+0000 7fad85000700 1 -- 172.21.15.90:0/208289150 learned_addr learned my addr 172.21.15.90:0/208289150 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:38.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.772+0000 7fad7fe00700 1 --2- 172.21.15.90:0/208289150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 0x55d9019645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.772+0000 7fad85a00700 1 --2- 172.21.15.90:0/208289150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d90195ed00 0x55d901973650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.772+0000 7fad85000700 1 -- 172.21.15.90:0/208289150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 msgr2=0x55d9019645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:38.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.773+0000 7fad85000700 1 --2- 172.21.15.90:0/208289150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 0x55d9019645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.773+0000 7fad85000700 1 -- 172.21.15.90:0/208289150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d90195ed00 msgr2=0x55d901973650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:38.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.773+0000 7fad85000700 1 --2- 172.21.15.90:0/208289150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d90195ed00 0x55d901973650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.773+0000 7fad85000700 1 -- 172.21.15.90:0/208289150 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90195b9c0 con 0x55d9017dfe40 2024-05-27T07:52:38.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.773+0000 7fad85000700 1 --2- 172.21.15.90:0/208289150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fad74000c00 tx=0x7fad74008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ac073f49731602 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:38.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.774+0000 7fad71600700 1 -- 172.21.15.90:0/208289150 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74017c90 con 0x55d9017dfe40 2024-05-27T07:52:38.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.774+0000 7fad71600700 1 -- 172.21.15.90:0/208289150 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad74017e30 con 0x55d9017dfe40 2024-05-27T07:52:38.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.774+0000 7fad71600700 1 -- 172.21.15.90:0/208289150 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74026520 con 0x55d9017dfe40 2024-05-27T07:52:38.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.775+0000 7fad9657eb80 1 -- 172.21.15.90:0/208289150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 msgr2=0x55d90195e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:38.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.775+0000 7fad9657eb80 1 --2- 172.21.15.90:0/208289150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e7c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fad74000c00 tx=0x7fad74008040 comp rx=0 tx=0).stop 2024-05-27T07:52:38.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.775+0000 7fad9657eb80 1 -- 172.21.15.90:0/208289150 shutdown_connections 2024-05-27T07:52:38.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.775+0000 7fad9657eb80 1 --2- 172.21.15.90:0/208289150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d90195ed00 0x55d901973650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.775+0000 7fad9657eb80 1 --2- 172.21.15.90:0/208289150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d9017dfe40 0x55d90195e7c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.775+0000 7fad9657eb80 1 --2- 172.21.15.90:0/208289150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 0x55d9019645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.775+0000 7fad9657eb80 1 -- 172.21.15.90:0/208289150 >> 172.21.15.90:0/208289150 conn(0x55d901879390 msgr2=0x55d90195e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:38.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.776+0000 7fad9657eb80 1 -- 172.21.15.90:0/208289150 shutdown_connections 2024-05-27T07:52:38.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.776+0000 7fad9657eb80 1 -- 172.21.15.90:0/208289150 wait complete. 2024-05-27T07:52:38.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.778+0000 7fad9657eb80 1 -- start start 2024-05-27T07:52:38.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.780+0000 7fad9657eb80 1 Processor -- start 2024-05-27T07:52:38.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.784+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 0x55d9019899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.784+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 0x55d901989f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.784+0000 7fad9657eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d90195ed00 0x55d90198a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.784+0000 7fad9657eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fad801ae860 con 0x55d9017dfe40 2024-05-27T07:52:38.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.784+0000 7fad9657eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad801ae190 con 0x55d90195ed00 2024-05-27T07:52:38.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.784+0000 7fad9657eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fad801aef30 con 0x55d9018cb9f0 2024-05-27T07:52:38.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.784+0000 7fad9657eb80 10 client.0 _ll_register_callbacks cb 0x55d901826c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:38.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.785+0000 7fad85000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 0x55d9019899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.785+0000 7fad85000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 0x55d9019899d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52306/0 (socket says 172.21.15.90:52306) 2024-05-27T07:52:38.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.785+0000 7fad85000700 1 -- 172.21.15.90:0/3370878473 learned_addr learned my addr 172.21.15.90:0/3370878473 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:38.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.786+0000 7fad7fe00700 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 0x55d901989f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.786+0000 7fad85a00700 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d90195ed00 0x55d90198a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.786+0000 7fad7fe00700 1 -- 172.21.15.90:0/3370878473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d90195ed00 msgr2=0x55d90198a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:38.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.786+0000 7fad7fe00700 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d90195ed00 0x55d90198a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.787+0000 7fad7fe00700 1 -- 172.21.15.90:0/3370878473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 msgr2=0x55d9019899d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:38.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.787+0000 7fad7fe00700 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 0x55d9019899d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.787+0000 7fad7fe00700 1 -- 172.21.15.90:0/3370878473 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d901956240 con 0x55d9018cb9f0 2024-05-27T07:52:38.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.787+0000 7fad7fe00700 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 0x55d901989f10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fad6c037c10 tx=0x7fad6c005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ae809706423dc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:38.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.787+0000 7fad66a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:52:38.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.788+0000 7fad66a00700 1 -- 172.21.15.90:0/3370878473 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00f910 con 0x55d9018cb9f0 2024-05-27T07:52:38.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.788+0000 7fad9657eb80 1 -- 172.21.15.90:0/3370878473 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad74017e30 con 0x55d9018cb9f0 2024-05-27T07:52:38.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.789+0000 7fad66a00700 1 -- 172.21.15.90:0/3370878473 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fad6c02b040 con 0x55d9018cb9f0 2024-05-27T07:52:38.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.789+0000 7fad66a00700 1 -- 172.21.15.90:0/3370878473 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fad6c00f910 con 0x55d9018cb9f0 2024-05-27T07:52:38.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.789+0000 7fad9657eb80 10 client.26281 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:38.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.789+0000 7fad9657eb80 1 -- 172.21.15.90:0/3370878473 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d9019f34d0 con 0x55d9018cb9f0 2024-05-27T07:52:38.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.790+0000 7fad66a00700 1 -- 172.21.15.90:0/3370878473 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fad74017e30 con 0x55d9018cb9f0 2024-05-27T07:52:38.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.790+0000 7fad66a00700 1 -- 172.21.15.90:0/3370878473 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fad6c022070 con 0x55d9018cb9f0 2024-05-27T07:52:38.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.790+0000 7fad9657eb80 10 client.26281 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:38.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.790+0000 7fad9657eb80 10 client.26281 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:52:38.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.790+0000 7fad9657eb80 20 client.26281 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:38.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.790+0000 7fad9657eb80 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d9019f3ae0 0x55d9019f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:38.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.791+0000 7fad9657eb80 4 client.26281 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:52:38.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.791+0000 7fad9657eb80 1 -- 172.21.15.90:0/3370878473 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d901888570 con 0x55d9019f3ae0 2024-05-27T07:52:38.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.792+0000 7fad85000700 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d9019f3ae0 0x55d9019f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:38.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.793+0000 7fad85000700 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d9019f3ae0 0x55d9019f5f00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82b09486231127d server_cookie=fb422eea7f276326 in_seq=0 out_seq=0 2024-05-27T07:52:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.793+0000 7fad66a00700 10 client.26281 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:52:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.794+0000 7fad66a00700 1 -- 172.21.15.90:0/3370878473 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d901888570 con 0x55d9019f3ae0 2024-05-27T07:52:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.794+0000 7fad66a00700 10 client.26281 handle_command_reply: tid=0 2024-05-27T07:52:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.794+0000 7fad9657eb80 1 client.26281 shutdown 2024-05-27T07:52:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.795+0000 7fad9657eb80 1 -- 172.21.15.90:0/3370878473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 msgr2=0x55d901989f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:38.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.795+0000 7fad9657eb80 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 0x55d901989f10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fad6c037c10 tx=0x7fad6c005780 comp rx=0 tx=0).stop 2024-05-27T07:52:38.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.795+0000 7fad9657eb80 1 -- 172.21.15.90:0/3370878473 shutdown_connections 2024-05-27T07:52:38.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.795+0000 7fad9657eb80 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d9019f3ae0 0x55d9019f5f00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.795+0000 7fad9657eb80 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d90195ed00 0x55d90198a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.795+0000 7fad9657eb80 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9017dfe40 0x55d9019899d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.795+0000 7fad9657eb80 1 --2- 172.21.15.90:0/3370878473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9018cb9f0 0x55d901989f10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.795+0000 7fad9657eb80 1 -- 172.21.15.90:0/3370878473 >> 172.21.15.90:0/3370878473 conn(0x55d901879390 msgr2=0x55d901959db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.796+0000 7fad66a00700 0 client.26281 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:52:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.797+0000 7fad9657eb80 1 -- 172.21.15.90:0/3370878473 shutdown_connections 2024-05-27T07:52:38.798 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:52:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.797+0000 7fad9657eb80 1 -- 172.21.15.90:0/3370878473 wait complete. 2024-05-27T07:52:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.798+0000 7fad9657eb80 20 client.26281 trim_cache size 0 max 16384 2024-05-27T07:52:38.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.799+0000 7fad87a00700 1 -- 172.21.15.90:0/3834981210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad680b2af0 msgr2=0x7fad680b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:38.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.799+0000 7fad87a00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad680b2af0 0x7fad680b5000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fad80102440 tx=0x7fad6c005fb0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.799+0000 7fad87a00700 1 -- 172.21.15.90:0/3834981210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad8010e8f0 msgr2=0x7fad801a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:38.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.799+0000 7fad87a00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad8010e8f0 0x7fad801a17c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fad7800efc0 tx=0x7fad7800c3a0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.799+0000 7fad87a00700 1 -- 172.21.15.90:0/3834981210 shutdown_connections 2024-05-27T07:52:38.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.799+0000 7fad87a00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad8010e8f0 0x7fad801a17c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.799+0000 7fad87a00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fad680b2af0 0x7fad680b5000 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.799+0000 7fad87a00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad80105510 0x7fad8019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.799+0000 7fad87a00700 1 --2- 172.21.15.90:0/3834981210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad80105e60 0x7fad8019d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:38.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.799+0000 7fad87a00700 1 -- 172.21.15.90:0/3834981210 >> 172.21.15.90:0/3834981210 conn(0x7fad800fedb0 msgr2=0x7fad80100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:38.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.801+0000 7fad87a00700 1 -- 172.21.15.90:0/3834981210 shutdown_connections 2024-05-27T07:52:38.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:38.801+0000 7fad87a00700 1 -- 172.21.15.90:0/3834981210 wait complete. 2024-05-27T07:52:38.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 140.0 s, 532.4 tps, lat 186.926 ms stddev 111.022 2024-05-27T07:52:40.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:40 smithi090 ceph-mon[31429]: pgmap v195: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.1 MiB/s wr, 832 op/s 2024-05-27T07:52:40.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:40 smithi136 ceph-mon[35324]: pgmap v195: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.1 MiB/s wr, 832 op/s 2024-05-27T07:52:40.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:40 smithi129 ceph-mon[35254]: pgmap v195: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.1 MiB/s wr, 832 op/s 2024-05-27T07:52:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.607+0000 7fd165800640 1 -- 172.21.15.90:0/2945388776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd160106b00 msgr2=0x7fd160106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.607+0000 7fd165800640 1 --2- 172.21.15.90:0/2945388776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd160106b00 0x7fd160106f30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009900 tx=0x7fd14c02a3f0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.607+0000 7fd165800640 1 -- 172.21.15.90:0/2945388776 shutdown_connections 2024-05-27T07:52:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.607+0000 7fd165800640 1 --2- 172.21.15.90:0/2945388776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd1601075c0 0x7fd16010ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.608+0000 7fd165800640 1 --2- 172.21.15.90:0/2945388776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd160106b00 0x7fd160106f30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.608+0000 7fd165800640 1 --2- 172.21.15.90:0/2945388776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd160102570 0x7fd160102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.608+0000 7fd165800640 1 -- 172.21.15.90:0/2945388776 >> 172.21.15.90:0/2945388776 conn(0x7fd1600779d0 msgr2=0x7fd160077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.608+0000 7fd165800640 1 -- 172.21.15.90:0/2945388776 shutdown_connections 2024-05-27T07:52:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.608+0000 7fd165800640 1 -- 172.21.15.90:0/2945388776 wait complete. 2024-05-27T07:52:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.610+0000 7fd165800640 1 Processor -- start 2024-05-27T07:52:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.610+0000 7fd165800640 1 -- start start 2024-05-27T07:52:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.611+0000 7fd165800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd160102570 0x7fd160110630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.611+0000 7fd165800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd160106b00 0x7fd160110b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.611+0000 7fd165800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1601075c0 0x7fd1601110b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.611+0000 7fd165800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd160113d60 con 0x7fd1601075c0 2024-05-27T07:52:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.611+0000 7fd165800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd160113be0 con 0x7fd160102570 2024-05-27T07:52:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.611+0000 7fd165800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd160113ee0 con 0x7fd160106b00 2024-05-27T07:52:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.611+0000 7fd15f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1601075c0 0x7fd1601110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.612+0000 7fd15ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd160102570 0x7fd160110630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.612+0000 7fd15e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd160106b00 0x7fd160110b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.612+0000 7fd15e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd160106b00 0x7fd160110b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37644/0 (socket says 172.21.15.90:37644) 2024-05-27T07:52:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.612+0000 7fd15e000640 1 -- 172.21.15.90:0/3747433521 learned_addr learned my addr 172.21.15.90:0/3747433521 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.613+0000 7fd15f400640 1 -- 172.21.15.90:0/3747433521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd160106b00 msgr2=0x7fd160110b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.613+0000 7fd15f400640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd160106b00 0x7fd160110b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.613+0000 7fd15f400640 1 -- 172.21.15.90:0/3747433521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd160102570 msgr2=0x7fd160110630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.613+0000 7fd15f400640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd160102570 0x7fd160110630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.613+0000 7fd15f400640 1 -- 172.21.15.90:0/3747433521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16019a510 con 0x7fd1601075c0 2024-05-27T07:52:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.613+0000 7fd15f400640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1601075c0 0x7fd1601110b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd15800e940 tx=0x7fd15800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.614+0000 7fd157400640 1 -- 172.21.15.90:0/3747433521 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158004d20 con 0x7fd1601075c0 2024-05-27T07:52:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.614+0000 7fd157400640 1 -- 172.21.15.90:0/3747433521 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd158005020 con 0x7fd1601075c0 2024-05-27T07:52:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.614+0000 7fd157400640 1 -- 172.21.15.90:0/3747433521 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1580116d0 con 0x7fd1601075c0 2024-05-27T07:52:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.614+0000 7fd165800640 1 -- 172.21.15.90:0/3747433521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16019a800 con 0x7fd1601075c0 2024-05-27T07:52:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.614+0000 7fd165800640 1 -- 172.21.15.90:0/3747433521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16019ad00 con 0x7fd1601075c0 2024-05-27T07:52:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.621+0000 7fd16c792740 1 Processor -- start 2024-05-27T07:52:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.623+0000 7fd157400640 1 -- 172.21.15.90:0/3747433521 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd15800cb20 con 0x7fd1601075c0 2024-05-27T07:52:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.623+0000 7fd16c792740 1 -- start start 2024-05-27T07:52:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.624+0000 7fd16c792740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 0x55eccfc61860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.624+0000 7fd16c792740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8962f0 0x55eccfc61da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.624+0000 7fd16c792740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf89e4d0 0x55eccfc622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.624+0000 7fd16c792740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eccfbd2b90 con 0x55eccf8bf1b0 2024-05-27T07:52:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.624+0000 7fd16c792740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55eccfbc2730 con 0x55eccf8962f0 2024-05-27T07:52:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.624+0000 7fd16c792740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eccfbc6770 con 0x55eccf89e4d0 2024-05-27T07:52:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.624+0000 7fd15e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf89e4d0 0x55eccfc622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.625+0000 7fd15e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf89e4d0 0x55eccfc622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37652/0 (socket says 172.21.15.90:37652) 2024-05-27T07:52:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.625+0000 7fd15e000640 1 -- 172.21.15.90:0/551990496 learned_addr learned my addr 172.21.15.90:0/551990496 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:40.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.625+0000 7fd15e000640 1 --2- 172.21.15.90:0/551990496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 0x55eccfc61860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.626+0000 7fd15e000640 1 -- 172.21.15.90:0/551990496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8962f0 msgr2=0x55eccfc61da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.626+0000 7fd15e000640 1 --2- 172.21.15.90:0/551990496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8962f0 0x55eccfc61da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.626+0000 7fd15e000640 1 -- 172.21.15.90:0/551990496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 msgr2=0x55eccfc61860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.626+0000 7fd15e000640 1 --2- 172.21.15.90:0/551990496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 0x55eccfc61860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.626+0000 7fd15e000640 1 -- 172.21.15.90:0/551990496 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eccfb5b8b0 con 0x55eccf89e4d0 2024-05-27T07:52:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.626+0000 7fd15e000640 1 --2- 172.21.15.90:0/551990496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf89e4d0 0x55eccfc622e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd14c02a910 tx=0x7fd14c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbd3a17ed6d41db0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.626+0000 7fd14be00640 1 -- 172.21.15.90:0/551990496 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c044020 con 0x55eccf89e4d0 2024-05-27T07:52:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.627+0000 7fd14be00640 1 -- 172.21.15.90:0/551990496 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd14c0046c0 con 0x55eccf89e4d0 2024-05-27T07:52:40.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.627+0000 7fd14be00640 1 -- 172.21.15.90:0/551990496 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c035070 con 0x55eccf89e4d0 2024-05-27T07:52:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.628+0000 7fd157400640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1440b29d0 0x7fd1440b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.628+0000 7fd15ea00640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1440b29d0 0x7fd1440b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.629+0000 7fd157400640 1 -- 172.21.15.90:0/3747433521 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fd1580e29d0 con 0x7fd1601075c0 2024-05-27T07:52:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.629+0000 7fd16c792740 1 -- 172.21.15.90:0/551990496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf89e4d0 msgr2=0x55eccfc622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.629+0000 7fd16c792740 1 --2- 172.21.15.90:0/551990496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf89e4d0 0x55eccfc622e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd14c02a910 tx=0x7fd14c009560 comp rx=0 tx=0).stop 2024-05-27T07:52:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.630+0000 7fd16c792740 1 -- 172.21.15.90:0/551990496 shutdown_connections 2024-05-27T07:52:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.630+0000 7fd16c792740 1 --2- 172.21.15.90:0/551990496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8962f0 0x55eccfc61da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.630+0000 7fd16c792740 1 --2- 172.21.15.90:0/551990496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf89e4d0 0x55eccfc622e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.630+0000 7fd16c792740 1 --2- 172.21.15.90:0/551990496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 0x55eccfc61860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.630+0000 7fd16c792740 1 -- 172.21.15.90:0/551990496 >> 172.21.15.90:0/551990496 conn(0x55eccf8b3300 msgr2=0x55eccfc5e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.630+0000 7fd15ea00640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1440b29d0 0x7fd1440b4ee0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd150005d00 tx=0x7fd150005c50 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.630+0000 7fd16c792740 1 -- 172.21.15.90:0/551990496 shutdown_connections 2024-05-27T07:52:40.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.630+0000 7fd16c792740 1 -- 172.21.15.90:0/551990496 wait complete. 2024-05-27T07:52:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.631+0000 7fd16c792740 1 -- start start 2024-05-27T07:52:40.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.632+0000 7fd16c792740 1 Processor -- start 2024-05-27T07:52:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.634+0000 7fd16c792740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8962f0 0x55eccfc6b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.634+0000 7fd16c792740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf89e4d0 0x55eccfc6bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.634+0000 7fd16c792740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.634+0000 7fd16c792740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eccf8a8790 con 0x55eccf89e4d0 2024-05-27T07:52:40.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.634+0000 7fd16c792740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55eccfb6abc0 con 0x55eccf8bf1b0 2024-05-27T07:52:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.634+0000 7fd16c792740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eccfc6e930 con 0x55eccf8962f0 2024-05-27T07:52:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.635+0000 7fd16c792740 10 client.0 _ll_register_callbacks cb 0x55eccfb48400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.636+0000 7fd15ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8962f0 0x55eccfc6b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.636+0000 7fd15f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.636+0000 7fd15e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf89e4d0 0x55eccfc6bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.636+0000 7fd15ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8962f0 0x55eccfc6b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37654/0 (socket says 172.21.15.90:37654) 2024-05-27T07:52:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.636+0000 7fd15ea00640 1 -- 172.21.15.90:0/3771187751 learned_addr learned my addr 172.21.15.90:0/3771187751 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.636+0000 7fd15ea00640 1 -- 172.21.15.90:0/3771187751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 msgr2=0x55eccfc77c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.636+0000 7fd15ea00640 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.636+0000 7fd15ea00640 1 -- 172.21.15.90:0/3771187751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf89e4d0 msgr2=0x55eccfc6bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.636+0000 7fd15ea00640 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf89e4d0 0x55eccfc6bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.636+0000 7fd15ea00640 1 -- 172.21.15.90:0/3771187751 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eccf8d7210 con 0x55eccf8962f0 2024-05-27T07:52:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.637+0000 7fd15f400640 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.637+0000 7fd15ea00640 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8962f0 0x55eccfc6b9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd150004700 tx=0x7fd150004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddc890fbbe2e50d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.637+0000 7fd148c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:52:40.638 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.637+0000 7fd15e000640 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf89e4d0 0x55eccfc6bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.638+0000 7fd148c00640 1 -- 172.21.15.90:0/3771187751 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150022070 con 0x55eccf8962f0 2024-05-27T07:52:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.638+0000 7fd148c00640 1 -- 172.21.15.90:0/3771187751 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fd15001e860 con 0x55eccf8962f0 2024-05-27T07:52:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.638+0000 7fd16c792740 1 -- 172.21.15.90:0/3771187751 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd14c0046c0 con 0x55eccf8962f0 2024-05-27T07:52:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.638+0000 7fd148c00640 1 -- 172.21.15.90:0/3771187751 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd15002e050 con 0x55eccf8962f0 2024-05-27T07:52:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.639+0000 7fd16c792740 10 client.26293 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.639+0000 7fd16c792740 1 -- 172.21.15.90:0/3771187751 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55eccf9a3620 con 0x55eccf8962f0 2024-05-27T07:52:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.639+0000 7fd148c00640 1 -- 172.21.15.90:0/3771187751 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd14c0046c0 con 0x55eccf8962f0 2024-05-27T07:52:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.639+0000 7fd148c00640 1 -- 172.21.15.90:0/3771187751 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fd1500109b0 con 0x55eccf8962f0 2024-05-27T07:52:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.640+0000 7fd16c792740 10 client.26293 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.640+0000 7fd16c792740 10 client.26293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.640+0000 7fd16c792740 20 client.26293 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:40.640 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.640+0000 7fd16c792740 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eccf87dd80 0x55eccfc7dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.640+0000 7fd16c792740 4 client.26293 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.640+0000 7fd16c792740 1 -- 172.21.15.90:0/3771187751 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eccfbd4ae0 con 0x55eccf87dd80 2024-05-27T07:52:40.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.641+0000 7fd15e000640 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eccf87dd80 0x55eccfc7dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.644+0000 7fd15e000640 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eccf87dd80 0x55eccfc7dd00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1aa819d7bb4ca4 server_cookie=c44e7e39694b0153 in_seq=0 out_seq=0 2024-05-27T07:52:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.644+0000 7fd148c00640 10 client.26293 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.649+0000 7fd148c00640 1 -- 172.21.15.90:0/3771187751 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eccfbd4ae0 con 0x55eccf87dd80 2024-05-27T07:52:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.649+0000 7fd148c00640 10 client.26293 handle_command_reply: tid=0 2024-05-27T07:52:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.649+0000 7fd16c792740 1 client.26293 shutdown 2024-05-27T07:52:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.650+0000 7fd16c792740 1 -- 172.21.15.90:0/3771187751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8962f0 msgr2=0x55eccfc6b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.650+0000 7fd16c792740 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8962f0 0x55eccfc6b9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd150004700 tx=0x7fd150004730 comp rx=0 tx=0).stop 2024-05-27T07:52:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.650+0000 7fd16c792740 1 -- 172.21.15.90:0/3771187751 shutdown_connections 2024-05-27T07:52:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.650+0000 7fd16c792740 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eccf87dd80 0x55eccfc7dd00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.650+0000 7fd16c792740 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.650+0000 7fd16c792740 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf89e4d0 0x55eccfc6bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.650+0000 7fd16c792740 1 --2- 172.21.15.90:0/3771187751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8962f0 0x55eccfc6b9b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.650+0000 7fd148c00640 0 client.26293 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.650+0000 7fd16c792740 1 -- 172.21.15.90:0/3771187751 >> 172.21.15.90:0/3771187751 conn(0x55eccf8b3300 msgr2=0x55eccfc63580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.651+0000 7fd16c792740 1 -- 172.21.15.90:0/3771187751 shutdown_connections 2024-05-27T07:52:40.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.651+0000 7fd16c792740 1 -- 172.21.15.90:0/3771187751 wait complete. 2024-05-27T07:52:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.654+0000 7fd16c792740 20 client.26293 trim_cache size 0 max 16384 2024-05-27T07:52:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.694+0000 7fd16c792740 1 Processor -- start 2024-05-27T07:52:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.694+0000 7fd16c792740 1 -- start start 2024-05-27T07:52:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.695+0000 7fd16c792740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf89e4d0 0x55eccfc6d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.695+0000 7fd16c792740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.696+0000 7fd16c792740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 0x55eccfc77f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.696+0000 7fd16c792740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601b92a0 con 0x55eccf8bf1b0 2024-05-27T07:52:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.696+0000 7fd16c792740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601b7090 con 0x55eccf89e4d0 2024-05-27T07:52:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.696+0000 7fd16c792740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601b8bd0 con 0x55eccf8531c0 2024-05-27T07:52:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.696+0000 7fd15f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 0x55eccfc77f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.696+0000 7fd15e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.696+0000 7fd15e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43992/0 (socket says 172.21.15.90:43992) 2024-05-27T07:52:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.696+0000 7fd15e000640 1 -- 172.21.15.90:0/3500175986 learned_addr learned my addr 172.21.15.90:0/3500175986 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.696+0000 7fd15ea00640 1 --2- 172.21.15.90:0/3500175986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf89e4d0 0x55eccfc6d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.697+0000 7fd15e000640 1 -- 172.21.15.90:0/3500175986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 msgr2=0x55eccfc77f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.697+0000 7fd15e000640 1 --2- 172.21.15.90:0/3500175986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 0x55eccfc77f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.697+0000 7fd15e000640 1 -- 172.21.15.90:0/3500175986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf89e4d0 msgr2=0x55eccfc6d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.697+0000 7fd15e000640 1 --2- 172.21.15.90:0/3500175986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf89e4d0 0x55eccfc6d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.697+0000 7fd15e000640 1 -- 172.21.15.90:0/3500175986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eccfc69f00 con 0x55eccf8bf1b0 2024-05-27T07:52:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.699+0000 7fd15e000640 1 --2- 172.21.15.90:0/3500175986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77a00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd14c02a910 tx=0x7fd14c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974a13bfa3875c3b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.700+0000 7fd14b400640 1 -- 172.21.15.90:0/3500175986 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c035050 con 0x55eccf8bf1b0 2024-05-27T07:52:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.700+0000 7fd14b400640 1 -- 172.21.15.90:0/3500175986 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd14c004fa0 con 0x55eccf8bf1b0 2024-05-27T07:52:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.700+0000 7fd14b400640 1 -- 172.21.15.90:0/3500175986 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c044040 con 0x55eccf8bf1b0 2024-05-27T07:52:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.704+0000 7fd16c792740 1 -- 172.21.15.90:0/3500175986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 msgr2=0x55eccfc77a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.704+0000 7fd16c792740 1 --2- 172.21.15.90:0/3500175986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77a00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd14c02a910 tx=0x7fd14c009260 comp rx=0 tx=0).stop 2024-05-27T07:52:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.705+0000 7fd16c792740 1 -- 172.21.15.90:0/3500175986 shutdown_connections 2024-05-27T07:52:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.705+0000 7fd16c792740 1 --2- 172.21.15.90:0/3500175986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 0x55eccfc77f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.705+0000 7fd16c792740 1 --2- 172.21.15.90:0/3500175986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf8bf1b0 0x55eccfc77a00 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.705+0000 7fd16c792740 1 --2- 172.21.15.90:0/3500175986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf89e4d0 0x55eccfc6d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.705+0000 7fd16c792740 1 -- 172.21.15.90:0/3500175986 >> 172.21.15.90:0/3500175986 conn(0x55eccf8b3300 msgr2=0x55eccfc6bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.705+0000 7fd16c792740 1 -- 172.21.15.90:0/3500175986 shutdown_connections 2024-05-27T07:52:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.705+0000 7fd16c792740 1 -- 172.21.15.90:0/3500175986 wait complete. 2024-05-27T07:52:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.706+0000 7fd16c792740 1 -- start start 2024-05-27T07:52:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.707+0000 7fd16c792740 1 Processor -- start 2024-05-27T07:52:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.708+0000 7fd16c792740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 0x55eccfc6efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.708+0000 7fd16c792740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf89e4d0 0x55eccfc6f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.708+0000 7fd16c792740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfce43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.708+0000 7fd16c792740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601b7e30 con 0x55eccf89e4d0 2024-05-27T07:52:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.708+0000 7fd16c792740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601b7760 con 0x55eccf8bf1b0 2024-05-27T07:52:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.708+0000 7fd16c792740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1601b8500 con 0x55eccf8531c0 2024-05-27T07:52:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.708+0000 7fd16c792740 10 client.0 _ll_register_callbacks cb 0x7fd150015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.709+0000 7fd15f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfce43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.709+0000 7fd15f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfce43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52542/0 (socket says 172.21.15.90:52542) 2024-05-27T07:52:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.709+0000 7fd15f400640 1 -- 172.21.15.90:0/3379246443 learned_addr learned my addr 172.21.15.90:0/3379246443 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.709+0000 7fd15ea00640 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 0x55eccfc6efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.710+0000 7fd15f400640 1 -- 172.21.15.90:0/3379246443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 msgr2=0x55eccfc6efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.710+0000 7fd15f400640 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 0x55eccfc6efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.710+0000 7fd15f400640 1 -- 172.21.15.90:0/3379246443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf89e4d0 msgr2=0x55eccfc6f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.710+0000 7fd15f400640 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf89e4d0 0x55eccfc6f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.710+0000 7fd15f400640 1 -- 172.21.15.90:0/3379246443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eccfce4900 con 0x55eccf8bf1b0 2024-05-27T07:52:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.710+0000 7fd15ea00640 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 0x55eccfc6efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.711+0000 7fd15f400640 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfce43c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd15800cfd0 tx=0x7fd1580a91e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2973e220c836acb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.711+0000 7fd148c00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:52:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.711+0000 7fd148c00640 1 -- 172.21.15.90:0/3379246443 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15801f040 con 0x55eccf8bf1b0 2024-05-27T07:52:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.711+0000 7fd148c00640 1 -- 172.21.15.90:0/3379246443 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fd1580a8ba0 con 0x55eccf8bf1b0 2024-05-27T07:52:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.711+0000 7fd16c792740 1 -- 172.21.15.90:0/3379246443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd14c004fa0 con 0x55eccf8bf1b0 2024-05-27T07:52:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.712+0000 7fd148c00640 1 -- 172.21.15.90:0/3379246443 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd15808fe60 con 0x55eccf8bf1b0 2024-05-27T07:52:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.712+0000 7fd16c792740 10 client.36242 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.712+0000 7fd16c792740 1 -- 172.21.15.90:0/3379246443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55eccfce5a40 con 0x55eccf8bf1b0 2024-05-27T07:52:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.713+0000 7fd148c00640 1 -- 172.21.15.90:0/3379246443 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd14c004fa0 con 0x55eccf8bf1b0 2024-05-27T07:52:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.713+0000 7fd148c00640 1 -- 172.21.15.90:0/3379246443 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fd15800f040 con 0x55eccf8bf1b0 2024-05-27T07:52:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.713+0000 7fd16c792740 10 client.36242 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.713+0000 7fd16c792740 10 client.36242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.713+0000 7fd16c792740 20 client.36242 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.713+0000 7fd16c792740 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eccfce61c0 0x55eccfce8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.713+0000 7fd16c792740 4 client.36242 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:52:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.713+0000 7fd16c792740 1 -- 172.21.15.90:0/3379246443 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eccfbce2c0 con 0x55eccfce61c0 2024-05-27T07:52:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.714+0000 7fd15ea00640 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eccfce61c0 0x55eccfce8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.717+0000 7fd15ea00640 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eccfce61c0 0x55eccfce8650 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8d838deb3a7275 server_cookie=76013a88bd50c7e4 in_seq=0 out_seq=0 2024-05-27T07:52:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.717+0000 7fd148c00640 10 client.36242 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.752+0000 7fd148c00640 1 -- 172.21.15.90:0/3379246443 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55eccfbce2c0 con 0x55eccfce61c0 2024-05-27T07:52:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.752+0000 7fd148c00640 10 client.36242 handle_command_reply: tid=0 2024-05-27T07:52:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.752+0000 7fd16c792740 1 client.36242 shutdown 2024-05-27T07:52:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.754+0000 7fd16c792740 1 -- 172.21.15.90:0/3379246443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 msgr2=0x55eccfce43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.754+0000 7fd16c792740 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfce43c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd15800cfd0 tx=0x7fd1580a91e0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.754+0000 7fd16c792740 1 -- 172.21.15.90:0/3379246443 shutdown_connections 2024-05-27T07:52:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.754+0000 7fd16c792740 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eccfce61c0 0x55eccfce8650 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.754+0000 7fd16c792740 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eccf8bf1b0 0x55eccfce43c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.754+0000 7fd16c792740 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eccf89e4d0 0x55eccfc6f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.754+0000 7fd16c792740 1 --2- 172.21.15.90:0/3379246443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eccf8531c0 0x55eccfc6efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.754+0000 7fd16c792740 1 -- 172.21.15.90:0/3379246443 >> 172.21.15.90:0/3379246443 conn(0x55eccf8b3300 msgr2=0x55eccfc6ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.754+0000 7fd148c00640 0 client.36242 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.755+0000 7fd16c792740 1 -- 172.21.15.90:0/3379246443 shutdown_connections 2024-05-27T07:52:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.755+0000 7fd16c792740 1 -- 172.21.15.90:0/3379246443 wait complete. 2024-05-27T07:52:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.755+0000 7fd16c792740 20 client.36242 trim_cache size 0 max 16384 2024-05-27T07:52:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:52:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:52:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:52:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.757+0000 7fd165800640 1 -- 172.21.15.90:0/3747433521 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1440b29d0 msgr2=0x7fd1440b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.757+0000 7fd165800640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1440b29d0 0x7fd1440b4ee0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd150005d00 tx=0x7fd150005c50 comp rx=0 tx=0).stop 2024-05-27T07:52:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.757+0000 7fd165800640 1 -- 172.21.15.90:0/3747433521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1601075c0 msgr2=0x7fd1601110b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.757+0000 7fd165800640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1601075c0 0x7fd1601110b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fd15800e940 tx=0x7fd15800ee00 comp rx=0 tx=0).stop 2024-05-27T07:52:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.757+0000 7fd165800640 1 -- 172.21.15.90:0/3747433521 shutdown_connections 2024-05-27T07:52:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.757+0000 7fd165800640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1440b29d0 0x7fd1440b4ee0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.757+0000 7fd165800640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1601075c0 0x7fd1601110b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.757+0000 7fd165800640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd160106b00 0x7fd160110b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.757+0000 7fd165800640 1 --2- 172.21.15.90:0/3747433521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd160102570 0x7fd160110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.757+0000 7fd165800640 1 -- 172.21.15.90:0/3747433521 >> 172.21.15.90:0/3747433521 conn(0x7fd1600779d0 msgr2=0x7fd160100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.758+0000 7fd165800640 1 -- 172.21.15.90:0/3747433521 shutdown_connections 2024-05-27T07:52:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:40.758+0000 7fd165800640 1 -- 172.21.15.90:0/3747433521 wait complete. 2024-05-27T07:52:41.471 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:52:42.163 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:42.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:42 smithi090 ceph-mon[31429]: pgmap v196: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-05-27T07:52:42.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:42 smithi136 ceph-mon[35324]: pgmap v196: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-05-27T07:52:42.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:42 smithi129 ceph-mon[35254]: pgmap v196: 129 pgs: 129 active+clean; 1.9 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.9 MiB/s wr, 565 op/s 2024-05-27T07:52:43.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 145.0 s, 570.4 tps, lat 175.804 ms stddev 97.408 2024-05-27T07:52:44.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:44 smithi090 ceph-mon[31429]: pgmap v197: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 743 op/s 2024-05-27T07:52:44.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:44 smithi136 ceph-mon[35324]: pgmap v197: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 743 op/s 2024-05-27T07:52:44.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:44 smithi129 ceph-mon[35254]: pgmap v197: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.3 MiB/s wr, 743 op/s 2024-05-27T07:52:45.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fac8d800700 time 2024-05-27T07:52:44.774517+0000 2024-05-27T07:52:45.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:52:45.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:52:45.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7faca32e2ba6] 2024-05-27T07:52:45.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7faca32e2dd1] 2024-05-27T07:52:45.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55835a63723d] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55835a622971] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 5: (SnapRealm::check_cache() const+0x13e) [0x55835a620fa0] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55835a621651] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55835a5b6347] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55835a2b287e] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55835a2b7f2f] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55835a2c2969] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 11: (MDLog::_replay_thread()+0x1f38) [0x55835a65b48c] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55835a6649cb] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 13: (Thread::entry_wrapper()+0x43) [0x7faca32b172d] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 14: (Thread::_entry_func(void*)+0xd) [0x7faca32b1749] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faca1e551ca] 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 16: clone() 2024-05-27T07:52:45.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: *** Caught signal (Aborted) ** 2024-05-27T07:52:45.216 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: in thread 7fac8d800700 thread_name:md_log_replay 2024-05-27T07:52:45.218 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:44.777+0000 7fac8d800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fac8d800700 time 2024-05-27T07:52:44.774517+0000 2024-05-27T07:52:45.218 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:52:45.218 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.218 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:52:45.218 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7faca32e2ba6] 2024-05-27T07:52:45.218 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7faca32e2dd1] 2024-05-27T07:52:45.218 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55835a63723d] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55835a622971] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 5: (SnapRealm::check_cache() const+0x13e) [0x55835a620fa0] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55835a621651] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55835a5b6347] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55835a2b287e] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55835a2b7f2f] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55835a2c2969] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 11: (MDLog::_replay_thread()+0x1f38) [0x55835a65b48c] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55835a6649cb] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 13: (Thread::entry_wrapper()+0x43) [0x7faca32b172d] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 14: (Thread::_entry_func(void*)+0xd) [0x7faca32b1749] 2024-05-27T07:52:45.219 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faca1e551ca] 2024-05-27T07:52:45.220 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 16: clone() 2024-05-27T07:52:45.220 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.220 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:52:45.220 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faca1e5fd20] 2024-05-27T07:52:45.220 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2: gsignal() 2024-05-27T07:52:45.220 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 3: abort() 2024-05-27T07:52:45.220 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7faca32e2ced] 2024-05-27T07:52:45.220 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7faca32e2dd1] 2024-05-27T07:52:45.220 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55835a63723d] 2024-05-27T07:52:45.220 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55835a622971] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 8: (SnapRealm::check_cache() const+0x13e) [0x55835a620fa0] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55835a621651] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55835a5b6347] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55835a2b287e] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55835a2b7f2f] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55835a2c2969] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 14: (MDLog::_replay_thread()+0x1f38) [0x55835a65b48c] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55835a6649cb] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 16: (Thread::entry_wrapper()+0x43) [0x7faca32b172d] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 17: (Thread::_entry_func(void*)+0xd) [0x7faca32b1749] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7faca1e551ca] 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 19: clone() 2024-05-27T07:52:45.221 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:44.781+0000 7fac8d800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: in thread 7fac8d800700 thread_name:md_log_replay 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faca1e5fd20] 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2: gsignal() 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 3: abort() 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7faca32e2ced] 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7faca32e2dd1] 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55835a63723d] 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55835a622971] 2024-05-27T07:52:45.222 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 8: (SnapRealm::check_cache() const+0x13e) [0x55835a620fa0] 2024-05-27T07:52:45.223 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55835a621651] 2024-05-27T07:52:45.223 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55835a5b6347] 2024-05-27T07:52:45.223 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55835a2b287e] 2024-05-27T07:52:45.223 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55835a2b7f2f] 2024-05-27T07:52:45.223 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55835a2c2969] 2024-05-27T07:52:45.223 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 14: (MDLog::_replay_thread()+0x1f38) [0x55835a65b48c] 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55835a6649cb] 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 16: (Thread::entry_wrapper()+0x43) [0x7faca32b172d] 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 17: (Thread::_entry_func(void*)+0xd) [0x7faca32b1749] 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7faca1e551ca] 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 19: clone() 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: -1> 2024-05-27T07:52:44.777+0000 7fac8d800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fac8d800700 time 2024-05-27T07:52:44.774517+0000 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:52:45.224 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7faca32e2ba6] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7faca32e2dd1] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55835a63723d] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55835a622971] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 5: (SnapRealm::check_cache() const+0x13e) [0x55835a620fa0] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55835a621651] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55835a5b6347] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55835a2b287e] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55835a2b7f2f] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55835a2c2969] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 11: (MDLog::_replay_thread()+0x1f38) [0x55835a65b48c] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55835a6649cb] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 13: (Thread::entry_wrapper()+0x43) [0x7faca32b172d] 2024-05-27T07:52:45.225 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 14: (Thread::_entry_func(void*)+0xd) [0x7faca32b1749] 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faca1e551ca] 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 16: clone() 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 0> 2024-05-27T07:52:44.781+0000 7fac8d800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: in thread 7fac8d800700 thread_name:md_log_replay 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faca1e5fd20] 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2: gsignal() 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 3: abort() 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7faca32e2ced] 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7faca32e2dd1] 2024-05-27T07:52:45.226 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55835a63723d] 2024-05-27T07:52:45.227 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55835a622971] 2024-05-27T07:52:45.227 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 8: (SnapRealm::check_cache() const+0x13e) [0x55835a620fa0] 2024-05-27T07:52:45.227 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55835a621651] 2024-05-27T07:52:45.227 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55835a5b6347] 2024-05-27T07:52:45.227 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55835a2b287e] 2024-05-27T07:52:45.227 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55835a2b7f2f] 2024-05-27T07:52:45.227 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55835a2c2969] 2024-05-27T07:52:45.227 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 14: (MDLog::_replay_thread()+0x1f38) [0x55835a65b48c] 2024-05-27T07:52:45.227 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55835a6649cb] 2024-05-27T07:52:45.227 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 16: (Thread::entry_wrapper()+0x43) [0x7faca32b172d] 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 17: (Thread::_entry_func(void*)+0xd) [0x7faca32b1749] 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7faca1e551ca] 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 19: clone() 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: -4> 2024-05-27T07:52:44.777+0000 7fac8d800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fac8d800700 time 2024-05-27T07:52:44.774517+0000 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7faca32e2ba6] 2024-05-27T07:52:45.228 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7faca32e2dd1] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55835a63723d] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55835a622971] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 5: (SnapRealm::check_cache() const+0x13e) [0x55835a620fa0] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55835a621651] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55835a5b6347] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55835a2b287e] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55835a2b7f2f] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55835a2c2969] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 11: (MDLog::_replay_thread()+0x1f38) [0x55835a65b48c] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55835a6649cb] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 13: (Thread::entry_wrapper()+0x43) [0x7faca32b172d] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 14: (Thread::_entry_func(void*)+0xd) [0x7faca32b1749] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faca1e551ca] 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 16: clone() 2024-05-27T07:52:45.229 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: -3> 2024-05-27T07:52:44.781+0000 7fac8d800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: in thread 7fac8d800700 thread_name:md_log_replay 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faca1e5fd20] 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2: gsignal() 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 3: abort() 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7faca32e2ced] 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7faca32e2dd1] 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55835a63723d] 2024-05-27T07:52:45.230 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55835a622971] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 8: (SnapRealm::check_cache() const+0x13e) [0x55835a620fa0] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55835a621651] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55835a5b6347] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55835a2b287e] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55835a2b7f2f] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55835a2c2969] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 14: (MDLog::_replay_thread()+0x1f38) [0x55835a65b48c] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55835a6649cb] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 16: (Thread::entry_wrapper()+0x43) [0x7faca32b172d] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 17: (Thread::_entry_func(void*)+0xd) [0x7faca32b1749] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7faca1e551ca] 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 19: clone() 2024-05-27T07:52:45.231 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:52:45.232 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[106057]: 2024-05-27T07:52:45.963 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:45 smithi090 podman[107387]: 2024-05-27 07:52:45.608713352 +0000 UTC m=+0.702950034 container died ad2feb8f6311c193d1a808b96ccaa83d082c256676ebf9929cd4d6e2ecceca58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-27T07:52:46.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:46 smithi090 ceph-mon[31429]: pgmap v198: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 619 op/s 2024-05-27T07:52:46.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:46 smithi136 ceph-mon[35324]: pgmap v198: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 619 op/s 2024-05-27T07:52:46.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:46 smithi129 ceph-mon[35254]: pgmap v198: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 619 op/s 2024-05-27T07:52:47.913 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:47 smithi090 podman[107387]: 2024-05-27 07:52:47.594176186 +0000 UTC m=+2.688412856 container remove ad2feb8f6311c193d1a808b96ccaa83d082c256676ebf9929cd4d6e2ecceca58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:52:47.913 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:47 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:52:48.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.417+0000 7fe0dc600700 1 -- 172.21.15.90:0/3238924197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d4110c80 msgr2=0x7fe0d41130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.417+0000 7fe0dc600700 1 --2- 172.21.15.90:0/3238924197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d4110c80 0x7fe0d41130a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009990 tx=0x7fe0cc009ca0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.417+0000 7fe0dc600700 1 -- 172.21.15.90:0/3238924197 shutdown_connections 2024-05-27T07:52:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.417+0000 7fe0dc600700 1 --2- 172.21.15.90:0/3238924197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d4110c80 0x7fe0d41130a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.417+0000 7fe0dc600700 1 --2- 172.21.15.90:0/3238924197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0d41030d0 0x7fe0d41034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.417+0000 7fe0dc600700 1 --2- 172.21.15.90:0/3238924197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0d4103a20 0x7fe0d4110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.417+0000 7fe0dc600700 1 -- 172.21.15.90:0/3238924197 >> 172.21.15.90:0/3238924197 conn(0x7fe0d40fedb0 msgr2=0x7fe0d41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.417+0000 7fe0dc600700 1 -- 172.21.15.90:0/3238924197 shutdown_connections 2024-05-27T07:52:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.418+0000 7fe0dc600700 1 -- 172.21.15.90:0/3238924197 wait complete. 2024-05-27T07:52:48.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.420+0000 7fe0dc600700 1 Processor -- start 2024-05-27T07:52:48.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.420+0000 7fe0dc600700 1 -- start start 2024-05-27T07:52:48.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.421+0000 7fe0dc600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0d41030d0 0x7fe0d410f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.423+0000 7fe0dc600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0d4103a20 0x7fe0d410f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.423+0000 7fe0dc600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d4110c80 0x7fe0d41aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.423+0000 7fe0dc600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d4105f90 con 0x7fe0d41030d0 2024-05-27T07:52:48.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.423+0000 7fe0dc600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d4105e10 con 0x7fe0d4110c80 2024-05-27T07:52:48.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.423+0000 7fe0dc600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d4106110 con 0x7fe0d4103a20 2024-05-27T07:52:48.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.423+0000 7fe0d9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0d41030d0 0x7fe0d410f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.423+0000 7fe0d9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0d41030d0 0x7fe0d410f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44024/0 (socket says 172.21.15.90:44024) 2024-05-27T07:52:48.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.423+0000 7fe0d9c00700 1 -- 172.21.15.90:0/1919308986 learned_addr learned my addr 172.21.15.90:0/1919308986 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:48.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.423+0000 7fe0da600700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d4110c80 0x7fe0d41aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.424+0000 7fe0d9200700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0d4103a20 0x7fe0d410f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.424+0000 7fe0d9c00700 1 -- 172.21.15.90:0/1919308986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0d4103a20 msgr2=0x7fe0d410f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.424+0000 7fe0d9c00700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0d4103a20 0x7fe0d410f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.424+0000 7fe0d9c00700 1 -- 172.21.15.90:0/1919308986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d4110c80 msgr2=0x7fe0d41aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.424+0000 7fe0d9c00700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d4110c80 0x7fe0d41aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.424+0000 7fe0d9c00700 1 -- 172.21.15.90:0/1919308986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d41ad4f0 con 0x7fe0d41030d0 2024-05-27T07:52:48.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.425+0000 7fe0d9c00700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0d41030d0 0x7fe0d410f180 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400ba50 tx=0x7fe0c400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.425+0000 7fe0d2000700 1 -- 172.21.15.90:0/1919308986 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4019070 con 0x7fe0d41030d0 2024-05-27T07:52:48.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.426+0000 7fe0dc600700 1 -- 172.21.15.90:0/1919308986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d41ad7e0 con 0x7fe0d41030d0 2024-05-27T07:52:48.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.426+0000 7fe0dc600700 1 -- 172.21.15.90:0/1919308986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d41add20 con 0x7fe0d41030d0 2024-05-27T07:52:48.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.426+0000 7fe0d2000700 1 -- 172.21.15.90:0/1919308986 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0c400cde0 con 0x7fe0d41030d0 2024-05-27T07:52:48.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.426+0000 7fe0d2000700 1 -- 172.21.15.90:0/1919308986 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4014e30 con 0x7fe0d41030d0 2024-05-27T07:52:48.438 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:48 smithi090 ceph-mon[31429]: pgmap v199: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 512 op/s 2024-05-27T07:52:48.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.434+0000 7fe0d2000700 1 -- 172.21.15.90:0/1919308986 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe0c401c070 con 0x7fe0d41030d0 2024-05-27T07:52:48.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.437+0000 7fe0eb094b80 1 Processor -- start 2024-05-27T07:52:48.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.437+0000 7fe0eb094b80 1 -- start start 2024-05-27T07:52:48.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.438+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad019b1e40 0x55ad01b367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.438+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 0x55ad01b30870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.438+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.438+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ad01a5ee40 con 0x55ad01b36d30 2024-05-27T07:52:48.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.438+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ad01a4e9e0 con 0x55ad019b1e40 2024-05-27T07:52:48.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.438+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ad01a52a20 con 0x55ad01b30db0 2024-05-27T07:52:48.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.439+0000 7fe0da600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad019b1e40 0x55ad01b367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.439+0000 7fe0d9200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.439+0000 7fe0da600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad019b1e40 0x55ad01b367f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52566/0 (socket says 172.21.15.90:52566) 2024-05-27T07:52:48.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.439+0000 7fe0da600700 1 -- 172.21.15.90:0/872732419 learned_addr learned my addr 172.21.15.90:0/872732419 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:48.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.439+0000 7fe0d9c00700 1 --2- 172.21.15.90:0/872732419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 0x55ad01b30870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.439+0000 7fe0d2000700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe0bc0b2c00 0x7fe0bc0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.439+0000 7fe0da600700 1 -- 172.21.15.90:0/872732419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 msgr2=0x55ad01b311c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.440+0000 7fe0d2000700 1 -- 172.21.15.90:0/1919308986 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fe0c40e1a10 con 0x7fe0d41030d0 2024-05-27T07:52:48.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.440+0000 7fe0da600700 1 --2- 172.21.15.90:0/872732419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b311c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.440+0000 7fe0da600700 1 -- 172.21.15.90:0/872732419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 msgr2=0x55ad01b30870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.440+0000 7fe0da600700 1 --2- 172.21.15.90:0/872732419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 0x55ad01b30870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.440+0000 7fe0da600700 1 -- 172.21.15.90:0/872732419 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad01a17420 con 0x55ad019b1e40 2024-05-27T07:52:48.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.441+0000 7fe0d9200700 1 --2- 172.21.15.90:0/872732419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:48.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.441+0000 7fe0da600700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe0bc0b2c00 0x7fe0bc0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.442+0000 7fe0da600700 1 --2- 172.21.15.90:0/872732419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad019b1e40 0x55ad01b367f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc003740 tx=0x7fe0cc005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f711e973997661 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.442+0000 7fe0da600700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe0bc0b2c00 0x7fe0bc0b5110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc02a000 tx=0x7fe0cc02d000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.443+0000 7fe0c2a00700 1 -- 172.21.15.90:0/872732419 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc003d20 con 0x55ad019b1e40 2024-05-27T07:52:48.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.443+0000 7fe0c2a00700 1 -- 172.21.15.90:0/872732419 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0cc00bc00 con 0x55ad019b1e40 2024-05-27T07:52:48.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.443+0000 7fe0c2a00700 1 -- 172.21.15.90:0/872732419 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc0178d0 con 0x55ad019b1e40 2024-05-27T07:52:48.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.444+0000 7fe0eb094b80 1 -- 172.21.15.90:0/872732419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad019b1e40 msgr2=0x55ad01b367f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.444+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/872732419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad019b1e40 0x55ad01b367f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc003740 tx=0x7fe0cc005050 comp rx=0 tx=0).stop 2024-05-27T07:52:48.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.445+0000 7fe0eb094b80 1 -- 172.21.15.90:0/872732419 shutdown_connections 2024-05-27T07:52:48.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.445+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/872732419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.445+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/872732419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad019b1e40 0x55ad01b367f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.445+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/872732419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 0x55ad01b30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.445+0000 7fe0eb094b80 1 -- 172.21.15.90:0/872732419 >> 172.21.15.90:0/872732419 conn(0x55ad01a11fc0 msgr2=0x55ad01b2b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.445+0000 7fe0eb094b80 1 -- 172.21.15.90:0/872732419 shutdown_connections 2024-05-27T07:52:48.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.445+0000 7fe0eb094b80 1 -- 172.21.15.90:0/872732419 wait complete. 2024-05-27T07:52:48.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.446+0000 7fe0eb094b80 1 -- start start 2024-05-27T07:52:48.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.446+0000 7fe0eb094b80 1 Processor -- start 2024-05-27T07:52:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.447+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad019b1e40 0x55ad01b3a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.447+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b30db0 0x55ad01b3a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.447+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 0x55ad01b3af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.447+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ad01954fa0 con 0x55ad01b36d30 2024-05-27T07:52:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.447+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ad019ef760 con 0x55ad01b30db0 2024-05-27T07:52:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.447+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ad01b3d800 con 0x55ad019b1e40 2024-05-27T07:52:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.448+0000 7fe0eb094b80 10 client.0 _ll_register_callbacks cb 0x55ad019f8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.448+0000 7fe0d9200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b30db0 0x55ad01b3a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.449+0000 7fe0d9c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad019b1e40 0x55ad01b3a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.449+0000 7fe0da600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 0x55ad01b3af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.449+0000 7fe0da600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 0x55ad01b3af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44040/0 (socket says 172.21.15.90:44040) 2024-05-27T07:52:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.449+0000 7fe0da600700 1 -- 172.21.15.90:0/294924873 learned_addr learned my addr 172.21.15.90:0/294924873 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.450+0000 7fe0da600700 1 -- 172.21.15.90:0/294924873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad019b1e40 msgr2=0x55ad01b3a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.450+0000 7fe0da600700 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad019b1e40 0x55ad01b3a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.450+0000 7fe0da600700 1 -- 172.21.15.90:0/294924873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b30db0 msgr2=0x55ad01b3a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.450+0000 7fe0da600700 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b30db0 0x55ad01b3a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.450+0000 7fe0da600700 1 -- 172.21.15.90:0/294924873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad0199e8d0 con 0x55ad01b36d30 2024-05-27T07:52:48.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.451+0000 7fe0da600700 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 0x55ad01b3af10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0098e0 tx=0x7fe0cc00bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92ebd3bf25b327c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.451+0000 7fe0b7400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:52:48.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.452+0000 7fe0b7400700 1 -- 172.21.15.90:0/294924873 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc03c060 con 0x55ad01b36d30 2024-05-27T07:52:48.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.452+0000 7fe0b7400700 1 -- 172.21.15.90:0/294924873 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fe0cc00b980 con 0x55ad01b36d30 2024-05-27T07:52:48.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.452+0000 7fe0eb094b80 1 -- 172.21.15.90:0/294924873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0cc00bc00 con 0x55ad01b36d30 2024-05-27T07:52:48.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.453+0000 7fe0b7400700 1 -- 172.21.15.90:0/294924873 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe0cc017570 con 0x55ad01b36d30 2024-05-27T07:52:48.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.453+0000 7fe0eb094b80 10 client.16752 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:48.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.453+0000 7fe0eb094b80 1 -- 172.21.15.90:0/294924873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ad01a1f060 con 0x55ad01b36d30 2024-05-27T07:52:48.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.454+0000 7fe0b7400700 1 -- 172.21.15.90:0/294924873 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0cc00bc00 con 0x55ad01b36d30 2024-05-27T07:52:48.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.454+0000 7fe0b7400700 1 -- 172.21.15.90:0/294924873 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fe0cc00f670 con 0x55ad01b36d30 2024-05-27T07:52:48.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.454+0000 7fe0eb094b80 10 client.16752 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:48.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.454+0000 7fe0eb094b80 10 client.16752 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:52:48.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.455+0000 7fe0eb094b80 20 client.16752 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:48.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.455+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55ad01b5b540 0x55ad01b5d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.455+0000 7fe0eb094b80 4 client.16752 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:48.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.455+0000 7fe0eb094b80 1 -- 172.21.15.90:0/294924873 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad01a60d90 con 0x55ad01b5b540 2024-05-27T07:52:48.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.456+0000 7fe0d9c00700 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55ad01b5b540 0x55ad01b5d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.459+0000 7fe0d9c00700 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55ad01b5b540 0x55ad01b5d960 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6610fb33317566ae server_cookie=3ac300c2c5c2c0f1 in_seq=0 out_seq=0 2024-05-27T07:52:48.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.459+0000 7fe0b7400700 10 client.16752 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:52:48.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.460+0000 7fe0b7400700 1 -- 172.21.15.90:0/294924873 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ad01a60d90 con 0x55ad01b5b540 2024-05-27T07:52:48.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.460+0000 7fe0b7400700 10 client.16752 handle_command_reply: tid=0 2024-05-27T07:52:48.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.461+0000 7fe0eb094b80 1 client.16752 shutdown 2024-05-27T07:52:48.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.461+0000 7fe0eb094b80 1 -- 172.21.15.90:0/294924873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 msgr2=0x55ad01b3af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.461+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 0x55ad01b3af10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0098e0 tx=0x7fe0cc00bfc0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.462+0000 7fe0eb094b80 1 -- 172.21.15.90:0/294924873 shutdown_connections 2024-05-27T07:52:48.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.462+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad01b36d30 0x55ad01b3af10 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.462+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad019b1e40 0x55ad01b3a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.462+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55ad01b5b540 0x55ad01b5d960 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.462+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/294924873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b30db0 0x55ad01b3a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.462+0000 7fe0b7400700 0 client.16752 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:52:48.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.462+0000 7fe0eb094b80 1 -- 172.21.15.90:0/294924873 >> 172.21.15.90:0/294924873 conn(0x55ad01a11fc0 msgr2=0x55ad01b2b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.463+0000 7fe0eb094b80 1 -- 172.21.15.90:0/294924873 shutdown_connections 2024-05-27T07:52:48.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.463+0000 7fe0eb094b80 1 -- 172.21.15.90:0/294924873 wait complete. 2024-05-27T07:52:48.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.463+0000 7fe0eb094b80 20 client.16752 trim_cache size 0 max 16384 2024-05-27T07:52:48.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.492+0000 7fe0eb094b80 1 Processor -- start 2024-05-27T07:52:48.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.492+0000 7fe0eb094b80 1 -- start start 2024-05-27T07:52:48.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.492+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad019b1e40 0x55ad01b30870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.492+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b34920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.492+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b3a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.492+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b80108d0 con 0x55ad019b1e40 2024-05-27T07:52:48.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.492+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b800e6c0 con 0x55ad01b36d30 2024-05-27T07:52:48.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.492+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b8010200 con 0x55ad01b30db0 2024-05-27T07:52:48.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.493+0000 7fe0d9c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad019b1e40 0x55ad01b30870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.494+0000 7fe0d9200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b34920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.494+0000 7fe0d9200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b34920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37714/0 (socket says 172.21.15.90:37714) 2024-05-27T07:52:48.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.494+0000 7fe0d9200700 1 -- 172.21.15.90:0/1950919425 learned_addr learned my addr 172.21.15.90:0/1950919425 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:48.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.494+0000 7fe0da600700 1 --2- 172.21.15.90:0/1950919425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b3a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.494+0000 7fe0da600700 1 -- 172.21.15.90:0/1950919425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 msgr2=0x55ad01b34920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.494+0000 7fe0da600700 1 --2- 172.21.15.90:0/1950919425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b34920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.494+0000 7fe0da600700 1 -- 172.21.15.90:0/1950919425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad019b1e40 msgr2=0x55ad01b30870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.494+0000 7fe0da600700 1 --2- 172.21.15.90:0/1950919425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad019b1e40 0x55ad01b30870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.495+0000 7fe0da600700 1 -- 172.21.15.90:0/1950919425 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad01b3d8a0 con 0x55ad01b36d30 2024-05-27T07:52:48.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.495+0000 7fe0da600700 1 --2- 172.21.15.90:0/1950919425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b3a270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00fa40 tx=0x7fe0cc009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d81aaa479cdb3d53 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.496+0000 7fe0c2000700 1 -- 172.21.15.90:0/1950919425 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc03c040 con 0x55ad01b36d30 2024-05-27T07:52:48.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.496+0000 7fe0c2000700 1 -- 172.21.15.90:0/1950919425 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0cc00b560 con 0x55ad01b36d30 2024-05-27T07:52:48.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.496+0000 7fe0c2000700 1 -- 172.21.15.90:0/1950919425 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc046080 con 0x55ad01b36d30 2024-05-27T07:52:48.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.497+0000 7fe0eb094b80 1 -- 172.21.15.90:0/1950919425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 msgr2=0x55ad01b3a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.498+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/1950919425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b3a270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00fa40 tx=0x7fe0cc009ee0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.498+0000 7fe0eb094b80 1 -- 172.21.15.90:0/1950919425 shutdown_connections 2024-05-27T07:52:48.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.498+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/1950919425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b3a270 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.498+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/1950919425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad019b1e40 0x55ad01b30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.498+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/1950919425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b34920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.498+0000 7fe0eb094b80 1 -- 172.21.15.90:0/1950919425 >> 172.21.15.90:0/1950919425 conn(0x55ad01a11fc0 msgr2=0x55ad01b34210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.498+0000 7fe0eb094b80 1 -- 172.21.15.90:0/1950919425 shutdown_connections 2024-05-27T07:52:48.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.498+0000 7fe0eb094b80 1 -- 172.21.15.90:0/1950919425 wait complete. 2024-05-27T07:52:48.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.499+0000 7fe0eb094b80 1 -- start start 2024-05-27T07:52:48.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.499+0000 7fe0eb094b80 1 Processor -- start 2024-05-27T07:52:48.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:48 smithi136 ceph-mon[35324]: pgmap v199: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 512 op/s 2024-05-27T07:52:48.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.500+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad019b1e40 0x55ad01b304b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.500+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b5ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.500+0000 7fe0eb094b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b5f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.500+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b800f460 con 0x55ad019b1e40 2024-05-27T07:52:48.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.500+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b800ed90 con 0x55ad01b36d30 2024-05-27T07:52:48.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.500+0000 7fe0eb094b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b800fb30 con 0x55ad01b30db0 2024-05-27T07:52:48.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.500+0000 7fe0eb094b80 10 client.0 _ll_register_callbacks cb 0x55ad019f8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:48.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.501+0000 7fe0d9200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b5ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.501+0000 7fe0da600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b5f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.501+0000 7fe0da600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b5f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52604/0 (socket says 172.21.15.90:52604) 2024-05-27T07:52:48.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.501+0000 7fe0da600700 1 -- 172.21.15.90:0/268863774 learned_addr learned my addr 172.21.15.90:0/268863774 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:48.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.501+0000 7fe0da600700 1 -- 172.21.15.90:0/268863774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 msgr2=0x55ad01b5ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.501+0000 7fe0da600700 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b5ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.501+0000 7fe0da600700 1 -- 172.21.15.90:0/268863774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad019b1e40 msgr2=0x55ad01b304b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:48.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.501+0000 7fe0da600700 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad019b1e40 0x55ad01b304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.501+0000 7fe0da600700 1 -- 172.21.15.90:0/268863774 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad01b30ac0 con 0x55ad01b36d30 2024-05-27T07:52:48.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.502+0000 7fe0da600700 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b5f480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc006010 tx=0x7fe0cc00bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3694c36566fe2720 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:48.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.502+0000 7fe0b7400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:52:48.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.503+0000 7fe0b7400700 1 -- 172.21.15.90:0/268863774 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc046020 con 0x55ad01b36d30 2024-05-27T07:52:48.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.503+0000 7fe0b7400700 1 -- 172.21.15.90:0/268863774 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fe0cc017710 con 0x55ad01b36d30 2024-05-27T07:52:48.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.504+0000 7fe0eb094b80 1 -- 172.21.15.90:0/268863774 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0cc00b560 con 0x55ad01b36d30 2024-05-27T07:52:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.505+0000 7fe0b7400700 1 -- 172.21.15.90:0/268863774 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe0cc03c070 con 0x55ad01b36d30 2024-05-27T07:52:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.505+0000 7fe0eb094b80 10 client.36260 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.505+0000 7fe0eb094b80 1 -- 172.21.15.90:0/268863774 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ad01bc5730 con 0x55ad01b36d30 2024-05-27T07:52:48.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.506+0000 7fe0b7400700 1 -- 172.21.15.90:0/268863774 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0cc00b560 con 0x55ad01b36d30 2024-05-27T07:52:48.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.506+0000 7fe0b7400700 1 -- 172.21.15.90:0/268863774 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fe0cc023b20 con 0x55ad01b36d30 2024-05-27T07:52:48.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.506+0000 7fe0eb094b80 10 client.36260 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:48.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.506+0000 7fe0eb094b80 10 client.36260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:52:48.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.506+0000 7fe0eb094b80 20 client.36260 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:48.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.506+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55ad01bc5c20 0x55ad01bc8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:48.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.506+0000 7fe0eb094b80 4 client.36260 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:52:48.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.506+0000 7fe0eb094b80 1 -- 172.21.15.90:0/268863774 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad01a5a570 con 0x55ad01bc5c20 2024-05-27T07:52:48.520 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:48 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-05-27T07:52:48.520 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:48 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.745s CPU time. 2024-05-27T07:52:48.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.521+0000 7fe0d9c00700 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55ad01bc5c20 0x55ad01bc8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:48.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.523+0000 7fe0d9c00700 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55ad01bc5c20 0x55ad01bc8040 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d8a94da743c3d35 server_cookie=cc51e8d579d5f049 in_seq=0 out_seq=0 2024-05-27T07:52:48.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.523+0000 7fe0b7400700 10 client.36260 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:52:48.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.524+0000 7fe0b7400700 1 -- 172.21.15.90:0/268863774 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad01a5a570 con 0x55ad01bc5c20 2024-05-27T07:52:48.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.524+0000 7fe0b7400700 10 client.36260 handle_command_reply: tid=0 2024-05-27T07:52:48.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.524+0000 7fe0eb094b80 1 client.36260 shutdown 2024-05-27T07:52:48.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.525+0000 7fe0eb094b80 1 -- 172.21.15.90:0/268863774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 msgr2=0x55ad01b5f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.525+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b5f480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc006010 tx=0x7fe0cc00bc30 comp rx=0 tx=0).stop 2024-05-27T07:52:48.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.525+0000 7fe0eb094b80 1 -- 172.21.15.90:0/268863774 shutdown_connections 2024-05-27T07:52:48.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.525+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55ad01bc5c20 0x55ad01bc8040 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.525+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ad01b36d30 0x55ad01b5f480 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.525+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ad019b1e40 0x55ad01b304b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.525+0000 7fe0eb094b80 1 --2- 172.21.15.90:0/268863774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ad01b30db0 0x55ad01b5ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.525+0000 7fe0eb094b80 1 -- 172.21.15.90:0/268863774 >> 172.21.15.90:0/268863774 conn(0x55ad01a11fc0 msgr2=0x55ad01b45b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.525+0000 7fe0b7400700 0 client.36260 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:52:48.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.526+0000 7fe0eb094b80 1 -- 172.21.15.90:0/268863774 shutdown_connections 2024-05-27T07:52:48.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.526+0000 7fe0eb094b80 1 -- 172.21.15.90:0/268863774 wait complete. 2024-05-27T07:52:48.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.526+0000 7fe0eb094b80 20 client.36260 trim_cache size 0 max 16384 2024-05-27T07:52:48.531 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:52:48.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.531+0000 7fe0dc600700 1 -- 172.21.15.90:0/1919308986 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe0bc0b2c00 msgr2=0x7fe0bc0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.531+0000 7fe0dc600700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe0bc0b2c00 0x7fe0bc0b5110 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc02a000 tx=0x7fe0cc02d000 comp rx=0 tx=0).stop 2024-05-27T07:52:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.531+0000 7fe0dc600700 1 -- 172.21.15.90:0/1919308986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0d41030d0 msgr2=0x7fe0d410f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.531+0000 7fe0dc600700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0d41030d0 0x7fe0d410f180 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400ba50 tx=0x7fe0c400bd60 comp rx=0 tx=0).stop 2024-05-27T07:52:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.532+0000 7fe0dc600700 1 -- 172.21.15.90:0/1919308986 shutdown_connections 2024-05-27T07:52:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.532+0000 7fe0dc600700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe0bc0b2c00 0x7fe0bc0b5110 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.532+0000 7fe0dc600700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d4110c80 0x7fe0d41aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.532+0000 7fe0dc600700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0d41030d0 0x7fe0d410f180 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.532+0000 7fe0dc600700 1 --2- 172.21.15.90:0/1919308986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0d4103a20 0x7fe0d410f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.532+0000 7fe0dc600700 1 -- 172.21.15.90:0/1919308986 >> 172.21.15.90:0/1919308986 conn(0x7fe0d40fedb0 msgr2=0x7fe0d4112800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.532+0000 7fe0dc600700 1 -- 172.21.15.90:0/1919308986 shutdown_connections 2024-05-27T07:52:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:48.532+0000 7fe0dc600700 1 -- 172.21.15.90:0/1919308986 wait complete. 2024-05-27T07:52:48.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:48 smithi129 ceph-mon[35254]: pgmap v199: 129 pgs: 129 active+clean; 1.9 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 4.4 MiB/s wr, 512 op/s 2024-05-27T07:52:48.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 150.0 s, 507.2 tps, lat 189.217 ms stddev 237.376 2024-05-27T07:52:49.483 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:52:49.982 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:50.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:50 smithi090 ceph-mon[31429]: pgmap v200: 129 pgs: 129 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 824 op/s 2024-05-27T07:52:50.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:50 smithi136 ceph-mon[35324]: pgmap v200: 129 pgs: 129 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 824 op/s 2024-05-27T07:52:50.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:50 smithi129 ceph-mon[35254]: pgmap v200: 129 pgs: 129 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 10 MiB/s wr, 824 op/s 2024-05-27T07:52:52.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:52 smithi090 ceph-mon[31429]: pgmap v201: 129 pgs: 129 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.7 MiB/s wr, 577 op/s 2024-05-27T07:52:52.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:52 smithi136 ceph-mon[35324]: pgmap v201: 129 pgs: 129 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.7 MiB/s wr, 577 op/s 2024-05-27T07:52:52.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.513+0000 7fd19fc00700 1 -- 172.21.15.90:0/1180235390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd198106eb0 msgr2=0x7fd198113340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.513+0000 7fd19fc00700 1 --2- 172.21.15.90:0/1180235390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd198106eb0 0x7fd198113340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd19000b0a0 tx=0x7fd19000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.514+0000 7fd19fc00700 1 -- 172.21.15.90:0/1180235390 shutdown_connections 2024-05-27T07:52:52.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.514+0000 7fd19fc00700 1 --2- 172.21.15.90:0/1180235390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd198106eb0 0x7fd198113340 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.514+0000 7fd19fc00700 1 --2- 172.21.15.90:0/1180235390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19810c5a0 0x7fd19810c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.514+0000 7fd19fc00700 1 --2- 172.21.15.90:0/1180235390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd198106560 0x7fd198106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.514+0000 7fd19fc00700 1 -- 172.21.15.90:0/1180235390 >> 172.21.15.90:0/1180235390 conn(0x7fd198102040 msgr2=0x7fd198104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.514+0000 7fd19fc00700 1 -- 172.21.15.90:0/1180235390 shutdown_connections 2024-05-27T07:52:52.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.514+0000 7fd19fc00700 1 -- 172.21.15.90:0/1180235390 wait complete. 2024-05-27T07:52:52.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.515+0000 7fd19fc00700 1 Processor -- start 2024-05-27T07:52:52.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.515+0000 7fd19fc00700 1 -- start start 2024-05-27T07:52:52.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.516+0000 7fd19fc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd198106560 0x7fd198114ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.516+0000 7fd19fc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd198106eb0 0x7fd1981150e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.516+0000 7fd19fc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19810c5a0 0x7fd19811bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.516+0000 7fd19fc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd198079620 con 0x7fd198106560 2024-05-27T07:52:52.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.516+0000 7fd19fc00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980794a0 con 0x7fd19810c5a0 2024-05-27T07:52:52.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.516+0000 7fd19fc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980797a0 con 0x7fd198106eb0 2024-05-27T07:52:52.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.517+0000 7fd19dc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19810c5a0 0x7fd19811bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.517+0000 7fd19dc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19810c5a0 0x7fd19811bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:35324/0 (socket says 172.21.15.90:35324) 2024-05-27T07:52:52.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.517+0000 7fd19dc00700 1 -- 172.21.15.90:0/1274099399 learned_addr learned my addr 172.21.15.90:0/1274099399 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:52.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.517+0000 7fd19dc00700 1 -- 172.21.15.90:0/1274099399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd198106eb0 msgr2=0x7fd1981150e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.517+0000 7fd19d200700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd198106560 0x7fd198114ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.517+0000 7fd19dc00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd198106eb0 0x7fd1981150e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.517+0000 7fd19dc00700 1 -- 172.21.15.90:0/1274099399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd198106560 msgr2=0x7fd198114ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.517+0000 7fd19dc00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd198106560 0x7fd198114ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.517+0000 7fd19dc00700 1 -- 172.21.15.90:0/1274099399 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1981159e0 con 0x7fd19810c5a0 2024-05-27T07:52:52.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.518+0000 7fd19dc00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19810c5a0 0x7fd19811bc00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd19000b070 tx=0x7fd190004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.518+0000 7fd195600700 1 -- 172.21.15.90:0/1274099399 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190021070 con 0x7fd19810c5a0 2024-05-27T07:52:52.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.519+0000 7fd19fc00700 1 -- 172.21.15.90:0/1274099399 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19811c200 con 0x7fd19810c5a0 2024-05-27T07:52:52.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.519+0000 7fd19fc00700 1 -- 172.21.15.90:0/1274099399 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19811c6e0 con 0x7fd19810c5a0 2024-05-27T07:52:52.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.519+0000 7fd195600700 1 -- 172.21.15.90:0/1274099399 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1900075c0 con 0x7fd19810c5a0 2024-05-27T07:52:52.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.519+0000 7fd195600700 1 -- 172.21.15.90:0/1274099399 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190004410 con 0x7fd19810c5a0 2024-05-27T07:52:52.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.526+0000 7fd1ae670b80 1 Processor -- start 2024-05-27T07:52:52.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.527+0000 7fd195600700 1 -- 172.21.15.90:0/1274099399 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd1900045b0 con 0x7fd19810c5a0 2024-05-27T07:52:52.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.528+0000 7fd1ae670b80 1 -- start start 2024-05-27T07:52:52.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.528+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 0x556bb47247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.528+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb471e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.528+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb471edb0 0x556bb471f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.528+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556bb464ce40 con 0x556bb459fe40 2024-05-27T07:52:52.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.528+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556bb463c9e0 con 0x556bb471edb0 2024-05-27T07:52:52.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.528+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556bb4640a20 con 0x556bb4724d30 2024-05-27T07:52:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.529+0000 7fd19d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb471e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.529+0000 7fd19d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb471e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39680/0 (socket says 172.21.15.90:39680) 2024-05-27T07:52:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.529+0000 7fd19d200700 1 -- 172.21.15.90:0/1651902891 learned_addr learned my addr 172.21.15.90:0/1651902891 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:52.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.529+0000 7fd197e00700 1 --2- 172.21.15.90:0/1651902891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb471edb0 0x556bb471f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.530+0000 7fd197e00700 1 --2- 172.21.15.90:0/1651902891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 0x556bb47247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.530+0000 7fd19d200700 1 -- 172.21.15.90:0/1651902891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb471edb0 msgr2=0x556bb471f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.530+0000 7fd19d200700 1 --2- 172.21.15.90:0/1651902891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb471edb0 0x556bb471f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.530+0000 7fd19d200700 1 -- 172.21.15.90:0/1651902891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 msgr2=0x556bb47247f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.530+0000 7fd195600700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1800b2a70 0x7fd1800b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.531+0000 7fd19d200700 1 --2- 172.21.15.90:0/1651902891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 0x556bb47247f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.531+0000 7fd195600700 1 -- 172.21.15.90:0/1274099399 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fd1900e3be0 con 0x7fd19810c5a0 2024-05-27T07:52:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.531+0000 7fd19d200700 1 -- 172.21.15.90:0/1651902891 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb4605420 con 0x556bb4724d30 2024-05-27T07:52:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.531+0000 7fd197e00700 1 --2- 172.21.15.90:0/1651902891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 0x556bb47247f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:52:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.531+0000 7fd19d200700 1 --2- 172.21.15.90:0/1651902891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb471e870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd184009c70 tx=0x7fd184009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621a259dbcd024e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.532+0000 7fd197e00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1800b2a70 0x7fd1800b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.533+0000 7fd18a000700 1 -- 172.21.15.90:0/1651902891 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18400fc20 con 0x556bb4724d30 2024-05-27T07:52:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.533+0000 7fd18a000700 1 -- 172.21.15.90:0/1651902891 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1840044b0 con 0x556bb4724d30 2024-05-27T07:52:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.533+0000 7fd18a000700 1 -- 172.21.15.90:0/1651902891 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd184011440 con 0x556bb4724d30 2024-05-27T07:52:52.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.535+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1651902891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 msgr2=0x556bb471e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.535+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/1651902891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb471e870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd184009c70 tx=0x7fd184009f80 comp rx=0 tx=0).stop 2024-05-27T07:52:52.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.535+0000 7fd197e00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1800b2a70 0x7fd1800b4f80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x556bb471fb30 tx=0x7fd18c009680 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.536+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1651902891 shutdown_connections 2024-05-27T07:52:52.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.536+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/1651902891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb471edb0 0x556bb471f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.536+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/1651902891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 0x556bb47247f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.536+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/1651902891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb471e870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.536+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1651902891 >> 172.21.15.90:0/1651902891 conn(0x556bb45fffc0 msgr2=0x556bb47198d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.537+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1651902891 shutdown_connections 2024-05-27T07:52:52.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.537+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1651902891 wait complete. 2024-05-27T07:52:52.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.538+0000 7fd1ae670b80 1 -- start start 2024-05-27T07:52:52.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.538+0000 7fd1ae670b80 1 Processor -- start 2024-05-27T07:52:52.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.539+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 0x556bb4728510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.539+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb471edb0 0x556bb4728a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.539+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb4724d30 0x556bb4728f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.539+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556bb4542fa0 con 0x556bb4724d30 2024-05-27T07:52:52.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.539+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556bb45dd760 con 0x556bb459fe40 2024-05-27T07:52:52.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.539+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556bb472b800 con 0x556bb471edb0 2024-05-27T07:52:52.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.539+0000 7fd1ae670b80 10 client.0 _ll_register_callbacks cb 0x556bb45e6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:52.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.540+0000 7fd197e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb471edb0 0x556bb4728a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.540+0000 7fd197e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb471edb0 0x556bb4728a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39684/0 (socket says 172.21.15.90:39684) 2024-05-27T07:52:52.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.540+0000 7fd197e00700 1 -- 172.21.15.90:0/3534137933 learned_addr learned my addr 172.21.15.90:0/3534137933 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:52.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.540+0000 7fd19dc00700 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb4724d30 0x556bb4728f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.540+0000 7fd19d200700 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 0x556bb4728510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.541+0000 7fd197e00700 1 -- 172.21.15.90:0/3534137933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 msgr2=0x556bb4728510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.541+0000 7fd197e00700 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 0x556bb4728510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.541+0000 7fd197e00700 1 -- 172.21.15.90:0/3534137933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb4724d30 msgr2=0x556bb4728f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.541+0000 7fd197e00700 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb4724d30 0x556bb4728f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.541+0000 7fd197e00700 1 -- 172.21.15.90:0/3534137933 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bb458c8d0 con 0x556bb471edb0 2024-05-27T07:52:52.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.541+0000 7fd19dc00700 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb4724d30 0x556bb4728f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:52.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.541+0000 7fd197e00700 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb471edb0 0x556bb4728a50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd18c010c20 tx=0x7fd18c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7341e8760b396e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.541+0000 7fd17ea00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:52:52.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.542+0000 7fd17ea00700 1 -- 172.21.15.90:0/3534137933 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00ec80 con 0x556bb471edb0 2024-05-27T07:52:52.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.542+0000 7fd17ea00700 1 -- 172.21.15.90:0/3534137933 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fd18c00ee20 con 0x556bb471edb0 2024-05-27T07:52:52.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.542+0000 7fd1ae670b80 1 -- 172.21.15.90:0/3534137933 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1840044b0 con 0x556bb471edb0 2024-05-27T07:52:52.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.542+0000 7fd17ea00700 1 -- 172.21.15.90:0/3534137933 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd18c004030 con 0x556bb471edb0 2024-05-27T07:52:52.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.542+0000 7fd1ae670b80 10 client.26320 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:52.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.542+0000 7fd1ae670b80 1 -- 172.21.15.90:0/3534137933 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556bb460d060 con 0x556bb471edb0 2024-05-27T07:52:52.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.543+0000 7fd17ea00700 1 -- 172.21.15.90:0/3534137933 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd1840044b0 con 0x556bb471edb0 2024-05-27T07:52:52.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.543+0000 7fd17ea00700 1 -- 172.21.15.90:0/3534137933 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fd18c005020 con 0x556bb471edb0 2024-05-27T07:52:52.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.544+0000 7fd1ae670b80 10 client.26320 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:52.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.544+0000 7fd1ae670b80 10 client.26320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:52.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.544+0000 7fd1ae670b80 20 client.26320 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:52.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.544+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556bb47387c0 0x556bb473ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.544+0000 7fd1ae670b80 4 client.26320 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:52.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.544+0000 7fd1ae670b80 1 -- 172.21.15.90:0/3534137933 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bb464ed90 con 0x556bb47387c0 2024-05-27T07:52:52.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.545+0000 7fd19d200700 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556bb47387c0 0x556bb473ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.547+0000 7fd19d200700 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556bb47387c0 0x556bb473ac70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb29e1b908d7e0b server_cookie=5d431b414c59cbb2 in_seq=0 out_seq=0 2024-05-27T07:52:52.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.547+0000 7fd17ea00700 10 client.26320 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:52.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd17ea00700 1 -- 172.21.15.90:0/3534137933 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556bb464ed90 con 0x556bb47387c0 2024-05-27T07:52:52.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd17ea00700 10 client.26320 handle_command_reply: tid=0 2024-05-27T07:52:52.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd1ae670b80 1 client.26320 shutdown 2024-05-27T07:52:52.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd1ae670b80 1 -- 172.21.15.90:0/3534137933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb471edb0 msgr2=0x556bb4728a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb471edb0 0x556bb4728a50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd18c010c20 tx=0x7fd18c004750 comp rx=0 tx=0).stop 2024-05-27T07:52:52.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd1ae670b80 1 -- 172.21.15.90:0/3534137933 shutdown_connections 2024-05-27T07:52:52.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556bb47387c0 0x556bb473ac70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb4724d30 0x556bb4728f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 0x556bb4728510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd17ea00700 0 client.26320 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:52.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/3534137933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb471edb0 0x556bb4728a50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.549+0000 7fd1ae670b80 1 -- 172.21.15.90:0/3534137933 >> 172.21.15.90:0/3534137933 conn(0x556bb45fffc0 msgr2=0x556bb4719580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.550+0000 7fd1ae670b80 1 -- 172.21.15.90:0/3534137933 shutdown_connections 2024-05-27T07:52:52.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.550+0000 7fd1ae670b80 1 -- 172.21.15.90:0/3534137933 wait complete. 2024-05-27T07:52:52.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.550+0000 7fd1ae670b80 20 client.26320 trim_cache size 0 max 16384 2024-05-27T07:52:52.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:52 smithi129 ceph-mon[35254]: pgmap v201: 129 pgs: 129 active+clean; 1.9 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.7 MiB/s wr, 577 op/s 2024-05-27T07:52:52.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.568+0000 7fd1ae670b80 1 Processor -- start 2024-05-27T07:52:52.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.568+0000 7fd1ae670b80 1 -- start start 2024-05-27T07:52:52.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.568+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 0x556bb471e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.568+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb471edb0 0x556bb4728be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.568+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb4740440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.568+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd178010780 con 0x556bb471edb0 2024-05-27T07:52:52.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.568+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd17800e570 con 0x556bb459fe40 2024-05-27T07:52:52.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.568+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1780100b0 con 0x556bb4724d30 2024-05-27T07:52:52.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.569+0000 7fd197e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb471edb0 0x556bb4728be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.569+0000 7fd197e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb471edb0 0x556bb4728be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44072/0 (socket says 172.21.15.90:44072) 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.569+0000 7fd197e00700 1 -- 172.21.15.90:0/2222523407 learned_addr learned my addr 172.21.15.90:0/2222523407 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.569+0000 7fd19dc00700 1 --2- 172.21.15.90:0/2222523407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb4740440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.569+0000 7fd19d200700 1 --2- 172.21.15.90:0/2222523407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 0x556bb471e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.569+0000 7fd197e00700 1 -- 172.21.15.90:0/2222523407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 msgr2=0x556bb4740440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.569+0000 7fd197e00700 1 --2- 172.21.15.90:0/2222523407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb4740440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.569+0000 7fd197e00700 1 -- 172.21.15.90:0/2222523407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 msgr2=0x556bb471e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.569+0000 7fd197e00700 1 --2- 172.21.15.90:0/2222523407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 0x556bb471e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.569+0000 7fd197e00700 1 -- 172.21.15.90:0/2222523407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bb472b6e0 con 0x556bb471edb0 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.570+0000 7fd19d200700 1 --2- 172.21.15.90:0/2222523407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 0x556bb471e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.570+0000 7fd197e00700 1 --2- 172.21.15.90:0/2222523407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb471edb0 0x556bb4728be0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0093d0 tx=0x7fd18c005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7256fb338fe3f81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.570+0000 7fd189600700 1 -- 172.21.15.90:0/2222523407 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c037070 con 0x556bb471edb0 2024-05-27T07:52:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.571+0000 7fd189600700 1 -- 172.21.15.90:0/2222523407 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd18c022070 con 0x556bb471edb0 2024-05-27T07:52:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.571+0000 7fd189600700 1 -- 172.21.15.90:0/2222523407 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c003d40 con 0x556bb471edb0 2024-05-27T07:52:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.572+0000 7fd1ae670b80 1 -- 172.21.15.90:0/2222523407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb471edb0 msgr2=0x556bb4728be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.572+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/2222523407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb471edb0 0x556bb4728be0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0093d0 tx=0x7fd18c005e40 comp rx=0 tx=0).stop 2024-05-27T07:52:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.572+0000 7fd1ae670b80 1 -- 172.21.15.90:0/2222523407 shutdown_connections 2024-05-27T07:52:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.572+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/2222523407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb4740440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.572+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/2222523407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb459fe40 0x556bb471e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.572+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/2222523407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb471edb0 0x556bb4728be0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.572+0000 7fd1ae670b80 1 -- 172.21.15.90:0/2222523407 >> 172.21.15.90:0/2222523407 conn(0x556bb45fffc0 msgr2=0x556bb47284d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.572+0000 7fd1ae670b80 1 -- 172.21.15.90:0/2222523407 shutdown_connections 2024-05-27T07:52:52.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.572+0000 7fd1ae670b80 1 -- 172.21.15.90:0/2222523407 wait complete. 2024-05-27T07:52:52.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.573+0000 7fd1ae670b80 1 -- start start 2024-05-27T07:52:52.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.573+0000 7fd1ae670b80 1 Processor -- start 2024-05-27T07:52:52.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.574+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 0x556bb4730af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.574+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb471edb0 0x556bb4731030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.574+0000 7fd1ae670b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb47338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.574+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd17800f310 con 0x556bb459fe40 2024-05-27T07:52:52.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.574+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd17800ec40 con 0x556bb471edb0 2024-05-27T07:52:52.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.574+0000 7fd1ae670b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd17800f9e0 con 0x556bb4724d30 2024-05-27T07:52:52.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.574+0000 7fd1ae670b80 10 client.0 _ll_register_callbacks cb 0x556bb45e6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:52.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.574+0000 7fd19dc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb47338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.575+0000 7fd19dc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb47338f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39702/0 (socket says 172.21.15.90:39702) 2024-05-27T07:52:52.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.575+0000 7fd19d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 0x556bb4730af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.575+0000 7fd19dc00700 1 -- 172.21.15.90:0/1595581879 learned_addr learned my addr 172.21.15.90:0/1595581879 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:52.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.575+0000 7fd197e00700 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb471edb0 0x556bb4731030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.575+0000 7fd19dc00700 1 -- 172.21.15.90:0/1595581879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb471edb0 msgr2=0x556bb4731030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.575+0000 7fd19dc00700 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb471edb0 0x556bb4731030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.575+0000 7fd19dc00700 1 -- 172.21.15.90:0/1595581879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 msgr2=0x556bb4730af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:52.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.575+0000 7fd19dc00700 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 0x556bb4730af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.575+0000 7fd19dc00700 1 -- 172.21.15.90:0/1595581879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bb47316f0 con 0x556bb4724d30 2024-05-27T07:52:52.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.575+0000 7fd19dc00700 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb47338f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd1900a7e50 tx=0x7fd1900ab0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40a29180df74198a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:52.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.576+0000 7fd17ea00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:52:52.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.576+0000 7fd17ea00700 1 -- 172.21.15.90:0/1595581879 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1900c4060 con 0x556bb4724d30 2024-05-27T07:52:52.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.576+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1595581879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18c022070 con 0x556bb4724d30 2024-05-27T07:52:52.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.576+0000 7fd17ea00700 1 -- 172.21.15.90:0/1595581879 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fd1900a71d0 con 0x556bb4724d30 2024-05-27T07:52:52.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.576+0000 7fd17ea00700 1 -- 172.21.15.90:0/1595581879 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd19002a040 con 0x556bb4724d30 2024-05-27T07:52:52.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.576+0000 7fd1ae670b80 10 client.26326 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:52.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.576+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1595581879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556bb47a2900 con 0x556bb4724d30 2024-05-27T07:52:52.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.577+0000 7fd17ea00700 1 -- 172.21.15.90:0/1595581879 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd18c022070 con 0x556bb4724d30 2024-05-27T07:52:52.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.577+0000 7fd17ea00700 1 -- 172.21.15.90:0/1595581879 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fd190087080 con 0x556bb4724d30 2024-05-27T07:52:52.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.577+0000 7fd1ae670b80 10 client.26326 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:52.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.577+0000 7fd1ae670b80 10 client.26326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:52.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.577+0000 7fd1ae670b80 20 client.26326 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:52.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.577+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556bb47a2f10 0x556bb47a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:52.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.577+0000 7fd1ae670b80 4 client.26326 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:52:52.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.577+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1595581879 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bb4648570 con 0x556bb47a2f10 2024-05-27T07:52:52.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.578+0000 7fd19d200700 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556bb47a2f10 0x556bb47a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:52.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.579+0000 7fd19d200700 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556bb47a2f10 0x556bb47a5330 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ff4b117f35adb5e server_cookie=77ec1e887de27036 in_seq=0 out_seq=0 2024-05-27T07:52:52.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.579+0000 7fd17ea00700 10 client.26326 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:52.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.579+0000 7fd17ea00700 1 -- 172.21.15.90:0/1595581879 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bb4648570 con 0x556bb47a2f10 2024-05-27T07:52:52.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.579+0000 7fd17ea00700 10 client.26326 handle_command_reply: tid=0 2024-05-27T07:52:52.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.579+0000 7fd1ae670b80 1 client.26326 shutdown 2024-05-27T07:52:52.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1595581879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 msgr2=0x556bb47338f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb47338f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd1900a7e50 tx=0x7fd1900ab0c0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1595581879 shutdown_connections 2024-05-27T07:52:52.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556bb47a2f10 0x556bb47a5330 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556bb4724d30 0x556bb47338f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bb459fe40 0x556bb4730af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd1ae670b80 1 --2- 172.21.15.90:0/1595581879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bb471edb0 0x556bb4731030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1595581879 >> 172.21.15.90:0/1595581879 conn(0x556bb45fffc0 msgr2=0x556bb4728430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd17ea00700 0 client.26326 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:52.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1595581879 shutdown_connections 2024-05-27T07:52:52.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.580+0000 7fd1ae670b80 1 -- 172.21.15.90:0/1595581879 wait complete. 2024-05-27T07:52:52.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd1ae670b80 20 client.26326 trim_cache size 0 max 16384 2024-05-27T07:52:52.582 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:52:52.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd19fc00700 1 -- 172.21.15.90:0/1274099399 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1800b2a70 msgr2=0x7fd1800b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd19fc00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1800b2a70 0x7fd1800b4f80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x556bb471fb30 tx=0x7fd18c009680 comp rx=0 tx=0).stop 2024-05-27T07:52:52.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd19fc00700 1 -- 172.21.15.90:0/1274099399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19810c5a0 msgr2=0x7fd19811bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:52.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd19fc00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19810c5a0 0x7fd19811bc00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd19000b070 tx=0x7fd190004d10 comp rx=0 tx=0).stop 2024-05-27T07:52:52.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd19fc00700 1 -- 172.21.15.90:0/1274099399 shutdown_connections 2024-05-27T07:52:52.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd19fc00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd1800b2a70 0x7fd1800b4f80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd19fc00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd19810c5a0 0x7fd19811bc00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd19fc00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd198106560 0x7fd198114ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd19fc00700 1 --2- 172.21.15.90:0/1274099399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd198106eb0 0x7fd1981150e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:52.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.581+0000 7fd19fc00700 1 -- 172.21.15.90:0/1274099399 >> 172.21.15.90:0/1274099399 conn(0x7fd198102040 msgr2=0x7fd198102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:52.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.582+0000 7fd19fc00700 1 -- 172.21.15.90:0/1274099399 shutdown_connections 2024-05-27T07:52:52.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:52.582+0000 7fd19fc00700 1 -- 172.21.15.90:0/1274099399 wait complete. 2024-05-27T07:52:53.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 155.0 s, 519.0 tps, lat 201.262 ms stddev 362.856 2024-05-27T07:52:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.077+0000 7fb73f000640 1 -- 172.21.15.90:0/1895780723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb73810fcd0 msgr2=0x7fb738110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.077+0000 7fb73f000640 1 --2- 172.21.15.90:0/1895780723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb73810fcd0 0x7fb738110100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb724009860 tx=0x7fb72402a340 comp rx=0 tx=0).stop 2024-05-27T07:52:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.077+0000 7fb73f000640 1 -- 172.21.15.90:0/1895780723 shutdown_connections 2024-05-27T07:52:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.077+0000 7fb73f000640 1 --2- 172.21.15.90:0/1895780723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb738110790 0x7fb738116d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.077+0000 7fb73f000640 1 --2- 172.21.15.90:0/1895780723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb73810fcd0 0x7fb738110100 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.077+0000 7fb73f000640 1 --2- 172.21.15.90:0/1895780723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb73810b1d0 0x7fb73810b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.077+0000 7fb73f000640 1 -- 172.21.15.90:0/1895780723 >> 172.21.15.90:0/1895780723 conn(0x7fb7380795c0 msgr2=0x7fb73807b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.077+0000 7fb73f000640 1 -- 172.21.15.90:0/1895780723 shutdown_connections 2024-05-27T07:52:54.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.077+0000 7fb73f000640 1 -- 172.21.15.90:0/1895780723 wait complete. 2024-05-27T07:52:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.079+0000 7fb73f000640 1 Processor -- start 2024-05-27T07:52:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.079+0000 7fb73f000640 1 -- start start 2024-05-27T07:52:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.080+0000 7fb73f000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb73810b1d0 0x7fb7381a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.080+0000 7fb73f000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb73810fcd0 0x7fb7381a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.080+0000 7fb73f000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb738110790 0x7fb7381a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.080+0000 7fb73f000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb73811a300 con 0x7fb73810b1d0 2024-05-27T07:52:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.080+0000 7fb73f000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb73811a180 con 0x7fb73810fcd0 2024-05-27T07:52:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.080+0000 7fb73f000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb73811a480 con 0x7fb738110790 2024-05-27T07:52:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.080+0000 7fb737e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb73810b1d0 0x7fb7381a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.081+0000 7fb73d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb738110790 0x7fb7381a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.081+0000 7fb737400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb73810fcd0 0x7fb7381a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.081+0000 7fb737e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb73810b1d0 0x7fb7381a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44092/0 (socket says 172.21.15.90:44092) 2024-05-27T07:52:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.081+0000 7fb737e00640 1 -- 172.21.15.90:0/1078761345 learned_addr learned my addr 172.21.15.90:0/1078761345 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.081+0000 7fb737400640 1 -- 172.21.15.90:0/1078761345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb738110790 msgr2=0x7fb7381a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.081+0000 7fb737400640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb738110790 0x7fb7381a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.081+0000 7fb737400640 1 -- 172.21.15.90:0/1078761345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb73810b1d0 msgr2=0x7fb7381a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.081+0000 7fb737400640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb73810b1d0 0x7fb7381a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.081+0000 7fb737400640 1 -- 172.21.15.90:0/1078761345 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7381a5b80 con 0x7fb73810fcd0 2024-05-27T07:52:54.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.081+0000 7fb737e00640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb73810b1d0 0x7fb7381a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:54.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.083+0000 7fb737400640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb73810fcd0 0x7fb7381a1280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb724009780 tx=0x7fb72402aef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.084+0000 7fb734c00640 1 -- 172.21.15.90:0/1078761345 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724037040 con 0x7fb73810fcd0 2024-05-27T07:52:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.084+0000 7fb734c00640 1 -- 172.21.15.90:0/1078761345 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb724030cf0 con 0x7fb73810fcd0 2024-05-27T07:52:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.084+0000 7fb734c00640 1 -- 172.21.15.90:0/1078761345 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7240035f0 con 0x7fb73810fcd0 2024-05-27T07:52:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.084+0000 7fb73f000640 1 -- 172.21.15.90:0/1078761345 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7381a5e10 con 0x7fb73810fcd0 2024-05-27T07:52:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.085+0000 7fb73f000640 1 -- 172.21.15.90:0/1078761345 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7381a6220 con 0x7fb73810fcd0 2024-05-27T07:52:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.090+0000 7fb746038740 1 Processor -- start 2024-05-27T07:52:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.091+0000 7fb746038740 1 -- start start 2024-05-27T07:52:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.091+0000 7fb734c00640 1 -- 172.21.15.90:0/1078761345 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb724043020 con 0x7fb73810fcd0 2024-05-27T07:52:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.091+0000 7fb746038740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dcf1b0 0x556c4a171860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.091+0000 7fb746038740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a171da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.091+0000 7fb746038740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a1722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.091+0000 7fb746038740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556c4a0e2b90 con 0x556c49da62f0 2024-05-27T07:52:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.091+0000 7fb746038740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556c4a0d2730 con 0x556c49dcf1b0 2024-05-27T07:52:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.091+0000 7fb746038740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556c4a0d6770 con 0x556c49dae4d0 2024-05-27T07:52:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.092+0000 7fb737400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a1722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.092+0000 7fb73d000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dcf1b0 0x556c4a171860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.092+0000 7fb737400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a1722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39728/0 (socket says 172.21.15.90:39728) 2024-05-27T07:52:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.092+0000 7fb737400640 1 -- 172.21.15.90:0/3499425100 learned_addr learned my addr 172.21.15.90:0/3499425100 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.092+0000 7fb737e00640 1 --2- 172.21.15.90:0/3499425100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a171da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.093+0000 7fb737400640 1 -- 172.21.15.90:0/3499425100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dcf1b0 msgr2=0x556c4a171860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.093+0000 7fb737400640 1 --2- 172.21.15.90:0/3499425100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dcf1b0 0x556c4a171860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.093+0000 7fb737400640 1 -- 172.21.15.90:0/3499425100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 msgr2=0x556c4a171da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.094+0000 7fb734c00640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb71c0b2950 0x7fb71c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.094+0000 7fb737400640 1 --2- 172.21.15.90:0/3499425100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a171da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.094+0000 7fb737400640 1 -- 172.21.15.90:0/3499425100 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4a06b8b0 con 0x556c49dae4d0 2024-05-27T07:52:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.094+0000 7fb737e00640 1 --2- 172.21.15.90:0/3499425100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a171da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.095+0000 7fb734c00640 1 -- 172.21.15.90:0/1078761345 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fb724101cf0 con 0x7fb73810fcd0 2024-05-27T07:52:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.095+0000 7fb73d000640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb71c0b2950 0x7fb71c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.096+0000 7fb737400640 1 --2- 172.21.15.90:0/3499425100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a1722e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb7240ff120 tx=0x7fb724049000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0428121fb518269 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.096+0000 7fb73d000640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb71c0b2950 0x7fb71c0b4e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x556c4a16e9d0 tx=0x7fb730009500 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.096+0000 7fb72d600640 1 -- 172.21.15.90:0/3499425100 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7240c8b40 con 0x556c49dae4d0 2024-05-27T07:52:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.097+0000 7fb72d600640 1 -- 172.21.15.90:0/3499425100 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7240c8880 con 0x556c49dae4d0 2024-05-27T07:52:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.098+0000 7fb746038740 1 -- 172.21.15.90:0/3499425100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 msgr2=0x556c4a1722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.098+0000 7fb746038740 1 --2- 172.21.15.90:0/3499425100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a1722e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb7240ff120 tx=0x7fb724049000 comp rx=0 tx=0).stop 2024-05-27T07:52:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.099+0000 7fb746038740 1 -- 172.21.15.90:0/3499425100 shutdown_connections 2024-05-27T07:52:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.099+0000 7fb746038740 1 --2- 172.21.15.90:0/3499425100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a1722e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.099+0000 7fb746038740 1 --2- 172.21.15.90:0/3499425100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a171da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.099+0000 7fb746038740 1 --2- 172.21.15.90:0/3499425100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dcf1b0 0x556c4a171860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.099+0000 7fb746038740 1 -- 172.21.15.90:0/3499425100 >> 172.21.15.90:0/3499425100 conn(0x556c49dc3300 msgr2=0x556c4a16e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.099+0000 7fb746038740 1 -- 172.21.15.90:0/3499425100 shutdown_connections 2024-05-27T07:52:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.100+0000 7fb746038740 1 -- 172.21.15.90:0/3499425100 wait complete. 2024-05-27T07:52:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.101+0000 7fb746038740 1 -- start start 2024-05-27T07:52:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.101+0000 7fb746038740 1 Processor -- start 2024-05-27T07:52:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.101+0000 7fb746038740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49da62f0 0x556c4a17ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.102+0000 7fb746038740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a17bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.102+0000 7fb746038740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a187a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.102+0000 7fb737e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49da62f0 0x556c4a17ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.102+0000 7fb737e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49da62f0 0x556c4a17ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:35398/0 (socket says 172.21.15.90:35398) 2024-05-27T07:52:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.103+0000 7fb737400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a17bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.103+0000 7fb737400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a17bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39742/0 (socket says 172.21.15.90:39742) 2024-05-27T07:52:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.103+0000 7fb73d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a187a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.103+0000 7fb73d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a187a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44122/0 (socket says 172.21.15.90:44122) 2024-05-27T07:52:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.103+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556c49db8790 con 0x556c49dcf1b0 2024-05-27T07:52:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.103+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556c4a07abc0 con 0x556c49da62f0 2024-05-27T07:52:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.103+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556c4a17e750 con 0x556c49dae4d0 2024-05-27T07:52:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.103+0000 7fb746038740 10 client.0 _ll_register_callbacks cb 0x556c4a058400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.103+0000 7fb737e00640 1 -- 172.21.15.90:0/2289852063 learned_addr learned my addr 172.21.15.90:0/2289852063 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.104+0000 7fb73d000640 1 -- 172.21.15.90:0/2289852063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 msgr2=0x556c4a17bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.104+0000 7fb73d000640 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a17bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.104+0000 7fb73d000640 1 -- 172.21.15.90:0/2289852063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49da62f0 msgr2=0x556c4a17ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.104+0000 7fb73d000640 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49da62f0 0x556c4a17ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.104+0000 7fb73d000640 1 -- 172.21.15.90:0/2289852063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c49de7210 con 0x556c49dcf1b0 2024-05-27T07:52:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.104+0000 7fb73d000640 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a187a00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb730005fd0 tx=0x7fb730004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabc7fe44ff266d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.104+0000 7fb722000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:52:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.105+0000 7fb722000640 1 -- 172.21.15.90:0/2289852063 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73000f870 con 0x556c49dcf1b0 2024-05-27T07:52:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.105+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7240c8880 con 0x556c49dcf1b0 2024-05-27T07:52:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.105+0000 7fb722000640 1 -- 172.21.15.90:0/2289852063 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fb73000fa10 con 0x556c49dcf1b0 2024-05-27T07:52:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.105+0000 7fb722000640 1 -- 172.21.15.90:0/2289852063 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb73000fc10 con 0x556c49dcf1b0 2024-05-27T07:52:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.106+0000 7fb746038740 10 client.16776 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.106+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556c49eb3620 con 0x556c49dcf1b0 2024-05-27T07:52:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.107+0000 7fb722000640 1 -- 172.21.15.90:0/2289852063 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7240c8880 con 0x556c49dcf1b0 2024-05-27T07:52:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.107+0000 7fb722000640 1 -- 172.21.15.90:0/2289852063 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fb730026070 con 0x556c49dcf1b0 2024-05-27T07:52:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.107+0000 7fb746038740 10 client.16776 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.107+0000 7fb746038740 10 client.16776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.107+0000 7fb746038740 20 client.16776 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.107+0000 7fb746038740 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556c49d8dd80 0x556c4a18dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.108+0000 7fb746038740 4 client.16776 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.108+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c4a0e4ae0 con 0x556c49d8dd80 2024-05-27T07:52:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.108+0000 7fb737e00640 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556c49d8dd80 0x556c4a18dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.110+0000 7fb737e00640 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556c49d8dd80 0x556c4a18dbc0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b43d54577be83cc server_cookie=dba57439285076ea in_seq=0 out_seq=0 2024-05-27T07:52:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.110+0000 7fb722000640 10 client.16776 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.112+0000 7fb722000640 1 -- 172.21.15.90:0/2289852063 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556c4a0e4ae0 con 0x556c49d8dd80 2024-05-27T07:52:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.112+0000 7fb722000640 10 client.16776 handle_command_reply: tid=0 2024-05-27T07:52:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.112+0000 7fb746038740 1 client.16776 shutdown 2024-05-27T07:52:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 msgr2=0x556c4a187a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb746038740 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a187a00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb730005fd0 tx=0x7fb730004ab0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 shutdown_connections 2024-05-27T07:52:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb746038740 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556c49d8dd80 0x556c4a18dbc0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb746038740 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a187a00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb746038740 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a17bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb746038740 1 --2- 172.21.15.90:0/2289852063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49da62f0 0x556c4a17ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb722000640 0 client.16776 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 >> 172.21.15.90:0/2289852063 conn(0x556c49dc3300 msgr2=0x556c4a173580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 shutdown_connections 2024-05-27T07:52:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.113+0000 7fb746038740 1 -- 172.21.15.90:0/2289852063 wait complete. 2024-05-27T07:52:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.114+0000 7fb746038740 20 client.16776 trim_cache size 0 max 16384 2024-05-27T07:52:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.142+0000 7fb746038740 1 Processor -- start 2024-05-27T07:52:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.142+0000 7fb746038740 1 -- start start 2024-05-27T07:52:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.143+0000 7fb746038740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a17e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.143+0000 7fb746038740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a16d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.143+0000 7fb746038740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dcf1b0 0x556c4a1877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.143+0000 7fb746038740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381c19b0 con 0x556c49da62f0 2024-05-27T07:52:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.143+0000 7fb746038740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381bf7a0 con 0x556c49dcf1b0 2024-05-27T07:52:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.143+0000 7fb746038740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381c12e0 con 0x556c49dae4d0 2024-05-27T07:52:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.144+0000 7fb737e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a17e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.144+0000 7fb737400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a16d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.145+0000 7fb737e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a17e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44138/0 (socket says 172.21.15.90:44138) 2024-05-27T07:52:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.145+0000 7fb737e00640 1 -- 172.21.15.90:0/3286547596 learned_addr learned my addr 172.21.15.90:0/3286547596 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.145+0000 7fb73d000640 1 --2- 172.21.15.90:0/3286547596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dcf1b0 0x556c4a1877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.145+0000 7fb737e00640 1 -- 172.21.15.90:0/3286547596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 msgr2=0x556c4a16d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.145+0000 7fb737e00640 1 --2- 172.21.15.90:0/3286547596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a16d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.145+0000 7fb737e00640 1 -- 172.21.15.90:0/3286547596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dcf1b0 msgr2=0x556c4a1877b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.145+0000 7fb737e00640 1 --2- 172.21.15.90:0/3286547596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dcf1b0 0x556c4a1877b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.145+0000 7fb737e00640 1 -- 172.21.15.90:0/3286547596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4a17e6f0 con 0x556c49da62f0 2024-05-27T07:52:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.145+0000 7fb737e00640 1 --2- 172.21.15.90:0/3286547596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a17e140 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb728005c80 tx=0x7fb728009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab7abc0f5fe3581 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.146+0000 7fb72cc00640 1 -- 172.21.15.90:0/3286547596 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728005220 con 0x556c49da62f0 2024-05-27T07:52:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.146+0000 7fb72cc00640 1 -- 172.21.15.90:0/3286547596 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb728004c30 con 0x556c49da62f0 2024-05-27T07:52:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.146+0000 7fb72cc00640 1 -- 172.21.15.90:0/3286547596 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728010910 con 0x556c49da62f0 2024-05-27T07:52:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.147+0000 7fb746038740 1 -- 172.21.15.90:0/3286547596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 msgr2=0x556c4a17e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.147+0000 7fb746038740 1 --2- 172.21.15.90:0/3286547596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a17e140 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb728005c80 tx=0x7fb728009260 comp rx=0 tx=0).stop 2024-05-27T07:52:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.147+0000 7fb746038740 1 -- 172.21.15.90:0/3286547596 shutdown_connections 2024-05-27T07:52:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.148+0000 7fb746038740 1 --2- 172.21.15.90:0/3286547596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dcf1b0 0x556c4a1877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.148+0000 7fb746038740 1 --2- 172.21.15.90:0/3286547596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49dae4d0 0x556c4a16d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.148+0000 7fb746038740 1 --2- 172.21.15.90:0/3286547596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49da62f0 0x556c4a17e140 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.148+0000 7fb746038740 1 -- 172.21.15.90:0/3286547596 >> 172.21.15.90:0/3286547596 conn(0x556c49dc3300 msgr2=0x556c4a172020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.148+0000 7fb746038740 1 -- 172.21.15.90:0/3286547596 shutdown_connections 2024-05-27T07:52:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.148+0000 7fb746038740 1 -- 172.21.15.90:0/3286547596 wait complete. 2024-05-27T07:52:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.149+0000 7fb746038740 1 -- start start 2024-05-27T07:52:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.149+0000 7fb746038740 1 Processor -- start 2024-05-27T07:52:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.150+0000 7fb746038740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dae4d0 0x556c4a175690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.150+0000 7fb746038740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a175bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.150+0000 7fb746038740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49d631c0 0x556c4a176110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.150+0000 7fb746038740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381c0540 con 0x556c49dcf1b0 2024-05-27T07:52:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.150+0000 7fb746038740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381bfe70 con 0x556c49dae4d0 2024-05-27T07:52:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.150+0000 7fb746038740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381c0c10 con 0x556c49d631c0 2024-05-27T07:52:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.150+0000 7fb746038740 10 client.0 _ll_register_callbacks cb 0x7fb730009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.150+0000 7fb737400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a175bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.150+0000 7fb737400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a175bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44148/0 (socket says 172.21.15.90:44148) 2024-05-27T07:52:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.150+0000 7fb737400640 1 -- 172.21.15.90:0/252604283 learned_addr learned my addr 172.21.15.90:0/252604283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.151+0000 7fb73d000640 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49d631c0 0x556c4a176110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.151+0000 7fb737e00640 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dae4d0 0x556c4a175690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.152+0000 7fb737e00640 1 -- 172.21.15.90:0/252604283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49d631c0 msgr2=0x556c4a176110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.152+0000 7fb737e00640 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49d631c0 0x556c4a176110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.152+0000 7fb737e00640 1 -- 172.21.15.90:0/252604283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 msgr2=0x556c4a175bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.152+0000 7fb737e00640 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a175bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.152+0000 7fb737e00640 1 -- 172.21.15.90:0/252604283 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4a1829a0 con 0x556c49dae4d0 2024-05-27T07:52:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.152+0000 7fb737400640 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a175bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:52:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.153+0000 7fb737e00640 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dae4d0 0x556c4a175690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x556c4a18a850 tx=0x7fb728004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb40dd0571dd1511 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.153+0000 7fb722000640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:52:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.154+0000 7fb722000640 1 -- 172.21.15.90:0/252604283 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728004a30 con 0x556c49dae4d0 2024-05-27T07:52:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.154+0000 7fb722000640 1 -- 172.21.15.90:0/252604283 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fb728004fd0 con 0x556c49dae4d0 2024-05-27T07:52:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.154+0000 7fb746038740 1 -- 172.21.15.90:0/252604283 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb728004c30 con 0x556c49dae4d0 2024-05-27T07:52:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.155+0000 7fb722000640 1 -- 172.21.15.90:0/252604283 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb7280051d0 con 0x556c49dae4d0 2024-05-27T07:52:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.155+0000 7fb746038740 10 client.36296 fetch_fsmap learned FSMap version 46 2024-05-27T07:52:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.155+0000 7fb746038740 1 -- 172.21.15.90:0/252604283 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x556c4a183c30 con 0x556c49dae4d0 2024-05-27T07:52:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.156+0000 7fb722000640 1 -- 172.21.15.90:0/252604283 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb728004c30 con 0x556c49dae4d0 2024-05-27T07:52:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.156+0000 7fb722000640 1 -- 172.21.15.90:0/252604283 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 46) ==== 7757+0+0 (secure 0 0 0) 0x7fb728027430 con 0x556c49dae4d0 2024-05-27T07:52:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.156+0000 7fb746038740 10 client.36296 fetch_fsmap finished waiting for FSMap version 46 2024-05-27T07:52:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.156+0000 7fb746038740 10 client.36296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.156+0000 7fb746038740 20 client.36296 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.156+0000 7fb746038740 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556c4a184230 0x556c4a1f8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.156+0000 7fb746038740 4 client.36296 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:52:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.156+0000 7fb746038740 1 -- 172.21.15.90:0/252604283 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c4a0de2c0 con 0x556c4a184230 2024-05-27T07:52:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.158+0000 7fb737400640 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556c4a184230 0x556c4a1f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.160+0000 7fb737400640 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556c4a184230 0x556c4a1f8260 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7d1b38eb1838234 server_cookie=1f196b3a7f384ff6 in_seq=0 out_seq=0 2024-05-27T07:52:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:54.160+0000 7fb722000640 10 client.36296 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:54.408 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:52:54.408 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:52:54.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:54 smithi090 ceph-mon[31429]: pgmap v202: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 807 op/s 2024-05-27T07:52:54.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:54 smithi136 ceph-mon[35324]: pgmap v202: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 807 op/s 2024-05-27T07:52:54.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:54 smithi129 ceph-mon[35254]: pgmap v202: 129 pgs: 129 active+clean; 1.9 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 13 MiB/s wr, 807 op/s 2024-05-27T07:52:55.079 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:52:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb722000640 1 -- 172.21.15.90:0/252604283 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556c4a0de2c0 con 0x556c4a184230 2024-05-27T07:52:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb722000640 10 client.36296 handle_command_reply: tid=0 2024-05-27T07:52:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb746038740 1 client.36296 shutdown 2024-05-27T07:52:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb746038740 1 -- 172.21.15.90:0/252604283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dae4d0 msgr2=0x556c4a175690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb746038740 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dae4d0 0x556c4a175690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x556c4a18a850 tx=0x7fb728004730 comp rx=0 tx=0).stop 2024-05-27T07:52:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb746038740 1 -- 172.21.15.90:0/252604283 shutdown_connections 2024-05-27T07:52:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb746038740 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556c4a184230 0x556c4a1f8260 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb746038740 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c49d631c0 0x556c4a176110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb746038740 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c49dcf1b0 0x556c4a175bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb746038740 1 --2- 172.21.15.90:0/252604283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c49dae4d0 0x556c4a175690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb746038740 1 -- 172.21.15.90:0/252604283 >> 172.21.15.90:0/252604283 conn(0x556c49dc3300 msgr2=0x556c4a171d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.554+0000 7fb722000640 0 client.36296 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.564+0000 7fb746038740 1 -- 172.21.15.90:0/252604283 shutdown_connections 2024-05-27T07:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.565+0000 7fb746038740 1 -- 172.21.15.90:0/252604283 wait complete. 2024-05-27T07:52:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.565+0000 7fb746038740 20 client.36296 trim_cache size 0 max 16384 2024-05-27T07:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.566+0000 7fb73f000640 1 -- 172.21.15.90:0/1078761345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb71c0b2950 msgr2=0x7fb71c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:55.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.566+0000 7fb73f000640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb71c0b2950 0x7fb71c0b4e60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x556c4a16e9d0 tx=0x7fb730009500 comp rx=0 tx=0).stop 2024-05-27T07:52:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.566+0000 7fb73f000640 1 -- 172.21.15.90:0/1078761345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb73810fcd0 msgr2=0x7fb7381a1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.566+0000 7fb73f000640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb73810fcd0 0x7fb7381a1280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb724009780 tx=0x7fb72402aef0 comp rx=0 tx=0).stop 2024-05-27T07:52:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.568+0000 7fb73f000640 1 -- 172.21.15.90:0/1078761345 shutdown_connections 2024-05-27T07:52:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.568+0000 7fb73f000640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb71c0b2950 0x7fb71c0b4e60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.568+0000 7fb73f000640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb73810fcd0 0x7fb7381a1280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.568+0000 7fb73f000640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb738110790 0x7fb7381a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.568+0000 7fb73f000640 1 --2- 172.21.15.90:0/1078761345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb73810b1d0 0x7fb7381a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.568+0000 7fb73f000640 1 -- 172.21.15.90:0/1078761345 >> 172.21.15.90:0/1078761345 conn(0x7fb7380795c0 msgr2=0x7fb73810bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.568+0000 7fb73f000640 1 -- 172.21.15.90:0/1078761345 shutdown_connections 2024-05-27T07:52:55.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:52:55.569+0000 7fb73f000640 1 -- 172.21.15.90:0/1078761345 wait complete. 2024-05-27T07:52:56.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:56 smithi090 ceph-mon[31429]: pgmap v203: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 746 op/s 2024-05-27T07:52:56.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:56 smithi136 ceph-mon[35324]: pgmap v203: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 746 op/s 2024-05-27T07:52:56.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:56 smithi129 ceph-mon[35254]: pgmap v203: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 746 op/s 2024-05-27T07:52:57.495 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:57 smithi090 ceph-mon[31429]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:52:57.495 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:57 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:52:57.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:57 smithi136 ceph-mon[35324]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:52:57.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:57 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:52:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:57 smithi129 ceph-mon[35254]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:52:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:57 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:52:57.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.903+0000 7fd3cf600700 1 -- 172.21.15.90:0/1235844217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3c8075af0 msgr2=0x7fd3c8075f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:57.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.903+0000 7fd3cf600700 1 --2- 172.21.15.90:0/1235844217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3c8075af0 0x7fd3c8075f00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009990 tx=0x7fd3b4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.904+0000 7fd3cf600700 1 -- 172.21.15.90:0/1235844217 shutdown_connections 2024-05-27T07:52:57.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.904+0000 7fd3cf600700 1 --2- 172.21.15.90:0/1235844217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3c81153e0 0x7fd3c8117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.904+0000 7fd3cf600700 1 --2- 172.21.15.90:0/1235844217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3c8075af0 0x7fd3c8075f00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.904+0000 7fd3cf600700 1 --2- 172.21.15.90:0/1235844217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3c8076440 0x7fd3c8114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.904+0000 7fd3cf600700 1 -- 172.21.15.90:0/1235844217 >> 172.21.15.90:0/1235844217 conn(0x7fd3c8100030 msgr2=0x7fd3c8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:57.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.904+0000 7fd3cf600700 1 -- 172.21.15.90:0/1235844217 shutdown_connections 2024-05-27T07:52:57.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.905+0000 7fd3cf600700 1 -- 172.21.15.90:0/1235844217 wait complete. 2024-05-27T07:52:57.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.906+0000 7fd3cf600700 1 Processor -- start 2024-05-27T07:52:57.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.907+0000 7fd3cf600700 1 -- start start 2024-05-27T07:52:57.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.907+0000 7fd3cf600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3c8075af0 0x7fd3c81a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.908+0000 7fd3cf600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3c8076440 0x7fd3c81a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.908+0000 7fd3cf600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3c81153e0 0x7fd3c81a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.908+0000 7fd3cf600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c811a9d0 con 0x7fd3c8075af0 2024-05-27T07:52:57.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.908+0000 7fd3cf600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c811a850 con 0x7fd3c8076440 2024-05-27T07:52:57.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.908+0000 7fd3cf600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c811ab50 con 0x7fd3c81153e0 2024-05-27T07:52:57.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.908+0000 7fd3cd600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3c81153e0 0x7fd3c81a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.908+0000 7fd3c7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3c8076440 0x7fd3c81a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.908+0000 7fd3cd600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3c81153e0 0x7fd3c81a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39786/0 (socket says 172.21.15.90:39786) 2024-05-27T07:52:57.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.908+0000 7fd3cd600700 1 -- 172.21.15.90:0/4275912577 learned_addr learned my addr 172.21.15.90:0/4275912577 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:57.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.909+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3c8075af0 0x7fd3c81a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.909+0000 7fd3ccc00700 1 -- 172.21.15.90:0/4275912577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3c81153e0 msgr2=0x7fd3c81a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:57.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.909+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3c81153e0 0x7fd3c81a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.909+0000 7fd3ccc00700 1 -- 172.21.15.90:0/4275912577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3c8076440 msgr2=0x7fd3c81a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:57.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.909+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3c8076440 0x7fd3c81a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.909+0000 7fd3ccc00700 1 -- 172.21.15.90:0/4275912577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c81a6270 con 0x7fd3c8075af0 2024-05-27T07:52:57.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.910+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3c8075af0 0x7fd3c81a13a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40098e0 tx=0x7fd3b40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:57.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.911+0000 7fd3c5600700 1 -- 172.21.15.90:0/4275912577 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400fb10 con 0x7fd3c8075af0 2024-05-27T07:52:57.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.911+0000 7fd3c5600700 1 -- 172.21.15.90:0/4275912577 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3b4004cc0 con 0x7fd3c8075af0 2024-05-27T07:52:57.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.911+0000 7fd3c5600700 1 -- 172.21.15.90:0/4275912577 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40053c0 con 0x7fd3c8075af0 2024-05-27T07:52:57.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.911+0000 7fd3cf600700 1 -- 172.21.15.90:0/4275912577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c81a6500 con 0x7fd3c8075af0 2024-05-27T07:52:57.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.912+0000 7fd3cf600700 1 -- 172.21.15.90:0/4275912577 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c81a6930 con 0x7fd3c8075af0 2024-05-27T07:52:57.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.919+0000 7fd3c5600700 1 -- 172.21.15.90:0/4275912577 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd3b4009d10 con 0x7fd3c8075af0 2024-05-27T07:52:57.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.920+0000 7fd3de1f0b80 1 Processor -- start 2024-05-27T07:52:57.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.920+0000 7fd3de1f0b80 1 -- start start 2024-05-27T07:52:57.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.921+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157f9d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.921+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 0x560157f978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.922+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157f97e30 0x560157f98240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.922+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560157ec5ec0 con 0x560157f97e30 2024-05-27T07:52:57.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.922+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560157eb5a60 con 0x560157f9ddb0 2024-05-27T07:52:57.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.922+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560157eb9aa0 con 0x560157e18ec0 2024-05-27T07:52:57.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.922+0000 7fd3cd600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157f9d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.922+0000 7fd3cd600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157f9d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39790/0 (socket says 172.21.15.90:39790) 2024-05-27T07:52:57.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.922+0000 7fd3cd600700 1 -- 172.21.15.90:0/703005015 learned_addr learned my addr 172.21.15.90:0/703005015 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:57.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.922+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/703005015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 0x560157f978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.922+0000 7fd3c7e00700 1 --2- 172.21.15.90:0/703005015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157f97e30 0x560157f98240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.923+0000 7fd3ccc00700 1 -- 172.21.15.90:0/703005015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 msgr2=0x560157f9d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.923+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/703005015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157f9d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.923+0000 7fd3ccc00700 1 -- 172.21.15.90:0/703005015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157f97e30 msgr2=0x560157f98240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.923+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/703005015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157f97e30 0x560157f98240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.923+0000 7fd3ccc00700 1 -- 172.21.15.90:0/703005015 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560157e7e4a0 con 0x560157f9ddb0 2024-05-27T07:52:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.924+0000 7fd3c5600700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3ac0b2bc0 0x7fd3ac0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.924+0000 7fd3c5600700 1 -- 172.21.15.90:0/4275912577 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fd3b40e39b0 con 0x7fd3c8075af0 2024-05-27T07:52:57.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.924+0000 7fd3cd600700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3ac0b2bc0 0x7fd3ac0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.925+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/703005015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 0x560157f978f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40e0d50 tx=0x7fd3b40e9ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52656018b17babb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:57.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.925+0000 7fd3be000700 1 -- 172.21.15.90:0/703005015 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40a6830 con 0x560157f9ddb0 2024-05-27T07:52:57.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.925+0000 7fd3be000700 1 -- 172.21.15.90:0/703005015 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3b40a60f0 con 0x560157f9ddb0 2024-05-27T07:52:57.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.925+0000 7fd3be000700 1 -- 172.21.15.90:0/703005015 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40aa0e0 con 0x560157f9ddb0 2024-05-27T07:52:57.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.927+0000 7fd3cd600700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3ac0b2bc0 0x7fd3ac0b50d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x560157f98bb0 tx=0x7fd3b8009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:57.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.931+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/703005015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 msgr2=0x560157f978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:57.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.931+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/703005015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 0x560157f978f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40e0d50 tx=0x7fd3b40e9ed0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.932+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/703005015 shutdown_connections 2024-05-27T07:52:57.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.932+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/703005015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157f97e30 0x560157f98240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.932+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/703005015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157f9d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.932+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/703005015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 0x560157f978f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.932+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/703005015 >> 172.21.15.90:0/703005015 conn(0x560157e79040 msgr2=0x560157f92950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:57.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.932+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/703005015 shutdown_connections 2024-05-27T07:52:57.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.932+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/703005015 wait complete. 2024-05-27T07:52:57.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.933+0000 7fd3de1f0b80 1 -- start start 2024-05-27T07:52:57.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.933+0000 7fd3de1f0b80 1 Processor -- start 2024-05-27T07:52:57.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.934+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157fa16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.934+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 0x560157fa1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.934+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157f9ddb0 0x560157fa2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.934+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560157dbc020 con 0x560157f9ddb0 2024-05-27T07:52:57.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.934+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560157e567e0 con 0x560157f97e30 2024-05-27T07:52:57.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.934+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560157fa4640 con 0x560157e18ec0 2024-05-27T07:52:57.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.935+0000 7fd3de1f0b80 10 client.0 _ll_register_callbacks cb 0x560157e5fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:57.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.935+0000 7fd3ccc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157fa16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.936+0000 7fd3ccc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157fa16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39794/0 (socket says 172.21.15.90:39794) 2024-05-27T07:52:57.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.936+0000 7fd3ccc00700 1 -- 172.21.15.90:0/87896460 learned_addr learned my addr 172.21.15.90:0/87896460 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:57.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.936+0000 7fd3cd600700 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157f9ddb0 0x560157fa2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.936+0000 7fd3c7e00700 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 0x560157fa1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.937+0000 7fd3ccc00700 1 -- 172.21.15.90:0/87896460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 msgr2=0x560157fa1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:57.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.937+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 0x560157fa1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.937+0000 7fd3ccc00700 1 -- 172.21.15.90:0/87896460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157f9ddb0 msgr2=0x560157fa2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:57.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.937+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157f9ddb0 0x560157fa2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.937+0000 7fd3ccc00700 1 -- 172.21.15.90:0/87896460 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560157e05950 con 0x560157e18ec0 2024-05-27T07:52:57.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.938+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157fa16a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40aa670 tx=0x7fd3b4005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebb33c4859ac4a5d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:57.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.938+0000 7fd3b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:52:57.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.938+0000 7fd3b2a00700 1 -- 172.21.15.90:0/87896460 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40aa2c0 con 0x560157e18ec0 2024-05-27T07:52:57.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.939+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/87896460 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3b40a60f0 con 0x560157e18ec0 2024-05-27T07:52:57.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.939+0000 7fd3b2a00700 1 -- 172.21.15.90:0/87896460 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fd3b4005560 con 0x560157e18ec0 2024-05-27T07:52:57.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.940+0000 7fd3b2a00700 1 -- 172.21.15.90:0/87896460 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd3b40aa740 con 0x560157e18ec0 2024-05-27T07:52:57.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.940+0000 7fd3de1f0b80 10 client.26356 fetch_fsmap learned FSMap version 47 2024-05-27T07:52:57.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.941+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/87896460 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560157e860e0 con 0x560157e18ec0 2024-05-27T07:52:57.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.941+0000 7fd3b2a00700 1 -- 172.21.15.90:0/87896460 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3b40a60f0 con 0x560157e18ec0 2024-05-27T07:52:57.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.942+0000 7fd3b2a00700 1 -- 172.21.15.90:0/87896460 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 47) ==== 7152+0+0 (secure 0 0 0) 0x7fd3b40aabd0 con 0x560157e18ec0 2024-05-27T07:52:57.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.942+0000 7fd3de1f0b80 10 client.26356 fetch_fsmap finished waiting for FSMap version 47 2024-05-27T07:52:57.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.942+0000 7fd3de1f0b80 10 client.26356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:57.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.942+0000 7fd3de1f0b80 20 client.26356 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:57.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.942+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560157fb1220 0x560157fb3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.942+0000 7fd3de1f0b80 4 client.26356 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:52:57.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.942+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/87896460 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560157ec7e10 con 0x560157fb1220 2024-05-27T07:52:57.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.943+0000 7fd3c7e00700 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560157fb1220 0x560157fb3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.944+0000 7fd3c7e00700 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560157fb1220 0x560157fb3640 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d38f0fd25c76b3d8 server_cookie=8f4439c0d348d34a in_seq=0 out_seq=0 2024-05-27T07:52:57.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.945+0000 7fd3b2a00700 10 client.26356 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:57.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.946+0000 7fd3b2a00700 1 -- 172.21.15.90:0/87896460 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560157ec7e10 con 0x560157fb1220 2024-05-27T07:52:57.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.947+0000 7fd3b2a00700 10 client.26356 handle_command_reply: tid=0 2024-05-27T07:52:57.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.947+0000 7fd3de1f0b80 1 client.26356 shutdown 2024-05-27T07:52:57.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.948+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/87896460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 msgr2=0x560157fa16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:57.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.948+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157fa16a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40aa670 tx=0x7fd3b4005f10 comp rx=0 tx=0).stop 2024-05-27T07:52:57.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.949+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/87896460 shutdown_connections 2024-05-27T07:52:57.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.949+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560157fb1220 0x560157fb3640 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.949+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157f9ddb0 0x560157fa2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.949+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157e18ec0 0x560157fa16a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.949+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/87896460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 0x560157fa1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.949+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/87896460 >> 172.21.15.90:0/87896460 conn(0x560157e79040 msgr2=0x560157f92600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:57.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.949+0000 7fd3b2a00700 0 client.26356 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:57.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.950+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/87896460 shutdown_connections 2024-05-27T07:52:57.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.950+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/87896460 wait complete. 2024-05-27T07:52:57.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.950+0000 7fd3de1f0b80 20 client.26356 trim_cache size 0 max 16384 2024-05-27T07:52:57.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.989+0000 7fd3de1f0b80 1 Processor -- start 2024-05-27T07:52:57.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.989+0000 7fd3de1f0b80 1 -- start start 2024-05-27T07:52:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.989+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157f978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.989+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f97e30 0x560157fad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.989+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 0x560157fa1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.989+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c81b55f0 con 0x560157e18ec0 2024-05-27T07:52:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.989+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c81b7ed0 con 0x560157f9ddb0 2024-05-27T07:52:57.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.990+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c81b5cc0 con 0x560157f97e30 2024-05-27T07:52:57.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.990+0000 7fd3ccc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157f978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.990+0000 7fd3ccc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157f978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44194/0 (socket says 172.21.15.90:44194) 2024-05-27T07:52:57.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.990+0000 7fd3ccc00700 1 -- 172.21.15.90:0/2625293461 learned_addr learned my addr 172.21.15.90:0/2625293461 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:57.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.990+0000 7fd3c7e00700 1 --2- 172.21.15.90:0/2625293461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f97e30 0x560157fad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.990+0000 7fd3cd600700 1 --2- 172.21.15.90:0/2625293461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 0x560157fa1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.991+0000 7fd3ccc00700 1 -- 172.21.15.90:0/2625293461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f97e30 msgr2=0x560157fad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.991+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/2625293461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f97e30 0x560157fad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.991+0000 7fd3ccc00700 1 -- 172.21.15.90:0/2625293461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 msgr2=0x560157fa1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.991+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/2625293461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 0x560157fa1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.991+0000 7fd3ccc00700 1 -- 172.21.15.90:0/2625293461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560157f8c1b0 con 0x560157e18ec0 2024-05-27T07:52:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.992+0000 7fd3c7e00700 1 --2- 172.21.15.90:0/2625293461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f97e30 0x560157fad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.992+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/2625293461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157f978f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40aa900 tx=0x7fd3b40e9ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad15d6283501bc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.993+0000 7fd3bd600700 1 -- 172.21.15.90:0/2625293461 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40e11e0 con 0x560157e18ec0 2024-05-27T07:52:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.993+0000 7fd3bd600700 1 -- 172.21.15.90:0/2625293461 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3b40a60f0 con 0x560157e18ec0 2024-05-27T07:52:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.993+0000 7fd3bd600700 1 -- 172.21.15.90:0/2625293461 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40ab070 con 0x560157e18ec0 2024-05-27T07:52:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.994+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2625293461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 msgr2=0x560157f978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.994+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/2625293461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157f978f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40aa900 tx=0x7fd3b40e9ed0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.995+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2625293461 shutdown_connections 2024-05-27T07:52:57.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.995+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/2625293461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f9ddb0 0x560157fa1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.995+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/2625293461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157f978f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.995+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/2625293461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f97e30 0x560157fad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:57.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.995+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2625293461 >> 172.21.15.90:0/2625293461 conn(0x560157e79040 msgr2=0x560157f9bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:57.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.995+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2625293461 shutdown_connections 2024-05-27T07:52:57.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.995+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2625293461 wait complete. 2024-05-27T07:52:57.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.996+0000 7fd3de1f0b80 1 -- start start 2024-05-27T07:52:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.996+0000 7fd3de1f0b80 1 Processor -- start 2024-05-27T07:52:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.998+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157fb41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.998+0000 7fd3ccc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157fb41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:57.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.998+0000 7fd3ccc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157fb41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44196/0 (socket says 172.21.15.90:44196) 2024-05-27T07:52:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.998+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 0x560157fb4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.998+0000 7fd3de1f0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f9ddb0 0x560157fb4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.999+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c81b6390 con 0x560157e18ec0 2024-05-27T07:52:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.999+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c81b7800 con 0x560157f97e30 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.999+0000 7fd3de1f0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c81b6a60 con 0x560157f9ddb0 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.999+0000 7fd3de1f0b80 10 client.0 _ll_register_callbacks cb 0x560157e5fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.999+0000 7fd3ccc00700 1 -- 172.21.15.90:0/2456206659 learned_addr learned my addr 172.21.15.90:0/2456206659 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.999+0000 7fd3c7e00700 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 0x560157fb4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:57.999+0000 7fd3cd600700 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f9ddb0 0x560157fb4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.001+0000 7fd3ccc00700 1 -- 172.21.15.90:0/2456206659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f9ddb0 msgr2=0x560157fb4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.001+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f9ddb0 0x560157fb4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.001+0000 7fd3ccc00700 1 -- 172.21.15.90:0/2456206659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 msgr2=0x560157fb4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.001+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 0x560157fb4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.001+0000 7fd3ccc00700 1 -- 172.21.15.90:0/2456206659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56015801ac50 con 0x560157e18ec0 2024-05-27T07:52:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.001+0000 7fd3c7e00700 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 0x560157fb4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:52:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.002+0000 7fd3ccc00700 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157fb41e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40e9b40 tx=0x7fd3b40e9fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50da7db7ad7e9713 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:52:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.002+0000 7fd3b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:52:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.003+0000 7fd3b2a00700 1 -- 172.21.15.90:0/2456206659 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40e11e0 con 0x560157e18ec0 2024-05-27T07:52:58.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.003+0000 7fd3b2a00700 1 -- 172.21.15.90:0/2456206659 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9408+0+0 (secure 0 0 0) 0x7fd3b40a6f60 con 0x560157e18ec0 2024-05-27T07:52:58.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.003+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2456206659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3b40a60f0 con 0x560157e18ec0 2024-05-27T07:52:58.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.004+0000 7fd3b2a00700 1 -- 172.21.15.90:0/2456206659 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd3b40aade0 con 0x560157e18ec0 2024-05-27T07:52:58.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.004+0000 7fd3de1f0b80 10 client.16800 fetch_fsmap learned FSMap version 47 2024-05-27T07:52:58.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.004+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2456206659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56015801ac50 con 0x560157e18ec0 2024-05-27T07:52:58.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.005+0000 7fd3b2a00700 1 -- 172.21.15.90:0/2456206659 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3b40a60f0 con 0x560157e18ec0 2024-05-27T07:52:58.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.005+0000 7fd3b2a00700 1 -- 172.21.15.90:0/2456206659 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 47) ==== 7152+0+0 (secure 0 0 0) 0x7fd3b40b5cc0 con 0x560157e18ec0 2024-05-27T07:52:58.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.005+0000 7fd3de1f0b80 10 client.16800 fetch_fsmap finished waiting for FSMap version 47 2024-05-27T07:52:58.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.005+0000 7fd3de1f0b80 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:52:58.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.005+0000 7fd3de1f0b80 20 client.16800 populate_metadata read hostname 'smithi090' 2024-05-27T07:52:58.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.005+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56015801c160 0x56015801e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:52:58.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.005+0000 7fd3de1f0b80 4 client.16800 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:52:58.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.005+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2456206659 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560157ec15f0 con 0x56015801c160 2024-05-27T07:52:58.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.006+0000 7fd3c7e00700 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56015801c160 0x56015801e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:52:58.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.008+0000 7fd3c7e00700 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56015801c160 0x56015801e580 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d097c04bf626d5 server_cookie=52e1317ab16b5885 in_seq=0 out_seq=0 2024-05-27T07:52:58.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.008+0000 7fd3b2a00700 10 client.16800 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:58.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.012+0000 7fd3b2a00700 1 -- 172.21.15.90:0/2456206659 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560157ec15f0 con 0x56015801c160 2024-05-27T07:52:58.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.012+0000 7fd3b2a00700 10 client.16800 handle_command_reply: tid=0 2024-05-27T07:52:58.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.012+0000 7fd3de1f0b80 1 client.16800 shutdown 2024-05-27T07:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.012+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2456206659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 msgr2=0x560157fb41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.012+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157fb41e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40e9b40 tx=0x7fd3b40e9fa0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.013+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2456206659 shutdown_connections 2024-05-27T07:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.013+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56015801c160 0x56015801e580 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.013+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560157f9ddb0 0x560157fb4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.013+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560157e18ec0 0x560157fb41e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.013+0000 7fd3de1f0b80 1 --2- 172.21.15.90:0/2456206659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560157f97e30 0x560157fb4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.013+0000 7fd3b2a00700 0 client.16800 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.013+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2456206659 >> 172.21.15.90:0/2456206659 conn(0x560157e79040 msgr2=0x560157f9bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.013+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2456206659 shutdown_connections 2024-05-27T07:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.013+0000 7fd3de1f0b80 1 -- 172.21.15.90:0/2456206659 wait complete. 2024-05-27T07:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.013+0000 7fd3de1f0b80 20 client.16800 trim_cache size 0 max 16384 2024-05-27T07:52:58.015 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:52:58.015 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:52:58.016 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "90280061-ff5f-4c59-aaf9-7fae09ca41fc", 2024-05-27T07:52:58.016 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:52:58.016 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.015+0000 7fd3cf600700 1 -- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3ac0b2bc0 msgr2=0x7fd3ac0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.015+0000 7fd3cf600700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3ac0b2bc0 0x7fd3ac0b50d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x560157f98bb0 tx=0x7fd3b8009260 comp rx=0 tx=0).stop 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.015+0000 7fd3cf600700 1 -- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3c8075af0 msgr2=0x7fd3c81a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.015+0000 7fd3cf600700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3c8075af0 0x7fd3c81a13a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40098e0 tx=0x7fd3b40047b0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.015+0000 7fd3cf600700 1 -- 172.21.15.90:0/4275912577 shutdown_connections 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.015+0000 7fd3cf600700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3c81153e0 0x7fd3c81a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.015+0000 7fd3cf600700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3c8075af0 0x7fd3c81a13a0 secure :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40098e0 tx=0x7fd3b40047b0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.015+0000 7fd3cf600700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3ac0b2bc0 0x7fd3ac0b50d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.015+0000 7fd3cf600700 1 --2- 172.21.15.90:0/4275912577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3c8076440 0x7fd3c81a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.015+0000 7fd3cf600700 1 -- 172.21.15.90:0/4275912577 >> 172.21.15.90:0/4275912577 conn(0x7fd3c8100030 msgr2=0x7fd3c8100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.016+0000 7fd3cf600700 1 -- 172.21.15.90:0/4275912577 shutdown_connections 2024-05-27T07:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:52:58.016+0000 7fd3cf600700 1 -- 172.21.15.90:0/4275912577 wait complete. 2024-05-27T07:52:58.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:58 smithi136 ceph-mon[35324]: pgmap v204: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 658 op/s 2024-05-27T07:52:58.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:58 smithi129 ceph-mon[35254]: pgmap v204: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 658 op/s 2024-05-27T07:52:58.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:58 smithi090 ceph-mon[31429]: pgmap v204: 129 pgs: 129 active+clean; 2.0 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 658 op/s 2024-05-27T07:52:58.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 160.0 s, 578.8 tps, lat 161.003 ms stddev 245.921 2024-05-27T07:52:59.213 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:58 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 3. 2024-05-27T07:52:59.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:58 smithi090 systemd[1]: Stopped Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:52:59.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:58 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.745s CPU time. 2024-05-27T07:52:59.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:58 smithi090 systemd[1]: Starting Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:52:59.486 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:52:59.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:59 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:52:59.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:59 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:59.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:59 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:52:59.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:59 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:59.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:52:59 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:52:59.555 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:59 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:52:59.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:59 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:59.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:59 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:52:59.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:59 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:59.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:52:59 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:52:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:59 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:52:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:59 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:59 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:52:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:59 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:52:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:52:59 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:53:00.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:52:59 smithi090 podman[108300]: 2024-05-27 07:52:59.824373342 +0000 UTC m=+0.060608027 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:53:00.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:00 smithi136 ceph-mon[35324]: pgmap v205: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 863 op/s 2024-05-27T07:53:00.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:00 smithi129 ceph-mon[35254]: pgmap v205: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 863 op/s 2024-05-27T07:53:00.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:00 smithi090 ceph-mon[31429]: pgmap v205: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 18 MiB/s wr, 863 op/s 2024-05-27T07:53:02.520 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:02 smithi090 ceph-mon[31429]: pgmap v206: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 552 op/s 2024-05-27T07:53:02.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:02 smithi129 ceph-mon[35254]: pgmap v206: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 552 op/s 2024-05-27T07:53:02.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:02 smithi136 ceph-mon[35324]: pgmap v206: 129 pgs: 129 active+clean; 2.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 13 MiB/s wr, 552 op/s 2024-05-27T07:53:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:03 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:53:03.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:03 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:53:03.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:03 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:53:03.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 165.0 s, 429.2 tps, lat 240.681 ms stddev 470.349 2024-05-27T07:53:04.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:04 smithi129 ceph-mon[35254]: pgmap v207: 129 pgs: 129 active+clean; 2.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 17 MiB/s wr, 824 op/s 2024-05-27T07:53:04.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:04 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:04 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:04 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:04 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:04 smithi090 ceph-mon[31429]: pgmap v207: 129 pgs: 129 active+clean; 2.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 17 MiB/s wr, 824 op/s 2024-05-27T07:53:04.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:04 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:04 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:04 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:04 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:04 smithi136 ceph-mon[35324]: pgmap v207: 129 pgs: 129 active+clean; 2.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 17 MiB/s wr, 824 op/s 2024-05-27T07:53:04.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:04 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:04 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:04 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:04.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:04 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:05.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:05 smithi090 ceph-mon[31429]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T07:53:05.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:05 smithi090 ceph-mon[31429]: pgmap v209: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-27T07:53:05.999 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:05 smithi136 ceph-mon[35324]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T07:53:06.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:05 smithi136 ceph-mon[35324]: pgmap v209: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-27T07:53:06.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:05 smithi129 ceph-mon[35254]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T07:53:06.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:05 smithi129 ceph-mon[35254]: pgmap v209: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-27T07:53:07.051 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:53:06 smithi090 podman[108300]: 2024-05-27 07:53:06.76189797 +0000 UTC m=+6.998132642 container create ac2afcf63a575480ab8be281e86b1bacfd8acaf9aa683a95ce211a444c554629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-27T07:53:07.311 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:53:07 smithi090 podman[108300]: 2024-05-27 07:53:07.049375779 +0000 UTC m=+7.285610466 container init ac2afcf63a575480ab8be281e86b1bacfd8acaf9aa683a95ce211a444c554629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T07:53:07.311 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:53:07 smithi090 podman[108300]: 2024-05-27 07:53:07.064318865 +0000 UTC m=+7.300553540 container start ac2afcf63a575480ab8be281e86b1bacfd8acaf9aa683a95ce211a444c554629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:53:07.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:53:07 smithi090 bash[108300]: ac2afcf63a575480ab8be281e86b1bacfd8acaf9aa683a95ce211a444c554629 2024-05-27T07:53:07.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:53:07 smithi090 systemd[1]: Started Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:53:07.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:53:07 smithi090 ceph-mds[108434]: -- 172.21.15.90:0/2399418224 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55620353ba40 con 0x556203583600 2024-05-27T07:53:07.356 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:07.567 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:53:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: starting mds.d at 2024-05-27T07:53:08.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:08 smithi090 ceph-mon[31429]: pgmap v210: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-27T07:53:08.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:08 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:53:08.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:08 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:53:08.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:08 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:08 smithi136 ceph-mon[35324]: pgmap v210: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-27T07:53:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:08 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:53:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:08 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:53:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:08 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:08.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:08 smithi129 ceph-mon[35254]: pgmap v210: 129 pgs: 129 active+clean; 2.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 14 MiB/s wr, 711 op/s 2024-05-27T07:53:08.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:08 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:53:08.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:08 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:53:08.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:08 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:53:08.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 170.0 s, 525.0 tps, lat 196.586 ms stddev 395.675 2024-05-27T07:53:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.924+0000 7f4320800640 1 -- 172.21.15.90:0/1104812354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4318075660 msgr2=0x7f4318075a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.924+0000 7f4320800640 1 --2- 172.21.15.90:0/1104812354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4318075660 0x7f4318075a70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f430c009900 tx=0x7f430c02a4a0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.924+0000 7f4320800640 1 -- 172.21.15.90:0/1104812354 shutdown_connections 2024-05-27T07:53:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.924+0000 7f4320800640 1 --2- 172.21.15.90:0/1104812354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43181104a0 0x7f4318112900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.924+0000 7f4320800640 1 --2- 172.21.15.90:0/1104812354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f431810db40 0x7f431810ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.924+0000 7f4320800640 1 --2- 172.21.15.90:0/1104812354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4318075660 0x7f4318075a70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.924+0000 7f4320800640 1 -- 172.21.15.90:0/1104812354 >> 172.21.15.90:0/1104812354 conn(0x7f43180ff480 msgr2=0x7f43181018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.925+0000 7f4320800640 1 -- 172.21.15.90:0/1104812354 shutdown_connections 2024-05-27T07:53:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.925+0000 7f4320800640 1 -- 172.21.15.90:0/1104812354 wait complete. 2024-05-27T07:53:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.927+0000 7f4320800640 1 Processor -- start 2024-05-27T07:53:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.927+0000 7f4320800640 1 -- start start 2024-05-27T07:53:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.927+0000 7f4320800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4318075660 0x7f431810aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.927+0000 7f4320800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f431810db40 0x7f431810b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.927+0000 7f4320800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43181104a0 0x7f431810b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.927+0000 7f4320800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4318115f40 con 0x7f431810db40 2024-05-27T07:53:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.927+0000 7f4320800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4318115dc0 con 0x7f4318075660 2024-05-27T07:53:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.927+0000 7f4320800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43181160c0 con 0x7f43181104a0 2024-05-27T07:53:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.928+0000 7f431de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4318075660 0x7f431810aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.928+0000 7f431d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f431810db40 0x7f431810b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.928+0000 7f431d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f431810db40 0x7f431810b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52494/0 (socket says 172.21.15.90:52494) 2024-05-27T07:53:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.928+0000 7f431d400640 1 -- 172.21.15.90:0/853763398 learned_addr learned my addr 172.21.15.90:0/853763398 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.929+0000 7f431e800640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43181104a0 0x7f431810b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.929+0000 7f431d400640 1 -- 172.21.15.90:0/853763398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43181104a0 msgr2=0x7f431810b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.929+0000 7f431d400640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43181104a0 0x7f431810b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.929+0000 7f431d400640 1 -- 172.21.15.90:0/853763398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4318075660 msgr2=0x7f431810aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.929+0000 7f431d400640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4318075660 0x7f431810aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.929+0000 7f431d400640 1 -- 172.21.15.90:0/853763398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43181acec0 con 0x7f431810db40 2024-05-27T07:53:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.929+0000 7f431de00640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4318075660 0x7f431810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.930+0000 7f431d400640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f431810db40 0x7f431810b030 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f430800ea60 tx=0x7f430800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.930+0000 7f4312a00640 1 -- 172.21.15.90:0/853763398 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430800cde0 con 0x7f431810db40 2024-05-27T07:53:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.930+0000 7f4312a00640 1 -- 172.21.15.90:0/853763398 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f43080035f0 con 0x7f431810db40 2024-05-27T07:53:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.930+0000 7f4312a00640 1 -- 172.21.15.90:0/853763398 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308010690 con 0x7f431810db40 2024-05-27T07:53:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.930+0000 7f4320800640 1 -- 172.21.15.90:0/853763398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43181ad1b0 con 0x7f431810db40 2024-05-27T07:53:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.931+0000 7f4320800640 1 -- 172.21.15.90:0/853763398 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43181ad6b0 con 0x7f431810db40 2024-05-27T07:53:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.935+0000 7f4327782740 1 Processor -- start 2024-05-27T07:53:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.937+0000 7f4327782740 1 -- start start 2024-05-27T07:53:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.938+0000 7f4327782740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d2b9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.938+0000 7f4327782740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.938+0000 7f4327782740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.938+0000 7f4327782740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560e6d22ab90 con 0x560e6cef64d0 2024-05-27T07:53:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.938+0000 7f4327782740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560e6d21a730 con 0x560e6cf171b0 2024-05-27T07:53:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.938+0000 7f4327782740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560e6d21e770 con 0x560e6ceee2f0 2024-05-27T07:53:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.938+0000 7f4312a00640 1 -- 172.21.15.90:0/853763398 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4308004000 con 0x7f431810db40 2024-05-27T07:53:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.938+0000 7f431de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.939+0000 7f431de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56182/0 (socket says 172.21.15.90:56182) 2024-05-27T07:53:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.939+0000 7f431de00640 1 -- 172.21.15.90:0/311963720 learned_addr learned my addr 172.21.15.90:0/311963720 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.939+0000 7f431e800640 1 --2- 172.21.15.90:0/311963720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d2b9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.939+0000 7f431d400640 1 --2- 172.21.15.90:0/311963720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.939+0000 7f431de00640 1 -- 172.21.15.90:0/311963720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 msgr2=0x560e6d2b9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.939+0000 7f431de00640 1 --2- 172.21.15.90:0/311963720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d2b9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.939+0000 7f431de00640 1 -- 172.21.15.90:0/311963720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 msgr2=0x560e6d2ba2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.939+0000 7f431de00640 1 --2- 172.21.15.90:0/311963720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2ba2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.939+0000 7f431de00640 1 -- 172.21.15.90:0/311963720 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6d1b38b0 con 0x560e6ceee2f0 2024-05-27T07:53:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.939+0000 7f431d400640 1 --2- 172.21.15.90:0/311963720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2ba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.940+0000 7f431de00640 1 --2- 172.21.15.90:0/311963720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b9da0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f430c0056f0 tx=0x7f430c004150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eb2c6213b117fdc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.940+0000 7f4312a00640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f43000b29d0 0x7f43000b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.940+0000 7f4312a00640 1 -- 172.21.15.90:0/853763398 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f43080e2e00 con 0x7f431810db40 2024-05-27T07:53:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.940+0000 7f431e800640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f43000b29d0 0x7f43000b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.940+0000 7f4307000640 1 -- 172.21.15.90:0/311963720 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c03f030 con 0x560e6ceee2f0 2024-05-27T07:53:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.941+0000 7f4307000640 1 -- 172.21.15.90:0/311963720 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f430c004370 con 0x560e6ceee2f0 2024-05-27T07:53:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.941+0000 7f4307000640 1 -- 172.21.15.90:0/311963720 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c035070 con 0x560e6ceee2f0 2024-05-27T07:53:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.942+0000 7f4327782740 1 -- 172.21.15.90:0/311963720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 msgr2=0x560e6d2b9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.942+0000 7f431e800640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f43000b29d0 0x7f43000b4ee0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x560e6d2b69d0 tx=0x7f4314009500 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.942+0000 7f4327782740 1 --2- 172.21.15.90:0/311963720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b9da0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f430c0056f0 tx=0x7f430c004150 comp rx=0 tx=0).stop 2024-05-27T07:53:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.942+0000 7f4327782740 1 -- 172.21.15.90:0/311963720 shutdown_connections 2024-05-27T07:53:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.942+0000 7f4327782740 1 --2- 172.21.15.90:0/311963720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2ba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.942+0000 7f4327782740 1 --2- 172.21.15.90:0/311963720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b9da0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.942+0000 7f4327782740 1 --2- 172.21.15.90:0/311963720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d2b9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.942+0000 7f4327782740 1 -- 172.21.15.90:0/311963720 >> 172.21.15.90:0/311963720 conn(0x560e6cf0b300 msgr2=0x560e6d2b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.942+0000 7f4327782740 1 -- 172.21.15.90:0/311963720 shutdown_connections 2024-05-27T07:53:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.943+0000 7f4327782740 1 -- 172.21.15.90:0/311963720 wait complete. 2024-05-27T07:53:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.943+0000 7f4327782740 1 -- start start 2024-05-27T07:53:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.944+0000 7f4327782740 1 Processor -- start 2024-05-27T07:53:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.944+0000 7f4327782740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6ceee2f0 0x560e6d2c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.944+0000 7f4327782740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6cef64d0 0x560e6d2c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.945+0000 7f431de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6ceee2f0 0x560e6d2c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.945+0000 7f431de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6ceee2f0 0x560e6d2c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52564/0 (socket says 172.21.15.90:52564) 2024-05-27T07:53:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.945+0000 7f431d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6cef64d0 0x560e6d2c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.945+0000 7f431d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6cef64d0 0x560e6d2c3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56196/0 (socket says 172.21.15.90:56196) 2024-05-27T07:53:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.945+0000 7f4327782740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cf171b0 0x560e6d2cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.945+0000 7f431de00640 1 -- 172.21.15.90:0/1353192162 learned_addr learned my addr 172.21.15.90:0/1353192162 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.945+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560e6cf00790 con 0x560e6cf171b0 2024-05-27T07:53:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.945+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560e6d1c2bc0 con 0x560e6ceee2f0 2024-05-27T07:53:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.945+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560e6d2c6930 con 0x560e6cef64d0 2024-05-27T07:53:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.946+0000 7f4327782740 10 client.0 _ll_register_callbacks cb 0x560e6d1a0400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.946+0000 7f431e800640 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cf171b0 0x560e6d2cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.947+0000 7f431de00640 1 -- 172.21.15.90:0/1353192162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6cef64d0 msgr2=0x560e6d2c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.947+0000 7f431de00640 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6cef64d0 0x560e6d2c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.947+0000 7f431de00640 1 -- 172.21.15.90:0/1353192162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cf171b0 msgr2=0x560e6d2cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.947+0000 7f431de00640 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cf171b0 0x560e6d2cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.947+0000 7f431de00640 1 -- 172.21.15.90:0/1353192162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6cf2f210 con 0x560e6ceee2f0 2024-05-27T07:53:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.947+0000 7f431e800640 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cf171b0 0x560e6d2cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.948+0000 7f431de00640 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6ceee2f0 0x560e6d2c3a20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c03b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8271eb90007b18f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.948+0000 7f42fb400640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.949+0000 7f42fb400640 1 -- 172.21.15.90:0/1353192162 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c048020 con 0x560e6ceee2f0 2024-05-27T07:53:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.949+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430c004370 con 0x560e6ceee2f0 2024-05-27T07:53:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.949+0000 7f42fb400640 1 -- 172.21.15.90:0/1353192162 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f430c03bc60 con 0x560e6ceee2f0 2024-05-27T07:53:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.950+0000 7f42fb400640 1 -- 172.21.15.90:0/1353192162 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f430c053050 con 0x560e6ceee2f0 2024-05-27T07:53:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.950+0000 7f4327782740 10 client.36326 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.950+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560e6cffb620 con 0x560e6ceee2f0 2024-05-27T07:53:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.951+0000 7f42fb400640 1 -- 172.21.15.90:0/1353192162 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f430c004370 con 0x560e6ceee2f0 2024-05-27T07:53:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.951+0000 7f42fb400640 1 -- 172.21.15.90:0/1353192162 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7f430c044410 con 0x560e6ceee2f0 2024-05-27T07:53:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.952+0000 7f4327782740 10 client.36326 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.952+0000 7f4327782740 10 client.36326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.952+0000 7f4327782740 20 client.36326 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.952+0000 7f4327782740 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560e6ced5d80 0x560e6d2d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.952+0000 7f4327782740 4 client.36326 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.952+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e6d22cae0 con 0x560e6ced5d80 2024-05-27T07:53:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.952+0000 7f431d400640 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560e6ced5d80 0x560e6d2d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.954+0000 7f431d400640 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560e6ced5d80 0x560e6d2d59d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe2df3fdfd275848 server_cookie=bdc977a1f4774ab1 in_seq=0 out_seq=0 2024-05-27T07:53:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.954+0000 7f42fb400640 10 client.36326 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f42fb400640 1 -- 172.21.15.90:0/1353192162 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560e6d22cae0 con 0x560e6ced5d80 2024-05-27T07:53:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f42fb400640 10 client.36326 handle_command_reply: tid=0 2024-05-27T07:53:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f4327782740 1 client.36326 shutdown 2024-05-27T07:53:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6ceee2f0 msgr2=0x560e6d2c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f4327782740 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6ceee2f0 0x560e6d2c3a20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c03b870 comp rx=0 tx=0).stop 2024-05-27T07:53:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 shutdown_connections 2024-05-27T07:53:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f4327782740 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560e6ced5d80 0x560e6d2d59d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f4327782740 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cf171b0 0x560e6d2cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f4327782740 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6cef64d0 0x560e6d2c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f4327782740 1 --2- 172.21.15.90:0/1353192162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6ceee2f0 0x560e6d2c3a20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f42fb400640 0 client.36326 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.955+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 >> 172.21.15.90:0/1353192162 conn(0x560e6cf0b300 msgr2=0x560e6d2b9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.956+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 shutdown_connections 2024-05-27T07:53:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.956+0000 7f4327782740 1 -- 172.21.15.90:0/1353192162 wait complete. 2024-05-27T07:53:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.956+0000 7f4327782740 20 client.36326 trim_cache size 0 max 16384 2024-05-27T07:53:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.981+0000 7f4327782740 1 Processor -- start 2024-05-27T07:53:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.981+0000 7f4327782740 1 -- start start 2024-05-27T07:53:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.982+0000 7f4327782740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.982+0000 7f4327782740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.982+0000 7f4327782740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d2cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.982+0000 7f4327782740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b9560 con 0x560e6cef64d0 2024-05-27T07:53:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.982+0000 7f4327782740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b7350 con 0x560e6cf171b0 2024-05-27T07:53:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.982+0000 7f4327782740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b8e90 con 0x560e6ceee2f0 2024-05-27T07:53:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.983+0000 7f431e800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d2cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.983+0000 7f431de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.983+0000 7f431de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56200/0 (socket says 172.21.15.90:56200) 2024-05-27T07:53:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.983+0000 7f431e800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d2cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52574/0 (socket says 172.21.15.90:52574) 2024-05-27T07:53:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.983+0000 7f431de00640 1 -- 172.21.15.90:0/2958801323 learned_addr learned my addr 172.21.15.90:0/2958801323 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.983+0000 7f431d400640 1 --2- 172.21.15.90:0/2958801323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.984+0000 7f431e800640 1 -- 172.21.15.90:0/2958801323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 msgr2=0x560e6d2b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.984+0000 7f431e800640 1 --2- 172.21.15.90:0/2958801323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.984+0000 7f431e800640 1 -- 172.21.15.90:0/2958801323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 msgr2=0x560e6d2c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.985+0000 7f431e800640 1 --2- 172.21.15.90:0/2958801323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.985+0000 7f431e800640 1 -- 172.21.15.90:0/2958801323 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e6d2d5d40 con 0x560e6cf171b0 2024-05-27T07:53:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.985+0000 7f431d400640 1 --2- 172.21.15.90:0/2958801323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.985+0000 7f431e800640 1 --2- 172.21.15.90:0/2958801323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d2cfa00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4314009fd0 tx=0x7f4314015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1e777213ee20c65 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.985+0000 7f4306600640 1 -- 172.21.15.90:0/2958801323 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314012db0 con 0x560e6cf171b0 2024-05-27T07:53:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.986+0000 7f4306600640 1 -- 172.21.15.90:0/2958801323 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f431401b8c0 con 0x560e6cf171b0 2024-05-27T07:53:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.986+0000 7f4306600640 1 -- 172.21.15.90:0/2958801323 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431401a880 con 0x560e6cf171b0 2024-05-27T07:53:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.987+0000 7f4327782740 1 -- 172.21.15.90:0/2958801323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 msgr2=0x560e6d2cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.987+0000 7f4327782740 1 --2- 172.21.15.90:0/2958801323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d2cfa00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4314009fd0 tx=0x7f4314015000 comp rx=0 tx=0).stop 2024-05-27T07:53:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.987+0000 7f4327782740 1 -- 172.21.15.90:0/2958801323 shutdown_connections 2024-05-27T07:53:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.987+0000 7f4327782740 1 --2- 172.21.15.90:0/2958801323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d2cfa00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.987+0000 7f4327782740 1 --2- 172.21.15.90:0/2958801323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.987+0000 7f4327782740 1 --2- 172.21.15.90:0/2958801323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.987+0000 7f4327782740 1 -- 172.21.15.90:0/2958801323 >> 172.21.15.90:0/2958801323 conn(0x560e6cf0b300 msgr2=0x560e6d2c5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.987+0000 7f4327782740 1 -- 172.21.15.90:0/2958801323 shutdown_connections 2024-05-27T07:53:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.987+0000 7f4327782740 1 -- 172.21.15.90:0/2958801323 wait complete. 2024-05-27T07:53:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.988+0000 7f4327782740 1 -- start start 2024-05-27T07:53:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.988+0000 7f4327782740 1 Processor -- start 2024-05-27T07:53:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.989+0000 7f4327782740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.989+0000 7f4327782740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2c7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.990+0000 7f4327782740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d335e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.990+0000 7f4327782740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b80f0 con 0x560e6cef64d0 2024-05-27T07:53:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.990+0000 7f4327782740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b7a20 con 0x560e6cf171b0 2024-05-27T07:53:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.990+0000 7f4327782740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f43181b87c0 con 0x560e6ceee2f0 2024-05-27T07:53:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.990+0000 7f4327782740 10 client.0 _ll_register_callbacks cb 0x7f43140098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.990+0000 7f431de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.990+0000 7f431de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56208/0 (socket says 172.21.15.90:56208) 2024-05-27T07:53:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.990+0000 7f431de00640 1 -- 172.21.15.90:0/902914563 learned_addr learned my addr 172.21.15.90:0/902914563 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.990+0000 7f431e800640 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d335e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.991+0000 7f431d400640 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.991+0000 7f431de00640 1 -- 172.21.15.90:0/902914563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 msgr2=0x560e6d335e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:08.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.991+0000 7f431de00640 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d335e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.991+0000 7f431de00640 1 -- 172.21.15.90:0/902914563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 msgr2=0x560e6d2c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.991+0000 7f431de00640 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.991+0000 7f431de00640 1 -- 172.21.15.90:0/902914563 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e6d3363b0 con 0x560e6ceee2f0 2024-05-27T07:53:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.991+0000 7f431e800640 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d335e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:53:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.992+0000 7f431de00640 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2c71d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecd7e465704bbaeb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.992+0000 7f42fb400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:53:08.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.993+0000 7f42fb400640 1 -- 172.21.15.90:0/902914563 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c048040 con 0x560e6ceee2f0 2024-05-27T07:53:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.993+0000 7f42fb400640 1 -- 172.21.15.90:0/902914563 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f430c044c20 con 0x560e6ceee2f0 2024-05-27T07:53:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.993+0000 7f4327782740 1 -- 172.21.15.90:0/902914563 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f431401b8c0 con 0x560e6ceee2f0 2024-05-27T07:53:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.994+0000 7f42fb400640 1 -- 172.21.15.90:0/902914563 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f430c035070 con 0x560e6ceee2f0 2024-05-27T07:53:08.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.994+0000 7f4327782740 10 client.26383 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.994+0000 7f4327782740 1 -- 172.21.15.90:0/902914563 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560e6d33d620 con 0x560e6ceee2f0 2024-05-27T07:53:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.995+0000 7f42fb400640 1 -- 172.21.15.90:0/902914563 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f431401b8c0 con 0x560e6ceee2f0 2024-05-27T07:53:08.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.995+0000 7f42fb400640 1 -- 172.21.15.90:0/902914563 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7f430c0037f0 con 0x560e6ceee2f0 2024-05-27T07:53:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.995+0000 7f4327782740 10 client.26383 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.995+0000 7f4327782740 10 client.26383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.995+0000 7f4327782740 20 client.26383 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.996+0000 7f4327782740 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560e6ceab1c0 0x560e6d33fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.996+0000 7f4327782740 4 client.26383 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:53:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.996+0000 7f4327782740 1 -- 172.21.15.90:0/902914563 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e6d2262c0 con 0x560e6ceab1c0 2024-05-27T07:53:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.996+0000 7f431d400640 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560e6ceab1c0 0x560e6d33fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.997+0000 7f42fb400640 10 client.26383 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:08.997+0000 7f431d400640 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560e6ceab1c0 0x560e6d33fc80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476e47b59493ead5 server_cookie=533b63c27338b9b2 in_seq=0 out_seq=0 2024-05-27T07:53:09.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:09 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6834/1963864918,v1:172.21.15.90:6835/1963864918] up:boot 2024-05-27T07:53:09.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:09 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:53:09.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:09 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:53:09.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:09 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6834/1963864918,v1:172.21.15.90:6835/1963864918] up:boot 2024-05-27T07:53:09.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:09 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:53:09.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:09 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:53:09.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:09 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6834/1963864918,v1:172.21.15.90:6835/1963864918] up:boot 2024-05-27T07:53:09.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:09 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:53:09.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:09 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:53:10.417 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:10 smithi090 ceph-mon[31429]: pgmap v211: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 13 MiB/s wr, 672 op/s 2024-05-27T07:53:10.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:10 smithi129 ceph-mon[35254]: pgmap v211: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 13 MiB/s wr, 672 op/s 2024-05-27T07:53:10.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:10 smithi136 ceph-mon[35324]: pgmap v211: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 13 MiB/s wr, 672 op/s 2024-05-27T07:53:10.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.776+0000 7fef05e00700 1 -- 172.21.15.90:0/3767301385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef001151e0 msgr2=0x7fef00117600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.776+0000 7fef05e00700 1 --2- 172.21.15.90:0/3767301385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef001151e0 0x7fef00117600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feef8009f90 tx=0x7feef800b330 comp rx=0 tx=0).stop 2024-05-27T07:53:10.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.777+0000 7fef05e00700 1 -- 172.21.15.90:0/3767301385 shutdown_connections 2024-05-27T07:53:10.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.777+0000 7fef05e00700 1 --2- 172.21.15.90:0/3767301385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef001151e0 0x7fef00117600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.777+0000 7fef05e00700 1 --2- 172.21.15.90:0/3767301385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef00077ce0 0x7fef000780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.777+0000 7fef05e00700 1 --2- 172.21.15.90:0/3767301385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef00078630 0x7fef00114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.777+0000 7fef05e00700 1 -- 172.21.15.90:0/3767301385 >> 172.21.15.90:0/3767301385 conn(0x7fef00102040 msgr2=0x7fef00104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:10.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.777+0000 7fef05e00700 1 -- 172.21.15.90:0/3767301385 shutdown_connections 2024-05-27T07:53:10.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.777+0000 7fef05e00700 1 -- 172.21.15.90:0/3767301385 wait complete. 2024-05-27T07:53:10.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.780+0000 7fef05e00700 1 Processor -- start 2024-05-27T07:53:10.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.780+0000 7fef05e00700 1 -- start start 2024-05-27T07:53:10.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.780+0000 7fef05e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef00077ce0 0x7fef001a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.780+0000 7fef05e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef00078630 0x7fef001a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.781+0000 7fef05e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef001151e0 0x7fef001a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.781+0000 7fef05e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fef0011a9f0 con 0x7fef00077ce0 2024-05-27T07:53:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.781+0000 7feefe000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef00078630 0x7fef001a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.781+0000 7fef05e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef0011a870 con 0x7fef00078630 2024-05-27T07:53:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.781+0000 7feefe000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef00078630 0x7fef001a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52606/0 (socket says 172.21.15.90:52606) 2024-05-27T07:53:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.781+0000 7feefe000700 1 -- 172.21.15.90:0/3713073887 learned_addr learned my addr 172.21.15.90:0/3713073887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.781+0000 7fef05e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef0011ab70 con 0x7fef001151e0 2024-05-27T07:53:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.782+0000 7feefea00700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef00077ce0 0x7fef001a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.782+0000 7feeff400700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef001151e0 0x7fef001a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.783+0000 7feeff400700 1 -- 172.21.15.90:0/3713073887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef00078630 msgr2=0x7fef001a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.783+0000 7feeff400700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef00078630 0x7fef001a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.783+0000 7feeff400700 1 -- 172.21.15.90:0/3713073887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef00077ce0 msgr2=0x7fef001a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.783+0000 7feeff400700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef00077ce0 0x7fef001a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.783+0000 7feeff400700 1 -- 172.21.15.90:0/3713073887 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef001a6270 con 0x7fef001151e0 2024-05-27T07:53:10.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.783+0000 7feeff400700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef001151e0 0x7fef001a5bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feef8015000 tx=0x7feef800cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:10.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.784+0000 7fef04a00700 1 -- 172.21.15.90:0/3713073887 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8021070 con 0x7fef001151e0 2024-05-27T07:53:10.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.784+0000 7fef04a00700 1 -- 172.21.15.90:0/3713073887 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feef8007ac0 con 0x7fef001151e0 2024-05-27T07:53:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.784+0000 7fef04a00700 1 -- 172.21.15.90:0/3713073887 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef801b680 con 0x7fef001151e0 2024-05-27T07:53:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.784+0000 7fef05e00700 1 -- 172.21.15.90:0/3713073887 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef001a6500 con 0x7fef001151e0 2024-05-27T07:53:10.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.785+0000 7fef05e00700 1 -- 172.21.15.90:0/3713073887 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef001a69f0 con 0x7fef001151e0 2024-05-27T07:53:10.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.792+0000 7fef148ddb80 1 Processor -- start 2024-05-27T07:53:10.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.798+0000 7fef04a00700 1 -- 172.21.15.90:0/3713073887 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7feef8024070 con 0x7fef001151e0 2024-05-27T07:53:10.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.798+0000 7fef148ddb80 1 -- start start 2024-05-27T07:53:10.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.799+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583964729f0 0x558396505330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.799+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 0x558396505870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.799+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396505e70 0x558396512110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.800+0000 7fef148ddb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558396433e40 con 0x558396505e70 2024-05-27T07:53:10.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.800+0000 7fef148ddb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5583964239e0 con 0x5583964729f0 2024-05-27T07:53:10.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.800+0000 7fef148ddb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558396427a20 con 0x558396386e40 2024-05-27T07:53:10.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.800+0000 7feefe000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583964729f0 0x558396505330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.800+0000 7feefe000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583964729f0 0x558396505330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52608/0 (socket says 172.21.15.90:52608) 2024-05-27T07:53:10.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.800+0000 7feefe000700 1 -- 172.21.15.90:0/623888469 learned_addr learned my addr 172.21.15.90:0/623888469 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:10.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.800+0000 7feefea00700 1 --2- 172.21.15.90:0/623888469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 0x558396505870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.801+0000 7feefe000700 1 -- 172.21.15.90:0/623888469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 msgr2=0x558396505870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:10.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.801+0000 7feefe000700 1 --2- 172.21.15.90:0/623888469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 0x558396505870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.801+0000 7feefe000700 1 -- 172.21.15.90:0/623888469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396505e70 msgr2=0x558396512110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:10.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.801+0000 7feefe000700 1 --2- 172.21.15.90:0/623888469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396505e70 0x558396512110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.801+0000 7feefe000700 1 -- 172.21.15.90:0/623888469 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583963ec420 con 0x5583964729f0 2024-05-27T07:53:10.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.802+0000 7fef04a00700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feee40b2aa0 0x7feee40b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.802+0000 7fef04a00700 1 -- 172.21.15.90:0/3713073887 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7feef800f020 con 0x7fef001151e0 2024-05-27T07:53:10.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.802+0000 7feefea00700 1 --2- 172.21.15.90:0/623888469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 0x558396505870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:10.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.803+0000 7feefe000700 1 --2- 172.21.15.90:0/623888469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583964729f0 0x558396505330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feee8009c00 tx=0x7feee8009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b68d64f81f8ce021 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:10.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.803+0000 7feefea00700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feee40b2aa0 0x7feee40b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.804+0000 7feef4c00700 1 -- 172.21.15.90:0/623888469 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8017070 con 0x5583964729f0 2024-05-27T07:53:10.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.804+0000 7feef4c00700 1 -- 172.21.15.90:0/623888469 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feee8013400 con 0x5583964729f0 2024-05-27T07:53:10.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.804+0000 7feef4c00700 1 -- 172.21.15.90:0/623888469 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8017070 con 0x5583964729f0 2024-05-27T07:53:10.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.806+0000 7fef148ddb80 1 -- 172.21.15.90:0/623888469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583964729f0 msgr2=0x558396505330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.806+0000 7fef148ddb80 1 --2- 172.21.15.90:0/623888469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583964729f0 0x558396505330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feee8009c00 tx=0x7feee8009f10 comp rx=0 tx=0).stop 2024-05-27T07:53:10.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.806+0000 7feefea00700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feee40b2aa0 0x7feee40b4fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x558396502790 tx=0x7feef0009310 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:10.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.806+0000 7fef148ddb80 1 -- 172.21.15.90:0/623888469 shutdown_connections 2024-05-27T07:53:10.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.806+0000 7fef148ddb80 1 --2- 172.21.15.90:0/623888469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396505e70 0x558396512110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.806+0000 7fef148ddb80 1 --2- 172.21.15.90:0/623888469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583964729f0 0x558396505330 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feee8009c00 tx=0x7feee8009f10 comp rx=0 tx=0).stop 2024-05-27T07:53:10.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.806+0000 7fef148ddb80 1 --2- 172.21.15.90:0/623888469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 0x558396505870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.806+0000 7fef148ddb80 1 -- 172.21.15.90:0/623888469 >> 172.21.15.90:0/623888469 conn(0x558396420390 msgr2=0x5583965014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:10.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.807+0000 7fef148ddb80 1 -- 172.21.15.90:0/623888469 shutdown_connections 2024-05-27T07:53:10.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.807+0000 7fef148ddb80 1 -- 172.21.15.90:0/623888469 wait complete. 2024-05-27T07:53:10.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.808+0000 7fef148ddb80 1 -- start start 2024-05-27T07:53:10.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.808+0000 7fef148ddb80 1 Processor -- start 2024-05-27T07:53:10.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.809+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 0x55839650f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.809+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396505e70 0x55839650fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.809+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839651beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.809+0000 7fef148ddb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558396329fa0 con 0x558396505e70 2024-05-27T07:53:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.809+0000 7fef148ddb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5583963c4760 con 0x5583965101e0 2024-05-27T07:53:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.809+0000 7fef148ddb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558396512b80 con 0x558396386e40 2024-05-27T07:53:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.809+0000 7fef148ddb80 10 client.0 _ll_register_callbacks cb 0x5583963cdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:10.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.810+0000 7feefea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 0x55839650f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.810+0000 7feefea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 0x55839650f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56258/0 (socket says 172.21.15.90:56258) 2024-05-27T07:53:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.810+0000 7feefea00700 1 -- 172.21.15.90:0/2311675485 learned_addr learned my addr 172.21.15.90:0/2311675485 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.810+0000 7feeff400700 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839651beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.810+0000 7feefe000700 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396505e70 0x55839650fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.810+0000 7feeff400700 1 -- 172.21.15.90:0/2311675485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 msgr2=0x55839650f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.810+0000 7feeff400700 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 0x55839650f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.810+0000 7feeff400700 1 -- 172.21.15.90:0/2311675485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396505e70 msgr2=0x55839650fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.811+0000 7feeff400700 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396505e70 0x55839650fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.811+0000 7feeff400700 1 -- 172.21.15.90:0/2311675485 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583963738d0 con 0x5583965101e0 2024-05-27T07:53:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.811+0000 7feefe000700 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396505e70 0x55839650fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.812+0000 7feeff400700 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839651beb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feef80e1200 tx=0x7feef80aa930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b0565235251269a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.812+0000 7feeed600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.813+0000 7feeed600700 1 -- 172.21.15.90:0/2311675485 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef80aae20 con 0x5583965101e0 2024-05-27T07:53:10.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.813+0000 7feeed600700 1 -- 172.21.15.90:0/2311675485 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7feef80ab290 con 0x5583965101e0 2024-05-27T07:53:10.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.813+0000 7fef148ddb80 1 -- 172.21.15.90:0/2311675485 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee8013400 con 0x5583965101e0 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.813+0000 7feeed600700 1 -- 172.21.15.90:0/2311675485 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feef80af1d0 con 0x5583965101e0 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.813+0000 7fef148ddb80 10 client.36344 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.814+0000 7fef148ddb80 1 -- 172.21.15.90:0/2311675485 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5583963f4060 con 0x5583965101e0 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.814+0000 7feeed600700 1 -- 172.21.15.90:0/2311675485 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feee8013400 con 0x5583965101e0 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.814+0000 7feeed600700 1 -- 172.21.15.90:0/2311675485 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7feef8011070 con 0x5583965101e0 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.814+0000 7fef148ddb80 10 client.36344 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.814+0000 7fef148ddb80 10 client.36344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.814+0000 7fef148ddb80 20 client.36344 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.814+0000 7fef148ddb80 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558396520050 0x558396522490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.814+0000 7fef148ddb80 4 client.36344 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.814+0000 7fef148ddb80 1 -- 172.21.15.90:0/2311675485 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558396435d90 con 0x558396520050 2024-05-27T07:53:10.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.815+0000 7feefea00700 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558396520050 0x558396522490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.816+0000 7feefea00700 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558396520050 0x558396522490 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3024747292765b server_cookie=3bec24f4a92cce96 in_seq=0 out_seq=0 2024-05-27T07:53:10.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.816+0000 7feeed600700 10 client.36344 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:10.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.819+0000 7feeed600700 1 -- 172.21.15.90:0/2311675485 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558396435d90 con 0x558396520050 2024-05-27T07:53:10.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.819+0000 7feeed600700 10 client.36344 handle_command_reply: tid=0 2024-05-27T07:53:10.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.819+0000 7fef148ddb80 1 client.36344 shutdown 2024-05-27T07:53:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.819+0000 7fef148ddb80 1 -- 172.21.15.90:0/2311675485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 msgr2=0x55839651beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.819+0000 7fef148ddb80 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839651beb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feef80e1200 tx=0x7feef80aa930 comp rx=0 tx=0).stop 2024-05-27T07:53:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.820+0000 7fef148ddb80 1 -- 172.21.15.90:0/2311675485 shutdown_connections 2024-05-27T07:53:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.820+0000 7fef148ddb80 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558396520050 0x558396522490 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.820+0000 7fef148ddb80 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839651beb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.820+0000 7fef148ddb80 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396386e40 0x55839650f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.820+0000 7fef148ddb80 1 --2- 172.21.15.90:0/2311675485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396505e70 0x55839650fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.820+0000 7fef148ddb80 1 -- 172.21.15.90:0/2311675485 >> 172.21.15.90:0/2311675485 conn(0x558396420390 msgr2=0x5583965053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:10.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.820+0000 7feeed600700 0 client.36344 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:10.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.820+0000 7fef148ddb80 1 -- 172.21.15.90:0/2311675485 shutdown_connections 2024-05-27T07:53:10.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.821+0000 7fef148ddb80 1 -- 172.21.15.90:0/2311675485 wait complete. 2024-05-27T07:53:10.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.821+0000 7fef148ddb80 20 client.36344 trim_cache size 0 max 16384 2024-05-27T07:53:10.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.864+0000 7fef148ddb80 1 Processor -- start 2024-05-27T07:53:10.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.864+0000 7fef148ddb80 1 -- start start 2024-05-27T07:53:10.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.864+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 0x55839650af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.864+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558396505e70 0x558396501630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.864+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583965101e0 0x55839651bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.864+0000 7fef148ddb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fef001b8650 con 0x558396386e40 2024-05-27T07:53:10.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.864+0000 7fef148ddb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef001b6440 con 0x558396505e70 2024-05-27T07:53:10.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.864+0000 7fef148ddb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef001b7f80 con 0x5583965101e0 2024-05-27T07:53:10.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.865+0000 7feefea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 0x55839650af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.865+0000 7feefea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 0x55839650af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52568/0 (socket says 172.21.15.90:52568) 2024-05-27T07:53:10.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.865+0000 7feefea00700 1 -- 172.21.15.90:0/734710336 learned_addr learned my addr 172.21.15.90:0/734710336 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:10.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.865+0000 7feefea00700 1 -- 172.21.15.90:0/734710336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583965101e0 msgr2=0x55839651bc20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:53:10.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.865+0000 7feefea00700 1 --2- 172.21.15.90:0/734710336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583965101e0 0x55839651bc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.865+0000 7feefea00700 1 -- 172.21.15.90:0/734710336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558396505e70 msgr2=0x558396501630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:53:10.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.866+0000 7feefe000700 1 --2- 172.21.15.90:0/734710336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558396505e70 0x558396501630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.866+0000 7feefea00700 1 --2- 172.21.15.90:0/734710336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558396505e70 0x558396501630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.866+0000 7feefea00700 1 -- 172.21.15.90:0/734710336 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583964fd240 con 0x558396386e40 2024-05-27T07:53:10.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.868+0000 7feefea00700 1 --2- 172.21.15.90:0/734710336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 0x55839650af90 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7feef0009fd0 tx=0x7feef001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6495a263a599165 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:10.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.870+0000 7feeefe00700 1 -- 172.21.15.90:0/734710336 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0034450 con 0x558396386e40 2024-05-27T07:53:10.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.870+0000 7feeefe00700 1 -- 172.21.15.90:0/734710336 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feef0034a80 con 0x558396386e40 2024-05-27T07:53:10.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.870+0000 7feeefe00700 1 -- 172.21.15.90:0/734710336 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef002a980 con 0x558396386e40 2024-05-27T07:53:10.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.872+0000 7fef148ddb80 1 -- 172.21.15.90:0/734710336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 msgr2=0x55839650af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.872+0000 7fef148ddb80 1 --2- 172.21.15.90:0/734710336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 0x55839650af90 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7feef0009fd0 tx=0x7feef001b000 comp rx=0 tx=0).stop 2024-05-27T07:53:10.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.873+0000 7fef148ddb80 1 -- 172.21.15.90:0/734710336 shutdown_connections 2024-05-27T07:53:10.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.873+0000 7fef148ddb80 1 --2- 172.21.15.90:0/734710336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583965101e0 0x55839651bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.873+0000 7fef148ddb80 1 --2- 172.21.15.90:0/734710336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 0x55839650af90 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.873+0000 7fef148ddb80 1 --2- 172.21.15.90:0/734710336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558396505e70 0x558396501630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.873+0000 7fef148ddb80 1 -- 172.21.15.90:0/734710336 >> 172.21.15.90:0/734710336 conn(0x558396420390 msgr2=0x558396500f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:10.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.873+0000 7fef148ddb80 1 -- 172.21.15.90:0/734710336 shutdown_connections 2024-05-27T07:53:10.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.874+0000 7fef148ddb80 1 -- 172.21.15.90:0/734710336 wait complete. 2024-05-27T07:53:10.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.875+0000 7fef148ddb80 1 -- start start 2024-05-27T07:53:10.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.875+0000 7fef148ddb80 1 Processor -- start 2024-05-27T07:53:10.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.876+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 0x558396518200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.876+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396505e70 0x558396509c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.876+0000 7fef148ddb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839650a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.876+0000 7fef148ddb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fef001b71e0 con 0x558396386e40 2024-05-27T07:53:10.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.876+0000 7fef148ddb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef001b6b10 con 0x5583965101e0 2024-05-27T07:53:10.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.876+0000 7fef148ddb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef001b78b0 con 0x558396505e70 2024-05-27T07:53:10.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.876+0000 7fef148ddb80 10 client.0 _ll_register_callbacks cb 0x5583963cdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:10.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.877+0000 7feeff400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839650a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.877+0000 7feeff400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839650a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52624/0 (socket says 172.21.15.90:52624) 2024-05-27T07:53:10.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.877+0000 7feeff400700 1 -- 172.21.15.90:0/3954897323 learned_addr learned my addr 172.21.15.90:0/3954897323 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:10.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.877+0000 7feefe000700 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396505e70 0x558396509c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.878+0000 7feefea00700 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 0x558396518200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.879+0000 7feeff400700 1 -- 172.21.15.90:0/3954897323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396505e70 msgr2=0x558396509c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:10.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.879+0000 7feeff400700 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396505e70 0x558396509c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.879+0000 7feeff400700 1 -- 172.21.15.90:0/3954897323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 msgr2=0x558396518200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:10.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.879+0000 7feeff400700 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 0x558396518200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.879+0000 7feeff400700 1 -- 172.21.15.90:0/3954897323 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55839650a6b0 con 0x5583965101e0 2024-05-27T07:53:10.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.880+0000 7feeff400700 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839650a170 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feef80aadb0 tx=0x7feef80097c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e1f70c54bcc0a24 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:10.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.880+0000 7feeed600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:10.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.881+0000 7feeed600700 1 -- 172.21.15.90:0/3954897323 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef80aae20 con 0x5583965101e0 2024-05-27T07:53:10.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.881+0000 7fef148ddb80 1 -- 172.21.15.90:0/3954897323 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feef0034a80 con 0x5583965101e0 2024-05-27T07:53:10.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.882+0000 7feeed600700 1 -- 172.21.15.90:0/3954897323 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7feef80a7410 con 0x5583965101e0 2024-05-27T07:53:10.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.882+0000 7feeed600700 1 -- 172.21.15.90:0/3954897323 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feef808e050 con 0x5583965101e0 2024-05-27T07:53:10.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.883+0000 7fef148ddb80 10 client.36347 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:10.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.883+0000 7fef148ddb80 1 -- 172.21.15.90:0/3954897323 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558396517ae0 con 0x5583965101e0 2024-05-27T07:53:10.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.884+0000 7feeed600700 1 -- 172.21.15.90:0/3954897323 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feef0034a80 con 0x5583965101e0 2024-05-27T07:53:10.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.884+0000 7feeed600700 1 -- 172.21.15.90:0/3954897323 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7feef8011070 con 0x5583965101e0 2024-05-27T07:53:10.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.884+0000 7fef148ddb80 10 client.36347 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:10.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.884+0000 7fef148ddb80 10 client.36347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:10.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.884+0000 7fef148ddb80 20 client.36347 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:10.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.884+0000 7fef148ddb80 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55839658acb0 0x55839658d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:10.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.884+0000 7fef148ddb80 4 client.36347 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:53:10.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.884+0000 7fef148ddb80 1 -- 172.21.15.90:0/3954897323 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55839642f570 con 0x55839658acb0 2024-05-27T07:53:10.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.885+0000 7feefea00700 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55839658acb0 0x55839658d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:10.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.887+0000 7feeed600700 10 client.36347 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:10.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.887+0000 7feefea00700 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55839658acb0 0x55839658d0d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3657df6a2cbacf54 server_cookie=875958d9584d101a in_seq=0 out_seq=0 2024-05-27T07:53:10.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.888+0000 7feeed600700 1 -- 172.21.15.90:0/3954897323 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55839642f570 con 0x55839658acb0 2024-05-27T07:53:10.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.888+0000 7feeed600700 10 client.36347 handle_command_reply: tid=0 2024-05-27T07:53:10.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.888+0000 7fef148ddb80 1 client.36347 shutdown 2024-05-27T07:53:10.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.889+0000 7fef148ddb80 1 -- 172.21.15.90:0/3954897323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 msgr2=0x55839650a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.889+0000 7fef148ddb80 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839650a170 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feef80aadb0 tx=0x7feef80097c0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.889+0000 7fef148ddb80 1 -- 172.21.15.90:0/3954897323 shutdown_connections 2024-05-27T07:53:10.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.889+0000 7fef148ddb80 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55839658acb0 0x55839658d0d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.889+0000 7fef148ddb80 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583965101e0 0x55839650a170 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.889+0000 7fef148ddb80 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558396386e40 0x558396518200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.889+0000 7fef148ddb80 1 --2- 172.21.15.90:0/3954897323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558396505e70 0x558396509c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.889+0000 7fef148ddb80 1 -- 172.21.15.90:0/3954897323 >> 172.21.15.90:0/3954897323 conn(0x558396420390 msgr2=0x55839650dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:10.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.889+0000 7feeed600700 0 client.36347 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:10.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.889+0000 7fef148ddb80 1 -- 172.21.15.90:0/3954897323 shutdown_connections 2024-05-27T07:53:10.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.890+0000 7fef148ddb80 1 -- 172.21.15.90:0/3954897323 wait complete. 2024-05-27T07:53:10.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.890+0000 7fef148ddb80 20 client.36347 trim_cache size 0 max 16384 2024-05-27T07:53:10.893 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:53:10.894 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:53:10.894 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:53:10.894 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:53:10.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 -- 172.21.15.90:0/3713073887 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feee40b2aa0 msgr2=0x7feee40b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feee40b2aa0 0x7feee40b4fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x558396502790 tx=0x7feef0009310 comp rx=0 tx=0).stop 2024-05-27T07:53:10.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 -- 172.21.15.90:0/3713073887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef001151e0 msgr2=0x7fef001a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef001151e0 0x7fef001a5bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feef8015000 tx=0x7feef800cd70 comp rx=0 tx=0).stop 2024-05-27T07:53:10.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 -- 172.21.15.90:0/3713073887 shutdown_connections 2024-05-27T07:53:10.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef001151e0 0x7fef001a5bd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef00077ce0 0x7fef001a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feee40b2aa0 0x7feee40b4fb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 --2- 172.21.15.90:0/3713073887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef00078630 0x7fef001a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 -- 172.21.15.90:0/3713073887 >> 172.21.15.90:0/3713073887 conn(0x7fef00102040 msgr2=0x7fef00104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:10.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 -- 172.21.15.90:0/3713073887 shutdown_connections 2024-05-27T07:53:10.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:10.892+0000 7fef05e00700 1 -- 172.21.15.90:0/3713073887 wait complete. 2024-05-27T07:53:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.953+0000 7f42fb400640 1 -- 172.21.15.90:0/902914563 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560e6d2262c0 con 0x560e6ceab1c0 2024-05-27T07:53:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.953+0000 7f42fb400640 10 client.26383 handle_command_reply: tid=0 2024-05-27T07:53:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.953+0000 7f4327782740 1 client.26383 shutdown 2024-05-27T07:53:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.953+0000 7f4327782740 1 -- 172.21.15.90:0/902914563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 msgr2=0x560e6d2c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.953+0000 7f4327782740 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2c71d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c009260 comp rx=0 tx=0).stop 2024-05-27T07:53:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.954+0000 7f4327782740 1 -- 172.21.15.90:0/902914563 shutdown_connections 2024-05-27T07:53:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.954+0000 7f4327782740 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560e6ceab1c0 0x560e6d33fc80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.954+0000 7f4327782740 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560e6cf171b0 0x560e6d335e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.954+0000 7f4327782740 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560e6cef64d0 0x560e6d2c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.954+0000 7f4327782740 1 --2- 172.21.15.90:0/902914563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ceee2f0 0x560e6d2c71d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.954+0000 7f4327782740 1 -- 172.21.15.90:0/902914563 >> 172.21.15.90:0/902914563 conn(0x560e6cf0b300 msgr2=0x560e6d2c4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.954+0000 7f42fb400640 0 client.26383 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.954+0000 7f4327782740 1 -- 172.21.15.90:0/902914563 shutdown_connections 2024-05-27T07:53:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.954+0000 7f4327782740 1 -- 172.21.15.90:0/902914563 wait complete. 2024-05-27T07:53:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.955+0000 7f4327782740 20 client.26383 trim_cache size 0 max 16384 2024-05-27T07:53:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:53:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:53:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:53:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.957+0000 7f4320800640 1 -- 172.21.15.90:0/853763398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f43000b29d0 msgr2=0x7f43000b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.957+0000 7f4320800640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f43000b29d0 0x7f43000b4ee0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x560e6d2b69d0 tx=0x7f4314009500 comp rx=0 tx=0).stop 2024-05-27T07:53:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.958+0000 7f4320800640 1 -- 172.21.15.90:0/853763398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f431810db40 msgr2=0x7f431810b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.958+0000 7f4320800640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f431810db40 0x7f431810b030 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f430800ea60 tx=0x7f430800ef20 comp rx=0 tx=0).stop 2024-05-27T07:53:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.958+0000 7f4320800640 1 -- 172.21.15.90:0/853763398 shutdown_connections 2024-05-27T07:53:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.958+0000 7f4320800640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f43000b29d0 0x7f43000b4ee0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.958+0000 7f4320800640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43181104a0 0x7f431810b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.958+0000 7f4320800640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f431810db40 0x7f431810b030 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.958+0000 7f4320800640 1 --2- 172.21.15.90:0/853763398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4318075660 0x7f431810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.958+0000 7f4320800640 1 -- 172.21.15.90:0/853763398 >> 172.21.15.90:0/853763398 conn(0x7f43180ff480 msgr2=0x7f43181108f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.959+0000 7f4320800640 1 -- 172.21.15.90:0/853763398 shutdown_connections 2024-05-27T07:53:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:10.959+0000 7f4320800640 1 -- 172.21.15.90:0/853763398 wait complete. 2024-05-27T07:53:11.777 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:53:11.778 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:53:12.370 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:12.537 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:12 smithi090 ceph-mon[31429]: pgmap v212: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 13 MiB/s wr, 672 op/s 2024-05-27T07:53:12.537 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:12 smithi090 ceph-mon[31429]: osdmap e100: 12 total, 12 up, 12 in 2024-05-27T07:53:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:12 smithi129 ceph-mon[35254]: pgmap v212: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 13 MiB/s wr, 672 op/s 2024-05-27T07:53:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:12 smithi129 ceph-mon[35254]: osdmap e100: 12 total, 12 up, 12 in 2024-05-27T07:53:12.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:12 smithi136 ceph-mon[35324]: pgmap v212: 129 pgs: 129 active+clean; 2.1 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 13 MiB/s wr, 672 op/s 2024-05-27T07:53:12.897 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:12 smithi136 ceph-mon[35324]: osdmap e100: 12 total, 12 up, 12 in 2024-05-27T07:53:13.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 175.0 s, 696.6 tps, lat 143.397 ms stddev 203.589 2024-05-27T07:53:14.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:14 smithi090 ceph-mon[31429]: pgmap v214: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 11 MiB/s wr, 406 op/s 2024-05-27T07:53:14.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:14 smithi129 ceph-mon[35254]: pgmap v214: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 11 MiB/s wr, 406 op/s 2024-05-27T07:53:14.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:14 smithi136 ceph-mon[35324]: pgmap v214: 129 pgs: 129 active+clean; 2.1 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 11 MiB/s wr, 406 op/s 2024-05-27T07:53:14.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.824+0000 7fc99e200700 1 -- 172.21.15.90:0/806262517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc99810e730 msgr2=0x7fc998110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:14.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.824+0000 7fc99e200700 1 --2- 172.21.15.90:0/806262517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc99810e730 0x7fc998110bd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc99000b0a0 tx=0x7fc99000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:53:14.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.825+0000 7fc99e200700 1 -- 172.21.15.90:0/806262517 shutdown_connections 2024-05-27T07:53:14.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.825+0000 7fc99e200700 1 --2- 172.21.15.90:0/806262517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc99810e730 0x7fc998110bd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:14.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.825+0000 7fc99e200700 1 --2- 172.21.15.90:0/806262517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc998105300 0x7fc998105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:14.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.825+0000 7fc99e200700 1 --2- 172.21.15.90:0/806262517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105c50 0x7fc99810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:14.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.825+0000 7fc99e200700 1 -- 172.21.15.90:0/806262517 >> 172.21.15.90:0/806262517 conn(0x7fc998078ab0 msgr2=0x7fc998078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:14.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.825+0000 7fc99e200700 1 -- 172.21.15.90:0/806262517 shutdown_connections 2024-05-27T07:53:14.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.825+0000 7fc99e200700 1 -- 172.21.15.90:0/806262517 wait complete. 2024-05-27T07:53:14.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.827+0000 7fc99e200700 1 Processor -- start 2024-05-27T07:53:14.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.827+0000 7fc99e200700 1 -- start start 2024-05-27T07:53:14.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.828+0000 7fc99e200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105300 0x7fc998118890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:14.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.828+0000 7fc99e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105c50 0x7fc998118df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:14.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.828+0000 7fc99e200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99810e730 0x7fc998113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:14.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.828+0000 7fc99e200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc998104130 con 0x7fc998105c50 2024-05-27T07:53:14.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.828+0000 7fc99e200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc998103fb0 con 0x7fc99810e730 2024-05-27T07:53:14.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.828+0000 7fc99e200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981042b0 con 0x7fc998105300 2024-05-27T07:53:14.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.828+0000 7fc99ce00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105300 0x7fc998118890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:14.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.828+0000 7fc99ce00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105300 0x7fc998118890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56308/0 (socket says 172.21.15.90:56308) 2024-05-27T07:53:14.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.828+0000 7fc99ce00700 1 -- 172.21.15.90:0/2525350861 learned_addr learned my addr 172.21.15.90:0/2525350861 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:14.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.828+0000 7fc997e00700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105c50 0x7fc998118df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:14.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.829+0000 7fc99d800700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99810e730 0x7fc998113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:14.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.829+0000 7fc99ce00700 1 -- 172.21.15.90:0/2525350861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99810e730 msgr2=0x7fc998113aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:14.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.829+0000 7fc99ce00700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99810e730 0x7fc998113aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:14.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.829+0000 7fc99ce00700 1 -- 172.21.15.90:0/2525350861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105c50 msgr2=0x7fc998118df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:14.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.829+0000 7fc99ce00700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105c50 0x7fc998118df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:14.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.829+0000 7fc99ce00700 1 -- 172.21.15.90:0/2525350861 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9981142d0 con 0x7fc998105300 2024-05-27T07:53:14.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.830+0000 7fc99ce00700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105300 0x7fc998118890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00d950 tx=0x7fc98c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:14.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.830+0000 7fc995600700 1 -- 172.21.15.90:0/2525350861 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c016070 con 0x7fc998105300 2024-05-27T07:53:14.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.830+0000 7fc99e200700 1 -- 172.21.15.90:0/2525350861 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc998114560 con 0x7fc998105300 2024-05-27T07:53:14.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.830+0000 7fc99e200700 1 -- 172.21.15.90:0/2525350861 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99811e780 con 0x7fc998105300 2024-05-27T07:53:14.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.831+0000 7fc995600700 1 -- 172.21.15.90:0/2525350861 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc98c00fd20 con 0x7fc998105300 2024-05-27T07:53:14.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.831+0000 7fc995600700 1 -- 172.21.15.90:0/2525350861 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00e3b0 con 0x7fc998105300 2024-05-27T07:53:14.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.832+0000 7fc99e200700 1 -- 172.21.15.90:0/2525350861 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc978005330 con 0x7fc998105300 2024-05-27T07:53:14.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.844+0000 7fc995600700 1 -- 172.21.15.90:0/2525350861 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc98c019070 con 0x7fc998105300 2024-05-27T07:53:14.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.848+0000 7fc995600700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9800b2b20 0x7fc9800b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:14.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.849+0000 7fc995600700 1 -- 172.21.15.90:0/2525350861 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fc98c0e2730 con 0x7fc998105300 2024-05-27T07:53:14.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.849+0000 7fc995600700 1 -- 172.21.15.90:0/2525350861 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc98c0e2b50 con 0x7fc998105300 2024-05-27T07:53:14.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.851+0000 7fc997e00700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9800b2b20 0x7fc9800b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:14.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:14.853+0000 7fc997e00700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9800b2b20 0x7fc9800b5030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc998102c10 tx=0x7fc984009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.289+0000 7fc99e200700 1 -- 172.21.15.90:0/2525350861 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc978005620 con 0x7fc998105300 2024-05-27T07:53:15.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.299+0000 7fc995600700 1 -- 172.21.15.90:0/2525350861 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11111 (secure 0 0 0) 0x7fc98c0a9ce0 con 0x7fc998105300 2024-05-27T07:53:15.302 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:53:15.302 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":36317,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1963864918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1963864918},{"type":"v1","addr":"172.21.15.90:6835","nonce":1963864918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:52:57.034433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26221":{"gid":26221,"name":"j","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6837/292961192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":292961192},{"type":"v1","addr":"172.21.15.90:6837","nonce":292961192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T07:53:15.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.303+0000 7fc99e200700 1 -- 172.21.15.90:0/2525350861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9800b2b20 msgr2=0x7fc9800b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:15.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.303+0000 7fc99e200700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9800b2b20 0x7fc9800b5030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc998102c10 tx=0x7fc984009260 comp rx=0 tx=0).stop 2024-05-27T07:53:15.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.303+0000 7fc99e200700 1 -- 172.21.15.90:0/2525350861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105300 msgr2=0x7fc998118890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:15.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.303+0000 7fc99e200700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105300 0x7fc998118890 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00d950 tx=0x7fc98c00dc60 comp rx=0 tx=0).stop 2024-05-27T07:53:15.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.303+0000 7fc99e200700 1 -- 172.21.15.90:0/2525350861 shutdown_connections 2024-05-27T07:53:15.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.303+0000 7fc99e200700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc9800b2b20 0x7fc9800b5030 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:15.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.303+0000 7fc99e200700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99810e730 0x7fc998113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:15.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.303+0000 7fc99e200700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105300 0x7fc998118890 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00d950 tx=0x7fc98c00dc60 comp rx=0 tx=0).stop 2024-05-27T07:53:15.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.303+0000 7fc99e200700 1 --2- 172.21.15.90:0/2525350861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105c50 0x7fc998118df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:15.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.303+0000 7fc99e200700 1 -- 172.21.15.90:0/2525350861 >> 172.21.15.90:0/2525350861 conn(0x7fc998078ab0 msgr2=0x7fc998106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:15.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.306+0000 7fc99e200700 1 -- 172.21.15.90:0/2525350861 shutdown_connections 2024-05-27T07:53:15.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:15.306+0000 7fc99e200700 1 -- 172.21.15.90:0/2525350861 wait complete. 2024-05-27T07:53:15.311 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 48 2024-05-27T07:53:15.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:15 smithi129 ceph-mon[35254]: pgmap v215: 129 pgs: 129 active+clean; 2.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 12 MiB/s wr, 479 op/s 2024-05-27T07:53:15.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:15 smithi090 ceph-mon[31429]: pgmap v215: 129 pgs: 129 active+clean; 2.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 12 MiB/s wr, 479 op/s 2024-05-27T07:53:15.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:15 smithi136 ceph-mon[35324]: pgmap v215: 129 pgs: 129 active+clean; 2.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 12 MiB/s wr, 479 op/s 2024-05-27T07:53:16.231 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:53:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:16 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2525350861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:53:16.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:16 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2525350861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:53:16.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:16 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2525350861' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:53:16.874 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:17.344 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:17 smithi090 ceph-mon[31429]: pgmap v216: 129 pgs: 129 active+clean; 2.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 12 MiB/s wr, 479 op/s 2024-05-27T07:53:17.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:17 smithi129 ceph-mon[35254]: pgmap v216: 129 pgs: 129 active+clean; 2.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 12 MiB/s wr, 479 op/s 2024-05-27T07:53:17.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:17 smithi136 ceph-mon[35324]: pgmap v216: 129 pgs: 129 active+clean; 2.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 12 MiB/s wr, 479 op/s 2024-05-27T07:53:18.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 180.0 s, 519.8 tps, lat 172.251 ms stddev 253.003 2024-05-27T07:53:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.034+0000 7fb590200700 1 -- 172.21.15.90:0/742524656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb588111bc0 msgr2=0x7fb588111fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.034+0000 7fb590200700 1 --2- 172.21.15.90:0/742524656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb588111bc0 0x7fb588111fd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009990 tx=0x7fb57c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.035+0000 7fb590200700 1 -- 172.21.15.90:0/742524656 shutdown_connections 2024-05-27T07:53:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.035+0000 7fb590200700 1 --2- 172.21.15.90:0/742524656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58810b270 0x7fb5881198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.035+0000 7fb590200700 1 --2- 172.21.15.90:0/742524656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb588111bc0 0x7fb588111fd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.035+0000 7fb590200700 1 --2- 172.21.15.90:0/742524656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb58810a920 0x7fb58810ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.035+0000 7fb590200700 1 -- 172.21.15.90:0/742524656 >> 172.21.15.90:0/742524656 conn(0x7fb58806f4c0 msgr2=0x7fb58806f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.036+0000 7fb590200700 1 -- 172.21.15.90:0/742524656 shutdown_connections 2024-05-27T07:53:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.036+0000 7fb590200700 1 -- 172.21.15.90:0/742524656 wait complete. 2024-05-27T07:53:20.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.038+0000 7fb590200700 1 Processor -- start 2024-05-27T07:53:20.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.038+0000 7fb590200700 1 -- start start 2024-05-27T07:53:20.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.039+0000 7fb590200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58810a920 0x7fb5881a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.039+0000 7fb590200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58810b270 0x7fb5881a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.040+0000 7fb590200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb588111bc0 0x7fb5881a7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.040+0000 7fb590200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb58811caf0 con 0x7fb588111bc0 2024-05-27T07:53:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.040+0000 7fb590200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb58811c970 con 0x7fb58810b270 2024-05-27T07:53:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.040+0000 7fb590200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb58811cc70 con 0x7fb58810a920 2024-05-27T07:53:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.040+0000 7fb58e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb588111bc0 0x7fb5881a7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.040+0000 7fb58ce00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58810b270 0x7fb5881a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.040+0000 7fb58d800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58810a920 0x7fb5881a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.040+0000 7fb58e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb588111bc0 0x7fb5881a7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41292/0 (socket says 172.21.15.90:41292) 2024-05-27T07:53:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.040+0000 7fb58e200700 1 -- 172.21.15.90:0/2912335086 learned_addr learned my addr 172.21.15.90:0/2912335086 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.041+0000 7fb58d800700 1 -- 172.21.15.90:0/2912335086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58810b270 msgr2=0x7fb5881a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.041+0000 7fb58d800700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58810b270 0x7fb5881a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.041+0000 7fb58d800700 1 -- 172.21.15.90:0/2912335086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb588111bc0 msgr2=0x7fb5881a7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.041+0000 7fb58d800700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb588111bc0 0x7fb5881a7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.041+0000 7fb58d800700 1 -- 172.21.15.90:0/2912335086 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5881a8360 con 0x7fb58810a920 2024-05-27T07:53:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.041+0000 7fb58ce00700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58810b270 0x7fb5881a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.042+0000 7fb58d800700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58810a920 0x7fb5881a34e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb57c000c00 tx=0x7fb57c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:20.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.043+0000 7fb586000700 1 -- 172.21.15.90:0/2912335086 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00fb10 con 0x7fb58810a920 2024-05-27T07:53:20.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.043+0000 7fb586000700 1 -- 172.21.15.90:0/2912335086 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb57c004cc0 con 0x7fb58810a920 2024-05-27T07:53:20.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.043+0000 7fb590200700 1 -- 172.21.15.90:0/2912335086 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5881a85f0 con 0x7fb58810a920 2024-05-27T07:53:20.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.043+0000 7fb586000700 1 -- 172.21.15.90:0/2912335086 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c005480 con 0x7fb58810a920 2024-05-27T07:53:20.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.044+0000 7fb590200700 1 -- 172.21.15.90:0/2912335086 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5881a8a70 con 0x7fb58810a920 2024-05-27T07:53:20.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.050+0000 7fb59edebb80 1 Processor -- start 2024-05-27T07:53:20.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.050+0000 7fb59edebb80 1 -- start start 2024-05-27T07:53:20.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.050+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca36e9f0 0x5591ca40b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.050+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca282e40 0x5591ca401ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.051+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca402830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.051+0000 7fb59edebb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5591ca32fe40 con 0x5591ca402420 2024-05-27T07:53:20.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.051+0000 7fb59edebb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5591ca31f9e0 con 0x5591ca36e9f0 2024-05-27T07:53:20.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.051+0000 7fb59edebb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5591ca323a20 con 0x5591ca282e40 2024-05-27T07:53:20.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.051+0000 7fb586000700 1 -- 172.21.15.90:0/2912335086 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb57c028060 con 0x7fb58810a920 2024-05-27T07:53:20.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.051+0000 7fb58ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca402830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.051+0000 7fb58e200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca36e9f0 0x5591ca40b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.051+0000 7fb58ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca402830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41300/0 (socket says 172.21.15.90:41300) 2024-05-27T07:53:20.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.051+0000 7fb58ce00700 1 -- 172.21.15.90:0/116876171 learned_addr learned my addr 172.21.15.90:0/116876171 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:20.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.052+0000 7fb58d800700 1 --2- 172.21.15.90:0/116876171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca282e40 0x5591ca401ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.052+0000 7fb58ce00700 1 -- 172.21.15.90:0/116876171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca282e40 msgr2=0x5591ca401ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:20.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.052+0000 7fb58ce00700 1 --2- 172.21.15.90:0/116876171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca282e40 0x5591ca401ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.052+0000 7fb58ce00700 1 -- 172.21.15.90:0/116876171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca36e9f0 msgr2=0x5591ca40b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:20.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.052+0000 7fb58ce00700 1 --2- 172.21.15.90:0/116876171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca36e9f0 0x5591ca40b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.052+0000 7fb58ce00700 1 -- 172.21.15.90:0/116876171 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ca2e8420 con 0x5591ca402420 2024-05-27T07:53:20.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.052+0000 7fb58e200700 1 --2- 172.21.15.90:0/116876171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca36e9f0 0x5591ca40b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:20.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.052+0000 7fb58ce00700 1 --2- 172.21.15.90:0/116876171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca402830 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb574009d80 tx=0x7fb5740116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09d85c8c9019b85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:20.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.053+0000 7fb586000700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb5700b2bc0 0x7fb5700b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.053+0000 7fb57aa00700 1 -- 172.21.15.90:0/116876171 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574012450 con 0x5591ca402420 2024-05-27T07:53:20.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.053+0000 7fb57aa00700 1 -- 172.21.15.90:0/116876171 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb574012a80 con 0x5591ca402420 2024-05-27T07:53:20.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.053+0000 7fb57aa00700 1 -- 172.21.15.90:0/116876171 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57400f840 con 0x5591ca402420 2024-05-27T07:53:20.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.053+0000 7fb586000700 1 -- 172.21.15.90:0/2912335086 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fb57c0e33e0 con 0x7fb58810a920 2024-05-27T07:53:20.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.055+0000 7fb58e200700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb5700b2bc0 0x7fb5700b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.057+0000 7fb59edebb80 1 -- 172.21.15.90:0/116876171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 msgr2=0x5591ca402830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:20.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.057+0000 7fb58e200700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb5700b2bc0 0x7fb5700b50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb580009fd0 tx=0x7fb58000b040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:20.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.057+0000 7fb59edebb80 1 --2- 172.21.15.90:0/116876171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca402830 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fb574009d80 tx=0x7fb5740116c0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.058+0000 7fb59edebb80 1 -- 172.21.15.90:0/116876171 shutdown_connections 2024-05-27T07:53:20.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.058+0000 7fb59edebb80 1 --2- 172.21.15.90:0/116876171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca402830 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.058+0000 7fb59edebb80 1 --2- 172.21.15.90:0/116876171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca36e9f0 0x5591ca40b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.058+0000 7fb59edebb80 1 --2- 172.21.15.90:0/116876171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca282e40 0x5591ca401ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.058+0000 7fb59edebb80 1 -- 172.21.15.90:0/116876171 >> 172.21.15.90:0/116876171 conn(0x5591ca31c390 msgr2=0x5591ca3fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:20.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.058+0000 7fb59edebb80 1 -- 172.21.15.90:0/116876171 shutdown_connections 2024-05-27T07:53:20.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.058+0000 7fb59edebb80 1 -- 172.21.15.90:0/116876171 wait complete. 2024-05-27T07:53:20.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.059+0000 7fb59edebb80 1 -- start start 2024-05-27T07:53:20.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.059+0000 7fb59edebb80 1 Processor -- start 2024-05-27T07:53:20.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.060+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 0x5591ca422b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.060+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca36e9f0 0x5591ca4230b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.060+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca41cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.061+0000 7fb59edebb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5591ca225fa0 con 0x5591ca402420 2024-05-27T07:53:20.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.061+0000 7fb59edebb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5591ca2c0760 con 0x5591ca282e40 2024-05-27T07:53:20.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.061+0000 7fb59edebb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5591ca406d10 con 0x5591ca36e9f0 2024-05-27T07:53:20.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.061+0000 7fb59edebb80 10 client.0 _ll_register_callbacks cb 0x5591ca2c9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:20.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.062+0000 7fb58e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca41cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.062+0000 7fb58e200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca41cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41304/0 (socket says 172.21.15.90:41304) 2024-05-27T07:53:20.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.062+0000 7fb58e200700 1 -- 172.21.15.90:0/2768801304 learned_addr learned my addr 172.21.15.90:0/2768801304 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:20.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.062+0000 7fb58d800700 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 0x5591ca422b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.062+0000 7fb58e200700 1 -- 172.21.15.90:0/2768801304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca36e9f0 msgr2=0x5591ca4230b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:20.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.062+0000 7fb58e200700 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca36e9f0 0x5591ca4230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.062+0000 7fb58e200700 1 -- 172.21.15.90:0/2768801304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 msgr2=0x5591ca422b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:20.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.062+0000 7fb58e200700 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 0x5591ca422b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.062+0000 7fb58e200700 1 -- 172.21.15.90:0/2768801304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ca26f8d0 con 0x5591ca402420 2024-05-27T07:53:20.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.063+0000 7fb58e200700 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca41cf70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb580016a60 tx=0x7fb580004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d27d177adbe8c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:20.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.063+0000 7fb56f400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:53:20.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.064+0000 7fb56f400700 1 -- 172.21.15.90:0/2768801304 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580027030 con 0x5591ca402420 2024-05-27T07:53:20.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.064+0000 7fb59edebb80 1 -- 172.21.15.90:0/2768801304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb574012a80 con 0x5591ca402420 2024-05-27T07:53:20.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.064+0000 7fb56f400700 1 -- 172.21.15.90:0/2768801304 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fb580003a00 con 0x5591ca402420 2024-05-27T07:53:20.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.065+0000 7fb56f400700 1 -- 172.21.15.90:0/2768801304 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb580031050 con 0x5591ca402420 2024-05-27T07:53:20.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.065+0000 7fb59edebb80 10 client.16839 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:20.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.065+0000 7fb59edebb80 1 -- 172.21.15.90:0/2768801304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5591ca2f0060 con 0x5591ca402420 2024-05-27T07:53:20.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.066+0000 7fb56f400700 1 -- 172.21.15.90:0/2768801304 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb574012a80 con 0x5591ca402420 2024-05-27T07:53:20.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.067+0000 7fb56f400700 1 -- 172.21.15.90:0/2768801304 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7fb5800040b0 con 0x5591ca402420 2024-05-27T07:53:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.068+0000 7fb59edebb80 10 client.16839 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.068+0000 7fb59edebb80 10 client.16839 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:53:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.068+0000 7fb59edebb80 20 client.16839 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:20.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.068+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5591ca42ca40 0x5591ca42ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.069+0000 7fb59edebb80 4 client.16839 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:20.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.069+0000 7fb59edebb80 1 -- 172.21.15.90:0/2768801304 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591ca331d90 con 0x5591ca42ca40 2024-05-27T07:53:20.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.070+0000 7fb58d800700 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5591ca42ca40 0x5591ca42ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.071+0000 7fb58d800700 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5591ca42ca40 0x5591ca42ef20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bff88a702cb26820 server_cookie=69eea14c3b8638df in_seq=0 out_seq=0 2024-05-27T07:53:20.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.071+0000 7fb56f400700 10 client.16839 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:53:20.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.073+0000 7fb56f400700 1 -- 172.21.15.90:0/2768801304 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5591ca331d90 con 0x5591ca42ca40 2024-05-27T07:53:20.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.073+0000 7fb56f400700 10 client.16839 handle_command_reply: tid=0 2024-05-27T07:53:20.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.073+0000 7fb59edebb80 1 client.16839 shutdown 2024-05-27T07:53:20.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.074+0000 7fb59edebb80 1 -- 172.21.15.90:0/2768801304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 msgr2=0x5591ca41cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:20.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.074+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca41cf70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb580016a60 tx=0x7fb580004b40 comp rx=0 tx=0).stop 2024-05-27T07:53:20.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.074+0000 7fb59edebb80 1 -- 172.21.15.90:0/2768801304 shutdown_connections 2024-05-27T07:53:20.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.074+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5591ca42ca40 0x5591ca42ef20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.074+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca41cf70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.074+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 0x5591ca422b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.074+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2768801304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca36e9f0 0x5591ca4230b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.074+0000 7fb59edebb80 1 -- 172.21.15.90:0/2768801304 >> 172.21.15.90:0/2768801304 conn(0x5591ca31c390 msgr2=0x5591ca407c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:20.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.075+0000 7fb56f400700 0 client.16839 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:53:20.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.075+0000 7fb59edebb80 1 -- 172.21.15.90:0/2768801304 shutdown_connections 2024-05-27T07:53:20.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.075+0000 7fb59edebb80 1 -- 172.21.15.90:0/2768801304 wait complete. 2024-05-27T07:53:20.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.075+0000 7fb59edebb80 20 client.16839 trim_cache size 0 max 16384 2024-05-27T07:53:20.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.108+0000 7fb59edebb80 1 Processor -- start 2024-05-27T07:53:20.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.108+0000 7fb59edebb80 1 -- start start 2024-05-27T07:53:20.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.108+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 0x5591ca4018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.108+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca36e9f0 0x5591ca408000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.109+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca402420 0x5591ca408540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.109+0000 7fb59edebb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881b85a0 con 0x5591ca36e9f0 2024-05-27T07:53:20.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.109+0000 7fb59edebb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881b6390 con 0x5591ca282e40 2024-05-27T07:53:20.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.109+0000 7fb59edebb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881b7ed0 con 0x5591ca402420 2024-05-27T07:53:20.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.109+0000 7fb58d800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 0x5591ca4018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.109+0000 7fb58d800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 0x5591ca4018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46336/0 (socket says 172.21.15.90:46336) 2024-05-27T07:53:20.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.109+0000 7fb58d800700 1 -- 172.21.15.90:0/2391675919 learned_addr learned my addr 172.21.15.90:0/2391675919 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:20.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.109+0000 7fb58e200700 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca402420 0x5591ca408540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.110+0000 7fb58ce00700 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca36e9f0 0x5591ca408000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.110+0000 7fb58d800700 1 -- 172.21.15.90:0/2391675919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca402420 msgr2=0x5591ca408540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:20.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.110+0000 7fb58d800700 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca402420 0x5591ca408540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.110+0000 7fb58d800700 1 -- 172.21.15.90:0/2391675919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca36e9f0 msgr2=0x5591ca408000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:20.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.110+0000 7fb58d800700 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca36e9f0 0x5591ca408000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.110+0000 7fb58d800700 1 -- 172.21.15.90:0/2391675919 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ca3fde10 con 0x5591ca282e40 2024-05-27T07:53:20.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.110+0000 7fb58e200700 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca402420 0x5591ca408540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:20.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.111+0000 7fb58ce00700 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca36e9f0 0x5591ca408000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:20.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.111+0000 7fb58d800700 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 0x5591ca4018a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb57c004f90 tx=0x7fb57c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=750076bf0ee7d078 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:20.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.112+0000 7fb57a000700 1 -- 172.21.15.90:0/2391675919 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0a9a90 con 0x5591ca282e40 2024-05-27T07:53:20.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.112+0000 7fb57a000700 1 -- 172.21.15.90:0/2391675919 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb57c0e0e50 con 0x5591ca282e40 2024-05-27T07:53:20.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.112+0000 7fb57a000700 1 -- 172.21.15.90:0/2391675919 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0b30f0 con 0x5591ca282e40 2024-05-27T07:53:20.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.114+0000 7fb59edebb80 1 -- 172.21.15.90:0/2391675919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 msgr2=0x5591ca4018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:20.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.114+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 0x5591ca4018a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb57c004f90 tx=0x7fb57c009310 comp rx=0 tx=0).stop 2024-05-27T07:53:20.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.114+0000 7fb59edebb80 1 -- 172.21.15.90:0/2391675919 shutdown_connections 2024-05-27T07:53:20.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.114+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca402420 0x5591ca408540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.114+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca282e40 0x5591ca4018a0 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb57c004f90 tx=0x7fb57c009310 comp rx=0 tx=0).stop 2024-05-27T07:53:20.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.114+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2391675919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca36e9f0 0x5591ca408000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.114+0000 7fb59edebb80 1 -- 172.21.15.90:0/2391675919 >> 172.21.15.90:0/2391675919 conn(0x5591ca31c390 msgr2=0x5591ca401190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:20.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.115+0000 7fb59edebb80 1 -- 172.21.15.90:0/2391675919 shutdown_connections 2024-05-27T07:53:20.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.115+0000 7fb59edebb80 1 -- 172.21.15.90:0/2391675919 wait complete. 2024-05-27T07:53:20.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.116+0000 7fb59edebb80 1 -- start start 2024-05-27T07:53:20.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.117+0000 7fb59edebb80 1 Processor -- start 2024-05-27T07:53:20.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.118+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca36e9f0 0x5591ca428c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.118+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca4291c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.118+0000 7fb59edebb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca426090 0x5591ca4264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.118+0000 7fb59edebb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881b7130 con 0x5591ca402420 2024-05-27T07:53:20.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.118+0000 7fb59edebb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881b6a60 con 0x5591ca426090 2024-05-27T07:53:20.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.118+0000 7fb59edebb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb5881b7800 con 0x5591ca36e9f0 2024-05-27T07:53:20.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.118+0000 7fb59edebb80 10 client.0 _ll_register_callbacks cb 0x5591ca2c9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:20.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.119+0000 7fb58e200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca426090 0x5591ca4264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.119+0000 7fb58e200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca426090 0x5591ca4264a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46340/0 (socket says 172.21.15.90:46340) 2024-05-27T07:53:20.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.119+0000 7fb58e200700 1 -- 172.21.15.90:0/2352098520 learned_addr learned my addr 172.21.15.90:0/2352098520 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:20.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.119+0000 7fb58d800700 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca36e9f0 0x5591ca428c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.119+0000 7fb58ce00700 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca4291c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.120+0000 7fb58e200700 1 -- 172.21.15.90:0/2352098520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca36e9f0 msgr2=0x5591ca428c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:20.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.120+0000 7fb58e200700 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca36e9f0 0x5591ca428c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.120+0000 7fb58e200700 1 -- 172.21.15.90:0/2352098520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 msgr2=0x5591ca4291c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:20.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.120+0000 7fb58e200700 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca4291c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.120+0000 7fb58e200700 1 -- 172.21.15.90:0/2352098520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ca426ba0 con 0x5591ca426090 2024-05-27T07:53:20.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.121+0000 7fb58ce00700 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca4291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:20.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.121+0000 7fb58d800700 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca36e9f0 0x5591ca428c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:20.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.121+0000 7fb58e200700 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca426090 0x5591ca4264a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb580000f80 tx=0x7fb580004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab1c77a30775bad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:20.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.121+0000 7fb56f400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:20.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.122+0000 7fb56f400700 1 -- 172.21.15.90:0/2352098520 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580027070 con 0x5591ca426090 2024-05-27T07:53:20.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.123+0000 7fb56f400700 1 -- 172.21.15.90:0/2352098520 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fb580015b40 con 0x5591ca426090 2024-05-27T07:53:20.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.123+0000 7fb59edebb80 1 -- 172.21.15.90:0/2352098520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb57c0e0e50 con 0x5591ca426090 2024-05-27T07:53:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.124+0000 7fb56f400700 1 -- 172.21.15.90:0/2352098520 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb580015d80 con 0x5591ca426090 2024-05-27T07:53:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.124+0000 7fb59edebb80 10 client.36371 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:20.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.124+0000 7fb59edebb80 1 -- 172.21.15.90:0/2352098520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5591ca497340 con 0x5591ca426090 2024-05-27T07:53:20.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.125+0000 7fb56f400700 1 -- 172.21.15.90:0/2352098520 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb57c0e0e50 con 0x5591ca426090 2024-05-27T07:53:20.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.125+0000 7fb56f400700 1 -- 172.21.15.90:0/2352098520 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7fb58003e440 con 0x5591ca426090 2024-05-27T07:53:20.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.125+0000 7fb59edebb80 10 client.36371 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:20.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.125+0000 7fb59edebb80 10 client.36371 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:53:20.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.125+0000 7fb59edebb80 20 client.36371 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:20.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.125+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5591ca4979e0 0x5591ca499e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:20.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.125+0000 7fb59edebb80 4 client.36371 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:53:20.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.125+0000 7fb59edebb80 1 -- 172.21.15.90:0/2352098520 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591ca32b570 con 0x5591ca4979e0 2024-05-27T07:53:20.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.126+0000 7fb58d800700 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5591ca4979e0 0x5591ca499e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:20.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.127+0000 7fb58d800700 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5591ca4979e0 0x5591ca499e00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c9e969b9ce95dbb8 server_cookie=6ecff4b972596408 in_seq=0 out_seq=0 2024-05-27T07:53:20.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.127+0000 7fb56f400700 10 client.36371 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:53:20.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb56f400700 1 -- 172.21.15.90:0/2352098520 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591ca32b570 con 0x5591ca4979e0 2024-05-27T07:53:20.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb56f400700 10 client.36371 handle_command_reply: tid=0 2024-05-27T07:53:20.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb59edebb80 1 client.36371 shutdown 2024-05-27T07:53:20.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb59edebb80 1 -- 172.21.15.90:0/2352098520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca426090 msgr2=0x5591ca4264a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:20.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca426090 0x5591ca4264a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb580000f80 tx=0x7fb580004ce0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb59edebb80 1 -- 172.21.15.90:0/2352098520 shutdown_connections 2024-05-27T07:53:20.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5591ca4979e0 0x5591ca499e00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591ca426090 0x5591ca4264a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591ca36e9f0 0x5591ca428c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.131 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:53:20.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb59edebb80 1 --2- 172.21.15.90:0/2352098520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ca402420 0x5591ca4291c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb56f400700 0 client.36371 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:53:20.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.128+0000 7fb59edebb80 1 -- 172.21.15.90:0/2352098520 >> 172.21.15.90:0/2352098520 conn(0x5591ca31c390 msgr2=0x5591ca3fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:20.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.129+0000 7fb59edebb80 1 -- 172.21.15.90:0/2352098520 shutdown_connections 2024-05-27T07:53:20.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.129+0000 7fb59edebb80 1 -- 172.21.15.90:0/2352098520 wait complete. 2024-05-27T07:53:20.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.129+0000 7fb59edebb80 20 client.36371 trim_cache size 0 max 16384 2024-05-27T07:53:20.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 -- 172.21.15.90:0/2912335086 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb5700b2bc0 msgr2=0x7fb5700b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:20.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb5700b2bc0 0x7fb5700b50d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb580009fd0 tx=0x7fb58000b040 comp rx=0 tx=0).stop 2024-05-27T07:53:20.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 -- 172.21.15.90:0/2912335086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58810a920 msgr2=0x7fb5881a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:20.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58810a920 0x7fb5881a34e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb57c000c00 tx=0x7fb57c0047b0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 -- 172.21.15.90:0/2912335086 shutdown_connections 2024-05-27T07:53:20.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb588111bc0 0x7fb5881a7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58810a920 0x7fb5881a34e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb5700b2bc0 0x7fb5700b50d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 --2- 172.21.15.90:0/2912335086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb58810b270 0x7fb5881a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:20.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 -- 172.21.15.90:0/2912335086 >> 172.21.15.90:0/2912335086 conn(0x7fb58806f4c0 msgr2=0x7fb5881108c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:20.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 -- 172.21.15.90:0/2912335086 shutdown_connections 2024-05-27T07:53:20.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:20.130+0000 7fb590200700 1 -- 172.21.15.90:0/2912335086 wait complete. 2024-05-27T07:53:20.455 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:20 smithi136 ceph-mon[35324]: pgmap v217: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 13 MiB/s wr, 612 op/s 2024-05-27T07:53:20.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:20 smithi090 ceph-mon[31429]: pgmap v217: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 13 MiB/s wr, 612 op/s 2024-05-27T07:53:20.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:20 smithi129 ceph-mon[35254]: pgmap v217: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 13 MiB/s wr, 612 op/s 2024-05-27T07:53:22.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:22 smithi090 ceph-mon[31429]: pgmap v218: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 13 MiB/s wr, 613 op/s 2024-05-27T07:53:22.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:22 smithi136 ceph-mon[35324]: pgmap v218: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 13 MiB/s wr, 613 op/s 2024-05-27T07:53:22.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:22 smithi129 ceph-mon[35254]: pgmap v218: 129 pgs: 129 active+clean; 2.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 13 MiB/s wr, 613 op/s 2024-05-27T07:53:23.249 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:53:23.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 185.0 s, 500.0 tps, lat 181.341 ms stddev 265.148 2024-05-27T07:53:23.891 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:24.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.281+0000 7fab8d800640 1 -- 172.21.15.90:0/2960154683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab8810a500 msgr2=0x7fab8810a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.281+0000 7fab8d800640 1 --2- 172.21.15.90:0/2960154683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab8810a500 0x7fab8810a910 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fab78009900 tx=0x7fab7802a410 comp rx=0 tx=0).stop 2024-05-27T07:53:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.281+0000 7fab8d800640 1 -- 172.21.15.90:0/2960154683 shutdown_connections 2024-05-27T07:53:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.281+0000 7fab8d800640 1 --2- 172.21.15.90:0/2960154683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8810f540 0x7fab88116ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.281+0000 7fab8d800640 1 --2- 172.21.15.90:0/2960154683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8810ea80 0x7fab8810eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.281+0000 7fab8d800640 1 --2- 172.21.15.90:0/2960154683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab8810a500 0x7fab8810a910 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.281+0000 7fab8d800640 1 -- 172.21.15.90:0/2960154683 >> 172.21.15.90:0/2960154683 conn(0x7fab88077190 msgr2=0x7fab880775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.281+0000 7fab8d800640 1 -- 172.21.15.90:0/2960154683 shutdown_connections 2024-05-27T07:53:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.282+0000 7fab8d800640 1 -- 172.21.15.90:0/2960154683 wait complete. 2024-05-27T07:53:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.284+0000 7fab8d800640 1 Processor -- start 2024-05-27T07:53:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.284+0000 7fab8d800640 1 -- start start 2024-05-27T07:53:24.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.285+0000 7fab8d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab8810a500 0x7fab8806daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.285+0000 7fab8d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8810ea80 0x7fab8806e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.286+0000 7fab8d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8810f540 0x7fab8806e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.286+0000 7fab8d800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab8811a300 con 0x7fab8810a500 2024-05-27T07:53:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.286+0000 7fab8d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab8811a180 con 0x7fab8810f540 2024-05-27T07:53:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.286+0000 7fab8d800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab8811a480 con 0x7fab8810ea80 2024-05-27T07:53:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.286+0000 7fab86a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab8810a500 0x7fab8806daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.286+0000 7fab86a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab8810a500 0x7fab8806daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41334/0 (socket says 172.21.15.90:41334) 2024-05-27T07:53:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.286+0000 7fab86a00640 1 -- 172.21.15.90:0/2825796852 learned_addr learned my addr 172.21.15.90:0/2825796852 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.287+0000 7fab87400640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8810f540 0x7fab8806e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.287+0000 7fab86000640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8810ea80 0x7fab8806e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.287+0000 7fab86a00640 1 -- 172.21.15.90:0/2825796852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8810ea80 msgr2=0x7fab8806e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.287+0000 7fab86a00640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8810ea80 0x7fab8806e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.287+0000 7fab86a00640 1 -- 172.21.15.90:0/2825796852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8810f540 msgr2=0x7fab8806e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.287+0000 7fab86a00640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8810f540 0x7fab8806e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.287+0000 7fab86a00640 1 -- 172.21.15.90:0/2825796852 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab880724f0 con 0x7fab8810a500 2024-05-27T07:53:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.288+0000 7fab86a00640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab8810a500 0x7fab8806daf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fab78009230 tx=0x7fab780319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.288+0000 7fab7f400640 1 -- 172.21.15.90:0/2825796852 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7803f030 con 0x7fab8810a500 2024-05-27T07:53:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.289+0000 7fab7f400640 1 -- 172.21.15.90:0/2825796852 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab780043d0 con 0x7fab8810a500 2024-05-27T07:53:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.289+0000 7fab8d800640 1 -- 172.21.15.90:0/2825796852 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab88072780 con 0x7fab8810a500 2024-05-27T07:53:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.289+0000 7fab7f400640 1 -- 172.21.15.90:0/2825796852 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab78035070 con 0x7fab8810a500 2024-05-27T07:53:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.289+0000 7fab8d800640 1 -- 172.21.15.90:0/2825796852 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab88072c70 con 0x7fab8810a500 2024-05-27T07:53:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.293+0000 7fab9482f740 1 Processor -- start 2024-05-27T07:53:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.293+0000 7fab9482f740 1 -- start start 2024-05-27T07:53:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.293+0000 7fab9482f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.294+0000 7fab9482f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102ce92f0 0x55a1030b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.294+0000 7fab9482f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cf14d0 0x55a1030b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.294+0000 7fab9482f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a103025b90 con 0x55a102cf14d0 2024-05-27T07:53:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.294+0000 7fab9482f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a103015730 con 0x55a102d121b0 2024-05-27T07:53:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.294+0000 7fab9482f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a103019770 con 0x55a102ce92f0 2024-05-27T07:53:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.294+0000 7fab87400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.295+0000 7fab87400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46370/0 (socket says 172.21.15.90:46370) 2024-05-27T07:53:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.295+0000 7fab87400640 1 -- 172.21.15.90:0/886211310 learned_addr learned my addr 172.21.15.90:0/886211310 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.295+0000 7fab87400640 1 -- 172.21.15.90:0/886211310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102ce92f0 msgr2=0x55a1030b4da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:53:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.295+0000 7fab87400640 1 --2- 172.21.15.90:0/886211310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102ce92f0 0x55a1030b4da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.295+0000 7fab87400640 1 -- 172.21.15.90:0/886211310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cf14d0 msgr2=0x55a1030b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.295+0000 7fab87400640 1 --2- 172.21.15.90:0/886211310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cf14d0 0x55a1030b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.295+0000 7fab87400640 1 -- 172.21.15.90:0/886211310 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a102fae8b0 con 0x55a102d121b0 2024-05-27T07:53:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.296+0000 7fab87400640 1 --2- 172.21.15.90:0/886211310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030b4860 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab80009f50 tx=0x7fab800118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a27c50b8954865d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.296+0000 7fab73e00640 1 -- 172.21.15.90:0/886211310 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80004b20 con 0x55a102d121b0 2024-05-27T07:53:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.296+0000 7fab73e00640 1 -- 172.21.15.90:0/886211310 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab80004cc0 con 0x55a102d121b0 2024-05-27T07:53:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.297+0000 7fab73e00640 1 -- 172.21.15.90:0/886211310 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80005260 con 0x55a102d121b0 2024-05-27T07:53:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.297+0000 7fab9482f740 1 -- 172.21.15.90:0/886211310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 msgr2=0x55a1030b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.297+0000 7fab9482f740 1 --2- 172.21.15.90:0/886211310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030b4860 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab80009f50 tx=0x7fab800118d0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.298+0000 7fab9482f740 1 -- 172.21.15.90:0/886211310 shutdown_connections 2024-05-27T07:53:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.298+0000 7fab9482f740 1 --2- 172.21.15.90:0/886211310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cf14d0 0x55a1030b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.298+0000 7fab9482f740 1 --2- 172.21.15.90:0/886211310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102ce92f0 0x55a1030b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.298+0000 7fab9482f740 1 --2- 172.21.15.90:0/886211310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030b4860 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.298+0000 7fab9482f740 1 -- 172.21.15.90:0/886211310 >> 172.21.15.90:0/886211310 conn(0x55a102d06300 msgr2=0x55a1030b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.300+0000 7fab7f400640 1 -- 172.21.15.90:0/2825796852 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fab78042030 con 0x7fab8810a500 2024-05-27T07:53:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.300+0000 7fab9482f740 1 -- 172.21.15.90:0/886211310 shutdown_connections 2024-05-27T07:53:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.300+0000 7fab9482f740 1 -- 172.21.15.90:0/886211310 wait complete. 2024-05-27T07:53:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.301+0000 7fab9482f740 1 -- start start 2024-05-27T07:53:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.301+0000 7fab9482f740 1 Processor -- start 2024-05-27T07:53:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.303+0000 7fab9482f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 0x55a1030bea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.303+0000 7fab7f400640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fab6c0b2860 0x7fab6c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.303+0000 7fab9482f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030befb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.303+0000 7fab9482f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.303+0000 7fab86a00640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fab6c0b2860 0x7fab6c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.303+0000 7fab7f400640 1 -- 172.21.15.90:0/2825796852 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fab78102e30 con 0x7fab8810a500 2024-05-27T07:53:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.304+0000 7fab9482f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a102cfb790 con 0x55a102cd0d80 2024-05-27T07:53:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.304+0000 7fab9482f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a102fbdbc0 con 0x55a102d121b0 2024-05-27T07:53:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.304+0000 7fab9482f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a1030c1990 con 0x55a102cf14d0 2024-05-27T07:53:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.304+0000 7fab9482f740 10 client.0 _ll_register_callbacks cb 0x55a102f9b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.304+0000 7fab87400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 0x55a1030bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.304+0000 7fab86000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.304+0000 7fab87400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.304+0000 7fab86000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030befb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46372/0 (socket says 172.21.15.90:46372) 2024-05-27T07:53:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.305+0000 7fab86000640 1 -- 172.21.15.90:0/4014461373 learned_addr learned my addr 172.21.15.90:0/4014461373 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.305+0000 7fab86a00640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fab6c0b2860 0x7fab6c0b4d70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55a1030b2350 tx=0x7fab78009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.306+0000 7fab86000640 1 -- 172.21.15.90:0/4014461373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 msgr2=0x55a1030bea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.306+0000 7fab86000640 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 0x55a1030bea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.306+0000 7fab86000640 1 -- 172.21.15.90:0/4014461373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 msgr2=0x55a1030cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.306+0000 7fab86000640 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.306+0000 7fab86000640 1 -- 172.21.15.90:0/4014461373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a102d2a210 con 0x55a102d121b0 2024-05-27T07:53:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.306+0000 7fab87400640 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:53:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.307+0000 7fab86000640 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030befb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab74011430 tx=0x7fab740118f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=826887bc2faec1a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:24.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.307+0000 7fab70c00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.307+0000 7fab70c00640 1 -- 172.21.15.90:0/4014461373 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74004220 con 0x55a102d121b0 2024-05-27T07:53:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.308+0000 7fab70c00640 1 -- 172.21.15.90:0/4014461373 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fab740043c0 con 0x55a102d121b0 2024-05-27T07:53:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.308+0000 7fab9482f740 1 -- 172.21.15.90:0/4014461373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab80004cc0 con 0x55a102d121b0 2024-05-27T07:53:24.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.308+0000 7fab70c00640 1 -- 172.21.15.90:0/4014461373 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fab74012720 con 0x55a102d121b0 2024-05-27T07:53:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.309+0000 7fab9482f740 10 client.36383 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.309+0000 7fab9482f740 1 -- 172.21.15.90:0/4014461373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a102df6620 con 0x55a102d121b0 2024-05-27T07:53:24.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.309+0000 7fab70c00640 1 -- 172.21.15.90:0/4014461373 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab80004cc0 con 0x55a102d121b0 2024-05-27T07:53:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.311+0000 7fab70c00640 1 -- 172.21.15.90:0/4014461373 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7fab74003af0 con 0x55a102d121b0 2024-05-27T07:53:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.311+0000 7fab9482f740 10 client.36383 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.311+0000 7fab9482f740 10 client.36383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.311+0000 7fab9482f740 20 client.36383 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.311+0000 7fab9482f740 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a102ca61c0 0x55a1030d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.311+0000 7fab9482f740 4 client.36383 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:24.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.311+0000 7fab9482f740 1 -- 172.21.15.90:0/4014461373 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a103027ae0 con 0x55a102ca61c0 2024-05-27T07:53:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.312+0000 7fab87400640 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a102ca61c0 0x55a1030d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.315+0000 7fab87400640 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a102ca61c0 0x55a1030d0d90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f2cba01a61e9546 server_cookie=34bb11b0782127e8 in_seq=0 out_seq=0 2024-05-27T07:53:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.315+0000 7fab70c00640 10 client.36383 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.317+0000 7fab70c00640 1 -- 172.21.15.90:0/4014461373 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a103027ae0 con 0x55a102ca61c0 2024-05-27T07:53:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab70c00640 10 client.36383 handle_command_reply: tid=0 2024-05-27T07:53:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab9482f740 1 client.36383 shutdown 2024-05-27T07:53:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab9482f740 1 -- 172.21.15.90:0/4014461373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 msgr2=0x55a1030befb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab9482f740 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030befb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab74011430 tx=0x7fab740118f0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab9482f740 1 -- 172.21.15.90:0/4014461373 shutdown_connections 2024-05-27T07:53:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab9482f740 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a102ca61c0 0x55a1030d0d90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab9482f740 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030befb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab9482f740 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab70c00640 0 client.36383 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab9482f740 1 --2- 172.21.15.90:0/4014461373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 0x55a1030bea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.318+0000 7fab9482f740 1 -- 172.21.15.90:0/4014461373 >> 172.21.15.90:0/4014461373 conn(0x55a102d06300 msgr2=0x55a1030b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.319+0000 7fab9482f740 1 -- 172.21.15.90:0/4014461373 shutdown_connections 2024-05-27T07:53:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.319+0000 7fab9482f740 1 -- 172.21.15.90:0/4014461373 wait complete. 2024-05-27T07:53:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.319+0000 7fab9482f740 20 client.36383 trim_cache size 0 max 16384 2024-05-27T07:53:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.350+0000 7fab9482f740 1 Processor -- start 2024-05-27T07:53:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.351+0000 7fab9482f740 1 -- start start 2024-05-27T07:53:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.351+0000 7fab9482f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.351+0000 7fab9482f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 0x55a1030b0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.351+0000 7fab9482f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.351+0000 7fab9482f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab881c1c20 con 0x55a102cd0d80 2024-05-27T07:53:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.351+0000 7fab9482f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab881bfa10 con 0x55a102d121b0 2024-05-27T07:53:24.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.351+0000 7fab9482f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab881c1550 con 0x55a102cf14d0 2024-05-27T07:53:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.352+0000 7fab87400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.352+0000 7fab86000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 0x55a1030b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.352+0000 7fab87400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030c0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41354/0 (socket says 172.21.15.90:41354) 2024-05-27T07:53:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.352+0000 7fab87400640 1 -- 172.21.15.90:0/3033289675 learned_addr learned my addr 172.21.15.90:0/3033289675 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.352+0000 7fab87400640 1 --2- 172.21.15.90:0/3033289675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.353+0000 7fab87400640 1 -- 172.21.15.90:0/3033289675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 msgr2=0x55a1030b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.353+0000 7fab87400640 1 --2- 172.21.15.90:0/3033289675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 0x55a1030b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.353+0000 7fab87400640 1 -- 172.21.15.90:0/3033289675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 msgr2=0x55a1030caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.353+0000 7fab87400640 1 --2- 172.21.15.90:0/3033289675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.353+0000 7fab87400640 1 -- 172.21.15.90:0/3033289675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1030c1310 con 0x55a102cd0d80 2024-05-27T07:53:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.354+0000 7fab87400640 1 --2- 172.21.15.90:0/3033289675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030c0c60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fab80003d00 tx=0x7fab800118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e5d44f48156816 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.354+0000 7fab86000640 1 --2- 172.21.15.90:0/3033289675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 0x55a1030b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:53:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.354+0000 7fab73400640 1 -- 172.21.15.90:0/3033289675 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8000f5f0 con 0x55a102cd0d80 2024-05-27T07:53:24.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.355+0000 7fab73400640 1 -- 172.21.15.90:0/3033289675 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab8000fc20 con 0x55a102cd0d80 2024-05-27T07:53:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.355+0000 7fab73400640 1 -- 172.21.15.90:0/3033289675 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80004430 con 0x55a102cd0d80 2024-05-27T07:53:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.356+0000 7fab9482f740 1 -- 172.21.15.90:0/3033289675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 msgr2=0x55a1030c0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.356+0000 7fab9482f740 1 --2- 172.21.15.90:0/3033289675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030c0c60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fab80003d00 tx=0x7fab800118d0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.356+0000 7fab9482f740 1 -- 172.21.15.90:0/3033289675 shutdown_connections 2024-05-27T07:53:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.356+0000 7fab9482f740 1 --2- 172.21.15.90:0/3033289675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102d121b0 0x55a1030caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.357+0000 7fab9482f740 1 --2- 172.21.15.90:0/3033289675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102cf14d0 0x55a1030b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.357+0000 7fab9482f740 1 --2- 172.21.15.90:0/3033289675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cd0d80 0x55a1030c0c60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.357+0000 7fab9482f740 1 -- 172.21.15.90:0/3033289675 >> 172.21.15.90:0/3033289675 conn(0x55a102d06300 msgr2=0x55a1030bed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.357+0000 7fab9482f740 1 -- 172.21.15.90:0/3033289675 shutdown_connections 2024-05-27T07:53:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.357+0000 7fab9482f740 1 -- 172.21.15.90:0/3033289675 wait complete. 2024-05-27T07:53:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.358+0000 7fab9482f740 1 -- start start 2024-05-27T07:53:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.358+0000 7fab9482f740 1 Processor -- start 2024-05-27T07:53:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.360+0000 7fab9482f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102cd0d80 0x55a1030c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.360+0000 7fab9482f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cf14d0 0x55a1030b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.360+0000 7fab9482f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102d121b0 0x55a1030b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.360+0000 7fab9482f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab881c07b0 con 0x55a102cf14d0 2024-05-27T07:53:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.360+0000 7fab87400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102cd0d80 0x55a1030c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.360+0000 7fab87400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102cd0d80 0x55a1030c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46388/0 (socket says 172.21.15.90:46388) 2024-05-27T07:53:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.360+0000 7fab87400640 1 -- 172.21.15.90:0/3507247545 learned_addr learned my addr 172.21.15.90:0/3507247545 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.361+0000 7fab9482f740 1 -- 172.21.15.90:0/3507247545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab881c00e0 con 0x55a102cd0d80 2024-05-27T07:53:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.361+0000 7fab9482f740 1 -- 172.21.15.90:0/3507247545 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab881c0e80 con 0x55a102d121b0 2024-05-27T07:53:24.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.361+0000 7fab9482f740 10 client.0 _ll_register_callbacks cb 0x7fab74009450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.361+0000 7fab86000640 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cf14d0 0x55a1030b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.361+0000 7fab87400640 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102d121b0 0x55a1030b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.362+0000 7fab87400640 1 -- 172.21.15.90:0/3507247545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102d121b0 msgr2=0x55a1030b95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.362+0000 7fab87400640 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102d121b0 0x55a1030b95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.362+0000 7fab87400640 1 -- 172.21.15.90:0/3507247545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cf14d0 msgr2=0x55a1030b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.362+0000 7fab87400640 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cf14d0 0x55a1030b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.362+0000 7fab87400640 1 -- 172.21.15.90:0/3507247545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1030b9ae0 con 0x55a102cd0d80 2024-05-27T07:53:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.363+0000 7fab70c00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.363+0000 7fab87400640 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102cd0d80 0x55a1030c71a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab80004e00 tx=0x7fab80003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83200b05cce95222 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.364+0000 7fab70c00640 1 -- 172.21.15.90:0/3507247545 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8000f5f0 con 0x55a102cd0d80 2024-05-27T07:53:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.364+0000 7fab70c00640 1 -- 172.21.15.90:0/3507247545 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fab8000f790 con 0x55a102cd0d80 2024-05-27T07:53:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.364+0000 7fab9482f740 1 -- 172.21.15.90:0/3507247545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab8000fc20 con 0x55a102cd0d80 2024-05-27T07:53:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.365+0000 7fab70c00640 1 -- 172.21.15.90:0/3507247545 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fab8004f050 con 0x55a102cd0d80 2024-05-27T07:53:24.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.365+0000 7fab9482f740 10 client.36389 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:24.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.365+0000 7fab9482f740 1 -- 172.21.15.90:0/3507247545 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a1030c5970 con 0x55a102cd0d80 2024-05-27T07:53:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.367+0000 7fab70c00640 1 -- 172.21.15.90:0/3507247545 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab8000fc20 con 0x55a102cd0d80 2024-05-27T07:53:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.367+0000 7fab70c00640 1 -- 172.21.15.90:0/3507247545 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7fab80028450 con 0x55a102cd0d80 2024-05-27T07:53:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.367+0000 7fab9482f740 10 client.36389 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:24.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.367+0000 7fab9482f740 10 client.36389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.367+0000 7fab9482f740 20 client.36389 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.367+0000 7fab9482f740 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a1031395b0 0x55a10313b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.367+0000 7fab9482f740 4 client.36389 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:53:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.367+0000 7fab9482f740 1 -- 172.21.15.90:0/3507247545 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1030212c0 con 0x55a1031395b0 2024-05-27T07:53:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.368+0000 7fab86000640 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a1031395b0 0x55a10313b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:24.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.369+0000 7fab86000640 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a1031395b0 0x55a10313b9d0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec517fe004025b4a server_cookie=c02fb95cf9a93fdc in_seq=0 out_seq=0 2024-05-27T07:53:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:24.369+0000 7fab70c00640 10 client.36389 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:24.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:24 smithi090 ceph-mon[31429]: pgmap v219: 129 pgs: 129 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 15 MiB/s wr, 712 op/s 2024-05-27T07:53:24.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:24 smithi136 ceph-mon[35324]: pgmap v219: 129 pgs: 129 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 15 MiB/s wr, 712 op/s 2024-05-27T07:53:24.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:24 smithi129 ceph-mon[35254]: pgmap v219: 129 pgs: 129 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 15 MiB/s wr, 712 op/s 2024-05-27T07:53:26.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:26 smithi090 ceph-mon[31429]: pgmap v220: 129 pgs: 129 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 670 op/s 2024-05-27T07:53:26.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:26 smithi136 ceph-mon[35324]: pgmap v220: 129 pgs: 129 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 670 op/s 2024-05-27T07:53:26.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:26 smithi129 ceph-mon[35254]: pgmap v220: 129 pgs: 129 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 13 MiB/s wr, 670 op/s 2024-05-27T07:53:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.964+0000 7fab70c00640 1 -- 172.21.15.90:0/3507247545 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a1030212c0 con 0x55a1031395b0 2024-05-27T07:53:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.964+0000 7fab70c00640 10 client.36389 handle_command_reply: tid=0 2024-05-27T07:53:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.964+0000 7fab9482f740 1 client.36389 shutdown 2024-05-27T07:53:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.965+0000 7fab9482f740 1 -- 172.21.15.90:0/3507247545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102cd0d80 msgr2=0x55a1030c71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.965+0000 7fab9482f740 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102cd0d80 0x55a1030c71a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab80004e00 tx=0x7fab80003e90 comp rx=0 tx=0).stop 2024-05-27T07:53:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.965+0000 7fab9482f740 1 -- 172.21.15.90:0/3507247545 shutdown_connections 2024-05-27T07:53:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.965+0000 7fab9482f740 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a1031395b0 0x55a10313b9d0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.965+0000 7fab9482f740 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a102d121b0 0x55a1030b95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.965+0000 7fab9482f740 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a102cf14d0 0x55a1030b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.965+0000 7fab9482f740 1 --2- 172.21.15.90:0/3507247545 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a102cd0d80 0x55a1030c71a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.965+0000 7fab9482f740 1 -- 172.21.15.90:0/3507247545 >> 172.21.15.90:0/3507247545 conn(0x55a102d06300 msgr2=0x55a1030b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.965+0000 7fab70c00640 0 client.36389 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.966+0000 7fab9482f740 1 -- 172.21.15.90:0/3507247545 shutdown_connections 2024-05-27T07:53:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.966+0000 7fab9482f740 1 -- 172.21.15.90:0/3507247545 wait complete. 2024-05-27T07:53:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.967+0000 7fab9482f740 20 client.36389 trim_cache size 0 max 16384 2024-05-27T07:53:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:53:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:53:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:53:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.968+0000 7fab8d800640 1 -- 172.21.15.90:0/2825796852 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fab6c0b2860 msgr2=0x7fab6c0b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.968+0000 7fab8d800640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fab6c0b2860 0x7fab6c0b4d70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55a1030b2350 tx=0x7fab78009260 comp rx=0 tx=0).stop 2024-05-27T07:53:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.968+0000 7fab8d800640 1 -- 172.21.15.90:0/2825796852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab8810a500 msgr2=0x7fab8806daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.968+0000 7fab8d800640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab8810a500 0x7fab8806daf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fab78009230 tx=0x7fab780319a0 comp rx=0 tx=0).stop 2024-05-27T07:53:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.969+0000 7fab8d800640 1 -- 172.21.15.90:0/2825796852 shutdown_connections 2024-05-27T07:53:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.969+0000 7fab8d800640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fab6c0b2860 0x7fab6c0b4d70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.969+0000 7fab8d800640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab8810f540 0x7fab8806e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.969+0000 7fab8d800640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8810ea80 0x7fab8806e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.969+0000 7fab8d800640 1 --2- 172.21.15.90:0/2825796852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab8810a500 0x7fab8806daf0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:27.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.969+0000 7fab8d800640 1 -- 172.21.15.90:0/2825796852 >> 172.21.15.90:0/2825796852 conn(0x7fab88077190 msgr2=0x7fab8807a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.969+0000 7fab8d800640 1 -- 172.21.15.90:0/2825796852 shutdown_connections 2024-05-27T07:53:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:27.969+0000 7fab8d800640 1 -- 172.21.15.90:0/2825796852 wait complete. 2024-05-27T07:53:28.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:28 smithi090 ceph-mon[31429]: pgmap v221: 129 pgs: 129 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 10 MiB/s wr, 558 op/s 2024-05-27T07:53:28.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:28 smithi136 ceph-mon[35324]: pgmap v221: 129 pgs: 129 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 10 MiB/s wr, 558 op/s 2024-05-27T07:53:28.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:28 smithi129 ceph-mon[35254]: pgmap v221: 129 pgs: 129 active+clean; 2.2 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 10 MiB/s wr, 558 op/s 2024-05-27T07:53:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 190.0 s, 505.0 tps, lat 189.787 ms stddev 282.408 2024-05-27T07:53:30.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:30 smithi090 ceph-mon[31429]: pgmap v222: 129 pgs: 129 active+clean; 2.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 15 MiB/s wr, 809 op/s 2024-05-27T07:53:30.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:30 smithi136 ceph-mon[35324]: pgmap v222: 129 pgs: 129 active+clean; 2.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 15 MiB/s wr, 809 op/s 2024-05-27T07:53:30.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:30 smithi129 ceph-mon[35254]: pgmap v222: 129 pgs: 129 active+clean; 2.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 15 MiB/s wr, 809 op/s 2024-05-27T07:53:31.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.225+0000 7f3b91600700 1 -- 172.21.15.90:0/920791411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8c11d920 msgr2=0x7f3b8c11fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:31.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.225+0000 7f3b91600700 1 --2- 172.21.15.90:0/920791411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8c11d920 0x7f3b8c11fd40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400b0a0 tx=0x7f3b8400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.226+0000 7f3b91600700 1 -- 172.21.15.90:0/920791411 shutdown_connections 2024-05-27T07:53:31.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.226+0000 7f3b91600700 1 --2- 172.21.15.90:0/920791411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8c11d920 0x7f3b8c11fd40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.226+0000 7f3b91600700 1 --2- 172.21.15.90:0/920791411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c10a920 0x7f3b8c10ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.226+0000 7f3b91600700 1 --2- 172.21.15.90:0/920791411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b8c10b270 0x7f3b8c11d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.226+0000 7f3b91600700 1 -- 172.21.15.90:0/920791411 >> 172.21.15.90:0/920791411 conn(0x7f3b8c06f4c0 msgr2=0x7f3b8c06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:31.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.226+0000 7f3b91600700 1 -- 172.21.15.90:0/920791411 shutdown_connections 2024-05-27T07:53:31.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.226+0000 7f3b91600700 1 -- 172.21.15.90:0/920791411 wait complete. 2024-05-27T07:53:31.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.228+0000 7f3b91600700 1 Processor -- start 2024-05-27T07:53:31.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.228+0000 7f3b91600700 1 -- start start 2024-05-27T07:53:31.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.229+0000 7f3b91600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8c10a920 0x7f3b8c1af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.230+0000 7f3b91600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c10b270 0x7f3b8c1afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.230+0000 7f3b91600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b8c11d920 0x7f3b8c1a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.230+0000 7f3b91600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c123160 con 0x7f3b8c10b270 2024-05-27T07:53:31.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.230+0000 7f3b91600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c122fe0 con 0x7f3b8c11d920 2024-05-27T07:53:31.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.230+0000 7f3b91600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1232e0 con 0x7f3b8c10a920 2024-05-27T07:53:31.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.230+0000 7f3b8aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8c10a920 0x7f3b8c1af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.231+0000 7f3b8aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8c10a920 0x7f3b8c1af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42242/0 (socket says 172.21.15.90:42242) 2024-05-27T07:53:31.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.231+0000 7f3b8aa00700 1 -- 172.21.15.90:0/158149291 learned_addr learned my addr 172.21.15.90:0/158149291 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:31.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.231+0000 7f3b8a000700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c10b270 0x7f3b8c1afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.232+0000 7f3b8aa00700 1 -- 172.21.15.90:0/158149291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b8c11d920 msgr2=0x7f3b8c1a9800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:53:31.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.232+0000 7f3b8aa00700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b8c11d920 0x7f3b8c1a9800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.232+0000 7f3b8aa00700 1 -- 172.21.15.90:0/158149291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c10b270 msgr2=0x7f3b8c1afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:31.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.233+0000 7f3b8aa00700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c10b270 0x7f3b8c1afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.233+0000 7f3b8aa00700 1 -- 172.21.15.90:0/158149291 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c1a9f30 con 0x7f3b8c10a920 2024-05-27T07:53:31.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.234+0000 7f3b8aa00700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8c10a920 0x7f3b8c1af5c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00e9a0 tx=0x7f3b7c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:31.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.234+0000 7f3b83400700 1 -- 172.21.15.90:0/158149291 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c00cbd0 con 0x7f3b8c10a920 2024-05-27T07:53:31.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.234+0000 7f3b83400700 1 -- 172.21.15.90:0/158149291 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b7c00cd70 con 0x7f3b8c10a920 2024-05-27T07:53:31.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.234+0000 7f3b83400700 1 -- 172.21.15.90:0/158149291 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c018680 con 0x7f3b8c10a920 2024-05-27T07:53:31.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.234+0000 7f3b91600700 1 -- 172.21.15.90:0/158149291 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c1aa220 con 0x7f3b8c10a920 2024-05-27T07:53:31.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.234+0000 7f3b91600700 1 -- 172.21.15.90:0/158149291 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c1b6420 con 0x7f3b8c10a920 2024-05-27T07:53:31.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.242+0000 7f3ba0131b80 1 Processor -- start 2024-05-27T07:53:31.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.244+0000 7f3ba0131b80 1 -- start start 2024-05-27T07:53:31.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.245+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94579f0 0x5621e94ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.245+0000 7f3b83400700 1 -- 172.21.15.90:0/158149291 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3b7c010950 con 0x7f3b8c10a920 2024-05-27T07:53:31.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.246+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e936be40 0x5621e94ea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.246+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94eae70 0x5621e94f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.246+0000 7f3b8b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94579f0 0x5621e94ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.246+0000 7f3b8b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94579f0 0x5621e94ea330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48548/0 (socket says 172.21.15.90:48548) 2024-05-27T07:53:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.246+0000 7f3b8b400700 1 -- 172.21.15.90:0/3606943130 learned_addr learned my addr 172.21.15.90:0/3606943130 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.246+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3606943130 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5621e9418e40 con 0x5621e94eae70 2024-05-27T07:53:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.246+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3606943130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5621e94089e0 con 0x5621e94579f0 2024-05-27T07:53:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.246+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3606943130 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5621e940ca20 con 0x5621e936be40 2024-05-27T07:53:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.247+0000 7f3b8b400700 1 -- 172.21.15.90:0/3606943130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e936be40 msgr2=0x5621e94ea870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:53:31.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.247+0000 7f3b8b400700 1 --2- 172.21.15.90:0/3606943130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e936be40 0x5621e94ea870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.247+0000 7f3b8b400700 1 -- 172.21.15.90:0/3606943130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94eae70 msgr2=0x5621e94f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:31.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.247+0000 7f3b8b400700 1 --2- 172.21.15.90:0/3606943130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94eae70 0x5621e94f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.247+0000 7f3b8b400700 1 -- 172.21.15.90:0/3606943130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e93d1420 con 0x5621e94579f0 2024-05-27T07:53:31.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.248+0000 7f3b8b400700 1 --2- 172.21.15.90:0/3606943130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94579f0 0x5621e94ea330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b84006070 tx=0x7f3b84009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ec6381f9410223 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:31.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.248+0000 7f3b77e00700 1 -- 172.21.15.90:0/3606943130 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84009450 con 0x5621e94579f0 2024-05-27T07:53:31.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.249+0000 7f3b83400700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3b700b2a70 0x7f3b700b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.249+0000 7f3b77e00700 1 -- 172.21.15.90:0/3606943130 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b8400bcc0 con 0x5621e94579f0 2024-05-27T07:53:31.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.249+0000 7f3b77e00700 1 -- 172.21.15.90:0/3606943130 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b84007c90 con 0x5621e94579f0 2024-05-27T07:53:31.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.251+0000 7f3b83400700 1 -- 172.21.15.90:0/158149291 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f3b7c0e2b00 con 0x7f3b8c10a920 2024-05-27T07:53:31.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.251+0000 7f3b8aa00700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3b700b2a70 0x7f3b700b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.251+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3606943130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94579f0 msgr2=0x5621e94ea330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:31.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.251+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3606943130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94579f0 0x5621e94ea330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b84006070 tx=0x7f3b84009d80 comp rx=0 tx=0).stop 2024-05-27T07:53:31.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.252+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3606943130 shutdown_connections 2024-05-27T07:53:31.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.252+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3606943130 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94eae70 0x5621e94f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.252+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3606943130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94579f0 0x5621e94ea330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.252+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3606943130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e936be40 0x5621e94ea870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.252+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3606943130 >> 172.21.15.90:0/3606943130 conn(0x5621e9405390 msgr2=0x5621e94e64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:31.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.253+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3606943130 shutdown_connections 2024-05-27T07:53:31.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.253+0000 7f3b8aa00700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3b700b2a70 0x7f3b700b4f80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5621e94e7a20 tx=0x7f3b7c091040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:31.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.253+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3606943130 wait complete. 2024-05-27T07:53:31.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.255+0000 7f3ba0131b80 1 -- start start 2024-05-27T07:53:31.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.255+0000 7f3ba0131b80 1 Processor -- start 2024-05-27T07:53:31.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.256+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e94f4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.256+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94579f0 0x5621e94f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.256+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94eae70 0x5621e94f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.256+0000 7f3ba0131b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5621e930efa0 con 0x5621e94579f0 2024-05-27T07:53:31.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.256+0000 7f3ba0131b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5621e93a9760 con 0x5621e936be40 2024-05-27T07:53:31.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.256+0000 7f3ba0131b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5621e94f7b20 con 0x5621e94eae70 2024-05-27T07:53:31.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.256+0000 7f3ba0131b80 10 client.0 _ll_register_callbacks cb 0x5621e93b2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:31.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.257+0000 7f3b8b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94eae70 0x5621e94f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.257+0000 7f3b8b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e94f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.257+0000 7f3b8b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e94f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48558/0 (socket says 172.21.15.90:48558) 2024-05-27T07:53:31.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.257+0000 7f3b8b400700 1 -- 172.21.15.90:0/3374025549 learned_addr learned my addr 172.21.15.90:0/3374025549 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:31.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.258+0000 7f3b8a000700 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94579f0 0x5621e94f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.258+0000 7f3b8b400700 1 -- 172.21.15.90:0/3374025549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94eae70 msgr2=0x5621e94f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.258+0000 7f3b8b400700 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94eae70 0x5621e94f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.259+0000 7f3b8b400700 1 -- 172.21.15.90:0/3374025549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94579f0 msgr2=0x5621e94f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.259+0000 7f3b8b400700 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94579f0 0x5621e94f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.259+0000 7f3b8b400700 1 -- 172.21.15.90:0/3374025549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e93588d0 con 0x5621e936be40 2024-05-27T07:53:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.259+0000 7f3b8b400700 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e94f4630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3b84007830 tx=0x7f3b840099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a39c8662e405f4fc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.259+0000 7f3b74c00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:31.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.261+0000 7f3b74c00700 1 -- 172.21.15.90:0/3374025549 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8401d470 con 0x5621e936be40 2024-05-27T07:53:31.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.261+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3374025549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b8400bcc0 con 0x5621e936be40 2024-05-27T07:53:31.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.261+0000 7f3b74c00700 1 -- 172.21.15.90:0/3374025549 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f3b84022040 con 0x5621e936be40 2024-05-27T07:53:31.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.262+0000 7f3b74c00700 1 -- 172.21.15.90:0/3374025549 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f3b8401d610 con 0x5621e936be40 2024-05-27T07:53:31.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.263+0000 7f3ba0131b80 10 client.36398 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:31.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.263+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3374025549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5621e93d9060 con 0x5621e936be40 2024-05-27T07:53:31.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.263+0000 7f3b74c00700 1 -- 172.21.15.90:0/3374025549 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b8400bcc0 con 0x5621e936be40 2024-05-27T07:53:31.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.264+0000 7f3b74c00700 1 -- 172.21.15.90:0/3374025549 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7f3b84011070 con 0x5621e936be40 2024-05-27T07:53:31.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.264+0000 7f3ba0131b80 10 client.36398 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:31.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.264+0000 7f3ba0131b80 10 client.36398 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:53:31.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.264+0000 7f3ba0131b80 20 client.36398 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:31.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.264+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5621e95048f0 0x5621e9506d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.264+0000 7f3ba0131b80 4 client.36398 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:31.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.264+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3374025549 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621e941ad90 con 0x5621e95048f0 2024-05-27T07:53:31.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.266+0000 7f3b8a000700 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5621e95048f0 0x5621e9506d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.267+0000 7f3b8a000700 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5621e95048f0 0x5621e9506d10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c114175117c7b52 server_cookie=582013ef637ab2b9 in_seq=0 out_seq=0 2024-05-27T07:53:31.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.268+0000 7f3b74c00700 10 client.36398 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:53:31.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3b74c00700 1 -- 172.21.15.90:0/3374025549 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5621e941ad90 con 0x5621e95048f0 2024-05-27T07:53:31.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3b74c00700 10 client.36398 handle_command_reply: tid=0 2024-05-27T07:53:31.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3ba0131b80 1 client.36398 shutdown 2024-05-27T07:53:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3374025549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 msgr2=0x5621e94f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e94f4630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3b84007830 tx=0x7f3b840099e0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3374025549 shutdown_connections 2024-05-27T07:53:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5621e95048f0 0x5621e9506d10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94eae70 0x5621e94f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e94f4630 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3374025549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94579f0 0x5621e94f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3b74c00700 0 client.36398 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:53:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.269+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3374025549 >> 172.21.15.90:0/3374025549 conn(0x5621e9405390 msgr2=0x5621e94ea310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:31.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.270+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3374025549 shutdown_connections 2024-05-27T07:53:31.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.271+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3374025549 wait complete. 2024-05-27T07:53:31.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.271+0000 7f3ba0131b80 20 client.36398 trim_cache size 0 max 16384 2024-05-27T07:53:31.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.298+0000 7f3ba0131b80 1 Processor -- start 2024-05-27T07:53:31.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.298+0000 7f3ba0131b80 1 -- start start 2024-05-27T07:53:31.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.299+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e936be40 0x5621e9500dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.299+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e94f1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.299+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94eae70 0x5621e94f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.299+0000 7f3ba0131b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1c0dc0 con 0x5621e936be40 2024-05-27T07:53:31.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.299+0000 7f3ba0131b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1bebb0 con 0x5621e94eae70 2024-05-27T07:53:31.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.299+0000 7f3ba0131b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1c06f0 con 0x5621e94579f0 2024-05-27T07:53:31.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.300+0000 7f3b8a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e94f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.300+0000 7f3b8a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e94f1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42276/0 (socket says 172.21.15.90:42276) 2024-05-27T07:53:31.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.300+0000 7f3b8a000700 1 -- 172.21.15.90:0/3152386512 learned_addr learned my addr 172.21.15.90:0/3152386512 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:31.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.300+0000 7f3b8b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e936be40 0x5621e9500dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.301+0000 7f3b8b400700 1 --2- 172.21.15.90:0/3152386512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94eae70 0x5621e94f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.302+0000 7f3b8a000700 1 -- 172.21.15.90:0/3152386512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94eae70 msgr2=0x5621e94f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:31.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.302+0000 7f3b8a000700 1 --2- 172.21.15.90:0/3152386512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94eae70 0x5621e94f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.302+0000 7f3b8a000700 1 -- 172.21.15.90:0/3152386512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e936be40 msgr2=0x5621e9500dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:31.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.302+0000 7f3b8a000700 1 --2- 172.21.15.90:0/3152386512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e936be40 0x5621e9500dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.302+0000 7f3b8a000700 1 -- 172.21.15.90:0/3152386512 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e94e7000 con 0x5621e94579f0 2024-05-27T07:53:31.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.303+0000 7f3b8a000700 1 --2- 172.21.15.90:0/3152386512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e94f1890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5621e94e5d20 tx=0x7f3b780058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a3c29c35efc679 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:31.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.304+0000 7f3b77400700 1 -- 172.21.15.90:0/3152386512 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78012dd0 con 0x5621e94579f0 2024-05-27T07:53:31.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.304+0000 7f3b77400700 1 -- 172.21.15.90:0/3152386512 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b78020070 con 0x5621e94579f0 2024-05-27T07:53:31.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.304+0000 7f3b77400700 1 -- 172.21.15.90:0/3152386512 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78003cb0 con 0x5621e94579f0 2024-05-27T07:53:31.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.305+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3152386512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 msgr2=0x5621e94f1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:31.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.305+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3152386512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e94f1890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5621e94e5d20 tx=0x7f3b780058e0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.305+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3152386512 shutdown_connections 2024-05-27T07:53:31.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.305+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3152386512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e94eae70 0x5621e94f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.305+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3152386512 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e936be40 0x5621e9500dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.305+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/3152386512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e94f1890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.305+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3152386512 >> 172.21.15.90:0/3152386512 conn(0x5621e9405390 msgr2=0x5621e94ea720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:31.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.306+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3152386512 shutdown_connections 2024-05-27T07:53:31.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.306+0000 7f3ba0131b80 1 -- 172.21.15.90:0/3152386512 wait complete. 2024-05-27T07:53:31.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.307+0000 7f3ba0131b80 1 -- start start 2024-05-27T07:53:31.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.307+0000 7f3ba0131b80 1 Processor -- start 2024-05-27T07:53:31.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.309+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e950a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.309+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e950a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.309+0000 7f3ba0131b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94eae70 0x5621e9507490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.309+0000 7f3ba0131b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1bf950 con 0x5621e94eae70 2024-05-27T07:53:31.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.309+0000 7f3ba0131b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1bf280 con 0x5621e936be40 2024-05-27T07:53:31.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.309+0000 7f3ba0131b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c1c0020 con 0x5621e94579f0 2024-05-27T07:53:31.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.309+0000 7f3ba0131b80 10 client.0 _ll_register_callbacks cb 0x5621e93b2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:31.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.310+0000 7f3b8b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e950a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.310+0000 7f3b8b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e950a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48570/0 (socket says 172.21.15.90:48570) 2024-05-27T07:53:31.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.310+0000 7f3b8b400700 1 -- 172.21.15.90:0/1704671932 learned_addr learned my addr 172.21.15.90:0/1704671932 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:31.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.310+0000 7f3b8b400700 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94eae70 0x5621e9507490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.311+0000 7f3b8a000700 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e950a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.311+0000 7f3b8b400700 1 -- 172.21.15.90:0/1704671932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 msgr2=0x5621e950a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:31.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.311+0000 7f3b8b400700 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e950a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.311+0000 7f3b8b400700 1 -- 172.21.15.90:0/1704671932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 msgr2=0x5621e950a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:31.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.311+0000 7f3b8b400700 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e950a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.311+0000 7f3b8b400700 1 -- 172.21.15.90:0/1704671932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e9507c30 con 0x5621e94eae70 2024-05-27T07:53:31.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.312+0000 7f3b8a000700 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e950a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:31.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.312+0000 7f3b8b400700 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94eae70 0x5621e9507490 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400b780 tx=0x7f3b84008fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0e5f4f02eda846 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:31.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.313+0000 7f3b74c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:53:31.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.314+0000 7f3b74c00700 1 -- 172.21.15.90:0/1704671932 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8401d470 con 0x5621e94eae70 2024-05-27T07:53:31.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.314+0000 7f3ba0131b80 1 -- 172.21.15.90:0/1704671932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b78020070 con 0x5621e94eae70 2024-05-27T07:53:31.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.314+0000 7f3b74c00700 1 -- 172.21.15.90:0/1704671932 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f3b8400f040 con 0x5621e94eae70 2024-05-27T07:53:31.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.315+0000 7f3b74c00700 1 -- 172.21.15.90:0/1704671932 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f3b84003a00 con 0x5621e94eae70 2024-05-27T07:53:31.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.316+0000 7f3ba0131b80 10 client.16872 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:31.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.316+0000 7f3ba0131b80 1 -- 172.21.15.90:0/1704671932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5621e956ec40 con 0x5621e94eae70 2024-05-27T07:53:31.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.317+0000 7f3b74c00700 1 -- 172.21.15.90:0/1704671932 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b78020070 con 0x5621e94eae70 2024-05-27T07:53:31.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.317+0000 7f3b74c00700 1 -- 172.21.15.90:0/1704671932 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7f3b84011040 con 0x5621e94eae70 2024-05-27T07:53:31.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.317+0000 7f3ba0131b80 10 client.16872 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.317+0000 7f3ba0131b80 10 client.16872 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:53:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.317+0000 7f3ba0131b80 20 client.16872 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.317+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5621e956f1e0 0x5621e9571600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.317+0000 7f3ba0131b80 4 client.16872 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:53:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.317+0000 7f3ba0131b80 1 -- 172.21.15.90:0/1704671932 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621e9414570 con 0x5621e956f1e0 2024-05-27T07:53:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.318+0000 7f3b8a000700 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5621e956f1e0 0x5621e9571600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.320+0000 7f3b8a000700 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5621e956f1e0 0x5621e9571600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=987a1ce09015a257 server_cookie=14d02d878b79d6ab in_seq=0 out_seq=0 2024-05-27T07:53:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.320+0000 7f3b74c00700 10 client.16872 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:53:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.321+0000 7f3b74c00700 1 -- 172.21.15.90:0/1704671932 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621e9414570 con 0x5621e956f1e0 2024-05-27T07:53:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.321+0000 7f3b74c00700 10 client.16872 handle_command_reply: tid=0 2024-05-27T07:53:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.321+0000 7f3ba0131b80 1 client.16872 shutdown 2024-05-27T07:53:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.323+0000 7f3ba0131b80 1 -- 172.21.15.90:0/1704671932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94eae70 msgr2=0x5621e9507490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.323+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94eae70 0x5621e9507490 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400b780 tx=0x7f3b84008fb0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.323+0000 7f3ba0131b80 1 -- 172.21.15.90:0/1704671932 shutdown_connections 2024-05-27T07:53:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.323+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5621e956f1e0 0x5621e9571600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.323+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5621e94eae70 0x5621e9507490 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.323+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621e936be40 0x5621e950a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.323+0000 7f3ba0131b80 1 --2- 172.21.15.90:0/1704671932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621e94579f0 0x5621e950a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.323+0000 7f3ba0131b80 1 -- 172.21.15.90:0/1704671932 >> 172.21.15.90:0/1704671932 conn(0x5621e9405390 msgr2=0x5621e94ea720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.324+0000 7f3b74c00700 0 client.16872 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:53:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.324+0000 7f3ba0131b80 1 -- 172.21.15.90:0/1704671932 shutdown_connections 2024-05-27T07:53:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.324+0000 7f3ba0131b80 1 -- 172.21.15.90:0/1704671932 wait complete. 2024-05-27T07:53:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.324+0000 7f3ba0131b80 20 client.16872 trim_cache size 0 max 16384 2024-05-27T07:53:31.326 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:53:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.325+0000 7f3b91600700 1 -- 172.21.15.90:0/158149291 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3b700b2a70 msgr2=0x7f3b700b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.325+0000 7f3b91600700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3b700b2a70 0x7f3b700b4f80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5621e94e7a20 tx=0x7f3b7c091040 comp rx=0 tx=0).stop 2024-05-27T07:53:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.326+0000 7f3b91600700 1 -- 172.21.15.90:0/158149291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8c10a920 msgr2=0x7f3b8c1af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.326+0000 7f3b91600700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8c10a920 0x7f3b8c1af5c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00e9a0 tx=0x7f3b7c00ed60 comp rx=0 tx=0).stop 2024-05-27T07:53:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.326+0000 7f3b91600700 1 -- 172.21.15.90:0/158149291 shutdown_connections 2024-05-27T07:53:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.326+0000 7f3b91600700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3b700b2a70 0x7f3b700b4f80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.326+0000 7f3b91600700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b8c11d920 0x7f3b8c1a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.326+0000 7f3b91600700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8c10a920 0x7f3b8c1af5c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.326+0000 7f3b91600700 1 --2- 172.21.15.90:0/158149291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b8c10b270 0x7f3b8c1afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.326+0000 7f3b91600700 1 -- 172.21.15.90:0/158149291 >> 172.21.15.90:0/158149291 conn(0x7f3b8c06f4c0 msgr2=0x7f3b8c11b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.326+0000 7f3b91600700 1 -- 172.21.15.90:0/158149291 shutdown_connections 2024-05-27T07:53:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:31.326+0000 7f3b91600700 1 -- 172.21.15.90:0/158149291 wait complete. 2024-05-27T07:53:32.271 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:53:32.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:32 smithi090 ceph-mon[31429]: pgmap v223: 129 pgs: 129 active+clean; 2.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 9.7 MiB/s wr, 525 op/s 2024-05-27T07:53:32.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:32 smithi136 ceph-mon[35324]: pgmap v223: 129 pgs: 129 active+clean; 2.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 9.7 MiB/s wr, 525 op/s 2024-05-27T07:53:32.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:32 smithi129 ceph-mon[35254]: pgmap v223: 129 pgs: 129 active+clean; 2.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 9.7 MiB/s wr, 525 op/s 2024-05-27T07:53:32.877 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:33.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 195.0 s, 738.3 tps, lat 168.088 ms stddev 289.976 2024-05-27T07:53:34.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:34 smithi090 ceph-mon[31429]: pgmap v224: 129 pgs: 129 active+clean; 2.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 14 MiB/s wr, 676 op/s 2024-05-27T07:53:34.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:34 smithi136 ceph-mon[35324]: pgmap v224: 129 pgs: 129 active+clean; 2.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 14 MiB/s wr, 676 op/s 2024-05-27T07:53:34.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:34 smithi129 ceph-mon[35254]: pgmap v224: 129 pgs: 129 active+clean; 2.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 14 MiB/s wr, 676 op/s 2024-05-27T07:53:35.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.380+0000 7f6a7c000700 1 -- 172.21.15.90:0/3794756654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a741034b0 msgr2=0x7f6a741038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:35.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.380+0000 7f6a7c000700 1 --2- 172.21.15.90:0/3794756654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a741034b0 0x7f6a741038c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800b0d0 tx=0x7f6a6800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.381+0000 7f6a7c000700 1 -- 172.21.15.90:0/3794756654 shutdown_connections 2024-05-27T07:53:35.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.381+0000 7f6a7c000700 1 --2- 172.21.15.90:0/3794756654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a74108730 0x7f6a74110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.381+0000 7f6a7c000700 1 --2- 172.21.15.90:0/3794756654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a741034b0 0x7f6a741038c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.381+0000 7f6a7c000700 1 --2- 172.21.15.90:0/3794756654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a74107de0 0x7f6a741081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.381+0000 7f6a7c000700 1 -- 172.21.15.90:0/3794756654 >> 172.21.15.90:0/3794756654 conn(0x7f6a740fedb0 msgr2=0x7f6a741011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:35.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.381+0000 7f6a7c000700 1 -- 172.21.15.90:0/3794756654 shutdown_connections 2024-05-27T07:53:35.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.383+0000 7f6a7c000700 1 -- 172.21.15.90:0/3794756654 wait complete. 2024-05-27T07:53:35.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.386+0000 7f6a7c000700 1 Processor -- start 2024-05-27T07:53:35.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.386+0000 7f6a7c000700 1 -- start start 2024-05-27T07:53:35.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.387+0000 7f6a7c000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a741034b0 0x7f6a7410d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.387+0000 7f6a7c000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a74107de0 0x7f6a7410d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.388+0000 7f6a7c000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a74108730 0x7f6a7410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.388+0000 7f6a7c000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74112190 con 0x7f6a741034b0 2024-05-27T07:53:35.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.388+0000 7f6a7c000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74112010 con 0x7f6a74107de0 2024-05-27T07:53:35.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.388+0000 7f6a7c000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a74112310 con 0x7f6a74108730 2024-05-27T07:53:35.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.388+0000 7f6a78c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a74107de0 0x7f6a7410d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.388+0000 7f6a78c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a74107de0 0x7f6a7410d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48584/0 (socket says 172.21.15.90:48584) 2024-05-27T07:53:35.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.388+0000 7f6a78c00700 1 -- 172.21.15.90:0/1950299496 learned_addr learned my addr 172.21.15.90:0/1950299496 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:35.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.389+0000 7f6a78c00700 1 -- 172.21.15.90:0/1950299496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a74108730 msgr2=0x7f6a7410def0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:53:35.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.389+0000 7f6a78c00700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a74108730 0x7f6a7410def0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.389+0000 7f6a78c00700 1 -- 172.21.15.90:0/1950299496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a741034b0 msgr2=0x7f6a7410d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:35.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.389+0000 7f6a78c00700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a741034b0 0x7f6a7410d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.389+0000 7f6a78c00700 1 -- 172.21.15.90:0/1950299496 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a741ad410 con 0x7f6a74107de0 2024-05-27T07:53:35.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.389+0000 7f6a78c00700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a74107de0 0x7f6a7410d9b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c9e0 tx=0x7f6a6000cda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:35.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.390+0000 7f6a72000700 1 -- 172.21.15.90:0/1950299496 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60007aa0 con 0x7f6a74107de0 2024-05-27T07:53:35.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.390+0000 7f6a72000700 1 -- 172.21.15.90:0/1950299496 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a60007c40 con 0x7f6a74107de0 2024-05-27T07:53:35.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.390+0000 7f6a7c000700 1 -- 172.21.15.90:0/1950299496 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a741ad700 con 0x7f6a74107de0 2024-05-27T07:53:35.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.390+0000 7f6a7c000700 1 -- 172.21.15.90:0/1950299496 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a741adc90 con 0x7f6a74107de0 2024-05-27T07:53:35.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.390+0000 7f6a72000700 1 -- 172.21.15.90:0/1950299496 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60018660 con 0x7f6a74107de0 2024-05-27T07:53:35.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.397+0000 7f6a72000700 1 -- 172.21.15.90:0/1950299496 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6a60018800 con 0x7f6a74107de0 2024-05-27T07:53:35.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.397+0000 7f6a8aab7b80 1 Processor -- start 2024-05-27T07:53:35.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.397+0000 7f6a8aab7b80 1 -- start start 2024-05-27T07:53:35.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.399+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1bbf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.399+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 0x5622f1bbf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.399+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1bbfe70 0x5622f1bcc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.399+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5622f1aede40 con 0x5622f1bbfe70 2024-05-27T07:53:35.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.399+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5622f1add9e0 con 0x5622f1a40e40 2024-05-27T07:53:35.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.399+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5622f1ae1a20 con 0x5622f1b2c9f0 2024-05-27T07:53:35.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.400+0000 7f6a7a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1bbf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.400+0000 7f6a78c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1bbfe70 0x5622f1bcc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.400+0000 7f6a7a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1bbf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42308/0 (socket says 172.21.15.90:42308) 2024-05-27T07:53:35.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.400+0000 7f6a7a000700 1 -- 172.21.15.90:0/1798845551 learned_addr learned my addr 172.21.15.90:0/1798845551 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:35.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.401+0000 7f6a72000700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6a5c0b2af0 0x7f6a5c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.401+0000 7f6a72000700 1 -- 172.21.15.90:0/1950299496 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f6a600e28a0 con 0x7f6a74107de0 2024-05-27T07:53:35.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.402+0000 7f6a79600700 1 --2- 172.21.15.90:0/1798845551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 0x5622f1bbf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.402+0000 7f6a7a000700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6a5c0b2af0 0x7f6a5c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.403+0000 7f6a7a000700 1 -- 172.21.15.90:0/1798845551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 msgr2=0x5622f1bbf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:35.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.403+0000 7f6a7a000700 1 --2- 172.21.15.90:0/1798845551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 0x5622f1bbf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.403+0000 7f6a7a000700 1 -- 172.21.15.90:0/1798845551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1bbfe70 msgr2=0x5622f1bcc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:35.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.403+0000 7f6a7a000700 1 --2- 172.21.15.90:0/1798845551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1bbfe70 0x5622f1bcc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.404+0000 7f6a7a000700 1 -- 172.21.15.90:0/1798845551 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f1aa6420 con 0x5622f1b2c9f0 2024-05-27T07:53:35.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.404+0000 7f6a79600700 1 --2- 172.21.15.90:0/1798845551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 0x5622f1bbf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:35.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.404+0000 7f6a78c00700 1 --2- 172.21.15.90:0/1798845551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1bbfe70 0x5622f1bcc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:53:35.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.405+0000 7f6a7a000700 1 --2- 172.21.15.90:0/1798845551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1bbf330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009e50 tx=0x7f6a6c0126d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3947f22c5ff230c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:35.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.406+0000 7f6a66a00700 1 -- 172.21.15.90:0/1798845551 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00ed60 con 0x5622f1b2c9f0 2024-05-27T07:53:35.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.406+0000 7f6a66a00700 1 -- 172.21.15.90:0/1798845551 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a6c020070 con 0x5622f1b2c9f0 2024-05-27T07:53:35.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.406+0000 7f6a66a00700 1 -- 172.21.15.90:0/1798845551 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0233b0 con 0x5622f1b2c9f0 2024-05-27T07:53:35.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.410+0000 7f6a7a000700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6a5c0b2af0 0x7f6a5c0b5000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c01f000 tx=0x7f6a6c019000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:35.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.411+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/1798845551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 msgr2=0x5622f1bbf330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:35.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.411+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/1798845551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1bbf330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009e50 tx=0x7f6a6c0126d0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.412+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/1798845551 shutdown_connections 2024-05-27T07:53:35.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.412+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/1798845551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1bbfe70 0x5622f1bcc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.412+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/1798845551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1bbf330 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.412+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/1798845551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 0x5622f1bbf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.412+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/1798845551 >> 172.21.15.90:0/1798845551 conn(0x5622f1ada390 msgr2=0x5622f1bbb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:35.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.412+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/1798845551 shutdown_connections 2024-05-27T07:53:35.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.412+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/1798845551 wait complete. 2024-05-27T07:53:35.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.413+0000 7f6a8aab7b80 1 -- start start 2024-05-27T07:53:35.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.414+0000 7f6a8aab7b80 1 Processor -- start 2024-05-27T07:53:35.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.414+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 0x5622f1bc95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.414+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 0x5622f1bc9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.414+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1bbfe70 0x5622f1bca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.414+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5622f19e3fa0 con 0x5622f1b2c9f0 2024-05-27T07:53:35.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.414+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5622f1a7e760 con 0x5622f1a40e40 2024-05-27T07:53:35.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.414+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5622f1bccb80 con 0x5622f1bbfe70 2024-05-27T07:53:35.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.414+0000 7f6a8aab7b80 10 client.0 _ll_register_callbacks cb 0x5622f1a87c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:35.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.415+0000 7f6a7a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1bbfe70 0x5622f1bca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.415+0000 7f6a78c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 0x5622f1bc9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.415+0000 7f6a78c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 0x5622f1bc9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52878/0 (socket says 172.21.15.90:52878) 2024-05-27T07:53:35.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.415+0000 7f6a78c00700 1 -- 172.21.15.90:0/3454925699 learned_addr learned my addr 172.21.15.90:0/3454925699 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:35.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.416+0000 7f6a79600700 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 0x5622f1bc95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.416+0000 7f6a78c00700 1 -- 172.21.15.90:0/3454925699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1bbfe70 msgr2=0x5622f1bca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:35.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.416+0000 7f6a78c00700 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1bbfe70 0x5622f1bca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.416+0000 7f6a78c00700 1 -- 172.21.15.90:0/3454925699 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 msgr2=0x5622f1bc95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:35.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.416+0000 7f6a78c00700 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 0x5622f1bc95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.416+0000 7f6a78c00700 1 -- 172.21.15.90:0/3454925699 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f1a2d8d0 con 0x5622f1b2c9f0 2024-05-27T07:53:35.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.416+0000 7f6a79600700 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 0x5622f1bc95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:35.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.416+0000 7f6a78c00700 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 0x5622f1bc9b30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6a600a69c0 tx=0x7f6a600a91d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cba3119b4f2ef0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:35.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.417+0000 7f6a5b400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:53:35.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.417+0000 7f6a5b400700 1 -- 172.21.15.90:0/3454925699 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a600a9750 con 0x5622f1b2c9f0 2024-05-27T07:53:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.417+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/3454925699 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a6c020070 con 0x5622f1b2c9f0 2024-05-27T07:53:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.418+0000 7f6a5b400700 1 -- 172.21.15.90:0/3454925699 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f6a600a5d40 con 0x5622f1b2c9f0 2024-05-27T07:53:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.418+0000 7f6a5b400700 1 -- 172.21.15.90:0/3454925699 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6a600adb00 con 0x5622f1b2c9f0 2024-05-27T07:53:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.418+0000 7f6a8aab7b80 10 client.16881 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.418+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/3454925699 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5622f1aae060 con 0x5622f1b2c9f0 2024-05-27T07:53:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.419+0000 7f6a5b400700 1 -- 172.21.15.90:0/3454925699 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a6c020070 con 0x5622f1b2c9f0 2024-05-27T07:53:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.419+0000 7f6a5b400700 1 -- 172.21.15.90:0/3454925699 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7f6a60020030 con 0x5622f1b2c9f0 2024-05-27T07:53:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.419+0000 7f6a8aab7b80 10 client.16881 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.419+0000 7f6a8aab7b80 10 client.16881 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:53:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.419+0000 7f6a8aab7b80 20 client.16881 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:35.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.419+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5622f1bd9b50 0x5622f1bdbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.419+0000 7f6a8aab7b80 4 client.16881 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:35.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.419+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/3454925699 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622f1aefd90 con 0x5622f1bd9b50 2024-05-27T07:53:35.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.421+0000 7f6a79600700 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5622f1bd9b50 0x5622f1bdbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.424+0000 7f6a5b400700 10 client.16881 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:53:35.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.424+0000 7f6a79600700 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5622f1bd9b50 0x5622f1bdbf70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0e1a26697572e59 server_cookie=be0d3889b3c3c607 in_seq=0 out_seq=0 2024-05-27T07:53:35.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.429+0000 7f6a5b400700 1 -- 172.21.15.90:0/3454925699 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5622f1aefd90 con 0x5622f1bd9b50 2024-05-27T07:53:35.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.429+0000 7f6a5b400700 10 client.16881 handle_command_reply: tid=0 2024-05-27T07:53:35.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.429+0000 7f6a8aab7b80 1 client.16881 shutdown 2024-05-27T07:53:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.430+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/3454925699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 msgr2=0x5622f1bc9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.430+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 0x5622f1bc9b30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6a600a69c0 tx=0x7f6a600a91d0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.430+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/3454925699 shutdown_connections 2024-05-27T07:53:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.430+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5622f1bd9b50 0x5622f1bdbf70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.430+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1bbfe70 0x5622f1bca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.430+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1a40e40 0x5622f1bc95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.430+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/3454925699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 0x5622f1bc9b30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.430+0000 7f6a5b400700 0 client.16881 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:53:35.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.431+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/3454925699 >> 172.21.15.90:0/3454925699 conn(0x5622f1ada390 msgr2=0x5622f1bbf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.431+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/3454925699 shutdown_connections 2024-05-27T07:53:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.432+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/3454925699 wait complete. 2024-05-27T07:53:35.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.432+0000 7f6a8aab7b80 20 client.16881 trim_cache size 0 max 16384 2024-05-27T07:53:35.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.467+0000 7f6a8aab7b80 1 Processor -- start 2024-05-27T07:53:35.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.468+0000 7f6a8aab7b80 1 -- start start 2024-05-27T07:53:35.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.468+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1a40e40 0x5622f1bc4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.468+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 0x5622f1bbf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.468+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1bbfe70 0x5622f1bd5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.468+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6a540108d0 con 0x5622f1b2c9f0 2024-05-27T07:53:35.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.468+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5400e6c0 con 0x5622f1bbfe70 2024-05-27T07:53:35.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.468+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54010200 con 0x5622f1a40e40 2024-05-27T07:53:35.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.469+0000 7f6a79600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1a40e40 0x5622f1bc4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.469+0000 7f6a79600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1a40e40 0x5622f1bc4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42316/0 (socket says 172.21.15.90:42316) 2024-05-27T07:53:35.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.469+0000 7f6a79600700 1 -- 172.21.15.90:0/4000998771 learned_addr learned my addr 172.21.15.90:0/4000998771 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:35.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.469+0000 7f6a7a000700 1 --2- 172.21.15.90:0/4000998771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1bbfe70 0x5622f1bd5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.470+0000 7f6a7a000700 1 -- 172.21.15.90:0/4000998771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1a40e40 msgr2=0x5622f1bc4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:35.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.470+0000 7f6a7a000700 1 --2- 172.21.15.90:0/4000998771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1a40e40 0x5622f1bc4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.470+0000 7f6a7a000700 1 -- 172.21.15.90:0/4000998771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 msgr2=0x5622f1bbf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:35.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.470+0000 7f6a7a000700 1 --2- 172.21.15.90:0/4000998771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 0x5622f1bbf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.470+0000 7f6a7a000700 1 -- 172.21.15.90:0/4000998771 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f1bca3e0 con 0x5622f1bbfe70 2024-05-27T07:53:35.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.471+0000 7f6a7a000700 1 --2- 172.21.15.90:0/4000998771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1bbfe70 0x5622f1bd5cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c012a20 tx=0x7f6a6c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1012fd006056f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:35.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.471+0000 7f6a66000700 1 -- 172.21.15.90:0/4000998771 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00eae0 con 0x5622f1bbfe70 2024-05-27T07:53:35.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.472+0000 7f6a66000700 1 -- 172.21.15.90:0/4000998771 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a6c020050 con 0x5622f1bbfe70 2024-05-27T07:53:35.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.472+0000 7f6a66000700 1 -- 172.21.15.90:0/4000998771 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c024bb0 con 0x5622f1bbfe70 2024-05-27T07:53:35.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.474+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/4000998771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1bbfe70 msgr2=0x5622f1bd5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:35.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.474+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/4000998771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1bbfe70 0x5622f1bd5cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c012a20 tx=0x7f6a6c011000 comp rx=0 tx=0).stop 2024-05-27T07:53:35.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.475+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/4000998771 shutdown_connections 2024-05-27T07:53:35.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.475+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/4000998771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1bbfe70 0x5622f1bd5cb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.476+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/4000998771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1a40e40 0x5622f1bc4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.476+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/4000998771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1b2c9f0 0x5622f1bbf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.477+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/4000998771 >> 172.21.15.90:0/4000998771 conn(0x5622f1ada390 msgr2=0x5622f1bbf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:35.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.477+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/4000998771 shutdown_connections 2024-05-27T07:53:35.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.478+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/4000998771 wait complete. 2024-05-27T07:53:35.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.479+0000 7f6a8aab7b80 1 -- start start 2024-05-27T07:53:35.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.480+0000 7f6a8aab7b80 1 Processor -- start 2024-05-27T07:53:35.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.482+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1a40e40 0x5622f1c41820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.482+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1c41d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.482+0000 7f6a8aab7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1bbfe70 0x5622f1bd1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.482+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5400f460 con 0x5622f1a40e40 2024-05-27T07:53:35.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.482+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5400ed90 con 0x5622f1bbfe70 2024-05-27T07:53:35.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.482+0000 7f6a8aab7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5400fb30 con 0x5622f1b2c9f0 2024-05-27T07:53:35.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.482+0000 7f6a8aab7b80 10 client.0 _ll_register_callbacks cb 0x5622f1a87c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:35.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.482+0000 7f6a78c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1c41d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.483+0000 7f6a78c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1c41d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42332/0 (socket says 172.21.15.90:42332) 2024-05-27T07:53:35.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.483+0000 7f6a78c00700 1 -- 172.21.15.90:0/802518530 learned_addr learned my addr 172.21.15.90:0/802518530 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:35.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.483+0000 7f6a79600700 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1a40e40 0x5622f1c41820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.483+0000 7f6a78c00700 1 -- 172.21.15.90:0/802518530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1bbfe70 msgr2=0x5622f1bd1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:35.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.483+0000 7f6a78c00700 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1bbfe70 0x5622f1bd1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.483+0000 7f6a78c00700 1 -- 172.21.15.90:0/802518530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1a40e40 msgr2=0x5622f1c41820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:35.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.483+0000 7f6a78c00700 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1a40e40 0x5622f1c41820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.484+0000 7f6a78c00700 1 -- 172.21.15.90:0/802518530 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f1bd23d0 con 0x5622f1b2c9f0 2024-05-27T07:53:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.486+0000 7f6a78c00700 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1c41d60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a600074a0 tx=0x7f6a600a8de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55b10d9c5f26633c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.486+0000 7f6a5b400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:53:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.486+0000 7f6a5b400700 1 -- 172.21.15.90:0/802518530 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a600a9750 con 0x5622f1b2c9f0 2024-05-27T07:53:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.486+0000 7f6a5b400700 1 -- 172.21.15.90:0/802518530 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f6a600e2410 con 0x5622f1b2c9f0 2024-05-27T07:53:35.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.486+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/802518530 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a6c020050 con 0x5622f1b2c9f0 2024-05-27T07:53:35.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.487+0000 7f6a5b400700 1 -- 172.21.15.90:0/802518530 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6a60091320 con 0x5622f1b2c9f0 2024-05-27T07:53:35.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.487+0000 7f6a8aab7b80 10 client.26458 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:35.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.487+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/802518530 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5622f1bc41e0 con 0x5622f1b2c9f0 2024-05-27T07:53:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.488+0000 7f6a5b400700 1 -- 172.21.15.90:0/802518530 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a6c020050 con 0x5622f1b2c9f0 2024-05-27T07:53:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.488+0000 7f6a5b400700 1 -- 172.21.15.90:0/802518530 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7f6a600a51d0 con 0x5622f1b2c9f0 2024-05-27T07:53:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.488+0000 7f6a8aab7b80 10 client.26458 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.488+0000 7f6a8aab7b80 10 client.26458 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:53:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.488+0000 7f6a8aab7b80 20 client.26458 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.489+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5622f1c66640 0x5622f1c68a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.489+0000 7f6a8aab7b80 4 client.26458 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:53:35.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.489+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/802518530 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622f1ae9570 con 0x5622f1c66640 2024-05-27T07:53:35.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.489+0000 7f6a79600700 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5622f1c66640 0x5622f1c68a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.491+0000 7f6a5b400700 10 client.26458 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:53:35.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.491+0000 7f6a79600700 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5622f1c66640 0x5622f1c68a60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7dd1e2f2c5f7ff6 server_cookie=f850fa4cb781921b in_seq=0 out_seq=0 2024-05-27T07:53:35.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.492+0000 7f6a5b400700 1 -- 172.21.15.90:0/802518530 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622f1ae9570 con 0x5622f1c66640 2024-05-27T07:53:35.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.492+0000 7f6a5b400700 10 client.26458 handle_command_reply: tid=0 2024-05-27T07:53:35.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.492+0000 7f6a8aab7b80 1 client.26458 shutdown 2024-05-27T07:53:35.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.493+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/802518530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 msgr2=0x5622f1c41d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:35.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.493+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1c41d60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a600074a0 tx=0x7f6a600a8de0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.494+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/802518530 shutdown_connections 2024-05-27T07:53:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.494+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5622f1c66640 0x5622f1c68a60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.494+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5622f1bbfe70 0x5622f1bd1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.494+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5622f1a40e40 0x5622f1c41820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.494+0000 7f6a8aab7b80 1 --2- 172.21.15.90:0/802518530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5622f1b2c9f0 0x5622f1c41d60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.494+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/802518530 >> 172.21.15.90:0/802518530 conn(0x5622f1ada390 msgr2=0x5622f1bc7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.494+0000 7f6a5b400700 0 client.26458 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:53:35.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.495+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/802518530 shutdown_connections 2024-05-27T07:53:35.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.495+0000 7f6a8aab7b80 1 -- 172.21.15.90:0/802518530 wait complete. 2024-05-27T07:53:35.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.496+0000 7f6a8aab7b80 20 client.26458 trim_cache size 0 max 16384 2024-05-27T07:53:35.496 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:53:35.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.497+0000 7f6a7c000700 1 -- 172.21.15.90:0/1950299496 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6a5c0b2af0 msgr2=0x7f6a5c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:35.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.497+0000 7f6a7c000700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6a5c0b2af0 0x7f6a5c0b5000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c01f000 tx=0x7f6a6c019000 comp rx=0 tx=0).stop 2024-05-27T07:53:35.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.497+0000 7f6a7c000700 1 -- 172.21.15.90:0/1950299496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a74107de0 msgr2=0x7f6a7410d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:35.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.497+0000 7f6a7c000700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a74107de0 0x7f6a7410d9b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c9e0 tx=0x7f6a6000cda0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.497+0000 7f6a7c000700 1 -- 172.21.15.90:0/1950299496 shutdown_connections 2024-05-27T07:53:35.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.497+0000 7f6a7c000700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a74108730 0x7f6a7410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.497+0000 7f6a7c000700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a741034b0 0x7f6a7410d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.497+0000 7f6a7c000700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6a5c0b2af0 0x7f6a5c0b5000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.497+0000 7f6a7c000700 1 --2- 172.21.15.90:0/1950299496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6a74107de0 0x7f6a7410d9b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:35.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.497+0000 7f6a7c000700 1 -- 172.21.15.90:0/1950299496 >> 172.21.15.90:0/1950299496 conn(0x7f6a740fedb0 msgr2=0x7f6a741011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.498+0000 7f6a7c000700 1 -- 172.21.15.90:0/1950299496 shutdown_connections 2024-05-27T07:53:35.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:35.498+0000 7f6a7c000700 1 -- 172.21.15.90:0/1950299496 wait complete. 2024-05-27T07:53:36.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:36 smithi090 ceph-mon[31429]: pgmap v225: 129 pgs: 129 active+clean; 2.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 12 MiB/s wr, 671 op/s 2024-05-27T07:53:36.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:36 smithi136 ceph-mon[35324]: pgmap v225: 129 pgs: 129 active+clean; 2.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 12 MiB/s wr, 671 op/s 2024-05-27T07:53:36.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:36 smithi129 ceph-mon[35254]: pgmap v225: 129 pgs: 129 active+clean; 2.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 12 MiB/s wr, 671 op/s 2024-05-27T07:53:36.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:53:37.559 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:38.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:38 smithi090 ceph-mon[31429]: pgmap v226: 129 pgs: 129 active+clean; 2.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 11 MiB/s wr, 542 op/s 2024-05-27T07:53:38.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:38 smithi136 ceph-mon[35324]: pgmap v226: 129 pgs: 129 active+clean; 2.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 11 MiB/s wr, 542 op/s 2024-05-27T07:53:38.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:38 smithi129 ceph-mon[35254]: pgmap v226: 129 pgs: 129 active+clean; 2.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 11 MiB/s wr, 542 op/s 2024-05-27T07:53:38.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 200.0 s, 546.0 tps, lat 182.962 ms stddev 265.679 2024-05-27T07:53:40.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:40 smithi136 ceph-mon[35324]: pgmap v227: 129 pgs: 129 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 16 MiB/s wr, 842 op/s 2024-05-27T07:53:40.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:40 smithi129 ceph-mon[35254]: pgmap v227: 129 pgs: 129 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 16 MiB/s wr, 842 op/s 2024-05-27T07:53:40.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:40 smithi090 ceph-mon[31429]: pgmap v227: 129 pgs: 129 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 16 MiB/s wr, 842 op/s 2024-05-27T07:53:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.337+0000 7fc252600640 1 -- 172.21.15.90:0/3703790169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc24c109cd0 msgr2=0x7fc24c103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.337+0000 7fc252600640 1 --2- 172.21.15.90:0/3703790169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc24c109cd0 0x7fc24c103870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009860 tx=0x7fc23c02a340 comp rx=0 tx=0).stop 2024-05-27T07:53:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.337+0000 7fc252600640 1 -- 172.21.15.90:0/3703790169 shutdown_connections 2024-05-27T07:53:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.337+0000 7fc252600640 1 --2- 172.21.15.90:0/3703790169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc24c104420 0x7fc24c1107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.338+0000 7fc252600640 1 --2- 172.21.15.90:0/3703790169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24c10a170 0x7fc24c103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.338+0000 7fc252600640 1 --2- 172.21.15.90:0/3703790169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc24c109cd0 0x7fc24c103870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.338+0000 7fc252600640 1 -- 172.21.15.90:0/3703790169 >> 172.21.15.90:0/3703790169 conn(0x7fc24c079440 msgr2=0x7fc24c079880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.338+0000 7fc252600640 1 -- 172.21.15.90:0/3703790169 shutdown_connections 2024-05-27T07:53:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.338+0000 7fc252600640 1 -- 172.21.15.90:0/3703790169 wait complete. 2024-05-27T07:53:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.339+0000 7fc252600640 1 Processor -- start 2024-05-27T07:53:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.340+0000 7fc252600640 1 -- start start 2024-05-27T07:53:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.340+0000 7fc252600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc24c104420 0x7fc24c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.341+0000 7fc252600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24c109cd0 0x7fc24c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.341+0000 7fc252600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc24c10a170 0x7fc24c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.341+0000 7fc252600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c113d50 con 0x7fc24c104420 2024-05-27T07:53:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.341+0000 7fc252600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c113bd0 con 0x7fc24c10a170 2024-05-27T07:53:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.341+0000 7fc252600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c113ed0 con 0x7fc24c109cd0 2024-05-27T07:53:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.341+0000 7fc24aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24c109cd0 0x7fc24c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.341+0000 7fc24aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24c109cd0 0x7fc24c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:45988/0 (socket says 172.21.15.90:45988) 2024-05-27T07:53:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.341+0000 7fc24aa00640 1 -- 172.21.15.90:0/4080984331 learned_addr learned my addr 172.21.15.90:0/4080984331 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.341+0000 7fc24be00640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc24c10a170 0x7fc24c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.342+0000 7fc24aa00640 1 -- 172.21.15.90:0/4080984331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc24c10a170 msgr2=0x7fc24c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.342+0000 7fc24aa00640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc24c10a170 0x7fc24c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.342+0000 7fc24aa00640 1 -- 172.21.15.90:0/4080984331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc24c104420 msgr2=0x7fc24c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.342+0000 7fc24aa00640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc24c104420 0x7fc24c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.342+0000 7fc24aa00640 1 -- 172.21.15.90:0/4080984331 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24c19f5e0 con 0x7fc24c109cd0 2024-05-27T07:53:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.342+0000 7fc24aa00640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24c109cd0 0x7fc24c19adb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc23800e9f0 tx=0x7fc23800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.343+0000 7fc243e00640 1 -- 172.21.15.90:0/4080984331 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23800ccc0 con 0x7fc24c109cd0 2024-05-27T07:53:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.343+0000 7fc252600640 1 -- 172.21.15.90:0/4080984331 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24c19f8d0 con 0x7fc24c109cd0 2024-05-27T07:53:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.343+0000 7fc252600640 1 -- 172.21.15.90:0/4080984331 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc24c1a72d0 con 0x7fc24c109cd0 2024-05-27T07:53:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.343+0000 7fc243e00640 1 -- 172.21.15.90:0/4080984331 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc23800ce60 con 0x7fc24c109cd0 2024-05-27T07:53:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.343+0000 7fc243e00640 1 -- 172.21.15.90:0/4080984331 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238010430 con 0x7fc24c109cd0 2024-05-27T07:53:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.349+0000 7fc259576740 1 Processor -- start 2024-05-27T07:53:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.350+0000 7fc259576740 1 -- start start 2024-05-27T07:53:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.350+0000 7fc259576740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e0c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.350+0000 7fc259576740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 0x55be45e0ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc259576740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a3a4d0 0x55be45e0b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc259576740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55be45d6eb90 con 0x55be45a322f0 2024-05-27T07:53:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc259576740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55be45d5e730 con 0x55be45a5b1b0 2024-05-27T07:53:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc259576740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55be45d62770 con 0x55be45a3a4d0 2024-05-27T07:53:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc24be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e0c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc24be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e0c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34000/0 (socket says 172.21.15.90:34000) 2024-05-27T07:53:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc24be00640 1 -- 172.21.15.90:0/3454868094 learned_addr learned my addr 172.21.15.90:0/3454868094 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc24be00640 1 -- 172.21.15.90:0/3454868094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a3a4d0 msgr2=0x55be45e0b570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:53:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc24be00640 1 --2- 172.21.15.90:0/3454868094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a3a4d0 0x55be45e0b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc24be00640 1 -- 172.21.15.90:0/3454868094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 msgr2=0x55be45e0ce10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:53:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc24be00640 1 --2- 172.21.15.90:0/3454868094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 0x55be45e0ce10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc24be00640 1 -- 172.21.15.90:0/3454868094 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be45cf78b0 con 0x55be45a5b1b0 2024-05-27T07:53:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.351+0000 7fc243e00640 1 -- 172.21.15.90:0/4080984331 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc238004020 con 0x7fc24c109cd0 2024-05-27T07:53:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.352+0000 7fc24be00640 1 --2- 172.21.15.90:0/3454868094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e0c8d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc244007f40 tx=0x7fc24400f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c18be37e87f5b50 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.353+0000 7fc240c00640 1 -- 172.21.15.90:0/3454868094 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244010900 con 0x55be45a5b1b0 2024-05-27T07:53:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.353+0000 7fc240c00640 1 -- 172.21.15.90:0/3454868094 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2440183f0 con 0x55be45a5b1b0 2024-05-27T07:53:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.353+0000 7fc240c00640 1 -- 172.21.15.90:0/3454868094 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244017600 con 0x55be45a5b1b0 2024-05-27T07:53:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.354+0000 7fc243e00640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2300b2950 0x7fc2300b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.354+0000 7fc243e00640 1 -- 172.21.15.90:0/4080984331 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fc2380e2c40 con 0x7fc24c109cd0 2024-05-27T07:53:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.355+0000 7fc259576740 1 -- 172.21.15.90:0/3454868094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 msgr2=0x55be45e0c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.355+0000 7fc259576740 1 --2- 172.21.15.90:0/3454868094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e0c8d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc244007f40 tx=0x7fc24400f850 comp rx=0 tx=0).stop 2024-05-27T07:53:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.355+0000 7fc24b400640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2300b2950 0x7fc2300b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.355+0000 7fc259576740 1 -- 172.21.15.90:0/3454868094 shutdown_connections 2024-05-27T07:53:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.355+0000 7fc259576740 1 --2- 172.21.15.90:0/3454868094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a3a4d0 0x55be45e0b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.355+0000 7fc259576740 1 --2- 172.21.15.90:0/3454868094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 0x55be45e0ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.355+0000 7fc259576740 1 --2- 172.21.15.90:0/3454868094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e0c8d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.355+0000 7fc259576740 1 -- 172.21.15.90:0/3454868094 >> 172.21.15.90:0/3454868094 conn(0x55be45a4f300 msgr2=0x55be45df99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.356+0000 7fc259576740 1 -- 172.21.15.90:0/3454868094 shutdown_connections 2024-05-27T07:53:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.356+0000 7fc259576740 1 -- 172.21.15.90:0/3454868094 wait complete. 2024-05-27T07:53:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.356+0000 7fc24b400640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2300b2950 0x7fc2300b4e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009780 tx=0x7fc23c005c90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.357+0000 7fc259576740 1 -- start start 2024-05-27T07:53:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.357+0000 7fc259576740 1 Processor -- start 2024-05-27T07:53:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.358+0000 7fc259576740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a322f0 0x55be45e10810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.358+0000 7fc259576740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 0x55be45e10d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.358+0000 7fc259576740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e11290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.358+0000 7fc259576740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55be45a44790 con 0x55be45a3a4d0 2024-05-27T07:53:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.358+0000 7fc259576740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55be45d06bc0 con 0x55be45a5b1b0 2024-05-27T07:53:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.358+0000 7fc259576740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55be45e0bfa0 con 0x55be45a322f0 2024-05-27T07:53:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.358+0000 7fc259576740 10 client.0 _ll_register_callbacks cb 0x55be45ce4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.358+0000 7fc24b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a322f0 0x55be45e10810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.358+0000 7fc24b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a322f0 0x55be45e10810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:45990/0 (socket says 172.21.15.90:45990) 2024-05-27T07:53:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.358+0000 7fc24b400640 1 -- 172.21.15.90:0/2188228043 learned_addr learned my addr 172.21.15.90:0/2188228043 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.359+0000 7fc24aa00640 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 0x55be45e10d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.359+0000 7fc24b400640 1 -- 172.21.15.90:0/2188228043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 msgr2=0x55be45e11290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.359+0000 7fc24be00640 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e11290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.359+0000 7fc24b400640 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e11290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.359+0000 7fc24b400640 1 -- 172.21.15.90:0/2188228043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 msgr2=0x55be45e10d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.359+0000 7fc24b400640 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 0x55be45e10d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.359+0000 7fc24b400640 1 -- 172.21.15.90:0/2188228043 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be45a73210 con 0x55be45a322f0 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.359+0000 7fc24aa00640 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 0x55be45e10d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.360+0000 7fc24be00640 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e11290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.360+0000 7fc235600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.360+0000 7fc24b400640 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a322f0 0x55be45e10810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc23c034000 tx=0x7fc23c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0e809bd4021202 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.360+0000 7fc235600640 1 -- 172.21.15.90:0/2188228043 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c044040 con 0x55be45a322f0 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.361+0000 7fc259576740 1 -- 172.21.15.90:0/2188228043 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2440183f0 con 0x55be45a322f0 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.361+0000 7fc235600640 1 -- 172.21.15.90:0/2188228043 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fc23c004380 con 0x55be45a322f0 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.361+0000 7fc235600640 1 -- 172.21.15.90:0/2188228043 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc23c044040 con 0x55be45a322f0 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.361+0000 7fc259576740 10 client.26467 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.361+0000 7fc259576740 1 -- 172.21.15.90:0/2188228043 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55be45b3f620 con 0x55be45a322f0 2024-05-27T07:53:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.362+0000 7fc235600640 1 -- 172.21.15.90:0/2188228043 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc2440183f0 con 0x55be45a322f0 2024-05-27T07:53:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.362+0000 7fc235600640 1 -- 172.21.15.90:0/2188228043 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7fc23c032440 con 0x55be45a322f0 2024-05-27T07:53:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.363+0000 7fc259576740 10 client.26467 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.363+0000 7fc259576740 10 client.26467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.363+0000 7fc259576740 20 client.26467 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.363+0000 7fc259576740 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55be45a19d80 0x55be45e334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.363+0000 7fc259576740 4 client.26467 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.363+0000 7fc259576740 1 -- 172.21.15.90:0/2188228043 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be45d70ae0 con 0x55be45a19d80 2024-05-27T07:53:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.363+0000 7fc24aa00640 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55be45a19d80 0x55be45e334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.364+0000 7fc235600640 10 client.26467 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.365+0000 7fc24aa00640 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55be45a19d80 0x55be45e334e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467a83d790e2b1af server_cookie=9eebb86b8f8501c7 in_seq=0 out_seq=0 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.366+0000 7fc235600640 1 -- 172.21.15.90:0/2188228043 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55be45d70ae0 con 0x55be45a19d80 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.366+0000 7fc235600640 10 client.26467 handle_command_reply: tid=0 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.366+0000 7fc259576740 1 client.26467 shutdown 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 1 -- 172.21.15.90:0/2188228043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a322f0 msgr2=0x55be45e10810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a322f0 0x55be45e10810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc23c034000 tx=0x7fc23c005f90 comp rx=0 tx=0).stop 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 1 -- 172.21.15.90:0/2188228043 shutdown_connections 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55be45a19d80 0x55be45e334e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a5b1b0 0x55be45e11290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 0x55be45e10d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 1 --2- 172.21.15.90:0/2188228043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a322f0 0x55be45e10810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 1 -- 172.21.15.90:0/2188228043 >> 172.21.15.90:0/2188228043 conn(0x55be45a4f300 msgr2=0x55be45df99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc235600640 0 client.26467 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 1 -- 172.21.15.90:0/2188228043 shutdown_connections 2024-05-27T07:53:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 1 -- 172.21.15.90:0/2188228043 wait complete. 2024-05-27T07:53:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.367+0000 7fc259576740 20 client.26467 trim_cache size 0 max 16384 2024-05-27T07:53:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.411+0000 7fc259576740 1 Processor -- start 2024-05-27T07:53:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.412+0000 7fc259576740 1 -- start start 2024-05-27T07:53:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.412+0000 7fc259576740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a322f0 0x55be45e2d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.412+0000 7fc259576740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 0x55be45e110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.412+0000 7fc259576740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 0x55be45e08c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.413+0000 7fc24b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a322f0 0x55be45e2d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.414+0000 7fc24b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a322f0 0x55be45e2d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34022/0 (socket says 172.21.15.90:34022) 2024-05-27T07:53:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.414+0000 7fc24aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 0x55be45e110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.414+0000 7fc24be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 0x55be45e08c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.414+0000 7fc24be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 0x55be45e08c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46004/0 (socket says 172.21.15.90:46004) 2024-05-27T07:53:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.414+0000 7fc24be00640 1 -- 172.21.15.90:0/430368738 learned_addr learned my addr 172.21.15.90:0/430368738 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.414+0000 7fc259576740 1 -- 172.21.15.90:0/430368738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c1a86f0 con 0x55be45a3a4d0 2024-05-27T07:53:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.414+0000 7fc259576740 1 -- 172.21.15.90:0/430368738 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c076130 con 0x55be45a322f0 2024-05-27T07:53:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.414+0000 7fc259576740 1 -- 172.21.15.90:0/430368738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c1a8020 con 0x55be45a5b1b0 2024-05-27T07:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.415+0000 7fc24b400640 1 -- 172.21.15.90:0/430368738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 msgr2=0x55be45e08c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.416+0000 7fc24b400640 1 --2- 172.21.15.90:0/430368738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 0x55be45e08c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.416+0000 7fc24b400640 1 -- 172.21.15.90:0/430368738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 msgr2=0x55be45e110b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.416+0000 7fc24b400640 1 --2- 172.21.15.90:0/430368738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 0x55be45e110b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.416+0000 7fc24b400640 1 -- 172.21.15.90:0/430368738 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be45e2db80 con 0x55be45a322f0 2024-05-27T07:53:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.416+0000 7fc24be00640 1 --2- 172.21.15.90:0/430368738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 0x55be45e08c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:53:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.417+0000 7fc24b400640 1 --2- 172.21.15.90:0/430368738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a322f0 0x55be45e2d640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc23c032830 tx=0x7fc23c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42c2bd6ef33b9884 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.417+0000 7fc237e00640 1 -- 172.21.15.90:0/430368738 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c044040 con 0x55be45a322f0 2024-05-27T07:53:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.417+0000 7fc237e00640 1 -- 172.21.15.90:0/430368738 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc23c04d070 con 0x55be45a322f0 2024-05-27T07:53:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.417+0000 7fc237e00640 1 -- 172.21.15.90:0/430368738 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c003ce0 con 0x55be45a322f0 2024-05-27T07:53:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.419+0000 7fc259576740 1 -- 172.21.15.90:0/430368738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a322f0 msgr2=0x55be45e2d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.420+0000 7fc259576740 1 --2- 172.21.15.90:0/430368738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a322f0 0x55be45e2d640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc23c032830 tx=0x7fc23c000bc0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.420+0000 7fc259576740 1 -- 172.21.15.90:0/430368738 shutdown_connections 2024-05-27T07:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.421+0000 7fc259576740 1 --2- 172.21.15.90:0/430368738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 0x55be45e08c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.421+0000 7fc259576740 1 --2- 172.21.15.90:0/430368738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a3a4d0 0x55be45e110b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.421+0000 7fc259576740 1 --2- 172.21.15.90:0/430368738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a322f0 0x55be45e2d640 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.421+0000 7fc259576740 1 -- 172.21.15.90:0/430368738 >> 172.21.15.90:0/430368738 conn(0x55be45a4f300 msgr2=0x55be45e08530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.422+0000 7fc259576740 1 -- 172.21.15.90:0/430368738 shutdown_connections 2024-05-27T07:53:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.422+0000 7fc259576740 1 -- 172.21.15.90:0/430368738 wait complete. 2024-05-27T07:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.423+0000 7fc259576740 1 -- start start 2024-05-27T07:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.423+0000 7fc259576740 1 Processor -- start 2024-05-27T07:53:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.425+0000 7fc259576740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 0x55be45e24e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.426+0000 7fc259576740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a3a4d0 0x55be45e25350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.426+0000 7fc259576740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 0x55be45e99ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.426+0000 7fc24b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 0x55be45e24e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.426+0000 7fc259576740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c076ed0 con 0x55be45a322f0 2024-05-27T07:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.426+0000 7fc259576740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c076800 con 0x55be45a3a4d0 2024-05-27T07:53:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.426+0000 7fc259576740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c1a7950 con 0x55be45a5b1b0 2024-05-27T07:53:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.426+0000 7fc24b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 0x55be45e24e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39196/0 (socket says 172.21.15.90:39196) 2024-05-27T07:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.426+0000 7fc24b400640 1 -- 172.21.15.90:0/2445365276 learned_addr learned my addr 172.21.15.90:0/2445365276 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.426+0000 7fc259576740 10 client.0 _ll_register_callbacks cb 0x7fc23c041040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.426+0000 7fc24be00640 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 0x55be45e99ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.427+0000 7fc24aa00640 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a3a4d0 0x55be45e25350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.427+0000 7fc24b400640 1 -- 172.21.15.90:0/2445365276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 msgr2=0x55be45e99ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.427+0000 7fc24b400640 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 0x55be45e99ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.428+0000 7fc24b400640 1 -- 172.21.15.90:0/2445365276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a3a4d0 msgr2=0x55be45e25350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.428+0000 7fc24b400640 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a3a4d0 0x55be45e25350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.428+0000 7fc24b400640 1 -- 172.21.15.90:0/2445365276 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be45e9a0e0 con 0x55be45a322f0 2024-05-27T07:53:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.429+0000 7fc24aa00640 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a3a4d0 0x55be45e25350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.429+0000 7fc24b400640 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 0x55be45e24e10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009860 tx=0x7fc23c0326f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab6f98d3a99a396 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:41.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.429+0000 7fc235600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:53:41.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.430+0000 7fc235600640 1 -- 172.21.15.90:0/2445365276 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c06b060 con 0x55be45a322f0 2024-05-27T07:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.431+0000 7fc235600640 1 -- 172.21.15.90:0/2445365276 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fc23c0035f0 con 0x55be45a322f0 2024-05-27T07:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.431+0000 7fc259576740 1 -- 172.21.15.90:0/2445365276 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc23c04d070 con 0x55be45a322f0 2024-05-27T07:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.431+0000 7fc235600640 1 -- 172.21.15.90:0/2445365276 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc23c072050 con 0x55be45a322f0 2024-05-27T07:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.431+0000 7fc259576740 10 client.16890 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.431+0000 7fc259576740 1 -- 172.21.15.90:0/2445365276 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55be45e9b2c0 con 0x55be45a322f0 2024-05-27T07:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.432+0000 7fc235600640 1 -- 172.21.15.90:0/2445365276 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc23c04d070 con 0x55be45a322f0 2024-05-27T07:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.432+0000 7fc235600640 1 -- 172.21.15.90:0/2445365276 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7fc23c05d9b0 con 0x55be45a322f0 2024-05-27T07:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.432+0000 7fc259576740 10 client.16890 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.432+0000 7fc259576740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.432+0000 7fc259576740 20 client.16890 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.433+0000 7fc259576740 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55be459ef1c0 0x55be45e9d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.433+0000 7fc259576740 4 client.16890 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.433+0000 7fc259576740 1 -- 172.21.15.90:0/2445365276 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be45d6a2c0 con 0x55be459ef1c0 2024-05-27T07:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.433+0000 7fc24aa00640 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55be459ef1c0 0x55be45e9d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.434+0000 7fc24aa00640 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55be459ef1c0 0x55be45e9d9f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6a564e98e76043f server_cookie=748dd12a4308f596 in_seq=0 out_seq=0 2024-05-27T07:53:41.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.434+0000 7fc235600640 10 client.16890 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.481+0000 7fc235600640 1 -- 172.21.15.90:0/2445365276 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55be45d6a2c0 con 0x55be459ef1c0 2024-05-27T07:53:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.481+0000 7fc235600640 10 client.16890 handle_command_reply: tid=0 2024-05-27T07:53:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.481+0000 7fc259576740 1 client.16890 shutdown 2024-05-27T07:53:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.481+0000 7fc259576740 1 -- 172.21.15.90:0/2445365276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 msgr2=0x55be45e24e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.481+0000 7fc259576740 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 0x55be45e24e10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009860 tx=0x7fc23c0326f0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.482+0000 7fc259576740 1 -- 172.21.15.90:0/2445365276 shutdown_connections 2024-05-27T07:53:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.482+0000 7fc259576740 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55be459ef1c0 0x55be45e9d9f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.482+0000 7fc259576740 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55be45a3a4d0 0x55be45e25350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.482+0000 7fc259576740 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55be45a5b1b0 0x55be45e99ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.482+0000 7fc259576740 1 --2- 172.21.15.90:0/2445365276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55be45a322f0 0x55be45e24e10 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.482+0000 7fc259576740 1 -- 172.21.15.90:0/2445365276 >> 172.21.15.90:0/2445365276 conn(0x55be45a4f300 msgr2=0x55be45e100a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:41.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.482+0000 7fc235600640 0 client.16890 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.484+0000 7fc259576740 1 -- 172.21.15.90:0/2445365276 shutdown_connections 2024-05-27T07:53:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.484+0000 7fc259576740 1 -- 172.21.15.90:0/2445365276 wait complete. 2024-05-27T07:53:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.484+0000 7fc259576740 20 client.16890 trim_cache size 0 max 16384 2024-05-27T07:53:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:53:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:53:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:53:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.487+0000 7fc252600640 1 -- 172.21.15.90:0/4080984331 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2300b2950 msgr2=0x7fc2300b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.487+0000 7fc252600640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2300b2950 0x7fc2300b4e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009780 tx=0x7fc23c005c90 comp rx=0 tx=0).stop 2024-05-27T07:53:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.487+0000 7fc252600640 1 -- 172.21.15.90:0/4080984331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24c109cd0 msgr2=0x7fc24c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.487+0000 7fc252600640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24c109cd0 0x7fc24c19adb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc23800e9f0 tx=0x7fc23800eeb0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.487+0000 7fc252600640 1 -- 172.21.15.90:0/4080984331 shutdown_connections 2024-05-27T07:53:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.487+0000 7fc252600640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc2300b2950 0x7fc2300b4e60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.487+0000 7fc252600640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc24c10a170 0x7fc24c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.487+0000 7fc252600640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24c109cd0 0x7fc24c19adb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.487+0000 7fc252600640 1 --2- 172.21.15.90:0/4080984331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc24c104420 0x7fc24c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:41.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.487+0000 7fc252600640 1 -- 172.21.15.90:0/4080984331 >> 172.21.15.90:0/4080984331 conn(0x7fc24c079440 msgr2=0x7fc24c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.488+0000 7fc252600640 1 -- 172.21.15.90:0/4080984331 shutdown_connections 2024-05-27T07:53:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:41.488+0000 7fc252600640 1 -- 172.21.15.90:0/4080984331 wait complete. 2024-05-27T07:53:42.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.419+0000 7f07c3c00700 1 -- 172.21.15.90:0/200709615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc10efc0 msgr2=0x7f07bc10f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:42.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.419+0000 7f07c3c00700 1 --2- 172.21.15.90:0/200709615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc10efc0 0x7f07bc10f3d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f07b400b0a0 tx=0x7f07b400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.419+0000 7f07c3c00700 1 -- 172.21.15.90:0/200709615 shutdown_connections 2024-05-27T07:53:42.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.419+0000 7f07c3c00700 1 --2- 172.21.15.90:0/200709615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07bc114240 0x7f07bc11bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.419+0000 7f07c3c00700 1 --2- 172.21.15.90:0/200709615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc10efc0 0x7f07bc10f3d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.419+0000 7f07c3c00700 1 --2- 172.21.15.90:0/200709615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07bc1138f0 0x7f07bc113d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.419+0000 7f07c3c00700 1 -- 172.21.15.90:0/200709615 >> 172.21.15.90:0/200709615 conn(0x7f07bc10ab50 msgr2=0x7f07bc10cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:42.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.419+0000 7f07c3c00700 1 -- 172.21.15.90:0/200709615 shutdown_connections 2024-05-27T07:53:42.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.419+0000 7f07c3c00700 1 -- 172.21.15.90:0/200709615 wait complete. 2024-05-27T07:53:42.422 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:42 smithi090 ceph-mon[31429]: pgmap v228: 129 pgs: 129 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 11 MiB/s wr, 590 op/s 2024-05-27T07:53:42.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.421+0000 7f07c3c00700 1 Processor -- start 2024-05-27T07:53:42.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.421+0000 7f07c3c00700 1 -- start start 2024-05-27T07:53:42.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.423+0000 7f07c3c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07bc10efc0 0x7f07bc119020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.423+0000 7f07c3c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07bc1138f0 0x7f07bc119560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.423+0000 7f07c3c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc114240 0x7f07bc119aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.423+0000 7f07c3c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc11ed20 con 0x7f07bc10efc0 2024-05-27T07:53:42.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.423+0000 7f07c3c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc11eba0 con 0x7f07bc114240 2024-05-27T07:53:42.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.423+0000 7f07c3c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc11eea0 con 0x7f07bc1138f0 2024-05-27T07:53:42.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.423+0000 7f07bbe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07bc1138f0 0x7f07bc119560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.424+0000 7f07bbe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07bc1138f0 0x7f07bc119560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46038/0 (socket says 172.21.15.90:46038) 2024-05-27T07:53:42.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.424+0000 7f07bbe00700 1 -- 172.21.15.90:0/1093645089 learned_addr learned my addr 172.21.15.90:0/1093645089 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:42.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.424+0000 7f07c1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07bc10efc0 0x7f07bc119020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.424+0000 7f07c1c00700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc114240 0x7f07bc119aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.424+0000 7f07c1c00700 1 -- 172.21.15.90:0/1093645089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07bc1138f0 msgr2=0x7f07bc119560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:42.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.424+0000 7f07c1c00700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07bc1138f0 0x7f07bc119560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.424+0000 7f07c1c00700 1 -- 172.21.15.90:0/1093645089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07bc10efc0 msgr2=0x7f07bc119020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:42.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.424+0000 7f07c1c00700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07bc10efc0 0x7f07bc119020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.424+0000 7f07c1c00700 1 -- 172.21.15.90:0/1093645089 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc1b5c60 con 0x7f07bc114240 2024-05-27T07:53:42.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.425+0000 7f07c1c00700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc114240 0x7f07bc119aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f07a800d720 tx=0x7f07a800da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:42.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.426+0000 7f07b9600700 1 -- 172.21.15.90:0/1093645089 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8016070 con 0x7f07bc114240 2024-05-27T07:53:42.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.426+0000 7f07b9600700 1 -- 172.21.15.90:0/1093645089 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a8005020 con 0x7f07bc114240 2024-05-27T07:53:42.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.426+0000 7f07c3c00700 1 -- 172.21.15.90:0/1093645089 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc1b5f50 con 0x7f07bc114240 2024-05-27T07:53:42.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.426+0000 7f07c3c00700 1 -- 172.21.15.90:0/1093645089 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc1b6400 con 0x7f07bc114240 2024-05-27T07:53:42.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.427+0000 7f07b9600700 1 -- 172.21.15.90:0/1093645089 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8016070 con 0x7f07bc114240 2024-05-27T07:53:42.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.434+0000 7f07d283fb80 1 Processor -- start 2024-05-27T07:53:42.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.435+0000 7f07b9600700 1 -- 172.21.15.90:0/1093645089 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f07a800f710 con 0x7f07bc114240 2024-05-27T07:53:42.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.435+0000 7f07d283fb80 1 -- start start 2024-05-27T07:53:42.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.436+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 0x5616f66a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.436+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f6526e40 0x5616f66a5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.436+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f66a5e70 0x5616f66b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.436+0000 7f07d283fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5616f65d3e40 con 0x5616f66a5e70 2024-05-27T07:53:42.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.436+0000 7f07d283fb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616f65c39e0 con 0x5616f6526e40 2024-05-27T07:53:42.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.436+0000 7f07d283fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5616f65c7a20 con 0x5616f66129f0 2024-05-27T07:53:42.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.439+0000 7f07c1200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f6526e40 0x5616f66a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.442+0000 7f07c1200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f6526e40 0x5616f66a5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34060/0 (socket says 172.21.15.90:34060) 2024-05-27T07:53:42.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.442+0000 7f07c1200700 1 -- 172.21.15.90:0/2244023332 learned_addr learned my addr 172.21.15.90:0/2244023332 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:42.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.440+0000 7f07bbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f66a5e70 0x5616f66b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.444+0000 7f07bbe00700 1 --2- 172.21.15.90:0/2244023332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 0x5616f66a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.445+0000 7f07bbe00700 1 -- 172.21.15.90:0/2244023332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 msgr2=0x5616f66a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.445+0000 7f07bbe00700 1 --2- 172.21.15.90:0/2244023332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 0x5616f66a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.445+0000 7f07bbe00700 1 -- 172.21.15.90:0/2244023332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f6526e40 msgr2=0x5616f66a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.445+0000 7f07bbe00700 1 --2- 172.21.15.90:0/2244023332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f6526e40 0x5616f66a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.447+0000 7f07bbe00700 1 -- 172.21.15.90:0/2244023332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f658c420 con 0x5616f66a5e70 2024-05-27T07:53:42.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.447+0000 7f07b9600700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f07a00b2df0 0x7f07a00b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.447+0000 7f07b9600700 1 -- 172.21.15.90:0/1093645089 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f07a8004830 con 0x7f07bc114240 2024-05-27T07:53:42.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.448+0000 7f07c1200700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f07a00b2df0 0x7f07a00b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.449+0000 7f07bbe00700 1 --2- 172.21.15.90:0/2244023332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f66a5e70 0x5616f66b2110 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f07bc112270 tx=0x7f07ac00a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ffccf2bfd3098b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:42.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.451+0000 7f07b2000700 1 -- 172.21.15.90:0/2244023332 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac004830 con 0x5616f66a5e70 2024-05-27T07:53:42.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.452+0000 7f07c1200700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f07a00b2df0 0x7f07a00b5300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5616f66a2790 tx=0x7f07b400b7f0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:42.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.452+0000 7f07b2000700 1 -- 172.21.15.90:0/2244023332 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07ac005020 con 0x5616f66a5e70 2024-05-27T07:53:42.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.452+0000 7f07b2000700 1 -- 172.21.15.90:0/2244023332 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac004830 con 0x5616f66a5e70 2024-05-27T07:53:42.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.454+0000 7f07d283fb80 1 -- 172.21.15.90:0/2244023332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f66a5e70 msgr2=0x5616f66b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:42.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.454+0000 7f07d283fb80 1 --2- 172.21.15.90:0/2244023332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f66a5e70 0x5616f66b2110 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f07bc112270 tx=0x7f07ac00a700 comp rx=0 tx=0).stop 2024-05-27T07:53:42.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.454+0000 7f07d283fb80 1 -- 172.21.15.90:0/2244023332 shutdown_connections 2024-05-27T07:53:42.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.454+0000 7f07d283fb80 1 --2- 172.21.15.90:0/2244023332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f66a5e70 0x5616f66b2110 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.454+0000 7f07d283fb80 1 --2- 172.21.15.90:0/2244023332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 0x5616f66a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.454+0000 7f07d283fb80 1 --2- 172.21.15.90:0/2244023332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f6526e40 0x5616f66a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.454+0000 7f07d283fb80 1 -- 172.21.15.90:0/2244023332 >> 172.21.15.90:0/2244023332 conn(0x5616f65c0390 msgr2=0x5616f66a14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:42.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.455+0000 7f07d283fb80 1 -- 172.21.15.90:0/2244023332 shutdown_connections 2024-05-27T07:53:42.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.455+0000 7f07d283fb80 1 -- 172.21.15.90:0/2244023332 wait complete. 2024-05-27T07:53:42.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.456+0000 7f07d283fb80 1 -- start start 2024-05-27T07:53:42.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.456+0000 7f07d283fb80 1 Processor -- start 2024-05-27T07:53:42.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.457+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 0x5616f66af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.458+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66129f0 0x5616f66afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.458+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66a5e70 0x5616f66b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.458+0000 7f07d283fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5616f64c9fa0 con 0x5616f6526e40 2024-05-27T07:53:42.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.458+0000 7f07d283fb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616f6564760 con 0x5616f66129f0 2024-05-27T07:53:42.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.458+0000 7f07d283fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5616f66b2b80 con 0x5616f66a5e70 2024-05-27T07:53:42.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.459+0000 7f07bbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66129f0 0x5616f66afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.459+0000 7f07d283fb80 10 client.0 _ll_register_callbacks cb 0x5616f656dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:42.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.459+0000 7f07c1c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66a5e70 0x5616f66b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.459+0000 7f07c1c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66a5e70 0x5616f66b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46056/0 (socket says 172.21.15.90:46056) 2024-05-27T07:53:42.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.459+0000 7f07c1c00700 1 -- 172.21.15.90:0/548467503 learned_addr learned my addr 172.21.15.90:0/548467503 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:42.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.459+0000 7f07c1200700 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 0x5616f66af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.459+0000 7f07c1c00700 1 -- 172.21.15.90:0/548467503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66129f0 msgr2=0x5616f66afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:42.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.460+0000 7f07c1c00700 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66129f0 0x5616f66afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.460+0000 7f07c1c00700 1 -- 172.21.15.90:0/548467503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 msgr2=0x5616f66af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:42.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.460+0000 7f07c1c00700 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 0x5616f66af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.460+0000 7f07c1c00700 1 -- 172.21.15.90:0/548467503 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f65138d0 con 0x5616f66a5e70 2024-05-27T07:53:42.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.460+0000 7f07c1c00700 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66a5e70 0x5616f66b0140 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f07a80a9760 tx=0x7f07a80a9570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16c95d2720e03b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:42.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.460+0000 7f07a6a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:53:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.461+0000 7f07a6a00700 1 -- 172.21.15.90:0/548467503 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8016030 con 0x5616f66a5e70 2024-05-27T07:53:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.461+0000 7f07a6a00700 1 -- 172.21.15.90:0/548467503 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f07a80a9ed0 con 0x5616f66a5e70 2024-05-27T07:53:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.461+0000 7f07d283fb80 1 -- 172.21.15.90:0/548467503 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07ac005020 con 0x5616f66a5e70 2024-05-27T07:53:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.461+0000 7f07a6a00700 1 -- 172.21.15.90:0/548467503 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f07a80ade10 con 0x5616f66a5e70 2024-05-27T07:53:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.461+0000 7f07d283fb80 10 client.26485 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:42.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.461+0000 7f07d283fb80 1 -- 172.21.15.90:0/548467503 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5616f6594060 con 0x5616f66a5e70 2024-05-27T07:53:42.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.462+0000 7f07a6a00700 1 -- 172.21.15.90:0/548467503 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07ac005020 con 0x5616f66a5e70 2024-05-27T07:53:42.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.463+0000 7f07a6a00700 1 -- 172.21.15.90:0/548467503 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7f07a806f1a0 con 0x5616f66a5e70 2024-05-27T07:53:42.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.463+0000 7f07d283fb80 10 client.26485 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:42.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.463+0000 7f07d283fb80 10 client.26485 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:53:42.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.463+0000 7f07d283fb80 20 client.26485 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:42.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.463+0000 7f07d283fb80 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5616f66bfb90 0x5616f66c1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.463+0000 7f07d283fb80 4 client.26485 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:42.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.463+0000 7f07d283fb80 1 -- 172.21.15.90:0/548467503 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616f65d5d90 con 0x5616f66bfb90 2024-05-27T07:53:42.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.464+0000 7f07c1200700 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5616f66bfb90 0x5616f66c1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.466+0000 7f07a6a00700 10 client.26485 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:53:42.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.467+0000 7f07c1200700 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5616f66bfb90 0x5616f66c1fb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c0ee8b85d665f53 server_cookie=78754edb5944e1c1 in_seq=0 out_seq=0 2024-05-27T07:53:42.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.468+0000 7f07a6a00700 1 -- 172.21.15.90:0/548467503 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5616f65d5d90 con 0x5616f66bfb90 2024-05-27T07:53:42.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.468+0000 7f07a6a00700 10 client.26485 handle_command_reply: tid=0 2024-05-27T07:53:42.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.469+0000 7f07d283fb80 1 client.26485 shutdown 2024-05-27T07:53:42.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.469+0000 7f07d283fb80 1 -- 172.21.15.90:0/548467503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66a5e70 msgr2=0x5616f66b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:42.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.469+0000 7f07d283fb80 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66a5e70 0x5616f66b0140 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f07a80a9760 tx=0x7f07a80a9570 comp rx=0 tx=0).stop 2024-05-27T07:53:42.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.471+0000 7f07d283fb80 1 -- 172.21.15.90:0/548467503 shutdown_connections 2024-05-27T07:53:42.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.471+0000 7f07d283fb80 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66a5e70 0x5616f66b0140 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.471+0000 7f07d283fb80 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 0x5616f66af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.471+0000 7f07d283fb80 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5616f66bfb90 0x5616f66c1fb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.471+0000 7f07a6a00700 0 client.26485 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:53:42.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.471+0000 7f07d283fb80 1 --2- 172.21.15.90:0/548467503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66129f0 0x5616f66afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.471+0000 7f07d283fb80 1 -- 172.21.15.90:0/548467503 >> 172.21.15.90:0/548467503 conn(0x5616f65c0390 msgr2=0x5616f66a53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:42.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.472+0000 7f07d283fb80 1 -- 172.21.15.90:0/548467503 shutdown_connections 2024-05-27T07:53:42.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.472+0000 7f07d283fb80 1 -- 172.21.15.90:0/548467503 wait complete. 2024-05-27T07:53:42.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.472+0000 7f07d283fb80 20 client.26485 trim_cache size 0 max 16384 2024-05-27T07:53:42.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:42 smithi136 ceph-mon[35324]: pgmap v228: 129 pgs: 129 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 11 MiB/s wr, 590 op/s 2024-05-27T07:53:42.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.505+0000 7f07d283fb80 1 Processor -- start 2024-05-27T07:53:42.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.505+0000 7f07d283fb80 1 -- start start 2024-05-27T07:53:42.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.506+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f6526e40 0x5616f66a5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.506+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f66129f0 0x5616f66c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.506+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.506+0000 7f07d283fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1c0ff0 con 0x5616f66129f0 2024-05-27T07:53:42.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.506+0000 7f07d283fb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1bede0 con 0x5616f66a5e70 2024-05-27T07:53:42.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.506+0000 7f07d283fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1c0920 con 0x5616f6526e40 2024-05-27T07:53:42.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.507+0000 7f07c1c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.507+0000 7f07c1c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34086/0 (socket says 172.21.15.90:34086) 2024-05-27T07:53:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.507+0000 7f07c1c00700 1 -- 172.21.15.90:0/3354831243 learned_addr learned my addr 172.21.15.90:0/3354831243 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.507+0000 7f07c1200700 1 --2- 172.21.15.90:0/3354831243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f6526e40 0x5616f66a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.508+0000 7f07c1c00700 1 -- 172.21.15.90:0/3354831243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f6526e40 msgr2=0x5616f66a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:42.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.508+0000 7f07c1c00700 1 --2- 172.21.15.90:0/3354831243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f6526e40 0x5616f66a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.508+0000 7f07c1c00700 1 -- 172.21.15.90:0/3354831243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f66129f0 msgr2=0x5616f66c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:42.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.508+0000 7f07c1c00700 1 --2- 172.21.15.90:0/3354831243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f66129f0 0x5616f66c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.508+0000 7f07c1c00700 1 -- 172.21.15.90:0/3354831243 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f66a6a50 con 0x5616f66a5e70 2024-05-27T07:53:42.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.508+0000 7f07c1c00700 1 --2- 172.21.15.90:0/3354831243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66c7af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07a80adfd0 tx=0x7f07a80a93b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c8eae9b249a2586 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:42.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.510+0000 7f07b1600700 1 -- 172.21.15.90:0/3354831243 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8016070 con 0x5616f66a5e70 2024-05-27T07:53:42.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.510+0000 7f07b1600700 1 -- 172.21.15.90:0/3354831243 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a80c7030 con 0x5616f66a5e70 2024-05-27T07:53:42.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.510+0000 7f07b1600700 1 -- 172.21.15.90:0/3354831243 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a80b2430 con 0x5616f66a5e70 2024-05-27T07:53:42.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.511+0000 7f07d283fb80 1 -- 172.21.15.90:0/3354831243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 msgr2=0x5616f66c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:42.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.511+0000 7f07d283fb80 1 --2- 172.21.15.90:0/3354831243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66c7af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07a80adfd0 tx=0x7f07a80a93b0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.512+0000 7f07d283fb80 1 -- 172.21.15.90:0/3354831243 shutdown_connections 2024-05-27T07:53:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.512+0000 7f07d283fb80 1 --2- 172.21.15.90:0/3354831243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66c7af0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.512+0000 7f07d283fb80 1 --2- 172.21.15.90:0/3354831243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f6526e40 0x5616f66a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.512+0000 7f07d283fb80 1 --2- 172.21.15.90:0/3354831243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f66129f0 0x5616f66c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.512+0000 7f07d283fb80 1 -- 172.21.15.90:0/3354831243 >> 172.21.15.90:0/3354831243 conn(0x5616f65c0390 msgr2=0x5616f66a5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.512+0000 7f07d283fb80 1 -- 172.21.15.90:0/3354831243 shutdown_connections 2024-05-27T07:53:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.513+0000 7f07d283fb80 1 -- 172.21.15.90:0/3354831243 wait complete. 2024-05-27T07:53:42.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.513+0000 7f07d283fb80 1 -- start start 2024-05-27T07:53:42.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.514+0000 7f07d283fb80 1 Processor -- start 2024-05-27T07:53:42.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.514+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 0x5616f66b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.514+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 0x5616f66b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.514+0000 7f07d283fb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.514+0000 7f07d283fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1bfb80 con 0x5616f6526e40 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.514+0000 7f07d283fb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1bf4b0 con 0x5616f66a5e70 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.514+0000 7f07d283fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc1c0250 con 0x5616f66129f0 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.514+0000 7f07d283fb80 10 client.0 _ll_register_callbacks cb 0x5616f656dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.515+0000 7f07bbe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 0x5616f66b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.515+0000 7f07c1c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.515+0000 7f07c1c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34102/0 (socket says 172.21.15.90:34102) 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.515+0000 7f07c1c00700 1 -- 172.21.15.90:0/2073009491 learned_addr learned my addr 172.21.15.90:0/2073009491 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.515+0000 7f07c1200700 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 0x5616f66b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.515+0000 7f07c1c00700 1 -- 172.21.15.90:0/2073009491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 msgr2=0x5616f66b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.517+0000 7f07c1c00700 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 0x5616f66b8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.517+0000 7f07c1c00700 1 -- 172.21.15.90:0/2073009491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 msgr2=0x5616f66b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.517+0000 7f07c1c00700 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 0x5616f66b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.517+0000 7f07c1c00700 1 -- 172.21.15.90:0/2073009491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f66b3d40 con 0x5616f66a5e70 2024-05-27T07:53:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.517+0000 7f07c1200700 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 0x5616f66b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:42.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.518+0000 7f07bbe00700 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 0x5616f66b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:42.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.518+0000 7f07c1c00700 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66b35a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f07a80df610 tx=0x7f07a80a9960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dca921eed5448eeb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.518+0000 7f07a6a00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.518+0000 7f07a6a00700 1 -- 172.21.15.90:0/2073009491 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8016070 con 0x5616f66a5e70 2024-05-27T07:53:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.518+0000 7f07a6a00700 1 -- 172.21.15.90:0/2073009491 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f07a806f1c0 con 0x5616f66a5e70 2024-05-27T07:53:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.518+0000 7f07d283fb80 1 -- 172.21.15.90:0/2073009491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a80c7030 con 0x5616f66a5e70 2024-05-27T07:53:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.519+0000 7f07a6a00700 1 -- 172.21.15.90:0/2073009491 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f07a80a8d10 con 0x5616f66a5e70 2024-05-27T07:53:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.519+0000 7f07d283fb80 10 client.36446 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.519+0000 7f07d283fb80 1 -- 172.21.15.90:0/2073009491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5616f6729f80 con 0x5616f66a5e70 2024-05-27T07:53:42.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.520+0000 7f07a6a00700 1 -- 172.21.15.90:0/2073009491 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f07a80c7030 con 0x5616f66a5e70 2024-05-27T07:53:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.521+0000 7f07a6a00700 1 -- 172.21.15.90:0/2073009491 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7f07a80a54f0 con 0x5616f66a5e70 2024-05-27T07:53:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.521+0000 7f07d283fb80 10 client.36446 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.521+0000 7f07d283fb80 10 client.36446 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:53:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.521+0000 7f07d283fb80 20 client.36446 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.521+0000 7f07d283fb80 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5616f66bfb90 0x5616f672c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.521+0000 7f07d283fb80 4 client.36446 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:53:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.521+0000 7f07d283fb80 1 -- 172.21.15.90:0/2073009491 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616f65cf570 con 0x5616f66bfb90 2024-05-27T07:53:42.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.522+0000 7f07c1200700 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5616f66bfb90 0x5616f672c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:42.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.524+0000 7f07c1200700 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5616f66bfb90 0x5616f672c3f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ae4ce96fbdbabf2 server_cookie=6823d7b271be642a in_seq=0 out_seq=0 2024-05-27T07:53:42.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.524+0000 7f07a6a00700 10 client.36446 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:53:42.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.524+0000 7f07a6a00700 1 -- 172.21.15.90:0/2073009491 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616f65cf570 con 0x5616f66bfb90 2024-05-27T07:53:42.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.524+0000 7f07a6a00700 10 client.36446 handle_command_reply: tid=0 2024-05-27T07:53:42.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.524+0000 7f07d283fb80 1 client.36446 shutdown 2024-05-27T07:53:42.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.524+0000 7f07d283fb80 1 -- 172.21.15.90:0/2073009491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 msgr2=0x5616f66b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:42.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.524+0000 7f07d283fb80 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66b35a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f07a80df610 tx=0x7f07a80a9960 comp rx=0 tx=0).stop 2024-05-27T07:53:42.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.525+0000 7f07d283fb80 1 -- 172.21.15.90:0/2073009491 shutdown_connections 2024-05-27T07:53:42.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.525+0000 7f07d283fb80 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5616f66bfb90 0x5616f672c3f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.525+0000 7f07d283fb80 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616f66a5e70 0x5616f66b35a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.525+0000 7f07d283fb80 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5616f6526e40 0x5616f66b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.525+0000 7f07d283fb80 1 --2- 172.21.15.90:0/2073009491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5616f66129f0 0x5616f66b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.525+0000 7f07d283fb80 1 -- 172.21.15.90:0/2073009491 >> 172.21.15.90:0/2073009491 conn(0x5616f65c0390 msgr2=0x5616f66af560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:42.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.525+0000 7f07a6a00700 0 client.36446 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:53:42.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.525+0000 7f07d283fb80 1 -- 172.21.15.90:0/2073009491 shutdown_connections 2024-05-27T07:53:42.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.525+0000 7f07d283fb80 1 -- 172.21.15.90:0/2073009491 wait complete. 2024-05-27T07:53:42.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.525+0000 7f07d283fb80 20 client.36446 trim_cache size 0 max 16384 2024-05-27T07:53:42.527 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:53:42.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.526+0000 7f07c3c00700 1 -- 172.21.15.90:0/1093645089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f07a00b2df0 msgr2=0x7f07a00b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:42.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.526+0000 7f07c3c00700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f07a00b2df0 0x7f07a00b5300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5616f66a2790 tx=0x7f07b400b7f0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.527+0000 7f07c3c00700 1 -- 172.21.15.90:0/1093645089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc114240 msgr2=0x7f07bc119aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:42.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.527+0000 7f07c3c00700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc114240 0x7f07bc119aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f07a800d720 tx=0x7f07a800da30 comp rx=0 tx=0).stop 2024-05-27T07:53:42.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.528+0000 7f07c3c00700 1 -- 172.21.15.90:0/1093645089 shutdown_connections 2024-05-27T07:53:42.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.528+0000 7f07c3c00700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f07a00b2df0 0x7f07a00b5300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.528+0000 7f07c3c00700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07bc114240 0x7f07bc119aa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.528+0000 7f07c3c00700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07bc10efc0 0x7f07bc119020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.528+0000 7f07c3c00700 1 --2- 172.21.15.90:0/1093645089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07bc1138f0 0x7f07bc119560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:42.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.528+0000 7f07c3c00700 1 -- 172.21.15.90:0/1093645089 >> 172.21.15.90:0/1093645089 conn(0x7f07bc10ab50 msgr2=0x7f07bc112ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:42.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.528+0000 7f07c3c00700 1 -- 172.21.15.90:0/1093645089 shutdown_connections 2024-05-27T07:53:42.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:42.529+0000 7f07c3c00700 1 -- 172.21.15.90:0/1093645089 wait complete. 2024-05-27T07:53:42.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:42 smithi129 ceph-mon[35254]: pgmap v228: 129 pgs: 129 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 11 MiB/s wr, 590 op/s 2024-05-27T07:53:43.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:53:43.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 205.0 s, 517.6 tps, lat 193.205 ms stddev 245.801 2024-05-27T07:53:44.149 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:44.495 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:44 smithi090 ceph-mon[31429]: pgmap v229: 129 pgs: 129 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 13 MiB/s wr, 717 op/s 2024-05-27T07:53:44.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:44 smithi136 ceph-mon[35324]: pgmap v229: 129 pgs: 129 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 13 MiB/s wr, 717 op/s 2024-05-27T07:53:44.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:44 smithi129 ceph-mon[35254]: pgmap v229: 129 pgs: 129 active+clean; 2.3 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 13 MiB/s wr, 717 op/s 2024-05-27T07:53:45.023 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f626be00700 time 2024-05-27T07:53:44.816074+0000 2024-05-27T07:53:45.023 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:53:45.023 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:53:45.023 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f628192cba6] 2024-05-27T07:53:45.023 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f628192cdd1] 2024-05-27T07:53:45.023 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55bef36ec23d] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55bef36d7971] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 5: (SnapRealm::check_cache() const+0x13e) [0x55bef36d5fa0] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55bef36d6651] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55bef366b347] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55bef336787e] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55bef336cf2f] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55bef3377969] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 11: (MDLog::_replay_thread()+0x1f38) [0x55bef371048c] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55bef37199cb] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 13: (Thread::entry_wrapper()+0x43) [0x7f62818fb72d] 2024-05-27T07:53:45.024 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 14: (Thread::_entry_func(void*)+0xd) [0x7f62818fb749] 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f628049f1ca] 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 16: clone() 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:44.819+0000 7f626be00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f626be00700 time 2024-05-27T07:53:44.816074+0000 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f628192cba6] 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f628192cdd1] 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55bef36ec23d] 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55bef36d7971] 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 5: (SnapRealm::check_cache() const+0x13e) [0x55bef36d5fa0] 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55bef36d6651] 2024-05-27T07:53:45.025 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55bef366b347] 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55bef336787e] 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55bef336cf2f] 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55bef3377969] 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 11: (MDLog::_replay_thread()+0x1f38) [0x55bef371048c] 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55bef37199cb] 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 13: (Thread::entry_wrapper()+0x43) [0x7f62818fb72d] 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 14: (Thread::_entry_func(void*)+0xd) [0x7f62818fb749] 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f628049f1ca] 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 16: clone() 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: *** Caught signal (Aborted) ** 2024-05-27T07:53:45.026 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: in thread 7f626be00700 thread_name:md_log_replay 2024-05-27T07:53:45.027 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:53:45.027 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f62804a9d20] 2024-05-27T07:53:45.027 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2: gsignal() 2024-05-27T07:53:45.027 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 3: abort() 2024-05-27T07:53:45.027 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f628192cced] 2024-05-27T07:53:45.028 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f628192cdd1] 2024-05-27T07:53:45.028 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55bef36ec23d] 2024-05-27T07:53:45.028 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55bef36d7971] 2024-05-27T07:53:45.028 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 8: (SnapRealm::check_cache() const+0x13e) [0x55bef36d5fa0] 2024-05-27T07:53:45.028 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55bef36d6651] 2024-05-27T07:53:45.028 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55bef366b347] 2024-05-27T07:53:45.028 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55bef336787e] 2024-05-27T07:53:45.028 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55bef336cf2f] 2024-05-27T07:53:45.028 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55bef3377969] 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 14: (MDLog::_replay_thread()+0x1f38) [0x55bef371048c] 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55bef37199cb] 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 16: (Thread::entry_wrapper()+0x43) [0x7f62818fb72d] 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 17: (Thread::_entry_func(void*)+0xd) [0x7f62818fb749] 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f628049f1ca] 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 19: clone() 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:44.822+0000 7f626be00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: in thread 7f626be00700 thread_name:md_log_replay 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f62804a9d20] 2024-05-27T07:53:45.029 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2: gsignal() 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 3: abort() 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f628192cced] 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f628192cdd1] 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55bef36ec23d] 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55bef36d7971] 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 8: (SnapRealm::check_cache() const+0x13e) [0x55bef36d5fa0] 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55bef36d6651] 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55bef366b347] 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55bef336787e] 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55bef336cf2f] 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55bef3377969] 2024-05-27T07:53:45.030 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 14: (MDLog::_replay_thread()+0x1f38) [0x55bef371048c] 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55bef37199cb] 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 16: (Thread::entry_wrapper()+0x43) [0x7f62818fb72d] 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 17: (Thread::_entry_func(void*)+0xd) [0x7f62818fb749] 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f628049f1ca] 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 19: clone() 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: -1> 2024-05-27T07:53:44.819+0000 7f626be00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f626be00700 time 2024-05-27T07:53:44.816074+0000 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f628192cba6] 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f628192cdd1] 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55bef36ec23d] 2024-05-27T07:53:45.031 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55bef36d7971] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 5: (SnapRealm::check_cache() const+0x13e) [0x55bef36d5fa0] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55bef36d6651] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55bef366b347] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55bef336787e] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55bef336cf2f] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55bef3377969] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 11: (MDLog::_replay_thread()+0x1f38) [0x55bef371048c] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55bef37199cb] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 13: (Thread::entry_wrapper()+0x43) [0x7f62818fb72d] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 14: (Thread::_entry_func(void*)+0xd) [0x7f62818fb749] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f628049f1ca] 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 16: clone() 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.032 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 0> 2024-05-27T07:53:44.822+0000 7f626be00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: in thread 7f626be00700 thread_name:md_log_replay 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f62804a9d20] 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2: gsignal() 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 3: abort() 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f628192cced] 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f628192cdd1] 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55bef36ec23d] 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55bef36d7971] 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 8: (SnapRealm::check_cache() const+0x13e) [0x55bef36d5fa0] 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55bef36d6651] 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55bef366b347] 2024-05-27T07:53:45.033 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55bef336787e] 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55bef336cf2f] 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55bef3377969] 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 14: (MDLog::_replay_thread()+0x1f38) [0x55bef371048c] 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55bef37199cb] 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 16: (Thread::entry_wrapper()+0x43) [0x7f62818fb72d] 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 17: (Thread::_entry_func(void*)+0xd) [0x7f62818fb749] 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f628049f1ca] 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 19: clone() 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: -1> 2024-05-27T07:53:44.819+0000 7f626be00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f626be00700 time 2024-05-27T07:53:44.816074+0000 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:53:45.034 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f628192cba6] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f628192cdd1] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55bef36ec23d] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55bef36d7971] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 5: (SnapRealm::check_cache() const+0x13e) [0x55bef36d5fa0] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55bef36d6651] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55bef366b347] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55bef336787e] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55bef336cf2f] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55bef3377969] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 11: (MDLog::_replay_thread()+0x1f38) [0x55bef371048c] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55bef37199cb] 2024-05-27T07:53:45.035 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 13: (Thread::entry_wrapper()+0x43) [0x7f62818fb72d] 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 14: (Thread::_entry_func(void*)+0xd) [0x7f62818fb749] 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f628049f1ca] 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 16: clone() 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 0> 2024-05-27T07:53:44.822+0000 7f626be00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: in thread 7f626be00700 thread_name:md_log_replay 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f62804a9d20] 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2: gsignal() 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 3: abort() 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f628192cced] 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f628192cdd1] 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55bef36ec23d] 2024-05-27T07:53:45.036 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55bef36d7971] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 8: (SnapRealm::check_cache() const+0x13e) [0x55bef36d5fa0] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55bef36d6651] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55bef366b347] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55bef336787e] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55bef336cf2f] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55bef3377969] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 14: (MDLog::_replay_thread()+0x1f38) [0x55bef371048c] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55bef37199cb] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 16: (Thread::entry_wrapper()+0x43) [0x7f62818fb72d] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 17: (Thread::_entry_func(void*)+0xd) [0x7f62818fb749] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f628049f1ca] 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 19: clone() 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:53:45.037 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[106524]: 2024-05-27T07:53:45.038 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:45 smithi090 podman[109794]: 2024-05-27 07:53:45.021223275 +0000 UTC m=+0.072069072 container died 42e738ac06199e7bde57b90adcf79b8ee997c96bb159dd843b0e0a0eadc1933d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:53:46.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:46 smithi136 ceph-mon[35324]: pgmap v230: 129 pgs: 129 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 12 MiB/s wr, 728 op/s 2024-05-27T07:53:46.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:46 smithi129 ceph-mon[35254]: pgmap v230: 129 pgs: 129 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 12 MiB/s wr, 728 op/s 2024-05-27T07:53:46.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:46 smithi090 ceph-mon[31429]: pgmap v230: 129 pgs: 129 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 12 MiB/s wr, 728 op/s 2024-05-27T07:53:47.213 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:46 smithi090 podman[109794]: 2024-05-27 07:53:46.949659318 +0000 UTC m=+2.000505085 container remove 42e738ac06199e7bde57b90adcf79b8ee997c96bb159dd843b0e0a0eadc1933d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-05-27T07:53:47.213 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:46 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:53:48.105 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:47 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-05-27T07:53:48.105 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:47 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 2.061s CPU time. 2024-05-27T07:53:48.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:48 smithi090 ceph-mon[31429]: pgmap v231: 129 pgs: 129 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 9.6 MiB/s wr, 588 op/s 2024-05-27T07:53:48.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:48 smithi136 ceph-mon[35324]: pgmap v231: 129 pgs: 129 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 9.6 MiB/s wr, 588 op/s 2024-05-27T07:53:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.531+0000 7fd389400700 1 -- 172.21.15.90:0/3555223122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3841054d0 msgr2=0x7fd3841058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.531+0000 7fd389400700 1 --2- 172.21.15.90:0/3555223122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3841054d0 0x7fd3841058e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd374009990 tx=0x7fd374009ca0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.531+0000 7fd389400700 1 -- 172.21.15.90:0/3555223122 shutdown_connections 2024-05-27T07:53:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.531+0000 7fd389400700 1 --2- 172.21.15.90:0/3555223122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd38410e900 0x7fd384110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.531+0000 7fd389400700 1 --2- 172.21.15.90:0/3555223122 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3841054d0 0x7fd3841058e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.531+0000 7fd389400700 1 --2- 172.21.15.90:0/3555223122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd384105e20 0x7fd38410e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.531+0000 7fd389400700 1 -- 172.21.15.90:0/3555223122 >> 172.21.15.90:0/3555223122 conn(0x7fd3840fedb0 msgr2=0x7fd3841011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.531+0000 7fd389400700 1 -- 172.21.15.90:0/3555223122 shutdown_connections 2024-05-27T07:53:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.531+0000 7fd389400700 1 -- 172.21.15.90:0/3555223122 wait complete. 2024-05-27T07:53:48.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.533+0000 7fd389400700 1 Processor -- start 2024-05-27T07:53:48.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.533+0000 7fd389400700 1 -- start start 2024-05-27T07:53:48.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.533+0000 7fd389400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3841054d0 0x7fd38410b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.533+0000 7fd389400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd384105e20 0x7fd38410bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.533+0000 7fd389400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410e900 0x7fd38410c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.533+0000 7fd389400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd384116550 con 0x7fd38410e900 2024-05-27T07:53:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.533+0000 7fd389400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3841163d0 con 0x7fd384105e20 2024-05-27T07:53:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.533+0000 7fd389400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3841166d0 con 0x7fd3841054d0 2024-05-27T07:53:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.534+0000 7fd382800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3841054d0 0x7fd38410b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.534+0000 7fd382800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3841054d0 0x7fd38410b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46096/0 (socket says 172.21.15.90:46096) 2024-05-27T07:53:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.534+0000 7fd382800700 1 -- 172.21.15.90:0/2282412793 learned_addr learned my addr 172.21.15.90:0/2282412793 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:48.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.535+0000 7fd383200700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410e900 0x7fd38410c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.535+0000 7fd381e00700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd384105e20 0x7fd38410bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.535+0000 7fd382800700 1 -- 172.21.15.90:0/2282412793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd384105e20 msgr2=0x7fd38410bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.535+0000 7fd382800700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd384105e20 0x7fd38410bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.535+0000 7fd382800700 1 -- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410e900 msgr2=0x7fd38410c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.535+0000 7fd382800700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410e900 0x7fd38410c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.535+0000 7fd382800700 1 -- 172.21.15.90:0/2282412793 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3841ad480 con 0x7fd3841054d0 2024-05-27T07:53:48.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.536+0000 7fd383200700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410e900 0x7fd38410c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:53:48.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.536+0000 7fd382800700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3841054d0 0x7fd38410b810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd3740098e0 tx=0x7fd374009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:48.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.537+0000 7fd37b400700 1 -- 172.21.15.90:0/2282412793 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374027070 con 0x7fd3841054d0 2024-05-27T07:53:48.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.537+0000 7fd37b400700 1 -- 172.21.15.90:0/2282412793 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd374022470 con 0x7fd3841054d0 2024-05-27T07:53:48.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.537+0000 7fd37b400700 1 -- 172.21.15.90:0/2282412793 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374017650 con 0x7fd3841054d0 2024-05-27T07:53:48.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.539+0000 7fd389400700 1 -- 172.21.15.90:0/2282412793 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3841ad710 con 0x7fd3841054d0 2024-05-27T07:53:48.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.539+0000 7fd389400700 1 -- 172.21.15.90:0/2282412793 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3841adba0 con 0x7fd3841054d0 2024-05-27T07:53:48.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.550+0000 7fd37b400700 1 -- 172.21.15.90:0/2282412793 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd374009d10 con 0x7fd3841054d0 2024-05-27T07:53:48.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.550+0000 7fd398008b80 1 Processor -- start 2024-05-27T07:53:48.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.551+0000 7fd398008b80 1 -- start start 2024-05-27T07:53:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.551+0000 7fd398008b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584309ce9f0 0x558430a61330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.552+0000 7fd398008b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 0x558430a61870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.552+0000 7fd398008b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558430a61e70 0x558430a6e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.552+0000 7fd398008b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55843098fe40 con 0x5584308e2e40 2024-05-27T07:53:48.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.552+0000 7fd398008b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55843097f9e0 con 0x558430a61e70 2024-05-27T07:53:48.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.552+0000 7fd398008b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558430983a20 con 0x5584309ce9f0 2024-05-27T07:53:48.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.553+0000 7fd382800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 0x558430a61870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.553+0000 7fd383200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584309ce9f0 0x558430a61330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.553+0000 7fd37b400700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3680b2b40 0x7fd3680b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.553+0000 7fd381e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558430a61e70 0x558430a6e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.553+0000 7fd382800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 0x558430a61870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39256/0 (socket says 172.21.15.90:39256) 2024-05-27T07:53:48.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.553+0000 7fd382800700 1 -- 172.21.15.90:0/4005439077 learned_addr learned my addr 172.21.15.90:0/4005439077 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:48.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.553+0000 7fd37b400700 1 -- 172.21.15.90:0/2282412793 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fd3740e3640 con 0x7fd3841054d0 2024-05-27T07:53:48.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.554+0000 7fd382800700 1 -- 172.21.15.90:0/4005439077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584309ce9f0 msgr2=0x558430a61330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.554+0000 7fd382800700 1 --2- 172.21.15.90:0/4005439077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584309ce9f0 0x558430a61330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.554+0000 7fd382800700 1 -- 172.21.15.90:0/4005439077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558430a61e70 msgr2=0x558430a6e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.554+0000 7fd382800700 1 --2- 172.21.15.90:0/4005439077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558430a61e70 0x558430a6e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.554+0000 7fd382800700 1 -- 172.21.15.90:0/4005439077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558430948420 con 0x5584308e2e40 2024-05-27T07:53:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.555+0000 7fd382800700 1 --2- 172.21.15.90:0/4005439077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 0x558430a61870 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd3740a7850 tx=0x7fd374009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4fb4f4b8a57594 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.555+0000 7fd36fe00700 1 -- 172.21.15.90:0/4005439077 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3740e3070 con 0x5584308e2e40 2024-05-27T07:53:48.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.555+0000 7fd36fe00700 1 -- 172.21.15.90:0/4005439077 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3740aace0 con 0x5584308e2e40 2024-05-27T07:53:48.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.556+0000 7fd36fe00700 1 -- 172.21.15.90:0/4005439077 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3740aa840 con 0x5584308e2e40 2024-05-27T07:53:48.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.556+0000 7fd383200700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3680b2b40 0x7fd3680b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.557+0000 7fd398008b80 1 -- 172.21.15.90:0/4005439077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 msgr2=0x558430a61870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:48.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.557+0000 7fd398008b80 1 --2- 172.21.15.90:0/4005439077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 0x558430a61870 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd3740a7850 tx=0x7fd374009310 comp rx=0 tx=0).stop 2024-05-27T07:53:48.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.557+0000 7fd398008b80 1 -- 172.21.15.90:0/4005439077 shutdown_connections 2024-05-27T07:53:48.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.558+0000 7fd398008b80 1 --2- 172.21.15.90:0/4005439077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558430a61e70 0x558430a6e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.558+0000 7fd398008b80 1 --2- 172.21.15.90:0/4005439077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584309ce9f0 0x558430a61330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.558+0000 7fd398008b80 1 --2- 172.21.15.90:0/4005439077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 0x558430a61870 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.558+0000 7fd398008b80 1 -- 172.21.15.90:0/4005439077 >> 172.21.15.90:0/4005439077 conn(0x55843097c390 msgr2=0x558430a5d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:48.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.559+0000 7fd398008b80 1 -- 172.21.15.90:0/4005439077 shutdown_connections 2024-05-27T07:53:48.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.560+0000 7fd383200700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3680b2b40 0x7fd3680b5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x558430a63ff0 tx=0x7fd37c004b40 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:48.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.560+0000 7fd398008b80 1 -- 172.21.15.90:0/4005439077 wait complete. 2024-05-27T07:53:48.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.561+0000 7fd398008b80 1 -- start start 2024-05-27T07:53:48.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.561+0000 7fd398008b80 1 Processor -- start 2024-05-27T07:53:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.562+0000 7fd398008b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584308e2e40 0x558430a6b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.562+0000 7fd398008b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 0x558430a6bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.562+0000 7fd398008b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558430a61e70 0x558430a6c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.562+0000 7fd398008b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558430885fa0 con 0x558430a61e70 2024-05-27T07:53:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.562+0000 7fd398008b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558430920760 con 0x5584309ce9f0 2024-05-27T07:53:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.562+0000 7fd398008b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558430a6e940 con 0x5584308e2e40 2024-05-27T07:53:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.562+0000 7fd398008b80 10 client.0 _ll_register_callbacks cb 0x558430929c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:48.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.563+0000 7fd383200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558430a61e70 0x558430a6c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.563+0000 7fd383200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558430a61e70 0x558430a6c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39268/0 (socket says 172.21.15.90:39268) 2024-05-27T07:53:48.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.563+0000 7fd383200700 1 -- 172.21.15.90:0/519471226 learned_addr learned my addr 172.21.15.90:0/519471226 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:48.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.563+0000 7fd381e00700 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 0x558430a6bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:48 smithi129 ceph-mon[35254]: pgmap v231: 129 pgs: 129 active+clean; 2.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 9.6 MiB/s wr, 588 op/s 2024-05-27T07:53:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.564+0000 7fd383200700 1 -- 172.21.15.90:0/519471226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584308e2e40 msgr2=0x558430a6b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.564+0000 7fd383200700 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584308e2e40 0x558430a6b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.564+0000 7fd383200700 1 -- 172.21.15.90:0/519471226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 msgr2=0x558430a6bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.564+0000 7fd383200700 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 0x558430a6bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.564+0000 7fd383200700 1 -- 172.21.15.90:0/519471226 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584308cf8d0 con 0x558430a61e70 2024-05-27T07:53:48.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.564+0000 7fd383200700 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558430a61e70 0x558430a6c200 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0197c0 tx=0x7fd37c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fd7eed1d3bb9ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:48.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.565+0000 7fd36cc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:53:48.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.565+0000 7fd36cc00700 1 -- 172.21.15.90:0/519471226 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c027060 con 0x558430a61e70 2024-05-27T07:53:48.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.566+0000 7fd36cc00700 1 -- 172.21.15.90:0/519471226 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fd37c014040 con 0x558430a61e70 2024-05-27T07:53:48.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.566+0000 7fd398008b80 1 -- 172.21.15.90:0/519471226 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3740aace0 con 0x558430a61e70 2024-05-27T07:53:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.567+0000 7fd36cc00700 1 -- 172.21.15.90:0/519471226 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd37c02e050 con 0x558430a61e70 2024-05-27T07:53:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.567+0000 7fd398008b80 10 client.16908 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.567+0000 7fd398008b80 1 -- 172.21.15.90:0/519471226 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558430950060 con 0x558430a61e70 2024-05-27T07:53:48.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.567+0000 7fd36cc00700 1 -- 172.21.15.90:0/519471226 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3740aace0 con 0x558430a61e70 2024-05-27T07:53:48.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.568+0000 7fd36cc00700 1 -- 172.21.15.90:0/519471226 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7fd37c016400 con 0x558430a61e70 2024-05-27T07:53:48.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.569+0000 7fd398008b80 10 client.16908 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:48.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.569+0000 7fd398008b80 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:48.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.569+0000 7fd398008b80 20 client.16908 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:48.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.569+0000 7fd398008b80 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558430a7b870 0x558430a7dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.569+0000 7fd398008b80 4 client.16908 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:48.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.569+0000 7fd398008b80 1 -- 172.21.15.90:0/519471226 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558430991d90 con 0x558430a7b870 2024-05-27T07:53:48.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.570+0000 7fd382800700 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558430a7b870 0x558430a7dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.571+0000 7fd382800700 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558430a7b870 0x558430a7dcd0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8015952e11c14ed server_cookie=53fd2829d2bc76e4 in_seq=0 out_seq=0 2024-05-27T07:53:48.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.571+0000 7fd36cc00700 10 client.16908 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:48.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.573+0000 7fd36cc00700 1 -- 172.21.15.90:0/519471226 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558430991d90 con 0x558430a7b870 2024-05-27T07:53:48.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.573+0000 7fd36cc00700 10 client.16908 handle_command_reply: tid=0 2024-05-27T07:53:48.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.574+0000 7fd398008b80 1 client.16908 shutdown 2024-05-27T07:53:48.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.574+0000 7fd398008b80 1 -- 172.21.15.90:0/519471226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558430a61e70 msgr2=0x558430a6c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:48.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.574+0000 7fd398008b80 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558430a61e70 0x558430a6c200 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0197c0 tx=0x7fd37c004ad0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.575+0000 7fd398008b80 1 -- 172.21.15.90:0/519471226 shutdown_connections 2024-05-27T07:53:48.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.575+0000 7fd398008b80 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558430a7b870 0x558430a7dcd0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.575+0000 7fd398008b80 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558430a61e70 0x558430a6c200 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.575+0000 7fd398008b80 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5584308e2e40 0x558430a6b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.575+0000 7fd398008b80 1 --2- 172.21.15.90:0/519471226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 0x558430a6bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.575+0000 7fd36cc00700 0 client.16908 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:48.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.575+0000 7fd398008b80 1 -- 172.21.15.90:0/519471226 >> 172.21.15.90:0/519471226 conn(0x55843097c390 msgr2=0x558430a613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:48.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.575+0000 7fd398008b80 1 -- 172.21.15.90:0/519471226 shutdown_connections 2024-05-27T07:53:48.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.576+0000 7fd398008b80 1 -- 172.21.15.90:0/519471226 wait complete. 2024-05-27T07:53:48.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.576+0000 7fd398008b80 20 client.16908 trim_cache size 0 max 16384 2024-05-27T07:53:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.605+0000 7fd398008b80 1 Processor -- start 2024-05-27T07:53:48.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.605+0000 7fd398008b80 1 -- start start 2024-05-27T07:53:48.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.606+0000 7fd398008b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584308e2e40 0x558430a66f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.606+0000 7fd398008b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584309ce9f0 0x558430a617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.606+0000 7fd398008b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558430a61e70 0x558430a779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.606+0000 7fd398008b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3841b86a0 con 0x5584309ce9f0 2024-05-27T07:53:48.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.606+0000 7fd398008b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3841b6490 con 0x5584308e2e40 2024-05-27T07:53:48.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.606+0000 7fd398008b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3841b7fd0 con 0x558430a61e70 2024-05-27T07:53:48.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.607+0000 7fd382800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584308e2e40 0x558430a66f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.607+0000 7fd382800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584308e2e40 0x558430a66f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34134/0 (socket says 172.21.15.90:34134) 2024-05-27T07:53:48.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.607+0000 7fd382800700 1 -- 172.21.15.90:0/3348242671 learned_addr learned my addr 172.21.15.90:0/3348242671 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:48.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.607+0000 7fd381e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584309ce9f0 0x558430a617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.607+0000 7fd382800700 1 -- 172.21.15.90:0/3348242671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558430a61e70 msgr2=0x558430a779a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:53:48.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.607+0000 7fd382800700 1 --2- 172.21.15.90:0/3348242671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558430a61e70 0x558430a779a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.607+0000 7fd382800700 1 -- 172.21.15.90:0/3348242671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584309ce9f0 msgr2=0x558430a617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:48.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.608+0000 7fd382800700 1 --2- 172.21.15.90:0/3348242671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584309ce9f0 0x558430a617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.608+0000 7fd382800700 1 -- 172.21.15.90:0/3348242671 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558430a59240 con 0x5584308e2e40 2024-05-27T07:53:48.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.608+0000 7fd381e00700 1 --2- 172.21.15.90:0/3348242671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584309ce9f0 0x558430a617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:48.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.613+0000 7fd382800700 1 --2- 172.21.15.90:0/3348242671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584308e2e40 0x558430a66f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3740a7850 tx=0x7fd374009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f686441cf7ae5664 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:48.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.614+0000 7fd36f400700 1 -- 172.21.15.90:0/3348242671 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3740e3040 con 0x5584308e2e40 2024-05-27T07:53:48.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.614+0000 7fd36f400700 1 -- 172.21.15.90:0/3348242671 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3740b33b0 con 0x5584308e2e40 2024-05-27T07:53:48.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.614+0000 7fd36f400700 1 -- 172.21.15.90:0/3348242671 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3740aaaa0 con 0x5584308e2e40 2024-05-27T07:53:48.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.616+0000 7fd398008b80 1 -- 172.21.15.90:0/3348242671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584308e2e40 msgr2=0x558430a66f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:48.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.616+0000 7fd398008b80 1 --2- 172.21.15.90:0/3348242671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584308e2e40 0x558430a66f30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3740a7850 tx=0x7fd374009ca0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.616+0000 7fd398008b80 1 -- 172.21.15.90:0/3348242671 shutdown_connections 2024-05-27T07:53:48.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.616+0000 7fd398008b80 1 --2- 172.21.15.90:0/3348242671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558430a61e70 0x558430a779a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.616+0000 7fd398008b80 1 --2- 172.21.15.90:0/3348242671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584308e2e40 0x558430a66f30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.616+0000 7fd398008b80 1 --2- 172.21.15.90:0/3348242671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584309ce9f0 0x558430a617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.616+0000 7fd398008b80 1 -- 172.21.15.90:0/3348242671 >> 172.21.15.90:0/3348242671 conn(0x55843097c390 msgr2=0x558430a6ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:48.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.617+0000 7fd398008b80 1 -- 172.21.15.90:0/3348242671 shutdown_connections 2024-05-27T07:53:48.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.617+0000 7fd398008b80 1 -- 172.21.15.90:0/3348242671 wait complete. 2024-05-27T07:53:48.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.617+0000 7fd398008b80 1 -- start start 2024-05-27T07:53:48.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.618+0000 7fd398008b80 1 Processor -- start 2024-05-27T07:53:48.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.619+0000 7fd398008b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 0x558430a655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.620+0000 7fd398008b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 0x558430a65b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.620+0000 7fd398008b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558430a61e70 0x558430a66060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.620+0000 7fd398008b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3841b7230 con 0x5584308e2e40 2024-05-27T07:53:48.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.620+0000 7fd398008b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3841b6b60 con 0x5584309ce9f0 2024-05-27T07:53:48.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.620+0000 7fd398008b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3841b7900 con 0x558430a61e70 2024-05-27T07:53:48.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.621+0000 7fd381e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 0x558430a65b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.621+0000 7fd398008b80 10 client.0 _ll_register_callbacks cb 0x558430929c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:48.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.621+0000 7fd381e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 0x558430a65b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34148/0 (socket says 172.21.15.90:34148) 2024-05-27T07:53:48.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.622+0000 7fd381e00700 1 -- 172.21.15.90:0/1400289076 learned_addr learned my addr 172.21.15.90:0/1400289076 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:48.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.622+0000 7fd383200700 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558430a61e70 0x558430a66060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.623+0000 7fd381e00700 1 -- 172.21.15.90:0/1400289076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558430a61e70 msgr2=0x558430a66060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:48.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.623+0000 7fd381e00700 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558430a61e70 0x558430a66060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.623+0000 7fd381e00700 1 -- 172.21.15.90:0/1400289076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 msgr2=0x558430a655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:48.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.623+0000 7fd381e00700 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 0x558430a655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.623+0000 7fd381e00700 1 -- 172.21.15.90:0/1400289076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558430a72c90 con 0x5584309ce9f0 2024-05-27T07:53:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.625+0000 7fd381e00700 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 0x558430a65b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd37000ff80 tx=0x7fd3700137f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80103c62d39a2ae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.625+0000 7fd36cc00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.626+0000 7fd36cc00700 1 -- 172.21.15.90:0/1400289076 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370004020 con 0x5584309ce9f0 2024-05-27T07:53:48.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.626+0000 7fd398008b80 1 -- 172.21.15.90:0/1400289076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3740b33b0 con 0x5584309ce9f0 2024-05-27T07:53:48.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.626+0000 7fd36cc00700 1 -- 172.21.15.90:0/1400289076 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fd3700041c0 con 0x5584309ce9f0 2024-05-27T07:53:48.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.629+0000 7fd36cc00700 1 -- 172.21.15.90:0/1400289076 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd370027050 con 0x5584309ce9f0 2024-05-27T07:53:48.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.629+0000 7fd398008b80 10 client.36464 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:48.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.629+0000 7fd398008b80 1 -- 172.21.15.90:0/1400289076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558430a73f20 con 0x5584309ce9f0 2024-05-27T07:53:48.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.630+0000 7fd36cc00700 1 -- 172.21.15.90:0/1400289076 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3740b33b0 con 0x5584309ce9f0 2024-05-27T07:53:48.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.630+0000 7fd36cc00700 1 -- 172.21.15.90:0/1400289076 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7fd370011780 con 0x5584309ce9f0 2024-05-27T07:53:48.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.630+0000 7fd398008b80 10 client.36464 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:48.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.630+0000 7fd398008b80 10 client.36464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:48.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.631+0000 7fd398008b80 20 client.36464 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:48.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.631+0000 7fd398008b80 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558430ae6360 0x558430ae8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:48.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.631+0000 7fd398008b80 4 client.36464 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:53:48.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.631+0000 7fd398008b80 1 -- 172.21.15.90:0/1400289076 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55843098b570 con 0x558430ae6360 2024-05-27T07:53:48.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.633+0000 7fd382800700 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558430ae6360 0x558430ae8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:48.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.634+0000 7fd36cc00700 10 client.36464 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:48.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.636+0000 7fd382800700 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558430ae6360 0x558430ae8780 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d4d8057c14fce6 server_cookie=3f7f00c5d81c4faa in_seq=0 out_seq=0 2024-05-27T07:53:48.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd36cc00700 1 -- 172.21.15.90:0/1400289076 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55843098b570 con 0x558430ae6360 2024-05-27T07:53:48.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd36cc00700 10 client.36464 handle_command_reply: tid=0 2024-05-27T07:53:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd398008b80 1 client.36464 shutdown 2024-05-27T07:53:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd398008b80 1 -- 172.21.15.90:0/1400289076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 msgr2=0x558430a65b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd398008b80 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 0x558430a65b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd37000ff80 tx=0x7fd3700137f0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd398008b80 1 -- 172.21.15.90:0/1400289076 shutdown_connections 2024-05-27T07:53:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd398008b80 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558430ae6360 0x558430ae8780 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd398008b80 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558430a61e70 0x558430a66060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd398008b80 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584308e2e40 0x558430a655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd398008b80 1 --2- 172.21.15.90:0/1400289076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5584309ce9f0 0x558430a65b20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd398008b80 1 -- 172.21.15.90:0/1400289076 >> 172.21.15.90:0/1400289076 conn(0x55843097c390 msgr2=0x558430a6cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:48.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.637+0000 7fd36cc00700 0 client.36464 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:48.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.638+0000 7fd398008b80 1 -- 172.21.15.90:0/1400289076 shutdown_connections 2024-05-27T07:53:48.642 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:53:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.641+0000 7fd398008b80 1 -- 172.21.15.90:0/1400289076 wait complete. 2024-05-27T07:53:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.641+0000 7fd398008b80 20 client.36464 trim_cache size 0 max 16384 2024-05-27T07:53:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 -- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3680b2b40 msgr2=0x7fd3680b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3680b2b40 0x7fd3680b5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x558430a63ff0 tx=0x7fd37c004b40 comp rx=0 tx=0).stop 2024-05-27T07:53:48.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 -- 172.21.15.90:0/2282412793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3841054d0 msgr2=0x7fd38410b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3841054d0 0x7fd38410b810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd3740098e0 tx=0x7fd374009550 comp rx=0 tx=0).stop 2024-05-27T07:53:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 -- 172.21.15.90:0/2282412793 shutdown_connections 2024-05-27T07:53:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd38410e900 0x7fd38410c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3841054d0 0x7fd38410b810 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd3740098e0 tx=0x7fd374009550 comp rx=0 tx=0).stop 2024-05-27T07:53:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3680b2b40 0x7fd3680b5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 --2- 172.21.15.90:0/2282412793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd384105e20 0x7fd38410bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 -- 172.21.15.90:0/2282412793 >> 172.21.15.90:0/2282412793 conn(0x7fd3840fedb0 msgr2=0x7fd3840ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 -- 172.21.15.90:0/2282412793 shutdown_connections 2024-05-27T07:53:48.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:48.642+0000 7fd389400700 1 -- 172.21.15.90:0/2282412793 wait complete. 2024-05-27T07:53:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 210.0 s, 592.6 tps, lat 160.045 ms stddev 94.914 2024-05-27T07:53:50.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:50 smithi129 ceph-mon[35254]: pgmap v232: 129 pgs: 129 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 14 MiB/s wr, 812 op/s 2024-05-27T07:53:50.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:50 smithi090 ceph-mon[31429]: pgmap v232: 129 pgs: 129 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 14 MiB/s wr, 812 op/s 2024-05-27T07:53:50.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:50 smithi136 ceph-mon[35324]: pgmap v232: 129 pgs: 129 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 14 MiB/s wr, 812 op/s 2024-05-27T07:53:51.391 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:53:51.391 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:53:52.124 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:52.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:52 smithi129 ceph-mon[35254]: pgmap v233: 129 pgs: 129 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 8.4 MiB/s wr, 512 op/s 2024-05-27T07:53:52.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:52 smithi090 ceph-mon[31429]: pgmap v233: 129 pgs: 129 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 8.4 MiB/s wr, 512 op/s 2024-05-27T07:53:52.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:52 smithi136 ceph-mon[35324]: pgmap v233: 129 pgs: 129 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 8.4 MiB/s wr, 512 op/s 2024-05-27T07:53:53.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 215.0 s, 614.0 tps, lat 171.308 ms stddev 238.535 2024-05-27T07:53:54.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:54 smithi129 ceph-mon[35254]: pgmap v234: 129 pgs: 129 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 9.9 MiB/s wr, 584 op/s 2024-05-27T07:53:54.654 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:54 smithi090 ceph-mon[31429]: pgmap v234: 129 pgs: 129 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 9.9 MiB/s wr, 584 op/s 2024-05-27T07:53:54.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:54 smithi136 ceph-mon[35324]: pgmap v234: 129 pgs: 129 active+clean; 2.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 9.9 MiB/s wr, 584 op/s 2024-05-27T07:53:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.885+0000 7f57bc800640 1 -- 172.21.15.90:0/1673329789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57b410dc20 msgr2=0x7f57b4110040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.885+0000 7f57bc800640 1 --2- 172.21.15.90:0/1673329789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57b410dc20 0x7f57b4110040 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009900 tx=0x7f57a802a480 comp rx=0 tx=0).stop 2024-05-27T07:53:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.886+0000 7f57bc800640 1 -- 172.21.15.90:0/1673329789 shutdown_connections 2024-05-27T07:53:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.886+0000 7f57bc800640 1 --2- 172.21.15.90:0/1673329789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57b4110580 0x7f57b41129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.886+0000 7f57bc800640 1 --2- 172.21.15.90:0/1673329789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57b410dc20 0x7f57b4110040 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.886+0000 7f57bc800640 1 --2- 172.21.15.90:0/1673329789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b41029e0 0x7f57b4102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.886+0000 7f57bc800640 1 -- 172.21.15.90:0/1673329789 >> 172.21.15.90:0/1673329789 conn(0x7f57b40782d0 msgr2=0x7f57b4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.886+0000 7f57bc800640 1 -- 172.21.15.90:0/1673329789 shutdown_connections 2024-05-27T07:53:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.886+0000 7f57bc800640 1 -- 172.21.15.90:0/1673329789 wait complete. 2024-05-27T07:53:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.888+0000 7f57bc800640 1 Processor -- start 2024-05-27T07:53:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.888+0000 7f57bc800640 1 -- start start 2024-05-27T07:53:54.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.889+0000 7f57bc800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b41029e0 0x7f57b41048a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.889+0000 7f57bc800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57b410dc20 0x7f57b4104de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.889+0000 7f57bc800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57b4110580 0x7f57b4105320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.889+0000 7f57bc800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4115ee0 con 0x7f57b410dc20 2024-05-27T07:53:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.889+0000 7f57bc800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4115d60 con 0x7f57b41029e0 2024-05-27T07:53:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.889+0000 7f57bc800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4116060 con 0x7f57b4110580 2024-05-27T07:53:54.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.890+0000 7f57ba800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57b4110580 0x7f57b4105320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.890+0000 7f57ba800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57b4110580 0x7f57b4105320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:49818/0 (socket says 172.21.15.90:49818) 2024-05-27T07:53:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.890+0000 7f57ba800640 1 -- 172.21.15.90:0/1765789983 learned_addr learned my addr 172.21.15.90:0/1765789983 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.890+0000 7f57b9e00640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b41029e0 0x7f57b41048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.891+0000 7f57b9400640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57b410dc20 0x7f57b4104de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.892+0000 7f57b9e00640 1 -- 172.21.15.90:0/1765789983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57b4110580 msgr2=0x7f57b4105320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.892+0000 7f57b9e00640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57b4110580 0x7f57b4105320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.892+0000 7f57b9e00640 1 -- 172.21.15.90:0/1765789983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57b410dc20 msgr2=0x7f57b4104de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.892+0000 7f57b9e00640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57b410dc20 0x7f57b4104de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.892+0000 7f57b9e00640 1 -- 172.21.15.90:0/1765789983 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b41acf90 con 0x7f57b41029e0 2024-05-27T07:53:54.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.892+0000 7f57b9e00640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b41029e0 0x7f57b41048a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57a400c970 tx=0x7f57a400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.893+0000 7f57aea00640 1 -- 172.21.15.90:0/1765789983 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4007bb0 con 0x7f57b41029e0 2024-05-27T07:53:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.893+0000 7f57aea00640 1 -- 172.21.15.90:0/1765789983 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57a4007d50 con 0x7f57b41029e0 2024-05-27T07:53:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.893+0000 7f57aea00640 1 -- 172.21.15.90:0/1765789983 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a40186f0 con 0x7f57b41029e0 2024-05-27T07:53:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.893+0000 7f57bc800640 1 -- 172.21.15.90:0/1765789983 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b41ad1c0 con 0x7f57b41029e0 2024-05-27T07:53:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.894+0000 7f57bc800640 1 -- 172.21.15.90:0/1765789983 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b41ad690 con 0x7f57b41029e0 2024-05-27T07:53:54.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.907+0000 7f57c396a740 1 Processor -- start 2024-05-27T07:53:54.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.907+0000 7f57c396a740 1 -- start start 2024-05-27T07:53:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.908+0000 7f57aea00640 1 -- 172.21.15.90:0/1765789983 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f57a400f450 con 0x7f57b41029e0 2024-05-27T07:53:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.908+0000 7f57c396a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c66b71b0 0x55c3c6a59860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.908+0000 7f57c396a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c668e2f0 0x55c3c6a59da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.908+0000 7f57c396a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a5a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.908+0000 7f57c396a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c3c69cab90 con 0x55c3c66b71b0 2024-05-27T07:53:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.908+0000 7f57c396a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c3c69ba730 con 0x55c3c66964d0 2024-05-27T07:53:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.908+0000 7f57c396a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c3c69be770 con 0x55c3c668e2f0 2024-05-27T07:53:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.909+0000 7f57b9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a5a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.909+0000 7f57b9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a5a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33694/0 (socket says 172.21.15.90:33694) 2024-05-27T07:53:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.909+0000 7f57b9400640 1 -- 172.21.15.90:0/4031171419 learned_addr learned my addr 172.21.15.90:0/4031171419 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.910+0000 7f57aea00640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f579c0b2a50 0x7f579c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.910+0000 7f57aea00640 1 -- 172.21.15.90:0/1765789983 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f57a40e2a00 con 0x7f57b41029e0 2024-05-27T07:53:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.911+0000 7f57b9e00640 1 --2- 172.21.15.90:0/4031171419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c668e2f0 0x55c3c6a59da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.911+0000 7f57ba800640 1 --2- 172.21.15.90:0/4031171419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c66b71b0 0x55c3c6a59860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.912+0000 7f57b9400640 1 -- 172.21.15.90:0/4031171419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c668e2f0 msgr2=0x55c3c6a59da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.912+0000 7f57b9400640 1 --2- 172.21.15.90:0/4031171419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c668e2f0 0x55c3c6a59da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.912+0000 7f57b9400640 1 -- 172.21.15.90:0/4031171419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c66b71b0 msgr2=0x55c3c6a59860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.912+0000 7f57b9400640 1 --2- 172.21.15.90:0/4031171419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c66b71b0 0x55c3c6a59860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.912+0000 7f57b9400640 1 -- 172.21.15.90:0/4031171419 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c69538b0 con 0x55c3c66964d0 2024-05-27T07:53:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.913+0000 7f57ba800640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f579c0b2a50 0x7f579c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.914+0000 7f57b9400640 1 --2- 172.21.15.90:0/4031171419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a5a2e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f57a8000c00 tx=0x7f57a8031d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f1666c39310729c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.915+0000 7f57a3000640 1 -- 172.21.15.90:0/4031171419 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8035030 con 0x55c3c66964d0 2024-05-27T07:53:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.915+0000 7f57a3000640 1 -- 172.21.15.90:0/4031171419 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57a8040070 con 0x55c3c66964d0 2024-05-27T07:53:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.915+0000 7f57a3000640 1 -- 172.21.15.90:0/4031171419 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8003b50 con 0x55c3c66964d0 2024-05-27T07:53:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.917+0000 7f57c396a740 1 -- 172.21.15.90:0/4031171419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 msgr2=0x55c3c6a5a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.917+0000 7f57c396a740 1 --2- 172.21.15.90:0/4031171419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a5a2e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f57a8000c00 tx=0x7f57a8031d50 comp rx=0 tx=0).stop 2024-05-27T07:53:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.917+0000 7f57ba800640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f579c0b2a50 0x7f579c0b4f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55c3c6a569d0 tx=0x7f57b0007560 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.917+0000 7f57c396a740 1 -- 172.21.15.90:0/4031171419 shutdown_connections 2024-05-27T07:53:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.917+0000 7f57c396a740 1 --2- 172.21.15.90:0/4031171419 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a5a2e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.917+0000 7f57c396a740 1 --2- 172.21.15.90:0/4031171419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c668e2f0 0x55c3c6a59da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.917+0000 7f57c396a740 1 --2- 172.21.15.90:0/4031171419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c66b71b0 0x55c3c6a59860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.917+0000 7f57c396a740 1 -- 172.21.15.90:0/4031171419 >> 172.21.15.90:0/4031171419 conn(0x55c3c66ab300 msgr2=0x55c3c6a56010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.918+0000 7f57c396a740 1 -- 172.21.15.90:0/4031171419 shutdown_connections 2024-05-27T07:53:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.918+0000 7f57c396a740 1 -- 172.21.15.90:0/4031171419 wait complete. 2024-05-27T07:53:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.920+0000 7f57c396a740 1 -- start start 2024-05-27T07:53:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.921+0000 7f57c396a740 1 Processor -- start 2024-05-27T07:53:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.923+0000 7f57c396a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c668e2f0 0x55c3c6a639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.923+0000 7f57c396a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66964d0 0x55c3c6a63f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.923+0000 7f57c396a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c66b71b0 0x55c3c6a6fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.923+0000 7f57c396a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c3c66a0790 con 0x55c3c66b71b0 2024-05-27T07:53:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.923+0000 7f57c396a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c3c6962bc0 con 0x55c3c668e2f0 2024-05-27T07:53:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.923+0000 7f57c396a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c3c6a66930 con 0x55c3c66964d0 2024-05-27T07:53:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.923+0000 7f57c396a740 10 client.0 _ll_register_callbacks cb 0x55c3c6940400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.924+0000 7f57b9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c668e2f0 0x55c3c6a639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.924+0000 7f57b9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c668e2f0 0x55c3c6a639e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33702/0 (socket says 172.21.15.90:33702) 2024-05-27T07:53:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.924+0000 7f57b9e00640 1 -- 172.21.15.90:0/1834988281 learned_addr learned my addr 172.21.15.90:0/1834988281 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.925+0000 7f57b9e00640 1 -- 172.21.15.90:0/1834988281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66964d0 msgr2=0x55c3c6a63f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:53:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.925+0000 7f57b9400640 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66964d0 0x55c3c6a63f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.925+0000 7f57b9e00640 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66964d0 0x55c3c6a63f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.925+0000 7f57b9e00640 1 -- 172.21.15.90:0/1834988281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c66b71b0 msgr2=0x55c3c6a6fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.925+0000 7f57b9e00640 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c66b71b0 0x55c3c6a6fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.925+0000 7f57b9e00640 1 -- 172.21.15.90:0/1834988281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c66cf210 con 0x55c3c668e2f0 2024-05-27T07:53:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.928+0000 7f57b9e00640 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c668e2f0 0x55c3c6a639e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f57a40dfd80 tx=0x7f57a400f970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f07aad804a49dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.928+0000 7f5797400640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.928+0000 7f5797400640 1 -- 172.21.15.90:0/1834988281 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a40adc60 con 0x55c3c668e2f0 2024-05-27T07:53:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.929+0000 7f5797400640 1 -- 172.21.15.90:0/1834988281 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f57a40dffa0 con 0x55c3c668e2f0 2024-05-27T07:53:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.930+0000 7f57c396a740 1 -- 172.21.15.90:0/1834988281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a8040070 con 0x55c3c668e2f0 2024-05-27T07:53:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.933+0000 7f5797400640 1 -- 172.21.15.90:0/1834988281 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f57a40e01a0 con 0x55c3c668e2f0 2024-05-27T07:53:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.933+0000 7f57c396a740 10 client.36473 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.933+0000 7f57c396a740 1 -- 172.21.15.90:0/1834988281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c3c679b620 con 0x55c3c668e2f0 2024-05-27T07:53:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.936+0000 7f5797400640 1 -- 172.21.15.90:0/1834988281 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57a8040070 con 0x55c3c668e2f0 2024-05-27T07:53:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.937+0000 7f5797400640 1 -- 172.21.15.90:0/1834988281 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7f57a40a8b10 con 0x55c3c668e2f0 2024-05-27T07:53:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.937+0000 7f57c396a740 10 client.36473 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.937+0000 7f57c396a740 10 client.36473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.938+0000 7f57c396a740 20 client.36473 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.938+0000 7f57c396a740 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c3c6675d80 0x55c3c6a75d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.938+0000 7f57c396a740 4 client.36473 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.938+0000 7f57c396a740 1 -- 172.21.15.90:0/1834988281 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3c69ccae0 con 0x55c3c6675d80 2024-05-27T07:53:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.939+0000 7f57b9400640 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c3c6675d80 0x55c3c6a75d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.942+0000 7f57b9400640 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c3c6675d80 0x55c3c6a75d10 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8115013c8fb6a0f0 server_cookie=c568ba8fa7c0b6f6 in_seq=0 out_seq=0 2024-05-27T07:53:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.942+0000 7f5797400640 10 client.36473 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.943+0000 7f5797400640 1 -- 172.21.15.90:0/1834988281 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c3c69ccae0 con 0x55c3c6675d80 2024-05-27T07:53:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.943+0000 7f5797400640 10 client.36473 handle_command_reply: tid=0 2024-05-27T07:53:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.943+0000 7f57c396a740 1 client.36473 shutdown 2024-05-27T07:53:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.945+0000 7f57c396a740 1 -- 172.21.15.90:0/1834988281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c668e2f0 msgr2=0x55c3c6a639e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:54.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.945+0000 7f57c396a740 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c668e2f0 0x55c3c6a639e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f57a40dfd80 tx=0x7f57a400f970 comp rx=0 tx=0).stop 2024-05-27T07:53:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.950+0000 7f57c396a740 1 -- 172.21.15.90:0/1834988281 shutdown_connections 2024-05-27T07:53:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.950+0000 7f57c396a740 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c3c6675d80 0x55c3c6a75d10 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.950+0000 7f57c396a740 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c66b71b0 0x55c3c6a6fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.950+0000 7f57c396a740 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66964d0 0x55c3c6a63f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.950+0000 7f57c396a740 1 --2- 172.21.15.90:0/1834988281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c668e2f0 0x55c3c6a639e0 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f57a40dfd80 tx=0x7f57a400f970 comp rx=0 tx=0).stop 2024-05-27T07:53:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.950+0000 7f57c396a740 1 -- 172.21.15.90:0/1834988281 >> 172.21.15.90:0/1834988281 conn(0x55c3c66ab300 msgr2=0x55c3c6a5b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.950+0000 7f5797400640 0 client.36473 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.952+0000 7f57c396a740 1 -- 172.21.15.90:0/1834988281 shutdown_connections 2024-05-27T07:53:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.953+0000 7f57c396a740 1 -- 172.21.15.90:0/1834988281 wait complete. 2024-05-27T07:53:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:54.953+0000 7f57c396a740 20 client.36473 trim_cache size 0 max 16384 2024-05-27T07:53:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.009+0000 7f57c396a740 1 Processor -- start 2024-05-27T07:53:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.010+0000 7f57c396a740 1 -- start start 2024-05-27T07:53:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.012+0000 7f57c396a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a6f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.012+0000 7f57c396a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66b71b0 0x55c3c6a6ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.012+0000 7f57c396a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 0x55c3c6a5d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.012+0000 7f57c396a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41b0b90 con 0x55c3c664b1c0 2024-05-27T07:53:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.012+0000 7f57c396a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41ae980 con 0x55c3c66964d0 2024-05-27T07:53:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.012+0000 7f57c396a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41b04c0 con 0x55c3c66b71b0 2024-05-27T07:53:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.012+0000 7f57b9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a6f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.015+0000 7f57b9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a6f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33714/0 (socket says 172.21.15.90:33714) 2024-05-27T07:53:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.015+0000 7f57b9e00640 1 -- 172.21.15.90:0/4123670788 learned_addr learned my addr 172.21.15.90:0/4123670788 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.012+0000 7f57b9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66b71b0 0x55c3c6a6ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.013+0000 7f57ba800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 0x55c3c6a5d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.016+0000 7f57b9e00640 1 -- 172.21.15.90:0/4123670788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66b71b0 msgr2=0x55c3c6a6ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.016+0000 7f57b9e00640 1 --2- 172.21.15.90:0/4123670788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66b71b0 0x55c3c6a6ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.016+0000 7f57b9e00640 1 -- 172.21.15.90:0/4123670788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 msgr2=0x55c3c6a5d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.016+0000 7f57b9e00640 1 --2- 172.21.15.90:0/4123670788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 0x55c3c6a5d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.016+0000 7f57b9e00640 1 -- 172.21.15.90:0/4123670788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c6a56c00 con 0x55c3c66964d0 2024-05-27T07:53:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.016+0000 7f57b9e00640 1 --2- 172.21.15.90:0/4123670788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a6f9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f57a40a95b0 tx=0x7f57a40a8f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cdc75a387ee0133 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.016+0000 7f57a2600640 1 -- 172.21.15.90:0/4123670788 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a40a5590 con 0x55c3c66964d0 2024-05-27T07:53:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.017+0000 7f57a2600640 1 -- 172.21.15.90:0/4123670788 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57a40a9fb0 con 0x55c3c66964d0 2024-05-27T07:53:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.017+0000 7f57a2600640 1 -- 172.21.15.90:0/4123670788 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a40b2280 con 0x55c3c66964d0 2024-05-27T07:53:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.018+0000 7f57c396a740 1 -- 172.21.15.90:0/4123670788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 msgr2=0x55c3c6a6f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.018+0000 7f57c396a740 1 --2- 172.21.15.90:0/4123670788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a6f9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f57a40a95b0 tx=0x7f57a40a8f70 comp rx=0 tx=0).stop 2024-05-27T07:53:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.020+0000 7f57c396a740 1 -- 172.21.15.90:0/4123670788 shutdown_connections 2024-05-27T07:53:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.020+0000 7f57c396a740 1 --2- 172.21.15.90:0/4123670788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 0x55c3c6a5d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.021+0000 7f57c396a740 1 --2- 172.21.15.90:0/4123670788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66b71b0 0x55c3c6a6ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.021+0000 7f57c396a740 1 --2- 172.21.15.90:0/4123670788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66964d0 0x55c3c6a6f9d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.021+0000 7f57c396a740 1 -- 172.21.15.90:0/4123670788 >> 172.21.15.90:0/4123670788 conn(0x55c3c66ab300 msgr2=0x55c3c6a63cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.021+0000 7f57c396a740 1 -- 172.21.15.90:0/4123670788 shutdown_connections 2024-05-27T07:53:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.021+0000 7f57c396a740 1 -- 172.21.15.90:0/4123670788 wait complete. 2024-05-27T07:53:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.022+0000 7f57c396a740 1 -- start start 2024-05-27T07:53:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.023+0000 7f57c396a740 1 Processor -- start 2024-05-27T07:53:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.024+0000 7f57c396a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 0x55c3c6a6c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.025+0000 7f57c396a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66964d0 0x55c3c6a676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.025+0000 7f57c396a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66b71b0 0x55c3c6a67c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.025+0000 7f57c396a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41af720 con 0x55c3c664b1c0 2024-05-27T07:53:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.025+0000 7f57c396a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41af050 con 0x55c3c66b71b0 2024-05-27T07:53:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.025+0000 7f57c396a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41afdf0 con 0x55c3c66964d0 2024-05-27T07:53:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.025+0000 7f57ba800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66b71b0 0x55c3c6a67c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.025+0000 7f57ba800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66b71b0 0x55c3c6a67c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33720/0 (socket says 172.21.15.90:33720) 2024-05-27T07:53:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.025+0000 7f57ba800640 1 -- 172.21.15.90:0/2370283588 learned_addr learned my addr 172.21.15.90:0/2370283588 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.026+0000 7f57c396a740 10 client.0 _ll_register_callbacks cb 0x7f57a40b20b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.026+0000 7f57b9e00640 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 0x55c3c6a6c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.027+0000 7f57b9400640 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66964d0 0x55c3c6a676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.027+0000 7f57ba800640 1 -- 172.21.15.90:0/2370283588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66964d0 msgr2=0x55c3c6a676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.027+0000 7f57ba800640 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66964d0 0x55c3c6a676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.027+0000 7f57ba800640 1 -- 172.21.15.90:0/2370283588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 msgr2=0x55c3c6a6c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.027+0000 7f57ba800640 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 0x55c3c6a6c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.027+0000 7f57ba800640 1 -- 172.21.15.90:0/2370283588 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c6a683d0 con 0x55c3c66b71b0 2024-05-27T07:53:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.029+0000 7f57b9e00640 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 0x55c3c6a6c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.029+0000 7f57ba800640 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66b71b0 0x55c3c6a67c40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f57b41110a0 tx=0x7f57b0007a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71abce5bc67678f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.029+0000 7f5797400640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.030+0000 7f5797400640 1 -- 172.21.15.90:0/2370283588 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0027030 con 0x55c3c66b71b0 2024-05-27T07:53:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.030+0000 7f5797400640 1 -- 172.21.15.90:0/2370283588 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f57b001d050 con 0x55c3c66b71b0 2024-05-27T07:53:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.030+0000 7f57c396a740 1 -- 172.21.15.90:0/2370283588 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a40a9fb0 con 0x55c3c66b71b0 2024-05-27T07:53:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.031+0000 7f5797400640 1 -- 172.21.15.90:0/2370283588 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f57b0018040 con 0x55c3c66b71b0 2024-05-27T07:53:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.031+0000 7f57c396a740 10 client.36479 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.031+0000 7f57c396a740 1 -- 172.21.15.90:0/2370283588 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c3c6ade370 con 0x55c3c66b71b0 2024-05-27T07:53:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.032+0000 7f5797400640 1 -- 172.21.15.90:0/2370283588 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57a40a9fb0 con 0x55c3c66b71b0 2024-05-27T07:53:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.033+0000 7f5797400640 1 -- 172.21.15.90:0/2370283588 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7f57b0022c40 con 0x55c3c66b71b0 2024-05-27T07:53:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.033+0000 7f57c396a740 10 client.36479 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.033+0000 7f57c396a740 10 client.36479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.033+0000 7f57c396a740 20 client.36479 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.034+0000 7f57c396a740 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c3c6adeac0 0x55c3c6ae0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.034+0000 7f57c396a740 4 client.36479 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:53:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.034+0000 7f57c396a740 1 -- 172.21.15.90:0/2370283588 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3c69c62c0 con 0x55c3c6adeac0 2024-05-27T07:53:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.035+0000 7f57b9e00640 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c3c6adeac0 0x55c3c6ae0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.035+0000 7f57b9e00640 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c3c6adeac0 0x55c3c6ae0f00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294d0e7f5eb21d05 server_cookie=d87340188b207f18 in_seq=0 out_seq=0 2024-05-27T07:53:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.036+0000 7f5797400640 10 client.36479 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.130+0000 7f5797400640 1 -- 172.21.15.90:0/2370283588 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c3c69c62c0 con 0x55c3c6adeac0 2024-05-27T07:53:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.130+0000 7f5797400640 10 client.36479 handle_command_reply: tid=0 2024-05-27T07:53:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.131+0000 7f57c396a740 1 client.36479 shutdown 2024-05-27T07:53:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.131+0000 7f57c396a740 1 -- 172.21.15.90:0/2370283588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66b71b0 msgr2=0x55c3c6a67c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.131+0000 7f57c396a740 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66b71b0 0x55c3c6a67c40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f57b41110a0 tx=0x7f57b0007a10 comp rx=0 tx=0).stop 2024-05-27T07:53:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.131+0000 7f57c396a740 1 -- 172.21.15.90:0/2370283588 shutdown_connections 2024-05-27T07:53:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.131+0000 7f57c396a740 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c3c6adeac0 0x55c3c6ae0f00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.131+0000 7f57c396a740 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3c66b71b0 0x55c3c6a67c40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.131+0000 7f57c396a740 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c3c66964d0 0x55c3c6a676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.131+0000 7f57c396a740 1 --2- 172.21.15.90:0/2370283588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c3c664b1c0 0x55c3c6a6c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.131+0000 7f57c396a740 1 -- 172.21.15.90:0/2370283588 >> 172.21.15.90:0/2370283588 conn(0x55c3c66ab300 msgr2=0x55c3c6a595f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.132+0000 7f5797400640 0 client.36479 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.132+0000 7f57c396a740 1 -- 172.21.15.90:0/2370283588 shutdown_connections 2024-05-27T07:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.132+0000 7f57c396a740 1 -- 172.21.15.90:0/2370283588 wait complete. 2024-05-27T07:53:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.132+0000 7f57c396a740 20 client.36479 trim_cache size 0 max 16384 2024-05-27T07:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.134+0000 7f57bc800640 1 -- 172.21.15.90:0/1765789983 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f579c0b2a50 msgr2=0x7f579c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.134+0000 7f57bc800640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f579c0b2a50 0x7f579c0b4f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55c3c6a569d0 tx=0x7f57b0007560 comp rx=0 tx=0).stop 2024-05-27T07:53:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.134+0000 7f57bc800640 1 -- 172.21.15.90:0/1765789983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b41029e0 msgr2=0x7f57b41048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.134+0000 7f57bc800640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b41029e0 0x7f57b41048a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f57a400c970 tx=0x7f57a400ce30 comp rx=0 tx=0).stop 2024-05-27T07:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.135+0000 7f57bc800640 1 -- 172.21.15.90:0/1765789983 shutdown_connections 2024-05-27T07:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.135+0000 7f57bc800640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f579c0b2a50 0x7f579c0b4f60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.135+0000 7f57bc800640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57b4110580 0x7f57b4105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.135+0000 7f57bc800640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57b410dc20 0x7f57b4104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.135+0000 7f57bc800640 1 --2- 172.21.15.90:0/1765789983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57b41029e0 0x7f57b41048a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.135+0000 7f57bc800640 1 -- 172.21.15.90:0/1765789983 >> 172.21.15.90:0/1765789983 conn(0x7f57b40782d0 msgr2=0x7f57b41012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.136+0000 7f57bc800640 1 -- 172.21.15.90:0/1765789983 shutdown_connections 2024-05-27T07:53:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:53:55.136+0000 7f57bc800640 1 -- 172.21.15.90:0/1765789983 wait complete. 2024-05-27T07:53:56.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.450+0000 7efccb200700 1 -- 172.21.15.90:0/968283450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efcc410a920 msgr2=0x7efcc410ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:56.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.450+0000 7efccb200700 1 --2- 172.21.15.90:0/968283450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efcc410a920 0x7efcc410ad30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7efcb400b0a0 tx=0x7efcb400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.450+0000 7efccb200700 1 -- 172.21.15.90:0/968283450 shutdown_connections 2024-05-27T07:53:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.450+0000 7efccb200700 1 --2- 172.21.15.90:0/968283450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc411d920 0x7efcc411fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.450+0000 7efccb200700 1 --2- 172.21.15.90:0/968283450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efcc410a920 0x7efcc410ad30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.450+0000 7efccb200700 1 --2- 172.21.15.90:0/968283450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc410b270 0x7efcc411d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.450+0000 7efccb200700 1 -- 172.21.15.90:0/968283450 >> 172.21.15.90:0/968283450 conn(0x7efcc406f4c0 msgr2=0x7efcc406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.450+0000 7efccb200700 1 -- 172.21.15.90:0/968283450 shutdown_connections 2024-05-27T07:53:56.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.450+0000 7efccb200700 1 -- 172.21.15.90:0/968283450 wait complete. 2024-05-27T07:53:56.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:56 smithi090 ceph-mon[31429]: pgmap v235: 129 pgs: 129 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 11 MiB/s wr, 651 op/s 2024-05-27T07:53:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.453+0000 7efccb200700 1 Processor -- start 2024-05-27T07:53:56.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.454+0000 7efccb200700 1 -- start start 2024-05-27T07:53:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.456+0000 7efccb200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc410a920 0x7efcc41af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.456+0000 7efccb200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efcc410b270 0x7efcc41afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.456+0000 7efccb200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc411d920 0x7efcc41a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.457+0000 7efccb200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efcc4123200 con 0x7efcc410b270 2024-05-27T07:53:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.457+0000 7efccb200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efcc4123080 con 0x7efcc411d920 2024-05-27T07:53:56.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.457+0000 7efccb200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efcc4123380 con 0x7efcc410a920 2024-05-27T07:53:56.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.457+0000 7efcc9200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc411d920 0x7efcc41a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.457+0000 7efcc9200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc411d920 0x7efcc41a99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33746/0 (socket says 172.21.15.90:33746) 2024-05-27T07:53:56.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.457+0000 7efcc9200700 1 -- 172.21.15.90:0/774708839 learned_addr learned my addr 172.21.15.90:0/774708839 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.458+0000 7efcc3400700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efcc410b270 0x7efcc41afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.458+0000 7efcc9200700 1 -- 172.21.15.90:0/774708839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc410a920 msgr2=0x7efcc41af780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:53:56.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.458+0000 7efcc9200700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc410a920 0x7efcc41af780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.458+0000 7efcc9200700 1 -- 172.21.15.90:0/774708839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efcc410b270 msgr2=0x7efcc41afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:56.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.458+0000 7efcc9200700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efcc410b270 0x7efcc41afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.458+0000 7efcc9200700 1 -- 172.21.15.90:0/774708839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc41aa050 con 0x7efcc411d920 2024-05-27T07:53:56.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.459+0000 7efcc9200700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc411d920 0x7efcc41a99a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00d950 tx=0x7efcbc00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:56.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.460+0000 7efcc0c00700 1 -- 172.21.15.90:0/774708839 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc017070 con 0x7efcc411d920 2024-05-27T07:53:56.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.460+0000 7efcc0c00700 1 -- 172.21.15.90:0/774708839 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcbc009dd0 con 0x7efcc411d920 2024-05-27T07:53:56.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.460+0000 7efcc0c00700 1 -- 172.21.15.90:0/774708839 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc005280 con 0x7efcc411d920 2024-05-27T07:53:56.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.460+0000 7efccb200700 1 -- 172.21.15.90:0/774708839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc41aa340 con 0x7efcc411d920 2024-05-27T07:53:56.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.460+0000 7efccb200700 1 -- 172.21.15.90:0/774708839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc41b6610 con 0x7efcc411d920 2024-05-27T07:53:56.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.469+0000 7efcd9c53b80 1 Processor -- start 2024-05-27T07:53:56.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.469+0000 7efcc0c00700 1 -- 172.21.15.90:0/774708839 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efcbc0049c0 con 0x7efcc411d920 2024-05-27T07:53:56.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.470+0000 7efcd9c53b80 1 -- start start 2024-05-27T07:53:56.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.471+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddef8ceec0 0x55ddefa53870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.471+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddefa53db0 0x55ddefa4d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.471+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa4e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.471+0000 7efcd9c53b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ddef97bec0 con 0x55ddef8ceec0 2024-05-27T07:53:56.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.471+0000 7efcd9c53b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ddef96ba60 con 0x55ddefa53db0 2024-05-27T07:53:56.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.471+0000 7efcd9c53b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ddef96faa0 con 0x55ddefa4de30 2024-05-27T07:53:56.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.471+0000 7efcc3e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddefa53db0 0x55ddefa4d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.472+0000 7efcc3e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddefa53db0 0x55ddefa4d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33748/0 (socket says 172.21.15.90:33748) 2024-05-27T07:53:56.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.472+0000 7efcc3e00700 1 -- 172.21.15.90:0/2681484004 learned_addr learned my addr 172.21.15.90:0/2681484004 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:56.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.472+0000 7efcc3e00700 1 -- 172.21.15.90:0/2681484004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 msgr2=0x55ddefa4e240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:53:56.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.472+0000 7efcc3e00700 1 --2- 172.21.15.90:0/2681484004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa4e240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.472+0000 7efcc3e00700 1 -- 172.21.15.90:0/2681484004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddef8ceec0 msgr2=0x55ddefa53870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:53:56.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.473+0000 7efcc3400700 1 --2- 172.21.15.90:0/2681484004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddef8ceec0 0x55ddefa53870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.474+0000 7efcc3e00700 1 --2- 172.21.15.90:0/2681484004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddef8ceec0 0x55ddefa53870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.474+0000 7efcc3e00700 1 -- 172.21.15.90:0/2681484004 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddef9344a0 con 0x55ddefa53db0 2024-05-27T07:53:56.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.476+0000 7efcc0c00700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efca80b2aa0 0x7efca80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.476+0000 7efcc3e00700 1 --2- 172.21.15.90:0/2681484004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddefa53db0 0x55ddefa4d8f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efcb40098b0 tx=0x7efcb40090d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a237d44575db6c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:56.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.477+0000 7efcc0c00700 1 -- 172.21.15.90:0/774708839 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7efcbc0e2240 con 0x7efcc411d920 2024-05-27T07:53:56.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.477+0000 7efcb9600700 1 -- 172.21.15.90:0/2681484004 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb401f030 con 0x55ddefa53db0 2024-05-27T07:53:56.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.477+0000 7efcb9600700 1 -- 172.21.15.90:0/2681484004 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcb4009320 con 0x55ddefa53db0 2024-05-27T07:53:56.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.477+0000 7efcb9600700 1 -- 172.21.15.90:0/2681484004 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4007d10 con 0x55ddefa53db0 2024-05-27T07:53:56.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.477+0000 7efcc3400700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efca80b2aa0 0x7efca80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.479+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2681484004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddefa53db0 msgr2=0x55ddefa4d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:56.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.479+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2681484004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddefa53db0 0x55ddefa4d8f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efcb40098b0 tx=0x7efcb40090d0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.479+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2681484004 shutdown_connections 2024-05-27T07:53:56.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.479+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2681484004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa4e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.479+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2681484004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddef8ceec0 0x55ddefa53870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.479+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2681484004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddefa53db0 0x55ddefa4d8f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.479+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2681484004 >> 172.21.15.90:0/2681484004 conn(0x55ddef92f040 msgr2=0x55ddefa48950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:56.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.480+0000 7efcc3400700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efca80b2aa0 0x7efca80b4fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efcb0005d90 tx=0x7efcb0005d00 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:56.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.480+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2681484004 shutdown_connections 2024-05-27T07:53:56.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.480+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2681484004 wait complete. 2024-05-27T07:53:56.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.481+0000 7efcd9c53b80 1 -- start start 2024-05-27T07:53:56.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.481+0000 7efcd9c53b80 1 Processor -- start 2024-05-27T07:53:56.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.482+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.482+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa57a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.482+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa53db0 0x55ddefa57f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.482+0000 7efcd9c53b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ddef872020 con 0x55ddefa53db0 2024-05-27T07:53:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.482+0000 7efcd9c53b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ddef90c7e0 con 0x55ddef8ceec0 2024-05-27T07:53:56.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.482+0000 7efcd9c53b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ddefa6b830 con 0x55ddefa4de30 2024-05-27T07:53:56.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.482+0000 7efcd9c53b80 10 client.0 _ll_register_callbacks cb 0x55ddef915cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:56.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.483+0000 7efcc3e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.483+0000 7efcc3400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa57a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.483+0000 7efcc3e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa57500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33758/0 (socket says 172.21.15.90:33758) 2024-05-27T07:53:56.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.483+0000 7efcc3e00700 1 -- 172.21.15.90:0/3821229616 learned_addr learned my addr 172.21.15.90:0/3821229616 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:56.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.483+0000 7efcc9200700 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa53db0 0x55ddefa57f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.484+0000 7efcc3e00700 1 -- 172.21.15.90:0/3821229616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 msgr2=0x55ddefa57a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:56.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.484+0000 7efcc3e00700 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa57a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.484+0000 7efcc3e00700 1 -- 172.21.15.90:0/3821229616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa53db0 msgr2=0x55ddefa57f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:56.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.484+0000 7efcc3e00700 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa53db0 0x55ddefa57f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.484+0000 7efcc3e00700 1 -- 172.21.15.90:0/3821229616 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddef8bb950 con 0x55ddef8ceec0 2024-05-27T07:53:56.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.484+0000 7efcc3e00700 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa57500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009a20 tx=0x7efcb4008920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e14c5a88b269be31 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:56.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.484+0000 7efcae000700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:53:56.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.485+0000 7efcae000700 1 -- 172.21.15.90:0/3821229616 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4027070 con 0x55ddef8ceec0 2024-05-27T07:53:56.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.485+0000 7efcae000700 1 -- 172.21.15.90:0/3821229616 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7efcb4008bf0 con 0x55ddef8ceec0 2024-05-27T07:53:56.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.485+0000 7efcd9c53b80 1 -- 172.21.15.90:0/3821229616 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb4009320 con 0x55ddef8ceec0 2024-05-27T07:53:56.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.485+0000 7efcae000700 1 -- 172.21.15.90:0/3821229616 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7efcb4031050 con 0x55ddef8ceec0 2024-05-27T07:53:56.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.485+0000 7efcd9c53b80 10 client.36491 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:56.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.485+0000 7efcd9c53b80 1 -- 172.21.15.90:0/3821229616 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ddef93c0e0 con 0x55ddef8ceec0 2024-05-27T07:53:56.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.486+0000 7efcae000700 1 -- 172.21.15.90:0/3821229616 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcb4009320 con 0x55ddef8ceec0 2024-05-27T07:53:56.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.487+0000 7efcae000700 1 -- 172.21.15.90:0/3821229616 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7efcb4011070 con 0x55ddef8ceec0 2024-05-27T07:53:56.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.487+0000 7efcd9c53b80 10 client.36491 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:56.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.487+0000 7efcd9c53b80 10 client.36491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:56.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.487+0000 7efcd9c53b80 20 client.36491 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:56.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.487+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddefa78530 0x55ddefa7a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.487+0000 7efcd9c53b80 4 client.36491 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:53:56.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.487+0000 7efcd9c53b80 1 -- 172.21.15.90:0/3821229616 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddef97de10 con 0x55ddefa78530 2024-05-27T07:53:56.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.488+0000 7efcc3400700 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddefa78530 0x55ddefa7a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.490+0000 7efcc3400700 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddefa78530 0x55ddefa7a950 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb61e3b5c1f9f2 server_cookie=f168b420dc74be4d in_seq=0 out_seq=0 2024-05-27T07:53:56.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.490+0000 7efcae000700 10 client.36491 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:56.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.492+0000 7efcae000700 1 -- 172.21.15.90:0/3821229616 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ddef97de10 con 0x55ddefa78530 2024-05-27T07:53:56.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.492+0000 7efcae000700 10 client.36491 handle_command_reply: tid=0 2024-05-27T07:53:56.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.492+0000 7efcd9c53b80 1 client.36491 shutdown 2024-05-27T07:53:56.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.492+0000 7efcd9c53b80 1 -- 172.21.15.90:0/3821229616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 msgr2=0x55ddefa57500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:56.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.492+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa57500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009a20 tx=0x7efcb4008920 comp rx=0 tx=0).stop 2024-05-27T07:53:56.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.493+0000 7efcd9c53b80 1 -- 172.21.15.90:0/3821229616 shutdown_connections 2024-05-27T07:53:56.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.493+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddefa78530 0x55ddefa7a950 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.493+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa53db0 0x55ddefa57f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.493+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa57500 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.493+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/3821229616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa57a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.493+0000 7efcd9c53b80 1 -- 172.21.15.90:0/3821229616 >> 172.21.15.90:0/3821229616 conn(0x55ddef92f040 msgr2=0x55ddefa48600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:56.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.493+0000 7efcae000700 0 client.36491 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:56.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.493+0000 7efcd9c53b80 1 -- 172.21.15.90:0/3821229616 shutdown_connections 2024-05-27T07:53:56.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.493+0000 7efcd9c53b80 1 -- 172.21.15.90:0/3821229616 wait complete. 2024-05-27T07:53:56.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.494+0000 7efcd9c53b80 20 client.36491 trim_cache size 0 max 16384 2024-05-27T07:53:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.533+0000 7efcd9c53b80 1 Processor -- start 2024-05-27T07:53:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.533+0000 7efcd9c53b80 1 -- start start 2024-05-27T07:53:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.534+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa4d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.534+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa4de30 0x55ddefa74a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.534+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa53db0 0x55ddefa599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.534+0000 7efcd9c53b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41b85f0 con 0x55ddefa4de30 2024-05-27T07:53:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.534+0000 7efcd9c53b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efcc411afc0 con 0x55ddef8ceec0 2024-05-27T07:53:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.534+0000 7efcd9c53b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41b7f20 con 0x55ddefa53db0 2024-05-27T07:53:56.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.535+0000 7efcc3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa4de30 0x55ddefa74a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.535+0000 7efcc9200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa53db0 0x55ddefa599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.535+0000 7efcc9200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa53db0 0x55ddefa599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:49894/0 (socket says 172.21.15.90:49894) 2024-05-27T07:53:56.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.535+0000 7efcc9200700 1 -- 172.21.15.90:0/2339034769 learned_addr learned my addr 172.21.15.90:0/2339034769 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:56.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.535+0000 7efcc3e00700 1 --2- 172.21.15.90:0/2339034769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa4d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.536+0000 7efcc9200700 1 -- 172.21.15.90:0/2339034769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 msgr2=0x55ddefa4d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:56.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.536+0000 7efcc9200700 1 --2- 172.21.15.90:0/2339034769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa4d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.536+0000 7efcc9200700 1 -- 172.21.15.90:0/2339034769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa4de30 msgr2=0x55ddefa74a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:56.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.536+0000 7efcc9200700 1 --2- 172.21.15.90:0/2339034769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa4de30 0x55ddefa74a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.536+0000 7efcc9200700 1 -- 172.21.15.90:0/2339034769 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddefa6b770 con 0x55ddefa53db0 2024-05-27T07:53:56.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.537+0000 7efcc3e00700 1 --2- 172.21.15.90:0/2339034769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa4d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:53:56.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.538+0000 7efcc9200700 1 --2- 172.21.15.90:0/2339034769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa53db0 0x55ddefa599a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0df670 tx=0x7efcbc0a9150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73bb485dca2cc421 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:56.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.538+0000 7efcb8c00700 1 -- 172.21.15.90:0/2339034769 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc020050 con 0x55ddefa53db0 2024-05-27T07:53:56.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.538+0000 7efcb8c00700 1 -- 172.21.15.90:0/2339034769 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcbc0b1fb0 con 0x55ddefa53db0 2024-05-27T07:53:56.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.539+0000 7efcb8c00700 1 -- 172.21.15.90:0/2339034769 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0b1d80 con 0x55ddefa53db0 2024-05-27T07:53:56.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.540+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2339034769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa53db0 msgr2=0x55ddefa599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:56.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.540+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2339034769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa53db0 0x55ddefa599a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0df670 tx=0x7efcbc0a9150 comp rx=0 tx=0).stop 2024-05-27T07:53:56.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.540+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2339034769 shutdown_connections 2024-05-27T07:53:56.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.540+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2339034769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa53db0 0x55ddefa599a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.540+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2339034769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa4d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.540+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2339034769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa4de30 0x55ddefa74a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.540+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2339034769 >> 172.21.15.90:0/2339034769 conn(0x55ddef92f040 msgr2=0x55ddefa572e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:56.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.540+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2339034769 shutdown_connections 2024-05-27T07:53:56.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.540+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2339034769 wait complete. 2024-05-27T07:53:56.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.541+0000 7efcd9c53b80 1 -- start start 2024-05-27T07:53:56.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.541+0000 7efcd9c53b80 1 Processor -- start 2024-05-27T07:53:56.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.543+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa7def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.543+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa7c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.543+0000 7efcd9c53b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa53db0 0x55ddefa7c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.544+0000 7efcc3e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa7def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.544+0000 7efcc3e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa7def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33772/0 (socket says 172.21.15.90:33772) 2024-05-27T07:53:56.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.544+0000 7efcc3e00700 1 -- 172.21.15.90:0/2123462222 learned_addr learned my addr 172.21.15.90:0/2123462222 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:53:56.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.544+0000 7efcc3400700 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa7c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.544+0000 7efcc9200700 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa53db0 0x55ddefa7c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.544+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41b7180 con 0x55ddefa53db0 2024-05-27T07:53:56.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.544+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41b6ab0 con 0x55ddef8ceec0 2024-05-27T07:53:56.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.544+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efcc41b7850 con 0x55ddefa4de30 2024-05-27T07:53:56.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.544+0000 7efcd9c53b80 10 client.0 _ll_register_callbacks cb 0x55ddef915cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:53:56.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.545+0000 7efcc3400700 1 -- 172.21.15.90:0/2123462222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 msgr2=0x55ddefa7def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:56.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.545+0000 7efcc3400700 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa7def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.545+0000 7efcc3400700 1 -- 172.21.15.90:0/2123462222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa53db0 msgr2=0x55ddefa7c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:53:56.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.545+0000 7efcc3400700 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa53db0 0x55ddefa7c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.545+0000 7efcc3400700 1 -- 172.21.15.90:0/2123462222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddefa7ce80 con 0x55ddefa4de30 2024-05-27T07:53:56.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.546+0000 7efcc3400700 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa7c350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efcb000d8a0 tx=0x7efcb0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bd52eade28348f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:53:56.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.546+0000 7efcae000700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:53:56.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.546+0000 7efcae000700 1 -- 172.21.15.90:0/2123462222 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0004830 con 0x55ddefa4de30 2024-05-27T07:53:56.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.546+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcbc0b1fb0 con 0x55ddefa4de30 2024-05-27T07:53:56.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.546+0000 7efcae000700 1 -- 172.21.15.90:0/2123462222 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7efcb00049d0 con 0x55ddefa4de30 2024-05-27T07:53:56.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.547+0000 7efcae000700 1 -- 172.21.15.90:0/2123462222 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7efcb0004bf0 con 0x55ddefa4de30 2024-05-27T07:53:56.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.547+0000 7efcd9c53b80 10 client.26524 fetch_fsmap learned FSMap version 48 2024-05-27T07:53:56.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.547+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ddefae3c20 con 0x55ddefa4de30 2024-05-27T07:53:56.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.547+0000 7efcae000700 1 -- 172.21.15.90:0/2123462222 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcbc0b1fb0 con 0x55ddefa4de30 2024-05-27T07:53:56.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.548+0000 7efcae000700 1 -- 172.21.15.90:0/2123462222 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 48) ==== 7757+0+0 (secure 0 0 0) 0x7efcb0028300 con 0x55ddefa4de30 2024-05-27T07:53:56.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.548+0000 7efcd9c53b80 10 client.26524 fetch_fsmap finished waiting for FSMap version 48 2024-05-27T07:53:56.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.548+0000 7efcd9c53b80 10 client.26524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:53:56.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.548+0000 7efcd9c53b80 20 client.26524 populate_metadata read hostname 'smithi090' 2024-05-27T07:53:56.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.548+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddefae4380 0x55ddefae67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:53:56.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.548+0000 7efcd9c53b80 4 client.26524 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:53:56.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.548+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ddef9775f0 con 0x55ddefae4380 2024-05-27T07:53:56.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.548+0000 7efcc3e00700 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddefae4380 0x55ddefae67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:53:56.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.549+0000 7efcae000700 10 client.26524 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:56.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.551+0000 7efcc3e00700 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddefae4380 0x55ddefae67a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=216c6f68a65b534a server_cookie=a8b9036af23d7830 in_seq=0 out_seq=0 2024-05-27T07:53:56.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.554+0000 7efcae000700 1 -- 172.21.15.90:0/2123462222 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ddef9775f0 con 0x55ddefae4380 2024-05-27T07:53:56.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcae000700 10 client.26524 handle_command_reply: tid=0 2024-05-27T07:53:56.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcd9c53b80 1 client.26524 shutdown 2024-05-27T07:53:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 msgr2=0x55ddefa7c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa7c350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efcb000d8a0 tx=0x7efcb0004750 comp rx=0 tx=0).stop 2024-05-27T07:53:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 shutdown_connections 2024-05-27T07:53:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddefae4380 0x55ddefae67a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddefa53db0 0x55ddefa7c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddef8ceec0 0x55ddefa7def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcd9c53b80 1 --2- 172.21.15.90:0/2123462222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddefa4de30 0x55ddefa7c350 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 >> 172.21.15.90:0/2123462222 conn(0x55ddef92f040 msgr2=0x55ddefa54ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:56.558 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:53:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:53:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "544909c8-1e7b-4d63-af3e-538e5bfbd432", 2024-05-27T07:53:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:53:56.558 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:53:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.555+0000 7efcae000700 0 client.26524 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:53:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.556+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 shutdown_connections 2024-05-27T07:53:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.556+0000 7efcd9c53b80 1 -- 172.21.15.90:0/2123462222 wait complete. 2024-05-27T07:53:56.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.557+0000 7efcd9c53b80 20 client.26524 trim_cache size 0 max 16384 2024-05-27T07:53:56.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.558+0000 7efccb200700 1 -- 172.21.15.90:0/774708839 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efca80b2aa0 msgr2=0x7efca80b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:56.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.558+0000 7efccb200700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efca80b2aa0 0x7efca80b4fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efcb0005d90 tx=0x7efcb0005d00 comp rx=0 tx=0).stop 2024-05-27T07:53:56.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.558+0000 7efccb200700 1 -- 172.21.15.90:0/774708839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc411d920 msgr2=0x7efcc41a99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:53:56.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.558+0000 7efccb200700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc411d920 0x7efcc41a99a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00d950 tx=0x7efcbc00dd10 comp rx=0 tx=0).stop 2024-05-27T07:53:56.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.559+0000 7efccb200700 1 -- 172.21.15.90:0/774708839 shutdown_connections 2024-05-27T07:53:56.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.559+0000 7efccb200700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efca80b2aa0 0x7efca80b4fb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.559+0000 7efccb200700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc411d920 0x7efcc41a99a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.559+0000 7efccb200700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcc410a920 0x7efcc41af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.559+0000 7efccb200700 1 --2- 172.21.15.90:0/774708839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efcc410b270 0x7efcc41afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:53:56.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.559+0000 7efccb200700 1 -- 172.21.15.90:0/774708839 >> 172.21.15.90:0/774708839 conn(0x7efcc406f4c0 msgr2=0x7efcc411b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:53:56.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.560+0000 7efccb200700 1 -- 172.21.15.90:0/774708839 shutdown_connections 2024-05-27T07:53:56.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:53:56.560+0000 7efccb200700 1 -- 172.21.15.90:0/774708839 wait complete. 2024-05-27T07:53:56.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:56 smithi129 ceph-mon[35254]: pgmap v235: 129 pgs: 129 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 11 MiB/s wr, 651 op/s 2024-05-27T07:53:56.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:56 smithi136 ceph-mon[35324]: pgmap v235: 129 pgs: 129 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 11 MiB/s wr, 651 op/s 2024-05-27T07:53:57.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:57 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:53:57.530 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:57 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:57.530 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:57 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:53:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:57 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:53:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:57 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:57.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:57 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:53:57.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:57 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:53:57.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:57 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:57.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:57 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:53:58.068 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:53:58.282 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:57 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 2. 2024-05-27T07:53:58.282 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:57 smithi090 systemd[1]: Stopped Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:53:58.283 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:57 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 2.061s CPU time. 2024-05-27T07:53:58.283 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:57 smithi090 systemd[1]: Starting Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:53:58.285 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:58 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:58.285 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:58 smithi090 ceph-mon[31429]: pgmap v236: 129 pgs: 129 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.6 MiB/s wr, 489 op/s 2024-05-27T07:53:58.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:58 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:58.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:58 smithi129 ceph-mon[35254]: pgmap v236: 129 pgs: 129 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.6 MiB/s wr, 489 op/s 2024-05-27T07:53:58.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:58 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:53:58.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:58 smithi136 ceph-mon[35324]: pgmap v236: 129 pgs: 129 active+clean; 2.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 8.6 MiB/s wr, 489 op/s 2024-05-27T07:53:58.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 220.0 s, 562.8 tps, lat 174.484 ms stddev 251.572 2024-05-27T07:53:58.861 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:58 smithi090 podman[110329]: 2024-05-27 07:53:58.444919126 +0000 UTC m=+0.059284228 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:53:58.861 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:58 smithi090 podman[110329]: 2024-05-27 07:53:58.556402244 +0000 UTC m=+0.170767332 container create 9c1c3a7308a298dabb331de08e212e7894d56e08db6380b4b317828260807ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS) 2024-05-27T07:53:58.861 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:58 smithi090 podman[110329]: 2024-05-27 07:53:58.860385363 +0000 UTC m=+0.474750479 container init 9c1c3a7308a298dabb331de08e212e7894d56e08db6380b4b317828260807ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-27T07:53:59.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:58 smithi090 podman[110329]: 2024-05-27 07:53:58.882254674 +0000 UTC m=+0.496619773 container start 9c1c3a7308a298dabb331de08e212e7894d56e08db6380b4b317828260807ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-27T07:53:59.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:59 smithi090 ceph-mds[110370]: -- 172.21.15.90:0/1817088523 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b676eb1a40 con 0x55b676ef9a80 2024-05-27T07:53:59.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:59 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: starting mds.j at 2024-05-27T07:53:59.518 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:53:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:59 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:53:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:53:59 smithi129 ceph-mon[35254]: daemon mds.j restarted 2024-05-27T07:53:59.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:59 smithi090 bash[110329]: 9c1c3a7308a298dabb331de08e212e7894d56e08db6380b4b317828260807ada 2024-05-27T07:53:59.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:53:59 smithi090 systemd[1]: Started Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:53:59.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:59 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:53:59.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:53:59 smithi090 ceph-mon[31429]: daemon mds.j restarted 2024-05-27T07:53:59.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:59 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:53:59.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:53:59 smithi136 ceph-mon[35324]: daemon mds.j restarted 2024-05-27T07:54:00.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:00 smithi129 ceph-mon[35254]: pgmap v237: 129 pgs: 129 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 13 MiB/s wr, 831 op/s 2024-05-27T07:54:00.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:00 smithi129 ceph-mon[35254]: osdmap e101: 12 total, 12 up, 12 in 2024-05-27T07:54:00.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:00 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6836/3316213432,v1:172.21.15.90:6837/3316213432] up:boot 2024-05-27T07:54:00.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:00 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:54:00.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:00 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:54:00.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:00 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:54:00.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:00 smithi090 ceph-mon[31429]: pgmap v237: 129 pgs: 129 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 13 MiB/s wr, 831 op/s 2024-05-27T07:54:00.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:00 smithi090 ceph-mon[31429]: osdmap e101: 12 total, 12 up, 12 in 2024-05-27T07:54:00.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:00 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6836/3316213432,v1:172.21.15.90:6837/3316213432] up:boot 2024-05-27T07:54:00.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:00 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:54:00.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:00 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:54:00.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:00 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:54:00.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:00 smithi136 ceph-mon[35324]: pgmap v237: 129 pgs: 129 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 13 MiB/s wr, 831 op/s 2024-05-27T07:54:00.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:00 smithi136 ceph-mon[35324]: osdmap e101: 12 total, 12 up, 12 in 2024-05-27T07:54:00.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:00 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6836/3316213432,v1:172.21.15.90:6837/3316213432] up:boot 2024-05-27T07:54:00.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:00 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:54:00.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:00 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:54:00.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:00 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:54:01.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:01 smithi090 ceph-mon[31429]: pgmap v239: 129 pgs: 129 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 11 MiB/s wr, 728 op/s 2024-05-27T07:54:01.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:01 smithi136 ceph-mon[35324]: pgmap v239: 129 pgs: 129 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 11 MiB/s wr, 728 op/s 2024-05-27T07:54:01.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:01 smithi129 ceph-mon[35254]: pgmap v239: 129 pgs: 129 active+clean; 2.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 11 MiB/s wr, 728 op/s 2024-05-27T07:54:03.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.609+0000 7f174b200700 1 -- 172.21.15.90:0/540666407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c115170 msgr2=0x7f174c117590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.609+0000 7f174b200700 1 --2- 172.21.15.90:0/540666407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c115170 0x7f174c117590 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f174400b0a0 tx=0x7f174400b3b0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.609+0000 7f174b200700 1 -- 172.21.15.90:0/540666407 shutdown_connections 2024-05-27T07:54:03.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.609+0000 7f174b200700 1 --2- 172.21.15.90:0/540666407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c115170 0x7f174c117590 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.609+0000 7f174b200700 1 --2- 172.21.15.90:0/540666407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f174c075b50 0x7f174c075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.609+0000 7f174b200700 1 --2- 172.21.15.90:0/540666407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f174c0764a0 0x7f174c114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.609+0000 7f174b200700 1 -- 172.21.15.90:0/540666407 >> 172.21.15.90:0/540666407 conn(0x7f174c079d00 msgr2=0x7f174c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.610+0000 7f174b200700 1 -- 172.21.15.90:0/540666407 shutdown_connections 2024-05-27T07:54:03.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.610+0000 7f174b200700 1 -- 172.21.15.90:0/540666407 wait complete. 2024-05-27T07:54:03.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.611+0000 7f174b200700 1 Processor -- start 2024-05-27T07:54:03.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.612+0000 7f174b200700 1 -- start start 2024-05-27T07:54:03.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.612+0000 7f174b200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f174c075b50 0x7f174c0753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.613+0000 7f174b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c0764a0 0x7f174c06e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.613+0000 7f174b200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f174c115170 0x7f174c06eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.613+0000 7f174b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f174c11a780 con 0x7f174c0764a0 2024-05-27T07:54:03.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.613+0000 7f174b200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f174c11a600 con 0x7f174c075b50 2024-05-27T07:54:03.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.613+0000 7f174b200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f174c11a900 con 0x7f174c115170 2024-05-27T07:54:03.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.613+0000 7f1749e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f174c075b50 0x7f174c0753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.614+0000 7f174a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f174c115170 0x7f174c06eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.614+0000 7f174a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f174c115170 0x7f174c06eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57290/0 (socket says 172.21.15.90:57290) 2024-05-27T07:54:03.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.614+0000 7f174a800700 1 -- 172.21.15.90:0/322241360 learned_addr learned my addr 172.21.15.90:0/322241360 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:03.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.614+0000 7f1749400700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c0764a0 0x7f174c06e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.615+0000 7f174a800700 1 -- 172.21.15.90:0/322241360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f174c075b50 msgr2=0x7f174c0753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.615+0000 7f174a800700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f174c075b50 0x7f174c0753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.615+0000 7f174a800700 1 -- 172.21.15.90:0/322241360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c0764a0 msgr2=0x7f174c06e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.615+0000 7f174a800700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c0764a0 0x7f174c06e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.615+0000 7f174a800700 1 -- 172.21.15.90:0/322241360 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c072ef0 con 0x7f174c115170 2024-05-27T07:54:03.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.615+0000 7f1749e00700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f174c075b50 0x7f174c0753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:03.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.616+0000 7f174a800700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f174c115170 0x7f174c06eb40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f174400b070 tx=0x7f1744004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.617+0000 7f1742a00700 1 -- 172.21.15.90:0/322241360 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744021070 con 0x7f174c115170 2024-05-27T07:54:03.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.617+0000 7f1742a00700 1 -- 172.21.15.90:0/322241360 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1744007b80 con 0x7f174c115170 2024-05-27T07:54:03.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.617+0000 7f1742a00700 1 -- 172.21.15.90:0/322241360 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744004650 con 0x7f174c115170 2024-05-27T07:54:03.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.617+0000 7f174b200700 1 -- 172.21.15.90:0/322241360 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c073180 con 0x7f174c115170 2024-05-27T07:54:03.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.617+0000 7f174b200700 1 -- 172.21.15.90:0/322241360 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c073610 con 0x7f174c115170 2024-05-27T07:54:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.627+0000 7f1742a00700 1 -- 172.21.15.90:0/322241360 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1744003a00 con 0x7f174c115170 2024-05-27T07:54:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.629+0000 7f1742a00700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f17300b2a70 0x7f17300b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.629+0000 7f1749e00700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f17300b2a70 0x7f17300b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.629+0000 7f1742a00700 1 -- 172.21.15.90:0/322241360 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9408+0+0 (secure 0 0 0) 0x7f174400f020 con 0x7f174c115170 2024-05-27T07:54:03.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.631+0000 7f1749e00700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f17300b2a70 0x7f17300b4f80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f174c06fd40 tx=0x7f173c00b410 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.636+0000 7f175faebb80 1 Processor -- start 2024-05-27T07:54:03.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.636+0000 7f175faebb80 1 -- start start 2024-05-27T07:54:03.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.637+0000 7f175faebb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5619f0 0x55d45f5f5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.637+0000 7f175faebb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.637+0000 7f175faebb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5ff2c0 0x55d45f6016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.638+0000 7f1749400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.638+0000 7f1749400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5f5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33434/0 (socket says 172.21.15.90:33434) 2024-05-27T07:54:03.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.638+0000 7f1749e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5619f0 0x55d45f5f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.638+0000 7f1749400700 1 -- 172.21.15.90:0/2882875391 learned_addr learned my addr 172.21.15.90:0/2882875391 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:03.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.638+0000 7f175faebb80 1 -- 172.21.15.90:0/2882875391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d45f522e40 con 0x55d45f5ff2c0 2024-05-27T07:54:03.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.638+0000 7f175faebb80 1 -- 172.21.15.90:0/2882875391 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d45f5129e0 con 0x55d45f475e40 2024-05-27T07:54:03.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.638+0000 7f175faebb80 1 -- 172.21.15.90:0/2882875391 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d45f516a20 con 0x55d45f5619f0 2024-05-27T07:54:03.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.639+0000 7f174a800700 1 --2- 172.21.15.90:0/2882875391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5ff2c0 0x55d45f6016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.641+0000 7f1749e00700 1 -- 172.21.15.90:0/2882875391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 msgr2=0x55d45f5f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.641+0000 7f1749e00700 1 --2- 172.21.15.90:0/2882875391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.641+0000 7f1749e00700 1 -- 172.21.15.90:0/2882875391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5ff2c0 msgr2=0x55d45f6016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.641+0000 7f1749e00700 1 --2- 172.21.15.90:0/2882875391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5ff2c0 0x55d45f6016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.641+0000 7f1749e00700 1 -- 172.21.15.90:0/2882875391 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d45f4db420 con 0x55d45f5619f0 2024-05-27T07:54:03.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.642+0000 7f1749400700 1 --2- 172.21.15.90:0/2882875391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:03.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.643+0000 7f1749e00700 1 --2- 172.21.15.90:0/2882875391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5619f0 0x55d45f5f5750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f173c005950 tx=0x7f173c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af316e996fe545c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.644+0000 7f1737400700 1 -- 172.21.15.90:0/2882875391 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c01e620 con 0x55d45f5619f0 2024-05-27T07:54:03.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.644+0000 7f1737400700 1 -- 172.21.15.90:0/2882875391 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f173c01ec50 con 0x55d45f5619f0 2024-05-27T07:54:03.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.644+0000 7f1737400700 1 -- 172.21.15.90:0/2882875391 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c01e620 con 0x55d45f5619f0 2024-05-27T07:54:03.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.647+0000 7f175faebb80 1 -- 172.21.15.90:0/2882875391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5619f0 msgr2=0x55d45f5f5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.647+0000 7f175faebb80 1 --2- 172.21.15.90:0/2882875391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5619f0 0x55d45f5f5750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f173c005950 tx=0x7f173c013000 comp rx=0 tx=0).stop 2024-05-27T07:54:03.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.647+0000 7f175faebb80 1 -- 172.21.15.90:0/2882875391 shutdown_connections 2024-05-27T07:54:03.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.647+0000 7f175faebb80 1 --2- 172.21.15.90:0/2882875391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5ff2c0 0x55d45f6016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.647+0000 7f175faebb80 1 --2- 172.21.15.90:0/2882875391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5619f0 0x55d45f5f5750 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.647+0000 7f175faebb80 1 --2- 172.21.15.90:0/2882875391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.647+0000 7f175faebb80 1 -- 172.21.15.90:0/2882875391 >> 172.21.15.90:0/2882875391 conn(0x55d45f50f390 msgr2=0x55d45f5f0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.649+0000 7f175faebb80 1 -- 172.21.15.90:0/2882875391 shutdown_connections 2024-05-27T07:54:03.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.649+0000 7f175faebb80 1 -- 172.21.15.90:0/2882875391 wait complete. 2024-05-27T07:54:03.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.650+0000 7f175faebb80 1 -- start start 2024-05-27T07:54:03.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.650+0000 7f175faebb80 1 Processor -- start 2024-05-27T07:54:03.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.652+0000 7f175faebb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5fc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.652+0000 7f175faebb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 0x55d45f5fc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.652+0000 7f175faebb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 0x55d45f5fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.652+0000 7f175faebb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d45f418fa0 con 0x55d45f5619f0 2024-05-27T07:54:03.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.652+0000 7f175faebb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d45f4b3760 con 0x55d45f475e40 2024-05-27T07:54:03.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.652+0000 7f175faebb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d45f601e00 con 0x55d45f5ff2c0 2024-05-27T07:54:03.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.652+0000 7f175faebb80 10 client.0 _ll_register_callbacks cb 0x55d45f4bcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:03.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.653+0000 7f1749400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 0x55d45f5fc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.653+0000 7f174a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 0x55d45f5fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.653+0000 7f174a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 0x55d45f5fcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57306/0 (socket says 172.21.15.90:57306) 2024-05-27T07:54:03.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.653+0000 7f174a800700 1 -- 172.21.15.90:0/3001906031 learned_addr learned my addr 172.21.15.90:0/3001906031 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:03.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.653+0000 7f1749e00700 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.653+0000 7f174a800700 1 -- 172.21.15.90:0/3001906031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 msgr2=0x55d45f5fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.653+0000 7f174a800700 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.653+0000 7f174a800700 1 -- 172.21.15.90:0/3001906031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 msgr2=0x55d45f5fc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.654+0000 7f174a800700 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 0x55d45f5fc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.654+0000 7f174a800700 1 -- 172.21.15.90:0/3001906031 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d45f4628d0 con 0x55d45f5ff2c0 2024-05-27T07:54:03.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.654+0000 7f1749400700 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 0x55d45f5fc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:03.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.654+0000 7f1749e00700 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:03.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.655+0000 7f174a800700 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 0x55d45f5fcd10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f17440e1110 tx=0x7f17440075c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b25ebcea0a228df server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.655+0000 7f172be00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:54:03.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.655+0000 7f172be00700 1 -- 172.21.15.90:0/3001906031 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17440ab3d0 con 0x55d45f5ff2c0 2024-05-27T07:54:03.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.655+0000 7f175faebb80 1 -- 172.21.15.90:0/3001906031 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f173c01ec50 con 0x55d45f5ff2c0 2024-05-27T07:54:03.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.655+0000 7f172be00700 1 -- 172.21.15.90:0/3001906031 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9408+0+0 (secure 0 0 0) 0x7f1744007d30 con 0x55d45f5ff2c0 2024-05-27T07:54:03.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.656+0000 7f172be00700 1 -- 172.21.15.90:0/3001906031 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1744004000 con 0x55d45f5ff2c0 2024-05-27T07:54:03.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.656+0000 7f175faebb80 10 client.26539 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:03.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.656+0000 7f175faebb80 1 -- 172.21.15.90:0/3001906031 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d45f4e3060 con 0x55d45f5ff2c0 2024-05-27T07:54:03.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.657+0000 7f172be00700 1 -- 172.21.15.90:0/3001906031 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f173c01ec50 con 0x55d45f5ff2c0 2024-05-27T07:54:03.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.657+0000 7f172be00700 1 -- 172.21.15.90:0/3001906031 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7f1744011070 con 0x55d45f5ff2c0 2024-05-27T07:54:03.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.657+0000 7f175faebb80 10 client.26539 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:03.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.657+0000 7f175faebb80 10 client.26539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:03.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.657+0000 7f175faebb80 20 client.26539 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:03.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.658+0000 7f175faebb80 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d45f60ecd0 0x55d45f611120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.658+0000 7f175faebb80 4 client.26539 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:03.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.658+0000 7f175faebb80 1 -- 172.21.15.90:0/3001906031 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d45f524d90 con 0x55d45f60ecd0 2024-05-27T07:54:03.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.658+0000 7f1749e00700 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d45f60ecd0 0x55d45f611120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.660+0000 7f172be00700 10 client.26539 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:03.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.660+0000 7f1749e00700 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d45f60ecd0 0x55d45f611120 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2f8db70f443e29 server_cookie=a6f85bffdae10ed0 in_seq=0 out_seq=0 2024-05-27T07:54:03.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.662+0000 7f172be00700 1 -- 172.21.15.90:0/3001906031 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d45f524d90 con 0x55d45f60ecd0 2024-05-27T07:54:03.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.662+0000 7f172be00700 10 client.26539 handle_command_reply: tid=0 2024-05-27T07:54:03.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.662+0000 7f175faebb80 1 client.26539 shutdown 2024-05-27T07:54:03.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.663+0000 7f175faebb80 1 -- 172.21.15.90:0/3001906031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 msgr2=0x55d45f5fcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.663+0000 7f175faebb80 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 0x55d45f5fcd10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f17440e1110 tx=0x7f17440075c0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.664+0000 7f175faebb80 1 -- 172.21.15.90:0/3001906031 shutdown_connections 2024-05-27T07:54:03.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.664+0000 7f175faebb80 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d45f60ecd0 0x55d45f611120 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.664+0000 7f175faebb80 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 0x55d45f5fcd10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.664+0000 7f175faebb80 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f475e40 0x55d45f5fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.664+0000 7f175faebb80 1 --2- 172.21.15.90:0/3001906031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 0x55d45f5fc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.664+0000 7f175faebb80 1 -- 172.21.15.90:0/3001906031 >> 172.21.15.90:0/3001906031 conn(0x55d45f50f390 msgr2=0x55d45f5f57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.664+0000 7f172be00700 0 client.26539 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:03.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.665+0000 7f175faebb80 1 -- 172.21.15.90:0/3001906031 shutdown_connections 2024-05-27T07:54:03.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.665+0000 7f175faebb80 1 -- 172.21.15.90:0/3001906031 wait complete. 2024-05-27T07:54:03.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.666+0000 7f175faebb80 20 client.26539 trim_cache size 0 max 16384 2024-05-27T07:54:03.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.706+0000 7f175faebb80 1 Processor -- start 2024-05-27T07:54:03.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.707+0000 7f175faebb80 1 -- start start 2024-05-27T07:54:03.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.707+0000 7f175faebb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f475e40 0x55d45f5f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.707+0000 7f175faebb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 0x55d45f5f0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.707+0000 7f175faebb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5ff2c0 0x55d45f60aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.707+0000 7f175faebb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1c0b60 con 0x55d45f5619f0 2024-05-27T07:54:03.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.707+0000 7f175faebb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1be950 con 0x55d45f5ff2c0 2024-05-27T07:54:03.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.707+0000 7f175faebb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1c0490 con 0x55d45f475e40 2024-05-27T07:54:03.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.708+0000 7f1749e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f475e40 0x55d45f5f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.708+0000 7f1749e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f475e40 0x55d45f5f7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57318/0 (socket says 172.21.15.90:57318) 2024-05-27T07:54:03.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.708+0000 7f1749e00700 1 -- 172.21.15.90:0/519170835 learned_addr learned my addr 172.21.15.90:0/519170835 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:03.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.708+0000 7f174a800700 1 --2- 172.21.15.90:0/519170835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5ff2c0 0x55d45f60aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.708+0000 7f1749400700 1 --2- 172.21.15.90:0/519170835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 0x55d45f5f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.709+0000 7f1749e00700 1 -- 172.21.15.90:0/519170835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5ff2c0 msgr2=0x55d45f60aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.709+0000 7f1749e00700 1 --2- 172.21.15.90:0/519170835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5ff2c0 0x55d45f60aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.709+0000 7f1749e00700 1 -- 172.21.15.90:0/519170835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 msgr2=0x55d45f5f0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.709+0000 7f1749e00700 1 --2- 172.21.15.90:0/519170835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 0x55d45f5f0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.709+0000 7f1749e00700 1 -- 172.21.15.90:0/519170835 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d45f5ea910 con 0x55d45f475e40 2024-05-27T07:54:03.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.709+0000 7f1749400700 1 --2- 172.21.15.90:0/519170835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 0x55d45f5f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:03.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.710+0000 7f1749e00700 1 --2- 172.21.15.90:0/519170835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f475e40 0x55d45f5f7aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f173c00b4f0 tx=0x7f173c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc16c8eaa78a89d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.711+0000 7f1736a00700 1 -- 172.21.15.90:0/519170835 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c01ca90 con 0x55d45f475e40 2024-05-27T07:54:03.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.712+0000 7f1736a00700 1 -- 172.21.15.90:0/519170835 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f173c0044b0 con 0x55d45f475e40 2024-05-27T07:54:03.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.712+0000 7f1736a00700 1 -- 172.21.15.90:0/519170835 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c01ca90 con 0x55d45f475e40 2024-05-27T07:54:03.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.713+0000 7f175faebb80 1 -- 172.21.15.90:0/519170835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f475e40 msgr2=0x55d45f5f7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.713+0000 7f175faebb80 1 --2- 172.21.15.90:0/519170835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f475e40 0x55d45f5f7aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f173c00b4f0 tx=0x7f173c016000 comp rx=0 tx=0).stop 2024-05-27T07:54:03.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.713+0000 7f175faebb80 1 -- 172.21.15.90:0/519170835 shutdown_connections 2024-05-27T07:54:03.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.713+0000 7f175faebb80 1 --2- 172.21.15.90:0/519170835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5ff2c0 0x55d45f60aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.713+0000 7f175faebb80 1 --2- 172.21.15.90:0/519170835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f475e40 0x55d45f5f7aa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.713+0000 7f175faebb80 1 --2- 172.21.15.90:0/519170835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f5619f0 0x55d45f5f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.713+0000 7f175faebb80 1 -- 172.21.15.90:0/519170835 >> 172.21.15.90:0/519170835 conn(0x55d45f50f390 msgr2=0x55d45f5f4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.714+0000 7f175faebb80 1 -- 172.21.15.90:0/519170835 shutdown_connections 2024-05-27T07:54:03.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.714+0000 7f175faebb80 1 -- 172.21.15.90:0/519170835 wait complete. 2024-05-27T07:54:03.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.714+0000 7f175faebb80 1 -- start start 2024-05-27T07:54:03.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.715+0000 7f175faebb80 1 Processor -- start 2024-05-27T07:54:03.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.716+0000 7f175faebb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f475e40 0x55d45f602d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.716+0000 7f175faebb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5619f0 0x55d45f603260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.716+0000 7f175faebb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 0x55d45f6037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.716+0000 7f175faebb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1bf6f0 con 0x55d45f475e40 2024-05-27T07:54:03.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.716+0000 7f175faebb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1bf020 con 0x55d45f5619f0 2024-05-27T07:54:03.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.716+0000 7f175faebb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f174c1bfdc0 con 0x55d45f5ff2c0 2024-05-27T07:54:03.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.716+0000 7f175faebb80 10 client.0 _ll_register_callbacks cb 0x55d45f4bcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:03.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.716+0000 7f1749400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5619f0 0x55d45f603260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.716+0000 7f1749400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5619f0 0x55d45f603260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33470/0 (socket says 172.21.15.90:33470) 2024-05-27T07:54:03.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.716+0000 7f1749400700 1 -- 172.21.15.90:0/3097497470 learned_addr learned my addr 172.21.15.90:0/3097497470 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:03.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.717+0000 7f1749e00700 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f475e40 0x55d45f602d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.717+0000 7f1749400700 1 -- 172.21.15.90:0/3097497470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 msgr2=0x55d45f6037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.717+0000 7f1749400700 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 0x55d45f6037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.717+0000 7f1749400700 1 -- 172.21.15.90:0/3097497470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f475e40 msgr2=0x55d45f602d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:03.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.717+0000 7f1749400700 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f475e40 0x55d45f602d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.717+0000 7f1749400700 1 -- 172.21.15.90:0/3097497470 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d45f6787d0 con 0x55d45f5619f0 2024-05-27T07:54:03.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.717+0000 7f1749e00700 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f475e40 0x55d45f602d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:03.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.718+0000 7f1749400700 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5619f0 0x55d45f603260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55d45f5f1970 tx=0x7f1738003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd38b5bff7693ce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:03.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.719+0000 7f172be00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:54:03.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.719+0000 7f172be00700 1 -- 172.21.15.90:0/3097497470 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738004a30 con 0x55d45f5619f0 2024-05-27T07:54:03.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.719+0000 7f172be00700 1 -- 172.21.15.90:0/3097497470 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9408+0+0 (secure 0 0 0) 0x7f1738004bd0 con 0x55d45f5619f0 2024-05-27T07:54:03.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.719+0000 7f175faebb80 1 -- 172.21.15.90:0/3097497470 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f173c0044b0 con 0x55d45f5619f0 2024-05-27T07:54:03.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.720+0000 7f172be00700 1 -- 172.21.15.90:0/3097497470 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1738004dd0 con 0x55d45f5619f0 2024-05-27T07:54:03.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.720+0000 7f175faebb80 10 client.36518 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:03.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.720+0000 7f175faebb80 1 -- 172.21.15.90:0/3097497470 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d45f679890 con 0x55d45f5619f0 2024-05-27T07:54:03.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.720+0000 7f172be00700 1 -- 172.21.15.90:0/3097497470 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f173c0044b0 con 0x55d45f5619f0 2024-05-27T07:54:03.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.721+0000 7f172be00700 1 -- 172.21.15.90:0/3097497470 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7f1738010a30 con 0x55d45f5619f0 2024-05-27T07:54:03.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.721+0000 7f175faebb80 10 client.36518 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:03.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.721+0000 7f175faebb80 10 client.36518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:03.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.721+0000 7f175faebb80 20 client.36518 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:03.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.721+0000 7f175faebb80 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d45f67a000 0x55d45f67c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:03.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.721+0000 7f175faebb80 4 client.36518 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:54:03.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.721+0000 7f175faebb80 1 -- 172.21.15.90:0/3097497470 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d45f51e570 con 0x55d45f67a000 2024-05-27T07:54:03.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.723+0000 7f1749e00700 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d45f67a000 0x55d45f67c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.725+0000 7f1749e00700 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d45f67a000 0x55d45f67c420 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba02bcb52b618e7 server_cookie=fb02b67ff230149c in_seq=0 out_seq=0 2024-05-27T07:54:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.725+0000 7f172be00700 10 client.36518 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.726+0000 7f172be00700 1 -- 172.21.15.90:0/3097497470 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d45f51e570 con 0x55d45f67a000 2024-05-27T07:54:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.726+0000 7f172be00700 10 client.36518 handle_command_reply: tid=0 2024-05-27T07:54:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.726+0000 7f175faebb80 1 client.36518 shutdown 2024-05-27T07:54:03.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.726+0000 7f175faebb80 1 -- 172.21.15.90:0/3097497470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5619f0 msgr2=0x55d45f603260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.726+0000 7f175faebb80 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5619f0 0x55d45f603260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55d45f5f1970 tx=0x7f1738003960 comp rx=0 tx=0).stop 2024-05-27T07:54:03.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.726+0000 7f175faebb80 1 -- 172.21.15.90:0/3097497470 shutdown_connections 2024-05-27T07:54:03.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.726+0000 7f175faebb80 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d45f67a000 0x55d45f67c420 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.726+0000 7f175faebb80 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d45f5ff2c0 0x55d45f6037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.726+0000 7f175faebb80 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d45f475e40 0x55d45f602d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.726+0000 7f175faebb80 1 --2- 172.21.15.90:0/3097497470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d45f5619f0 0x55d45f603260 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.727+0000 7f172be00700 0 client.36518 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:03.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.727+0000 7f175faebb80 1 -- 172.21.15.90:0/3097497470 >> 172.21.15.90:0/3097497470 conn(0x55d45f50f390 msgr2=0x55d45f5f4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.727+0000 7f175faebb80 1 -- 172.21.15.90:0/3097497470 shutdown_connections 2024-05-27T07:54:03.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.728+0000 7f175faebb80 1 -- 172.21.15.90:0/3097497470 wait complete. 2024-05-27T07:54:03.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.728+0000 7f175faebb80 20 client.36518 trim_cache size 0 max 16384 2024-05-27T07:54:03.729 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:54:03.729 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:54:03.729 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:54:03.729 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:54:03.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 -- 172.21.15.90:0/322241360 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f17300b2a70 msgr2=0x7f17300b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f17300b2a70 0x7f17300b4f80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f174c06fd40 tx=0x7f173c00b410 comp rx=0 tx=0).stop 2024-05-27T07:54:03.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 -- 172.21.15.90:0/322241360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f174c115170 msgr2=0x7f174c06eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:03.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f174c115170 0x7f174c06eb40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f174400b070 tx=0x7f1744004c80 comp rx=0 tx=0).stop 2024-05-27T07:54:03.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 -- 172.21.15.90:0/322241360 shutdown_connections 2024-05-27T07:54:03.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f174c115170 0x7f174c06eb40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f17300b2a70 0x7f17300b4f80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f174c075b50 0x7f174c0753c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 --2- 172.21.15.90:0/322241360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f174c0764a0 0x7f174c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:03.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 -- 172.21.15.90:0/322241360 >> 172.21.15.90:0/322241360 conn(0x7f174c079d00 msgr2=0x7f174c100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:03.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 -- 172.21.15.90:0/322241360 shutdown_connections 2024-05-27T07:54:03.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:03.729+0000 7f174b200700 1 -- 172.21.15.90:0/322241360 wait complete. 2024-05-27T07:54:03.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 225.0 s, 609.0 tps, lat 165.761 ms stddev 244.595 2024-05-27T07:54:04.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:04 smithi090 ceph-mon[31429]: pgmap v240: 129 pgs: 129 active+clean; 2.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 11 MiB/s wr, 729 op/s 2024-05-27T07:54:04.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:04 smithi136 ceph-mon[35324]: pgmap v240: 129 pgs: 129 active+clean; 2.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 11 MiB/s wr, 729 op/s 2024-05-27T07:54:04.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:04 smithi129 ceph-mon[35254]: pgmap v240: 129 pgs: 129 active+clean; 2.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 11 MiB/s wr, 729 op/s 2024-05-27T07:54:05.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:05 smithi090 ceph-mon[31429]: osdmap e102: 12 total, 12 up, 12 in 2024-05-27T07:54:05.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:05 smithi136 ceph-mon[35324]: osdmap e102: 12 total, 12 up, 12 in 2024-05-27T07:54:05.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:05 smithi129 ceph-mon[35254]: osdmap e102: 12 total, 12 up, 12 in 2024-05-27T07:54:06.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:06 smithi090 ceph-mon[31429]: pgmap v242: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 14 MiB/s wr, 959 op/s 2024-05-27T07:54:06.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:06 smithi136 ceph-mon[35324]: pgmap v242: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 14 MiB/s wr, 959 op/s 2024-05-27T07:54:06.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:06 smithi129 ceph-mon[35254]: pgmap v242: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 14 MiB/s wr, 959 op/s 2024-05-27T07:54:06.638 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:54:06.639 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:54:07.315 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.497+0000 7fbee0c00640 1 -- 172.21.15.90:0/3819892420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbedc1043f0 msgr2=0x7fbedc1107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.497+0000 7fbee0c00640 1 --2- 172.21.15.90:0/3819892420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbedc1043f0 0x7fbedc1107c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009ca0 tx=0x7fbed402a8c0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.497+0000 7fbee0c00640 1 -- 172.21.15.90:0/3819892420 shutdown_connections 2024-05-27T07:54:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.497+0000 7fbee0c00640 1 --2- 172.21.15.90:0/3819892420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbedc1043f0 0x7fbedc1107c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.497+0000 7fbee0c00640 1 --2- 172.21.15.90:0/3819892420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbedc10a140 0x7fbedc103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.497+0000 7fbee0c00640 1 --2- 172.21.15.90:0/3819892420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbedc109ca0 0x7fbedc103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.497+0000 7fbee0c00640 1 -- 172.21.15.90:0/3819892420 >> 172.21.15.90:0/3819892420 conn(0x7fbedc079390 msgr2=0x7fbedc0797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.497+0000 7fbee0c00640 1 -- 172.21.15.90:0/3819892420 shutdown_connections 2024-05-27T07:54:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.497+0000 7fbee0c00640 1 -- 172.21.15.90:0/3819892420 wait complete. 2024-05-27T07:54:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:08 smithi136 ceph-mon[35324]: pgmap v243: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 7.2 MiB/s wr, 446 op/s 2024-05-27T07:54:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:08 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:54:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.498+0000 7fbee0c00640 1 Processor -- start 2024-05-27T07:54:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.498+0000 7fbee0c00640 1 -- start start 2024-05-27T07:54:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.499+0000 7fbee0c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbedc1043f0 0x7fbedc1a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.499+0000 7fbee0c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbedc109ca0 0x7fbedc1a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.499+0000 7fbee0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbedc10a140 0x7fbedc19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.499+0000 7fbee0c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc113db0 con 0x7fbedc1043f0 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.499+0000 7fbee0c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc113c30 con 0x7fbedc109ca0 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.499+0000 7fbee0c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc113f30 con 0x7fbedc10a140 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.499+0000 7fbed9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbedc1043f0 0x7fbedc1a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.499+0000 7fbed9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbedc1043f0 0x7fbedc1a25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51254/0 (socket says 172.21.15.90:51254) 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.499+0000 7fbed9e00640 1 -- 172.21.15.90:0/1593527142 learned_addr learned my addr 172.21.15.90:0/1593527142 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.499+0000 7fbed9400640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbedc109ca0 0x7fbedc1a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.500+0000 7fbeda800640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbedc10a140 0x7fbedc19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.500+0000 7fbed9e00640 1 -- 172.21.15.90:0/1593527142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbedc10a140 msgr2=0x7fbedc19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.500+0000 7fbed9e00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbedc10a140 0x7fbedc19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.500+0000 7fbed9e00640 1 -- 172.21.15.90:0/1593527142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbedc109ca0 msgr2=0x7fbedc1a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.500+0000 7fbed9e00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbedc109ca0 0x7fbedc1a2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.500+0000 7fbed9e00640 1 -- 172.21.15.90:0/1593527142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbedc19ced0 con 0x7fbedc1043f0 2024-05-27T07:54:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.500+0000 7fbed9e00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbedc1043f0 0x7fbedc1a25c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00d770 tx=0x7fbecc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.500+0000 7fbed2a00640 1 -- 172.21.15.90:0/1593527142 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc017070 con 0x7fbedc1043f0 2024-05-27T07:54:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.501+0000 7fbed2a00640 1 -- 172.21.15.90:0/1593527142 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbecc009c30 con 0x7fbedc1043f0 2024-05-27T07:54:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.501+0000 7fbed2a00640 1 -- 172.21.15.90:0/1593527142 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00f620 con 0x7fbedc1043f0 2024-05-27T07:54:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.501+0000 7fbee0c00640 1 -- 172.21.15.90:0/1593527142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbedc19d1c0 con 0x7fbedc1043f0 2024-05-27T07:54:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.501+0000 7fbee0c00640 1 -- 172.21.15.90:0/1593527142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbedc1a9410 con 0x7fbedc1043f0 2024-05-27T07:54:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.505+0000 7fbee7b8b740 1 Processor -- start 2024-05-27T07:54:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.505+0000 7fbee7b8b740 1 -- start start 2024-05-27T07:54:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.506+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e131860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.506+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e131da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.506+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e1322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.506+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56406e0a2b90 con 0x56406dd8f1b0 2024-05-27T07:54:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.506+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56406e092730 con 0x56406dd6e4d0 2024-05-27T07:54:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.506+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56406e096770 con 0x56406dd662f0 2024-05-27T07:54:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.506+0000 7fbed2a00640 1 -- 172.21.15.90:0/1593527142 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbecc01a070 con 0x7fbedc1043f0 2024-05-27T07:54:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.506+0000 7fbeda800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e131860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.506+0000 7fbeda800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e131860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51260/0 (socket says 172.21.15.90:51260) 2024-05-27T07:54:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.506+0000 7fbeda800640 1 -- 172.21.15.90:0/3377133869 learned_addr learned my addr 172.21.15.90:0/3377133869 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.507+0000 7fbed9400640 1 --2- 172.21.15.90:0/3377133869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e1322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.507+0000 7fbed9e00640 1 --2- 172.21.15.90:0/3377133869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e131da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.507+0000 7fbeda800640 1 -- 172.21.15.90:0/3377133869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 msgr2=0x56406e131da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.507+0000 7fbeda800640 1 --2- 172.21.15.90:0/3377133869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e131da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.507+0000 7fbeda800640 1 -- 172.21.15.90:0/3377133869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 msgr2=0x56406e1322e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.507+0000 7fbeda800640 1 --2- 172.21.15.90:0/3377133869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e1322e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.507+0000 7fbeda800640 1 -- 172.21.15.90:0/3377133869 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56406e02b8b0 con 0x56406dd8f1b0 2024-05-27T07:54:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.508+0000 7fbeda800640 1 --2- 172.21.15.90:0/3377133869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e131860 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbed4005e30 tx=0x7fbed4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c8363e8da25ee7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.508+0000 7fbed2a00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbec00b29d0 0x7fbec00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.508+0000 7fbed2a00640 1 -- 172.21.15.90:0/1593527142 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7fbecc0e2620 con 0x7fbedc1043f0 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.508+0000 7fbec7000640 1 -- 172.21.15.90:0/3377133869 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4031980 con 0x56406dd8f1b0 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.508+0000 7fbec7000640 1 -- 172.21.15.90:0/3377133869 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbed4036040 con 0x56406dd8f1b0 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.508+0000 7fbec7000640 1 -- 172.21.15.90:0/3377133869 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed40043c0 con 0x56406dd8f1b0 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.509+0000 7fbed9e00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbec00b29d0 0x7fbec00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.510+0000 7fbee7b8b740 1 -- 172.21.15.90:0/3377133869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 msgr2=0x56406e131860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.510+0000 7fbed9e00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbec00b29d0 0x7fbec00b4ee0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56406e12f350 tx=0x7fbecc00de80 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.510+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/3377133869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e131860 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbed4005e30 tx=0x7fbed4009a20 comp rx=0 tx=0).stop 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.510+0000 7fbee7b8b740 1 -- 172.21.15.90:0/3377133869 shutdown_connections 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.510+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/3377133869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e1322e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.510+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/3377133869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e131da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.510+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/3377133869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e131860 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.510+0000 7fbee7b8b740 1 -- 172.21.15.90:0/3377133869 >> 172.21.15.90:0/3377133869 conn(0x56406dd83300 msgr2=0x56406e12e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.510+0000 7fbee7b8b740 1 -- 172.21.15.90:0/3377133869 shutdown_connections 2024-05-27T07:54:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.510+0000 7fbee7b8b740 1 -- 172.21.15.90:0/3377133869 wait complete. 2024-05-27T07:54:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.511+0000 7fbee7b8b740 1 -- start start 2024-05-27T07:54:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.511+0000 7fbee7b8b740 1 Processor -- start 2024-05-27T07:54:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e13ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e13bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e147c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56406dd78790 con 0x56406dd8f1b0 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56406e03abc0 con 0x56406dd6e4d0 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56406e13e930 con 0x56406dd662f0 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbee7b8b740 10 client.0 _ll_register_callbacks cb 0x56406e018400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbeda800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbeda800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e147c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51268/0 (socket says 172.21.15.90:51268) 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbeda800640 1 -- 172.21.15.90:0/2392812766 learned_addr learned my addr 172.21.15.90:0/2392812766 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbeda800640 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e13ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.512+0000 7fbed9400640 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e13bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.513+0000 7fbeda800640 1 -- 172.21.15.90:0/2392812766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 msgr2=0x56406e13ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.513+0000 7fbeda800640 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e13ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.513+0000 7fbeda800640 1 -- 172.21.15.90:0/2392812766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 msgr2=0x56406e13bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.513+0000 7fbeda800640 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e13bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.513+0000 7fbeda800640 1 -- 172.21.15.90:0/2392812766 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56406dda7210 con 0x56406dd8f1b0 2024-05-27T07:54:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.513+0000 7fbeda800640 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e147c60 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fbed40040a0 tx=0x7fbed403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e8e72acba04d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.513+0000 7fbebb400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:54:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.514+0000 7fbebb400640 1 -- 172.21.15.90:0/2392812766 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4059060 con 0x56406dd8f1b0 2024-05-27T07:54:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.514+0000 7fbebb400640 1 -- 172.21.15.90:0/2392812766 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7fbed4031b80 con 0x56406dd8f1b0 2024-05-27T07:54:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.514+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2392812766 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed4036040 con 0x56406dd8f1b0 2024-05-27T07:54:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.514+0000 7fbebb400640 1 -- 172.21.15.90:0/2392812766 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fbed4031d80 con 0x56406dd8f1b0 2024-05-27T07:54:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.514+0000 7fbee7b8b740 10 client.16968 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.514+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2392812766 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56406de73620 con 0x56406dd8f1b0 2024-05-27T07:54:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.514+0000 7fbebb400640 1 -- 172.21.15.90:0/2392812766 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbed4036040 con 0x56406dd8f1b0 2024-05-27T07:54:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.515+0000 7fbebb400640 1 -- 172.21.15.90:0/2392812766 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7fbed40305c0 con 0x56406dd8f1b0 2024-05-27T07:54:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.515+0000 7fbee7b8b740 10 client.16968 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.515+0000 7fbee7b8b740 10 client.16968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.515+0000 7fbee7b8b740 20 client.16968 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.515+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56406dd4dd80 0x56406e14da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.515+0000 7fbee7b8b740 4 client.16968 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.515+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2392812766 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56406e0a4ae0 con 0x56406dd4dd80 2024-05-27T07:54:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.515+0000 7fbed9400640 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56406dd4dd80 0x56406e14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.516+0000 7fbed9400640 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56406dd4dd80 0x56406e14da80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f4bcabdd4e4fd6 server_cookie=53228e26b6bc6826 in_seq=0 out_seq=0 2024-05-27T07:54:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.516+0000 7fbebb400640 10 client.16968 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.518+0000 7fbebb400640 1 -- 172.21.15.90:0/2392812766 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56406e0a4ae0 con 0x56406dd4dd80 2024-05-27T07:54:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.518+0000 7fbebb400640 10 client.16968 handle_command_reply: tid=0 2024-05-27T07:54:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.518+0000 7fbee7b8b740 1 client.16968 shutdown 2024-05-27T07:54:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2392812766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 msgr2=0x56406e147c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e147c60 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fbed40040a0 tx=0x7fbed403e040 comp rx=0 tx=0).stop 2024-05-27T07:54:08.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2392812766 shutdown_connections 2024-05-27T07:54:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56406dd4dd80 0x56406e14da80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e13bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbebb400640 0 client.16968 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e147c60 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2392812766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e13ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2392812766 >> 172.21.15.90:0/2392812766 conn(0x56406dd83300 msgr2=0x56406e131690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2392812766 shutdown_connections 2024-05-27T07:54:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2392812766 wait complete. 2024-05-27T07:54:08.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.519+0000 7fbee7b8b740 20 client.16968 trim_cache size 0 max 16384 2024-05-27T07:54:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.546+0000 7fbee7b8b740 1 Processor -- start 2024-05-27T07:54:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.547+0000 7fbee7b8b740 1 -- start start 2024-05-27T07:54:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.547+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd662f0 0x56406e13bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.547+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd6e4d0 0x56406e147bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.547+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd8f1b0 0x56406e138c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.547+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc1b5160 con 0x56406dd6e4d0 2024-05-27T07:54:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.547+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc1b2f50 con 0x56406dd662f0 2024-05-27T07:54:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.548+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc1b4a90 con 0x56406dd8f1b0 2024-05-27T07:54:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.549+0000 7fbed9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd6e4d0 0x56406e147bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.549+0000 7fbeda800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd662f0 0x56406e13bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.549+0000 7fbeda800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd662f0 0x56406e13bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33488/0 (socket says 172.21.15.90:33488) 2024-05-27T07:54:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.549+0000 7fbeda800640 1 -- 172.21.15.90:0/965863904 learned_addr learned my addr 172.21.15.90:0/965863904 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.549+0000 7fbed9400640 1 -- 172.21.15.90:0/965863904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd8f1b0 msgr2=0x56406e138c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.549+0000 7fbed9400640 1 --2- 172.21.15.90:0/965863904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd8f1b0 0x56406e138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.550+0000 7fbed9400640 1 -- 172.21.15.90:0/965863904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd662f0 msgr2=0x56406e13bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.550+0000 7fbed9400640 1 --2- 172.21.15.90:0/965863904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd662f0 0x56406e13bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.550+0000 7fbed9400640 1 -- 172.21.15.90:0/965863904 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56406e13c820 con 0x56406dd6e4d0 2024-05-27T07:54:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.550+0000 7fbed9400640 1 --2- 172.21.15.90:0/965863904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd6e4d0 0x56406e147bd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x56406e13d010 tx=0x7fbec80044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3945a24cb02f0f4e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.550+0000 7fbec6600640 1 -- 172.21.15.90:0/965863904 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8003e40 con 0x56406dd6e4d0 2024-05-27T07:54:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.551+0000 7fbec6600640 1 -- 172.21.15.90:0/965863904 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbec801e8b0 con 0x56406dd6e4d0 2024-05-27T07:54:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.551+0000 7fbec6600640 1 -- 172.21.15.90:0/965863904 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8012aa0 con 0x56406dd6e4d0 2024-05-27T07:54:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.552+0000 7fbee7b8b740 1 -- 172.21.15.90:0/965863904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd6e4d0 msgr2=0x56406e147bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.552+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/965863904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd6e4d0 0x56406e147bd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x56406e13d010 tx=0x7fbec80044d0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.552+0000 7fbee7b8b740 1 -- 172.21.15.90:0/965863904 shutdown_connections 2024-05-27T07:54:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.552+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/965863904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd8f1b0 0x56406e138c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.552+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/965863904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd6e4d0 0x56406e147bd0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.552+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/965863904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd662f0 0x56406e13bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.552+0000 7fbee7b8b740 1 -- 172.21.15.90:0/965863904 >> 172.21.15.90:0/965863904 conn(0x56406dd83300 msgr2=0x56406e13db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.552+0000 7fbee7b8b740 1 -- 172.21.15.90:0/965863904 shutdown_connections 2024-05-27T07:54:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.553+0000 7fbee7b8b740 1 -- 172.21.15.90:0/965863904 wait complete. 2024-05-27T07:54:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.554+0000 7fbee7b8b740 1 -- start start 2024-05-27T07:54:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.554+0000 7fbee7b8b740 1 Processor -- start 2024-05-27T07:54:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.555+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e1ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.555+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e1ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.555+0000 7fbee7b8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e150050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.555+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc1b3cf0 con 0x56406dd8f1b0 2024-05-27T07:54:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.555+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc1b3620 con 0x56406dd6e4d0 2024-05-27T07:54:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.555+0000 7fbee7b8b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbedc1b43c0 con 0x56406dd662f0 2024-05-27T07:54:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.555+0000 7fbee7b8b740 10 client.0 _ll_register_callbacks cb 0x7fbecc0e5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.555+0000 7fbed9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e1ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.555+0000 7fbed9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e1ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33494/0 (socket says 172.21.15.90:33494) 2024-05-27T07:54:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.555+0000 7fbed9400640 1 -- 172.21.15.90:0/2406989783 learned_addr learned my addr 172.21.15.90:0/2406989783 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.556+0000 7fbeda800640 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e1ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.556+0000 7fbed9400640 1 -- 172.21.15.90:0/2406989783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 msgr2=0x56406e1ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.556+0000 7fbed9400640 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e1ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.556+0000 7fbeda800640 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.556+0000 7fbed9400640 1 -- 172.21.15.90:0/2406989783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 msgr2=0x56406e150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.556+0000 7fbed9400640 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.556+0000 7fbed9400640 1 -- 172.21.15.90:0/2406989783 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56406e150660 con 0x56406dd6e4d0 2024-05-27T07:54:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.556+0000 7fbeda800640 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.557+0000 7fbed9400640 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e1ad700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbec8005890 tx=0x7fbec801ea60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cfb8597b611d5bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.557+0000 7fbebb400640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:54:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.557+0000 7fbebb400640 1 -- 172.21.15.90:0/2406989783 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8003c40 con 0x56406dd6e4d0 2024-05-27T07:54:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.557+0000 7fbebb400640 1 -- 172.21.15.90:0/2406989783 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7fbec8003de0 con 0x56406dd6e4d0 2024-05-27T07:54:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.557+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2406989783 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec801e8b0 con 0x56406dd6e4d0 2024-05-27T07:54:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbebb400640 1 -- 172.21.15.90:0/2406989783 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fbec8038050 con 0x56406dd6e4d0 2024-05-27T07:54:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbee7b8b740 10 client.36536 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2406989783 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56406e1b58c0 con 0x56406dd6e4d0 2024-05-27T07:54:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbebb400640 1 -- 172.21.15.90:0/2406989783 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbec8005050 con 0x56406dd6e4d0 2024-05-27T07:54:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbebb400640 1 -- 172.21.15.90:0/2406989783 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7fbec803b020 con 0x56406dd6e4d0 2024-05-27T07:54:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbee7b8b740 10 client.36536 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbee7b8b740 10 client.36536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbee7b8b740 20 client.36536 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56406dd231c0 0x56406e1b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbee7b8b740 4 client.36536 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:54:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.558+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2406989783 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fbec801e8b0 con 0x56406dd231c0 2024-05-27T07:54:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.559+0000 7fbeda800640 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56406dd231c0 0x56406e1b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.559+0000 7fbeda800640 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56406dd231c0 0x56406e1b80a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79f3dcab0a6b0fc4 server_cookie=2dee4309b32defa8 in_seq=0 out_seq=0 2024-05-27T07:54:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.559+0000 7fbebb400640 10 client.36536 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:08.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:08 smithi129 ceph-mon[35254]: pgmap v243: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 7.2 MiB/s wr, 446 op/s 2024-05-27T07:54:08.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:08 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:54:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbebb400640 1 -- 172.21.15.90:0/2406989783 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fbec801e8b0 con 0x56406dd231c0 2024-05-27T07:54:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbebb400640 10 client.36536 handle_command_reply: tid=0 2024-05-27T07:54:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 client.36536 shutdown 2024-05-27T07:54:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2406989783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 msgr2=0x56406e1ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e1ad700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbec8005890 tx=0x7fbec801ea60 comp rx=0 tx=0).stop 2024-05-27T07:54:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2406989783 shutdown_connections 2024-05-27T07:54:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56406dd231c0 0x56406e1b80a0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56406dd6e4d0 0x56406e1ad700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56406dd8f1b0 0x56406e150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 --2- 172.21.15.90:0/2406989783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56406dd662f0 0x56406e1ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2406989783 >> 172.21.15.90:0/2406989783 conn(0x56406dd83300 msgr2=0x56406e136140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbebb400640 0 client.36536 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2406989783 shutdown_connections 2024-05-27T07:54:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.596+0000 7fbee7b8b740 1 -- 172.21.15.90:0/2406989783 wait complete. 2024-05-27T07:54:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.597+0000 7fbee7b8b740 20 client.36536 trim_cache size 0 max 16384 2024-05-27T07:54:08.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:54:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:54:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:54:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 -- 172.21.15.90:0/1593527142 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbec00b29d0 msgr2=0x7fbec00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbec00b29d0 0x7fbec00b4ee0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56406e12f350 tx=0x7fbecc00de80 comp rx=0 tx=0).stop 2024-05-27T07:54:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 -- 172.21.15.90:0/1593527142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbedc1043f0 msgr2=0x7fbedc1a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbedc1043f0 0x7fbedc1a25c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00d770 tx=0x7fbecc00dc30 comp rx=0 tx=0).stop 2024-05-27T07:54:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 -- 172.21.15.90:0/1593527142 shutdown_connections 2024-05-27T07:54:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbec00b29d0 0x7fbec00b4ee0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbedc109ca0 0x7fbedc1a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbedc10a140 0x7fbedc19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 --2- 172.21.15.90:0/1593527142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbedc1043f0 0x7fbedc1a25c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 -- 172.21.15.90:0/1593527142 >> 172.21.15.90:0/1593527142 conn(0x7fbedc079390 msgr2=0x7fbedc10eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 -- 172.21.15.90:0/1593527142 shutdown_connections 2024-05-27T07:54:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:08.598+0000 7fbee0c00640 1 -- 172.21.15.90:0/1593527142 wait complete. 2024-05-27T07:54:08.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:08 smithi090 ceph-mon[31429]: pgmap v243: 129 pgs: 129 active+clean; 2.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 7.2 MiB/s wr, 446 op/s 2024-05-27T07:54:08.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:08 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:54:08.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 230.0 s, 429.6 tps, lat 176.457 ms stddev 140.558 2024-05-27T07:54:10.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:10 smithi090 ceph-mon[31429]: pgmap v244: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 10 MiB/s wr, 745 op/s 2024-05-27T07:54:10.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:10 smithi136 ceph-mon[35324]: pgmap v244: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 10 MiB/s wr, 745 op/s 2024-05-27T07:54:10.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.506+0000 7fb1f4c00700 1 -- 172.21.15.90:0/3771504072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1f0103b90 msgr2=0x7fb1f01080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:10.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.506+0000 7fb1f4c00700 1 --2- 172.21.15.90:0/3771504072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1f0103b90 0x7fb1f01080c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0098f0 tx=0x7fb1dc009c00 comp rx=0 tx=0).stop 2024-05-27T07:54:10.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.506+0000 7fb1f4c00700 1 -- 172.21.15.90:0/3771504072 shutdown_connections 2024-05-27T07:54:10.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.506+0000 7fb1f4c00700 1 --2- 172.21.15.90:0/3771504072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1f0108690 0x7fb1f0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:10.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.506+0000 7fb1f4c00700 1 --2- 172.21.15.90:0/3771504072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f0103240 0x7fb1f0103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:10.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.506+0000 7fb1f4c00700 1 --2- 172.21.15.90:0/3771504072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1f0103b90 0x7fb1f01080c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:10.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.506+0000 7fb1f4c00700 1 -- 172.21.15.90:0/3771504072 >> 172.21.15.90:0/3771504072 conn(0x7fb1f00fedb0 msgr2=0x7fb1f01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:10.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.507+0000 7fb1f4c00700 1 -- 172.21.15.90:0/3771504072 shutdown_connections 2024-05-27T07:54:10.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.507+0000 7fb1f4c00700 1 -- 172.21.15.90:0/3771504072 wait complete. 2024-05-27T07:54:10.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.509+0000 7fb1f4c00700 1 Processor -- start 2024-05-27T07:54:10.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.509+0000 7fb1f4c00700 1 -- start start 2024-05-27T07:54:10.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.509+0000 7fb1f4c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f0103240 0x7fb1f01a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:10.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.509+0000 7fb1f4c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1f0103b90 0x7fb1f01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:10.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.510+0000 7fb1f4c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1f0108690 0x7fb1f01a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:10.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.510+0000 7fb1f4c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f011a8e0 con 0x7fb1f0108690 2024-05-27T07:54:10.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.510+0000 7fb1f4c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f011a760 con 0x7fb1f0103b90 2024-05-27T07:54:10.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.510+0000 7fb1f4c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f011aa60 con 0x7fb1f0103240 2024-05-27T07:54:10.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.511+0000 7fb1ed400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1f0103b90 0x7fb1f01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:10.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.511+0000 7fb1ed400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1f0103b90 0x7fb1f01a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34244/0 (socket says 172.21.15.90:34244) 2024-05-27T07:54:10.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.511+0000 7fb1ed400700 1 -- 172.21.15.90:0/368897427 learned_addr learned my addr 172.21.15.90:0/368897427 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:10.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.511+0000 7fb1ede00700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f0103240 0x7fb1f01a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:10.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.511+0000 7fb1ee800700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1f0108690 0x7fb1f01a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:10.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.512+0000 7fb1ed400700 1 -- 172.21.15.90:0/368897427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f0103240 msgr2=0x7fb1f01a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:10.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.512+0000 7fb1ed400700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f0103240 0x7fb1f01a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:10.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.512+0000 7fb1ed400700 1 -- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1f0108690 msgr2=0x7fb1f01a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:10.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.512+0000 7fb1ed400700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1f0108690 0x7fb1f01a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:10.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.512+0000 7fb1ed400700 1 -- 172.21.15.90:0/368897427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f01a62e0 con 0x7fb1f0103b90 2024-05-27T07:54:10.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.512+0000 7fb1ee800700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1f0108690 0x7fb1f01a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.513+0000 7fb1ed400700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1f0103b90 0x7fb1f01a19b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00fa00 tx=0x7fb1dc00fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.514+0000 7fb1e6a00700 1 -- 172.21.15.90:0/368897427 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc027070 con 0x7fb1f0103b90 2024-05-27T07:54:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.514+0000 7fb1e6a00700 1 -- 172.21.15.90:0/368897427 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1dc005040 con 0x7fb1f0103b90 2024-05-27T07:54:10.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.514+0000 7fb1e6a00700 1 -- 172.21.15.90:0/368897427 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc004930 con 0x7fb1f0103b90 2024-05-27T07:54:10.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.514+0000 7fb1f4c00700 1 -- 172.21.15.90:0/368897427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f01a6570 con 0x7fb1f0103b90 2024-05-27T07:54:10.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.514+0000 7fb1f4c00700 1 -- 172.21.15.90:0/368897427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01a6a00 con 0x7fb1f0103b90 2024-05-27T07:54:10.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.518+0000 7fb1f4c00700 1 -- 172.21.15.90:0/368897427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f00512b0 con 0x7fb1f0103b90 2024-05-27T07:54:10.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.519+0000 7fb1e6a00700 1 -- 172.21.15.90:0/368897427 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb1dc02a070 con 0x7fb1f0103b90 2024-05-27T07:54:10.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.522+0000 7fb1e6a00700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb1d40b2a70 0x7fb1d40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:10.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.522+0000 7fb1e6a00700 1 -- 172.21.15.90:0/368897427 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9584+0+0 (secure 0 0 0) 0x7fb1dc0e34e0 con 0x7fb1f0103b90 2024-05-27T07:54:10.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.523+0000 7fb1ede00700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb1d40b2a70 0x7fb1d40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:10.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.524+0000 7fb1ede00700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb1d40b2a70 0x7fb1d40b4f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0005950 tx=0x7fb1e000b410 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:10.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:10.528+0000 7fb1e6a00700 1 -- 172.21.15.90:0/368897427 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb1dc013030 con 0x7fb1f0103b90 2024-05-27T07:54:10.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:10 smithi129 ceph-mon[35254]: pgmap v244: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 10 MiB/s wr, 745 op/s 2024-05-27T07:54:11.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.001+0000 7fb1f4c00700 1 -- 172.21.15.90:0/368897427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1f006d290 con 0x7fb1f0103b90 2024-05-27T07:54:11.004 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:54:11.004 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":16941,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3316213432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3316213432},{"type":"v1","addr":"172.21.15.90:6837","nonce":3316213432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:53:59.292813+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_36317":{"gid":36317,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/1963864918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1963864918},{"type":"v1","addr":"172.21.15.90:6835","nonce":1963864918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34472,34442,34424,24478,34406]},"id":1}]} 2024-05-27T07:54:11.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.002+0000 7fb1e6a00700 1 -- 172.21.15.90:0/368897427 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11114 (secure 0 0 0) 0x7fb1dc0aaaf0 con 0x7fb1f0103b90 2024-05-27T07:54:11.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.007+0000 7fb1f4c00700 1 -- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb1d40b2a70 msgr2=0x7fb1d40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:11.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.007+0000 7fb1f4c00700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb1d40b2a70 0x7fb1d40b4f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0005950 tx=0x7fb1e000b410 comp rx=0 tx=0).stop 2024-05-27T07:54:11.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.007+0000 7fb1f4c00700 1 -- 172.21.15.90:0/368897427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1f0103b90 msgr2=0x7fb1f01a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:11.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.007+0000 7fb1f4c00700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1f0103b90 0x7fb1f01a19b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00fa00 tx=0x7fb1dc00fa30 comp rx=0 tx=0).stop 2024-05-27T07:54:11.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.007+0000 7fb1f4c00700 1 -- 172.21.15.90:0/368897427 shutdown_connections 2024-05-27T07:54:11.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.007+0000 7fb1f4c00700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1f0108690 0x7fb1f01a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:11.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.007+0000 7fb1f4c00700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f0103240 0x7fb1f01a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:11.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.007+0000 7fb1f4c00700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb1d40b2a70 0x7fb1d40b4f80 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0005950 tx=0x7fb1e000b410 comp rx=0 tx=0).stop 2024-05-27T07:54:11.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.007+0000 7fb1f4c00700 1 --2- 172.21.15.90:0/368897427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb1f0103b90 0x7fb1f01a19b0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00fa00 tx=0x7fb1dc00fa30 comp rx=0 tx=0).stop 2024-05-27T07:54:11.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.007+0000 7fb1f4c00700 1 -- 172.21.15.90:0/368897427 >> 172.21.15.90:0/368897427 conn(0x7fb1f00fedb0 msgr2=0x7fb1f0105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:11.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.008+0000 7fb1f4c00700 1 -- 172.21.15.90:0/368897427 shutdown_connections 2024-05-27T07:54:11.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:11.009+0000 7fb1f4c00700 1 -- 172.21.15.90:0/368897427 wait complete. 2024-05-27T07:54:11.015 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 50 2024-05-27T07:54:11.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:11 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:54:11.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:11 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:54:11.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:11 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:54:11.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:11 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/368897427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:54:11.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:11 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:54:11.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:11 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:54:11.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:11 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:54:11.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:11 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/368897427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:54:11.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:11 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:54:11.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:11 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:54:11.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:11 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:54:11.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:11 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/368897427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:54:12.184 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:54:12.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:12 smithi090 ceph-mon[31429]: pgmap v245: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 10 MiB/s wr, 733 op/s 2024-05-27T07:54:12.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:12 smithi090 ceph-mon[31429]: osdmap e103: 12 total, 12 up, 12 in 2024-05-27T07:54:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:12 smithi129 ceph-mon[35254]: pgmap v245: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 10 MiB/s wr, 733 op/s 2024-05-27T07:54:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:12 smithi129 ceph-mon[35254]: osdmap e103: 12 total, 12 up, 12 in 2024-05-27T07:54:12.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:12 smithi136 ceph-mon[35324]: pgmap v245: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 10 MiB/s wr, 733 op/s 2024-05-27T07:54:12.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:12 smithi136 ceph-mon[35324]: osdmap e103: 12 total, 12 up, 12 in 2024-05-27T07:54:12.897 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:13.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:13 smithi090 ceph-mon[31429]: pgmap v247: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.8 MiB/s wr, 477 op/s 2024-05-27T07:54:13.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:13 smithi136 ceph-mon[35324]: pgmap v247: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.8 MiB/s wr, 477 op/s 2024-05-27T07:54:13.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:13 smithi129 ceph-mon[35254]: pgmap v247: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.8 MiB/s wr, 477 op/s 2024-05-27T07:54:13.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 235.0 s, 700.4 tps, lat 177.417 ms stddev 221.781 2024-05-27T07:54:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:16 smithi090 ceph-mon[31429]: pgmap v248: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.7 MiB/s wr, 621 op/s 2024-05-27T07:54:16.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:16 smithi136 ceph-mon[35324]: pgmap v248: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.7 MiB/s wr, 621 op/s 2024-05-27T07:54:16.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:16 smithi129 ceph-mon[35254]: pgmap v248: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.7 MiB/s wr, 621 op/s 2024-05-27T07:54:17.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.394+0000 7faaa2400700 1 -- 172.21.15.90:0/374148720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa9c103240 msgr2=0x7faa9c103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:17.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.394+0000 7faaa2400700 1 --2- 172.21.15.90:0/374148720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa9c103240 0x7faa9c103650 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0098f0 tx=0x7faa8c009c00 comp rx=0 tx=0).stop 2024-05-27T07:54:17.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.394+0000 7faaa2400700 1 -- 172.21.15.90:0/374148720 shutdown_connections 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.394+0000 7faaa2400700 1 --2- 172.21.15.90:0/374148720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c108690 0x7faa9c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.394+0000 7faaa2400700 1 --2- 172.21.15.90:0/374148720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa9c103240 0x7faa9c103650 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.394+0000 7faaa2400700 1 --2- 172.21.15.90:0/374148720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa9c103b90 0x7faa9c1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.394+0000 7faaa2400700 1 -- 172.21.15.90:0/374148720 >> 172.21.15.90:0/374148720 conn(0x7faa9c0fedb0 msgr2=0x7faa9c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.394+0000 7faaa2400700 1 -- 172.21.15.90:0/374148720 shutdown_connections 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.394+0000 7faaa2400700 1 -- 172.21.15.90:0/374148720 wait complete. 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.396+0000 7faaa2400700 1 Processor -- start 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.396+0000 7faaa2400700 1 -- start start 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.396+0000 7faaa2400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c103240 0x7faa9c075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.396+0000 7faaa2400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa9c103b90 0x7faa9c06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.396+0000 7faaa2400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa9c108690 0x7faa9c06ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.396+0000 7faaa2400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c11a990 con 0x7faa9c103b90 2024-05-27T07:54:17.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.396+0000 7faaa2400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c11a810 con 0x7faa9c108690 2024-05-27T07:54:17.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.396+0000 7faaa2400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c11ab10 con 0x7faa9c103240 2024-05-27T07:54:17.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.397+0000 7faa9b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c103240 0x7faa9c075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.397+0000 7faa93e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa9c103b90 0x7faa9c06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.397+0000 7faa93e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa9c103b90 0x7faa9c06e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58078/0 (socket says 172.21.15.90:58078) 2024-05-27T07:54:17.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.397+0000 7faa93e00700 1 -- 172.21.15.90:0/3892959926 learned_addr learned my addr 172.21.15.90:0/3892959926 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:17.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.397+0000 7faa9be00700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa9c108690 0x7faa9c06ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.398+0000 7faa93e00700 1 -- 172.21.15.90:0/3892959926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c103240 msgr2=0x7faa9c075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:17.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.398+0000 7faa93e00700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c103240 0x7faa9c075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.398+0000 7faa93e00700 1 -- 172.21.15.90:0/3892959926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa9c108690 msgr2=0x7faa9c06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:17.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.398+0000 7faa93e00700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa9c108690 0x7faa9c06ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.398+0000 7faa93e00700 1 -- 172.21.15.90:0/3892959926 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c072f50 con 0x7faa9c103b90 2024-05-27T07:54:17.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.398+0000 7faa9be00700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa9c108690 0x7faa9c06ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:17.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.399+0000 7faa93e00700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa9c103b90 0x7faa9c06e5a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7faa8400e9c0 tx=0x7faa8400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:17.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.400+0000 7faa98c00700 1 -- 172.21.15.90:0/3892959926 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8400cbd0 con 0x7faa9c103b90 2024-05-27T07:54:17.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.400+0000 7faaa2400700 1 -- 172.21.15.90:0/3892959926 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c073240 con 0x7faa9c103b90 2024-05-27T07:54:17.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.400+0000 7faaa2400700 1 -- 172.21.15.90:0/3892959926 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c0737d0 con 0x7faa9c103b90 2024-05-27T07:54:17.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.401+0000 7faa98c00700 1 -- 172.21.15.90:0/3892959926 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa8400cd70 con 0x7faa9c103b90 2024-05-27T07:54:17.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.401+0000 7faa98c00700 1 -- 172.21.15.90:0/3892959926 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84005410 con 0x7faa9c103b90 2024-05-27T07:54:17.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.409+0000 7faab0ef3b80 1 Processor -- start 2024-05-27T07:54:17.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.411+0000 7faab0ef3b80 1 -- start start 2024-05-27T07:54:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.411+0000 7faa98c00700 1 -- 172.21.15.90:0/3892959926 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faa84004750 con 0x7faa9c103b90 2024-05-27T07:54:17.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.412+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559650f39e40 0x5596510be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.412+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 0x5596510b8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.413+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 0x5596510b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.413+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559650fe6e40 con 0x559650f39e40 2024-05-27T07:54:17.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.413+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559650fd69e0 con 0x5596510b8db0 2024-05-27T07:54:17.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.413+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559650fdaa20 con 0x5596510bed30 2024-05-27T07:54:17.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.414+0000 7faa9be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559650f39e40 0x5596510be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.414+0000 7faa9be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559650f39e40 0x5596510be7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58082/0 (socket says 172.21.15.90:58082) 2024-05-27T07:54:17.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.414+0000 7faa9be00700 1 -- 172.21.15.90:0/2835347746 learned_addr learned my addr 172.21.15.90:0/2835347746 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:17.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.414+0000 7faa9be00700 1 -- 172.21.15.90:0/2835347746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 msgr2=0x5596510b8870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:54:17.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.415+0000 7faa93e00700 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 0x5596510b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.415+0000 7faa98c00700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7faa800b2af0 0x7faa800b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.416+0000 7faa98c00700 1 -- 172.21.15.90:0/3892959926 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7faa840e2ad0 con 0x7faa9c103b90 2024-05-27T07:54:17.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.416+0000 7faa9b400700 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 0x5596510b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.416+0000 7faa9be00700 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 0x5596510b8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.417+0000 7faa9be00700 1 -- 172.21.15.90:0/2835347746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 msgr2=0x5596510b91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:17.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.417+0000 7faa9be00700 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 0x5596510b91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.417+0000 7faa9be00700 1 -- 172.21.15.90:0/2835347746 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559650f9f420 con 0x559650f39e40 2024-05-27T07:54:17.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.418+0000 7faa93e00700 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 0x5596510b91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:17.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.418+0000 7faa9b400700 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 0x5596510b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:17.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.418+0000 7faa9be00700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7faa800b2af0 0x7faa800b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.419+0000 7faa9be00700 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559650f39e40 0x5596510be7f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7faa94007f30 tx=0x7faa9400c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c99222320bc67c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:17.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.420+0000 7faa90c00700 1 -- 172.21.15.90:0/2835347746 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9401c850 con 0x559650f39e40 2024-05-27T07:54:17.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.421+0000 7faa90c00700 1 -- 172.21.15.90:0/2835347746 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa94014050 con 0x559650f39e40 2024-05-27T07:54:17.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.421+0000 7faa90c00700 1 -- 172.21.15.90:0/2835347746 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9400e5e0 con 0x559650f39e40 2024-05-27T07:54:17.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.422+0000 7faab0ef3b80 1 -- 172.21.15.90:0/2835347746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559650f39e40 msgr2=0x5596510be7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:17.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.423+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559650f39e40 0x5596510be7f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7faa94007f30 tx=0x7faa9400c5a0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.422+0000 7faa9be00700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7faa800b2af0 0x7faa800b5000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa9401b000 tx=0x7faa9401e000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:17.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.424+0000 7faab0ef3b80 1 -- 172.21.15.90:0/2835347746 shutdown_connections 2024-05-27T07:54:17.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.424+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 0x5596510b91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.424+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559650f39e40 0x5596510be7f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.424+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/2835347746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 0x5596510b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.424+0000 7faab0ef3b80 1 -- 172.21.15.90:0/2835347746 >> 172.21.15.90:0/2835347746 conn(0x559650f99fc0 msgr2=0x5596510b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:17.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.425+0000 7faab0ef3b80 1 -- 172.21.15.90:0/2835347746 shutdown_connections 2024-05-27T07:54:17.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.425+0000 7faab0ef3b80 1 -- 172.21.15.90:0/2835347746 wait complete. 2024-05-27T07:54:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.426+0000 7faab0ef3b80 1 -- start start 2024-05-27T07:54:17.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.427+0000 7faab0ef3b80 1 Processor -- start 2024-05-27T07:54:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.428+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559650f39e40 0x5596510c2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.428+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510b8db0 0x5596510c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.428+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 0x5596510c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.428+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559650edcfa0 con 0x5596510bed30 2024-05-27T07:54:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.428+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559650f77760 con 0x559650f39e40 2024-05-27T07:54:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.428+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5596510c5800 con 0x5596510b8db0 2024-05-27T07:54:17.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.428+0000 7faab0ef3b80 10 client.0 _ll_register_callbacks cb 0x559650f80c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:17.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.429+0000 7faa93e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510b8db0 0x5596510c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.429+0000 7faa9b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559650f39e40 0x5596510c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.429+0000 7faa93e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510b8db0 0x5596510c29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42012/0 (socket says 172.21.15.90:42012) 2024-05-27T07:54:17.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.429+0000 7faa9be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 0x5596510c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.429+0000 7faa9be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 0x5596510c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58086/0 (socket says 172.21.15.90:58086) 2024-05-27T07:54:17.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.429+0000 7faa9be00700 1 -- 172.21.15.90:0/3425747544 learned_addr learned my addr 172.21.15.90:0/3425747544 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:17.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.430+0000 7faa9be00700 1 -- 172.21.15.90:0/3425747544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510b8db0 msgr2=0x5596510c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:17.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.430+0000 7faa9be00700 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510b8db0 0x5596510c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.430+0000 7faa9be00700 1 -- 172.21.15.90:0/3425747544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559650f39e40 msgr2=0x5596510c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:17.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.430+0000 7faa9be00700 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559650f39e40 0x5596510c2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.430+0000 7faa9be00700 1 -- 172.21.15.90:0/3425747544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559650f268d0 con 0x5596510bed30 2024-05-27T07:54:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.431+0000 7faa9be00700 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 0x5596510c2ee0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faa94004c20 tx=0x7faa9401b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d05ba2a520f52c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.431+0000 7faa89600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:54:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.432+0000 7faa89600700 1 -- 172.21.15.90:0/3425747544 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9401c7d0 con 0x5596510bed30 2024-05-27T07:54:17.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.432+0000 7faa89600700 1 -- 172.21.15.90:0/3425747544 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7faa9401c970 con 0x5596510bed30 2024-05-27T07:54:17.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.432+0000 7faab0ef3b80 1 -- 172.21.15.90:0/3425747544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa94014050 con 0x5596510bed30 2024-05-27T07:54:17.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.432+0000 7faa89600700 1 -- 172.21.15.90:0/3425747544 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7faa9401cb70 con 0x5596510bed30 2024-05-27T07:54:17.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.433+0000 7faab0ef3b80 10 client.16989 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:17.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.433+0000 7faab0ef3b80 1 -- 172.21.15.90:0/3425747544 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x559650fa7060 con 0x5596510bed30 2024-05-27T07:54:17.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.433+0000 7faa89600700 1 -- 172.21.15.90:0/3425747544 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa94014050 con 0x5596510bed30 2024-05-27T07:54:17.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.434+0000 7faa89600700 1 -- 172.21.15.90:0/3425747544 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7faa94031890 con 0x5596510bed30 2024-05-27T07:54:17.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.434+0000 7faab0ef3b80 10 client.16989 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:17.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.434+0000 7faab0ef3b80 10 client.16989 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:54:17.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.434+0000 7faab0ef3b80 20 client.16989 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:17.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.435+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5596510e3440 0x5596510e5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.435+0000 7faab0ef3b80 4 client.16989 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:17.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.435+0000 7faab0ef3b80 1 -- 172.21.15.90:0/3425747544 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559650fe8d90 con 0x5596510e3440 2024-05-27T07:54:17.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.437+0000 7faa9b400700 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5596510e3440 0x5596510e5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.438+0000 7faa9b400700 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5596510e3440 0x5596510e5860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e6fac43f96ecf7eb server_cookie=6cd3bdd94f5ce53f in_seq=0 out_seq=0 2024-05-27T07:54:17.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.438+0000 7faa89600700 10 client.16989 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:54:17.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.439+0000 7faa89600700 1 -- 172.21.15.90:0/3425747544 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559650fe8d90 con 0x5596510e3440 2024-05-27T07:54:17.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.439+0000 7faa89600700 10 client.16989 handle_command_reply: tid=0 2024-05-27T07:54:17.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.441+0000 7faab0ef3b80 1 client.16989 shutdown 2024-05-27T07:54:17.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.442+0000 7faab0ef3b80 1 -- 172.21.15.90:0/3425747544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 msgr2=0x5596510c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:17.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.442+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 0x5596510c2ee0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faa94004c20 tx=0x7faa9401b880 comp rx=0 tx=0).stop 2024-05-27T07:54:17.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.443+0000 7faab0ef3b80 1 -- 172.21.15.90:0/3425747544 shutdown_connections 2024-05-27T07:54:17.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.443+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5596510e3440 0x5596510e5860 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.443+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 0x5596510c2ee0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.443+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559650f39e40 0x5596510c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.443+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/3425747544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510b8db0 0x5596510c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.443+0000 7faa89600700 0 client.16989 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:54:17.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.443+0000 7faab0ef3b80 1 -- 172.21.15.90:0/3425747544 >> 172.21.15.90:0/3425747544 conn(0x559650f99fc0 msgr2=0x5596510b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:17.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.443+0000 7faab0ef3b80 1 -- 172.21.15.90:0/3425747544 shutdown_connections 2024-05-27T07:54:17.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.444+0000 7faab0ef3b80 1 -- 172.21.15.90:0/3425747544 wait complete. 2024-05-27T07:54:17.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.444+0000 7faab0ef3b80 20 client.16989 trim_cache size 0 max 16384 2024-05-27T07:54:17.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.475+0000 7faab0ef3b80 1 Processor -- start 2024-05-27T07:54:17.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.476+0000 7faab0ef3b80 1 -- start start 2024-05-27T07:54:17.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.476+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559650f39e40 0x5596510b8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.476+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510b8db0 0x5596510bc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.476+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 0x5596510bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.476+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faa78010880 con 0x5596510b8db0 2024-05-27T07:54:17.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.476+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800e670 con 0x559650f39e40 2024-05-27T07:54:17.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.476+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa780101b0 con 0x5596510bed30 2024-05-27T07:54:17.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.477+0000 7faa9be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 0x5596510bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.477+0000 7faa93e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510b8db0 0x5596510bc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.477+0000 7faa93e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510b8db0 0x5596510bc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58094/0 (socket says 172.21.15.90:58094) 2024-05-27T07:54:17.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.477+0000 7faa93e00700 1 -- 172.21.15.90:0/2645169703 learned_addr learned my addr 172.21.15.90:0/2645169703 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:17.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.477+0000 7faa9b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559650f39e40 0x5596510b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.478+0000 7faa93e00700 1 -- 172.21.15.90:0/2645169703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 msgr2=0x5596510bf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:17.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.478+0000 7faa93e00700 1 --2- 172.21.15.90:0/2645169703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 0x5596510bf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.478+0000 7faa93e00700 1 -- 172.21.15.90:0/2645169703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559650f39e40 msgr2=0x5596510b8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:17.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.478+0000 7faa93e00700 1 --2- 172.21.15.90:0/2645169703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559650f39e40 0x5596510b8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.478+0000 7faa93e00700 1 -- 172.21.15.90:0/2645169703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596510c3250 con 0x5596510b8db0 2024-05-27T07:54:17.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.481+0000 7faa93e00700 1 --2- 172.21.15.90:0/2645169703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510b8db0 0x5596510bc850 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7faa840a6bf0 tx=0x7faa84000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc2748c31daa9cd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:17.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.482+0000 7faa8be00700 1 -- 172.21.15.90:0/2645169703 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84005670 con 0x5596510b8db0 2024-05-27T07:54:17.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.482+0000 7faa8be00700 1 -- 172.21.15.90:0/2645169703 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faa840b2120 con 0x5596510b8db0 2024-05-27T07:54:17.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.482+0000 7faa8be00700 1 -- 172.21.15.90:0/2645169703 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84005670 con 0x5596510b8db0 2024-05-27T07:54:17.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.484+0000 7faab0ef3b80 1 -- 172.21.15.90:0/2645169703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510b8db0 msgr2=0x5596510bc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:17.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.484+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/2645169703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510b8db0 0x5596510bc850 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7faa840a6bf0 tx=0x7faa84000bc0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.486+0000 7faab0ef3b80 1 -- 172.21.15.90:0/2645169703 shutdown_connections 2024-05-27T07:54:17.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.486+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/2645169703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5596510bed30 0x5596510bf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.486+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/2645169703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559650f39e40 0x5596510b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.486+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/2645169703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510b8db0 0x5596510bc850 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.486+0000 7faab0ef3b80 1 -- 172.21.15.90:0/2645169703 >> 172.21.15.90:0/2645169703 conn(0x559650f99fc0 msgr2=0x5596510bc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:17.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.487+0000 7faab0ef3b80 1 -- 172.21.15.90:0/2645169703 shutdown_connections 2024-05-27T07:54:17.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.487+0000 7faab0ef3b80 1 -- 172.21.15.90:0/2645169703 wait complete. 2024-05-27T07:54:17.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.488+0000 7faab0ef3b80 1 -- start start 2024-05-27T07:54:17.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.488+0000 7faab0ef3b80 1 Processor -- start 2024-05-27T07:54:17.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.489+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559650f39e40 0x5596510cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.489+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 0x5596510cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.489+0000 7faab0ef3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 0x5596510ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.489+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800f410 con 0x5596510bed30 2024-05-27T07:54:17.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.489+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800ed40 con 0x5596510b8db0 2024-05-27T07:54:17.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.489+0000 7faab0ef3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800fae0 con 0x559650f39e40 2024-05-27T07:54:17.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.489+0000 7faab0ef3b80 10 client.0 _ll_register_callbacks cb 0x559650f80c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:17.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.490+0000 7faa9b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559650f39e40 0x5596510cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.490+0000 7faa9b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559650f39e40 0x5596510cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42042/0 (socket says 172.21.15.90:42042) 2024-05-27T07:54:17.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.490+0000 7faa9b400700 1 -- 172.21.15.90:0/776907012 learned_addr learned my addr 172.21.15.90:0/776907012 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:17.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.490+0000 7faa9be00700 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 0x5596510ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.490+0000 7faa93e00700 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 0x5596510cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.491+0000 7faa9b400700 1 -- 172.21.15.90:0/776907012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 msgr2=0x5596510cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:17.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.491+0000 7faa9b400700 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 0x5596510cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.491+0000 7faa9b400700 1 -- 172.21.15.90:0/776907012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 msgr2=0x5596510ea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:17.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.491+0000 7faa9b400700 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 0x5596510ea270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.491+0000 7faa9b400700 1 -- 172.21.15.90:0/776907012 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596510ea9f0 con 0x559650f39e40 2024-05-27T07:54:17.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.492+0000 7faa9b400700 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559650f39e40 0x5596510cad10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0098c0 tx=0x7faa8c00f550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96105a433eed077c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:17.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.492+0000 7faa89600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:54:17.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.492+0000 7faa89600700 1 -- 172.21.15.90:0/776907012 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c005240 con 0x559650f39e40 2024-05-27T07:54:17.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.492+0000 7faa89600700 1 -- 172.21.15.90:0/776907012 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7faa8c0053e0 con 0x559650f39e40 2024-05-27T07:54:17.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.492+0000 7faab0ef3b80 1 -- 172.21.15.90:0/776907012 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559650fe2570 con 0x559650f39e40 2024-05-27T07:54:17.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.493+0000 7faa89600700 1 -- 172.21.15.90:0/776907012 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7faa8c0055e0 con 0x559650f39e40 2024-05-27T07:54:17.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.493+0000 7faab0ef3b80 10 client.26575 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:17.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.493+0000 7faab0ef3b80 1 -- 172.21.15.90:0/776907012 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55965114d710 con 0x559650f39e40 2024-05-27T07:54:17.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.494+0000 7faa89600700 1 -- 172.21.15.90:0/776907012 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559650fe2570 con 0x559650f39e40 2024-05-27T07:54:17.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.495+0000 7faa89600700 1 -- 172.21.15.90:0/776907012 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7faa8c00b790 con 0x559650f39e40 2024-05-27T07:54:17.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.495+0000 7faab0ef3b80 10 client.26575 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:17.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.495+0000 7faab0ef3b80 10 client.26575 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:54:17.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.495+0000 7faab0ef3b80 20 client.26575 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:17.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.495+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55965114de20 0x559651150240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:17.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.495+0000 7faab0ef3b80 4 client.26575 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:54:17.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.495+0000 7faab0ef3b80 1 -- 172.21.15.90:0/776907012 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559650e59250 con 0x55965114de20 2024-05-27T07:54:17.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.495+0000 7faa93e00700 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55965114de20 0x559651150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:17.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.496+0000 7faa93e00700 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55965114de20 0x559651150240 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e34ea0b8f0fc9a20 server_cookie=cd8ab7896607cd92 in_seq=0 out_seq=0 2024-05-27T07:54:17.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.496+0000 7faa89600700 10 client.26575 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:54:17.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faa89600700 1 -- 172.21.15.90:0/776907012 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559650e59250 con 0x55965114de20 2024-05-27T07:54:17.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faa89600700 10 client.26575 handle_command_reply: tid=0 2024-05-27T07:54:17.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faab0ef3b80 1 client.26575 shutdown 2024-05-27T07:54:17.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faab0ef3b80 1 -- 172.21.15.90:0/776907012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559650f39e40 msgr2=0x5596510cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:17.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559650f39e40 0x5596510cad10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faa8c0098c0 tx=0x7faa8c00f550 comp rx=0 tx=0).stop 2024-05-27T07:54:17.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faab0ef3b80 1 -- 172.21.15.90:0/776907012 shutdown_connections 2024-05-27T07:54:17.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55965114de20 0x559651150240 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5596510bed30 0x5596510ea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559650f39e40 0x5596510cad10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faab0ef3b80 1 --2- 172.21.15.90:0/776907012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596510b8db0 0x5596510cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.497+0000 7faab0ef3b80 1 -- 172.21.15.90:0/776907012 >> 172.21.15.90:0/776907012 conn(0x559650f99fc0 msgr2=0x5596510bdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:17.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.498+0000 7faa89600700 0 client.26575 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:54:17.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.498+0000 7faab0ef3b80 1 -- 172.21.15.90:0/776907012 shutdown_connections 2024-05-27T07:54:17.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.498+0000 7faab0ef3b80 1 -- 172.21.15.90:0/776907012 wait complete. 2024-05-27T07:54:17.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.499+0000 7faab0ef3b80 20 client.26575 trim_cache size 0 max 16384 2024-05-27T07:54:17.501 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:54:17.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.501+0000 7faaa2400700 1 -- 172.21.15.90:0/3892959926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7faa800b2af0 msgr2=0x7faa800b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:17.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.501+0000 7faaa2400700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7faa800b2af0 0x7faa800b5000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faa9401b000 tx=0x7faa9401e000 comp rx=0 tx=0).stop 2024-05-27T07:54:17.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.501+0000 7faaa2400700 1 -- 172.21.15.90:0/3892959926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa9c103b90 msgr2=0x7faa9c06e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:17.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.501+0000 7faaa2400700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa9c103b90 0x7faa9c06e5a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7faa8400e9c0 tx=0x7faa8400ecd0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.501+0000 7faaa2400700 1 -- 172.21.15.90:0/3892959926 shutdown_connections 2024-05-27T07:54:17.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.501+0000 7faaa2400700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7faa800b2af0 0x7faa800b5000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.501+0000 7faaa2400700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa9c108690 0x7faa9c06ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.501+0000 7faaa2400700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa9c103240 0x7faa9c075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.501+0000 7faaa2400700 1 --2- 172.21.15.90:0/3892959926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa9c103b90 0x7faa9c06e5a0 secure :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7faa8400e9c0 tx=0x7faa8400ecd0 comp rx=0 tx=0).stop 2024-05-27T07:54:17.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.501+0000 7faaa2400700 1 -- 172.21.15.90:0/3892959926 >> 172.21.15.90:0/3892959926 conn(0x7faa9c0fedb0 msgr2=0x7faa9c106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:17.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.503+0000 7faaa2400700 1 -- 172.21.15.90:0/3892959926 shutdown_connections 2024-05-27T07:54:17.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:17.507+0000 7faaa2400700 1 -- 172.21.15.90:0/3892959926 wait complete. 2024-05-27T07:54:18.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:18 smithi090 ceph-mon[31429]: pgmap v249: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.7 MiB/s wr, 621 op/s 2024-05-27T07:54:18.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:18 smithi136 ceph-mon[35324]: pgmap v249: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.7 MiB/s wr, 621 op/s 2024-05-27T07:54:18.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:18 smithi129 ceph-mon[35254]: pgmap v249: 129 pgs: 129 active+clean; 2.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 8.7 MiB/s wr, 621 op/s 2024-05-27T07:54:18.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 240.0 s, 596.9 tps, lat 167.908 ms stddev 244.887 2024-05-27T07:54:19.391 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:54:20.099 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:20.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:20 smithi090 ceph-mon[31429]: pgmap v250: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.7 MiB/s wr, 579 op/s 2024-05-27T07:54:20.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:20 smithi136 ceph-mon[35324]: pgmap v250: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.7 MiB/s wr, 579 op/s 2024-05-27T07:54:20.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:20 smithi129 ceph-mon[35254]: pgmap v250: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.7 MiB/s wr, 579 op/s 2024-05-27T07:54:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.873+0000 7f1170200640 1 -- 172.21.15.90:0/3715184918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116810beb0 msgr2=0x7f1168105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.873+0000 7f1170200640 1 --2- 172.21.15.90:0/3715184918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116810beb0 0x7f1168105a50 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f115c013f10 tx=0x7f115c032980 comp rx=0 tx=0).stop 2024-05-27T07:54:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.874+0000 7f1170200640 1 -- 172.21.15.90:0/3715184918 shutdown_connections 2024-05-27T07:54:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.874+0000 7f1170200640 1 --2- 172.21.15.90:0/3715184918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f116810c350 0x7f1168105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.874+0000 7f1170200640 1 --2- 172.21.15.90:0/3715184918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1168106600 0x7f11681129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.874+0000 7f1170200640 1 --2- 172.21.15.90:0/3715184918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116810beb0 0x7f1168105a50 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.874+0000 7f1170200640 1 -- 172.21.15.90:0/3715184918 >> 172.21.15.90:0/3715184918 conn(0x7f11681016b0 msgr2=0x7f1168103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.874+0000 7f1170200640 1 -- 172.21.15.90:0/3715184918 shutdown_connections 2024-05-27T07:54:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.875+0000 7f1170200640 1 -- 172.21.15.90:0/3715184918 wait complete. 2024-05-27T07:54:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.877+0000 7f1170200640 1 Processor -- start 2024-05-27T07:54:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.877+0000 7f1170200640 1 -- start start 2024-05-27T07:54:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.879+0000 7f1170200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1168106600 0x7f11680744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.879+0000 7f1170200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116810beb0 0x7f116806c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.879+0000 7f1170200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116810c350 0x7f116806cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.879+0000 7f1170200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f116807a8b0 con 0x7f116810c350 2024-05-27T07:54:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.879+0000 7f1170200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f116807a730 con 0x7f1168106600 2024-05-27T07:54:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.879+0000 7f1170200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f116807aa30 con 0x7f116810beb0 2024-05-27T07:54:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.880+0000 7f116e200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116810c350 0x7f116806cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.880+0000 7f116d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1168106600 0x7f11680744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.880+0000 7f116d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1168106600 0x7f11680744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39898/0 (socket says 172.21.15.90:39898) 2024-05-27T07:54:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.880+0000 7f116d800640 1 -- 172.21.15.90:0/529801140 learned_addr learned my addr 172.21.15.90:0/529801140 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.880+0000 7f116ce00640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116810beb0 0x7f116806c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.881+0000 7f116ce00640 1 -- 172.21.15.90:0/529801140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1168106600 msgr2=0x7f11680744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.881+0000 7f116ce00640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1168106600 0x7f11680744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.881+0000 7f116ce00640 1 -- 172.21.15.90:0/529801140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116810c350 msgr2=0x7f116806cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.881+0000 7f116ce00640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116810c350 0x7f116806cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.881+0000 7f116ce00640 1 -- 172.21.15.90:0/529801140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116806d160 con 0x7f116810beb0 2024-05-27T07:54:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.881+0000 7f116e200640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116810c350 0x7f116806cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.882+0000 7f116ce00640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116810beb0 0x7f116806c6e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f116806c670 tx=0x7f115400ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.882+0000 7f1166000640 1 -- 172.21.15.90:0/529801140 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154007b30 con 0x7f116810beb0 2024-05-27T07:54:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.882+0000 7f1166000640 1 -- 172.21.15.90:0/529801140 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1154007cd0 con 0x7f116810beb0 2024-05-27T07:54:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.882+0000 7f1166000640 1 -- 172.21.15.90:0/529801140 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154007b30 con 0x7f116810beb0 2024-05-27T07:54:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.882+0000 7f1170200640 1 -- 172.21.15.90:0/529801140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116806d450 con 0x7f116810beb0 2024-05-27T07:54:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.883+0000 7f1170200640 1 -- 172.21.15.90:0/529801140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11681ad820 con 0x7f116810beb0 2024-05-27T07:54:21.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.889+0000 7f117721a740 1 Processor -- start 2024-05-27T07:54:21.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.889+0000 7f117721a740 1 -- start start 2024-05-27T07:54:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.890+0000 7f1166000640 1 -- 172.21.15.90:0/529801140 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f115400f450 con 0x7f116810beb0 2024-05-27T07:54:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.890+0000 7f117721a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e29e1b0 0x55f63e640860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.890+0000 7f117721a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 0x55f63e640da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.890+0000 7f117721a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e6412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.890+0000 7f117721a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f63e5b1b90 con 0x55f63e29e1b0 2024-05-27T07:54:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.890+0000 7f117721a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f63e5a1730 con 0x55f63e27d4d0 2024-05-27T07:54:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.890+0000 7f117721a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f63e5a5770 con 0x55f63e2752f0 2024-05-27T07:54:21.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.891+0000 7f116d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 0x55f63e640da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.891+0000 7f116d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 0x55f63e640da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59128/0 (socket says 172.21.15.90:59128) 2024-05-27T07:54:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.891+0000 7f116d800640 1 -- 172.21.15.90:0/425792334 learned_addr learned my addr 172.21.15.90:0/425792334 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.892+0000 7f116e200640 1 --2- 172.21.15.90:0/425792334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e29e1b0 0x55f63e640860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.892+0000 7f1166000640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2c50 0x7f11500b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.892+0000 7f116d800640 1 -- 172.21.15.90:0/425792334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 msgr2=0x55f63e6412e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:54:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.892+0000 7f116d800640 1 --2- 172.21.15.90:0/425792334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e6412e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.892+0000 7f116d800640 1 -- 172.21.15.90:0/425792334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e29e1b0 msgr2=0x55f63e640860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.892+0000 7f1166000640 1 -- 172.21.15.90:0/529801140 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f11540e26a0 con 0x7f116810beb0 2024-05-27T07:54:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.892+0000 7f116d800640 1 --2- 172.21.15.90:0/425792334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e29e1b0 0x55f63e640860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.892+0000 7f116d800640 1 -- 172.21.15.90:0/425792334 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f63e53a8b0 con 0x55f63e2752f0 2024-05-27T07:54:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.892+0000 7f116d800640 1 --2- 172.21.15.90:0/425792334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 0x55f63e640da0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f115c012fb0 tx=0x7f115c005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeedfb4c586690d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.893+0000 7f116e200640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2c50 0x7f11500b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.893+0000 7f115aa00640 1 -- 172.21.15.90:0/425792334 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c005460 con 0x55f63e2752f0 2024-05-27T07:54:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.893+0000 7f115aa00640 1 -- 172.21.15.90:0/425792334 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115c005600 con 0x55f63e2752f0 2024-05-27T07:54:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.893+0000 7f115aa00640 1 -- 172.21.15.90:0/425792334 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c0035f0 con 0x55f63e2752f0 2024-05-27T07:54:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.894+0000 7f117721a740 1 -- 172.21.15.90:0/425792334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 msgr2=0x55f63e640da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.894+0000 7f117721a740 1 --2- 172.21.15.90:0/425792334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 0x55f63e640da0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f115c012fb0 tx=0x7f115c005040 comp rx=0 tx=0).stop 2024-05-27T07:54:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.894+0000 7f116e200640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2c50 0x7f11500b5160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1160005c40 tx=0x7f1160004a40 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.895+0000 7f117721a740 1 -- 172.21.15.90:0/425792334 shutdown_connections 2024-05-27T07:54:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.895+0000 7f117721a740 1 --2- 172.21.15.90:0/425792334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e6412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.895+0000 7f117721a740 1 --2- 172.21.15.90:0/425792334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 0x55f63e640da0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.895+0000 7f117721a740 1 --2- 172.21.15.90:0/425792334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e29e1b0 0x55f63e640860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.895+0000 7f117721a740 1 -- 172.21.15.90:0/425792334 >> 172.21.15.90:0/425792334 conn(0x55f63e292300 msgr2=0x55f63e63d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.895+0000 7f117721a740 1 -- 172.21.15.90:0/425792334 shutdown_connections 2024-05-27T07:54:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.895+0000 7f117721a740 1 -- 172.21.15.90:0/425792334 wait complete. 2024-05-27T07:54:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.895+0000 7f117721a740 1 -- start start 2024-05-27T07:54:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.896+0000 7f117721a740 1 Processor -- start 2024-05-27T07:54:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f117721a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e2752f0 0x55f63e64ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f117721a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e64b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f117721a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e667c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f117721a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f63e287790 con 0x55f63e2752f0 2024-05-27T07:54:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f117721a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f63e549bc0 con 0x55f63e27d4d0 2024-05-27T07:54:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f117721a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f63e65e8e0 con 0x55f63e29e1b0 2024-05-27T07:54:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f117721a740 10 client.0 _ll_register_callbacks cb 0x55f63e527400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f116d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e2752f0 0x55f63e64ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f116ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e64b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f116d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e2752f0 0x55f63e64ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48624/0 (socket says 172.21.15.90:48624) 2024-05-27T07:54:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.897+0000 7f116d800640 1 -- 172.21.15.90:0/1506576852 learned_addr learned my addr 172.21.15.90:0/1506576852 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.898+0000 7f116e200640 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e667c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.898+0000 7f116ce00640 1 -- 172.21.15.90:0/1506576852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 msgr2=0x55f63e667c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.898+0000 7f116ce00640 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e667c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.898+0000 7f116ce00640 1 -- 172.21.15.90:0/1506576852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e2752f0 msgr2=0x55f63e64ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.898+0000 7f116ce00640 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e2752f0 0x55f63e64ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.898+0000 7f116ce00640 1 -- 172.21.15.90:0/1506576852 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63e2b6210 con 0x55f63e27d4d0 2024-05-27T07:54:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.898+0000 7f116d800640 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e2752f0 0x55f63e64ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.899+0000 7f116e200640 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e667c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.899+0000 7f116ce00640 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e64b270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11540df9c0 tx=0x7f11540a8d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a2968b2c0bfc4a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.899+0000 7f114f400640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:54:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.899+0000 7f114f400640 1 -- 172.21.15.90:0/1506576852 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11540ad900 con 0x55f63e27d4d0 2024-05-27T07:54:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.899+0000 7f117721a740 1 -- 172.21.15.90:0/1506576852 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115c005600 con 0x55f63e27d4d0 2024-05-27T07:54:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.900+0000 7f114f400640 1 -- 172.21.15.90:0/1506576852 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f11540a57e0 con 0x55f63e27d4d0 2024-05-27T07:54:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.900+0000 7f114f400640 1 -- 172.21.15.90:0/1506576852 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f11540b23b0 con 0x55f63e27d4d0 2024-05-27T07:54:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.900+0000 7f117721a740 10 client.36560 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.900+0000 7f117721a740 1 -- 172.21.15.90:0/1506576852 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55f63e382620 con 0x55f63e27d4d0 2024-05-27T07:54:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.900+0000 7f114f400640 1 -- 172.21.15.90:0/1506576852 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115c005600 con 0x55f63e27d4d0 2024-05-27T07:54:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.902+0000 7f114f400640 1 -- 172.21.15.90:0/1506576852 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7f11540b1f20 con 0x55f63e27d4d0 2024-05-27T07:54:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.902+0000 7f117721a740 10 client.36560 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.902+0000 7f117721a740 10 client.36560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.902+0000 7f117721a740 20 client.36560 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.902+0000 7f117721a740 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f63e25cd80 0x55f63e66e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.902+0000 7f117721a740 4 client.36560 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.902+0000 7f117721a740 1 -- 172.21.15.90:0/1506576852 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f63e5b3ae0 con 0x55f63e25cd80 2024-05-27T07:54:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.903+0000 7f116d800640 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f63e25cd80 0x55f63e66e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.904+0000 7f116d800640 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f63e25cd80 0x55f63e66e120 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8bc77dc77d3d9a2 server_cookie=c337fcd21491e370 in_seq=0 out_seq=0 2024-05-27T07:54:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.904+0000 7f114f400640 10 client.36560 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.905+0000 7f114f400640 1 -- 172.21.15.90:0/1506576852 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f63e5b3ae0 con 0x55f63e25cd80 2024-05-27T07:54:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.905+0000 7f114f400640 10 client.36560 handle_command_reply: tid=0 2024-05-27T07:54:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.905+0000 7f117721a740 1 client.36560 shutdown 2024-05-27T07:54:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.906+0000 7f117721a740 1 -- 172.21.15.90:0/1506576852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 msgr2=0x55f63e64b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.906+0000 7f117721a740 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e64b270 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11540df9c0 tx=0x7f11540a8d80 comp rx=0 tx=0).stop 2024-05-27T07:54:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.906+0000 7f117721a740 1 -- 172.21.15.90:0/1506576852 shutdown_connections 2024-05-27T07:54:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.906+0000 7f117721a740 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f63e25cd80 0x55f63e66e120 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.906+0000 7f117721a740 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e64b270 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.906+0000 7f117721a740 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e667c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.906+0000 7f117721a740 1 --2- 172.21.15.90:0/1506576852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e2752f0 0x55f63e64ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.906+0000 7f117721a740 1 -- 172.21.15.90:0/1506576852 >> 172.21.15.90:0/1506576852 conn(0x55f63e292300 msgr2=0x55f63e642580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.906+0000 7f114f400640 0 client.36560 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.907+0000 7f117721a740 1 -- 172.21.15.90:0/1506576852 shutdown_connections 2024-05-27T07:54:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.908+0000 7f117721a740 1 -- 172.21.15.90:0/1506576852 wait complete. 2024-05-27T07:54:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.908+0000 7f117721a740 20 client.36560 trim_cache size 0 max 16384 2024-05-27T07:54:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.943+0000 7f117721a740 1 Processor -- start 2024-05-27T07:54:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.943+0000 7f117721a740 1 -- start start 2024-05-27T07:54:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.943+0000 7f117721a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e2752f0 0x55f63e6462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.944+0000 7f117721a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e27d4d0 0x55f63e641210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.944+0000 7f117721a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e64c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.944+0000 7f117721a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b91e0 con 0x55f63e27d4d0 2024-05-27T07:54:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.944+0000 7f117721a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b6fd0 con 0x55f63e2752f0 2024-05-27T07:54:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.944+0000 7f117721a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b8b10 con 0x55f63e29e1b0 2024-05-27T07:54:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.944+0000 7f116e200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e64c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.944+0000 7f116e200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e64c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59148/0 (socket says 172.21.15.90:59148) 2024-05-27T07:54:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.944+0000 7f116e200640 1 -- 172.21.15.90:0/2482122266 learned_addr learned my addr 172.21.15.90:0/2482122266 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.944+0000 7f116d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e2752f0 0x55f63e6462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.945+0000 7f116e200640 1 -- 172.21.15.90:0/2482122266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e2752f0 msgr2=0x55f63e6462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.945+0000 7f116e200640 1 --2- 172.21.15.90:0/2482122266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e2752f0 0x55f63e6462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.945+0000 7f116e200640 1 -- 172.21.15.90:0/2482122266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e27d4d0 msgr2=0x55f63e641210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.946+0000 7f116ce00640 1 --2- 172.21.15.90:0/2482122266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e27d4d0 0x55f63e641210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.946+0000 7f116e200640 1 --2- 172.21.15.90:0/2482122266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e27d4d0 0x55f63e641210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.946+0000 7f116e200640 1 -- 172.21.15.90:0/2482122266 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f63e63dba0 con 0x55f63e29e1b0 2024-05-27T07:54:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.946+0000 7f116d800640 1 --2- 172.21.15.90:0/2482122266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e2752f0 0x55f63e6462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.948+0000 7f116e200640 1 --2- 172.21.15.90:0/2482122266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e64c950 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1160013830 tx=0x7f1160014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d074fe42201722b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.948+0000 7f115a000640 1 -- 172.21.15.90:0/2482122266 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116001eb90 con 0x55f63e29e1b0 2024-05-27T07:54:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.948+0000 7f115a000640 1 -- 172.21.15.90:0/2482122266 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f116001ed30 con 0x55f63e29e1b0 2024-05-27T07:54:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.948+0000 7f115a000640 1 -- 172.21.15.90:0/2482122266 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600263e0 con 0x55f63e29e1b0 2024-05-27T07:54:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.950+0000 7f117721a740 1 -- 172.21.15.90:0/2482122266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 msgr2=0x55f63e64c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.950+0000 7f117721a740 1 --2- 172.21.15.90:0/2482122266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e64c950 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1160013830 tx=0x7f1160014000 comp rx=0 tx=0).stop 2024-05-27T07:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.950+0000 7f117721a740 1 -- 172.21.15.90:0/2482122266 shutdown_connections 2024-05-27T07:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.950+0000 7f117721a740 1 --2- 172.21.15.90:0/2482122266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e29e1b0 0x55f63e64c950 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.950+0000 7f117721a740 1 --2- 172.21.15.90:0/2482122266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e27d4d0 0x55f63e641210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.950+0000 7f117721a740 1 --2- 172.21.15.90:0/2482122266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e2752f0 0x55f63e6462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.950+0000 7f117721a740 1 -- 172.21.15.90:0/2482122266 >> 172.21.15.90:0/2482122266 conn(0x55f63e292300 msgr2=0x55f63e64c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.954+0000 7f117721a740 1 -- 172.21.15.90:0/2482122266 shutdown_connections 2024-05-27T07:54:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.954+0000 7f117721a740 1 -- 172.21.15.90:0/2482122266 wait complete. 2024-05-27T07:54:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.955+0000 7f117721a740 1 -- start start 2024-05-27T07:54:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.955+0000 7f117721a740 1 Processor -- start 2024-05-27T07:54:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.956+0000 7f117721a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 0x55f63e65eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.956+0000 7f117721a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e65f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.957+0000 7f117721a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e29e1b0 0x55f63e65f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.957+0000 7f116ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e65f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.957+0000 7f116ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e65f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39926/0 (socket says 172.21.15.90:39926) 2024-05-27T07:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.957+0000 7f117721a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b7d70 con 0x55f63e29e1b0 2024-05-27T07:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.957+0000 7f117721a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b76a0 con 0x55f63e27d4d0 2024-05-27T07:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.957+0000 7f117721a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11681b8440 con 0x55f63e2752f0 2024-05-27T07:54:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.957+0000 7f117721a740 10 client.0 _ll_register_callbacks cb 0x7f1154007e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.958+0000 7f116d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 0x55f63e65eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.958+0000 7f116ce00640 1 -- 172.21.15.90:0/465440075 learned_addr learned my addr 172.21.15.90:0/465440075 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.958+0000 7f116e200640 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e29e1b0 0x55f63e65f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.959+0000 7f116ce00640 1 -- 172.21.15.90:0/465440075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 msgr2=0x55f63e65eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.959+0000 7f116ce00640 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 0x55f63e65eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.959+0000 7f116ce00640 1 -- 172.21.15.90:0/465440075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e29e1b0 msgr2=0x55f63e65f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.959+0000 7f116ce00640 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e29e1b0 0x55f63e65f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.959+0000 7f116ce00640 1 -- 172.21.15.90:0/465440075 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63e6d4810 con 0x55f63e27d4d0 2024-05-27T07:54:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.960+0000 7f116ce00640 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e65f070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f11540a9250 tx=0x7f11540aef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d023005213521e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.960+0000 7f114f400640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:54:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.961+0000 7f114f400640 1 -- 172.21.15.90:0/465440075 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11540aadb0 con 0x55f63e27d4d0 2024-05-27T07:54:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.961+0000 7f114f400640 1 -- 172.21.15.90:0/465440075 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f11540a97b0 con 0x55f63e27d4d0 2024-05-27T07:54:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.961+0000 7f117721a740 1 -- 172.21.15.90:0/465440075 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f116001ed30 con 0x55f63e27d4d0 2024-05-27T07:54:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.962+0000 7f114f400640 1 -- 172.21.15.90:0/465440075 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f115408b050 con 0x55f63e27d4d0 2024-05-27T07:54:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.962+0000 7f117721a740 10 client.36566 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.962+0000 7f117721a740 1 -- 172.21.15.90:0/465440075 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55f63e6d5a80 con 0x55f63e27d4d0 2024-05-27T07:54:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.963+0000 7f114f400640 1 -- 172.21.15.90:0/465440075 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f116001ed30 con 0x55f63e27d4d0 2024-05-27T07:54:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.964+0000 7f114f400640 1 -- 172.21.15.90:0/465440075 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7f11540b1f20 con 0x55f63e27d4d0 2024-05-27T07:54:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.964+0000 7f117721a740 10 client.36566 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.964+0000 7f117721a740 10 client.36566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.964+0000 7f117721a740 20 client.36566 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.964+0000 7f117721a740 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f63e2321c0 0x55f63e6d8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.964+0000 7f117721a740 4 client.36566 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:54:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.964+0000 7f117721a740 1 -- 172.21.15.90:0/465440075 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f63e5ad2c0 con 0x55f63e2321c0 2024-05-27T07:54:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.965+0000 7f116d800640 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f63e2321c0 0x55f63e6d8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.966+0000 7f116d800640 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f63e2321c0 0x55f63e6d8200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77be46f77b52ed1d server_cookie=8988e586cf0884ff in_seq=0 out_seq=0 2024-05-27T07:54:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:21.966+0000 7f114f400640 10 client.36566 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.004+0000 7f114f400640 1 -- 172.21.15.90:0/465440075 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f63e5ad2c0 con 0x55f63e2321c0 2024-05-27T07:54:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.005+0000 7f114f400640 10 client.36566 handle_command_reply: tid=0 2024-05-27T07:54:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.005+0000 7f117721a740 1 client.36566 shutdown 2024-05-27T07:54:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.006+0000 7f117721a740 1 -- 172.21.15.90:0/465440075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 msgr2=0x55f63e65f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.006+0000 7f117721a740 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e65f070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f11540a9250 tx=0x7f11540aef90 comp rx=0 tx=0).stop 2024-05-27T07:54:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.006+0000 7f117721a740 1 -- 172.21.15.90:0/465440075 shutdown_connections 2024-05-27T07:54:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.006+0000 7f117721a740 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f63e2321c0 0x55f63e6d8200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.006+0000 7f114f400640 0 client.36566 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.006+0000 7f117721a740 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f63e27d4d0 0x55f63e65f070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.006+0000 7f117721a740 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f63e29e1b0 0x55f63e65f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.006+0000 7f117721a740 1 --2- 172.21.15.90:0/465440075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f63e2752f0 0x55f63e65eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.006+0000 7f117721a740 1 -- 172.21.15.90:0/465440075 >> 172.21.15.90:0/465440075 conn(0x55f63e292300 msgr2=0x55f63e640ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.007+0000 7f117721a740 1 -- 172.21.15.90:0/465440075 shutdown_connections 2024-05-27T07:54:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.009+0000 7f117721a740 1 -- 172.21.15.90:0/465440075 wait complete. 2024-05-27T07:54:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.009+0000 7f117721a740 20 client.36566 trim_cache size 0 max 16384 2024-05-27T07:54:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:54:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:54:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:54:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.012+0000 7f1170200640 1 -- 172.21.15.90:0/529801140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2c50 msgr2=0x7f11500b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.012+0000 7f1170200640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2c50 0x7f11500b5160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1160005c40 tx=0x7f1160004a40 comp rx=0 tx=0).stop 2024-05-27T07:54:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.012+0000 7f1170200640 1 -- 172.21.15.90:0/529801140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116810beb0 msgr2=0x7f116806c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.012+0000 7f1170200640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116810beb0 0x7f116806c6e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f116806c670 tx=0x7f115400ce50 comp rx=0 tx=0).stop 2024-05-27T07:54:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.012+0000 7f1170200640 1 -- 172.21.15.90:0/529801140 shutdown_connections 2024-05-27T07:54:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.012+0000 7f1170200640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2c50 0x7f11500b5160 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.012+0000 7f1170200640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116810c350 0x7f116806cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.012+0000 7f1170200640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116810beb0 0x7f116806c6e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.012+0000 7f1170200640 1 --2- 172.21.15.90:0/529801140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1168106600 0x7f11680744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.012+0000 7f1170200640 1 -- 172.21.15.90:0/529801140 >> 172.21.15.90:0/529801140 conn(0x7f11681016b0 msgr2=0x7f1168110bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.013+0000 7f1170200640 1 -- 172.21.15.90:0/529801140 shutdown_connections 2024-05-27T07:54:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:22.013+0000 7f1170200640 1 -- 172.21.15.90:0/529801140 wait complete. 2024-05-27T07:54:22.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:22 smithi136 ceph-mon[35324]: pgmap v251: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.7 MiB/s wr, 579 op/s 2024-05-27T07:54:22.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:22 smithi129 ceph-mon[35254]: pgmap v251: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.7 MiB/s wr, 579 op/s 2024-05-27T07:54:22.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:22 smithi090 ceph-mon[31429]: pgmap v251: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.7 MiB/s wr, 579 op/s 2024-05-27T07:54:23.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 245.0 s, 523.2 tps, lat 191.282 ms stddev 241.077 2024-05-27T07:54:24.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:24 smithi136 ceph-mon[35324]: pgmap v252: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 9.4 MiB/s wr, 598 op/s 2024-05-27T07:54:24.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:24 smithi129 ceph-mon[35254]: pgmap v252: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 9.4 MiB/s wr, 598 op/s 2024-05-27T07:54:24.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:24 smithi090 ceph-mon[31429]: pgmap v252: 129 pgs: 129 active+clean; 2.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 9.4 MiB/s wr, 598 op/s 2024-05-27T07:54:26.331 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:26 smithi090 ceph-mon[31429]: pgmap v253: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 12 MiB/s wr, 764 op/s 2024-05-27T07:54:26.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:26 smithi136 ceph-mon[35324]: pgmap v253: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 12 MiB/s wr, 764 op/s 2024-05-27T07:54:26.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:26 smithi129 ceph-mon[35254]: pgmap v253: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 12 MiB/s wr, 764 op/s 2024-05-27T07:54:26.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.655+0000 7fe05be00700 1 -- 172.21.15.90:0/1113706383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054105490 msgr2=0x7fe0541058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.655+0000 7fe05be00700 1 --2- 172.21.15.90:0/1113706383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054105490 0x7fe0541058a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe048009f90 tx=0x7fe04800b330 comp rx=0 tx=0).stop 2024-05-27T07:54:26.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.656+0000 7fe05be00700 1 -- 172.21.15.90:0/1113706383 shutdown_connections 2024-05-27T07:54:26.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.656+0000 7fe05be00700 1 --2- 172.21.15.90:0/1113706383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe05410e900 0x7fe054110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.656+0000 7fe05be00700 1 --2- 172.21.15.90:0/1113706383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054105490 0x7fe0541058a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.656+0000 7fe05be00700 1 --2- 172.21.15.90:0/1113706383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe054105de0 0x7fe05410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.656+0000 7fe05be00700 1 -- 172.21.15.90:0/1113706383 >> 172.21.15.90:0/1113706383 conn(0x7fe0540fedb0 msgr2=0x7fe0541011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.656+0000 7fe05be00700 1 -- 172.21.15.90:0/1113706383 shutdown_connections 2024-05-27T07:54:26.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.657+0000 7fe05be00700 1 -- 172.21.15.90:0/1113706383 wait complete. 2024-05-27T07:54:26.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.659+0000 7fe05be00700 1 Processor -- start 2024-05-27T07:54:26.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.659+0000 7fe05be00700 1 -- start start 2024-05-27T07:54:26.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.660+0000 7fe05be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe054105490 0x7fe054103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.660+0000 7fe05be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054105de0 0x7fe0541036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.660+0000 7fe05be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe05410e900 0x7fe054103c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.660+0000 7fe05be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe054114350 con 0x7fe054105490 2024-05-27T07:54:26.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.660+0000 7fe05be00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0541141d0 con 0x7fe054105de0 2024-05-27T07:54:26.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.660+0000 7fe05be00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0541144d0 con 0x7fe05410e900 2024-05-27T07:54:26.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.660+0000 7fe058a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054105de0 0x7fe0541036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.660+0000 7fe058a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054105de0 0x7fe0541036c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39944/0 (socket says 172.21.15.90:39944) 2024-05-27T07:54:26.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.661+0000 7fe058a00700 1 -- 172.21.15.90:0/3343486398 learned_addr learned my addr 172.21.15.90:0/3343486398 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:26.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.661+0000 7fe059e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe05410e900 0x7fe054103c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.661+0000 7fe058a00700 1 -- 172.21.15.90:0/3343486398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe05410e900 msgr2=0x7fe054103c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.661+0000 7fe058a00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe05410e900 0x7fe054103c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.661+0000 7fe058a00700 1 -- 172.21.15.90:0/3343486398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe054105490 msgr2=0x7fe054103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.661+0000 7fe058a00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe054105490 0x7fe054103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.661+0000 7fe058a00700 1 -- 172.21.15.90:0/3343486398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0541a9100 con 0x7fe054105de0 2024-05-27T07:54:26.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.662+0000 7fe058a00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054105de0 0x7fe0541036c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe04000e9f0 tx=0x7fe04000edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.663+0000 7fe052000700 1 -- 172.21.15.90:0/3343486398 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04000cbd0 con 0x7fe054105de0 2024-05-27T07:54:26.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.663+0000 7fe052000700 1 -- 172.21.15.90:0/3343486398 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe04000cd70 con 0x7fe054105de0 2024-05-27T07:54:26.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.663+0000 7fe052000700 1 -- 172.21.15.90:0/3343486398 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040018420 con 0x7fe054105de0 2024-05-27T07:54:26.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.665+0000 7fe05be00700 1 -- 172.21.15.90:0/3343486398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0541a93f0 con 0x7fe054105de0 2024-05-27T07:54:26.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.665+0000 7fe05be00700 1 -- 172.21.15.90:0/3343486398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0541a98b0 con 0x7fe054105de0 2024-05-27T07:54:26.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.671+0000 7fe06a919b80 1 Processor -- start 2024-05-27T07:54:26.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.671+0000 7fe06a919b80 1 -- start start 2024-05-27T07:54:26.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.672+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 0x5595c10177f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.672+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1017d30 0x5595c1011870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.672+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1011db0 0x5595c10121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.672+0000 7fe06a919b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5595c0f3fe40 con 0x5595c1011db0 2024-05-27T07:54:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.672+0000 7fe06a919b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5595c0f2f9e0 con 0x5595c1017d30 2024-05-27T07:54:26.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.672+0000 7fe06a919b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5595c0f33a20 con 0x5595c0e92e40 2024-05-27T07:54:26.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.674+0000 7fe059e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 0x5595c10177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.674+0000 7fe059e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 0x5595c10177f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59198/0 (socket says 172.21.15.90:59198) 2024-05-27T07:54:26.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.674+0000 7fe059e00700 1 -- 172.21.15.90:0/2123746494 learned_addr learned my addr 172.21.15.90:0/2123746494 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:26.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.674+0000 7fe052000700 1 -- 172.21.15.90:0/3343486398 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe04001f020 con 0x7fe054105de0 2024-05-27T07:54:26.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.675+0000 7fe058a00700 1 --2- 172.21.15.90:0/2123746494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1011db0 0x5595c10121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.676+0000 7fe058a00700 1 -- 172.21.15.90:0/2123746494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 msgr2=0x5595c10177f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.676+0000 7fe058a00700 1 --2- 172.21.15.90:0/2123746494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 0x5595c10177f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.676+0000 7fe058a00700 1 -- 172.21.15.90:0/2123746494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1017d30 msgr2=0x5595c1011870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.676+0000 7fe058a00700 1 --2- 172.21.15.90:0/2123746494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1017d30 0x5595c1011870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.676+0000 7fe058a00700 1 -- 172.21.15.90:0/2123746494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c0ef8420 con 0x5595c1011db0 2024-05-27T07:54:26.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.676+0000 7fe059e00700 1 --2- 172.21.15.90:0/2123746494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 0x5595c10177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:26.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.677+0000 7fe058a00700 1 --2- 172.21.15.90:0/2123746494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1011db0 0x5595c10121c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe0400dfd10 tx=0x7fe0400e8ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc5a7b0dff2be6a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.677+0000 7fe052000700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe03c0b2a70 0x7fe03c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.677+0000 7fe059e00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe03c0b2a70 0x7fe03c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.677+0000 7fe052000700 1 -- 172.21.15.90:0/3343486398 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe0400e29f0 con 0x7fe054105de0 2024-05-27T07:54:26.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.678+0000 7fe046a00700 1 -- 172.21.15.90:0/2123746494 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400f1070 con 0x5595c1011db0 2024-05-27T07:54:26.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.678+0000 7fe046a00700 1 -- 172.21.15.90:0/2123746494 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0400a93b0 con 0x5595c1011db0 2024-05-27T07:54:26.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.678+0000 7fe046a00700 1 -- 172.21.15.90:0/2123746494 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400a9670 con 0x5595c1011db0 2024-05-27T07:54:26.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.680+0000 7fe06a919b80 1 -- 172.21.15.90:0/2123746494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1011db0 msgr2=0x5595c10121c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.680+0000 7fe06a919b80 1 --2- 172.21.15.90:0/2123746494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1011db0 0x5595c10121c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fe0400dfd10 tx=0x7fe0400e8ed0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.680+0000 7fe059e00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe03c0b2a70 0x7fe03c0b4f80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5595c1012b30 tx=0x7fe04c009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.680+0000 7fe06a919b80 1 -- 172.21.15.90:0/2123746494 shutdown_connections 2024-05-27T07:54:26.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.680+0000 7fe06a919b80 1 --2- 172.21.15.90:0/2123746494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1011db0 0x5595c10121c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.680+0000 7fe06a919b80 1 --2- 172.21.15.90:0/2123746494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 0x5595c10177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.680+0000 7fe06a919b80 1 --2- 172.21.15.90:0/2123746494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1017d30 0x5595c1011870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.680+0000 7fe06a919b80 1 -- 172.21.15.90:0/2123746494 >> 172.21.15.90:0/2123746494 conn(0x5595c0ef2fc0 msgr2=0x5595c100c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.680+0000 7fe06a919b80 1 -- 172.21.15.90:0/2123746494 shutdown_connections 2024-05-27T07:54:26.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.680+0000 7fe06a919b80 1 -- 172.21.15.90:0/2123746494 wait complete. 2024-05-27T07:54:26.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.681+0000 7fe06a919b80 1 -- start start 2024-05-27T07:54:26.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.682+0000 7fe06a919b80 1 Processor -- start 2024-05-27T07:54:26.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.683+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c0e92e40 0x5595c101b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.683+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1011db0 0x5595c101ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.683+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1017d30 0x5595c101bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.683+0000 7fe06a919b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5595c0e35fa0 con 0x5595c1017d30 2024-05-27T07:54:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.683+0000 7fe06a919b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5595c0ed0760 con 0x5595c0e92e40 2024-05-27T07:54:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.683+0000 7fe06a919b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5595c101e800 con 0x5595c1011db0 2024-05-27T07:54:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.683+0000 7fe06a919b80 10 client.0 _ll_register_callbacks cb 0x5595c0ed9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.683+0000 7fe058a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1011db0 0x5595c101ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.683+0000 7fe058a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1011db0 0x5595c101ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59210/0 (socket says 172.21.15.90:59210) 2024-05-27T07:54:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.683+0000 7fe058a00700 1 -- 172.21.15.90:0/3371989453 learned_addr learned my addr 172.21.15.90:0/3371989453 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.684+0000 7fe058a00700 1 -- 172.21.15.90:0/3371989453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c0e92e40 msgr2=0x5595c101b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.684+0000 7fe058a00700 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c0e92e40 0x5595c101b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.684+0000 7fe058a00700 1 -- 172.21.15.90:0/3371989453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1017d30 msgr2=0x5595c101bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.684+0000 7fe058a00700 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1017d30 0x5595c101bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.684+0000 7fe058a00700 1 -- 172.21.15.90:0/3371989453 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c0e7f8d0 con 0x5595c1011db0 2024-05-27T07:54:26.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.688+0000 7fe058a00700 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1011db0 0x5595c101ba30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe04000e9c0 tx=0x7fe0400e2c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1e2a134e4037bfa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.688+0000 7fe03b400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:54:26.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.688+0000 7fe03b400700 1 -- 172.21.15.90:0/3371989453 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400f1070 con 0x5595c1011db0 2024-05-27T07:54:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.688+0000 7fe03b400700 1 -- 172.21.15.90:0/3371989453 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe0400a9b00 con 0x5595c1011db0 2024-05-27T07:54:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.688+0000 7fe06a919b80 1 -- 172.21.15.90:0/3371989453 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0400a93b0 con 0x5595c1011db0 2024-05-27T07:54:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.689+0000 7fe03b400700 1 -- 172.21.15.90:0/3371989453 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe0400a9010 con 0x5595c1011db0 2024-05-27T07:54:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.689+0000 7fe06a919b80 10 client.26605 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.689+0000 7fe06a919b80 1 -- 172.21.15.90:0/3371989453 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5595c0f00060 con 0x5595c1011db0 2024-05-27T07:54:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.690+0000 7fe03b400700 1 -- 172.21.15.90:0/3371989453 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0400a93b0 con 0x5595c1011db0 2024-05-27T07:54:26.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.690+0000 7fe03b400700 1 -- 172.21.15.90:0/3371989453 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7fe0400aabd0 con 0x5595c1011db0 2024-05-27T07:54:26.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.691+0000 7fe06a919b80 10 client.26605 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:26.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.691+0000 7fe06a919b80 10 client.26605 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:54:26.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.691+0000 7fe06a919b80 20 client.26605 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:26.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.691+0000 7fe06a919b80 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5595c102b580 0x5595c102d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.691+0000 7fe06a919b80 4 client.26605 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:26.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.691+0000 7fe06a919b80 1 -- 172.21.15.90:0/3371989453 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595c0f41d90 con 0x5595c102b580 2024-05-27T07:54:26.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.693+0000 7fe059400700 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5595c102b580 0x5595c102d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.695+0000 7fe059400700 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5595c102b580 0x5595c102d9a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c952ac8b803a87c server_cookie=5efff82771fb59f6 in_seq=0 out_seq=0 2024-05-27T07:54:26.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.695+0000 7fe03b400700 10 client.26605 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:54:26.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.700+0000 7fe03b400700 1 -- 172.21.15.90:0/3371989453 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5595c0f41d90 con 0x5595c102b580 2024-05-27T07:54:26.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.700+0000 7fe03b400700 10 client.26605 handle_command_reply: tid=0 2024-05-27T07:54:26.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.700+0000 7fe06a919b80 1 client.26605 shutdown 2024-05-27T07:54:26.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.702+0000 7fe06a919b80 1 -- 172.21.15.90:0/3371989453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1011db0 msgr2=0x5595c101ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.702+0000 7fe06a919b80 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1011db0 0x5595c101ba30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe04000e9c0 tx=0x7fe0400e2c40 comp rx=0 tx=0).stop 2024-05-27T07:54:26.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.702+0000 7fe06a919b80 1 -- 172.21.15.90:0/3371989453 shutdown_connections 2024-05-27T07:54:26.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.702+0000 7fe06a919b80 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5595c102b580 0x5595c102d9a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.702+0000 7fe06a919b80 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1017d30 0x5595c101bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.702+0000 7fe06a919b80 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c0e92e40 0x5595c101b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.702+0000 7fe06a919b80 1 --2- 172.21.15.90:0/3371989453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1011db0 0x5595c101ba30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.702+0000 7fe06a919b80 1 -- 172.21.15.90:0/3371989453 >> 172.21.15.90:0/3371989453 conn(0x5595c0ef2fc0 msgr2=0x5595c100c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.702+0000 7fe03b400700 0 client.26605 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:54:26.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.703+0000 7fe06a919b80 1 -- 172.21.15.90:0/3371989453 shutdown_connections 2024-05-27T07:54:26.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.703+0000 7fe06a919b80 1 -- 172.21.15.90:0/3371989453 wait complete. 2024-05-27T07:54:26.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.703+0000 7fe06a919b80 20 client.26605 trim_cache size 0 max 16384 2024-05-27T07:54:26.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.727+0000 7fe06a919b80 1 Processor -- start 2024-05-27T07:54:26.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.728+0000 7fe06a919b80 1 -- start start 2024-05-27T07:54:26.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.728+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c0e92e40 0x5595c1011870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.728+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c101d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.728+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1017d30 0x5595c1027a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.728+0000 7fe06a919b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0541b4420 con 0x5595c0e92e40 2024-05-27T07:54:26.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.728+0000 7fe06a919b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0541b2210 con 0x5595c1011db0 2024-05-27T07:54:26.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.728+0000 7fe06a919b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0541b3d50 con 0x5595c1017d30 2024-05-27T07:54:26.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.729+0000 7fe058a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c101d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.729+0000 7fe058a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c101d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39960/0 (socket says 172.21.15.90:39960) 2024-05-27T07:54:26.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.729+0000 7fe058a00700 1 -- 172.21.15.90:0/1282782209 learned_addr learned my addr 172.21.15.90:0/1282782209 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:26.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.729+0000 7fe059e00700 1 --2- 172.21.15.90:0/1282782209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1017d30 0x5595c1027a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.729+0000 7fe059400700 1 --2- 172.21.15.90:0/1282782209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c0e92e40 0x5595c1011870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.730+0000 7fe058a00700 1 -- 172.21.15.90:0/1282782209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1017d30 msgr2=0x5595c1027a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.730+0000 7fe058a00700 1 --2- 172.21.15.90:0/1282782209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1017d30 0x5595c1027a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.730+0000 7fe058a00700 1 -- 172.21.15.90:0/1282782209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c0e92e40 msgr2=0x5595c1011870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.730+0000 7fe058a00700 1 --2- 172.21.15.90:0/1282782209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c0e92e40 0x5595c1011870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.730+0000 7fe058a00700 1 -- 172.21.15.90:0/1282782209 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c1019a60 con 0x5595c1011db0 2024-05-27T07:54:26.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.730+0000 7fe058a00700 1 --2- 172.21.15.90:0/1282782209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c101d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe04000c790 tx=0x7fe0400adcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d468935ee9ba179 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.730+0000 7fe046000700 1 -- 172.21.15.90:0/1282782209 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400f1070 con 0x5595c1011db0 2024-05-27T07:54:26.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.730+0000 7fe046000700 1 -- 172.21.15.90:0/1282782209 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0400e8300 con 0x5595c1011db0 2024-05-27T07:54:26.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.730+0000 7fe046000700 1 -- 172.21.15.90:0/1282782209 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400a91f0 con 0x5595c1011db0 2024-05-27T07:54:26.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.731+0000 7fe06a919b80 1 -- 172.21.15.90:0/1282782209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 msgr2=0x5595c101d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.731+0000 7fe06a919b80 1 --2- 172.21.15.90:0/1282782209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c101d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe04000c790 tx=0x7fe0400adcf0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.732+0000 7fe06a919b80 1 -- 172.21.15.90:0/1282782209 shutdown_connections 2024-05-27T07:54:26.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.732+0000 7fe06a919b80 1 --2- 172.21.15.90:0/1282782209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c1017d30 0x5595c1027a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.732+0000 7fe06a919b80 1 --2- 172.21.15.90:0/1282782209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c0e92e40 0x5595c1011870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.732+0000 7fe06a919b80 1 --2- 172.21.15.90:0/1282782209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c101d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.732+0000 7fe06a919b80 1 -- 172.21.15.90:0/1282782209 >> 172.21.15.90:0/1282782209 conn(0x5595c0ef2fc0 msgr2=0x5595c1015140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.733+0000 7fe06a919b80 1 -- 172.21.15.90:0/1282782209 shutdown_connections 2024-05-27T07:54:26.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.733+0000 7fe06a919b80 1 -- 172.21.15.90:0/1282782209 wait complete. 2024-05-27T07:54:26.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.734+0000 7fe06a919b80 1 -- start start 2024-05-27T07:54:26.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.734+0000 7fe06a919b80 1 Processor -- start 2024-05-27T07:54:26.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.735+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 0x5595c1030680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.736+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c1030bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.736+0000 7fe06a919b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1017d30 0x5595c102dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.736+0000 7fe058a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c1030bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.736+0000 7fe058a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c1030bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39970/0 (socket says 172.21.15.90:39970) 2024-05-27T07:54:26.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.736+0000 7fe06a919b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0541b2fb0 con 0x5595c1017d30 2024-05-27T07:54:26.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.736+0000 7fe06a919b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0541b28e0 con 0x5595c1011db0 2024-05-27T07:54:26.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.736+0000 7fe058a00700 1 -- 172.21.15.90:0/1774446007 learned_addr learned my addr 172.21.15.90:0/1774446007 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:26.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.736+0000 7fe059400700 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 0x5595c1030680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.736+0000 7fe06a919b80 1 -- 172.21.15.90:0/1774446007 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0541b3680 con 0x5595c0e92e40 2024-05-27T07:54:26.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.737+0000 7fe06a919b80 10 client.0 _ll_register_callbacks cb 0x5595c0ed9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:26.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.737+0000 7fe059e00700 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1017d30 0x5595c102dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.737+0000 7fe058a00700 1 -- 172.21.15.90:0/1774446007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 msgr2=0x5595c1030680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.737+0000 7fe058a00700 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 0x5595c1030680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.737+0000 7fe058a00700 1 -- 172.21.15.90:0/1774446007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1017d30 msgr2=0x5595c102dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:26.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.737+0000 7fe058a00700 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1017d30 0x5595c102dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.738+0000 7fe058a00700 1 -- 172.21.15.90:0/1774446007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c102e230 con 0x5595c1011db0 2024-05-27T07:54:26.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.738+0000 7fe058a00700 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c1030bc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0400b0fd0 tx=0x7fe0400aae00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5cc7f8b927acad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:26.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.738+0000 7fe03b400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:54:26.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.738+0000 7fe03b400700 1 -- 172.21.15.90:0/1774446007 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400f1070 con 0x5595c1011db0 2024-05-27T07:54:26.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.738+0000 7fe03b400700 1 -- 172.21.15.90:0/1774446007 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe0400a5e90 con 0x5595c1011db0 2024-05-27T07:54:26.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.738+0000 7fe06a919b80 1 -- 172.21.15.90:0/1774446007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0400e8300 con 0x5595c1011db0 2024-05-27T07:54:26.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.739+0000 7fe03b400700 1 -- 172.21.15.90:0/1774446007 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe0400c2050 con 0x5595c1011db0 2024-05-27T07:54:26.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.739+0000 7fe06a919b80 10 client.36578 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:26.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.739+0000 7fe06a919b80 1 -- 172.21.15.90:0/1774446007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5595c102e230 con 0x5595c1011db0 2024-05-27T07:54:26.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.739+0000 7fe03b400700 1 -- 172.21.15.90:0/1774446007 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0400e8300 con 0x5595c1011db0 2024-05-27T07:54:26.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.740+0000 7fe03b400700 1 -- 172.21.15.90:0/1774446007 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7fe0400b2270 con 0x5595c1011db0 2024-05-27T07:54:26.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.740+0000 7fe06a919b80 10 client.36578 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:26.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.740+0000 7fe06a919b80 10 client.36578 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:54:26.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.740+0000 7fe06a919b80 20 client.36578 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:26.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.740+0000 7fe06a919b80 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5595c1095aa0 0x5595c1097ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:26.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.740+0000 7fe06a919b80 4 client.36578 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:54:26.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.740+0000 7fe06a919b80 1 -- 172.21.15.90:0/1774446007 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595c0f3b570 con 0x5595c1095aa0 2024-05-27T07:54:26.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.741+0000 7fe059400700 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5595c1095aa0 0x5595c1097ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:26.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.742+0000 7fe03b400700 10 client.36578 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:54:26.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.742+0000 7fe059400700 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5595c1095aa0 0x5595c1097ec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=caeb0055691d39d9 server_cookie=3f2809893e74e254 in_seq=0 out_seq=0 2024-05-27T07:54:26.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.742+0000 7fe03b400700 1 -- 172.21.15.90:0/1774446007 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595c0f3b570 con 0x5595c1095aa0 2024-05-27T07:54:26.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.742+0000 7fe03b400700 10 client.36578 handle_command_reply: tid=0 2024-05-27T07:54:26.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.742+0000 7fe06a919b80 1 client.36578 shutdown 2024-05-27T07:54:26.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.743+0000 7fe06a919b80 1 -- 172.21.15.90:0/1774446007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 msgr2=0x5595c1030bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.743+0000 7fe06a919b80 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c1030bc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe0400b0fd0 tx=0x7fe0400aae00 comp rx=0 tx=0).stop 2024-05-27T07:54:26.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.743+0000 7fe06a919b80 1 -- 172.21.15.90:0/1774446007 shutdown_connections 2024-05-27T07:54:26.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.743+0000 7fe06a919b80 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x5595c1095aa0 0x5595c1097ec0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.743+0000 7fe06a919b80 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595c1017d30 0x5595c102dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.743+0000 7fe06a919b80 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c0e92e40 0x5595c1030680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.743+0000 7fe06a919b80 1 --2- 172.21.15.90:0/1774446007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595c1011db0 0x5595c1030bc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.743+0000 7fe06a919b80 1 -- 172.21.15.90:0/1774446007 >> 172.21.15.90:0/1774446007 conn(0x5595c0ef2fc0 msgr2=0x5595c101b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.743+0000 7fe03b400700 0 client.36578 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:54:26.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.743+0000 7fe06a919b80 1 -- 172.21.15.90:0/1774446007 shutdown_connections 2024-05-27T07:54:26.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.744+0000 7fe06a919b80 1 -- 172.21.15.90:0/1774446007 wait complete. 2024-05-27T07:54:26.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.744+0000 7fe06a919b80 20 client.36578 trim_cache size 0 max 16384 2024-05-27T07:54:26.746 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:54:26.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 -- 172.21.15.90:0/3343486398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe03c0b2a70 msgr2=0x7fe03c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe03c0b2a70 0x7fe03c0b4f80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5595c1012b30 tx=0x7fe04c009260 comp rx=0 tx=0).stop 2024-05-27T07:54:26.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 -- 172.21.15.90:0/3343486398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054105de0 msgr2=0x7fe0541036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:26.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054105de0 0x7fe0541036c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe04000e9f0 tx=0x7fe04000edb0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 -- 172.21.15.90:0/3343486398 shutdown_connections 2024-05-27T07:54:26.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe05410e900 0x7fe054103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe054105490 0x7fe054103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe03c0b2a70 0x7fe03c0b4f80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 --2- 172.21.15.90:0/3343486398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054105de0 0x7fe0541036c0 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe04000e9f0 tx=0x7fe04000edb0 comp rx=0 tx=0).stop 2024-05-27T07:54:26.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 -- 172.21.15.90:0/3343486398 >> 172.21.15.90:0/3343486398 conn(0x7fe0540fedb0 msgr2=0x7fe0540ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:26.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.745+0000 7fe05be00700 1 -- 172.21.15.90:0/3343486398 shutdown_connections 2024-05-27T07:54:26.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:26.746+0000 7fe05be00700 1 -- 172.21.15.90:0/3343486398 wait complete. 2024-05-27T07:54:27.640 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:54:27.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7efdf4800700 time 2024-05-27T07:54:27.626093+0000 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7efe0a380ba6] 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7efe0a380dd1] 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5561d902823d] 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5561d9013971] 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 5: (SnapRealm::check_cache() const+0x13e) [0x5561d9011fa0] 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5561d9012651] 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5561d8fa7347] 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5561d8ca387e] 2024-05-27T07:54:27.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5561d8ca8e94] 2024-05-27T07:54:27.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5561d8cb3969] 2024-05-27T07:54:27.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 11: (MDLog::_replay_thread()+0x1f38) [0x5561d904c48c] 2024-05-27T07:54:27.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5561d90559cb] 2024-05-27T07:54:27.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 13: (Thread::entry_wrapper()+0x43) [0x7efe0a34f72d] 2024-05-27T07:54:27.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 14: (Thread::_entry_func(void*)+0xd) [0x7efe0a34f749] 2024-05-27T07:54:27.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7efe08ef31ca] 2024-05-27T07:54:27.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 16: clone() 2024-05-27T07:54:27.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: *** Caught signal (Aborted) ** 2024-05-27T07:54:27.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: in thread 7efdf4800700 thread_name:md_log_replay 2024-05-27T07:54:27.730 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.628+0000 7efdf4800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7efdf4800700 time 2024-05-27T07:54:27.626093+0000 2024-05-27T07:54:27.730 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:54:27.730 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.730 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:27.730 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7efe0a380ba6] 2024-05-27T07:54:27.730 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7efe0a380dd1] 2024-05-27T07:54:27.730 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5561d902823d] 2024-05-27T07:54:27.730 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5561d9013971] 2024-05-27T07:54:27.731 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 5: (SnapRealm::check_cache() const+0x13e) [0x5561d9011fa0] 2024-05-27T07:54:27.731 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5561d9012651] 2024-05-27T07:54:27.731 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5561d8fa7347] 2024-05-27T07:54:27.731 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5561d8ca387e] 2024-05-27T07:54:27.731 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5561d8ca8e94] 2024-05-27T07:54:27.731 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5561d8cb3969] 2024-05-27T07:54:27.731 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 11: (MDLog::_replay_thread()+0x1f38) [0x5561d904c48c] 2024-05-27T07:54:27.731 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5561d90559cb] 2024-05-27T07:54:27.731 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 13: (Thread::entry_wrapper()+0x43) [0x7efe0a34f72d] 2024-05-27T07:54:27.731 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 14: (Thread::_entry_func(void*)+0xd) [0x7efe0a34f749] 2024-05-27T07:54:27.732 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7efe08ef31ca] 2024-05-27T07:54:27.732 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 16: clone() 2024-05-27T07:54:27.732 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.732 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:27.732 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7efe08efdd20] 2024-05-27T07:54:27.732 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2: gsignal() 2024-05-27T07:54:27.732 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 3: abort() 2024-05-27T07:54:27.732 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7efe0a380ced] 2024-05-27T07:54:27.733 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7efe0a380dd1] 2024-05-27T07:54:27.733 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5561d902823d] 2024-05-27T07:54:27.733 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5561d9013971] 2024-05-27T07:54:27.733 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 8: (SnapRealm::check_cache() const+0x13e) [0x5561d9011fa0] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5561d9012651] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5561d8fa7347] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5561d8ca387e] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5561d8ca8e94] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5561d8cb3969] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 14: (MDLog::_replay_thread()+0x1f38) [0x5561d904c48c] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5561d90559cb] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 16: (Thread::entry_wrapper()+0x43) [0x7efe0a34f72d] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 17: (Thread::_entry_func(void*)+0xd) [0x7efe0a34f749] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7efe08ef31ca] 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 19: clone() 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.631+0000 7efdf4800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:54:27.734 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: in thread 7efdf4800700 thread_name:md_log_replay 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7efe08efdd20] 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2: gsignal() 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 3: abort() 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7efe0a380ced] 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7efe0a380dd1] 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5561d902823d] 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5561d9013971] 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 8: (SnapRealm::check_cache() const+0x13e) [0x5561d9011fa0] 2024-05-27T07:54:27.735 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5561d9012651] 2024-05-27T07:54:27.736 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5561d8fa7347] 2024-05-27T07:54:27.736 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5561d8ca387e] 2024-05-27T07:54:27.736 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5561d8ca8e94] 2024-05-27T07:54:27.736 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5561d8cb3969] 2024-05-27T07:54:27.736 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 14: (MDLog::_replay_thread()+0x1f38) [0x5561d904c48c] 2024-05-27T07:54:27.736 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5561d90559cb] 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 16: (Thread::entry_wrapper()+0x43) [0x7efe0a34f72d] 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 17: (Thread::_entry_func(void*)+0xd) [0x7efe0a34f749] 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7efe08ef31ca] 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 19: clone() 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: -1> 2024-05-27T07:54:27.628+0000 7efdf4800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7efdf4800700 time 2024-05-27T07:54:27.626093+0000 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7efe0a380ba6] 2024-05-27T07:54:27.737 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7efe0a380dd1] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5561d902823d] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5561d9013971] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 5: (SnapRealm::check_cache() const+0x13e) [0x5561d9011fa0] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5561d9012651] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5561d8fa7347] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5561d8ca387e] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5561d8ca8e94] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5561d8cb3969] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 11: (MDLog::_replay_thread()+0x1f38) [0x5561d904c48c] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5561d90559cb] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 13: (Thread::entry_wrapper()+0x43) [0x7efe0a34f72d] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 14: (Thread::_entry_func(void*)+0xd) [0x7efe0a34f749] 2024-05-27T07:54:27.738 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7efe08ef31ca] 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 16: clone() 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 0> 2024-05-27T07:54:27.631+0000 7efdf4800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: in thread 7efdf4800700 thread_name:md_log_replay 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7efe08efdd20] 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2: gsignal() 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 3: abort() 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7efe0a380ced] 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7efe0a380dd1] 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5561d902823d] 2024-05-27T07:54:27.739 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5561d9013971] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 8: (SnapRealm::check_cache() const+0x13e) [0x5561d9011fa0] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5561d9012651] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5561d8fa7347] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5561d8ca387e] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5561d8ca8e94] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5561d8cb3969] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 14: (MDLog::_replay_thread()+0x1f38) [0x5561d904c48c] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5561d90559cb] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 16: (Thread::entry_wrapper()+0x43) [0x7efe0a34f72d] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 17: (Thread::_entry_func(void*)+0xd) [0x7efe0a34f749] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7efe08ef31ca] 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 19: clone() 2024-05-27T07:54:27.740 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: -1> 2024-05-27T07:54:27.628+0000 7efdf4800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7efdf4800700 time 2024-05-27T07:54:27.626093+0000 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7efe0a380ba6] 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7efe0a380dd1] 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5561d902823d] 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5561d9013971] 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 5: (SnapRealm::check_cache() const+0x13e) [0x5561d9011fa0] 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5561d9012651] 2024-05-27T07:54:27.741 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5561d8fa7347] 2024-05-27T07:54:27.742 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5561d8ca387e] 2024-05-27T07:54:27.742 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5561d8ca8e94] 2024-05-27T07:54:27.742 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5561d8cb3969] 2024-05-27T07:54:27.742 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 11: (MDLog::_replay_thread()+0x1f38) [0x5561d904c48c] 2024-05-27T07:54:27.742 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5561d90559cb] 2024-05-27T07:54:27.742 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 13: (Thread::entry_wrapper()+0x43) [0x7efe0a34f72d] 2024-05-27T07:54:27.742 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 14: (Thread::_entry_func(void*)+0xd) [0x7efe0a34f749] 2024-05-27T07:54:27.742 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7efe08ef31ca] 2024-05-27T07:54:27.742 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 16: clone() 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 0> 2024-05-27T07:54:27.631+0000 7efdf4800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: in thread 7efdf4800700 thread_name:md_log_replay 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7efe08efdd20] 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2: gsignal() 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 3: abort() 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7efe0a380ced] 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7efe0a380dd1] 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5561d902823d] 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5561d9013971] 2024-05-27T07:54:27.743 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 8: (SnapRealm::check_cache() const+0x13e) [0x5561d9011fa0] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5561d9012651] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5561d8fa7347] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5561d8ca387e] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x2702) [0x5561d8ca8e94] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5561d8cb3969] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 14: (MDLog::_replay_thread()+0x1f38) [0x5561d904c48c] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5561d90559cb] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 16: (Thread::entry_wrapper()+0x43) [0x7efe0a34f72d] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 17: (Thread::_entry_func(void*)+0xd) [0x7efe0a34f749] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7efe08ef31ca] 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 19: clone() 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:54:27.744 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[108426]: 2024-05-27T07:54:28.047 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:27 smithi090 podman[111294]: 2024-05-27 07:54:27.816744062 +0000 UTC m=+0.052472644 container died ac2afcf63a575480ab8be281e86b1bacfd8acaf9aa683a95ce211a444c554629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T07:54:28.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:28 smithi090 ceph-mon[31429]: pgmap v254: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 8.9 MiB/s wr, 601 op/s 2024-05-27T07:54:28.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:28 smithi136 ceph-mon[35324]: pgmap v254: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 8.9 MiB/s wr, 601 op/s 2024-05-27T07:54:28.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:28 smithi129 ceph-mon[35254]: pgmap v254: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 8.9 MiB/s wr, 601 op/s 2024-05-27T07:54:28.716 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:28.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 250.0 s, 668.0 tps, lat 146.948 ms stddev 82.288 2024-05-27T07:54:28.925 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:28 smithi090 podman[111294]: 2024-05-27 07:54:28.560366989 +0000 UTC m=+0.796095558 container remove ac2afcf63a575480ab8be281e86b1bacfd8acaf9aa683a95ce211a444c554629 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-27T07:54:28.925 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:28 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:54:30.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:30 smithi090 ceph-mon[31429]: pgmap v255: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 12 MiB/s wr, 838 op/s 2024-05-27T07:54:30.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:30 smithi136 ceph-mon[35324]: pgmap v255: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 12 MiB/s wr, 838 op/s 2024-05-27T07:54:30.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:30 smithi129 ceph-mon[35254]: pgmap v255: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 12 MiB/s wr, 838 op/s 2024-05-27T07:54:31.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:31 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-05-27T07:54:31.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:31 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.880s CPU time. 2024-05-27T07:54:32.416 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:32 smithi090 ceph-mon[31429]: pgmap v256: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 7.6 MiB/s wr, 560 op/s 2024-05-27T07:54:32.499 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:32 smithi136 ceph-mon[35324]: pgmap v256: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 7.6 MiB/s wr, 560 op/s 2024-05-27T07:54:32.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:32 smithi129 ceph-mon[35254]: pgmap v256: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 7.6 MiB/s wr, 560 op/s 2024-05-27T07:54:33.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.086+0000 7f115b600700 1 -- 172.21.15.90:0/3025487734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1154078df0 msgr2=0x7f1154079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:33.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.086+0000 7f115b600700 1 --2- 172.21.15.90:0/3025487734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1154078df0 0x7f1154079200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f114c00b0a0 tx=0x7f114c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.086+0000 7f115b600700 1 -- 172.21.15.90:0/3025487734 shutdown_connections 2024-05-27T07:54:33.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.086+0000 7f115b600700 1 --2- 172.21.15.90:0/3025487734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1154115210 0x7f1154117630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.086+0000 7f115b600700 1 --2- 172.21.15.90:0/3025487734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1154078df0 0x7f1154079200 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.086+0000 7f115b600700 1 --2- 172.21.15.90:0/3025487734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1154079740 0x7f1154114cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.086+0000 7f115b600700 1 -- 172.21.15.90:0/3025487734 >> 172.21.15.90:0/3025487734 conn(0x7f1154102070 msgr2=0x7f11541044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:33.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.086+0000 7f115b600700 1 -- 172.21.15.90:0/3025487734 shutdown_connections 2024-05-27T07:54:33.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.086+0000 7f115b600700 1 -- 172.21.15.90:0/3025487734 wait complete. 2024-05-27T07:54:33.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.088+0000 7f115b600700 1 Processor -- start 2024-05-27T07:54:33.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.088+0000 7f115b600700 1 -- start start 2024-05-27T07:54:33.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.089+0000 7f115b600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1154078df0 0x7f1154075410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.089+0000 7f115b600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1154079740 0x7f115406e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.090+0000 7f115b600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1154115210 0x7f115406ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.090+0000 7f115b600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f115411a9a0 con 0x7f1154079740 2024-05-27T07:54:33.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.090+0000 7f115b600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f115411a820 con 0x7f1154115210 2024-05-27T07:54:33.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.090+0000 7f115b600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f115411ab20 con 0x7f1154078df0 2024-05-27T07:54:33.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.090+0000 7f1159600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1154115210 0x7f115406ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.090+0000 7f1159600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1154115210 0x7f115406ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33822/0 (socket says 172.21.15.90:33822) 2024-05-27T07:54:33.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.090+0000 7f1159600700 1 -- 172.21.15.90:0/1577396406 learned_addr learned my addr 172.21.15.90:0/1577396406 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:33.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.091+0000 7f1153e00700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1154079740 0x7f115406e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.091+0000 7f1158c00700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1154078df0 0x7f1154075410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.091+0000 7f1159600700 1 -- 172.21.15.90:0/1577396406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1154078df0 msgr2=0x7f1154075410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:33.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.091+0000 7f1159600700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1154078df0 0x7f1154075410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.091+0000 7f1159600700 1 -- 172.21.15.90:0/1577396406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1154079740 msgr2=0x7f115406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:33.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.091+0000 7f1159600700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1154079740 0x7f115406e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.091+0000 7f1159600700 1 -- 172.21.15.90:0/1577396406 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1154072f40 con 0x7f1154115210 2024-05-27T07:54:33.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.092+0000 7f1159600700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1154115210 0x7f115406ebe0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f114400e940 tx=0x7f114400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:33.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.092+0000 7f1151600700 1 -- 172.21.15.90:0/1577396406 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114400cbd0 con 0x7f1154115210 2024-05-27T07:54:33.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.092+0000 7f115b600700 1 -- 172.21.15.90:0/1577396406 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1154073230 con 0x7f1154115210 2024-05-27T07:54:33.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.093+0000 7f115b600700 1 -- 172.21.15.90:0/1577396406 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1154073770 con 0x7f1154115210 2024-05-27T07:54:33.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.093+0000 7f1151600700 1 -- 172.21.15.90:0/1577396406 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f114400cd70 con 0x7f1154115210 2024-05-27T07:54:33.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.093+0000 7f1151600700 1 -- 172.21.15.90:0/1577396406 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144005570 con 0x7f1154115210 2024-05-27T07:54:33.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.100+0000 7f116a15eb80 1 Processor -- start 2024-05-27T07:54:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.102+0000 7f1151600700 1 -- 172.21.15.90:0/1577396406 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1144004750 con 0x7f1154115210 2024-05-27T07:54:33.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.103+0000 7f116a15eb80 1 -- start start 2024-05-27T07:54:33.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.104+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402223330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.106+0000 7f1151600700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11380b29f0 0x7f11380b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.106+0000 7f1153e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402223330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.106+0000 7f1153e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402223330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33834/0 (socket says 172.21.15.90:33834) 2024-05-27T07:54:33.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.106+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 0x556402223870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.106+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x556402230110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.106+0000 7f116a15eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556402151e40 con 0x556402223e70 2024-05-27T07:54:33.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.106+0000 7f116a15eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5564021419e0 con 0x5564021909f0 2024-05-27T07:54:33.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.106+0000 7f116a15eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556402145a20 con 0x5564020a4e40 2024-05-27T07:54:33.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.106+0000 7f1153e00700 1 -- 172.21.15.90:0/1818542520 learned_addr learned my addr 172.21.15.90:0/1818542520 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:33.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.107+0000 7f1153e00700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11380b29f0 0x7f11380b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.107+0000 7f1158c00700 1 --2- 172.21.15.90:0/1818542520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 0x556402223870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.108+0000 7f1151600700 1 -- 172.21.15.90:0/1577396406 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f11440e2b40 con 0x7f1154115210 2024-05-27T07:54:33.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.108+0000 7f1158c00700 1 -- 172.21.15.90:0/1818542520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 msgr2=0x556402223330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:33.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.108+0000 7f1158c00700 1 --2- 172.21.15.90:0/1818542520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402223330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.108+0000 7f1158c00700 1 -- 172.21.15.90:0/1818542520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 msgr2=0x556402230110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.109+0000 7f1158c00700 1 --2- 172.21.15.90:0/1818542520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x556402230110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.109+0000 7f1158c00700 1 -- 172.21.15.90:0/1818542520 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55640210a420 con 0x5564020a4e40 2024-05-27T07:54:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.109+0000 7f1153e00700 1 --2- 172.21.15.90:0/1818542520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402223330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.109+0000 7f1158c00700 1 --2- 172.21.15.90:0/1818542520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 0x556402223870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f115406fd90 tx=0x7f114c009d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a095faedaac4c310 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.109+0000 7f1153e00700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11380b29f0 0x7f11380b4f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f114000bfd0 tx=0x7f114000b820 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.110+0000 7f114a000700 1 -- 172.21.15.90:0/1818542520 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c00bab0 con 0x5564020a4e40 2024-05-27T07:54:33.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.110+0000 7f114a000700 1 -- 172.21.15.90:0/1818542520 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f114c00bc50 con 0x5564020a4e40 2024-05-27T07:54:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.111+0000 7f116a15eb80 1 -- 172.21.15.90:0/1818542520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 msgr2=0x556402223870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.111+0000 7f116a15eb80 1 --2- 172.21.15.90:0/1818542520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 0x556402223870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f115406fd90 tx=0x7f114c009d80 comp rx=0 tx=0).stop 2024-05-27T07:54:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.112+0000 7f116a15eb80 1 -- 172.21.15.90:0/1818542520 shutdown_connections 2024-05-27T07:54:33.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.112+0000 7f116a15eb80 1 --2- 172.21.15.90:0/1818542520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x556402230110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.112+0000 7f116a15eb80 1 --2- 172.21.15.90:0/1818542520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402223330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.112+0000 7f116a15eb80 1 --2- 172.21.15.90:0/1818542520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 0x556402223870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.112+0000 7f116a15eb80 1 -- 172.21.15.90:0/1818542520 >> 172.21.15.90:0/1818542520 conn(0x55640213e390 msgr2=0x55640221f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:33.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.113+0000 7f116a15eb80 1 -- 172.21.15.90:0/1818542520 shutdown_connections 2024-05-27T07:54:33.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.113+0000 7f116a15eb80 1 -- 172.21.15.90:0/1818542520 wait complete. 2024-05-27T07:54:33.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.114+0000 7f116a15eb80 1 -- start start 2024-05-27T07:54:33.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.114+0000 7f116a15eb80 1 Processor -- start 2024-05-27T07:54:33.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.115+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 0x55640222d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.116+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564021909f0 0x55640222da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.116+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x55640222dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.116+0000 7f116a15eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556402047fa0 con 0x556402223e70 2024-05-27T07:54:33.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.116+0000 7f116a15eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5564020e2760 con 0x5564020a4e40 2024-05-27T07:54:33.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.116+0000 7f116a15eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556402230bc0 con 0x5564021909f0 2024-05-27T07:54:33.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.116+0000 7f116a15eb80 10 client.0 _ll_register_callbacks cb 0x5564020ebc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:33.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.118+0000 7f1153e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564021909f0 0x55640222da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.118+0000 7f1153e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564021909f0 0x55640222da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36276/0 (socket says 172.21.15.90:36276) 2024-05-27T07:54:33.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.118+0000 7f1153e00700 1 -- 172.21.15.90:0/3088745313 learned_addr learned my addr 172.21.15.90:0/3088745313 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:33.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.118+0000 7f1159600700 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x55640222dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.118+0000 7f1158c00700 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 0x55640222d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.119+0000 7f1153e00700 1 -- 172.21.15.90:0/3088745313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 msgr2=0x55640222d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:33.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.119+0000 7f1153e00700 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 0x55640222d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.119+0000 7f1153e00700 1 -- 172.21.15.90:0/3088745313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 msgr2=0x55640222dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:33.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.119+0000 7f1153e00700 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x55640222dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.119+0000 7f1153e00700 1 -- 172.21.15.90:0/3088745313 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564020918d0 con 0x5564021909f0 2024-05-27T07:54:33.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.120+0000 7f1153e00700 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564021909f0 0x55640222da70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1140014fd0 tx=0x7f114000b520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17527052ed4f4a91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:33.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.120+0000 7f113ea00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:54:33.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.120+0000 7f113ea00700 1 -- 172.21.15.90:0/3088745313 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140025030 con 0x5564021909f0 2024-05-27T07:54:33.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.120+0000 7f113ea00700 1 -- 172.21.15.90:0/3088745313 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f11400147f0 con 0x5564021909f0 2024-05-27T07:54:33.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.120+0000 7f116a15eb80 1 -- 172.21.15.90:0/3088745313 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114c00bc50 con 0x5564021909f0 2024-05-27T07:54:33.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.121+0000 7f113ea00700 1 -- 172.21.15.90:0/3088745313 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1140032050 con 0x5564021909f0 2024-05-27T07:54:33.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.121+0000 7f116a15eb80 10 client.26623 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:33.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.121+0000 7f116a15eb80 1 -- 172.21.15.90:0/3088745313 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556402112060 con 0x5564021909f0 2024-05-27T07:54:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.122+0000 7f113ea00700 1 -- 172.21.15.90:0/3088745313 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f114c00bc50 con 0x5564021909f0 2024-05-27T07:54:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.122+0000 7f113ea00700 1 -- 172.21.15.90:0/3088745313 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7f114001d070 con 0x5564021909f0 2024-05-27T07:54:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.123+0000 7f116a15eb80 10 client.26623 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.123+0000 7f116a15eb80 10 client.26623 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:54:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.123+0000 7f116a15eb80 20 client.26623 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.123+0000 7f116a15eb80 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55640223da70 0x55640223fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.123+0000 7f116a15eb80 4 client.26623 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.123+0000 7f116a15eb80 1 -- 172.21.15.90:0/3088745313 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556402153d90 con 0x55640223da70 2024-05-27T07:54:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.124+0000 7f1158c00700 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55640223da70 0x55640223fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.125+0000 7f1158c00700 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55640223da70 0x55640223fe90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bb01c121f45f2e7 server_cookie=c16e2cbf4ac2031c in_seq=0 out_seq=0 2024-05-27T07:54:33.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.125+0000 7f113ea00700 10 client.26623 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:54:33.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.127+0000 7f113ea00700 1 -- 172.21.15.90:0/3088745313 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556402153d90 con 0x55640223da70 2024-05-27T07:54:33.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.127+0000 7f113ea00700 10 client.26623 handle_command_reply: tid=0 2024-05-27T07:54:33.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.127+0000 7f116a15eb80 1 client.26623 shutdown 2024-05-27T07:54:33.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.129+0000 7f116a15eb80 1 -- 172.21.15.90:0/3088745313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564021909f0 msgr2=0x55640222da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:33.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.129+0000 7f116a15eb80 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564021909f0 0x55640222da70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1140014fd0 tx=0x7f114000b520 comp rx=0 tx=0).stop 2024-05-27T07:54:33.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.130+0000 7f116a15eb80 1 -- 172.21.15.90:0/3088745313 shutdown_connections 2024-05-27T07:54:33.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.130+0000 7f116a15eb80 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55640223da70 0x55640223fe90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.130+0000 7f116a15eb80 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x55640222dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.130+0000 7f116a15eb80 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 0x55640222d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.130+0000 7f116a15eb80 1 --2- 172.21.15.90:0/3088745313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564021909f0 0x55640222da70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.130+0000 7f116a15eb80 1 -- 172.21.15.90:0/3088745313 >> 172.21.15.90:0/3088745313 conn(0x55640213e390 msgr2=0x556402223310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:33.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.130+0000 7f113ea00700 0 client.26623 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:54:33.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.131+0000 7f116a15eb80 1 -- 172.21.15.90:0/3088745313 shutdown_connections 2024-05-27T07:54:33.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.131+0000 7f116a15eb80 1 -- 172.21.15.90:0/3088745313 wait complete. 2024-05-27T07:54:33.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.131+0000 7f116a15eb80 20 client.26623 trim_cache size 0 max 16384 2024-05-27T07:54:33.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.167+0000 7f116a15eb80 1 Processor -- start 2024-05-27T07:54:33.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.167+0000 7f116a15eb80 1 -- start start 2024-05-27T07:54:33.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.168+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 0x5564022237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.168+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402239d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.168+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x556402245690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.168+0000 7f116a15eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11340109f0 con 0x556402223e70 2024-05-27T07:54:33.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.168+0000 7f116a15eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f113400e7e0 con 0x5564021909f0 2024-05-27T07:54:33.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.168+0000 7f116a15eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1134010320 con 0x5564020a4e40 2024-05-27T07:54:33.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.168+0000 7f1153e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402239d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.168+0000 7f1153e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402239d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33856/0 (socket says 172.21.15.90:33856) 2024-05-27T07:54:33.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.168+0000 7f1153e00700 1 -- 172.21.15.90:0/158235017 learned_addr learned my addr 172.21.15.90:0/158235017 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:33.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.169+0000 7f1159600700 1 --2- 172.21.15.90:0/158235017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x556402245690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.169+0000 7f1158c00700 1 --2- 172.21.15.90:0/158235017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 0x5564022237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.169+0000 7f1153e00700 1 -- 172.21.15.90:0/158235017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 msgr2=0x5564022237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:33.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.169+0000 7f1153e00700 1 --2- 172.21.15.90:0/158235017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 0x5564022237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.169+0000 7f1153e00700 1 -- 172.21.15.90:0/158235017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 msgr2=0x556402245690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:33.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.169+0000 7f1153e00700 1 --2- 172.21.15.90:0/158235017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x556402245690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.169+0000 7f1153e00700 1 -- 172.21.15.90:0/158235017 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55640221b240 con 0x5564021909f0 2024-05-27T07:54:33.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.169+0000 7f1159600700 1 --2- 172.21.15.90:0/158235017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x556402245690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:33.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.170+0000 7f1153e00700 1 --2- 172.21.15.90:0/158235017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402239d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1140011c00 tx=0x7f1140004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=351f2486cab25dd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:33.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.170+0000 7f1149600700 1 -- 172.21.15.90:0/158235017 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140025040 con 0x5564021909f0 2024-05-27T07:54:33.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.171+0000 7f1149600700 1 -- 172.21.15.90:0/158235017 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11400054d0 con 0x5564021909f0 2024-05-27T07:54:33.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.171+0000 7f1149600700 1 -- 172.21.15.90:0/158235017 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11400047c0 con 0x5564021909f0 2024-05-27T07:54:33.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.172+0000 7f116a15eb80 1 -- 172.21.15.90:0/158235017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 msgr2=0x556402239d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:33.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.172+0000 7f116a15eb80 1 --2- 172.21.15.90:0/158235017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402239d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1140011c00 tx=0x7f1140004600 comp rx=0 tx=0).stop 2024-05-27T07:54:33.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.172+0000 7f116a15eb80 1 -- 172.21.15.90:0/158235017 shutdown_connections 2024-05-27T07:54:33.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.172+0000 7f116a15eb80 1 --2- 172.21.15.90:0/158235017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556402223e70 0x556402245690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.172+0000 7f116a15eb80 1 --2- 172.21.15.90:0/158235017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5564020a4e40 0x5564022237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.172+0000 7f116a15eb80 1 --2- 172.21.15.90:0/158235017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564021909f0 0x556402239d40 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1140011c00 tx=0x7f1140004600 comp rx=0 tx=0).stop 2024-05-27T07:54:33.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.172+0000 7f116a15eb80 1 -- 172.21.15.90:0/158235017 >> 172.21.15.90:0/158235017 conn(0x55640213e390 msgr2=0x5564022230b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:33.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.173+0000 7f116a15eb80 1 -- 172.21.15.90:0/158235017 shutdown_connections 2024-05-27T07:54:33.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.173+0000 7f116a15eb80 1 -- 172.21.15.90:0/158235017 wait complete. 2024-05-27T07:54:33.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.174+0000 7f116a15eb80 1 -- start start 2024-05-27T07:54:33.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.174+0000 7f116a15eb80 1 Processor -- start 2024-05-27T07:54:33.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.176+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 0x5564022316d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.176+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556402223e70 0x556402231c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.176+0000 7f116a15eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564022a0590 0x5564022a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.176+0000 7f1158c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 0x5564022316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.177+0000 7f1158c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 0x5564022316d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33862/0 (socket says 172.21.15.90:33862) 2024-05-27T07:54:33.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.177+0000 7f1153e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556402223e70 0x556402231c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.177+0000 7f1153e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556402223e70 0x556402231c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36292/0 (socket says 172.21.15.90:36292) 2024-05-27T07:54:33.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.177+0000 7f116a15eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f113400f580 con 0x5564022a0590 2024-05-27T07:54:33.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.177+0000 7f116a15eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f113400eeb0 con 0x5564020a4e40 2024-05-27T07:54:33.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.177+0000 7f116a15eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f113400fc50 con 0x556402223e70 2024-05-27T07:54:33.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.177+0000 7f1159600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564022a0590 0x5564022a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.177+0000 7f1158c00700 1 -- 172.21.15.90:0/440494964 learned_addr learned my addr 172.21.15.90:0/440494964 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:33.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.178+0000 7f116a15eb80 10 client.0 _ll_register_callbacks cb 0x5564020ebc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:33.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.178+0000 7f1158c00700 1 -- 172.21.15.90:0/440494964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556402223e70 msgr2=0x556402231c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:33.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.178+0000 7f1158c00700 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556402223e70 0x556402231c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.178+0000 7f1158c00700 1 -- 172.21.15.90:0/440494964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564022a0590 msgr2=0x5564022a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:33.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.179+0000 7f1158c00700 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564022a0590 0x5564022a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.179+0000 7f1158c00700 1 -- 172.21.15.90:0/440494964 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564022a0f90 con 0x5564020a4e40 2024-05-27T07:54:33.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.179+0000 7f1159600700 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564022a0590 0x5564022a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:33.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.180+0000 7f1158c00700 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 0x5564022316d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x556402220a20 tx=0x7f114c00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e31a99a44258732 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:33.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.180+0000 7f113ea00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:54:33.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.181+0000 7f113ea00700 1 -- 172.21.15.90:0/440494964 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c032070 con 0x5564020a4e40 2024-05-27T07:54:33.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.181+0000 7f116a15eb80 1 -- 172.21.15.90:0/440494964 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11400054d0 con 0x5564020a4e40 2024-05-27T07:54:33.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.181+0000 7f113ea00700 1 -- 172.21.15.90:0/440494964 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f114c00f040 con 0x5564020a4e40 2024-05-27T07:54:33.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.182+0000 7f113ea00700 1 -- 172.21.15.90:0/440494964 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f114c03e050 con 0x5564020a4e40 2024-05-27T07:54:33.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.182+0000 7f116a15eb80 10 client.36596 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:33.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.182+0000 7f116a15eb80 1 -- 172.21.15.90:0/440494964 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5564022a8290 con 0x5564020a4e40 2024-05-27T07:54:33.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.182+0000 7f113ea00700 1 -- 172.21.15.90:0/440494964 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11400054d0 con 0x5564020a4e40 2024-05-27T07:54:33.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.183+0000 7f113ea00700 1 -- 172.21.15.90:0/440494964 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7f114c011070 con 0x5564020a4e40 2024-05-27T07:54:33.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.183+0000 7f116a15eb80 10 client.36596 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:33.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.183+0000 7f116a15eb80 10 client.36596 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:54:33.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.183+0000 7f116a15eb80 20 client.36596 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:33.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.183+0000 7f116a15eb80 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564022a84c0 0x5564022aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:33.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.183+0000 7f116a15eb80 4 client.36596 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:54:33.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.183+0000 7f116a15eb80 1 -- 172.21.15.90:0/440494964 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55640214d570 con 0x5564022a84c0 2024-05-27T07:54:33.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.184+0000 7f1153e00700 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564022a84c0 0x5564022aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:33.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.185+0000 7f1153e00700 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564022a84c0 0x5564022aa8e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee701580476bc60 server_cookie=4ebbd46d6000c783 in_seq=0 out_seq=0 2024-05-27T07:54:33.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.185+0000 7f113ea00700 10 client.36596 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:54:33.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.185+0000 7f113ea00700 1 -- 172.21.15.90:0/440494964 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55640214d570 con 0x5564022a84c0 2024-05-27T07:54:33.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.185+0000 7f113ea00700 10 client.36596 handle_command_reply: tid=0 2024-05-27T07:54:33.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.185+0000 7f116a15eb80 1 client.36596 shutdown 2024-05-27T07:54:33.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.186+0000 7f116a15eb80 1 -- 172.21.15.90:0/440494964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 msgr2=0x5564022316d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:33.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.186+0000 7f116a15eb80 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 0x5564022316d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x556402220a20 tx=0x7f114c00b3b0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.187+0000 7f116a15eb80 1 -- 172.21.15.90:0/440494964 shutdown_connections 2024-05-27T07:54:33.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.187+0000 7f116a15eb80 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5564022a84c0 0x5564022aa8e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.187+0000 7f116a15eb80 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5564022a0590 0x5564022a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.187+0000 7f116a15eb80 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564020a4e40 0x5564022316d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.187+0000 7f116a15eb80 1 --2- 172.21.15.90:0/440494964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556402223e70 0x556402231c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.187+0000 7f116a15eb80 1 -- 172.21.15.90:0/440494964 >> 172.21.15.90:0/440494964 conn(0x55640213e390 msgr2=0x55640222d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:33.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.187+0000 7f113ea00700 0 client.36596 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:54:33.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.187+0000 7f116a15eb80 1 -- 172.21.15.90:0/440494964 shutdown_connections 2024-05-27T07:54:33.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.188+0000 7f116a15eb80 1 -- 172.21.15.90:0/440494964 wait complete. 2024-05-27T07:54:33.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.188+0000 7f116a15eb80 20 client.36596 trim_cache size 0 max 16384 2024-05-27T07:54:33.190 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.189+0000 7f115b600700 1 -- 172.21.15.90:0/1577396406 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11380b29f0 msgr2=0x7f11380b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.189+0000 7f115b600700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11380b29f0 0x7f11380b4f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f114000bfd0 tx=0x7f114000b820 comp rx=0 tx=0).stop 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.189+0000 7f115b600700 1 -- 172.21.15.90:0/1577396406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1154115210 msgr2=0x7f115406ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.189+0000 7f115b600700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1154115210 0x7f115406ebe0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f114400e940 tx=0x7f114400ed00 comp rx=0 tx=0).stop 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.190+0000 7f115b600700 1 -- 172.21.15.90:0/1577396406 shutdown_connections 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.190+0000 7f115b600700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11380b29f0 0x7f11380b4f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.190+0000 7f115b600700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1154115210 0x7f115406ebe0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.190+0000 7f115b600700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1154078df0 0x7f1154075410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.190+0000 7f115b600700 1 --2- 172.21.15.90:0/1577396406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1154079740 0x7f115406e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.190+0000 7f115b600700 1 -- 172.21.15.90:0/1577396406 >> 172.21.15.90:0/1577396406 conn(0x7f1154102070 msgr2=0x7f1154104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:33.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.190+0000 7f115b600700 1 -- 172.21.15.90:0/1577396406 shutdown_connections 2024-05-27T07:54:33.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:33.190+0000 7f115b600700 1 -- 172.21.15.90:0/1577396406 wait complete. 2024-05-27T07:54:33.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 255.0 s, 537.4 tps, lat 189.501 ms stddev 92.722 2024-05-27T07:54:34.236 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:54:34.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:34 smithi090 ceph-mon[31429]: pgmap v257: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.3 MiB/s wr, 618 op/s 2024-05-27T07:54:34.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:34 smithi129 ceph-mon[35254]: pgmap v257: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.3 MiB/s wr, 618 op/s 2024-05-27T07:54:34.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:34 smithi136 ceph-mon[35324]: pgmap v257: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 8.3 MiB/s wr, 618 op/s 2024-05-27T07:54:34.951 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.355+0000 7f3285200640 1 -- 172.21.15.90:0/928537969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280075480 msgr2=0x7f3280075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.355+0000 7f3285200640 1 --2- 172.21.15.90:0/928537969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280075480 0x7f3280075890 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3278009860 tx=0x7f327802a3d0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.356+0000 7f3285200640 1 -- 172.21.15.90:0/928537969 shutdown_connections 2024-05-27T07:54:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.356+0000 7f3285200640 1 --2- 172.21.15.90:0/928537969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114ac0 0x7f3280116ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.356+0000 7f3285200640 1 --2- 172.21.15.90:0/928537969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280112160 0x7f3280114580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.356+0000 7f3285200640 1 --2- 172.21.15.90:0/928537969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280075480 0x7f3280075890 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.356+0000 7f3285200640 1 -- 172.21.15.90:0/928537969 >> 172.21.15.90:0/928537969 conn(0x7f3280079240 msgr2=0x7f3280079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.357+0000 7f3285200640 1 -- 172.21.15.90:0/928537969 shutdown_connections 2024-05-27T07:54:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.357+0000 7f3285200640 1 -- 172.21.15.90:0/928537969 wait complete. 2024-05-27T07:54:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.359+0000 7f3285200640 1 Processor -- start 2024-05-27T07:54:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.359+0000 7f3285200640 1 -- start start 2024-05-27T07:54:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.360+0000 7f3285200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280075480 0x7f32801a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.360+0000 7f3285200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280112160 0x7f32801a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.360+0000 7f3285200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114ac0 0x7f32801a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.360+0000 7f3285200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f328011a140 con 0x7f3280075480 2024-05-27T07:54:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.360+0000 7f3285200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3280119fc0 con 0x7f3280114ac0 2024-05-27T07:54:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.360+0000 7f3285200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f328011a2c0 con 0x7f3280112160 2024-05-27T07:54:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.361+0000 7f327f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280075480 0x7f32801a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.361+0000 7f327fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114ac0 0x7f32801a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.361+0000 7f327f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280075480 0x7f32801a66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45674/0 (socket says 172.21.15.90:45674) 2024-05-27T07:54:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.361+0000 7f327fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114ac0 0x7f32801a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33880/0 (socket says 172.21.15.90:33880) 2024-05-27T07:54:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.361+0000 7f327f400640 1 -- 172.21.15.90:0/3685075103 learned_addr learned my addr 172.21.15.90:0/3685075103 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.361+0000 7f327ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280112160 0x7f32801a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.362+0000 7f327fe00640 1 -- 172.21.15.90:0/3685075103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280112160 msgr2=0x7f32801a6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.362+0000 7f327fe00640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280112160 0x7f32801a6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.362+0000 7f327fe00640 1 -- 172.21.15.90:0/3685075103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280075480 msgr2=0x7f32801a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.362+0000 7f327fe00640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280075480 0x7f32801a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.362+0000 7f327fe00640 1 -- 172.21.15.90:0/3685075103 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32801a0ea0 con 0x7f3280114ac0 2024-05-27T07:54:35.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.362+0000 7f327fe00640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114ac0 0x7f32801a08b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f327000ef00 tx=0x7f327000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.363+0000 7f3277e00640 1 -- 172.21.15.90:0/3685075103 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270004b20 con 0x7f3280114ac0 2024-05-27T07:54:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.363+0000 7f3285200640 1 -- 172.21.15.90:0/3685075103 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32801a1190 con 0x7f3280114ac0 2024-05-27T07:54:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.363+0000 7f3285200640 1 -- 172.21.15.90:0/3685075103 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32801ad520 con 0x7f3280114ac0 2024-05-27T07:54:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.363+0000 7f3277e00640 1 -- 172.21.15.90:0/3685075103 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3270003e00 con 0x7f3280114ac0 2024-05-27T07:54:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.363+0000 7f3277e00640 1 -- 172.21.15.90:0/3685075103 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270004770 con 0x7f3280114ac0 2024-05-27T07:54:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.372+0000 7f3277e00640 1 -- 172.21.15.90:0/3685075103 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f327001d020 con 0x7f3280114ac0 2024-05-27T07:54:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.372+0000 7f328d83a740 1 Processor -- start 2024-05-27T07:54:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.372+0000 7f328d83a740 1 -- start start 2024-05-27T07:54:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.374+0000 7f328d83a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9aa11b0 0x5558a9e43a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.374+0000 7f328d83a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 0x5558a9e43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.374+0000 7f328d83a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.374+0000 7f328d83a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5558a9db4b90 con 0x5558a9a782f0 2024-05-27T07:54:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.374+0000 7f328d83a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5558a9da4730 con 0x5558a9a804d0 2024-05-27T07:54:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.374+0000 7f328d83a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5558a9da8770 con 0x5558a9aa11b0 2024-05-27T07:54:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.375+0000 7f3277e00640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2a50 0x7f32640b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.376+0000 7f3277e00640 1 -- 172.21.15.90:0/3685075103 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f32700e2700 con 0x7f3280114ac0 2024-05-27T07:54:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.376+0000 7f327ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9aa11b0 0x5558a9e43a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.376+0000 7f327ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.376+0000 7f327fe00640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2a50 0x7f32640b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.377+0000 7f327f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 0x5558a9e43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.377+0000 7f327ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e444e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33882/0 (socket says 172.21.15.90:33882) 2024-05-27T07:54:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.377+0000 7f327ea00640 1 -- 172.21.15.90:0/1285869346 learned_addr learned my addr 172.21.15.90:0/1285869346 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.378+0000 7f327fe00640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2a50 0x7f32640b4f60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f32700a9540 tx=0x7f32700e2950 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.379+0000 7f327f400640 1 -- 172.21.15.90:0/1285869346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9aa11b0 msgr2=0x5558a9e43a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.379+0000 7f327f400640 1 --2- 172.21.15.90:0/1285869346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9aa11b0 0x5558a9e43a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.379+0000 7f327f400640 1 -- 172.21.15.90:0/1285869346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 msgr2=0x5558a9e444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.379+0000 7f327f400640 1 --2- 172.21.15.90:0/1285869346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.379+0000 7f327f400640 1 -- 172.21.15.90:0/1285869346 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a9d3d8b0 con 0x5558a9a782f0 2024-05-27T07:54:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.380+0000 7f327f400640 1 --2- 172.21.15.90:0/1285869346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 0x5558a9e43fa0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3278009830 tx=0x7f3278030640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941696cdddb564fe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.381+0000 7f3274c00640 1 -- 172.21.15.90:0/1285869346 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278030cf0 con 0x5558a9a782f0 2024-05-27T07:54:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.381+0000 7f3274c00640 1 -- 172.21.15.90:0/1285869346 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3278041070 con 0x5558a9a782f0 2024-05-27T07:54:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.381+0000 7f3274c00640 1 -- 172.21.15.90:0/1285869346 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278004aa0 con 0x5558a9a782f0 2024-05-27T07:54:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.383+0000 7f328d83a740 1 -- 172.21.15.90:0/1285869346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 msgr2=0x5558a9e43fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.383+0000 7f328d83a740 1 --2- 172.21.15.90:0/1285869346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 0x5558a9e43fa0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3278009830 tx=0x7f3278030640 comp rx=0 tx=0).stop 2024-05-27T07:54:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.383+0000 7f328d83a740 1 -- 172.21.15.90:0/1285869346 shutdown_connections 2024-05-27T07:54:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.383+0000 7f328d83a740 1 --2- 172.21.15.90:0/1285869346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.383+0000 7f328d83a740 1 --2- 172.21.15.90:0/1285869346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 0x5558a9e43fa0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.383+0000 7f328d83a740 1 --2- 172.21.15.90:0/1285869346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9aa11b0 0x5558a9e43a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.383+0000 7f328d83a740 1 -- 172.21.15.90:0/1285869346 >> 172.21.15.90:0/1285869346 conn(0x5558a9a95300 msgr2=0x5558a9e40210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.384+0000 7f328d83a740 1 -- 172.21.15.90:0/1285869346 shutdown_connections 2024-05-27T07:54:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.384+0000 7f328d83a740 1 -- 172.21.15.90:0/1285869346 wait complete. 2024-05-27T07:54:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.386+0000 7f328d83a740 1 -- start start 2024-05-27T07:54:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.386+0000 7f328d83a740 1 Processor -- start 2024-05-27T07:54:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.387+0000 7f328d83a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 0x5558a9e4dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.387+0000 7f328d83a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.388+0000 7f328d83a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9aa11b0 0x5558a9e59ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.388+0000 7f328d83a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5558a9a8a790 con 0x5558a9a782f0 2024-05-27T07:54:35.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.388+0000 7f328d83a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5558a9d4cbc0 con 0x5558a9a804d0 2024-05-27T07:54:35.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.389+0000 7f327f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.389+0000 7f328d83a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5558a9e50b30 con 0x5558a9aa11b0 2024-05-27T07:54:35.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.389+0000 7f327f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e4e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33898/0 (socket says 172.21.15.90:33898) 2024-05-27T07:54:35.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.389+0000 7f327f400640 1 -- 172.21.15.90:0/324198798 learned_addr learned my addr 172.21.15.90:0/324198798 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.389+0000 7f328d83a740 10 client.0 _ll_register_callbacks cb 0x5558a9d2a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.390+0000 7f327ea00640 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9aa11b0 0x5558a9e59ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.390+0000 7f327ea00640 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 0x5558a9e4dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.391+0000 7f327f400640 1 -- 172.21.15.90:0/324198798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9aa11b0 msgr2=0x5558a9e59ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.391+0000 7f327f400640 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9aa11b0 0x5558a9e59ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.391+0000 7f327f400640 1 -- 172.21.15.90:0/324198798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 msgr2=0x5558a9e4dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.391+0000 7f327f400640 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 0x5558a9e4dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.391+0000 7f327f400640 1 -- 172.21.15.90:0/324198798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a9ab9210 con 0x5558a9a804d0 2024-05-27T07:54:35.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.392+0000 7f327f400640 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e4e120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3278009250 tx=0x7f32780466a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16a19d0b3e47ea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:35.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.392+0000 7f3269600640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:54:35.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.393+0000 7f3269600640 1 -- 172.21.15.90:0/324198798 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278052030 con 0x5558a9a804d0 2024-05-27T07:54:35.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.393+0000 7f3269600640 1 -- 172.21.15.90:0/324198798 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f3278046a20 con 0x5558a9a804d0 2024-05-27T07:54:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.394+0000 7f328d83a740 1 -- 172.21.15.90:0/324198798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3278041070 con 0x5558a9a804d0 2024-05-27T07:54:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.395+0000 7f3269600640 1 -- 172.21.15.90:0/324198798 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3278046c20 con 0x5558a9a804d0 2024-05-27T07:54:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.396+0000 7f328d83a740 10 client.36608 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.396+0000 7f328d83a740 1 -- 172.21.15.90:0/324198798 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5558a9b85620 con 0x5558a9a804d0 2024-05-27T07:54:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.397+0000 7f3269600640 1 -- 172.21.15.90:0/324198798 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3278041070 con 0x5558a9a804d0 2024-05-27T07:54:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.397+0000 7f3269600640 1 -- 172.21.15.90:0/324198798 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7f327803dbb0 con 0x5558a9a804d0 2024-05-27T07:54:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.397+0000 7f328d83a740 10 client.36608 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.397+0000 7f328d83a740 10 client.36608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.397+0000 7f328d83a740 20 client.36608 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.398+0000 7f328d83a740 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5558a9a5fd80 0x5558a9e5fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.398+0000 7f328d83a740 4 client.36608 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.398+0000 7f328d83a740 1 -- 172.21.15.90:0/324198798 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558a9db6ae0 con 0x5558a9a5fd80 2024-05-27T07:54:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.398+0000 7f327ea00640 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5558a9a5fd80 0x5558a9e5fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.400+0000 7f327ea00640 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5558a9a5fd80 0x5558a9e5fc90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c6844a9f1779e server_cookie=4cd55190de681c53 in_seq=0 out_seq=0 2024-05-27T07:54:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.400+0000 7f3269600640 10 client.36608 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.403+0000 7f3269600640 1 -- 172.21.15.90:0/324198798 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5558a9db6ae0 con 0x5558a9a5fd80 2024-05-27T07:54:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.403+0000 7f3269600640 10 client.36608 handle_command_reply: tid=0 2024-05-27T07:54:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.404+0000 7f328d83a740 1 client.36608 shutdown 2024-05-27T07:54:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.406+0000 7f328d83a740 1 -- 172.21.15.90:0/324198798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 msgr2=0x5558a9e4e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.406+0000 7f328d83a740 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e4e120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3278009250 tx=0x7f32780466a0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.406+0000 7f328d83a740 1 -- 172.21.15.90:0/324198798 shutdown_connections 2024-05-27T07:54:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.406+0000 7f328d83a740 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5558a9a5fd80 0x5558a9e5fc90 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.406+0000 7f3269600640 0 client.36608 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.406+0000 7f328d83a740 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9a804d0 0x5558a9e4e120 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.406+0000 7f328d83a740 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9aa11b0 0x5558a9e59ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.406+0000 7f328d83a740 1 --2- 172.21.15.90:0/324198798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a782f0 0x5558a9e4dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.406+0000 7f328d83a740 1 -- 172.21.15.90:0/324198798 >> 172.21.15.90:0/324198798 conn(0x5558a9a95300 msgr2=0x5558a9e43890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.407+0000 7f328d83a740 1 -- 172.21.15.90:0/324198798 shutdown_connections 2024-05-27T07:54:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.407+0000 7f328d83a740 1 -- 172.21.15.90:0/324198798 wait complete. 2024-05-27T07:54:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.408+0000 7f328d83a740 20 client.36608 trim_cache size 0 max 16384 2024-05-27T07:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.438+0000 7f328d83a740 1 Processor -- start 2024-05-27T07:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.438+0000 7f328d83a740 1 -- start start 2024-05-27T07:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.439+0000 7f328d83a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 0x5558a9e50310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.439+0000 7f328d83a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 0x5558a9e59e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.439+0000 7f328d83a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 0x5558a9e62cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.439+0000 7f328d83a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3260011a40 con 0x5558a9a804d0 2024-05-27T07:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.439+0000 7f328d83a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f326000f830 con 0x5558a9aa11b0 2024-05-27T07:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.439+0000 7f328d83a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3260011370 con 0x5558a9a782f0 2024-05-27T07:54:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.440+0000 7f327f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 0x5558a9e59e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.440+0000 7f327ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 0x5558a9e62cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.440+0000 7f327ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 0x5558a9e50310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.440+0000 7f327ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 0x5558a9e62cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:33914/0 (socket says 172.21.15.90:33914) 2024-05-27T07:54:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.440+0000 7f327ea00640 1 -- 172.21.15.90:0/2832465882 learned_addr learned my addr 172.21.15.90:0/2832465882 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.442+0000 7f327ea00640 1 -- 172.21.15.90:0/2832465882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 msgr2=0x5558a9e50310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.442+0000 7f327ea00640 1 --2- 172.21.15.90:0/2832465882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 0x5558a9e50310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.442+0000 7f327ea00640 1 -- 172.21.15.90:0/2832465882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 msgr2=0x5558a9e59e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.442+0000 7f327ea00640 1 --2- 172.21.15.90:0/2832465882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 0x5558a9e59e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.442+0000 7f327ea00640 1 -- 172.21.15.90:0/2832465882 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a9e48180 con 0x5558a9aa11b0 2024-05-27T07:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.442+0000 7f327f400640 1 --2- 172.21.15.90:0/2832465882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 0x5558a9e59e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.443+0000 7f327ea00640 1 --2- 172.21.15.90:0/2832465882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 0x5558a9e62cb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5558a9a8a870 tx=0x7f326c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b74c816925dd07 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.443+0000 7f326be00640 1 -- 172.21.15.90:0/2832465882 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c029030 con 0x5558a9aa11b0 2024-05-27T07:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.443+0000 7f326be00640 1 -- 172.21.15.90:0/2832465882 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f326c02a040 con 0x5558a9aa11b0 2024-05-27T07:54:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.443+0000 7f326be00640 1 -- 172.21.15.90:0/2832465882 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c023a70 con 0x5558a9aa11b0 2024-05-27T07:54:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.445+0000 7f328d83a740 1 -- 172.21.15.90:0/2832465882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 msgr2=0x5558a9e62cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.445+0000 7f328d83a740 1 --2- 172.21.15.90:0/2832465882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 0x5558a9e62cb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5558a9a8a870 tx=0x7f326c004730 comp rx=0 tx=0).stop 2024-05-27T07:54:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.447+0000 7f328d83a740 1 -- 172.21.15.90:0/2832465882 shutdown_connections 2024-05-27T07:54:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.447+0000 7f328d83a740 1 --2- 172.21.15.90:0/2832465882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 0x5558a9e62cb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.447+0000 7f328d83a740 1 --2- 172.21.15.90:0/2832465882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 0x5558a9e59e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.447+0000 7f328d83a740 1 --2- 172.21.15.90:0/2832465882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 0x5558a9e50310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.447+0000 7f328d83a740 1 -- 172.21.15.90:0/2832465882 >> 172.21.15.90:0/2832465882 conn(0x5558a9a95300 msgr2=0x5558a9e4db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.447+0000 7f328d83a740 1 -- 172.21.15.90:0/2832465882 shutdown_connections 2024-05-27T07:54:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.448+0000 7f328d83a740 1 -- 172.21.15.90:0/2832465882 wait complete. 2024-05-27T07:54:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.448+0000 7f328d83a740 1 -- start start 2024-05-27T07:54:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.448+0000 7f328d83a740 1 Processor -- start 2024-05-27T07:54:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.451+0000 7f328d83a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 0x5558a9e601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.452+0000 7f328d83a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 0x5558a9e60730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.452+0000 7f328d83a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 0x5558a9ec60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.452+0000 7f328d83a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f32600105d0 con 0x5558a9a804d0 2024-05-27T07:54:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.452+0000 7f328d83a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f326000ff00 con 0x5558a9aa11b0 2024-05-27T07:54:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.452+0000 7f328d83a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3260010ca0 con 0x5558a9a782f0 2024-05-27T07:54:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.452+0000 7f328d83a740 10 client.0 _ll_register_callbacks cb 0x7f326c00aa60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.453+0000 7f327ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 0x5558a9e601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.453+0000 7f327ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 0x5558a9e601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36364/0 (socket says 172.21.15.90:36364) 2024-05-27T07:54:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.453+0000 7f327ea00640 1 -- 172.21.15.90:0/3906263629 learned_addr learned my addr 172.21.15.90:0/3906263629 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.454+0000 7f327ea00640 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 0x5558a9ec60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.454+0000 7f327f400640 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 0x5558a9e60730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.454+0000 7f327ea00640 1 -- 172.21.15.90:0/3906263629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 msgr2=0x5558a9ec60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.454+0000 7f327ea00640 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 0x5558a9ec60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.454+0000 7f327ea00640 1 -- 172.21.15.90:0/3906263629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 msgr2=0x5558a9e60730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.455+0000 7f327ea00640 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 0x5558a9e60730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.455+0000 7f327ea00640 1 -- 172.21.15.90:0/3906263629 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a9ec6610 con 0x5558a9a782f0 2024-05-27T07:54:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.455+0000 7f327f400640 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 0x5558a9e60730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.456+0000 7f327ea00640 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 0x5558a9e601f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f326c004700 tx=0x7f326c005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b10ee887cf4f921f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.456+0000 7f3269600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:54:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.457+0000 7f3269600640 1 -- 172.21.15.90:0/3906263629 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c029070 con 0x5558a9a782f0 2024-05-27T07:54:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.457+0000 7f3269600640 1 -- 172.21.15.90:0/3906263629 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7f326c012b00 con 0x5558a9a782f0 2024-05-27T07:54:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.457+0000 7f328d83a740 1 -- 172.21.15.90:0/3906263629 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f326c02a040 con 0x5558a9a782f0 2024-05-27T07:54:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.458+0000 7f3269600640 1 -- 172.21.15.90:0/3906263629 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f326c012d00 con 0x5558a9a782f0 2024-05-27T07:54:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.458+0000 7f328d83a740 10 client.26644 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.458+0000 7f328d83a740 1 -- 172.21.15.90:0/3906263629 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5558a9ec78a0 con 0x5558a9a782f0 2024-05-27T07:54:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.459+0000 7f3269600640 1 -- 172.21.15.90:0/3906263629 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f326c02a040 con 0x5558a9a782f0 2024-05-27T07:54:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.459+0000 7f3269600640 1 -- 172.21.15.90:0/3906263629 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7f326c02d020 con 0x5558a9a782f0 2024-05-27T07:54:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.459+0000 7f328d83a740 10 client.26644 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.459+0000 7f328d83a740 10 client.26644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.459+0000 7f328d83a740 20 client.26644 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.460+0000 7f328d83a740 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5558a9a351c0 0x5558a9ec9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.460+0000 7f328d83a740 4 client.26644 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:54:35.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.460+0000 7f328d83a740 1 -- 172.21.15.90:0/3906263629 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558a9db02c0 con 0x5558a9a351c0 2024-05-27T07:54:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.460+0000 7f327f400640 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5558a9a351c0 0x5558a9ec9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.462+0000 7f327f400640 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5558a9a351c0 0x5558a9ec9e50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943e8efe5d43f4ac server_cookie=5dc427827aec9a00 in_seq=0 out_seq=0 2024-05-27T07:54:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.462+0000 7f3269600640 10 client.26644 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.549+0000 7f3269600640 1 -- 172.21.15.90:0/3906263629 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5558a9db02c0 con 0x5558a9a351c0 2024-05-27T07:54:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.549+0000 7f3269600640 10 client.26644 handle_command_reply: tid=0 2024-05-27T07:54:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.549+0000 7f328d83a740 1 client.26644 shutdown 2024-05-27T07:54:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.551+0000 7f328d83a740 1 -- 172.21.15.90:0/3906263629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 msgr2=0x5558a9e601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.551+0000 7f328d83a740 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 0x5558a9e601f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f326c004700 tx=0x7f326c005b70 comp rx=0 tx=0).stop 2024-05-27T07:54:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.552+0000 7f328d83a740 1 -- 172.21.15.90:0/3906263629 shutdown_connections 2024-05-27T07:54:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.552+0000 7f328d83a740 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5558a9a351c0 0x5558a9ec9e50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.552+0000 7f328d83a740 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5558a9aa11b0 0x5558a9ec60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.552+0000 7f328d83a740 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5558a9a804d0 0x5558a9e60730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.552+0000 7f328d83a740 1 --2- 172.21.15.90:0/3906263629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5558a9a782f0 0x5558a9e601f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.552+0000 7f3269600640 0 client.26644 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.552+0000 7f328d83a740 1 -- 172.21.15.90:0/3906263629 >> 172.21.15.90:0/3906263629 conn(0x5558a9a95300 msgr2=0x5558a9e4d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.553+0000 7f328d83a740 1 -- 172.21.15.90:0/3906263629 shutdown_connections 2024-05-27T07:54:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.553+0000 7f328d83a740 1 -- 172.21.15.90:0/3906263629 wait complete. 2024-05-27T07:54:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.553+0000 7f328d83a740 20 client.26644 trim_cache size 0 max 16384 2024-05-27T07:54:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:54:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:54:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:54:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.556+0000 7f3285200640 1 -- 172.21.15.90:0/3685075103 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2a50 msgr2=0x7f32640b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.556+0000 7f3285200640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2a50 0x7f32640b4f60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f32700a9540 tx=0x7f32700e2950 comp rx=0 tx=0).stop 2024-05-27T07:54:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.556+0000 7f3285200640 1 -- 172.21.15.90:0/3685075103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114ac0 msgr2=0x7f32801a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.556+0000 7f3285200640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114ac0 0x7f32801a08b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f327000ef00 tx=0x7f327000c740 comp rx=0 tx=0).stop 2024-05-27T07:54:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.556+0000 7f3285200640 1 -- 172.21.15.90:0/3685075103 shutdown_connections 2024-05-27T07:54:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.556+0000 7f3285200640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2a50 0x7f32640b4f60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.556+0000 7f3285200640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114ac0 0x7f32801a08b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.556+0000 7f3285200640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280112160 0x7f32801a6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.556+0000 7f3285200640 1 --2- 172.21.15.90:0/3685075103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280075480 0x7f32801a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.556+0000 7f3285200640 1 -- 172.21.15.90:0/3685075103 >> 172.21.15.90:0/3685075103 conn(0x7f3280079240 msgr2=0x7f3280112570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.557+0000 7f3285200640 1 -- 172.21.15.90:0/3685075103 shutdown_connections 2024-05-27T07:54:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:35.558+0000 7f3285200640 1 -- 172.21.15.90:0/3685075103 wait complete. 2024-05-27T07:54:36.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:36 smithi129 ceph-mon[35254]: pgmap v258: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 9.3 MiB/s wr, 710 op/s 2024-05-27T07:54:36.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:36 smithi090 ceph-mon[31429]: pgmap v258: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 9.3 MiB/s wr, 710 op/s 2024-05-27T07:54:36.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:36 smithi136 ceph-mon[35324]: pgmap v258: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 9.3 MiB/s wr, 710 op/s 2024-05-27T07:54:38.524 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:38 smithi090 ceph-mon[31429]: pgmap v259: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.5 MiB/s wr, 455 op/s 2024-05-27T07:54:38.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:38 smithi129 ceph-mon[35254]: pgmap v259: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.5 MiB/s wr, 455 op/s 2024-05-27T07:54:38.749 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:38 smithi136 ceph-mon[35324]: pgmap v259: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.5 MiB/s wr, 455 op/s 2024-05-27T07:54:38.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.789+0000 7fe124000700 1 -- 172.21.15.90:0/846868998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe11c1054d0 msgr2=0x7fe11c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:38.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.789+0000 7fe124000700 1 --2- 172.21.15.90:0/846868998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe11c1054d0 0x7fe11c1058e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fe1100098f0 tx=0x7fe110009c00 comp rx=0 tx=0).stop 2024-05-27T07:54:38.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.790+0000 7fe124000700 1 -- 172.21.15.90:0/846868998 shutdown_connections 2024-05-27T07:54:38.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.790+0000 7fe124000700 1 --2- 172.21.15.90:0/846868998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe11c10e900 0x7fe11c110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.790+0000 7fe124000700 1 --2- 172.21.15.90:0/846868998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe11c1054d0 0x7fe11c1058e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.790+0000 7fe124000700 1 --2- 172.21.15.90:0/846868998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe11c105e20 0x7fe11c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.790+0000 7fe124000700 1 -- 172.21.15.90:0/846868998 >> 172.21.15.90:0/846868998 conn(0x7fe11c0fedb0 msgr2=0x7fe11c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:38.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.790+0000 7fe124000700 1 -- 172.21.15.90:0/846868998 shutdown_connections 2024-05-27T07:54:38.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.791+0000 7fe124000700 1 -- 172.21.15.90:0/846868998 wait complete. 2024-05-27T07:54:38.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.792+0000 7fe124000700 1 Processor -- start 2024-05-27T07:54:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.793+0000 7fe124000700 1 -- start start 2024-05-27T07:54:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.794+0000 7fe124000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe11c1054d0 0x7fe11c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.794+0000 7fe124000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe11c105e20 0x7fe11c19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.794+0000 7fe124000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe11c10e900 0x7fe11c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.796+0000 7fe120c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe11c105e20 0x7fe11c19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.796+0000 7fe120c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe11c105e20 0x7fe11c19d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36370/0 (socket says 172.21.15.90:36370) 2024-05-27T07:54:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.796+0000 7fe120c00700 1 -- 172.21.15.90:0/719771307 learned_addr learned my addr 172.21.15.90:0/719771307 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.796+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c116500 con 0x7fe11c10e900 2024-05-27T07:54:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.796+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c116380 con 0x7fe11c1054d0 2024-05-27T07:54:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.796+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c116680 con 0x7fe11c105e20 2024-05-27T07:54:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.796+0000 7fe122000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe11c10e900 0x7fe11c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.796+0000 7fe121600700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe11c1054d0 0x7fe11c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.797+0000 7fe122000700 1 -- 172.21.15.90:0/719771307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe11c105e20 msgr2=0x7fe11c19d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.797+0000 7fe122000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe11c105e20 0x7fe11c19d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.797+0000 7fe122000700 1 -- 172.21.15.90:0/719771307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe11c1054d0 msgr2=0x7fe11c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.797+0000 7fe122000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe11c1054d0 0x7fe11c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.797+0000 7fe122000700 1 -- 172.21.15.90:0/719771307 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c1a1e20 con 0x7fe11c10e900 2024-05-27T07:54:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.798+0000 7fe122000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe11c10e900 0x7fe11c1a1780 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe11400efc0 tx=0x7fe11400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:38.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.799+0000 7fe11a000700 1 -- 172.21.15.90:0/719771307 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1140092e0 con 0x7fe11c10e900 2024-05-27T07:54:38.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.799+0000 7fe11a000700 1 -- 172.21.15.90:0/719771307 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe114004bb0 con 0x7fe11c10e900 2024-05-27T07:54:38.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.799+0000 7fe11a000700 1 -- 172.21.15.90:0/719771307 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114004020 con 0x7fe11c10e900 2024-05-27T07:54:38.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.799+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11c1a20b0 con 0x7fe11c10e900 2024-05-27T07:54:38.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.799+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11c1a25a0 con 0x7fe11c10e900 2024-05-27T07:54:38.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.825+0000 7fe132b48b80 1 Processor -- start 2024-05-27T07:54:38.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.826+0000 7fe11a000700 1 -- 172.21.15.90:0/719771307 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe11401b030 con 0x7fe11c10e900 2024-05-27T07:54:38.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.826+0000 7fe132b48b80 1 -- start start 2024-05-27T07:54:38.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 260.0 s, 591.6 tps, lat 166.410 ms stddev 49.859 2024-05-27T07:54:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.827+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa50330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.827+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa50870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.827+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa50e70 0x55fb6aa5d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.827+0000 7fe132b48b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a97ee40 con 0x55fb6a8d1e40 2024-05-27T07:54:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.827+0000 7fe132b48b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a96e9e0 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.827+0000 7fe132b48b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a972a20 con 0x55fb6aa50e70 2024-05-27T07:54:38.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.830+0000 7fe120c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa50e70 0x55fb6aa5d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.830+0000 7fe11a000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1040b2af0 0x7fe1040b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.830+0000 7fe120c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa50330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.830+0000 7fe120c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa50330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49676/0 (socket says 172.21.15.90:49676) 2024-05-27T07:54:38.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.830+0000 7fe120c00700 1 -- 172.21.15.90:0/3178295630 learned_addr learned my addr 172.21.15.90:0/3178295630 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:38.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.830+0000 7fe11a000700 1 -- 172.21.15.90:0/719771307 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe1140e2410 con 0x7fe11c10e900 2024-05-27T07:54:38.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.831+0000 7fe122000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1040b2af0 0x7fe1040b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.831+0000 7fe120c00700 1 -- 172.21.15.90:0/3178295630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a9bd9f0 msgr2=0x55fb6aa50330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:38.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.831+0000 7fe120c00700 1 --2- 172.21.15.90:0/3178295630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa50330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.831+0000 7fe120c00700 1 -- 172.21.15.90:0/3178295630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6a8d1e40 msgr2=0x55fb6aa50870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:54:38.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.831+0000 7fe120c00700 1 --2- 172.21.15.90:0/3178295630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa50870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.831+0000 7fe120c00700 1 -- 172.21.15.90:0/3178295630 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6a937420 con 0x55fb6aa50e70 2024-05-27T07:54:38.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.832+0000 7fe120c00700 1 --2- 172.21.15.90:0/3178295630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa50e70 0x55fb6aa5d110 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe108014790 tx=0x7fe10800b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f7bd92179e94a21 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:38.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.832+0000 7fe10ea00700 1 -- 172.21.15.90:0/3178295630 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108004030 con 0x55fb6aa50e70 2024-05-27T07:54:38.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.833+0000 7fe122000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1040b2af0 0x7fe1040b5000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1140a53d0 tx=0x7fe114009040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:38.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.833+0000 7fe10ea00700 1 -- 172.21.15.90:0/3178295630 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe108004830 con 0x55fb6aa50e70 2024-05-27T07:54:38.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.833+0000 7fe10ea00700 1 -- 172.21.15.90:0/3178295630 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108005280 con 0x55fb6aa50e70 2024-05-27T07:54:38.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.834+0000 7fe132b48b80 1 -- 172.21.15.90:0/3178295630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa50e70 msgr2=0x55fb6aa5d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:38.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.834+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3178295630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa50e70 0x55fb6aa5d110 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe108014790 tx=0x7fe10800b370 comp rx=0 tx=0).stop 2024-05-27T07:54:38.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.835+0000 7fe132b48b80 1 -- 172.21.15.90:0/3178295630 shutdown_connections 2024-05-27T07:54:38.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.835+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3178295630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa50e70 0x55fb6aa5d110 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.835+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3178295630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa50330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.835+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3178295630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa50870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.835+0000 7fe132b48b80 1 -- 172.21.15.90:0/3178295630 >> 172.21.15.90:0/3178295630 conn(0x55fb6a96b390 msgr2=0x55fb6aa4c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:38.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.836+0000 7fe132b48b80 1 -- 172.21.15.90:0/3178295630 shutdown_connections 2024-05-27T07:54:38.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.836+0000 7fe132b48b80 1 -- 172.21.15.90:0/3178295630 wait complete. 2024-05-27T07:54:38.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.837+0000 7fe132b48b80 1 -- start start 2024-05-27T07:54:38.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.837+0000 7fe132b48b80 1 Processor -- start 2024-05-27T07:54:38.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.840+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa5a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.840+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa5aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.840+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 0x55fb6aa5af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.841+0000 7fe120c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa5a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.841+0000 7fe120c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa5a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49682/0 (socket says 172.21.15.90:49682) 2024-05-27T07:54:38.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.841+0000 7fe132b48b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a874fa0 con 0x55fb6aa50e70 2024-05-27T07:54:38.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.841+0000 7fe132b48b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb6a90f760 con 0x55fb6a8d1e40 2024-05-27T07:54:38.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.841+0000 7fe132b48b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fb6aa66430 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.841+0000 7fe132b48b80 10 client.0 _ll_register_callbacks cb 0x55fb6a918c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:38.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.841+0000 7fe120c00700 1 -- 172.21.15.90:0/3184205822 learned_addr learned my addr 172.21.15.90:0/3184205822 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:38.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.842+0000 7fe121600700 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa5aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.843+0000 7fe121600700 1 -- 172.21.15.90:0/3184205822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 msgr2=0x55fb6aa5a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:38.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.843+0000 7fe121600700 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa5a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.843+0000 7fe121600700 1 -- 172.21.15.90:0/3184205822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 msgr2=0x55fb6aa5af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:38.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.842+0000 7fe120c00700 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 0x55fb6aa5af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.846+0000 7fe121600700 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 0x55fb6aa5af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.846+0000 7fe121600700 1 -- 172.21.15.90:0/3184205822 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6a8be8d0 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.846+0000 7fe120c00700 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 0x55fb6aa5af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:38.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.846+0000 7fe121600700 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa5aa50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1100098c0 tx=0x7fe110009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a1645452fe35579 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:38.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.846+0000 7fe103400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:54:38.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.847+0000 7fe103400700 1 -- 172.21.15.90:0/3184205822 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110027060 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.847+0000 7fe103400700 1 -- 172.21.15.90:0/3184205822 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe110005040 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.849+0000 7fe132b48b80 1 -- 172.21.15.90:0/3184205822 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe108004830 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.850+0000 7fe103400700 1 -- 172.21.15.90:0/3184205822 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe110005240 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.850+0000 7fe132b48b80 10 client.26656 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:38.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.850+0000 7fe132b48b80 1 -- 172.21.15.90:0/3184205822 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55fb6a93f060 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.851+0000 7fe103400700 1 -- 172.21.15.90:0/3184205822 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe108004830 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.853+0000 7fe103400700 1 -- 172.21.15.90:0/3184205822 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7fe110021070 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.853+0000 7fe132b48b80 10 client.26656 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:38.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.853+0000 7fe132b48b80 10 client.26656 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:54:38.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.854+0000 7fe132b48b80 20 client.26656 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:38.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.854+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55fb6aa73270 0x55fb6aa756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.854+0000 7fe132b48b80 4 client.26656 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:38.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.855+0000 7fe132b48b80 1 -- 172.21.15.90:0/3184205822 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb6a980d90 con 0x55fb6aa73270 2024-05-27T07:54:38.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.855+0000 7fe120c00700 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55fb6aa73270 0x55fb6aa756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.860+0000 7fe103400700 10 client.26656 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:54:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.860+0000 7fe120c00700 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55fb6aa73270 0x55fb6aa756b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5595da686c50305 server_cookie=7c62c610ffe9bbbf in_seq=0 out_seq=0 2024-05-27T07:54:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.861+0000 7fe103400700 1 -- 172.21.15.90:0/3184205822 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fb6a980d90 con 0x55fb6aa73270 2024-05-27T07:54:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.861+0000 7fe103400700 10 client.26656 handle_command_reply: tid=0 2024-05-27T07:54:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe132b48b80 1 client.26656 shutdown 2024-05-27T07:54:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe132b48b80 1 -- 172.21.15.90:0/3184205822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 msgr2=0x55fb6aa5aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa5aa50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe1100098c0 tx=0x7fe110009610 comp rx=0 tx=0).stop 2024-05-27T07:54:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe132b48b80 1 -- 172.21.15.90:0/3184205822 shutdown_connections 2024-05-27T07:54:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 0x55fb6aa5af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55fb6aa73270 0x55fb6aa756b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa5a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3184205822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa5aa50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe103400700 0 client.26656 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:54:38.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe132b48b80 1 -- 172.21.15.90:0/3184205822 >> 172.21.15.90:0/3184205822 conn(0x55fb6a96b390 msgr2=0x55fb6aa50310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:38.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.862+0000 7fe132b48b80 1 -- 172.21.15.90:0/3184205822 shutdown_connections 2024-05-27T07:54:38.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.863+0000 7fe132b48b80 1 -- 172.21.15.90:0/3184205822 wait complete. 2024-05-27T07:54:38.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.863+0000 7fe132b48b80 20 client.26656 trim_cache size 0 max 16384 2024-05-27T07:54:38.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.896+0000 7fe132b48b80 1 Processor -- start 2024-05-27T07:54:38.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.896+0000 7fe132b48b80 1 -- start start 2024-05-27T07:54:38.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.897+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.897+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa55ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.897+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6aa50e70 0x55fb6aa545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.897+0000 7fe132b48b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc010880 con 0x55fb6a8d1e40 2024-05-27T07:54:38.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.897+0000 7fe132b48b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc00e670 con 0x55fb6aa50e70 2024-05-27T07:54:38.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.897+0000 7fe132b48b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc0101b0 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.897+0000 7fe120c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.898+0000 7fe120c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34146/0 (socket says 172.21.15.90:34146) 2024-05-27T07:54:38.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.898+0000 7fe120c00700 1 -- 172.21.15.90:0/3867177089 learned_addr learned my addr 172.21.15.90:0/3867177089 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:38.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.898+0000 7fe121600700 1 --2- 172.21.15.90:0/3867177089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa55ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.898+0000 7fe120c00700 1 --2- 172.21.15.90:0/3867177089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6aa50e70 0x55fb6aa545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.900+0000 7fe121600700 1 -- 172.21.15.90:0/3867177089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6aa50e70 msgr2=0x55fb6aa545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:38.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.900+0000 7fe121600700 1 --2- 172.21.15.90:0/3867177089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6aa50e70 0x55fb6aa545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.900+0000 7fe121600700 1 -- 172.21.15.90:0/3867177089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6a8d1e40 msgr2=0x55fb6aa594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:38.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.900+0000 7fe121600700 1 --2- 172.21.15.90:0/3867177089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.900+0000 7fe121600700 1 -- 172.21.15.90:0/3867177089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb6aa4d000 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.900+0000 7fe120c00700 1 --2- 172.21.15.90:0/3867177089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6aa50e70 0x55fb6aa545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:38.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.901+0000 7fe121600700 1 --2- 172.21.15.90:0/3867177089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa55ce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ac3238dd79ab09f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:38.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.902+0000 7fe10e000700 1 -- 172.21.15.90:0/3867177089 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110027070 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.902+0000 7fe10e000700 1 -- 172.21.15.90:0/3867177089 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe11000b8a0 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.902+0000 7fe10e000700 1 -- 172.21.15.90:0/3867177089 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110031080 con 0x55fb6a9bd9f0 2024-05-27T07:54:38.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.904+0000 7fe132b48b80 1 -- 172.21.15.90:0/3867177089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 msgr2=0x55fb6aa55ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.904+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3867177089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa55ce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110011000 comp rx=0 tx=0).stop 2024-05-27T07:54:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.904+0000 7fe132b48b80 1 -- 172.21.15.90:0/3867177089 shutdown_connections 2024-05-27T07:54:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.904+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3867177089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6aa50e70 0x55fb6aa545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.904+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3867177089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.904+0000 7fe132b48b80 1 --2- 172.21.15.90:0/3867177089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6a9bd9f0 0x55fb6aa55ce0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.904+0000 7fe132b48b80 1 -- 172.21.15.90:0/3867177089 >> 172.21.15.90:0/3867177089 conn(0x55fb6a96b390 msgr2=0x55fb6aa53ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.904+0000 7fe132b48b80 1 -- 172.21.15.90:0/3867177089 shutdown_connections 2024-05-27T07:54:38.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.904+0000 7fe132b48b80 1 -- 172.21.15.90:0/3867177089 wait complete. 2024-05-27T07:54:38.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.905+0000 7fe132b48b80 1 -- start start 2024-05-27T07:54:38.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.905+0000 7fe132b48b80 1 Processor -- start 2024-05-27T07:54:38.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.907+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa6b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.907+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 0x55fb6aa6bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.907+0000 7fe132b48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa66d50 0x55fb6aa67220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.907+0000 7fe132b48b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc00f410 con 0x55fb6aa50e70 2024-05-27T07:54:38.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.907+0000 7fe132b48b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc00ed40 con 0x55fb6a8d1e40 2024-05-27T07:54:38.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.907+0000 7fe132b48b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc00fae0 con 0x55fb6aa66d50 2024-05-27T07:54:38.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.907+0000 7fe132b48b80 10 client.0 _ll_register_callbacks cb 0x55fb6a918c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:38.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.907+0000 7fe120c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa6b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.907+0000 7fe120c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa6b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49696/0 (socket says 172.21.15.90:49696) 2024-05-27T07:54:38.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.907+0000 7fe120c00700 1 -- 172.21.15.90:0/492655373 learned_addr learned my addr 172.21.15.90:0/492655373 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:38.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.908+0000 7fe120c00700 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa66d50 0x55fb6aa67220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.908+0000 7fe121600700 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 0x55fb6aa6bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.910+0000 7fe120c00700 1 -- 172.21.15.90:0/492655373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa66d50 msgr2=0x55fb6aa67220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:38.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.910+0000 7fe120c00700 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa66d50 0x55fb6aa67220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.910+0000 7fe120c00700 1 -- 172.21.15.90:0/492655373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 msgr2=0x55fb6aa6bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:38.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.910+0000 7fe120c00700 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 0x55fb6aa6bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.910+0000 7fe120c00700 1 -- 172.21.15.90:0/492655373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb6aa67a10 con 0x55fb6a8d1e40 2024-05-27T07:54:38.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.911+0000 7fe103400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:54:38.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.911+0000 7fe120c00700 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa6b9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe108016040 tx=0x7fe108026040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bca5df45e9362aee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:38.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.914+0000 7fe103400700 1 -- 172.21.15.90:0/492655373 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080040f0 con 0x55fb6a8d1e40 2024-05-27T07:54:38.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.914+0000 7fe103400700 1 -- 172.21.15.90:0/492655373 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9496+0+0 (secure 0 0 0) 0x7fe108028e00 con 0x55fb6a8d1e40 2024-05-27T07:54:38.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.915+0000 7fe132b48b80 1 -- 172.21.15.90:0/492655373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe11000b8a0 con 0x55fb6a8d1e40 2024-05-27T07:54:38.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.916+0000 7fe103400700 1 -- 172.21.15.90:0/492655373 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe108004290 con 0x55fb6a8d1e40 2024-05-27T07:54:38.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.916+0000 7fe132b48b80 10 client.36629 fetch_fsmap learned FSMap version 50 2024-05-27T07:54:38.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.916+0000 7fe132b48b80 1 -- 172.21.15.90:0/492655373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55fb6aaddb60 con 0x55fb6a8d1e40 2024-05-27T07:54:38.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.920+0000 7fe103400700 1 -- 172.21.15.90:0/492655373 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe11000b8a0 con 0x55fb6a8d1e40 2024-05-27T07:54:38.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.920+0000 7fe103400700 1 -- 172.21.15.90:0/492655373 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 50) ==== 7757+0+0 (secure 0 0 0) 0x7fe10803e020 con 0x55fb6a8d1e40 2024-05-27T07:54:38.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.920+0000 7fe132b48b80 10 client.36629 fetch_fsmap finished waiting for FSMap version 50 2024-05-27T07:54:38.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.920+0000 7fe132b48b80 10 client.36629 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:54:38.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.920+0000 7fe132b48b80 20 client.36629 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:38.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.920+0000 7fe132b48b80 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55fb6aade270 0x55fb6aae0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:38.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.920+0000 7fe132b48b80 4 client.36629 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:54:38.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.920+0000 7fe132b48b80 1 -- 172.21.15.90:0/492655373 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb6a97a570 con 0x55fb6aade270 2024-05-27T07:54:38.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.921+0000 7fe121600700 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55fb6aade270 0x55fb6aae0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:38.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.922+0000 7fe103400700 10 client.36629 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:54:38.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.922+0000 7fe121600700 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55fb6aade270 0x55fb6aae0690 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aaac907523d05d14 server_cookie=52bc124327b6ebd4 in_seq=0 out_seq=0 2024-05-27T07:54:38.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe103400700 1 -- 172.21.15.90:0/492655373 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb6a97a570 con 0x55fb6aade270 2024-05-27T07:54:38.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe103400700 10 client.36629 handle_command_reply: tid=0 2024-05-27T07:54:38.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe132b48b80 1 client.36629 shutdown 2024-05-27T07:54:38.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe132b48b80 1 -- 172.21.15.90:0/492655373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 msgr2=0x55fb6aa6b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:38.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe132b48b80 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa6b9f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe108016040 tx=0x7fe108026040 comp rx=0 tx=0).stop 2024-05-27T07:54:38.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe132b48b80 1 -- 172.21.15.90:0/492655373 shutdown_connections 2024-05-27T07:54:38.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe132b48b80 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb6aa66d50 0x55fb6aa67220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe132b48b80 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55fb6aade270 0x55fb6aae0690 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe132b48b80 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb6a8d1e40 0x55fb6aa6b9f0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe108016040 tx=0x7fe108026040 comp rx=0 tx=0).stop 2024-05-27T07:54:38.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe132b48b80 1 --2- 172.21.15.90:0/492655373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb6aa50e70 0x55fb6aa6bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe103400700 0 client.36629 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:54:38.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.923+0000 7fe132b48b80 1 -- 172.21.15.90:0/492655373 >> 172.21.15.90:0/492655373 conn(0x55fb6a96b390 msgr2=0x55fb6aa5a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:38.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.925+0000 7fe132b48b80 1 -- 172.21.15.90:0/492655373 shutdown_connections 2024-05-27T07:54:38.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.925+0000 7fe132b48b80 1 -- 172.21.15.90:0/492655373 wait complete. 2024-05-27T07:54:38.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.925+0000 7fe132b48b80 20 client.36629 trim_cache size 0 max 16384 2024-05-27T07:54:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:54:38.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.928+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1040b2af0 msgr2=0x7fe1040b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:38.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.928+0000 7fe124000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1040b2af0 0x7fe1040b5000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1140a53d0 tx=0x7fe114009040 comp rx=0 tx=0).stop 2024-05-27T07:54:38.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.928+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe11c10e900 msgr2=0x7fe11c1a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:38.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.928+0000 7fe124000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe11c10e900 0x7fe11c1a1780 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe11400efc0 tx=0x7fe11400c3a0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.928+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 shutdown_connections 2024-05-27T07:54:38.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.928+0000 7fe124000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe11c10e900 0x7fe11c1a1780 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.928+0000 7fe124000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe1040b2af0 0x7fe1040b5000 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.928+0000 7fe124000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe11c1054d0 0x7fe11c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.928+0000 7fe124000700 1 --2- 172.21.15.90:0/719771307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe11c105e20 0x7fe11c19d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:38.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.929+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 >> 172.21.15.90:0/719771307 conn(0x7fe11c0fedb0 msgr2=0x7fe11c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:38.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.929+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 shutdown_connections 2024-05-27T07:54:38.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:38.929+0000 7fe124000700 1 -- 172.21.15.90:0/719771307 wait complete. 2024-05-27T07:54:40.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:40 smithi090 ceph-mon[31429]: pgmap v260: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.4 MiB/s wr, 733 op/s 2024-05-27T07:54:40.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:40 smithi129 ceph-mon[35254]: pgmap v260: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.4 MiB/s wr, 733 op/s 2024-05-27T07:54:40.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:40 smithi136 ceph-mon[35324]: pgmap v260: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.4 MiB/s wr, 733 op/s 2024-05-27T07:54:41.431 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:54:41.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:41 smithi090 ceph-mon[31429]: pgmap v261: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.2 MiB/s wr, 496 op/s 2024-05-27T07:54:41.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:41 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 4. 2024-05-27T07:54:41.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:41 smithi136 ceph-mon[35324]: pgmap v261: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.2 MiB/s wr, 496 op/s 2024-05-27T07:54:41.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:41 smithi129 ceph-mon[35254]: pgmap v261: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.2 MiB/s wr, 496 op/s 2024-05-27T07:54:42.202 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:42.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:41 smithi090 systemd[1]: Stopped Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:54:42.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:41 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.880s CPU time. 2024-05-27T07:54:42.215 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:41 smithi090 systemd[1]: Starting Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:54:42.531 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:42 smithi090 podman[111814]: 2024-05-27 07:54:42.190236318 +0000 UTC m=+0.066563360 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:54:42.532 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:42 smithi090 podman[111814]: 2024-05-27 07:54:42.317174548 +0000 UTC m=+0.193501607 container create 4d810d32844eb6d420b8d7896e8a9f72fd16c444ab07642f0a46637c759aa3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:54:43.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:43 smithi090 podman[111814]: 2024-05-27 07:54:43.024799645 +0000 UTC m=+0.901126673 container init 4d810d32844eb6d420b8d7896e8a9f72fd16c444ab07642f0a46637c759aa3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:54:43.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:43 smithi090 podman[111814]: 2024-05-27 07:54:43.036910065 +0000 UTC m=+0.913237099 container start 4d810d32844eb6d420b8d7896e8a9f72fd16c444ab07642f0a46637c759aa3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T07:54:43.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:43 smithi090 bash[111814]: 4d810d32844eb6d420b8d7896e8a9f72fd16c444ab07642f0a46637c759aa3ae 2024-05-27T07:54:43.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:43 smithi090 systemd[1]: Started Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:54:43.545 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:43 smithi090 ceph-mds[111860]: -- 172.21.15.90:0/2153144053 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5592299bba40 con 0x559229a03180 2024-05-27T07:54:43.545 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:43 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: starting mds.d at 2024-05-27T07:54:43.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 265.0 s, 542.4 tps, lat 186.293 ms stddev 126.401 2024-05-27T07:54:44.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:44 smithi136 ceph-mon[35324]: pgmap v262: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-05-27T07:54:44.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:44 smithi136 ceph-mon[35324]: daemon mds.d restarted 2024-05-27T07:54:44.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:44 smithi129 ceph-mon[35254]: pgmap v262: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-05-27T07:54:44.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:44 smithi129 ceph-mon[35254]: daemon mds.d restarted 2024-05-27T07:54:44.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:44 smithi090 ceph-mon[31429]: pgmap v262: 129 pgs: 129 active+clean; 2.6 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-05-27T07:54:44.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:44 smithi090 ceph-mon[31429]: daemon mds.d restarted 2024-05-27T07:54:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc813a00700 time 2024-05-27T07:54:44.369899+0000 2024-05-27T07:54:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:54:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc829544ba6] 2024-05-27T07:54:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc829544dd1] 2024-05-27T07:54:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5591fb1c823d] 2024-05-27T07:54:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5591fb1b3971] 2024-05-27T07:54:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 5: (SnapRealm::check_cache() const+0x13e) [0x5591fb1b1fa0] 2024-05-27T07:54:44.715 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5591fb1b2651] 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5591fb147347] 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5591fae4387e] 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5591fae48f2f] 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5591fae53969] 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 11: (MDLog::_replay_thread()+0x1f38) [0x5591fb1ec48c] 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5591fb1f59cb] 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 13: (Thread::entry_wrapper()+0x43) [0x7fc82951372d] 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc829513749] 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc8280b71ca] 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 16: clone() 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: *** Caught signal (Aborted) ** 2024-05-27T07:54:44.716 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: in thread 7fc813a00700 thread_name:md_log_replay 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.374+0000 7fc813a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc813a00700 time 2024-05-27T07:54:44.369899+0000 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc829544ba6] 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc829544dd1] 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5591fb1c823d] 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5591fb1b3971] 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 5: (SnapRealm::check_cache() const+0x13e) [0x5591fb1b1fa0] 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5591fb1b2651] 2024-05-27T07:54:44.717 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5591fb147347] 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5591fae4387e] 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5591fae48f2f] 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5591fae53969] 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 11: (MDLog::_replay_thread()+0x1f38) [0x5591fb1ec48c] 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5591fb1f59cb] 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 13: (Thread::entry_wrapper()+0x43) [0x7fc82951372d] 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc829513749] 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc8280b71ca] 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 16: clone() 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc8280c1d20] 2024-05-27T07:54:44.718 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2: gsignal() 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 3: abort() 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc829544ced] 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc829544dd1] 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5591fb1c823d] 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5591fb1b3971] 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 8: (SnapRealm::check_cache() const+0x13e) [0x5591fb1b1fa0] 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5591fb1b2651] 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5591fb147347] 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5591fae4387e] 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5591fae48f2f] 2024-05-27T07:54:44.719 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5591fae53969] 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 14: (MDLog::_replay_thread()+0x1f38) [0x5591fb1ec48c] 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5591fb1f59cb] 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 16: (Thread::entry_wrapper()+0x43) [0x7fc82951372d] 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc829513749] 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc8280b71ca] 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 19: clone() 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.379+0000 7fc813a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: in thread 7fc813a00700 thread_name:md_log_replay 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc8280c1d20] 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2: gsignal() 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 3: abort() 2024-05-27T07:54:44.720 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc829544ced] 2024-05-27T07:54:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc829544dd1] 2024-05-27T07:54:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5591fb1c823d] 2024-05-27T07:54:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5591fb1b3971] 2024-05-27T07:54:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 8: (SnapRealm::check_cache() const+0x13e) [0x5591fb1b1fa0] 2024-05-27T07:54:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5591fb1b2651] 2024-05-27T07:54:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5591fb147347] 2024-05-27T07:54:44.721 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5591fae4387e] 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5591fae48f2f] 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5591fae53969] 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 14: (MDLog::_replay_thread()+0x1f38) [0x5591fb1ec48c] 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5591fb1f59cb] 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 16: (Thread::entry_wrapper()+0x43) [0x7fc82951372d] 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc829513749] 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc8280b71ca] 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 19: clone() 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: -1> 2024-05-27T07:54:44.374+0000 7fc813a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc813a00700 time 2024-05-27T07:54:44.369899+0000 2024-05-27T07:54:44.722 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc829544ba6] 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc829544dd1] 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5591fb1c823d] 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5591fb1b3971] 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 5: (SnapRealm::check_cache() const+0x13e) [0x5591fb1b1fa0] 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5591fb1b2651] 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5591fb147347] 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5591fae4387e] 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5591fae48f2f] 2024-05-27T07:54:44.723 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5591fae53969] 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 11: (MDLog::_replay_thread()+0x1f38) [0x5591fb1ec48c] 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5591fb1f59cb] 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 13: (Thread::entry_wrapper()+0x43) [0x7fc82951372d] 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc829513749] 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc8280b71ca] 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 16: clone() 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 0> 2024-05-27T07:54:44.379+0000 7fc813a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: in thread 7fc813a00700 thread_name:md_log_replay 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc8280c1d20] 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2: gsignal() 2024-05-27T07:54:44.724 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 3: abort() 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc829544ced] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc829544dd1] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5591fb1c823d] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5591fb1b3971] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 8: (SnapRealm::check_cache() const+0x13e) [0x5591fb1b1fa0] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5591fb1b2651] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5591fb147347] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5591fae4387e] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5591fae48f2f] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5591fae53969] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 14: (MDLog::_replay_thread()+0x1f38) [0x5591fb1ec48c] 2024-05-27T07:54:44.725 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5591fb1f59cb] 2024-05-27T07:54:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 16: (Thread::entry_wrapper()+0x43) [0x7fc82951372d] 2024-05-27T07:54:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc829513749] 2024-05-27T07:54:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc8280b71ca] 2024-05-27T07:54:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 19: clone() 2024-05-27T07:54:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:54:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: -1> 2024-05-27T07:54:44.374+0000 7fc813a00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fc813a00700 time 2024-05-27T07:54:44.369899+0000 2024-05-27T07:54:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:54:44.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fc829544ba6] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc829544dd1] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5591fb1c823d] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 4: (SnapRealm::build_snap_set() const+0x323) [0x5591fb1b3971] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 5: (SnapRealm::check_cache() const+0x13e) [0x5591fb1b1fa0] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5591fb1b2651] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5591fb147347] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5591fae4387e] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5591fae48f2f] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x5591fae53969] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 11: (MDLog::_replay_thread()+0x1f38) [0x5591fb1ec48c] 2024-05-27T07:54:44.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5591fb1f59cb] 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 13: (Thread::entry_wrapper()+0x43) [0x7fc82951372d] 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 14: (Thread::_entry_func(void*)+0xd) [0x7fc829513749] 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc8280b71ca] 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 16: clone() 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 0> 2024-05-27T07:54:44.379+0000 7fc813a00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: in thread 7fc813a00700 thread_name:md_log_replay 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc8280c1d20] 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2: gsignal() 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 3: abort() 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fc829544ced] 2024-05-27T07:54:44.728 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fc829544dd1] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5591fb1c823d] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 7: (SnapRealm::build_snap_set() const+0x323) [0x5591fb1b3971] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 8: (SnapRealm::check_cache() const+0x13e) [0x5591fb1b1fa0] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x5591fb1b2651] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x5591fb147347] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x5591fae4387e] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x5591fae48f2f] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x5591fae53969] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 14: (MDLog::_replay_thread()+0x1f38) [0x5591fb1ec48c] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5591fb1f59cb] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 16: (Thread::entry_wrapper()+0x43) [0x7fc82951372d] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 17: (Thread::_entry_func(void*)+0xd) [0x7fc829513749] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fc8280b71ca] 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 19: clone() 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:54:44.729 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[111848]: 2024-05-27T07:54:45.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:45 smithi136 ceph-mon[35324]: osdmap e104: 12 total, 12 up, 12 in 2024-05-27T07:54:45.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:45 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6834/1262132318,v1:172.21.15.90:6835/1262132318] up:boot 2024-05-27T07:54:45.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:45 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:54:45.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:45 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:54:45.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:45 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:54:45.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:45 smithi129 ceph-mon[35254]: osdmap e104: 12 total, 12 up, 12 in 2024-05-27T07:54:45.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:45 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6834/1262132318,v1:172.21.15.90:6835/1262132318] up:boot 2024-05-27T07:54:45.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:45 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:54:45.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:45 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:54:45.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:45 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:54:45.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:45 smithi090 ceph-mon[31429]: osdmap e104: 12 total, 12 up, 12 in 2024-05-27T07:54:45.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:45 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6834/1262132318,v1:172.21.15.90:6835/1262132318] up:boot 2024-05-27T07:54:45.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:45 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-05-27T07:54:45.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:45 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:54:45.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:45 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:54:46.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:46 smithi136 ceph-mon[35324]: pgmap v264: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.6 MiB/s wr, 626 op/s 2024-05-27T07:54:46.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:46 smithi129 ceph-mon[35254]: pgmap v264: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.6 MiB/s wr, 626 op/s 2024-05-27T07:54:46.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:46 smithi090 ceph-mon[31429]: pgmap v264: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.6 MiB/s wr, 626 op/s 2024-05-27T07:54:48.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:48 smithi136 ceph-mon[35324]: pgmap v265: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.6 MiB/s wr, 626 op/s 2024-05-27T07:54:48.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:48 smithi129 ceph-mon[35254]: pgmap v265: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.6 MiB/s wr, 626 op/s 2024-05-27T07:54:48.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:48 smithi090 ceph-mon[31429]: pgmap v265: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.6 MiB/s wr, 626 op/s 2024-05-27T07:54:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 270.0 s, 355.6 tps, lat 262.450 ms stddev 197.138 2024-05-27T07:54:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.865+0000 7fcfcb600640 1 -- 172.21.15.90:0/2995010252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc4108000 msgr2=0x7fcfc4108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.865+0000 7fcfcb600640 1 --2- 172.21.15.90:0/2995010252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc4108000 0x7fcfc4108490 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009be0 tx=0x7fcfbc02a6d0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.866+0000 7fcfcb600640 1 -- 172.21.15.90:0/2995010252 shutdown_connections 2024-05-27T07:54:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.866+0000 7fcfcb600640 1 --2- 172.21.15.90:0/2995010252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc4108000 0x7fcfc4108490 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.866+0000 7fcfcb600640 1 --2- 172.21.15.90:0/2995010252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfc41075d0 0x7fcfc41079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.866+0000 7fcfcb600640 1 --2- 172.21.15.90:0/2995010252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc4102b80 0x7fcfc4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.866+0000 7fcfcb600640 1 -- 172.21.15.90:0/2995010252 >> 172.21.15.90:0/2995010252 conn(0x7fcfc40782d0 msgr2=0x7fcfc4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.868+0000 7fcfcb600640 1 -- 172.21.15.90:0/2995010252 shutdown_connections 2024-05-27T07:54:48.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.868+0000 7fcfcb600640 1 -- 172.21.15.90:0/2995010252 wait complete. 2024-05-27T07:54:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.871+0000 7fcfcb600640 1 Processor -- start 2024-05-27T07:54:48.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.872+0000 7fcfcb600640 1 -- start start 2024-05-27T07:54:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.872+0000 7fcfcb600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc4102b80 0x7fcfc4116e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.872+0000 7fcfcb600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfc41075d0 0x7fcfc410ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.872+0000 7fcfcb600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc4108000 0x7fcfc41106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.872+0000 7fcfcb600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc411a2c0 con 0x7fcfc4102b80 2024-05-27T07:54:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.872+0000 7fcfcb600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc411a140 con 0x7fcfc4108000 2024-05-27T07:54:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.872+0000 7fcfcb600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc411a440 con 0x7fcfc41075d0 2024-05-27T07:54:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.873+0000 7fcfc8c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc4102b80 0x7fcfc4116e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.873+0000 7fcfc9600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc4108000 0x7fcfc41106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.873+0000 7fcfc8c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc4102b80 0x7fcfc4116e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56180/0 (socket says 172.21.15.90:56180) 2024-05-27T07:54:48.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.873+0000 7fcfc8c00640 1 -- 172.21.15.90:0/4093653001 learned_addr learned my addr 172.21.15.90:0/4093653001 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.874+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfc41075d0 0x7fcfc410ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.874+0000 7fcfc9600640 1 -- 172.21.15.90:0/4093653001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfc41075d0 msgr2=0x7fcfc410ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.874+0000 7fcfc9600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfc41075d0 0x7fcfc410ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.874+0000 7fcfc9600640 1 -- 172.21.15.90:0/4093653001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc4102b80 msgr2=0x7fcfc4116e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.874+0000 7fcfc9600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc4102b80 0x7fcfc4116e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.874+0000 7fcfc9600640 1 -- 172.21.15.90:0/4093653001 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc4114980 con 0x7fcfc4108000 2024-05-27T07:54:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.875+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfc41075d0 0x7fcfc410ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.875+0000 7fcfc8c00640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc4102b80 0x7fcfc4116e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.877+0000 7fcfc9600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc4108000 0x7fcfc41106a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009b30 tx=0x7fcfbc030d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.878+0000 7fcfc1600640 1 -- 172.21.15.90:0/4093653001 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc041020 con 0x7fcfc4108000 2024-05-27T07:54:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.878+0000 7fcfc1600640 1 -- 172.21.15.90:0/4093653001 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfbc037040 con 0x7fcfc4108000 2024-05-27T07:54:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.878+0000 7fcfc1600640 1 -- 172.21.15.90:0/4093653001 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc036070 con 0x7fcfc4108000 2024-05-27T07:54:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.878+0000 7fcfcb600640 1 -- 172.21.15.90:0/4093653001 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc4114c10 con 0x7fcfc4108000 2024-05-27T07:54:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.878+0000 7fcfcb600640 1 -- 172.21.15.90:0/4093653001 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc4115080 con 0x7fcfc4108000 2024-05-27T07:54:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.882+0000 7fcfd258d740 1 Processor -- start 2024-05-27T07:54:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.885+0000 7fcfc1600640 1 -- 172.21.15.90:0/4093653001 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcfbc044020 con 0x7fcfc4108000 2024-05-27T07:54:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.886+0000 7fcfd258d740 1 -- start start 2024-05-27T07:54:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.887+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f6f5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.887+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f32a2f0 0x557a0f6f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.887+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f6f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.887+0000 7fcfd258d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557a0f666b90 con 0x557a0f3531b0 2024-05-27T07:54:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.887+0000 7fcfd258d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557a0f656730 con 0x557a0f32a2f0 2024-05-27T07:54:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.887+0000 7fcfd258d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557a0f65a770 con 0x557a0f3324d0 2024-05-27T07:54:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.888+0000 7fcfc1600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcfac0b2a50 0x7fcfac0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.888+0000 7fcfc3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f6f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.888+0000 7fcfc8c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f32a2f0 0x557a0f6f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.888+0000 7fcfc3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f6f5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56196/0 (socket says 172.21.15.90:56196) 2024-05-27T07:54:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.888+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2205361349 learned_addr learned my addr 172.21.15.90:0/2205361349 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.888+0000 7fcfc1600640 1 -- 172.21.15.90:0/4093653001 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7fcfbc103170 con 0x7fcfc4108000 2024-05-27T07:54:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.888+0000 7fcfc9600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcfac0b2a50 0x7fcfac0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.888+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2205361349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f6f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.889+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2205361349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 msgr2=0x557a0f6f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.889+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2205361349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f6f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.889+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2205361349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f32a2f0 msgr2=0x557a0f6f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.889+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2205361349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f32a2f0 0x557a0f6f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.889+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2205361349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a0f5ef8b0 con 0x557a0f3531b0 2024-05-27T07:54:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.889+0000 7fcfc9600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcfac0b2a50 0x7fcfac0b4f60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0c6230 tx=0x7fcfbc1033c0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.892+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2205361349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f6f5860 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8012790 tx=0x7fcfb8009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275c9f6e465c0b78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.893+0000 7fcfb1c00640 1 -- 172.21.15.90:0/2205361349 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8012d10 con 0x557a0f3531b0 2024-05-27T07:54:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.893+0000 7fcfb1c00640 1 -- 172.21.15.90:0/2205361349 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfb8010c60 con 0x557a0f3531b0 2024-05-27T07:54:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.893+0000 7fcfb1c00640 1 -- 172.21.15.90:0/2205361349 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb801ea20 con 0x557a0f3531b0 2024-05-27T07:54:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.896+0000 7fcfd258d740 1 -- 172.21.15.90:0/2205361349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 msgr2=0x557a0f6f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.896+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2205361349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f6f5860 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8012790 tx=0x7fcfb8009690 comp rx=0 tx=0).stop 2024-05-27T07:54:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.897+0000 7fcfd258d740 1 -- 172.21.15.90:0/2205361349 shutdown_connections 2024-05-27T07:54:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.897+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2205361349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f32a2f0 0x557a0f6f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.897+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2205361349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f6f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.897+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2205361349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f6f5860 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.898+0000 7fcfd258d740 1 -- 172.21.15.90:0/2205361349 >> 172.21.15.90:0/2205361349 conn(0x557a0f347300 msgr2=0x557a0f6f2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.898+0000 7fcfd258d740 1 -- 172.21.15.90:0/2205361349 shutdown_connections 2024-05-27T07:54:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.898+0000 7fcfd258d740 1 -- 172.21.15.90:0/2205361349 wait complete. 2024-05-27T07:54:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.899+0000 7fcfd258d740 1 -- start start 2024-05-27T07:54:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.901+0000 7fcfd258d740 1 Processor -- start 2024-05-27T07:54:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.902+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.903+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f3324d0 0x557a0f6fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.903+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3531b0 0x557a0f70bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.903+0000 7fcfd258d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557a0f33c790 con 0x557a0f32a2f0 2024-05-27T07:54:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.903+0000 7fcfd258d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557a0f5febc0 con 0x557a0f3324d0 2024-05-27T07:54:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.903+0000 7fcfd258d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557a0f702990 con 0x557a0f3531b0 2024-05-27T07:54:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.903+0000 7fcfd258d740 10 client.0 _ll_register_callbacks cb 0x557a0f5dc400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.904+0000 7fcfc3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.904+0000 7fcfc3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56208/0 (socket says 172.21.15.90:56208) 2024-05-27T07:54:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.904+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2483366145 learned_addr learned my addr 172.21.15.90:0/2483366145 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.904+0000 7fcfc8c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f3324d0 0x557a0f6fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.905+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3531b0 0x557a0f70bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.905+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2483366145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3531b0 msgr2=0x557a0f70bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.905+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3531b0 0x557a0f70bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.905+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2483366145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f3324d0 msgr2=0x557a0f6fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.905+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f3324d0 0x557a0f6fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.905+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2483366145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a0f36b210 con 0x557a0f32a2f0 2024-05-27T07:54:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.906+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6ffa20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80095e0 tx=0x7fcfb8009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959733679d07c7f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.906+0000 7fcfa6000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:54:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.906+0000 7fcfa6000640 1 -- 172.21.15.90:0/2483366145 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8005240 con 0x557a0f32a2f0 2024-05-27T07:54:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.906+0000 7fcfa6000640 1 -- 172.21.15.90:0/2483366145 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7fcfb80053e0 con 0x557a0f32a2f0 2024-05-27T07:54:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.906+0000 7fcfd258d740 1 -- 172.21.15.90:0/2483366145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb8010c60 con 0x557a0f32a2f0 2024-05-27T07:54:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.907+0000 7fcfa6000640 1 -- 172.21.15.90:0/2483366145 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fcfb8030050 con 0x557a0f32a2f0 2024-05-27T07:54:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.907+0000 7fcfd258d740 10 client.17073 fetch_fsmap learned FSMap version 52 2024-05-27T07:54:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.907+0000 7fcfd258d740 1 -- 172.21.15.90:0/2483366145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x557a0f437620 con 0x557a0f32a2f0 2024-05-27T07:54:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.908+0000 7fcfa6000640 1 -- 172.21.15.90:0/2483366145 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfb8010c60 con 0x557a0f32a2f0 2024-05-27T07:54:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.908+0000 7fcfa6000640 1 -- 172.21.15.90:0/2483366145 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 52) ==== 7757+0+0 (secure 0 0 0) 0x7fcfb8004900 con 0x557a0f32a2f0 2024-05-27T07:54:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.908+0000 7fcfd258d740 10 client.17073 fetch_fsmap finished waiting for FSMap version 52 2024-05-27T07:54:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.908+0000 7fcfd258d740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.908+0000 7fcfd258d740 20 client.17073 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.908+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a0f311d80 0x557a0f711b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.909+0000 7fcfd258d740 4 client.17073 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.909+0000 7fcfd258d740 1 -- 172.21.15.90:0/2483366145 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a0f668ae0 con 0x557a0f311d80 2024-05-27T07:54:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.910+0000 7fcfc8c00640 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a0f311d80 0x557a0f711b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.912+0000 7fcfc8c00640 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a0f311d80 0x557a0f711b60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab9d130cfc29e5fb server_cookie=7c63affff7e2a7d6 in_seq=0 out_seq=0 2024-05-27T07:54:48.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.912+0000 7fcfa6000640 10 client.17073 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.919+0000 7fcfa6000640 1 -- 172.21.15.90:0/2483366145 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557a0f668ae0 con 0x557a0f311d80 2024-05-27T07:54:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.919+0000 7fcfa6000640 10 client.17073 handle_command_reply: tid=0 2024-05-27T07:54:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.919+0000 7fcfd258d740 1 client.17073 shutdown 2024-05-27T07:54:48.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.920+0000 7fcfd258d740 1 -- 172.21.15.90:0/2483366145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 msgr2=0x557a0f6ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.920+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6ffa20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80095e0 tx=0x7fcfb8009530 comp rx=0 tx=0).stop 2024-05-27T07:54:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.921+0000 7fcfd258d740 1 -- 172.21.15.90:0/2483366145 shutdown_connections 2024-05-27T07:54:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.921+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a0f311d80 0x557a0f711b60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.921+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f3324d0 0x557a0f6fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.921+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3531b0 0x557a0f70bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.921+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2483366145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6ffa20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.921+0000 7fcfd258d740 1 -- 172.21.15.90:0/2483366145 >> 172.21.15.90:0/2483366145 conn(0x557a0f347300 msgr2=0x557a0f6f5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.921+0000 7fcfa6000640 0 client.17073 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.922+0000 7fcfd258d740 1 -- 172.21.15.90:0/2483366145 shutdown_connections 2024-05-27T07:54:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.922+0000 7fcfd258d740 1 -- 172.21.15.90:0/2483366145 wait complete. 2024-05-27T07:54:48.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.922+0000 7fcfd258d740 20 client.17073 trim_cache size 0 max 16384 2024-05-27T07:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.951+0000 7fcfd258d740 1 Processor -- start 2024-05-27T07:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.951+0000 7fcfd258d740 1 -- start start 2024-05-27T07:54:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.952+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6fb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.952+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f6f1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.952+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f3531b0 0x557a0f6ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.952+0000 7fcfd258d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41c1a00 con 0x557a0f32a2f0 2024-05-27T07:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.952+0000 7fcfd258d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41bf7f0 con 0x557a0f3531b0 2024-05-27T07:54:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.952+0000 7fcfd258d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41c1330 con 0x557a0f3324d0 2024-05-27T07:54:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.953+0000 7fcfc3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.953+0000 7fcfc3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6fb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56220/0 (socket says 172.21.15.90:56220) 2024-05-27T07:54:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.953+0000 7fcfc3e00640 1 -- 172.21.15.90:0/909436793 learned_addr learned my addr 172.21.15.90:0/909436793 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.953+0000 7fcfc8c00640 1 --2- 172.21.15.90:0/909436793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f6f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.954+0000 7fcfc3e00640 1 -- 172.21.15.90:0/909436793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 msgr2=0x557a0f6f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.954+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/909436793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f6f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.954+0000 7fcfc3e00640 1 -- 172.21.15.90:0/909436793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f3531b0 msgr2=0x557a0f6ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.954+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/909436793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f3531b0 0x557a0f6ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.954+0000 7fcfc3e00640 1 -- 172.21.15.90:0/909436793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a0f6f6c40 con 0x557a0f32a2f0 2024-05-27T07:54:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.954+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/909436793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6fb240 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8005910 tx=0x7fcfb801f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b354ea52ca76fa22 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.955+0000 7fcfb1200640 1 -- 172.21.15.90:0/909436793 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8010c60 con 0x557a0f32a2f0 2024-05-27T07:54:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.955+0000 7fcfb1200640 1 -- 172.21.15.90:0/909436793 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfb8025040 con 0x557a0f32a2f0 2024-05-27T07:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.956+0000 7fcfb1200640 1 -- 172.21.15.90:0/909436793 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb801f990 con 0x557a0f32a2f0 2024-05-27T07:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.957+0000 7fcfd258d740 1 -- 172.21.15.90:0/909436793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 msgr2=0x557a0f6fb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.957+0000 7fcfd258d740 1 --2- 172.21.15.90:0/909436793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6fb240 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8005910 tx=0x7fcfb801f540 comp rx=0 tx=0).stop 2024-05-27T07:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.957+0000 7fcfd258d740 1 -- 172.21.15.90:0/909436793 shutdown_connections 2024-05-27T07:54:48.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.957+0000 7fcfd258d740 1 --2- 172.21.15.90:0/909436793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f3531b0 0x557a0f6ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.957+0000 7fcfd258d740 1 --2- 172.21.15.90:0/909436793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f6f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.957+0000 7fcfd258d740 1 --2- 172.21.15.90:0/909436793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f32a2f0 0x557a0f6fb240 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.957+0000 7fcfd258d740 1 -- 172.21.15.90:0/909436793 >> 172.21.15.90:0/909436793 conn(0x557a0f347300 msgr2=0x557a0f6ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.958+0000 7fcfd258d740 1 -- 172.21.15.90:0/909436793 shutdown_connections 2024-05-27T07:54:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.958+0000 7fcfd258d740 1 -- 172.21.15.90:0/909436793 wait complete. 2024-05-27T07:54:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.959+0000 7fcfd258d740 1 -- start start 2024-05-27T07:54:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.959+0000 7fcfd258d740 1 Processor -- start 2024-05-27T07:54:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.960+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f32a2f0 0x557a0f702cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.960+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f703230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.960+0000 7fcfd258d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f778050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.960+0000 7fcfd258d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41c0590 con 0x557a0f3531b0 2024-05-27T07:54:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.960+0000 7fcfd258d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41bfec0 con 0x557a0f32a2f0 2024-05-27T07:54:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.960+0000 7fcfd258d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc41c0c60 con 0x557a0f3324d0 2024-05-27T07:54:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.960+0000 7fcfd258d740 10 client.0 _ll_register_callbacks cb 0x7fcfb4009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f778050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f778050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56224/0 (socket says 172.21.15.90:56224) 2024-05-27T07:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2556224611 learned_addr learned my addr 172.21.15.90:0/2556224611 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f32a2f0 0x557a0f702cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc8c00640 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f703230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2556224611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 msgr2=0x557a0f703230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f703230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2556224611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f32a2f0 msgr2=0x557a0f702cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f32a2f0 0x557a0f702cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc3e00640 1 -- 172.21.15.90:0/2556224611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a0f703a50 con 0x557a0f3531b0 2024-05-27T07:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.961+0000 7fcfc8c00640 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f703230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.962+0000 7fcfc3e00640 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f778050 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8005000 tx=0x7fcfb8009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f022a52d24978ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.962+0000 7fcfa6000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.962+0000 7fcfa6000640 1 -- 172.21.15.90:0/2556224611 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8005240 con 0x557a0f3531b0 2024-05-27T07:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.962+0000 7fcfa6000640 1 -- 172.21.15.90:0/2556224611 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7fcfb80053e0 con 0x557a0f3531b0 2024-05-27T07:54:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.963+0000 7fcfd258d740 1 -- 172.21.15.90:0/2556224611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb8025040 con 0x557a0f3531b0 2024-05-27T07:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.963+0000 7fcfa6000640 1 -- 172.21.15.90:0/2556224611 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fcfb80055e0 con 0x557a0f3531b0 2024-05-27T07:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.964+0000 7fcfd258d740 10 client.17079 fetch_fsmap learned FSMap version 52 2024-05-27T07:54:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.964+0000 7fcfd258d740 1 -- 172.21.15.90:0/2556224611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x557a0f78a580 con 0x557a0f3531b0 2024-05-27T07:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.964+0000 7fcfa6000640 1 -- 172.21.15.90:0/2556224611 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcfb8025040 con 0x557a0f3531b0 2024-05-27T07:54:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.965+0000 7fcfa6000640 1 -- 172.21.15.90:0/2556224611 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 52) ==== 7757+0+0 (secure 0 0 0) 0x7fcfb802c510 con 0x557a0f3531b0 2024-05-27T07:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.965+0000 7fcfd258d740 10 client.17079 fetch_fsmap finished waiting for FSMap version 52 2024-05-27T07:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.965+0000 7fcfd258d740 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.965+0000 7fcfd258d740 20 client.17079 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.965+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a0f2e71c0 0x557a0f78cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.965+0000 7fcfd258d740 4 client.17079 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:54:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.965+0000 7fcfd258d740 1 -- 172.21.15.90:0/2556224611 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a0f6622c0 con 0x557a0f2e71c0 2024-05-27T07:54:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.967+0000 7fcfc8c00640 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a0f2e71c0 0x557a0f78cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.968+0000 7fcfc8c00640 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a0f2e71c0 0x557a0f78cbc0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f413efe6418a18a1 server_cookie=a88a29ef6b6ab6b in_seq=0 out_seq=0 2024-05-27T07:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:48.968+0000 7fcfa6000640 10 client.17079 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:49.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.013+0000 7fcfa6000640 1 -- 172.21.15.90:0/2556224611 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557a0f6622c0 con 0x557a0f2e71c0 2024-05-27T07:54:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.013+0000 7fcfa6000640 10 client.17079 handle_command_reply: tid=0 2024-05-27T07:54:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.013+0000 7fcfd258d740 1 client.17079 shutdown 2024-05-27T07:54:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.014+0000 7fcfd258d740 1 -- 172.21.15.90:0/2556224611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 msgr2=0x557a0f778050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:49.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.014+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f778050 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8005000 tx=0x7fcfb8009fd0 comp rx=0 tx=0).stop 2024-05-27T07:54:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.014+0000 7fcfd258d740 1 -- 172.21.15.90:0/2556224611 shutdown_connections 2024-05-27T07:54:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.014+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557a0f2e71c0 0x557a0f78cbc0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.014+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557a0f3531b0 0x557a0f778050 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.015+0000 7fcfa6000640 0 client.17079 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.014+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557a0f3324d0 0x557a0f703230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.015+0000 7fcfd258d740 1 --2- 172.21.15.90:0/2556224611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557a0f32a2f0 0x557a0f702cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.015+0000 7fcfd258d740 1 -- 172.21.15.90:0/2556224611 >> 172.21.15.90:0/2556224611 conn(0x557a0f347300 msgr2=0x557a0f6ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.016+0000 7fcfd258d740 1 -- 172.21.15.90:0/2556224611 shutdown_connections 2024-05-27T07:54:49.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.016+0000 7fcfd258d740 1 -- 172.21.15.90:0/2556224611 wait complete. 2024-05-27T07:54:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.017+0000 7fcfd258d740 20 client.17079 trim_cache size 0 max 16384 2024-05-27T07:54:49.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:54:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:54:49.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:54:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.018+0000 7fcfcb600640 1 -- 172.21.15.90:0/4093653001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcfac0b2a50 msgr2=0x7fcfac0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.018+0000 7fcfcb600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcfac0b2a50 0x7fcfac0b4f60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0c6230 tx=0x7fcfbc1033c0 comp rx=0 tx=0).stop 2024-05-27T07:54:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.018+0000 7fcfcb600640 1 -- 172.21.15.90:0/4093653001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc4108000 msgr2=0x7fcfc41106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.018+0000 7fcfcb600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc4108000 0x7fcfc41106a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009b30 tx=0x7fcfbc030d50 comp rx=0 tx=0).stop 2024-05-27T07:54:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.019+0000 7fcfcb600640 1 -- 172.21.15.90:0/4093653001 shutdown_connections 2024-05-27T07:54:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.019+0000 7fcfcb600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fcfac0b2a50 0x7fcfac0b4f60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.019+0000 7fcfcb600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfc4108000 0x7fcfc41106a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.019+0000 7fcfcb600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfc41075d0 0x7fcfc410ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.019+0000 7fcfcb600640 1 --2- 172.21.15.90:0/4093653001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc4102b80 0x7fcfc4116e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.019+0000 7fcfcb600640 1 -- 172.21.15.90:0/4093653001 >> 172.21.15.90:0/4093653001 conn(0x7fcfc40782d0 msgr2=0x7fcfc41051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:49.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.019+0000 7fcfcb600640 1 -- 172.21.15.90:0/4093653001 shutdown_connections 2024-05-27T07:54:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:54:49.020+0000 7fcfcb600640 1 -- 172.21.15.90:0/4093653001 wait complete. 2024-05-27T07:54:49.963 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:49 smithi090 podman[111910]: 2024-05-27 07:54:49.680971819 +0000 UTC m=+5.211157822 container died 4d810d32844eb6d420b8d7896e8a9f72fd16c444ab07642f0a46637c759aa3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T07:54:50.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:50 smithi090 ceph-mon[31429]: pgmap v266: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2024-05-27T07:54:50.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:50 smithi136 ceph-mon[35324]: pgmap v266: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2024-05-27T07:54:50.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:50 smithi129 ceph-mon[35254]: pgmap v266: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2024-05-27T07:54:50.856 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:50 smithi090 podman[111910]: 2024-05-27 07:54:50.485641939 +0000 UTC m=+6.015827936 container remove 4d810d32844eb6d420b8d7896e8a9f72fd16c444ab07642f0a46637c759aa3ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-27T07:54:50.856 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:50 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:54:51.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.524+0000 7f18fca00700 1 -- 172.21.15.90:0/2922063500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f8078630 msgr2=0x7f18f8114ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:51.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.524+0000 7f18fca00700 1 --2- 172.21.15.90:0/2922063500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f8078630 0x7f18f8114ae0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009990 tx=0x7f18e0009ca0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.524+0000 7f18fca00700 1 -- 172.21.15.90:0/2922063500 shutdown_connections 2024-05-27T07:54:51.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.524+0000 7f18fca00700 1 --2- 172.21.15.90:0/2922063500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f8115020 0x7f18f8117440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.524+0000 7f18fca00700 1 --2- 172.21.15.90:0/2922063500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18f8077ce0 0x7f18f80780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.524+0000 7f18fca00700 1 --2- 172.21.15.90:0/2922063500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f8078630 0x7f18f8114ae0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.524+0000 7f18fca00700 1 -- 172.21.15.90:0/2922063500 >> 172.21.15.90:0/2922063500 conn(0x7f18f8102020 msgr2=0x7f18f8104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:51.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.525+0000 7f18fca00700 1 -- 172.21.15.90:0/2922063500 shutdown_connections 2024-05-27T07:54:51.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.527+0000 7f18fca00700 1 -- 172.21.15.90:0/2922063500 wait complete. 2024-05-27T07:54:51.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.529+0000 7f18fca00700 1 Processor -- start 2024-05-27T07:54:51.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.529+0000 7f18fca00700 1 -- start start 2024-05-27T07:54:51.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.531+0000 7f18fca00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f8077ce0 0x7f18f8075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.531+0000 7f18fca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f8078630 0x7f18f806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.531+0000 7f18fca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18f8115020 0x7f18f806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.532+0000 7f18fca00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f18f811a880 con 0x7f18f8078630 2024-05-27T07:54:51.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.532+0000 7f18fca00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18f811a700 con 0x7f18f8115020 2024-05-27T07:54:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.532+0000 7f18fca00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18f811aa00 con 0x7f18f8077ce0 2024-05-27T07:54:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.532+0000 7f18f7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f8077ce0 0x7f18f8075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.532+0000 7f18f7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18f8115020 0x7f18f806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.532+0000 7f18f7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18f8115020 0x7f18f806eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47228/0 (socket says 172.21.15.90:47228) 2024-05-27T07:54:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.532+0000 7f18f7e00700 1 -- 172.21.15.90:0/2367835270 learned_addr learned my addr 172.21.15.90:0/2367835270 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.533+0000 7f18f7e00700 1 -- 172.21.15.90:0/2367835270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f8077ce0 msgr2=0x7f18f8075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:51.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.534+0000 7f18eea00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f8078630 0x7f18f806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.535+0000 7f18f7e00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f8077ce0 0x7f18f8075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.535+0000 7f18f7e00700 1 -- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f8078630 msgr2=0x7f18f806e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.535+0000 7f18f7e00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f8078630 0x7f18f806e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.535+0000 7f18f7e00700 1 -- 172.21.15.90:0/2367835270 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f8072ea0 con 0x7f18f8115020 2024-05-27T07:54:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.535+0000 7f18eea00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f8078630 0x7f18f806e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:51.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.536+0000 7f18f7400700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f8077ce0 0x7f18f8075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:51.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.537+0000 7f18f7e00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18f8115020 0x7f18f806eaa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f18e800b780 tx=0x7f18e800bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:51.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.538+0000 7f18f4c00700 1 -- 172.21.15.90:0/2367835270 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e800f840 con 0x7f18f8115020 2024-05-27T07:54:51.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.538+0000 7f18f4c00700 1 -- 172.21.15.90:0/2367835270 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18e8017400 con 0x7f18f8115020 2024-05-27T07:54:51.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.538+0000 7f18f4c00700 1 -- 172.21.15.90:0/2367835270 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e800d630 con 0x7f18f8115020 2024-05-27T07:54:51.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.538+0000 7f18fca00700 1 -- 172.21.15.90:0/2367835270 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f8073130 con 0x7f18f8115020 2024-05-27T07:54:51.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.538+0000 7f18fca00700 1 -- 172.21.15.90:0/2367835270 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f8073620 con 0x7f18f8115020 2024-05-27T07:54:51.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.551+0000 7f18f4c00700 1 -- 172.21.15.90:0/2367835270 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f18e80175b0 con 0x7f18f8115020 2024-05-27T07:54:51.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.553+0000 7f190ca9db80 1 Processor -- start 2024-05-27T07:54:51.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.554+0000 7f190ca9db80 1 -- start start 2024-05-27T07:54:51.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.554+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf20750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.555+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cda0e40 0x564a0cf20c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.555+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cf2a2c0 0x564a0cf2c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.555+0000 7f190ca9db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564a0ce4de40 con 0x564a0ce8c9f0 2024-05-27T07:54:51.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.555+0000 7f190ca9db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564a0ce3d9e0 con 0x564a0cf2a2c0 2024-05-27T07:54:51.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.555+0000 7f190ca9db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564a0ce41a20 con 0x564a0cda0e40 2024-05-27T07:54:51.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18f7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cda0e40 0x564a0cf20c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18f4c00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18dc0b2b20 0x7f18dc0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18f7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cda0e40 0x564a0cf20c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39942/0 (socket says 172.21.15.90:39942) 2024-05-27T07:54:51.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18f7400700 1 -- 172.21.15.90:0/3237167906 learned_addr learned my addr 172.21.15.90:0/3237167906 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:51.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18eea00700 1 --2- 172.21.15.90:0/3237167906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cf2a2c0 0x564a0cf2c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18eea00700 1 --2- 172.21.15.90:0/3237167906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf20750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18f4c00700 1 -- 172.21.15.90:0/2367835270 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7f18e80e27e0 con 0x7f18f8115020 2024-05-27T07:54:51.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18f7400700 1 -- 172.21.15.90:0/3237167906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cf2a2c0 msgr2=0x564a0cf2c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:51.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18f7400700 1 --2- 172.21.15.90:0/3237167906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cf2a2c0 0x564a0cf2c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18f7e00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18dc0b2b20 0x7f18dc0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.556+0000 7f18f7400700 1 -- 172.21.15.90:0/3237167906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 msgr2=0x564a0cf20750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:51.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.560+0000 7f18f7400700 1 --2- 172.21.15.90:0/3237167906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf20750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.560+0000 7f18f7400700 1 -- 172.21.15.90:0/3237167906 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0ce06420 con 0x564a0cda0e40 2024-05-27T07:54:51.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.560+0000 7f18eea00700 1 --2- 172.21.15.90:0/3237167906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf20750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:51.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.560+0000 7f18f7400700 1 --2- 172.21.15.90:0/3237167906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cda0e40 0x564a0cf20c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009cd0 tx=0x7f18f000e2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0649a7d7826a97f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:51.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.561+0000 7f18ecc00700 1 -- 172.21.15.90:0/3237167906 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0016070 con 0x564a0cda0e40 2024-05-27T07:54:51.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.561+0000 7f18ecc00700 1 -- 172.21.15.90:0/3237167906 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18f00044b0 con 0x564a0cda0e40 2024-05-27T07:54:51.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.561+0000 7f18ecc00700 1 -- 172.21.15.90:0/3237167906 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0005020 con 0x564a0cda0e40 2024-05-27T07:54:51.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.561+0000 7f18f7e00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18dc0b2b20 0x7f18dc0b5030 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18e8017c50 tx=0x7f18e80e2a30 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:51.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.563+0000 7f190ca9db80 1 -- 172.21.15.90:0/3237167906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cda0e40 msgr2=0x564a0cf20c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:51.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.563+0000 7f190ca9db80 1 --2- 172.21.15.90:0/3237167906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cda0e40 0x564a0cf20c90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009cd0 tx=0x7f18f000e2f0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.563+0000 7f190ca9db80 1 -- 172.21.15.90:0/3237167906 shutdown_connections 2024-05-27T07:54:51.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.563+0000 7f190ca9db80 1 --2- 172.21.15.90:0/3237167906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cf2a2c0 0x564a0cf2c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.564+0000 7f190ca9db80 1 --2- 172.21.15.90:0/3237167906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf20750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.564+0000 7f190ca9db80 1 --2- 172.21.15.90:0/3237167906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cda0e40 0x564a0cf20c90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.564+0000 7f190ca9db80 1 -- 172.21.15.90:0/3237167906 >> 172.21.15.90:0/3237167906 conn(0x564a0ce3a390 msgr2=0x564a0cf1b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:51.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.564+0000 7f190ca9db80 1 -- 172.21.15.90:0/3237167906 shutdown_connections 2024-05-27T07:54:51.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.564+0000 7f190ca9db80 1 -- 172.21.15.90:0/3237167906 wait complete. 2024-05-27T07:54:51.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.566+0000 7f190ca9db80 1 -- start start 2024-05-27T07:54:51.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.566+0000 7f190ca9db80 1 Processor -- start 2024-05-27T07:54:51.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.567+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cda0e40 0x564a0cf27220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.567+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf27760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.567+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf2a2c0 0x564a0cf27ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.567+0000 7f190ca9db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564a0cd43fa0 con 0x564a0ce8c9f0 2024-05-27T07:54:51.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.567+0000 7f190ca9db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564a0cdde760 con 0x564a0cda0e40 2024-05-27T07:54:51.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.567+0000 7f190ca9db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564a0cf3de10 con 0x564a0cf2a2c0 2024-05-27T07:54:51.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.567+0000 7f190ca9db80 10 client.0 _ll_register_callbacks cb 0x564a0cde7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:51.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.569+0000 7f18eea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf2a2c0 0x564a0cf27ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.569+0000 7f18eea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf2a2c0 0x564a0cf27ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39948/0 (socket says 172.21.15.90:39948) 2024-05-27T07:54:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.569+0000 7f18eea00700 1 -- 172.21.15.90:0/2269501084 learned_addr learned my addr 172.21.15.90:0/2269501084 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.569+0000 7f18eea00700 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cda0e40 0x564a0cf27220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.570+0000 7f18eea00700 1 -- 172.21.15.90:0/2269501084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cda0e40 msgr2=0x564a0cf27220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.572+0000 7f18f7400700 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf27760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.572+0000 7f18eea00700 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cda0e40 0x564a0cf27220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.572+0000 7f18eea00700 1 -- 172.21.15.90:0/2269501084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 msgr2=0x564a0cf27760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.572+0000 7f18eea00700 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf27760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.572+0000 7f18eea00700 1 -- 172.21.15.90:0/2269501084 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0cd8d8d0 con 0x564a0cf2a2c0 2024-05-27T07:54:51.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.573+0000 7f18f7400700 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf27760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:51.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.573+0000 7f18eea00700 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf2a2c0 0x564a0cf27ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f18e0006010 tx=0x7f18e0019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f47fc11d8cc3b7f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:51.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.573+0000 7f18e5600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:54:51.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.576+0000 7f18e5600700 1 -- 172.21.15.90:0/2269501084 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e00052d0 con 0x564a0cf2a2c0 2024-05-27T07:54:51.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.576+0000 7f18e5600700 1 -- 172.21.15.90:0/2269501084 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7f18e0005470 con 0x564a0cf2a2c0 2024-05-27T07:54:51.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.576+0000 7f190ca9db80 1 -- 172.21.15.90:0/2269501084 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f00044b0 con 0x564a0cf2a2c0 2024-05-27T07:54:51.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.578+0000 7f18e5600700 1 -- 172.21.15.90:0/2269501084 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f18e0005670 con 0x564a0cf2a2c0 2024-05-27T07:54:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.578+0000 7f190ca9db80 10 client.26689 fetch_fsmap learned FSMap version 52 2024-05-27T07:54:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.578+0000 7f190ca9db80 1 -- 172.21.15.90:0/2269501084 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564a0ce0e060 con 0x564a0cf2a2c0 2024-05-27T07:54:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.580+0000 7f18e5600700 1 -- 172.21.15.90:0/2269501084 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18f00044b0 con 0x564a0cf2a2c0 2024-05-27T07:54:51.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.580+0000 7f18e5600700 1 -- 172.21.15.90:0/2269501084 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 52) ==== 7757+0+0 (secure 0 0 0) 0x7f18e000fb90 con 0x564a0cf2a2c0 2024-05-27T07:54:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.582+0000 7f190ca9db80 10 client.26689 fetch_fsmap finished waiting for FSMap version 52 2024-05-27T07:54:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.582+0000 7f190ca9db80 10 client.26689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.582+0000 7f190ca9db80 20 client.26689 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.582+0000 7f190ca9db80 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564a0cf4ac90 0x564a0cf4d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.582+0000 7f190ca9db80 4 client.26689 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.582+0000 7f190ca9db80 1 -- 172.21.15.90:0/2269501084 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a0ce4fd90 con 0x564a0cf4ac90 2024-05-27T07:54:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.583+0000 7f18f7400700 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564a0cf4ac90 0x564a0cf4d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.586+0000 7f18e5600700 10 client.26689 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.586+0000 7f18f7400700 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564a0cf4ac90 0x564a0cf4d0d0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6660655eebbe9bb8 server_cookie=130f1abc3d0c8531 in_seq=0 out_seq=0 2024-05-27T07:54:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.587+0000 7f18e5600700 1 -- 172.21.15.90:0/2269501084 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x564a0ce4fd90 con 0x564a0cf4ac90 2024-05-27T07:54:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.587+0000 7f18e5600700 10 client.26689 handle_command_reply: tid=0 2024-05-27T07:54:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.587+0000 7f190ca9db80 1 client.26689 shutdown 2024-05-27T07:54:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.588+0000 7f190ca9db80 1 -- 172.21.15.90:0/2269501084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf2a2c0 msgr2=0x564a0cf27ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.588+0000 7f190ca9db80 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf2a2c0 0x564a0cf27ca0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f18e0006010 tx=0x7f18e0019040 comp rx=0 tx=0).stop 2024-05-27T07:54:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.588+0000 7f190ca9db80 1 -- 172.21.15.90:0/2269501084 shutdown_connections 2024-05-27T07:54:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.588+0000 7f190ca9db80 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564a0cf4ac90 0x564a0cf4d0d0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.588+0000 7f190ca9db80 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf2a2c0 0x564a0cf27ca0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.588+0000 7f190ca9db80 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cda0e40 0x564a0cf27220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.588+0000 7f190ca9db80 1 --2- 172.21.15.90:0/2269501084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf27760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.588+0000 7f190ca9db80 1 -- 172.21.15.90:0/2269501084 >> 172.21.15.90:0/2269501084 conn(0x564a0ce3a390 msgr2=0x564a0cf20750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:51.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.589+0000 7f18e5600700 0 client.26689 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:51.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.589+0000 7f190ca9db80 1 -- 172.21.15.90:0/2269501084 shutdown_connections 2024-05-27T07:54:51.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.589+0000 7f190ca9db80 1 -- 172.21.15.90:0/2269501084 wait complete. 2024-05-27T07:54:51.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.589+0000 7f190ca9db80 20 client.26689 trim_cache size 0 max 16384 2024-05-27T07:54:51.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.616+0000 7f190ca9db80 1 Processor -- start 2024-05-27T07:54:51.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.616+0000 7f190ca9db80 1 -- start start 2024-05-27T07:54:51.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.617+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cda0e40 0x564a0cf1b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.617+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf22aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.617+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf4d310 0x564a0cf46eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.617+0000 7f190ca9db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81b8460 con 0x564a0ce8c9f0 2024-05-27T07:54:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.617+0000 7f190ca9db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18f810f9a0 con 0x564a0cda0e40 2024-05-27T07:54:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.617+0000 7f190ca9db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81b7d90 con 0x564a0cf4d310 2024-05-27T07:54:51.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.617+0000 7f18f7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf22aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.618+0000 7f18eea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cda0e40 0x564a0cf1b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.618+0000 7f18f7400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf22aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56286/0 (socket says 172.21.15.90:56286) 2024-05-27T07:54:51.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.618+0000 7f18f7400700 1 -- 172.21.15.90:0/2637757805 learned_addr learned my addr 172.21.15.90:0/2637757805 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:51.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.618+0000 7f18eea00700 1 --2- 172.21.15.90:0/2637757805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf4d310 0x564a0cf46eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.618+0000 7f18f7400700 1 -- 172.21.15.90:0/2637757805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf4d310 msgr2=0x564a0cf46eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:51.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.618+0000 7f18f7400700 1 --2- 172.21.15.90:0/2637757805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf4d310 0x564a0cf46eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.618+0000 7f18f7400700 1 -- 172.21.15.90:0/2637757805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cda0e40 msgr2=0x564a0cf1b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:51.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.618+0000 7f18f7400700 1 --2- 172.21.15.90:0/2637757805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cda0e40 0x564a0cf1b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.618+0000 7f18f7400700 1 -- 172.21.15.90:0/2637757805 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0cf28010 con 0x564a0ce8c9f0 2024-05-27T07:54:51.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.619+0000 7f18f7400700 1 --2- 172.21.15.90:0/2637757805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf22aa0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x564a0cf1c970 tx=0x7f18f000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f257ccd2fb34f8b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:51.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.619+0000 7f18e7e00700 1 -- 172.21.15.90:0/2637757805 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f002c070 con 0x564a0ce8c9f0 2024-05-27T07:54:51.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.619+0000 7f18e7e00700 1 -- 172.21.15.90:0/2637757805 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18f00268e0 con 0x564a0ce8c9f0 2024-05-27T07:54:51.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.619+0000 7f18e7e00700 1 -- 172.21.15.90:0/2637757805 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0016040 con 0x564a0ce8c9f0 2024-05-27T07:54:51.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.621+0000 7f190ca9db80 1 -- 172.21.15.90:0/2637757805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 msgr2=0x564a0cf22aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:51.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.621+0000 7f190ca9db80 1 --2- 172.21.15.90:0/2637757805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf22aa0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x564a0cf1c970 tx=0x7f18f000ec00 comp rx=0 tx=0).stop 2024-05-27T07:54:51.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.621+0000 7f190ca9db80 1 -- 172.21.15.90:0/2637757805 shutdown_connections 2024-05-27T07:54:51.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.621+0000 7f190ca9db80 1 --2- 172.21.15.90:0/2637757805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0cf4d310 0x564a0cf46eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.621+0000 7f190ca9db80 1 --2- 172.21.15.90:0/2637757805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cda0e40 0x564a0cf1b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.621+0000 7f190ca9db80 1 --2- 172.21.15.90:0/2637757805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0ce8c9f0 0x564a0cf22aa0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.621+0000 7f190ca9db80 1 -- 172.21.15.90:0/2637757805 >> 172.21.15.90:0/2637757805 conn(0x564a0ce3a390 msgr2=0x564a0cf1f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:51.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.622+0000 7f190ca9db80 1 -- 172.21.15.90:0/2637757805 shutdown_connections 2024-05-27T07:54:51.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.622+0000 7f190ca9db80 1 -- 172.21.15.90:0/2637757805 wait complete. 2024-05-27T07:54:51.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.623+0000 7f190ca9db80 1 -- start start 2024-05-27T07:54:51.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.623+0000 7f190ca9db80 1 Processor -- start 2024-05-27T07:54:51.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.624+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0cda0e40 0x564a0cf437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.624+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0ce8c9f0 0x564a0cf3eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.624+0000 7f190ca9db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cf4d310 0x564a0cf3f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.624+0000 7f190ca9db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81b6ff0 con 0x564a0cda0e40 2024-05-27T07:54:51.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.624+0000 7f190ca9db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18f8110070 con 0x564a0cf4d310 2024-05-27T07:54:51.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.624+0000 7f190ca9db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18f81b76c0 con 0x564a0ce8c9f0 2024-05-27T07:54:51.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.624+0000 7f190ca9db80 10 client.0 _ll_register_callbacks cb 0x564a0cde7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:51.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.625+0000 7f18f7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0ce8c9f0 0x564a0cf3eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.625+0000 7f18eea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0cda0e40 0x564a0cf437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.626+0000 7f18f7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0ce8c9f0 0x564a0cf3eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39970/0 (socket says 172.21.15.90:39970) 2024-05-27T07:54:51.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.626+0000 7f18f7400700 1 -- 172.21.15.90:0/1686652556 learned_addr learned my addr 172.21.15.90:0/1686652556 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:51.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.626+0000 7f18eea00700 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cf4d310 0x564a0cf3f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.628+0000 7f18f7400700 1 -- 172.21.15.90:0/1686652556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cf4d310 msgr2=0x564a0cf3f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:51.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.629+0000 7f18f7400700 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cf4d310 0x564a0cf3f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.629+0000 7f18f7400700 1 -- 172.21.15.90:0/1686652556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0cda0e40 msgr2=0x564a0cf437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:51.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.629+0000 7f18f7400700 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0cda0e40 0x564a0cf437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.629+0000 7f18f7400700 1 -- 172.21.15.90:0/1686652556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a0cf3f7c0 con 0x564a0ce8c9f0 2024-05-27T07:54:51.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.629+0000 7f18eea00700 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0cda0e40 0x564a0cf437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:54:51.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.632+0000 7f18f7400700 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0ce8c9f0 0x564a0cf3eb00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f18f00047a0 tx=0x7f18f000ea40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b9548f3fdcd29 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:51.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.633+0000 7f18e5600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:54:51.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.633+0000 7f18e5600700 1 -- 172.21.15.90:0/1686652556 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0034020 con 0x564a0ce8c9f0 2024-05-27T07:54:51.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.634+0000 7f18e5600700 1 -- 172.21.15.90:0/1686652556 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7f18f0026a90 con 0x564a0ce8c9f0 2024-05-27T07:54:51.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.635+0000 7f190ca9db80 1 -- 172.21.15.90:0/1686652556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f00268e0 con 0x564a0ce8c9f0 2024-05-27T07:54:51.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.635+0000 7f18e5600700 1 -- 172.21.15.90:0/1686652556 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f18f0016050 con 0x564a0ce8c9f0 2024-05-27T07:54:51.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.635+0000 7f190ca9db80 10 client.26695 fetch_fsmap learned FSMap version 52 2024-05-27T07:54:51.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.635+0000 7f190ca9db80 1 -- 172.21.15.90:0/1686652556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564a0cfb5670 con 0x564a0ce8c9f0 2024-05-27T07:54:51.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.637+0000 7f18e5600700 1 -- 172.21.15.90:0/1686652556 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18f00268e0 con 0x564a0ce8c9f0 2024-05-27T07:54:51.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.637+0000 7f18e5600700 1 -- 172.21.15.90:0/1686652556 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 52) ==== 7757+0+0 (secure 0 0 0) 0x7f18f002f070 con 0x564a0ce8c9f0 2024-05-27T07:54:51.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.637+0000 7f190ca9db80 10 client.26695 fetch_fsmap finished waiting for FSMap version 52 2024-05-27T07:54:51.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.637+0000 7f190ca9db80 10 client.26695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:51.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.637+0000 7f190ca9db80 20 client.26695 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:51.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.637+0000 7f190ca9db80 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564a0cfb5bc0 0x564a0cfb7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:51.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.638+0000 7f190ca9db80 4 client.26695 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:54:51.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.638+0000 7f190ca9db80 1 -- 172.21.15.90:0/1686652556 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a0ce49570 con 0x564a0cfb5bc0 2024-05-27T07:54:51.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.639+0000 7f18eea00700 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564a0cfb5bc0 0x564a0cfb7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:51.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.641+0000 7f18eea00700 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564a0cfb5bc0 0x564a0cfb7fe0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea1bb7b9db106eb server_cookie=19465396cb1aca3a in_seq=0 out_seq=0 2024-05-27T07:54:51.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.641+0000 7f18e5600700 10 client.26695 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:51.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.642+0000 7f18e5600700 1 -- 172.21.15.90:0/1686652556 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a0ce49570 con 0x564a0cfb5bc0 2024-05-27T07:54:51.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.642+0000 7f18e5600700 10 client.26695 handle_command_reply: tid=0 2024-05-27T07:54:51.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.642+0000 7f190ca9db80 1 client.26695 shutdown 2024-05-27T07:54:51.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.643+0000 7f190ca9db80 1 -- 172.21.15.90:0/1686652556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0ce8c9f0 msgr2=0x564a0cf3eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:51.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.643+0000 7f190ca9db80 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0ce8c9f0 0x564a0cf3eb00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f18f00047a0 tx=0x7f18f000ea40 comp rx=0 tx=0).stop 2024-05-27T07:54:51.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.643+0000 7f190ca9db80 1 -- 172.21.15.90:0/1686652556 shutdown_connections 2024-05-27T07:54:51.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.643+0000 7f190ca9db80 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x564a0cfb5bc0 0x564a0cfb7fe0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.643+0000 7f190ca9db80 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564a0cf4d310 0x564a0cf3f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.643+0000 7f190ca9db80 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564a0cda0e40 0x564a0cf437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.643+0000 7f190ca9db80 1 --2- 172.21.15.90:0/1686652556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a0ce8c9f0 0x564a0cf3eb00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.643+0000 7f18e5600700 0 client.26695 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:51.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.643+0000 7f190ca9db80 1 -- 172.21.15.90:0/1686652556 >> 172.21.15.90:0/1686652556 conn(0x564a0ce3a390 msgr2=0x564a0cf22800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:51.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.643+0000 7f190ca9db80 1 -- 172.21.15.90:0/1686652556 shutdown_connections 2024-05-27T07:54:51.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.644+0000 7f190ca9db80 1 -- 172.21.15.90:0/1686652556 wait complete. 2024-05-27T07:54:51.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.644+0000 7f190ca9db80 20 client.26695 trim_cache size 0 max 16384 2024-05-27T07:54:51.645 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:54:51.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.645+0000 7f18fca00700 1 -- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18dc0b2b20 msgr2=0x7f18dc0b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:51.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.645+0000 7f18fca00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18dc0b2b20 0x7f18dc0b5030 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f18e8017c50 tx=0x7f18e80e2a30 comp rx=0 tx=0).stop 2024-05-27T07:54:51.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.645+0000 7f18fca00700 1 -- 172.21.15.90:0/2367835270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18f8115020 msgr2=0x7f18f806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:51.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.645+0000 7f18fca00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18f8115020 0x7f18f806eaa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f18e800b780 tx=0x7f18e800bb40 comp rx=0 tx=0).stop 2024-05-27T07:54:51.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.646+0000 7f18fca00700 1 -- 172.21.15.90:0/2367835270 shutdown_connections 2024-05-27T07:54:51.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.646+0000 7f18fca00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18dc0b2b20 0x7f18dc0b5030 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.646+0000 7f18fca00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18f8115020 0x7f18f806eaa0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.646+0000 7f18fca00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18f8077ce0 0x7f18f8075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.646+0000 7f18fca00700 1 --2- 172.21.15.90:0/2367835270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18f8078630 0x7f18f806e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:51.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.646+0000 7f18fca00700 1 -- 172.21.15.90:0/2367835270 >> 172.21.15.90:0/2367835270 conn(0x7f18f8102020 msgr2=0x7f18f807af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:51.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.646+0000 7f18fca00700 1 -- 172.21.15.90:0/2367835270 shutdown_connections 2024-05-27T07:54:51.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:51.647+0000 7f18fca00700 1 -- 172.21.15.90:0/2367835270 wait complete. 2024-05-27T07:54:51.963 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:51 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-05-27T07:54:51.963 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:54:51 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.532s CPU time. 2024-05-27T07:54:52.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:52 smithi136 ceph-mon[35324]: pgmap v267: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2024-05-27T07:54:52.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:52 smithi129 ceph-mon[35254]: pgmap v267: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2024-05-27T07:54:52.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:52 smithi090 ceph-mon[31429]: pgmap v267: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.8 MiB/s wr, 544 op/s 2024-05-27T07:54:53.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 275.0 s, 603.0 tps, lat 177.275 ms stddev 117.277 2024-05-27T07:54:54.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:54 smithi129 ceph-mon[35254]: pgmap v268: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.2 MiB/s wr, 522 op/s 2024-05-27T07:54:54.686 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:54:54.687 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:54:54.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:54 smithi090 ceph-mon[31429]: pgmap v268: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.2 MiB/s wr, 522 op/s 2024-05-27T07:54:54.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:54 smithi136 ceph-mon[35324]: pgmap v268: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.2 MiB/s wr, 522 op/s 2024-05-27T07:54:55.449 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:54:56.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:56 smithi090 ceph-mon[31429]: pgmap v269: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.1 MiB/s wr, 486 op/s 2024-05-27T07:54:56.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:56 smithi129 ceph-mon[35254]: pgmap v269: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.1 MiB/s wr, 486 op/s 2024-05-27T07:54:56.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:56 smithi136 ceph-mon[35324]: pgmap v269: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.1 MiB/s wr, 486 op/s 2024-05-27T07:54:57.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.972+0000 7f62db800700 1 -- 172.21.15.90:0/1276656416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d410c830 msgr2=0x7f62d410cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:57.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.972+0000 7f62db800700 1 --2- 172.21.15.90:0/1276656416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d410c830 0x7f62d410cc40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f62c800d960 tx=0x7f62c800dc70 comp rx=0 tx=0).stop 2024-05-27T07:54:57.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.972+0000 7f62db800700 1 -- 172.21.15.90:0/1276656416 shutdown_connections 2024-05-27T07:54:57.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.972+0000 7f62db800700 1 --2- 172.21.15.90:0/1276656416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62d4110fa0 0x7f62d41188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:57.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.972+0000 7f62db800700 1 --2- 172.21.15.90:0/1276656416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d410c830 0x7f62d410cc40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:57.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.972+0000 7f62db800700 1 --2- 172.21.15.90:0/1276656416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d4110650 0x7f62d4110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:57.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.972+0000 7f62db800700 1 -- 172.21.15.90:0/1276656416 >> 172.21.15.90:0/1276656416 conn(0x7f62d406f4c0 msgr2=0x7f62d406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:57.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.972+0000 7f62db800700 1 -- 172.21.15.90:0/1276656416 shutdown_connections 2024-05-27T07:54:57.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.973+0000 7f62db800700 1 -- 172.21.15.90:0/1276656416 wait complete. 2024-05-27T07:54:57.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.974+0000 7f62db800700 1 Processor -- start 2024-05-27T07:54:57.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.975+0000 7f62db800700 1 -- start start 2024-05-27T07:54:57.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.975+0000 7f62db800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d410c830 0x7f62d41af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:57.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.975+0000 7f62db800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62d4110650 0x7f62d41afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:57.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.976+0000 7f62db800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d4110fa0 0x7f62d41a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.976+0000 7f62db800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f62d411aa20 con 0x7f62d410c830 2024-05-27T07:54:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.976+0000 7f62db800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62d411a8a0 con 0x7f62d4110650 2024-05-27T07:54:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.976+0000 7f62db800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f62d411aba0 con 0x7f62d4110fa0 2024-05-27T07:54:57.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.976+0000 7f62d9800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d4110fa0 0x7f62d41a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:57.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.976+0000 7f62d9800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d4110fa0 0x7f62d41a99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39978/0 (socket says 172.21.15.90:39978) 2024-05-27T07:54:57.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.976+0000 7f62d9800700 1 -- 172.21.15.90:0/1596844097 learned_addr learned my addr 172.21.15.90:0/1596844097 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:57.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.977+0000 7f62d8e00700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d410c830 0x7f62d41af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:57.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.977+0000 7f62d3e00700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62d4110650 0x7f62d41afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:57.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.977+0000 7f62d9800700 1 -- 172.21.15.90:0/1596844097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62d4110650 msgr2=0x7f62d41afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:57.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.977+0000 7f62d9800700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62d4110650 0x7f62d41afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:57.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.977+0000 7f62d9800700 1 -- 172.21.15.90:0/1596844097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d410c830 msgr2=0x7f62d41af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:57.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.977+0000 7f62d9800700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d410c830 0x7f62d41af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:57.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.977+0000 7f62d9800700 1 -- 172.21.15.90:0/1596844097 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d41aa080 con 0x7f62d4110fa0 2024-05-27T07:54:57.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.978+0000 7f62d9800700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d4110fa0 0x7f62d41a99d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00c110 tx=0x7f62cc00c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:57.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.979+0000 7f62d1600700 1 -- 172.21.15.90:0/1596844097 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0092e0 con 0x7f62d4110fa0 2024-05-27T07:54:57.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.979+0000 7f62d1600700 1 -- 172.21.15.90:0/1596844097 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62cc004bb0 con 0x7f62d4110fa0 2024-05-27T07:54:57.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.979+0000 7f62d1600700 1 -- 172.21.15.90:0/1596844097 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc007bd0 con 0x7f62d4110fa0 2024-05-27T07:54:57.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.979+0000 7f62db800700 1 -- 172.21.15.90:0/1596844097 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d41aa310 con 0x7f62d4110fa0 2024-05-27T07:54:57.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.979+0000 7f62db800700 1 -- 172.21.15.90:0/1596844097 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d41aa800 con 0x7f62d4110fa0 2024-05-27T07:54:57.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.986+0000 7f62d1600700 1 -- 172.21.15.90:0/1596844097 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f62cc01a020 con 0x7f62d4110fa0 2024-05-27T07:54:57.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.990+0000 7f62d1600700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62bc0b2c00 0x7f62bc0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:57.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.990+0000 7f62d1600700 1 -- 172.21.15.90:0/1596844097 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7f62cc0e2400 con 0x7f62d4110fa0 2024-05-27T07:54:57.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.991+0000 7f62d8e00700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62bc0b2c00 0x7f62bc0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:57.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.992+0000 7f62d8e00700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62bc0b2c00 0x7f62bc0b5110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62c800d930 tx=0x7f62c800ffe0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.993+0000 7f62ea345b80 1 Processor -- start 2024-05-27T07:54:57.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.993+0000 7f62ea345b80 1 -- start start 2024-05-27T07:54:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.994+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c98870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.994+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1c928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.994+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c93240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.994+0000 7f62ea345b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559da1bc0ec0 con 0x559da1b13ec0 2024-05-27T07:54:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.994+0000 7f62ea345b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559da1bb0a60 con 0x559da1c98db0 2024-05-27T07:54:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.994+0000 7f62ea345b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559da1bb4aa0 con 0x559da1c92e30 2024-05-27T07:54:57.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.995+0000 7f62d8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c98870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:57.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.995+0000 7f62d8e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c98870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56330/0 (socket says 172.21.15.90:56330) 2024-05-27T07:54:57.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.995+0000 7f62d8e00700 1 -- 172.21.15.90:0/3561405951 learned_addr learned my addr 172.21.15.90:0/3561405951 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:57.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.995+0000 7f62d9800700 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c93240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.995+0000 7f62d3e00700 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1c928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.996+0000 7f62d9800700 1 -- 172.21.15.90:0/3561405951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 msgr2=0x559da1c928f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.996+0000 7f62d9800700 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1c928f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.996+0000 7f62d9800700 1 -- 172.21.15.90:0/3561405951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 msgr2=0x559da1c98870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.996+0000 7f62d9800700 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c98870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.996+0000 7f62d9800700 1 -- 172.21.15.90:0/3561405951 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da1b794a0 con 0x559da1c92e30 2024-05-27T07:54:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.996+0000 7f62d3e00700 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1c928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.997+0000 7f62d8e00700 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c98870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:57.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.997+0000 7f62d9800700 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c93240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0df7a0 tx=0x7f62cc0a8af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde14384c8adf71e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:57.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.998+0000 7f62c6000700 1 -- 172.21.15.90:0/3561405951 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0a9a30 con 0x559da1c92e30 2024-05-27T07:54:57.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.998+0000 7f62c6000700 1 -- 172.21.15.90:0/3561405951 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62cc0a9330 con 0x559da1c92e30 2024-05-27T07:54:57.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:57.998+0000 7f62c6000700 1 -- 172.21.15.90:0/3561405951 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0a97f0 con 0x559da1c92e30 2024-05-27T07:54:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.000+0000 7f62ea345b80 1 -- 172.21.15.90:0/3561405951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 msgr2=0x559da1c93240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:58.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.000+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c93240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0df7a0 tx=0x7f62cc0a8af0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.000+0000 7f62ea345b80 1 -- 172.21.15.90:0/3561405951 shutdown_connections 2024-05-27T07:54:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.000+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c93240 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.000+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c98870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.000+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3561405951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1c928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.000+0000 7f62ea345b80 1 -- 172.21.15.90:0/3561405951 >> 172.21.15.90:0/3561405951 conn(0x559da1b74040 msgr2=0x559da1c8d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.000+0000 7f62ea345b80 1 -- 172.21.15.90:0/3561405951 shutdown_connections 2024-05-27T07:54:58.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.000+0000 7f62ea345b80 1 -- 172.21.15.90:0/3561405951 wait complete. 2024-05-27T07:54:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.001+0000 7f62ea345b80 1 -- start start 2024-05-27T07:54:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.001+0000 7f62ea345b80 1 Processor -- start 2024-05-27T07:54:58.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.002+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c9c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.002+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c9cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.002+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1c9cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.002+0000 7f62ea345b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559da1ab7020 con 0x559da1b13ec0 2024-05-27T07:54:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.002+0000 7f62ea345b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559da1b517e0 con 0x559da1c98db0 2024-05-27T07:54:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.002+0000 7f62ea345b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559da1c9f880 con 0x559da1c92e30 2024-05-27T07:54:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.002+0000 7f62ea345b80 10 client.0 _ll_register_callbacks cb 0x559da1b5acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.003+0000 7f62d3e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c9cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.003+0000 7f62d3e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c9cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39998/0 (socket says 172.21.15.90:39998) 2024-05-27T07:54:58.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.003+0000 7f62d3e00700 1 -- 172.21.15.90:0/911225539 learned_addr learned my addr 172.21.15.90:0/911225539 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:58.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.003+0000 7f62d9800700 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1c9cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.003+0000 7f62d8e00700 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c9c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.004+0000 7f62d3e00700 1 -- 172.21.15.90:0/911225539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 msgr2=0x559da1c9cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:58.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.004+0000 7f62d3e00700 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1c9cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.004+0000 7f62d3e00700 1 -- 172.21.15.90:0/911225539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 msgr2=0x559da1c9c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:58.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.004+0000 7f62d3e00700 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c9c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.004+0000 7f62d3e00700 1 -- 172.21.15.90:0/911225539 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da1b00950 con 0x559da1c92e30 2024-05-27T07:54:58.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.004+0000 7f62d8e00700 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c9c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:54:58.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.005+0000 7f62d3e00700 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c9cab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f62c00106a0 tx=0x7f62c00109b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca78eb4a677555f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:58.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.005+0000 7f62baa00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:54:58.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.005+0000 7f62baa00700 1 -- 172.21.15.90:0/911225539 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0004440 con 0x559da1c92e30 2024-05-27T07:54:58.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.006+0000 7f62ea345b80 1 -- 172.21.15.90:0/911225539 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62cc0a9330 con 0x559da1c92e30 2024-05-27T07:54:58.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.006+0000 7f62baa00700 1 -- 172.21.15.90:0/911225539 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7f62c0003680 con 0x559da1c92e30 2024-05-27T07:54:58.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.006+0000 7f62baa00700 1 -- 172.21.15.90:0/911225539 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f62c0003880 con 0x559da1c92e30 2024-05-27T07:54:58.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.006+0000 7f62ea345b80 10 client.26707 fetch_fsmap learned FSMap version 52 2024-05-27T07:54:58.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.006+0000 7f62ea345b80 1 -- 172.21.15.90:0/911225539 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x559da1b810e0 con 0x559da1c92e30 2024-05-27T07:54:58.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.007+0000 7f62baa00700 1 -- 172.21.15.90:0/911225539 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62cc0a9330 con 0x559da1c92e30 2024-05-27T07:54:58.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.007+0000 7f62baa00700 1 -- 172.21.15.90:0/911225539 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 52) ==== 7757+0+0 (secure 0 0 0) 0x7f62c000f040 con 0x559da1c92e30 2024-05-27T07:54:58.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.007+0000 7f62ea345b80 10 client.26707 fetch_fsmap finished waiting for FSMap version 52 2024-05-27T07:54:58.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.007+0000 7f62ea345b80 10 client.26707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:58.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.007+0000 7f62ea345b80 20 client.26707 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:58.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.007+0000 7f62ea345b80 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559da1cac6e0 0x559da1caeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.007+0000 7f62ea345b80 4 client.26707 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:54:58.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.007+0000 7f62ea345b80 1 -- 172.21.15.90:0/911225539 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559da1bc2e10 con 0x559da1cac6e0 2024-05-27T07:54:58.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.008+0000 7f62d8e00700 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559da1cac6e0 0x559da1caeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.011+0000 7f62d8e00700 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559da1cac6e0 0x559da1caeb00 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dc0448cbca8ba66 server_cookie=512c28c78fa33a07 in_seq=0 out_seq=0 2024-05-27T07:54:58.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.011+0000 7f62baa00700 10 client.26707 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.013+0000 7f62baa00700 1 -- 172.21.15.90:0/911225539 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559da1bc2e10 con 0x559da1cac6e0 2024-05-27T07:54:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.013+0000 7f62baa00700 10 client.26707 handle_command_reply: tid=0 2024-05-27T07:54:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.013+0000 7f62ea345b80 1 client.26707 shutdown 2024-05-27T07:54:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.014+0000 7f62ea345b80 1 -- 172.21.15.90:0/911225539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 msgr2=0x559da1c9cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.014+0000 7f62ea345b80 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c9cab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f62c00106a0 tx=0x7f62c00109b0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.014+0000 7f62ea345b80 1 -- 172.21.15.90:0/911225539 shutdown_connections 2024-05-27T07:54:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.014+0000 7f62ea345b80 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559da1cac6e0 0x559da1caeb00 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.014+0000 7f62ea345b80 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1c9cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.014+0000 7f62ea345b80 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1b13ec0 0x559da1c9c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.014+0000 7f62ea345b80 1 --2- 172.21.15.90:0/911225539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c92e30 0x559da1c9cab0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.014+0000 7f62ea345b80 1 -- 172.21.15.90:0/911225539 >> 172.21.15.90:0/911225539 conn(0x559da1b74040 msgr2=0x559da1c8d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.014+0000 7f62baa00700 0 client.26707 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.015+0000 7f62ea345b80 1 -- 172.21.15.90:0/911225539 shutdown_connections 2024-05-27T07:54:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.015+0000 7f62ea345b80 1 -- 172.21.15.90:0/911225539 wait complete. 2024-05-27T07:54:58.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.016+0000 7f62ea345b80 20 client.26707 trim_cache size 0 max 16384 2024-05-27T07:54:58.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.060+0000 7f62ea345b80 1 Processor -- start 2024-05-27T07:54:58.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.061+0000 7f62ea345b80 1 -- start start 2024-05-27T07:54:58.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.061+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1b13ec0 0x559da1c928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.061+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1c9cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.061+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1ca88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.061+0000 7f62ea345b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41c0ec0 con 0x559da1c92e30 2024-05-27T07:54:58.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.061+0000 7f62ea345b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41becb0 con 0x559da1c98db0 2024-05-27T07:54:58.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.062+0000 7f62ea345b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41c07f0 con 0x559da1b13ec0 2024-05-27T07:54:58.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.062+0000 7f62d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1c9cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.063+0000 7f62d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1c9cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56356/0 (socket says 172.21.15.90:56356) 2024-05-27T07:54:58.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.063+0000 7f62d3e00700 1 -- 172.21.15.90:0/3076022507 learned_addr learned my addr 172.21.15.90:0/3076022507 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:58.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.063+0000 7f62d8e00700 1 --2- 172.21.15.90:0/3076022507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1b13ec0 0x559da1c928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.063+0000 7f62d9800700 1 --2- 172.21.15.90:0/3076022507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1ca88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.064+0000 7f62d3e00700 1 -- 172.21.15.90:0/3076022507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1b13ec0 msgr2=0x559da1c928f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.064+0000 7f62d3e00700 1 --2- 172.21.15.90:0/3076022507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1b13ec0 0x559da1c928f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.064+0000 7f62d3e00700 1 -- 172.21.15.90:0/3076022507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 msgr2=0x559da1ca88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.064+0000 7f62d3e00700 1 --2- 172.21.15.90:0/3076022507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1ca88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.064+0000 7f62d3e00700 1 -- 172.21.15.90:0/3076022507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da1c9aae0 con 0x559da1c92e30 2024-05-27T07:54:58.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.065+0000 7f62d3e00700 1 --2- 172.21.15.90:0/3076022507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1c9cf60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f62c00129b0 tx=0x7f62c000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8683cdbdd463c6cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:58.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.066+0000 7f62c5600700 1 -- 172.21.15.90:0/3076022507 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0033070 con 0x559da1c92e30 2024-05-27T07:54:58.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.066+0000 7f62c5600700 1 -- 172.21.15.90:0/3076022507 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62c0012d60 con 0x559da1c92e30 2024-05-27T07:54:58.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.066+0000 7f62c5600700 1 -- 172.21.15.90:0/3076022507 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c002a040 con 0x559da1c92e30 2024-05-27T07:54:58.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.067+0000 7f62ea345b80 1 -- 172.21.15.90:0/3076022507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 msgr2=0x559da1c9cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:58.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.067+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3076022507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1c9cf60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f62c00129b0 tx=0x7f62c000b6c0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.068+0000 7f62ea345b80 1 -- 172.21.15.90:0/3076022507 shutdown_connections 2024-05-27T07:54:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.068+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3076022507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1c98db0 0x559da1ca88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.068+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3076022507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1b13ec0 0x559da1c928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.068+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3076022507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1c9cf60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.068+0000 7f62ea345b80 1 -- 172.21.15.90:0/3076022507 >> 172.21.15.90:0/3076022507 conn(0x559da1b74040 msgr2=0x559da1c9dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.068+0000 7f62ea345b80 1 -- 172.21.15.90:0/3076022507 shutdown_connections 2024-05-27T07:54:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.068+0000 7f62ea345b80 1 -- 172.21.15.90:0/3076022507 wait complete. 2024-05-27T07:54:58.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.068+0000 7f62ea345b80 1 -- start start 2024-05-27T07:54:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.069+0000 7f62ea345b80 1 Processor -- start 2024-05-27T07:54:58.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1b13ec0 0x559da1d12a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1d11160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62ea345b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c98db0 0x559da1d116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62ea345b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41bfa50 con 0x559da1c92e30 2024-05-27T07:54:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62ea345b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41bf380 con 0x559da1b13ec0 2024-05-27T07:54:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62ea345b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f62d41c0120 con 0x559da1c98db0 2024-05-27T07:54:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62ea345b80 10 client.0 _ll_register_callbacks cb 0x559da1b5acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:54:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1d11160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62d3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1d11160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56362/0 (socket says 172.21.15.90:56362) 2024-05-27T07:54:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62d3e00700 1 -- 172.21.15.90:0/3753305695 learned_addr learned my addr 172.21.15.90:0/3753305695 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:54:58.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.070+0000 7f62d9800700 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c98db0 0x559da1d116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.071+0000 7f62d8e00700 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1b13ec0 0x559da1d12a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.073+0000 7f62d3e00700 1 -- 172.21.15.90:0/3753305695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c98db0 msgr2=0x559da1d116a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.073+0000 7f62d3e00700 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c98db0 0x559da1d116a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.073+0000 7f62d3e00700 1 -- 172.21.15.90:0/3753305695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1b13ec0 msgr2=0x559da1d12a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:54:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.073+0000 7f62d3e00700 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1b13ec0 0x559da1d12a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.073+0000 7f62d3e00700 1 -- 172.21.15.90:0/3753305695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da1d11dd0 con 0x559da1c92e30 2024-05-27T07:54:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.073+0000 7f62d8e00700 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1b13ec0 0x559da1d12a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:54:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.074+0000 7f62d3e00700 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1d11160 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f62c0014020 tx=0x7f62c000bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f82f03edd0112d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:54:58.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.074+0000 7f62baa00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:54:58.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.074+0000 7f62baa00700 1 -- 172.21.15.90:0/3753305695 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c003d060 con 0x559da1c92e30 2024-05-27T07:54:58.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.074+0000 7f62baa00700 1 -- 172.21.15.90:0/3753305695 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7f62c0004440 con 0x559da1c92e30 2024-05-27T07:54:58.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.075+0000 7f62ea345b80 1 -- 172.21.15.90:0/3753305695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c0012d60 con 0x559da1c92e30 2024-05-27T07:54:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.075+0000 7f62baa00700 1 -- 172.21.15.90:0/3753305695 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f62c002a070 con 0x559da1c92e30 2024-05-27T07:54:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.075+0000 7f62ea345b80 10 client.17106 fetch_fsmap learned FSMap version 52 2024-05-27T07:54:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.075+0000 7f62ea345b80 1 -- 172.21.15.90:0/3753305695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x559da1ca49d0 con 0x559da1c92e30 2024-05-27T07:54:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.075+0000 7f62baa00700 1 -- 172.21.15.90:0/3753305695 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62c0012d60 con 0x559da1c92e30 2024-05-27T07:54:58.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.076+0000 7f62baa00700 1 -- 172.21.15.90:0/3753305695 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 52) ==== 7757+0+0 (secure 0 0 0) 0x7f62c00052f0 con 0x559da1c92e30 2024-05-27T07:54:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.076+0000 7f62ea345b80 10 client.17106 fetch_fsmap finished waiting for FSMap version 52 2024-05-27T07:54:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.076+0000 7f62ea345b80 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:54:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.076+0000 7f62ea345b80 20 client.17106 populate_metadata read hostname 'smithi090' 2024-05-27T07:54:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.076+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559da1ca50c0 0x559da1d2c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:54:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.076+0000 7f62ea345b80 4 client.17106 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:54:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.076+0000 7f62ea345b80 1 -- 172.21.15.90:0/3753305695 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559da1bbc5f0 con 0x559da1ca50c0 2024-05-27T07:54:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.077+0000 7f62d8e00700 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559da1ca50c0 0x559da1d2c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:54:58.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.083+0000 7f62d8e00700 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559da1ca50c0 0x559da1d2c1c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f634540672236c0f server_cookie=ef360e1ccedcb2e4 in_seq=0 out_seq=0 2024-05-27T07:54:58.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.083+0000 7f62baa00700 10 client.17106 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:58.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.089+0000 7f62baa00700 1 -- 172.21.15.90:0/3753305695 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559da1bbc5f0 con 0x559da1ca50c0 2024-05-27T07:54:58.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.089+0000 7f62baa00700 10 client.17106 handle_command_reply: tid=0 2024-05-27T07:54:58.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.089+0000 7f62ea345b80 1 client.17106 shutdown 2024-05-27T07:54:58.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.089+0000 7f62ea345b80 1 -- 172.21.15.90:0/3753305695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 msgr2=0x559da1d11160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:58.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.089+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1d11160 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f62c0014020 tx=0x7f62c000bfa0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.090+0000 7f62ea345b80 1 -- 172.21.15.90:0/3753305695 shutdown_connections 2024-05-27T07:54:58.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.090+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559da1ca50c0 0x559da1d2c1c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.090+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559da1c98db0 0x559da1d116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.090+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559da1b13ec0 0x559da1d12a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.090+0000 7f62ea345b80 1 --2- 172.21.15.90:0/3753305695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559da1c92e30 0x559da1d11160 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.090+0000 7f62ea345b80 1 -- 172.21.15.90:0/3753305695 >> 172.21.15.90:0/3753305695 conn(0x559da1b74040 msgr2=0x559da1c9ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:58.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.090+0000 7f62baa00700 0 client.17106 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:54:58.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.090+0000 7f62ea345b80 1 -- 172.21.15.90:0/3753305695 shutdown_connections 2024-05-27T07:54:58.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.091+0000 7f62ea345b80 1 -- 172.21.15.90:0/3753305695 wait complete. 2024-05-27T07:54:58.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.091+0000 7f62ea345b80 20 client.17106 trim_cache size 0 max 16384 2024-05-27T07:54:58.095 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:54:58.095 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:54:58.095 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "aaff9bdd-bdf2-4c84-9f62-17245517829d", 2024-05-27T07:54:58.095 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:54:58.095 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.092+0000 7f62db800700 1 -- 172.21.15.90:0/1596844097 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62bc0b2c00 msgr2=0x7f62bc0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.092+0000 7f62db800700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62bc0b2c00 0x7f62bc0b5110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62c800d930 tx=0x7f62c800ffe0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.092+0000 7f62db800700 1 -- 172.21.15.90:0/1596844097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d4110fa0 msgr2=0x7f62d41a99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.092+0000 7f62db800700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d4110fa0 0x7f62d41a99d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00c110 tx=0x7f62cc00c420 comp rx=0 tx=0).stop 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.092+0000 7f62db800700 1 -- 172.21.15.90:0/1596844097 shutdown_connections 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.092+0000 7f62db800700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d4110fa0 0x7f62d41a99d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.092+0000 7f62db800700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d410c830 0x7f62d41af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.092+0000 7f62db800700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f62bc0b2c00 0x7f62bc0b5110 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.092+0000 7f62db800700 1 --2- 172.21.15.90:0/1596844097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62d4110650 0x7f62d41afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.092+0000 7f62db800700 1 -- 172.21.15.90:0/1596844097 >> 172.21.15.90:0/1596844097 conn(0x7f62d406f4c0 msgr2=0x7f62d410f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.094+0000 7f62db800700 1 -- 172.21.15.90:0/1596844097 shutdown_connections 2024-05-27T07:54:58.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:54:58.094+0000 7f62db800700 1 -- 172.21.15.90:0/1596844097 wait complete. 2024-05-27T07:54:58.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:58 smithi129 ceph-mon[35254]: pgmap v270: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.7 MiB/s wr, 443 op/s 2024-05-27T07:54:58.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:58 smithi090 ceph-mon[31429]: pgmap v270: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.7 MiB/s wr, 443 op/s 2024-05-27T07:54:58.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:58 smithi136 ceph-mon[35324]: pgmap v270: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 4.7 MiB/s wr, 443 op/s 2024-05-27T07:54:58.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 280.0 s, 601.2 tps, lat 165.184 ms stddev 48.831 2024-05-27T07:54:59.123 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:54:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:59 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:54:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:59 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:59 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:54:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:59 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:59.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:54:59 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:54:59.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:59 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:54:59.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:59 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:59.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:59 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:54:59.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:59 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:59.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:54:59 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:54:59.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:59 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:54:59.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:59 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:59.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:59 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:54:59.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:59 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:54:59.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:54:59 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:54:59.908 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:00.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:00 smithi090 ceph-mon[31429]: pgmap v271: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.5 MiB/s wr, 753 op/s 2024-05-27T07:55:00.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:00 smithi129 ceph-mon[35254]: pgmap v271: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.5 MiB/s wr, 753 op/s 2024-05-27T07:55:00.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:00 smithi136 ceph-mon[35324]: pgmap v271: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.5 MiB/s wr, 753 op/s 2024-05-27T07:55:01.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:01 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 5. 2024-05-27T07:55:01.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:01 smithi090 systemd[1]: Stopped Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:55:01.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:01 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.532s CPU time. 2024-05-27T07:55:01.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:01 smithi090 systemd[1]: Starting Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:55:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.349+0000 7fedda600640 1 -- 172.21.15.90:0/238018986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd4111fc0 msgr2=0x7fedd41143e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.349+0000 7fedda600640 1 --2- 172.21.15.90:0/238018986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd4111fc0 0x7fedd41143e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009860 tx=0x7fedc002a360 comp rx=0 tx=0).stop 2024-05-27T07:55:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.349+0000 7fedda600640 1 -- 172.21.15.90:0/238018986 shutdown_connections 2024-05-27T07:55:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.349+0000 7fedda600640 1 --2- 172.21.15.90:0/238018986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd4111fc0 0x7fedd41143e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.349+0000 7fedda600640 1 --2- 172.21.15.90:0/238018986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd4114920 0x7fedd4116d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.349+0000 7fedda600640 1 --2- 172.21.15.90:0/238018986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd4077610 0x7fedd4077a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.349+0000 7fedda600640 1 -- 172.21.15.90:0/238018986 >> 172.21.15.90:0/238018986 conn(0x7fedd41013d0 msgr2=0x7fedd4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.350+0000 7fedda600640 1 -- 172.21.15.90:0/238018986 shutdown_connections 2024-05-27T07:55:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.350+0000 7fedda600640 1 -- 172.21.15.90:0/238018986 wait complete. 2024-05-27T07:55:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.351+0000 7fedda600640 1 Processor -- start 2024-05-27T07:55:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.352+0000 7fedda600640 1 -- start start 2024-05-27T07:55:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.354+0000 7fedda600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd4077610 0x7fedd41a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.354+0000 7fedda600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd4111fc0 0x7fedd41a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.354+0000 7fedda600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd4114920 0x7fedd41a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.354+0000 7fedda600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fedd411a120 con 0x7fedd4077610 2024-05-27T07:55:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.354+0000 7fedda600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fedd4119fa0 con 0x7fedd4114920 2024-05-27T07:55:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.354+0000 7fedda600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fedd411a2a0 con 0x7fedd4111fc0 2024-05-27T07:55:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.355+0000 7fedd9c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd4114920 0x7fedd41a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.355+0000 7fedd9c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd4114920 0x7fedd41a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34860/0 (socket says 172.21.15.90:34860) 2024-05-27T07:55:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.355+0000 7fedd9c00640 1 -- 172.21.15.90:0/2252735866 learned_addr learned my addr 172.21.15.90:0/2252735866 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.355+0000 7fedd3e00640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd4111fc0 0x7fedd41a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.355+0000 7fedd9200640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd4077610 0x7fedd41a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.356+0000 7fedd9c00640 1 -- 172.21.15.90:0/2252735866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd4111fc0 msgr2=0x7fedd41a6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.356+0000 7fedd9c00640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd4111fc0 0x7fedd41a6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.356+0000 7fedd9c00640 1 -- 172.21.15.90:0/2252735866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd4077610 msgr2=0x7fedd41a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.356+0000 7fedd9c00640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd4077610 0x7fedd41a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.356+0000 7fedd9c00640 1 -- 172.21.15.90:0/2252735866 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd41a10e0 con 0x7fedd4114920 2024-05-27T07:55:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.356+0000 7fedd9c00640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd4114920 0x7fedd41a0a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00eb10 tx=0x7fedcc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.357+0000 7fedd1600640 1 -- 172.21.15.90:0/2252735866 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc00ce20 con 0x7fedd4114920 2024-05-27T07:55:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.357+0000 7fedd1600640 1 -- 172.21.15.90:0/2252735866 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedcc005020 con 0x7fedd4114920 2024-05-27T07:55:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.357+0000 7fedd1600640 1 -- 172.21.15.90:0/2252735866 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc010690 con 0x7fedd4114920 2024-05-27T07:55:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.357+0000 7fedda600640 1 -- 172.21.15.90:0/2252735866 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd41a13d0 con 0x7fedd4114920 2024-05-27T07:55:02.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.358+0000 7fedda600640 1 -- 172.21.15.90:0/2252735866 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd41ad6b0 con 0x7fedd4114920 2024-05-27T07:55:02.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.362+0000 7fede2c2d740 1 Processor -- start 2024-05-27T07:55:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.367+0000 7fedd1600640 1 -- 172.21.15.90:0/2252735866 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fedcc0051d0 con 0x7fedd4114920 2024-05-27T07:55:02.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.368+0000 7fede2c2d740 1 -- start start 2024-05-27T07:55:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.370+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0ea41b0 0x5631a1246a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.370+0000 7fedd1600640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fedbc0b2aa0 0x7fedbc0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.370+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a1246fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.370+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 0x5631a12474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.370+0000 7fede2c2d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5631a11b7b90 con 0x5631a0e7b2f0 2024-05-27T07:55:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.370+0000 7fede2c2d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5631a11a7730 con 0x5631a0ea41b0 2024-05-27T07:55:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.370+0000 7fede2c2d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631a11ab770 con 0x5631a0e834d0 2024-05-27T07:55:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.370+0000 7fedd1600640 1 -- 172.21.15.90:0/2252735866 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7fedcc0e2da0 con 0x7fedd4114920 2024-05-27T07:55:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.371+0000 7fedd9200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0ea41b0 0x5631a1246a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.371+0000 7fedd3e00640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fedbc0b2aa0 0x7fedbc0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.371+0000 7fedd9200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0ea41b0 0x5631a1246a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34870/0 (socket says 172.21.15.90:34870) 2024-05-27T07:55:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.371+0000 7fedd9200640 1 -- 172.21.15.90:0/3190213859 learned_addr learned my addr 172.21.15.90:0/3190213859 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.371+0000 7fedd3e00640 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a1246fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.372+0000 7fedd9c00640 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 0x5631a12474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.372+0000 7fedd9200640 1 -- 172.21.15.90:0/3190213859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 msgr2=0x5631a12474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.372+0000 7fedd9200640 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 0x5631a12474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.372+0000 7fedd9200640 1 -- 172.21.15.90:0/3190213859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 msgr2=0x5631a1246fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.372+0000 7fedd9200640 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a1246fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.372+0000 7fedd9200640 1 -- 172.21.15.90:0/3190213859 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a11408b0 con 0x5631a0ea41b0 2024-05-27T07:55:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.373+0000 7fedd3e00640 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a1246fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.373+0000 7fedd9c00640 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 0x5631a12474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.373+0000 7fedd3e00640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fedbc0b2aa0 0x7fedbc0b4fb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fedc0035040 tx=0x7fedc0030b30 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.373+0000 7fedd9200640 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0ea41b0 0x5631a1246a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009c90 tx=0x7fedc8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a87668054cd895 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.374+0000 7fedc6000640 1 -- 172.21.15.90:0/3190213859 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8004230 con 0x5631a0ea41b0 2024-05-27T07:55:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.374+0000 7fedc6000640 1 -- 172.21.15.90:0/3190213859 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedc80043d0 con 0x5631a0ea41b0 2024-05-27T07:55:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.374+0000 7fedc6000640 1 -- 172.21.15.90:0/3190213859 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8019640 con 0x5631a0ea41b0 2024-05-27T07:55:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.375+0000 7fede2c2d740 1 -- 172.21.15.90:0/3190213859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0ea41b0 msgr2=0x5631a1246a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.375+0000 7fede2c2d740 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0ea41b0 0x5631a1246a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009c90 tx=0x7fedc8011810 comp rx=0 tx=0).stop 2024-05-27T07:55:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.375+0000 7fede2c2d740 1 -- 172.21.15.90:0/3190213859 shutdown_connections 2024-05-27T07:55:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.375+0000 7fede2c2d740 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 0x5631a12474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.375+0000 7fede2c2d740 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a1246fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.375+0000 7fede2c2d740 1 --2- 172.21.15.90:0/3190213859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0ea41b0 0x5631a1246a60 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009c90 tx=0x7fedc8011810 comp rx=0 tx=0).stop 2024-05-27T07:55:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.375+0000 7fede2c2d740 1 -- 172.21.15.90:0/3190213859 >> 172.21.15.90:0/3190213859 conn(0x5631a0e98300 msgr2=0x5631a1243210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.376+0000 7fede2c2d740 1 -- 172.21.15.90:0/3190213859 shutdown_connections 2024-05-27T07:55:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.376+0000 7fede2c2d740 1 -- 172.21.15.90:0/3190213859 wait complete. 2024-05-27T07:55:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.377+0000 7fede2c2d740 1 -- start start 2024-05-27T07:55:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.378+0000 7fede2c2d740 1 Processor -- start 2024-05-27T07:55:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.378+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 0x5631a1250ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.379+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 0x5631a1251420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.379+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e62d80 0x5631a125cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.379+0000 7fede2c2d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5631a0e8d790 con 0x5631a0e62d80 2024-05-27T07:55:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.379+0000 7fede2c2d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5631a114fbc0 con 0x5631a0e7b2f0 2024-05-27T07:55:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.379+0000 7fede2c2d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631a1253be0 con 0x5631a0e834d0 2024-05-27T07:55:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.379+0000 7fede2c2d740 10 client.0 _ll_register_callbacks cb 0x5631a112d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.379+0000 7fedd9c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e62d80 0x5631a125cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.379+0000 7fedd9c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e62d80 0x5631a125cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47210/0 (socket says 172.21.15.90:47210) 2024-05-27T07:55:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.379+0000 7fedd9c00640 1 -- 172.21.15.90:0/3770793873 learned_addr learned my addr 172.21.15.90:0/3770793873 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.379+0000 7fedd9200640 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 0x5631a1250ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.380+0000 7fedd3e00640 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 0x5631a1251420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.380+0000 7fedd9c00640 1 -- 172.21.15.90:0/3770793873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 msgr2=0x5631a1251420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.380+0000 7fedd9c00640 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 0x5631a1251420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.380+0000 7fedd9c00640 1 -- 172.21.15.90:0/3770793873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 msgr2=0x5631a1250ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.380+0000 7fedd9c00640 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 0x5631a1250ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.380+0000 7fedd9c00640 1 -- 172.21.15.90:0/3770793873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a0ebc210 con 0x5631a0e62d80 2024-05-27T07:55:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.381+0000 7fedd9c00640 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e62d80 0x5631a125cf10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0a6f80 tx=0x7fedcc0a98e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b8f89ceeb3a019 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.381+0000 7fedbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:55:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.381+0000 7fedbaa00640 1 -- 172.21.15.90:0/3770793873 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0ae0c0 con 0x5631a0e62d80 2024-05-27T07:55:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.381+0000 7fedbaa00640 1 -- 172.21.15.90:0/3770793873 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7fedcc0b28f0 con 0x5631a0e62d80 2024-05-27T07:55:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.381+0000 7fede2c2d740 1 -- 172.21.15.90:0/3770793873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc80043d0 con 0x5631a0e62d80 2024-05-27T07:55:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.381+0000 7fedbaa00640 1 -- 172.21.15.90:0/3770793873 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fedcc0b2b70 con 0x5631a0e62d80 2024-05-27T07:55:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.381+0000 7fede2c2d740 10 client.17118 fetch_fsmap learned FSMap version 53 2024-05-27T07:55:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.382+0000 7fede2c2d740 1 -- 172.21.15.90:0/3770793873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5631a0f88620 con 0x5631a0e62d80 2024-05-27T07:55:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.382+0000 7fedbaa00640 1 -- 172.21.15.90:0/3770793873 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedc80043d0 con 0x5631a0e62d80 2024-05-27T07:55:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.383+0000 7fedbaa00640 1 -- 172.21.15.90:0/3770793873 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 53) ==== 7152+0+0 (secure 0 0 0) 0x7fedcc0a89e0 con 0x5631a0e62d80 2024-05-27T07:55:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.383+0000 7fede2c2d740 10 client.17118 fetch_fsmap finished waiting for FSMap version 53 2024-05-27T07:55:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.383+0000 7fede2c2d740 10 client.17118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.383+0000 7fede2c2d740 20 client.17118 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.383+0000 7fede2c2d740 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631a0e381c0 0x5631a12630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.383+0000 7fede2c2d740 4 client.17118 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.383+0000 7fede2c2d740 1 -- 172.21.15.90:0/3770793873 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631a11b9ae0 con 0x5631a0e381c0 2024-05-27T07:55:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.384+0000 7fedd9200640 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631a0e381c0 0x5631a12630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.385+0000 7fedd9200640 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631a0e381c0 0x5631a12630c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b35d93ecbf378d server_cookie=15ba7627325cb5dc in_seq=0 out_seq=0 2024-05-27T07:55:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.385+0000 7fedbaa00640 10 client.17118 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.387+0000 7fedbaa00640 1 -- 172.21.15.90:0/3770793873 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5631a11b9ae0 con 0x5631a0e381c0 2024-05-27T07:55:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.387+0000 7fedbaa00640 10 client.17118 handle_command_reply: tid=0 2024-05-27T07:55:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.387+0000 7fede2c2d740 1 client.17118 shutdown 2024-05-27T07:55:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 1 -- 172.21.15.90:0/3770793873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e62d80 msgr2=0x5631a125cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e62d80 0x5631a125cf10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0a6f80 tx=0x7fedcc0a98e0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 1 -- 172.21.15.90:0/3770793873 shutdown_connections 2024-05-27T07:55:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631a0e381c0 0x5631a12630c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e62d80 0x5631a125cf10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e834d0 0x5631a1251420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 1 --2- 172.21.15.90:0/3770793873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 0x5631a1250ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 1 -- 172.21.15.90:0/3770793873 >> 172.21.15.90:0/3770793873 conn(0x5631a0e98300 msgr2=0x5631a12469a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fedbaa00640 0 client.17118 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 1 -- 172.21.15.90:0/3770793873 shutdown_connections 2024-05-27T07:55:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 1 -- 172.21.15.90:0/3770793873 wait complete. 2024-05-27T07:55:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.388+0000 7fede2c2d740 20 client.17118 trim_cache size 0 max 16384 2024-05-27T07:55:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.415+0000 7fede2c2d740 1 Processor -- start 2024-05-27T07:55:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.416+0000 7fede2c2d740 1 -- start start 2024-05-27T07:55:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.416+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e62d80 0x5631a124c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.416+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a12432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.416+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e834d0 0x5631a125cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.416+0000 7fede2c2d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b5fa0 con 0x5631a0e7b2f0 2024-05-27T07:55:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.416+0000 7fede2c2d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b8880 con 0x5631a0e834d0 2024-05-27T07:55:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.416+0000 7fede2c2d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b6670 con 0x5631a0e62d80 2024-05-27T07:55:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.416+0000 7fedd3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a12432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.417+0000 7fedd3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a12432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47212/0 (socket says 172.21.15.90:47212) 2024-05-27T07:55:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.417+0000 7fedd3e00640 1 -- 172.21.15.90:0/59192511 learned_addr learned my addr 172.21.15.90:0/59192511 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.417+0000 7fedd9200640 1 --2- 172.21.15.90:0/59192511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e62d80 0x5631a124c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.417+0000 7fedd9c00640 1 --2- 172.21.15.90:0/59192511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e834d0 0x5631a125cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.417+0000 7fedd3e00640 1 -- 172.21.15.90:0/59192511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e62d80 msgr2=0x5631a124c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.417+0000 7fedd3e00640 1 --2- 172.21.15.90:0/59192511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e62d80 0x5631a124c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.417+0000 7fedd3e00640 1 -- 172.21.15.90:0/59192511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e834d0 msgr2=0x5631a125cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.417+0000 7fedd3e00640 1 --2- 172.21.15.90:0/59192511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e834d0 0x5631a125cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.418+0000 7fedd3e00640 1 -- 172.21.15.90:0/59192511 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a124f220 con 0x5631a0e7b2f0 2024-05-27T07:55:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.419+0000 7fedd3e00640 1 --2- 172.21.15.90:0/59192511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a12432e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fedc0034000 tx=0x7fedc0038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c086fc9aba4b6a22 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.419+0000 7fedc5600640 1 -- 172.21.15.90:0/59192511 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0037040 con 0x5631a0e7b2f0 2024-05-27T07:55:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.419+0000 7fedc5600640 1 -- 172.21.15.90:0/59192511 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedc0051070 con 0x5631a0e7b2f0 2024-05-27T07:55:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.419+0000 7fedc5600640 1 -- 172.21.15.90:0/59192511 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0004c50 con 0x5631a0e7b2f0 2024-05-27T07:55:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.421+0000 7fede2c2d740 1 -- 172.21.15.90:0/59192511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 msgr2=0x5631a12432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.421+0000 7fede2c2d740 1 --2- 172.21.15.90:0/59192511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a12432e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fedc0034000 tx=0x7fedc0038000 comp rx=0 tx=0).stop 2024-05-27T07:55:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.421+0000 7fede2c2d740 1 -- 172.21.15.90:0/59192511 shutdown_connections 2024-05-27T07:55:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.421+0000 7fede2c2d740 1 --2- 172.21.15.90:0/59192511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e834d0 0x5631a125cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.421+0000 7fede2c2d740 1 --2- 172.21.15.90:0/59192511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e7b2f0 0x5631a12432e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.421+0000 7fede2c2d740 1 --2- 172.21.15.90:0/59192511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e62d80 0x5631a124c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.421+0000 7fede2c2d740 1 -- 172.21.15.90:0/59192511 >> 172.21.15.90:0/59192511 conn(0x5631a0e98300 msgr2=0x5631a1242bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.423+0000 7fede2c2d740 1 -- 172.21.15.90:0/59192511 shutdown_connections 2024-05-27T07:55:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.423+0000 7fede2c2d740 1 -- 172.21.15.90:0/59192511 wait complete. 2024-05-27T07:55:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.424+0000 7fede2c2d740 1 -- start start 2024-05-27T07:55:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.424+0000 7fede2c2d740 1 Processor -- start 2024-05-27T07:55:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.425+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e62d80 0x5631a12c7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.425+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 0x5631a1258e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.425+0000 7fede2c2d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e834d0 0x5631a12593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.425+0000 7fede2c2d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b6d40 con 0x5631a0e834d0 2024-05-27T07:55:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.425+0000 7fede2c2d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b81b0 con 0x5631a0e7b2f0 2024-05-27T07:55:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.425+0000 7fede2c2d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fedd41b7410 con 0x5631a0e62d80 2024-05-27T07:55:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.425+0000 7fede2c2d740 10 client.0 _ll_register_callbacks cb 0x7fedcc00c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.426+0000 7fedd9c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e834d0 0x5631a12593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.426+0000 7fedd9200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e62d80 0x5631a12c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.426+0000 7fedd9c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e834d0 0x5631a12593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47226/0 (socket says 172.21.15.90:47226) 2024-05-27T07:55:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.426+0000 7fedd9c00640 1 -- 172.21.15.90:0/1792026590 learned_addr learned my addr 172.21.15.90:0/1792026590 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.426+0000 7fedd3e00640 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 0x5631a1258e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.427+0000 7fedd3e00640 1 -- 172.21.15.90:0/1792026590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e62d80 msgr2=0x5631a12c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.427+0000 7fedd3e00640 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e62d80 0x5631a12c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.427+0000 7fedd3e00640 1 -- 172.21.15.90:0/1792026590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e834d0 msgr2=0x5631a12593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.427+0000 7fedd3e00640 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e834d0 0x5631a12593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.427+0000 7fedd3e00640 1 -- 172.21.15.90:0/1792026590 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a12c4420 con 0x5631a0e7b2f0 2024-05-27T07:55:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.427+0000 7fedd9c00640 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e834d0 0x5631a12593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.427+0000 7fedd3e00640 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 0x5631a1258e70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5631a1252040 tx=0x7fedc0030950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a47a06407c7b860c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.428+0000 7fedbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:55:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.428+0000 7fedbaa00640 1 -- 172.21.15.90:0/1792026590 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0059030 con 0x5631a0e7b2f0 2024-05-27T07:55:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.428+0000 7fedbaa00640 1 -- 172.21.15.90:0/1792026590 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9408+0+0 (secure 0 0 0) 0x7fedc0047070 con 0x5631a0e7b2f0 2024-05-27T07:55:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.428+0000 7fede2c2d740 1 -- 172.21.15.90:0/1792026590 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc0051070 con 0x5631a0e7b2f0 2024-05-27T07:55:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.429+0000 7fedbaa00640 1 -- 172.21.15.90:0/1792026590 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fedc0066050 con 0x5631a0e7b2f0 2024-05-27T07:55:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.429+0000 7fede2c2d740 10 client.36686 fetch_fsmap learned FSMap version 53 2024-05-27T07:55:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.429+0000 7fede2c2d740 1 -- 172.21.15.90:0/1792026590 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5631a12dc880 con 0x5631a0e7b2f0 2024-05-27T07:55:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.430+0000 7fedbaa00640 1 -- 172.21.15.90:0/1792026590 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fedc0051070 con 0x5631a0e7b2f0 2024-05-27T07:55:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.431+0000 7fedbaa00640 1 -- 172.21.15.90:0/1792026590 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 53) ==== 7152+0+0 (secure 0 0 0) 0x7fedc003d040 con 0x5631a0e7b2f0 2024-05-27T07:55:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.431+0000 7fede2c2d740 10 client.36686 fetch_fsmap finished waiting for FSMap version 53 2024-05-27T07:55:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.431+0000 7fede2c2d740 10 client.36686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.431+0000 7fede2c2d740 20 client.36686 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.431+0000 7fede2c2d740 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631a12dcef0 0x5631a12df310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.431+0000 7fede2c2d740 4 client.36686 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:55:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.431+0000 7fede2c2d740 1 -- 172.21.15.90:0/1792026590 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631a11b32c0 con 0x5631a12dcef0 2024-05-27T07:55:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.432+0000 7fedd9200640 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631a12dcef0 0x5631a12df310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.433+0000 7fedd9200640 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631a12dcef0 0x5631a12df310 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368fcd745bb4bc71 server_cookie=dc6217966670478d in_seq=0 out_seq=0 2024-05-27T07:55:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.433+0000 7fedbaa00640 10 client.36686 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:02.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:02 smithi090 ceph-mon[31429]: pgmap v272: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.3 MiB/s wr, 544 op/s 2024-05-27T07:55:02.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:02 smithi090 ceph-mon[31429]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:02.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:02 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:02.467 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4f5c200700 time 2024-05-27T07:55:02.149604+0000 2024-05-27T07:55:02.467 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:02.467 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4f71dccba6] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4f71dccdd1] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55b64678d23d] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55b646778971] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 5: (SnapRealm::check_cache() const+0x13e) [0x55b646776fa0] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55b646777651] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55b64670c347] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55b64640887e] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55b64640df2f] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55b646418969] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 11: (MDLog::_replay_thread()+0x1f38) [0x55b6467b148c] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55b6467ba9cb] 2024-05-27T07:55:02.468 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 13: (Thread::entry_wrapper()+0x43) [0x7f4f71d9b72d] 2024-05-27T07:55:02.469 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 14: (Thread::_entry_func(void*)+0xd) [0x7f4f71d9b749] 2024-05-27T07:55:02.469 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4f7093f1ca] 2024-05-27T07:55:02.469 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 16: clone() 2024-05-27T07:55:02.469 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: *** Caught signal (Aborted) ** 2024-05-27T07:55:02.469 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: in thread 7f4f5c200700 thread_name:md_log_replay 2024-05-27T07:55:02.469 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.155+0000 7f4f5c200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4f5c200700 time 2024-05-27T07:55:02.149604+0000 2024-05-27T07:55:02.469 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:02.469 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.469 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:02.469 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4f71dccba6] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4f71dccdd1] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55b64678d23d] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55b646778971] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 5: (SnapRealm::check_cache() const+0x13e) [0x55b646776fa0] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55b646777651] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55b64670c347] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55b64640887e] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55b64640df2f] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55b646418969] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 11: (MDLog::_replay_thread()+0x1f38) [0x55b6467b148c] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55b6467ba9cb] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 13: (Thread::entry_wrapper()+0x43) [0x7f4f71d9b72d] 2024-05-27T07:55:02.470 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 14: (Thread::_entry_func(void*)+0xd) [0x7f4f71d9b749] 2024-05-27T07:55:02.471 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4f7093f1ca] 2024-05-27T07:55:02.471 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 16: clone() 2024-05-27T07:55:02.471 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.471 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:02.471 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4f70949d20] 2024-05-27T07:55:02.471 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2: gsignal() 2024-05-27T07:55:02.471 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 3: abort() 2024-05-27T07:55:02.471 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4f71dccced] 2024-05-27T07:55:02.472 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4f71dccdd1] 2024-05-27T07:55:02.472 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55b64678d23d] 2024-05-27T07:55:02.472 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55b646778971] 2024-05-27T07:55:02.472 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 8: (SnapRealm::check_cache() const+0x13e) [0x55b646776fa0] 2024-05-27T07:55:02.472 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55b646777651] 2024-05-27T07:55:02.472 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55b64670c347] 2024-05-27T07:55:02.472 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55b64640887e] 2024-05-27T07:55:02.472 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55b64640df2f] 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55b646418969] 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 14: (MDLog::_replay_thread()+0x1f38) [0x55b6467b148c] 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55b6467ba9cb] 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 16: (Thread::entry_wrapper()+0x43) [0x7f4f71d9b72d] 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 17: (Thread::_entry_func(void*)+0xd) [0x7f4f71d9b749] 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4f7093f1ca] 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 19: clone() 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.163+0000 7f4f5c200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: in thread 7f4f5c200700 thread_name:md_log_replay 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:02.473 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4f70949d20] 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2: gsignal() 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 3: abort() 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4f71dccced] 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4f71dccdd1] 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55b64678d23d] 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55b646778971] 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 8: (SnapRealm::check_cache() const+0x13e) [0x55b646776fa0] 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55b646777651] 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55b64670c347] 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55b64640887e] 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55b64640df2f] 2024-05-27T07:55:02.474 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55b646418969] 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 14: (MDLog::_replay_thread()+0x1f38) [0x55b6467b148c] 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55b6467ba9cb] 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 16: (Thread::entry_wrapper()+0x43) [0x7f4f71d9b72d] 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 17: (Thread::_entry_func(void*)+0xd) [0x7f4f71d9b749] 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4f7093f1ca] 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 19: clone() 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: -1> 2024-05-27T07:55:02.155+0000 7f4f5c200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4f5c200700 time 2024-05-27T07:55:02.149604+0000 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:02.475 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4f71dccba6] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4f71dccdd1] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55b64678d23d] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55b646778971] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 5: (SnapRealm::check_cache() const+0x13e) [0x55b646776fa0] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55b646777651] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55b64670c347] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55b64640887e] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55b64640df2f] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55b646418969] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 11: (MDLog::_replay_thread()+0x1f38) [0x55b6467b148c] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55b6467ba9cb] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 13: (Thread::entry_wrapper()+0x43) [0x7f4f71d9b72d] 2024-05-27T07:55:02.476 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 14: (Thread::_entry_func(void*)+0xd) [0x7f4f71d9b749] 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4f7093f1ca] 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 16: clone() 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 0> 2024-05-27T07:55:02.163+0000 7f4f5c200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: in thread 7f4f5c200700 thread_name:md_log_replay 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4f70949d20] 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2: gsignal() 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 3: abort() 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4f71dccced] 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4f71dccdd1] 2024-05-27T07:55:02.477 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55b64678d23d] 2024-05-27T07:55:02.478 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55b646778971] 2024-05-27T07:55:02.478 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 8: (SnapRealm::check_cache() const+0x13e) [0x55b646776fa0] 2024-05-27T07:55:02.478 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55b646777651] 2024-05-27T07:55:02.478 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55b64670c347] 2024-05-27T07:55:02.478 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55b64640887e] 2024-05-27T07:55:02.478 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55b64640df2f] 2024-05-27T07:55:02.478 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55b646418969] 2024-05-27T07:55:02.478 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 14: (MDLog::_replay_thread()+0x1f38) [0x55b6467b148c] 2024-05-27T07:55:02.478 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55b6467ba9cb] 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 16: (Thread::entry_wrapper()+0x43) [0x7f4f71d9b72d] 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 17: (Thread::_entry_func(void*)+0xd) [0x7f4f71d9b749] 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4f7093f1ca] 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 19: clone() 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: -1> 2024-05-27T07:55:02.155+0000 7f4f5c200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f4f5c200700 time 2024-05-27T07:55:02.149604+0000 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:02.479 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f4f71dccba6] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4f71dccdd1] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55b64678d23d] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55b646778971] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 5: (SnapRealm::check_cache() const+0x13e) [0x55b646776fa0] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55b646777651] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55b64670c347] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55b64640887e] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55b64640df2f] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x55b646418969] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 11: (MDLog::_replay_thread()+0x1f38) [0x55b6467b148c] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55b6467ba9cb] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 13: (Thread::entry_wrapper()+0x43) [0x7f4f71d9b72d] 2024-05-27T07:55:02.480 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 14: (Thread::_entry_func(void*)+0xd) [0x7f4f71d9b749] 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f4f7093f1ca] 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 16: clone() 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 0> 2024-05-27T07:55:02.163+0000 7f4f5c200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: in thread 7f4f5c200700 thread_name:md_log_replay 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4f70949d20] 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2: gsignal() 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 3: abort() 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f4f71dccced] 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f4f71dccdd1] 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55b64678d23d] 2024-05-27T07:55:02.481 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55b646778971] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 8: (SnapRealm::check_cache() const+0x13e) [0x55b646776fa0] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55b646777651] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55b64670c347] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55b64640887e] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55b64640df2f] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x55b646418969] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 14: (MDLog::_replay_thread()+0x1f38) [0x55b6467b148c] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55b6467ba9cb] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 16: (Thread::entry_wrapper()+0x43) [0x7f4f71d9b72d] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 17: (Thread::_entry_func(void*)+0xd) [0x7f4f71d9b749] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f4f7093f1ca] 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 19: clone() 2024-05-27T07:55:02.482 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:02.483 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[110354]: 2024-05-27T07:55:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.488+0000 7fedbaa00640 1 -- 172.21.15.90:0/1792026590 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5631a11b32c0 con 0x5631a12dcef0 2024-05-27T07:55:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.488+0000 7fedbaa00640 10 client.36686 handle_command_reply: tid=0 2024-05-27T07:55:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.489+0000 7fede2c2d740 1 client.36686 shutdown 2024-05-27T07:55:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.489+0000 7fede2c2d740 1 -- 172.21.15.90:0/1792026590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 msgr2=0x5631a1258e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.489+0000 7fede2c2d740 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 0x5631a1258e70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x5631a1252040 tx=0x7fedc0030950 comp rx=0 tx=0).stop 2024-05-27T07:55:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.489+0000 7fede2c2d740 1 -- 172.21.15.90:0/1792026590 shutdown_connections 2024-05-27T07:55:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.489+0000 7fede2c2d740 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631a12dcef0 0x5631a12df310 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.489+0000 7fede2c2d740 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631a0e7b2f0 0x5631a1258e70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.489+0000 7fede2c2d740 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631a0e834d0 0x5631a12593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.489+0000 7fede2c2d740 1 --2- 172.21.15.90:0/1792026590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631a0e62d80 0x5631a12c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.489+0000 7fedbaa00640 0 client.36686 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.489+0000 7fede2c2d740 1 -- 172.21.15.90:0/1792026590 >> 172.21.15.90:0/1792026590 conn(0x5631a0e98300 msgr2=0x5631a124be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.490+0000 7fede2c2d740 1 -- 172.21.15.90:0/1792026590 shutdown_connections 2024-05-27T07:55:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.490+0000 7fede2c2d740 1 -- 172.21.15.90:0/1792026590 wait complete. 2024-05-27T07:55:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.491+0000 7fede2c2d740 20 client.36686 trim_cache size 0 max 16384 2024-05-27T07:55:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:55:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:55:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:55:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.493+0000 7fedda600640 1 -- 172.21.15.90:0/2252735866 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fedbc0b2aa0 msgr2=0x7fedbc0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.493+0000 7fedda600640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fedbc0b2aa0 0x7fedbc0b4fb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fedc0035040 tx=0x7fedc0030b30 comp rx=0 tx=0).stop 2024-05-27T07:55:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.493+0000 7fedda600640 1 -- 172.21.15.90:0/2252735866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd4114920 msgr2=0x7fedd41a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.493+0000 7fedda600640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd4114920 0x7fedd41a0a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00eb10 tx=0x7fedcc00ef10 comp rx=0 tx=0).stop 2024-05-27T07:55:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.494+0000 7fedda600640 1 -- 172.21.15.90:0/2252735866 shutdown_connections 2024-05-27T07:55:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.494+0000 7fedda600640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fedbc0b2aa0 0x7fedbc0b4fb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.494+0000 7fedda600640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd4114920 0x7fedd41a0a40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.494+0000 7fedda600640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd4111fc0 0x7fedd41a6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.494+0000 7fedda600640 1 --2- 172.21.15.90:0/2252735866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fedd4077610 0x7fedd41a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.494+0000 7fedda600640 1 -- 172.21.15.90:0/2252735866 >> 172.21.15.90:0/2252735866 conn(0x7fedd41013d0 msgr2=0x7fedd41017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.494+0000 7fedda600640 1 -- 172.21.15.90:0/2252735866 shutdown_connections 2024-05-27T07:55:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:02.494+0000 7fedda600640 1 -- 172.21.15.90:0/2252735866 wait complete. 2024-05-27T07:55:02.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:02 smithi129 ceph-mon[35254]: pgmap v272: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.3 MiB/s wr, 544 op/s 2024-05-27T07:55:02.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:02 smithi129 ceph-mon[35254]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:02.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:02 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:02.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:02 smithi136 ceph-mon[35324]: pgmap v272: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.3 MiB/s wr, 544 op/s 2024-05-27T07:55:02.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:02 smithi136 ceph-mon[35324]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:02.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:02 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:03.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:03 smithi129 ceph-mon[35254]: pgmap v273: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 5.8 MiB/s wr, 626 op/s 2024-05-27T07:55:03.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:03 smithi090 ceph-mon[31429]: pgmap v273: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 5.8 MiB/s wr, 626 op/s 2024-05-27T07:55:03.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:03 smithi136 ceph-mon[35324]: pgmap v273: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 5.8 MiB/s wr, 626 op/s 2024-05-27T07:55:03.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 285.0 s, 538.0 tps, lat 171.274 ms stddev 92.727 2024-05-27T07:55:04.315 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:04 smithi090 podman[112809]: 2024-05-27 07:55:04.282570215 +0000 UTC m=+2.041940817 container died 9c1c3a7308a298dabb331de08e212e7894d56e08db6380b4b317828260807ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:55:05.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:05 smithi090 ceph-mon[31429]: osdmap e105: 12 total, 12 up, 12 in 2024-05-27T07:55:05.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:05 smithi090 ceph-mon[31429]: pgmap v275: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 6.4 MiB/s wr, 738 op/s 2024-05-27T07:55:05.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:05 smithi136 ceph-mon[35324]: osdmap e105: 12 total, 12 up, 12 in 2024-05-27T07:55:05.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:05 smithi136 ceph-mon[35324]: pgmap v275: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 6.4 MiB/s wr, 738 op/s 2024-05-27T07:55:05.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:05 smithi129 ceph-mon[35254]: osdmap e105: 12 total, 12 up, 12 in 2024-05-27T07:55:05.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:05 smithi129 ceph-mon[35254]: pgmap v275: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 6.4 MiB/s wr, 738 op/s 2024-05-27T07:55:06.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:05 smithi090 podman[112809]: 2024-05-27 07:55:05.939683885 +0000 UTC m=+3.699054462 container remove 9c1c3a7308a298dabb331de08e212e7894d56e08db6380b4b317828260807ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-05-27T07:55:06.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:05 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:55:06.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.440+0000 7fd60ba00700 1 -- 172.21.15.90:0/2273343339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd60410a920 msgr2=0x7fd60410ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.440+0000 7fd60ba00700 1 --2- 172.21.15.90:0/2273343339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd60410a920 0x7fd60410ad30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009990 tx=0x7fd5f8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.441+0000 7fd60ba00700 1 -- 172.21.15.90:0/2273343339 shutdown_connections 2024-05-27T07:55:06.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.441+0000 7fd60ba00700 1 --2- 172.21.15.90:0/2273343339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd60411d920 0x7fd60411fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.441+0000 7fd60ba00700 1 --2- 172.21.15.90:0/2273343339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd60410a920 0x7fd60410ad30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.441+0000 7fd60ba00700 1 --2- 172.21.15.90:0/2273343339 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd60410b270 0x7fd60411d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.441+0000 7fd60ba00700 1 -- 172.21.15.90:0/2273343339 >> 172.21.15.90:0/2273343339 conn(0x7fd60406f4c0 msgr2=0x7fd60406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.441+0000 7fd60ba00700 1 -- 172.21.15.90:0/2273343339 shutdown_connections 2024-05-27T07:55:06.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.441+0000 7fd60ba00700 1 -- 172.21.15.90:0/2273343339 wait complete. 2024-05-27T07:55:06.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.443+0000 7fd60ba00700 1 Processor -- start 2024-05-27T07:55:06.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.445+0000 7fd60ba00700 1 -- start start 2024-05-27T07:55:06.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.446+0000 7fd60ba00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd60410a920 0x7fd6041af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.446+0000 7fd60ba00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd60410b270 0x7fd6041afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.446+0000 7fd60ba00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd60411d920 0x7fd6041a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.446+0000 7fd60ba00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd604123200 con 0x7fd60410b270 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.446+0000 7fd60ba00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd604123080 con 0x7fd60411d920 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.446+0000 7fd60ba00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd604123380 con 0x7fd60410a920 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.449+0000 7fd609000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd60410a920 0x7fd6041af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.449+0000 7fd609000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd60410a920 0x7fd6041af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:35122/0 (socket says 172.21.15.90:35122) 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.449+0000 7fd609000700 1 -- 172.21.15.90:0/3321197569 learned_addr learned my addr 172.21.15.90:0/3321197569 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.451+0000 7fd609a00700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd60411d920 0x7fd6041a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.451+0000 7fd603e00700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd60410b270 0x7fd6041afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.452+0000 7fd609000700 1 -- 172.21.15.90:0/3321197569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd60411d920 msgr2=0x7fd6041a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.452+0000 7fd609000700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd60411d920 0x7fd6041a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.452+0000 7fd609000700 1 -- 172.21.15.90:0/3321197569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd60410b270 msgr2=0x7fd6041afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.452+0000 7fd609000700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd60410b270 0x7fd6041afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.452+0000 7fd609000700 1 -- 172.21.15.90:0/3321197569 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6041aa150 con 0x7fd60410a920 2024-05-27T07:55:06.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.454+0000 7fd609000700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd60410a920 0x7fd6041af880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80057c0 tx=0x7fd5f80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.455+0000 7fd601600700 1 -- 172.21.15.90:0/3321197569 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8026070 con 0x7fd60410a920 2024-05-27T07:55:06.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.455+0000 7fd60ba00700 1 -- 172.21.15.90:0/3321197569 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6041aa3e0 con 0x7fd60410a920 2024-05-27T07:55:06.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.455+0000 7fd60ba00700 1 -- 172.21.15.90:0/3321197569 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6041b6710 con 0x7fd60410a920 2024-05-27T07:55:06.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.455+0000 7fd601600700 1 -- 172.21.15.90:0/3321197569 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5f8021410 con 0x7fd60410a920 2024-05-27T07:55:06.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.455+0000 7fd601600700 1 -- 172.21.15.90:0/3321197569 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8030060 con 0x7fd60410a920 2024-05-27T07:55:06.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.462+0000 7fd601600700 1 -- 172.21.15.90:0/3321197569 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd5f8029070 con 0x7fd60410a920 2024-05-27T07:55:06.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.464+0000 7fd601600700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd5ec0b2cf0 0x7fd5ec0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.464+0000 7fd601600700 1 -- 172.21.15.90:0/3321197569 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9584+0+0 (secure 0 0 0) 0x7fd5f80e4190 con 0x7fd60410a920 2024-05-27T07:55:06.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.466+0000 7fd61a58ab80 1 Processor -- start 2024-05-27T07:55:06.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.467+0000 7fd603e00700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd5ec0b2cf0 0x7fd5ec0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.468+0000 7fd61a58ab80 1 -- start start 2024-05-27T07:55:06.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.468+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdb97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.468+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdb3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.468+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdb41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.470+0000 7fd603e00700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd5ec0b2cf0 0x7fd5ec0b5200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd60411fe00 tx=0x7fd5f0009190 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.470+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dc7fce1e40 con 0x55dc7fc34e40 2024-05-27T07:55:06.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.470+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc7fcd19e0 con 0x55dc7fdb3db0 2024-05-27T07:55:06.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.470+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc7fcd5a20 con 0x55dc7fdb9d30 2024-05-27T07:55:06.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.470+0000 7fd603e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdb3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.470+0000 7fd609000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdb97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.470+0000 7fd609a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdb41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.470+0000 7fd609a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdb41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34906/0 (socket says 172.21.15.90:34906) 2024-05-27T07:55:06.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.470+0000 7fd609a00700 1 -- 172.21.15.90:0/837161887 learned_addr learned my addr 172.21.15.90:0/837161887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:06.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.471+0000 7fd609a00700 1 -- 172.21.15.90:0/837161887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 msgr2=0x55dc7fdb3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.471+0000 7fd609a00700 1 --2- 172.21.15.90:0/837161887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdb3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.471+0000 7fd609a00700 1 -- 172.21.15.90:0/837161887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 msgr2=0x55dc7fdb97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.471+0000 7fd609a00700 1 --2- 172.21.15.90:0/837161887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdb97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.471+0000 7fd609a00700 1 -- 172.21.15.90:0/837161887 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7fc9a420 con 0x55dc7fdb3db0 2024-05-27T07:55:06.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.472+0000 7fd609000700 1 --2- 172.21.15.90:0/837161887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdb97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:06.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.474+0000 7fd609a00700 1 --2- 172.21.15.90:0/837161887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdb41c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0093a0 tx=0x7fd5fc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffa385df12c9e977 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.475+0000 7fd5f6000700 1 -- 172.21.15.90:0/837161887 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc003c00 con 0x55dc7fdb3db0 2024-05-27T07:55:06.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.476+0000 7fd5f6000700 1 -- 172.21.15.90:0/837161887 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5fc019070 con 0x55dc7fdb3db0 2024-05-27T07:55:06.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.476+0000 7fd5f6000700 1 -- 172.21.15.90:0/837161887 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc003c00 con 0x55dc7fdb3db0 2024-05-27T07:55:06.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.477+0000 7fd61a58ab80 1 -- 172.21.15.90:0/837161887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 msgr2=0x55dc7fdb41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.477+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/837161887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdb41c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0093a0 tx=0x7fd5fc009920 comp rx=0 tx=0).stop 2024-05-27T07:55:06.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.477+0000 7fd61a58ab80 1 -- 172.21.15.90:0/837161887 shutdown_connections 2024-05-27T07:55:06.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.477+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/837161887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdb41c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.477+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/837161887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdb97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.477+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/837161887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdb3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.477+0000 7fd61a58ab80 1 -- 172.21.15.90:0/837161887 >> 172.21.15.90:0/837161887 conn(0x55dc7fc94fc0 msgr2=0x55dc7fdae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.478+0000 7fd61a58ab80 1 -- 172.21.15.90:0/837161887 shutdown_connections 2024-05-27T07:55:06.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.479+0000 7fd61a58ab80 1 -- 172.21.15.90:0/837161887 wait complete. 2024-05-27T07:55:06.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.480+0000 7fd61a58ab80 1 -- start start 2024-05-27T07:55:06.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.480+0000 7fd61a58ab80 1 Processor -- start 2024-05-27T07:55:06.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.482+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdbd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.482+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdbdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.482+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.483+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dc7fbd7fa0 con 0x55dc7fc34e40 2024-05-27T07:55:06.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.483+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc7fc72760 con 0x55dc7fdb3db0 2024-05-27T07:55:06.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.483+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc7fdc0800 con 0x55dc7fdb9d30 2024-05-27T07:55:06.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.483+0000 7fd61a58ab80 10 client.0 _ll_register_callbacks cb 0x55dc7fc7bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:06.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.483+0000 7fd603e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdbdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.483+0000 7fd603e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdbdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34916/0 (socket says 172.21.15.90:34916) 2024-05-27T07:55:06.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.483+0000 7fd609a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.483+0000 7fd603e00700 1 -- 172.21.15.90:0/3436900867 learned_addr learned my addr 172.21.15.90:0/3436900867 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:06.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.483+0000 7fd609000700 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdbd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.484+0000 7fd603e00700 1 -- 172.21.15.90:0/3436900867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 msgr2=0x55dc7fdbe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.484+0000 7fd603e00700 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.484+0000 7fd603e00700 1 -- 172.21.15.90:0/3436900867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 msgr2=0x55dc7fdbd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.484+0000 7fd603e00700 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdbd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.484+0000 7fd603e00700 1 -- 172.21.15.90:0/3436900867 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7fc218d0 con 0x55dc7fdb3db0 2024-05-27T07:55:06.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.484+0000 7fd609a00700 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:06.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.485+0000 7fd603e00700 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdbdac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0015700 tx=0x7fd5f0004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1720adca689c309 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.485+0000 7fd5eaa00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:55:06.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.485+0000 7fd5eaa00700 1 -- 172.21.15.90:0/3436900867 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0025020 con 0x55dc7fdb3db0 2024-05-27T07:55:06.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.486+0000 7fd5eaa00700 1 -- 172.21.15.90:0/3436900867 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9584+0+0 (secure 0 0 0) 0x7fd5f000f040 con 0x55dc7fdb3db0 2024-05-27T07:55:06.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.486+0000 7fd61a58ab80 1 -- 172.21.15.90:0/3436900867 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5fc019070 con 0x55dc7fdb3db0 2024-05-27T07:55:06.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.487+0000 7fd5eaa00700 1 -- 172.21.15.90:0/3436900867 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fd5f0030050 con 0x55dc7fdb3db0 2024-05-27T07:55:06.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.487+0000 7fd61a58ab80 10 client.36695 fetch_fsmap learned FSMap version 53 2024-05-27T07:55:06.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.487+0000 7fd61a58ab80 1 -- 172.21.15.90:0/3436900867 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55dc7fca2060 con 0x55dc7fdb3db0 2024-05-27T07:55:06.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.488+0000 7fd5eaa00700 1 -- 172.21.15.90:0/3436900867 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5fc019070 con 0x55dc7fdb3db0 2024-05-27T07:55:06.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.489+0000 7fd5eaa00700 1 -- 172.21.15.90:0/3436900867 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 53) ==== 7152+0+0 (secure 0 0 0) 0x7fd5f0021bb0 con 0x55dc7fdb3db0 2024-05-27T07:55:06.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.489+0000 7fd61a58ab80 10 client.36695 fetch_fsmap finished waiting for FSMap version 53 2024-05-27T07:55:06.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.490+0000 7fd61a58ab80 10 client.36695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:06.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.490+0000 7fd61a58ab80 20 client.36695 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:06.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.490+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc7fdcd5d0 0x55dc7fdcfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.490+0000 7fd61a58ab80 4 client.36695 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:06.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.490+0000 7fd61a58ab80 1 -- 172.21.15.90:0/3436900867 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc7fce3d90 con 0x55dc7fdcd5d0 2024-05-27T07:55:06.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.493+0000 7fd609000700 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc7fdcd5d0 0x55dc7fdcfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.494+0000 7fd609000700 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc7fdcd5d0 0x55dc7fdcfa10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8682afa5c1e0a61 server_cookie=db0cc5bedae764a in_seq=0 out_seq=0 2024-05-27T07:55:06.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.494+0000 7fd5eaa00700 10 client.36695 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:06.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.495+0000 7fd5eaa00700 1 -- 172.21.15.90:0/3436900867 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dc7fce3d90 con 0x55dc7fdcd5d0 2024-05-27T07:55:06.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.495+0000 7fd5eaa00700 10 client.36695 handle_command_reply: tid=0 2024-05-27T07:55:06.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.495+0000 7fd61a58ab80 1 client.36695 shutdown 2024-05-27T07:55:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.499+0000 7fd61a58ab80 1 -- 172.21.15.90:0/3436900867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 msgr2=0x55dc7fdbdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.499+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdbdac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0015700 tx=0x7fd5f0004860 comp rx=0 tx=0).stop 2024-05-27T07:55:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.499+0000 7fd61a58ab80 1 -- 172.21.15.90:0/3436900867 shutdown_connections 2024-05-27T07:55:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.499+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc7fdcd5d0 0x55dc7fdcfa10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.499+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.499+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdbd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.499+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/3436900867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdb3db0 0x55dc7fdbdac0 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0015700 tx=0x7fd5f0004860 comp rx=0 tx=0).stop 2024-05-27T07:55:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.499+0000 7fd61a58ab80 1 -- 172.21.15.90:0/3436900867 >> 172.21.15.90:0/3436900867 conn(0x55dc7fc94fc0 msgr2=0x55dc7fdae580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.500+0000 7fd5eaa00700 0 client.36695 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.504+0000 7fd61a58ab80 1 -- 172.21.15.90:0/3436900867 shutdown_connections 2024-05-27T07:55:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.505+0000 7fd61a58ab80 1 -- 172.21.15.90:0/3436900867 wait complete. 2024-05-27T07:55:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.505+0000 7fd61a58ab80 20 client.36695 trim_cache size 0 max 16384 2024-05-27T07:55:06.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.545+0000 7fd61a58ab80 1 Processor -- start 2024-05-27T07:55:06.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.545+0000 7fd61a58ab80 1 -- start start 2024-05-27T07:55:06.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.546+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdb3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.546+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.546+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdbdf70 0x55dc7fdbf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.546+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd6041be040 con 0x55dc7fc34e40 2024-05-27T07:55:06.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.546+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6041c0920 con 0x55dc7fdbdf70 2024-05-27T07:55:06.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.546+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6041be710 con 0x55dc7fdb9d30 2024-05-27T07:55:06.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.546+0000 7fd603e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.547+0000 7fd603e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:35146/0 (socket says 172.21.15.90:35146) 2024-05-27T07:55:06.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.547+0000 7fd603e00700 1 -- 172.21.15.90:0/823451611 learned_addr learned my addr 172.21.15.90:0/823451611 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:06.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.547+0000 7fd609a00700 1 --2- 172.21.15.90:0/823451611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdbdf70 0x55dc7fdbf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.547+0000 7fd609000700 1 --2- 172.21.15.90:0/823451611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdb3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.547+0000 7fd603e00700 1 -- 172.21.15.90:0/823451611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdbdf70 msgr2=0x55dc7fdbf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.547+0000 7fd603e00700 1 --2- 172.21.15.90:0/823451611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdbdf70 0x55dc7fdbf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.547+0000 7fd603e00700 1 -- 172.21.15.90:0/823451611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 msgr2=0x55dc7fdb3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.547+0000 7fd603e00700 1 --2- 172.21.15.90:0/823451611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdb3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.547+0000 7fd603e00700 1 -- 172.21.15.90:0/823451611 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7fdb41c0 con 0x55dc7fdb9d30 2024-05-27T07:55:06.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.548+0000 7fd603e00700 1 --2- 172.21.15.90:0/823451611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbc560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0013610 tx=0x7fd5f0006e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9b666c9455c288 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.550+0000 7fd5f5600700 1 -- 172.21.15.90:0/823451611 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0038070 con 0x55dc7fdb9d30 2024-05-27T07:55:06.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.550+0000 7fd5f5600700 1 -- 172.21.15.90:0/823451611 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5f0013a40 con 0x55dc7fdb9d30 2024-05-27T07:55:06.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.550+0000 7fd5f5600700 1 -- 172.21.15.90:0/823451611 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0025020 con 0x55dc7fdb9d30 2024-05-27T07:55:06.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.552+0000 7fd61a58ab80 1 -- 172.21.15.90:0/823451611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 msgr2=0x55dc7fdbc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.552+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/823451611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbc560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0013610 tx=0x7fd5f0006e10 comp rx=0 tx=0).stop 2024-05-27T07:55:06.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.553+0000 7fd61a58ab80 1 -- 172.21.15.90:0/823451611 shutdown_connections 2024-05-27T07:55:06.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.553+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/823451611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fdbdf70 0x55dc7fdbf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.553+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/823451611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fc34e40 0x55dc7fdb3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.553+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/823451611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdbc560 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.553+0000 7fd61a58ab80 1 -- 172.21.15.90:0/823451611 >> 172.21.15.90:0/823451611 conn(0x55dc7fc94fc0 msgr2=0x55dc7fdb7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.554+0000 7fd61a58ab80 1 -- 172.21.15.90:0/823451611 shutdown_connections 2024-05-27T07:55:06.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.554+0000 7fd61a58ab80 1 -- 172.21.15.90:0/823451611 wait complete. 2024-05-27T07:55:06.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.555+0000 7fd61a58ab80 1 -- start start 2024-05-27T07:55:06.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.555+0000 7fd61a58ab80 1 Processor -- start 2024-05-27T07:55:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.560+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fc34e40 0x55dc7fdc5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.560+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdd47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.560+0000 7fd61a58ab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fdbdf70 0x55dc7fdd4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.561+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd6041bede0 con 0x55dc7fdbdf70 2024-05-27T07:55:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.561+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6041c0250 con 0x55dc7fc34e40 2024-05-27T07:55:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.561+0000 7fd61a58ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd6041bf4b0 con 0x55dc7fdb9d30 2024-05-27T07:55:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.561+0000 7fd609000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fc34e40 0x55dc7fdc5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.561+0000 7fd609000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fc34e40 0x55dc7fdc5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34924/0 (socket says 172.21.15.90:34924) 2024-05-27T07:55:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.561+0000 7fd609000700 1 -- 172.21.15.90:0/2712700609 learned_addr learned my addr 172.21.15.90:0/2712700609 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:06.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.561+0000 7fd603e00700 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdd47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.561+0000 7fd61a58ab80 10 client.0 _ll_register_callbacks cb 0x55dc7fc7bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:06.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.562+0000 7fd609000700 1 -- 172.21.15.90:0/2712700609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 msgr2=0x55dc7fdd47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.563+0000 7fd609000700 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdd47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.563+0000 7fd609000700 1 -- 172.21.15.90:0/2712700609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fdbdf70 msgr2=0x55dc7fdd4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:06.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.562+0000 7fd609a00700 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fdbdf70 0x55dc7fdd4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.563+0000 7fd609000700 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fdbdf70 0x55dc7fdd4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.563+0000 7fd609000700 1 -- 172.21.15.90:0/2712700609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7fdd5260 con 0x55dc7fc34e40 2024-05-27T07:55:06.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.564+0000 7fd603e00700 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdd47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:06.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.564+0000 7fd609a00700 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fdbdf70 0x55dc7fdd4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:06.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.565+0000 7fd609000700 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fc34e40 0x55dc7fdc5e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800c010 tx=0x7fd5f80e19e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e95e8ac2d659653c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:06.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.565+0000 7fd5eaa00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:55:06.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.568+0000 7fd5eaa00700 1 -- 172.21.15.90:0/2712700609 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8026040 con 0x55dc7fc34e40 2024-05-27T07:55:06.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.568+0000 7fd5eaa00700 1 -- 172.21.15.90:0/2712700609 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9584+0+0 (secure 0 0 0) 0x7fd5f80e1de0 con 0x55dc7fc34e40 2024-05-27T07:55:06.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.568+0000 7fd61a58ab80 1 -- 172.21.15.90:0/2712700609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f0013a40 con 0x55dc7fc34e40 2024-05-27T07:55:06.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.569+0000 7fd5eaa00700 1 -- 172.21.15.90:0/2712700609 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fd5f80ab7a0 con 0x55dc7fc34e40 2024-05-27T07:55:06.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.569+0000 7fd61a58ab80 10 client.36701 fetch_fsmap learned FSMap version 53 2024-05-27T07:55:06.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.569+0000 7fd61a58ab80 1 -- 172.21.15.90:0/2712700609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55dc7fe37cb0 con 0x55dc7fc34e40 2024-05-27T07:55:06.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.569+0000 7fd5eaa00700 1 -- 172.21.15.90:0/2712700609 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd5f0013a40 con 0x55dc7fc34e40 2024-05-27T07:55:06.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.570+0000 7fd5eaa00700 1 -- 172.21.15.90:0/2712700609 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 53) ==== 7152+0+0 (secure 0 0 0) 0x7fd5f80b39e0 con 0x55dc7fc34e40 2024-05-27T07:55:06.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.570+0000 7fd61a58ab80 10 client.36701 fetch_fsmap finished waiting for FSMap version 53 2024-05-27T07:55:06.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.570+0000 7fd61a58ab80 10 client.36701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:06.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.570+0000 7fd61a58ab80 20 client.36701 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:06.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.570+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc7fe382d0 0x55dc7fe3a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:06.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.570+0000 7fd61a58ab80 4 client.36701 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:55:06.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.570+0000 7fd61a58ab80 1 -- 172.21.15.90:0/2712700609 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc7fcdd570 con 0x55dc7fe382d0 2024-05-27T07:55:06.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.571+0000 7fd603e00700 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc7fe382d0 0x55dc7fe3a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:06.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.572+0000 7fd5eaa00700 10 client.36701 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:06.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.572+0000 7fd603e00700 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc7fe382d0 0x55dc7fe3a6f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dadd8d28a371347 server_cookie=74a31eeb2ffb3276 in_seq=0 out_seq=0 2024-05-27T07:55:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.574+0000 7fd5eaa00700 1 -- 172.21.15.90:0/2712700609 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dc7fcdd570 con 0x55dc7fe382d0 2024-05-27T07:55:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.574+0000 7fd5eaa00700 10 client.36701 handle_command_reply: tid=0 2024-05-27T07:55:06.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.574+0000 7fd61a58ab80 1 client.36701 shutdown 2024-05-27T07:55:06.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.574+0000 7fd61a58ab80 1 -- 172.21.15.90:0/2712700609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fc34e40 msgr2=0x55dc7fdc5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.574+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fc34e40 0x55dc7fdc5e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800c010 tx=0x7fd5f80e19e0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.575+0000 7fd61a58ab80 1 -- 172.21.15.90:0/2712700609 shutdown_connections 2024-05-27T07:55:06.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.575+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc7fe382d0 0x55dc7fe3a6f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.575+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc7fdbdf70 0x55dc7fdd4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.575+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc7fc34e40 0x55dc7fdc5e70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.575+0000 7fd61a58ab80 1 --2- 172.21.15.90:0/2712700609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc7fdb9d30 0x55dc7fdd47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.575+0000 7fd61a58ab80 1 -- 172.21.15.90:0/2712700609 >> 172.21.15.90:0/2712700609 conn(0x55dc7fc94fc0 msgr2=0x55dc7fdbba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.575+0000 7fd5eaa00700 0 client.36701 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:06.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.575+0000 7fd61a58ab80 1 -- 172.21.15.90:0/2712700609 shutdown_connections 2024-05-27T07:55:06.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.575+0000 7fd61a58ab80 1 -- 172.21.15.90:0/2712700609 wait complete. 2024-05-27T07:55:06.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.576+0000 7fd61a58ab80 20 client.36701 trim_cache size 0 max 16384 2024-05-27T07:55:06.578 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:55:06.578 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:55:06.578 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:55:06.578 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:55:06.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.577+0000 7fd60ba00700 1 -- 172.21.15.90:0/3321197569 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd5ec0b2cf0 msgr2=0x7fd5ec0b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.577+0000 7fd60ba00700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd5ec0b2cf0 0x7fd5ec0b5200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd60411fe00 tx=0x7fd5f0009190 comp rx=0 tx=0).stop 2024-05-27T07:55:06.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.578+0000 7fd60ba00700 1 -- 172.21.15.90:0/3321197569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd60410a920 msgr2=0x7fd6041af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:06.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.578+0000 7fd60ba00700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd60410a920 0x7fd6041af880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80057c0 tx=0x7fd5f80057f0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.579+0000 7fd60ba00700 1 -- 172.21.15.90:0/3321197569 shutdown_connections 2024-05-27T07:55:06.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.579+0000 7fd60ba00700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd5ec0b2cf0 0x7fd5ec0b5200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.579+0000 7fd60ba00700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd60411d920 0x7fd6041a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.579+0000 7fd60ba00700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd60410a920 0x7fd6041af880 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.579+0000 7fd60ba00700 1 --2- 172.21.15.90:0/3321197569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd60410b270 0x7fd6041afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:06.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.579+0000 7fd60ba00700 1 -- 172.21.15.90:0/3321197569 >> 172.21.15.90:0/3321197569 conn(0x7fd60406f4c0 msgr2=0x7fd60411b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:06.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.579+0000 7fd60ba00700 1 -- 172.21.15.90:0/3321197569 shutdown_connections 2024-05-27T07:55:06.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:06.579+0000 7fd60ba00700 1 -- 172.21.15.90:0/3321197569 wait complete. 2024-05-27T07:55:06.713 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:06 smithi090 podman[112917]: 2024-05-27 07:55:06.356585369 +0000 UTC m=+0.067843392 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:55:06.714 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:06 smithi090 podman[112917]: 2024-05-27 07:55:06.49509352 +0000 UTC m=+0.206351584 container create 6a3722d62ee5c36ba1b9824740938cf51259c5f1d43d5546a0b09919c1c2bdd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:55:07.213 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:07 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-05-27T07:55:07.213 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:07 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.528s CPU time. 2024-05-27T07:55:07.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:06 smithi090 podman[112917]: 2024-05-27 07:55:06.85414447 +0000 UTC m=+0.565402512 container init 6a3722d62ee5c36ba1b9824740938cf51259c5f1d43d5546a0b09919c1c2bdd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T07:55:07.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:06 smithi090 podman[112917]: 2024-05-27 07:55:06.869357088 +0000 UTC m=+0.580615105 container start 6a3722d62ee5c36ba1b9824740938cf51259c5f1d43d5546a0b09919c1c2bdd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-27T07:55:07.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:06 smithi090 bash[112917]: 6a3722d62ee5c36ba1b9824740938cf51259c5f1d43d5546a0b09919c1c2bdd3 2024-05-27T07:55:07.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:06 smithi090 systemd[1]: Started Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:55:07.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:07 smithi090 ceph-mds[112999]: -- 172.21.15.90:0/4266964302 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557003fc1a40 con 0x557004009600 2024-05-27T07:55:07.214 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: starting mds.d at 2024-05-27T07:55:07.832 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:55:07.833 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:55:08.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:08 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6834/4022956786,v1:172.21.15.90:6835/4022956786] up:boot 2024-05-27T07:55:08.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:08 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:55:08.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:08 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:55:08.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:08 smithi090 ceph-mon[31429]: pgmap v276: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 6.4 MiB/s wr, 738 op/s 2024-05-27T07:55:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:08 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6834/4022956786,v1:172.21.15.90:6835/4022956786] up:boot 2024-05-27T07:55:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:08 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:55:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:08 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:55:08.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:08 smithi136 ceph-mon[35324]: pgmap v276: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 6.4 MiB/s wr, 738 op/s 2024-05-27T07:55:08.550 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:08.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:08 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6834/4022956786,v1:172.21.15.90:6835/4022956786] up:boot 2024-05-27T07:55:08.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:08 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:55:08.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:08 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T07:55:08.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:08 smithi129 ceph-mon[35254]: pgmap v276: 129 pgs: 129 active+clean; 2.7 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 6.4 MiB/s wr, 738 op/s 2024-05-27T07:55:08.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 290.0 s, 555.8 tps, lat 194.972 ms stddev 150.797 2024-05-27T07:55:10.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:10 smithi136 ceph-mon[35324]: pgmap v277: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 5.4 MiB/s wr, 594 op/s 2024-05-27T07:55:10.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:10 smithi129 ceph-mon[35254]: pgmap v277: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 5.4 MiB/s wr, 594 op/s 2024-05-27T07:55:10.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:10 smithi090 ceph-mon[31429]: pgmap v277: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 5.4 MiB/s wr, 594 op/s 2024-05-27T07:55:11.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:11 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:55:11.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:11 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:55:11.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:11 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:55:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:12 smithi129 ceph-mon[35254]: pgmap v278: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.8 MiB/s wr, 632 op/s 2024-05-27T07:55:12.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:12 smithi129 ceph-mon[35254]: osdmap e106: 12 total, 12 up, 12 in 2024-05-27T07:55:12.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:12 smithi090 ceph-mon[31429]: pgmap v278: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.8 MiB/s wr, 632 op/s 2024-05-27T07:55:12.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:12 smithi090 ceph-mon[31429]: osdmap e106: 12 total, 12 up, 12 in 2024-05-27T07:55:12.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:12 smithi136 ceph-mon[35324]: pgmap v278: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 5.8 MiB/s wr, 632 op/s 2024-05-27T07:55:12.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:12 smithi136 ceph-mon[35324]: osdmap e106: 12 total, 12 up, 12 in 2024-05-27T07:55:13.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 295.0 s, 489.0 tps, lat 198.498 ms stddev 177.078 2024-05-27T07:55:14.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:14 smithi090 ceph-mon[31429]: pgmap v280: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.6 MiB/s wr, 361 op/s 2024-05-27T07:55:14.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:14 smithi129 ceph-mon[35254]: pgmap v280: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.6 MiB/s wr, 361 op/s 2024-05-27T07:55:14.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:14 smithi136 ceph-mon[35324]: pgmap v280: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.6 MiB/s wr, 361 op/s 2024-05-27T07:55:15.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:15 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:55:15.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:15 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:55:15.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:15 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:55:15.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:15 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:55:15.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:15 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:55:15.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:15 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:55:15.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:15 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:55:15.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:15 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:55:15.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:15 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:55:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.877+0000 7fdcc4c00640 1 -- 172.21.15.90:0/1913043083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc0102ab0 msgr2=0x7fdcc0102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.877+0000 7fdcc4c00640 1 --2- 172.21.15.90:0/1913043083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc0102ab0 0x7fdcc0102ec0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009900 tx=0x7fdcac02a400 comp rx=0 tx=0).stop 2024-05-27T07:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.878+0000 7fdcc4c00640 1 -- 172.21.15.90:0/1913043083 shutdown_connections 2024-05-27T07:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.878+0000 7fdcc4c00640 1 --2- 172.21.15.90:0/1913043083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcc010d340 0x7fdcc010f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.878+0000 7fdcc4c00640 1 --2- 172.21.15.90:0/1913043083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc0102ab0 0x7fdcc0102ec0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.878+0000 7fdcc4c00640 1 --2- 172.21.15.90:0/1913043083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc01089e0 0x7fdcc0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.879+0000 7fdcc4c00640 1 -- 172.21.15.90:0/1913043083 >> 172.21.15.90:0/1913043083 conn(0x7fdcc00fe210 msgr2=0x7fdcc0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.879+0000 7fdcc4c00640 1 -- 172.21.15.90:0/1913043083 shutdown_connections 2024-05-27T07:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.879+0000 7fdcc4c00640 1 -- 172.21.15.90:0/1913043083 wait complete. 2024-05-27T07:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.881+0000 7fdcc4c00640 1 Processor -- start 2024-05-27T07:55:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.881+0000 7fdcc4c00640 1 -- start start 2024-05-27T07:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.882+0000 7fdcc4c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc0102ab0 0x7fdcc01173a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.882+0000 7fdcc4c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcc01089e0 0x7fdcc01134d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.882+0000 7fdcc4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc010d340 0x7fdcc0113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.882+0000 7fdcc4c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01119a0 con 0x7fdcc010d340 2024-05-27T07:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.882+0000 7fdcc4c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111820 con 0x7fdcc0102ab0 2024-05-27T07:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.882+0000 7fdcc4c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111b20 con 0x7fdcc01089e0 2024-05-27T07:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.882+0000 7fdcbfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc010d340 0x7fdcc0113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.882+0000 7fdcbfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc010d340 0x7fdcc0113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55562/0 (socket says 172.21.15.90:55562) 2024-05-27T07:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.882+0000 7fdcbfe00640 1 -- 172.21.15.90:0/3895443935 learned_addr learned my addr 172.21.15.90:0/3895443935 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.883+0000 7fdcbf400640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc0102ab0 0x7fdcc01173a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.883+0000 7fdcbea00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcc01089e0 0x7fdcc01134d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.883+0000 7fdcbfe00640 1 -- 172.21.15.90:0/3895443935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcc01089e0 msgr2=0x7fdcc01134d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.883+0000 7fdcbfe00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcc01089e0 0x7fdcc01134d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.883+0000 7fdcbfe00640 1 -- 172.21.15.90:0/3895443935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc0102ab0 msgr2=0x7fdcc01173a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.883+0000 7fdcbfe00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc0102ab0 0x7fdcc01173a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.883+0000 7fdcbfe00640 1 -- 172.21.15.90:0/3895443935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc0114140 con 0x7fdcc010d340 2024-05-27T07:55:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.883+0000 7fdcbf400640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc0102ab0 0x7fdcc01173a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.884+0000 7fdcbfe00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc010d340 0x7fdcc0113a10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800cad0 tx=0x7fdcb800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.885+0000 7fdcb7e00640 1 -- 172.21.15.90:0/3895443935 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800ae20 con 0x7fdcc010d340 2024-05-27T07:55:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.885+0000 7fdcb7e00640 1 -- 172.21.15.90:0/3895443935 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcb8005020 con 0x7fdcc010d340 2024-05-27T07:55:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.885+0000 7fdcc4c00640 1 -- 172.21.15.90:0/3895443935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc01ad0f0 con 0x7fdcc010d340 2024-05-27T07:55:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.886+0000 7fdcb7e00640 1 -- 172.21.15.90:0/3895443935 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800f6b0 con 0x7fdcc010d340 2024-05-27T07:55:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.887+0000 7fdcc4c00640 1 -- 172.21.15.90:0/3895443935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01ad500 con 0x7fdcc010d340 2024-05-27T07:55:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.893+0000 7fdcb7e00640 1 -- 172.21.15.90:0/3895443935 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdcb80051d0 con 0x7fdcc010d340 2024-05-27T07:55:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.894+0000 7fdccd1c6740 1 Processor -- start 2024-05-27T07:55:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.895+0000 7fdccd1c6740 1 -- start start 2024-05-27T07:55:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.896+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c597ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.896+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c597cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.896+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b94d0 0x55e1c597d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.896+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e1c58edb90 con 0x55e1c55da1b0 2024-05-27T07:55:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.896+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e1c58dd730 con 0x55e1c55b12f0 2024-05-27T07:55:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.896+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e1c58e1770 con 0x55e1c55b94d0 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.897+0000 7fdcbea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b94d0 0x55e1c597d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.897+0000 7fdcbf400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c597cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.897+0000 7fdcbf400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c597cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56226/0 (socket says 172.21.15.90:56226) 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.897+0000 7fdcbf400640 1 -- 172.21.15.90:0/639858100 learned_addr learned my addr 172.21.15.90:0/639858100 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.897+0000 7fdcbea00640 1 --2- 172.21.15.90:0/639858100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c597ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.898+0000 7fdcbf400640 1 -- 172.21.15.90:0/639858100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b94d0 msgr2=0x55e1c597d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.898+0000 7fdcbf400640 1 --2- 172.21.15.90:0/639858100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b94d0 0x55e1c597d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.898+0000 7fdcbf400640 1 -- 172.21.15.90:0/639858100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 msgr2=0x55e1c597ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.898+0000 7fdcbf400640 1 --2- 172.21.15.90:0/639858100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c597ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.898+0000 7fdcbf400640 1 -- 172.21.15.90:0/639858100 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c58768b0 con 0x55e1c55b12f0 2024-05-27T07:55:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.899+0000 7fdcbf400640 1 --2- 172.21.15.90:0/639858100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c597cfd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000bd30 tx=0x7fdcb000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8c31bc27f9e3e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.899+0000 7fdcb7e00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdca40b2a50 0x7fdca40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.899+0000 7fdcb7e00640 1 -- 172.21.15.90:0/3895443935 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fdcb80e2e10 con 0x7fdcc010d340 2024-05-27T07:55:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.899+0000 7fdcabe00640 1 -- 172.21.15.90:0/639858100 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0015070 con 0x55e1c55b12f0 2024-05-27T07:55:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.899+0000 7fdcbea00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdca40b2a50 0x7fdca40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.899+0000 7fdcabe00640 1 -- 172.21.15.90:0/639858100 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcb000be20 con 0x55e1c55b12f0 2024-05-27T07:55:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.899+0000 7fdcabe00640 1 -- 172.21.15.90:0/639858100 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0011840 con 0x55e1c55b12f0 2024-05-27T07:55:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.901+0000 7fdccd1c6740 1 -- 172.21.15.90:0/639858100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 msgr2=0x55e1c597cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.901+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/639858100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c597cfd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000bd30 tx=0x7fdcb000f730 comp rx=0 tx=0).stop 2024-05-27T07:55:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.901+0000 7fdcbea00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdca40b2a50 0x7fdca40b4f60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdcac02a920 tx=0x7fdcac02a400 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.901+0000 7fdccd1c6740 1 -- 172.21.15.90:0/639858100 shutdown_connections 2024-05-27T07:55:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.901+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/639858100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c597cfd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.901+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/639858100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b94d0 0x55e1c597d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.901+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/639858100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c597ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.901+0000 7fdccd1c6740 1 -- 172.21.15.90:0/639858100 >> 172.21.15.90:0/639858100 conn(0x55e1c55ce300 msgr2=0x55e1c5979240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.901+0000 7fdccd1c6740 1 -- 172.21.15.90:0/639858100 shutdown_connections 2024-05-27T07:55:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.901+0000 7fdccd1c6740 1 -- 172.21.15.90:0/639858100 wait complete. 2024-05-27T07:55:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.902+0000 7fdccd1c6740 1 -- start start 2024-05-27T07:55:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.903+0000 7fdccd1c6740 1 Processor -- start 2024-05-27T07:55:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.905+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c5986c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.905+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b94d0 0x55e1c5987140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.905+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c5992ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.905+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e1c55c3790 con 0x55e1c55da1b0 2024-05-27T07:55:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.905+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e1c5885bc0 con 0x55e1c55b94d0 2024-05-27T07:55:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.905+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e1c5989bc0 con 0x55e1c55b12f0 2024-05-27T07:55:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.905+0000 7fdccd1c6740 10 client.0 _ll_register_callbacks cb 0x55e1c5863400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.905+0000 7fdcbf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c5986c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.905+0000 7fdcbf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c5986c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:40974/0 (socket says 172.21.15.90:40974) 2024-05-27T07:55:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.905+0000 7fdcbf400640 1 -- 172.21.15.90:0/496342067 learned_addr learned my addr 172.21.15.90:0/496342067 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.906+0000 7fdcbfe00640 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c5992ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.906+0000 7fdcbf400640 1 -- 172.21.15.90:0/496342067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b94d0 msgr2=0x55e1c5987140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.906+0000 7fdcbf400640 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b94d0 0x55e1c5987140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.906+0000 7fdcbf400640 1 -- 172.21.15.90:0/496342067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 msgr2=0x55e1c5992ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.906+0000 7fdcbf400640 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c5992ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.906+0000 7fdcbf400640 1 -- 172.21.15.90:0/496342067 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c55f2210 con 0x55e1c55b12f0 2024-05-27T07:55:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.906+0000 7fdcbfe00640 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c5992ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.907+0000 7fdcbf400640 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c5986c00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0019000 tx=0x7fdcb0005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78a9e8cdd3407990 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.907+0000 7fdca8c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:55:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.907+0000 7fdca8c00640 1 -- 172.21.15.90:0/496342067 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0028060 con 0x55e1c55b12f0 2024-05-27T07:55:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.907+0000 7fdca8c00640 1 -- 172.21.15.90:0/496342067 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fdcb00048c0 con 0x55e1c55b12f0 2024-05-27T07:55:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.907+0000 7fdccd1c6740 1 -- 172.21.15.90:0/496342067 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb000be20 con 0x55e1c55b12f0 2024-05-27T07:55:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.908+0000 7fdca8c00640 1 -- 172.21.15.90:0/496342067 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdcb002f050 con 0x55e1c55b12f0 2024-05-27T07:55:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.908+0000 7fdccd1c6740 10 client.26761 fetch_fsmap learned FSMap version 54 2024-05-27T07:55:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.908+0000 7fdccd1c6740 1 -- 172.21.15.90:0/496342067 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e1c56be620 con 0x55e1c55b12f0 2024-05-27T07:55:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.908+0000 7fdca8c00640 1 -- 172.21.15.90:0/496342067 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcb000be20 con 0x55e1c55b12f0 2024-05-27T07:55:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.908+0000 7fdca8c00640 1 -- 172.21.15.90:0/496342067 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 54) ==== 7757+0+0 (secure 0 0 0) 0x7fdcb0022030 con 0x55e1c55b12f0 2024-05-27T07:55:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.909+0000 7fdccd1c6740 10 client.26761 fetch_fsmap finished waiting for FSMap version 54 2024-05-27T07:55:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.909+0000 7fdccd1c6740 10 client.26761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.909+0000 7fdccd1c6740 20 client.26761 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.909+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e1c5598d80 0x55e1c5998d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.909+0000 7fdccd1c6740 4 client.26761 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.909+0000 7fdccd1c6740 1 -- 172.21.15.90:0/496342067 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1c58efae0 con 0x55e1c5598d80 2024-05-27T07:55:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.910+0000 7fdcbea00640 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e1c5598d80 0x55e1c5998d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.913+0000 7fdcbea00640 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e1c5598d80 0x55e1c5998d10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edccb4797d8a1dae server_cookie=6b50d5a4e2b2e705 in_seq=0 out_seq=0 2024-05-27T07:55:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.913+0000 7fdca8c00640 10 client.26761 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.916+0000 7fdca8c00640 1 -- 172.21.15.90:0/496342067 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e1c58efae0 con 0x55e1c5598d80 2024-05-27T07:55:15.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.916+0000 7fdca8c00640 10 client.26761 handle_command_reply: tid=0 2024-05-27T07:55:15.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.916+0000 7fdccd1c6740 1 client.26761 shutdown 2024-05-27T07:55:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.918+0000 7fdccd1c6740 1 -- 172.21.15.90:0/496342067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 msgr2=0x55e1c5986c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:15.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.918+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c5986c00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0019000 tx=0x7fdcb0005270 comp rx=0 tx=0).stop 2024-05-27T07:55:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.918+0000 7fdccd1c6740 1 -- 172.21.15.90:0/496342067 shutdown_connections 2024-05-27T07:55:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.918+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e1c5598d80 0x55e1c5998d10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.918+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b94d0 0x55e1c5987140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.918+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c5992ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.918+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/496342067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c5986c00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.918+0000 7fdccd1c6740 1 -- 172.21.15.90:0/496342067 >> 172.21.15.90:0/496342067 conn(0x55e1c55ce300 msgr2=0x55e1c597e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.918+0000 7fdca8c00640 0 client.26761 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.919+0000 7fdccd1c6740 1 -- 172.21.15.90:0/496342067 shutdown_connections 2024-05-27T07:55:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.919+0000 7fdccd1c6740 1 -- 172.21.15.90:0/496342067 wait complete. 2024-05-27T07:55:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.920+0000 7fdccd1c6740 20 client.26761 trim_cache size 0 max 16384 2024-05-27T07:55:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.951+0000 7fdccd1c6740 1 Processor -- start 2024-05-27T07:55:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.952+0000 7fdccd1c6740 1 -- start start 2024-05-27T07:55:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.952+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c597d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.952+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b94d0 0x55e1c5992c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.952+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c59931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.953+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01b91e0 con 0x55e1c55da1b0 2024-05-27T07:55:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.953+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01b6fd0 con 0x55e1c55b94d0 2024-05-27T07:55:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.953+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01b8b10 con 0x55e1c55b12f0 2024-05-27T07:55:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.953+0000 7fdcbf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c597d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.953+0000 7fdcbfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c59931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.953+0000 7fdcbf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c597d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:40988/0 (socket says 172.21.15.90:40988) 2024-05-27T07:55:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.953+0000 7fdcbf400640 1 -- 172.21.15.90:0/4028699211 learned_addr learned my addr 172.21.15.90:0/4028699211 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.953+0000 7fdcbea00640 1 --2- 172.21.15.90:0/4028699211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b94d0 0x55e1c5992c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.954+0000 7fdcbf400640 1 -- 172.21.15.90:0/4028699211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b94d0 msgr2=0x55e1c5992c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.954+0000 7fdcbf400640 1 --2- 172.21.15.90:0/4028699211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b94d0 0x55e1c5992c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.954+0000 7fdcbf400640 1 -- 172.21.15.90:0/4028699211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 msgr2=0x55e1c59931a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.954+0000 7fdcbf400640 1 --2- 172.21.15.90:0/4028699211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c59931a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.954+0000 7fdcbf400640 1 -- 172.21.15.90:0/4028699211 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c5985ca0 con 0x55e1c55b12f0 2024-05-27T07:55:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.954+0000 7fdcbfe00640 1 --2- 172.21.15.90:0/4028699211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c59931a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.954+0000 7fdcbf400640 1 --2- 172.21.15.90:0/4028699211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c597d830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b420 tx=0x7fdcb00059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d24e10ffbd475236 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.955+0000 7fdcab400640 1 -- 172.21.15.90:0/4028699211 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0028070 con 0x55e1c55b12f0 2024-05-27T07:55:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.956+0000 7fdcab400640 1 -- 172.21.15.90:0/4028699211 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcb001a460 con 0x55e1c55b12f0 2024-05-27T07:55:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.956+0000 7fdcab400640 1 -- 172.21.15.90:0/4028699211 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0015040 con 0x55e1c55b12f0 2024-05-27T07:55:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.957+0000 7fdccd1c6740 1 -- 172.21.15.90:0/4028699211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 msgr2=0x55e1c597d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.957+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/4028699211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c597d830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b420 tx=0x7fdcb00059b0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.957+0000 7fdccd1c6740 1 -- 172.21.15.90:0/4028699211 shutdown_connections 2024-05-27T07:55:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.957+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/4028699211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b94d0 0x55e1c5992c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.957+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/4028699211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55da1b0 0x55e1c59931a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.957+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/4028699211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55b12f0 0x55e1c597d830 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.957+0000 7fdccd1c6740 1 -- 172.21.15.90:0/4028699211 >> 172.21.15.90:0/4028699211 conn(0x55e1c55ce300 msgr2=0x55e1c5981e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.957+0000 7fdccd1c6740 1 -- 172.21.15.90:0/4028699211 shutdown_connections 2024-05-27T07:55:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.957+0000 7fdccd1c6740 1 -- 172.21.15.90:0/4028699211 wait complete. 2024-05-27T07:55:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.958+0000 7fdccd1c6740 1 -- start start 2024-05-27T07:55:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.958+0000 7fdccd1c6740 1 Processor -- start 2024-05-27T07:55:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.960+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c5989e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.960+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55b94d0 0x55e1c598a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.960+0000 7fdccd1c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55da1b0 0x55e1c598a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.960+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01b7d70 con 0x55e1c55b94d0 2024-05-27T07:55:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.960+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01b76a0 con 0x55e1c55b12f0 2024-05-27T07:55:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.960+0000 7fdccd1c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01b8440 con 0x55e1c55da1b0 2024-05-27T07:55:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.960+0000 7fdccd1c6740 10 client.0 _ll_register_callbacks cb 0x7fdcac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.960+0000 7fdcbf400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c5989e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.961+0000 7fdcbea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55b94d0 0x55e1c598a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.961+0000 7fdcbea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55b94d0 0x55e1c598a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55584/0 (socket says 172.21.15.90:55584) 2024-05-27T07:55:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.961+0000 7fdcbf400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c5989e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56242/0 (socket says 172.21.15.90:56242) 2024-05-27T07:55:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.961+0000 7fdcbf400640 1 -- 172.21.15.90:0/2287701234 learned_addr learned my addr 172.21.15.90:0/2287701234 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.961+0000 7fdcbfe00640 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55da1b0 0x55e1c598a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.961+0000 7fdcbea00640 1 -- 172.21.15.90:0/2287701234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55da1b0 msgr2=0x55e1c598a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.961+0000 7fdcbea00640 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55da1b0 0x55e1c598a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.961+0000 7fdcbea00640 1 -- 172.21.15.90:0/2287701234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 msgr2=0x55e1c5989e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.961+0000 7fdcbea00640 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c5989e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.961+0000 7fdcbea00640 1 -- 172.21.15.90:0/2287701234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c59ff360 con 0x55e1c55b94d0 2024-05-27T07:55:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.962+0000 7fdcbea00640 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55b94d0 0x55e1c598a350 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fdcac02a9f0 tx=0x7fdcac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cce1ea3cd710db server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.962+0000 7fdca8c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:55:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.962+0000 7fdca8c00640 1 -- 172.21.15.90:0/2287701234 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac035040 con 0x55e1c55b94d0 2024-05-27T07:55:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.962+0000 7fdca8c00640 1 -- 172.21.15.90:0/2287701234 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fdcac03d9e0 con 0x55e1c55b94d0 2024-05-27T07:55:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.962+0000 7fdccd1c6740 1 -- 172.21.15.90:0/2287701234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb001a460 con 0x55e1c55b94d0 2024-05-27T07:55:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.963+0000 7fdca8c00640 1 -- 172.21.15.90:0/2287701234 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdcac063050 con 0x55e1c55b94d0 2024-05-27T07:55:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.963+0000 7fdccd1c6740 10 client.17154 fetch_fsmap learned FSMap version 54 2024-05-27T07:55:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.963+0000 7fdccd1c6740 1 -- 172.21.15.90:0/2287701234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e1c5a005f0 con 0x55e1c55b94d0 2024-05-27T07:55:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.964+0000 7fdca8c00640 1 -- 172.21.15.90:0/2287701234 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdcb001a460 con 0x55e1c55b94d0 2024-05-27T07:55:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.964+0000 7fdca8c00640 1 -- 172.21.15.90:0/2287701234 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 54) ==== 7757+0+0 (secure 0 0 0) 0x7fdcac048040 con 0x55e1c55b94d0 2024-05-27T07:55:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.964+0000 7fdccd1c6740 10 client.17154 fetch_fsmap finished waiting for FSMap version 54 2024-05-27T07:55:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.964+0000 7fdccd1c6740 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.964+0000 7fdccd1c6740 20 client.17154 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.964+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e1c556e1c0 0x55e1c5a02e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.964+0000 7fdccd1c6740 4 client.17154 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:55:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.964+0000 7fdccd1c6740 1 -- 172.21.15.90:0/2287701234 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1c58e92c0 con 0x55e1c556e1c0 2024-05-27T07:55:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.965+0000 7fdcbf400640 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e1c556e1c0 0x55e1c5a02e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.966+0000 7fdcbf400640 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e1c556e1c0 0x55e1c5a02e30 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7a5e4bada024690 server_cookie=64035bd9a0fa8834 in_seq=0 out_seq=0 2024-05-27T07:55:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:15.966+0000 7fdca8c00640 10 client.17154 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.017+0000 7fdca8c00640 1 -- 172.21.15.90:0/2287701234 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e1c58e92c0 con 0x55e1c556e1c0 2024-05-27T07:55:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.017+0000 7fdca8c00640 10 client.17154 handle_command_reply: tid=0 2024-05-27T07:55:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.017+0000 7fdccd1c6740 1 client.17154 shutdown 2024-05-27T07:55:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.018+0000 7fdccd1c6740 1 -- 172.21.15.90:0/2287701234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55b94d0 msgr2=0x55e1c598a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.018+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55b94d0 0x55e1c598a350 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fdcac02a9f0 tx=0x7fdcac009260 comp rx=0 tx=0).stop 2024-05-27T07:55:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.018+0000 7fdccd1c6740 1 -- 172.21.15.90:0/2287701234 shutdown_connections 2024-05-27T07:55:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.018+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e1c556e1c0 0x55e1c5a02e30 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.018+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e1c55da1b0 0x55e1c598a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.018+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1c55b94d0 0x55e1c598a350 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.018+0000 7fdccd1c6740 1 --2- 172.21.15.90:0/2287701234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e1c55b12f0 0x55e1c5989e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.018+0000 7fdccd1c6740 1 -- 172.21.15.90:0/2287701234 >> 172.21.15.90:0/2287701234 conn(0x55e1c55ce300 msgr2=0x55e1c59882b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.018+0000 7fdca8c00640 0 client.17154 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.018+0000 7fdccd1c6740 1 -- 172.21.15.90:0/2287701234 shutdown_connections 2024-05-27T07:55:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.019+0000 7fdccd1c6740 1 -- 172.21.15.90:0/2287701234 wait complete. 2024-05-27T07:55:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.019+0000 7fdccd1c6740 20 client.17154 trim_cache size 0 max 16384 2024-05-27T07:55:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:55:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:55:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:55:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.020+0000 7fdcc4c00640 1 -- 172.21.15.90:0/3895443935 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdca40b2a50 msgr2=0x7fdca40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.020+0000 7fdcc4c00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdca40b2a50 0x7fdca40b4f60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdcac02a920 tx=0x7fdcac02a400 comp rx=0 tx=0).stop 2024-05-27T07:55:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.020+0000 7fdcc4c00640 1 -- 172.21.15.90:0/3895443935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc010d340 msgr2=0x7fdcc0113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.020+0000 7fdcc4c00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc010d340 0x7fdcc0113a10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800cad0 tx=0x7fdcb800cf90 comp rx=0 tx=0).stop 2024-05-27T07:55:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.021+0000 7fdcc4c00640 1 -- 172.21.15.90:0/3895443935 shutdown_connections 2024-05-27T07:55:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.021+0000 7fdcc4c00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdca40b2a50 0x7fdca40b4f60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.021+0000 7fdcc4c00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcc010d340 0x7fdcc0113a10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.021+0000 7fdcc4c00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcc01089e0 0x7fdcc01134d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.021+0000 7fdcc4c00640 1 --2- 172.21.15.90:0/3895443935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcc0102ab0 0x7fdcc01173a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.021+0000 7fdcc4c00640 1 -- 172.21.15.90:0/3895443935 >> 172.21.15.90:0/3895443935 conn(0x7fdcc00fe210 msgr2=0x7fdcc0104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.021+0000 7fdcc4c00640 1 -- 172.21.15.90:0/3895443935 shutdown_connections 2024-05-27T07:55:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:16.021+0000 7fdcc4c00640 1 -- 172.21.15.90:0/3895443935 wait complete. 2024-05-27T07:55:16.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:16 smithi129 ceph-mon[35254]: pgmap v281: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-05-27T07:55:16.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.662+0000 7f000b400700 1 -- 172.21.15.90:0/4094848878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000410b270 msgr2=0x7f000411d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.662+0000 7f000b400700 1 --2- 172.21.15.90:0/4094848878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000410b270 0x7f000411d3e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efff80098f0 tx=0x7efff8009c00 comp rx=0 tx=0).stop 2024-05-27T07:55:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.663+0000 7f000b400700 1 -- 172.21.15.90:0/4094848878 shutdown_connections 2024-05-27T07:55:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.663+0000 7f000b400700 1 --2- 172.21.15.90:0/4094848878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000411d920 0x7f000411fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.663+0000 7f000b400700 1 --2- 172.21.15.90:0/4094848878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f000410a920 0x7f000410ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.663+0000 7f000b400700 1 --2- 172.21.15.90:0/4094848878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000410b270 0x7f000411d3e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.663+0000 7f000b400700 1 -- 172.21.15.90:0/4094848878 >> 172.21.15.90:0/4094848878 conn(0x7f000406f4c0 msgr2=0x7f000406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:16.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:16 smithi090 ceph-mon[31429]: pgmap v281: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-05-27T07:55:16.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.663+0000 7f000b400700 1 -- 172.21.15.90:0/4094848878 shutdown_connections 2024-05-27T07:55:16.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.665+0000 7f000b400700 1 -- 172.21.15.90:0/4094848878 wait complete. 2024-05-27T07:55:16.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.667+0000 7f000b400700 1 Processor -- start 2024-05-27T07:55:16.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.667+0000 7f000b400700 1 -- start start 2024-05-27T07:55:16.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.668+0000 7f000b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000410a920 0x7f000411be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:16.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.668+0000 7f000b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000410b270 0x7f0004114fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:16.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.668+0000 7f000b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f000411d920 0x7f0004115660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:16.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.669+0000 7f000b400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0004123190 con 0x7f000411d920 2024-05-27T07:55:16.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.669+0000 7f0008a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000410a920 0x7f000411be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:16.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.669+0000 7f0009400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f000411d920 0x7f0004115660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:16.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.669+0000 7f0008a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000410a920 0x7f000411be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:41020/0 (socket says 172.21.15.90:41020) 2024-05-27T07:55:16.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.669+0000 7f0008a00700 1 -- 172.21.15.90:0/3294979388 learned_addr learned my addr 172.21.15.90:0/3294979388 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:16.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.669+0000 7f000b400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0004123010 con 0x7f000410b270 2024-05-27T07:55:16.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.669+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0004123310 con 0x7f000410a920 2024-05-27T07:55:16.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.670+0000 7f0003e00700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000410b270 0x7f0004114fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:16.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.671+0000 7f0003e00700 1 -- 172.21.15.90:0/3294979388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000410a920 msgr2=0x7f000411be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:16.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.671+0000 7f0003e00700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000410a920 0x7f000411be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.671+0000 7f0003e00700 1 -- 172.21.15.90:0/3294979388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f000411d920 msgr2=0x7f0004115660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:16.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.671+0000 7f0003e00700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f000411d920 0x7f0004115660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:16.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.671+0000 7f0003e00700 1 -- 172.21.15.90:0/3294979388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0004119980 con 0x7f000410b270 2024-05-27T07:55:16.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.671+0000 7f0008a00700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000410a920 0x7f000411be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:16.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.671+0000 7f0003e00700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000410b270 0x7f0004114fa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efff8009fd0 tx=0x7efff800f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:16.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.672+0000 7f0001600700 1 -- 172.21.15.90:0/3294979388 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff800b560 con 0x7f000410b270 2024-05-27T07:55:16.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.672+0000 7f0001600700 1 -- 172.21.15.90:0/3294979388 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efff800bb90 con 0x7f000410b270 2024-05-27T07:55:16.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.672+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0004119c10 con 0x7f000410b270 2024-05-27T07:55:16.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.672+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000411a140 con 0x7f000410b270 2024-05-27T07:55:16.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.672+0000 7f0001600700 1 -- 172.21.15.90:0/3294979388 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff80038e0 con 0x7f000410b270 2024-05-27T07:55:16.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.675+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe8005330 con 0x7f000410b270 2024-05-27T07:55:16.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.682+0000 7f0001600700 1 -- 172.21.15.90:0/3294979388 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efff800b700 con 0x7f000410b270 2024-05-27T07:55:16.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.684+0000 7f0001600700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7effec0b2af0 0x7effec0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:16.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.684+0000 7f0001600700 1 -- 172.21.15.90:0/3294979388 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7efff80e3e50 con 0x7f000410b270 2024-05-27T07:55:16.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.685+0000 7f0008a00700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7effec0b2af0 0x7effec0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:16.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.687+0000 7f0008a00700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7effec0b2af0 0x7effec0b5000 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f00041167d0 tx=0x7efff4009310 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:16.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:16.689+0000 7f0001600700 1 -- 172.21.15.90:0/3294979388 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efff8013030 con 0x7f000410b270 2024-05-27T07:55:16.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:16 smithi136 ceph-mon[35324]: pgmap v281: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-05-27T07:55:17.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.181+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effe8005620 con 0x7f000410b270 2024-05-27T07:55:17.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.184+0000 7f0001600700 1 -- 172.21.15.90:0/3294979388 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+10289 (secure 0 0 0) 0x7efff80ab490 con 0x7f000410b270 2024-05-27T07:55:17.186 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:55:17.186 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":55,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:55:17.105769+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26752":{"gid":26752,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6835/4022956786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":4022956786},{"type":"v1","addr":"172.21.15.90:6835","nonce":4022956786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34406,24478,34424,34472,34442]},"id":1}]} 2024-05-27T07:55:17.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.188+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7effec0b2af0 msgr2=0x7effec0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:17.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.188+0000 7f000b400700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7effec0b2af0 0x7effec0b5000 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f00041167d0 tx=0x7efff4009310 comp rx=0 tx=0).stop 2024-05-27T07:55:17.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.189+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000410b270 msgr2=0x7f0004114fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:17.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.189+0000 7f000b400700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000410b270 0x7f0004114fa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efff8009fd0 tx=0x7efff800f990 comp rx=0 tx=0).stop 2024-05-27T07:55:17.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.189+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 shutdown_connections 2024-05-27T07:55:17.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.189+0000 7f000b400700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f000411d920 0x7f0004115660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:17.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.189+0000 7f000b400700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000410a920 0x7f000411be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:17.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.189+0000 7f000b400700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7effec0b2af0 0x7effec0b5000 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:17.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.189+0000 7f000b400700 1 --2- 172.21.15.90:0/3294979388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f000410b270 0x7f0004114fa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:17.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.189+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 >> 172.21.15.90:0/3294979388 conn(0x7f000406f4c0 msgr2=0x7f0004072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:17.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.192+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 shutdown_connections 2024-05-27T07:55:17.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:17.192+0000 7f000b400700 1 -- 172.21.15.90:0/3294979388 wait complete. 2024-05-27T07:55:17.199 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 55 2024-05-27T07:55:17.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:17 smithi090 ceph-mon[31429]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:17.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:17 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:17.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:17 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3294979388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:55:17.464 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:17 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 3. 2024-05-27T07:55:17.464 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:17 smithi090 systemd[1]: Stopped Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:55:17.464 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:17 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.528s CPU time. 2024-05-27T07:55:17.465 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:17 smithi090 systemd[1]: Starting Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:55:17.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:17 smithi129 ceph-mon[35254]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:17.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:17 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:17.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:17 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3294979388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:55:17.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:17 smithi136 ceph-mon[35324]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:17.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:17 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:17.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:17 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3294979388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:55:18.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:55:18.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:18 smithi090 ceph-mon[31429]: pgmap v282: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-05-27T07:55:18.373 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f2574e00700 time 2024-05-27T07:55:18.174378+0000 2024-05-27T07:55:18.373 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:18.373 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:18.373 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f258a90aba6] 2024-05-27T07:55:18.373 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f258a90add1] 2024-05-27T07:55:18.373 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556ffe9fc23d] 2024-05-27T07:55:18.374 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556ffe9e7971] 2024-05-27T07:55:18.374 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 5: (SnapRealm::check_cache() const+0x13e) [0x556ffe9e5fa0] 2024-05-27T07:55:18.374 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556ffe9e6651] 2024-05-27T07:55:18.374 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556ffe97b347] 2024-05-27T07:55:18.374 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556ffe67787e] 2024-05-27T07:55:18.374 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x556ffe67cf2f] 2024-05-27T07:55:18.374 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 10: (EOpen::replay(MDSRank*)+0x57) [0x556ffe6833e9] 2024-05-27T07:55:18.375 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 11: (MDLog::_replay_thread()+0x1f38) [0x556ffea2048c] 2024-05-27T07:55:18.375 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556ffea299cb] 2024-05-27T07:55:18.375 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 13: (Thread::entry_wrapper()+0x43) [0x7f258a8d972d] 2024-05-27T07:55:18.375 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 14: (Thread::_entry_func(void*)+0xd) [0x7f258a8d9749] 2024-05-27T07:55:18.375 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f258947d1ca] 2024-05-27T07:55:18.375 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 16: clone() 2024-05-27T07:55:18.375 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: *** Caught signal (Aborted) ** 2024-05-27T07:55:18.375 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: in thread 7f2574e00700 thread_name:md_log_replay 2024-05-27T07:55:18.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.176+0000 7f2574e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f2574e00700 time 2024-05-27T07:55:18.174378+0000 2024-05-27T07:55:18.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:18.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:18.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f258a90aba6] 2024-05-27T07:55:18.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f258a90add1] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556ffe9fc23d] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556ffe9e7971] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 5: (SnapRealm::check_cache() const+0x13e) [0x556ffe9e5fa0] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556ffe9e6651] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556ffe97b347] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556ffe67787e] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x556ffe67cf2f] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 10: (EOpen::replay(MDSRank*)+0x57) [0x556ffe6833e9] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 11: (MDLog::_replay_thread()+0x1f38) [0x556ffea2048c] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556ffea299cb] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 13: (Thread::entry_wrapper()+0x43) [0x7f258a8d972d] 2024-05-27T07:55:18.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 14: (Thread::_entry_func(void*)+0xd) [0x7f258a8d9749] 2024-05-27T07:55:18.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f258947d1ca] 2024-05-27T07:55:18.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 16: clone() 2024-05-27T07:55:18.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:18.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2589487d20] 2024-05-27T07:55:18.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2: gsignal() 2024-05-27T07:55:18.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 3: abort() 2024-05-27T07:55:18.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f258a90aced] 2024-05-27T07:55:18.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f258a90add1] 2024-05-27T07:55:18.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556ffe9fc23d] 2024-05-27T07:55:18.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556ffe9e7971] 2024-05-27T07:55:18.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 8: (SnapRealm::check_cache() const+0x13e) [0x556ffe9e5fa0] 2024-05-27T07:55:18.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556ffe9e6651] 2024-05-27T07:55:18.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556ffe97b347] 2024-05-27T07:55:18.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556ffe67787e] 2024-05-27T07:55:18.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x556ffe67cf2f] 2024-05-27T07:55:18.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 13: (EOpen::replay(MDSRank*)+0x57) [0x556ffe6833e9] 2024-05-27T07:55:18.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 14: (MDLog::_replay_thread()+0x1f38) [0x556ffea2048c] 2024-05-27T07:55:18.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556ffea299cb] 2024-05-27T07:55:18.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 16: (Thread::entry_wrapper()+0x43) [0x7f258a8d972d] 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 17: (Thread::_entry_func(void*)+0xd) [0x7f258a8d9749] 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f258947d1ca] 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 19: clone() 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.179+0000 7f2574e00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: in thread 7f2574e00700 thread_name:md_log_replay 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2589487d20] 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2: gsignal() 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 3: abort() 2024-05-27T07:55:18.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f258a90aced] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f258a90add1] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556ffe9fc23d] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556ffe9e7971] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 8: (SnapRealm::check_cache() const+0x13e) [0x556ffe9e5fa0] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556ffe9e6651] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556ffe97b347] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556ffe67787e] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x556ffe67cf2f] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 13: (EOpen::replay(MDSRank*)+0x57) [0x556ffe6833e9] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 14: (MDLog::_replay_thread()+0x1f38) [0x556ffea2048c] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556ffea299cb] 2024-05-27T07:55:18.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 16: (Thread::entry_wrapper()+0x43) [0x7f258a8d972d] 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 17: (Thread::_entry_func(void*)+0xd) [0x7f258a8d9749] 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f258947d1ca] 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 19: clone() 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: -1> 2024-05-27T07:55:18.176+0000 7f2574e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f2574e00700 time 2024-05-27T07:55:18.174378+0000 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f258a90aba6] 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f258a90add1] 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556ffe9fc23d] 2024-05-27T07:55:18.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556ffe9e7971] 2024-05-27T07:55:18.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 5: (SnapRealm::check_cache() const+0x13e) [0x556ffe9e5fa0] 2024-05-27T07:55:18.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556ffe9e6651] 2024-05-27T07:55:18.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556ffe97b347] 2024-05-27T07:55:18.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556ffe67787e] 2024-05-27T07:55:18.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x556ffe67cf2f] 2024-05-27T07:55:18.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 10: (EOpen::replay(MDSRank*)+0x57) [0x556ffe6833e9] 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 11: (MDLog::_replay_thread()+0x1f38) [0x556ffea2048c] 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556ffea299cb] 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 13: (Thread::entry_wrapper()+0x43) [0x7f258a8d972d] 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 14: (Thread::_entry_func(void*)+0xd) [0x7f258a8d9749] 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f258947d1ca] 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 16: clone() 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 0> 2024-05-27T07:55:18.179+0000 7f2574e00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: in thread 7f2574e00700 thread_name:md_log_replay 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2589487d20] 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2: gsignal() 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 3: abort() 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f258a90aced] 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f258a90add1] 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556ffe9fc23d] 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556ffe9e7971] 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 8: (SnapRealm::check_cache() const+0x13e) [0x556ffe9e5fa0] 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556ffe9e6651] 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556ffe97b347] 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556ffe67787e] 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x556ffe67cf2f] 2024-05-27T07:55:18.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 13: (EOpen::replay(MDSRank*)+0x57) [0x556ffe6833e9] 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 14: (MDLog::_replay_thread()+0x1f38) [0x556ffea2048c] 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556ffea299cb] 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 16: (Thread::entry_wrapper()+0x43) [0x7f258a8d972d] 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 17: (Thread::_entry_func(void*)+0xd) [0x7f258a8d9749] 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f258947d1ca] 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 19: clone() 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: -1> 2024-05-27T07:55:18.176+0000 7f2574e00700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f2574e00700 time 2024-05-27T07:55:18.174378+0000 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:18.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f258a90aba6] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f258a90add1] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556ffe9fc23d] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 4: (SnapRealm::build_snap_set() const+0x323) [0x556ffe9e7971] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 5: (SnapRealm::check_cache() const+0x13e) [0x556ffe9e5fa0] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556ffe9e6651] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556ffe97b347] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556ffe67787e] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x556ffe67cf2f] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 10: (EOpen::replay(MDSRank*)+0x57) [0x556ffe6833e9] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 11: (MDLog::_replay_thread()+0x1f38) [0x556ffea2048c] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 12: (MDLog::ReplayThread::entry()+0x11) [0x556ffea299cb] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 13: (Thread::entry_wrapper()+0x43) [0x7f258a8d972d] 2024-05-27T07:55:18.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 14: (Thread::_entry_func(void*)+0xd) [0x7f258a8d9749] 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f258947d1ca] 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 16: clone() 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 0> 2024-05-27T07:55:18.179+0000 7f2574e00700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: in thread 7f2574e00700 thread_name:md_log_replay 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2589487d20] 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2: gsignal() 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 3: abort() 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f258a90aced] 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f258a90add1] 2024-05-27T07:55:18.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x556ffe9fc23d] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 7: (SnapRealm::build_snap_set() const+0x323) [0x556ffe9e7971] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 8: (SnapRealm::check_cache() const+0x13e) [0x556ffe9e5fa0] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x556ffe9e6651] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x556ffe97b347] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x556ffe67787e] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x556ffe67cf2f] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 13: (EOpen::replay(MDSRank*)+0x57) [0x556ffe6833e9] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 14: (MDLog::_replay_thread()+0x1f38) [0x556ffea2048c] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 15: (MDLog::ReplayThread::entry()+0x11) [0x556ffea299cb] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 16: (Thread::entry_wrapper()+0x43) [0x7f258a8d972d] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 17: (Thread::_entry_func(void*)+0xd) [0x7f258a8d9749] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f258947d1ca] 2024-05-27T07:55:18.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 19: clone() 2024-05-27T07:55:18.392 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:18.392 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d[112986]: 2024-05-27T07:55:18.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:18 smithi129 ceph-mon[35254]: pgmap v282: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-05-27T07:55:18.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:18 smithi090 podman[113411]: 2024-05-27 07:55:18.281852168 +0000 UTC m=+0.089594814 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:55:18.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:18 smithi090 podman[113411]: 2024-05-27 07:55:18.437811308 +0000 UTC m=+0.245553944 container create e6e05b5296f9da10e0512003a1d00d116009eef42691772765799a4251b6f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-27T07:55:18.676 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:18 smithi090 podman[113434]: 2024-05-27 07:55:18.673491392 +0000 UTC m=+0.415832403 container died 6a3722d62ee5c36ba1b9824740938cf51259c5f1d43d5546a0b09919c1c2bdd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5) 2024-05-27T07:55:18.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:18 smithi136 ceph-mon[35324]: pgmap v282: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.5 MiB/s wr, 536 op/s 2024-05-27T07:55:18.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 300.0 s, 611.6 tps, lat 168.747 ms stddev 57.737 2024-05-27T07:55:19.845 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:19 smithi090 podman[113434]: 2024-05-27 07:55:19.48036733 +0000 UTC m=+1.222708333 container remove 6a3722d62ee5c36ba1b9824740938cf51259c5f1d43d5546a0b09919c1c2bdd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T07:55:19.846 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:19 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:55:19.846 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:19 smithi090 podman[113411]: 2024-05-27 07:55:19.599154195 +0000 UTC m=+1.406896851 container init e6e05b5296f9da10e0512003a1d00d116009eef42691772765799a4251b6f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-27T07:55:19.846 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:19 smithi090 podman[113411]: 2024-05-27 07:55:19.610660193 +0000 UTC m=+1.418402832 container start e6e05b5296f9da10e0512003a1d00d116009eef42691772765799a4251b6f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:55:19.847 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:19 smithi090 bash[113411]: e6e05b5296f9da10e0512003a1d00d116009eef42691772765799a4251b6f09b 2024-05-27T07:55:19.847 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:19 smithi090 systemd[1]: Started Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:55:19.847 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: starting mds.j at 2024-05-27T07:55:19.849 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:20.522 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:20 smithi090 ceph-mon[31429]: pgmap v283: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 569 op/s 2024-05-27T07:55:20.551 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:20 smithi136 ceph-mon[35324]: pgmap v283: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 569 op/s 2024-05-27T07:55:20.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:20 smithi129 ceph-mon[35254]: pgmap v283: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 569 op/s 2024-05-27T07:55:21.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:21 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6834/1395232286,v1:172.21.15.90:6835/1395232286] up:boot 2024-05-27T07:55:21.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:21 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:55:21.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:21 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:55:21.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:21 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6834/1395232286,v1:172.21.15.90:6835/1395232286] up:boot 2024-05-27T07:55:21.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:21 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:55:21.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:21 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:55:21.640 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:21 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-05-27T07:55:21.640 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:21 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.535s CPU time. 2024-05-27T07:55:21.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:21 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6834/1395232286,v1:172.21.15.90:6835/1395232286] up:boot 2024-05-27T07:55:21.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:21 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:55:21.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:21 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:55:22.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:22 smithi090 ceph-mon[31429]: pgmap v284: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 580 op/s 2024-05-27T07:55:22.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:22 smithi136 ceph-mon[35324]: pgmap v284: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 580 op/s 2024-05-27T07:55:22.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:22 smithi129 ceph-mon[35254]: pgmap v284: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 580 op/s 2024-05-27T07:55:22.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.923+0000 7f8b61000700 1 -- 172.21.15.90:0/1077029551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b5c109840 msgr2=0x7f8b5c1111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:22.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.923+0000 7f8b61000700 1 --2- 172.21.15.90:0/1077029551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b5c109840 0x7f8b5c1111a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009fc0 tx=0x7f8b5400b330 comp rx=0 tx=0).stop 2024-05-27T07:55:22.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.923+0000 7f8b61000700 1 -- 172.21.15.90:0/1077029551 shutdown_connections 2024-05-27T07:55:22.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.923+0000 7f8b61000700 1 --2- 172.21.15.90:0/1077029551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b5c109840 0x7f8b5c1111a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.923+0000 7f8b61000700 1 --2- 172.21.15.90:0/1077029551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b5c1045c0 0x7f8b5c1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.923+0000 7f8b61000700 1 --2- 172.21.15.90:0/1077029551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b5c108ef0 0x7f8b5c109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.923+0000 7f8b61000700 1 -- 172.21.15.90:0/1077029551 >> 172.21.15.90:0/1077029551 conn(0x7f8b5c0ffe70 msgr2=0x7f8b5c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:22.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.924+0000 7f8b61000700 1 -- 172.21.15.90:0/1077029551 shutdown_connections 2024-05-27T07:55:22.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.924+0000 7f8b61000700 1 -- 172.21.15.90:0/1077029551 wait complete. 2024-05-27T07:55:22.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.925+0000 7f8b61000700 1 Processor -- start 2024-05-27T07:55:22.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.926+0000 7f8b61000700 1 -- start start 2024-05-27T07:55:22.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.926+0000 7f8b61000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b5c1045c0 0x7f8b5c1a1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.926+0000 7f8b61000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b5c108ef0 0x7f8b5c1a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.926+0000 7f8b61000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b5c109840 0x7f8b5c1a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.926+0000 7f8b61000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c11a990 con 0x7f8b5c109840 2024-05-27T07:55:22.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.926+0000 7f8b61000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c11a810 con 0x7f8b5c1045c0 2024-05-27T07:55:22.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.926+0000 7f8b61000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c11ab10 con 0x7f8b5c108ef0 2024-05-27T07:55:22.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.927+0000 7f8b59e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b5c1045c0 0x7f8b5c1a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.927+0000 7f8b59400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b5c108ef0 0x7f8b5c1a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.927+0000 7f8b59e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b5c1045c0 0x7f8b5c1a1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50946/0 (socket says 172.21.15.90:50946) 2024-05-27T07:55:22.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.927+0000 7f8b59e00700 1 -- 172.21.15.90:0/3831896988 learned_addr learned my addr 172.21.15.90:0/3831896988 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:22.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.927+0000 7f8b59400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b5c108ef0 0x7f8b5c1a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50096/0 (socket says 172.21.15.90:50096) 2024-05-27T07:55:22.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.927+0000 7f8b5a800700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b5c109840 0x7f8b5c1a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.928+0000 7f8b59e00700 1 -- 172.21.15.90:0/3831896988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b5c108ef0 msgr2=0x7f8b5c1a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:22.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.928+0000 7f8b59e00700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b5c108ef0 0x7f8b5c1a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.928+0000 7f8b59e00700 1 -- 172.21.15.90:0/3831896988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b5c109840 msgr2=0x7f8b5c1a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:22.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.928+0000 7f8b59e00700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b5c109840 0x7f8b5c1a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.928+0000 7f8b59e00700 1 -- 172.21.15.90:0/3831896988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b5c1a6220 con 0x7f8b5c1045c0 2024-05-27T07:55:22.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.928+0000 7f8b59e00700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b5c1045c0 0x7f8b5c1a1350 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00e9c0 tx=0x7f8b4c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:22.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.929+0000 7f8b52a00700 1 -- 172.21.15.90:0/3831896988 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c004a30 con 0x7f8b5c1045c0 2024-05-27T07:55:22.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.929+0000 7f8b52a00700 1 -- 172.21.15.90:0/3831896988 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b4c00cc50 con 0x7f8b5c1045c0 2024-05-27T07:55:22.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.929+0000 7f8b52a00700 1 -- 172.21.15.90:0/3831896988 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c005420 con 0x7f8b5c1045c0 2024-05-27T07:55:22.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.930+0000 7f8b61000700 1 -- 172.21.15.90:0/3831896988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b5c1a6510 con 0x7f8b5c1045c0 2024-05-27T07:55:22.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.930+0000 7f8b61000700 1 -- 172.21.15.90:0/3831896988 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5c1ade30 con 0x7f8b5c1045c0 2024-05-27T07:55:22.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.935+0000 7f8b52a00700 1 -- 172.21.15.90:0/3831896988 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8b4c0055c0 con 0x7f8b5c1045c0 2024-05-27T07:55:22.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.938+0000 7f8b52a00700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8b400b2ab0 0x7f8b400b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.938+0000 7f8b52a00700 1 -- 172.21.15.90:0/3831896988 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8b4c0e2a50 con 0x7f8b5c1045c0 2024-05-27T07:55:22.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.939+0000 7f8b59400700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8b400b2ab0 0x7f8b400b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.939+0000 7f8b6fc37b80 1 Processor -- start 2024-05-27T07:55:22.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.940+0000 7f8b59400700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8b400b2ab0 0x7f8b400b4fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009680 tx=0x7f8b48009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:22.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.940+0000 7f8b6fc37b80 1 -- start start 2024-05-27T07:55:22.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.941+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 0x556c2883a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.941+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c28834870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.941+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c288351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.941+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556c28762e40 con 0x556c286b5e40 2024-05-27T07:55:22.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.941+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556c287529e0 con 0x556c2883ad30 2024-05-27T07:55:22.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.941+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556c28756a20 con 0x556c28834db0 2024-05-27T07:55:22.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.942+0000 7f8b59e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 0x556c2883a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.942+0000 7f8b59400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c28834870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.942+0000 7f8b5a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c288351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.942+0000 7f8b59400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c28834870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50948/0 (socket says 172.21.15.90:50948) 2024-05-27T07:55:22.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.942+0000 7f8b59400700 1 -- 172.21.15.90:0/294955614 learned_addr learned my addr 172.21.15.90:0/294955614 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:22.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.942+0000 7f8b5a800700 1 --2- 172.21.15.90:0/294955614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c288351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50112/0 (socket says 172.21.15.90:50112) 2024-05-27T07:55:22.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.943+0000 7f8b59400700 1 -- 172.21.15.90:0/294955614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 msgr2=0x556c288351c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:22.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.943+0000 7f8b59400700 1 --2- 172.21.15.90:0/294955614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c288351c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.943+0000 7f8b59400700 1 -- 172.21.15.90:0/294955614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 msgr2=0x556c2883a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:22.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.943+0000 7f8b59400700 1 --2- 172.21.15.90:0/294955614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 0x556c2883a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.943+0000 7f8b59400700 1 -- 172.21.15.90:0/294955614 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2871b420 con 0x556c2883ad30 2024-05-27T07:55:22.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.943+0000 7f8b59400700 1 --2- 172.21.15.90:0/294955614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c28834870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8b48011d00 tx=0x7f8b48014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32a1bed8e76aecf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:22.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.944+0000 7f8b47400700 1 -- 172.21.15.90:0/294955614 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48011d30 con 0x556c2883ad30 2024-05-27T07:55:22.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.944+0000 7f8b47400700 1 -- 172.21.15.90:0/294955614 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b48004020 con 0x556c2883ad30 2024-05-27T07:55:22.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.944+0000 7f8b47400700 1 -- 172.21.15.90:0/294955614 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48003b60 con 0x556c2883ad30 2024-05-27T07:55:22.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.946+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/294955614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 msgr2=0x556c28834870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:22.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.946+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/294955614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c28834870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8b48011d00 tx=0x7f8b48014000 comp rx=0 tx=0).stop 2024-05-27T07:55:22.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.946+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/294955614 shutdown_connections 2024-05-27T07:55:22.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.946+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/294955614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c288351c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.946+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/294955614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 0x556c2883a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.946+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/294955614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c28834870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.946+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/294955614 >> 172.21.15.90:0/294955614 conn(0x556c28715fc0 msgr2=0x556c2882f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:22.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.946+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/294955614 shutdown_connections 2024-05-27T07:55:22.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.947+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/294955614 wait complete. 2024-05-27T07:55:22.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.947+0000 7f8b6fc37b80 1 -- start start 2024-05-27T07:55:22.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.948+0000 7f8b6fc37b80 1 Processor -- start 2024-05-27T07:55:22.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 0x556c2883e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c2883e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c2883ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556c28658fa0 con 0x556c286b5e40 2024-05-27T07:55:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556c286f3760 con 0x556c2883ad30 2024-05-27T07:55:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556c28841800 con 0x556c28834db0 2024-05-27T07:55:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b6fc37b80 10 client.0 _ll_register_callbacks cb 0x556c286fcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b59e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 0x556c2883e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b5a800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c2883ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b59e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 0x556c2883e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32868/0 (socket says 172.21.15.90:32868) 2024-05-27T07:55:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b5a800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c2883ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50952/0 (socket says 172.21.15.90:50952) 2024-05-27T07:55:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.949+0000 7f8b59e00700 1 -- 172.21.15.90:0/2515195863 learned_addr learned my addr 172.21.15.90:0/2515195863 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:22.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.950+0000 7f8b59400700 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c2883e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.950+0000 7f8b59e00700 1 -- 172.21.15.90:0/2515195863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 msgr2=0x556c2883e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:22.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.950+0000 7f8b59e00700 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c2883e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.950+0000 7f8b59e00700 1 -- 172.21.15.90:0/2515195863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 msgr2=0x556c2883ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:22.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.950+0000 7f8b59e00700 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c2883ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.950+0000 7f8b59e00700 1 -- 172.21.15.90:0/2515195863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c286a28d0 con 0x556c286b5e40 2024-05-27T07:55:22.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.950+0000 7f8b59400700 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c2883e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.950+0000 7f8b59e00700 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 0x556c2883e480 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c005950 tx=0x7f8b4c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce9753d727b0da5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.951+0000 7f8b3be00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:55:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.951+0000 7f8b3be00700 1 -- 172.21.15.90:0/2515195863 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0b2100 con 0x556c286b5e40 2024-05-27T07:55:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.951+0000 7f8b3be00700 1 -- 172.21.15.90:0/2515195863 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8b4c0a5f50 con 0x556c286b5e40 2024-05-27T07:55:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.951+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2515195863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b48004020 con 0x556c286b5e40 2024-05-27T07:55:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.952+0000 7f8b3be00700 1 -- 172.21.15.90:0/2515195863 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f8b4c0a9470 con 0x556c286b5e40 2024-05-27T07:55:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.952+0000 7f8b6fc37b80 10 client.17178 fetch_fsmap learned FSMap version 56 2024-05-27T07:55:22.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.952+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2515195863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x556c28723060 con 0x556c286b5e40 2024-05-27T07:55:22.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.953+0000 7f8b3be00700 1 -- 172.21.15.90:0/2515195863 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b48004020 con 0x556c286b5e40 2024-05-27T07:55:22.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.953+0000 7f8b3be00700 1 -- 172.21.15.90:0/2515195863 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 56) ==== 7757+0+0 (secure 0 0 0) 0x7f8b4c0b2330 con 0x556c286b5e40 2024-05-27T07:55:22.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.953+0000 7f8b6fc37b80 10 client.17178 fetch_fsmap finished waiting for FSMap version 56 2024-05-27T07:55:22.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.953+0000 7f8b6fc37b80 10 client.17178 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:55:22.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.953+0000 7f8b6fc37b80 20 client.17178 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:22.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.953+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c2884e620 0x556c28850a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.953+0000 7f8b6fc37b80 4 client.17178 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:22.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.953+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2515195863 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c28764d90 con 0x556c2884e620 2024-05-27T07:55:22.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.956+0000 7f8b59400700 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c2884e620 0x556c28850a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.957+0000 7f8b59400700 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c2884e620 0x556c28850a40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b6d358a6ff80961 server_cookie=b015d152906e7b28 in_seq=0 out_seq=0 2024-05-27T07:55:22.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.957+0000 7f8b3be00700 10 client.17178 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b3be00700 1 -- 172.21.15.90:0/2515195863 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556c28764d90 con 0x556c2884e620 2024-05-27T07:55:22.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b3be00700 10 client.17178 handle_command_reply: tid=0 2024-05-27T07:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b6fc37b80 1 client.17178 shutdown 2024-05-27T07:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2515195863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 msgr2=0x556c2883e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 0x556c2883e480 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c005950 tx=0x7f8b4c004ed0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2515195863 shutdown_connections 2024-05-27T07:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c2884e620 0x556c28850a40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c2883ad30 0x556c2883ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c286b5e40 0x556c2883e480 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/2515195863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c2883e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.959+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2515195863 >> 172.21.15.90:0/2515195863 conn(0x556c28715fc0 msgr2=0x556c288351c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:22.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.960+0000 7f8b3be00700 0 client.17178 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:55:22.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.960+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2515195863 shutdown_connections 2024-05-27T07:55:22.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.960+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2515195863 wait complete. 2024-05-27T07:55:22.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.961+0000 7f8b6fc37b80 20 client.17178 trim_cache size 0 max 16384 2024-05-27T07:55:22.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.991+0000 7f8b6fc37b80 1 Processor -- start 2024-05-27T07:55:22.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.992+0000 7f8b6fc37b80 1 -- start start 2024-05-27T07:55:22.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.992+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c28834870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.992+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c2883d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.992+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 0x556c2884a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.992+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1b8790 con 0x556c2883ad30 2024-05-27T07:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.992+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1b6580 con 0x556c286b5e40 2024-05-27T07:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.992+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1b80c0 con 0x556c28834db0 2024-05-27T07:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.993+0000 7f8b59e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c28834870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.993+0000 7f8b59e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c28834870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50956/0 (socket says 172.21.15.90:50956) 2024-05-27T07:55:22.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.993+0000 7f8b59e00700 1 -- 172.21.15.90:0/2014263173 learned_addr learned my addr 172.21.15.90:0/2014263173 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:22.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.993+0000 7f8b5a800700 1 --2- 172.21.15.90:0/2014263173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 0x556c2884a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.994+0000 7f8b59400700 1 --2- 172.21.15.90:0/2014263173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c2883d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:22.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.994+0000 7f8b59e00700 1 -- 172.21.15.90:0/2014263173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 msgr2=0x556c2883d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:22.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.994+0000 7f8b59e00700 1 --2- 172.21.15.90:0/2014263173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c2883d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.995+0000 7f8b59e00700 1 -- 172.21.15.90:0/2014263173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 msgr2=0x556c2884a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.995+0000 7f8b59e00700 1 --2- 172.21.15.90:0/2014263173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 0x556c2884a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.995+0000 7f8b59e00700 1 -- 172.21.15.90:0/2014263173 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c28838f90 con 0x556c286b5e40 2024-05-27T07:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.995+0000 7f8b5a800700 1 --2- 172.21.15.90:0/2014263173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 0x556c2884a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.996+0000 7f8b59e00700 1 --2- 172.21.15.90:0/2014263173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c28834870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0058b0 tx=0x7f8b4c0a5820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43874b2cc094cc3a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:22.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.997+0000 7f8b46a00700 1 -- 172.21.15.90:0/2014263173 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c088080 con 0x556c286b5e40 2024-05-27T07:55:22.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.997+0000 7f8b46a00700 1 -- 172.21.15.90:0/2014263173 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b4c0c4040 con 0x556c286b5e40 2024-05-27T07:55:22.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.997+0000 7f8b46a00700 1 -- 172.21.15.90:0/2014263173 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0b27c0 con 0x556c286b5e40 2024-05-27T07:55:23.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.999+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2014263173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 msgr2=0x556c28834870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:23.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.999+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/2014263173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c28834870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0058b0 tx=0x7f8b4c0a5820 comp rx=0 tx=0).stop 2024-05-27T07:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.999+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2014263173 shutdown_connections 2024-05-27T07:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.999+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/2014263173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 0x556c2884a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.999+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/2014263173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c28834870 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.999+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/2014263173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c2883d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.999+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2014263173 >> 172.21.15.90:0/2014263173 conn(0x556c28715fc0 msgr2=0x556c28838880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:22.999+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2014263173 shutdown_connections 2024-05-27T07:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.000+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/2014263173 wait complete. 2024-05-27T07:55:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.001+0000 7f8b6fc37b80 1 -- start start 2024-05-27T07:55:23.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.001+0000 7f8b6fc37b80 1 Processor -- start 2024-05-27T07:55:23.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.002+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c288472d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:23.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.002+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c28855c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:23.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.002+0000 7f8b6fc37b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 0x556c288562b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:23.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.002+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1b7320 con 0x556c2883ad30 2024-05-27T07:55:23.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.002+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1b6c50 con 0x556c286b5e40 2024-05-27T07:55:23.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.002+0000 7f8b6fc37b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5c1b79f0 con 0x556c28834db0 2024-05-27T07:55:23.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.002+0000 7f8b6fc37b80 10 client.0 _ll_register_callbacks cb 0x556c286fcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.003+0000 7f8b59e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c288472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.003+0000 7f8b59e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c288472d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50960/0 (socket says 172.21.15.90:50960) 2024-05-27T07:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.003+0000 7f8b59e00700 1 -- 172.21.15.90:0/519219556 learned_addr learned my addr 172.21.15.90:0/519219556 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.003+0000 7f8b59400700 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c28855c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.004+0000 7f8b5a800700 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 0x556c288562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.004+0000 7f8b59e00700 1 -- 172.21.15.90:0/519219556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 msgr2=0x556c28855c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:23.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.004+0000 7f8b59e00700 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c28855c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.004+0000 7f8b59e00700 1 -- 172.21.15.90:0/519219556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 msgr2=0x556c288562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:23.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.004+0000 7f8b59e00700 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 0x556c288562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.004+0000 7f8b59e00700 1 -- 172.21.15.90:0/519219556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c28856a30 con 0x556c286b5e40 2024-05-27T07:55:23.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.005+0000 7f8b5a800700 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 0x556c288562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:23.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.005+0000 7f8b59400700 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c28855c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:23.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.006+0000 7f8b59e00700 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c288472d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c005950 tx=0x7f8b4c0a9f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f988b84b17fd2b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:23.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.006+0000 7f8b3be00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:55:23.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.006+0000 7f8b3be00700 1 -- 172.21.15.90:0/519219556 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c0aa060 con 0x556c286b5e40 2024-05-27T07:55:23.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.007+0000 7f8b3be00700 1 -- 172.21.15.90:0/519219556 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8b4c0a8bc0 con 0x556c286b5e40 2024-05-27T07:55:23.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.007+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/519219556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b4c0c4040 con 0x556c286b5e40 2024-05-27T07:55:23.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.008+0000 7f8b3be00700 1 -- 172.21.15.90:0/519219556 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f8b4c0a9960 con 0x556c286b5e40 2024-05-27T07:55:23.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.008+0000 7f8b6fc37b80 10 client.36752 fetch_fsmap learned FSMap version 56 2024-05-27T07:55:23.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.008+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/519219556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x556c288ca150 con 0x556c286b5e40 2024-05-27T07:55:23.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.012+0000 7f8b3be00700 1 -- 172.21.15.90:0/519219556 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b4c0c4040 con 0x556c286b5e40 2024-05-27T07:55:23.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.012+0000 7f8b3be00700 1 -- 172.21.15.90:0/519219556 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 56) ==== 7757+0+0 (secure 0 0 0) 0x7f8b4c0a8630 con 0x556c286b5e40 2024-05-27T07:55:23.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.013+0000 7f8b6fc37b80 10 client.36752 fetch_fsmap finished waiting for FSMap version 56 2024-05-27T07:55:23.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.013+0000 7f8b6fc37b80 10 client.36752 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:55:23.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.013+0000 7f8b6fc37b80 20 client.36752 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:23.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.013+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c288ca4b0 0x556c288cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:23.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.013+0000 7f8b6fc37b80 4 client.36752 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:55:23.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.013+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/519219556 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c2875e570 con 0x556c288ca4b0 2024-05-27T07:55:23.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.014+0000 7f8b59400700 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c288ca4b0 0x556c288cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:23.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.015+0000 7f8b59400700 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c288ca4b0 0x556c288cc920 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=39b1d7092e72c3ba server_cookie=6d179f6c9f6a5302 in_seq=0 out_seq=0 2024-05-27T07:55:23.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.015+0000 7f8b3be00700 10 client.36752 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:55:23.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.016+0000 7f8b3be00700 1 -- 172.21.15.90:0/519219556 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c2875e570 con 0x556c288ca4b0 2024-05-27T07:55:23.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.016+0000 7f8b3be00700 10 client.36752 handle_command_reply: tid=0 2024-05-27T07:55:23.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.016+0000 7f8b6fc37b80 1 client.36752 shutdown 2024-05-27T07:55:23.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.017+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/519219556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 msgr2=0x556c288472d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:23.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.017+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c288472d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c005950 tx=0x7f8b4c0a9f40 comp rx=0 tx=0).stop 2024-05-27T07:55:23.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.017+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/519219556 shutdown_connections 2024-05-27T07:55:23.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.017+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x556c288ca4b0 0x556c288cc920 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.017+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c2883ad30 0x556c288562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.017+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c286b5e40 0x556c288472d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.017+0000 7f8b6fc37b80 1 --2- 172.21.15.90:0/519219556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556c28834db0 0x556c28855c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.017+0000 7f8b3be00700 0 client.36752 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:55:23.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.017+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/519219556 >> 172.21.15.90:0/519219556 conn(0x556c28715fc0 msgr2=0x556c28838880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:23.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.018+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/519219556 shutdown_connections 2024-05-27T07:55:23.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.018+0000 7f8b6fc37b80 1 -- 172.21.15.90:0/519219556 wait complete. 2024-05-27T07:55:23.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.019+0000 7f8b6fc37b80 20 client.36752 trim_cache size 0 max 16384 2024-05-27T07:55:23.022 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:55:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.021+0000 7f8b61000700 1 -- 172.21.15.90:0/3831896988 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8b400b2ab0 msgr2=0x7f8b400b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.021+0000 7f8b61000700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8b400b2ab0 0x7f8b400b4fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009680 tx=0x7f8b48009260 comp rx=0 tx=0).stop 2024-05-27T07:55:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.021+0000 7f8b61000700 1 -- 172.21.15.90:0/3831896988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b5c1045c0 msgr2=0x7f8b5c1a1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.021+0000 7f8b61000700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b5c1045c0 0x7f8b5c1a1350 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00e9c0 tx=0x7f8b4c00ecd0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.022+0000 7f8b61000700 1 -- 172.21.15.90:0/3831896988 shutdown_connections 2024-05-27T07:55:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.022+0000 7f8b61000700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b5c109840 0x7f8b5c1a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.022+0000 7f8b61000700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8b400b2ab0 0x7f8b400b4fc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.022+0000 7f8b61000700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b5c1045c0 0x7f8b5c1a1350 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.022+0000 7f8b61000700 1 --2- 172.21.15.90:0/3831896988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b5c108ef0 0x7f8b5c1a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:23.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.022+0000 7f8b61000700 1 -- 172.21.15.90:0/3831896988 >> 172.21.15.90:0/3831896988 conn(0x7f8b5c0ffe70 msgr2=0x7f8b5c102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:23.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.022+0000 7f8b61000700 1 -- 172.21.15.90:0/3831896988 shutdown_connections 2024-05-27T07:55:23.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:23.023+0000 7f8b61000700 1 -- 172.21.15.90:0/3831896988 wait complete. 2024-05-27T07:55:23.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:23 smithi090 ceph-mon[31429]: pgmap v285: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 583 op/s 2024-05-27T07:55:23.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:23 smithi136 ceph-mon[35324]: pgmap v285: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 583 op/s 2024-05-27T07:55:23.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:23 smithi129 ceph-mon[35254]: pgmap v285: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 583 op/s 2024-05-27T07:55:23.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 305.0 s, 604.2 tps, lat 162.668 ms stddev 47.536 2024-05-27T07:55:25.641 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:55:26.375 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:26.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:26 smithi136 ceph-mon[35324]: pgmap v286: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 702 op/s 2024-05-27T07:55:26.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:26 smithi129 ceph-mon[35254]: pgmap v286: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 702 op/s 2024-05-27T07:55:26.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:26 smithi090 ceph-mon[31429]: pgmap v286: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 702 op/s 2024-05-27T07:55:28.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:28 smithi090 ceph-mon[31429]: pgmap v287: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 520 op/s 2024-05-27T07:55:28.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:28 smithi129 ceph-mon[35254]: pgmap v287: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 520 op/s 2024-05-27T07:55:28.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:28 smithi136 ceph-mon[35324]: pgmap v287: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.2 MiB/s wr, 520 op/s 2024-05-27T07:55:28.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 310.0 s, 607.6 tps, lat 167.263 ms stddev 81.803 2024-05-27T07:55:30.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:30 smithi129 ceph-mon[35254]: pgmap v288: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 743 op/s 2024-05-27T07:55:30.652 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:30 smithi090 ceph-mon[31429]: pgmap v288: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 743 op/s 2024-05-27T07:55:30.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:30 smithi136 ceph-mon[35324]: pgmap v288: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 743 op/s 2024-05-27T07:55:31.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:31 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 6. 2024-05-27T07:55:31.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:31 smithi090 systemd[1]: Stopped Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:55:31.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:31 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Consumed 1.535s CPU time. 2024-05-27T07:55:31.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:31 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Start request repeated too quickly. 2024-05-27T07:55:31.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:31 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-05-27T07:55:31.964 INFO:journalctl@ceph.mds.d.smithi090.stdout:May 27 07:55:31 smithi090 systemd[1]: Failed to start Ceph mds.d for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:55:32.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:32 smithi129 ceph-mon[35254]: pgmap v289: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.6 MiB/s wr, 567 op/s 2024-05-27T07:55:32.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:32 smithi129 ceph-mon[35254]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:32.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:32 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:32.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:32 smithi090 ceph-mon[31429]: pgmap v289: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.6 MiB/s wr, 567 op/s 2024-05-27T07:55:32.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:32 smithi090 ceph-mon[31429]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:32.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:32 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:32.672 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f95f8200700 time 2024-05-27T07:55:32.188139+0000 2024-05-27T07:55:32.673 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:32.673 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:32.673 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f960dc25ba6] 2024-05-27T07:55:32.673 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f960dc25dd1] 2024-05-27T07:55:32.673 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eec223023d] 2024-05-27T07:55:32.673 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55eec221b971] 2024-05-27T07:55:32.673 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 5: (SnapRealm::check_cache() const+0x13e) [0x55eec2219fa0] 2024-05-27T07:55:32.673 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eec221a651] 2024-05-27T07:55:32.673 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eec21af347] 2024-05-27T07:55:32.673 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eec1eab87e] 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55eec1eb0f2f] 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 10: (EOpen::replay(MDSRank*)+0x57) [0x55eec1eb73e9] 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 11: (MDLog::_replay_thread()+0x1f38) [0x55eec225448c] 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55eec225d9cb] 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 13: (Thread::entry_wrapper()+0x43) [0x7f960dbf472d] 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 14: (Thread::_entry_func(void*)+0xd) [0x7f960dbf4749] 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f960c7981ca] 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 16: clone() 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.195+0000 7f95f8200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f95f8200700 time 2024-05-27T07:55:32.188139+0000 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:32.674 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f960dc25ba6] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f960dc25dd1] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eec223023d] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55eec221b971] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 5: (SnapRealm::check_cache() const+0x13e) [0x55eec2219fa0] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eec221a651] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eec21af347] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eec1eab87e] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55eec1eb0f2f] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 10: (EOpen::replay(MDSRank*)+0x57) [0x55eec1eb73e9] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 11: (MDLog::_replay_thread()+0x1f38) [0x55eec225448c] 2024-05-27T07:55:32.675 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55eec225d9cb] 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 13: (Thread::entry_wrapper()+0x43) [0x7f960dbf472d] 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 14: (Thread::_entry_func(void*)+0xd) [0x7f960dbf4749] 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f960c7981ca] 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 16: clone() 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: *** Caught signal (Aborted) ** 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: in thread 7f95f8200700 thread_name:md_log_replay 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f960c7a2d20] 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2: gsignal() 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 3: abort() 2024-05-27T07:55:32.676 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f960dc25ced] 2024-05-27T07:55:32.677 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f960dc25dd1] 2024-05-27T07:55:32.677 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eec223023d] 2024-05-27T07:55:32.677 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55eec221b971] 2024-05-27T07:55:32.677 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 8: (SnapRealm::check_cache() const+0x13e) [0x55eec2219fa0] 2024-05-27T07:55:32.677 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eec221a651] 2024-05-27T07:55:32.677 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eec21af347] 2024-05-27T07:55:32.677 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eec1eab87e] 2024-05-27T07:55:32.677 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55eec1eb0f2f] 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 13: (EOpen::replay(MDSRank*)+0x57) [0x55eec1eb73e9] 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 14: (MDLog::_replay_thread()+0x1f38) [0x55eec225448c] 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55eec225d9cb] 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 16: (Thread::entry_wrapper()+0x43) [0x7f960dbf472d] 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 17: (Thread::_entry_func(void*)+0xd) [0x7f960dbf4749] 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f960c7981ca] 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 19: clone() 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.202+0000 7f95f8200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: in thread 7f95f8200700 thread_name:md_log_replay 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:32.678 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f960c7a2d20] 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2: gsignal() 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 3: abort() 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f960dc25ced] 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f960dc25dd1] 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eec223023d] 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55eec221b971] 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 8: (SnapRealm::check_cache() const+0x13e) [0x55eec2219fa0] 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eec221a651] 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eec21af347] 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eec1eab87e] 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55eec1eb0f2f] 2024-05-27T07:55:32.679 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 13: (EOpen::replay(MDSRank*)+0x57) [0x55eec1eb73e9] 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 14: (MDLog::_replay_thread()+0x1f38) [0x55eec225448c] 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55eec225d9cb] 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 16: (Thread::entry_wrapper()+0x43) [0x7f960dbf472d] 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 17: (Thread::_entry_func(void*)+0xd) [0x7f960dbf4749] 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f960c7981ca] 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 19: clone() 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: -1> 2024-05-27T07:55:32.195+0000 7f95f8200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f95f8200700 time 2024-05-27T07:55:32.188139+0000 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:32.680 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f960dc25ba6] 2024-05-27T07:55:32.681 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f960dc25dd1] 2024-05-27T07:55:32.681 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eec223023d] 2024-05-27T07:55:32.681 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55eec221b971] 2024-05-27T07:55:32.681 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 5: (SnapRealm::check_cache() const+0x13e) [0x55eec2219fa0] 2024-05-27T07:55:32.681 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eec221a651] 2024-05-27T07:55:32.681 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eec21af347] 2024-05-27T07:55:32.681 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eec1eab87e] 2024-05-27T07:55:32.682 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55eec1eb0f2f] 2024-05-27T07:55:32.682 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 10: (EOpen::replay(MDSRank*)+0x57) [0x55eec1eb73e9] 2024-05-27T07:55:32.682 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 11: (MDLog::_replay_thread()+0x1f38) [0x55eec225448c] 2024-05-27T07:55:32.682 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55eec225d9cb] 2024-05-27T07:55:32.682 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 13: (Thread::entry_wrapper()+0x43) [0x7f960dbf472d] 2024-05-27T07:55:32.682 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 14: (Thread::_entry_func(void*)+0xd) [0x7f960dbf4749] 2024-05-27T07:55:32.682 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f960c7981ca] 2024-05-27T07:55:32.682 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 16: clone() 2024-05-27T07:55:32.682 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.682 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 0> 2024-05-27T07:55:32.202+0000 7f95f8200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: in thread 7f95f8200700 thread_name:md_log_replay 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f960c7a2d20] 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2: gsignal() 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 3: abort() 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f960dc25ced] 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f960dc25dd1] 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eec223023d] 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55eec221b971] 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 8: (SnapRealm::check_cache() const+0x13e) [0x55eec2219fa0] 2024-05-27T07:55:32.683 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eec221a651] 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eec21af347] 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eec1eab87e] 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55eec1eb0f2f] 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 13: (EOpen::replay(MDSRank*)+0x57) [0x55eec1eb73e9] 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 14: (MDLog::_replay_thread()+0x1f38) [0x55eec225448c] 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55eec225d9cb] 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 16: (Thread::entry_wrapper()+0x43) [0x7f960dbf472d] 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 17: (Thread::_entry_func(void*)+0xd) [0x7f960dbf4749] 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f960c7981ca] 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 19: clone() 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.684 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: -1> 2024-05-27T07:55:32.195+0000 7f95f8200700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f95f8200700 time 2024-05-27T07:55:32.188139+0000 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f960dc25ba6] 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f960dc25dd1] 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eec223023d] 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 4: (SnapRealm::build_snap_set() const+0x323) [0x55eec221b971] 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 5: (SnapRealm::check_cache() const+0x13e) [0x55eec2219fa0] 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eec221a651] 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eec21af347] 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eec1eab87e] 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55eec1eb0f2f] 2024-05-27T07:55:32.685 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 10: (EOpen::replay(MDSRank*)+0x57) [0x55eec1eb73e9] 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 11: (MDLog::_replay_thread()+0x1f38) [0x55eec225448c] 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 12: (MDLog::ReplayThread::entry()+0x11) [0x55eec225d9cb] 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 13: (Thread::entry_wrapper()+0x43) [0x7f960dbf472d] 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 14: (Thread::_entry_func(void*)+0xd) [0x7f960dbf4749] 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f960c7981ca] 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 16: clone() 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 0> 2024-05-27T07:55:32.202+0000 7f95f8200700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: in thread 7f95f8200700 thread_name:md_log_replay 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f960c7a2d20] 2024-05-27T07:55:32.686 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2: gsignal() 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 3: abort() 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f960dc25ced] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f960dc25dd1] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x55eec223023d] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 7: (SnapRealm::build_snap_set() const+0x323) [0x55eec221b971] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 8: (SnapRealm::check_cache() const+0x13e) [0x55eec2219fa0] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x55eec221a651] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x55eec21af347] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x55eec1eab87e] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x55eec1eb0f2f] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 13: (EOpen::replay(MDSRank*)+0x57) [0x55eec1eb73e9] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 14: (MDLog::_replay_thread()+0x1f38) [0x55eec225448c] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 15: (MDLog::ReplayThread::entry()+0x11) [0x55eec225d9cb] 2024-05-27T07:55:32.687 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 16: (Thread::entry_wrapper()+0x43) [0x7f960dbf472d] 2024-05-27T07:55:32.688 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 17: (Thread::_entry_func(void*)+0xd) [0x7f960dbf4749] 2024-05-27T07:55:32.688 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f960c7981ca] 2024-05-27T07:55:32.688 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 19: clone() 2024-05-27T07:55:32.688 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:32.688 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[113465]: 2024-05-27T07:55:32.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:32 smithi136 ceph-mon[35324]: pgmap v289: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.6 MiB/s wr, 567 op/s 2024-05-27T07:55:32.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:32 smithi136 ceph-mon[35324]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:32.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:32 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:33.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 315.0 s, 573.2 tps, lat 168.740 ms stddev 118.779 2024-05-27T07:55:34.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:34 smithi129 ceph-mon[35254]: pgmap v290: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.7 MiB/s wr, 581 op/s 2024-05-27T07:55:34.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:34 smithi090 ceph-mon[31429]: pgmap v290: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.7 MiB/s wr, 581 op/s 2024-05-27T07:55:34.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:34 smithi136 ceph-mon[35324]: pgmap v290: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.7 MiB/s wr, 581 op/s 2024-05-27T07:55:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.799+0000 7f4dbde00640 1 -- 172.21.15.90:0/2217580670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4db8075480 msgr2=0x7f4db8075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.799+0000 7f4dbde00640 1 --2- 172.21.15.90:0/2217580670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4db8075480 0x7f4db8075890 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009900 tx=0x7f4da802a490 comp rx=0 tx=0).stop 2024-05-27T07:55:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.800+0000 7f4dbde00640 1 -- 172.21.15.90:0/2217580670 shutdown_connections 2024-05-27T07:55:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.800+0000 7f4dbde00640 1 --2- 172.21.15.90:0/2217580670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4db8112310 0x7f4db8114730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.800+0000 7f4dbde00640 1 --2- 172.21.15.90:0/2217580670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4db8114c70 0x7f4db8117090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.800+0000 7f4dbde00640 1 --2- 172.21.15.90:0/2217580670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4db8075480 0x7f4db8075890 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.800+0000 7f4dbde00640 1 -- 172.21.15.90:0/2217580670 >> 172.21.15.90:0/2217580670 conn(0x7f4db80ff480 msgr2=0x7f4db81018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.800+0000 7f4dbde00640 1 -- 172.21.15.90:0/2217580670 shutdown_connections 2024-05-27T07:55:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.801+0000 7f4dbde00640 1 -- 172.21.15.90:0/2217580670 wait complete. 2024-05-27T07:55:35.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.803+0000 7f4dbde00640 1 Processor -- start 2024-05-27T07:55:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.803+0000 7f4dbde00640 1 -- start start 2024-05-27T07:55:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.804+0000 7f4dbde00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4db8075480 0x7f4db81a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.804+0000 7f4dbde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4db8112310 0x7f4db81a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.804+0000 7f4dbde00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4db8114c70 0x7f4db81a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.804+0000 7f4dbde00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4db811a310 con 0x7f4db8114c70 2024-05-27T07:55:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.804+0000 7f4dbde00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4db811a190 con 0x7f4db8075480 2024-05-27T07:55:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.804+0000 7f4dbde00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4db811a490 con 0x7f4db8112310 2024-05-27T07:55:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.805+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4db8075480 0x7f4db81a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.805+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4db8075480 0x7f4db81a0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:35450/0 (socket says 172.21.15.90:35450) 2024-05-27T07:55:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.805+0000 7f4db6a00640 1 -- 172.21.15.90:0/2823399453 learned_addr learned my addr 172.21.15.90:0/2823399453 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.805+0000 7f4db6000640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4db8112310 0x7f4db81a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.805+0000 7f4db6000640 1 -- 172.21.15.90:0/2823399453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4db8075480 msgr2=0x7f4db81a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.805+0000 7f4db6000640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4db8075480 0x7f4db81a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.805+0000 7f4db6000640 1 -- 172.21.15.90:0/2823399453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4db8114c70 msgr2=0x7f4db81a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.805+0000 7f4db6000640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4db8114c70 0x7f4db81a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.805+0000 7f4db6000640 1 -- 172.21.15.90:0/2823399453 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db81a5c30 con 0x7f4db8112310 2024-05-27T07:55:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.806+0000 7f4db6000640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4db8112310 0x7f4db81a13d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4da400e9f0 tx=0x7f4da400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.806+0000 7f4dbca00640 1 -- 172.21.15.90:0/2823399453 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da400ccc0 con 0x7f4db8112310 2024-05-27T07:55:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.807+0000 7f4dbca00640 1 -- 172.21.15.90:0/2823399453 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4da4005020 con 0x7f4db8112310 2024-05-27T07:55:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.807+0000 7f4dbde00640 1 -- 172.21.15.90:0/2823399453 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db81a5f20 con 0x7f4db8112310 2024-05-27T07:55:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.807+0000 7f4dbde00640 1 -- 172.21.15.90:0/2823399453 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db81a63c0 con 0x7f4db8112310 2024-05-27T07:55:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.808+0000 7f4dbca00640 1 -- 172.21.15.90:0/2823399453 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da400ce60 con 0x7f4db8112310 2024-05-27T07:55:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.817+0000 7f4dbca00640 1 -- 172.21.15.90:0/2823399453 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4da4004b20 con 0x7f4db8112310 2024-05-27T07:55:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.819+0000 7f4dbca00640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4d9c0b2950 0x7f4d9c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.819+0000 7f4dbca00640 1 -- 172.21.15.90:0/2823399453 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f4da40e2a30 con 0x7f4db8112310 2024-05-27T07:55:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.820+0000 7f4db6a00640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4d9c0b2950 0x7f4d9c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.821+0000 7f4db6a00640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4d9c0b2950 0x7f4d9c0b4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4da8000c00 tx=0x7f4da8009580 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.832+0000 7f4dc4e4d740 1 Processor -- start 2024-05-27T07:55:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.832+0000 7f4dc4e4d740 1 -- start start 2024-05-27T07:55:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.833+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003c01b0 0x559c00762860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.833+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 0x559c00762da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.833+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 0x559c007632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.833+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559c006d3b90 con 0x559c0039f4d0 2024-05-27T07:55:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.833+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559c006c3730 con 0x559c003972f0 2024-05-27T07:55:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.833+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559c006c7770 con 0x559c003c01b0 2024-05-27T07:55:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.833+0000 7f4db7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 0x559c007632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.833+0000 7f4db7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 0x559c007632e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58676/0 (socket says 172.21.15.90:58676) 2024-05-27T07:55:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.833+0000 7f4db7400640 1 -- 172.21.15.90:0/294808199 learned_addr learned my addr 172.21.15.90:0/294808199 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.833+0000 7f4db6a00640 1 --2- 172.21.15.90:0/294808199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003c01b0 0x559c00762860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.834+0000 7f4db6000640 1 --2- 172.21.15.90:0/294808199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 0x559c00762da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.834+0000 7f4db6a00640 1 -- 172.21.15.90:0/294808199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 msgr2=0x559c00762da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.834+0000 7f4db6a00640 1 --2- 172.21.15.90:0/294808199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 0x559c00762da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.834+0000 7f4db6a00640 1 -- 172.21.15.90:0/294808199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 msgr2=0x559c007632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.834+0000 7f4db6a00640 1 --2- 172.21.15.90:0/294808199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 0x559c007632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.834+0000 7f4db6a00640 1 -- 172.21.15.90:0/294808199 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0065c8b0 con 0x559c003c01b0 2024-05-27T07:55:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.834+0000 7f4db6000640 1 --2- 172.21.15.90:0/294808199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 0x559c00762da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.834+0000 7f4db6a00640 1 --2- 172.21.15.90:0/294808199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003c01b0 0x559c00762860 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4da80057c0 tx=0x7f4da803e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6c2b37796216369 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.835+0000 7f4da3e00640 1 -- 172.21.15.90:0/294808199 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da804c070 con 0x559c003c01b0 2024-05-27T07:55:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.835+0000 7f4da3e00640 1 -- 172.21.15.90:0/294808199 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4da8004300 con 0x559c003c01b0 2024-05-27T07:55:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.835+0000 7f4da3e00640 1 -- 172.21.15.90:0/294808199 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8035050 con 0x559c003c01b0 2024-05-27T07:55:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.836+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/294808199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003c01b0 msgr2=0x559c00762860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.836+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/294808199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003c01b0 0x559c00762860 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4da80057c0 tx=0x7f4da803e000 comp rx=0 tx=0).stop 2024-05-27T07:55:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.837+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/294808199 shutdown_connections 2024-05-27T07:55:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.837+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/294808199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 0x559c00762da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.837+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/294808199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 0x559c007632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.837+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/294808199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003c01b0 0x559c00762860 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.838+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/294808199 >> 172.21.15.90:0/294808199 conn(0x559c003b4300 msgr2=0x559c0075f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.838+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/294808199 shutdown_connections 2024-05-27T07:55:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.839+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/294808199 wait complete. 2024-05-27T07:55:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.840+0000 7f4dc4e4d740 1 -- start start 2024-05-27T07:55:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.840+0000 7f4dc4e4d740 1 Processor -- start 2024-05-27T07:55:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.841+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 0x559c0076ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.841+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c0039f4d0 0x559c0076cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.841+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 0x559c00778c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.841+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559c003a9790 con 0x559c003c01b0 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.841+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559c0066bbc0 con 0x559c003972f0 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.841+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559c0076f930 con 0x559c0039f4d0 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.841+0000 7f4db6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c0039f4d0 0x559c0076cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.841+0000 7f4db6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c0039f4d0 0x559c0076cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:55942/0 (socket says 172.21.15.90:55942) 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.841+0000 7f4db6000640 1 -- 172.21.15.90:0/3330926682 learned_addr learned my addr 172.21.15.90:0/3330926682 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.842+0000 7f4dc4e4d740 10 client.0 _ll_register_callbacks cb 0x559c00649400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.842+0000 7f4db6a00640 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 0x559c0076ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.842+0000 7f4db7400640 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 0x559c00778c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.843+0000 7f4db7400640 1 -- 172.21.15.90:0/3330926682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c0039f4d0 msgr2=0x559c0076cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.843+0000 7f4db7400640 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c0039f4d0 0x559c0076cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.843+0000 7f4db7400640 1 -- 172.21.15.90:0/3330926682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 msgr2=0x559c0076ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.843+0000 7f4db7400640 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 0x559c0076ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.843+0000 7f4db7400640 1 -- 172.21.15.90:0/3330926682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c003d8210 con 0x559c003c01b0 2024-05-27T07:55:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.843+0000 7f4db6a00640 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 0x559c0076ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.844+0000 7f4db7400640 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 0x559c00778c60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4db00049d0 tx=0x7f4db0010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec827a9722afa90 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.844+0000 7f4da0c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:55:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.845+0000 7f4da0c00640 1 -- 172.21.15.90:0/3330926682 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0017070 con 0x559c003c01b0 2024-05-27T07:55:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.845+0000 7f4da0c00640 1 -- 172.21.15.90:0/3330926682 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f4db0004d20 con 0x559c003c01b0 2024-05-27T07:55:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.845+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3330926682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da8004300 con 0x559c003c01b0 2024-05-27T07:55:35.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.847+0000 7f4da0c00640 1 -- 172.21.15.90:0/3330926682 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4db0024050 con 0x559c003c01b0 2024-05-27T07:55:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.848+0000 7f4dc4e4d740 10 client.17190 fetch_fsmap learned FSMap version 57 2024-05-27T07:55:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.848+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3330926682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559c004a4620 con 0x559c003c01b0 2024-05-27T07:55:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.850+0000 7f4da0c00640 1 -- 172.21.15.90:0/3330926682 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4da8004300 con 0x559c003c01b0 2024-05-27T07:55:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.850+0000 7f4da0c00640 1 -- 172.21.15.90:0/3330926682 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 57) ==== 7152+0+0 (secure 0 0 0) 0x7f4db0011be0 con 0x559c003c01b0 2024-05-27T07:55:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.850+0000 7f4dc4e4d740 10 client.17190 fetch_fsmap finished waiting for FSMap version 57 2024-05-27T07:55:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.850+0000 7f4dc4e4d740 10 client.17190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.850+0000 7f4dc4e4d740 20 client.17190 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.850+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559c0037ed80 0x559c0077ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.852+0000 7f4dc4e4d740 4 client.17190 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.852+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3330926682 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c006d5ae0 con 0x559c0037ed80 2024-05-27T07:55:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.852+0000 7f4db6a00640 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559c0037ed80 0x559c0077ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.854+0000 7f4db6a00640 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559c0037ed80 0x559c0077ebe0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ca6e71a30bcf90 server_cookie=3e66b63b18f1e302 in_seq=0 out_seq=0 2024-05-27T07:55:35.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.854+0000 7f4da0c00640 10 client.17190 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.857+0000 7f4da0c00640 1 -- 172.21.15.90:0/3330926682 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559c006d5ae0 con 0x559c0037ed80 2024-05-27T07:55:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.858+0000 7f4da0c00640 10 client.17190 handle_command_reply: tid=0 2024-05-27T07:55:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.858+0000 7f4dc4e4d740 1 client.17190 shutdown 2024-05-27T07:55:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.860+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3330926682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 msgr2=0x559c00778c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:35.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.860+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 0x559c00778c60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4db00049d0 tx=0x7f4db0010ca0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.860+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3330926682 shutdown_connections 2024-05-27T07:55:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.860+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559c0037ed80 0x559c0077ebe0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.860+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 0x559c00778c60 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.861+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c0039f4d0 0x559c0076cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.861+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/3330926682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003972f0 0x559c0076ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.861+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3330926682 >> 172.21.15.90:0/3330926682 conn(0x559c003b4300 msgr2=0x559c00762630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.860+0000 7f4da0c00640 0 client.17190 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.861+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3330926682 shutdown_connections 2024-05-27T07:55:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.862+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3330926682 wait complete. 2024-05-27T07:55:35.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.862+0000 7f4dc4e4d740 20 client.17190 trim_cache size 0 max 16384 2024-05-27T07:55:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.899+0000 7f4dc4e4d740 1 Processor -- start 2024-05-27T07:55:35.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.900+0000 7f4dc4e4d740 1 -- start start 2024-05-27T07:55:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.900+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 0x559c0076f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.900+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 0x559c007789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.901+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003c01b0 0x559c00778f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.901+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81b6360 con 0x559c0039f4d0 2024-05-27T07:55:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.901+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81b8c40 con 0x559c003c01b0 2024-05-27T07:55:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.901+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81b6a30 con 0x559c003972f0 2024-05-27T07:55:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.901+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 0x559c0076f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.901+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 0x559c0076f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:55944/0 (socket says 172.21.15.90:55944) 2024-05-27T07:55:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.901+0000 7f4db6a00640 1 -- 172.21.15.90:0/3159819752 learned_addr learned my addr 172.21.15.90:0/3159819752 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.901+0000 7f4db7400640 1 --2- 172.21.15.90:0/3159819752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003c01b0 0x559c00778f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.901+0000 7f4db6000640 1 --2- 172.21.15.90:0/3159819752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 0x559c007789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.902+0000 7f4db6000640 1 -- 172.21.15.90:0/3159819752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 msgr2=0x559c0076f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.902+0000 7f4db6000640 1 --2- 172.21.15.90:0/3159819752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 0x559c0076f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.902+0000 7f4db6000640 1 -- 172.21.15.90:0/3159819752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003c01b0 msgr2=0x559c00778f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.902+0000 7f4db6000640 1 --2- 172.21.15.90:0/3159819752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003c01b0 0x559c00778f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.902+0000 7f4db6000640 1 -- 172.21.15.90:0/3159819752 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c00763520 con 0x559c0039f4d0 2024-05-27T07:55:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.902+0000 7f4db6a00640 1 --2- 172.21.15.90:0/3159819752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 0x559c0076f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.903+0000 7f4db6000640 1 --2- 172.21.15.90:0/3159819752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 0x559c007789d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4da40a59c0 tx=0x7f4da40a50d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9928ff7bc098b67e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.904+0000 7f4da3400640 1 -- 172.21.15.90:0/3159819752 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da40c3070 con 0x559c0039f4d0 2024-05-27T07:55:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.904+0000 7f4da3400640 1 -- 172.21.15.90:0/3159819752 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4da40aa040 con 0x559c0039f4d0 2024-05-27T07:55:35.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.904+0000 7f4da3400640 1 -- 172.21.15.90:0/3159819752 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da401c070 con 0x559c0039f4d0 2024-05-27T07:55:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.907+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3159819752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 msgr2=0x559c007789d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:35.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.907+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/3159819752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 0x559c007789d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4da40a59c0 tx=0x7f4da40a50d0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.907+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3159819752 shutdown_connections 2024-05-27T07:55:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.907+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/3159819752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c003c01b0 0x559c00778f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.907+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/3159819752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c0039f4d0 0x559c007789d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.907+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/3159819752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 0x559c0076f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.907+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3159819752 >> 172.21.15.90:0/3159819752 conn(0x559c003b4300 msgr2=0x559c0076c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.908+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3159819752 shutdown_connections 2024-05-27T07:55:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.908+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/3159819752 wait complete. 2024-05-27T07:55:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.909+0000 7f4dc4e4d740 1 -- start start 2024-05-27T07:55:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.909+0000 7f4dc4e4d740 1 Processor -- start 2024-05-27T07:55:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 0x559c00775440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c0039f4d0 0x559c007717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4dc4e4d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 0x559c00771e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81b7100 con 0x559c003c01b0 2024-05-27T07:55:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81b8570 con 0x559c0039f4d0 2024-05-27T07:55:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4dc4e4d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81b77d0 con 0x559c003972f0 2024-05-27T07:55:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4dc4e4d740 10 client.0 _ll_register_callbacks cb 0x7f4db00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4db7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 0x559c00771e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4db7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 0x559c00771e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58704/0 (socket says 172.21.15.90:58704) 2024-05-27T07:55:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4db7400640 1 -- 172.21.15.90:0/1023343493 learned_addr learned my addr 172.21.15.90:0/1023343493 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.910+0000 7f4db6a00640 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 0x559c00775440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.911+0000 7f4db6000640 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c0039f4d0 0x559c007717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.912+0000 7f4db6000640 1 -- 172.21.15.90:0/1023343493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 msgr2=0x559c00775440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.912+0000 7f4db6000640 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 0x559c00775440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.912+0000 7f4db6000640 1 -- 172.21.15.90:0/1023343493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 msgr2=0x559c00771e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.912+0000 7f4db6000640 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 0x559c00771e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.912+0000 7f4db6000640 1 -- 172.21.15.90:0/1023343493 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c007725d0 con 0x559c0039f4d0 2024-05-27T07:55:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.912+0000 7f4db6000640 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c0039f4d0 0x559c007717f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x559c0075f5f0 tx=0x7f4da4005460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2205a2e6b835561 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.912+0000 7f4da0c00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:55:35.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.913+0000 7f4da0c00640 1 -- 172.21.15.90:0/1023343493 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da40c3040 con 0x559c0039f4d0 2024-05-27T07:55:35.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.913+0000 7f4da0c00640 1 -- 172.21.15.90:0/1023343493 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f4da40cc0d0 con 0x559c0039f4d0 2024-05-27T07:55:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.915+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/1023343493 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da40aa040 con 0x559c0039f4d0 2024-05-27T07:55:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.915+0000 7f4da0c00640 1 -- 172.21.15.90:0/1023343493 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4da401c040 con 0x559c0039f4d0 2024-05-27T07:55:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.916+0000 7f4dc4e4d740 10 client.36767 fetch_fsmap learned FSMap version 57 2024-05-27T07:55:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.916+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/1023343493 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559c007e6280 con 0x559c0039f4d0 2024-05-27T07:55:35.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.916+0000 7f4da0c00640 1 -- 172.21.15.90:0/1023343493 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4da40aa040 con 0x559c0039f4d0 2024-05-27T07:55:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.918+0000 7f4da0c00640 1 -- 172.21.15.90:0/1023343493 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 57) ==== 7152+0+0 (secure 0 0 0) 0x7f4da40b2310 con 0x559c0039f4d0 2024-05-27T07:55:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.919+0000 7f4dc4e4d740 10 client.36767 fetch_fsmap finished waiting for FSMap version 57 2024-05-27T07:55:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.919+0000 7f4dc4e4d740 10 client.36767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.919+0000 7f4dc4e4d740 20 client.36767 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.919+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559c003541c0 0x559c007e88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.919+0000 7f4dc4e4d740 4 client.36767 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:55:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.919+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/1023343493 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c006cf2c0 con 0x559c003541c0 2024-05-27T07:55:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.920+0000 7f4db6a00640 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559c003541c0 0x559c007e88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.922+0000 7f4db6a00640 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559c003541c0 0x559c007e88d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d314baf0a06ca4 server_cookie=cbe7330c3ff7471c in_seq=0 out_seq=0 2024-05-27T07:55:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.922+0000 7f4da0c00640 10 client.36767 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.968+0000 7f4da0c00640 1 -- 172.21.15.90:0/1023343493 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559c006cf2c0 con 0x559c003541c0 2024-05-27T07:55:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.968+0000 7f4da0c00640 10 client.36767 handle_command_reply: tid=0 2024-05-27T07:55:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.968+0000 7f4dc4e4d740 1 client.36767 shutdown 2024-05-27T07:55:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.969+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/1023343493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c0039f4d0 msgr2=0x559c007717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.969+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c0039f4d0 0x559c007717f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x559c0075f5f0 tx=0x7f4da4005460 comp rx=0 tx=0).stop 2024-05-27T07:55:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.969+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/1023343493 shutdown_connections 2024-05-27T07:55:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.969+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559c003541c0 0x559c007e88d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.969+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c0039f4d0 0x559c007717f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.969+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559c003c01b0 0x559c00771e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.969+0000 7f4dc4e4d740 1 --2- 172.21.15.90:0/1023343493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c003972f0 0x559c00775440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.969+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/1023343493 >> 172.21.15.90:0/1023343493 conn(0x559c003b4300 msgr2=0x559c0076cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.969+0000 7f4da0c00640 0 client.36767 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.969+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/1023343493 shutdown_connections 2024-05-27T07:55:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.970+0000 7f4dc4e4d740 1 -- 172.21.15.90:0/1023343493 wait complete. 2024-05-27T07:55:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.970+0000 7f4dc4e4d740 20 client.36767 trim_cache size 0 max 16384 2024-05-27T07:55:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:55:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:55:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:55:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.972+0000 7f4dbde00640 1 -- 172.21.15.90:0/2823399453 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4d9c0b2950 msgr2=0x7f4d9c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.973+0000 7f4dbde00640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4d9c0b2950 0x7f4d9c0b4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4da8000c00 tx=0x7f4da8009580 comp rx=0 tx=0).stop 2024-05-27T07:55:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.973+0000 7f4dbde00640 1 -- 172.21.15.90:0/2823399453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4db8112310 msgr2=0x7f4db81a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.973+0000 7f4dbde00640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4db8112310 0x7f4db81a13d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4da400e9f0 tx=0x7f4da400eeb0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.973+0000 7f4dbde00640 1 -- 172.21.15.90:0/2823399453 shutdown_connections 2024-05-27T07:55:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.973+0000 7f4dbde00640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4d9c0b2950 0x7f4d9c0b4e60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.973+0000 7f4dbde00640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4db8114c70 0x7f4db81a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.973+0000 7f4dbde00640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4db8112310 0x7f4db81a13d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.973+0000 7f4dbde00640 1 --2- 172.21.15.90:0/2823399453 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4db8075480 0x7f4db81a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.973+0000 7f4dbde00640 1 -- 172.21.15.90:0/2823399453 >> 172.21.15.90:0/2823399453 conn(0x7f4db80ff480 msgr2=0x7f4db80ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.974+0000 7f4dbde00640 1 -- 172.21.15.90:0/2823399453 shutdown_connections 2024-05-27T07:55:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:35.974+0000 7f4dbde00640 1 -- 172.21.15.90:0/2823399453 wait complete. 2024-05-27T07:55:36.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:35 smithi090 podman[113753]: 2024-05-27 07:55:35.867368032 +0000 UTC m=+2.587319273 container died e6e05b5296f9da10e0512003a1d00d116009eef42691772765799a4251b6f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-27T07:55:36.554 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:36 smithi090 ceph-mon[31429]: pgmap v291: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-05-27T07:55:36.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:36 smithi129 ceph-mon[35254]: pgmap v291: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-05-27T07:55:36.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:36 smithi136 ceph-mon[35324]: pgmap v291: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.2 MiB/s wr, 724 op/s 2024-05-27T07:55:38.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:38 smithi129 ceph-mon[35254]: pgmap v292: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.0 MiB/s wr, 515 op/s 2024-05-27T07:55:38.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:38 smithi090 ceph-mon[31429]: pgmap v292: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.0 MiB/s wr, 515 op/s 2024-05-27T07:55:38.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:38 smithi136 ceph-mon[35324]: pgmap v292: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.0 MiB/s wr, 515 op/s 2024-05-27T07:55:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 320.0 s, 495.0 tps, lat 206.643 ms stddev 194.322 2024-05-27T07:55:40.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:40 smithi129 ceph-mon[35254]: pgmap v293: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-05-27T07:55:40.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:40 smithi090 ceph-mon[31429]: pgmap v293: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-05-27T07:55:40.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:40 smithi136 ceph-mon[35324]: pgmap v293: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-05-27T07:55:41.197 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:40 smithi090 podman[113753]: 2024-05-27 07:55:40.911869565 +0000 UTC m=+7.631820783 container remove e6e05b5296f9da10e0512003a1d00d116009eef42691772765799a4251b6f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:55:41.197 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:40 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:55:41.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:41 smithi090 ceph-mon[31429]: pgmap v294: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-05-27T07:55:41.761 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:41 smithi136 ceph-mon[35324]: pgmap v294: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-05-27T07:55:41.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:41 smithi129 ceph-mon[35254]: pgmap v294: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.9 MiB/s wr, 505 op/s 2024-05-27T07:55:43.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.055+0000 7f22dfe00700 1 -- 172.21.15.90:0/2670651534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22d81054b0 msgr2=0x7f22d81058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:43.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.055+0000 7f22dfe00700 1 --2- 172.21.15.90:0/2670651534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22d81054b0 0x7f22d81058c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f22d000b0a0 tx=0x7f22d000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.057+0000 7f22dfe00700 1 -- 172.21.15.90:0/2670651534 shutdown_connections 2024-05-27T07:55:43.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.057+0000 7f22dfe00700 1 --2- 172.21.15.90:0/2670651534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22d810e8e0 0x7f22d8110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.057+0000 7f22dfe00700 1 --2- 172.21.15.90:0/2670651534 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22d81054b0 0x7f22d81058c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.057+0000 7f22dfe00700 1 --2- 172.21.15.90:0/2670651534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22d8105e00 0x7f22d810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.057+0000 7f22dfe00700 1 -- 172.21.15.90:0/2670651534 >> 172.21.15.90:0/2670651534 conn(0x7f22d80fedb0 msgr2=0x7f22d81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:43.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.057+0000 7f22dfe00700 1 -- 172.21.15.90:0/2670651534 shutdown_connections 2024-05-27T07:55:43.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.058+0000 7f22dfe00700 1 -- 172.21.15.90:0/2670651534 wait complete. 2024-05-27T07:55:43.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.059+0000 7f22dfe00700 1 Processor -- start 2024-05-27T07:55:43.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.059+0000 7f22dfe00700 1 -- start start 2024-05-27T07:55:43.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.060+0000 7f22dfe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22d81054b0 0x7f22d80753f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.060+0000 7f22dfe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22d8105e00 0x7f22d806e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.060+0000 7f22dfe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22d810e8e0 0x7f22d806ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.060+0000 7f22dfe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f22d81143b0 con 0x7f22d810e8e0 2024-05-27T07:55:43.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.060+0000 7f22dfe00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8114230 con 0x7f22d8105e00 2024-05-27T07:55:43.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.060+0000 7f22dfe00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f22d8114530 con 0x7f22d81054b0 2024-05-27T07:55:43.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.061+0000 7f22dca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22d8105e00 0x7f22d806e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.061+0000 7f22dca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22d8105e00 0x7f22d806e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:36450/0 (socket says 172.21.15.90:36450) 2024-05-27T07:55:43.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.061+0000 7f22dca00700 1 -- 172.21.15.90:0/2867913137 learned_addr learned my addr 172.21.15.90:0/2867913137 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:43.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.062+0000 7f22dd400700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22d81054b0 0x7f22d80753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.062+0000 7f22dca00700 1 -- 172.21.15.90:0/2867913137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22d81054b0 msgr2=0x7f22d80753f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:43.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.062+0000 7f22dca00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22d81054b0 0x7f22d80753f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.062+0000 7f22dca00700 1 -- 172.21.15.90:0/2867913137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22d810e8e0 msgr2=0x7f22d806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:43.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.062+0000 7f22dca00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22d810e8e0 0x7f22d806ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.062+0000 7f22dca00700 1 -- 172.21.15.90:0/2867913137 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d8072f20 con 0x7f22d8105e00 2024-05-27T07:55:43.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.063+0000 7f22dca00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22d8105e00 0x7f22d806e520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b530 tx=0x7f22cc00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:43.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.064+0000 7f22d6000700 1 -- 172.21.15.90:0/2867913137 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc015020 con 0x7f22d8105e00 2024-05-27T07:55:43.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.064+0000 7f22d6000700 1 -- 172.21.15.90:0/2867913137 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22cc011cc0 con 0x7f22d8105e00 2024-05-27T07:55:43.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.064+0000 7f22d6000700 1 -- 172.21.15.90:0/2867913137 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc015020 con 0x7f22d8105e00 2024-05-27T07:55:43.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.064+0000 7f22dfe00700 1 -- 172.21.15.90:0/2867913137 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d80731b0 con 0x7f22d8105e00 2024-05-27T07:55:43.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.065+0000 7f22dfe00700 1 -- 172.21.15.90:0/2867913137 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d80736a0 con 0x7f22d8105e00 2024-05-27T07:55:43.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.077+0000 7f22d6000700 1 -- 172.21.15.90:0/2867913137 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f22cc018020 con 0x7f22d8105e00 2024-05-27T07:55:43.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.081+0000 7f22ee848b80 1 Processor -- start 2024-05-27T07:55:43.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.081+0000 7f22d6000700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f22c00b2cf0 0x7f22c00b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.081+0000 7f22d6000700 1 -- 172.21.15.90:0/2867913137 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f22cc0e1d00 con 0x7f22d8105e00 2024-05-27T07:55:43.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.082+0000 7f22ee848b80 1 -- start start 2024-05-27T07:55:43.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.083+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab9877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.083+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab981870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.083+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab9821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.083+0000 7f22ee848b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bbab8afe40 con 0x55bbab981db0 2024-05-27T07:55:43.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.083+0000 7f22ee848b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bbab89f9e0 con 0x55bbab987d30 2024-05-27T07:55:43.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.083+0000 7f22ee848b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bbab8a3a20 con 0x55bbab802e40 2024-05-27T07:55:43.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.083+0000 7f22dde00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f22c00b2cf0 0x7f22c00b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.084+0000 7f22dd400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab9877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.084+0000 7f22dd400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab9877f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57652/0 (socket says 172.21.15.90:57652) 2024-05-27T07:55:43.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.084+0000 7f22dd400700 1 -- 172.21.15.90:0/1885158542 learned_addr learned my addr 172.21.15.90:0/1885158542 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:43.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.084+0000 7f22dd400700 1 -- 172.21.15.90:0/1885158542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 msgr2=0x55bbab981870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:43.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.084+0000 7f22dd400700 1 --2- 172.21.15.90:0/1885158542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab981870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.085+0000 7f22dd400700 1 -- 172.21.15.90:0/1885158542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 msgr2=0x55bbab9821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:43.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.085+0000 7f22dd400700 1 --2- 172.21.15.90:0/1885158542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab9821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.085+0000 7f22dd400700 1 -- 172.21.15.90:0/1885158542 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbab868420 con 0x55bbab802e40 2024-05-27T07:55:43.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.086+0000 7f22dd400700 1 --2- 172.21.15.90:0/1885158542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab9877f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f22d0000f80 tx=0x7f22d00092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d259ff4a5baa534b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:43.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.086+0000 7f22caa00700 1 -- 172.21.15.90:0/1885158542 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0020070 con 0x55bbab802e40 2024-05-27T07:55:43.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.086+0000 7f22dde00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f22c00b2cf0 0x7f22c00b5200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22d80759b0 tx=0x7f22c400a3b0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:43.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.087+0000 7f22caa00700 1 -- 172.21.15.90:0/1885158542 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22d0003c00 con 0x55bbab802e40 2024-05-27T07:55:43.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.087+0000 7f22caa00700 1 -- 172.21.15.90:0/1885158542 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d00052a0 con 0x55bbab802e40 2024-05-27T07:55:43.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.088+0000 7f22ee848b80 1 -- 172.21.15.90:0/1885158542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 msgr2=0x55bbab9877f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:43.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.088+0000 7f22ee848b80 1 --2- 172.21.15.90:0/1885158542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab9877f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f22d0000f80 tx=0x7f22d00092e0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.088+0000 7f22ee848b80 1 -- 172.21.15.90:0/1885158542 shutdown_connections 2024-05-27T07:55:43.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.088+0000 7f22ee848b80 1 --2- 172.21.15.90:0/1885158542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab9821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.088+0000 7f22ee848b80 1 --2- 172.21.15.90:0/1885158542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab9877f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.088+0000 7f22ee848b80 1 --2- 172.21.15.90:0/1885158542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab981870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.088+0000 7f22ee848b80 1 -- 172.21.15.90:0/1885158542 >> 172.21.15.90:0/1885158542 conn(0x55bbab862fc0 msgr2=0x55bbab97c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:43.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.089+0000 7f22ee848b80 1 -- 172.21.15.90:0/1885158542 shutdown_connections 2024-05-27T07:55:43.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.089+0000 7f22ee848b80 1 -- 172.21.15.90:0/1885158542 wait complete. 2024-05-27T07:55:43.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.090+0000 7f22ee848b80 1 -- start start 2024-05-27T07:55:43.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.090+0000 7f22ee848b80 1 Processor -- start 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.091+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab98b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.091+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab981db0 0x55bbab98baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.091+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab987d30 0x55bbab98bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.091+0000 7f22ee848b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bbab7a5fa0 con 0x55bbab987d30 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.091+0000 7f22ee848b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bbab840760 con 0x55bbab981db0 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.092+0000 7f22ee848b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bbab98e7a0 con 0x55bbab802e40 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.092+0000 7f22ee848b80 10 client.0 _ll_register_callbacks cb 0x55bbab849c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.092+0000 7f22dde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab987d30 0x55bbab98bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.092+0000 7f22dde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab987d30 0x55bbab98bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34952/0 (socket says 172.21.15.90:34952) 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.092+0000 7f22dde00700 1 -- 172.21.15.90:0/2539505178 learned_addr learned my addr 172.21.15.90:0/2539505178 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:43.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.093+0000 7f22dca00700 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab981db0 0x55bbab98baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.093+0000 7f22dd400700 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab98b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.093+0000 7f22dca00700 1 -- 172.21.15.90:0/2539505178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 msgr2=0x55bbab98b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:43.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.093+0000 7f22dca00700 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab98b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.093+0000 7f22dca00700 1 -- 172.21.15.90:0/2539505178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab987d30 msgr2=0x55bbab98bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:43.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.093+0000 7f22dca00700 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab987d30 0x55bbab98bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.093+0000 7f22dca00700 1 -- 172.21.15.90:0/2539505178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbab7ef8d0 con 0x55bbab981db0 2024-05-27T07:55:43.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.094+0000 7f22dca00700 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab981db0 0x55bbab98baa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0df130 tx=0x7f22cc010780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c11e38ea4ba361 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:43.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.094+0000 7f22bf400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:55:43.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.095+0000 7f22bf400700 1 -- 172.21.15.90:0/2539505178 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc0c0070 con 0x55bbab981db0 2024-05-27T07:55:43.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.095+0000 7f22bf400700 1 -- 172.21.15.90:0/2539505178 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f22cc0172c0 con 0x55bbab981db0 2024-05-27T07:55:43.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.096+0000 7f22ee848b80 1 -- 172.21.15.90:0/2539505178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22d0003c00 con 0x55bbab981db0 2024-05-27T07:55:43.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.096+0000 7f22bf400700 1 -- 172.21.15.90:0/2539505178 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f22cc0174c0 con 0x55bbab981db0 2024-05-27T07:55:43.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.096+0000 7f22ee848b80 10 client.36773 fetch_fsmap learned FSMap version 57 2024-05-27T07:55:43.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.096+0000 7f22ee848b80 1 -- 172.21.15.90:0/2539505178 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55bbab870060 con 0x55bbab981db0 2024-05-27T07:55:43.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.097+0000 7f22bf400700 1 -- 172.21.15.90:0/2539505178 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22cc0a93c0 con 0x55bbab981db0 2024-05-27T07:55:43.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.097+0000 7f22bf400700 1 -- 172.21.15.90:0/2539505178 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 57) ==== 7152+0+0 (secure 0 0 0) 0x7f22cc0c3020 con 0x55bbab981db0 2024-05-27T07:55:43.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.097+0000 7f22ee848b80 10 client.36773 fetch_fsmap finished waiting for FSMap version 57 2024-05-27T07:55:43.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.097+0000 7f22ee848b80 10 client.36773 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:55:43.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.097+0000 7f22ee848b80 20 client.36773 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:43.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.097+0000 7f22ee848b80 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55bbab9926c0 0x55bbab994ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.097+0000 7f22ee848b80 4 client.36773 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:43.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.097+0000 7f22ee848b80 1 -- 172.21.15.90:0/2539505178 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f22d0003c00 con 0x55bbab9926c0 2024-05-27T07:55:43.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.098+0000 7f22dd400700 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55bbab9926c0 0x55bbab994ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.099+0000 7f22dd400700 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55bbab9926c0 0x55bbab994ae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=55a51cb4bf6c49c5 server_cookie=e1134c0e6f82df0d in_seq=0 out_seq=0 2024-05-27T07:55:43.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.099+0000 7f22bf400700 10 client.36773 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:55:43.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22bf400700 1 -- 172.21.15.90:0/2539505178 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f22d0003c00 con 0x55bbab9926c0 2024-05-27T07:55:43.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22bf400700 10 client.36773 handle_command_reply: tid=0 2024-05-27T07:55:43.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22ee848b80 1 client.36773 shutdown 2024-05-27T07:55:43.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22ee848b80 1 -- 172.21.15.90:0/2539505178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab981db0 msgr2=0x55bbab98baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:43.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22ee848b80 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab981db0 0x55bbab98baa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0df130 tx=0x7f22cc010780 comp rx=0 tx=0).stop 2024-05-27T07:55:43.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22ee848b80 1 -- 172.21.15.90:0/2539505178 shutdown_connections 2024-05-27T07:55:43.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22ee848b80 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55bbab9926c0 0x55bbab994ae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22ee848b80 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab987d30 0x55bbab98bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22ee848b80 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab98b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22ee848b80 1 --2- 172.21.15.90:0/2539505178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab981db0 0x55bbab98baa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.101+0000 7f22ee848b80 1 -- 172.21.15.90:0/2539505178 >> 172.21.15.90:0/2539505178 conn(0x55bbab862fc0 msgr2=0x55bbab97c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.102+0000 7f22bf400700 0 client.36773 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:55:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.102+0000 7f22ee848b80 1 -- 172.21.15.90:0/2539505178 shutdown_connections 2024-05-27T07:55:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.102+0000 7f22ee848b80 1 -- 172.21.15.90:0/2539505178 wait complete. 2024-05-27T07:55:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.102+0000 7f22ee848b80 20 client.36773 trim_cache size 0 max 16384 2024-05-27T07:55:43.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.133+0000 7f22ee848b80 1 Processor -- start 2024-05-27T07:55:43.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.133+0000 7f22ee848b80 1 -- start start 2024-05-27T07:55:43.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.133+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab981870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.133+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab98eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.133+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab98b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.133+0000 7f22ee848b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f22d81b56a0 con 0x55bbab981db0 2024-05-27T07:55:43.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.133+0000 7f22ee848b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f22d81b7f80 con 0x55bbab987d30 2024-05-27T07:55:43.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.133+0000 7f22ee848b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f22d81b5d70 con 0x55bbab802e40 2024-05-27T07:55:43.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.134+0000 7f22dde00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab98b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.134+0000 7f22dde00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab98b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:36486/0 (socket says 172.21.15.90:36486) 2024-05-27T07:55:43.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.134+0000 7f22dde00700 1 -- 172.21.15.90:0/368038627 learned_addr learned my addr 172.21.15.90:0/368038627 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:43.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.135+0000 7f22dd400700 1 --2- 172.21.15.90:0/368038627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab981870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.135+0000 7f22dd400700 1 -- 172.21.15.90:0/368038627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 msgr2=0x55bbab98b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:43.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.136+0000 7f22dca00700 1 --2- 172.21.15.90:0/368038627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab98eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.136+0000 7f22dd400700 1 --2- 172.21.15.90:0/368038627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab98b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.136+0000 7f22dd400700 1 -- 172.21.15.90:0/368038627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 msgr2=0x55bbab98eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:43.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.136+0000 7f22dd400700 1 --2- 172.21.15.90:0/368038627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab98eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.136+0000 7f22dd400700 1 -- 172.21.15.90:0/368038627 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbab989ad0 con 0x55bbab802e40 2024-05-27T07:55:43.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.136+0000 7f22dde00700 1 --2- 172.21.15.90:0/368038627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab98b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:43.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.137+0000 7f22dd400700 1 --2- 172.21.15.90:0/368038627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab981870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f22d000b070 tx=0x7f22d000b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=467cf95e53bc7e12 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:43.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.137+0000 7f22ca000700 1 -- 172.21.15.90:0/368038627 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0020040 con 0x55bbab802e40 2024-05-27T07:55:43.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.138+0000 7f22ca000700 1 -- 172.21.15.90:0/368038627 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22d0007ad0 con 0x55bbab802e40 2024-05-27T07:55:43.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.138+0000 7f22ca000700 1 -- 172.21.15.90:0/368038627 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d002d980 con 0x55bbab802e40 2024-05-27T07:55:43.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.139+0000 7f22ee848b80 1 -- 172.21.15.90:0/368038627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 msgr2=0x55bbab981870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:43.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.139+0000 7f22ee848b80 1 --2- 172.21.15.90:0/368038627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab981870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f22d000b070 tx=0x7f22d000b3b0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.140+0000 7f22ee848b80 1 -- 172.21.15.90:0/368038627 shutdown_connections 2024-05-27T07:55:43.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.140+0000 7f22ee848b80 1 --2- 172.21.15.90:0/368038627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab98b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.140+0000 7f22ee848b80 1 --2- 172.21.15.90:0/368038627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab981870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.140+0000 7f22ee848b80 1 --2- 172.21.15.90:0/368038627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab98eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.140+0000 7f22ee848b80 1 -- 172.21.15.90:0/368038627 >> 172.21.15.90:0/368038627 conn(0x55bbab862fc0 msgr2=0x55bbab989e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:43.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.142+0000 7f22ee848b80 1 -- 172.21.15.90:0/368038627 shutdown_connections 2024-05-27T07:55:43.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.143+0000 7f22ee848b80 1 -- 172.21.15.90:0/368038627 wait complete. 2024-05-27T07:55:43.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.143+0000 7f22ee848b80 1 -- start start 2024-05-27T07:55:43.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.144+0000 7f22ee848b80 1 Processor -- start 2024-05-27T07:55:43.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.149+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab99a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.149+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab99aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.150+0000 7f22ee848b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab9b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.150+0000 7f22dd400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab99a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.150+0000 7f22dd400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab99a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57686/0 (socket says 172.21.15.90:57686) 2024-05-27T07:55:43.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.150+0000 7f22dca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab99aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.150+0000 7f22dd400700 1 -- 172.21.15.90:0/1858071958 learned_addr learned my addr 172.21.15.90:0/1858071958 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:43.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.150+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f22d81b6440 con 0x55bbab981db0 2024-05-27T07:55:43.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.151+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f22d81b78b0 con 0x55bbab987d30 2024-05-27T07:55:43.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.151+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f22d81b6b10 con 0x55bbab802e40 2024-05-27T07:55:43.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.151+0000 7f22ee848b80 10 client.0 _ll_register_callbacks cb 0x55bbab849c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:43.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.151+0000 7f22dde00700 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab9b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.152+0000 7f22dde00700 1 -- 172.21.15.90:0/1858071958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 msgr2=0x55bbab99a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:43.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.152+0000 7f22dde00700 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab99a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.152+0000 7f22dde00700 1 -- 172.21.15.90:0/1858071958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 msgr2=0x55bbab99aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:43.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.152+0000 7f22dde00700 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab99aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.152+0000 7f22dde00700 1 -- 172.21.15.90:0/1858071958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbab9b5af0 con 0x55bbab987d30 2024-05-27T07:55:43.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.155+0000 7f22dde00700 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab9b54b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55bbab98f120 tx=0x7f22c4011750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=673fea2e0d50ed73 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:43.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.155+0000 7f22bf400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:55:43.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.155+0000 7f22bf400700 1 -- 172.21.15.90:0/1858071958 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4025030 con 0x55bbab987d30 2024-05-27T07:55:43.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.155+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22d0007ad0 con 0x55bbab987d30 2024-05-27T07:55:43.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.155+0000 7f22bf400700 1 -- 172.21.15.90:0/1858071958 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f22c4021460 con 0x55bbab987d30 2024-05-27T07:55:43.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.156+0000 7f22bf400700 1 -- 172.21.15.90:0/1858071958 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f22c401a040 con 0x55bbab987d30 2024-05-27T07:55:43.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.156+0000 7f22ee848b80 10 client.36779 fetch_fsmap learned FSMap version 57 2024-05-27T07:55:43.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.156+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55bbaba16b10 con 0x55bbab987d30 2024-05-27T07:55:43.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.156+0000 7f22bf400700 1 -- 172.21.15.90:0/1858071958 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22d0007ad0 con 0x55bbab987d30 2024-05-27T07:55:43.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.157+0000 7f22bf400700 1 -- 172.21.15.90:0/1858071958 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 57) ==== 7152+0+0 (secure 0 0 0) 0x7f22c400f040 con 0x55bbab987d30 2024-05-27T07:55:43.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.157+0000 7f22ee848b80 10 client.36779 fetch_fsmap finished waiting for FSMap version 57 2024-05-27T07:55:43.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.157+0000 7f22ee848b80 10 client.36779 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:55:43.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.157+0000 7f22ee848b80 20 client.36779 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:43.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.157+0000 7f22ee848b80 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55bbaba17230 0x55bbaba19650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:43.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.157+0000 7f22ee848b80 4 client.36779 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:55:43.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.157+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbab8b1d90 con 0x55bbaba17230 2024-05-27T07:55:43.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.158+0000 7f22dd400700 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55bbaba17230 0x55bbaba19650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:43.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.159+0000 7f22bf400700 10 client.36779 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:55:43.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.158+0000 7f22dd400700 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55bbaba17230 0x55bbaba19650 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=25adafe2c53668eb server_cookie=dc7a8916ef95d84b in_seq=0 out_seq=0 2024-05-27T07:55:43.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.159+0000 7f22bf400700 1 -- 172.21.15.90:0/1858071958 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbab8b1d90 con 0x55bbaba17230 2024-05-27T07:55:43.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.159+0000 7f22bf400700 10 client.36779 handle_command_reply: tid=0 2024-05-27T07:55:43.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.159+0000 7f22ee848b80 1 client.36779 shutdown 2024-05-27T07:55:43.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 msgr2=0x55bbab9b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:43.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22ee848b80 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab9b54b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55bbab98f120 tx=0x7f22c4011750 comp rx=0 tx=0).stop 2024-05-27T07:55:43.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 shutdown_connections 2024-05-27T07:55:43.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22ee848b80 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55bbaba17230 0x55bbaba19650 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22ee848b80 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbab987d30 0x55bbab9b54b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22ee848b80 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bbab802e40 0x55bbab99a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22bf400700 0 client.36779 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:55:43.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22ee848b80 1 --2- 172.21.15.90:0/1858071958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bbab981db0 0x55bbab99aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 >> 172.21.15.90:0/1858071958 conn(0x55bbab862fc0 msgr2=0x55bbab98e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:43.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 shutdown_connections 2024-05-27T07:55:43.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.160+0000 7f22ee848b80 1 -- 172.21.15.90:0/1858071958 wait complete. 2024-05-27T07:55:43.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.161+0000 7f22ee848b80 20 client.36779 trim_cache size 0 max 16384 2024-05-27T07:55:43.163 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:55:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.162+0000 7f22dfe00700 1 -- 172.21.15.90:0/2867913137 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f22c00b2cf0 msgr2=0x7f22c00b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.162+0000 7f22dfe00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f22c00b2cf0 0x7f22c00b5200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22d80759b0 tx=0x7f22c400a3b0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.162+0000 7f22dfe00700 1 -- 172.21.15.90:0/2867913137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22d8105e00 msgr2=0x7f22d806e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.162+0000 7f22dfe00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22d8105e00 0x7f22d806e520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b530 tx=0x7f22cc00b840 comp rx=0 tx=0).stop 2024-05-27T07:55:43.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.163+0000 7f22dfe00700 1 -- 172.21.15.90:0/2867913137 shutdown_connections 2024-05-27T07:55:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.163+0000 7f22dfe00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22d810e8e0 0x7f22d806ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.163+0000 7f22dfe00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22d81054b0 0x7f22d80753f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.163+0000 7f22dfe00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f22c00b2cf0 0x7f22c00b5200 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.163+0000 7f22dfe00700 1 --2- 172.21.15.90:0/2867913137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22d8105e00 0x7f22d806e520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.163+0000 7f22dfe00700 1 -- 172.21.15.90:0/2867913137 >> 172.21.15.90:0/2867913137 conn(0x7f22d80fedb0 msgr2=0x7f22d80ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.163+0000 7f22dfe00700 1 -- 172.21.15.90:0/2867913137 shutdown_connections 2024-05-27T07:55:43.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:43.164+0000 7f22dfe00700 1 -- 172.21.15.90:0/2867913137 wait complete. 2024-05-27T07:55:43.698 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:43 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-05-27T07:55:43.698 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:43 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.568s CPU time. 2024-05-27T07:55:43.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 325.0 s, 499.6 tps, lat 201.259 ms stddev 128.737 2024-05-27T07:55:44.103 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:55:44.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:44 smithi090 ceph-mon[31429]: pgmap v295: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.9 MiB/s wr, 509 op/s 2024-05-27T07:55:44.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:44 smithi129 ceph-mon[35254]: pgmap v295: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.9 MiB/s wr, 509 op/s 2024-05-27T07:55:44.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:44 smithi136 ceph-mon[35324]: pgmap v295: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 4.9 MiB/s wr, 509 op/s 2024-05-27T07:55:45.022 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:46.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:46 smithi129 ceph-mon[35254]: pgmap v296: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.1 MiB/s wr, 617 op/s 2024-05-27T07:55:46.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:46 smithi136 ceph-mon[35324]: pgmap v296: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.1 MiB/s wr, 617 op/s 2024-05-27T07:55:47.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:46 smithi090 ceph-mon[31429]: pgmap v296: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.1 MiB/s wr, 617 op/s 2024-05-27T07:55:47.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:47 smithi129 ceph-mon[35254]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:47.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:47 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-05-27T07:55:47.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:47 smithi090 ceph-mon[31429]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:47.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:47 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-05-27T07:55:47.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:47 smithi136 ceph-mon[35324]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:55:47.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:47 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-05-27T07:55:48.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:48 smithi129 ceph-mon[35254]: pgmap v297: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-05-27T07:55:48.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:48 smithi090 ceph-mon[31429]: pgmap v297: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-05-27T07:55:48.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:48 smithi136 ceph-mon[35324]: pgmap v297: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.1 MiB/s wr, 420 op/s 2024-05-27T07:55:48.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 330.0 s, 495.4 tps, lat 202.289 ms stddev 246.896 2024-05-27T07:55:50.554 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:50 smithi136 ceph-mon[35324]: pgmap v298: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.2 MiB/s wr, 642 op/s 2024-05-27T07:55:50.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:50 smithi129 ceph-mon[35254]: pgmap v298: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.2 MiB/s wr, 642 op/s 2024-05-27T07:55:50.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:50 smithi090 ceph-mon[31429]: pgmap v298: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.2 MiB/s wr, 642 op/s 2024-05-27T07:55:50.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.973+0000 7f4431000640 1 -- 172.21.15.90:0/2831908513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c106600 msgr2=0x7f442c1129d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.973+0000 7f4431000640 1 --2- 172.21.15.90:0/2831908513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c106600 0x7f442c1129d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4424009ca0 tx=0x7f442402a8b0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.974+0000 7f4431000640 1 -- 172.21.15.90:0/2831908513 shutdown_connections 2024-05-27T07:55:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.974+0000 7f4431000640 1 --2- 172.21.15.90:0/2831908513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c106600 0x7f442c1129d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.974+0000 7f4431000640 1 --2- 172.21.15.90:0/2831908513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f442c10c350 0x7f442c105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.974+0000 7f4431000640 1 --2- 172.21.15.90:0/2831908513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442c10beb0 0x7f442c105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.974+0000 7f4431000640 1 -- 172.21.15.90:0/2831908513 >> 172.21.15.90:0/2831908513 conn(0x7f442c1016b0 msgr2=0x7f442c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.974+0000 7f4431000640 1 -- 172.21.15.90:0/2831908513 shutdown_connections 2024-05-27T07:55:50.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.974+0000 7f4431000640 1 -- 172.21.15.90:0/2831908513 wait complete. 2024-05-27T07:55:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.977+0000 7f4431000640 1 Processor -- start 2024-05-27T07:55:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.977+0000 7f4431000640 1 -- start start 2024-05-27T07:55:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.978+0000 7f4431000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f442c106600 0x7f442c1a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.978+0000 7f4431000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c10beb0 0x7f442c1a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.979+0000 7f4431000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442c10c350 0x7f442c1acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.979+0000 7f4431000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f442c079bf0 con 0x7f442c106600 2024-05-27T07:55:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.979+0000 7f4431000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f442c079a70 con 0x7f442c10c350 2024-05-27T07:55:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.979+0000 7f4431000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f442c079d70 con 0x7f442c10beb0 2024-05-27T07:55:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.979+0000 7f4429400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c10beb0 0x7f442c1a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.979+0000 7f4429400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c10beb0 0x7f442c1a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38496/0 (socket says 172.21.15.90:38496) 2024-05-27T07:55:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.979+0000 7f4429400640 1 -- 172.21.15.90:0/2368642659 learned_addr learned my addr 172.21.15.90:0/2368642659 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.980+0000 7f4429e00640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f442c106600 0x7f442c1a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.980+0000 7f442a800640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442c10c350 0x7f442c1acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.981+0000 7f442a800640 1 -- 172.21.15.90:0/2368642659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c10beb0 msgr2=0x7f442c1a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.981+0000 7f442a800640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c10beb0 0x7f442c1a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.981+0000 7f442a800640 1 -- 172.21.15.90:0/2368642659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f442c106600 msgr2=0x7f442c1a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.981+0000 7f442a800640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f442c106600 0x7f442c1a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.981+0000 7f442a800640 1 -- 172.21.15.90:0/2368642659 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c1ad050 con 0x7f442c10c350 2024-05-27T07:55:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.981+0000 7f4429400640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c10beb0 0x7f442c1a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.981+0000 7f442a800640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442c10c350 0x7f442c1acb10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f442402add0 tx=0x7f4424030d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.982+0000 7f4422a00640 1 -- 172.21.15.90:0/2368642659 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424004040 con 0x7f442c10c350 2024-05-27T07:55:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.982+0000 7f4431000640 1 -- 172.21.15.90:0/2368642659 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c1ad220 con 0x7f442c10c350 2024-05-27T07:55:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.982+0000 7f4422a00640 1 -- 172.21.15.90:0/2368642659 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4424040070 con 0x7f442c10c350 2024-05-27T07:55:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.982+0000 7f4422a00640 1 -- 172.21.15.90:0/2368642659 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424045af0 con 0x7f442c10c350 2024-05-27T07:55:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.982+0000 7f4431000640 1 -- 172.21.15.90:0/2368642659 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c1ad6e0 con 0x7f442c10c350 2024-05-27T07:55:50.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.989+0000 7f4422a00640 1 -- 172.21.15.90:0/2368642659 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4424045c90 con 0x7f442c10c350 2024-05-27T07:55:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.991+0000 7f4422a00640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44100b2950 0x7f44100b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.991+0000 7f4422a00640 1 -- 172.21.15.90:0/2368642659 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f4424103ac0 con 0x7f442c10c350 2024-05-27T07:55:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.993+0000 7f4429e00640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44100b2950 0x7f44100b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.995+0000 7f4429e00640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44100b2950 0x7f44100b4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f442c106b90 tx=0x7f441c00b390 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:50.999+0000 7f4438184740 1 Processor -- start 2024-05-27T07:55:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.000+0000 7f4438184740 1 -- start start 2024-05-27T07:55:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.000+0000 7f4438184740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701248860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.000+0000 7f4438184740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701248da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.000+0000 7f4438184740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 0x5627012492e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.000+0000 7f4438184740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5627011b9b90 con 0x562700e7d2f0 2024-05-27T07:55:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.000+0000 7f4438184740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5627011a9730 con 0x562700e854d0 2024-05-27T07:55:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.000+0000 7f4438184740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5627011ad770 con 0x562700ea61b0 2024-05-27T07:55:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.001+0000 7f4429400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701248da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.001+0000 7f4429e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701248860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.001+0000 7f4429e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701248860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38508/0 (socket says 172.21.15.90:38508) 2024-05-27T07:55:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.001+0000 7f4429e00640 1 -- 172.21.15.90:0/2161461142 learned_addr learned my addr 172.21.15.90:0/2161461142 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.001+0000 7f442a800640 1 --2- 172.21.15.90:0/2161461142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 0x5627012492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.002+0000 7f4429e00640 1 -- 172.21.15.90:0/2161461142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 msgr2=0x5627012492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.002+0000 7f4429e00640 1 --2- 172.21.15.90:0/2161461142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 0x5627012492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.002+0000 7f4429e00640 1 -- 172.21.15.90:0/2161461142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 msgr2=0x562701248da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.002+0000 7f4429e00640 1 --2- 172.21.15.90:0/2161461142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701248da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.002+0000 7f4429e00640 1 -- 172.21.15.90:0/2161461142 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627011428b0 con 0x562700ea61b0 2024-05-27T07:55:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.002+0000 7f4429400640 1 --2- 172.21.15.90:0/2161461142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701248da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.002+0000 7f4429e00640 1 --2- 172.21.15.90:0/2161461142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701248860 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f441c00b6e0 tx=0x7f441c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd2138372b9c70e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.003+0000 7f4417400640 1 -- 172.21.15.90:0/2161461142 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c005220 con 0x562700ea61b0 2024-05-27T07:55:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.003+0000 7f4417400640 1 -- 172.21.15.90:0/2161461142 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f441c004c30 con 0x562700ea61b0 2024-05-27T07:55:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.003+0000 7f4417400640 1 -- 172.21.15.90:0/2161461142 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c01e920 con 0x562700ea61b0 2024-05-27T07:55:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.004+0000 7f4438184740 1 -- 172.21.15.90:0/2161461142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 msgr2=0x562701248860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.004+0000 7f4438184740 1 --2- 172.21.15.90:0/2161461142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701248860 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f441c00b6e0 tx=0x7f441c013000 comp rx=0 tx=0).stop 2024-05-27T07:55:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.004+0000 7f4438184740 1 -- 172.21.15.90:0/2161461142 shutdown_connections 2024-05-27T07:55:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.004+0000 7f4438184740 1 --2- 172.21.15.90:0/2161461142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 0x5627012492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.004+0000 7f4438184740 1 --2- 172.21.15.90:0/2161461142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701248da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.004+0000 7f4438184740 1 --2- 172.21.15.90:0/2161461142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701248860 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.004+0000 7f4438184740 1 -- 172.21.15.90:0/2161461142 >> 172.21.15.90:0/2161461142 conn(0x562700e9a300 msgr2=0x562701245010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.005+0000 7f4438184740 1 -- 172.21.15.90:0/2161461142 shutdown_connections 2024-05-27T07:55:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.005+0000 7f4438184740 1 -- 172.21.15.90:0/2161461142 wait complete. 2024-05-27T07:55:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.006+0000 7f4438184740 1 -- start start 2024-05-27T07:55:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.006+0000 7f4438184740 1 Processor -- start 2024-05-27T07:55:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.007+0000 7f4438184740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701252a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.007+0000 7f4438184740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e854d0 0x562701252f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.007+0000 7f4438184740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x56270125ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.007+0000 7f4438184740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562700e8f790 con 0x562700e7d2f0 2024-05-27T07:55:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.007+0000 7f4438184740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562701151bc0 con 0x562700ea61b0 2024-05-27T07:55:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.007+0000 7f4438184740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562701255990 con 0x562700e854d0 2024-05-27T07:55:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.007+0000 7f4438184740 10 client.0 _ll_register_callbacks cb 0x56270112f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.007+0000 7f4429400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e854d0 0x562701252f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.007+0000 7f4429400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e854d0 0x562701252f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38516/0 (socket says 172.21.15.90:38516) 2024-05-27T07:55:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.007+0000 7f4429400640 1 -- 172.21.15.90:0/695177429 learned_addr learned my addr 172.21.15.90:0/695177429 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.008+0000 7f442a800640 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x56270125ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.008+0000 7f4429400640 1 -- 172.21.15.90:0/695177429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 msgr2=0x56270125ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.008+0000 7f4429400640 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x56270125ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.008+0000 7f4429400640 1 -- 172.21.15.90:0/695177429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 msgr2=0x562701252a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.008+0000 7f4429400640 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701252a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.008+0000 7f4429400640 1 -- 172.21.15.90:0/695177429 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562700ebe210 con 0x562700e854d0 2024-05-27T07:55:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.008+0000 7f442a800640 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x56270125ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.008+0000 7f4429400640 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e854d0 0x562701252f90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f441800bf10 tx=0x7f4418012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e43b85286e738b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.008+0000 7f440be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:55:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.009+0000 7f440be00640 1 -- 172.21.15.90:0/695177429 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418005220 con 0x562700e854d0 2024-05-27T07:55:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.009+0000 7f4438184740 1 -- 172.21.15.90:0/695177429 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f441c004c30 con 0x562700e854d0 2024-05-27T07:55:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.009+0000 7f440be00640 1 -- 172.21.15.90:0/695177429 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f44180053c0 con 0x562700e854d0 2024-05-27T07:55:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.010+0000 7f440be00640 1 -- 172.21.15.90:0/695177429 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4418025050 con 0x562700e854d0 2024-05-27T07:55:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.010+0000 7f4438184740 10 client.26842 fetch_fsmap learned FSMap version 58 2024-05-27T07:55:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.010+0000 7f4438184740 1 -- 172.21.15.90:0/695177429 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562700f8a620 con 0x562700e854d0 2024-05-27T07:55:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.011+0000 7f440be00640 1 -- 172.21.15.90:0/695177429 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f441c004c30 con 0x562700e854d0 2024-05-27T07:55:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.012+0000 7f440be00640 1 -- 172.21.15.90:0/695177429 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 58) ==== 6559+0+0 (secure 0 0 0) 0x7f4418010a20 con 0x562700e854d0 2024-05-27T07:55:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.012+0000 7f4438184740 10 client.26842 fetch_fsmap finished waiting for FSMap version 58 2024-05-27T07:55:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.012+0000 7f4438184740 10 client.26842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.012+0000 7f4438184740 20 client.26842 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.013+0000 7f4438184740 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562700e64d80 0x562701265040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.013+0000 7f4438184740 4 client.26842 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.013+0000 7f4438184740 1 -- 172.21.15.90:0/695177429 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627011bbae0 con 0x562700e64d80 2024-05-27T07:55:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.014+0000 7f4429e00640 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562700e64d80 0x562701265040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.016+0000 7f4429e00640 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562700e64d80 0x562701265040 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be28d21bdb53e057 server_cookie=e882b39317afe220 in_seq=0 out_seq=0 2024-05-27T07:55:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.016+0000 7f440be00640 10 client.26842 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.018+0000 7f440be00640 1 -- 172.21.15.90:0/695177429 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5627011bbae0 con 0x562700e64d80 2024-05-27T07:55:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.018+0000 7f440be00640 10 client.26842 handle_command_reply: tid=0 2024-05-27T07:55:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.018+0000 7f4438184740 1 client.26842 shutdown 2024-05-27T07:55:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.019+0000 7f4438184740 1 -- 172.21.15.90:0/695177429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e854d0 msgr2=0x562701252f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.019+0000 7f4438184740 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e854d0 0x562701252f90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f441800bf10 tx=0x7f4418012820 comp rx=0 tx=0).stop 2024-05-27T07:55:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.021+0000 7f4438184740 1 -- 172.21.15.90:0/695177429 shutdown_connections 2024-05-27T07:55:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.021+0000 7f4438184740 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562700e64d80 0x562701265040 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.021+0000 7f4438184740 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x56270125ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.021+0000 7f4438184740 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e854d0 0x562701252f90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.021+0000 7f4438184740 1 --2- 172.21.15.90:0/695177429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701252a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.021+0000 7f440be00640 0 client.26842 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.021+0000 7f4438184740 1 -- 172.21.15.90:0/695177429 >> 172.21.15.90:0/695177429 conn(0x562700e9a300 msgr2=0x562701248630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.022+0000 7f4438184740 1 -- 172.21.15.90:0/695177429 shutdown_connections 2024-05-27T07:55:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.022+0000 7f4438184740 1 -- 172.21.15.90:0/695177429 wait complete. 2024-05-27T07:55:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.022+0000 7f4438184740 20 client.26842 trim_cache size 0 max 16384 2024-05-27T07:55:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.057+0000 7f4438184740 1 Processor -- start 2024-05-27T07:55:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.059+0000 7f4438184740 1 -- start start 2024-05-27T07:55:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.060+0000 7f4438184740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701255180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.060+0000 7f4438184740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 0x5627012529f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.060+0000 7f4438184740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701252f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.060+0000 7f4438184740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b92c0 con 0x562700e7d2f0 2024-05-27T07:55:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.060+0000 7f4438184740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b6310 con 0x562700e854d0 2024-05-27T07:55:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.060+0000 7f4438184740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b8520 con 0x562700ea61b0 2024-05-27T07:55:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.061+0000 7f4429400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 0x5627012529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.061+0000 7f4429400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 0x5627012529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:54388/0 (socket says 172.21.15.90:54388) 2024-05-27T07:55:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.061+0000 7f4429400640 1 -- 172.21.15.90:0/3005168533 learned_addr learned my addr 172.21.15.90:0/3005168533 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.061+0000 7f442a800640 1 --2- 172.21.15.90:0/3005168533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701252f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.061+0000 7f4429e00640 1 --2- 172.21.15.90:0/3005168533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701255180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.062+0000 7f442a800640 1 -- 172.21.15.90:0/3005168533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 msgr2=0x5627012529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.062+0000 7f442a800640 1 --2- 172.21.15.90:0/3005168533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 0x5627012529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.062+0000 7f442a800640 1 -- 172.21.15.90:0/3005168533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 msgr2=0x562701255180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.062+0000 7f442a800640 1 --2- 172.21.15.90:0/3005168533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701255180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.062+0000 7f442a800640 1 -- 172.21.15.90:0/3005168533 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562701249520 con 0x562700ea61b0 2024-05-27T07:55:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.063+0000 7f4429e00640 1 --2- 172.21.15.90:0/3005168533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701255180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.063+0000 7f442a800640 1 --2- 172.21.15.90:0/3005168533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701252f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4424100f50 tx=0x7f44240b2040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e646eee2dcd83dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.063+0000 7f4416a00640 1 -- 172.21.15.90:0/3005168533 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240ca5f0 con 0x562700ea61b0 2024-05-27T07:55:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.064+0000 7f4416a00640 1 -- 172.21.15.90:0/3005168533 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4424036040 con 0x562700ea61b0 2024-05-27T07:55:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.064+0000 7f4416a00640 1 -- 172.21.15.90:0/3005168533 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240cad40 con 0x562700ea61b0 2024-05-27T07:55:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.065+0000 7f4438184740 1 -- 172.21.15.90:0/3005168533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 msgr2=0x562701252f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.065+0000 7f4438184740 1 --2- 172.21.15.90:0/3005168533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701252f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4424100f50 tx=0x7f44240b2040 comp rx=0 tx=0).stop 2024-05-27T07:55:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.068+0000 7f4438184740 1 -- 172.21.15.90:0/3005168533 shutdown_connections 2024-05-27T07:55:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.068+0000 7f4438184740 1 --2- 172.21.15.90:0/3005168533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700e854d0 0x5627012529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.068+0000 7f4438184740 1 --2- 172.21.15.90:0/3005168533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700ea61b0 0x562701252f30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.068+0000 7f4438184740 1 --2- 172.21.15.90:0/3005168533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e7d2f0 0x562701255180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.068+0000 7f4438184740 1 -- 172.21.15.90:0/3005168533 >> 172.21.15.90:0/3005168533 conn(0x562700e9a300 msgr2=0x5627012522e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.069+0000 7f4438184740 1 -- 172.21.15.90:0/3005168533 shutdown_connections 2024-05-27T07:55:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.069+0000 7f4438184740 1 -- 172.21.15.90:0/3005168533 wait complete. 2024-05-27T07:55:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.071+0000 7f4438184740 1 -- start start 2024-05-27T07:55:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.072+0000 7f4438184740 1 Processor -- start 2024-05-27T07:55:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.073+0000 7f4438184740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e7d2f0 0x5627012579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.073+0000 7f4438184740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e854d0 0x562701257f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.073+0000 7f4438184740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x562701258470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.074+0000 7f4438184740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b7e50 con 0x562700e854d0 2024-05-27T07:55:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.074+0000 7f4438184740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b69e0 con 0x562700ea61b0 2024-05-27T07:55:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.074+0000 7f4438184740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1b70b0 con 0x562700e7d2f0 2024-05-27T07:55:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.074+0000 7f4438184740 10 client.0 _ll_register_callbacks cb 0x7f441800b870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.074+0000 7f442a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x562701258470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.074+0000 7f442a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x562701258470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:54392/0 (socket says 172.21.15.90:54392) 2024-05-27T07:55:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.074+0000 7f442a800640 1 -- 172.21.15.90:0/2922777595 learned_addr learned my addr 172.21.15.90:0/2922777595 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.075+0000 7f4429400640 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e854d0 0x562701257f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.075+0000 7f442a800640 1 -- 172.21.15.90:0/2922777595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e7d2f0 msgr2=0x5627012579f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:55:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.075+0000 7f442a800640 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e7d2f0 0x5627012579f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.075+0000 7f442a800640 1 -- 172.21.15.90:0/2922777595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e854d0 msgr2=0x562701257f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.075+0000 7f442a800640 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e854d0 0x562701257f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.075+0000 7f442a800640 1 -- 172.21.15.90:0/2922777595 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627012cb710 con 0x562700ea61b0 2024-05-27T07:55:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.075+0000 7f4429400640 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e854d0 0x562701257f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:55:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.076+0000 7f440be00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:55:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.076+0000 7f442a800640 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x562701258470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f44240caa70 tx=0x7f44240caf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d7f3453dea3f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.077+0000 7f440be00640 1 -- 172.21.15.90:0/2922777595 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442408e070 con 0x562700ea61b0 2024-05-27T07:55:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.077+0000 7f440be00640 1 -- 172.21.15.90:0/2922777595 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f44240d36c0 con 0x562700ea61b0 2024-05-27T07:55:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.077+0000 7f4438184740 1 -- 172.21.15.90:0/2922777595 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4424036040 con 0x562700ea61b0 2024-05-27T07:55:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.080+0000 7f440be00640 1 -- 172.21.15.90:0/2922777595 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f44240ca5f0 con 0x562700ea61b0 2024-05-27T07:55:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.080+0000 7f4438184740 10 client.36797 fetch_fsmap learned FSMap version 58 2024-05-27T07:55:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.080+0000 7f4438184740 1 -- 172.21.15.90:0/2922777595 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5627012cc960 con 0x562700ea61b0 2024-05-27T07:55:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.081+0000 7f440be00640 1 -- 172.21.15.90:0/2922777595 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4424036040 con 0x562700ea61b0 2024-05-27T07:55:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.081+0000 7f440be00640 1 -- 172.21.15.90:0/2922777595 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 58) ==== 6559+0+0 (secure 0 0 0) 0x7f442404d030 con 0x562700ea61b0 2024-05-27T07:55:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.082+0000 7f4438184740 10 client.36797 fetch_fsmap finished waiting for FSMap version 58 2024-05-27T07:55:51.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.082+0000 7f4438184740 10 client.36797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:55:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.082+0000 7f4438184740 20 client.36797 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.082+0000 7f4438184740 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562700e3a1c0 0x5627012cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.082+0000 7f4438184740 4 client.36797 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:55:51.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.082+0000 7f4438184740 1 -- 172.21.15.90:0/2922777595 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562701249130 con 0x562700e3a1c0 2024-05-27T07:55:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.083+0000 7f4429e00640 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562700e3a1c0 0x5627012cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.085+0000 7f4429e00640 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562700e3a1c0 0x5627012cee00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93685e2d04855bb server_cookie=2927ddcdcc06f36a in_seq=0 out_seq=0 2024-05-27T07:55:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.085+0000 7f440be00640 10 client.36797 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.136+0000 7f440be00640 1 -- 172.21.15.90:0/2922777595 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562701249130 con 0x562700e3a1c0 2024-05-27T07:55:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.136+0000 7f440be00640 10 client.36797 handle_command_reply: tid=0 2024-05-27T07:55:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.136+0000 7f4438184740 1 client.36797 shutdown 2024-05-27T07:55:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.137+0000 7f4438184740 1 -- 172.21.15.90:0/2922777595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 msgr2=0x562701258470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.137+0000 7f4438184740 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x562701258470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f44240caa70 tx=0x7f44240caf50 comp rx=0 tx=0).stop 2024-05-27T07:55:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.139+0000 7f4438184740 1 -- 172.21.15.90:0/2922777595 shutdown_connections 2024-05-27T07:55:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.139+0000 7f4438184740 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x562700e3a1c0 0x5627012cee00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.139+0000 7f4438184740 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562700ea61b0 0x562701258470 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.139+0000 7f4438184740 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562700e854d0 0x562701257f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.139+0000 7f4438184740 1 --2- 172.21.15.90:0/2922777595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562700e7d2f0 0x5627012579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.139+0000 7f4438184740 1 -- 172.21.15.90:0/2922777595 >> 172.21.15.90:0/2922777595 conn(0x562700e9a300 msgr2=0x562701252a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.139+0000 7f440be00640 0 client.36797 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:55:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.140+0000 7f4438184740 1 -- 172.21.15.90:0/2922777595 shutdown_connections 2024-05-27T07:55:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.141+0000 7f4438184740 1 -- 172.21.15.90:0/2922777595 wait complete. 2024-05-27T07:55:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.141+0000 7f4438184740 20 client.36797 trim_cache size 0 max 16384 2024-05-27T07:55:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:55:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:55:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:55:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.143+0000 7f4431000640 1 -- 172.21.15.90:0/2368642659 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44100b2950 msgr2=0x7f44100b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.143+0000 7f4431000640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44100b2950 0x7f44100b4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f442c106b90 tx=0x7f441c00b390 comp rx=0 tx=0).stop 2024-05-27T07:55:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.143+0000 7f4431000640 1 -- 172.21.15.90:0/2368642659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442c10c350 msgr2=0x7f442c1acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.143+0000 7f4431000640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442c10c350 0x7f442c1acb10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f442402add0 tx=0x7f4424030d50 comp rx=0 tx=0).stop 2024-05-27T07:55:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.143+0000 7f4431000640 1 -- 172.21.15.90:0/2368642659 shutdown_connections 2024-05-27T07:55:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.143+0000 7f4431000640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44100b2950 0x7f44100b4e60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.143+0000 7f4431000640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f442c10c350 0x7f442c1acb10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.143+0000 7f4431000640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c10beb0 0x7f442c1a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.143+0000 7f4431000640 1 --2- 172.21.15.90:0/2368642659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f442c106600 0x7f442c1a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.143+0000 7f4431000640 1 -- 172.21.15.90:0/2368642659 >> 172.21.15.90:0/2368642659 conn(0x7f442c1016b0 msgr2=0x7f442c07b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.144+0000 7f4431000640 1 -- 172.21.15.90:0/2368642659 shutdown_connections 2024-05-27T07:55:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:55:51.144+0000 7f4431000640 1 -- 172.21.15.90:0/2368642659 wait complete. 2024-05-27T07:55:51.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.707+0000 7fe2f4800700 1 -- 172.21.15.90:0/3426584258 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2ec106eb0 msgr2=0x7fe2ec117b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.707+0000 7fe2f4800700 1 --2- 172.21.15.90:0/3426584258 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2ec106eb0 0x7fe2ec117b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800b0a0 tx=0x7fe2e800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.707+0000 7fe2f4800700 1 -- 172.21.15.90:0/3426584258 shutdown_connections 2024-05-27T07:55:51.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.707+0000 7fe2f4800700 1 --2- 172.21.15.90:0/3426584258 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2ec106eb0 0x7fe2ec117b30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.707+0000 7fe2f4800700 1 --2- 172.21.15.90:0/3426584258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2ec10c5a0 0x7fe2ec10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.707+0000 7fe2f4800700 1 --2- 172.21.15.90:0/3426584258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2ec106560 0x7fe2ec106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.707+0000 7fe2f4800700 1 -- 172.21.15.90:0/3426584258 >> 172.21.15.90:0/3426584258 conn(0x7fe2ec102040 msgr2=0x7fe2ec104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.708+0000 7fe2f4800700 1 -- 172.21.15.90:0/3426584258 shutdown_connections 2024-05-27T07:55:51.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.708+0000 7fe2f4800700 1 -- 172.21.15.90:0/3426584258 wait complete. 2024-05-27T07:55:51.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.709+0000 7fe2f4800700 1 Processor -- start 2024-05-27T07:55:51.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.709+0000 7fe2f4800700 1 -- start start 2024-05-27T07:55:51.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.710+0000 7fe2f4800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2ec106560 0x7fe2ec117770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.710+0000 7fe2f4800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2ec106eb0 0x7fe2ec1138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.710+0000 7fe2f4800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2ec10c5a0 0x7fe2ec113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.710+0000 7fe2f4800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec10f950 con 0x7fe2ec10c5a0 2024-05-27T07:55:51.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.710+0000 7fe2f4800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec10f7d0 con 0x7fe2ec106560 2024-05-27T07:55:51.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.710+0000 7fe2f4800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec10fad0 con 0x7fe2ec106eb0 2024-05-27T07:55:51.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.710+0000 7fe2f1400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2ec106eb0 0x7fe2ec1138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.710+0000 7fe2f1400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2ec106eb0 0x7fe2ec1138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38542/0 (socket says 172.21.15.90:38542) 2024-05-27T07:55:51.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.711+0000 7fe2f1400700 1 -- 172.21.15.90:0/4210010772 learned_addr learned my addr 172.21.15.90:0/4210010772 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:51.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.711+0000 7fe2f1e00700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2ec106560 0x7fe2ec117770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.711+0000 7fe2f2800700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2ec10c5a0 0x7fe2ec113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.712+0000 7fe2f1400700 1 -- 172.21.15.90:0/4210010772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2ec106560 msgr2=0x7fe2ec117770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.712+0000 7fe2f1400700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2ec106560 0x7fe2ec117770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.712+0000 7fe2f1400700 1 -- 172.21.15.90:0/4210010772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2ec10c5a0 msgr2=0x7fe2ec113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.712+0000 7fe2f1400700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2ec10c5a0 0x7fe2ec113df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.712+0000 7fe2f1400700 1 -- 172.21.15.90:0/4210010772 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec1143e0 con 0x7fe2ec106eb0 2024-05-27T07:55:51.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.712+0000 7fe2f1400700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2ec106eb0 0x7fe2ec1138b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00d950 tx=0x7fe2dc00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.712+0000 7fe2e6a00700 1 -- 172.21.15.90:0/4210010772 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc017070 con 0x7fe2ec106eb0 2024-05-27T07:55:51.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.713+0000 7fe2e6a00700 1 -- 172.21.15.90:0/4210010772 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2dc00fd20 con 0x7fe2ec106eb0 2024-05-27T07:55:51.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.713+0000 7fe2e6a00700 1 -- 172.21.15.90:0/4210010772 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00e3b0 con 0x7fe2ec106eb0 2024-05-27T07:55:51.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.713+0000 7fe2f4800700 1 -- 172.21.15.90:0/4210010772 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ec114670 con 0x7fe2ec106eb0 2024-05-27T07:55:51.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.713+0000 7fe2f4800700 1 -- 172.21.15.90:0/4210010772 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ec06d490 con 0x7fe2ec106eb0 2024-05-27T07:55:51.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.720+0000 7fe2e6a00700 1 -- 172.21.15.90:0/4210010772 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe2dc01a070 con 0x7fe2ec106eb0 2024-05-27T07:55:51.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.721+0000 7fe3032e5b80 1 Processor -- start 2024-05-27T07:55:51.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.723+0000 7fe3032e5b80 1 -- start start 2024-05-27T07:55:51.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.723+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010e18e40 0x55d010f9d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.723+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010f9dd30 0x55d010f97870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.723+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f97db0 0x55d010f981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.723+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d010ec5e40 con 0x55d010f97db0 2024-05-27T07:55:51.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.723+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d010eb59e0 con 0x55d010e18e40 2024-05-27T07:55:51.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.723+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d010eb9a20 con 0x55d010f9dd30 2024-05-27T07:55:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.724+0000 7fe2e6a00700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2d40b2b20 0x7fe2d40b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.724+0000 7fe2f2800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010e18e40 0x55d010f9d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.724+0000 7fe2f2800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010e18e40 0x55d010f9d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:54420/0 (socket says 172.21.15.90:54420) 2024-05-27T07:55:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.724+0000 7fe2f2800700 1 -- 172.21.15.90:0/1061519383 learned_addr learned my addr 172.21.15.90:0/1061519383 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.725+0000 7fe2f2800700 1 -- 172.21.15.90:0/1061519383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010f9dd30 msgr2=0x55d010f97870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.725+0000 7fe2f2800700 1 --2- 172.21.15.90:0/1061519383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010f9dd30 0x55d010f97870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.725+0000 7fe2f1400700 1 --2- 172.21.15.90:0/1061519383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f97db0 0x55d010f981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.725+0000 7fe2f2800700 1 -- 172.21.15.90:0/1061519383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f97db0 msgr2=0x55d010f981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.725+0000 7fe2f2800700 1 --2- 172.21.15.90:0/1061519383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f97db0 0x55d010f981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.725+0000 7fe2f2800700 1 -- 172.21.15.90:0/1061519383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d010e7e420 con 0x55d010e18e40 2024-05-27T07:55:51.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.725+0000 7fe2e6a00700 1 -- 172.21.15.90:0/4210010772 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fe2dc0e2210 con 0x7fe2ec106eb0 2024-05-27T07:55:51.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.726+0000 7fe2f1400700 1 --2- 172.21.15.90:0/1061519383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f97db0 0x55d010f981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:55:51.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.727+0000 7fe2f2800700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2d40b2b20 0x7fe2d40b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.728+0000 7fe2f2800700 1 --2- 172.21.15.90:0/1061519383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010e18e40 0x55d010f9d7f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80093c0 tx=0x7fe2e80093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c090e16dcd4881e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.728+0000 7fe2db400700 1 -- 172.21.15.90:0/1061519383 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8003a00 con 0x55d010e18e40 2024-05-27T07:55:51.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.729+0000 7fe2db400700 1 -- 172.21.15.90:0/1061519383 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2e802a470 con 0x55d010e18e40 2024-05-27T07:55:51.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.729+0000 7fe2db400700 1 -- 172.21.15.90:0/1061519383 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8003e60 con 0x55d010e18e40 2024-05-27T07:55:51.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.730+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1061519383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010e18e40 msgr2=0x55d010f9d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.730+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1061519383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010e18e40 0x55d010f9d7f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe2e80093c0 tx=0x7fe2e80093f0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.730+0000 7fe2f2800700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2d40b2b20 0x7fe2d40b5030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8029000 tx=0x7fe2e802a000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.730+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1061519383 shutdown_connections 2024-05-27T07:55:51.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.730+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1061519383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f97db0 0x55d010f981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.730+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1061519383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010e18e40 0x55d010f9d7f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.730+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1061519383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010f9dd30 0x55d010f97870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.730+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1061519383 >> 172.21.15.90:0/1061519383 conn(0x55d010e78fc0 msgr2=0x55d010f928d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.731+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1061519383 shutdown_connections 2024-05-27T07:55:51.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.731+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1061519383 wait complete. 2024-05-27T07:55:51.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.732+0000 7fe3032e5b80 1 -- start start 2024-05-27T07:55:51.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.732+0000 7fe3032e5b80 1 Processor -- start 2024-05-27T07:55:51.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.733+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010fa14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.733+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 0x55d010fa1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.733+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f9dd30 0x55d010fa1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.733+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d010dbbfa0 con 0x55d010f9dd30 2024-05-27T07:55:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.733+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d010e56760 con 0x55d010f97db0 2024-05-27T07:55:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.733+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d010fa4800 con 0x55d010e18e40 2024-05-27T07:55:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.733+0000 7fe3032e5b80 10 client.0 _ll_register_callbacks cb 0x55d010e5fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.734+0000 7fe2f1400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 0x55d010fa1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.734+0000 7fe2f1e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010fa14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.734+0000 7fe2f1e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010fa14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38552/0 (socket says 172.21.15.90:38552) 2024-05-27T07:55:51.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.734+0000 7fe2f1e00700 1 -- 172.21.15.90:0/2402498679 learned_addr learned my addr 172.21.15.90:0/2402498679 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:51.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.734+0000 7fe2f1e00700 1 -- 172.21.15.90:0/2402498679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 msgr2=0x55d010fa1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.734+0000 7fe2f1e00700 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 0x55d010fa1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.734+0000 7fe2f1e00700 1 -- 172.21.15.90:0/2402498679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f9dd30 msgr2=0x55d010fa1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.734+0000 7fe2f1e00700 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f9dd30 0x55d010fa1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.734+0000 7fe2f1e00700 1 -- 172.21.15.90:0/2402498679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d010e058d0 con 0x55d010e18e40 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.735+0000 7fe2f1400700 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 0x55d010fa1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.735+0000 7fe2f1e00700 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010fa14d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00226d0 tx=0x7fe2e00229e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f940213ddd9027ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.735+0000 7fe2cfe00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.735+0000 7fe2cfe00700 1 -- 172.21.15.90:0/2402498679 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e002f060 con 0x55d010e18e40 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.735+0000 7fe3032e5b80 1 -- 172.21.15.90:0/2402498679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e802a470 con 0x55d010e18e40 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.736+0000 7fe2cfe00700 1 -- 172.21.15.90:0/2402498679 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fe2e0004260 con 0x55d010e18e40 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.736+0000 7fe2cfe00700 1 -- 172.21.15.90:0/2402498679 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fe2e0036050 con 0x55d010e18e40 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.736+0000 7fe3032e5b80 10 client.26854 fetch_fsmap learned FSMap version 58 2024-05-27T07:55:51.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.736+0000 7fe3032e5b80 1 -- 172.21.15.90:0/2402498679 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d010e86060 con 0x55d010e18e40 2024-05-27T07:55:51.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.736+0000 7fe2cfe00700 1 -- 172.21.15.90:0/2402498679 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2e802a470 con 0x55d010e18e40 2024-05-27T07:55:51.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.736+0000 7fe2cfe00700 1 -- 172.21.15.90:0/2402498679 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 58) ==== 6559+0+0 (secure 0 0 0) 0x7fe2e0020480 con 0x55d010e18e40 2024-05-27T07:55:51.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.737+0000 7fe3032e5b80 10 client.26854 fetch_fsmap finished waiting for FSMap version 58 2024-05-27T07:55:51.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.737+0000 7fe3032e5b80 10 client.26854 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:55:51.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.737+0000 7fe3032e5b80 20 client.26854 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:51.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.737+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d010fb1860 0x55d010fb3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.737+0000 7fe3032e5b80 4 client.26854 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:55:51.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.737+0000 7fe3032e5b80 1 -- 172.21.15.90:0/2402498679 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d010ec7d90 con 0x55d010fb1860 2024-05-27T07:55:51.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.737+0000 7fe2f1400700 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d010fb1860 0x55d010fb3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.738+0000 7fe2f1400700 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d010fb1860 0x55d010fb3d10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20f06306082956d2 server_cookie=172456b248dec07d in_seq=0 out_seq=0 2024-05-27T07:55:51.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.738+0000 7fe2cfe00700 10 client.26854 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:55:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.740+0000 7fe2cfe00700 1 -- 172.21.15.90:0/2402498679 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d010ec7d90 con 0x55d010fb1860 2024-05-27T07:55:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.740+0000 7fe2cfe00700 10 client.26854 handle_command_reply: tid=0 2024-05-27T07:55:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.740+0000 7fe3032e5b80 1 client.26854 shutdown 2024-05-27T07:55:51.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.741+0000 7fe3032e5b80 1 -- 172.21.15.90:0/2402498679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 msgr2=0x55d010fa14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.741+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010fa14d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00226d0 tx=0x7fe2e00229e0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.741+0000 7fe3032e5b80 1 -- 172.21.15.90:0/2402498679 shutdown_connections 2024-05-27T07:55:51.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.741+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d010fb1860 0x55d010fb3d10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.741+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f9dd30 0x55d010fa1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.742+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010fa14d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.742+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/2402498679 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 0x55d010fa1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.742+0000 7fe3032e5b80 1 -- 172.21.15.90:0/2402498679 >> 172.21.15.90:0/2402498679 conn(0x55d010e78fc0 msgr2=0x55d010f92580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.742+0000 7fe2cfe00700 0 client.26854 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:55:51.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.742+0000 7fe3032e5b80 1 -- 172.21.15.90:0/2402498679 shutdown_connections 2024-05-27T07:55:51.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.742+0000 7fe3032e5b80 1 -- 172.21.15.90:0/2402498679 wait complete. 2024-05-27T07:55:51.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.742+0000 7fe3032e5b80 20 client.26854 trim_cache size 0 max 16384 2024-05-27T07:55:51.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.768+0000 7fe3032e5b80 1 Processor -- start 2024-05-27T07:55:51.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.769+0000 7fe3032e5b80 1 -- start start 2024-05-27T07:55:51.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.769+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010f97870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.769+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 0x55d010f9cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.769+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f9dd30 0x55d010fa0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.769+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b8670 con 0x55d010f9dd30 2024-05-27T07:55:51.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.769+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec074f70 con 0x55d010f97db0 2024-05-27T07:55:51.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.769+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b78d0 con 0x55d010e18e40 2024-05-27T07:55:51.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.770+0000 7fe2f2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f9dd30 0x55d010fa0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.770+0000 7fe2f1400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 0x55d010f9cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.770+0000 7fe2f1e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010f97870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.770+0000 7fe2f1e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010f97870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38568/0 (socket says 172.21.15.90:38568) 2024-05-27T07:55:51.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.770+0000 7fe2f1e00700 1 -- 172.21.15.90:0/1804878829 learned_addr learned my addr 172.21.15.90:0/1804878829 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.772+0000 7fe2f1e00700 1 -- 172.21.15.90:0/1804878829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 msgr2=0x55d010f9cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.772+0000 7fe2f1e00700 1 --2- 172.21.15.90:0/1804878829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 0x55d010f9cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.772+0000 7fe2f1e00700 1 -- 172.21.15.90:0/1804878829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f9dd30 msgr2=0x55d010fa0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.772+0000 7fe2f1e00700 1 --2- 172.21.15.90:0/1804878829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f9dd30 0x55d010fa0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.772+0000 7fe2f1e00700 1 -- 172.21.15.90:0/1804878829 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d010fa22c0 con 0x55d010e18e40 2024-05-27T07:55:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.772+0000 7fe2f1400700 1 --2- 172.21.15.90:0/1804878829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 0x55d010f9cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:55:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.773+0000 7fe2f1e00700 1 --2- 172.21.15.90:0/1804878829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010f97870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0005950 tx=0x7fe2e0005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e662f6fd3d029a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.774+0000 7fe2daa00700 1 -- 172.21.15.90:0/1804878829 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0020ac0 con 0x55d010e18e40 2024-05-27T07:55:51.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.774+0000 7fe2daa00700 1 -- 172.21.15.90:0/1804878829 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2e0020c60 con 0x55d010e18e40 2024-05-27T07:55:51.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.774+0000 7fe2daa00700 1 -- 172.21.15.90:0/1804878829 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0031300 con 0x55d010e18e40 2024-05-27T07:55:51.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.775+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1804878829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 msgr2=0x55d010f97870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.775+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1804878829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010f97870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0005950 tx=0x7fe2e0005ec0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.775+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1804878829 shutdown_connections 2024-05-27T07:55:51.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.775+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1804878829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010f9dd30 0x55d010fa0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.775+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1804878829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010e18e40 0x55d010f97870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.775+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1804878829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f97db0 0x55d010f9cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.775+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1804878829 >> 172.21.15.90:0/1804878829 conn(0x55d010e78fc0 msgr2=0x55d010f9b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.777+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1804878829 shutdown_connections 2024-05-27T07:55:51.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.777+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1804878829 wait complete. 2024-05-27T07:55:51.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.778+0000 7fe3032e5b80 1 -- start start 2024-05-27T07:55:51.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.778+0000 7fe3032e5b80 1 Processor -- start 2024-05-27T07:55:51.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.779+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010e18e40 0x55d010fb9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.779+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010f97db0 0x55d010fa9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.779+0000 7fe3032e5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f9dd30 0x55d010fa9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.779+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b7200 con 0x55d010e18e40 2024-05-27T07:55:51.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.779+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec075640 con 0x55d010f9dd30 2024-05-27T07:55:51.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.779+0000 7fe3032e5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ec1b6460 con 0x55d010f97db0 2024-05-27T07:55:51.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.779+0000 7fe3032e5b80 10 client.0 _ll_register_callbacks cb 0x55d010e5fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:55:51.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.780+0000 7fe2f1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010e18e40 0x55d010fb9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.780+0000 7fe2f1400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010f97db0 0x55d010fa9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.780+0000 7fe2f1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010e18e40 0x55d010fb9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45390/0 (socket says 172.21.15.90:45390) 2024-05-27T07:55:51.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.781+0000 7fe2f1e00700 1 -- 172.21.15.90:0/1809734550 learned_addr learned my addr 172.21.15.90:0/1809734550 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:55:51.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.782+0000 7fe2f1e00700 1 -- 172.21.15.90:0/1809734550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010f97db0 msgr2=0x55d010fa9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.782+0000 7fe2f1e00700 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010f97db0 0x55d010fa9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.782+0000 7fe2f1e00700 1 -- 172.21.15.90:0/1809734550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f9dd30 msgr2=0x55d010fa9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:55:51.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.782+0000 7fe2f1e00700 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f9dd30 0x55d010fa9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.782+0000 7fe2f1e00700 1 -- 172.21.15.90:0/1809734550 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d010fb8500 con 0x55d010e18e40 2024-05-27T07:55:51.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.782+0000 7fe2f1e00700 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010e18e40 0x55d010fb9ca0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0022a20 tx=0x7fe2e0005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62a1762d22585ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:55:51.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.782+0000 7fe2cfe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:55:51.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.783+0000 7fe2cfe00700 1 -- 172.21.15.90:0/1809734550 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0020e90 con 0x55d010e18e40 2024-05-27T07:55:51.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.783+0000 7fe2cfe00700 1 -- 172.21.15.90:0/1809734550 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fe2e0031d80 con 0x55d010e18e40 2024-05-27T07:55:51.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.783+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1809734550 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2e0020c60 con 0x55d010e18e40 2024-05-27T07:55:51.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.783+0000 7fe2cfe00700 1 -- 172.21.15.90:0/1809734550 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fe2e0020a70 con 0x55d010e18e40 2024-05-27T07:55:51.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.783+0000 7fe3032e5b80 10 client.17229 fetch_fsmap learned FSMap version 58 2024-05-27T07:55:51.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.783+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1809734550 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d01101b580 con 0x55d010e18e40 2024-05-27T07:55:51.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.784+0000 7fe2cfe00700 1 -- 172.21.15.90:0/1809734550 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2e0020c60 con 0x55d010e18e40 2024-05-27T07:55:51.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.784+0000 7fe2cfe00700 1 -- 172.21.15.90:0/1809734550 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 58) ==== 6559+0+0 (secure 0 0 0) 0x7fe2e0039070 con 0x55d010e18e40 2024-05-27T07:55:51.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.784+0000 7fe3032e5b80 10 client.17229 fetch_fsmap finished waiting for FSMap version 58 2024-05-27T07:55:51.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.784+0000 7fe3032e5b80 10 client.17229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:55:51.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.784+0000 7fe3032e5b80 20 client.17229 populate_metadata read hostname 'smithi090' 2024-05-27T07:55:51.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.784+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d01101bbb0 0x55d01101e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:55:51.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.784+0000 7fe3032e5b80 4 client.17229 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:55:51.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.784+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1809734550 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d010ec1570 con 0x55d01101bbb0 2024-05-27T07:55:51.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.786+0000 7fe2f1400700 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d01101bbb0 0x55d01101e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:55:51.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.788+0000 7fe2cfe00700 10 client.17229 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:55:51.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.788+0000 7fe2f1400700 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d01101bbb0 0x55d01101e010 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa1f963e77073c44 server_cookie=f37f37edfc68c0b3 in_seq=0 out_seq=0 2024-05-27T07:55:51.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.789+0000 7fe2cfe00700 1 -- 172.21.15.90:0/1809734550 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d010ec1570 con 0x55d01101bbb0 2024-05-27T07:55:51.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.789+0000 7fe2cfe00700 10 client.17229 handle_command_reply: tid=0 2024-05-27T07:55:51.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.789+0000 7fe3032e5b80 1 client.17229 shutdown 2024-05-27T07:55:51.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1809734550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010e18e40 msgr2=0x55d010fb9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010e18e40 0x55d010fb9ca0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0022a20 tx=0x7fe2e0005530 comp rx=0 tx=0).stop 2024-05-27T07:55:51.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1809734550 shutdown_connections 2024-05-27T07:55:51.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55d01101bbb0 0x55d01101e010 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d010f9dd30 0x55d010fa9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d010e18e40 0x55d010fb9ca0 secure :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0022a20 tx=0x7fe2e0005530 comp rx=0 tx=0).stop 2024-05-27T07:55:51.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe3032e5b80 1 --2- 172.21.15.90:0/1809734550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d010f97db0 0x55d010fa9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1809734550 >> 172.21.15.90:0/1809734550 conn(0x55d010e78fc0 msgr2=0x55d010f9b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe2cfe00700 0 client.17229 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:55:51.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1809734550 shutdown_connections 2024-05-27T07:55:51.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.790+0000 7fe3032e5b80 1 -- 172.21.15.90:0/1809734550 wait complete. 2024-05-27T07:55:51.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.791+0000 7fe3032e5b80 20 client.17229 trim_cache size 0 max 16384 2024-05-27T07:55:51.792 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:55:51.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.791+0000 7fe2f4800700 1 -- 172.21.15.90:0/4210010772 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2d40b2b20 msgr2=0x7fe2d40b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.791+0000 7fe2f4800700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2d40b2b20 0x7fe2d40b5030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8029000 tx=0x7fe2e802a000 comp rx=0 tx=0).stop 2024-05-27T07:55:51.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.791+0000 7fe2f4800700 1 -- 172.21.15.90:0/4210010772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2ec106eb0 msgr2=0x7fe2ec1138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:55:51.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.791+0000 7fe2f4800700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2ec106eb0 0x7fe2ec1138b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00d950 tx=0x7fe2dc00dc60 comp rx=0 tx=0).stop 2024-05-27T07:55:51.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.792+0000 7fe2f4800700 1 -- 172.21.15.90:0/4210010772 shutdown_connections 2024-05-27T07:55:51.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.792+0000 7fe2f4800700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2ec10c5a0 0x7fe2ec113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.792+0000 7fe2f4800700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2d40b2b20 0x7fe2d40b5030 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.792+0000 7fe2f4800700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe2ec106560 0x7fe2ec117770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.792+0000 7fe2f4800700 1 --2- 172.21.15.90:0/4210010772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2ec106eb0 0x7fe2ec1138b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:55:51.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.792+0000 7fe2f4800700 1 -- 172.21.15.90:0/4210010772 >> 172.21.15.90:0/4210010772 conn(0x7fe2ec102040 msgr2=0x7fe2ec1034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:55:51.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.792+0000 7fe2f4800700 1 -- 172.21.15.90:0/4210010772 shutdown_connections 2024-05-27T07:55:51.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:55:51.792+0000 7fe2f4800700 1 -- 172.21.15.90:0/4210010772 wait complete. 2024-05-27T07:55:52.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:52 smithi090 ceph-mon[31429]: pgmap v299: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.0 MiB/s wr, 503 op/s 2024-05-27T07:55:52.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:52 smithi129 ceph-mon[35254]: pgmap v299: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.0 MiB/s wr, 503 op/s 2024-05-27T07:55:52.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:52 smithi136 ceph-mon[35324]: pgmap v299: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.0 MiB/s wr, 503 op/s 2024-05-27T07:55:53.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:53 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 4. 2024-05-27T07:55:53.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:53 smithi090 systemd[1]: Stopped Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:55:53.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:53 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.568s CPU time. 2024-05-27T07:55:53.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:53 smithi090 systemd[1]: Starting Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:55:53.827 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 335.0 s, 600.8 tps, lat 165.473 ms stddev 60.881 2024-05-27T07:55:54.017 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:55:54.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:54 smithi090 ceph-mon[31429]: pgmap v300: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.2 MiB/s wr, 525 op/s 2024-05-27T07:55:54.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:54 smithi129 ceph-mon[35254]: pgmap v300: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.2 MiB/s wr, 525 op/s 2024-05-27T07:55:54.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:54 smithi136 ceph-mon[35324]: pgmap v300: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.2 MiB/s wr, 525 op/s 2024-05-27T07:55:54.778 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:54 smithi090 podman[114416]: 2024-05-27 07:55:54.394266785 +0000 UTC m=+0.069344646 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:55:54.779 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:54 smithi090 podman[114416]: 2024-05-27 07:55:54.532790724 +0000 UTC m=+0.207868571 container create 238e14834e321fde3dfacd449cdf939e0e39b68aca0ca7c0867d46a2b1824500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True) 2024-05-27T07:55:55.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:54 smithi090 podman[114416]: 2024-05-27 07:55:54.870156954 +0000 UTC m=+0.545234823 container init 238e14834e321fde3dfacd449cdf939e0e39b68aca0ca7c0867d46a2b1824500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T07:55:55.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:54 smithi090 podman[114416]: 2024-05-27 07:55:54.883831523 +0000 UTC m=+0.558909362 container start 238e14834e321fde3dfacd449cdf939e0e39b68aca0ca7c0867d46a2b1824500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-27T07:55:55.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:54 smithi090 bash[114416]: 238e14834e321fde3dfacd449cdf939e0e39b68aca0ca7c0867d46a2b1824500 2024-05-27T07:55:55.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:55 smithi090 systemd[1]: Started Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:55:55.215 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:55 smithi090 ceph-mds[114452]: -- 172.21.15.90:0/4100523395 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e10153a40 con 0x561e1019ba80 2024-05-27T07:55:55.966 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:55 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: starting mds.j at 2024-05-27T07:55:56.083 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:55:56.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:56 smithi090 ceph-mon[31429]: pgmap v301: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.9 MiB/s wr, 686 op/s 2024-05-27T07:55:56.718 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fd087800700 time 2024-05-27T07:55:56.373356+0000 2024-05-27T07:55:56.718 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:56.718 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:56.718 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fd09d312ba6] 2024-05-27T07:55:56.718 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fd09d312dd1] 2024-05-27T07:55:56.718 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561ddb3e723d] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561ddb3d2971] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 5: (SnapRealm::check_cache() const+0x13e) [0x561ddb3d0fa0] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561ddb3d1651] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561ddb366347] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561ddb06287e] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561ddb067f2f] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561ddb072969] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 11: (MDLog::_replay_thread()+0x1f38) [0x561ddb40b48c] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561ddb4149cb] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 13: (Thread::entry_wrapper()+0x43) [0x7fd09d2e172d] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 14: (Thread::_entry_func(void*)+0xd) [0x7fd09d2e1749] 2024-05-27T07:55:56.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fd09be851ca] 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 16: clone() 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: *** Caught signal (Aborted) ** 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: in thread 7fd087800700 thread_name:md_log_replay 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.376+0000 7fd087800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fd087800700 time 2024-05-27T07:55:56.373356+0000 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fd09d312ba6] 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fd09d312dd1] 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561ddb3e723d] 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561ddb3d2971] 2024-05-27T07:55:56.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 5: (SnapRealm::check_cache() const+0x13e) [0x561ddb3d0fa0] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561ddb3d1651] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561ddb366347] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561ddb06287e] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561ddb067f2f] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561ddb072969] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 11: (MDLog::_replay_thread()+0x1f38) [0x561ddb40b48c] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561ddb4149cb] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 13: (Thread::entry_wrapper()+0x43) [0x7fd09d2e172d] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 14: (Thread::_entry_func(void*)+0xd) [0x7fd09d2e1749] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fd09be851ca] 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 16: clone() 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:56.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd09be8fd20] 2024-05-27T07:55:56.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2: gsignal() 2024-05-27T07:55:56.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 3: abort() 2024-05-27T07:55:56.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fd09d312ced] 2024-05-27T07:55:56.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fd09d312dd1] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561ddb3e723d] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561ddb3d2971] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 8: (SnapRealm::check_cache() const+0x13e) [0x561ddb3d0fa0] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561ddb3d1651] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561ddb366347] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561ddb06287e] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561ddb067f2f] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561ddb072969] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 14: (MDLog::_replay_thread()+0x1f38) [0x561ddb40b48c] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561ddb4149cb] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 16: (Thread::entry_wrapper()+0x43) [0x7fd09d2e172d] 2024-05-27T07:55:56.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 17: (Thread::_entry_func(void*)+0xd) [0x7fd09d2e1749] 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fd09be851ca] 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 19: clone() 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.383+0000 7fd087800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: in thread 7fd087800700 thread_name:md_log_replay 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd09be8fd20] 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2: gsignal() 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 3: abort() 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fd09d312ced] 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fd09d312dd1] 2024-05-27T07:55:56.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561ddb3e723d] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561ddb3d2971] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 8: (SnapRealm::check_cache() const+0x13e) [0x561ddb3d0fa0] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561ddb3d1651] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561ddb366347] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561ddb06287e] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561ddb067f2f] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561ddb072969] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 14: (MDLog::_replay_thread()+0x1f38) [0x561ddb40b48c] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561ddb4149cb] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 16: (Thread::entry_wrapper()+0x43) [0x7fd09d2e172d] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 17: (Thread::_entry_func(void*)+0xd) [0x7fd09d2e1749] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fd09be851ca] 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 19: clone() 2024-05-27T07:55:56.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: -1> 2024-05-27T07:55:56.376+0000 7fd087800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fd087800700 time 2024-05-27T07:55:56.373356+0000 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fd09d312ba6] 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fd09d312dd1] 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561ddb3e723d] 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561ddb3d2971] 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 5: (SnapRealm::check_cache() const+0x13e) [0x561ddb3d0fa0] 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561ddb3d1651] 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561ddb366347] 2024-05-27T07:55:56.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561ddb06287e] 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561ddb067f2f] 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561ddb072969] 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 11: (MDLog::_replay_thread()+0x1f38) [0x561ddb40b48c] 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561ddb4149cb] 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 13: (Thread::entry_wrapper()+0x43) [0x7fd09d2e172d] 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 14: (Thread::_entry_func(void*)+0xd) [0x7fd09d2e1749] 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fd09be851ca] 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 16: clone() 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 0> 2024-05-27T07:55:56.383+0000 7fd087800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: in thread 7fd087800700 thread_name:md_log_replay 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd09be8fd20] 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2: gsignal() 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 3: abort() 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fd09d312ced] 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fd09d312dd1] 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561ddb3e723d] 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561ddb3d2971] 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 8: (SnapRealm::check_cache() const+0x13e) [0x561ddb3d0fa0] 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561ddb3d1651] 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561ddb366347] 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561ddb06287e] 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561ddb067f2f] 2024-05-27T07:55:56.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561ddb072969] 2024-05-27T07:55:56.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 14: (MDLog::_replay_thread()+0x1f38) [0x561ddb40b48c] 2024-05-27T07:55:56.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561ddb4149cb] 2024-05-27T07:55:56.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 16: (Thread::entry_wrapper()+0x43) [0x7fd09d2e172d] 2024-05-27T07:55:56.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 17: (Thread::_entry_func(void*)+0xd) [0x7fd09d2e1749] 2024-05-27T07:55:56.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fd09be851ca] 2024-05-27T07:55:56.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 19: clone() 2024-05-27T07:55:56.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:56.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: -1> 2024-05-27T07:55:56.376+0000 7fd087800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7fd087800700 time 2024-05-27T07:55:56.373356+0000 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7fd09d312ba6] 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fd09d312dd1] 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561ddb3e723d] 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561ddb3d2971] 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 5: (SnapRealm::check_cache() const+0x13e) [0x561ddb3d0fa0] 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561ddb3d1651] 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561ddb366347] 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561ddb06287e] 2024-05-27T07:55:56.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561ddb067f2f] 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 10: (EUpdate::replay(MDSRank*)+0x63) [0x561ddb072969] 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 11: (MDLog::_replay_thread()+0x1f38) [0x561ddb40b48c] 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 12: (MDLog::ReplayThread::entry()+0x11) [0x561ddb4149cb] 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 13: (Thread::entry_wrapper()+0x43) [0x7fd09d2e172d] 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 14: (Thread::_entry_func(void*)+0xd) [0x7fd09d2e1749] 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fd09be851ca] 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 16: clone() 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 0> 2024-05-27T07:55:56.383+0000 7fd087800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: in thread 7fd087800700 thread_name:md_log_replay 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd09be8fd20] 2024-05-27T07:55:56.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2: gsignal() 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 3: abort() 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7fd09d312ced] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7fd09d312dd1] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x561ddb3e723d] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561ddb3d2971] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 8: (SnapRealm::check_cache() const+0x13e) [0x561ddb3d0fa0] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561ddb3d1651] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561ddb366347] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561ddb06287e] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561ddb067f2f] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 13: (EUpdate::replay(MDSRank*)+0x63) [0x561ddb072969] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 14: (MDLog::_replay_thread()+0x1f38) [0x561ddb40b48c] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 15: (MDLog::ReplayThread::entry()+0x11) [0x561ddb4149cb] 2024-05-27T07:55:56.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 16: (Thread::entry_wrapper()+0x43) [0x7fd09d2e172d] 2024-05-27T07:55:56.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 17: (Thread::_entry_func(void*)+0xd) [0x7fd09d2e1749] 2024-05-27T07:55:56.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7fd09be851ca] 2024-05-27T07:55:56.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 19: clone() 2024-05-27T07:55:56.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:55:56.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114437]: 2024-05-27T07:55:56.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:56 smithi090 podman[114508]: 2024-05-27 07:55:56.516202131 +0000 UTC m=+0.063522548 container died 238e14834e321fde3dfacd449cdf939e0e39b68aca0ca7c0867d46a2b1824500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-27T07:55:56.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:56 smithi136 ceph-mon[35324]: pgmap v301: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.9 MiB/s wr, 686 op/s 2024-05-27T07:55:56.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:56 smithi129 ceph-mon[35254]: pgmap v301: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.9 MiB/s wr, 686 op/s 2024-05-27T07:55:57.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:57 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6834/53311746,v1:172.21.15.90:6835/53311746] up:boot 2024-05-27T07:55:57.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:57 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:55:57.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:57 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:55:57.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:57 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:57.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:55:57 smithi090 ceph-mon[31429]: pgmap v302: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-05-27T07:55:57.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:57 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6834/53311746,v1:172.21.15.90:6835/53311746] up:boot 2024-05-27T07:55:57.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:57 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:55:57.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:57 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:55:57.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:57 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:57.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:55:57 smithi136 ceph-mon[35324]: pgmap v302: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-05-27T07:55:57.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:57 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6834/53311746,v1:172.21.15.90:6835/53311746] up:boot 2024-05-27T07:55:57.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:57 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:55:57.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:57 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:55:57.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:57 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:55:57.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:55:57 smithi129 ceph-mon[35254]: pgmap v302: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 5.2 MiB/s wr, 524 op/s 2024-05-27T07:55:58.213 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:57 smithi090 podman[114508]: 2024-05-27 07:55:57.921361993 +0000 UTC m=+1.468682409 container remove 238e14834e321fde3dfacd449cdf939e0e39b68aca0ca7c0867d46a2b1824500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T07:55:58.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:55:57 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:55:58.831 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 340.0 s, 550.2 tps, lat 157.695 ms stddev 57.555 2024-05-27T07:56:00.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:00 smithi129 ceph-mon[35254]: pgmap v303: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.2 MiB/s wr, 727 op/s 2024-05-27T07:56:00.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:00 smithi090 ceph-mon[31429]: pgmap v303: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.2 MiB/s wr, 727 op/s 2024-05-27T07:56:00.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:00 smithi136 ceph-mon[35324]: pgmap v303: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.2 MiB/s wr, 727 op/s 2024-05-27T07:56:02.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.072+0000 7f0bd2a00700 1 -- 172.21.15.90:0/87539093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc110fa0 msgr2=0x7f0bcc1188c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:02.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.072+0000 7f0bd2a00700 1 --2- 172.21.15.90:0/87539093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc110fa0 0x7f0bcc1188c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400b0d0 tx=0x7f0bc400b3e0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.072+0000 7f0bd2a00700 1 -- 172.21.15.90:0/87539093 shutdown_connections 2024-05-27T07:56:02.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.072+0000 7f0bd2a00700 1 --2- 172.21.15.90:0/87539093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc110fa0 0x7f0bcc1188c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.072+0000 7f0bd2a00700 1 --2- 172.21.15.90:0/87539093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc10c7c0 0x7f0bcc10cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.072+0000 7f0bd2a00700 1 --2- 172.21.15.90:0/87539093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc110650 0x7f0bcc110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.072+0000 7f0bd2a00700 1 -- 172.21.15.90:0/87539093 >> 172.21.15.90:0/87539093 conn(0x7f0bcc06f980 msgr2=0x7f0bcc06fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:02.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.074+0000 7f0bd2a00700 1 -- 172.21.15.90:0/87539093 shutdown_connections 2024-05-27T07:56:02.076 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:01 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-05-27T07:56:02.076 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:01 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.466s CPU time. 2024-05-27T07:56:02.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.075+0000 7f0bd2a00700 1 -- 172.21.15.90:0/87539093 wait complete. 2024-05-27T07:56:02.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.079+0000 7f0bd2a00700 1 Processor -- start 2024-05-27T07:56:02.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.079+0000 7f0bd2a00700 1 -- start start 2024-05-27T07:56:02.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.080+0000 7f0bd2a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc10c7c0 0x7f0bcc117690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.080+0000 7f0bd2a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc110650 0x7f0bcc117c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.080+0000 7f0bd2a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc110fa0 0x7f0bcc1ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.080+0000 7f0bd2a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc11a4f0 con 0x7f0bcc10c7c0 2024-05-27T07:56:02.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.080+0000 7f0bd2a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc11a370 con 0x7f0bcc110650 2024-05-27T07:56:02.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.080+0000 7f0bd2a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc11a670 con 0x7f0bcc110fa0 2024-05-27T07:56:02.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.081+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc10c7c0 0x7f0bcc117690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.081+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc10c7c0 0x7f0bcc117690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46528/0 (socket says 172.21.15.90:46528) 2024-05-27T07:56:02.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.081+0000 7f0bcbe00700 1 -- 172.21.15.90:0/1750887786 learned_addr learned my addr 172.21.15.90:0/1750887786 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:02.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.081+0000 7f0bcb400700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc110650 0x7f0bcc117c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.082+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc110fa0 0x7f0bcc1ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.082+0000 7f0bcbe00700 1 -- 172.21.15.90:0/1750887786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc110fa0 msgr2=0x7f0bcc1ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:02.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.082+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc110fa0 0x7f0bcc1ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.082+0000 7f0bcbe00700 1 -- 172.21.15.90:0/1750887786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc110650 msgr2=0x7f0bcc117c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:02.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.082+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc110650 0x7f0bcc117c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.082+0000 7f0bcbe00700 1 -- 172.21.15.90:0/1750887786 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc1aea00 con 0x7f0bcc10c7c0 2024-05-27T07:56:02.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.083+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc10c7c0 0x7f0bcc117690 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00d950 tx=0x7f0bbc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:02.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.084+0000 7f0bc8c00700 1 -- 172.21.15.90:0/1750887786 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc017070 con 0x7f0bcc10c7c0 2024-05-27T07:56:02.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.084+0000 7f0bc8c00700 1 -- 172.21.15.90:0/1750887786 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bbc00fd20 con 0x7f0bcc10c7c0 2024-05-27T07:56:02.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.084+0000 7f0bc8c00700 1 -- 172.21.15.90:0/1750887786 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc00e3b0 con 0x7f0bcc10c7c0 2024-05-27T07:56:02.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.084+0000 7f0bd2a00700 1 -- 172.21.15.90:0/1750887786 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc1aecf0 con 0x7f0bcc10c7c0 2024-05-27T07:56:02.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.085+0000 7f0bd2a00700 1 -- 172.21.15.90:0/1750887786 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc1b6650 con 0x7f0bcc10c7c0 2024-05-27T07:56:02.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.091+0000 7f0bc8c00700 1 -- 172.21.15.90:0/1750887786 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0bbc01a070 con 0x7f0bcc10c7c0 2024-05-27T07:56:02.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.093+0000 7f0be14e0b80 1 Processor -- start 2024-05-27T07:56:02.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.094+0000 7f0be14e0b80 1 -- start start 2024-05-27T07:56:02.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.095+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a968e9f0 0x55e2a9721330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.095+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a95a2e40 0x55e2a9721870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.095+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a9721e70 0x55e2a972e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.095+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e2a964fe40 con 0x55e2a9721e70 2024-05-27T07:56:02.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.095+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2a963f9e0 con 0x55e2a95a2e40 2024-05-27T07:56:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.095+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e2a9643a20 con 0x55e2a968e9f0 2024-05-27T07:56:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.096+0000 7f0bd0a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a968e9f0 0x55e2a9721330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.096+0000 7f0bd0a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a968e9f0 0x55e2a9721330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33042/0 (socket says 172.21.15.90:33042) 2024-05-27T07:56:02.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.096+0000 7f0bd0a00700 1 -- 172.21.15.90:0/1713854778 learned_addr learned my addr 172.21.15.90:0/1713854778 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:02.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.097+0000 7f0bc8c00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a60 0x7f0bb00b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.097+0000 7f0bd0a00700 1 -- 172.21.15.90:0/1713854778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a95a2e40 msgr2=0x55e2a9721870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:02.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.097+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a95a2e40 0x55e2a9721870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.097+0000 7f0bcb400700 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a9721e70 0x55e2a972e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.097+0000 7f0bc8c00700 1 -- 172.21.15.90:0/1750887786 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f0bbc0e21c0 con 0x7f0bcc10c7c0 2024-05-27T07:56:02.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.098+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a95a2e40 0x55e2a9721870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.098+0000 7f0bd0a00700 1 -- 172.21.15.90:0/1713854778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a9721e70 msgr2=0x55e2a972e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.098+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a9721e70 0x55e2a972e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.098+0000 7f0bd0a00700 1 -- 172.21.15.90:0/1713854778 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a9608420 con 0x55e2a968e9f0 2024-05-27T07:56:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.098+0000 7f0bcb400700 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a9721e70 0x55e2a972e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.098+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a95a2e40 0x55e2a9721870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.099+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a60 0x7f0bb00b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.099+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a968e9f0 0x55e2a9721330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4015000 tx=0x7f0bc40091f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e99fdb625581b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.100+0000 7f0bc1600700 1 -- 172.21.15.90:0/1713854778 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc402d070 con 0x55e2a968e9f0 2024-05-27T07:56:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.100+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a60 0x7f0bb00b4f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0bc402a000 tx=0x7f0bc402b000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.100+0000 7f0bc1600700 1 -- 172.21.15.90:0/1713854778 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc4003e30 con 0x55e2a968e9f0 2024-05-27T07:56:02.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.100+0000 7f0bc1600700 1 -- 172.21.15.90:0/1713854778 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4005630 con 0x55e2a968e9f0 2024-05-27T07:56:02.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.101+0000 7f0be14e0b80 1 -- 172.21.15.90:0/1713854778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a968e9f0 msgr2=0x55e2a9721330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:02.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.101+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a968e9f0 0x55e2a9721330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4015000 tx=0x7f0bc40091f0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.101+0000 7f0be14e0b80 1 -- 172.21.15.90:0/1713854778 shutdown_connections 2024-05-27T07:56:02.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.101+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a9721e70 0x55e2a972e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.101+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a968e9f0 0x55e2a9721330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.101+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/1713854778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a95a2e40 0x55e2a9721870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.102+0000 7f0be14e0b80 1 -- 172.21.15.90:0/1713854778 >> 172.21.15.90:0/1713854778 conn(0x55e2a963c390 msgr2=0x55e2a971d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:02.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.102+0000 7f0be14e0b80 1 -- 172.21.15.90:0/1713854778 shutdown_connections 2024-05-27T07:56:02.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.102+0000 7f0be14e0b80 1 -- 172.21.15.90:0/1713854778 wait complete. 2024-05-27T07:56:02.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.103+0000 7f0be14e0b80 1 -- start start 2024-05-27T07:56:02.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.103+0000 7f0be14e0b80 1 Processor -- start 2024-05-27T07:56:02.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.104+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a95a2e40 0x55e2a972b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.104+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a972bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.104+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 0x55e2a972c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.104+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e2a9545fa0 con 0x55e2a968e9f0 2024-05-27T07:56:02.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.104+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2a95e0760 con 0x55e2a95a2e40 2024-05-27T07:56:02.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.104+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e2a972eb80 con 0x55e2a9721e70 2024-05-27T07:56:02.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.104+0000 7f0be14e0b80 10 client.0 _ll_register_callbacks cb 0x55e2a95e9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:02.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.105+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a972bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.105+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a972bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46540/0 (socket says 172.21.15.90:46540) 2024-05-27T07:56:02.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.105+0000 7f0bcb400700 1 -- 172.21.15.90:0/2084308594 learned_addr learned my addr 172.21.15.90:0/2084308594 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:02.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.105+0000 7f0bcb400700 1 -- 172.21.15.90:0/2084308594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 msgr2=0x55e2a972c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:02.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.105+0000 7f0bcb400700 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 0x55e2a972c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.105+0000 7f0bcb400700 1 -- 172.21.15.90:0/2084308594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a95a2e40 msgr2=0x55e2a972b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:02.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.105+0000 7f0bcb400700 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a95a2e40 0x55e2a972b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.105+0000 7f0bcb400700 1 -- 172.21.15.90:0/2084308594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a958f8d0 con 0x55e2a968e9f0 2024-05-27T07:56:02.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.106+0000 7f0bcb400700 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a972bb40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8007ec0 tx=0x7f0bb80116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8a3f000a35f21f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:02.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.106+0000 7f0bb6000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:56:02.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.106+0000 7f0bb6000700 1 -- 172.21.15.90:0/2084308594 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8012450 con 0x55e2a968e9f0 2024-05-27T07:56:02.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.106+0000 7f0bb6000700 1 -- 172.21.15.90:0/2084308594 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f0bb80125f0 con 0x55e2a968e9f0 2024-05-27T07:56:02.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.106+0000 7f0be14e0b80 1 -- 172.21.15.90:0/2084308594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc4003e30 con 0x55e2a968e9f0 2024-05-27T07:56:02.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.106+0000 7f0bb6000700 1 -- 172.21.15.90:0/2084308594 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0bb80127f0 con 0x55e2a968e9f0 2024-05-27T07:56:02.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.106+0000 7f0be14e0b80 10 client.17241 fetch_fsmap learned FSMap version 60 2024-05-27T07:56:02.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.106+0000 7f0be14e0b80 1 -- 172.21.15.90:0/2084308594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55e2a9610060 con 0x55e2a968e9f0 2024-05-27T07:56:02.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.107+0000 7f0bb6000700 1 -- 172.21.15.90:0/2084308594 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc4003e30 con 0x55e2a968e9f0 2024-05-27T07:56:02.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.108+0000 7f0bb6000700 1 -- 172.21.15.90:0/2084308594 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 60) ==== 7152+0+0 (secure 0 0 0) 0x7f0bb8025b20 con 0x55e2a968e9f0 2024-05-27T07:56:02.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.108+0000 7f0be14e0b80 10 client.17241 fetch_fsmap finished waiting for FSMap version 60 2024-05-27T07:56:02.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.108+0000 7f0be14e0b80 10 client.17241 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:56:02.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.108+0000 7f0be14e0b80 20 client.17241 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:02.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.108+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e2a973bb30 0x55e2a973e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.108+0000 7f0be14e0b80 4 client.17241 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:02.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.108+0000 7f0be14e0b80 1 -- 172.21.15.90:0/2084308594 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a9651d90 con 0x55e2a973bb30 2024-05-27T07:56:02.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.110+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e2a973bb30 0x55e2a973e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.112+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e2a973bb30 0x55e2a973e000 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c32e431717c0bdb server_cookie=ab889ec6fec077aa in_seq=0 out_seq=0 2024-05-27T07:56:02.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.112+0000 7f0bb6000700 10 client.17241 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:56:02.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.116+0000 7f0bb6000700 1 -- 172.21.15.90:0/2084308594 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e2a9651d90 con 0x55e2a973bb30 2024-05-27T07:56:02.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.116+0000 7f0bb6000700 10 client.17241 handle_command_reply: tid=0 2024-05-27T07:56:02.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.117+0000 7f0be14e0b80 1 client.17241 shutdown 2024-05-27T07:56:02.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.118+0000 7f0be14e0b80 1 -- 172.21.15.90:0/2084308594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 msgr2=0x55e2a972bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:02.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.118+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a972bb40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8007ec0 tx=0x7f0bb80116c0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.121+0000 7f0be14e0b80 1 -- 172.21.15.90:0/2084308594 shutdown_connections 2024-05-27T07:56:02.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.121+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 0x55e2a972c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.121+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e2a973bb30 0x55e2a973e000 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.121+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a95a2e40 0x55e2a972b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.121+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/2084308594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a972bb40 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.121+0000 7f0be14e0b80 1 -- 172.21.15.90:0/2084308594 >> 172.21.15.90:0/2084308594 conn(0x55e2a963c390 msgr2=0x55e2a9721310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:02.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.121+0000 7f0bb6000700 0 client.17241 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:56:02.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.122+0000 7f0be14e0b80 1 -- 172.21.15.90:0/2084308594 shutdown_connections 2024-05-27T07:56:02.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.122+0000 7f0be14e0b80 1 -- 172.21.15.90:0/2084308594 wait complete. 2024-05-27T07:56:02.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.122+0000 7f0be14e0b80 20 client.17241 trim_cache size 0 max 16384 2024-05-27T07:56:02.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.157+0000 7f0be14e0b80 1 Processor -- start 2024-05-27T07:56:02.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.157+0000 7f0be14e0b80 1 -- start start 2024-05-27T07:56:02.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a95a2e40 0x55e2a972a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a968e9f0 0x55e2a9721870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 0x55e2a9726df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1bde00 con 0x55e2a95a2e40 2024-05-27T07:56:02.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1c06e0 con 0x55e2a968e9f0 2024-05-27T07:56:02.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1be4d0 con 0x55e2a9721e70 2024-05-27T07:56:02.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a968e9f0 0x55e2a9721870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0bd0a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 0x55e2a9726df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a968e9f0 0x55e2a9721870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38176/0 (socket says 172.21.15.90:38176) 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0bcb400700 1 -- 172.21.15.90:0/3127661189 learned_addr learned my addr 172.21.15.90:0/3127661189 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.158+0000 7f0bd0a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 0x55e2a9726df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33066/0 (socket says 172.21.15.90:33066) 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.159+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/3127661189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a95a2e40 0x55e2a972a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.159+0000 7f0bd0a00700 1 -- 172.21.15.90:0/3127661189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a968e9f0 msgr2=0x55e2a9721870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.159+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/3127661189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a968e9f0 0x55e2a9721870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.159+0000 7f0bd0a00700 1 -- 172.21.15.90:0/3127661189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a95a2e40 msgr2=0x55e2a972a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.159+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/3127661189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a95a2e40 0x55e2a972a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.159+0000 7f0bd0a00700 1 -- 172.21.15.90:0/3127661189 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a9729b70 con 0x55e2a9721e70 2024-05-27T07:56:02.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.160+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/3127661189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 0x55e2a9726df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009310 tx=0x7f0bc40099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d31e8c3928991c85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:02.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.160+0000 7f0bc0c00700 1 -- 172.21.15.90:0/3127661189 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc402d040 con 0x55e2a9721e70 2024-05-27T07:56:02.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.160+0000 7f0bc0c00700 1 -- 172.21.15.90:0/3127661189 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc4007d10 con 0x55e2a9721e70 2024-05-27T07:56:02.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.160+0000 7f0bc0c00700 1 -- 172.21.15.90:0/3127661189 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40059f0 con 0x55e2a9721e70 2024-05-27T07:56:02.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.162+0000 7f0be14e0b80 1 -- 172.21.15.90:0/3127661189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 msgr2=0x55e2a9726df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:02.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.162+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/3127661189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 0x55e2a9726df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009310 tx=0x7f0bc40099e0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.163+0000 7f0be14e0b80 1 -- 172.21.15.90:0/3127661189 shutdown_connections 2024-05-27T07:56:02.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.163+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/3127661189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a9721e70 0x55e2a9726df0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.163+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/3127661189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a95a2e40 0x55e2a972a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.163+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/3127661189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a968e9f0 0x55e2a9721870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.163+0000 7f0be14e0b80 1 -- 172.21.15.90:0/3127661189 >> 172.21.15.90:0/3127661189 conn(0x55e2a963c390 msgr2=0x55e2a9722ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:02.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.164+0000 7f0be14e0b80 1 -- 172.21.15.90:0/3127661189 shutdown_connections 2024-05-27T07:56:02.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.165+0000 7f0be14e0b80 1 -- 172.21.15.90:0/3127661189 wait complete. 2024-05-27T07:56:02.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.166+0000 7f0be14e0b80 1 -- start start 2024-05-27T07:56:02.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.166+0000 7f0be14e0b80 1 Processor -- start 2024-05-27T07:56:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.167+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a95a2e40 0x55e2a97254d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.167+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a9725a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.167+0000 7f0be14e0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a9721e70 0x55e2a9725f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.167+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1beba0 con 0x55e2a968e9f0 2024-05-27T07:56:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.167+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1c0010 con 0x55e2a9721e70 2024-05-27T07:56:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.167+0000 7f0be14e0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1bf270 con 0x55e2a95a2e40 2024-05-27T07:56:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.167+0000 7f0be14e0b80 10 client.0 _ll_register_callbacks cb 0x55e2a95e9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:02.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.168+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a9725a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.168+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a9725a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46548/0 (socket says 172.21.15.90:46548) 2024-05-27T07:56:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.168+0000 7f0bcb400700 1 -- 172.21.15.90:0/304960851 learned_addr learned my addr 172.21.15.90:0/304960851 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.168+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a9721e70 0x55e2a9725f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.169+0000 7f0bcb400700 1 -- 172.21.15.90:0/304960851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a95a2e40 msgr2=0x55e2a97254d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:56:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.169+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a95a2e40 0x55e2a97254d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.169+0000 7f0bcb400700 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a95a2e40 0x55e2a97254d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.169+0000 7f0bcb400700 1 -- 172.21.15.90:0/304960851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a9721e70 msgr2=0x55e2a9725f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:02.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.169+0000 7f0bcb400700 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a9721e70 0x55e2a9725f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.169+0000 7f0bcb400700 1 -- 172.21.15.90:0/304960851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a9733070 con 0x55e2a968e9f0 2024-05-27T07:56:02.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.169+0000 7f0bd0a00700 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a9721e70 0x55e2a9725f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:56:02.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.169+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a95a2e40 0x55e2a97254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:02.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.170+0000 7f0bcb400700 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a9725a10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8007ec0 tx=0x7f0bb8012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473ed18f3d661003 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:02.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.170+0000 7f0bb6000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:56:02.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.170+0000 7f0bb6000700 1 -- 172.21.15.90:0/304960851 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb802b040 con 0x55e2a968e9f0 2024-05-27T07:56:02.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.170+0000 7f0bb6000700 1 -- 172.21.15.90:0/304960851 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f0bb801b9f0 con 0x55e2a968e9f0 2024-05-27T07:56:02.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.171+0000 7f0be14e0b80 1 -- 172.21.15.90:0/304960851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc4007d10 con 0x55e2a968e9f0 2024-05-27T07:56:02.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.171+0000 7f0bb6000700 1 -- 172.21.15.90:0/304960851 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f0bb801bbf0 con 0x55e2a968e9f0 2024-05-27T07:56:02.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.171+0000 7f0be14e0b80 10 client.17247 fetch_fsmap learned FSMap version 60 2024-05-27T07:56:02.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.172+0000 7f0be14e0b80 1 -- 172.21.15.90:0/304960851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55e2a97342d0 con 0x55e2a968e9f0 2024-05-27T07:56:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.173+0000 7f0bb6000700 1 -- 172.21.15.90:0/304960851 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc4007d10 con 0x55e2a968e9f0 2024-05-27T07:56:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.173+0000 7f0bb6000700 1 -- 172.21.15.90:0/304960851 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 60) ==== 7152+0+0 (secure 0 0 0) 0x7f0bb80051c0 con 0x55e2a968e9f0 2024-05-27T07:56:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.173+0000 7f0be14e0b80 10 client.17247 fetch_fsmap finished waiting for FSMap version 60 2024-05-27T07:56:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.173+0000 7f0be14e0b80 10 client.17247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:56:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.173+0000 7f0be14e0b80 20 client.17247 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.173+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e2a97b76e0 0x55e2a97b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.173+0000 7f0be14e0b80 4 client.17247 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:56:02.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.173+0000 7f0be14e0b80 1 -- 172.21.15.90:0/304960851 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2a964b570 con 0x55e2a97b76e0 2024-05-27T07:56:02.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.174+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e2a97b76e0 0x55e2a97b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:02.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.175+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e2a97b76e0 0x55e2a97b9b00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b60b69fbf68b5ec4 server_cookie=b121f4a594042b39 in_seq=0 out_seq=0 2024-05-27T07:56:02.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0bb6000700 10 client.17247 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0bb6000700 1 -- 172.21.15.90:0/304960851 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2a964b570 con 0x55e2a97b76e0 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0bb6000700 10 client.17247 handle_command_reply: tid=0 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0be14e0b80 1 client.17247 shutdown 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0be14e0b80 1 -- 172.21.15.90:0/304960851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 msgr2=0x55e2a9725a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a9725a10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8007ec0 tx=0x7f0bb8012700 comp rx=0 tx=0).stop 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0be14e0b80 1 -- 172.21.15.90:0/304960851 shutdown_connections 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e2a97b76e0 0x55e2a97b9b00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2a9721e70 0x55e2a9725f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0bb6000700 0 client.17247 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e2a95a2e40 0x55e2a97254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0be14e0b80 1 --2- 172.21.15.90:0/304960851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e2a968e9f0 0x55e2a9725a10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.178+0000 7f0be14e0b80 1 -- 172.21.15.90:0/304960851 >> 172.21.15.90:0/304960851 conn(0x55e2a963c390 msgr2=0x55e2a972a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:02.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.179+0000 7f0be14e0b80 1 -- 172.21.15.90:0/304960851 shutdown_connections 2024-05-27T07:56:02.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.179+0000 7f0be14e0b80 1 -- 172.21.15.90:0/304960851 wait complete. 2024-05-27T07:56:02.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.179+0000 7f0be14e0b80 20 client.17247 trim_cache size 0 max 16384 2024-05-27T07:56:02.182 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:56:02.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 -- 172.21.15.90:0/1750887786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a60 msgr2=0x7f0bb00b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:02.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a60 0x7f0bb00b4f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0bc402a000 tx=0x7f0bc402b000 comp rx=0 tx=0).stop 2024-05-27T07:56:02.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 -- 172.21.15.90:0/1750887786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc10c7c0 msgr2=0x7f0bcc117690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:02.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc10c7c0 0x7f0bcc117690 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00d950 tx=0x7f0bbc00dc60 comp rx=0 tx=0).stop 2024-05-27T07:56:02.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 -- 172.21.15.90:0/1750887786 shutdown_connections 2024-05-27T07:56:02.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc110fa0 0x7f0bcc1ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc10c7c0 0x7f0bcc117690 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a60 0x7f0bb00b4f70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 --2- 172.21.15.90:0/1750887786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc110650 0x7f0bcc117c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:02.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 -- 172.21.15.90:0/1750887786 >> 172.21.15.90:0/1750887786 conn(0x7f0bcc06f980 msgr2=0x7f0bcc10fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:02.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.181+0000 7f0bd2a00700 1 -- 172.21.15.90:0/1750887786 shutdown_connections 2024-05-27T07:56:02.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:02.182+0000 7f0bd2a00700 1 -- 172.21.15.90:0/1750887786 wait complete. 2024-05-27T07:56:02.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:02 smithi090 ceph-mon[31429]: pgmap v304: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 541 op/s 2024-05-27T07:56:02.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:02 smithi129 ceph-mon[35254]: pgmap v304: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 541 op/s 2024-05-27T07:56:02.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:02 smithi136 ceph-mon[35324]: pgmap v304: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 541 op/s 2024-05-27T07:56:03.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 345.0 s, 523.7 tps, lat 216.487 ms stddev 219.451 2024-05-27T07:56:04.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:04 smithi129 ceph-mon[35254]: pgmap v305: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.5 MiB/s wr, 558 op/s 2024-05-27T07:56:04.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:04 smithi090 ceph-mon[31429]: pgmap v305: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.5 MiB/s wr, 558 op/s 2024-05-27T07:56:04.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:04 smithi136 ceph-mon[35324]: pgmap v305: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.5 MiB/s wr, 558 op/s 2024-05-27T07:56:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.393+0000 7f1940c00640 1 -- 172.21.15.90:0/2063751582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f193c109030 msgr2=0x7f193c102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.393+0000 7f1940c00640 1 --2- 172.21.15.90:0/2063751582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f193c109030 0x7f193c102c70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f192402a410 comp rx=0 tx=0).stop 2024-05-27T07:56:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.393+0000 7f1940c00640 1 -- 172.21.15.90:0/2063751582 shutdown_connections 2024-05-27T07:56:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.393+0000 7f1940c00640 1 --2- 172.21.15.90:0/2063751582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f193c1032e0 0x7f193c10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.393+0000 7f1940c00640 1 --2- 172.21.15.90:0/2063751582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f193c109030 0x7f193c102c70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.393+0000 7f1940c00640 1 --2- 172.21.15.90:0/2063751582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f193c108b90 0x7f193c102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.393+0000 7f1940c00640 1 -- 172.21.15.90:0/2063751582 >> 172.21.15.90:0/2063751582 conn(0x7f193c0782d0 msgr2=0x7f193c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.393+0000 7f1940c00640 1 -- 172.21.15.90:0/2063751582 shutdown_connections 2024-05-27T07:56:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.394+0000 7f1940c00640 1 -- 172.21.15.90:0/2063751582 wait complete. 2024-05-27T07:56:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.397+0000 7f1940c00640 1 Processor -- start 2024-05-27T07:56:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.397+0000 7f1940c00640 1 -- start start 2024-05-27T07:56:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.398+0000 7f1940c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f193c1032e0 0x7f193c1a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.398+0000 7f1940c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f193c108b90 0x7f193c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.398+0000 7f1940c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f193c109030 0x7f193c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.398+0000 7f1940c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f193c112c80 con 0x7f193c109030 2024-05-27T07:56:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.398+0000 7f1940c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f193c112b00 con 0x7f193c1032e0 2024-05-27T07:56:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.398+0000 7f1940c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f193c112e00 con 0x7f193c108b90 2024-05-27T07:56:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.399+0000 7f1939e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f193c1032e0 0x7f193c1a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.400+0000 7f1939e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f193c1032e0 0x7f193c1a0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38198/0 (socket says 172.21.15.90:38198) 2024-05-27T07:56:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.400+0000 7f1939e00640 1 -- 172.21.15.90:0/3154964919 learned_addr learned my addr 172.21.15.90:0/3154964919 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.400+0000 7f1933e00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f193c108b90 0x7f193c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.400+0000 7f193a800640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f193c109030 0x7f193c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.401+0000 7f1939e00640 1 -- 172.21.15.90:0/3154964919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f193c108b90 msgr2=0x7f193c1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.402+0000 7f1939e00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f193c108b90 0x7f193c1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.402+0000 7f1939e00640 1 -- 172.21.15.90:0/3154964919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f193c109030 msgr2=0x7f193c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.402+0000 7f1939e00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f193c109030 0x7f193c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.402+0000 7f1939e00640 1 -- 172.21.15.90:0/3154964919 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c19ae20 con 0x7f193c1032e0 2024-05-27T07:56:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.402+0000 7f1933e00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f193c108b90 0x7f193c1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:56:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.402+0000 7f1939e00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f193c1032e0 0x7f193c1a0520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f192c005fd0 tx=0x7f192c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.405+0000 7f1932a00640 1 -- 172.21.15.90:0/3154964919 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c017070 con 0x7f193c1032e0 2024-05-27T07:56:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.405+0000 7f1932a00640 1 -- 172.21.15.90:0/3154964919 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f192c009c30 con 0x7f193c1032e0 2024-05-27T07:56:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.405+0000 7f1932a00640 1 -- 172.21.15.90:0/3154964919 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c003cb0 con 0x7f193c1032e0 2024-05-27T07:56:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.405+0000 7f1940c00640 1 -- 172.21.15.90:0/3154964919 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c19b0b0 con 0x7f193c1032e0 2024-05-27T07:56:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.405+0000 7f1940c00640 1 -- 172.21.15.90:0/3154964919 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c10d6a0 con 0x7f193c1032e0 2024-05-27T07:56:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.410+0000 7f1947ba5740 1 Processor -- start 2024-05-27T07:56:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.410+0000 7f1947ba5740 1 -- start start 2024-05-27T07:56:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.411+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 0x55975e9fd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.411+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975e9fdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.411+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975e9fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.411+0000 7f1947ba5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55975e96eb90 con 0x55975e63a4d0 2024-05-27T07:56:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.411+0000 7f1947ba5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55975e95e730 con 0x55975e6322f0 2024-05-27T07:56:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.411+0000 7f1947ba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55975e962770 con 0x55975e65b1b0 2024-05-27T07:56:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.412+0000 7f1933e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975e9fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.412+0000 7f1933e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975e9fe1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46574/0 (socket says 172.21.15.90:46574) 2024-05-27T07:56:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.412+0000 7f1933e00640 1 -- 172.21.15.90:0/1397177608 learned_addr learned my addr 172.21.15.90:0/1397177608 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.412+0000 7f193a800640 1 --2- 172.21.15.90:0/1397177608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 0x55975e9fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.412+0000 7f1933e00640 1 -- 172.21.15.90:0/1397177608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 msgr2=0x55975e9fd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.412+0000 7f1933e00640 1 --2- 172.21.15.90:0/1397177608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 0x55975e9fd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.412+0000 7f1933e00640 1 -- 172.21.15.90:0/1397177608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 msgr2=0x55975e9fdc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:56:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.412+0000 7f1933e00640 1 --2- 172.21.15.90:0/1397177608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975e9fdc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.412+0000 7f1933e00640 1 -- 172.21.15.90:0/1397177608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975e8f78b0 con 0x55975e63a4d0 2024-05-27T07:56:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.413+0000 7f1933e00640 1 --2- 172.21.15.90:0/1397177608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975e9fe1b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f19240057c0 tx=0x7f1924003670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6f38d2d83bca61 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.415+0000 7f192b000640 1 -- 172.21.15.90:0/1397177608 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924035070 con 0x55975e63a4d0 2024-05-27T07:56:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.415+0000 7f192b000640 1 -- 172.21.15.90:0/1397177608 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1924040030 con 0x55975e63a4d0 2024-05-27T07:56:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.415+0000 7f192b000640 1 -- 172.21.15.90:0/1397177608 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192403cdd0 con 0x55975e63a4d0 2024-05-27T07:56:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.415+0000 7f1932a00640 1 -- 172.21.15.90:0/3154964919 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f192c004020 con 0x7f193c1032e0 2024-05-27T07:56:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.416+0000 7f1947ba5740 1 -- 172.21.15.90:0/1397177608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 msgr2=0x55975e9fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.416+0000 7f1947ba5740 1 --2- 172.21.15.90:0/1397177608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975e9fe1b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f19240057c0 tx=0x7f1924003670 comp rx=0 tx=0).stop 2024-05-27T07:56:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.417+0000 7f1947ba5740 1 -- 172.21.15.90:0/1397177608 shutdown_connections 2024-05-27T07:56:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.417+0000 7f1947ba5740 1 --2- 172.21.15.90:0/1397177608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975e9fdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.417+0000 7f1947ba5740 1 --2- 172.21.15.90:0/1397177608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975e9fe1b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.417+0000 7f1947ba5740 1 --2- 172.21.15.90:0/1397177608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 0x55975e9fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.417+0000 7f1947ba5740 1 -- 172.21.15.90:0/1397177608 >> 172.21.15.90:0/1397177608 conn(0x55975e64f300 msgr2=0x55975e9fa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.417+0000 7f1947ba5740 1 -- 172.21.15.90:0/1397177608 shutdown_connections 2024-05-27T07:56:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.418+0000 7f1947ba5740 1 -- 172.21.15.90:0/1397177608 wait complete. 2024-05-27T07:56:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.418+0000 7f1947ba5740 1 -- start start 2024-05-27T07:56:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.419+0000 7f1947ba5740 1 Processor -- start 2024-05-27T07:56:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.420+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea10270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.420+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 0x55975ea107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.420+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 0x55975ea1c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.420+0000 7f1947ba5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55975e644790 con 0x55975e65b1b0 2024-05-27T07:56:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.420+0000 7f1947ba5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55975e906bc0 con 0x55975e6322f0 2024-05-27T07:56:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.420+0000 7f1947ba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55975ea131f0 con 0x55975e63a4d0 2024-05-27T07:56:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.420+0000 7f1947ba5740 10 client.0 _ll_register_callbacks cb 0x55975e8e4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.420+0000 7f1939e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 0x55975ea107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.420+0000 7f1939e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 0x55975ea107b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33120/0 (socket says 172.21.15.90:33120) 2024-05-27T07:56:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.420+0000 7f1939e00640 1 -- 172.21.15.90:0/2888080524 learned_addr learned my addr 172.21.15.90:0/2888080524 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.421+0000 7f193a800640 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea10270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.421+0000 7f1933e00640 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 0x55975ea1c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.422+0000 7f1932a00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f19200b2a30 0x7f19200b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.422+0000 7f1939e00640 1 -- 172.21.15.90:0/2888080524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 msgr2=0x55975ea10270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.422+0000 7f1939e00640 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea10270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.422+0000 7f1939e00640 1 -- 172.21.15.90:0/2888080524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 msgr2=0x55975ea1c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.422+0000 7f1932a00640 1 -- 172.21.15.90:0/3154964919 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f192c0e2300 con 0x7f193c1032e0 2024-05-27T07:56:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.422+0000 7f1939e00640 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 0x55975ea1c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.422+0000 7f1939e00640 1 -- 172.21.15.90:0/2888080524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975e673210 con 0x55975e63a4d0 2024-05-27T07:56:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.422+0000 7f1933e00640 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 0x55975ea1c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.423+0000 7f193a800640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f19200b2a30 0x7f19200b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.423+0000 7f1939e00640 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 0x55975ea107b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f192c0df590 tx=0x7f192c076e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=437aa5a344391fe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.423+0000 7f191f400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:56:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.423+0000 7f191f400640 1 -- 172.21.15.90:0/2888080524 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c0a5070 con 0x55975e63a4d0 2024-05-27T07:56:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.423+0000 7f1947ba5740 1 -- 172.21.15.90:0/2888080524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1924040030 con 0x55975e63a4d0 2024-05-27T07:56:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.423+0000 7f191f400640 1 -- 172.21.15.90:0/2888080524 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f192c004310 con 0x55975e63a4d0 2024-05-27T07:56:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.424+0000 7f191f400640 1 -- 172.21.15.90:0/2888080524 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f192c0e5d40 con 0x55975e63a4d0 2024-05-27T07:56:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.424+0000 7f193a800640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f19200b2a30 0x7f19200b4f40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55975ea0fad0 tx=0x7f1934004ab0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.424+0000 7f1947ba5740 10 client.26887 fetch_fsmap learned FSMap version 60 2024-05-27T07:56:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.424+0000 7f1947ba5740 1 -- 172.21.15.90:0/2888080524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55975e73f620 con 0x55975e63a4d0 2024-05-27T07:56:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.425+0000 7f191f400640 1 -- 172.21.15.90:0/2888080524 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1924040030 con 0x55975e63a4d0 2024-05-27T07:56:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.426+0000 7f191f400640 1 -- 172.21.15.90:0/2888080524 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 60) ==== 7152+0+0 (secure 0 0 0) 0x7f192c0df8a0 con 0x55975e63a4d0 2024-05-27T07:56:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.427+0000 7f1947ba5740 10 client.26887 fetch_fsmap finished waiting for FSMap version 60 2024-05-27T07:56:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.427+0000 7f1947ba5740 10 client.26887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.427+0000 7f1947ba5740 20 client.26887 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.427+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55975e619d80 0x55975ea22500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.427+0000 7f1947ba5740 4 client.26887 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.427+0000 7f1947ba5740 1 -- 172.21.15.90:0/2888080524 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55975e970ae0 con 0x55975e619d80 2024-05-27T07:56:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.427+0000 7f1933e00640 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55975e619d80 0x55975ea22500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.428+0000 7f191f400640 10 client.26887 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.428+0000 7f1933e00640 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55975e619d80 0x55975ea22500 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80cfe7a49fa472b9 server_cookie=898836f37580d581 in_seq=0 out_seq=0 2024-05-27T07:56:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.429+0000 7f191f400640 1 -- 172.21.15.90:0/2888080524 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55975e970ae0 con 0x55975e619d80 2024-05-27T07:56:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.429+0000 7f191f400640 10 client.26887 handle_command_reply: tid=0 2024-05-27T07:56:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.429+0000 7f1947ba5740 1 client.26887 shutdown 2024-05-27T07:56:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 1 -- 172.21.15.90:0/2888080524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 msgr2=0x55975ea107b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 0x55975ea107b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f192c0df590 tx=0x7f192c076e00 comp rx=0 tx=0).stop 2024-05-27T07:56:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 1 -- 172.21.15.90:0/2888080524 shutdown_connections 2024-05-27T07:56:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55975e619d80 0x55975ea22500 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 0x55975ea1c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 0x55975ea107b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2888080524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea10270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 1 -- 172.21.15.90:0/2888080524 >> 172.21.15.90:0/2888080524 conn(0x55975e64f300 msgr2=0x55975e9fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f191f400640 0 client.26887 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 1 -- 172.21.15.90:0/2888080524 shutdown_connections 2024-05-27T07:56:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 1 -- 172.21.15.90:0/2888080524 wait complete. 2024-05-27T07:56:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.430+0000 7f1947ba5740 20 client.26887 trim_cache size 0 max 16384 2024-05-27T07:56:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.456+0000 7f1947ba5740 1 Processor -- start 2024-05-27T07:56:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.457+0000 7f1947ba5740 1 -- start start 2024-05-27T07:56:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.457+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea0baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.457+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 0x55975ea09ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.457+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 0x55975ea0a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.457+0000 7f1947ba5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1adb20 con 0x55975e65b1b0 2024-05-27T07:56:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.457+0000 7f1947ba5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1b0400 con 0x55975e6322f0 2024-05-27T07:56:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.457+0000 7f1947ba5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1ae1f0 con 0x55975e63a4d0 2024-05-27T07:56:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.458+0000 7f1939e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea0baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.458+0000 7f1939e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea0baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38226/0 (socket says 172.21.15.90:38226) 2024-05-27T07:56:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.458+0000 7f1939e00640 1 -- 172.21.15.90:0/3854639799 learned_addr learned my addr 172.21.15.90:0/3854639799 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.459+0000 7f193a800640 1 --2- 172.21.15.90:0/3854639799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 0x55975ea0a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.460+0000 7f1933e00640 1 --2- 172.21.15.90:0/3854639799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 0x55975ea09ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.462+0000 7f1939e00640 1 -- 172.21.15.90:0/3854639799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 msgr2=0x55975ea09ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.462+0000 7f1939e00640 1 --2- 172.21.15.90:0/3854639799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 0x55975ea09ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.462+0000 7f1939e00640 1 -- 172.21.15.90:0/3854639799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 msgr2=0x55975ea0a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.462+0000 7f1939e00640 1 --2- 172.21.15.90:0/3854639799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 0x55975ea0a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.462+0000 7f1939e00640 1 -- 172.21.15.90:0/3854639799 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975ea11180 con 0x55975e6322f0 2024-05-27T07:56:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.463+0000 7f1939e00640 1 --2- 172.21.15.90:0/3854639799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea0baa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f192c000c00 tx=0x7f192c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57998ec666743955 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.463+0000 7f192a600640 1 -- 172.21.15.90:0/3854639799 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c0a8920 con 0x55975e6322f0 2024-05-27T07:56:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.464+0000 7f192a600640 1 -- 172.21.15.90:0/3854639799 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f192c0035f0 con 0x55975e6322f0 2024-05-27T07:56:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.464+0000 7f192a600640 1 -- 172.21.15.90:0/3854639799 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c0a8b00 con 0x55975e6322f0 2024-05-27T07:56:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.465+0000 7f1947ba5740 1 -- 172.21.15.90:0/3854639799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 msgr2=0x55975ea0baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.465+0000 7f1947ba5740 1 --2- 172.21.15.90:0/3854639799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea0baa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f192c000c00 tx=0x7f192c005d30 comp rx=0 tx=0).stop 2024-05-27T07:56:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.465+0000 7f1947ba5740 1 -- 172.21.15.90:0/3854639799 shutdown_connections 2024-05-27T07:56:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.465+0000 7f1947ba5740 1 --2- 172.21.15.90:0/3854639799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e65b1b0 0x55975ea0a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.465+0000 7f1947ba5740 1 --2- 172.21.15.90:0/3854639799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e63a4d0 0x55975ea09ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.465+0000 7f1947ba5740 1 --2- 172.21.15.90:0/3854639799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea0baa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.465+0000 7f1947ba5740 1 -- 172.21.15.90:0/3854639799 >> 172.21.15.90:0/3854639799 conn(0x55975e64f300 msgr2=0x55975ea10530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.465+0000 7f1947ba5740 1 -- 172.21.15.90:0/3854639799 shutdown_connections 2024-05-27T07:56:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.466+0000 7f1947ba5740 1 -- 172.21.15.90:0/3854639799 wait complete. 2024-05-27T07:56:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.467+0000 7f1947ba5740 1 -- start start 2024-05-27T07:56:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.467+0000 7f1947ba5740 1 Processor -- start 2024-05-27T07:56:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.468+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea86550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.469+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975ea848d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.469+0000 7f1947ba5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 0x55975ea84e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.469+0000 7f1947ba5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1ae8c0 con 0x55975e63a4d0 2024-05-27T07:56:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.469+0000 7f1947ba5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1afd30 con 0x55975e6322f0 2024-05-27T07:56:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.469+0000 7f193a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 0x55975ea84e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.469+0000 7f193a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 0x55975ea84e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33132/0 (socket says 172.21.15.90:33132) 2024-05-27T07:56:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.469+0000 7f193a800640 1 -- 172.21.15.90:0/2056289844 learned_addr learned my addr 172.21.15.90:0/2056289844 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.470+0000 7f1947ba5740 1 -- 172.21.15.90:0/2056289844 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f193c1aef90 con 0x55975e65b1b0 2024-05-27T07:56:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.470+0000 7f1939e00640 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea86550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.470+0000 7f1933e00640 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975ea848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.470+0000 7f1947ba5740 10 client.0 _ll_register_callbacks cb 0x7f192c0e5b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.471+0000 7f1933e00640 1 -- 172.21.15.90:0/2056289844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 msgr2=0x55975ea84e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.471+0000 7f1933e00640 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 0x55975ea84e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.471+0000 7f1933e00640 1 -- 172.21.15.90:0/2056289844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 msgr2=0x55975ea86550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.471+0000 7f1933e00640 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea86550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.471+0000 7f1933e00640 1 -- 172.21.15.90:0/2056289844 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975ea85670 con 0x55975e63a4d0 2024-05-27T07:56:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.471+0000 7f193a800640 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 0x55975ea84e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.472+0000 7f1933e00640 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975ea848d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f192402aa00 tx=0x7f192403cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d66bd69d2a499b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.472+0000 7f191f400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:56:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.473+0000 7f191f400640 1 -- 172.21.15.90:0/2056289844 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192403de60 con 0x55975e63a4d0 2024-05-27T07:56:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.473+0000 7f191f400640 1 -- 172.21.15.90:0/2056289844 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f192403c640 con 0x55975e63a4d0 2024-05-27T07:56:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.473+0000 7f1947ba5740 1 -- 172.21.15.90:0/2056289844 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55975e96a2c0 con 0x55975e63a4d0 2024-05-27T07:56:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.473+0000 7f191f400640 1 -- 172.21.15.90:0/2056289844 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f1924061050 con 0x55975e63a4d0 2024-05-27T07:56:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.473+0000 7f1947ba5740 10 client.17265 fetch_fsmap learned FSMap version 60 2024-05-27T07:56:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.474+0000 7f1947ba5740 1 -- 172.21.15.90:0/2056289844 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55975ea184d0 con 0x55975e63a4d0 2024-05-27T07:56:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.474+0000 7f191f400640 1 -- 172.21.15.90:0/2056289844 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1924003db0 con 0x55975e63a4d0 2024-05-27T07:56:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.474+0000 7f191f400640 1 -- 172.21.15.90:0/2056289844 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 60) ==== 7152+0+0 (secure 0 0 0) 0x7f19240537b0 con 0x55975e63a4d0 2024-05-27T07:56:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.475+0000 7f1947ba5740 10 client.17265 fetch_fsmap finished waiting for FSMap version 60 2024-05-27T07:56:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.475+0000 7f1947ba5740 10 client.17265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.475+0000 7f1947ba5740 20 client.17265 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.475+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55975e5ef1c0 0x55975ea189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.475+0000 7f1947ba5740 4 client.17265 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:56:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.475+0000 7f1947ba5740 1 -- 172.21.15.90:0/2056289844 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55975e96a2c0 con 0x55975e5ef1c0 2024-05-27T07:56:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.476+0000 7f1939e00640 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55975e5ef1c0 0x55975ea189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.477+0000 7f1939e00640 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55975e5ef1c0 0x55975ea189d0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f09d1cd8f3312aac server_cookie=2be10f0e59c37600 in_seq=0 out_seq=0 2024-05-27T07:56:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.477+0000 7f191f400640 10 client.17265 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.522+0000 7f191f400640 1 -- 172.21.15.90:0/2056289844 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55975e96a2c0 con 0x55975e5ef1c0 2024-05-27T07:56:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.522+0000 7f191f400640 10 client.17265 handle_command_reply: tid=0 2024-05-27T07:56:06.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.522+0000 7f1947ba5740 1 client.17265 shutdown 2024-05-27T07:56:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.524+0000 7f1947ba5740 1 -- 172.21.15.90:0/2056289844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 msgr2=0x55975ea848d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.524+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975ea848d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f192402aa00 tx=0x7f192403cc50 comp rx=0 tx=0).stop 2024-05-27T07:56:06.528 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.524+0000 7f1947ba5740 1 -- 172.21.15.90:0/2056289844 shutdown_connections 2024-05-27T07:56:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.524+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55975e5ef1c0 0x55975ea189d0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.524+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55975e65b1b0 0x55975ea84e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.524+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55975e63a4d0 0x55975ea848d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.524+0000 7f1947ba5740 1 --2- 172.21.15.90:0/2056289844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55975e6322f0 0x55975ea86550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.524+0000 7f1947ba5740 1 -- 172.21.15.90:0/2056289844 >> 172.21.15.90:0/2056289844 conn(0x55975e64f300 msgr2=0x55975ea10290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.525+0000 7f191f400640 0 client.17265 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.527+0000 7f1947ba5740 1 -- 172.21.15.90:0/2056289844 shutdown_connections 2024-05-27T07:56:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.527+0000 7f1947ba5740 1 -- 172.21.15.90:0/2056289844 wait complete. 2024-05-27T07:56:06.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.527+0000 7f1947ba5740 20 client.17265 trim_cache size 0 max 16384 2024-05-27T07:56:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:56:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:56:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:56:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.533+0000 7f1940c00640 1 -- 172.21.15.90:0/3154964919 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f19200b2a30 msgr2=0x7f19200b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.534+0000 7f1940c00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f19200b2a30 0x7f19200b4f40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55975ea0fad0 tx=0x7f1934004ab0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.534+0000 7f1940c00640 1 -- 172.21.15.90:0/3154964919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f193c1032e0 msgr2=0x7f193c1a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.534+0000 7f1940c00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f193c1032e0 0x7f193c1a0520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f192c005fd0 tx=0x7f192c00dde0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.535+0000 7f1940c00640 1 -- 172.21.15.90:0/3154964919 shutdown_connections 2024-05-27T07:56:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.535+0000 7f1940c00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f19200b2a30 0x7f19200b4f40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.535+0000 7f1940c00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f193c109030 0x7f193c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.535+0000 7f1940c00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f193c108b90 0x7f193c1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.535+0000 7f1940c00640 1 --2- 172.21.15.90:0/3154964919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f193c1032e0 0x7f193c1a0520 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.535+0000 7f1940c00640 1 -- 172.21.15.90:0/3154964919 >> 172.21.15.90:0/3154964919 conn(0x7f193c0782d0 msgr2=0x7f193c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.535+0000 7f1940c00640 1 -- 172.21.15.90:0/3154964919 shutdown_connections 2024-05-27T07:56:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:06.535+0000 7f1940c00640 1 -- 172.21.15.90:0/3154964919 wait complete. 2024-05-27T07:56:06.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:06 smithi129 ceph-mon[35254]: pgmap v306: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.2 MiB/s wr, 659 op/s 2024-05-27T07:56:06.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:06 smithi090 ceph-mon[31429]: pgmap v306: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.2 MiB/s wr, 659 op/s 2024-05-27T07:56:06.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:06 smithi136 ceph-mon[35324]: pgmap v306: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 6.2 MiB/s wr, 659 op/s 2024-05-27T07:56:07.349 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:56:08.118 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:08.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:08 smithi129 ceph-mon[35254]: pgmap v307: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.2 MiB/s wr, 452 op/s 2024-05-27T07:56:08.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:08 smithi090 ceph-mon[31429]: pgmap v307: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.2 MiB/s wr, 452 op/s 2024-05-27T07:56:08.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:08 smithi136 ceph-mon[35324]: pgmap v307: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 4.2 MiB/s wr, 452 op/s 2024-05-27T07:56:08.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 350.0 s, 520.8 tps, lat 188.648 ms stddev 101.196 2024-05-27T07:56:10.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:10 smithi090 ceph-mon[31429]: pgmap v308: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.0 MiB/s wr, 648 op/s 2024-05-27T07:56:10.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:10 smithi090 ceph-mon[31429]: osdmap e107: 12 total, 12 up, 12 in 2024-05-27T07:56:10.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:10 smithi136 ceph-mon[35324]: pgmap v308: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.0 MiB/s wr, 648 op/s 2024-05-27T07:56:10.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:10 smithi136 ceph-mon[35324]: osdmap e107: 12 total, 12 up, 12 in 2024-05-27T07:56:10.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:10 smithi129 ceph-mon[35254]: pgmap v308: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.0 MiB/s wr, 648 op/s 2024-05-27T07:56:10.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:10 smithi129 ceph-mon[35254]: osdmap e107: 12 total, 12 up, 12 in 2024-05-27T07:56:11.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:11 smithi090 ceph-mon[31429]: pgmap v310: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.9 MiB/s wr, 535 op/s 2024-05-27T07:56:11.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:11 smithi136 ceph-mon[35324]: pgmap v310: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.9 MiB/s wr, 535 op/s 2024-05-27T07:56:11.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:11 smithi129 ceph-mon[35254]: pgmap v310: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.9 MiB/s wr, 535 op/s 2024-05-27T07:56:12.307 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:11 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 5. 2024-05-27T07:56:12.307 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:11 smithi090 systemd[1]: Stopped Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:56:12.307 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:11 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.466s CPU time. 2024-05-27T07:56:12.307 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:11 smithi090 systemd[1]: Starting Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T07:56:12.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:12 smithi090 ceph-mon[31429]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:56:12.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:12 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-05-27T07:56:12.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:12 smithi136 ceph-mon[35324]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:56:12.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:12 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-05-27T07:56:12.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:12 smithi129 ceph-mon[35254]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:56:12.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:12 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-05-27T07:56:13.430 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:13 smithi090 ceph-mon[31429]: pgmap v311: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.5 MiB/s wr, 526 op/s 2024-05-27T07:56:13.714 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:13 smithi090 podman[114848]: 2024-05-27 07:56:13.351335615 +0000 UTC m=+0.312173690 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T07:56:13.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:13 smithi136 ceph-mon[35324]: pgmap v311: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.5 MiB/s wr, 526 op/s 2024-05-27T07:56:13.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:13 smithi129 ceph-mon[35254]: pgmap v311: 129 pgs: 2 active+clean+snaptrim, 122 active+clean, 5 active+clean+snaptrim_wait; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.5 MiB/s wr, 526 op/s 2024-05-27T07:56:13.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 355.0 s, 442.0 tps, lat 206.576 ms stddev 164.395 2024-05-27T07:56:14.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:14 smithi090 podman[114848]: 2024-05-27 07:56:14.090659212 +0000 UTC m=+1.051497269 container create 3d2a19f9d09934938d702e5ce11225f02175a67a99574b98a2acd2f7f695f299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux ) 2024-05-27T07:56:15.910 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:56:15.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.701+0000 7fd072400700 1 -- 172.21.15.90:0/244385380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd06c109670 msgr2=0x7fd06c110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:15.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.701+0000 7fd072400700 1 --2- 172.21.15.90:0/244385380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd06c109670 0x7fd06c110fd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd0580099c0 tx=0x7fd058009cd0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.704+0000 7fd072400700 1 -- 172.21.15.90:0/244385380 shutdown_connections 2024-05-27T07:56:15.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.704+0000 7fd072400700 1 --2- 172.21.15.90:0/244385380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd06c109670 0x7fd06c110fd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.704+0000 7fd072400700 1 --2- 172.21.15.90:0/244385380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd06c1043f0 0x7fd06c104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.704+0000 7fd072400700 1 --2- 172.21.15.90:0/244385380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd06c108d20 0x7fd06c109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.704+0000 7fd072400700 1 -- 172.21.15.90:0/244385380 >> 172.21.15.90:0/244385380 conn(0x7fd06c079d00 msgr2=0x7fd06c07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.706+0000 7fd072400700 1 -- 172.21.15.90:0/244385380 shutdown_connections 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.706+0000 7fd072400700 1 -- 172.21.15.90:0/244385380 wait complete. 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.709+0000 7fd072400700 1 Processor -- start 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.710+0000 7fd072400700 1 -- start start 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.711+0000 7fd072400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd06c1043f0 0x7fd06c075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.711+0000 7fd072400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd06c108d20 0x7fd06c075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.711+0000 7fd072400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd06c109670 0x7fd06c076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.711+0000 7fd072400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c114220 con 0x7fd06c109670 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.711+0000 7fd072400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1140a0 con 0x7fd06c108d20 2024-05-27T07:56:15.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.711+0000 7fd072400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1143a0 con 0x7fd06c1043f0 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.712+0000 7fd06be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd06c108d20 0x7fd06c075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.712+0000 7fd071a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd06c109670 0x7fd06c076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.712+0000 7fd071000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd06c1043f0 0x7fd06c075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.712+0000 7fd071000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd06c1043f0 0x7fd06c075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37512/0 (socket says 172.21.15.90:37512) 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.712+0000 7fd071000700 1 -- 172.21.15.90:0/560544129 learned_addr learned my addr 172.21.15.90:0/560544129 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.712+0000 7fd071a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd06c109670 0x7fd06c076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49446/0 (socket says 172.21.15.90:49446) 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.713+0000 7fd071000700 1 -- 172.21.15.90:0/560544129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd06c108d20 msgr2=0x7fd06c075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.713+0000 7fd071000700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd06c108d20 0x7fd06c075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.713+0000 7fd071000700 1 -- 172.21.15.90:0/560544129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd06c109670 msgr2=0x7fd06c076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.713+0000 7fd071000700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd06c109670 0x7fd06c076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.713+0000 7fd071000700 1 -- 172.21.15.90:0/560544129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c1a8f90 con 0x7fd06c1043f0 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.713+0000 7fd071a00700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd06c109670 0x7fd06c076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:15.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.713+0000 7fd071000700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd06c1043f0 0x7fd06c075ab0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd06400ba50 tx=0x7fd06400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.714+0000 7fd069600700 1 -- 172.21.15.90:0/560544129 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064019070 con 0x7fd06c1043f0 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.714+0000 7fd069600700 1 -- 172.21.15.90:0/560544129 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd064014440 con 0x7fd06c1043f0 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.714+0000 7fd069600700 1 -- 172.21.15.90:0/560544129 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064019070 con 0x7fd06c1043f0 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.715+0000 7fd072400700 1 -- 172.21.15.90:0/560544129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06c1a9280 con 0x7fd06c1043f0 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.715+0000 7fd072400700 1 -- 172.21.15.90:0/560544129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06c1a9770 con 0x7fd06c1043f0 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.722+0000 7fd0824aab80 1 Processor -- start 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.722+0000 7fd0824aab80 1 -- start start 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.723+0000 7fd069600700 1 -- 172.21.15.90:0/560544129 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd06401c070 con 0x7fd06c1043f0 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.724+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf80c8e40 0x561cf8246ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.724+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8247410 0x561cf8251ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.724+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf8254020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.724+0000 7fd0824aab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561cf8175e40 con 0x561cf8247410 2024-05-27T07:56:15.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.724+0000 7fd0824aab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561cf81659e0 con 0x561cf80c8e40 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.724+0000 7fd071a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf80c8e40 0x561cf8246ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.725+0000 7fd071a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf80c8e40 0x561cf8246ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:34604/0 (socket says 172.21.15.90:34604) 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.725+0000 7fd071a00700 1 -- 172.21.15.90:0/1551076571 learned_addr learned my addr 172.21.15.90:0/1551076571 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.725+0000 7fd0824aab80 1 -- 172.21.15.90:0/1551076571 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cf8169a20 con 0x561cf8247820 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.726+0000 7fd071a00700 1 -- 172.21.15.90:0/1551076571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 msgr2=0x561cf8254020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.726+0000 7fd071a00700 1 --2- 172.21.15.90:0/1551076571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf8254020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.726+0000 7fd069600700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd0500b2cf0 0x7fd0500b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.726+0000 7fd071a00700 1 -- 172.21.15.90:0/1551076571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8247410 msgr2=0x561cf8251ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.726+0000 7fd071a00700 1 --2- 172.21.15.90:0/1551076571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8247410 0x561cf8251ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.727+0000 7fd071a00700 1 -- 172.21.15.90:0/1551076571 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf812e420 con 0x561cf80c8e40 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.727+0000 7fd069600700 1 -- 172.21.15.90:0/560544129 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7fd0640e1970 con 0x7fd06c1043f0 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.727+0000 7fd071a00700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd0500b2cf0 0x7fd0500b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.728+0000 7fd071a00700 1 --2- 172.21.15.90:0/1551076571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf80c8e40 0x561cf8246ed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd058015000 tx=0x7fd058003700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6673a4c0caf8cda server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.729+0000 7fd062000700 1 -- 172.21.15.90:0/1551076571 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058003db0 con 0x561cf80c8e40 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.729+0000 7fd062000700 1 -- 172.21.15.90:0/1551076571 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd058005500 con 0x561cf80c8e40 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.729+0000 7fd062000700 1 -- 172.21.15.90:0/1551076571 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800fc50 con 0x561cf80c8e40 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.729+0000 7fd071a00700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd0500b2cf0 0x7fd0500b5200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd058029000 tx=0x7fd05802c000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.730+0000 7fd0824aab80 1 -- 172.21.15.90:0/1551076571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf80c8e40 msgr2=0x561cf8246ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.730+0000 7fd0824aab80 1 --2- 172.21.15.90:0/1551076571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf80c8e40 0x561cf8246ed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd058015000 tx=0x7fd058003700 comp rx=0 tx=0).stop 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.731+0000 7fd0824aab80 1 -- 172.21.15.90:0/1551076571 shutdown_connections 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.731+0000 7fd0824aab80 1 --2- 172.21.15.90:0/1551076571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf8254020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.731+0000 7fd0824aab80 1 --2- 172.21.15.90:0/1551076571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf80c8e40 0x561cf8246ed0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.731+0000 7fd0824aab80 1 --2- 172.21.15.90:0/1551076571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf8247410 0x561cf8251ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.731+0000 7fd0824aab80 1 -- 172.21.15.90:0/1551076571 >> 172.21.15.90:0/1551076571 conn(0x561cf8128fc0 msgr2=0x561cf8242d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.731+0000 7fd0824aab80 1 -- 172.21.15.90:0/1551076571 shutdown_connections 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.733+0000 7fd0824aab80 1 -- 172.21.15.90:0/1551076571 wait complete. 2024-05-27T07:56:15.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.734+0000 7fd0824aab80 1 -- start start 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.734+0000 7fd0824aab80 1 Processor -- start 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.735+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf8251680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.735+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247410 0x561cf8251bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.735+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf8252100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.735+0000 7fd0824aab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561cf806bfa0 con 0x561cf80c8e40 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.735+0000 7fd0824aab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561cf8106760 con 0x561cf8247410 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.736+0000 7fd0824aab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cf8254990 con 0x561cf8247820 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.736+0000 7fd0824aab80 10 client.0 _ll_register_callbacks cb 0x561cf810fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.736+0000 7fd071a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf8252100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.736+0000 7fd071a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf8252100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37528/0 (socket says 172.21.15.90:37528) 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.736+0000 7fd071a00700 1 -- 172.21.15.90:0/2184076110 learned_addr learned my addr 172.21.15.90:0/2184076110 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.737+0000 7fd071000700 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf8251680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.737+0000 7fd071a00700 1 -- 172.21.15.90:0/2184076110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247410 msgr2=0x561cf8251bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:56:15.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.737+0000 7fd071a00700 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247410 0x561cf8251bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.737+0000 7fd071a00700 1 -- 172.21.15.90:0/2184076110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 msgr2=0x561cf8251680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.737+0000 7fd071a00700 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf8251680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.737+0000 7fd071a00700 1 -- 172.21.15.90:0/2184076110 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf80b58d0 con 0x561cf8247820 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.737+0000 7fd071000700 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf8251680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.738+0000 7fd071a00700 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf8252100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd05800f8f0 tx=0x7fd05800ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea67d76ee547a14 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.738+0000 7fd056a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.738+0000 7fd056a00700 1 -- 172.21.15.90:0/2184076110 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05803d060 con 0x561cf8247820 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.738+0000 7fd056a00700 1 -- 172.21.15.90:0/2184076110 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7fd05800f950 con 0x561cf8247820 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.739+0000 7fd0824aab80 1 -- 172.21.15.90:0/2184076110 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd058005500 con 0x561cf8247820 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.739+0000 7fd056a00700 1 -- 172.21.15.90:0/2184076110 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd05800fb50 con 0x561cf8247820 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.739+0000 7fd0824aab80 10 client.26902 fetch_fsmap learned FSMap version 61 2024-05-27T07:56:15.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.739+0000 7fd0824aab80 1 -- 172.21.15.90:0/2184076110 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x561cf8136060 con 0x561cf8247820 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.740+0000 7fd056a00700 1 -- 172.21.15.90:0/2184076110 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd058033050 con 0x561cf8247820 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.740+0000 7fd056a00700 1 -- 172.21.15.90:0/2184076110 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 61) ==== 6559+0+0 (secure 0 0 0) 0x7fd058038b70 con 0x561cf8247820 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.740+0000 7fd0824aab80 10 client.26902 fetch_fsmap finished waiting for FSMap version 61 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.740+0000 7fd0824aab80 10 client.26902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.740+0000 7fd0824aab80 20 client.26902 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.740+0000 7fd0824aab80 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561cf8258800 0x561cf825ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.740+0000 7fd0824aab80 4 client.26902 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.740+0000 7fd0824aab80 1 -- 172.21.15.90:0/2184076110 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd058005500 con 0x561cf8258800 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.741+0000 7fd071000700 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561cf8258800 0x561cf825ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.743+0000 7fd056a00700 10 client.26902 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.743+0000 7fd071000700 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561cf8258800 0x561cf825ac20 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b14cfc69a5858f6 server_cookie=9d20706f04ebb9cd in_seq=0 out_seq=0 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd056a00700 1 -- 172.21.15.90:0/2184076110 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7fd058005500 con 0x561cf8258800 2024-05-27T07:56:15.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd056a00700 10 client.26902 handle_command_reply: tid=0 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd0824aab80 1 client.26902 shutdown 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd0824aab80 1 -- 172.21.15.90:0/2184076110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 msgr2=0x561cf8252100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd0824aab80 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf8252100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd05800f8f0 tx=0x7fd05800ff70 comp rx=0 tx=0).stop 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd0824aab80 1 -- 172.21.15.90:0/2184076110 shutdown_connections 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd0824aab80 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561cf8258800 0x561cf825ac20 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd0824aab80 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf8252100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd0824aab80 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf8251680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd0824aab80 1 --2- 172.21.15.90:0/2184076110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247410 0x561cf8251bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.745+0000 7fd0824aab80 1 -- 172.21.15.90:0/2184076110 >> 172.21.15.90:0/2184076110 conn(0x561cf8128fc0 msgr2=0x561cf8241ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.746+0000 7fd056a00700 0 client.26902 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.747+0000 7fd0824aab80 1 -- 172.21.15.90:0/2184076110 shutdown_connections 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.747+0000 7fd0824aab80 1 -- 172.21.15.90:0/2184076110 wait complete. 2024-05-27T07:56:15.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.747+0000 7fd0824aab80 20 client.26902 trim_cache size 0 max 16384 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.778+0000 7fd0824aab80 1 Processor -- start 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.779+0000 7fd0824aab80 1 -- start start 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.779+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf824ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.779+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247410 0x561cf824b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.779+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247820 0x561cf827c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.779+0000 7fd0824aab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1afde0 con 0x561cf80c8e40 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.779+0000 7fd0824aab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1ace30 con 0x561cf8247820 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.779+0000 7fd0824aab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1af040 con 0x561cf8247410 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.780+0000 7fd06be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247410 0x561cf824b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.780+0000 7fd06be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247410 0x561cf824b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37538/0 (socket says 172.21.15.90:37538) 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.780+0000 7fd06be00700 1 -- 172.21.15.90:0/3137355501 learned_addr learned my addr 172.21.15.90:0/3137355501 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.780+0000 7fd071a00700 1 --2- 172.21.15.90:0/3137355501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247820 0x561cf827c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.781+0000 7fd071000700 1 --2- 172.21.15.90:0/3137355501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf824ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.781+0000 7fd06be00700 1 -- 172.21.15.90:0/3137355501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247820 msgr2=0x561cf827c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.781+0000 7fd06be00700 1 --2- 172.21.15.90:0/3137355501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247820 0x561cf827c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.781+0000 7fd06be00700 1 -- 172.21.15.90:0/3137355501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 msgr2=0x561cf824ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.781+0000 7fd06be00700 1 --2- 172.21.15.90:0/3137355501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf824ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.782+0000 7fd06be00700 1 -- 172.21.15.90:0/3137355501 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf823d1d0 con 0x561cf8247410 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.782+0000 7fd071000700 1 --2- 172.21.15.90:0/3137355501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf824ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.782+0000 7fd06be00700 1 --2- 172.21.15.90:0/3137355501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247410 0x561cf824b7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd05c005950 tx=0x7fd05c00cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3e20c6bebb6450c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.783+0000 7fd061600700 1 -- 172.21.15.90:0/3137355501 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c00ec20 con 0x561cf8247410 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.783+0000 7fd061600700 1 -- 172.21.15.90:0/3137355501 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd05c0044b0 con 0x561cf8247410 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.783+0000 7fd061600700 1 -- 172.21.15.90:0/3137355501 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c005020 con 0x561cf8247410 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.784+0000 7fd0824aab80 1 -- 172.21.15.90:0/3137355501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247410 msgr2=0x561cf824b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:15.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.784+0000 7fd0824aab80 1 --2- 172.21.15.90:0/3137355501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247410 0x561cf824b7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd05c005950 tx=0x7fd05c00cc70 comp rx=0 tx=0).stop 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.785+0000 7fd0824aab80 1 -- 172.21.15.90:0/3137355501 shutdown_connections 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.785+0000 7fd0824aab80 1 --2- 172.21.15.90:0/3137355501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247820 0x561cf827c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.785+0000 7fd0824aab80 1 --2- 172.21.15.90:0/3137355501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf824ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.785+0000 7fd0824aab80 1 --2- 172.21.15.90:0/3137355501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247410 0x561cf824b7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.785+0000 7fd0824aab80 1 -- 172.21.15.90:0/3137355501 >> 172.21.15.90:0/3137355501 conn(0x561cf8128fc0 msgr2=0x561cf82509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.785+0000 7fd0824aab80 1 -- 172.21.15.90:0/3137355501 shutdown_connections 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.786+0000 7fd0824aab80 1 -- 172.21.15.90:0/3137355501 wait complete. 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.786+0000 7fd0824aab80 1 -- start start 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.786+0000 7fd0824aab80 1 Processor -- start 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.788+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf825b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.788+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247410 0x561cf825bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.789+0000 7fd0824aab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf82d5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.789+0000 7fd0824aab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1ae970 con 0x561cf80c8e40 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.789+0000 7fd0824aab80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1ad500 con 0x561cf8247410 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.789+0000 7fd0824aab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c1adbd0 con 0x561cf8247820 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.789+0000 7fd0824aab80 10 client.0 _ll_register_callbacks cb 0x561cf810fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.790+0000 7fd071a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf82d5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.790+0000 7fd071a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf82d5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37546/0 (socket says 172.21.15.90:37546) 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.790+0000 7fd071a00700 1 -- 172.21.15.90:0/3521514475 learned_addr learned my addr 172.21.15.90:0/3521514475 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.791+0000 7fd06be00700 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247410 0x561cf825bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.791+0000 7fd071000700 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf825b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.791+0000 7fd071a00700 1 -- 172.21.15.90:0/3521514475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247410 msgr2=0x561cf825bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.791+0000 7fd071a00700 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247410 0x561cf825bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.792+0000 7fd071a00700 1 -- 172.21.15.90:0/3521514475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 msgr2=0x561cf825b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:15.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.792+0000 7fd071a00700 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf825b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.792+0000 7fd071a00700 1 -- 172.21.15.90:0/3521514475 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf82510b0 con 0x561cf8247820 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.794+0000 7fd071a00700 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf82d5410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd058043f70 tx=0x7fd05802d490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc9295e0254e49a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.795+0000 7fd056a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.795+0000 7fd056a00700 1 -- 172.21.15.90:0/3521514475 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05803d040 con 0x561cf8247820 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.795+0000 7fd056a00700 1 -- 172.21.15.90:0/3521514475 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7fd05802d710 con 0x561cf8247820 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.795+0000 7fd0824aab80 1 -- 172.21.15.90:0/3521514475 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd05c0044b0 con 0x561cf8247820 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.796+0000 7fd056a00700 1 -- 172.21.15.90:0/3521514475 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd058047080 con 0x561cf8247820 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.797+0000 7fd0824aab80 10 client.26908 fetch_fsmap learned FSMap version 61 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.797+0000 7fd0824aab80 1 -- 172.21.15.90:0/3521514475 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x561cf8244350 con 0x561cf8247820 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.798+0000 7fd056a00700 1 -- 172.21.15.90:0/3521514475 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd05c0044b0 con 0x561cf8247820 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.799+0000 7fd056a00700 1 -- 172.21.15.90:0/3521514475 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 61) ==== 6559+0+0 (secure 0 0 0) 0x7fd058038450 con 0x561cf8247820 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.799+0000 7fd0824aab80 10 client.26908 fetch_fsmap finished waiting for FSMap version 61 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.799+0000 7fd0824aab80 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.799+0000 7fd0824aab80 20 client.26908 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:15.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.799+0000 7fd0824aab80 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561cf82dcec0 0x561cf82df2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.799+0000 7fd0824aab80 4 client.26908 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.799+0000 7fd0824aab80 1 -- 172.21.15.90:0/3521514475 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cf8177d90 con 0x561cf82dcec0 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.801+0000 7fd071000700 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561cf82dcec0 0x561cf82df2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.803+0000 7fd071000700 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561cf82dcec0 0x561cf82df2e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d7718f77650734 server_cookie=49f8bf37abc5548 in_seq=0 out_seq=0 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.803+0000 7fd056a00700 10 client.26908 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.804+0000 7fd056a00700 1 -- 172.21.15.90:0/3521514475 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cf8177d90 con 0x561cf82dcec0 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.804+0000 7fd056a00700 10 client.26908 handle_command_reply: tid=0 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.804+0000 7fd0824aab80 1 client.26908 shutdown 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.804+0000 7fd0824aab80 1 -- 172.21.15.90:0/3521514475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 msgr2=0x561cf82d5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.804+0000 7fd0824aab80 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf82d5410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd058043f70 tx=0x7fd05802d490 comp rx=0 tx=0).stop 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.805+0000 7fd0824aab80 1 -- 172.21.15.90:0/3521514475 shutdown_connections 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.805+0000 7fd0824aab80 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561cf82dcec0 0x561cf82df2e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.805+0000 7fd0824aab80 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cf8247820 0x561cf82d5410 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.805+0000 7fd056a00700 0 client.26908 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.805+0000 7fd0824aab80 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cf80c8e40 0x561cf825b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.805+0000 7fd0824aab80 1 --2- 172.21.15.90:0/3521514475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561cf8247410 0x561cf825bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.805+0000 7fd0824aab80 1 -- 172.21.15.90:0/3521514475 >> 172.21.15.90:0/3521514475 conn(0x561cf8128fc0 msgr2=0x561cf8250700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.805+0000 7fd0824aab80 1 -- 172.21.15.90:0/3521514475 shutdown_connections 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.805+0000 7fd0824aab80 1 -- 172.21.15.90:0/3521514475 wait complete. 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.806+0000 7fd0824aab80 20 client.26908 trim_cache size 0 max 16384 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.810+0000 7fd072400700 1 -- 172.21.15.90:0/560544129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd0500b2cf0 msgr2=0x7fd0500b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.810+0000 7fd072400700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd0500b2cf0 0x7fd0500b5200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd058029000 tx=0x7fd05802c000 comp rx=0 tx=0).stop 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.810+0000 7fd072400700 1 -- 172.21.15.90:0/560544129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd06c1043f0 msgr2=0x7fd06c075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.810+0000 7fd072400700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd06c1043f0 0x7fd06c075ab0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd06400ba50 tx=0x7fd06400be10 comp rx=0 tx=0).stop 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.811+0000 7fd072400700 1 -- 172.21.15.90:0/560544129 shutdown_connections 2024-05-27T07:56:15.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.811+0000 7fd072400700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd06c109670 0x7fd06c076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.811+0000 7fd072400700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd06c1043f0 0x7fd06c075ab0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.811+0000 7fd072400700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd0500b2cf0 0x7fd0500b5200 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.811+0000 7fd072400700 1 --2- 172.21.15.90:0/560544129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd06c108d20 0x7fd06c075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:15.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.811+0000 7fd072400700 1 -- 172.21.15.90:0/560544129 >> 172.21.15.90:0/560544129 conn(0x7fd06c079d00 msgr2=0x7fd06c101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:15.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.811+0000 7fd072400700 1 -- 172.21.15.90:0/560544129 shutdown_connections 2024-05-27T07:56:15.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:14.812+0000 7fd072400700 1 -- 172.21.15.90:0/560544129 wait complete. 2024-05-27T07:56:15.928 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:15 smithi090 podman[114848]: 2024-05-27 07:56:15.646909066 +0000 UTC m=+2.607747152 container init 3d2a19f9d09934938d702e5ce11225f02175a67a99574b98a2acd2f7f695f299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-27T07:56:15.928 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:15 smithi090 podman[114848]: 2024-05-27 07:56:15.671462412 +0000 UTC m=+2.632300478 container start 3d2a19f9d09934938d702e5ce11225f02175a67a99574b98a2acd2f7f695f299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-27T07:56:15.928 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:15 smithi090 bash[114848]: 3d2a19f9d09934938d702e5ce11225f02175a67a99574b98a2acd2f7f695f299 2024-05-27T07:56:15.929 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:15 smithi090 systemd[1]: Started Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:56:15.929 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:15 smithi090 ceph-mds[114965]: -- 172.21.15.90:0/2501005011 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561759293a40 con 0x5617592db180 2024-05-27T07:56:15.929 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: starting mds.j at 2024-05-27T07:56:16.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:16 smithi129 ceph-mon[35254]: pgmap v312: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.2 MiB/s wr, 559 op/s 2024-05-27T07:56:16.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:16 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:56:16.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:16 smithi090 ceph-mon[31429]: pgmap v312: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.2 MiB/s wr, 559 op/s 2024-05-27T07:56:16.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:16 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:56:16.714 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f90e1800700 time 2024-05-27T07:56:16.347985+0000 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f90f723cba6] 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f90f723cdd1] 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5617240a723d] 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561724092971] 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 5: (SnapRealm::check_cache() const+0x13e) [0x561724090fa0] 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561724091651] 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561724026347] 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561723d2287e] 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561723d27f2f] 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 10: (EOpen::replay(MDSRank*)+0x57) [0x561723d2e3e9] 2024-05-27T07:56:16.715 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 11: (MDLog::_replay_thread()+0x1f38) [0x5617240cb48c] 2024-05-27T07:56:16.716 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5617240d49cb] 2024-05-27T07:56:16.716 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 13: (Thread::entry_wrapper()+0x43) [0x7f90f720b72d] 2024-05-27T07:56:16.716 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 14: (Thread::_entry_func(void*)+0xd) [0x7f90f720b749] 2024-05-27T07:56:16.716 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f90f5daf1ca] 2024-05-27T07:56:16.716 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 16: clone() 2024-05-27T07:56:16.716 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.350+0000 7f90e1800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f90e1800700 time 2024-05-27T07:56:16.347985+0000 2024-05-27T07:56:16.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:56:16.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:56:16.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f90f723cba6] 2024-05-27T07:56:16.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f90f723cdd1] 2024-05-27T07:56:16.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5617240a723d] 2024-05-27T07:56:16.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561724092971] 2024-05-27T07:56:16.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 5: (SnapRealm::check_cache() const+0x13e) [0x561724090fa0] 2024-05-27T07:56:16.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561724091651] 2024-05-27T07:56:16.719 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561724026347] 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561723d2287e] 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561723d27f2f] 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 10: (EOpen::replay(MDSRank*)+0x57) [0x561723d2e3e9] 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 11: (MDLog::_replay_thread()+0x1f38) [0x5617240cb48c] 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5617240d49cb] 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 13: (Thread::entry_wrapper()+0x43) [0x7f90f720b72d] 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 14: (Thread::_entry_func(void*)+0xd) [0x7f90f720b749] 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f90f5daf1ca] 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 16: clone() 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: *** Caught signal (Aborted) ** 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: in thread 7f90e1800700 thread_name:md_log_replay 2024-05-27T07:56:16.720 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:56:16.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f90f5db9d20] 2024-05-27T07:56:16.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2: gsignal() 2024-05-27T07:56:16.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 3: abort() 2024-05-27T07:56:16.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f90f723cced] 2024-05-27T07:56:16.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f90f723cdd1] 2024-05-27T07:56:16.721 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5617240a723d] 2024-05-27T07:56:16.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561724092971] 2024-05-27T07:56:16.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 8: (SnapRealm::check_cache() const+0x13e) [0x561724090fa0] 2024-05-27T07:56:16.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561724091651] 2024-05-27T07:56:16.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561724026347] 2024-05-27T07:56:16.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561723d2287e] 2024-05-27T07:56:16.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561723d27f2f] 2024-05-27T07:56:16.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 13: (EOpen::replay(MDSRank*)+0x57) [0x561723d2e3e9] 2024-05-27T07:56:16.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 14: (MDLog::_replay_thread()+0x1f38) [0x5617240cb48c] 2024-05-27T07:56:16.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5617240d49cb] 2024-05-27T07:56:16.722 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 16: (Thread::entry_wrapper()+0x43) [0x7f90f720b72d] 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 17: (Thread::_entry_func(void*)+0xd) [0x7f90f720b749] 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f90f5daf1ca] 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 19: clone() 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.354+0000 7f90e1800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: in thread 7f90e1800700 thread_name:md_log_replay 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f90f5db9d20] 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2: gsignal() 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 3: abort() 2024-05-27T07:56:16.723 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f90f723cced] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f90f723cdd1] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5617240a723d] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561724092971] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 8: (SnapRealm::check_cache() const+0x13e) [0x561724090fa0] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561724091651] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561724026347] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561723d2287e] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561723d27f2f] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 13: (EOpen::replay(MDSRank*)+0x57) [0x561723d2e3e9] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 14: (MDLog::_replay_thread()+0x1f38) [0x5617240cb48c] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5617240d49cb] 2024-05-27T07:56:16.724 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 16: (Thread::entry_wrapper()+0x43) [0x7f90f720b72d] 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 17: (Thread::_entry_func(void*)+0xd) [0x7f90f720b749] 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f90f5daf1ca] 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 19: clone() 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: -1> 2024-05-27T07:56:16.350+0000 7f90e1800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f90e1800700 time 2024-05-27T07:56:16.347985+0000 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f90f723cba6] 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f90f723cdd1] 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5617240a723d] 2024-05-27T07:56:16.725 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561724092971] 2024-05-27T07:56:16.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 5: (SnapRealm::check_cache() const+0x13e) [0x561724090fa0] 2024-05-27T07:56:16.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561724091651] 2024-05-27T07:56:16.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561724026347] 2024-05-27T07:56:16.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561723d2287e] 2024-05-27T07:56:16.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561723d27f2f] 2024-05-27T07:56:16.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 10: (EOpen::replay(MDSRank*)+0x57) [0x561723d2e3e9] 2024-05-27T07:56:16.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 11: (MDLog::_replay_thread()+0x1f38) [0x5617240cb48c] 2024-05-27T07:56:16.726 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5617240d49cb] 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 13: (Thread::entry_wrapper()+0x43) [0x7f90f720b72d] 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 14: (Thread::_entry_func(void*)+0xd) [0x7f90f720b749] 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f90f5daf1ca] 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 16: clone() 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 0> 2024-05-27T07:56:16.354+0000 7f90e1800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: in thread 7f90e1800700 thread_name:md_log_replay 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f90f5db9d20] 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2: gsignal() 2024-05-27T07:56:16.727 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 3: abort() 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f90f723cced] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f90f723cdd1] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5617240a723d] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561724092971] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 8: (SnapRealm::check_cache() const+0x13e) [0x561724090fa0] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561724091651] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561724026347] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561723d2287e] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561723d27f2f] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 13: (EOpen::replay(MDSRank*)+0x57) [0x561723d2e3e9] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 14: (MDLog::_replay_thread()+0x1f38) [0x5617240cb48c] 2024-05-27T07:56:16.728 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5617240d49cb] 2024-05-27T07:56:16.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 16: (Thread::entry_wrapper()+0x43) [0x7f90f720b72d] 2024-05-27T07:56:16.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 17: (Thread::_entry_func(void*)+0xd) [0x7f90f720b749] 2024-05-27T07:56:16.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f90f5daf1ca] 2024-05-27T07:56:16.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 19: clone() 2024-05-27T07:56:16.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:56:16.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: -1> 2024-05-27T07:56:16.350+0000 7f90e1800700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: In function 'std::set SnapClient::filter(const std::set&) const' thread 7f90e1800700 time 2024-05-27T07:56:16.347985+0000 2024-05-27T07:56:16.729 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3872-g38f9d900/rpm/el8/BUILD/ceph-19.0.0-3872-g38f9d900/src/mds/SnapClient.cc: 197: FAILED ceph_assert(cached_version > 0) 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x127) [0x7f90f723cba6] 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f90f723cdd1] 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 3: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5617240a723d] 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 4: (SnapRealm::build_snap_set() const+0x323) [0x561724092971] 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 5: (SnapRealm::check_cache() const+0x13e) [0x561724090fa0] 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 6: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561724091651] 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 7: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561724026347] 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 8: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561723d2287e] 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 9: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561723d27f2f] 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 10: (EOpen::replay(MDSRank*)+0x57) [0x561723d2e3e9] 2024-05-27T07:56:16.730 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 11: (MDLog::_replay_thread()+0x1f38) [0x5617240cb48c] 2024-05-27T07:56:16.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 12: (MDLog::ReplayThread::entry()+0x11) [0x5617240d49cb] 2024-05-27T07:56:16.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 13: (Thread::entry_wrapper()+0x43) [0x7f90f720b72d] 2024-05-27T07:56:16.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 14: (Thread::_entry_func(void*)+0xd) [0x7f90f720b749] 2024-05-27T07:56:16.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f90f5daf1ca] 2024-05-27T07:56:16.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 16: clone() 2024-05-27T07:56:16.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 0> 2024-05-27T07:56:16.354+0000 7f90e1800700 -1 *** Caught signal (Aborted) ** 2024-05-27T07:56:16.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: in thread 7f90e1800700 thread_name:md_log_replay 2024-05-27T07:56:16.731 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: ceph version 19.0.0-3872-g38f9d900 (38f9d900c83c63799fbdbe61acc9a11b0d3554a6) squid (dev) 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f90f5db9d20] 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2: gsignal() 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 3: abort() 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x26e) [0x7f90f723cced] 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 5: (ceph::register_assert_context(ceph::common::CephContext*)+0) [0x7f90f723cdd1] 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 6: (SnapClient::filter(std::set, std::allocator > const&) const+0x81) [0x5617240a723d] 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 7: (SnapRealm::build_snap_set() const+0x323) [0x561724092971] 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 8: (SnapRealm::check_cache() const+0x13e) [0x561724090fa0] 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 9: (operator<<(std::ostream&, SnapRealm const&)+0x1fd) [0x561724091651] 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 10: (CInode::decode_snap_blob(ceph::buffer::v15_2_0::list const&)+0x271) [0x561724026347] 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 11: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x22c) [0x561723d2287e] 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 12: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x279d) [0x561723d27f2f] 2024-05-27T07:56:16.732 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 13: (EOpen::replay(MDSRank*)+0x57) [0x561723d2e3e9] 2024-05-27T07:56:16.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 14: (MDLog::_replay_thread()+0x1f38) [0x5617240cb48c] 2024-05-27T07:56:16.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 15: (MDLog::ReplayThread::entry()+0x11) [0x5617240d49cb] 2024-05-27T07:56:16.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 16: (Thread::entry_wrapper()+0x43) [0x7f90f720b72d] 2024-05-27T07:56:16.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 17: (Thread::_entry_func(void*)+0xd) [0x7f90f720b749] 2024-05-27T07:56:16.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 18: /lib64/libpthread.so.0(+0x81ca) [0x7f90f5daf1ca] 2024-05-27T07:56:16.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 19: clone() 2024-05-27T07:56:16.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-27T07:56:16.733 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j[114877]: 2024-05-27T07:56:16.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:16 smithi136 ceph-mon[35324]: pgmap v312: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.2 MiB/s wr, 559 op/s 2024-05-27T07:56:16.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:16 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:56:17.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:17 smithi129 ceph-mon[35254]: mds.? [v2:172.21.15.90:6834/1271298304,v1:172.21.15.90:6835/1271298304] up:boot 2024-05-27T07:56:17.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:17 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:56:17.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:17 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:56:17.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:17 smithi129 ceph-mon[35254]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:56:17.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:17 smithi129 ceph-mon[35254]: osdmap e108: 12 total, 12 up, 12 in 2024-05-27T07:56:17.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:17 smithi090 ceph-mon[31429]: mds.? [v2:172.21.15.90:6834/1271298304,v1:172.21.15.90:6835/1271298304] up:boot 2024-05-27T07:56:17.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:17 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:56:17.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:17 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:56:17.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:17 smithi090 ceph-mon[31429]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:56:17.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:17 smithi090 ceph-mon[31429]: osdmap e108: 12 total, 12 up, 12 in 2024-05-27T07:56:17.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:17 smithi136 ceph-mon[35324]: mds.? [v2:172.21.15.90:6834/1271298304,v1:172.21.15.90:6835/1271298304] up:boot 2024-05-27T07:56:17.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:17 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-05-27T07:56:17.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:17 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T07:56:17.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:17 smithi136 ceph-mon[35324]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-05-27T07:56:17.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:17 smithi136 ceph-mon[35324]: osdmap e108: 12 total, 12 up, 12 in 2024-05-27T07:56:18.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:18 smithi129 ceph-mon[35254]: pgmap v314: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 3.8 MiB/s wr, 406 op/s 2024-05-27T07:56:18.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:18 smithi090 ceph-mon[31429]: pgmap v314: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 3.8 MiB/s wr, 406 op/s 2024-05-27T07:56:18.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:18 smithi136 ceph-mon[35324]: pgmap v314: 129 pgs: 129 active+clean; 2.6 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 3.8 MiB/s wr, 406 op/s 2024-05-27T07:56:18.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 360.0 s, 536.6 tps, lat 204.725 ms stddev 203.532 2024-05-27T07:56:18.947 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:18 smithi090 podman[115038]: 2024-05-27 07:56:18.565639897 +0000 UTC m=+1.854349666 container died 3d2a19f9d09934938d702e5ce11225f02175a67a99574b98a2acd2f7f695f299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5) 2024-05-27T07:56:19.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:19 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:56:19.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:19 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:56:19.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:19 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:56:19.572 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:56:19.573 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:56:19.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:19 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:56:19.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:19 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:56:19.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:19 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:56:19.670 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:19 smithi090 podman[115038]: 2024-05-27 07:56:19.257990734 +0000 UTC m=+2.546700488 container remove 3d2a19f9d09934938d702e5ce11225f02175a67a99574b98a2acd2f7f695f299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-j, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=) 2024-05-27T07:56:19.671 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:19 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-05-27T07:56:19.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:19 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:56:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:19 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:56:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:19 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' 2024-05-27T07:56:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.959+0000 7f61e3000640 1 -- 172.21.15.90:0/1491176503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61dc10a200 msgr2=0x7f61dc10a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.959+0000 7f61e3000640 1 --2- 172.21.15.90:0/1491176503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61dc10a200 0x7f61dc10a690 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009ca0 tx=0x7f61d402a8e0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.960+0000 7f61e3000640 1 -- 172.21.15.90:0/1491176503 shutdown_connections 2024-05-27T07:56:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.960+0000 7f61e3000640 1 --2- 172.21.15.90:0/1491176503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61dc1103e0 0x7f61dc109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.960+0000 7f61e3000640 1 --2- 172.21.15.90:0/1491176503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61dc10a200 0x7f61dc10a690 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.960+0000 7f61e3000640 1 --2- 172.21.15.90:0/1491176503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61dc10fa00 0x7f61dc10fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.960+0000 7f61e3000640 1 -- 172.21.15.90:0/1491176503 >> 172.21.15.90:0/1491176503 conn(0x7f61dc06f800 msgr2=0x7f61dc06fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.960+0000 7f61e3000640 1 -- 172.21.15.90:0/1491176503 shutdown_connections 2024-05-27T07:56:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.960+0000 7f61e3000640 1 -- 172.21.15.90:0/1491176503 wait complete. 2024-05-27T07:56:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.961+0000 7f61e3000640 1 Processor -- start 2024-05-27T07:56:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.962+0000 7f61e3000640 1 -- start start 2024-05-27T07:56:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.962+0000 7f61e3000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61dc10a200 0x7f61dc1a9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.963+0000 7f61e3000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61dc10fa00 0x7f61dc1a9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.963+0000 7f61e3000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61dc1103e0 0x7f61dc1ad940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.963+0000 7f61e3000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc122960 con 0x7f61dc10a200 2024-05-27T07:56:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.963+0000 7f61e3000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1227e0 con 0x7f61dc1103e0 2024-05-27T07:56:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.963+0000 7f61e1c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61dc10a200 0x7f61dc1a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.963+0000 7f61e1c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61dc10a200 0x7f61dc1a9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38504/0 (socket says 172.21.15.90:38504) 2024-05-27T07:56:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.963+0000 7f61e1c00640 1 -- 172.21.15.90:0/3050507882 learned_addr learned my addr 172.21.15.90:0/3050507882 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.964+0000 7f61e1200640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61dc10fa00 0x7f61dc1a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.963+0000 7f61e3000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc122ae0 con 0x7f61dc10fa00 2024-05-27T07:56:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.964+0000 7f61e2600640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61dc1103e0 0x7f61dc1ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.965+0000 7f61e2600640 1 -- 172.21.15.90:0/3050507882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61dc10fa00 msgr2=0x7f61dc1a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.965+0000 7f61e2600640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61dc10fa00 0x7f61dc1a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.965+0000 7f61e2600640 1 -- 172.21.15.90:0/3050507882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61dc10a200 msgr2=0x7f61dc1a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.965+0000 7f61e2600640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61dc10a200 0x7f61dc1a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.967 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.966+0000 7f61e2600640 1 -- 172.21.15.90:0/3050507882 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc1adf80 con 0x7f61dc1103e0 2024-05-27T07:56:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.966+0000 7f61e1c00640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61dc10a200 0x7f61dc1a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:19.968 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.966+0000 7f61e1200640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61dc10fa00 0x7f61dc1a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.967+0000 7f61e2600640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61dc1103e0 0x7f61dc1ad940 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f61d402ae00 tx=0x7f61d4003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.967+0000 7f61da000640 1 -- 172.21.15.90:0/3050507882 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4004040 con 0x7f61dc1103e0 2024-05-27T07:56:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.967+0000 7f61e3000640 1 -- 172.21.15.90:0/3050507882 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1ae210 con 0x7f61dc1103e0 2024-05-27T07:56:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.967+0000 7f61e3000640 1 -- 172.21.15.90:0/3050507882 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1b5bf0 con 0x7f61dc1103e0 2024-05-27T07:56:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.968+0000 7f61da000640 1 -- 172.21.15.90:0/3050507882 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61d4004670 con 0x7f61dc1103e0 2024-05-27T07:56:19.969 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.968+0000 7f61da000640 1 -- 172.21.15.90:0/3050507882 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4030c40 con 0x7f61dc1103e0 2024-05-27T07:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.972+0000 7f61eb5fe740 1 Processor -- start 2024-05-27T07:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.973+0000 7f61eb5fe740 1 -- start start 2024-05-27T07:56:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.973+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4888a1b0 0x55aa48c2ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.973+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488612f0 0x55aa48c2cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.973+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa488694d0 0x55aa48c2d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.973+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aa48b9db90 con 0x55aa4888a1b0 2024-05-27T07:56:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.973+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aa48b8d730 con 0x55aa488694d0 2024-05-27T07:56:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.973+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aa48b91770 con 0x55aa488612f0 2024-05-27T07:56:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.974+0000 7f61e1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4888a1b0 0x55aa48c2ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.974+0000 7f61e1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4888a1b0 0x55aa48c2ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38520/0 (socket says 172.21.15.90:38520) 2024-05-27T07:56:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.974+0000 7f61e1200640 1 -- 172.21.15.90:0/3447422582 learned_addr learned my addr 172.21.15.90:0/3447422582 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.974+0000 7f61e1c00640 1 --2- 172.21.15.90:0/3447422582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488612f0 0x55aa48c2cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.974+0000 7f61e1200640 1 --2- 172.21.15.90:0/3447422582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa488694d0 0x55aa48c2d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.975+0000 7f61e1200640 1 -- 172.21.15.90:0/3447422582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488612f0 msgr2=0x55aa48c2cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.975+0000 7f61e1200640 1 --2- 172.21.15.90:0/3447422582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488612f0 0x55aa48c2cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.975+0000 7f61e1200640 1 -- 172.21.15.90:0/3447422582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa488694d0 msgr2=0x55aa48c2d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.975+0000 7f61e1200640 1 --2- 172.21.15.90:0/3447422582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa488694d0 0x55aa48c2d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.975+0000 7f61e1200640 1 -- 172.21.15.90:0/3447422582 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa48b268b0 con 0x55aa4888a1b0 2024-05-27T07:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.975+0000 7f61e1200640 1 --2- 172.21.15.90:0/3447422582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4888a1b0 0x55aa48c2ca60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f61c8014490 tx=0x7f61c80095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dbdbd7fb11ccb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.976+0000 7f61cea00640 1 -- 172.21.15.90:0/3447422582 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8004230 con 0x55aa4888a1b0 2024-05-27T07:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.976+0000 7f61cea00640 1 -- 172.21.15.90:0/3447422582 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61c801b070 con 0x55aa4888a1b0 2024-05-27T07:56:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.977+0000 7f61cea00640 1 -- 172.21.15.90:0/3447422582 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c800e650 con 0x55aa4888a1b0 2024-05-27T07:56:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.979+0000 7f61da000640 1 -- 172.21.15.90:0/3050507882 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f61d40041e0 con 0x7f61dc1103e0 2024-05-27T07:56:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.979+0000 7f61eb5fe740 1 -- 172.21.15.90:0/3447422582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4888a1b0 msgr2=0x55aa48c2ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.979+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/3447422582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4888a1b0 0x55aa48c2ca60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f61c8014490 tx=0x7f61c80095b0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.979+0000 7f61eb5fe740 1 -- 172.21.15.90:0/3447422582 shutdown_connections 2024-05-27T07:56:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.979+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/3447422582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa488694d0 0x55aa48c2d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.979+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/3447422582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488612f0 0x55aa48c2cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.979+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/3447422582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4888a1b0 0x55aa48c2ca60 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.979+0000 7f61eb5fe740 1 -- 172.21.15.90:0/3447422582 >> 172.21.15.90:0/3447422582 conn(0x55aa4887e300 msgr2=0x55aa48c29210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.979+0000 7f61eb5fe740 1 -- 172.21.15.90:0/3447422582 shutdown_connections 2024-05-27T07:56:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.980+0000 7f61eb5fe740 1 -- 172.21.15.90:0/3447422582 wait complete. 2024-05-27T07:56:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.980+0000 7f61da000640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61c40b2960 0x7f61c40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.980+0000 7f61eb5fe740 1 -- start start 2024-05-27T07:56:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.981+0000 7f61da000640 1 -- 172.21.15.90:0/3050507882 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f61d4103e30 con 0x7f61dc1103e0 2024-05-27T07:56:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.981+0000 7f61e1200640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61c40b2960 0x7f61c40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.981+0000 7f61eb5fe740 1 Processor -- start 2024-05-27T07:56:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.982+0000 7f61e1200640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61c40b2960 0x7f61c40b4e70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f61c8014490 tx=0x7f61c80093b0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.983+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488612f0 0x55aa48c36b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.983+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488694d0 0x55aa48c370a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.983+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 0x55aa48c37620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.983+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aa48873790 con 0x55aa488612f0 2024-05-27T07:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.983+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aa48b35bc0 con 0x55aa4888a1b0 2024-05-27T07:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.984+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aa48c39b90 con 0x55aa488694d0 2024-05-27T07:56:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.984+0000 7f61eb5fe740 10 client.0 _ll_register_callbacks cb 0x55aa48b13400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.985+0000 7f61e2600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 0x55aa48c37620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.985+0000 7f61e2600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 0x55aa48c37620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56570/0 (socket says 172.21.15.90:56570) 2024-05-27T07:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.985+0000 7f61e2600640 1 -- 172.21.15.90:0/4171120874 learned_addr learned my addr 172.21.15.90:0/4171120874 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.985+0000 7f61e1200640 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488694d0 0x55aa48c370a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.985+0000 7f61e1c00640 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488612f0 0x55aa48c36b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.986+0000 7f61e2600640 1 -- 172.21.15.90:0/4171120874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488694d0 msgr2=0x55aa48c370a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.986+0000 7f61e2600640 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488694d0 0x55aa48c370a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.986+0000 7f61e2600640 1 -- 172.21.15.90:0/4171120874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488612f0 msgr2=0x55aa48c36b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.986+0000 7f61e2600640 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488612f0 0x55aa48c36b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.986+0000 7f61e2600640 1 -- 172.21.15.90:0/4171120874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa488a2210 con 0x55aa4888a1b0 2024-05-27T07:56:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.986+0000 7f61e1c00640 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488612f0 0x55aa48c36b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.987+0000 7f61e2600640 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 0x55aa48c37620 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009bf0 tx=0x7f61d40044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01ac99c9cec1e91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.987+0000 7f61c3400640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.987+0000 7f61c3400640 1 -- 172.21.15.90:0/4171120874 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40cb4f0 con 0x55aa4888a1b0 2024-05-27T07:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.987+0000 7f61c3400640 1 -- 172.21.15.90:0/4171120874 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f61d41013d0 con 0x55aa4888a1b0 2024-05-27T07:56:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.987+0000 7f61eb5fe740 1 -- 172.21.15.90:0/4171120874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c801b070 con 0x55aa4888a1b0 2024-05-27T07:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.988+0000 7f61c3400640 1 -- 172.21.15.90:0/4171120874 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f61d41015d0 con 0x55aa4888a1b0 2024-05-27T07:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.988+0000 7f61eb5fe740 10 client.36872 fetch_fsmap learned FSMap version 63 2024-05-27T07:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.988+0000 7f61eb5fe740 1 -- 172.21.15.90:0/4171120874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55aa4896e620 con 0x55aa4888a1b0 2024-05-27T07:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.988+0000 7f61c3400640 1 -- 172.21.15.90:0/4171120874 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61c801b070 con 0x55aa4888a1b0 2024-05-27T07:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.988+0000 7f61c3400640 1 -- 172.21.15.90:0/4171120874 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 63) ==== 7152+0+0 (secure 0 0 0) 0x7f61d40d37c0 con 0x55aa4888a1b0 2024-05-27T07:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.989+0000 7f61eb5fe740 10 client.36872 fetch_fsmap finished waiting for FSMap version 63 2024-05-27T07:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.989+0000 7f61eb5fe740 10 client.36872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.989+0000 7f61eb5fe740 20 client.36872 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.989+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa48848d80 0x55aa48c48fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.989+0000 7f61eb5fe740 4 client.36872 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.989+0000 7f61eb5fe740 1 -- 172.21.15.90:0/4171120874 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa48b9fae0 con 0x55aa48848d80 2024-05-27T07:56:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.990+0000 7f61e1c00640 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa48848d80 0x55aa48c48fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.991+0000 7f61e1c00640 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa48848d80 0x55aa48c48fb0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19eedf4bbb50cc39 server_cookie=da3b1bccd4536099 in_seq=0 out_seq=0 2024-05-27T07:56:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.991+0000 7f61c3400640 10 client.36872 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.994+0000 7f61c3400640 1 -- 172.21.15.90:0/4171120874 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55aa48b9fae0 con 0x55aa48848d80 2024-05-27T07:56:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.994+0000 7f61c3400640 10 client.36872 handle_command_reply: tid=0 2024-05-27T07:56:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.994+0000 7f61eb5fe740 1 client.36872 shutdown 2024-05-27T07:56:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.995+0000 7f61eb5fe740 1 -- 172.21.15.90:0/4171120874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 msgr2=0x55aa48c37620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.995+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 0x55aa48c37620 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009bf0 tx=0x7f61d40044d0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.996+0000 7f61eb5fe740 1 -- 172.21.15.90:0/4171120874 shutdown_connections 2024-05-27T07:56:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.996+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa48848d80 0x55aa48c48fb0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.996+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 0x55aa48c37620 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.996+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488694d0 0x55aa48c370a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.996+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/4171120874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488612f0 0x55aa48c36b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.996+0000 7f61c3400640 0 client.36872 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.996+0000 7f61eb5fe740 1 -- 172.21.15.90:0/4171120874 >> 172.21.15.90:0/4171120874 conn(0x55aa4887e300 msgr2=0x55aa48c2c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:19.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.996+0000 7f61eb5fe740 1 -- 172.21.15.90:0/4171120874 shutdown_connections 2024-05-27T07:56:19.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.997+0000 7f61eb5fe740 1 -- 172.21.15.90:0/4171120874 wait complete. 2024-05-27T07:56:19.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:19.997+0000 7f61eb5fe740 20 client.36872 trim_cache size 0 max 16384 2024-05-27T07:56:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.028+0000 7f61eb5fe740 1 Processor -- start 2024-05-27T07:56:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.028+0000 7f61eb5fe740 1 -- start start 2024-05-27T07:56:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.028+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa488612f0 0x55aa48c39290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:20.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.028+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488694d0 0x55aa48c4e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.028+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa4888a1b0 0x55aa48c4ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.028+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc00ea20 con 0x55aa488694d0 2024-05-27T07:56:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.028+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc011300 con 0x55aa488612f0 2024-05-27T07:56:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.028+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc00f0f0 con 0x55aa4888a1b0 2024-05-27T07:56:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.029+0000 7f61e1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488694d0 0x55aa48c4e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.029+0000 7f61e1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488694d0 0x55aa48c4e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38538/0 (socket says 172.21.15.90:38538) 2024-05-27T07:56:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.029+0000 7f61e1200640 1 -- 172.21.15.90:0/2628180932 learned_addr learned my addr 172.21.15.90:0/2628180932 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.029+0000 7f61e1200640 1 -- 172.21.15.90:0/2628180932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa4888a1b0 msgr2=0x55aa48c4ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.029+0000 7f61e1200640 1 --2- 172.21.15.90:0/2628180932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa4888a1b0 0x55aa48c4ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.029+0000 7f61e1200640 1 -- 172.21.15.90:0/2628180932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa488612f0 msgr2=0x55aa48c39290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.029+0000 7f61e1200640 1 --2- 172.21.15.90:0/2628180932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa488612f0 0x55aa48c39290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.030+0000 7f61e1200640 1 -- 172.21.15.90:0/2628180932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa48c42f70 con 0x55aa488694d0 2024-05-27T07:56:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.030+0000 7f61e1200640 1 --2- 172.21.15.90:0/2628180932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488694d0 0x55aa48c4e770 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009420 tx=0x7f61c8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bab20f43155129f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.030+0000 7f61ce000640 1 -- 172.21.15.90:0/2628180932 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8004250 con 0x55aa488694d0 2024-05-27T07:56:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.031+0000 7f61ce000640 1 -- 172.21.15.90:0/2628180932 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61c801b070 con 0x55aa488694d0 2024-05-27T07:56:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.031+0000 7f61ce000640 1 -- 172.21.15.90:0/2628180932 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8020cb0 con 0x55aa488694d0 2024-05-27T07:56:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.032+0000 7f61eb5fe740 1 -- 172.21.15.90:0/2628180932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488694d0 msgr2=0x55aa48c4e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.032+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/2628180932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488694d0 0x55aa48c4e770 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009420 tx=0x7f61c8021000 comp rx=0 tx=0).stop 2024-05-27T07:56:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.032+0000 7f61eb5fe740 1 -- 172.21.15.90:0/2628180932 shutdown_connections 2024-05-27T07:56:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.033+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/2628180932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa4888a1b0 0x55aa48c4ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.033+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/2628180932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa488694d0 0x55aa48c4e770 secure :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f61c8009420 tx=0x7f61c8021000 comp rx=0 tx=0).stop 2024-05-27T07:56:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.033+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/2628180932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa488612f0 0x55aa48c39290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.033+0000 7f61eb5fe740 1 -- 172.21.15.90:0/2628180932 >> 172.21.15.90:0/2628180932 conn(0x55aa4887e300 msgr2=0x55aa48c426e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.033+0000 7f61eb5fe740 1 -- 172.21.15.90:0/2628180932 shutdown_connections 2024-05-27T07:56:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.033+0000 7f61eb5fe740 1 -- 172.21.15.90:0/2628180932 wait complete. 2024-05-27T07:56:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.034+0000 7f61eb5fe740 1 -- start start 2024-05-27T07:56:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.034+0000 7f61eb5fe740 1 Processor -- start 2024-05-27T07:56:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.037+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488612f0 0x55aa48c3e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.038+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 0x55aa48c3e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.038+0000 7f61eb5fe740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4881e1c0 0x55aa48caf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.038+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc00f7c0 con 0x55aa4881e1c0 2024-05-27T07:56:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.038+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc010c30 con 0x55aa4888a1b0 2024-05-27T07:56:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.038+0000 7f61eb5fe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc00fe90 con 0x55aa488612f0 2024-05-27T07:56:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.038+0000 7f61e2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4881e1c0 0x55aa48caf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.038+0000 7f61e2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4881e1c0 0x55aa48caf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38550/0 (socket says 172.21.15.90:38550) 2024-05-27T07:56:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.038+0000 7f61eb5fe740 10 client.0 _ll_register_callbacks cb 0x7f61d4009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.038+0000 7f61e2600640 1 -- 172.21.15.90:0/416786265 learned_addr learned my addr 172.21.15.90:0/416786265 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.038+0000 7f61e1200640 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 0x55aa48c3e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.039+0000 7f61e1c00640 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488612f0 0x55aa48c3e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.039+0000 7f61e2600640 1 -- 172.21.15.90:0/416786265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488612f0 msgr2=0x55aa48c3e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.039+0000 7f61e2600640 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488612f0 0x55aa48c3e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.039+0000 7f61e2600640 1 -- 172.21.15.90:0/416786265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 msgr2=0x55aa48c3e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.039+0000 7f61e2600640 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 0x55aa48c3e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.039+0000 7f61e2600640 1 -- 172.21.15.90:0/416786265 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa48c27610 con 0x55aa4881e1c0 2024-05-27T07:56:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.039+0000 7f61e2600640 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4881e1c0 0x55aa48caf650 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f61d40c8060 tx=0x7f61d40c6da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f27cbb0fef27ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.039+0000 7f61c3400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:56:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.040+0000 7f61c3400640 1 -- 172.21.15.90:0/416786265 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d40ca740 con 0x55aa4881e1c0 2024-05-27T07:56:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.040+0000 7f61c3400640 1 -- 172.21.15.90:0/416786265 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f61d40d37c0 con 0x55aa4881e1c0 2024-05-27T07:56:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.040+0000 7f61eb5fe740 1 -- 172.21.15.90:0/416786265 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c801b070 con 0x55aa4881e1c0 2024-05-27T07:56:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.041+0000 7f61c3400640 1 -- 172.21.15.90:0/416786265 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f61d40305e0 con 0x55aa4881e1c0 2024-05-27T07:56:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.041+0000 7f61eb5fe740 10 client.17289 fetch_fsmap learned FSMap version 63 2024-05-27T07:56:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.041+0000 7f61eb5fe740 1 -- 172.21.15.90:0/416786265 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55aa48c2de40 con 0x55aa4881e1c0 2024-05-27T07:56:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.042+0000 7f61c3400640 1 -- 172.21.15.90:0/416786265 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61c801b070 con 0x55aa4881e1c0 2024-05-27T07:56:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.042+0000 7f61c3400640 1 -- 172.21.15.90:0/416786265 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 63) ==== 7152+0+0 (secure 0 0 0) 0x7f61d40c6760 con 0x55aa4881e1c0 2024-05-27T07:56:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.042+0000 7f61eb5fe740 10 client.17289 fetch_fsmap finished waiting for FSMap version 63 2024-05-27T07:56:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.042+0000 7f61eb5fe740 10 client.17289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.042+0000 7f61eb5fe740 20 client.17289 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.042+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa48cb0f80 0x55aa48cb33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.042+0000 7f61eb5fe740 4 client.17289 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:56:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.042+0000 7f61eb5fe740 1 -- 172.21.15.90:0/416786265 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa48b992c0 con 0x55aa48cb0f80 2024-05-27T07:56:20.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.043+0000 7f61e1c00640 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa48cb0f80 0x55aa48cb33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.045+0000 7f61e1c00640 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa48cb0f80 0x55aa48cb33a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1faa48a14a9e7e server_cookie=7acf291081e7167c in_seq=0 out_seq=0 2024-05-27T07:56:20.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.045+0000 7f61c3400640 10 client.17289 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.097+0000 7f61c3400640 1 -- 172.21.15.90:0/416786265 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55aa48b992c0 con 0x55aa48cb0f80 2024-05-27T07:56:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.098+0000 7f61c3400640 10 client.17289 handle_command_reply: tid=0 2024-05-27T07:56:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.098+0000 7f61eb5fe740 1 client.17289 shutdown 2024-05-27T07:56:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.098+0000 7f61eb5fe740 1 -- 172.21.15.90:0/416786265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4881e1c0 msgr2=0x55aa48caf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.098+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4881e1c0 0x55aa48caf650 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f61d40c8060 tx=0x7f61d40c6da0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.099+0000 7f61eb5fe740 1 -- 172.21.15.90:0/416786265 shutdown_connections 2024-05-27T07:56:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.099+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa48cb0f80 0x55aa48cb33a0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.099+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa4888a1b0 0x55aa48c3e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.099+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa4881e1c0 0x55aa48caf650 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.099+0000 7f61c3400640 0 client.17289 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.099+0000 7f61eb5fe740 1 --2- 172.21.15.90:0/416786265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa488612f0 0x55aa48c3e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.099+0000 7f61eb5fe740 1 -- 172.21.15.90:0/416786265 >> 172.21.15.90:0/416786265 conn(0x55aa4887e300 msgr2=0x55aa48c31300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.100+0000 7f61eb5fe740 1 -- 172.21.15.90:0/416786265 shutdown_connections 2024-05-27T07:56:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.100+0000 7f61eb5fe740 1 -- 172.21.15.90:0/416786265 wait complete. 2024-05-27T07:56:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.100+0000 7f61eb5fe740 20 client.17289 trim_cache size 0 max 16384 2024-05-27T07:56:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:56:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:56:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:56:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.102+0000 7f61e3000640 1 -- 172.21.15.90:0/3050507882 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61c40b2960 msgr2=0x7f61c40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.102+0000 7f61e3000640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61c40b2960 0x7f61c40b4e70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f61c8014490 tx=0x7f61c80093b0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.102+0000 7f61e3000640 1 -- 172.21.15.90:0/3050507882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61dc1103e0 msgr2=0x7f61dc1ad940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.102+0000 7f61e3000640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61dc1103e0 0x7f61dc1ad940 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f61d402ae00 tx=0x7f61d4003ba0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.102+0000 7f61e3000640 1 -- 172.21.15.90:0/3050507882 shutdown_connections 2024-05-27T07:56:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.102+0000 7f61e3000640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61c40b2960 0x7f61c40b4e70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.102+0000 7f61e3000640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f61dc1103e0 0x7f61dc1ad940 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.102+0000 7f61e3000640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61dc10fa00 0x7f61dc1a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.103+0000 7f61e3000640 1 --2- 172.21.15.90:0/3050507882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61dc10a200 0x7f61dc1a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.103+0000 7f61e3000640 1 -- 172.21.15.90:0/3050507882 >> 172.21.15.90:0/3050507882 conn(0x7f61dc06f800 msgr2=0x7f61dc10d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.103+0000 7f61e3000640 1 -- 172.21.15.90:0/3050507882 shutdown_connections 2024-05-27T07:56:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:20.104+0000 7f61e3000640 1 -- 172.21.15.90:0/3050507882 wait complete. 2024-05-27T07:56:20.308 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:19 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-05-27T07:56:20.308 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:19 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.566s CPU time. 2024-05-27T07:56:20.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:20 smithi129 ceph-mon[35254]: pgmap v315: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.7 MiB/s wr, 527 op/s 2024-05-27T07:56:20.651 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:20 smithi136 ceph-mon[35324]: pgmap v315: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.7 MiB/s wr, 527 op/s 2024-05-27T07:56:20.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:20 smithi090 ceph-mon[31429]: pgmap v315: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 4.7 MiB/s wr, 527 op/s 2024-05-27T07:56:21.185 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:22.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:22 smithi090 ceph-mon[31429]: pgmap v316: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-05-27T07:56:22.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:22 smithi136 ceph-mon[35324]: pgmap v316: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-05-27T07:56:22.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:22 smithi129 ceph-mon[35254]: pgmap v316: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-05-27T07:56:23.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 365.0 s, 586.8 tps, lat 171.231 ms stddev 128.641 2024-05-27T07:56:24.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:24 smithi090 ceph-mon[31429]: pgmap v317: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.1 MiB/s wr, 526 op/s 2024-05-27T07:56:24.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:24 smithi136 ceph-mon[35324]: pgmap v317: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.1 MiB/s wr, 526 op/s 2024-05-27T07:56:24.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:24 smithi129 ceph-mon[35254]: pgmap v317: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.1 MiB/s wr, 526 op/s 2024-05-27T07:56:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.057+0000 7ff5dde00700 1 -- 172.21.15.90:0/876155934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5d0097720 msgr2=0x7ff5d0097b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.058+0000 7ff5dde00700 1 --2- 172.21.15.90:0/876155934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5d0097720 0x7ff5d0097b30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc0098f0 tx=0x7ff5cc009c00 comp rx=0 tx=0).stop 2024-05-27T07:56:26.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.058+0000 7ff5dde00700 1 -- 172.21.15.90:0/876155934 shutdown_connections 2024-05-27T07:56:26.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.058+0000 7ff5dde00700 1 --2- 172.21.15.90:0/876155934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5d00a0b90 0x7ff5d00a3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.058+0000 7ff5dde00700 1 --2- 172.21.15.90:0/876155934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5d0097720 0x7ff5d0097b30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.058+0000 7ff5dde00700 1 --2- 172.21.15.90:0/876155934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5d0098070 0x7ff5d00a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.058+0000 7ff5dde00700 1 -- 172.21.15.90:0/876155934 >> 172.21.15.90:0/876155934 conn(0x7ff5d000b050 msgr2=0x7ff5d000b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.058+0000 7ff5dde00700 1 -- 172.21.15.90:0/876155934 shutdown_connections 2024-05-27T07:56:26.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.058+0000 7ff5dde00700 1 -- 172.21.15.90:0/876155934 wait complete. 2024-05-27T07:56:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.059+0000 7ff5dde00700 1 Processor -- start 2024-05-27T07:56:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.060+0000 7ff5dde00700 1 -- start start 2024-05-27T07:56:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.060+0000 7ff5dde00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5d0097720 0x7ff5d0095410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.060+0000 7ff5dde00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5d0098070 0x7ff5d0095950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.060+0000 7ff5dde00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5d00a0b90 0x7ff5d0095e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.060+0000 7ff5dde00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d00a65e0 con 0x7ff5d0097720 2024-05-27T07:56:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.060+0000 7ff5dde00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d00a6460 con 0x7ff5d00a0b90 2024-05-27T07:56:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.060+0000 7ff5dde00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d00a6760 con 0x7ff5d0098070 2024-05-27T07:56:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.061+0000 7ff5dd400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5d00a0b90 0x7ff5d0095e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.061+0000 7ff5dd400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5d00a0b90 0x7ff5d0095e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56598/0 (socket says 172.21.15.90:56598) 2024-05-27T07:56:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.061+0000 7ff5dd400700 1 -- 172.21.15.90:0/482509001 learned_addr learned my addr 172.21.15.90:0/482509001 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.062+0000 7ff5dd400700 1 -- 172.21.15.90:0/482509001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5d0098070 msgr2=0x7ff5d0095950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T07:56:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.062+0000 7ff5dd400700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5d0098070 0x7ff5d0095950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.062+0000 7ff5dd400700 1 -- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5d0097720 msgr2=0x7ff5d0095410 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:56:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.062+0000 7ff5dca00700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5d0097720 0x7ff5d0095410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.062+0000 7ff5dd400700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5d0097720 0x7ff5d0095410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.062+0000 7ff5dd400700 1 -- 172.21.15.90:0/482509001 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d013b3d0 con 0x7ff5d00a0b90 2024-05-27T07:56:26.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.062+0000 7ff5dca00700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5d0097720 0x7ff5d0095410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.063+0000 7ff5dd400700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5d00a0b90 0x7ff5d0095e90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8052100 tx=0x7ff5d806bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.063+0000 7ff5d5600700 1 -- 172.21.15.90:0/482509001 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8084070 con 0x7ff5d00a0b90 2024-05-27T07:56:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.064+0000 7ff5dde00700 1 -- 172.21.15.90:0/482509001 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d013b6c0 con 0x7ff5d00a0b90 2024-05-27T07:56:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.064+0000 7ff5d5600700 1 -- 172.21.15.90:0/482509001 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5d807dc30 con 0x7ff5d00a0b90 2024-05-27T07:56:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.064+0000 7ff5d5600700 1 -- 172.21.15.90:0/482509001 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8080b80 con 0x7ff5d00a0b90 2024-05-27T07:56:26.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.064+0000 7ff5dde00700 1 -- 172.21.15.90:0/482509001 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d013bb80 con 0x7ff5d00a0b90 2024-05-27T07:56:26.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.075+0000 7ff5ede5db80 1 Processor -- start 2024-05-27T07:56:26.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.075+0000 7ff5d5600700 1 -- 172.21.15.90:0/482509001 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff5d807d7a0 con 0x7ff5d00a0b90 2024-05-27T07:56:26.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.076+0000 7ff5ede5db80 1 -- start start 2024-05-27T07:56:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.076+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62182aa70 0x55d6218be7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.076+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218bed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.076+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6218c8340 0x55d6218ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.076+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d6217ebec0 con 0x55d62173eec0 2024-05-27T07:56:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.076+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d6217dba60 con 0x55d6218c8340 2024-05-27T07:56:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.076+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d6217dfaa0 con 0x55d62182aa70 2024-05-27T07:56:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.077+0000 7ff5dca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218bed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.077+0000 7ff5dca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218bed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38562/0 (socket says 172.21.15.90:38562) 2024-05-27T07:56:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.077+0000 7ff5dca00700 1 -- 172.21.15.90:0/65539002 learned_addr learned my addr 172.21.15.90:0/65539002 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.077+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/65539002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62182aa70 0x55d6218be7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.078+0000 7ff5d5600700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff5c00b2a70 0x7ff5c00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.078+0000 7ff5d5600700 1 -- 172.21.15.90:0/482509001 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7ff5d81503f0 con 0x7ff5d00a0b90 2024-05-27T07:56:26.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.078+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/65539002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6218c8340 0x55d6218ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.079+0000 7ff5dca00700 1 -- 172.21.15.90:0/65539002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62182aa70 msgr2=0x55d6218be7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.079+0000 7ff5dca00700 1 --2- 172.21.15.90:0/65539002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62182aa70 0x55d6218be7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.079+0000 7ff5dca00700 1 -- 172.21.15.90:0/65539002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6218c8340 msgr2=0x55d6218ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.079+0000 7ff5dca00700 1 --2- 172.21.15.90:0/65539002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6218c8340 0x55d6218ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.079+0000 7ff5dca00700 1 -- 172.21.15.90:0/65539002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6217a44a0 con 0x55d62173eec0 2024-05-27T07:56:26.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.079+0000 7ff5dd400700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff5c00b2a70 0x7ff5c00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.080+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/65539002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6218c8340 0x55d6218ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:26.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.081+0000 7ff5dca00700 1 --2- 172.21.15.90:0/65539002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218bed10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc005ef0 tx=0x7ff5cc00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d3d8c4392c4cfd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.082+0000 7ff5dd400700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff5c00b2a70 0x7ff5c00b4f80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8052c50 tx=0x7ff5d806a340 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.083+0000 7ff5ca000700 1 -- 172.21.15.90:0/65539002 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00fdd0 con 0x55d62173eec0 2024-05-27T07:56:26.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.083+0000 7ff5ca000700 1 -- 172.21.15.90:0/65539002 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5cc00f8f0 con 0x55d62173eec0 2024-05-27T07:56:26.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.083+0000 7ff5ca000700 1 -- 172.21.15.90:0/65539002 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc0056e0 con 0x55d62173eec0 2024-05-27T07:56:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.085+0000 7ff5ede5db80 1 -- 172.21.15.90:0/65539002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 msgr2=0x55d6218bed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.086+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/65539002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218bed10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc005ef0 tx=0x7ff5cc00fbb0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.087+0000 7ff5ede5db80 1 -- 172.21.15.90:0/65539002 shutdown_connections 2024-05-27T07:56:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.087+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/65539002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d6218c8340 0x55d6218ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.087+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/65539002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62182aa70 0x55d6218be7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.087+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/65539002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218bed10 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.087+0000 7ff5ede5db80 1 -- 172.21.15.90:0/65539002 >> 172.21.15.90:0/65539002 conn(0x55d6217d8410 msgr2=0x55d6218b9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.087+0000 7ff5ede5db80 1 -- 172.21.15.90:0/65539002 shutdown_connections 2024-05-27T07:56:26.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.088+0000 7ff5ede5db80 1 -- 172.21.15.90:0/65539002 wait complete. 2024-05-27T07:56:26.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.089+0000 7ff5ede5db80 1 -- start start 2024-05-27T07:56:26.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.090+0000 7ff5ede5db80 1 Processor -- start 2024-05-27T07:56:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.091+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.091+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.091+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6218c8340 0x55d6218c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.091+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d6216e2020 con 0x55d62173eec0 2024-05-27T07:56:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.091+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d62177c7e0 con 0x55d62182aa70 2024-05-27T07:56:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.091+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d6218cae20 con 0x55d6218c8340 2024-05-27T07:56:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.091+0000 7ff5ede5db80 10 client.0 _ll_register_callbacks cb 0x55d621785cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.092+0000 7ff5d7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.093+0000 7ff5dca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.093+0000 7ff5dca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56616/0 (socket says 172.21.15.90:56616) 2024-05-27T07:56:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.093+0000 7ff5dca00700 1 -- 172.21.15.90:0/4079535475 learned_addr learned my addr 172.21.15.90:0/4079535475 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.093+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6218c8340 0x55d6218c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.094+0000 7ff5dca00700 1 -- 172.21.15.90:0/4079535475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6218c8340 msgr2=0x55d6218c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.094+0000 7ff5dca00700 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6218c8340 0x55d6218c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.094+0000 7ff5dca00700 1 -- 172.21.15.90:0/4079535475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 msgr2=0x55d6218c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.094+0000 7ff5dca00700 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.094+0000 7ff5dca00700 1 -- 172.21.15.90:0/4079535475 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d62172b950 con 0x55d62182aa70 2024-05-27T07:56:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.095+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.095+0000 7ff5dca00700 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218c5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00b790 tx=0x7ff5cc00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77b4d5ce23e6167 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.095+0000 7ff5bea00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:56:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.096+0000 7ff5bea00700 1 -- 172.21.15.90:0/4079535475 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc0056c0 con 0x55d62182aa70 2024-05-27T07:56:26.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.096+0000 7ff5bea00700 1 -- 172.21.15.90:0/4079535475 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7ff5cc00bcd0 con 0x55d62182aa70 2024-05-27T07:56:26.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.096+0000 7ff5ede5db80 1 -- 172.21.15.90:0/4079535475 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5cc00f8f0 con 0x55d62182aa70 2024-05-27T07:56:26.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.097+0000 7ff5bea00700 1 -- 172.21.15.90:0/4079535475 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7ff5cc036050 con 0x55d62182aa70 2024-05-27T07:56:26.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.097+0000 7ff5ede5db80 10 client.36881 fetch_fsmap learned FSMap version 63 2024-05-27T07:56:26.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.097+0000 7ff5ede5db80 1 -- 172.21.15.90:0/4079535475 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55d6217ac0e0 con 0x55d62182aa70 2024-05-27T07:56:26.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.098+0000 7ff5bea00700 1 -- 172.21.15.90:0/4079535475 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5cc00f8f0 con 0x55d62182aa70 2024-05-27T07:56:26.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.099+0000 7ff5bea00700 1 -- 172.21.15.90:0/4079535475 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 63) ==== 7152+0+0 (secure 0 0 0) 0x7ff5cc01f070 con 0x55d62182aa70 2024-05-27T07:56:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.099+0000 7ff5ede5db80 10 client.36881 fetch_fsmap finished waiting for FSMap version 63 2024-05-27T07:56:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.099+0000 7ff5ede5db80 10 client.36881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.099+0000 7ff5ede5db80 20 client.36881 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.099+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d6218e0110 0x55d6218e2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.100+0000 7ff5ede5db80 4 client.36881 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:26.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.100+0000 7ff5ede5db80 1 -- 172.21.15.90:0/4079535475 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6217ede10 con 0x55d6218e0110 2024-05-27T07:56:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.100+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d6218e0110 0x55d6218e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.101+0000 7ff5bea00700 10 client.36881 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.101+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d6218e0110 0x55d6218e2530 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d433b1f04b02eb2d server_cookie=a2932de84f0bb103 in_seq=0 out_seq=0 2024-05-27T07:56:26.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.102+0000 7ff5bea00700 1 -- 172.21.15.90:0/4079535475 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d6217ede10 con 0x55d6218e0110 2024-05-27T07:56:26.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.102+0000 7ff5bea00700 10 client.36881 handle_command_reply: tid=0 2024-05-27T07:56:26.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.102+0000 7ff5ede5db80 1 client.36881 shutdown 2024-05-27T07:56:26.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.103+0000 7ff5ede5db80 1 -- 172.21.15.90:0/4079535475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 msgr2=0x55d6218c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.103+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218c5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00b790 tx=0x7ff5cc00b870 comp rx=0 tx=0).stop 2024-05-27T07:56:26.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.104+0000 7ff5ede5db80 1 -- 172.21.15.90:0/4079535475 shutdown_connections 2024-05-27T07:56:26.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.104+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d6218e0110 0x55d6218e2530 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.104+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6218c8340 0x55d6218c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.104+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d62173eec0 0x55d6218c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.104+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/4079535475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218c5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.104+0000 7ff5ede5db80 1 -- 172.21.15.90:0/4079535475 >> 172.21.15.90:0/4079535475 conn(0x55d6217d8410 msgr2=0x55d6218b9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.104+0000 7ff5bea00700 0 client.36881 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:26.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.105+0000 7ff5ede5db80 1 -- 172.21.15.90:0/4079535475 shutdown_connections 2024-05-27T07:56:26.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.105+0000 7ff5ede5db80 1 -- 172.21.15.90:0/4079535475 wait complete. 2024-05-27T07:56:26.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.105+0000 7ff5ede5db80 20 client.36881 trim_cache size 0 max 16384 2024-05-27T07:56:26.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.149+0000 7ff5ede5db80 1 Processor -- start 2024-05-27T07:56:26.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.149+0000 7ff5ede5db80 1 -- start start 2024-05-27T07:56:26.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.150+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 0x55d6218b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.150+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218dc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.150+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6218c8340 0x55d6218beee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.150+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d013f130 con 0x55d6218c8340 2024-05-27T07:56:26.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.150+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0141a10 con 0x55d62182aa70 2024-05-27T07:56:26.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.150+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d013f800 con 0x55d62173eec0 2024-05-27T07:56:26.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.151+0000 7ff5dca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218dc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.151+0000 7ff5dca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218dc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56620/0 (socket says 172.21.15.90:56620) 2024-05-27T07:56:26.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.151+0000 7ff5dca00700 1 -- 172.21.15.90:0/3659753806 learned_addr learned my addr 172.21.15.90:0/3659753806 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:26.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.151+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/3659753806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 0x55d6218b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.151+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/3659753806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6218c8340 0x55d6218beee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.152+0000 7ff5dca00700 1 -- 172.21.15.90:0/3659753806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 msgr2=0x55d6218b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.152+0000 7ff5dca00700 1 --2- 172.21.15.90:0/3659753806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 0x55d6218b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.152+0000 7ff5dca00700 1 -- 172.21.15.90:0/3659753806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6218c8340 msgr2=0x55d6218beee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.152+0000 7ff5dca00700 1 --2- 172.21.15.90:0/3659753806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6218c8340 0x55d6218beee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.152+0000 7ff5dca00700 1 -- 172.21.15.90:0/3659753806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6218c37c0 con 0x55d62182aa70 2024-05-27T07:56:26.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.152+0000 7ff5dca00700 1 --2- 172.21.15.90:0/3659753806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218dc7e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009810 tx=0x7ff5cc009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5901a4a311ba6203 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.153+0000 7ff5c9600700 1 -- 172.21.15.90:0/3659753806 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00bcd0 con 0x55d62182aa70 2024-05-27T07:56:26.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.153+0000 7ff5c9600700 1 -- 172.21.15.90:0/3659753806 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5cc02d070 con 0x55d62182aa70 2024-05-27T07:56:26.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.153+0000 7ff5c9600700 1 -- 172.21.15.90:0/3659753806 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc0305b0 con 0x55d62182aa70 2024-05-27T07:56:26.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.154+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3659753806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 msgr2=0x55d6218dc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.154+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/3659753806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218dc7e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009810 tx=0x7ff5cc009290 comp rx=0 tx=0).stop 2024-05-27T07:56:26.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.155+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3659753806 shutdown_connections 2024-05-27T07:56:26.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.155+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/3659753806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6218c8340 0x55d6218beee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.155+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/3659753806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 0x55d6218b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.155+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/3659753806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218dc7e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.155+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3659753806 >> 172.21.15.90:0/3659753806 conn(0x55d6217d8410 msgr2=0x55d6218be7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.155+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3659753806 shutdown_connections 2024-05-27T07:56:26.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.155+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3659753806 wait complete. 2024-05-27T07:56:26.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.156+0000 7ff5ede5db80 1 -- start start 2024-05-27T07:56:26.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.156+0000 7ff5ede5db80 1 Processor -- start 2024-05-27T07:56:26.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.157+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 0x55d6218d49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.158+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.158+0000 7ff5ede5db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6218c8340 0x55d6218d54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.159+0000 7ff5d7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 0x55d6218d49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.159+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d013fed0 con 0x55d6218c8340 2024-05-27T07:56:26.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.159+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0141340 con 0x55d62182aa70 2024-05-27T07:56:26.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.159+0000 7ff5ede5db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01405a0 con 0x55d62173eec0 2024-05-27T07:56:26.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.159+0000 7ff5ede5db80 10 client.0 _ll_register_callbacks cb 0x55d621785cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:26.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.159+0000 7ff5dca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.159+0000 7ff5dca00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56628/0 (socket says 172.21.15.90:56628) 2024-05-27T07:56:26.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.159+0000 7ff5dca00700 1 -- 172.21.15.90:0/3586008357 learned_addr learned my addr 172.21.15.90:0/3586008357 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:26.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.160+0000 7ff5d7e00700 1 -- 172.21.15.90:0/3586008357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 msgr2=0x55d6218d4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.161+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218d4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.161+0000 7ff5d7e00700 1 -- 172.21.15.90:0/3586008357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6218c8340 msgr2=0x55d6218d54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:26.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.161+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6218c8340 0x55d6218d54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.161+0000 7ff5d7e00700 1 -- 172.21.15.90:0/3586008357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d621949d10 con 0x55d62173eec0 2024-05-27T07:56:26.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.161+0000 7ff5dca00700 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:56:26.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.162+0000 7ff5d7e00700 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 0x55d6218d49e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55d6218e2b50 tx=0x7ff5c4020610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c6209cc9f5c8417 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:26.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.162+0000 7ff5bea00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:56:26.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.163+0000 7ff5bea00700 1 -- 172.21.15.90:0/3586008357 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4004030 con 0x55d62173eec0 2024-05-27T07:56:26.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.163+0000 7ff5bea00700 1 -- 172.21.15.90:0/3586008357 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7ff5c40041d0 con 0x55d62173eec0 2024-05-27T07:56:26.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.163+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3586008357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5cc02d070 con 0x55d62173eec0 2024-05-27T07:56:26.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.164+0000 7ff5bea00700 1 -- 172.21.15.90:0/3586008357 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7ff5c40043d0 con 0x55d62173eec0 2024-05-27T07:56:26.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.164+0000 7ff5ede5db80 10 client.26938 fetch_fsmap learned FSMap version 63 2024-05-27T07:56:26.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.164+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3586008357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55d62194b010 con 0x55d62173eec0 2024-05-27T07:56:26.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.165+0000 7ff5bea00700 1 -- 172.21.15.90:0/3586008357 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5cc02d070 con 0x55d62173eec0 2024-05-27T07:56:26.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.165+0000 7ff5bea00700 1 -- 172.21.15.90:0/3586008357 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 63) ==== 7152+0+0 (secure 0 0 0) 0x7ff5c40124e0 con 0x55d62173eec0 2024-05-27T07:56:26.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.165+0000 7ff5ede5db80 10 client.26938 fetch_fsmap finished waiting for FSMap version 63 2024-05-27T07:56:26.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.165+0000 7ff5ede5db80 10 client.26938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:26.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.165+0000 7ff5ede5db80 20 client.26938 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:26.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.165+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d62194b680 0x55d62194daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:26.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.166+0000 7ff5ede5db80 4 client.26938 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:56:26.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.166+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3586008357 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d6217e75f0 con 0x55d62194b680 2024-05-27T07:56:26.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.166+0000 7ff5dca00700 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d62194b680 0x55d62194daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:26.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.167+0000 7ff5bea00700 10 client.26938 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:26.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.167+0000 7ff5dca00700 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d62194b680 0x55d62194daa0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead8431a8cddb830 server_cookie=8028296e98d1d471 in_seq=0 out_seq=0 2024-05-27T07:56:26.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.169+0000 7ff5bea00700 1 -- 172.21.15.90:0/3586008357 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d6217e75f0 con 0x55d62194b680 2024-05-27T07:56:26.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.169+0000 7ff5bea00700 10 client.26938 handle_command_reply: tid=0 2024-05-27T07:56:26.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.169+0000 7ff5ede5db80 1 client.26938 shutdown 2024-05-27T07:56:26.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.169+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3586008357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 msgr2=0x55d6218d49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.169+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 0x55d6218d49e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55d6218e2b50 tx=0x7ff5c4020610 comp rx=0 tx=0).stop 2024-05-27T07:56:26.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.170+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3586008357 shutdown_connections 2024-05-27T07:56:26.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.170+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d62194b680 0x55d62194daa0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.170+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6218c8340 0x55d6218d54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.170+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d62173eec0 0x55d6218d49e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.170+0000 7ff5ede5db80 1 --2- 172.21.15.90:0/3586008357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d62182aa70 0x55d6218d4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.170+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3586008357 >> 172.21.15.90:0/3586008357 conn(0x55d6217d8410 msgr2=0x55d6218bd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.170+0000 7ff5bea00700 0 client.26938 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:26.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.170+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3586008357 shutdown_connections 2024-05-27T07:56:26.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.170+0000 7ff5ede5db80 1 -- 172.21.15.90:0/3586008357 wait complete. 2024-05-27T07:56:26.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.171+0000 7ff5ede5db80 20 client.26938 trim_cache size 0 max 16384 2024-05-27T07:56:26.173 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:56:26.174 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:56:26.174 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "ba86ce88-5878-4393-b662-bd7e8498ce45", 2024-05-27T07:56:26.174 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:56:26.174 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:56:26.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.172+0000 7ff5dde00700 1 -- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff5c00b2a70 msgr2=0x7ff5c00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.172+0000 7ff5dde00700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff5c00b2a70 0x7ff5c00b4f80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8052c50 tx=0x7ff5d806a340 comp rx=0 tx=0).stop 2024-05-27T07:56:26.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.172+0000 7ff5dde00700 1 -- 172.21.15.90:0/482509001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5d00a0b90 msgr2=0x7ff5d0095e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:26.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.172+0000 7ff5dde00700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5d00a0b90 0x7ff5d0095e90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8052100 tx=0x7ff5d806bc10 comp rx=0 tx=0).stop 2024-05-27T07:56:26.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.172+0000 7ff5dde00700 1 -- 172.21.15.90:0/482509001 shutdown_connections 2024-05-27T07:56:26.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.172+0000 7ff5dde00700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff5c00b2a70 0x7ff5c00b4f80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.172+0000 7ff5dde00700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5d00a0b90 0x7ff5d0095e90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.172+0000 7ff5dde00700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5d0097720 0x7ff5d0095410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.172+0000 7ff5dde00700 1 --2- 172.21.15.90:0/482509001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5d0098070 0x7ff5d0095950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:26.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.172+0000 7ff5dde00700 1 -- 172.21.15.90:0/482509001 >> 172.21.15.90:0/482509001 conn(0x7ff5d000b050 msgr2=0x7ff5d00924d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:26.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.173+0000 7ff5dde00700 1 -- 172.21.15.90:0/482509001 shutdown_connections 2024-05-27T07:56:26.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:26.174+0000 7ff5dde00700 1 -- 172.21.15.90:0/482509001 wait complete. 2024-05-27T07:56:26.616 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:26 smithi129 ceph-mon[35254]: pgmap v318: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-05-27T07:56:26.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:26 smithi090 ceph-mon[31429]: pgmap v318: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-05-27T07:56:26.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:26 smithi136 ceph-mon[35324]: pgmap v318: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-05-27T07:56:27.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:27 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:56:27.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:27 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:27.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:27 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:56:27.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:27 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:27.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:27 smithi090 ceph-mon[31429]: pgmap v319: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.6 MiB/s wr, 480 op/s 2024-05-27T07:56:27.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:27 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:56:27.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:27 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:27.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:27 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:56:27.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:27 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:27.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:27 smithi136 ceph-mon[35324]: pgmap v319: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.6 MiB/s wr, 480 op/s 2024-05-27T07:56:27.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:27 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:56:27.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:27 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:27.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:27 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:56:27.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:27 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:56:27.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:27 smithi129 ceph-mon[35254]: pgmap v319: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 4.6 MiB/s wr, 480 op/s 2024-05-27T07:56:28.727 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:56:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 370.0 s, 572.6 tps, lat 175.401 ms stddev 98.764 2024-05-27T07:56:29.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:28 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:56:29.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:28 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:56:29.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:28 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:56:29.908 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:30.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:29 smithi090 ceph-mon[31429]: pgmap v320: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.7 MiB/s wr, 681 op/s 2024-05-27T07:56:30.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:30 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 6. 2024-05-27T07:56:30.214 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:30 smithi090 systemd[1]: Stopped Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:56:30.215 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:30 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Consumed 1.566s CPU time. 2024-05-27T07:56:30.215 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:30 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Start request repeated too quickly. 2024-05-27T07:56:30.215 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:30 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-05-27T07:56:30.215 INFO:journalctl@ceph.mds.j.smithi090.stdout:May 27 07:56:30 smithi090 systemd[1]: Failed to start Ceph mds.j for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T07:56:30.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:29 smithi136 ceph-mon[35324]: pgmap v320: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.7 MiB/s wr, 681 op/s 2024-05-27T07:56:30.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:29 smithi129 ceph-mon[35254]: pgmap v320: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.7 MiB/s wr, 681 op/s 2024-05-27T07:56:32.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:32 smithi129 ceph-mon[35254]: pgmap v321: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 MiB/s wr, 567 op/s 2024-05-27T07:56:32.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:32 smithi129 ceph-mon[35254]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:56:32.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:32 smithi129 ceph-mon[35254]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-05-27T07:56:32.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:32 smithi090 ceph-mon[31429]: pgmap v321: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 MiB/s wr, 567 op/s 2024-05-27T07:56:32.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:32 smithi090 ceph-mon[31429]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:56:32.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:32 smithi090 ceph-mon[31429]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-05-27T07:56:32.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:32 smithi136 ceph-mon[35324]: pgmap v321: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 MiB/s wr, 567 op/s 2024-05-27T07:56:32.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:32 smithi136 ceph-mon[35324]: MDS daemon mds.j is removed because it is dead or otherwise unavailable. 2024-05-27T07:56:32.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:32 smithi136 ceph-mon[35324]: fsmap cephfs:5 4 up:standby-replay 5 up:active 2024-05-27T07:56:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 375.0 s, 458.6 tps, lat 213.651 ms stddev 162.835 2024-05-27T07:56:34.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:34 smithi129 ceph-mon[35254]: pgmap v322: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 MiB/s wr, 581 op/s 2024-05-27T07:56:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.699+0000 7f3aaca00640 1 -- 172.21.15.90:0/4206674225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3aa8104210 msgr2=0x7f3aa81105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.699+0000 7f3aaca00640 1 --2- 172.21.15.90:0/4206674225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3aa8104210 0x7f3aa81105e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009ca0 tx=0x7f3aa002a8c0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.699+0000 7f3aaca00640 1 -- 172.21.15.90:0/4206674225 shutdown_connections 2024-05-27T07:56:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.699+0000 7f3aaca00640 1 --2- 172.21.15.90:0/4206674225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3aa8104210 0x7f3aa81105e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.700+0000 7f3aaca00640 1 --2- 172.21.15.90:0/4206674225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa8109f60 0x7f3aa8103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.700+0000 7f3aaca00640 1 --2- 172.21.15.90:0/4206674225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3aa8109ac0 0x7f3aa8103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.700+0000 7f3aaca00640 1 -- 172.21.15.90:0/4206674225 >> 172.21.15.90:0/4206674225 conn(0x7f3aa80ff180 msgr2=0x7f3aa81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:34.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.700+0000 7f3aaca00640 1 -- 172.21.15.90:0/4206674225 shutdown_connections 2024-05-27T07:56:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.701+0000 7f3aaca00640 1 -- 172.21.15.90:0/4206674225 wait complete. 2024-05-27T07:56:34.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.702+0000 7f3aaca00640 1 Processor -- start 2024-05-27T07:56:34.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.703+0000 7f3aaca00640 1 -- start start 2024-05-27T07:56:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.704+0000 7f3aaca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa8104210 0x7f3aa810d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.704+0000 7f3aaca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3aa8109ac0 0x7f3aa810ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.705+0000 7f3aaca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3aa8109f60 0x7f3aa810e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.705+0000 7f3aa7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa8104210 0x7f3aa810d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.705+0000 7f3aa7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa8104210 0x7f3aa810d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46646/0 (socket says 172.21.15.90:46646) 2024-05-27T07:56:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.705+0000 7f3aaca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa8077860 con 0x7f3aa8104210 2024-05-27T07:56:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.705+0000 7f3aa7400640 1 -- 172.21.15.90:0/2820423444 learned_addr learned my addr 172.21.15.90:0/2820423444 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.705+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3aa8109ac0 0x7f3aa810ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.706+0000 7f3aaca00640 1 -- 172.21.15.90:0/2820423444 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa80776e0 con 0x7f3aa8109f60 2024-05-27T07:56:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.706+0000 7f3aaca00640 1 -- 172.21.15.90:0/2820423444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa80779e0 con 0x7f3aa8109ac0 2024-05-27T07:56:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.706+0000 7f3aa7e00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3aa8109f60 0x7f3aa810e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.706+0000 7f3aa6a00640 1 -- 172.21.15.90:0/2820423444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3aa8109f60 msgr2=0x7f3aa810e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.706+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3aa8109f60 0x7f3aa810e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.707+0000 7f3aa6a00640 1 -- 172.21.15.90:0/2820423444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa8104210 msgr2=0x7f3aa810d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.707+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa8104210 0x7f3aa810d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.707+0000 7f3aa6a00640 1 -- 172.21.15.90:0/2820423444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa81acce0 con 0x7f3aa8109ac0 2024-05-27T07:56:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.707+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3aa8109ac0 0x7f3aa810ddb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3aa806b4a0 tx=0x7f3a9800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.708+0000 7f3a9fe00640 1 -- 172.21.15.90:0/2820423444 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800cc40 con 0x7f3aa8109ac0 2024-05-27T07:56:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.708+0000 7f3a9fe00640 1 -- 172.21.15.90:0/2820423444 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a980044b0 con 0x7f3aa8109ac0 2024-05-27T07:56:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.708+0000 7f3a9fe00640 1 -- 172.21.15.90:0/2820423444 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98003e10 con 0x7f3aa8109ac0 2024-05-27T07:56:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.708+0000 7f3aaca00640 1 -- 172.21.15.90:0/2820423444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa81acfd0 con 0x7f3aa8109ac0 2024-05-27T07:56:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.708+0000 7f3aaca00640 1 -- 172.21.15.90:0/2820423444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa81ad3e0 con 0x7f3aa8109ac0 2024-05-27T07:56:34.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:34 smithi090 ceph-mon[31429]: pgmap v322: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 MiB/s wr, 581 op/s 2024-05-27T07:56:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.717+0000 7f3ab4f8d740 1 Processor -- start 2024-05-27T07:56:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.717+0000 7f3ab4f8d740 1 -- start start 2024-05-27T07:56:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.718+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fbfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.718+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fbffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.718+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fc04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.718+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eeb2f30b90 con 0x55eeb2bf42f0 2024-05-27T07:56:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.718+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55eeb2f20730 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.718+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eeb2f24770 con 0x55eeb2c1d1b0 2024-05-27T07:56:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.719+0000 7f3a9fe00640 1 -- 172.21.15.90:0/2820423444 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3a98004000 con 0x7f3aa8109ac0 2024-05-27T07:56:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.719+0000 7f3aa7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fbfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.719+0000 7f3aa7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fbfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57840/0 (socket says 172.21.15.90:57840) 2024-05-27T07:56:34.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.719+0000 7f3aa7e00640 1 -- 172.21.15.90:0/2328748961 learned_addr learned my addr 172.21.15.90:0/2328748961 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.721+0000 7f3aa7400640 1 --2- 172.21.15.90:0/2328748961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fbffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.721+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/2328748961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fc04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.721+0000 7f3a9fe00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3a8c0b2a50 0x7f3a8c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.721+0000 7f3a9fe00640 1 -- 172.21.15.90:0/2820423444 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f3a980e2d40 con 0x7f3aa8109ac0 2024-05-27T07:56:34.723 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.722+0000 7f3aa7e00640 1 -- 172.21.15.90:0/2328748961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 msgr2=0x55eeb2fc04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.722+0000 7f3aa7e00640 1 --2- 172.21.15.90:0/2328748961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fc04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.724 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.722+0000 7f3aa7e00640 1 -- 172.21.15.90:0/2328748961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 msgr2=0x55eeb2fbffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.722+0000 7f3aa7e00640 1 --2- 172.21.15.90:0/2328748961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fbffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.722+0000 7f3aa7e00640 1 -- 172.21.15.90:0/2328748961 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb2eb98b0 con 0x55eeb2c1d1b0 2024-05-27T07:56:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.723+0000 7f3aa7e00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3a8c0b2a50 0x7f3a8c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.725+0000 7f3aa7e00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3a8c0b2a50 0x7f3a8c0b4f60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0044000 tx=0x7f3aa0046000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.726+0000 7f3aa7e00640 1 --2- 172.21.15.90:0/2328748961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fbfa60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3aa002aeb0 tx=0x7f3aa0031450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a85a6146e9fd277 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.727+0000 7f3a93e00640 1 -- 172.21.15.90:0/2328748961 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0042040 con 0x55eeb2c1d1b0 2024-05-27T07:56:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.727+0000 7f3a93e00640 1 -- 172.21.15.90:0/2328748961 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3aa0036070 con 0x55eeb2c1d1b0 2024-05-27T07:56:34.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.727+0000 7f3a93e00640 1 -- 172.21.15.90:0/2328748961 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa003c8a0 con 0x55eeb2c1d1b0 2024-05-27T07:56:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.730+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/2328748961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2c1d1b0 msgr2=0x55eeb2fbfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.730+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/2328748961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fbfa60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3aa002aeb0 tx=0x7f3aa0031450 comp rx=0 tx=0).stop 2024-05-27T07:56:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.732+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/2328748961 shutdown_connections 2024-05-27T07:56:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.732+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/2328748961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fc04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.732+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/2328748961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fbffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.732+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/2328748961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fbfa60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.732+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/2328748961 >> 172.21.15.90:0/2328748961 conn(0x55eeb2c11300 msgr2=0x55eeb2fbc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.732+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/2328748961 shutdown_connections 2024-05-27T07:56:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.734+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/2328748961 wait complete. 2024-05-27T07:56:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.734+0000 7f3ab4f8d740 1 -- start start 2024-05-27T07:56:34.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.735+0000 7f3ab4f8d740 1 Processor -- start 2024-05-27T07:56:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.736+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fc9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.737+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.737+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fd5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.738+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eeb2c06790 con 0x55eeb2bf42f0 2024-05-27T07:56:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.738+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55eeb2ec8bc0 con 0x55eeb2c1d1b0 2024-05-27T07:56:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.738+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eeb2fcc9b0 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.738+0000 7f3aa6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.738+0000 7f3aa6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57842/0 (socket says 172.21.15.90:57842) 2024-05-27T07:56:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.738+0000 7f3aa6a00640 1 -- 172.21.15.90:0/1500053016 learned_addr learned my addr 172.21.15.90:0/1500053016 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.738+0000 7f3aa7e00640 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fd5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.739+0000 7f3aa6a00640 1 -- 172.21.15.90:0/1500053016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2c1d1b0 msgr2=0x55eeb2fd5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.739+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fd5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.739+0000 7f3aa6a00640 1 -- 172.21.15.90:0/1500053016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 msgr2=0x55eeb2fc9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.738+0000 7f3ab4f8d740 10 client.0 _ll_register_callbacks cb 0x55eeb2ea6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.740+0000 7f3aa7400640 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fc9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.740+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fc9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.740+0000 7f3aa6a00640 1 -- 172.21.15.90:0/1500053016 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb2c35210 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.740+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/1500053016 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa0036070 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.741+0000 7f3aa7400640 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fc9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.741+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fca180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a980a9b80 tx=0x7f3a980a5bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eb31066812898dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.741+0000 7f3a90c00640 10 client.26950 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.741+0000 7f3a90c00640 1 -- 172.21.15.90:0/1500053016 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a980e4070 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.741+0000 7f3a90c00640 1 -- 172.21.15.90:0/1500053016 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f3a980a9050 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.742+0000 7f3a90c00640 1 -- 172.21.15.90:0/1500053016 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3a980a5380 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.742+0000 7f3ab4f8d740 10 client.26950 fetch_fsmap learned FSMap version 64 2024-05-27T07:56:34.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.742+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/1500053016 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=64}) -- 0x55eeb2d01620 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.743+0000 7f3a90c00640 1 -- 172.21.15.90:0/1500053016 <== mon.1 v2:172.21.15.136:3300/0 4 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f3a980a5560 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.744+0000 7f3ab4f8d740 10 client.26950 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:56:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.744+0000 7f3ab4f8d740 10 client.26950 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.744+0000 7f3ab4f8d740 20 client.26950 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.744+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eeb2bdbd80 0x55eeb2fdbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.744+0000 7f3ab4f8d740 4 client.26950 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:34.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.744+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/1500053016 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeb2f32ae0 con 0x55eeb2bdbd80 2024-05-27T07:56:34.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.745+0000 7f3aa7400640 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eeb2bdbd80 0x55eeb2fdbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.747+0000 7f3aa7400640 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eeb2bdbd80 0x55eeb2fdbaa0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf78d1618c4cd83 server_cookie=5f37bf2f8cf43c25 in_seq=0 out_seq=0 2024-05-27T07:56:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.747+0000 7f3a90c00640 10 client.26950 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:34.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:34 smithi136 ceph-mon[35324]: pgmap v322: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 4.6 MiB/s wr, 581 op/s 2024-05-27T07:56:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.749+0000 7f3a90c00640 1 -- 172.21.15.90:0/1500053016 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eeb2f32ae0 con 0x55eeb2bdbd80 2024-05-27T07:56:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.749+0000 7f3a90c00640 10 client.26950 handle_command_reply: tid=0 2024-05-27T07:56:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.749+0000 7f3ab4f8d740 1 client.26950 shutdown 2024-05-27T07:56:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.751+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/1500053016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bfc4d0 msgr2=0x55eeb2fca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.751+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fca180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3a980a9b80 tx=0x7f3a980a5bd0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.751+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/1500053016 shutdown_connections 2024-05-27T07:56:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.751+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eeb2bdbd80 0x55eeb2fdbaa0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.751+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fd5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.751+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fca180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.751+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/1500053016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fc9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.751+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/1500053016 >> 172.21.15.90:0/1500053016 conn(0x55eeb2c11300 msgr2=0x55eeb2fbf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.751+0000 7f3a90c00640 0 client.26950 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.753+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/1500053016 shutdown_connections 2024-05-27T07:56:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.754+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/1500053016 wait complete. 2024-05-27T07:56:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.756+0000 7f3ab4f8d740 20 client.26950 trim_cache size 0 max 16384 2024-05-27T07:56:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.785+0000 7f3ab4f8d740 1 Processor -- start 2024-05-27T07:56:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.785+0000 7f3ab4f8d740 1 -- start start 2024-05-27T07:56:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.786+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fc5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.786+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fcc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.786+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.786+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81b0850 con 0x55eeb2c1d1b0 2024-05-27T07:56:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.786+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81ad8a0 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.786+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81afab0 con 0x55eeb2bf42f0 2024-05-27T07:56:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.786+0000 7f3aa7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.786+0000 7f3aa7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fba810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46660/0 (socket says 172.21.15.90:46660) 2024-05-27T07:56:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.786+0000 7f3aa7e00640 1 -- 172.21.15.90:0/675606760 learned_addr learned my addr 172.21.15.90:0/675606760 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.786+0000 7f3aa6a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fcc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.787+0000 7f3aa7400640 1 --2- 172.21.15.90:0/675606760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fc5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.787+0000 7f3aa6a00640 1 -- 172.21.15.90:0/675606760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bf42f0 msgr2=0x55eeb2fc5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.787+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/675606760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fc5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.787+0000 7f3aa6a00640 1 -- 172.21.15.90:0/675606760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 msgr2=0x55eeb2fba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.787+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/675606760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.787+0000 7f3aa6a00640 1 -- 172.21.15.90:0/675606760 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb2fc0b60 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.788+0000 7f3aa6a00640 1 --2- 172.21.15.90:0/675606760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fcc370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a980e0d60 tx=0x7f3a980cc050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0d0c7593296f58 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.788+0000 7f3a93400640 1 -- 172.21.15.90:0/675606760 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a980a9360 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.788+0000 7f3a93400640 1 -- 172.21.15.90:0/675606760 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a980e0e50 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.788+0000 7f3a93400640 1 -- 172.21.15.90:0/675606760 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a980adfa0 con 0x55eeb2bfc4d0 2024-05-27T07:56:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.790+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/675606760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 msgr2=0x55eeb2fcc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.790+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/675606760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fcc370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a980e0d60 tx=0x7f3a980cc050 comp rx=0 tx=0).stop 2024-05-27T07:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.790+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/675606760 shutdown_connections 2024-05-27T07:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.790+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/675606760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bfc4d0 0x55eeb2fcc370 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a980e0d60 tx=0x7f3a980cc050 comp rx=0 tx=0).stop 2024-05-27T07:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.790+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/675606760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.790+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/675606760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fc5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.790+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/675606760 >> 172.21.15.90:0/675606760 conn(0x55eeb2c11300 msgr2=0x55eeb2fc0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.791+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/675606760 shutdown_connections 2024-05-27T07:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.791+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/675606760 wait complete. 2024-05-27T07:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.792+0000 7f3ab4f8d740 1 -- start start 2024-05-27T07:56:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.792+0000 7f3ab4f8d740 1 Processor -- start 2024-05-27T07:56:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.793+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fd20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.793+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fcd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.794+0000 7f3ab4f8d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bb11c0 0x55eeb2fcd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.794+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81af3e0 con 0x55eeb2c1d1b0 2024-05-27T07:56:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.794+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81adf70 con 0x55eeb2bb11c0 2024-05-27T07:56:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.794+0000 7f3ab4f8d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81ae640 con 0x55eeb2bf42f0 2024-05-27T07:56:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.794+0000 7f3ab4f8d740 10 client.0 _ll_register_callbacks cb 0x7f3a980a62c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.794+0000 7f3aa7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bb11c0 0x55eeb2fcd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.794+0000 7f3aa7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bb11c0 0x55eeb2fcd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:36254/0 (socket says 172.21.15.90:36254) 2024-05-27T07:56:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.794+0000 7f3aa6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fcd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.794+0000 7f3aa7e00640 1 -- 172.21.15.90:0/3889816955 learned_addr learned my addr 172.21.15.90:0/3889816955 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.797+0000 7f3aa7e00640 1 -- 172.21.15.90:0/3889816955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bf42f0 msgr2=0x55eeb2fd20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.797+0000 7f3aa7e00640 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fd20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.797+0000 7f3aa7e00640 1 -- 172.21.15.90:0/3889816955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 msgr2=0x55eeb2fcd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.797+0000 7f3aa7e00640 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fcd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.797+0000 7f3aa7e00640 1 -- 172.21.15.90:0/3889816955 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb2fdbe60 con 0x55eeb2bb11c0 2024-05-27T07:56:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.799+0000 7f3a90c00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:56:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.799+0000 7f3aa7e00640 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bb11c0 0x55eeb2fcd980 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3aa002ade0 tx=0x7f3aa00315c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d236d556d30aa58 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.801+0000 7f3a90c00640 1 -- 172.21.15.90:0/3889816955 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa005e060 con 0x55eeb2bb11c0 2024-05-27T07:56:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.801+0000 7f3a90c00640 1 -- 172.21.15.90:0/3889816955 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f3aa003cdb0 con 0x55eeb2bb11c0 2024-05-27T07:56:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.801+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/3889816955 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a980e0e50 con 0x55eeb2bb11c0 2024-05-27T07:56:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.804+0000 7f3a90c00640 1 -- 172.21.15.90:0/3889816955 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3aa0042040 con 0x55eeb2bb11c0 2024-05-27T07:56:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.804+0000 7f3ab4f8d740 10 client.36905 fetch_fsmap learned FSMap version 64 2024-05-27T07:56:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.804+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/3889816955 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55eeb3043c40 con 0x55eeb2bb11c0 2024-05-27T07:56:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.806+0000 7f3a90c00640 1 -- 172.21.15.90:0/3889816955 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a980e0e50 con 0x55eeb2bb11c0 2024-05-27T07:56:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.806+0000 7f3a90c00640 1 -- 172.21.15.90:0/3889816955 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f3aa00596e0 con 0x55eeb2bb11c0 2024-05-27T07:56:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.806+0000 7f3ab4f8d740 10 client.36905 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:56:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.806+0000 7f3ab4f8d740 10 client.36905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.806+0000 7f3ab4f8d740 20 client.36905 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.806+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eeb3044170 0x55eeb3046590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.806+0000 7f3ab4f8d740 4 client.36905 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:56:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.806+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/3889816955 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eeb2f2c2c0 con 0x55eeb3044170 2024-05-27T07:56:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.809+0000 7f3aa7400640 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eeb3044170 0x55eeb3046590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.813+0000 7f3aa7400640 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eeb3044170 0x55eeb3046590 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed86b59297f3c42 server_cookie=1f671b0ed6a6e133 in_seq=0 out_seq=0 2024-05-27T07:56:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.813+0000 7f3a90c00640 10 client.36905 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.865+0000 7f3a90c00640 1 -- 172.21.15.90:0/3889816955 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55eeb2f2c2c0 con 0x55eeb3044170 2024-05-27T07:56:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.865+0000 7f3a90c00640 10 client.36905 handle_command_reply: tid=0 2024-05-27T07:56:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.865+0000 7f3ab4f8d740 1 client.36905 shutdown 2024-05-27T07:56:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.866+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/3889816955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bb11c0 msgr2=0x55eeb2fcd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.866+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bb11c0 0x55eeb2fcd980 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3aa002ade0 tx=0x7f3aa00315c0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.866+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/3889816955 shutdown_connections 2024-05-27T07:56:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.866+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eeb3044170 0x55eeb3046590 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.866+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eeb2bb11c0 0x55eeb2fcd980 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.866+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eeb2c1d1b0 0x55eeb2fcd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.866+0000 7f3ab4f8d740 1 --2- 172.21.15.90:0/3889816955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eeb2bf42f0 0x55eeb2fd20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.866+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/3889816955 >> 172.21.15.90:0/3889816955 conn(0x55eeb2c11300 msgr2=0x55eeb2fbff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.866+0000 7f3a90c00640 0 client.36905 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.868+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/3889816955 shutdown_connections 2024-05-27T07:56:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.868+0000 7f3ab4f8d740 1 -- 172.21.15.90:0/3889816955 wait complete. 2024-05-27T07:56:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.869+0000 7f3ab4f8d740 20 client.36905 trim_cache size 0 max 16384 2024-05-27T07:56:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:56:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:56:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:56:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.871+0000 7f3aaca00640 1 -- 172.21.15.90:0/2820423444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3a8c0b2a50 msgr2=0x7f3a8c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.871+0000 7f3aaca00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3a8c0b2a50 0x7f3a8c0b4f60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0044000 tx=0x7f3aa0046000 comp rx=0 tx=0).stop 2024-05-27T07:56:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.871+0000 7f3aaca00640 1 -- 172.21.15.90:0/2820423444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3aa8109ac0 msgr2=0x7f3aa810ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.871+0000 7f3aaca00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3aa8109ac0 0x7f3aa810ddb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3aa806b4a0 tx=0x7f3a9800ee50 comp rx=0 tx=0).stop 2024-05-27T07:56:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.871+0000 7f3aaca00640 1 -- 172.21.15.90:0/2820423444 shutdown_connections 2024-05-27T07:56:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.872+0000 7f3aaca00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3a8c0b2a50 0x7f3a8c0b4f60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.872+0000 7f3aaca00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3aa8109f60 0x7f3aa810e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.872+0000 7f3aaca00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3aa8109ac0 0x7f3aa810ddb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.872+0000 7f3aaca00640 1 --2- 172.21.15.90:0/2820423444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa8104210 0x7f3aa810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.872+0000 7f3aaca00640 1 -- 172.21.15.90:0/2820423444 >> 172.21.15.90:0/2820423444 conn(0x7f3aa80ff180 msgr2=0x7f3aa8106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.872+0000 7f3aaca00640 1 -- 172.21.15.90:0/2820423444 shutdown_connections 2024-05-27T07:56:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:34.872+0000 7f3aaca00640 1 -- 172.21.15.90:0/2820423444 wait complete. 2024-05-27T07:56:35.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.017+0000 7ff875400700 1 -- 172.21.15.90:0/3132596735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8680965c0 msgr2=0x7ff8680969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:35.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.017+0000 7ff875400700 1 --2- 172.21.15.90:0/3132596735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8680965c0 0x7ff8680969d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff8640098f0 tx=0x7ff864009c00 comp rx=0 tx=0).stop 2024-05-27T07:56:35.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.018+0000 7ff875400700 1 -- 172.21.15.90:0/3132596735 shutdown_connections 2024-05-27T07:56:35.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.018+0000 7ff875400700 1 --2- 172.21.15.90:0/3132596735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff868096f10 0x7ff8680a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.018+0000 7ff875400700 1 --2- 172.21.15.90:0/3132596735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff86809c5e0 0x7ff86809c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.018+0000 7ff875400700 1 --2- 172.21.15.90:0/3132596735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8680965c0 0x7ff8680969d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.018+0000 7ff875400700 1 -- 172.21.15.90:0/3132596735 >> 172.21.15.90:0/3132596735 conn(0x7ff86800d2f0 msgr2=0x7ff86800d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:35.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.022+0000 7ff875400700 1 -- 172.21.15.90:0/3132596735 shutdown_connections 2024-05-27T07:56:35.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.024+0000 7ff875400700 1 -- 172.21.15.90:0/3132596735 wait complete. 2024-05-27T07:56:35.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.025+0000 7ff875400700 1 Processor -- start 2024-05-27T07:56:35.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.025+0000 7ff875400700 1 -- start start 2024-05-27T07:56:35.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.027+0000 7ff875400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8680965c0 0x7ff86812ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.027+0000 7ff875400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff868096f10 0x7ff86812b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.027+0000 7ff875400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff86809c5e0 0x7ff86812f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.027+0000 7ff875400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff8680a4430 con 0x7ff86809c5e0 2024-05-27T07:56:35.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.027+0000 7ff875400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8680a42b0 con 0x7ff8680965c0 2024-05-27T07:56:35.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.027+0000 7ff875400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff8680a45b0 con 0x7ff868096f10 2024-05-27T07:56:35.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.028+0000 7ff86f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff868096f10 0x7ff86812b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.028+0000 7ff874a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff86809c5e0 0x7ff86812f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.028+0000 7ff86f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff868096f10 0x7ff86812b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57868/0 (socket says 172.21.15.90:57868) 2024-05-27T07:56:35.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.028+0000 7ff86f400700 1 -- 172.21.15.90:0/1601106863 learned_addr learned my addr 172.21.15.90:0/1601106863 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:35.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.029+0000 7ff86f400700 1 -- 172.21.15.90:0/1601106863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8680965c0 msgr2=0x7ff86812ae70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:56:35.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.029+0000 7ff86f400700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8680965c0 0x7ff86812ae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.029+0000 7ff86f400700 1 -- 172.21.15.90:0/1601106863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff86809c5e0 msgr2=0x7ff86812f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:35.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.029+0000 7ff86f400700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff86809c5e0 0x7ff86812f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.029+0000 7ff86f400700 1 -- 172.21.15.90:0/1601106863 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86812fc90 con 0x7ff868096f10 2024-05-27T07:56:35.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.029+0000 7ff874a00700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff86809c5e0 0x7ff86812f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:56:35.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.030+0000 7ff86f400700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff868096f10 0x7ff86812b410 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff864009230 tx=0x7ff86400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:35.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.030+0000 7ff86cc00700 1 -- 172.21.15.90:0/1601106863 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86400b790 con 0x7ff868096f10 2024-05-27T07:56:35.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.031+0000 7ff86cc00700 1 -- 172.21.15.90:0/1601106863 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8640054d0 con 0x7ff868096f10 2024-05-27T07:56:35.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.031+0000 7ff86cc00700 1 -- 172.21.15.90:0/1601106863 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864003a40 con 0x7ff868096f10 2024-05-27T07:56:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.033+0000 7ff875400700 1 -- 172.21.15.90:0/1601106863 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86812ff20 con 0x7ff868096f10 2024-05-27T07:56:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.033+0000 7ff875400700 1 -- 172.21.15.90:0/1601106863 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff868130320 con 0x7ff868096f10 2024-05-27T07:56:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.044+0000 7ff86cc00700 1 -- 172.21.15.90:0/1601106863 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff86400b930 con 0x7ff868096f10 2024-05-27T07:56:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.046+0000 7ff86cc00700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff8580b2a50 0x7ff8580b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.046+0000 7ff88551cb80 1 Processor -- start 2024-05-27T07:56:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.046+0000 7ff86cc00700 1 -- 172.21.15.90:0/1601106863 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7ff8640e3480 con 0x7ff868096f10 2024-05-27T07:56:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.046+0000 7ff88551cb80 1 -- start start 2024-05-27T07:56:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.047+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c91f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.047+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 0x5608c91feb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.047+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c9201060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.047+0000 7ff88551cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5608c9122e40 con 0x5608c91f43a0 2024-05-27T07:56:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.047+0000 7ff88551cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5608c91129e0 con 0x5608c91f47b0 2024-05-27T07:56:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.047+0000 7ff88551cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5608c9116a20 con 0x5608c9075e40 2024-05-27T07:56:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.047+0000 7ff86fe00700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff8580b2a50 0x7ff8580b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.047+0000 7ff874a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c9201060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.047+0000 7ff874a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c9201060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:36282/0 (socket says 172.21.15.90:36282) 2024-05-27T07:56:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.047+0000 7ff874a00700 1 -- 172.21.15.90:0/480748038 learned_addr learned my addr 172.21.15.90:0/480748038 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.048+0000 7ff874a00700 1 -- 172.21.15.90:0/480748038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 msgr2=0x5608c91f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.048+0000 7ff874a00700 1 --2- 172.21.15.90:0/480748038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c91f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.048+0000 7ff874a00700 1 -- 172.21.15.90:0/480748038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 msgr2=0x5608c91feb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.048+0000 7ff874a00700 1 --2- 172.21.15.90:0/480748038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 0x5608c91feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.048+0000 7ff874a00700 1 -- 172.21.15.90:0/480748038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608c90db420 con 0x5608c91f47b0 2024-05-27T07:56:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.049+0000 7ff874a00700 1 --2- 172.21.15.90:0/480748038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c9201060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff85c011520 tx=0x7ff85c011830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d667b62a0c35928e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.049+0000 7ff861600700 1 -- 172.21.15.90:0/480748038 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c004220 con 0x5608c91f47b0 2024-05-27T07:56:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.049+0000 7ff861600700 1 -- 172.21.15.90:0/480748038 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff85c0043c0 con 0x5608c91f47b0 2024-05-27T07:56:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.049+0000 7ff861600700 1 -- 172.21.15.90:0/480748038 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0198a0 con 0x5608c91f47b0 2024-05-27T07:56:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.051+0000 7ff88551cb80 1 -- 172.21.15.90:0/480748038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 msgr2=0x5608c9201060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:35.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.051+0000 7ff88551cb80 1 --2- 172.21.15.90:0/480748038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c9201060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff85c011520 tx=0x7ff85c011830 comp rx=0 tx=0).stop 2024-05-27T07:56:35.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.051+0000 7ff88551cb80 1 -- 172.21.15.90:0/480748038 shutdown_connections 2024-05-27T07:56:35.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.051+0000 7ff88551cb80 1 --2- 172.21.15.90:0/480748038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c9201060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.051+0000 7ff88551cb80 1 --2- 172.21.15.90:0/480748038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c91f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.051+0000 7ff88551cb80 1 --2- 172.21.15.90:0/480748038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 0x5608c91feb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.051+0000 7ff88551cb80 1 -- 172.21.15.90:0/480748038 >> 172.21.15.90:0/480748038 conn(0x5608c90d5fc0 msgr2=0x5608c91efd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.052+0000 7ff88551cb80 1 -- 172.21.15.90:0/480748038 shutdown_connections 2024-05-27T07:56:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.052+0000 7ff86fe00700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff8580b2a50 0x7ff8580b4f60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff870052100 tx=0x7ff870051980 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.052+0000 7ff88551cb80 1 -- 172.21.15.90:0/480748038 wait complete. 2024-05-27T07:56:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.053+0000 7ff88551cb80 1 -- start start 2024-05-27T07:56:35.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.054+0000 7ff88551cb80 1 Processor -- start 2024-05-27T07:56:35.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.055+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c91fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.055+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 0x5608c91feb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.055+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c91ff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.055+0000 7ff88551cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5608c9018fa0 con 0x5608c91f43a0 2024-05-27T07:56:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.055+0000 7ff88551cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5608c90b3760 con 0x5608c91f47b0 2024-05-27T07:56:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.055+0000 7ff88551cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5608c92019d0 con 0x5608c9075e40 2024-05-27T07:56:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.055+0000 7ff88551cb80 10 client.0 _ll_register_callbacks cb 0x5608c90bcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.056+0000 7ff874a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c91ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.056+0000 7ff874a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c91ff060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:36294/0 (socket says 172.21.15.90:36294) 2024-05-27T07:56:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.056+0000 7ff874a00700 1 -- 172.21.15.90:0/1318529513 learned_addr learned my addr 172.21.15.90:0/1318529513 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:35.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.056+0000 7ff86fe00700 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c91fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.057+0000 7ff874a00700 1 -- 172.21.15.90:0/1318529513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 msgr2=0x5608c91fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:35.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.057+0000 7ff874a00700 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c91fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.057+0000 7ff874a00700 1 -- 172.21.15.90:0/1318529513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 msgr2=0x5608c91feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:35.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.057+0000 7ff874a00700 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 0x5608c91feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.057+0000 7ff874a00700 1 -- 172.21.15.90:0/1318529513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608c90628d0 con 0x5608c91f47b0 2024-05-27T07:56:35.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.057+0000 7ff874a00700 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c91ff060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff85c01b040 tx=0x7ff85c003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3685e39a47d374 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:35.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.058+0000 7ff856000700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:56:35.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.058+0000 7ff856000700 1 -- 172.21.15.90:0/1318529513 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c027070 con 0x5608c91f47b0 2024-05-27T07:56:35.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.058+0000 7ff856000700 1 -- 172.21.15.90:0/1318529513 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7ff85c005450 con 0x5608c91f47b0 2024-05-27T07:56:35.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.058+0000 7ff88551cb80 1 -- 172.21.15.90:0/1318529513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85c0043c0 con 0x5608c91f47b0 2024-05-27T07:56:35.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.059+0000 7ff856000700 1 -- 172.21.15.90:0/1318529513 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff85c033050 con 0x5608c91f47b0 2024-05-27T07:56:35.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.059+0000 7ff88551cb80 10 client.36914 fetch_fsmap learned FSMap version 64 2024-05-27T07:56:35.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.059+0000 7ff88551cb80 1 -- 172.21.15.90:0/1318529513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5608c90e3060 con 0x5608c91f47b0 2024-05-27T07:56:35.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.060+0000 7ff856000700 1 -- 172.21.15.90:0/1318529513 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff85c0043c0 con 0x5608c91f47b0 2024-05-27T07:56:35.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.061+0000 7ff856000700 1 -- 172.21.15.90:0/1318529513 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7ff85c0235c0 con 0x5608c91f47b0 2024-05-27T07:56:35.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.061+0000 7ff88551cb80 10 client.36914 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:56:35.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.061+0000 7ff88551cb80 10 client.36914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:35.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.061+0000 7ff88551cb80 20 client.36914 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:35.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.061+0000 7ff88551cb80 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5608c921f9b0 0x5608c9221e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.061+0000 7ff88551cb80 4 client.36914 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:35.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.061+0000 7ff88551cb80 1 -- 172.21.15.90:0/1318529513 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608c9124d90 con 0x5608c921f9b0 2024-05-27T07:56:35.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.062+0000 7ff86fe00700 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5608c921f9b0 0x5608c9221e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.064+0000 7ff856000700 10 client.36914 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:35.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.064+0000 7ff86fe00700 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5608c921f9b0 0x5608c9221e80 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c01977c29390da server_cookie=24633e3ecd8aa230 in_seq=0 out_seq=0 2024-05-27T07:56:35.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.066+0000 7ff856000700 1 -- 172.21.15.90:0/1318529513 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5608c9124d90 con 0x5608c921f9b0 2024-05-27T07:56:35.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.066+0000 7ff856000700 10 client.36914 handle_command_reply: tid=0 2024-05-27T07:56:35.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.066+0000 7ff88551cb80 1 client.36914 shutdown 2024-05-27T07:56:35.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.067+0000 7ff88551cb80 1 -- 172.21.15.90:0/1318529513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 msgr2=0x5608c91ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:35.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.067+0000 7ff88551cb80 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c91ff060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff85c01b040 tx=0x7ff85c003960 comp rx=0 tx=0).stop 2024-05-27T07:56:35.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.067+0000 7ff88551cb80 1 -- 172.21.15.90:0/1318529513 shutdown_connections 2024-05-27T07:56:35.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.067+0000 7ff88551cb80 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5608c921f9b0 0x5608c9221e80 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.067+0000 7ff88551cb80 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f47b0 0x5608c91ff060 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.067+0000 7ff88551cb80 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c91fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.067+0000 7ff88551cb80 1 --2- 172.21.15.90:0/1318529513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 0x5608c91feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.067+0000 7ff856000700 0 client.36914 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.067+0000 7ff88551cb80 1 -- 172.21.15.90:0/1318529513 >> 172.21.15.90:0/1318529513 conn(0x5608c90d5fc0 msgr2=0x5608c91eeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.068+0000 7ff88551cb80 1 -- 172.21.15.90:0/1318529513 shutdown_connections 2024-05-27T07:56:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.068+0000 7ff88551cb80 1 -- 172.21.15.90:0/1318529513 wait complete. 2024-05-27T07:56:35.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.068+0000 7ff88551cb80 20 client.36914 trim_cache size 0 max 16384 2024-05-27T07:56:35.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.105+0000 7ff88551cb80 1 Processor -- start 2024-05-27T07:56:35.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.106+0000 7ff88551cb80 1 -- start start 2024-05-27T07:56:35.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.106+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c9075e40 0x5608c91f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.106+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 0x5608c9200250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.106+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c91f47b0 0x5608c9200790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.106+0000 7ff88551cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff868141e70 con 0x5608c91f43a0 2024-05-27T07:56:35.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.106+0000 7ff88551cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff86813eec0 con 0x5608c9075e40 2024-05-27T07:56:35.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.106+0000 7ff88551cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681410d0 con 0x5608c91f47b0 2024-05-27T07:56:35.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.107+0000 7ff874a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c91f47b0 0x5608c9200790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.107+0000 7ff874a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c91f47b0 0x5608c9200790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57884/0 (socket says 172.21.15.90:57884) 2024-05-27T07:56:35.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.107+0000 7ff874a00700 1 -- 172.21.15.90:0/1966130259 learned_addr learned my addr 172.21.15.90:0/1966130259 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:35.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.107+0000 7ff86f400700 1 --2- 172.21.15.90:0/1966130259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 0x5608c9200250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.108+0000 7ff86fe00700 1 --2- 172.21.15.90:0/1966130259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c9075e40 0x5608c91f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.108+0000 7ff874a00700 1 -- 172.21.15.90:0/1966130259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c9075e40 msgr2=0x5608c91f9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.108+0000 7ff874a00700 1 --2- 172.21.15.90:0/1966130259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c9075e40 0x5608c91f9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.108+0000 7ff874a00700 1 -- 172.21.15.90:0/1966130259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 msgr2=0x5608c9200250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.108+0000 7ff874a00700 1 --2- 172.21.15.90:0/1966130259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 0x5608c9200250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.108+0000 7ff874a00700 1 -- 172.21.15.90:0/1966130259 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608c91f0930 con 0x5608c91f47b0 2024-05-27T07:56:35.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.110+0000 7ff874a00700 1 --2- 172.21.15.90:0/1966130259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c91f47b0 0x5608c9200790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff85c023990 tx=0x7ff85c0094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa0f91459fd94a38 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:35.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.111+0000 7ff860c00700 1 -- 172.21.15.90:0/1966130259 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c027070 con 0x5608c91f47b0 2024-05-27T07:56:35.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.112+0000 7ff860c00700 1 -- 172.21.15.90:0/1966130259 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff85c003c80 con 0x5608c91f47b0 2024-05-27T07:56:35.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.112+0000 7ff860c00700 1 -- 172.21.15.90:0/1966130259 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c02ba50 con 0x5608c91f47b0 2024-05-27T07:56:35.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.113+0000 7ff88551cb80 1 -- 172.21.15.90:0/1966130259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c91f47b0 msgr2=0x5608c9200790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:35.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.113+0000 7ff88551cb80 1 --2- 172.21.15.90:0/1966130259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c91f47b0 0x5608c9200790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff85c023990 tx=0x7ff85c0094b0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.114+0000 7ff88551cb80 1 -- 172.21.15.90:0/1966130259 shutdown_connections 2024-05-27T07:56:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.114+0000 7ff88551cb80 1 --2- 172.21.15.90:0/1966130259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c91f47b0 0x5608c9200790 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.114+0000 7ff88551cb80 1 --2- 172.21.15.90:0/1966130259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c9075e40 0x5608c91f9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.114+0000 7ff88551cb80 1 --2- 172.21.15.90:0/1966130259 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f43a0 0x5608c9200250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.114+0000 7ff88551cb80 1 -- 172.21.15.90:0/1966130259 >> 172.21.15.90:0/1966130259 conn(0x5608c90d5fc0 msgr2=0x5608c91ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.114+0000 7ff88551cb80 1 -- 172.21.15.90:0/1966130259 shutdown_connections 2024-05-27T07:56:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.115+0000 7ff88551cb80 1 -- 172.21.15.90:0/1966130259 wait complete. 2024-05-27T07:56:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.116+0000 7ff88551cb80 1 -- start start 2024-05-27T07:56:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.116+0000 7ff88551cb80 1 Processor -- start 2024-05-27T07:56:35.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.117+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c9207410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.117+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f43a0 0x5608c9202770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.118+0000 7ff88551cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f47b0 0x5608c9202d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.118+0000 7ff88551cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff868140a00 con 0x5608c91f47b0 2024-05-27T07:56:35.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.118+0000 7ff88551cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff86813f590 con 0x5608c91f43a0 2024-05-27T07:56:35.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.118+0000 7ff88551cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff86813fc60 con 0x5608c9075e40 2024-05-27T07:56:35.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.118+0000 7ff88551cb80 10 client.0 _ll_register_callbacks cb 0x5608c90bcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:35.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.118+0000 7ff86fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c9207410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.118+0000 7ff86fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c9207410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57886/0 (socket says 172.21.15.90:57886) 2024-05-27T07:56:35.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.118+0000 7ff86fe00700 1 -- 172.21.15.90:0/2685695917 learned_addr learned my addr 172.21.15.90:0/2685695917 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:35.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.118+0000 7ff86f400700 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f43a0 0x5608c9202770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.119+0000 7ff874a00700 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f47b0 0x5608c9202d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.119+0000 7ff874a00700 1 -- 172.21.15.90:0/2685695917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 msgr2=0x5608c9207410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:35.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.119+0000 7ff874a00700 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c9207410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.119+0000 7ff874a00700 1 -- 172.21.15.90:0/2685695917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f43a0 msgr2=0x5608c9202770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:35.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.119+0000 7ff874a00700 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f43a0 0x5608c9202770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.119+0000 7ff874a00700 1 -- 172.21.15.90:0/2685695917 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608c92034b0 con 0x5608c91f47b0 2024-05-27T07:56:35.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.120+0000 7ff856000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:56:35.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.121+0000 7ff874a00700 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f47b0 0x5608c9202d10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff85c023990 tx=0x7ff85c023de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d79b941af38210 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:35.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.122+0000 7ff856000700 1 -- 172.21.15.90:0/2685695917 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c03b070 con 0x5608c91f47b0 2024-05-27T07:56:35.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.122+0000 7ff88551cb80 1 -- 172.21.15.90:0/2685695917 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85c003c80 con 0x5608c91f47b0 2024-05-27T07:56:35.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.122+0000 7ff856000700 1 -- 172.21.15.90:0/2685695917 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7ff85c023670 con 0x5608c91f47b0 2024-05-27T07:56:35.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.123+0000 7ff856000700 1 -- 172.21.15.90:0/2685695917 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff85c03b070 con 0x5608c91f47b0 2024-05-27T07:56:35.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.123+0000 7ff88551cb80 10 client.17325 fetch_fsmap learned FSMap version 64 2024-05-27T07:56:35.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.123+0000 7ff88551cb80 1 -- 172.21.15.90:0/2685695917 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5608c928a4b0 con 0x5608c91f47b0 2024-05-27T07:56:35.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.124+0000 7ff856000700 1 -- 172.21.15.90:0/2685695917 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff85c003c80 con 0x5608c91f47b0 2024-05-27T07:56:35.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.124+0000 7ff856000700 1 -- 172.21.15.90:0/2685695917 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7ff85c005250 con 0x5608c91f47b0 2024-05-27T07:56:35.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.124+0000 7ff88551cb80 10 client.17325 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:56:35.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.124+0000 7ff88551cb80 10 client.17325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:35.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.124+0000 7ff88551cb80 20 client.17325 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:35.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.124+0000 7ff88551cb80 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5608c928aa40 0x5608c928ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:35.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.124+0000 7ff88551cb80 4 client.17325 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:56:35.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.124+0000 7ff88551cb80 1 -- 172.21.15.90:0/2685695917 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608c911e570 con 0x5608c928aa40 2024-05-27T07:56:35.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.125+0000 7ff86fe00700 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5608c928aa40 0x5608c928ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:35.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.126+0000 7ff86fe00700 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5608c928aa40 0x5608c928ce60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fda464b8711a8b5 server_cookie=30d59387e83bf2b4 in_seq=0 out_seq=0 2024-05-27T07:56:35.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.127+0000 7ff856000700 10 client.17325 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:35.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.127+0000 7ff856000700 1 -- 172.21.15.90:0/2685695917 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5608c911e570 con 0x5608c928aa40 2024-05-27T07:56:35.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.127+0000 7ff856000700 10 client.17325 handle_command_reply: tid=0 2024-05-27T07:56:35.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.127+0000 7ff88551cb80 1 client.17325 shutdown 2024-05-27T07:56:35.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.127+0000 7ff88551cb80 1 -- 172.21.15.90:0/2685695917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f47b0 msgr2=0x5608c9202d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:35.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.127+0000 7ff88551cb80 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f47b0 0x5608c9202d10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff85c023990 tx=0x7ff85c023de0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.128+0000 7ff88551cb80 1 -- 172.21.15.90:0/2685695917 shutdown_connections 2024-05-27T07:56:35.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.128+0000 7ff88551cb80 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5608c928aa40 0x5608c928ce60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.128+0000 7ff88551cb80 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5608c91f47b0 0x5608c9202d10 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.128+0000 7ff88551cb80 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5608c9075e40 0x5608c9207410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.128+0000 7ff88551cb80 1 --2- 172.21.15.90:0/2685695917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5608c91f43a0 0x5608c9202770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.128+0000 7ff88551cb80 1 -- 172.21.15.90:0/2685695917 >> 172.21.15.90:0/2685695917 conn(0x5608c90d5fc0 msgr2=0x5608c920a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:35.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.128+0000 7ff856000700 0 client.17325 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:35.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.128+0000 7ff88551cb80 1 -- 172.21.15.90:0/2685695917 shutdown_connections 2024-05-27T07:56:35.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.129+0000 7ff88551cb80 1 -- 172.21.15.90:0/2685695917 wait complete. 2024-05-27T07:56:35.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.129+0000 7ff88551cb80 20 client.17325 trim_cache size 0 max 16384 2024-05-27T07:56:35.132 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:56:35.132 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:56:35.132 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:56:35.132 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:56:35.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.130+0000 7ff875400700 1 -- 172.21.15.90:0/1601106863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff8580b2a50 msgr2=0x7ff8580b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:35.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.130+0000 7ff875400700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff8580b2a50 0x7ff8580b4f60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff870052100 tx=0x7ff870051980 comp rx=0 tx=0).stop 2024-05-27T07:56:35.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.130+0000 7ff875400700 1 -- 172.21.15.90:0/1601106863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff868096f10 msgr2=0x7ff86812b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:35.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.130+0000 7ff875400700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff868096f10 0x7ff86812b410 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff864009230 tx=0x7ff86400fa00 comp rx=0 tx=0).stop 2024-05-27T07:56:35.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.131+0000 7ff875400700 1 -- 172.21.15.90:0/1601106863 shutdown_connections 2024-05-27T07:56:35.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.131+0000 7ff875400700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff86809c5e0 0x7ff86812f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.131+0000 7ff875400700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff8580b2a50 0x7ff8580b4f60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.131+0000 7ff875400700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8680965c0 0x7ff86812ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.131+0000 7ff875400700 1 --2- 172.21.15.90:0/1601106863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff868096f10 0x7ff86812b410 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:35.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.131+0000 7ff875400700 1 -- 172.21.15.90:0/1601106863 >> 172.21.15.90:0/1601106863 conn(0x7ff86800d2f0 msgr2=0x7ff868094d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:35.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.131+0000 7ff875400700 1 -- 172.21.15.90:0/1601106863 shutdown_connections 2024-05-27T07:56:35.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:35.131+0000 7ff875400700 1 -- 172.21.15.90:0/1601106863 wait complete. 2024-05-27T07:56:36.173 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:56:36.173 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:56:36.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:36 smithi090 ceph-mon[31429]: pgmap v323: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.4 MiB/s wr, 676 op/s 2024-05-27T07:56:36.564 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:36 smithi129 ceph-mon[35254]: pgmap v323: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.4 MiB/s wr, 676 op/s 2024-05-27T07:56:36.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:36 smithi136 ceph-mon[35324]: pgmap v323: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.4 MiB/s wr, 676 op/s 2024-05-27T07:56:38.203 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:38.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:38 smithi129 ceph-mon[35254]: pgmap v324: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.0 MiB/s wr, 531 op/s 2024-05-27T07:56:38.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:38 smithi090 ceph-mon[31429]: pgmap v324: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.0 MiB/s wr, 531 op/s 2024-05-27T07:56:38.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:38 smithi136 ceph-mon[35324]: pgmap v324: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.0 MiB/s wr, 531 op/s 2024-05-27T07:56:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 380.0 s, 633.4 tps, lat 161.012 ms stddev 89.860 2024-05-27T07:56:40.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:40 smithi129 ceph-mon[35254]: pgmap v325: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.0 MiB/s wr, 744 op/s 2024-05-27T07:56:40.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:40 smithi090 ceph-mon[31429]: pgmap v325: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.0 MiB/s wr, 744 op/s 2024-05-27T07:56:40.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:40 smithi136 ceph-mon[35324]: pgmap v325: 129 pgs: 129 active+clean; 2.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.0 MiB/s wr, 744 op/s 2024-05-27T07:56:42.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:42 smithi090 ceph-mon[31429]: pgmap v326: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-05-27T07:56:42.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:42 smithi136 ceph-mon[35324]: pgmap v326: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-05-27T07:56:42.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:42 smithi129 ceph-mon[35254]: pgmap v326: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 500 op/s 2024-05-27T07:56:43.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 385.0 s, 600.6 tps, lat 166.122 ms stddev 66.798 2024-05-27T07:56:44.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:44 smithi090 ceph-mon[31429]: pgmap v327: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.1 MiB/s wr, 541 op/s 2024-05-27T07:56:44.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:44 smithi136 ceph-mon[35324]: pgmap v327: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.1 MiB/s wr, 541 op/s 2024-05-27T07:56:44.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:44 smithi129 ceph-mon[35254]: pgmap v327: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.1 MiB/s wr, 541 op/s 2024-05-27T07:56:46.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:46 smithi090 ceph-mon[31429]: pgmap v328: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.7 MiB/s wr, 642 op/s 2024-05-27T07:56:46.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:46 smithi136 ceph-mon[35324]: pgmap v328: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.7 MiB/s wr, 642 op/s 2024-05-27T07:56:46.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:46 smithi129 ceph-mon[35254]: pgmap v328: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.7 MiB/s wr, 642 op/s 2024-05-27T07:56:47.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.920+0000 7f49f5c00700 1 -- 172.21.15.90:0/1530520211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49e809b540 msgr2=0x7f49e809b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:47.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.920+0000 7f49f5c00700 1 --2- 172.21.15.90:0/1530520211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49e809b540 0x7f49e809b950 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009990 tx=0x7f49e4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.921+0000 7f49f5c00700 1 -- 172.21.15.90:0/1530520211 shutdown_connections 2024-05-27T07:56:47.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.921+0000 7f49f5c00700 1 --2- 172.21.15.90:0/1530520211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49e8095e50 0x7f49e80a22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.921+0000 7f49f5c00700 1 --2- 172.21.15.90:0/1530520211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49e809b540 0x7f49e809b950 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.921+0000 7f49f5c00700 1 --2- 172.21.15.90:0/1530520211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e8095500 0x7f49e8095910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.921+0000 7f49f5c00700 1 -- 172.21.15.90:0/1530520211 >> 172.21.15.90:0/1530520211 conn(0x7f49e800b040 msgr2=0x7f49e800b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:47.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.924+0000 7f49f5c00700 1 -- 172.21.15.90:0/1530520211 shutdown_connections 2024-05-27T07:56:47.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.924+0000 7f49f5c00700 1 -- 172.21.15.90:0/1530520211 wait complete. 2024-05-27T07:56:47.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.926+0000 7f49f5c00700 1 Processor -- start 2024-05-27T07:56:47.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.927+0000 7f49f5c00700 1 -- start start 2024-05-27T07:56:47.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.927+0000 7f49f5c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49e8095500 0x7f49e80a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:47.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.927+0000 7f49f5c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e8095e50 0x7f49e80a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:47.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.927+0000 7f49f5c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49e809b540 0x7f49e8139000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:47.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.927+0000 7f49f5c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f49e80a3d40 con 0x7f49e809b540 2024-05-27T07:56:47.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.927+0000 7f49f5c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49e80a3bc0 con 0x7f49e8095500 2024-05-27T07:56:47.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.927+0000 7f49f5c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49e80a3ec0 con 0x7f49e8095e50 2024-05-27T07:56:47.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.928+0000 7f49f5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49e809b540 0x7f49e8139000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:47.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.928+0000 7f49efe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49e8095500 0x7f49e80a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:47.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.928+0000 7f49efe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49e8095500 0x7f49e80a0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:42346/0 (socket says 172.21.15.90:42346) 2024-05-27T07:56:47.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.928+0000 7f49ef400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e8095e50 0x7f49e80a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:47.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.928+0000 7f49efe00700 1 -- 172.21.15.90:0/4196449648 learned_addr learned my addr 172.21.15.90:0/4196449648 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:47.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.929+0000 7f49efe00700 1 -- 172.21.15.90:0/4196449648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e8095e50 msgr2=0x7f49e80a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:47.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.929+0000 7f49efe00700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e8095e50 0x7f49e80a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.929+0000 7f49efe00700 1 -- 172.21.15.90:0/4196449648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49e809b540 msgr2=0x7f49e8139000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.929+0000 7f49efe00700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49e809b540 0x7f49e8139000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.929+0000 7f49efe00700 1 -- 172.21.15.90:0/4196449648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e81396a0 con 0x7f49e8095500 2024-05-27T07:56:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.929+0000 7f49f5200700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49e809b540 0x7f49e8139000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:56:47.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.930+0000 7f49efe00700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49e8095500 0x7f49e80a0ec0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49e40098e0 tx=0x7f49e40057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:47.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.930+0000 7f49ecc00700 1 -- 172.21.15.90:0/4196449648 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4027070 con 0x7f49e8095500 2024-05-27T07:56:47.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.930+0000 7f49f5c00700 1 -- 172.21.15.90:0/4196449648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e8139930 con 0x7f49e8095500 2024-05-27T07:56:47.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.931+0000 7f49f5c00700 1 -- 172.21.15.90:0/4196449648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e81400d0 con 0x7f49e8095500 2024-05-27T07:56:47.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.931+0000 7f49ecc00700 1 -- 172.21.15.90:0/4196449648 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49e4009510 con 0x7f49e8095500 2024-05-27T07:56:47.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.931+0000 7f49ecc00700 1 -- 172.21.15.90:0/4196449648 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4022e30 con 0x7f49e8095500 2024-05-27T07:56:47.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.932+0000 7f49f5c00700 1 -- 172.21.15.90:0/4196449648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d0005330 con 0x7f49e8095500 2024-05-27T07:56:47.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.941+0000 7f49ecc00700 1 -- 172.21.15.90:0/4196449648 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f49e4009d10 con 0x7f49e8095500 2024-05-27T07:56:47.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.944+0000 7f49ecc00700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f49d80b2ae0 0x7f49d80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:47.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.947+0000 7f49ef400700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f49d80b2ae0 0x7f49d80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:47.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.949+0000 7f49ecc00700 1 -- 172.21.15.90:0/4196449648 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f49e40e3bd0 con 0x7f49e8095500 2024-05-27T07:56:47.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.951+0000 7f49ef400700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f49d80b2ae0 0x7f49d80b4ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f49e80a5590 tx=0x7f49dc009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:47.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:47.958+0000 7f49ecc00700 1 -- 172.21.15.90:0/4196449648 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49e4013070 con 0x7f49e8095500 2024-05-27T07:56:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.451+0000 7f70e3600640 1 -- 172.21.15.90:0/3729363966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70dc1043b0 msgr2=0x7f70dc110780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.451+0000 7f70e3600640 1 --2- 172.21.15.90:0/3729363966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70dc1043b0 0x7f70dc110780 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009ca0 tx=0x7f70d402a8c0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.452+0000 7f70e3600640 1 -- 172.21.15.90:0/3729363966 shutdown_connections 2024-05-27T07:56:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.452+0000 7f70e3600640 1 --2- 172.21.15.90:0/3729363966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70dc10a100 0x7f70dc103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.453+0000 7f70e3600640 1 --2- 172.21.15.90:0/3729363966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70dc1043b0 0x7f70dc110780 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.453+0000 7f70e3600640 1 --2- 172.21.15.90:0/3729363966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70dc109c60 0x7f70dc103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.453+0000 7f70e3600640 1 -- 172.21.15.90:0/3729363966 >> 172.21.15.90:0/3729363966 conn(0x7f70dc0ff320 msgr2=0x7f70dc101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.454+0000 7f70e3600640 1 -- 172.21.15.90:0/3729363966 shutdown_connections 2024-05-27T07:56:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.455+0000 7f70e3600640 1 -- 172.21.15.90:0/3729363966 wait complete. 2024-05-27T07:56:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.456+0000 7f70e3600640 1 Processor -- start 2024-05-27T07:56:48.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.458+0000 7f70e3600640 1 -- start start 2024-05-27T07:56:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.460+0000 7f70e3600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70dc1043b0 0x7f70dc10da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.461+0000 7f70e3600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70dc109c60 0x7f70dc10dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.461+0000 7f70e0c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70dc1043b0 0x7f70dc10da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.461+0000 7f70e3600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70dc10a100 0x7f70dc10e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.461+0000 7f70e3600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc0778e0 con 0x7f70dc1043b0 2024-05-27T07:56:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.461+0000 7f70e3600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc077760 con 0x7f70dc10a100 2024-05-27T07:56:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.461+0000 7f70e3600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc077a60 con 0x7f70dc109c60 2024-05-27T07:56:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.462+0000 7f70dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70dc109c60 0x7f70dc10dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.462+0000 7f70e1600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70dc10a100 0x7f70dc10e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.462+0000 7f70e1600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70dc10a100 0x7f70dc10e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:42362/0 (socket says 172.21.15.90:42362) 2024-05-27T07:56:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.462+0000 7f70e1600640 1 -- 172.21.15.90:0/1718670685 learned_addr learned my addr 172.21.15.90:0/1718670685 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.463+0000 7f70dbe00640 1 -- 172.21.15.90:0/1718670685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70dc10a100 msgr2=0x7f70dc10e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.463+0000 7f70dbe00640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70dc10a100 0x7f70dc10e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.463+0000 7f70dbe00640 1 -- 172.21.15.90:0/1718670685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70dc1043b0 msgr2=0x7f70dc10da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.463+0000 7f70dbe00640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70dc1043b0 0x7f70dc10da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.463+0000 7f70dbe00640 1 -- 172.21.15.90:0/1718670685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc0737b0 con 0x7f70dc109c60 2024-05-27T07:56:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.465+0000 7f70dbe00640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70dc109c60 0x7f70dc10dfd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f70dc10a530 tx=0x7f70c800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.466+0000 7f70d9600640 1 -- 172.21.15.90:0/1718670685 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8017070 con 0x7f70dc109c60 2024-05-27T07:56:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.466+0000 7f70d9600640 1 -- 172.21.15.90:0/1718670685 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70c8009c00 con 0x7f70dc109c60 2024-05-27T07:56:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.466+0000 7f70d9600640 1 -- 172.21.15.90:0/1718670685 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c800f3c0 con 0x7f70dc109c60 2024-05-27T07:56:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.467+0000 7f70e3600640 1 -- 172.21.15.90:0/1718670685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc073aa0 con 0x7f70dc109c60 2024-05-27T07:56:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.467+0000 7f70e3600640 1 -- 172.21.15.90:0/1718670685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc073fa0 con 0x7f70dc109c60 2024-05-27T07:56:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.479+0000 7f70d9600640 1 -- 172.21.15.90:0/1718670685 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f70c801a070 con 0x7f70dc109c60 2024-05-27T07:56:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.479+0000 7f70ea5c5740 1 Processor -- start 2024-05-27T07:56:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.480+0000 7f70ea5c5740 1 -- start start 2024-05-27T07:56:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.480+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 0x55f96f114860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.480+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 0x55f96f114da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.481+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.481+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f96f085b90 con 0x55f96ed721b0 2024-05-27T07:56:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.481+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f96f075730 con 0x55f96ed492f0 2024-05-27T07:56:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.481+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f96f079770 con 0x55f96ed514d0 2024-05-27T07:56:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.481+0000 7f70e1600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 0x55f96f114860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.481+0000 7f70dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.482+0000 7f70dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1152e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:41666/0 (socket says 172.21.15.90:41666) 2024-05-27T07:56:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.482+0000 7f70dbe00640 1 -- 172.21.15.90:0/1468694757 learned_addr learned my addr 172.21.15.90:0/1468694757 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.482+0000 7f70e0c00640 1 --2- 172.21.15.90:0/1468694757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 0x55f96f114da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.482+0000 7f70dbe00640 1 -- 172.21.15.90:0/1468694757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 msgr2=0x55f96f114da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.482+0000 7f70dbe00640 1 --2- 172.21.15.90:0/1468694757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 0x55f96f114da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.483+0000 7f70dbe00640 1 -- 172.21.15.90:0/1468694757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 msgr2=0x55f96f114860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.483+0000 7f70dbe00640 1 --2- 172.21.15.90:0/1468694757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 0x55f96f114860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.483+0000 7f70dbe00640 1 -- 172.21.15.90:0/1468694757 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96f00e8b0 con 0x55f96ed514d0 2024-05-27T07:56:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.483+0000 7f70e1600640 1 --2- 172.21.15.90:0/1468694757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 0x55f96f114860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:56:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.484+0000 7f70dbe00640 1 --2- 172.21.15.90:0/1468694757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1152e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70c80df2d0 tx=0x7f70c80ea6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f2d6a704098b7a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.485+0000 7f70d9600640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f70c40b29d0 0x7f70c40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.485+0000 7f70cdc00640 1 -- 172.21.15.90:0/1468694757 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c80f5070 con 0x55f96ed514d0 2024-05-27T07:56:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.486+0000 7f70cdc00640 1 -- 172.21.15.90:0/1468694757 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70c80e42e0 con 0x55f96ed514d0 2024-05-27T07:56:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.486+0000 7f70cdc00640 1 -- 172.21.15.90:0/1468694757 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c80f07f0 con 0x55f96ed514d0 2024-05-27T07:56:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.486+0000 7f70e1600640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f70c40b29d0 0x7f70c40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.486+0000 7f70d9600640 1 -- 172.21.15.90:0/1718670685 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f70c80e1fe0 con 0x7f70dc109c60 2024-05-27T07:56:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.490+0000 7f70ea5c5740 1 -- 172.21.15.90:0/1468694757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 msgr2=0x55f96f1152e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.490+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/1468694757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1152e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f70c80df2d0 tx=0x7f70c80ea6f0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.490+0000 7f70ea5c5740 1 -- 172.21.15.90:0/1468694757 shutdown_connections 2024-05-27T07:56:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.490+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/1468694757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 0x55f96f114da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.491+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/1468694757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1152e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.491+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/1468694757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 0x55f96f114860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.491+0000 7f70ea5c5740 1 -- 172.21.15.90:0/1468694757 >> 172.21.15.90:0/1468694757 conn(0x55f96ed66300 msgr2=0x55f96f111010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.492+0000 7f70e1600640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f70c40b29d0 0x7f70c40b4ee0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70d40094e0 tx=0x7f70d4004a80 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:48.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.492+0000 7f70ea5c5740 1 -- 172.21.15.90:0/1468694757 shutdown_connections 2024-05-27T07:56:48.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.492+0000 7f70ea5c5740 1 -- 172.21.15.90:0/1468694757 wait complete. 2024-05-27T07:56:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.493+0000 7f70ea5c5740 1 -- start start 2024-05-27T07:56:48.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.493+0000 7f70ea5c5740 1 Processor -- start 2024-05-27T07:56:48.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.494+0000 7f49f5c00700 1 -- 172.21.15.90:0/4196449648 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49d0005620 con 0x7f49e8095500 2024-05-27T07:56:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.496+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 0x55f96f11ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.497+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f11ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.497+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 0x55f96f12ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.498+0000 7f49ecc00700 1 -- 172.21.15.90:0/4196449648 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+9456 (secure 0 0 0) 0x7f49e40b3070 con 0x7f49e8095500 2024-05-27T07:56:48.501 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:56:48.501 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":64,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:56:32.161929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T07:56:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.499+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f96ed5b790 con 0x55f96ed721b0 2024-05-27T07:56:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.499+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f96f01dbc0 con 0x55f96ed492f0 2024-05-27T07:56:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.499+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f96f121990 con 0x55f96ed514d0 2024-05-27T07:56:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.499+0000 7f70ea5c5740 10 client.0 _ll_register_callbacks cb 0x55f96effb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.500+0000 7f70dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f11ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.500+0000 7f70e0c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 0x55f96f11ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.500+0000 7f70e1600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 0x55f96f12ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.500+0000 7f70e0c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 0x55f96f11ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:42384/0 (socket says 172.21.15.90:42384) 2024-05-27T07:56:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.500+0000 7f70e0c00640 1 -- 172.21.15.90:0/919525807 learned_addr learned my addr 172.21.15.90:0/919525807 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.501+0000 7f70dbe00640 1 -- 172.21.15.90:0/919525807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 msgr2=0x55f96f11ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.502+0000 7f70dbe00640 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 0x55f96f11ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.502+0000 7f70dbe00640 1 -- 172.21.15.90:0/919525807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 msgr2=0x55f96f12ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.502+0000 7f70dbe00640 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 0x55f96f12ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.502+0000 7f70dbe00640 1 -- 172.21.15.90:0/919525807 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96ed8a210 con 0x55f96ed514d0 2024-05-27T07:56:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.503+0000 7f70dbe00640 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f11ef60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f70c80a4f70 tx=0x7f70c800bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2998fcfacfadfde0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.503+0000 7f70c2000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:56:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.502+0000 7f49f5c00700 1 -- 172.21.15.90:0/4196449648 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f49d80b2ae0 msgr2=0x7f49d80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.502+0000 7f49f5c00700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f49d80b2ae0 0x7f49d80b4ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f49e80a5590 tx=0x7f49dc009260 comp rx=0 tx=0).stop 2024-05-27T07:56:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.502+0000 7f49f5c00700 1 -- 172.21.15.90:0/4196449648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49e8095500 msgr2=0x7f49e80a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.502+0000 7f49f5c00700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49e8095500 0x7f49e80a0ec0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49e40098e0 tx=0x7f49e40057c0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.503+0000 7f70c2000640 1 -- 172.21.15.90:0/919525807 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8021070 con 0x55f96ed514d0 2024-05-27T07:56:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.503+0000 7f70c2000640 1 -- 172.21.15.90:0/919525807 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f70c80a5180 con 0x55f96ed514d0 2024-05-27T07:56:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.504+0000 7f70ea5c5740 1 -- 172.21.15.90:0/919525807 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70c80e42e0 con 0x55f96ed514d0 2024-05-27T07:56:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.504+0000 7f70c2000640 1 -- 172.21.15.90:0/919525807 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f70c80a4db0 con 0x55f96ed514d0 2024-05-27T07:56:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.505+0000 7f70ea5c5740 10 client.26983 fetch_fsmap learned FSMap version 64 2024-05-27T07:56:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.505+0000 7f70ea5c5740 1 -- 172.21.15.90:0/919525807 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f96ee56620 con 0x55f96ed514d0 2024-05-27T07:56:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.503+0000 7f49f5c00700 1 -- 172.21.15.90:0/4196449648 shutdown_connections 2024-05-27T07:56:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.503+0000 7f49f5c00700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49e809b540 0x7f49e8139000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.503+0000 7f49f5c00700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f49d80b2ae0 0x7f49d80b4ff0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.503+0000 7f49f5c00700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49e8095500 0x7f49e80a0ec0 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f49e40098e0 tx=0x7f49e40057c0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.503+0000 7f49f5c00700 1 --2- 172.21.15.90:0/4196449648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e8095e50 0x7f49e80a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.503+0000 7f49f5c00700 1 -- 172.21.15.90:0/4196449648 >> 172.21.15.90:0/4196449648 conn(0x7f49e800b040 msgr2=0x7f49e80a6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.504+0000 7f49f5c00700 1 -- 172.21.15.90:0/4196449648 shutdown_connections 2024-05-27T07:56:48.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:48.504+0000 7f49f5c00700 1 -- 172.21.15.90:0/4196449648 wait complete. 2024-05-27T07:56:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.505+0000 7f70c2000640 1 -- 172.21.15.90:0/919525807 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70c80e42e0 con 0x55f96ed514d0 2024-05-27T07:56:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.505+0000 7f70c2000640 1 -- 172.21.15.90:0/919525807 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f70c8024060 con 0x55f96ed514d0 2024-05-27T07:56:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.506+0000 7f70ea5c5740 10 client.26983 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:56:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.506+0000 7f70ea5c5740 10 client.26983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.506+0000 7f70ea5c5740 20 client.26983 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.506+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f96ed30d80 0x55f96f130b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.506+0000 7f70ea5c5740 4 client.26983 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.506+0000 7f70ea5c5740 1 -- 172.21.15.90:0/919525807 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f96f087ae0 con 0x55f96ed30d80 2024-05-27T07:56:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.507+0000 7f70e0c00640 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f96ed30d80 0x55f96f130b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.510+0000 7f70c2000640 10 client.26983 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.510+0000 7f70e0c00640 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f96ed30d80 0x55f96f130b60 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a245059139cf828d server_cookie=1fff5d11ec717fad in_seq=0 out_seq=0 2024-05-27T07:56:48.512 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 64 2024-05-27T07:56:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.513+0000 7f70c2000640 1 -- 172.21.15.90:0/919525807 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f96f087ae0 con 0x55f96ed30d80 2024-05-27T07:56:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.513+0000 7f70c2000640 10 client.26983 handle_command_reply: tid=0 2024-05-27T07:56:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.514+0000 7f70ea5c5740 1 client.26983 shutdown 2024-05-27T07:56:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.516+0000 7f70ea5c5740 1 -- 172.21.15.90:0/919525807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 msgr2=0x55f96f11ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.516+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f11ef60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f70c80a4f70 tx=0x7f70c800bed0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.516+0000 7f70ea5c5740 1 -- 172.21.15.90:0/919525807 shutdown_connections 2024-05-27T07:56:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.516+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f96ed30d80 0x55f96f130b60 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.516+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed721b0 0x55f96f12ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.516+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f11ef60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.516+0000 7f70c2000640 0 client.26983 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.516+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/919525807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed492f0 0x55f96f11ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.517+0000 7f70ea5c5740 1 -- 172.21.15.90:0/919525807 >> 172.21.15.90:0/919525807 conn(0x55f96ed66300 msgr2=0x55f96f116580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.517+0000 7f70ea5c5740 1 -- 172.21.15.90:0/919525807 shutdown_connections 2024-05-27T07:56:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.517+0000 7f70ea5c5740 1 -- 172.21.15.90:0/919525807 wait complete. 2024-05-27T07:56:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.518+0000 7f70ea5c5740 20 client.26983 trim_cache size 0 max 16384 2024-05-27T07:56:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.560+0000 7f70ea5c5740 1 Processor -- start 2024-05-27T07:56:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.560+0000 7f70ea5c5740 1 -- start start 2024-05-27T07:56:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.561+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed492f0 0x55f96f110330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.562+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed514d0 0x55f96f120b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.562+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed721b0 0x55f96f114ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.562+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1b9320 con 0x55f96ed492f0 2024-05-27T07:56:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.562+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc11e640 con 0x55f96ed514d0 2024-05-27T07:56:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.562+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1b8580 con 0x55f96ed721b0 2024-05-27T07:56:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.563+0000 7f70dbe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed514d0 0x55f96f120b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.563+0000 7f70dbe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed514d0 0x55f96f120b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:42388/0 (socket says 172.21.15.90:42388) 2024-05-27T07:56:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.563+0000 7f70dbe00640 1 -- 172.21.15.90:0/841423301 learned_addr learned my addr 172.21.15.90:0/841423301 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.563+0000 7f70e0c00640 1 --2- 172.21.15.90:0/841423301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed492f0 0x55f96f110330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.564+0000 7f70e1600640 1 --2- 172.21.15.90:0/841423301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed721b0 0x55f96f114ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.565+0000 7f70e0c00640 1 -- 172.21.15.90:0/841423301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed721b0 msgr2=0x55f96f114ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.565+0000 7f70e0c00640 1 --2- 172.21.15.90:0/841423301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed721b0 0x55f96f114ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.565+0000 7f70e0c00640 1 -- 172.21.15.90:0/841423301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed514d0 msgr2=0x55f96f120b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.565+0000 7f70e0c00640 1 --2- 172.21.15.90:0/841423301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed514d0 0x55f96f120b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.565+0000 7f70e0c00640 1 -- 172.21.15.90:0/841423301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96f111ef0 con 0x55f96ed492f0 2024-05-27T07:56:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.566+0000 7f70e0c00640 1 --2- 172.21.15.90:0/841423301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed492f0 0x55f96f110330 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x55f96f11f730 tx=0x7f70d001c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681f82bab902ec75 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.566+0000 7f70cd200640 1 -- 172.21.15.90:0/841423301 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d001f040 con 0x55f96ed492f0 2024-05-27T07:56:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.567+0000 7f70cd200640 1 -- 172.21.15.90:0/841423301 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70d0031030 con 0x55f96ed492f0 2024-05-27T07:56:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.567+0000 7f70cd200640 1 -- 172.21.15.90:0/841423301 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d002db90 con 0x55f96ed492f0 2024-05-27T07:56:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.569+0000 7f70ea5c5740 1 -- 172.21.15.90:0/841423301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed492f0 msgr2=0x55f96f110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.569+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/841423301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed492f0 0x55f96f110330 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x55f96f11f730 tx=0x7f70d001c270 comp rx=0 tx=0).stop 2024-05-27T07:56:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.571+0000 7f70ea5c5740 1 -- 172.21.15.90:0/841423301 shutdown_connections 2024-05-27T07:56:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.571+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/841423301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed514d0 0x55f96f120b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.571+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/841423301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed721b0 0x55f96f114ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.571+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/841423301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed492f0 0x55f96f110330 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.571+0000 7f70ea5c5740 1 -- 172.21.15.90:0/841423301 >> 172.21.15.90:0/841423301 conn(0x55f96ed66300 msgr2=0x55f96f1147e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.573+0000 7f70ea5c5740 1 -- 172.21.15.90:0/841423301 shutdown_connections 2024-05-27T07:56:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.573+0000 7f70ea5c5740 1 -- 172.21.15.90:0/841423301 wait complete. 2024-05-27T07:56:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.574+0000 7f70ea5c5740 1 -- start start 2024-05-27T07:56:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.577+0000 7f70ea5c5740 1 Processor -- start 2024-05-27T07:56:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.578+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed492f0 0x55f96f127350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.578+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.578+0000 7f70ea5c5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed721b0 0x55f96f119630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.578+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1b7eb0 con 0x55f96ed492f0 2024-05-27T07:56:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.578+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc11ed10 con 0x55f96ed721b0 2024-05-27T07:56:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.578+0000 7f70ea5c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc1b7110 con 0x55f96ed514d0 2024-05-27T07:56:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.578+0000 7f70ea5c5740 10 client.0 _ll_register_callbacks cb 0x7f70c80f3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.578+0000 7f70dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.579+0000 7f70dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1190f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:41702/0 (socket says 172.21.15.90:41702) 2024-05-27T07:56:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.579+0000 7f70dbe00640 1 -- 172.21.15.90:0/4173210944 learned_addr learned my addr 172.21.15.90:0/4173210944 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.579+0000 7f70dbe00640 1 -- 172.21.15.90:0/4173210944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed721b0 msgr2=0x55f96f119630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:56:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.579+0000 7f70dbe00640 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed721b0 0x55f96f119630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.579+0000 7f70dbe00640 1 -- 172.21.15.90:0/4173210944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed492f0 msgr2=0x55f96f127350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:56:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.579+0000 7f70dbe00640 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed492f0 0x55f96f127350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.580+0000 7f70dbe00640 1 -- 172.21.15.90:0/4173210944 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96f119ca0 con 0x55f96ed514d0 2024-05-27T07:56:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.580+0000 7f70dbe00640 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1190f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f70c80a8bf0 tx=0x7f70c80e5b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7578b414390c4c2b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.580+0000 7f70c2000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:56:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.581+0000 7f70c2000640 1 -- 172.21.15.90:0/4173210944 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8021070 con 0x55f96ed514d0 2024-05-27T07:56:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.581+0000 7f70c2000640 1 -- 172.21.15.90:0/4173210944 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f70c80e5c60 con 0x55f96ed514d0 2024-05-27T07:56:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.582+0000 7f70ea5c5740 1 -- 172.21.15.90:0/4173210944 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70d0031030 con 0x55f96ed514d0 2024-05-27T07:56:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.582+0000 7f70c2000640 1 -- 172.21.15.90:0/4173210944 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f70c80a8660 con 0x55f96ed514d0 2024-05-27T07:56:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.584+0000 7f70ea5c5740 10 client.26989 fetch_fsmap learned FSMap version 64 2024-05-27T07:56:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.584+0000 7f70ea5c5740 1 -- 172.21.15.90:0/4173210944 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f96f119ca0 con 0x55f96ed514d0 2024-05-27T07:56:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.585+0000 7f70c2000640 1 -- 172.21.15.90:0/4173210944 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f70d0031030 con 0x55f96ed514d0 2024-05-27T07:56:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.587+0000 7f70c2000640 1 -- 172.21.15.90:0/4173210944 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f70c80b18c0 con 0x55f96ed514d0 2024-05-27T07:56:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.587+0000 7f70ea5c5740 10 client.26989 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:56:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.587+0000 7f70ea5c5740 10 client.26989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:56:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.587+0000 7f70ea5c5740 20 client.26989 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.587+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f96ed061c0 0x55f96f19b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.587+0000 7f70ea5c5740 4 client.26989 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:56:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.587+0000 7f70ea5c5740 1 -- 172.21.15.90:0/4173210944 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f96f0812c0 con 0x55f96ed061c0 2024-05-27T07:56:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.589+0000 7f70e0c00640 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f96ed061c0 0x55f96f19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.591+0000 7f70e0c00640 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f96ed061c0 0x55f96f19b300 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5254d9e8380a51c2 server_cookie=16cc13f78535c514 in_seq=0 out_seq=0 2024-05-27T07:56:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.591+0000 7f70c2000640 10 client.26989 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:48.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:48 smithi090 ceph-mon[31429]: pgmap v329: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.6 MiB/s wr, 507 op/s 2024-05-27T07:56:48.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:48 smithi136 ceph-mon[35324]: pgmap v329: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.6 MiB/s wr, 507 op/s 2024-05-27T07:56:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.755+0000 7f70c2000640 1 -- 172.21.15.90:0/4173210944 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f96f0812c0 con 0x55f96ed061c0 2024-05-27T07:56:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.756+0000 7f70c2000640 10 client.26989 handle_command_reply: tid=0 2024-05-27T07:56:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.756+0000 7f70ea5c5740 1 client.26989 shutdown 2024-05-27T07:56:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.756+0000 7f70ea5c5740 1 -- 172.21.15.90:0/4173210944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 msgr2=0x55f96f1190f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.757+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1190f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f70c80a8bf0 tx=0x7f70c80e5b30 comp rx=0 tx=0).stop 2024-05-27T07:56:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.758+0000 7f70ea5c5740 1 -- 172.21.15.90:0/4173210944 shutdown_connections 2024-05-27T07:56:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.758+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f96ed061c0 0x55f96f19b300 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.758+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f96ed721b0 0x55f96f119630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.758+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f96ed514d0 0x55f96f1190f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.758+0000 7f70c2000640 0 client.26989 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:56:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.758+0000 7f70ea5c5740 1 --2- 172.21.15.90:0/4173210944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f96ed492f0 0x55f96f127350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.759+0000 7f70ea5c5740 1 -- 172.21.15.90:0/4173210944 >> 172.21.15.90:0/4173210944 conn(0x55f96ed66300 msgr2=0x55f96f120860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.759+0000 7f70ea5c5740 1 -- 172.21.15.90:0/4173210944 shutdown_connections 2024-05-27T07:56:48.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.760+0000 7f70ea5c5740 1 -- 172.21.15.90:0/4173210944 wait complete. 2024-05-27T07:56:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.760+0000 7f70ea5c5740 20 client.26989 trim_cache size 0 max 16384 2024-05-27T07:56:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:56:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:56:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:56:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.763+0000 7f70e3600640 1 -- 172.21.15.90:0/1718670685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f70c40b29d0 msgr2=0x7f70c40b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.763+0000 7f70e3600640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f70c40b29d0 0x7f70c40b4ee0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f70d40094e0 tx=0x7f70d4004a80 comp rx=0 tx=0).stop 2024-05-27T07:56:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.763+0000 7f70e3600640 1 -- 172.21.15.90:0/1718670685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70dc109c60 msgr2=0x7f70dc10dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.763+0000 7f70e3600640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70dc109c60 0x7f70dc10dfd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f70dc10a530 tx=0x7f70c800dde0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.764+0000 7f70e3600640 1 -- 172.21.15.90:0/1718670685 shutdown_connections 2024-05-27T07:56:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.764+0000 7f70e3600640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f70c40b29d0 0x7f70c40b4ee0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.764+0000 7f70e3600640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70dc10a100 0x7f70dc10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.764+0000 7f70e3600640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70dc109c60 0x7f70dc10dfd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.764+0000 7f70e3600640 1 --2- 172.21.15.90:0/1718670685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70dc1043b0 0x7f70dc10da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.764+0000 7f70e3600640 1 -- 172.21.15.90:0/1718670685 >> 172.21.15.90:0/1718670685 conn(0x7f70dc0ff320 msgr2=0x7f70dc106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.765+0000 7f70e3600640 1 -- 172.21.15.90:0/1718670685 shutdown_connections 2024-05-27T07:56:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:56:48.765+0000 7f70e3600640 1 -- 172.21.15.90:0/1718670685 wait complete. 2024-05-27T07:56:48.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:48 smithi129 ceph-mon[35254]: pgmap v329: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.6 MiB/s wr, 507 op/s 2024-05-27T07:56:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 390.0 s, 523.8 tps, lat 186.859 ms stddev 104.791 2024-05-27T07:56:49.502 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:56:49.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:49 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/4196449648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:56:49.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:49 smithi090 ceph-mon[31429]: pgmap v330: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.7 MiB/s wr, 735 op/s 2024-05-27T07:56:49.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:49 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/4196449648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:56:49.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:49 smithi136 ceph-mon[35324]: pgmap v330: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.7 MiB/s wr, 735 op/s 2024-05-27T07:56:49.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:49 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/4196449648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:56:49.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:49 smithi129 ceph-mon[35254]: pgmap v330: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.7 MiB/s wr, 735 op/s 2024-05-27T07:56:50.310 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:52.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:52 smithi090 ceph-mon[31429]: pgmap v331: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.1 MiB/s wr, 565 op/s 2024-05-27T07:56:52.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:52 smithi129 ceph-mon[35254]: pgmap v331: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.1 MiB/s wr, 565 op/s 2024-05-27T07:56:52.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:52 smithi136 ceph-mon[35324]: pgmap v331: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.1 MiB/s wr, 565 op/s 2024-05-27T07:56:53.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 395.0 s, 558.8 tps, lat 180.370 ms stddev 125.853 2024-05-27T07:56:54.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:54 smithi129 ceph-mon[35254]: pgmap v332: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.2 MiB/s wr, 584 op/s 2024-05-27T07:56:54.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:54 smithi090 ceph-mon[31429]: pgmap v332: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.2 MiB/s wr, 584 op/s 2024-05-27T07:56:54.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:54 smithi136 ceph-mon[35324]: pgmap v332: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.2 MiB/s wr, 584 op/s 2024-05-27T07:56:55.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.907+0000 7efc6be00700 1 -- 172.21.15.90:0/1687033845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc6c105cb0 msgr2=0x7efc6c10e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:55.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.907+0000 7efc6be00700 1 --2- 172.21.15.90:0/1687033845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc6c105cb0 0x7efc6c10e200 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7efc58009990 tx=0x7efc58009ca0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.907+0000 7efc6be00700 1 -- 172.21.15.90:0/1687033845 shutdown_connections 2024-05-27T07:56:55.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.907+0000 7efc6be00700 1 --2- 172.21.15.90:0/1687033845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc6c10e740 0x7efc6c110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.907+0000 7efc6be00700 1 --2- 172.21.15.90:0/1687033845 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc6c105360 0x7efc6c105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.907+0000 7efc6be00700 1 --2- 172.21.15.90:0/1687033845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc6c105cb0 0x7efc6c10e200 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.907+0000 7efc6be00700 1 -- 172.21.15.90:0/1687033845 >> 172.21.15.90:0/1687033845 conn(0x7efc6c078ab0 msgr2=0x7efc6c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:55.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.907+0000 7efc6be00700 1 -- 172.21.15.90:0/1687033845 shutdown_connections 2024-05-27T07:56:55.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.907+0000 7efc6be00700 1 -- 172.21.15.90:0/1687033845 wait complete. 2024-05-27T07:56:55.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.908+0000 7efc6be00700 1 Processor -- start 2024-05-27T07:56:55.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.908+0000 7efc6be00700 1 -- start start 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc6c105360 0x7efc6c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc6c105cb0 0x7efc6c19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc6c10e740 0x7efc6c1a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c114130 con 0x7efc6c105360 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6be00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c113fb0 con 0x7efc6c105cb0 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6be00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c1142b0 con 0x7efc6c10e740 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc6c105360 0x7efc6c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc6c105360 0x7efc6c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57294/0 (socket says 172.21.15.90:57294) 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6aa00700 1 -- 172.21.15.90:0/1167481188 learned_addr learned my addr 172.21.15.90:0/1167481188 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6b400700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc6c10e740 0x7efc6c1a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6a000700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc6c105cb0 0x7efc6c19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6aa00700 1 -- 172.21.15.90:0/1167481188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc6c10e740 msgr2=0x7efc6c1a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.909+0000 7efc6aa00700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc6c10e740 0x7efc6c1a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.910+0000 7efc6aa00700 1 -- 172.21.15.90:0/1167481188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc6c105cb0 msgr2=0x7efc6c19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.910+0000 7efc6aa00700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc6c105cb0 0x7efc6c19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.910+0000 7efc6aa00700 1 -- 172.21.15.90:0/1167481188 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c1a1d50 con 0x7efc6c105360 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.910+0000 7efc6a000700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc6c105cb0 0x7efc6c19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.910+0000 7efc6aa00700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc6c105360 0x7efc6c19ce80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00c970 tx=0x7efc5c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.910+0000 7efc63400700 1 -- 172.21.15.90:0/1167481188 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c007aa0 con 0x7efc6c105360 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.910+0000 7efc63400700 1 -- 172.21.15.90:0/1167481188 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc5c004cc0 con 0x7efc6c105360 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.910+0000 7efc6be00700 1 -- 172.21.15.90:0/1167481188 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6c1a2040 con 0x7efc6c105360 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.910+0000 7efc6be00700 1 -- 172.21.15.90:0/1167481188 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6c1a9960 con 0x7efc6c105360 2024-05-27T07:56:55.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.910+0000 7efc63400700 1 -- 172.21.15.90:0/1167481188 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c005500 con 0x7efc6c105360 2024-05-27T07:56:55.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.914+0000 7efc63400700 1 -- 172.21.15.90:0/1167481188 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efc5c00f450 con 0x7efc6c105360 2024-05-27T07:56:55.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.915+0000 7efc805e5b80 1 Processor -- start 2024-05-27T07:56:55.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.915+0000 7efc805e5b80 1 -- start start 2024-05-27T07:56:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.915+0000 7efc63400700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efc500b29f0 0x7efc500b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.915+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15fd89f0 0x55fa1606c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.915+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 0x55fa1606cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.915+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa160786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.916+0000 7efc805e5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fa15f99e40 con 0x55fa160762c0 2024-05-27T07:56:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.916+0000 7efc805e5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fa15f899e0 con 0x55fa15eece40 2024-05-27T07:56:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.916+0000 7efc805e5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fa15f8da20 con 0x55fa15fd89f0 2024-05-27T07:56:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.916+0000 7efc63400700 1 -- 172.21.15.90:0/1167481188 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7efc5c0e27f0 con 0x7efc6c105360 2024-05-27T07:56:55.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.916+0000 7efc6b400700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efc500b29f0 0x7efc500b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.916+0000 7efc6b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa160786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.916+0000 7efc6aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15fd89f0 0x55fa1606c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.916+0000 7efc6b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa160786e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57306/0 (socket says 172.21.15.90:57306) 2024-05-27T07:56:55.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.916+0000 7efc6b400700 1 -- 172.21.15.90:0/3706119170 learned_addr learned my addr 172.21.15.90:0/3706119170 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:55.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc6a000700 1 --2- 172.21.15.90:0/3706119170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 0x55fa1606cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc6b400700 1 -- 172.21.15.90:0/3706119170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15fd89f0 msgr2=0x55fa1606c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc6b400700 1 --2- 172.21.15.90:0/3706119170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15fd89f0 0x55fa1606c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc6b400700 1 -- 172.21.15.90:0/3706119170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 msgr2=0x55fa1606cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc6b400700 1 --2- 172.21.15.90:0/3706119170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 0x55fa1606cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc6b400700 1 -- 172.21.15.90:0/3706119170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa15f52420 con 0x55fa160762c0 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc6a000700 1 --2- 172.21.15.90:0/3706119170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 0x55fa1606cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc6b400700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efc500b29f0 0x7efc500b4f00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc64007fd0 tx=0x7efc6400a490 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc6b400700 1 --2- 172.21.15.90:0/3706119170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa160786e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7efc64017d60 tx=0x7efc6401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443fcd02605a952e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc57e00700 1 -- 172.21.15.90:0/3706119170 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64020070 con 0x55fa160762c0 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc57e00700 1 -- 172.21.15.90:0/3706119170 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc6400a090 con 0x55fa160762c0 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.917+0000 7efc57e00700 1 -- 172.21.15.90:0/3706119170 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64019e30 con 0x55fa160762c0 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.918+0000 7efc805e5b80 1 -- 172.21.15.90:0/3706119170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 msgr2=0x55fa160786e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:55.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.918+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3706119170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa160786e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7efc64017d60 tx=0x7efc6401a000 comp rx=0 tx=0).stop 2024-05-27T07:56:55.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.918+0000 7efc805e5b80 1 -- 172.21.15.90:0/3706119170 shutdown_connections 2024-05-27T07:56:55.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.918+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3706119170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa160786e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.918+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3706119170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15fd89f0 0x55fa1606c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.918+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3706119170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 0x55fa1606cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.918+0000 7efc805e5b80 1 -- 172.21.15.90:0/3706119170 >> 172.21.15.90:0/3706119170 conn(0x55fa15f86390 msgr2=0x55fa16067710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:55.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.919+0000 7efc805e5b80 1 -- 172.21.15.90:0/3706119170 shutdown_connections 2024-05-27T07:56:55.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.919+0000 7efc805e5b80 1 -- 172.21.15.90:0/3706119170 wait complete. 2024-05-27T07:56:55.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.919+0000 7efc805e5b80 1 -- start start 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.920+0000 7efc805e5b80 1 Processor -- start 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.920+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15eece40 0x55fa16073240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.920+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15fd89f0 0x55fa16073780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.920+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa160762c0 0x55fa16073cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.920+0000 7efc805e5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fa15e8ffa0 con 0x55fa15eece40 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.920+0000 7efc805e5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fa15f2a760 con 0x55fa160762c0 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.920+0000 7efc805e5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fa16078da0 con 0x55fa15fd89f0 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.920+0000 7efc805e5b80 10 client.0 _ll_register_callbacks cb 0x55fa15f33c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15eece40 0x55fa16073240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15eece40 0x55fa16073240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57318/0 (socket says 172.21.15.90:57318) 2024-05-27T07:56:55.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6aa00700 1 -- 172.21.15.90:0/1848429222 learned_addr learned my addr 172.21.15.90:0/1848429222 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6a000700 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15fd89f0 0x55fa16073780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6b400700 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa160762c0 0x55fa16073cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6aa00700 1 -- 172.21.15.90:0/1848429222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15fd89f0 msgr2=0x55fa16073780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6aa00700 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15fd89f0 0x55fa16073780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6aa00700 1 -- 172.21.15.90:0/1848429222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa160762c0 msgr2=0x55fa16073cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6aa00700 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa160762c0 0x55fa16073cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6aa00700 1 -- 172.21.15.90:0/1848429222 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa15ed98d0 con 0x55fa15eece40 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc6aa00700 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15eece40 0x55fa16073240 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0a9660 tx=0x7efc5c0bc0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6061a8e1d5d0e89 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc54c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.921+0000 7efc54c00700 1 -- 172.21.15.90:0/1848429222 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c0a50f0 con 0x55fa15eece40 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc54c00700 1 -- 172.21.15.90:0/1848429222 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7efc5c007c40 con 0x55fa15eece40 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc805e5b80 1 -- 172.21.15.90:0/1848429222 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc6400a090 con 0x55fa15eece40 2024-05-27T07:56:55.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc54c00700 1 -- 172.21.15.90:0/1848429222 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7efc5c0dffa0 con 0x55fa15eece40 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc805e5b80 10 client.17358 fetch_fsmap learned FSMap version 64 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc805e5b80 1 -- 172.21.15.90:0/1848429222 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55fa15f5a060 con 0x55fa15eece40 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc54c00700 1 -- 172.21.15.90:0/1848429222 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc6400a090 con 0x55fa15eece40 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc54c00700 1 -- 172.21.15.90:0/1848429222 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7efc5c0c3c00 con 0x55fa15eece40 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc805e5b80 10 client.17358 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc805e5b80 10 client.17358 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc805e5b80 20 client.17358 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc805e5b80 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fa16085ea0 0x55fa160882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc805e5b80 4 client.17358 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.922+0000 7efc805e5b80 1 -- 172.21.15.90:0/1848429222 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa15f9bd90 con 0x55fa16085ea0 2024-05-27T07:56:55.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.923+0000 7efc6a000700 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fa16085ea0 0x55fa160882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.924+0000 7efc6a000700 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fa16085ea0 0x55fa160882f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=51060cd48451cca2 server_cookie=42ebfbbf1a0acd74 in_seq=0 out_seq=0 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.924+0000 7efc54c00700 10 client.17358 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc54c00700 1 -- 172.21.15.90:0/1848429222 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fa15f9bd90 con 0x55fa16085ea0 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc54c00700 10 client.17358 handle_command_reply: tid=0 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc805e5b80 1 client.17358 shutdown 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc805e5b80 1 -- 172.21.15.90:0/1848429222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15eece40 msgr2=0x55fa16073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc805e5b80 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15eece40 0x55fa16073240 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7efc5c0a9660 tx=0x7efc5c0bc0a0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc805e5b80 1 -- 172.21.15.90:0/1848429222 shutdown_connections 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc805e5b80 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fa16085ea0 0x55fa160882f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc805e5b80 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa160762c0 0x55fa16073cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc805e5b80 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15eece40 0x55fa16073240 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc805e5b80 1 --2- 172.21.15.90:0/1848429222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15fd89f0 0x55fa16073780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc805e5b80 1 -- 172.21.15.90:0/1848429222 >> 172.21.15.90:0/1848429222 conn(0x55fa15f86390 msgr2=0x55fa16067300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:55.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.926+0000 7efc54c00700 0 client.17358 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:56:55.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.927+0000 7efc805e5b80 1 -- 172.21.15.90:0/1848429222 shutdown_connections 2024-05-27T07:56:55.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.927+0000 7efc805e5b80 1 -- 172.21.15.90:0/1848429222 wait complete. 2024-05-27T07:56:55.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.927+0000 7efc805e5b80 20 client.17358 trim_cache size 0 max 16384 2024-05-27T07:56:55.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.948+0000 7efc805e5b80 1 Processor -- start 2024-05-27T07:56:55.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.949+0000 7efc805e5b80 1 -- start start 2024-05-27T07:56:55.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.949+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15eece40 0x55fa160721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.949+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15fd89f0 0x55fa16081ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.949+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa1606b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.949+0000 7efc805e5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c010720 con 0x55fa160762c0 2024-05-27T07:56:55.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.949+0000 7efc805e5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c00d770 con 0x55fa15fd89f0 2024-05-27T07:56:55.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.949+0000 7efc805e5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c00f980 con 0x55fa15eece40 2024-05-27T07:56:55.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.950+0000 7efc6b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa1606b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.950+0000 7efc6b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa1606b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57332/0 (socket says 172.21.15.90:57332) 2024-05-27T07:56:55.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.950+0000 7efc6b400700 1 -- 172.21.15.90:0/3338880401 learned_addr learned my addr 172.21.15.90:0/3338880401 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:55.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.950+0000 7efc6b400700 1 -- 172.21.15.90:0/3338880401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15eece40 msgr2=0x55fa160721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:55.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.950+0000 7efc6b400700 1 --2- 172.21.15.90:0/3338880401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15eece40 0x55fa160721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.950+0000 7efc6b400700 1 -- 172.21.15.90:0/3338880401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15fd89f0 msgr2=0x55fa16081ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:55.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.950+0000 7efc6b400700 1 --2- 172.21.15.90:0/3338880401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15fd89f0 0x55fa16081ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.950+0000 7efc6b400700 1 -- 172.21.15.90:0/3338880401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa16067c50 con 0x55fa160762c0 2024-05-27T07:56:55.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.952+0000 7efc6b400700 1 --2- 172.21.15.90:0/3338880401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa1606b830 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7efc640199c0 tx=0x7efc6400a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606b8b2ce74fab82 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:55.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.952+0000 7efc57400700 1 -- 172.21.15.90:0/3338880401 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64033070 con 0x55fa160762c0 2024-05-27T07:56:55.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.953+0000 7efc57400700 1 -- 172.21.15.90:0/3338880401 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc6400a090 con 0x55fa160762c0 2024-05-27T07:56:55.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.953+0000 7efc57400700 1 -- 172.21.15.90:0/3338880401 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64020040 con 0x55fa160762c0 2024-05-27T07:56:55.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.954+0000 7efc805e5b80 1 -- 172.21.15.90:0/3338880401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 msgr2=0x55fa1606b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:55.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.954+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3338880401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa1606b830 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7efc640199c0 tx=0x7efc6400a7e0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.954+0000 7efc805e5b80 1 -- 172.21.15.90:0/3338880401 shutdown_connections 2024-05-27T07:56:55.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.954+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3338880401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa160762c0 0x55fa1606b830 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.954+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3338880401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa15eece40 0x55fa160721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.954+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3338880401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15fd89f0 0x55fa16081ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.954+0000 7efc805e5b80 1 -- 172.21.15.90:0/3338880401 >> 172.21.15.90:0/3338880401 conn(0x55fa15f86390 msgr2=0x55fa1606b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:55.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.955+0000 7efc805e5b80 1 -- 172.21.15.90:0/3338880401 shutdown_connections 2024-05-27T07:56:55.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.955+0000 7efc805e5b80 1 -- 172.21.15.90:0/3338880401 wait complete. 2024-05-27T07:56:55.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.955+0000 7efc805e5b80 1 -- start start 2024-05-27T07:56:55.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.956+0000 7efc805e5b80 1 Processor -- start 2024-05-27T07:56:55.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.956+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 0x55fa1607e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.956+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15fd89f0 0x55fa16079870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.956+0000 7efc805e5b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa160762c0 0x55fa16079db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.956+0000 7efc805e5b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c00f2b0 con 0x55fa15fd89f0 2024-05-27T07:56:55.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.956+0000 7efc805e5b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c00de40 con 0x55fa15eece40 2024-05-27T07:56:55.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.956+0000 7efc805e5b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c00e510 con 0x55fa160762c0 2024-05-27T07:56:55.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.956+0000 7efc805e5b80 10 client.0 _ll_register_callbacks cb 0x55fa15f33c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:56:55.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.958+0000 7efc6aa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 0x55fa1607e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.958+0000 7efc6aa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 0x55fa1607e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45750/0 (socket says 172.21.15.90:45750) 2024-05-27T07:56:55.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.958+0000 7efc6aa00700 1 -- 172.21.15.90:0/3633765795 learned_addr learned my addr 172.21.15.90:0/3633765795 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:56:55.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.958+0000 7efc6b400700 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa160762c0 0x55fa16079db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.959+0000 7efc6b400700 1 -- 172.21.15.90:0/3633765795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 msgr2=0x55fa1607e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:55.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.959+0000 7efc6b400700 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 0x55fa1607e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.959+0000 7efc6b400700 1 -- 172.21.15.90:0/3633765795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15fd89f0 msgr2=0x55fa16079870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:56:55.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.959+0000 7efc6b400700 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15fd89f0 0x55fa16079870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.959+0000 7efc6b400700 1 -- 172.21.15.90:0/3633765795 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa16074030 con 0x55fa160762c0 2024-05-27T07:56:55.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.960+0000 7efc6b400700 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa160762c0 0x55fa16079db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55fa16072730 tx=0x7efc6400a240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f1889104b3cdae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:56:55.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.960+0000 7efc54c00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:56:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.965+0000 7efc54c00700 1 -- 172.21.15.90:0/3633765795 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64033070 con 0x55fa160762c0 2024-05-27T07:56:55.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.965+0000 7efc805e5b80 1 -- 172.21.15.90:0/3633765795 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc6400a090 con 0x55fa160762c0 2024-05-27T07:56:55.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.966+0000 7efc54c00700 1 -- 172.21.15.90:0/3633765795 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7efc64003a00 con 0x55fa160762c0 2024-05-27T07:56:55.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.967+0000 7efc54c00700 1 -- 172.21.15.90:0/3633765795 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7efc64020040 con 0x55fa160762c0 2024-05-27T07:56:55.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.968+0000 7efc805e5b80 10 client.27001 fetch_fsmap learned FSMap version 64 2024-05-27T07:56:55.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.968+0000 7efc805e5b80 1 -- 172.21.15.90:0/3633765795 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55fa16101430 con 0x55fa160762c0 2024-05-27T07:56:55.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.972+0000 7efc54c00700 1 -- 172.21.15.90:0/3633765795 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc6400a090 con 0x55fa160762c0 2024-05-27T07:56:55.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.973+0000 7efc54c00700 1 -- 172.21.15.90:0/3633765795 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7efc6402d070 con 0x55fa160762c0 2024-05-27T07:56:55.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.973+0000 7efc805e5b80 10 client.27001 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:56:55.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.973+0000 7efc805e5b80 10 client.27001 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:56:55.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.973+0000 7efc805e5b80 20 client.27001 populate_metadata read hostname 'smithi090' 2024-05-27T07:56:55.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.973+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fa16101a50 0x55fa16103e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:56:55.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.973+0000 7efc805e5b80 4 client.27001 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:56:55.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.973+0000 7efc805e5b80 1 -- 172.21.15.90:0/3633765795 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa15f95570 con 0x55fa16101a50 2024-05-27T07:56:55.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.976+0000 7efc6aa00700 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fa16101a50 0x55fa16103e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:56:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.979+0000 7efc54c00700 10 client.27001 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:56:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.979+0000 7efc6aa00700 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fa16101a50 0x55fa16103e70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=90d76e979bdfde68 server_cookie=10b6172a027d82ef in_seq=0 out_seq=0 2024-05-27T07:56:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.980+0000 7efc54c00700 1 -- 172.21.15.90:0/3633765795 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa15f95570 con 0x55fa16101a50 2024-05-27T07:56:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.980+0000 7efc54c00700 10 client.27001 handle_command_reply: tid=0 2024-05-27T07:56:55.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.980+0000 7efc805e5b80 1 client.27001 shutdown 2024-05-27T07:56:55.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.981+0000 7efc805e5b80 1 -- 172.21.15.90:0/3633765795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa160762c0 msgr2=0x55fa16079db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:55.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.981+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa160762c0 0x55fa16079db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55fa16072730 tx=0x7efc6400a240 comp rx=0 tx=0).stop 2024-05-27T07:56:55.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.982+0000 7efc805e5b80 1 -- 172.21.15.90:0/3633765795 shutdown_connections 2024-05-27T07:56:55.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.982+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55fa16101a50 0x55fa16103e70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.982+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fa160762c0 0x55fa16079db0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.982+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa15eece40 0x55fa1607e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.982+0000 7efc805e5b80 1 --2- 172.21.15.90:0/3633765795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fa15fd89f0 0x55fa16079870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.982+0000 7efc805e5b80 1 -- 172.21.15.90:0/3633765795 >> 172.21.15.90:0/3633765795 conn(0x55fa15f86390 msgr2=0x55fa1606e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:55.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.982+0000 7efc54c00700 0 client.27001 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:56:55.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.982+0000 7efc805e5b80 1 -- 172.21.15.90:0/3633765795 shutdown_connections 2024-05-27T07:56:55.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.982+0000 7efc805e5b80 1 -- 172.21.15.90:0/3633765795 wait complete. 2024-05-27T07:56:55.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.983+0000 7efc805e5b80 20 client.27001 trim_cache size 0 max 16384 2024-05-27T07:56:55.985 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:56:55.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.986+0000 7efc6be00700 1 -- 172.21.15.90:0/1167481188 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efc500b29f0 msgr2=0x7efc500b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:55.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.986+0000 7efc6be00700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efc500b29f0 0x7efc500b4f00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efc64007fd0 tx=0x7efc6400a490 comp rx=0 tx=0).stop 2024-05-27T07:56:55.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.986+0000 7efc6be00700 1 -- 172.21.15.90:0/1167481188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc6c105360 msgr2=0x7efc6c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:56:55.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.986+0000 7efc6be00700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc6c105360 0x7efc6c19ce80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00c970 tx=0x7efc5c00cd30 comp rx=0 tx=0).stop 2024-05-27T07:56:55.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.986+0000 7efc6be00700 1 -- 172.21.15.90:0/1167481188 shutdown_connections 2024-05-27T07:56:55.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.986+0000 7efc6be00700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc6c10e740 0x7efc6c1a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.986+0000 7efc6be00700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc6c105360 0x7efc6c19ce80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.986+0000 7efc6be00700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efc500b29f0 0x7efc500b4f00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.986+0000 7efc6be00700 1 --2- 172.21.15.90:0/1167481188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc6c105cb0 0x7efc6c19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:56:55.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.986+0000 7efc6be00700 1 -- 172.21.15.90:0/1167481188 >> 172.21.15.90:0/1167481188 conn(0x7efc6c078ab0 msgr2=0x7efc6c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:56:55.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.988+0000 7efc6be00700 1 -- 172.21.15.90:0/1167481188 shutdown_connections 2024-05-27T07:56:55.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:56:55.988+0000 7efc6be00700 1 -- 172.21.15.90:0/1167481188 wait complete. 2024-05-27T07:56:56.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:56 smithi090 ceph-mon[31429]: pgmap v333: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.0 MiB/s wr, 680 op/s 2024-05-27T07:56:56.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:56 smithi136 ceph-mon[35324]: pgmap v333: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.0 MiB/s wr, 680 op/s 2024-05-27T07:56:56.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:56 smithi129 ceph-mon[35254]: pgmap v333: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.0 MiB/s wr, 680 op/s 2024-05-27T07:56:57.391 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:56:58.213 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:56:58.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:56:58 smithi090 ceph-mon[31429]: pgmap v334: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-05-27T07:56:58.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:56:58 smithi136 ceph-mon[35324]: pgmap v334: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-05-27T07:56:58.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:56:58 smithi129 ceph-mon[35254]: pgmap v334: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.6 MiB/s wr, 511 op/s 2024-05-27T07:56:58.831 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 400.0 s, 502.1 tps, lat 198.523 ms stddev 188.828 2024-05-27T07:57:00.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:00 smithi090 ceph-mon[31429]: pgmap v335: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.9 MiB/s wr, 674 op/s 2024-05-27T07:57:00.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:00 smithi136 ceph-mon[35324]: pgmap v335: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.9 MiB/s wr, 674 op/s 2024-05-27T07:57:00.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:00 smithi129 ceph-mon[35254]: pgmap v335: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5.9 MiB/s wr, 674 op/s 2024-05-27T07:57:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.254+0000 7f6380e00640 1 -- 172.21.15.90:0/3957924084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f637c10ee70 msgr2=0x7f637c10f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.254+0000 7f6380e00640 1 --2- 172.21.15.90:0/3957924084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f637c10ee70 0x7f637c10f2a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f636c009900 tx=0x7f636c02a410 comp rx=0 tx=0).stop 2024-05-27T07:57:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.254+0000 7f6380e00640 1 -- 172.21.15.90:0/3957924084 shutdown_connections 2024-05-27T07:57:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.254+0000 7f6380e00640 1 --2- 172.21.15.90:0/3957924084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f637c10f930 0x7f637c1170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.254+0000 7f6380e00640 1 --2- 172.21.15.90:0/3957924084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f637c10ee70 0x7f637c10f2a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.254+0000 7f6380e00640 1 --2- 172.21.15.90:0/3957924084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c10a8e0 0x7f637c10acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.254+0000 7f6380e00640 1 -- 172.21.15.90:0/3957924084 >> 172.21.15.90:0/3957924084 conn(0x7f637c06e8f0 msgr2=0x7f637c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.254+0000 7f6380e00640 1 -- 172.21.15.90:0/3957924084 shutdown_connections 2024-05-27T07:57:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.255+0000 7f6380e00640 1 -- 172.21.15.90:0/3957924084 wait complete. 2024-05-27T07:57:02.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.257+0000 7f6380e00640 1 Processor -- start 2024-05-27T07:57:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.257+0000 7f6380e00640 1 -- start start 2024-05-27T07:57:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.258+0000 7f6380e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f637c10a8e0 0x7f637c11ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.258+0000 7f6380e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f637c10ee70 0x7f637c11bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.258+0000 7f6380e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c10f930 0x7f637c11c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.258+0000 7f6380e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f637c11a2f0 con 0x7f637c10a8e0 2024-05-27T07:57:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.258+0000 7f6380e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f637c11a170 con 0x7f637c10f930 2024-05-27T07:57:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.258+0000 7f6380e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f637c11a470 con 0x7f637c10ee70 2024-05-27T07:57:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.259+0000 7f6379e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f637c10a8e0 0x7f637c11ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.259+0000 7f6379e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f637c10a8e0 0x7f637c11ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51038/0 (socket says 172.21.15.90:51038) 2024-05-27T07:57:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.259+0000 7f6379e00640 1 -- 172.21.15.90:0/912819075 learned_addr learned my addr 172.21.15.90:0/912819075 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.259+0000 7f6379400640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f637c10ee70 0x7f637c11bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.259+0000 7f637a800640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c10f930 0x7f637c11c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.260+0000 7f6379e00640 1 -- 172.21.15.90:0/912819075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f637c10ee70 msgr2=0x7f637c11bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.260+0000 7f6379e00640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f637c10ee70 0x7f637c11bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.260+0000 7f6379e00640 1 -- 172.21.15.90:0/912819075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c10f930 msgr2=0x7f637c11c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.260+0000 7f6379e00640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c10f930 0x7f637c11c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.260+0000 7f6379e00640 1 -- 172.21.15.90:0/912819075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637c11c940 con 0x7f637c10a8e0 2024-05-27T07:57:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.260+0000 7f6379e00640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f637c10a8e0 0x7f637c11ed30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f636800e9f0 tx=0x7f636800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.261+0000 7f6372a00640 1 -- 172.21.15.90:0/912819075 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636800cd80 con 0x7f637c10a8e0 2024-05-27T07:57:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.261+0000 7f6372a00640 1 -- 172.21.15.90:0/912819075 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63680108c0 con 0x7f637c10a8e0 2024-05-27T07:57:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.261+0000 7f6380e00640 1 -- 172.21.15.90:0/912819075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637c11cc30 con 0x7f637c10a8e0 2024-05-27T07:57:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.261+0000 7f6372a00640 1 -- 172.21.15.90:0/912819075 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63680187c0 con 0x7f637c10a8e0 2024-05-27T07:57:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.261+0000 7f6380e00640 1 -- 172.21.15.90:0/912819075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637c1ab6c0 con 0x7f637c10a8e0 2024-05-27T07:57:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.268+0000 7f6372a00640 1 -- 172.21.15.90:0/912819075 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6368010430 con 0x7f637c10a8e0 2024-05-27T07:57:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.271+0000 7f6372a00640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63600b29a0 0x7f63600b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.272+0000 7f6372a00640 1 -- 172.21.15.90:0/912819075 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f63680e2e60 con 0x7f637c10a8e0 2024-05-27T07:57:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.272+0000 7f6379400640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63600b29a0 0x7f63600b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.273+0000 7f6379400640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63600b29a0 0x7f63600b4eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f636c005ab0 tx=0x7f636c000bc0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.326+0000 7f6387ddf740 1 Processor -- start 2024-05-27T07:57:02.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.327+0000 7f6387ddf740 1 -- start start 2024-05-27T07:57:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.327+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 0x561672fd7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.327+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fd7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.327+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c144d0 0x561672fd82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.327+0000 7f6387ddf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561672f48b90 con 0x561672c0c2f0 2024-05-27T07:57:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.327+0000 7f6387ddf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561672f38730 con 0x561672c351b0 2024-05-27T07:57:02.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.327+0000 7f6387ddf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561672f3c770 con 0x561672c144d0 2024-05-27T07:57:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.328+0000 7f6379400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fd7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.328+0000 7f6379400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fd7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51046/0 (socket says 172.21.15.90:51046) 2024-05-27T07:57:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.328+0000 7f6379400640 1 -- 172.21.15.90:0/797541651 learned_addr learned my addr 172.21.15.90:0/797541651 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:02.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.328+0000 7f637a800640 1 --2- 172.21.15.90:0/797541651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c144d0 0x561672fd82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.328+0000 7f6379e00640 1 --2- 172.21.15.90:0/797541651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 0x561672fd7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.329+0000 7f6379400640 1 -- 172.21.15.90:0/797541651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c144d0 msgr2=0x561672fd82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:02.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.329+0000 7f6379400640 1 --2- 172.21.15.90:0/797541651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c144d0 0x561672fd82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.329+0000 7f6379400640 1 -- 172.21.15.90:0/797541651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 msgr2=0x561672fd7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.329+0000 7f6379400640 1 --2- 172.21.15.90:0/797541651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 0x561672fd7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.329+0000 7f6379400640 1 -- 172.21.15.90:0/797541651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561672ed18b0 con 0x561672c0c2f0 2024-05-27T07:57:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.329+0000 7f6379400640 1 --2- 172.21.15.90:0/797541651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fd7da0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f636c033000 tx=0x7f636c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db25c41dac5d7c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.330+0000 7f6367400640 1 -- 172.21.15.90:0/797541651 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c035040 con 0x561672c0c2f0 2024-05-27T07:57:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.330+0000 7f6367400640 1 -- 172.21.15.90:0/797541651 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f636c04e070 con 0x561672c0c2f0 2024-05-27T07:57:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.330+0000 7f6367400640 1 -- 172.21.15.90:0/797541651 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c0496e0 con 0x561672c0c2f0 2024-05-27T07:57:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.331+0000 7f6387ddf740 1 -- 172.21.15.90:0/797541651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 msgr2=0x561672fd7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.331+0000 7f6387ddf740 1 --2- 172.21.15.90:0/797541651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fd7da0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f636c033000 tx=0x7f636c03d000 comp rx=0 tx=0).stop 2024-05-27T07:57:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.334+0000 7f6387ddf740 1 -- 172.21.15.90:0/797541651 shutdown_connections 2024-05-27T07:57:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.334+0000 7f6387ddf740 1 --2- 172.21.15.90:0/797541651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c144d0 0x561672fd82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.334+0000 7f6387ddf740 1 --2- 172.21.15.90:0/797541651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fd7da0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.334+0000 7f6387ddf740 1 --2- 172.21.15.90:0/797541651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 0x561672fd7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.334+0000 7f6387ddf740 1 -- 172.21.15.90:0/797541651 >> 172.21.15.90:0/797541651 conn(0x561672c29300 msgr2=0x561672fd4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.334+0000 7f6387ddf740 1 -- 172.21.15.90:0/797541651 shutdown_connections 2024-05-27T07:57:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.334+0000 7f6387ddf740 1 -- 172.21.15.90:0/797541651 wait complete. 2024-05-27T07:57:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.335+0000 7f6387ddf740 1 -- start start 2024-05-27T07:57:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.335+0000 7f6387ddf740 1 Processor -- start 2024-05-27T07:57:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.336+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fe1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.336+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c144d0 0x561672fe1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.336+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 0x561672fedc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.336+0000 7f6387ddf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561672c1e790 con 0x561672c0c2f0 2024-05-27T07:57:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.336+0000 7f6387ddf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561672ee0bc0 con 0x561672c351b0 2024-05-27T07:57:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.336+0000 7f6387ddf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561672fe4930 con 0x561672c144d0 2024-05-27T07:57:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.336+0000 7f6387ddf740 10 client.0 _ll_register_callbacks cb 0x561672ebe400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f6379e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fe1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f6379e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fe1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51058/0 (socket says 172.21.15.90:51058) 2024-05-27T07:57:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f6379e00640 1 -- 172.21.15.90:0/848341400 learned_addr learned my addr 172.21.15.90:0/848341400 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:02.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f6379400640 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c144d0 0x561672fe1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f637a800640 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 0x561672fedc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f6379e00640 1 -- 172.21.15.90:0/848341400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c144d0 msgr2=0x561672fe1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f6379e00640 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c144d0 0x561672fe1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f6379e00640 1 -- 172.21.15.90:0/848341400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 msgr2=0x561672fedc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f6379e00640 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 0x561672fedc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f6379e00640 1 -- 172.21.15.90:0/848341400 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561672c4d210 con 0x561672c0c2f0 2024-05-27T07:57:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.337+0000 7f637a800640 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 0x561672fedc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.338+0000 7f6379e00640 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fe1a30 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f63680a6fe0 tx=0x7f6368010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9527087f96a417e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.338+0000 7f635be00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:57:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.339+0000 7f635be00640 1 -- 172.21.15.90:0/848341400 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63680b2bd0 con 0x561672c0c2f0 2024-05-27T07:57:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.339+0000 7f635be00640 1 -- 172.21.15.90:0/848341400 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f63680a8fd0 con 0x561672c0c2f0 2024-05-27T07:57:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.339+0000 7f6387ddf740 1 -- 172.21.15.90:0/848341400 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f636c04e070 con 0x561672c0c2f0 2024-05-27T07:57:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.339+0000 7f635be00640 1 -- 172.21.15.90:0/848341400 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f63680c43e0 con 0x561672c0c2f0 2024-05-27T07:57:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.339+0000 7f6387ddf740 10 client.17373 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.339+0000 7f6387ddf740 1 -- 172.21.15.90:0/848341400 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561672d19620 con 0x561672c0c2f0 2024-05-27T07:57:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.341+0000 7f635be00640 1 -- 172.21.15.90:0/848341400 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f636c04e070 con 0x561672c0c2f0 2024-05-27T07:57:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.342+0000 7f635be00640 1 -- 172.21.15.90:0/848341400 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f63680cac20 con 0x561672c0c2f0 2024-05-27T07:57:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.342+0000 7f6387ddf740 10 client.17373 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.342+0000 7f6387ddf740 10 client.17373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.342+0000 7f6387ddf740 20 client.17373 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.342+0000 7f6387ddf740 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561672bf3d80 0x561672ff3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.342+0000 7f6387ddf740 4 client.17373 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.342+0000 7f6387ddf740 1 -- 172.21.15.90:0/848341400 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561672f4aae0 con 0x561672bf3d80 2024-05-27T07:57:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.343+0000 7f6379400640 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561672bf3d80 0x561672ff3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.344+0000 7f6379400640 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561672bf3d80 0x561672ff3cb0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d961e5c1eeedeb6 server_cookie=c7e952b92ec9f206 in_seq=0 out_seq=0 2024-05-27T07:57:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.344+0000 7f635be00640 10 client.17373 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.347+0000 7f635be00640 1 -- 172.21.15.90:0/848341400 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561672f4aae0 con 0x561672bf3d80 2024-05-27T07:57:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.347+0000 7f635be00640 10 client.17373 handle_command_reply: tid=0 2024-05-27T07:57:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.347+0000 7f6387ddf740 1 client.17373 shutdown 2024-05-27T07:57:02.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.348+0000 7f6387ddf740 1 -- 172.21.15.90:0/848341400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 msgr2=0x561672fe1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.349+0000 7f6387ddf740 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fe1a30 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f63680a6fe0 tx=0x7f6368010ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.349+0000 7f6387ddf740 1 -- 172.21.15.90:0/848341400 shutdown_connections 2024-05-27T07:57:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.349+0000 7f6387ddf740 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561672bf3d80 0x561672ff3cb0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.349+0000 7f6387ddf740 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c351b0 0x561672fedc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.349+0000 7f6387ddf740 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c144d0 0x561672fe1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.349+0000 7f6387ddf740 1 --2- 172.21.15.90:0/848341400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c0c2f0 0x561672fe1a30 secure :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f63680a6fe0 tx=0x7f6368010ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.349+0000 7f6387ddf740 1 -- 172.21.15.90:0/848341400 >> 172.21.15.90:0/848341400 conn(0x561672c29300 msgr2=0x561672fd7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.349+0000 7f635be00640 0 client.17373 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.351+0000 7f6387ddf740 1 -- 172.21.15.90:0/848341400 shutdown_connections 2024-05-27T07:57:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.351+0000 7f6387ddf740 1 -- 172.21.15.90:0/848341400 wait complete. 2024-05-27T07:57:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.352+0000 7f6387ddf740 20 client.17373 trim_cache size 0 max 16384 2024-05-27T07:57:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.420+0000 7f6387ddf740 1 Processor -- start 2024-05-27T07:57:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.420+0000 7f6387ddf740 1 -- start start 2024-05-27T07:57:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.420+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c144d0 0x561672fdd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.420+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c351b0 0x561672fe3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.420+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672bc91c0 0x561672fe19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.420+0000 7f6387ddf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f637c07f5a0 con 0x561672c351b0 2024-05-27T07:57:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.420+0000 7f6387ddf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f637c07c5f0 con 0x561672c144d0 2024-05-27T07:57:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.420+0000 7f6387ddf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f637c07e800 con 0x561672bc91c0 2024-05-27T07:57:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.422+0000 7f637a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672bc91c0 0x561672fe19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.422+0000 7f637a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672bc91c0 0x561672fe19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:43820/0 (socket says 172.21.15.90:43820) 2024-05-27T07:57:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.422+0000 7f637a800640 1 -- 172.21.15.90:0/2887836429 learned_addr learned my addr 172.21.15.90:0/2887836429 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.423+0000 7f637a800640 1 -- 172.21.15.90:0/2887836429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c144d0 msgr2=0x561672fdd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.423+0000 7f637a800640 1 --2- 172.21.15.90:0/2887836429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c144d0 0x561672fdd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.423+0000 7f637a800640 1 -- 172.21.15.90:0/2887836429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c351b0 msgr2=0x561672fe3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.423+0000 7f637a800640 1 --2- 172.21.15.90:0/2887836429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c351b0 0x561672fe3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.423+0000 7f637a800640 1 -- 172.21.15.90:0/2887836429 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561672fd4c10 con 0x561672bc91c0 2024-05-27T07:57:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.424+0000 7f637a800640 1 --2- 172.21.15.90:0/2887836429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672bc91c0 0x561672fe19d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6374009fd0 tx=0x7f63740128e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f45bfb4fb5a5ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.425+0000 7f6366a00640 1 -- 172.21.15.90:0/2887836429 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374003a30 con 0x561672bc91c0 2024-05-27T07:57:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.425+0000 7f6366a00640 1 -- 172.21.15.90:0/2887836429 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6374003bd0 con 0x561672bc91c0 2024-05-27T07:57:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.425+0000 7f6366a00640 1 -- 172.21.15.90:0/2887836429 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374010770 con 0x561672bc91c0 2024-05-27T07:57:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.427+0000 7f6387ddf740 1 -- 172.21.15.90:0/2887836429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672bc91c0 msgr2=0x561672fe19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.428+0000 7f6387ddf740 1 --2- 172.21.15.90:0/2887836429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672bc91c0 0x561672fe19d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6374009fd0 tx=0x7f63740128e0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.428+0000 7f6387ddf740 1 -- 172.21.15.90:0/2887836429 shutdown_connections 2024-05-27T07:57:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.428+0000 7f6387ddf740 1 --2- 172.21.15.90:0/2887836429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672bc91c0 0x561672fe19d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.428+0000 7f6387ddf740 1 --2- 172.21.15.90:0/2887836429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672c351b0 0x561672fe3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.428+0000 7f6387ddf740 1 --2- 172.21.15.90:0/2887836429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c144d0 0x561672fdd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.428+0000 7f6387ddf740 1 -- 172.21.15.90:0/2887836429 >> 172.21.15.90:0/2887836429 conn(0x561672c29300 msgr2=0x561672fe12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.429+0000 7f6387ddf740 1 -- 172.21.15.90:0/2887836429 shutdown_connections 2024-05-27T07:57:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.429+0000 7f6387ddf740 1 -- 172.21.15.90:0/2887836429 wait complete. 2024-05-27T07:57:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.429+0000 7f6387ddf740 1 -- start start 2024-05-27T07:57:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.430+0000 7f6387ddf740 1 Processor -- start 2024-05-27T07:57:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.432+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672bc91c0 0x561672fe9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.432+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c144d0 0x561672fea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.432+0000 7f6387ddf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c351b0 0x561672fe7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.432+0000 7f6387ddf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f637c07e130 con 0x561672bc91c0 2024-05-27T07:57:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.432+0000 7f6387ddf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f637c07ccc0 con 0x561672c144d0 2024-05-27T07:57:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.432+0000 7f6387ddf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f637c07d390 con 0x561672c351b0 2024-05-27T07:57:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.432+0000 7f6387ddf740 10 client.0 _ll_register_callbacks cb 0x7f63680a9b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.433+0000 7f6379e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672bc91c0 0x561672fe9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.434+0000 7f637a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c351b0 0x561672fe7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.434+0000 7f637a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c351b0 0x561672fe7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:43830/0 (socket says 172.21.15.90:43830) 2024-05-27T07:57:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.434+0000 7f637a800640 1 -- 172.21.15.90:0/2917738599 learned_addr learned my addr 172.21.15.90:0/2917738599 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.435+0000 7f6379400640 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c144d0 0x561672fea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.435+0000 7f637a800640 1 -- 172.21.15.90:0/2917738599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c144d0 msgr2=0x561672fea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.435+0000 7f637a800640 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c144d0 0x561672fea430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.435+0000 7f637a800640 1 -- 172.21.15.90:0/2917738599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672bc91c0 msgr2=0x561672fe9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.435+0000 7f637a800640 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672bc91c0 0x561672fe9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.435+0000 7f637a800640 1 -- 172.21.15.90:0/2917738599 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561672fe7ae0 con 0x561672c351b0 2024-05-27T07:57:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.438+0000 7f637a800640 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c351b0 0x561672fe7350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6374004f40 tx=0x7f6374009320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e190b0183df2e872 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.438+0000 7f635be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:57:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.439+0000 7f635be00640 1 -- 172.21.15.90:0/2917738599 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374021040 con 0x561672c351b0 2024-05-27T07:57:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.440+0000 7f635be00640 1 -- 172.21.15.90:0/2917738599 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f6374003d80 con 0x561672c351b0 2024-05-27T07:57:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.441+0000 7f6387ddf740 1 -- 172.21.15.90:0/2917738599 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6374003bd0 con 0x561672c351b0 2024-05-27T07:57:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.441+0000 7f635be00640 1 -- 172.21.15.90:0/2917738599 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f6374004000 con 0x561672c351b0 2024-05-27T07:57:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.441+0000 7f6387ddf740 10 client.27019 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.441+0000 7f6387ddf740 1 -- 172.21.15.90:0/2917738599 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56167305be50 con 0x561672c351b0 2024-05-27T07:57:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.442+0000 7f635be00640 1 -- 172.21.15.90:0/2917738599 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6374003bd0 con 0x561672c351b0 2024-05-27T07:57:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.443+0000 7f635be00640 1 -- 172.21.15.90:0/2917738599 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f637402d030 con 0x561672c351b0 2024-05-27T07:57:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.444+0000 7f6387ddf740 10 client.27019 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.444+0000 7f6387ddf740 10 client.27019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.444+0000 7f6387ddf740 20 client.27019 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.444+0000 7f6387ddf740 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56167305c540 0x56167305e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.444+0000 7f6387ddf740 4 client.27019 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:57:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.444+0000 7f6387ddf740 1 -- 172.21.15.90:0/2917738599 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561672f442c0 con 0x56167305c540 2024-05-27T07:57:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.445+0000 7f6379e00640 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56167305c540 0x56167305e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.447+0000 7f635be00640 10 client.27019 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.447+0000 7f6379e00640 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56167305c540 0x56167305e960 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab78baf3589361f7 server_cookie=8781705967ea9ce9 in_seq=0 out_seq=0 2024-05-27T07:57:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.530+0000 7f635be00640 1 -- 172.21.15.90:0/2917738599 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561672f442c0 con 0x56167305c540 2024-05-27T07:57:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.530+0000 7f635be00640 10 client.27019 handle_command_reply: tid=0 2024-05-27T07:57:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.530+0000 7f6387ddf740 1 client.27019 shutdown 2024-05-27T07:57:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.530+0000 7f6387ddf740 1 -- 172.21.15.90:0/2917738599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c351b0 msgr2=0x561672fe7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.531+0000 7f6387ddf740 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c351b0 0x561672fe7350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6374004f40 tx=0x7f6374009320 comp rx=0 tx=0).stop 2024-05-27T07:57:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.531+0000 7f6387ddf740 1 -- 172.21.15.90:0/2917738599 shutdown_connections 2024-05-27T07:57:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.531+0000 7f6387ddf740 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56167305c540 0x56167305e960 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.531+0000 7f6387ddf740 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561672c144d0 0x561672fea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.531+0000 7f6387ddf740 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561672c351b0 0x561672fe7350 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.531+0000 7f6387ddf740 1 --2- 172.21.15.90:0/2917738599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561672bc91c0 0x561672fe9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.531+0000 7f635be00640 0 client.27019 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.531+0000 7f6387ddf740 1 -- 172.21.15.90:0/2917738599 >> 172.21.15.90:0/2917738599 conn(0x561672c29300 msgr2=0x561672fe3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.533+0000 7f6387ddf740 1 -- 172.21.15.90:0/2917738599 shutdown_connections 2024-05-27T07:57:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.533+0000 7f6387ddf740 1 -- 172.21.15.90:0/2917738599 wait complete. 2024-05-27T07:57:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.534+0000 7f6387ddf740 20 client.27019 trim_cache size 0 max 16384 2024-05-27T07:57:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:57:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:57:02.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:57:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.536+0000 7f6380e00640 1 -- 172.21.15.90:0/912819075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63600b29a0 msgr2=0x7f63600b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.536+0000 7f6380e00640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63600b29a0 0x7f63600b4eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f636c005ab0 tx=0x7f636c000bc0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.536+0000 7f6380e00640 1 -- 172.21.15.90:0/912819075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f637c10a8e0 msgr2=0x7f637c11ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.536+0000 7f6380e00640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f637c10a8e0 0x7f637c11ed30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f636800e9f0 tx=0x7f636800eeb0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.537+0000 7f6380e00640 1 -- 172.21.15.90:0/912819075 shutdown_connections 2024-05-27T07:57:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.537+0000 7f6380e00640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63600b29a0 0x7f63600b4eb0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f636c005ab0 tx=0x7f636c000bc0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.537+0000 7f6380e00640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c10f930 0x7f637c11c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.537+0000 7f6380e00640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f637c10ee70 0x7f637c11bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.537+0000 7f6380e00640 1 --2- 172.21.15.90:0/912819075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f637c10a8e0 0x7f637c11ed30 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.537+0000 7f6380e00640 1 -- 172.21.15.90:0/912819075 >> 172.21.15.90:0/912819075 conn(0x7f637c06e8f0 msgr2=0x7f637c115600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.539+0000 7f6380e00640 1 -- 172.21.15.90:0/912819075 shutdown_connections 2024-05-27T07:57:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:02.539+0000 7f6380e00640 1 -- 172.21.15.90:0/912819075 wait complete. 2024-05-27T07:57:02.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:02 smithi090 ceph-mon[31429]: pgmap v336: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 MiB/s wr, 511 op/s 2024-05-27T07:57:02.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:02 smithi136 ceph-mon[35324]: pgmap v336: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 MiB/s wr, 511 op/s 2024-05-27T07:57:02.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:02 smithi129 ceph-mon[35254]: pgmap v336: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 MiB/s wr, 511 op/s 2024-05-27T07:57:03.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:03 smithi090 ceph-mon[31429]: pgmap v337: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.5 MiB/s wr, 512 op/s 2024-05-27T07:57:03.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:03 smithi136 ceph-mon[35324]: pgmap v337: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.5 MiB/s wr, 512 op/s 2024-05-27T07:57:03.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:03 smithi129 ceph-mon[35254]: pgmap v337: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.5 MiB/s wr, 512 op/s 2024-05-27T07:57:03.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 405.0 s, 492.1 tps, lat 205.359 ms stddev 209.750 2024-05-27T07:57:05.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:05 smithi090 ceph-mon[31429]: osdmap e109: 12 total, 12 up, 12 in 2024-05-27T07:57:05.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:05 smithi090 ceph-mon[31429]: pgmap v339: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-05-27T07:57:05.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:05 smithi136 ceph-mon[35324]: osdmap e109: 12 total, 12 up, 12 in 2024-05-27T07:57:05.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:05 smithi136 ceph-mon[35324]: pgmap v339: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-05-27T07:57:05.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:05 smithi129 ceph-mon[35254]: osdmap e109: 12 total, 12 up, 12 in 2024-05-27T07:57:05.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:05 smithi129 ceph-mon[35254]: pgmap v339: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 3.9 MiB/s wr, 455 op/s 2024-05-27T07:57:08.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:08 smithi129 ceph-mon[35254]: pgmap v340: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 508 op/s 2024-05-27T07:57:08.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:08 smithi090 ceph-mon[31429]: pgmap v340: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 508 op/s 2024-05-27T07:57:08.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:08 smithi136 ceph-mon[35324]: pgmap v340: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 508 op/s 2024-05-27T07:57:08.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 410.0 s, 524.0 tps, lat 191.886 ms stddev 192.584 2024-05-27T07:57:09.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.664+0000 7fc99d200700 1 -- 172.21.15.90:0/2475553858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105530 msgr2=0x7fc998105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:09.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.664+0000 7fc99d200700 1 --2- 172.21.15.90:0/2475553858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105530 0x7fc998105940 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc98800dd20 tx=0x7fc98800f330 comp rx=0 tx=0).stop 2024-05-27T07:57:09.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.664+0000 7fc99d200700 1 -- 172.21.15.90:0/2475553858 shutdown_connections 2024-05-27T07:57:09.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.664+0000 7fc99d200700 1 --2- 172.21.15.90:0/2475553858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc99810e910 0x7fc998110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.664+0000 7fc99d200700 1 --2- 172.21.15.90:0/2475553858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105530 0x7fc998105940 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.664+0000 7fc99d200700 1 --2- 172.21.15.90:0/2475553858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc998105e80 0x7fc99810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.664+0000 7fc99d200700 1 -- 172.21.15.90:0/2475553858 >> 172.21.15.90:0/2475553858 conn(0x7fc9980fef40 msgr2=0x7fc998101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:09.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.664+0000 7fc99d200700 1 -- 172.21.15.90:0/2475553858 shutdown_connections 2024-05-27T07:57:09.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.664+0000 7fc99d200700 1 -- 172.21.15.90:0/2475553858 wait complete. 2024-05-27T07:57:09.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.666+0000 7fc99d200700 1 Processor -- start 2024-05-27T07:57:09.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.667+0000 7fc99d200700 1 -- start start 2024-05-27T07:57:09.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.667+0000 7fc99d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105530 0x7fc998198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.667+0000 7fc99d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105e80 0x7fc9981992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.667+0000 7fc99d200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99810e910 0x7fc99819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.668+0000 7fc995400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105e80 0x7fc9981992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.668+0000 7fc995400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105e80 0x7fc9981992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53454/0 (socket says 172.21.15.90:53454) 2024-05-27T07:57:09.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.668+0000 7fc99d200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc998112270 con 0x7fc998105530 2024-05-27T07:57:09.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.668+0000 7fc99d200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981120f0 con 0x7fc99810e910 2024-05-27T07:57:09.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.668+0000 7fc99d200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981123f0 con 0x7fc998105e80 2024-05-27T07:57:09.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.668+0000 7fc995400700 1 -- 172.21.15.90:0/3093246104 learned_addr learned my addr 172.21.15.90:0/3093246104 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:09.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.669+0000 7fc995e00700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105530 0x7fc998198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.669+0000 7fc996800700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99810e910 0x7fc99819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.669+0000 7fc995400700 1 -- 172.21.15.90:0/3093246104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99810e910 msgr2=0x7fc99819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:09.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.669+0000 7fc995400700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99810e910 0x7fc99819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.669+0000 7fc995400700 1 -- 172.21.15.90:0/3093246104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105530 msgr2=0x7fc998198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:09.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.669+0000 7fc995400700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105530 0x7fc998198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.669+0000 7fc995400700 1 -- 172.21.15.90:0/3093246104 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99819dc10 con 0x7fc998105e80 2024-05-27T07:57:09.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.670+0000 7fc995400700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105e80 0x7fc9981992e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc98400b4d0 tx=0x7fc98400b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:09.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.670+0000 7fc98ea00700 1 -- 172.21.15.90:0/3093246104 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984015020 con 0x7fc998105e80 2024-05-27T07:57:09.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.670+0000 7fc98ea00700 1 -- 172.21.15.90:0/3093246104 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc984011cc0 con 0x7fc998105e80 2024-05-27T07:57:09.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.670+0000 7fc98ea00700 1 -- 172.21.15.90:0/3093246104 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984010c10 con 0x7fc998105e80 2024-05-27T07:57:09.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.670+0000 7fc99d200700 1 -- 172.21.15.90:0/3093246104 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99819df00 con 0x7fc998105e80 2024-05-27T07:57:09.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.671+0000 7fc99d200700 1 -- 172.21.15.90:0/3093246104 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981a5820 con 0x7fc998105e80 2024-05-27T07:57:09.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.677+0000 7fc98ea00700 1 -- 172.21.15.90:0/3093246104 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc984018020 con 0x7fc998105e80 2024-05-27T07:57:09.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.677+0000 7fc9abdb1b80 1 Processor -- start 2024-05-27T07:57:09.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.678+0000 7fc9abdb1b80 1 -- start start 2024-05-27T07:57:09.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.678+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609978e69f0 0x560997981b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.678+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609977fae40 0x560997982090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.678+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x56099798e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.678+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5609978a7e40 con 0x5609977fae40 2024-05-27T07:57:09.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.678+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5609978979e0 con 0x5609978e69f0 2024-05-27T07:57:09.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.678+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56099789ba20 con 0x560997982690 2024-05-27T07:57:09.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.679+0000 7fc995e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609977fae40 0x560997982090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.679+0000 7fc995e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609977fae40 0x560997982090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35400/0 (socket says 172.21.15.90:35400) 2024-05-27T07:57:09.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.679+0000 7fc995e00700 1 -- 172.21.15.90:0/2949413793 learned_addr learned my addr 172.21.15.90:0/2949413793 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:09.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.680+0000 7fc995e00700 1 -- 172.21.15.90:0/2949413793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 msgr2=0x56099798e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:09.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.680+0000 7fc996800700 1 --2- 172.21.15.90:0/2949413793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609978e69f0 0x560997981b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.680+0000 7fc995e00700 1 --2- 172.21.15.90:0/2949413793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x56099798e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.681+0000 7fc995e00700 1 -- 172.21.15.90:0/2949413793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609978e69f0 msgr2=0x560997981b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:09.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.681+0000 7fc995e00700 1 --2- 172.21.15.90:0/2949413793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609978e69f0 0x560997981b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.681+0000 7fc995e00700 1 -- 172.21.15.90:0/2949413793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560997860420 con 0x5609977fae40 2024-05-27T07:57:09.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.681+0000 7fc996800700 1 --2- 172.21.15.90:0/2949413793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609978e69f0 0x560997981b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:09.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.681+0000 7fc98ea00700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc97c0b2a70 0x7fc97c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.681+0000 7fc98ea00700 1 -- 172.21.15.90:0/3093246104 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9584+0+0 (secure 0 0 0) 0x7fc9840e2090 con 0x7fc998105e80 2024-05-27T07:57:09.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.682+0000 7fc995e00700 1 --2- 172.21.15.90:0/2949413793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609977fae40 0x560997982090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc98800dd20 tx=0x7fc98800dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943bbc576aa79b63 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:09.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.684+0000 7fc983400700 1 -- 172.21.15.90:0/2949413793 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98801b070 con 0x5609977fae40 2024-05-27T07:57:09.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.684+0000 7fc983400700 1 -- 172.21.15.90:0/2949413793 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc98800d2e0 con 0x5609977fae40 2024-05-27T07:57:09.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.684+0000 7fc983400700 1 -- 172.21.15.90:0/2949413793 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988003ba0 con 0x5609977fae40 2024-05-27T07:57:09.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.684+0000 7fc996800700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc97c0b2a70 0x7fc97c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.686+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2949413793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609977fae40 msgr2=0x560997982090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:09.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.686+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/2949413793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609977fae40 0x560997982090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fc98800dd20 tx=0x7fc98800dad0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.686+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2949413793 shutdown_connections 2024-05-27T07:57:09.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.686+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/2949413793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x56099798e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.686+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/2949413793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609978e69f0 0x560997981b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.686+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/2949413793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609977fae40 0x560997982090 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.686+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2949413793 >> 172.21.15.90:0/2949413793 conn(0x560997894390 msgr2=0x560997975320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:09.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.688+0000 7fc996800700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc97c0b2a70 0x7fc97c0b4f80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5609979836e0 tx=0x7fc990009650 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:09.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.690+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2949413793 shutdown_connections 2024-05-27T07:57:09.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.690+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2949413793 wait complete. 2024-05-27T07:57:09.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.691+0000 7fc9abdb1b80 1 -- start start 2024-05-27T07:57:09.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.691+0000 7fc9abdb1b80 1 Processor -- start 2024-05-27T07:57:09.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.692+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x56099798c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.693+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609978e69f0 0x56099798c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.693+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560997982690 0x56099798caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.693+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56099779dfa0 con 0x560997982690 2024-05-27T07:57:09.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.693+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560997838760 con 0x5609977fae40 2024-05-27T07:57:09.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.693+0000 7fc995e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x56099798c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.694+0000 7fc995e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x56099798c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49622/0 (socket says 172.21.15.90:49622) 2024-05-27T07:57:09.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.694+0000 7fc995e00700 1 -- 172.21.15.90:0/2318045540 learned_addr learned my addr 172.21.15.90:0/2318045540 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:09.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.694+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56099798f340 con 0x5609978e69f0 2024-05-27T07:57:09.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.694+0000 7fc995400700 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609978e69f0 0x56099798c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.694+0000 7fc996800700 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560997982690 0x56099798caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.694+0000 7fc9abdb1b80 10 client.0 _ll_register_callbacks cb 0x560997841c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.695+0000 7fc996800700 1 -- 172.21.15.90:0/2318045540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609978e69f0 msgr2=0x56099798c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.695+0000 7fc996800700 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609978e69f0 0x56099798c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.695+0000 7fc996800700 1 -- 172.21.15.90:0/2318045540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 msgr2=0x56099798c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.695+0000 7fc996800700 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x56099798c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.695+0000 7fc996800700 1 -- 172.21.15.90:0/2318045540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609977e78d0 con 0x560997982690 2024-05-27T07:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.695+0000 7fc995400700 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609978e69f0 0x56099798c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.696+0000 7fc996800700 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560997982690 0x56099798caa0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc9900199b0 tx=0x7fc990004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95401d85f3a67f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:09.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.696+0000 7fc977e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:57:09.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.697+0000 7fc977e00700 1 -- 172.21.15.90:0/2318045540 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990027060 con 0x560997982690 2024-05-27T07:57:09.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.697+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2318045540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98800d2e0 con 0x560997982690 2024-05-27T07:57:09.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.697+0000 7fc977e00700 1 -- 172.21.15.90:0/2318045540 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9584+0+0 (secure 0 0 0) 0x7fc990022410 con 0x560997982690 2024-05-27T07:57:09.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.697+0000 7fc977e00700 1 -- 172.21.15.90:0/2318045540 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc99002e050 con 0x560997982690 2024-05-27T07:57:09.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.698+0000 7fc9abdb1b80 10 client.17385 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:09.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.698+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2318045540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560997868060 con 0x560997982690 2024-05-27T07:57:09.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.698+0000 7fc977e00700 1 -- 172.21.15.90:0/2318045540 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc98800d2e0 con 0x560997982690 2024-05-27T07:57:09.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.699+0000 7fc977e00700 1 -- 172.21.15.90:0/2318045540 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fc990017cc0 con 0x560997982690 2024-05-27T07:57:09.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.699+0000 7fc9abdb1b80 10 client.17385 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:09.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.699+0000 7fc9abdb1b80 10 client.17385 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:57:09.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.699+0000 7fc9abdb1b80 20 client.17385 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:09.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.699+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56099799c360 0x56099799e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.699+0000 7fc9abdb1b80 4 client.17385 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:09.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.699+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2318045540 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609978a9d90 con 0x56099799c360 2024-05-27T07:57:09.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.700+0000 7fc995e00700 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56099799c360 0x56099799e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.703+0000 7fc995e00700 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56099799c360 0x56099799e7b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ba7a674e8a4ace6 server_cookie=61995079aa811bfa in_seq=0 out_seq=0 2024-05-27T07:57:09.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.703+0000 7fc977e00700 10 client.17385 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:57:09.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.707+0000 7fc977e00700 1 -- 172.21.15.90:0/2318045540 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5609978a9d90 con 0x56099799c360 2024-05-27T07:57:09.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.707+0000 7fc977e00700 10 client.17385 handle_command_reply: tid=0 2024-05-27T07:57:09.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.707+0000 7fc9abdb1b80 1 client.17385 shutdown 2024-05-27T07:57:09.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.711+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2318045540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560997982690 msgr2=0x56099798caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:09.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.711+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560997982690 0x56099798caa0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc9900199b0 tx=0x7fc990004be0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.711+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2318045540 shutdown_connections 2024-05-27T07:57:09.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.711+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56099799c360 0x56099799e7b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.712+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560997982690 0x56099798caa0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.712+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x56099798c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.712+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/2318045540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609978e69f0 0x56099798c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.712+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2318045540 >> 172.21.15.90:0/2318045540 conn(0x560997894390 msgr2=0x560997981c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:09.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.712+0000 7fc977e00700 0 client.17385 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:57:09.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.712+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2318045540 shutdown_connections 2024-05-27T07:57:09.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.712+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/2318045540 wait complete. 2024-05-27T07:57:09.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.712+0000 7fc9abdb1b80 20 client.17385 trim_cache size 0 max 16384 2024-05-27T07:57:09.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.746+0000 7fc9abdb1b80 1 Processor -- start 2024-05-27T07:57:09.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.747+0000 7fc9abdb1b80 1 -- start start 2024-05-27T07:57:09.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.747+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x560997987880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.747+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609978e69f0 0x56099798af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.747+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x560997986150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.747+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981b00c0 con 0x5609978e69f0 2024-05-27T07:57:09.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.747+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981ad110 con 0x5609977fae40 2024-05-27T07:57:09.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.747+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981af320 con 0x560997982690 2024-05-27T07:57:09.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.748+0000 7fc995e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x560997987880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.748+0000 7fc995e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x560997987880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49632/0 (socket says 172.21.15.90:49632) 2024-05-27T07:57:09.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.748+0000 7fc995e00700 1 -- 172.21.15.90:0/1295467673 learned_addr learned my addr 172.21.15.90:0/1295467673 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:09.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.748+0000 7fc996800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x560997986150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.749+0000 7fc996800700 1 -- 172.21.15.90:0/1295467673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 msgr2=0x560997987880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:09.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.749+0000 7fc996800700 1 --2- 172.21.15.90:0/1295467673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x560997987880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.749+0000 7fc996800700 1 -- 172.21.15.90:0/1295467673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609978e69f0 msgr2=0x56099798af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:09.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.749+0000 7fc996800700 1 --2- 172.21.15.90:0/1295467673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609978e69f0 0x56099798af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.749+0000 7fc996800700 1 -- 172.21.15.90:0/1295467673 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56099798ce10 con 0x560997982690 2024-05-27T07:57:09.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.749+0000 7fc996800700 1 --2- 172.21.15.90:0/1295467673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x560997986150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc990030890 tx=0x7fc990030f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f252ef967cb9eff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:09.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.750+0000 7fc982a00700 1 -- 172.21.15.90:0/1295467673 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990030980 con 0x560997982690 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.750+0000 7fc982a00700 1 -- 172.21.15.90:0/1295467673 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc990030b20 con 0x560997982690 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.750+0000 7fc982a00700 1 -- 172.21.15.90:0/1295467673 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990016d80 con 0x560997982690 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.751+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1295467673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 msgr2=0x560997986150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.751+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/1295467673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x560997986150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc990030890 tx=0x7fc990030f10 comp rx=0 tx=0).stop 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.751+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1295467673 shutdown_connections 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.751+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/1295467673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x560997986150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.752+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/1295467673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x560997987880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.752+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/1295467673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609978e69f0 0x56099798af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.752+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1295467673 >> 172.21.15.90:0/1295467673 conn(0x560997894390 msgr2=0x56099798d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.752+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1295467673 shutdown_connections 2024-05-27T07:57:09.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.752+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1295467673 wait complete. 2024-05-27T07:57:09.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.753+0000 7fc9abdb1b80 1 -- start start 2024-05-27T07:57:09.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.753+0000 7fc9abdb1b80 1 Processor -- start 2024-05-27T07:57:09.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.754+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x56099798f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.754+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609978e69f0 0x56099798fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.754+0000 7fc9abdb1b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x560997990010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.754+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981aec50 con 0x5609978e69f0 2024-05-27T07:57:09.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.754+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981ad7e0 con 0x5609977fae40 2024-05-27T07:57:09.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.754+0000 7fc9abdb1b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9981adeb0 con 0x560997982690 2024-05-27T07:57:09.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.754+0000 7fc9abdb1b80 10 client.0 _ll_register_callbacks cb 0x560997841c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:09.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.755+0000 7fc995e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x56099798f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.755+0000 7fc995400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609978e69f0 0x56099798fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.755+0000 7fc995e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x56099798f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49640/0 (socket says 172.21.15.90:49640) 2024-05-27T07:57:09.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.755+0000 7fc995e00700 1 -- 172.21.15.90:0/1872477834 learned_addr learned my addr 172.21.15.90:0/1872477834 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:09.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.755+0000 7fc996800700 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x560997990010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.756+0000 7fc996800700 1 -- 172.21.15.90:0/1872477834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 msgr2=0x56099798f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:09.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.756+0000 7fc996800700 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x56099798f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.756+0000 7fc996800700 1 -- 172.21.15.90:0/1872477834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609978e69f0 msgr2=0x56099798fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:09.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.756+0000 7fc996800700 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609978e69f0 0x56099798fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.756+0000 7fc996800700 1 -- 172.21.15.90:0/1872477834 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560997a15f80 con 0x560997982690 2024-05-27T07:57:09.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.757+0000 7fc996800700 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x560997990010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc990000f80 tx=0x7fc990030590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2455f4577ce2105d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:09.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.757+0000 7fc977e00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:57:09.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.757+0000 7fc977e00700 1 -- 172.21.15.90:0/1872477834 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc990030cd0 con 0x560997982690 2024-05-27T07:57:09.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.757+0000 7fc977e00700 1 -- 172.21.15.90:0/1872477834 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9584+0+0 (secure 0 0 0) 0x7fc99001f040 con 0x560997982690 2024-05-27T07:57:09.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.758+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1872477834 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc990030b20 con 0x560997982690 2024-05-27T07:57:09.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.758+0000 7fc977e00700 1 -- 172.21.15.90:0/1872477834 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc99000a950 con 0x560997982690 2024-05-27T07:57:09.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.758+0000 7fc9abdb1b80 10 client.27034 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:09.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.758+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1872477834 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560997a171e0 con 0x560997982690 2024-05-27T07:57:09.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.759+0000 7fc977e00700 1 -- 172.21.15.90:0/1872477834 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc990030b20 con 0x560997982690 2024-05-27T07:57:09.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.759+0000 7fc977e00700 1 -- 172.21.15.90:0/1872477834 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fc99003b690 con 0x560997982690 2024-05-27T07:57:09.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.759+0000 7fc9abdb1b80 10 client.27034 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:09.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.759+0000 7fc9abdb1b80 10 client.27034 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:57:09.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.759+0000 7fc9abdb1b80 20 client.27034 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:09.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.759+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x560997a177d0 0x560997a19bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:09.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.759+0000 7fc9abdb1b80 4 client.27034 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:57:09.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.759+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1872477834 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609978a3570 con 0x560997a177d0 2024-05-27T07:57:09.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.760+0000 7fc995e00700 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x560997a177d0 0x560997a19bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:09.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.761+0000 7fc995e00700 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x560997a177d0 0x560997a19bf0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b819cf186c72309d server_cookie=cf6b926f810ac7fb in_seq=0 out_seq=0 2024-05-27T07:57:09.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.761+0000 7fc977e00700 10 client.27034 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:57:09.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.762+0000 7fc977e00700 1 -- 172.21.15.90:0/1872477834 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609978a3570 con 0x560997a177d0 2024-05-27T07:57:09.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.762+0000 7fc977e00700 10 client.27034 handle_command_reply: tid=0 2024-05-27T07:57:09.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.762+0000 7fc9abdb1b80 1 client.27034 shutdown 2024-05-27T07:57:09.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.762+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1872477834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 msgr2=0x560997990010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:09.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.762+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x560997990010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc990000f80 tx=0x7fc990030590 comp rx=0 tx=0).stop 2024-05-27T07:57:09.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.763+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1872477834 shutdown_connections 2024-05-27T07:57:09.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.763+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x560997a177d0 0x560997a19bf0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.763+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560997982690 0x560997990010 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.763+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609977fae40 0x56099798f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.763+0000 7fc977e00700 0 client.27034 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:57:09.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.763+0000 7fc9abdb1b80 1 --2- 172.21.15.90:0/1872477834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609978e69f0 0x56099798fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.763+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1872477834 >> 172.21.15.90:0/1872477834 conn(0x560997894390 msgr2=0x56099798c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:09.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.763+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1872477834 shutdown_connections 2024-05-27T07:57:09.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.763+0000 7fc9abdb1b80 1 -- 172.21.15.90:0/1872477834 wait complete. 2024-05-27T07:57:09.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.763+0000 7fc9abdb1b80 20 client.27034 trim_cache size 0 max 16384 2024-05-27T07:57:09.770 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:57:09.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.769+0000 7fc99d200700 1 -- 172.21.15.90:0/3093246104 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc97c0b2a70 msgr2=0x7fc97c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:09.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.769+0000 7fc99d200700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc97c0b2a70 0x7fc97c0b4f80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5609979836e0 tx=0x7fc990009650 comp rx=0 tx=0).stop 2024-05-27T07:57:09.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.769+0000 7fc99d200700 1 -- 172.21.15.90:0/3093246104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105e80 msgr2=0x7fc9981992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:09.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.769+0000 7fc99d200700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105e80 0x7fc9981992e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc98400b4d0 tx=0x7fc98400b7e0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.770+0000 7fc99d200700 1 -- 172.21.15.90:0/3093246104 shutdown_connections 2024-05-27T07:57:09.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.770+0000 7fc99d200700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc97c0b2a70 0x7fc97c0b4f80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.770+0000 7fc99d200700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99810e910 0x7fc99819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.770+0000 7fc99d200700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc998105530 0x7fc998198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.770+0000 7fc99d200700 1 --2- 172.21.15.90:0/3093246104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc998105e80 0x7fc9981992e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:09.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.770+0000 7fc99d200700 1 -- 172.21.15.90:0/3093246104 >> 172.21.15.90:0/3093246104 conn(0x7fc9980fef40 msgr2=0x7fc9980ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:09.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.770+0000 7fc99d200700 1 -- 172.21.15.90:0/3093246104 shutdown_connections 2024-05-27T07:57:09.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:09.771+0000 7fc99d200700 1 -- 172.21.15.90:0/3093246104 wait complete. 2024-05-27T07:57:10.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:10 smithi090 ceph-mon[31429]: pgmap v341: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 524 op/s 2024-05-27T07:57:10.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:10 smithi136 ceph-mon[35324]: pgmap v341: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 524 op/s 2024-05-27T07:57:10.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:10 smithi129 ceph-mon[35254]: pgmap v341: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 MiB/s wr, 524 op/s 2024-05-27T07:57:11.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:11 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:57:11.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:11 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:57:11.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:11 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:57:11.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:11 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:57:11.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:11 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:57:11.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:11 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:57:11.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:11 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:57:11.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:11 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:57:11.803 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:11 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T07:57:11.803 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:11 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T07:57:11.803 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:11 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T07:57:11.803 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:11 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:57:12.496 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:57:13.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:13 smithi090 ceph-mon[31429]: osdmap e110: 12 total, 12 up, 12 in 2024-05-27T07:57:13.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:13 smithi136 ceph-mon[35324]: osdmap e110: 12 total, 12 up, 12 in 2024-05-27T07:57:13.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:13 smithi129 ceph-mon[35254]: osdmap e110: 12 total, 12 up, 12 in 2024-05-27T07:57:13.582 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:13.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 415.0 s, 496.0 tps, lat 198.241 ms stddev 199.760 2024-05-27T07:57:14.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:14 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.753 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:14 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:14.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:14 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:15.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:15 smithi090 ceph-mon[31429]: pgmap v342: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 489 op/s 2024-05-27T07:57:15.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:15 smithi090 ceph-mon[31429]: pgmap v344: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 410 op/s 2024-05-27T07:57:15.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:15 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi129 to 176.4M 2024-05-27T07:57:15.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:15 smithi090 ceph-mon[31429]: Unable to set osd_memory_target on smithi129 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-05-27T07:57:15.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:15 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi136 to 176.4M 2024-05-27T07:57:15.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:15 smithi090 ceph-mon[31429]: Unable to set osd_memory_target on smithi136 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-27T07:57:15.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:15 smithi136 ceph-mon[35324]: pgmap v342: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 489 op/s 2024-05-27T07:57:15.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:15 smithi136 ceph-mon[35324]: pgmap v344: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 410 op/s 2024-05-27T07:57:15.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:15 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi129 to 176.4M 2024-05-27T07:57:15.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:15 smithi136 ceph-mon[35324]: Unable to set osd_memory_target on smithi129 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-05-27T07:57:15.755 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:15 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi136 to 176.4M 2024-05-27T07:57:15.756 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:15 smithi136 ceph-mon[35324]: Unable to set osd_memory_target on smithi136 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-27T07:57:15.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:15 smithi129 ceph-mon[35254]: pgmap v342: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.3 MiB/s wr, 489 op/s 2024-05-27T07:57:15.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:15 smithi129 ceph-mon[35254]: pgmap v344: 129 pgs: 129 active+clean; 2.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.7 MiB/s wr, 410 op/s 2024-05-27T07:57:15.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:15 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi129 to 176.4M 2024-05-27T07:57:15.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:15 smithi129 ceph-mon[35254]: Unable to set osd_memory_target on smithi129 to 185061273: error parsing value: Value '185061273' is below minimum 939524096 2024-05-27T07:57:15.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:15 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi136 to 176.4M 2024-05-27T07:57:15.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:15 smithi129 ceph-mon[35254]: Unable to set osd_memory_target on smithi136 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-05-27T07:57:16.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:16 smithi090 ceph-mon[31429]: pgmap v345: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 497 op/s 2024-05-27T07:57:16.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:16 smithi136 ceph-mon[35324]: pgmap v345: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 497 op/s 2024-05-27T07:57:16.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:16 smithi129 ceph-mon[35254]: pgmap v345: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.4 MiB/s wr, 497 op/s 2024-05-27T07:57:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.138+0000 7f20ba800640 1 -- 172.21.15.90:0/3597888952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b4106600 msgr2=0x7f20b41129d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.138+0000 7f20ba800640 1 --2- 172.21.15.90:0/3597888952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b4106600 0x7f20b41129d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009ca0 tx=0x7f20ac02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.138+0000 7f20ba800640 1 -- 172.21.15.90:0/3597888952 shutdown_connections 2024-05-27T07:57:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.138+0000 7f20ba800640 1 --2- 172.21.15.90:0/3597888952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b4106600 0x7f20b41129d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.138+0000 7f20ba800640 1 --2- 172.21.15.90:0/3597888952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b410c350 0x7f20b4105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.138+0000 7f20ba800640 1 --2- 172.21.15.90:0/3597888952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20b410beb0 0x7f20b4105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.138+0000 7f20ba800640 1 -- 172.21.15.90:0/3597888952 >> 172.21.15.90:0/3597888952 conn(0x7f20b41016b0 msgr2=0x7f20b4103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.138+0000 7f20ba800640 1 -- 172.21.15.90:0/3597888952 shutdown_connections 2024-05-27T07:57:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.138+0000 7f20ba800640 1 -- 172.21.15.90:0/3597888952 wait complete. 2024-05-27T07:57:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.140+0000 7f20ba800640 1 Processor -- start 2024-05-27T07:57:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.140+0000 7f20ba800640 1 -- start start 2024-05-27T07:57:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.141+0000 7f20ba800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b4106600 0x7f20b40748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.141+0000 7f20ba800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b410beb0 0x7f20b406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.141+0000 7f20ba800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20b410c350 0x7f20b406e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.141+0000 7f20ba800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4115fc0 con 0x7f20b4106600 2024-05-27T07:57:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.141+0000 7f20ba800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4115e40 con 0x7f20b410c350 2024-05-27T07:57:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.141+0000 7f20ba800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4116140 con 0x7f20b410beb0 2024-05-27T07:57:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.141+0000 7f20b2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b410beb0 0x7f20b406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.141+0000 7f20b3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b4106600 0x7f20b40748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.141+0000 7f20b2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b410beb0 0x7f20b406da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53514/0 (socket says 172.21.15.90:53514) 2024-05-27T07:57:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.141+0000 7f20b2a00640 1 -- 172.21.15.90:0/1044217995 learned_addr learned my addr 172.21.15.90:0/1044217995 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.142+0000 7f20b3e00640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20b410c350 0x7f20b406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.143+0000 7f20b2a00640 1 -- 172.21.15.90:0/1044217995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20b410c350 msgr2=0x7f20b406e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.143+0000 7f20b2a00640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20b410c350 0x7f20b406e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.143+0000 7f20b2a00640 1 -- 172.21.15.90:0/1044217995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b4106600 msgr2=0x7f20b40748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.143+0000 7f20b2a00640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b4106600 0x7f20b40748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.143+0000 7f20b2a00640 1 -- 172.21.15.90:0/1044217995 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b406e7f0 con 0x7f20b410beb0 2024-05-27T07:57:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.144+0000 7f20b2a00640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b410beb0 0x7f20b406da10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f20a000c970 tx=0x7f20a000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.144+0000 7f20b3400640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b4106600 0x7f20b40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.144+0000 7f20abe00640 1 -- 172.21.15.90:0/1044217995 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0007c70 con 0x7f20b410beb0 2024-05-27T07:57:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.145+0000 7f20abe00640 1 -- 172.21.15.90:0/1044217995 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20a0007e10 con 0x7f20b410beb0 2024-05-27T07:57:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.145+0000 7f20abe00640 1 -- 172.21.15.90:0/1044217995 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0018780 con 0x7f20b410beb0 2024-05-27T07:57:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.145+0000 7f20ba800640 1 -- 172.21.15.90:0/1044217995 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b40724d0 con 0x7f20b410beb0 2024-05-27T07:57:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.145+0000 7f20ba800640 1 -- 172.21.15.90:0/1044217995 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b40729d0 con 0x7f20b410beb0 2024-05-27T07:57:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.152+0000 7f20abe00640 1 -- 172.21.15.90:0/1044217995 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f20a000fb20 con 0x7f20b410beb0 2024-05-27T07:57:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.154+0000 7f20c182c740 1 Processor -- start 2024-05-27T07:57:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.155+0000 7f20c182c740 1 -- start start 2024-05-27T07:57:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.155+0000 7f20c182c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5c21b0 0x55eb4b964860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.155+0000 7f20c182c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5992f0 0x55eb4b964da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.155+0000 7f20c182c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b9652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.155+0000 7f20c182c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eb4b8d5b90 con 0x55eb4b5992f0 2024-05-27T07:57:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.155+0000 7f20c182c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55eb4b8c5730 con 0x55eb4b5a14d0 2024-05-27T07:57:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.155+0000 7f20c182c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eb4b8c9770 con 0x55eb4b5c21b0 2024-05-27T07:57:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.155+0000 7f20abe00640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f20980b29d0 0x7f20980b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.156+0000 7f20b3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5c21b0 0x55eb4b964860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.156+0000 7f20b3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5c21b0 0x55eb4b964860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53518/0 (socket says 172.21.15.90:53518) 2024-05-27T07:57:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.156+0000 7f20b3e00640 1 -- 172.21.15.90:0/4203509592 learned_addr learned my addr 172.21.15.90:0/4203509592 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.156+0000 7f20b3e00640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f20980b29d0 0x7f20980b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.156+0000 7f20abe00640 1 -- 172.21.15.90:0/1044217995 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f20a00e2a60 con 0x7f20b410beb0 2024-05-27T07:57:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.156+0000 7f20b3400640 1 --2- 172.21.15.90:0/4203509592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5992f0 0x55eb4b964da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.157+0000 7f20b2a00640 1 --2- 172.21.15.90:0/4203509592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b9652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.157+0000 7f20b3e00640 1 -- 172.21.15.90:0/4203509592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 msgr2=0x55eb4b9652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.157+0000 7f20b3e00640 1 --2- 172.21.15.90:0/4203509592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b9652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.157+0000 7f20b3e00640 1 -- 172.21.15.90:0/4203509592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5992f0 msgr2=0x55eb4b964da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.157+0000 7f20b3e00640 1 --2- 172.21.15.90:0/4203509592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5992f0 0x55eb4b964da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.157+0000 7f20b3e00640 1 -- 172.21.15.90:0/4203509592 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb4b85e8b0 con 0x55eb4b5c21b0 2024-05-27T07:57:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.157+0000 7f20b3e00640 1 --2- 172.21.15.90:0/4203509592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5c21b0 0x55eb4b964860 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009ca0 tx=0x7f20ac003a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ece33882c378c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.158+0000 7f20b3e00640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f20980b29d0 0x7f20980b4ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f20ac035000 tx=0x7f20ac04c000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.158+0000 7f20a8c00640 1 -- 172.21.15.90:0/4203509592 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac003c60 con 0x55eb4b5c21b0 2024-05-27T07:57:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.158+0000 7f20a8c00640 1 -- 172.21.15.90:0/4203509592 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20ac036040 con 0x55eb4b5c21b0 2024-05-27T07:57:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.158+0000 7f20a8c00640 1 -- 172.21.15.90:0/4203509592 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac030820 con 0x55eb4b5c21b0 2024-05-27T07:57:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.159+0000 7f20c182c740 1 -- 172.21.15.90:0/4203509592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5c21b0 msgr2=0x55eb4b964860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.159+0000 7f20c182c740 1 --2- 172.21.15.90:0/4203509592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5c21b0 0x55eb4b964860 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009ca0 tx=0x7f20ac003a70 comp rx=0 tx=0).stop 2024-05-27T07:57:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.160+0000 7f20c182c740 1 -- 172.21.15.90:0/4203509592 shutdown_connections 2024-05-27T07:57:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.160+0000 7f20c182c740 1 --2- 172.21.15.90:0/4203509592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b9652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.160+0000 7f20c182c740 1 --2- 172.21.15.90:0/4203509592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5992f0 0x55eb4b964da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.160+0000 7f20c182c740 1 --2- 172.21.15.90:0/4203509592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5c21b0 0x55eb4b964860 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.160+0000 7f20c182c740 1 -- 172.21.15.90:0/4203509592 >> 172.21.15.90:0/4203509592 conn(0x55eb4b5b6300 msgr2=0x55eb4b961010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.161+0000 7f20c182c740 1 -- 172.21.15.90:0/4203509592 shutdown_connections 2024-05-27T07:57:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.162+0000 7f20c182c740 1 -- 172.21.15.90:0/4203509592 wait complete. 2024-05-27T07:57:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.162+0000 7f20c182c740 1 -- start start 2024-05-27T07:57:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.163+0000 7f20c182c740 1 Processor -- start 2024-05-27T07:57:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.164+0000 7f20c182c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5992f0 0x55eb4b96ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.165+0000 7f20c182c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5a14d0 0x55eb4b96ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.165+0000 7f20c182c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5c21b0 0x55eb4b97ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.165+0000 7f20c182c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eb4b5ab790 con 0x55eb4b5992f0 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.165+0000 7f20c182c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55eb4b86dbc0 con 0x55eb4b5c21b0 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.165+0000 7f20c182c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55eb4b9717b0 con 0x55eb4b5a14d0 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.165+0000 7f20c182c740 10 client.0 _ll_register_callbacks cb 0x55eb4b84b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.165+0000 7f20b3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5992f0 0x55eb4b96ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.165+0000 7f20b2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5a14d0 0x55eb4b96ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.166+0000 7f20b3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5c21b0 0x55eb4b97ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.166+0000 7f20b3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5c21b0 0x55eb4b97ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49694/0 (socket says 172.21.15.90:49694) 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.166+0000 7f20b3e00640 1 -- 172.21.15.90:0/2746616202 learned_addr learned my addr 172.21.15.90:0/2746616202 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.167+0000 7f20b3e00640 1 -- 172.21.15.90:0/2746616202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5a14d0 msgr2=0x55eb4b96ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.167+0000 7f20b3e00640 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5a14d0 0x55eb4b96ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.167+0000 7f20b3e00640 1 -- 172.21.15.90:0/2746616202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5992f0 msgr2=0x55eb4b96ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.167+0000 7f20b3e00640 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5992f0 0x55eb4b96ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.167+0000 7f20b3e00640 1 -- 172.21.15.90:0/2746616202 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb4b5da210 con 0x55eb4b5c21b0 2024-05-27T07:57:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.167+0000 7f20b3e00640 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5c21b0 0x55eb4b97ab10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20ac02b0b0 tx=0x7f20ac02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3fd247e70dd0410 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.167+0000 7f209d600640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:57:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.168+0000 7f209d600640 1 -- 172.21.15.90:0/2746616202 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac003c60 con 0x55eb4b5c21b0 2024-05-27T07:57:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.168+0000 7f209d600640 1 -- 172.21.15.90:0/2746616202 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f20ac003e00 con 0x55eb4b5c21b0 2024-05-27T07:57:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.169+0000 7f20c182c740 1 -- 172.21.15.90:0/2746616202 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20ac036040 con 0x55eb4b5c21b0 2024-05-27T07:57:17.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.170+0000 7f209d600640 1 -- 172.21.15.90:0/2746616202 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f20ac040b20 con 0x55eb4b5c21b0 2024-05-27T07:57:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.171+0000 7f20c182c740 10 client.36980 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.171+0000 7f20c182c740 1 -- 172.21.15.90:0/2746616202 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55eb4b6a6620 con 0x55eb4b5c21b0 2024-05-27T07:57:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.171+0000 7f209d600640 1 -- 172.21.15.90:0/2746616202 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20ac036040 con 0x55eb4b5c21b0 2024-05-27T07:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.172+0000 7f209d600640 1 -- 172.21.15.90:0/2746616202 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f20ac057020 con 0x55eb4b5c21b0 2024-05-27T07:57:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.172+0000 7f20c182c740 10 client.36980 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.173+0000 7f20c182c740 10 client.36980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.173+0000 7f20c182c740 20 client.36980 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.173+0000 7f20c182c740 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eb4b580d80 0x55eb4b9808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.173+0000 7f20c182c740 4 client.36980 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.173+0000 7f20c182c740 1 -- 172.21.15.90:0/2746616202 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb4b8d7ae0 con 0x55eb4b580d80 2024-05-27T07:57:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.174+0000 7f20b3400640 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eb4b580d80 0x55eb4b9808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.176+0000 7f20b3400640 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eb4b580d80 0x55eb4b9808c0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=443fb93bd6364a4d server_cookie=550befbab1215ae4 in_seq=0 out_seq=0 2024-05-27T07:57:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.176+0000 7f209d600640 10 client.36980 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.179+0000 7f209d600640 1 -- 172.21.15.90:0/2746616202 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55eb4b8d7ae0 con 0x55eb4b580d80 2024-05-27T07:57:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.179+0000 7f209d600640 10 client.36980 handle_command_reply: tid=0 2024-05-27T07:57:17.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.179+0000 7f20c182c740 1 client.36980 shutdown 2024-05-27T07:57:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.180+0000 7f20c182c740 1 -- 172.21.15.90:0/2746616202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5c21b0 msgr2=0x55eb4b97ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.180+0000 7f20c182c740 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5c21b0 0x55eb4b97ab10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20ac02b0b0 tx=0x7f20ac02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.180+0000 7f20c182c740 1 -- 172.21.15.90:0/2746616202 shutdown_connections 2024-05-27T07:57:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.180+0000 7f20c182c740 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eb4b580d80 0x55eb4b9808c0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.180+0000 7f20c182c740 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5c21b0 0x55eb4b97ab10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.180+0000 7f20c182c740 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5a14d0 0x55eb4b96ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.180+0000 7f20c182c740 1 --2- 172.21.15.90:0/2746616202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5992f0 0x55eb4b96ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.180+0000 7f20c182c740 1 -- 172.21.15.90:0/2746616202 >> 172.21.15.90:0/2746616202 conn(0x55eb4b5b6300 msgr2=0x55eb4b964690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.181+0000 7f209d600640 0 client.36980 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.182+0000 7f20c182c740 1 -- 172.21.15.90:0/2746616202 shutdown_connections 2024-05-27T07:57:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.182+0000 7f20c182c740 1 -- 172.21.15.90:0/2746616202 wait complete. 2024-05-27T07:57:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.182+0000 7f20c182c740 20 client.36980 trim_cache size 0 max 16384 2024-05-27T07:57:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.241+0000 7f20c182c740 1 Processor -- start 2024-05-27T07:57:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.241+0000 7f20c182c740 1 -- start start 2024-05-27T07:57:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.242+0000 7f20c182c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 0x55eb4b96a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.242+0000 7f20c182c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b960330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.242+0000 7f20c182c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b96e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.242+0000 7f20c182c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b92d0 con 0x55eb4b5c21b0 2024-05-27T07:57:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.242+0000 7f20c182c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b6320 con 0x55eb4b5a14d0 2024-05-27T07:57:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.242+0000 7f20c182c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b8530 con 0x55eb4b5992f0 2024-05-27T07:57:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.243+0000 7f20b3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b96e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.243+0000 7f20b3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b96e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35480/0 (socket says 172.21.15.90:35480) 2024-05-27T07:57:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.243+0000 7f20b3e00640 1 -- 172.21.15.90:0/3655565313 learned_addr learned my addr 172.21.15.90:0/3655565313 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.243+0000 7f20b2a00640 1 --2- 172.21.15.90:0/3655565313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b960330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.244+0000 7f20b3400640 1 --2- 172.21.15.90:0/3655565313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 0x55eb4b96a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.245+0000 7f20b3e00640 1 -- 172.21.15.90:0/3655565313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 msgr2=0x55eb4b96a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.245+0000 7f20b3e00640 1 --2- 172.21.15.90:0/3655565313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 0x55eb4b96a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.245+0000 7f20b3e00640 1 -- 172.21.15.90:0/3655565313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 msgr2=0x55eb4b960330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.245+0000 7f20b3e00640 1 --2- 172.21.15.90:0/3655565313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b960330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.245+0000 7f20b3e00640 1 -- 172.21.15.90:0/3655565313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb4b965c40 con 0x55eb4b5c21b0 2024-05-27T07:57:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.248+0000 7f20b3e00640 1 --2- 172.21.15.90:0/3655565313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b96e9f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f20ac004410 tx=0x7f20ac003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be084a912c98428 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.248+0000 7f209fe00640 1 -- 172.21.15.90:0/3655565313 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac02f3f0 con 0x55eb4b5c21b0 2024-05-27T07:57:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.249+0000 7f209fe00640 1 -- 172.21.15.90:0/3655565313 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20ac036040 con 0x55eb4b5c21b0 2024-05-27T07:57:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.249+0000 7f209fe00640 1 -- 172.21.15.90:0/3655565313 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac02f3f0 con 0x55eb4b5c21b0 2024-05-27T07:57:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.250+0000 7f20c182c740 1 -- 172.21.15.90:0/3655565313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 msgr2=0x55eb4b96e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.250+0000 7f20c182c740 1 --2- 172.21.15.90:0/3655565313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b96e9f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f20ac004410 tx=0x7f20ac003af0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.251+0000 7f20c182c740 1 -- 172.21.15.90:0/3655565313 shutdown_connections 2024-05-27T07:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.251+0000 7f20c182c740 1 --2- 172.21.15.90:0/3655565313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b960330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.251+0000 7f20c182c740 1 --2- 172.21.15.90:0/3655565313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b96e9f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.251+0000 7f20c182c740 1 --2- 172.21.15.90:0/3655565313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 0x55eb4b96a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.251+0000 7f20c182c740 1 -- 172.21.15.90:0/3655565313 >> 172.21.15.90:0/3655565313 conn(0x55eb4b5b6300 msgr2=0x55eb4b96e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.251+0000 7f20c182c740 1 -- 172.21.15.90:0/3655565313 shutdown_connections 2024-05-27T07:57:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.251+0000 7f20c182c740 1 -- 172.21.15.90:0/3655565313 wait complete. 2024-05-27T07:57:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.252+0000 7f20c182c740 1 -- start start 2024-05-27T07:57:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.252+0000 7f20c182c740 1 Processor -- start 2024-05-27T07:57:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.253+0000 7f20c182c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 0x55eb4b971a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.253+0000 7f20c182c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b971f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.253+0000 7f20c182c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b9724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.254+0000 7f20b3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 0x55eb4b971a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.254+0000 7f20b3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 0x55eb4b971a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53552/0 (socket says 172.21.15.90:53552) 2024-05-27T07:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.254+0000 7f20b2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b971f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.254+0000 7f20b2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b971f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49716/0 (socket says 172.21.15.90:49716) 2024-05-27T07:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.255+0000 7f20c182c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b7e60 con 0x55eb4b5c21b0 2024-05-27T07:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.255+0000 7f20c182c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b69f0 con 0x55eb4b5a14d0 2024-05-27T07:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.255+0000 7f20c182c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41b70c0 con 0x55eb4b5992f0 2024-05-27T07:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.255+0000 7f20c182c740 10 client.0 _ll_register_callbacks cb 0x7f20a400b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.255+0000 7f20b3400640 1 -- 172.21.15.90:0/3341222516 learned_addr learned my addr 172.21.15.90:0/3341222516 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.255+0000 7f20b3e00640 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b9724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.256+0000 7f20b3e00640 1 -- 172.21.15.90:0/3341222516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 msgr2=0x55eb4b971a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.256+0000 7f20b3e00640 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 0x55eb4b971a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.256+0000 7f20b3e00640 1 -- 172.21.15.90:0/3341222516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 msgr2=0x55eb4b971f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.256+0000 7f20b3e00640 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b971f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.256+0000 7f20b3e00640 1 -- 172.21.15.90:0/3341222516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb4b9e6db0 con 0x55eb4b5c21b0 2024-05-27T07:57:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.256+0000 7f20b3400640 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 0x55eb4b971a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.257+0000 7f20b3e00640 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b9724b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f20ac005e30 tx=0x7f20ac02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9348f6e3e9706c87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.257+0000 7f209d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.258+0000 7f209d600640 1 -- 172.21.15.90:0/3341222516 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac004860 con 0x55eb4b5c21b0 2024-05-27T07:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.258+0000 7f209d600640 1 -- 172.21.15.90:0/3341222516 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f20ac040cc0 con 0x55eb4b5c21b0 2024-05-27T07:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.258+0000 7f20c182c740 1 -- 172.21.15.90:0/3341222516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20ac036040 con 0x55eb4b5c21b0 2024-05-27T07:57:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.259+0000 7f209d600640 1 -- 172.21.15.90:0/3341222516 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f20ac07b050 con 0x55eb4b5c21b0 2024-05-27T07:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.259+0000 7f20c182c740 10 client.17403 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.259+0000 7f20c182c740 1 -- 172.21.15.90:0/3341222516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55eb4b9e7f60 con 0x55eb4b5c21b0 2024-05-27T07:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.259+0000 7f209d600640 1 -- 172.21.15.90:0/3341222516 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20ac036040 con 0x55eb4b5c21b0 2024-05-27T07:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.260+0000 7f209d600640 1 -- 172.21.15.90:0/3341222516 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f20ac057040 con 0x55eb4b5c21b0 2024-05-27T07:57:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.260+0000 7f20c182c740 10 client.17403 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.260+0000 7f20c182c740 10 client.17403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.260+0000 7f20c182c740 20 client.17403 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.260+0000 7f20c182c740 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eb4b5561c0 0x55eb4b9ea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.260+0000 7f20c182c740 4 client.17403 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.260+0000 7f20c182c740 1 -- 172.21.15.90:0/3341222516 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb4b969060 con 0x55eb4b5561c0 2024-05-27T07:57:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.260+0000 7f20b3400640 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eb4b5561c0 0x55eb4b9ea5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.263+0000 7f209d600640 10 client.17403 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.263+0000 7f20b3400640 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eb4b5561c0 0x55eb4b9ea5f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e9af2faa34fbaa server_cookie=1c8f30eb876d0585 in_seq=0 out_seq=0 2024-05-27T07:57:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.297+0000 7f209d600640 1 -- 172.21.15.90:0/3341222516 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55eb4b969060 con 0x55eb4b5561c0 2024-05-27T07:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.297+0000 7f209d600640 10 client.17403 handle_command_reply: tid=0 2024-05-27T07:57:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.297+0000 7f20c182c740 1 client.17403 shutdown 2024-05-27T07:57:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.298+0000 7f20c182c740 1 -- 172.21.15.90:0/3341222516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 msgr2=0x55eb4b9724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.298+0000 7f20c182c740 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b9724b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f20ac005e30 tx=0x7f20ac02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.298+0000 7f20c182c740 1 -- 172.21.15.90:0/3341222516 shutdown_connections 2024-05-27T07:57:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.298+0000 7f20c182c740 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eb4b5561c0 0x55eb4b9ea5f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.298+0000 7f209d600640 0 client.17403 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.298+0000 7f20c182c740 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55eb4b5a14d0 0x55eb4b971f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.298+0000 7f20c182c740 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eb4b5c21b0 0x55eb4b9724b0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.298+0000 7f20c182c740 1 --2- 172.21.15.90:0/3341222516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eb4b5992f0 0x55eb4b971a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.298+0000 7f20c182c740 1 -- 172.21.15.90:0/3341222516 >> 172.21.15.90:0/3341222516 conn(0x55eb4b5b6300 msgr2=0x55eb4b96ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.302+0000 7f20c182c740 1 -- 172.21.15.90:0/3341222516 shutdown_connections 2024-05-27T07:57:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.302+0000 7f20c182c740 1 -- 172.21.15.90:0/3341222516 wait complete. 2024-05-27T07:57:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.303+0000 7f20c182c740 20 client.17403 trim_cache size 0 max 16384 2024-05-27T07:57:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.304+0000 7f20ba800640 1 -- 172.21.15.90:0/1044217995 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f20980b29d0 msgr2=0x7f20980b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.304+0000 7f20ba800640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f20980b29d0 0x7f20980b4ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f20ac035000 tx=0x7f20ac04c000 comp rx=0 tx=0).stop 2024-05-27T07:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.304+0000 7f20ba800640 1 -- 172.21.15.90:0/1044217995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b410beb0 msgr2=0x7f20b406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.304+0000 7f20ba800640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b410beb0 0x7f20b406da10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f20a000c970 tx=0x7f20a000ce30 comp rx=0 tx=0).stop 2024-05-27T07:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.305+0000 7f20ba800640 1 -- 172.21.15.90:0/1044217995 shutdown_connections 2024-05-27T07:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.305+0000 7f20ba800640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f20980b29d0 0x7f20980b4ee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.305+0000 7f20ba800640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f20b410c350 0x7f20b406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.305+0000 7f20ba800640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b410beb0 0x7f20b406da10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.305+0000 7f20ba800640 1 --2- 172.21.15.90:0/1044217995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20b4106600 0x7f20b40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.305+0000 7f20ba800640 1 -- 172.21.15.90:0/1044217995 >> 172.21.15.90:0/1044217995 conn(0x7f20b41016b0 msgr2=0x7f20b41087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.305+0000 7f20ba800640 1 -- 172.21.15.90:0/1044217995 shutdown_connections 2024-05-27T07:57:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:17.305+0000 7f20ba800640 1 -- 172.21.15.90:0/1044217995 wait complete. 2024-05-27T07:57:18.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:18 smithi136 ceph-mon[35324]: pgmap v346: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.6 MiB/s wr, 502 op/s 2024-05-27T07:57:18.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:18 smithi129 ceph-mon[35254]: pgmap v346: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.6 MiB/s wr, 502 op/s 2024-05-27T07:57:18.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 420.0 s, 541.8 tps, lat 186.404 ms stddev 219.830 2024-05-27T07:57:18.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:18 smithi090 ceph-mon[31429]: pgmap v346: 129 pgs: 129 active+clean; 2.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.6 MiB/s wr, 502 op/s 2024-05-27T07:57:19.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.810+0000 7f6198600700 1 -- 172.21.15.90:0/422263312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6190110c60 msgr2=0x7f6190113080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.810+0000 7f6198600700 1 --2- 172.21.15.90:0/422263312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6190110c60 0x7f6190113080 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f617c009990 tx=0x7f617c009ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.810+0000 7f6198600700 1 -- 172.21.15.90:0/422263312 shutdown_connections 2024-05-27T07:57:19.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.810+0000 7f6198600700 1 --2- 172.21.15.90:0/422263312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6190110c60 0x7f6190113080 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.810+0000 7f6198600700 1 --2- 172.21.15.90:0/422263312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61901030b0 0x7f61901034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.810+0000 7f6198600700 1 --2- 172.21.15.90:0/422263312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6190103a00 0x7f6190110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.810+0000 7f6198600700 1 -- 172.21.15.90:0/422263312 >> 172.21.15.90:0/422263312 conn(0x7f61900fedb0 msgr2=0x7f61901011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.812+0000 7f6198600700 1 -- 172.21.15.90:0/422263312 shutdown_connections 2024-05-27T07:57:19.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.812+0000 7f6198600700 1 -- 172.21.15.90:0/422263312 wait complete. 2024-05-27T07:57:19.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.813+0000 7f6198600700 1 Processor -- start 2024-05-27T07:57:19.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.814+0000 7f6198600700 1 -- start start 2024-05-27T07:57:19.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.814+0000 7f6198600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61901030b0 0x7f619010f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.814+0000 7f6198600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6190103a00 0x7f619010f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.814+0000 7f6198600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6190110c60 0x7f61901acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.814+0000 7f6198600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6190105e00 con 0x7f6190110c60 2024-05-27T07:57:19.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.814+0000 7f6198600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6190105c80 con 0x7f6190103a00 2024-05-27T07:57:19.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.814+0000 7f6198600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6190105f80 con 0x7f61901030b0 2024-05-27T07:57:19.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.815+0000 7f6195200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6190103a00 0x7f619010f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.815+0000 7f6195200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6190103a00 0x7f619010f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:59966/0 (socket says 172.21.15.90:59966) 2024-05-27T07:57:19.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.815+0000 7f6195200700 1 -- 172.21.15.90:0/293937319 learned_addr learned my addr 172.21.15.90:0/293937319 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:19.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.815+0000 7f6195c00700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61901030b0 0x7f619010f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.816+0000 7f6195200700 1 -- 172.21.15.90:0/293937319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61901030b0 msgr2=0x7f619010f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.816+0000 7f6196600700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6190110c60 0x7f61901acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.816+0000 7f6195200700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61901030b0 0x7f619010f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.817+0000 7f6195200700 1 -- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6190110c60 msgr2=0x7f61901acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.817+0000 7f6195200700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6190110c60 0x7f61901acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.817+0000 7f6195200700 1 -- 172.21.15.90:0/293937319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61901ad500 con 0x7f6190103a00 2024-05-27T07:57:19.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.817+0000 7f6195c00700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61901030b0 0x7f619010f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:19.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.818+0000 7f6195200700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6190103a00 0x7f619010f580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f618400e970 tx=0x7f618400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.818+0000 7f6196600700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6190110c60 0x7f61901acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:19.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.820+0000 7f618e000700 1 -- 172.21.15.90:0/293937319 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618400cbd0 con 0x7f6190103a00 2024-05-27T07:57:19.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.820+0000 7f618e000700 1 -- 172.21.15.90:0/293937319 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f618400cd70 con 0x7f6190103a00 2024-05-27T07:57:19.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.820+0000 7f6198600700 1 -- 172.21.15.90:0/293937319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61901ad7f0 con 0x7f6190103a00 2024-05-27T07:57:19.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.820+0000 7f618e000700 1 -- 172.21.15.90:0/293937319 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184005610 con 0x7f6190103a00 2024-05-27T07:57:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.820+0000 7f6198600700 1 -- 172.21.15.90:0/293937319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61901add30 con 0x7f6190103a00 2024-05-27T07:57:19.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.826+0000 7f61a7114b80 1 Processor -- start 2024-05-27T07:57:19.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.826+0000 7f61a7114b80 1 -- start start 2024-05-27T07:57:19.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.827+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 0x558ce17c1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.828+0000 7f6196600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 0x558ce17c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.828+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.828+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.828+0000 7f61a7114b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558ce16efe40 con 0x558ce1642e40 2024-05-27T07:57:19.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.828+0000 7f61a7114b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558ce16df9e0 con 0x558ce17c1e70 2024-05-27T07:57:19.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.828+0000 7f61a7114b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ce16e3a20 con 0x558ce172e9f0 2024-05-27T07:57:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.828+0000 7f6196600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 0x558ce17c1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57474/0 (socket says 172.21.15.90:57474) 2024-05-27T07:57:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.829+0000 7f6196600700 1 -- 172.21.15.90:0/1099183066 learned_addr learned my addr 172.21.15.90:0/1099183066 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.829+0000 7f6196600700 1 -- 172.21.15.90:0/1099183066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 msgr2=0x558ce17ce110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:57:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.829+0000 7f6196600700 1 --2- 172.21.15.90:0/1099183066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17ce110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.829+0000 7f6196600700 1 -- 172.21.15.90:0/1099183066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 msgr2=0x558ce17c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.829+0000 7f6196600700 1 --2- 172.21.15.90:0/1099183066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.829+0000 7f6196600700 1 -- 172.21.15.90:0/1099183066 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce16a8420 con 0x558ce172e9f0 2024-05-27T07:57:19.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.830+0000 7f6196600700 1 --2- 172.21.15.90:0/1099183066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 0x558ce17c1330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f617c00b2e0 tx=0x7f617c009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51529464c401ae1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.833+0000 7f618e000700 1 -- 172.21.15.90:0/293937319 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6184004750 con 0x7f6190103a00 2024-05-27T07:57:19.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.838+0000 7f6182a00700 1 -- 172.21.15.90:0/1099183066 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c009580 con 0x558ce172e9f0 2024-05-27T07:57:19.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.838+0000 7f6182a00700 1 -- 172.21.15.90:0/1099183066 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f617c004830 con 0x558ce172e9f0 2024-05-27T07:57:19.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.838+0000 7f6182a00700 1 -- 172.21.15.90:0/1099183066 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c0053a0 con 0x558ce172e9f0 2024-05-27T07:57:19.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.838+0000 7f618e000700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61780b2b40 0x7f61780b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.839+0000 7f61a7114b80 1 -- 172.21.15.90:0/1099183066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 msgr2=0x558ce17c1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.839+0000 7f61a7114b80 1 --2- 172.21.15.90:0/1099183066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 0x558ce17c1330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f617c00b2e0 tx=0x7f617c009510 comp rx=0 tx=0).stop 2024-05-27T07:57:19.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.839+0000 7f618e000700 1 -- 172.21.15.90:0/293937319 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f61840e2bb0 con 0x7f6190103a00 2024-05-27T07:57:19.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.840+0000 7f6195c00700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61780b2b40 0x7f61780b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.840+0000 7f61a7114b80 1 -- 172.21.15.90:0/1099183066 shutdown_connections 2024-05-27T07:57:19.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.840+0000 7f61a7114b80 1 --2- 172.21.15.90:0/1099183066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.840+0000 7f61a7114b80 1 --2- 172.21.15.90:0/1099183066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 0x558ce17c1330 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.840+0000 7f61a7114b80 1 --2- 172.21.15.90:0/1099183066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.840+0000 7f61a7114b80 1 -- 172.21.15.90:0/1099183066 >> 172.21.15.90:0/1099183066 conn(0x558ce16dc390 msgr2=0x558ce17bd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.841+0000 7f6195c00700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61780b2b40 0x7f61780b5050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x558ce17be790 tx=0x7f6188009920 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.841+0000 7f61a7114b80 1 -- 172.21.15.90:0/1099183066 shutdown_connections 2024-05-27T07:57:19.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.841+0000 7f61a7114b80 1 -- 172.21.15.90:0/1099183066 wait complete. 2024-05-27T07:57:19.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.842+0000 7f61a7114b80 1 -- start start 2024-05-27T07:57:19.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.843+0000 7f61a7114b80 1 Processor -- start 2024-05-27T07:57:19.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.843+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.844+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 0x558ce17cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.844+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.844+0000 7f6195c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.845+0000 7f6195c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34124/0 (socket says 172.21.15.90:34124) 2024-05-27T07:57:19.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.845+0000 7f6195c00700 1 -- 172.21.15.90:0/141799703 learned_addr learned my addr 172.21.15.90:0/141799703 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:19.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.845+0000 7f6195200700 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 0x558ce17cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.845+0000 7f61a7114b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558ce15e5fa0 con 0x558ce1642e40 2024-05-27T07:57:19.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.845+0000 7f61a7114b80 1 -- 172.21.15.90:0/141799703 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558ce1680760 con 0x558ce17c1e70 2024-05-27T07:57:19.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.845+0000 7f61a7114b80 1 -- 172.21.15.90:0/141799703 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558ce17ceac0 con 0x558ce172e9f0 2024-05-27T07:57:19.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.845+0000 7f6196600700 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.846+0000 7f61a7114b80 10 client.0 _ll_register_callbacks cb 0x558ce1689c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:19.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.846+0000 7f6195c00700 1 -- 172.21.15.90:0/141799703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 msgr2=0x558ce17cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.846+0000 7f6195c00700 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 0x558ce17cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.846+0000 7f6195c00700 1 -- 172.21.15.90:0/141799703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 msgr2=0x558ce17cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.846+0000 7f6195c00700 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.846+0000 7f6195c00700 1 -- 172.21.15.90:0/141799703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce162f8d0 con 0x558ce1642e40 2024-05-27T07:57:19.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.846+0000 7f6195c00700 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17cb6a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f6188015e30 tx=0x7f6188004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14199d2eead7149 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.846+0000 7f6177400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:57:19.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.847+0000 7f6177400700 1 -- 172.21.15.90:0/141799703 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618801c050 con 0x558ce1642e40 2024-05-27T07:57:19.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.847+0000 7f6177400700 1 -- 172.21.15.90:0/141799703 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f6188016ce0 con 0x558ce1642e40 2024-05-27T07:57:19.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.847+0000 7f61a7114b80 1 -- 172.21.15.90:0/141799703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f617c004830 con 0x558ce1642e40 2024-05-27T07:57:19.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.848+0000 7f6177400700 1 -- 172.21.15.90:0/141799703 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f618802b450 con 0x558ce1642e40 2024-05-27T07:57:19.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.848+0000 7f61a7114b80 10 client.17409 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:19.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.848+0000 7f61a7114b80 1 -- 172.21.15.90:0/141799703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558ce16b0060 con 0x558ce1642e40 2024-05-27T07:57:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.849+0000 7f6177400700 1 -- 172.21.15.90:0/141799703 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f617c004830 con 0x558ce1642e40 2024-05-27T07:57:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.849+0000 7f6177400700 1 -- 172.21.15.90:0/141799703 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f6188021530 con 0x558ce1642e40 2024-05-27T07:57:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.850+0000 7f61a7114b80 10 client.17409 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.850+0000 7f61a7114b80 10 client.17409 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:57:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.850+0000 7f61a7114b80 20 client.17409 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.850+0000 7f61a7114b80 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558ce17eccc0 0x558ce17ef170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.850+0000 7f61a7114b80 4 client.17409 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:19.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.850+0000 7f61a7114b80 1 -- 172.21.15.90:0/141799703 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ce16f1d90 con 0x558ce17eccc0 2024-05-27T07:57:19.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.851+0000 7f6195200700 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558ce17eccc0 0x558ce17ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.852+0000 7f6177400700 10 client.17409 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:57:19.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.852+0000 7f6195200700 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558ce17eccc0 0x558ce17ef170 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12749823ff8ebafd server_cookie=e77b5a0a08bee478 in_seq=0 out_seq=0 2024-05-27T07:57:19.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.854+0000 7f6177400700 1 -- 172.21.15.90:0/141799703 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558ce16f1d90 con 0x558ce17eccc0 2024-05-27T07:57:19.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.854+0000 7f6177400700 10 client.17409 handle_command_reply: tid=0 2024-05-27T07:57:19.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.855+0000 7f61a7114b80 1 client.17409 shutdown 2024-05-27T07:57:19.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.855+0000 7f61a7114b80 1 -- 172.21.15.90:0/141799703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 msgr2=0x558ce17cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.855+0000 7f61a7114b80 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17cb6a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f6188015e30 tx=0x7f6188004b40 comp rx=0 tx=0).stop 2024-05-27T07:57:19.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.855+0000 7f61a7114b80 1 -- 172.21.15.90:0/141799703 shutdown_connections 2024-05-27T07:57:19.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.855+0000 7f61a7114b80 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558ce17eccc0 0x558ce17ef170 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.855+0000 7f61a7114b80 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.855+0000 7f61a7114b80 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17cb6a0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.855+0000 7f61a7114b80 1 --2- 172.21.15.90:0/141799703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce172e9f0 0x558ce17cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.855+0000 7f61a7114b80 1 -- 172.21.15.90:0/141799703 >> 172.21.15.90:0/141799703 conn(0x558ce16dc390 msgr2=0x558ce17c1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.855+0000 7f6177400700 0 client.17409 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:57:19.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.856+0000 7f61a7114b80 1 -- 172.21.15.90:0/141799703 shutdown_connections 2024-05-27T07:57:19.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.856+0000 7f61a7114b80 1 -- 172.21.15.90:0/141799703 wait complete. 2024-05-27T07:57:19.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.856+0000 7f61a7114b80 20 client.17409 trim_cache size 0 max 16384 2024-05-27T07:57:19.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.886+0000 7f61a7114b80 1 Processor -- start 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.886+0000 7f61a7114b80 1 -- start start 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.887+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.887+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce172e9f0 0x558ce17ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.887+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce17c1e70 0x558ce17c17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.888+0000 7f61a7114b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f61901afec0 con 0x558ce1642e40 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.888+0000 7f61a7114b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f61900748a0 con 0x558ce172e9f0 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.888+0000 7f61a7114b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f61901af120 con 0x558ce17c1e70 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.888+0000 7f6196600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce17c1e70 0x558ce17c17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.888+0000 7f6196600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce17c1e70 0x558ce17c17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57498/0 (socket says 172.21.15.90:57498) 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.888+0000 7f6196600700 1 -- 172.21.15.90:0/3685987745 learned_addr learned my addr 172.21.15.90:0/3685987745 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.889+0000 7f6195200700 1 --2- 172.21.15.90:0/3685987745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce172e9f0 0x558ce17ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.889+0000 7f6196600700 1 -- 172.21.15.90:0/3685987745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce172e9f0 msgr2=0x558ce17ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.889+0000 7f6196600700 1 --2- 172.21.15.90:0/3685987745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce172e9f0 0x558ce17ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.889+0000 7f6196600700 1 -- 172.21.15.90:0/3685987745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 msgr2=0x558ce17c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.889+0000 7f6196600700 1 --2- 172.21.15.90:0/3685987745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.889+0000 7f6196600700 1 -- 172.21.15.90:0/3685987745 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce17cc490 con 0x558ce17c1e70 2024-05-27T07:57:19.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.891+0000 7f6196600700 1 --2- 172.21.15.90:0/3685987745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce17c1e70 0x558ce17c17c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f617c006010 tx=0x7f617c00bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1303a4ac5c8b228 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.892+0000 7f6182000700 1 -- 172.21.15.90:0/3685987745 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c00bd10 con 0x558ce17c1e70 2024-05-27T07:57:19.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.893+0000 7f6182000700 1 -- 172.21.15.90:0/3685987745 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f617c004cc0 con 0x558ce17c1e70 2024-05-27T07:57:19.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.893+0000 7f6182000700 1 -- 172.21.15.90:0/3685987745 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c00f460 con 0x558ce17c1e70 2024-05-27T07:57:19.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.895+0000 7f61a7114b80 1 -- 172.21.15.90:0/3685987745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce17c1e70 msgr2=0x558ce17c17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.895+0000 7f61a7114b80 1 --2- 172.21.15.90:0/3685987745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce17c1e70 0x558ce17c17c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f617c006010 tx=0x7f617c00bb70 comp rx=0 tx=0).stop 2024-05-27T07:57:19.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.895+0000 7f61a7114b80 1 -- 172.21.15.90:0/3685987745 shutdown_connections 2024-05-27T07:57:19.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.895+0000 7f61a7114b80 1 --2- 172.21.15.90:0/3685987745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce17c1e70 0x558ce17c17c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.895+0000 7f61a7114b80 1 --2- 172.21.15.90:0/3685987745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce1642e40 0x558ce17c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.895+0000 7f61a7114b80 1 --2- 172.21.15.90:0/3685987745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce172e9f0 0x558ce17ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.895+0000 7f61a7114b80 1 -- 172.21.15.90:0/3685987745 >> 172.21.15.90:0/3685987745 conn(0x558ce16dc390 msgr2=0x558ce17c10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.896+0000 7f61a7114b80 1 -- 172.21.15.90:0/3685987745 shutdown_connections 2024-05-27T07:57:19.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.896+0000 7f61a7114b80 1 -- 172.21.15.90:0/3685987745 wait complete. 2024-05-27T07:57:19.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.897+0000 7f61a7114b80 1 -- start start 2024-05-27T07:57:19.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.898+0000 7f61a7114b80 1 Processor -- start 2024-05-27T07:57:19.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.899+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce1642e40 0x558ce17c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.900+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce172e9f0 0x558ce17c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.900+0000 7f61a7114b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.900+0000 7f6195200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce172e9f0 0x558ce17c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.900+0000 7f61a7114b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f61901aea50 con 0x558ce172e9f0 2024-05-27T07:57:19.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.900+0000 7f61a7114b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6190074f70 con 0x558ce17c1e70 2024-05-27T07:57:19.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.900+0000 7f61a7114b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6190075640 con 0x558ce1642e40 2024-05-27T07:57:19.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.901+0000 7f61a7114b80 10 client.0 _ll_register_callbacks cb 0x558ce1689c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:19.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.901+0000 7f6195200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce172e9f0 0x558ce17c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34152/0 (socket says 172.21.15.90:34152) 2024-05-27T07:57:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.901+0000 7f6195200700 1 -- 172.21.15.90:0/860825296 learned_addr learned my addr 172.21.15.90:0/860825296 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.900+0000 7f6195c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce1642e40 0x558ce17c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.901+0000 7f6196600700 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.902+0000 7f6195c00700 1 -- 172.21.15.90:0/860825296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 msgr2=0x558ce17c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.902+0000 7f6195c00700 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.902+0000 7f6195c00700 1 -- 172.21.15.90:0/860825296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce172e9f0 msgr2=0x558ce17c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:19.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.903+0000 7f6195c00700 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce172e9f0 0x558ce17c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.903+0000 7f6195c00700 1 -- 172.21.15.90:0/860825296 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce17e3f60 con 0x558ce1642e40 2024-05-27T07:57:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.903+0000 7f6195c00700 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce1642e40 0x558ce17c5540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6188013cb0 tx=0x7f6188015f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77645b6addcd38f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.904+0000 7f6177400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:57:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.904+0000 7f6177400700 1 -- 172.21.15.90:0/860825296 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618803a020 con 0x558ce1642e40 2024-05-27T07:57:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.904+0000 7f6177400700 1 -- 172.21.15.90:0/860825296 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f6188021640 con 0x558ce1642e40 2024-05-27T07:57:19.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.904+0000 7f61a7114b80 1 -- 172.21.15.90:0/860825296 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f617c004cc0 con 0x558ce1642e40 2024-05-27T07:57:19.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.905+0000 7f6177400700 1 -- 172.21.15.90:0/860825296 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f618801c050 con 0x558ce1642e40 2024-05-27T07:57:19.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.905+0000 7f61a7114b80 10 client.27070 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:19.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.905+0000 7f61a7114b80 1 -- 172.21.15.90:0/860825296 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558ce17e51a0 con 0x558ce1642e40 2024-05-27T07:57:19.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.906+0000 7f6177400700 1 -- 172.21.15.90:0/860825296 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6188026070 con 0x558ce1642e40 2024-05-27T07:57:19.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.906+0000 7f6177400700 1 -- 172.21.15.90:0/860825296 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f6188033030 con 0x558ce1642e40 2024-05-27T07:57:19.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.906+0000 7f61a7114b80 10 client.27070 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:19.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.906+0000 7f61a7114b80 10 client.27070 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:57:19.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.906+0000 7f61a7114b80 20 client.27070 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:19.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.906+0000 7f61a7114b80 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558ce1857800 0x558ce1859c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:19.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.906+0000 7f61a7114b80 4 client.27070 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:57:19.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.906+0000 7f61a7114b80 1 -- 172.21.15.90:0/860825296 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f617c004cc0 con 0x558ce1857800 2024-05-27T07:57:19.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.907+0000 7f6195200700 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558ce1857800 0x558ce1859c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:19.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.908+0000 7f6195200700 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558ce1857800 0x558ce1859c20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b16eaa7ceb48c79 server_cookie=904fc14986f8db51 in_seq=0 out_seq=0 2024-05-27T07:57:19.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.908+0000 7f6177400700 10 client.27070 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:57:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.909+0000 7f6177400700 1 -- 172.21.15.90:0/860825296 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f617c004cc0 con 0x558ce1857800 2024-05-27T07:57:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.909+0000 7f6177400700 10 client.27070 handle_command_reply: tid=0 2024-05-27T07:57:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.909+0000 7f61a7114b80 1 client.27070 shutdown 2024-05-27T07:57:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.909+0000 7f61a7114b80 1 -- 172.21.15.90:0/860825296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce1642e40 msgr2=0x558ce17c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.909+0000 7f61a7114b80 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce1642e40 0x558ce17c5540 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6188013cb0 tx=0x7f6188015f90 comp rx=0 tx=0).stop 2024-05-27T07:57:19.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.910+0000 7f61a7114b80 1 -- 172.21.15.90:0/860825296 shutdown_connections 2024-05-27T07:57:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.910+0000 7f61a7114b80 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x558ce1857800 0x558ce1859c20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.910+0000 7f61a7114b80 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558ce17c1e70 0x558ce17c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.910+0000 7f61a7114b80 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558ce1642e40 0x558ce17c5540 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.910+0000 7f61a7114b80 1 --2- 172.21.15.90:0/860825296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558ce172e9f0 0x558ce17c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.910+0000 7f61a7114b80 1 -- 172.21.15.90:0/860825296 >> 172.21.15.90:0/860825296 conn(0x558ce16dc390 msgr2=0x558ce17cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.910+0000 7f6177400700 0 client.27070 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:57:19.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.911+0000 7f61a7114b80 1 -- 172.21.15.90:0/860825296 shutdown_connections 2024-05-27T07:57:19.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.911+0000 7f61a7114b80 1 -- 172.21.15.90:0/860825296 wait complete. 2024-05-27T07:57:19.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.912+0000 7f61a7114b80 20 client.27070 trim_cache size 0 max 16384 2024-05-27T07:57:19.934 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:57:19.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.923+0000 7f6198600700 1 -- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61780b2b40 msgr2=0x7f61780b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.923+0000 7f6198600700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61780b2b40 0x7f61780b5050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x558ce17be790 tx=0x7f6188009920 comp rx=0 tx=0).stop 2024-05-27T07:57:19.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.923+0000 7f6198600700 1 -- 172.21.15.90:0/293937319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6190103a00 msgr2=0x7f619010f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:19.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.923+0000 7f6198600700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6190103a00 0x7f619010f580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f618400e970 tx=0x7f618400ed30 comp rx=0 tx=0).stop 2024-05-27T07:57:19.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.923+0000 7f6198600700 1 -- 172.21.15.90:0/293937319 shutdown_connections 2024-05-27T07:57:19.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.923+0000 7f6198600700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6190110c60 0x7f61901acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.923+0000 7f6198600700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61901030b0 0x7f619010f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:19.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.923+0000 7f6198600700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f61780b2b40 0x7f61780b5050 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x558ce17be790 tx=0x7f6188009920 comp rx=0 tx=0).stop 2024-05-27T07:57:19.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.923+0000 7f6198600700 1 --2- 172.21.15.90:0/293937319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6190103a00 0x7f619010f580 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f618400e970 tx=0x7f618400ed30 comp rx=0 tx=0).stop 2024-05-27T07:57:19.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.923+0000 7f6198600700 1 -- 172.21.15.90:0/293937319 >> 172.21.15.90:0/293937319 conn(0x7f61900fedb0 msgr2=0x7f6190112810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:19.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.936+0000 7f6198600700 1 -- 172.21.15.90:0/293937319 shutdown_connections 2024-05-27T07:57:19.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:19.948+0000 7f6198600700 1 -- 172.21.15.90:0/293937319 wait complete. 2024-05-27T07:57:20.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:20 smithi136 ceph-mon[35324]: pgmap v347: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.4 MiB/s wr, 496 op/s 2024-05-27T07:57:20.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:20 smithi129 ceph-mon[35254]: pgmap v347: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.4 MiB/s wr, 496 op/s 2024-05-27T07:57:20.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:20 smithi090 ceph-mon[31429]: pgmap v347: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.4 MiB/s wr, 496 op/s 2024-05-27T07:57:21.691 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:57:22.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:22 smithi136 ceph-mon[35324]: pgmap v348: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-05-27T07:57:22.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:22 smithi129 ceph-mon[35254]: pgmap v348: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-05-27T07:57:22.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:22 smithi090 ceph-mon[31429]: pgmap v348: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.5 MiB/s wr, 496 op/s 2024-05-27T07:57:23.102 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:23.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 425.0 s, 537.2 tps, lat 183.945 ms stddev 109.974 2024-05-27T07:57:24.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:24 smithi090 ceph-mon[31429]: pgmap v349: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.5 MiB/s wr, 500 op/s 2024-05-27T07:57:24.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:24 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:24.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:24 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:24.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:24 smithi129 ceph-mon[35254]: pgmap v349: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.5 MiB/s wr, 500 op/s 2024-05-27T07:57:24.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:24 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:24.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:24 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:24.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:24 smithi136 ceph-mon[35324]: pgmap v349: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 4.5 MiB/s wr, 500 op/s 2024-05-27T07:57:24.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:24 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:24.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:24 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:26.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:26 smithi090 ceph-mon[31429]: pgmap v350: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.0 MiB/s wr, 548 op/s 2024-05-27T07:57:26.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:26 smithi136 ceph-mon[35324]: pgmap v350: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.0 MiB/s wr, 548 op/s 2024-05-27T07:57:26.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:26 smithi129 ceph-mon[35254]: pgmap v350: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.0 MiB/s wr, 548 op/s 2024-05-27T07:57:27.334 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.714 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.714 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.714 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.714 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.714 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.714 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.715 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.715 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.715 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-05-27T07:57:27.715 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-05-27T07:57:27.715 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:57:27.715 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:57:27.715 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: pgmap v351: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-05-27T07:57:27.715 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:27 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:27.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-05-27T07:57:27.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-05-27T07:57:27.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:57:27.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:57:27.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: pgmap v351: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-05-27T07:57:27.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:27 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:27.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: Unable to set osd_memory_target on smithi090 to 185041920: error parsing value: Value '185041920' is below minimum 939524096 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: pgmap v351: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-05-27T07:57:27.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:27 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:57:28.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.025+0000 7f4425800700 1 -- 172.21.15.90:0/1464356229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4420107de0 msgr2=0x7f44201081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:28.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.025+0000 7f4425800700 1 --2- 172.21.15.90:0/1464356229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4420107de0 0x7f44201081f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44080098f0 tx=0x7f4408009c00 comp rx=0 tx=0).stop 2024-05-27T07:57:28.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.027+0000 7f4425800700 1 -- 172.21.15.90:0/1464356229 shutdown_connections 2024-05-27T07:57:28.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.027+0000 7f4425800700 1 --2- 172.21.15.90:0/1464356229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4420108730 0x7f4420110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.027+0000 7f4425800700 1 --2- 172.21.15.90:0/1464356229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44201034b0 0x7f44201038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.027+0000 7f4425800700 1 --2- 172.21.15.90:0/1464356229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4420107de0 0x7f44201081f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.027+0000 7f4425800700 1 -- 172.21.15.90:0/1464356229 >> 172.21.15.90:0/1464356229 conn(0x7f44200fedb0 msgr2=0x7f44201011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:28.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.027+0000 7f4425800700 1 -- 172.21.15.90:0/1464356229 shutdown_connections 2024-05-27T07:57:28.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.027+0000 7f4425800700 1 -- 172.21.15.90:0/1464356229 wait complete. 2024-05-27T07:57:28.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.030+0000 7f4425800700 1 Processor -- start 2024-05-27T07:57:28.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.030+0000 7f4425800700 1 -- start start 2024-05-27T07:57:28.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.031+0000 7f4425800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44201034b0 0x7f44201a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.031+0000 7f4425800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4420107de0 0x7f44201a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.031+0000 7f4425800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4420108730 0x7f44201a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.031+0000 7f4425800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f44201121c0 con 0x7f44201034b0 2024-05-27T07:57:28.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.031+0000 7f4425800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4420112040 con 0x7f4420108730 2024-05-27T07:57:28.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.031+0000 7f4425800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4420112340 con 0x7f4420107de0 2024-05-27T07:57:28.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.032+0000 7f441ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4420107de0 0x7f44201a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.032+0000 7f441ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4420107de0 0x7f44201a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57522/0 (socket says 172.21.15.90:57522) 2024-05-27T07:57:28.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.032+0000 7f441ea00700 1 -- 172.21.15.90:0/4256873468 learned_addr learned my addr 172.21.15.90:0/4256873468 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:28.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.033+0000 7f441f400700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4420108730 0x7f44201a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.033+0000 7f4416a00700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44201034b0 0x7f44201a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.033+0000 7f441ea00700 1 -- 172.21.15.90:0/4256873468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4420108730 msgr2=0x7f44201a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:28.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.033+0000 7f441ea00700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4420108730 0x7f44201a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.033+0000 7f441ea00700 1 -- 172.21.15.90:0/4256873468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44201034b0 msgr2=0x7f44201a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:28.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.033+0000 7f441ea00700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44201034b0 0x7f44201a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.033+0000 7f441ea00700 1 -- 172.21.15.90:0/4256873468 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44201a6290 con 0x7f4420107de0 2024-05-27T07:57:28.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.034+0000 7f441ea00700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4420107de0 0x7f44201a1960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4408009230 tx=0x7f4408009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:28.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.035+0000 7f4417e00700 1 -- 172.21.15.90:0/4256873468 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408025070 con 0x7f4420107de0 2024-05-27T07:57:28.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.035+0000 7f4417e00700 1 -- 172.21.15.90:0/4256873468 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4408005040 con 0x7f4420107de0 2024-05-27T07:57:28.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.035+0000 7f4417e00700 1 -- 172.21.15.90:0/4256873468 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4408003aa0 con 0x7f4420107de0 2024-05-27T07:57:28.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.035+0000 7f4425800700 1 -- 172.21.15.90:0/4256873468 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44201a6520 con 0x7f4420107de0 2024-05-27T07:57:28.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.035+0000 7f4425800700 1 -- 172.21.15.90:0/4256873468 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44201a69b0 con 0x7f4420107de0 2024-05-27T07:57:28.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.043+0000 7f4417e00700 1 -- 172.21.15.90:0/4256873468 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4408028070 con 0x7f4420107de0 2024-05-27T07:57:28.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.045+0000 7f4417e00700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44040b2b20 0x7f44040b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.045+0000 7f4417e00700 1 -- 172.21.15.90:0/4256873468 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f44080e31d0 con 0x7f4420107de0 2024-05-27T07:57:28.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.046+0000 7f4416a00700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44040b2b20 0x7f44040b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.049+0000 7f4416a00700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44040b2b20 0x7f44040b5030 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4410009750 tx=0x7f4410009310 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:28.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.082+0000 7f4434326b80 1 Processor -- start 2024-05-27T07:57:28.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.082+0000 7f4434326b80 1 -- start start 2024-05-27T07:57:28.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.083+0000 7f4434326b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462bc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.083+0000 7f4434326b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 0x563c462bc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.083+0000 7f4434326b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462c9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.083+0000 7f4434326b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563c461eae40 con 0x563c462299f0 2024-05-27T07:57:28.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.083+0000 7f4434326b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563c461da9e0 con 0x563c4613de40 2024-05-27T07:57:28.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.083+0000 7f4434326b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563c461dea20 con 0x563c462bce70 2024-05-27T07:57:28.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.084+0000 7f4416a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462bc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.084+0000 7f441f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.084+0000 7f441f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57526/0 (socket says 172.21.15.90:57526) 2024-05-27T07:57:28.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.084+0000 7f441f400700 1 -- 172.21.15.90:0/527829866 learned_addr learned my addr 172.21.15.90:0/527829866 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:28.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.085+0000 7f441ea00700 1 --2- 172.21.15.90:0/527829866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 0x563c462bc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.085+0000 7f441f400700 1 -- 172.21.15.90:0/527829866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 msgr2=0x563c462bc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:28.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.085+0000 7f441f400700 1 --2- 172.21.15.90:0/527829866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 0x563c462bc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.085+0000 7f441f400700 1 -- 172.21.15.90:0/527829866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 msgr2=0x563c462bc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:28.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.085+0000 7f441f400700 1 --2- 172.21.15.90:0/527829866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462bc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.085+0000 7f441f400700 1 -- 172.21.15.90:0/527829866 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c461a3420 con 0x563c462bce70 2024-05-27T07:57:28.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.088+0000 7f441f400700 1 --2- 172.21.15.90:0/527829866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462c9110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4418009440 tx=0x7f4418009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57158b8d32cc5b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:28.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.090+0000 7f440fe00700 1 -- 172.21.15.90:0/527829866 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418013750 con 0x563c462bce70 2024-05-27T07:57:28.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.090+0000 7f440fe00700 1 -- 172.21.15.90:0/527829866 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4418013d80 con 0x563c462bce70 2024-05-27T07:57:28.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.090+0000 7f440fe00700 1 -- 172.21.15.90:0/527829866 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418015e40 con 0x563c462bce70 2024-05-27T07:57:28.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.093+0000 7f4434326b80 1 -- 172.21.15.90:0/527829866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 msgr2=0x563c462c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:28.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.093+0000 7f4434326b80 1 --2- 172.21.15.90:0/527829866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462c9110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4418009440 tx=0x7f4418009920 comp rx=0 tx=0).stop 2024-05-27T07:57:28.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.094+0000 7f4434326b80 1 -- 172.21.15.90:0/527829866 shutdown_connections 2024-05-27T07:57:28.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.094+0000 7f4434326b80 1 --2- 172.21.15.90:0/527829866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462c9110 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.094+0000 7f4434326b80 1 --2- 172.21.15.90:0/527829866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462bc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.094+0000 7f4434326b80 1 --2- 172.21.15.90:0/527829866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 0x563c462bc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.094+0000 7f4434326b80 1 -- 172.21.15.90:0/527829866 >> 172.21.15.90:0/527829866 conn(0x563c461d7390 msgr2=0x563c462b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:28.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.094+0000 7f4434326b80 1 -- 172.21.15.90:0/527829866 shutdown_connections 2024-05-27T07:57:28.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.095+0000 7f4434326b80 1 -- 172.21.15.90:0/527829866 wait complete. 2024-05-27T07:57:28.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.096+0000 7f4434326b80 1 -- start start 2024-05-27T07:57:28.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.097+0000 7f4434326b80 1 Processor -- start 2024-05-27T07:57:28.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.099+0000 7f4434326b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 0x563c462c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.099+0000 7f4434326b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.099+0000 7f4434326b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.099+0000 7f4434326b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563c460e0fa0 con 0x563c462299f0 2024-05-27T07:57:28.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.099+0000 7f4434326b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563c4617b760 con 0x563c4613de40 2024-05-27T07:57:28.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.099+0000 7f4434326b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563c462c9b80 con 0x563c462bce70 2024-05-27T07:57:28.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.099+0000 7f4434326b80 10 client.0 _ll_register_callbacks cb 0x563c46184c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:28.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.100+0000 7f4416a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 0x563c462c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.100+0000 7f4416a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 0x563c462c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:60014/0 (socket says 172.21.15.90:60014) 2024-05-27T07:57:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.100+0000 7f441ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.100+0000 7f4416a00700 1 -- 172.21.15.90:0/825958088 learned_addr learned my addr 172.21.15.90:0/825958088 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.100+0000 7f441f400700 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.101+0000 7f4416a00700 1 -- 172.21.15.90:0/825958088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 msgr2=0x563c462c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.101+0000 7f4416a00700 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.101+0000 7f4416a00700 1 -- 172.21.15.90:0/825958088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 msgr2=0x563c462c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.101+0000 7f4416a00700 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.101+0000 7f4416a00700 1 -- 172.21.15.90:0/825958088 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c4612a8d0 con 0x563c4613de40 2024-05-27T07:57:28.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.101+0000 7f441ea00700 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.102+0000 7f4416a00700 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 0x563c462c6680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4410010ea0 tx=0x7f44100047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330627e7e633f8b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.102+0000 7f440cc00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:57:28.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.103+0000 7f440cc00700 1 -- 172.21.15.90:0/825958088 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410020430 con 0x563c4613de40 2024-05-27T07:57:28.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.103+0000 7f440cc00700 1 -- 172.21.15.90:0/825958088 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f44100205d0 con 0x563c4613de40 2024-05-27T07:57:28.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.104+0000 7f4434326b80 1 -- 172.21.15.90:0/825958088 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4418013d80 con 0x563c4613de40 2024-05-27T07:57:28.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.106+0000 7f440cc00700 1 -- 172.21.15.90:0/825958088 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f44100207d0 con 0x563c4613de40 2024-05-27T07:57:28.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.106+0000 7f4434326b80 10 client.37013 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:28.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.106+0000 7f4434326b80 1 -- 172.21.15.90:0/825958088 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x563c461ab060 con 0x563c4613de40 2024-05-27T07:57:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.107+0000 7f440cc00700 1 -- 172.21.15.90:0/825958088 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4418013d80 con 0x563c4613de40 2024-05-27T07:57:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.107+0000 7f440cc00700 1 -- 172.21.15.90:0/825958088 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f441000f040 con 0x563c4613de40 2024-05-27T07:57:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.107+0000 7f4434326b80 10 client.37013 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.107+0000 7f4434326b80 10 client.37013 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:57:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.107+0000 7f4434326b80 20 client.37013 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.107+0000 7f4434326b80 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x563c462d6c60 0x563c462d90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.107+0000 7f4434326b80 4 client.37013 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.107+0000 7f4434326b80 1 -- 172.21.15.90:0/825958088 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c461ecd90 con 0x563c462d6c60 2024-05-27T07:57:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.108+0000 7f441ea00700 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x563c462d6c60 0x563c462d90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.109+0000 7f441ea00700 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x563c462d6c60 0x563c462d90e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a719d0b0834c56d server_cookie=f12c6ae3f18d5b6d in_seq=0 out_seq=0 2024-05-27T07:57:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.109+0000 7f440cc00700 10 client.37013 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:57:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.111+0000 7f440cc00700 1 -- 172.21.15.90:0/825958088 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563c461ecd90 con 0x563c462d6c60 2024-05-27T07:57:28.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.111+0000 7f440cc00700 10 client.37013 handle_command_reply: tid=0 2024-05-27T07:57:28.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.111+0000 7f4434326b80 1 client.37013 shutdown 2024-05-27T07:57:28.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.112+0000 7f4434326b80 1 -- 172.21.15.90:0/825958088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 msgr2=0x563c462c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:28.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.112+0000 7f4434326b80 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 0x563c462c6680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4410010ea0 tx=0x7f44100047c0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.113+0000 7f4434326b80 1 -- 172.21.15.90:0/825958088 shutdown_connections 2024-05-27T07:57:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.113+0000 7f4434326b80 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.113+0000 7f4434326b80 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x563c462d6c60 0x563c462d90e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.113+0000 7f4434326b80 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c4613de40 0x563c462c6680 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4410010ea0 tx=0x7f44100047c0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.113+0000 7f4434326b80 1 --2- 172.21.15.90:0/825958088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.113+0000 7f4434326b80 1 -- 172.21.15.90:0/825958088 >> 172.21.15.90:0/825958088 conn(0x563c461d7390 msgr2=0x563c462bc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:28.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.113+0000 7f440cc00700 0 client.37013 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:57:28.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.113+0000 7f4434326b80 1 -- 172.21.15.90:0/825958088 shutdown_connections 2024-05-27T07:57:28.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.114+0000 7f4434326b80 1 -- 172.21.15.90:0/825958088 wait complete. 2024-05-27T07:57:28.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.114+0000 7f4434326b80 20 client.37013 trim_cache size 0 max 16384 2024-05-27T07:57:28.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.146+0000 7f4434326b80 1 Processor -- start 2024-05-27T07:57:28.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.146+0000 7f4434326b80 1 -- start start 2024-05-27T07:57:28.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.147+0000 7f4434326b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c461d7390 0x563c462c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.147+0000 7f4434326b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c462299f0 0x563c462bc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.147+0000 7f4434326b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462bce70 0x563c462d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.147+0000 7f4434326b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f44201b84d0 con 0x563c462bce70 2024-05-27T07:57:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.147+0000 7f4434326b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44201b5520 con 0x563c462299f0 2024-05-27T07:57:28.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.147+0000 7f4434326b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f44201b7730 con 0x563c461d7390 2024-05-27T07:57:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.147+0000 7f4416a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c461d7390 0x563c462c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.147+0000 7f4416a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c461d7390 0x563c462c1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57550/0 (socket says 172.21.15.90:57550) 2024-05-27T07:57:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.147+0000 7f4416a00700 1 -- 172.21.15.90:0/2217548956 learned_addr learned my addr 172.21.15.90:0/2217548956 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.147+0000 7f441ea00700 1 --2- 172.21.15.90:0/2217548956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c462299f0 0x563c462bc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.148+0000 7f441ea00700 1 -- 172.21.15.90:0/2217548956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c461d7390 msgr2=0x563c462c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.148+0000 7f441ea00700 1 --2- 172.21.15.90:0/2217548956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c461d7390 0x563c462c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.148+0000 7f441ea00700 1 -- 172.21.15.90:0/2217548956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462bce70 msgr2=0x563c462d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:28.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.148+0000 7f441ea00700 1 --2- 172.21.15.90:0/2217548956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462bce70 0x563c462d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.148+0000 7f441ea00700 1 -- 172.21.15.90:0/2217548956 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c462b4240 con 0x563c462299f0 2024-05-27T07:57:28.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.148+0000 7f441ea00700 1 --2- 172.21.15.90:0/2217548956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c462299f0 0x563c462bc7c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f44080e04f0 tx=0x7f4408005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3392e4f1939e7a82 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:28.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.148+0000 7f440f400700 1 -- 172.21.15.90:0/2217548956 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44080c4070 con 0x563c462299f0 2024-05-27T07:57:28.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.149+0000 7f440f400700 1 -- 172.21.15.90:0/2217548956 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44080ae4c0 con 0x563c462299f0 2024-05-27T07:57:28.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.149+0000 7f440f400700 1 -- 172.21.15.90:0/2217548956 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44080aa370 con 0x563c462299f0 2024-05-27T07:57:28.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.150+0000 7f4434326b80 1 -- 172.21.15.90:0/2217548956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c462299f0 msgr2=0x563c462bc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:28.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.150+0000 7f4434326b80 1 --2- 172.21.15.90:0/2217548956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c462299f0 0x563c462bc7c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f44080e04f0 tx=0x7f4408005f00 comp rx=0 tx=0).stop 2024-05-27T07:57:28.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.150+0000 7f4434326b80 1 -- 172.21.15.90:0/2217548956 shutdown_connections 2024-05-27T07:57:28.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.150+0000 7f4434326b80 1 --2- 172.21.15.90:0/2217548956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462bce70 0x563c462d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.150+0000 7f4434326b80 1 --2- 172.21.15.90:0/2217548956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c461d7390 0x563c462c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.151+0000 7f4434326b80 1 --2- 172.21.15.90:0/2217548956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c462299f0 0x563c462bc7c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.151+0000 7f4434326b80 1 -- 172.21.15.90:0/2217548956 >> 172.21.15.90:0/2217548956 conn(0x563c462d9430 msgr2=0x563c462bc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:28.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.151+0000 7f4434326b80 1 -- 172.21.15.90:0/2217548956 shutdown_connections 2024-05-27T07:57:28.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.151+0000 7f4434326b80 1 -- 172.21.15.90:0/2217548956 wait complete. 2024-05-27T07:57:28.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.152+0000 7f4434326b80 1 -- start start 2024-05-27T07:57:28.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.153+0000 7f4434326b80 1 Processor -- start 2024-05-27T07:57:28.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.154+0000 7f4434326b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c461d7390 0x563c462cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.154+0000 7f4434326b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.155+0000 7f4434326b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.155+0000 7f4434326b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f44201b7060 con 0x563c462299f0 2024-05-27T07:57:28.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.155+0000 7f4434326b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44201b5bf0 con 0x563c461d7390 2024-05-27T07:57:28.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.155+0000 7f4434326b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f44201b62c0 con 0x563c462bce70 2024-05-27T07:57:28.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.155+0000 7f441ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.155+0000 7f4434326b80 10 client.0 _ll_register_callbacks cb 0x563c462b7220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:28.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.155+0000 7f441f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.155+0000 7f441f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57552/0 (socket says 172.21.15.90:57552) 2024-05-27T07:57:28.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.155+0000 7f441f400700 1 -- 172.21.15.90:0/1724783826 learned_addr learned my addr 172.21.15.90:0/1724783826 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:28.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.156+0000 7f441f400700 1 -- 172.21.15.90:0/1724783826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c461d7390 msgr2=0x563c462cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:28.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.156+0000 7f441f400700 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c461d7390 0x563c462cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.156+0000 7f441f400700 1 -- 172.21.15.90:0/1724783826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 msgr2=0x563c462c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:28.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.156+0000 7f441f400700 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.156+0000 7f441f400700 1 -- 172.21.15.90:0/1724783826 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c462cabc0 con 0x563c462bce70 2024-05-27T07:57:28.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.157+0000 7f441ea00700 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:28.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.157+0000 7f440cc00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:57:28.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.157+0000 7f441f400700 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462ca420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4418009440 tx=0x7f4418015670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc22349562ce81e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:28.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.158+0000 7f440cc00700 1 -- 172.21.15.90:0/1724783826 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418015e20 con 0x563c462bce70 2024-05-27T07:57:28.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.158+0000 7f440cc00700 1 -- 172.21.15.90:0/1724783826 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f4418013550 con 0x563c462bce70 2024-05-27T07:57:28.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.158+0000 7f4434326b80 1 -- 172.21.15.90:0/1724783826 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44080ae4c0 con 0x563c462bce70 2024-05-27T07:57:28.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.159+0000 7f440cc00700 1 -- 172.21.15.90:0/1724783826 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f441802e050 con 0x563c462bce70 2024-05-27T07:57:28.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.159+0000 7f4434326b80 10 client.27088 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:28.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.159+0000 7f4434326b80 1 -- 172.21.15.90:0/1724783826 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x563c463407a0 con 0x563c462bce70 2024-05-27T07:57:28.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.160+0000 7f440cc00700 1 -- 172.21.15.90:0/1724783826 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f44080ae4c0 con 0x563c462bce70 2024-05-27T07:57:28.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.160+0000 7f440cc00700 1 -- 172.21.15.90:0/1724783826 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f4418029680 con 0x563c462bce70 2024-05-27T07:57:28.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.161+0000 7f4434326b80 10 client.27088 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:28.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.161+0000 7f4434326b80 10 client.27088 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:57:28.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.161+0000 7f4434326b80 20 client.27088 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:28.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.161+0000 7f4434326b80 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x563c46340d90 0x563c463431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:28.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.161+0000 7f4434326b80 4 client.27088 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:57:28.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.161+0000 7f4434326b80 1 -- 172.21.15.90:0/1724783826 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c461e6570 con 0x563c46340d90 2024-05-27T07:57:28.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.161+0000 7f4416a00700 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x563c46340d90 0x563c463431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:28.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.162+0000 7f4416a00700 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x563c46340d90 0x563c463431b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d16dac3ca8da264c server_cookie=f9a16543bf7ce3da in_seq=0 out_seq=0 2024-05-27T07:57:28.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.162+0000 7f440cc00700 10 client.27088 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:57:28.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f440cc00700 1 -- 172.21.15.90:0/1724783826 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c461e6570 con 0x563c46340d90 2024-05-27T07:57:28.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f440cc00700 10 client.27088 handle_command_reply: tid=0 2024-05-27T07:57:28.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f4434326b80 1 client.27088 shutdown 2024-05-27T07:57:28.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f4434326b80 1 -- 172.21.15.90:0/1724783826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 msgr2=0x563c462ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:28.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f4434326b80 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462ca420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4418009440 tx=0x7f4418015670 comp rx=0 tx=0).stop 2024-05-27T07:57:28.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f4434326b80 1 -- 172.21.15.90:0/1724783826 shutdown_connections 2024-05-27T07:57:28.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f4434326b80 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c462bce70 0x563c462ca420 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f4434326b80 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x563c46340d90 0x563c463431b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f4434326b80 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c461d7390 0x563c462cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f4434326b80 1 --2- 172.21.15.90:0/1724783826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c462299f0 0x563c462c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f4434326b80 1 -- 172.21.15.90:0/1724783826 >> 172.21.15.90:0/1724783826 conn(0x563c462d9430 msgr2=0x563c462c6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:28.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f440cc00700 0 client.27088 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:57:28.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.163+0000 7f4434326b80 1 -- 172.21.15.90:0/1724783826 shutdown_connections 2024-05-27T07:57:28.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.164+0000 7f4434326b80 1 -- 172.21.15.90:0/1724783826 wait complete. 2024-05-27T07:57:28.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.164+0000 7f4434326b80 20 client.27088 trim_cache size 0 max 16384 2024-05-27T07:57:28.169 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:57:28.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.165+0000 7f4425800700 1 -- 172.21.15.90:0/4256873468 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44040b2b20 msgr2=0x7f44040b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:28.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.165+0000 7f4425800700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44040b2b20 0x7f44040b5030 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4410009750 tx=0x7f4410009310 comp rx=0 tx=0).stop 2024-05-27T07:57:28.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.165+0000 7f4425800700 1 -- 172.21.15.90:0/4256873468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4420107de0 msgr2=0x7f44201a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:28.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.165+0000 7f4425800700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4420107de0 0x7f44201a1960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4408009230 tx=0x7f4408009400 comp rx=0 tx=0).stop 2024-05-27T07:57:28.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.166+0000 7f4425800700 1 -- 172.21.15.90:0/4256873468 shutdown_connections 2024-05-27T07:57:28.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.166+0000 7f4425800700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f44040b2b20 0x7f44040b5030 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.166+0000 7f4425800700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4420108730 0x7f44201a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.166+0000 7f4425800700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44201034b0 0x7f44201a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.166+0000 7f4425800700 1 --2- 172.21.15.90:0/4256873468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4420107de0 0x7f44201a1960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:28.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.166+0000 7f4425800700 1 -- 172.21.15.90:0/4256873468 >> 172.21.15.90:0/4256873468 conn(0x7f44200fedb0 msgr2=0x7f44201011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:28.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.166+0000 7f4425800700 1 -- 172.21.15.90:0/4256873468 shutdown_connections 2024-05-27T07:57:28.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:28.166+0000 7f4425800700 1 -- 172.21.15.90:0/4256873468 wait complete. 2024-05-27T07:57:28.337 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:28 smithi090 ceph-mon[31429]: Health check failed: 3 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-27T07:57:28.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:28 smithi136 ceph-mon[35324]: Health check failed: 3 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-27T07:57:28.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:28 smithi129 ceph-mon[35254]: Health check failed: 3 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-27T07:57:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 430.0 s, 589.2 tps, lat 172.637 ms stddev 129.451 2024-05-27T07:57:29.075 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:57:29.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:29 smithi090 ceph-mon[31429]: pgmap v352: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.8 MiB/s wr, 531 op/s 2024-05-27T07:57:29.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:29 smithi136 ceph-mon[35324]: pgmap v352: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.8 MiB/s wr, 531 op/s 2024-05-27T07:57:29.814 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:29 smithi129 ceph-mon[35254]: pgmap v352: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.8 MiB/s wr, 531 op/s 2024-05-27T07:57:29.816 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.660+0000 7fe7fba00640 1 -- 172.21.15.90:0/4102226859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7f4109c90 msgr2=0x7fe7f4103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.660+0000 7fe7fba00640 1 --2- 172.21.15.90:0/4102226859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7f4109c90 0x7fe7f4103830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009860 tx=0x7fe7e802a340 comp rx=0 tx=0).stop 2024-05-27T07:57:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.660+0000 7fe7fba00640 1 -- 172.21.15.90:0/4102226859 shutdown_connections 2024-05-27T07:57:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.660+0000 7fe7fba00640 1 --2- 172.21.15.90:0/4102226859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f41043e0 0x7fe7f41107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.660+0000 7fe7fba00640 1 --2- 172.21.15.90:0/4102226859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7f410a130 0x7fe7f4103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.660+0000 7fe7fba00640 1 --2- 172.21.15.90:0/4102226859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7f4109c90 0x7fe7f4103830 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.660+0000 7fe7fba00640 1 -- 172.21.15.90:0/4102226859 >> 172.21.15.90:0/4102226859 conn(0x7fe7f4079390 msgr2=0x7fe7f40797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.661+0000 7fe7fba00640 1 -- 172.21.15.90:0/4102226859 shutdown_connections 2024-05-27T07:57:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.661+0000 7fe7fba00640 1 -- 172.21.15.90:0/4102226859 wait complete. 2024-05-27T07:57:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.663+0000 7fe7fba00640 1 Processor -- start 2024-05-27T07:57:30.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.663+0000 7fe7fba00640 1 -- start start 2024-05-27T07:57:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.664+0000 7fe7fba00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7f41043e0 0x7fe7f4076b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.664+0000 7fe7fba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4109c90 0x7fe7f4074fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.664+0000 7fe7fba00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7f410a130 0x7fe7f40755c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.664+0000 7fe7fba00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4113d30 con 0x7fe7f41043e0 2024-05-27T07:57:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.664+0000 7fe7fba00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4113bb0 con 0x7fe7f410a130 2024-05-27T07:57:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.664+0000 7fe7fba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4113eb0 con 0x7fe7f4109c90 2024-05-27T07:57:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.665+0000 7fe7f9a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7f410a130 0x7fe7f40755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.665+0000 7fe7f3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4109c90 0x7fe7f4074fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.665+0000 7fe7f9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7f41043e0 0x7fe7f4076b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.665+0000 7fe7f9a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7f410a130 0x7fe7f40755c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:35964/0 (socket says 172.21.15.90:35964) 2024-05-27T07:57:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.665+0000 7fe7f9a00640 1 -- 172.21.15.90:0/3198446809 learned_addr learned my addr 172.21.15.90:0/3198446809 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:30.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.665+0000 7fe7f3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4109c90 0x7fe7f4074fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44636/0 (socket says 172.21.15.90:44636) 2024-05-27T07:57:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.665+0000 7fe7f9a00640 1 -- 172.21.15.90:0/3198446809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4109c90 msgr2=0x7fe7f4074fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.665+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4109c90 0x7fe7f4074fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.665+0000 7fe7f9a00640 1 -- 172.21.15.90:0/3198446809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7f41043e0 msgr2=0x7fe7f4076b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.665+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7f41043e0 0x7fe7f4076b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.666+0000 7fe7f9a00640 1 -- 172.21.15.90:0/3198446809 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f4075db0 con 0x7fe7f410a130 2024-05-27T07:57:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.666+0000 7fe7f9000640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7f41043e0 0x7fe7f4076b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.666+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7f410a130 0x7fe7f40755c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00eaa0 tx=0x7fe7ec00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.667+0000 7fe7f1600640 1 -- 172.21.15.90:0/3198446809 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec00cdb0 con 0x7fe7f410a130 2024-05-27T07:57:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.667+0000 7fe7f1600640 1 -- 172.21.15.90:0/3198446809 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7ec005020 con 0x7fe7f410a130 2024-05-27T07:57:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.667+0000 7fe7f1600640 1 -- 172.21.15.90:0/3198446809 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0116d0 con 0x7fe7f410a130 2024-05-27T07:57:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.667+0000 7fe7fba00640 1 -- 172.21.15.90:0/3198446809 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f419eab0 con 0x7fe7f410a130 2024-05-27T07:57:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.668+0000 7fe7fba00640 1 -- 172.21.15.90:0/3198446809 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f419efb0 con 0x7fe7f410a130 2024-05-27T07:57:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.675+0000 7fe7f1600640 1 -- 172.21.15.90:0/3198446809 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe7ec0051d0 con 0x7fe7f410a130 2024-05-27T07:57:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.678+0000 7fe7f1600640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7dc0b29d0 0x7fe7dc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.678+0000 7fe7f9000640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7dc0b29d0 0x7fe7dc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.679+0000 7fe7f1600640 1 -- 172.21.15.90:0/3198446809 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7fe7ec0e2ab0 con 0x7fe7f410a130 2024-05-27T07:57:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.679+0000 7fe7f9000640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7dc0b29d0 0x7fe7dc0b4ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005940 tx=0x7fe7e80058b0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.680+0000 7fe8029d0740 1 Processor -- start 2024-05-27T07:57:30.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.680+0000 7fe8029d0740 1 -- start start 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.681+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106ccb41b0 0x56106d056860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.681+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106cc8b2f0 0x56106d056da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.681+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc934d0 0x56106d0572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.681+0000 7fe8029d0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56106cfc7b90 con 0x56106cc934d0 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.681+0000 7fe8029d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56106cfb7730 con 0x56106cc8b2f0 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.681+0000 7fe8029d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56106cfbb770 con 0x56106ccb41b0 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.682+0000 7fe7f9000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106ccb41b0 0x56106d056860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.682+0000 7fe7f9000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106ccb41b0 0x56106d056860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44642/0 (socket says 172.21.15.90:44642) 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.682+0000 7fe7f9000640 1 -- 172.21.15.90:0/2823867462 learned_addr learned my addr 172.21.15.90:0/2823867462 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.682+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/2823867462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc934d0 0x56106d0572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.683+0000 7fe7f9000640 1 -- 172.21.15.90:0/2823867462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106cc8b2f0 msgr2=0x56106d056da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:57:30.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.683+0000 7fe7f9000640 1 --2- 172.21.15.90:0/2823867462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106cc8b2f0 0x56106d056da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.683+0000 7fe7f9000640 1 -- 172.21.15.90:0/2823867462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc934d0 msgr2=0x56106d0572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.683+0000 7fe7f9000640 1 --2- 172.21.15.90:0/2823867462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc934d0 0x56106d0572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.683+0000 7fe7f9000640 1 -- 172.21.15.90:0/2823867462 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106cf508b0 con 0x56106ccb41b0 2024-05-27T07:57:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.683+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/2823867462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc934d0 0x56106d0572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.684+0000 7fe7f9000640 1 --2- 172.21.15.90:0/2823867462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106ccb41b0 0x56106d056860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8000c00 tx=0x7fe7e803d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=630415f1b66a9a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.686+0000 7fe7e6000640 1 -- 172.21.15.90:0/2823867462 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8037040 con 0x56106ccb41b0 2024-05-27T07:57:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.686+0000 7fe7e6000640 1 -- 172.21.15.90:0/2823867462 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7e8044070 con 0x56106ccb41b0 2024-05-27T07:57:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.686+0000 7fe7e6000640 1 -- 172.21.15.90:0/2823867462 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8037040 con 0x56106ccb41b0 2024-05-27T07:57:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.688+0000 7fe8029d0740 1 -- 172.21.15.90:0/2823867462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106ccb41b0 msgr2=0x56106d056860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.688+0000 7fe8029d0740 1 --2- 172.21.15.90:0/2823867462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106ccb41b0 0x56106d056860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8000c00 tx=0x7fe7e803d000 comp rx=0 tx=0).stop 2024-05-27T07:57:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.689+0000 7fe8029d0740 1 -- 172.21.15.90:0/2823867462 shutdown_connections 2024-05-27T07:57:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.689+0000 7fe8029d0740 1 --2- 172.21.15.90:0/2823867462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106cc8b2f0 0x56106d056da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.689+0000 7fe8029d0740 1 --2- 172.21.15.90:0/2823867462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc934d0 0x56106d0572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.689+0000 7fe8029d0740 1 --2- 172.21.15.90:0/2823867462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106ccb41b0 0x56106d056860 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.689+0000 7fe8029d0740 1 -- 172.21.15.90:0/2823867462 >> 172.21.15.90:0/2823867462 conn(0x56106cca8300 msgr2=0x56106d053010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.689+0000 7fe8029d0740 1 -- 172.21.15.90:0/2823867462 shutdown_connections 2024-05-27T07:57:30.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.690+0000 7fe8029d0740 1 -- 172.21.15.90:0/2823867462 wait complete. 2024-05-27T07:57:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.691+0000 7fe8029d0740 1 -- start start 2024-05-27T07:57:30.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.691+0000 7fe8029d0740 1 Processor -- start 2024-05-27T07:57:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.696+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 0x56106d060a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.696+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d060fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.696+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.697+0000 7fe7f3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d060fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.697+0000 7fe7f3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d060fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44644/0 (socket says 172.21.15.90:44644) 2024-05-27T07:57:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.697+0000 7fe7f3e00640 1 -- 172.21.15.90:0/4138091072 learned_addr learned my addr 172.21.15.90:0/4138091072 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.697+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.698+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56106cc9d790 con 0x56106cc8b2f0 2024-05-27T07:57:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.698+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56106cf5fbc0 con 0x56106ccb41b0 2024-05-27T07:57:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.699+0000 7fe7f9000640 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 0x56106d060a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.699+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56106d063980 con 0x56106cc934d0 2024-05-27T07:57:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.700+0000 7fe8029d0740 10 client.0 _ll_register_callbacks cb 0x56106cf3d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.700+0000 7fe7f9a00640 1 -- 172.21.15.90:0/4138091072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 msgr2=0x56106d060fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.700+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d060fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.700+0000 7fe7f9a00640 1 -- 172.21.15.90:0/4138091072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 msgr2=0x56106d060a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.700+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 0x56106d060a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.700+0000 7fe7f9a00640 1 -- 172.21.15.90:0/4138091072 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106cccc210 con 0x56106ccb41b0 2024-05-27T07:57:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.700+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e8044070 con 0x56106ccb41b0 2024-05-27T07:57:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.701+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d06ccb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0dfe30 tx=0x7fe7ec0a9650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=214aa09c4fc2e68a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:30.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.701+0000 7fe7daa00640 10 client.37025 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:57:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.701+0000 7fe7daa00640 1 -- 172.21.15.90:0/4138091072 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec01f040 con 0x56106ccb41b0 2024-05-27T07:57:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.701+0000 7fe7daa00640 1 -- 172.21.15.90:0/4138091072 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7fe7ec0a8cd0 con 0x56106ccb41b0 2024-05-27T07:57:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.702+0000 7fe7daa00640 1 -- 172.21.15.90:0/4138091072 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe7ec0a8ef0 con 0x56106ccb41b0 2024-05-27T07:57:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.702+0000 7fe8029d0740 10 client.37025 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.702+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({fsmap=64}) -- 0x56106cd98620 con 0x56106ccb41b0 2024-05-27T07:57:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.702+0000 7fe7daa00640 1 -- 172.21.15.90:0/4138091072 <== mon.2 v2:172.21.15.129:3300/0 4 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe7ec00f040 con 0x56106ccb41b0 2024-05-27T07:57:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.703+0000 7fe8029d0740 10 client.37025 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.703+0000 7fe8029d0740 10 client.37025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.703+0000 7fe8029d0740 20 client.37025 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.703+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56106cc72d80 0x56106d072c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.703+0000 7fe8029d0740 4 client.37025 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.703+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56106cfc9ae0 con 0x56106cc72d80 2024-05-27T07:57:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.703+0000 7fe7f9000640 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56106cc72d80 0x56106d072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.704+0000 7fe7f9000640 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56106cc72d80 0x56106d072c70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2973b1d72f731af7 server_cookie=3912b59692553ed8 in_seq=0 out_seq=0 2024-05-27T07:57:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.704+0000 7fe7daa00640 10 client.37025 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.706+0000 7fe7daa00640 1 -- 172.21.15.90:0/4138091072 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56106cfc9ae0 con 0x56106cc72d80 2024-05-27T07:57:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.706+0000 7fe7daa00640 10 client.37025 handle_command_reply: tid=0 2024-05-27T07:57:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.706+0000 7fe8029d0740 1 client.37025 shutdown 2024-05-27T07:57:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 msgr2=0x56106d06ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d06ccb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0dfe30 tx=0x7fe7ec0a9650 comp rx=0 tx=0).stop 2024-05-27T07:57:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 shutdown_connections 2024-05-27T07:57:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56106cc72d80 0x56106d072c70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d06ccb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d060fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4138091072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 0x56106d060a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 >> 172.21.15.90:0/4138091072 conn(0x56106cca8300 msgr2=0x56106d056690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe7daa00640 0 client.37025 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 shutdown_connections 2024-05-27T07:57:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.707+0000 7fe8029d0740 1 -- 172.21.15.90:0/4138091072 wait complete. 2024-05-27T07:57:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.708+0000 7fe8029d0740 20 client.37025 trim_cache size 0 max 16384 2024-05-27T07:57:30.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.748+0000 7fe8029d0740 1 Processor -- start 2024-05-27T07:57:30.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.748+0000 7fe8029d0740 1 -- start start 2024-05-27T07:57:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.749+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 0x56106d05c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.749+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d052330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.749+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.749+0000 7fe8029d0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41c1c70 con 0x56106cc8b2f0 2024-05-27T07:57:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.749+0000 7fe8029d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f4076530 con 0x56106ccb41b0 2024-05-27T07:57:30.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.749+0000 7fe8029d0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41c0ed0 con 0x56106cc934d0 2024-05-27T07:57:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.750+0000 7fe7f9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 0x56106d05c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.750+0000 7fe7f9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 0x56106d05c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38726/0 (socket says 172.21.15.90:38726) 2024-05-27T07:57:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.750+0000 7fe7f9000640 1 -- 172.21.15.90:0/3144204842 learned_addr learned my addr 172.21.15.90:0/3144204842 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.750+0000 7fe7f3e00640 1 --2- 172.21.15.90:0/3144204842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d052330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.750+0000 7fe7f9000640 1 -- 172.21.15.90:0/3144204842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 msgr2=0x56106d052330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.750+0000 7fe7f9000640 1 --2- 172.21.15.90:0/3144204842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d052330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.750+0000 7fe7f9000640 1 -- 172.21.15.90:0/3144204842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 msgr2=0x56106d06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.750+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/3144204842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.751+0000 7fe7f9000640 1 --2- 172.21.15.90:0/3144204842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d06ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.751+0000 7fe7f9000640 1 -- 172.21.15.90:0/3144204842 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106d04d050 con 0x56106cc8b2f0 2024-05-27T07:57:30.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.751+0000 7fe7f9000640 1 --2- 172.21.15.90:0/3144204842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 0x56106d05c280 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x56106d0539d0 tx=0x7fe7e8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf867ebf2fc8f08 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.752+0000 7fe7e5600640 1 -- 172.21.15.90:0/3144204842 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e805b040 con 0x56106cc8b2f0 2024-05-27T07:57:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.752+0000 7fe7e5600640 1 -- 172.21.15.90:0/3144204842 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7e8030400 con 0x56106cc8b2f0 2024-05-27T07:57:30.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.752+0000 7fe7e5600640 1 -- 172.21.15.90:0/3144204842 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8003970 con 0x56106cc8b2f0 2024-05-27T07:57:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.753+0000 7fe8029d0740 1 -- 172.21.15.90:0/3144204842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 msgr2=0x56106d05c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:30.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.753+0000 7fe8029d0740 1 --2- 172.21.15.90:0/3144204842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 0x56106d05c280 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x56106d0539d0 tx=0x7fe7e8005780 comp rx=0 tx=0).stop 2024-05-27T07:57:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.754+0000 7fe8029d0740 1 -- 172.21.15.90:0/3144204842 shutdown_connections 2024-05-27T07:57:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.754+0000 7fe8029d0740 1 --2- 172.21.15.90:0/3144204842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d06ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.754+0000 7fe8029d0740 1 --2- 172.21.15.90:0/3144204842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d052330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.754+0000 7fe8029d0740 1 --2- 172.21.15.90:0/3144204842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc8b2f0 0x56106d05c280 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.754+0000 7fe8029d0740 1 -- 172.21.15.90:0/3144204842 >> 172.21.15.90:0/3144204842 conn(0x56106cca8300 msgr2=0x56106d062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.754+0000 7fe8029d0740 1 -- 172.21.15.90:0/3144204842 shutdown_connections 2024-05-27T07:57:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.754+0000 7fe8029d0740 1 -- 172.21.15.90:0/3144204842 wait complete. 2024-05-27T07:57:30.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.755+0000 7fe8029d0740 1 -- start start 2024-05-27T07:57:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.755+0000 7fe8029d0740 1 Processor -- start 2024-05-27T07:57:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.756+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d068e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.756+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d0693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.756+0000 7fe8029d0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc481c0 0x56106d064290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.756+0000 7fe8029d0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41c0800 con 0x56106cc481c0 2024-05-27T07:57:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.756+0000 7fe7f9a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc481c0 0x56106d064290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.757+0000 7fe7f9a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc481c0 0x56106d064290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38728/0 (socket says 172.21.15.90:38728) 2024-05-27T07:57:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.757+0000 7fe7f9a00640 1 -- 172.21.15.90:0/4188634943 learned_addr learned my addr 172.21.15.90:0/4188634943 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.757+0000 7fe8029d0740 1 -- 172.21.15.90:0/4188634943 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41bf390 con 0x56106ccb41b0 2024-05-27T07:57:30.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.757+0000 7fe8029d0740 1 -- 172.21.15.90:0/4188634943 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f41bfa60 con 0x56106cc934d0 2024-05-27T07:57:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.757+0000 7fe8029d0740 10 client.0 _ll_register_callbacks cb 0x7fe7ec0a9580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.757+0000 7fe7f9000640 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.757+0000 7fe7f3e00640 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d0693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.759+0000 7fe7f9a00640 1 -- 172.21.15.90:0/4188634943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 msgr2=0x56106d068e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.759+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d068e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.759+0000 7fe7f9a00640 1 -- 172.21.15.90:0/4188634943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 msgr2=0x56106d0693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:30.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.759+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d0693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.759+0000 7fe7f9a00640 1 -- 172.21.15.90:0/4188634943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106d064a20 con 0x56106cc481c0 2024-05-27T07:57:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.760+0000 7fe7f9000640 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d068e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.760+0000 7fe7f3e00640 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d0693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.761+0000 7fe7f9a00640 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc481c0 0x56106d064290 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0a9a00 tx=0x7fe7ec013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b48308533f75f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.761+0000 7fe7daa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:57:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.761+0000 7fe7daa00640 1 -- 172.21.15.90:0/4188634943 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec01f040 con 0x56106cc481c0 2024-05-27T07:57:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.761+0000 7fe7daa00640 1 -- 172.21.15.90:0/4188634943 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7fe7ec0a55f0 con 0x56106cc481c0 2024-05-27T07:57:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.761+0000 7fe8029d0740 1 -- 172.21.15.90:0/4188634943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e8030400 con 0x56106cc481c0 2024-05-27T07:57:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.762+0000 7fe7daa00640 1 -- 172.21.15.90:0/4188634943 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe7ec0b2160 con 0x56106cc481c0 2024-05-27T07:57:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.762+0000 7fe8029d0740 10 client.17442 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:30.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.762+0000 7fe8029d0740 1 -- 172.21.15.90:0/4188634943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56106d0dad60 con 0x56106cc481c0 2024-05-27T07:57:30.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.763+0000 7fe7daa00640 1 -- 172.21.15.90:0/4188634943 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7e8030400 con 0x56106cc481c0 2024-05-27T07:57:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.765+0000 7fe7daa00640 1 -- 172.21.15.90:0/4188634943 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe7ec0a8740 con 0x56106cc481c0 2024-05-27T07:57:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.765+0000 7fe8029d0740 10 client.17442 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.765+0000 7fe8029d0740 10 client.17442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.765+0000 7fe8029d0740 20 client.17442 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.765+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56106d0db1f0 0x56106d0dd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.765+0000 7fe8029d0740 4 client.17442 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:57:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.765+0000 7fe8029d0740 1 -- 172.21.15.90:0/4188634943 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56106cfc32c0 con 0x56106d0db1f0 2024-05-27T07:57:30.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.765+0000 7fe7f9000640 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56106d0db1f0 0x56106d0dd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.766+0000 7fe7daa00640 10 client.17442 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:30.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.766+0000 7fe7f9000640 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56106d0db1f0 0x56106d0dd610 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40424213faf85ca5 server_cookie=3d4a256d28969c52 in_seq=0 out_seq=0 2024-05-27T07:57:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.795+0000 7fe7daa00640 1 -- 172.21.15.90:0/4188634943 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56106cfc32c0 con 0x56106d0db1f0 2024-05-27T07:57:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.795+0000 7fe7daa00640 10 client.17442 handle_command_reply: tid=0 2024-05-27T07:57:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.795+0000 7fe8029d0740 1 client.17442 shutdown 2024-05-27T07:57:30.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 1 -- 172.21.15.90:0/4188634943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc481c0 msgr2=0x56106d064290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc481c0 0x56106d064290 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0a9a00 tx=0x7fe7ec013040 comp rx=0 tx=0).stop 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 1 -- 172.21.15.90:0/4188634943 shutdown_connections 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56106d0db1f0 0x56106d0dd610 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56106ccb41b0 0x56106d0693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56106cc481c0 0x56106d064290 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 1 --2- 172.21.15.90:0/4188634943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56106cc934d0 0x56106d068e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe7daa00640 0 client.17442 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 1 -- 172.21.15.90:0/4188634943 >> 172.21.15.90:0/4188634943 conn(0x56106cca8300 msgr2=0x56106d05ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 1 -- 172.21.15.90:0/4188634943 shutdown_connections 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 1 -- 172.21.15.90:0/4188634943 wait complete. 2024-05-27T07:57:30.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.796+0000 7fe8029d0740 20 client.17442 trim_cache size 0 max 16384 2024-05-27T07:57:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:57:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:57:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:57:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 -- 172.21.15.90:0/3198446809 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7dc0b29d0 msgr2=0x7fe7dc0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7dc0b29d0 0x7fe7dc0b4ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005940 tx=0x7fe7e80058b0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 -- 172.21.15.90:0/3198446809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7f410a130 msgr2=0x7fe7f40755c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7f410a130 0x7fe7f40755c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00eaa0 tx=0x7fe7ec00ef60 comp rx=0 tx=0).stop 2024-05-27T07:57:30.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 -- 172.21.15.90:0/3198446809 shutdown_connections 2024-05-27T07:57:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe7dc0b29d0 0x7fe7dc0b4ee0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7f410a130 0x7fe7f40755c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f4109c90 0x7fe7f4074fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 --2- 172.21.15.90:0/3198446809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7f41043e0 0x7fe7f4076b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 -- 172.21.15.90:0/3198446809 >> 172.21.15.90:0/3198446809 conn(0x7fe7f4079390 msgr2=0x7fe7f41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 -- 172.21.15.90:0/3198446809 shutdown_connections 2024-05-27T07:57:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:30.797+0000 7fe7fba00640 1 -- 172.21.15.90:0/3198446809 wait complete. 2024-05-27T07:57:31.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:31 smithi090 ceph-mon[31429]: pgmap v353: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.5 MiB/s wr, 491 op/s 2024-05-27T07:57:32.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:31 smithi136 ceph-mon[35324]: pgmap v353: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.5 MiB/s wr, 491 op/s 2024-05-27T07:57:32.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:31 smithi129 ceph-mon[35254]: pgmap v353: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.5 MiB/s wr, 491 op/s 2024-05-27T07:57:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 435.0 s, 634.2 tps, lat 157.224 ms stddev 71.864 2024-05-27T07:57:33.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:33 smithi090 ceph-mon[31429]: pgmap v354: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.0 MiB/s wr, 452 op/s 2024-05-27T07:57:34.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:33 smithi136 ceph-mon[35324]: pgmap v354: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.0 MiB/s wr, 452 op/s 2024-05-27T07:57:34.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:33 smithi129 ceph-mon[35254]: pgmap v354: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.0 MiB/s wr, 452 op/s 2024-05-27T07:57:35.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:35 smithi090 ceph-mon[31429]: pgmap v355: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.8 MiB/s wr, 647 op/s 2024-05-27T07:57:36.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:35 smithi136 ceph-mon[35324]: pgmap v355: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.8 MiB/s wr, 647 op/s 2024-05-27T07:57:36.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:35 smithi129 ceph-mon[35254]: pgmap v355: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.8 MiB/s wr, 647 op/s 2024-05-27T07:57:36.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.078+0000 7fb502400700 1 -- 172.21.15.90:0/3732957088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4fc0775b0 msgr2=0x7fb4fc114e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:36.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.078+0000 7fb502400700 1 --2- 172.21.15.90:0/3732957088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4fc0775b0 0x7fb4fc114e00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009990 tx=0x7fb4e8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.079+0000 7fb502400700 1 -- 172.21.15.90:0/3732957088 shutdown_connections 2024-05-27T07:57:36.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.079+0000 7fb502400700 1 --2- 172.21.15.90:0/3732957088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc115340 0x7fb4fc117760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.079+0000 7fb502400700 1 --2- 172.21.15.90:0/3732957088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4fc076c60 0x7fb4fc077070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.079+0000 7fb502400700 1 --2- 172.21.15.90:0/3732957088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4fc0775b0 0x7fb4fc114e00 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.079+0000 7fb502400700 1 -- 172.21.15.90:0/3732957088 >> 172.21.15.90:0/3732957088 conn(0x7fb4fc0ffe90 msgr2=0x7fb4fc1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:36.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.080+0000 7fb502400700 1 -- 172.21.15.90:0/3732957088 shutdown_connections 2024-05-27T07:57:36.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.080+0000 7fb502400700 1 -- 172.21.15.90:0/3732957088 wait complete. 2024-05-27T07:57:36.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.082+0000 7fb502400700 1 Processor -- start 2024-05-27T07:57:36.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.083+0000 7fb502400700 1 -- start start 2024-05-27T07:57:36.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.084+0000 7fb502400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4fc076c60 0x7fb4fc1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.084+0000 7fb502400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4fc0775b0 0x7fb4fc1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.084+0000 7fb502400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc115340 0x7fb4fc1a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.084+0000 7fb4faa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4fc0775b0 0x7fb4fc1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.084+0000 7fb4faa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4fc0775b0 0x7fb4fc1a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:35994/0 (socket says 172.21.15.90:35994) 2024-05-27T07:57:36.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.085+0000 7fb502400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc11a970 con 0x7fb4fc076c60 2024-05-27T07:57:36.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.085+0000 7fb502400700 1 -- 172.21.15.90:0/1470685006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc11a7f0 con 0x7fb4fc0775b0 2024-05-27T07:57:36.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.085+0000 7fb502400700 1 -- 172.21.15.90:0/1470685006 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4fc11aaf0 con 0x7fb4fc115340 2024-05-27T07:57:36.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.085+0000 7fb4fb400700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4fc076c60 0x7fb4fc1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.085+0000 7fb4faa00700 1 -- 172.21.15.90:0/1470685006 learned_addr learned my addr 172.21.15.90:0/1470685006 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:36.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.085+0000 7fb4fbe00700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc115340 0x7fb4fc1a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.086+0000 7fb4fbe00700 1 -- 172.21.15.90:0/1470685006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4fc0775b0 msgr2=0x7fb4fc1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:36.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.086+0000 7fb4fbe00700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4fc0775b0 0x7fb4fc1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.086+0000 7fb4fbe00700 1 -- 172.21.15.90:0/1470685006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4fc076c60 msgr2=0x7fb4fc1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:36.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.086+0000 7fb4fbe00700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4fc076c60 0x7fb4fc1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.086+0000 7fb4fbe00700 1 -- 172.21.15.90:0/1470685006 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4fc1a6290 con 0x7fb4fc115340 2024-05-27T07:57:36.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.086+0000 7fb4faa00700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4fc0775b0 0x7fb4fc1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:36.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.087+0000 7fb4fbe00700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc115340 0x7fb4fc1a5bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4007f10 tx=0x7fb4f400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:36.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.089+0000 7fb4f3e00700 1 -- 172.21.15.90:0/1470685006 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4004db0 con 0x7fb4fc115340 2024-05-27T07:57:36.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.090+0000 7fb4f3e00700 1 -- 172.21.15.90:0/1470685006 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4f4003a00 con 0x7fb4fc115340 2024-05-27T07:57:36.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.092+0000 7fb502400700 1 -- 172.21.15.90:0/1470685006 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4fc1a6580 con 0x7fb4fc115340 2024-05-27T07:57:36.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.092+0000 7fb502400700 1 -- 172.21.15.90:0/1470685006 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4fc1adea0 con 0x7fb4fc115340 2024-05-27T07:57:36.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.092+0000 7fb4f3e00700 1 -- 172.21.15.90:0/1470685006 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4004420 con 0x7fb4fc115340 2024-05-27T07:57:36.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.099+0000 7fb4f3e00700 1 -- 172.21.15.90:0/1470685006 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb4f401c020 con 0x7fb4fc115340 2024-05-27T07:57:36.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.103+0000 7fb4f3e00700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb4e00b2a70 0x7fb4e00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.103+0000 7fb4f3e00700 1 -- 172.21.15.90:0/1470685006 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7fb4f40e2030 con 0x7fb4fc115340 2024-05-27T07:57:36.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.103+0000 7fb4fb400700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb4e00b2a70 0x7fb4e00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.108+0000 7fb4fb400700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb4e00b2a70 0x7fb4e00b4f80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009a10 tx=0x7fb4ec009190 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:36.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.122+0000 7fb510f1bb80 1 Processor -- start 2024-05-27T07:57:36.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.122+0000 7fb510f1bb80 1 -- start start 2024-05-27T07:57:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.123+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 0x55d4d5222330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.123+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d5222870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.123+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d522f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.123+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d4d5150e40 con 0x55d4d518f9f0 2024-05-27T07:57:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.123+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d4d51409e0 con 0x55d4d5222e70 2024-05-27T07:57:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.123+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d4d5144a20 con 0x55d4d50a3e40 2024-05-27T07:57:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.124+0000 7fb4fbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d522f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.124+0000 7fb4fbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d522f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:36002/0 (socket says 172.21.15.90:36002) 2024-05-27T07:57:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.124+0000 7fb4fbe00700 1 -- 172.21.15.90:0/832942358 learned_addr learned my addr 172.21.15.90:0/832942358 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:36.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.125+0000 7fb4fb400700 1 --2- 172.21.15.90:0/832942358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 0x55d4d5222330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.125+0000 7fb4faa00700 1 --2- 172.21.15.90:0/832942358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d5222870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.126+0000 7fb4fb400700 1 -- 172.21.15.90:0/832942358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 msgr2=0x55d4d5222870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:36.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.126+0000 7fb4fb400700 1 --2- 172.21.15.90:0/832942358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d5222870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.126+0000 7fb4fb400700 1 -- 172.21.15.90:0/832942358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 msgr2=0x55d4d522f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:36.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.126+0000 7fb4fb400700 1 --2- 172.21.15.90:0/832942358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d522f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.127+0000 7fb4fb400700 1 -- 172.21.15.90:0/832942358 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d5109420 con 0x55d4d518f9f0 2024-05-27T07:57:36.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.127+0000 7fb4fbe00700 1 --2- 172.21.15.90:0/832942358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d522f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:36.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.127+0000 7fb4fb400700 1 --2- 172.21.15.90:0/832942358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 0x55d4d5222330 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00e9e0 tx=0x7fb4ec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28296f17cfd3cfc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:36.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.128+0000 7fb4f0c00700 1 -- 172.21.15.90:0/832942358 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec01b040 con 0x55d4d518f9f0 2024-05-27T07:57:36.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.129+0000 7fb4f0c00700 1 -- 172.21.15.90:0/832942358 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4ec024070 con 0x55d4d518f9f0 2024-05-27T07:57:36.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.129+0000 7fb4f0c00700 1 -- 172.21.15.90:0/832942358 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0136b0 con 0x55d4d518f9f0 2024-05-27T07:57:36.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.130+0000 7fb510f1bb80 1 -- 172.21.15.90:0/832942358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 msgr2=0x55d4d5222330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:36.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.130+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/832942358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 0x55d4d5222330 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00e9e0 tx=0x7fb4ec011000 comp rx=0 tx=0).stop 2024-05-27T07:57:36.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.131+0000 7fb510f1bb80 1 -- 172.21.15.90:0/832942358 shutdown_connections 2024-05-27T07:57:36.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.131+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/832942358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d522f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.131+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/832942358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 0x55d4d5222330 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.131+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/832942358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d5222870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.131+0000 7fb510f1bb80 1 -- 172.21.15.90:0/832942358 >> 172.21.15.90:0/832942358 conn(0x55d4d513d390 msgr2=0x55d4d521e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:36.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.131+0000 7fb510f1bb80 1 -- 172.21.15.90:0/832942358 shutdown_connections 2024-05-27T07:57:36.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.131+0000 7fb510f1bb80 1 -- 172.21.15.90:0/832942358 wait complete. 2024-05-27T07:57:36.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.132+0000 7fb510f1bb80 1 -- start start 2024-05-27T07:57:36.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.133+0000 7fb510f1bb80 1 Processor -- start 2024-05-27T07:57:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.136+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d522c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.137+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d518f9f0 0x55d4d522ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.137+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d5222e70 0x55d4d522cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.137+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d4d5046fa0 con 0x55d4d5222e70 2024-05-27T07:57:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.137+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d4d50e1760 con 0x55d4d518f9f0 2024-05-27T07:57:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.137+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d4d522fc20 con 0x55d4d50a3e40 2024-05-27T07:57:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.138+0000 7fb510f1bb80 10 client.0 _ll_register_callbacks cb 0x55d4d50eac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.138+0000 7fb4fb400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d522c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.137+0000 7fb4faa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d518f9f0 0x55d4d522ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.139+0000 7fb4fb400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d522c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44698/0 (socket says 172.21.15.90:44698) 2024-05-27T07:57:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.139+0000 7fb4fb400700 1 -- 172.21.15.90:0/2165638693 learned_addr learned my addr 172.21.15.90:0/2165638693 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:36.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.139+0000 7fb4fbe00700 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d5222e70 0x55d4d522cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.140+0000 7fb4fbe00700 1 -- 172.21.15.90:0/2165638693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 msgr2=0x55d4d522c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:36.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.140+0000 7fb4fbe00700 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d522c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.140+0000 7fb4fbe00700 1 -- 172.21.15.90:0/2165638693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d518f9f0 msgr2=0x55d4d522ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:36.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.141+0000 7fb4fbe00700 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d518f9f0 0x55d4d522ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.141+0000 7fb4fbe00700 1 -- 172.21.15.90:0/2165638693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4d50908d0 con 0x55d4d5222e70 2024-05-27T07:57:36.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.141+0000 7fb4faa00700 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d518f9f0 0x55d4d522ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:36.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.142+0000 7fb4fbe00700 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d5222e70 0x55d4d522cf70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4007ee0 tx=0x7fb4f40a8bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce7049cd2267b57 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:36.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.142+0000 7fb4e5600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:57:36.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.142+0000 7fb4e5600700 1 -- 172.21.15.90:0/2165638693 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4019070 con 0x55d4d5222e70 2024-05-27T07:57:36.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.142+0000 7fb4e5600700 1 -- 172.21.15.90:0/2165638693 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7fb4f4003dc0 con 0x55d4d5222e70 2024-05-27T07:57:36.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.143+0000 7fb510f1bb80 1 -- 172.21.15.90:0/2165638693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4ec024070 con 0x55d4d5222e70 2024-05-27T07:57:36.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.143+0000 7fb4e5600700 1 -- 172.21.15.90:0/2165638693 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb4f40ad2f0 con 0x55d4d5222e70 2024-05-27T07:57:36.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.144+0000 7fb510f1bb80 10 client.17454 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:36.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.144+0000 7fb510f1bb80 1 -- 172.21.15.90:0/2165638693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55d4d5111060 con 0x55d4d5222e70 2024-05-27T07:57:36.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.144+0000 7fb4e5600700 1 -- 172.21.15.90:0/2165638693 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4ec024070 con 0x55d4d5222e70 2024-05-27T07:57:36.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.146+0000 7fb4e5600700 1 -- 172.21.15.90:0/2165638693 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fb4f40a8230 con 0x55d4d5222e70 2024-05-27T07:57:36.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.146+0000 7fb510f1bb80 10 client.17454 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:36.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.146+0000 7fb510f1bb80 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:36.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.146+0000 7fb510f1bb80 20 client.17454 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:36.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.146+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d4d523cd10 0x55d4d523f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.146+0000 7fb510f1bb80 4 client.17454 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:36.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.146+0000 7fb510f1bb80 1 -- 172.21.15.90:0/2165638693 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4d5152d90 con 0x55d4d523cd10 2024-05-27T07:57:36.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.147+0000 7fb4fb400700 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d4d523cd10 0x55d4d523f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.149+0000 7fb4fb400700 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d4d523cd10 0x55d4d523f180 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba57df0254878a8e server_cookie=8536b1bbfec4e888 in_seq=0 out_seq=0 2024-05-27T07:57:36.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.150+0000 7fb4e5600700 10 client.17454 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:36.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.151+0000 7fb4e5600700 1 -- 172.21.15.90:0/2165638693 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d4d5152d90 con 0x55d4d523cd10 2024-05-27T07:57:36.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.151+0000 7fb4e5600700 10 client.17454 handle_command_reply: tid=0 2024-05-27T07:57:36.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.151+0000 7fb510f1bb80 1 client.17454 shutdown 2024-05-27T07:57:36.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.152+0000 7fb510f1bb80 1 -- 172.21.15.90:0/2165638693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d5222e70 msgr2=0x55d4d522cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.152+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d5222e70 0x55d4d522cf70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4007ee0 tx=0x7fb4f40a8bb0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.153+0000 7fb510f1bb80 1 -- 172.21.15.90:0/2165638693 shutdown_connections 2024-05-27T07:57:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.153+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d4d523cd10 0x55d4d523f180 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.153+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d5222e70 0x55d4d522cf70 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.153+0000 7fb4e5600700 0 client.17454 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.153+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d522c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.153+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/2165638693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d518f9f0 0x55d4d522ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.153+0000 7fb510f1bb80 1 -- 172.21.15.90:0/2165638693 >> 172.21.15.90:0/2165638693 conn(0x55d4d513d390 msgr2=0x55d4d5222310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:36.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.153+0000 7fb510f1bb80 1 -- 172.21.15.90:0/2165638693 shutdown_connections 2024-05-27T07:57:36.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.154+0000 7fb510f1bb80 1 -- 172.21.15.90:0/2165638693 wait complete. 2024-05-27T07:57:36.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.154+0000 7fb510f1bb80 20 client.17454 trim_cache size 0 max 16384 2024-05-27T07:57:36.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.198+0000 7fb510f1bb80 1 Processor -- start 2024-05-27T07:57:36.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.198+0000 7fb510f1bb80 1 -- start start 2024-05-27T07:57:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.199+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d50a3e40 0x55d4d522b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.199+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d518f9f0 0x55d4d5227cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.199+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d52227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.199+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc0108a0 con 0x55d4d50a3e40 2024-05-27T07:57:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.199+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc00d8f0 con 0x55d4d5222e70 2024-05-27T07:57:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.199+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc00fb00 con 0x55d4d518f9f0 2024-05-27T07:57:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.200+0000 7fb4fbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d52227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.200+0000 7fb4fbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d52227c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:36010/0 (socket says 172.21.15.90:36010) 2024-05-27T07:57:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.200+0000 7fb4fbe00700 1 -- 172.21.15.90:0/4018124461 learned_addr learned my addr 172.21.15.90:0/4018124461 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.200+0000 7fb4fb400700 1 --2- 172.21.15.90:0/4018124461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d50a3e40 0x55d4d522b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.200+0000 7fb4faa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d518f9f0 0x55d4d5227cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.201+0000 7fb4fb400700 1 -- 172.21.15.90:0/4018124461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d518f9f0 msgr2=0x55d4d5227cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.202+0000 7fb4fb400700 1 --2- 172.21.15.90:0/4018124461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d518f9f0 0x55d4d5227cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.202+0000 7fb4fb400700 1 -- 172.21.15.90:0/4018124461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 msgr2=0x55d4d52227c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.202+0000 7fb4fb400700 1 --2- 172.21.15.90:0/4018124461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d52227c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.202+0000 7fb4fb400700 1 -- 172.21.15.90:0/4018124461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4d522aa60 con 0x55d4d50a3e40 2024-05-27T07:57:36.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.202+0000 7fb4fbe00700 1 --2- 172.21.15.90:0/4018124461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d52227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:36.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.203+0000 7fb4fb400700 1 --2- 172.21.15.90:0/4018124461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d50a3e40 0x55d4d522b450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0048a0 tx=0x7fb4ec015480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28577c8bee065126 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:36.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.203+0000 7fb4e7e00700 1 -- 172.21.15.90:0/4018124461 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec039070 con 0x55d4d50a3e40 2024-05-27T07:57:36.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.203+0000 7fb4e7e00700 1 -- 172.21.15.90:0/4018124461 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4ec024040 con 0x55d4d50a3e40 2024-05-27T07:57:36.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.203+0000 7fb4e7e00700 1 -- 172.21.15.90:0/4018124461 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec031070 con 0x55d4d50a3e40 2024-05-27T07:57:36.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.204+0000 7fb510f1bb80 1 -- 172.21.15.90:0/4018124461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d50a3e40 msgr2=0x55d4d522b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:36.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.204+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/4018124461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d50a3e40 0x55d4d522b450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0048a0 tx=0x7fb4ec015480 comp rx=0 tx=0).stop 2024-05-27T07:57:36.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.206+0000 7fb510f1bb80 1 -- 172.21.15.90:0/4018124461 shutdown_connections 2024-05-27T07:57:36.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.206+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/4018124461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d52227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.206+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/4018124461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d50a3e40 0x55d4d522b450 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.206+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/4018124461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d518f9f0 0x55d4d5227cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.206+0000 7fb510f1bb80 1 -- 172.21.15.90:0/4018124461 >> 172.21.15.90:0/4018124461 conn(0x55d4d513d390 msgr2=0x55d4d52220b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:36.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.207+0000 7fb510f1bb80 1 -- 172.21.15.90:0/4018124461 shutdown_connections 2024-05-27T07:57:36.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.207+0000 7fb510f1bb80 1 -- 172.21.15.90:0/4018124461 wait complete. 2024-05-27T07:57:36.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.207+0000 7fb510f1bb80 1 -- start start 2024-05-27T07:57:36.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.208+0000 7fb510f1bb80 1 Processor -- start 2024-05-27T07:57:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.209+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d5227c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.209+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 0x55d4d5226020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.209+0000 7fb510f1bb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d52265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.209+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc00f430 con 0x55d4d518f9f0 2024-05-27T07:57:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.209+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc00dfc0 con 0x55d4d5222e70 2024-05-27T07:57:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.209+0000 7fb510f1bb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc00e690 con 0x55d4d50a3e40 2024-05-27T07:57:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.209+0000 7fb510f1bb80 10 client.0 _ll_register_callbacks cb 0x55d4d50eac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.212+0000 7fb4fb400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d5227c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.212+0000 7fb4fb400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d5227c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44706/0 (socket says 172.21.15.90:44706) 2024-05-27T07:57:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.212+0000 7fb4fb400700 1 -- 172.21.15.90:0/3875761040 learned_addr learned my addr 172.21.15.90:0/3875761040 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.212+0000 7fb4faa00700 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 0x55d4d5226020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.212+0000 7fb4faa00700 1 -- 172.21.15.90:0/3875761040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 msgr2=0x55d4d5227c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.212+0000 7fb4faa00700 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d5227c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.212+0000 7fb4faa00700 1 -- 172.21.15.90:0/3875761040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 msgr2=0x55d4d52265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.212+0000 7fb4faa00700 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d52265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.213+0000 7fb4faa00700 1 -- 172.21.15.90:0/3875761040 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4d5226d60 con 0x55d4d518f9f0 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.213+0000 7fb4faa00700 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 0x55d4d5226020 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8006010 tx=0x7fb4e8005390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491de11aecd53467 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.213+0000 7fb4e5600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.213+0000 7fb4e5600700 1 -- 172.21.15.90:0/3875761040 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80039c0 con 0x55d4d518f9f0 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.214+0000 7fb4e5600700 1 -- 172.21.15.90:0/3875761040 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7fb4e8003b60 con 0x55d4d518f9f0 2024-05-27T07:57:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.214+0000 7fb510f1bb80 1 -- 172.21.15.90:0/3875761040 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4ec024040 con 0x55d4d518f9f0 2024-05-27T07:57:36.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.214+0000 7fb4e5600700 1 -- 172.21.15.90:0/3875761040 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb4e8033050 con 0x55d4d518f9f0 2024-05-27T07:57:36.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.215+0000 7fb510f1bb80 10 client.17460 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:36.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.215+0000 7fb510f1bb80 1 -- 172.21.15.90:0/3875761040 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55d4d5234e70 con 0x55d4d518f9f0 2024-05-27T07:57:36.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.215+0000 7fb4e5600700 1 -- 172.21.15.90:0/3875761040 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4ec024040 con 0x55d4d518f9f0 2024-05-27T07:57:36.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.215+0000 7fb4e5600700 1 -- 172.21.15.90:0/3875761040 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fb4e800f460 con 0x55d4d518f9f0 2024-05-27T07:57:36.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.215+0000 7fb510f1bb80 10 client.17460 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:36.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.215+0000 7fb510f1bb80 10 client.17460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:36.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.215+0000 7fb510f1bb80 20 client.17460 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:36.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.215+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d4d5235600 0x55d4d52ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:36.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.215+0000 7fb510f1bb80 4 client.17460 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:57:36.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.215+0000 7fb510f1bb80 1 -- 172.21.15.90:0/3875761040 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4d514c570 con 0x55d4d5235600 2024-05-27T07:57:36.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.216+0000 7fb4fb400700 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d4d5235600 0x55d4d52ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:36.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.218+0000 7fb4e5600700 10 client.17460 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:36.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.218+0000 7fb4fb400700 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d4d5235600 0x55d4d52ba870 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f13017ad549889 server_cookie=ef5602b23241a516 in_seq=0 out_seq=0 2024-05-27T07:57:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.220+0000 7fb4e5600700 1 -- 172.21.15.90:0/3875761040 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4d514c570 con 0x55d4d5235600 2024-05-27T07:57:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.221+0000 7fb4e5600700 10 client.17460 handle_command_reply: tid=0 2024-05-27T07:57:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.221+0000 7fb510f1bb80 1 client.17460 shutdown 2024-05-27T07:57:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.221+0000 7fb510f1bb80 1 -- 172.21.15.90:0/3875761040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 msgr2=0x55d4d5226020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:36.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.221+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 0x55d4d5226020 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8006010 tx=0x7fb4e8005390 comp rx=0 tx=0).stop 2024-05-27T07:57:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.222+0000 7fb510f1bb80 1 -- 172.21.15.90:0/3875761040 shutdown_connections 2024-05-27T07:57:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.222+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d4d5235600 0x55d4d52ba870 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.222+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4d5222e70 0x55d4d52265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.222+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d4d50a3e40 0x55d4d5227c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.222+0000 7fb510f1bb80 1 --2- 172.21.15.90:0/3875761040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d4d518f9f0 0x55d4d5226020 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.222+0000 7fb4e5600700 0 client.17460 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.222+0000 7fb510f1bb80 1 -- 172.21.15.90:0/3875761040 >> 172.21.15.90:0/3875761040 conn(0x55d4d513d390 msgr2=0x55d4d5223ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.223+0000 7fb510f1bb80 1 -- 172.21.15.90:0/3875761040 shutdown_connections 2024-05-27T07:57:36.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.224+0000 7fb510f1bb80 1 -- 172.21.15.90:0/3875761040 wait complete. 2024-05-27T07:57:36.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.224+0000 7fb510f1bb80 20 client.17460 trim_cache size 0 max 16384 2024-05-27T07:57:36.228 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:57:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.225+0000 7fb502400700 1 -- 172.21.15.90:0/1470685006 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb4e00b2a70 msgr2=0x7fb4e00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.225+0000 7fb502400700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb4e00b2a70 0x7fb4e00b4f80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009a10 tx=0x7fb4ec009190 comp rx=0 tx=0).stop 2024-05-27T07:57:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.225+0000 7fb502400700 1 -- 172.21.15.90:0/1470685006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc115340 msgr2=0x7fb4fc1a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:36.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.225+0000 7fb502400700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc115340 0x7fb4fc1a5bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4007f10 tx=0x7fb4f400d3b0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.226+0000 7fb502400700 1 -- 172.21.15.90:0/1470685006 shutdown_connections 2024-05-27T07:57:36.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.226+0000 7fb502400700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4fc115340 0x7fb4fc1a5bf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.226+0000 7fb502400700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4fc076c60 0x7fb4fc1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.226+0000 7fb502400700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb4e00b2a70 0x7fb4e00b4f80 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009a10 tx=0x7fb4ec009190 comp rx=0 tx=0).stop 2024-05-27T07:57:36.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.226+0000 7fb502400700 1 --2- 172.21.15.90:0/1470685006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4fc0775b0 0x7fb4fc1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:36.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.226+0000 7fb502400700 1 -- 172.21.15.90:0/1470685006 >> 172.21.15.90:0/1470685006 conn(0x7fb4fc0ffe90 msgr2=0x7fb4fc100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:36.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.227+0000 7fb502400700 1 -- 172.21.15.90:0/1470685006 shutdown_connections 2024-05-27T07:57:36.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:36.227+0000 7fb502400700 1 -- 172.21.15.90:0/1470685006 wait complete. 2024-05-27T07:57:38.010 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:37 smithi136 ceph-mon[35324]: pgmap v356: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.8 MiB/s wr, 545 op/s 2024-05-27T07:57:38.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:37 smithi129 ceph-mon[35254]: pgmap v356: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.8 MiB/s wr, 545 op/s 2024-05-27T07:57:38.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:37 smithi090 ceph-mon[31429]: pgmap v356: 129 pgs: 129 active+clean; 2.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.8 MiB/s wr, 545 op/s 2024-05-27T07:57:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 440.0 s, 461.6 tps, lat 214.574 ms stddev 145.093 2024-05-27T07:57:39.452 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:57:39.453 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:57:39.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:39 smithi090 ceph-mon[31429]: pgmap v357: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.8 MiB/s wr, 553 op/s 2024-05-27T07:57:40.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:39 smithi136 ceph-mon[35324]: pgmap v357: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.8 MiB/s wr, 553 op/s 2024-05-27T07:57:40.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:39 smithi129 ceph-mon[35254]: pgmap v357: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.8 MiB/s wr, 553 op/s 2024-05-27T07:57:40.815 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:41.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:41 smithi090 ceph-mon[31429]: pgmap v358: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.3 MiB/s wr, 607 op/s 2024-05-27T07:57:42.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:41 smithi136 ceph-mon[35324]: pgmap v358: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.3 MiB/s wr, 607 op/s 2024-05-27T07:57:42.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:41 smithi129 ceph-mon[35254]: pgmap v358: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.3 MiB/s wr, 607 op/s 2024-05-27T07:57:43.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 445.0 s, 595.3 tps, lat 170.336 ms stddev 90.496 2024-05-27T07:57:43.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:43 smithi090 ceph-mon[31429]: pgmap v359: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.2 MiB/s wr, 474 op/s 2024-05-27T07:57:44.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:43 smithi136 ceph-mon[35324]: pgmap v359: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.2 MiB/s wr, 474 op/s 2024-05-27T07:57:44.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:43 smithi129 ceph-mon[35254]: pgmap v359: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.2 MiB/s wr, 474 op/s 2024-05-27T07:57:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.190+0000 7f18af200640 1 -- 172.21.15.90:0/1559435232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010d340 msgr2=0x7f18b010f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.190+0000 7f18af200640 1 --2- 172.21.15.90:0/1559435232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010d340 0x7f18b010f760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009900 tx=0x7f18a802a420 comp rx=0 tx=0).stop 2024-05-27T07:57:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.190+0000 7f18af200640 1 -- 172.21.15.90:0/1559435232 shutdown_connections 2024-05-27T07:57:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.190+0000 7f18af200640 1 --2- 172.21.15.90:0/1559435232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010d340 0x7f18b010f760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.190+0000 7f18af200640 1 --2- 172.21.15.90:0/1559435232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b0102ab0 0x7f18b0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.190+0000 7f18af200640 1 --2- 172.21.15.90:0/1559435232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01089e0 0x7f18b0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.190+0000 7f18af200640 1 -- 172.21.15.90:0/1559435232 >> 172.21.15.90:0/1559435232 conn(0x7f18b00fe210 msgr2=0x7f18b0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.194+0000 7f18af200640 1 -- 172.21.15.90:0/1559435232 shutdown_connections 2024-05-27T07:57:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.195+0000 7f18af200640 1 -- 172.21.15.90:0/1559435232 wait complete. 2024-05-27T07:57:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.196+0000 7f18af200640 1 Processor -- start 2024-05-27T07:57:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.197+0000 7f18af200640 1 -- start start 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.199+0000 7f18af200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b0102ab0 0x7f18b0074800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.199+0000 7f18af200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b01089e0 0x7f18b006d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.199+0000 7f18af200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010d340 0x7f18b006de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.199+0000 7f18af200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0112b20 con 0x7f18b01089e0 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.199+0000 7f18af200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01129a0 con 0x7f18b0102ab0 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.199+0000 7f18af200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0112ca0 con 0x7f18b010d340 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.200+0000 7f18ae800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010d340 0x7f18b006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.200+0000 7f18ade00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b0102ab0 0x7f18b0074800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.200+0000 7f18ade00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b0102ab0 0x7f18b0074800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45356/0 (socket says 172.21.15.90:45356) 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.200+0000 7f18ade00640 1 -- 172.21.15.90:0/418102706 learned_addr learned my addr 172.21.15.90:0/418102706 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.201+0000 7f18ade00640 1 -- 172.21.15.90:0/418102706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010d340 msgr2=0x7f18b006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.201+0000 7f18ade00640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010d340 0x7f18b006de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.201+0000 7f18ade00640 1 -- 172.21.15.90:0/418102706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b01089e0 msgr2=0x7f18b006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.201+0000 7f18ade00640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b01089e0 0x7f18b006d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.201+0000 7f18ade00640 1 -- 172.21.15.90:0/418102706 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b006e6a0 con 0x7f18b0102ab0 2024-05-27T07:57:44.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.202+0000 7f18ade00640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b0102ab0 0x7f18b0074800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f18a000ea80 tx=0x7f18a000ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.203+0000 7f18a7400640 1 -- 172.21.15.90:0/418102706 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a000ce40 con 0x7f18b0102ab0 2024-05-27T07:57:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.203+0000 7f18a7400640 1 -- 172.21.15.90:0/418102706 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a0011900 con 0x7f18b0102ab0 2024-05-27T07:57:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.203+0000 7f18a7400640 1 -- 172.21.15.90:0/418102706 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a001a9c0 con 0x7f18b0102ab0 2024-05-27T07:57:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.203+0000 7f18af200640 1 -- 172.21.15.90:0/418102706 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b0072380 con 0x7f18b0102ab0 2024-05-27T07:57:44.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.203+0000 7f18af200640 1 -- 172.21.15.90:0/418102706 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b0072880 con 0x7f18b0102ab0 2024-05-27T07:57:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.209+0000 7f18bbfe1740 1 Processor -- start 2024-05-27T07:57:44.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.209+0000 7f18bbfe1740 1 -- start start 2024-05-27T07:57:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.211+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc60bb1b0 0x557cc645da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.211+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60922f0 0x557cc645dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.211+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc645e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.211+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557cc63ceb90 con 0x557cc60922f0 2024-05-27T07:57:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.211+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557cc63be730 con 0x557cc609a4d0 2024-05-27T07:57:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.211+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557cc63c2770 con 0x557cc60bb1b0 2024-05-27T07:57:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.212+0000 7f18a5400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc645e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.212+0000 7f18ae800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc60bb1b0 0x557cc645da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.212+0000 7f18ae800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc60bb1b0 0x557cc645da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38166/0 (socket says 172.21.15.90:38166) 2024-05-27T07:57:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.212+0000 7f18ae800640 1 -- 172.21.15.90:0/3990131435 learned_addr learned my addr 172.21.15.90:0/3990131435 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.215+0000 7f18a5400640 1 -- 172.21.15.90:0/3990131435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc60bb1b0 msgr2=0x557cc645da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.215+0000 7f18a5400640 1 --2- 172.21.15.90:0/3990131435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc60bb1b0 0x557cc645da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.215+0000 7f18a5400640 1 -- 172.21.15.90:0/3990131435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60922f0 msgr2=0x557cc645dfa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:57:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.215+0000 7f18a5400640 1 --2- 172.21.15.90:0/3990131435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60922f0 0x557cc645dfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.215+0000 7f18a5400640 1 -- 172.21.15.90:0/3990131435 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc63578b0 con 0x557cc609a4d0 2024-05-27T07:57:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.215+0000 7f18ae800640 1 --2- 172.21.15.90:0/3990131435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc60bb1b0 0x557cc645da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.216+0000 7f18a5400640 1 --2- 172.21.15.90:0/3990131435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc645e4e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f18b0075fb0 tx=0x7f189800bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00305b18ccccf41 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.217+0000 7f18a7400640 1 -- 172.21.15.90:0/418102706 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f18a0011470 con 0x7f18b0102ab0 2024-05-27T07:57:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.218+0000 7f189f400640 1 -- 172.21.15.90:0/3990131435 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898016070 con 0x557cc609a4d0 2024-05-27T07:57:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.218+0000 7f189f400640 1 -- 172.21.15.90:0/3990131435 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18980054b0 con 0x557cc609a4d0 2024-05-27T07:57:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.218+0000 7f189f400640 1 -- 172.21.15.90:0/3990131435 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898004d60 con 0x557cc609a4d0 2024-05-27T07:57:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.220+0000 7f18a7400640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18940b2b20 0x7f18940b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.220+0000 7f18a7400640 1 -- 172.21.15.90:0/418102706 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f18a00e2ec0 con 0x7f18b0102ab0 2024-05-27T07:57:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.221+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3990131435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 msgr2=0x557cc645e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.221+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/3990131435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc645e4e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f18b0075fb0 tx=0x7f189800bec0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.221+0000 7f18ae800640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18940b2b20 0x7f18940b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.225+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3990131435 shutdown_connections 2024-05-27T07:57:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.225+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/3990131435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc645e4e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.225+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/3990131435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60922f0 0x557cc645dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.225+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/3990131435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc60bb1b0 0x557cc645da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.225+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3990131435 >> 172.21.15.90:0/3990131435 conn(0x557cc60af300 msgr2=0x557cc645a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.226+0000 7f18ae800640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18940b2b20 0x7f18940b5030 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f18a8006010 tx=0x7f18a80046c0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.227+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3990131435 shutdown_connections 2024-05-27T07:57:44.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.227+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3990131435 wait complete. 2024-05-27T07:57:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.228+0000 7f18bbfe1740 1 -- start start 2024-05-27T07:57:44.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.228+0000 7f18bbfe1740 1 Processor -- start 2024-05-27T07:57:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.230+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc60922f0 0x557cc6467b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.230+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc609a4d0 0x557cc6468080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.230+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64685e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.230+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557cc60a4790 con 0x557cc60bb1b0 2024-05-27T07:57:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.230+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557cc6366bc0 con 0x557cc60922f0 2024-05-27T07:57:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.230+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557cc646abe0 con 0x557cc609a4d0 2024-05-27T07:57:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.230+0000 7f18bbfe1740 10 client.0 _ll_register_callbacks cb 0x557cc6344400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.231+0000 7f18ade00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc60922f0 0x557cc6467b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.231+0000 7f18a5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc609a4d0 0x557cc6468080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.231+0000 7f18ade00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc60922f0 0x557cc6467b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45366/0 (socket says 172.21.15.90:45366) 2024-05-27T07:57:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.231+0000 7f18a5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc609a4d0 0x557cc6468080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38172/0 (socket says 172.21.15.90:38172) 2024-05-27T07:57:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.231+0000 7f18ade00640 1 -- 172.21.15.90:0/3170633800 learned_addr learned my addr 172.21.15.90:0/3170633800 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.231+0000 7f18ae800640 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.231+0000 7f18ade00640 1 -- 172.21.15.90:0/3170633800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc609a4d0 msgr2=0x557cc6468080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.231+0000 7f18ade00640 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc609a4d0 0x557cc6468080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.231+0000 7f18ade00640 1 -- 172.21.15.90:0/3170633800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 msgr2=0x557cc64685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.231+0000 7f18ade00640 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.232+0000 7f18ade00640 1 -- 172.21.15.90:0/3170633800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc60d3210 con 0x557cc60922f0 2024-05-27T07:57:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.232+0000 7f18ae800640 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.232+0000 7f18ade00640 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc60922f0 0x557cc6467b40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f18a00e0240 tx=0x7f18a00a9610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da46b67d47d6692a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.232+0000 7f1893e00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:57:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.233+0000 7f1893e00640 1 -- 172.21.15.90:0/3170633800 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a00e0670 con 0x557cc60922f0 2024-05-27T07:57:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.233+0000 7f1893e00640 1 -- 172.21.15.90:0/3170633800 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f18a001ab60 con 0x557cc60922f0 2024-05-27T07:57:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.233+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3170633800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18980054b0 con 0x557cc60922f0 2024-05-27T07:57:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.234+0000 7f1893e00640 1 -- 172.21.15.90:0/3170633800 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f18a00c4460 con 0x557cc60922f0 2024-05-27T07:57:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.234+0000 7f18bbfe1740 10 client.37049 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.234+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3170633800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x557cc619f620 con 0x557cc60922f0 2024-05-27T07:57:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.234+0000 7f1893e00640 1 -- 172.21.15.90:0/3170633800 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18980054b0 con 0x557cc60922f0 2024-05-27T07:57:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.236+0000 7f1893e00640 1 -- 172.21.15.90:0/3170633800 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f18a0022030 con 0x557cc60922f0 2024-05-27T07:57:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.236+0000 7f18bbfe1740 10 client.37049 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.236+0000 7f18bbfe1740 10 client.37049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.236+0000 7f18bbfe1740 20 client.37049 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.236+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557cc6079d80 0x557cc6479f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.236+0000 7f18bbfe1740 4 client.37049 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.236+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3170633800 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cc63d0ae0 con 0x557cc6079d80 2024-05-27T07:57:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.237+0000 7f18a5400640 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557cc6079d80 0x557cc6479f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.238+0000 7f18a5400640 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557cc6079d80 0x557cc6479f00 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bcd31cb69cde245 server_cookie=dfc009b8f27ff07a in_seq=0 out_seq=0 2024-05-27T07:57:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.238+0000 7f1893e00640 10 client.37049 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.239+0000 7f1893e00640 1 -- 172.21.15.90:0/3170633800 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x557cc63d0ae0 con 0x557cc6079d80 2024-05-27T07:57:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.239+0000 7f1893e00640 10 client.37049 handle_command_reply: tid=0 2024-05-27T07:57:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.239+0000 7f18bbfe1740 1 client.37049 shutdown 2024-05-27T07:57:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.240+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3170633800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc60922f0 msgr2=0x557cc6467b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.240+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc60922f0 0x557cc6467b40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f18a00e0240 tx=0x7f18a00a9610 comp rx=0 tx=0).stop 2024-05-27T07:57:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.240+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3170633800 shutdown_connections 2024-05-27T07:57:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.240+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557cc6079d80 0x557cc6479f00 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.240+0000 7f1893e00640 0 client.37049 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.240+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.240+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc609a4d0 0x557cc6468080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.240+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/3170633800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc60922f0 0x557cc6467b40 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f18a00e0240 tx=0x7f18a00a9610 comp rx=0 tx=0).stop 2024-05-27T07:57:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.240+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3170633800 >> 172.21.15.90:0/3170633800 conn(0x557cc60af300 msgr2=0x557cc645f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.241+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3170633800 shutdown_connections 2024-05-27T07:57:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.241+0000 7f18bbfe1740 1 -- 172.21.15.90:0/3170633800 wait complete. 2024-05-27T07:57:44.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.241+0000 7f18bbfe1740 20 client.37049 trim_cache size 0 max 16384 2024-05-27T07:57:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.272+0000 7f18bbfe1740 1 Processor -- start 2024-05-27T07:57:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.273+0000 7f18bbfe1740 1 -- start start 2024-05-27T07:57:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.273+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc646a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.273+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc6467b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.273+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc6468050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.273+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01bd5f0 con 0x557cc60bb1b0 2024-05-27T07:57:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.273+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01ba640 con 0x557cc609a4d0 2024-05-27T07:57:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.273+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01bc850 con 0x557cc604f1c0 2024-05-27T07:57:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.274+0000 7f18ae800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc6468050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.274+0000 7f18ae800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc6468050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38176/0 (socket says 172.21.15.90:38176) 2024-05-27T07:57:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.274+0000 7f18ae800640 1 -- 172.21.15.90:0/1550967007 learned_addr learned my addr 172.21.15.90:0/1550967007 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.275+0000 7f18a5400640 1 --2- 172.21.15.90:0/1550967007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc6467b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.275+0000 7f18ae800640 1 -- 172.21.15.90:0/1550967007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 msgr2=0x557cc646a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.275+0000 7f18ae800640 1 --2- 172.21.15.90:0/1550967007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc646a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.275+0000 7f18ae800640 1 -- 172.21.15.90:0/1550967007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 msgr2=0x557cc6467b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.276+0000 7f18ae800640 1 --2- 172.21.15.90:0/1550967007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc6467b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.276+0000 7f18ae800640 1 -- 172.21.15.90:0/1550967007 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc6466090 con 0x557cc604f1c0 2024-05-27T07:57:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.277+0000 7f18ae800640 1 --2- 172.21.15.90:0/1550967007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc6468050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f18a802aa10 tx=0x7f18a803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abcbe6d58012b7a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.277+0000 7f189ea00640 1 -- 172.21.15.90:0/1550967007 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8035070 con 0x557cc604f1c0 2024-05-27T07:57:44.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.278+0000 7f189ea00640 1 -- 172.21.15.90:0/1550967007 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a8050910 con 0x557cc604f1c0 2024-05-27T07:57:44.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.278+0000 7f189ea00640 1 -- 172.21.15.90:0/1550967007 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a803c800 con 0x557cc604f1c0 2024-05-27T07:57:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.279+0000 7f18bbfe1740 1 -- 172.21.15.90:0/1550967007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 msgr2=0x557cc6468050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.279+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/1550967007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc6468050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f18a802aa10 tx=0x7f18a803f000 comp rx=0 tx=0).stop 2024-05-27T07:57:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.279+0000 7f18bbfe1740 1 -- 172.21.15.90:0/1550967007 shutdown_connections 2024-05-27T07:57:44.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.279+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/1550967007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc6468050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.280+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/1550967007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc6467b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.280+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/1550967007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc646a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.280+0000 7f18bbfe1740 1 -- 172.21.15.90:0/1550967007 >> 172.21.15.90:0/1550967007 conn(0x557cc60af300 msgr2=0x557cc6467400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.281+0000 7f18bbfe1740 1 -- 172.21.15.90:0/1550967007 shutdown_connections 2024-05-27T07:57:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.281+0000 7f18bbfe1740 1 -- 172.21.15.90:0/1550967007 wait complete. 2024-05-27T07:57:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.282+0000 7f18bbfe1740 1 -- start start 2024-05-27T07:57:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.282+0000 7f18bbfe1740 1 Processor -- start 2024-05-27T07:57:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.283+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc646b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.284+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc646b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.284+0000 7f18ade00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc646b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.284+0000 7f18bbfe1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.285+0000 7f18a5400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc646b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.285+0000 7f18a5400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc646b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45386/0 (socket says 172.21.15.90:45386) 2024-05-27T07:57:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.285+0000 7f18ae800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.285+0000 7f18ae800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64da590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51864/0 (socket says 172.21.15.90:51864) 2024-05-27T07:57:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.285+0000 7f18ade00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc646b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38186/0 (socket says 172.21.15.90:38186) 2024-05-27T07:57:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.285+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01bc180 con 0x557cc60bb1b0 2024-05-27T07:57:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.285+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01bad10 con 0x557cc609a4d0 2024-05-27T07:57:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.285+0000 7f18bbfe1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01bb3e0 con 0x557cc604f1c0 2024-05-27T07:57:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.285+0000 7f18bbfe1740 10 client.0 _ll_register_callbacks cb 0x7f18a00a9790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.285+0000 7f18a5400640 1 -- 172.21.15.90:0/987041142 learned_addr learned my addr 172.21.15.90:0/987041142 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.286+0000 7f18ade00640 1 -- 172.21.15.90:0/987041142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 msgr2=0x557cc646b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.286+0000 7f18ade00640 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc646b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.286+0000 7f18ade00640 1 -- 172.21.15.90:0/987041142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 msgr2=0x557cc64da590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.286+0000 7f18ade00640 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64da590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.287+0000 7f18ade00640 1 -- 172.21.15.90:0/987041142 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc64daad0 con 0x557cc604f1c0 2024-05-27T07:57:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.287+0000 7f18a5400640 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc646b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:44.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.287+0000 7f18ae800640 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.288+0000 7f18ade00640 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc646b450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x557cc6468c50 tx=0x7f18a00a9200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb83df3f2f5fdafc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.291+0000 7f1893e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:57:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.291+0000 7f1893e00640 1 -- 172.21.15.90:0/987041142 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a00e0670 con 0x557cc604f1c0 2024-05-27T07:57:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.291+0000 7f18bbfe1740 1 -- 172.21.15.90:0/987041142 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a8050910 con 0x557cc604f1c0 2024-05-27T07:57:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.292+0000 7f1893e00640 1 -- 172.21.15.90:0/987041142 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f18a00a6360 con 0x557cc604f1c0 2024-05-27T07:57:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.292+0000 7f1893e00640 1 -- 172.21.15.90:0/987041142 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f18a00e0670 con 0x557cc604f1c0 2024-05-27T07:57:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.292+0000 7f18bbfe1740 10 client.27142 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.292+0000 7f18bbfe1740 1 -- 172.21.15.90:0/987041142 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x557cc64e1d30 con 0x557cc604f1c0 2024-05-27T07:57:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.293+0000 7f1893e00640 1 -- 172.21.15.90:0/987041142 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18a8050910 con 0x557cc604f1c0 2024-05-27T07:57:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.293+0000 7f1893e00640 1 -- 172.21.15.90:0/987041142 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f18a0022040 con 0x557cc604f1c0 2024-05-27T07:57:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.293+0000 7f18bbfe1740 10 client.27142 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.293+0000 7f18bbfe1740 10 client.27142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.293+0000 7f18bbfe1740 20 client.27142 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.293+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557cc64e22d0 0x557cc64e26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:44.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.293+0000 7f18bbfe1740 4 client.27142 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:57:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.293+0000 7f18bbfe1740 1 -- 172.21.15.90:0/987041142 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557cc63ca2c0 con 0x557cc64e22d0 2024-05-27T07:57:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.294+0000 7f18a5400640 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557cc64e22d0 0x557cc64e26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.296+0000 7f18a5400640 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557cc64e22d0 0x557cc64e26e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a36ab7d9d8f0262f server_cookie=1037041056fbf08e in_seq=0 out_seq=0 2024-05-27T07:57:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.296+0000 7f1893e00640 10 client.27142 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:44.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.869+0000 7f1893e00640 1 -- 172.21.15.90:0/987041142 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557cc63ca2c0 con 0x557cc64e22d0 2024-05-27T07:57:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.869+0000 7f1893e00640 10 client.27142 handle_command_reply: tid=0 2024-05-27T07:57:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.869+0000 7f18bbfe1740 1 client.27142 shutdown 2024-05-27T07:57:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.870+0000 7f18bbfe1740 1 -- 172.21.15.90:0/987041142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 msgr2=0x557cc646b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.870+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc646b450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x557cc6468c50 tx=0x7f18a00a9200 comp rx=0 tx=0).stop 2024-05-27T07:57:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.870+0000 7f18bbfe1740 1 -- 172.21.15.90:0/987041142 shutdown_connections 2024-05-27T07:57:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.870+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x557cc64e22d0 0x557cc64e26e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.870+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557cc609a4d0 0x557cc646b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.870+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557cc60bb1b0 0x557cc64da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.870+0000 7f18bbfe1740 1 --2- 172.21.15.90:0/987041142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557cc604f1c0 0x557cc646b450 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.870+0000 7f18bbfe1740 1 -- 172.21.15.90:0/987041142 >> 172.21.15.90:0/987041142 conn(0x557cc60af300 msgr2=0x557cc6467e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.871+0000 7f1893e00640 0 client.27142 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.872+0000 7f18bbfe1740 1 -- 172.21.15.90:0/987041142 shutdown_connections 2024-05-27T07:57:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.872+0000 7f18bbfe1740 1 -- 172.21.15.90:0/987041142 wait complete. 2024-05-27T07:57:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.873+0000 7f18bbfe1740 20 client.27142 trim_cache size 0 max 16384 2024-05-27T07:57:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:57:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:57:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:57:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 -- 172.21.15.90:0/418102706 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18940b2b20 msgr2=0x7f18940b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18940b2b20 0x7f18940b5030 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f18a8006010 tx=0x7f18a80046c0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 -- 172.21.15.90:0/418102706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b0102ab0 msgr2=0x7f18b0074800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b0102ab0 0x7f18b0074800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f18a000ea80 tx=0x7f18a000ef40 comp rx=0 tx=0).stop 2024-05-27T07:57:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 -- 172.21.15.90:0/418102706 shutdown_connections 2024-05-27T07:57:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f18940b2b20 0x7f18940b5030 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18b010d340 0x7f18b006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18b01089e0 0x7f18b006d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 --2- 172.21.15.90:0/418102706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b0102ab0 0x7f18b0074800 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 -- 172.21.15.90:0/418102706 >> 172.21.15.90:0/418102706 conn(0x7f18b00fe210 msgr2=0x7f18b0104070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 -- 172.21.15.90:0/418102706 shutdown_connections 2024-05-27T07:57:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:44.874+0000 7f18af200640 1 -- 172.21.15.90:0/418102706 wait complete. 2024-05-27T07:57:45.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.943+0000 7fed80200700 1 -- 172.21.15.90:0/616633337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed78108f50 msgr2=0x7fed78109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:45.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.943+0000 7fed80200700 1 --2- 172.21.15.90:0/616633337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed78108f50 0x7fed78109360 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fed64009920 tx=0x7fed64009c30 comp rx=0 tx=0).stop 2024-05-27T07:57:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.944+0000 7fed80200700 1 -- 172.21.15.90:0/616633337 shutdown_connections 2024-05-27T07:57:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.944+0000 7fed80200700 1 --2- 172.21.15.90:0/616633337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed781098a0 0x7fed781111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.944+0000 7fed80200700 1 --2- 172.21.15.90:0/616633337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed78102590 0x7fed781029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.944+0000 7fed80200700 1 --2- 172.21.15.90:0/616633337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed78108f50 0x7fed78109360 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.944+0000 7fed80200700 1 -- 172.21.15.90:0/616633337 >> 172.21.15.90:0/616633337 conn(0x7fed780fe0c0 msgr2=0x7fed781004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.944+0000 7fed80200700 1 -- 172.21.15.90:0/616633337 shutdown_connections 2024-05-27T07:57:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.944+0000 7fed80200700 1 -- 172.21.15.90:0/616633337 wait complete. 2024-05-27T07:57:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.946+0000 7fed80200700 1 Processor -- start 2024-05-27T07:57:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.946+0000 7fed80200700 1 -- start start 2024-05-27T07:57:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.947+0000 7fed80200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed78102590 0x7fed7810e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.947+0000 7fed80200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed78108f50 0x7fed7810eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.947+0000 7fed80200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed781098a0 0x7fed7810f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.947+0000 7fed80200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fed78105650 con 0x7fed78102590 2024-05-27T07:57:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.947+0000 7fed80200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed781054d0 con 0x7fed781098a0 2024-05-27T07:57:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.947+0000 7fed80200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fed781057d0 con 0x7fed78108f50 2024-05-27T07:57:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.947+0000 7fed7e200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed781098a0 0x7fed7810f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.947+0000 7fed7e200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed781098a0 0x7fed7810f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45398/0 (socket says 172.21.15.90:45398) 2024-05-27T07:57:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.948+0000 7fed7e200700 1 -- 172.21.15.90:0/981860275 learned_addr learned my addr 172.21.15.90:0/981860275 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.948+0000 7fed7d800700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed78102590 0x7fed7810e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.948+0000 7fed7ce00700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed78108f50 0x7fed7810eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.948+0000 7fed7e200700 1 -- 172.21.15.90:0/981860275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed78108f50 msgr2=0x7fed7810eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:45.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.948+0000 7fed7e200700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed78108f50 0x7fed7810eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.948+0000 7fed7e200700 1 -- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed78102590 msgr2=0x7fed7810e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.948+0000 7fed7e200700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed78102590 0x7fed7810e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.948+0000 7fed7e200700 1 -- 172.21.15.90:0/981860275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed781ad4e0 con 0x7fed781098a0 2024-05-27T07:57:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.949+0000 7fed7d800700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed78102590 0x7fed7810e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.949+0000 7fed7e200700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed781098a0 0x7fed7810f0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed7000e3c0 tx=0x7fed7000e6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.950+0000 7fed76000700 1 -- 172.21.15.90:0/981860275 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700092d0 con 0x7fed781098a0 2024-05-27T07:57:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.950+0000 7fed80200700 1 -- 172.21.15.90:0/981860275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed781ad7d0 con 0x7fed781098a0 2024-05-27T07:57:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.950+0000 7fed80200700 1 -- 172.21.15.90:0/981860275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed781add30 con 0x7fed781098a0 2024-05-27T07:57:45.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.950+0000 7fed76000700 1 -- 172.21.15.90:0/981860275 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed7000ee70 con 0x7fed781098a0 2024-05-27T07:57:45.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.950+0000 7fed76000700 1 -- 172.21.15.90:0/981860275 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70007d80 con 0x7fed781098a0 2024-05-27T07:57:45.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.956+0000 7fed76000700 1 -- 172.21.15.90:0/981860275 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fed70012050 con 0x7fed781098a0 2024-05-27T07:57:45.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.959+0000 7fed76000700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fed600b2a70 0x7fed600b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.960+0000 7fed76000700 1 -- 172.21.15.90:0/981860275 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7fed700e23f0 con 0x7fed781098a0 2024-05-27T07:57:45.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.961+0000 7fed7d800700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fed600b2a70 0x7fed600b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:45.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.963+0000 7fed7d800700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fed600b2a70 0x7fed600b4f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fed78103380 tx=0x7fed6c009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:45.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.972+0000 7fed8ecd4b80 1 Processor -- start 2024-05-27T07:57:45.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.973+0000 7fed8ecd4b80 1 -- start start 2024-05-27T07:57:45.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.973+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa03ea7ec0 0x55aa0402c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.973+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa0402cdb0 0x55aa040268f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.973+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 0x55aa04027240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.973+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aa03f54ec0 con 0x55aa03ea7ec0 2024-05-27T07:57:45.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.973+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aa03f44a60 con 0x55aa04026e30 2024-05-27T07:57:45.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.973+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aa03f48aa0 con 0x55aa0402cdb0 2024-05-27T07:57:45.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.974+0000 7fed7ce00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa0402cdb0 0x55aa040268f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:45.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.974+0000 7fed7e200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 0x55aa04027240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:45.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.974+0000 7fed7e200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 0x55aa04027240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45412/0 (socket says 172.21.15.90:45412) 2024-05-27T07:57:45.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.974+0000 7fed7e200700 1 -- 172.21.15.90:0/1976483255 learned_addr learned my addr 172.21.15.90:0/1976483255 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:45.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.975+0000 7fed7d800700 1 --2- 172.21.15.90:0/1976483255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa03ea7ec0 0x55aa0402c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:45.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.975+0000 7fed7ce00700 1 -- 172.21.15.90:0/1976483255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 msgr2=0x55aa04027240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:45.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.975+0000 7fed7ce00700 1 --2- 172.21.15.90:0/1976483255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 0x55aa04027240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.975+0000 7fed7ce00700 1 -- 172.21.15.90:0/1976483255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa03ea7ec0 msgr2=0x55aa0402c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:45.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.975+0000 7fed7ce00700 1 --2- 172.21.15.90:0/1976483255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa03ea7ec0 0x55aa0402c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.975+0000 7fed7ce00700 1 -- 172.21.15.90:0/1976483255 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa03f0d4a0 con 0x55aa0402cdb0 2024-05-27T07:57:45.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.976+0000 7fed7ce00700 1 --2- 172.21.15.90:0/1976483255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa0402cdb0 0x55aa040268f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fed64000c00 tx=0x7fed6400bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31a7e9eec73fa252 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:45.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.976+0000 7fed6aa00700 1 -- 172.21.15.90:0/1976483255 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6400f460 con 0x55aa0402cdb0 2024-05-27T07:57:45.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.976+0000 7fed6aa00700 1 -- 172.21.15.90:0/1976483255 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed64005040 con 0x55aa0402cdb0 2024-05-27T07:57:45.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.978+0000 7fed6aa00700 1 -- 172.21.15.90:0/1976483255 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6400f460 con 0x55aa0402cdb0 2024-05-27T07:57:45.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.978+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/1976483255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa0402cdb0 msgr2=0x55aa040268f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:45.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.978+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/1976483255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa0402cdb0 0x55aa040268f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fed64000c00 tx=0x7fed6400bf40 comp rx=0 tx=0).stop 2024-05-27T07:57:45.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.978+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/1976483255 shutdown_connections 2024-05-27T07:57:45.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.978+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/1976483255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 0x55aa04027240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.978+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/1976483255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa03ea7ec0 0x55aa0402c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.978+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/1976483255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa0402cdb0 0x55aa040268f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.978+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/1976483255 >> 172.21.15.90:0/1976483255 conn(0x55aa03f08040 msgr2=0x55aa04021950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:45.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.979+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/1976483255 shutdown_connections 2024-05-27T07:57:45.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.982+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/1976483255 wait complete. 2024-05-27T07:57:45.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.983+0000 7fed8ecd4b80 1 -- start start 2024-05-27T07:57:45.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.983+0000 7fed8ecd4b80 1 Processor -- start 2024-05-27T07:57:45.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.984+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 0x55aa040304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.984+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa04026e30 0x55aa04030a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.984+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa0402cdb0 0x55aa04030f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.984+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55aa03e4b020 con 0x55aa04026e30 2024-05-27T07:57:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.984+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aa03ee57e0 con 0x55aa03ea7ec0 2024-05-27T07:57:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.984+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aa04033820 con 0x55aa0402cdb0 2024-05-27T07:57:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.984+0000 7fed8ecd4b80 10 client.0 _ll_register_callbacks cb 0x55aa03eeecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.985+0000 7fed7d800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 0x55aa040304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.985+0000 7fed7d800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 0x55aa040304f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:45422/0 (socket says 172.21.15.90:45422) 2024-05-27T07:57:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.985+0000 7fed7d800700 1 -- 172.21.15.90:0/4046919692 learned_addr learned my addr 172.21.15.90:0/4046919692 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:45.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.985+0000 7fed7ce00700 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa04026e30 0x55aa04030a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:45.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.986+0000 7fed7d800700 1 -- 172.21.15.90:0/4046919692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa0402cdb0 msgr2=0x55aa04030f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:45.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.986+0000 7fed7d800700 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa0402cdb0 0x55aa04030f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.986+0000 7fed7d800700 1 -- 172.21.15.90:0/4046919692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa04026e30 msgr2=0x55aa04030a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.987+0000 7fed7d800700 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa04026e30 0x55aa04030a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.987+0000 7fed7d800700 1 -- 172.21.15.90:0/4046919692 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa03e94950 con 0x55aa03ea7ec0 2024-05-27T07:57:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.987+0000 7fed7ce00700 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa04026e30 0x55aa04030a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:45.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.988+0000 7fed7d800700 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 0x55aa040304f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fed6c012890 tx=0x7fed6c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b34324e22d4b1465 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:45.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.988+0000 7fed5f400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:57:45.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.989+0000 7fed5f400700 1 -- 172.21.15.90:0/4046919692 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c01e970 con 0x55aa03ea7ec0 2024-05-27T07:57:45.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.989+0000 7fed5f400700 1 -- 172.21.15.90:0/4046919692 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7fed6c01eb10 con 0x55aa03ea7ec0 2024-05-27T07:57:45.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.990+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/4046919692 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed64005040 con 0x55aa03ea7ec0 2024-05-27T07:57:45.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.991+0000 7fed5f400700 1 -- 172.21.15.90:0/4046919692 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fed6c01ed10 con 0x55aa03ea7ec0 2024-05-27T07:57:45.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.991+0000 7fed8ecd4b80 10 client.37064 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:45.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.991+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/4046919692 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55aa03f150e0 con 0x55aa03ea7ec0 2024-05-27T07:57:45.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.992+0000 7fed5f400700 1 -- 172.21.15.90:0/4046919692 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed64005040 con 0x55aa03ea7ec0 2024-05-27T07:57:45.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.992+0000 7fed5f400700 1 -- 172.21.15.90:0/4046919692 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fed6c0273f0 con 0x55aa03ea7ec0 2024-05-27T07:57:45.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.992+0000 7fed8ecd4b80 10 client.37064 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:45.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.992+0000 7fed8ecd4b80 10 client.37064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:45.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.992+0000 7fed8ecd4b80 20 client.37064 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:45.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.992+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa040408a0 0x55aa04042d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:45.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.992+0000 7fed8ecd4b80 4 client.37064 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:45.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.992+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/4046919692 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa03f56e10 con 0x55aa040408a0 2024-05-27T07:57:45.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.993+0000 7fed7ce00700 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa040408a0 0x55aa04042d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:45.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.994+0000 7fed7ce00700 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa040408a0 0x55aa04042d90 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8897bcfe2194f313 server_cookie=c9c565fcd6430ff1 in_seq=0 out_seq=0 2024-05-27T07:57:45.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.994+0000 7fed5f400700 10 client.37064 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:45.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:45 smithi090 ceph-mon[31429]: pgmap v360: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.9 MiB/s wr, 665 op/s 2024-05-27T07:57:46.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:45 smithi136 ceph-mon[35324]: pgmap v360: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.9 MiB/s wr, 665 op/s 2024-05-27T07:57:46.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.998+0000 7fed5f400700 1 -- 172.21.15.90:0/4046919692 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55aa03f56e10 con 0x55aa040408a0 2024-05-27T07:57:46.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:45.998+0000 7fed5f400700 10 client.37064 handle_command_reply: tid=0 2024-05-27T07:57:46.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.000+0000 7fed8ecd4b80 1 client.37064 shutdown 2024-05-27T07:57:46.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.000+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/4046919692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 msgr2=0x55aa040304f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.000+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 0x55aa040304f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fed6c012890 tx=0x7fed6c0047c0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.000+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/4046919692 shutdown_connections 2024-05-27T07:57:46.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.000+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa040408a0 0x55aa04042d90 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.000+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa0402cdb0 0x55aa04030f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.001+0000 7fed5f400700 0 client.37064 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:46.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.001+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 0x55aa040304f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.001+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/4046919692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa04026e30 0x55aa04030a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.001+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/4046919692 >> 172.21.15.90:0/4046919692 conn(0x55aa03f08040 msgr2=0x55aa04021600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.001+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/4046919692 shutdown_connections 2024-05-27T07:57:46.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.002+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/4046919692 wait complete. 2024-05-27T07:57:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.002+0000 7fed8ecd4b80 20 client.37064 trim_cache size 0 max 16384 2024-05-27T07:57:46.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.046+0000 7fed8ecd4b80 1 Processor -- start 2024-05-27T07:57:46.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.046+0000 7fed8ecd4b80 1 -- start start 2024-05-27T07:57:46.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.046+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 0x55aa040268f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.047+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa04026e30 0x55aa04032990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.047+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 0x55aa0402abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.047+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fed781b8560 con 0x55aa0402cdb0 2024-05-27T07:57:46.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.047+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed781b55b0 con 0x55aa03ea7ec0 2024-05-27T07:57:46.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.047+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fed781b77c0 con 0x55aa04026e30 2024-05-27T07:57:46.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.047+0000 7fed7ce00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa04026e30 0x55aa04032990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.047+0000 7fed7ce00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa04026e30 0x55aa04032990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38228/0 (socket says 172.21.15.90:38228) 2024-05-27T07:57:46.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.048+0000 7fed7ce00700 1 -- 172.21.15.90:0/1020714294 learned_addr learned my addr 172.21.15.90:0/1020714294 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:46.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.048+0000 7fed7e200700 1 --2- 172.21.15.90:0/1020714294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 0x55aa0402abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.049+0000 7fed7d800700 1 --2- 172.21.15.90:0/1020714294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 0x55aa040268f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.049+0000 7fed7e200700 1 -- 172.21.15.90:0/1020714294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa04026e30 msgr2=0x55aa04032990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.049+0000 7fed7e200700 1 --2- 172.21.15.90:0/1020714294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa04026e30 0x55aa04032990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.049+0000 7fed7e200700 1 -- 172.21.15.90:0/1020714294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 msgr2=0x55aa040268f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.049+0000 7fed7e200700 1 --2- 172.21.15.90:0/1020714294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 0x55aa040268f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.049+0000 7fed7e200700 1 -- 172.21.15.90:0/1020714294 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa0402ea60 con 0x55aa0402cdb0 2024-05-27T07:57:46.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.049+0000 7fed7e200700 1 --2- 172.21.15.90:0/1020714294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 0x55aa0402abf0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fed700a5410 tx=0x7fed700a9230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a7154b8c3329ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.050+0000 7fed6a000700 1 -- 172.21.15.90:0/1020714294 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700a9a90 con 0x55aa0402cdb0 2024-05-27T07:57:46.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.050+0000 7fed6a000700 1 -- 172.21.15.90:0/1020714294 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed700a9390 con 0x55aa0402cdb0 2024-05-27T07:57:46.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.050+0000 7fed6a000700 1 -- 172.21.15.90:0/1020714294 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700a95f0 con 0x55aa0402cdb0 2024-05-27T07:57:46.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.051+0000 7fed7ce00700 1 --2- 172.21.15.90:0/1020714294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa04026e30 0x55aa04032990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:46.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.052+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/1020714294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 msgr2=0x55aa0402abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.052+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/1020714294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 0x55aa0402abf0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fed700a5410 tx=0x7fed700a9230 comp rx=0 tx=0).stop 2024-05-27T07:57:46.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.052+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/1020714294 shutdown_connections 2024-05-27T07:57:46.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.052+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/1020714294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 0x55aa0402abf0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.052+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/1020714294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa03ea7ec0 0x55aa040268f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.052+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/1020714294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa04026e30 0x55aa04032990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.052+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/1020714294 >> 172.21.15.90:0/1020714294 conn(0x55aa03f08040 msgr2=0x55aa04030b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.053+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/1020714294 shutdown_connections 2024-05-27T07:57:46.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.053+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/1020714294 wait complete. 2024-05-27T07:57:46.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.054+0000 7fed8ecd4b80 1 -- start start 2024-05-27T07:57:46.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.055+0000 7fed8ecd4b80 1 Processor -- start 2024-05-27T07:57:46.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.056+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa03ea7ec0 0x55aa04035be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.057+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 0x55aa04036120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.057+0000 7fed8ecd4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 0x55aa040ab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.058+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fed781b70f0 con 0x55aa0402cdb0 2024-05-27T07:57:46.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.058+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed781b5c80 con 0x55aa04026e30 2024-05-27T07:57:46.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.058+0000 7fed8ecd4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fed781b6350 con 0x55aa03ea7ec0 2024-05-27T07:57:46.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.058+0000 7fed8ecd4b80 10 client.0 _ll_register_callbacks cb 0x55aa03eeecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.058+0000 7fed7d800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa03ea7ec0 0x55aa04035be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.059+0000 7fed7d800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa03ea7ec0 0x55aa04035be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:38238/0 (socket says 172.21.15.90:38238) 2024-05-27T07:57:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.059+0000 7fed7d800700 1 -- 172.21.15.90:0/925334918 learned_addr learned my addr 172.21.15.90:0/925334918 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.059+0000 7fed7ce00700 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 0x55aa04036120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.059+0000 7fed7e200700 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 0x55aa040ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.061+0000 7fed7ce00700 1 -- 172.21.15.90:0/925334918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa03ea7ec0 msgr2=0x55aa04035be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.061+0000 7fed7ce00700 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa03ea7ec0 0x55aa04035be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.061+0000 7fed7ce00700 1 -- 172.21.15.90:0/925334918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 msgr2=0x55aa040ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.062+0000 7fed7ce00700 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 0x55aa040ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.062+0000 7fed7ce00700 1 -- 172.21.15.90:0/925334918 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa040ab970 con 0x55aa04026e30 2024-05-27T07:57:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.062+0000 7fed7e200700 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 0x55aa040ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:57:46.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:45 smithi129 ceph-mon[35254]: pgmap v360: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.9 MiB/s wr, 665 op/s 2024-05-27T07:57:46.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.063+0000 7fed7ce00700 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 0x55aa04036120 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fed64005950 tx=0x7fed640058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e370721248e3eaa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:46.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.063+0000 7fed5f400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:57:46.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.066+0000 7fed5f400700 1 -- 172.21.15.90:0/925334918 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64037060 con 0x55aa04026e30 2024-05-27T07:57:46.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.067+0000 7fed5f400700 1 -- 172.21.15.90:0/925334918 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7fed64031e00 con 0x55aa04026e30 2024-05-27T07:57:46.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.067+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/925334918 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed700a9390 con 0x55aa04026e30 2024-05-27T07:57:46.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.068+0000 7fed5f400700 1 -- 172.21.15.90:0/925334918 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fed64003890 con 0x55aa04026e30 2024-05-27T07:57:46.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.068+0000 7fed8ecd4b80 10 client.37070 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:46.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.068+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/925334918 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55aa040aca50 con 0x55aa04026e30 2024-05-27T07:57:46.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.070+0000 7fed5f400700 1 -- 172.21.15.90:0/925334918 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed700a9390 con 0x55aa04026e30 2024-05-27T07:57:46.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.070+0000 7fed5f400700 1 -- 172.21.15.90:0/925334918 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fed64020070 con 0x55aa04026e30 2024-05-27T07:57:46.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.070+0000 7fed8ecd4b80 10 client.37070 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:46.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.070+0000 7fed8ecd4b80 10 client.37070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:46.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.070+0000 7fed8ecd4b80 20 client.37070 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:46.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.070+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa040408a0 0x55aa040aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:46.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.070+0000 7fed8ecd4b80 4 client.37070 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:57:46.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.070+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/925334918 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55aa03f505f0 con 0x55aa040408a0 2024-05-27T07:57:46.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.071+0000 7fed7d800700 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa040408a0 0x55aa040aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:46.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.075+0000 7fed7d800700 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa040408a0 0x55aa040aeed0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6684f94bae0363ba server_cookie=bc648e7169f80fbe in_seq=0 out_seq=0 2024-05-27T07:57:46.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.075+0000 7fed5f400700 10 client.37070 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:46.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.077+0000 7fed5f400700 1 -- 172.21.15.90:0/925334918 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aa03f505f0 con 0x55aa040408a0 2024-05-27T07:57:46.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.077+0000 7fed5f400700 10 client.37070 handle_command_reply: tid=0 2024-05-27T07:57:46.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.078+0000 7fed8ecd4b80 1 client.37070 shutdown 2024-05-27T07:57:46.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.080+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/925334918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 msgr2=0x55aa04036120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.080+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 0x55aa04036120 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fed64005950 tx=0x7fed640058e0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.080+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/925334918 shutdown_connections 2024-05-27T07:57:46.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.080+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55aa040408a0 0x55aa040aeed0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.080+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55aa0402cdb0 0x55aa040ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.080+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aa03ea7ec0 0x55aa04035be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.080+0000 7fed8ecd4b80 1 --2- 172.21.15.90:0/925334918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa04026e30 0x55aa04036120 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.080+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/925334918 >> 172.21.15.90:0/925334918 conn(0x55aa03f08040 msgr2=0x55aa0402d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.080+0000 7fed5f400700 0 client.37070 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:46.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.080+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/925334918 shutdown_connections 2024-05-27T07:57:46.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.081+0000 7fed8ecd4b80 1 -- 172.21.15.90:0/925334918 wait complete. 2024-05-27T07:57:46.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.081+0000 7fed8ecd4b80 20 client.37070 trim_cache size 0 max 16384 2024-05-27T07:57:46.084 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:57:46.084 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:57:46.084 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "8eff7722-c368-4dd7-b198-8a27ee377986", 2024-05-27T07:57:46.084 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:57:46.084 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:57:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.082+0000 7fed80200700 1 -- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fed600b2a70 msgr2=0x7fed600b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.082+0000 7fed80200700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fed600b2a70 0x7fed600b4f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fed78103380 tx=0x7fed6c009260 comp rx=0 tx=0).stop 2024-05-27T07:57:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.082+0000 7fed80200700 1 -- 172.21.15.90:0/981860275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed781098a0 msgr2=0x7fed7810f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.082+0000 7fed80200700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed781098a0 0x7fed7810f0c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fed7000e3c0 tx=0x7fed7000e6d0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.082+0000 7fed80200700 1 -- 172.21.15.90:0/981860275 shutdown_connections 2024-05-27T07:57:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.082+0000 7fed80200700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fed600b2a70 0x7fed600b4f80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.082+0000 7fed80200700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed781098a0 0x7fed7810f0c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.082+0000 7fed80200700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed78102590 0x7fed7810e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.082+0000 7fed80200700 1 --2- 172.21.15.90:0/981860275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed78108f50 0x7fed7810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:46.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.082+0000 7fed80200700 1 -- 172.21.15.90:0/981860275 >> 172.21.15.90:0/981860275 conn(0x7fed780fe0c0 msgr2=0x7fed781071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:46.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.083+0000 7fed80200700 1 -- 172.21.15.90:0/981860275 shutdown_connections 2024-05-27T07:57:46.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:46.083+0000 7fed80200700 1 -- 172.21.15.90:0/981860275 wait complete. 2024-05-27T07:57:47.208 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:57:48.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:47 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:57:48.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:47 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:48.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:47 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:57:48.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:47 smithi129 ceph-mon[35254]: pgmap v361: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-05-27T07:57:48.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:47 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:48.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:47 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:57:48.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:47 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:48.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:47 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:57:48.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:47 smithi090 ceph-mon[31429]: pgmap v361: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-05-27T07:57:48.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:47 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:48.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:47 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:57:48.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:47 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:48.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:47 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:57:48.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:47 smithi136 ceph-mon[35324]: pgmap v361: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.1 MiB/s wr, 466 op/s 2024-05-27T07:57:48.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:47 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:57:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 450.0 s, 476.2 tps, lat 208.604 ms stddev 154.623 2024-05-27T07:57:48.835 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:50.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:49 smithi090 ceph-mon[31429]: pgmap v362: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.8 MiB/s wr, 544 op/s 2024-05-27T07:57:50.060 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:49 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:57:50.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:49 smithi129 ceph-mon[35254]: pgmap v362: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.8 MiB/s wr, 544 op/s 2024-05-27T07:57:50.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:49 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:57:50.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:49 smithi136 ceph-mon[35324]: pgmap v362: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.8 MiB/s wr, 544 op/s 2024-05-27T07:57:50.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:49 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:57:51.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.557+0000 7ff38c400700 1 -- 172.21.15.90:0/714435075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff384103a50 msgr2=0x7ff3841106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.557+0000 7ff38c400700 1 --2- 172.21.15.90:0/714435075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff384103a50 0x7ff3841106e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff3700098f0 tx=0x7ff370009c00 comp rx=0 tx=0).stop 2024-05-27T07:57:51.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.558+0000 7ff38c400700 1 -- 172.21.15.90:0/714435075 shutdown_connections 2024-05-27T07:57:51.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.558+0000 7ff38c400700 1 --2- 172.21.15.90:0/714435075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff384110c20 0x7ff384113040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.558+0000 7ff38c400700 1 --2- 172.21.15.90:0/714435075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff384103070 0x7ff384103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.558+0000 7ff38c400700 1 --2- 172.21.15.90:0/714435075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff384103a50 0x7ff3841106e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.558+0000 7ff38c400700 1 -- 172.21.15.90:0/714435075 >> 172.21.15.90:0/714435075 conn(0x7ff3840fedb0 msgr2=0x7ff3841011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:51.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.559+0000 7ff38c400700 1 -- 172.21.15.90:0/714435075 shutdown_connections 2024-05-27T07:57:51.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.559+0000 7ff38c400700 1 -- 172.21.15.90:0/714435075 wait complete. 2024-05-27T07:57:51.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.561+0000 7ff38c400700 1 Processor -- start 2024-05-27T07:57:51.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.561+0000 7ff38c400700 1 -- start start 2024-05-27T07:57:51.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.562+0000 7ff38c400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff384103070 0x7ff38410dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.562+0000 7ff38c400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff384103a50 0x7ff38410e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.562+0000 7ff38c400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff384110c20 0x7ff38410e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.562+0000 7ff38c400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff384116520 con 0x7ff384110c20 2024-05-27T07:57:51.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.562+0000 7ff38c400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff3841163a0 con 0x7ff384103070 2024-05-27T07:57:51.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.562+0000 7ff38c400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3841166a0 con 0x7ff384103a50 2024-05-27T07:57:51.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.563+0000 7ff389a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff384103070 0x7ff38410dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.563+0000 7ff389a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff384103070 0x7ff38410dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48874/0 (socket says 172.21.15.90:48874) 2024-05-27T07:57:51.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.563+0000 7ff389a00700 1 -- 172.21.15.90:0/2598323230 learned_addr learned my addr 172.21.15.90:0/2598323230 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:51.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.564+0000 7ff389000700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff384103a50 0x7ff38410e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.564+0000 7ff38a400700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff384110c20 0x7ff38410e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.564+0000 7ff389a00700 1 -- 172.21.15.90:0/2598323230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff384103a50 msgr2=0x7ff38410e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.564+0000 7ff389a00700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff384103a50 0x7ff38410e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.564+0000 7ff389a00700 1 -- 172.21.15.90:0/2598323230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff384110c20 msgr2=0x7ff38410e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.564+0000 7ff389a00700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff384110c20 0x7ff38410e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.565+0000 7ff389a00700 1 -- 172.21.15.90:0/2598323230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38419bb20 con 0x7ff384103070 2024-05-27T07:57:51.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.565+0000 7ff389000700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff384103a50 0x7ff38410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:51.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.566+0000 7ff389a00700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff384103070 0x7ff38410dbf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff37800e9f0 tx=0x7ff37800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:51.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.566+0000 7ff382000700 1 -- 172.21.15.90:0/2598323230 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378004a30 con 0x7ff384103070 2024-05-27T07:57:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.566+0000 7ff382000700 1 -- 172.21.15.90:0/2598323230 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff37800cc50 con 0x7ff384103070 2024-05-27T07:57:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.567+0000 7ff382000700 1 -- 172.21.15.90:0/2598323230 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378004a30 con 0x7ff384103070 2024-05-27T07:57:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.568+0000 7ff38c400700 1 -- 172.21.15.90:0/2598323230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38419be10 con 0x7ff384103070 2024-05-27T07:57:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.568+0000 7ff38c400700 1 -- 172.21.15.90:0/2598323230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38419c270 con 0x7ff384103070 2024-05-27T07:57:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.578+0000 7ff382000700 1 -- 172.21.15.90:0/2598323230 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff378005590 con 0x7ff384103070 2024-05-27T07:57:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.580+0000 7ff382000700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff36c0b2bf0 0x7ff36c0b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.581+0000 7ff382000700 1 -- 172.21.15.90:0/2598323230 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7ff3780e2a70 con 0x7ff384103070 2024-05-27T07:57:51.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.582+0000 7ff389000700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff36c0b2bf0 0x7ff36c0b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.583+0000 7ff39ae4eb80 1 Processor -- start 2024-05-27T07:57:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.583+0000 7ff39ae4eb80 1 -- start start 2024-05-27T07:57:51.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.583+0000 7ff389000700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff36c0b2bf0 0x7ff36c0b5100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff370005e90 tx=0x7ff370019040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.584+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 0x55b497fc1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.584+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 0x55b497fc1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.584+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 0x55b497fce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.584+0000 7ff39ae4eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b497eefe40 con 0x55b497e42e40 2024-05-27T07:57:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.584+0000 7ff39ae4eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b497edf9e0 con 0x55b497f2e9f0 2024-05-27T07:57:51.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.584+0000 7ff39ae4eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b497ee3a20 con 0x55b497fc1e70 2024-05-27T07:57:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.585+0000 7ff389a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 0x55b497fc1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.585+0000 7ff389a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 0x55b497fc1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48886/0 (socket says 172.21.15.90:48886) 2024-05-27T07:57:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.585+0000 7ff389a00700 1 -- 172.21.15.90:0/3744659465 learned_addr learned my addr 172.21.15.90:0/3744659465 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:51.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.585+0000 7ff389000700 1 --2- 172.21.15.90:0/3744659465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 0x55b497fc1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.585+0000 7ff38a400700 1 --2- 172.21.15.90:0/3744659465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 0x55b497fce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.585+0000 7ff389a00700 1 -- 172.21.15.90:0/3744659465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 msgr2=0x55b497fce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.585+0000 7ff389a00700 1 --2- 172.21.15.90:0/3744659465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 0x55b497fce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.586+0000 7ff389a00700 1 -- 172.21.15.90:0/3744659465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 msgr2=0x55b497fc1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.586+0000 7ff389a00700 1 --2- 172.21.15.90:0/3744659465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 0x55b497fc1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.586+0000 7ff389a00700 1 -- 172.21.15.90:0/3744659465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b497ea8420 con 0x55b497f2e9f0 2024-05-27T07:57:51.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.586+0000 7ff389a00700 1 --2- 172.21.15.90:0/3744659465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 0x55b497fc1330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff3780a5a00 tx=0x7ff3780a5200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e901ed4957191e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:51.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.587+0000 7ff376a00700 1 -- 172.21.15.90:0/3744659465 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3780b2130 con 0x55b497f2e9f0 2024-05-27T07:57:51.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.587+0000 7ff376a00700 1 -- 172.21.15.90:0/3744659465 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3780add30 con 0x55b497f2e9f0 2024-05-27T07:57:51.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.587+0000 7ff376a00700 1 -- 172.21.15.90:0/3744659465 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3780cc0d0 con 0x55b497f2e9f0 2024-05-27T07:57:51.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.593+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3744659465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 msgr2=0x55b497fc1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.593+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3744659465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 0x55b497fc1330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff3780a5a00 tx=0x7ff3780a5200 comp rx=0 tx=0).stop 2024-05-27T07:57:51.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.593+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3744659465 shutdown_connections 2024-05-27T07:57:51.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.593+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3744659465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 0x55b497fce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.593+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3744659465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 0x55b497fc1330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.593+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3744659465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 0x55b497fc1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.593+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3744659465 >> 172.21.15.90:0/3744659465 conn(0x55b497edc390 msgr2=0x55b497fbd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:51.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.594+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3744659465 shutdown_connections 2024-05-27T07:57:51.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.594+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3744659465 wait complete. 2024-05-27T07:57:51.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.594+0000 7ff39ae4eb80 1 -- start start 2024-05-27T07:57:51.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.595+0000 7ff39ae4eb80 1 Processor -- start 2024-05-27T07:57:51.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.596+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497e42e40 0x55b497fcb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.596+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497f2e9f0 0x55b497fcbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.596+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497fc1e70 0x55b497fcc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.596+0000 7ff39ae4eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b497de5fa0 con 0x55b497fc1e70 2024-05-27T07:57:51.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.596+0000 7ff39ae4eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b497e80760 con 0x55b497e42e40 2024-05-27T07:57:51.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.596+0000 7ff39ae4eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b497fceac0 con 0x55b497f2e9f0 2024-05-27T07:57:51.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.596+0000 7ff39ae4eb80 10 client.0 _ll_register_callbacks cb 0x55b497e89c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:51.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.596+0000 7ff389000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497f2e9f0 0x55b497fcbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.597+0000 7ff389a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497e42e40 0x55b497fcb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.597+0000 7ff389a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497e42e40 0x55b497fcb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48894/0 (socket says 172.21.15.90:48894) 2024-05-27T07:57:51.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.597+0000 7ff389a00700 1 -- 172.21.15.90:0/1910635973 learned_addr learned my addr 172.21.15.90:0/1910635973 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:51.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.597+0000 7ff389000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497f2e9f0 0x55b497fcbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:60592/0 (socket says 172.21.15.90:60592) 2024-05-27T07:57:51.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.597+0000 7ff38a400700 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497fc1e70 0x55b497fcc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.598+0000 7ff389a00700 1 -- 172.21.15.90:0/1910635973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497f2e9f0 msgr2=0x55b497fcbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:51.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.598+0000 7ff389a00700 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497f2e9f0 0x55b497fcbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.598+0000 7ff389a00700 1 -- 172.21.15.90:0/1910635973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497fc1e70 msgr2=0x55b497fcc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:51.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.598+0000 7ff389a00700 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497fc1e70 0x55b497fcc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.598+0000 7ff389a00700 1 -- 172.21.15.90:0/1910635973 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b497e2f8d0 con 0x55b497e42e40 2024-05-27T07:57:51.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.598+0000 7ff389a00700 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497e42e40 0x55b497fcb7b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff37800e9c0 tx=0x7ff3780cc380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b8c5cd04a13f0de server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:51.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.598+0000 7ff36b400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:57:51.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.598+0000 7ff36b400700 1 -- 172.21.15.90:0/1910635973 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3780a5840 con 0x55b497e42e40 2024-05-27T07:57:51.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.598+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/1910635973 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3780add30 con 0x55b497e42e40 2024-05-27T07:57:51.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.599+0000 7ff36b400700 1 -- 172.21.15.90:0/1910635973 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7ff37800ce00 con 0x55b497e42e40 2024-05-27T07:57:51.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.599+0000 7ff36b400700 1 -- 172.21.15.90:0/1910635973 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff3780b7050 con 0x55b497e42e40 2024-05-27T07:57:51.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.599+0000 7ff39ae4eb80 10 client.37082 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:51.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.599+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/1910635973 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b497eb0060 con 0x55b497e42e40 2024-05-27T07:57:51.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.599+0000 7ff36b400700 1 -- 172.21.15.90:0/1910635973 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3780add30 con 0x55b497e42e40 2024-05-27T07:57:51.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.600+0000 7ff36b400700 1 -- 172.21.15.90:0/1910635973 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7ff3780a9be0 con 0x55b497e42e40 2024-05-27T07:57:51.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.600+0000 7ff39ae4eb80 10 client.37082 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:51.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.600+0000 7ff39ae4eb80 10 client.37082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:51.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.600+0000 7ff39ae4eb80 20 client.37082 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:51.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.600+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b497fdb8e0 0x55b497fddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.600+0000 7ff39ae4eb80 4 client.37082 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:51.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.600+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/1910635973 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b497ef1d90 con 0x55b497fdb8e0 2024-05-27T07:57:51.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.601+0000 7ff389000700 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b497fdb8e0 0x55b497fddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.602+0000 7ff36b400700 10 client.37082 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:51.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.602+0000 7ff389000700 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b497fdb8e0 0x55b497fddd00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c25299107048b9e server_cookie=f54bf49546210177 in_seq=0 out_seq=0 2024-05-27T07:57:51.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.604+0000 7ff36b400700 1 -- 172.21.15.90:0/1910635973 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b497ef1d90 con 0x55b497fdb8e0 2024-05-27T07:57:51.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.604+0000 7ff36b400700 10 client.37082 handle_command_reply: tid=0 2024-05-27T07:57:51.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.604+0000 7ff39ae4eb80 1 client.37082 shutdown 2024-05-27T07:57:51.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.605+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/1910635973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497e42e40 msgr2=0x55b497fcb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.605+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497e42e40 0x55b497fcb7b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff37800e9c0 tx=0x7ff3780cc380 comp rx=0 tx=0).stop 2024-05-27T07:57:51.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.606+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/1910635973 shutdown_connections 2024-05-27T07:57:51.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.606+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b497fdb8e0 0x55b497fddd00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.606+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497fc1e70 0x55b497fcc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.606+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497e42e40 0x55b497fcb7b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.606+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/1910635973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497f2e9f0 0x55b497fcbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.606+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/1910635973 >> 172.21.15.90:0/1910635973 conn(0x55b497edc390 msgr2=0x55b497fc1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:51.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.606+0000 7ff36b400700 0 client.37082 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:51.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.606+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/1910635973 shutdown_connections 2024-05-27T07:57:51.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.607+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/1910635973 wait complete. 2024-05-27T07:57:51.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.607+0000 7ff39ae4eb80 20 client.37082 trim_cache size 0 max 16384 2024-05-27T07:57:51.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.644+0000 7ff39ae4eb80 1 Processor -- start 2024-05-27T07:57:51.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.644+0000 7ff39ae4eb80 1 -- start start 2024-05-27T07:57:51.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.645+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497e42e40 0x55b497fcc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.645+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497f2e9f0 0x55b497fc6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.645+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497fc1e70 0x55b497fca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.645+0000 7ff39ae4eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff3841c0d60 con 0x55b497f2e9f0 2024-05-27T07:57:51.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.645+0000 7ff39ae4eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff3841a3bc0 con 0x55b497fc1e70 2024-05-27T07:57:51.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.645+0000 7ff39ae4eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3841bffc0 con 0x55b497e42e40 2024-05-27T07:57:51.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.645+0000 7ff389000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497f2e9f0 0x55b497fc6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.646+0000 7ff389000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497f2e9f0 0x55b497fc6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35900/0 (socket says 172.21.15.90:35900) 2024-05-27T07:57:51.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.646+0000 7ff389000700 1 -- 172.21.15.90:0/3622479680 learned_addr learned my addr 172.21.15.90:0/3622479680 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:51.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.646+0000 7ff38a400700 1 --2- 172.21.15.90:0/3622479680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497fc1e70 0x55b497fca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.646+0000 7ff389000700 1 -- 172.21.15.90:0/3622479680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497e42e40 msgr2=0x55b497fcc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:51.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.646+0000 7ff389000700 1 --2- 172.21.15.90:0/3622479680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497e42e40 0x55b497fcc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.646+0000 7ff389000700 1 -- 172.21.15.90:0/3622479680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497fc1e70 msgr2=0x55b497fca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:51.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.646+0000 7ff389000700 1 --2- 172.21.15.90:0/3622479680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497fc1e70 0x55b497fca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.646+0000 7ff389000700 1 -- 172.21.15.90:0/3622479680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b497fbe000 con 0x55b497f2e9f0 2024-05-27T07:57:51.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.646+0000 7ff389000700 1 --2- 172.21.15.90:0/3622479680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497f2e9f0 0x55b497fc6f60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff370009680 tx=0x7ff37001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808dddd04185dd88 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:51.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.647+0000 7ff376000700 1 -- 172.21.15.90:0/3622479680 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370029030 con 0x55b497f2e9f0 2024-05-27T07:57:51.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.647+0000 7ff376000700 1 -- 172.21.15.90:0/3622479680 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff370036040 con 0x55b497f2e9f0 2024-05-27T07:57:51.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.647+0000 7ff376000700 1 -- 172.21.15.90:0/3622479680 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37000b900 con 0x55b497f2e9f0 2024-05-27T07:57:51.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.648+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3622479680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497f2e9f0 msgr2=0x55b497fc6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.648+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3622479680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497f2e9f0 0x55b497fc6f60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff370009680 tx=0x7ff37001f000 comp rx=0 tx=0).stop 2024-05-27T07:57:51.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.649+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3622479680 shutdown_connections 2024-05-27T07:57:51.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.649+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3622479680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497fc1e70 0x55b497fca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.649+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3622479680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497e42e40 0x55b497fcc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.649+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3622479680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497f2e9f0 0x55b497fc6f60 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.649+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3622479680 >> 172.21.15.90:0/3622479680 conn(0x55b497edc390 msgr2=0x55b497fc9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:51.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.649+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3622479680 shutdown_connections 2024-05-27T07:57:51.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.650+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3622479680 wait complete. 2024-05-27T07:57:51.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.651+0000 7ff39ae4eb80 1 -- start start 2024-05-27T07:57:51.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.652+0000 7ff39ae4eb80 1 Processor -- start 2024-05-27T07:57:51.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.653+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 0x55b497fd4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.654+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 0x55b497fc5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.654+0000 7ff389a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 0x55b497fd4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.654+0000 7ff39ae4eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 0x55b497fc6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.654+0000 7ff389a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 0x55b497fd4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35914/0 (socket says 172.21.15.90:35914) 2024-05-27T07:57:51.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.654+0000 7ff389a00700 1 -- 172.21.15.90:0/3188035890 learned_addr learned my addr 172.21.15.90:0/3188035890 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:51.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.654+0000 7ff39ae4eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff3841bf8f0 con 0x55b497e42e40 2024-05-27T07:57:51.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.655+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3188035890 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff3841be480 con 0x55b497f2e9f0 2024-05-27T07:57:51.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.655+0000 7ff389000700 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 0x55b497fc5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.655+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3188035890 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3841beb50 con 0x55b497fc1e70 2024-05-27T07:57:51.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.655+0000 7ff38a400700 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 0x55b497fc6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.655+0000 7ff39ae4eb80 10 client.0 _ll_register_callbacks cb 0x55b497e89c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:51.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.656+0000 7ff38a400700 1 -- 172.21.15.90:0/3188035890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 msgr2=0x55b497fc5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:51.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.656+0000 7ff38a400700 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 0x55b497fc5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.656+0000 7ff38a400700 1 -- 172.21.15.90:0/3188035890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 msgr2=0x55b497fd4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:51.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.656+0000 7ff38a400700 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 0x55b497fd4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.656+0000 7ff38a400700 1 -- 172.21.15.90:0/3188035890 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b497fc67b0 con 0x55b497fc1e70 2024-05-27T07:57:51.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.657+0000 7ff38a400700 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 0x55b497fc6270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff37c007ed0 tx=0x7ff37c016620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e95aee5a7c44f729 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:51.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.657+0000 7ff36b400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:57:51.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.657+0000 7ff36b400700 1 -- 172.21.15.90:0/3188035890 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37c01c070 con 0x55b497fc1e70 2024-05-27T07:57:51.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.658+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3188035890 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff370036040 con 0x55b497fc1e70 2024-05-27T07:57:51.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.658+0000 7ff36b400700 1 -- 172.21.15.90:0/3188035890 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7ff37c018420 con 0x55b497fc1e70 2024-05-27T07:57:51.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.658+0000 7ff36b400700 1 -- 172.21.15.90:0/3188035890 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff37c003a00 con 0x55b497fc1e70 2024-05-27T07:57:51.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.658+0000 7ff39ae4eb80 10 client.27172 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:51.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.658+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3188035890 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b497fd2a30 con 0x55b497fc1e70 2024-05-27T07:57:51.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.659+0000 7ff36b400700 1 -- 172.21.15.90:0/3188035890 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff370036040 con 0x55b497fc1e70 2024-05-27T07:57:51.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.659+0000 7ff36b400700 1 -- 172.21.15.90:0/3188035890 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7ff37c018c40 con 0x55b497fc1e70 2024-05-27T07:57:51.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.659+0000 7ff39ae4eb80 10 client.27172 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:51.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.659+0000 7ff39ae4eb80 10 client.27172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:51.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.659+0000 7ff39ae4eb80 20 client.27172 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:51.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.659+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b4980465d0 0x55b4980489f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:51.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.659+0000 7ff39ae4eb80 4 client.27172 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:57:51.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.659+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3188035890 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b497fbc120 con 0x55b4980465d0 2024-05-27T07:57:51.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.660+0000 7ff389a00700 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b4980465d0 0x55b4980489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:51.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.662+0000 7ff389a00700 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b4980465d0 0x55b4980489f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc431f2b4494fac server_cookie=e9e070a6f489c74f in_seq=0 out_seq=0 2024-05-27T07:57:51.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.662+0000 7ff36b400700 10 client.27172 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:51.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff36b400700 1 -- 172.21.15.90:0/3188035890 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b497fbc120 con 0x55b4980465d0 2024-05-27T07:57:51.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff36b400700 10 client.27172 handle_command_reply: tid=0 2024-05-27T07:57:51.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff39ae4eb80 1 client.27172 shutdown 2024-05-27T07:57:51.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3188035890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 msgr2=0x55b497fc6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 0x55b497fc6270 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff37c007ed0 tx=0x7ff37c016620 comp rx=0 tx=0).stop 2024-05-27T07:57:51.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3188035890 shutdown_connections 2024-05-27T07:57:51.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b4980465d0 0x55b4980489f0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b497fc1e70 0x55b497fc6270 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b497e42e40 0x55b497fd4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff39ae4eb80 1 --2- 172.21.15.90:0/3188035890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b497f2e9f0 0x55b497fc5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3188035890 >> 172.21.15.90:0/3188035890 conn(0x55b497edc390 msgr2=0x55b497fca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:51.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff36b400700 0 client.27172 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:51.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.663+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3188035890 shutdown_connections 2024-05-27T07:57:51.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.664+0000 7ff39ae4eb80 1 -- 172.21.15.90:0/3188035890 wait complete. 2024-05-27T07:57:51.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.664+0000 7ff39ae4eb80 20 client.27172 trim_cache size 0 max 16384 2024-05-27T07:57:51.667 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:57:51.667 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:57:51.667 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:57:51.667 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:57:51.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.665+0000 7ff38c400700 1 -- 172.21.15.90:0/2598323230 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff36c0b2bf0 msgr2=0x7ff36c0b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.665+0000 7ff38c400700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff36c0b2bf0 0x7ff36c0b5100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff370005e90 tx=0x7ff370019040 comp rx=0 tx=0).stop 2024-05-27T07:57:51.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.665+0000 7ff38c400700 1 -- 172.21.15.90:0/2598323230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff384103070 msgr2=0x7ff38410dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:51.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.665+0000 7ff38c400700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff384103070 0x7ff38410dbf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff37800e9f0 tx=0x7ff37800ed00 comp rx=0 tx=0).stop 2024-05-27T07:57:51.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.665+0000 7ff38c400700 1 -- 172.21.15.90:0/2598323230 shutdown_connections 2024-05-27T07:57:51.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.666+0000 7ff38c400700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff384110c20 0x7ff38410e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.666+0000 7ff38c400700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff36c0b2bf0 0x7ff36c0b5100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.666+0000 7ff38c400700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff384103070 0x7ff38410dbf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.666+0000 7ff38c400700 1 --2- 172.21.15.90:0/2598323230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff384103a50 0x7ff38410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:51.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.666+0000 7ff38c400700 1 -- 172.21.15.90:0/2598323230 >> 172.21.15.90:0/2598323230 conn(0x7ff3840fedb0 msgr2=0x7ff384101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:51.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.668+0000 7ff38c400700 1 -- 172.21.15.90:0/2598323230 shutdown_connections 2024-05-27T07:57:51.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:57:51.668+0000 7ff38c400700 1 -- 172.21.15.90:0/2598323230 wait complete. 2024-05-27T07:57:51.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:51 smithi090 ceph-mon[31429]: pgmap v363: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.5 MiB/s wr, 687 op/s 2024-05-27T07:57:52.053 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:51 smithi136 ceph-mon[35324]: pgmap v363: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.5 MiB/s wr, 687 op/s 2024-05-27T07:57:52.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:51 smithi129 ceph-mon[35254]: pgmap v363: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 6.5 MiB/s wr, 687 op/s 2024-05-27T07:57:53.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:53 smithi136 ceph-mon[35324]: pgmap v364: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.2 MiB/s wr, 549 op/s 2024-05-27T07:57:53.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:53 smithi129 ceph-mon[35254]: pgmap v364: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.2 MiB/s wr, 549 op/s 2024-05-27T07:57:53.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:53 smithi090 ceph-mon[31429]: pgmap v364: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.2 MiB/s wr, 549 op/s 2024-05-27T07:57:53.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 455.0 s, 473.2 tps, lat 201.847 ms stddev 107.164 2024-05-27T07:57:54.849 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:57:54.850 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:57:55.551 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:57:55.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:55 smithi090 ceph-mon[31429]: pgmap v365: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.6 MiB/s wr, 753 op/s 2024-05-27T07:57:56.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:55 smithi136 ceph-mon[35324]: pgmap v365: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.6 MiB/s wr, 753 op/s 2024-05-27T07:57:56.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:55 smithi129 ceph-mon[35254]: pgmap v365: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.6 MiB/s wr, 753 op/s 2024-05-27T07:57:57.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:57 smithi090 ceph-mon[31429]: pgmap v366: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.6 MiB/s wr, 603 op/s 2024-05-27T07:57:58.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:57 smithi136 ceph-mon[35324]: pgmap v366: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.6 MiB/s wr, 603 op/s 2024-05-27T07:57:58.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:57 smithi129 ceph-mon[35254]: pgmap v366: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.6 MiB/s wr, 603 op/s 2024-05-27T07:57:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.152+0000 7f337f200640 1 -- 172.21.15.90:0/4100561241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f338010e000 msgr2=0x7f33801104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.152+0000 7f337f200640 1 --2- 172.21.15.90:0/4100561241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f338010e000 0x7f33801104f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3378009900 tx=0x7f337802a3f0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.152+0000 7f337f200640 1 -- 172.21.15.90:0/4100561241 shutdown_connections 2024-05-27T07:57:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.152+0000 7f337f200640 1 --2- 172.21.15.90:0/4100561241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f338010b610 0x7f338010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.152+0000 7f337f200640 1 --2- 172.21.15.90:0/4100561241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f338010e000 0x7f33801104f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.152+0000 7f337f200640 1 --2- 172.21.15.90:0/4100561241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3380104bb0 0x7f3380104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.152+0000 7f337f200640 1 -- 172.21.15.90:0/4100561241 >> 172.21.15.90:0/4100561241 conn(0x7f33800fe210 msgr2=0x7f3380100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.152+0000 7f337f200640 1 -- 172.21.15.90:0/4100561241 shutdown_connections 2024-05-27T07:57:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.152+0000 7f337f200640 1 -- 172.21.15.90:0/4100561241 wait complete. 2024-05-27T07:57:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.153+0000 7f337f200640 1 Processor -- start 2024-05-27T07:57:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.153+0000 7f337f200640 1 -- start start 2024-05-27T07:57:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.154+0000 7f337f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3380104bb0 0x7f338019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.154+0000 7f337f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f338010b610 0x7f338019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.154+0000 7f337f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f338010e000 0x7f338019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.154+0000 7f337f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3380113b10 con 0x7f3380104bb0 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.154+0000 7f337f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3380113990 con 0x7f338010b610 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.154+0000 7f337f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3380113c90 con 0x7f338010e000 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.154+0000 7f337de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3380104bb0 0x7f338019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.154+0000 7f337de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3380104bb0 0x7f338019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35920/0 (socket says 172.21.15.90:35920) 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.154+0000 7f337de00640 1 -- 172.21.15.90:0/852621959 learned_addr learned my addr 172.21.15.90:0/852621959 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337de00640 1 -- 172.21.15.90:0/852621959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f338010e000 msgr2=0x7f338019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337d400640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f338010b610 0x7f338019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337e800640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f338010e000 0x7f338019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337de00640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f338010e000 0x7f338019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337de00640 1 -- 172.21.15.90:0/852621959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f338010b610 msgr2=0x7f338019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337de00640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f338010b610 0x7f338019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337de00640 1 -- 172.21.15.90:0/852621959 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338019f330 con 0x7f3380104bb0 2024-05-27T07:57:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337d400640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f338010b610 0x7f338019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337de00640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3380104bb0 0x7f338019a550 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f337000cd10 tx=0x7f337000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f3376a00640 1 -- 172.21.15.90:0/852621959 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370004b20 con 0x7f3380104bb0 2024-05-27T07:57:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f3376a00640 1 -- 172.21.15.90:0/852621959 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3370004cc0 con 0x7f3380104bb0 2024-05-27T07:57:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f3376a00640 1 -- 172.21.15.90:0/852621959 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337000f670 con 0x7f3380104bb0 2024-05-27T07:57:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337f200640 1 -- 172.21.15.90:0/852621959 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338019f620 con 0x7f3380104bb0 2024-05-27T07:57:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.155+0000 7f337f200640 1 -- 172.21.15.90:0/852621959 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338019fb20 con 0x7f3380104bb0 2024-05-27T07:57:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.158+0000 7f338c26c740 1 Processor -- start 2024-05-27T07:57:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.158+0000 7f338c26c740 1 -- start start 2024-05-27T07:57:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.159+0000 7f338c26c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa99ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.159+0000 7f338c26c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa99bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.159+0000 7f338c26c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa99c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.159+0000 7f338c26c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b7fa90cb90 con 0x55b7fa5f91b0 2024-05-27T07:57:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.159+0000 7f338c26c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b7fa8fc730 con 0x55b7fa5d84d0 2024-05-27T07:57:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.159+0000 7f338c26c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b7fa900770 con 0x55b7fa5d02f0 2024-05-27T07:57:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.159+0000 7f3376a00640 1 -- 172.21.15.90:0/852621959 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3370005540 con 0x7f3380104bb0 2024-05-27T07:57:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.159+0000 7f337e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa99ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.159+0000 7f337e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa99ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35930/0 (socket says 172.21.15.90:35930) 2024-05-27T07:57:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.159+0000 7f337e800640 1 -- 172.21.15.90:0/177051098 learned_addr learned my addr 172.21.15.90:0/177051098 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.160+0000 7f337d400640 1 --2- 172.21.15.90:0/177051098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa99c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.160+0000 7f337de00640 1 --2- 172.21.15.90:0/177051098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa99bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.160+0000 7f337e800640 1 -- 172.21.15.90:0/177051098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 msgr2=0x55b7fa99bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.160+0000 7f337e800640 1 --2- 172.21.15.90:0/177051098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa99bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.160+0000 7f337e800640 1 -- 172.21.15.90:0/177051098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 msgr2=0x55b7fa99c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.160+0000 7f337e800640 1 --2- 172.21.15.90:0/177051098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa99c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.160+0000 7f337e800640 1 -- 172.21.15.90:0/177051098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7fa8958b0 con 0x55b7fa5f91b0 2024-05-27T07:57:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.160+0000 7f337e800640 1 --2- 172.21.15.90:0/177051098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa99ba60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3378009a20 tx=0x7f3378032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de19394c1ba029ec server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.160+0000 7f336f400640 1 -- 172.21.15.90:0/177051098 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337803f070 con 0x55b7fa5f91b0 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.161+0000 7f336f400640 1 -- 172.21.15.90:0/177051098 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f33780044c0 con 0x55b7fa5f91b0 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.161+0000 7f336f400640 1 -- 172.21.15.90:0/177051098 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378035070 con 0x55b7fa5f91b0 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.161+0000 7f3376a00640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f33640b29d0 0x7f33640b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.161+0000 7f3376a00640 1 -- 172.21.15.90:0/852621959 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f33700e2b30 con 0x7f3380104bb0 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.161+0000 7f337de00640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f33640b29d0 0x7f33640b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.162+0000 7f338c26c740 1 -- 172.21.15.90:0/177051098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 msgr2=0x55b7fa99ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.162+0000 7f338c26c740 1 --2- 172.21.15.90:0/177051098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa99ba60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3378009a20 tx=0x7f3378032c40 comp rx=0 tx=0).stop 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.162+0000 7f338c26c740 1 -- 172.21.15.90:0/177051098 shutdown_connections 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.162+0000 7f338c26c740 1 --2- 172.21.15.90:0/177051098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa99c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.162+0000 7f338c26c740 1 --2- 172.21.15.90:0/177051098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa99bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.162+0000 7f338c26c740 1 --2- 172.21.15.90:0/177051098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa99ba60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.162+0000 7f338c26c740 1 -- 172.21.15.90:0/177051098 >> 172.21.15.90:0/177051098 conn(0x55b7fa5ed300 msgr2=0x55b7fa998210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.162+0000 7f338c26c740 1 -- 172.21.15.90:0/177051098 shutdown_connections 2024-05-27T07:57:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.162+0000 7f338c26c740 1 -- 172.21.15.90:0/177051098 wait complete. 2024-05-27T07:57:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.162+0000 7f337de00640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f33640b29d0 0x7f33640b4ee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55b7fa999550 tx=0x7f3370005830 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.163+0000 7f338c26c740 1 -- start start 2024-05-27T07:57:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.163+0000 7f338c26c740 1 Processor -- start 2024-05-27T07:57:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.163+0000 7f338c26c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.163+0000 7f338c26c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.163+0000 7f338c26c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa9aa530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.163+0000 7f338c26c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b7fa5e2790 con 0x55b7fa5f91b0 2024-05-27T07:57:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.163+0000 7f338c26c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b7fa8a4bc0 con 0x55b7fa5d84d0 2024-05-27T07:57:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.163+0000 7f338c26c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b7fa9a8b30 con 0x55b7fa5d02f0 2024-05-27T07:57:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.163+0000 7f338c26c740 10 client.0 _ll_register_callbacks cb 0x55b7fa882400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.164+0000 7f337e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa9aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.164+0000 7f337e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa9aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35942/0 (socket says 172.21.15.90:35942) 2024-05-27T07:57:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.164+0000 7f337e800640 1 -- 172.21.15.90:0/415901189 learned_addr learned my addr 172.21.15.90:0/415901189 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.164+0000 7f337d400640 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.164+0000 7f337e800640 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.164+0000 7f337e800640 1 -- 172.21.15.90:0/415901189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 msgr2=0x55b7fa9b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.164+0000 7f337e800640 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.164+0000 7f337e800640 1 -- 172.21.15.90:0/415901189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 msgr2=0x55b7fa9b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.164+0000 7f337e800640 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.164+0000 7f337e800640 1 -- 172.21.15.90:0/415901189 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7fa611210 con 0x55b7fa5f91b0 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.165+0000 7f337e800640 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa9aa530 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3378005180 tx=0x7f3378009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b0496bf08a43c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.165+0000 7f3363e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.165+0000 7f3363e00640 1 -- 172.21.15.90:0/415901189 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378035070 con 0x55b7fa5f91b0 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.165+0000 7f3363e00640 1 -- 172.21.15.90:0/415901189 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f3378031610 con 0x55b7fa5f91b0 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.165+0000 7f338c26c740 1 -- 172.21.15.90:0/415901189 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b7fa90eae0 con 0x55b7fa5f91b0 2024-05-27T07:57:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.165+0000 7f3363e00640 1 -- 172.21.15.90:0/415901189 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f337803f040 con 0x55b7fa5f91b0 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.165+0000 7f338c26c740 10 client.17517 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.165+0000 7f338c26c740 1 -- 172.21.15.90:0/415901189 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b7fa6dd620 con 0x55b7fa5f91b0 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.166+0000 7f3363e00640 1 -- 172.21.15.90:0/415901189 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b7fa90eae0 con 0x55b7fa5f91b0 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.166+0000 7f3363e00640 1 -- 172.21.15.90:0/415901189 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f337803b580 con 0x55b7fa5f91b0 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.166+0000 7f338c26c740 10 client.17517 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.166+0000 7f338c26c740 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.166+0000 7f338c26c740 20 client.17517 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.166+0000 7f338c26c740 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b7fa5b7d80 0x55b7fa9bc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.166+0000 7f338c26c740 4 client.17517 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.166+0000 7f338c26c740 1 -- 172.21.15.90:0/415901189 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7fa9082c0 con 0x55b7fa5b7d80 2024-05-27T07:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.167+0000 7f337d400640 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b7fa5b7d80 0x55b7fa9bc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.168+0000 7f337d400640 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b7fa5b7d80 0x55b7fa9bc070 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e98193d6ce287c8 server_cookie=b6fad768c0bfc83c in_seq=0 out_seq=0 2024-05-27T07:57:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.168+0000 7f3363e00640 10 client.17517 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f3363e00640 1 -- 172.21.15.90:0/415901189 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b7fa9082c0 con 0x55b7fa5b7d80 2024-05-27T07:57:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f3363e00640 10 client.17517 handle_command_reply: tid=0 2024-05-27T07:57:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 client.17517 shutdown 2024-05-27T07:57:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 -- 172.21.15.90:0/415901189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 msgr2=0x55b7fa9aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa9aa530 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3378005180 tx=0x7f3378009260 comp rx=0 tx=0).stop 2024-05-27T07:57:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 -- 172.21.15.90:0/415901189 shutdown_connections 2024-05-27T07:57:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b7fa5b7d80 0x55b7fa9bc070 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7fa9aa530 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f3363e00640 0 client.17517 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 --2- 172.21.15.90:0/415901189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 -- 172.21.15.90:0/415901189 >> 172.21.15.90:0/415901189 conn(0x55b7fa5ed300 msgr2=0x55b7fa99b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 -- 172.21.15.90:0/415901189 shutdown_connections 2024-05-27T07:57:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 1 -- 172.21.15.90:0/415901189 wait complete. 2024-05-27T07:57:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.170+0000 7f338c26c740 20 client.17517 trim_cache size 0 max 16384 2024-05-27T07:57:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.192+0000 7f338c26c740 1 Processor -- start 2024-05-27T07:57:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.192+0000 7f338c26c740 1 -- start start 2024-05-27T07:57:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.192+0000 7f338c26c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.192+0000 7f338c26c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.192+0000 7f338c26c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5f91b0 0x55b7fa9b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.192+0000 7f338c26c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f33801b07b0 con 0x55b7fa5d84d0 2024-05-27T07:57:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.192+0000 7f338c26c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33801ad800 con 0x55b7fa5f91b0 2024-05-27T07:57:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.192+0000 7f338c26c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33801afa10 con 0x55b7fa5d02f0 2024-05-27T07:57:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.193+0000 7f337d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.193+0000 7f337d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9a57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35954/0 (socket says 172.21.15.90:35954) 2024-05-27T07:57:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.193+0000 7f337d400640 1 -- 172.21.15.90:0/295527686 learned_addr learned my addr 172.21.15.90:0/295527686 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.193+0000 7f337e800640 1 --2- 172.21.15.90:0/295527686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.193+0000 7f337e800640 1 --2- 172.21.15.90:0/295527686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5f91b0 0x55b7fa9b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.194+0000 7f337e800640 1 -- 172.21.15.90:0/295527686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5f91b0 msgr2=0x55b7fa9b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.194+0000 7f337e800640 1 --2- 172.21.15.90:0/295527686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5f91b0 0x55b7fa9b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.194+0000 7f337e800640 1 -- 172.21.15.90:0/295527686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5d84d0 msgr2=0x55b7fa9a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.194+0000 7f337e800640 1 --2- 172.21.15.90:0/295527686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.194+0000 7f337e800640 1 -- 172.21.15.90:0/295527686 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7fa9a9860 con 0x55b7fa5d02f0 2024-05-27T07:57:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.194+0000 7f337d400640 1 --2- 172.21.15.90:0/295527686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:57:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.195+0000 7f337e800640 1 --2- 172.21.15.90:0/295527686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9aa540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f33780428c0 tx=0x7f337802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9317c4c5aebdec4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.195+0000 7f336ea00640 1 -- 172.21.15.90:0/295527686 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378035040 con 0x55b7fa5d02f0 2024-05-27T07:57:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.195+0000 7f336ea00640 1 -- 172.21.15.90:0/295527686 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3378031610 con 0x55b7fa5d02f0 2024-05-27T07:57:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.195+0000 7f336ea00640 1 -- 172.21.15.90:0/295527686 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378032c40 con 0x55b7fa5d02f0 2024-05-27T07:57:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.197+0000 7f338c26c740 1 -- 172.21.15.90:0/295527686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 msgr2=0x55b7fa9aa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:57:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.197+0000 7f338c26c740 1 --2- 172.21.15.90:0/295527686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9aa540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f33780428c0 tx=0x7f337802a3f0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.197+0000 7f338c26c740 1 -- 172.21.15.90:0/295527686 shutdown_connections 2024-05-27T07:57:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.197+0000 7f338c26c740 1 --2- 172.21.15.90:0/295527686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5f91b0 0x55b7fa9b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.197+0000 7f338c26c740 1 --2- 172.21.15.90:0/295527686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.197+0000 7f338c26c740 1 --2- 172.21.15.90:0/295527686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9aa540 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.197+0000 7f338c26c740 1 -- 172.21.15.90:0/295527686 >> 172.21.15.90:0/295527686 conn(0x55b7fa5ed300 msgr2=0x55b7fa9afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:57:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.197+0000 7f338c26c740 1 -- 172.21.15.90:0/295527686 shutdown_connections 2024-05-27T07:57:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.197+0000 7f338c26c740 1 -- 172.21.15.90:0/295527686 wait complete. 2024-05-27T07:57:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.198+0000 7f338c26c740 1 -- start start 2024-05-27T07:57:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.198+0000 7f338c26c740 1 Processor -- start 2024-05-27T07:57:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f338c26c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f338c26c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f338c26c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7faa1e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f338c26c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f33801af340 con 0x55b7fa5f91b0 2024-05-27T07:57:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f338c26c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33801aded0 con 0x55b7fa5d84d0 2024-05-27T07:57:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f338c26c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33801ae5a0 con 0x55b7fa5d02f0 2024-05-27T07:57:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f338c26c740 10 client.0 _ll_register_callbacks cb 0x7f3378032b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:57:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f337e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7faa1e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f337e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7faa1e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35966/0 (socket says 172.21.15.90:35966) 2024-05-27T07:57:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f337e800640 1 -- 172.21.15.90:0/1807170614 learned_addr learned my addr 172.21.15.90:0/1807170614 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:57:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.199+0000 7f337e800640 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.200+0000 7f337d400640 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.200+0000 7f337e800640 1 -- 172.21.15.90:0/1807170614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 msgr2=0x55b7fa9b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.200+0000 7f337e800640 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.200+0000 7f337e800640 1 -- 172.21.15.90:0/1807170614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 msgr2=0x55b7fa9b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:57:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.200+0000 7f337e800640 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:57:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.200+0000 7f337e800640 1 -- 172.21.15.90:0/1807170614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7fa9a5890 con 0x55b7fa5f91b0 2024-05-27T07:57:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.200+0000 7f337d400640 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:57:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.200+0000 7f337e800640 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7faa1e630 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3378005820 tx=0x7f337803cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79334cdfc92c9d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:57:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.200+0000 7f3363e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:57:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.201+0000 7f3363e00640 1 -- 172.21.15.90:0/1807170614 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378035040 con 0x55b7fa5f91b0 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.201+0000 7f3363e00640 1 -- 172.21.15.90:0/1807170614 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f3378065b90 con 0x55b7fa5f91b0 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.201+0000 7f338c26c740 1 -- 172.21.15.90:0/1807170614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3378031610 con 0x55b7fa5f91b0 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.201+0000 7f3363e00640 1 -- 172.21.15.90:0/1807170614 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3378005420 con 0x55b7fa5f91b0 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.201+0000 7f338c26c740 10 client.17523 fetch_fsmap learned FSMap version 64 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.201+0000 7f338c26c740 1 -- 172.21.15.90:0/1807170614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b7faa1f8f0 con 0x55b7fa5f91b0 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.202+0000 7f3363e00640 1 -- 172.21.15.90:0/1807170614 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3378031610 con 0x55b7fa5f91b0 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.202+0000 7f3363e00640 1 -- 172.21.15.90:0/1807170614 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f3378077ba0 con 0x55b7fa5f91b0 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.202+0000 7f338c26c740 10 client.17523 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.202+0000 7f338c26c740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.202+0000 7f338c26c740 20 client.17523 populate_metadata read hostname 'smithi090' 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.202+0000 7f338c26c740 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b7fa58d1c0 0x55b7faa21dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.202+0000 7f338c26c740 4 client.17523 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:57:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.202+0000 7f338c26c740 1 -- 172.21.15.90:0/1807170614 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7fa8824b0 con 0x55b7fa58d1c0 2024-05-27T07:57:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.203+0000 7f337d400640 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b7fa58d1c0 0x55b7faa21dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:57:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.204+0000 7f337d400640 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b7fa58d1c0 0x55b7faa21dc0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=677e9174af065b94 server_cookie=4c8b8aac884db07a in_seq=0 out_seq=0 2024-05-27T07:57:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:57:58.204+0000 7f3363e00640 10 client.17523 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:57:58.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 460.0 s, 521.6 tps, lat 195.397 ms stddev 244.955 2024-05-27T07:58:00.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:57:59 smithi136 ceph-mon[35324]: pgmap v367: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.2 MiB/s wr, 642 op/s 2024-05-27T07:58:00.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:57:59 smithi129 ceph-mon[35254]: pgmap v367: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.2 MiB/s wr, 642 op/s 2024-05-27T07:58:00.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:57:59 smithi090 ceph-mon[31429]: pgmap v367: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.2 MiB/s wr, 642 op/s 2024-05-27T07:58:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.070+0000 7f3363e00640 1 -- 172.21.15.90:0/1807170614 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b7fa8824b0 con 0x55b7fa58d1c0 2024-05-27T07:58:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.070+0000 7f3363e00640 10 client.17523 handle_command_reply: tid=0 2024-05-27T07:58:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.071+0000 7f338c26c740 1 client.17523 shutdown 2024-05-27T07:58:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.071+0000 7f338c26c740 1 -- 172.21.15.90:0/1807170614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 msgr2=0x55b7faa1e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.071+0000 7f338c26c740 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7faa1e630 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3378005820 tx=0x7f337803cfc0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.072+0000 7f338c26c740 1 -- 172.21.15.90:0/1807170614 shutdown_connections 2024-05-27T07:58:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.072+0000 7f338c26c740 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b7fa58d1c0 0x55b7faa21dc0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.072+0000 7f338c26c740 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b7fa5d84d0 0x55b7fa9b4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.072+0000 7f338c26c740 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b7fa5f91b0 0x55b7faa1e630 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.072+0000 7f338c26c740 1 --2- 172.21.15.90:0/1807170614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b7fa5d02f0 0x55b7fa9b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.072+0000 7f338c26c740 1 -- 172.21.15.90:0/1807170614 >> 172.21.15.90:0/1807170614 conn(0x55b7fa5ed300 msgr2=0x55b7fa9aa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.072+0000 7f3363e00640 0 client.17523 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.072+0000 7f338c26c740 1 -- 172.21.15.90:0/1807170614 shutdown_connections 2024-05-27T07:58:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.072+0000 7f338c26c740 1 -- 172.21.15.90:0/1807170614 wait complete. 2024-05-27T07:58:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.073+0000 7f338c26c740 20 client.17523 trim_cache size 0 max 16384 2024-05-27T07:58:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:58:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:58:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:58:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.074+0000 7f337f200640 1 -- 172.21.15.90:0/852621959 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f33640b29d0 msgr2=0x7f33640b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.074+0000 7f337f200640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f33640b29d0 0x7f33640b4ee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55b7fa999550 tx=0x7f3370005830 comp rx=0 tx=0).stop 2024-05-27T07:58:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.074+0000 7f337f200640 1 -- 172.21.15.90:0/852621959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3380104bb0 msgr2=0x7f338019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.074+0000 7f337f200640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3380104bb0 0x7f338019a550 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f337000cd10 tx=0x7f337000a620 comp rx=0 tx=0).stop 2024-05-27T07:58:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.074+0000 7f337f200640 1 -- 172.21.15.90:0/852621959 shutdown_connections 2024-05-27T07:58:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.074+0000 7f337f200640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f33640b29d0 0x7f33640b4ee0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.074+0000 7f337f200640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f338010b610 0x7f338019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.074+0000 7f337f200640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f338010e000 0x7f338019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.074+0000 7f337f200640 1 --2- 172.21.15.90:0/852621959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3380104bb0 0x7f338019a550 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.074+0000 7f337f200640 1 -- 172.21.15.90:0/852621959 >> 172.21.15.90:0/852621959 conn(0x7f33800fe210 msgr2=0x7f338010e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.075+0000 7f337f200640 1 -- 172.21.15.90:0/852621959 shutdown_connections 2024-05-27T07:58:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:01.075+0000 7f337f200640 1 -- 172.21.15.90:0/852621959 wait complete. 2024-05-27T07:58:01.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.483+0000 7f5dbce00700 1 -- 172.21.15.90:0/3136718056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db8109670 msgr2=0x7f5db8110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:01.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.484+0000 7f5dbce00700 1 --2- 172.21.15.90:0/3136718056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db8109670 0x7f5db8110fd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5db000b0d0 tx=0x7f5db000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.484+0000 7f5dbce00700 1 -- 172.21.15.90:0/3136718056 shutdown_connections 2024-05-27T07:58:01.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.484+0000 7f5dbce00700 1 --2- 172.21.15.90:0/3136718056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db8109670 0x7f5db8110fd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.484+0000 7f5dbce00700 1 --2- 172.21.15.90:0/3136718056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db81043f0 0x7f5db8104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.484+0000 7f5dbce00700 1 --2- 172.21.15.90:0/3136718056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db8108d20 0x7f5db8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.484+0000 7f5dbce00700 1 -- 172.21.15.90:0/3136718056 >> 172.21.15.90:0/3136718056 conn(0x7f5db8079d00 msgr2=0x7f5db807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:01.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.485+0000 7f5dbce00700 1 -- 172.21.15.90:0/3136718056 shutdown_connections 2024-05-27T07:58:01.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.485+0000 7f5dbce00700 1 -- 172.21.15.90:0/3136718056 wait complete. 2024-05-27T07:58:01.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.486+0000 7f5dbce00700 1 Processor -- start 2024-05-27T07:58:01.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.491+0000 7f5dbce00700 1 -- start start 2024-05-27T07:58:01.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.491+0000 7f5dbce00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db81043f0 0x7f5db81a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:01.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.491+0000 7f5dbce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db8108d20 0x7f5db81a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:01.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.491+0000 7f5dbce00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db8109670 0x7f5db81acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:01.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.491+0000 7f5dbce00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81141a0 con 0x7f5db8108d20 2024-05-27T07:58:01.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.491+0000 7f5dbce00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8114020 con 0x7f5db81043f0 2024-05-27T07:58:01.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.491+0000 7f5dbce00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8114320 con 0x7f5db8109670 2024-05-27T07:58:01.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.492+0000 7f5db7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db8109670 0x7f5db81acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:01.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.492+0000 7f5db7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db8109670 0x7f5db81acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56850/0 (socket says 172.21.15.90:56850) 2024-05-27T07:58:01.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.492+0000 7f5db7e00700 1 -- 172.21.15.90:0/1950597054 learned_addr learned my addr 172.21.15.90:0/1950597054 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:01.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.492+0000 7f5db6a00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db8108d20 0x7f5db81a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:01.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.493+0000 7f5db7400700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db81043f0 0x7f5db81a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:01.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.493+0000 7f5db7e00700 1 -- 172.21.15.90:0/1950597054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db81043f0 msgr2=0x7f5db81a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:01.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.493+0000 7f5db7e00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db81043f0 0x7f5db81a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.493+0000 7f5db7e00700 1 -- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db8108d20 msgr2=0x7f5db81a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:01.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.493+0000 7f5db7e00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db8108d20 0x7f5db81a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.493+0000 7f5db7e00700 1 -- 172.21.15.90:0/1950597054 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db81ad440 con 0x7f5db8109670 2024-05-27T07:58:01.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.493+0000 7f5db6a00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db8108d20 0x7f5db81a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:01.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.493+0000 7f5db7400700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db81043f0 0x7f5db81a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:01.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.494+0000 7f5db7e00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db8109670 0x7f5db81acf00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5db0007d10 tx=0x7f5db0007df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:01.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.494+0000 7f5dafe00700 1 -- 172.21.15.90:0/1950597054 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db001f070 con 0x7f5db8109670 2024-05-27T07:58:01.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.494+0000 7f5dbce00700 1 -- 172.21.15.90:0/1950597054 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db81ad670 con 0x7f5db8109670 2024-05-27T07:58:01.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.495+0000 7f5dbce00700 1 -- 172.21.15.90:0/1950597054 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db81adb20 con 0x7f5db8109670 2024-05-27T07:58:01.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.495+0000 7f5dafe00700 1 -- 172.21.15.90:0/1950597054 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5db0003e10 con 0x7f5db8109670 2024-05-27T07:58:01.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.495+0000 7f5dafe00700 1 -- 172.21.15.90:0/1950597054 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db001b820 con 0x7f5db8109670 2024-05-27T07:58:01.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.496+0000 7f5dbce00700 1 -- 172.21.15.90:0/1950597054 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d98005330 con 0x7f5db8109670 2024-05-27T07:58:01.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.502+0000 7f5dafe00700 1 -- 172.21.15.90:0/1950597054 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5db0022070 con 0x7f5db8109670 2024-05-27T07:58:01.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.504+0000 7f5dafe00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d9c0b2af0 0x7f5d9c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:01.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.504+0000 7f5dafe00700 1 -- 172.21.15.90:0/1950597054 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9496+0+0 (secure 0 0 0) 0x7f5db000f020 con 0x7f5db8109670 2024-05-27T07:58:01.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.504+0000 7f5db7400700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d9c0b2af0 0x7f5d9c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:01.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.506+0000 7f5db7400700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d9c0b2af0 0x7f5d9c0b5000 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5da8005fd0 tx=0x7f5da8005ee0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:01.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.515+0000 7f5dafe00700 1 -- 172.21.15.90:0/1950597054 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5db0029020 con 0x7f5db8109670 2024-05-27T07:58:01.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:01 smithi090 ceph-mon[31429]: pgmap v368: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.7 MiB/s wr, 689 op/s 2024-05-27T07:58:01.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.968+0000 7f5dbce00700 1 -- 172.21.15.90:0/1950597054 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d98005620 con 0x7f5db8109670 2024-05-27T07:58:01.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.970+0000 7f5dafe00700 1 -- 172.21.15.90:0/1950597054 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+9456 (secure 0 0 0) 0x7f5db00ab900 con 0x7f5db8109670 2024-05-27T07:58:01.973 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:58:01.973 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":64,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:56:32.161929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T07:58:01.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.974+0000 7f5dbce00700 1 -- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d9c0b2af0 msgr2=0x7f5d9c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:01.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.974+0000 7f5dbce00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d9c0b2af0 0x7f5d9c0b5000 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5da8005fd0 tx=0x7f5da8005ee0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.974+0000 7f5dbce00700 1 -- 172.21.15.90:0/1950597054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db8109670 msgr2=0x7f5db81acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:01.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.974+0000 7f5dbce00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db8109670 0x7f5db81acf00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5db0007d10 tx=0x7f5db0007df0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.974+0000 7f5dbce00700 1 -- 172.21.15.90:0/1950597054 shutdown_connections 2024-05-27T07:58:01.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.974+0000 7f5dbce00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5db8109670 0x7f5db81acf00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.974+0000 7f5dbce00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f5d9c0b2af0 0x7f5d9c0b5000 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.974+0000 7f5dbce00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db81043f0 0x7f5db81a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.974+0000 7f5dbce00700 1 --2- 172.21.15.90:0/1950597054 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5db8108d20 0x7f5db81a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:01.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.974+0000 7f5dbce00700 1 -- 172.21.15.90:0/1950597054 >> 172.21.15.90:0/1950597054 conn(0x7f5db8079d00 msgr2=0x7f5db8101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:01.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.975+0000 7f5dbce00700 1 -- 172.21.15.90:0/1950597054 shutdown_connections 2024-05-27T07:58:01.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:01.975+0000 7f5dbce00700 1 -- 172.21.15.90:0/1950597054 wait complete. 2024-05-27T07:58:01.979 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 64 2024-05-27T07:58:02.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:01 smithi136 ceph-mon[35324]: pgmap v368: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.7 MiB/s wr, 689 op/s 2024-05-27T07:58:02.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:01 smithi129 ceph-mon[35254]: pgmap v368: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 9.7 MiB/s wr, 689 op/s 2024-05-27T07:58:02.895 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:58:03.049 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:02 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1950597054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:58:03.064 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:02 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1950597054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:58:03.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:02 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1950597054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:58:03.703 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:03.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 465.0 s, 384.6 tps, lat 259.727 ms stddev 460.421 2024-05-27T07:58:04.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:03 smithi129 ceph-mon[35254]: pgmap v369: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7.5 MiB/s wr, 480 op/s 2024-05-27T07:58:04.087 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:03 smithi090 ceph-mon[31429]: pgmap v369: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7.5 MiB/s wr, 480 op/s 2024-05-27T07:58:04.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:03 smithi136 ceph-mon[35324]: pgmap v369: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7.5 MiB/s wr, 480 op/s 2024-05-27T07:58:06.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:05 smithi129 ceph-mon[35254]: pgmap v370: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9.5 MiB/s wr, 601 op/s 2024-05-27T07:58:06.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:05 smithi129 ceph-mon[35254]: osdmap e111: 12 total, 12 up, 12 in 2024-05-27T07:58:06.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:05 smithi090 ceph-mon[31429]: pgmap v370: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9.5 MiB/s wr, 601 op/s 2024-05-27T07:58:06.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:05 smithi090 ceph-mon[31429]: osdmap e111: 12 total, 12 up, 12 in 2024-05-27T07:58:06.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:05 smithi136 ceph-mon[35324]: pgmap v370: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 9.5 MiB/s wr, 601 op/s 2024-05-27T07:58:06.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:05 smithi136 ceph-mon[35324]: osdmap e111: 12 total, 12 up, 12 in 2024-05-27T07:58:08.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:07 smithi129 ceph-mon[35254]: pgmap v372: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 474 op/s 2024-05-27T07:58:08.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:07 smithi090 ceph-mon[31429]: pgmap v372: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 474 op/s 2024-05-27T07:58:08.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:07 smithi136 ceph-mon[35324]: pgmap v372: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.5 MiB/s wr, 474 op/s 2024-05-27T07:58:08.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 470.0 s, 533.8 tps, lat 188.459 ms stddev 198.917 2024-05-27T07:58:10.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:09 smithi129 ceph-mon[35254]: pgmap v373: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 492 op/s 2024-05-27T07:58:10.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:09 smithi090 ceph-mon[31429]: pgmap v373: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 492 op/s 2024-05-27T07:58:10.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:09 smithi136 ceph-mon[35324]: pgmap v373: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 492 op/s 2024-05-27T07:58:12.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:11 smithi129 ceph-mon[35254]: pgmap v374: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 407 op/s 2024-05-27T07:58:12.067 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:11 smithi090 ceph-mon[31429]: pgmap v374: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 407 op/s 2024-05-27T07:58:12.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:11 smithi136 ceph-mon[35324]: pgmap v374: 129 pgs: 5 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 122 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.7 MiB/s wr, 407 op/s 2024-05-27T07:58:13.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.006+0000 7fef8a600700 1 -- 172.21.15.90:0/2219600646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef841045e0 msgr2=0x7fef841049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:13.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.006+0000 7fef8a600700 1 --2- 172.21.15.90:0/2219600646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef841045e0 0x7fef841049f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fef740099c0 tx=0x7fef74009cd0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.009+0000 7fef8a600700 1 -- 172.21.15.90:0/2219600646 shutdown_connections 2024-05-27T07:58:13.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.009+0000 7fef8a600700 1 --2- 172.21.15.90:0/2219600646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef84109860 0x7fef841111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.009+0000 7fef8a600700 1 --2- 172.21.15.90:0/2219600646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef841045e0 0x7fef841049f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.009+0000 7fef8a600700 1 --2- 172.21.15.90:0/2219600646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef84108f10 0x7fef84109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.009+0000 7fef8a600700 1 -- 172.21.15.90:0/2219600646 >> 172.21.15.90:0/2219600646 conn(0x7fef84100030 msgr2=0x7fef84102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:13.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.010+0000 7fef8a600700 1 -- 172.21.15.90:0/2219600646 shutdown_connections 2024-05-27T07:58:13.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.010+0000 7fef8a600700 1 -- 172.21.15.90:0/2219600646 wait complete. 2024-05-27T07:58:13.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.012+0000 7fef8a600700 1 Processor -- start 2024-05-27T07:58:13.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.012+0000 7fef8a600700 1 -- start start 2024-05-27T07:58:13.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.012+0000 7fef8a600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef841045e0 0x7fef841a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.012+0000 7fef8a600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef84108f10 0x7fef841a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.013+0000 7fef8a600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef84109860 0x7fef841a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.013+0000 7fef8a600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fef84112210 con 0x7fef84108f10 2024-05-27T07:58:13.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.013+0000 7fef8a600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef84112090 con 0x7fef841045e0 2024-05-27T07:58:13.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.013+0000 7fef8a600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef84112390 con 0x7fef84109860 2024-05-27T07:58:13.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.013+0000 7fef83400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef841045e0 0x7fef841a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.013+0000 7fef83400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef841045e0 0x7fef841a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43600/0 (socket says 172.21.15.90:43600) 2024-05-27T07:58:13.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.013+0000 7fef83400700 1 -- 172.21.15.90:0/4146907014 learned_addr learned my addr 172.21.15.90:0/4146907014 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:13.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.014+0000 7fef82a00700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef84108f10 0x7fef841a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.014+0000 7fef83e00700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef84109860 0x7fef841a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.014+0000 7fef83400700 1 -- 172.21.15.90:0/4146907014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef84109860 msgr2=0x7fef841a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:13.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.014+0000 7fef83400700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef84109860 0x7fef841a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.014+0000 7fef83400700 1 -- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef84108f10 msgr2=0x7fef841a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:13.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.014+0000 7fef83400700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef84108f10 0x7fef841a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.014+0000 7fef83400700 1 -- 172.21.15.90:0/4146907014 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef841a62a0 con 0x7fef841045e0 2024-05-27T07:58:13.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.014+0000 7fef82a00700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef84108f10 0x7fef841a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:13.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.014+0000 7fef83e00700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef84109860 0x7fef841a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:13.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.015+0000 7fef83400700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef841045e0 0x7fef841a13d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef74005910 tx=0x7fef74004690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:13.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.015+0000 7fef7be00700 1 -- 172.21.15.90:0/4146907014 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74026070 con 0x7fef841045e0 2024-05-27T07:58:13.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.015+0000 7fef7be00700 1 -- 172.21.15.90:0/4146907014 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef7400f460 con 0x7fef841045e0 2024-05-27T07:58:13.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.015+0000 7fef7be00700 1 -- 172.21.15.90:0/4146907014 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74009e60 con 0x7fef841045e0 2024-05-27T07:58:13.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.015+0000 7fef8a600700 1 -- 172.21.15.90:0/4146907014 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef841a6530 con 0x7fef841045e0 2024-05-27T07:58:13.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.015+0000 7fef8a600700 1 -- 172.21.15.90:0/4146907014 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef841a69c0 con 0x7fef841045e0 2024-05-27T07:58:13.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.022+0000 7fef7be00700 1 -- 172.21.15.90:0/4146907014 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fef7402a3f0 con 0x7fef841045e0 2024-05-27T07:58:13.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.025+0000 7fef7be00700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fef680b2a70 0x7fef680b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.025+0000 7fef7be00700 1 -- 172.21.15.90:0/4146907014 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9584+0+0 (secure 0 0 0) 0x7fef740e43b0 con 0x7fef841045e0 2024-05-27T07:58:13.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.026+0000 7fef82a00700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fef680b2a70 0x7fef680b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.245+0000 7fef82a00700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fef680b2a70 0x7fef680b4f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fef84107500 tx=0x7fef70009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:13.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.246+0000 7fef99147b80 1 Processor -- start 2024-05-27T07:58:13.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.247+0000 7fef99147b80 1 -- start start 2024-05-27T07:58:13.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.248+0000 7fef99147b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 0x5563c734a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.248+0000 7fef99147b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c734ad30 0x5563c7344870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.248+0000 7fef99147b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c73451c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.248+0000 7fef99147b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5563c7272e40 con 0x5563c7344db0 2024-05-27T07:58:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.248+0000 7fef99147b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5563c72629e0 con 0x5563c71c5e40 2024-05-27T07:58:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.248+0000 7fef99147b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5563c7266a20 con 0x5563c734ad30 2024-05-27T07:58:13.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.248+0000 7fef82a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c734ad30 0x5563c7344870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.248+0000 7fef82a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c734ad30 0x5563c7344870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37256/0 (socket says 172.21.15.90:37256) 2024-05-27T07:58:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.248+0000 7fef82a00700 1 -- 172.21.15.90:0/3487093257 learned_addr learned my addr 172.21.15.90:0/3487093257 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.248+0000 7fef83400700 1 --2- 172.21.15.90:0/3487093257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 0x5563c734a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.249+0000 7fef82a00700 1 -- 172.21.15.90:0/3487093257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 msgr2=0x5563c734a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.249+0000 7fef82a00700 1 --2- 172.21.15.90:0/3487093257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 0x5563c734a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.249+0000 7fef82a00700 1 -- 172.21.15.90:0/3487093257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 msgr2=0x5563c73451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.249+0000 7fef82a00700 1 --2- 172.21.15.90:0/3487093257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c73451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.249+0000 7fef82a00700 1 -- 172.21.15.90:0/3487093257 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c722b420 con 0x5563c734ad30 2024-05-27T07:58:13.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.251+0000 7fef82a00700 1 --2- 172.21.15.90:0/3487093257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c734ad30 0x5563c7344870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef70010cd0 tx=0x7fef70013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7a04e325c5ddfc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:13.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.251+0000 7fef78c00700 1 -- 172.21.15.90:0/3487093257 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70023070 con 0x5563c734ad30 2024-05-27T07:58:13.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.251+0000 7fef78c00700 1 -- 172.21.15.90:0/3487093257 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef70012b90 con 0x5563c734ad30 2024-05-27T07:58:13.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.252+0000 7fef78c00700 1 -- 172.21.15.90:0/3487093257 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7001fe60 con 0x5563c734ad30 2024-05-27T07:58:13.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.253+0000 7fef99147b80 1 -- 172.21.15.90:0/3487093257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c734ad30 msgr2=0x5563c7344870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:13.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.253+0000 7fef99147b80 1 --2- 172.21.15.90:0/3487093257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c734ad30 0x5563c7344870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fef70010cd0 tx=0x7fef70013000 comp rx=0 tx=0).stop 2024-05-27T07:58:13.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.253+0000 7fef99147b80 1 -- 172.21.15.90:0/3487093257 shutdown_connections 2024-05-27T07:58:13.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.253+0000 7fef99147b80 1 --2- 172.21.15.90:0/3487093257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c73451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.253+0000 7fef99147b80 1 --2- 172.21.15.90:0/3487093257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 0x5563c734a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.253+0000 7fef99147b80 1 --2- 172.21.15.90:0/3487093257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c734ad30 0x5563c7344870 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.253+0000 7fef99147b80 1 -- 172.21.15.90:0/3487093257 >> 172.21.15.90:0/3487093257 conn(0x5563c7225fc0 msgr2=0x5563c733f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:13.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.254+0000 7fef99147b80 1 -- 172.21.15.90:0/3487093257 shutdown_connections 2024-05-27T07:58:13.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.254+0000 7fef99147b80 1 -- 172.21.15.90:0/3487093257 wait complete. 2024-05-27T07:58:13.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.255+0000 7fef99147b80 1 -- start start 2024-05-27T07:58:13.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.256+0000 7fef99147b80 1 Processor -- start 2024-05-27T07:58:13.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.258+0000 7fef99147b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c71c5e40 0x5563c734e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.259+0000 7fef99147b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c734ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.259+0000 7fef99147b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 0x5563c734ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.259+0000 7fef99147b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5563c7168fa0 con 0x5563c7344db0 2024-05-27T07:58:13.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.259+0000 7fef99147b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5563c7203760 con 0x5563c734ad30 2024-05-27T07:58:13.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.259+0000 7fef99147b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5563c7351800 con 0x5563c71c5e40 2024-05-27T07:58:13.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.259+0000 7fef99147b80 10 client.0 _ll_register_callbacks cb 0x5563c720cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.262+0000 7fef83400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c71c5e40 0x5563c734e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.262+0000 7fef83400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c71c5e40 0x5563c734e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37264/0 (socket says 172.21.15.90:37264) 2024-05-27T07:58:13.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.262+0000 7fef83400700 1 -- 172.21.15.90:0/1959254433 learned_addr learned my addr 172.21.15.90:0/1959254433 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:13.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.264+0000 7fef83e00700 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 0x5563c734ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.266+0000 7fef82a00700 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c734ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.267+0000 7fef83e00700 1 -- 172.21.15.90:0/1959254433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c71c5e40 msgr2=0x5563c734e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.267+0000 7fef83e00700 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c71c5e40 0x5563c734e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.267+0000 7fef83e00700 1 -- 172.21.15.90:0/1959254433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 msgr2=0x5563c734ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.267+0000 7fef83e00700 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c734ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.267+0000 7fef83e00700 1 -- 172.21.15.90:0/1959254433 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c71b28d0 con 0x5563c734ad30 2024-05-27T07:58:13.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.267+0000 7fef83400700 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c71c5e40 0x5563c734e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:58:13.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.268+0000 7fef83e00700 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 0x5563c734ef70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00b6e0 tx=0x7fef7c00d390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d837ed1b1e9332a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:13.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.270+0000 7fef6d600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:58:13.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.270+0000 7fef6d600700 1 -- 172.21.15.90:0/1959254433 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c003a00 con 0x5563c734ad30 2024-05-27T07:58:13.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.270+0000 7fef6d600700 1 -- 172.21.15.90:0/1959254433 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9584+0+0 (secure 0 0 0) 0x7fef7c003ba0 con 0x5563c734ad30 2024-05-27T07:58:13.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.272+0000 7fef99147b80 1 -- 172.21.15.90:0/1959254433 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef70012b90 con 0x5563c734ad30 2024-05-27T07:58:13.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.273+0000 7fef6d600700 1 -- 172.21.15.90:0/1959254433 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fef7c003da0 con 0x5563c734ad30 2024-05-27T07:58:13.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.273+0000 7fef99147b80 10 client.37112 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:13.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.273+0000 7fef99147b80 1 -- 172.21.15.90:0/1959254433 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5563c7233060 con 0x5563c734ad30 2024-05-27T07:58:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.274+0000 7fef6d600700 1 -- 172.21.15.90:0/1959254433 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef70012b90 con 0x5563c734ad30 2024-05-27T07:58:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.274+0000 7fef6d600700 1 -- 172.21.15.90:0/1959254433 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fef7c0042b0 con 0x5563c734ad30 2024-05-27T07:58:13.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.275+0000 7fef99147b80 10 client.37112 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:13.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.275+0000 7fef99147b80 10 client.37112 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:58:13.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.275+0000 7fef99147b80 20 client.37112 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:13.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.275+0000 7fef99147b80 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5563c735e8e0 0x5563c7360d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.275+0000 7fef99147b80 4 client.37112 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:13.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.275+0000 7fef99147b80 1 -- 172.21.15.90:0/1959254433 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563c7274d90 con 0x5563c735e8e0 2024-05-27T07:58:13.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.276+0000 7fef83400700 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5563c735e8e0 0x5563c7360d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.279+0000 7fef83400700 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5563c735e8e0 0x5563c7360d30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee62481d3353760a server_cookie=dec3d4be9a7ebe7 in_seq=0 out_seq=0 2024-05-27T07:58:13.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.279+0000 7fef6d600700 10 client.37112 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:58:13.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.280+0000 7fef6d600700 1 -- 172.21.15.90:0/1959254433 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5563c7274d90 con 0x5563c735e8e0 2024-05-27T07:58:13.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.280+0000 7fef6d600700 10 client.37112 handle_command_reply: tid=0 2024-05-27T07:58:13.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.280+0000 7fef99147b80 1 client.37112 shutdown 2024-05-27T07:58:13.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.281+0000 7fef99147b80 1 -- 172.21.15.90:0/1959254433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 msgr2=0x5563c734ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:13.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.281+0000 7fef99147b80 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 0x5563c734ef70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00b6e0 tx=0x7fef7c00d390 comp rx=0 tx=0).stop 2024-05-27T07:58:13.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.282+0000 7fef99147b80 1 -- 172.21.15.90:0/1959254433 shutdown_connections 2024-05-27T07:58:13.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.282+0000 7fef99147b80 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5563c735e8e0 0x5563c7360d30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.282+0000 7fef6d600700 0 client.37112 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:58:13.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.282+0000 7fef99147b80 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 0x5563c734ef70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.282+0000 7fef99147b80 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c71c5e40 0x5563c734e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.282+0000 7fef99147b80 1 --2- 172.21.15.90:0/1959254433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c734ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.282+0000 7fef99147b80 1 -- 172.21.15.90:0/1959254433 >> 172.21.15.90:0/1959254433 conn(0x5563c7225fc0 msgr2=0x5563c733f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:13.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.282+0000 7fef99147b80 1 -- 172.21.15.90:0/1959254433 shutdown_connections 2024-05-27T07:58:13.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.282+0000 7fef99147b80 1 -- 172.21.15.90:0/1959254433 wait complete. 2024-05-27T07:58:13.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.283+0000 7fef99147b80 20 client.37112 trim_cache size 0 max 16384 2024-05-27T07:58:13.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.318+0000 7fef99147b80 1 Processor -- start 2024-05-27T07:58:13.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.318+0000 7fef99147b80 1 -- start start 2024-05-27T07:58:13.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.318+0000 7fef99147b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 0x5563c7344870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.318+0000 7fef99147b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c7344db0 0x5563c733b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.318+0000 7fef99147b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c734ad30 0x5563c734d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.318+0000 7fef99147b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b8670 con 0x5563c734ad30 2024-05-27T07:58:13.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.318+0000 7fef99147b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b56c0 con 0x5563c71c5e40 2024-05-27T07:58:13.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.318+0000 7fef99147b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b78d0 con 0x5563c7344db0 2024-05-27T07:58:13.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.319+0000 7fef82a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c7344db0 0x5563c733b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.319+0000 7fef83400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 0x5563c7344870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.319+0000 7fef82a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c7344db0 0x5563c733b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37278/0 (socket says 172.21.15.90:37278) 2024-05-27T07:58:13.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.319+0000 7fef82a00700 1 -- 172.21.15.90:0/988599628 learned_addr learned my addr 172.21.15.90:0/988599628 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:13.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.319+0000 7fef83e00700 1 --2- 172.21.15.90:0/988599628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c734ad30 0x5563c734d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.320+0000 7fef83400700 1 -- 172.21.15.90:0/988599628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c7344db0 msgr2=0x5563c733b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:13.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.320+0000 7fef83400700 1 --2- 172.21.15.90:0/988599628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c7344db0 0x5563c733b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.320+0000 7fef83400700 1 -- 172.21.15.90:0/988599628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c734ad30 msgr2=0x5563c734d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:13.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.320+0000 7fef83400700 1 --2- 172.21.15.90:0/988599628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c734ad30 0x5563c734d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.320+0000 7fef83400700 1 -- 172.21.15.90:0/988599628 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c735b2c0 con 0x5563c71c5e40 2024-05-27T07:58:13.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.321+0000 7fef83400700 1 --2- 172.21.15.90:0/988599628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 0x5563c7344870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fef740e1700 tx=0x7fef7400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1936ffca6ecaeb8d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:13.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.322+0000 7fef6fe00700 1 -- 172.21.15.90:0/988599628 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef740c4020 con 0x5563c71c5e40 2024-05-27T07:58:13.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.322+0000 7fef6fe00700 1 -- 172.21.15.90:0/988599628 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef740af670 con 0x5563c71c5e40 2024-05-27T07:58:13.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.322+0000 7fef6fe00700 1 -- 172.21.15.90:0/988599628 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7400f910 con 0x5563c71c5e40 2024-05-27T07:58:13.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.324+0000 7fef99147b80 1 -- 172.21.15.90:0/988599628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 msgr2=0x5563c7344870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:13.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.324+0000 7fef99147b80 1 --2- 172.21.15.90:0/988599628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 0x5563c7344870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fef740e1700 tx=0x7fef7400bf60 comp rx=0 tx=0).stop 2024-05-27T07:58:13.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.324+0000 7fef99147b80 1 -- 172.21.15.90:0/988599628 shutdown_connections 2024-05-27T07:58:13.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.324+0000 7fef99147b80 1 --2- 172.21.15.90:0/988599628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c734ad30 0x5563c734d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.324+0000 7fef99147b80 1 --2- 172.21.15.90:0/988599628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c71c5e40 0x5563c7344870 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fef740e1700 tx=0x7fef7400bf60 comp rx=0 tx=0).stop 2024-05-27T07:58:13.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.324+0000 7fef99147b80 1 --2- 172.21.15.90:0/988599628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c7344db0 0x5563c733b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.324+0000 7fef99147b80 1 -- 172.21.15.90:0/988599628 >> 172.21.15.90:0/988599628 conn(0x5563c7225fc0 msgr2=0x5563c7348210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:13.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.324+0000 7fef99147b80 1 -- 172.21.15.90:0/988599628 shutdown_connections 2024-05-27T07:58:13.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.325+0000 7fef99147b80 1 -- 172.21.15.90:0/988599628 wait complete. 2024-05-27T07:58:13.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.326+0000 7fef99147b80 1 -- start start 2024-05-27T07:58:13.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.327+0000 7fef99147b80 1 Processor -- start 2024-05-27T07:58:13.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.328+0000 7fef99147b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c7356a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.328+0000 7fef99147b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 0x5563c7356f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.328+0000 7fef99147b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c735e8e0 0x5563c73654e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.328+0000 7fef99147b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b7200 con 0x5563c7344db0 2024-05-27T07:58:13.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.328+0000 7fef99147b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b5d90 con 0x5563c734ad30 2024-05-27T07:58:13.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.328+0000 7fef99147b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef841b6460 con 0x5563c735e8e0 2024-05-27T07:58:13.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.328+0000 7fef99147b80 10 client.0 _ll_register_callbacks cb 0x5563c720cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:13.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.329+0000 7fef83e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c735e8e0 0x5563c73654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.329+0000 7fef82a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 0x5563c7356f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.330+0000 7fef83400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c7356a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.330+0000 7fef82a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 0x5563c7356f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43658/0 (socket says 172.21.15.90:43658) 2024-05-27T07:58:13.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.330+0000 7fef82a00700 1 -- 172.21.15.90:0/2700496878 learned_addr learned my addr 172.21.15.90:0/2700496878 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:13.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.331+0000 7fef83e00700 1 -- 172.21.15.90:0/2700496878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 msgr2=0x5563c7356f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:13.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.331+0000 7fef83e00700 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 0x5563c7356f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.331+0000 7fef83e00700 1 -- 172.21.15.90:0/2700496878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 msgr2=0x5563c7356a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:13.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.332+0000 7fef83e00700 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c7356a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.332+0000 7fef83e00700 1 -- 172.21.15.90:0/2700496878 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c7365a20 con 0x5563c735e8e0 2024-05-27T07:58:13.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.332+0000 7fef83400700 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c7356a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:13.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.333+0000 7fef83e00700 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c735e8e0 0x5563c73654e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fef7c015460 tx=0x7fef7c0021f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c912fc70d6ade8c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:13.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.335+0000 7fef6d600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:58:13.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.335+0000 7fef6d600700 1 -- 172.21.15.90:0/2700496878 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c02d060 con 0x5563c735e8e0 2024-05-27T07:58:13.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.335+0000 7fef6d600700 1 -- 172.21.15.90:0/2700496878 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9584+0+0 (secure 0 0 0) 0x7fef7c013040 con 0x5563c735e8e0 2024-05-27T07:58:13.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.335+0000 7fef99147b80 1 -- 172.21.15.90:0/2700496878 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef740af670 con 0x5563c735e8e0 2024-05-27T07:58:13.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.336+0000 7fef6d600700 1 -- 172.21.15.90:0/2700496878 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fef7c034050 con 0x5563c735e8e0 2024-05-27T07:58:13.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.336+0000 7fef99147b80 10 client.27208 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:13.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.336+0000 7fef99147b80 1 -- 172.21.15.90:0/2700496878 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5563c73c8a60 con 0x5563c735e8e0 2024-05-27T07:58:13.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.336+0000 7fef6d600700 1 -- 172.21.15.90:0/2700496878 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef740af670 con 0x5563c735e8e0 2024-05-27T07:58:13.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.337+0000 7fef6d600700 1 -- 172.21.15.90:0/2700496878 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fef7c030cb0 con 0x5563c735e8e0 2024-05-27T07:58:13.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.337+0000 7fef99147b80 10 client.27208 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:13.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.337+0000 7fef99147b80 10 client.27208 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:58:13.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.337+0000 7fef99147b80 20 client.27208 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:13.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.337+0000 7fef99147b80 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5563c73c90c0 0x5563c73cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:13.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.337+0000 7fef99147b80 4 client.27208 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:58:13.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.337+0000 7fef99147b80 1 -- 172.21.15.90:0/2700496878 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563c726e570 con 0x5563c73c90c0 2024-05-27T07:58:13.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.339+0000 7fef83400700 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5563c73c90c0 0x5563c73cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:13.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.340+0000 7fef83400700 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5563c73c90c0 0x5563c73cb4e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8e12052949dc17c5 server_cookie=d2a40457d1a5b1fe in_seq=0 out_seq=0 2024-05-27T07:58:13.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.341+0000 7fef6d600700 10 client.27208 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:58:13.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.341+0000 7fef6d600700 1 -- 172.21.15.90:0/2700496878 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563c726e570 con 0x5563c73c90c0 2024-05-27T07:58:13.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.341+0000 7fef6d600700 10 client.27208 handle_command_reply: tid=0 2024-05-27T07:58:13.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.341+0000 7fef99147b80 1 client.27208 shutdown 2024-05-27T07:58:13.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef99147b80 1 -- 172.21.15.90:0/2700496878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c735e8e0 msgr2=0x5563c73654e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:13.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef99147b80 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c735e8e0 0x5563c73654e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fef7c015460 tx=0x7fef7c0021f0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef99147b80 1 -- 172.21.15.90:0/2700496878 shutdown_connections 2024-05-27T07:58:13.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef99147b80 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5563c73c90c0 0x5563c73cb4e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef99147b80 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563c735e8e0 0x5563c73654e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef99147b80 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5563c7344db0 0x5563c7356a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef99147b80 1 --2- 172.21.15.90:0/2700496878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563c734ad30 0x5563c7356f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef6d600700 0 client.27208 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:58:13.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef99147b80 1 -- 172.21.15.90:0/2700496878 >> 172.21.15.90:0/2700496878 conn(0x5563c7225fc0 msgr2=0x5563c734e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:13.344 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:58:13.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef99147b80 1 -- 172.21.15.90:0/2700496878 shutdown_connections 2024-05-27T07:58:13.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.342+0000 7fef99147b80 1 -- 172.21.15.90:0/2700496878 wait complete. 2024-05-27T07:58:13.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.343+0000 7fef99147b80 20 client.27208 trim_cache size 0 max 16384 2024-05-27T07:58:13.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.343+0000 7fef8a600700 1 -- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fef680b2a70 msgr2=0x7fef680b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:13.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.343+0000 7fef8a600700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fef680b2a70 0x7fef680b4f80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fef84107500 tx=0x7fef70009260 comp rx=0 tx=0).stop 2024-05-27T07:58:13.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.344+0000 7fef8a600700 1 -- 172.21.15.90:0/4146907014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef841045e0 msgr2=0x7fef841a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:13.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.344+0000 7fef8a600700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef841045e0 0x7fef841a13d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fef74005910 tx=0x7fef74004690 comp rx=0 tx=0).stop 2024-05-27T07:58:13.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.344+0000 7fef8a600700 1 -- 172.21.15.90:0/4146907014 shutdown_connections 2024-05-27T07:58:13.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.344+0000 7fef8a600700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef84109860 0x7fef841a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.344+0000 7fef8a600700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fef680b2a70 0x7fef680b4f80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.344+0000 7fef8a600700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef841045e0 0x7fef841a13d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.344+0000 7fef8a600700 1 --2- 172.21.15.90:0/4146907014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef84108f10 0x7fef841a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:13.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.344+0000 7fef8a600700 1 -- 172.21.15.90:0/4146907014 >> 172.21.15.90:0/4146907014 conn(0x7fef84100030 msgr2=0x7fef84102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:13.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.344+0000 7fef8a600700 1 -- 172.21.15.90:0/4146907014 shutdown_connections 2024-05-27T07:58:13.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:13.344+0000 7fef8a600700 1 -- 172.21.15.90:0/4146907014 wait complete. 2024-05-27T07:58:13.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 475.0 s, 613.6 tps, lat 164.857 ms stddev 124.599 2024-05-27T07:58:14.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:13 smithi129 ceph-mon[35254]: pgmap v375: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.0 MiB/s wr, 441 op/s 2024-05-27T07:58:14.072 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:13 smithi090 ceph-mon[31429]: pgmap v375: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.0 MiB/s wr, 441 op/s 2024-05-27T07:58:14.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:13 smithi136 ceph-mon[35324]: pgmap v375: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.0 MiB/s wr, 441 op/s 2024-05-27T07:58:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.574+0000 7f4aa8e00640 1 -- 172.21.15.90:0/4231568582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4aa4108000 msgr2=0x7f4aa4108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.574+0000 7f4aa8e00640 1 --2- 172.21.15.90:0/4231568582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4aa4108000 0x7f4aa4108490 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009ca0 tx=0x7f4a9c02a8c0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.574+0000 7f4aa8e00640 1 -- 172.21.15.90:0/4231568582 shutdown_connections 2024-05-27T07:58:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.574+0000 7f4aa8e00640 1 --2- 172.21.15.90:0/4231568582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4aa41075d0 0x7f4aa41079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.574+0000 7f4aa8e00640 1 --2- 172.21.15.90:0/4231568582 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4aa4108000 0x7f4aa4108490 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.574+0000 7f4aa8e00640 1 --2- 172.21.15.90:0/4231568582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4102b80 0x7f4aa4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.574+0000 7f4aa8e00640 1 -- 172.21.15.90:0/4231568582 >> 172.21.15.90:0/4231568582 conn(0x7f4aa40782d0 msgr2=0x7f4aa4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.575+0000 7f4aa8e00640 1 -- 172.21.15.90:0/4231568582 shutdown_connections 2024-05-27T07:58:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.575+0000 7f4aa8e00640 1 -- 172.21.15.90:0/4231568582 wait complete. 2024-05-27T07:58:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.577+0000 7f4aa8e00640 1 Processor -- start 2024-05-27T07:58:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.577+0000 7f4aa8e00640 1 -- start start 2024-05-27T07:58:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.578+0000 7f4aa8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4aa4102b80 0x7f4aa41a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.578+0000 7f4aa8e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4aa41075d0 0x7f4aa41a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.578+0000 7f4aa8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4108000 0x7f4aa41a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.578+0000 7f4aa8e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa411a310 con 0x7f4aa4102b80 2024-05-27T07:58:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.578+0000 7f4aa8e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa411a190 con 0x7f4aa41075d0 2024-05-27T07:58:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.578+0000 7f4aa8e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa411a490 con 0x7f4aa4108000 2024-05-27T07:58:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.579+0000 7f4aa2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4108000 0x7f4aa41a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.579+0000 7f4aa2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4108000 0x7f4aa41a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37288/0 (socket says 172.21.15.90:37288) 2024-05-27T07:58:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.579+0000 7f4aa2800640 1 -- 172.21.15.90:0/1080726786 learned_addr learned my addr 172.21.15.90:0/1080726786 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.579+0000 7f4aa1400640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4aa41075d0 0x7f4aa41a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.579+0000 7f4aa2800640 1 -- 172.21.15.90:0/1080726786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4aa41075d0 msgr2=0x7f4aa41a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.579+0000 7f4aa2800640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4aa41075d0 0x7f4aa41a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.579+0000 7f4aa2800640 1 -- 172.21.15.90:0/1080726786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4aa4102b80 msgr2=0x7f4aa41a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.579+0000 7f4aa2800640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4aa4102b80 0x7f4aa41a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.579+0000 7f4aa2800640 1 -- 172.21.15.90:0/1080726786 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa41a5c10 con 0x7f4aa4108000 2024-05-27T07:58:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.580+0000 7f4aa2800640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4108000 0x7f4aa41a55d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c031040 tx=0x7f4a9c003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.581+0000 7f4a9aa00640 1 -- 172.21.15.90:0/1080726786 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c032c90 con 0x7f4aa4108000 2024-05-27T07:58:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.581+0000 7f4a9aa00640 1 -- 172.21.15.90:0/1080726786 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a9c032e30 con 0x7f4aa4108000 2024-05-27T07:58:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.581+0000 7f4aa8e00640 1 -- 172.21.15.90:0/1080726786 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa41a5ea0 con 0x7f4aa4108000 2024-05-27T07:58:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.581+0000 7f4aa8e00640 1 -- 172.21.15.90:0/1080726786 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa41a63f0 con 0x7f4aa4108000 2024-05-27T07:58:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.581+0000 7f4a9aa00640 1 -- 172.21.15.90:0/1080726786 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c030b30 con 0x7f4aa4108000 2024-05-27T07:58:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.586+0000 7f4aafe66740 1 Processor -- start 2024-05-27T07:58:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.589+0000 7f4a9aa00640 1 -- 172.21.15.90:0/1080726786 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4a9c030cd0 con 0x7f4aa4108000 2024-05-27T07:58:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.589+0000 7f4aafe66740 1 -- start start 2024-05-27T07:58:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.589+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 0x560ed1599860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.589+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 0x560ed1599da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.589+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11d64d0 0x560ed159a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.589+0000 7f4aafe66740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560ed150ab90 con 0x560ed11ce2f0 2024-05-27T07:58:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.589+0000 7f4aafe66740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560ed14fa730 con 0x560ed11d64d0 2024-05-27T07:58:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.590+0000 7f4aa1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11d64d0 0x560ed159a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.590+0000 7f4aa1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11d64d0 0x560ed159a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43694/0 (socket says 172.21.15.90:43694) 2024-05-27T07:58:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.590+0000 7f4aa1400640 1 -- 172.21.15.90:0/2674305778 learned_addr learned my addr 172.21.15.90:0/2674305778 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.590+0000 7f4aafe66740 1 -- 172.21.15.90:0/2674305778 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560ed14fe770 con 0x560ed11f71b0 2024-05-27T07:58:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.591+0000 7f4aa1400640 1 --2- 172.21.15.90:0/2674305778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 0x560ed1599860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.591+0000 7f4a9aa00640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4a880b2950 0x7f4a880b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.591+0000 7f4a9aa00640 1 -- 172.21.15.90:0/1080726786 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9584+0+0 (secure 0 0 0) 0x7f4a9c103f90 con 0x7f4aa4108000 2024-05-27T07:58:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.591+0000 7f4aa2800640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4a880b2950 0x7f4a880b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.592+0000 7f4aa1400640 1 -- 172.21.15.90:0/2674305778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 msgr2=0x560ed1599860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.592+0000 7f4aa1400640 1 --2- 172.21.15.90:0/2674305778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 0x560ed1599860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.592+0000 7f4aa1400640 1 -- 172.21.15.90:0/2674305778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 msgr2=0x560ed1599da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.592+0000 7f4aa1400640 1 --2- 172.21.15.90:0/2674305778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 0x560ed1599da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.592+0000 7f4aa1400640 1 -- 172.21.15.90:0/2674305778 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ed14938b0 con 0x560ed11d64d0 2024-05-27T07:58:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.593+0000 7f4aa2800640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4a880b2950 0x7f4a880b4e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0094e0 tx=0x7f4a9c0095c0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.593+0000 7f4aa1400640 1 --2- 172.21.15.90:0/2674305778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11d64d0 0x560ed159a2e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x560ed15969d0 tx=0x7f4a900095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81acdca370e180ac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.597+0000 7f4a8f400640 1 -- 172.21.15.90:0/2674305778 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90014110 con 0x560ed11d64d0 2024-05-27T07:58:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.597+0000 7f4a8f400640 1 -- 172.21.15.90:0/2674305778 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a900142b0 con 0x560ed11d64d0 2024-05-27T07:58:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.597+0000 7f4a8f400640 1 -- 172.21.15.90:0/2674305778 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9000fe30 con 0x560ed11d64d0 2024-05-27T07:58:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.598+0000 7f4aafe66740 1 -- 172.21.15.90:0/2674305778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11d64d0 msgr2=0x560ed159a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.598+0000 7f4aafe66740 1 --2- 172.21.15.90:0/2674305778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11d64d0 0x560ed159a2e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x560ed15969d0 tx=0x7f4a900095b0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.599+0000 7f4aafe66740 1 -- 172.21.15.90:0/2674305778 shutdown_connections 2024-05-27T07:58:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.599+0000 7f4aafe66740 1 --2- 172.21.15.90:0/2674305778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11d64d0 0x560ed159a2e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.599+0000 7f4aafe66740 1 --2- 172.21.15.90:0/2674305778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 0x560ed1599da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.599+0000 7f4aafe66740 1 --2- 172.21.15.90:0/2674305778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 0x560ed1599860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.599+0000 7f4aafe66740 1 -- 172.21.15.90:0/2674305778 >> 172.21.15.90:0/2674305778 conn(0x560ed11eb300 msgr2=0x560ed1596010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.599+0000 7f4aafe66740 1 -- 172.21.15.90:0/2674305778 shutdown_connections 2024-05-27T07:58:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.599+0000 7f4aafe66740 1 -- 172.21.15.90:0/2674305778 wait complete. 2024-05-27T07:58:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.600+0000 7f4aafe66740 1 -- start start 2024-05-27T07:58:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.600+0000 7f4aafe66740 1 Processor -- start 2024-05-27T07:58:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.601+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 0x560ed15a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.601+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11d64d0 0x560ed15a4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.601+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11f71b0 0x560ed15c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.601+0000 7f4aafe66740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560ed11e0790 con 0x560ed11ce2f0 2024-05-27T07:58:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.601+0000 7f4aafe66740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560ed14a2bc0 con 0x560ed11f71b0 2024-05-27T07:58:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.601+0000 7f4aafe66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560ed15b79a0 con 0x560ed11d64d0 2024-05-27T07:58:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.601+0000 7f4aafe66740 10 client.0 _ll_register_callbacks cb 0x560ed1480400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.602+0000 7f4aa1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 0x560ed15a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.602+0000 7f4aa1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 0x560ed15a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58488/0 (socket says 172.21.15.90:58488) 2024-05-27T07:58:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.602+0000 7f4aa1400640 1 -- 172.21.15.90:0/3741260555 learned_addr learned my addr 172.21.15.90:0/3741260555 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.602+0000 7f4aa1400640 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11f71b0 0x560ed15c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.602+0000 7f4aa1e00640 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11d64d0 0x560ed15a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.603+0000 7f4aa1400640 1 -- 172.21.15.90:0/3741260555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11d64d0 msgr2=0x560ed15a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.603+0000 7f4aa1400640 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11d64d0 0x560ed15a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.603+0000 7f4aa1400640 1 -- 172.21.15.90:0/3741260555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11f71b0 msgr2=0x560ed15c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.603+0000 7f4aa1400640 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11f71b0 0x560ed15c0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.603+0000 7f4aa1400640 1 -- 172.21.15.90:0/3741260555 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ed120f210 con 0x560ed11ce2f0 2024-05-27T07:58:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.604+0000 7f4aa1400640 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 0x560ed15a3dc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009d00 tx=0x7f4a90005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13ded118c39d244 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.604+0000 7f4a83e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:58:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.605+0000 7f4a83e00640 1 -- 172.21.15.90:0/3741260555 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90014110 con 0x560ed11ce2f0 2024-05-27T07:58:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.605+0000 7f4aafe66740 1 -- 172.21.15.90:0/3741260555 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ed150cae0 con 0x560ed11ce2f0 2024-05-27T07:58:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.605+0000 7f4a83e00640 1 -- 172.21.15.90:0/3741260555 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9584+0+0 (secure 0 0 0) 0x7f4a90004020 con 0x560ed11ce2f0 2024-05-27T07:58:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.605+0000 7f4a83e00640 1 -- 172.21.15.90:0/3741260555 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4a90033050 con 0x560ed11ce2f0 2024-05-27T07:58:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.605+0000 7f4aafe66740 10 client.17544 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.606+0000 7f4aafe66740 1 -- 172.21.15.90:0/3741260555 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560ed12db620 con 0x560ed11ce2f0 2024-05-27T07:58:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.606+0000 7f4a83e00640 1 -- 172.21.15.90:0/3741260555 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560ed150cae0 con 0x560ed11ce2f0 2024-05-27T07:58:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.607+0000 7f4a83e00640 1 -- 172.21.15.90:0/3741260555 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f4a900035f0 con 0x560ed11ce2f0 2024-05-27T07:58:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.607+0000 7f4aafe66740 10 client.17544 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.607+0000 7f4aafe66740 10 client.17544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.607+0000 7f4aafe66740 20 client.17544 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.607+0000 7f4aafe66740 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560ed11b5d80 0x560ed15d7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.607+0000 7f4aafe66740 4 client.17544 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.607+0000 7f4aafe66740 1 -- 172.21.15.90:0/3741260555 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ed15062c0 con 0x560ed11b5d80 2024-05-27T07:58:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.608+0000 7f4aa1e00640 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560ed11b5d80 0x560ed15d7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.609+0000 7f4aa1e00640 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560ed11b5d80 0x560ed15d7f90 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16c657921467cb server_cookie=3009cac9e59ab0f7 in_seq=0 out_seq=0 2024-05-27T07:58:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.609+0000 7f4a83e00640 10 client.17544 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.610+0000 7f4a83e00640 1 -- 172.21.15.90:0/3741260555 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560ed15062c0 con 0x560ed11b5d80 2024-05-27T07:58:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.610+0000 7f4a83e00640 10 client.17544 handle_command_reply: tid=0 2024-05-27T07:58:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.611+0000 7f4aafe66740 1 client.17544 shutdown 2024-05-27T07:58:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.611+0000 7f4aafe66740 1 -- 172.21.15.90:0/3741260555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 msgr2=0x560ed15a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.611+0000 7f4aafe66740 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 0x560ed15a3dc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009d00 tx=0x7f4a90005f90 comp rx=0 tx=0).stop 2024-05-27T07:58:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.611+0000 7f4aafe66740 1 -- 172.21.15.90:0/3741260555 shutdown_connections 2024-05-27T07:58:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.611+0000 7f4aafe66740 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560ed11b5d80 0x560ed15d7f90 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.611+0000 7f4aafe66740 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed11f71b0 0x560ed15c0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.611+0000 7f4aafe66740 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11d64d0 0x560ed15a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.611+0000 7f4aafe66740 1 --2- 172.21.15.90:0/3741260555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11ce2f0 0x560ed15a3dc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.611+0000 7f4aafe66740 1 -- 172.21.15.90:0/3741260555 >> 172.21.15.90:0/3741260555 conn(0x560ed11eb300 msgr2=0x560ed159b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.611+0000 7f4a83e00640 0 client.17544 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.612+0000 7f4aafe66740 1 -- 172.21.15.90:0/3741260555 shutdown_connections 2024-05-27T07:58:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.612+0000 7f4aafe66740 1 -- 172.21.15.90:0/3741260555 wait complete. 2024-05-27T07:58:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.612+0000 7f4aafe66740 20 client.17544 trim_cache size 0 max 16384 2024-05-27T07:58:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.645+0000 7f4aafe66740 1 Processor -- start 2024-05-27T07:58:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.645+0000 7f4aafe66740 1 -- start start 2024-05-27T07:58:14.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.645+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11d64d0 0x560ed159f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.645+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 0x560ed1595330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.645+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 0x560ed159a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.645+0000 7f4aafe66740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa410d840 con 0x560ed11d64d0 2024-05-27T07:58:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.645+0000 7f4aafe66740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa410a890 con 0x560ed118b1c0 2024-05-27T07:58:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.645+0000 7f4aafe66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa410caa0 con 0x560ed11f71b0 2024-05-27T07:58:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.646+0000 7f4aa1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 0x560ed1595330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.646+0000 7f4aa1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 0x560ed1595330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37306/0 (socket says 172.21.15.90:37306) 2024-05-27T07:58:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.646+0000 7f4aa1e00640 1 -- 172.21.15.90:0/3789632800 learned_addr learned my addr 172.21.15.90:0/3789632800 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.646+0000 7f4aa1400640 1 --2- 172.21.15.90:0/3789632800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11d64d0 0x560ed159f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.647+0000 7f4aa1400640 1 --2- 172.21.15.90:0/3789632800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 0x560ed159a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.647+0000 7f4aa1e00640 1 -- 172.21.15.90:0/3789632800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 msgr2=0x560ed159a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.647+0000 7f4aa1e00640 1 --2- 172.21.15.90:0/3789632800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 0x560ed159a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.647+0000 7f4aa1e00640 1 -- 172.21.15.90:0/3789632800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11d64d0 msgr2=0x560ed159f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.647+0000 7f4aa1e00640 1 --2- 172.21.15.90:0/3789632800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11d64d0 0x560ed159f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.647+0000 7f4aa1e00640 1 -- 172.21.15.90:0/3789632800 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ed159b0b0 con 0x560ed11f71b0 2024-05-27T07:58:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.649+0000 7f4aa1e00640 1 --2- 172.21.15.90:0/3789632800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 0x560ed1595330 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x560ed1596fa0 tx=0x7f4a9400b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53233e1910f81a1a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.649+0000 7f4a8ea00640 1 -- 172.21.15.90:0/3789632800 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9401e030 con 0x560ed11f71b0 2024-05-27T07:58:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.649+0000 7f4a8ea00640 1 -- 172.21.15.90:0/3789632800 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a9400be50 con 0x560ed11f71b0 2024-05-27T07:58:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.649+0000 7f4a8ea00640 1 -- 172.21.15.90:0/3789632800 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9401e030 con 0x560ed11f71b0 2024-05-27T07:58:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.651+0000 7f4aafe66740 1 -- 172.21.15.90:0/3789632800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 msgr2=0x560ed1595330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.651+0000 7f4aafe66740 1 --2- 172.21.15.90:0/3789632800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 0x560ed1595330 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x560ed1596fa0 tx=0x7f4a9400b4a0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.651+0000 7f4aafe66740 1 -- 172.21.15.90:0/3789632800 shutdown_connections 2024-05-27T07:58:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.651+0000 7f4aafe66740 1 --2- 172.21.15.90:0/3789632800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 0x560ed159a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.651+0000 7f4aafe66740 1 --2- 172.21.15.90:0/3789632800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11f71b0 0x560ed1595330 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.651+0000 7f4aafe66740 1 --2- 172.21.15.90:0/3789632800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11d64d0 0x560ed159f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.651+0000 7f4aafe66740 1 -- 172.21.15.90:0/3789632800 >> 172.21.15.90:0/3789632800 conn(0x560ed11eb300 msgr2=0x560ed15a2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.651+0000 7f4aafe66740 1 -- 172.21.15.90:0/3789632800 shutdown_connections 2024-05-27T07:58:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.651+0000 7f4aafe66740 1 -- 172.21.15.90:0/3789632800 wait complete. 2024-05-27T07:58:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.652+0000 7f4aafe66740 1 -- start start 2024-05-27T07:58:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.653+0000 7f4aafe66740 1 Processor -- start 2024-05-27T07:58:14.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.654+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 0x560ed15bcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.654+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11d64d0 0x560ed15bd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.654+0000 7f4aafe66740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11f71b0 0x560ed159cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.654+0000 7f4aafe66740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa410c3d0 con 0x560ed11f71b0 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.654+0000 7f4aafe66740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa410af60 con 0x560ed118b1c0 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.654+0000 7f4aafe66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa410b630 con 0x560ed11d64d0 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.654+0000 7f4aafe66740 10 client.0 _ll_register_callbacks cb 0x7f4a90009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.655+0000 7f4aa1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 0x560ed15bcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.655+0000 7f4aa1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 0x560ed15bcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43714/0 (socket says 172.21.15.90:43714) 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.655+0000 7f4aa1400640 1 -- 172.21.15.90:0/4151457270 learned_addr learned my addr 172.21.15.90:0/4151457270 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.655+0000 7f4aa1400640 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11f71b0 0x560ed159cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.656+0000 7f4aa1e00640 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11d64d0 0x560ed15bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.656+0000 7f4aa1400640 1 -- 172.21.15.90:0/4151457270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11d64d0 msgr2=0x560ed15bd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.657+0000 7f4aa1400640 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11d64d0 0x560ed15bd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.657+0000 7f4aa1400640 1 -- 172.21.15.90:0/4151457270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 msgr2=0x560ed15bcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:14.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.657+0000 7f4aa1400640 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 0x560ed15bcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.657+0000 7f4aa1400640 1 -- 172.21.15.90:0/4151457270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ed159d120 con 0x560ed11f71b0 2024-05-27T07:58:14.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.658+0000 7f4a83e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:58:14.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.658+0000 7f4aa1400640 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11f71b0 0x560ed159cbe0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4a900039c0 tx=0x7f4a90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505c6c97064a01ad server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:14.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.659+0000 7f4a83e00640 1 -- 172.21.15.90:0/4151457270 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90014110 con 0x560ed11f71b0 2024-05-27T07:58:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.659+0000 7f4a83e00640 1 -- 172.21.15.90:0/4151457270 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9584+0+0 (secure 0 0 0) 0x7f4a90028040 con 0x560ed11f71b0 2024-05-27T07:58:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.660+0000 7f4aafe66740 1 -- 172.21.15.90:0/4151457270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a9400be50 con 0x560ed11f71b0 2024-05-27T07:58:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.663+0000 7f4a83e00640 1 -- 172.21.15.90:0/4151457270 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4a90014110 con 0x560ed11f71b0 2024-05-27T07:58:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.663+0000 7f4aafe66740 10 client.17550 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.663+0000 7f4aafe66740 1 -- 172.21.15.90:0/4151457270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560ed163ed60 con 0x560ed11f71b0 2024-05-27T07:58:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.665+0000 7f4a83e00640 1 -- 172.21.15.90:0/4151457270 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a9400be50 con 0x560ed11f71b0 2024-05-27T07:58:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.665+0000 7f4a83e00640 1 -- 172.21.15.90:0/4151457270 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f4a9000f840 con 0x560ed11f71b0 2024-05-27T07:58:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.665+0000 7f4aafe66740 10 client.17550 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.665+0000 7f4aafe66740 10 client.17550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.665+0000 7f4aafe66740 20 client.17550 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.665+0000 7f4aafe66740 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560ed163f340 0x560ed1641760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.665+0000 7f4aafe66740 4 client.17550 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:58:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.665+0000 7f4aafe66740 1 -- 172.21.15.90:0/4151457270 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ed14804b0 con 0x560ed163f340 2024-05-27T07:58:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.666+0000 7f4aa1e00640 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560ed163f340 0x560ed1641760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.668+0000 7f4aa1e00640 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560ed163f340 0x560ed1641760 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b141ac6958389602 server_cookie=c7f69103797e131d in_seq=0 out_seq=0 2024-05-27T07:58:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.668+0000 7f4a83e00640 10 client.17550 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.730+0000 7f4a83e00640 1 -- 172.21.15.90:0/4151457270 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560ed14804b0 con 0x560ed163f340 2024-05-27T07:58:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.730+0000 7f4a83e00640 10 client.17550 handle_command_reply: tid=0 2024-05-27T07:58:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.730+0000 7f4aafe66740 1 client.17550 shutdown 2024-05-27T07:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4aafe66740 1 -- 172.21.15.90:0/4151457270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11f71b0 msgr2=0x560ed159cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4aafe66740 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11f71b0 0x560ed159cbe0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4a900039c0 tx=0x7f4a90000bc0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4aafe66740 1 -- 172.21.15.90:0/4151457270 shutdown_connections 2024-05-27T07:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4aafe66740 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560ed163f340 0x560ed1641760 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4aafe66740 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ed11f71b0 0x560ed159cbe0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4aafe66740 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ed11d64d0 0x560ed15bd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4aafe66740 1 --2- 172.21.15.90:0/4151457270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ed118b1c0 0x560ed15bcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4aafe66740 1 -- 172.21.15.90:0/4151457270 >> 172.21.15.90:0/4151457270 conn(0x560ed11eb300 msgr2=0x560ed15a3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4a83e00640 0 client.17550 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4aafe66740 1 -- 172.21.15.90:0/4151457270 shutdown_connections 2024-05-27T07:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.731+0000 7f4aafe66740 1 -- 172.21.15.90:0/4151457270 wait complete. 2024-05-27T07:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.732+0000 7f4aafe66740 20 client.17550 trim_cache size 0 max 16384 2024-05-27T07:58:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.733+0000 7f4aa8e00640 1 -- 172.21.15.90:0/1080726786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4a880b2950 msgr2=0x7f4a880b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.734+0000 7f4aa8e00640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4a880b2950 0x7f4a880b4e60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c0094e0 tx=0x7f4a9c0095c0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.734+0000 7f4aa8e00640 1 -- 172.21.15.90:0/1080726786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4108000 msgr2=0x7f4aa41a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.734+0000 7f4aa8e00640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4108000 0x7f4aa41a55d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c031040 tx=0x7f4a9c003f80 comp rx=0 tx=0).stop 2024-05-27T07:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.734+0000 7f4aa8e00640 1 -- 172.21.15.90:0/1080726786 shutdown_connections 2024-05-27T07:58:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.734+0000 7f4aa8e00640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4a880b2950 0x7f4a880b4e60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.734+0000 7f4aa8e00640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4aa41075d0 0x7f4aa41a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.734+0000 7f4aa8e00640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4108000 0x7f4aa41a55d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.734+0000 7f4aa8e00640 1 --2- 172.21.15.90:0/1080726786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4aa4102b80 0x7f4aa41a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.734+0000 7f4aa8e00640 1 -- 172.21.15.90:0/1080726786 >> 172.21.15.90:0/1080726786 conn(0x7f4aa40782d0 msgr2=0x7f4aa41051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.735+0000 7f4aa8e00640 1 -- 172.21.15.90:0/1080726786 shutdown_connections 2024-05-27T07:58:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:14.735+0000 7f4aa8e00640 1 -- 172.21.15.90:0/1080726786 wait complete. 2024-05-27T07:58:15.143 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:58:15.844 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:16.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:15 smithi090 ceph-mon[31429]: pgmap v376: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.6 MiB/s wr, 516 op/s 2024-05-27T07:58:16.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:15 smithi136 ceph-mon[35324]: pgmap v376: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.6 MiB/s wr, 516 op/s 2024-05-27T07:58:16.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:15 smithi129 ceph-mon[35254]: pgmap v376: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.6 MiB/s wr, 516 op/s 2024-05-27T07:58:18.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:17 smithi090 ceph-mon[31429]: pgmap v377: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.5 MiB/s wr, 472 op/s 2024-05-27T07:58:18.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:17 smithi090 ceph-mon[31429]: osdmap e112: 12 total, 12 up, 12 in 2024-05-27T07:58:18.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:17 smithi136 ceph-mon[35324]: pgmap v377: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.5 MiB/s wr, 472 op/s 2024-05-27T07:58:18.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:17 smithi136 ceph-mon[35324]: osdmap e112: 12 total, 12 up, 12 in 2024-05-27T07:58:18.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:17 smithi129 ceph-mon[35254]: pgmap v377: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.5 MiB/s wr, 472 op/s 2024-05-27T07:58:18.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:17 smithi129 ceph-mon[35254]: osdmap e112: 12 total, 12 up, 12 in 2024-05-27T07:58:18.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 480.0 s, 582.2 tps, lat 173.699 ms stddev 79.790 2024-05-27T07:58:20.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:19 smithi090 ceph-mon[31429]: pgmap v379: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 496 op/s 2024-05-27T07:58:20.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:19 smithi136 ceph-mon[35324]: pgmap v379: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 496 op/s 2024-05-27T07:58:20.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:19 smithi129 ceph-mon[35254]: pgmap v379: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.6 MiB/s wr, 496 op/s 2024-05-27T07:58:22.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.011+0000 7f8f64e00700 1 -- 172.21.15.90:0/2970332466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f60108730 msgr2=0x7f8f60110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:22.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.011+0000 7f8f64e00700 1 --2- 172.21.15.90:0/2970332466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f60108730 0x7f8f60110090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800b0d0 tx=0x7f8f5800b3e0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.012+0000 7f8f64e00700 1 -- 172.21.15.90:0/2970332466 shutdown_connections 2024-05-27T07:58:22.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.012+0000 7f8f64e00700 1 --2- 172.21.15.90:0/2970332466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f60108730 0x7f8f60110090 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.012+0000 7f8f64e00700 1 --2- 172.21.15.90:0/2970332466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f601034b0 0x7f8f601038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.012+0000 7f8f64e00700 1 --2- 172.21.15.90:0/2970332466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f60107de0 0x7f8f601081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.012+0000 7f8f64e00700 1 -- 172.21.15.90:0/2970332466 >> 172.21.15.90:0/2970332466 conn(0x7f8f600fedb0 msgr2=0x7f8f601011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:22.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.013+0000 7f8f64e00700 1 -- 172.21.15.90:0/2970332466 shutdown_connections 2024-05-27T07:58:22.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.013+0000 7f8f64e00700 1 -- 172.21.15.90:0/2970332466 wait complete. 2024-05-27T07:58:22.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.016+0000 7f8f64e00700 1 Processor -- start 2024-05-27T07:58:22.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.016+0000 7f8f64e00700 1 -- start start 2024-05-27T07:58:22.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.017+0000 7f8f64e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f601034b0 0x7f8f60110a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.017+0000 7f8f64e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f60107de0 0x7f8f60110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.017+0000 7f8f64e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f60108730 0x7f8f601114f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.017+0000 7f8f64e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8f60114380 con 0x7f8f60107de0 2024-05-27T07:58:22.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.017+0000 7f8f64e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f60114200 con 0x7f8f60108730 2024-05-27T07:58:22.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.017+0000 7f8f64e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8f60114500 con 0x7f8f601034b0 2024-05-27T07:58:22.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.017+0000 7f8f5d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f60107de0 0x7f8f60110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.018+0000 7f8f5e800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f60108730 0x7f8f601114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.018+0000 7f8f5e800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f60108730 0x7f8f601114f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48924/0 (socket says 172.21.15.90:48924) 2024-05-27T07:58:22.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.018+0000 7f8f5e800700 1 -- 172.21.15.90:0/4098713979 learned_addr learned my addr 172.21.15.90:0/4098713979 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:22.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.018+0000 7f8f5de00700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f601034b0 0x7f8f60110a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.019+0000 7f8f5e800700 1 -- 172.21.15.90:0/4098713979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f601034b0 msgr2=0x7f8f60110a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:22.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.019+0000 7f8f5e800700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f601034b0 0x7f8f60110a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.019+0000 7f8f5e800700 1 -- 172.21.15.90:0/4098713979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f60107de0 msgr2=0x7f8f60110fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:22.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.019+0000 7f8f5e800700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f60107de0 0x7f8f60110fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.019+0000 7f8f5e800700 1 -- 172.21.15.90:0/4098713979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f601a90d0 con 0x7f8f60108730 2024-05-27T07:58:22.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.019+0000 7f8f5d400700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f60107de0 0x7f8f60110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:22.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.019+0000 7f8f5e800700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f60108730 0x7f8f601114f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f58007d90 tx=0x7f8f58007dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:22.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.020+0000 7f8f56a00700 1 -- 172.21.15.90:0/4098713979 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5801f030 con 0x7f8f60108730 2024-05-27T07:58:22.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.020+0000 7f8f64e00700 1 -- 172.21.15.90:0/4098713979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f601a9360 con 0x7f8f60108730 2024-05-27T07:58:22.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.020+0000 7f8f64e00700 1 -- 172.21.15.90:0/4098713979 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f601a9860 con 0x7f8f60108730 2024-05-27T07:58:22.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.021+0000 7f8f56a00700 1 -- 172.21.15.90:0/4098713979 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f580075c0 con 0x7f8f60108730 2024-05-27T07:58:22.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.021+0000 7f8f56a00700 1 -- 172.21.15.90:0/4098713979 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f580125e0 con 0x7f8f60108730 2024-05-27T07:58:22.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.027+0000 7f8f56a00700 1 -- 172.21.15.90:0/4098713979 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8f58022030 con 0x7f8f60108730 2024-05-27T07:58:22.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.030+0000 7f8f56a00700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8f440b2af0 0x7f8f440b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.030+0000 7f8f56a00700 1 -- 172.21.15.90:0/4098713979 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f8f580e42a0 con 0x7f8f60108730 2024-05-27T07:58:22.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.031+0000 7f8f5de00700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8f440b2af0 0x7f8f440b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.034+0000 7f8f5de00700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8f440b2af0 0x7f8f440b5000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009a10 tx=0x7f8f50009190 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:22.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.037+0000 7f8f73a36b80 1 Processor -- start 2024-05-27T07:58:22.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.039+0000 7f8f73a36b80 1 -- start start 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.040+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 0x559f01de77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.041+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de7d30 0x559f01de1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.041+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 0x559f01de21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.041+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559f01d0fe40 con 0x559f01de1db0 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.041+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559f01cff9e0 con 0x559f01de7d30 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.041+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559f01d03a20 con 0x559f01c62e40 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.041+0000 7f8f5de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 0x559f01de77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.041+0000 7f8f5de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 0x559f01de77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53816/0 (socket says 172.21.15.90:53816) 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.041+0000 7f8f5de00700 1 -- 172.21.15.90:0/1510288471 learned_addr learned my addr 172.21.15.90:0/1510288471 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.041+0000 7f8f5d400700 1 --2- 172.21.15.90:0/1510288471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de7d30 0x559f01de1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.042+0000 7f8f5de00700 1 -- 172.21.15.90:0/1510288471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de7d30 msgr2=0x559f01de1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.042+0000 7f8f5de00700 1 --2- 172.21.15.90:0/1510288471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de7d30 0x559f01de1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.042+0000 7f8f5de00700 1 -- 172.21.15.90:0/1510288471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 msgr2=0x559f01de21c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:58:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.042+0000 7f8f5de00700 1 --2- 172.21.15.90:0/1510288471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 0x559f01de21c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.042+0000 7f8f5de00700 1 -- 172.21.15.90:0/1510288471 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f01cc8420 con 0x559f01c62e40 2024-05-27T07:58:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.043+0000 7f8f5de00700 1 --2- 172.21.15.90:0/1510288471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 0x559f01de77f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000ea00 tx=0x7f8f50011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4797cc635f790da6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.043+0000 7f8f4b400700 1 -- 172.21.15.90:0/1510288471 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5002b050 con 0x559f01c62e40 2024-05-27T07:58:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.044+0000 7f8f4b400700 1 -- 172.21.15.90:0/1510288471 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f50035070 con 0x559f01c62e40 2024-05-27T07:58:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.044+0000 7f8f4b400700 1 -- 172.21.15.90:0/1510288471 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50013850 con 0x559f01c62e40 2024-05-27T07:58:22.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.046+0000 7f8f73a36b80 1 -- 172.21.15.90:0/1510288471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 msgr2=0x559f01de77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:22.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.046+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/1510288471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 0x559f01de77f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000ea00 tx=0x7f8f50011000 comp rx=0 tx=0).stop 2024-05-27T07:58:22.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.047+0000 7f8f73a36b80 1 -- 172.21.15.90:0/1510288471 shutdown_connections 2024-05-27T07:58:22.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.047+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/1510288471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 0x559f01de21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.047+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/1510288471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 0x559f01de77f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.047+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/1510288471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de7d30 0x559f01de1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.047+0000 7f8f73a36b80 1 -- 172.21.15.90:0/1510288471 >> 172.21.15.90:0/1510288471 conn(0x559f01cc2fc0 msgr2=0x559f01ddc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:22.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.047+0000 7f8f73a36b80 1 -- 172.21.15.90:0/1510288471 shutdown_connections 2024-05-27T07:58:22.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.048+0000 7f8f73a36b80 1 -- 172.21.15.90:0/1510288471 wait complete. 2024-05-27T07:58:22.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.049+0000 7f8f73a36b80 1 -- start start 2024-05-27T07:58:22.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.050+0000 7f8f73a36b80 1 Processor -- start 2024-05-27T07:58:22.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.051+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 0x559f01deb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.051+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 0x559f01deb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.051+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de7d30 0x559f01debf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.051+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559f01c05fa0 con 0x559f01de1db0 2024-05-27T07:58:22.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.051+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559f01ca0760 con 0x559f01c62e40 2024-05-27T07:58:22.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.051+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559f01dee7a0 con 0x559f01de7d30 2024-05-27T07:58:22.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.051+0000 7f8f73a36b80 10 client.0 _ll_register_callbacks cb 0x559f01ca9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:22.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.052+0000 7f8f5d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 0x559f01deb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.052+0000 7f8f5d400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 0x559f01deb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51452/0 (socket says 172.21.15.90:51452) 2024-05-27T07:58:22.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.052+0000 7f8f5d400700 1 -- 172.21.15.90:0/389437305 learned_addr learned my addr 172.21.15.90:0/389437305 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:22.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.052+0000 7f8f5de00700 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 0x559f01deb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.053+0000 7f8f5e800700 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de7d30 0x559f01debf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.053+0000 7f8f5d400700 1 -- 172.21.15.90:0/389437305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de7d30 msgr2=0x559f01debf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.053+0000 7f8f5d400700 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de7d30 0x559f01debf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.053+0000 7f8f5d400700 1 -- 172.21.15.90:0/389437305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 msgr2=0x559f01deb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.053+0000 7f8f5d400700 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 0x559f01deb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.053+0000 7f8f5d400700 1 -- 172.21.15.90:0/389437305 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f01c4f8d0 con 0x559f01de1db0 2024-05-27T07:58:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.054+0000 7f8f5de00700 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 0x559f01deb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:22.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.054+0000 7f8f5d400700 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 0x559f01deb9c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c010a50 tx=0x7f8f4c010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f166d3d3663d6f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:22.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.054+0000 7f8f3fe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:58:22.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.055+0000 7f8f3fe00700 1 -- 172.21.15.90:0/389437305 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c01d060 con 0x559f01de1db0 2024-05-27T07:58:22.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.055+0000 7f8f3fe00700 1 -- 172.21.15.90:0/389437305 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f8f4c00eab0 con 0x559f01de1db0 2024-05-27T07:58:22.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.055+0000 7f8f73a36b80 1 -- 172.21.15.90:0/389437305 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f50035070 con 0x559f01de1db0 2024-05-27T07:58:22.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.057+0000 7f8f3fe00700 1 -- 172.21.15.90:0/389437305 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8f4c024050 con 0x559f01de1db0 2024-05-27T07:58:22.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.057+0000 7f8f73a36b80 10 client.17553 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:22.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.057+0000 7f8f73a36b80 1 -- 172.21.15.90:0/389437305 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x559f01cd0060 con 0x559f01de1db0 2024-05-27T07:58:22.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.059+0000 7f8f3fe00700 1 -- 172.21.15.90:0/389437305 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f50035070 con 0x559f01de1db0 2024-05-27T07:58:22.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.060+0000 7f8f3fe00700 1 -- 172.21.15.90:0/389437305 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f8f4c00dc20 con 0x559f01de1db0 2024-05-27T07:58:22.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.061+0000 7f8f73a36b80 10 client.17553 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.061+0000 7f8f73a36b80 10 client.17553 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:58:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.061+0000 7f8f73a36b80 20 client.17553 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.061+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x559f01dfba40 0x559f01dfdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.061+0000 7f8f73a36b80 4 client.17553 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.061+0000 7f8f73a36b80 1 -- 172.21.15.90:0/389437305 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f01d11d90 con 0x559f01dfba40 2024-05-27T07:58:22.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.062+0000 7f8f5de00700 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x559f01dfba40 0x559f01dfdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.064+0000 7f8f3fe00700 10 client.17553 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:58:22.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.064+0000 7f8f5de00700 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x559f01dfba40 0x559f01dfdeb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f4bb29bc847d6d9 server_cookie=a31fdcfef54f133a in_seq=0 out_seq=0 2024-05-27T07:58:22.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.066+0000 7f8f3fe00700 1 -- 172.21.15.90:0/389437305 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559f01d11d90 con 0x559f01dfba40 2024-05-27T07:58:22.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.066+0000 7f8f3fe00700 10 client.17553 handle_command_reply: tid=0 2024-05-27T07:58:22.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.067+0000 7f8f73a36b80 1 client.17553 shutdown 2024-05-27T07:58:22.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.067+0000 7f8f73a36b80 1 -- 172.21.15.90:0/389437305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 msgr2=0x559f01deb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:22.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.067+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 0x559f01deb9c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c010a50 tx=0x7f8f4c010e10 comp rx=0 tx=0).stop 2024-05-27T07:58:22.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.068+0000 7f8f73a36b80 1 -- 172.21.15.90:0/389437305 shutdown_connections 2024-05-27T07:58:22.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.068+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x559f01dfba40 0x559f01dfdeb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.068+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de7d30 0x559f01debf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.068+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 0x559f01deb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.068+0000 7f8f3fe00700 0 client.17553 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:58:22.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.068+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/389437305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de1db0 0x559f01deb9c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.068+0000 7f8f73a36b80 1 -- 172.21.15.90:0/389437305 >> 172.21.15.90:0/389437305 conn(0x559f01cc2fc0 msgr2=0x559f01ddc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:22.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.069+0000 7f8f73a36b80 1 -- 172.21.15.90:0/389437305 shutdown_connections 2024-05-27T07:58:22.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.069+0000 7f8f73a36b80 1 -- 172.21.15.90:0/389437305 wait complete. 2024-05-27T07:58:22.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.069+0000 7f8f73a36b80 20 client.17553 trim_cache size 0 max 16384 2024-05-27T07:58:22.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.099+0000 7f8f73a36b80 1 Processor -- start 2024-05-27T07:58:22.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.099+0000 7f8f73a36b80 1 -- start start 2024-05-27T07:58:22.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.100+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 0x559f01de1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.100+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de1db0 0x559f01de5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.100+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 0x559f01e03670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.100+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8f400107c0 con 0x559f01de7d30 2024-05-27T07:58:22.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.100+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4000d810 con 0x559f01c62e40 2024-05-27T07:58:22.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.100+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4000fa20 con 0x559f01de1db0 2024-05-27T07:58:22.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:21 smithi090 ceph-mon[31429]: pgmap v380: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 530 op/s 2024-05-27T07:58:22.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.102+0000 7f8f5d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de1db0 0x559f01de5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.102+0000 7f8f5de00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 0x559f01de1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.102+0000 7f8f5d400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de1db0 0x559f01de5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53826/0 (socket says 172.21.15.90:53826) 2024-05-27T07:58:22.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.102+0000 7f8f5d400700 1 -- 172.21.15.90:0/1162355202 learned_addr learned my addr 172.21.15.90:0/1162355202 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:22.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.103+0000 7f8f5e800700 1 --2- 172.21.15.90:0/1162355202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 0x559f01e03670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.103+0000 7f8f5d400700 1 -- 172.21.15.90:0/1162355202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 msgr2=0x559f01de1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.103+0000 7f8f5d400700 1 --2- 172.21.15.90:0/1162355202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 0x559f01de1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.103+0000 7f8f5d400700 1 -- 172.21.15.90:0/1162355202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 msgr2=0x559f01e03670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.103+0000 7f8f5d400700 1 --2- 172.21.15.90:0/1162355202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 0x559f01e03670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.103+0000 7f8f5d400700 1 -- 172.21.15.90:0/1162355202 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f01dea750 con 0x559f01de1db0 2024-05-27T07:58:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.103+0000 7f8f5d400700 1 --2- 172.21.15.90:0/1162355202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de1db0 0x559f01de5920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00e010 tx=0x7f8f4c005c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0de18fda6ffa3fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.104+0000 7f8f5de00700 1 --2- 172.21.15.90:0/1162355202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 0x559f01de1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:58:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.104+0000 7f8f4aa00700 1 -- 172.21.15.90:0/1162355202 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c00be90 con 0x559f01de1db0 2024-05-27T07:58:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.105+0000 7f8f4aa00700 1 -- 172.21.15.90:0/1162355202 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f4c019450 con 0x559f01de1db0 2024-05-27T07:58:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.105+0000 7f8f4aa00700 1 -- 172.21.15.90:0/1162355202 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c028670 con 0x559f01de1db0 2024-05-27T07:58:22.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.107+0000 7f8f73a36b80 1 -- 172.21.15.90:0/1162355202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de1db0 msgr2=0x559f01de5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.107+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/1162355202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de1db0 0x559f01de5920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00e010 tx=0x7f8f4c005c30 comp rx=0 tx=0).stop 2024-05-27T07:58:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.107+0000 7f8f73a36b80 1 -- 172.21.15.90:0/1162355202 shutdown_connections 2024-05-27T07:58:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.107+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/1162355202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 0x559f01e03670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.107+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/1162355202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01c62e40 0x559f01de1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.107+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/1162355202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01de1db0 0x559f01de5920 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.107+0000 7f8f73a36b80 1 -- 172.21.15.90:0/1162355202 >> 172.21.15.90:0/1162355202 conn(0x559f01cc2fc0 msgr2=0x559f01de5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:22.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.108+0000 7f8f73a36b80 1 -- 172.21.15.90:0/1162355202 shutdown_connections 2024-05-27T07:58:22.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.108+0000 7f8f73a36b80 1 -- 172.21.15.90:0/1162355202 wait complete. 2024-05-27T07:58:22.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.109+0000 7f8f73a36b80 1 -- start start 2024-05-27T07:58:22.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.110+0000 7f8f73a36b80 1 Processor -- start 2024-05-27T07:58:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.111+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 0x559f01df1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.111+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de1db0 0x559f01df1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.111+0000 7f8f73a36b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 0x559f01df1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.111+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4000f350 con 0x559f01de7d30 2024-05-27T07:58:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.111+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4000dee0 con 0x559f01de1db0 2024-05-27T07:58:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.111+0000 7f8f73a36b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4000e5b0 con 0x559f01c62e40 2024-05-27T07:58:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.111+0000 7f8f73a36b80 10 client.0 _ll_register_callbacks cb 0x559f01ca9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:22.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.112+0000 7f8f5d400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de1db0 0x559f01df1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.112+0000 7f8f5d400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de1db0 0x559f01df1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48966/0 (socket says 172.21.15.90:48966) 2024-05-27T07:58:22.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.112+0000 7f8f5d400700 1 -- 172.21.15.90:0/716848127 learned_addr learned my addr 172.21.15.90:0/716848127 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.112+0000 7f8f5d400700 1 -- 172.21.15.90:0/716848127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 msgr2=0x559f01df1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.113+0000 7f8f5d400700 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 0x559f01df1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.113+0000 7f8f5d400700 1 -- 172.21.15.90:0/716848127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 msgr2=0x559f01df1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.113+0000 7f8f5e800700 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 0x559f01df1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.113+0000 7f8f5d400700 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 0x559f01df1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.114+0000 7f8f5d400700 1 -- 172.21.15.90:0/716848127 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f01e76550 con 0x559f01de1db0 2024-05-27T07:58:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.114+0000 7f8f5e800700 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 0x559f01df1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.115+0000 7f8f5d400700 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de1db0 0x559f01df1590 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c019fb0 tx=0x7f8f4c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=290428aa7c03ae28 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.115+0000 7f8f3fe00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:58:22.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.121+0000 7f8f3fe00700 1 -- 172.21.15.90:0/716848127 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c01d040 con 0x559f01de1db0 2024-05-27T07:58:22.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.121+0000 7f8f3fe00700 1 -- 172.21.15.90:0/716848127 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f8f4c00b420 con 0x559f01de1db0 2024-05-27T07:58:22.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.121+0000 7f8f73a36b80 1 -- 172.21.15.90:0/716848127 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4c019450 con 0x559f01de1db0 2024-05-27T07:58:22.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.123+0000 7f8f3fe00700 1 -- 172.21.15.90:0/716848127 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8f4c040050 con 0x559f01de1db0 2024-05-27T07:58:22.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.123+0000 7f8f73a36b80 10 client.37148 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:22.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.123+0000 7f8f73a36b80 1 -- 172.21.15.90:0/716848127 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x559f01e77630 con 0x559f01de1db0 2024-05-27T07:58:22.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.124+0000 7f8f3fe00700 1 -- 172.21.15.90:0/716848127 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8f4c019450 con 0x559f01de1db0 2024-05-27T07:58:22.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.124+0000 7f8f3fe00700 1 -- 172.21.15.90:0/716848127 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f8f4c031050 con 0x559f01de1db0 2024-05-27T07:58:22.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.124+0000 7f8f73a36b80 10 client.37148 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:22.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.124+0000 7f8f73a36b80 10 client.37148 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:58:22.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.124+0000 7f8f73a36b80 20 client.37148 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:22.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.125+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x559f01dfba40 0x559f01e79c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:22.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.125+0000 7f8f73a36b80 4 client.37148 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:58:22.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.125+0000 7f8f73a36b80 1 -- 172.21.15.90:0/716848127 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f01d0b570 con 0x559f01dfba40 2024-05-27T07:58:22.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.126+0000 7f8f5de00700 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x559f01dfba40 0x559f01e79c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:22.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.128+0000 7f8f5de00700 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x559f01dfba40 0x559f01e79c80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f0e64203767af88 server_cookie=46e755215f6fcddd in_seq=0 out_seq=0 2024-05-27T07:58:22.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.128+0000 7f8f3fe00700 10 client.37148 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:58:22.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.129+0000 7f8f3fe00700 1 -- 172.21.15.90:0/716848127 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f01d0b570 con 0x559f01dfba40 2024-05-27T07:58:22.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.129+0000 7f8f3fe00700 10 client.37148 handle_command_reply: tid=0 2024-05-27T07:58:22.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.129+0000 7f8f73a36b80 1 client.37148 shutdown 2024-05-27T07:58:22.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.131+0000 7f8f73a36b80 1 -- 172.21.15.90:0/716848127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de1db0 msgr2=0x559f01df1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:22.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.131+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de1db0 0x559f01df1590 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c019fb0 tx=0x7f8f4c00ecc0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.131+0000 7f8f73a36b80 1 -- 172.21.15.90:0/716848127 shutdown_connections 2024-05-27T07:58:22.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.132+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x559f01dfba40 0x559f01e79c80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.132+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f01de7d30 0x559f01df1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.132+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559f01c62e40 0x559f01df1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.132+0000 7f8f73a36b80 1 --2- 172.21.15.90:0/716848127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559f01de1db0 0x559f01df1590 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.132+0000 7f8f73a36b80 1 -- 172.21.15.90:0/716848127 >> 172.21.15.90:0/716848127 conn(0x559f01cc2fc0 msgr2=0x559f01de6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:22.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.132+0000 7f8f3fe00700 0 client.37148 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:58:22.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.132+0000 7f8f73a36b80 1 -- 172.21.15.90:0/716848127 shutdown_connections 2024-05-27T07:58:22.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.133+0000 7f8f73a36b80 1 -- 172.21.15.90:0/716848127 wait complete. 2024-05-27T07:58:22.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.133+0000 7f8f73a36b80 20 client.37148 trim_cache size 0 max 16384 2024-05-27T07:58:22.135 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:58:22.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 -- 172.21.15.90:0/4098713979 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8f440b2af0 msgr2=0x7f8f440b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:22.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8f440b2af0 0x7f8f440b5000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009a10 tx=0x7f8f50009190 comp rx=0 tx=0).stop 2024-05-27T07:58:22.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 -- 172.21.15.90:0/4098713979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f60108730 msgr2=0x7f8f601114f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:22.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f60108730 0x7f8f601114f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f58007d90 tx=0x7f8f58007dc0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 -- 172.21.15.90:0/4098713979 shutdown_connections 2024-05-27T07:58:22.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8f440b2af0 0x7f8f440b5000 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f60108730 0x7f8f601114f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f601034b0 0x7f8f60110a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 --2- 172.21.15.90:0/4098713979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8f60107de0 0x7f8f60110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:22.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 -- 172.21.15.90:0/4098713979 >> 172.21.15.90:0/4098713979 conn(0x7f8f600fedb0 msgr2=0x7f8f60109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:22.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.136+0000 7f8f64e00700 1 -- 172.21.15.90:0/4098713979 shutdown_connections 2024-05-27T07:58:22.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:22.137+0000 7f8f64e00700 1 -- 172.21.15.90:0/4098713979 wait complete. 2024-05-27T07:58:22.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:21 smithi136 ceph-mon[35324]: pgmap v380: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 530 op/s 2024-05-27T07:58:22.314 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:21 smithi129 ceph-mon[35254]: pgmap v380: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 530 op/s 2024-05-27T07:58:23.076 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:58:23.825 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:23.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 485.0 s, 655.6 tps, lat 152.044 ms stddev 56.791 2024-05-27T07:58:24.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:23 smithi090 ceph-mon[31429]: pgmap v381: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 538 op/s 2024-05-27T07:58:24.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:23 smithi136 ceph-mon[35324]: pgmap v381: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 538 op/s 2024-05-27T07:58:24.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:23 smithi129 ceph-mon[35254]: pgmap v381: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 538 op/s 2024-05-27T07:58:26.118 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:25 smithi129 ceph-mon[35254]: pgmap v382: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 591 op/s 2024-05-27T07:58:26.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:25 smithi090 ceph-mon[31429]: pgmap v382: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 591 op/s 2024-05-27T07:58:26.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:25 smithi136 ceph-mon[35324]: pgmap v382: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 591 op/s 2024-05-27T07:58:28.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:27 smithi090 ceph-mon[31429]: pgmap v383: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.6 MiB/s wr, 598 op/s 2024-05-27T07:58:28.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:27 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:58:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.215+0000 7f11fd400640 1 -- 172.21.15.90:0/3079095346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11f00b5db0 msgr2=0x7f11f00b81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:28.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.215+0000 7f11fd400640 1 --2- 172.21.15.90:0/3079095346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11f00b5db0 0x7f11f00b81d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009900 tx=0x7f11e402a480 comp rx=0 tx=0).stop 2024-05-27T07:58:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.216+0000 7f11fd400640 1 -- 172.21.15.90:0/3079095346 shutdown_connections 2024-05-27T07:58:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.216+0000 7f11fd400640 1 --2- 172.21.15.90:0/3079095346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11f00b5db0 0x7f11f00b81d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.216+0000 7f11fd400640 1 --2- 172.21.15.90:0/3079095346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f00b8710 0x7f11f00bab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.216+0000 7f11fd400640 1 --2- 172.21.15.90:0/3079095346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11f00a5840 0x7f11f00a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.216+0000 7f11fd400640 1 -- 172.21.15.90:0/3079095346 >> 172.21.15.90:0/3079095346 conn(0x7f11f001af20 msgr2=0x7f11f001b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.216+0000 7f11fd400640 1 -- 172.21.15.90:0/3079095346 shutdown_connections 2024-05-27T07:58:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.216+0000 7f11fd400640 1 -- 172.21.15.90:0/3079095346 wait complete. 2024-05-27T07:58:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.217+0000 7f11fd400640 1 Processor -- start 2024-05-27T07:58:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.217+0000 7f11fd400640 1 -- start start 2024-05-27T07:58:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.218+0000 7f11fd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f00a5840 0x7f11f014a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.218+0000 7f11fd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11f00b5db0 0x7f11f014adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.218+0000 7f11fd400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11f00b8710 0x7f11f0144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.218+0000 7f11fd400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11f00be140 con 0x7f11f00b5db0 2024-05-27T07:58:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.218+0000 7f11fd400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11f00bdfc0 con 0x7f11f00b8710 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.218+0000 7f11fd400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11f00be2c0 con 0x7f11f00a5840 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.219+0000 7f11f5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11f00b5db0 0x7f11f014adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.219+0000 7f11f7200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11f00b8710 0x7f11f0144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.219+0000 7f11f7200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11f00b8710 0x7f11f0144a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48986/0 (socket says 172.21.15.90:48986) 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.219+0000 7f11f7200640 1 -- 172.21.15.90:0/1394742275 learned_addr learned my addr 172.21.15.90:0/1394742275 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.219+0000 7f11f6800640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f00a5840 0x7f11f014a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.219+0000 7f11f5e00640 1 -- 172.21.15.90:0/1394742275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f00a5840 msgr2=0x7f11f014a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.219+0000 7f11f5e00640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f00a5840 0x7f11f014a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.219+0000 7f11f5e00640 1 -- 172.21.15.90:0/1394742275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11f00b8710 msgr2=0x7f11f0144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.219+0000 7f11f5e00640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11f00b8710 0x7f11f0144a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.219+0000 7f11f5e00640 1 -- 172.21.15.90:0/1394742275 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f0145120 con 0x7f11f00b5db0 2024-05-27T07:58:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.220+0000 7f11f7200640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11f00b8710 0x7f11f0144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:58:28.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.220+0000 7f11f5e00640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11f00b5db0 0x7f11f014adb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009230 tx=0x7f11e4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:28.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.221+0000 7f11ef400640 1 -- 172.21.15.90:0/1394742275 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e403f070 con 0x7f11f00b5db0 2024-05-27T07:58:28.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.222+0000 7f11fd400640 1 -- 172.21.15.90:0/1394742275 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f01453b0 con 0x7f11f00b5db0 2024-05-27T07:58:28.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.222+0000 7f11fd400640 1 -- 172.21.15.90:0/1394742275 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f01458c0 con 0x7f11f00b5db0 2024-05-27T07:58:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.223+0000 7f11ef400640 1 -- 172.21.15.90:0/1394742275 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11e403bd00 con 0x7f11f00b5db0 2024-05-27T07:58:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.223+0000 7f11ef400640 1 -- 172.21.15.90:0/1394742275 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4035030 con 0x7f11f00b5db0 2024-05-27T07:58:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.227+0000 7f12043ba740 1 Processor -- start 2024-05-27T07:58:28.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.227+0000 7f12043ba740 1 -- start start 2024-05-27T07:58:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.229+0000 7f12043ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13cd21b0 0x55ab14074a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.229+0000 7f12043ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 0x55ab14074fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.229+0000 7f12043ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab14075510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.229+0000 7f12043ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ab13fe5b90 con 0x55ab13ca92f0 2024-05-27T07:58:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.229+0000 7f12043ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ab13fd5730 con 0x55ab13cb14d0 2024-05-27T07:58:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.229+0000 7f12043ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ab13fd9770 con 0x55ab13cd21b0 2024-05-27T07:58:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.230+0000 7f11f7200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13cd21b0 0x55ab14074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.230+0000 7f11f6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 0x55ab14074fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.231+0000 7f11f6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 0x55ab14074fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51510/0 (socket says 172.21.15.90:51510) 2024-05-27T07:58:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.231+0000 7f11f6800640 1 -- 172.21.15.90:0/2285001435 learned_addr learned my addr 172.21.15.90:0/2285001435 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:28.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.232+0000 7f11f6800640 1 -- 172.21.15.90:0/2285001435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13cd21b0 msgr2=0x55ab14074a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.232+0000 7f11f6800640 1 --2- 172.21.15.90:0/2285001435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13cd21b0 0x55ab14074a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.232+0000 7f11f6800640 1 -- 172.21.15.90:0/2285001435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 msgr2=0x55ab14075510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:58:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.232+0000 7f11f6800640 1 --2- 172.21.15.90:0/2285001435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab14075510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.232+0000 7f11f6800640 1 -- 172.21.15.90:0/2285001435 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab13f6e8b0 con 0x55ab13ca92f0 2024-05-27T07:58:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.232+0000 7f11f6800640 1 --2- 172.21.15.90:0/2285001435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 0x55ab14074fd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009a40 tx=0x7f11e8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66482eb080c381ae server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.233+0000 7f11e3400640 1 -- 172.21.15.90:0/2285001435 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8017070 con 0x55ab13ca92f0 2024-05-27T07:58:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.233+0000 7f11e3400640 1 -- 172.21.15.90:0/2285001435 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11e8013470 con 0x55ab13ca92f0 2024-05-27T07:58:28.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.233+0000 7f11e3400640 1 -- 172.21.15.90:0/2285001435 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8017070 con 0x55ab13ca92f0 2024-05-27T07:58:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.235+0000 7f11ef400640 1 -- 172.21.15.90:0/1394742275 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f11e4042070 con 0x7f11f00b5db0 2024-05-27T07:58:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.236+0000 7f12043ba740 1 -- 172.21.15.90:0/2285001435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 msgr2=0x55ab14074fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:28.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.236+0000 7f12043ba740 1 --2- 172.21.15.90:0/2285001435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 0x55ab14074fd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009a40 tx=0x7f11e8009f00 comp rx=0 tx=0).stop 2024-05-27T07:58:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.238+0000 7f12043ba740 1 -- 172.21.15.90:0/2285001435 shutdown_connections 2024-05-27T07:58:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.238+0000 7f12043ba740 1 --2- 172.21.15.90:0/2285001435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab14075510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.238+0000 7f12043ba740 1 --2- 172.21.15.90:0/2285001435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 0x55ab14074fd0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.238+0000 7f12043ba740 1 --2- 172.21.15.90:0/2285001435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13cd21b0 0x55ab14074a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.238+0000 7f12043ba740 1 -- 172.21.15.90:0/2285001435 >> 172.21.15.90:0/2285001435 conn(0x55ab13cc6300 msgr2=0x55ab14071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.238+0000 7f11ef400640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11dc0b2a90 0x7f11dc0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.239+0000 7f11ef400640 1 -- 172.21.15.90:0/1394742275 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f11e4102ff0 con 0x7f11f00b5db0 2024-05-27T07:58:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.239+0000 7f11f6800640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11dc0b2a90 0x7f11dc0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.239+0000 7f12043ba740 1 -- 172.21.15.90:0/2285001435 shutdown_connections 2024-05-27T07:58:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.240+0000 7f12043ba740 1 -- 172.21.15.90:0/2285001435 wait complete. 2024-05-27T07:58:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.241+0000 7f12043ba740 1 -- start start 2024-05-27T07:58:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.241+0000 7f11f6800640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11dc0b2a90 0x7f11dc0b4fa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009a10 tx=0x7f11e8008040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.241+0000 7f12043ba740 1 Processor -- start 2024-05-27T07:58:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.242+0000 7f12043ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab1407ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.242+0000 7f12043ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1407f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.242+0000 7f12043ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 0x55ab1408af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.242+0000 7f12043ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ab13cbb790 con 0x55ab13cd21b0 2024-05-27T07:58:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.242+0000 7f12043ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ab13f7dbc0 con 0x55ab13cb14d0 2024-05-27T07:58:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.242+0000 7f12043ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ab14081bc0 con 0x55ab13ca92f0 2024-05-27T07:58:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.242+0000 7f12043ba740 10 client.0 _ll_register_callbacks cb 0x55ab13f5b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.243+0000 7f11f6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab1407ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.243+0000 7f11f6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab1407ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53854/0 (socket says 172.21.15.90:53854) 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.243+0000 7f11f6800640 1 -- 172.21.15.90:0/2523740923 learned_addr learned my addr 172.21.15.90:0/2523740923 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.243+0000 7f11f7200640 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 0x55ab1408af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.243+0000 7f11f5e00640 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1407f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.243+0000 7f11f6800640 1 -- 172.21.15.90:0/2523740923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 msgr2=0x55ab1407f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.243+0000 7f11f6800640 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1407f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.243+0000 7f11f6800640 1 -- 172.21.15.90:0/2523740923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 msgr2=0x55ab1408af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.243+0000 7f11f6800640 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 0x55ab1408af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.243+0000 7f11f6800640 1 -- 172.21.15.90:0/2523740923 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab13cea210 con 0x55ab13ca92f0 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.244+0000 7f11f6800640 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab1407ec20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11e8019000 tx=0x7f11e8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23b7032c7987584c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.244+0000 7f11d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:58:28.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.244+0000 7f11d7e00640 1 -- 172.21.15.90:0/2523740923 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8017050 con 0x55ab13ca92f0 2024-05-27T07:58:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.244+0000 7f12043ba740 1 -- 172.21.15.90:0/2523740923 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e8013470 con 0x55ab13ca92f0 2024-05-27T07:58:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.244+0000 7f11f5e00640 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1407f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.245+0000 7f11d7e00640 1 -- 172.21.15.90:0/2523740923 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f11e8028610 con 0x55ab13ca92f0 2024-05-27T07:58:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.245+0000 7f11d7e00640 1 -- 172.21.15.90:0/2523740923 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f11e8017050 con 0x55ab13ca92f0 2024-05-27T07:58:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.246+0000 7f12043ba740 10 client.27250 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:28.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.246+0000 7f12043ba740 1 -- 172.21.15.90:0/2523740923 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ab13db6620 con 0x55ab13ca92f0 2024-05-27T07:58:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.246+0000 7f11d7e00640 1 -- 172.21.15.90:0/2523740923 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11e8013470 con 0x55ab13ca92f0 2024-05-27T07:58:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.247+0000 7f11d7e00640 1 -- 172.21.15.90:0/2523740923 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f11e8026710 con 0x55ab13ca92f0 2024-05-27T07:58:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.247+0000 7f12043ba740 10 client.27250 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.247+0000 7f12043ba740 10 client.27250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.247+0000 7f12043ba740 20 client.27250 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.247+0000 7f12043ba740 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ab13c90d80 0x55ab14090f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.247+0000 7f12043ba740 4 client.27250 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:28.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.247+0000 7f12043ba740 1 -- 172.21.15.90:0/2523740923 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab13fe7ae0 con 0x55ab13c90d80 2024-05-27T07:58:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:27 smithi136 ceph-mon[35324]: pgmap v383: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.6 MiB/s wr, 598 op/s 2024-05-27T07:58:28.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:27 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:58:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.251+0000 7f11f5e00640 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ab13c90d80 0x55ab14090f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.253+0000 7f11f5e00640 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ab13c90d80 0x55ab14090f60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c30ebc7eecc40da server_cookie=fc0be1e7f00cd9bc in_seq=0 out_seq=0 2024-05-27T07:58:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.253+0000 7f11d7e00640 10 client.27250 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.254+0000 7f11d7e00640 1 -- 172.21.15.90:0/2523740923 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ab13fe7ae0 con 0x55ab13c90d80 2024-05-27T07:58:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.254+0000 7f11d7e00640 10 client.27250 handle_command_reply: tid=0 2024-05-27T07:58:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.254+0000 7f12043ba740 1 client.27250 shutdown 2024-05-27T07:58:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.254+0000 7f12043ba740 1 -- 172.21.15.90:0/2523740923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 msgr2=0x55ab1407ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.254+0000 7f12043ba740 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab1407ec20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11e8019000 tx=0x7f11e8000bc0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.255+0000 7f12043ba740 1 -- 172.21.15.90:0/2523740923 shutdown_connections 2024-05-27T07:58:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.255+0000 7f12043ba740 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ab13c90d80 0x55ab14090f60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.255+0000 7f12043ba740 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1407f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.255+0000 7f12043ba740 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 0x55ab1408af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.255+0000 7f12043ba740 1 --2- 172.21.15.90:0/2523740923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab1407ec20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.255+0000 7f12043ba740 1 -- 172.21.15.90:0/2523740923 >> 172.21.15.90:0/2523740923 conn(0x55ab13cc6300 msgr2=0x55ab140767b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.256+0000 7f11d7e00640 0 client.27250 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.256+0000 7f12043ba740 1 -- 172.21.15.90:0/2523740923 shutdown_connections 2024-05-27T07:58:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.256+0000 7f12043ba740 1 -- 172.21.15.90:0/2523740923 wait complete. 2024-05-27T07:58:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.257+0000 7f12043ba740 20 client.27250 trim_cache size 0 max 16384 2024-05-27T07:58:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.288+0000 7f12043ba740 1 Processor -- start 2024-05-27T07:58:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.288+0000 7f12043ba740 1 -- start start 2024-05-27T07:58:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.289+0000 7f12043ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 0x55ab14081350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.289+0000 7f12043ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1408acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.289+0000 7f12043ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13cd21b0 0x55ab1408b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.289+0000 7f12043ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11d80119f0 con 0x55ab13ca92f0 2024-05-27T07:58:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.289+0000 7f12043ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11d800ea40 con 0x55ab13cb14d0 2024-05-27T07:58:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.289+0000 7f12043ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8010c50 con 0x55ab13cd21b0 2024-05-27T07:58:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.289+0000 7f11f6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 0x55ab14081350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.289+0000 7f11f5e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1408acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.290+0000 7f11f5e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1408acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49000/0 (socket says 172.21.15.90:49000) 2024-05-27T07:58:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.290+0000 7f11f5e00640 1 -- 172.21.15.90:0/1464006245 learned_addr learned my addr 172.21.15.90:0/1464006245 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.290+0000 7f11f7200640 1 --2- 172.21.15.90:0/1464006245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13cd21b0 0x55ab1408b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.290+0000 7f11f5e00640 1 -- 172.21.15.90:0/1464006245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13cd21b0 msgr2=0x55ab1408b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.290+0000 7f11f5e00640 1 --2- 172.21.15.90:0/1464006245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13cd21b0 0x55ab1408b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.290+0000 7f11f5e00640 1 -- 172.21.15.90:0/1464006245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 msgr2=0x55ab14081350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.290+0000 7f11f5e00640 1 --2- 172.21.15.90:0/1464006245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 0x55ab14081350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.290+0000 7f11f5e00640 1 -- 172.21.15.90:0/1464006245 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1407d170 con 0x55ab13cb14d0 2024-05-27T07:58:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.291+0000 7f11f5e00640 1 --2- 172.21.15.90:0/1464006245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1408acc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f11e40c7110 tx=0x7f11e402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee54bc1040f8062 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.292+0000 7f11e2a00640 1 -- 172.21.15.90:0/1464006245 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4049040 con 0x55ab13cb14d0 2024-05-27T07:58:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.292+0000 7f11e2a00640 1 -- 172.21.15.90:0/1464006245 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11e40c9ea0 con 0x55ab13cb14d0 2024-05-27T07:58:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.292+0000 7f11e2a00640 1 -- 172.21.15.90:0/1464006245 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4100590 con 0x55ab13cb14d0 2024-05-27T07:58:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.293+0000 7f12043ba740 1 -- 172.21.15.90:0/1464006245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 msgr2=0x55ab1408acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.293+0000 7f12043ba740 1 --2- 172.21.15.90:0/1464006245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1408acc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f11e40c7110 tx=0x7f11e402a480 comp rx=0 tx=0).stop 2024-05-27T07:58:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.293+0000 7f12043ba740 1 -- 172.21.15.90:0/1464006245 shutdown_connections 2024-05-27T07:58:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.293+0000 7f12043ba740 1 --2- 172.21.15.90:0/1464006245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13cb14d0 0x55ab1408acc0 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f11e40c7110 tx=0x7f11e402a480 comp rx=0 tx=0).stop 2024-05-27T07:58:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.293+0000 7f12043ba740 1 --2- 172.21.15.90:0/1464006245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13cd21b0 0x55ab1408b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.293+0000 7f12043ba740 1 --2- 172.21.15.90:0/1464006245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13ca92f0 0x55ab14081350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.293+0000 7f12043ba740 1 -- 172.21.15.90:0/1464006245 >> 172.21.15.90:0/1464006245 conn(0x55ab13cc6300 msgr2=0x55ab14075300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.295+0000 7f12043ba740 1 -- 172.21.15.90:0/1464006245 shutdown_connections 2024-05-27T07:58:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.295+0000 7f12043ba740 1 -- 172.21.15.90:0/1464006245 wait complete. 2024-05-27T07:58:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.296+0000 7f12043ba740 1 -- start start 2024-05-27T07:58:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.296+0000 7f12043ba740 1 Processor -- start 2024-05-27T07:58:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.297+0000 7f12043ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab14087760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.297+0000 7f12043ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 0x55ab14081e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.297+0000 7f12043ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13c661c0 0x55ab14082510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.297+0000 7f12043ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8010580 con 0x55ab13cd21b0 2024-05-27T07:58:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.297+0000 7f12043ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11d800f110 con 0x55ab13c661c0 2024-05-27T07:58:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.297+0000 7f12043ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11d800f7e0 con 0x55ab13ca92f0 2024-05-27T07:58:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.297+0000 7f12043ba740 10 client.0 _ll_register_callbacks cb 0x7f11e8009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.298+0000 7f11f6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab14087760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.298+0000 7f11f6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab14087760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:53864/0 (socket says 172.21.15.90:53864) 2024-05-27T07:58:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.298+0000 7f11f6800640 1 -- 172.21.15.90:0/4160867925 learned_addr learned my addr 172.21.15.90:0/4160867925 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.298+0000 7f11f5e00640 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 0x55ab14081e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.299+0000 7f11f7200640 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13c661c0 0x55ab14082510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.299+0000 7f11f5e00640 1 -- 172.21.15.90:0/4160867925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 msgr2=0x55ab14087760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.299+0000 7f11f5e00640 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab14087760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.299+0000 7f11f5e00640 1 -- 172.21.15.90:0/4160867925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13c661c0 msgr2=0x55ab14082510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.299+0000 7f11f5e00640 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13c661c0 0x55ab14082510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.299+0000 7f11f5e00640 1 -- 172.21.15.90:0/4160867925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab14082c80 con 0x55ab13cd21b0 2024-05-27T07:58:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.300+0000 7f11f5e00640 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 0x55ab14081e70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x55ab14072240 tx=0x7f11e4031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a210b0f76388a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.300+0000 7f11d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:58:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.305+0000 7f11d7e00640 1 -- 172.21.15.90:0/4160867925 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4049030 con 0x55ab13cd21b0 2024-05-27T07:58:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.305+0000 7f11d7e00640 1 -- 172.21.15.90:0/4160867925 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f11e40c6490 con 0x55ab13cd21b0 2024-05-27T07:58:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.305+0000 7f12043ba740 1 -- 172.21.15.90:0/4160867925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e40c9ea0 con 0x55ab13cd21b0 2024-05-27T07:58:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.305+0000 7f11d7e00640 1 -- 172.21.15.90:0/4160867925 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f11e41007a0 con 0x55ab13cd21b0 2024-05-27T07:58:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.306+0000 7f12043ba740 10 client.17565 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.306+0000 7f12043ba740 1 -- 172.21.15.90:0/4160867925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ab140f8650 con 0x55ab13cd21b0 2024-05-27T07:58:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.307+0000 7f11d7e00640 1 -- 172.21.15.90:0/4160867925 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11e40c9ea0 con 0x55ab13cd21b0 2024-05-27T07:58:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.307+0000 7f11d7e00640 1 -- 172.21.15.90:0/4160867925 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f11e40e0070 con 0x55ab13cd21b0 2024-05-27T07:58:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.308+0000 7f12043ba740 10 client.17565 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.308+0000 7f12043ba740 10 client.17565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.308+0000 7f12043ba740 20 client.17565 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.308+0000 7f12043ba740 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ab140f8c30 0x55ab140fb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.308+0000 7f12043ba740 4 client.17565 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:58:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.311+0000 7f12043ba740 1 -- 172.21.15.90:0/4160867925 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab13fe12c0 con 0x55ab140f8c30 2024-05-27T07:58:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.311+0000 7f11f6800640 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ab140f8c30 0x55ab140fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.313+0000 7f11f6800640 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ab140f8c30 0x55ab140fb050 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a41efc514c8464 server_cookie=71172b1a0bc27fc3 in_seq=0 out_seq=0 2024-05-27T07:58:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.313+0000 7f11d7e00640 10 client.17565 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:28.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:27 smithi129 ceph-mon[35254]: pgmap v383: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.6 MiB/s wr, 598 op/s 2024-05-27T07:58:28.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:27 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:58:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.825+0000 7f11d7e00640 1 -- 172.21.15.90:0/4160867925 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ab13fe12c0 con 0x55ab140f8c30 2024-05-27T07:58:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.825+0000 7f11d7e00640 10 client.17565 handle_command_reply: tid=0 2024-05-27T07:58:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.826+0000 7f12043ba740 1 client.17565 shutdown 2024-05-27T07:58:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.827+0000 7f12043ba740 1 -- 172.21.15.90:0/4160867925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 msgr2=0x55ab14081e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.827+0000 7f12043ba740 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 0x55ab14081e70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x55ab14072240 tx=0x7f11e4031630 comp rx=0 tx=0).stop 2024-05-27T07:58:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 490.0 s, 577.2 tps, lat 172.981 ms stddev 69.362 2024-05-27T07:58:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.827+0000 7f12043ba740 1 -- 172.21.15.90:0/4160867925 shutdown_connections 2024-05-27T07:58:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.827+0000 7f12043ba740 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ab140f8c30 0x55ab140fb050 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.827+0000 7f12043ba740 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ab13c661c0 0x55ab14082510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.827+0000 7f12043ba740 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ab13cd21b0 0x55ab14081e70 secure :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0x55ab14072240 tx=0x7f11e4031630 comp rx=0 tx=0).stop 2024-05-27T07:58:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.827+0000 7f12043ba740 1 --2- 172.21.15.90:0/4160867925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab13ca92f0 0x55ab14087760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.827+0000 7f11d7e00640 0 client.17565 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.827+0000 7f12043ba740 1 -- 172.21.15.90:0/4160867925 >> 172.21.15.90:0/4160867925 conn(0x55ab13cc6300 msgr2=0x55ab14079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.828+0000 7f12043ba740 1 -- 172.21.15.90:0/4160867925 shutdown_connections 2024-05-27T07:58:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.828+0000 7f12043ba740 1 -- 172.21.15.90:0/4160867925 wait complete. 2024-05-27T07:58:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.829+0000 7f12043ba740 20 client.17565 trim_cache size 0 max 16384 2024-05-27T07:58:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:58:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:58:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:58:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.831+0000 7f11fd400640 1 -- 172.21.15.90:0/1394742275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11dc0b2a90 msgr2=0x7f11dc0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.831+0000 7f11fd400640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11dc0b2a90 0x7f11dc0b4fa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009a10 tx=0x7f11e8008040 comp rx=0 tx=0).stop 2024-05-27T07:58:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.831+0000 7f11fd400640 1 -- 172.21.15.90:0/1394742275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11f00b5db0 msgr2=0x7f11f014adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.831+0000 7f11fd400640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11f00b5db0 0x7f11f014adb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009230 tx=0x7f11e4003cd0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.831+0000 7f11fd400640 1 -- 172.21.15.90:0/1394742275 shutdown_connections 2024-05-27T07:58:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.831+0000 7f11fd400640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11dc0b2a90 0x7f11dc0b4fa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.831+0000 7f11fd400640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11f00b8710 0x7f11f0144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.831+0000 7f11fd400640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11f00b5db0 0x7f11f014adb0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.831+0000 7f11fd400640 1 --2- 172.21.15.90:0/1394742275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11f00a5840 0x7f11f014a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.832+0000 7f11fd400640 1 -- 172.21.15.90:0/1394742275 >> 172.21.15.90:0/1394742275 conn(0x7f11f001af20 msgr2=0x7f11f00b61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.833+0000 7f11fd400640 1 -- 172.21.15.90:0/1394742275 shutdown_connections 2024-05-27T07:58:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:28.833+0000 7f11fd400640 1 -- 172.21.15.90:0/1394742275 wait complete. 2024-05-27T07:58:30.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:29 smithi090 ceph-mon[31429]: pgmap v384: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.2 MiB/s wr, 606 op/s 2024-05-27T07:58:30.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:29 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:58:30.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:29 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:58:30.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:29 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:58:30.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:29 smithi136 ceph-mon[35324]: pgmap v384: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.2 MiB/s wr, 606 op/s 2024-05-27T07:58:30.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:29 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:58:30.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:29 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:58:30.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:29 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:58:30.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.250+0000 7f7d13e00700 1 -- 172.21.15.90:0/2644137172 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400bab0 con 0x7f7d1410a920 2024-05-27T07:58:30.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.251+0000 7f7d1c000700 1 -- 172.21.15.90:0/2644137172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d1410a920 msgr2=0x7f7d1410ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:30.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.251+0000 7f7d1c000700 1 --2- 172.21.15.90:0/2644137172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d1410a920 0x7f7d1410ad30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7d04009990 tx=0x7f7d04009ca0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.251+0000 7f7d1c000700 1 -- 172.21.15.90:0/2644137172 shutdown_connections 2024-05-27T07:58:30.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.251+0000 7f7d1c000700 1 --2- 172.21.15.90:0/2644137172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d1410b270 0x7f7d141198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.251+0000 7f7d1c000700 1 --2- 172.21.15.90:0/2644137172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d14111bc0 0x7f7d14111fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.251+0000 7f7d1c000700 1 --2- 172.21.15.90:0/2644137172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d1410a920 0x7f7d1410ad30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.251+0000 7f7d1c000700 1 -- 172.21.15.90:0/2644137172 >> 172.21.15.90:0/2644137172 conn(0x7f7d1406f4c0 msgr2=0x7f7d1406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:30.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.253+0000 7f7d1c000700 1 -- 172.21.15.90:0/2644137172 shutdown_connections 2024-05-27T07:58:30.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.253+0000 7f7d1c000700 1 -- 172.21.15.90:0/2644137172 wait complete. 2024-05-27T07:58:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.255+0000 7f7d1c000700 1 Processor -- start 2024-05-27T07:58:30.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.255+0000 7f7d1c000700 1 -- start start 2024-05-27T07:58:30.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.257+0000 7f7d1c000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d1410a920 0x7f7d141a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.257+0000 7f7d1c000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d1410b270 0x7f7d141a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.257+0000 7f7d1c000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d14111bc0 0x7f7d141a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.257+0000 7f7d1c000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1411cb90 con 0x7f7d1410a920 2024-05-27T07:58:30.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.257+0000 7f7d1c000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1411ca10 con 0x7f7d1410b270 2024-05-27T07:58:30.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.257+0000 7f7d1c000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1411cd10 con 0x7f7d14111bc0 2024-05-27T07:58:30.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.258+0000 7f7d18c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d1410b270 0x7f7d141a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.258+0000 7f7d18c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d1410b270 0x7f7d141a3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:55382/0 (socket says 172.21.15.90:55382) 2024-05-27T07:58:30.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.258+0000 7f7d18c00700 1 -- 172.21.15.90:0/1047299294 learned_addr learned my addr 172.21.15.90:0/1047299294 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:30.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.258+0000 7f7d19600700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d1410a920 0x7f7d141a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.259+0000 7f7d1a000700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d14111bc0 0x7f7d141a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.259+0000 7f7d18c00700 1 -- 172.21.15.90:0/1047299294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d14111bc0 msgr2=0x7f7d141a7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:30.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.259+0000 7f7d18c00700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d14111bc0 0x7f7d141a7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.259+0000 7f7d18c00700 1 -- 172.21.15.90:0/1047299294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d1410a920 msgr2=0x7f7d141a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:30.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.259+0000 7f7d18c00700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d1410a920 0x7f7d141a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.259+0000 7f7d18c00700 1 -- 172.21.15.90:0/1047299294 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d141a83f0 con 0x7f7d1410b270 2024-05-27T07:58:30.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.260+0000 7f7d18c00700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d1410b270 0x7f7d141a3ac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7d040098e0 tx=0x7f7d04009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:30.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.260+0000 7f7d12000700 1 -- 172.21.15.90:0/1047299294 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04027070 con 0x7f7d1410b270 2024-05-27T07:58:30.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.261+0000 7f7d12000700 1 -- 172.21.15.90:0/1047299294 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d04022470 con 0x7f7d1410b270 2024-05-27T07:58:30.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.261+0000 7f7d12000700 1 -- 172.21.15.90:0/1047299294 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04017650 con 0x7f7d1410b270 2024-05-27T07:58:30.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.261+0000 7f7d1c000700 1 -- 172.21.15.90:0/1047299294 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d141a8680 con 0x7f7d1410b270 2024-05-27T07:58:30.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.261+0000 7f7d1c000700 1 -- 172.21.15.90:0/1047299294 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d141a8b10 con 0x7f7d1410b270 2024-05-27T07:58:30.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.268+0000 7f7d12000700 1 -- 172.21.15.90:0/1047299294 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7d0402a070 con 0x7f7d1410b270 2024-05-27T07:58:30.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.270+0000 7f7d12000700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7cfc0b2af0 0x7f7cfc0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.270+0000 7f7d12000700 1 -- 172.21.15.90:0/1047299294 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f7d040e4920 con 0x7f7d1410b270 2024-05-27T07:58:30.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.271+0000 7f7d19600700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7cfc0b2af0 0x7f7cfc0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:29 smithi129 ceph-mon[35254]: pgmap v384: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.2 MiB/s wr, 606 op/s 2024-05-27T07:58:30.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:29 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:58:30.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:29 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:58:30.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:29 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:58:30.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.402+0000 7f7d19600700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7cfc0b2af0 0x7f7cfc0b5000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7d1410bc70 tx=0x7f7d08009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:30.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.404+0000 7f7d2abc3b80 1 Processor -- start 2024-05-27T07:58:30.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.405+0000 7f7d2abc3b80 1 -- start start 2024-05-27T07:58:30.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.406+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c965b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.406+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c965bd30 0x5651c9655870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.406+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9655db0 0x5651c96561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.406+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5651c9583e40 con 0x5651c94d6e40 2024-05-27T07:58:30.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.406+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5651c95739e0 con 0x5651c965bd30 2024-05-27T07:58:30.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.406+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5651c9577a20 con 0x5651c9655db0 2024-05-27T07:58:30.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.406+0000 7f7d18c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c965bd30 0x5651c9655870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.406+0000 7f7d1a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9655db0 0x5651c96561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.407+0000 7f7d18c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c965bd30 0x5651c9655870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:55398/0 (socket says 172.21.15.90:55398) 2024-05-27T07:58:30.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.407+0000 7f7d18c00700 1 -- 172.21.15.90:0/2503631275 learned_addr learned my addr 172.21.15.90:0/2503631275 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:30.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.407+0000 7f7d1a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9655db0 0x5651c96561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33668/0 (socket says 172.21.15.90:33668) 2024-05-27T07:58:30.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.407+0000 7f7d19600700 1 --2- 172.21.15.90:0/2503631275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c965b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.408+0000 7f7d18c00700 1 -- 172.21.15.90:0/2503631275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9655db0 msgr2=0x5651c96561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:30.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.408+0000 7f7d18c00700 1 --2- 172.21.15.90:0/2503631275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9655db0 0x5651c96561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.408+0000 7f7d18c00700 1 -- 172.21.15.90:0/2503631275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 msgr2=0x5651c965b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:30.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.408+0000 7f7d18c00700 1 --2- 172.21.15.90:0/2503631275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c965b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.408+0000 7f7d18c00700 1 -- 172.21.15.90:0/2503631275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c953c420 con 0x5651c965bd30 2024-05-27T07:58:30.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.408+0000 7f7d18c00700 1 --2- 172.21.15.90:0/2503631275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c965bd30 0x5651c9655870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7d040ab7f0 tx=0x7f7d040096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bfb44b1aacc220 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:30.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.408+0000 7f7d02a00700 1 -- 172.21.15.90:0/2503631275 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d040ab860 con 0x5651c965bd30 2024-05-27T07:58:30.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.409+0000 7f7d02a00700 1 -- 172.21.15.90:0/2503631275 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d040afc10 con 0x5651c965bd30 2024-05-27T07:58:30.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.409+0000 7f7d02a00700 1 -- 172.21.15.90:0/2503631275 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d040e2060 con 0x5651c965bd30 2024-05-27T07:58:30.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.410+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2503631275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c965bd30 msgr2=0x5651c9655870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:30.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.410+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/2503631275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c965bd30 0x5651c9655870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7d040ab7f0 tx=0x7f7d040096b0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.411+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2503631275 shutdown_connections 2024-05-27T07:58:30.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.411+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/2503631275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9655db0 0x5651c96561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.411+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/2503631275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c965b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.411+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/2503631275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c965bd30 0x5651c9655870 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.411+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2503631275 >> 172.21.15.90:0/2503631275 conn(0x5651c9536fc0 msgr2=0x5651c96508d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:30.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.413+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2503631275 shutdown_connections 2024-05-27T07:58:30.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.413+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2503631275 wait complete. 2024-05-27T07:58:30.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.415+0000 7f7d2abc3b80 1 -- start start 2024-05-27T07:58:30.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.416+0000 7f7d2abc3b80 1 Processor -- start 2024-05-27T07:58:30.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.416+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c965f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.416+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9655db0 0x5651c965fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.416+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9660100 0x5651c9674070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.417+0000 7f7d19600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c965f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.417+0000 7f7d19600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c965f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40378/0 (socket says 172.21.15.90:40378) 2024-05-27T07:58:30.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.417+0000 7f7d18c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9655db0 0x5651c965fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.417+0000 7f7d18c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9655db0 0x5651c965fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:55400/0 (socket says 172.21.15.90:55400) 2024-05-27T07:58:30.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.418+0000 7f7d1a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9660100 0x5651c9674070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.418+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5651c9479fa0 con 0x5651c94d6e40 2024-05-27T07:58:30.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.418+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5651c9514760 con 0x5651c9655db0 2024-05-27T07:58:30.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.418+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5651c96625c0 con 0x5651c9660100 2024-05-27T07:58:30.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.418+0000 7f7d2abc3b80 10 client.0 _ll_register_callbacks cb 0x5651c951dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:30.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.418+0000 7f7d19600700 1 -- 172.21.15.90:0/2012451751 learned_addr learned my addr 172.21.15.90:0/2012451751 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:30.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.418+0000 7f7d18c00700 1 -- 172.21.15.90:0/2012451751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9660100 msgr2=0x5651c9674070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:30.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.419+0000 7f7d18c00700 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9660100 0x5651c9674070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.419+0000 7f7d18c00700 1 -- 172.21.15.90:0/2012451751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 msgr2=0x5651c965f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:30.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.419+0000 7f7d18c00700 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c965f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.419+0000 7f7d18c00700 1 -- 172.21.15.90:0/2012451751 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c94c38d0 con 0x5651c9655db0 2024-05-27T07:58:30.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.419+0000 7f7d19600700 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c965f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:30.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.420+0000 7f7d18c00700 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9655db0 0x5651c965fbc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5651c9651550 tx=0x7f7d040ab740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f44f8a2552d7563 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:30.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.420+0000 7f7cf7400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:58:30.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.421+0000 7f7cf7400700 1 -- 172.21.15.90:0/2012451751 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d040ab860 con 0x5651c9655db0 2024-05-27T07:58:30.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.421+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2012451751 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d040afc10 con 0x5651c9655db0 2024-05-27T07:58:30.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.421+0000 7f7cf7400700 1 -- 172.21.15.90:0/2012451751 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f7d040a7e50 con 0x5651c9655db0 2024-05-27T07:58:30.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.422+0000 7f7cf7400700 1 -- 172.21.15.90:0/2012451751 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7d040abcd0 con 0x5651c9655db0 2024-05-27T07:58:30.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.422+0000 7f7d2abc3b80 10 client.37172 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:30.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.422+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2012451751 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5651c9544060 con 0x5651c9655db0 2024-05-27T07:58:30.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.423+0000 7f7cf7400700 1 -- 172.21.15.90:0/2012451751 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d040afc10 con 0x5651c9655db0 2024-05-27T07:58:30.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.424+0000 7f7cf7400700 1 -- 172.21.15.90:0/2012451751 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f7d040050d0 con 0x5651c9655db0 2024-05-27T07:58:30.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.425+0000 7f7d2abc3b80 10 client.37172 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:30.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.425+0000 7f7d2abc3b80 10 client.37172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:58:30.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.425+0000 7f7d2abc3b80 20 client.37172 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:30.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.425+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5651c9678120 0x5651c967a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.425+0000 7f7d2abc3b80 4 client.37172 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:30.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.425+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2012451751 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651c9585d90 con 0x5651c9678120 2024-05-27T07:58:30.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.426+0000 7f7d19600700 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5651c9678120 0x5651c967a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.426+0000 7f7d19600700 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5651c9678120 0x5651c967a540 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f57bb80f44f6f0ec server_cookie=a4907c20814b38e3 in_seq=0 out_seq=0 2024-05-27T07:58:30.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.427+0000 7f7cf7400700 10 client.37172 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:58:30.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.428+0000 7f7cf7400700 1 -- 172.21.15.90:0/2012451751 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5651c9585d90 con 0x5651c9678120 2024-05-27T07:58:30.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.428+0000 7f7cf7400700 10 client.37172 handle_command_reply: tid=0 2024-05-27T07:58:30.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.428+0000 7f7d2abc3b80 1 client.37172 shutdown 2024-05-27T07:58:30.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.429+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2012451751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9655db0 msgr2=0x5651c965fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:30.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.429+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9655db0 0x5651c965fbc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5651c9651550 tx=0x7f7d040ab740 comp rx=0 tx=0).stop 2024-05-27T07:58:30.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.429+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2012451751 shutdown_connections 2024-05-27T07:58:30.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.429+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5651c9678120 0x5651c967a540 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.429+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9660100 0x5651c9674070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.429+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c965f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.429+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/2012451751 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9655db0 0x5651c965fbc0 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5651c9651550 tx=0x7f7d040ab740 comp rx=0 tx=0).stop 2024-05-27T07:58:30.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.429+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2012451751 >> 172.21.15.90:0/2012451751 conn(0x5651c9536fc0 msgr2=0x5651c9650580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:30.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.429+0000 7f7cf7400700 0 client.37172 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:58:30.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.430+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2012451751 shutdown_connections 2024-05-27T07:58:30.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.430+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/2012451751 wait complete. 2024-05-27T07:58:30.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.430+0000 7f7d2abc3b80 20 client.37172 trim_cache size 0 max 16384 2024-05-27T07:58:30.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.458+0000 7f7d2abc3b80 1 Processor -- start 2024-05-27T07:58:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.458+0000 7f7d2abc3b80 1 -- start start 2024-05-27T07:58:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.459+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c94d6e40 0x5651c9655870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.459+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 0x5651c965fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.459+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c9650b60 0x5651c9674060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.459+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141b85e0 con 0x5651c9650b60 2024-05-27T07:58:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.459+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141b5630 con 0x5651c9660100 2024-05-27T07:58:30.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.459+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141b7840 con 0x5651c94d6e40 2024-05-27T07:58:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.460+0000 7f7d1a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c9650b60 0x5651c9674060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.460+0000 7f7d19600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c94d6e40 0x5651c9655870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.460+0000 7f7d19600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c94d6e40 0x5651c9655870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33674/0 (socket says 172.21.15.90:33674) 2024-05-27T07:58:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.460+0000 7f7d1a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c9650b60 0x5651c9674060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40380/0 (socket says 172.21.15.90:40380) 2024-05-27T07:58:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.460+0000 7f7d19600700 1 -- 172.21.15.90:0/392306288 learned_addr learned my addr 172.21.15.90:0/392306288 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:30.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.460+0000 7f7d18c00700 1 --2- 172.21.15.90:0/392306288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 0x5651c965fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.462+0000 7f7d1a000700 1 -- 172.21.15.90:0/392306288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c94d6e40 msgr2=0x5651c9655870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.462+0000 7f7d1a000700 1 --2- 172.21.15.90:0/392306288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c94d6e40 0x5651c9655870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.462+0000 7f7d1a000700 1 -- 172.21.15.90:0/392306288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 msgr2=0x5651c965fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.462+0000 7f7d1a000700 1 --2- 172.21.15.90:0/392306288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 0x5651c965fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.462+0000 7f7d1a000700 1 -- 172.21.15.90:0/392306288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651c965dbf0 con 0x5651c9650b60 2024-05-27T07:58:30.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.462+0000 7f7d1a000700 1 --2- 172.21.15.90:0/392306288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c9650b60 0x5651c9674060 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009e20 tx=0x7f7d0c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6384669404c5111b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:30.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.462+0000 7f7d02000700 1 -- 172.21.15.90:0/392306288 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c01b070 con 0x5651c9650b60 2024-05-27T07:58:30.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.463+0000 7f7d02000700 1 -- 172.21.15.90:0/392306288 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d0c004db0 con 0x5651c9650b60 2024-05-27T07:58:30.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.463+0000 7f7d02000700 1 -- 172.21.15.90:0/392306288 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c005280 con 0x5651c9650b60 2024-05-27T07:58:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.464+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/392306288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c9650b60 msgr2=0x5651c9674060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:30.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.464+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/392306288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c9650b60 0x5651c9674060 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009e20 tx=0x7f7d0c0077e0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.465+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/392306288 shutdown_connections 2024-05-27T07:58:30.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.465+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/392306288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c9650b60 0x5651c9674060 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.465+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/392306288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c94d6e40 0x5651c9655870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.465+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/392306288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 0x5651c965fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.465+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/392306288 >> 172.21.15.90:0/392306288 conn(0x5651c9536fc0 msgr2=0x5651c965f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:30.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.466+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/392306288 shutdown_connections 2024-05-27T07:58:30.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.467+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/392306288 wait complete. 2024-05-27T07:58:30.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.467+0000 7f7d2abc3b80 1 -- start start 2024-05-27T07:58:30.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.467+0000 7f7d2abc3b80 1 Processor -- start 2024-05-27T07:58:30.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.469+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c967bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.469+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9650b60 0x5651c967c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.469+0000 7f7d2abc3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 0x5651c967c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.470+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141b7170 con 0x5651c94d6e40 2024-05-27T07:58:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.470+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141b5d00 con 0x5651c9660100 2024-05-27T07:58:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.470+0000 7f7d2abc3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141b63d0 con 0x5651c9650b60 2024-05-27T07:58:30.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.470+0000 7f7d2abc3b80 10 client.0 _ll_register_callbacks cb 0x5651c951dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.470+0000 7f7d18c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9650b60 0x5651c967c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.470+0000 7f7d18c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9650b60 0x5651c967c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33682/0 (socket says 172.21.15.90:33682) 2024-05-27T07:58:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.470+0000 7f7d18c00700 1 -- 172.21.15.90:0/3456902286 learned_addr learned my addr 172.21.15.90:0/3456902286 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.470+0000 7f7d19600700 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c967bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.470+0000 7f7d1a000700 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 0x5651c967c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.471+0000 7f7d1a000700 1 -- 172.21.15.90:0/3456902286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9650b60 msgr2=0x5651c967c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:30.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.471+0000 7f7d1a000700 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9650b60 0x5651c967c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.471+0000 7f7d1a000700 1 -- 172.21.15.90:0/3456902286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 msgr2=0x5651c967bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:30.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.471+0000 7f7d1a000700 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c967bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.471+0000 7f7d1a000700 1 -- 172.21.15.90:0/3456902286 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651c96e1a10 con 0x5651c9660100 2024-05-27T07:58:30.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.471+0000 7f7d18c00700 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9650b60 0x5651c967c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:58:30.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.472+0000 7f7d1a000700 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 0x5651c967c5a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c004fa0 tx=0x7f7d0c017440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=321d9951a203bcc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:30.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.472+0000 7f7cf7400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:58:30.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.472+0000 7f7cf7400700 1 -- 172.21.15.90:0/3456902286 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c01b040 con 0x5651c9660100 2024-05-27T07:58:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.472+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/3456902286 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0c004db0 con 0x5651c9660100 2024-05-27T07:58:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.472+0000 7f7cf7400700 1 -- 172.21.15.90:0/3456902286 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f7d0c0177f0 con 0x5651c9660100 2024-05-27T07:58:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.473+0000 7f7cf7400700 1 -- 172.21.15.90:0/3456902286 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7d0c017a30 con 0x5651c9660100 2024-05-27T07:58:30.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.473+0000 7f7d2abc3b80 10 client.37178 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.473+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/3456902286 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5651c96e2c50 con 0x5651c9660100 2024-05-27T07:58:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.473+0000 7f7cf7400700 1 -- 172.21.15.90:0/3456902286 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d0c004db0 con 0x5651c9660100 2024-05-27T07:58:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.473+0000 7f7cf7400700 1 -- 172.21.15.90:0/3456902286 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f7d0c004680 con 0x5651c9660100 2024-05-27T07:58:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.474+0000 7f7d2abc3b80 10 client.37178 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.474+0000 7f7d2abc3b80 10 client.37178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:58:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.474+0000 7f7d2abc3b80 20 client.37178 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.474+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5651c96e3200 0x5651c96e5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.474+0000 7f7d2abc3b80 4 client.37178 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:58:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.474+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/3456902286 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651c957f570 con 0x5651c96e3200 2024-05-27T07:58:30.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.474+0000 7f7d19600700 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5651c96e3200 0x5651c96e5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:30.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.476+0000 7f7d19600700 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5651c96e3200 0x5651c96e5620 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5a7e8086e493a13 server_cookie=3c51c2d5c6569508 in_seq=0 out_seq=0 2024-05-27T07:58:30.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.476+0000 7f7cf7400700 10 client.37178 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:58:30.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.477+0000 7f7cf7400700 1 -- 172.21.15.90:0/3456902286 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651c957f570 con 0x5651c96e3200 2024-05-27T07:58:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.477+0000 7f7cf7400700 10 client.37178 handle_command_reply: tid=0 2024-05-27T07:58:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.477+0000 7f7d2abc3b80 1 client.37178 shutdown 2024-05-27T07:58:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.477+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/3456902286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 msgr2=0x5651c967c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:30.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.477+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 0x5651c967c5a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c004fa0 tx=0x7f7d0c017440 comp rx=0 tx=0).stop 2024-05-27T07:58:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.478+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/3456902286 shutdown_connections 2024-05-27T07:58:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.478+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5651c96e3200 0x5651c96e5620 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.478+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651c9660100 0x5651c967c5a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.478+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651c94d6e40 0x5651c967bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.478+0000 7f7d2abc3b80 1 --2- 172.21.15.90:0/3456902286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651c9650b60 0x5651c967c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.478+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/3456902286 >> 172.21.15.90:0/3456902286 conn(0x5651c9536fc0 msgr2=0x5651c964fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:30.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.478+0000 7f7cf7400700 0 client.37178 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:58:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.479+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/3456902286 shutdown_connections 2024-05-27T07:58:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.479+0000 7f7d2abc3b80 1 -- 172.21.15.90:0/3456902286 wait complete. 2024-05-27T07:58:30.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.479+0000 7f7d2abc3b80 20 client.37178 trim_cache size 0 max 16384 2024-05-27T07:58:30.483 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:58:30.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.484+0000 7f7d1c000700 1 -- 172.21.15.90:0/1047299294 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7cfc0b2af0 msgr2=0x7f7cfc0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:30.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.484+0000 7f7d1c000700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7cfc0b2af0 0x7f7cfc0b5000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7d1410bc70 tx=0x7f7d08009260 comp rx=0 tx=0).stop 2024-05-27T07:58:30.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.484+0000 7f7d1c000700 1 -- 172.21.15.90:0/1047299294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d1410b270 msgr2=0x7f7d141a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:30.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.484+0000 7f7d1c000700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d1410b270 0x7f7d141a3ac0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7d040098e0 tx=0x7f7d04009550 comp rx=0 tx=0).stop 2024-05-27T07:58:30.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.484+0000 7f7d1c000700 1 -- 172.21.15.90:0/1047299294 shutdown_connections 2024-05-27T07:58:30.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.484+0000 7f7d1c000700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d14111bc0 0x7f7d141a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.484+0000 7f7d1c000700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d1410a920 0x7f7d141a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.484+0000 7f7d1c000700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7cfc0b2af0 0x7f7cfc0b5000 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.484+0000 7f7d1c000700 1 --2- 172.21.15.90:0/1047299294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d1410b270 0x7f7d141a3ac0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:30.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.484+0000 7f7d1c000700 1 -- 172.21.15.90:0/1047299294 >> 172.21.15.90:0/1047299294 conn(0x7f7d1406f4c0 msgr2=0x7f7d14110630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:30.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.485+0000 7f7d1c000700 1 -- 172.21.15.90:0/1047299294 shutdown_connections 2024-05-27T07:58:30.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:30.485+0000 7f7d1c000700 1 -- 172.21.15.90:0/1047299294 wait complete. 2024-05-27T07:58:31.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:30 smithi090 ceph-mon[31429]: pgmap v385: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 699 op/s 2024-05-27T07:58:31.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:30 smithi090 ceph-mon[31429]: pgmap v386: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.3 MiB/s wr, 694 op/s 2024-05-27T07:58:31.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:30 smithi136 ceph-mon[35324]: pgmap v385: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 699 op/s 2024-05-27T07:58:31.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:30 smithi136 ceph-mon[35324]: pgmap v386: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.3 MiB/s wr, 694 op/s 2024-05-27T07:58:31.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:30 smithi129 ceph-mon[35254]: pgmap v385: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 10 MiB/s wr, 699 op/s 2024-05-27T07:58:31.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:30 smithi129 ceph-mon[35254]: pgmap v386: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 9.3 MiB/s wr, 694 op/s 2024-05-27T07:58:31.607 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:58:32.690 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:33.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:32 smithi090 ceph-mon[31429]: pgmap v387: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.8 MiB/s wr, 701 op/s 2024-05-27T07:58:33.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:32 smithi136 ceph-mon[35324]: pgmap v387: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.8 MiB/s wr, 701 op/s 2024-05-27T07:58:33.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:32 smithi129 ceph-mon[35254]: pgmap v387: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.8 MiB/s wr, 701 op/s 2024-05-27T07:58:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 495.0 s, 591.4 tps, lat 167.742 ms stddev 94.051 2024-05-27T07:58:35.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:34 smithi090 ceph-mon[31429]: pgmap v388: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 501 op/s 2024-05-27T07:58:35.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:34 smithi136 ceph-mon[35324]: pgmap v388: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 501 op/s 2024-05-27T07:58:35.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:34 smithi129 ceph-mon[35254]: pgmap v388: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 501 op/s 2024-05-27T07:58:36.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:35 smithi090 ceph-mon[31429]: pgmap v389: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 615 op/s 2024-05-27T07:58:36.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:35 smithi136 ceph-mon[35324]: pgmap v389: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 615 op/s 2024-05-27T07:58:36.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:35 smithi129 ceph-mon[35254]: pgmap v389: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 MiB/s wr, 615 op/s 2024-05-27T07:58:38.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 500.0 s, 539.8 tps, lat 187.284 ms stddev 134.982 2024-05-27T07:58:38.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:38 smithi090 ceph-mon[31429]: pgmap v390: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.3 MiB/s wr, 566 op/s 2024-05-27T07:58:39.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:38 smithi136 ceph-mon[35324]: pgmap v390: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.3 MiB/s wr, 566 op/s 2024-05-27T07:58:39.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:38 smithi129 ceph-mon[35254]: pgmap v390: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.3 MiB/s wr, 566 op/s 2024-05-27T07:58:39.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.687+0000 7fd97d000700 1 -- 172.21.15.90:0/3161303390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd978109860 msgr2=0x7fd9781111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.687+0000 7fd97d000700 1 --2- 172.21.15.90:0/3161303390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd978109860 0x7fd9781111c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b0d0 tx=0x7fd97000b3e0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.687+0000 7fd97d000700 1 -- 172.21.15.90:0/3161303390 shutdown_connections 2024-05-27T07:58:39.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.687+0000 7fd97d000700 1 --2- 172.21.15.90:0/3161303390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd978109860 0x7fd9781111c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.687+0000 7fd97d000700 1 --2- 172.21.15.90:0/3161303390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9781045e0 0x7fd9781049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.687+0000 7fd97d000700 1 --2- 172.21.15.90:0/3161303390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd978108f10 0x7fd978109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.687+0000 7fd97d000700 1 -- 172.21.15.90:0/3161303390 >> 172.21.15.90:0/3161303390 conn(0x7fd978100030 msgr2=0x7fd978102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.688+0000 7fd97d000700 1 -- 172.21.15.90:0/3161303390 shutdown_connections 2024-05-27T07:58:39.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.688+0000 7fd97d000700 1 -- 172.21.15.90:0/3161303390 wait complete. 2024-05-27T07:58:39.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.689+0000 7fd97d000700 1 Processor -- start 2024-05-27T07:58:39.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.689+0000 7fd97d000700 1 -- start start 2024-05-27T07:58:39.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.690+0000 7fd97d000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9781045e0 0x7fd97819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.690+0000 7fd97d000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd978108f10 0x7fd97819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.690+0000 7fd97d000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd978109860 0x7fd9781a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.690+0000 7fd97d000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd978114360 con 0x7fd978108f10 2024-05-27T07:58:39.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.690+0000 7fd97d000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781141e0 con 0x7fd9781045e0 2024-05-27T07:58:39.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.690+0000 7fd97d000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781144e0 con 0x7fd978109860 2024-05-27T07:58:39.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.691+0000 7fd975400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd978108f10 0x7fd97819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.691+0000 7fd975400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd978108f10 0x7fd97819d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35832/0 (socket says 172.21.15.90:35832) 2024-05-27T07:58:39.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.691+0000 7fd975400700 1 -- 172.21.15.90:0/623596645 learned_addr learned my addr 172.21.15.90:0/623596645 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:39.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.691+0000 7fd975e00700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9781045e0 0x7fd97819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.691+0000 7fd976800700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd978109860 0x7fd9781a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.691+0000 7fd975400700 1 -- 172.21.15.90:0/623596645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd978109860 msgr2=0x7fd9781a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.691+0000 7fd975400700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd978109860 0x7fd9781a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.691+0000 7fd975400700 1 -- 172.21.15.90:0/623596645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9781045e0 msgr2=0x7fd97819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.691+0000 7fd975400700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9781045e0 0x7fd97819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.691+0000 7fd975400700 1 -- 172.21.15.90:0/623596645 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9781a1dc0 con 0x7fd978108f10 2024-05-27T07:58:39.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.694+0000 7fd975400700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd978108f10 0x7fd97819d490 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd96400d950 tx=0x7fd96400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.696+0000 7fd96ea00700 1 -- 172.21.15.90:0/623596645 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964017070 con 0x7fd978108f10 2024-05-27T07:58:39.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.696+0000 7fd96ea00700 1 -- 172.21.15.90:0/623596645 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd964005020 con 0x7fd978108f10 2024-05-27T07:58:39.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.696+0000 7fd96ea00700 1 -- 172.21.15.90:0/623596645 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96400f980 con 0x7fd978108f10 2024-05-27T07:58:39.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.697+0000 7fd97d000700 1 -- 172.21.15.90:0/623596645 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9781a20b0 con 0x7fd978108f10 2024-05-27T07:58:39.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.697+0000 7fd97d000700 1 -- 172.21.15.90:0/623596645 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9781a99d0 con 0x7fd978108f10 2024-05-27T07:58:39.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.704+0000 7fd96ea00700 1 -- 172.21.15.90:0/623596645 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd96400fb20 con 0x7fd978108f10 2024-05-27T07:58:39.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.706+0000 7fd96ea00700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd95c0b2bc0 0x7fd95c0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.706+0000 7fd96ea00700 1 -- 172.21.15.90:0/623596645 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7fd9640e1cd0 con 0x7fd978108f10 2024-05-27T07:58:39.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.707+0000 7fd975e00700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd95c0b2bc0 0x7fd95c0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.709+0000 7fd975e00700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd95c0b2bc0 0x7fd95c0b50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd968005950 tx=0x7fd96800b410 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.736+0000 7fd98bac7b80 1 Processor -- start 2024-05-27T07:58:39.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.736+0000 7fd98bac7b80 1 -- start start 2024-05-27T07:58:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.736+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.736+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e05023e40 0x559e051a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.736+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e051af110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.736+0000 7fd98bac7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559e050d0e40 con 0x559e05023e40 2024-05-27T07:58:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.736+0000 7fd98bac7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559e050c09e0 con 0x559e051a2e70 2024-05-27T07:58:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.736+0000 7fd98bac7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559e050c4a20 con 0x559e0510f9f0 2024-05-27T07:58:39.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.737+0000 7fd976800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e051af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.737+0000 7fd975e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.737+0000 7fd976800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e051af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49776/0 (socket says 172.21.15.90:49776) 2024-05-27T07:58:39.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.737+0000 7fd976800700 1 -- 172.21.15.90:0/2779361290 learned_addr learned my addr 172.21.15.90:0/2779361290 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.738+0000 7fd976800700 1 -- 172.21.15.90:0/2779361290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 msgr2=0x559e051a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.738+0000 7fd976800700 1 --2- 172.21.15.90:0/2779361290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.738+0000 7fd976800700 1 -- 172.21.15.90:0/2779361290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e05023e40 msgr2=0x559e051a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.738+0000 7fd976800700 1 --2- 172.21.15.90:0/2779361290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e05023e40 0x559e051a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.738+0000 7fd976800700 1 -- 172.21.15.90:0/2779361290 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e05089420 con 0x559e051a2e70 2024-05-27T07:58:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.739+0000 7fd975e00700 1 --2- 172.21.15.90:0/2779361290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:39.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.739+0000 7fd976800700 1 --2- 172.21.15.90:0/2779361290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e051af110 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd970009c30 tx=0x7fd970009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d48e2a4830c7336 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.741+0000 7fd963400700 1 -- 172.21.15.90:0/2779361290 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9700093d0 con 0x559e051a2e70 2024-05-27T07:58:39.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.741+0000 7fd963400700 1 -- 172.21.15.90:0/2779361290 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd970020070 con 0x559e051a2e70 2024-05-27T07:58:39.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.741+0000 7fd963400700 1 -- 172.21.15.90:0/2779361290 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970005860 con 0x559e051a2e70 2024-05-27T07:58:39.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.742+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2779361290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 msgr2=0x559e051af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.742+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2779361290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e051af110 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd970009c30 tx=0x7fd970009c60 comp rx=0 tx=0).stop 2024-05-27T07:58:39.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.742+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2779361290 shutdown_connections 2024-05-27T07:58:39.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.742+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2779361290 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e051af110 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.742+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2779361290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.742+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2779361290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e05023e40 0x559e051a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.742+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2779361290 >> 172.21.15.90:0/2779361290 conn(0x559e050bd390 msgr2=0x559e0519e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.743+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2779361290 shutdown_connections 2024-05-27T07:58:39.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.743+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2779361290 wait complete. 2024-05-27T07:58:39.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.743+0000 7fd98bac7b80 1 -- start start 2024-05-27T07:58:39.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.744+0000 7fd98bac7b80 1 Processor -- start 2024-05-27T07:58:39.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.745+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e05023e40 0x559e051ac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.745+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e0510f9f0 0x559e051acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.745+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e051a2e70 0x559e051ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.745+0000 7fd98bac7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559e04fc6fa0 con 0x559e05023e40 2024-05-27T07:58:39.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.745+0000 7fd98bac7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559e05061760 con 0x559e0510f9f0 2024-05-27T07:58:39.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.745+0000 7fd98bac7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559e051b82d0 con 0x559e051a2e70 2024-05-27T07:58:39.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.745+0000 7fd98bac7b80 10 client.0 _ll_register_callbacks cb 0x559e0506ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:39.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.746+0000 7fd975400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e0510f9f0 0x559e051acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.746+0000 7fd976800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e051a2e70 0x559e051ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.746+0000 7fd975e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e05023e40 0x559e051ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.746+0000 7fd975400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e0510f9f0 0x559e051acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49778/0 (socket says 172.21.15.90:49778) 2024-05-27T07:58:39.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.746+0000 7fd975400700 1 -- 172.21.15.90:0/2108851724 learned_addr learned my addr 172.21.15.90:0/2108851724 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:39.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.747+0000 7fd975e00700 1 -- 172.21.15.90:0/2108851724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e051a2e70 msgr2=0x559e051ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.747+0000 7fd975e00700 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e051a2e70 0x559e051ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.747+0000 7fd975e00700 1 -- 172.21.15.90:0/2108851724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e0510f9f0 msgr2=0x559e051acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.747+0000 7fd975e00700 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e0510f9f0 0x559e051acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.747+0000 7fd975e00700 1 -- 172.21.15.90:0/2108851724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e050108d0 con 0x559e05023e40 2024-05-27T07:58:39.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.749+0000 7fd975e00700 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e05023e40 0x559e051ac680 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd968010a40 tx=0x7fd9680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c4b5d8e7ebebf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.749+0000 7fd957e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:58:39.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.750+0000 7fd957e00700 1 -- 172.21.15.90:0/2108851724 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968027470 con 0x559e05023e40 2024-05-27T07:58:39.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.750+0000 7fd957e00700 1 -- 172.21.15.90:0/2108851724 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7fd968027610 con 0x559e05023e40 2024-05-27T07:58:39.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.750+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2108851724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd970020070 con 0x559e05023e40 2024-05-27T07:58:39.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.751+0000 7fd957e00700 1 -- 172.21.15.90:0/2108851724 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd968027810 con 0x559e05023e40 2024-05-27T07:58:39.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.751+0000 7fd98bac7b80 10 client.17592 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:39.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.751+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2108851724 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x559e05091060 con 0x559e05023e40 2024-05-27T07:58:39.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.752+0000 7fd957e00700 1 -- 172.21.15.90:0/2108851724 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd970020070 con 0x559e05023e40 2024-05-27T07:58:39.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.753+0000 7fd957e00700 1 -- 172.21.15.90:0/2108851724 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fd968031aa0 con 0x559e05023e40 2024-05-27T07:58:39.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.755+0000 7fd98bac7b80 10 client.17592 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:39.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.755+0000 7fd98bac7b80 10 client.17592 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:58:39.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.755+0000 7fd98bac7b80 20 client.17592 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:39.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.755+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559e051d6330 0x559e051d8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.755+0000 7fd98bac7b80 4 client.17592 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:39.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.755+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2108851724 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e050d2d90 con 0x559e051d6330 2024-05-27T07:58:39.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.756+0000 7fd975400700 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559e051d6330 0x559e051d8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.758+0000 7fd975400700 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559e051d6330 0x559e051d8790 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d70a9ac216456f25 server_cookie=658a22a289a8dadf in_seq=0 out_seq=0 2024-05-27T07:58:39.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.758+0000 7fd957e00700 10 client.17592 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:58:39.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.761+0000 7fd957e00700 1 -- 172.21.15.90:0/2108851724 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559e050d2d90 con 0x559e051d6330 2024-05-27T07:58:39.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.761+0000 7fd957e00700 10 client.17592 handle_command_reply: tid=0 2024-05-27T07:58:39.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.761+0000 7fd98bac7b80 1 client.17592 shutdown 2024-05-27T07:58:39.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.762+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2108851724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e05023e40 msgr2=0x559e051ac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.762+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e05023e40 0x559e051ac680 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd968010a40 tx=0x7fd9680047c0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.762+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2108851724 shutdown_connections 2024-05-27T07:58:39.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.762+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e051a2e70 0x559e051ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.762+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e05023e40 0x559e051ac680 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.762+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559e051d6330 0x559e051d8790 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.762+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2108851724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e0510f9f0 0x559e051acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.762+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2108851724 >> 172.21.15.90:0/2108851724 conn(0x559e050bd390 msgr2=0x559e051a2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.762+0000 7fd957e00700 0 client.17592 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:58:39.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.763+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2108851724 shutdown_connections 2024-05-27T07:58:39.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.763+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2108851724 wait complete. 2024-05-27T07:58:39.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.763+0000 7fd98bac7b80 20 client.17592 trim_cache size 0 max 16384 2024-05-27T07:58:39.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.793+0000 7fd98bac7b80 1 Processor -- start 2024-05-27T07:58:39.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.793+0000 7fd98bac7b80 1 -- start start 2024-05-27T07:58:39.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.793+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e05023e40 0x559e051a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.794+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.794+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e051a2e70 0x559e051ddf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.794+0000 7fd98bac7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd978073340 con 0x559e051a2e70 2024-05-27T07:58:39.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.794+0000 7fd98bac7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd978070390 con 0x559e05023e40 2024-05-27T07:58:39.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.794+0000 7fd98bac7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780725a0 con 0x559e0510f9f0 2024-05-27T07:58:39.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.794+0000 7fd975400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.794+0000 7fd975400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44246/0 (socket says 172.21.15.90:44246) 2024-05-27T07:58:39.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.794+0000 7fd975400700 1 -- 172.21.15.90:0/1757754158 learned_addr learned my addr 172.21.15.90:0/1757754158 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:39.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.795+0000 7fd976800700 1 --2- 172.21.15.90:0/1757754158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e051a2e70 0x559e051ddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.795+0000 7fd975e00700 1 --2- 172.21.15.90:0/1757754158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e05023e40 0x559e051a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.795+0000 7fd975400700 1 -- 172.21.15.90:0/1757754158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e05023e40 msgr2=0x559e051a7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.795+0000 7fd975400700 1 --2- 172.21.15.90:0/1757754158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e05023e40 0x559e051a7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.795+0000 7fd975400700 1 -- 172.21.15.90:0/1757754158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e051a2e70 msgr2=0x559e051ddf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.795+0000 7fd975400700 1 --2- 172.21.15.90:0/1757754158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e051a2e70 0x559e051ddf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.795+0000 7fd975400700 1 -- 172.21.15.90:0/1757754158 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0519a240 con 0x559e0510f9f0 2024-05-27T07:58:39.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.796+0000 7fd975400700 1 --2- 172.21.15.90:0/1757754158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a65f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd9640a8b90 tx=0x7fd9640a4ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bf57a30327d8872 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.796+0000 7fd962a00700 1 -- 172.21.15.90:0/1757754158 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640c8070 con 0x559e0510f9f0 2024-05-27T07:58:39.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.796+0000 7fd962a00700 1 -- 172.21.15.90:0/1757754158 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9640a8c00 con 0x559e0510f9f0 2024-05-27T07:58:39.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.796+0000 7fd962a00700 1 -- 172.21.15.90:0/1757754158 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964020040 con 0x559e0510f9f0 2024-05-27T07:58:39.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.797+0000 7fd98bac7b80 1 -- 172.21.15.90:0/1757754158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 msgr2=0x559e051a65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.798+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/1757754158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a65f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd9640a8b90 tx=0x7fd9640a4ac0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.798+0000 7fd98bac7b80 1 -- 172.21.15.90:0/1757754158 shutdown_connections 2024-05-27T07:58:39.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.798+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/1757754158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e051a2e70 0x559e051ddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.798+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/1757754158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e05023e40 0x559e051a7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.798+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/1757754158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e0510f9f0 0x559e051a65f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.798+0000 7fd98bac7b80 1 -- 172.21.15.90:0/1757754158 >> 172.21.15.90:0/1757754158 conn(0x559e050bd390 msgr2=0x559e051a5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.798+0000 7fd98bac7b80 1 -- 172.21.15.90:0/1757754158 shutdown_connections 2024-05-27T07:58:39.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.798+0000 7fd98bac7b80 1 -- 172.21.15.90:0/1757754158 wait complete. 2024-05-27T07:58:39.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.799+0000 7fd98bac7b80 1 -- start start 2024-05-27T07:58:39.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.799+0000 7fd98bac7b80 1 Processor -- start 2024-05-27T07:58:39.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.800+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e05023e40 0x559e051b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.800+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e0510f9f0 0x559e051b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.801+0000 7fd98bac7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e0523ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.801+0000 7fd975e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e05023e40 0x559e051b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.801+0000 7fd975400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e0510f9f0 0x559e051b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.801+0000 7fd975e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e05023e40 0x559e051b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44256/0 (socket says 172.21.15.90:44256) 2024-05-27T07:58:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.801+0000 7fd975e00700 1 -- 172.21.15.90:0/2972200482 learned_addr learned my addr 172.21.15.90:0/2972200482 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.801+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd978071ed0 con 0x559e0510f9f0 2024-05-27T07:58:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.801+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd978070a60 con 0x559e051a2e70 2024-05-27T07:58:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.801+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd978071130 con 0x559e05023e40 2024-05-27T07:58:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.801+0000 7fd98bac7b80 10 client.0 _ll_register_callbacks cb 0x559e0506ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.802+0000 7fd976800700 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e0523ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.803+0000 7fd976800700 1 -- 172.21.15.90:0/2972200482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e05023e40 msgr2=0x559e051b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.803+0000 7fd976800700 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e05023e40 0x559e051b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.803+0000 7fd976800700 1 -- 172.21.15.90:0/2972200482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e0510f9f0 msgr2=0x559e051b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.803+0000 7fd976800700 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e0510f9f0 0x559e051b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.803+0000 7fd976800700 1 -- 172.21.15.90:0/2972200482 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e051b93a0 con 0x559e051a2e70 2024-05-27T07:58:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.803+0000 7fd975400700 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e0510f9f0 0x559e051b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:39.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.804+0000 7fd976800700 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e0523ee10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd970009c00 tx=0x7fd970005400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91699b1292e3ab8c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:39.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.804+0000 7fd957e00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:58:39.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.804+0000 7fd957e00700 1 -- 172.21.15.90:0/2972200482 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970032060 con 0x559e051a2e70 2024-05-27T07:58:39.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.804+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9640a8c00 con 0x559e051a2e70 2024-05-27T07:58:39.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.806+0000 7fd957e00700 1 -- 172.21.15.90:0/2972200482 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7fd9700057f0 con 0x559e051a2e70 2024-05-27T07:58:39.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.806+0000 7fd957e00700 1 -- 172.21.15.90:0/2972200482 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd970039050 con 0x559e051a2e70 2024-05-27T07:58:39.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.806+0000 7fd98bac7b80 10 client.37196 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:39.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.806+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x559e05240340 con 0x559e051a2e70 2024-05-27T07:58:39.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.807+0000 7fd957e00700 1 -- 172.21.15.90:0/2972200482 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd9640a8c00 con 0x559e051a2e70 2024-05-27T07:58:39.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.807+0000 7fd957e00700 1 -- 172.21.15.90:0/2972200482 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fd970011070 con 0x559e051a2e70 2024-05-27T07:58:39.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.807+0000 7fd98bac7b80 10 client.37196 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:39.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.807+0000 7fd98bac7b80 10 client.37196 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:58:39.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.807+0000 7fd98bac7b80 20 client.37196 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:39.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.807+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559e05240750 0x559e05242b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:39.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.807+0000 7fd98bac7b80 4 client.37196 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:58:39.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.807+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e050cc570 con 0x559e05240750 2024-05-27T07:58:39.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.808+0000 7fd975e00700 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559e05240750 0x559e05242b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:39.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.809+0000 7fd957e00700 10 client.37196 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:58:39.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.810+0000 7fd975e00700 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559e05240750 0x559e05242b70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db6370bccd72bf29 server_cookie=fa975d9e64acfe7f in_seq=0 out_seq=0 2024-05-27T07:58:39.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.810+0000 7fd957e00700 1 -- 172.21.15.90:0/2972200482 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e050cc570 con 0x559e05240750 2024-05-27T07:58:39.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.810+0000 7fd957e00700 10 client.37196 handle_command_reply: tid=0 2024-05-27T07:58:39.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.810+0000 7fd98bac7b80 1 client.37196 shutdown 2024-05-27T07:58:39.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.810+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 msgr2=0x559e0523ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.810+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e0523ee10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd970009c00 tx=0x7fd970005400 comp rx=0 tx=0).stop 2024-05-27T07:58:39.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.811+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 shutdown_connections 2024-05-27T07:58:39.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.811+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x559e05240750 0x559e05242b70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.811+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559e051a2e70 0x559e0523ee10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.811+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559e05023e40 0x559e051b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.811+0000 7fd98bac7b80 1 --2- 172.21.15.90:0/2972200482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e0510f9f0 0x559e051b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.811+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 >> 172.21.15.90:0/2972200482 conn(0x559e050bd390 msgr2=0x559e051adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.811+0000 7fd957e00700 0 client.37196 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:58:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.811+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 shutdown_connections 2024-05-27T07:58:39.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.811+0000 7fd98bac7b80 1 -- 172.21.15.90:0/2972200482 wait complete. 2024-05-27T07:58:39.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.812+0000 7fd98bac7b80 20 client.37196 trim_cache size 0 max 16384 2024-05-27T07:58:39.817 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:58:39.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.813+0000 7fd97d000700 1 -- 172.21.15.90:0/623596645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd95c0b2bc0 msgr2=0x7fd95c0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.813+0000 7fd97d000700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd95c0b2bc0 0x7fd95c0b50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd968005950 tx=0x7fd96800b410 comp rx=0 tx=0).stop 2024-05-27T07:58:39.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.813+0000 7fd97d000700 1 -- 172.21.15.90:0/623596645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd978108f10 msgr2=0x7fd97819d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:39.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.813+0000 7fd97d000700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd978108f10 0x7fd97819d490 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd96400d950 tx=0x7fd96400dc60 comp rx=0 tx=0).stop 2024-05-27T07:58:39.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.813+0000 7fd97d000700 1 -- 172.21.15.90:0/623596645 shutdown_connections 2024-05-27T07:58:39.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.813+0000 7fd97d000700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd978109860 0x7fd9781a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.813+0000 7fd97d000700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd95c0b2bc0 0x7fd95c0b50d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.813+0000 7fd97d000700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd9781045e0 0x7fd97819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.813+0000 7fd97d000700 1 --2- 172.21.15.90:0/623596645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd978108f10 0x7fd97819d490 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:39.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.813+0000 7fd97d000700 1 -- 172.21.15.90:0/623596645 >> 172.21.15.90:0/623596645 conn(0x7fd978100030 msgr2=0x7fd97810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:39.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.814+0000 7fd97d000700 1 -- 172.21.15.90:0/623596645 shutdown_connections 2024-05-27T07:58:39.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:39.814+0000 7fd97d000700 1 -- 172.21.15.90:0/623596645 wait complete. 2024-05-27T07:58:40.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:40 smithi090 ceph-mon[31429]: pgmap v391: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 568 op/s 2024-05-27T07:58:40.840 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:58:41.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:40 smithi136 ceph-mon[35324]: pgmap v391: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 568 op/s 2024-05-27T07:58:41.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:40 smithi129 ceph-mon[35254]: pgmap v391: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.0 MiB/s wr, 568 op/s 2024-05-27T07:58:41.917 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.210+0000 7fa80a400640 1 -- 172.21.15.90:0/2934363402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7fc0a5050 msgr2=0x7fa7fc0a7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.210+0000 7fa80a400640 1 --2- 172.21.15.90:0/2934363402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7fc0a5050 0x7fa7fc0a7470 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009860 tx=0x7fa7f002a3d0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.211+0000 7fa80a400640 1 -- 172.21.15.90:0/2934363402 shutdown_connections 2024-05-27T07:58:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.211+0000 7fa80a400640 1 --2- 172.21.15.90:0/2934363402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc0a79b0 0x7fa7fc0a9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.211+0000 7fa80a400640 1 --2- 172.21.15.90:0/2934363402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7fc0a5050 0x7fa7fc0a7470 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.211+0000 7fa80a400640 1 --2- 172.21.15.90:0/2934363402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7fc0075f0 0x7fa7fc007a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.211+0000 7fa80a400640 1 -- 172.21.15.90:0/2934363402 >> 172.21.15.90:0/2934363402 conn(0x7fa7fc00d3c0 msgr2=0x7fa7fc00d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.211+0000 7fa80a400640 1 -- 172.21.15.90:0/2934363402 shutdown_connections 2024-05-27T07:58:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.211+0000 7fa80a400640 1 -- 172.21.15.90:0/2934363402 wait complete. 2024-05-27T07:58:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.213+0000 7fa80a400640 1 Processor -- start 2024-05-27T07:58:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.214+0000 7fa80a400640 1 -- start start 2024-05-27T07:58:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.214+0000 7fa80a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7fc0075f0 0x7fa7fc139780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.214+0000 7fa80a400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc0a5050 0x7fa7fc139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.214+0000 7fa80a400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7fc0a79b0 0x7fa7fc133990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.214+0000 7fa80a400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0ad090 con 0x7fa7fc0a79b0 2024-05-27T07:58:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.214+0000 7fa80a400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0acf10 con 0x7fa7fc0a5050 2024-05-27T07:58:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.214+0000 7fa80a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0ad210 con 0x7fa7fc0075f0 2024-05-27T07:58:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.215+0000 7fa809a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7fc0a79b0 0x7fa7fc133990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.215+0000 7fa809000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7fc0075f0 0x7fa7fc139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.215+0000 7fa809a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7fc0a79b0 0x7fa7fc133990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35868/0 (socket says 172.21.15.90:35868) 2024-05-27T07:58:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.215+0000 7fa809a00640 1 -- 172.21.15.90:0/3208051232 learned_addr learned my addr 172.21.15.90:0/3208051232 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.216+0000 7fa809a00640 1 -- 172.21.15.90:0/3208051232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7fc0075f0 msgr2=0x7fa7fc139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.216+0000 7fa809a00640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7fc0075f0 0x7fa7fc139780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.216+0000 7fa809a00640 1 -- 172.21.15.90:0/3208051232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc0a5050 msgr2=0x7fa7fc139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.217+0000 7fa803e00640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc0a5050 0x7fa7fc139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.217+0000 7fa809a00640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc0a5050 0x7fa7fc139cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.218+0000 7fa809a00640 1 -- 172.21.15.90:0/3208051232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc134030 con 0x7fa7fc0a79b0 2024-05-27T07:58:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.218+0000 7fa809000640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7fc0075f0 0x7fa7fc139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:58:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.220+0000 7fa809a00640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7fc0a79b0 0x7fa7fc133990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa804052210 tx=0x7fa804069e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.220+0000 7fa801600640 1 -- 172.21.15.90:0/3208051232 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804082070 con 0x7fa7fc0a79b0 2024-05-27T07:58:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.221+0000 7fa80a400640 1 -- 172.21.15.90:0/3208051232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc134320 con 0x7fa7fc0a79b0 2024-05-27T07:58:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.221+0000 7fa80a400640 1 -- 172.21.15.90:0/3208051232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc140600 con 0x7fa7fc0a79b0 2024-05-27T07:58:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.222+0000 7fa801600640 1 -- 172.21.15.90:0/3208051232 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa80407bdb0 con 0x7fa7fc0a79b0 2024-05-27T07:58:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.222+0000 7fa801600640 1 -- 172.21.15.90:0/3208051232 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80408c060 con 0x7fa7fc0a79b0 2024-05-27T07:58:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.228+0000 7fa8129b7740 1 Processor -- start 2024-05-27T07:58:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.230+0000 7fa801600640 1 -- 172.21.15.90:0/3208051232 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa804085070 con 0x7fa7fc0a79b0 2024-05-27T07:58:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.231+0000 7fa8129b7740 1 -- start start 2024-05-27T07:58:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.231+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7458011b0 0x55a745ba3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.231+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7457d82f0 0x55a745ba3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.231+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 0x55a745ba4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.231+0000 7fa8129b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a745b14b90 con 0x55a7457e04d0 2024-05-27T07:58:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.231+0000 7fa8129b7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a745b04730 con 0x55a7457d82f0 2024-05-27T07:58:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.231+0000 7fa8129b7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a745b08770 con 0x55a7458011b0 2024-05-27T07:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.232+0000 7fa809000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7457d82f0 0x55a745ba3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.232+0000 7fa803e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 0x55a745ba4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.232+0000 7fa809000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7457d82f0 0x55a745ba3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49842/0 (socket says 172.21.15.90:49842) 2024-05-27T07:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.232+0000 7fa809000640 1 -- 172.21.15.90:0/2623371052 learned_addr learned my addr 172.21.15.90:0/2623371052 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.232+0000 7fa801600640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa7ec0b2a90 0x7fa7ec0b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.232+0000 7fa803e00640 1 --2- 172.21.15.90:0/2623371052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7458011b0 0x55a745ba3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.232+0000 7fa801600640 1 -- 172.21.15.90:0/3208051232 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7fa80414f980 con 0x7fa7fc0a79b0 2024-05-27T07:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.233+0000 7fa809000640 1 -- 172.21.15.90:0/2623371052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7458011b0 msgr2=0x55a745ba3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.233+0000 7fa809000640 1 --2- 172.21.15.90:0/2623371052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7458011b0 0x55a745ba3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.233+0000 7fa809000640 1 -- 172.21.15.90:0/2623371052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 msgr2=0x55a745ba4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.233+0000 7fa809000640 1 --2- 172.21.15.90:0/2623371052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 0x55a745ba4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.233+0000 7fa809000640 1 -- 172.21.15.90:0/2623371052 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a745a9d8b0 con 0x55a7457d82f0 2024-05-27T07:58:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.233+0000 7fa809a00640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa7ec0b2a90 0x7fa7ec0b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.233+0000 7fa809000640 1 --2- 172.21.15.90:0/2623371052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7457d82f0 0x55a745ba3fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80114f0 tx=0x7fa7f80119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab101a28300cc438 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.234+0000 7fa7f6000640 1 -- 172.21.15.90:0/2623371052 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8005020 con 0x55a7457d82f0 2024-05-27T07:58:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.234+0000 7fa7f6000640 1 -- 172.21.15.90:0/2623371052 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7f80047a0 con 0x55a7457d82f0 2024-05-27T07:58:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.234+0000 7fa7f6000640 1 -- 172.21.15.90:0/2623371052 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800f8e0 con 0x55a7457d82f0 2024-05-27T07:58:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.235+0000 7fa809a00640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa7ec0b2a90 0x7fa7ec0b4fa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa804116640 tx=0x7fa804069100 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.235+0000 7fa8129b7740 1 -- 172.21.15.90:0/2623371052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7457d82f0 msgr2=0x55a745ba3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.235+0000 7fa8129b7740 1 --2- 172.21.15.90:0/2623371052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7457d82f0 0x55a745ba3fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80114f0 tx=0x7fa7f80119b0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.235+0000 7fa8129b7740 1 -- 172.21.15.90:0/2623371052 shutdown_connections 2024-05-27T07:58:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.235+0000 7fa8129b7740 1 --2- 172.21.15.90:0/2623371052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7457d82f0 0x55a745ba3fd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.235+0000 7fa8129b7740 1 --2- 172.21.15.90:0/2623371052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 0x55a745ba4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.235+0000 7fa8129b7740 1 --2- 172.21.15.90:0/2623371052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7458011b0 0x55a745ba3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.235+0000 7fa8129b7740 1 -- 172.21.15.90:0/2623371052 >> 172.21.15.90:0/2623371052 conn(0x55a7457f5300 msgr2=0x55a745ba0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.235+0000 7fa8129b7740 1 -- 172.21.15.90:0/2623371052 shutdown_connections 2024-05-27T07:58:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.235+0000 7fa8129b7740 1 -- 172.21.15.90:0/2623371052 wait complete. 2024-05-27T07:58:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.236+0000 7fa8129b7740 1 -- start start 2024-05-27T07:58:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.237+0000 7fa8129b7740 1 Processor -- start 2024-05-27T07:58:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.237+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745badae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.237+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 0x55a745bae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.237+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bae560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.237+0000 7fa8129b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a7457ea790 con 0x55a7457d82f0 2024-05-27T07:58:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.237+0000 7fa8129b7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a745aacbc0 con 0x55a7458011b0 2024-05-27T07:58:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.237+0000 7fa8129b7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a745bb0c60 con 0x55a7457e04d0 2024-05-27T07:58:42.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.237+0000 7fa8129b7740 10 client.0 _ll_register_callbacks cb 0x55a745a8a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.238+0000 7fa809000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 0x55a745bae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.238+0000 7fa809000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 0x55a745bae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44270/0 (socket says 172.21.15.90:44270) 2024-05-27T07:58:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.238+0000 7fa809000640 1 -- 172.21.15.90:0/3113746784 learned_addr learned my addr 172.21.15.90:0/3113746784 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.238+0000 7fa803e00640 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745badae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.239+0000 7fa809000640 1 -- 172.21.15.90:0/3113746784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 msgr2=0x55a745bae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.239+0000 7fa809000640 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.239+0000 7fa809000640 1 -- 172.21.15.90:0/3113746784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 msgr2=0x55a745badae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.239+0000 7fa809000640 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745badae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.239+0000 7fa809000640 1 -- 172.21.15.90:0/3113746784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a745819210 con 0x55a7457e04d0 2024-05-27T07:58:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.240+0000 7fa803e00640 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745badae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:58:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.241+0000 7fa809000640 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 0x55a745bae020 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009690 tx=0x7fa7f8005150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed5a9936789af2db server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.241+0000 7fa7eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:58:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.241+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3113746784 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8004030 con 0x55a7457e04d0 2024-05-27T07:58:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.241+0000 7fa8129b7740 1 -- 172.21.15.90:0/3113746784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f80047a0 con 0x55a7457e04d0 2024-05-27T07:58:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.243+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3113746784 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7fa7f80041d0 con 0x55a7457e04d0 2024-05-27T07:58:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.244+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3113746784 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa7f8033050 con 0x55a7457e04d0 2024-05-27T07:58:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.244+0000 7fa8129b7740 10 client.27301 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.244+0000 7fa8129b7740 1 -- 172.21.15.90:0/3113746784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a7458e5620 con 0x55a7457e04d0 2024-05-27T07:58:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.245+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3113746784 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7f80047a0 con 0x55a7457e04d0 2024-05-27T07:58:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.245+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3113746784 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fa7f800fce0 con 0x55a7457e04d0 2024-05-27T07:58:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.245+0000 7fa8129b7740 10 client.27301 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.245+0000 7fa8129b7740 10 client.27301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.245+0000 7fa8129b7740 20 client.27301 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.245+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a7457bfd80 0x55a745bc0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.245+0000 7fa8129b7740 4 client.27301 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.245+0000 7fa8129b7740 1 -- 172.21.15.90:0/3113746784 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a745b16ae0 con 0x55a7457bfd80 2024-05-27T07:58:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.247+0000 7fa803e00640 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a7457bfd80 0x55a745bc0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.248+0000 7fa803e00640 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a7457bfd80 0x55a745bc0000 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4b46ca66fcc8d2 server_cookie=446dd9527df75623 in_seq=0 out_seq=0 2024-05-27T07:58:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.248+0000 7fa7eaa00640 10 client.27301 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.252+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3113746784 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a745b16ae0 con 0x55a7457bfd80 2024-05-27T07:58:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.252+0000 7fa7eaa00640 10 client.27301 handle_command_reply: tid=0 2024-05-27T07:58:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.252+0000 7fa8129b7740 1 client.27301 shutdown 2024-05-27T07:58:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.254+0000 7fa8129b7740 1 -- 172.21.15.90:0/3113746784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 msgr2=0x55a745bae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.254+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 0x55a745bae020 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009690 tx=0x7fa7f8005150 comp rx=0 tx=0).stop 2024-05-27T07:58:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.254+0000 7fa8129b7740 1 -- 172.21.15.90:0/3113746784 shutdown_connections 2024-05-27T07:58:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.254+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a7457bfd80 0x55a745bc0000 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.254+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.254+0000 7fa7eaa00640 0 client.27301 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.254+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 0x55a745bae020 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.254+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3113746784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745badae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.254+0000 7fa8129b7740 1 -- 172.21.15.90:0/3113746784 >> 172.21.15.90:0/3113746784 conn(0x55a7457f5300 msgr2=0x55a745ba3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.255+0000 7fa8129b7740 1 -- 172.21.15.90:0/3113746784 shutdown_connections 2024-05-27T07:58:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.255+0000 7fa8129b7740 1 -- 172.21.15.90:0/3113746784 wait complete. 2024-05-27T07:58:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.255+0000 7fa8129b7740 20 client.27301 trim_cache size 0 max 16384 2024-05-27T07:58:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.280+0000 7fa8129b7740 1 Processor -- start 2024-05-27T07:58:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.280+0000 7fa8129b7740 1 -- start start 2024-05-27T07:58:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457d82f0 0x55a745bb0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 0x55a745bb9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bc57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa8129b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e4011990 con 0x55a7457e04d0 2024-05-27T07:58:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa8129b7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e400e9e0 con 0x55a7458011b0 2024-05-27T07:58:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa8129b7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e4010bf0 con 0x55a7457d82f0 2024-05-27T07:58:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa809000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 0x55a745bb9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa809000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 0x55a745bb9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35902/0 (socket says 172.21.15.90:35902) 2024-05-27T07:58:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa809000640 1 -- 172.21.15.90:0/1436304240 learned_addr learned my addr 172.21.15.90:0/1436304240 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa803e00640 1 --2- 172.21.15.90:0/1436304240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bc57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa803e00640 1 --2- 172.21.15.90:0/1436304240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457d82f0 0x55a745bb0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa809000640 1 -- 172.21.15.90:0/1436304240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457d82f0 msgr2=0x55a745bb0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa809000640 1 --2- 172.21.15.90:0/1436304240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457d82f0 0x55a745bb0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.281+0000 7fa809000640 1 -- 172.21.15.90:0/1436304240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 msgr2=0x55a745bc57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.282+0000 7fa809000640 1 --2- 172.21.15.90:0/1436304240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bc57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.282+0000 7fa809000640 1 -- 172.21.15.90:0/1436304240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a745bac030 con 0x55a7457e04d0 2024-05-27T07:58:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.282+0000 7fa803e00640 1 --2- 172.21.15.90:0/1436304240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bc57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.282+0000 7fa809000640 1 --2- 172.21.15.90:0/1436304240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 0x55a745bb9de0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fa7f802ef10 tx=0x7fa7f80059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a55652aaf8e723 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.282+0000 7fa7f5600640 1 -- 172.21.15.90:0/1436304240 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8005750 con 0x55a7457e04d0 2024-05-27T07:58:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.282+0000 7fa7f5600640 1 -- 172.21.15.90:0/1436304240 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7f80037b0 con 0x55a7457e04d0 2024-05-27T07:58:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.282+0000 7fa7f5600640 1 -- 172.21.15.90:0/1436304240 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8021d70 con 0x55a7457e04d0 2024-05-27T07:58:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.283+0000 7fa8129b7740 1 -- 172.21.15.90:0/1436304240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 msgr2=0x55a745bb9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.283+0000 7fa8129b7740 1 --2- 172.21.15.90:0/1436304240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 0x55a745bb9de0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fa7f802ef10 tx=0x7fa7f80059b0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.283+0000 7fa8129b7740 1 -- 172.21.15.90:0/1436304240 shutdown_connections 2024-05-27T07:58:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.283+0000 7fa8129b7740 1 --2- 172.21.15.90:0/1436304240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bc57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.283+0000 7fa8129b7740 1 --2- 172.21.15.90:0/1436304240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457e04d0 0x55a745bb9de0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.283+0000 7fa8129b7740 1 --2- 172.21.15.90:0/1436304240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457d82f0 0x55a745bb0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.283+0000 7fa8129b7740 1 -- 172.21.15.90:0/1436304240 >> 172.21.15.90:0/1436304240 conn(0x55a7457f5300 msgr2=0x55a745b9f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.283+0000 7fa8129b7740 1 -- 172.21.15.90:0/1436304240 shutdown_connections 2024-05-27T07:58:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.283+0000 7fa8129b7740 1 -- 172.21.15.90:0/1436304240 wait complete. 2024-05-27T07:58:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.284+0000 7fa8129b7740 1 -- start start 2024-05-27T07:58:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.284+0000 7fa8129b7740 1 Processor -- start 2024-05-27T07:58:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.284+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745bb6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.284+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 0x55a745bb2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.284+0000 7fa8129b7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bb2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.284+0000 7fa8129b7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e4010520 con 0x55a7457d82f0 2024-05-27T07:58:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.284+0000 7fa8129b7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e400f0b0 con 0x55a7458011b0 2024-05-27T07:58:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.284+0000 7fa8129b7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e400f780 con 0x55a7457e04d0 2024-05-27T07:58:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.284+0000 7fa8129b7740 10 client.0 _ll_register_callbacks cb 0x7fa7f8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa803e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745bb6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa803e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745bb6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35904/0 (socket says 172.21.15.90:35904) 2024-05-27T07:58:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa803e00640 1 -- 172.21.15.90:0/3701475386 learned_addr learned my addr 172.21.15.90:0/3701475386 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa809000640 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 0x55a745bb2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa803e00640 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bb2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa803e00640 1 -- 172.21.15.90:0/3701475386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 msgr2=0x55a745bb2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa803e00640 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 0x55a745bb2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa803e00640 1 -- 172.21.15.90:0/3701475386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 msgr2=0x55a745bb2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa803e00640 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bb2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa803e00640 1 -- 172.21.15.90:0/3701475386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a745bb3520 con 0x55a7457d82f0 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa803e00640 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745bb6610 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009250 tx=0x7fa7f8004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a008c1f0bdd1cc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.285+0000 7fa7eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.286+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3701475386 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f003da70 con 0x55a7457d82f0 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.286+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3701475386 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7fa7f0048040 con 0x55a7457d82f0 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.286+0000 7fa8129b7740 1 -- 172.21.15.90:0/3701475386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f80037b0 con 0x55a7457d82f0 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.286+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3701475386 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa7f005f050 con 0x55a7457d82f0 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.286+0000 7fa8129b7740 10 client.17613 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.286+0000 7fa8129b7740 1 -- 172.21.15.90:0/3701475386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a745c27530 con 0x55a7457d82f0 2024-05-27T07:58:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.286+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3701475386 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7f80037b0 con 0x55a7457d82f0 2024-05-27T07:58:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.286+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3701475386 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fa7f004c350 con 0x55a7457d82f0 2024-05-27T07:58:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.287+0000 7fa8129b7740 10 client.17613 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.287+0000 7fa8129b7740 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.287+0000 7fa8129b7740 20 client.17613 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.287+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a7457951c0 0x55a745c29c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.287+0000 7fa8129b7740 4 client.17613 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:58:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.287+0000 7fa8129b7740 1 -- 172.21.15.90:0/3701475386 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a745b102c0 con 0x55a7457951c0 2024-05-27T07:58:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.287+0000 7fa809000640 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a7457951c0 0x55a745c29c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.288+0000 7fa809000640 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a7457951c0 0x55a745c29c70 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d2596f0a946aad server_cookie=e3e1313e1b5100b in_seq=0 out_seq=0 2024-05-27T07:58:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.288+0000 7fa7eaa00640 10 client.17613 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa7eaa00640 1 -- 172.21.15.90:0/3701475386 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a745b102c0 con 0x55a7457951c0 2024-05-27T07:58:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa7eaa00640 10 client.17613 handle_command_reply: tid=0 2024-05-27T07:58:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 client.17613 shutdown 2024-05-27T07:58:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 -- 172.21.15.90:0/3701475386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 msgr2=0x55a745bb6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745bb6610 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0009250 tx=0x7fa7f8004c60 comp rx=0 tx=0).stop 2024-05-27T07:58:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 -- 172.21.15.90:0/3701475386 shutdown_connections 2024-05-27T07:58:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a7457951c0 0x55a745c29c70 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a7458011b0 0x55a745bb2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a7457e04d0 0x55a745bb2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 --2- 172.21.15.90:0/3701475386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a7457d82f0 0x55a745bb6610 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa7eaa00640 0 client.17613 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 -- 172.21.15.90:0/3701475386 >> 172.21.15.90:0/3701475386 conn(0x55a7457f5300 msgr2=0x55a745b9e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 -- 172.21.15.90:0/3701475386 shutdown_connections 2024-05-27T07:58:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.583+0000 7fa8129b7740 1 -- 172.21.15.90:0/3701475386 wait complete. 2024-05-27T07:58:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.584+0000 7fa8129b7740 20 client.17613 trim_cache size 0 max 16384 2024-05-27T07:58:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:58:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:58:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.584+0000 7fa80a400640 1 -- 172.21.15.90:0/3208051232 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa7ec0b2a90 msgr2=0x7fa7ec0b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.584+0000 7fa80a400640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa7ec0b2a90 0x7fa7ec0b4fa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa804116640 tx=0x7fa804069100 comp rx=0 tx=0).stop 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.584+0000 7fa80a400640 1 -- 172.21.15.90:0/3208051232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7fc0a79b0 msgr2=0x7fa7fc133990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.584+0000 7fa80a400640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7fc0a79b0 0x7fa7fc133990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa804052210 tx=0x7fa804069e80 comp rx=0 tx=0).stop 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.584+0000 7fa80a400640 1 -- 172.21.15.90:0/3208051232 shutdown_connections 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.584+0000 7fa80a400640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa7ec0b2a90 0x7fa7ec0b4fa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.584+0000 7fa80a400640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc0a5050 0x7fa7fc139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.584+0000 7fa80a400640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7fc0a79b0 0x7fa7fc133990 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.585+0000 7fa80a400640 1 --2- 172.21.15.90:0/3208051232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7fc0075f0 0x7fa7fc139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.585+0000 7fa80a400640 1 -- 172.21.15.90:0/3208051232 >> 172.21.15.90:0/3208051232 conn(0x7fa7fc00d3c0 msgr2=0x7fa7fc0a5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.585+0000 7fa80a400640 1 -- 172.21.15.90:0/3208051232 shutdown_connections 2024-05-27T07:58:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:42.585+0000 7fa80a400640 1 -- 172.21.15.90:0/3208051232 wait complete. 2024-05-27T07:58:42.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:42 smithi090 ceph-mon[31429]: pgmap v392: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.9 MiB/s wr, 503 op/s 2024-05-27T07:58:43.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:42 smithi136 ceph-mon[35324]: pgmap v392: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.9 MiB/s wr, 503 op/s 2024-05-27T07:58:43.068 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:42 smithi129 ceph-mon[35254]: pgmap v392: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.9 MiB/s wr, 503 op/s 2024-05-27T07:58:43.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 505.0 s, 591.6 tps, lat 168.134 ms stddev 137.291 2024-05-27T07:58:44.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:44 smithi090 ceph-mon[31429]: pgmap v393: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.1 MiB/s wr, 532 op/s 2024-05-27T07:58:45.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:44 smithi136 ceph-mon[35324]: pgmap v393: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.1 MiB/s wr, 532 op/s 2024-05-27T07:58:45.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:44 smithi129 ceph-mon[35254]: pgmap v393: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.1 MiB/s wr, 532 op/s 2024-05-27T07:58:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.060+0000 7f87d3600700 1 -- 172.21.15.90:0/2308999295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87cc108f10 msgr2=0x7f87cc109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.060+0000 7f87d3600700 1 --2- 172.21.15.90:0/2308999295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87cc108f10 0x7f87cc109320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009990 tx=0x7f87b8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.061+0000 7f87d3600700 1 -- 172.21.15.90:0/2308999295 shutdown_connections 2024-05-27T07:58:46.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.061+0000 7f87d3600700 1 --2- 172.21.15.90:0/2308999295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc109860 0x7f87cc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.061+0000 7f87d3600700 1 --2- 172.21.15.90:0/2308999295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc1045e0 0x7f87cc1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.061+0000 7f87d3600700 1 --2- 172.21.15.90:0/2308999295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87cc108f10 0x7f87cc109320 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.061+0000 7f87d3600700 1 -- 172.21.15.90:0/2308999295 >> 172.21.15.90:0/2308999295 conn(0x7f87cc100030 msgr2=0x7f87cc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.061+0000 7f87d3600700 1 -- 172.21.15.90:0/2308999295 shutdown_connections 2024-05-27T07:58:46.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.061+0000 7f87d3600700 1 -- 172.21.15.90:0/2308999295 wait complete. 2024-05-27T07:58:46.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.063+0000 7f87d3600700 1 Processor -- start 2024-05-27T07:58:46.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.064+0000 7f87d3600700 1 -- start start 2024-05-27T07:58:46.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.064+0000 7f87d3600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc1045e0 0x7f87cc10d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.064+0000 7f87d3600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87cc108f10 0x7f87cc10db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.064+0000 7f87d3600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc109860 0x7f87cc10e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.064+0000 7f87d3600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0762e0 con 0x7f87cc1045e0 2024-05-27T07:58:46.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.064+0000 7f87d3600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc076160 con 0x7f87cc108f10 2024-05-27T07:58:46.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.064+0000 7f87d3600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc076460 con 0x7f87cc109860 2024-05-27T07:58:46.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.065+0000 7f87cbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87cc108f10 0x7f87cc10db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.065+0000 7f87d0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc1045e0 0x7f87cc10d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.065+0000 7f87cbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87cc108f10 0x7f87cc10db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49898/0 (socket says 172.21.15.90:49898) 2024-05-27T07:58:46.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.065+0000 7f87cbe00700 1 -- 172.21.15.90:0/2007727638 learned_addr learned my addr 172.21.15.90:0/2007727638 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:46.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.065+0000 7f87d1600700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc109860 0x7f87cc10e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.066+0000 7f87d0c00700 1 -- 172.21.15.90:0/2007727638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc109860 msgr2=0x7f87cc10e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.066+0000 7f87d0c00700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc109860 0x7f87cc10e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.066+0000 7f87d0c00700 1 -- 172.21.15.90:0/2007727638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87cc108f10 msgr2=0x7f87cc10db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.066+0000 7f87d0c00700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87cc108f10 0x7f87cc10db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.066+0000 7f87d0c00700 1 -- 172.21.15.90:0/2007727638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc1ad690 con 0x7f87cc1045e0 2024-05-27T07:58:46.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.066+0000 7f87d1600700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc109860 0x7f87cc10e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:46.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.066+0000 7f87d0c00700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc1045e0 0x7f87cc10d5d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f87c0006fd0 tx=0x7f87c000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.067+0000 7f87c9600700 1 -- 172.21.15.90:0/2007727638 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0015020 con 0x7f87cc1045e0 2024-05-27T07:58:46.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.067+0000 7f87c9600700 1 -- 172.21.15.90:0/2007727638 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87c0011cc0 con 0x7f87cc1045e0 2024-05-27T07:58:46.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.067+0000 7f87c9600700 1 -- 172.21.15.90:0/2007727638 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0010cd0 con 0x7f87cc1045e0 2024-05-27T07:58:46.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.068+0000 7f87d3600700 1 -- 172.21.15.90:0/2007727638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87cc1ad980 con 0x7f87cc1045e0 2024-05-27T07:58:46.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.068+0000 7f87d3600700 1 -- 172.21.15.90:0/2007727638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87cc1ade70 con 0x7f87cc1045e0 2024-05-27T07:58:46.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.074+0000 7f87c9600700 1 -- 172.21.15.90:0/2007727638 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f87c0018020 con 0x7f87cc1045e0 2024-05-27T07:58:46.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.077+0000 7f87c9600700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87b40b2a70 0x7f87b40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.077+0000 7f87c9600700 1 -- 172.21.15.90:0/2007727638 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f87c00e20a0 con 0x7f87cc1045e0 2024-05-27T07:58:46.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.080+0000 7f87cbe00700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87b40b2a70 0x7f87b40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.082+0000 7f87cbe00700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87b40b2a70 0x7f87b40b4f80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009750 tx=0x7f87b801a040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.098+0000 7f87e21a4b80 1 Processor -- start 2024-05-27T07:58:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.099+0000 7f87e21a4b80 1 -- start start 2024-05-27T07:58:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.099+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 0x5585ba2837f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.100+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 0x5585ba27d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.100+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 0x5585ba27e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.100+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5585ba1abe40 con 0x5585ba283d30 2024-05-27T07:58:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.100+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5585ba19b9e0 con 0x5585ba0fee40 2024-05-27T07:58:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.100+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5585ba19fa20 con 0x5585ba27ddb0 2024-05-27T07:58:46.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.100+0000 7f87cbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 0x5585ba27d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.100+0000 7f87cbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 0x5585ba27d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35918/0 (socket says 172.21.15.90:35918) 2024-05-27T07:58:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.100+0000 7f87cbe00700 1 -- 172.21.15.90:0/1681549052 learned_addr learned my addr 172.21.15.90:0/1681549052 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.100+0000 7f87d0c00700 1 --2- 172.21.15.90:0/1681549052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 0x5585ba2837f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.101+0000 7f87d1600700 1 --2- 172.21.15.90:0/1681549052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 0x5585ba27e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.102+0000 7f87cbe00700 1 -- 172.21.15.90:0/1681549052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 msgr2=0x5585ba27e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.102+0000 7f87cbe00700 1 --2- 172.21.15.90:0/1681549052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 0x5585ba27e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.102+0000 7f87cbe00700 1 -- 172.21.15.90:0/1681549052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 msgr2=0x5585ba2837f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.102+0000 7f87cbe00700 1 --2- 172.21.15.90:0/1681549052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 0x5585ba2837f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.102+0000 7f87cbe00700 1 -- 172.21.15.90:0/1681549052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ba164420 con 0x5585ba283d30 2024-05-27T07:58:46.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.103+0000 7f87cbe00700 1 --2- 172.21.15.90:0/1681549052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 0x5585ba27d870 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f87b800bb80 tx=0x7f87b801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5374bfa5ddd2b430 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.104+0000 7f87be000700 1 -- 172.21.15.90:0/1681549052 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b800fa70 con 0x5585ba283d30 2024-05-27T07:58:46.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.104+0000 7f87be000700 1 -- 172.21.15.90:0/1681549052 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87b8031070 con 0x5585ba283d30 2024-05-27T07:58:46.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.104+0000 7f87be000700 1 -- 172.21.15.90:0/1681549052 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8022d60 con 0x5585ba283d30 2024-05-27T07:58:46.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.106+0000 7f87e21a4b80 1 -- 172.21.15.90:0/1681549052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 msgr2=0x5585ba27d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.106+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/1681549052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 0x5585ba27d870 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f87b800bb80 tx=0x7f87b801d000 comp rx=0 tx=0).stop 2024-05-27T07:58:46.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.106+0000 7f87e21a4b80 1 -- 172.21.15.90:0/1681549052 shutdown_connections 2024-05-27T07:58:46.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.106+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/1681549052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 0x5585ba27e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.106+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/1681549052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 0x5585ba2837f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.106+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/1681549052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 0x5585ba27d870 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.106+0000 7f87e21a4b80 1 -- 172.21.15.90:0/1681549052 >> 172.21.15.90:0/1681549052 conn(0x5585ba15efc0 msgr2=0x5585ba2788d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.107+0000 7f87e21a4b80 1 -- 172.21.15.90:0/1681549052 shutdown_connections 2024-05-27T07:58:46.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.108+0000 7f87e21a4b80 1 -- 172.21.15.90:0/1681549052 wait complete. 2024-05-27T07:58:46.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.109+0000 7f87e21a4b80 1 -- start start 2024-05-27T07:58:46.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.109+0000 7f87e21a4b80 1 Processor -- start 2024-05-27T07:58:46.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.111+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba0fee40 0x5585ba287440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.112+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 0x5585ba287980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.113+0000 7f87d0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba0fee40 0x5585ba287440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.113+0000 7f87d0c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba0fee40 0x5585ba287440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35922/0 (socket says 172.21.15.90:35922) 2024-05-27T07:58:46.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.113+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba283d30 0x5585ba287ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.113+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5585ba0a1fa0 con 0x5585ba0fee40 2024-05-27T07:58:46.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.113+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5585ba13c760 con 0x5585ba283d30 2024-05-27T07:58:46.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.113+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5585ba29b7b0 con 0x5585ba27ddb0 2024-05-27T07:58:46.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.113+0000 7f87e21a4b80 10 client.0 _ll_register_callbacks cb 0x5585ba145c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:46.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.114+0000 7f87d0c00700 1 -- 172.21.15.90:0/3532048330 learned_addr learned my addr 172.21.15.90:0/3532048330 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.115+0000 7f87d1600700 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba283d30 0x5585ba287ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.115+0000 7f87cbe00700 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 0x5585ba287980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.116+0000 7f87d0c00700 1 -- 172.21.15.90:0/3532048330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 msgr2=0x5585ba287980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.116+0000 7f87d0c00700 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 0x5585ba287980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.117+0000 7f87d0c00700 1 -- 172.21.15.90:0/3532048330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba283d30 msgr2=0x5585ba287ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.117+0000 7f87d0c00700 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba283d30 0x5585ba287ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.117+0000 7f87d0c00700 1 -- 172.21.15.90:0/3532048330 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ba0eb8d0 con 0x5585ba0fee40 2024-05-27T07:58:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.117+0000 7f87d1600700 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba283d30 0x5585ba287ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:58:46.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.117+0000 7f87cbe00700 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 0x5585ba287980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:58:46.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.118+0000 7f87d0c00700 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba0fee40 0x5585ba287440 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f87c00a9050 tx=0x7f87c00a97c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f405b39af492083f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.118+0000 7f87b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:58:46.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.118+0000 7f87b2a00700 1 -- 172.21.15.90:0/3532048330 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00ad470 con 0x5585ba0fee40 2024-05-27T07:58:46.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.119+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3532048330 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87b8031070 con 0x5585ba0fee40 2024-05-27T07:58:46.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.119+0000 7f87b2a00700 1 -- 172.21.15.90:0/3532048330 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f87c0010e70 con 0x5585ba0fee40 2024-05-27T07:58:46.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.120+0000 7f87b2a00700 1 -- 172.21.15.90:0/3532048330 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f87c00b1f20 con 0x5585ba0fee40 2024-05-27T07:58:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.120+0000 7f87e21a4b80 10 client.17622 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.120+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3532048330 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5585ba16c060 con 0x5585ba0fee40 2024-05-27T07:58:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.120+0000 7f87b2a00700 1 -- 172.21.15.90:0/3532048330 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87b8031070 con 0x5585ba0fee40 2024-05-27T07:58:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.121+0000 7f87b2a00700 1 -- 172.21.15.90:0/3532048330 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f87c0011830 con 0x5585ba0fee40 2024-05-27T07:58:46.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.121+0000 7f87e21a4b80 10 client.17622 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.121+0000 7f87e21a4b80 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.121+0000 7f87e21a4b80 20 client.17622 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.121+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5585ba2a8770 0x5585ba2aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.121+0000 7f87e21a4b80 4 client.17622 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.121+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3532048330 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585ba1add90 con 0x5585ba2a8770 2024-05-27T07:58:46.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.123+0000 7f87cbe00700 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5585ba2a8770 0x5585ba2aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.124+0000 7f87b2a00700 10 client.17622 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:46.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.124+0000 7f87cbe00700 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5585ba2a8770 0x5585ba2aabf0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea0cd488388c7b5 server_cookie=fbac5a32f1fe0008 in_seq=0 out_seq=0 2024-05-27T07:58:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.125+0000 7f87b2a00700 1 -- 172.21.15.90:0/3532048330 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5585ba1add90 con 0x5585ba2a8770 2024-05-27T07:58:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.125+0000 7f87b2a00700 10 client.17622 handle_command_reply: tid=0 2024-05-27T07:58:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.125+0000 7f87e21a4b80 1 client.17622 shutdown 2024-05-27T07:58:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.126+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3532048330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba0fee40 msgr2=0x5585ba287440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.126+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba0fee40 0x5585ba287440 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f87c00a9050 tx=0x7f87c00a97c0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.126+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3532048330 shutdown_connections 2024-05-27T07:58:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.126+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5585ba2a8770 0x5585ba2aabf0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.126+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba283d30 0x5585ba287ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.126+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba0fee40 0x5585ba287440 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.126+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3532048330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba27ddb0 0x5585ba287980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.126+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3532048330 >> 172.21.15.90:0/3532048330 conn(0x5585ba15efc0 msgr2=0x5585ba278580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.126+0000 7f87b2a00700 0 client.17622 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.127+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3532048330 shutdown_connections 2024-05-27T07:58:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.127+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3532048330 wait complete. 2024-05-27T07:58:46.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.127+0000 7f87e21a4b80 20 client.17622 trim_cache size 0 max 16384 2024-05-27T07:58:46.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.158+0000 7f87e21a4b80 1 Processor -- start 2024-05-27T07:58:46.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.159+0000 7f87e21a4b80 1 -- start start 2024-05-27T07:58:46.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.159+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 0x5585ba27d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.159+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba27ddb0 0x5585ba281b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.159+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba283d30 0x5585ba2897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.159+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1afe90 con 0x5585ba27ddb0 2024-05-27T07:58:46.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.159+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1a4230 con 0x5585ba0fee40 2024-05-27T07:58:46.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.159+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1af0f0 con 0x5585ba283d30 2024-05-27T07:58:46.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.160+0000 7f87d0c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 0x5585ba27d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.160+0000 7f87d0c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 0x5585ba27d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49924/0 (socket says 172.21.15.90:49924) 2024-05-27T07:58:46.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.160+0000 7f87d0c00700 1 -- 172.21.15.90:0/2429368970 learned_addr learned my addr 172.21.15.90:0/2429368970 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:46.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.160+0000 7f87cbe00700 1 --2- 172.21.15.90:0/2429368970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba27ddb0 0x5585ba281b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.161+0000 7f87d1600700 1 --2- 172.21.15.90:0/2429368970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba283d30 0x5585ba2897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.161+0000 7f87d0c00700 1 -- 172.21.15.90:0/2429368970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba283d30 msgr2=0x5585ba2897a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.161+0000 7f87d0c00700 1 --2- 172.21.15.90:0/2429368970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba283d30 0x5585ba2897a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.161+0000 7f87d0c00700 1 -- 172.21.15.90:0/2429368970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba27ddb0 msgr2=0x5585ba281b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.161+0000 7f87d0c00700 1 --2- 172.21.15.90:0/2429368970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba27ddb0 0x5585ba281b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.161+0000 7f87d0c00700 1 -- 172.21.15.90:0/2429368970 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ba279740 con 0x5585ba0fee40 2024-05-27T07:58:46.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.161+0000 7f87d1600700 1 --2- 172.21.15.90:0/2429368970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba283d30 0x5585ba2897a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:46.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.162+0000 7f87d0c00700 1 --2- 172.21.15.90:0/2429368970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 0x5585ba27d870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f87c00093d0 tx=0x7f87c000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=389c53095bae9a9d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.162+0000 7f87bd600700 1 -- 172.21.15.90:0/2429368970 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0087070 con 0x5585ba0fee40 2024-05-27T07:58:46.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.162+0000 7f87bd600700 1 -- 172.21.15.90:0/2429368970 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87c00ad470 con 0x5585ba0fee40 2024-05-27T07:58:46.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.162+0000 7f87bd600700 1 -- 172.21.15.90:0/2429368970 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00119b0 con 0x5585ba0fee40 2024-05-27T07:58:46.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.164+0000 7f87e21a4b80 1 -- 172.21.15.90:0/2429368970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 msgr2=0x5585ba27d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.164+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/2429368970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 0x5585ba27d870 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f87c00093d0 tx=0x7f87c000bf90 comp rx=0 tx=0).stop 2024-05-27T07:58:46.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.164+0000 7f87e21a4b80 1 -- 172.21.15.90:0/2429368970 shutdown_connections 2024-05-27T07:58:46.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.164+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/2429368970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba283d30 0x5585ba2897a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.164+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/2429368970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba0fee40 0x5585ba27d870 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.164+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/2429368970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba27ddb0 0x5585ba281b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.164+0000 7f87e21a4b80 1 -- 172.21.15.90:0/2429368970 >> 172.21.15.90:0/2429368970 conn(0x5585ba15efc0 msgr2=0x5585ba2863a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.165+0000 7f87e21a4b80 1 -- 172.21.15.90:0/2429368970 shutdown_connections 2024-05-27T07:58:46.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.166+0000 7f87e21a4b80 1 -- 172.21.15.90:0/2429368970 wait complete. 2024-05-27T07:58:46.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.166+0000 7f87e21a4b80 1 -- start start 2024-05-27T07:58:46.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.167+0000 7f87e21a4b80 1 Processor -- start 2024-05-27T07:58:46.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.168+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba0fee40 0x5585ba2a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.169+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba27ddb0 0x5585ba2af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.169+0000 7f87e21a4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 0x5585ba2afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.169+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1aea20 con 0x5585ba283d30 2024-05-27T07:58:46.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.169+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1a4900 con 0x5585ba27ddb0 2024-05-27T07:58:46.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.169+0000 7f87e21a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc1a4fd0 con 0x5585ba0fee40 2024-05-27T07:58:46.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.169+0000 7f87e21a4b80 10 client.0 _ll_register_callbacks cb 0x5585ba145c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:46.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.170+0000 7f87cbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba27ddb0 0x5585ba2af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.170+0000 7f87cbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba27ddb0 0x5585ba2af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:49936/0 (socket says 172.21.15.90:49936) 2024-05-27T07:58:46.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.170+0000 7f87cbe00700 1 -- 172.21.15.90:0/3165551179 learned_addr learned my addr 172.21.15.90:0/3165551179 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:46.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.170+0000 7f87d0c00700 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba0fee40 0x5585ba2a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.170+0000 7f87d1600700 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 0x5585ba2afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.171+0000 7f87cbe00700 1 -- 172.21.15.90:0/3165551179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba0fee40 msgr2=0x5585ba2a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.171+0000 7f87cbe00700 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba0fee40 0x5585ba2a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.171+0000 7f87cbe00700 1 -- 172.21.15.90:0/3165551179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 msgr2=0x5585ba2afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:46.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.171+0000 7f87cbe00700 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 0x5585ba2afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.171+0000 7f87cbe00700 1 -- 172.21.15.90:0/3165551179 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ba2b0430 con 0x5585ba27ddb0 2024-05-27T07:58:46.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.172+0000 7f87cbe00700 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba27ddb0 0x5585ba2af9b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87b8006010 tx=0x7f87b80186d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e7543a60f900f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:46.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.172+0000 7f87b2a00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:58:46.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.173+0000 7f87b2a00700 1 -- 172.21.15.90:0/3165551179 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b800fe60 con 0x5585ba27ddb0 2024-05-27T07:58:46.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.174+0000 7f87b2a00700 1 -- 172.21.15.90:0/3165551179 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f87b8018a00 con 0x5585ba27ddb0 2024-05-27T07:58:46.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.174+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3165551179 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c00ad470 con 0x5585ba27ddb0 2024-05-27T07:58:46.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.175+0000 7f87b2a00700 1 -- 172.21.15.90:0/3165551179 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f87b8018c00 con 0x5585ba27ddb0 2024-05-27T07:58:46.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.175+0000 7f87e21a4b80 10 client.37226 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:46.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.175+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3165551179 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5585ba312e40 con 0x5585ba27ddb0 2024-05-27T07:58:46.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.176+0000 7f87b2a00700 1 -- 172.21.15.90:0/3165551179 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87c00ad470 con 0x5585ba27ddb0 2024-05-27T07:58:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.176+0000 7f87b2a00700 1 -- 172.21.15.90:0/3165551179 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f87b8045030 con 0x5585ba27ddb0 2024-05-27T07:58:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.176+0000 7f87e21a4b80 10 client.37226 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.176+0000 7f87e21a4b80 10 client.37226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.176+0000 7f87e21a4b80 20 client.37226 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.176+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5585ba313380 0x5585ba3157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.176+0000 7f87e21a4b80 4 client.37226 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:58:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.176+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3165551179 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5585ba1a7570 con 0x5585ba313380 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.177+0000 7f87d0c00700 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5585ba313380 0x5585ba3157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.178+0000 7f87d0c00700 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5585ba313380 0x5585ba3157a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d7c0b733fb82c41 server_cookie=4e4457d118b2ac4b in_seq=0 out_seq=0 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.178+0000 7f87b2a00700 10 client.37226 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.178+0000 7f87b2a00700 1 -- 172.21.15.90:0/3165551179 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5585ba1a7570 con 0x5585ba313380 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.178+0000 7f87b2a00700 10 client.37226 handle_command_reply: tid=0 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.179+0000 7f87e21a4b80 1 client.37226 shutdown 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.179+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3165551179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba27ddb0 msgr2=0x5585ba2af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.179+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba27ddb0 0x5585ba2af9b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87b8006010 tx=0x7f87b80186d0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.179+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3165551179 shutdown_connections 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.179+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5585ba313380 0x5585ba3157a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.179+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5585ba283d30 0x5585ba2afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.179+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585ba0fee40 0x5585ba2a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.179+0000 7f87e21a4b80 1 --2- 172.21.15.90:0/3165551179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585ba27ddb0 0x5585ba2af9b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.179+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3165551179 >> 172.21.15.90:0/3165551179 conn(0x5585ba15efc0 msgr2=0x5585ba2843a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.179+0000 7f87b2a00700 0 client.37226 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.180+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3165551179 shutdown_connections 2024-05-27T07:58:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.180+0000 7f87e21a4b80 1 -- 172.21.15.90:0/3165551179 wait complete. 2024-05-27T07:58:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.181+0000 7f87e21a4b80 20 client.37226 trim_cache size 0 max 16384 2024-05-27T07:58:46.186 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:58:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.185+0000 7f87d3600700 1 -- 172.21.15.90:0/2007727638 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87b40b2a70 msgr2=0x7f87b40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.185+0000 7f87d3600700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87b40b2a70 0x7f87b40b4f80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009750 tx=0x7f87b801a040 comp rx=0 tx=0).stop 2024-05-27T07:58:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.185+0000 7f87d3600700 1 -- 172.21.15.90:0/2007727638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc1045e0 msgr2=0x7f87cc10d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.185+0000 7f87d3600700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc1045e0 0x7f87cc10d5d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f87c0006fd0 tx=0x7f87c000b7e0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.185+0000 7f87d3600700 1 -- 172.21.15.90:0/2007727638 shutdown_connections 2024-05-27T07:58:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.185+0000 7f87d3600700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc109860 0x7f87cc10e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.185+0000 7f87d3600700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87cc1045e0 0x7f87cc10d5d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.185+0000 7f87d3600700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87b40b2a70 0x7f87b40b4f80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.185+0000 7f87d3600700 1 --2- 172.21.15.90:0/2007727638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87cc108f10 0x7f87cc10db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.185+0000 7f87d3600700 1 -- 172.21.15.90:0/2007727638 >> 172.21.15.90:0/2007727638 conn(0x7f87cc100030 msgr2=0x7f87cc102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.186+0000 7f87d3600700 1 -- 172.21.15.90:0/2007727638 shutdown_connections 2024-05-27T07:58:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:46.186+0000 7f87d3600700 1 -- 172.21.15.90:0/2007727638 wait complete. 2024-05-27T07:58:46.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:46 smithi090 ceph-mon[31429]: pgmap v394: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 580 op/s 2024-05-27T07:58:47.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:46 smithi136 ceph-mon[35324]: pgmap v394: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 580 op/s 2024-05-27T07:58:47.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:46 smithi129 ceph-mon[35254]: pgmap v394: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 MiB/s wr, 580 op/s 2024-05-27T07:58:48.213 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:58:48.214 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:58:48.659 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:48 smithi090 ceph-mon[31429]: pgmap v395: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.5 MiB/s wr, 491 op/s 2024-05-27T07:58:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 510.0 s, 486.4 tps, lat 206.249 ms stddev 143.620 2024-05-27T07:58:49.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:48 smithi136 ceph-mon[35324]: pgmap v395: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.5 MiB/s wr, 491 op/s 2024-05-27T07:58:49.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:48 smithi129 ceph-mon[35254]: pgmap v395: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.5 MiB/s wr, 491 op/s 2024-05-27T07:58:49.478 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:50.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:50 smithi090 ceph-mon[31429]: pgmap v396: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 649 op/s 2024-05-27T07:58:51.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:50 smithi136 ceph-mon[35324]: pgmap v396: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 649 op/s 2024-05-27T07:58:51.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:50 smithi129 ceph-mon[35254]: pgmap v396: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 649 op/s 2024-05-27T07:58:52.926 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:52 smithi090 ceph-mon[31429]: pgmap v397: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.2 MiB/s wr, 499 op/s 2024-05-27T07:58:53.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:52 smithi136 ceph-mon[35324]: pgmap v397: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.2 MiB/s wr, 499 op/s 2024-05-27T07:58:53.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:52 smithi129 ceph-mon[35254]: pgmap v397: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 6.2 MiB/s wr, 499 op/s 2024-05-27T07:58:53.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 515.0 s, 512.6 tps, lat 184.277 ms stddev 160.482 2024-05-27T07:58:54.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:54 smithi090 ceph-mon[31429]: pgmap v398: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.6 MiB/s wr, 529 op/s 2024-05-27T07:58:55.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:54 smithi136 ceph-mon[35324]: pgmap v398: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.6 MiB/s wr, 529 op/s 2024-05-27T07:58:55.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:54 smithi129 ceph-mon[35254]: pgmap v398: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 6.6 MiB/s wr, 529 op/s 2024-05-27T07:58:55.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.496+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1417009846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd810c5a0 msgr2=0x7f0bd810c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.496+0000 7f0bdcc00700 1 --2- 172.21.15.90:0/1417009846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd810c5a0 0x7f0bd810c9b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009990 tx=0x7f0bc8009ca0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.497+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1417009846 shutdown_connections 2024-05-27T07:58:55.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.497+0000 7f0bdcc00700 1 --2- 172.21.15.90:0/1417009846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8106eb0 0x7f0bd8113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.497+0000 7f0bdcc00700 1 --2- 172.21.15.90:0/1417009846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd810c5a0 0x7f0bd810c9b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.497+0000 7f0bdcc00700 1 --2- 172.21.15.90:0/1417009846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bd8106560 0x7f0bd8106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.497+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1417009846 >> 172.21.15.90:0/1417009846 conn(0x7f0bd8102040 msgr2=0x7f0bd8104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.498+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1417009846 shutdown_connections 2024-05-27T07:58:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.498+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1417009846 wait complete. 2024-05-27T07:58:55.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.500+0000 7f0bdcc00700 1 Processor -- start 2024-05-27T07:58:55.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.501+0000 7f0bdcc00700 1 -- start start 2024-05-27T07:58:55.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.503+0000 7f0bdcc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd8106560 0x7f0bd8077cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.503+0000 7f0bdcc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8106eb0 0x7f0bd8078200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.503+0000 7f0bdcc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bd810c5a0 0x7f0bd8078740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.503+0000 7f0bd5e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd8106560 0x7f0bd8077cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.503+0000 7f0bd5e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd8106560 0x7f0bd8077cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53506/0 (socket says 172.21.15.90:53506) 2024-05-27T07:58:55.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.504+0000 7f0bd5e00700 1 -- 172.21.15.90:0/1517696450 learned_addr learned my addr 172.21.15.90:0/1517696450 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.504+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8116600 con 0x7f0bd810c5a0 2024-05-27T07:58:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.504+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8116480 con 0x7f0bd8106560 2024-05-27T07:58:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.504+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8116780 con 0x7f0bd8106eb0 2024-05-27T07:58:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.504+0000 7f0bd5400700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8106eb0 0x7f0bd8078200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.504+0000 7f0bd5400700 1 -- 172.21.15.90:0/1517696450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd8106560 msgr2=0x7f0bd8077cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.504+0000 7f0bd5400700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd8106560 0x7f0bd8077cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.504+0000 7f0bd5400700 1 -- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bd810c5a0 msgr2=0x7f0bd8078740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.504+0000 7f0bd6800700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bd810c5a0 0x7f0bd8078740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.505+0000 7f0bd5400700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bd810c5a0 0x7f0bd8078740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.505+0000 7f0bd5400700 1 -- 172.21.15.90:0/1517696450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd81ad550 con 0x7f0bd8106eb0 2024-05-27T07:58:55.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.505+0000 7f0bd6800700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bd810c5a0 0x7f0bd8078740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:55.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.505+0000 7f0bd5400700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8106eb0 0x7f0bd8078200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8075b70 tx=0x7f0bc400ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.506+0000 7f0bcea00700 1 -- 172.21.15.90:0/1517696450 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400c9c0 con 0x7f0bd8106eb0 2024-05-27T07:58:55.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.506+0000 7f0bcea00700 1 -- 172.21.15.90:0/1517696450 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc4010430 con 0x7f0bd8106eb0 2024-05-27T07:58:55.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.506+0000 7f0bcea00700 1 -- 172.21.15.90:0/1517696450 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4018680 con 0x7f0bd8106eb0 2024-05-27T07:58:55.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.506+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd81ad780 con 0x7f0bd8106eb0 2024-05-27T07:58:55.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.506+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd81adc40 con 0x7f0bd8106eb0 2024-05-27T07:58:55.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.514+0000 7f0bcea00700 1 -- 172.21.15.90:0/1517696450 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0bc40108e0 con 0x7f0bd8106eb0 2024-05-27T07:58:55.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.516+0000 7f0bcea00700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bbc0b2c00 0x7f0bbc0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.516+0000 7f0bcea00700 1 -- 172.21.15.90:0/1517696450 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f0bc40e2a60 con 0x7f0bd8106eb0 2024-05-27T07:58:55.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.517+0000 7f0bd5e00700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bbc0b2c00 0x7f0bbc0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.520+0000 7f0bd5e00700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bbc0b2c00 0x7f0bbc0b5110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009750 tx=0x7f0bc80058e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.543+0000 7f0beb661b80 1 Processor -- start 2024-05-27T07:58:55.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.544+0000 7f0beb661b80 1 -- start start 2024-05-27T07:58:55.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.544+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 0x561eab4f1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.544+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4f1db0 0x561eab4eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.544+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 0x561eab4ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.544+0000 7f0beb661b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561eab419ec0 con 0x561eab4ebe30 2024-05-27T07:58:55.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.544+0000 7f0beb661b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561eab409a60 con 0x561eab36cec0 2024-05-27T07:58:55.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.544+0000 7f0beb661b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561eab40daa0 con 0x561eab4f1db0 2024-05-27T07:58:55.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.545+0000 7f0bd5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4f1db0 0x561eab4eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.545+0000 7f0bd5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4f1db0 0x561eab4eb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42656/0 (socket says 172.21.15.90:42656) 2024-05-27T07:58:55.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.545+0000 7f0bd5400700 1 -- 172.21.15.90:0/461194716 learned_addr learned my addr 172.21.15.90:0/461194716 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:55.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.545+0000 7f0bd5e00700 1 --2- 172.21.15.90:0/461194716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 0x561eab4f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.545+0000 7f0bd5400700 1 -- 172.21.15.90:0/461194716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 msgr2=0x561eab4f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.545+0000 7f0bd5400700 1 --2- 172.21.15.90:0/461194716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 0x561eab4f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.545+0000 7f0bd5400700 1 -- 172.21.15.90:0/461194716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 msgr2=0x561eab4ec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.545+0000 7f0bd5400700 1 --2- 172.21.15.90:0/461194716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 0x561eab4ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.546+0000 7f0bd5400700 1 -- 172.21.15.90:0/461194716 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eab3d24a0 con 0x561eab4f1db0 2024-05-27T07:58:55.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.546+0000 7f0bd5400700 1 --2- 172.21.15.90:0/461194716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4f1db0 0x561eab4eb8f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40a5990 tx=0x7f0bc40a9810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11e4d9b6478fa643 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.546+0000 7f0bc3400700 1 -- 172.21.15.90:0/461194716 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40a9910 con 0x561eab4f1db0 2024-05-27T07:58:55.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.547+0000 7f0bc3400700 1 -- 172.21.15.90:0/461194716 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc40adcc0 con 0x561eab4f1db0 2024-05-27T07:58:55.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.547+0000 7f0bc3400700 1 -- 172.21.15.90:0/461194716 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40a9dd0 con 0x561eab4f1db0 2024-05-27T07:58:55.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.548+0000 7f0beb661b80 1 -- 172.21.15.90:0/461194716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4f1db0 msgr2=0x561eab4eb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.548+0000 7f0beb661b80 1 --2- 172.21.15.90:0/461194716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4f1db0 0x561eab4eb8f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40a5990 tx=0x7f0bc40a9810 comp rx=0 tx=0).stop 2024-05-27T07:58:55.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.550+0000 7f0beb661b80 1 -- 172.21.15.90:0/461194716 shutdown_connections 2024-05-27T07:58:55.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.550+0000 7f0beb661b80 1 --2- 172.21.15.90:0/461194716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 0x561eab4ec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.550+0000 7f0beb661b80 1 --2- 172.21.15.90:0/461194716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 0x561eab4f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.550+0000 7f0beb661b80 1 --2- 172.21.15.90:0/461194716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4f1db0 0x561eab4eb8f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.550+0000 7f0beb661b80 1 -- 172.21.15.90:0/461194716 >> 172.21.15.90:0/461194716 conn(0x561eab3cd040 msgr2=0x561eab4e6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.550+0000 7f0beb661b80 1 -- 172.21.15.90:0/461194716 shutdown_connections 2024-05-27T07:58:55.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.550+0000 7f0beb661b80 1 -- 172.21.15.90:0/461194716 wait complete. 2024-05-27T07:58:55.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.551+0000 7f0beb661b80 1 -- start start 2024-05-27T07:58:55.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.552+0000 7f0beb661b80 1 Processor -- start 2024-05-27T07:58:55.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.552+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab36cec0 0x561eab4f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.552+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 0x561eab4f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.552+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 0x561eab4f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.552+0000 7f0beb661b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561eab310020 con 0x561eab36cec0 2024-05-27T07:58:55.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.552+0000 7f0beb661b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561eab3aa7e0 con 0x561eab4f1db0 2024-05-27T07:58:55.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.553+0000 7f0beb661b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561eab4f8880 con 0x561eab4ebe30 2024-05-27T07:58:55.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.553+0000 7f0beb661b80 10 client.0 _ll_register_callbacks cb 0x561eab3b3cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:55.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.554+0000 7f0bd5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab36cec0 0x561eab4f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.554+0000 7f0bd5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab36cec0 0x561eab4f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56388/0 (socket says 172.21.15.90:56388) 2024-05-27T07:58:55.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.554+0000 7f0bd5e00700 1 -- 172.21.15.90:0/1193585448 learned_addr learned my addr 172.21.15.90:0/1193585448 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:55.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.554+0000 7f0bd6800700 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 0x561eab4f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.554+0000 7f0bd5400700 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 0x561eab4f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.555+0000 7f0bd6800700 1 -- 172.21.15.90:0/1193585448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 msgr2=0x561eab4f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.555+0000 7f0bd6800700 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 0x561eab4f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.555+0000 7f0bd6800700 1 -- 172.21.15.90:0/1193585448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab36cec0 msgr2=0x561eab4f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.555+0000 7f0bd6800700 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab36cec0 0x561eab4f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.555+0000 7f0bd6800700 1 -- 172.21.15.90:0/1193585448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eab359950 con 0x561eab4f1db0 2024-05-27T07:58:55.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.556+0000 7f0bd6800700 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 0x561eab4f60a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000d910 tx=0x7f0bd000dc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f961915703baff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.556+0000 7f0bb7e00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:58:55.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.556+0000 7f0bb7e00700 1 -- 172.21.15.90:0/1193585448 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0016070 con 0x561eab4f1db0 2024-05-27T07:58:55.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.556+0000 7f0bb7e00700 1 -- 172.21.15.90:0/1193585448 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f0bd0004bb0 con 0x561eab4f1db0 2024-05-27T07:58:55.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.557+0000 7f0beb661b80 1 -- 172.21.15.90:0/1193585448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc40adcc0 con 0x561eab4f1db0 2024-05-27T07:58:55.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.557+0000 7f0bb7e00700 1 -- 172.21.15.90:0/1193585448 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f0bd001f050 con 0x561eab4f1db0 2024-05-27T07:58:55.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.557+0000 7f0beb661b80 10 client.37238 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:55.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.557+0000 7f0beb661b80 1 -- 172.21.15.90:0/1193585448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561eab3da0e0 con 0x561eab4f1db0 2024-05-27T07:58:55.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.558+0000 7f0bb7e00700 1 -- 172.21.15.90:0/1193585448 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bc40adcc0 con 0x561eab4f1db0 2024-05-27T07:58:55.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.558+0000 7f0bb7e00700 1 -- 172.21.15.90:0/1193585448 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f0bd0005270 con 0x561eab4f1db0 2024-05-27T07:58:55.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.558+0000 7f0beb661b80 10 client.37238 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:55.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.558+0000 7f0beb661b80 10 client.37238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:55.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.558+0000 7f0beb661b80 20 client.37238 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:55.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.558+0000 7f0beb661b80 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561eab516cb0 0x561eab519160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.558+0000 7f0beb661b80 4 client.37238 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:55.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.558+0000 7f0beb661b80 1 -- 172.21.15.90:0/1193585448 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561eab41be10 con 0x561eab516cb0 2024-05-27T07:58:55.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.560+0000 7f0bd5e00700 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561eab516cb0 0x561eab519160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.561+0000 7f0bd5e00700 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561eab516cb0 0x561eab519160 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bccb210e84a4c889 server_cookie=65e8c71c932ed484 in_seq=0 out_seq=0 2024-05-27T07:58:55.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.561+0000 7f0bb7e00700 10 client.37238 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:55.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.562+0000 7f0bb7e00700 1 -- 172.21.15.90:0/1193585448 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561eab41be10 con 0x561eab516cb0 2024-05-27T07:58:55.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.562+0000 7f0bb7e00700 10 client.37238 handle_command_reply: tid=0 2024-05-27T07:58:55.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.563+0000 7f0beb661b80 1 client.37238 shutdown 2024-05-27T07:58:55.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.563+0000 7f0beb661b80 1 -- 172.21.15.90:0/1193585448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 msgr2=0x561eab4f60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.563+0000 7f0beb661b80 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 0x561eab4f60a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000d910 tx=0x7f0bd000dc20 comp rx=0 tx=0).stop 2024-05-27T07:58:55.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.564+0000 7f0beb661b80 1 -- 172.21.15.90:0/1193585448 shutdown_connections 2024-05-27T07:58:55.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.564+0000 7f0beb661b80 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561eab516cb0 0x561eab519160 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.564+0000 7f0beb661b80 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 0x561eab4f60a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.564+0000 7f0beb661b80 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab36cec0 0x561eab4f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.564+0000 7f0beb661b80 1 --2- 172.21.15.90:0/1193585448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 0x561eab4f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.564+0000 7f0beb661b80 1 -- 172.21.15.90:0/1193585448 >> 172.21.15.90:0/1193585448 conn(0x561eab3cd040 msgr2=0x561eab4e6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.564+0000 7f0bb7e00700 0 client.37238 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:55.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.565+0000 7f0beb661b80 1 -- 172.21.15.90:0/1193585448 shutdown_connections 2024-05-27T07:58:55.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.566+0000 7f0beb661b80 1 -- 172.21.15.90:0/1193585448 wait complete. 2024-05-27T07:58:55.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.567+0000 7f0beb661b80 20 client.37238 trim_cache size 0 max 16384 2024-05-27T07:58:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.605+0000 7f0beb661b80 1 Processor -- start 2024-05-27T07:58:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.605+0000 7f0beb661b80 1 -- start start 2024-05-27T07:58:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.606+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab36cec0 0x561eab4eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.606+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 0x561eab4f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.606+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 0x561eab4f4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.606+0000 7f0beb661b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81afe60 con 0x561eab4ebe30 2024-05-27T07:58:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.606+0000 7f0beb661b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8074920 con 0x561eab4f1db0 2024-05-27T07:58:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.606+0000 7f0beb661b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81af0c0 con 0x561eab36cec0 2024-05-27T07:58:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.606+0000 7f0bd5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 0x561eab4f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.607+0000 7f0bd5e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab36cec0 0x561eab4eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.607+0000 7f0bd5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 0x561eab4f3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56402/0 (socket says 172.21.15.90:56402) 2024-05-27T07:58:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.607+0000 7f0bd5400700 1 -- 172.21.15.90:0/2631347090 learned_addr learned my addr 172.21.15.90:0/2631347090 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.607+0000 7f0bd6800700 1 --2- 172.21.15.90:0/2631347090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 0x561eab4f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.608+0000 7f0bd6800700 1 -- 172.21.15.90:0/2631347090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab36cec0 msgr2=0x561eab4eb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.608+0000 7f0bd6800700 1 --2- 172.21.15.90:0/2631347090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab36cec0 0x561eab4eb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.608+0000 7f0bd6800700 1 -- 172.21.15.90:0/2631347090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 msgr2=0x561eab4f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.608+0000 7f0bd6800700 1 --2- 172.21.15.90:0/2631347090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 0x561eab4f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.608+0000 7f0bd6800700 1 -- 172.21.15.90:0/2631347090 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eab4f3b90 con 0x561eab4f1db0 2024-05-27T07:58:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.608+0000 7f0bd5400700 1 --2- 172.21.15.90:0/2631347090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 0x561eab4f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:55.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.609+0000 7f0bd6800700 1 --2- 172.21.15.90:0/2631347090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 0x561eab4f4450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00223a0 tx=0x7f0bd000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42a46ce3676e930 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.612+0000 7f0bc2a00700 1 -- 172.21.15.90:0/2631347090 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd001f050 con 0x561eab4f1db0 2024-05-27T07:58:55.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.612+0000 7f0bc2a00700 1 -- 172.21.15.90:0/2631347090 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bd0005980 con 0x561eab4f1db0 2024-05-27T07:58:55.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.612+0000 7f0bc2a00700 1 -- 172.21.15.90:0/2631347090 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0022710 con 0x561eab4f1db0 2024-05-27T07:58:55.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.613+0000 7f0beb661b80 1 -- 172.21.15.90:0/2631347090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 msgr2=0x561eab4f4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.613+0000 7f0beb661b80 1 --2- 172.21.15.90:0/2631347090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 0x561eab4f4450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00223a0 tx=0x7f0bd000ee90 comp rx=0 tx=0).stop 2024-05-27T07:58:55.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.614+0000 7f0beb661b80 1 -- 172.21.15.90:0/2631347090 shutdown_connections 2024-05-27T07:58:55.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.614+0000 7f0beb661b80 1 --2- 172.21.15.90:0/2631347090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab4f1db0 0x561eab4f4450 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.614+0000 7f0beb661b80 1 --2- 172.21.15.90:0/2631347090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab36cec0 0x561eab4eb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.614+0000 7f0beb661b80 1 --2- 172.21.15.90:0/2631347090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4ebe30 0x561eab4f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.614+0000 7f0beb661b80 1 -- 172.21.15.90:0/2631347090 >> 172.21.15.90:0/2631347090 conn(0x561eab3cd040 msgr2=0x561eab4efbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.615+0000 7f0beb661b80 1 -- 172.21.15.90:0/2631347090 shutdown_connections 2024-05-27T07:58:55.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.615+0000 7f0beb661b80 1 -- 172.21.15.90:0/2631347090 wait complete. 2024-05-27T07:58:55.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.615+0000 7f0beb661b80 1 -- start start 2024-05-27T07:58:55.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.616+0000 7f0beb661b80 1 Processor -- start 2024-05-27T07:58:55.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.617+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 0x561eab4fdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.617+0000 7f0beb661b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 0x561eab4fe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.618+0000 7f0bd5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 0x561eab4fe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.618+0000 7f0bd5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 0x561eab4fe050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42688/0 (socket says 172.21.15.90:42688) 2024-05-27T07:58:55.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.618+0000 7f0bd5e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 0x561eab4fdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.619+0000 7f0bd5e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 0x561eab4fdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53536/0 (socket says 172.21.15.90:53536) 2024-05-27T07:58:55.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.619+0000 7f0bd5400700 1 -- 172.21.15.90:0/3032962587 learned_addr learned my addr 172.21.15.90:0/3032962587 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:55.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.619+0000 7f0beb661b80 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4f1db0 0x561eab4faae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.620+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd81ae9f0 con 0x561eab4f1db0 2024-05-27T07:58:55.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.620+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8074ff0 con 0x561eab36cec0 2024-05-27T07:58:55.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.620+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd80756c0 con 0x561eab4ebe30 2024-05-27T07:58:55.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.620+0000 7f0beb661b80 10 client.0 _ll_register_callbacks cb 0x561eab3b3cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:55.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.620+0000 7f0bd6800700 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4f1db0 0x561eab4faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.621+0000 7f0bd5400700 1 -- 172.21.15.90:0/3032962587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 msgr2=0x561eab4fdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.621+0000 7f0bd5400700 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 0x561eab4fdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.621+0000 7f0bd5400700 1 -- 172.21.15.90:0/3032962587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4f1db0 msgr2=0x561eab4faae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:55.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.621+0000 7f0bd5400700 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4f1db0 0x561eab4faae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.621+0000 7f0bd5400700 1 -- 172.21.15.90:0/3032962587 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eab4fb020 con 0x561eab4ebe30 2024-05-27T07:58:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.622+0000 7f0bd5400700 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 0x561eab4fe050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40a6b80 tx=0x7f0bc40a9da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44bcb0a86cf255aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.622+0000 7f0bb7e00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:58:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.622+0000 7f0bb7e00700 1 -- 172.21.15.90:0/3032962587 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40a57d0 con 0x561eab4ebe30 2024-05-27T07:58:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.622+0000 7f0bb7e00700 1 -- 172.21.15.90:0/3032962587 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f0bc40a5f00 con 0x561eab4ebe30 2024-05-27T07:58:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.623+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bd0005980 con 0x561eab4ebe30 2024-05-27T07:58:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.626+0000 7f0bb7e00700 1 -- 172.21.15.90:0/3032962587 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f0bc40adcc0 con 0x561eab4ebe30 2024-05-27T07:58:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.626+0000 7f0beb661b80 10 client.27334 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:55.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.626+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561eab581ef0 con 0x561eab4ebe30 2024-05-27T07:58:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.626+0000 7f0bb7e00700 1 -- 172.21.15.90:0/3032962587 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bd0005980 con 0x561eab4ebe30 2024-05-27T07:58:55.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.627+0000 7f0bb7e00700 1 -- 172.21.15.90:0/3032962587 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f0bc40cace0 con 0x561eab4ebe30 2024-05-27T07:58:55.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.627+0000 7f0beb661b80 10 client.27334 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:55.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.627+0000 7f0beb661b80 10 client.27334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:55.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.627+0000 7f0beb661b80 20 client.27334 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:55.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.627+0000 7f0beb661b80 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561eab5823b0 0x561eab5847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:55.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.627+0000 7f0beb661b80 4 client.27334 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T07:58:55.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.627+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561eab4155f0 con 0x561eab5823b0 2024-05-27T07:58:55.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.628+0000 7f0bd5e00700 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561eab5823b0 0x561eab5847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:55.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.631+0000 7f0bd5e00700 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561eab5823b0 0x561eab5847d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=debcb599f37dc17a server_cookie=8c436f048d4fc038 in_seq=0 out_seq=0 2024-05-27T07:58:55.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.631+0000 7f0bb7e00700 10 client.27334 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.634+0000 7f0bb7e00700 1 -- 172.21.15.90:0/3032962587 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561eab4155f0 con 0x561eab5823b0 2024-05-27T07:58:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.634+0000 7f0bb7e00700 10 client.27334 handle_command_reply: tid=0 2024-05-27T07:58:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.634+0000 7f0beb661b80 1 client.27334 shutdown 2024-05-27T07:58:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.634+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 msgr2=0x561eab4fe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.634+0000 7f0beb661b80 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 0x561eab4fe050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40a6b80 tx=0x7f0bc40a9da0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.635+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 shutdown_connections 2024-05-27T07:58:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.635+0000 7f0beb661b80 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561eab5823b0 0x561eab5847d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.635+0000 7f0beb661b80 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561eab4f1db0 0x561eab4faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.635+0000 7f0beb661b80 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561eab36cec0 0x561eab4fdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.635+0000 7f0beb661b80 1 --2- 172.21.15.90:0/3032962587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561eab4ebe30 0x561eab4fe050 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.635+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 >> 172.21.15.90:0/3032962587 conn(0x561eab3cd040 msgr2=0x561eab501340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.635+0000 7f0bb7e00700 0 client.27334 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.635+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 shutdown_connections 2024-05-27T07:58:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.635+0000 7f0beb661b80 1 -- 172.21.15.90:0/3032962587 wait complete. 2024-05-27T07:58:55.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.635+0000 7f0beb661b80 20 client.27334 trim_cache size 0 max 16384 2024-05-27T07:58:55.639 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:58:55.639 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T07:58:55.639 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "fb3fd793-bafe-4163-9acc-1384bf3c1689", 2024-05-27T07:58:55.639 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T07:58:55.639 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:58:55.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.637+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bbc0b2c00 msgr2=0x7f0bbc0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.637+0000 7f0bdcc00700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bbc0b2c00 0x7f0bbc0b5110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009750 tx=0x7f0bc80058e0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.637+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8106eb0 msgr2=0x7f0bd8078200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:55.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.637+0000 7f0bdcc00700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8106eb0 0x7f0bd8078200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8075b70 tx=0x7f0bc400ec20 comp rx=0 tx=0).stop 2024-05-27T07:58:55.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.638+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 shutdown_connections 2024-05-27T07:58:55.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.638+0000 7f0bdcc00700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bd810c5a0 0x7f0bd8078740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.638+0000 7f0bdcc00700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bbc0b2c00 0x7f0bbc0b5110 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.638+0000 7f0bdcc00700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bd8106560 0x7f0bd8077cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.638+0000 7f0bdcc00700 1 --2- 172.21.15.90:0/1517696450 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd8106eb0 0x7f0bd8078200 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:55.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.638+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 >> 172.21.15.90:0/1517696450 conn(0x7f0bd8102040 msgr2=0x7f0bd8103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:55.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.639+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 shutdown_connections 2024-05-27T07:58:55.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:58:55.639+0000 7f0bdcc00700 1 -- 172.21.15.90:0/1517696450 wait complete. 2024-05-27T07:58:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.008+0000 7f10cca00640 1 -- 172.21.15.90:0/1259090622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10c810da60 msgr2=0x7f10c810fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:56.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.008+0000 7f10cca00640 1 --2- 172.21.15.90:0/1259090622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10c810da60 0x7f10c810fe80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009900 tx=0x7f10b402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.010+0000 7f10cca00640 1 -- 172.21.15.90:0/1259090622 shutdown_connections 2024-05-27T07:58:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.010+0000 7f10cca00640 1 --2- 172.21.15.90:0/1259090622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10c810da60 0x7f10c810fe80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.010+0000 7f10cca00640 1 --2- 172.21.15.90:0/1259090622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10c81103c0 0x7f10c81127e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.010+0000 7f10cca00640 1 --2- 172.21.15.90:0/1259090622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c8102820 0x7f10c8102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.010+0000 7f10cca00640 1 -- 172.21.15.90:0/1259090622 >> 172.21.15.90:0/1259090622 conn(0x7f10c80fe210 msgr2=0x7f10c8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.010+0000 7f10cca00640 1 -- 172.21.15.90:0/1259090622 shutdown_connections 2024-05-27T07:58:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.010+0000 7f10cca00640 1 -- 172.21.15.90:0/1259090622 wait complete. 2024-05-27T07:58:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.012+0000 7f10cca00640 1 Processor -- start 2024-05-27T07:58:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.012+0000 7f10cca00640 1 -- start start 2024-05-27T07:58:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.013+0000 7f10cca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10c8102820 0x7f10c8073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.013+0000 7f10cca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c810da60 0x7f10c8074150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.013+0000 7f10cca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10c81103c0 0x7f10c8106620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.013+0000 7f10cca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10c806ce00 con 0x7f10c810da60 2024-05-27T07:58:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.013+0000 7f10cca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10c806cc80 con 0x7f10c81103c0 2024-05-27T07:58:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.013+0000 7f10cca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f10c806cf80 con 0x7f10c8102820 2024-05-27T07:58:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.014+0000 7f10c6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10c8102820 0x7f10c8073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.014+0000 7f10c6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10c8102820 0x7f10c8073c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42712/0 (socket says 172.21.15.90:42712) 2024-05-27T07:58:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.014+0000 7f10c6800640 1 -- 172.21.15.90:0/3335045485 learned_addr learned my addr 172.21.15.90:0/3335045485 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.014+0000 7f10c7200640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10c81103c0 0x7f10c8106620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.014+0000 7f10c5e00640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c810da60 0x7f10c8074150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.015+0000 7f10c7200640 1 -- 172.21.15.90:0/3335045485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10c8102820 msgr2=0x7f10c8073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.015+0000 7f10c7200640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10c8102820 0x7f10c8073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.015+0000 7f10c7200640 1 -- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c810da60 msgr2=0x7f10c8074150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.015+0000 7f10c7200640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c810da60 0x7f10c8074150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.015+0000 7f10c7200640 1 -- 172.21.15.90:0/3335045485 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c8106b60 con 0x7f10c81103c0 2024-05-27T07:58:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.015+0000 7f10c5e00640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c810da60 0x7f10c8074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:58:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.017+0000 7f10c7200640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10c81103c0 0x7f10c8106620 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10c000e940 tx=0x7f10c000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.018+0000 7f10bf400640 1 -- 172.21.15.90:0/3335045485 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0004d20 con 0x7f10c81103c0 2024-05-27T07:58:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.018+0000 7f10bf400640 1 -- 172.21.15.90:0/3335045485 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10c0005020 con 0x7f10c81103c0 2024-05-27T07:58:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.018+0000 7f10bf400640 1 -- 172.21.15.90:0/3335045485 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c00116d0 con 0x7f10c81103c0 2024-05-27T07:58:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.019+0000 7f10cca00640 1 -- 172.21.15.90:0/3335045485 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c8106e50 con 0x7f10c81103c0 2024-05-27T07:58:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.019+0000 7f10cca00640 1 -- 172.21.15.90:0/3335045485 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c8107320 con 0x7f10c81103c0 2024-05-27T07:58:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.023+0000 7f10d43f7740 1 Processor -- start 2024-05-27T07:58:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.028+0000 7f10bf400640 1 -- 172.21.15.90:0/3335045485 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f10c000cb20 con 0x7f10c81103c0 2024-05-27T07:58:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.028+0000 7f10d43f7740 1 -- start start 2024-05-27T07:58:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.029+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef501b0 0x5583af2f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.029+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.030+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 0x5583af2f3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.030+0000 7f10d43f7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5583af263b90 con 0x5583aef501b0 2024-05-27T07:58:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.030+0000 7f10d43f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5583af253730 con 0x5583aef2f4d0 2024-05-27T07:58:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.030+0000 7f10d43f7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5583af257770 con 0x5583aef272f0 2024-05-27T07:58:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.030+0000 7f10c6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.030+0000 7f10c6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2f2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42724/0 (socket says 172.21.15.90:42724) 2024-05-27T07:58:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.030+0000 7f10c6800640 1 -- 172.21.15.90:0/3818984983 learned_addr learned my addr 172.21.15.90:0/3818984983 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.032+0000 7f10c6800640 1 -- 172.21.15.90:0/3818984983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 msgr2=0x5583af2f3510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:58:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.032+0000 7f10c6800640 1 --2- 172.21.15.90:0/3818984983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 0x5583af2f3510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.032+0000 7f10c6800640 1 -- 172.21.15.90:0/3818984983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef501b0 msgr2=0x5583af2f2a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:58:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.032+0000 7f10c6800640 1 --2- 172.21.15.90:0/3818984983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef501b0 0x5583af2f2a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.032+0000 7f10c6800640 1 -- 172.21.15.90:0/3818984983 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583af1ec8b0 con 0x5583aef272f0 2024-05-27T07:58:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.033+0000 7f10bf400640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f10ac0b2b60 0x7f10ac0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.034+0000 7f10bf400640 1 -- 172.21.15.90:0/3335045485 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f10c00e2ca0 con 0x7f10c81103c0 2024-05-27T07:58:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.035+0000 7f10c5e00640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f10ac0b2b60 0x7f10ac0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.036+0000 7f10c6800640 1 --2- 172.21.15.90:0/3818984983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2f2fd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f10b800ad30 tx=0x7f10b800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c01aa24528d309d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.037+0000 7f10b3400640 1 -- 172.21.15.90:0/3818984983 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b801d070 con 0x5583aef272f0 2024-05-27T07:58:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.037+0000 7f10b3400640 1 -- 172.21.15.90:0/3818984983 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10b800ae20 con 0x5583aef272f0 2024-05-27T07:58:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.037+0000 7f10b3400640 1 -- 172.21.15.90:0/3818984983 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8018b30 con 0x5583aef272f0 2024-05-27T07:58:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.039+0000 7f10c5e00640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f10ac0b2b60 0x7f10ac0b5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f10b40057c0 tx=0x7f10b4000bc0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.040+0000 7f10d43f7740 1 -- 172.21.15.90:0/3818984983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 msgr2=0x5583af2f2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.040+0000 7f10d43f7740 1 --2- 172.21.15.90:0/3818984983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2f2fd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f10b800ad30 tx=0x7f10b800f790 comp rx=0 tx=0).stop 2024-05-27T07:58:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.040+0000 7f10d43f7740 1 -- 172.21.15.90:0/3818984983 shutdown_connections 2024-05-27T07:58:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.040+0000 7f10d43f7740 1 --2- 172.21.15.90:0/3818984983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 0x5583af2f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.040+0000 7f10d43f7740 1 --2- 172.21.15.90:0/3818984983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2f2fd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.040+0000 7f10d43f7740 1 --2- 172.21.15.90:0/3818984983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef501b0 0x5583af2f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.040+0000 7f10d43f7740 1 -- 172.21.15.90:0/3818984983 >> 172.21.15.90:0/3818984983 conn(0x5583aef44300 msgr2=0x5583af2ef240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.040+0000 7f10d43f7740 1 -- 172.21.15.90:0/3818984983 shutdown_connections 2024-05-27T07:58:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.040+0000 7f10d43f7740 1 -- 172.21.15.90:0/3818984983 wait complete. 2024-05-27T07:58:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.042+0000 7f10d43f7740 1 -- start start 2024-05-27T07:58:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.042+0000 7f10d43f7740 1 Processor -- start 2024-05-27T07:58:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.043+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef272f0 0x5583af2fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.043+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef2f4d0 0x5583af2fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.044+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef501b0 0x5583af308f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.044+0000 7f10d43f7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5583aef39790 con 0x5583aef2f4d0 2024-05-27T07:58:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.044+0000 7f10d43f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5583af1fbbc0 con 0x5583aef272f0 2024-05-27T07:58:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.044+0000 7f10d43f7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5583af2ffbc0 con 0x5583aef501b0 2024-05-27T07:58:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.044+0000 7f10d43f7740 10 client.0 _ll_register_callbacks cb 0x5583af1d9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.045+0000 7f10c6800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef272f0 0x5583af2fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.045+0000 7f10c6800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef272f0 0x5583af2fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53560/0 (socket says 172.21.15.90:53560) 2024-05-27T07:58:56.047 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.045+0000 7f10c6800640 1 -- 172.21.15.90:0/8754356 learned_addr learned my addr 172.21.15.90:0/8754356 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.046+0000 7f10c7200640 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef501b0 0x5583af308f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.046+0000 7f10c5e00640 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef2f4d0 0x5583af2fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.046+0000 7f10c6800640 1 -- 172.21.15.90:0/8754356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef501b0 msgr2=0x5583af308f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.046+0000 7f10c6800640 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef501b0 0x5583af308f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.046+0000 7f10c6800640 1 -- 172.21.15.90:0/8754356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef2f4d0 msgr2=0x5583af2fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.046+0000 7f10c6800640 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef2f4d0 0x5583af2fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.046+0000 7f10c6800640 1 -- 172.21.15.90:0/8754356 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583aef68210 con 0x5583aef272f0 2024-05-27T07:58:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.046+0000 7f10c5e00640 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef2f4d0 0x5583af2fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:58:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.047+0000 7f10c6800640 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef272f0 0x5583af2fcc80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f10b801a000 tx=0x7f10b8018aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fabcbedf1040e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.047+0000 7f10a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:58:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.047+0000 7f10a7e00640 1 -- 172.21.15.90:0/8754356 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b801d070 con 0x5583aef272f0 2024-05-27T07:58:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.048+0000 7f10a7e00640 1 -- 172.21.15.90:0/8754356 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f10b8005240 con 0x5583aef272f0 2024-05-27T07:58:56.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.048+0000 7f10d43f7740 1 -- 172.21.15.90:0/8754356 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b800ae20 con 0x5583aef272f0 2024-05-27T07:58:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.049+0000 7f10a7e00640 1 -- 172.21.15.90:0/8754356 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f10b8031050 con 0x5583aef272f0 2024-05-27T07:58:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.049+0000 7f10d43f7740 10 client.37253 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.049+0000 7f10d43f7740 1 -- 172.21.15.90:0/8754356 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5583af034620 con 0x5583aef272f0 2024-05-27T07:58:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.049+0000 7f10a7e00640 1 -- 172.21.15.90:0/8754356 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10b800ae20 con 0x5583aef272f0 2024-05-27T07:58:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.050+0000 7f10a7e00640 1 -- 172.21.15.90:0/8754356 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f10b80106d0 con 0x5583aef272f0 2024-05-27T07:58:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.050+0000 7f10d43f7740 10 client.37253 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.050+0000 7f10d43f7740 10 client.37253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:56.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.050+0000 7f10d43f7740 20 client.37253 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.050+0000 7f10d43f7740 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5583aef0ed80 0x5583af30f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.050+0000 7f10d43f7740 4 client.37253 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:58:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.050+0000 7f10d43f7740 1 -- 172.21.15.90:0/8754356 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583af265ae0 con 0x5583aef0ed80 2024-05-27T07:58:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.051+0000 7f10c5e00640 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5583aef0ed80 0x5583af30f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.052+0000 7f10c5e00640 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5583aef0ed80 0x5583af30f0a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa9e3e974bc818d9 server_cookie=a23d5f194d69fbd5 in_seq=0 out_seq=0 2024-05-27T07:58:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.053+0000 7f10a7e00640 10 client.37253 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.054+0000 7f10a7e00640 1 -- 172.21.15.90:0/8754356 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5583af265ae0 con 0x5583aef0ed80 2024-05-27T07:58:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.054+0000 7f10a7e00640 10 client.37253 handle_command_reply: tid=0 2024-05-27T07:58:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.054+0000 7f10d43f7740 1 client.37253 shutdown 2024-05-27T07:58:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.055+0000 7f10d43f7740 1 -- 172.21.15.90:0/8754356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef272f0 msgr2=0x5583af2fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.055+0000 7f10d43f7740 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef272f0 0x5583af2fcc80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f10b801a000 tx=0x7f10b8018aa0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.055+0000 7f10d43f7740 1 -- 172.21.15.90:0/8754356 shutdown_connections 2024-05-27T07:58:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.055+0000 7f10d43f7740 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5583aef0ed80 0x5583af30f0a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.055+0000 7f10d43f7740 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef501b0 0x5583af308f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.055+0000 7f10d43f7740 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef2f4d0 0x5583af2fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.055+0000 7f10d43f7740 1 --2- 172.21.15.90:0/8754356 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef272f0 0x5583af2fcc80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.056+0000 7f10d43f7740 1 -- 172.21.15.90:0/8754356 >> 172.21.15.90:0/8754356 conn(0x5583aef44300 msgr2=0x5583af2f47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.056+0000 7f10a7e00640 0 client.37253 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.057+0000 7f10d43f7740 1 -- 172.21.15.90:0/8754356 shutdown_connections 2024-05-27T07:58:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.057+0000 7f10d43f7740 1 -- 172.21.15.90:0/8754356 wait complete. 2024-05-27T07:58:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.058+0000 7f10d43f7740 20 client.37253 trim_cache size 0 max 16384 2024-05-27T07:58:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.102+0000 7f10d43f7740 1 Processor -- start 2024-05-27T07:58:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.102+0000 7f10d43f7740 1 -- start start 2024-05-27T07:58:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.104+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2fd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.104+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 0x5583af2fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.104+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef501b0 0x5583af2f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.105+0000 7f10d43f7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10c81ca0a0 con 0x5583aef501b0 2024-05-27T07:58:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.105+0000 7f10d43f7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10c81c7110 con 0x5583aef2f4d0 2024-05-27T07:58:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.105+0000 7f10d43f7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f10c81c9300 con 0x5583aef272f0 2024-05-27T07:58:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.104+0000 7f10c6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2fd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.105+0000 7f10c6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2fd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42740/0 (socket says 172.21.15.90:42740) 2024-05-27T07:58:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.105+0000 7f10c6800640 1 -- 172.21.15.90:0/350299332 learned_addr learned my addr 172.21.15.90:0/350299332 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.105+0000 7f10c7200640 1 --2- 172.21.15.90:0/350299332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef501b0 0x5583af2f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.106+0000 7f10c6800640 1 -- 172.21.15.90:0/350299332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 msgr2=0x5583af2fedd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T07:58:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.106+0000 7f10c6800640 1 --2- 172.21.15.90:0/350299332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 0x5583af2fedd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.106+0000 7f10c6800640 1 -- 172.21.15.90:0/350299332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef501b0 msgr2=0x5583af2f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.106+0000 7f10c6800640 1 --2- 172.21.15.90:0/350299332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef501b0 0x5583af2f2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.106+0000 7f10c6800640 1 -- 172.21.15.90:0/350299332 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583af30f220 con 0x5583aef272f0 2024-05-27T07:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.106+0000 7f10c6800640 1 --2- 172.21.15.90:0/350299332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2fd1d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f10b8010aa0 tx=0x7f10b800abd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eae5d6bf8357d4f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.107+0000 7f10b2a00640 1 -- 172.21.15.90:0/350299332 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b801d040 con 0x5583aef272f0 2024-05-27T07:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.107+0000 7f10b2a00640 1 -- 172.21.15.90:0/350299332 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10b801e040 con 0x5583aef272f0 2024-05-27T07:58:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.107+0000 7f10b2a00640 1 -- 172.21.15.90:0/350299332 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8010640 con 0x5583aef272f0 2024-05-27T07:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.108+0000 7f10d43f7740 1 -- 172.21.15.90:0/350299332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 msgr2=0x5583af2fd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.108+0000 7f10d43f7740 1 --2- 172.21.15.90:0/350299332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2fd1d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f10b8010aa0 tx=0x7f10b800abd0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.109+0000 7f10d43f7740 1 -- 172.21.15.90:0/350299332 shutdown_connections 2024-05-27T07:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.109+0000 7f10d43f7740 1 --2- 172.21.15.90:0/350299332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 0x5583af2fedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.109+0000 7f10d43f7740 1 --2- 172.21.15.90:0/350299332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef501b0 0x5583af2f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.109+0000 7f10d43f7740 1 --2- 172.21.15.90:0/350299332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef272f0 0x5583af2fd1d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.109+0000 7f10d43f7740 1 -- 172.21.15.90:0/350299332 >> 172.21.15.90:0/350299332 conn(0x5583aef44300 msgr2=0x5583af2f2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.109+0000 7f10d43f7740 1 -- 172.21.15.90:0/350299332 shutdown_connections 2024-05-27T07:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.110+0000 7f10d43f7740 1 -- 172.21.15.90:0/350299332 wait complete. 2024-05-27T07:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.110+0000 7f10d43f7740 1 -- start start 2024-05-27T07:58:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.110+0000 7f10d43f7740 1 Processor -- start 2024-05-27T07:58:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.111+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef272f0 0x5583af305110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.111+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 0x5583af305650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.111+0000 7f10d43f7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef501b0 0x5583af2ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10c6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef272f0 0x5583af305110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10c6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef272f0 0x5583af305110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56456/0 (socket says 172.21.15.90:56456) 2024-05-27T07:58:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10c6800640 1 -- 172.21.15.90:0/2600021407 learned_addr learned my addr 172.21.15.90:0/2600021407 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:58:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f10c81c8c30 con 0x5583aef272f0 2024-05-27T07:58:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f10c81c77c0 con 0x5583aef2f4d0 2024-05-27T07:58:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f10c81c7e90 con 0x5583aef501b0 2024-05-27T07:58:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10d43f7740 10 client.0 _ll_register_callbacks cb 0x7f10b801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:58:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10c5e00640 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 0x5583af305650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10c7200640 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef501b0 0x5583af2ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10c6800640 1 -- 172.21.15.90:0/2600021407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef501b0 msgr2=0x5583af2ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.112+0000 7f10c6800640 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef501b0 0x5583af2ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.113+0000 7f10c6800640 1 -- 172.21.15.90:0/2600021407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 msgr2=0x5583af305650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:58:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.113+0000 7f10c6800640 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 0x5583af305650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.113+0000 7f10c6800640 1 -- 172.21.15.90:0/2600021407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583af3005f0 con 0x5583aef272f0 2024-05-27T07:58:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.113+0000 7f10c6800640 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef272f0 0x5583af305110 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f10b8018040 tx=0x7f10b8010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480672781e74e30 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:58:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.113+0000 7f10a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:58:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.114+0000 7f10a7e00640 1 -- 172.21.15.90:0/2600021407 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b801d070 con 0x5583aef272f0 2024-05-27T07:58:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.114+0000 7f10a7e00640 1 -- 172.21.15.90:0/2600021407 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f10b8010850 con 0x5583aef272f0 2024-05-27T07:58:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.114+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b801e040 con 0x5583aef272f0 2024-05-27T07:58:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.114+0000 7f10a7e00640 1 -- 172.21.15.90:0/2600021407 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f10b8005040 con 0x5583aef272f0 2024-05-27T07:58:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.114+0000 7f10d43f7740 10 client.17643 fetch_fsmap learned FSMap version 64 2024-05-27T07:58:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.114+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5583af376150 con 0x5583aef272f0 2024-05-27T07:58:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.116+0000 7f10a7e00640 1 -- 172.21.15.90:0/2600021407 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10b801e040 con 0x5583aef272f0 2024-05-27T07:58:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.116+0000 7f10a7e00640 1 -- 172.21.15.90:0/2600021407 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f10b8004c40 con 0x5583aef272f0 2024-05-27T07:58:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.116+0000 7f10d43f7740 10 client.17643 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:58:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.116+0000 7f10d43f7740 10 client.17643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:58:56.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.116+0000 7f10d43f7740 20 client.17643 populate_metadata read hostname 'smithi090' 2024-05-27T07:58:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.116+0000 7f10d43f7740 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5583aeee41c0 0x5583af378810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:58:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.116+0000 7f10d43f7740 4 client.17643 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:58:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.116+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583af25f2c0 con 0x5583aeee41c0 2024-05-27T07:58:56.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.117+0000 7f10c5e00640 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5583aeee41c0 0x5583af378810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:58:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.119+0000 7f10c5e00640 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5583aeee41c0 0x5583af378810 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ebb8f8e1cc3fa8 server_cookie=1a3ef1be4be6b924 in_seq=0 out_seq=0 2024-05-27T07:58:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.119+0000 7f10a7e00640 10 client.17643 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.179+0000 7f10a7e00640 1 -- 172.21.15.90:0/2600021407 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5583af25f2c0 con 0x5583aeee41c0 2024-05-27T07:58:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.179+0000 7f10a7e00640 10 client.17643 handle_command_reply: tid=0 2024-05-27T07:58:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.179+0000 7f10d43f7740 1 client.17643 shutdown 2024-05-27T07:58:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.180+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef272f0 msgr2=0x5583af305110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.180+0000 7f10d43f7740 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef272f0 0x5583af305110 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f10b8018040 tx=0x7f10b8010f70 comp rx=0 tx=0).stop 2024-05-27T07:58:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.180+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 shutdown_connections 2024-05-27T07:58:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.180+0000 7f10d43f7740 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5583aeee41c0 0x5583af378810 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.180+0000 7f10d43f7740 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583aef2f4d0 0x5583af305650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.180+0000 7f10d43f7740 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5583aef501b0 0x5583af2ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.180+0000 7f10d43f7740 1 --2- 172.21.15.90:0/2600021407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5583aef272f0 0x5583af305110 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.180+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 >> 172.21.15.90:0/2600021407 conn(0x5583aef44300 msgr2=0x5583af2f3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.180+0000 7f10a7e00640 0 client.17643 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:58:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.180+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 shutdown_connections 2024-05-27T07:58:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.181+0000 7f10d43f7740 1 -- 172.21.15.90:0/2600021407 wait complete. 2024-05-27T07:58:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.181+0000 7f10d43f7740 20 client.17643 trim_cache size 0 max 16384 2024-05-27T07:58:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:58:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:58:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:58:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.183+0000 7f10cca00640 1 -- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f10ac0b2b60 msgr2=0x7f10ac0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.183+0000 7f10cca00640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f10ac0b2b60 0x7f10ac0b5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f10b40057c0 tx=0x7f10b4000bc0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.183+0000 7f10cca00640 1 -- 172.21.15.90:0/3335045485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10c81103c0 msgr2=0x7f10c8106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:58:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.183+0000 7f10cca00640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10c81103c0 0x7f10c8106620 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f10c000e940 tx=0x7f10c000ee00 comp rx=0 tx=0).stop 2024-05-27T07:58:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.185+0000 7f10cca00640 1 -- 172.21.15.90:0/3335045485 shutdown_connections 2024-05-27T07:58:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.185+0000 7f10cca00640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f10ac0b2b60 0x7f10ac0b5070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.185+0000 7f10cca00640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f10c81103c0 0x7f10c8106620 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.185+0000 7f10cca00640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c810da60 0x7f10c8074150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.185+0000 7f10cca00640 1 --2- 172.21.15.90:0/3335045485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10c8102820 0x7f10c8073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:58:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.185+0000 7f10cca00640 1 -- 172.21.15.90:0/3335045485 >> 172.21.15.90:0/3335045485 conn(0x7f10c80fe210 msgr2=0x7f10c8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:58:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.186+0000 7f10cca00640 1 -- 172.21.15.90:0/3335045485 shutdown_connections 2024-05-27T07:58:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:58:56.186+0000 7f10cca00640 1 -- 172.21.15.90:0/3335045485 wait complete. 2024-05-27T07:58:56.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:56 smithi090 ceph-mon[31429]: pgmap v399: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 570 op/s 2024-05-27T07:58:56.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:56 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T07:58:56.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:56 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:56.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:56 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T07:58:56.943 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T07:58:57.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:56 smithi136 ceph-mon[35324]: pgmap v399: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 570 op/s 2024-05-27T07:58:57.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:56 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T07:58:57.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:56 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:57.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:56 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T07:58:57.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:56 smithi129 ceph-mon[35254]: pgmap v399: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 570 op/s 2024-05-27T07:58:57.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:56 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T07:58:57.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:56 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:57.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:56 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T07:58:58.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:57 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:58.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:57 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:58.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:57 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T07:58:58.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 520.0 s, 402.0 tps, lat 194.475 ms stddev 152.344 2024-05-27T07:58:59.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:58 smithi136 ceph-mon[35324]: pgmap v400: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 MiB/s wr, 494 op/s 2024-05-27T07:58:59.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:58 smithi129 ceph-mon[35254]: pgmap v400: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 MiB/s wr, 494 op/s 2024-05-27T07:58:59.194 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:58:59.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:58 smithi090 ceph-mon[31429]: pgmap v400: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.2 MiB/s wr, 494 op/s 2024-05-27T07:59:00.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:58:59 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T07:59:00.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:58:59 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T07:59:00.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:58:59 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T07:59:00.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:00 smithi090 ceph-mon[31429]: pgmap v401: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-27T07:59:01.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:00 smithi136 ceph-mon[35324]: pgmap v401: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-27T07:59:01.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:00 smithi129 ceph-mon[35254]: pgmap v401: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.8 MiB/s wr, 774 op/s 2024-05-27T07:59:03.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:02 smithi129 ceph-mon[35254]: pgmap v402: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.0 MiB/s wr, 637 op/s 2024-05-27T07:59:03.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:02 smithi090 ceph-mon[31429]: pgmap v402: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.0 MiB/s wr, 637 op/s 2024-05-27T07:59:03.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:02 smithi136 ceph-mon[35324]: pgmap v402: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.0 MiB/s wr, 637 op/s 2024-05-27T07:59:03.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 525.0 s, 658.9 tps, lat 189.970 ms stddev 333.196 2024-05-27T07:59:05.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:04 smithi129 ceph-mon[35254]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.8 MiB/s wr, 650 op/s 2024-05-27T07:59:05.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:04 smithi090 ceph-mon[31429]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.8 MiB/s wr, 650 op/s 2024-05-27T07:59:05.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:04 smithi136 ceph-mon[35324]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.8 MiB/s wr, 650 op/s 2024-05-27T07:59:05.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.722+0000 7f184d200700 1 -- 172.21.15.90:0/2664986264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f184810b890 msgr2=0x7f184810bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:05.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.722+0000 7f184d200700 1 --2- 172.21.15.90:0/2664986264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f184810b890 0x7f184810bca0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f18380099c0 tx=0x7f1838009cd0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.725+0000 7f184d200700 1 -- 172.21.15.90:0/2664986264 shutdown_connections 2024-05-27T07:59:05.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.725+0000 7f184d200700 1 --2- 172.21.15.90:0/2664986264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1848110fa0 0x7f18481177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.725+0000 7f184d200700 1 --2- 172.21.15.90:0/2664986264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f184810b890 0x7f184810bca0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.725+0000 7f184d200700 1 --2- 172.21.15.90:0/2664986264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1848110650 0x7f1848110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.725+0000 7f184d200700 1 -- 172.21.15.90:0/2664986264 >> 172.21.15.90:0/2664986264 conn(0x7f184807a010 msgr2=0x7f184807c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:05.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.725+0000 7f184d200700 1 -- 172.21.15.90:0/2664986264 shutdown_connections 2024-05-27T07:59:05.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.725+0000 7f184d200700 1 -- 172.21.15.90:0/2664986264 wait complete. 2024-05-27T07:59:05.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.727+0000 7f184d200700 1 Processor -- start 2024-05-27T07:59:05.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.727+0000 7f184d200700 1 -- start start 2024-05-27T07:59:05.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f184d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f184810b890 0x7f18481a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f184d200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1848110650 0x7f18481a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f184d200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1848110fa0 0x7f18481a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f184d200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f184811aa30 con 0x7f1848110650 2024-05-27T07:59:05.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f184d200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f184811a8b0 con 0x7f1848110fa0 2024-05-27T07:59:05.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f184d200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f184811abb0 con 0x7f184810b890 2024-05-27T07:59:05.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f1845e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f184810b890 0x7f18481a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f1845e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f184810b890 0x7f18481a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:48982/0 (socket says 172.21.15.90:48982) 2024-05-27T07:59:05.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f1845e00700 1 -- 172.21.15.90:0/785593823 learned_addr learned my addr 172.21.15.90:0/785593823 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:05.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f1845400700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1848110650 0x7f18481a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.728+0000 7f1846800700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1848110fa0 0x7f18481a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.729+0000 7f1845e00700 1 -- 172.21.15.90:0/785593823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1848110fa0 msgr2=0x7f18481a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:05.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.729+0000 7f1845e00700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1848110fa0 0x7f18481a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.729+0000 7f1845e00700 1 -- 172.21.15.90:0/785593823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1848110650 msgr2=0x7f18481a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:05.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.729+0000 7f1845e00700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1848110650 0x7f18481a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.729+0000 7f1845e00700 1 -- 172.21.15.90:0/785593823 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18481a6310 con 0x7f184810b890 2024-05-27T07:59:05.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.729+0000 7f1845e00700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f184810b890 0x7f18481a1440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1838006010 tx=0x7f1838005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:05.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.730+0000 7f183ea00700 1 -- 172.21.15.90:0/785593823 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838037070 con 0x7f184810b890 2024-05-27T07:59:05.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.730+0000 7f183ea00700 1 -- 172.21.15.90:0/785593823 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f18380040a0 con 0x7f184810b890 2024-05-27T07:59:05.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.730+0000 7f183ea00700 1 -- 172.21.15.90:0/785593823 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183800fc90 con 0x7f184810b890 2024-05-27T07:59:05.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.730+0000 7f184d200700 1 -- 172.21.15.90:0/785593823 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18481a65a0 con 0x7f184810b890 2024-05-27T07:59:05.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.730+0000 7f184d200700 1 -- 172.21.15.90:0/785593823 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18481a6a30 con 0x7f184810b890 2024-05-27T07:59:05.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.737+0000 7f183ea00700 1 -- 172.21.15.90:0/785593823 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f183803b3f0 con 0x7f184810b890 2024-05-27T07:59:05.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.740+0000 7f183ea00700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f182c0b2b40 0x7f182c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.740+0000 7f183ea00700 1 -- 172.21.15.90:0/785593823 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f18380f5750 con 0x7f184810b890 2024-05-27T07:59:05.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.740+0000 7f1845400700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f182c0b2b40 0x7f182c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.750+0000 7f1845400700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f182c0b2b40 0x7f182c0b5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f18481100d0 tx=0x7f183400a300 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:05.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.753+0000 7f185bdd3b80 1 Processor -- start 2024-05-27T07:59:05.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.753+0000 7f185bdd3b80 1 -- start start 2024-05-27T07:59:05.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.754+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 0x5595cc1ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.754+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 0x5595cc1ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.754+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.754+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5595cc0d9e40 con 0x5595cc1189f0 2024-05-27T07:59:05.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.754+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5595cc0c99e0 con 0x5595cc02ce40 2024-05-27T07:59:05.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.754+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5595cc0cda20 con 0x5595cc1abe70 2024-05-27T07:59:05.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.755+0000 7f1846800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.755+0000 7f1845400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 0x5595cc1ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.755+0000 7f1846800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:48990/0 (socket says 172.21.15.90:48990) 2024-05-27T07:59:05.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.755+0000 7f1846800700 1 -- 172.21.15.90:0/1505190066 learned_addr learned my addr 172.21.15.90:0/1505190066 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:05.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.756+0000 7f1845e00700 1 --2- 172.21.15.90:0/1505190066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 0x5595cc1ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.757+0000 7f1846800700 1 -- 172.21.15.90:0/1505190066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 msgr2=0x5595cc1ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:05.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.757+0000 7f1846800700 1 --2- 172.21.15.90:0/1505190066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 0x5595cc1ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.757+0000 7f1846800700 1 -- 172.21.15.90:0/1505190066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 msgr2=0x5595cc1ab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:05.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.757+0000 7f1846800700 1 --2- 172.21.15.90:0/1505190066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 0x5595cc1ab330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.757+0000 7f1846800700 1 -- 172.21.15.90:0/1505190066 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cc092420 con 0x5595cc1abe70 2024-05-27T07:59:05.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.758+0000 7f1845400700 1 --2- 172.21.15.90:0/1505190066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 0x5595cc1ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:05.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.759+0000 7f1846800700 1 --2- 172.21.15.90:0/1505190066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b8110 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f184000b500 tx=0x7f1840009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a17b6c68562940e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:05.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.760+0000 7f1833400700 1 -- 172.21.15.90:0/1505190066 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184000a7f0 con 0x5595cc1abe70 2024-05-27T07:59:05.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.761+0000 7f1833400700 1 -- 172.21.15.90:0/1505190066 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1840019070 con 0x5595cc1abe70 2024-05-27T07:59:05.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.761+0000 7f1833400700 1 -- 172.21.15.90:0/1505190066 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184000a7f0 con 0x5595cc1abe70 2024-05-27T07:59:05.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.763+0000 7f185bdd3b80 1 -- 172.21.15.90:0/1505190066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 msgr2=0x5595cc1b8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:05.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.763+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/1505190066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b8110 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f184000b500 tx=0x7f1840009630 comp rx=0 tx=0).stop 2024-05-27T07:59:05.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.763+0000 7f185bdd3b80 1 -- 172.21.15.90:0/1505190066 shutdown_connections 2024-05-27T07:59:05.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.763+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/1505190066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b8110 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.763+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/1505190066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 0x5595cc1ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.763+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/1505190066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 0x5595cc1ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.763+0000 7f185bdd3b80 1 -- 172.21.15.90:0/1505190066 >> 172.21.15.90:0/1505190066 conn(0x5595cc0c6390 msgr2=0x5595cc1a74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:05.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.763+0000 7f185bdd3b80 1 -- 172.21.15.90:0/1505190066 shutdown_connections 2024-05-27T07:59:05.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.764+0000 7f185bdd3b80 1 -- 172.21.15.90:0/1505190066 wait complete. 2024-05-27T07:59:05.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.765+0000 7f185bdd3b80 1 -- start start 2024-05-27T07:59:05.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.766+0000 7f185bdd3b80 1 Processor -- start 2024-05-27T07:59:05.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.776+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 0x5595cc1b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.776+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 0x5595cc1b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.776+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.777+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5595cbfcffa0 con 0x5595cc1189f0 2024-05-27T07:59:05.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.777+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5595cc06a760 con 0x5595cc02ce40 2024-05-27T07:59:05.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.777+0000 7f1845400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 0x5595cc1b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.777+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5595cc1c9b90 con 0x5595cc1abe70 2024-05-27T07:59:05.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.777+0000 7f1846800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.777+0000 7f1846800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:49006/0 (socket says 172.21.15.90:49006) 2024-05-27T07:59:05.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.777+0000 7f1846800700 1 -- 172.21.15.90:0/680938237 learned_addr learned my addr 172.21.15.90:0/680938237 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:05.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.778+0000 7f1845e00700 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 0x5595cc1b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.778+0000 7f1845e00700 1 -- 172.21.15.90:0/680938237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 msgr2=0x5595cc1b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:05.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.779+0000 7f185bdd3b80 10 client.0 _ll_register_callbacks cb 0x5595cc073c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:05.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.779+0000 7f1845e00700 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.779+0000 7f1845e00700 1 -- 172.21.15.90:0/680938237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 msgr2=0x5595cc1b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:05.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.779+0000 7f1845e00700 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 0x5595cc1b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.779+0000 7f1845e00700 1 -- 172.21.15.90:0/680938237 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cc0198d0 con 0x5595cc02ce40 2024-05-27T07:59:05.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.779+0000 7f185bdd3b80 1 -- 172.21.15.90:0/680938237 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1840019070 con 0x5595cc02ce40 2024-05-27T07:59:05.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.779+0000 7f1845400700 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 0x5595cc1b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:05.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.780+0000 7f1846800700 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:05.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.780+0000 7f1845e00700 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 0x5595cc1b5630 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f18380f2ad0 tx=0x7f18380b8c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b110c3d74312a59c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:05.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.780+0000 7f1827e00700 10 client.37265 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:59:05.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.781+0000 7f1827e00700 1 -- 172.21.15.90:0/680938237 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380bc200 con 0x5595cc02ce40 2024-05-27T07:59:05.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.781+0000 7f1827e00700 1 -- 172.21.15.90:0/680938237 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f18380bcb60 con 0x5595cc02ce40 2024-05-27T07:59:05.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.781+0000 7f1827e00700 1 -- 172.21.15.90:0/680938237 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f18380bcdf0 con 0x5595cc02ce40 2024-05-27T07:59:05.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.783+0000 7f185bdd3b80 10 client.37265 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:05.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.783+0000 7f185bdd3b80 1 -- 172.21.15.90:0/680938237 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({fsmap=64}) -- 0x5595cc09a060 con 0x5595cc02ce40 2024-05-27T07:59:05.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.784+0000 7f1827e00700 1 -- 172.21.15.90:0/680938237 <== mon.2 v2:172.21.15.129:3300/0 4 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f18380c5030 con 0x5595cc02ce40 2024-05-27T07:59:05.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.784+0000 7f185bdd3b80 10 client.37265 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:05.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.784+0000 7f185bdd3b80 10 client.37265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:05.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.784+0000 7f185bdd3b80 20 client.37265 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:05.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.784+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5595cc1d66c0 0x5595cc1d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.784+0000 7f185bdd3b80 4 client.37265 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:05.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.784+0000 7f185bdd3b80 1 -- 172.21.15.90:0/680938237 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595cc0dbd90 con 0x5595cc1d66c0 2024-05-27T07:59:05.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.785+0000 7f1845400700 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5595cc1d66c0 0x5595cc1d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.787+0000 7f1827e00700 10 client.37265 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:05.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.787+0000 7f1845400700 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5595cc1d66c0 0x5595cc1d8b00 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=accf0da667d8e414 server_cookie=d251da4cae60e2f in_seq=0 out_seq=0 2024-05-27T07:59:05.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.789+0000 7f1827e00700 1 -- 172.21.15.90:0/680938237 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5595cc0dbd90 con 0x5595cc1d66c0 2024-05-27T07:59:05.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.789+0000 7f1827e00700 10 client.37265 handle_command_reply: tid=0 2024-05-27T07:59:05.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.789+0000 7f185bdd3b80 1 client.37265 shutdown 2024-05-27T07:59:05.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.790+0000 7f185bdd3b80 1 -- 172.21.15.90:0/680938237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 msgr2=0x5595cc1b5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:05.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.790+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 0x5595cc1b5630 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f18380f2ad0 tx=0x7f18380b8c00 comp rx=0 tx=0).stop 2024-05-27T07:59:05.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.790+0000 7f185bdd3b80 1 -- 172.21.15.90:0/680938237 shutdown_connections 2024-05-27T07:59:05.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.790+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5595cc1d66c0 0x5595cc1d8b00 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.790+0000 7f1827e00700 0 client.37265 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:05.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.791+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1abe70 0x5595cc1b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.791+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc02ce40 0x5595cc1b5630 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.791+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/680938237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1189f0 0x5595cc1b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.791+0000 7f185bdd3b80 1 -- 172.21.15.90:0/680938237 >> 172.21.15.90:0/680938237 conn(0x5595cc0c6390 msgr2=0x5595cc1ab3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:05.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.792+0000 7f185bdd3b80 1 -- 172.21.15.90:0/680938237 shutdown_connections 2024-05-27T07:59:05.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.793+0000 7f185bdd3b80 1 -- 172.21.15.90:0/680938237 wait complete. 2024-05-27T07:59:05.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.793+0000 7f185bdd3b80 20 client.37265 trim_cache size 0 max 16384 2024-05-27T07:59:05.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.835+0000 7f185bdd3b80 1 Processor -- start 2024-05-27T07:59:05.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.835+0000 7f185bdd3b80 1 -- start start 2024-05-27T07:59:05.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.836+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc02ce40 0x5595cc1b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.836+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1189f0 0x5595cc1d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.836+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1abe70 0x5595cc1de2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.836+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f184806eff0 con 0x5595cc02ce40 2024-05-27T07:59:05.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.836+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18480866b0 con 0x5595cc1abe70 2024-05-27T07:59:05.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.836+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f184806e250 con 0x5595cc1189f0 2024-05-27T07:59:05.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.837+0000 7f1845e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc02ce40 0x5595cc1b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.837+0000 7f1846800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1abe70 0x5595cc1de2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.837+0000 7f1845400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1189f0 0x5595cc1d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.837+0000 7f1846800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1abe70 0x5595cc1de2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:36250/0 (socket says 172.21.15.90:36250) 2024-05-27T07:59:05.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.837+0000 7f1846800700 1 -- 172.21.15.90:0/349728443 learned_addr learned my addr 172.21.15.90:0/349728443 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:05.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.838+0000 7f1845e00700 1 -- 172.21.15.90:0/349728443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1189f0 msgr2=0x5595cc1d2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:05.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.838+0000 7f1845e00700 1 --2- 172.21.15.90:0/349728443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1189f0 0x5595cc1d2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.838+0000 7f1845e00700 1 -- 172.21.15.90:0/349728443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1abe70 msgr2=0x5595cc1de2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:05.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.838+0000 7f1845e00700 1 --2- 172.21.15.90:0/349728443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1abe70 0x5595cc1de2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.838+0000 7f1845e00700 1 -- 172.21.15.90:0/349728443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cc1a3240 con 0x5595cc02ce40 2024-05-27T07:59:05.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.838+0000 7f1846800700 1 --2- 172.21.15.90:0/349728443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1abe70 0x5595cc1de2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:05.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.839+0000 7f1845e00700 1 --2- 172.21.15.90:0/349728443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc02ce40 0x5595cc1b0de0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f18380bae20 tx=0x7f18380c3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcc14bff79da442 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:05.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.840+0000 7f1832a00700 1 -- 172.21.15.90:0/349728443 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380c0aa0 con 0x5595cc02ce40 2024-05-27T07:59:05.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.840+0000 7f1832a00700 1 -- 172.21.15.90:0/349728443 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1840019070 con 0x5595cc02ce40 2024-05-27T07:59:05.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.840+0000 7f1832a00700 1 -- 172.21.15.90:0/349728443 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380b8ce0 con 0x5595cc02ce40 2024-05-27T07:59:05.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.841+0000 7f185bdd3b80 1 -- 172.21.15.90:0/349728443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc02ce40 msgr2=0x5595cc1b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:05.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.841+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/349728443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc02ce40 0x5595cc1b0de0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f18380bae20 tx=0x7f18380c3000 comp rx=0 tx=0).stop 2024-05-27T07:59:05.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.841+0000 7f185bdd3b80 1 -- 172.21.15.90:0/349728443 shutdown_connections 2024-05-27T07:59:05.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.841+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/349728443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1abe70 0x5595cc1de2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.841+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/349728443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc02ce40 0x5595cc1b0de0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.841+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/349728443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc1189f0 0x5595cc1d2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.841+0000 7f185bdd3b80 1 -- 172.21.15.90:0/349728443 >> 172.21.15.90:0/349728443 conn(0x5595cc0c6390 msgr2=0x5595cc1b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:05.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.842+0000 7f185bdd3b80 1 -- 172.21.15.90:0/349728443 shutdown_connections 2024-05-27T07:59:05.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.842+0000 7f185bdd3b80 1 -- 172.21.15.90:0/349728443 wait complete. 2024-05-27T07:59:05.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.842+0000 7f185bdd3b80 1 -- start start 2024-05-27T07:59:05.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.843+0000 7f185bdd3b80 1 Processor -- start 2024-05-27T07:59:05.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.843+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc02ce40 0x5595cc1cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.843+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1189f0 0x5595cc1ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.843+0000 7f185bdd3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1abe70 0x5595cc1caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.844+0000 7f1845400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1189f0 0x5595cc1ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.844+0000 7f1845400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1189f0 0x5595cc1ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:36266/0 (socket says 172.21.15.90:36266) 2024-05-27T07:59:05.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.844+0000 7f1845e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc02ce40 0x5595cc1cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.845+0000 7f1845e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc02ce40 0x5595cc1cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:49020/0 (socket says 172.21.15.90:49020) 2024-05-27T07:59:05.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.845+0000 7f1846800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1abe70 0x5595cc1caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.845+0000 7f1846800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1abe70 0x5595cc1caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35520/0 (socket says 172.21.15.90:35520) 2024-05-27T07:59:05.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.845+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f184806db80 con 0x5595cc1abe70 2024-05-27T07:59:05.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.845+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1848086d80 con 0x5595cc1189f0 2024-05-27T07:59:05.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.845+0000 7f185bdd3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1848087450 con 0x5595cc02ce40 2024-05-27T07:59:05.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.845+0000 7f1845400700 1 -- 172.21.15.90:0/2883150714 learned_addr learned my addr 172.21.15.90:0/2883150714 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:05.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.846+0000 7f185bdd3b80 10 client.0 _ll_register_callbacks cb 0x5595cc073c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:05.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.846+0000 7f1845e00700 1 -- 172.21.15.90:0/2883150714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1189f0 msgr2=0x5595cc1ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:05.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.847+0000 7f1845e00700 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1189f0 0x5595cc1ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.847+0000 7f1845e00700 1 -- 172.21.15.90:0/2883150714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1abe70 msgr2=0x5595cc1caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:05.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.847+0000 7f1845e00700 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1abe70 0x5595cc1caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.847+0000 7f1845e00700 1 -- 172.21.15.90:0/2883150714 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cc1b6420 con 0x5595cc02ce40 2024-05-27T07:59:05.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.847+0000 7f1845400700 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1189f0 0x5595cc1ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:05.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.848+0000 7f1845e00700 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc02ce40 0x5595cc1cf1e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f183800f7b0 tx=0x7f18380bc6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c2fa8c18956fea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:05.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.848+0000 7f1827e00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:59:05.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.849+0000 7f1827e00700 1 -- 172.21.15.90:0/2883150714 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380c54c0 con 0x5595cc02ce40 2024-05-27T07:59:05.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.849+0000 7f1827e00700 1 -- 172.21.15.90:0/2883150714 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9496+0+0 (secure 0 0 0) 0x7f18380db050 con 0x5595cc02ce40 2024-05-27T07:59:05.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.849+0000 7f185bdd3b80 1 -- 172.21.15.90:0/2883150714 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1840019070 con 0x5595cc02ce40 2024-05-27T07:59:05.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.849+0000 7f1827e00700 1 -- 172.21.15.90:0/2883150714 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f18380bc200 con 0x5595cc02ce40 2024-05-27T07:59:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.851+0000 7f185bdd3b80 10 client.27364 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:05.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.851+0000 7f185bdd3b80 1 -- 172.21.15.90:0/2883150714 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5595cc1b6420 con 0x5595cc02ce40 2024-05-27T07:59:05.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.852+0000 7f1827e00700 1 -- 172.21.15.90:0/2883150714 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1840019070 con 0x5595cc02ce40 2024-05-27T07:59:05.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.852+0000 7f1827e00700 1 -- 172.21.15.90:0/2883150714 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f18380baac0 con 0x5595cc02ce40 2024-05-27T07:59:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.852+0000 7f185bdd3b80 10 client.27364 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.852+0000 7f185bdd3b80 10 client.27364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.852+0000 7f185bdd3b80 20 client.27364 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.854+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5595cc1d66c0 0x5595cc242d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.854+0000 7f185bdd3b80 4 client.27364 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T07:59:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.854+0000 7f185bdd3b80 1 -- 172.21.15.90:0/2883150714 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595cc1a6120 con 0x5595cc1d66c0 2024-05-27T07:59:05.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.854+0000 7f1845400700 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5595cc1d66c0 0x5595cc242d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.855+0000 7f1845400700 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5595cc1d66c0 0x5595cc242d50 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2edcf29e9c42b76 server_cookie=af42afe3c94b44df in_seq=0 out_seq=0 2024-05-27T07:59:05.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.855+0000 7f1827e00700 10 client.27364 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.856+0000 7f1827e00700 1 -- 172.21.15.90:0/2883150714 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5595cc1a6120 con 0x5595cc1d66c0 2024-05-27T07:59:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.856+0000 7f1827e00700 10 client.27364 handle_command_reply: tid=0 2024-05-27T07:59:05.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.856+0000 7f185bdd3b80 1 client.27364 shutdown 2024-05-27T07:59:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.857+0000 7f185bdd3b80 1 -- 172.21.15.90:0/2883150714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc02ce40 msgr2=0x5595cc1cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.857+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc02ce40 0x5595cc1cf1e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f183800f7b0 tx=0x7f18380bc6f0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.857+0000 7f185bdd3b80 1 -- 172.21.15.90:0/2883150714 shutdown_connections 2024-05-27T07:59:05.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.857+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5595cc1d66c0 0x5595cc242d50 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.857+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5595cc1abe70 0x5595cc1caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.857+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595cc02ce40 0x5595cc1cf1e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.857+0000 7f185bdd3b80 1 --2- 172.21.15.90:0/2883150714 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5595cc1189f0 0x5595cc1ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.857+0000 7f185bdd3b80 1 -- 172.21.15.90:0/2883150714 >> 172.21.15.90:0/2883150714 conn(0x5595cc0c6390 msgr2=0x5595cc1b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:05.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.858+0000 7f1827e00700 0 client.27364 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:05.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.859+0000 7f185bdd3b80 1 -- 172.21.15.90:0/2883150714 shutdown_connections 2024-05-27T07:59:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.859+0000 7f185bdd3b80 1 -- 172.21.15.90:0/2883150714 wait complete. 2024-05-27T07:59:05.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.860+0000 7f185bdd3b80 20 client.27364 trim_cache size 0 max 16384 2024-05-27T07:59:05.861 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T07:59:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T07:59:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T07:59:05.862 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T07:59:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.863+0000 7f184d200700 1 -- 172.21.15.90:0/785593823 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f182c0b2b40 msgr2=0x7f182c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:05.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.863+0000 7f184d200700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f182c0b2b40 0x7f182c0b5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f18481100d0 tx=0x7f183400a300 comp rx=0 tx=0).stop 2024-05-27T07:59:05.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.863+0000 7f184d200700 1 -- 172.21.15.90:0/785593823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f184810b890 msgr2=0x7f18481a1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:05.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.863+0000 7f184d200700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f184810b890 0x7f18481a1440 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1838006010 tx=0x7f1838005f00 comp rx=0 tx=0).stop 2024-05-27T07:59:05.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.863+0000 7f184d200700 1 -- 172.21.15.90:0/785593823 shutdown_connections 2024-05-27T07:59:05.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.863+0000 7f184d200700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f182c0b2b40 0x7f182c0b5050 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f18481100d0 tx=0x7f183400a300 comp rx=0 tx=0).stop 2024-05-27T07:59:05.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.863+0000 7f184d200700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1848110fa0 0x7f18481a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.863+0000 7f184d200700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f184810b890 0x7f18481a1440 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1838006010 tx=0x7f1838005f00 comp rx=0 tx=0).stop 2024-05-27T07:59:05.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.863+0000 7f184d200700 1 --2- 172.21.15.90:0/785593823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1848110650 0x7f18481a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:05.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.863+0000 7f184d200700 1 -- 172.21.15.90:0/785593823 >> 172.21.15.90:0/785593823 conn(0x7f184807a010 msgr2=0x7f184807ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:05.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.864+0000 7f184d200700 1 -- 172.21.15.90:0/785593823 shutdown_connections 2024-05-27T07:59:05.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:05.865+0000 7f184d200700 1 -- 172.21.15.90:0/785593823 wait complete. 2024-05-27T07:59:06.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:06 smithi090 ceph-mon[31429]: pgmap v404: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.0 MiB/s wr, 655 op/s 2024-05-27T07:59:06.987 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T07:59:06.987 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T07:59:07.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:06 smithi129 ceph-mon[35254]: pgmap v404: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.0 MiB/s wr, 655 op/s 2024-05-27T07:59:07.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:06 smithi136 ceph-mon[35324]: pgmap v404: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.0 MiB/s wr, 655 op/s 2024-05-27T07:59:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:08.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 530.0 s, 584.2 tps, lat 174.572 ms stddev 304.920 2024-05-27T07:59:09.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:08 smithi129 ceph-mon[35254]: pgmap v405: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.6 MiB/s wr, 583 op/s 2024-05-27T07:59:09.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:08 smithi090 ceph-mon[31429]: pgmap v405: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.6 MiB/s wr, 583 op/s 2024-05-27T07:59:09.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:08 smithi136 ceph-mon[35324]: pgmap v405: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 7.6 MiB/s wr, 583 op/s 2024-05-27T07:59:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.534+0000 7fe263400640 1 -- 172.21.15.90:0/2727885466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe25c11d240 msgr2=0x7fe25c11f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.534+0000 7fe263400640 1 --2- 172.21.15.90:0/2727885466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe25c11d240 0x7fe25c11f660 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe254009860 tx=0x7fe25402a3f0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.541+0000 7fe263400640 1 -- 172.21.15.90:0/2727885466 shutdown_connections 2024-05-27T07:59:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.541+0000 7fe263400640 1 --2- 172.21.15.90:0/2727885466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25c11a8e0 0x7fe25c11cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.541+0000 7fe263400640 1 --2- 172.21.15.90:0/2727885466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe25c11d240 0x7fe25c11f660 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.541+0000 7fe263400640 1 --2- 172.21.15.90:0/2727885466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe25c10a1f0 0x7fe25c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.541+0000 7fe263400640 1 -- 172.21.15.90:0/2727885466 >> 172.21.15.90:0/2727885466 conn(0x7fe25c06e8f0 msgr2=0x7fe25c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.545+0000 7fe263400640 1 -- 172.21.15.90:0/2727885466 shutdown_connections 2024-05-27T07:59:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.545+0000 7fe263400640 1 -- 172.21.15.90:0/2727885466 wait complete. 2024-05-27T07:59:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.548+0000 7fe263400640 1 Processor -- start 2024-05-27T07:59:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.550+0000 7fe263400640 1 -- start start 2024-05-27T07:59:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.551+0000 7fe263400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe25c10a1f0 0x7fe25c1a9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.551+0000 7fe263400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25c11a8e0 0x7fe25c1a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.551+0000 7fe263400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe25c11d240 0x7fe25c1add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.553+0000 7fe260a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe25c10a1f0 0x7fe25c1a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.554+0000 7fe253e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25c11a8e0 0x7fe25c1a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.554+0000 7fe253e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25c11a8e0 0x7fe25c1a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48654/0 (socket says 172.21.15.90:48654) 2024-05-27T07:59:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.554+0000 7fe253e00640 1 -- 172.21.15.90:0/2011908109 learned_addr learned my addr 172.21.15.90:0/2011908109 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.554+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c122b20 con 0x7fe25c10a1f0 2024-05-27T07:59:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.554+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1229a0 con 0x7fe25c11a8e0 2024-05-27T07:59:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.555+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c122ca0 con 0x7fe25c11d240 2024-05-27T07:59:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.555+0000 7fe261400640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe25c11d240 0x7fe25c1add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.557+0000 7fe253e00640 1 -- 172.21.15.90:0/2011908109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe25c11d240 msgr2=0x7fe25c1add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.557+0000 7fe253e00640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe25c11d240 0x7fe25c1add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.557+0000 7fe253e00640 1 -- 172.21.15.90:0/2011908109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe25c10a1f0 msgr2=0x7fe25c1a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.557+0000 7fe253e00640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe25c10a1f0 0x7fe25c1a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.557+0000 7fe253e00640 1 -- 172.21.15.90:0/2011908109 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c1ae3a0 con 0x7fe25c11a8e0 2024-05-27T07:59:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.558+0000 7fe260a00640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe25c10a1f0 0x7fe25c1a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.558+0000 7fe253e00640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25c11a8e0 0x7fe25c1a9aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe24400e9c0 tx=0x7fe24400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.561+0000 7fe25a000640 1 -- 172.21.15.90:0/2011908109 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24400ccc0 con 0x7fe25c11a8e0 2024-05-27T07:59:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.561+0000 7fe25a000640 1 -- 172.21.15.90:0/2011908109 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2440054b0 con 0x7fe25c11a8e0 2024-05-27T07:59:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.562+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25c1ae690 con 0x7fe25c11a8e0 2024-05-27T07:59:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.562+0000 7fe25a000640 1 -- 172.21.15.90:0/2011908109 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244010430 con 0x7fe25c11a8e0 2024-05-27T07:59:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.562+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25c1b6010 con 0x7fe25c11a8e0 2024-05-27T07:59:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.575+0000 7fe25a000640 1 -- 172.21.15.90:0/2011908109 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe2440046c0 con 0x7fe25c11a8e0 2024-05-27T07:59:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.575+0000 7fe26a42b740 1 Processor -- start 2024-05-27T07:59:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.576+0000 7fe26a42b740 1 -- start start 2024-05-27T07:59:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.578+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e41860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.578+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a762f0 0x55ffc7e41da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.578+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.578+0000 7fe26a42b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7db2b90 con 0x55ffc7a762f0 2024-05-27T07:59:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.578+0000 7fe26a42b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7da2730 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.578+0000 7fe26a42b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7da6770 con 0x55ffc7a7e4d0 2024-05-27T07:59:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.578+0000 7fe25a000640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2400b29d0 0x7fe2400b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.579+0000 7fe25a000640 1 -- 172.21.15.90:0/2011908109 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9584+0+0 (secure 0 0 0) 0x7fe2440e2d20 con 0x7fe25c11a8e0 2024-05-27T07:59:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.581+0000 7fe261400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e41860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.582+0000 7fe261400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e41860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48666/0 (socket says 172.21.15.90:48666) 2024-05-27T07:59:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.582+0000 7fe261400640 1 -- 172.21.15.90:0/3357530301 learned_addr learned my addr 172.21.15.90:0/3357530301 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.582+0000 7fe261400640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2400b29d0 0x7fe2400b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.582+0000 7fe261400640 1 -- 172.21.15.90:0/3357530301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a7e4d0 msgr2=0x55ffc7e422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.581+0000 7fe260a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a762f0 0x55ffc7e41da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.581+0000 7fe253e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.583+0000 7fe261400640 1 --2- 172.21.15.90:0/3357530301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.583+0000 7fe261400640 1 -- 172.21.15.90:0/3357530301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a762f0 msgr2=0x55ffc7e41da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.583+0000 7fe261400640 1 --2- 172.21.15.90:0/3357530301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a762f0 0x55ffc7e41da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.583+0000 7fe261400640 1 -- 172.21.15.90:0/3357530301 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc7d3b8b0 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.583+0000 7fe260a00640 1 --2- 172.21.15.90:0/3357530301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a762f0 0x55ffc7e41da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.583+0000 7fe253e00640 1 --2- 172.21.15.90:0/3357530301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.583+0000 7fe261400640 1 --2- 172.21.15.90:0/3357530301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e41860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe25402a9e0 tx=0x7fe254030b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd87303616c2f4eb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.584+0000 7fe251c00640 1 -- 172.21.15.90:0/3357530301 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2540041b0 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.584+0000 7fe251c00640 1 -- 172.21.15.90:0/3357530301 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe254004350 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.585+0000 7fe26a42b740 1 -- 172.21.15.90:0/3357530301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 msgr2=0x55ffc7e41860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.585+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3357530301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e41860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe25402a9e0 tx=0x7fe254030b40 comp rx=0 tx=0).stop 2024-05-27T07:59:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.586+0000 7fe261400640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2400b29d0 0x7fe2400b4ee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe254005dc0 tx=0x7fe25404b000 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.588+0000 7fe26a42b740 1 -- 172.21.15.90:0/3357530301 shutdown_connections 2024-05-27T07:59:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.588+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3357530301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.588+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3357530301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a762f0 0x55ffc7e41da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.588+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3357530301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e41860 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.588+0000 7fe26a42b740 1 -- 172.21.15.90:0/3357530301 >> 172.21.15.90:0/3357530301 conn(0x55ffc7a93300 msgr2=0x55ffc7e3e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:09.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.590+0000 7fe26a42b740 1 -- 172.21.15.90:0/3357530301 shutdown_connections 2024-05-27T07:59:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.590+0000 7fe26a42b740 1 -- 172.21.15.90:0/3357530301 wait complete. 2024-05-27T07:59:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.590+0000 7fe26a42b740 1 -- start start 2024-05-27T07:59:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.591+0000 7fe26a42b740 1 Processor -- start 2024-05-27T07:59:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.593+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.593+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e4bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.593+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e57d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.593+0000 7fe260a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.593+0000 7fe260a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:58964/0 (socket says 172.21.15.90:58964) 2024-05-27T07:59:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.594+0000 7fe253e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e4bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.594+0000 7fe253e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e4bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48672/0 (socket says 172.21.15.90:48672) 2024-05-27T07:59:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.595+0000 7fe26a42b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7a88790 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.595+0000 7fe26a42b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7d4abc0 con 0x55ffc7a7e4d0 2024-05-27T07:59:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.595+0000 7fe26a42b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ffc7e4e9e0 con 0x55ffc7a762f0 2024-05-27T07:59:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.595+0000 7fe26a42b740 10 client.0 _ll_register_callbacks cb 0x55ffc7d28400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.595+0000 7fe260a00640 1 -- 172.21.15.90:0/281276076 learned_addr learned my addr 172.21.15.90:0/281276076 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.596+0000 7fe261400640 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e57d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.596+0000 7fe260a00640 1 -- 172.21.15.90:0/281276076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a7e4d0 msgr2=0x55ffc7e4bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.596+0000 7fe260a00640 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e4bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.596+0000 7fe260a00640 1 -- 172.21.15.90:0/281276076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a9f1b0 msgr2=0x55ffc7e57d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.597+0000 7fe260a00640 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e57d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.597+0000 7fe260a00640 1 -- 172.21.15.90:0/281276076 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc7ab7210 con 0x55ffc7a762f0 2024-05-27T07:59:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.597+0000 7fe261400640 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e57d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.597+0000 7fe260a00640 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4b980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe24c011930 tx=0x7fe24c011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a624ad30cedfbb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.597+0000 7fe24a000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:59:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.599+0000 7fe24a000640 1 -- 172.21.15.90:0/281276076 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c003d80 con 0x55ffc7a762f0 2024-05-27T07:59:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.599+0000 7fe24a000640 1 -- 172.21.15.90:0/281276076 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9584+0+0 (secure 0 0 0) 0x7fe24c00c910 con 0x55ffc7a762f0 2024-05-27T07:59:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.599+0000 7fe26a42b740 1 -- 172.21.15.90:0/281276076 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe254004350 con 0x55ffc7a762f0 2024-05-27T07:59:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.600+0000 7fe24a000640 1 -- 172.21.15.90:0/281276076 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe24c00cb10 con 0x55ffc7a762f0 2024-05-27T07:59:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.600+0000 7fe26a42b740 10 client.27373 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.600+0000 7fe26a42b740 1 -- 172.21.15.90:0/281276076 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ffc7b83620 con 0x55ffc7a762f0 2024-05-27T07:59:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.601+0000 7fe24a000640 1 -- 172.21.15.90:0/281276076 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe254004350 con 0x55ffc7a762f0 2024-05-27T07:59:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.601+0000 7fe24a000640 1 -- 172.21.15.90:0/281276076 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe24c01b470 con 0x55ffc7a762f0 2024-05-27T07:59:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.602+0000 7fe26a42b740 10 client.27373 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.602+0000 7fe26a42b740 10 client.27373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.602+0000 7fe26a42b740 20 client.27373 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.602+0000 7fe26a42b740 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ffc7a5dd80 0x55ffc7e5dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.602+0000 7fe26a42b740 4 client.27373 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.602+0000 7fe26a42b740 1 -- 172.21.15.90:0/281276076 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffc7db4ae0 con 0x55ffc7a5dd80 2024-05-27T07:59:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.603+0000 7fe253e00640 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ffc7a5dd80 0x55ffc7e5dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.604+0000 7fe253e00640 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ffc7a5dd80 0x55ffc7e5dd80 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0f0c5d59b03aa5 server_cookie=9463ac09e5db3ca5 in_seq=0 out_seq=0 2024-05-27T07:59:09.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.605+0000 7fe24a000640 10 client.27373 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.605+0000 7fe24a000640 1 -- 172.21.15.90:0/281276076 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ffc7db4ae0 con 0x55ffc7a5dd80 2024-05-27T07:59:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.605+0000 7fe24a000640 10 client.27373 handle_command_reply: tid=0 2024-05-27T07:59:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.606+0000 7fe26a42b740 1 client.27373 shutdown 2024-05-27T07:59:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.606+0000 7fe26a42b740 1 -- 172.21.15.90:0/281276076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 msgr2=0x55ffc7e4b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.606+0000 7fe26a42b740 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4b980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe24c011930 tx=0x7fe24c011d30 comp rx=0 tx=0).stop 2024-05-27T07:59:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.606+0000 7fe26a42b740 1 -- 172.21.15.90:0/281276076 shutdown_connections 2024-05-27T07:59:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.606+0000 7fe26a42b740 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ffc7a5dd80 0x55ffc7e5dd80 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.606+0000 7fe26a42b740 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e4bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.606+0000 7fe26a42b740 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e57d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.606+0000 7fe26a42b740 1 --2- 172.21.15.90:0/281276076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4b980 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.606+0000 7fe26a42b740 1 -- 172.21.15.90:0/281276076 >> 172.21.15.90:0/281276076 conn(0x55ffc7a93300 msgr2=0x55ffc7e41690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.607+0000 7fe24a000640 0 client.27373 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.607+0000 7fe26a42b740 1 -- 172.21.15.90:0/281276076 shutdown_connections 2024-05-27T07:59:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.610+0000 7fe26a42b740 1 -- 172.21.15.90:0/281276076 wait complete. 2024-05-27T07:59:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.610+0000 7fe26a42b740 20 client.27373 trim_cache size 0 max 16384 2024-05-27T07:59:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.664+0000 7fe26a42b740 1 Processor -- start 2024-05-27T07:59:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.665+0000 7fe26a42b740 1 -- start start 2024-05-27T07:59:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.665+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.667+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e3d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.667+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.667+0000 7fe26a42b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1c1c70 con 0x55ffc7a7e4d0 2024-05-27T07:59:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.667+0000 7fe26a42b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1becc0 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.667+0000 7fe26a42b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1c0ed0 con 0x55ffc7a762f0 2024-05-27T07:59:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.667+0000 7fe261400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.667+0000 7fe261400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48678/0 (socket says 172.21.15.90:48678) 2024-05-27T07:59:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.667+0000 7fe261400640 1 -- 172.21.15.90:0/3645685976 learned_addr learned my addr 172.21.15.90:0/3645685976 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.668+0000 7fe260a00640 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.668+0000 7fe261400640 1 -- 172.21.15.90:0/3645685976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 msgr2=0x55ffc7e471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.669+0000 7fe253e00640 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e3d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.669+0000 7fe261400640 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.669+0000 7fe261400640 1 -- 172.21.15.90:0/3645685976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 msgr2=0x55ffc7e3d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.671+0000 7fe261400640 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e3d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.671+0000 7fe261400640 1 -- 172.21.15.90:0/3645685976 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc7e420c0 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.672+0000 7fe260a00640 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.672+0000 7fe261400640 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4b920 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe254005eb0 tx=0x7fe254030b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2185d8c72f6f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.672+0000 7fe253e00640 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e3d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.673+0000 7fe251200640 1 -- 172.21.15.90:0/3645685976 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2540046c0 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.673+0000 7fe251200640 1 -- 172.21.15.90:0/3645685976 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe254030550 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.673+0000 7fe251200640 1 -- 172.21.15.90:0/3645685976 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2540046c0 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.675+0000 7fe26a42b740 1 -- 172.21.15.90:0/3645685976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 msgr2=0x55ffc7e4b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.675+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4b920 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe254005eb0 tx=0x7fe254030b40 comp rx=0 tx=0).stop 2024-05-27T07:59:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.676+0000 7fe26a42b740 1 -- 172.21.15.90:0/3645685976 shutdown_connections 2024-05-27T07:59:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.676+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4b920 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.676+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e3d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.676+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3645685976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.676+0000 7fe26a42b740 1 -- 172.21.15.90:0/3645685976 >> 172.21.15.90:0/3645685976 conn(0x55ffc7a93300 msgr2=0x55ffc7e4b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.677+0000 7fe26a42b740 1 -- 172.21.15.90:0/3645685976 shutdown_connections 2024-05-27T07:59:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.677+0000 7fe26a42b740 1 -- 172.21.15.90:0/3645685976 wait complete. 2024-05-27T07:59:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.679+0000 7fe26a42b740 1 -- start start 2024-05-27T07:59:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.679+0000 7fe26a42b740 1 Processor -- start 2024-05-27T07:59:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.681+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.682+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e4f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.682+0000 7fe26a42b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.683+0000 7fe26a42b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1c0800 con 0x55ffc7a7e4d0 2024-05-27T07:59:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.683+0000 7fe26a42b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1bf390 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.683+0000 7fe26a42b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c1bfa60 con 0x55ffc7a762f0 2024-05-27T07:59:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.683+0000 7fe26a42b740 10 client.0 _ll_register_callbacks cb 0x7fe24c009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.683+0000 7fe261400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.683+0000 7fe261400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48688/0 (socket says 172.21.15.90:48688) 2024-05-27T07:59:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.683+0000 7fe261400640 1 -- 172.21.15.90:0/3248285942 learned_addr learned my addr 172.21.15.90:0/3248285942 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.683+0000 7fe260a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.683+0000 7fe261400640 1 -- 172.21.15.90:0/3248285942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 msgr2=0x55ffc7e4ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.683+0000 7fe261400640 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.684+0000 7fe261400640 1 -- 172.21.15.90:0/3248285942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 msgr2=0x55ffc7e4f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.684+0000 7fe253e00640 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e4f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.684+0000 7fe261400640 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e4f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.684+0000 7fe261400640 1 -- 172.21.15.90:0/3248285942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc7ec4530 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.684+0000 7fe260a00640 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.684+0000 7fe253e00640 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e4f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.685+0000 7fe24a000640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:59:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.685+0000 7fe261400640 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4f700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe254009250 tx=0x7fe254005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=239a11b7a19aad5c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.685+0000 7fe24a000640 1 -- 172.21.15.90:0/3248285942 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2540041b0 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.686+0000 7fe24a000640 1 -- 172.21.15.90:0/3248285942 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9584+0+0 (secure 0 0 0) 0x7fe254064070 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.686+0000 7fe26a42b740 1 -- 172.21.15.90:0/3248285942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe254030550 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.687+0000 7fe24a000640 1 -- 172.21.15.90:0/3248285942 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe2540041b0 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.687+0000 7fe26a42b740 10 client.37289 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.687+0000 7fe26a42b740 1 -- 172.21.15.90:0/3248285942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ffc7ec5680 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.688+0000 7fe24a000640 1 -- 172.21.15.90:0/3248285942 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe254030550 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.688+0000 7fe24a000640 1 -- 172.21.15.90:0/3248285942 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe25405f460 con 0x55ffc7a9f1b0 2024-05-27T07:59:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.688+0000 7fe26a42b740 10 client.37289 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.688+0000 7fe26a42b740 10 client.37289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.688+0000 7fe26a42b740 20 client.37289 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.688+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ffc7a331c0 0x55ffc7ec7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.688+0000 7fe26a42b740 4 client.37289 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:59:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.688+0000 7fe26a42b740 1 -- 172.21.15.90:0/3248285942 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ffc7dae2c0 con 0x55ffc7a331c0 2024-05-27T07:59:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.689+0000 7fe260a00640 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ffc7a331c0 0x55ffc7ec7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.690+0000 7fe24a000640 10 client.37289 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.690+0000 7fe260a00640 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ffc7a331c0 0x55ffc7ec7b60 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa695e3c4de6565 server_cookie=3ddc195c829a6cfd in_seq=0 out_seq=0 2024-05-27T07:59:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.864+0000 7fe24a000640 1 -- 172.21.15.90:0/3248285942 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ffc7dae2c0 con 0x55ffc7a331c0 2024-05-27T07:59:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.864+0000 7fe24a000640 10 client.37289 handle_command_reply: tid=0 2024-05-27T07:59:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.864+0000 7fe26a42b740 1 client.37289 shutdown 2024-05-27T07:59:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.864+0000 7fe26a42b740 1 -- 172.21.15.90:0/3248285942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 msgr2=0x55ffc7e4f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.864+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4f700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe254009250 tx=0x7fe254005270 comp rx=0 tx=0).stop 2024-05-27T07:59:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.865+0000 7fe26a42b740 1 -- 172.21.15.90:0/3248285942 shutdown_connections 2024-05-27T07:59:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.865+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ffc7a331c0 0x55ffc7ec7b60 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.865+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ffc7a9f1b0 0x55ffc7e4f700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.865+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ffc7a7e4d0 0x55ffc7e4f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.865+0000 7fe26a42b740 1 --2- 172.21.15.90:0/3248285942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffc7a762f0 0x55ffc7e4ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.865+0000 7fe26a42b740 1 -- 172.21.15.90:0/3248285942 >> 172.21.15.90:0/3248285942 conn(0x55ffc7a93300 msgr2=0x55ffc7e4bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.865+0000 7fe24a000640 0 client.37289 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.866+0000 7fe26a42b740 1 -- 172.21.15.90:0/3248285942 shutdown_connections 2024-05-27T07:59:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.866+0000 7fe26a42b740 1 -- 172.21.15.90:0/3248285942 wait complete. 2024-05-27T07:59:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.866+0000 7fe26a42b740 20 client.37289 trim_cache size 0 max 16384 2024-05-27T07:59:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:59:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:59:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:59:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.870+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2400b29d0 msgr2=0x7fe2400b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.870+0000 7fe263400640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2400b29d0 0x7fe2400b4ee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe254005dc0 tx=0x7fe25404b000 comp rx=0 tx=0).stop 2024-05-27T07:59:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.870+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25c11a8e0 msgr2=0x7fe25c1a9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.870+0000 7fe263400640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25c11a8e0 0x7fe25c1a9aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe24400e9c0 tx=0x7fe24400ee80 comp rx=0 tx=0).stop 2024-05-27T07:59:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.870+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 shutdown_connections 2024-05-27T07:59:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.870+0000 7fe263400640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe2400b29d0 0x7fe2400b4ee0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.870+0000 7fe263400640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe25c11a8e0 0x7fe25c1a9aa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.870+0000 7fe263400640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe25c11d240 0x7fe25c1add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.870+0000 7fe263400640 1 --2- 172.21.15.90:0/2011908109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe25c10a1f0 0x7fe25c1a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.871+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 >> 172.21.15.90:0/2011908109 conn(0x7fe25c06e8f0 msgr2=0x7fe25c11acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.874+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 shutdown_connections 2024-05-27T07:59:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:09.874+0000 7fe263400640 1 -- 172.21.15.90:0/2011908109 wait complete. 2024-05-27T07:59:10.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:09 smithi129 ceph-mon[35254]: osdmap e113: 12 total, 12 up, 12 in 2024-05-27T07:59:10.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:09 smithi090 ceph-mon[31429]: osdmap e113: 12 total, 12 up, 12 in 2024-05-27T07:59:10.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:09 smithi136 ceph-mon[35324]: osdmap e113: 12 total, 12 up, 12 in 2024-05-27T07:59:11.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:10 smithi129 ceph-mon[35254]: pgmap v407: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.9 MiB/s wr, 440 op/s 2024-05-27T07:59:11.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:10 smithi090 ceph-mon[31429]: pgmap v407: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.9 MiB/s wr, 440 op/s 2024-05-27T07:59:11.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:10 smithi136 ceph-mon[35324]: pgmap v407: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.9 MiB/s wr, 440 op/s 2024-05-27T07:59:13.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:12 smithi090 ceph-mon[31429]: pgmap v408: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 436 op/s 2024-05-27T07:59:13.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:12 smithi136 ceph-mon[35324]: pgmap v408: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 436 op/s 2024-05-27T07:59:13.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:12 smithi129 ceph-mon[35254]: pgmap v408: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.9 MiB/s wr, 436 op/s 2024-05-27T07:59:13.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 535.0 s, 380.0 tps, lat 201.779 ms stddev 130.330 2024-05-27T07:59:15.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.154+0000 7fd3fea00700 1 -- 172.21.15.90:0/2495696976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3f8103a20 msgr2=0x7fd3f8110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:15.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.154+0000 7fd3fea00700 1 --2- 172.21.15.90:0/2495696976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3f8103a20 0x7fd3f8110740 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009990 tx=0x7fd3e4009ca0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:14 smithi090 ceph-mon[31429]: pgmap v409: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 481 op/s 2024-05-27T07:59:15.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.154+0000 7fd3fea00700 1 -- 172.21.15.90:0/2495696976 shutdown_connections 2024-05-27T07:59:15.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.154+0000 7fd3fea00700 1 --2- 172.21.15.90:0/2495696976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3f8110c80 0x7fd3f81130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.154+0000 7fd3fea00700 1 --2- 172.21.15.90:0/2495696976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3f81030d0 0x7fd3f81034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.154+0000 7fd3fea00700 1 --2- 172.21.15.90:0/2495696976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3f8103a20 0x7fd3f8110740 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.154+0000 7fd3fea00700 1 -- 172.21.15.90:0/2495696976 >> 172.21.15.90:0/2495696976 conn(0x7fd3f80fedb0 msgr2=0x7fd3f81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:15.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.155+0000 7fd3fea00700 1 -- 172.21.15.90:0/2495696976 shutdown_connections 2024-05-27T07:59:15.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.155+0000 7fd3fea00700 1 -- 172.21.15.90:0/2495696976 wait complete. 2024-05-27T07:59:15.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.156+0000 7fd3fea00700 1 Processor -- start 2024-05-27T07:59:15.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.156+0000 7fd3fea00700 1 -- start start 2024-05-27T07:59:15.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.156+0000 7fd3fea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3f81030d0 0x7fd3f819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:15.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.157+0000 7fd3fea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3f8103a20 0x7fd3f819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:15.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.157+0000 7fd3fea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3f8110c80 0x7fd3f81a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:15.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.157+0000 7fd3fea00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f8116540 con 0x7fd3f8110c80 2024-05-27T07:59:15.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.157+0000 7fd3fea00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f81163c0 con 0x7fd3f8103a20 2024-05-27T07:59:15.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.157+0000 7fd3fea00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f81166c0 con 0x7fd3f81030d0 2024-05-27T07:59:15.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.157+0000 7fd3fca00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3f8110c80 0x7fd3f81a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:15.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.157+0000 7fd3f7400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3f8103a20 0x7fd3f819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:15.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.157+0000 7fd3f7400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3f8103a20 0x7fd3f819d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48700/0 (socket says 172.21.15.90:48700) 2024-05-27T07:59:15.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.157+0000 7fd3f7400700 1 -- 172.21.15.90:0/2697003928 learned_addr learned my addr 172.21.15.90:0/2697003928 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:15.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.157+0000 7fd3f7e00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3f81030d0 0x7fd3f819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:15.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.158+0000 7fd3fca00700 1 -- 172.21.15.90:0/2697003928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3f81030d0 msgr2=0x7fd3f819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:15.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.158+0000 7fd3fca00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3f81030d0 0x7fd3f819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.158+0000 7fd3fca00700 1 -- 172.21.15.90:0/2697003928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3f8103a20 msgr2=0x7fd3f819d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:15.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.158+0000 7fd3fca00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3f8103a20 0x7fd3f819d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.158+0000 7fd3fca00700 1 -- 172.21.15.90:0/2697003928 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f81a1d90 con 0x7fd3f8110c80 2024-05-27T07:59:15.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.158+0000 7fd3f7400700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3f8103a20 0x7fd3f819d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:15.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.159+0000 7fd3fca00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3f8110c80 0x7fd3f81a17a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0007f10 tx=0x7fd3f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:15.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.159+0000 7fd3f4c00700 1 -- 172.21.15.90:0/2697003928 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0004db0 con 0x7fd3f8110c80 2024-05-27T07:59:15.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.159+0000 7fd3f4c00700 1 -- 172.21.15.90:0/2697003928 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd3f0009e20 con 0x7fd3f8110c80 2024-05-27T07:59:15.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.159+0000 7fd3f4c00700 1 -- 172.21.15.90:0/2697003928 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0015400 con 0x7fd3f8110c80 2024-05-27T07:59:15.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.159+0000 7fd3fea00700 1 -- 172.21.15.90:0/2697003928 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f81a2080 con 0x7fd3f8110c80 2024-05-27T07:59:15.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.160+0000 7fd3fea00700 1 -- 172.21.15.90:0/2697003928 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f81a2570 con 0x7fd3f8110c80 2024-05-27T07:59:15.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.161+0000 7fd3fea00700 1 -- 172.21.15.90:0/2697003928 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f80512b0 con 0x7fd3f8110c80 2024-05-27T07:59:15.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.165+0000 7fd3f4c00700 1 -- 172.21.15.90:0/2697003928 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd3f0005240 con 0x7fd3f8110c80 2024-05-27T07:59:15.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.167+0000 7fd3f4c00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3dc0b2b40 0x7fd3dc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:15.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.167+0000 7fd3f4c00700 1 -- 172.21.15.90:0/2697003928 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9584+0+0 (secure 0 0 0) 0x7fd3f00e2ac0 con 0x7fd3f8110c80 2024-05-27T07:59:15.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.168+0000 7fd3f7e00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3dc0b2b40 0x7fd3dc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:15.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.169+0000 7fd3f7e00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3dc0b2b40 0x7fd3dc0b5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800ac50 tx=0x7fd3e800a380 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:15.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.172+0000 7fd3f4c00700 1 -- 172.21.15.90:0/2697003928 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3f0010070 con 0x7fd3f8110c80 2024-05-27T07:59:15.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:14 smithi136 ceph-mon[35324]: pgmap v409: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 481 op/s 2024-05-27T07:59:15.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:14 smithi129 ceph-mon[35254]: pgmap v409: 129 pgs: 4 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.8 MiB/s wr, 481 op/s 2024-05-27T07:59:15.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.615+0000 7fd3fea00700 1 -- 172.21.15.90:0/2697003928 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3f819e2d0 con 0x7fd3f8110c80 2024-05-27T07:59:15.619 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T07:59:15.619 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":64,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:56:32.161929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T07:59:15.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.617+0000 7fd3f4c00700 1 -- 172.21.15.90:0/2697003928 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+9456 (secure 0 0 0) 0x7fd3f00aa0c0 con 0x7fd3f8110c80 2024-05-27T07:59:15.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.621+0000 7fd3fea00700 1 -- 172.21.15.90:0/2697003928 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3dc0b2b40 msgr2=0x7fd3dc0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.621+0000 7fd3fea00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3dc0b2b40 0x7fd3dc0b5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800ac50 tx=0x7fd3e800a380 comp rx=0 tx=0).stop 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.621+0000 7fd3fea00700 1 -- 172.21.15.90:0/2697003928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3f8110c80 msgr2=0x7fd3f81a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.621+0000 7fd3fea00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3f8110c80 0x7fd3f81a17a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0007f10 tx=0x7fd3f000d3b0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.622+0000 7fd3fea00700 1 -- 172.21.15.90:0/2697003928 shutdown_connections 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.622+0000 7fd3fea00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3f8110c80 0x7fd3f81a17a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.622+0000 7fd3fea00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3f81030d0 0x7fd3f819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.622+0000 7fd3fea00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd3dc0b2b40 0x7fd3dc0b5050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.622+0000 7fd3fea00700 1 --2- 172.21.15.90:0/2697003928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd3f8103a20 0x7fd3f819d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.622+0000 7fd3fea00700 1 -- 172.21.15.90:0/2697003928 >> 172.21.15.90:0/2697003928 conn(0x7fd3f80fedb0 msgr2=0x7fd3f8101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.622+0000 7fd3fea00700 1 -- 172.21.15.90:0/2697003928 shutdown_connections 2024-05-27T07:59:15.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:15.622+0000 7fd3fea00700 1 -- 172.21.15.90:0/2697003928 wait complete. 2024-05-27T07:59:15.626 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 64 2024-05-27T07:59:15.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:15 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2697003928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:59:16.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:15 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2697003928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:59:16.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:15 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2697003928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T07:59:16.674 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T07:59:16.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:16 smithi090 ceph-mon[31429]: pgmap v410: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.5 MiB/s wr, 575 op/s 2024-05-27T07:59:17.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:16 smithi136 ceph-mon[35324]: pgmap v410: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.5 MiB/s wr, 575 op/s 2024-05-27T07:59:17.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:16 smithi129 ceph-mon[35254]: pgmap v410: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.5 MiB/s wr, 575 op/s 2024-05-27T07:59:18.239 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:18.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:18 smithi090 ceph-mon[31429]: osdmap e114: 12 total, 12 up, 12 in 2024-05-27T07:59:18.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:18 smithi090 ceph-mon[31429]: pgmap v412: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 435 op/s 2024-05-27T07:59:18.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:18 smithi129 ceph-mon[35254]: osdmap e114: 12 total, 12 up, 12 in 2024-05-27T07:59:18.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:18 smithi129 ceph-mon[35254]: pgmap v412: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 435 op/s 2024-05-27T07:59:18.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:18 smithi136 ceph-mon[35324]: osdmap e114: 12 total, 12 up, 12 in 2024-05-27T07:59:18.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:18 smithi136 ceph-mon[35324]: pgmap v412: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 MiB/s wr, 435 op/s 2024-05-27T07:59:18.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 540.0 s, 682.3 tps, lat 180.325 ms stddev 261.629 2024-05-27T07:59:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:20 smithi090 ceph-mon[31429]: pgmap v413: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 522 op/s 2024-05-27T07:59:21.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:20 smithi136 ceph-mon[35324]: pgmap v413: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 522 op/s 2024-05-27T07:59:21.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:20 smithi129 ceph-mon[35254]: pgmap v413: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.7 MiB/s wr, 522 op/s 2024-05-27T07:59:21.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.742+0000 7fe375c00700 1 -- 172.21.15.90:0/3259909659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe370109860 msgr2=0x7fe3701111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:21.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.742+0000 7fe375c00700 1 --2- 172.21.15.90:0/3259909659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe370109860 0x7fe3701111c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe36800b0a0 tx=0x7fe36800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:59:21.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.743+0000 7fe375c00700 1 -- 172.21.15.90:0/3259909659 shutdown_connections 2024-05-27T07:59:21.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.743+0000 7fe375c00700 1 --2- 172.21.15.90:0/3259909659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe370109860 0x7fe3701111c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:21.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.743+0000 7fe375c00700 1 --2- 172.21.15.90:0/3259909659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3701045e0 0x7fe3701049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:21.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.743+0000 7fe375c00700 1 --2- 172.21.15.90:0/3259909659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe370108f10 0x7fe370109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:21.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.743+0000 7fe375c00700 1 -- 172.21.15.90:0/3259909659 >> 172.21.15.90:0/3259909659 conn(0x7fe370100030 msgr2=0x7fe370102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:21.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.743+0000 7fe375c00700 1 -- 172.21.15.90:0/3259909659 shutdown_connections 2024-05-27T07:59:21.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.743+0000 7fe375c00700 1 -- 172.21.15.90:0/3259909659 wait complete. 2024-05-27T07:59:21.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.745+0000 7fe375c00700 1 Processor -- start 2024-05-27T07:59:21.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.746+0000 7fe375c00700 1 -- start start 2024-05-27T07:59:21.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.747+0000 7fe375c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3701045e0 0x7fe3700753b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:21.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.747+0000 7fe375c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe370108f10 0x7fe37006e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:21.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.747+0000 7fe375c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe370109860 0x7fe37006eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:21.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.748+0000 7fe375c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe370114390 con 0x7fe3701045e0 2024-05-27T07:59:21.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.748+0000 7fe375c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe370114210 con 0x7fe370108f10 2024-05-27T07:59:21.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.748+0000 7fe375c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe370114510 con 0x7fe370109860 2024-05-27T07:59:21.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.748+0000 7fe36e000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe370108f10 0x7fe37006e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:21.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.748+0000 7fe36e000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe370108f10 0x7fe37006e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47142/0 (socket says 172.21.15.90:47142) 2024-05-27T07:59:21.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.748+0000 7fe36e000700 1 -- 172.21.15.90:0/205905231 learned_addr learned my addr 172.21.15.90:0/205905231 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:21.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.748+0000 7fe36ea00700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3701045e0 0x7fe3700753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:21.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.749+0000 7fe36ea00700 1 -- 172.21.15.90:0/205905231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe370109860 msgr2=0x7fe37006eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:21.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.749+0000 7fe36f400700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe370109860 0x7fe37006eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:21.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.750+0000 7fe36ea00700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe370109860 0x7fe37006eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:21.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.750+0000 7fe36ea00700 1 -- 172.21.15.90:0/205905231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe370108f10 msgr2=0x7fe37006e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:21.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.750+0000 7fe36ea00700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe370108f10 0x7fe37006e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:21.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.750+0000 7fe36ea00700 1 -- 172.21.15.90:0/205905231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe370072ee0 con 0x7fe3701045e0 2024-05-27T07:59:21.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.750+0000 7fe36f400700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe370109860 0x7fe37006eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:21.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.750+0000 7fe36ea00700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3701045e0 0x7fe3700753b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe36000e9f0 tx=0x7fe36000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:21.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.751+0000 7fe367400700 1 -- 172.21.15.90:0/205905231 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36000cbd0 con 0x7fe3701045e0 2024-05-27T07:59:21.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.751+0000 7fe375c00700 1 -- 172.21.15.90:0/205905231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3700731d0 con 0x7fe3701045e0 2024-05-27T07:59:21.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.751+0000 7fe375c00700 1 -- 172.21.15.90:0/205905231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe370073760 con 0x7fe3701045e0 2024-05-27T07:59:21.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.752+0000 7fe367400700 1 -- 172.21.15.90:0/205905231 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe36000cd70 con 0x7fe3701045e0 2024-05-27T07:59:21.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.752+0000 7fe367400700 1 -- 172.21.15.90:0/205905231 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360026060 con 0x7fe3701045e0 2024-05-27T07:59:21.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.763+0000 7fe367400700 1 -- 172.21.15.90:0/205905231 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe360018420 con 0x7fe3701045e0 2024-05-27T07:59:21.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.765+0000 7fe367400700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3540b2c70 0x7fe3540b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:21.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.765+0000 7fe367400700 1 -- 172.21.15.90:0/205905231 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fe3600e3700 con 0x7fe3701045e0 2024-05-27T07:59:21.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:21.766+0000 7fe36e000700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3540b2c70 0x7fe3540b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.210+0000 7fe36e000700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3540b2c70 0x7fe3540b5180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe370107a60 tx=0x7fe35c009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:22.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.213+0000 7fe384745b80 1 Processor -- start 2024-05-27T07:59:22.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.213+0000 7fe384745b80 1 -- start start 2024-05-27T07:59:22.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.217+0000 7fe384745b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac408be40 0x55bac42107f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.217+0000 7fe384745b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4210d30 0x55bac420a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.218+0000 7fe384745b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 0x55bac420b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.218+0000 7fe384745b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bac4138e40 con 0x55bac4210d30 2024-05-27T07:59:22.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.218+0000 7fe36ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac408be40 0x55bac42107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.218+0000 7fe36ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac408be40 0x55bac42107f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:57844/0 (socket says 172.21.15.90:57844) 2024-05-27T07:59:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.218+0000 7fe36ea00700 1 -- 172.21.15.90:0/4278824799 learned_addr learned my addr 172.21.15.90:0/4278824799 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.218+0000 7fe36e000700 1 --2- 172.21.15.90:0/4278824799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4210d30 0x55bac420a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.218+0000 7fe36f400700 1 --2- 172.21.15.90:0/4278824799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 0x55bac420b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.219+0000 7fe384745b80 1 -- 172.21.15.90:0/4278824799 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bac41289e0 con 0x55bac420adb0 2024-05-27T07:59:22.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.219+0000 7fe384745b80 1 -- 172.21.15.90:0/4278824799 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bac412ca20 con 0x55bac408be40 2024-05-27T07:59:22.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.220+0000 7fe36ea00700 1 -- 172.21.15.90:0/4278824799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 msgr2=0x55bac420b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:22.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.220+0000 7fe36ea00700 1 --2- 172.21.15.90:0/4278824799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 0x55bac420b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.220+0000 7fe36ea00700 1 -- 172.21.15.90:0/4278824799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4210d30 msgr2=0x55bac420a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:22.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.220+0000 7fe36ea00700 1 --2- 172.21.15.90:0/4278824799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4210d30 0x55bac420a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.220+0000 7fe36ea00700 1 -- 172.21.15.90:0/4278824799 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac40f1420 con 0x55bac408be40 2024-05-27T07:59:22.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.224+0000 7fe36ea00700 1 --2- 172.21.15.90:0/4278824799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac408be40 0x55bac42107f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c5b0 tx=0x7fe3600a9df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f255d2d2f7b4b4b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:22.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.224+0000 7fe35be00700 1 -- 172.21.15.90:0/4278824799 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3600e2070 con 0x55bac408be40 2024-05-27T07:59:22.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.224+0000 7fe35be00700 1 -- 172.21.15.90:0/4278824799 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3600aacb0 con 0x55bac408be40 2024-05-27T07:59:22.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.224+0000 7fe35be00700 1 -- 172.21.15.90:0/4278824799 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3600e3270 con 0x55bac408be40 2024-05-27T07:59:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.226+0000 7fe384745b80 1 -- 172.21.15.90:0/4278824799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac408be40 msgr2=0x55bac42107f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.226+0000 7fe384745b80 1 --2- 172.21.15.90:0/4278824799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac408be40 0x55bac42107f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c5b0 tx=0x7fe3600a9df0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.226+0000 7fe384745b80 1 -- 172.21.15.90:0/4278824799 shutdown_connections 2024-05-27T07:59:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.226+0000 7fe384745b80 1 --2- 172.21.15.90:0/4278824799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 0x55bac420b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.226+0000 7fe384745b80 1 --2- 172.21.15.90:0/4278824799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac408be40 0x55bac42107f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.226+0000 7fe384745b80 1 --2- 172.21.15.90:0/4278824799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac4210d30 0x55bac420a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.226+0000 7fe384745b80 1 -- 172.21.15.90:0/4278824799 >> 172.21.15.90:0/4278824799 conn(0x55bac40ebfc0 msgr2=0x55bac42058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:22.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.226+0000 7fe384745b80 1 -- 172.21.15.90:0/4278824799 shutdown_connections 2024-05-27T07:59:22.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.226+0000 7fe384745b80 1 -- 172.21.15.90:0/4278824799 wait complete. 2024-05-27T07:59:22.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.227+0000 7fe384745b80 1 -- start start 2024-05-27T07:59:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.228+0000 7fe384745b80 1 Processor -- start 2024-05-27T07:59:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.228+0000 7fe384745b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac408be40 0x55bac42144e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.228+0000 7fe384745b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 0x55bac4214a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.228+0000 7fe384745b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac4214f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.228+0000 7fe384745b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bac402efa0 con 0x55bac420adb0 2024-05-27T07:59:22.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.228+0000 7fe384745b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bac40c9760 con 0x55bac408be40 2024-05-27T07:59:22.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.228+0000 7fe384745b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bac4217800 con 0x55bac4210d30 2024-05-27T07:59:22.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.228+0000 7fe384745b80 10 client.0 _ll_register_callbacks cb 0x55bac40d2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.229+0000 7fe36ea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac408be40 0x55bac42144e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.229+0000 7fe36ea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac408be40 0x55bac42144e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47162/0 (socket says 172.21.15.90:47162) 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.229+0000 7fe36ea00700 1 -- 172.21.15.90:0/3907670154 learned_addr learned my addr 172.21.15.90:0/3907670154 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.229+0000 7fe36f400700 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac4214f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.230+0000 7fe36ea00700 1 -- 172.21.15.90:0/3907670154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 msgr2=0x55bac4214f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.230+0000 7fe36ea00700 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac4214f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.230+0000 7fe36ea00700 1 -- 172.21.15.90:0/3907670154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 msgr2=0x55bac4214a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.230+0000 7fe36ea00700 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 0x55bac4214a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.230+0000 7fe36ea00700 1 -- 172.21.15.90:0/3907670154 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac40788d0 con 0x55bac408be40 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.230+0000 7fe36ea00700 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac408be40 0x55bac42144e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe36000e9c0 tx=0x7fe3600e3520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70be996c9f9c9ac1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.230+0000 7fe358c00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T07:59:22.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.230+0000 7fe358c00700 1 -- 172.21.15.90:0/3907670154 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3600e2070 con 0x55bac408be40 2024-05-27T07:59:22.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.231+0000 7fe358c00700 1 -- 172.21.15.90:0/3907670154 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fe3600a6ba0 con 0x55bac408be40 2024-05-27T07:59:22.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.231+0000 7fe384745b80 1 -- 172.21.15.90:0/3907670154 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3600aacb0 con 0x55bac408be40 2024-05-27T07:59:22.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.231+0000 7fe358c00700 1 -- 172.21.15.90:0/3907670154 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe3600ae960 con 0x55bac408be40 2024-05-27T07:59:22.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.231+0000 7fe384745b80 10 client.37304 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:22.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.231+0000 7fe384745b80 1 -- 172.21.15.90:0/3907670154 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55bac40f9060 con 0x55bac408be40 2024-05-27T07:59:22.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.232+0000 7fe358c00700 1 -- 172.21.15.90:0/3907670154 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3600aacb0 con 0x55bac408be40 2024-05-27T07:59:22.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.232+0000 7fe358c00700 1 -- 172.21.15.90:0/3907670154 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe3600b1ce0 con 0x55bac408be40 2024-05-27T07:59:22.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.233+0000 7fe384745b80 10 client.37304 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:22.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.233+0000 7fe384745b80 10 client.37304 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:59:22.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.233+0000 7fe384745b80 20 client.37304 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:22.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.233+0000 7fe384745b80 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55bac422cda0 0x55bac422f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.233+0000 7fe384745b80 4 client.37304 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:22.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.233+0000 7fe384745b80 1 -- 172.21.15.90:0/3907670154 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bac413ad90 con 0x55bac422cda0 2024-05-27T07:59:22.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.234+0000 7fe36e000700 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55bac422cda0 0x55bac422f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.235+0000 7fe36e000700 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55bac422cda0 0x55bac422f1c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7638d3ff40eac7de server_cookie=1d9a750a988ad633 in_seq=0 out_seq=0 2024-05-27T07:59:22.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.235+0000 7fe358c00700 10 client.37304 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:59:22.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.235+0000 7fe358c00700 1 -- 172.21.15.90:0/3907670154 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bac413ad90 con 0x55bac422cda0 2024-05-27T07:59:22.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.235+0000 7fe358c00700 10 client.37304 handle_command_reply: tid=0 2024-05-27T07:59:22.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 client.37304 shutdown 2024-05-27T07:59:22.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 -- 172.21.15.90:0/3907670154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac408be40 msgr2=0x55bac42144e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:22.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac408be40 0x55bac42144e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe36000e9c0 tx=0x7fe3600e3520 comp rx=0 tx=0).stop 2024-05-27T07:59:22.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 -- 172.21.15.90:0/3907670154 shutdown_connections 2024-05-27T07:59:22.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55bac422cda0 0x55bac422f1c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac4214f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac408be40 0x55bac42144e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 --2- 172.21.15.90:0/3907670154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 0x55bac4214a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 -- 172.21.15.90:0/3907670154 >> 172.21.15.90:0/3907670154 conn(0x55bac40ebfc0 msgr2=0x55bac4204b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:22.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe358c00700 0 client.37304 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:59:22.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 -- 172.21.15.90:0/3907670154 shutdown_connections 2024-05-27T07:59:22.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.236+0000 7fe384745b80 1 -- 172.21.15.90:0/3907670154 wait complete. 2024-05-27T07:59:22.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.237+0000 7fe384745b80 20 client.37304 trim_cache size 0 max 16384 2024-05-27T07:59:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.261+0000 7fe384745b80 1 Processor -- start 2024-05-27T07:59:22.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.261+0000 7fe384745b80 1 -- start start 2024-05-27T07:59:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.261+0000 7fe384745b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 0x55bac420a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.261+0000 7fe384745b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac4213800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.261+0000 7fe384745b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420fcf0 0x55bac4216840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.261+0000 7fe384745b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe3500107c0 con 0x55bac420fcf0 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.261+0000 7fe384745b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe35000d810 con 0x55bac420adb0 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.261+0000 7fe384745b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe35000fa20 con 0x55bac4210d30 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.261+0000 7fe36f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420fcf0 0x55bac4216840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.261+0000 7fe36ea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 0x55bac420a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.262+0000 7fe36f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420fcf0 0x55bac4216840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57382/0 (socket says 172.21.15.90:57382) 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.262+0000 7fe36ea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 0x55bac420a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47166/0 (socket says 172.21.15.90:47166) 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.262+0000 7fe36f400700 1 -- 172.21.15.90:0/3603093684 learned_addr learned my addr 172.21.15.90:0/3603093684 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.262+0000 7fe36e000700 1 --2- 172.21.15.90:0/3603093684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac4213800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.262+0000 7fe36ea00700 1 -- 172.21.15.90:0/3603093684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 msgr2=0x55bac4213800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.262+0000 7fe36ea00700 1 --2- 172.21.15.90:0/3603093684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac4213800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.262+0000 7fe36ea00700 1 -- 172.21.15.90:0/3603093684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420fcf0 msgr2=0x55bac4216840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.262+0000 7fe36ea00700 1 --2- 172.21.15.90:0/3603093684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420fcf0 0x55bac4216840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.262+0000 7fe36ea00700 1 -- 172.21.15.90:0/3603093684 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac4216d80 con 0x55bac420adb0 2024-05-27T07:59:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.262+0000 7fe36ea00700 1 --2- 172.21.15.90:0/3603093684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 0x55bac420a870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe3600050d0 tx=0x7fe36000ccd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78da1fe90d63388f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:22.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.263+0000 7fe35b400700 1 -- 172.21.15.90:0/3603093684 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3600e2070 con 0x55bac420adb0 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.263+0000 7fe35b400700 1 -- 172.21.15.90:0/3603093684 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3600b3410 con 0x55bac420adb0 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.263+0000 7fe35b400700 1 -- 172.21.15.90:0/3603093684 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3600a9280 con 0x55bac420adb0 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.264+0000 7fe384745b80 1 -- 172.21.15.90:0/3603093684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 msgr2=0x55bac420a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.264+0000 7fe384745b80 1 --2- 172.21.15.90:0/3603093684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 0x55bac420a870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe3600050d0 tx=0x7fe36000ccd0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.264+0000 7fe384745b80 1 -- 172.21.15.90:0/3603093684 shutdown_connections 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.264+0000 7fe384745b80 1 --2- 172.21.15.90:0/3603093684 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420fcf0 0x55bac4216840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.264+0000 7fe384745b80 1 --2- 172.21.15.90:0/3603093684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420adb0 0x55bac420a870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.264+0000 7fe384745b80 1 --2- 172.21.15.90:0/3603093684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac4213800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.264+0000 7fe384745b80 1 -- 172.21.15.90:0/3603093684 >> 172.21.15.90:0/3603093684 conn(0x55bac40ebfc0 msgr2=0x55bac420e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.264+0000 7fe384745b80 1 -- 172.21.15.90:0/3603093684 shutdown_connections 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.264+0000 7fe384745b80 1 -- 172.21.15.90:0/3603093684 wait complete. 2024-05-27T07:59:22.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.265+0000 7fe384745b80 1 -- start start 2024-05-27T07:59:22.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.265+0000 7fe384745b80 1 Processor -- start 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe384745b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 0x55bac42258a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe384745b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420fcf0 0x55bac4233fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe384745b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac42344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe384745b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe35000f350 con 0x55bac420adb0 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe384745b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe35000dee0 con 0x55bac420fcf0 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe384745b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe35000e5b0 con 0x55bac4210d30 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe384745b80 10 client.0 _ll_register_callbacks cb 0x55bac40d2c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe36ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 0x55bac42258a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe36ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 0x55bac42258a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57394/0 (socket says 172.21.15.90:57394) 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe36ea00700 1 -- 172.21.15.90:0/1569581348 learned_addr learned my addr 172.21.15.90:0/1569581348 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe36e000700 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420fcf0 0x55bac4233fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.266+0000 7fe36f400700 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac42344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.267+0000 7fe36ea00700 1 -- 172.21.15.90:0/1569581348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 msgr2=0x55bac42344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.267+0000 7fe36ea00700 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac42344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.267+0000 7fe36ea00700 1 -- 172.21.15.90:0/1569581348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420fcf0 msgr2=0x55bac4233fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.267+0000 7fe36ea00700 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420fcf0 0x55bac4233fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.267+0000 7fe36ea00700 1 -- 172.21.15.90:0/1569581348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac4234aa0 con 0x55bac420adb0 2024-05-27T07:59:22.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.267+0000 7fe36ea00700 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 0x55bac42258a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe36000e9c0 tx=0x7fe360009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903f4043902d7d84 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:22.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.267+0000 7fe358c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:59:22.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.267+0000 7fe358c00700 1 -- 172.21.15.90:0/1569581348 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3600e2040 con 0x55bac420adb0 2024-05-27T07:59:22.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.267+0000 7fe358c00700 1 -- 172.21.15.90:0/1569581348 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fe3600a6ba0 con 0x55bac420adb0 2024-05-27T07:59:22.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.267+0000 7fe384745b80 1 -- 172.21.15.90:0/1569581348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3600b3410 con 0x55bac420adb0 2024-05-27T07:59:22.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe358c00700 1 -- 172.21.15.90:0/1569581348 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe3600aa5b0 con 0x55bac420adb0 2024-05-27T07:59:22.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe384745b80 10 client.17685 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:22.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe384745b80 1 -- 172.21.15.90:0/1569581348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55bac4297510 con 0x55bac420adb0 2024-05-27T07:59:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe358c00700 1 -- 172.21.15.90:0/1569581348 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe3600b3410 con 0x55bac420adb0 2024-05-27T07:59:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe358c00700 1 -- 172.21.15.90:0/1569581348 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe3600b1ce0 con 0x55bac420adb0 2024-05-27T07:59:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe384745b80 10 client.17685 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe384745b80 10 client.17685 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T07:59:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe384745b80 20 client.17685 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe384745b80 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55bac4297920 0x55bac4299d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe384745b80 4 client.17685 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:59:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.268+0000 7fe384745b80 1 -- 172.21.15.90:0/1569581348 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bac4134570 con 0x55bac4297920 2024-05-27T07:59:22.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.269+0000 7fe36e000700 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55bac4297920 0x55bac4299d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:22.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.269+0000 7fe358c00700 10 client.17685 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T07:59:22.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.269+0000 7fe36e000700 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55bac4297920 0x55bac4299d40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da00a47a585dab04 server_cookie=f027c34dc4732706 in_seq=0 out_seq=0 2024-05-27T07:59:22.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe358c00700 1 -- 172.21.15.90:0/1569581348 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bac4134570 con 0x55bac4297920 2024-05-27T07:59:22.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe358c00700 10 client.17685 handle_command_reply: tid=0 2024-05-27T07:59:22.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 client.17685 shutdown 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 -- 172.21.15.90:0/1569581348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 msgr2=0x55bac42258a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 0x55bac42258a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe36000e9c0 tx=0x7fe360009ef0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 -- 172.21.15.90:0/1569581348 shutdown_connections 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x55bac4297920 0x55bac4299d40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bac4210d30 0x55bac42344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bac420adb0 0x55bac42258a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 --2- 172.21.15.90:0/1569581348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bac420fcf0 0x55bac4233fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 -- 172.21.15.90:0/1569581348 >> 172.21.15.90:0/1569581348 conn(0x55bac40ebfc0 msgr2=0x55bac4213560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe358c00700 0 client.17685 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 -- 172.21.15.90:0/1569581348 shutdown_connections 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 1 -- 172.21.15.90:0/1569581348 wait complete. 2024-05-27T07:59:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.270+0000 7fe384745b80 20 client.17685 trim_cache size 0 max 16384 2024-05-27T07:59:22.274 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:59:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.271+0000 7fe375c00700 1 -- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3540b2c70 msgr2=0x7fe3540b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.271+0000 7fe375c00700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3540b2c70 0x7fe3540b5180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe370107a60 tx=0x7fe35c009260 comp rx=0 tx=0).stop 2024-05-27T07:59:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.271+0000 7fe375c00700 1 -- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3701045e0 msgr2=0x7fe3700753b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:22.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.271+0000 7fe375c00700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3701045e0 0x7fe3700753b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe36000e9f0 tx=0x7fe36000ed00 comp rx=0 tx=0).stop 2024-05-27T07:59:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.271+0000 7fe375c00700 1 -- 172.21.15.90:0/205905231 shutdown_connections 2024-05-27T07:59:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.271+0000 7fe375c00700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe370109860 0x7fe37006eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.271+0000 7fe375c00700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3701045e0 0x7fe3700753b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.271+0000 7fe375c00700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe3540b2c70 0x7fe3540b5180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.271+0000 7fe375c00700 1 --2- 172.21.15.90:0/205905231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe370108f10 0x7fe37006e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.271+0000 7fe375c00700 1 -- 172.21.15.90:0/205905231 >> 172.21.15.90:0/205905231 conn(0x7fe370100030 msgr2=0x7fe37010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.272+0000 7fe375c00700 1 -- 172.21.15.90:0/205905231 shutdown_connections 2024-05-27T07:59:22.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:22.272+0000 7fe375c00700 1 -- 172.21.15.90:0/205905231 wait complete. 2024-05-27T07:59:22.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:22 smithi090 ceph-mon[31429]: pgmap v414: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 MiB/s wr, 547 op/s 2024-05-27T07:59:23.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:22 smithi136 ceph-mon[35324]: pgmap v414: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 MiB/s wr, 547 op/s 2024-05-27T07:59:23.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:22 smithi129 ceph-mon[35254]: pgmap v414: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 MiB/s wr, 547 op/s 2024-05-27T07:59:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.117+0000 7f1173800640 1 -- 172.21.15.90:0/1551137270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116c10a1f0 msgr2=0x7f116c10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.117+0000 7f1173800640 1 --2- 172.21.15.90:0/1551137270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116c10a1f0 0x7f116c10a600 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1164009ca0 tx=0x7f116402a940 comp rx=0 tx=0).stop 2024-05-27T07:59:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.117+0000 7f1173800640 1 -- 172.21.15.90:0/1551137270 shutdown_connections 2024-05-27T07:59:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.117+0000 7f1173800640 1 --2- 172.21.15.90:0/1551137270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f116c11d240 0x7f116c11f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.117+0000 7f1173800640 1 --2- 172.21.15.90:0/1551137270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116c11a8e0 0x7f116c11cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.117+0000 7f1173800640 1 --2- 172.21.15.90:0/1551137270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116c10a1f0 0x7f116c10a600 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.117+0000 7f1173800640 1 -- 172.21.15.90:0/1551137270 >> 172.21.15.90:0/1551137270 conn(0x7f116c06e8f0 msgr2=0x7f116c06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.118+0000 7f1173800640 1 -- 172.21.15.90:0/1551137270 shutdown_connections 2024-05-27T07:59:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.118+0000 7f1173800640 1 -- 172.21.15.90:0/1551137270 wait complete. 2024-05-27T07:59:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.122+0000 7f1173800640 1 Processor -- start 2024-05-27T07:59:23.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.122+0000 7f1173800640 1 -- start start 2024-05-27T07:59:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.124+0000 7f1173800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116c10a1f0 0x7f116c1a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.124+0000 7f1173800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116c11a8e0 0x7f116c1a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.124+0000 7f1173800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f116c11d240 0x7f116c1addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.124+0000 7f1173800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f116c122b20 con 0x7f116c10a1f0 2024-05-27T07:59:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.124+0000 7f1173800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1229a0 con 0x7f116c11d240 2024-05-27T07:59:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.124+0000 7f1173800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f116c122ca0 con 0x7f116c11a8e0 2024-05-27T07:59:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.125+0000 7f1171800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f116c11d240 0x7f116c1addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.125+0000 7f1170e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116c10a1f0 0x7f116c1a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.125+0000 7f1170e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116c10a1f0 0x7f116c1a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57414/0 (socket says 172.21.15.90:57414) 2024-05-27T07:59:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.125+0000 7f1170e00640 1 -- 172.21.15.90:0/2434048846 learned_addr learned my addr 172.21.15.90:0/2434048846 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.125+0000 7f116be00640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116c11a8e0 0x7f116c1a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.126+0000 7f1170e00640 1 -- 172.21.15.90:0/2434048846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116c11a8e0 msgr2=0x7f116c1a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.126+0000 7f1170e00640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116c11a8e0 0x7f116c1a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.127+0000 7f1170e00640 1 -- 172.21.15.90:0/2434048846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f116c11d240 msgr2=0x7f116c1addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.127+0000 7f1170e00640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f116c11d240 0x7f116c1addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.127+0000 7f1170e00640 1 -- 172.21.15.90:0/2434048846 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c1ae3f0 con 0x7f116c10a1f0 2024-05-27T07:59:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.127+0000 7f1170e00640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116c10a1f0 0x7f116c1a95e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1164004af0 tx=0x7f1164003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.128+0000 7f1169600640 1 -- 172.21.15.90:0/2434048846 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164034040 con 0x7f116c10a1f0 2024-05-27T07:59:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.128+0000 7f1169600640 1 -- 172.21.15.90:0/2434048846 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1164031e80 con 0x7f116c10a1f0 2024-05-27T07:59:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.128+0000 7f1173800640 1 -- 172.21.15.90:0/2434048846 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116c1ae680 con 0x7f116c10a1f0 2024-05-27T07:59:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.128+0000 7f1169600640 1 -- 172.21.15.90:0/2434048846 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164004640 con 0x7f116c10a1f0 2024-05-27T07:59:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.129+0000 7f1173800640 1 -- 172.21.15.90:0/2434048846 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116c1aeb90 con 0x7f116c10a1f0 2024-05-27T07:59:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.136+0000 7f1169600640 1 -- 172.21.15.90:0/2434048846 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1164037020 con 0x7f116c10a1f0 2024-05-27T07:59:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.137+0000 7f117a855740 1 Processor -- start 2024-05-27T07:59:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.138+0000 7f117a855740 1 -- start start 2024-05-27T07:59:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.138+0000 7f117a855740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be24860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.138+0000 7f1169600640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2a50 0x7f11500b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.138+0000 7f117a855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba592f0 0x55c84be24da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.138+0000 7f117a855740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba614d0 0x55c84be252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.138+0000 7f117a855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c84bd95b90 con 0x55c84ba592f0 2024-05-27T07:59:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.138+0000 7f117a855740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c84bd85730 con 0x55c84ba821b0 2024-05-27T07:59:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.138+0000 7f117a855740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c84bd89770 con 0x55c84ba614d0 2024-05-27T07:59:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.139+0000 7f1169600640 1 -- 172.21.15.90:0/2434048846 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f1164103860 con 0x7f116c10a1f0 2024-05-27T07:59:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.139+0000 7f1170e00640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2a50 0x7f11500b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.140+0000 7f116be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba592f0 0x55c84be24da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.140+0000 7f1171800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be24860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.140+0000 7f1171800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be24860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47212/0 (socket says 172.21.15.90:47212) 2024-05-27T07:59:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.140+0000 7f1171800640 1 -- 172.21.15.90:0/3585969007 learned_addr learned my addr 172.21.15.90:0/3585969007 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.140+0000 7f1171800640 1 --2- 172.21.15.90:0/3585969007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba614d0 0x55c84be252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.141+0000 7f1170e00640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2a50 0x7f11500b4f60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11640c6ad0 tx=0x7f1164009460 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.141+0000 7f1171800640 1 -- 172.21.15.90:0/3585969007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba614d0 msgr2=0x55c84be252e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.141+0000 7f1171800640 1 --2- 172.21.15.90:0/3585969007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba614d0 0x55c84be252e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.141+0000 7f1171800640 1 -- 172.21.15.90:0/3585969007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba592f0 msgr2=0x55c84be24da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.141+0000 7f1171800640 1 --2- 172.21.15.90:0/3585969007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba592f0 0x55c84be24da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.141+0000 7f1171800640 1 -- 172.21.15.90:0/3585969007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c84bd1e8b0 con 0x55c84ba821b0 2024-05-27T07:59:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.141+0000 7f116be00640 1 --2- 172.21.15.90:0/3585969007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba592f0 0x55c84be24da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.142+0000 7f1171800640 1 --2- 172.21.15.90:0/3585969007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be24860 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f115c0126f0 tx=0x7f115c00b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6daba11a87375e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.142+0000 7f1162000640 1 -- 172.21.15.90:0/3585969007 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c012c70 con 0x55c84ba821b0 2024-05-27T07:59:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.142+0000 7f1162000640 1 -- 172.21.15.90:0/3585969007 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115c012e10 con 0x55c84ba821b0 2024-05-27T07:59:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.142+0000 7f1162000640 1 -- 172.21.15.90:0/3585969007 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c004e10 con 0x55c84ba821b0 2024-05-27T07:59:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.143+0000 7f117a855740 1 -- 172.21.15.90:0/3585969007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 msgr2=0x55c84be24860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.143+0000 7f117a855740 1 --2- 172.21.15.90:0/3585969007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be24860 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f115c0126f0 tx=0x7f115c00b6d0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.144+0000 7f117a855740 1 -- 172.21.15.90:0/3585969007 shutdown_connections 2024-05-27T07:59:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.144+0000 7f117a855740 1 --2- 172.21.15.90:0/3585969007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba614d0 0x55c84be252e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.144+0000 7f117a855740 1 --2- 172.21.15.90:0/3585969007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba592f0 0x55c84be24da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.144+0000 7f117a855740 1 --2- 172.21.15.90:0/3585969007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be24860 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.144+0000 7f117a855740 1 -- 172.21.15.90:0/3585969007 >> 172.21.15.90:0/3585969007 conn(0x55c84ba76300 msgr2=0x55c84be21010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.144+0000 7f117a855740 1 -- 172.21.15.90:0/3585969007 shutdown_connections 2024-05-27T07:59:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.144+0000 7f117a855740 1 -- 172.21.15.90:0/3585969007 wait complete. 2024-05-27T07:59:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.145+0000 7f117a855740 1 -- start start 2024-05-27T07:59:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.145+0000 7f117a855740 1 Processor -- start 2024-05-27T07:59:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.146+0000 7f117a855740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba592f0 0x55c84be2ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.146+0000 7f117a855740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba614d0 0x55c84be2f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.146+0000 7f117a855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 0x55c84be3acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.146+0000 7f117a855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c84ba6b790 con 0x55c84ba821b0 2024-05-27T07:59:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.146+0000 7f117a855740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c84bd2dbc0 con 0x55c84ba592f0 2024-05-27T07:59:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.146+0000 7f117a855740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c84be31990 con 0x55c84ba614d0 2024-05-27T07:59:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.146+0000 7f117a855740 10 client.0 _ll_register_callbacks cb 0x55c84bd0b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.147+0000 7f1171800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 0x55c84be3acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.147+0000 7f1171800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 0x55c84be3acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57432/0 (socket says 172.21.15.90:57432) 2024-05-27T07:59:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.147+0000 7f1171800640 1 -- 172.21.15.90:0/3102276569 learned_addr learned my addr 172.21.15.90:0/3102276569 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.147+0000 7f1171800640 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba592f0 0x55c84be2ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.147+0000 7f116be00640 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba614d0 0x55c84be2f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.147+0000 7f1171800640 1 -- 172.21.15.90:0/3102276569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba614d0 msgr2=0x55c84be2f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.147+0000 7f1171800640 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba614d0 0x55c84be2f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.147+0000 7f1171800640 1 -- 172.21.15.90:0/3102276569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba592f0 msgr2=0x55c84be2ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.147+0000 7f1171800640 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba592f0 0x55c84be2ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.147+0000 7f1171800640 1 -- 172.21.15.90:0/3102276569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c84ba9a210 con 0x55c84ba821b0 2024-05-27T07:59:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.148+0000 7f1171800640 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 0x55c84be3acc0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x55c84be2e330 tx=0x7f115c01b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0291bfbd5367aac server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.148+0000 7f1156a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:59:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.148+0000 7f1156a00640 1 -- 172.21.15.90:0/3102276569 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c004460 con 0x55c84ba821b0 2024-05-27T07:59:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.148+0000 7f1156a00640 1 -- 172.21.15.90:0/3102276569 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f115c010770 con 0x55c84ba821b0 2024-05-27T07:59:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.148+0000 7f117a855740 1 -- 172.21.15.90:0/3102276569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115c012e10 con 0x55c84ba821b0 2024-05-27T07:59:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.149+0000 7f1156a00640 1 -- 172.21.15.90:0/3102276569 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f115c036050 con 0x55c84ba821b0 2024-05-27T07:59:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.149+0000 7f117a855740 10 client.17697 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.149+0000 7f117a855740 1 -- 172.21.15.90:0/3102276569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55c84bb66620 con 0x55c84ba821b0 2024-05-27T07:59:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.150+0000 7f1156a00640 1 -- 172.21.15.90:0/3102276569 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115c012e10 con 0x55c84ba821b0 2024-05-27T07:59:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.150+0000 7f1156a00640 1 -- 172.21.15.90:0/3102276569 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f115c005320 con 0x55c84ba821b0 2024-05-27T07:59:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.150+0000 7f117a855740 10 client.17697 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.150+0000 7f117a855740 10 client.17697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.150+0000 7f117a855740 20 client.17697 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.150+0000 7f117a855740 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c84ba40d80 0x55c84be40ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.150+0000 7f117a855740 4 client.17697 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.150+0000 7f117a855740 1 -- 172.21.15.90:0/3102276569 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c84bd97ae0 con 0x55c84ba40d80 2024-05-27T07:59:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.151+0000 7f116be00640 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c84ba40d80 0x55c84be40ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.152+0000 7f116be00640 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c84ba40d80 0x55c84be40ca0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526f32873b014cd3 server_cookie=f130a9759ecbe600 in_seq=0 out_seq=0 2024-05-27T07:59:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.152+0000 7f1156a00640 10 client.17697 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.154+0000 7f1156a00640 1 -- 172.21.15.90:0/3102276569 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c84bd97ae0 con 0x55c84ba40d80 2024-05-27T07:59:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.154+0000 7f1156a00640 10 client.17697 handle_command_reply: tid=0 2024-05-27T07:59:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.154+0000 7f117a855740 1 client.17697 shutdown 2024-05-27T07:59:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.156+0000 7f117a855740 1 -- 172.21.15.90:0/3102276569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 msgr2=0x55c84be3acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.156+0000 7f117a855740 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 0x55c84be3acc0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x55c84be2e330 tx=0x7f115c01b040 comp rx=0 tx=0).stop 2024-05-27T07:59:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.156+0000 7f117a855740 1 -- 172.21.15.90:0/3102276569 shutdown_connections 2024-05-27T07:59:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.156+0000 7f117a855740 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c84ba40d80 0x55c84be40ca0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.156+0000 7f117a855740 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 0x55c84be3acc0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.156+0000 7f117a855740 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba614d0 0x55c84be2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.156+0000 7f117a855740 1 --2- 172.21.15.90:0/3102276569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba592f0 0x55c84be2ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.156+0000 7f117a855740 1 -- 172.21.15.90:0/3102276569 >> 172.21.15.90:0/3102276569 conn(0x55c84ba76300 msgr2=0x55c84be24630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.156+0000 7f1156a00640 0 client.17697 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.157+0000 7f117a855740 1 -- 172.21.15.90:0/3102276569 shutdown_connections 2024-05-27T07:59:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.157+0000 7f117a855740 1 -- 172.21.15.90:0/3102276569 wait complete. 2024-05-27T07:59:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.157+0000 7f117a855740 20 client.17697 trim_cache size 0 max 16384 2024-05-27T07:59:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.190+0000 7f117a855740 1 Processor -- start 2024-05-27T07:59:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.190+0000 7f117a855740 1 -- start start 2024-05-27T07:59:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.191+0000 7f117a855740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba592f0 0x55c84be2f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.191+0000 7f117a855740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba614d0 0x55c84be3aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.191+0000 7f117a855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 0x55c84be3af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.191+0000 7f117a855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1c1b50 con 0x55c84ba821b0 2024-05-27T07:59:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.191+0000 7f117a855740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1beba0 con 0x55c84ba614d0 2024-05-27T07:59:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.191+0000 7f117a855740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1c0db0 con 0x55c84ba592f0 2024-05-27T07:59:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.192+0000 7f116be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba614d0 0x55c84be3aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.192+0000 7f1171800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba592f0 0x55c84be2f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.192+0000 7f116be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba614d0 0x55c84be3aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47222/0 (socket says 172.21.15.90:47222) 2024-05-27T07:59:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.192+0000 7f116be00640 1 -- 172.21.15.90:0/3490223807 learned_addr learned my addr 172.21.15.90:0/3490223807 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.192+0000 7f1171800640 1 --2- 172.21.15.90:0/3490223807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 0x55c84be3af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.192+0000 7f116be00640 1 -- 172.21.15.90:0/3490223807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba592f0 msgr2=0x55c84be2f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.192+0000 7f116be00640 1 --2- 172.21.15.90:0/3490223807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba592f0 0x55c84be2f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.192+0000 7f116be00640 1 -- 172.21.15.90:0/3490223807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 msgr2=0x55c84be3af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.192+0000 7f116be00640 1 --2- 172.21.15.90:0/3490223807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 0x55c84be3af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.192+0000 7f116be00640 1 -- 172.21.15.90:0/3490223807 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c84be25520 con 0x55c84ba614d0 2024-05-27T07:59:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.193+0000 7f116be00640 1 --2- 172.21.15.90:0/3490223807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba614d0 0x55c84be3aa30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55c84be21fa0 tx=0x7f115800b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fdf5b90826e1a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.194+0000 7f1161600640 1 -- 172.21.15.90:0/3490223807 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115800bc90 con 0x55c84ba614d0 2024-05-27T07:59:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.194+0000 7f1161600640 1 -- 172.21.15.90:0/3490223807 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115800be30 con 0x55c84ba614d0 2024-05-27T07:59:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.194+0000 7f1161600640 1 -- 172.21.15.90:0/3490223807 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115801d6e0 con 0x55c84ba614d0 2024-05-27T07:59:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.195+0000 7f117a855740 1 -- 172.21.15.90:0/3490223807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba614d0 msgr2=0x55c84be3aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.195+0000 7f117a855740 1 --2- 172.21.15.90:0/3490223807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba614d0 0x55c84be3aa30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55c84be21fa0 tx=0x7f115800b390 comp rx=0 tx=0).stop 2024-05-27T07:59:23.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.196+0000 7f117a855740 1 -- 172.21.15.90:0/3490223807 shutdown_connections 2024-05-27T07:59:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.196+0000 7f117a855740 1 --2- 172.21.15.90:0/3490223807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba614d0 0x55c84be3aa30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.196+0000 7f117a855740 1 --2- 172.21.15.90:0/3490223807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba821b0 0x55c84be3af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.196+0000 7f117a855740 1 --2- 172.21.15.90:0/3490223807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba592f0 0x55c84be2f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.196+0000 7f117a855740 1 -- 172.21.15.90:0/3490223807 >> 172.21.15.90:0/3490223807 conn(0x55c84ba76300 msgr2=0x55c84be2ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.196+0000 7f117a855740 1 -- 172.21.15.90:0/3490223807 shutdown_connections 2024-05-27T07:59:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.196+0000 7f117a855740 1 -- 172.21.15.90:0/3490223807 wait complete. 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.197+0000 7f117a855740 1 -- start start 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.197+0000 7f117a855740 1 Processor -- start 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.198+0000 7f117a855740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba592f0 0x55c84be33970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.198+0000 7f117a855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba614d0 0x55c84be33eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.198+0000 7f117a855740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.198+0000 7f117a855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1c06e0 con 0x55c84ba614d0 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.198+0000 7f117a855740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1bf270 con 0x55c84ba821b0 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.198+0000 7f117a855740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1bf940 con 0x55c84ba592f0 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.198+0000 7f117a855740 10 client.0 _ll_register_callbacks cb 0x7f115800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f116be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba614d0 0x55c84be33eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f116be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba614d0 0x55c84be33eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57440/0 (socket says 172.21.15.90:57440) 2024-05-27T07:59:23.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f116be00640 1 -- 172.21.15.90:0/4100302041 learned_addr learned my addr 172.21.15.90:0/4100302041 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f1171800640 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f1171800640 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba592f0 0x55c84be33970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f116be00640 1 -- 172.21.15.90:0/4100302041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba592f0 msgr2=0x55c84be33970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f116be00640 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba592f0 0x55c84be33970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f116be00640 1 -- 172.21.15.90:0/4100302041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 msgr2=0x55c84be343f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f116be00640 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be343f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f116be00640 1 -- 172.21.15.90:0/4100302041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c84bea7330 con 0x55c84ba614d0 2024-05-27T07:59:23.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.199+0000 7f1171800640 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be343f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.200+0000 7f116be00640 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba614d0 0x55c84be33eb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1158005920 tx=0x7f115800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f86cc027796866 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.200+0000 7f1156a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:59:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.201+0000 7f1156a00640 1 -- 172.21.15.90:0/4100302041 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158005020 con 0x55c84ba614d0 2024-05-27T07:59:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.201+0000 7f1156a00640 1 -- 172.21.15.90:0/4100302041 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f11580051c0 con 0x55c84ba614d0 2024-05-27T07:59:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.201+0000 7f117a855740 1 -- 172.21.15.90:0/4100302041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115800be30 con 0x55c84ba614d0 2024-05-27T07:59:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.201+0000 7f1156a00640 1 -- 172.21.15.90:0/4100302041 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f11580053c0 con 0x55c84ba614d0 2024-05-27T07:59:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.201+0000 7f117a855740 10 client.17703 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.202+0000 7f117a855740 1 -- 172.21.15.90:0/4100302041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55c84bea8500 con 0x55c84ba614d0 2024-05-27T07:59:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.202+0000 7f1156a00640 1 -- 172.21.15.90:0/4100302041 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f115800be30 con 0x55c84ba614d0 2024-05-27T07:59:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.203+0000 7f1156a00640 1 -- 172.21.15.90:0/4100302041 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f1158004730 con 0x55c84ba614d0 2024-05-27T07:59:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.203+0000 7f117a855740 10 client.17703 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:23.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.204+0000 7f117a855740 10 client.17703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.204+0000 7f117a855740 20 client.17703 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.204+0000 7f117a855740 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c84ba161c0 0x55c84beaaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.204+0000 7f117a855740 4 client.17703 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:59:23.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.204+0000 7f117a855740 1 -- 172.21.15.90:0/4100302041 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c84be25130 con 0x55c84ba161c0 2024-05-27T07:59:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.205+0000 7f1171800640 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c84ba161c0 0x55c84beaaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:23.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.206+0000 7f1171800640 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c84ba161c0 0x55c84beaaab0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc38a456847b2679 server_cookie=6149e5a26fdff2dd in_seq=0 out_seq=0 2024-05-27T07:59:23.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.207+0000 7f1156a00640 10 client.17703 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:23.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 545.0 s, 360.2 tps, lat 278.238 ms stddev 432.195 2024-05-27T07:59:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.829+0000 7f1156a00640 1 -- 172.21.15.90:0/4100302041 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c84be25130 con 0x55c84ba161c0 2024-05-27T07:59:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.829+0000 7f1156a00640 10 client.17703 handle_command_reply: tid=0 2024-05-27T07:59:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.829+0000 7f117a855740 1 client.17703 shutdown 2024-05-27T07:59:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.829+0000 7f117a855740 1 -- 172.21.15.90:0/4100302041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba614d0 msgr2=0x55c84be33eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:23.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.829+0000 7f117a855740 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba614d0 0x55c84be33eb0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1158005920 tx=0x7f115800b900 comp rx=0 tx=0).stop 2024-05-27T07:59:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.830+0000 7f117a855740 1 -- 172.21.15.90:0/4100302041 shutdown_connections 2024-05-27T07:59:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.830+0000 7f117a855740 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c84ba161c0 0x55c84beaaab0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.830+0000 7f117a855740 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c84ba821b0 0x55c84be343f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.830+0000 7f117a855740 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c84ba614d0 0x55c84be33eb0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.830+0000 7f117a855740 1 --2- 172.21.15.90:0/4100302041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c84ba592f0 0x55c84be33970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.830+0000 7f117a855740 1 -- 172.21.15.90:0/4100302041 >> 172.21.15.90:0/4100302041 conn(0x55c84ba76300 msgr2=0x55c84be2f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:23.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.830+0000 7f1156a00640 0 client.17703 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.830+0000 7f117a855740 1 -- 172.21.15.90:0/4100302041 shutdown_connections 2024-05-27T07:59:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.831+0000 7f117a855740 1 -- 172.21.15.90:0/4100302041 wait complete. 2024-05-27T07:59:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.831+0000 7f117a855740 20 client.17703 trim_cache size 0 max 16384 2024-05-27T07:59:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:59:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:59:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:59:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.833+0000 7f1173800640 1 -- 172.21.15.90:0/2434048846 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2a50 msgr2=0x7f11500b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.833+0000 7f1173800640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2a50 0x7f11500b4f60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11640c6ad0 tx=0x7f1164009460 comp rx=0 tx=0).stop 2024-05-27T07:59:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.833+0000 7f1173800640 1 -- 172.21.15.90:0/2434048846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116c10a1f0 msgr2=0x7f116c1a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.833+0000 7f1173800640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116c10a1f0 0x7f116c1a95e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1164004af0 tx=0x7f1164003c20 comp rx=0 tx=0).stop 2024-05-27T07:59:23.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.834+0000 7f1173800640 1 -- 172.21.15.90:0/2434048846 shutdown_connections 2024-05-27T07:59:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.834+0000 7f1173800640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f11500b2a50 0x7f11500b4f60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.834+0000 7f1173800640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f116c11d240 0x7f116c1addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.834+0000 7f1173800640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f116c11a8e0 0x7f116c1a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.834+0000 7f1173800640 1 --2- 172.21.15.90:0/2434048846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f116c10a1f0 0x7f116c1a95e0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.834+0000 7f1173800640 1 -- 172.21.15.90:0/2434048846 >> 172.21.15.90:0/2434048846 conn(0x7f116c06e8f0 msgr2=0x7f116c11acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.834+0000 7f1173800640 1 -- 172.21.15.90:0/2434048846 shutdown_connections 2024-05-27T07:59:23.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:23.834+0000 7f1173800640 1 -- 172.21.15.90:0/2434048846 wait complete. 2024-05-27T07:59:24.724 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T07:59:24.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:24 smithi090 ceph-mon[31429]: pgmap v415: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.6 MiB/s wr, 524 op/s 2024-05-27T07:59:25.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:24 smithi136 ceph-mon[35324]: pgmap v415: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.6 MiB/s wr, 524 op/s 2024-05-27T07:59:25.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:24 smithi129 ceph-mon[35254]: pgmap v415: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.6 MiB/s wr, 524 op/s 2024-05-27T07:59:25.447 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:26.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:26 smithi090 ceph-mon[31429]: pgmap v416: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.9 MiB/s wr, 482 op/s 2024-05-27T07:59:27.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:26 smithi136 ceph-mon[35324]: pgmap v416: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.9 MiB/s wr, 482 op/s 2024-05-27T07:59:27.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:26 smithi129 ceph-mon[35254]: pgmap v416: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.9 MiB/s wr, 482 op/s 2024-05-27T07:59:28.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:28 smithi090 ceph-mon[31429]: pgmap v417: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 460 op/s 2024-05-27T07:59:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 550.0 s, 637.2 tps, lat 157.390 ms stddev 111.437 2024-05-27T07:59:29.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:28 smithi136 ceph-mon[35324]: pgmap v417: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 460 op/s 2024-05-27T07:59:29.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:28 smithi129 ceph-mon[35254]: pgmap v417: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.8 MiB/s wr, 460 op/s 2024-05-27T07:59:30.968 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:30 smithi090 ceph-mon[31429]: pgmap v418: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.5 MiB/s wr, 525 op/s 2024-05-27T07:59:30.968 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:30 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:59:31.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:30 smithi136 ceph-mon[35324]: pgmap v418: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.5 MiB/s wr, 525 op/s 2024-05-27T07:59:31.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:30 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:59:31.078 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:30 smithi129 ceph-mon[35254]: pgmap v418: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.5 MiB/s wr, 525 op/s 2024-05-27T07:59:31.078 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:30 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T07:59:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.147+0000 7f9549a00700 1 -- 172.21.15.90:0/3362009981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95441034b0 msgr2=0x7f95441038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.147+0000 7f9549a00700 1 --2- 172.21.15.90:0/3362009981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95441034b0 0x7f95441038c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f95340099c0 tx=0x7f9534009cd0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.148+0000 7f9549a00700 1 -- 172.21.15.90:0/3362009981 shutdown_connections 2024-05-27T07:59:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.148+0000 7f9549a00700 1 --2- 172.21.15.90:0/3362009981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9544108730 0x7f9544110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.148+0000 7f9549a00700 1 --2- 172.21.15.90:0/3362009981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95441034b0 0x7f95441038c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.148+0000 7f9549a00700 1 --2- 172.21.15.90:0/3362009981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9544107de0 0x7f95441081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.148+0000 7f9549a00700 1 -- 172.21.15.90:0/3362009981 >> 172.21.15.90:0/3362009981 conn(0x7f95440fedb0 msgr2=0x7f95441011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.148+0000 7f9549a00700 1 -- 172.21.15.90:0/3362009981 shutdown_connections 2024-05-27T07:59:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.148+0000 7f9549a00700 1 -- 172.21.15.90:0/3362009981 wait complete. 2024-05-27T07:59:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.150+0000 7f9549a00700 1 Processor -- start 2024-05-27T07:59:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.150+0000 7f9549a00700 1 -- start start 2024-05-27T07:59:31.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.151+0000 7f9549a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95441034b0 0x7f95440750a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.151+0000 7f9549a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9544107de0 0x7f954406e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.151+0000 7f9549a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9544108730 0x7f954406e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.151+0000 7f9549a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f954406bc10 con 0x7f95441034b0 2024-05-27T07:59:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.151+0000 7f9549a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f954406ba90 con 0x7f9544107de0 2024-05-27T07:59:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.151+0000 7f9549a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f954406bd90 con 0x7f9544108730 2024-05-27T07:59:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.152+0000 7f9542000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9544107de0 0x7f954406e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.152+0000 7f9542000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9544107de0 0x7f954406e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38708/0 (socket says 172.21.15.90:38708) 2024-05-27T07:59:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.152+0000 7f9542000700 1 -- 172.21.15.90:0/1919853075 learned_addr learned my addr 172.21.15.90:0/1919853075 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.152+0000 7f9543400700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9544108730 0x7f954406e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.152+0000 7f9542a00700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95441034b0 0x7f95440750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.153+0000 7f9542000700 1 -- 172.21.15.90:0/1919853075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9544108730 msgr2=0x7f954406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.153+0000 7f9542000700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9544108730 0x7f954406e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.153+0000 7f9542000700 1 -- 172.21.15.90:0/1919853075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95441034b0 msgr2=0x7f95440750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.154+0000 7f9542000700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95441034b0 0x7f95440750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.155+0000 7f9542000700 1 -- 172.21.15.90:0/1919853075 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954406ecd0 con 0x7f9544107de0 2024-05-27T07:59:31.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.156+0000 7f9542000700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9544107de0 0x7f954406e1d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f953000c9e0 tx=0x7f953000ccf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:31.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.156+0000 7f953b400700 1 -- 172.21.15.90:0/1919853075 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530007aa0 con 0x7f9544107de0 2024-05-27T07:59:31.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.157+0000 7f9549a00700 1 -- 172.21.15.90:0/1919853075 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954406efc0 con 0x7f9544107de0 2024-05-27T07:59:31.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.157+0000 7f9549a00700 1 -- 172.21.15.90:0/1919853075 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9544072f40 con 0x7f9544107de0 2024-05-27T07:59:31.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.157+0000 7f953b400700 1 -- 172.21.15.90:0/1919853075 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9530007c40 con 0x7f9544107de0 2024-05-27T07:59:31.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.157+0000 7f953b400700 1 -- 172.21.15.90:0/1919853075 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530018400 con 0x7f9544107de0 2024-05-27T07:59:31.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.166+0000 7f953b400700 1 -- 172.21.15.90:0/1919853075 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9530004850 con 0x7f9544107de0 2024-05-27T07:59:31.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.170+0000 7f953b400700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f95280b2a70 0x7f95280b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.170+0000 7f953b400700 1 -- 172.21.15.90:0/1919853075 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f95300e3760 con 0x7f9544107de0 2024-05-27T07:59:31.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.170+0000 7f9542a00700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f95280b2a70 0x7f95280b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.173+0000 7f9542a00700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f95280b2a70 0x7f95280b4f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9534009990 tx=0x7f9534005fb0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:31.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.176+0000 7f95585a0b80 1 Processor -- start 2024-05-27T07:59:31.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.177+0000 7f95585a0b80 1 -- start start 2024-05-27T07:59:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.178+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c33e9f0 0x56350c3d1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.178+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c252e40 0x56350c3d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.178+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3de110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.178+0000 7f95585a0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56350c2ffe40 con 0x56350c252e40 2024-05-27T07:59:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.178+0000 7f95585a0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56350c2ef9e0 con 0x56350c33e9f0 2024-05-27T07:59:31.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.178+0000 7f95585a0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56350c2f3a20 con 0x56350c3d1e70 2024-05-27T07:59:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.178+0000 7f9543400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.178+0000 7f9543400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3de110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:35874/0 (socket says 172.21.15.90:35874) 2024-05-27T07:59:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.178+0000 7f9543400700 1 -- 172.21.15.90:0/30144804 learned_addr learned my addr 172.21.15.90:0/30144804 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.178+0000 7f9542a00700 1 --2- 172.21.15.90:0/30144804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c33e9f0 0x56350c3d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.179+0000 7f9543400700 1 -- 172.21.15.90:0/30144804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c33e9f0 msgr2=0x56350c3d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.179+0000 7f9543400700 1 --2- 172.21.15.90:0/30144804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c33e9f0 0x56350c3d1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.179+0000 7f9543400700 1 -- 172.21.15.90:0/30144804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c252e40 msgr2=0x56350c3d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:31.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.179+0000 7f9543400700 1 --2- 172.21.15.90:0/30144804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c252e40 0x56350c3d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.179+0000 7f9543400700 1 -- 172.21.15.90:0/30144804 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350c2b8420 con 0x56350c3d1e70 2024-05-27T07:59:31.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.179+0000 7f9543400700 1 --2- 172.21.15.90:0/30144804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3de110 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f953c003fa0 tx=0x7f953c009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4991898f3d774265 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:31.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.180+0000 7f952fe00700 1 -- 172.21.15.90:0/30144804 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c004bb0 con 0x56350c3d1e70 2024-05-27T07:59:31.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.181+0000 7f952fe00700 1 -- 172.21.15.90:0/30144804 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f953c004d50 con 0x56350c3d1e70 2024-05-27T07:59:31.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.181+0000 7f952fe00700 1 -- 172.21.15.90:0/30144804 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c007d80 con 0x56350c3d1e70 2024-05-27T07:59:31.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.182+0000 7f95585a0b80 1 -- 172.21.15.90:0/30144804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 msgr2=0x56350c3de110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:31.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.182+0000 7f95585a0b80 1 --2- 172.21.15.90:0/30144804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3de110 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f953c003fa0 tx=0x7f953c009ba0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.182+0000 7f95585a0b80 1 -- 172.21.15.90:0/30144804 shutdown_connections 2024-05-27T07:59:31.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.182+0000 7f95585a0b80 1 --2- 172.21.15.90:0/30144804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3de110 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.182+0000 7f95585a0b80 1 --2- 172.21.15.90:0/30144804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c33e9f0 0x56350c3d1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.182+0000 7f95585a0b80 1 --2- 172.21.15.90:0/30144804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c252e40 0x56350c3d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.182+0000 7f95585a0b80 1 -- 172.21.15.90:0/30144804 >> 172.21.15.90:0/30144804 conn(0x56350c2ec390 msgr2=0x56350c3cd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:31.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.183+0000 7f95585a0b80 1 -- 172.21.15.90:0/30144804 shutdown_connections 2024-05-27T07:59:31.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.183+0000 7f95585a0b80 1 -- 172.21.15.90:0/30144804 wait complete. 2024-05-27T07:59:31.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.184+0000 7f95585a0b80 1 -- start start 2024-05-27T07:59:31.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.185+0000 7f95585a0b80 1 Processor -- start 2024-05-27T07:59:31.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.186+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c252e40 0x56350c3db680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.186+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c33e9f0 0x56350c3dbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.186+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.187+0000 7f95585a0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56350c1f5fa0 con 0x56350c33e9f0 2024-05-27T07:59:31.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.187+0000 7f95585a0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56350c290760 con 0x56350c252e40 2024-05-27T07:59:31.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.187+0000 7f95585a0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56350c3e7390 con 0x56350c3d1e70 2024-05-27T07:59:31.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.187+0000 7f95585a0b80 10 client.0 _ll_register_callbacks cb 0x56350c299c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:31.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.188+0000 7f9542000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c33e9f0 0x56350c3dbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.188+0000 7f9542000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c33e9f0 0x56350c3dbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56572/0 (socket says 172.21.15.90:56572) 2024-05-27T07:59:31.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.188+0000 7f9542000700 1 -- 172.21.15.90:0/1777505477 learned_addr learned my addr 172.21.15.90:0/1777505477 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:31.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.188+0000 7f9543400700 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.188+0000 7f9542a00700 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c252e40 0x56350c3db680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.189+0000 7f9542000700 1 -- 172.21.15.90:0/1777505477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 msgr2=0x56350c3dc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:31.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.189+0000 7f9542000700 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3dc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.189+0000 7f9542000700 1 -- 172.21.15.90:0/1777505477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c252e40 msgr2=0x56350c3db680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:31.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.189+0000 7f9542000700 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c252e40 0x56350c3db680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.189+0000 7f9542000700 1 -- 172.21.15.90:0/1777505477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350c23f8d0 con 0x56350c33e9f0 2024-05-27T07:59:31.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.189+0000 7f9542000700 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c33e9f0 0x56350c3dbbc0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f95300e0b90 tx=0x7f95300a9e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f571b6fa26e38c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:31.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.189+0000 7f952cc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:59:31.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.190+0000 7f952cc00700 1 -- 172.21.15.90:0/1777505477 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95300aa670 con 0x56350c33e9f0 2024-05-27T07:59:31.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.190+0000 7f952cc00700 1 -- 172.21.15.90:0/1777505477 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f95300aa8d0 con 0x56350c33e9f0 2024-05-27T07:59:31.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.190+0000 7f95585a0b80 1 -- 172.21.15.90:0/1777505477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f953c004d50 con 0x56350c33e9f0 2024-05-27T07:59:31.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.191+0000 7f952cc00700 1 -- 172.21.15.90:0/1777505477 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f95300aea20 con 0x56350c33e9f0 2024-05-27T07:59:31.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.191+0000 7f95585a0b80 10 client.17712 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:31.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.191+0000 7f95585a0b80 1 -- 172.21.15.90:0/1777505477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56350c2c0060 con 0x56350c33e9f0 2024-05-27T07:59:31.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.193+0000 7f952cc00700 1 -- 172.21.15.90:0/1777505477 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f953c004d50 con 0x56350c33e9f0 2024-05-27T07:59:31.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.193+0000 7f952cc00700 1 -- 172.21.15.90:0/1777505477 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f95300b3040 con 0x56350c33e9f0 2024-05-27T07:59:31.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.193+0000 7f95585a0b80 10 client.17712 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:31.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.193+0000 7f95585a0b80 10 client.17712 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:59:31.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.193+0000 7f95585a0b80 20 client.17712 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:31.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.193+0000 7f95585a0b80 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56350c3f44d0 0x56350c3f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.193+0000 7f95585a0b80 4 client.17712 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:31.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.193+0000 7f95585a0b80 1 -- 172.21.15.90:0/1777505477 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56350c301d90 con 0x56350c3f44d0 2024-05-27T07:59:31.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.197+0000 7f9542a00700 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56350c3f44d0 0x56350c3f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.200+0000 7f9542a00700 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56350c3f44d0 0x56350c3f68f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b4e37d0932c0c7fc server_cookie=78e2ecfd10dfbc60 in_seq=0 out_seq=0 2024-05-27T07:59:31.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.200+0000 7f952cc00700 10 client.17712 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:59:31.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.202+0000 7f952cc00700 1 -- 172.21.15.90:0/1777505477 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56350c301d90 con 0x56350c3f44d0 2024-05-27T07:59:31.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.202+0000 7f952cc00700 10 client.17712 handle_command_reply: tid=0 2024-05-27T07:59:31.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.203+0000 7f95585a0b80 1 client.17712 shutdown 2024-05-27T07:59:31.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.203+0000 7f95585a0b80 1 -- 172.21.15.90:0/1777505477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c33e9f0 msgr2=0x56350c3dbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:31.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.203+0000 7f95585a0b80 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c33e9f0 0x56350c3dbbc0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f95300e0b90 tx=0x7f95300a9e90 comp rx=0 tx=0).stop 2024-05-27T07:59:31.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.204+0000 7f95585a0b80 1 -- 172.21.15.90:0/1777505477 shutdown_connections 2024-05-27T07:59:31.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.204+0000 7f95585a0b80 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56350c3f44d0 0x56350c3f68f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.204+0000 7f95585a0b80 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3dc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.204+0000 7f95585a0b80 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c252e40 0x56350c3db680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.204+0000 7f95585a0b80 1 --2- 172.21.15.90:0/1777505477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c33e9f0 0x56350c3dbbc0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.204+0000 7f95585a0b80 1 -- 172.21.15.90:0/1777505477 >> 172.21.15.90:0/1777505477 conn(0x56350c2ec390 msgr2=0x56350c3d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:31.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.204+0000 7f952cc00700 0 client.17712 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:59:31.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.205+0000 7f95585a0b80 1 -- 172.21.15.90:0/1777505477 shutdown_connections 2024-05-27T07:59:31.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.205+0000 7f95585a0b80 1 -- 172.21.15.90:0/1777505477 wait complete. 2024-05-27T07:59:31.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.205+0000 7f95585a0b80 20 client.17712 trim_cache size 0 max 16384 2024-05-27T07:59:31.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.238+0000 7f95585a0b80 1 Processor -- start 2024-05-27T07:59:31.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.238+0000 7f95585a0b80 1 -- start start 2024-05-27T07:59:31.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.238+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c252e40 0x56350c3d6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.238+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c3d1e70 0x56350c3fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.238+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3d4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.238+0000 7f95585a0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9524010720 con 0x56350c3fc610 2024-05-27T07:59:31.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.238+0000 7f95585a0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f952400d770 con 0x56350c3d1e70 2024-05-27T07:59:31.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.238+0000 7f95585a0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f952400f980 con 0x56350c252e40 2024-05-27T07:59:31.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.239+0000 7f9542000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c3d1e70 0x56350c3fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.240+0000 7f9543400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3d4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.240+0000 7f9543400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3d4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56582/0 (socket says 172.21.15.90:56582) 2024-05-27T07:59:31.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.240+0000 7f9543400700 1 -- 172.21.15.90:0/682773948 learned_addr learned my addr 172.21.15.90:0/682773948 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:31.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.240+0000 7f9542a00700 1 --2- 172.21.15.90:0/682773948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c252e40 0x56350c3d6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.242+0000 7f9542a00700 1 -- 172.21.15.90:0/682773948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c3d1e70 msgr2=0x56350c3fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:31.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.242+0000 7f9542a00700 1 --2- 172.21.15.90:0/682773948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c3d1e70 0x56350c3fc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.242+0000 7f9542a00700 1 -- 172.21.15.90:0/682773948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 msgr2=0x56350c3d4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:31.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.242+0000 7f9542a00700 1 --2- 172.21.15.90:0/682773948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3d4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.242+0000 7f9542a00700 1 -- 172.21.15.90:0/682773948 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350c3d9bf0 con 0x56350c252e40 2024-05-27T07:59:31.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.242+0000 7f9543400700 1 --2- 172.21.15.90:0/682773948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3d4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:31.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.242+0000 7f9542a00700 1 --2- 172.21.15.90:0/682773948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c252e40 0x56350c3d6e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f95340058e0 tx=0x7f953401e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a23a4ba4b9b7710 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:31.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.244+0000 7f952f400700 1 -- 172.21.15.90:0/682773948 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534005420 con 0x56350c252e40 2024-05-27T07:59:31.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.244+0000 7f952f400700 1 -- 172.21.15.90:0/682773948 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56350c301d90 con 0x56350c252e40 2024-05-27T07:59:31.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.244+0000 7f952f400700 1 -- 172.21.15.90:0/682773948 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953400f460 con 0x56350c252e40 2024-05-27T07:59:31.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.245+0000 7f95585a0b80 1 -- 172.21.15.90:0/682773948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c252e40 msgr2=0x56350c3d6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:31.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.245+0000 7f95585a0b80 1 --2- 172.21.15.90:0/682773948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c252e40 0x56350c3d6e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f95340058e0 tx=0x7f953401e000 comp rx=0 tx=0).stop 2024-05-27T07:59:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.246+0000 7f95585a0b80 1 -- 172.21.15.90:0/682773948 shutdown_connections 2024-05-27T07:59:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.246+0000 7f95585a0b80 1 --2- 172.21.15.90:0/682773948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3d4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.246+0000 7f95585a0b80 1 --2- 172.21.15.90:0/682773948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c252e40 0x56350c3d6e90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.246+0000 7f95585a0b80 1 --2- 172.21.15.90:0/682773948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c3d1e70 0x56350c3fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.246+0000 7f95585a0b80 1 -- 172.21.15.90:0/682773948 >> 172.21.15.90:0/682773948 conn(0x56350c2ec390 msgr2=0x56350c3f64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.246+0000 7f95585a0b80 1 -- 172.21.15.90:0/682773948 shutdown_connections 2024-05-27T07:59:31.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.246+0000 7f95585a0b80 1 -- 172.21.15.90:0/682773948 wait complete. 2024-05-27T07:59:31.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.247+0000 7f95585a0b80 1 -- start start 2024-05-27T07:59:31.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.247+0000 7f95585a0b80 1 Processor -- start 2024-05-27T07:59:31.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.248+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c252e40 0x56350c3ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.248+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.248+0000 7f95585a0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.249+0000 7f95585a0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f952400f2b0 con 0x56350c3fc610 2024-05-27T07:59:31.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.249+0000 7f95585a0b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f952400de40 con 0x56350c252e40 2024-05-27T07:59:31.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.249+0000 7f95585a0b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f952400e510 con 0x56350c3d1e70 2024-05-27T07:59:31.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.249+0000 7f95585a0b80 10 client.0 _ll_register_callbacks cb 0x56350c299c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:31.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.249+0000 7f9542000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.249+0000 7f9542000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3ecf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:35902/0 (socket says 172.21.15.90:35902) 2024-05-27T07:59:31.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.249+0000 7f9542000700 1 -- 172.21.15.90:0/3550340916 learned_addr learned my addr 172.21.15.90:0/3550340916 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:31.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.249+0000 7f9542a00700 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c252e40 0x56350c3ec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.249+0000 7f9543400700 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.251+0000 7f9543400700 1 -- 172.21.15.90:0/3550340916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 msgr2=0x56350c3ecf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:31.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.251+0000 7f9543400700 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3ecf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.251+0000 7f9543400700 1 -- 172.21.15.90:0/3550340916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c252e40 msgr2=0x56350c3ec9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:31.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.251+0000 7f9543400700 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c252e40 0x56350c3ec9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.251+0000 7f9543400700 1 -- 172.21.15.90:0/3550340916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350c3e8660 con 0x56350c3fc610 2024-05-27T07:59:31.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.254+0000 7f9543400700 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3e7ec0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f953c003f40 tx=0x7f953c008d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f52dd28f53b796 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:31.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.255+0000 7f952cc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:59:31.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.255+0000 7f952cc00700 1 -- 172.21.15.90:0/3550340916 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c004700 con 0x56350c3fc610 2024-05-27T07:59:31.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.256+0000 7f952cc00700 1 -- 172.21.15.90:0/3550340916 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f953c0048a0 con 0x56350c3fc610 2024-05-27T07:59:31.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.256+0000 7f95585a0b80 1 -- 172.21.15.90:0/3550340916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56350c301d90 con 0x56350c3fc610 2024-05-27T07:59:31.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.256+0000 7f952cc00700 1 -- 172.21.15.90:0/3550340916 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f953c02e050 con 0x56350c3fc610 2024-05-27T07:59:31.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.257+0000 7f95585a0b80 10 client.17718 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:31.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.257+0000 7f95585a0b80 1 -- 172.21.15.90:0/3550340916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56350c45eb20 con 0x56350c3fc610 2024-05-27T07:59:31.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.258+0000 7f952cc00700 1 -- 172.21.15.90:0/3550340916 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56350c301d90 con 0x56350c3fc610 2024-05-27T07:59:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.258+0000 7f952cc00700 1 -- 172.21.15.90:0/3550340916 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f953c015ad0 con 0x56350c3fc610 2024-05-27T07:59:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.258+0000 7f95585a0b80 10 client.17718 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.258+0000 7f95585a0b80 10 client.17718 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T07:59:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.258+0000 7f95585a0b80 20 client.17718 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:31.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.258+0000 7f95585a0b80 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56350c45f180 0x56350c4615a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:31.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.258+0000 7f95585a0b80 4 client.17718 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:59:31.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.258+0000 7f95585a0b80 1 -- 172.21.15.90:0/3550340916 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56350c3cc120 con 0x56350c45f180 2024-05-27T07:59:31.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.259+0000 7f9542a00700 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56350c45f180 0x56350c4615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:31.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.260+0000 7f9542a00700 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56350c45f180 0x56350c4615a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c568d985b7b4c7c server_cookie=8592b7fc30917766 in_seq=0 out_seq=0 2024-05-27T07:59:31.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.260+0000 7f952cc00700 10 client.17718 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T07:59:31.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.263+0000 7f952cc00700 1 -- 172.21.15.90:0/3550340916 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56350c3cc120 con 0x56350c45f180 2024-05-27T07:59:31.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.263+0000 7f952cc00700 10 client.17718 handle_command_reply: tid=0 2024-05-27T07:59:31.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.263+0000 7f95585a0b80 1 client.17718 shutdown 2024-05-27T07:59:31.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.264+0000 7f95585a0b80 1 -- 172.21.15.90:0/3550340916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 msgr2=0x56350c3e7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:31.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.264+0000 7f95585a0b80 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3e7ec0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f953c003f40 tx=0x7f953c008d00 comp rx=0 tx=0).stop 2024-05-27T07:59:31.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.265+0000 7f95585a0b80 1 -- 172.21.15.90:0/3550340916 shutdown_connections 2024-05-27T07:59:31.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.265+0000 7f95585a0b80 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56350c45f180 0x56350c4615a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.265+0000 7f952cc00700 0 client.17718 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T07:59:31.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.265+0000 7f95585a0b80 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56350c3fc610 0x56350c3e7ec0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.265+0000 7f95585a0b80 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56350c252e40 0x56350c3ec9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.265+0000 7f95585a0b80 1 --2- 172.21.15.90:0/3550340916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56350c3d1e70 0x56350c3ecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.265+0000 7f95585a0b80 1 -- 172.21.15.90:0/3550340916 >> 172.21.15.90:0/3550340916 conn(0x56350c2ec390 msgr2=0x56350c3d9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:31.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.268+0000 7f95585a0b80 1 -- 172.21.15.90:0/3550340916 shutdown_connections 2024-05-27T07:59:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.268+0000 7f95585a0b80 1 -- 172.21.15.90:0/3550340916 wait complete. 2024-05-27T07:59:31.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.269+0000 7f95585a0b80 20 client.17718 trim_cache size 0 max 16384 2024-05-27T07:59:31.271 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:59:31.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.271+0000 7f9549a00700 1 -- 172.21.15.90:0/1919853075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f95280b2a70 msgr2=0x7f95280b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:31.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.271+0000 7f9549a00700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f95280b2a70 0x7f95280b4f80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9534009990 tx=0x7f9534005fb0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.271+0000 7f9549a00700 1 -- 172.21.15.90:0/1919853075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9544107de0 msgr2=0x7f954406e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:31.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.271+0000 7f9549a00700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9544107de0 0x7f954406e1d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f953000c9e0 tx=0x7f953000ccf0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.271+0000 7f9549a00700 1 -- 172.21.15.90:0/1919853075 shutdown_connections 2024-05-27T07:59:31.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.271+0000 7f9549a00700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9544108730 0x7f954406e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.271+0000 7f9549a00700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95441034b0 0x7f95440750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.271+0000 7f9549a00700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f95280b2a70 0x7f95280b4f80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.271+0000 7f9549a00700 1 --2- 172.21.15.90:0/1919853075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9544107de0 0x7f954406e1d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:31.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.271+0000 7f9549a00700 1 -- 172.21.15.90:0/1919853075 >> 172.21.15.90:0/1919853075 conn(0x7f95440fedb0 msgr2=0x7f9544105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:31.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.272+0000 7f9549a00700 1 -- 172.21.15.90:0/1919853075 shutdown_connections 2024-05-27T07:59:31.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:31.273+0000 7f9549a00700 1 -- 172.21.15.90:0/1919853075 wait complete. 2024-05-27T07:59:32.250 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T07:59:32.811 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:32.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:32 smithi090 ceph-mon[31429]: pgmap v419: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 460 op/s 2024-05-27T07:59:33.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:32 smithi136 ceph-mon[35324]: pgmap v419: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 460 op/s 2024-05-27T07:59:33.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:32 smithi129 ceph-mon[35254]: pgmap v419: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.6 MiB/s wr, 460 op/s 2024-05-27T07:59:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 555.0 s, 465.0 tps, lat 215.149 ms stddev 343.699 2024-05-27T07:59:33.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:59:33.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:33 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:59:33.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:33 smithi090 ceph-mon[31429]: pgmap v420: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 400 op/s 2024-05-27T07:59:33.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:33 smithi090 ceph-mon[31429]: pgmap v421: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-05-27T07:59:33.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:33 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:59:34.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:59:34.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:33 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:59:34.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:33 smithi136 ceph-mon[35324]: pgmap v420: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 400 op/s 2024-05-27T07:59:34.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:33 smithi136 ceph-mon[35324]: pgmap v421: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-05-27T07:59:34.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:33 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:59:34.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T07:59:34.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:33 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T07:59:34.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:33 smithi129 ceph-mon[35254]: pgmap v420: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 400 op/s 2024-05-27T07:59:34.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:33 smithi129 ceph-mon[35254]: pgmap v421: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.3 MiB/s wr, 404 op/s 2024-05-27T07:59:34.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:33 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T07:59:35.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:34 smithi136 ceph-mon[35324]: pgmap v422: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 522 op/s 2024-05-27T07:59:35.335 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:34 smithi129 ceph-mon[35254]: pgmap v422: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 522 op/s 2024-05-27T07:59:35.412 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:34 smithi090 ceph-mon[31429]: pgmap v422: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 522 op/s 2024-05-27T07:59:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.243+0000 7f983b200640 1 -- 172.21.15.90:0/2245806895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9834105cf0 msgr2=0x7f9834110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.243+0000 7f983b200640 1 --2- 172.21.15.90:0/2245806895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9834105cf0 0x7f9834110690 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9830009ca0 tx=0x7f983002a950 comp rx=0 tx=0).stop 2024-05-27T07:59:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.244+0000 7f983b200640 1 -- 172.21.15.90:0/2245806895 shutdown_connections 2024-05-27T07:59:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.244+0000 7f983b200640 1 --2- 172.21.15.90:0/2245806895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9834105cf0 0x7f9834110690 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.244+0000 7f983b200640 1 --2- 172.21.15.90:0/2245806895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9834104260 0x7f9834105680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.244+0000 7f983b200640 1 --2- 172.21.15.90:0/2245806895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9834103e50 0x7f9834105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.244+0000 7f983b200640 1 -- 172.21.15.90:0/2245806895 >> 172.21.15.90:0/2245806895 conn(0x7f98340791a0 msgr2=0x7f98340795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.245+0000 7f983b200640 1 -- 172.21.15.90:0/2245806895 shutdown_connections 2024-05-27T07:59:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.245+0000 7f983b200640 1 -- 172.21.15.90:0/2245806895 wait complete. 2024-05-27T07:59:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.246+0000 7f983b200640 1 Processor -- start 2024-05-27T07:59:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.247+0000 7f983b200640 1 -- start start 2024-05-27T07:59:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.247+0000 7f983b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9834103e50 0x7f98341a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.247+0000 7f983b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9834104260 0x7f98341a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.247+0000 7f983b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9834105cf0 0x7f98341ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.247+0000 7f983b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9834113bf0 con 0x7f9834103e50 2024-05-27T07:59:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.247+0000 7f983b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9834113a70 con 0x7f9834104260 2024-05-27T07:59:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.247+0000 7f983b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9834113d70 con 0x7f9834105cf0 2024-05-27T07:59:37.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.249+0000 7f9839400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9834104260 0x7f98341a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.249+0000 7f9839400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9834104260 0x7f98341a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38746/0 (socket says 172.21.15.90:38746) 2024-05-27T07:59:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.249+0000 7f9839400640 1 -- 172.21.15.90:0/322599993 learned_addr learned my addr 172.21.15.90:0/322599993 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:37.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:36 smithi136 ceph-mon[35324]: pgmap v423: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.1 MiB/s wr, 544 op/s 2024-05-27T07:59:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.249+0000 7f9839e00640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9834103e50 0x7f98341a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.250+0000 7f983a800640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9834105cf0 0x7f98341ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.253+0000 7f9839e00640 1 -- 172.21.15.90:0/322599993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9834105cf0 msgr2=0x7f98341ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.253+0000 7f9839e00640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9834105cf0 0x7f98341ac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.253+0000 7f9839e00640 1 -- 172.21.15.90:0/322599993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9834104260 msgr2=0x7f98341a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.253+0000 7f9839e00640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9834104260 0x7f98341a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.253+0000 7f9839e00640 1 -- 172.21.15.90:0/322599993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98341ace90 con 0x7f9834103e50 2024-05-27T07:59:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.253+0000 7f983a800640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9834105cf0 0x7f98341ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.253+0000 7f9839400640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9834104260 0x7f98341a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.254+0000 7f9839e00640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9834103e50 0x7f98341a18d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f982800d6d0 tx=0x7f982800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.254+0000 7f982ea00640 1 -- 172.21.15.90:0/322599993 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828017070 con 0x7f9834103e50 2024-05-27T07:59:37.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.255+0000 7f983b200640 1 -- 172.21.15.90:0/322599993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98341ad0c0 con 0x7f9834103e50 2024-05-27T07:59:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.255+0000 7f983b200640 1 -- 172.21.15.90:0/322599993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98341ad660 con 0x7f9834103e50 2024-05-27T07:59:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.255+0000 7f982ea00640 1 -- 172.21.15.90:0/322599993 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9828009820 con 0x7f9834103e50 2024-05-27T07:59:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.256+0000 7f982ea00640 1 -- 172.21.15.90:0/322599993 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828004bc0 con 0x7f9834103e50 2024-05-27T07:59:37.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.260+0000 7f98437cc740 1 Processor -- start 2024-05-27T07:59:37.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.261+0000 7f98437cc740 1 -- start start 2024-05-27T07:59:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.261+0000 7f98437cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 0x55c6cc5f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.261+0000 7f98437cc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc5f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.261+0000 7f98437cc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 0x55c6cc5f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.261+0000 7f98437cc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc564b90 con 0x55c6cc2304d0 2024-05-27T07:59:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.262+0000 7f98437cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc554730 con 0x55c6cc2511b0 2024-05-27T07:59:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.262+0000 7f98437cc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc558770 con 0x55c6cc2282f0 2024-05-27T07:59:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.262+0000 7f983a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 0x55c6cc5f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.262+0000 7f9839400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 0x55c6cc5f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.262+0000 7f983a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 0x55c6cc5f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38762/0 (socket says 172.21.15.90:38762) 2024-05-27T07:59:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.262+0000 7f983a800640 1 -- 172.21.15.90:0/1016747832 learned_addr learned my addr 172.21.15.90:0/1016747832 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.263+0000 7f9839400640 1 -- 172.21.15.90:0/1016747832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 msgr2=0x55c6cc5f3fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T07:59:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.263+0000 7f9839400640 1 --2- 172.21.15.90:0/1016747832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc5f3fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.263+0000 7f9839400640 1 -- 172.21.15.90:0/1016747832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 msgr2=0x55c6cc5f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.263+0000 7f9839400640 1 --2- 172.21.15.90:0/1016747832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 0x55c6cc5f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.263+0000 7f9839400640 1 -- 172.21.15.90:0/1016747832 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cc4ed8b0 con 0x55c6cc2304d0 2024-05-27T07:59:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.264+0000 7f9839400640 1 --2- 172.21.15.90:0/1016747832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 0x55c6cc5f44e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f9824010450 tx=0x7f982400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43c0aff32ff66f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.264+0000 7f9823000640 1 -- 172.21.15.90:0/1016747832 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824011820 con 0x55c6cc2304d0 2024-05-27T07:59:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.265+0000 7f9823000640 1 -- 172.21.15.90:0/1016747832 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9824011e50 con 0x55c6cc2304d0 2024-05-27T07:59:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.265+0000 7f9823000640 1 -- 172.21.15.90:0/1016747832 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824011820 con 0x55c6cc2304d0 2024-05-27T07:59:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.266+0000 7f982ea00640 1 -- 172.21.15.90:0/322599993 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9828005220 con 0x7f9834103e50 2024-05-27T07:59:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.266+0000 7f98437cc740 1 -- 172.21.15.90:0/1016747832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 msgr2=0x55c6cc5f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.266+0000 7f98437cc740 1 --2- 172.21.15.90:0/1016747832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 0x55c6cc5f44e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f9824010450 tx=0x7f982400bf90 comp rx=0 tx=0).stop 2024-05-27T07:59:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.267+0000 7f98437cc740 1 -- 172.21.15.90:0/1016747832 shutdown_connections 2024-05-27T07:59:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.267+0000 7f98437cc740 1 --2- 172.21.15.90:0/1016747832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 0x55c6cc5f44e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.267+0000 7f98437cc740 1 --2- 172.21.15.90:0/1016747832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc5f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.267+0000 7f98437cc740 1 --2- 172.21.15.90:0/1016747832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 0x55c6cc5f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.267+0000 7f98437cc740 1 -- 172.21.15.90:0/1016747832 >> 172.21.15.90:0/1016747832 conn(0x55c6cc245300 msgr2=0x55c6cc5f0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.268+0000 7f98437cc740 1 -- 172.21.15.90:0/1016747832 shutdown_connections 2024-05-27T07:59:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.268+0000 7f98437cc740 1 -- 172.21.15.90:0/1016747832 wait complete. 2024-05-27T07:59:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.269+0000 7f982ea00640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f981c0b2940 0x7f981c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.269+0000 7f98437cc740 1 -- start start 2024-05-27T07:59:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.269+0000 7f982ea00640 1 -- 172.21.15.90:0/322599993 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f98280e21d0 con 0x7f9834103e50 2024-05-27T07:59:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.270+0000 7f98437cc740 1 Processor -- start 2024-05-27T07:59:37.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.270+0000 7f983a800640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f981c0b2940 0x7f981c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.271+0000 7f98437cc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc5fdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.271+0000 7f98437cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2304d0 0x55c6cc5fdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.271+0000 7f98437cc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 0x55c6cc5fe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.271+0000 7f983a800640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f981c0b2940 0x7f981c0b4e50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9830009bf0 tx=0x7f9830005dc0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.272+0000 7f98437cc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc23a790 con 0x55c6cc2511b0 2024-05-27T07:59:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.272+0000 7f98437cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc4fcbc0 con 0x55c6cc2304d0 2024-05-27T07:59:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.272+0000 7f98437cc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c6cc600be0 con 0x55c6cc2282f0 2024-05-27T07:59:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.272+0000 7f98437cc740 10 client.0 _ll_register_callbacks cb 0x55c6cc4da400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.272+0000 7f983a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 0x55c6cc5fe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.272+0000 7f983a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 0x55c6cc5fe520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56618/0 (socket says 172.21.15.90:56618) 2024-05-27T07:59:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.272+0000 7f983a800640 1 -- 172.21.15.90:0/931699092 learned_addr learned my addr 172.21.15.90:0/931699092 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.272+0000 7f9839400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2304d0 0x55c6cc5fdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.273+0000 7f983a800640 1 -- 172.21.15.90:0/931699092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 msgr2=0x55c6cc5fdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.273+0000 7f983a800640 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc5fdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.273+0000 7f983a800640 1 -- 172.21.15.90:0/931699092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2304d0 msgr2=0x55c6cc5fdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.273+0000 7f983a800640 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2304d0 0x55c6cc5fdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.273+0000 7f983a800640 1 -- 172.21.15.90:0/931699092 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cc269210 con 0x55c6cc2511b0 2024-05-27T07:59:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.274+0000 7f983a800640 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 0x55c6cc5fe520 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9830035000 tx=0x7f983002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33870c38004ae857 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:37.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.274+0000 7f9817400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:59:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.275+0000 7f9817400640 1 -- 172.21.15.90:0/931699092 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830034040 con 0x55c6cc2511b0 2024-05-27T07:59:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.275+0000 7f9817400640 1 -- 172.21.15.90:0/931699092 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f9830003fe0 con 0x55c6cc2511b0 2024-05-27T07:59:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.275+0000 7f98437cc740 1 -- 172.21.15.90:0/931699092 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9824011e50 con 0x55c6cc2511b0 2024-05-27T07:59:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.276+0000 7f9817400640 1 -- 172.21.15.90:0/931699092 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f983002fea0 con 0x55c6cc2511b0 2024-05-27T07:59:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.276+0000 7f98437cc740 10 client.17730 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.276+0000 7f98437cc740 1 -- 172.21.15.90:0/931699092 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55c6cc335620 con 0x55c6cc2511b0 2024-05-27T07:59:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.277+0000 7f9817400640 1 -- 172.21.15.90:0/931699092 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9824011e50 con 0x55c6cc2511b0 2024-05-27T07:59:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.277+0000 7f9817400640 1 -- 172.21.15.90:0/931699092 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f9830052980 con 0x55c6cc2511b0 2024-05-27T07:59:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.277+0000 7f98437cc740 10 client.17730 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.277+0000 7f98437cc740 10 client.17730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.277+0000 7f98437cc740 20 client.17730 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.278+0000 7f98437cc740 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c6cc20fd80 0x55c6cc6100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.278+0000 7f98437cc740 4 client.17730 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.278+0000 7f98437cc740 1 -- 172.21.15.90:0/931699092 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6cc566ae0 con 0x55c6cc20fd80 2024-05-27T07:59:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.279+0000 7f9839e00640 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c6cc20fd80 0x55c6cc6100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.281+0000 7f9839e00640 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c6cc20fd80 0x55c6cc6100d0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a39acebb028c2810 server_cookie=8008b5aef77f1ed0 in_seq=0 out_seq=0 2024-05-27T07:59:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.281+0000 7f9817400640 10 client.17730 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.284+0000 7f9817400640 1 -- 172.21.15.90:0/931699092 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c6cc566ae0 con 0x55c6cc20fd80 2024-05-27T07:59:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.284+0000 7f9817400640 10 client.17730 handle_command_reply: tid=0 2024-05-27T07:59:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.284+0000 7f98437cc740 1 client.17730 shutdown 2024-05-27T07:59:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.285+0000 7f98437cc740 1 -- 172.21.15.90:0/931699092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 msgr2=0x55c6cc5fe520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.285+0000 7f98437cc740 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 0x55c6cc5fe520 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9830035000 tx=0x7f983002a950 comp rx=0 tx=0).stop 2024-05-27T07:59:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.285+0000 7f98437cc740 1 -- 172.21.15.90:0/931699092 shutdown_connections 2024-05-27T07:59:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.285+0000 7f98437cc740 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c6cc20fd80 0x55c6cc6100d0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.285+0000 7f98437cc740 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2304d0 0x55c6cc5fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.285+0000 7f98437cc740 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 0x55c6cc5fe520 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.285+0000 7f98437cc740 1 --2- 172.21.15.90:0/931699092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc5fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.286+0000 7f98437cc740 1 -- 172.21.15.90:0/931699092 >> 172.21.15.90:0/931699092 conn(0x55c6cc245300 msgr2=0x55c6cc5f5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.286+0000 7f9817400640 0 client.17730 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.286+0000 7f98437cc740 1 -- 172.21.15.90:0/931699092 shutdown_connections 2024-05-27T07:59:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.286+0000 7f98437cc740 1 -- 172.21.15.90:0/931699092 wait complete. 2024-05-27T07:59:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.286+0000 7f98437cc740 20 client.17730 trim_cache size 0 max 16384 2024-05-27T07:59:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.318+0000 7f98437cc740 1 Processor -- start 2024-05-27T07:59:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.318+0000 7f98437cc740 1 -- start start 2024-05-27T07:59:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.319+0000 7f98437cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2282f0 0x55c6cc5ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.319+0000 7f98437cc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2304d0 0x55c6cc5fe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.319+0000 7f98437cc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 0x55c6cc615890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.319+0000 7f98437cc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9818011ab0 con 0x55c6cc2511b0 2024-05-27T07:59:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.319+0000 7f98437cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f981800eb00 con 0x55c6cc2282f0 2024-05-27T07:59:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.319+0000 7f98437cc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9818010d10 con 0x55c6cc2304d0 2024-05-27T07:59:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.319+0000 7f983a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 0x55c6cc615890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.320+0000 7f9839400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2304d0 0x55c6cc5fe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.320+0000 7f9839400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2304d0 0x55c6cc5fe340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:35928/0 (socket says 172.21.15.90:35928) 2024-05-27T07:59:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.320+0000 7f9839400640 1 -- 172.21.15.90:0/3045759755 learned_addr learned my addr 172.21.15.90:0/3045759755 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.320+0000 7f9839e00640 1 --2- 172.21.15.90:0/3045759755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2282f0 0x55c6cc5ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.321+0000 7f9839400640 1 -- 172.21.15.90:0/3045759755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2282f0 msgr2=0x55c6cc5ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.321+0000 7f9839400640 1 --2- 172.21.15.90:0/3045759755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2282f0 0x55c6cc5ff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.321+0000 7f9839400640 1 -- 172.21.15.90:0/3045759755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 msgr2=0x55c6cc615890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.321+0000 7f9839400640 1 --2- 172.21.15.90:0/3045759755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 0x55c6cc615890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.321+0000 7f9839400640 1 -- 172.21.15.90:0/3045759755 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cc600ca0 con 0x55c6cc2304d0 2024-05-27T07:59:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.322+0000 7f9839400640 1 --2- 172.21.15.90:0/3045759755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2304d0 0x55c6cc5fe340 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9824010da0 tx=0x7f9824005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=150c2b574fb2a509 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.323+0000 7f9822600640 1 -- 172.21.15.90:0/3045759755 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98240054f0 con 0x55c6cc2304d0 2024-05-27T07:59:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.323+0000 7f9822600640 1 -- 172.21.15.90:0/3045759755 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9824004af0 con 0x55c6cc2304d0 2024-05-27T07:59:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.323+0000 7f9822600640 1 -- 172.21.15.90:0/3045759755 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824011af0 con 0x55c6cc2304d0 2024-05-27T07:59:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.325+0000 7f98437cc740 1 -- 172.21.15.90:0/3045759755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2304d0 msgr2=0x55c6cc5fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.325+0000 7f98437cc740 1 --2- 172.21.15.90:0/3045759755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2304d0 0x55c6cc5fe340 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9824010da0 tx=0x7f9824005280 comp rx=0 tx=0).stop 2024-05-27T07:59:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.327+0000 7f98437cc740 1 -- 172.21.15.90:0/3045759755 shutdown_connections 2024-05-27T07:59:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.327+0000 7f98437cc740 1 --2- 172.21.15.90:0/3045759755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2511b0 0x55c6cc615890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.327+0000 7f98437cc740 1 --2- 172.21.15.90:0/3045759755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2304d0 0x55c6cc5fe340 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.327+0000 7f98437cc740 1 --2- 172.21.15.90:0/3045759755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2282f0 0x55c6cc5ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.328+0000 7f98437cc740 1 -- 172.21.15.90:0/3045759755 >> 172.21.15.90:0/3045759755 conn(0x55c6cc245300 msgr2=0x55c6cc5ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.328+0000 7f98437cc740 1 -- 172.21.15.90:0/3045759755 shutdown_connections 2024-05-27T07:59:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.329+0000 7f98437cc740 1 -- 172.21.15.90:0/3045759755 wait complete. 2024-05-27T07:59:37.331 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:36 smithi129 ceph-mon[35254]: pgmap v423: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.1 MiB/s wr, 544 op/s 2024-05-27T07:59:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.330+0000 7f98437cc740 1 -- start start 2024-05-27T07:59:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.330+0000 7f98437cc740 1 Processor -- start 2024-05-27T07:59:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.332+0000 7f98437cc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc600f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.332+0000 7f98437cc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 0x55c6cc601450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.333+0000 7f98437cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 0x55c6cc601990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.333+0000 7f9839400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 0x55c6cc601450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.334+0000 7f9839e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc600f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.334+0000 7f9839e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc600f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:35932/0 (socket says 172.21.15.90:35932) 2024-05-27T07:59:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.334+0000 7f98437cc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9818010640 con 0x55c6cc2304d0 2024-05-27T07:59:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.334+0000 7f98437cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f981800f1d0 con 0x55c6cc2511b0 2024-05-27T07:59:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.334+0000 7f98437cc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f981800f8a0 con 0x55c6cc2282f0 2024-05-27T07:59:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.334+0000 7f9839e00640 1 -- 172.21.15.90:0/2782949695 learned_addr learned my addr 172.21.15.90:0/2782949695 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.334+0000 7f98437cc740 10 client.0 _ll_register_callbacks cb 0x7f9830009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.335+0000 7f983a800640 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 0x55c6cc601990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.336+0000 7f9839e00640 1 -- 172.21.15.90:0/2782949695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 msgr2=0x55c6cc601990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.336+0000 7f9839e00640 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 0x55c6cc601990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.336+0000 7f9839e00640 1 -- 172.21.15.90:0/2782949695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 msgr2=0x55c6cc601450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.336+0000 7f9839e00640 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 0x55c6cc601450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.336+0000 7f9839e00640 1 -- 172.21.15.90:0/2782949695 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cc676900 con 0x55c6cc2282f0 2024-05-27T07:59:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.336+0000 7f983a800640 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 0x55c6cc601990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.336+0000 7f9839400640 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 0x55c6cc601450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.337+0000 7f9839e00640 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc600f10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9828009610 tx=0x7f98280a8970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55e904309281663d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.337+0000 7f9817400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:59:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.339+0000 7f9817400640 1 -- 172.21.15.90:0/2782949695 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828014070 con 0x55c6cc2282f0 2024-05-27T07:59:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.339+0000 7f9817400640 1 -- 172.21.15.90:0/2782949695 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f98280ad160 con 0x55c6cc2282f0 2024-05-27T07:59:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.339+0000 7f98437cc740 1 -- 172.21.15.90:0/2782949695 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9824004af0 con 0x55c6cc2282f0 2024-05-27T07:59:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.340+0000 7f9817400640 1 -- 172.21.15.90:0/2782949695 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f9828020040 con 0x55c6cc2282f0 2024-05-27T07:59:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.342+0000 7f98437cc740 10 client.27439 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.342+0000 7f98437cc740 1 -- 172.21.15.90:0/2782949695 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55c6cc677b90 con 0x55c6cc2282f0 2024-05-27T07:59:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.342+0000 7f9817400640 1 -- 172.21.15.90:0/2782949695 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f98280df7d0 con 0x55c6cc2282f0 2024-05-27T07:59:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.343+0000 7f9817400640 1 -- 172.21.15.90:0/2782949695 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f98280a4be0 con 0x55c6cc2282f0 2024-05-27T07:59:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.343+0000 7f98437cc740 10 client.27439 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.343+0000 7f98437cc740 10 client.27439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.343+0000 7f98437cc740 20 client.27439 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.343+0000 7f98437cc740 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c6cc1e51c0 0x55c6cc67a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.343+0000 7f98437cc740 4 client.27439 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:59:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.343+0000 7f98437cc740 1 -- 172.21.15.90:0/2782949695 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9824004af0 con 0x55c6cc1e51c0 2024-05-27T07:59:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.344+0000 7f9839400640 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c6cc1e51c0 0x55c6cc67a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.345+0000 7f9839400640 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c6cc1e51c0 0x55c6cc67a2d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=503f31e988d042e8 server_cookie=6471dd7dcd42ee0d in_seq=0 out_seq=0 2024-05-27T07:59:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.345+0000 7f9817400640 10 client.27439 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:37.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:36 smithi090 ceph-mon[31429]: pgmap v423: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.1 MiB/s wr, 544 op/s 2024-05-27T07:59:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.430+0000 7f9817400640 1 -- 172.21.15.90:0/2782949695 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f9824004af0 con 0x55c6cc1e51c0 2024-05-27T07:59:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.430+0000 7f9817400640 10 client.27439 handle_command_reply: tid=0 2024-05-27T07:59:37.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.430+0000 7f98437cc740 1 client.27439 shutdown 2024-05-27T07:59:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.430+0000 7f98437cc740 1 -- 172.21.15.90:0/2782949695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 msgr2=0x55c6cc600f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.430+0000 7f98437cc740 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc600f10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9828009610 tx=0x7f98280a8970 comp rx=0 tx=0).stop 2024-05-27T07:59:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.431+0000 7f98437cc740 1 -- 172.21.15.90:0/2782949695 shutdown_connections 2024-05-27T07:59:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.431+0000 7f98437cc740 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c6cc1e51c0 0x55c6cc67a2d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.431+0000 7f98437cc740 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c6cc2511b0 0x55c6cc601990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.431+0000 7f98437cc740 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6cc2304d0 0x55c6cc601450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.431+0000 7f98437cc740 1 --2- 172.21.15.90:0/2782949695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6cc2282f0 0x55c6cc600f10 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9828009610 tx=0x7f98280a8970 comp rx=0 tx=0).stop 2024-05-27T07:59:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.431+0000 7f98437cc740 1 -- 172.21.15.90:0/2782949695 >> 172.21.15.90:0/2782949695 conn(0x55c6cc245300 msgr2=0x55c6cc5f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:37.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.431+0000 7f9817400640 0 client.27439 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.431+0000 7f98437cc740 1 -- 172.21.15.90:0/2782949695 shutdown_connections 2024-05-27T07:59:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.431+0000 7f98437cc740 1 -- 172.21.15.90:0/2782949695 wait complete. 2024-05-27T07:59:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.432+0000 7f98437cc740 20 client.27439 trim_cache size 0 max 16384 2024-05-27T07:59:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:59:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:59:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:59:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.432+0000 7f983b200640 1 -- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f981c0b2940 msgr2=0x7f981c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.433+0000 7f983b200640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f981c0b2940 0x7f981c0b4e50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9830009bf0 tx=0x7f9830005dc0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.433+0000 7f983b200640 1 -- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9834103e50 msgr2=0x7f98341a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:37.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.433+0000 7f983b200640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9834103e50 0x7f98341a18d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f982800d6d0 tx=0x7f982800db90 comp rx=0 tx=0).stop 2024-05-27T07:59:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.433+0000 7f983b200640 1 -- 172.21.15.90:0/322599993 shutdown_connections 2024-05-27T07:59:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.433+0000 7f983b200640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f981c0b2940 0x7f981c0b4e50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.433+0000 7f983b200640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9834104260 0x7f98341a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.433+0000 7f983b200640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9834105cf0 0x7f98341ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.433+0000 7f983b200640 1 --2- 172.21.15.90:0/322599993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9834103e50 0x7f98341a18d0 secure :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f982800d6d0 tx=0x7f982800db90 comp rx=0 tx=0).stop 2024-05-27T07:59:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.433+0000 7f983b200640 1 -- 172.21.15.90:0/322599993 >> 172.21.15.90:0/322599993 conn(0x7f98340791a0 msgr2=0x7f9834106250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.433+0000 7f983b200640 1 -- 172.21.15.90:0/322599993 shutdown_connections 2024-05-27T07:59:37.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:37.434+0000 7f983b200640 1 -- 172.21.15.90:0/322599993 wait complete. 2024-05-27T07:59:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 560.0 s, 567.4 tps, lat 174.968 ms stddev 104.231 2024-05-27T07:59:39.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:38 smithi136 ceph-mon[35324]: pgmap v424: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 565 op/s 2024-05-27T07:59:39.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:38 smithi129 ceph-mon[35254]: pgmap v424: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 565 op/s 2024-05-27T07:59:39.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:38 smithi090 ceph-mon[31429]: pgmap v424: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.1 MiB/s wr, 565 op/s 2024-05-27T07:59:40.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.898+0000 7f3f7be00700 1 -- 172.21.15.90:0/386243643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f7c110fc0 msgr2=0x7f3f7c1176c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:40.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.898+0000 7f3f7be00700 1 --2- 172.21.15.90:0/386243643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f7c110fc0 0x7f3f7c1176c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3f740092b0 tx=0x7f3f740095c0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.899+0000 7f3f7be00700 1 -- 172.21.15.90:0/386243643 shutdown_connections 2024-05-27T07:59:40.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.899+0000 7f3f7be00700 1 --2- 172.21.15.90:0/386243643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f7c110fc0 0x7f3f7c1176c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.899+0000 7f3f7be00700 1 --2- 172.21.15.90:0/386243643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f7c10af00 0x7f3f7c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.899+0000 7f3f7be00700 1 --2- 172.21.15.90:0/386243643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f7c10b3a0 0x7f3f7c110930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.899+0000 7f3f7be00700 1 -- 172.21.15.90:0/386243643 >> 172.21.15.90:0/386243643 conn(0x7f3f7c06f280 msgr2=0x7f3f7c06f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.900+0000 7f3f7be00700 1 -- 172.21.15.90:0/386243643 shutdown_connections 2024-05-27T07:59:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.900+0000 7f3f7be00700 1 -- 172.21.15.90:0/386243643 wait complete. 2024-05-27T07:59:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.904+0000 7f3f7be00700 1 Processor -- start 2024-05-27T07:59:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.904+0000 7f3f7be00700 1 -- start start 2024-05-27T07:59:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.905+0000 7f3f7be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f7c10af00 0x7f3f7c118700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.905+0000 7f3f7be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f7c110fc0 0x7f3f7c118c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.905+0000 7f3f7be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f7c119180 0x7f3f7c1b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.905+0000 7f3f7be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c11ebd0 con 0x7f3f7c119180 2024-05-27T07:59:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.905+0000 7f3f7be00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c11ea50 con 0x7f3f7c110fc0 2024-05-27T07:59:40.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.905+0000 7f3f7be00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c11ed50 con 0x7f3f7c10af00 2024-05-27T07:59:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.906+0000 7f3f7b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f7c119180 0x7f3f7c1b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.906+0000 7f3f7b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f7c119180 0x7f3f7c1b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35238/0 (socket says 172.21.15.90:35238) 2024-05-27T07:59:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.906+0000 7f3f7b400700 1 -- 172.21.15.90:0/1629392856 learned_addr learned my addr 172.21.15.90:0/1629392856 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.906+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f7c10af00 0x7f3f7c118700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.907+0000 7f3f7b400700 1 -- 172.21.15.90:0/1629392856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f7c10af00 msgr2=0x7f3f7c118700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.907+0000 7f3f7b400700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f7c10af00 0x7f3f7c118700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.907+0000 7f3f7b400700 1 -- 172.21.15.90:0/1629392856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f7c110fc0 msgr2=0x7f3f7c118c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:59:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.907+0000 7f3f7b400700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f7c110fc0 0x7f3f7c118c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.907+0000 7f3f7b400700 1 -- 172.21.15.90:0/1629392856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c1b5c40 con 0x7f3f7c119180 2024-05-27T07:59:40.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.909+0000 7f3f7b400700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f7c119180 0x7f3f7c1b5700 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3f74006010 tx=0x7f3f7400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:40.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.910+0000 7f3f73e00700 1 -- 172.21.15.90:0/1629392856 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74004230 con 0x7f3f7c119180 2024-05-27T07:59:40.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.910+0000 7f3f7be00700 1 -- 172.21.15.90:0/1629392856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c1b5e10 con 0x7f3f7c119180 2024-05-27T07:59:40.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.910+0000 7f3f7be00700 1 -- 172.21.15.90:0/1629392856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c1b61b0 con 0x7f3f7c119180 2024-05-27T07:59:40.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.911+0000 7f3f73e00700 1 -- 172.21.15.90:0/1629392856 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f7400f460 con 0x7f3f7c119180 2024-05-27T07:59:40.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.911+0000 7f3f73e00700 1 -- 172.21.15.90:0/1629392856 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74030060 con 0x7f3f7c119180 2024-05-27T07:59:40.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.918+0000 7f3f73e00700 1 -- 172.21.15.90:0/1629392856 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3f74022420 con 0x7f3f7c119180 2024-05-27T07:59:40.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.921+0000 7f3f73e00700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3f600b2a50 0x7f3f600b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.923+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3f600b2a50 0x7f3f600b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.924+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3f600b2a50 0x7f3f600b4f60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00a400 tx=0x7f3f6c00d040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:40.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.925+0000 7f3f73e00700 1 -- 172.21.15.90:0/1629392856 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f3f740a7a80 con 0x7f3f7c119180 2024-05-27T07:59:40.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.926+0000 7f3f90768b80 1 Processor -- start 2024-05-27T07:59:40.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.926+0000 7f3f90768b80 1 -- start start 2024-05-27T07:59:40.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.927+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9549f0 0x55780d9e8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.927+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d868e40 0x55780d9e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.927+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9f22c0 0x55780d9f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.927+0000 7f3f90768b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55780d915e40 con 0x55780d868e40 2024-05-27T07:59:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.927+0000 7f3f90768b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55780d9059e0 con 0x55780d9f22c0 2024-05-27T07:59:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.927+0000 7f3f90768b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55780d909a20 con 0x55780d9549f0 2024-05-27T07:59:40.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.928+0000 7f3f7aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9549f0 0x55780d9e8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.928+0000 7f3f7aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9549f0 0x55780d9e8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36684/0 (socket says 172.21.15.90:36684) 2024-05-27T07:59:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.928+0000 7f3f7aa00700 1 -- 172.21.15.90:0/1189862850 learned_addr learned my addr 172.21.15.90:0/1189862850 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.928+0000 7f3f72000700 1 --2- 172.21.15.90:0/1189862850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d868e40 0x55780d9e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.929+0000 7f3f7b400700 1 --2- 172.21.15.90:0/1189862850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9f22c0 0x55780d9f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.929+0000 7f3f7aa00700 1 -- 172.21.15.90:0/1189862850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9f22c0 msgr2=0x55780d9f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.929+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/1189862850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9f22c0 0x55780d9f46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.929+0000 7f3f7aa00700 1 -- 172.21.15.90:0/1189862850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d868e40 msgr2=0x55780d9e8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.929+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/1189862850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d868e40 0x55780d9e8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.929+0000 7f3f7aa00700 1 -- 172.21.15.90:0/1189862850 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55780d8ce420 con 0x55780d9549f0 2024-05-27T07:59:40.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.931+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/1189862850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9549f0 0x55780d9e8750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009900 tx=0x7f3f6c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cc69d76940236 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:40.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.933+0000 7f3f6be00700 1 -- 172.21.15.90:0/1189862850 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0205f0 con 0x55780d9549f0 2024-05-27T07:59:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.934+0000 7f3f6be00700 1 -- 172.21.15.90:0/1189862850 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f6c025070 con 0x55780d9549f0 2024-05-27T07:59:40.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.934+0000 7f3f6be00700 1 -- 172.21.15.90:0/1189862850 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0158c0 con 0x55780d9549f0 2024-05-27T07:59:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.935+0000 7f3f90768b80 1 -- 172.21.15.90:0/1189862850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9549f0 msgr2=0x55780d9e8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:40.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.935+0000 7f3f90768b80 1 --2- 172.21.15.90:0/1189862850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9549f0 0x55780d9e8750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009900 tx=0x7f3f6c011000 comp rx=0 tx=0).stop 2024-05-27T07:59:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.939+0000 7f3f90768b80 1 -- 172.21.15.90:0/1189862850 shutdown_connections 2024-05-27T07:59:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.939+0000 7f3f90768b80 1 --2- 172.21.15.90:0/1189862850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9f22c0 0x55780d9f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.939+0000 7f3f90768b80 1 --2- 172.21.15.90:0/1189862850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9549f0 0x55780d9e8750 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.939+0000 7f3f90768b80 1 --2- 172.21.15.90:0/1189862850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d868e40 0x55780d9e8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.939+0000 7f3f90768b80 1 -- 172.21.15.90:0/1189862850 >> 172.21.15.90:0/1189862850 conn(0x55780d902390 msgr2=0x55780d9e3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.939+0000 7f3f90768b80 1 -- 172.21.15.90:0/1189862850 shutdown_connections 2024-05-27T07:59:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.940+0000 7f3f90768b80 1 -- 172.21.15.90:0/1189862850 wait complete. 2024-05-27T07:59:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.941+0000 7f3f90768b80 1 -- start start 2024-05-27T07:59:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.941+0000 7f3f90768b80 1 Processor -- start 2024-05-27T07:59:40.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.942+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d868e40 0x55780d9ef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.942+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9549f0 0x55780d9ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.942+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9f22c0 0x55780d9efce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.942+0000 7f3f90768b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55780d80bfa0 con 0x55780d9549f0 2024-05-27T07:59:40.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.942+0000 7f3f90768b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55780d8a6760 con 0x55780d868e40 2024-05-27T07:59:40.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.942+0000 7f3f90768b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55780d9f4e00 con 0x55780d9f22c0 2024-05-27T07:59:40.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.942+0000 7f3f90768b80 10 client.0 _ll_register_callbacks cb 0x55780d8afc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:40.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.943+0000 7f3f72000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9549f0 0x55780d9ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.943+0000 7f3f7b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9f22c0 0x55780d9efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.943+0000 7f3f72000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9549f0 0x55780d9ef7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35250/0 (socket says 172.21.15.90:35250) 2024-05-27T07:59:40.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.943+0000 7f3f72000700 1 -- 172.21.15.90:0/1210484961 learned_addr learned my addr 172.21.15.90:0/1210484961 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:40.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.944+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d868e40 0x55780d9ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.944+0000 7f3f72000700 1 -- 172.21.15.90:0/1210484961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9f22c0 msgr2=0x55780d9efce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:40.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.944+0000 7f3f72000700 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9f22c0 0x55780d9efce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.945+0000 7f3f72000700 1 -- 172.21.15.90:0/1210484961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d868e40 msgr2=0x55780d9ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:40.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.945+0000 7f3f72000700 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d868e40 0x55780d9ef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.945+0000 7f3f72000700 1 -- 172.21.15.90:0/1210484961 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55780d8558d0 con 0x55780d9549f0 2024-05-27T07:59:40.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.945+0000 7f3f7b400700 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9f22c0 0x55780d9efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:40.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.946+0000 7f3f72000700 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9549f0 0x55780d9ef7a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400bfd0 tx=0x7f3f6400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173cd37c57ce30fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:40.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.946+0000 7f3f68c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:59:40.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.948+0000 7f3f68c00700 1 -- 172.21.15.90:0/1210484961 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64004a30 con 0x55780d9549f0 2024-05-27T07:59:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.948+0000 7f3f68c00700 1 -- 172.21.15.90:0/1210484961 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f3f64004bd0 con 0x55780d9549f0 2024-05-27T07:59:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.948+0000 7f3f90768b80 1 -- 172.21.15.90:0/1210484961 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f6c025070 con 0x55780d9549f0 2024-05-27T07:59:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.950+0000 7f3f68c00700 1 -- 172.21.15.90:0/1210484961 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3f6400d7d0 con 0x55780d9549f0 2024-05-27T07:59:40.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.951+0000 7f3f90768b80 10 client.17745 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:40.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.951+0000 7f3f90768b80 1 -- 172.21.15.90:0/1210484961 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55780d8d6060 con 0x55780d9549f0 2024-05-27T07:59:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.952+0000 7f3f68c00700 1 -- 172.21.15.90:0/1210484961 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f6c025070 con 0x55780d9549f0 2024-05-27T07:59:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.952+0000 7f3f68c00700 1 -- 172.21.15.90:0/1210484961 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f3f6400b040 con 0x55780d9549f0 2024-05-27T07:59:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.952+0000 7f3f90768b80 10 client.17745 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.952+0000 7f3f90768b80 10 client.17745 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:59:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.953+0000 7f3f90768b80 20 client.17745 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.953+0000 7f3f90768b80 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55780da02020 0x55780da044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.953+0000 7f3f90768b80 4 client.17745 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.953+0000 7f3f90768b80 1 -- 172.21.15.90:0/1210484961 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55780d917d90 con 0x55780da02020 2024-05-27T07:59:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.953+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55780da02020 0x55780da044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.955+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55780da02020 0x55780da044b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c61e82b1e1b5f784 server_cookie=e57b7cb20684defb in_seq=0 out_seq=0 2024-05-27T07:59:40.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.955+0000 7f3f68c00700 10 client.17745 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:59:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.956+0000 7f3f68c00700 1 -- 172.21.15.90:0/1210484961 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55780d917d90 con 0x55780da02020 2024-05-27T07:59:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.956+0000 7f3f68c00700 10 client.17745 handle_command_reply: tid=0 2024-05-27T07:59:40.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.957+0000 7f3f90768b80 1 client.17745 shutdown 2024-05-27T07:59:40.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.959+0000 7f3f90768b80 1 -- 172.21.15.90:0/1210484961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9549f0 msgr2=0x55780d9ef7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:40.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.959+0000 7f3f90768b80 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9549f0 0x55780d9ef7a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400bfd0 tx=0x7f3f6400fb60 comp rx=0 tx=0).stop 2024-05-27T07:59:40.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.959+0000 7f3f90768b80 1 -- 172.21.15.90:0/1210484961 shutdown_connections 2024-05-27T07:59:40.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.959+0000 7f3f90768b80 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55780da02020 0x55780da044b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.959+0000 7f3f90768b80 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d9f22c0 0x55780d9efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.959+0000 7f3f90768b80 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d868e40 0x55780d9ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.959+0000 7f3f90768b80 1 --2- 172.21.15.90:0/1210484961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9549f0 0x55780d9ef7a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.959+0000 7f3f68c00700 0 client.17745 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:59:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.959+0000 7f3f90768b80 1 -- 172.21.15.90:0/1210484961 >> 172.21.15.90:0/1210484961 conn(0x55780d902390 msgr2=0x55780d9e3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.959+0000 7f3f90768b80 1 -- 172.21.15.90:0/1210484961 shutdown_connections 2024-05-27T07:59:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.960+0000 7f3f90768b80 1 -- 172.21.15.90:0/1210484961 wait complete. 2024-05-27T07:59:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.960+0000 7f3f90768b80 20 client.17745 trim_cache size 0 max 16384 2024-05-27T07:59:40.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.992+0000 7f3f90768b80 1 Processor -- start 2024-05-27T07:59:40.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.992+0000 7f3f90768b80 1 -- start start 2024-05-27T07:59:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.992+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.992+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9549f0 0x55780d9fdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.992+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9f22c0 0x55780da09cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.992+0000 7f3f90768b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c1c0c20 con 0x55780d9f22c0 2024-05-27T07:59:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.992+0000 7f3f90768b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c1bdc70 con 0x55780d9549f0 2024-05-27T07:59:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.992+0000 7f3f90768b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c1bfe80 con 0x55780d868e40 2024-05-27T07:59:40.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.993+0000 7f3f7aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.993+0000 7f3f7aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9ea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36708/0 (socket says 172.21.15.90:36708) 2024-05-27T07:59:40.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.993+0000 7f3f7aa00700 1 -- 172.21.15.90:0/886038892 learned_addr learned my addr 172.21.15.90:0/886038892 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:40.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.994+0000 7f3f72000700 1 --2- 172.21.15.90:0/886038892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9549f0 0x55780d9fdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:40.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.994+0000 7f3f7aa00700 1 -- 172.21.15.90:0/886038892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9549f0 msgr2=0x55780d9fdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:40.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.994+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/886038892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9549f0 0x55780d9fdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.994+0000 7f3f7aa00700 1 -- 172.21.15.90:0/886038892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9f22c0 msgr2=0x55780da09cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:40.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.994+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/886038892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9f22c0 0x55780da09cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:40.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.994+0000 7f3f7aa00700 1 -- 172.21.15.90:0/886038892 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55780d9f0050 con 0x55780d868e40 2024-05-27T07:59:40.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.995+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/886038892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9ea880 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55780d9e4a90 tx=0x7f3f6c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74cdd038950cd9e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:40.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.996+0000 7f3f6b400700 1 -- 172.21.15.90:0/886038892 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c020830 con 0x55780d868e40 2024-05-27T07:59:40.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.996+0000 7f3f6b400700 1 -- 172.21.15.90:0/886038892 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f6c037070 con 0x55780d868e40 2024-05-27T07:59:40.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.996+0000 7f3f6b400700 1 -- 172.21.15.90:0/886038892 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c032910 con 0x55780d868e40 2024-05-27T07:59:40.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.997+0000 7f3f90768b80 1 -- 172.21.15.90:0/886038892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 msgr2=0x55780d9ea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:40.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.997+0000 7f3f90768b80 1 --2- 172.21.15.90:0/886038892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9ea880 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55780d9e4a90 tx=0x7f3f6c015000 comp rx=0 tx=0).stop 2024-05-27T07:59:41.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.998+0000 7f3f90768b80 1 -- 172.21.15.90:0/886038892 shutdown_connections 2024-05-27T07:59:41.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.998+0000 7f3f90768b80 1 --2- 172.21.15.90:0/886038892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9f22c0 0x55780da09cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.998+0000 7f3f90768b80 1 --2- 172.21.15.90:0/886038892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9ea880 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.998+0000 7f3f90768b80 1 --2- 172.21.15.90:0/886038892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9549f0 0x55780d9fdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.999+0000 7f3f90768b80 1 -- 172.21.15.90:0/886038892 >> 172.21.15.90:0/886038892 conn(0x55780d902390 msgr2=0x55780d9ea170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:41.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.999+0000 7f3f90768b80 1 -- 172.21.15.90:0/886038892 shutdown_connections 2024-05-27T07:59:41.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:40.999+0000 7f3f90768b80 1 -- 172.21.15.90:0/886038892 wait complete. 2024-05-27T07:59:41.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.000+0000 7f3f90768b80 1 -- start start 2024-05-27T07:59:41.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.001+0000 7f3f90768b80 1 Processor -- start 2024-05-27T07:59:41.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.004+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:41.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.004+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9549f0 0x55780d9f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:41.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.004+0000 7f3f90768b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9f22c0 0x55780d9f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:41.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.005+0000 7f3f7aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:41.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.005+0000 7f3f7aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9f50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36720/0 (socket says 172.21.15.90:36720) 2024-05-27T07:59:41.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.005+0000 7f3f7aa00700 1 -- 172.21.15.90:0/4151855382 learned_addr learned my addr 172.21.15.90:0/4151855382 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:41.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.005+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c1bf7b0 con 0x55780d9f22c0 2024-05-27T07:59:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.005+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c1be340 con 0x55780d9549f0 2024-05-27T07:59:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.005+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c1bea10 con 0x55780d868e40 2024-05-27T07:59:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.005+0000 7f3f90768b80 10 client.0 _ll_register_callbacks cb 0x55780d8afc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.005+0000 7f3f72000700 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9549f0 0x55780d9f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.006+0000 7f3f7aa00700 1 -- 172.21.15.90:0/4151855382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9549f0 msgr2=0x55780d9f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.006+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9549f0 0x55780d9f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.006+0000 7f3f7aa00700 1 -- 172.21.15.90:0/4151855382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9f22c0 msgr2=0x55780d9f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.006+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9f22c0 0x55780d9f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.006+0000 7f3f7aa00700 1 -- 172.21.15.90:0/4151855382 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55780da6acf0 con 0x55780d868e40 2024-05-27T07:59:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.007+0000 7f3f7aa00700 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9f50b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c034040 tx=0x7f3f6c0208e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c698f6c0ae7f3158 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.007+0000 7f3f68c00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:59:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.007+0000 7f3f68c00700 1 -- 172.21.15.90:0/4151855382 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0322d0 con 0x55780d868e40 2024-05-27T07:59:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.008+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f6c037070 con 0x55780d868e40 2024-05-27T07:59:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.008+0000 7f3f68c00700 1 -- 172.21.15.90:0/4151855382 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7f3f6c032470 con 0x55780d868e40 2024-05-27T07:59:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.009+0000 7f3f68c00700 1 -- 172.21.15.90:0/4151855382 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3f6c04b050 con 0x55780d868e40 2024-05-27T07:59:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.009+0000 7f3f90768b80 10 client.27457 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.009+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55780da6bf50 con 0x55780d868e40 2024-05-27T07:59:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.009+0000 7f3f68c00700 1 -- 172.21.15.90:0/4151855382 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f6c037070 con 0x55780d868e40 2024-05-27T07:59:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.010+0000 7f3f68c00700 1 -- 172.21.15.90:0/4151855382 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f3f6c03d920 con 0x55780d868e40 2024-05-27T07:59:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.011+0000 7f3f90768b80 10 client.27457 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.011+0000 7f3f90768b80 10 client.27457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T07:59:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.011+0000 7f3f90768b80 20 client.27457 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.011+0000 7f3f90768b80 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55780da6c570 0x55780da6e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.011+0000 7f3f90768b80 4 client.27457 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:59:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.011+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55780d911570 con 0x55780da6c570 2024-05-27T07:59:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.012+0000 7f3f72000700 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55780da6c570 0x55780da6e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.012+0000 7f3f72000700 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55780da6c570 0x55780da6e990 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=309137e8d9971978 server_cookie=a20a6b80ac08880c in_seq=0 out_seq=0 2024-05-27T07:59:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.013+0000 7f3f68c00700 10 client.27457 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T07:59:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.013+0000 7f3f68c00700 1 -- 172.21.15.90:0/4151855382 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55780d911570 con 0x55780da6c570 2024-05-27T07:59:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.013+0000 7f3f68c00700 10 client.27457 handle_command_reply: tid=0 2024-05-27T07:59:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.014+0000 7f3f90768b80 1 client.27457 shutdown 2024-05-27T07:59:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.015+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 msgr2=0x55780d9f50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.015+0000 7f3f90768b80 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9f50b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c034040 tx=0x7f3f6c0208e0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.016+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 shutdown_connections 2024-05-27T07:59:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.016+0000 7f3f90768b80 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55780da6c570 0x55780da6e990 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.016+0000 7f3f90768b80 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55780d9f22c0 0x55780d9f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.016+0000 7f3f90768b80 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55780d868e40 0x55780d9f50b0 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c034040 tx=0x7f3f6c0208e0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.016+0000 7f3f90768b80 1 --2- 172.21.15.90:0/4151855382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55780d9549f0 0x55780d9f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.016+0000 7f3f68c00700 0 client.27457 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T07:59:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.016+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 >> 172.21.15.90:0/4151855382 conn(0x55780d902390 msgr2=0x55780d9e7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.017+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 shutdown_connections 2024-05-27T07:59:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.017+0000 7f3f90768b80 1 -- 172.21.15.90:0/4151855382 wait complete. 2024-05-27T07:59:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.018+0000 7f3f90768b80 20 client.27457 trim_cache size 0 max 16384 2024-05-27T07:59:41.021 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:59:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.020+0000 7f3f7be00700 1 -- 172.21.15.90:0/1629392856 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3f600b2a50 msgr2=0x7f3f600b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.020+0000 7f3f7be00700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3f600b2a50 0x7f3f600b4f60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00a400 tx=0x7f3f6c00d040 comp rx=0 tx=0).stop 2024-05-27T07:59:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.020+0000 7f3f7be00700 1 -- 172.21.15.90:0/1629392856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f7c119180 msgr2=0x7f3f7c1b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.020+0000 7f3f7be00700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f7c119180 0x7f3f7c1b5700 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3f74006010 tx=0x7f3f7400b920 comp rx=0 tx=0).stop 2024-05-27T07:59:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.020+0000 7f3f7be00700 1 -- 172.21.15.90:0/1629392856 shutdown_connections 2024-05-27T07:59:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.020+0000 7f3f7be00700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f7c119180 0x7f3f7c1b5700 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.020+0000 7f3f7be00700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f7c10af00 0x7f3f7c118700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.020+0000 7f3f7be00700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3f600b2a50 0x7f3f600b4f60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.020+0000 7f3f7be00700 1 --2- 172.21.15.90:0/1629392856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f7c110fc0 0x7f3f7c118c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.020+0000 7f3f7be00700 1 -- 172.21.15.90:0/1629392856 >> 172.21.15.90:0/1629392856 conn(0x7f3f7c06f280 msgr2=0x7f3f7c10c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.021+0000 7f3f7be00700 1 -- 172.21.15.90:0/1629392856 shutdown_connections 2024-05-27T07:59:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:41.021+0000 7f3f7be00700 1 -- 172.21.15.90:0/1629392856 wait complete. 2024-05-27T07:59:41.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:40 smithi090 ceph-mon[31429]: pgmap v425: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 537 op/s 2024-05-27T07:59:41.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:40 smithi136 ceph-mon[35324]: pgmap v425: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 537 op/s 2024-05-27T07:59:41.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:40 smithi129 ceph-mon[35254]: pgmap v425: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 537 op/s 2024-05-27T07:59:42.233 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T07:59:42.866 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:43.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:42 smithi090 ceph-mon[31429]: pgmap v426: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 MiB/s wr, 535 op/s 2024-05-27T07:59:43.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:42 smithi136 ceph-mon[35324]: pgmap v426: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 MiB/s wr, 535 op/s 2024-05-27T07:59:43.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:42 smithi129 ceph-mon[35254]: pgmap v426: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 MiB/s wr, 535 op/s 2024-05-27T07:59:43.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 565.0 s, 481.8 tps, lat 208.083 ms stddev 373.205 2024-05-27T07:59:45.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:44 smithi136 ceph-mon[35324]: pgmap v427: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 643 op/s 2024-05-27T07:59:45.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:44 smithi129 ceph-mon[35254]: pgmap v427: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 643 op/s 2024-05-27T07:59:45.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:44 smithi090 ceph-mon[31429]: pgmap v427: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.8 MiB/s wr, 643 op/s 2024-05-27T07:59:47.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:46 smithi090 ceph-mon[31429]: pgmap v428: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 481 op/s 2024-05-27T07:59:47.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:46 smithi136 ceph-mon[35324]: pgmap v428: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 481 op/s 2024-05-27T07:59:47.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:46 smithi129 ceph-mon[35254]: pgmap v428: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.1 MiB/s wr, 481 op/s 2024-05-27T07:59:48.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.114+0000 7fac35600700 1 -- 172.21.15.90:0/1434008072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac300764a0 msgr2=0x7fac30114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:48.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.114+0000 7fac35600700 1 --2- 172.21.15.90:0/1434008072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac300764a0 0x7fac30114e20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fac20009990 tx=0x7fac20009ca0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.114+0000 7fac35600700 1 -- 172.21.15.90:0/1434008072 shutdown_connections 2024-05-27T07:59:48.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.114+0000 7fac35600700 1 --2- 172.21.15.90:0/1434008072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac30115360 0x7fac30117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.114+0000 7fac35600700 1 --2- 172.21.15.90:0/1434008072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac30075b50 0x7fac30075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.114+0000 7fac35600700 1 --2- 172.21.15.90:0/1434008072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac300764a0 0x7fac30114e20 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fac20009990 tx=0x7fac20009ca0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.114+0000 7fac35600700 1 -- 172.21.15.90:0/1434008072 >> 172.21.15.90:0/1434008072 conn(0x7fac30100030 msgr2=0x7fac30102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:48.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.115+0000 7fac35600700 1 -- 172.21.15.90:0/1434008072 shutdown_connections 2024-05-27T07:59:48.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.116+0000 7fac35600700 1 -- 172.21.15.90:0/1434008072 wait complete. 2024-05-27T07:59:48.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.118+0000 7fac35600700 1 Processor -- start 2024-05-27T07:59:48.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.118+0000 7fac35600700 1 -- start start 2024-05-27T07:59:48.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.118+0000 7fac35600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac30075b50 0x7fac3006e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.119+0000 7fac35600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30115360 0x7fac3006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.119+0000 7fac35600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac3006f0c0 0x7fac30073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.119+0000 7fac35600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac3011a930 con 0x7fac30075b50 2024-05-27T07:59:48.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.119+0000 7fac35600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac3011a7b0 con 0x7fac3006f0c0 2024-05-27T07:59:48.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.119+0000 7fac35600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac3011aab0 con 0x7fac30115360 2024-05-27T07:59:48.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.120+0000 7fac2e000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30115360 0x7fac3006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.120+0000 7fac2f400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac3006f0c0 0x7fac30073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.120+0000 7fac2e000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30115360 0x7fac3006eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36746/0 (socket says 172.21.15.90:36746) 2024-05-27T07:59:48.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.120+0000 7fac2e000700 1 -- 172.21.15.90:0/3640766252 learned_addr learned my addr 172.21.15.90:0/3640766252 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:48.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.121+0000 7fac2e000700 1 -- 172.21.15.90:0/3640766252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac3006f0c0 msgr2=0x7fac30073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:48.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.121+0000 7fac2e000700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac3006f0c0 0x7fac30073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.121+0000 7fac2e000700 1 -- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac30075b50 msgr2=0x7fac3006e640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T07:59:48.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.121+0000 7fac2ea00700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac30075b50 0x7fac3006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.122+0000 7fac2e000700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac30075b50 0x7fac3006e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.122+0000 7fac2e000700 1 -- 172.21.15.90:0/3640766252 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac30073580 con 0x7fac30115360 2024-05-27T07:59:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.122+0000 7fac2ea00700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac30075b50 0x7fac3006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.124+0000 7fac2e000700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30115360 0x7fac3006eb80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fac20006010 tx=0x7fac200047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:48.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.124+0000 7fac27400700 1 -- 172.21.15.90:0/3640766252 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20026070 con 0x7fac30115360 2024-05-27T07:59:48.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.125+0000 7fac27400700 1 -- 172.21.15.90:0/3640766252 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac2000fc00 con 0x7fac30115360 2024-05-27T07:59:48.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.125+0000 7fac35600700 1 -- 172.21.15.90:0/3640766252 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac30073810 con 0x7fac30115360 2024-05-27T07:59:48.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.125+0000 7fac27400700 1 -- 172.21.15.90:0/3640766252 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20021b80 con 0x7fac30115360 2024-05-27T07:59:48.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.125+0000 7fac35600700 1 -- 172.21.15.90:0/3640766252 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac30073cf0 con 0x7fac30115360 2024-05-27T07:59:48.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.132+0000 7fac27400700 1 -- 172.21.15.90:0/3640766252 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fac20029070 con 0x7fac30115360 2024-05-27T07:59:48.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.136+0000 7fac27400700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fac140b2b40 0x7fac140b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.137+0000 7fac27400700 1 -- 172.21.15.90:0/3640766252 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fac200e4420 con 0x7fac30115360 2024-05-27T07:59:48.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.137+0000 7fac2ea00700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fac140b2b40 0x7fac140b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.140+0000 7fac2ea00700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fac140b2b40 0x7fac140b5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac1c005fd0 tx=0x7fac1c005de0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:48.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.143+0000 7fac44180b80 1 Processor -- start 2024-05-27T07:59:48.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.143+0000 7fac44180b80 1 -- start start 2024-05-27T07:59:48.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.144+0000 7fac44180b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e55322ee40 0x55e5533b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.144+0000 7fac44180b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533b3d30 0x55e5533ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.144+0000 7fac44180b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533addb0 0x55e5533ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.144+0000 7fac44180b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e5532dbe40 con 0x55e55322ee40 2024-05-27T07:59:48.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.144+0000 7fac44180b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e5532cb9e0 con 0x55e5533b3d30 2024-05-27T07:59:48.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.144+0000 7fac44180b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e5532cfa20 con 0x55e5533addb0 2024-05-27T07:59:48.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.145+0000 7fac2f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533addb0 0x55e5533ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.145+0000 7fac2f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533addb0 0x55e5533ae1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36762/0 (socket says 172.21.15.90:36762) 2024-05-27T07:59:48.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.145+0000 7fac2f400700 1 -- 172.21.15.90:0/3597952653 learned_addr learned my addr 172.21.15.90:0/3597952653 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:48.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.145+0000 7fac2ea00700 1 --2- 172.21.15.90:0/3597952653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e55322ee40 0x55e5533b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.145+0000 7fac2f400700 1 -- 172.21.15.90:0/3597952653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533b3d30 msgr2=0x55e5533ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:48.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.145+0000 7fac2f400700 1 --2- 172.21.15.90:0/3597952653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533b3d30 0x55e5533ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.145+0000 7fac2f400700 1 -- 172.21.15.90:0/3597952653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e55322ee40 msgr2=0x55e5533b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:48.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.145+0000 7fac2f400700 1 --2- 172.21.15.90:0/3597952653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e55322ee40 0x55e5533b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.145+0000 7fac2f400700 1 -- 172.21.15.90:0/3597952653 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e553294420 con 0x55e5533addb0 2024-05-27T07:59:48.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.146+0000 7fac2f400700 1 --2- 172.21.15.90:0/3597952653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533addb0 0x55e5533ae1c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fac28009d20 tx=0x7fac280077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5038663a0cd90b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:48.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.146+0000 7fac1be00700 1 -- 172.21.15.90:0/3597952653 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28004bb0 con 0x55e5533addb0 2024-05-27T07:59:48.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.146+0000 7fac1be00700 1 -- 172.21.15.90:0/3597952653 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac28004d50 con 0x55e5533addb0 2024-05-27T07:59:48.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.146+0000 7fac1be00700 1 -- 172.21.15.90:0/3597952653 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28014670 con 0x55e5533addb0 2024-05-27T07:59:48.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.148+0000 7fac44180b80 1 -- 172.21.15.90:0/3597952653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533addb0 msgr2=0x55e5533ae1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:48.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.148+0000 7fac44180b80 1 --2- 172.21.15.90:0/3597952653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533addb0 0x55e5533ae1c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fac28009d20 tx=0x7fac280077e0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.148+0000 7fac44180b80 1 -- 172.21.15.90:0/3597952653 shutdown_connections 2024-05-27T07:59:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.148+0000 7fac44180b80 1 --2- 172.21.15.90:0/3597952653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533addb0 0x55e5533ae1c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.148+0000 7fac44180b80 1 --2- 172.21.15.90:0/3597952653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e55322ee40 0x55e5533b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.148+0000 7fac44180b80 1 --2- 172.21.15.90:0/3597952653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533b3d30 0x55e5533ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.148+0000 7fac44180b80 1 -- 172.21.15.90:0/3597952653 >> 172.21.15.90:0/3597952653 conn(0x55e55328efc0 msgr2=0x55e5533a88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:48.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.148+0000 7fac44180b80 1 -- 172.21.15.90:0/3597952653 shutdown_connections 2024-05-27T07:59:48.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.149+0000 7fac44180b80 1 -- 172.21.15.90:0/3597952653 wait complete. 2024-05-27T07:59:48.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.150+0000 7fac44180b80 1 -- start start 2024-05-27T07:59:48.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.150+0000 7fac44180b80 1 Processor -- start 2024-05-27T07:59:48.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.152+0000 7fac44180b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.152+0000 7fac44180b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e5533addb0 0x55e5533b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.152+0000 7fac44180b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533b3d30 0x55e5533b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.152+0000 7fac44180b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e5531d1fa0 con 0x55e5533addb0 2024-05-27T07:59:48.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.152+0000 7fac44180b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e55326c760 con 0x55e5533b3d30 2024-05-27T07:59:48.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.152+0000 7fac44180b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e5533ba800 con 0x55e55322ee40 2024-05-27T07:59:48.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.152+0000 7fac44180b80 10 client.0 _ll_register_callbacks cb 0x55e553275c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:48.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.153+0000 7fac2ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.153+0000 7fac2ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533b7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36764/0 (socket says 172.21.15.90:36764) 2024-05-27T07:59:48.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.153+0000 7fac2ea00700 1 -- 172.21.15.90:0/2793620769 learned_addr learned my addr 172.21.15.90:0/2793620769 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:48.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.153+0000 7fac2f400700 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533b3d30 0x55e5533b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.154+0000 7fac2ea00700 1 -- 172.21.15.90:0/2793620769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533b3d30 msgr2=0x55e5533b80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:48.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.154+0000 7fac2ea00700 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533b3d30 0x55e5533b80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.154+0000 7fac2ea00700 1 -- 172.21.15.90:0/2793620769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e5533addb0 msgr2=0x55e5533b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:48.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.154+0000 7fac2ea00700 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e5533addb0 0x55e5533b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.154+0000 7fac2ea00700 1 -- 172.21.15.90:0/2793620769 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55321b8d0 con 0x55e55322ee40 2024-05-27T07:59:48.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.155+0000 7fac18c00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:59:48.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.155+0000 7fac2ea00700 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533b7650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00f6d0 tx=0x7fac1c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24d8642d61d1abc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:48.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.155+0000 7fac18c00700 1 -- 172.21.15.90:0/2793620769 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c01d630 con 0x55e55322ee40 2024-05-27T07:59:48.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.155+0000 7fac18c00700 1 -- 172.21.15.90:0/2793620769 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fac1c01d7d0 con 0x55e55322ee40 2024-05-27T07:59:48.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.156+0000 7fac44180b80 1 -- 172.21.15.90:0/2793620769 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac28004d50 con 0x55e55322ee40 2024-05-27T07:59:48.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.156+0000 7fac18c00700 1 -- 172.21.15.90:0/2793620769 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fac1c01d9d0 con 0x55e55322ee40 2024-05-27T07:59:48.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.157+0000 7fac44180b80 10 client.27469 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:48.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.157+0000 7fac44180b80 1 -- 172.21.15.90:0/2793620769 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e55329c060 con 0x55e55322ee40 2024-05-27T07:59:48.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.157+0000 7fac18c00700 1 -- 172.21.15.90:0/2793620769 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac28004d50 con 0x55e55322ee40 2024-05-27T07:59:48.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.158+0000 7fac18c00700 1 -- 172.21.15.90:0/2793620769 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fac1c02fae0 con 0x55e55322ee40 2024-05-27T07:59:48.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.158+0000 7fac44180b80 10 client.27469 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:48.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.158+0000 7fac44180b80 10 client.27469 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:59:48.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.158+0000 7fac44180b80 20 client.27469 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:48.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.158+0000 7fac44180b80 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e5533c7780 0x55e5533c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.158+0000 7fac44180b80 4 client.27469 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:48.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.158+0000 7fac44180b80 1 -- 172.21.15.90:0/2793620769 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5532ddd90 con 0x55e5533c7780 2024-05-27T07:59:48.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.159+0000 7fac2e000700 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e5533c7780 0x55e5533c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.160+0000 7fac2e000700 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e5533c7780 0x55e5533c9c30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc6ffb684607f430 server_cookie=8acdad07df01763c in_seq=0 out_seq=0 2024-05-27T07:59:48.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.160+0000 7fac18c00700 10 client.27469 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:59:48.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.162+0000 7fac18c00700 1 -- 172.21.15.90:0/2793620769 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e5532ddd90 con 0x55e5533c7780 2024-05-27T07:59:48.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.162+0000 7fac18c00700 10 client.27469 handle_command_reply: tid=0 2024-05-27T07:59:48.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.162+0000 7fac44180b80 1 client.27469 shutdown 2024-05-27T07:59:48.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.163+0000 7fac44180b80 1 -- 172.21.15.90:0/2793620769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 msgr2=0x55e5533b7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:48.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.163+0000 7fac44180b80 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533b7650 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00f6d0 tx=0x7fac1c004750 comp rx=0 tx=0).stop 2024-05-27T07:59:48.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.163+0000 7fac44180b80 1 -- 172.21.15.90:0/2793620769 shutdown_connections 2024-05-27T07:59:48.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.163+0000 7fac44180b80 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e5533c7780 0x55e5533c9c30 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.163+0000 7fac44180b80 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533b3d30 0x55e5533b80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.163+0000 7fac44180b80 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533b7650 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.163+0000 7fac44180b80 1 --2- 172.21.15.90:0/2793620769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e5533addb0 0x55e5533b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.163+0000 7fac44180b80 1 -- 172.21.15.90:0/2793620769 >> 172.21.15.90:0/2793620769 conn(0x55e55328efc0 msgr2=0x55e5533a8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:48.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.163+0000 7fac18c00700 0 client.27469 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:59:48.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.163+0000 7fac44180b80 1 -- 172.21.15.90:0/2793620769 shutdown_connections 2024-05-27T07:59:48.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.164+0000 7fac44180b80 1 -- 172.21.15.90:0/2793620769 wait complete. 2024-05-27T07:59:48.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.164+0000 7fac44180b80 20 client.27469 trim_cache size 0 max 16384 2024-05-27T07:59:48.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.188+0000 7fac44180b80 1 Processor -- start 2024-05-27T07:59:48.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.188+0000 7fac44180b80 1 -- start start 2024-05-27T07:59:48.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.189+0000 7fac44180b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.189+0000 7fac44180b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 0x55e5533b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.189+0000 7fac44180b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e5533b3d30 0x55e5533b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.189+0000 7fac44180b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac301c1250 con 0x55e5533b3d30 2024-05-27T07:59:48.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.189+0000 7fac44180b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac301be2a0 con 0x55e5533addb0 2024-05-27T07:59:48.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.189+0000 7fac44180b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac301c04b0 con 0x55e55322ee40 2024-05-27T07:59:48.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.190+0000 7fac2e000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 0x55e5533b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.190+0000 7fac2ea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.190+0000 7fac2e000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 0x55e5533b65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:60740/0 (socket says 172.21.15.90:60740) 2024-05-27T07:59:48.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.191+0000 7fac2e000700 1 -- 172.21.15.90:0/4164664333 learned_addr learned my addr 172.21.15.90:0/4164664333 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:48.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.191+0000 7fac2f400700 1 --2- 172.21.15.90:0/4164664333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e5533b3d30 0x55e5533b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.192+0000 7fac2ea00700 1 -- 172.21.15.90:0/4164664333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 msgr2=0x55e5533b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:48.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.192+0000 7fac2ea00700 1 --2- 172.21.15.90:0/4164664333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 0x55e5533b65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.192+0000 7fac2ea00700 1 -- 172.21.15.90:0/4164664333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e5533b3d30 msgr2=0x55e5533b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:48.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.192+0000 7fac2ea00700 1 --2- 172.21.15.90:0/4164664333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e5533b3d30 0x55e5533b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.192+0000 7fac2ea00700 1 -- 172.21.15.90:0/4164664333 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5533ca1c0 con 0x55e55322ee40 2024-05-27T07:59:48.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.193+0000 7fac2e000700 1 --2- 172.21.15.90:0/4164664333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 0x55e5533b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:48.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.193+0000 7fac2ea00700 1 --2- 172.21.15.90:0/4164664333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533ad870 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fac1c01a770 tx=0x7fac1c026f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4af907461ab63fbc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:48.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.194+0000 7fac1b400700 1 -- 172.21.15.90:0/4164664333 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c034040 con 0x55e55322ee40 2024-05-27T07:59:48.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.194+0000 7fac1b400700 1 -- 172.21.15.90:0/4164664333 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac1c01a540 con 0x55e55322ee40 2024-05-27T07:59:48.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.194+0000 7fac1b400700 1 -- 172.21.15.90:0/4164664333 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c01ab50 con 0x55e55322ee40 2024-05-27T07:59:48.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.196+0000 7fac44180b80 1 -- 172.21.15.90:0/4164664333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 msgr2=0x55e5533ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:48.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.196+0000 7fac44180b80 1 --2- 172.21.15.90:0/4164664333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533ad870 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fac1c01a770 tx=0x7fac1c026f20 comp rx=0 tx=0).stop 2024-05-27T07:59:48.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.196+0000 7fac44180b80 1 -- 172.21.15.90:0/4164664333 shutdown_connections 2024-05-27T07:59:48.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.196+0000 7fac44180b80 1 --2- 172.21.15.90:0/4164664333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e5533b3d30 0x55e5533b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.196+0000 7fac44180b80 1 --2- 172.21.15.90:0/4164664333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e55322ee40 0x55e5533ad870 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.196+0000 7fac44180b80 1 --2- 172.21.15.90:0/4164664333 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 0x55e5533b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.196+0000 7fac44180b80 1 -- 172.21.15.90:0/4164664333 >> 172.21.15.90:0/4164664333 conn(0x55e55328efc0 msgr2=0x55e5533b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:48.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.196+0000 7fac44180b80 1 -- 172.21.15.90:0/4164664333 shutdown_connections 2024-05-27T07:59:48.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.196+0000 7fac44180b80 1 -- 172.21.15.90:0/4164664333 wait complete. 2024-05-27T07:59:48.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.197+0000 7fac44180b80 1 -- start start 2024-05-27T07:59:48.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.197+0000 7fac44180b80 1 Processor -- start 2024-05-27T07:59:48.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.198+0000 7fac44180b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e55322ee40 0x55e5533bfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac44180b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 0x55e5533bff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac44180b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533b3d30 0x55e5533ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac44180b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac301bfde0 con 0x55e55322ee40 2024-05-27T07:59:48.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac44180b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac301be970 con 0x55e5533addb0 2024-05-27T07:59:48.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac44180b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fac301bf040 con 0x55e5533b3d30 2024-05-27T07:59:48.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac44180b80 10 client.0 _ll_register_callbacks cb 0x55e553275c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:48.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac2ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e55322ee40 0x55e5533bfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac2f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533b3d30 0x55e5533ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac2f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533b3d30 0x55e5533ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36792/0 (socket says 172.21.15.90:36792) 2024-05-27T07:59:48.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac2f400700 1 -- 172.21.15.90:0/4117002267 learned_addr learned my addr 172.21.15.90:0/4117002267 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:48.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.199+0000 7fac2e000700 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 0x55e5533bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.200+0000 7fac2f400700 1 -- 172.21.15.90:0/4117002267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 msgr2=0x55e5533bff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:48.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.200+0000 7fac2f400700 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 0x55e5533bff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.200+0000 7fac2f400700 1 -- 172.21.15.90:0/4117002267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e55322ee40 msgr2=0x55e5533bfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:48.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.200+0000 7fac2f400700 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e55322ee40 0x55e5533bfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.200+0000 7fac2f400700 1 -- 172.21.15.90:0/4117002267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5533cfe90 con 0x55e5533b3d30 2024-05-27T07:59:48.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.201+0000 7fac2f400700 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533b3d30 0x55e5533ce420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fac28005570 tx=0x7fac280055a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87b451ef000f52d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:48.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.201+0000 7fac18c00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:59:48.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.202+0000 7fac18c00700 1 -- 172.21.15.90:0/4117002267 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28004e70 con 0x55e5533b3d30 2024-05-27T07:59:48.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.202+0000 7fac18c00700 1 -- 172.21.15.90:0/4117002267 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fac280059d0 con 0x55e5533b3d30 2024-05-27T07:59:48.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.202+0000 7fac44180b80 1 -- 172.21.15.90:0/4117002267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1c01a540 con 0x55e5533b3d30 2024-05-27T07:59:48.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.203+0000 7fac18c00700 1 -- 172.21.15.90:0/4117002267 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fac2801f430 con 0x55e5533b3d30 2024-05-27T07:59:48.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.203+0000 7fac44180b80 10 client.27475 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:48.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.203+0000 7fac44180b80 1 -- 172.21.15.90:0/4117002267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e553431740 con 0x55e5533b3d30 2024-05-27T07:59:48.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.204+0000 7fac18c00700 1 -- 172.21.15.90:0/4117002267 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac1c01a540 con 0x55e5533b3d30 2024-05-27T07:59:48.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.204+0000 7fac18c00700 1 -- 172.21.15.90:0/4117002267 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fac280315b0 con 0x55e5533b3d30 2024-05-27T07:59:48.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.204+0000 7fac44180b80 10 client.27475 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:48.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.204+0000 7fac44180b80 10 client.27475 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T07:59:48.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.204+0000 7fac44180b80 20 client.27475 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:48.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.204+0000 7fac44180b80 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e553431c70 0x55e553434090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:48.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.204+0000 7fac44180b80 4 client.27475 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:59:48.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.204+0000 7fac44180b80 1 -- 172.21.15.90:0/4117002267 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5532d7570 con 0x55e553431c70 2024-05-27T07:59:48.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.205+0000 7fac2ea00700 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e553431c70 0x55e553434090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:48.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.211+0000 7fac18c00700 10 client.27475 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T07:59:48.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.211+0000 7fac2ea00700 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e553431c70 0x55e553434090 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3edb7da7fcc00e56 server_cookie=ba4bc5fb2039b97c in_seq=0 out_seq=0 2024-05-27T07:59:48.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.212+0000 7fac18c00700 1 -- 172.21.15.90:0/4117002267 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5532d7570 con 0x55e553431c70 2024-05-27T07:59:48.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.212+0000 7fac18c00700 10 client.27475 handle_command_reply: tid=0 2024-05-27T07:59:48.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.212+0000 7fac44180b80 1 client.27475 shutdown 2024-05-27T07:59:48.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 1 -- 172.21.15.90:0/4117002267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533b3d30 msgr2=0x55e5533ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:48.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533b3d30 0x55e5533ce420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fac28005570 tx=0x7fac280055a0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 1 -- 172.21.15.90:0/4117002267 shutdown_connections 2024-05-27T07:59:48.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e5533b3d30 0x55e5533ce420 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e55322ee40 0x55e5533bfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x55e553431c70 0x55e553434090 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 1 --2- 172.21.15.90:0/4117002267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e5533addb0 0x55e5533bff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac18c00700 0 client.27475 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T07:59:48.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 1 -- 172.21.15.90:0/4117002267 >> 172.21.15.90:0/4117002267 conn(0x55e55328efc0 msgr2=0x55e5533b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:48.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 1 -- 172.21.15.90:0/4117002267 shutdown_connections 2024-05-27T07:59:48.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 1 -- 172.21.15.90:0/4117002267 wait complete. 2024-05-27T07:59:48.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.213+0000 7fac44180b80 20 client.27475 trim_cache size 0 max 16384 2024-05-27T07:59:48.218 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:59:48.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.215+0000 7fac35600700 1 -- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fac140b2b40 msgr2=0x7fac140b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:48.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.215+0000 7fac35600700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fac140b2b40 0x7fac140b5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac1c005fd0 tx=0x7fac1c005de0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.215+0000 7fac35600700 1 -- 172.21.15.90:0/3640766252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30115360 msgr2=0x7fac3006eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.215+0000 7fac35600700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30115360 0x7fac3006eb80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fac20006010 tx=0x7fac200047b0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.216+0000 7fac35600700 1 -- 172.21.15.90:0/3640766252 shutdown_connections 2024-05-27T07:59:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.216+0000 7fac35600700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fac140b2b40 0x7fac140b5050 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac1c005fd0 tx=0x7fac1c005de0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.216+0000 7fac35600700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac3006f0c0 0x7fac30073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.216+0000 7fac35600700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac30075b50 0x7fac3006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.216+0000 7fac35600700 1 --2- 172.21.15.90:0/3640766252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30115360 0x7fac3006eb80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.216+0000 7fac35600700 1 -- 172.21.15.90:0/3640766252 >> 172.21.15.90:0/3640766252 conn(0x7fac30100030 msgr2=0x7fac30100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.216+0000 7fac35600700 1 -- 172.21.15.90:0/3640766252 shutdown_connections 2024-05-27T07:59:48.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:48.216+0000 7fac35600700 1 -- 172.21.15.90:0/3640766252 wait complete. 2024-05-27T07:59:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 570.0 s, 459.0 tps, lat 214.769 ms stddev 177.356 2024-05-27T07:59:49.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:48 smithi090 ceph-mon[31429]: pgmap v429: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 591 op/s 2024-05-27T07:59:49.246 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T07:59:49.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:48 smithi136 ceph-mon[35324]: pgmap v429: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 591 op/s 2024-05-27T07:59:49.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:48 smithi129 ceph-mon[35254]: pgmap v429: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 MiB/s wr, 591 op/s 2024-05-27T07:59:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.775+0000 7fd88d800640 1 -- 172.21.15.90:0/894028763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd88810fa00 msgr2=0x7fd88810fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.775+0000 7fd88d800640 1 --2- 172.21.15.90:0/894028763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd88810fa00 0x7fd88810fe10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd878009900 tx=0x7fd87802a420 comp rx=0 tx=0).stop 2024-05-27T07:59:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.775+0000 7fd88d800640 1 -- 172.21.15.90:0/894028763 shutdown_connections 2024-05-27T07:59:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.775+0000 7fd88d800640 1 --2- 172.21.15.90:0/894028763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd88810a200 0x7fd88810a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.775+0000 7fd88d800640 1 --2- 172.21.15.90:0/894028763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8881103e0 0x7fd888109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.775+0000 7fd88d800640 1 --2- 172.21.15.90:0/894028763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd88810fa00 0x7fd88810fe10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.775+0000 7fd88d800640 1 -- 172.21.15.90:0/894028763 >> 172.21.15.90:0/894028763 conn(0x7fd88806f800 msgr2=0x7fd88806fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.776+0000 7fd88d800640 1 -- 172.21.15.90:0/894028763 shutdown_connections 2024-05-27T07:59:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.776+0000 7fd88d800640 1 -- 172.21.15.90:0/894028763 wait complete. 2024-05-27T07:59:50.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.777+0000 7fd88d800640 1 Processor -- start 2024-05-27T07:59:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.777+0000 7fd88d800640 1 -- start start 2024-05-27T07:59:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.778+0000 7fd88d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd88810a200 0x7fd8881a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.778+0000 7fd88d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd88810fa00 0x7fd8881a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.778+0000 7fd88d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8881103e0 0x7fd8881adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.778+0000 7fd88d800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd888122a20 con 0x7fd88810a200 2024-05-27T07:59:50.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.778+0000 7fd88d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881228a0 con 0x7fd88810fa00 2024-05-27T07:59:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.778+0000 7fd88d800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd888122ba0 con 0x7fd8881103e0 2024-05-27T07:59:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.779+0000 7fd88ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8881103e0 0x7fd8881adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.779+0000 7fd88ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8881103e0 0x7fd8881adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:43220/0 (socket says 172.21.15.90:43220) 2024-05-27T07:59:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.779+0000 7fd88ce00640 1 -- 172.21.15.90:0/4261917670 learned_addr learned my addr 172.21.15.90:0/4261917670 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:50.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.779+0000 7fd887e00640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd88810a200 0x7fd8881a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.780+0000 7fd887400640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd88810fa00 0x7fd8881a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.780+0000 7fd88ce00640 1 -- 172.21.15.90:0/4261917670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd88810fa00 msgr2=0x7fd8881a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:50.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.780+0000 7fd88ce00640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd88810fa00 0x7fd8881a9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.780+0000 7fd88ce00640 1 -- 172.21.15.90:0/4261917670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd88810a200 msgr2=0x7fd8881a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.780+0000 7fd88ce00640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd88810a200 0x7fd8881a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.780+0000 7fd88ce00640 1 -- 172.21.15.90:0/4261917670 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8881ae300 con 0x7fd8881103e0 2024-05-27T07:59:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.780+0000 7fd887400640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd88810fa00 0x7fd8881a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.781+0000 7fd88ce00640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8881103e0 0x7fd8881adcc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd88000b970 tx=0x7fd88000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:50.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.781+0000 7fd884c00640 1 -- 172.21.15.90:0/4261917670 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880019070 con 0x7fd8881103e0 2024-05-27T07:59:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.781+0000 7fd884c00640 1 -- 172.21.15.90:0/4261917670 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd880014440 con 0x7fd8881103e0 2024-05-27T07:59:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.781+0000 7fd88d800640 1 -- 172.21.15.90:0/4261917670 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8881ae5f0 con 0x7fd8881103e0 2024-05-27T07:59:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.781+0000 7fd884c00640 1 -- 172.21.15.90:0/4261917670 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800075d0 con 0x7fd8881103e0 2024-05-27T07:59:50.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.782+0000 7fd88d800640 1 -- 172.21.15.90:0/4261917670 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8881aeaf0 con 0x7fd8881103e0 2024-05-27T07:59:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.787+0000 7fd895dcf740 1 Processor -- start 2024-05-27T07:59:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.788+0000 7fd884c00640 1 -- 172.21.15.90:0/4261917670 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd88001c070 con 0x7fd8881103e0 2024-05-27T07:59:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.788+0000 7fd895dcf740 1 -- start start 2024-05-27T07:59:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.789+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d731b0 0x55dc66115a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.789+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d4a2f0 0x55dc66115fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.789+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d524d0 0x55dc661164e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.789+0000 7fd895dcf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dc66086b90 con 0x55dc65d731b0 2024-05-27T07:59:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.789+0000 7fd895dcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc66076730 con 0x55dc65d524d0 2024-05-27T07:59:50.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.789+0000 7fd895dcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc6607a770 con 0x55dc65d4a2f0 2024-05-27T07:59:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.790+0000 7fd887400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d731b0 0x55dc66115a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.790+0000 7fd887400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d731b0 0x55dc66115a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59008/0 (socket says 172.21.15.90:59008) 2024-05-27T07:59:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.790+0000 7fd887400640 1 -- 172.21.15.90:0/4255279463 learned_addr learned my addr 172.21.15.90:0/4255279463 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.790+0000 7fd887e00640 1 --2- 172.21.15.90:0/4255279463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d4a2f0 0x55dc66115fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.791+0000 7fd884c00640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd86c0b2950 0x7fd86c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.791+0000 7fd884c00640 1 -- 172.21.15.90:0/4261917670 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fd8800e1e70 con 0x7fd8881103e0 2024-05-27T07:59:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.791+0000 7fd887400640 1 -- 172.21.15.90:0/4255279463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d4a2f0 msgr2=0x55dc66115fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.792+0000 7fd887400640 1 --2- 172.21.15.90:0/4255279463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d4a2f0 0x55dc66115fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.792+0000 7fd88ce00640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd86c0b2950 0x7fd86c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.792+0000 7fd887400640 1 -- 172.21.15.90:0/4255279463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d524d0 msgr2=0x55dc661164e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.792+0000 7fd887400640 1 --2- 172.21.15.90:0/4255279463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d524d0 0x55dc661164e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.792+0000 7fd887400640 1 -- 172.21.15.90:0/4255279463 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc6600f8b0 con 0x55dc65d731b0 2024-05-27T07:59:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.793+0000 7fd88ce00640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd86c0b2950 0x7fd86c0b4e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd880016000 tx=0x7fd880004b50 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.794+0000 7fd887400640 1 --2- 172.21.15.90:0/4255279463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d731b0 0x55dc66115a60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd87400aaa0 tx=0x7fd87400af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05ed74f039b7288 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.794+0000 7fd87d600640 1 -- 172.21.15.90:0/4255279463 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874015070 con 0x55dc65d731b0 2024-05-27T07:59:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.795+0000 7fd87d600640 1 -- 172.21.15.90:0/4255279463 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd87400cdb0 con 0x55dc65d731b0 2024-05-27T07:59:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.795+0000 7fd87d600640 1 -- 172.21.15.90:0/4255279463 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8740185c0 con 0x55dc65d731b0 2024-05-27T07:59:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.796+0000 7fd895dcf740 1 -- 172.21.15.90:0/4255279463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d731b0 msgr2=0x55dc66115a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.796+0000 7fd895dcf740 1 --2- 172.21.15.90:0/4255279463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d731b0 0x55dc66115a60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd87400aaa0 tx=0x7fd87400af60 comp rx=0 tx=0).stop 2024-05-27T07:59:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.796+0000 7fd895dcf740 1 -- 172.21.15.90:0/4255279463 shutdown_connections 2024-05-27T07:59:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.796+0000 7fd895dcf740 1 --2- 172.21.15.90:0/4255279463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d524d0 0x55dc661164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.796+0000 7fd895dcf740 1 --2- 172.21.15.90:0/4255279463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d4a2f0 0x55dc66115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.796+0000 7fd895dcf740 1 --2- 172.21.15.90:0/4255279463 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d731b0 0x55dc66115a60 secure :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd87400aaa0 tx=0x7fd87400af60 comp rx=0 tx=0).stop 2024-05-27T07:59:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.796+0000 7fd895dcf740 1 -- 172.21.15.90:0/4255279463 >> 172.21.15.90:0/4255279463 conn(0x55dc65d67300 msgr2=0x55dc66112210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.799+0000 7fd895dcf740 1 -- 172.21.15.90:0/4255279463 shutdown_connections 2024-05-27T07:59:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.799+0000 7fd895dcf740 1 -- 172.21.15.90:0/4255279463 wait complete. 2024-05-27T07:59:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.801+0000 7fd895dcf740 1 -- start start 2024-05-27T07:59:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.801+0000 7fd895dcf740 1 Processor -- start 2024-05-27T07:59:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.803+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 0x55dc6611ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.804+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d524d0 0x55dc661204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.804+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d31d80 0x55dc6612bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.804+0000 7fd895dcf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dc65d5c790 con 0x55dc65d31d80 2024-05-27T07:59:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.804+0000 7fd895dcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc6601ebc0 con 0x55dc65d4a2f0 2024-05-27T07:59:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.804+0000 7fd895dcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc66122b90 con 0x55dc65d524d0 2024-05-27T07:59:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.804+0000 7fd895dcf740 10 client.0 _ll_register_callbacks cb 0x55dc65ffc400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.805+0000 7fd887e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d524d0 0x55dc661204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.806+0000 7fd887e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d524d0 0x55dc661204e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:43248/0 (socket says 172.21.15.90:43248) 2024-05-27T07:59:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.806+0000 7fd887e00640 1 -- 172.21.15.90:0/2392231830 learned_addr learned my addr 172.21.15.90:0/2392231830 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.806+0000 7fd887400640 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d31d80 0x55dc6612bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.807+0000 7fd887400640 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 0x55dc6611ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.807+0000 7fd887e00640 1 -- 172.21.15.90:0/2392231830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 msgr2=0x55dc6611ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.807+0000 7fd887e00640 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 0x55dc6611ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.807+0000 7fd887e00640 1 -- 172.21.15.90:0/2392231830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d31d80 msgr2=0x55dc6612bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.807+0000 7fd887e00640 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d31d80 0x55dc6612bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.807+0000 7fd887e00640 1 -- 172.21.15.90:0/2392231830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc65d8b210 con 0x55dc65d524d0 2024-05-27T07:59:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.808+0000 7fd887e00640 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d524d0 0x55dc661204e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd878009900 tx=0x7fd878004f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c23028e1c6da8d5c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.808+0000 7fd872000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:59:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.808+0000 7fd872000640 1 -- 172.21.15.90:0/2392231830 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878035070 con 0x55dc65d524d0 2024-05-27T07:59:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.808+0000 7fd872000640 1 -- 172.21.15.90:0/2392231830 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fd878005000 con 0x55dc65d524d0 2024-05-27T07:59:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.808+0000 7fd895dcf740 1 -- 172.21.15.90:0/2392231830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd87400cdb0 con 0x55dc65d524d0 2024-05-27T07:59:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.809+0000 7fd872000640 1 -- 172.21.15.90:0/2392231830 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd87804d050 con 0x55dc65d524d0 2024-05-27T07:59:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.809+0000 7fd895dcf740 10 client.27484 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.809+0000 7fd895dcf740 1 -- 172.21.15.90:0/2392231830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55dc65e57620 con 0x55dc65d524d0 2024-05-27T07:59:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.810+0000 7fd872000640 1 -- 172.21.15.90:0/2392231830 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd87400cdb0 con 0x55dc65d524d0 2024-05-27T07:59:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.811+0000 7fd872000640 1 -- 172.21.15.90:0/2392231830 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fd87803b430 con 0x55dc65d524d0 2024-05-27T07:59:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.811+0000 7fd895dcf740 10 client.27484 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.811+0000 7fd895dcf740 10 client.27484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.811+0000 7fd895dcf740 20 client.27484 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.811+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc65d071c0 0x55dc661322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.811+0000 7fd895dcf740 4 client.27484 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.811+0000 7fd895dcf740 1 -- 172.21.15.90:0/2392231830 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc66088ae0 con 0x55dc65d071c0 2024-05-27T07:59:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.812+0000 7fd887400640 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc65d071c0 0x55dc661322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.813+0000 7fd872000640 10 client.27484 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.813+0000 7fd887400640 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc65d071c0 0x55dc661322c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12cd699c8cab08 server_cookie=47bc4a0c1652dba6 in_seq=0 out_seq=0 2024-05-27T07:59:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.816+0000 7fd872000640 1 -- 172.21.15.90:0/2392231830 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dc66088ae0 con 0x55dc65d071c0 2024-05-27T07:59:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.816+0000 7fd872000640 10 client.27484 handle_command_reply: tid=0 2024-05-27T07:59:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.817+0000 7fd895dcf740 1 client.27484 shutdown 2024-05-27T07:59:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.817+0000 7fd895dcf740 1 -- 172.21.15.90:0/2392231830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d524d0 msgr2=0x55dc661204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.817+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d524d0 0x55dc661204e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd878009900 tx=0x7fd878004f00 comp rx=0 tx=0).stop 2024-05-27T07:59:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.818+0000 7fd895dcf740 1 -- 172.21.15.90:0/2392231830 shutdown_connections 2024-05-27T07:59:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.818+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc65d071c0 0x55dc661322c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.818+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d31d80 0x55dc6612bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.818+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d524d0 0x55dc661204e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.818+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2392231830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 0x55dc6611ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.818+0000 7fd895dcf740 1 -- 172.21.15.90:0/2392231830 >> 172.21.15.90:0/2392231830 conn(0x55dc65d67300 msgr2=0x55dc661159a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.818+0000 7fd872000640 0 client.27484 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.819+0000 7fd895dcf740 1 -- 172.21.15.90:0/2392231830 shutdown_connections 2024-05-27T07:59:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.819+0000 7fd895dcf740 1 -- 172.21.15.90:0/2392231830 wait complete. 2024-05-27T07:59:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.819+0000 7fd895dcf740 20 client.27484 trim_cache size 0 max 16384 2024-05-27T07:59:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.846+0000 7fd895dcf740 1 Processor -- start 2024-05-27T07:59:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.846+0000 7fd895dcf740 1 -- start start 2024-05-27T07:59:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.847+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 0x55dc6611b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.847+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 0x55dc66121bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.847+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d524d0 0x55dc66122100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.847+0000 7fd895dcf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881ca090 con 0x55dc65d524d0 2024-05-27T07:59:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.847+0000 7fd895dcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881c7100 con 0x55dc65d4a2f0 2024-05-27T07:59:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.847+0000 7fd895dcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881c92f0 con 0x55dc65d31d80 2024-05-27T07:59:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.848+0000 7fd887400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d524d0 0x55dc66122100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.848+0000 7fd887e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 0x55dc66121bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.849+0000 7fd887e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 0x55dc66121bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:57548/0 (socket says 172.21.15.90:57548) 2024-05-27T07:59:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.849+0000 7fd887e00640 1 -- 172.21.15.90:0/3495075552 learned_addr learned my addr 172.21.15.90:0/3495075552 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.851+0000 7fd887e00640 1 -- 172.21.15.90:0/3495075552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 msgr2=0x55dc6611b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.852+0000 7fd887400640 1 --2- 172.21.15.90:0/3495075552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 0x55dc6611b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.853+0000 7fd887e00640 1 --2- 172.21.15.90:0/3495075552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 0x55dc6611b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.853+0000 7fd887e00640 1 -- 172.21.15.90:0/3495075552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d524d0 msgr2=0x55dc66122100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.853+0000 7fd887e00640 1 --2- 172.21.15.90:0/3495075552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d524d0 0x55dc66122100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.853+0000 7fd887e00640 1 -- 172.21.15.90:0/3495075552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc65d073f0 con 0x55dc65d4a2f0 2024-05-27T07:59:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.853+0000 7fd887400640 1 --2- 172.21.15.90:0/3495075552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 0x55dc6611b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T07:59:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.855+0000 7fd887e00640 1 --2- 172.21.15.90:0/3495075552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 0x55dc66121bc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd87803b9c0 tx=0x7fd878000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=263d03ffc6af90e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:50.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.857+0000 7fd87cc00640 1 -- 172.21.15.90:0/3495075552 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878035070 con 0x55dc65d4a2f0 2024-05-27T07:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.857+0000 7fd87cc00640 1 -- 172.21.15.90:0/3495075552 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd878005490 con 0x55dc65d4a2f0 2024-05-27T07:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.857+0000 7fd87cc00640 1 -- 172.21.15.90:0/3495075552 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8780035f0 con 0x55dc65d4a2f0 2024-05-27T07:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.858+0000 7fd895dcf740 1 -- 172.21.15.90:0/3495075552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 msgr2=0x55dc66121bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:50.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.858+0000 7fd895dcf740 1 --2- 172.21.15.90:0/3495075552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 0x55dc66121bc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd87803b9c0 tx=0x7fd878000bc0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.858+0000 7fd895dcf740 1 -- 172.21.15.90:0/3495075552 shutdown_connections 2024-05-27T07:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.858+0000 7fd895dcf740 1 --2- 172.21.15.90:0/3495075552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d4a2f0 0x55dc66121bc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.858+0000 7fd895dcf740 1 --2- 172.21.15.90:0/3495075552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d524d0 0x55dc66122100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.858+0000 7fd895dcf740 1 --2- 172.21.15.90:0/3495075552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 0x55dc6611b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.858+0000 7fd895dcf740 1 -- 172.21.15.90:0/3495075552 >> 172.21.15.90:0/3495075552 conn(0x55dc65d67300 msgr2=0x55dc661214b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.859+0000 7fd895dcf740 1 -- 172.21.15.90:0/3495075552 shutdown_connections 2024-05-27T07:59:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.859+0000 7fd895dcf740 1 -- 172.21.15.90:0/3495075552 wait complete. 2024-05-27T07:59:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.860+0000 7fd895dcf740 1 -- start start 2024-05-27T07:59:50.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.860+0000 7fd895dcf740 1 Processor -- start 2024-05-27T07:59:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.862+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 0x55dc66128140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.862+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d4a2f0 0x55dc66128680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.862+0000 7fd895dcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d524d0 0x55dc66125610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.862+0000 7fd895dcf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881c8c20 con 0x55dc65d4a2f0 2024-05-27T07:59:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.862+0000 7fd895dcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881c77b0 con 0x55dc65d524d0 2024-05-27T07:59:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.862+0000 7fd895dcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881c7e80 con 0x55dc65d31d80 2024-05-27T07:59:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.862+0000 7fd895dcf740 10 client.0 _ll_register_callbacks cb 0x7fd878005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.864+0000 7fd887400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d524d0 0x55dc66125610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.865+0000 7fd887e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d4a2f0 0x55dc66128680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.866+0000 7fd887400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 0x55dc66128140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.866+0000 7fd887400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 0x55dc66128140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:43278/0 (socket says 172.21.15.90:43278) 2024-05-27T07:59:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.866+0000 7fd887400640 1 -- 172.21.15.90:0/2691409190 learned_addr learned my addr 172.21.15.90:0/2691409190 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.867+0000 7fd887e00640 1 -- 172.21.15.90:0/2691409190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 msgr2=0x55dc66128140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.867+0000 7fd887e00640 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 0x55dc66128140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.867+0000 7fd887e00640 1 -- 172.21.15.90:0/2691409190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d524d0 msgr2=0x55dc66125610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.867+0000 7fd887e00640 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d524d0 0x55dc66125610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.867+0000 7fd887e00640 1 -- 172.21.15.90:0/2691409190 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc66125d80 con 0x55dc65d4a2f0 2024-05-27T07:59:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.867+0000 7fd887e00640 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d4a2f0 0x55dc66128680 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd878039350 tx=0x7fd878005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f9ef544bb7d27b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.868+0000 7fd872000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.868+0000 7fd872000640 1 -- 172.21.15.90:0/2691409190 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87805f060 con 0x55dc65d4a2f0 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.868+0000 7fd872000640 1 -- 172.21.15.90:0/2691409190 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fd87803bab0 con 0x55dc65d4a2f0 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.868+0000 7fd895dcf740 1 -- 172.21.15.90:0/2691409190 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd878005490 con 0x55dc65d4a2f0 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.868+0000 7fd872000640 1 -- 172.21.15.90:0/2691409190 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd878035050 con 0x55dc65d4a2f0 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.869+0000 7fd895dcf740 10 client.17772 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.869+0000 7fd895dcf740 1 -- 172.21.15.90:0/2691409190 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55dc6619a420 con 0x55dc65d4a2f0 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.869+0000 7fd872000640 1 -- 172.21.15.90:0/2691409190 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd878005490 con 0x55dc65d4a2f0 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.869+0000 7fd872000640 1 -- 172.21.15.90:0/2691409190 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fd878003ba0 con 0x55dc65d4a2f0 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.870+0000 7fd895dcf740 10 client.17772 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.870+0000 7fd895dcf740 10 client.17772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.870+0000 7fd895dcf740 20 client.17772 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.870+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc6619a9d0 0x55dc6619cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.870+0000 7fd895dcf740 4 client.17772 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T07:59:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.870+0000 7fd895dcf740 1 -- 172.21.15.90:0/2691409190 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc660822c0 con 0x55dc6619a9d0 2024-05-27T07:59:50.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.871+0000 7fd887400640 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc6619a9d0 0x55dc6619cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.872+0000 7fd887400640 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc6619a9d0 0x55dc6619cee0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ade3b2fdc87ff45 server_cookie=36925577a809d7f7 in_seq=0 out_seq=0 2024-05-27T07:59:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.872+0000 7fd872000640 10 client.17772 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.995+0000 7fd872000640 1 -- 172.21.15.90:0/2691409190 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dc660822c0 con 0x55dc6619a9d0 2024-05-27T07:59:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.995+0000 7fd872000640 10 client.17772 handle_command_reply: tid=0 2024-05-27T07:59:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.995+0000 7fd895dcf740 1 client.17772 shutdown 2024-05-27T07:59:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.999+0000 7fd895dcf740 1 -- 172.21.15.90:0/2691409190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d4a2f0 msgr2=0x55dc66128680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:50.999+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d4a2f0 0x55dc66128680 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd878039350 tx=0x7fd878005d50 comp rx=0 tx=0).stop 2024-05-27T07:59:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.001+0000 7fd895dcf740 1 -- 172.21.15.90:0/2691409190 shutdown_connections 2024-05-27T07:59:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.001+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55dc6619a9d0 0x55dc6619cee0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.001+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc65d524d0 0x55dc66125610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.001+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc65d4a2f0 0x55dc66128680 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.001+0000 7fd895dcf740 1 --2- 172.21.15.90:0/2691409190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc65d31d80 0x55dc66128140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.001+0000 7fd895dcf740 1 -- 172.21.15.90:0/2691409190 >> 172.21.15.90:0/2691409190 conn(0x55dc65d67300 msgr2=0x55dc66136ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.001+0000 7fd872000640 0 client.17772 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.003+0000 7fd895dcf740 1 -- 172.21.15.90:0/2691409190 shutdown_connections 2024-05-27T07:59:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.004+0000 7fd895dcf740 1 -- 172.21.15.90:0/2691409190 wait complete. 2024-05-27T07:59:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.004+0000 7fd895dcf740 20 client.17772 trim_cache size 0 max 16384 2024-05-27T07:59:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T07:59:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T07:59:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T07:59:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.007+0000 7fd88d800640 1 -- 172.21.15.90:0/4261917670 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd86c0b2950 msgr2=0x7fd86c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.007+0000 7fd88d800640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd86c0b2950 0x7fd86c0b4e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd880016000 tx=0x7fd880004b50 comp rx=0 tx=0).stop 2024-05-27T07:59:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.007+0000 7fd88d800640 1 -- 172.21.15.90:0/4261917670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8881103e0 msgr2=0x7fd8881adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.007+0000 7fd88d800640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8881103e0 0x7fd8881adcc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd88000b970 tx=0x7fd88000be30 comp rx=0 tx=0).stop 2024-05-27T07:59:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.007+0000 7fd88d800640 1 -- 172.21.15.90:0/4261917670 shutdown_connections 2024-05-27T07:59:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.007+0000 7fd88d800640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd86c0b2950 0x7fd86c0b4e60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.007+0000 7fd88d800640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd88810fa00 0x7fd8881a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.007+0000 7fd88d800640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8881103e0 0x7fd8881adcc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.007+0000 7fd88d800640 1 --2- 172.21.15.90:0/4261917670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd88810a200 0x7fd8881a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.007+0000 7fd88d800640 1 -- 172.21.15.90:0/4261917670 >> 172.21.15.90:0/4261917670 conn(0x7fd88806f800 msgr2=0x7fd88810d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.008+0000 7fd88d800640 1 -- 172.21.15.90:0/4261917670 shutdown_connections 2024-05-27T07:59:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T07:59:51.008+0000 7fd88d800640 1 -- 172.21.15.90:0/4261917670 wait complete. 2024-05-27T07:59:51.248 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:50 smithi090 ceph-mon[31429]: pgmap v430: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-05-27T07:59:51.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:50 smithi136 ceph-mon[35324]: pgmap v430: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-05-27T07:59:51.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:50 smithi129 ceph-mon[35254]: pgmap v430: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-05-27T07:59:53.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:52 smithi136 ceph-mon[35324]: pgmap v431: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.3 MiB/s wr, 478 op/s 2024-05-27T07:59:53.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:52 smithi129 ceph-mon[35254]: pgmap v431: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.3 MiB/s wr, 478 op/s 2024-05-27T07:59:53.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:52 smithi090 ceph-mon[31429]: pgmap v431: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.3 MiB/s wr, 478 op/s 2024-05-27T07:59:53.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 575.0 s, 467.3 tps, lat 184.206 ms stddev 136.083 2024-05-27T07:59:55.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.049+0000 7fb46be00700 1 -- 172.21.15.90:0/1112996626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4641034b0 msgr2=0x7fb4641038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.049+0000 7fb46be00700 1 --2- 172.21.15.90:0/1112996626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4641034b0 0x7fb4641038c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb45800b0a0 tx=0x7fb45800b3b0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.049+0000 7fb46be00700 1 -- 172.21.15.90:0/1112996626 shutdown_connections 2024-05-27T07:59:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.049+0000 7fb46be00700 1 --2- 172.21.15.90:0/1112996626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb464108730 0x7fb4640759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.049+0000 7fb46be00700 1 --2- 172.21.15.90:0/1112996626 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4641034b0 0x7fb4641038c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.049+0000 7fb46be00700 1 --2- 172.21.15.90:0/1112996626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb464107de0 0x7fb4641081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.049+0000 7fb46be00700 1 -- 172.21.15.90:0/1112996626 >> 172.21.15.90:0/1112996626 conn(0x7fb4640fede0 msgr2=0x7fb464101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.050+0000 7fb46be00700 1 -- 172.21.15.90:0/1112996626 shutdown_connections 2024-05-27T07:59:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.050+0000 7fb46be00700 1 -- 172.21.15.90:0/1112996626 wait complete. 2024-05-27T07:59:55.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:54 smithi090 ceph-mon[31429]: pgmap v432: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 664 op/s 2024-05-27T07:59:55.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.051+0000 7fb46be00700 1 Processor -- start 2024-05-27T07:59:55.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.052+0000 7fb46be00700 1 -- start start 2024-05-27T07:59:55.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.052+0000 7fb46be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4641034b0 0x7fb46419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.052+0000 7fb46be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb464107de0 0x7fb46419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.053+0000 7fb46be00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb464108730 0x7fb4641a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.053+0000 7fb46be00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641166b0 con 0x7fb464108730 2024-05-27T07:59:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.053+0000 7fb46be00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb464110000 con 0x7fb4641034b0 2024-05-27T07:59:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.053+0000 7fb46be00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb464116830 con 0x7fb464107de0 2024-05-27T07:59:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.053+0000 7fb469400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4641034b0 0x7fb46419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.053+0000 7fb469400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4641034b0 0x7fb46419cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:57570/0 (socket says 172.21.15.90:57570) 2024-05-27T07:59:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.053+0000 7fb469400700 1 -- 172.21.15.90:0/1887759143 learned_addr learned my addr 172.21.15.90:0/1887759143 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.053+0000 7fb469e00700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb464108730 0x7fb4641a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.053+0000 7fb468a00700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb464107de0 0x7fb46419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.054+0000 7fb469400700 1 -- 172.21.15.90:0/1887759143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb464107de0 msgr2=0x7fb46419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.054+0000 7fb469400700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb464107de0 0x7fb46419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.054+0000 7fb469400700 1 -- 172.21.15.90:0/1887759143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb464108730 msgr2=0x7fb4641a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.054+0000 7fb469400700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb464108730 0x7fb4641a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.054+0000 7fb469400700 1 -- 172.21.15.90:0/1887759143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4641a1ea0 con 0x7fb4641034b0 2024-05-27T07:59:55.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.054+0000 7fb469400700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4641034b0 0x7fb46419cfd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb458000f80 tx=0x7fb458004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.055+0000 7fb462000700 1 -- 172.21.15.90:0/1887759143 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458021070 con 0x7fb4641034b0 2024-05-27T07:59:55.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.055+0000 7fb46be00700 1 -- 172.21.15.90:0/1887759143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4641a2130 con 0x7fb4641034b0 2024-05-27T07:59:55.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.055+0000 7fb46be00700 1 -- 172.21.15.90:0/1887759143 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4641a9ab0 con 0x7fb4641034b0 2024-05-27T07:59:55.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.055+0000 7fb462000700 1 -- 172.21.15.90:0/1887759143 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb458007b80 con 0x7fb4641034b0 2024-05-27T07:59:55.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.055+0000 7fb462000700 1 -- 172.21.15.90:0/1887759143 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4580046f0 con 0x7fb4641034b0 2024-05-27T07:59:55.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.062+0000 7fb462000700 1 -- 172.21.15.90:0/1887759143 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb45801d450 con 0x7fb4641034b0 2024-05-27T07:59:55.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.064+0000 7fb462000700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb44c0b2af0 0x7fb44c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.064+0000 7fb462000700 1 -- 172.21.15.90:0/1887759143 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fb45800f020 con 0x7fb4641034b0 2024-05-27T07:59:55.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.064+0000 7fb468a00700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb44c0b2af0 0x7fb44c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.079+0000 7fb468a00700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb44c0b2af0 0x7fb44c0b5000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb464102640 tx=0x7fb45000b410 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.083+0000 7fb47a9c6b80 1 Processor -- start 2024-05-27T07:59:55.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.083+0000 7fb47a9c6b80 1 -- start start 2024-05-27T07:59:55.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.084+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3ec39f0 0x55bea3f5eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.084+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3dd7e40 0x55bea3f5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.084+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 0x55bea3f6b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.084+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bea3e84e40 con 0x55bea3dd7e40 2024-05-27T07:59:55.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.084+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bea3e749e0 con 0x55bea3f5f690 2024-05-27T07:59:55.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.084+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bea3e78a20 con 0x55bea3ec39f0 2024-05-27T07:59:55.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.085+0000 7fb469400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3ec39f0 0x55bea3f5eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.085+0000 7fb468a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3dd7e40 0x55bea3f5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.085+0000 7fb468a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3dd7e40 0x55bea3f5f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59068/0 (socket says 172.21.15.90:59068) 2024-05-27T07:59:55.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.085+0000 7fb468a00700 1 -- 172.21.15.90:0/2600293493 learned_addr learned my addr 172.21.15.90:0/2600293493 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:55.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.085+0000 7fb469e00700 1 --2- 172.21.15.90:0/2600293493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 0x55bea3f6b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.086+0000 7fb468a00700 1 -- 172.21.15.90:0/2600293493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3ec39f0 msgr2=0x55bea3f5eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.086+0000 7fb468a00700 1 --2- 172.21.15.90:0/2600293493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3ec39f0 0x55bea3f5eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.086+0000 7fb468a00700 1 -- 172.21.15.90:0/2600293493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 msgr2=0x55bea3f6b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.086+0000 7fb468a00700 1 --2- 172.21.15.90:0/2600293493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 0x55bea3f6b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.086+0000 7fb468a00700 1 -- 172.21.15.90:0/2600293493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea3e3d420 con 0x55bea3dd7e40 2024-05-27T07:59:55.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.086+0000 7fb469400700 1 --2- 172.21.15.90:0/2600293493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3ec39f0 0x55bea3f5eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T07:59:55.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.087+0000 7fb468a00700 1 --2- 172.21.15.90:0/2600293493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3dd7e40 0x55bea3f5f090 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb450010f40 tx=0x7fb450013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79607e6bf69793f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.087+0000 7fb456a00700 1 -- 172.21.15.90:0/2600293493 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45000be60 con 0x55bea3dd7e40 2024-05-27T07:59:55.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.088+0000 7fb456a00700 1 -- 172.21.15.90:0/2600293493 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb450011db0 con 0x55bea3dd7e40 2024-05-27T07:59:55.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.088+0000 7fb456a00700 1 -- 172.21.15.90:0/2600293493 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45001c650 con 0x55bea3dd7e40 2024-05-27T07:59:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.090+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2600293493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3dd7e40 msgr2=0x55bea3f5f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.090+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/2600293493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3dd7e40 0x55bea3f5f090 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb450010f40 tx=0x7fb450013000 comp rx=0 tx=0).stop 2024-05-27T07:59:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.090+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2600293493 shutdown_connections 2024-05-27T07:59:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.090+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/2600293493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 0x55bea3f6b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.090+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/2600293493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3ec39f0 0x55bea3f5eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.090+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/2600293493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3dd7e40 0x55bea3f5f090 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.090+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2600293493 >> 172.21.15.90:0/2600293493 conn(0x55bea3e71390 msgr2=0x55bea3f52320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.090+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2600293493 shutdown_connections 2024-05-27T07:59:55.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.090+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2600293493 wait complete. 2024-05-27T07:59:55.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.091+0000 7fb47a9c6b80 1 -- start start 2024-05-27T07:59:55.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.091+0000 7fb47a9c6b80 1 Processor -- start 2024-05-27T07:59:55.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.094+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3dd7e40 0x55bea3f68f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.094+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.094+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 0x55bea3f699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.094+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bea3d7afa0 con 0x55bea3ec39f0 2024-05-27T07:59:55.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.094+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bea3e15760 con 0x55bea3dd7e40 2024-05-27T07:59:55.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.094+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bea3f6c3a0 con 0x55bea3f5f690 2024-05-27T07:59:55.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.094+0000 7fb47a9c6b80 10 client.0 _ll_register_callbacks cb 0x55bea3e1ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:55.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.095+0000 7fb468a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.096+0000 7fb468a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f694b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59084/0 (socket says 172.21.15.90:59084) 2024-05-27T07:59:55.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.096+0000 7fb468a00700 1 -- 172.21.15.90:0/3200240741 learned_addr learned my addr 172.21.15.90:0/3200240741 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:55.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.096+0000 7fb469e00700 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 0x55bea3f699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.096+0000 7fb469400700 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3dd7e40 0x55bea3f68f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.097+0000 7fb469e00700 1 -- 172.21.15.90:0/3200240741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3dd7e40 msgr2=0x55bea3f68f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.097+0000 7fb469e00700 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3dd7e40 0x55bea3f68f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.097+0000 7fb469e00700 1 -- 172.21.15.90:0/3200240741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 msgr2=0x55bea3f694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.097+0000 7fb469e00700 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.097+0000 7fb469e00700 1 -- 172.21.15.90:0/3200240741 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea3dc48d0 con 0x55bea3f5f690 2024-05-27T07:59:55.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.097+0000 7fb468a00700 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T07:59:55.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.098+0000 7fb469e00700 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 0x55bea3f699f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb45c010720 tx=0x7fb45c010ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22ed624d91ecba07 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.098+0000 7fb44b400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T07:59:55.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.099+0000 7fb44b400700 1 -- 172.21.15.90:0/3200240741 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c01e060 con 0x55bea3f5f690 2024-05-27T07:59:55.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.099+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/3200240741 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb450011db0 con 0x55bea3f5f690 2024-05-27T07:59:55.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.099+0000 7fb44b400700 1 -- 172.21.15.90:0/3200240741 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fb45c00f770 con 0x55bea3f5f690 2024-05-27T07:59:55.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.099+0000 7fb44b400700 1 -- 172.21.15.90:0/3200240741 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb45c025050 con 0x55bea3f5f690 2024-05-27T07:59:55.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.099+0000 7fb47a9c6b80 10 client.27496 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:55.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.099+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/3200240741 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55bea3e45060 con 0x55bea3f5f690 2024-05-27T07:59:55.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.100+0000 7fb44b400700 1 -- 172.21.15.90:0/3200240741 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb450011db0 con 0x55bea3f5f690 2024-05-27T07:59:55.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.101+0000 7fb44b400700 1 -- 172.21.15.90:0/3200240741 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fb45c00eb20 con 0x55bea3f5f690 2024-05-27T07:59:55.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.101+0000 7fb47a9c6b80 10 client.27496 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:55.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.101+0000 7fb47a9c6b80 10 client.27496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:55.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.101+0000 7fb47a9c6b80 20 client.27496 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:55.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.101+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bea3f79430 0x55bea3f7b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.101+0000 7fb47a9c6b80 4 client.27496 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T07:59:55.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.101+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/3200240741 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bea3e86d90 con 0x55bea3f79430 2024-05-27T07:59:55.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.102+0000 7fb469400700 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bea3f79430 0x55bea3f7b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.103+0000 7fb469400700 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bea3f79430 0x55bea3f7b8e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b615aefdbad35763 server_cookie=7632ce8faa65271d in_seq=0 out_seq=0 2024-05-27T07:59:55.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.103+0000 7fb44b400700 10 client.27496 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.105+0000 7fb44b400700 1 -- 172.21.15.90:0/3200240741 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55bea3e86d90 con 0x55bea3f79430 2024-05-27T07:59:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.105+0000 7fb44b400700 10 client.27496 handle_command_reply: tid=0 2024-05-27T07:59:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.105+0000 7fb47a9c6b80 1 client.27496 shutdown 2024-05-27T07:59:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.106+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/3200240741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 msgr2=0x55bea3f699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.106+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 0x55bea3f699f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb45c010720 tx=0x7fb45c010ae0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.106+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/3200240741 shutdown_connections 2024-05-27T07:59:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.106+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bea3f79430 0x55bea3f7b8e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.106+0000 7fb44b400700 0 client.27496 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.106+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 0x55bea3f699f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.106+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3dd7e40 0x55bea3f68f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.106+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/3200240741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.106+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/3200240741 >> 172.21.15.90:0/3200240741 conn(0x55bea3e71390 msgr2=0x55bea3f5ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.107+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/3200240741 shutdown_connections 2024-05-27T07:59:55.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.107+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/3200240741 wait complete. 2024-05-27T07:59:55.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.107+0000 7fb47a9c6b80 20 client.27496 trim_cache size 0 max 16384 2024-05-27T07:59:55.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.135+0000 7fb47a9c6b80 1 Processor -- start 2024-05-27T07:59:55.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.135+0000 7fb47a9c6b80 1 -- start start 2024-05-27T07:59:55.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.136+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3dd7e40 0x55bea3f5f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.136+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f64770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.136+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 0x55bea3f75470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.136+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4440107c0 con 0x55bea3ec39f0 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.136+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb44400d810 con 0x55bea3f5f690 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.136+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb44400fa20 con 0x55bea3dd7e40 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.136+0000 7fb468a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f64770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.137+0000 7fb468a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f64770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59092/0 (socket says 172.21.15.90:59092) 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.137+0000 7fb468a00700 1 -- 172.21.15.90:0/4162168881 learned_addr learned my addr 172.21.15.90:0/4162168881 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.137+0000 7fb469e00700 1 --2- 172.21.15.90:0/4162168881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 0x55bea3f75470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.137+0000 7fb469400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3dd7e40 0x55bea3f5f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.138+0000 7fb469e00700 1 -- 172.21.15.90:0/4162168881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3dd7e40 msgr2=0x55bea3f5f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.138+0000 7fb469e00700 1 --2- 172.21.15.90:0/4162168881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3dd7e40 0x55bea3f5f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.138+0000 7fb469e00700 1 -- 172.21.15.90:0/4162168881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 msgr2=0x55bea3f64770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.138+0000 7fb469e00700 1 --2- 172.21.15.90:0/4162168881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f64770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.138+0000 7fb469e00700 1 -- 172.21.15.90:0/4162168881 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea3f4e240 con 0x55bea3f5f690 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.138+0000 7fb469e00700 1 --2- 172.21.15.90:0/4162168881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 0x55bea3f75470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb45c020470 tx=0x7fb45c00a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daa872a1e75d799b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.139+0000 7fb456000700 1 -- 172.21.15.90:0/4162168881 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c02e070 con 0x55bea3f5f690 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.139+0000 7fb456000700 1 -- 172.21.15.90:0/4162168881 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb45c00d480 con 0x55bea3f5f690 2024-05-27T07:59:55.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.139+0000 7fb456000700 1 -- 172.21.15.90:0/4162168881 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c01e070 con 0x55bea3f5f690 2024-05-27T07:59:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.141+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/4162168881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 msgr2=0x55bea3f75470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.141+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/4162168881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 0x55bea3f75470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb45c020470 tx=0x7fb45c00a360 comp rx=0 tx=0).stop 2024-05-27T07:59:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.141+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/4162168881 shutdown_connections 2024-05-27T07:59:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.141+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/4162168881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3f5f690 0x55bea3f75470 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.141+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/4162168881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3dd7e40 0x55bea3f5f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.141+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/4162168881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f64770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.141+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/4162168881 >> 172.21.15.90:0/4162168881 conn(0x55bea3e71390 msgr2=0x55bea3f626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.141+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/4162168881 shutdown_connections 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.141+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/4162168881 wait complete. 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.142+0000 7fb47a9c6b80 1 -- start start 2024-05-27T07:59:55.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.142+0000 7fb47a9c6b80 1 Processor -- start 2024-05-27T07:59:55.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.143+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3dd7e40 0x55bea3f71660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.143+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f71ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.143+0000 7fb47a9c6b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 0x55bea3f6cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.143+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb44400f350 con 0x55bea3ec39f0 2024-05-27T07:59:55.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.143+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb44400dee0 con 0x55bea3dd7e40 2024-05-27T07:59:55.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.143+0000 7fb47a9c6b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb44400e5b0 con 0x55bea3f5f690 2024-05-27T07:59:55.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.143+0000 7fb47a9c6b80 10 client.0 _ll_register_callbacks cb 0x55bea3e1ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T07:59:55.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.143+0000 7fb468a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f71ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.143+0000 7fb468a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f71ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59100/0 (socket says 172.21.15.90:59100) 2024-05-27T07:59:55.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.143+0000 7fb468a00700 1 -- 172.21.15.90:0/2247314521 learned_addr learned my addr 172.21.15.90:0/2247314521 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T07:59:55.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.144+0000 7fb469e00700 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 0x55bea3f6cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.144+0000 7fb468a00700 1 -- 172.21.15.90:0/2247314521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 msgr2=0x55bea3f6cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.144+0000 7fb468a00700 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 0x55bea3f6cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.144+0000 7fb468a00700 1 -- 172.21.15.90:0/2247314521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3dd7e40 msgr2=0x55bea3f71660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T07:59:55.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.144+0000 7fb469400700 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3dd7e40 0x55bea3f71660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.144+0000 7fb468a00700 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3dd7e40 0x55bea3f71660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.144+0000 7fb468a00700 1 -- 172.21.15.90:0/2247314521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea3f6d2b0 con 0x55bea3ec39f0 2024-05-27T07:59:55.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.144+0000 7fb468a00700 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f71ba0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb450011fa0 tx=0x7fb450011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afce4f59e440bf8a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T07:59:55.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.144+0000 7fb44b400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T07:59:55.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.145+0000 7fb44b400700 1 -- 172.21.15.90:0/2247314521 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450005590 con 0x55bea3ec39f0 2024-05-27T07:59:55.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.145+0000 7fb44b400700 1 -- 172.21.15.90:0/2247314521 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9496+0+0 (secure 0 0 0) 0x7fb4500046d0 con 0x55bea3ec39f0 2024-05-27T07:59:55.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.145+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2247314521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb45c00d480 con 0x55bea3ec39f0 2024-05-27T07:59:55.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.145+0000 7fb44b400700 1 -- 172.21.15.90:0/2247314521 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb45003f050 con 0x55bea3ec39f0 2024-05-27T07:59:55.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.146+0000 7fb47a9c6b80 10 client.17790 fetch_fsmap learned FSMap version 64 2024-05-27T07:59:55.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.146+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2247314521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55bea3fe38b0 con 0x55bea3ec39f0 2024-05-27T07:59:55.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.146+0000 7fb44b400700 1 -- 172.21.15.90:0/2247314521 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb45c00d480 con 0x55bea3ec39f0 2024-05-27T07:59:55.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.147+0000 7fb44b400700 1 -- 172.21.15.90:0/2247314521 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fb45001e790 con 0x55bea3ec39f0 2024-05-27T07:59:55.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.147+0000 7fb47a9c6b80 10 client.17790 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T07:59:55.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.147+0000 7fb47a9c6b80 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T07:59:55.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.147+0000 7fb47a9c6b80 20 client.17790 populate_metadata read hostname 'smithi090' 2024-05-27T07:59:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.147+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bea3f79430 0x55bea3fe5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T07:59:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.147+0000 7fb47a9c6b80 4 client.17790 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T07:59:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.147+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2247314521 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bea3e80570 con 0x55bea3f79430 2024-05-27T07:59:55.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.149+0000 7fb469400700 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bea3f79430 0x55bea3fe5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T07:59:55.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.151+0000 7fb469400700 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bea3f79430 0x55bea3fe5fd0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dfa3cd4177767dd server_cookie=c7e4bf56d60bf008 in_seq=0 out_seq=0 2024-05-27T07:59:55.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.151+0000 7fb44b400700 10 client.17790 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:55.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.152+0000 7fb44b400700 1 -- 172.21.15.90:0/2247314521 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bea3e80570 con 0x55bea3f79430 2024-05-27T07:59:55.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.152+0000 7fb44b400700 10 client.17790 handle_command_reply: tid=0 2024-05-27T07:59:55.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.152+0000 7fb47a9c6b80 1 client.17790 shutdown 2024-05-27T07:59:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.152+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2247314521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 msgr2=0x55bea3f71ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.152+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f71ba0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fb450011fa0 tx=0x7fb450011f10 comp rx=0 tx=0).stop 2024-05-27T07:59:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.152+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2247314521 shutdown_connections 2024-05-27T07:59:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.153+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55bea3f79430 0x55bea3fe5fd0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.153+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bea3f5f690 0x55bea3f6cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.153+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bea3dd7e40 0x55bea3f71660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.153+0000 7fb47a9c6b80 1 --2- 172.21.15.90:0/2247314521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bea3ec39f0 0x55bea3f71ba0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.153+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2247314521 >> 172.21.15.90:0/2247314521 conn(0x55bea3e71390 msgr2=0x55bea3f69780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.153+0000 7fb44b400700 0 client.17790 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T07:59:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.153+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2247314521 shutdown_connections 2024-05-27T07:59:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.153+0000 7fb47a9c6b80 1 -- 172.21.15.90:0/2247314521 wait complete. 2024-05-27T07:59:55.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.154+0000 7fb47a9c6b80 20 client.17790 trim_cache size 0 max 16384 2024-05-27T07:59:55.157 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T07:59:55.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.155+0000 7fb46be00700 1 -- 172.21.15.90:0/1887759143 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb44c0b2af0 msgr2=0x7fb44c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.155+0000 7fb46be00700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb44c0b2af0 0x7fb44c0b5000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb464102640 tx=0x7fb45000b410 comp rx=0 tx=0).stop 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.155+0000 7fb46be00700 1 -- 172.21.15.90:0/1887759143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4641034b0 msgr2=0x7fb46419cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.155+0000 7fb46be00700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4641034b0 0x7fb46419cfd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb458000f80 tx=0x7fb458004d00 comp rx=0 tx=0).stop 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.155+0000 7fb46be00700 1 -- 172.21.15.90:0/1887759143 shutdown_connections 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.155+0000 7fb46be00700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb464108730 0x7fb4641a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.155+0000 7fb46be00700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb44c0b2af0 0x7fb44c0b5000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.155+0000 7fb46be00700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4641034b0 0x7fb46419cfd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.155+0000 7fb46be00700 1 --2- 172.21.15.90:0/1887759143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb464107de0 0x7fb46419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.155+0000 7fb46be00700 1 -- 172.21.15.90:0/1887759143 >> 172.21.15.90:0/1887759143 conn(0x7fb4640fede0 msgr2=0x7fb464101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.156+0000 7fb46be00700 1 -- 172.21.15.90:0/1887759143 shutdown_connections 2024-05-27T07:59:55.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T07:59:55.157+0000 7fb46be00700 1 -- 172.21.15.90:0/1887759143 wait complete. 2024-05-27T07:59:55.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:54 smithi136 ceph-mon[35324]: pgmap v432: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 664 op/s 2024-05-27T07:59:55.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:54 smithi129 ceph-mon[35254]: pgmap v432: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.8 MiB/s wr, 664 op/s 2024-05-27T07:59:56.978 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T07:59:56.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T07:59:57.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:56 smithi136 ceph-mon[35324]: pgmap v433: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 498 op/s 2024-05-27T07:59:57.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:56 smithi129 ceph-mon[35254]: pgmap v433: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 498 op/s 2024-05-27T07:59:57.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:56 smithi090 ceph-mon[31429]: pgmap v433: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 498 op/s 2024-05-27T07:59:57.572 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T07:59:58.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 580.0 s, 677.0 tps, lat 152.299 ms stddev 259.684 2024-05-27T07:59:59.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 07:59:58 smithi136 ceph-mon[35324]: pgmap v434: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 582 op/s 2024-05-27T07:59:59.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 07:59:58 smithi129 ceph-mon[35254]: pgmap v434: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 582 op/s 2024-05-27T07:59:59.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 07:59:58 smithi090 ceph-mon[31429]: pgmap v434: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 582 op/s 2024-05-27T08:00:00.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:00 smithi129 ceph-mon[35254]: Health detail: HEALTH_WARN 3 failed cephadm daemon(s) 2024-05-27T08:00:00.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:00 smithi129 ceph-mon[35254]: [WRN] CEPHADM_FAILED_DAEMON: 3 failed cephadm daemon(s) 2024-05-27T08:00:00.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:00 smithi129 ceph-mon[35254]: daemon mds.j on smithi090 is in error state 2024-05-27T08:00:00.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:00 smithi129 ceph-mon[35254]: daemon mds.d on smithi090 is in error state 2024-05-27T08:00:00.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:00 smithi129 ceph-mon[35254]: daemon mds.c on smithi136 is in error state 2024-05-27T08:00:00.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:00 smithi090 ceph-mon[31429]: Health detail: HEALTH_WARN 3 failed cephadm daemon(s) 2024-05-27T08:00:00.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:00 smithi090 ceph-mon[31429]: [WRN] CEPHADM_FAILED_DAEMON: 3 failed cephadm daemon(s) 2024-05-27T08:00:00.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:00 smithi090 ceph-mon[31429]: daemon mds.j on smithi090 is in error state 2024-05-27T08:00:00.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:00 smithi090 ceph-mon[31429]: daemon mds.d on smithi090 is in error state 2024-05-27T08:00:00.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:00 smithi090 ceph-mon[31429]: daemon mds.c on smithi136 is in error state 2024-05-27T08:00:00.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:00 smithi136 ceph-mon[35324]: Health detail: HEALTH_WARN 3 failed cephadm daemon(s) 2024-05-27T08:00:00.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:00 smithi136 ceph-mon[35324]: [WRN] CEPHADM_FAILED_DAEMON: 3 failed cephadm daemon(s) 2024-05-27T08:00:00.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:00 smithi136 ceph-mon[35324]: daemon mds.j on smithi090 is in error state 2024-05-27T08:00:00.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:00 smithi136 ceph-mon[35324]: daemon mds.d on smithi090 is in error state 2024-05-27T08:00:00.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:00 smithi136 ceph-mon[35324]: daemon mds.c on smithi136 is in error state 2024-05-27T08:00:01.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:01 smithi129 ceph-mon[35254]: pgmap v435: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 495 op/s 2024-05-27T08:00:01.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:01 smithi090 ceph-mon[31429]: pgmap v435: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 495 op/s 2024-05-27T08:00:01.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:01 smithi136 ceph-mon[35324]: pgmap v435: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 495 op/s 2024-05-27T08:00:03.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:02 smithi136 ceph-mon[35324]: pgmap v436: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.9 MiB/s wr, 511 op/s 2024-05-27T08:00:03.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:02 smithi129 ceph-mon[35254]: pgmap v436: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.9 MiB/s wr, 511 op/s 2024-05-27T08:00:03.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:02 smithi090 ceph-mon[31429]: pgmap v436: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.9 MiB/s wr, 511 op/s 2024-05-27T08:00:03.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 585.0 s, 549.0 tps, lat 199.552 ms stddev 212.371 2024-05-27T08:00:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.378+0000 7fa447200640 1 -- 172.21.15.90:0/3543194229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa448104250 msgr2=0x7fa4481105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.378+0000 7fa447200640 1 --2- 172.21.15.90:0/3543194229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa448104250 0x7fa4481105d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa440009ca0 tx=0x7fa44002a8e0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.378+0000 7fa447200640 1 -- 172.21.15.90:0/3543194229 shutdown_connections 2024-05-27T08:00:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.378+0000 7fa447200640 1 --2- 172.21.15.90:0/3543194229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa448109f50 0x7fa448103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.378+0000 7fa447200640 1 --2- 172.21.15.90:0/3543194229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa448104250 0x7fa4481105d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.378+0000 7fa447200640 1 --2- 172.21.15.90:0/3543194229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa448109ab0 0x7fa4481036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.378+0000 7fa447200640 1 -- 172.21.15.90:0/3543194229 >> 172.21.15.90:0/3543194229 conn(0x7fa448079240 msgr2=0x7fa448079680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.378+0000 7fa447200640 1 -- 172.21.15.90:0/3543194229 shutdown_connections 2024-05-27T08:00:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.380+0000 7fa447200640 1 -- 172.21.15.90:0/3543194229 wait complete. 2024-05-27T08:00:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.381+0000 7fa447200640 1 Processor -- start 2024-05-27T08:00:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.382+0000 7fa447200640 1 -- start start 2024-05-27T08:00:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.382+0000 7fa447200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa448104250 0x7fa4481a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.382+0000 7fa447200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa448109ab0 0x7fa4481a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.382+0000 7fa447200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa448109f50 0x7fa44819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.382+0000 7fa447200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa448113bc0 con 0x7fa448109f50 2024-05-27T08:00:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.382+0000 7fa447200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa448113a40 con 0x7fa448104250 2024-05-27T08:00:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.382+0000 7fa447200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa448113d40 con 0x7fa448109ab0 2024-05-27T08:00:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.383+0000 7fa445e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa448104250 0x7fa4481a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.383+0000 7fa445e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa448104250 0x7fa4481a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47960/0 (socket says 172.21.15.90:47960) 2024-05-27T08:00:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.383+0000 7fa445e00640 1 -- 172.21.15.90:0/4205384525 learned_addr learned my addr 172.21.15.90:0/4205384525 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.383+0000 7fa445400640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa448109ab0 0x7fa4481a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.384+0000 7fa445e00640 1 -- 172.21.15.90:0/4205384525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa448109ab0 msgr2=0x7fa4481a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.384+0000 7fa445e00640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa448109ab0 0x7fa4481a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.384+0000 7fa445e00640 1 -- 172.21.15.90:0/4205384525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa448109f50 msgr2=0x7fa44819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.384+0000 7fa445e00640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa448109f50 0x7fa44819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.384+0000 7fa445e00640 1 -- 172.21.15.90:0/4205384525 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44819cd40 con 0x7fa448104250 2024-05-27T08:00:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.384+0000 7fa445e00640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa448104250 0x7fa4481a2580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa43800c970 tx=0x7fa43800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.385+0000 7fa43ea00640 1 -- 172.21.15.90:0/4205384525 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438014070 con 0x7fa448104250 2024-05-27T08:00:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.385+0000 7fa43ea00640 1 -- 172.21.15.90:0/4205384525 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa438004440 con 0x7fa448104250 2024-05-27T08:00:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.385+0000 7fa43ea00640 1 -- 172.21.15.90:0/4205384525 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438003ce0 con 0x7fa448104250 2024-05-27T08:00:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.386+0000 7fa447200640 1 -- 172.21.15.90:0/4205384525 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44819d030 con 0x7fa448104250 2024-05-27T08:00:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.386+0000 7fa447200640 1 -- 172.21.15.90:0/4205384525 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44819d530 con 0x7fa448104250 2024-05-27T08:00:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.394+0000 7fa43ea00640 1 -- 172.21.15.90:0/4205384525 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa438017070 con 0x7fa448104250 2024-05-27T08:00:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.397+0000 7fa43ea00640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa42c0b2aa0 0x7fa42c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.397+0000 7fa453a23740 1 Processor -- start 2024-05-27T08:00:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.397+0000 7fa43ea00640 1 -- 172.21.15.90:0/4205384525 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9584+0+0 (secure 0 0 0) 0x7fa4380e28f0 con 0x7fa448104250 2024-05-27T08:00:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.398+0000 7fa453a23740 1 -- start start 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.399+0000 7fa445400640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa42c0b2aa0 0x7fa42c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.399+0000 7fa453a23740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a14201b0 0x5639a17c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.399+0000 7fa453a23740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13f72f0 0x5639a17c2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.399+0000 7fa453a23740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 0x5639a17c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.399+0000 7fa453a23740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5639a1733b90 con 0x5639a14201b0 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.399+0000 7fa453a23740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5639a1723730 con 0x5639a13f72f0 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.399+0000 7fa453a23740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5639a1727770 con 0x5639a13ff4d0 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.400+0000 7fa445e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a14201b0 0x5639a17c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.400+0000 7fa445e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a14201b0 0x5639a17c2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38362/0 (socket says 172.21.15.90:38362) 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.400+0000 7fa445e00640 1 -- 172.21.15.90:0/2120940660 learned_addr learned my addr 172.21.15.90:0/2120940660 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.400+0000 7fa445400640 1 --2- 172.21.15.90:0/2120940660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13f72f0 0x5639a17c2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.400+0000 7fa445e00640 1 -- 172.21.15.90:0/2120940660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 msgr2=0x5639a17c34e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:00:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.400+0000 7fa445e00640 1 --2- 172.21.15.90:0/2120940660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 0x5639a17c34e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.400+0000 7fa445e00640 1 -- 172.21.15.90:0/2120940660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13f72f0 msgr2=0x5639a17c2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.400+0000 7fa445e00640 1 --2- 172.21.15.90:0/2120940660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13f72f0 0x5639a17c2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.400+0000 7fa445e00640 1 -- 172.21.15.90:0/2120940660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a16bc8b0 con 0x5639a14201b0 2024-05-27T08:00:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.401+0000 7fa445e00640 1 --2- 172.21.15.90:0/2120940660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a14201b0 0x5639a17c2a60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa4380dfbc0 tx=0x7fa4380a9280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbf55b9b82e5b80 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.402+0000 7fa433000640 1 -- 172.21.15.90:0/2120940660 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4380c3070 con 0x5639a14201b0 2024-05-27T08:00:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.402+0000 7fa433000640 1 -- 172.21.15.90:0/2120940660 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4380a9830 con 0x5639a14201b0 2024-05-27T08:00:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.402+0000 7fa433000640 1 -- 172.21.15.90:0/2120940660 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4380a8a90 con 0x5639a14201b0 2024-05-27T08:00:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.404+0000 7fa445400640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa42c0b2aa0 0x7fa42c0b4fb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa434005fd0 tx=0x7fa43400b760 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.404+0000 7fa453a23740 1 -- 172.21.15.90:0/2120940660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a14201b0 msgr2=0x5639a17c2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.404+0000 7fa453a23740 1 --2- 172.21.15.90:0/2120940660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a14201b0 0x5639a17c2a60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa4380dfbc0 tx=0x7fa4380a9280 comp rx=0 tx=0).stop 2024-05-27T08:00:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.404+0000 7fa453a23740 1 -- 172.21.15.90:0/2120940660 shutdown_connections 2024-05-27T08:00:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.404+0000 7fa453a23740 1 --2- 172.21.15.90:0/2120940660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13f72f0 0x5639a17c2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.404+0000 7fa453a23740 1 --2- 172.21.15.90:0/2120940660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 0x5639a17c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.404+0000 7fa453a23740 1 --2- 172.21.15.90:0/2120940660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a14201b0 0x5639a17c2a60 secure :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa4380dfbc0 tx=0x7fa4380a9280 comp rx=0 tx=0).stop 2024-05-27T08:00:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.404+0000 7fa453a23740 1 -- 172.21.15.90:0/2120940660 >> 172.21.15.90:0/2120940660 conn(0x5639a1414300 msgr2=0x5639a17bf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.405+0000 7fa453a23740 1 -- 172.21.15.90:0/2120940660 shutdown_connections 2024-05-27T08:00:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.405+0000 7fa453a23740 1 -- 172.21.15.90:0/2120940660 wait complete. 2024-05-27T08:00:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.406+0000 7fa453a23740 1 -- start start 2024-05-27T08:00:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.406+0000 7fa453a23740 1 Processor -- start 2024-05-27T08:00:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.407+0000 7fa453a23740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 0x5639a17c5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.407+0000 7fa453a23740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ff4d0 0x5639a17c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.408+0000 7fa453a23740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 0x5639a17c6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.408+0000 7fa445400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ff4d0 0x5639a17c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.408+0000 7fa453a23740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5639a1409790 con 0x5639a13ff4d0 2024-05-27T08:00:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.409+0000 7fa445400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ff4d0 0x5639a17c5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38378/0 (socket says 172.21.15.90:38378) 2024-05-27T08:00:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.409+0000 7fa445400640 1 -- 172.21.15.90:0/3149519204 learned_addr learned my addr 172.21.15.90:0/3149519204 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.410+0000 7fa446800640 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 0x5639a17c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.410+0000 7fa453a23740 1 -- 172.21.15.90:0/3149519204 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5639a16cbbc0 con 0x5639a13ded80 2024-05-27T08:00:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.410+0000 7fa453a23740 1 -- 172.21.15.90:0/3149519204 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5639a17cfb30 con 0x5639a13f72f0 2024-05-27T08:00:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.410+0000 7fa445e00640 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 0x5639a17c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.410+0000 7fa453a23740 10 client.0 _ll_register_callbacks cb 0x5639a16a9400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.412+0000 7fa445e00640 1 -- 172.21.15.90:0/3149519204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 msgr2=0x5639a17c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.412+0000 7fa445e00640 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 0x5639a17c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.412+0000 7fa445e00640 1 -- 172.21.15.90:0/3149519204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ff4d0 msgr2=0x5639a17c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.412+0000 7fa445e00640 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ff4d0 0x5639a17c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.412+0000 7fa445e00640 1 -- 172.21.15.90:0/3149519204 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a1438210 con 0x5639a13f72f0 2024-05-27T08:00:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.413+0000 7fa446800640 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 0x5639a17c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.413+0000 7fa445400640 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ff4d0 0x5639a17c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.414+0000 7fa445e00640 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 0x5639a17c5840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa4380dfbc0 tx=0x7fa4380abda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb33e519cb3fa6cb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.415+0000 7fa427400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:00:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.415+0000 7fa427400640 1 -- 172.21.15.90:0/3149519204 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4380c3050 con 0x5639a13f72f0 2024-05-27T08:00:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.415+0000 7fa427400640 1 -- 172.21.15.90:0/3149519204 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9584+0+0 (secure 0 0 0) 0x7fa4380a5e20 con 0x5639a13f72f0 2024-05-27T08:00:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.415+0000 7fa453a23740 1 -- 172.21.15.90:0/3149519204 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4380a9830 con 0x5639a13f72f0 2024-05-27T08:00:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.416+0000 7fa427400640 1 -- 172.21.15.90:0/3149519204 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa4380adbe0 con 0x5639a13f72f0 2024-05-27T08:00:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.416+0000 7fa453a23740 10 client.27511 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.416+0000 7fa453a23740 1 -- 172.21.15.90:0/3149519204 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5639a1504620 con 0x5639a13f72f0 2024-05-27T08:00:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.418+0000 7fa427400640 1 -- 172.21.15.90:0/3149519204 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4380a9830 con 0x5639a13f72f0 2024-05-27T08:00:04.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.418+0000 7fa427400640 1 -- 172.21.15.90:0/3149519204 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fa4380b9cb0 con 0x5639a13f72f0 2024-05-27T08:00:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.419+0000 7fa453a23740 10 client.27511 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.419+0000 7fa453a23740 10 client.27511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.419+0000 7fa453a23740 20 client.27511 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.419+0000 7fa453a23740 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5639a13b41c0 0x5639a17dee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.419+0000 7fa453a23740 4 client.27511 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.419+0000 7fa453a23740 1 -- 172.21.15.90:0/3149519204 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639a1735ae0 con 0x5639a13b41c0 2024-05-27T08:00:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.419+0000 7fa445400640 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5639a13b41c0 0x5639a17dee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.421+0000 7fa427400640 10 client.27511 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.421+0000 7fa445400640 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5639a13b41c0 0x5639a17dee40 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a8840c2649f3ae server_cookie=12ea950c8214d02b in_seq=0 out_seq=0 2024-05-27T08:00:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.424+0000 7fa427400640 1 -- 172.21.15.90:0/3149519204 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5639a1735ae0 con 0x5639a13b41c0 2024-05-27T08:00:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.424+0000 7fa427400640 10 client.27511 handle_command_reply: tid=0 2024-05-27T08:00:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.427+0000 7fa453a23740 1 client.27511 shutdown 2024-05-27T08:00:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.427+0000 7fa453a23740 1 -- 172.21.15.90:0/3149519204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 msgr2=0x5639a17c5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.427+0000 7fa453a23740 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 0x5639a17c5840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa4380dfbc0 tx=0x7fa4380abda0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.427+0000 7fa453a23740 1 -- 172.21.15.90:0/3149519204 shutdown_connections 2024-05-27T08:00:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.427+0000 7fa453a23740 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5639a13b41c0 0x5639a17dee40 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.427+0000 7fa453a23740 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 0x5639a17c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.427+0000 7fa453a23740 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ff4d0 0x5639a17c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.427+0000 7fa427400640 0 client.27511 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.427+0000 7fa453a23740 1 --2- 172.21.15.90:0/3149519204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 0x5639a17c5840 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.427+0000 7fa453a23740 1 -- 172.21.15.90:0/3149519204 >> 172.21.15.90:0/3149519204 conn(0x5639a1414300 msgr2=0x5639a17c29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.429+0000 7fa453a23740 1 -- 172.21.15.90:0/3149519204 shutdown_connections 2024-05-27T08:00:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.429+0000 7fa453a23740 1 -- 172.21.15.90:0/3149519204 wait complete. 2024-05-27T08:00:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.429+0000 7fa453a23740 20 client.27511 trim_cache size 0 max 16384 2024-05-27T08:00:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.475+0000 7fa453a23740 1 Processor -- start 2024-05-27T08:00:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.475+0000 7fa453a23740 1 -- start start 2024-05-27T08:00:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.475+0000 7fa453a23740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ded80 0x5639a17c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.475+0000 7fa453a23740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 0x5639a17c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.475+0000 7fa453a23740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ff4d0 0x5639a17e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.475+0000 7fa453a23740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481b0a00 con 0x5639a13ded80 2024-05-27T08:00:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.475+0000 7fa453a23740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481ada50 con 0x5639a13ff4d0 2024-05-27T08:00:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.475+0000 7fa453a23740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481afc60 con 0x5639a13f72f0 2024-05-27T08:00:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.476+0000 7fa445e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ded80 0x5639a17c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.476+0000 7fa445400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 0x5639a17c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.477+0000 7fa445400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 0x5639a17c5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:49880/0 (socket says 172.21.15.90:49880) 2024-05-27T08:00:04.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.477+0000 7fa445400640 1 -- 172.21.15.90:0/3502650241 learned_addr learned my addr 172.21.15.90:0/3502650241 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.477+0000 7fa446800640 1 --2- 172.21.15.90:0/3502650241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ff4d0 0x5639a17e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.477+0000 7fa445e00640 1 -- 172.21.15.90:0/3502650241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 msgr2=0x5639a17c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.477+0000 7fa445e00640 1 --2- 172.21.15.90:0/3502650241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 0x5639a17c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.477+0000 7fa445e00640 1 -- 172.21.15.90:0/3502650241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ff4d0 msgr2=0x5639a17e44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.477+0000 7fa445e00640 1 --2- 172.21.15.90:0/3502650241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ff4d0 0x5639a17e44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.477+0000 7fa445e00640 1 -- 172.21.15.90:0/3502650241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a17b9210 con 0x5639a13ded80 2024-05-27T08:00:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.478+0000 7fa446800640 1 --2- 172.21.15.90:0/3502650241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ff4d0 0x5639a17e44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.478+0000 7fa445e00640 1 --2- 172.21.15.90:0/3502650241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ded80 0x5639a17c5720 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa4380ad520 tx=0x7fa4380045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dc34ad229f29c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.481+0000 7fa432600640 1 -- 172.21.15.90:0/3502650241 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4380c3070 con 0x5639a13ded80 2024-05-27T08:00:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.481+0000 7fa432600640 1 -- 172.21.15.90:0/3502650241 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4380c6070 con 0x5639a13ded80 2024-05-27T08:00:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.481+0000 7fa432600640 1 -- 172.21.15.90:0/3502650241 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4380b2690 con 0x5639a13ded80 2024-05-27T08:00:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.482+0000 7fa453a23740 1 -- 172.21.15.90:0/3502650241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ded80 msgr2=0x5639a17c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.482+0000 7fa453a23740 1 --2- 172.21.15.90:0/3502650241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ded80 0x5639a17c5720 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa4380ad520 tx=0x7fa4380045f0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.482+0000 7fa453a23740 1 -- 172.21.15.90:0/3502650241 shutdown_connections 2024-05-27T08:00:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.482+0000 7fa453a23740 1 --2- 172.21.15.90:0/3502650241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ff4d0 0x5639a17e44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.482+0000 7fa453a23740 1 --2- 172.21.15.90:0/3502650241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13f72f0 0x5639a17c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.482+0000 7fa453a23740 1 --2- 172.21.15.90:0/3502650241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13ded80 0x5639a17c5720 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.482+0000 7fa453a23740 1 -- 172.21.15.90:0/3502650241 >> 172.21.15.90:0/3502650241 conn(0x5639a1414300 msgr2=0x5639a17d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.482+0000 7fa453a23740 1 -- 172.21.15.90:0/3502650241 shutdown_connections 2024-05-27T08:00:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.483+0000 7fa453a23740 1 -- 172.21.15.90:0/3502650241 wait complete. 2024-05-27T08:00:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.483+0000 7fa453a23740 1 -- start start 2024-05-27T08:00:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.484+0000 7fa453a23740 1 Processor -- start 2024-05-27T08:00:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.487+0000 7fa453a23740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 0x5639a183e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.487+0000 7fa453a23740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13f72f0 0x5639a183ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.487+0000 7fa453a23740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 0x5639a17c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.488+0000 7fa453a23740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481af590 con 0x5639a13f72f0 2024-05-27T08:00:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.488+0000 7fa446800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 0x5639a17c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.488+0000 7fa446800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 0x5639a17c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:49896/0 (socket says 172.21.15.90:49896) 2024-05-27T08:00:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.488+0000 7fa446800640 1 -- 172.21.15.90:0/3383698154 learned_addr learned my addr 172.21.15.90:0/3383698154 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.488+0000 7fa453a23740 1 -- 172.21.15.90:0/3383698154 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481ae120 con 0x5639a13ded80 2024-05-27T08:00:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.489+0000 7fa445e00640 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 0x5639a183e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.489+0000 7fa453a23740 1 -- 172.21.15.90:0/3383698154 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa4481ae7f0 con 0x5639a13ff4d0 2024-05-27T08:00:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.489+0000 7fa453a23740 10 client.0 _ll_register_callbacks cb 0x7fa4380c1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.490+0000 7fa446800640 1 -- 172.21.15.90:0/3383698154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 msgr2=0x5639a183e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.490+0000 7fa446800640 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 0x5639a183e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.490+0000 7fa446800640 1 -- 172.21.15.90:0/3383698154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13f72f0 msgr2=0x5639a183ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.490+0000 7fa446800640 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13f72f0 0x5639a183ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.490+0000 7fa446800640 1 -- 172.21.15.90:0/3383698154 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a17c8ff0 con 0x5639a13ff4d0 2024-05-27T08:00:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.491+0000 7fa446800640 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 0x5639a17c8900 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa440009ca0 tx=0x7fa440030810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe4dad394248f4f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.491+0000 7fa427400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:00:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.492+0000 7fa427400640 1 -- 172.21.15.90:0/3383698154 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440047020 con 0x5639a13ff4d0 2024-05-27T08:00:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.493+0000 7fa427400640 1 -- 172.21.15.90:0/3383698154 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9584+0+0 (secure 0 0 0) 0x7fa440030a90 con 0x5639a13ff4d0 2024-05-27T08:00:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.493+0000 7fa453a23740 1 -- 172.21.15.90:0/3383698154 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4380c6070 con 0x5639a13ff4d0 2024-05-27T08:00:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.493+0000 7fa427400640 1 -- 172.21.15.90:0/3383698154 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fa44004f050 con 0x5639a13ff4d0 2024-05-27T08:00:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.494+0000 7fa453a23740 10 client.27517 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.494+0000 7fa453a23740 1 -- 172.21.15.90:0/3383698154 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5639a1846d60 con 0x5639a13ff4d0 2024-05-27T08:00:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.494+0000 7fa427400640 1 -- 172.21.15.90:0/3383698154 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4380c6070 con 0x5639a13ff4d0 2024-05-27T08:00:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.496+0000 7fa427400640 1 -- 172.21.15.90:0/3383698154 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fa440042070 con 0x5639a13ff4d0 2024-05-27T08:00:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.496+0000 7fa453a23740 10 client.27517 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:04.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.496+0000 7fa453a23740 10 client.27517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.496+0000 7fa453a23740 20 client.27517 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.496+0000 7fa453a23740 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5639a1847500 0x5639a1849950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.496+0000 7fa453a23740 4 client.27517 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:00:04.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.496+0000 7fa453a23740 1 -- 172.21.15.90:0/3383698154 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639a172f2c0 con 0x5639a1847500 2024-05-27T08:00:04.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.498+0000 7fa445e00640 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5639a1847500 0x5639a1849950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.500+0000 7fa445e00640 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5639a1847500 0x5639a1849950 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234035510bd883df server_cookie=9f9328b66e61977b in_seq=0 out_seq=0 2024-05-27T08:00:04.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:04.501+0000 7fa427400640 10 client.27517 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:04.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.739+0000 7f0147200700 1 -- 172.21.15.90:0/966186125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01400b8340 msgr2=0x7f01400ba760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.739+0000 7f0147200700 1 --2- 172.21.15.90:0/966186125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01400b8340 0x7f01400ba760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0148069810 tx=0x7f014806a480 comp rx=0 tx=0).stop 2024-05-27T08:00:04.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.740+0000 7f0147200700 1 -- 172.21.15.90:0/966186125 shutdown_connections 2024-05-27T08:00:04.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.740+0000 7f0147200700 1 --2- 172.21.15.90:0/966186125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01400b8340 0x7f01400ba760 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.740+0000 7f0147200700 1 --2- 172.21.15.90:0/966186125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01400a53e0 0x7f01400a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.740+0000 7f0147200700 1 --2- 172.21.15.90:0/966186125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01400a5d30 0x7f01400b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.740+0000 7f0147200700 1 -- 172.21.15.90:0/966186125 >> 172.21.15.90:0/966186125 conn(0x7f014001af40 msgr2=0x7f014001b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.740+0000 7f0147200700 1 -- 172.21.15.90:0/966186125 shutdown_connections 2024-05-27T08:00:04.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.740+0000 7f0147200700 1 -- 172.21.15.90:0/966186125 wait complete. 2024-05-27T08:00:04.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.745+0000 7f0147200700 1 Processor -- start 2024-05-27T08:00:04.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.745+0000 7f0147200700 1 -- start start 2024-05-27T08:00:04.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.746+0000 7f0147200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01400a53e0 0x7f01400b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.746+0000 7f0147200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01400a5d30 0x7f01400b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.746+0000 7f0147200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01400b8340 0x7f01400b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.746+0000 7f0147200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f01400bdb60 con 0x7f01400b8340 2024-05-27T08:00:04.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.746+0000 7f0147200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01400bd9e0 con 0x7f01400a5d30 2024-05-27T08:00:04.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.746+0000 7f0147200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01400bdce0 con 0x7f01400a53e0 2024-05-27T08:00:04.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.746+0000 7f0145e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01400a53e0 0x7f01400b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.747+0000 7f0145e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01400a53e0 0x7f01400b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:49914/0 (socket says 172.21.15.90:49914) 2024-05-27T08:00:04.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.747+0000 7f0145e00700 1 -- 172.21.15.90:0/3179452089 learned_addr learned my addr 172.21.15.90:0/3179452089 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:04.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.747+0000 7f0146800700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01400b8340 0x7f01400b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.747+0000 7f0145400700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01400a5d30 0x7f01400b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.747+0000 7f0145e00700 1 -- 172.21.15.90:0/3179452089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01400a5d30 msgr2=0x7f01400b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.747+0000 7f0145e00700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01400a5d30 0x7f01400b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.747+0000 7f0145e00700 1 -- 172.21.15.90:0/3179452089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01400b8340 msgr2=0x7f01400b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.747+0000 7f0145e00700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01400b8340 0x7f01400b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.747+0000 7f0145e00700 1 -- 172.21.15.90:0/3179452089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01400b58d0 con 0x7f01400a53e0 2024-05-27T08:00:04.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.747+0000 7f0145400700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01400a5d30 0x7f01400b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:04.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.748+0000 7f0145e00700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01400a53e0 0x7f01400b7da0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f013800c920 tx=0x7f013800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.748+0000 7f013ea00700 1 -- 172.21.15.90:0/3179452089 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138004a30 con 0x7f01400a53e0 2024-05-27T08:00:04.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.748+0000 7f0147200700 1 -- 172.21.15.90:0/3179452089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01400b5bc0 con 0x7f01400a53e0 2024-05-27T08:00:04.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.748+0000 7f0147200700 1 -- 172.21.15.90:0/3179452089 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01400b6150 con 0x7f01400a53e0 2024-05-27T08:00:04.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.749+0000 7f013ea00700 1 -- 172.21.15.90:0/3179452089 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01380054b0 con 0x7f01400a53e0 2024-05-27T08:00:04.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.749+0000 7f013ea00700 1 -- 172.21.15.90:0/3179452089 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138007b70 con 0x7f01400a53e0 2024-05-27T08:00:04.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.754+0000 7f013ea00700 1 -- 172.21.15.90:0/3179452089 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f013800f450 con 0x7f01400a53e0 2024-05-27T08:00:04.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.756+0000 7f013ea00700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f012c0b2af0 0x7f012c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.757+0000 7f013ea00700 1 -- 172.21.15.90:0/3179452089 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9584+0+0 (secure 0 0 0) 0x7f01380e3400 con 0x7f01400a53e0 2024-05-27T08:00:04.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.757+0000 7f0145400700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f012c0b2af0 0x7f012c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.770+0000 7f0145400700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f012c0b2af0 0x7f012c0b5000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f01400ba7f0 tx=0x7f013400a380 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.772+0000 7f015b951b80 1 Processor -- start 2024-05-27T08:00:04.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.773+0000 7f015b951b80 1 -- start start 2024-05-27T08:00:04.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.773+0000 7f015b951b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 0x5644e0f307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.773+0000 7f015b951b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f30d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.773+0000 7f015b951b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0f3a340 0x5644e0f3c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.773+0000 7f015b951b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5644e0e5dec0 con 0x5644e0f3a340 2024-05-27T08:00:04.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.773+0000 7f015b951b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5644e0e4da60 con 0x5644e0e9ca70 2024-05-27T08:00:04.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.773+0000 7f015b951b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5644e0e51aa0 con 0x5644e0db0ec0 2024-05-27T08:00:04.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.774+0000 7f0146800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0f3a340 0x5644e0f3c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.774+0000 7f0145400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f30d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.774+0000 7f0145400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f30d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:49924/0 (socket says 172.21.15.90:49924) 2024-05-27T08:00:04.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.775+0000 7f0145400700 1 -- 172.21.15.90:0/818461635 learned_addr learned my addr 172.21.15.90:0/818461635 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.775+0000 7f0145400700 1 -- 172.21.15.90:0/818461635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 msgr2=0x5644e0f307d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.775+0000 7f0145400700 1 --2- 172.21.15.90:0/818461635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 0x5644e0f307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.775+0000 7f0145400700 1 -- 172.21.15.90:0/818461635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0f3a340 msgr2=0x5644e0f3c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.775+0000 7f0145400700 1 --2- 172.21.15.90:0/818461635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0f3a340 0x5644e0f3c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.775+0000 7f0145400700 1 -- 172.21.15.90:0/818461635 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e0e164a0 con 0x5644e0db0ec0 2024-05-27T08:00:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.776+0000 7f0145400700 1 --2- 172.21.15.90:0/818461635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f30d10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0134010560 tx=0x7f0134012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6200e60049f628c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.776+0000 7f0133400700 1 -- 172.21.15.90:0/818461635 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134016070 con 0x5644e0db0ec0 2024-05-27T08:00:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.777+0000 7f0133400700 1 -- 172.21.15.90:0/818461635 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0134010cd0 con 0x5644e0db0ec0 2024-05-27T08:00:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.777+0000 7f0133400700 1 -- 172.21.15.90:0/818461635 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013401b680 con 0x5644e0db0ec0 2024-05-27T08:00:04.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.779+0000 7f015b951b80 1 -- 172.21.15.90:0/818461635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 msgr2=0x5644e0f30d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.779+0000 7f015b951b80 1 --2- 172.21.15.90:0/818461635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f30d10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0134010560 tx=0x7f0134012000 comp rx=0 tx=0).stop 2024-05-27T08:00:04.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.781+0000 7f015b951b80 1 -- 172.21.15.90:0/818461635 shutdown_connections 2024-05-27T08:00:04.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.781+0000 7f015b951b80 1 --2- 172.21.15.90:0/818461635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0f3a340 0x5644e0f3c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.781+0000 7f015b951b80 1 --2- 172.21.15.90:0/818461635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 0x5644e0f307d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.781+0000 7f015b951b80 1 --2- 172.21.15.90:0/818461635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f30d10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.781+0000 7f015b951b80 1 -- 172.21.15.90:0/818461635 >> 172.21.15.90:0/818461635 conn(0x5644e0e4a410 msgr2=0x5644e0f2b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.781+0000 7f015b951b80 1 -- 172.21.15.90:0/818461635 shutdown_connections 2024-05-27T08:00:04.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.781+0000 7f015b951b80 1 -- 172.21.15.90:0/818461635 wait complete. 2024-05-27T08:00:04.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.782+0000 7f015b951b80 1 -- start start 2024-05-27T08:00:04.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.782+0000 7f015b951b80 1 Processor -- start 2024-05-27T08:00:04.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.783+0000 7f015b951b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0db0ec0 0x5644e0f37440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.783+0000 7f015b951b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0e9ca70 0x5644e0f37980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.783+0000 7f015b951b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0f3a340 0x5644e0f37ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.783+0000 7f015b951b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5644e0d54020 con 0x5644e0e9ca70 2024-05-27T08:00:04.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.783+0000 7f015b951b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5644e0dee7e0 con 0x5644e0db0ec0 2024-05-27T08:00:04.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.783+0000 7f015b951b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5644e0f3ce80 con 0x5644e0f3a340 2024-05-27T08:00:04.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.783+0000 7f015b951b80 10 client.0 _ll_register_callbacks cb 0x5644e0df7cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:04.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.785+0000 7f0145e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0db0ec0 0x5644e0f37440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.785+0000 7f0145e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0db0ec0 0x5644e0f37440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48042/0 (socket says 172.21.15.90:48042) 2024-05-27T08:00:04.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.785+0000 7f0145e00700 1 -- 172.21.15.90:0/4095762513 learned_addr learned my addr 172.21.15.90:0/4095762513 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:04.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.785+0000 7f0145400700 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0e9ca70 0x5644e0f37980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.786+0000 7f0146800700 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0f3a340 0x5644e0f37ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.786+0000 7f0145e00700 1 -- 172.21.15.90:0/4095762513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0f3a340 msgr2=0x5644e0f37ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.786+0000 7f0145e00700 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0f3a340 0x5644e0f37ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.786+0000 7f0145e00700 1 -- 172.21.15.90:0/4095762513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0e9ca70 msgr2=0x5644e0f37980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.786+0000 7f0145e00700 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0e9ca70 0x5644e0f37980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.786+0000 7f0145e00700 1 -- 172.21.15.90:0/4095762513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e0d9d950 con 0x5644e0db0ec0 2024-05-27T08:00:04.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.787+0000 7f0145e00700 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0db0ec0 0x5644e0f37440 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01380a9f80 tx=0x7f0138007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a62df4b38d991a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.787+0000 7f0127e00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:00:04.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.789+0000 7f0127e00700 1 -- 172.21.15.90:0/4095762513 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013801d070 con 0x5644e0db0ec0 2024-05-27T08:00:04.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.789+0000 7f0127e00700 1 -- 172.21.15.90:0/4095762513 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9584+0+0 (secure 0 0 0) 0x7f01380e0970 con 0x5644e0db0ec0 2024-05-27T08:00:04.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.789+0000 7f015b951b80 1 -- 172.21.15.90:0/4095762513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0134010cd0 con 0x5644e0db0ec0 2024-05-27T08:00:04.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.790+0000 7f0127e00700 1 -- 172.21.15.90:0/4095762513 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f01380e0b70 con 0x5644e0db0ec0 2024-05-27T08:00:04.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.790+0000 7f015b951b80 10 client.37433 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:04.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.790+0000 7f015b951b80 1 -- 172.21.15.90:0/4095762513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5644e0e1e0e0 con 0x5644e0db0ec0 2024-05-27T08:00:04.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.791+0000 7f0127e00700 1 -- 172.21.15.90:0/4095762513 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0134010cd0 con 0x5644e0db0ec0 2024-05-27T08:00:04.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.791+0000 7f0127e00700 1 -- 172.21.15.90:0/4095762513 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f01380b2c50 con 0x5644e0db0ec0 2024-05-27T08:00:04.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.791+0000 7f015b951b80 10 client.37433 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:04.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.791+0000 7f015b951b80 10 client.37433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:04.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.791+0000 7f015b951b80 20 client.37433 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:04.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.791+0000 7f015b951b80 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5644e0f49fe0 0x5644e0f4c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.791+0000 7f015b951b80 4 client.37433 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:04.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.791+0000 7f015b951b80 1 -- 172.21.15.90:0/4095762513 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644e0e5fe10 con 0x5644e0f49fe0 2024-05-27T08:00:04.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.792+0000 7f0145400700 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5644e0f49fe0 0x5644e0f4c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.793+0000 7f0127e00700 10 client.37433 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:04.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.793+0000 7f0145400700 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5644e0f49fe0 0x5644e0f4c400 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d323743aacbcf179 server_cookie=1e5534105e45df09 in_seq=0 out_seq=0 2024-05-27T08:00:04.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.795+0000 7f0127e00700 1 -- 172.21.15.90:0/4095762513 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5644e0e5fe10 con 0x5644e0f49fe0 2024-05-27T08:00:04.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.795+0000 7f0127e00700 10 client.37433 handle_command_reply: tid=0 2024-05-27T08:00:04.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.795+0000 7f015b951b80 1 client.37433 shutdown 2024-05-27T08:00:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.796+0000 7f015b951b80 1 -- 172.21.15.90:0/4095762513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0db0ec0 msgr2=0x5644e0f37440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.796+0000 7f015b951b80 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0db0ec0 0x5644e0f37440 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01380a9f80 tx=0x7f0138007f90 comp rx=0 tx=0).stop 2024-05-27T08:00:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.796+0000 7f015b951b80 1 -- 172.21.15.90:0/4095762513 shutdown_connections 2024-05-27T08:00:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.796+0000 7f015b951b80 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5644e0f49fe0 0x5644e0f4c400 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.796+0000 7f015b951b80 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0f3a340 0x5644e0f37ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.796+0000 7f015b951b80 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0db0ec0 0x5644e0f37440 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.796+0000 7f015b951b80 1 --2- 172.21.15.90:0/4095762513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0e9ca70 0x5644e0f37980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.796+0000 7f015b951b80 1 -- 172.21.15.90:0/4095762513 >> 172.21.15.90:0/4095762513 conn(0x5644e0e4a410 msgr2=0x5644e0f2b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.796+0000 7f0127e00700 0 client.37433 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.797+0000 7f015b951b80 1 -- 172.21.15.90:0/4095762513 shutdown_connections 2024-05-27T08:00:04.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.797+0000 7f015b951b80 1 -- 172.21.15.90:0/4095762513 wait complete. 2024-05-27T08:00:04.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.797+0000 7f015b951b80 20 client.37433 trim_cache size 0 max 16384 2024-05-27T08:00:04.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.836+0000 7f015b951b80 1 Processor -- start 2024-05-27T08:00:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.837+0000 7f015b951b80 1 -- start start 2024-05-27T08:00:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.837+0000 7f015b951b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f32c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.837+0000 7f015b951b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 0x5644e0f2b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.837+0000 7f015b951b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0f3a340 0x5644e0f51bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.837+0000 7f015b951b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f01280109e0 con 0x5644e0f3a340 2024-05-27T08:00:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.837+0000 7f015b951b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f012800da30 con 0x5644e0e9ca70 2024-05-27T08:00:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.837+0000 7f015b951b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f012800fc40 con 0x5644e0db0ec0 2024-05-27T08:00:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.841+0000 7f0145400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 0x5644e0f2b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.841+0000 7f0146800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0f3a340 0x5644e0f51bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.841+0000 7f0145400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 0x5644e0f2b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48048/0 (socket says 172.21.15.90:48048) 2024-05-27T08:00:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.841+0000 7f0145400700 1 -- 172.21.15.90:0/434736819 learned_addr learned my addr 172.21.15.90:0/434736819 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.841+0000 7f0145e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f32c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.842+0000 7f0145400700 1 -- 172.21.15.90:0/434736819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 msgr2=0x5644e0f32c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.842+0000 7f0145400700 1 --2- 172.21.15.90:0/434736819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f32c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.842+0000 7f0145400700 1 -- 172.21.15.90:0/434736819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0f3a340 msgr2=0x5644e0f51bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.842+0000 7f0145400700 1 --2- 172.21.15.90:0/434736819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0f3a340 0x5644e0f51bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.842+0000 7f0145400700 1 -- 172.21.15.90:0/434736819 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644e0f38230 con 0x5644e0e9ca70 2024-05-27T08:00:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.843+0000 7f0145400700 1 --2- 172.21.15.90:0/434736819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 0x5644e0f2b7c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f013400fde0 tx=0x7f0134010b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be68f4903f7f73c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.843+0000 7f0132a00700 1 -- 172.21.15.90:0/434736819 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0134016070 con 0x5644e0e9ca70 2024-05-27T08:00:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.843+0000 7f0132a00700 1 -- 172.21.15.90:0/434736819 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0134026e00 con 0x5644e0e9ca70 2024-05-27T08:00:04.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.843+0000 7f0132a00700 1 -- 172.21.15.90:0/434736819 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01340313f0 con 0x5644e0e9ca70 2024-05-27T08:00:04.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.845+0000 7f015b951b80 1 -- 172.21.15.90:0/434736819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 msgr2=0x5644e0f2b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.845+0000 7f015b951b80 1 --2- 172.21.15.90:0/434736819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 0x5644e0f2b7c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f013400fde0 tx=0x7f0134010b10 comp rx=0 tx=0).stop 2024-05-27T08:00:04.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.846+0000 7f015b951b80 1 -- 172.21.15.90:0/434736819 shutdown_connections 2024-05-27T08:00:04.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.847+0000 7f015b951b80 1 --2- 172.21.15.90:0/434736819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0f3a340 0x5644e0f51bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.847+0000 7f015b951b80 1 --2- 172.21.15.90:0/434736819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f32c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.847+0000 7f015b951b80 1 --2- 172.21.15.90:0/434736819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0e9ca70 0x5644e0f2b7c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.847+0000 7f015b951b80 1 -- 172.21.15.90:0/434736819 >> 172.21.15.90:0/434736819 conn(0x5644e0e4a410 msgr2=0x5644e0f2f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.847+0000 7f015b951b80 1 -- 172.21.15.90:0/434736819 shutdown_connections 2024-05-27T08:00:04.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.847+0000 7f015b951b80 1 -- 172.21.15.90:0/434736819 wait complete. 2024-05-27T08:00:04.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.848+0000 7f015b951b80 1 -- start start 2024-05-27T08:00:04.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.848+0000 7f015b951b80 1 Processor -- start 2024-05-27T08:00:04.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.849+0000 7f015b951b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f42370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.849+0000 7f015b951b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0e9ca70 0x5644e0f428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.849+0000 7f015b951b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0f3a340 0x5644e0f3f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.850+0000 7f015b951b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f012800f570 con 0x5644e0e9ca70 2024-05-27T08:00:04.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.850+0000 7f015b951b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f012800e100 con 0x5644e0f3a340 2024-05-27T08:00:04.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.850+0000 7f015b951b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f012800e7d0 con 0x5644e0db0ec0 2024-05-27T08:00:04.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.850+0000 7f015b951b80 10 client.0 _ll_register_callbacks cb 0x5644e0df7cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.850+0000 7f0145e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f42370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.850+0000 7f0146800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0f3a340 0x5644e0f3f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.850+0000 7f0146800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0f3a340 0x5644e0f3f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48064/0 (socket says 172.21.15.90:48064) 2024-05-27T08:00:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.850+0000 7f0146800700 1 -- 172.21.15.90:0/3461881795 learned_addr learned my addr 172.21.15.90:0/3461881795 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.851+0000 7f0145400700 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0e9ca70 0x5644e0f428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.852+0000 7f0145e00700 1 -- 172.21.15.90:0/3461881795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0f3a340 msgr2=0x5644e0f3f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.852+0000 7f0145e00700 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0f3a340 0x5644e0f3f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.852+0000 7f0145e00700 1 -- 172.21.15.90:0/3461881795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0e9ca70 msgr2=0x5644e0f428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:04.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.852+0000 7f0145e00700 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0e9ca70 0x5644e0f428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.852+0000 7f0145e00700 1 -- 172.21.15.90:0/3461881795 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644e0f3f7f0 con 0x5644e0db0ec0 2024-05-27T08:00:04.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.852+0000 7f0145e00700 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f42370 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01380a8930 tx=0x7f0138012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5901eb8236133d1f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:04.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.852+0000 7f0127e00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:00:04.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.852+0000 7f0127e00700 1 -- 172.21.15.90:0/3461881795 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380a6140 con 0x5644e0db0ec0 2024-05-27T08:00:04.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.854+0000 7f0127e00700 1 -- 172.21.15.90:0/3461881795 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9584+0+0 (secure 0 0 0) 0x7f01380e0970 con 0x5644e0db0ec0 2024-05-27T08:00:04.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.855+0000 7f015b951b80 1 -- 172.21.15.90:0/3461881795 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0134026e00 con 0x5644e0db0ec0 2024-05-27T08:00:04.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.855+0000 7f0127e00700 1 -- 172.21.15.90:0/3461881795 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f01380aa280 con 0x5644e0db0ec0 2024-05-27T08:00:04.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.856+0000 7f015b951b80 10 client.27535 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:04.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.856+0000 7f015b951b80 1 -- 172.21.15.90:0/3461881795 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5644e0fb53e0 con 0x5644e0db0ec0 2024-05-27T08:00:04.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.857+0000 7f0127e00700 1 -- 172.21.15.90:0/3461881795 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0134026e00 con 0x5644e0db0ec0 2024-05-27T08:00:04.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.857+0000 7f0127e00700 1 -- 172.21.15.90:0/3461881795 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f0138011a40 con 0x5644e0db0ec0 2024-05-27T08:00:04.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.857+0000 7f015b951b80 10 client.27535 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:04.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.857+0000 7f015b951b80 10 client.27535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:04.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.857+0000 7f015b951b80 20 client.27535 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:04.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.857+0000 7f015b951b80 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5644e0fb5a30 0x5644e0fb7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:04.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.857+0000 7f015b951b80 4 client.27535 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:00:04.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.857+0000 7f015b951b80 1 -- 172.21.15.90:0/3461881795 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5644e0e595f0 con 0x5644e0fb5a30 2024-05-27T08:00:04.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.858+0000 7f0145400700 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5644e0fb5a30 0x5644e0fb7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:04.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.858+0000 7f0145400700 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5644e0fb5a30 0x5644e0fb7e50 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81528a7a1ebac576 server_cookie=dae6a35ecc48477d in_seq=0 out_seq=0 2024-05-27T08:00:04.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.858+0000 7f0127e00700 10 client.27535 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:04.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.860+0000 7f0127e00700 1 -- 172.21.15.90:0/3461881795 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5644e0e595f0 con 0x5644e0fb5a30 2024-05-27T08:00:04.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.860+0000 7f0127e00700 10 client.27535 handle_command_reply: tid=0 2024-05-27T08:00:04.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.860+0000 7f015b951b80 1 client.27535 shutdown 2024-05-27T08:00:04.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.860+0000 7f015b951b80 1 -- 172.21.15.90:0/3461881795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 msgr2=0x5644e0f42370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.860+0000 7f015b951b80 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f42370 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01380a8930 tx=0x7f0138012000 comp rx=0 tx=0).stop 2024-05-27T08:00:04.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.860+0000 7f015b951b80 1 -- 172.21.15.90:0/3461881795 shutdown_connections 2024-05-27T08:00:04.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.860+0000 7f015b951b80 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5644e0fb5a30 0x5644e0fb7e50 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.861+0000 7f015b951b80 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644e0f3a340 0x5644e0f3f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.861+0000 7f0127e00700 0 client.27535 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:04.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.861+0000 7f015b951b80 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5644e0db0ec0 0x5644e0f42370 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.861+0000 7f015b951b80 1 --2- 172.21.15.90:0/3461881795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5644e0e9ca70 0x5644e0f428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.861+0000 7f015b951b80 1 -- 172.21.15.90:0/3461881795 >> 172.21.15.90:0/3461881795 conn(0x5644e0e4a410 msgr2=0x5644e0f320f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.861+0000 7f015b951b80 1 -- 172.21.15.90:0/3461881795 shutdown_connections 2024-05-27T08:00:04.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.861+0000 7f015b951b80 1 -- 172.21.15.90:0/3461881795 wait complete. 2024-05-27T08:00:04.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.861+0000 7f015b951b80 20 client.27535 trim_cache size 0 max 16384 2024-05-27T08:00:04.865 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:00:04.865 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T08:00:04.865 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "b4155383-a76f-49ad-9e57-0c8b07473963", 2024-05-27T08:00:04.865 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T08:00:04.865 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:00:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.863+0000 7f0147200700 1 -- 172.21.15.90:0/3179452089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f012c0b2af0 msgr2=0x7f012c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.863+0000 7f0147200700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f012c0b2af0 0x7f012c0b5000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f01400ba7f0 tx=0x7f013400a380 comp rx=0 tx=0).stop 2024-05-27T08:00:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.863+0000 7f0147200700 1 -- 172.21.15.90:0/3179452089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01400a53e0 msgr2=0x7f01400b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.863+0000 7f0147200700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01400a53e0 0x7f01400b7da0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f013800c920 tx=0x7f013800cc30 comp rx=0 tx=0).stop 2024-05-27T08:00:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.864+0000 7f0147200700 1 -- 172.21.15.90:0/3179452089 shutdown_connections 2024-05-27T08:00:04.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.864+0000 7f0147200700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01400b8340 0x7f01400b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.864+0000 7f0147200700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01400a53e0 0x7f01400b7da0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.864+0000 7f0147200700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f012c0b2af0 0x7f012c0b5000 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.864+0000 7f0147200700 1 --2- 172.21.15.90:0/3179452089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01400a5d30 0x7f01400b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:04.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.864+0000 7f0147200700 1 -- 172.21.15.90:0/3179452089 >> 172.21.15.90:0/3179452089 conn(0x7f014001af40 msgr2=0x7f01400b8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:04.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.864+0000 7f0147200700 1 -- 172.21.15.90:0/3179452089 shutdown_connections 2024-05-27T08:00:04.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:04.864+0000 7f0147200700 1 -- 172.21.15.90:0/3179452089 wait complete. 2024-05-27T08:00:05.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:04 smithi136 ceph-mon[35324]: osdmap e115: 12 total, 12 up, 12 in 2024-05-27T08:00:05.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:04 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T08:00:05.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:04 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:05.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:04 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T08:00:05.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:04 smithi136 ceph-mon[35324]: pgmap v438: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 544 op/s 2024-05-27T08:00:05.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:04 smithi129 ceph-mon[35254]: osdmap e115: 12 total, 12 up, 12 in 2024-05-27T08:00:05.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:04 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T08:00:05.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:04 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:05.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:04 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T08:00:05.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:04 smithi129 ceph-mon[35254]: pgmap v438: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 544 op/s 2024-05-27T08:00:05.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:04 smithi090 ceph-mon[31429]: osdmap e115: 12 total, 12 up, 12 in 2024-05-27T08:00:05.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:04 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T08:00:05.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:04 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:05.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:04 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T08:00:05.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:04 smithi090 ceph-mon[31429]: pgmap v438: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 544 op/s 2024-05-27T08:00:06.176 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:00:06.712 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:00:07.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:06 smithi090 ceph-mon[31429]: pgmap v439: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.0 MiB/s wr, 561 op/s 2024-05-27T08:00:07.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:06 smithi136 ceph-mon[35324]: pgmap v439: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.0 MiB/s wr, 561 op/s 2024-05-27T08:00:07.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:06 smithi129 ceph-mon[35254]: pgmap v439: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.0 MiB/s wr, 561 op/s 2024-05-27T08:00:08.831 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 590.0 s, 54.4 tps, lat 531.866 ms stddev 450.551 2024-05-27T08:00:09.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:08 smithi136 ceph-mon[35324]: pgmap v440: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-05-27T08:00:09.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:08 smithi129 ceph-mon[35254]: pgmap v440: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-05-27T08:00:09.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:08 smithi090 ceph-mon[31429]: pgmap v440: 129 pgs: 4 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 116 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-05-27T08:00:11.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:10 smithi136 ceph-mon[35324]: pgmap v441: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 4.2 MiB/s wr, 559 op/s 2024-05-27T08:00:11.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:10 smithi129 ceph-mon[35254]: pgmap v441: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 4.2 MiB/s wr, 559 op/s 2024-05-27T08:00:11.395 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:10 smithi090 ceph-mon[31429]: pgmap v441: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 4.2 MiB/s wr, 559 op/s 2024-05-27T08:00:12.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.443+0000 7ff799200700 1 -- 172.21.15.90:0/1821135353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff794108ef0 msgr2=0x7ff794109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:12.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.443+0000 7ff799200700 1 --2- 172.21.15.90:0/1821135353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff794108ef0 0x7ff794109300 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff7800099c0 tx=0x7ff780009cd0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.443+0000 7ff799200700 1 -- 172.21.15.90:0/1821135353 shutdown_connections 2024-05-27T08:00:12.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.443+0000 7ff799200700 1 --2- 172.21.15.90:0/1821135353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff794109840 0x7ff7941111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.443+0000 7ff799200700 1 --2- 172.21.15.90:0/1821135353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7941045c0 0x7ff7941049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.443+0000 7ff799200700 1 --2- 172.21.15.90:0/1821135353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff794108ef0 0x7ff794109300 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.443+0000 7ff799200700 1 -- 172.21.15.90:0/1821135353 >> 172.21.15.90:0/1821135353 conn(0x7ff7940ffe90 msgr2=0x7ff7941022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:12.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.443+0000 7ff799200700 1 -- 172.21.15.90:0/1821135353 shutdown_connections 2024-05-27T08:00:12.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.443+0000 7ff799200700 1 -- 172.21.15.90:0/1821135353 wait complete. 2024-05-27T08:00:12.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.444+0000 7ff799200700 1 Processor -- start 2024-05-27T08:00:12.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.444+0000 7ff799200700 1 -- start start 2024-05-27T08:00:12.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.445+0000 7ff799200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7941045c0 0x7ff794076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.445+0000 7ff799200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff794108ef0 0x7ff794077100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.445+0000 7ff799200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff794109840 0x7ff794077640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.445+0000 7ff799200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff794114370 con 0x7ff794109840 2024-05-27T08:00:12.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.445+0000 7ff799200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941141f0 con 0x7ff794108ef0 2024-05-27T08:00:12.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.445+0000 7ff799200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941144f0 con 0x7ff7941045c0 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.445+0000 7ff792800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff794109840 0x7ff794077640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.445+0000 7ff792800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff794109840 0x7ff794077640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44416/0 (socket says 172.21.15.90:44416) 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.445+0000 7ff792800700 1 -- 172.21.15.90:0/2660790095 learned_addr learned my addr 172.21.15.90:0/2660790095 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.446+0000 7ff791e00700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7941045c0 0x7ff794076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.446+0000 7ff791400700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff794108ef0 0x7ff794077100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.446+0000 7ff792800700 1 -- 172.21.15.90:0/2660790095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7941045c0 msgr2=0x7ff794076bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.446+0000 7ff792800700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7941045c0 0x7ff794076bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.446+0000 7ff792800700 1 -- 172.21.15.90:0/2660790095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff794108ef0 msgr2=0x7ff794077100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.446+0000 7ff792800700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff794108ef0 0x7ff794077100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.446+0000 7ff792800700 1 -- 172.21.15.90:0/2660790095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7941a9110 con 0x7ff794109840 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.446+0000 7ff792800700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff794109840 0x7ff794077640 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff78c007ee0 tx=0x7ff78c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:12.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.447+0000 7ff78aa00700 1 -- 172.21.15.90:0/2660790095 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c00dbd0 con 0x7ff794109840 2024-05-27T08:00:12.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.447+0000 7ff78aa00700 1 -- 172.21.15.90:0/2660790095 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff78c00dd70 con 0x7ff794109840 2024-05-27T08:00:12.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.447+0000 7ff78aa00700 1 -- 172.21.15.90:0/2660790095 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c0058d0 con 0x7ff794109840 2024-05-27T08:00:12.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.447+0000 7ff799200700 1 -- 172.21.15.90:0/2660790095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7941a9400 con 0x7ff794109840 2024-05-27T08:00:12.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.447+0000 7ff799200700 1 -- 172.21.15.90:0/2660790095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7941a9910 con 0x7ff794109840 2024-05-27T08:00:12.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.451+0000 7ff78aa00700 1 -- 172.21.15.90:0/2660790095 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff78c01a020 con 0x7ff794109840 2024-05-27T08:00:12.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.452+0000 7ff78aa00700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff7780b29f0 0x7ff7780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.452+0000 7ff78aa00700 1 -- 172.21.15.90:0/2660790095 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7ff78c0e2140 con 0x7ff794109840 2024-05-27T08:00:12.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.452+0000 7ff791e00700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff7780b29f0 0x7ff7780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.453+0000 7ff791e00700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff7780b29f0 0x7ff7780b4f00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff784005950 tx=0x7ff78400b410 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:12.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.465+0000 7ff7a7cfcb80 1 Processor -- start 2024-05-27T08:00:12.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.465+0000 7ff7a7cfcb80 1 -- start start 2024-05-27T08:00:12.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.465+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb384c9f0 0x55ddb38df330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb3760e40 0x55ddb38df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb38ec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ddb380de40 con 0x55ddb384c9f0 2024-05-27T08:00:12.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ddb37fd9e0 con 0x55ddb3760e40 2024-05-27T08:00:12.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ddb3801a20 con 0x55ddb38dfe70 2024-05-27T08:00:12.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff791e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb384c9f0 0x55ddb38df330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff791e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb384c9f0 0x55ddb38df330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44420/0 (socket says 172.21.15.90:44420) 2024-05-27T08:00:12.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff791e00700 1 -- 172.21.15.90:0/152527518 learned_addr learned my addr 172.21.15.90:0/152527518 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:12.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff791400700 1 --2- 172.21.15.90:0/152527518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb3760e40 0x55ddb38df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff792800700 1 --2- 172.21.15.90:0/152527518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb38ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff791e00700 1 -- 172.21.15.90:0/152527518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 msgr2=0x55ddb38ec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:12.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff791e00700 1 --2- 172.21.15.90:0/152527518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb38ec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff791e00700 1 -- 172.21.15.90:0/152527518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb3760e40 msgr2=0x55ddb38df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:12.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff791e00700 1 --2- 172.21.15.90:0/152527518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb3760e40 0x55ddb38df870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.466+0000 7ff791e00700 1 -- 172.21.15.90:0/152527518 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb37c6420 con 0x55ddb384c9f0 2024-05-27T08:00:12.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.467+0000 7ff791e00700 1 --2- 172.21.15.90:0/152527518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb384c9f0 0x55ddb38df330 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff784010fa0 tx=0x7ff784013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aef67842b2ed236 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:12.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.467+0000 7ff77f400700 1 -- 172.21.15.90:0/152527518 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78400be60 con 0x55ddb384c9f0 2024-05-27T08:00:12.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.467+0000 7ff77f400700 1 -- 172.21.15.90:0/152527518 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff78401eab0 con 0x55ddb384c9f0 2024-05-27T08:00:12.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.467+0000 7ff77f400700 1 -- 172.21.15.90:0/152527518 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78401caa0 con 0x55ddb384c9f0 2024-05-27T08:00:12.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.468+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/152527518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb384c9f0 msgr2=0x55ddb38df330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:12.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.468+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/152527518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb384c9f0 0x55ddb38df330 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff784010fa0 tx=0x7ff784013000 comp rx=0 tx=0).stop 2024-05-27T08:00:12.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.468+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/152527518 shutdown_connections 2024-05-27T08:00:12.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.468+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/152527518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb38ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.468+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/152527518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb384c9f0 0x55ddb38df330 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.468+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/152527518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb3760e40 0x55ddb38df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.468+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/152527518 >> 172.21.15.90:0/152527518 conn(0x55ddb37fa390 msgr2=0x55ddb38db4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:12.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.469+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/152527518 shutdown_connections 2024-05-27T08:00:12.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.469+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/152527518 wait complete. 2024-05-27T08:00:12.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.469+0000 7ff7a7cfcb80 1 -- start start 2024-05-27T08:00:12.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.469+0000 7ff7a7cfcb80 1 Processor -- start 2024-05-27T08:00:12.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.470+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.470+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.470+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb38ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.470+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ddb3703fa0 con 0x55ddb3760e40 2024-05-27T08:00:12.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.470+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ddb379e760 con 0x55ddb384c9f0 2024-05-27T08:00:12.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.470+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ddb38ecb80 con 0x55ddb38dfe70 2024-05-27T08:00:12.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.470+0000 7ff7a7cfcb80 10 client.0 _ll_register_callbacks cb 0x55ddb37a7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:12.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.470+0000 7ff791e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.470+0000 7ff791e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44428/0 (socket says 172.21.15.90:44428) 2024-05-27T08:00:12.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.470+0000 7ff791e00700 1 -- 172.21.15.90:0/1200721557 learned_addr learned my addr 172.21.15.90:0/1200721557 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:12.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff791400700 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff792800700 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb38ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff791e00700 1 -- 172.21.15.90:0/1200721557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 msgr2=0x55ddb38ea0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:12.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff791e00700 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb38ea0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff791e00700 1 -- 172.21.15.90:0/1200721557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 msgr2=0x55ddb38e9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:12.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff791e00700 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38e9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff791e00700 1 -- 172.21.15.90:0/1200721557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb374d8d0 con 0x55ddb3760e40 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff791400700 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38e9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff792800700 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb38ea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff791e00700 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e9620 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff784005d90 tx=0x7ff78401e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb1433620a385c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.471+0000 7ff773e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.472+0000 7ff773e00700 1 -- 172.21.15.90:0/1200721557 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784036060 con 0x55ddb3760e40 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.472+0000 7ff773e00700 1 -- 172.21.15.90:0/1200721557 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7ff784005040 con 0x55ddb3760e40 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.472+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/1200721557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78401eab0 con 0x55ddb3760e40 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.472+0000 7ff773e00700 1 -- 172.21.15.90:0/1200721557 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff78403d050 con 0x55ddb3760e40 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.472+0000 7ff7a7cfcb80 10 client.17823 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.472+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/1200721557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ddb37ce060 con 0x55ddb3760e40 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.472+0000 7ff773e00700 1 -- 172.21.15.90:0/1200721557 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff78401eab0 con 0x55ddb3760e40 2024-05-27T08:00:12.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.473+0000 7ff773e00700 1 -- 172.21.15.90:0/1200721557 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7ff784004ce0 con 0x55ddb3760e40 2024-05-27T08:00:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.473+0000 7ff7a7cfcb80 10 client.17823 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.473+0000 7ff7a7cfcb80 10 client.17823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.473+0000 7ff7a7cfcb80 20 client.17823 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.473+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddb38f9b80 0x55ddb38fc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.473+0000 7ff7a7cfcb80 4 client.17823 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.473+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/1200721557 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddb380fd90 con 0x55ddb38f9b80 2024-05-27T08:00:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.473+0000 7ff791400700 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddb38f9b80 0x55ddb38fc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.475+0000 7ff791400700 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddb38f9b80 0x55ddb38fc030 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70406f4b6bccedd server_cookie=352ac7a54288202d in_seq=0 out_seq=0 2024-05-27T08:00:12.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.475+0000 7ff773e00700 10 client.17823 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:12.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff773e00700 1 -- 172.21.15.90:0/1200721557 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ddb380fd90 con 0x55ddb38f9b80 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff773e00700 10 client.17823 handle_command_reply: tid=0 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 client.17823 shutdown 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/1200721557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 msgr2=0x55ddb38e9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e9620 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff784005d90 tx=0x7ff78401e420 comp rx=0 tx=0).stop 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/1200721557 shutdown_connections 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddb38f9b80 0x55ddb38fc030 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb38ea0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e9620 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/1200721557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38e9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/1200721557 >> 172.21.15.90:0/1200721557 conn(0x55ddb37fa390 msgr2=0x55ddb38df310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:12.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff773e00700 0 client.17823 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/1200721557 shutdown_connections 2024-05-27T08:00:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/1200721557 wait complete. 2024-05-27T08:00:12.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.477+0000 7ff7a7cfcb80 20 client.17823 trim_cache size 0 max 16384 2024-05-27T08:00:12.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.500+0000 7ff7a7cfcb80 1 Processor -- start 2024-05-27T08:00:12.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.500+0000 7ff7a7cfcb80 1 -- start start 2024-05-27T08:00:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.500+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.500+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.500+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb39017f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.500+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941b4140 con 0x55ddb3760e40 2024-05-27T08:00:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.500+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941b1190 con 0x55ddb384c9f0 2024-05-27T08:00:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.500+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941b33a0 con 0x55ddb38dfe70 2024-05-27T08:00:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44436/0 (socket says 172.21.15.90:44436) 2024-05-27T08:00:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791e00700 1 -- 172.21.15.90:0/832050561 learned_addr learned my addr 172.21.15.90:0/832050561 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:12.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff792800700 1 --2- 172.21.15.90:0/832050561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb39017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791400700 1 --2- 172.21.15.90:0/832050561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791e00700 1 -- 172.21.15.90:0/832050561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 msgr2=0x55ddb39017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791e00700 1 --2- 172.21.15.90:0/832050561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb39017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791e00700 1 -- 172.21.15.90:0/832050561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 msgr2=0x55ddb38f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791e00700 1 --2- 172.21.15.90:0/832050561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791e00700 1 -- 172.21.15.90:0/832050561 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb38e7b90 con 0x55ddb3760e40 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791400700 1 --2- 172.21.15.90:0/832050561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.501+0000 7ff791e00700 1 --2- 172.21.15.90:0/832050561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e35f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff78401c5e0 tx=0x7ff78401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8d6cc20b8c26bf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.502+0000 7ff77ea00700 1 -- 172.21.15.90:0/832050561 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784036070 con 0x55ddb3760e40 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.502+0000 7ff77ea00700 1 -- 172.21.15.90:0/832050561 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff78401c610 con 0x55ddb3760e40 2024-05-27T08:00:12.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.502+0000 7ff77ea00700 1 -- 172.21.15.90:0/832050561 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784005030 con 0x55ddb3760e40 2024-05-27T08:00:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.502+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/832050561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 msgr2=0x55ddb38e35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.502+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/832050561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e35f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff78401c5e0 tx=0x7ff78401c9f0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.503+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/832050561 shutdown_connections 2024-05-27T08:00:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.503+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/832050561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb39017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.503+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/832050561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38e35f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.503+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/832050561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.503+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/832050561 >> 172.21.15.90:0/832050561 conn(0x55ddb37fa390 msgr2=0x55ddb38e2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.503+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/832050561 shutdown_connections 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.503+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/832050561 wait complete. 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.503+0000 7ff7a7cfcb80 1 -- start start 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.503+0000 7ff7a7cfcb80 1 Processor -- start 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.504+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.504+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38ed950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.504+0000 7ff7a7cfcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb39626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.504+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941b2cd0 con 0x55ddb3760e40 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.504+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941b1860 con 0x55ddb384c9f0 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.504+0000 7ff7a7cfcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7941b1f30 con 0x55ddb38dfe70 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.504+0000 7ff7a7cfcb80 10 client.0 _ll_register_callbacks cb 0x55ddb37a7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.504+0000 7ff791e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.504+0000 7ff791e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38ed410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44450/0 (socket says 172.21.15.90:44450) 2024-05-27T08:00:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.504+0000 7ff791e00700 1 -- 172.21.15.90:0/4161234683 learned_addr learned my addr 172.21.15.90:0/4161234683 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.505+0000 7ff791400700 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.505+0000 7ff792800700 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb39626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.505+0000 7ff791e00700 1 -- 172.21.15.90:0/4161234683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 msgr2=0x55ddb39626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.505+0000 7ff791e00700 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb39626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.505+0000 7ff791e00700 1 -- 172.21.15.90:0/4161234683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 msgr2=0x55ddb38ed950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.505+0000 7ff791e00700 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38ed950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.505+0000 7ff791e00700 1 -- 172.21.15.90:0/4161234683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb3962c20 con 0x55ddb3760e40 2024-05-27T08:00:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.505+0000 7ff791e00700 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38ed410 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff78401b5e0 tx=0x7ff784032ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8603967c96704f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.505+0000 7ff773e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.505+0000 7ff773e00700 1 -- 172.21.15.90:0/4161234683 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784047060 con 0x55ddb3760e40 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff773e00700 1 -- 172.21.15.90:0/4161234683 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7ff784005040 con 0x55ddb3760e40 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/4161234683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff78401c610 con 0x55ddb3760e40 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff773e00700 1 -- 172.21.15.90:0/4161234683 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff78404e050 con 0x55ddb3760e40 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff7a7cfcb80 10 client.17829 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/4161234683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ddb3963e00 con 0x55ddb3760e40 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff773e00700 1 -- 172.21.15.90:0/4161234683 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff78401c610 con 0x55ddb3760e40 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff773e00700 1 -- 172.21.15.90:0/4161234683 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7ff784040020 con 0x55ddb3760e40 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff7a7cfcb80 10 client.17829 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff7a7cfcb80 10 client.17829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff7a7cfcb80 20 client.17829 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.506+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddb3964370 0x55ddb3966790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.507+0000 7ff7a7cfcb80 4 client.17829 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:00:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.507+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/4161234683 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ddb3809570 con 0x55ddb3964370 2024-05-27T08:00:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.507+0000 7ff791400700 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddb3964370 0x55ddb3966790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.508+0000 7ff791400700 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddb3964370 0x55ddb3966790 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9647713dff9bdb9d server_cookie=5f98fad1ea935f5c in_seq=0 out_seq=0 2024-05-27T08:00:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.508+0000 7ff773e00700 10 client.17829 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff773e00700 1 -- 172.21.15.90:0/4161234683 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55ddb3809570 con 0x55ddb3964370 2024-05-27T08:00:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff773e00700 10 client.17829 handle_command_reply: tid=0 2024-05-27T08:00:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 client.17829 shutdown 2024-05-27T08:00:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/4161234683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 msgr2=0x55ddb38ed410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38ed410 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff78401b5e0 tx=0x7ff784032ee0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/4161234683 shutdown_connections 2024-05-27T08:00:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55ddb3964370 0x55ddb3966790 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ddb38dfe70 0x55ddb39626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ddb3760e40 0x55ddb38ed410 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 --2- 172.21.15.90:0/4161234683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ddb384c9f0 0x55ddb38ed950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/4161234683 >> 172.21.15.90:0/4161234683 conn(0x55ddb37fa390 msgr2=0x55ddb38eac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:12.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff773e00700 0 client.17829 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:12.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/4161234683 shutdown_connections 2024-05-27T08:00:12.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 1 -- 172.21.15.90:0/4161234683 wait complete. 2024-05-27T08:00:12.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.509+0000 7ff7a7cfcb80 20 client.17829 trim_cache size 0 max 16384 2024-05-27T08:00:12.513 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:00:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "scrub active (1 inodes in the stack)", 2024-05-27T08:00:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": { 2024-05-27T08:00:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "b4155383-a76f-49ad-9e57-0c8b07473963": { 2024-05-27T08:00:12.513 INFO:teuthology.orchestra.run.smithi090.stdout: "path": "/", 2024-05-27T08:00:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: "tag": "b4155383-a76f-49ad-9e57-0c8b07473963", 2024-05-27T08:00:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: "options": "recursive,force" 2024-05-27T08:00:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:00:12.514 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:00:12.514 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.510+0000 7ff799200700 1 -- 172.21.15.90:0/2660790095 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff7780b29f0 msgr2=0x7ff7780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.510+0000 7ff799200700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff7780b29f0 0x7ff7780b4f00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff784005950 tx=0x7ff78400b410 comp rx=0 tx=0).stop 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.510+0000 7ff799200700 1 -- 172.21.15.90:0/2660790095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff794109840 msgr2=0x7ff794077640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.510+0000 7ff799200700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff794109840 0x7ff794077640 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff78c007ee0 tx=0x7ff78c00d3b0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.510+0000 7ff799200700 1 -- 172.21.15.90:0/2660790095 shutdown_connections 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.510+0000 7ff799200700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff794109840 0x7ff794077640 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.510+0000 7ff799200700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7941045c0 0x7ff794076bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.510+0000 7ff799200700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff7780b29f0 0x7ff7780b4f00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.510+0000 7ff799200700 1 --2- 172.21.15.90:0/2660790095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff794108ef0 0x7ff794077100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.510+0000 7ff799200700 1 -- 172.21.15.90:0/2660790095 >> 172.21.15.90:0/2660790095 conn(0x7ff7940ffe90 msgr2=0x7ff79410a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.511+0000 7ff799200700 1 -- 172.21.15.90:0/2660790095 shutdown_connections 2024-05-27T08:00:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:12.511+0000 7ff799200700 1 -- 172.21.15.90:0/2660790095 wait complete. 2024-05-27T08:00:13.356 INFO:tasks.cephfs.filesystem:scrub status for tag:b4155383-a76f-49ad-9e57-0c8b07473963 - {'path': '/', 'tag': 'b4155383-a76f-49ad-9e57-0c8b07473963', 'options': 'recursive,force'} 2024-05-27T08:00:13.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:13 smithi129 ceph-mon[35254]: osdmap e116: 12 total, 12 up, 12 in 2024-05-27T08:00:13.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:13 smithi129 ceph-mon[35254]: pgmap v443: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.9 MiB/s wr, 356 op/s 2024-05-27T08:00:13.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:13 smithi090 ceph-mon[31429]: osdmap e116: 12 total, 12 up, 12 in 2024-05-27T08:00:13.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:13 smithi090 ceph-mon[31429]: pgmap v443: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.9 MiB/s wr, 356 op/s 2024-05-27T08:00:13.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:13 smithi136 ceph-mon[35324]: osdmap e116: 12 total, 12 up, 12 in 2024-05-27T08:00:13.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:13 smithi136 ceph-mon[35324]: pgmap v443: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.9 MiB/s wr, 356 op/s 2024-05-27T08:00:13.833 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 595.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-05-27T08:00:14.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.138+0000 7fa427400640 1 -- 172.21.15.90:0/3383698154 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5639a172f2c0 con 0x5639a1847500 2024-05-27T08:00:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.138+0000 7fa427400640 10 client.27517 handle_command_reply: tid=0 2024-05-27T08:00:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.138+0000 7fa453a23740 1 client.27517 shutdown 2024-05-27T08:00:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.139+0000 7fa453a23740 1 -- 172.21.15.90:0/3383698154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 msgr2=0x5639a17c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:14.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.139+0000 7fa453a23740 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 0x5639a17c8900 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa440009ca0 tx=0x7fa440030810 comp rx=0 tx=0).stop 2024-05-27T08:00:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.140+0000 7fa453a23740 1 -- 172.21.15.90:0/3383698154 shutdown_connections 2024-05-27T08:00:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.140+0000 7fa453a23740 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5639a1847500 0x5639a1849950 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.140+0000 7fa427400640 0 client.27517 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.140+0000 7fa453a23740 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5639a13ff4d0 0x5639a17c8900 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.140+0000 7fa453a23740 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639a13f72f0 0x5639a183ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.140+0000 7fa453a23740 1 --2- 172.21.15.90:0/3383698154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5639a13ded80 0x5639a183e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:14.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.140+0000 7fa453a23740 1 -- 172.21.15.90:0/3383698154 >> 172.21.15.90:0/3383698154 conn(0x5639a1414300 msgr2=0x5639a17c56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.140+0000 7fa453a23740 1 -- 172.21.15.90:0/3383698154 shutdown_connections 2024-05-27T08:00:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.140+0000 7fa453a23740 1 -- 172.21.15.90:0/3383698154 wait complete. 2024-05-27T08:00:14.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.141+0000 7fa453a23740 20 client.27517 trim_cache size 0 max 16384 2024-05-27T08:00:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:00:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:00:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:00:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.142+0000 7fa447200640 1 -- 172.21.15.90:0/4205384525 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa42c0b2aa0 msgr2=0x7fa42c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.142+0000 7fa447200640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa42c0b2aa0 0x7fa42c0b4fb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa434005fd0 tx=0x7fa43400b760 comp rx=0 tx=0).stop 2024-05-27T08:00:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.142+0000 7fa447200640 1 -- 172.21.15.90:0/4205384525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa448104250 msgr2=0x7fa4481a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.142+0000 7fa447200640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa448104250 0x7fa4481a2580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa43800c970 tx=0x7fa43800ce30 comp rx=0 tx=0).stop 2024-05-27T08:00:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.143+0000 7fa447200640 1 -- 172.21.15.90:0/4205384525 shutdown_connections 2024-05-27T08:00:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.143+0000 7fa447200640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa42c0b2aa0 0x7fa42c0b4fb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.143+0000 7fa447200640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa448109f50 0x7fa44819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.143+0000 7fa447200640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa448109ab0 0x7fa4481a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.143+0000 7fa447200640 1 --2- 172.21.15.90:0/4205384525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa448104250 0x7fa4481a2580 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:14.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.143+0000 7fa447200640 1 -- 172.21.15.90:0/4205384525 >> 172.21.15.90:0/4205384525 conn(0x7fa448079240 msgr2=0x7fa448105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:14.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.143+0000 7fa447200640 1 -- 172.21.15.90:0/4205384525 shutdown_connections 2024-05-27T08:00:14.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:14.145+0000 7fa447200640 1 -- 172.21.15.90:0/4205384525 wait complete. 2024-05-27T08:00:15.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:14 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:15.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:14 smithi136 ceph-mon[35324]: pgmap v444: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.8 MiB/s wr, 345 op/s 2024-05-27T08:00:15.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:14 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:15.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:14 smithi129 ceph-mon[35254]: pgmap v444: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.8 MiB/s wr, 345 op/s 2024-05-27T08:00:15.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:14 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T08:00:15.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:14 smithi090 ceph-mon[31429]: pgmap v444: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.8 MiB/s wr, 345 op/s 2024-05-27T08:00:17.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:16 smithi136 ceph-mon[35324]: pgmap v445: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-05-27T08:00:17.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:16 smithi129 ceph-mon[35254]: pgmap v445: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-05-27T08:00:17.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:16 smithi090 ceph-mon[31429]: pgmap v445: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.8 MiB/s wr, 315 op/s 2024-05-27T08:00:18.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 600.0 s, 877.2 tps, lat 312.249 ms stddev 1350.109 2024-05-27T08:00:19.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:18 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T08:00:19.266 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:18 smithi136 ceph-mon[35324]: pgmap v446: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-05-27T08:00:19.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:19 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T08:00:19.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:19 smithi129 ceph-mon[35254]: pgmap v446: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-05-27T08:00:19.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:18 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T08:00:19.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:18 smithi090 ceph-mon[31429]: pgmap v446: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-05-27T08:00:21.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:20 smithi136 ceph-mon[35324]: pgmap v447: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-05-27T08:00:21.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:20 smithi129 ceph-mon[35254]: pgmap v447: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-05-27T08:00:21.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:20 smithi090 ceph-mon[31429]: pgmap v447: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.9 MiB/s wr, 277 op/s 2024-05-27T08:00:23.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:22 smithi136 ceph-mon[35324]: pgmap v448: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.4 MiB/s wr, 299 op/s 2024-05-27T08:00:23.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:22 smithi129 ceph-mon[35254]: pgmap v448: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.4 MiB/s wr, 299 op/s 2024-05-27T08:00:23.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:22 smithi090 ceph-mon[31429]: pgmap v448: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 3.4 MiB/s wr, 299 op/s 2024-05-27T08:00:23.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 605.0 s, 533.2 tps, lat 188.662 ms stddev 171.908 2024-05-27T08:00:26.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:25 smithi136 ceph-mon[35324]: pgmap v449: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.1 MiB/s wr, 473 op/s 2024-05-27T08:00:26.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:25 smithi129 ceph-mon[35254]: pgmap v449: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.1 MiB/s wr, 473 op/s 2024-05-27T08:00:26.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:25 smithi090 ceph-mon[31429]: pgmap v449: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.1 MiB/s wr, 473 op/s 2024-05-27T08:00:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.478+0000 7fe5c8c00640 1 -- 172.21.15.90:0/3073481462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c41075d0 msgr2=0x7fe5c41079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.478+0000 7fe5c8c00640 1 --2- 172.21.15.90:0/3073481462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c41075d0 0x7fe5c41079e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009900 tx=0x7fe5b002a420 comp rx=0 tx=0).stop 2024-05-27T08:00:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.479+0000 7fe5c8c00640 1 -- 172.21.15.90:0/3073481462 shutdown_connections 2024-05-27T08:00:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.479+0000 7fe5c8c00640 1 --2- 172.21.15.90:0/3073481462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c4108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.479+0000 7fe5c8c00640 1 --2- 172.21.15.90:0/3073481462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c41075d0 0x7fe5c41079e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.479+0000 7fe5c8c00640 1 --2- 172.21.15.90:0/3073481462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c4102b80 0x7fe5c4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.479+0000 7fe5c8c00640 1 -- 172.21.15.90:0/3073481462 >> 172.21.15.90:0/3073481462 conn(0x7fe5c40782d0 msgr2=0x7fe5c4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.479+0000 7fe5c8c00640 1 -- 172.21.15.90:0/3073481462 shutdown_connections 2024-05-27T08:00:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.479+0000 7fe5c8c00640 1 -- 172.21.15.90:0/3073481462 wait complete. 2024-05-27T08:00:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.480+0000 7fe5c8c00640 1 Processor -- start 2024-05-27T08:00:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.480+0000 7fe5c8c00640 1 -- start start 2024-05-27T08:00:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.481+0000 7fe5c8c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 0x7fe5c41a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.482+0000 7fe5c8c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 0x7fe5c41a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.482+0000 7fe5c8c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.482+0000 7fe5c8c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c411a310 con 0x7fe5c41075d0 2024-05-27T08:00:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.482+0000 7fe5c8c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c411a190 con 0x7fe5c4108000 2024-05-27T08:00:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.482+0000 7fe5c8c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c411a490 con 0x7fe5c4102b80 2024-05-27T08:00:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.482+0000 7fe5c1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 0x7fe5c41a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.482+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 0x7fe5c41a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.482+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.482+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 0x7fe5c41a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51608/0 (socket says 172.21.15.90:51608) 2024-05-27T08:00:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.482+0000 7fe5c1400640 1 -- 172.21.15.90:0/1811963231 learned_addr learned my addr 172.21.15.90:0/1811963231 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.483+0000 7fe5c2800640 1 -- 172.21.15.90:0/1811963231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 msgr2=0x7fe5c41a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.483+0000 7fe5c2800640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 0x7fe5c41a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.483+0000 7fe5c2800640 1 -- 172.21.15.90:0/1811963231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 msgr2=0x7fe5c41a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.483+0000 7fe5c2800640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 0x7fe5c41a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.483+0000 7fe5c2800640 1 -- 172.21.15.90:0/1811963231 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c41a5bf0 con 0x7fe5c4108000 2024-05-27T08:00:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.483+0000 7fe5c1e00640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 0x7fe5c41a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.483+0000 7fe5c1400640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 0x7fe5c41a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.484+0000 7fe5c2800640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00e940 tx=0x7fe5bc00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.484+0000 7fe5baa00640 1 -- 172.21.15.90:0/1811963231 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc004d20 con 0x7fe5c4108000 2024-05-27T08:00:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.484+0000 7fe5baa00640 1 -- 172.21.15.90:0/1811963231 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5bc005020 con 0x7fe5c4108000 2024-05-27T08:00:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.484+0000 7fe5baa00640 1 -- 172.21.15.90:0/1811963231 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0116d0 con 0x7fe5c4108000 2024-05-27T08:00:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.484+0000 7fe5c8c00640 1 -- 172.21.15.90:0/1811963231 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c41a5ee0 con 0x7fe5c4108000 2024-05-27T08:00:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.484+0000 7fe5c8c00640 1 -- 172.21.15.90:0/1811963231 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c41a63e0 con 0x7fe5c4108000 2024-05-27T08:00:27.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.489+0000 7fe5cfb86740 1 Processor -- start 2024-05-27T08:00:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.491+0000 7fe5baa00640 1 -- 172.21.15.90:0/1811963231 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe5bc00cb20 con 0x7fe5c4108000 2024-05-27T08:00:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.492+0000 7fe5cfb86740 1 -- start start 2024-05-27T08:00:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.493+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d0a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.493+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b293f2f0 0x5580b2d0ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.493+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d0b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.493+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d0a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.493+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d0a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53636/0 (socket says 172.21.15.90:53636) 2024-05-27T08:00:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.493+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5580b2c7bb90 con 0x5580b293f2f0 2024-05-27T08:00:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.493+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5580b2c6b730 con 0x5580b29681b0 2024-05-27T08:00:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.493+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5580b2c6f770 con 0x5580b29474d0 2024-05-27T08:00:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.494+0000 7fe5c1400640 1 -- 172.21.15.90:0/540305553 learned_addr learned my addr 172.21.15.90:0/540305553 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.494+0000 7fe5c1400640 1 --2- 172.21.15.90:0/540305553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d0b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.494+0000 7fe5c1400640 1 -- 172.21.15.90:0/540305553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 msgr2=0x5580b2d0b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.494+0000 7fe5c1400640 1 --2- 172.21.15.90:0/540305553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d0b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.494+0000 7fe5c1400640 1 -- 172.21.15.90:0/540305553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b293f2f0 msgr2=0x5580b2d0ada0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:00:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.494+0000 7fe5c1400640 1 --2- 172.21.15.90:0/540305553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b293f2f0 0x5580b2d0ada0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.494+0000 7fe5c1400640 1 -- 172.21.15.90:0/540305553 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b2c048b0 con 0x5580b29681b0 2024-05-27T08:00:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.495+0000 7fe5c1400640 1 --2- 172.21.15.90:0/540305553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d0a860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0003820 tx=0x7fe5b0005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7679b08a8485a94e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.496+0000 7fe5af000640 1 -- 172.21.15.90:0/540305553 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0004990 con 0x5580b29681b0 2024-05-27T08:00:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.496+0000 7fe5af000640 1 -- 172.21.15.90:0/540305553 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5b0005670 con 0x5580b29681b0 2024-05-27T08:00:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.496+0000 7fe5af000640 1 -- 172.21.15.90:0/540305553 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b003fd80 con 0x5580b29681b0 2024-05-27T08:00:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.497+0000 7fe5baa00640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b29d0 0x7fe5a80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.497+0000 7fe5baa00640 1 -- 172.21.15.90:0/1811963231 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7fe5bc0e2a00 con 0x7fe5c4108000 2024-05-27T08:00:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.498+0000 7fe5cfb86740 1 -- 172.21.15.90:0/540305553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 msgr2=0x5580b2d0a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.498+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/540305553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d0a860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0003820 tx=0x7fe5b0005d50 comp rx=0 tx=0).stop 2024-05-27T08:00:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.498+0000 7fe5c1e00640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b29d0 0x7fe5a80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.498+0000 7fe5cfb86740 1 -- 172.21.15.90:0/540305553 shutdown_connections 2024-05-27T08:00:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.498+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/540305553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d0b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.498+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/540305553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b293f2f0 0x5580b2d0ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.498+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/540305553 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d0a860 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.498+0000 7fe5cfb86740 1 -- 172.21.15.90:0/540305553 >> 172.21.15.90:0/540305553 conn(0x5580b295c300 msgr2=0x5580b2d07010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.499+0000 7fe5c1e00640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b29d0 0x7fe5a80b4ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5580b2d0bc10 tx=0x7fe5b4005f90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.499+0000 7fe5cfb86740 1 -- 172.21.15.90:0/540305553 shutdown_connections 2024-05-27T08:00:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.499+0000 7fe5cfb86740 1 -- 172.21.15.90:0/540305553 wait complete. 2024-05-27T08:00:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.500+0000 7fe5cfb86740 1 -- start start 2024-05-27T08:00:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.504+0000 7fe5cfb86740 1 Processor -- start 2024-05-27T08:00:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b293f2f0 0x5580b2d14a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d14f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d20cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5580b2951790 con 0x5580b293f2f0 2024-05-27T08:00:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5580b2c13bc0 con 0x5580b29681b0 2024-05-27T08:00:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5580b2d17930 con 0x5580b29474d0 2024-05-27T08:00:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5cfb86740 10 client.0 _ll_register_callbacks cb 0x5580b2bf1400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d14f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d20cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d14f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50336/0 (socket says 172.21.15.90:50336) 2024-05-27T08:00:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d20cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53652/0 (socket says 172.21.15.90:53652) 2024-05-27T08:00:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.508+0000 7fe5c1400640 1 -- 172.21.15.90:0/2033924622 learned_addr learned my addr 172.21.15.90:0/2033924622 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.509+0000 7fe5c1400640 1 -- 172.21.15.90:0/2033924622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 msgr2=0x5580b2d20cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.509+0000 7fe5c1400640 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d20cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.509+0000 7fe5c1400640 1 -- 172.21.15.90:0/2033924622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b293f2f0 msgr2=0x5580b2d14a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:00:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.510+0000 7fe5c1400640 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b293f2f0 0x5580b2d14a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.510+0000 7fe5c1400640 1 -- 172.21.15.90:0/2033924622 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b2980210 con 0x5580b29474d0 2024-05-27T08:00:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.512+0000 7fe5c1400640 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d14f80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0005f70 tx=0x7fe5b0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a8f6f7173a3088 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.513+0000 7fe5a3400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:00:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.513+0000 7fe5a3400640 1 -- 172.21.15.90:0/2033924622 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0004990 con 0x5580b29474d0 2024-05-27T08:00:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.514+0000 7fe5a3400640 1 -- 172.21.15.90:0/2033924622 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7fe5b0004b30 con 0x5580b29474d0 2024-05-27T08:00:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.514+0000 7fe5cfb86740 1 -- 172.21.15.90:0/2033924622 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b0005670 con 0x5580b29474d0 2024-05-27T08:00:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.515+0000 7fe5a3400640 1 -- 172.21.15.90:0/2033924622 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe5b006a050 con 0x5580b29474d0 2024-05-27T08:00:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.515+0000 7fe5cfb86740 10 client.27565 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.515+0000 7fe5cfb86740 1 -- 172.21.15.90:0/2033924622 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5580b2a4c620 con 0x5580b29474d0 2024-05-27T08:00:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.516+0000 7fe5a3400640 1 -- 172.21.15.90:0/2033924622 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5b0005670 con 0x5580b29474d0 2024-05-27T08:00:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.516+0000 7fe5a3400640 1 -- 172.21.15.90:0/2033924622 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe5b0031440 con 0x5580b29474d0 2024-05-27T08:00:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.517+0000 7fe5cfb86740 10 client.27565 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.517+0000 7fe5cfb86740 10 client.27565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.517+0000 7fe5cfb86740 20 client.27565 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.517+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580b2926d80 0x5580b2d26ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.518+0000 7fe5cfb86740 4 client.27565 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.518+0000 7fe5cfb86740 1 -- 172.21.15.90:0/2033924622 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580b2c7dae0 con 0x5580b2926d80 2024-05-27T08:00:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.518+0000 7fe5c1e00640 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580b2926d80 0x5580b2d26ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.520+0000 7fe5c1e00640 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580b2926d80 0x5580b2d26ae0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a80185fb59c3aa server_cookie=78a3d81336f03421 in_seq=0 out_seq=0 2024-05-27T08:00:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.521+0000 7fe5a3400640 10 client.27565 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.522+0000 7fe5a3400640 1 -- 172.21.15.90:0/2033924622 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5580b2c7dae0 con 0x5580b2926d80 2024-05-27T08:00:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.522+0000 7fe5a3400640 10 client.27565 handle_command_reply: tid=0 2024-05-27T08:00:27.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.522+0000 7fe5cfb86740 1 client.27565 shutdown 2024-05-27T08:00:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.523+0000 7fe5cfb86740 1 -- 172.21.15.90:0/2033924622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 msgr2=0x5580b2d14f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.523+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d14f80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0005f70 tx=0x7fe5b0000bc0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.523+0000 7fe5cfb86740 1 -- 172.21.15.90:0/2033924622 shutdown_connections 2024-05-27T08:00:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.523+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580b2926d80 0x5580b2d26ae0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.523+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b29681b0 0x5580b2d20cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.523+0000 7fe5a3400640 0 client.27565 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.523+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d14f80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.523+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/2033924622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b293f2f0 0x5580b2d14a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.524+0000 7fe5cfb86740 1 -- 172.21.15.90:0/2033924622 >> 172.21.15.90:0/2033924622 conn(0x5580b295c300 msgr2=0x5580b2d0a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.524+0000 7fe5cfb86740 1 -- 172.21.15.90:0/2033924622 shutdown_connections 2024-05-27T08:00:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.524+0000 7fe5cfb86740 1 -- 172.21.15.90:0/2033924622 wait complete. 2024-05-27T08:00:27.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.524+0000 7fe5cfb86740 20 client.27565 trim_cache size 0 max 16384 2024-05-27T08:00:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.565+0000 7fe5cfb86740 1 Processor -- start 2024-05-27T08:00:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.565+0000 7fe5cfb86740 1 -- start start 2024-05-27T08:00:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.566+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 0x5580b2d10240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.566+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d20a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.566+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29681b0 0x5580b2d20f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.566+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b0a60 con 0x5580b29681b0 2024-05-27T08:00:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.566+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41adab0 con 0x5580b293f2f0 2024-05-27T08:00:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.566+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41afcc0 con 0x5580b29474d0 2024-05-27T08:00:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.567+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29681b0 0x5580b2d20f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.567+0000 7fe5c1e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 0x5580b2d10240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.567+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d20a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.567+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d20a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50350/0 (socket says 172.21.15.90:50350) 2024-05-27T08:00:27.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.567+0000 7fe5c1400640 1 -- 172.21.15.90:0/3898779170 learned_addr learned my addr 172.21.15.90:0/3898779170 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.568+0000 7fe5c1400640 1 -- 172.21.15.90:0/3898779170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 msgr2=0x5580b2d10240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.568+0000 7fe5c1400640 1 --2- 172.21.15.90:0/3898779170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 0x5580b2d10240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.568+0000 7fe5c1400640 1 -- 172.21.15.90:0/3898779170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29681b0 msgr2=0x5580b2d20f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.568+0000 7fe5c1400640 1 --2- 172.21.15.90:0/3898779170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29681b0 0x5580b2d20f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.568+0000 7fe5c1400640 1 -- 172.21.15.90:0/3898779170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580b2d0bc40 con 0x5580b29474d0 2024-05-27T08:00:27.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.569+0000 7fe5c2800640 1 --2- 172.21.15.90:0/3898779170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29681b0 0x5580b2d20f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:27.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.570+0000 7fe5c1400640 1 --2- 172.21.15.90:0/3898779170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d20a30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0005940 tx=0x7fe5b00054a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f79e678d22cb2789 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.570+0000 7fe5ae600640 1 -- 172.21.15.90:0/3898779170 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0073070 con 0x5580b29474d0 2024-05-27T08:00:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.571+0000 7fe5ae600640 1 -- 172.21.15.90:0/3898779170 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5b0060070 con 0x5580b29474d0 2024-05-27T08:00:27.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.571+0000 7fe5ae600640 1 -- 172.21.15.90:0/3898779170 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0043070 con 0x5580b29474d0 2024-05-27T08:00:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.572+0000 7fe5cfb86740 1 -- 172.21.15.90:0/3898779170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 msgr2=0x5580b2d20a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.573+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/3898779170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d20a30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0005940 tx=0x7fe5b00054a0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.573+0000 7fe5cfb86740 1 -- 172.21.15.90:0/3898779170 shutdown_connections 2024-05-27T08:00:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.573+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/3898779170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29681b0 0x5580b2d20f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.573+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/3898779170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29474d0 0x5580b2d20a30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.573+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/3898779170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 0x5580b2d10240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.573+0000 7fe5cfb86740 1 -- 172.21.15.90:0/3898779170 >> 172.21.15.90:0/3898779170 conn(0x5580b295c300 msgr2=0x5580b2d14d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.577+0000 7fe5cfb86740 1 -- 172.21.15.90:0/3898779170 shutdown_connections 2024-05-27T08:00:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.577+0000 7fe5cfb86740 1 -- 172.21.15.90:0/3898779170 wait complete. 2024-05-27T08:00:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.578+0000 7fe5cfb86740 1 -- start start 2024-05-27T08:00:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.578+0000 7fe5cfb86740 1 Processor -- start 2024-05-27T08:00:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.579+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 0x5580b2d86f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.579+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29474d0 0x5580b2d874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.579+0000 7fe5cfb86740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29681b0 0x5580b2d87a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.579+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41af5f0 con 0x5580b29474d0 2024-05-27T08:00:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.579+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41ae180 con 0x5580b293f2f0 2024-05-27T08:00:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.579+0000 7fe5cfb86740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41ae850 con 0x5580b29681b0 2024-05-27T08:00:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.579+0000 7fe5cfb86740 10 client.0 _ll_register_callbacks cb 0x7fe5b00053f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.580+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29681b0 0x5580b2d87a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.580+0000 7fe5c1e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 0x5580b2d86f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.580+0000 7fe5c1e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 0x5580b2d86f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53674/0 (socket says 172.21.15.90:53674) 2024-05-27T08:00:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.581+0000 7fe5c2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29681b0 0x5580b2d87a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50364/0 (socket says 172.21.15.90:50364) 2024-05-27T08:00:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.581+0000 7fe5c1e00640 1 -- 172.21.15.90:0/1820787741 learned_addr learned my addr 172.21.15.90:0/1820787741 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.580+0000 7fe5c1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29474d0 0x5580b2d874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.583+0000 7fe5c1e00640 1 -- 172.21.15.90:0/1820787741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29681b0 msgr2=0x5580b2d87a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.583+0000 7fe5c1e00640 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29681b0 0x5580b2d87a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.583+0000 7fe5c1e00640 1 -- 172.21.15.90:0/1820787741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29474d0 msgr2=0x5580b2d874c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:27.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.583+0000 7fe5c1e00640 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29474d0 0x5580b2d874c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.583+0000 7fe5c1e00640 1 -- 172.21.15.90:0/1820787741 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580b2d1bc20 con 0x5580b293f2f0 2024-05-27T08:00:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.583+0000 7fe5c2800640 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29681b0 0x5580b2d87a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.584+0000 7fe5c1400640 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29474d0 0x5580b2d874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.586+0000 7fe5c1e00640 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 0x5580b2d86f80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4000c00 tx=0x7fe5b4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2be5f9fb5f99ada server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.586+0000 7fe5a3400640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:00:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.586+0000 7fe5a3400640 1 -- 172.21.15.90:0/1820787741 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4027640 con 0x5580b293f2f0 2024-05-27T08:00:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.587+0000 7fe5a3400640 1 -- 172.21.15.90:0/1820787741 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7fe5b40277e0 con 0x5580b293f2f0 2024-05-27T08:00:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.587+0000 7fe5cfb86740 1 -- 172.21.15.90:0/1820787741 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b0060070 con 0x5580b293f2f0 2024-05-27T08:00:27.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.587+0000 7fe5a3400640 1 -- 172.21.15.90:0/1820787741 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe5b4039050 con 0x5580b293f2f0 2024-05-27T08:00:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.587+0000 7fe5cfb86740 10 client.37472 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.587+0000 7fe5cfb86740 1 -- 172.21.15.90:0/1820787741 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5580b2d1ced0 con 0x5580b293f2f0 2024-05-27T08:00:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.588+0000 7fe5a3400640 1 -- 172.21.15.90:0/1820787741 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5b402b070 con 0x5580b293f2f0 2024-05-27T08:00:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.588+0000 7fe5a3400640 1 -- 172.21.15.90:0/1820787741 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe5b401f420 con 0x5580b293f2f0 2024-05-27T08:00:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.588+0000 7fe5cfb86740 10 client.37472 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.588+0000 7fe5cfb86740 10 client.37472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.588+0000 7fe5cfb86740 20 client.37472 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.588+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580b28fc1c0 0x5580b2da2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.588+0000 7fe5cfb86740 4 client.37472 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:00:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.588+0000 7fe5cfb86740 1 -- 172.21.15.90:0/1820787741 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe5b0060070 con 0x5580b28fc1c0 2024-05-27T08:00:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.591+0000 7fe5c1400640 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580b28fc1c0 0x5580b2da2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.593+0000 7fe5c1400640 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580b28fc1c0 0x5580b2da2110 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9e02255ec82e33 server_cookie=2af2f1ddbbfef1e1 in_seq=0 out_seq=0 2024-05-27T08:00:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.593+0000 7fe5a3400640 10 client.37472 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.665+0000 7fe5a3400640 1 -- 172.21.15.90:0/1820787741 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fe5b0060070 con 0x5580b28fc1c0 2024-05-27T08:00:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.665+0000 7fe5a3400640 10 client.37472 handle_command_reply: tid=0 2024-05-27T08:00:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.665+0000 7fe5cfb86740 1 client.37472 shutdown 2024-05-27T08:00:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.665+0000 7fe5cfb86740 1 -- 172.21.15.90:0/1820787741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 msgr2=0x5580b2d86f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.665+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 0x5580b2d86f80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4000c00 tx=0x7fe5b4004730 comp rx=0 tx=0).stop 2024-05-27T08:00:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.666+0000 7fe5cfb86740 1 -- 172.21.15.90:0/1820787741 shutdown_connections 2024-05-27T08:00:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.666+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580b28fc1c0 0x5580b2da2110 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.666+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580b29681b0 0x5580b2d87a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.666+0000 7fe5a3400640 0 client.37472 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.666+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580b29474d0 0x5580b2d874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.666+0000 7fe5cfb86740 1 --2- 172.21.15.90:0/1820787741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580b293f2f0 0x5580b2d86f80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.666+0000 7fe5cfb86740 1 -- 172.21.15.90:0/1820787741 >> 172.21.15.90:0/1820787741 conn(0x5580b295c300 msgr2=0x5580b2d14a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.666+0000 7fe5cfb86740 1 -- 172.21.15.90:0/1820787741 shutdown_connections 2024-05-27T08:00:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.666+0000 7fe5cfb86740 1 -- 172.21.15.90:0/1820787741 wait complete. 2024-05-27T08:00:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.667+0000 7fe5cfb86740 20 client.37472 trim_cache size 0 max 16384 2024-05-27T08:00:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:00:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:00:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:00:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.668+0000 7fe5c8c00640 1 -- 172.21.15.90:0/1811963231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b29d0 msgr2=0x7fe5a80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.668+0000 7fe5c8c00640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b29d0 0x7fe5a80b4ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5580b2d0bc10 tx=0x7fe5b4005f90 comp rx=0 tx=0).stop 2024-05-27T08:00:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.669+0000 7fe5c8c00640 1 -- 172.21.15.90:0/1811963231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 msgr2=0x7fe5c41a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.669+0000 7fe5c8c00640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00e940 tx=0x7fe5bc00ee00 comp rx=0 tx=0).stop 2024-05-27T08:00:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.669+0000 7fe5c8c00640 1 -- 172.21.15.90:0/1811963231 shutdown_connections 2024-05-27T08:00:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.669+0000 7fe5c8c00640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b29d0 0x7fe5a80b4ee0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.669+0000 7fe5c8c00640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.669+0000 7fe5c8c00640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 0x7fe5c41a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.669+0000 7fe5c8c00640 1 --2- 172.21.15.90:0/1811963231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 0x7fe5c41a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.669+0000 7fe5c8c00640 1 -- 172.21.15.90:0/1811963231 >> 172.21.15.90:0/1811963231 conn(0x7fe5c40782d0 msgr2=0x7fe5c4105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.673+0000 7fe5c8c00640 1 -- 172.21.15.90:0/1811963231 shutdown_connections 2024-05-27T08:00:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:27.673+0000 7fe5c8c00640 1 -- 172.21.15.90:0/1811963231 wait complete. 2024-05-27T08:00:28.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:27 smithi129 ceph-mon[35254]: pgmap v450: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 457 op/s 2024-05-27T08:00:28.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:27 smithi090 ceph-mon[31429]: pgmap v450: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 457 op/s 2024-05-27T08:00:28.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:28 smithi136 ceph-mon[35324]: pgmap v450: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 457 op/s 2024-05-27T08:00:28.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 610.0 s, 695.4 tps, lat 143.423 ms stddev 53.774 2024-05-27T08:00:30.319 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:30 smithi129 ceph-mon[35254]: pgmap v451: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-05-27T08:00:30.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:30 smithi090 ceph-mon[31429]: pgmap v451: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-05-27T08:00:30.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:30 smithi136 ceph-mon[35324]: pgmap v451: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.8 MiB/s wr, 610 op/s 2024-05-27T08:00:32.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:32 smithi129 ceph-mon[35254]: pgmap v452: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.5 MiB/s wr, 524 op/s 2024-05-27T08:00:32.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:32 smithi090 ceph-mon[31429]: pgmap v452: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.5 MiB/s wr, 524 op/s 2024-05-27T08:00:32.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:32 smithi136 ceph-mon[35324]: pgmap v452: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.5 MiB/s wr, 524 op/s 2024-05-27T08:00:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 615.0 s, 603.2 tps, lat 163.135 ms stddev 147.499 2024-05-27T08:00:34.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:34 smithi129 ceph-mon[35254]: pgmap v453: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 533 op/s 2024-05-27T08:00:34.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:34 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:00:34.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:34 smithi090 ceph-mon[31429]: pgmap v453: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 533 op/s 2024-05-27T08:00:34.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:34 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:00:34.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:34 smithi136 ceph-mon[35324]: pgmap v453: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 533 op/s 2024-05-27T08:00:34.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:34 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:00:36.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:36 smithi129 ceph-mon[35254]: pgmap v454: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 750 op/s 2024-05-27T08:00:36.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:36 smithi090 ceph-mon[31429]: pgmap v454: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 750 op/s 2024-05-27T08:00:36.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:36 smithi136 ceph-mon[35324]: pgmap v454: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 750 op/s 2024-05-27T08:00:37.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:37 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:00:37.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:37 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:00:37.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:37 smithi129 ceph-mon[35254]: pgmap v455: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.2 MiB/s wr, 582 op/s 2024-05-27T08:00:37.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:37 smithi129 ceph-mon[35254]: pgmap v456: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.3 MiB/s wr, 664 op/s 2024-05-27T08:00:37.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:37 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:00:37.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:37 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:00:37.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:37 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:00:37.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:37 smithi090 ceph-mon[31429]: pgmap v455: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.2 MiB/s wr, 582 op/s 2024-05-27T08:00:37.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:37 smithi090 ceph-mon[31429]: pgmap v456: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.3 MiB/s wr, 664 op/s 2024-05-27T08:00:37.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:37 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:00:37.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:37 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:00:37.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:37 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:00:37.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:37 smithi136 ceph-mon[35324]: pgmap v455: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 6.2 MiB/s wr, 582 op/s 2024-05-27T08:00:37.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:37 smithi136 ceph-mon[35324]: pgmap v456: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 7.3 MiB/s wr, 664 op/s 2024-05-27T08:00:37.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:37 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:00:38.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 620.0 s, 630.6 tps, lat 160.177 ms stddev 71.724 2024-05-27T08:00:39.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:39 smithi090 ceph-mon[31429]: pgmap v457: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.1 MiB/s wr, 488 op/s 2024-05-27T08:00:39.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:39 smithi136 ceph-mon[35324]: pgmap v457: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.1 MiB/s wr, 488 op/s 2024-05-27T08:00:39.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:39 smithi129 ceph-mon[35254]: pgmap v457: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.1 MiB/s wr, 488 op/s 2024-05-27T08:00:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.046+0000 7fe702400640 1 -- 172.21.15.90:0/369792105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f409ebe0 msgr2=0x7fe6f4098780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.046+0000 7fe702400640 1 --2- 172.21.15.90:0/369792105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f409ebe0 0x7fe6f4098780 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009860 tx=0x7fe6f002a340 comp rx=0 tx=0).stop 2024-05-27T08:00:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.047+0000 7fe702400640 1 -- 172.21.15.90:0/369792105 shutdown_connections 2024-05-27T08:00:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.047+0000 7fe702400640 1 --2- 172.21.15.90:0/369792105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f4099330 0x7fe6f40a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.047+0000 7fe702400640 1 --2- 172.21.15.90:0/369792105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f409f080 0x7fe6f4098cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.047+0000 7fe702400640 1 --2- 172.21.15.90:0/369792105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f409ebe0 0x7fe6f4098780 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.047+0000 7fe702400640 1 -- 172.21.15.90:0/369792105 >> 172.21.15.90:0/369792105 conn(0x7fe6f40942c0 msgr2=0x7fe6f4096710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.047+0000 7fe702400640 1 -- 172.21.15.90:0/369792105 shutdown_connections 2024-05-27T08:00:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.047+0000 7fe702400640 1 -- 172.21.15.90:0/369792105 wait complete. 2024-05-27T08:00:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.049+0000 7fe702400640 1 Processor -- start 2024-05-27T08:00:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.050+0000 7fe702400640 1 -- start start 2024-05-27T08:00:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.051+0000 7fe702400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f4099330 0x7fe6f40092a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.051+0000 7fe702400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f409ebe0 0x7fe6f40097e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.051+0000 7fe702400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f409f080 0x7fe6f4009d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.051+0000 7fe702400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40b9c90 con 0x7fe6f409ebe0 2024-05-27T08:00:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.051+0000 7fe702400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40b9b10 con 0x7fe6f409f080 2024-05-27T08:00:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.051+0000 7fe702400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40b9e10 con 0x7fe6f4099330 2024-05-27T08:00:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.052+0000 7fe6fbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f409ebe0 0x7fe6f40097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.052+0000 7fe701000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f4099330 0x7fe6f40092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.052+0000 7fe701000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f4099330 0x7fe6f40092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:34210/0 (socket says 172.21.15.90:34210) 2024-05-27T08:00:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.052+0000 7fe701000640 1 -- 172.21.15.90:0/1859029235 learned_addr learned my addr 172.21.15.90:0/1859029235 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.053+0000 7fe701a00640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f409f080 0x7fe6f4009d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.053+0000 7fe701000640 1 -- 172.21.15.90:0/1859029235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f409f080 msgr2=0x7fe6f4009d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.053+0000 7fe701000640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f409f080 0x7fe6f4009d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.053+0000 7fe701000640 1 -- 172.21.15.90:0/1859029235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f409ebe0 msgr2=0x7fe6f40097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.053+0000 7fe701000640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f409ebe0 0x7fe6f40097e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.053+0000 7fe701000640 1 -- 172.21.15.90:0/1859029235 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f4150d20 con 0x7fe6f4099330 2024-05-27T08:00:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.054+0000 7fe701000640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f4099330 0x7fe6f40092a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0035040 tx=0x7fe6f002aef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.056+0000 7fe6f9600640 1 -- 172.21.15.90:0/1859029235 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0037040 con 0x7fe6f4099330 2024-05-27T08:00:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.056+0000 7fe6f9600640 1 -- 172.21.15.90:0/1859029235 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6f0030c30 con 0x7fe6f4099330 2024-05-27T08:00:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.056+0000 7fe6f9600640 1 -- 172.21.15.90:0/1859029235 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00035f0 con 0x7fe6f4099330 2024-05-27T08:00:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.056+0000 7fe702400640 1 -- 172.21.15.90:0/1859029235 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f4150f50 con 0x7fe6f4099330 2024-05-27T08:00:41.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.058+0000 7fe702400640 1 -- 172.21.15.90:0/1859029235 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f4151300 con 0x7fe6f4099330 2024-05-27T08:00:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.064+0000 7fe70a9fd740 1 Processor -- start 2024-05-27T08:00:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.066+0000 7fe70a9fd740 1 -- start start 2024-05-27T08:00:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.067+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f62da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.067+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f62dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.067+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 0x55f19f62e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.067+0000 7fe70a9fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f19f59eb90 con 0x55f19f26a4d0 2024-05-27T08:00:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.067+0000 7fe70a9fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f19f58e730 con 0x55f19f2622f0 2024-05-27T08:00:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.067+0000 7fe70a9fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f19f592770 con 0x55f19f28b1b0 2024-05-27T08:00:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.067+0000 7fe6f9600640 1 -- 172.21.15.90:0/1859029235 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe6f0043020 con 0x7fe6f4099330 2024-05-27T08:00:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.068+0000 7fe701000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f62dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.069+0000 7fe701000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f62dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37840/0 (socket says 172.21.15.90:37840) 2024-05-27T08:00:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.069+0000 7fe701000640 1 -- 172.21.15.90:0/1884264095 learned_addr learned my addr 172.21.15.90:0/1884264095 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.069+0000 7fe6fbe00640 1 --2- 172.21.15.90:0/1884264095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 0x55f19f62e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.069+0000 7fe701a00640 1 --2- 172.21.15.90:0/1884264095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f62da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.069+0000 7fe701000640 1 -- 172.21.15.90:0/1884264095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 msgr2=0x55f19f62da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.070+0000 7fe701000640 1 --2- 172.21.15.90:0/1884264095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f62da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.070+0000 7fe701000640 1 -- 172.21.15.90:0/1884264095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 msgr2=0x55f19f62e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.071+0000 7fe701000640 1 --2- 172.21.15.90:0/1884264095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 0x55f19f62e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.071+0000 7fe701000640 1 -- 172.21.15.90:0/1884264095 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19f5278b0 con 0x55f19f2622f0 2024-05-27T08:00:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.072+0000 7fe6fbe00640 1 --2- 172.21.15.90:0/1884264095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 0x55f19f62e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.072+0000 7fe6f9600640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe6e40b2b30 0x7fe6e40b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.072+0000 7fe6f9600640 1 -- 172.21.15.90:0/1859029235 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7fe6f0101900 con 0x7fe6f4099330 2024-05-27T08:00:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.072+0000 7fe701000640 1 --2- 172.21.15.90:0/1884264095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f62dfd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00fec70 tx=0x7fe6f0107b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76ec4e114c7cb91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.073+0000 7fe701a00640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe6e40b2b30 0x7fe6e40b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.073+0000 7fe6ee000640 1 -- 172.21.15.90:0/1884264095 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0104e60 con 0x55f19f2622f0 2024-05-27T08:00:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.074+0000 7fe701a00640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe6e40b2b30 0x7fe6e40b5040 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc051db0 tx=0x7fe6fc06b040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.074+0000 7fe6ee000640 1 -- 172.21.15.90:0/1884264095 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6f00c83d0 con 0x55f19f2622f0 2024-05-27T08:00:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.074+0000 7fe6ee000640 1 -- 172.21.15.90:0/1884264095 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00c4780 con 0x55f19f2622f0 2024-05-27T08:00:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.076+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1884264095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 msgr2=0x55f19f62dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.076+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1884264095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f62dfd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00fec70 tx=0x7fe6f0107b20 comp rx=0 tx=0).stop 2024-05-27T08:00:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.078+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1884264095 shutdown_connections 2024-05-27T08:00:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.078+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1884264095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f62dfd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.078+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1884264095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 0x55f19f62e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.078+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1884264095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f62da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.078+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1884264095 >> 172.21.15.90:0/1884264095 conn(0x55f19f27f300 msgr2=0x55f19f62a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.079+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1884264095 shutdown_connections 2024-05-27T08:00:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.079+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1884264095 wait complete. 2024-05-27T08:00:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.081+0000 7fe70a9fd740 1 -- start start 2024-05-27T08:00:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.081+0000 7fe70a9fd740 1 Processor -- start 2024-05-27T08:00:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.082+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f637bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.082+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 0x55f19f638130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.082+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f643b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.082+0000 7fe70a9fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f19f274790 con 0x55f19f26a4d0 2024-05-27T08:00:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.082+0000 7fe70a9fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f19f536bc0 con 0x55f19f2622f0 2024-05-27T08:00:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.082+0000 7fe70a9fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f19f58e730 con 0x55f19f28b1b0 2024-05-27T08:00:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.082+0000 7fe70a9fd740 10 client.0 _ll_register_callbacks cb 0x55f19f514400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.083+0000 7fe701000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f637bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.083+0000 7fe701000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f637bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37856/0 (socket says 172.21.15.90:37856) 2024-05-27T08:00:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.083+0000 7fe701000640 1 -- 172.21.15.90:0/673816895 learned_addr learned my addr 172.21.15.90:0/673816895 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.083+0000 7fe701a00640 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f643b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.084+0000 7fe6fbe00640 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 0x55f19f638130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.085+0000 7fe701000640 1 -- 172.21.15.90:0/673816895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 msgr2=0x55f19f643b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.085+0000 7fe701000640 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f643b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.085+0000 7fe701000640 1 -- 172.21.15.90:0/673816895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 msgr2=0x55f19f638130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.085+0000 7fe701000640 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 0x55f19f638130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.085+0000 7fe701000640 1 -- 172.21.15.90:0/673816895 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19f2a3210 con 0x55f19f2622f0 2024-05-27T08:00:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.085+0000 7fe6fbe00640 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 0x55f19f638130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.085+0000 7fe701000640 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f637bf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00309e0 tx=0x7fe6f0005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a849bce4e0fc8302 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.085+0000 7fe6e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:00:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.086+0000 7fe6e2a00640 1 -- 172.21.15.90:0/673816895 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00c8030 con 0x55f19f2622f0 2024-05-27T08:00:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.086+0000 7fe6e2a00640 1 -- 172.21.15.90:0/673816895 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7fe6f00eb0c0 con 0x55f19f2622f0 2024-05-27T08:00:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.086+0000 7fe70a9fd740 1 -- 172.21.15.90:0/673816895 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6f00c83d0 con 0x55f19f2622f0 2024-05-27T08:00:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.087+0000 7fe6e2a00640 1 -- 172.21.15.90:0/673816895 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe6f00c8210 con 0x55f19f2622f0 2024-05-27T08:00:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.088+0000 7fe70a9fd740 10 client.37484 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.088+0000 7fe70a9fd740 1 -- 172.21.15.90:0/673816895 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f19f36f620 con 0x55f19f2622f0 2024-05-27T08:00:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.088+0000 7fe6e2a00640 1 -- 172.21.15.90:0/673816895 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6f00c83d0 con 0x55f19f2622f0 2024-05-27T08:00:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.090+0000 7fe6e2a00640 1 -- 172.21.15.90:0/673816895 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe6f00c4b50 con 0x55f19f2622f0 2024-05-27T08:00:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.090+0000 7fe70a9fd740 10 client.37484 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.090+0000 7fe70a9fd740 10 client.37484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.090+0000 7fe70a9fd740 20 client.37484 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.090+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f19f249d80 0x55f19f649920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.090+0000 7fe70a9fd740 4 client.37484 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:41.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.090+0000 7fe70a9fd740 1 -- 172.21.15.90:0/673816895 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f19f5a0ae0 con 0x55f19f249d80 2024-05-27T08:00:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.091+0000 7fe6fbe00640 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f19f249d80 0x55f19f649920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.093+0000 7fe6fbe00640 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f19f249d80 0x55f19f649920 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e98e65d7616cd57 server_cookie=58f1da53f8a37a1d in_seq=0 out_seq=0 2024-05-27T08:00:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.093+0000 7fe6e2a00640 10 client.37484 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe6e2a00640 1 -- 172.21.15.90:0/673816895 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f19f5a0ae0 con 0x55f19f249d80 2024-05-27T08:00:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe6e2a00640 10 client.37484 handle_command_reply: tid=0 2024-05-27T08:00:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe70a9fd740 1 client.37484 shutdown 2024-05-27T08:00:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe70a9fd740 1 -- 172.21.15.90:0/673816895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 msgr2=0x55f19f637bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f637bf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00309e0 tx=0x7fe6f0005c90 comp rx=0 tx=0).stop 2024-05-27T08:00:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe70a9fd740 1 -- 172.21.15.90:0/673816895 shutdown_connections 2024-05-27T08:00:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f19f249d80 0x55f19f649920 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f643b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f26a4d0 0x55f19f638130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe6e2a00640 0 client.37484 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/673816895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f2622f0 0x55f19f637bf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.096+0000 7fe70a9fd740 1 -- 172.21.15.90:0/673816895 >> 172.21.15.90:0/673816895 conn(0x55f19f27f300 msgr2=0x55f19f62d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.097+0000 7fe70a9fd740 1 -- 172.21.15.90:0/673816895 shutdown_connections 2024-05-27T08:00:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.097+0000 7fe70a9fd740 1 -- 172.21.15.90:0/673816895 wait complete. 2024-05-27T08:00:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.097+0000 7fe70a9fd740 20 client.37484 trim_cache size 0 max 16384 2024-05-27T08:00:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.132+0000 7fe70a9fd740 1 Processor -- start 2024-05-27T08:00:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.133+0000 7fe70a9fd740 1 -- start start 2024-05-27T08:00:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.133+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f26a4d0 0x55f19f6333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.133+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f62d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.133+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 0x55f19f643900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.133+0000 7fe70a9fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f415d100 con 0x55f19f21f1c0 2024-05-27T08:00:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.133+0000 7fe70a9fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f415a150 con 0x55f19f26a4d0 2024-05-27T08:00:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.133+0000 7fe70a9fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f415c360 con 0x55f19f28b1b0 2024-05-27T08:00:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.134+0000 7fe6fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f62d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.134+0000 7fe6fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f62d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:34238/0 (socket says 172.21.15.90:34238) 2024-05-27T08:00:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.134+0000 7fe6fbe00640 1 -- 172.21.15.90:0/1150452584 learned_addr learned my addr 172.21.15.90:0/1150452584 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.134+0000 7fe701a00640 1 --2- 172.21.15.90:0/1150452584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 0x55f19f643900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.134+0000 7fe701000640 1 --2- 172.21.15.90:0/1150452584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f26a4d0 0x55f19f6333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.135+0000 7fe701000640 1 -- 172.21.15.90:0/1150452584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 msgr2=0x55f19f62d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.135+0000 7fe701000640 1 --2- 172.21.15.90:0/1150452584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f62d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.135+0000 7fe701000640 1 -- 172.21.15.90:0/1150452584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 msgr2=0x55f19f643900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.135+0000 7fe701000640 1 --2- 172.21.15.90:0/1150452584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 0x55f19f643900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.135+0000 7fe701000640 1 -- 172.21.15.90:0/1150452584 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19f643e40 con 0x55f19f26a4d0 2024-05-27T08:00:41.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.135+0000 7fe701a00640 1 --2- 172.21.15.90:0/1150452584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 0x55f19f643900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.136+0000 7fe701000640 1 --2- 172.21.15.90:0/1150452584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f26a4d0 0x55f19f6333d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe6f01016f0 tx=0x7fe6f0107b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c92471dbcc3a78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.136+0000 7fe6ed600640 1 -- 172.21.15.90:0/1150452584 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0105070 con 0x55f19f26a4d0 2024-05-27T08:00:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.136+0000 7fe6ed600640 1 -- 172.21.15.90:0/1150452584 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6f00c83d0 con 0x55f19f26a4d0 2024-05-27T08:00:41.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.136+0000 7fe6ed600640 1 -- 172.21.15.90:0/1150452584 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f00c8690 con 0x55f19f26a4d0 2024-05-27T08:00:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.137+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1150452584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f26a4d0 msgr2=0x55f19f6333d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.137+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1150452584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f26a4d0 0x55f19f6333d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe6f01016f0 tx=0x7fe6f0107b20 comp rx=0 tx=0).stop 2024-05-27T08:00:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.137+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1150452584 shutdown_connections 2024-05-27T08:00:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.137+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1150452584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 0x55f19f643900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.137+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1150452584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f28b1b0 0x55f19f62d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.137+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1150452584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f26a4d0 0x55f19f6333d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.137+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1150452584 >> 172.21.15.90:0/1150452584 conn(0x55f19f27f300 msgr2=0x55f19f637eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.138+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1150452584 shutdown_connections 2024-05-27T08:00:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.138+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1150452584 wait complete. 2024-05-27T08:00:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.139+0000 7fe70a9fd740 1 -- start start 2024-05-27T08:00:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.139+0000 7fe70a9fd740 1 Processor -- start 2024-05-27T08:00:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.140+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 0x55f19f640220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.140+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f26a4d0 0x55f19f63b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.140+0000 7fe70a9fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f28b1b0 0x55f19f63bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.140+0000 7fe701000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 0x55f19f640220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.141+0000 7fe701000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 0x55f19f640220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33788/0 (socket says 172.21.15.90:33788) 2024-05-27T08:00:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.141+0000 7fe6fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f26a4d0 0x55f19f63b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.141+0000 7fe6fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f26a4d0 0x55f19f63b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:34240/0 (socket says 172.21.15.90:34240) 2024-05-27T08:00:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.141+0000 7fe701a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f28b1b0 0x55f19f63bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.141+0000 7fe701a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f28b1b0 0x55f19f63bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37882/0 (socket says 172.21.15.90:37882) 2024-05-27T08:00:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.141+0000 7fe701000640 1 -- 172.21.15.90:0/1343040401 learned_addr learned my addr 172.21.15.90:0/1343040401 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.141+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f415bc90 con 0x55f19f21f1c0 2024-05-27T08:00:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.141+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f415a820 con 0x55f19f28b1b0 2024-05-27T08:00:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.142+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f415aef0 con 0x55f19f26a4d0 2024-05-27T08:00:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.143+0000 7fe70a9fd740 10 client.0 _ll_register_callbacks cb 0x7fe6f01078e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.143+0000 7fe701a00640 1 -- 172.21.15.90:0/1343040401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f26a4d0 msgr2=0x55f19f63b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.143+0000 7fe701a00640 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f26a4d0 0x55f19f63b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.143+0000 7fe701a00640 1 -- 172.21.15.90:0/1343040401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 msgr2=0x55f19f640220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.143+0000 7fe701a00640 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 0x55f19f640220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.143+0000 7fe701a00640 1 -- 172.21.15.90:0/1343040401 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19f63c220 con 0x55f19f28b1b0 2024-05-27T08:00:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.143+0000 7fe6fbe00640 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f26a4d0 0x55f19f63b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:00:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.144+0000 7fe701a00640 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f28b1b0 0x55f19f63bab0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc069170 tx=0x7fe6fc068f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c499de03911951aa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.144+0000 7fe6e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:00:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.144+0000 7fe6e2a00640 1 -- 172.21.15.90:0/1343040401 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0744f0 con 0x55f19f28b1b0 2024-05-27T08:00:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.144+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6f00c83d0 con 0x55f19f28b1b0 2024-05-27T08:00:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.145+0000 7fe6e2a00640 1 -- 172.21.15.90:0/1343040401 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7fe6fc078040 con 0x55f19f28b1b0 2024-05-27T08:00:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.145+0000 7fe6e2a00640 1 -- 172.21.15.90:0/1343040401 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe6fc097050 con 0x55f19f28b1b0 2024-05-27T08:00:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.145+0000 7fe70a9fd740 10 client.37490 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.145+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f19f6b1ed0 con 0x55f19f28b1b0 2024-05-27T08:00:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.146+0000 7fe6e2a00640 1 -- 172.21.15.90:0/1343040401 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe6f00c83d0 con 0x55f19f28b1b0 2024-05-27T08:00:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.146+0000 7fe6e2a00640 1 -- 172.21.15.90:0/1343040401 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe6fc08b730 con 0x55f19f28b1b0 2024-05-27T08:00:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.146+0000 7fe70a9fd740 10 client.37490 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.146+0000 7fe70a9fd740 10 client.37490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.146+0000 7fe70a9fd740 20 client.37490 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.146+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f19f6b25d0 0x55f19f6b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.146+0000 7fe70a9fd740 4 client.37490 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:00:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.146+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f19f59a2c0 con 0x55f19f6b25d0 2024-05-27T08:00:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.147+0000 7fe701000640 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f19f6b25d0 0x55f19f6b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.149+0000 7fe701000640 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f19f6b25d0 0x55f19f6b4a10 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1307b1867011798d server_cookie=dfbd61bc95c56da0 in_seq=0 out_seq=0 2024-05-27T08:00:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.149+0000 7fe6e2a00640 10 client.37490 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.196+0000 7fe6e2a00640 1 -- 172.21.15.90:0/1343040401 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f19f59a2c0 con 0x55f19f6b25d0 2024-05-27T08:00:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.196+0000 7fe6e2a00640 10 client.37490 handle_command_reply: tid=0 2024-05-27T08:00:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.196+0000 7fe70a9fd740 1 client.37490 shutdown 2024-05-27T08:00:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.196+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f28b1b0 msgr2=0x55f19f63bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.196+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f28b1b0 0x55f19f63bab0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc069170 tx=0x7fe6fc068f80 comp rx=0 tx=0).stop 2024-05-27T08:00:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.197+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 shutdown_connections 2024-05-27T08:00:41.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.197+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f19f6b25d0 0x55f19f6b4a10 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.197+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f19f28b1b0 0x55f19f63bab0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.197+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f19f26a4d0 0x55f19f63b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.197+0000 7fe70a9fd740 1 --2- 172.21.15.90:0/1343040401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f19f21f1c0 0x55f19f640220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.197+0000 7fe6e2a00640 0 client.37490 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.197+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 >> 172.21.15.90:0/1343040401 conn(0x55f19f27f300 msgr2=0x55f19f6428f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.197+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 shutdown_connections 2024-05-27T08:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.197+0000 7fe70a9fd740 1 -- 172.21.15.90:0/1343040401 wait complete. 2024-05-27T08:00:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.197+0000 7fe70a9fd740 20 client.37490 trim_cache size 0 max 16384 2024-05-27T08:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:00:41.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.199+0000 7fe702400640 1 -- 172.21.15.90:0/1859029235 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe6e40b2b30 msgr2=0x7fe6e40b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.199+0000 7fe702400640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe6e40b2b30 0x7fe6e40b5040 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc051db0 tx=0x7fe6fc06b040 comp rx=0 tx=0).stop 2024-05-27T08:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.200+0000 7fe702400640 1 -- 172.21.15.90:0/1859029235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f4099330 msgr2=0x7fe6f40092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.200+0000 7fe702400640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f4099330 0x7fe6f40092a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0035040 tx=0x7fe6f002aef0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.201+0000 7fe702400640 1 -- 172.21.15.90:0/1859029235 shutdown_connections 2024-05-27T08:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.201+0000 7fe702400640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe6e40b2b30 0x7fe6e40b5040 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.201+0000 7fe702400640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6f409f080 0x7fe6f4009d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.201+0000 7fe702400640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe6f409ebe0 0x7fe6f40097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.201+0000 7fe702400640 1 --2- 172.21.15.90:0/1859029235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6f4099330 0x7fe6f40092a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.201+0000 7fe702400640 1 -- 172.21.15.90:0/1859029235 >> 172.21.15.90:0/1859029235 conn(0x7fe6f40942c0 msgr2=0x7fe6f40946d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.201+0000 7fe702400640 1 -- 172.21.15.90:0/1859029235 shutdown_connections 2024-05-27T08:00:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:41.201+0000 7fe702400640 1 -- 172.21.15.90:0/1859029235 wait complete. 2024-05-27T08:00:41.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:41 smithi090 ceph-mon[31429]: pgmap v458: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.3 MiB/s wr, 713 op/s 2024-05-27T08:00:41.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:41 smithi136 ceph-mon[35324]: pgmap v458: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.3 MiB/s wr, 713 op/s 2024-05-27T08:00:41.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:41 smithi129 ceph-mon[35254]: pgmap v458: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.3 MiB/s wr, 713 op/s 2024-05-27T08:00:43.358 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:00:43.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:43 smithi090 ceph-mon[31429]: pgmap v459: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.6 MiB/s wr, 723 op/s 2024-05-27T08:00:43.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:43 smithi136 ceph-mon[35324]: pgmap v459: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.6 MiB/s wr, 723 op/s 2024-05-27T08:00:43.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:43 smithi129 ceph-mon[35254]: pgmap v459: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.6 MiB/s wr, 723 op/s 2024-05-27T08:00:43.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 625.0 s, 593.6 tps, lat 164.043 ms stddev 113.528 2024-05-27T08:00:44.702 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:00:45.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:45 smithi090 ceph-mon[31429]: pgmap v460: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-05-27T08:00:45.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:45 smithi136 ceph-mon[35324]: pgmap v460: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-05-27T08:00:45.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:45 smithi129 ceph-mon[35254]: pgmap v460: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.6 MiB/s wr, 598 op/s 2024-05-27T08:00:47.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:47 smithi090 ceph-mon[31429]: pgmap v461: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-05-27T08:00:47.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:47 smithi136 ceph-mon[35324]: pgmap v461: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-05-27T08:00:47.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:47 smithi129 ceph-mon[35254]: pgmap v461: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-05-27T08:00:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 630.0 s, 442.0 tps, lat 230.574 ms stddev 220.090 2024-05-27T08:00:49.443 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:49 smithi090 ceph-mon[31429]: pgmap v462: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.0 MiB/s wr, 521 op/s 2024-05-27T08:00:49.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:49 smithi136 ceph-mon[35324]: pgmap v462: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.0 MiB/s wr, 521 op/s 2024-05-27T08:00:49.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:49 smithi129 ceph-mon[35254]: pgmap v462: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.0 MiB/s wr, 521 op/s 2024-05-27T08:00:51.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:51 smithi136 ceph-mon[35324]: pgmap v463: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 MiB/s wr, 677 op/s 2024-05-27T08:00:51.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:51 smithi129 ceph-mon[35254]: pgmap v463: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 MiB/s wr, 677 op/s 2024-05-27T08:00:51.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:51 smithi090 ceph-mon[31429]: pgmap v463: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.6 MiB/s wr, 677 op/s 2024-05-27T08:00:53.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:53 smithi136 ceph-mon[35324]: pgmap v464: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-05-27T08:00:53.508 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:53 smithi090 ceph-mon[31429]: pgmap v464: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-05-27T08:00:53.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:53 smithi129 ceph-mon[35254]: pgmap v464: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.8 MiB/s wr, 496 op/s 2024-05-27T08:00:53.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 635.0 s, 369.8 tps, lat 266.681 ms stddev 258.534 2024-05-27T08:00:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.450+0000 7efedd600640 1 -- 172.21.15.90:0/1680295893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efed810a320 msgr2=0x7efed810a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.450+0000 7efedd600640 1 --2- 172.21.15.90:0/1680295893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efed810a320 0x7efed810a730 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7efec8009900 tx=0x7efec802a410 comp rx=0 tx=0).stop 2024-05-27T08:00:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.451+0000 7efedd600640 1 -- 172.21.15.90:0/1680295893 shutdown_connections 2024-05-27T08:00:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.451+0000 7efedd600640 1 --2- 172.21.15.90:0/1680295893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efed810f370 0x7efed8116b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.451+0000 7efedd600640 1 --2- 172.21.15.90:0/1680295893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed810e8b0 0x7efed810ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.451+0000 7efedd600640 1 --2- 172.21.15.90:0/1680295893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efed810a320 0x7efed810a730 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.451+0000 7efedd600640 1 -- 172.21.15.90:0/1680295893 >> 172.21.15.90:0/1680295893 conn(0x7efed8076070 msgr2=0x7efed8076480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.451+0000 7efedd600640 1 -- 172.21.15.90:0/1680295893 shutdown_connections 2024-05-27T08:00:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.451+0000 7efedd600640 1 -- 172.21.15.90:0/1680295893 wait complete. 2024-05-27T08:00:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.452+0000 7efedd600640 1 Processor -- start 2024-05-27T08:00:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.452+0000 7efedd600640 1 -- start start 2024-05-27T08:00:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.452+0000 7efedd600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efed810a320 0x7efed81a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.452+0000 7efedd600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed810e8b0 0x7efed81a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.452+0000 7efedd600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efed810f370 0x7efed81a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.452+0000 7efedd600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efed811a150 con 0x7efed810a320 2024-05-27T08:00:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.452+0000 7efedd600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efed8119fd0 con 0x7efed810f370 2024-05-27T08:00:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.452+0000 7efedd600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efed811a2d0 con 0x7efed810e8b0 2024-05-27T08:00:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.453+0000 7efed7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efed810a320 0x7efed81a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.453+0000 7efed7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efed810a320 0x7efed81a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55806/0 (socket says 172.21.15.90:55806) 2024-05-27T08:00:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.453+0000 7efed7e00640 1 -- 172.21.15.90:0/2891458132 learned_addr learned my addr 172.21.15.90:0/2891458132 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.453+0000 7efedcc00640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efed810f370 0x7efed81a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.453+0000 7efed7400640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed810e8b0 0x7efed81a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.453+0000 7efed7e00640 1 -- 172.21.15.90:0/2891458132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed810e8b0 msgr2=0x7efed81a6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.453+0000 7efed7e00640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed810e8b0 0x7efed81a6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.453+0000 7efed7e00640 1 -- 172.21.15.90:0/2891458132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efed810f370 msgr2=0x7efed81a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.453+0000 7efed7e00640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efed810f370 0x7efed81a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.453+0000 7efed7e00640 1 -- 172.21.15.90:0/2891458132 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed81a1170 con 0x7efed810a320 2024-05-27T08:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.454+0000 7efed7e00640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efed810a320 0x7efed81a68a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7efec8009230 tx=0x7efec8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.454+0000 7efed4c00640 1 -- 172.21.15.90:0/2891458132 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec803f030 con 0x7efed810a320 2024-05-27T08:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.454+0000 7efedd600640 1 -- 172.21.15.90:0/2891458132 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed81a1400 con 0x7efed810a320 2024-05-27T08:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.454+0000 7efedd600640 1 -- 172.21.15.90:0/2891458132 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed81ad740 con 0x7efed810a320 2024-05-27T08:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.454+0000 7efed4c00640 1 -- 172.21.15.90:0/2891458132 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efec8003a80 con 0x7efed810a320 2024-05-27T08:00:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.454+0000 7efed4c00640 1 -- 172.21.15.90:0/2891458132 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8035070 con 0x7efed810a320 2024-05-27T08:00:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.457+0000 7efee5bcd740 1 Processor -- start 2024-05-27T08:00:54.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efee5bcd740 1 -- start start 2024-05-27T08:00:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efed4c00640 1 -- 172.21.15.90:0/2891458132 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efec8042030 con 0x7efed810a320 2024-05-27T08:00:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0a71b0 0x561e7e449a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e07e2f0 0x561e7e449fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e44a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efee5bcd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561e7e3bab90 con 0x561e7e0864d0 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efee5bcd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561e7e3aa730 con 0x561e7e0a71b0 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efee5bcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561e7e3ae770 con 0x561e7e07e2f0 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efed7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e44a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efed7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e44a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55816/0 (socket says 172.21.15.90:55816) 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.458+0000 7efed7400640 1 -- 172.21.15.90:0/2597708361 learned_addr learned my addr 172.21.15.90:0/2597708361 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.459+0000 7efed7e00640 1 --2- 172.21.15.90:0/2597708361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e07e2f0 0x561e7e449fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.459+0000 7efedcc00640 1 --2- 172.21.15.90:0/2597708361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0a71b0 0x561e7e449a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.459+0000 7efed7400640 1 -- 172.21.15.90:0/2597708361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e07e2f0 msgr2=0x561e7e449fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.459+0000 7efed7400640 1 --2- 172.21.15.90:0/2597708361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e07e2f0 0x561e7e449fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.459+0000 7efed7400640 1 -- 172.21.15.90:0/2597708361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0a71b0 msgr2=0x561e7e449a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.459+0000 7efed7400640 1 --2- 172.21.15.90:0/2597708361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0a71b0 0x561e7e449a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.459+0000 7efed7400640 1 -- 172.21.15.90:0/2597708361 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e7e3438b0 con 0x561e7e0864d0 2024-05-27T08:00:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.459+0000 7efedcc00640 1 --2- 172.21.15.90:0/2597708361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0a71b0 0x561e7e449a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:00:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.459+0000 7efed7400640 1 --2- 172.21.15.90:0/2597708361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e44a4e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7efec400bc20 tx=0x7efec400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475674f5dcd900f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.460+0000 7efecd200640 1 -- 172.21.15.90:0/2597708361 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4015070 con 0x561e7e0864d0 2024-05-27T08:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.460+0000 7efecd200640 1 -- 172.21.15.90:0/2597708361 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efec401a8f0 con 0x561e7e0864d0 2024-05-27T08:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.460+0000 7efecd200640 1 -- 172.21.15.90:0/2597708361 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4011860 con 0x561e7e0864d0 2024-05-27T08:00:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.460+0000 7efed4c00640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efebc0b2950 0x7efebc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.460+0000 7efed4c00640 1 -- 172.21.15.90:0/2891458132 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7efec81030b0 con 0x7efed810a320 2024-05-27T08:00:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.460+0000 7efedcc00640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efebc0b2950 0x7efebc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.461+0000 7efee5bcd740 1 -- 172.21.15.90:0/2597708361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 msgr2=0x561e7e44a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.461+0000 7efee5bcd740 1 --2- 172.21.15.90:0/2597708361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e44a4e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7efec400bc20 tx=0x7efec400f730 comp rx=0 tx=0).stop 2024-05-27T08:00:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.461+0000 7efee5bcd740 1 -- 172.21.15.90:0/2597708361 shutdown_connections 2024-05-27T08:00:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.461+0000 7efee5bcd740 1 --2- 172.21.15.90:0/2597708361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e44a4e0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.461+0000 7efee5bcd740 1 --2- 172.21.15.90:0/2597708361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e07e2f0 0x561e7e449fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.461+0000 7efee5bcd740 1 --2- 172.21.15.90:0/2597708361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0a71b0 0x561e7e449a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.461+0000 7efee5bcd740 1 -- 172.21.15.90:0/2597708361 >> 172.21.15.90:0/2597708361 conn(0x561e7e09b300 msgr2=0x561e7e446210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.461+0000 7efedcc00640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efebc0b2950 0x7efebc0b4e60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x561e7e446bd0 tx=0x7efed00094a0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.461+0000 7efee5bcd740 1 -- 172.21.15.90:0/2597708361 shutdown_connections 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.461+0000 7efee5bcd740 1 -- 172.21.15.90:0/2597708361 wait complete. 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.462+0000 7efee5bcd740 1 -- start start 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.462+0000 7efee5bcd740 1 Processor -- start 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.462+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e07e2f0 0x561e7e453c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.462+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0864d0 0x561e7e454140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.462+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0a71b0 0x561e7e45fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.462+0000 7efee5bcd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561e7e090790 con 0x561e7e0a71b0 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.462+0000 7efee5bcd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561e7e352bc0 con 0x561e7e0864d0 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.462+0000 7efee5bcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561e7e456b90 con 0x561e7e07e2f0 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efee5bcd740 10 client.0 _ll_register_callbacks cb 0x561e7e330400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:54.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efedcc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0a71b0 0x561e7e45fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efedcc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0a71b0 0x561e7e45fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55832/0 (socket says 172.21.15.90:55832) 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efedcc00640 1 -- 172.21.15.90:0/3802072863 learned_addr learned my addr 172.21.15.90:0/3802072863 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efed7400640 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0864d0 0x561e7e454140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efed7e00640 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e07e2f0 0x561e7e453c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efedcc00640 1 -- 172.21.15.90:0/3802072863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e07e2f0 msgr2=0x561e7e453c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efedcc00640 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e07e2f0 0x561e7e453c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efedcc00640 1 -- 172.21.15.90:0/3802072863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0864d0 msgr2=0x561e7e454140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efedcc00640 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0864d0 0x561e7e454140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.463+0000 7efedcc00640 1 -- 172.21.15.90:0/3802072863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7e0bf210 con 0x561e7e0a71b0 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.464+0000 7efedcc00640 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0a71b0 0x561e7e45fec0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7efed0012b50 tx=0x7efed0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5a35fcf1b5e96 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.464+0000 7efec1600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:00:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.464+0000 7efec1600640 1 -- 172.21.15.90:0/3802072863 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0023070 con 0x561e7e0a71b0 2024-05-27T08:00:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.464+0000 7efee5bcd740 1 -- 172.21.15.90:0/3802072863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec401a8f0 con 0x561e7e0a71b0 2024-05-27T08:00:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.464+0000 7efec1600640 1 -- 172.21.15.90:0/3802072863 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7efed001f420 con 0x561e7e0a71b0 2024-05-27T08:00:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.464+0000 7efec1600640 1 -- 172.21.15.90:0/3802072863 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7efed002f050 con 0x561e7e0a71b0 2024-05-27T08:00:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.464+0000 7efee5bcd740 10 client.17871 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.464+0000 7efee5bcd740 1 -- 172.21.15.90:0/3802072863 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561e7e18b620 con 0x561e7e0a71b0 2024-05-27T08:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.465+0000 7efec1600640 1 -- 172.21.15.90:0/3802072863 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efec401a8f0 con 0x561e7e0a71b0 2024-05-27T08:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.465+0000 7efec1600640 1 -- 172.21.15.90:0/3802072863 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7efed000f050 con 0x561e7e0a71b0 2024-05-27T08:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.465+0000 7efee5bcd740 10 client.17871 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.465+0000 7efee5bcd740 10 client.17871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.465+0000 7efee5bcd740 20 client.17871 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.465+0000 7efee5bcd740 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561e7e065d80 0x561e7e465fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.465+0000 7efee5bcd740 4 client.17871 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.465+0000 7efee5bcd740 1 -- 172.21.15.90:0/3802072863 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e7e3bcae0 con 0x561e7e065d80 2024-05-27T08:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.466+0000 7efed7e00640 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561e7e065d80 0x561e7e465fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.467+0000 7efed7e00640 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561e7e065d80 0x561e7e465fc0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b919deefbeb5f34 server_cookie=7462d5b3b7eea1a9 in_seq=0 out_seq=0 2024-05-27T08:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.467+0000 7efec1600640 10 client.17871 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.469+0000 7efec1600640 1 -- 172.21.15.90:0/3802072863 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561e7e3bcae0 con 0x561e7e065d80 2024-05-27T08:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.469+0000 7efec1600640 10 client.17871 handle_command_reply: tid=0 2024-05-27T08:00:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.469+0000 7efee5bcd740 1 client.17871 shutdown 2024-05-27T08:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.469+0000 7efee5bcd740 1 -- 172.21.15.90:0/3802072863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0a71b0 msgr2=0x561e7e45fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.469+0000 7efee5bcd740 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0a71b0 0x561e7e45fec0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7efed0012b50 tx=0x7efed0004a40 comp rx=0 tx=0).stop 2024-05-27T08:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.470+0000 7efee5bcd740 1 -- 172.21.15.90:0/3802072863 shutdown_connections 2024-05-27T08:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.470+0000 7efee5bcd740 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561e7e065d80 0x561e7e465fc0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.470+0000 7efee5bcd740 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0864d0 0x561e7e454140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.470+0000 7efee5bcd740 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0a71b0 0x561e7e45fec0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.470+0000 7efee5bcd740 1 --2- 172.21.15.90:0/3802072863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e07e2f0 0x561e7e453c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.470+0000 7efec1600640 0 client.17871 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.470+0000 7efee5bcd740 1 -- 172.21.15.90:0/3802072863 >> 172.21.15.90:0/3802072863 conn(0x561e7e09b300 msgr2=0x561e7e44b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.470+0000 7efee5bcd740 1 -- 172.21.15.90:0/3802072863 shutdown_connections 2024-05-27T08:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.470+0000 7efee5bcd740 1 -- 172.21.15.90:0/3802072863 wait complete. 2024-05-27T08:00:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.470+0000 7efee5bcd740 20 client.17871 trim_cache size 0 max 16384 2024-05-27T08:00:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.489+0000 7efee5bcd740 1 Processor -- start 2024-05-27T08:00:54.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.490+0000 7efee5bcd740 1 -- start start 2024-05-27T08:00:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.490+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e07e2f0 0x561e7e456330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.490+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e0864d0 0x561e7e45fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.490+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0a71b0 0x561e7e460170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.490+0000 7efee5bcd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b90c0 con 0x561e7e07e2f0 2024-05-27T08:00:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.490+0000 7efee5bcd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b6110 con 0x561e7e0a71b0 2024-05-27T08:00:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.490+0000 7efee5bcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b8320 con 0x561e7e0864d0 2024-05-27T08:00:54.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.490+0000 7efed7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e07e2f0 0x561e7e456330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.490+0000 7efed7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e07e2f0 0x561e7e456330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55836/0 (socket says 172.21.15.90:55836) 2024-05-27T08:00:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.490+0000 7efed7e00640 1 -- 172.21.15.90:0/1011827773 learned_addr learned my addr 172.21.15.90:0/1011827773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.491+0000 7efed7400640 1 --2- 172.21.15.90:0/1011827773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e0864d0 0x561e7e45fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.491+0000 7efed7e00640 1 -- 172.21.15.90:0/1011827773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e0864d0 msgr2=0x561e7e45fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.491+0000 7efed7e00640 1 --2- 172.21.15.90:0/1011827773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e0864d0 0x561e7e45fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.491+0000 7efed7e00640 1 -- 172.21.15.90:0/1011827773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0a71b0 msgr2=0x561e7e460170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.491+0000 7efed7e00640 1 --2- 172.21.15.90:0/1011827773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0a71b0 0x561e7e460170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.491+0000 7efed7e00640 1 -- 172.21.15.90:0/1011827773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e7e446de0 con 0x561e7e07e2f0 2024-05-27T08:00:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.491+0000 7efed7e00640 1 --2- 172.21.15.90:0/1011827773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e07e2f0 0x561e7e456330 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7efec80c71a0 tx=0x7efec802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b870494ec5fdf8c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.492+0000 7efec3e00640 1 -- 172.21.15.90:0/1011827773 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8049040 con 0x561e7e07e2f0 2024-05-27T08:00:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.492+0000 7efec3e00640 1 -- 172.21.15.90:0/1011827773 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efec80ca6c0 con 0x561e7e07e2f0 2024-05-27T08:00:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.492+0000 7efec3e00640 1 -- 172.21.15.90:0/1011827773 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80c61c0 con 0x561e7e07e2f0 2024-05-27T08:00:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.493+0000 7efee5bcd740 1 -- 172.21.15.90:0/1011827773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e07e2f0 msgr2=0x561e7e456330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.493+0000 7efee5bcd740 1 --2- 172.21.15.90:0/1011827773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e07e2f0 0x561e7e456330 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7efec80c71a0 tx=0x7efec802a410 comp rx=0 tx=0).stop 2024-05-27T08:00:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.493+0000 7efee5bcd740 1 -- 172.21.15.90:0/1011827773 shutdown_connections 2024-05-27T08:00:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.493+0000 7efee5bcd740 1 --2- 172.21.15.90:0/1011827773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e0a71b0 0x561e7e460170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.493+0000 7efee5bcd740 1 --2- 172.21.15.90:0/1011827773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e0864d0 0x561e7e45fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.493+0000 7efee5bcd740 1 --2- 172.21.15.90:0/1011827773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e07e2f0 0x561e7e456330 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.493+0000 7efee5bcd740 1 -- 172.21.15.90:0/1011827773 >> 172.21.15.90:0/1011827773 conn(0x561e7e09b300 msgr2=0x561e7e44a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.493+0000 7efee5bcd740 1 -- 172.21.15.90:0/1011827773 shutdown_connections 2024-05-27T08:00:54.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.493+0000 7efee5bcd740 1 -- 172.21.15.90:0/1011827773 wait complete. 2024-05-27T08:00:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.494+0000 7efee5bcd740 1 -- start start 2024-05-27T08:00:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.494+0000 7efee5bcd740 1 Processor -- start 2024-05-27T08:00:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.494+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e45be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.494+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e0a71b0 0x561e7e45c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.494+0000 7efee5bcd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e03b1c0 0x561e7e4573e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.494+0000 7efee5bcd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b7c50 con 0x561e7e0864d0 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.494+0000 7efee5bcd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b67e0 con 0x561e7e03b1c0 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.494+0000 7efee5bcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b6eb0 con 0x561e7e0a71b0 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.494+0000 7efee5bcd740 10 client.0 _ll_register_callbacks cb 0x7efed0009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efed7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e45be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efed7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e45be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55852/0 (socket says 172.21.15.90:55852) 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efed7e00640 1 -- 172.21.15.90:0/2484063015 learned_addr learned my addr 172.21.15.90:0/2484063015 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efedcc00640 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e03b1c0 0x561e7e4573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efed7e00640 1 -- 172.21.15.90:0/2484063015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e0a71b0 msgr2=0x561e7e45c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efed7e00640 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e0a71b0 0x561e7e45c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efed7e00640 1 -- 172.21.15.90:0/2484063015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e03b1c0 msgr2=0x561e7e4573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efed7e00640 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e03b1c0 0x561e7e4573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efed7e00640 1 -- 172.21.15.90:0/2484063015 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7e457aa0 con 0x561e7e0864d0 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efedcc00640 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e03b1c0 0x561e7e4573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efed7e00640 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e45be70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7efec81004e0 tx=0x7efec8003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbb0294e0646090 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.495+0000 7efec1600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.496+0000 7efec1600640 1 -- 172.21.15.90:0/2484063015 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8049070 con 0x561e7e0864d0 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.496+0000 7efec1600640 1 -- 172.21.15.90:0/2484063015 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7efec80c61c0 con 0x561e7e0864d0 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.496+0000 7efee5bcd740 1 -- 172.21.15.90:0/2484063015 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efec80ca6c0 con 0x561e7e0864d0 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.496+0000 7efec1600640 1 -- 172.21.15.90:0/2484063015 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7efec80d2e20 con 0x561e7e0864d0 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.496+0000 7efee5bcd740 10 client.17877 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.496+0000 7efee5bcd740 1 -- 172.21.15.90:0/2484063015 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561e7e4cdfe0 con 0x561e7e0864d0 2024-05-27T08:00:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.496+0000 7efec1600640 1 -- 172.21.15.90:0/2484063015 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efec80ca6c0 con 0x561e7e0864d0 2024-05-27T08:00:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.496+0000 7efec1600640 1 -- 172.21.15.90:0/2484063015 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7efec80cc030 con 0x561e7e0864d0 2024-05-27T08:00:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.497+0000 7efee5bcd740 10 client.17877 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.497+0000 7efee5bcd740 10 client.17877 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.497+0000 7efee5bcd740 20 client.17877 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.497+0000 7efee5bcd740 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561e7e4ce5f0 0x561e7e4d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.497+0000 7efee5bcd740 4 client.17877 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:00:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.497+0000 7efee5bcd740 1 -- 172.21.15.90:0/2484063015 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e7e3b62c0 con 0x561e7e4ce5f0 2024-05-27T08:00:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.497+0000 7efed7400640 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561e7e4ce5f0 0x561e7e4d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.499+0000 7efec1600640 10 client.17877 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.499+0000 7efed7400640 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561e7e4ce5f0 0x561e7e4d0a40 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9620efb940339da9 server_cookie=5cc46870d239696b in_seq=0 out_seq=0 2024-05-27T08:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.761+0000 7efec1600640 1 -- 172.21.15.90:0/2484063015 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561e7e3b62c0 con 0x561e7e4ce5f0 2024-05-27T08:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.761+0000 7efec1600640 10 client.17877 handle_command_reply: tid=0 2024-05-27T08:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.761+0000 7efee5bcd740 1 client.17877 shutdown 2024-05-27T08:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.761+0000 7efee5bcd740 1 -- 172.21.15.90:0/2484063015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 msgr2=0x561e7e45be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.761+0000 7efee5bcd740 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e45be70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7efec81004e0 tx=0x7efec8003e40 comp rx=0 tx=0).stop 2024-05-27T08:00:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.763+0000 7efee5bcd740 1 -- 172.21.15.90:0/2484063015 shutdown_connections 2024-05-27T08:00:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.763+0000 7efee5bcd740 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561e7e4ce5f0 0x561e7e4d0a40 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.763+0000 7efee5bcd740 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561e7e03b1c0 0x561e7e4573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.763+0000 7efee5bcd740 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561e7e0a71b0 0x561e7e45c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.763+0000 7efee5bcd740 1 --2- 172.21.15.90:0/2484063015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561e7e0864d0 0x561e7e45be70 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.763+0000 7efee5bcd740 1 -- 172.21.15.90:0/2484063015 >> 172.21.15.90:0/2484063015 conn(0x561e7e09b300 msgr2=0x561e7e44a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.763+0000 7efec1600640 0 client.17877 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.763+0000 7efee5bcd740 1 -- 172.21.15.90:0/2484063015 shutdown_connections 2024-05-27T08:00:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.763+0000 7efee5bcd740 1 -- 172.21.15.90:0/2484063015 wait complete. 2024-05-27T08:00:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.764+0000 7efee5bcd740 20 client.17877 trim_cache size 0 max 16384 2024-05-27T08:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:00:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.767+0000 7efedd600640 1 -- 172.21.15.90:0/2891458132 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efebc0b2950 msgr2=0x7efebc0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.767+0000 7efedd600640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efebc0b2950 0x7efebc0b4e60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x561e7e446bd0 tx=0x7efed00094a0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.767+0000 7efedd600640 1 -- 172.21.15.90:0/2891458132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efed810a320 msgr2=0x7efed81a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.767+0000 7efedd600640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efed810a320 0x7efed81a68a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7efec8009230 tx=0x7efec8004270 comp rx=0 tx=0).stop 2024-05-27T08:00:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.768+0000 7efedd600640 1 -- 172.21.15.90:0/2891458132 shutdown_connections 2024-05-27T08:00:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.768+0000 7efedd600640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7efebc0b2950 0x7efebc0b4e60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.768+0000 7efedd600640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efed810f370 0x7efed81a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.768+0000 7efedd600640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed810e8b0 0x7efed81a6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.768+0000 7efedd600640 1 --2- 172.21.15.90:0/2891458132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efed810a320 0x7efed81a68a0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.768+0000 7efedd600640 1 -- 172.21.15.90:0/2891458132 >> 172.21.15.90:0/2891458132 conn(0x7efed8076070 msgr2=0x7efed8107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.769+0000 7efedd600640 1 -- 172.21.15.90:0/2891458132 shutdown_connections 2024-05-27T08:00:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:00:54.769+0000 7efedd600640 1 -- 172.21.15.90:0/2891458132 wait complete. 2024-05-27T08:00:55.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:55 smithi090 ceph-mon[31429]: pgmap v465: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 615 op/s 2024-05-27T08:00:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.482+0000 7f204f000700 1 -- 172.21.15.90:0/1023289797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2048108f10 msgr2=0x7f2048109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.482+0000 7f204f000700 1 --2- 172.21.15.90:0/1023289797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2048108f10 0x7f2048109320 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f20340099c0 tx=0x7f2034009cd0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.483+0000 7f204f000700 1 -- 172.21.15.90:0/1023289797 shutdown_connections 2024-05-27T08:00:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.483+0000 7f204f000700 1 --2- 172.21.15.90:0/1023289797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2048109860 0x7f20481111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.483+0000 7f204f000700 1 --2- 172.21.15.90:0/1023289797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20481045e0 0x7f20481049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.483+0000 7f204f000700 1 --2- 172.21.15.90:0/1023289797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2048108f10 0x7f2048109320 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.483+0000 7f204f000700 1 -- 172.21.15.90:0/1023289797 >> 172.21.15.90:0/1023289797 conn(0x7f2048100030 msgr2=0x7f2048102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.484+0000 7f204f000700 1 -- 172.21.15.90:0/1023289797 shutdown_connections 2024-05-27T08:00:55.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.485+0000 7f204f000700 1 -- 172.21.15.90:0/1023289797 wait complete. 2024-05-27T08:00:55.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.487+0000 7f204f000700 1 Processor -- start 2024-05-27T08:00:55.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.487+0000 7f204f000700 1 -- start start 2024-05-27T08:00:55.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.488+0000 7f204f000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20481045e0 0x7f2048118b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.488+0000 7f204f000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2048108f10 0x7f20481190b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.489+0000 7f204f000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2048109860 0x7f2048113d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.489+0000 7f2047e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20481045e0 0x7f2048118b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.489+0000 7f204f000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2048076bd0 con 0x7f20481045e0 2024-05-27T08:00:55.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.489+0000 7f204f000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2048076a50 con 0x7f2048109860 2024-05-27T08:00:55.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.489+0000 7f204f000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2048076d50 con 0x7f2048108f10 2024-05-27T08:00:55.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.489+0000 7f2047e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20481045e0 0x7f2048118b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55860/0 (socket says 172.21.15.90:55860) 2024-05-27T08:00:55.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.489+0000 7f2047e00700 1 -- 172.21.15.90:0/4162265604 learned_addr learned my addr 172.21.15.90:0/4162265604 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:55.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.490+0000 7f2047e00700 1 -- 172.21.15.90:0/4162265604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2048108f10 msgr2=0x7f20481190b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.490+0000 7f2047e00700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2048108f10 0x7f20481190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.490+0000 7f2047e00700 1 -- 172.21.15.90:0/4162265604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2048109860 msgr2=0x7f2048113d10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T08:00:55.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.490+0000 7f2047e00700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2048109860 0x7f2048113d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.490+0000 7f2047e00700 1 -- 172.21.15.90:0/4162265604 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2048114540 con 0x7f20481045e0 2024-05-27T08:00:55.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.491+0000 7f2047e00700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20481045e0 0x7f2048118b50 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f203800e9a0 tx=0x7f203800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.491+0000 7f2044c00700 1 -- 172.21.15.90:0/4162265604 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203800cbd0 con 0x7f20481045e0 2024-05-27T08:00:55.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.491+0000 7f204f000700 1 -- 172.21.15.90:0/4162265604 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2048114830 con 0x7f20481045e0 2024-05-27T08:00:55.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.491+0000 7f204f000700 1 -- 172.21.15.90:0/4162265604 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204811ea40 con 0x7f20481045e0 2024-05-27T08:00:55.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.492+0000 7f2044c00700 1 -- 172.21.15.90:0/4162265604 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f203800cd70 con 0x7f20481045e0 2024-05-27T08:00:55.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.492+0000 7f2044c00700 1 -- 172.21.15.90:0/4162265604 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203800cbd0 con 0x7f20481045e0 2024-05-27T08:00:55.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:55 smithi136 ceph-mon[35324]: pgmap v465: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 615 op/s 2024-05-27T08:00:55.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.502+0000 7f2044c00700 1 -- 172.21.15.90:0/4162265604 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f20380184a0 con 0x7f20481045e0 2024-05-27T08:00:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.505+0000 7f2044c00700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f202c0b2ca0 0x7f202c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.505+0000 7f2044c00700 1 -- 172.21.15.90:0/4162265604 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7f20380e3790 con 0x7f20481045e0 2024-05-27T08:00:55.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.506+0000 7f2047400700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f202c0b2ca0 0x7f202c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.512+0000 7f2047400700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f202c0b2ca0 0x7f202c0b51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20340098e0 tx=0x7f2034000bc0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.516+0000 7f205db94b80 1 Processor -- start 2024-05-27T08:00:55.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.517+0000 7f205db94b80 1 -- start start 2024-05-27T08:00:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.517+0000 7f205db94b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e827330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.517+0000 7f205db94b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e827870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.517+0000 7f205db94b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 0x560a0e834110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.518+0000 7f205db94b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560a0e755e40 con 0x560a0e6a8e40 2024-05-27T08:00:55.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.518+0000 7f205db94b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560a0e7459e0 con 0x560a0e7949f0 2024-05-27T08:00:55.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.518+0000 7f205db94b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560a0e749a20 con 0x560a0e827e70 2024-05-27T08:00:55.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.518+0000 7f2047e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e827330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.518+0000 7f204d000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 0x560a0e834110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.518+0000 7f2047e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e827330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38374/0 (socket says 172.21.15.90:38374) 2024-05-27T08:00:55.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.518+0000 7f2047e00700 1 -- 172.21.15.90:0/2708831296 learned_addr learned my addr 172.21.15.90:0/2708831296 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:55.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.519+0000 7f2047400700 1 --2- 172.21.15.90:0/2708831296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e827870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.519+0000 7f2047400700 1 -- 172.21.15.90:0/2708831296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 msgr2=0x560a0e834110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.519+0000 7f2047400700 1 --2- 172.21.15.90:0/2708831296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 0x560a0e834110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.519+0000 7f2047400700 1 -- 172.21.15.90:0/2708831296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 msgr2=0x560a0e827330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.519+0000 7f2047400700 1 --2- 172.21.15.90:0/2708831296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e827330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.519+0000 7f2047400700 1 -- 172.21.15.90:0/2708831296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a0e70e420 con 0x560a0e6a8e40 2024-05-27T08:00:55.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.520+0000 7f2047400700 1 --2- 172.21.15.90:0/2708831296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e827870 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2034006010 tx=0x7f203401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3431d0fb14579264 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.521+0000 7f203d600700 1 -- 172.21.15.90:0/2708831296 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2034005630 con 0x560a0e6a8e40 2024-05-27T08:00:55.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.521+0000 7f203d600700 1 -- 172.21.15.90:0/2708831296 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f203400bc20 con 0x560a0e6a8e40 2024-05-27T08:00:55.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.521+0000 7f203d600700 1 -- 172.21.15.90:0/2708831296 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203400f460 con 0x560a0e6a8e40 2024-05-27T08:00:55.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.523+0000 7f205db94b80 1 -- 172.21.15.90:0/2708831296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 msgr2=0x560a0e827870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.523+0000 7f205db94b80 1 --2- 172.21.15.90:0/2708831296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e827870 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2034006010 tx=0x7f203401c000 comp rx=0 tx=0).stop 2024-05-27T08:00:55.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.524+0000 7f205db94b80 1 -- 172.21.15.90:0/2708831296 shutdown_connections 2024-05-27T08:00:55.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.524+0000 7f205db94b80 1 --2- 172.21.15.90:0/2708831296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 0x560a0e834110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.524+0000 7f205db94b80 1 --2- 172.21.15.90:0/2708831296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e827330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.524+0000 7f205db94b80 1 --2- 172.21.15.90:0/2708831296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e827870 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.524+0000 7f205db94b80 1 -- 172.21.15.90:0/2708831296 >> 172.21.15.90:0/2708831296 conn(0x560a0e742390 msgr2=0x560a0e8234c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.524+0000 7f205db94b80 1 -- 172.21.15.90:0/2708831296 shutdown_connections 2024-05-27T08:00:55.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.525+0000 7f205db94b80 1 -- 172.21.15.90:0/2708831296 wait complete. 2024-05-27T08:00:55.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.525+0000 7f205db94b80 1 -- start start 2024-05-27T08:00:55.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.526+0000 7f205db94b80 1 Processor -- start 2024-05-27T08:00:55.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.527+0000 7f205db94b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e831740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.527+0000 7f205db94b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e831c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.527+0000 7f205db94b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 0x560a0e8321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.527+0000 7f205db94b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560a0e64bfa0 con 0x560a0e6a8e40 2024-05-27T08:00:55.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.527+0000 7f205db94b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560a0e6e6760 con 0x560a0e7949f0 2024-05-27T08:00:55.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.527+0000 7f205db94b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560a0e834b20 con 0x560a0e827e70 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.527+0000 7f205db94b80 10 client.0 _ll_register_callbacks cb 0x560a0e6efc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.528+0000 7f2047e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e831740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.528+0000 7f204d000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 0x560a0e8321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.528+0000 7f204d000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 0x560a0e8321c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:32860/0 (socket says 172.21.15.90:32860) 2024-05-27T08:00:55.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.528+0000 7f2047400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e831c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.528+0000 7f2047400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e831c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38382/0 (socket says 172.21.15.90:38382) 2024-05-27T08:00:55.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.528+0000 7f2047400700 1 -- 172.21.15.90:0/2354467293 learned_addr learned my addr 172.21.15.90:0/2354467293 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:55.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.528+0000 7f2047e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e831740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55886/0 (socket says 172.21.15.90:55886) 2024-05-27T08:00:55.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.529+0000 7f2047e00700 1 -- 172.21.15.90:0/2354467293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 msgr2=0x560a0e8321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.530+0000 7f2047e00700 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 0x560a0e8321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.530+0000 7f2047e00700 1 -- 172.21.15.90:0/2354467293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 msgr2=0x560a0e831c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.530+0000 7f2047e00700 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e831c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.530+0000 7f2047e00700 1 -- 172.21.15.90:0/2354467293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a0e6958d0 con 0x560a0e6a8e40 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.531+0000 7f2047e00700 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e831740 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f20380a6720 tx=0x7f2038092330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7afa692fb062919 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.531+0000 7f2032000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:00:55.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.531+0000 7f2032000700 1 -- 172.21.15.90:0/2354467293 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20380c3020 con 0x560a0e6a8e40 2024-05-27T08:00:55.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.532+0000 7f205db94b80 1 -- 172.21.15.90:0/2354467293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f203400bc20 con 0x560a0e6a8e40 2024-05-27T08:00:55.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.532+0000 7f2032000700 1 -- 172.21.15.90:0/2354467293 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7f20380b3280 con 0x560a0e6a8e40 2024-05-27T08:00:55.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.532+0000 7f2032000700 1 -- 172.21.15.90:0/2354467293 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f20380aea50 con 0x560a0e6a8e40 2024-05-27T08:00:55.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.532+0000 7f205db94b80 10 client.17889 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:55.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.532+0000 7f205db94b80 1 -- 172.21.15.90:0/2354467293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560a0e716060 con 0x560a0e6a8e40 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.533+0000 7f2032000700 1 -- 172.21.15.90:0/2354467293 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f203400bc20 con 0x560a0e6a8e40 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.533+0000 7f2032000700 1 -- 172.21.15.90:0/2354467293 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f20380e3340 con 0x560a0e6a8e40 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.534+0000 7f205db94b80 10 client.17889 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.534+0000 7f205db94b80 10 client.17889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.534+0000 7f205db94b80 20 client.17889 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:55.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.534+0000 7f205db94b80 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560a0e841c00 0x560a0e8440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.534+0000 7f205db94b80 4 client.17889 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:00:55.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.534+0000 7f205db94b80 1 -- 172.21.15.90:0/2354467293 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a0e757d90 con 0x560a0e841c00 2024-05-27T08:00:55.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.535+0000 7f2047400700 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560a0e841c00 0x560a0e8440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.536+0000 7f2032000700 10 client.17889 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:55.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.536+0000 7f2047400700 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560a0e841c00 0x560a0e8440b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8237b809274a1eb7 server_cookie=a8f739df5a1691f7 in_seq=0 out_seq=0 2024-05-27T08:00:55.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.538+0000 7f2032000700 1 -- 172.21.15.90:0/2354467293 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560a0e757d90 con 0x560a0e841c00 2024-05-27T08:00:55.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.538+0000 7f2032000700 10 client.17889 handle_command_reply: tid=0 2024-05-27T08:00:55.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.538+0000 7f205db94b80 1 client.17889 shutdown 2024-05-27T08:00:55.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.539+0000 7f205db94b80 1 -- 172.21.15.90:0/2354467293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 msgr2=0x560a0e831740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.539+0000 7f205db94b80 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e831740 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f20380a6720 tx=0x7f2038092330 comp rx=0 tx=0).stop 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.539+0000 7f205db94b80 1 -- 172.21.15.90:0/2354467293 shutdown_connections 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.539+0000 7f205db94b80 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560a0e841c00 0x560a0e8440b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.539+0000 7f205db94b80 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e827e70 0x560a0e8321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.539+0000 7f205db94b80 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e6a8e40 0x560a0e831740 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.539+0000 7f205db94b80 1 --2- 172.21.15.90:0/2354467293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e831c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.539+0000 7f205db94b80 1 -- 172.21.15.90:0/2354467293 >> 172.21.15.90:0/2354467293 conn(0x560a0e742390 msgr2=0x560a0e8273b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.539+0000 7f2032000700 0 client.17889 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:55.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.540+0000 7f205db94b80 1 -- 172.21.15.90:0/2354467293 shutdown_connections 2024-05-27T08:00:55.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.540+0000 7f205db94b80 1 -- 172.21.15.90:0/2354467293 wait complete. 2024-05-27T08:00:55.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.543+0000 7f205db94b80 20 client.17889 trim_cache size 0 max 16384 2024-05-27T08:00:55.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:55 smithi129 ceph-mon[35254]: pgmap v465: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 615 op/s 2024-05-27T08:00:55.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.584+0000 7f205db94b80 1 Processor -- start 2024-05-27T08:00:55.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.584+0000 7f205db94b80 1 -- start start 2024-05-27T08:00:55.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.585+0000 7f205db94b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 0x560a0e82cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.585+0000 7f205db94b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e83dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.585+0000 7f205db94b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e827e70 0x560a0e82b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.585+0000 7f205db94b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2028010810 con 0x560a0e827e70 2024-05-27T08:00:55.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.585+0000 7f205db94b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f202800d860 con 0x560a0e7949f0 2024-05-27T08:00:55.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.585+0000 7f205db94b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f202800fa70 con 0x560a0e6a8e40 2024-05-27T08:00:55.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.586+0000 7f2047e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 0x560a0e82cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.586+0000 7f2047400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e83dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.586+0000 7f2047400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e83dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:38394/0 (socket says 172.21.15.90:38394) 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.586+0000 7f2047400700 1 -- 172.21.15.90:0/4228136498 learned_addr learned my addr 172.21.15.90:0/4228136498 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.586+0000 7f204d000700 1 --2- 172.21.15.90:0/4228136498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e827e70 0x560a0e82b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.587+0000 7f204d000700 1 -- 172.21.15.90:0/4228136498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 msgr2=0x560a0e82cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.587+0000 7f204d000700 1 --2- 172.21.15.90:0/4228136498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 0x560a0e82cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.587+0000 7f204d000700 1 -- 172.21.15.90:0/4228136498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 msgr2=0x560a0e83dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.587+0000 7f204d000700 1 --2- 172.21.15.90:0/4228136498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e83dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.587+0000 7f204d000700 1 -- 172.21.15.90:0/4228136498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a0e827830 con 0x560a0e827e70 2024-05-27T08:00:55.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.588+0000 7f204d000700 1 --2- 172.21.15.90:0/4228136498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e827e70 0x560a0e82b5f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f20400099b0 tx=0x7f20400099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5722a03de82f877 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.588+0000 7f203cc00700 1 -- 172.21.15.90:0/4228136498 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204001a070 con 0x560a0e827e70 2024-05-27T08:00:55.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.588+0000 7f203cc00700 1 -- 172.21.15.90:0/4228136498 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2040004db0 con 0x560a0e827e70 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.589+0000 7f203cc00700 1 -- 172.21.15.90:0/4228136498 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20400079d0 con 0x560a0e827e70 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.590+0000 7f205db94b80 1 -- 172.21.15.90:0/4228136498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e827e70 msgr2=0x560a0e82b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.590+0000 7f205db94b80 1 --2- 172.21.15.90:0/4228136498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e827e70 0x560a0e82b5f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f20400099b0 tx=0x7f20400099e0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.591+0000 7f205db94b80 1 -- 172.21.15.90:0/4228136498 shutdown_connections 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.591+0000 7f205db94b80 1 --2- 172.21.15.90:0/4228136498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e827e70 0x560a0e82b5f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.591+0000 7f205db94b80 1 --2- 172.21.15.90:0/4228136498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 0x560a0e82cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.591+0000 7f205db94b80 1 --2- 172.21.15.90:0/4228136498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e7949f0 0x560a0e83dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.591+0000 7f205db94b80 1 -- 172.21.15.90:0/4228136498 >> 172.21.15.90:0/4228136498 conn(0x560a0e742390 msgr2=0x560a0e82aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.592+0000 7f205db94b80 1 -- 172.21.15.90:0/4228136498 shutdown_connections 2024-05-27T08:00:55.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.593+0000 7f205db94b80 1 -- 172.21.15.90:0/4228136498 wait complete. 2024-05-27T08:00:55.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.594+0000 7f205db94b80 1 -- start start 2024-05-27T08:00:55.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.594+0000 7f205db94b80 1 Processor -- start 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.595+0000 7f205db94b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 0x560a0e83a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.595+0000 7f205db94b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e7949f0 0x560a0e8355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.595+0000 7f205db94b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e827e70 0x560a0e835b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.595+0000 7f205db94b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f202800f3a0 con 0x560a0e7949f0 2024-05-27T08:00:55.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.595+0000 7f205db94b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f202800df30 con 0x560a0e827e70 2024-05-27T08:00:55.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.595+0000 7f205db94b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f202800e600 con 0x560a0e6a8e40 2024-05-27T08:00:55.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.595+0000 7f205db94b80 10 client.0 _ll_register_callbacks cb 0x560a0e6efc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:00:55.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.597+0000 7f2047e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 0x560a0e83a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.597+0000 7f2047400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e7949f0 0x560a0e8355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.597+0000 7f2047e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 0x560a0e83a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:32892/0 (socket says 172.21.15.90:32892) 2024-05-27T08:00:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.597+0000 7f2047e00700 1 -- 172.21.15.90:0/3112810172 learned_addr learned my addr 172.21.15.90:0/3112810172 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:00:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.597+0000 7f204d000700 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e827e70 0x560a0e835b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.598+0000 7f2047e00700 1 -- 172.21.15.90:0/3112810172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e827e70 msgr2=0x560a0e835b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.598+0000 7f2047e00700 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e827e70 0x560a0e835b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.598+0000 7f2047e00700 1 -- 172.21.15.90:0/3112810172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e7949f0 msgr2=0x560a0e8355c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:00:55.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.598+0000 7f2047e00700 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e7949f0 0x560a0e8355c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.598+0000 7f2047e00700 1 -- 172.21.15.90:0/3112810172 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a0e836040 con 0x560a0e6a8e40 2024-05-27T08:00:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.598+0000 7f2047e00700 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 0x560a0e83a260 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f20380aa630 tx=0x7f2038009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3dc237f63cfeb29 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:00:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.598+0000 7f2032000700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:00:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.599+0000 7f2032000700 1 -- 172.21.15.90:0/3112810172 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20380c3070 con 0x560a0e6a8e40 2024-05-27T08:00:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.599+0000 7f2032000700 1 -- 172.21.15.90:0/3112810172 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7f20380bfaf0 con 0x560a0e6a8e40 2024-05-27T08:00:55.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.599+0000 7f205db94b80 1 -- 172.21.15.90:0/3112810172 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2040004db0 con 0x560a0e6a8e40 2024-05-27T08:00:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.600+0000 7f2032000700 1 -- 172.21.15.90:0/3112810172 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f20380c9030 con 0x560a0e6a8e40 2024-05-27T08:00:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.600+0000 7f205db94b80 10 client.27616 fetch_fsmap learned FSMap version 64 2024-05-27T08:00:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.600+0000 7f205db94b80 1 -- 172.21.15.90:0/3112810172 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560a0e8ac3e0 con 0x560a0e6a8e40 2024-05-27T08:00:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.601+0000 7f2032000700 1 -- 172.21.15.90:0/3112810172 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2040004db0 con 0x560a0e6a8e40 2024-05-27T08:00:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.602+0000 7f2032000700 1 -- 172.21.15.90:0/3112810172 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f20380a6010 con 0x560a0e6a8e40 2024-05-27T08:00:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.602+0000 7f205db94b80 10 client.27616 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:00:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.602+0000 7f205db94b80 10 client.27616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:00:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.602+0000 7f205db94b80 20 client.27616 populate_metadata read hostname 'smithi090' 2024-05-27T08:00:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.602+0000 7f205db94b80 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560a0e8ac950 0x560a0e8aed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:00:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.602+0000 7f205db94b80 4 client.27616 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:00:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.602+0000 7f205db94b80 1 -- 172.21.15.90:0/3112810172 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a0e751570 con 0x560a0e8ac950 2024-05-27T08:00:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.603+0000 7f2047400700 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560a0e8ac950 0x560a0e8aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:00:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.604+0000 7f2047400700 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560a0e8ac950 0x560a0e8aed70 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb1cc483aef3c21 server_cookie=ae16ae70edd4ff6d in_seq=0 out_seq=0 2024-05-27T08:00:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.604+0000 7f2032000700 10 client.27616 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.605+0000 7f2032000700 1 -- 172.21.15.90:0/3112810172 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560a0e751570 con 0x560a0e8ac950 2024-05-27T08:00:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.605+0000 7f2032000700 10 client.27616 handle_command_reply: tid=0 2024-05-27T08:00:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.605+0000 7f205db94b80 1 client.27616 shutdown 2024-05-27T08:00:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.605+0000 7f205db94b80 1 -- 172.21.15.90:0/3112810172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 msgr2=0x560a0e83a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.605+0000 7f205db94b80 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 0x560a0e83a260 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f20380aa630 tx=0x7f2038009620 comp rx=0 tx=0).stop 2024-05-27T08:00:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.606+0000 7f205db94b80 1 -- 172.21.15.90:0/3112810172 shutdown_connections 2024-05-27T08:00:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.606+0000 7f205db94b80 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560a0e8ac950 0x560a0e8aed70 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.606+0000 7f205db94b80 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a0e827e70 0x560a0e835b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.606+0000 7f205db94b80 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560a0e6a8e40 0x560a0e83a260 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.606+0000 7f205db94b80 1 --2- 172.21.15.90:0/3112810172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a0e7949f0 0x560a0e8355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.606+0000 7f205db94b80 1 -- 172.21.15.90:0/3112810172 >> 172.21.15.90:0/3112810172 conn(0x560a0e742390 msgr2=0x560a0e832bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.606+0000 7f2032000700 0 client.27616 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:00:55.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.606+0000 7f205db94b80 1 -- 172.21.15.90:0/3112810172 shutdown_connections 2024-05-27T08:00:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.608+0000 7f205db94b80 1 -- 172.21.15.90:0/3112810172 wait complete. 2024-05-27T08:00:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.608+0000 7f205db94b80 20 client.27616 trim_cache size 0 max 16384 2024-05-27T08:00:55.611 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:00:55.611 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T08:00:55.611 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T08:00:55.611 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:00:55.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.609+0000 7f204f000700 1 -- 172.21.15.90:0/4162265604 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f202c0b2ca0 msgr2=0x7f202c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.609+0000 7f204f000700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f202c0b2ca0 0x7f202c0b51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20340098e0 tx=0x7f2034000bc0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.609+0000 7f204f000700 1 -- 172.21.15.90:0/4162265604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20481045e0 msgr2=0x7f2048118b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:00:55.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.609+0000 7f204f000700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20481045e0 0x7f2048118b50 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f203800e9a0 tx=0x7f203800ed60 comp rx=0 tx=0).stop 2024-05-27T08:00:55.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.610+0000 7f204f000700 1 -- 172.21.15.90:0/4162265604 shutdown_connections 2024-05-27T08:00:55.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.610+0000 7f204f000700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f202c0b2ca0 0x7f202c0b51b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.610+0000 7f204f000700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2048109860 0x7f2048113d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.610+0000 7f204f000700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20481045e0 0x7f2048118b50 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.610+0000 7f204f000700 1 --2- 172.21.15.90:0/4162265604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2048108f10 0x7f20481190b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:00:55.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.610+0000 7f204f000700 1 -- 172.21.15.90:0/4162265604 >> 172.21.15.90:0/4162265604 conn(0x7f2048100030 msgr2=0x7f20481067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:00:55.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.610+0000 7f204f000700 1 -- 172.21.15.90:0/4162265604 shutdown_connections 2024-05-27T08:00:55.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:00:55.610+0000 7f204f000700 1 -- 172.21.15.90:0/4162265604 wait complete. 2024-05-27T08:00:56.787 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:00:56.788 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:00:57.522 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:00:57.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:57 smithi129 ceph-mon[35254]: pgmap v466: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.1 MiB/s wr, 494 op/s 2024-05-27T08:00:57.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:57 smithi090 ceph-mon[31429]: pgmap v466: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.1 MiB/s wr, 494 op/s 2024-05-27T08:00:57.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:57 smithi136 ceph-mon[35324]: pgmap v466: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.1 MiB/s wr, 494 op/s 2024-05-27T08:00:58.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 640.0 s, 379.4 tps, lat 265.089 ms stddev 343.164 2024-05-27T08:00:59.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:00:59 smithi090 ceph-mon[31429]: pgmap v467: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.7 MiB/s wr, 535 op/s 2024-05-27T08:00:59.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:00:59 smithi129 ceph-mon[35254]: pgmap v467: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.7 MiB/s wr, 535 op/s 2024-05-27T08:00:59.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:00:59 smithi136 ceph-mon[35324]: pgmap v467: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.7 MiB/s wr, 535 op/s 2024-05-27T08:01:01.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:01 smithi090 ceph-mon[31429]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.3 MiB/s wr, 675 op/s 2024-05-27T08:01:01.528 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:01 smithi129 ceph-mon[35254]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.3 MiB/s wr, 675 op/s 2024-05-27T08:01:01.690 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:01 smithi136 ceph-mon[35324]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.3 MiB/s wr, 675 op/s 2024-05-27T08:01:03.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:03 smithi129 ceph-mon[35254]: pgmap v469: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.4 MiB/s wr, 480 op/s 2024-05-27T08:01:03.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:03 smithi090 ceph-mon[31429]: pgmap v469: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.4 MiB/s wr, 480 op/s 2024-05-27T08:01:03.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:03 smithi136 ceph-mon[35324]: pgmap v469: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.4 MiB/s wr, 480 op/s 2024-05-27T08:01:03.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 645.0 s, 389.4 tps, lat 210.317 ms stddev 280.504 2024-05-27T08:01:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.097+0000 7f06b5200700 1 -- 172.21.15.90:0/1845973816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b0116ec0 msgr2=0x7f06b01193d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.097+0000 7f06b5200700 1 --2- 172.21.15.90:0/1845973816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b0116ec0 0x7f06b01193d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f069c009990 tx=0x7f069c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:01:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.098+0000 7f06b5200700 1 -- 172.21.15.90:0/1845973816 shutdown_connections 2024-05-27T08:01:04.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.098+0000 7f06b5200700 1 --2- 172.21.15.90:0/1845973816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b0116ec0 0x7f06b01193d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:04.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.098+0000 7f06b5200700 1 --2- 172.21.15.90:0/1845973816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06b010aa40 0x7f06b010ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:04.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.098+0000 7f06b5200700 1 --2- 172.21.15.90:0/1845973816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06b010b390 0x7f06b0116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:04.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.098+0000 7f06b5200700 1 -- 172.21.15.90:0/1845973816 >> 172.21.15.90:0/1845973816 conn(0x7f06b006f280 msgr2=0x7f06b006f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:04.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.098+0000 7f06b5200700 1 -- 172.21.15.90:0/1845973816 shutdown_connections 2024-05-27T08:01:04.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.099+0000 7f06b5200700 1 -- 172.21.15.90:0/1845973816 wait complete. 2024-05-27T08:01:04.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.102+0000 7f06b5200700 1 Processor -- start 2024-05-27T08:01:04.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.104+0000 7f06b5200700 1 -- start start 2024-05-27T08:01:04.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.105+0000 7f06b5200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06b010aa40 0x7f06b01a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:04.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.105+0000 7f06b5200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06b010b390 0x7f06b01a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:04.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.105+0000 7f06b5200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b0116ec0 0x7f06b01a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:04.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.106+0000 7f06ade00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06b010aa40 0x7f06b01a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:04.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.106+0000 7f06b5200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f06b011a7f0 con 0x7f06b010aa40 2024-05-27T08:01:04.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.106+0000 7f06b5200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06b011a670 con 0x7f06b0116ec0 2024-05-27T08:01:04.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.106+0000 7f06b5200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06b011a970 con 0x7f06b010b390 2024-05-27T08:01:04.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.106+0000 7f06ade00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06b010aa40 0x7f06b01a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43872/0 (socket says 172.21.15.90:43872) 2024-05-27T08:01:04.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.106+0000 7f06ade00700 1 -- 172.21.15.90:0/3726869863 learned_addr learned my addr 172.21.15.90:0/3726869863 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:04.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.106+0000 7f06ae800700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b0116ec0 0x7f06b01a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:04.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.106+0000 7f06ad400700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06b010b390 0x7f06b01a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:04.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.107+0000 7f06ae800700 1 -- 172.21.15.90:0/3726869863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06b010b390 msgr2=0x7f06b01a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:04.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.109+0000 7f06ae800700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06b010b390 0x7f06b01a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:04.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.109+0000 7f06ae800700 1 -- 172.21.15.90:0/3726869863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06b010aa40 msgr2=0x7f06b01a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:04.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.109+0000 7f06ae800700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06b010aa40 0x7f06b01a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:04.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.109+0000 7f06ae800700 1 -- 172.21.15.90:0/3726869863 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b01a5f40 con 0x7f06b0116ec0 2024-05-27T08:01:04.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.109+0000 7f06ade00700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06b010aa40 0x7f06b01a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:01:04.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.109+0000 7f06ad400700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06b010b390 0x7f06b01a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:01:04.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.109+0000 7f06ae800700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b0116ec0 0x7f06b01a5950 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f069c009990 tx=0x7f069c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:04.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.110+0000 7f06a6a00700 1 -- 172.21.15.90:0/3726869863 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c026070 con 0x7f06b0116ec0 2024-05-27T08:01:04.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.110+0000 7f06b5200700 1 -- 172.21.15.90:0/3726869863 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b01a61d0 con 0x7f06b0116ec0 2024-05-27T08:01:04.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.110+0000 7f06b5200700 1 -- 172.21.15.90:0/3726869863 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b01a6700 con 0x7f06b0116ec0 2024-05-27T08:01:04.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.110+0000 7f06a6a00700 1 -- 172.21.15.90:0/3726869863 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f069c00fbc0 con 0x7f06b0116ec0 2024-05-27T08:01:04.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.110+0000 7f06a6a00700 1 -- 172.21.15.90:0/3726869863 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c021b30 con 0x7f06b0116ec0 2024-05-27T08:01:04.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.112+0000 7f06b5200700 1 -- 172.21.15.90:0/3726869863 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0690005330 con 0x7f06b0116ec0 2024-05-27T08:01:04.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.120+0000 7f06a6a00700 1 -- 172.21.15.90:0/3726869863 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f069c00f730 con 0x7f06b0116ec0 2024-05-27T08:01:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.122+0000 7f06a6a00700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f06940b2aa0 0x7f06940b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.123+0000 7f06ade00700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f06940b2aa0 0x7f06940b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.124+0000 7f06a6a00700 1 -- 172.21.15.90:0/3726869863 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7f069c0e3fb0 con 0x7f06b0116ec0 2024-05-27T08:01:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.124+0000 7f06ade00700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f06940b2aa0 0x7f06940b4fb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f06a80060b0 tx=0x7f06a8009040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:04.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.125+0000 7f06a6a00700 1 -- 172.21.15.90:0/3726869863 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f069c02f070 con 0x7f06b0116ec0 2024-05-27T08:01:04.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.637+0000 7f06b5200700 1 -- 172.21.15.90:0/3726869863 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0690005620 con 0x7f06b0116ec0 2024-05-27T08:01:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:01:04.642 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":64,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:56:32.161929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T08:01:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.639+0000 7f06a6a00700 1 -- 172.21.15.90:0/3726869863 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+9456 (secure 0 0 0) 0x7f069c013070 con 0x7f06b0116ec0 2024-05-27T08:01:04.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.643+0000 7f06b5200700 1 -- 172.21.15.90:0/3726869863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f06940b2aa0 msgr2=0x7f06940b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:04.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.643+0000 7f06b5200700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f06940b2aa0 0x7f06940b4fb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f06a80060b0 tx=0x7f06a8009040 comp rx=0 tx=0).stop 2024-05-27T08:01:04.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.643+0000 7f06b5200700 1 -- 172.21.15.90:0/3726869863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b0116ec0 msgr2=0x7f06b01a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:04.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.643+0000 7f06b5200700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b0116ec0 0x7f06b01a5950 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f069c009990 tx=0x7f069c009580 comp rx=0 tx=0).stop 2024-05-27T08:01:04.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.643+0000 7f06b5200700 1 -- 172.21.15.90:0/3726869863 shutdown_connections 2024-05-27T08:01:04.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.643+0000 7f06b5200700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f06940b2aa0 0x7f06940b4fb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:04.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.643+0000 7f06b5200700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06b0116ec0 0x7f06b01a5950 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:04.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.643+0000 7f06b5200700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06b010aa40 0x7f06b01a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:04.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.643+0000 7f06b5200700 1 --2- 172.21.15.90:0/3726869863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06b010b390 0x7f06b01a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:04.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.644+0000 7f06b5200700 1 -- 172.21.15.90:0/3726869863 >> 172.21.15.90:0/3726869863 conn(0x7f06b006f280 msgr2=0x7f06b0072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:04.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.644+0000 7f06b5200700 1 -- 172.21.15.90:0/3726869863 shutdown_connections 2024-05-27T08:01:04.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:04.644+0000 7f06b5200700 1 -- 172.21.15.90:0/3726869863 wait complete. 2024-05-27T08:01:04.649 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 64 2024-05-27T08:01:05.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:05 smithi129 ceph-mon[35254]: pgmap v470: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.8 MiB/s wr, 696 op/s 2024-05-27T08:01:05.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:05 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3726869863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:01:05.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:05 smithi090 ceph-mon[31429]: pgmap v470: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.8 MiB/s wr, 696 op/s 2024-05-27T08:01:05.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:05 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3726869863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:01:05.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:05 smithi136 ceph-mon[35324]: pgmap v470: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.8 MiB/s wr, 696 op/s 2024-05-27T08:01:05.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:05 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3726869863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:01:07.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:07 smithi129 ceph-mon[35254]: pgmap v471: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 587 op/s 2024-05-27T08:01:07.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:07 smithi090 ceph-mon[31429]: pgmap v471: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 587 op/s 2024-05-27T08:01:07.713 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T08:01:07.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:07 smithi136 ceph-mon[35324]: pgmap v471: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.4 MiB/s wr, 587 op/s 2024-05-27T08:01:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.172+0000 7fc672400640 1 -- 172.21.15.90:0/933080756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6640a1390 msgr2=0x7fc6640a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.172+0000 7fc672400640 1 --2- 172.21.15.90:0/933080756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6640a1390 0x7fc6640a37b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc66c051db0 tx=0x7fc66c097ae0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.174+0000 7fc672400640 1 -- 172.21.15.90:0/933080756 shutdown_connections 2024-05-27T08:01:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.174+0000 7fc672400640 1 --2- 172.21.15.90:0/933080756 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6640a1390 0x7fc6640a37b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.174+0000 7fc672400640 1 --2- 172.21.15.90:0/933080756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc664096b00 0x7fc664096f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.174+0000 7fc672400640 1 --2- 172.21.15.90:0/933080756 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc66409ca30 0x7fc6640965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.174+0000 7fc672400640 1 -- 172.21.15.90:0/933080756 >> 172.21.15.90:0/933080756 conn(0x7fc66400c120 msgr2=0x7fc66400c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.174+0000 7fc672400640 1 -- 172.21.15.90:0/933080756 shutdown_connections 2024-05-27T08:01:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.174+0000 7fc672400640 1 -- 172.21.15.90:0/933080756 wait complete. 2024-05-27T08:01:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.177+0000 7fc672400640 1 Processor -- start 2024-05-27T08:01:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.177+0000 7fc672400640 1 -- start start 2024-05-27T08:01:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.179+0000 7fc672400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc664096b00 0x7fc66412bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.179+0000 7fc672400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc66409ca30 0x7fc66412c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.180+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc66409ca30 0x7fc66412c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.180+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc66409ca30 0x7fc66412c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43888/0 (socket says 172.21.15.90:43888) 2024-05-27T08:01:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.180+0000 7fc672400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6640a1390 0x7fc66412c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.180+0000 7fc66aa00640 1 -- 172.21.15.90:0/1536120749 learned_addr learned my addr 172.21.15.90:0/1536120749 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.181+0000 7fc66be00640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6640a1390 0x7fc66412c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.181+0000 7fc66b400640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc664096b00 0x7fc66412bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.181+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc6640a6a80 con 0x7fc66409ca30 2024-05-27T08:01:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.181+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6640a6900 con 0x7fc664096b00 2024-05-27T08:01:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.181+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc6640a6c00 con 0x7fc6640a1390 2024-05-27T08:01:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.182+0000 7fc66be00640 1 -- 172.21.15.90:0/1536120749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc664096b00 msgr2=0x7fc66412bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.182+0000 7fc66be00640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc664096b00 0x7fc66412bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.182+0000 7fc66be00640 1 -- 172.21.15.90:0/1536120749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc66409ca30 msgr2=0x7fc66412c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.182+0000 7fc66be00640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc66409ca30 0x7fc66412c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.182+0000 7fc66be00640 1 -- 172.21.15.90:0/1536120749 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66413fbe0 con 0x7fc6640a1390 2024-05-27T08:01:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.182+0000 7fc66aa00640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc66409ca30 0x7fc66412c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.183+0000 7fc66be00640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6640a1390 0x7fc66412c9d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc66c068820 tx=0x7fc66c06b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.183+0000 7fc65fe00640 1 -- 172.21.15.90:0/1536120749 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c09f070 con 0x7fc6640a1390 2024-05-27T08:01:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.183+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66413fe70 con 0x7fc6640a1390 2024-05-27T08:01:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.184+0000 7fc65fe00640 1 -- 172.21.15.90:0/1536120749 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc66c09bd10 con 0x7fc6640a1390 2024-05-27T08:01:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.184+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6641403b0 con 0x7fc6640a1390 2024-05-27T08:01:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.184+0000 7fc65fe00640 1 -- 172.21.15.90:0/1536120749 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0a5e10 con 0x7fc6640a1390 2024-05-27T08:01:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.192+0000 7fc67938b740 1 Processor -- start 2024-05-27T08:01:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.193+0000 7fc65fe00640 1 -- 172.21.15.90:0/1536120749 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc66c0a2020 con 0x7fc6640a1390 2024-05-27T08:01:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.193+0000 7fc67938b740 1 -- start start 2024-05-27T08:01:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.194+0000 7fc67938b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3241b0 0x55f0fb6c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.194+0000 7fc67938b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.194+0000 7fc67938b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb3034d0 0x55f0fb6c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.194+0000 7fc67938b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f0fb637b90 con 0x55f0fb3241b0 2024-05-27T08:01:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.194+0000 7fc67938b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f0fb627730 con 0x55f0fb2fb2f0 2024-05-27T08:01:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.194+0000 7fc67938b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f0fb62b770 con 0x55f0fb3034d0 2024-05-27T08:01:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.196+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3241b0 0x55f0fb6c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.196+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3241b0 0x55f0fb6c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43904/0 (socket says 172.21.15.90:43904) 2024-05-27T08:01:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.196+0000 7fc66aa00640 1 -- 172.21.15.90:0/1099131914 learned_addr learned my addr 172.21.15.90:0/1099131914 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.197+0000 7fc66aa00640 1 --2- 172.21.15.90:0/1099131914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb3034d0 0x55f0fb6c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.197+0000 7fc66b400640 1 --2- 172.21.15.90:0/1099131914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.197+0000 7fc66aa00640 1 -- 172.21.15.90:0/1099131914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb3034d0 msgr2=0x55f0fb6c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.198+0000 7fc66aa00640 1 --2- 172.21.15.90:0/1099131914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb3034d0 0x55f0fb6c7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.198+0000 7fc66aa00640 1 -- 172.21.15.90:0/1099131914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb2fb2f0 msgr2=0x55f0fb6c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.198+0000 7fc66aa00640 1 --2- 172.21.15.90:0/1099131914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6c6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.198+0000 7fc66aa00640 1 -- 172.21.15.90:0/1099131914 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0fb5c08b0 con 0x55f0fb3241b0 2024-05-27T08:01:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.198+0000 7fc66b400640 1 --2- 172.21.15.90:0/1099131914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6c6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.200+0000 7fc65fe00640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6540b2a10 0x7fc6540b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.200+0000 7fc65fe00640 1 -- 172.21.15.90:0/1536120749 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7fc66c16fc00 con 0x7fc6640a1390 2024-05-27T08:01:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.200+0000 7fc66aa00640 1 --2- 172.21.15.90:0/1099131914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3241b0 0x55f0fb6c6a90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc6600135e0 tx=0x7fc6600097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9424efd7b9958ff server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.201+0000 7fc66b400640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6540b2a10 0x7fc6540b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.202+0000 7fc65cc00640 1 -- 172.21.15.90:0/1099131914 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660005020 con 0x55f0fb3241b0 2024-05-27T08:01:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.202+0000 7fc65cc00640 1 -- 172.21.15.90:0/1099131914 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6600047a0 con 0x55f0fb3241b0 2024-05-27T08:01:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.202+0000 7fc65cc00640 1 -- 172.21.15.90:0/1099131914 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660005650 con 0x55f0fb3241b0 2024-05-27T08:01:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.203+0000 7fc66b400640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6540b2a10 0x7fc6540b4f20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55f0fb6c7e40 tx=0x7fc658008040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.204+0000 7fc67938b740 1 -- 172.21.15.90:0/1099131914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3241b0 msgr2=0x55f0fb6c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.204+0000 7fc67938b740 1 --2- 172.21.15.90:0/1099131914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3241b0 0x55f0fb6c6a90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc6600135e0 tx=0x7fc6600097e0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.205+0000 7fc67938b740 1 -- 172.21.15.90:0/1099131914 shutdown_connections 2024-05-27T08:01:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.205+0000 7fc67938b740 1 --2- 172.21.15.90:0/1099131914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6c6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.205+0000 7fc67938b740 1 --2- 172.21.15.90:0/1099131914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb3034d0 0x55f0fb6c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.205+0000 7fc67938b740 1 --2- 172.21.15.90:0/1099131914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3241b0 0x55f0fb6c6a90 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.205+0000 7fc67938b740 1 -- 172.21.15.90:0/1099131914 >> 172.21.15.90:0/1099131914 conn(0x55f0fb318300 msgr2=0x55f0fb6c3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.206+0000 7fc67938b740 1 -- 172.21.15.90:0/1099131914 shutdown_connections 2024-05-27T08:01:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.206+0000 7fc67938b740 1 -- 172.21.15.90:0/1099131914 wait complete. 2024-05-27T08:01:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.207+0000 7fc67938b740 1 -- start start 2024-05-27T08:01:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.207+0000 7fc67938b740 1 Processor -- start 2024-05-27T08:01:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.208+0000 7fc67938b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.209+0000 7fc67938b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 0x55f0fb6d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.209+0000 7fc66b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.209+0000 7fc66b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43906/0 (socket says 172.21.15.90:43906) 2024-05-27T08:01:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.210+0000 7fc66b400640 1 -- 172.21.15.90:0/552350481 learned_addr learned my addr 172.21.15.90:0/552350481 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.211+0000 7fc67938b740 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb3241b0 0x55f0fb6edf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.211+0000 7fc66aa00640 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 0x55f0fb6d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.211+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f0fb30d790 con 0x55f0fb2fb2f0 2024-05-27T08:01:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.211+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f0fb5cfbc0 con 0x55f0fb3034d0 2024-05-27T08:01:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.211+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f0fb6e4bd0 con 0x55f0fb3241b0 2024-05-27T08:01:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.211+0000 7fc67938b740 10 client.0 _ll_register_callbacks cb 0x55f0fb5ad400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:08.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.212+0000 7fc66be00640 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb3241b0 0x55f0fb6edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.213+0000 7fc66b400640 1 -- 172.21.15.90:0/552350481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb3241b0 msgr2=0x55f0fb6edf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.213+0000 7fc66b400640 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb3241b0 0x55f0fb6edf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.213+0000 7fc66b400640 1 -- 172.21.15.90:0/552350481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 msgr2=0x55f0fb6d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.213+0000 7fc66b400640 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 0x55f0fb6d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.213+0000 7fc66b400640 1 -- 172.21.15.90:0/552350481 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0fb33c210 con 0x55f0fb2fb2f0 2024-05-27T08:01:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.215+0000 7fc66b400640 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6d0f50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc6580156a0 tx=0x7fc6580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d56a0c8cf53c209 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.215+0000 7fc651600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:01:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.217+0000 7fc651600640 1 -- 172.21.15.90:0/552350481 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc658025020 con 0x55f0fb2fb2f0 2024-05-27T08:01:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.217+0000 7fc651600640 1 -- 172.21.15.90:0/552350481 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7fc658021440 con 0x55f0fb2fb2f0 2024-05-27T08:01:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.217+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6600047a0 con 0x55f0fb2fb2f0 2024-05-27T08:01:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.220+0000 7fc651600640 1 -- 172.21.15.90:0/552350481 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc658021640 con 0x55f0fb2fb2f0 2024-05-27T08:01:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.222+0000 7fc67938b740 10 client.17904 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.222+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f0fb408620 con 0x55f0fb2fb2f0 2024-05-27T08:01:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.224+0000 7fc651600640 1 -- 172.21.15.90:0/552350481 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6600047a0 con 0x55f0fb2fb2f0 2024-05-27T08:01:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.224+0000 7fc651600640 1 -- 172.21.15.90:0/552350481 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fc658014960 con 0x55f0fb2fb2f0 2024-05-27T08:01:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.224+0000 7fc67938b740 10 client.17904 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.224+0000 7fc67938b740 10 client.17904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.224+0000 7fc67938b740 20 client.17904 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.224+0000 7fc67938b740 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f0fb2e2d80 0x55f0fb6f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.224+0000 7fc67938b740 4 client.17904 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.224+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0fb639ae0 con 0x55f0fb2e2d80 2024-05-27T08:01:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.225+0000 7fc66aa00640 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f0fb2e2d80 0x55f0fb6f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.228+0000 7fc651600640 10 client.17904 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.229+0000 7fc66aa00640 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f0fb2e2d80 0x55f0fb6f4340 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fae6d31ea4e95ca server_cookie=279e4f877675ff9b in_seq=0 out_seq=0 2024-05-27T08:01:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.232+0000 7fc651600640 1 -- 172.21.15.90:0/552350481 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f0fb639ae0 con 0x55f0fb2e2d80 2024-05-27T08:01:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.232+0000 7fc651600640 10 client.17904 handle_command_reply: tid=0 2024-05-27T08:01:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.232+0000 7fc67938b740 1 client.17904 shutdown 2024-05-27T08:01:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.233+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb2fb2f0 msgr2=0x55f0fb6d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.233+0000 7fc67938b740 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6d0f50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc6580156a0 tx=0x7fc6580046c0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.233+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 shutdown_connections 2024-05-27T08:01:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.233+0000 7fc67938b740 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f0fb2e2d80 0x55f0fb6f4340 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.233+0000 7fc67938b740 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 0x55f0fb6d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.233+0000 7fc67938b740 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb3241b0 0x55f0fb6edf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.233+0000 7fc67938b740 1 --2- 172.21.15.90:0/552350481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb2fb2f0 0x55f0fb6d0f50 secure :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc6580156a0 tx=0x7fc6580046c0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.233+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 >> 172.21.15.90:0/552350481 conn(0x55f0fb318300 msgr2=0x55f0fb6c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.233+0000 7fc651600640 0 client.17904 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.235+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 shutdown_connections 2024-05-27T08:01:08.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.235+0000 7fc67938b740 1 -- 172.21.15.90:0/552350481 wait complete. 2024-05-27T08:01:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.237+0000 7fc67938b740 20 client.17904 trim_cache size 0 max 16384 2024-05-27T08:01:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.281+0000 7fc67938b740 1 Processor -- start 2024-05-27T08:01:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.281+0000 7fc67938b740 1 -- start start 2024-05-27T08:01:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.282+0000 7fc67938b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3034d0 0x55f0fb6cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.282+0000 7fc67938b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3241b0 0x55f0fb6edd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.282+0000 7fc67938b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 0x55f0fb6ee240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.282+0000 7fc67938b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c0119e0 con 0x55f0fb3034d0 2024-05-27T08:01:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.282+0000 7fc67938b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c00ea30 con 0x55f0fb3241b0 2024-05-27T08:01:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.282+0000 7fc67938b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c010c40 con 0x55f0fb2b81c0 2024-05-27T08:01:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.283+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3241b0 0x55f0fb6edd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.283+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3241b0 0x55f0fb6edd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:60030/0 (socket says 172.21.15.90:60030) 2024-05-27T08:01:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.283+0000 7fc66aa00640 1 -- 172.21.15.90:0/725242584 learned_addr learned my addr 172.21.15.90:0/725242584 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.283+0000 7fc66be00640 1 --2- 172.21.15.90:0/725242584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 0x55f0fb6ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.284+0000 7fc66aa00640 1 -- 172.21.15.90:0/725242584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 msgr2=0x55f0fb6ee240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.284+0000 7fc66aa00640 1 --2- 172.21.15.90:0/725242584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 0x55f0fb6ee240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.284+0000 7fc66aa00640 1 -- 172.21.15.90:0/725242584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3034d0 msgr2=0x55f0fb6cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.284+0000 7fc66aa00640 1 --2- 172.21.15.90:0/725242584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3034d0 0x55f0fb6cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.284+0000 7fc66aa00640 1 -- 172.21.15.90:0/725242584 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0fb6cf290 con 0x55f0fb3241b0 2024-05-27T08:01:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.284+0000 7fc66be00640 1 --2- 172.21.15.90:0/725242584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 0x55f0fb6ee240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.285+0000 7fc66aa00640 1 --2- 172.21.15.90:0/725242584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3241b0 0x55f0fb6edd00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc660000c00 tx=0x7fc660021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a729a8772342500e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.286+0000 7fc653e00640 1 -- 172.21.15.90:0/725242584 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660004040 con 0x55f0fb3241b0 2024-05-27T08:01:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.286+0000 7fc653e00640 1 -- 172.21.15.90:0/725242584 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc660005020 con 0x55f0fb3241b0 2024-05-27T08:01:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.286+0000 7fc653e00640 1 -- 172.21.15.90:0/725242584 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660004a00 con 0x55f0fb3241b0 2024-05-27T08:01:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.287+0000 7fc67938b740 1 -- 172.21.15.90:0/725242584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3241b0 msgr2=0x55f0fb6edd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.287+0000 7fc67938b740 1 --2- 172.21.15.90:0/725242584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3241b0 0x55f0fb6edd00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc660000c00 tx=0x7fc660021040 comp rx=0 tx=0).stop 2024-05-27T08:01:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.287+0000 7fc67938b740 1 -- 172.21.15.90:0/725242584 shutdown_connections 2024-05-27T08:01:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.287+0000 7fc67938b740 1 --2- 172.21.15.90:0/725242584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3241b0 0x55f0fb6edd00 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc660000c00 tx=0x7fc660021040 comp rx=0 tx=0).stop 2024-05-27T08:01:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.287+0000 7fc67938b740 1 --2- 172.21.15.90:0/725242584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 0x55f0fb6ee240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.287+0000 7fc67938b740 1 --2- 172.21.15.90:0/725242584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb3034d0 0x55f0fb6cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.287+0000 7fc67938b740 1 -- 172.21.15.90:0/725242584 >> 172.21.15.90:0/725242584 conn(0x55f0fb318300 msgr2=0x55f0fb6d2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.287+0000 7fc67938b740 1 -- 172.21.15.90:0/725242584 shutdown_connections 2024-05-27T08:01:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.288+0000 7fc67938b740 1 -- 172.21.15.90:0/725242584 wait complete. 2024-05-27T08:01:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.288+0000 7fc67938b740 1 -- start start 2024-05-27T08:01:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.289+0000 7fc67938b740 1 Processor -- start 2024-05-27T08:01:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.292+0000 7fc67938b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 0x55f0fb6e6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.292+0000 7fc67938b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 0x55f0fb6e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.292+0000 7fc67938b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb6e76d0 0x55f0fb75a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.292+0000 7fc67938b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c010570 con 0x55f0fb6e76d0 2024-05-27T08:01:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.293+0000 7fc67938b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c00f100 con 0x55f0fb3034d0 2024-05-27T08:01:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.293+0000 7fc67938b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c00f7d0 con 0x55f0fb2b81c0 2024-05-27T08:01:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.293+0000 7fc67938b740 10 client.0 _ll_register_callbacks cb 0x7fc6580093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.293+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 0x55f0fb6e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.293+0000 7fc66aa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 0x55f0fb6e7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:60036/0 (socket says 172.21.15.90:60036) 2024-05-27T08:01:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.293+0000 7fc66aa00640 1 -- 172.21.15.90:0/88519074 learned_addr learned my addr 172.21.15.90:0/88519074 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.293+0000 7fc66b400640 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 0x55f0fb6e6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.294+0000 7fc66aa00640 1 -- 172.21.15.90:0/88519074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 msgr2=0x55f0fb6e6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.294+0000 7fc66aa00640 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 0x55f0fb6e6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.294+0000 7fc66aa00640 1 -- 172.21.15.90:0/88519074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb6e76d0 msgr2=0x55f0fb75a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.294+0000 7fc66be00640 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb6e76d0 0x55f0fb75a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.294+0000 7fc66aa00640 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb6e76d0 0x55f0fb75a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.294+0000 7fc66aa00640 1 -- 172.21.15.90:0/88519074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0fb6eaaf0 con 0x55f0fb3034d0 2024-05-27T08:01:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.294+0000 7fc66be00640 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb6e76d0 0x55f0fb75a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:01:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.295+0000 7fc651600640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:01:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.295+0000 7fc66aa00640 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 0x55f0fb6e7190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc660015040 tx=0x7fc6600097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd806e983fbb7547 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.295+0000 7fc651600640 1 -- 172.21.15.90:0/88519074 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660004450 con 0x55f0fb3034d0 2024-05-27T08:01:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.295+0000 7fc651600640 1 -- 172.21.15.90:0/88519074 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9496+0+0 (secure 0 0 0) 0x7fc6600051d0 con 0x55f0fb3034d0 2024-05-27T08:01:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.295+0000 7fc67938b740 1 -- 172.21.15.90:0/88519074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc660005020 con 0x55f0fb3034d0 2024-05-27T08:01:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.296+0000 7fc651600640 1 -- 172.21.15.90:0/88519074 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc6600053d0 con 0x55f0fb3034d0 2024-05-27T08:01:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.296+0000 7fc67938b740 10 client.37541 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.296+0000 7fc67938b740 1 -- 172.21.15.90:0/88519074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f0fb75bfb0 con 0x55f0fb3034d0 2024-05-27T08:01:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.297+0000 7fc651600640 1 -- 172.21.15.90:0/88519074 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc660005020 con 0x55f0fb3034d0 2024-05-27T08:01:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.297+0000 7fc651600640 1 -- 172.21.15.90:0/88519074 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fc66003f030 con 0x55f0fb3034d0 2024-05-27T08:01:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.298+0000 7fc67938b740 10 client.37541 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.298+0000 7fc67938b740 10 client.37541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.298+0000 7fc67938b740 20 client.37541 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.298+0000 7fc67938b740 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f0fb75c540 0x55f0fb75e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.298+0000 7fc67938b740 4 client.37541 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:01:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.298+0000 7fc67938b740 1 -- 172.21.15.90:0/88519074 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0fb6332c0 con 0x55f0fb75c540 2024-05-27T08:01:08.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.299+0000 7fc66b400640 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f0fb75c540 0x55f0fb75e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.300+0000 7fc66b400640 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f0fb75c540 0x55f0fb75e960 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c58f8b438fca068 server_cookie=997af473c8a46e82 in_seq=0 out_seq=0 2024-05-27T08:01:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.300+0000 7fc651600640 10 client.37541 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.343+0000 7fc651600640 1 -- 172.21.15.90:0/88519074 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f0fb6332c0 con 0x55f0fb75c540 2024-05-27T08:01:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.343+0000 7fc651600640 10 client.37541 handle_command_reply: tid=0 2024-05-27T08:01:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.345+0000 7fc67938b740 1 client.37541 shutdown 2024-05-27T08:01:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.345+0000 7fc67938b740 1 -- 172.21.15.90:0/88519074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 msgr2=0x55f0fb6e7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.345+0000 7fc67938b740 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 0x55f0fb6e7190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc660015040 tx=0x7fc6600097b0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.345+0000 7fc67938b740 1 -- 172.21.15.90:0/88519074 shutdown_connections 2024-05-27T08:01:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.345+0000 7fc67938b740 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f0fb75c540 0x55f0fb75e960 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.346+0000 7fc67938b740 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f0fb3034d0 0x55f0fb6e7190 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.346+0000 7fc67938b740 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f0fb6e76d0 0x55f0fb75a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.346+0000 7fc67938b740 1 --2- 172.21.15.90:0/88519074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f0fb2b81c0 0x55f0fb6e6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.346+0000 7fc67938b740 1 -- 172.21.15.90:0/88519074 >> 172.21.15.90:0/88519074 conn(0x55f0fb318300 msgr2=0x55f0fb6d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.346+0000 7fc651600640 0 client.37541 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.346+0000 7fc67938b740 1 -- 172.21.15.90:0/88519074 shutdown_connections 2024-05-27T08:01:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:01:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:01:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:01:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.347+0000 7fc67938b740 1 -- 172.21.15.90:0/88519074 wait complete. 2024-05-27T08:01:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.348+0000 7fc67938b740 20 client.37541 trim_cache size 0 max 16384 2024-05-27T08:01:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.349+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6540b2a10 msgr2=0x7fc6540b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.350+0000 7fc672400640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6540b2a10 0x7fc6540b4f20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55f0fb6c7e40 tx=0x7fc658008040 comp rx=0 tx=0).stop 2024-05-27T08:01:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.350+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6640a1390 msgr2=0x7fc66412c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.350+0000 7fc672400640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6640a1390 0x7fc66412c9d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc66c068820 tx=0x7fc66c06b6d0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.350+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 shutdown_connections 2024-05-27T08:01:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.350+0000 7fc672400640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6540b2a10 0x7fc6540b4f20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.350+0000 7fc672400640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6640a1390 0x7fc66412c9d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.350+0000 7fc672400640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc66409ca30 0x7fc66412c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.350+0000 7fc672400640 1 --2- 172.21.15.90:0/1536120749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc664096b00 0x7fc66412bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.350+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 >> 172.21.15.90:0/1536120749 conn(0x7fc66400c120 msgr2=0x7fc6640079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.350+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 shutdown_connections 2024-05-27T08:01:08.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:08.351+0000 7fc672400640 1 -- 172.21.15.90:0/1536120749 wait complete. 2024-05-27T08:01:08.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 650.0 s, 466.8 tps, lat 252.166 ms stddev 381.313 2024-05-27T08:01:09.073 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:09.523 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:09 smithi090 ceph-mon[31429]: pgmap v472: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 546 op/s 2024-05-27T08:01:09.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:09 smithi129 ceph-mon[35254]: pgmap v472: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 546 op/s 2024-05-27T08:01:09.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:09 smithi136 ceph-mon[35324]: pgmap v472: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 546 op/s 2024-05-27T08:01:10.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:10 smithi090 ceph-mon[31429]: osdmap e117: 12 total, 12 up, 12 in 2024-05-27T08:01:10.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:10 smithi136 ceph-mon[35324]: osdmap e117: 12 total, 12 up, 12 in 2024-05-27T08:01:10.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:10 smithi129 ceph-mon[35254]: osdmap e117: 12 total, 12 up, 12 in 2024-05-27T08:01:11.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:11 smithi090 ceph-mon[31429]: pgmap v474: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 597 op/s 2024-05-27T08:01:11.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:11 smithi136 ceph-mon[35324]: pgmap v474: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 597 op/s 2024-05-27T08:01:11.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:11 smithi129 ceph-mon[35254]: pgmap v474: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 597 op/s 2024-05-27T08:01:13.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:13 smithi090 ceph-mon[31429]: pgmap v475: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 606 op/s 2024-05-27T08:01:13.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:13 smithi136 ceph-mon[35324]: pgmap v475: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 606 op/s 2024-05-27T08:01:13.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:13 smithi129 ceph-mon[35254]: pgmap v475: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.9 MiB/s wr, 606 op/s 2024-05-27T08:01:13.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 655.0 s, 137.4 tps, lat 604.421 ms stddev 811.515 2024-05-27T08:01:15.390 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:15 smithi090 ceph-mon[31429]: pgmap v476: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.0 MiB/s wr, 516 op/s 2024-05-27T08:01:15.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:15 smithi136 ceph-mon[35324]: pgmap v476: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.0 MiB/s wr, 516 op/s 2024-05-27T08:01:15.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:15 smithi129 ceph-mon[35254]: pgmap v476: 129 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 114 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.0 MiB/s wr, 516 op/s 2024-05-27T08:01:16.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.452+0000 7ff027200700 1 -- 172.21.15.90:0/4090541026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0280764a0 msgr2=0x7ff028114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:16.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.452+0000 7ff027200700 1 --2- 172.21.15.90:0/4090541026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0280764a0 0x7ff028114c30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff018009990 tx=0x7ff018009ca0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.452+0000 7ff027200700 1 -- 172.21.15.90:0/4090541026 shutdown_connections 2024-05-27T08:01:16.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.452+0000 7ff027200700 1 --2- 172.21.15.90:0/4090541026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff028115170 0x7ff028117590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.452+0000 7ff027200700 1 --2- 172.21.15.90:0/4090541026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff028075b50 0x7ff028075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.452+0000 7ff027200700 1 --2- 172.21.15.90:0/4090541026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0280764a0 0x7ff028114c30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.452+0000 7ff027200700 1 -- 172.21.15.90:0/4090541026 >> 172.21.15.90:0/4090541026 conn(0x7ff028079d00 msgr2=0x7ff02807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:16.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.453+0000 7ff027200700 1 -- 172.21.15.90:0/4090541026 shutdown_connections 2024-05-27T08:01:16.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.453+0000 7ff027200700 1 -- 172.21.15.90:0/4090541026 wait complete. 2024-05-27T08:01:16.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.457+0000 7ff027200700 1 Processor -- start 2024-05-27T08:01:16.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.459+0000 7ff027200700 1 -- start start 2024-05-27T08:01:16.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.460+0000 7ff027200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff028075b50 0x7ff028072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.460+0000 7ff027200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0280764a0 0x7ff028073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.460+0000 7ff027200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff028115170 0x7ff0281142a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.461+0000 7ff027200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff02810aa80 con 0x7ff0280764a0 2024-05-27T08:01:16.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.461+0000 7ff027200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff02810a900 con 0x7ff028075b50 2024-05-27T08:01:16.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.461+0000 7ff027200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff02810ac00 con 0x7ff028115170 2024-05-27T08:01:16.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.461+0000 7ff026800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff028115170 0x7ff0281142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.461+0000 7ff026800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff028115170 0x7ff0281142a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33466/0 (socket says 172.21.15.90:33466) 2024-05-27T08:01:16.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.461+0000 7ff026800700 1 -- 172.21.15.90:0/3464953129 learned_addr learned my addr 172.21.15.90:0/3464953129 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:16.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.461+0000 7ff025e00700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff028075b50 0x7ff028072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.462+0000 7ff025400700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0280764a0 0x7ff028073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.462+0000 7ff026800700 1 -- 172.21.15.90:0/3464953129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff028075b50 msgr2=0x7ff028072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:16.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.462+0000 7ff026800700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff028075b50 0x7ff028072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.462+0000 7ff026800700 1 -- 172.21.15.90:0/3464953129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0280764a0 msgr2=0x7ff028073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:16.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.462+0000 7ff026800700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0280764a0 0x7ff028073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.462+0000 7ff026800700 1 -- 172.21.15.90:0/3464953129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0281147e0 con 0x7ff028115170 2024-05-27T08:01:16.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.464+0000 7ff026800700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff028115170 0x7ff0281142a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff02000ef30 tx=0x7ff02000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:16.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.464+0000 7ff01ea00700 1 -- 172.21.15.90:0/3464953129 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200092e0 con 0x7ff028115170 2024-05-27T08:01:16.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.465+0000 7ff027200700 1 -- 172.21.15.90:0/3464953129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff028114a70 con 0x7ff028115170 2024-05-27T08:01:16.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.465+0000 7ff027200700 1 -- 172.21.15.90:0/3464953129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff028114f80 con 0x7ff028115170 2024-05-27T08:01:16.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.465+0000 7ff01ea00700 1 -- 172.21.15.90:0/3464953129 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff020004bb0 con 0x7ff028115170 2024-05-27T08:01:16.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.465+0000 7ff01ea00700 1 -- 172.21.15.90:0/3464953129 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020004020 con 0x7ff028115170 2024-05-27T08:01:16.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.470+0000 7ff01ea00700 1 -- 172.21.15.90:0/3464953129 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff02001b030 con 0x7ff028115170 2024-05-27T08:01:16.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.473+0000 7ff01ea00700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff00c0c3b00 0x7ff00c0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.473+0000 7ff01ea00700 1 -- 172.21.15.90:0/3464953129 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9584+0+0 (secure 0 0 0) 0x7ff0200e3360 con 0x7ff028115170 2024-05-27T08:01:16.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.475+0000 7ff025e00700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff00c0c3b00 0x7ff00c0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.494+0000 7ff025e00700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff00c0c3b00 0x7ff00c0c6010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff010009910 tx=0x7ff010008040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:16.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.497+0000 7ff03b9f7b80 1 Processor -- start 2024-05-27T08:01:16.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.498+0000 7ff03b9f7b80 1 -- start start 2024-05-27T08:01:16.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.499+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcf907e40 0x561dcfa85ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.499+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcfa86410 0x561dcfa90ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.499+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86820 0x561dcfa93020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.499+0000 7ff03b9f7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561dcf9b4e40 con 0x561dcf907e40 2024-05-27T08:01:16.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.499+0000 7ff03b9f7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561dcf9a49e0 con 0x561dcfa86820 2024-05-27T08:01:16.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.499+0000 7ff03b9f7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561dcf9a8a20 con 0x561dcfa86410 2024-05-27T08:01:16.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.499+0000 7ff025400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcfa86410 0x561dcfa90ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.500+0000 7ff025400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcfa86410 0x561dcfa90ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33472/0 (socket says 172.21.15.90:33472) 2024-05-27T08:01:16.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.500+0000 7ff025400700 1 -- 172.21.15.90:0/1097501560 learned_addr learned my addr 172.21.15.90:0/1097501560 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:16.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.500+0000 7ff025e00700 1 --2- 172.21.15.90:0/1097501560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcf907e40 0x561dcfa85ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.500+0000 7ff026800700 1 --2- 172.21.15.90:0/1097501560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86820 0x561dcfa93020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.500+0000 7ff025400700 1 -- 172.21.15.90:0/1097501560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86820 msgr2=0x561dcfa93020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:16.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.500+0000 7ff025400700 1 --2- 172.21.15.90:0/1097501560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86820 0x561dcfa93020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.500+0000 7ff025400700 1 -- 172.21.15.90:0/1097501560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcf907e40 msgr2=0x561dcfa85ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:16.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.500+0000 7ff025400700 1 --2- 172.21.15.90:0/1097501560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcf907e40 0x561dcfa85ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.500+0000 7ff025400700 1 -- 172.21.15.90:0/1097501560 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dcf96d420 con 0x561dcfa86410 2024-05-27T08:01:16.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.500+0000 7ff025e00700 1 --2- 172.21.15.90:0/1097501560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcf907e40 0x561dcfa85ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:16.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.501+0000 7ff025400700 1 --2- 172.21.15.90:0/1097501560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcfa86410 0x561dcfa90ad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff018000c00 tx=0x7ff018005100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154122ba7999d0dd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:16.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.502+0000 7ff017400700 1 -- 172.21.15.90:0/1097501560 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018026070 con 0x561dcfa86410 2024-05-27T08:01:16.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.502+0000 7ff017400700 1 -- 172.21.15.90:0/1097501560 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff01800bbe0 con 0x561dcfa86410 2024-05-27T08:01:16.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.502+0000 7ff017400700 1 -- 172.21.15.90:0/1097501560 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018003e10 con 0x561dcfa86410 2024-05-27T08:01:16.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.503+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/1097501560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcfa86410 msgr2=0x561dcfa90ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:16.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.503+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/1097501560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcfa86410 0x561dcfa90ad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff018000c00 tx=0x7ff018005100 comp rx=0 tx=0).stop 2024-05-27T08:01:16.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.504+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/1097501560 shutdown_connections 2024-05-27T08:01:16.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.504+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/1097501560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86820 0x561dcfa93020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.504+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/1097501560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcf907e40 0x561dcfa85ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.504+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/1097501560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcfa86410 0x561dcfa90ad0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.504+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/1097501560 >> 172.21.15.90:0/1097501560 conn(0x561dcf967fc0 msgr2=0x561dcfa81d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:16.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.504+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/1097501560 shutdown_connections 2024-05-27T08:01:16.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.504+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/1097501560 wait complete. 2024-05-27T08:01:16.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.505+0000 7ff03b9f7b80 1 -- start start 2024-05-27T08:01:16.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.506+0000 7ff03b9f7b80 1 Processor -- start 2024-05-27T08:01:16.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.507+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa90520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.507+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa90a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.507+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa90fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.507+0000 7ff03b9f7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561dcf8aafa0 con 0x561dcfa86820 2024-05-27T08:01:16.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.507+0000 7ff03b9f7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561dcf945760 con 0x561dcfa86410 2024-05-27T08:01:16.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.507+0000 7ff03b9f7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561dcfa93930 con 0x561dcf907e40 2024-05-27T08:01:16.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.507+0000 7ff03b9f7b80 10 client.0 _ll_register_callbacks cb 0x561dcf94ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:16.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.508+0000 7ff025400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa90a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.508+0000 7ff025e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa90520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.508+0000 7ff025400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa90a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:54704/0 (socket says 172.21.15.90:54704) 2024-05-27T08:01:16.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.508+0000 7ff025400700 1 -- 172.21.15.90:0/209908523 learned_addr learned my addr 172.21.15.90:0/209908523 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:16.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.508+0000 7ff026800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa90fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.509+0000 7ff026800700 1 -- 172.21.15.90:0/209908523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 msgr2=0x561dcfa90520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:16.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.509+0000 7ff026800700 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa90520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.509+0000 7ff026800700 1 -- 172.21.15.90:0/209908523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 msgr2=0x561dcfa90a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:16.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.509+0000 7ff026800700 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa90a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.509+0000 7ff026800700 1 -- 172.21.15.90:0/209908523 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dcf8f48d0 con 0x561dcfa86820 2024-05-27T08:01:16.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.510+0000 7ff026800700 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa90fa0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff0200e0650 tx=0x7ff0200aa7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a170ff3afd47aad8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:16.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.510+0000 7ff00be00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:01:16.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.512+0000 7ff00be00700 1 -- 172.21.15.90:0/209908523 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200c4060 con 0x561dcfa86820 2024-05-27T08:01:16.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.512+0000 7ff00be00700 1 -- 172.21.15.90:0/209908523 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9584+0+0 (secure 0 0 0) 0x7ff0200a96e0 con 0x561dcfa86820 2024-05-27T08:01:16.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.512+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/209908523 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff01800bbe0 con 0x561dcfa86820 2024-05-27T08:01:16.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.513+0000 7ff00be00700 1 -- 172.21.15.90:0/209908523 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff0200c9050 con 0x561dcfa86820 2024-05-27T08:01:16.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.513+0000 7ff03b9f7b80 10 client.17916 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:16.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.513+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/209908523 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561dcf975060 con 0x561dcfa86820 2024-05-27T08:01:16.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.513+0000 7ff00be00700 1 -- 172.21.15.90:0/209908523 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff01800bbe0 con 0x561dcfa86820 2024-05-27T08:01:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.513+0000 7ff00be00700 1 -- 172.21.15.90:0/209908523 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7ff0200ae170 con 0x561dcfa86820 2024-05-27T08:01:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.513+0000 7ff03b9f7b80 10 client.17916 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.513+0000 7ff03b9f7b80 10 client.17916 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T08:01:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.513+0000 7ff03b9f7b80 20 client.17916 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.513+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x561dcfaa08f0 0x561dcfaa2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.514+0000 7ff03b9f7b80 4 client.17916 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.514+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/209908523 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dcf9b6d90 con 0x561dcfaa08f0 2024-05-27T08:01:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.514+0000 7ff025e00700 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x561dcfaa08f0 0x561dcfaa2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.515+0000 7ff025e00700 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x561dcfaa08f0 0x561dcfaa2d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ba2664e974c5dd6 server_cookie=80d2ea26aacbb250 in_seq=0 out_seq=0 2024-05-27T08:01:16.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.515+0000 7ff00be00700 10 client.17916 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T08:01:16.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.518+0000 7ff00be00700 1 -- 172.21.15.90:0/209908523 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561dcf9b6d90 con 0x561dcfaa08f0 2024-05-27T08:01:16.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.518+0000 7ff00be00700 10 client.17916 handle_command_reply: tid=0 2024-05-27T08:01:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.518+0000 7ff03b9f7b80 1 client.17916 shutdown 2024-05-27T08:01:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.519+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/209908523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 msgr2=0x561dcfa90fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.519+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa90fa0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff0200e0650 tx=0x7ff0200aa7b0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.520+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/209908523 shutdown_connections 2024-05-27T08:01:16.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.520+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x561dcfaa08f0 0x561dcfaa2d80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.520+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa90fa0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.520+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa90520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.520+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/209908523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa90a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.520+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/209908523 >> 172.21.15.90:0/209908523 conn(0x561dcf967fc0 msgr2=0x561dcfa80ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:16.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.520+0000 7ff00be00700 0 client.17916 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T08:01:16.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.520+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/209908523 shutdown_connections 2024-05-27T08:01:16.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.520+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/209908523 wait complete. 2024-05-27T08:01:16.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.521+0000 7ff03b9f7b80 20 client.17916 trim_cache size 0 max 16384 2024-05-27T08:01:16.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.552+0000 7ff03b9f7b80 1 Processor -- start 2024-05-27T08:01:16.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.552+0000 7ff03b9f7b80 1 -- start start 2024-05-27T08:01:16.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.553+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa8bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.553+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa92990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.553+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa9ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.553+0000 7ff03b9f7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff004010a20 con 0x561dcfa86820 2024-05-27T08:01:16.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.553+0000 7ff03b9f7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff00400da70 con 0x561dcfa86410 2024-05-27T08:01:16.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.553+0000 7ff03b9f7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff00400fc80 con 0x561dcf907e40 2024-05-27T08:01:16.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.554+0000 7ff025400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa92990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.554+0000 7ff025400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa92990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:54706/0 (socket says 172.21.15.90:54706) 2024-05-27T08:01:16.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.554+0000 7ff025400700 1 -- 172.21.15.90:0/2904365655 learned_addr learned my addr 172.21.15.90:0/2904365655 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:16.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.554+0000 7ff025e00700 1 --2- 172.21.15.90:0/2904365655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa8bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.555+0000 7ff026800700 1 --2- 172.21.15.90:0/2904365655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa9ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.556+0000 7ff025e00700 1 -- 172.21.15.90:0/2904365655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 msgr2=0x561dcfa92990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:16.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.556+0000 7ff025e00700 1 --2- 172.21.15.90:0/2904365655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa92990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.556+0000 7ff025e00700 1 -- 172.21.15.90:0/2904365655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 msgr2=0x561dcfa9ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:16.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.556+0000 7ff025e00700 1 --2- 172.21.15.90:0/2904365655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa9ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.556+0000 7ff025e00700 1 -- 172.21.15.90:0/2904365655 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dcfa91530 con 0x561dcf907e40 2024-05-27T08:01:16.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.557+0000 7ff026800700 1 --2- 172.21.15.90:0/2904365655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa9ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:01:16.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.557+0000 7ff025e00700 1 --2- 172.21.15.90:0/2904365655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa8bd90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x561dcfaa3520 tx=0x7ff010013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe9a250079aeed1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:16.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.558+0000 7ff016a00700 1 -- 172.21.15.90:0/2904365655 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010014810 con 0x561dcf907e40 2024-05-27T08:01:16.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.558+0000 7ff016a00700 1 -- 172.21.15.90:0/2904365655 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff010014e40 con 0x561dcf907e40 2024-05-27T08:01:16.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.558+0000 7ff016a00700 1 -- 172.21.15.90:0/2904365655 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0100229d0 con 0x561dcf907e40 2024-05-27T08:01:16.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.560+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/2904365655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 msgr2=0x561dcfa8bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:16.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.560+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/2904365655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa8bd90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x561dcfaa3520 tx=0x7ff010013000 comp rx=0 tx=0).stop 2024-05-27T08:01:16.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.560+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/2904365655 shutdown_connections 2024-05-27T08:01:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.560+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/2904365655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa9ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.560+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/2904365655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa8bd90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.560+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/2904365655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa92990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.561+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/2904365655 >> 172.21.15.90:0/2904365655 conn(0x561dcf967fc0 msgr2=0x561dcfa904e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:16.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.561+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/2904365655 shutdown_connections 2024-05-27T08:01:16.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.561+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/2904365655 wait complete. 2024-05-27T08:01:16.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.562+0000 7ff03b9f7b80 1 -- start start 2024-05-27T08:01:16.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.562+0000 7ff03b9f7b80 1 Processor -- start 2024-05-27T08:01:16.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.563+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa94960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.564+0000 7ff03b9f7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa94ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.564+0000 7ff025e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa94960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.564+0000 7ff025e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa94960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33496/0 (socket says 172.21.15.90:33496) 2024-05-27T08:01:16.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.564+0000 7ff025e00700 1 -- 172.21.15.90:0/920794963 learned_addr learned my addr 172.21.15.90:0/920794963 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:16.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.564+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.565+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff00400f5b0 con 0x561dcfa86820 2024-05-27T08:01:16.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.565+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff00400e140 con 0x561dcfa86410 2024-05-27T08:01:16.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.565+0000 7ff025400700 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa94ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.565+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff00400e810 con 0x561dcf907e40 2024-05-27T08:01:16.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.565+0000 7ff03b9f7b80 10 client.0 _ll_register_callbacks cb 0x561dcf94ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:16.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.566+0000 7ff026800700 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.566+0000 7ff025e00700 1 -- 172.21.15.90:0/920794963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 msgr2=0x561dcfa94ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:16.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.566+0000 7ff025e00700 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa94ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.566+0000 7ff025e00700 1 -- 172.21.15.90:0/920794963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 msgr2=0x561dcfa953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:16.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.567+0000 7ff025e00700 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.567+0000 7ff025e00700 1 -- 172.21.15.90:0/920794963 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dcfa83350 con 0x561dcf907e40 2024-05-27T08:01:16.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.567+0000 7ff026800700 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:01:16.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.567+0000 7ff025400700 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa94ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:16.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.568+0000 7ff025e00700 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa94960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff010016010 tx=0x7ff010022410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98c3297b86a69870 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:16.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.568+0000 7ff00be00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:01:16.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.568+0000 7ff00be00700 1 -- 172.21.15.90:0/920794963 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010014a20 con 0x561dcf907e40 2024-05-27T08:01:16.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.569+0000 7ff00be00700 1 -- 172.21.15.90:0/920794963 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9584+0+0 (secure 0 0 0) 0x7ff01000f040 con 0x561dcf907e40 2024-05-27T08:01:16.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.569+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561dcf9b0570 con 0x561dcf907e40 2024-05-27T08:01:16.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.570+0000 7ff00be00700 1 -- 172.21.15.90:0/920794963 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff010014bc0 con 0x561dcf907e40 2024-05-27T08:01:16.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.570+0000 7ff03b9f7b80 10 client.27655 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:16.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.570+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561dcfa7c1d0 con 0x561dcf907e40 2024-05-27T08:01:16.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.571+0000 7ff00be00700 1 -- 172.21.15.90:0/920794963 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561dcf9b0570 con 0x561dcf907e40 2024-05-27T08:01:16.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.572+0000 7ff00be00700 1 -- 172.21.15.90:0/920794963 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7ff010013070 con 0x561dcf907e40 2024-05-27T08:01:16.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.572+0000 7ff03b9f7b80 10 client.27655 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:16.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.572+0000 7ff03b9f7b80 10 client.27655 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T08:01:16.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.572+0000 7ff03b9f7b80 20 client.27655 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:16.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.572+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x561dcfb0b820 0x561dcfb0dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:16.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.572+0000 7ff03b9f7b80 4 client.27655 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:01:16.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.572+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561dcf827250 con 0x561dcfb0b820 2024-05-27T08:01:16.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.573+0000 7ff025400700 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x561dcfb0b820 0x561dcfb0dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:16.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.574+0000 7ff025400700 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x561dcfb0b820 0x561dcfb0dc40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da8456d562b3cad9 server_cookie=6f2b0525a613679d in_seq=0 out_seq=0 2024-05-27T08:01:16.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.574+0000 7ff00be00700 10 client.27655 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T08:01:16.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.575+0000 7ff00be00700 1 -- 172.21.15.90:0/920794963 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561dcf827250 con 0x561dcfb0b820 2024-05-27T08:01:16.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.575+0000 7ff00be00700 10 client.27655 handle_command_reply: tid=0 2024-05-27T08:01:16.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.575+0000 7ff03b9f7b80 1 client.27655 shutdown 2024-05-27T08:01:16.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 msgr2=0x561dcfa94960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:16.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa94960 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff010016010 tx=0x7ff010022410 comp rx=0 tx=0).stop 2024-05-27T08:01:16.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 shutdown_connections 2024-05-27T08:01:16.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x561dcfb0b820 0x561dcfb0dc40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561dcfa86820 0x561dcfa953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561dcf907e40 0x561dcfa94960 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff03b9f7b80 1 --2- 172.21.15.90:0/920794963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561dcfa86410 0x561dcfa94ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 >> 172.21.15.90:0/920794963 conn(0x561dcf967fc0 msgr2=0x561dcfa918e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:16.579 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:01:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff00be00700 0 client.27655 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T08:01:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 shutdown_connections 2024-05-27T08:01:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.576+0000 7ff03b9f7b80 1 -- 172.21.15.90:0/920794963 wait complete. 2024-05-27T08:01:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.577+0000 7ff03b9f7b80 20 client.27655 trim_cache size 0 max 16384 2024-05-27T08:01:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.578+0000 7ff027200700 1 -- 172.21.15.90:0/3464953129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff00c0c3b00 msgr2=0x7ff00c0c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.578+0000 7ff027200700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff00c0c3b00 0x7ff00c0c6010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff010009910 tx=0x7ff010008040 comp rx=0 tx=0).stop 2024-05-27T08:01:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.578+0000 7ff027200700 1 -- 172.21.15.90:0/3464953129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff028115170 msgr2=0x7ff0281142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:16.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.578+0000 7ff027200700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff028115170 0x7ff0281142a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff02000ef30 tx=0x7ff02000c3a0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.579+0000 7ff027200700 1 -- 172.21.15.90:0/3464953129 shutdown_connections 2024-05-27T08:01:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.579+0000 7ff027200700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff028115170 0x7ff0281142a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.579+0000 7ff027200700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff00c0c3b00 0x7ff00c0c6010 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.579+0000 7ff027200700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff028075b50 0x7ff028072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.579+0000 7ff027200700 1 --2- 172.21.15.90:0/3464953129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0280764a0 0x7ff028073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.579+0000 7ff027200700 1 -- 172.21.15.90:0/3464953129 >> 172.21.15.90:0/3464953129 conn(0x7ff028079d00 msgr2=0x7ff028101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.579+0000 7ff027200700 1 -- 172.21.15.90:0/3464953129 shutdown_connections 2024-05-27T08:01:16.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:16.579+0000 7ff027200700 1 -- 172.21.15.90:0/3464953129 wait complete. 2024-05-27T08:01:17.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:17 smithi090 ceph-mon[31429]: pgmap v477: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.9 MiB/s wr, 504 op/s 2024-05-27T08:01:17.467 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:17 smithi090 ceph-mon[31429]: osdmap e118: 12 total, 12 up, 12 in 2024-05-27T08:01:17.728 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T08:01:17.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:17 smithi136 ceph-mon[35324]: pgmap v477: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.9 MiB/s wr, 504 op/s 2024-05-27T08:01:17.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:17 smithi136 ceph-mon[35324]: osdmap e118: 12 total, 12 up, 12 in 2024-05-27T08:01:17.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:17 smithi129 ceph-mon[35254]: pgmap v477: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.9 MiB/s wr, 504 op/s 2024-05-27T08:01:17.817 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:17 smithi129 ceph-mon[35254]: osdmap e118: 12 total, 12 up, 12 in 2024-05-27T08:01:18.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 660.0 s, 595.8 tps, lat 188.526 ms stddev 327.809 2024-05-27T08:01:19.203 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:19.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:19 smithi090 ceph-mon[31429]: pgmap v479: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-05-27T08:01:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:19 smithi136 ceph-mon[35324]: pgmap v479: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-05-27T08:01:19.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:19 smithi129 ceph-mon[35254]: pgmap v479: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-05-27T08:01:21.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:21 smithi090 ceph-mon[31429]: pgmap v480: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 MiB/s wr, 515 op/s 2024-05-27T08:01:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.736+0000 7f9712600640 1 -- 172.21.15.90:0/2820559097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f970c1029a0 msgr2=0x7f970c102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.736+0000 7f9712600640 1 --2- 172.21.15.90:0/2820559097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f970c1029a0 0x7f970c102db0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9704009900 tx=0x7f970402a400 comp rx=0 tx=0).stop 2024-05-27T08:01:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.740+0000 7f9712600640 1 -- 172.21.15.90:0/2820559097 shutdown_connections 2024-05-27T08:01:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.740+0000 7f9712600640 1 --2- 172.21.15.90:0/2820559097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970c110540 0x7f970c112960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.740+0000 7f9712600640 1 --2- 172.21.15.90:0/2820559097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f970c10dbe0 0x7f970c110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.740+0000 7f9712600640 1 --2- 172.21.15.90:0/2820559097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f970c1029a0 0x7f970c102db0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.740+0000 7f9712600640 1 -- 172.21.15.90:0/2820559097 >> 172.21.15.90:0/2820559097 conn(0x7f970c0782d0 msgr2=0x7f970c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.741+0000 7f9712600640 1 -- 172.21.15.90:0/2820559097 shutdown_connections 2024-05-27T08:01:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.741+0000 7f9712600640 1 -- 172.21.15.90:0/2820559097 wait complete. 2024-05-27T08:01:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.743+0000 7f9712600640 1 Processor -- start 2024-05-27T08:01:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.743+0000 7f9712600640 1 -- start start 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.744+0000 7f9712600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f970c1029a0 0x7f970c1af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.744+0000 7f9712600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f970c10dbe0 0x7f970c1af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.744+0000 7f9712600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970c110540 0x7f970c1a9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.744+0000 7f9712600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f970c106ed0 con 0x7f970c10dbe0 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.744+0000 7f9712600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f970c106d50 con 0x7f970c110540 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.744+0000 7f9712600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f970c107050 con 0x7f970c1029a0 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.744+0000 7f970b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f970c1029a0 0x7f970c1af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.744+0000 7f970b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f970c1029a0 0x7f970c1af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:41424/0 (socket says 172.21.15.90:41424) 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.744+0000 7f970b400640 1 -- 172.21.15.90:0/1985857906 learned_addr learned my addr 172.21.15.90:0/1985857906 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.744+0000 7f970aa00640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f970c10dbe0 0x7f970c1af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.745+0000 7f970be00640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970c110540 0x7f970c1a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.745+0000 7f970b400640 1 -- 172.21.15.90:0/1985857906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970c110540 msgr2=0x7f970c1a9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.745+0000 7f970b400640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970c110540 0x7f970c1a9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.745+0000 7f970b400640 1 -- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f970c10dbe0 msgr2=0x7f970c1af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.746+0000 7f970b400640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f970c10dbe0 0x7f970c1af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.746+0000 7f970b400640 1 -- 172.21.15.90:0/1985857906 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c1a9c60 con 0x7f970c1029a0 2024-05-27T08:01:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.746+0000 7f970aa00640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f970c10dbe0 0x7f970c1af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:01:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.746+0000 7f970b400640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f970c1029a0 0x7f970c1af230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f970402a970 tx=0x7f9704003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.747+0000 7f9703e00640 1 -- 172.21.15.90:0/1985857906 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970403f070 con 0x7f970c1029a0 2024-05-27T08:01:21.754 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:21 smithi136 ceph-mon[35324]: pgmap v480: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 MiB/s wr, 515 op/s 2024-05-27T08:01:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.747+0000 7f9703e00640 1 -- 172.21.15.90:0/1985857906 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f970403baf0 con 0x7f970c1029a0 2024-05-27T08:01:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.747+0000 7f9703e00640 1 -- 172.21.15.90:0/1985857906 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704035030 con 0x7f970c1029a0 2024-05-27T08:01:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.747+0000 7f9712600640 1 -- 172.21.15.90:0/1985857906 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c1a9e90 con 0x7f970c1029a0 2024-05-27T08:01:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.747+0000 7f9712600640 1 -- 172.21.15.90:0/1985857906 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c1aa330 con 0x7f970c1029a0 2024-05-27T08:01:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.754+0000 7f9719774740 1 Processor -- start 2024-05-27T08:01:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.754+0000 7f9719774740 1 -- start start 2024-05-27T08:01:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.755+0000 7f9719774740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f3a1b0 0x55b2312dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.755+0000 7f9719774740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f112f0 0x55b2312dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.755+0000 7f9719774740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f194d0 0x55b2312dd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.755+0000 7f9719774740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b23124db90 con 0x55b230f3a1b0 2024-05-27T08:01:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.755+0000 7f9719774740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b23123d730 con 0x55b230f112f0 2024-05-27T08:01:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.755+0000 7f9719774740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b231241770 con 0x55b230f194d0 2024-05-27T08:01:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.755+0000 7f970aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f194d0 0x55b2312dd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.755+0000 7f970aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f194d0 0x55b2312dd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:41436/0 (socket says 172.21.15.90:41436) 2024-05-27T08:01:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.755+0000 7f970aa00640 1 -- 172.21.15.90:0/3127086005 learned_addr learned my addr 172.21.15.90:0/3127086005 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.757+0000 7f970aa00640 1 -- 172.21.15.90:0/3127086005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f112f0 msgr2=0x55b2312dcda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:01:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.757+0000 7f970aa00640 1 --2- 172.21.15.90:0/3127086005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f112f0 0x55b2312dcda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.757+0000 7f970aa00640 1 -- 172.21.15.90:0/3127086005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f3a1b0 msgr2=0x55b2312dc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.757+0000 7f970aa00640 1 --2- 172.21.15.90:0/3127086005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f3a1b0 0x55b2312dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.757+0000 7f970aa00640 1 -- 172.21.15.90:0/3127086005 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2311d68b0 con 0x55b230f194d0 2024-05-27T08:01:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.759+0000 7f970aa00640 1 --2- 172.21.15.90:0/3127086005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f194d0 0x55b2312dd2e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009ae0 tx=0x7f96f80113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87401c1866eafedc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.759+0000 7f9700c00640 1 -- 172.21.15.90:0/3127086005 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8011e70 con 0x55b230f194d0 2024-05-27T08:01:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.760+0000 7f9700c00640 1 -- 172.21.15.90:0/3127086005 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96f80134d0 con 0x55b230f194d0 2024-05-27T08:01:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.760+0000 7f9700c00640 1 -- 172.21.15.90:0/3127086005 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f8012aa0 con 0x55b230f194d0 2024-05-27T08:01:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.760+0000 7f9703e00640 1 -- 172.21.15.90:0/1985857906 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9704042070 con 0x7f970c1029a0 2024-05-27T08:01:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.763+0000 7f9703e00640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f96f00b2aa0 0x7f96f00b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.763+0000 7f9703e00640 1 -- 172.21.15.90:0/1985857906 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f97041031f0 con 0x7f970c1029a0 2024-05-27T08:01:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.764+0000 7f9719774740 1 -- 172.21.15.90:0/3127086005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f194d0 msgr2=0x55b2312dd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.764+0000 7f9719774740 1 --2- 172.21.15.90:0/3127086005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f194d0 0x55b2312dd2e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009ae0 tx=0x7f96f80113e0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.764+0000 7f970be00640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f96f00b2aa0 0x7f96f00b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.765+0000 7f9719774740 1 -- 172.21.15.90:0/3127086005 shutdown_connections 2024-05-27T08:01:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.766+0000 7f9719774740 1 --2- 172.21.15.90:0/3127086005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f112f0 0x55b2312dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.766+0000 7f9719774740 1 --2- 172.21.15.90:0/3127086005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f194d0 0x55b2312dd2e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.766+0000 7f9719774740 1 --2- 172.21.15.90:0/3127086005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f3a1b0 0x55b2312dc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.766+0000 7f9719774740 1 -- 172.21.15.90:0/3127086005 >> 172.21.15.90:0/3127086005 conn(0x55b230f2e300 msgr2=0x55b2312d9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.767+0000 7f9719774740 1 -- 172.21.15.90:0/3127086005 shutdown_connections 2024-05-27T08:01:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.768+0000 7f970be00640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f96f00b2aa0 0x7f96f00b4fb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0092f0 tx=0x7f96fc009280 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:21.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.768+0000 7f9719774740 1 -- 172.21.15.90:0/3127086005 wait complete. 2024-05-27T08:01:21.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.769+0000 7f9719774740 1 -- start start 2024-05-27T08:01:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.770+0000 7f9719774740 1 Processor -- start 2024-05-27T08:01:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.770+0000 7f9719774740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 0x55b2312e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.770+0000 7f9719774740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f194d0 0x55b2312e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.770+0000 7f9719774740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.771+0000 7f9719774740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b230f23790 con 0x55b230f112f0 2024-05-27T08:01:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.771+0000 7f9719774740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2311e5bc0 con 0x55b230f194d0 2024-05-27T08:01:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.771+0000 7f9719774740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b2312e9990 con 0x55b230f3a1b0 2024-05-27T08:01:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.771+0000 7f9719774740 10 client.0 _ll_register_callbacks cb 0x55b2311c3400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.772+0000 7f970be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.772+0000 7f970be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312f2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:41452/0 (socket says 172.21.15.90:41452) 2024-05-27T08:01:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.772+0000 7f970be00640 1 -- 172.21.15.90:0/1634513747 learned_addr learned my addr 172.21.15.90:0/1634513747 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.772+0000 7f970aa00640 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f194d0 0x55b2312e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.772+0000 7f970b400640 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 0x55b2312e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.773+0000 7f970be00640 1 -- 172.21.15.90:0/1634513747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f194d0 msgr2=0x55b2312e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.773+0000 7f970be00640 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f194d0 0x55b2312e6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.773+0000 7f970be00640 1 -- 172.21.15.90:0/1634513747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 msgr2=0x55b2312e6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.773+0000 7f970be00640 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 0x55b2312e6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.774+0000 7f970be00640 1 -- 172.21.15.90:0/1634513747 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b230f52210 con 0x55b230f3a1b0 2024-05-27T08:01:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.775+0000 7f970be00640 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312f2cc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f96fc022be0 tx=0x7f96fc005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f971fadb6e6fd6b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.775+0000 7f96f5600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:01:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.775+0000 7f96f5600640 1 -- 172.21.15.90:0/1634513747 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc02cc40 con 0x55b230f3a1b0 2024-05-27T08:01:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.775+0000 7f96f5600640 1 -- 172.21.15.90:0/1634513747 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f96fc02cde0 con 0x55b230f3a1b0 2024-05-27T08:01:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.776+0000 7f9719774740 1 -- 172.21.15.90:0/1634513747 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f80134d0 con 0x55b230f3a1b0 2024-05-27T08:01:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.777+0000 7f96f5600640 1 -- 172.21.15.90:0/1634513747 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f96fc038410 con 0x55b230f3a1b0 2024-05-27T08:01:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.777+0000 7f9719774740 10 client.27667 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.777+0000 7f9719774740 1 -- 172.21.15.90:0/1634513747 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b23101e620 con 0x55b230f3a1b0 2024-05-27T08:01:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.777+0000 7f96f5600640 1 -- 172.21.15.90:0/1634513747 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96f80134d0 con 0x55b230f3a1b0 2024-05-27T08:01:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.778+0000 7f96f5600640 1 -- 172.21.15.90:0/1634513747 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f96fc01f050 con 0x55b230f3a1b0 2024-05-27T08:01:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.778+0000 7f9719774740 10 client.27667 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.778+0000 7f9719774740 10 client.27667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.778+0000 7f9719774740 20 client.27667 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.778+0000 7f9719774740 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b230ef8d80 0x55b2312f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.778+0000 7f9719774740 4 client.27667 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.778+0000 7f9719774740 1 -- 172.21.15.90:0/1634513747 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b23124fae0 con 0x55b230ef8d80 2024-05-27T08:01:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.779+0000 7f970b400640 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b230ef8d80 0x55b2312f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.780+0000 7f970b400640 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b230ef8d80 0x55b2312f8e00 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7556304d05a31ddc server_cookie=3ade77af5b8e26ea in_seq=0 out_seq=0 2024-05-27T08:01:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.780+0000 7f96f5600640 10 client.27667 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.781+0000 7f96f5600640 1 -- 172.21.15.90:0/1634513747 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b23124fae0 con 0x55b230ef8d80 2024-05-27T08:01:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.781+0000 7f96f5600640 10 client.27667 handle_command_reply: tid=0 2024-05-27T08:01:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.781+0000 7f9719774740 1 client.27667 shutdown 2024-05-27T08:01:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f9719774740 1 -- 172.21.15.90:0/1634513747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 msgr2=0x55b2312f2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f9719774740 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312f2cc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f96fc022be0 tx=0x7f96fc005bd0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f9719774740 1 -- 172.21.15.90:0/1634513747 shutdown_connections 2024-05-27T08:01:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f9719774740 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b230ef8d80 0x55b2312f8e00 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f9719774740 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f194d0 0x55b2312e6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f9719774740 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312f2cc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f96f5600640 0 client.27667 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f9719774740 1 --2- 172.21.15.90:0/1634513747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 0x55b2312e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f9719774740 1 -- 172.21.15.90:0/1634513747 >> 172.21.15.90:0/1634513747 conn(0x55b230f2e300 msgr2=0x55b2312de580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f9719774740 1 -- 172.21.15.90:0/1634513747 shutdown_connections 2024-05-27T08:01:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.782+0000 7f9719774740 1 -- 172.21.15.90:0/1634513747 wait complete. 2024-05-27T08:01:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.783+0000 7f9719774740 20 client.27667 trim_cache size 0 max 16384 2024-05-27T08:01:21.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:21 smithi129 ceph-mon[35254]: pgmap v480: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 MiB/s wr, 515 op/s 2024-05-27T08:01:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.823+0000 7f9719774740 1 Processor -- start 2024-05-27T08:01:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.824+0000 7f9719774740 1 -- start start 2024-05-27T08:01:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.824+0000 7f9719774740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f112f0 0x55b2312dd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.824+0000 7f9719774740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f194d0 0x55b2312e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.824+0000 7f9719774740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312d8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.824+0000 7f9719774740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f970c1b92c0 con 0x55b230f194d0 2024-05-27T08:01:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.824+0000 7f9719774740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f970c1b6310 con 0x55b230f112f0 2024-05-27T08:01:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.824+0000 7f9719774740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f970c1b8520 con 0x55b230f3a1b0 2024-05-27T08:01:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.825+0000 7f970aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f194d0 0x55b2312e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.826+0000 7f970be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.826+0000 7f970be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312d8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:41466/0 (socket says 172.21.15.90:41466) 2024-05-27T08:01:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.826+0000 7f970be00640 1 -- 172.21.15.90:0/2506822226 learned_addr learned my addr 172.21.15.90:0/2506822226 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.826+0000 7f970be00640 1 -- 172.21.15.90:0/2506822226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f112f0 msgr2=0x55b2312dd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.826+0000 7f970be00640 1 --2- 172.21.15.90:0/2506822226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f112f0 0x55b2312dd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.826+0000 7f970be00640 1 -- 172.21.15.90:0/2506822226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f194d0 msgr2=0x55b2312e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.826+0000 7f970be00640 1 --2- 172.21.15.90:0/2506822226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f194d0 0x55b2312e8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.826+0000 7f970be00640 1 -- 172.21.15.90:0/2506822226 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2312d3050 con 0x55b230f3a1b0 2024-05-27T08:01:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.828+0000 7f970be00640 1 --2- 172.21.15.90:0/2506822226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312d8330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0427d0 tx=0x7f96fc009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71ff2fc6893bb73 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:21.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.828+0000 7f96f7e00640 1 -- 172.21.15.90:0/2506822226 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc042a40 con 0x55b230f3a1b0 2024-05-27T08:01:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.829+0000 7f96f7e00640 1 -- 172.21.15.90:0/2506822226 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96fc042be0 con 0x55b230f3a1b0 2024-05-27T08:01:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.829+0000 7f96f7e00640 1 -- 172.21.15.90:0/2506822226 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc038830 con 0x55b230f3a1b0 2024-05-27T08:01:21.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.830+0000 7f9719774740 1 -- 172.21.15.90:0/2506822226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 msgr2=0x55b2312d8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.830+0000 7f9719774740 1 --2- 172.21.15.90:0/2506822226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312d8330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0427d0 tx=0x7f96fc009560 comp rx=0 tx=0).stop 2024-05-27T08:01:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.830+0000 7f9719774740 1 -- 172.21.15.90:0/2506822226 shutdown_connections 2024-05-27T08:01:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.832+0000 7f9719774740 1 --2- 172.21.15.90:0/2506822226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312d8330 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.832+0000 7f9719774740 1 --2- 172.21.15.90:0/2506822226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f194d0 0x55b2312e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.832+0000 7f9719774740 1 --2- 172.21.15.90:0/2506822226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f112f0 0x55b2312dd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.832+0000 7f9719774740 1 -- 172.21.15.90:0/2506822226 >> 172.21.15.90:0/2506822226 conn(0x55b230f2e300 msgr2=0x55b2312e5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.833+0000 7f9719774740 1 -- 172.21.15.90:0/2506822226 shutdown_connections 2024-05-27T08:01:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.833+0000 7f9719774740 1 -- 172.21.15.90:0/2506822226 wait complete. 2024-05-27T08:01:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.834+0000 7f9719774740 1 -- start start 2024-05-27T08:01:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.835+0000 7f9719774740 1 Processor -- start 2024-05-27T08:01:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.836+0000 7f9719774740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 0x55b2312e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.837+0000 7f9719774740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f194d0 0x55b2312e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.837+0000 7f9719774740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312e1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.837+0000 7f9719774740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f970c1b7e50 con 0x55b230f112f0 2024-05-27T08:01:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.837+0000 7f9719774740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f970c1b69e0 con 0x55b230f194d0 2024-05-27T08:01:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.837+0000 7f9719774740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f970c1b70b0 con 0x55b230f3a1b0 2024-05-27T08:01:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.837+0000 7f9719774740 10 client.0 _ll_register_callbacks cb 0x7f96fc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.837+0000 7f970be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312e1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.838+0000 7f970b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 0x55b2312e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.838+0000 7f970b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 0x55b2312e0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43778/0 (socket says 172.21.15.90:43778) 2024-05-27T08:01:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.838+0000 7f970b400640 1 -- 172.21.15.90:0/81466872 learned_addr learned my addr 172.21.15.90:0/81466872 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.838+0000 7f970aa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f194d0 0x55b2312e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.839+0000 7f970b400640 1 -- 172.21.15.90:0/81466872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 msgr2=0x55b2312e1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.839+0000 7f970b400640 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312e1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.839+0000 7f970b400640 1 -- 172.21.15.90:0/81466872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f194d0 msgr2=0x55b2312e0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.839+0000 7f970b400640 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f194d0 0x55b2312e0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.839+0000 7f970b400640 1 -- 172.21.15.90:0/81466872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2312ede70 con 0x55b230f112f0 2024-05-27T08:01:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.841+0000 7f970b400640 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 0x55b2312e0a80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f97040c6120 tx=0x7f97040ca860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0794424f10ac3e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.841+0000 7f96f5600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:01:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.841+0000 7f96f5600640 1 -- 172.21.15.90:0/81466872 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704049040 con 0x55b230f112f0 2024-05-27T08:01:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.842+0000 7f96f5600640 1 -- 172.21.15.90:0/81466872 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f97040ce0d0 con 0x55b230f112f0 2024-05-27T08:01:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.842+0000 7f9719774740 1 -- 172.21.15.90:0/81466872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96fc042be0 con 0x55b230f112f0 2024-05-27T08:01:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.843+0000 7f96f5600640 1 -- 172.21.15.90:0/81466872 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f97040d2f00 con 0x55b230f112f0 2024-05-27T08:01:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.843+0000 7f9719774740 10 client.17931 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.843+0000 7f9719774740 1 -- 172.21.15.90:0/81466872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b2312ef100 con 0x55b230f112f0 2024-05-27T08:01:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.844+0000 7f96f5600640 1 -- 172.21.15.90:0/81466872 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96fc042be0 con 0x55b230f112f0 2024-05-27T08:01:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.844+0000 7f96f5600640 1 -- 172.21.15.90:0/81466872 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f97040d2a70 con 0x55b230f112f0 2024-05-27T08:01:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.844+0000 7f9719774740 10 client.17931 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.844+0000 7f9719774740 10 client.17931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.844+0000 7f9719774740 20 client.17931 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.844+0000 7f9719774740 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b230ece1c0 0x55b2313634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.845+0000 7f9719774740 4 client.17931 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:01:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.845+0000 7f9719774740 1 -- 172.21.15.90:0/81466872 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2312492c0 con 0x55b230ece1c0 2024-05-27T08:01:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.845+0000 7f970aa00640 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b230ece1c0 0x55b2313634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.846+0000 7f970aa00640 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b230ece1c0 0x55b2313634a0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d6487d870957b12 server_cookie=caa4dde871397f0f in_seq=0 out_seq=0 2024-05-27T08:01:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.847+0000 7f96f5600640 10 client.17931 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f96f5600640 1 -- 172.21.15.90:0/81466872 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b2312492c0 con 0x55b230ece1c0 2024-05-27T08:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f96f5600640 10 client.17931 handle_command_reply: tid=0 2024-05-27T08:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f9719774740 1 client.17931 shutdown 2024-05-27T08:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f9719774740 1 -- 172.21.15.90:0/81466872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 msgr2=0x55b2312e0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f9719774740 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 0x55b2312e0a80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f97040c6120 tx=0x7f97040ca860 comp rx=0 tx=0).stop 2024-05-27T08:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f9719774740 1 -- 172.21.15.90:0/81466872 shutdown_connections 2024-05-27T08:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f9719774740 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b230ece1c0 0x55b2313634a0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f9719774740 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b230f194d0 0x55b2312e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f9719774740 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b230f3a1b0 0x55b2312e1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f9719774740 1 --2- 172.21.15.90:0/81466872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b230f112f0 0x55b2312e0a80 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f9719774740 1 -- 172.21.15.90:0/81466872 >> 172.21.15.90:0/81466872 conn(0x55b230f2e300 msgr2=0x55b2312e5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.878+0000 7f96f5600640 0 client.17931 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.880+0000 7f9719774740 1 -- 172.21.15.90:0/81466872 shutdown_connections 2024-05-27T08:01:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.880+0000 7f9719774740 1 -- 172.21.15.90:0/81466872 wait complete. 2024-05-27T08:01:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.880+0000 7f9719774740 20 client.17931 trim_cache size 0 max 16384 2024-05-27T08:01:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:01:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:01:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:01:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.883+0000 7f9712600640 1 -- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f96f00b2aa0 msgr2=0x7f96f00b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.883+0000 7f9712600640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f96f00b2aa0 0x7f96f00b4fb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0092f0 tx=0x7f96fc009280 comp rx=0 tx=0).stop 2024-05-27T08:01:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.883+0000 7f9712600640 1 -- 172.21.15.90:0/1985857906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f970c1029a0 msgr2=0x7f970c1af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.883+0000 7f9712600640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f970c1029a0 0x7f970c1af230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f970402a970 tx=0x7f9704003c00 comp rx=0 tx=0).stop 2024-05-27T08:01:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.883+0000 7f9712600640 1 -- 172.21.15.90:0/1985857906 shutdown_connections 2024-05-27T08:01:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.883+0000 7f9712600640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f96f00b2aa0 0x7f96f00b4fb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.883+0000 7f9712600640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f970c110540 0x7f970c1a9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.883+0000 7f9712600640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f970c10dbe0 0x7f970c1af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.883+0000 7f9712600640 1 --2- 172.21.15.90:0/1985857906 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f970c1029a0 0x7f970c1af230 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.883+0000 7f9712600640 1 -- 172.21.15.90:0/1985857906 >> 172.21.15.90:0/1985857906 conn(0x7f970c0782d0 msgr2=0x7f970c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.884+0000 7f9712600640 1 -- 172.21.15.90:0/1985857906 shutdown_connections 2024-05-27T08:01:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:21.884+0000 7f9712600640 1 -- 172.21.15.90:0/1985857906 wait complete. 2024-05-27T08:01:22.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.003+0000 7f524aa00700 1 -- 172.21.15.90:0/3466627801 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c009480 con 0x7f524c111bc0 2024-05-27T08:01:22.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.003+0000 7f5252400700 1 -- 172.21.15.90:0/3466627801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524c111bc0 msgr2=0x7f524c111fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:22.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.003+0000 7f5252400700 1 --2- 172.21.15.90:0/3466627801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524c111bc0 0x7f524c111fd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f523c00b0a0 tx=0x7f523c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.003+0000 7f5252400700 1 -- 172.21.15.90:0/3466627801 shutdown_connections 2024-05-27T08:01:22.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.003+0000 7f5252400700 1 --2- 172.21.15.90:0/3466627801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f524c10b270 0x7f524c1198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.003+0000 7f5252400700 1 --2- 172.21.15.90:0/3466627801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524c111bc0 0x7f524c111fd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.003+0000 7f5252400700 1 --2- 172.21.15.90:0/3466627801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c10a920 0x7f524c10ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.003+0000 7f5252400700 1 -- 172.21.15.90:0/3466627801 >> 172.21.15.90:0/3466627801 conn(0x7f524c06f4c0 msgr2=0x7f524c06f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:22.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.004+0000 7f5252400700 1 -- 172.21.15.90:0/3466627801 shutdown_connections 2024-05-27T08:01:22.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.004+0000 7f5252400700 1 -- 172.21.15.90:0/3466627801 wait complete. 2024-05-27T08:01:22.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.005+0000 7f5252400700 1 Processor -- start 2024-05-27T08:01:22.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.006+0000 7f5252400700 1 -- start start 2024-05-27T08:01:22.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.006+0000 7f5252400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f524c10a920 0x7f524c114a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.006+0000 7f5252400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c10b270 0x7f524c114f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.006+0000 7f5252400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524c111bc0 0x7f524c115480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.007+0000 7f5252400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f524c11cb90 con 0x7f524c10b270 2024-05-27T08:01:22.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.007+0000 7f5252400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f524c11ca10 con 0x7f524c10a920 2024-05-27T08:01:22.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.007+0000 7f5252400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f524c11cd10 con 0x7f524c111bc0 2024-05-27T08:01:22.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.007+0000 7f5242a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c10b270 0x7f524c114f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.007+0000 7f5242a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c10b270 0x7f524c114f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43800/0 (socket says 172.21.15.90:43800) 2024-05-27T08:01:22.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.007+0000 7f5242a00700 1 -- 172.21.15.90:0/2037556778 learned_addr learned my addr 172.21.15.90:0/2037556778 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:22.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.007+0000 7f524b400700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f524c10a920 0x7f524c114a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.007+0000 7f524be00700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524c111bc0 0x7f524c115480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.008+0000 7f5242a00700 1 -- 172.21.15.90:0/2037556778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524c111bc0 msgr2=0x7f524c115480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:22.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.008+0000 7f5242a00700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524c111bc0 0x7f524c115480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.008+0000 7f5242a00700 1 -- 172.21.15.90:0/2037556778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f524c10a920 msgr2=0x7f524c114a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:22.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.008+0000 7f5242a00700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f524c10a920 0x7f524c114a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.008+0000 7f5242a00700 1 -- 172.21.15.90:0/2037556778 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c1a5280 con 0x7f524c10b270 2024-05-27T08:01:22.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.008+0000 7f524b400700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f524c10a920 0x7f524c114a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:22.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.009+0000 7f5242a00700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c10b270 0x7f524c114f40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f523400e9a0 tx=0x7f523400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:22.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.010+0000 7f5248c00700 1 -- 172.21.15.90:0/2037556778 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523400cbd0 con 0x7f524c10b270 2024-05-27T08:01:22.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.010+0000 7f5252400700 1 -- 172.21.15.90:0/2037556778 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524c1a5570 con 0x7f524c10b270 2024-05-27T08:01:22.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.010+0000 7f5252400700 1 -- 172.21.15.90:0/2037556778 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524c1a5b00 con 0x7f524c10b270 2024-05-27T08:01:22.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.010+0000 7f5248c00700 1 -- 172.21.15.90:0/2037556778 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f523400cd70 con 0x7f524c10b270 2024-05-27T08:01:22.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.010+0000 7f5248c00700 1 -- 172.21.15.90:0/2037556778 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234018680 con 0x7f524c10b270 2024-05-27T08:01:22.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.019+0000 7f5248c00700 1 -- 172.21.15.90:0/2037556778 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5234018820 con 0x7f524c10b270 2024-05-27T08:01:22.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.022+0000 7f5248c00700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f52300b2af0 0x7f52300b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.022+0000 7f5248c00700 1 -- 172.21.15.90:0/2037556778 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f52340e3b10 con 0x7f524c10b270 2024-05-27T08:01:22.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.023+0000 7f524b400700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f52300b2af0 0x7f52300b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.029+0000 7f524b400700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f52300b2af0 0x7f52300b5000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f523c00b780 tx=0x7f523c0099e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:22.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.031+0000 7f5260e9eb80 1 Processor -- start 2024-05-27T08:01:22.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.032+0000 7f5260e9eb80 1 -- start start 2024-05-27T08:01:22.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.032+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017a88de40 0x56017aa127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.032+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa12d30 0x56017aa0c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.032+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa0cdb0 0x56017aa0d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.032+0000 7f5260e9eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56017a93ae40 con 0x56017aa12d30 2024-05-27T08:01:22.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.032+0000 7f5260e9eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56017a92a9e0 con 0x56017a88de40 2024-05-27T08:01:22.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.032+0000 7f5260e9eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56017a92ea20 con 0x56017aa0cdb0 2024-05-27T08:01:22.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.033+0000 7f5242a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa12d30 0x56017aa0c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.033+0000 7f5242a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa12d30 0x56017aa0c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43804/0 (socket says 172.21.15.90:43804) 2024-05-27T08:01:22.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.033+0000 7f5242a00700 1 -- 172.21.15.90:0/1581103881 learned_addr learned my addr 172.21.15.90:0/1581103881 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:22.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.034+0000 7f5242a00700 1 -- 172.21.15.90:0/1581103881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa0cdb0 msgr2=0x56017aa0d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:22.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.034+0000 7f5242a00700 1 --2- 172.21.15.90:0/1581103881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa0cdb0 0x56017aa0d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.034+0000 7f5242a00700 1 -- 172.21.15.90:0/1581103881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017a88de40 msgr2=0x56017aa127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:22.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.034+0000 7f5242a00700 1 --2- 172.21.15.90:0/1581103881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017a88de40 0x56017aa127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.034+0000 7f5242a00700 1 -- 172.21.15.90:0/1581103881 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017a8f3420 con 0x56017aa12d30 2024-05-27T08:01:22.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.034+0000 7f5242a00700 1 --2- 172.21.15.90:0/1581103881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa12d30 0x56017aa0c870 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f523400e970 tx=0x7f52340aa230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56798b023145d586 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:22.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.035+0000 7f5240c00700 1 -- 172.21.15.90:0/1581103881 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340e12c0 con 0x56017aa12d30 2024-05-27T08:01:22.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.035+0000 7f5240c00700 1 -- 172.21.15.90:0/1581103881 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52340ab0c0 con 0x56017aa12d30 2024-05-27T08:01:22.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.035+0000 7f5240c00700 1 -- 172.21.15.90:0/1581103881 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340aac20 con 0x56017aa12d30 2024-05-27T08:01:22.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.038+0000 7f5260e9eb80 1 -- 172.21.15.90:0/1581103881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa12d30 msgr2=0x56017aa0c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:22.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.038+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/1581103881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa12d30 0x56017aa0c870 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f523400e970 tx=0x7f52340aa230 comp rx=0 tx=0).stop 2024-05-27T08:01:22.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.039+0000 7f5260e9eb80 1 -- 172.21.15.90:0/1581103881 shutdown_connections 2024-05-27T08:01:22.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.039+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/1581103881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa0cdb0 0x56017aa0d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.039+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/1581103881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017a88de40 0x56017aa127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.039+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/1581103881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa12d30 0x56017aa0c870 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.039+0000 7f5260e9eb80 1 -- 172.21.15.90:0/1581103881 >> 172.21.15.90:0/1581103881 conn(0x56017a8edfc0 msgr2=0x56017aa078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:22.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.039+0000 7f5260e9eb80 1 -- 172.21.15.90:0/1581103881 shutdown_connections 2024-05-27T08:01:22.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.040+0000 7f5260e9eb80 1 -- 172.21.15.90:0/1581103881 wait complete. 2024-05-27T08:01:22.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.040+0000 7f5260e9eb80 1 -- start start 2024-05-27T08:01:22.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.041+0000 7f5260e9eb80 1 Processor -- start 2024-05-27T08:01:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.042+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017a88de40 0x56017aa16630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.043+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa0cdb0 0x56017aa16b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.043+0000 7f524b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017a88de40 0x56017aa16630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.043+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 0x56017aa170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.043+0000 7f524b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017a88de40 0x56017aa16630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:59570/0 (socket says 172.21.15.90:59570) 2024-05-27T08:01:22.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.044+0000 7f5260e9eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56017a830fa0 con 0x56017aa0cdb0 2024-05-27T08:01:22.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.044+0000 7f5260e9eb80 1 -- 172.21.15.90:0/2442945666 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56017a8cb760 con 0x56017a88de40 2024-05-27T08:01:22.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.044+0000 7f5260e9eb80 1 -- 172.21.15.90:0/2442945666 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56017aa19800 con 0x56017aa12d30 2024-05-27T08:01:22.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.044+0000 7f524b400700 1 -- 172.21.15.90:0/2442945666 learned_addr learned my addr 172.21.15.90:0/2442945666 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:22.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.044+0000 7f5260e9eb80 10 client.0 _ll_register_callbacks cb 0x56017a8d4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:22.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.044+0000 7f5242a00700 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa0cdb0 0x56017aa16b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.045+0000 7f524be00700 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 0x56017aa170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.046+0000 7f5242a00700 1 -- 172.21.15.90:0/2442945666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 msgr2=0x56017aa170b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:22.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.046+0000 7f5242a00700 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 0x56017aa170b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.046+0000 7f5242a00700 1 -- 172.21.15.90:0/2442945666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017a88de40 msgr2=0x56017aa16630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:22.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.046+0000 7f5242a00700 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017a88de40 0x56017aa16630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.046+0000 7f5242a00700 1 -- 172.21.15.90:0/2442945666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017a87a8d0 con 0x56017aa0cdb0 2024-05-27T08:01:22.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.047+0000 7f5242a00700 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa0cdb0 0x56017aa16b70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f523400c5b0 tx=0x7f52340cd380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff0b03665c142db server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:22.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.047+0000 7f5239600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:01:22.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.048+0000 7f5239600700 1 -- 172.21.15.90:0/2442945666 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340c3070 con 0x56017aa0cdb0 2024-05-27T08:01:22.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.048+0000 7f5239600700 1 -- 172.21.15.90:0/2442945666 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f52340aae30 con 0x56017aa0cdb0 2024-05-27T08:01:22.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.048+0000 7f5260e9eb80 1 -- 172.21.15.90:0/2442945666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52340ab0c0 con 0x56017aa0cdb0 2024-05-27T08:01:22.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.049+0000 7f5239600700 1 -- 172.21.15.90:0/2442945666 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f5234088050 con 0x56017aa0cdb0 2024-05-27T08:01:22.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.049+0000 7f5260e9eb80 10 client.17943 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:22.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.049+0000 7f5260e9eb80 1 -- 172.21.15.90:0/2442945666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56017a8fb060 con 0x56017aa0cdb0 2024-05-27T08:01:22.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.051+0000 7f5239600700 1 -- 172.21.15.90:0/2442945666 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52340ab0c0 con 0x56017aa0cdb0 2024-05-27T08:01:22.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.051+0000 7f5239600700 1 -- 172.21.15.90:0/2442945666 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f52340a6330 con 0x56017aa0cdb0 2024-05-27T08:01:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.051+0000 7f5260e9eb80 10 client.17943 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.051+0000 7f5260e9eb80 10 client.17943 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T08:01:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.051+0000 7f5260e9eb80 20 client.17943 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.051+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56017aa266e0 0x56017aa28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.051+0000 7f5260e9eb80 4 client.17943 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.051+0000 7f5260e9eb80 1 -- 172.21.15.90:0/2442945666 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56017a93cd90 con 0x56017aa266e0 2024-05-27T08:01:22.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.052+0000 7f524b400700 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56017aa266e0 0x56017aa28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.054+0000 7f524b400700 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56017aa266e0 0x56017aa28b00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=11e7f3215e76c034 server_cookie=498fd7b3def6c816 in_seq=0 out_seq=0 2024-05-27T08:01:22.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.054+0000 7f5239600700 10 client.17943 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T08:01:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.056+0000 7f5239600700 1 -- 172.21.15.90:0/2442945666 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56017a93cd90 con 0x56017aa266e0 2024-05-27T08:01:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.056+0000 7f5239600700 10 client.17943 handle_command_reply: tid=0 2024-05-27T08:01:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.057+0000 7f5260e9eb80 1 client.17943 shutdown 2024-05-27T08:01:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.057+0000 7f5260e9eb80 1 -- 172.21.15.90:0/2442945666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa0cdb0 msgr2=0x56017aa16b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.057+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa0cdb0 0x56017aa16b70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f523400c5b0 tx=0x7f52340cd380 comp rx=0 tx=0).stop 2024-05-27T08:01:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.058+0000 7f5260e9eb80 1 -- 172.21.15.90:0/2442945666 shutdown_connections 2024-05-27T08:01:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.058+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56017aa266e0 0x56017aa28b00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.058+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 0x56017aa170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.058+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017a88de40 0x56017aa16630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.058+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/2442945666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017aa0cdb0 0x56017aa16b70 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.058+0000 7f5260e9eb80 1 -- 172.21.15.90:0/2442945666 >> 172.21.15.90:0/2442945666 conn(0x56017a8edfc0 msgr2=0x56017aa07580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:22.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.058+0000 7f5239600700 0 client.17943 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T08:01:22.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.060+0000 7f5260e9eb80 1 -- 172.21.15.90:0/2442945666 shutdown_connections 2024-05-27T08:01:22.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.063+0000 7f5260e9eb80 1 -- 172.21.15.90:0/2442945666 wait complete. 2024-05-27T08:01:22.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.063+0000 7f5260e9eb80 20 client.17943 trim_cache size 0 max 16384 2024-05-27T08:01:22.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.090+0000 7f5260e9eb80 1 Processor -- start 2024-05-27T08:01:22.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.090+0000 7f5260e9eb80 1 -- start start 2024-05-27T08:01:22.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.091+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 0x56017aa0c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.091+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017aa0cdb0 0x56017aa15590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.091+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 0x56017aa10920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.091+0000 7f5260e9eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5228010840 con 0x56017a88de40 2024-05-27T08:01:22.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.091+0000 7f5260e9eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f522800d890 con 0x56017aa0cdb0 2024-05-27T08:01:22.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.091+0000 7f5260e9eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f522800faa0 con 0x56017aa12d30 2024-05-27T08:01:22.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.092+0000 7f524be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 0x56017aa10920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.092+0000 7f524be00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 0x56017aa10920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:41534/0 (socket says 172.21.15.90:41534) 2024-05-27T08:01:22.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.092+0000 7f524be00700 1 -- 172.21.15.90:0/1613257343 learned_addr learned my addr 172.21.15.90:0/1613257343 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:22.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.092+0000 7f5242a00700 1 --2- 172.21.15.90:0/1613257343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017aa0cdb0 0x56017aa15590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.092+0000 7f524be00700 1 -- 172.21.15.90:0/1613257343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017aa0cdb0 msgr2=0x56017aa15590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:22.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.092+0000 7f524be00700 1 --2- 172.21.15.90:0/1613257343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017aa0cdb0 0x56017aa15590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.092+0000 7f524be00700 1 -- 172.21.15.90:0/1613257343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 msgr2=0x56017aa0c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:22.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.092+0000 7f524be00700 1 --2- 172.21.15.90:0/1613257343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 0x56017aa0c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.092+0000 7f524be00700 1 -- 172.21.15.90:0/1613257343 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017aa14ba0 con 0x56017aa12d30 2024-05-27T08:01:22.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.092+0000 7f5242a00700 1 --2- 172.21.15.90:0/1613257343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017aa0cdb0 0x56017aa15590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:22.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.095+0000 7f524be00700 1 --2- 172.21.15.90:0/1613257343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 0x56017aa10920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5244007db0 tx=0x7f5244011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3640c443532800f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:22.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.095+0000 7f523be00700 1 -- 172.21.15.90:0/1613257343 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244004830 con 0x56017aa12d30 2024-05-27T08:01:22.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.095+0000 7f523be00700 1 -- 172.21.15.90:0/1613257343 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5244005020 con 0x56017aa12d30 2024-05-27T08:01:22.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.095+0000 7f523be00700 1 -- 172.21.15.90:0/1613257343 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524401b6d0 con 0x56017aa12d30 2024-05-27T08:01:22.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.096+0000 7f5260e9eb80 1 -- 172.21.15.90:0/1613257343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 msgr2=0x56017aa10920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:22.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.097+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/1613257343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 0x56017aa10920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5244007db0 tx=0x7f5244011720 comp rx=0 tx=0).stop 2024-05-27T08:01:22.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.097+0000 7f5260e9eb80 1 -- 172.21.15.90:0/1613257343 shutdown_connections 2024-05-27T08:01:22.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.098+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/1613257343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa12d30 0x56017aa10920 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.098+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/1613257343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 0x56017aa0c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.098+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/1613257343 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017aa0cdb0 0x56017aa15590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.098+0000 7f5260e9eb80 1 -- 172.21.15.90:0/1613257343 >> 172.21.15.90:0/1613257343 conn(0x56017a8edfc0 msgr2=0x56017aa10210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:22.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.099+0000 7f5260e9eb80 1 -- 172.21.15.90:0/1613257343 shutdown_connections 2024-05-27T08:01:22.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.099+0000 7f5260e9eb80 1 -- 172.21.15.90:0/1613257343 wait complete. 2024-05-27T08:01:22.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.100+0000 7f5260e9eb80 1 -- start start 2024-05-27T08:01:22.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.102+0000 7f5260e9eb80 1 Processor -- start 2024-05-27T08:01:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.104+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 0x56017aa1ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.105+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa0cdb0 0x56017aa1f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.105+0000 7f5260e9eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017aa12d30 0x56017aa0b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.105+0000 7f524b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 0x56017aa1ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.105+0000 7f524b400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 0x56017aa1ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43842/0 (socket says 172.21.15.90:43842) 2024-05-27T08:01:22.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.105+0000 7f524b400700 1 -- 172.21.15.90:0/3019931074 learned_addr learned my addr 172.21.15.90:0/3019931074 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:22.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.106+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f522800f3d0 con 0x56017a88de40 2024-05-27T08:01:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.106+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f522800df60 con 0x56017aa12d30 2024-05-27T08:01:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.106+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f522800e630 con 0x56017aa0cdb0 2024-05-27T08:01:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.106+0000 7f5260e9eb80 10 client.0 _ll_register_callbacks cb 0x56017a8d4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.106+0000 7f524b400700 1 -- 172.21.15.90:0/3019931074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa0cdb0 msgr2=0x56017aa1f200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:01:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.106+0000 7f524b400700 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa0cdb0 0x56017aa1f200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.106+0000 7f524b400700 1 -- 172.21.15.90:0/3019931074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017aa12d30 msgr2=0x56017aa0b860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:01:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.106+0000 7f524b400700 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017aa12d30 0x56017aa0b860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.106+0000 7f524b400700 1 -- 172.21.15.90:0/3019931074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017aa0bea0 con 0x56017a88de40 2024-05-27T08:01:22.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.107+0000 7f524b400700 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 0x56017aa1ecc0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f524c10bc70 tx=0x7f523c004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f899385e44e95d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:22.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.107+0000 7f5239600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:01:22.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.107+0000 7f5239600700 1 -- 172.21.15.90:0/3019931074 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c007d30 con 0x56017a88de40 2024-05-27T08:01:22.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.107+0000 7f5239600700 1 -- 172.21.15.90:0/3019931074 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f523c02b450 con 0x56017a88de40 2024-05-27T08:01:22.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.107+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5244005020 con 0x56017a88de40 2024-05-27T08:01:22.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.108+0000 7f5239600700 1 -- 172.21.15.90:0/3019931074 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f523c004e50 con 0x56017a88de40 2024-05-27T08:01:22.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.108+0000 7f5260e9eb80 10 client.17946 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:22.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.108+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56017aa90970 con 0x56017a88de40 2024-05-27T08:01:22.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.109+0000 7f5239600700 1 -- 172.21.15.90:0/3019931074 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5244005020 con 0x56017a88de40 2024-05-27T08:01:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.109+0000 7f5239600700 1 -- 172.21.15.90:0/3019931074 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f523c028040 con 0x56017a88de40 2024-05-27T08:01:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.110+0000 7f5260e9eb80 10 client.17946 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.110+0000 7f5260e9eb80 10 client.17946 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T08:01:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.110+0000 7f5260e9eb80 20 client.17946 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.110+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56017aa91060 0x56017aa93480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.110+0000 7f5260e9eb80 4 client.17946 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:01:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.110+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56017a936570 con 0x56017aa91060 2024-05-27T08:01:22.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.111+0000 7f5242a00700 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56017aa91060 0x56017aa93480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.112+0000 7f5242a00700 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56017aa91060 0x56017aa93480 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bbbc5f672948582e server_cookie=6a8b9208381403d8 in_seq=0 out_seq=0 2024-05-27T08:01:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.112+0000 7f5239600700 10 client.17946 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T08:01:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.113+0000 7f5239600700 1 -- 172.21.15.90:0/3019931074 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56017a936570 con 0x56017aa91060 2024-05-27T08:01:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.113+0000 7f5239600700 10 client.17946 handle_command_reply: tid=0 2024-05-27T08:01:22.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.113+0000 7f5260e9eb80 1 client.17946 shutdown 2024-05-27T08:01:22.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.114+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 msgr2=0x56017aa1ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:22.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.114+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 0x56017aa1ecc0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f524c10bc70 tx=0x7f523c004d30 comp rx=0 tx=0).stop 2024-05-27T08:01:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.114+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 shutdown_connections 2024-05-27T08:01:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.114+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x56017aa91060 0x56017aa93480 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.114+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56017aa12d30 0x56017aa0b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.114+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56017a88de40 0x56017aa1ecc0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.114+0000 7f5260e9eb80 1 --2- 172.21.15.90:0/3019931074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56017aa0cdb0 0x56017aa1f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.114+0000 7f5239600700 0 client.17946 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T08:01:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.114+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 >> 172.21.15.90:0/3019931074 conn(0x56017a8edfc0 msgr2=0x56017aa16410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:22.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.115+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 shutdown_connections 2024-05-27T08:01:22.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.115+0000 7f5260e9eb80 1 -- 172.21.15.90:0/3019931074 wait complete. 2024-05-27T08:01:22.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.115+0000 7f5260e9eb80 20 client.17946 trim_cache size 0 max 16384 2024-05-27T08:01:22.121 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.117+0000 7f5252400700 1 -- 172.21.15.90:0/2037556778 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f52300b2af0 msgr2=0x7f52300b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.117+0000 7f5252400700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f52300b2af0 0x7f52300b5000 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f523c00b780 tx=0x7f523c0099e0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.117+0000 7f5252400700 1 -- 172.21.15.90:0/2037556778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c10b270 msgr2=0x7f524c114f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.117+0000 7f5252400700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c10b270 0x7f524c114f40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f523400e9a0 tx=0x7f523400ed60 comp rx=0 tx=0).stop 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.119+0000 7f5252400700 1 -- 172.21.15.90:0/2037556778 shutdown_connections 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.119+0000 7f5252400700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524c111bc0 0x7f524c115480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.119+0000 7f5252400700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f52300b2af0 0x7f52300b5000 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.119+0000 7f5252400700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f524c10a920 0x7f524c114a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.119+0000 7f5252400700 1 --2- 172.21.15.90:0/2037556778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c10b270 0x7f524c114f40 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.119+0000 7f5252400700 1 -- 172.21.15.90:0/2037556778 >> 172.21.15.90:0/2037556778 conn(0x7f524c06f4c0 msgr2=0x7f524c118d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:22.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.120+0000 7f5252400700 1 -- 172.21.15.90:0/2037556778 shutdown_connections 2024-05-27T08:01:22.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:22.120+0000 7f5252400700 1 -- 172.21.15.90:0/2037556778 wait complete. 2024-05-27T08:01:23.225 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:01:23.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:23 smithi090 ceph-mon[31429]: pgmap v481: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 MiB/s wr, 516 op/s 2024-05-27T08:01:23.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:23 smithi136 ceph-mon[35324]: pgmap v481: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 MiB/s wr, 516 op/s 2024-05-27T08:01:23.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:23 smithi129 ceph-mon[35254]: pgmap v481: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 MiB/s wr, 516 op/s 2024-05-27T08:01:23.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 665.0 s, 553.0 tps, lat 191.935 ms stddev 112.609 2024-05-27T08:01:24.012 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:25.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:25 smithi090 ceph-mon[31429]: pgmap v482: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.8 MiB/s wr, 547 op/s 2024-05-27T08:01:25.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:25 smithi136 ceph-mon[35324]: pgmap v482: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.8 MiB/s wr, 547 op/s 2024-05-27T08:01:25.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:25 smithi129 ceph-mon[35254]: pgmap v482: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.8 MiB/s wr, 547 op/s 2024-05-27T08:01:27.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:27 smithi090 ceph-mon[31429]: pgmap v483: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.6 MiB/s wr, 548 op/s 2024-05-27T08:01:27.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:27 smithi136 ceph-mon[35324]: pgmap v483: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.6 MiB/s wr, 548 op/s 2024-05-27T08:01:27.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:27 smithi129 ceph-mon[35254]: pgmap v483: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 6.6 MiB/s wr, 548 op/s 2024-05-27T08:01:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 670.0 s, 451.3 tps, lat 221.421 ms stddev 295.081 2024-05-27T08:01:29.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:29 smithi090 ceph-mon[31429]: pgmap v484: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.1 MiB/s wr, 507 op/s 2024-05-27T08:01:29.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:29 smithi136 ceph-mon[35324]: pgmap v484: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.1 MiB/s wr, 507 op/s 2024-05-27T08:01:29.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:29 smithi129 ceph-mon[35254]: pgmap v484: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.1 MiB/s wr, 507 op/s 2024-05-27T08:01:31.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:31 smithi090 ceph-mon[31429]: pgmap v485: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.1 MiB/s wr, 678 op/s 2024-05-27T08:01:31.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:31 smithi136 ceph-mon[35324]: pgmap v485: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.1 MiB/s wr, 678 op/s 2024-05-27T08:01:31.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:31 smithi129 ceph-mon[35254]: pgmap v485: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 8.1 MiB/s wr, 678 op/s 2024-05-27T08:01:32.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.870+0000 7f63d5800700 1 -- 172.21.15.90:0/1166131006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63d0108f10 msgr2=0x7f63d0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:32.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.870+0000 7f63d5800700 1 --2- 172.21.15.90:0/1166131006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63d0108f10 0x7f63d0109320 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009990 tx=0x7f63bc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.871+0000 7f63d5800700 1 -- 172.21.15.90:0/1166131006 shutdown_connections 2024-05-27T08:01:32.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.871+0000 7f63d5800700 1 --2- 172.21.15.90:0/1166131006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d0109860 0x7f63d01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.871+0000 7f63d5800700 1 --2- 172.21.15.90:0/1166131006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d01045e0 0x7f63d01049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.871+0000 7f63d5800700 1 --2- 172.21.15.90:0/1166131006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63d0108f10 0x7f63d0109320 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.871+0000 7f63d5800700 1 -- 172.21.15.90:0/1166131006 >> 172.21.15.90:0/1166131006 conn(0x7f63d0100030 msgr2=0x7f63d0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:32.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.871+0000 7f63d5800700 1 -- 172.21.15.90:0/1166131006 shutdown_connections 2024-05-27T08:01:32.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.871+0000 7f63d5800700 1 -- 172.21.15.90:0/1166131006 wait complete. 2024-05-27T08:01:32.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.873+0000 7f63d5800700 1 Processor -- start 2024-05-27T08:01:32.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.873+0000 7f63d5800700 1 -- start start 2024-05-27T08:01:32.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63d5800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63d01045e0 0x7f63d019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63d5800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d0108f10 0x7f63d019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63d5800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d0109860 0x7f63d01a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63d5800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f63d0076fa0 con 0x7f63d0109860 2024-05-27T08:01:32.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63d5800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63d0076e20 con 0x7f63d01045e0 2024-05-27T08:01:32.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63d5800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63d0077120 con 0x7f63d0108f10 2024-05-27T08:01:32.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63cea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63d01045e0 0x7f63d019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63cea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63d01045e0 0x7f63d019d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48576/0 (socket says 172.21.15.90:48576) 2024-05-27T08:01:32.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63cea00700 1 -- 172.21.15.90:0/2000968822 learned_addr learned my addr 172.21.15.90:0/2000968822 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:32.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63cf400700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d0109860 0x7f63d01a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.874+0000 7f63ce000700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d0108f10 0x7f63d019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.875+0000 7f63cea00700 1 -- 172.21.15.90:0/2000968822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d0108f10 msgr2=0x7f63d019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:32.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.875+0000 7f63cea00700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d0108f10 0x7f63d019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.875+0000 7f63cea00700 1 -- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d0109860 msgr2=0x7f63d01a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:32.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.875+0000 7f63cea00700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d0109860 0x7f63d01a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.875+0000 7f63cea00700 1 -- 172.21.15.90:0/2000968822 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d01a1f30 con 0x7f63d01045e0 2024-05-27T08:01:32.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.875+0000 7f63cf400700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d0109860 0x7f63d01a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:32.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.875+0000 7f63ce000700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d0108f10 0x7f63d019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:32.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.876+0000 7f63cea00700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63d01045e0 0x7f63d019d060 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f63c000b530 tx=0x7f63c000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:32.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.876+0000 7f63c7400700 1 -- 172.21.15.90:0/2000968822 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0015020 con 0x7f63d01045e0 2024-05-27T08:01:32.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.877+0000 7f63c7400700 1 -- 172.21.15.90:0/2000968822 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63c0011cc0 con 0x7f63d01045e0 2024-05-27T08:01:32.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.877+0000 7f63d5800700 1 -- 172.21.15.90:0/2000968822 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d01a2220 con 0x7f63d01045e0 2024-05-27T08:01:32.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.877+0000 7f63c7400700 1 -- 172.21.15.90:0/2000968822 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0010d30 con 0x7f63d01045e0 2024-05-27T08:01:32.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.877+0000 7f63d5800700 1 -- 172.21.15.90:0/2000968822 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d01a9b40 con 0x7f63d01045e0 2024-05-27T08:01:32.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.884+0000 7f63c7400700 1 -- 172.21.15.90:0/2000968822 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f63c0018020 con 0x7f63d01045e0 2024-05-27T08:01:32.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.887+0000 7f63c7400700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63b40b2af0 0x7f63b40b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.887+0000 7f63c7400700 1 -- 172.21.15.90:0/2000968822 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f63c00e2100 con 0x7f63d01045e0 2024-05-27T08:01:32.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.887+0000 7f63ce000700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63b40b2af0 0x7f63b40b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.889+0000 7f63ce000700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63b40b2af0 0x7f63b40b5000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f63d010a8e0 tx=0x7f63bc005fd0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:32.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.906+0000 7f63e4307b80 1 Processor -- start 2024-05-27T08:01:32.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.907+0000 7f63e4307b80 1 -- start start 2024-05-27T08:01:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.907+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c7a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.907+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c7ad30 0x562ea8c74870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.907+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c74db0 0x562ea8c751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.907+0000 7f63e4307b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ea8ba2e40 con 0x562ea8c7ad30 2024-05-27T08:01:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.907+0000 7f63e4307b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ea8b929e0 con 0x562ea8c74db0 2024-05-27T08:01:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.907+0000 7f63e4307b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562ea8b96a20 con 0x562ea8af5e40 2024-05-27T08:01:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.908+0000 7f63cea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c7a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.908+0000 7f63cea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c7a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46312/0 (socket says 172.21.15.90:46312) 2024-05-27T08:01:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.908+0000 7f63cea00700 1 -- 172.21.15.90:0/1183348278 learned_addr learned my addr 172.21.15.90:0/1183348278 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:32.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.908+0000 7f63ce000700 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c7ad30 0x562ea8c74870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.908+0000 7f63cf400700 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c74db0 0x562ea8c751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.909+0000 7f63cea00700 1 -- 172.21.15.90:0/1183348278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c74db0 msgr2=0x562ea8c751c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:32.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.909+0000 7f63cea00700 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c74db0 0x562ea8c751c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.909+0000 7f63cea00700 1 -- 172.21.15.90:0/1183348278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c7ad30 msgr2=0x562ea8c74870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:32.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.909+0000 7f63cea00700 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c7ad30 0x562ea8c74870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.909+0000 7f63cea00700 1 -- 172.21.15.90:0/1183348278 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea8b5b420 con 0x562ea8af5e40 2024-05-27T08:01:32.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.909+0000 7f63cf400700 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c74db0 0x562ea8c751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:32.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.909+0000 7f63ce000700 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c7ad30 0x562ea8c74870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:01:32.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.910+0000 7f63cea00700 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c7a7f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f63c00a5140 tx=0x7f63c00a88b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80a075f8cac292d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:32.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.910+0000 7f63bbe00700 1 -- 172.21.15.90:0/1183348278 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c00c3070 con 0x562ea8af5e40 2024-05-27T08:01:32.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.910+0000 7f63bbe00700 1 -- 172.21.15.90:0/1183348278 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63c00a97c0 con 0x562ea8af5e40 2024-05-27T08:01:32.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.910+0000 7f63bbe00700 1 -- 172.21.15.90:0/1183348278 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c00df6a0 con 0x562ea8af5e40 2024-05-27T08:01:32.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.912+0000 7f63e4307b80 1 -- 172.21.15.90:0/1183348278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 msgr2=0x562ea8c7a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:32.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.912+0000 7f63e4307b80 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c7a7f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f63c00a5140 tx=0x7f63c00a88b0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.912+0000 7f63e4307b80 1 -- 172.21.15.90:0/1183348278 shutdown_connections 2024-05-27T08:01:32.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.912+0000 7f63e4307b80 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c74db0 0x562ea8c751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.912+0000 7f63e4307b80 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c7a7f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.913+0000 7f63e4307b80 1 --2- 172.21.15.90:0/1183348278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c7ad30 0x562ea8c74870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.913+0000 7f63e4307b80 1 -- 172.21.15.90:0/1183348278 >> 172.21.15.90:0/1183348278 conn(0x562ea8b55fc0 msgr2=0x562ea8c6f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:32.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.913+0000 7f63e4307b80 1 -- 172.21.15.90:0/1183348278 shutdown_connections 2024-05-27T08:01:32.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.913+0000 7f63e4307b80 1 -- 172.21.15.90:0/1183348278 wait complete. 2024-05-27T08:01:32.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.914+0000 7f63e4307b80 1 -- start start 2024-05-27T08:01:32.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.915+0000 7f63e4307b80 1 Processor -- start 2024-05-27T08:01:32.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.915+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c7e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.915+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 0x562ea8c7ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.915+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.916+0000 7f63e4307b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ea8a98fa0 con 0x562ea8c74db0 2024-05-27T08:01:32.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.916+0000 7f63e4307b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ea8b33760 con 0x562ea8c7ad30 2024-05-27T08:01:32.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.916+0000 7f63e4307b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562ea8c92810 con 0x562ea8af5e40 2024-05-27T08:01:32.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.916+0000 7f63e4307b80 10 client.0 _ll_register_callbacks cb 0x562ea8b3cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:32.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.916+0000 7f63cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.916+0000 7f63cea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c7e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.916+0000 7f63cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48592/0 (socket says 172.21.15.90:48592) 2024-05-27T08:01:32.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.916+0000 7f63cf400700 1 -- 172.21.15.90:0/2863660461 learned_addr learned my addr 172.21.15.90:0/2863660461 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:32.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.916+0000 7f63ce000700 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 0x562ea8c7ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.917+0000 7f63cf400700 1 -- 172.21.15.90:0/2863660461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 msgr2=0x562ea8c7e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:32.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.917+0000 7f63cf400700 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c7e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.917+0000 7f63cf400700 1 -- 172.21.15.90:0/2863660461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 msgr2=0x562ea8c7ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:32.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.917+0000 7f63cf400700 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 0x562ea8c7ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.917+0000 7f63cf400700 1 -- 172.21.15.90:0/2863660461 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea8ae28d0 con 0x562ea8c7ad30 2024-05-27T08:01:32.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.918+0000 7f63ce000700 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 0x562ea8c7ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:32.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.918+0000 7f63cf400700 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7efc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f63c8007f00 tx=0x7f63c8012240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6638971401ac98e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:32.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.918+0000 7f63b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:01:32.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.918+0000 7f63b8c00700 1 -- 172.21.15.90:0/2863660461 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c8012d80 con 0x562ea8c7ad30 2024-05-27T08:01:32.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.918+0000 7f63b8c00700 1 -- 172.21.15.90:0/2863660461 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f63c8016420 con 0x562ea8c7ad30 2024-05-27T08:01:32.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.918+0000 7f63e4307b80 1 -- 172.21.15.90:0/2863660461 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63c00a97c0 con 0x562ea8c7ad30 2024-05-27T08:01:32.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.919+0000 7f63b8c00700 1 -- 172.21.15.90:0/2863660461 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f63c8016620 con 0x562ea8c7ad30 2024-05-27T08:01:32.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.919+0000 7f63e4307b80 10 client.37592 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:32.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.919+0000 7f63e4307b80 1 -- 172.21.15.90:0/2863660461 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562ea8b63060 con 0x562ea8c7ad30 2024-05-27T08:01:32.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.919+0000 7f63b8c00700 1 -- 172.21.15.90:0/2863660461 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63c00a97c0 con 0x562ea8c7ad30 2024-05-27T08:01:32.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.920+0000 7f63b8c00700 1 -- 172.21.15.90:0/2863660461 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f63c8027020 con 0x562ea8c7ad30 2024-05-27T08:01:32.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.920+0000 7f63e4307b80 10 client.37592 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:32.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.920+0000 7f63e4307b80 10 client.37592 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:01:32.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.920+0000 7f63e4307b80 20 client.37592 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:32.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.920+0000 7f63e4307b80 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x562ea8c9f730 0x562ea8ca1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.920+0000 7f63e4307b80 4 client.37592 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:32.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.920+0000 7f63e4307b80 1 -- 172.21.15.90:0/2863660461 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ea8ba4d90 con 0x562ea8c9f730 2024-05-27T08:01:32.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.921+0000 7f63cea00700 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x562ea8c9f730 0x562ea8ca1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.922+0000 7f63cea00700 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x562ea8c9f730 0x562ea8ca1b80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f9e3e2f7d92a717 server_cookie=a139ffa648988a8d in_seq=0 out_seq=0 2024-05-27T08:01:32.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.922+0000 7f63b8c00700 10 client.37592 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T08:01:32.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.923+0000 7f63b8c00700 1 -- 172.21.15.90:0/2863660461 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562ea8ba4d90 con 0x562ea8c9f730 2024-05-27T08:01:32.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.923+0000 7f63b8c00700 10 client.37592 handle_command_reply: tid=0 2024-05-27T08:01:32.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.924+0000 7f63e4307b80 1 client.37592 shutdown 2024-05-27T08:01:32.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.925+0000 7f63e4307b80 1 -- 172.21.15.90:0/2863660461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 msgr2=0x562ea8c7efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:32.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.925+0000 7f63e4307b80 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7efc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f63c8007f00 tx=0x7f63c8012240 comp rx=0 tx=0).stop 2024-05-27T08:01:32.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.926+0000 7f63e4307b80 1 -- 172.21.15.90:0/2863660461 shutdown_connections 2024-05-27T08:01:32.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.926+0000 7f63e4307b80 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x562ea8c9f730 0x562ea8ca1b80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.926+0000 7f63e4307b80 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7efc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.926+0000 7f63e4307b80 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c7e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.926+0000 7f63e4307b80 1 --2- 172.21.15.90:0/2863660461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 0x562ea8c7ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.926+0000 7f63e4307b80 1 -- 172.21.15.90:0/2863660461 >> 172.21.15.90:0/2863660461 conn(0x562ea8b55fc0 msgr2=0x562ea8c6f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:32.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.926+0000 7f63b8c00700 0 client.37592 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T08:01:32.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.926+0000 7f63e4307b80 1 -- 172.21.15.90:0/2863660461 shutdown_connections 2024-05-27T08:01:32.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.926+0000 7f63e4307b80 1 -- 172.21.15.90:0/2863660461 wait complete. 2024-05-27T08:01:32.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.927+0000 7f63e4307b80 20 client.37592 trim_cache size 0 max 16384 2024-05-27T08:01:32.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.957+0000 7f63e4307b80 1 Processor -- start 2024-05-27T08:01:32.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.958+0000 7f63e4307b80 1 -- start start 2024-05-27T08:01:32.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.958+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c74870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.958+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 0x562ea8c79dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.958+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.958+0000 7f63e4307b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f63d01abb80 con 0x562ea8c74db0 2024-05-27T08:01:32.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.958+0000 7f63e4307b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63d0074830 con 0x562ea8c7ad30 2024-05-27T08:01:32.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.958+0000 7f63e4307b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63d01aade0 con 0x562ea8af5e40 2024-05-27T08:01:32.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.959+0000 7f63cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.959+0000 7f63cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48596/0 (socket says 172.21.15.90:48596) 2024-05-27T08:01:32.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.959+0000 7f63cf400700 1 -- 172.21.15.90:0/1348393140 learned_addr learned my addr 172.21.15.90:0/1348393140 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:32.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.959+0000 7f63cea00700 1 --2- 172.21.15.90:0/1348393140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c74870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.960+0000 7f63ce000700 1 --2- 172.21.15.90:0/1348393140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 0x562ea8c79dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.960+0000 7f63cf400700 1 -- 172.21.15.90:0/1348393140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 msgr2=0x562ea8c74870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:32.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.960+0000 7f63cf400700 1 --2- 172.21.15.90:0/1348393140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c74870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.960+0000 7f63cf400700 1 -- 172.21.15.90:0/1348393140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 msgr2=0x562ea8c79dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:32.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.961+0000 7f63cf400700 1 --2- 172.21.15.90:0/1348393140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 0x562ea8c79dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.961+0000 7f63cf400700 1 -- 172.21.15.90:0/1348393140 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea8c7cab0 con 0x562ea8c7ad30 2024-05-27T08:01:32.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.961+0000 7f63ce000700 1 --2- 172.21.15.90:0/1348393140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 0x562ea8c79dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:01:32.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.962+0000 7f63cf400700 1 --2- 172.21.15.90:0/1348393140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7e320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f63c8007ed0 tx=0x7f63c8012640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4fb218ab2c447f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:32.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.962+0000 7f63bb400700 1 -- 172.21.15.90:0/1348393140 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c801fd10 con 0x562ea8c7ad30 2024-05-27T08:01:32.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.962+0000 7f63bb400700 1 -- 172.21.15.90:0/1348393140 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63c801f430 con 0x562ea8c7ad30 2024-05-27T08:01:32.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.963+0000 7f63e4307b80 1 -- 172.21.15.90:0/1348393140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 msgr2=0x562ea8c7e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:32.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.963+0000 7f63e4307b80 1 --2- 172.21.15.90:0/1348393140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7e320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f63c8007ed0 tx=0x7f63c8012640 comp rx=0 tx=0).stop 2024-05-27T08:01:32.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.964+0000 7f63e4307b80 1 -- 172.21.15.90:0/1348393140 shutdown_connections 2024-05-27T08:01:32.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.964+0000 7f63e4307b80 1 --2- 172.21.15.90:0/1348393140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c7e320 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.964+0000 7f63e4307b80 1 --2- 172.21.15.90:0/1348393140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8af5e40 0x562ea8c74870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.964+0000 7f63e4307b80 1 --2- 172.21.15.90:0/1348393140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8c74db0 0x562ea8c79dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.964+0000 7f63e4307b80 1 -- 172.21.15.90:0/1348393140 >> 172.21.15.90:0/1348393140 conn(0x562ea8b55fc0 msgr2=0x562ea8c796b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:32.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.965+0000 7f63e4307b80 1 -- 172.21.15.90:0/1348393140 shutdown_connections 2024-05-27T08:01:32.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.965+0000 7f63e4307b80 1 -- 172.21.15.90:0/1348393140 wait complete. 2024-05-27T08:01:32.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.966+0000 7f63e4307b80 1 -- start start 2024-05-27T08:01:32.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.966+0000 7f63e4307b80 1 Processor -- start 2024-05-27T08:01:32.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.967+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8af5e40 0x562ea8ca79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.967+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8c74db0 0x562ea8c97aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.967+0000 7f63e4307b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c97fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.967+0000 7f63e4307b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f63d01aa710 con 0x562ea8af5e40 2024-05-27T08:01:32.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.967+0000 7f63e4307b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63d0074f00 con 0x562ea8c7ad30 2024-05-27T08:01:32.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.967+0000 7f63e4307b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63d00755d0 con 0x562ea8c74db0 2024-05-27T08:01:32.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.967+0000 7f63e4307b80 10 client.0 _ll_register_callbacks cb 0x562ea8b3cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:32.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.968+0000 7f63cea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8af5e40 0x562ea8ca79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.968+0000 7f63cea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8af5e40 0x562ea8ca79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34372/0 (socket says 172.21.15.90:34372) 2024-05-27T08:01:32.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.968+0000 7f63cea00700 1 -- 172.21.15.90:0/3663682903 learned_addr learned my addr 172.21.15.90:0/3663682903 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:32.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.968+0000 7f63ce000700 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8c74db0 0x562ea8c97aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.969+0000 7f63cf400700 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c97fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.970+0000 7f63cf400700 1 -- 172.21.15.90:0/3663682903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8c74db0 msgr2=0x562ea8c97aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:32.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.971+0000 7f63cf400700 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8c74db0 0x562ea8c97aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.971+0000 7f63cf400700 1 -- 172.21.15.90:0/3663682903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8af5e40 msgr2=0x562ea8ca79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:32.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.971+0000 7f63cf400700 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8af5e40 0x562ea8ca79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.971+0000 7f63cf400700 1 -- 172.21.15.90:0/3663682903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea8c98680 con 0x562ea8c7ad30 2024-05-27T08:01:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.972+0000 7f63cf400700 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c97fe0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x562ea8c70f70 tx=0x7f63c8009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23c556805283fca server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.972+0000 7f63b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:01:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.973+0000 7f63b8c00700 1 -- 172.21.15.90:0/3663682903 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c8003e40 con 0x562ea8c7ad30 2024-05-27T08:01:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.973+0000 7f63e4307b80 1 -- 172.21.15.90:0/3663682903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63c801f430 con 0x562ea8c7ad30 2024-05-27T08:01:32.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.973+0000 7f63b8c00700 1 -- 172.21.15.90:0/3663682903 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f63c8004a70 con 0x562ea8c7ad30 2024-05-27T08:01:32.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.973+0000 7f63b8c00700 1 -- 172.21.15.90:0/3663682903 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f63c8045050 con 0x562ea8c7ad30 2024-05-27T08:01:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.974+0000 7f63e4307b80 10 client.37598 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.974+0000 7f63e4307b80 1 -- 172.21.15.90:0/3663682903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562ea8d11a80 con 0x562ea8c7ad30 2024-05-27T08:01:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.974+0000 7f63b8c00700 1 -- 172.21.15.90:0/3663682903 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63c801f430 con 0x562ea8c7ad30 2024-05-27T08:01:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.976+0000 7f63b8c00700 1 -- 172.21.15.90:0/3663682903 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f63c8027070 con 0x562ea8c7ad30 2024-05-27T08:01:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.976+0000 7f63e4307b80 10 client.37598 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.976+0000 7f63e4307b80 10 client.37598 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:01:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.976+0000 7f63e4307b80 20 client.37598 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.976+0000 7f63e4307b80 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x562ea8d12080 0x562ea8d144a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.977+0000 7f63e4307b80 4 client.37598 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:01:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.977+0000 7f63e4307b80 1 -- 172.21.15.90:0/3663682903 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ea8b9e570 con 0x562ea8d12080 2024-05-27T08:01:32.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.978+0000 7f63cea00700 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x562ea8d12080 0x562ea8d144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:32.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.979+0000 7f63b8c00700 10 client.37598 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T08:01:32.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.979+0000 7f63cea00700 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x562ea8d12080 0x562ea8d144a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d3c545552998773 server_cookie=5e48c97a80255438 in_seq=0 out_seq=0 2024-05-27T08:01:32.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.980+0000 7f63b8c00700 1 -- 172.21.15.90:0/3663682903 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ea8b9e570 con 0x562ea8d12080 2024-05-27T08:01:32.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.980+0000 7f63b8c00700 10 client.37598 handle_command_reply: tid=0 2024-05-27T08:01:32.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.980+0000 7f63e4307b80 1 client.37598 shutdown 2024-05-27T08:01:32.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.981+0000 7f63e4307b80 1 -- 172.21.15.90:0/3663682903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 msgr2=0x562ea8c97fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:32.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.981+0000 7f63e4307b80 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c97fe0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x562ea8c70f70 tx=0x7f63c8009a10 comp rx=0 tx=0).stop 2024-05-27T08:01:32.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.983+0000 7f63e4307b80 1 -- 172.21.15.90:0/3663682903 shutdown_connections 2024-05-27T08:01:32.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.983+0000 7f63e4307b80 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x562ea8d12080 0x562ea8d144a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.983+0000 7f63e4307b80 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ea8c7ad30 0x562ea8c97fe0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.983+0000 7f63e4307b80 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ea8af5e40 0x562ea8ca79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.983+0000 7f63e4307b80 1 --2- 172.21.15.90:0/3663682903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562ea8c74db0 0x562ea8c97aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.983+0000 7f63b8c00700 0 client.37598 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T08:01:32.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.983+0000 7f63e4307b80 1 -- 172.21.15.90:0/3663682903 >> 172.21.15.90:0/3663682903 conn(0x562ea8b55fc0 msgr2=0x562ea8c7b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:32.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.983+0000 7f63e4307b80 1 -- 172.21.15.90:0/3663682903 shutdown_connections 2024-05-27T08:01:32.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.983+0000 7f63e4307b80 1 -- 172.21.15.90:0/3663682903 wait complete. 2024-05-27T08:01:32.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.984+0000 7f63e4307b80 20 client.37598 trim_cache size 0 max 16384 2024-05-27T08:01:32.985 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:01:32.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.986+0000 7f63d5800700 1 -- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63b40b2af0 msgr2=0x7f63b40b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:32.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.986+0000 7f63d5800700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63b40b2af0 0x7f63b40b5000 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f63d010a8e0 tx=0x7f63bc005fd0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.986+0000 7f63d5800700 1 -- 172.21.15.90:0/2000968822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63d01045e0 msgr2=0x7f63d019d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:32.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.986+0000 7f63d5800700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63d01045e0 0x7f63d019d060 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f63c000b530 tx=0x7f63c000b8f0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.986+0000 7f63d5800700 1 -- 172.21.15.90:0/2000968822 shutdown_connections 2024-05-27T08:01:32.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.986+0000 7f63d5800700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63d0109860 0x7f63d01a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.986+0000 7f63d5800700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63b40b2af0 0x7f63b40b5000 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.986+0000 7f63d5800700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63d01045e0 0x7f63d019d060 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.986+0000 7f63d5800700 1 --2- 172.21.15.90:0/2000968822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63d0108f10 0x7f63d019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:32.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.986+0000 7f63d5800700 1 -- 172.21.15.90:0/2000968822 >> 172.21.15.90:0/2000968822 conn(0x7f63d0100030 msgr2=0x7f63d0106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:32.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.987+0000 7f63d5800700 1 -- 172.21.15.90:0/2000968822 shutdown_connections 2024-05-27T08:01:32.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:32.987+0000 7f63d5800700 1 -- 172.21.15.90:0/2000968822 wait complete. 2024-05-27T08:01:33.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:33 smithi136 ceph-mon[35324]: pgmap v486: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 MiB/s wr, 492 op/s 2024-05-27T08:01:33.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:33 smithi129 ceph-mon[35254]: pgmap v486: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 MiB/s wr, 492 op/s 2024-05-27T08:01:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 675.0 s, 271.2 tps, lat 354.409 ms stddev 442.030 2024-05-27T08:01:33.895 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:33 smithi090 ceph-mon[31429]: pgmap v486: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.7 MiB/s wr, 492 op/s 2024-05-27T08:01:33.959 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:01:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.254+0000 7fb383600640 1 -- 172.21.15.90:0/1817231452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c108cb0 msgr2=0x7fb37c1018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.254+0000 7fb383600640 1 --2- 172.21.15.90:0/1817231452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c108cb0 0x7fb37c1018c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb370009900 tx=0x7fb37002a480 comp rx=0 tx=0).stop 2024-05-27T08:01:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.255+0000 7fb383600640 1 -- 172.21.15.90:0/1817231452 shutdown_connections 2024-05-27T08:01:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.255+0000 7fb383600640 1 --2- 172.21.15.90:0/1817231452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb37c10e550 0x7fb37c110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.255+0000 7fb383600640 1 --2- 172.21.15.90:0/1817231452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb37c101e00 0x7fb37c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.255+0000 7fb383600640 1 --2- 172.21.15.90:0/1817231452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c108cb0 0x7fb37c1018c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.255+0000 7fb383600640 1 -- 172.21.15.90:0/1817231452 >> 172.21.15.90:0/1817231452 conn(0x7fb37c0fd6a0 msgr2=0x7fb37c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.256+0000 7fb383600640 1 -- 172.21.15.90:0/1817231452 shutdown_connections 2024-05-27T08:01:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.256+0000 7fb383600640 1 -- 172.21.15.90:0/1817231452 wait complete. 2024-05-27T08:01:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.258+0000 7fb383600640 1 Processor -- start 2024-05-27T08:01:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.258+0000 7fb383600640 1 -- start start 2024-05-27T08:01:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.258+0000 7fb383600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb37c101e00 0x7fb37c073fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.258+0000 7fb383600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c108cb0 0x7fb37c0744e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.258+0000 7fb383600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb37c10e550 0x7fb37c1032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.258+0000 7fb383600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c113d30 con 0x7fb37c10e550 2024-05-27T08:01:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.258+0000 7fb383600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c113bb0 con 0x7fb37c101e00 2024-05-27T08:01:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.258+0000 7fb383600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c113eb0 con 0x7fb37c108cb0 2024-05-27T08:01:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.260+0000 7fb37be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c108cb0 0x7fb37c0744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.260+0000 7fb380c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb37c101e00 0x7fb37c073fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.260+0000 7fb37be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c108cb0 0x7fb37c0744e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46350/0 (socket says 172.21.15.90:46350) 2024-05-27T08:01:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.260+0000 7fb37be00640 1 -- 172.21.15.90:0/3129588547 learned_addr learned my addr 172.21.15.90:0/3129588547 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.261+0000 7fb381600640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb37c10e550 0x7fb37c1032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.261+0000 7fb37be00640 1 -- 172.21.15.90:0/3129588547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb37c101e00 msgr2=0x7fb37c073fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.261+0000 7fb37be00640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb37c101e00 0x7fb37c073fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.261+0000 7fb37be00640 1 -- 172.21.15.90:0/3129588547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb37c10e550 msgr2=0x7fb37c1032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.261+0000 7fb37be00640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb37c10e550 0x7fb37c1032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.262+0000 7fb37be00640 1 -- 172.21.15.90:0/3129588547 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c074b50 con 0x7fb37c108cb0 2024-05-27T08:01:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.263+0000 7fb37be00640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c108cb0 0x7fb37c0744e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb36800d950 tx=0x7fb36800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.264+0000 7fb379600640 1 -- 172.21.15.90:0/3129588547 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368017070 con 0x7fb37c108cb0 2024-05-27T08:01:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.264+0000 7fb379600640 1 -- 172.21.15.90:0/3129588547 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb368009c30 con 0x7fb37c108cb0 2024-05-27T08:01:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.264+0000 7fb379600640 1 -- 172.21.15.90:0/3129588547 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36800f620 con 0x7fb37c108cb0 2024-05-27T08:01:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.264+0000 7fb383600640 1 -- 172.21.15.90:0/3129588547 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c103820 con 0x7fb37c108cb0 2024-05-27T08:01:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.265+0000 7fb383600640 1 -- 172.21.15.90:0/3129588547 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c103d20 con 0x7fb37c108cb0 2024-05-27T08:01:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.271+0000 7fb38a5f2740 1 Processor -- start 2024-05-27T08:01:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.271+0000 7fb38a5f2740 1 -- start start 2024-05-27T08:01:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.272+0000 7fb38a5f2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e26a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.272+0000 7fb379600640 1 -- 172.21.15.90:0/3129588547 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb368010460 con 0x7fb37c108cb0 2024-05-27T08:01:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.274+0000 7fb381600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e26a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.274+0000 7fb381600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e26a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46364/0 (socket says 172.21.15.90:46364) 2024-05-27T08:01:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.274+0000 7fb381600640 1 -- 172.21.15.90:0/450521486 learned_addr learned my addr 172.21.15.90:0/450521486 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.274+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/450521486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124de9f2f0 0x56124e26ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.274+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/450521486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dea74d0 0x56124e26b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.274+0000 7fb38a5f2740 1 -- 172.21.15.90:0/450521486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56124e1dbb90 con 0x56124de9f2f0 2024-05-27T08:01:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.274+0000 7fb38a5f2740 1 -- 172.21.15.90:0/450521486 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56124e1cb730 con 0x56124dea74d0 2024-05-27T08:01:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.274+0000 7fb38a5f2740 1 -- 172.21.15.90:0/450521486 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56124e1cf770 con 0x56124dec81b0 2024-05-27T08:01:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.275+0000 7fb380c00640 1 --2- 172.21.15.90:0/450521486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124de9f2f0 0x56124e26ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.275+0000 7fb381600640 1 -- 172.21.15.90:0/450521486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dea74d0 msgr2=0x56124e26b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.275+0000 7fb381600640 1 --2- 172.21.15.90:0/450521486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dea74d0 0x56124e26b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.275+0000 7fb381600640 1 -- 172.21.15.90:0/450521486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124de9f2f0 msgr2=0x56124e26ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.275+0000 7fb381600640 1 --2- 172.21.15.90:0/450521486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124de9f2f0 0x56124e26ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.275+0000 7fb381600640 1 -- 172.21.15.90:0/450521486 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56124e1648b0 con 0x56124dec81b0 2024-05-27T08:01:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.276+0000 7fb381600640 1 --2- 172.21.15.90:0/450521486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e26a860 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb374009f90 tx=0x7fb374010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6420f47d4ea45215 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.276+0000 7fb379600640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb3640b2ae0 0x7fb3640b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.276+0000 7fb379600640 1 -- 172.21.15.90:0/3129588547 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7fb3680e21e0 con 0x7fb37c108cb0 2024-05-27T08:01:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.277+0000 7fb36dc00640 1 -- 172.21.15.90:0/450521486 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374011700 con 0x56124dec81b0 2024-05-27T08:01:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.277+0000 7fb380c00640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb3640b2ae0 0x7fb3640b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.277+0000 7fb36dc00640 1 -- 172.21.15.90:0/450521486 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb374011d30 con 0x56124dec81b0 2024-05-27T08:01:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.277+0000 7fb36dc00640 1 -- 172.21.15.90:0/450521486 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374019d30 con 0x56124dec81b0 2024-05-27T08:01:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.278+0000 7fb38a5f2740 1 -- 172.21.15.90:0/450521486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 msgr2=0x56124e26a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.278+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/450521486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e26a860 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb374009f90 tx=0x7fb374010860 comp rx=0 tx=0).stop 2024-05-27T08:01:35.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.279+0000 7fb38a5f2740 1 -- 172.21.15.90:0/450521486 shutdown_connections 2024-05-27T08:01:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.279+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/450521486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dea74d0 0x56124e26b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.280+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/450521486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124de9f2f0 0x56124e26ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.280+0000 7fb380c00640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb3640b2ae0 0x7fb3640b4ff0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb37002a9f0 tx=0x7fb37002a980 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.280+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/450521486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e26a860 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.280+0000 7fb38a5f2740 1 -- 172.21.15.90:0/450521486 >> 172.21.15.90:0/450521486 conn(0x56124debc300 msgr2=0x56124e267010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.280+0000 7fb38a5f2740 1 -- 172.21.15.90:0/450521486 shutdown_connections 2024-05-27T08:01:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.280+0000 7fb38a5f2740 1 -- 172.21.15.90:0/450521486 wait complete. 2024-05-27T08:01:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.281+0000 7fb38a5f2740 1 -- start start 2024-05-27T08:01:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.282+0000 7fb38a5f2740 1 Processor -- start 2024-05-27T08:01:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.284+0000 7fb38a5f2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124de9f2f0 0x56124e27d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.285+0000 7fb38a5f2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124dea74d0 0x56124e27da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.285+0000 7fb38a5f2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e289520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.285+0000 7fb38a5f2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56124deb1790 con 0x56124dea74d0 2024-05-27T08:01:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.285+0000 7fb38a5f2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56124e173bc0 con 0x56124de9f2f0 2024-05-27T08:01:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.285+0000 7fb380c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124de9f2f0 0x56124e27d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.286+0000 7fb381600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e289520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.286+0000 7fb381600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e289520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46376/0 (socket says 172.21.15.90:46376) 2024-05-27T08:01:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.286+0000 7fb380c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124de9f2f0 0x56124e27d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48640/0 (socket says 172.21.15.90:48640) 2024-05-27T08:01:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.286+0000 7fb381600640 1 -- 172.21.15.90:0/2685933647 learned_addr learned my addr 172.21.15.90:0/2685933647 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.286+0000 7fb38a5f2740 1 -- 172.21.15.90:0/2685933647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56124e2801a0 con 0x56124dec81b0 2024-05-27T08:01:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.286+0000 7fb38a5f2740 10 client.0 _ll_register_callbacks cb 0x56124e151400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.287+0000 7fb380c00640 1 -- 172.21.15.90:0/2685933647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 msgr2=0x56124e289520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.287+0000 7fb380c00640 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e289520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.287+0000 7fb380c00640 1 -- 172.21.15.90:0/2685933647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124dea74d0 msgr2=0x56124e27da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.287+0000 7fb380c00640 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124dea74d0 0x56124e27da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.287+0000 7fb380c00640 1 -- 172.21.15.90:0/2685933647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56124dee0210 con 0x56124de9f2f0 2024-05-27T08:01:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.287+0000 7fb381600640 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e289520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:01:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.288+0000 7fb380c00640 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124de9f2f0 0x56124e27d4c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb370009850 tx=0x7fb370009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28a61cd1c0c4360d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.288+0000 7fb362000640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:01:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.289+0000 7fb362000640 1 -- 172.21.15.90:0/2685933647 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37004c070 con 0x56124de9f2f0 2024-05-27T08:01:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.289+0000 7fb362000640 1 -- 172.21.15.90:0/2685933647 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7fb370004d90 con 0x56124de9f2f0 2024-05-27T08:01:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.289+0000 7fb38a5f2740 1 -- 172.21.15.90:0/2685933647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb374011d30 con 0x56124de9f2f0 2024-05-27T08:01:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.290+0000 7fb362000640 1 -- 172.21.15.90:0/2685933647 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb370056050 con 0x56124de9f2f0 2024-05-27T08:01:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.290+0000 7fb38a5f2740 10 client.37607 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.290+0000 7fb38a5f2740 1 -- 172.21.15.90:0/2685933647 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56124dfac620 con 0x56124de9f2f0 2024-05-27T08:01:35.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.290+0000 7fb362000640 1 -- 172.21.15.90:0/2685933647 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb374011d30 con 0x56124de9f2f0 2024-05-27T08:01:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.290+0000 7fb362000640 1 -- 172.21.15.90:0/2685933647 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fb3700485f0 con 0x56124de9f2f0 2024-05-27T08:01:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.291+0000 7fb38a5f2740 10 client.37607 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.291+0000 7fb38a5f2740 10 client.37607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.291+0000 7fb38a5f2740 20 client.37607 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.291+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56124de86d80 0x56124e28f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.292+0000 7fb38a5f2740 4 client.37607 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.292+0000 7fb38a5f2740 1 -- 172.21.15.90:0/2685933647 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56124e1ddae0 con 0x56124de86d80 2024-05-27T08:01:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.293+0000 7fb37be00640 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56124de86d80 0x56124e28f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.295+0000 7fb37be00640 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56124de86d80 0x56124e28f6c0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f388e4a7f297dc81 server_cookie=58c0a01c399e3f60 in_seq=0 out_seq=0 2024-05-27T08:01:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.295+0000 7fb362000640 10 client.37607 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:35.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.297+0000 7fb362000640 1 -- 172.21.15.90:0/2685933647 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56124e1ddae0 con 0x56124de86d80 2024-05-27T08:01:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.297+0000 7fb362000640 10 client.37607 handle_command_reply: tid=0 2024-05-27T08:01:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.297+0000 7fb38a5f2740 1 client.37607 shutdown 2024-05-27T08:01:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.300+0000 7fb38a5f2740 1 -- 172.21.15.90:0/2685933647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124de9f2f0 msgr2=0x56124e27d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.300+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124de9f2f0 0x56124e27d4c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb370009850 tx=0x7fb370009260 comp rx=0 tx=0).stop 2024-05-27T08:01:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.300+0000 7fb38a5f2740 1 -- 172.21.15.90:0/2685933647 shutdown_connections 2024-05-27T08:01:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.300+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56124de86d80 0x56124e28f6c0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.300+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dec81b0 0x56124e289520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.300+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124dea74d0 0x56124e27da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.300+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/2685933647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124de9f2f0 0x56124e27d4c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.300+0000 7fb38a5f2740 1 -- 172.21.15.90:0/2685933647 >> 172.21.15.90:0/2685933647 conn(0x56124debc300 msgr2=0x56124e26c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.300+0000 7fb362000640 0 client.37607 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.301+0000 7fb38a5f2740 1 -- 172.21.15.90:0/2685933647 shutdown_connections 2024-05-27T08:01:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.301+0000 7fb38a5f2740 1 -- 172.21.15.90:0/2685933647 wait complete. 2024-05-27T08:01:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.301+0000 7fb38a5f2740 20 client.37607 trim_cache size 0 max 16384 2024-05-27T08:01:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.332+0000 7fb38a5f2740 1 Processor -- start 2024-05-27T08:01:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.333+0000 7fb38a5f2740 1 -- start start 2024-05-27T08:01:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.333+0000 7fb38a5f2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124de9f2f0 0x56124e289290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.333+0000 7fb38a5f2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dea74d0 0x56124e2897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.333+0000 7fb38a5f2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dec81b0 0x56124e276ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.333+0000 7fb38a5f2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1b92e0 con 0x56124de9f2f0 2024-05-27T08:01:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.333+0000 7fb38a5f2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1b6330 con 0x56124dec81b0 2024-05-27T08:01:35.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.333+0000 7fb38a5f2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1b8540 con 0x56124dea74d0 2024-05-27T08:01:35.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.335+0000 7fb381600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dec81b0 0x56124e276ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.335+0000 7fb37be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dea74d0 0x56124e2897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.335+0000 7fb37be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dea74d0 0x56124e2897d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46386/0 (socket says 172.21.15.90:46386) 2024-05-27T08:01:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.335+0000 7fb37be00640 1 -- 172.21.15.90:0/3679701107 learned_addr learned my addr 172.21.15.90:0/3679701107 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:35.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.335+0000 7fb381600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dec81b0 0x56124e276ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48642/0 (socket says 172.21.15.90:48642) 2024-05-27T08:01:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.336+0000 7fb37be00640 1 -- 172.21.15.90:0/3679701107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dec81b0 msgr2=0x56124e276ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.336+0000 7fb37be00640 1 --2- 172.21.15.90:0/3679701107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dec81b0 0x56124e276ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.336+0000 7fb37be00640 1 -- 172.21.15.90:0/3679701107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124de9f2f0 msgr2=0x56124e289290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:01:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.336+0000 7fb37be00640 1 --2- 172.21.15.90:0/3679701107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124de9f2f0 0x56124e289290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.336+0000 7fb37be00640 1 -- 172.21.15.90:0/3679701107 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56124e27b800 con 0x56124dea74d0 2024-05-27T08:01:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.337+0000 7fb37be00640 1 --2- 172.21.15.90:0/3679701107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dea74d0 0x56124e2897d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb3680a51f0 tx=0x7fb368005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1126339f819cd87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.337+0000 7fb36d200640 1 -- 172.21.15.90:0/3679701107 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3680c2070 con 0x56124dea74d0 2024-05-27T08:01:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.337+0000 7fb36d200640 1 -- 172.21.15.90:0/3679701107 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3680a9170 con 0x56124dea74d0 2024-05-27T08:01:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.337+0000 7fb36d200640 1 -- 172.21.15.90:0/3679701107 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3680ad170 con 0x56124dea74d0 2024-05-27T08:01:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.339+0000 7fb38a5f2740 1 -- 172.21.15.90:0/3679701107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dea74d0 msgr2=0x56124e2897d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.339+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/3679701107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dea74d0 0x56124e2897d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb3680a51f0 tx=0x7fb368005d30 comp rx=0 tx=0).stop 2024-05-27T08:01:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.339+0000 7fb38a5f2740 1 -- 172.21.15.90:0/3679701107 shutdown_connections 2024-05-27T08:01:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.339+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/3679701107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dec81b0 0x56124e276ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.339+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/3679701107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124dea74d0 0x56124e2897d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.339+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/3679701107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124de9f2f0 0x56124e289290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.339+0000 7fb38a5f2740 1 -- 172.21.15.90:0/3679701107 >> 172.21.15.90:0/3679701107 conn(0x56124debc300 msgr2=0x56124e26b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.340+0000 7fb38a5f2740 1 -- 172.21.15.90:0/3679701107 shutdown_connections 2024-05-27T08:01:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.340+0000 7fb38a5f2740 1 -- 172.21.15.90:0/3679701107 wait complete. 2024-05-27T08:01:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.341+0000 7fb38a5f2740 1 -- start start 2024-05-27T08:01:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.342+0000 7fb38a5f2740 1 Processor -- start 2024-05-27T08:01:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.344+0000 7fb38a5f2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124de9f2f0 0x56124e285490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.344+0000 7fb38a5f2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dea74d0 0x56124e2859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.344+0000 7fb38a5f2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124dec81b0 0x56124e280440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.344+0000 7fb38a5f2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1b7e70 con 0x56124dec81b0 2024-05-27T08:01:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.344+0000 7fb38a5f2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1b6a00 con 0x56124dea74d0 2024-05-27T08:01:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.344+0000 7fb38a5f2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1b70d0 con 0x56124de9f2f0 2024-05-27T08:01:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.344+0000 7fb38a5f2740 10 client.0 _ll_register_callbacks cb 0x7fb3700041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.344+0000 7fb380c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124de9f2f0 0x56124e285490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.345+0000 7fb380c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124de9f2f0 0x56124e285490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46388/0 (socket says 172.21.15.90:46388) 2024-05-27T08:01:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.345+0000 7fb380c00640 1 -- 172.21.15.90:0/5419186 learned_addr learned my addr 172.21.15.90:0/5419186 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.345+0000 7fb37be00640 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dea74d0 0x56124e2859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.345+0000 7fb381600640 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124dec81b0 0x56124e280440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.345+0000 7fb380c00640 1 -- 172.21.15.90:0/5419186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dea74d0 msgr2=0x56124e2859d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.345+0000 7fb380c00640 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dea74d0 0x56124e2859d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.345+0000 7fb380c00640 1 -- 172.21.15.90:0/5419186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124dec81b0 msgr2=0x56124e280440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.345+0000 7fb380c00640 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124dec81b0 0x56124e280440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.345+0000 7fb380c00640 1 -- 172.21.15.90:0/5419186 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56124e286010 con 0x56124de9f2f0 2024-05-27T08:01:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.346+0000 7fb380c00640 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124de9f2f0 0x56124e285490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb3700057c0 tx=0x7fb37005af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f22748fe99b91f97 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.346+0000 7fb362000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:01:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.346+0000 7fb362000640 1 -- 172.21.15.90:0/5419186 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37005f070 con 0x56124de9f2f0 2024-05-27T08:01:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.347+0000 7fb362000640 1 -- 172.21.15.90:0/5419186 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7fb370003a00 con 0x56124de9f2f0 2024-05-27T08:01:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.347+0000 7fb38a5f2740 1 -- 172.21.15.90:0/5419186 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3680a9170 con 0x56124de9f2f0 2024-05-27T08:01:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.348+0000 7fb362000640 1 -- 172.21.15.90:0/5419186 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb37006b050 con 0x56124de9f2f0 2024-05-27T08:01:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.348+0000 7fb38a5f2740 10 client.27718 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.348+0000 7fb38a5f2740 1 -- 172.21.15.90:0/5419186 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56124e273c50 con 0x56124de9f2f0 2024-05-27T08:01:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.349+0000 7fb362000640 1 -- 172.21.15.90:0/5419186 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3680a9170 con 0x56124de9f2f0 2024-05-27T08:01:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.349+0000 7fb362000640 1 -- 172.21.15.90:0/5419186 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fb370067a80 con 0x56124de9f2f0 2024-05-27T08:01:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.349+0000 7fb38a5f2740 10 client.27718 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.349+0000 7fb38a5f2740 10 client.27718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.349+0000 7fb38a5f2740 20 client.27718 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.349+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56124de5c1c0 0x56124e2763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.349+0000 7fb38a5f2740 4 client.27718 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:01:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.349+0000 7fb38a5f2740 1 -- 172.21.15.90:0/5419186 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56124e1d72c0 con 0x56124de5c1c0 2024-05-27T08:01:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.350+0000 7fb37be00640 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56124de5c1c0 0x56124e2763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.351+0000 7fb37be00640 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56124de5c1c0 0x56124e2763c0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddeedd5562522f5a server_cookie=b3666b02c4603d14 in_seq=0 out_seq=0 2024-05-27T08:01:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.351+0000 7fb362000640 10 client.27718 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.399+0000 7fb362000640 1 -- 172.21.15.90:0/5419186 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56124e1d72c0 con 0x56124de5c1c0 2024-05-27T08:01:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.399+0000 7fb362000640 10 client.27718 handle_command_reply: tid=0 2024-05-27T08:01:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.399+0000 7fb38a5f2740 1 client.27718 shutdown 2024-05-27T08:01:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.399+0000 7fb38a5f2740 1 -- 172.21.15.90:0/5419186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124de9f2f0 msgr2=0x56124e285490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.399+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124de9f2f0 0x56124e285490 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb3700057c0 tx=0x7fb37005af90 comp rx=0 tx=0).stop 2024-05-27T08:01:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.400+0000 7fb38a5f2740 1 -- 172.21.15.90:0/5419186 shutdown_connections 2024-05-27T08:01:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.400+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56124de5c1c0 0x56124e2763c0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.400+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56124dea74d0 0x56124e2859d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.400+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56124dec81b0 0x56124e280440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.400+0000 7fb38a5f2740 1 --2- 172.21.15.90:0/5419186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56124de9f2f0 0x56124e285490 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.400+0000 7fb38a5f2740 1 -- 172.21.15.90:0/5419186 >> 172.21.15.90:0/5419186 conn(0x56124debc300 msgr2=0x56124e26b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.400+0000 7fb362000640 0 client.27718 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.402+0000 7fb38a5f2740 1 -- 172.21.15.90:0/5419186 shutdown_connections 2024-05-27T08:01:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.402+0000 7fb38a5f2740 1 -- 172.21.15.90:0/5419186 wait complete. 2024-05-27T08:01:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.402+0000 7fb38a5f2740 20 client.27718 trim_cache size 0 max 16384 2024-05-27T08:01:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:01:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:01:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:01:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.407+0000 7fb383600640 1 -- 172.21.15.90:0/3129588547 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb3640b2ae0 msgr2=0x7fb3640b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.407+0000 7fb383600640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb3640b2ae0 0x7fb3640b4ff0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb37002a9f0 tx=0x7fb37002a980 comp rx=0 tx=0).stop 2024-05-27T08:01:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.407+0000 7fb383600640 1 -- 172.21.15.90:0/3129588547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c108cb0 msgr2=0x7fb37c0744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.407+0000 7fb383600640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c108cb0 0x7fb37c0744e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb36800d950 tx=0x7fb36800de10 comp rx=0 tx=0).stop 2024-05-27T08:01:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.407+0000 7fb383600640 1 -- 172.21.15.90:0/3129588547 shutdown_connections 2024-05-27T08:01:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.407+0000 7fb383600640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb3640b2ae0 0x7fb3640b4ff0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.407+0000 7fb383600640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb37c10e550 0x7fb37c1032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.407+0000 7fb383600640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb37c108cb0 0x7fb37c0744e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.407+0000 7fb383600640 1 --2- 172.21.15.90:0/3129588547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb37c101e00 0x7fb37c073fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.407+0000 7fb383600640 1 -- 172.21.15.90:0/3129588547 >> 172.21.15.90:0/3129588547 conn(0x7fb37c0fd6a0 msgr2=0x7fb37c106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.408+0000 7fb383600640 1 -- 172.21.15.90:0/3129588547 shutdown_connections 2024-05-27T08:01:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:35.408+0000 7fb383600640 1 -- 172.21.15.90:0/3129588547 wait complete. 2024-05-27T08:01:35.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:35 smithi136 ceph-mon[35324]: pgmap v487: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 MiB/s wr, 725 op/s 2024-05-27T08:01:35.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:35 smithi129 ceph-mon[35254]: pgmap v487: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 MiB/s wr, 725 op/s 2024-05-27T08:01:35.916 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:35.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:35 smithi090 ceph-mon[31429]: pgmap v487: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.2 MiB/s wr, 725 op/s 2024-05-27T08:01:37.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:37 smithi136 ceph-mon[35324]: pgmap v488: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 6.6 MiB/s wr, 580 op/s 2024-05-27T08:01:37.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:37 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:01:37.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:37 smithi129 ceph-mon[35254]: pgmap v488: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 6.6 MiB/s wr, 580 op/s 2024-05-27T08:01:37.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:37 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:01:37.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:37 smithi090 ceph-mon[31429]: pgmap v488: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 6.6 MiB/s wr, 580 op/s 2024-05-27T08:01:37.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:37 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:01:38.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 680.0 s, 481.0 tps, lat 215.614 ms stddev 212.254 2024-05-27T08:01:39.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:39 smithi136 ceph-mon[35324]: pgmap v489: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6.2 MiB/s wr, 535 op/s 2024-05-27T08:01:39.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:39 smithi129 ceph-mon[35254]: pgmap v489: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6.2 MiB/s wr, 535 op/s 2024-05-27T08:01:39.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:39 smithi090 ceph-mon[31429]: pgmap v489: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6.2 MiB/s wr, 535 op/s 2024-05-27T08:01:40.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:40 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:01:40.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:40 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:01:40.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:40 smithi136 ceph-mon[35324]: pgmap v490: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 8.9 MiB/s wr, 760 op/s 2024-05-27T08:01:40.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:40 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:01:40.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:40 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:01:40.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:40 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:01:40.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:40 smithi129 ceph-mon[35254]: pgmap v490: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 8.9 MiB/s wr, 760 op/s 2024-05-27T08:01:40.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:40 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:01:40.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:40 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:01:40.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:40 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:01:40.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:40 smithi090 ceph-mon[31429]: pgmap v490: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 8.9 MiB/s wr, 760 op/s 2024-05-27T08:01:40.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:40 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:01:42.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.669+0000 7f2c67600700 1 -- 172.21.15.90:0/3947894875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c60107de0 msgr2=0x7f2c601081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:42.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.669+0000 7f2c67600700 1 --2- 172.21.15.90:0/3947894875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c60107de0 0x7f2c601081f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00d990 tx=0x7f2c4c00dca0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.670+0000 7f2c67600700 1 -- 172.21.15.90:0/3947894875 shutdown_connections 2024-05-27T08:01:42.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.670+0000 7f2c67600700 1 --2- 172.21.15.90:0/3947894875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c60108730 0x7f2c60110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.670+0000 7f2c67600700 1 --2- 172.21.15.90:0/3947894875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c601034b0 0x7f2c601038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.670+0000 7f2c67600700 1 --2- 172.21.15.90:0/3947894875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c60107de0 0x7f2c601081f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.670+0000 7f2c67600700 1 -- 172.21.15.90:0/3947894875 >> 172.21.15.90:0/3947894875 conn(0x7f2c600fedb0 msgr2=0x7f2c601011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:42.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.670+0000 7f2c67600700 1 -- 172.21.15.90:0/3947894875 shutdown_connections 2024-05-27T08:01:42.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.670+0000 7f2c67600700 1 -- 172.21.15.90:0/3947894875 wait complete. 2024-05-27T08:01:42.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.672+0000 7f2c67600700 1 Processor -- start 2024-05-27T08:01:42.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.675+0000 7f2c67600700 1 -- start start 2024-05-27T08:01:42.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.675+0000 7f2c67600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c601034b0 0x7f2c6019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.675+0000 7f2c67600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c60107de0 0x7f2c6019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.675+0000 7f2c67600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c60108730 0x7f2c601a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.675+0000 7f2c67600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60112180 con 0x7f2c60108730 2024-05-27T08:01:42.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.675+0000 7f2c67600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60112000 con 0x7f2c60107de0 2024-05-27T08:01:42.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.675+0000 7f2c67600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60112300 con 0x7f2c601034b0 2024-05-27T08:01:42.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.676+0000 7f2c5fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c60107de0 0x7f2c6019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.676+0000 7f2c5fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c60107de0 0x7f2c6019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56334/0 (socket says 172.21.15.90:56334) 2024-05-27T08:01:42.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.676+0000 7f2c5fe00700 1 -- 172.21.15.90:0/3612959990 learned_addr learned my addr 172.21.15.90:0/3612959990 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:42.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.677+0000 7f2c5fe00700 1 -- 172.21.15.90:0/3612959990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c601034b0 msgr2=0x7f2c6019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:42.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.677+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c601034b0 0x7f2c6019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.677+0000 7f2c5fe00700 1 -- 172.21.15.90:0/3612959990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c60108730 msgr2=0x7f2c601a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:42.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.677+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c60108730 0x7f2c601a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.677+0000 7f2c5fe00700 1 -- 172.21.15.90:0/3612959990 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c601a1ed0 con 0x7f2c60107de0 2024-05-27T08:01:42.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.678+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c60107de0 0x7f2c6019d5a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c000c00 tx=0x7f2c4c005390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:42.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.678+0000 7f2c5d600700 1 -- 172.21.15.90:0/3612959990 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c01a070 con 0x7f2c60107de0 2024-05-27T08:01:42.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.678+0000 7f2c5d600700 1 -- 172.21.15.90:0/3612959990 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c4c009c00 con 0x7f2c60107de0 2024-05-27T08:01:42.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.678+0000 7f2c5d600700 1 -- 172.21.15.90:0/3612959990 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c003cf0 con 0x7f2c60107de0 2024-05-27T08:01:42.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.680+0000 7f2c67600700 1 -- 172.21.15.90:0/3612959990 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c601a2160 con 0x7f2c60107de0 2024-05-27T08:01:42.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.680+0000 7f2c67600700 1 -- 172.21.15.90:0/3612959990 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c601a25c0 con 0x7f2c60107de0 2024-05-27T08:01:42.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.691+0000 7f2c5d600700 1 -- 172.21.15.90:0/3612959990 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2c4c00c040 con 0x7f2c60107de0 2024-05-27T08:01:42.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.693+0000 7f2c5d600700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2c480b2a70 0x7f2c480b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.694+0000 7f2c5d600700 1 -- 172.21.15.90:0/3612959990 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f2c4c0e2ed0 con 0x7f2c60107de0 2024-05-27T08:01:42.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.694+0000 7f2c64c00700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2c480b2a70 0x7f2c480b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.698+0000 7f2c64c00700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2c480b2a70 0x7f2c480b4f80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c54005950 tx=0x7f2c540058e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:42.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.698+0000 7f2c761e8b80 1 Processor -- start 2024-05-27T08:01:42.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.698+0000 7f2c761e8b80 1 -- start start 2024-05-27T08:01:42.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.700+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 0x558af024ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.700+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af024b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.700+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af0257920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.701+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558af0170e40 con 0x558af01af9f0 2024-05-27T08:01:42.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.701+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558af01609e0 con 0x558af00c3e40 2024-05-27T08:01:42.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.701+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558af0164a20 con 0x558af024b690 2024-05-27T08:01:42.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.701+0000 7f2c5fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af024b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.701+0000 7f2c65600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af0257920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.701+0000 7f2c65600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af0257920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:32880/0 (socket says 172.21.15.90:32880) 2024-05-27T08:01:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.701+0000 7f2c65600700 1 -- 172.21.15.90:0/1697429536 learned_addr learned my addr 172.21.15.90:0/1697429536 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.701+0000 7f2c64c00700 1 --2- 172.21.15.90:0/1697429536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 0x558af024ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.702+0000 7f2c65600700 1 -- 172.21.15.90:0/1697429536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 msgr2=0x558af024b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.702+0000 7f2c65600700 1 --2- 172.21.15.90:0/1697429536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af024b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.702+0000 7f2c65600700 1 -- 172.21.15.90:0/1697429536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 msgr2=0x558af024ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.702+0000 7f2c65600700 1 --2- 172.21.15.90:0/1697429536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 0x558af024ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.702+0000 7f2c65600700 1 -- 172.21.15.90:0/1697429536 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af0129420 con 0x558af024b690 2024-05-27T08:01:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.702+0000 7f2c64c00700 1 --2- 172.21.15.90:0/1697429536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 0x558af024ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.703+0000 7f2c65600700 1 --2- 172.21.15.90:0/1697429536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af0257920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800e380 tx=0x7f2c5800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9477cba337d7eb99 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:42.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.704+0000 7f2c52000700 1 -- 172.21.15.90:0/1697429536 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800ee30 con 0x558af024b690 2024-05-27T08:01:42.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.704+0000 7f2c52000700 1 -- 172.21.15.90:0/1697429536 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c58004bb0 con 0x558af024b690 2024-05-27T08:01:42.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.704+0000 7f2c52000700 1 -- 172.21.15.90:0/1697429536 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58004280 con 0x558af024b690 2024-05-27T08:01:42.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.705+0000 7f2c761e8b80 1 -- 172.21.15.90:0/1697429536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 msgr2=0x558af0257920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:42.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.705+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/1697429536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af0257920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800e380 tx=0x7f2c5800b040 comp rx=0 tx=0).stop 2024-05-27T08:01:42.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.706+0000 7f2c761e8b80 1 -- 172.21.15.90:0/1697429536 shutdown_connections 2024-05-27T08:01:42.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.706+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/1697429536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af0257920 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.706+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/1697429536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 0x558af024ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.706+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/1697429536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af024b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.706+0000 7f2c761e8b80 1 -- 172.21.15.90:0/1697429536 >> 172.21.15.90:0/1697429536 conn(0x558af015d390 msgr2=0x558af023e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:42.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.706+0000 7f2c761e8b80 1 -- 172.21.15.90:0/1697429536 shutdown_connections 2024-05-27T08:01:42.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.707+0000 7f2c761e8b80 1 -- 172.21.15.90:0/1697429536 wait complete. 2024-05-27T08:01:42.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.708+0000 7f2c761e8b80 1 -- start start 2024-05-27T08:01:42.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.708+0000 7f2c761e8b80 1 Processor -- start 2024-05-27T08:01:42.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.711+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af0254f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.711+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0255470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.711+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af024b690 0x558af02559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.711+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558af0066fa0 con 0x558af024b690 2024-05-27T08:01:42.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.711+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558af0101760 con 0x558af00c3e40 2024-05-27T08:01:42.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.711+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558af0258390 con 0x558af01af9f0 2024-05-27T08:01:42.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.711+0000 7f2c761e8b80 10 client.0 _ll_register_callbacks cb 0x558af010ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.712+0000 7f2c5fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0255470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.712+0000 7f2c5fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0255470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:32882/0 (socket says 172.21.15.90:32882) 2024-05-27T08:01:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.712+0000 7f2c5fe00700 1 -- 172.21.15.90:0/4072901460 learned_addr learned my addr 172.21.15.90:0/4072901460 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:42.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.712+0000 7f2c64c00700 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af0254f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.715+0000 7f2c65600700 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af024b690 0x558af02559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.716+0000 7f2c5fe00700 1 -- 172.21.15.90:0/4072901460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 msgr2=0x558af0254f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.716+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af0254f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.716+0000 7f2c5fe00700 1 -- 172.21.15.90:0/4072901460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af024b690 msgr2=0x558af02559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.716+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af024b690 0x558af02559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.716+0000 7f2c5fe00700 1 -- 172.21.15.90:0/4072901460 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af00b08d0 con 0x558af01af9f0 2024-05-27T08:01:42.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.719+0000 7f2c46a00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:01:42.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.719+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0255470 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0e0850 tx=0x7f2c4c026f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c84eaf042a66085 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:42.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.720+0000 7f2c46a00700 1 -- 172.21.15.90:0/4072901460 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c0a95e0 con 0x558af01af9f0 2024-05-27T08:01:42.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.720+0000 7f2c46a00700 1 -- 172.21.15.90:0/4072901460 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f2c4c0a6460 con 0x558af01af9f0 2024-05-27T08:01:42.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.720+0000 7f2c761e8b80 1 -- 172.21.15.90:0/4072901460 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c58004bb0 con 0x558af01af9f0 2024-05-27T08:01:42.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.720+0000 7f2c46a00700 1 -- 172.21.15.90:0/4072901460 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f2c4c0a95e0 con 0x558af01af9f0 2024-05-27T08:01:42.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.720+0000 7f2c761e8b80 10 client.27727 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:42.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.721+0000 7f2c761e8b80 1 -- 172.21.15.90:0/4072901460 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558af0131060 con 0x558af01af9f0 2024-05-27T08:01:42.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.721+0000 7f2c46a00700 1 -- 172.21.15.90:0/4072901460 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c4c003680 con 0x558af01af9f0 2024-05-27T08:01:42.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.721+0000 7f2c46a00700 1 -- 172.21.15.90:0/4072901460 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f2c4c0a90d0 con 0x558af01af9f0 2024-05-27T08:01:42.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.721+0000 7f2c761e8b80 10 client.27727 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:42.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.721+0000 7f2c761e8b80 10 client.27727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:01:42.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.721+0000 7f2c761e8b80 20 client.27727 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:42.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.721+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558af02652f0 0x558af0267740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.721+0000 7f2c761e8b80 4 client.27727 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:42.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.721+0000 7f2c761e8b80 1 -- 172.21.15.90:0/4072901460 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2c58004bb0 con 0x558af02652f0 2024-05-27T08:01:42.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.722+0000 7f2c64c00700 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558af02652f0 0x558af0267740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.727+0000 7f2c64c00700 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558af02652f0 0x558af0267740 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f355e4a34ad78d80 server_cookie=b9ab27f540eef91b in_seq=0 out_seq=0 2024-05-27T08:01:42.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.727+0000 7f2c46a00700 10 client.27727 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T08:01:42.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.728+0000 7f2c46a00700 1 -- 172.21.15.90:0/4072901460 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f2c58004bb0 con 0x558af02652f0 2024-05-27T08:01:42.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.728+0000 7f2c46a00700 10 client.27727 handle_command_reply: tid=0 2024-05-27T08:01:42.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.728+0000 7f2c761e8b80 1 client.27727 shutdown 2024-05-27T08:01:42.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.728+0000 7f2c761e8b80 1 -- 172.21.15.90:0/4072901460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 msgr2=0x558af0255470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:42.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.728+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0255470 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0e0850 tx=0x7f2c4c026f50 comp rx=0 tx=0).stop 2024-05-27T08:01:42.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.729+0000 7f2c761e8b80 1 -- 172.21.15.90:0/4072901460 shutdown_connections 2024-05-27T08:01:42.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.729+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af024b690 0x558af02559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.729+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558af02652f0 0x558af0267740 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.729+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af0254f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.729+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/4072901460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0255470 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.729+0000 7f2c46a00700 0 client.27727 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T08:01:42.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.729+0000 7f2c761e8b80 1 -- 172.21.15.90:0/4072901460 >> 172.21.15.90:0/4072901460 conn(0x558af015d390 msgr2=0x558af024abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:42.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.729+0000 7f2c761e8b80 1 -- 172.21.15.90:0/4072901460 shutdown_connections 2024-05-27T08:01:42.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.729+0000 7f2c761e8b80 1 -- 172.21.15.90:0/4072901460 wait complete. 2024-05-27T08:01:42.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.729+0000 7f2c761e8b80 20 client.27727 trim_cache size 0 max 16384 2024-05-27T08:01:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.768+0000 7f2c761e8b80 1 Processor -- start 2024-05-27T08:01:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.768+0000 7f2c761e8b80 1 -- start start 2024-05-27T08:01:42.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.769+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af00c3e40 0x558af0250700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.770+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0261480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.770+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af024b690 0x558af024ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.770+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601afee0 con 0x558af00c3e40 2024-05-27T08:01:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.770+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601acf30 con 0x558af024b690 2024-05-27T08:01:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.770+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601af140 con 0x558af01af9f0 2024-05-27T08:01:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.770+0000 7f2c5fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0261480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.770+0000 7f2c5fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0261480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:32896/0 (socket says 172.21.15.90:32896) 2024-05-27T08:01:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.770+0000 7f2c5fe00700 1 -- 172.21.15.90:0/1228177775 learned_addr learned my addr 172.21.15.90:0/1228177775 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:42.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.770+0000 7f2c65600700 1 --2- 172.21.15.90:0/1228177775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af024b690 0x558af024ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.771+0000 7f2c65600700 1 -- 172.21.15.90:0/1228177775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 msgr2=0x558af0261480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.771+0000 7f2c65600700 1 --2- 172.21.15.90:0/1228177775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0261480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.771+0000 7f2c65600700 1 -- 172.21.15.90:0/1228177775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af00c3e40 msgr2=0x558af0250700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.771+0000 7f2c65600700 1 --2- 172.21.15.90:0/1228177775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af00c3e40 0x558af0250700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.771+0000 7f2c65600700 1 -- 172.21.15.90:0/1228177775 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af023a240 con 0x558af024b690 2024-05-27T08:01:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.771+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/1228177775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0261480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:01:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.772+0000 7f2c65600700 1 --2- 172.21.15.90:0/1228177775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af024b690 0x558af024ce20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800b010 tx=0x7f2c5800efc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e81d84c1337ae304 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.773+0000 7f2c51600700 1 -- 172.21.15.90:0/1228177775 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58004c70 con 0x558af024b690 2024-05-27T08:01:42.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.773+0000 7f2c51600700 1 -- 172.21.15.90:0/1228177775 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c58004e10 con 0x558af024b690 2024-05-27T08:01:42.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.773+0000 7f2c51600700 1 -- 172.21.15.90:0/1228177775 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58007d90 con 0x558af024b690 2024-05-27T08:01:42.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.775+0000 7f2c761e8b80 1 -- 172.21.15.90:0/1228177775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af024b690 msgr2=0x558af024ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:42.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.775+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/1228177775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af024b690 0x558af024ce20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800b010 tx=0x7f2c5800efc0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.775+0000 7f2c761e8b80 1 -- 172.21.15.90:0/1228177775 shutdown_connections 2024-05-27T08:01:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.775+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/1228177775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af024b690 0x558af024ce20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.775+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/1228177775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af00c3e40 0x558af0250700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.775+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/1228177775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af01af9f0 0x558af0261480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.775+0000 7f2c761e8b80 1 -- 172.21.15.90:0/1228177775 >> 172.21.15.90:0/1228177775 conn(0x558af015d390 msgr2=0x558af024c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.776+0000 7f2c761e8b80 1 -- 172.21.15.90:0/1228177775 shutdown_connections 2024-05-27T08:01:42.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.777+0000 7f2c761e8b80 1 -- 172.21.15.90:0/1228177775 wait complete. 2024-05-27T08:01:42.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.777+0000 7f2c761e8b80 1 -- start start 2024-05-27T08:01:42.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.777+0000 7f2c761e8b80 1 Processor -- start 2024-05-27T08:01:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.780+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af024f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.781+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 0x558af024f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.781+0000 7f2c761e8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af025c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.781+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601aea70 con 0x558af01af9f0 2024-05-27T08:01:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.781+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601ad600 con 0x558af00c3e40 2024-05-27T08:01:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.781+0000 7f2c761e8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c601adcd0 con 0x558af024b690 2024-05-27T08:01:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.781+0000 7f2c761e8b80 10 client.0 _ll_register_callbacks cb 0x558af010ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.782+0000 7f2c65600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af025c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.782+0000 7f2c65600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af025c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:32900/0 (socket says 172.21.15.90:32900) 2024-05-27T08:01:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.782+0000 7f2c65600700 1 -- 172.21.15.90:0/3465238500 learned_addr learned my addr 172.21.15.90:0/3465238500 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.782+0000 7f2c64c00700 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af024f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.782+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 0x558af024f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.782+0000 7f2c64c00700 1 -- 172.21.15.90:0/3465238500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 msgr2=0x558af025c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.782+0000 7f2c64c00700 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af025c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.783+0000 7f2c64c00700 1 -- 172.21.15.90:0/3465238500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 msgr2=0x558af024f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.783+0000 7f2c64c00700 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 0x558af024f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.783+0000 7f2c64c00700 1 -- 172.21.15.90:0/3465238500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af025cbb0 con 0x558af00c3e40 2024-05-27T08:01:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.783+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 0x558af024f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:01:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.784+0000 7f2c64c00700 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af024f2b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x558af0267e70 tx=0x7f2c54004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d105ffa27db0881 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:42.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.784+0000 7f2c46a00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:01:42.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.785+0000 7f2c46a00700 1 -- 172.21.15.90:0/3465238500 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5401dba0 con 0x558af00c3e40 2024-05-27T08:01:42.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.785+0000 7f2c46a00700 1 -- 172.21.15.90:0/3465238500 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f2c5401dd40 con 0x558af00c3e40 2024-05-27T08:01:42.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.785+0000 7f2c761e8b80 1 -- 172.21.15.90:0/3465238500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c58004e10 con 0x558af00c3e40 2024-05-27T08:01:42.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.786+0000 7f2c46a00700 1 -- 172.21.15.90:0/3465238500 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f2c54004830 con 0x558af00c3e40 2024-05-27T08:01:42.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.786+0000 7f2c761e8b80 10 client.37631 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:42.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.786+0000 7f2c761e8b80 1 -- 172.21.15.90:0/3465238500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558af025de70 con 0x558af00c3e40 2024-05-27T08:01:42.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:42 smithi090 ceph-mon[31429]: pgmap v491: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6.7 MiB/s wr, 563 op/s 2024-05-27T08:01:42.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.787+0000 7f2c46a00700 1 -- 172.21.15.90:0/3465238500 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c58004e10 con 0x558af00c3e40 2024-05-27T08:01:42.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.787+0000 7f2c46a00700 1 -- 172.21.15.90:0/3465238500 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f2c54030030 con 0x558af00c3e40 2024-05-27T08:01:42.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.787+0000 7f2c761e8b80 10 client.37631 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:42.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.787+0000 7f2c761e8b80 10 client.37631 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:01:42.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.787+0000 7f2c761e8b80 20 client.37631 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:42.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.787+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558af02cfdd0 0x558af02d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:42.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.787+0000 7f2c761e8b80 4 client.37631 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:01:42.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.787+0000 7f2c761e8b80 1 -- 172.21.15.90:0/3465238500 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558af0172d90 con 0x558af02cfdd0 2024-05-27T08:01:42.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.789+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558af02cfdd0 0x558af02d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:42.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.792+0000 7f2c46a00700 10 client.37631 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T08:01:42.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.792+0000 7f2c5fe00700 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558af02cfdd0 0x558af02d21f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53db163459b2a54a server_cookie=d095cfdef4b099e8 in_seq=0 out_seq=0 2024-05-27T08:01:42.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.793+0000 7f2c46a00700 1 -- 172.21.15.90:0/3465238500 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558af0172d90 con 0x558af02cfdd0 2024-05-27T08:01:42.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.793+0000 7f2c46a00700 10 client.37631 handle_command_reply: tid=0 2024-05-27T08:01:42.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.793+0000 7f2c761e8b80 1 client.37631 shutdown 2024-05-27T08:01:42.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.794+0000 7f2c761e8b80 1 -- 172.21.15.90:0/3465238500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 msgr2=0x558af024f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:42.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.794+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af024f2b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x558af0267e70 tx=0x7f2c54004750 comp rx=0 tx=0).stop 2024-05-27T08:01:42.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.796+0000 7f2c761e8b80 1 -- 172.21.15.90:0/3465238500 shutdown_connections 2024-05-27T08:01:42.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.796+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558af024b690 0x558af025c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.796+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558af02cfdd0 0x558af02d21f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.796+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558af00c3e40 0x558af024f2b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.796+0000 7f2c761e8b80 1 --2- 172.21.15.90:0/3465238500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558af01af9f0 0x558af024f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.796+0000 7f2c46a00700 0 client.37631 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T08:01:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.796+0000 7f2c761e8b80 1 -- 172.21.15.90:0/3465238500 >> 172.21.15.90:0/3465238500 conn(0x558af015d390 msgr2=0x558af0254d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.796+0000 7f2c761e8b80 1 -- 172.21.15.90:0/3465238500 shutdown_connections 2024-05-27T08:01:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.796+0000 7f2c761e8b80 1 -- 172.21.15.90:0/3465238500 wait complete. 2024-05-27T08:01:42.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.796+0000 7f2c761e8b80 20 client.37631 trim_cache size 0 max 16384 2024-05-27T08:01:42.799 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:01:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.798+0000 7f2c67600700 1 -- 172.21.15.90:0/3612959990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2c480b2a70 msgr2=0x7f2c480b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.798+0000 7f2c67600700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2c480b2a70 0x7f2c480b4f80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c54005950 tx=0x7f2c540058e0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.798+0000 7f2c67600700 1 -- 172.21.15.90:0/3612959990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c60107de0 msgr2=0x7f2c6019d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.798+0000 7f2c67600700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c60107de0 0x7f2c6019d5a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c000c00 tx=0x7f2c4c005390 comp rx=0 tx=0).stop 2024-05-27T08:01:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.799+0000 7f2c67600700 1 -- 172.21.15.90:0/3612959990 shutdown_connections 2024-05-27T08:01:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.799+0000 7f2c67600700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c60108730 0x7f2c601a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.799+0000 7f2c67600700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c601034b0 0x7f2c6019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.799+0000 7f2c67600700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2c480b2a70 0x7f2c480b4f80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.799+0000 7f2c67600700 1 --2- 172.21.15.90:0/3612959990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c60107de0 0x7f2c6019d5a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:42.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.799+0000 7f2c67600700 1 -- 172.21.15.90:0/3612959990 >> 172.21.15.90:0/3612959990 conn(0x7f2c600fedb0 msgr2=0x7f2c600ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:42.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.799+0000 7f2c67600700 1 -- 172.21.15.90:0/3612959990 shutdown_connections 2024-05-27T08:01:42.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:42.799+0000 7f2c67600700 1 -- 172.21.15.90:0/3612959990 wait complete. 2024-05-27T08:01:42.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:42 smithi129 ceph-mon[35254]: pgmap v491: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6.7 MiB/s wr, 563 op/s 2024-05-27T08:01:43.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:42 smithi136 ceph-mon[35324]: pgmap v491: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6.7 MiB/s wr, 563 op/s 2024-05-27T08:01:43.719 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:01:43.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 685.0 s, 433.0 tps, lat 232.053 ms stddev 225.294 2024-05-27T08:01:44.464 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:44.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:44 smithi129 ceph-mon[35254]: pgmap v492: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.9 MiB/s wr, 595 op/s 2024-05-27T08:01:44.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:44 smithi090 ceph-mon[31429]: pgmap v492: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.9 MiB/s wr, 595 op/s 2024-05-27T08:01:45.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:44 smithi136 ceph-mon[35324]: pgmap v492: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.9 MiB/s wr, 595 op/s 2024-05-27T08:01:46.663 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:46 smithi090 ceph-mon[31429]: pgmap v493: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.6 MiB/s wr, 546 op/s 2024-05-27T08:01:46.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:46 smithi129 ceph-mon[35254]: pgmap v493: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.6 MiB/s wr, 546 op/s 2024-05-27T08:01:47.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:46 smithi136 ceph-mon[35324]: pgmap v493: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.6 MiB/s wr, 546 op/s 2024-05-27T08:01:47.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.085+0000 7fae29c00700 1 -- 172.21.15.90:0/3205151149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae24110960 msgr2=0x7fae24112dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:47.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.086+0000 7fae29c00700 1 --2- 172.21.15.90:0/3205151149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae24110960 0x7fae24112dc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009990 tx=0x7fae1c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.086+0000 7fae29c00700 1 -- 172.21.15.90:0/3205151149 shutdown_connections 2024-05-27T08:01:47.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.086+0000 7fae29c00700 1 --2- 172.21.15.90:0/3205151149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae24110960 0x7fae24112dc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.086+0000 7fae29c00700 1 --2- 172.21.15.90:0/3205151149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae24075d20 0x7fae24076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.086+0000 7fae29c00700 1 --2- 172.21.15.90:0/3205151149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae24076670 0x7fae24110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.086+0000 7fae29c00700 1 -- 172.21.15.90:0/3205151149 >> 172.21.15.90:0/3205151149 conn(0x7fae240ffda0 msgr2=0x7fae241021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:47.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.088+0000 7fae29c00700 1 -- 172.21.15.90:0/3205151149 shutdown_connections 2024-05-27T08:01:47.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.089+0000 7fae29c00700 1 -- 172.21.15.90:0/3205151149 wait complete. 2024-05-27T08:01:47.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.091+0000 7fae29c00700 1 Processor -- start 2024-05-27T08:01:47.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.091+0000 7fae29c00700 1 -- start start 2024-05-27T08:01:47.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.092+0000 7fae29c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae24075d20 0x7fae2419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.092+0000 7fae29c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae24076670 0x7fae2419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.092+0000 7fae29c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae24110960 0x7fae241a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.093+0000 7fae29c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae24116430 con 0x7fae24076670 2024-05-27T08:01:47.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.093+0000 7fae29c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae241162b0 con 0x7fae24075d20 2024-05-27T08:01:47.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.093+0000 7fae29c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae241165b0 con 0x7fae24110960 2024-05-27T08:01:47.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.093+0000 7fae23e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae24075d20 0x7fae2419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.093+0000 7fae23e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae24075d20 0x7fae2419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56366/0 (socket says 172.21.15.90:56366) 2024-05-27T08:01:47.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.093+0000 7fae29200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae24110960 0x7fae241a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.093+0000 7fae23e00700 1 -- 172.21.15.90:0/718738505 learned_addr learned my addr 172.21.15.90:0/718738505 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:47.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.094+0000 7fae29200700 1 -- 172.21.15.90:0/718738505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae24075d20 msgr2=0x7fae2419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:47.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.094+0000 7fae29200700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae24075d20 0x7fae2419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.094+0000 7fae29200700 1 -- 172.21.15.90:0/718738505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae24076670 msgr2=0x7fae2419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:47.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.094+0000 7fae29200700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae24076670 0x7fae2419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.094+0000 7fae29200700 1 -- 172.21.15.90:0/718738505 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae241a1d30 con 0x7fae24110960 2024-05-27T08:01:47.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.094+0000 7fae29200700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae24110960 0x7fae241a1690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fae1c006010 tx=0x7fae1c00f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:47.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.095+0000 7fae21600700 1 -- 172.21.15.90:0/718738505 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c00f840 con 0x7fae24110960 2024-05-27T08:01:47.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.095+0000 7fae21600700 1 -- 172.21.15.90:0/718738505 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae1c004800 con 0x7fae24110960 2024-05-27T08:01:47.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.096+0000 7fae21600700 1 -- 172.21.15.90:0/718738505 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0052b0 con 0x7fae24110960 2024-05-27T08:01:47.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.096+0000 7fae29c00700 1 -- 172.21.15.90:0/718738505 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae241a1fc0 con 0x7fae24110960 2024-05-27T08:01:47.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.096+0000 7fae29c00700 1 -- 172.21.15.90:0/718738505 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae241a23f0 con 0x7fae24110960 2024-05-27T08:01:47.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.104+0000 7fae21600700 1 -- 172.21.15.90:0/718738505 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fae1c005450 con 0x7fae24110960 2024-05-27T08:01:47.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.108+0000 7fae39cbcb80 1 Processor -- start 2024-05-27T08:01:47.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.108+0000 7fae39cbcb80 1 -- start start 2024-05-27T08:01:47.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.108+0000 7fae21600700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fae080b2c00 0x7fae080b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.109+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cce76e40 0x55b8ccff4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.109+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8ccfffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.109+0000 7fae21600700 1 -- 172.21.15.90:0/718738505 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7fae1c0e3690 con 0x7fae24110960 2024-05-27T08:01:47.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.110+0000 7fae1b400700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fae080b2c00 0x7fae080b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.111+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ccff5820 0x55b8cd002020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.111+0000 7fae23e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cce76e40 0x55b8ccff4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.111+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b8ccf23e40 con 0x55b8cce76e40 2024-05-27T08:01:47.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.111+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b8ccf139e0 con 0x55b8ccff5410 2024-05-27T08:01:47.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.111+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8ccf17a20 con 0x55b8ccff5820 2024-05-27T08:01:47.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.111+0000 7fae1b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8ccfffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.112+0000 7fae1b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8ccfffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56380/0 (socket says 172.21.15.90:56380) 2024-05-27T08:01:47.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.112+0000 7fae1b400700 1 -- 172.21.15.90:0/2429542382 learned_addr learned my addr 172.21.15.90:0/2429542382 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:47.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.112+0000 7fae29200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ccff5820 0x55b8cd002020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.113+0000 7fae29200700 1 -- 172.21.15.90:0/2429542382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 msgr2=0x55b8ccfffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:47.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.113+0000 7fae29200700 1 --2- 172.21.15.90:0/2429542382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8ccfffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.113+0000 7fae29200700 1 -- 172.21.15.90:0/2429542382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cce76e40 msgr2=0x55b8ccff4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:47.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.114+0000 7fae1b400700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fae080b2c00 0x7fae080b5110 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00afd0 tx=0x7fae0c00a4f0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:47.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.115+0000 7fae29200700 1 --2- 172.21.15.90:0/2429542382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cce76e40 0x55b8ccff4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.115+0000 7fae29200700 1 -- 172.21.15.90:0/2429542382 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ccedc420 con 0x55b8ccff5820 2024-05-27T08:01:47.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.116+0000 7fae23e00700 1 --2- 172.21.15.90:0/2429542382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cce76e40 0x55b8ccff4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:01:47.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.117+0000 7fae29200700 1 --2- 172.21.15.90:0/2429542382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ccff5820 0x55b8cd002020 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0e09b0 tx=0x7fae1c005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20ad00caf96a7ed9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:47.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.117+0000 7fae19600700 1 -- 172.21.15.90:0/2429542382 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0c4070 con 0x55b8ccff5820 2024-05-27T08:01:47.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.118+0000 7fae19600700 1 -- 172.21.15.90:0/2429542382 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae1c0aacd0 con 0x55b8ccff5820 2024-05-27T08:01:47.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.118+0000 7fae19600700 1 -- 172.21.15.90:0/2429542382 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0e3070 con 0x55b8ccff5820 2024-05-27T08:01:47.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.120+0000 7fae39cbcb80 1 -- 172.21.15.90:0/2429542382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ccff5820 msgr2=0x55b8cd002020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:47.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.120+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/2429542382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ccff5820 0x55b8cd002020 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0e09b0 tx=0x7fae1c005760 comp rx=0 tx=0).stop 2024-05-27T08:01:47.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.121+0000 7fae39cbcb80 1 -- 172.21.15.90:0/2429542382 shutdown_connections 2024-05-27T08:01:47.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.122+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/2429542382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ccff5820 0x55b8cd002020 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.122+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/2429542382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8cce76e40 0x55b8ccff4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.122+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/2429542382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8ccfffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.123+0000 7fae39cbcb80 1 -- 172.21.15.90:0/2429542382 >> 172.21.15.90:0/2429542382 conn(0x55b8cced6fc0 msgr2=0x55b8ccff0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:47.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.123+0000 7fae39cbcb80 1 -- 172.21.15.90:0/2429542382 shutdown_connections 2024-05-27T08:01:47.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.124+0000 7fae39cbcb80 1 -- 172.21.15.90:0/2429542382 wait complete. 2024-05-27T08:01:47.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.125+0000 7fae39cbcb80 1 -- start start 2024-05-27T08:01:47.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.126+0000 7fae39cbcb80 1 Processor -- start 2024-05-27T08:01:47.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.127+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cce76e40 0x55b8ccfff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.127+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5410 0x55b8ccfffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.127+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5820 0x55b8cd000090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.127+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b8cce19fa0 con 0x55b8ccff5410 2024-05-27T08:01:47.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.127+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b8cceb4760 con 0x55b8ccff5820 2024-05-27T08:01:47.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.127+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8cd002750 con 0x55b8cce76e40 2024-05-27T08:01:47.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.127+0000 7fae39cbcb80 10 client.0 _ll_register_callbacks cb 0x55b8ccebdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:47.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.128+0000 7fae23e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cce76e40 0x55b8ccfff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.128+0000 7fae23e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cce76e40 0x55b8ccfff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:32922/0 (socket says 172.21.15.90:32922) 2024-05-27T08:01:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.128+0000 7fae23e00700 1 -- 172.21.15.90:0/7556283 learned_addr learned my addr 172.21.15.90:0/7556283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.129+0000 7fae1b400700 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5410 0x55b8ccfffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.129+0000 7fae29200700 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5820 0x55b8cd000090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.132+0000 7fae29200700 1 -- 172.21.15.90:0/7556283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cce76e40 msgr2=0x55b8ccfff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:47.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.132+0000 7fae29200700 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cce76e40 0x55b8ccfff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.132+0000 7fae29200700 1 -- 172.21.15.90:0/7556283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5410 msgr2=0x55b8ccfffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:47.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.132+0000 7fae29200700 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5410 0x55b8ccfffb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.132+0000 7fae29200700 1 -- 172.21.15.90:0/7556283 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8cce638d0 con 0x55b8ccff5820 2024-05-27T08:01:47.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.133+0000 7fae29200700 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5820 0x55b8cd000090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0aa560 tx=0x7fae1c0a65c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27c2821d06cf1f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:47.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.133+0000 7fae12000700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:01:47.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.134+0000 7fae12000700 1 -- 172.21.15.90:0/7556283 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0c4040 con 0x55b8ccff5820 2024-05-27T08:01:47.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.135+0000 7fae12000700 1 -- 172.21.15.90:0/7556283 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7fae1c0aa830 con 0x55b8ccff5820 2024-05-27T08:01:47.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.135+0000 7fae39cbcb80 1 -- 172.21.15.90:0/7556283 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae1c0aacd0 con 0x55b8ccff5820 2024-05-27T08:01:47.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.137+0000 7fae12000700 1 -- 172.21.15.90:0/7556283 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fae1c0e3040 con 0x55b8ccff5820 2024-05-27T08:01:47.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.137+0000 7fae39cbcb80 10 client.37643 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:47.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.138+0000 7fae39cbcb80 1 -- 172.21.15.90:0/7556283 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b8ccee4060 con 0x55b8ccff5820 2024-05-27T08:01:47.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.139+0000 7fae12000700 1 -- 172.21.15.90:0/7556283 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae1c0aacd0 con 0x55b8ccff5820 2024-05-27T08:01:47.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.140+0000 7fae12000700 1 -- 172.21.15.90:0/7556283 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fae1c028020 con 0x55b8ccff5820 2024-05-27T08:01:47.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.140+0000 7fae39cbcb80 10 client.37643 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:47.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.140+0000 7fae39cbcb80 10 client.37643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:47.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.140+0000 7fae39cbcb80 20 client.37643 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:47.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.140+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b8cd020500 0x55b8cd022920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.140+0000 7fae39cbcb80 4 client.37643 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:47.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.140+0000 7fae39cbcb80 1 -- 172.21.15.90:0/7556283 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8ccf25d90 con 0x55b8cd020500 2024-05-27T08:01:47.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.142+0000 7fae23e00700 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b8cd020500 0x55b8cd022920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.143+0000 7fae12000700 10 client.37643 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:47.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.144+0000 7fae23e00700 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b8cd020500 0x55b8cd022920 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467361086802b4d2 server_cookie=4b48abe172f76781 in_seq=0 out_seq=0 2024-05-27T08:01:47.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.145+0000 7fae12000700 1 -- 172.21.15.90:0/7556283 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b8ccf25d90 con 0x55b8cd020500 2024-05-27T08:01:47.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.145+0000 7fae12000700 10 client.37643 handle_command_reply: tid=0 2024-05-27T08:01:47.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.145+0000 7fae39cbcb80 1 client.37643 shutdown 2024-05-27T08:01:47.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.147+0000 7fae39cbcb80 1 -- 172.21.15.90:0/7556283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5820 msgr2=0x55b8cd000090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:47.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.147+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5820 0x55b8cd000090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0aa560 tx=0x7fae1c0a65c0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.148+0000 7fae39cbcb80 1 -- 172.21.15.90:0/7556283 shutdown_connections 2024-05-27T08:01:47.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.148+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b8cd020500 0x55b8cd022920 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.148+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5820 0x55b8cd000090 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.148+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cce76e40 0x55b8ccfff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.148+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/7556283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5410 0x55b8ccfffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.148+0000 7fae39cbcb80 1 -- 172.21.15.90:0/7556283 >> 172.21.15.90:0/7556283 conn(0x55b8cced6fc0 msgr2=0x55b8ccfefff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:47.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.148+0000 7fae12000700 0 client.37643 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.150+0000 7fae39cbcb80 1 -- 172.21.15.90:0/7556283 shutdown_connections 2024-05-27T08:01:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.150+0000 7fae39cbcb80 1 -- 172.21.15.90:0/7556283 wait complete. 2024-05-27T08:01:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.150+0000 7fae39cbcb80 20 client.37643 trim_cache size 0 max 16384 2024-05-27T08:01:47.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.194+0000 7fae39cbcb80 1 Processor -- start 2024-05-27T08:01:47.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.194+0000 7fae39cbcb80 1 -- start start 2024-05-27T08:01:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.195+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 0x55b8ccffae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.195+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8cce76e40 0x55b8cd000000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.195+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5410 0x55b8ccff97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.195+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae241afbc0 con 0x55b8ccff5410 2024-05-27T08:01:47.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.195+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae241acc10 con 0x55b8cce76e40 2024-05-27T08:01:47.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.195+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae241aee20 con 0x55b8cced6fc0 2024-05-27T08:01:47.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.195+0000 7fae1b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8cce76e40 0x55b8cd000000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.195+0000 7fae1b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8cce76e40 0x55b8cd000000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56400/0 (socket says 172.21.15.90:56400) 2024-05-27T08:01:47.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.195+0000 7fae1b400700 1 -- 172.21.15.90:0/199802858 learned_addr learned my addr 172.21.15.90:0/199802858 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:47.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.197+0000 7fae1b400700 1 -- 172.21.15.90:0/199802858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 msgr2=0x55b8ccffae30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:01:47.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.197+0000 7fae1b400700 1 --2- 172.21.15.90:0/199802858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 0x55b8ccffae30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.197+0000 7fae1b400700 1 -- 172.21.15.90:0/199802858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5410 msgr2=0x55b8ccff97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:47.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.197+0000 7fae1b400700 1 --2- 172.21.15.90:0/199802858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5410 0x55b8ccff97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.197+0000 7fae1b400700 1 -- 172.21.15.90:0/199802858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ccff2350 con 0x55b8cce76e40 2024-05-27T08:01:47.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.198+0000 7fae1b400700 1 --2- 172.21.15.90:0/199802858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8cce76e40 0x55b8cd000000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00a950 tx=0x7fae0c006cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b492393505a275f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:47.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.199+0000 7fae18c00700 1 -- 172.21.15.90:0/199802858 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c021620 con 0x55b8cce76e40 2024-05-27T08:01:47.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.199+0000 7fae18c00700 1 -- 172.21.15.90:0/199802858 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae0c026070 con 0x55b8cce76e40 2024-05-27T08:01:47.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.199+0000 7fae18c00700 1 -- 172.21.15.90:0/199802858 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c0165d0 con 0x55b8cce76e40 2024-05-27T08:01:47.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.201+0000 7fae39cbcb80 1 -- 172.21.15.90:0/199802858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8cce76e40 msgr2=0x55b8cd000000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:47.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.201+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/199802858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8cce76e40 0x55b8cd000000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00a950 tx=0x7fae0c006cc0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.201+0000 7fae39cbcb80 1 -- 172.21.15.90:0/199802858 shutdown_connections 2024-05-27T08:01:47.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.201+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/199802858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5410 0x55b8ccff97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.201+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/199802858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 0x55b8ccffae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.201+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/199802858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8cce76e40 0x55b8cd000000 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00a950 tx=0x7fae0c006cc0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.201+0000 7fae39cbcb80 1 -- 172.21.15.90:0/199802858 >> 172.21.15.90:0/199802858 conn(0x55b8ccfe9f60 msgr2=0x55b8cd00a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:47.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.202+0000 7fae39cbcb80 1 -- 172.21.15.90:0/199802858 shutdown_connections 2024-05-27T08:01:47.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.202+0000 7fae39cbcb80 1 -- 172.21.15.90:0/199802858 wait complete. 2024-05-27T08:01:47.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.203+0000 7fae39cbcb80 1 -- start start 2024-05-27T08:01:47.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.204+0000 7fae39cbcb80 1 Processor -- start 2024-05-27T08:01:47.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.205+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 0x55b8cd003900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.205+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8cd003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.206+0000 7fae39cbcb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5820 0x55b8cd004380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.206+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae241ae750 con 0x55b8ccff5820 2024-05-27T08:01:47.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.206+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae241ad2e0 con 0x55b8ccff5410 2024-05-27T08:01:47.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.206+0000 7fae39cbcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae241ad9b0 con 0x55b8cced6fc0 2024-05-27T08:01:47.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.206+0000 7fae39cbcb80 10 client.0 _ll_register_callbacks cb 0x55b8ccebdc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:47.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.206+0000 7fae23e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 0x55b8cd003900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.206+0000 7fae1b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8cd003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.207+0000 7fae23e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 0x55b8cd003900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:32942/0 (socket says 172.21.15.90:32942) 2024-05-27T08:01:47.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.207+0000 7fae1b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8cd003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:56414/0 (socket says 172.21.15.90:56414) 2024-05-27T08:01:47.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.207+0000 7fae1b400700 1 -- 172.21.15.90:0/202143875 learned_addr learned my addr 172.21.15.90:0/202143875 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:47.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.208+0000 7fae1b400700 1 -- 172.21.15.90:0/202143875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 msgr2=0x55b8cd003900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:47.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.208+0000 7fae1b400700 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 0x55b8cd003900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.208+0000 7fae1b400700 1 -- 172.21.15.90:0/202143875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5820 msgr2=0x55b8cd004380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:47.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.208+0000 7fae1b400700 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5820 0x55b8cd004380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.208+0000 7fae1b400700 1 -- 172.21.15.90:0/202143875 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8cd08a060 con 0x55b8ccff5410 2024-05-27T08:01:47.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.208+0000 7fae1b400700 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8cd003e40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae0c014cc0 tx=0x7fae0c014e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ceeff04a997b2f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:47.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.208+0000 7fae12000700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:01:47.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.209+0000 7fae12000700 1 -- 172.21.15.90:0/202143875 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0c02e070 con 0x55b8ccff5410 2024-05-27T08:01:47.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.209+0000 7fae12000700 1 -- 172.21.15.90:0/202143875 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7fae0c021930 con 0x55b8ccff5410 2024-05-27T08:01:47.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.209+0000 7fae39cbcb80 1 -- 172.21.15.90:0/202143875 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae0c026070 con 0x55b8ccff5410 2024-05-27T08:01:47.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.211+0000 7fae23e00700 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 0x55b8cd003900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:01:47.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.212+0000 7fae12000700 1 -- 172.21.15.90:0/202143875 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fae0c021b30 con 0x55b8ccff5410 2024-05-27T08:01:47.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.212+0000 7fae39cbcb80 10 client.37649 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:47.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.212+0000 7fae39cbcb80 1 -- 172.21.15.90:0/202143875 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b8cd08b2a0 con 0x55b8ccff5410 2024-05-27T08:01:47.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.213+0000 7fae12000700 1 -- 172.21.15.90:0/202143875 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae0c026070 con 0x55b8ccff5410 2024-05-27T08:01:47.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.213+0000 7fae12000700 1 -- 172.21.15.90:0/202143875 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fae0c03d420 con 0x55b8ccff5410 2024-05-27T08:01:47.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.213+0000 7fae39cbcb80 10 client.37649 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:47.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.213+0000 7fae39cbcb80 10 client.37649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:47.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.213+0000 7fae39cbcb80 20 client.37649 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:47.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.213+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b8cd08b920 0x55b8cd08dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:47.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.213+0000 7fae39cbcb80 4 client.37649 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:01:47.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.213+0000 7fae39cbcb80 1 -- 172.21.15.90:0/202143875 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8ccf1f570 con 0x55b8cd08b920 2024-05-27T08:01:47.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.215+0000 7fae23e00700 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b8cd08b920 0x55b8cd08dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:47.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.216+0000 7fae12000700 10 client.37649 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:47.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.217+0000 7fae23e00700 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b8cd08b920 0x55b8cd08dd40 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=672c441179a460e server_cookie=c9996513c5134fd in_seq=0 out_seq=0 2024-05-27T08:01:47.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.218+0000 7fae12000700 1 -- 172.21.15.90:0/202143875 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8ccf1f570 con 0x55b8cd08b920 2024-05-27T08:01:47.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.218+0000 7fae12000700 10 client.37649 handle_command_reply: tid=0 2024-05-27T08:01:47.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.218+0000 7fae39cbcb80 1 client.37649 shutdown 2024-05-27T08:01:47.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.219+0000 7fae39cbcb80 1 -- 172.21.15.90:0/202143875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 msgr2=0x55b8cd003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:47.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.219+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8cd003e40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae0c014cc0 tx=0x7fae0c014e80 comp rx=0 tx=0).stop 2024-05-27T08:01:47.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.221+0000 7fae39cbcb80 1 -- 172.21.15.90:0/202143875 shutdown_connections 2024-05-27T08:01:47.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.221+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b8cd08b920 0x55b8cd08dd40 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.221+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8ccff5820 0x55b8cd004380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.221+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8cced6fc0 0x55b8cd003900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.221+0000 7fae39cbcb80 1 --2- 172.21.15.90:0/202143875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8ccff5410 0x55b8cd003e40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.221+0000 7fae39cbcb80 1 -- 172.21.15.90:0/202143875 >> 172.21.15.90:0/202143875 conn(0x55b8ccfe9f60 msgr2=0x55b8ccfffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:47.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.221+0000 7fae12000700 0 client.37649 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:47.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.221+0000 7fae39cbcb80 1 -- 172.21.15.90:0/202143875 shutdown_connections 2024-05-27T08:01:47.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.222+0000 7fae39cbcb80 1 -- 172.21.15.90:0/202143875 wait complete. 2024-05-27T08:01:47.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.222+0000 7fae39cbcb80 20 client.37649 trim_cache size 0 max 16384 2024-05-27T08:01:47.224 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:01:47.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.223+0000 7fae29c00700 1 -- 172.21.15.90:0/718738505 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fae080b2c00 msgr2=0x7fae080b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:47.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.223+0000 7fae29c00700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fae080b2c00 0x7fae080b5110 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00afd0 tx=0x7fae0c00a4f0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.223+0000 7fae29c00700 1 -- 172.21.15.90:0/718738505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae24110960 msgr2=0x7fae241a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:47.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.223+0000 7fae29c00700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae24110960 0x7fae241a1690 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fae1c006010 tx=0x7fae1c00f510 comp rx=0 tx=0).stop 2024-05-27T08:01:47.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.223+0000 7fae29c00700 1 -- 172.21.15.90:0/718738505 shutdown_connections 2024-05-27T08:01:47.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.223+0000 7fae29c00700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae24110960 0x7fae241a1690 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.223+0000 7fae29c00700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fae080b2c00 0x7fae080b5110 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.224+0000 7fae29c00700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae24075d20 0x7fae2419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.224+0000 7fae29c00700 1 --2- 172.21.15.90:0/718738505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae24076670 0x7fae2419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:47.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.224+0000 7fae29c00700 1 -- 172.21.15.90:0/718738505 >> 172.21.15.90:0/718738505 conn(0x7fae240ffda0 msgr2=0x7fae24100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:47.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.226+0000 7fae29c00700 1 -- 172.21.15.90:0/718738505 shutdown_connections 2024-05-27T08:01:47.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:01:47.226+0000 7fae29c00700 1 -- 172.21.15.90:0/718738505 wait complete. 2024-05-27T08:01:48.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:48 smithi129 ceph-mon[35254]: pgmap v494: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.8 MiB/s wr, 488 op/s 2024-05-27T08:01:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 690.0 s, 440.4 tps, lat 223.185 ms stddev 120.348 2024-05-27T08:01:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.854+0000 7f3285a00640 1 -- 172.21.15.90:0/2059842283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280077610 msgr2=0x7f3280077a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.854+0000 7f3285a00640 1 --2- 172.21.15.90:0/2059842283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280077610 0x7f3280077a20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3270009860 tx=0x7f327002a340 comp rx=0 tx=0).stop 2024-05-27T08:01:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.854+0000 7f3285a00640 1 -- 172.21.15.90:0/2059842283 shutdown_connections 2024-05-27T08:01:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.854+0000 7f3285a00640 1 --2- 172.21.15.90:0/2059842283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114af0 0x7f3280116f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.854+0000 7f3285a00640 1 --2- 172.21.15.90:0/2059842283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280112190 0x7f32801145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.854+0000 7f3285a00640 1 --2- 172.21.15.90:0/2059842283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280077610 0x7f3280077a20 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.854+0000 7f3285a00640 1 -- 172.21.15.90:0/2059842283 >> 172.21.15.90:0/2059842283 conn(0x7f3280101520 msgr2=0x7f3280103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:48.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.854+0000 7f3285a00640 1 -- 172.21.15.90:0/2059842283 shutdown_connections 2024-05-27T08:01:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.854+0000 7f3285a00640 1 -- 172.21.15.90:0/2059842283 wait complete. 2024-05-27T08:01:48.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.857+0000 7f3285a00640 1 Processor -- start 2024-05-27T08:01:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.861+0000 7f3285a00640 1 -- start start 2024-05-27T08:01:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.862+0000 7f3285a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280077610 0x7f32801a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.862+0000 7f3285a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280112190 0x7f32801a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.862+0000 7f3285a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114af0 0x7f32801a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.862+0000 7f3285a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f328011a330 con 0x7f3280112190 2024-05-27T08:01:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.862+0000 7f3285a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f328011a1b0 con 0x7f3280114af0 2024-05-27T08:01:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.862+0000 7f3285a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f328011a4b0 con 0x7f3280077610 2024-05-27T08:01:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.863+0000 7f327ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280077610 0x7f32801a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.863+0000 7f327ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280077610 0x7f32801a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:55250/0 (socket says 172.21.15.90:55250) 2024-05-27T08:01:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.863+0000 7f327ea00640 1 -- 172.21.15.90:0/775487691 learned_addr learned my addr 172.21.15.90:0/775487691 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.863+0000 7f327f400640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114af0 0x7f32801a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.863+0000 7f3276000640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280112190 0x7f32801a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.863+0000 7f327f400640 1 -- 172.21.15.90:0/775487691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280077610 msgr2=0x7f32801a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.863+0000 7f327f400640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280077610 0x7f32801a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.863+0000 7f327f400640 1 -- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280112190 msgr2=0x7f32801a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.863+0000 7f327f400640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280112190 0x7f32801a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.863+0000 7f327f400640 1 -- 172.21.15.90:0/775487691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32801a5c90 con 0x7f3280114af0 2024-05-27T08:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.864+0000 7f3276000640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280112190 0x7f32801a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:01:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.864+0000 7f327f400640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114af0 0x7f32801a5650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f327800efc0 tx=0x7f327800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.865+0000 7f3277e00640 1 -- 172.21.15.90:0/775487691 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278003970 con 0x7f3280114af0 2024-05-27T08:01:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.865+0000 7f3285a00640 1 -- 172.21.15.90:0/775487691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32801a5f80 con 0x7f3280114af0 2024-05-27T08:01:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.865+0000 7f3285a00640 1 -- 172.21.15.90:0/775487691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32801ad900 con 0x7f3280114af0 2024-05-27T08:01:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.865+0000 7f3277e00640 1 -- 172.21.15.90:0/775487691 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3278009880 con 0x7f3280114af0 2024-05-27T08:01:48.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.867+0000 7f3277e00640 1 -- 172.21.15.90:0/775487691 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278003970 con 0x7f3280114af0 2024-05-27T08:01:48.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.874+0000 7f328c9b0740 1 Processor -- start 2024-05-27T08:01:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.875+0000 7f3277e00640 1 -- 172.21.15.90:0/775487691 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f327801d020 con 0x7f3280114af0 2024-05-27T08:01:48.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.875+0000 7f328c9b0740 1 -- start start 2024-05-27T08:01:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.876+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297c0f1b0 0x55c297fb1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.876+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297be62f0 0x55c297fb1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.876+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 0x55c297fb22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.876+0000 7f328c9b0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c297f22b90 con 0x55c297bee4d0 2024-05-27T08:01:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.876+0000 7f328c9b0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c297f12730 con 0x55c297be62f0 2024-05-27T08:01:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.876+0000 7f328c9b0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c297f16770 con 0x55c297c0f1b0 2024-05-27T08:01:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.877+0000 7f3276000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 0x55c297fb22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.878+0000 7f3276000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 0x55c297fb22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46978/0 (socket says 172.21.15.90:46978) 2024-05-27T08:01:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.878+0000 7f3276000640 1 -- 172.21.15.90:0/3911521464 learned_addr learned my addr 172.21.15.90:0/3911521464 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.878+0000 7f327ea00640 1 --2- 172.21.15.90:0/3911521464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297be62f0 0x55c297fb1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.878+0000 7f3276000640 1 --2- 172.21.15.90:0/3911521464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297c0f1b0 0x55c297fb1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.878+0000 7f327ea00640 1 -- 172.21.15.90:0/3911521464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297c0f1b0 msgr2=0x55c297fb1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.878+0000 7f327ea00640 1 --2- 172.21.15.90:0/3911521464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297c0f1b0 0x55c297fb1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.878+0000 7f327ea00640 1 -- 172.21.15.90:0/3911521464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 msgr2=0x55c297fb22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.878+0000 7f327ea00640 1 --2- 172.21.15.90:0/3911521464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 0x55c297fb22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.878+0000 7f327ea00640 1 -- 172.21.15.90:0/3911521464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c297eab8b0 con 0x55c297be62f0 2024-05-27T08:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.878+0000 7f3276000640 1 --2- 172.21.15.90:0/3911521464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297c0f1b0 0x55c297fb1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.879+0000 7f327ea00640 1 --2- 172.21.15.90:0/3911521464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297be62f0 0x55c297fb1da0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3270009830 tx=0x7f32700095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29d313ea1a352dd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.879+0000 7f3277e00640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2d50 0x7f32640b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.880+0000 7f326fe00640 1 -- 172.21.15.90:0/3911521464 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270043070 con 0x55c297be62f0 2024-05-27T08:01:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.881+0000 7f326fe00640 1 -- 172.21.15.90:0/3911521464 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f32700044d0 con 0x55c297be62f0 2024-05-27T08:01:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.881+0000 7f326fe00640 1 -- 172.21.15.90:0/3911521464 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270043070 con 0x55c297be62f0 2024-05-27T08:01:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.881+0000 7f3276000640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2d50 0x7f32640b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.881+0000 7f3277e00640 1 -- 172.21.15.90:0/775487691 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f3278013070 con 0x7f3280114af0 2024-05-27T08:01:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.882+0000 7f3276000640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2d50 0x7f32640b5260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55c297fae9d0 tx=0x7f3268009b80 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.882+0000 7f328c9b0740 1 -- 172.21.15.90:0/3911521464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297be62f0 msgr2=0x55c297fb1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.882+0000 7f328c9b0740 1 --2- 172.21.15.90:0/3911521464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297be62f0 0x55c297fb1da0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3270009830 tx=0x7f32700095a0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.883+0000 7f328c9b0740 1 -- 172.21.15.90:0/3911521464 shutdown_connections 2024-05-27T08:01:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.883+0000 7f328c9b0740 1 --2- 172.21.15.90:0/3911521464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297be62f0 0x55c297fb1da0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.883+0000 7f328c9b0740 1 --2- 172.21.15.90:0/3911521464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 0x55c297fb22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.883+0000 7f328c9b0740 1 --2- 172.21.15.90:0/3911521464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297c0f1b0 0x55c297fb1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.884+0000 7f328c9b0740 1 -- 172.21.15.90:0/3911521464 >> 172.21.15.90:0/3911521464 conn(0x55c297c03300 msgr2=0x55c297fae010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.884+0000 7f328c9b0740 1 -- 172.21.15.90:0/3911521464 shutdown_connections 2024-05-27T08:01:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.884+0000 7f328c9b0740 1 -- 172.21.15.90:0/3911521464 wait complete. 2024-05-27T08:01:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.886+0000 7f328c9b0740 1 -- start start 2024-05-27T08:01:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.886+0000 7f328c9b0740 1 Processor -- start 2024-05-27T08:01:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.887+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 0x55c297fbb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.887+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 0x55c297fbbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.887+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 0x55c297fc7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.888+0000 7f328c9b0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c297bf8790 con 0x55c297bee4d0 2024-05-27T08:01:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.888+0000 7f328c9b0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c297ebabc0 con 0x55c297c0f1b0 2024-05-27T08:01:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.888+0000 7f327ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 0x55c297fbb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.888+0000 7f328c9b0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c297fbe930 con 0x55c297be62f0 2024-05-27T08:01:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.888+0000 7f328c9b0740 10 client.0 _ll_register_callbacks cb 0x55c297e98400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.888+0000 7f327f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 0x55c297fc7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.888+0000 7f327ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 0x55c297fbb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:55260/0 (socket says 172.21.15.90:55260) 2024-05-27T08:01:48.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.888+0000 7f327ea00640 1 -- 172.21.15.90:0/2123481029 learned_addr learned my addr 172.21.15.90:0/2123481029 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.889+0000 7f3276000640 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 0x55c297fbbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.889+0000 7f327f400640 1 -- 172.21.15.90:0/2123481029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 msgr2=0x55c297fbb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.889+0000 7f327f400640 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 0x55c297fbb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.889+0000 7f327f400640 1 -- 172.21.15.90:0/2123481029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 msgr2=0x55c297fbbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.889+0000 7f327f400640 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 0x55c297fbbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.889+0000 7f327f400640 1 -- 172.21.15.90:0/2123481029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c297c27210 con 0x55c297c0f1b0 2024-05-27T08:01:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.889+0000 7f3276000640 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 0x55c297fbbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.890+0000 7f327f400640 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 0x55c297fc7cc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f32780df050 tx=0x7f3278004620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7c20362e5d473e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.890+0000 7f326cc00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:01:48.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.890+0000 7f326cc00640 1 -- 172.21.15.90:0/2123481029 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780a8c70 con 0x55c297c0f1b0 2024-05-27T08:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.890+0000 7f326cc00640 1 -- 172.21.15.90:0/2123481029 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f32780b17c0 con 0x55c297c0f1b0 2024-05-27T08:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.891+0000 7f328c9b0740 1 -- 172.21.15.90:0/2123481029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32700044d0 con 0x55c297c0f1b0 2024-05-27T08:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.891+0000 7f326cc00640 1 -- 172.21.15.90:0/2123481029 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f32780c8050 con 0x55c297c0f1b0 2024-05-27T08:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.891+0000 7f328c9b0740 10 client.37661 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.891+0000 7f328c9b0740 1 -- 172.21.15.90:0/2123481029 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55c297cf3620 con 0x55c297c0f1b0 2024-05-27T08:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.892+0000 7f326cc00640 1 -- 172.21.15.90:0/2123481029 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3278009880 con 0x55c297c0f1b0 2024-05-27T08:01:48.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.892+0000 7f326cc00640 1 -- 172.21.15.90:0/2123481029 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f327800b040 con 0x55c297c0f1b0 2024-05-27T08:01:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.892+0000 7f328c9b0740 10 client.37661 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.892+0000 7f328c9b0740 10 client.37661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.892+0000 7f328c9b0740 20 client.37661 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.892+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c297bcdd80 0x55c297fcdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.892+0000 7f328c9b0740 4 client.37661 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:01:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.892+0000 7f328c9b0740 1 -- 172.21.15.90:0/2123481029 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f32700044d0 con 0x55c297bcdd80 2024-05-27T08:01:48.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.893+0000 7f327ea00640 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c297bcdd80 0x55c297fcdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.894+0000 7f327ea00640 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c297bcdd80 0x55c297fcdc30 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e088174e0b475eff server_cookie=ecf6f263dd6f021c in_seq=0 out_seq=0 2024-05-27T08:01:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.894+0000 7f326cc00640 10 client.37661 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.895+0000 7f326cc00640 1 -- 172.21.15.90:0/2123481029 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f32700044d0 con 0x55c297bcdd80 2024-05-27T08:01:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.895+0000 7f326cc00640 10 client.37661 handle_command_reply: tid=0 2024-05-27T08:01:48.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.896+0000 7f328c9b0740 1 client.37661 shutdown 2024-05-27T08:01:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.897+0000 7f328c9b0740 1 -- 172.21.15.90:0/2123481029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 msgr2=0x55c297fc7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.897+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 0x55c297fc7cc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f32780df050 tx=0x7f3278004620 comp rx=0 tx=0).stop 2024-05-27T08:01:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.898+0000 7f328c9b0740 1 -- 172.21.15.90:0/2123481029 shutdown_connections 2024-05-27T08:01:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.898+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c297bcdd80 0x55c297fcdc30 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.898+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 0x55c297fc7cc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.898+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297bee4d0 0x55c297fbbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.898+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2123481029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 0x55c297fbb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.898+0000 7f328c9b0740 1 -- 172.21.15.90:0/2123481029 >> 172.21.15.90:0/2123481029 conn(0x55c297c03300 msgr2=0x55c297fb1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.898+0000 7f326cc00640 0 client.37661 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.898+0000 7f328c9b0740 1 -- 172.21.15.90:0/2123481029 shutdown_connections 2024-05-27T08:01:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.899+0000 7f328c9b0740 1 -- 172.21.15.90:0/2123481029 wait complete. 2024-05-27T08:01:48.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.899+0000 7f328c9b0740 20 client.37661 trim_cache size 0 max 16384 2024-05-27T08:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.929+0000 7f328c9b0740 1 Processor -- start 2024-05-27T08:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.930+0000 7f328c9b0740 1 -- start start 2024-05-27T08:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.930+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 0x55c297fbe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.930+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297bee4d0 0x55c297fc7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.930+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297c0f1b0 0x55c297fc7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.930+0000 7f328c9b0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f325c011940 con 0x55c297c0f1b0 2024-05-27T08:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.930+0000 7f328c9b0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f325c00e990 con 0x55c297bee4d0 2024-05-27T08:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.930+0000 7f328c9b0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f325c010ba0 con 0x55c297be62f0 2024-05-27T08:01:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.931+0000 7f3276000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297bee4d0 0x55c297fc7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.931+0000 7f3276000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297bee4d0 0x55c297fc7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:60348/0 (socket says 172.21.15.90:60348) 2024-05-27T08:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.931+0000 7f3276000640 1 -- 172.21.15.90:0/1200344071 learned_addr learned my addr 172.21.15.90:0/1200344071 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.932+0000 7f327ea00640 1 --2- 172.21.15.90:0/1200344071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 0x55c297fbe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.932+0000 7f3276000640 1 -- 172.21.15.90:0/1200344071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 msgr2=0x55c297fbe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.932+0000 7f3276000640 1 --2- 172.21.15.90:0/1200344071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 0x55c297fbe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.932+0000 7f3276000640 1 -- 172.21.15.90:0/1200344071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297c0f1b0 msgr2=0x55c297fc7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.932+0000 7f3276000640 1 --2- 172.21.15.90:0/1200344071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297c0f1b0 0x55c297fc7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.932+0000 7f3276000640 1 -- 172.21.15.90:0/1200344071 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c297fb9f20 con 0x55c297bee4d0 2024-05-27T08:01:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.932+0000 7f327ea00640 1 --2- 172.21.15.90:0/1200344071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 0x55c297fbe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:01:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.934+0000 7f3276000640 1 --2- 172.21.15.90:0/1200344071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297bee4d0 0x55c297fc7a30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f326800ed80 tx=0x7f32680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b76a253955bdf873 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.935+0000 7f326f400640 1 -- 172.21.15.90:0/1200344071 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326800edb0 con 0x55c297bee4d0 2024-05-27T08:01:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.935+0000 7f326f400640 1 -- 172.21.15.90:0/1200344071 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f326801e460 con 0x55c297bee4d0 2024-05-27T08:01:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.935+0000 7f326f400640 1 -- 172.21.15.90:0/1200344071 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326801f400 con 0x55c297bee4d0 2024-05-27T08:01:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.936+0000 7f328c9b0740 1 -- 172.21.15.90:0/1200344071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297bee4d0 msgr2=0x55c297fc7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.936+0000 7f328c9b0740 1 --2- 172.21.15.90:0/1200344071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297bee4d0 0x55c297fc7a30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f326800ed80 tx=0x7f32680046c0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.936+0000 7f328c9b0740 1 -- 172.21.15.90:0/1200344071 shutdown_connections 2024-05-27T08:01:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.936+0000 7f328c9b0740 1 --2- 172.21.15.90:0/1200344071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297bee4d0 0x55c297fc7a30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.936+0000 7f328c9b0740 1 --2- 172.21.15.90:0/1200344071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297c0f1b0 0x55c297fc7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.936+0000 7f328c9b0740 1 --2- 172.21.15.90:0/1200344071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297be62f0 0x55c297fbe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.936+0000 7f328c9b0740 1 -- 172.21.15.90:0/1200344071 >> 172.21.15.90:0/1200344071 conn(0x55c297c03300 msgr2=0x55c297fb6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.936+0000 7f328c9b0740 1 -- 172.21.15.90:0/1200344071 shutdown_connections 2024-05-27T08:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.937+0000 7f328c9b0740 1 -- 172.21.15.90:0/1200344071 wait complete. 2024-05-27T08:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.937+0000 7f328c9b0740 1 -- start start 2024-05-27T08:01:48.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.937+0000 7f328c9b0740 1 Processor -- start 2024-05-27T08:01:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.938+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297be62f0 0x55c297fc1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.938+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297bee4d0 0x55c297fc1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.938+0000 7f328c9b0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 0x55c2980342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.939+0000 7f328c9b0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f325c0104d0 con 0x55c297be62f0 2024-05-27T08:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.939+0000 7f328c9b0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f325c00f060 con 0x55c297c0f1b0 2024-05-27T08:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.939+0000 7f328c9b0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f325c00f730 con 0x55c297bee4d0 2024-05-27T08:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.939+0000 7f3276000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297bee4d0 0x55c297fc1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.939+0000 7f327f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 0x55c2980342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.939+0000 7f3276000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297bee4d0 0x55c297fc1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:55286/0 (socket says 172.21.15.90:55286) 2024-05-27T08:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.939+0000 7f3276000640 1 -- 172.21.15.90:0/2102860886 learned_addr learned my addr 172.21.15.90:0/2102860886 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:01:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.939+0000 7f327ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297be62f0 0x55c297fc1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.939+0000 7f328c9b0740 10 client.0 _ll_register_callbacks cb 0x7f3278004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:01:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.940+0000 7f327f400640 1 -- 172.21.15.90:0/2102860886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297bee4d0 msgr2=0x55c297fc1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.940+0000 7f327f400640 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297bee4d0 0x55c297fc1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.940+0000 7f327f400640 1 -- 172.21.15.90:0/2102860886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297be62f0 msgr2=0x55c297fc1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:01:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.940+0000 7f327f400640 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297be62f0 0x55c297fc1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.940+0000 7f327f400640 1 -- 172.21.15.90:0/2102860886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c298034800 con 0x55c297c0f1b0 2024-05-27T08:01:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.940+0000 7f3276000640 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297bee4d0 0x55c297fc1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:01:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.941+0000 7f327f400640 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 0x55c2980342c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f32780af8c0 tx=0x7f32780a44d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccd82d23d599db04 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:01:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.943+0000 7f326cc00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:01:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.943+0000 7f326cc00640 1 -- 172.21.15.90:0/2102860886 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780b5060 con 0x55c297c0f1b0 2024-05-27T08:01:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.943+0000 7f326cc00640 1 -- 172.21.15.90:0/2102860886 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f32780a47d0 con 0x55c297c0f1b0 2024-05-27T08:01:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.943+0000 7f328c9b0740 1 -- 172.21.15.90:0/2102860886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f326801e460 con 0x55c297c0f1b0 2024-05-27T08:01:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.943+0000 7f326cc00640 1 -- 172.21.15.90:0/2102860886 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f32780a8c70 con 0x55c297c0f1b0 2024-05-27T08:01:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.943+0000 7f328c9b0740 10 client.37667 fetch_fsmap learned FSMap version 64 2024-05-27T08:01:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.943+0000 7f328c9b0740 1 -- 172.21.15.90:0/2102860886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55c298035840 con 0x55c297c0f1b0 2024-05-27T08:01:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.944+0000 7f326cc00640 1 -- 172.21.15.90:0/2102860886 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f326801e460 con 0x55c297c0f1b0 2024-05-27T08:01:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.944+0000 7f326cc00640 1 -- 172.21.15.90:0/2102860886 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f327800e0b0 con 0x55c297c0f1b0 2024-05-27T08:01:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.944+0000 7f328c9b0740 10 client.37667 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:01:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.944+0000 7f328c9b0740 10 client.37667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:01:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.944+0000 7f328c9b0740 20 client.37667 populate_metadata read hostname 'smithi090' 2024-05-27T08:01:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.944+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c297ba31c0 0x55c298037f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:01:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.944+0000 7f328c9b0740 4 client.37667 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:01:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.944+0000 7f328c9b0740 1 -- 172.21.15.90:0/2102860886 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c297f24ae0 con 0x55c297ba31c0 2024-05-27T08:01:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.945+0000 7f327ea00640 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c297ba31c0 0x55c298037f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:01:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.946+0000 7f326cc00640 10 client.37667 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.946+0000 7f327ea00640 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c297ba31c0 0x55c298037f20 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2960a30ae95e6fec server_cookie=94aa1c4585a620a0 in_seq=0 out_seq=0 2024-05-27T08:01:48.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:48 smithi090 ceph-mon[31429]: pgmap v494: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.8 MiB/s wr, 488 op/s 2024-05-27T08:01:49.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:48 smithi136 ceph-mon[35324]: pgmap v494: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.8 MiB/s wr, 488 op/s 2024-05-27T08:01:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.999+0000 7f326cc00640 1 -- 172.21.15.90:0/2102860886 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c297f24ae0 con 0x55c297ba31c0 2024-05-27T08:01:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.999+0000 7f326cc00640 10 client.37667 handle_command_reply: tid=0 2024-05-27T08:01:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.999+0000 7f328c9b0740 1 client.37667 shutdown 2024-05-27T08:01:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.999+0000 7f328c9b0740 1 -- 172.21.15.90:0/2102860886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 msgr2=0x55c2980342c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:48.999+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 0x55c2980342c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f32780af8c0 tx=0x7f32780a44d0 comp rx=0 tx=0).stop 2024-05-27T08:01:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.000+0000 7f328c9b0740 1 -- 172.21.15.90:0/2102860886 shutdown_connections 2024-05-27T08:01:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.000+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c297ba31c0 0x55c298037f20 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.000+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c297c0f1b0 0x55c2980342c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.000+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c297bee4d0 0x55c297fc1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.000+0000 7f328c9b0740 1 --2- 172.21.15.90:0/2102860886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c297be62f0 0x55c297fc1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.000+0000 7f328c9b0740 1 -- 172.21.15.90:0/2102860886 >> 172.21.15.90:0/2102860886 conn(0x55c297c03300 msgr2=0x55c297fbd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.000+0000 7f326cc00640 0 client.37667 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:01:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.000+0000 7f328c9b0740 1 -- 172.21.15.90:0/2102860886 shutdown_connections 2024-05-27T08:01:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.001+0000 7f328c9b0740 1 -- 172.21.15.90:0/2102860886 wait complete. 2024-05-27T08:01:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.002+0000 7f328c9b0740 20 client.37667 trim_cache size 0 max 16384 2024-05-27T08:01:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:01:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:01:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:01:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.003+0000 7f3285a00640 1 -- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2d50 msgr2=0x7f32640b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.003+0000 7f3285a00640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2d50 0x7f32640b5260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55c297fae9d0 tx=0x7f3268009b80 comp rx=0 tx=0).stop 2024-05-27T08:01:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.003+0000 7f3285a00640 1 -- 172.21.15.90:0/775487691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114af0 msgr2=0x7f32801a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:01:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.003+0000 7f3285a00640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114af0 0x7f32801a5650 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f327800efc0 tx=0x7f327800c800 comp rx=0 tx=0).stop 2024-05-27T08:01:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.003+0000 7f3285a00640 1 -- 172.21.15.90:0/775487691 shutdown_connections 2024-05-27T08:01:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.003+0000 7f3285a00640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f32640b2d50 0x7f32640b5260 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.003+0000 7f3285a00640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3280114af0 0x7f32801a5650 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.003+0000 7f3285a00640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3280112190 0x7f32801a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.003+0000 7f3285a00640 1 --2- 172.21.15.90:0/775487691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3280077610 0x7f32801a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:01:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.003+0000 7f3285a00640 1 -- 172.21.15.90:0/775487691 >> 172.21.15.90:0/775487691 conn(0x7f3280101520 msgr2=0x7f3280113630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:01:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.004+0000 7f3285a00640 1 -- 172.21.15.90:0/775487691 shutdown_connections 2024-05-27T08:01:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:01:49.004+0000 7f3285a00640 1 -- 172.21.15.90:0/775487691 wait complete. 2024-05-27T08:01:50.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:50 smithi090 ceph-mon[31429]: pgmap v495: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 8.1 MiB/s wr, 703 op/s 2024-05-27T08:01:51.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:50 smithi136 ceph-mon[35324]: pgmap v495: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 8.1 MiB/s wr, 703 op/s 2024-05-27T08:01:51.034 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:01:51.034 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:01:51.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:50 smithi129 ceph-mon[35254]: pgmap v495: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 8.1 MiB/s wr, 703 op/s 2024-05-27T08:01:52.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:52 smithi090 ceph-mon[31429]: pgmap v496: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-05-27T08:01:53.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:52 smithi136 ceph-mon[35324]: pgmap v496: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-05-27T08:01:53.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:52 smithi129 ceph-mon[35254]: pgmap v496: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 6.1 MiB/s wr, 516 op/s 2024-05-27T08:01:53.147 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:01:53.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 695.0 s, 546.2 tps, lat 186.391 ms stddev 154.530 2024-05-27T08:01:54.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:54 smithi090 ceph-mon[31429]: pgmap v497: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 6.3 MiB/s wr, 544 op/s 2024-05-27T08:01:55.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:54 smithi136 ceph-mon[35324]: pgmap v497: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 6.3 MiB/s wr, 544 op/s 2024-05-27T08:01:55.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:54 smithi129 ceph-mon[35254]: pgmap v497: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 6.3 MiB/s wr, 544 op/s 2024-05-27T08:01:56.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:56 smithi090 ceph-mon[31429]: pgmap v498: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.5 MiB/s wr, 661 op/s 2024-05-27T08:01:57.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:56 smithi136 ceph-mon[35324]: pgmap v498: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.5 MiB/s wr, 661 op/s 2024-05-27T08:01:57.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:56 smithi129 ceph-mon[35254]: pgmap v498: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.5 MiB/s wr, 661 op/s 2024-05-27T08:01:58.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 700.0 s, 299.0 tps, lat 309.909 ms stddev 304.166 2024-05-27T08:01:58.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:01:58 smithi090 ceph-mon[31429]: pgmap v499: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 MiB/s wr, 468 op/s 2024-05-27T08:01:59.005 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:01:58 smithi136 ceph-mon[35324]: pgmap v499: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 MiB/s wr, 468 op/s 2024-05-27T08:01:59.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:01:58 smithi129 ceph-mon[35254]: pgmap v499: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 MiB/s wr, 468 op/s 2024-05-27T08:02:00.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.669+0000 7f4b25e00700 1 -- 172.21.15.90:0/568515935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2010c5d0 msgr2=0x7f4b2010c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:00.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.669+0000 7f4b25e00700 1 --2- 172.21.15.90:0/568515935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2010c5d0 0x7f4b2010c9e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009990 tx=0x7f4b10009ca0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.669+0000 7f4b25e00700 1 -- 172.21.15.90:0/568515935 shutdown_connections 2024-05-27T08:02:00.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.669+0000 7f4b25e00700 1 --2- 172.21.15.90:0/568515935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b20106ee0 0x7f4b20113370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.669+0000 7f4b25e00700 1 --2- 172.21.15.90:0/568515935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2010c5d0 0x7f4b2010c9e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.669+0000 7f4b25e00700 1 --2- 172.21.15.90:0/568515935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20106590 0x7f4b201069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.669+0000 7f4b25e00700 1 -- 172.21.15.90:0/568515935 >> 172.21.15.90:0/568515935 conn(0x7f4b20102070 msgr2=0x7f4b201044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:00.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.669+0000 7f4b25e00700 1 -- 172.21.15.90:0/568515935 shutdown_connections 2024-05-27T08:02:00.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.669+0000 7f4b25e00700 1 -- 172.21.15.90:0/568515935 wait complete. 2024-05-27T08:02:00.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.671+0000 7f4b25e00700 1 Processor -- start 2024-05-27T08:02:00.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.671+0000 7f4b25e00700 1 -- start start 2024-05-27T08:02:00.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.672+0000 7f4b25e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b20106590 0x7f4b20110900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.672+0000 7f4b25e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20106ee0 0x7f4b20110e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.672+0000 7f4b25e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b201113d0 0x7f4b201ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.672+0000 7f4b25e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4b201165e0 con 0x7f4b201113d0 2024-05-27T08:02:00.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.672+0000 7f4b25e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20116460 con 0x7f4b20106590 2024-05-27T08:02:00.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.672+0000 7f4b25e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20116760 con 0x7f4b20106ee0 2024-05-27T08:02:00.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.672+0000 7f4b1e000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20106ee0 0x7f4b20110e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.672+0000 7f4b1e000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20106ee0 0x7f4b20110e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:40868/0 (socket says 172.21.15.90:40868) 2024-05-27T08:02:00.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.672+0000 7f4b1e000700 1 -- 172.21.15.90:0/3613425461 learned_addr learned my addr 172.21.15.90:0/3613425461 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:00.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.672+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b20106590 0x7f4b20110900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.673+0000 7f4b1e000700 1 -- 172.21.15.90:0/3613425461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b20106590 msgr2=0x7f4b20110900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:00.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.673+0000 7f4b1e000700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b20106590 0x7f4b20110900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.673+0000 7f4b1e000700 1 -- 172.21.15.90:0/3613425461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b201113d0 msgr2=0x7f4b201ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:00.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.673+0000 7f4b1e000700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b201113d0 0x7f4b201ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.673+0000 7f4b1e000700 1 -- 172.21.15.90:0/3613425461 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b201adb30 con 0x7f4b20106ee0 2024-05-27T08:02:00.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.673+0000 7f4b1e000700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20106ee0 0x7f4b20110e40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00d950 tx=0x7f4b0c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:00.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.675+0000 7f4b24a00700 1 -- 172.21.15.90:0/3613425461 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c017070 con 0x7f4b20106ee0 2024-05-27T08:02:00.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.675+0000 7f4b24a00700 1 -- 172.21.15.90:0/3613425461 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b0c009dc0 con 0x7f4b20106ee0 2024-05-27T08:02:00.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.675+0000 7f4b24a00700 1 -- 172.21.15.90:0/3613425461 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c010de0 con 0x7f4b20106ee0 2024-05-27T08:02:00.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.675+0000 7f4b25e00700 1 -- 172.21.15.90:0/3613425461 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b201ade20 con 0x7f4b20106ee0 2024-05-27T08:02:00.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.675+0000 7f4b25e00700 1 -- 172.21.15.90:0/3613425461 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b201ae220 con 0x7f4b20106ee0 2024-05-27T08:02:00.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.683+0000 7f4b24a00700 1 -- 172.21.15.90:0/3613425461 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4b0c01a070 con 0x7f4b20106ee0 2024-05-27T08:02:00.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.685+0000 7f4b24a00700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4b040b2b40 0x7f4b040b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.685+0000 7f4b24a00700 1 -- 172.21.15.90:0/3613425461 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f4b0c0e1de0 con 0x7f4b20106ee0 2024-05-27T08:02:00.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.686+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4b040b2b40 0x7f4b040b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.687+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4b040b2b40 0x7f4b040b5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4b10000c00 tx=0x7f4b10005fb0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:00.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.852+0000 7f4b3491db80 1 Processor -- start 2024-05-27T08:02:00.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.852+0000 7f4b3491db80 1 -- start start 2024-05-27T08:02:00.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.853+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a771d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.853+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a771d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.853+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a772a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.854+0000 7f4b1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a772a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.854+0000 7f4b1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a772a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59710/0 (socket says 172.21.15.90:59710) 2024-05-27T08:02:00.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.854+0000 7f4b1f400700 1 -- 172.21.15.90:0/10780556 learned_addr learned my addr 172.21.15.90:0/10780556 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:00.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.854+0000 7f4b3491db80 1 -- 172.21.15.90:0/10780556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5601a764bec0 con 0x5601a771def0 2024-05-27T08:02:00.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.854+0000 7f4b3491db80 1 -- 172.21.15.90:0/10780556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5601a763ba60 con 0x5601a759eec0 2024-05-27T08:02:00.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.854+0000 7f4b3491db80 1 -- 172.21.15.90:0/10780556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601a763faa0 con 0x5601a768aa70 2024-05-27T08:02:00.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.854+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a771d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.854+0000 7f4b1e000700 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a771d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.855+0000 7f4b1ea00700 1 -- 172.21.15.90:0/10780556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 msgr2=0x5601a771d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:00.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.855+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a771d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.855+0000 7f4b1ea00700 1 -- 172.21.15.90:0/10780556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 msgr2=0x5601a772a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:00.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.855+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a772a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.855+0000 7f4b1ea00700 1 -- 172.21.15.90:0/10780556 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a76044a0 con 0x5601a768aa70 2024-05-27T08:02:00.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.856+0000 7f4b1f400700 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a772a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:00.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.856+0000 7f4b1e000700 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a771d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:00.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.857+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a771d3b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4b201076b0 tx=0x7f4b1001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82bfcef3f25abe8b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:00.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.857+0000 7f4b14c00700 1 -- 172.21.15.90:0/10780556 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10003680 con 0x5601a768aa70 2024-05-27T08:02:00.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.858+0000 7f4b14c00700 1 -- 172.21.15.90:0/10780556 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b10003cb0 con 0x5601a768aa70 2024-05-27T08:02:00.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.858+0000 7f4b14c00700 1 -- 172.21.15.90:0/10780556 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10025b90 con 0x5601a768aa70 2024-05-27T08:02:00.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:00 smithi090 ceph-mon[31429]: pgmap v500: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.5 MiB/s wr, 681 op/s 2024-05-27T08:02:00.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.859+0000 7f4b3491db80 1 -- 172.21.15.90:0/10780556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 msgr2=0x5601a771d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:00.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.859+0000 7f4b3491db80 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a771d3b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4b201076b0 tx=0x7f4b1001e000 comp rx=0 tx=0).stop 2024-05-27T08:02:00.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.859+0000 7f4b3491db80 1 -- 172.21.15.90:0/10780556 shutdown_connections 2024-05-27T08:02:00.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.859+0000 7f4b3491db80 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a772a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.859+0000 7f4b3491db80 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a771d3b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.859+0000 7f4b3491db80 1 --2- 172.21.15.90:0/10780556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a771d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.859+0000 7f4b3491db80 1 -- 172.21.15.90:0/10780556 >> 172.21.15.90:0/10780556 conn(0x5601a7638410 msgr2=0x5601a7719540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:00.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.861+0000 7f4b3491db80 1 -- 172.21.15.90:0/10780556 shutdown_connections 2024-05-27T08:02:00.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.861+0000 7f4b3491db80 1 -- 172.21.15.90:0/10780556 wait complete. 2024-05-27T08:02:00.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.862+0000 7f4b3491db80 1 -- start start 2024-05-27T08:02:00.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.862+0000 7f4b3491db80 1 Processor -- start 2024-05-27T08:02:00.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.863+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a759eec0 0x5601a77276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.863+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a768aa70 0x5601a7727be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.863+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a7728120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.863+0000 7f4b3491db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5601a7542020 con 0x5601a771def0 2024-05-27T08:02:00.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.863+0000 7f4b3491db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5601a75dc7e0 con 0x5601a768aa70 2024-05-27T08:02:00.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.863+0000 7f4b3491db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601a772aba0 con 0x5601a759eec0 2024-05-27T08:02:00.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.863+0000 7f4b3491db80 10 client.0 _ll_register_callbacks cb 0x5601a75e5cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:00.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.863+0000 7f4b1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a7728120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.864+0000 7f4b1f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a7728120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59716/0 (socket says 172.21.15.90:59716) 2024-05-27T08:02:00.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.864+0000 7f4b1f400700 1 -- 172.21.15.90:0/2101551023 learned_addr learned my addr 172.21.15.90:0/2101551023 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:00.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.864+0000 7f4b1e000700 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a768aa70 0x5601a7727be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.864+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a759eec0 0x5601a77276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.865+0000 7f4b1f400700 1 -- 172.21.15.90:0/2101551023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a759eec0 msgr2=0x5601a77276a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:00.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.865+0000 7f4b1f400700 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a759eec0 0x5601a77276a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.865+0000 7f4b1f400700 1 -- 172.21.15.90:0/2101551023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a768aa70 msgr2=0x5601a7727be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:00.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.865+0000 7f4b1f400700 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a768aa70 0x5601a7727be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.865+0000 7f4b1f400700 1 -- 172.21.15.90:0/2101551023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a758b950 con 0x5601a771def0 2024-05-27T08:02:00.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.865+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a759eec0 0x5601a77276a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:00.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.865+0000 7f4b1f400700 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a7728120 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4b18011db0 tx=0x7f4b1800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1a03ea038746c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:00.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.866+0000 7f4b09600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:00.869 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:00 smithi129 ceph-mon[35254]: pgmap v500: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.5 MiB/s wr, 681 op/s 2024-05-27T08:02:00.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.866+0000 7f4b09600700 1 -- 172.21.15.90:0/2101551023 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18019030 con 0x5601a771def0 2024-05-27T08:02:00.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.866+0000 7f4b09600700 1 -- 172.21.15.90:0/2101551023 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f4b1801a070 con 0x5601a771def0 2024-05-27T08:02:00.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.869+0000 7f4b3491db80 1 -- 172.21.15.90:0/2101551023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b10003cb0 con 0x5601a771def0 2024-05-27T08:02:00.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.870+0000 7f4b09600700 1 -- 172.21.15.90:0/2101551023 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4b18019030 con 0x5601a771def0 2024-05-27T08:02:00.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.870+0000 7f4b3491db80 10 client.18009 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:00.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.871+0000 7f4b3491db80 1 -- 172.21.15.90:0/2101551023 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5601a760c0e0 con 0x5601a771def0 2024-05-27T08:02:00.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.873+0000 7f4b09600700 1 -- 172.21.15.90:0/2101551023 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b180092e0 con 0x5601a771def0 2024-05-27T08:02:00.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.873+0000 7f4b09600700 1 -- 172.21.15.90:0/2101551023 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f4b18004020 con 0x5601a771def0 2024-05-27T08:02:00.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.873+0000 7f4b3491db80 10 client.18009 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:00.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.873+0000 7f4b3491db80 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:00.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.874+0000 7f4b3491db80 20 client.18009 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:00.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.874+0000 7f4b3491db80 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5601a7748c70 0x5601a774b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.874+0000 7f4b3491db80 4 client.18009 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:00.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.874+0000 7f4b3491db80 1 -- 172.21.15.90:0/2101551023 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4b10003cb0 con 0x5601a7748c70 2024-05-27T08:02:00.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.874+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5601a7748c70 0x5601a774b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.877+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5601a7748c70 0x5601a774b0c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6900dd5f4b894e69 server_cookie=17b6ef0cf7351abc in_seq=0 out_seq=0 2024-05-27T08:02:00.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.877+0000 7f4b09600700 10 client.18009 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:00.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.879+0000 7f4b09600700 1 -- 172.21.15.90:0/2101551023 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f4b10003cb0 con 0x5601a7748c70 2024-05-27T08:02:00.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.880+0000 7f4b09600700 10 client.18009 handle_command_reply: tid=0 2024-05-27T08:02:00.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.880+0000 7f4b3491db80 1 client.18009 shutdown 2024-05-27T08:02:00.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.881+0000 7f4b3491db80 1 -- 172.21.15.90:0/2101551023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 msgr2=0x5601a7728120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:00.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.881+0000 7f4b3491db80 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a7728120 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4b18011db0 tx=0x7f4b1800f490 comp rx=0 tx=0).stop 2024-05-27T08:02:00.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.881+0000 7f4b3491db80 1 -- 172.21.15.90:0/2101551023 shutdown_connections 2024-05-27T08:02:00.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.881+0000 7f4b3491db80 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5601a7748c70 0x5601a774b0c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.881+0000 7f4b3491db80 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a7728120 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.881+0000 7f4b3491db80 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a759eec0 0x5601a77276a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.881+0000 7f4b3491db80 1 --2- 172.21.15.90:0/2101551023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a768aa70 0x5601a7727be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.881+0000 7f4b3491db80 1 -- 172.21.15.90:0/2101551023 >> 172.21.15.90:0/2101551023 conn(0x5601a7638410 msgr2=0x5601a771d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:00.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.882+0000 7f4b09600700 0 client.18009 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:00.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.882+0000 7f4b3491db80 1 -- 172.21.15.90:0/2101551023 shutdown_connections 2024-05-27T08:02:00.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.882+0000 7f4b3491db80 1 -- 172.21.15.90:0/2101551023 wait complete. 2024-05-27T08:02:00.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.882+0000 7f4b3491db80 20 client.18009 trim_cache size 0 max 16384 2024-05-27T08:02:00.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.923+0000 7f4b3491db80 1 Processor -- start 2024-05-27T08:02:00.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.923+0000 7f4b3491db80 1 -- start start 2024-05-27T08:02:00.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.924+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a7722e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.924+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a7729780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.924+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a77210b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.924+0000 7f4b3491db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4b201b0660 con 0x5601a771def0 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.924+0000 7f4b3491db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b201a33e0 con 0x5601a759eec0 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.924+0000 7f4b3491db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b201af8c0 con 0x5601a768aa70 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.925+0000 7f4b1e000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a7729780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.925+0000 7f4b1e000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a7729780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:40904/0 (socket says 172.21.15.90:40904) 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.925+0000 7f4b1e000700 1 -- 172.21.15.90:0/712977547 learned_addr learned my addr 172.21.15.90:0/712977547 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.925+0000 7f4b1f400700 1 --2- 172.21.15.90:0/712977547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a77210b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.926+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/712977547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a7722e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.926+0000 7f4b1e000700 1 -- 172.21.15.90:0/712977547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 msgr2=0x5601a7722e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.926+0000 7f4b1e000700 1 --2- 172.21.15.90:0/712977547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a7722e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.926+0000 7f4b1e000700 1 -- 172.21.15.90:0/712977547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 msgr2=0x5601a77210b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:00.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.926+0000 7f4b1e000700 1 --2- 172.21.15.90:0/712977547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a77210b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.926+0000 7f4b1e000700 1 -- 172.21.15.90:0/712977547 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a771dc60 con 0x5601a768aa70 2024-05-27T08:02:00.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.926+0000 7f4b1f400700 1 --2- 172.21.15.90:0/712977547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a77210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:00.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.927+0000 7f4b1e000700 1 --2- 172.21.15.90:0/712977547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a7729780 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0df210 tx=0x7f4b0c0cb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ae93dcb824a61b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:00.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.927+0000 7f4b0be00700 1 -- 172.21.15.90:0/712977547 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c021040 con 0x5601a768aa70 2024-05-27T08:02:00.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.928+0000 7f4b0be00700 1 -- 172.21.15.90:0/712977547 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b0c0b1b50 con 0x5601a768aa70 2024-05-27T08:02:00.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.928+0000 7f4b0be00700 1 -- 172.21.15.90:0/712977547 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0b1920 con 0x5601a768aa70 2024-05-27T08:02:00.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.930+0000 7f4b3491db80 1 -- 172.21.15.90:0/712977547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 msgr2=0x5601a7729780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:00.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.930+0000 7f4b3491db80 1 --2- 172.21.15.90:0/712977547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a7729780 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0df210 tx=0x7f4b0c0cb040 comp rx=0 tx=0).stop 2024-05-27T08:02:00.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.930+0000 7f4b3491db80 1 -- 172.21.15.90:0/712977547 shutdown_connections 2024-05-27T08:02:00.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.930+0000 7f4b3491db80 1 --2- 172.21.15.90:0/712977547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a771def0 0x5601a77210b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.930+0000 7f4b3491db80 1 --2- 172.21.15.90:0/712977547 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a7722e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.930+0000 7f4b3491db80 1 --2- 172.21.15.90:0/712977547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a768aa70 0x5601a7729780 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.930+0000 7f4b3491db80 1 -- 172.21.15.90:0/712977547 >> 172.21.15.90:0/712977547 conn(0x5601a7638410 msgr2=0x5601a7726600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:00.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.930+0000 7f4b3491db80 1 -- 172.21.15.90:0/712977547 shutdown_connections 2024-05-27T08:02:00.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.930+0000 7f4b3491db80 1 -- 172.21.15.90:0/712977547 wait complete. 2024-05-27T08:02:00.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.931+0000 7f4b3491db80 1 -- start start 2024-05-27T08:02:00.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.931+0000 7f4b3491db80 1 Processor -- start 2024-05-27T08:02:00.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.932+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a7730490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.932+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a768aa70 0x5601a772adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.932+0000 7f4b3491db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a771def0 0x5601a772b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.932+0000 7f4b3491db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4b201af1f0 con 0x5601a768aa70 2024-05-27T08:02:00.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.932+0000 7f4b3491db80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b201a3ab0 con 0x5601a759eec0 2024-05-27T08:02:00.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.932+0000 7f4b3491db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b201ae590 con 0x5601a771def0 2024-05-27T08:02:00.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.932+0000 7f4b3491db80 10 client.0 _ll_register_callbacks cb 0x5601a75e5cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:00.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.933+0000 7f4b1ea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a7730490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.933+0000 7f4b1ea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a7730490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39756/0 (socket says 172.21.15.90:39756) 2024-05-27T08:02:00.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.933+0000 7f4b1ea00700 1 -- 172.21.15.90:0/3404035424 learned_addr learned my addr 172.21.15.90:0/3404035424 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:00.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.933+0000 7f4b1e000700 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a768aa70 0x5601a772adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.934+0000 7f4b1f400700 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a771def0 0x5601a772b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.934+0000 7f4b1ea00700 1 -- 172.21.15.90:0/3404035424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a771def0 msgr2=0x5601a772b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.934+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a771def0 0x5601a772b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.934+0000 7f4b1ea00700 1 -- 172.21.15.90:0/3404035424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a768aa70 msgr2=0x5601a772adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.934+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a768aa70 0x5601a772adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.934+0000 7f4b1ea00700 1 -- 172.21.15.90:0/3404035424 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a772ba60 con 0x5601a759eec0 2024-05-27T08:02:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.935+0000 7f4b1ea00700 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a7730490 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5601a7720070 tx=0x7f4b10005400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f678cb160edac550 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.935+0000 7f4b09600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:02:00.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.936+0000 7f4b09600700 1 -- 172.21.15.90:0/3404035424 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10025b90 con 0x5601a759eec0 2024-05-27T08:02:00.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.936+0000 7f4b09600700 1 -- 172.21.15.90:0/3404035424 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f4b10025d30 con 0x5601a759eec0 2024-05-27T08:02:00.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.936+0000 7f4b3491db80 1 -- 172.21.15.90:0/3404035424 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b0c0b1b50 con 0x5601a759eec0 2024-05-27T08:02:00.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.937+0000 7f4b09600700 1 -- 172.21.15.90:0/3404035424 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4b1004c050 con 0x5601a759eec0 2024-05-27T08:02:00.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.937+0000 7f4b3491db80 10 client.37685 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:00.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.937+0000 7f4b3491db80 1 -- 172.21.15.90:0/3404035424 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5601a77b4670 con 0x5601a759eec0 2024-05-27T08:02:00.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.938+0000 7f4b09600700 1 -- 172.21.15.90:0/3404035424 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4b0c0b1b50 con 0x5601a759eec0 2024-05-27T08:02:00.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.938+0000 7f4b09600700 1 -- 172.21.15.90:0/3404035424 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f4b1002e020 con 0x5601a759eec0 2024-05-27T08:02:00.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.939+0000 7f4b3491db80 10 client.37685 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:00.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.939+0000 7f4b3491db80 10 client.37685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:00.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.939+0000 7f4b3491db80 20 client.37685 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:00.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.939+0000 7f4b3491db80 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5601a77b4b60 0x5601a77b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:00.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.939+0000 7f4b3491db80 4 client.37685 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:02:00.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.939+0000 7f4b3491db80 1 -- 172.21.15.90:0/3404035424 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5601a764de10 con 0x5601a77b4b60 2024-05-27T08:02:00.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.940+0000 7f4b1e000700 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5601a77b4b60 0x5601a77b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:00.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.941+0000 7f4b1e000700 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5601a77b4b60 0x5601a77b6f80 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a28d14db2e93019 server_cookie=73fff05dfffd5467 in_seq=0 out_seq=0 2024-05-27T08:02:00.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.941+0000 7f4b09600700 10 client.37685 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:00.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.945+0000 7f4b09600700 1 -- 172.21.15.90:0/3404035424 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5601a764de10 con 0x5601a77b4b60 2024-05-27T08:02:00.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.945+0000 7f4b09600700 10 client.37685 handle_command_reply: tid=0 2024-05-27T08:02:00.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.945+0000 7f4b3491db80 1 client.37685 shutdown 2024-05-27T08:02:00.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.945+0000 7f4b3491db80 1 -- 172.21.15.90:0/3404035424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 msgr2=0x5601a7730490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:00.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.945+0000 7f4b3491db80 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a7730490 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5601a7720070 tx=0x7f4b10005400 comp rx=0 tx=0).stop 2024-05-27T08:02:00.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.945+0000 7f4b3491db80 1 -- 172.21.15.90:0/3404035424 shutdown_connections 2024-05-27T08:02:00.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.945+0000 7f4b3491db80 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5601a77b4b60 0x5601a77b6f80 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.945+0000 7f4b3491db80 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a771def0 0x5601a772b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.945+0000 7f4b3491db80 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5601a759eec0 0x5601a7730490 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.945+0000 7f4b3491db80 1 --2- 172.21.15.90:0/3404035424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a768aa70 0x5601a772adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.946+0000 7f4b3491db80 1 -- 172.21.15.90:0/3404035424 >> 172.21.15.90:0/3404035424 conn(0x5601a7638410 msgr2=0x5601a7728c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:00.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.946+0000 7f4b09600700 0 client.37685 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:00.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.946+0000 7f4b3491db80 1 -- 172.21.15.90:0/3404035424 shutdown_connections 2024-05-27T08:02:00.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.946+0000 7f4b3491db80 1 -- 172.21.15.90:0/3404035424 wait complete. 2024-05-27T08:02:00.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.946+0000 7f4b3491db80 20 client.37685 trim_cache size 0 max 16384 2024-05-27T08:02:00.950 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:02:00.950 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T08:02:00.950 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "5a02219f-4692-4335-88c9-b6c9c8ac8f85", 2024-05-27T08:02:00.950 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T08:02:00.950 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:02:00.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 -- 172.21.15.90:0/3613425461 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4b040b2b40 msgr2=0x7f4b040b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:00.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4b040b2b40 0x7f4b040b5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4b10000c00 tx=0x7f4b10005fb0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 -- 172.21.15.90:0/3613425461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20106ee0 msgr2=0x7f4b20110e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:00.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20106ee0 0x7f4b20110e40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00d950 tx=0x7f4b0c00dc60 comp rx=0 tx=0).stop 2024-05-27T08:02:00.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 -- 172.21.15.90:0/3613425461 shutdown_connections 2024-05-27T08:02:00.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b201113d0 0x7f4b201ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f4b040b2b40 0x7f4b040b5050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b20106590 0x7f4b20110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 --2- 172.21.15.90:0/3613425461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20106ee0 0x7f4b20110e40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:00.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 -- 172.21.15.90:0/3613425461 >> 172.21.15.90:0/3613425461 conn(0x7f4b20102070 msgr2=0x7f4b201039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:00.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.948+0000 7f4b25e00700 1 -- 172.21.15.90:0/3613425461 shutdown_connections 2024-05-27T08:02:00.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:00.949+0000 7f4b25e00700 1 -- 172.21.15.90:0/3613425461 wait complete. 2024-05-27T08:02:01.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:00 smithi136 ceph-mon[35324]: pgmap v500: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 7.5 MiB/s wr, 681 op/s 2024-05-27T08:02:01.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:01 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T08:02:01.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:01 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:01.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:01 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T08:02:02.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:01 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T08:02:02.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:01 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:02.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:01 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T08:02:02.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:01 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T08:02:02.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:01 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:02.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:01 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T08:02:02.383 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:02:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.385+0000 7f87f1000640 1 -- 172.21.15.90:0/2296247293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ec110510 msgr2=0x7f87ec10a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.385+0000 7f87f1000640 1 --2- 172.21.15.90:0/2296247293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ec110510 0x7f87ec10a110 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009900 tx=0x7f87dc02a3f0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.385+0000 7f87f1000640 1 -- 172.21.15.90:0/2296247293 shutdown_connections 2024-05-27T08:02:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.385+0000 7f87f1000640 1 --2- 172.21.15.90:0/2296247293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87ec114e10 0x7f87ec117230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.385+0000 7f87f1000640 1 --2- 172.21.15.90:0/2296247293 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87ec10a650 0x7f87ec10aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.385+0000 7f87f1000640 1 --2- 172.21.15.90:0/2296247293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ec110510 0x7f87ec10a110 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.385+0000 7f87f1000640 1 -- 172.21.15.90:0/2296247293 >> 172.21.15.90:0/2296247293 conn(0x7f87ec06f800 msgr2=0x7f87ec06fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.387+0000 7f87f1000640 1 -- 172.21.15.90:0/2296247293 shutdown_connections 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.387+0000 7f87f1000640 1 -- 172.21.15.90:0/2296247293 wait complete. 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.389+0000 7f87f1000640 1 Processor -- start 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.389+0000 7f87f1000640 1 -- start start 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.389+0000 7f87f1000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ec10a650 0x7f87ec10f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.389+0000 7f87f1000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87ec110510 0x7f87ec10d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.389+0000 7f87f1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87ec114e10 0x7f87ec10db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.389+0000 7f87f1000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec118a30 con 0x7f87ec10a650 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.389+0000 7f87f1000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1188b0 con 0x7f87ec110510 2024-05-27T08:02:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.389+0000 7f87f1000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec118bb0 con 0x7f87ec114e10 2024-05-27T08:02:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.391+0000 7f87eb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ec10a650 0x7f87ec10f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.391+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87ec114e10 0x7f87ec10db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.391+0000 7f87eaa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87ec110510 0x7f87ec10d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.391+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87ec114e10 0x7f87ec10db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:40938/0 (socket says 172.21.15.90:40938) 2024-05-27T08:02:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.391+0000 7f87ebe00640 1 -- 172.21.15.90:0/762527052 learned_addr learned my addr 172.21.15.90:0/762527052 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.392+0000 7f87ebe00640 1 -- 172.21.15.90:0/762527052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87ec110510 msgr2=0x7f87ec10d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.392+0000 7f87ebe00640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87ec110510 0x7f87ec10d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.392+0000 7f87ebe00640 1 -- 172.21.15.90:0/762527052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ec10a650 msgr2=0x7f87ec10f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.392+0000 7f87ebe00640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ec10a650 0x7f87ec10f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.392+0000 7f87ebe00640 1 -- 172.21.15.90:0/762527052 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ec10e3b0 con 0x7f87ec114e10 2024-05-27T08:02:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.392+0000 7f87ebe00640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87ec114e10 0x7f87ec10db90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f87e400efd0 tx=0x7f87e400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.393+0000 7f87e3e00640 1 -- 172.21.15.90:0/762527052 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4003b70 con 0x7f87ec114e10 2024-05-27T08:02:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.393+0000 7f87e3e00640 1 -- 172.21.15.90:0/762527052 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87e4003d10 con 0x7f87ec114e10 2024-05-27T08:02:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.393+0000 7f87e3e00640 1 -- 172.21.15.90:0/762527052 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4010850 con 0x7f87ec114e10 2024-05-27T08:02:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.395+0000 7f87f1000640 1 -- 172.21.15.90:0/762527052 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ec1b57f0 con 0x7f87ec114e10 2024-05-27T08:02:02.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.395+0000 7f87f1000640 1 -- 172.21.15.90:0/762527052 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ec1b5cf0 con 0x7f87ec114e10 2024-05-27T08:02:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.402+0000 7f87f95ef740 1 Processor -- start 2024-05-27T08:02:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.405+0000 7f87e3e00640 1 -- 172.21.15.90:0/762527052 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f87e40109f0 con 0x7f87ec114e10 2024-05-27T08:02:02.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.405+0000 7f87f95ef740 1 -- start start 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.406+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095441b0 0x5610098e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.406+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x5610098e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.406+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095234d0 0x5610098e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.406+0000 7f87f95ef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561009857b90 con 0x5610095234d0 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.406+0000 7f87f95ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561009847730 con 0x56100951b2f0 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.406+0000 7f87f95ef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56100984b770 con 0x5610095441b0 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.407+0000 7f87eaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095234d0 0x5610098e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.407+0000 7f87eaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095234d0 0x5610098e74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59748/0 (socket says 172.21.15.90:59748) 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.407+0000 7f87eaa00640 1 -- 172.21.15.90:0/3728358155 learned_addr learned my addr 172.21.15.90:0/3728358155 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.408+0000 7f87eb400640 1 --2- 172.21.15.90:0/3728358155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x5610098e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.408+0000 7f87eaa00640 1 --2- 172.21.15.90:0/3728358155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095441b0 0x5610098e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.408+0000 7f87eaa00640 1 -- 172.21.15.90:0/3728358155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095441b0 msgr2=0x5610098e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.408+0000 7f87eaa00640 1 --2- 172.21.15.90:0/3728358155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095441b0 0x5610098e6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.408+0000 7f87eaa00640 1 -- 172.21.15.90:0/3728358155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 msgr2=0x5610098e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.408+0000 7f87eaa00640 1 --2- 172.21.15.90:0/3728358155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x5610098e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.408+0000 7f87eaa00640 1 -- 172.21.15.90:0/3728358155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610097e08b0 con 0x5610095234d0 2024-05-27T08:02:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.409+0000 7f87e3e00640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87d00b2a20 0x7f87d00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.410+0000 7f87e3e00640 1 -- 172.21.15.90:0/762527052 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f87e40e2dc0 con 0x7f87ec114e10 2024-05-27T08:02:02.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.410+0000 7f87eb400640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87d00b2a20 0x7f87d00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.410+0000 7f87eaa00640 1 --2- 172.21.15.90:0/3728358155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095234d0 0x5610098e74e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f87d80093e0 tx=0x7f87d8009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3140e21873cc58 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.413+0000 7f87e0c00640 1 -- 172.21.15.90:0/3728358155 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8004050 con 0x5610095234d0 2024-05-27T08:02:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.413+0000 7f87e0c00640 1 -- 172.21.15.90:0/3728358155 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87d800f810 con 0x5610095234d0 2024-05-27T08:02:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.413+0000 7f87e0c00640 1 -- 172.21.15.90:0/3728358155 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8003850 con 0x5610095234d0 2024-05-27T08:02:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.414+0000 7f87f95ef740 1 -- 172.21.15.90:0/3728358155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095234d0 msgr2=0x5610098e74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.414+0000 7f87eb400640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87d00b2a20 0x7f87d00b4f30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f87dc02a960 tx=0x7f87dc02a8f0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.414+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3728358155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095234d0 0x5610098e74e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f87d80093e0 tx=0x7f87d8009370 comp rx=0 tx=0).stop 2024-05-27T08:02:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.417+0000 7f87f95ef740 1 -- 172.21.15.90:0/3728358155 shutdown_connections 2024-05-27T08:02:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.417+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3728358155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x5610098e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.417+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3728358155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095234d0 0x5610098e74e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.417+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3728358155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095441b0 0x5610098e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.417+0000 7f87f95ef740 1 -- 172.21.15.90:0/3728358155 >> 172.21.15.90:0/3728358155 conn(0x561009538300 msgr2=0x5610098e3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.418+0000 7f87f95ef740 1 -- 172.21.15.90:0/3728358155 shutdown_connections 2024-05-27T08:02:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.420+0000 7f87f95ef740 1 -- 172.21.15.90:0/3728358155 wait complete. 2024-05-27T08:02:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.420+0000 7f87f95ef740 1 -- start start 2024-05-27T08:02:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.421+0000 7f87f95ef740 1 Processor -- start 2024-05-27T08:02:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.422+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x5610098f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.422+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x5610098f1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.422+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095441b0 0x5610098fcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.422+0000 7f87f95ef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56100952d790 con 0x5610095441b0 2024-05-27T08:02:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.422+0000 7f87f95ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5610097efbc0 con 0x56100951b2f0 2024-05-27T08:02:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.422+0000 7f87f95ef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5610098f3b90 con 0x5610095234d0 2024-05-27T08:02:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.422+0000 7f87f95ef740 10 client.0 _ll_register_callbacks cb 0x5610097cd400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.426+0000 7f87eb400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x5610098f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.426+0000 7f87eaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x5610098f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.426+0000 7f87eaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x5610098f1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:40962/0 (socket says 172.21.15.90:40962) 2024-05-27T08:02:02.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.426+0000 7f87eaa00640 1 -- 172.21.15.90:0/3555365542 learned_addr learned my addr 172.21.15.90:0/3555365542 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.427+0000 7f87eb400640 1 -- 172.21.15.90:0/3555365542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 msgr2=0x5610098f1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.427+0000 7f87eb400640 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x5610098f1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.427+0000 7f87eb400640 1 -- 172.21.15.90:0/3555365542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095441b0 msgr2=0x5610098fcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.427+0000 7f87eb400640 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095441b0 0x5610098fcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.427+0000 7f87eb400640 1 -- 172.21.15.90:0/3555365542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56100955c210 con 0x56100951b2f0 2024-05-27T08:02:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.428+0000 7f87eb400640 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x5610098f0f20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009850 tx=0x7f87dc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb0817b6547c9f01 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.428+0000 7f87d5600640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:02:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.429+0000 7f87d5600640 1 -- 172.21.15.90:0/3555365542 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc035040 con 0x56100951b2f0 2024-05-27T08:02:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.430+0000 7f87d5600640 1 -- 172.21.15.90:0/3555365542 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f87dc004d10 con 0x56100951b2f0 2024-05-27T08:02:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.430+0000 7f87f95ef740 1 -- 172.21.15.90:0/3555365542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87d800f810 con 0x56100951b2f0 2024-05-27T08:02:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.430+0000 7f87d5600640 1 -- 172.21.15.90:0/3555365542 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f87dc0035f0 con 0x56100951b2f0 2024-05-27T08:02:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.431+0000 7f87f95ef740 10 client.37697 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.431+0000 7f87f95ef740 1 -- 172.21.15.90:0/3555365542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561009628620 con 0x56100951b2f0 2024-05-27T08:02:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.431+0000 7f87d5600640 1 -- 172.21.15.90:0/3555365542 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f87d800f810 con 0x56100951b2f0 2024-05-27T08:02:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.432+0000 7f87d5600640 1 -- 172.21.15.90:0/3555365542 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f87dc044030 con 0x56100951b2f0 2024-05-27T08:02:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.432+0000 7f87f95ef740 10 client.37697 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.432+0000 7f87f95ef740 10 client.37697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.432+0000 7f87f95ef740 20 client.37697 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.432+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561009502d80 0x561009903130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.432+0000 7f87f95ef740 4 client.37697 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:02.435 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.432+0000 7f87f95ef740 1 -- 172.21.15.90:0/3555365542 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561009859ae0 con 0x561009502d80 2024-05-27T08:02:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.435+0000 7f87eaa00640 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561009502d80 0x561009903130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.438+0000 7f87eaa00640 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561009502d80 0x561009903130 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a0a78b0e2c9d287 server_cookie=7f129278bc540c91 in_seq=0 out_seq=0 2024-05-27T08:02:02.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.439+0000 7f87d5600640 10 client.37697 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.440+0000 7f87d5600640 1 -- 172.21.15.90:0/3555365542 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561009859ae0 con 0x561009502d80 2024-05-27T08:02:02.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.440+0000 7f87d5600640 10 client.37697 handle_command_reply: tid=0 2024-05-27T08:02:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.440+0000 7f87f95ef740 1 client.37697 shutdown 2024-05-27T08:02:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.441+0000 7f87f95ef740 1 -- 172.21.15.90:0/3555365542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 msgr2=0x5610098f0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:02.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.441+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x5610098f0f20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f87dc009850 tx=0x7f87dc009260 comp rx=0 tx=0).stop 2024-05-27T08:02:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.444+0000 7f87f95ef740 1 -- 172.21.15.90:0/3555365542 shutdown_connections 2024-05-27T08:02:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.444+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561009502d80 0x561009903130 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.444+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095441b0 0x5610098fcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.444+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x5610098f1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.444+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3555365542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x5610098f0f20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.444+0000 7f87f95ef740 1 -- 172.21.15.90:0/3555365542 >> 172.21.15.90:0/3555365542 conn(0x561009538300 msgr2=0x5610098e8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.444+0000 7f87d5600640 0 client.37697 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.445+0000 7f87f95ef740 1 -- 172.21.15.90:0/3555365542 shutdown_connections 2024-05-27T08:02:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.446+0000 7f87f95ef740 1 -- 172.21.15.90:0/3555365542 wait complete. 2024-05-27T08:02:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.447+0000 7f87f95ef740 20 client.37697 trim_cache size 0 max 16384 2024-05-27T08:02:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.479+0000 7f87f95ef740 1 Processor -- start 2024-05-27T08:02:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.480+0000 7f87f95ef740 1 -- start start 2024-05-27T08:02:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.481+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56100951b2f0 0x5610098ec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.481+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x5610098e7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.481+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610095441b0 0x5610098fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.481+0000 7f87f95ef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1c1880 con 0x56100951b2f0 2024-05-27T08:02:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.481+0000 7f87f95ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1be8d0 con 0x5610095441b0 2024-05-27T08:02:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.481+0000 7f87f95ef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1c0ae0 con 0x5610095234d0 2024-05-27T08:02:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.483+0000 7f87eb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56100951b2f0 0x5610098ec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.483+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610095441b0 0x5610098fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.483+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610095441b0 0x5610098fcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39790/0 (socket says 172.21.15.90:39790) 2024-05-27T08:02:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.483+0000 7f87ebe00640 1 -- 172.21.15.90:0/1740001073 learned_addr learned my addr 172.21.15.90:0/1740001073 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.483+0000 7f87eaa00640 1 --2- 172.21.15.90:0/1740001073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x5610098e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.484+0000 7f87eaa00640 1 -- 172.21.15.90:0/1740001073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610095441b0 msgr2=0x5610098fcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.484+0000 7f87eaa00640 1 --2- 172.21.15.90:0/1740001073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610095441b0 0x5610098fcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.487+0000 7f87eaa00640 1 -- 172.21.15.90:0/1740001073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56100951b2f0 msgr2=0x5610098ec4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.487+0000 7f87eaa00640 1 --2- 172.21.15.90:0/1740001073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56100951b2f0 0x5610098ec4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.487+0000 7f87eaa00640 1 -- 172.21.15.90:0/1740001073 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610098e67f0 con 0x5610095234d0 2024-05-27T08:02:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.487+0000 7f87eaa00640 1 --2- 172.21.15.90:0/1740001073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x5610098e7410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009960 tx=0x7f87d8009c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d221127faa4f7f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.488+0000 7f87d7e00640 1 -- 172.21.15.90:0/1740001073 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d800fce0 con 0x5610095234d0 2024-05-27T08:02:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.488+0000 7f87d7e00640 1 -- 172.21.15.90:0/1740001073 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561009859ae0 con 0x5610095234d0 2024-05-27T08:02:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.488+0000 7f87d7e00640 1 -- 172.21.15.90:0/1740001073 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8004a80 con 0x5610095234d0 2024-05-27T08:02:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.489+0000 7f87eb400640 1 --2- 172.21.15.90:0/1740001073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56100951b2f0 0x5610098ec4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.491+0000 7f87f95ef740 1 -- 172.21.15.90:0/1740001073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 msgr2=0x5610098e7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.491+0000 7f87f95ef740 1 --2- 172.21.15.90:0/1740001073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x5610098e7410 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009960 tx=0x7f87d8009c70 comp rx=0 tx=0).stop 2024-05-27T08:02:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.495+0000 7f87f95ef740 1 -- 172.21.15.90:0/1740001073 shutdown_connections 2024-05-27T08:02:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.495+0000 7f87f95ef740 1 --2- 172.21.15.90:0/1740001073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610095441b0 0x5610098fcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.495+0000 7f87f95ef740 1 --2- 172.21.15.90:0/1740001073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x5610098e7410 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.495+0000 7f87f95ef740 1 --2- 172.21.15.90:0/1740001073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56100951b2f0 0x5610098ec4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.495+0000 7f87f95ef740 1 -- 172.21.15.90:0/1740001073 >> 172.21.15.90:0/1740001073 conn(0x561009538300 msgr2=0x5610098e1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.495+0000 7f87f95ef740 1 -- 172.21.15.90:0/1740001073 shutdown_connections 2024-05-27T08:02:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.496+0000 7f87f95ef740 1 -- 172.21.15.90:0/1740001073 wait complete. 2024-05-27T08:02:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.496+0000 7f87f95ef740 1 -- start start 2024-05-27T08:02:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.497+0000 7f87f95ef740 1 Processor -- start 2024-05-27T08:02:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.499+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x561009968800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.499+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x561009968d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.499+0000 7f87f95ef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095441b0 0x561009969280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.500+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095441b0 0x561009969280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.500+0000 7f87eaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x561009968d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.500+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095441b0 0x561009969280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59772/0 (socket says 172.21.15.90:59772) 2024-05-27T08:02:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.501+0000 7f87ebe00640 1 -- 172.21.15.90:0/3973352847 learned_addr learned my addr 172.21.15.90:0/3973352847 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.501+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1c0410 con 0x5610095441b0 2024-05-27T08:02:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.501+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1befa0 con 0x56100951b2f0 2024-05-27T08:02:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.501+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1bf670 con 0x5610095234d0 2024-05-27T08:02:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.501+0000 7f87f95ef740 10 client.0 _ll_register_callbacks cb 0x7f87dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.501+0000 7f87eaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x561009968d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:40986/0 (socket says 172.21.15.90:40986) 2024-05-27T08:02:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.504+0000 7f87eb400640 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x561009968800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.504+0000 7f87eaa00640 1 -- 172.21.15.90:0/3973352847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 msgr2=0x561009968800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.504+0000 7f87eaa00640 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x561009968800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.504+0000 7f87eaa00640 1 -- 172.21.15.90:0/3973352847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095441b0 msgr2=0x561009969280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.505+0000 7f87eaa00640 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095441b0 0x561009969280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.505+0000 7f87eaa00640 1 -- 172.21.15.90:0/3973352847 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610098f8e10 con 0x5610095234d0 2024-05-27T08:02:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.505+0000 7f87eb400640 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x561009968800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.506+0000 7f87eaa00640 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x561009968d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5610098e1660 tx=0x7f87d80094e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604459934d99c87b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.506+0000 7f87d5600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:02:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.506+0000 7f87d5600640 1 -- 172.21.15.90:0/3973352847 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8039070 con 0x5610095234d0 2024-05-27T08:02:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.507+0000 7f87d5600640 1 -- 172.21.15.90:0/3973352847 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9496+0+0 (secure 0 0 0) 0x7f87d8011220 con 0x5610095234d0 2024-05-27T08:02:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.507+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561009859ae0 con 0x5610095234d0 2024-05-27T08:02:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.508+0000 7f87d5600640 1 -- 172.21.15.90:0/3973352847 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f87d8044050 con 0x5610095234d0 2024-05-27T08:02:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.508+0000 7f87f95ef740 10 client.27793 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.508+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5610098eabd0 con 0x5610095234d0 2024-05-27T08:02:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.508+0000 7f87d5600640 1 -- 172.21.15.90:0/3973352847 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561009859ae0 con 0x5610095234d0 2024-05-27T08:02:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.512+0000 7f87d5600640 1 -- 172.21.15.90:0/3973352847 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f87d802a430 con 0x5610095234d0 2024-05-27T08:02:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.512+0000 7f87f95ef740 10 client.27793 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.512+0000 7f87f95ef740 10 client.27793 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.512+0000 7f87f95ef740 20 client.27793 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.512+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5610094d81c0 0x5610098eb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.512+0000 7f87f95ef740 4 client.27793 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:02:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.512+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610098532c0 con 0x5610094d81c0 2024-05-27T08:02:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.513+0000 7f87eb400640 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5610094d81c0 0x5610098eb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.514+0000 7f87eb400640 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5610094d81c0 0x5610098eb2b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa400174f74206a5 server_cookie=5775eab01f2f472f in_seq=0 out_seq=0 2024-05-27T08:02:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.514+0000 7f87d5600640 10 client.27793 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.904+0000 7f87d5600640 1 -- 172.21.15.90:0/3973352847 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5610098532c0 con 0x5610094d81c0 2024-05-27T08:02:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.904+0000 7f87d5600640 10 client.27793 handle_command_reply: tid=0 2024-05-27T08:02:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.904+0000 7f87f95ef740 1 client.27793 shutdown 2024-05-27T08:02:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.904+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 msgr2=0x561009968d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.905+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x561009968d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x5610098e1660 tx=0x7f87d80094e0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.905+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 shutdown_connections 2024-05-27T08:02:02.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.905+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5610094d81c0 0x5610098eb2b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.905+0000 7f87d5600640 0 client.27793 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.905+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610095441b0 0x561009969280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.905+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610095234d0 0x561009968d40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.905+0000 7f87f95ef740 1 --2- 172.21.15.90:0/3973352847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56100951b2f0 0x561009968800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.905+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 >> 172.21.15.90:0/3973352847 conn(0x561009538300 msgr2=0x5610098e2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.905+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 shutdown_connections 2024-05-27T08:02:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.906+0000 7f87f95ef740 1 -- 172.21.15.90:0/3973352847 wait complete. 2024-05-27T08:02:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.906+0000 7f87f95ef740 20 client.27793 trim_cache size 0 max 16384 2024-05-27T08:02:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:02:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:02:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:02:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.910+0000 7f87f1000640 1 -- 172.21.15.90:0/762527052 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87d00b2a20 msgr2=0x7f87d00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.910+0000 7f87f1000640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87d00b2a20 0x7f87d00b4f30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f87dc02a960 tx=0x7f87dc02a8f0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.910+0000 7f87f1000640 1 -- 172.21.15.90:0/762527052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87ec114e10 msgr2=0x7f87ec10db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.910+0000 7f87f1000640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87ec114e10 0x7f87ec10db90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f87e400efd0 tx=0x7f87e400c6a0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.911+0000 7f87f1000640 1 -- 172.21.15.90:0/762527052 shutdown_connections 2024-05-27T08:02:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.911+0000 7f87f1000640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f87d00b2a20 0x7f87d00b4f30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.911+0000 7f87f1000640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87ec110510 0x7f87ec10d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.911+0000 7f87f1000640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87ec114e10 0x7f87ec10db90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.911+0000 7f87f1000640 1 --2- 172.21.15.90:0/762527052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ec10a650 0x7f87ec10f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.911+0000 7f87f1000640 1 -- 172.21.15.90:0/762527052 >> 172.21.15.90:0/762527052 conn(0x7f87ec06f800 msgr2=0x7f87ec1191d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.911+0000 7f87f1000640 1 -- 172.21.15.90:0/762527052 shutdown_connections 2024-05-27T08:02:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:02.911+0000 7f87f1000640 1 -- 172.21.15.90:0/762527052 wait complete. 2024-05-27T08:02:02.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:02 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:02.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:02 smithi090 ceph-mon[31429]: pgmap v501: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.5 MiB/s wr, 502 op/s 2024-05-27T08:02:03.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:02 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:03.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:02 smithi136 ceph-mon[35324]: pgmap v501: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.5 MiB/s wr, 502 op/s 2024-05-27T08:02:03.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:02 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T08:02:03.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:02 smithi129 ceph-mon[35254]: pgmap v501: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.5 MiB/s wr, 502 op/s 2024-05-27T08:02:03.315 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:03.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 705.0 s, 399.4 tps, lat 253.315 ms stddev 300.233 2024-05-27T08:02:04.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:04 smithi090 ceph-mon[31429]: pgmap v502: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.9 MiB/s wr, 573 op/s 2024-05-27T08:02:04.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:04 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T08:02:05.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:04 smithi136 ceph-mon[35324]: pgmap v502: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.9 MiB/s wr, 573 op/s 2024-05-27T08:02:05.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:04 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T08:02:05.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:04 smithi129 ceph-mon[35254]: pgmap v502: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 5.9 MiB/s wr, 573 op/s 2024-05-27T08:02:05.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:04 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T08:02:05.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:05 smithi090 ceph-mon[31429]: osdmap e119: 12 total, 12 up, 12 in 2024-05-27T08:02:06.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:05 smithi136 ceph-mon[35324]: osdmap e119: 12 total, 12 up, 12 in 2024-05-27T08:02:06.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:05 smithi129 ceph-mon[35254]: osdmap e119: 12 total, 12 up, 12 in 2024-05-27T08:02:06.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:06 smithi090 ceph-mon[31429]: pgmap v504: 129 pgs: 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 126 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 6.2 MiB/s wr, 679 op/s 2024-05-27T08:02:07.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:06 smithi136 ceph-mon[35324]: pgmap v504: 129 pgs: 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 126 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 6.2 MiB/s wr, 679 op/s 2024-05-27T08:02:07.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:06 smithi129 ceph-mon[35254]: pgmap v504: 129 pgs: 2 active+clean+snaptrim, 1 active+clean+snaptrim_wait, 126 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 6.2 MiB/s wr, 679 op/s 2024-05-27T08:02:08.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 710.0 s, 121.2 tps, lat 706.719 ms stddev 781.678 2024-05-27T08:02:08.976 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:08 smithi090 ceph-mon[31429]: pgmap v505: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 6.4 MiB/s wr, 714 op/s 2024-05-27T08:02:09.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:08 smithi136 ceph-mon[35324]: pgmap v505: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 6.4 MiB/s wr, 714 op/s 2024-05-27T08:02:09.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:08 smithi129 ceph-mon[35254]: pgmap v505: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 6.4 MiB/s wr, 714 op/s 2024-05-27T08:02:09.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.172+0000 7f7ef6e00700 1 -- 172.21.15.90:0/4006161932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ef010b270 msgr2=0x7f7ef011d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.172+0000 7f7ef6e00700 1 --2- 172.21.15.90:0/4006161932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ef010b270 0x7f7ef011d3e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009990 tx=0x7f7ee0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.172+0000 7f7ef6e00700 1 -- 172.21.15.90:0/4006161932 shutdown_connections 2024-05-27T08:02:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.172+0000 7f7ef6e00700 1 --2- 172.21.15.90:0/4006161932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef011d920 0x7f7ef011fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.173+0000 7f7ef6e00700 1 --2- 172.21.15.90:0/4006161932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ef010a920 0x7f7ef010ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.173+0000 7f7ef6e00700 1 --2- 172.21.15.90:0/4006161932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ef010b270 0x7f7ef011d3e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.173+0000 7f7ef6e00700 1 -- 172.21.15.90:0/4006161932 >> 172.21.15.90:0/4006161932 conn(0x7f7ef006f4c0 msgr2=0x7f7ef006f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:09.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.176+0000 7f7ef6e00700 1 -- 172.21.15.90:0/4006161932 shutdown_connections 2024-05-27T08:02:09.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.176+0000 7f7ef6e00700 1 -- 172.21.15.90:0/4006161932 wait complete. 2024-05-27T08:02:09.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.177+0000 7f7ef6e00700 1 Processor -- start 2024-05-27T08:02:09.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.178+0000 7f7ef6e00700 1 -- start start 2024-05-27T08:02:09.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.179+0000 7f7ef6e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ef010a920 0x7f7ef01af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.179+0000 7f7ef6e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ef010b270 0x7f7ef01afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.179+0000 7f7ef6e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef011d920 0x7f7ef01a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.180+0000 7f7ef6e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0123140 con 0x7f7ef010b270 2024-05-27T08:02:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.180+0000 7f7ef6e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0122fc0 con 0x7f7ef011d920 2024-05-27T08:02:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.180+0000 7f7eefe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ef010a920 0x7f7ef01af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.180+0000 7f7eefe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ef010a920 0x7f7ef01af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37564/0 (socket says 172.21.15.90:37564) 2024-05-27T08:02:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.180+0000 7f7eefe00700 1 -- 172.21.15.90:0/2898016773 learned_addr learned my addr 172.21.15.90:0/2898016773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.180+0000 7f7eef400700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ef010b270 0x7f7ef01afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.180+0000 7f7ef4e00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef011d920 0x7f7ef01a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.181+0000 7f7ef6e00700 1 -- 172.21.15.90:0/2898016773 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01232c0 con 0x7f7ef010a920 2024-05-27T08:02:09.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.182+0000 7f7eef400700 1 -- 172.21.15.90:0/2898016773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ef010a920 msgr2=0x7f7ef01af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:09.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.182+0000 7f7eef400700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ef010a920 0x7f7ef01af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.182+0000 7f7eef400700 1 -- 172.21.15.90:0/2898016773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef011d920 msgr2=0x7f7ef01a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:09.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.182+0000 7f7eef400700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef011d920 0x7f7ef01a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.182+0000 7f7eef400700 1 -- 172.21.15.90:0/2898016773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef01aa010 con 0x7f7ef010b270 2024-05-27T08:02:09.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.182+0000 7f7ef4e00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef011d920 0x7f7ef01a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:09.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.183+0000 7f7eef400700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ef010b270 0x7f7ef01afc80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:09.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.184+0000 7f7eecc00700 1 -- 172.21.15.90:0/2898016773 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0027070 con 0x7f7ef010b270 2024-05-27T08:02:09.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.184+0000 7f7eecc00700 1 -- 172.21.15.90:0/2898016773 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ee0005da0 con 0x7f7ef010b270 2024-05-27T08:02:09.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.184+0000 7f7eecc00700 1 -- 172.21.15.90:0/2898016773 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00173f0 con 0x7f7ef010b270 2024-05-27T08:02:09.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.184+0000 7f7ef6e00700 1 -- 172.21.15.90:0/2898016773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef01aa2a0 con 0x7f7ef010b270 2024-05-27T08:02:09.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.184+0000 7f7ef6e00700 1 -- 172.21.15.90:0/2898016773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01aa780 con 0x7f7ef010b270 2024-05-27T08:02:09.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.190+0000 7f7eecc00700 1 -- 172.21.15.90:0/2898016773 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7ee0009d10 con 0x7f7ef010b270 2024-05-27T08:02:09.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.193+0000 7f7eecc00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ed40b2b40 0x7f7ed40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.193+0000 7f7eecc00700 1 -- 172.21.15.90:0/2898016773 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9584+0+0 (secure 0 0 0) 0x7f7ee00e4350 con 0x7f7ef010b270 2024-05-27T08:02:09.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.194+0000 7f7eefe00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ed40b2b40 0x7f7ed40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.209+0000 7f7eefe00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ed40b2b40 0x7f7ed40b5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ef01aaaa0 tx=0x7f7ed8009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:09.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.211+0000 7f7f059f3b80 1 Processor -- start 2024-05-27T08:02:09.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.212+0000 7f7f059f3b80 1 -- start start 2024-05-27T08:02:09.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.212+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be32ae40 0x5619be4af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.212+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4afd30 0x5619be4a9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.212+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4a9db0 0x5619be4aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.212+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5619be3d7e40 con 0x5619be4a9db0 2024-05-27T08:02:09.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.212+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5619be3c79e0 con 0x5619be32ae40 2024-05-27T08:02:09.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.212+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5619be3cba20 con 0x5619be4afd30 2024-05-27T08:02:09.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.213+0000 7f7ef4e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4a9db0 0x5619be4aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.213+0000 7f7ef4e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4a9db0 0x5619be4aa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45298/0 (socket says 172.21.15.90:45298) 2024-05-27T08:02:09.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.213+0000 7f7ef4e00700 1 -- 172.21.15.90:0/1502613037 learned_addr learned my addr 172.21.15.90:0/1502613037 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:09.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.213+0000 7f7eefe00700 1 --2- 172.21.15.90:0/1502613037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be32ae40 0x5619be4af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.214+0000 7f7ef4e00700 1 -- 172.21.15.90:0/1502613037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4afd30 msgr2=0x5619be4a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:09.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.214+0000 7f7ef4e00700 1 --2- 172.21.15.90:0/1502613037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4afd30 0x5619be4a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.214+0000 7f7ef4e00700 1 -- 172.21.15.90:0/1502613037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be32ae40 msgr2=0x5619be4af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:09.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.214+0000 7f7ef4e00700 1 --2- 172.21.15.90:0/1502613037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be32ae40 0x5619be4af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.214+0000 7f7ef4e00700 1 -- 172.21.15.90:0/1502613037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619be390420 con 0x5619be4a9db0 2024-05-27T08:02:09.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.215+0000 7f7ef4e00700 1 --2- 172.21.15.90:0/1502613037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4a9db0 0x5619be4aa1c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8007680 tx=0x7f7ee8007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06e5af2f8736568 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:09.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.215+0000 7f7ee5600700 1 -- 172.21.15.90:0/1502613037 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8019020 con 0x5619be4a9db0 2024-05-27T08:02:09.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.215+0000 7f7ee5600700 1 -- 172.21.15.90:0/1502613037 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ee8015ce0 con 0x5619be4a9db0 2024-05-27T08:02:09.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.216+0000 7f7f059f3b80 1 -- 172.21.15.90:0/1502613037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4a9db0 msgr2=0x5619be4aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:09.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.216+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/1502613037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4a9db0 0x5619be4aa1c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8007680 tx=0x7f7ee8007990 comp rx=0 tx=0).stop 2024-05-27T08:02:09.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.217+0000 7f7f059f3b80 1 -- 172.21.15.90:0/1502613037 shutdown_connections 2024-05-27T08:02:09.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.217+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/1502613037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4a9db0 0x5619be4aa1c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.217+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/1502613037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be32ae40 0x5619be4af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.217+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/1502613037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4afd30 0x5619be4a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.217+0000 7f7f059f3b80 1 -- 172.21.15.90:0/1502613037 >> 172.21.15.90:0/1502613037 conn(0x5619be38afc0 msgr2=0x5619be4a48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:09.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.217+0000 7f7f059f3b80 1 -- 172.21.15.90:0/1502613037 shutdown_connections 2024-05-27T08:02:09.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.217+0000 7f7f059f3b80 1 -- 172.21.15.90:0/1502613037 wait complete. 2024-05-27T08:02:09.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.218+0000 7f7f059f3b80 1 -- start start 2024-05-27T08:02:09.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.218+0000 7f7f059f3b80 1 Processor -- start 2024-05-27T08:02:09.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.219+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 0x5619be4b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.219+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 0x5619be4b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.219+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 0x5619be4b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.219+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5619be2cdfa0 con 0x5619be32ae40 2024-05-27T08:02:09.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.219+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5619be368760 con 0x5619be4afd30 2024-05-27T08:02:09.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.219+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5619be4b6800 con 0x5619be4a9db0 2024-05-27T08:02:09.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.219+0000 7f7f059f3b80 10 client.0 _ll_register_callbacks cb 0x5619be371c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:09.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.222+0000 7f7eefe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 0x5619be4b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.222+0000 7f7eefe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 0x5619be4b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45310/0 (socket says 172.21.15.90:45310) 2024-05-27T08:02:09.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.222+0000 7f7eefe00700 1 -- 172.21.15.90:0/908837532 learned_addr learned my addr 172.21.15.90:0/908837532 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:09.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.222+0000 7f7eef400700 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 0x5619be4b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.222+0000 7f7ef4e00700 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 0x5619be4b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.224+0000 7f7eef400700 1 -- 172.21.15.90:0/908837532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 msgr2=0x5619be4b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.224+0000 7f7eef400700 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 0x5619be4b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.224+0000 7f7eef400700 1 -- 172.21.15.90:0/908837532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 msgr2=0x5619be4b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.224+0000 7f7eef400700 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 0x5619be4b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.224+0000 7f7eef400700 1 -- 172.21.15.90:0/908837532 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619be3178d0 con 0x5619be4a9db0 2024-05-27T08:02:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.225+0000 7f7eef400700 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 0x5619be4b3ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00e1680 tx=0x7f7ee0009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6843587099eea24 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.225+0000 7f7ede000700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:02:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.225+0000 7f7ede000700 1 -- 172.21.15.90:0/908837532 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00ab9a0 con 0x5619be4a9db0 2024-05-27T08:02:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.225+0000 7f7f059f3b80 1 -- 172.21.15.90:0/908837532 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee8015ce0 con 0x5619be4a9db0 2024-05-27T08:02:09.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.225+0000 7f7ede000700 1 -- 172.21.15.90:0/908837532 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9584+0+0 (secure 0 0 0) 0x7f7ee00b3df0 con 0x5619be4a9db0 2024-05-27T08:02:09.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.226+0000 7f7ede000700 1 -- 172.21.15.90:0/908837532 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7ee0022470 con 0x5619be4a9db0 2024-05-27T08:02:09.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.226+0000 7f7f059f3b80 10 client.27802 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:09.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.226+0000 7f7f059f3b80 1 -- 172.21.15.90:0/908837532 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5619be398060 con 0x5619be4a9db0 2024-05-27T08:02:09.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.227+0000 7f7ede000700 1 -- 172.21.15.90:0/908837532 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ee8015ce0 con 0x5619be4a9db0 2024-05-27T08:02:09.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.229+0000 7f7ede000700 1 -- 172.21.15.90:0/908837532 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f7ee002a030 con 0x5619be4a9db0 2024-05-27T08:02:09.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.229+0000 7f7f059f3b80 10 client.27802 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:09.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.229+0000 7f7f059f3b80 10 client.27802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:09.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.229+0000 7f7f059f3b80 20 client.27802 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:09.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.229+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5619be4c3760 0x5619be4c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.229+0000 7f7f059f3b80 4 client.27802 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:09.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.229+0000 7f7f059f3b80 1 -- 172.21.15.90:0/908837532 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619be3d9d90 con 0x5619be4c3760 2024-05-27T08:02:09.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.230+0000 7f7eefe00700 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5619be4c3760 0x5619be4c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.233+0000 7f7eefe00700 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5619be4c3760 0x5619be4c5bb0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=729e5c57ab09359b server_cookie=e3f7799108c212e3 in_seq=0 out_seq=0 2024-05-27T08:02:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.233+0000 7f7ede000700 10 client.27802 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:09.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.235+0000 7f7ede000700 1 -- 172.21.15.90:0/908837532 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5619be3d9d90 con 0x5619be4c3760 2024-05-27T08:02:09.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.235+0000 7f7ede000700 10 client.27802 handle_command_reply: tid=0 2024-05-27T08:02:09.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.236+0000 7f7f059f3b80 1 client.27802 shutdown 2024-05-27T08:02:09.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.236+0000 7f7f059f3b80 1 -- 172.21.15.90:0/908837532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 msgr2=0x5619be4b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:09.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.236+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 0x5619be4b3ae0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00e1680 tx=0x7f7ee0009710 comp rx=0 tx=0).stop 2024-05-27T08:02:09.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.237+0000 7f7f059f3b80 1 -- 172.21.15.90:0/908837532 shutdown_connections 2024-05-27T08:02:09.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.237+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5619be4c3760 0x5619be4c5bb0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.237+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 0x5619be4b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.237+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 0x5619be4b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.237+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/908837532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 0x5619be4b3ae0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.237+0000 7f7f059f3b80 1 -- 172.21.15.90:0/908837532 >> 172.21.15.90:0/908837532 conn(0x5619be38afc0 msgr2=0x5619be4a4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:09.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.237+0000 7f7ede000700 0 client.27802 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:09.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.238+0000 7f7f059f3b80 1 -- 172.21.15.90:0/908837532 shutdown_connections 2024-05-27T08:02:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.238+0000 7f7f059f3b80 1 -- 172.21.15.90:0/908837532 wait complete. 2024-05-27T08:02:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.238+0000 7f7f059f3b80 20 client.27802 trim_cache size 0 max 16384 2024-05-27T08:02:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.292+0000 7f7f059f3b80 1 Processor -- start 2024-05-27T08:02:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.292+0000 7f7f059f3b80 1 -- start start 2024-05-27T08:02:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.292+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 0x5619be4a9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.292+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 0x5619be4b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.292+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 0x5619be4b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.292+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01c0ea0 con 0x5619be32ae40 2024-05-27T08:02:09.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.292+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01bdef0 con 0x5619be4afd30 2024-05-27T08:02:09.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.292+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01c0100 con 0x5619be4a9db0 2024-05-27T08:02:09.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.294+0000 7f7ef4e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 0x5619be4b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.295+0000 7f7ef4e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 0x5619be4b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:51226/0 (socket says 172.21.15.90:51226) 2024-05-27T08:02:09.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.295+0000 7f7ef4e00700 1 -- 172.21.15.90:0/1360958313 learned_addr learned my addr 172.21.15.90:0/1360958313 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:09.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.295+0000 7f7eef400700 1 --2- 172.21.15.90:0/1360958313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 0x5619be4b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.296+0000 7f7eefe00700 1 --2- 172.21.15.90:0/1360958313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 0x5619be4a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.298+0000 7f7eef400700 1 -- 172.21.15.90:0/1360958313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 msgr2=0x5619be4b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:09.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.298+0000 7f7eef400700 1 --2- 172.21.15.90:0/1360958313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 0x5619be4b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.298+0000 7f7eef400700 1 -- 172.21.15.90:0/1360958313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 msgr2=0x5619be4a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:09.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.298+0000 7f7eef400700 1 --2- 172.21.15.90:0/1360958313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 0x5619be4a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.298+0000 7f7eef400700 1 -- 172.21.15.90:0/1360958313 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619be4b1b30 con 0x5619be4a9db0 2024-05-27T08:02:09.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.301+0000 7f7eef400700 1 --2- 172.21.15.90:0/1360958313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 0x5619be4b2580 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00abe70 tx=0x7f7ee0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d794f0998ad238 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:09.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.303+0000 7f7ee4c00700 1 -- 172.21.15.90:0/1360958313 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00cc2a0 con 0x5619be4a9db0 2024-05-27T08:02:09.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.303+0000 7f7ee4c00700 1 -- 172.21.15.90:0/1360958313 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ee00ab9a0 con 0x5619be4a9db0 2024-05-27T08:02:09.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.303+0000 7f7ee4c00700 1 -- 172.21.15.90:0/1360958313 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00abc30 con 0x5619be4a9db0 2024-05-27T08:02:09.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.306+0000 7f7f059f3b80 1 -- 172.21.15.90:0/1360958313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 msgr2=0x5619be4b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:09.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.306+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/1360958313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 0x5619be4b2580 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00abe70 tx=0x7f7ee0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.306+0000 7f7f059f3b80 1 -- 172.21.15.90:0/1360958313 shutdown_connections 2024-05-27T08:02:09.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.306+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/1360958313 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4afd30 0x5619be4b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.307+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/1360958313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be32ae40 0x5619be4a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.308+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/1360958313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be4a9db0 0x5619be4b2580 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.308+0000 7f7f059f3b80 1 -- 172.21.15.90:0/1360958313 >> 172.21.15.90:0/1360958313 conn(0x5619be38afc0 msgr2=0x5619be4ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:09.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.309+0000 7f7f059f3b80 1 -- 172.21.15.90:0/1360958313 shutdown_connections 2024-05-27T08:02:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.309+0000 7f7f059f3b80 1 -- 172.21.15.90:0/1360958313 wait complete. 2024-05-27T08:02:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.310+0000 7f7f059f3b80 1 -- start start 2024-05-27T08:02:09.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.311+0000 7f7f059f3b80 1 Processor -- start 2024-05-27T08:02:09.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.312+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be32ae40 0x5619be4cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.313+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4a9db0 0x5619be4cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.313+0000 7f7f059f3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4afd30 0x5619be4cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.314+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01bfa30 con 0x5619be4afd30 2024-05-27T08:02:09.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.314+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01be5c0 con 0x5619be4a9db0 2024-05-27T08:02:09.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.314+0000 7f7f059f3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01bec90 con 0x5619be32ae40 2024-05-27T08:02:09.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.314+0000 7f7f059f3b80 10 client.0 _ll_register_callbacks cb 0x5619be371c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:09.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.314+0000 7f7eefe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be32ae40 0x5619be4cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.314+0000 7f7eefe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be32ae40 0x5619be4cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37600/0 (socket says 172.21.15.90:37600) 2024-05-27T08:02:09.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.314+0000 7f7eefe00700 1 -- 172.21.15.90:0/2838024018 learned_addr learned my addr 172.21.15.90:0/2838024018 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:09.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.314+0000 7f7eef400700 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4a9db0 0x5619be4cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.315+0000 7f7eefe00700 1 -- 172.21.15.90:0/2838024018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4a9db0 msgr2=0x5619be4cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:09.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.315+0000 7f7eefe00700 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4a9db0 0x5619be4cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.315+0000 7f7ef4e00700 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4afd30 0x5619be4cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.315+0000 7f7eefe00700 1 -- 172.21.15.90:0/2838024018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4afd30 msgr2=0x5619be4cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:09.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.315+0000 7f7eefe00700 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4afd30 0x5619be4cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.315+0000 7f7eefe00700 1 -- 172.21.15.90:0/2838024018 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619be52d260 con 0x5619be32ae40 2024-05-27T08:02:09.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.315+0000 7f7eef400700 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4a9db0 0x5619be4cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:09.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.316+0000 7f7eefe00700 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be32ae40 0x5619be4cafc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5619be4c6300 tx=0x7f7ed80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4109515b9154a042 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:09.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.316+0000 7f7ede000700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:02:09.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.318+0000 7f7ede000700 1 -- 172.21.15.90:0/2838024018 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed801ca70 con 0x5619be32ae40 2024-05-27T08:02:09.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.318+0000 7f7ede000700 1 -- 172.21.15.90:0/2838024018 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9584+0+0 (secure 0 0 0) 0x7f7ed801cc10 con 0x5619be32ae40 2024-05-27T08:02:09.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.318+0000 7f7f059f3b80 1 -- 172.21.15.90:0/2838024018 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee00ab9a0 con 0x5619be32ae40 2024-05-27T08:02:09.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.319+0000 7f7ede000700 1 -- 172.21.15.90:0/2838024018 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7ed801ce10 con 0x5619be32ae40 2024-05-27T08:02:09.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.319+0000 7f7f059f3b80 10 client.27808 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:09.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.319+0000 7f7f059f3b80 1 -- 172.21.15.90:0/2838024018 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5619be52e520 con 0x5619be32ae40 2024-05-27T08:02:09.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.320+0000 7f7ede000700 1 -- 172.21.15.90:0/2838024018 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ee00ab9a0 con 0x5619be32ae40 2024-05-27T08:02:09.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.323+0000 7f7ede000700 1 -- 172.21.15.90:0/2838024018 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f7ed8033440 con 0x5619be32ae40 2024-05-27T08:02:09.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.324+0000 7f7f059f3b80 10 client.27808 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:09.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.324+0000 7f7f059f3b80 10 client.27808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:09.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.324+0000 7f7f059f3b80 20 client.27808 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:09.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.324+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5619be52ebb0 0x5619be530fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:09.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.324+0000 7f7f059f3b80 4 client.27808 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:02:09.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.324+0000 7f7f059f3b80 1 -- 172.21.15.90:0/2838024018 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619be3d3570 con 0x5619be52ebb0 2024-05-27T08:02:09.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.325+0000 7f7eef400700 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5619be52ebb0 0x5619be530fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:09.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.326+0000 7f7ede000700 10 client.27808 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:09.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.326+0000 7f7eef400700 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5619be52ebb0 0x5619be530fd0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ce2f5b89d5a28c4 server_cookie=494d69694d528523 in_seq=0 out_seq=0 2024-05-27T08:02:09.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.326+0000 7f7ede000700 1 -- 172.21.15.90:0/2838024018 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5619be3d3570 con 0x5619be52ebb0 2024-05-27T08:02:09.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.326+0000 7f7ede000700 10 client.27808 handle_command_reply: tid=0 2024-05-27T08:02:09.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.326+0000 7f7f059f3b80 1 client.27808 shutdown 2024-05-27T08:02:09.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.331+0000 7f7f059f3b80 1 -- 172.21.15.90:0/2838024018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be32ae40 msgr2=0x5619be4cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:09.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.331+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be32ae40 0x5619be4cafc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x5619be4c6300 tx=0x7f7ed80047c0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.331+0000 7f7f059f3b80 1 -- 172.21.15.90:0/2838024018 shutdown_connections 2024-05-27T08:02:09.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.331+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5619be52ebb0 0x5619be530fd0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.331+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619be4afd30 0x5619be4cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.331+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619be32ae40 0x5619be4cafc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.331+0000 7f7f059f3b80 1 --2- 172.21.15.90:0/2838024018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5619be4a9db0 0x5619be4cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.331+0000 7f7ede000700 0 client.27808 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:09.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.331+0000 7f7f059f3b80 1 -- 172.21.15.90:0/2838024018 >> 172.21.15.90:0/2838024018 conn(0x5619be38afc0 msgr2=0x5619be4b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:09.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.332+0000 7f7f059f3b80 1 -- 172.21.15.90:0/2838024018 shutdown_connections 2024-05-27T08:02:09.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.332+0000 7f7f059f3b80 1 -- 172.21.15.90:0/2838024018 wait complete. 2024-05-27T08:02:09.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.332+0000 7f7f059f3b80 20 client.27808 trim_cache size 0 max 16384 2024-05-27T08:02:09.335 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:02:09.336 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T08:02:09.336 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T08:02:09.336 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:02:09.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.333+0000 7f7ef6e00700 1 -- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ed40b2b40 msgr2=0x7f7ed40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:09.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.333+0000 7f7ef6e00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ed40b2b40 0x7f7ed40b5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ef01aaaa0 tx=0x7f7ed8009260 comp rx=0 tx=0).stop 2024-05-27T08:02:09.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.333+0000 7f7ef6e00700 1 -- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ef010b270 msgr2=0x7f7ef01afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:09.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.333+0000 7f7ef6e00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ef010b270 0x7f7ef01afc80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee00057c0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.334+0000 7f7ef6e00700 1 -- 172.21.15.90:0/2898016773 shutdown_connections 2024-05-27T08:02:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.334+0000 7f7ef6e00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ed40b2b40 0x7f7ed40b5050 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.334+0000 7f7ef6e00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ef011d920 0x7f7ef01a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.334+0000 7f7ef6e00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ef010a920 0x7f7ef01af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.334+0000 7f7ef6e00700 1 --2- 172.21.15.90:0/2898016773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ef010b270 0x7f7ef01afc80 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.334+0000 7f7ef6e00700 1 -- 172.21.15.90:0/2898016773 >> 172.21.15.90:0/2898016773 conn(0x7f7ef006f4c0 msgr2=0x7f7ef011b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:09.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.334+0000 7f7ef6e00700 1 -- 172.21.15.90:0/2898016773 shutdown_connections 2024-05-27T08:02:09.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:09.334+0000 7f7ef6e00700 1 -- 172.21.15.90:0/2898016773 wait complete. 2024-05-27T08:02:10.358 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:02:10.359 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:02:10.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:10 smithi090 ceph-mon[31429]: pgmap v506: 129 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.5 MiB/s wr, 699 op/s 2024-05-27T08:02:11.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:10 smithi136 ceph-mon[35324]: pgmap v506: 129 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.5 MiB/s wr, 699 op/s 2024-05-27T08:02:11.055 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:11.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:10 smithi129 ceph-mon[35254]: pgmap v506: 129 pgs: 7 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 110 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.5 MiB/s wr, 699 op/s 2024-05-27T08:02:13.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:12 smithi136 ceph-mon[35324]: pgmap v507: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 7.1 MiB/s wr, 729 op/s 2024-05-27T08:02:13.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:12 smithi136 ceph-mon[35324]: osdmap e120: 12 total, 12 up, 12 in 2024-05-27T08:02:13.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:12 smithi129 ceph-mon[35254]: pgmap v507: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 7.1 MiB/s wr, 729 op/s 2024-05-27T08:02:13.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:12 smithi129 ceph-mon[35254]: osdmap e120: 12 total, 12 up, 12 in 2024-05-27T08:02:13.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:12 smithi090 ceph-mon[31429]: pgmap v507: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 7.1 MiB/s wr, 729 op/s 2024-05-27T08:02:13.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:12 smithi090 ceph-mon[31429]: osdmap e120: 12 total, 12 up, 12 in 2024-05-27T08:02:13.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 715.0 s, 401.0 tps, lat 294.566 ms stddev 511.336 2024-05-27T08:02:15.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:14 smithi136 ceph-mon[35324]: pgmap v509: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 452 op/s 2024-05-27T08:02:15.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:14 smithi129 ceph-mon[35254]: pgmap v509: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 452 op/s 2024-05-27T08:02:15.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:14 smithi090 ceph-mon[31429]: pgmap v509: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.4 MiB/s wr, 452 op/s 2024-05-27T08:02:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.285+0000 7f2f5e600640 1 -- 172.21.15.90:0/4040206766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f58115090 msgr2=0x7f2f58117530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.285+0000 7f2f5e600640 1 --2- 172.21.15.90:0/4040206766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f58115090 0x7f2f58117530 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2f50009ca0 tx=0x7f2f5002a8e0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.285+0000 7f2f5e600640 1 -- 172.21.15.90:0/4040206766 shutdown_connections 2024-05-27T08:02:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.285+0000 7f2f5e600640 1 --2- 172.21.15.90:0/4040206766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f5810f330 0x7f2f5810f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.285+0000 7f2f5e600640 1 --2- 172.21.15.90:0/4040206766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f58115090 0x7f2f58117530 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.285+0000 7f2f5e600640 1 --2- 172.21.15.90:0/4040206766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f5810ad60 0x7f2f5810edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.286+0000 7f2f5e600640 1 -- 172.21.15.90:0/4040206766 >> 172.21.15.90:0/4040206766 conn(0x7f2f5806eda0 msgr2=0x7f2f5806f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.286+0000 7f2f5e600640 1 -- 172.21.15.90:0/4040206766 shutdown_connections 2024-05-27T08:02:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.286+0000 7f2f5e600640 1 -- 172.21.15.90:0/4040206766 wait complete. 2024-05-27T08:02:16.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.288+0000 7f2f5e600640 1 Processor -- start 2024-05-27T08:02:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.288+0000 7f2f5e600640 1 -- start start 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.289+0000 7f2f5e600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f5810ad60 0x7f2f581a6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.289+0000 7f2f5e600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f5810f330 0x7f2f581a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.289+0000 7f2f5e600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f58115090 0x7f2f581b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.289+0000 7f2f5e600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5811c5d0 con 0x7f2f58115090 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.289+0000 7f2f5e600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5811c450 con 0x7f2f5810f330 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.289+0000 7f2f5e600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5811c750 con 0x7f2f5810ad60 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.289+0000 7f2f57400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f5810ad60 0x7f2f581a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.290+0000 7f2f56a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f5810f330 0x7f2f581a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.290+0000 7f2f56a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f5810f330 0x7f2f581a65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:51244/0 (socket says 172.21.15.90:51244) 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.290+0000 7f2f56a00640 1 -- 172.21.15.90:0/3916802693 learned_addr learned my addr 172.21.15.90:0/3916802693 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.290+0000 7f2f57e00640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f58115090 0x7f2f581b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.291+0000 7f2f57400640 1 -- 172.21.15.90:0/3916802693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f5810f330 msgr2=0x7f2f581a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.291+0000 7f2f57400640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f5810f330 0x7f2f581a65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.291+0000 7f2f57400640 1 -- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f58115090 msgr2=0x7f2f581b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.294+0000 7f2f57400640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f58115090 0x7f2f581b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.294+0000 7f2f57400640 1 -- 172.21.15.90:0/3916802693 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f581b1470 con 0x7f2f5810ad60 2024-05-27T08:02:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.294+0000 7f2f57e00640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f58115090 0x7f2f581b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.294+0000 7f2f57400640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f5810ad60 0x7f2f581a6090 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800b3c0 tx=0x7f2f4800b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.296+0000 7f2f4fe00640 1 -- 172.21.15.90:0/3916802693 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48011830 con 0x7f2f5810ad60 2024-05-27T08:02:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.296+0000 7f2f4fe00640 1 -- 172.21.15.90:0/3916802693 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f48016070 con 0x7f2f5810ad60 2024-05-27T08:02:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.296+0000 7f2f4fe00640 1 -- 172.21.15.90:0/3916802693 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48010cb0 con 0x7f2f5810ad60 2024-05-27T08:02:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.296+0000 7f2f5e600640 1 -- 172.21.15.90:0/3916802693 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f58074f50 con 0x7f2f5810ad60 2024-05-27T08:02:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.296+0000 7f2f5e600640 1 -- 172.21.15.90:0/3916802693 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f58075420 con 0x7f2f5810ad60 2024-05-27T08:02:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.301+0000 7f2f6559a740 1 Processor -- start 2024-05-27T08:02:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.302+0000 7f2f6559a740 1 -- start start 2024-05-27T08:02:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.304+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36ae95860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.304+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 0x55c36ae95da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.304+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 0x55c36ae962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.304+0000 7f2f6559a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c36ae06b90 con 0x55c36aad24d0 2024-05-27T08:02:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.304+0000 7f2f6559a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c36adf6730 con 0x55c36aaf31b0 2024-05-27T08:02:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.304+0000 7f2f6559a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c36adfa770 con 0x55c36aaca2f0 2024-05-27T08:02:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.304+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36ae95860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.304+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36ae95860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:51260/0 (socket says 172.21.15.90:51260) 2024-05-27T08:02:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.304+0000 7f2f57e00640 1 -- 172.21.15.90:0/2536082317 learned_addr learned my addr 172.21.15.90:0/2536082317 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.304+0000 7f2f56a00640 1 --2- 172.21.15.90:0/2536082317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 0x55c36ae962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.305+0000 7f2f4fe00640 1 -- 172.21.15.90:0/3916802693 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2f480119d0 con 0x7f2f5810ad60 2024-05-27T08:02:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.305+0000 7f2f56a00640 1 -- 172.21.15.90:0/2536082317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 msgr2=0x55c36ae95da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:02:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.305+0000 7f2f56a00640 1 --2- 172.21.15.90:0/2536082317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 0x55c36ae95da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.305+0000 7f2f56a00640 1 -- 172.21.15.90:0/2536082317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 msgr2=0x55c36ae95860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.305+0000 7f2f56a00640 1 --2- 172.21.15.90:0/2536082317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36ae95860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.305+0000 7f2f56a00640 1 -- 172.21.15.90:0/2536082317 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c36ad8f8b0 con 0x55c36aad24d0 2024-05-27T08:02:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.306+0000 7f2f56a00640 1 --2- 172.21.15.90:0/2536082317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 0x55c36ae962e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2f440099f0 tx=0x7f2f44009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1feed37f7143c3c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.307+0000 7f2f4cc00640 1 -- 172.21.15.90:0/2536082317 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44017070 con 0x55c36aad24d0 2024-05-27T08:02:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.307+0000 7f2f4cc00640 1 -- 172.21.15.90:0/2536082317 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f44013470 con 0x55c36aad24d0 2024-05-27T08:02:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.307+0000 7f2f4cc00640 1 -- 172.21.15.90:0/2536082317 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44012630 con 0x55c36aad24d0 2024-05-27T08:02:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.307+0000 7f2f4fe00640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f3c0b2aa0 0x7f2f3c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.308+0000 7f2f4fe00640 1 -- 172.21.15.90:0/3916802693 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9508+0+0 (secure 0 0 0) 0x7f2f480e1930 con 0x7f2f5810ad60 2024-05-27T08:02:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.308+0000 7f2f57e00640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f3c0b2aa0 0x7f2f3c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.308+0000 7f2f6559a740 1 -- 172.21.15.90:0/2536082317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 msgr2=0x55c36ae962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.308+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2536082317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 0x55c36ae962e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2f440099f0 tx=0x7f2f44009eb0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.309+0000 7f2f6559a740 1 -- 172.21.15.90:0/2536082317 shutdown_connections 2024-05-27T08:02:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.309+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2536082317 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 0x55c36ae962e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.309+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2536082317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 0x55c36ae95da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.309+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2536082317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36ae95860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.309+0000 7f2f6559a740 1 -- 172.21.15.90:0/2536082317 >> 172.21.15.90:0/2536082317 conn(0x55c36aae7300 msgr2=0x55c36ae92010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:16.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.311+0000 7f2f6559a740 1 -- 172.21.15.90:0/2536082317 shutdown_connections 2024-05-27T08:02:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.311+0000 7f2f57e00640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f3c0b2aa0 0x7f2f3c0b4fb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f50031c90 tx=0x7f2f5005f040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:16.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.311+0000 7f2f6559a740 1 -- 172.21.15.90:0/2536082317 wait complete. 2024-05-27T08:02:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.312+0000 7f2f6559a740 1 -- start start 2024-05-27T08:02:16.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.313+0000 7f2f6559a740 1 Processor -- start 2024-05-27T08:02:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.315+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36ae9f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.315+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aad24d0 0x55c36ae9ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.315+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36aeabd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.315+0000 7f2f6559a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c36aadc790 con 0x55c36aaca2f0 2024-05-27T08:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.315+0000 7f2f6559a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c36ad9ebc0 con 0x55c36aaf31b0 2024-05-27T08:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.315+0000 7f2f6559a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c36aea2990 con 0x55c36aad24d0 2024-05-27T08:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.315+0000 7f2f6559a740 10 client.0 _ll_register_callbacks cb 0x55c36ad7c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.315+0000 7f2f57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36ae9f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.316+0000 7f2f57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36ae9f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45364/0 (socket says 172.21.15.90:45364) 2024-05-27T08:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.316+0000 7f2f57400640 1 -- 172.21.15.90:0/2014882660 learned_addr learned my addr 172.21.15.90:0/2014882660 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.316+0000 7f2f57e00640 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36aeabd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.317+0000 7f2f57400640 1 -- 172.21.15.90:0/2014882660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aad24d0 msgr2=0x55c36ae9ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.317+0000 7f2f57400640 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aad24d0 0x55c36ae9ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.317+0000 7f2f57400640 1 -- 172.21.15.90:0/2014882660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 msgr2=0x55c36aeabd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:16.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.317+0000 7f2f57400640 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36aeabd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.317+0000 7f2f57400640 1 -- 172.21.15.90:0/2014882660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c36ab0b210 con 0x55c36aaca2f0 2024-05-27T08:02:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.317+0000 7f2f57400640 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36ae9f9c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2f480a49f0 tx=0x7f2f480b1140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dc3cba84c06944 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.317+0000 7f2f41600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.318+0000 7f2f41600640 1 -- 172.21.15.90:0/2014882660 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480a8970 con 0x55c36aaca2f0 2024-05-27T08:02:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.318+0000 7f2f41600640 1 -- 172.21.15.90:0/2014882660 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9508+0+0 (secure 0 0 0) 0x7f2f480cb0b0 con 0x55c36aaca2f0 2024-05-27T08:02:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.318+0000 7f2f6559a740 1 -- 172.21.15.90:0/2014882660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f44013470 con 0x55c36aaca2f0 2024-05-27T08:02:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.318+0000 7f2f41600640 1 -- 172.21.15.90:0/2014882660 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f2f480acd20 con 0x55c36aaca2f0 2024-05-27T08:02:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.318+0000 7f2f6559a740 10 client.18057 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.318+0000 7f2f6559a740 1 -- 172.21.15.90:0/2014882660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55c36abd7620 con 0x55c36aaca2f0 2024-05-27T08:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.319+0000 7f2f41600640 1 -- 172.21.15.90:0/2014882660 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f44013470 con 0x55c36aaca2f0 2024-05-27T08:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.319+0000 7f2f41600640 1 -- 172.21.15.90:0/2014882660 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f2f480af0d0 con 0x55c36aaca2f0 2024-05-27T08:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.319+0000 7f2f6559a740 10 client.18057 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.319+0000 7f2f6559a740 10 client.18057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.319+0000 7f2f6559a740 20 client.18057 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.319+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c36aab1d80 0x55c36aeb1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.319+0000 7f2f6559a740 4 client.18057 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.319+0000 7f2f6559a740 1 -- 172.21.15.90:0/2014882660 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c36ae08ae0 con 0x55c36aab1d80 2024-05-27T08:02:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.321+0000 7f2f56a00640 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c36aab1d80 0x55c36aeb1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.321+0000 7f2f56a00640 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c36aab1d80 0x55c36aeb1de0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cfb2e8f296fa5d5 server_cookie=98527e67be8a29da in_seq=0 out_seq=0 2024-05-27T08:02:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.322+0000 7f2f41600640 10 client.18057 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.323+0000 7f2f41600640 1 -- 172.21.15.90:0/2014882660 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c36ae08ae0 con 0x55c36aab1d80 2024-05-27T08:02:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.323+0000 7f2f41600640 10 client.18057 handle_command_reply: tid=0 2024-05-27T08:02:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.323+0000 7f2f6559a740 1 client.18057 shutdown 2024-05-27T08:02:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.324+0000 7f2f6559a740 1 -- 172.21.15.90:0/2014882660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 msgr2=0x55c36ae9f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.324+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36ae9f9c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2f480a49f0 tx=0x7f2f480b1140 comp rx=0 tx=0).stop 2024-05-27T08:02:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.324+0000 7f2f6559a740 1 -- 172.21.15.90:0/2014882660 shutdown_connections 2024-05-27T08:02:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.324+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c36aab1d80 0x55c36aeb1de0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.324+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36aeabd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.324+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aad24d0 0x55c36ae9ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.324+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2014882660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36ae9f9c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.324+0000 7f2f6559a740 1 -- 172.21.15.90:0/2014882660 >> 172.21.15.90:0/2014882660 conn(0x55c36aae7300 msgr2=0x55c36ae97580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.324+0000 7f2f41600640 0 client.18057 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.324+0000 7f2f6559a740 1 -- 172.21.15.90:0/2014882660 shutdown_connections 2024-05-27T08:02:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.327+0000 7f2f6559a740 1 -- 172.21.15.90:0/2014882660 wait complete. 2024-05-27T08:02:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.328+0000 7f2f6559a740 20 client.18057 trim_cache size 0 max 16384 2024-05-27T08:02:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.377+0000 7f2f6559a740 1 Processor -- start 2024-05-27T08:02:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.377+0000 7f2f6559a740 1 -- start start 2024-05-27T08:02:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.379+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 0x55c36ae9b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.379+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 0x55c36aea1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.379+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36ae9f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.379+0000 7f2f6559a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f58081500 con 0x55c36aad24d0 2024-05-27T08:02:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.379+0000 7f2f6559a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5810ba00 con 0x55c36aaf31b0 2024-05-27T08:02:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.379+0000 7f2f6559a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f58080760 con 0x55c36aaca2f0 2024-05-27T08:02:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.380+0000 7f2f57400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 0x55c36ae9b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.380+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36ae9f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.380+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36ae9f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:51288/0 (socket says 172.21.15.90:51288) 2024-05-27T08:02:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.380+0000 7f2f57400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 0x55c36ae9b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37642/0 (socket says 172.21.15.90:37642) 2024-05-27T08:02:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.380+0000 7f2f57e00640 1 -- 172.21.15.90:0/2540101748 learned_addr learned my addr 172.21.15.90:0/2540101748 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.381+0000 7f2f56a00640 1 --2- 172.21.15.90:0/2540101748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 0x55c36aea1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.381+0000 7f2f57400640 1 -- 172.21.15.90:0/2540101748 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 msgr2=0x55c36ae9f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.381+0000 7f2f57400640 1 --2- 172.21.15.90:0/2540101748 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36ae9f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.381+0000 7f2f57400640 1 -- 172.21.15.90:0/2540101748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 msgr2=0x55c36aea1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.381+0000 7f2f57400640 1 --2- 172.21.15.90:0/2540101748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 0x55c36aea1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.381+0000 7f2f57400640 1 -- 172.21.15.90:0/2540101748 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c36ae97f70 con 0x55c36aaca2f0 2024-05-27T08:02:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.384+0000 7f2f57400640 1 --2- 172.21.15.90:0/2540101748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 0x55c36ae9b210 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2f480a9aa0 tx=0x7f2f4801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c2327063f2310e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.385+0000 7f2f43e00640 1 -- 172.21.15.90:0/2540101748 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480a82c0 con 0x55c36aaca2f0 2024-05-27T08:02:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.385+0000 7f2f43e00640 1 -- 172.21.15.90:0/2540101748 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f480df090 con 0x55c36aaca2f0 2024-05-27T08:02:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.385+0000 7f2f43e00640 1 -- 172.21.15.90:0/2540101748 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480a4830 con 0x55c36aaca2f0 2024-05-27T08:02:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.386+0000 7f2f6559a740 1 -- 172.21.15.90:0/2540101748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 msgr2=0x55c36ae9b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.386+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2540101748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 0x55c36ae9b210 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2f480a9aa0 tx=0x7f2f4801e000 comp rx=0 tx=0).stop 2024-05-27T08:02:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.390+0000 7f2f6559a740 1 -- 172.21.15.90:0/2540101748 shutdown_connections 2024-05-27T08:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.390+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2540101748 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36ae9f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.390+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2540101748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aad24d0 0x55c36aea1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.390+0000 7f2f6559a740 1 --2- 172.21.15.90:0/2540101748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aaca2f0 0x55c36ae9b210 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.390+0000 7f2f6559a740 1 -- 172.21.15.90:0/2540101748 >> 172.21.15.90:0/2540101748 conn(0x55c36aae7300 msgr2=0x55c36ae9f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.391+0000 7f2f6559a740 1 -- 172.21.15.90:0/2540101748 shutdown_connections 2024-05-27T08:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.391+0000 7f2f6559a740 1 -- 172.21.15.90:0/2540101748 wait complete. 2024-05-27T08:02:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.392+0000 7f2f6559a740 1 -- start start 2024-05-27T08:02:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.392+0000 7f2f6559a740 1 Processor -- start 2024-05-27T08:02:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.395+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36aea2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.395+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aad24d0 0x55c36aea3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.395+0000 7f2f6559a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36aea36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.396+0000 7f2f6559a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f58080090 con 0x55c36aaca2f0 2024-05-27T08:02:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.396+0000 7f2f6559a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5807ec20 con 0x55c36aaf31b0 2024-05-27T08:02:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.396+0000 7f2f6559a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5807f2f0 con 0x55c36aad24d0 2024-05-27T08:02:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.396+0000 7f2f57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36aea2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.396+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36aea36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.396+0000 7f2f57e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36aea36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:51290/0 (socket says 172.21.15.90:51290) 2024-05-27T08:02:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.396+0000 7f2f57e00640 1 -- 172.21.15.90:0/798063478 learned_addr learned my addr 172.21.15.90:0/798063478 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.396+0000 7f2f6559a740 10 client.0 _ll_register_callbacks cb 0x7f2f4800bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.397+0000 7f2f56a00640 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aad24d0 0x55c36aea3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.398+0000 7f2f57400640 1 -- 172.21.15.90:0/798063478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aad24d0 msgr2=0x55c36aea3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.399+0000 7f2f57400640 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aad24d0 0x55c36aea3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.399+0000 7f2f57400640 1 -- 172.21.15.90:0/798063478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 msgr2=0x55c36aea36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.399+0000 7f2f57400640 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36aea36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.399+0000 7f2f57400640 1 -- 172.21.15.90:0/798063478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c36af20e30 con 0x55c36aaca2f0 2024-05-27T08:02:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.399+0000 7f2f56a00640 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aad24d0 0x55c36aea3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.400+0000 7f2f57e00640 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36aea36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.402+0000 7f2f57400640 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36aea2c30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x55c36ae97160 tx=0x7f2f480a80e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed341e7e3caf0fde server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.402+0000 7f2f41600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.404+0000 7f2f41600640 1 -- 172.21.15.90:0/798063478 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480a4830 con 0x55c36aaca2f0 2024-05-27T08:02:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.404+0000 7f2f41600640 1 -- 172.21.15.90:0/798063478 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9508+0+0 (secure 0 0 0) 0x7f2f480a4f60 con 0x55c36aaca2f0 2024-05-27T08:02:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.404+0000 7f2f6559a740 1 -- 172.21.15.90:0/798063478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f480df090 con 0x55c36aaca2f0 2024-05-27T08:02:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.405+0000 7f2f41600640 1 -- 172.21.15.90:0/798063478 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f2f480a82c0 con 0x55c36aaca2f0 2024-05-27T08:02:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.405+0000 7f2f6559a740 10 client.18060 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.405+0000 7f2f6559a740 1 -- 172.21.15.90:0/798063478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55c36af20e30 con 0x55c36aaca2f0 2024-05-27T08:02:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.407+0000 7f2f41600640 1 -- 172.21.15.90:0/798063478 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f480df090 con 0x55c36aaca2f0 2024-05-27T08:02:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.409+0000 7f2f41600640 1 -- 172.21.15.90:0/798063478 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f2f480a4480 con 0x55c36aaca2f0 2024-05-27T08:02:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.409+0000 7f2f6559a740 10 client.18060 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.409+0000 7f2f6559a740 10 client.18060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.409+0000 7f2f6559a740 20 client.18060 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.409+0000 7f2f6559a740 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c36aa871c0 0x55c36af24300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.409+0000 7f2f6559a740 4 client.18060 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:02:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.409+0000 7f2f6559a740 1 -- 172.21.15.90:0/798063478 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c36ae022c0 con 0x55c36aa871c0 2024-05-27T08:02:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.410+0000 7f2f56a00640 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c36aa871c0 0x55c36af24300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.414+0000 7f2f41600640 10 client.18060 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.414+0000 7f2f56a00640 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c36aa871c0 0x55c36af24300 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b92f2dbaac22c20 server_cookie=6cce558f896ee1d5 in_seq=0 out_seq=0 2024-05-27T08:02:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.480+0000 7f2f41600640 1 -- 172.21.15.90:0/798063478 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c36ae022c0 con 0x55c36aa871c0 2024-05-27T08:02:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.480+0000 7f2f41600640 10 client.18060 handle_command_reply: tid=0 2024-05-27T08:02:16.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.480+0000 7f2f6559a740 1 client.18060 shutdown 2024-05-27T08:02:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f6559a740 1 -- 172.21.15.90:0/798063478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 msgr2=0x55c36aea2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f6559a740 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36aea2c30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x55c36ae97160 tx=0x7f2f480a80e0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f6559a740 1 -- 172.21.15.90:0/798063478 shutdown_connections 2024-05-27T08:02:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f6559a740 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c36aa871c0 0x55c36af24300 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f6559a740 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c36aaf31b0 0x55c36aea36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f6559a740 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c36aad24d0 0x55c36aea3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f6559a740 1 --2- 172.21.15.90:0/798063478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c36aaca2f0 0x55c36aea2c30 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f6559a740 1 -- 172.21.15.90:0/798063478 >> 172.21.15.90:0/798063478 conn(0x55c36aae7300 msgr2=0x55c36aea17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f41600640 0 client.18060 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f6559a740 1 -- 172.21.15.90:0/798063478 shutdown_connections 2024-05-27T08:02:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.481+0000 7f2f6559a740 1 -- 172.21.15.90:0/798063478 wait complete. 2024-05-27T08:02:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.482+0000 7f2f6559a740 20 client.18060 trim_cache size 0 max 16384 2024-05-27T08:02:16.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:02:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:02:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:02:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.483+0000 7f2f5e600640 1 -- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f3c0b2aa0 msgr2=0x7f2f3c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.483+0000 7f2f5e600640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f3c0b2aa0 0x7f2f3c0b4fb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f50031c90 tx=0x7f2f5005f040 comp rx=0 tx=0).stop 2024-05-27T08:02:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.483+0000 7f2f5e600640 1 -- 172.21.15.90:0/3916802693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f5810ad60 msgr2=0x7f2f581a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.483+0000 7f2f5e600640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f5810ad60 0x7f2f581a6090 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800b3c0 tx=0x7f2f4800b880 comp rx=0 tx=0).stop 2024-05-27T08:02:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.484+0000 7f2f5e600640 1 -- 172.21.15.90:0/3916802693 shutdown_connections 2024-05-27T08:02:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.484+0000 7f2f5e600640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f2f3c0b2aa0 0x7f2f3c0b4fb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.484+0000 7f2f5e600640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f5810f330 0x7f2f581a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.484+0000 7f2f5e600640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f58115090 0x7f2f581b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.484+0000 7f2f5e600640 1 --2- 172.21.15.90:0/3916802693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f5810ad60 0x7f2f581a6090 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.484+0000 7f2f5e600640 1 -- 172.21.15.90:0/3916802693 >> 172.21.15.90:0/3916802693 conn(0x7f2f5806eda0 msgr2=0x7f2f5810bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.484+0000 7f2f5e600640 1 -- 172.21.15.90:0/3916802693 shutdown_connections 2024-05-27T08:02:16.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:16.484+0000 7f2f5e600640 1 -- 172.21.15.90:0/3916802693 wait complete. 2024-05-27T08:02:17.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:16 smithi136 ceph-mon[35324]: pgmap v510: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 511 op/s 2024-05-27T08:02:17.024 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:16 smithi090 ceph-mon[31429]: pgmap v510: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 511 op/s 2024-05-27T08:02:17.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:16 smithi129 ceph-mon[35254]: pgmap v510: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.4 MiB/s wr, 511 op/s 2024-05-27T08:02:18.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.045+0000 7f1c0fa00700 1 -- 172.21.15.90:0/2698940319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c08110b90 msgr2=0x7f1c08112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:18.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.045+0000 7f1c0fa00700 1 --2- 172.21.15.90:0/2698940319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c08110b90 0x7f1c08112ff0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800b0a0 tx=0x7f1bf800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.045+0000 7f1c0fa00700 1 -- 172.21.15.90:0/2698940319 shutdown_connections 2024-05-27T08:02:18.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.045+0000 7f1c0fa00700 1 --2- 172.21.15.90:0/2698940319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c08110b90 0x7f1c08112ff0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.045+0000 7f1c0fa00700 1 --2- 172.21.15.90:0/2698940319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c08075d20 0x7f1c08076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.045+0000 7f1c0fa00700 1 --2- 172.21.15.90:0/2698940319 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c08076670 0x7f1c08110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.045+0000 7f1c0fa00700 1 -- 172.21.15.90:0/2698940319 >> 172.21.15.90:0/2698940319 conn(0x7f1c08100030 msgr2=0x7f1c08102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:18.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.046+0000 7f1c0fa00700 1 -- 172.21.15.90:0/2698940319 shutdown_connections 2024-05-27T08:02:18.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.046+0000 7f1c0fa00700 1 -- 172.21.15.90:0/2698940319 wait complete. 2024-05-27T08:02:18.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.047+0000 7f1c0fa00700 1 Processor -- start 2024-05-27T08:02:18.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.048+0000 7f1c0fa00700 1 -- start start 2024-05-27T08:02:18.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.048+0000 7f1c0fa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c08075d20 0x7f1c0819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:18.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.048+0000 7f1c0fa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c08076670 0x7f1c0819d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:18.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.049+0000 7f1c0fa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c08110b90 0x7f1c081a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:18.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.049+0000 7f1c0fa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08116570 con 0x7f1c08110b90 2024-05-27T08:02:18.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.049+0000 7f1c0fa00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081163f0 con 0x7f1c08076670 2024-05-27T08:02:18.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.049+0000 7f1c0fa00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081166f0 con 0x7f1c08075d20 2024-05-27T08:02:18.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.049+0000 7f1c07e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c08076670 0x7f1c0819d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.049+0000 7f1c07e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c08076670 0x7f1c0819d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:51312/0 (socket says 172.21.15.90:51312) 2024-05-27T08:02:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.049+0000 7f1c07e00700 1 -- 172.21.15.90:0/3794834549 learned_addr learned my addr 172.21.15.90:0/3794834549 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.050+0000 7f1c0d000700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c08075d20 0x7f1c0819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.050+0000 7f1c0da00700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c08110b90 0x7f1c081a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.051+0000 7f1c0da00700 1 -- 172.21.15.90:0/3794834549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c08075d20 msgr2=0x7f1c0819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:18.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.051+0000 7f1c0da00700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c08075d20 0x7f1c0819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.051+0000 7f1c0da00700 1 -- 172.21.15.90:0/3794834549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c08076670 msgr2=0x7f1c0819d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.051+0000 7f1c0da00700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c08076670 0x7f1c0819d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.051+0000 7f1c0da00700 1 -- 172.21.15.90:0/3794834549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c081a1c90 con 0x7f1c08110b90 2024-05-27T08:02:18.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.052+0000 7f1c0d000700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c08075d20 0x7f1c0819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:18.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.052+0000 7f1c0da00700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c08110b90 0x7f1c081a15f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800b780 tx=0x7f1bf800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:18.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.053+0000 7f1c05600700 1 -- 172.21.15.90:0/3794834549 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8021070 con 0x7f1c08110b90 2024-05-27T08:02:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.054+0000 7f1c05600700 1 -- 172.21.15.90:0/3794834549 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1bf8007d00 con 0x7f1c08110b90 2024-05-27T08:02:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.054+0000 7f1c05600700 1 -- 172.21.15.90:0/3794834549 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8005430 con 0x7f1c08110b90 2024-05-27T08:02:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.054+0000 7f1c0fa00700 1 -- 172.21.15.90:0/3794834549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c081a1f20 con 0x7f1c08110b90 2024-05-27T08:02:18.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.054+0000 7f1c0fa00700 1 -- 172.21.15.90:0/3794834549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c081a2380 con 0x7f1c08110b90 2024-05-27T08:02:18.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.056+0000 7f1c0fa00700 1 -- 172.21.15.90:0/3794834549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c080512b0 con 0x7f1c08110b90 2024-05-27T08:02:18.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.061+0000 7f1c05600700 1 -- 172.21.15.90:0/3794834549 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1bf8024070 con 0x7f1c08110b90 2024-05-27T08:02:18.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.064+0000 7f1c05600700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1bec0b2bc0 0x7f1bec0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:18.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.064+0000 7f1c05600700 1 -- 172.21.15.90:0/3794834549 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f1bf80e3f10 con 0x7f1c08110b90 2024-05-27T08:02:18.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.068+0000 7f1c0d000700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1bec0b2bc0 0x7f1bec0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:18.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.069+0000 7f1c05600700 1 -- 172.21.15.90:0/3794834549 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1bf8009cb0 con 0x7f1c08110b90 2024-05-27T08:02:18.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.070+0000 7f1c0d000700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1bec0b2bc0 0x7f1bec0b50d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000bc10 tx=0x7f1c0000b5f0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:18.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.546+0000 7f1c0fa00700 1 -- 172.21.15.90:0/3794834549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c0802e5e0 con 0x7f1c08110b90 2024-05-27T08:02:18.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.550+0000 7f1c05600700 1 -- 172.21.15.90:0/3794834549 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+9456 (secure 0 0 0) 0x7f1bf80ab5e0 con 0x7f1c08110b90 2024-05-27T08:02:18.554 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:02:18.554 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":64,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:56:32.161929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T08:02:18.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.555+0000 7f1c0fa00700 1 -- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1bec0b2bc0 msgr2=0x7f1bec0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:18.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.555+0000 7f1c0fa00700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1bec0b2bc0 0x7f1bec0b50d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000bc10 tx=0x7f1c0000b5f0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.555+0000 7f1c0fa00700 1 -- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c08110b90 msgr2=0x7f1c081a15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:18.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.555+0000 7f1c0fa00700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c08110b90 0x7f1c081a15f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1bf800b780 tx=0x7f1bf800bf40 comp rx=0 tx=0).stop 2024-05-27T08:02:18.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.556+0000 7f1c0fa00700 1 -- 172.21.15.90:0/3794834549 shutdown_connections 2024-05-27T08:02:18.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.556+0000 7f1c0fa00700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c08110b90 0x7f1c081a15f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.556+0000 7f1c0fa00700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c08075d20 0x7f1c0819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.556+0000 7f1c0fa00700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1bec0b2bc0 0x7f1bec0b50d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.556+0000 7f1c0fa00700 1 --2- 172.21.15.90:0/3794834549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c08076670 0x7f1c0819d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:18.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.556+0000 7f1c0fa00700 1 -- 172.21.15.90:0/3794834549 >> 172.21.15.90:0/3794834549 conn(0x7f1c08100030 msgr2=0x7f1c08101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:18.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.557+0000 7f1c0fa00700 1 -- 172.21.15.90:0/3794834549 shutdown_connections 2024-05-27T08:02:18.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:18.558+0000 7f1c0fa00700 1 -- 172.21.15.90:0/3794834549 wait complete. 2024-05-27T08:02:18.563 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 64 2024-05-27T08:02:18.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:18 smithi090 ceph-mon[31429]: osdmap e121: 12 total, 12 up, 12 in 2024-05-27T08:02:18.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:18 smithi090 ceph-mon[31429]: pgmap v512: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.1 MiB/s wr, 302 op/s 2024-05-27T08:02:18.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:18 smithi136 ceph-mon[35324]: osdmap e121: 12 total, 12 up, 12 in 2024-05-27T08:02:18.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:18 smithi136 ceph-mon[35324]: pgmap v512: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.1 MiB/s wr, 302 op/s 2024-05-27T08:02:18.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:18 smithi129 ceph-mon[35254]: osdmap e121: 12 total, 12 up, 12 in 2024-05-27T08:02:18.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:18 smithi129 ceph-mon[35254]: pgmap v512: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.1 MiB/s wr, 302 op/s 2024-05-27T08:02:18.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 720.0 s, 420.4 tps, lat 241.749 ms stddev 179.957 2024-05-27T08:02:19.567 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T08:02:19.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:19 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/3794834549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:02:19.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:19 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/3794834549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:02:19.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:19 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/3794834549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:02:20.351 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:20.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:20 smithi090 ceph-mon[31429]: pgmap v513: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.8 MiB/s wr, 514 op/s 2024-05-27T08:02:20.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:20 smithi136 ceph-mon[35324]: pgmap v513: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.8 MiB/s wr, 514 op/s 2024-05-27T08:02:20.824 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:20 smithi129 ceph-mon[35254]: pgmap v513: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.8 MiB/s wr, 514 op/s 2024-05-27T08:02:22.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:22 smithi090 ceph-mon[31429]: pgmap v514: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 487 op/s 2024-05-27T08:02:23.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:22 smithi136 ceph-mon[35324]: pgmap v514: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 487 op/s 2024-05-27T08:02:23.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:22 smithi129 ceph-mon[35254]: pgmap v514: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.3 MiB/s wr, 487 op/s 2024-05-27T08:02:23.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 725.0 s, 447.8 tps, lat 217.826 ms stddev 158.828 2024-05-27T08:02:24.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:24 smithi090 ceph-mon[31429]: pgmap v515: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 MiB/s wr, 463 op/s 2024-05-27T08:02:25.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:24 smithi136 ceph-mon[35324]: pgmap v515: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 MiB/s wr, 463 op/s 2024-05-27T08:02:25.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:24 smithi129 ceph-mon[35254]: pgmap v515: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.6 MiB/s wr, 463 op/s 2024-05-27T08:02:25.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.634+0000 7f3b09600700 1 -- 172.21.15.90:0/219596485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b04108690 msgr2=0x7f3b04108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:25.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.634+0000 7f3b09600700 1 --2- 172.21.15.90:0/219596485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b04108690 0x7f3b04108b60 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00b0a0 tx=0x7f3afc00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:02:25.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.635+0000 7f3b09600700 1 -- 172.21.15.90:0/219596485 shutdown_connections 2024-05-27T08:02:25.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.635+0000 7f3b09600700 1 --2- 172.21.15.90:0/219596485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b04108690 0x7f3b04108b60 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:25.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.635+0000 7f3b09600700 1 --2- 172.21.15.90:0/219596485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b04103240 0x7f3b04103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:25.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.635+0000 7f3b09600700 1 --2- 172.21.15.90:0/219596485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b04103b90 0x7f3b041080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:25.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.635+0000 7f3b09600700 1 -- 172.21.15.90:0/219596485 >> 172.21.15.90:0/219596485 conn(0x7f3b040fedb0 msgr2=0x7f3b041011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:25.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.635+0000 7f3b09600700 1 -- 172.21.15.90:0/219596485 shutdown_connections 2024-05-27T08:02:25.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.635+0000 7f3b09600700 1 -- 172.21.15.90:0/219596485 wait complete. 2024-05-27T08:02:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.638+0000 7f3b09600700 1 Processor -- start 2024-05-27T08:02:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.638+0000 7f3b09600700 1 -- start start 2024-05-27T08:02:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.638+0000 7f3b09600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b04103240 0x7f3b041a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.638+0000 7f3b09600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b04103b90 0x7f3b041a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.639+0000 7f3b09600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b04108690 0x7f3b041a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.639+0000 7f3b09600700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0411a910 con 0x7f3b04103b90 2024-05-27T08:02:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.639+0000 7f3b09600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0411a790 con 0x7f3b04108690 2024-05-27T08:02:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.639+0000 7f3b09600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0411aa90 con 0x7f3b04103240 2024-05-27T08:02:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.639+0000 7f3b02000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b04103b90 0x7f3b041a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.639+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b04103240 0x7f3b041a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.639+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b04103240 0x7f3b041a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36838/0 (socket says 172.21.15.90:36838) 2024-05-27T08:02:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.640+0000 7f3b02a00700 1 -- 172.21.15.90:0/2858731283 learned_addr learned my addr 172.21.15.90:0/2858731283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.640+0000 7f3b03400700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b04108690 0x7f3b041a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.640+0000 7f3b02000700 1 -- 172.21.15.90:0/2858731283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b04103240 msgr2=0x7f3b041a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.640+0000 7f3b02000700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b04103240 0x7f3b041a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.640+0000 7f3b02000700 1 -- 172.21.15.90:0/2858731283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b04108690 msgr2=0x7f3b041a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.640+0000 7f3b02000700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b04108690 0x7f3b041a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.640+0000 7f3b02000700 1 -- 172.21.15.90:0/2858731283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b041a6180 con 0x7f3b04103b90 2024-05-27T08:02:25.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.641+0000 7f3b02000700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b04103b90 0x7f3b041a18a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3af000e9f0 tx=0x7f3af000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:25.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.641+0000 7f3afb400700 1 -- 172.21.15.90:0/2858731283 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af000c9c0 con 0x7f3b04103b90 2024-05-27T08:02:25.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.642+0000 7f3afb400700 1 -- 172.21.15.90:0/2858731283 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3af0010430 con 0x7f3b04103b90 2024-05-27T08:02:25.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.642+0000 7f3b09600700 1 -- 172.21.15.90:0/2858731283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b041a6410 con 0x7f3b04103b90 2024-05-27T08:02:25.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.642+0000 7f3afb400700 1 -- 172.21.15.90:0/2858731283 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0018680 con 0x7f3b04103b90 2024-05-27T08:02:25.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.642+0000 7f3b09600700 1 -- 172.21.15.90:0/2858731283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b041add90 con 0x7f3b04103b90 2024-05-27T08:02:25.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.649+0000 7f3afb400700 1 -- 172.21.15.90:0/2858731283 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3af000cb60 con 0x7f3b04103b90 2024-05-27T08:02:25.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.651+0000 7f3afb400700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3ae80b2a70 0x7f3ae80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:25.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.651+0000 7f3afb400700 1 -- 172.21.15.90:0/2858731283 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f3af00e29a0 con 0x7f3b04103b90 2024-05-27T08:02:25.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.651+0000 7f3b02a00700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3ae80b2a70 0x7f3ae80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:25.653+0000 7f3b02a00700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3ae80b2a70 0x7f3ae80b4f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005950 tx=0x7f3af400b500 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.037+0000 7f3b180deb80 1 Processor -- start 2024-05-27T08:02:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.037+0000 7f3b180deb80 1 -- start start 2024-05-27T08:02:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.038+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33a1a9f0 0x562c33aad330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.038+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c3392ee40 0x562c33aad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.038+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 0x562c33aba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.039+0000 7f3b180deb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562c339dbe40 con 0x562c3392ee40 2024-05-27T08:02:26.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.039+0000 7f3b180deb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c339cb9e0 con 0x562c33aade70 2024-05-27T08:02:26.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.039+0000 7f3b180deb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562c339cfa20 con 0x562c33a1a9f0 2024-05-27T08:02:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.040+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33a1a9f0 0x562c33aad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.040+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33a1a9f0 0x562c33aad330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36844/0 (socket says 172.21.15.90:36844) 2024-05-27T08:02:26.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.040+0000 7f3b02a00700 1 -- 172.21.15.90:0/3644391999 learned_addr learned my addr 172.21.15.90:0/3644391999 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:26.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.041+0000 7f3b03400700 1 --2- 172.21.15.90:0/3644391999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 0x562c33aba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.041+0000 7f3b02000700 1 --2- 172.21.15.90:0/3644391999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c3392ee40 0x562c33aad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.041+0000 7f3b03400700 1 -- 172.21.15.90:0/3644391999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33a1a9f0 msgr2=0x562c33aad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.041+0000 7f3b03400700 1 --2- 172.21.15.90:0/3644391999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33a1a9f0 0x562c33aad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.041+0000 7f3b03400700 1 -- 172.21.15.90:0/3644391999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c3392ee40 msgr2=0x562c33aad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.041+0000 7f3b03400700 1 --2- 172.21.15.90:0/3644391999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c3392ee40 0x562c33aad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.041+0000 7f3b03400700 1 -- 172.21.15.90:0/3644391999 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c33994420 con 0x562c33aade70 2024-05-27T08:02:26.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.042+0000 7f3b03400700 1 --2- 172.21.15.90:0/3644391999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 0x562c33aba110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009fd0 tx=0x7f3afc0092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7353d4f33c63099 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.042+0000 7f3aefe00700 1 -- 172.21.15.90:0/3644391999 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc020070 con 0x562c33aade70 2024-05-27T08:02:26.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.043+0000 7f3aefe00700 1 -- 172.21.15.90:0/3644391999 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3afc00bcb0 con 0x562c33aade70 2024-05-27T08:02:26.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.043+0000 7f3aefe00700 1 -- 172.21.15.90:0/3644391999 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc007b00 con 0x562c33aade70 2024-05-27T08:02:26.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.044+0000 7f3b180deb80 1 -- 172.21.15.90:0/3644391999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 msgr2=0x562c33aba110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.044+0000 7f3b180deb80 1 --2- 172.21.15.90:0/3644391999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 0x562c33aba110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009fd0 tx=0x7f3afc0092e0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.048+0000 7f3b180deb80 1 -- 172.21.15.90:0/3644391999 shutdown_connections 2024-05-27T08:02:26.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.048+0000 7f3b180deb80 1 --2- 172.21.15.90:0/3644391999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 0x562c33aba110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.048+0000 7f3b180deb80 1 --2- 172.21.15.90:0/3644391999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33a1a9f0 0x562c33aad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.048+0000 7f3b180deb80 1 --2- 172.21.15.90:0/3644391999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c3392ee40 0x562c33aad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.048+0000 7f3b180deb80 1 -- 172.21.15.90:0/3644391999 >> 172.21.15.90:0/3644391999 conn(0x562c339c8390 msgr2=0x562c33aa94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.048+0000 7f3b180deb80 1 -- 172.21.15.90:0/3644391999 shutdown_connections 2024-05-27T08:02:26.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.048+0000 7f3b180deb80 1 -- 172.21.15.90:0/3644391999 wait complete. 2024-05-27T08:02:26.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.049+0000 7f3b180deb80 1 -- start start 2024-05-27T08:02:26.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.049+0000 7f3b180deb80 1 Processor -- start 2024-05-27T08:02:26.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.051+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c3392ee40 0x562c33ab7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.051+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 0x562c33ab7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.051+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33aade70 0x562c33ab80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.051+0000 7f3b180deb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562c338d1fa0 con 0x562c3392ee40 2024-05-27T08:02:26.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.051+0000 7f3b180deb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c3396c760 con 0x562c33a1a9f0 2024-05-27T08:02:26.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.051+0000 7f3b180deb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562c33abab20 con 0x562c33aade70 2024-05-27T08:02:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.051+0000 7f3b180deb80 10 client.0 _ll_register_callbacks cb 0x562c33975c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.052+0000 7f3b02000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 0x562c33ab7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.052+0000 7f3b02000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 0x562c33ab7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47940/0 (socket says 172.21.15.90:47940) 2024-05-27T08:02:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.052+0000 7f3b02000700 1 -- 172.21.15.90:0/3837478248 learned_addr learned my addr 172.21.15.90:0/3837478248 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.052+0000 7f3b03400700 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33aade70 0x562c33ab80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.053+0000 7f3b02a00700 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c3392ee40 0x562c33ab7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.053+0000 7f3b03400700 1 -- 172.21.15.90:0/3837478248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 msgr2=0x562c33ab7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.053+0000 7f3b03400700 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 0x562c33ab7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.053+0000 7f3b03400700 1 -- 172.21.15.90:0/3837478248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c3392ee40 msgr2=0x562c33ab7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.053+0000 7f3b03400700 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c3392ee40 0x562c33ab7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.053+0000 7f3b03400700 1 -- 172.21.15.90:0/3837478248 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c3391b8d0 con 0x562c33aade70 2024-05-27T08:02:26.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.054+0000 7f3b03400700 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33aade70 0x562c33ab80e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00a000 tx=0x7f3afc0077f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1e37944d2d2209b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.055+0000 7f3aecc00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:02:26.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.055+0000 7f3aecc00700 1 -- 172.21.15.90:0/3837478248 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc020040 con 0x562c33aade70 2024-05-27T08:02:26.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.055+0000 7f3aecc00700 1 -- 172.21.15.90:0/3837478248 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f3afc00bab0 con 0x562c33aade70 2024-05-27T08:02:26.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.055+0000 7f3b180deb80 1 -- 172.21.15.90:0/3837478248 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3afc00bcb0 con 0x562c33aade70 2024-05-27T08:02:26.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.056+0000 7f3aecc00700 1 -- 172.21.15.90:0/3837478248 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3afc00be60 con 0x562c33aade70 2024-05-27T08:02:26.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.056+0000 7f3b180deb80 10 client.27835 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:26.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.056+0000 7f3b180deb80 1 -- 172.21.15.90:0/3837478248 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562c3399c060 con 0x562c33aade70 2024-05-27T08:02:26.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.057+0000 7f3aecc00700 1 -- 172.21.15.90:0/3837478248 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3afc00bcb0 con 0x562c33aade70 2024-05-27T08:02:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.057+0000 7f3aecc00700 1 -- 172.21.15.90:0/3837478248 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f3afc011070 con 0x562c33aade70 2024-05-27T08:02:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.057+0000 7f3b180deb80 10 client.27835 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.057+0000 7f3b180deb80 10 client.27835 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T08:02:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.057+0000 7f3b180deb80 20 client.27835 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.058+0000 7f3b180deb80 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562c33ac7710 0x562c33ac9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.058+0000 7f3b180deb80 4 client.27835 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:26.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.058+0000 7f3b180deb80 1 -- 172.21.15.90:0/3837478248 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c339ddd90 con 0x562c33ac7710 2024-05-27T08:02:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.058+0000 7f3b02a00700 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562c33ac7710 0x562c33ac9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.059+0000 7f3aecc00700 10 client.27835 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T08:02:26.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.059+0000 7f3b02a00700 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562c33ac7710 0x562c33ac9b90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=388207de5f9ddf14 server_cookie=26860865390ac702 in_seq=0 out_seq=0 2024-05-27T08:02:26.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.060+0000 7f3aecc00700 1 -- 172.21.15.90:0/3837478248 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562c339ddd90 con 0x562c33ac7710 2024-05-27T08:02:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.060+0000 7f3aecc00700 10 client.27835 handle_command_reply: tid=0 2024-05-27T08:02:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.060+0000 7f3b180deb80 1 client.27835 shutdown 2024-05-27T08:02:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.061+0000 7f3b180deb80 1 -- 172.21.15.90:0/3837478248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33aade70 msgr2=0x562c33ab80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.061+0000 7f3b180deb80 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33aade70 0x562c33ab80e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00a000 tx=0x7f3afc0077f0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.061+0000 7f3b180deb80 1 -- 172.21.15.90:0/3837478248 shutdown_connections 2024-05-27T08:02:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.061+0000 7f3b180deb80 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562c33ac7710 0x562c33ac9b90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.061+0000 7f3b180deb80 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c33aade70 0x562c33ab80e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.061+0000 7f3b180deb80 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c3392ee40 0x562c33ab7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.061+0000 7f3b180deb80 1 --2- 172.21.15.90:0/3837478248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 0x562c33ab7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.061+0000 7f3b180deb80 1 -- 172.21.15.90:0/3837478248 >> 172.21.15.90:0/3837478248 conn(0x562c339c8390 msgr2=0x562c33aad310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.062+0000 7f3aecc00700 0 client.27835 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T08:02:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.062+0000 7f3b180deb80 1 -- 172.21.15.90:0/3837478248 shutdown_connections 2024-05-27T08:02:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.062+0000 7f3b180deb80 1 -- 172.21.15.90:0/3837478248 wait complete. 2024-05-27T08:02:26.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.062+0000 7f3b180deb80 20 client.27835 trim_cache size 0 max 16384 2024-05-27T08:02:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.092+0000 7f3b180deb80 1 Processor -- start 2024-05-27T08:02:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.092+0000 7f3b180deb80 1 -- start start 2024-05-27T08:02:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.093+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33ab2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.093+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33a1a9f0 0x562c33ac3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.093+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 0x562c33ab6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.093+0000 7f3b180deb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041b85f0 con 0x562c33a1a9f0 2024-05-27T08:02:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.093+0000 7f3b180deb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041b5640 con 0x562c33aade70 2024-05-27T08:02:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.093+0000 7f3b180deb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041b7850 con 0x562c3392ee40 2024-05-27T08:02:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.094+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33ab2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.094+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33ab2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36874/0 (socket says 172.21.15.90:36874) 2024-05-27T08:02:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.094+0000 7f3b02a00700 1 -- 172.21.15.90:0/1470624903 learned_addr learned my addr 172.21.15.90:0/1470624903 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.094+0000 7f3b02000700 1 --2- 172.21.15.90:0/1470624903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33a1a9f0 0x562c33ac3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.094+0000 7f3b03400700 1 --2- 172.21.15.90:0/1470624903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 0x562c33ab6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.095+0000 7f3b03400700 1 -- 172.21.15.90:0/1470624903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 msgr2=0x562c33ab2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.095+0000 7f3b03400700 1 --2- 172.21.15.90:0/1470624903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33ab2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.095+0000 7f3b03400700 1 -- 172.21.15.90:0/1470624903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33a1a9f0 msgr2=0x562c33ac3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.095+0000 7f3b03400700 1 --2- 172.21.15.90:0/1470624903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33a1a9f0 0x562c33ac3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.095+0000 7f3b03400700 1 -- 172.21.15.90:0/1470624903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c33ab5bd0 con 0x562c33aade70 2024-05-27T08:02:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.095+0000 7f3b02000700 1 --2- 172.21.15.90:0/1470624903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33a1a9f0 0x562c33ac3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:26.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.096+0000 7f3b03400700 1 --2- 172.21.15.90:0/1470624903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 0x562c33ab6530 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3afc01a480 tx=0x7f3afc00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a834401d5f9c4f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.097+0000 7f3aef400700 1 -- 172.21.15.90:0/1470624903 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc020040 con 0x562c33aade70 2024-05-27T08:02:26.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.097+0000 7f3aef400700 1 -- 172.21.15.90:0/1470624903 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3afc009780 con 0x562c33aade70 2024-05-27T08:02:26.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.097+0000 7f3aef400700 1 -- 172.21.15.90:0/1470624903 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc005c60 con 0x562c33aade70 2024-05-27T08:02:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.098+0000 7f3b180deb80 1 -- 172.21.15.90:0/1470624903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 msgr2=0x562c33ab6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.098+0000 7f3b180deb80 1 --2- 172.21.15.90:0/1470624903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 0x562c33ab6530 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3afc01a480 tx=0x7f3afc00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.099+0000 7f3b180deb80 1 -- 172.21.15.90:0/1470624903 shutdown_connections 2024-05-27T08:02:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.099+0000 7f3b180deb80 1 --2- 172.21.15.90:0/1470624903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33aade70 0x562c33ab6530 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.099+0000 7f3b180deb80 1 --2- 172.21.15.90:0/1470624903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33ab2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.099+0000 7f3b180deb80 1 --2- 172.21.15.90:0/1470624903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33a1a9f0 0x562c33ac3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.099+0000 7f3b180deb80 1 -- 172.21.15.90:0/1470624903 >> 172.21.15.90:0/1470624903 conn(0x562c339c8390 msgr2=0x562c33ab5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.101+0000 7f3b180deb80 1 -- 172.21.15.90:0/1470624903 shutdown_connections 2024-05-27T08:02:26.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.101+0000 7f3b180deb80 1 -- 172.21.15.90:0/1470624903 wait complete. 2024-05-27T08:02:26.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.102+0000 7f3b180deb80 1 -- start start 2024-05-27T08:02:26.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.102+0000 7f3b180deb80 1 Processor -- start 2024-05-27T08:02:26.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.104+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33accd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.104+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 0x562c33ac7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.105+0000 7f3b180deb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33aade70 0x562c33ac7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.105+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33accd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.106+0000 7f3b02a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33accd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36890/0 (socket says 172.21.15.90:36890) 2024-05-27T08:02:26.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.106+0000 7f3b02a00700 1 -- 172.21.15.90:0/698522014 learned_addr learned my addr 172.21.15.90:0/698522014 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:26.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.106+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041b7180 con 0x562c33aade70 2024-05-27T08:02:26.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.106+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041b5d10 con 0x562c33a1a9f0 2024-05-27T08:02:26.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.106+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3b041b63e0 con 0x562c3392ee40 2024-05-27T08:02:26.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.106+0000 7f3b180deb80 10 client.0 _ll_register_callbacks cb 0x562c33975c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:26.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.106+0000 7f3b03400700 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33aade70 0x562c33ac7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.106+0000 7f3b02000700 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 0x562c33ac7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.107+0000 7f3b03400700 1 -- 172.21.15.90:0/698522014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 msgr2=0x562c33accd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.107+0000 7f3b03400700 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33accd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.107+0000 7f3b03400700 1 -- 172.21.15.90:0/698522014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 msgr2=0x562c33ac7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:26.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.108+0000 7f3b03400700 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 0x562c33ac7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.108+0000 7f3b03400700 1 -- 172.21.15.90:0/698522014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c33ac82f0 con 0x562c33aade70 2024-05-27T08:02:26.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.108+0000 7f3b02a00700 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33accd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:26.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.108+0000 7f3b02000700 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 0x562c33ac7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:26.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.109+0000 7f3b03400700 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33aade70 0x562c33ac7c50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3afc01a480 tx=0x7f3afc0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b884eaba5a8c13df server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:26.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.109+0000 7f3aecc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:26.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.110+0000 7f3aecc00700 1 -- 172.21.15.90:0/698522014 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc020070 con 0x562c33aade70 2024-05-27T08:02:26.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.110+0000 7f3aecc00700 1 -- 172.21.15.90:0/698522014 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f3afc036810 con 0x562c33aade70 2024-05-27T08:02:26.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.110+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3afc009780 con 0x562c33aade70 2024-05-27T08:02:26.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.111+0000 7f3aecc00700 1 -- 172.21.15.90:0/698522014 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3afc036a10 con 0x562c33aade70 2024-05-27T08:02:26.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.111+0000 7f3b180deb80 10 client.18081 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:26.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.111+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562c33b30ee0 con 0x562c33aade70 2024-05-27T08:02:26.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.112+0000 7f3aecc00700 1 -- 172.21.15.90:0/698522014 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3afc009780 con 0x562c33aade70 2024-05-27T08:02:26.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.112+0000 7f3aecc00700 1 -- 172.21.15.90:0/698522014 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f3afc011040 con 0x562c33aade70 2024-05-27T08:02:26.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.113+0000 7f3b180deb80 10 client.18081 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:26.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.113+0000 7f3b180deb80 10 client.18081 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T08:02:26.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.113+0000 7f3b180deb80 20 client.18081 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:26.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.113+0000 7f3b180deb80 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562c33b31480 0x562c33b338a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:26.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.113+0000 7f3b180deb80 4 client.18081 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:02:26.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.113+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c339d7570 con 0x562c33b31480 2024-05-27T08:02:26.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.114+0000 7f3b02a00700 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562c33b31480 0x562c33b338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:26.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.115+0000 7f3b02a00700 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562c33b31480 0x562c33b338a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e38b4af10da0b834 server_cookie=13477467a338f1d8 in_seq=0 out_seq=0 2024-05-27T08:02:26.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.115+0000 7f3aecc00700 10 client.18081 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T08:02:26.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.116+0000 7f3aecc00700 1 -- 172.21.15.90:0/698522014 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c339d7570 con 0x562c33b31480 2024-05-27T08:02:26.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.116+0000 7f3aecc00700 10 client.18081 handle_command_reply: tid=0 2024-05-27T08:02:26.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.118+0000 7f3b180deb80 1 client.18081 shutdown 2024-05-27T08:02:26.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.119+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33aade70 msgr2=0x562c33ac7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.119+0000 7f3b180deb80 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33aade70 0x562c33ac7c50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3afc01a480 tx=0x7f3afc0077d0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.119+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 shutdown_connections 2024-05-27T08:02:26.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.119+0000 7f3b180deb80 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x562c33b31480 0x562c33b338a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.119+0000 7f3b180deb80 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562c33aade70 0x562c33ac7c50 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.119+0000 7f3b180deb80 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c3392ee40 0x562c33accd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.119+0000 7f3b180deb80 1 --2- 172.21.15.90:0/698522014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c33a1a9f0 0x562c33ac7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.119+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 >> 172.21.15.90:0/698522014 conn(0x562c339c8390 msgr2=0x562c33ab8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.119+0000 7f3aecc00700 0 client.18081 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T08:02:26.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.119+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 shutdown_connections 2024-05-27T08:02:26.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.120+0000 7f3b180deb80 1 -- 172.21.15.90:0/698522014 wait complete. 2024-05-27T08:02:26.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.121+0000 7f3b180deb80 20 client.18081 trim_cache size 0 max 16384 2024-05-27T08:02:26.124 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:02:26.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.123+0000 7f3b09600700 1 -- 172.21.15.90:0/2858731283 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3ae80b2a70 msgr2=0x7f3ae80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.123+0000 7f3b09600700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3ae80b2a70 0x7f3ae80b4f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005950 tx=0x7f3af400b500 comp rx=0 tx=0).stop 2024-05-27T08:02:26.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.123+0000 7f3b09600700 1 -- 172.21.15.90:0/2858731283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b04103b90 msgr2=0x7f3b041a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:26.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.123+0000 7f3b09600700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b04103b90 0x7f3b041a18a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3af000e9f0 tx=0x7f3af000ed00 comp rx=0 tx=0).stop 2024-05-27T08:02:26.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.123+0000 7f3b09600700 1 -- 172.21.15.90:0/2858731283 shutdown_connections 2024-05-27T08:02:26.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.123+0000 7f3b09600700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3ae80b2a70 0x7f3ae80b4f80 secure :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005950 tx=0x7f3af400b500 comp rx=0 tx=0).stop 2024-05-27T08:02:26.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.123+0000 7f3b09600700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b04108690 0x7f3b041a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.123+0000 7f3b09600700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b04103240 0x7f3b041a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.123+0000 7f3b09600700 1 --2- 172.21.15.90:0/2858731283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b04103b90 0x7f3b041a18a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:26.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.123+0000 7f3b09600700 1 -- 172.21.15.90:0/2858731283 >> 172.21.15.90:0/2858731283 conn(0x7f3b040fedb0 msgr2=0x7f3b04105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:26.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.124+0000 7f3b09600700 1 -- 172.21.15.90:0/2858731283 shutdown_connections 2024-05-27T08:02:26.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:26.124+0000 7f3b09600700 1 -- 172.21.15.90:0/2858731283 wait complete. 2024-05-27T08:02:26.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:26 smithi090 ceph-mon[31429]: pgmap v516: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.9 MiB/s wr, 553 op/s 2024-05-27T08:02:27.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:26 smithi136 ceph-mon[35324]: pgmap v516: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.9 MiB/s wr, 553 op/s 2024-05-27T08:02:27.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:26 smithi129 ceph-mon[35254]: pgmap v516: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.9 MiB/s wr, 553 op/s 2024-05-27T08:02:27.235 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T08:02:28.403 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 730.0 s, 611.8 tps, lat 167.800 ms stddev 125.130 2024-05-27T08:02:28.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:28 smithi090 ceph-mon[31429]: pgmap v517: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.9 MiB/s wr, 561 op/s 2024-05-27T08:02:29.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:28 smithi136 ceph-mon[35324]: pgmap v517: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.9 MiB/s wr, 561 op/s 2024-05-27T08:02:29.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:28 smithi129 ceph-mon[35254]: pgmap v517: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 4.9 MiB/s wr, 561 op/s 2024-05-27T08:02:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.872+0000 7f63ebc00640 1 -- 172.21.15.90:0/808714497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63e410f540 msgr2=0x7f63e4116ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.872+0000 7f63ebc00640 1 --2- 172.21.15.90:0/808714497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63e410f540 0x7f63e4116ce0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009ca0 tx=0x7f63dc02a8b0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.872+0000 7f63ebc00640 1 -- 172.21.15.90:0/808714497 shutdown_connections 2024-05-27T08:02:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.872+0000 7f63ebc00640 1 --2- 172.21.15.90:0/808714497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63e410f540 0x7f63e4116ce0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.872+0000 7f63ebc00640 1 --2- 172.21.15.90:0/808714497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63e410ea80 0x7f63e410eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.872+0000 7f63ebc00640 1 --2- 172.21.15.90:0/808714497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e410a500 0x7f63e410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.872+0000 7f63ebc00640 1 -- 172.21.15.90:0/808714497 >> 172.21.15.90:0/808714497 conn(0x7f63e4077190 msgr2=0x7f63e40775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.872+0000 7f63ebc00640 1 -- 172.21.15.90:0/808714497 shutdown_connections 2024-05-27T08:02:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.872+0000 7f63ebc00640 1 -- 172.21.15.90:0/808714497 wait complete. 2024-05-27T08:02:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.874+0000 7f63ebc00640 1 Processor -- start 2024-05-27T08:02:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.874+0000 7f63ebc00640 1 -- start start 2024-05-27T08:02:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.874+0000 7f63ebc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63e410a500 0x7f63e41a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.875+0000 7f63ebc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63e410ea80 0x7f63e41a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.875+0000 7f63ebc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e410f540 0x7f63e41a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.875+0000 7f63ebc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f63e411a3b0 con 0x7f63e410a500 2024-05-27T08:02:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.875+0000 7f63ebc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63e411a230 con 0x7f63e410ea80 2024-05-27T08:02:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.875+0000 7f63ebc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63e411a530 con 0x7f63e410f540 2024-05-27T08:02:29.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.875+0000 7f63e9200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63e410a500 0x7f63e41a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.875+0000 7f63e9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e410f540 0x7f63e41a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.875+0000 7f63e9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e410f540 0x7f63e41a5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:45498/0 (socket says 172.21.15.90:45498) 2024-05-27T08:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.875+0000 7f63e9c00640 1 -- 172.21.15.90:0/1333513072 learned_addr learned my addr 172.21.15.90:0/1333513072 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.876+0000 7f63e3e00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63e410ea80 0x7f63e41a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.876+0000 7f63e9c00640 1 -- 172.21.15.90:0/1333513072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63e410ea80 msgr2=0x7f63e41a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.876+0000 7f63e9c00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63e410ea80 0x7f63e41a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.876+0000 7f63e9c00640 1 -- 172.21.15.90:0/1333513072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63e410a500 msgr2=0x7f63e41a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.876+0000 7f63e9c00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63e410a500 0x7f63e41a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.876+0000 7f63e9c00640 1 -- 172.21.15.90:0/1333513072 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e41a5cb0 con 0x7f63e410f540 2024-05-27T08:02:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.876+0000 7f63e9200640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63e410a500 0x7f63e41a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.878+0000 7f63e9c00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e410f540 0x7f63e41a5670 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f63dc030d50 tx=0x7f63dc030ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.879+0000 7f63e1600640 1 -- 172.21.15.90:0/1333513072 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc004040 con 0x7f63e410f540 2024-05-27T08:02:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.879+0000 7f63ebc00640 1 -- 172.21.15.90:0/1333513072 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e41a5f40 con 0x7f63e410f540 2024-05-27T08:02:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.879+0000 7f63e1600640 1 -- 172.21.15.90:0/1333513072 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63dc040070 con 0x7f63e410f540 2024-05-27T08:02:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.879+0000 7f63ebc00640 1 -- 172.21.15.90:0/1333513072 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e41a6490 con 0x7f63e410f540 2024-05-27T08:02:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.879+0000 7f63e1600640 1 -- 172.21.15.90:0/1333513072 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc045a30 con 0x7f63e410f540 2024-05-27T08:02:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.892+0000 7f63e1600640 1 -- 172.21.15.90:0/1333513072 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f63dc0041e0 con 0x7f63e410f540 2024-05-27T08:02:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.893+0000 7f63f2bce740 1 Processor -- start 2024-05-27T08:02:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.893+0000 7f63f2bce740 1 -- start start 2024-05-27T08:02:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.894+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 0x55fe41782860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.894+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe41782da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.894+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 0x55fe417832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.894+0000 7f63f2bce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fe416f3b90 con 0x55fe413e01b0 2024-05-27T08:02:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.894+0000 7f63f2bce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fe416e3730 con 0x55fe413b72f0 2024-05-27T08:02:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.894+0000 7f63f2bce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fe416e7770 con 0x55fe413bf4d0 2024-05-27T08:02:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.894+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 0x55fe41782860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e1600640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63c80b2950 0x7f63c80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 0x55fe41782860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44684/0 (socket says 172.21.15.90:44684) 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e3e00640 1 -- 172.21.15.90:0/3670572144 learned_addr learned my addr 172.21.15.90:0/3670572144 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e3e00640 1 --2- 172.21.15.90:0/3670572144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 0x55fe417832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e1600640 1 -- 172.21.15.90:0/1333513072 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f63dc1041f0 con 0x7f63e410f540 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e9c00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63c80b2950 0x7f63c80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e9200640 1 --2- 172.21.15.90:0/3670572144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe41782da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e3e00640 1 -- 172.21.15.90:0/3670572144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 msgr2=0x55fe417832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e3e00640 1 --2- 172.21.15.90:0/3670572144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 0x55fe417832e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e3e00640 1 -- 172.21.15.90:0/3670572144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 msgr2=0x55fe41782da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e3e00640 1 --2- 172.21.15.90:0/3670572144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe41782da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.895+0000 7f63e3e00640 1 -- 172.21.15.90:0/3670572144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe4167c8b0 con 0x55fe413e01b0 2024-05-27T08:02:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.896+0000 7f63e3e00640 1 --2- 172.21.15.90:0/3670572144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 0x55fe41782860 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f63d00136c0 tx=0x7f63d00096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4deea16da714ba server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.897+0000 7f63da000640 1 -- 172.21.15.90:0/3670572144 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d00110d0 con 0x55fe413e01b0 2024-05-27T08:02:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.897+0000 7f63e9c00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63c80b2950 0x7f63c80b4e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0cb240 tx=0x7f63dc02a8b0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.897+0000 7f63da000640 1 -- 172.21.15.90:0/3670572144 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63d0013c40 con 0x55fe413e01b0 2024-05-27T08:02:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.897+0000 7f63da000640 1 -- 172.21.15.90:0/3670572144 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d000f6c0 con 0x55fe413e01b0 2024-05-27T08:02:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.898+0000 7f63f2bce740 1 -- 172.21.15.90:0/3670572144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 msgr2=0x55fe41782860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.898+0000 7f63f2bce740 1 --2- 172.21.15.90:0/3670572144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 0x55fe41782860 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f63d00136c0 tx=0x7f63d00096a0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.899+0000 7f63f2bce740 1 -- 172.21.15.90:0/3670572144 shutdown_connections 2024-05-27T08:02:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.899+0000 7f63f2bce740 1 --2- 172.21.15.90:0/3670572144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe41782da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.899+0000 7f63f2bce740 1 --2- 172.21.15.90:0/3670572144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 0x55fe417832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.899+0000 7f63f2bce740 1 --2- 172.21.15.90:0/3670572144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 0x55fe41782860 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.899+0000 7f63f2bce740 1 -- 172.21.15.90:0/3670572144 >> 172.21.15.90:0/3670572144 conn(0x55fe413d4300 msgr2=0x55fe4177f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.901+0000 7f63f2bce740 1 -- 172.21.15.90:0/3670572144 shutdown_connections 2024-05-27T08:02:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.901+0000 7f63f2bce740 1 -- 172.21.15.90:0/3670572144 wait complete. 2024-05-27T08:02:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.902+0000 7f63f2bce740 1 -- start start 2024-05-27T08:02:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.904+0000 7f63f2bce740 1 Processor -- start 2024-05-27T08:02:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.905+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe4178cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.905+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 0x55fe4178d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.905+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 0x55fe41798c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.905+0000 7f63f2bce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fe413c9790 con 0x55fe413e01b0 2024-05-27T08:02:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.905+0000 7f63f2bce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fe4168bbc0 con 0x55fe413b72f0 2024-05-27T08:02:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.905+0000 7f63f2bce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fe4178f930 con 0x55fe413bf4d0 2024-05-27T08:02:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.905+0000 7f63f2bce740 10 client.0 _ll_register_callbacks cb 0x55fe41669400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.906+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe4178cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.906+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe4178cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:55768/0 (socket says 172.21.15.90:55768) 2024-05-27T08:02:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.906+0000 7f63e3e00640 1 -- 172.21.15.90:0/371845531 learned_addr learned my addr 172.21.15.90:0/371845531 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.906+0000 7f63e3e00640 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 0x55fe41798c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.906+0000 7f63e9200640 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 0x55fe4178d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.907+0000 7f63e3e00640 1 -- 172.21.15.90:0/371845531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 msgr2=0x55fe4178d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.907+0000 7f63e3e00640 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 0x55fe4178d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.907+0000 7f63e3e00640 1 -- 172.21.15.90:0/371845531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 msgr2=0x55fe41798c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.907+0000 7f63e3e00640 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 0x55fe41798c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.907+0000 7f63e3e00640 1 -- 172.21.15.90:0/371845531 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe413f8210 con 0x55fe413b72f0 2024-05-27T08:02:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.907+0000 7f63e9200640 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 0x55fe4178d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.907+0000 7f63e3e00640 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe4178cdd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63d0011f60 tx=0x7f63d0018040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b671a0a662106bf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.907+0000 7f63cea00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:02:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.908+0000 7f63cea00640 1 -- 172.21.15.90:0/371845531 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0011d30 con 0x55fe413b72f0 2024-05-27T08:02:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.908+0000 7f63f2bce740 1 -- 172.21.15.90:0/371845531 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d0013c40 con 0x55fe413b72f0 2024-05-27T08:02:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.908+0000 7f63cea00640 1 -- 172.21.15.90:0/371845531 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f63d0026030 con 0x55fe413b72f0 2024-05-27T08:02:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.908+0000 7f63cea00640 1 -- 172.21.15.90:0/371845531 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f63d0033050 con 0x55fe413b72f0 2024-05-27T08:02:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.908+0000 7f63f2bce740 10 client.37769 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.908+0000 7f63f2bce740 1 -- 172.21.15.90:0/371845531 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55fe414c4620 con 0x55fe413b72f0 2024-05-27T08:02:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.909+0000 7f63cea00640 1 -- 172.21.15.90:0/371845531 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63d0013c40 con 0x55fe413b72f0 2024-05-27T08:02:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.910+0000 7f63cea00640 1 -- 172.21.15.90:0/371845531 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f63d000f480 con 0x55fe413b72f0 2024-05-27T08:02:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.910+0000 7f63f2bce740 10 client.37769 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.910+0000 7f63f2bce740 10 client.37769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.910+0000 7f63f2bce740 20 client.37769 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.910+0000 7f63f2bce740 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55fe4139ed80 0x55fe417afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.910+0000 7f63f2bce740 4 client.37769 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.910+0000 7f63f2bce740 1 -- 172.21.15.90:0/371845531 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe416f5ae0 con 0x55fe4139ed80 2024-05-27T08:02:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.911+0000 7f63e9200640 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55fe4139ed80 0x55fe417afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.912+0000 7f63e9200640 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55fe4139ed80 0x55fe417afe30 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c398724d8f0bb6 server_cookie=99f982d730168247 in_seq=0 out_seq=0 2024-05-27T08:02:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.912+0000 7f63cea00640 10 client.37769 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63cea00640 1 -- 172.21.15.90:0/371845531 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fe416f5ae0 con 0x55fe4139ed80 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63cea00640 10 client.37769 handle_command_reply: tid=0 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 client.37769 shutdown 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 -- 172.21.15.90:0/371845531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 msgr2=0x55fe4178cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe4178cdd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63d0011f60 tx=0x7f63d0018040 comp rx=0 tx=0).stop 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 -- 172.21.15.90:0/371845531 shutdown_connections 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55fe4139ed80 0x55fe417afe30 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413e01b0 0x55fe41798c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413bf4d0 0x55fe4178d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 --2- 172.21.15.90:0/371845531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe4178cdd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63cea00640 0 client.37769 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 -- 172.21.15.90:0/371845531 >> 172.21.15.90:0/371845531 conn(0x55fe413d4300 msgr2=0x55fe41782690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 -- 172.21.15.90:0/371845531 shutdown_connections 2024-05-27T08:02:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 1 -- 172.21.15.90:0/371845531 wait complete. 2024-05-27T08:02:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.914+0000 7f63f2bce740 20 client.37769 trim_cache size 0 max 16384 2024-05-27T08:02:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.930+0000 7f63f2bce740 1 Processor -- start 2024-05-27T08:02:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.930+0000 7f63f2bce740 1 -- start start 2024-05-27T08:02:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe417883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413bf4d0 0x55fe4177e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413e01b0 0x55fe417989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63f2bce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41b0a20 con 0x55fe413bf4d0 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63f2bce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41ada70 con 0x55fe413b72f0 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63f2bce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41afc80 con 0x55fe413e01b0 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63e9200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413bf4d0 0x55fe4177e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63e9200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413bf4d0 0x55fe4177e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44708/0 (socket says 172.21.15.90:44708) 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63e9200640 1 -- 172.21.15.90:0/1555650647 learned_addr learned my addr 172.21.15.90:0/1555650647 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63e3e00640 1 --2- 172.21.15.90:0/1555650647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe417883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63e9200640 1 -- 172.21.15.90:0/1555650647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413e01b0 msgr2=0x55fe417989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63e9200640 1 --2- 172.21.15.90:0/1555650647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413e01b0 0x55fe417989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63e9200640 1 -- 172.21.15.90:0/1555650647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 msgr2=0x55fe417883c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63e9200640 1 --2- 172.21.15.90:0/1555650647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe417883c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.931+0000 7f63e9200640 1 -- 172.21.15.90:0/1555650647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe4178b110 con 0x55fe413bf4d0 2024-05-27T08:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.932+0000 7f63e9200640 1 --2- 172.21.15.90:0/1555650647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413bf4d0 0x55fe4177e330 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f63d400bde0 tx=0x7f63d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc24b3e72ae2d79 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.932+0000 7f63d9600640 1 -- 172.21.15.90:0/1555650647 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4016db0 con 0x55fe413bf4d0 2024-05-27T08:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.932+0000 7f63d9600640 1 -- 172.21.15.90:0/1555650647 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63d4020030 con 0x55fe413bf4d0 2024-05-27T08:02:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.932+0000 7f63d9600640 1 -- 172.21.15.90:0/1555650647 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d40244c0 con 0x55fe413bf4d0 2024-05-27T08:02:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.933+0000 7f63f2bce740 1 -- 172.21.15.90:0/1555650647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413bf4d0 msgr2=0x55fe4177e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.933+0000 7f63f2bce740 1 --2- 172.21.15.90:0/1555650647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413bf4d0 0x55fe4177e330 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f63d400bde0 tx=0x7f63d4008040 comp rx=0 tx=0).stop 2024-05-27T08:02:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.933+0000 7f63f2bce740 1 -- 172.21.15.90:0/1555650647 shutdown_connections 2024-05-27T08:02:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.933+0000 7f63f2bce740 1 --2- 172.21.15.90:0/1555650647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413e01b0 0x55fe417989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.933+0000 7f63f2bce740 1 --2- 172.21.15.90:0/1555650647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413bf4d0 0x55fe4177e330 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.933+0000 7f63f2bce740 1 --2- 172.21.15.90:0/1555650647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413b72f0 0x55fe417883c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.933+0000 7f63f2bce740 1 -- 172.21.15.90:0/1555650647 >> 172.21.15.90:0/1555650647 conn(0x55fe413d4300 msgr2=0x55fe41783020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.933+0000 7f63f2bce740 1 -- 172.21.15.90:0/1555650647 shutdown_connections 2024-05-27T08:02:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.933+0000 7f63f2bce740 1 -- 172.21.15.90:0/1555650647 wait complete. 2024-05-27T08:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.933+0000 7f63f2bce740 1 -- start start 2024-05-27T08:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63f2bce740 1 Processor -- start 2024-05-27T08:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413b72f0 0x55fe417880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413bf4d0 0x55fe41786430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63f2bce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413e01b0 0x55fe41786970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63f2bce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41af5b0 con 0x55fe413b72f0 2024-05-27T08:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63f2bce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41ae140 con 0x55fe413bf4d0 2024-05-27T08:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63f2bce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63e41ae810 con 0x55fe413e01b0 2024-05-27T08:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63f2bce740 10 client.0 _ll_register_callbacks cb 0x7f63d0009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413b72f0 0x55fe417880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413b72f0 0x55fe417880b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44716/0 (socket says 172.21.15.90:44716) 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.934+0000 7f63e3e00640 1 -- 172.21.15.90:0/3422533979 learned_addr learned my addr 172.21.15.90:0/3422533979 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63e9200640 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413bf4d0 0x55fe41786430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63e3e00640 1 -- 172.21.15.90:0/3422533979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413e01b0 msgr2=0x55fe41786970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63e3e00640 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413e01b0 0x55fe41786970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63e3e00640 1 -- 172.21.15.90:0/3422533979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413bf4d0 msgr2=0x55fe41786430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63e3e00640 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413bf4d0 0x55fe41786430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63e3e00640 1 -- 172.21.15.90:0/3422533979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe41786ee0 con 0x55fe413b72f0 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63e3e00640 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413b72f0 0x55fe417880b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f63d002c6c0 tx=0x7f63d0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d885682cdae6ad0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63cea00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63cea00640 1 -- 172.21.15.90:0/3422533979 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d000f690 con 0x55fe413b72f0 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63cea00640 1 -- 172.21.15.90:0/3422533979 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f63d002a040 con 0x55fe413b72f0 2024-05-27T08:02:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.935+0000 7f63f2bce740 1 -- 172.21.15.90:0/3422533979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d4020030 con 0x55fe413b72f0 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63cea00640 1 -- 172.21.15.90:0/3422533979 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f63d003e050 con 0x55fe413b72f0 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63f2bce740 10 client.18099 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63f2bce740 1 -- 172.21.15.90:0/3422533979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55fe417948e0 con 0x55fe413b72f0 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63cea00640 1 -- 172.21.15.90:0/3422533979 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63d4020030 con 0x55fe413b72f0 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63cea00640 1 -- 172.21.15.90:0/3422533979 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f63d000fa70 con 0x55fe413b72f0 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63f2bce740 10 client.18099 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63f2bce740 10 client.18099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63f2bce740 20 client.18099 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63f2bce740 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55fe413741c0 0x55fe41795010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63f2bce740 4 client.18099 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.936+0000 7f63f2bce740 1 -- 172.21.15.90:0/3422533979 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe416ef2c0 con 0x55fe413741c0 2024-05-27T08:02:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.937+0000 7f63e9200640 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55fe413741c0 0x55fe41795010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.937+0000 7f63cea00640 10 client.18099 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:29.937+0000 7f63e9200640 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55fe413741c0 0x55fe41795010 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9590b72fc994e8d6 server_cookie=875024691fce97b1 in_seq=0 out_seq=0 2024-05-27T08:02:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.337+0000 7f63cea00640 1 -- 172.21.15.90:0/3422533979 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fe416ef2c0 con 0x55fe413741c0 2024-05-27T08:02:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.337+0000 7f63cea00640 10 client.18099 handle_command_reply: tid=0 2024-05-27T08:02:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.337+0000 7f63f2bce740 1 client.18099 shutdown 2024-05-27T08:02:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.338+0000 7f63f2bce740 1 -- 172.21.15.90:0/3422533979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413b72f0 msgr2=0x55fe417880b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.338+0000 7f63f2bce740 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413b72f0 0x55fe417880b0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f63d002c6c0 tx=0x7f63d0009c30 comp rx=0 tx=0).stop 2024-05-27T08:02:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.338+0000 7f63f2bce740 1 -- 172.21.15.90:0/3422533979 shutdown_connections 2024-05-27T08:02:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.338+0000 7f63f2bce740 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55fe413741c0 0x55fe41795010 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.338+0000 7f63f2bce740 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe413bf4d0 0x55fe41786430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.338+0000 7f63f2bce740 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fe413e01b0 0x55fe41786970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.338+0000 7f63f2bce740 1 --2- 172.21.15.90:0/3422533979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fe413b72f0 0x55fe417880b0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.338+0000 7f63f2bce740 1 -- 172.21.15.90:0/3422533979 >> 172.21.15.90:0/3422533979 conn(0x55fe413d4300 msgr2=0x55fe41783020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.338+0000 7f63cea00640 0 client.18099 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.339+0000 7f63f2bce740 1 -- 172.21.15.90:0/3422533979 shutdown_connections 2024-05-27T08:02:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.340+0000 7f63f2bce740 1 -- 172.21.15.90:0/3422533979 wait complete. 2024-05-27T08:02:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.340+0000 7f63f2bce740 20 client.18099 trim_cache size 0 max 16384 2024-05-27T08:02:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:02:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:02:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:02:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.341+0000 7f63ebc00640 1 -- 172.21.15.90:0/1333513072 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63c80b2950 msgr2=0x7f63c80b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63c80b2950 0x7f63c80b4e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0cb240 tx=0x7f63dc02a8b0 comp rx=0 tx=0).stop 2024-05-27T08:02:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 -- 172.21.15.90:0/1333513072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e410f540 msgr2=0x7f63e41a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e410f540 0x7f63e41a5670 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f63dc030d50 tx=0x7f63dc030ff0 comp rx=0 tx=0).stop 2024-05-27T08:02:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 -- 172.21.15.90:0/1333513072 shutdown_connections 2024-05-27T08:02:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f63c80b2950 0x7f63c80b4e60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63e410ea80 0x7f63e41a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e410f540 0x7f63e41a5670 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 --2- 172.21.15.90:0/1333513072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63e410a500 0x7f63e41a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 -- 172.21.15.90:0/1333513072 >> 172.21.15.90:0/1333513072 conn(0x7f63e4077190 msgr2=0x7f63e407a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 -- 172.21.15.90:0/1333513072 shutdown_connections 2024-05-27T08:02:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:30.342+0000 7f63ebc00640 1 -- 172.21.15.90:0/1333513072 wait complete. 2024-05-27T08:02:30.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:30 smithi090 ceph-mon[31429]: pgmap v518: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.9 MiB/s wr, 654 op/s 2024-05-27T08:02:31.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:30 smithi136 ceph-mon[35324]: pgmap v518: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.9 MiB/s wr, 654 op/s 2024-05-27T08:02:31.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:30 smithi129 ceph-mon[35254]: pgmap v518: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.9 MiB/s wr, 654 op/s 2024-05-27T08:02:32.967 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:32 smithi090 ceph-mon[31429]: pgmap v519: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-05-27T08:02:33.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:32 smithi136 ceph-mon[35324]: pgmap v519: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-05-27T08:02:33.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:32 smithi129 ceph-mon[35254]: pgmap v519: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 MiB/s wr, 491 op/s 2024-05-27T08:02:33.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.695+0000 7fd87a000700 1 -- 172.21.15.90:0/1015462214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8741045e0 msgr2=0x7fd8741049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:33.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.695+0000 7fd87a000700 1 --2- 172.21.15.90:0/1015462214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8741045e0 0x7fd8741049f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd864009fc0 tx=0x7fd86400b330 comp rx=0 tx=0).stop 2024-05-27T08:02:33.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.695+0000 7fd87a000700 1 -- 172.21.15.90:0/1015462214 shutdown_connections 2024-05-27T08:02:33.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.695+0000 7fd87a000700 1 --2- 172.21.15.90:0/1015462214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd874109860 0x7fd8741111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.695+0000 7fd87a000700 1 --2- 172.21.15.90:0/1015462214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8741045e0 0x7fd8741049f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.695+0000 7fd87a000700 1 --2- 172.21.15.90:0/1015462214 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd874108f10 0x7fd874109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.695+0000 7fd87a000700 1 -- 172.21.15.90:0/1015462214 >> 172.21.15.90:0/1015462214 conn(0x7fd874100030 msgr2=0x7fd874102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:33.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.696+0000 7fd87a000700 1 -- 172.21.15.90:0/1015462214 shutdown_connections 2024-05-27T08:02:33.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.696+0000 7fd87a000700 1 -- 172.21.15.90:0/1015462214 wait complete. 2024-05-27T08:02:33.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.697+0000 7fd87a000700 1 Processor -- start 2024-05-27T08:02:33.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.698+0000 7fd87a000700 1 -- start start 2024-05-27T08:02:33.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.698+0000 7fd87a000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8741045e0 0x7fd8740753f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.699+0000 7fd87a000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd874108f10 0x7fd87406e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.699+0000 7fd87a000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd874109860 0x7fd87406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.699+0000 7fd873400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8741045e0 0x7fd8740753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.699+0000 7fd87a000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741143b0 con 0x7fd8741045e0 2024-05-27T08:02:33.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.699+0000 7fd873400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8741045e0 0x7fd8740753f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44726/0 (socket says 172.21.15.90:44726) 2024-05-27T08:02:33.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.700+0000 7fd87a000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd874114230 con 0x7fd874109860 2024-05-27T08:02:33.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.700+0000 7fd86aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd874108f10 0x7fd87406e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.700+0000 7fd873e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd874109860 0x7fd87406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.700+0000 7fd86aa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd874108f10 0x7fd87406e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:45548/0 (socket says 172.21.15.90:45548) 2024-05-27T08:02:33.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.700+0000 7fd86aa00700 1 -- 172.21.15.90:0/1783052592 learned_addr learned my addr 172.21.15.90:0/1783052592 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:33.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.700+0000 7fd87a000700 1 -- 172.21.15.90:0/1783052592 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd874114530 con 0x7fd874108f10 2024-05-27T08:02:33.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.701+0000 7fd873e00700 1 -- 172.21.15.90:0/1783052592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd874108f10 msgr2=0x7fd87406e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:33.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.701+0000 7fd873e00700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd874108f10 0x7fd87406e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.701+0000 7fd873e00700 1 -- 172.21.15.90:0/1783052592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8741045e0 msgr2=0x7fd8740753f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:33.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.701+0000 7fd873e00700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8741045e0 0x7fd8740753f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.701+0000 7fd873e00700 1 -- 172.21.15.90:0/1783052592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd874072f20 con 0x7fd874109860 2024-05-27T08:02:33.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.701+0000 7fd873e00700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd874109860 0x7fd87406ec10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00e970 tx=0x7fd86c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:33.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.703+0000 7fd870c00700 1 -- 172.21.15.90:0/1783052592 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c004220 con 0x7fd874109860 2024-05-27T08:02:33.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.703+0000 7fd870c00700 1 -- 172.21.15.90:0/1783052592 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86c0043c0 con 0x7fd874109860 2024-05-27T08:02:33.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.703+0000 7fd870c00700 1 -- 172.21.15.90:0/1783052592 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c00ccc0 con 0x7fd874109860 2024-05-27T08:02:33.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.704+0000 7fd87a000700 1 -- 172.21.15.90:0/1783052592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd874073210 con 0x7fd874109860 2024-05-27T08:02:33.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.704+0000 7fd87a000700 1 -- 172.21.15.90:0/1783052592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd874073700 con 0x7fd874109860 2024-05-27T08:02:33.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.712+0000 7fd870c00700 1 -- 172.21.15.90:0/1783052592 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd86c004750 con 0x7fd874109860 2024-05-27T08:02:33.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.714+0000 7fd870c00700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8580b2b40 0x7fd8580b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.714+0000 7fd870c00700 1 -- 172.21.15.90:0/1783052592 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7fd86c0e2ca0 con 0x7fd874109860 2024-05-27T08:02:33.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.714+0000 7fd873400700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8580b2b40 0x7fd8580b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.718+0000 7fd873400700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8580b2b40 0x7fd8580b5050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd864000f80 tx=0x7fd864009f10 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:33.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.728+0000 7fd888c3bb80 1 Processor -- start 2024-05-27T08:02:33.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.730+0000 7fd888c3bb80 1 -- start start 2024-05-27T08:02:33.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.731+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a4ca9f0 0x55e62a561730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.731+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a3dee40 0x55e62a561c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.731+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a572cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.731+0000 7fd888c3bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e62a48be40 con 0x55e62a3dee40 2024-05-27T08:02:33.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.731+0000 7fd888c3bb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e62a47b9e0 con 0x55e62a5621b0 2024-05-27T08:02:33.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.731+0000 7fd888c3bb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e62a47fa20 con 0x55e62a4ca9f0 2024-05-27T08:02:33.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.732+0000 7fd873e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a572cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.732+0000 7fd873e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a572cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:55808/0 (socket says 172.21.15.90:55808) 2024-05-27T08:02:33.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.732+0000 7fd873e00700 1 -- 172.21.15.90:0/717732016 learned_addr learned my addr 172.21.15.90:0/717732016 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:33.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.732+0000 7fd86aa00700 1 --2- 172.21.15.90:0/717732016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a3dee40 0x55e62a561c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.732+0000 7fd873400700 1 --2- 172.21.15.90:0/717732016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a4ca9f0 0x55e62a561730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.732+0000 7fd873400700 1 -- 172.21.15.90:0/717732016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 msgr2=0x55e62a572cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:33.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.733+0000 7fd873400700 1 --2- 172.21.15.90:0/717732016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a572cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.733+0000 7fd873400700 1 -- 172.21.15.90:0/717732016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a3dee40 msgr2=0x55e62a561c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:33.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.733+0000 7fd873400700 1 --2- 172.21.15.90:0/717732016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a3dee40 0x55e62a561c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.733+0000 7fd873400700 1 -- 172.21.15.90:0/717732016 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62a444420 con 0x55e62a4ca9f0 2024-05-27T08:02:33.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.733+0000 7fd86aa00700 1 --2- 172.21.15.90:0/717732016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a3dee40 0x55e62a561c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:33.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.734+0000 7fd873400700 1 --2- 172.21.15.90:0/717732016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a4ca9f0 0x55e62a561730 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd87406fd70 tx=0x7fd86401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdb156113450f764 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.734+0000 7fd868c00700 1 -- 172.21.15.90:0/717732016 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86402d070 con 0x55e62a4ca9f0 2024-05-27T08:02:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.735+0000 7fd868c00700 1 -- 172.21.15.90:0/717732016 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86400bcb0 con 0x55e62a4ca9f0 2024-05-27T08:02:33.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.735+0000 7fd868c00700 1 -- 172.21.15.90:0/717732016 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640296b0 con 0x55e62a4ca9f0 2024-05-27T08:02:33.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.737+0000 7fd888c3bb80 1 -- 172.21.15.90:0/717732016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a4ca9f0 msgr2=0x55e62a561730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:33.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.737+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/717732016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a4ca9f0 0x55e62a561730 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd87406fd70 tx=0x7fd86401d000 comp rx=0 tx=0).stop 2024-05-27T08:02:33.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.738+0000 7fd888c3bb80 1 -- 172.21.15.90:0/717732016 shutdown_connections 2024-05-27T08:02:33.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.738+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/717732016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a572cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.738+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/717732016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a4ca9f0 0x55e62a561730 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.738+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/717732016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a3dee40 0x55e62a561c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.738+0000 7fd888c3bb80 1 -- 172.21.15.90:0/717732016 >> 172.21.15.90:0/717732016 conn(0x55e62a478390 msgr2=0x55e62a559350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:33.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.738+0000 7fd888c3bb80 1 -- 172.21.15.90:0/717732016 shutdown_connections 2024-05-27T08:02:33.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.739+0000 7fd888c3bb80 1 -- 172.21.15.90:0/717732016 wait complete. 2024-05-27T08:02:33.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.739+0000 7fd888c3bb80 1 -- start start 2024-05-27T08:02:33.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.740+0000 7fd888c3bb80 1 Processor -- start 2024-05-27T08:02:33.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.741+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a3dee40 0x55e62a570270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.741+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a4ca9f0 0x55e62a5707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.741+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a5621b0 0x55e62a570cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.741+0000 7fd888c3bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e62a381fa0 con 0x55e62a4ca9f0 2024-05-27T08:02:33.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.741+0000 7fd888c3bb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e62a41c760 con 0x55e62a3dee40 2024-05-27T08:02:33.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.741+0000 7fd888c3bb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e62a573360 con 0x55e62a5621b0 2024-05-27T08:02:33.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.741+0000 7fd888c3bb80 10 client.0 _ll_register_callbacks cb 0x55e62a425c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:33.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.742+0000 7fd873400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a3dee40 0x55e62a570270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.742+0000 7fd873400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a3dee40 0x55e62a570270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:55816/0 (socket says 172.21.15.90:55816) 2024-05-27T08:02:33.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.742+0000 7fd873400700 1 -- 172.21.15.90:0/3100462646 learned_addr learned my addr 172.21.15.90:0/3100462646 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:33.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.742+0000 7fd873e00700 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a5621b0 0x55e62a570cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.743+0000 7fd873400700 1 -- 172.21.15.90:0/3100462646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a5621b0 msgr2=0x55e62a570cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:33.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.743+0000 7fd873400700 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a5621b0 0x55e62a570cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.743+0000 7fd873400700 1 -- 172.21.15.90:0/3100462646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a4ca9f0 msgr2=0x55e62a5707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:33.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.743+0000 7fd873400700 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a4ca9f0 0x55e62a5707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.743+0000 7fd873400700 1 -- 172.21.15.90:0/3100462646 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62a3cb8d0 con 0x55e62a3dee40 2024-05-27T08:02:33.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.744+0000 7fd873400700 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a3dee40 0x55e62a570270 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd864009840 tx=0x7fd86402af70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c34b64f808797080 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:33.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.744+0000 7fd861600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:02:33.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.745+0000 7fd861600700 1 -- 172.21.15.90:0/3100462646 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86402d040 con 0x55e62a3dee40 2024-05-27T08:02:33.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.745+0000 7fd861600700 1 -- 172.21.15.90:0/3100462646 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7fd8640077d0 con 0x55e62a3dee40 2024-05-27T08:02:33.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.746+0000 7fd888c3bb80 1 -- 172.21.15.90:0/3100462646 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86400bcb0 con 0x55e62a3dee40 2024-05-27T08:02:33.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.746+0000 7fd861600700 1 -- 172.21.15.90:0/3100462646 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd86400be60 con 0x55e62a3dee40 2024-05-27T08:02:33.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.747+0000 7fd888c3bb80 10 client.37787 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:33.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.747+0000 7fd888c3bb80 1 -- 172.21.15.90:0/3100462646 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e62a44c060 con 0x55e62a3dee40 2024-05-27T08:02:33.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.747+0000 7fd861600700 1 -- 172.21.15.90:0/3100462646 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86400bcb0 con 0x55e62a3dee40 2024-05-27T08:02:33.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.747+0000 7fd861600700 1 -- 172.21.15.90:0/3100462646 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fd864011070 con 0x55e62a3dee40 2024-05-27T08:02:33.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.749+0000 7fd888c3bb80 10 client.37787 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:33.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.749+0000 7fd888c3bb80 10 client.37787 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T08:02:33.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.749+0000 7fd888c3bb80 20 client.37787 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:33.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.749+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e62a580170 0x55e62a582590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.749+0000 7fd888c3bb80 4 client.37787 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:33.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.749+0000 7fd888c3bb80 1 -- 172.21.15.90:0/3100462646 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e62a48dd90 con 0x55e62a580170 2024-05-27T08:02:33.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.749+0000 7fd86aa00700 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e62a580170 0x55e62a582590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.751+0000 7fd86aa00700 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e62a580170 0x55e62a582590 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=caf9bfa079452ee server_cookie=570bb295a5c5c13f in_seq=0 out_seq=0 2024-05-27T08:02:33.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.751+0000 7fd861600700 10 client.37787 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T08:02:33.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.752+0000 7fd861600700 1 -- 172.21.15.90:0/3100462646 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e62a48dd90 con 0x55e62a580170 2024-05-27T08:02:33.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.752+0000 7fd861600700 10 client.37787 handle_command_reply: tid=0 2024-05-27T08:02:33.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.752+0000 7fd888c3bb80 1 client.37787 shutdown 2024-05-27T08:02:33.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.754+0000 7fd888c3bb80 1 -- 172.21.15.90:0/3100462646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a3dee40 msgr2=0x55e62a570270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:33.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.754+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a3dee40 0x55e62a570270 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd864009840 tx=0x7fd86402af70 comp rx=0 tx=0).stop 2024-05-27T08:02:33.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.754+0000 7fd888c3bb80 1 -- 172.21.15.90:0/3100462646 shutdown_connections 2024-05-27T08:02:33.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.754+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e62a580170 0x55e62a582590 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.754+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a5621b0 0x55e62a570cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.754+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a3dee40 0x55e62a570270 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.754+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/3100462646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a4ca9f0 0x55e62a5707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.754+0000 7fd888c3bb80 1 -- 172.21.15.90:0/3100462646 >> 172.21.15.90:0/3100462646 conn(0x55e62a478390 msgr2=0x55e62a561710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:33.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.754+0000 7fd861600700 0 client.37787 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T08:02:33.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.755+0000 7fd888c3bb80 1 -- 172.21.15.90:0/3100462646 shutdown_connections 2024-05-27T08:02:33.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.755+0000 7fd888c3bb80 1 -- 172.21.15.90:0/3100462646 wait complete. 2024-05-27T08:02:33.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.755+0000 7fd888c3bb80 20 client.37787 trim_cache size 0 max 16384 2024-05-27T08:02:33.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.785+0000 7fd888c3bb80 1 Processor -- start 2024-05-27T08:02:33.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.786+0000 7fd888c3bb80 1 -- start start 2024-05-27T08:02:33.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.786+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a3dee40 0x55e62a572710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.786+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a4ca9f0 0x55e62a55dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.786+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a57c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.787+0000 7fd888c3bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741b8610 con 0x55e62a4ca9f0 2024-05-27T08:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.787+0000 7fd888c3bb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741b5660 con 0x55e62a5621b0 2024-05-27T08:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.787+0000 7fd888c3bb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741b7870 con 0x55e62a3dee40 2024-05-27T08:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.787+0000 7fd873e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a57c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.787+0000 7fd873e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a57c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:55818/0 (socket says 172.21.15.90:55818) 2024-05-27T08:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.787+0000 7fd873e00700 1 -- 172.21.15.90:0/2160413085 learned_addr learned my addr 172.21.15.90:0/2160413085 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.788+0000 7fd873e00700 1 -- 172.21.15.90:0/2160413085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a3dee40 msgr2=0x55e62a572710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.788+0000 7fd873e00700 1 --2- 172.21.15.90:0/2160413085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a3dee40 0x55e62a572710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.788+0000 7fd86aa00700 1 --2- 172.21.15.90:0/2160413085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a4ca9f0 0x55e62a55dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.788+0000 7fd873e00700 1 -- 172.21.15.90:0/2160413085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a4ca9f0 msgr2=0x55e62a55dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:33.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.788+0000 7fd873e00700 1 --2- 172.21.15.90:0/2160413085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a4ca9f0 0x55e62a55dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.788+0000 7fd873e00700 1 -- 172.21.15.90:0/2160413085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62a55a830 con 0x55e62a5621b0 2024-05-27T08:02:33.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.788+0000 7fd86aa00700 1 --2- 172.21.15.90:0/2160413085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a4ca9f0 0x55e62a55dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:33.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.789+0000 7fd873e00700 1 --2- 172.21.15.90:0/2160413085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a57c3c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd86c004620 tx=0x7fd86c0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bea4c1156aad69 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:33.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.790+0000 7fd863e00700 1 -- 172.21.15.90:0/2160413085 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0a9670 con 0x55e62a5621b0 2024-05-27T08:02:33.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.790+0000 7fd863e00700 1 -- 172.21.15.90:0/2160413085 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86c0c9070 con 0x55e62a5621b0 2024-05-27T08:02:33.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.790+0000 7fd863e00700 1 -- 172.21.15.90:0/2160413085 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0c4200 con 0x55e62a5621b0 2024-05-27T08:02:33.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.791+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2160413085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 msgr2=0x55e62a57c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:33.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.791+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2160413085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a57c3c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd86c004620 tx=0x7fd86c0cc040 comp rx=0 tx=0).stop 2024-05-27T08:02:33.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.792+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2160413085 shutdown_connections 2024-05-27T08:02:33.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.792+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2160413085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a5621b0 0x55e62a57c3c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.792+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2160413085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a3dee40 0x55e62a572710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.792+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2160413085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a4ca9f0 0x55e62a55dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.792+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2160413085 >> 172.21.15.90:0/2160413085 conn(0x55e62a478390 msgr2=0x55e62a5632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:33.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.792+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2160413085 shutdown_connections 2024-05-27T08:02:33.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.792+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2160413085 wait complete. 2024-05-27T08:02:33.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.793+0000 7fd888c3bb80 1 -- start start 2024-05-27T08:02:33.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.793+0000 7fd888c3bb80 1 Processor -- start 2024-05-27T08:02:33.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.795+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a3dee40 0x55e62a578bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.795+0000 7fd888c3bb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a4ca9f0 0x55e62a573570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.795+0000 7fd873400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a3dee40 0x55e62a578bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.795+0000 7fd873400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a3dee40 0x55e62a578bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:45594/0 (socket says 172.21.15.90:45594) 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.796+0000 7fd873400700 1 -- 172.21.15.90:0/2455367129 learned_addr learned my addr 172.21.15.90:0/2455367129 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.796+0000 7fd86aa00700 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a4ca9f0 0x55e62a573570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.796+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a5621b0 0x55e62a573ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.796+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741b71a0 con 0x55e62a5621b0 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.796+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741b5d30 con 0x55e62a4ca9f0 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.797+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741b6400 con 0x55e62a3dee40 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.797+0000 7fd888c3bb80 10 client.0 _ll_register_callbacks cb 0x55e62a425c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.798+0000 7fd86aa00700 1 -- 172.21.15.90:0/2455367129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a3dee40 msgr2=0x55e62a578bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.798+0000 7fd86aa00700 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a3dee40 0x55e62a578bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.798+0000 7fd86aa00700 1 -- 172.21.15.90:0/2455367129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a5621b0 msgr2=0x55e62a573ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:33.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.798+0000 7fd86aa00700 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a5621b0 0x55e62a573ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.798+0000 7fd86aa00700 1 -- 172.21.15.90:0/2455367129 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62a574150 con 0x55e62a4ca9f0 2024-05-27T08:02:33.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.799+0000 7fd861600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:02:33.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.799+0000 7fd86aa00700 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a4ca9f0 0x55e62a573570 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55e62a5627c0 tx=0x7fd85c010710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84ab44b87dae9a76 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:33.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.799+0000 7fd861600700 1 -- 172.21.15.90:0/2455367129 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c023060 con 0x55e62a4ca9f0 2024-05-27T08:02:33.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.800+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd86c0c9070 con 0x55e62a4ca9f0 2024-05-27T08:02:33.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.800+0000 7fd861600700 1 -- 172.21.15.90:0/2455367129 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7fd85c005260 con 0x55e62a4ca9f0 2024-05-27T08:02:33.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.800+0000 7fd861600700 1 -- 172.21.15.90:0/2455367129 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd85c023060 con 0x55e62a4ca9f0 2024-05-27T08:02:33.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.800+0000 7fd888c3bb80 10 client.37793 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:33.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.800+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e62a56b220 con 0x55e62a4ca9f0 2024-05-27T08:02:33.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.801+0000 7fd861600700 1 -- 172.21.15.90:0/2455367129 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd86c0c9070 con 0x55e62a4ca9f0 2024-05-27T08:02:33.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.801+0000 7fd861600700 1 -- 172.21.15.90:0/2455367129 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fd85c01d430 con 0x55e62a4ca9f0 2024-05-27T08:02:33.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.801+0000 7fd888c3bb80 10 client.37793 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:33.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.801+0000 7fd888c3bb80 10 client.37793 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T08:02:33.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.801+0000 7fd888c3bb80 20 client.37793 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:33.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.801+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e62a56b880 0x55e62a56dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:33.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.801+0000 7fd888c3bb80 4 client.37793 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:02:33.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.801+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e62a487570 con 0x55e62a56b880 2024-05-27T08:02:33.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.802+0000 7fd873400700 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e62a56b880 0x55e62a56dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:33.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.803+0000 7fd873400700 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e62a56b880 0x55e62a56dca0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b974b142527f1da server_cookie=d15631089e04ed2c in_seq=0 out_seq=0 2024-05-27T08:02:33.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.803+0000 7fd861600700 10 client.37793 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T08:02:33.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.804+0000 7fd861600700 1 -- 172.21.15.90:0/2455367129 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e62a487570 con 0x55e62a56b880 2024-05-27T08:02:33.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.804+0000 7fd861600700 10 client.37793 handle_command_reply: tid=0 2024-05-27T08:02:33.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.804+0000 7fd888c3bb80 1 client.37793 shutdown 2024-05-27T08:02:33.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.805+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a4ca9f0 msgr2=0x55e62a573570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:33.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.805+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a4ca9f0 0x55e62a573570 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55e62a5627c0 tx=0x7fd85c010710 comp rx=0 tx=0).stop 2024-05-27T08:02:33.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.805+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 shutdown_connections 2024-05-27T08:02:33.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.805+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e62a56b880 0x55e62a56dca0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.805+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e62a5621b0 0x55e62a573ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.805+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e62a3dee40 0x55e62a578bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.805+0000 7fd861600700 0 client.37793 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T08:02:33.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.805+0000 7fd888c3bb80 1 --2- 172.21.15.90:0/2455367129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e62a4ca9f0 0x55e62a573570 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.805+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 >> 172.21.15.90:0/2455367129 conn(0x55e62a478390 msgr2=0x55e62a5632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:33.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.806+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 shutdown_connections 2024-05-27T08:02:33.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.807+0000 7fd888c3bb80 1 -- 172.21.15.90:0/2455367129 wait complete. 2024-05-27T08:02:33.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.808+0000 7fd888c3bb80 20 client.37793 trim_cache size 0 max 16384 2024-05-27T08:02:33.810 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:02:33.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.809+0000 7fd87a000700 1 -- 172.21.15.90:0/1783052592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8580b2b40 msgr2=0x7fd8580b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:33.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.809+0000 7fd87a000700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8580b2b40 0x7fd8580b5050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd864000f80 tx=0x7fd864009f10 comp rx=0 tx=0).stop 2024-05-27T08:02:33.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.809+0000 7fd87a000700 1 -- 172.21.15.90:0/1783052592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd874109860 msgr2=0x7fd87406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:33.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.809+0000 7fd87a000700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd874109860 0x7fd87406ec10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00e970 tx=0x7fd86c00ec80 comp rx=0 tx=0).stop 2024-05-27T08:02:33.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.810+0000 7fd87a000700 1 -- 172.21.15.90:0/1783052592 shutdown_connections 2024-05-27T08:02:33.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.810+0000 7fd87a000700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8580b2b40 0x7fd8580b5050 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.810+0000 7fd87a000700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd874109860 0x7fd87406ec10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.810+0000 7fd87a000700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8741045e0 0x7fd8740753f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.810+0000 7fd87a000700 1 --2- 172.21.15.90:0/1783052592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd874108f10 0x7fd87406e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:33.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.810+0000 7fd87a000700 1 -- 172.21.15.90:0/1783052592 >> 172.21.15.90:0/1783052592 conn(0x7fd874100030 msgr2=0x7fd874102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:33.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.810+0000 7fd87a000700 1 -- 172.21.15.90:0/1783052592 shutdown_connections 2024-05-27T08:02:33.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:33.810+0000 7fd87a000700 1 -- 172.21.15.90:0/1783052592 wait complete. 2024-05-27T08:02:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 735.0 s, 427.0 tps, lat 231.708 ms stddev 199.805 2024-05-27T08:02:34.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:34 smithi090 ceph-mon[31429]: pgmap v520: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.6 MiB/s wr, 519 op/s 2024-05-27T08:02:35.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:34 smithi136 ceph-mon[35324]: pgmap v520: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.6 MiB/s wr, 519 op/s 2024-05-27T08:02:35.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:34 smithi129 ceph-mon[35254]: pgmap v520: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.6 MiB/s wr, 519 op/s 2024-05-27T08:02:36.720 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:02:36.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:36 smithi090 ceph-mon[31429]: pgmap v521: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-05-27T08:02:37.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:36 smithi136 ceph-mon[35324]: pgmap v521: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-05-27T08:02:37.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:36 smithi129 ceph-mon[35254]: pgmap v521: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-05-27T08:02:37.522 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:38.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 740.0 s, 452.6 tps, lat 221.334 ms stddev 192.462 2024-05-27T08:02:38.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:38 smithi090 ceph-mon[31429]: pgmap v522: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-05-27T08:02:39.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:38 smithi136 ceph-mon[35324]: pgmap v522: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-05-27T08:02:39.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:38 smithi129 ceph-mon[35254]: pgmap v522: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.0 MiB/s wr, 450 op/s 2024-05-27T08:02:40.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:40 smithi090 ceph-mon[31429]: pgmap v523: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 582 op/s 2024-05-27T08:02:40.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:40 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:02:41.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:40 smithi136 ceph-mon[35324]: pgmap v523: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 582 op/s 2024-05-27T08:02:41.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:40 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:02:41.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:40 smithi129 ceph-mon[35254]: pgmap v523: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.3 MiB/s wr, 582 op/s 2024-05-27T08:02:41.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:40 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:02:43.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:42 smithi090 ceph-mon[31429]: pgmap v524: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-05-27T08:02:43.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:42 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:02:43.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:42 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:02:43.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.227+0000 7f5050c00700 1 -- 172.21.15.90:0/2922965924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f504c1045e0 msgr2=0x7f504c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.227+0000 7f5050c00700 1 --2- 172.21.15.90:0/2922965924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f504c1045e0 0x7f504c1049f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f503c0099c0 tx=0x7f503c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.228+0000 7f5050c00700 1 -- 172.21.15.90:0/2922965924 shutdown_connections 2024-05-27T08:02:43.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.228+0000 7f5050c00700 1 --2- 172.21.15.90:0/2922965924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f504c109860 0x7f504c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.228+0000 7f5050c00700 1 --2- 172.21.15.90:0/2922965924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f504c1045e0 0x7f504c1049f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.228+0000 7f5050c00700 1 --2- 172.21.15.90:0/2922965924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f504c108f10 0x7f504c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.228+0000 7f5050c00700 1 -- 172.21.15.90:0/2922965924 >> 172.21.15.90:0/2922965924 conn(0x7f504c100030 msgr2=0x7f504c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.228+0000 7f5050c00700 1 -- 172.21.15.90:0/2922965924 shutdown_connections 2024-05-27T08:02:43.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.228+0000 7f5050c00700 1 -- 172.21.15.90:0/2922965924 wait complete. 2024-05-27T08:02:43.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.229+0000 7f5050c00700 1 Processor -- start 2024-05-27T08:02:43.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.230+0000 7f5050c00700 1 -- start start 2024-05-27T08:02:43.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.230+0000 7f5050c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f504c1045e0 0x7f504c111070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.230+0000 7f5050c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f504c108f10 0x7f504c10e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.230+0000 7f5050c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f504c109860 0x7f504c10e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.231+0000 7f5050c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f504c076fa0 con 0x7f504c108f10 2024-05-27T08:02:43.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.231+0000 7f5050c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f504c076e20 con 0x7f504c1045e0 2024-05-27T08:02:43.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.231+0000 7f5050c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f504c077120 con 0x7f504c109860 2024-05-27T08:02:43.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.231+0000 7f5049400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f504c108f10 0x7f504c10e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.231+0000 7f5049400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f504c108f10 0x7f504c10e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44858/0 (socket says 172.21.15.90:44858) 2024-05-27T08:02:43.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.231+0000 7f5049400700 1 -- 172.21.15.90:0/3751013384 learned_addr learned my addr 172.21.15.90:0/3751013384 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:43.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.231+0000 7f5049e00700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f504c1045e0 0x7f504c111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.232+0000 7f504a800700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f504c109860 0x7f504c10e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.232+0000 7f5049400700 1 -- 172.21.15.90:0/3751013384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f504c109860 msgr2=0x7f504c10e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.232+0000 7f5049400700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f504c109860 0x7f504c10e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.232+0000 7f5049400700 1 -- 172.21.15.90:0/3751013384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f504c1045e0 msgr2=0x7f504c111070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.232+0000 7f5049400700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f504c1045e0 0x7f504c111070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.233+0000 7f5049400700 1 -- 172.21.15.90:0/3751013384 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c10ef50 con 0x7f504c108f10 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.234+0000 7f5049400700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f504c108f10 0x7f504c10e1c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f503800ea30 tx=0x7f503800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.235+0000 7f5042a00700 1 -- 172.21.15.90:0/3751013384 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503800cbd0 con 0x7f504c108f10 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.235+0000 7f5042a00700 1 -- 172.21.15.90:0/3751013384 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50380044b0 con 0x7f504c108f10 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.235+0000 7f5042a00700 1 -- 172.21.15.90:0/3751013384 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038003cf0 con 0x7f504c108f10 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.235+0000 7f5050c00700 1 -- 172.21.15.90:0/3751013384 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c1ad820 con 0x7f504c108f10 2024-05-27T08:02:43.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.235+0000 7f5050c00700 1 -- 172.21.15.90:0/3751013384 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c1adce0 con 0x7f504c108f10 2024-05-27T08:02:43.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.245+0000 7f5042a00700 1 -- 172.21.15.90:0/3751013384 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5038021440 con 0x7f504c108f10 2024-05-27T08:02:43.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.248+0000 7f5042a00700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f50300b2a70 0x7f50300b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.248+0000 7f5042a00700 1 -- 172.21.15.90:0/3751013384 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f50380e3910 con 0x7f504c108f10 2024-05-27T08:02:43.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:42 smithi136 ceph-mon[35324]: pgmap v524: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-05-27T08:02:43.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:42 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:02:43.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:42 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:02:43.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.249+0000 7f5049e00700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f50300b2a70 0x7f50300b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.252+0000 7f5049e00700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f50300b2a70 0x7f50300b4f80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f503c005920 tx=0x7f503c01a040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.254+0000 7f505f768b80 1 Processor -- start 2024-05-27T08:02:43.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.254+0000 7f505f768b80 1 -- start start 2024-05-27T08:02:43.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.255+0000 7f505f768b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 0x5612216c0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.255+0000 7f505f768b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.255+0000 7f505f768b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 0x5612216cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.255+0000 7f505f768b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5612215eee40 con 0x5612216c0e70 2024-05-27T08:02:43.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.255+0000 7f505f768b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5612215de9e0 con 0x56122162d9f0 2024-05-27T08:02:43.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.255+0000 7f505f768b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5612215e2a20 con 0x561221541e40 2024-05-27T08:02:43.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.255+0000 7f5049e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 0x5612216c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.256+0000 7f5049400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.256+0000 7f5049e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 0x5612216c0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47284/0 (socket says 172.21.15.90:47284) 2024-05-27T08:02:43.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.256+0000 7f5049e00700 1 -- 172.21.15.90:0/4168504136 learned_addr learned my addr 172.21.15.90:0/4168504136 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:43.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.256+0000 7f504a800700 1 --2- 172.21.15.90:0/4168504136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 0x5612216cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.256+0000 7f5049e00700 1 -- 172.21.15.90:0/4168504136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 msgr2=0x5612216c0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.256+0000 7f5049e00700 1 --2- 172.21.15.90:0/4168504136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.257+0000 7f5049e00700 1 -- 172.21.15.90:0/4168504136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 msgr2=0x5612216cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.257+0000 7f5049e00700 1 --2- 172.21.15.90:0/4168504136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 0x5612216cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.257+0000 7f5049e00700 1 -- 172.21.15.90:0/4168504136 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612215a7420 con 0x56122162d9f0 2024-05-27T08:02:43.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.257+0000 7f5049400700 1 --2- 172.21.15.90:0/4168504136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:43.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.258+0000 7f5049e00700 1 --2- 172.21.15.90:0/4168504136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 0x5612216c0330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f503c015000 tx=0x7f503c009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ffbb9a04efa3c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.259+0000 7f5037400700 1 -- 172.21.15.90:0/4168504136 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c00bad0 con 0x56122162d9f0 2024-05-27T08:02:43.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.259+0000 7f5037400700 1 -- 172.21.15.90:0/4168504136 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f503c00bc70 con 0x56122162d9f0 2024-05-27T08:02:43.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.259+0000 7f5037400700 1 -- 172.21.15.90:0/4168504136 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c00f6c0 con 0x56122162d9f0 2024-05-27T08:02:43.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.260+0000 7f505f768b80 1 -- 172.21.15.90:0/4168504136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 msgr2=0x5612216c0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.260+0000 7f505f768b80 1 --2- 172.21.15.90:0/4168504136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 0x5612216c0330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f503c015000 tx=0x7f503c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.261+0000 7f505f768b80 1 -- 172.21.15.90:0/4168504136 shutdown_connections 2024-05-27T08:02:43.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.261+0000 7f505f768b80 1 --2- 172.21.15.90:0/4168504136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 0x5612216cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.261+0000 7f505f768b80 1 --2- 172.21.15.90:0/4168504136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 0x5612216c0330 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.261+0000 7f505f768b80 1 --2- 172.21.15.90:0/4168504136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.261+0000 7f505f768b80 1 -- 172.21.15.90:0/4168504136 >> 172.21.15.90:0/4168504136 conn(0x5612215db390 msgr2=0x5612216bc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.262+0000 7f505f768b80 1 -- 172.21.15.90:0/4168504136 shutdown_connections 2024-05-27T08:02:43.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.263+0000 7f505f768b80 1 -- 172.21.15.90:0/4168504136 wait complete. 2024-05-27T08:02:43.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.263+0000 7f505f768b80 1 -- start start 2024-05-27T08:02:43.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.264+0000 7f505f768b80 1 Processor -- start 2024-05-27T08:02:43.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.265+0000 7f505f768b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.265+0000 7f505f768b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 0x5612216caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.265+0000 7f505f768b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 0x5612216cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.265+0000 7f505f768b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5612214e4fa0 con 0x5612216c0e70 2024-05-27T08:02:43.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.265+0000 7f505f768b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56122157f760 con 0x56122162d9f0 2024-05-27T08:02:43.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.265+0000 7f505f768b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5612216cdac0 con 0x561221541e40 2024-05-27T08:02:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.265+0000 7f505f768b80 10 client.0 _ll_register_callbacks cb 0x561221588c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.265+0000 7f5049e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.266+0000 7f504a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 0x5612216cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.266+0000 7f504a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 0x5612216cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44876/0 (socket says 172.21.15.90:44876) 2024-05-27T08:02:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.266+0000 7f504a800700 1 -- 172.21.15.90:0/1174356552 learned_addr learned my addr 172.21.15.90:0/1174356552 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.266+0000 7f5049400700 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 0x5612216caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.266+0000 7f504a800700 1 -- 172.21.15.90:0/1174356552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 msgr2=0x5612216ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.266+0000 7f504a800700 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.266+0000 7f504a800700 1 -- 172.21.15.90:0/1174356552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 msgr2=0x5612216caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.266+0000 7f504a800700 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 0x5612216caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.267+0000 7f504a800700 1 -- 172.21.15.90:0/1174356552 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122152e8d0 con 0x5612216c0e70 2024-05-27T08:02:43.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.267+0000 7f504a800700 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 0x5612216cb1e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5044011f20 tx=0x7f5044014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb46c6d25a35fda3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.267+0000 7f502be00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:43.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.268+0000 7f502be00700 1 -- 172.21.15.90:0/1174356552 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044009310 con 0x5612216c0e70 2024-05-27T08:02:43.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.268+0000 7f505f768b80 1 -- 172.21.15.90:0/1174356552 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f503c00bc70 con 0x5612216c0e70 2024-05-27T08:02:43.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.269+0000 7f502be00700 1 -- 172.21.15.90:0/1174356552 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f5044003a00 con 0x5612216c0e70 2024-05-27T08:02:43.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.270+0000 7f502be00700 1 -- 172.21.15.90:0/1174356552 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f5044003c20 con 0x5612216c0e70 2024-05-27T08:02:43.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.270+0000 7f505f768b80 10 client.18120 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:43.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.270+0000 7f505f768b80 1 -- 172.21.15.90:0/1174356552 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5612215af060 con 0x5612216c0e70 2024-05-27T08:02:43.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.272+0000 7f502be00700 1 -- 172.21.15.90:0/1174356552 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f503c00bc70 con 0x5612216c0e70 2024-05-27T08:02:43.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.272+0000 7f502be00700 1 -- 172.21.15.90:0/1174356552 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f5044007500 con 0x5612216c0e70 2024-05-27T08:02:43.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.272+0000 7f505f768b80 10 client.18120 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:43.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.272+0000 7f505f768b80 10 client.18120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:02:43.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.272+0000 7f505f768b80 20 client.18120 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:43.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.272+0000 7f505f768b80 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5612216daba0 0x5612216dd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.272+0000 7f505f768b80 4 client.18120 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:43.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.272+0000 7f505f768b80 1 -- 172.21.15.90:0/1174356552 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612215f0d90 con 0x5612216daba0 2024-05-27T08:02:43.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.273+0000 7f5049e00700 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5612216daba0 0x5612216dd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.274+0000 7f5049e00700 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5612216daba0 0x5612216dd030 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=738f2c102af0ab67 server_cookie=124fd307a4102b1a in_seq=0 out_seq=0 2024-05-27T08:02:43.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.275+0000 7f502be00700 10 client.18120 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T08:02:43.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.277+0000 7f502be00700 1 -- 172.21.15.90:0/1174356552 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5612215f0d90 con 0x5612216daba0 2024-05-27T08:02:43.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.277+0000 7f502be00700 10 client.18120 handle_command_reply: tid=0 2024-05-27T08:02:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.277+0000 7f505f768b80 1 client.18120 shutdown 2024-05-27T08:02:43.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.277+0000 7f505f768b80 1 -- 172.21.15.90:0/1174356552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 msgr2=0x5612216cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.277+0000 7f505f768b80 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 0x5612216cb1e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5044011f20 tx=0x7f5044014600 comp rx=0 tx=0).stop 2024-05-27T08:02:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.278+0000 7f505f768b80 1 -- 172.21.15.90:0/1174356552 shutdown_connections 2024-05-27T08:02:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.278+0000 7f505f768b80 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5612216daba0 0x5612216dd030 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.278+0000 7f505f768b80 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5612216c0e70 0x5612216cb1e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.278+0000 7f505f768b80 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.278+0000 7f505f768b80 1 --2- 172.21.15.90:0/1174356552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56122162d9f0 0x5612216caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.278+0000 7f505f768b80 1 -- 172.21.15.90:0/1174356552 >> 172.21.15.90:0/1174356552 conn(0x5612215db390 msgr2=0x5612216c03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.278+0000 7f502be00700 0 client.18120 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T08:02:43.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.279+0000 7f505f768b80 1 -- 172.21.15.90:0/1174356552 shutdown_connections 2024-05-27T08:02:43.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.279+0000 7f505f768b80 1 -- 172.21.15.90:0/1174356552 wait complete. 2024-05-27T08:02:43.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.279+0000 7f505f768b80 20 client.18120 trim_cache size 0 max 16384 2024-05-27T08:02:43.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.309+0000 7f505f768b80 1 Processor -- start 2024-05-27T08:02:43.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.310+0000 7f505f768b80 1 -- start start 2024-05-27T08:02:43.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.310+0000 7f505f768b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561221541e40 0x5612216c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.310+0000 7f505f768b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56122162d9f0 0x5612216bc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.310+0000 7f505f768b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612216c0e70 0x5612216d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.310+0000 7f505f768b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1b8700 con 0x561221541e40 2024-05-27T08:02:43.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.310+0000 7f505f768b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1b5750 con 0x5612216c0e70 2024-05-27T08:02:43.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.310+0000 7f505f768b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1b7960 con 0x56122162d9f0 2024-05-27T08:02:43.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.311+0000 7f5049400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56122162d9f0 0x5612216bc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.311+0000 7f5049e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561221541e40 0x5612216c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.311+0000 7f5049e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561221541e40 0x5612216c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44884/0 (socket says 172.21.15.90:44884) 2024-05-27T08:02:43.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.311+0000 7f5049400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56122162d9f0 0x5612216bc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37748/0 (socket says 172.21.15.90:37748) 2024-05-27T08:02:43.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.311+0000 7f5049e00700 1 -- 172.21.15.90:0/2866026631 learned_addr learned my addr 172.21.15.90:0/2866026631 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:43.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.311+0000 7f504a800700 1 --2- 172.21.15.90:0/2866026631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612216c0e70 0x5612216d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.312+0000 7f5049400700 1 -- 172.21.15.90:0/2866026631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612216c0e70 msgr2=0x5612216d6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.312+0000 7f5049400700 1 --2- 172.21.15.90:0/2866026631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612216c0e70 0x5612216d6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.312+0000 7f5049400700 1 -- 172.21.15.90:0/2866026631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561221541e40 msgr2=0x5612216c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.312+0000 7f5049400700 1 --2- 172.21.15.90:0/2866026631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561221541e40 0x5612216c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.312+0000 7f5049400700 1 -- 172.21.15.90:0/2866026631 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612216b8240 con 0x56122162d9f0 2024-05-27T08:02:43.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.312+0000 7f5049400700 1 --2- 172.21.15.90:0/2866026631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56122162d9f0 0x5612216bc4f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f50380e0ca0 tx=0x7f50380098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d0881177eba702 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.313+0000 7f5036a00700 1 -- 172.21.15.90:0/2866026631 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038003680 con 0x56122162d9f0 2024-05-27T08:02:43.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:42 smithi129 ceph-mon[35254]: pgmap v524: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-05-27T08:02:43.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:42 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:02:43.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:42 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:02:43.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.313+0000 7f5036a00700 1 -- 172.21.15.90:0/2866026631 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50380aa340 con 0x56122162d9f0 2024-05-27T08:02:43.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.313+0000 7f5036a00700 1 -- 172.21.15.90:0/2866026631 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50380a6ac0 con 0x56122162d9f0 2024-05-27T08:02:43.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.315+0000 7f505f768b80 1 -- 172.21.15.90:0/2866026631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56122162d9f0 msgr2=0x5612216bc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.315+0000 7f505f768b80 1 --2- 172.21.15.90:0/2866026631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56122162d9f0 0x5612216bc4f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f50380e0ca0 tx=0x7f50380098e0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.315+0000 7f505f768b80 1 -- 172.21.15.90:0/2866026631 shutdown_connections 2024-05-27T08:02:43.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.315+0000 7f505f768b80 1 --2- 172.21.15.90:0/2866026631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612216c0e70 0x5612216d6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.315+0000 7f505f768b80 1 --2- 172.21.15.90:0/2866026631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561221541e40 0x5612216c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.315+0000 7f505f768b80 1 --2- 172.21.15.90:0/2866026631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56122162d9f0 0x5612216bc4f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.315+0000 7f505f768b80 1 -- 172.21.15.90:0/2866026631 >> 172.21.15.90:0/2866026631 conn(0x5612215db390 msgr2=0x5612216c9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.316+0000 7f505f768b80 1 -- 172.21.15.90:0/2866026631 shutdown_connections 2024-05-27T08:02:43.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.320+0000 7f505f768b80 1 -- 172.21.15.90:0/2866026631 wait complete. 2024-05-27T08:02:43.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.320+0000 7f505f768b80 1 -- start start 2024-05-27T08:02:43.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.321+0000 7f505f768b80 1 Processor -- start 2024-05-27T08:02:43.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.324+0000 7f505f768b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.324+0000 7f505f768b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56122162d9f0 0x5612216c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.324+0000 7f505f768b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612216c0e70 0x5612216c4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.326+0000 7f5049e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.326+0000 7f5049e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37754/0 (socket says 172.21.15.90:37754) 2024-05-27T08:02:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.326+0000 7f505f768b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1b7290 con 0x56122162d9f0 2024-05-27T08:02:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.326+0000 7f505f768b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1b5e20 con 0x5612216c0e70 2024-05-27T08:02:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.326+0000 7f505f768b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f504c1b64f0 con 0x561221541e40 2024-05-27T08:02:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.326+0000 7f505f768b80 10 client.0 _ll_register_callbacks cb 0x561221588c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.326+0000 7f5049e00700 1 -- 172.21.15.90:0/3537041613 learned_addr learned my addr 172.21.15.90:0/3537041613 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.327+0000 7f5049400700 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56122162d9f0 0x5612216c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.327+0000 7f504a800700 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612216c0e70 0x5612216c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.327+0000 7f5049e00700 1 -- 172.21.15.90:0/3537041613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612216c0e70 msgr2=0x5612216c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.327+0000 7f5049e00700 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612216c0e70 0x5612216c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.327+0000 7f5049e00700 1 -- 172.21.15.90:0/3537041613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56122162d9f0 msgr2=0x5612216c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.327+0000 7f5049e00700 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56122162d9f0 0x5612216c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.327+0000 7f5049e00700 1 -- 172.21.15.90:0/3537041613 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612216c4ba0 con 0x561221541e40 2024-05-27T08:02:43.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.329+0000 7f5049e00700 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c5c90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f503c0098e0 tx=0x7f503c0050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a29f2c67f5bec210 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.329+0000 7f502be00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:02:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.330+0000 7f502be00700 1 -- 172.21.15.90:0/3537041613 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c00f980 con 0x561221541e40 2024-05-27T08:02:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.330+0000 7f502be00700 1 -- 172.21.15.90:0/3537041613 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f503c00fb20 con 0x561221541e40 2024-05-27T08:02:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.330+0000 7f505f768b80 1 -- 172.21.15.90:0/3537041613 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50380aa340 con 0x561221541e40 2024-05-27T08:02:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.331+0000 7f502be00700 1 -- 172.21.15.90:0/3537041613 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f503c00fd20 con 0x561221541e40 2024-05-27T08:02:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.332+0000 7f505f768b80 10 client.27883 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.332+0000 7f505f768b80 1 -- 172.21.15.90:0/3537041613 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5612216d2b00 con 0x561221541e40 2024-05-27T08:02:43.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.332+0000 7f502be00700 1 -- 172.21.15.90:0/3537041613 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f50380aa340 con 0x561221541e40 2024-05-27T08:02:43.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.333+0000 7f502be00700 1 -- 172.21.15.90:0/3537041613 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f503c003ab0 con 0x561221541e40 2024-05-27T08:02:43.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.333+0000 7f505f768b80 10 client.27883 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:43.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.333+0000 7f505f768b80 10 client.27883 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:02:43.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.333+0000 7f505f768b80 20 client.27883 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:43.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.333+0000 7f505f768b80 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5612216d2fe0 0x5612216d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.333+0000 7f505f768b80 4 client.27883 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:02:43.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.333+0000 7f505f768b80 1 -- 172.21.15.90:0/3537041613 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612215ea570 con 0x5612216d2fe0 2024-05-27T08:02:43.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.337+0000 7f5049400700 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5612216d2fe0 0x5612216d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.338+0000 7f502be00700 10 client.27883 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T08:02:43.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.338+0000 7f5049400700 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5612216d2fe0 0x5612216d33f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a431915cd3d9a14 server_cookie=2aad66fa91d2a2fc in_seq=0 out_seq=0 2024-05-27T08:02:43.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.339+0000 7f502be00700 1 -- 172.21.15.90:0/3537041613 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612215ea570 con 0x5612216d2fe0 2024-05-27T08:02:43.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.339+0000 7f502be00700 10 client.27883 handle_command_reply: tid=0 2024-05-27T08:02:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.344+0000 7f505f768b80 1 client.27883 shutdown 2024-05-27T08:02:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.344+0000 7f505f768b80 1 -- 172.21.15.90:0/3537041613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 msgr2=0x5612216c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.344+0000 7f505f768b80 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c5c90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f503c0098e0 tx=0x7f503c0050b0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.346+0000 7f505f768b80 1 -- 172.21.15.90:0/3537041613 shutdown_connections 2024-05-27T08:02:43.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.346+0000 7f505f768b80 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x5612216d2fe0 0x5612216d33f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.346+0000 7f505f768b80 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612216c0e70 0x5612216c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.346+0000 7f505f768b80 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561221541e40 0x5612216c5c90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.346+0000 7f505f768b80 1 --2- 172.21.15.90:0/3537041613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56122162d9f0 0x5612216c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.346+0000 7f505f768b80 1 -- 172.21.15.90:0/3537041613 >> 172.21.15.90:0/3537041613 conn(0x5612215db390 msgr2=0x5612216c1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.347+0000 7f502be00700 0 client.27883 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T08:02:43.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.347+0000 7f505f768b80 1 -- 172.21.15.90:0/3537041613 shutdown_connections 2024-05-27T08:02:43.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.347+0000 7f505f768b80 1 -- 172.21.15.90:0/3537041613 wait complete. 2024-05-27T08:02:43.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.347+0000 7f505f768b80 20 client.27883 trim_cache size 0 max 16384 2024-05-27T08:02:43.350 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:02:43.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.351+0000 7f5050c00700 1 -- 172.21.15.90:0/3751013384 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f50300b2a70 msgr2=0x7f50300b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.351+0000 7f5050c00700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f50300b2a70 0x7f50300b4f80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f503c005920 tx=0x7f503c01a040 comp rx=0 tx=0).stop 2024-05-27T08:02:43.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.351+0000 7f5050c00700 1 -- 172.21.15.90:0/3751013384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f504c108f10 msgr2=0x7f504c10e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.351+0000 7f5050c00700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f504c108f10 0x7f504c10e1c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f503800ea30 tx=0x7f503800ed40 comp rx=0 tx=0).stop 2024-05-27T08:02:43.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.352+0000 7f5050c00700 1 -- 172.21.15.90:0/3751013384 shutdown_connections 2024-05-27T08:02:43.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.352+0000 7f5050c00700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f504c109860 0x7f504c10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.352+0000 7f5050c00700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f50300b2a70 0x7f50300b4f80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.352+0000 7f5050c00700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f504c1045e0 0x7f504c111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.352+0000 7f5050c00700 1 --2- 172.21.15.90:0/3751013384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f504c108f10 0x7f504c10e1c0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.352+0000 7f5050c00700 1 -- 172.21.15.90:0/3751013384 >> 172.21.15.90:0/3751013384 conn(0x7f504c100030 msgr2=0x7f504c114e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.352+0000 7f5050c00700 1 -- 172.21.15.90:0/3751013384 shutdown_connections 2024-05-27T08:02:43.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:43.353+0000 7f5050c00700 1 -- 172.21.15.90:0/3751013384 wait complete. 2024-05-27T08:02:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.754+0000 7fbb2d200640 1 -- 172.21.15.90:0/2209479346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb2810e8c0 msgr2=0x7fbb2810ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.754+0000 7fbb2d200640 1 --2- 172.21.15.90:0/2209479346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb2810e8c0 0x7fbb2810ecd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009900 tx=0x7fbb1802a420 comp rx=0 tx=0).stop 2024-05-27T08:02:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.756+0000 7fbb2d200640 1 -- 172.21.15.90:0/2209479346 shutdown_connections 2024-05-27T08:02:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.756+0000 7fbb2d200640 1 --2- 172.21.15.90:0/2209479346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb28113910 0x7fbb2811b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.756+0000 7fbb2d200640 1 --2- 172.21.15.90:0/2209479346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb28112e50 0x7fbb28113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.756+0000 7fbb2d200640 1 --2- 172.21.15.90:0/2209479346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb2810e8c0 0x7fbb2810ecd0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.756+0000 7fbb2d200640 1 -- 172.21.15.90:0/2209479346 >> 172.21.15.90:0/2209479346 conn(0x7fbb28109ea0 msgr2=0x7fbb2810c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.757+0000 7fbb2d200640 1 -- 172.21.15.90:0/2209479346 shutdown_connections 2024-05-27T08:02:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.758+0000 7fbb2d200640 1 -- 172.21.15.90:0/2209479346 wait complete. 2024-05-27T08:02:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.760+0000 7fbb2d200640 1 Processor -- start 2024-05-27T08:02:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.760+0000 7fbb2d200640 1 -- start start 2024-05-27T08:02:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.761+0000 7fbb2d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb2810e8c0 0x7fbb281a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.761+0000 7fbb2d200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb28112e50 0x7fbb281a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.762+0000 7fbb25e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb2810e8c0 0x7fbb281a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.762+0000 7fbb2d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb28113910 0x7fbb281a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.762+0000 7fbb2d200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2811e800 con 0x7fbb28113910 2024-05-27T08:02:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.762+0000 7fbb2d200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2811e680 con 0x7fbb28112e50 2024-05-27T08:02:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.762+0000 7fbb2d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2811e980 con 0x7fbb2810e8c0 2024-05-27T08:02:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.762+0000 7fbb25e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb2810e8c0 0x7fbb281a5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37770/0 (socket says 172.21.15.90:37770) 2024-05-27T08:02:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.762+0000 7fbb25e00640 1 -- 172.21.15.90:0/1302214028 learned_addr learned my addr 172.21.15.90:0/1302214028 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.762+0000 7fbb25400640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb28112e50 0x7fbb281a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.763+0000 7fbb26800640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb28113910 0x7fbb281a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.763+0000 7fbb25400640 1 -- 172.21.15.90:0/1302214028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb2810e8c0 msgr2=0x7fbb281a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.763+0000 7fbb25400640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb2810e8c0 0x7fbb281a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.763+0000 7fbb25400640 1 -- 172.21.15.90:0/1302214028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb28113910 msgr2=0x7fbb281a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.763+0000 7fbb25400640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb28113910 0x7fbb281a9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.764+0000 7fbb25400640 1 -- 172.21.15.90:0/1302214028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb281aa180 con 0x7fbb28112e50 2024-05-27T08:02:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.766+0000 7fbb25400640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb28112e50 0x7fbb281a5880 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400b3c0 tx=0x7fbb1400b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.767+0000 7fbb1ea00640 1 -- 172.21.15.90:0/1302214028 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14011830 con 0x7fbb28112e50 2024-05-27T08:02:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.767+0000 7fbb1ea00640 1 -- 172.21.15.90:0/1302214028 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb14016070 con 0x7fbb28112e50 2024-05-27T08:02:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.767+0000 7fbb1ea00640 1 -- 172.21.15.90:0/1302214028 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14010d00 con 0x7fbb28112e50 2024-05-27T08:02:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.767+0000 7fbb2d200640 1 -- 172.21.15.90:0/1302214028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb281aa470 con 0x7fbb28112e50 2024-05-27T08:02:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.767+0000 7fbb2d200640 1 -- 172.21.15.90:0/1302214028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb281aa8e0 con 0x7fbb28112e50 2024-05-27T08:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.777+0000 7fbb341c4740 1 Processor -- start 2024-05-27T08:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.780+0000 7fbb341c4740 1 -- start start 2024-05-27T08:02:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.780+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 0x562636b97860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.781+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 0x562636b97da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.781+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 0x562636b982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.781+0000 7fbb341c4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562636b08b90 con 0x5626367d44d0 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.781+0000 7fbb26800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 0x562636b97860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.781+0000 7fbb341c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562636af8730 con 0x5626367f51b0 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.781+0000 7fbb26800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 0x562636b97860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47350/0 (socket says 172.21.15.90:47350) 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.781+0000 7fbb26800640 1 -- 172.21.15.90:0/1496842664 learned_addr learned my addr 172.21.15.90:0/1496842664 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.781+0000 7fbb341c4740 1 -- 172.21.15.90:0/1496842664 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562636afc770 con 0x5626367cc2f0 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.782+0000 7fbb1ea00640 1 -- 172.21.15.90:0/1302214028 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbb14010380 con 0x7fbb28112e50 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.782+0000 7fbb26800640 1 -- 172.21.15.90:0/1496842664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 msgr2=0x562636b97da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.782+0000 7fbb26800640 1 --2- 172.21.15.90:0/1496842664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 0x562636b97da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.782+0000 7fbb26800640 1 -- 172.21.15.90:0/1496842664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 msgr2=0x562636b982e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:02:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.782+0000 7fbb26800640 1 --2- 172.21.15.90:0/1496842664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 0x562636b982e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.782+0000 7fbb26800640 1 -- 172.21.15.90:0/1496842664 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562636a918b0 con 0x5626367f51b0 2024-05-27T08:02:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.783+0000 7fbb26800640 1 --2- 172.21.15.90:0/1496842664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 0x562636b97860 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbb20007d90 tx=0x7fbb2000c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aad9ccc8d6e62d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.785+0000 7fbb13400640 1 -- 172.21.15.90:0/1496842664 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20015020 con 0x5626367f51b0 2024-05-27T08:02:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.787+0000 7fbb13400640 1 -- 172.21.15.90:0/1496842664 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb20011ac0 con 0x5626367f51b0 2024-05-27T08:02:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.787+0000 7fbb13400640 1 -- 172.21.15.90:0/1496842664 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20010930 con 0x5626367f51b0 2024-05-27T08:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.788+0000 7fbb1ea00640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbb0c0b2ae0 0x7fbb0c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.788+0000 7fbb1ea00640 1 -- 172.21.15.90:0/1302214028 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7fbb140e1a10 con 0x7fbb28112e50 2024-05-27T08:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.789+0000 7fbb341c4740 1 -- 172.21.15.90:0/1496842664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 msgr2=0x562636b97860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.789+0000 7fbb341c4740 1 --2- 172.21.15.90:0/1496842664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 0x562636b97860 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbb20007d90 tx=0x7fbb2000c560 comp rx=0 tx=0).stop 2024-05-27T08:02:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.789+0000 7fbb25e00640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbb0c0b2ae0 0x7fbb0c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.790+0000 7fbb341c4740 1 -- 172.21.15.90:0/1496842664 shutdown_connections 2024-05-27T08:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.790+0000 7fbb341c4740 1 --2- 172.21.15.90:0/1496842664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 0x562636b982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.790+0000 7fbb341c4740 1 --2- 172.21.15.90:0/1496842664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 0x562636b97da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.790+0000 7fbb341c4740 1 --2- 172.21.15.90:0/1496842664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 0x562636b97860 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.790+0000 7fbb341c4740 1 -- 172.21.15.90:0/1496842664 >> 172.21.15.90:0/1496842664 conn(0x5626367e9300 msgr2=0x562636b94010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.791+0000 7fbb25e00640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbb0c0b2ae0 0x7fbb0c0b4ff0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x562636b94fa0 tx=0x7fbb1802a920 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.792+0000 7fbb341c4740 1 -- 172.21.15.90:0/1496842664 shutdown_connections 2024-05-27T08:02:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.792+0000 7fbb341c4740 1 -- 172.21.15.90:0/1496842664 wait complete. 2024-05-27T08:02:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.793+0000 7fbb341c4740 1 -- start start 2024-05-27T08:02:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.793+0000 7fbb341c4740 1 Processor -- start 2024-05-27T08:02:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.795+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367cc2f0 0x562636ba1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.796+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367d44d0 0x562636ba2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.796+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.796+0000 7fbb25e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367cc2f0 0x562636ba1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.796+0000 7fbb25e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367cc2f0 0x562636ba1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:47360/0 (socket says 172.21.15.90:47360) 2024-05-27T08:02:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.797+0000 7fbb25400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367d44d0 0x562636ba2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.797+0000 7fbb25400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367d44d0 0x562636ba2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:37784/0 (socket says 172.21.15.90:37784) 2024-05-27T08:02:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.797+0000 7fbb26800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.797+0000 7fbb26800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44912/0 (socket says 172.21.15.90:44912) 2024-05-27T08:02:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.797+0000 7fbb25e00640 1 -- 172.21.15.90:0/1779227472 learned_addr learned my addr 172.21.15.90:0/1779227472 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.798+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5626367de790 con 0x5626367f51b0 2024-05-27T08:02:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.798+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562636aa0bc0 con 0x5626367cc2f0 2024-05-27T08:02:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.798+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562636ba4990 con 0x5626367d44d0 2024-05-27T08:02:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.798+0000 7fbb341c4740 10 client.0 _ll_register_callbacks cb 0x562636a7e400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.799+0000 7fbb26800640 1 -- 172.21.15.90:0/1779227472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367d44d0 msgr2=0x562636ba2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.799+0000 7fbb26800640 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367d44d0 0x562636ba2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.799+0000 7fbb26800640 1 -- 172.21.15.90:0/1779227472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367cc2f0 msgr2=0x562636ba1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.799+0000 7fbb26800640 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367cc2f0 0x562636ba1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.799+0000 7fbb26800640 1 -- 172.21.15.90:0/1779227472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56263680d210 con 0x5626367f51b0 2024-05-27T08:02:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.799+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb20011ac0 con 0x5626367f51b0 2024-05-27T08:02:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.800+0000 7fbb26800640 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badcc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbb20000f80 tx=0x7fbb20011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d652697420335b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.801+0000 7fbb25e00640 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367cc2f0 0x562636ba1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.800+0000 7fbb07e00640 10 client.18126 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.802+0000 7fbb07e00640 1 -- 172.21.15.90:0/1779227472 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20028060 con 0x5626367f51b0 2024-05-27T08:02:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.803+0000 7fbb07e00640 1 -- 172.21.15.90:0/1779227472 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7fbb200108a0 con 0x5626367f51b0 2024-05-27T08:02:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.803+0000 7fbb07e00640 1 -- 172.21.15.90:0/1779227472 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fbb20028060 con 0x5626367f51b0 2024-05-27T08:02:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.803+0000 7fbb341c4740 10 client.18126 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.803+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({fsmap=64}) -- 0x5626368d9620 con 0x5626367f51b0 2024-05-27T08:02:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.804+0000 7fbb07e00640 1 -- 172.21.15.90:0/1779227472 <== mon.0 v2:172.21.15.90:3300/0 4 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fbb20022030 con 0x5626367f51b0 2024-05-27T08:02:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.805+0000 7fbb341c4740 10 client.18126 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.805+0000 7fbb341c4740 10 client.18126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.805+0000 7fbb341c4740 20 client.18126 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.805+0000 7fbb341c4740 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626367b3d80 0x562636bb3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.805+0000 7fbb341c4740 4 client.18126 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.805+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562636b0aae0 con 0x5626367b3d80 2024-05-27T08:02:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.806+0000 7fbb25e00640 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626367b3d80 0x562636bb3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.808+0000 7fbb25e00640 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626367b3d80 0x562636bb3c60 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98660110400a2823 server_cookie=6b358c94646c56a4 in_seq=0 out_seq=0 2024-05-27T08:02:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.809+0000 7fbb07e00640 10 client.18126 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.818+0000 7fbb07e00640 1 -- 172.21.15.90:0/1779227472 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x562636b0aae0 con 0x5626367b3d80 2024-05-27T08:02:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.818+0000 7fbb07e00640 10 client.18126 handle_command_reply: tid=0 2024-05-27T08:02:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.818+0000 7fbb341c4740 1 client.18126 shutdown 2024-05-27T08:02:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.819+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 msgr2=0x562636badcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.819+0000 7fbb341c4740 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badcc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbb20000f80 tx=0x7fbb20011a30 comp rx=0 tx=0).stop 2024-05-27T08:02:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.822+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 shutdown_connections 2024-05-27T08:02:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.822+0000 7fbb341c4740 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626367b3d80 0x562636bb3c60 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.822+0000 7fbb341c4740 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badcc0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.822+0000 7fbb341c4740 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367d44d0 0x562636ba2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.822+0000 7fbb341c4740 1 --2- 172.21.15.90:0/1779227472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367cc2f0 0x562636ba1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.822+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 >> 172.21.15.90:0/1779227472 conn(0x5626367e9300 msgr2=0x562636b99580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.822+0000 7fbb07e00640 0 client.18126 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.822+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 shutdown_connections 2024-05-27T08:02:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.822+0000 7fbb341c4740 1 -- 172.21.15.90:0/1779227472 wait complete. 2024-05-27T08:02:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.822+0000 7fbb341c4740 20 client.18126 trim_cache size 0 max 16384 2024-05-27T08:02:43.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 745.0 s, 450.0 tps, lat 218.815 ms stddev 184.720 2024-05-27T08:02:43.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.870+0000 7fbb341c4740 1 Processor -- start 2024-05-27T08:02:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.870+0000 7fbb341c4740 1 -- start start 2024-05-27T08:02:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.870+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 0x562636ba0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.870+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367d44d0 0x562636bada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.870+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.870+0000 7fbb341c4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281c1b90 con 0x5626367f51b0 2024-05-27T08:02:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.870+0000 7fbb341c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281bebe0 con 0x5626367d44d0 2024-05-27T08:02:43.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.870+0000 7fbb341c4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281c0df0 con 0x5626367cc2f0 2024-05-27T08:02:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.871+0000 7fbb26800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.872+0000 7fbb26800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44916/0 (socket says 172.21.15.90:44916) 2024-05-27T08:02:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.872+0000 7fbb26800640 1 -- 172.21.15.90:0/3661851594 learned_addr learned my addr 172.21.15.90:0/3661851594 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:43.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.872+0000 7fbb25400640 1 --2- 172.21.15.90:0/3661851594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367d44d0 0x562636bada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.875+0000 7fbb26800640 1 -- 172.21.15.90:0/3661851594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 msgr2=0x562636ba0a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:02:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.875+0000 7fbb26800640 1 --2- 172.21.15.90:0/3661851594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 0x562636ba0a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.875+0000 7fbb26800640 1 -- 172.21.15.90:0/3661851594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367d44d0 msgr2=0x562636bada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.875+0000 7fbb26800640 1 --2- 172.21.15.90:0/3661851594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367d44d0 0x562636bada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.875+0000 7fbb26800640 1 -- 172.21.15.90:0/3661851594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562636b98960 con 0x5626367f51b0 2024-05-27T08:02:43.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.876+0000 7fbb26800640 1 --2- 172.21.15.90:0/3661851594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badf70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbb20004b90 tx=0x7fbb2001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8358b3125e5006 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.877+0000 7fbb12a00640 1 -- 172.21.15.90:0/3661851594 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20015070 con 0x5626367f51b0 2024-05-27T08:02:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.878+0000 7fbb12a00640 1 -- 172.21.15.90:0/3661851594 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb20011ac0 con 0x5626367f51b0 2024-05-27T08:02:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.878+0000 7fbb12a00640 1 -- 172.21.15.90:0/3661851594 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2002b770 con 0x5626367f51b0 2024-05-27T08:02:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.879+0000 7fbb341c4740 1 -- 172.21.15.90:0/3661851594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 msgr2=0x562636badf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.879+0000 7fbb341c4740 1 --2- 172.21.15.90:0/3661851594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badf70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbb20004b90 tx=0x7fbb2001e000 comp rx=0 tx=0).stop 2024-05-27T08:02:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.881+0000 7fbb341c4740 1 -- 172.21.15.90:0/3661851594 shutdown_connections 2024-05-27T08:02:43.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.883+0000 7fbb341c4740 1 --2- 172.21.15.90:0/3661851594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367d44d0 0x562636bada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.883+0000 7fbb341c4740 1 --2- 172.21.15.90:0/3661851594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367f51b0 0x562636badf70 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.883+0000 7fbb341c4740 1 --2- 172.21.15.90:0/3661851594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 0x562636ba0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.883+0000 7fbb341c4740 1 -- 172.21.15.90:0/3661851594 >> 172.21.15.90:0/3661851594 conn(0x5626367e9300 msgr2=0x562636b98020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.884+0000 7fbb341c4740 1 -- 172.21.15.90:0/3661851594 shutdown_connections 2024-05-27T08:02:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.884+0000 7fbb341c4740 1 -- 172.21.15.90:0/3661851594 wait complete. 2024-05-27T08:02:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.885+0000 7fbb341c4740 1 -- start start 2024-05-27T08:02:43.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.886+0000 7fbb341c4740 1 Processor -- start 2024-05-27T08:02:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.888+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 0x562636baa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.888+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 0x562636ba4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.888+0000 7fbb341c4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 0x562636ba50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.889+0000 7fbb341c4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281c0720 con 0x5626367d44d0 2024-05-27T08:02:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.889+0000 7fbb341c4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281bf2b0 con 0x5626367f51b0 2024-05-27T08:02:43.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.889+0000 7fbb341c4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb281bf980 con 0x5626367cc2f0 2024-05-27T08:02:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.889+0000 7fbb341c4740 10 client.0 _ll_register_callbacks cb 0x7fbb20007be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.889+0000 7fbb25400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 0x562636ba4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.891+0000 7fbb25400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 0x562636ba4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44930/0 (socket says 172.21.15.90:44930) 2024-05-27T08:02:43.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.891+0000 7fbb25400640 1 -- 172.21.15.90:0/2679952305 learned_addr learned my addr 172.21.15.90:0/2679952305 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.891+0000 7fbb25400640 1 -- 172.21.15.90:0/2679952305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 msgr2=0x562636baa230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:02:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.891+0000 7fbb25400640 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 0x562636baa230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.891+0000 7fbb25400640 1 -- 172.21.15.90:0/2679952305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 msgr2=0x562636ba50f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:02:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.891+0000 7fbb25400640 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 0x562636ba50f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.891+0000 7fbb25400640 1 -- 172.21.15.90:0/2679952305 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562636ba5680 con 0x5626367d44d0 2024-05-27T08:02:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.892+0000 7fbb25400640 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 0x562636ba4bb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbb140df3a0 tx=0x7fbb140a81b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ab0ac1bff31b67 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.892+0000 7fbb07e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:43.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.893+0000 7fbb07e00640 1 -- 172.21.15.90:0/2679952305 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb140c1070 con 0x5626367d44d0 2024-05-27T08:02:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.893+0000 7fbb07e00640 1 -- 172.21.15.90:0/2679952305 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7fbb140cb2d0 con 0x5626367d44d0 2024-05-27T08:02:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.893+0000 7fbb341c4740 1 -- 172.21.15.90:0/2679952305 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb20011ac0 con 0x5626367d44d0 2024-05-27T08:02:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.895+0000 7fbb07e00640 1 -- 172.21.15.90:0/2679952305 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fbb140b1040 con 0x5626367d44d0 2024-05-27T08:02:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.895+0000 7fbb341c4740 10 client.18132 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:43.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.895+0000 7fbb341c4740 1 -- 172.21.15.90:0/2679952305 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x562636c1b040 con 0x5626367d44d0 2024-05-27T08:02:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.896+0000 7fbb07e00640 1 -- 172.21.15.90:0/2679952305 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb20011ac0 con 0x5626367d44d0 2024-05-27T08:02:43.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.896+0000 7fbb07e00640 1 -- 172.21.15.90:0/2679952305 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fbb140b6b50 con 0x5626367d44d0 2024-05-27T08:02:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.898+0000 7fbb341c4740 10 client.18132 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.898+0000 7fbb341c4740 10 client.18132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.898+0000 7fbb341c4740 20 client.18132 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.898+0000 7fbb341c4740 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626367891c0 0x562636c1d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.898+0000 7fbb341c4740 4 client.18132 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:02:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.898+0000 7fbb341c4740 1 -- 172.21.15.90:0/2679952305 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562636b042c0 con 0x5626367891c0 2024-05-27T08:02:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.898+0000 7fbb25e00640 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626367891c0 0x562636c1d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.900+0000 7fbb25e00640 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626367891c0 0x562636c1d790 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa38a6175179f71 server_cookie=4bcca0c31f6b95b7 in_seq=0 out_seq=0 2024-05-27T08:02:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.900+0000 7fbb07e00640 10 client.18132 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:43.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.927+0000 7fbb07e00640 1 -- 172.21.15.90:0/2679952305 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562636b042c0 con 0x5626367891c0 2024-05-27T08:02:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.927+0000 7fbb07e00640 10 client.18132 handle_command_reply: tid=0 2024-05-27T08:02:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.927+0000 7fbb341c4740 1 client.18132 shutdown 2024-05-27T08:02:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.927+0000 7fbb341c4740 1 -- 172.21.15.90:0/2679952305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 msgr2=0x562636ba4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.927+0000 7fbb341c4740 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 0x562636ba4bb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbb140df3a0 tx=0x7fbb140a81b0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.929+0000 7fbb341c4740 1 -- 172.21.15.90:0/2679952305 shutdown_connections 2024-05-27T08:02:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.929+0000 7fbb341c4740 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5626367891c0 0x562636c1d790 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.929+0000 7fbb341c4740 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626367f51b0 0x562636ba50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.929+0000 7fbb341c4740 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5626367d44d0 0x562636ba4bb0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.929+0000 7fbb341c4740 1 --2- 172.21.15.90:0/2679952305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626367cc2f0 0x562636baa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.929+0000 7fbb341c4740 1 -- 172.21.15.90:0/2679952305 >> 172.21.15.90:0/2679952305 conn(0x5626367e9300 msgr2=0x562636b98020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.929+0000 7fbb07e00640 0 client.18132 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.929+0000 7fbb341c4740 1 -- 172.21.15.90:0/2679952305 shutdown_connections 2024-05-27T08:02:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.929+0000 7fbb341c4740 1 -- 172.21.15.90:0/2679952305 wait complete. 2024-05-27T08:02:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.930+0000 7fbb341c4740 20 client.18132 trim_cache size 0 max 16384 2024-05-27T08:02:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:02:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:02:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:02:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.931+0000 7fbb2d200640 1 -- 172.21.15.90:0/1302214028 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbb0c0b2ae0 msgr2=0x7fbb0c0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbb0c0b2ae0 0x7fbb0c0b4ff0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x562636b94fa0 tx=0x7fbb1802a920 comp rx=0 tx=0).stop 2024-05-27T08:02:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 -- 172.21.15.90:0/1302214028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb28112e50 msgr2=0x7fbb281a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb28112e50 0x7fbb281a5880 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400b3c0 tx=0x7fbb1400b880 comp rx=0 tx=0).stop 2024-05-27T08:02:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 -- 172.21.15.90:0/1302214028 shutdown_connections 2024-05-27T08:02:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fbb0c0b2ae0 0x7fbb0c0b4ff0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb28112e50 0x7fbb281a5880 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb28113910 0x7fbb281a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 --2- 172.21.15.90:0/1302214028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb2810e8c0 0x7fbb281a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 -- 172.21.15.90:0/1302214028 >> 172.21.15.90:0/1302214028 conn(0x7fbb28109ea0 msgr2=0x7fbb281112e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 -- 172.21.15.90:0/1302214028 shutdown_connections 2024-05-27T08:02:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:43.932+0000 7fbb2d200640 1 -- 172.21.15.90:0/1302214028 wait complete. 2024-05-27T08:02:44.384 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:02:45.112 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:45.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:44 smithi090 ceph-mon[31429]: pgmap v525: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-05-27T08:02:45.217 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:44 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:02:45.217 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:44 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:02:45.217 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:44 smithi090 ceph-mon[31429]: pgmap v526: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 446 op/s 2024-05-27T08:02:45.217 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:44 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:02:45.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:44 smithi136 ceph-mon[35324]: pgmap v525: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-05-27T08:02:45.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:44 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:02:45.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:44 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:02:45.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:44 smithi136 ceph-mon[35324]: pgmap v526: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 446 op/s 2024-05-27T08:02:45.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:44 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:02:45.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:44 smithi129 ceph-mon[35254]: pgmap v525: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.9 MiB/s wr, 442 op/s 2024-05-27T08:02:45.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:44 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:02:45.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:44 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:02:45.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:44 smithi129 ceph-mon[35254]: pgmap v526: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.9 MiB/s wr, 446 op/s 2024-05-27T08:02:45.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:44 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:02:47.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:46 smithi136 ceph-mon[35324]: pgmap v527: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2024-05-27T08:02:47.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:46 smithi129 ceph-mon[35254]: pgmap v527: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2024-05-27T08:02:47.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:46 smithi090 ceph-mon[31429]: pgmap v527: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.2 MiB/s wr, 469 op/s 2024-05-27T08:02:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 750.0 s, 387.2 tps, lat 260.259 ms stddev 244.538 2024-05-27T08:02:49.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:48 smithi136 ceph-mon[35324]: pgmap v528: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 438 op/s 2024-05-27T08:02:49.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:48 smithi129 ceph-mon[35254]: pgmap v528: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 438 op/s 2024-05-27T08:02:49.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:48 smithi090 ceph-mon[31429]: pgmap v528: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.9 MiB/s wr, 438 op/s 2024-05-27T08:02:49.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.895+0000 7f7dc5400700 1 -- 172.21.15.90:0/3731557208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7dc0109860 msgr2=0x7f7dc01111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.895+0000 7f7dc5400700 1 --2- 172.21.15.90:0/3731557208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7dc0109860 0x7f7dc01111c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7db800b0d0 tx=0x7f7db800b3e0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.897+0000 7f7dc5400700 1 -- 172.21.15.90:0/3731557208 shutdown_connections 2024-05-27T08:02:49.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.897+0000 7f7dc5400700 1 --2- 172.21.15.90:0/3731557208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7dc0109860 0x7f7dc01111c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.897+0000 7f7dc5400700 1 --2- 172.21.15.90:0/3731557208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7dc01045e0 0x7f7dc01049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.897+0000 7f7dc5400700 1 --2- 172.21.15.90:0/3731557208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dc0108f10 0x7f7dc0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.897+0000 7f7dc5400700 1 -- 172.21.15.90:0/3731557208 >> 172.21.15.90:0/3731557208 conn(0x7f7dc0100030 msgr2=0x7f7dc0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:49.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.898+0000 7f7dc5400700 1 -- 172.21.15.90:0/3731557208 shutdown_connections 2024-05-27T08:02:49.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.898+0000 7f7dc5400700 1 -- 172.21.15.90:0/3731557208 wait complete. 2024-05-27T08:02:49.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.900+0000 7f7dc5400700 1 Processor -- start 2024-05-27T08:02:49.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.901+0000 7f7dc5400700 1 -- start start 2024-05-27T08:02:49.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.902+0000 7f7dc5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dc01045e0 0x7f7dc01a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.902+0000 7f7dc5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7dc0108f10 0x7f7dc01a2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.902+0000 7f7dc5400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7dc0109860 0x7f7dc01ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.902+0000 7f7dc5400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0114480 con 0x7f7dc01045e0 2024-05-27T08:02:49.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.902+0000 7f7dc5400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0114300 con 0x7f7dc0109860 2024-05-27T08:02:49.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.902+0000 7f7dc5400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0114600 con 0x7f7dc0108f10 2024-05-27T08:02:49.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.904+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7dc0108f10 0x7f7dc01a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.905+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7dc0108f10 0x7f7dc01a2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46636/0 (socket says 172.21.15.90:46636) 2024-05-27T08:02:49.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.905+0000 7f7db7e00700 1 -- 172.21.15.90:0/2519476466 learned_addr learned my addr 172.21.15.90:0/2519476466 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:49.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.905+0000 7f7dbe800700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dc01045e0 0x7f7dc01a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.905+0000 7f7dbf200700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7dc0109860 0x7f7dc01ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.906+0000 7f7dbe800700 1 -- 172.21.15.90:0/2519476466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7dc0108f10 msgr2=0x7f7dc01a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.906+0000 7f7dbe800700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7dc0108f10 0x7f7dc01a2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.906+0000 7f7dbe800700 1 -- 172.21.15.90:0/2519476466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7dc0109860 msgr2=0x7f7dc01ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.906+0000 7f7dbe800700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7dc0109860 0x7f7dc01ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.906+0000 7f7dbe800700 1 -- 172.21.15.90:0/2519476466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc01ad720 con 0x7f7dc01045e0 2024-05-27T08:02:49.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.907+0000 7f7db7e00700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7dc0108f10 0x7f7dc01a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:49.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.907+0000 7f7dbe800700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dc01045e0 0x7f7dc01a2330 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7db000b450 tx=0x7f7db000b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.907+0000 7f7db7400700 1 -- 172.21.15.90:0/2519476466 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0015020 con 0x7f7dc01045e0 2024-05-27T08:02:49.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.907+0000 7f7db7400700 1 -- 172.21.15.90:0/2519476466 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7db0011cc0 con 0x7f7dc01045e0 2024-05-27T08:02:49.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.907+0000 7f7db7400700 1 -- 172.21.15.90:0/2519476466 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0010be0 con 0x7f7dc01045e0 2024-05-27T08:02:49.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.908+0000 7f7dc5400700 1 -- 172.21.15.90:0/2519476466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc01ada10 con 0x7f7dc01045e0 2024-05-27T08:02:49.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.908+0000 7f7dc5400700 1 -- 172.21.15.90:0/2519476466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc01adf00 con 0x7f7dc01045e0 2024-05-27T08:02:49.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.914+0000 7f7db7400700 1 -- 172.21.15.90:0/2519476466 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7db0018020 con 0x7f7dc01045e0 2024-05-27T08:02:49.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.914+0000 7f7dd3e62b80 1 Processor -- start 2024-05-27T08:02:49.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.915+0000 7f7dd3e62b80 1 -- start start 2024-05-27T08:02:49.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.915+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c97a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.915+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 0x56008c974870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.915+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c974db0 0x56008c9751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.915+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56008c8a2e40 con 0x56008c974db0 2024-05-27T08:02:49.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.915+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56008c8929e0 con 0x56008c97ad30 2024-05-27T08:02:49.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.915+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56008c896a20 con 0x56008c7f5e40 2024-05-27T08:02:49.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.916+0000 7f7dbf200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c97a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.916+0000 7f7dbf200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c97a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46650/0 (socket says 172.21.15.90:46650) 2024-05-27T08:02:49.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.916+0000 7f7dbf200700 1 -- 172.21.15.90:0/3349788566 learned_addr learned my addr 172.21.15.90:0/3349788566 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:49.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.917+0000 7f7db7e00700 1 --2- 172.21.15.90:0/3349788566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c974db0 0x56008c9751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.918+0000 7f7db7e00700 1 -- 172.21.15.90:0/3349788566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 msgr2=0x56008c97a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.918+0000 7f7db7e00700 1 --2- 172.21.15.90:0/3349788566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c97a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.918+0000 7f7db7e00700 1 -- 172.21.15.90:0/3349788566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 msgr2=0x56008c974870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.918+0000 7f7dbe800700 1 --2- 172.21.15.90:0/3349788566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 0x56008c974870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.919+0000 7f7db7e00700 1 --2- 172.21.15.90:0/3349788566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 0x56008c974870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.919+0000 7f7db7e00700 1 -- 172.21.15.90:0/3349788566 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56008c85b420 con 0x56008c974db0 2024-05-27T08:02:49.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.920+0000 7f7db7e00700 1 --2- 172.21.15.90:0/3349788566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c974db0 0x56008c9751c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009a00 tx=0x7f7da8009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db829453c5a94def server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.923+0000 7f7db7400700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7da40b2aa0 0x7f7da40b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.923+0000 7f7db7400700 1 -- 172.21.15.90:0/2519476466 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f7db00e1ff0 con 0x7f7dc01045e0 2024-05-27T08:02:49.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.924+0000 7f7dafe00700 1 -- 172.21.15.90:0/3349788566 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8017020 con 0x56008c974db0 2024-05-27T08:02:49.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.924+0000 7f7dafe00700 1 -- 172.21.15.90:0/3349788566 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7da800ecf0 con 0x56008c974db0 2024-05-27T08:02:49.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.924+0000 7f7dafe00700 1 -- 172.21.15.90:0/3349788566 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da800f040 con 0x56008c974db0 2024-05-27T08:02:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.925+0000 7f7dbf200700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7da40b2aa0 0x7f7da40b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.926+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3349788566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c974db0 msgr2=0x56008c9751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.926+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/3349788566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c974db0 0x56008c9751c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7da8009a00 tx=0x7f7da8009d10 comp rx=0 tx=0).stop 2024-05-27T08:02:49.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.926+0000 7f7dbf200700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7da40b2aa0 0x7f7da40b4fb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009900 tx=0x7f7db800b040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.927+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3349788566 shutdown_connections 2024-05-27T08:02:49.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.927+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/3349788566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c974db0 0x56008c9751c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.927+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/3349788566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c97a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.927+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/3349788566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 0x56008c974870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.927+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3349788566 >> 172.21.15.90:0/3349788566 conn(0x56008c855fc0 msgr2=0x56008c96f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:49.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.928+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3349788566 shutdown_connections 2024-05-27T08:02:49.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.928+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3349788566 wait complete. 2024-05-27T08:02:49.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.929+0000 7f7dd3e62b80 1 -- start start 2024-05-27T08:02:49.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.929+0000 7f7dd3e62b80 1 Processor -- start 2024-05-27T08:02:49.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.930+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 0x56008c97e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.930+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.930+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c97ad30 0x56008c97f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.930+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56008c798fa0 con 0x56008c7f5e40 2024-05-27T08:02:49.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.930+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56008c833760 con 0x56008c974db0 2024-05-27T08:02:49.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.930+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56008c981800 con 0x56008c97ad30 2024-05-27T08:02:49.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.930+0000 7f7dd3e62b80 10 client.0 _ll_register_callbacks cb 0x56008c83cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:49.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.931+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.932+0000 7f7dbf200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c97ad30 0x56008c97f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.932+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39362/0 (socket says 172.21.15.90:39362) 2024-05-27T08:02:49.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.932+0000 7f7dbe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 0x56008c97e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.932+0000 7f7dbe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 0x56008c97e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34758/0 (socket says 172.21.15.90:34758) 2024-05-27T08:02:49.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.932+0000 7f7dbe800700 1 -- 172.21.15.90:0/2484976628 learned_addr learned my addr 172.21.15.90:0/2484976628 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:49.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.932+0000 7f7dbf200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c97ad30 0x56008c97f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46664/0 (socket says 172.21.15.90:46664) 2024-05-27T08:02:49.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.933+0000 7f7dbe800700 1 -- 172.21.15.90:0/2484976628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c97ad30 msgr2=0x56008c97f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.933+0000 7f7dbe800700 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c97ad30 0x56008c97f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.933+0000 7f7dbe800700 1 -- 172.21.15.90:0/2484976628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 msgr2=0x56008c97eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.933+0000 7f7dbe800700 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.933+0000 7f7dbe800700 1 -- 172.21.15.90:0/2484976628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008c7e28d0 con 0x56008c7f5e40 2024-05-27T08:02:49.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.933+0000 7f7db7e00700 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:49.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.934+0000 7f7dbe800700 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 0x56008c97e5f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7db0000c00 tx=0x7f7db00a8910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481eb1fd8df57c5a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.935+0000 7f7dacc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:49.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.935+0000 7f7dacc00700 1 -- 172.21.15.90:0/2484976628 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db00c3060 con 0x56008c7f5e40 2024-05-27T08:02:49.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.935+0000 7f7dacc00700 1 -- 172.21.15.90:0/2484976628 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f7db00b1ba0 con 0x56008c7f5e40 2024-05-27T08:02:49.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.936+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/2484976628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7da800ecf0 con 0x56008c7f5e40 2024-05-27T08:02:49.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.936+0000 7f7dacc00700 1 -- 172.21.15.90:0/2484976628 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7db0010380 con 0x56008c7f5e40 2024-05-27T08:02:49.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.936+0000 7f7dd3e62b80 10 client.18141 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:49.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.936+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/2484976628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56008c863060 con 0x56008c7f5e40 2024-05-27T08:02:49.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.938+0000 7f7dacc00700 1 -- 172.21.15.90:0/2484976628 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7da800ecf0 con 0x56008c7f5e40 2024-05-27T08:02:49.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.938+0000 7f7dacc00700 1 -- 172.21.15.90:0/2484976628 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f7db00ab030 con 0x56008c7f5e40 2024-05-27T08:02:49.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.938+0000 7f7dd3e62b80 10 client.18141 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:49.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.938+0000 7f7dd3e62b80 10 client.18141 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:02:49.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.938+0000 7f7dd3e62b80 20 client.18141 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:49.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.938+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56008c98e7d0 0x56008c990ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.938+0000 7f7dd3e62b80 4 client.18141 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:49.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.938+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/2484976628 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56008c8a4d90 con 0x56008c98e7d0 2024-05-27T08:02:49.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.939+0000 7f7db7e00700 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56008c98e7d0 0x56008c990ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.940+0000 7f7db7e00700 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56008c98e7d0 0x56008c990ca0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ded777c9ac987376 server_cookie=f0421402c35b1a73 in_seq=0 out_seq=0 2024-05-27T08:02:49.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.940+0000 7f7dacc00700 10 client.18141 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T08:02:49.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.943+0000 7f7dacc00700 1 -- 172.21.15.90:0/2484976628 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56008c8a4d90 con 0x56008c98e7d0 2024-05-27T08:02:49.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.943+0000 7f7dacc00700 10 client.18141 handle_command_reply: tid=0 2024-05-27T08:02:49.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.943+0000 7f7dd3e62b80 1 client.18141 shutdown 2024-05-27T08:02:49.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.944+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/2484976628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 msgr2=0x56008c97e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.944+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 0x56008c97e5f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7db0000c00 tx=0x7f7db00a8910 comp rx=0 tx=0).stop 2024-05-27T08:02:49.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.945+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/2484976628 shutdown_connections 2024-05-27T08:02:49.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.945+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c97ad30 0x56008c97f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.945+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 0x56008c97e5f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.945+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56008c98e7d0 0x56008c990ca0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.945+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/2484976628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.945+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/2484976628 >> 172.21.15.90:0/2484976628 conn(0x56008c855fc0 msgr2=0x56008c96f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:49.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.945+0000 7f7dacc00700 0 client.18141 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T08:02:49.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.945+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/2484976628 shutdown_connections 2024-05-27T08:02:49.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.945+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/2484976628 wait complete. 2024-05-27T08:02:49.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.945+0000 7f7dd3e62b80 20 client.18141 trim_cache size 0 max 16384 2024-05-27T08:02:49.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.976+0000 7f7dd3e62b80 1 Processor -- start 2024-05-27T08:02:49.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.976+0000 7f7dd3e62b80 1 -- start start 2024-05-27T08:02:49.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.976+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c974870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.976+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.976+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c97ad30 0x56008c979d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.976+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01b8830 con 0x56008c97ad30 2024-05-27T08:02:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.976+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01b5880 con 0x56008c974db0 2024-05-27T08:02:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.976+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01b7a90 con 0x56008c7f5e40 2024-05-27T08:02:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.977+0000 7f7dbe800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c974870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.978+0000 7f7dbf200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c97ad30 0x56008c979d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.978+0000 7f7dbe800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c974870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46678/0 (socket says 172.21.15.90:46678) 2024-05-27T08:02:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.978+0000 7f7dbe800700 1 -- 172.21.15.90:0/147301738 learned_addr learned my addr 172.21.15.90:0/147301738 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:49.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.978+0000 7f7db7e00700 1 --2- 172.21.15.90:0/147301738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.978+0000 7f7dbe800700 1 -- 172.21.15.90:0/147301738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 msgr2=0x56008c97d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.978+0000 7f7dbe800700 1 --2- 172.21.15.90:0/147301738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.978+0000 7f7dbe800700 1 -- 172.21.15.90:0/147301738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c97ad30 msgr2=0x56008c979d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.978+0000 7f7dbe800700 1 --2- 172.21.15.90:0/147301738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c97ad30 0x56008c979d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.978+0000 7f7dbe800700 1 -- 172.21.15.90:0/147301738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56008c97a400 con 0x56008c7f5e40 2024-05-27T08:02:49.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.979+0000 7f7db7e00700 1 --2- 172.21.15.90:0/147301738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:49.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.980+0000 7f7dbe800700 1 --2- 172.21.15.90:0/147301738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c974870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7db00a8cc0 tx=0x7f7db000bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e956363a10af1c3f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.980+0000 7f7daf400700 1 -- 172.21.15.90:0/147301738 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db00a8ad0 con 0x56008c7f5e40 2024-05-27T08:02:49.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.980+0000 7f7daf400700 1 -- 172.21.15.90:0/147301738 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7db00c7020 con 0x56008c7f5e40 2024-05-27T08:02:49.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.980+0000 7f7daf400700 1 -- 172.21.15.90:0/147301738 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db00b1ba0 con 0x56008c7f5e40 2024-05-27T08:02:49.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.982+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/147301738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 msgr2=0x56008c974870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:49.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.982+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/147301738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c974870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7db00a8cc0 tx=0x7f7db000bd10 comp rx=0 tx=0).stop 2024-05-27T08:02:49.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.982+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/147301738 shutdown_connections 2024-05-27T08:02:49.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.982+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/147301738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c97ad30 0x56008c979d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.982+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/147301738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c7f5e40 0x56008c974870 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.982+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/147301738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c974db0 0x56008c97d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.982+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/147301738 >> 172.21.15.90:0/147301738 conn(0x56008c855fc0 msgr2=0x56008c97e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:49.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.983+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/147301738 shutdown_connections 2024-05-27T08:02:49.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.983+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/147301738 wait complete. 2024-05-27T08:02:49.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.984+0000 7f7dd3e62b80 1 -- start start 2024-05-27T08:02:49.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.984+0000 7f7dd3e62b80 1 Processor -- start 2024-05-27T08:02:49.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.985+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 0x56008c986a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.986+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c974db0 0x56008c986f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.986+0000 7f7dd3e62b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 0x56008c995450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.986+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01b73c0 con 0x56008c7f5e40 2024-05-27T08:02:49.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.986+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01b5f50 con 0x56008c97ad30 2024-05-27T08:02:49.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.986+0000 7f7dd3e62b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01b6620 con 0x56008c974db0 2024-05-27T08:02:49.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.986+0000 7f7dd3e62b80 10 client.0 _ll_register_callbacks cb 0x56008c83cc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:49.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.987+0000 7f7dbe800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 0x56008c986a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.987+0000 7f7db7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c974db0 0x56008c986f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.987+0000 7f7dbf200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 0x56008c995450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:49.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.987+0000 7f7dbf200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 0x56008c995450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39368/0 (socket says 172.21.15.90:39368) 2024-05-27T08:02:49.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.987+0000 7f7dbf200700 1 -- 172.21.15.90:0/3104773992 learned_addr learned my addr 172.21.15.90:0/3104773992 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:49.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.988+0000 7f7db7e00700 1 -- 172.21.15.90:0/3104773992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 msgr2=0x56008c995450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.988+0000 7f7db7e00700 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 0x56008c995450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.988+0000 7f7db7e00700 1 -- 172.21.15.90:0/3104773992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 msgr2=0x56008c986a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:49.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.988+0000 7f7db7e00700 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 0x56008c986a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:49.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.989+0000 7f7db7e00700 1 -- 172.21.15.90:0/3104773992 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008c995990 con 0x56008c974db0 2024-05-27T08:02:49.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.990+0000 7f7db7e00700 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c974db0 0x56008c986f90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7da8010010 tx=0x7f7da80092b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7800f0e6ab8d9b1d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:49.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.990+0000 7f7dacc00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:02:49.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.993+0000 7f7dacc00700 1 -- 172.21.15.90:0/3104773992 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8017040 con 0x56008c974db0 2024-05-27T08:02:49.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.994+0000 7f7dacc00700 1 -- 172.21.15.90:0/3104773992 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f7da80138f0 con 0x56008c974db0 2024-05-27T08:02:49.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.994+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3104773992 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7db00c7020 con 0x56008c974db0 2024-05-27T08:02:49.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.994+0000 7f7dacc00700 1 -- 172.21.15.90:0/3104773992 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7da8039050 con 0x56008c974db0 2024-05-27T08:02:49.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.995+0000 7f7dd3e62b80 10 client.27907 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:49.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.995+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3104773992 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56008c9f8a10 con 0x56008c974db0 2024-05-27T08:02:49.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.995+0000 7f7dacc00700 1 -- 172.21.15.90:0/3104773992 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7db00c7020 con 0x56008c974db0 2024-05-27T08:02:49.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.996+0000 7f7dacc00700 1 -- 172.21.15.90:0/3104773992 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f7da8026040 con 0x56008c974db0 2024-05-27T08:02:49.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.996+0000 7f7dd3e62b80 10 client.27907 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:49.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.996+0000 7f7dd3e62b80 10 client.27907 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:02:49.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.996+0000 7f7dd3e62b80 20 client.27907 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:49.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.996+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56008c9f90f0 0x56008c9fb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:49.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.996+0000 7f7dd3e62b80 4 client.27907 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:02:49.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.996+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3104773992 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56008c89e570 con 0x56008c9f90f0 2024-05-27T08:02:50.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.997+0000 7f7dbe800700 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56008c9f90f0 0x56008c9fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:50.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.998+0000 7f7dbe800700 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56008c9f90f0 0x56008c9fb510 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a020cf531396186f server_cookie=451683227d9ddcf0 in_seq=0 out_seq=0 2024-05-27T08:02:50.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.998+0000 7f7dacc00700 10 client.27907 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T08:02:50.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.998+0000 7f7dacc00700 1 -- 172.21.15.90:0/3104773992 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56008c89e570 con 0x56008c9f90f0 2024-05-27T08:02:50.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.998+0000 7f7dacc00700 10 client.27907 handle_command_reply: tid=0 2024-05-27T08:02:50.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.998+0000 7f7dd3e62b80 1 client.27907 shutdown 2024-05-27T08:02:50.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3104773992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c974db0 msgr2=0x56008c986f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:50.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c974db0 0x56008c986f90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7da8010010 tx=0x7f7da80092b0 comp rx=0 tx=0).stop 2024-05-27T08:02:50.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3104773992 shutdown_connections 2024-05-27T08:02:50.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56008c9f90f0 0x56008c9fb510 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:50.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56008c97ad30 0x56008c995450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:50.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56008c7f5e40 0x56008c986a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:50.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dd3e62b80 1 --2- 172.21.15.90:0/3104773992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56008c974db0 0x56008c986f90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:50.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3104773992 >> 172.21.15.90:0/3104773992 conn(0x56008c855fc0 msgr2=0x56008c97d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:50.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dacc00700 0 client.27907 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T08:02:50.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3104773992 shutdown_connections 2024-05-27T08:02:50.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:49.999+0000 7f7dd3e62b80 1 -- 172.21.15.90:0/3104773992 wait complete. 2024-05-27T08:02:50.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.001+0000 7f7dd3e62b80 20 client.27907 trim_cache size 0 max 16384 2024-05-27T08:02:50.003 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:02:50.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.001+0000 7f7dc5400700 1 -- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7da40b2aa0 msgr2=0x7f7da40b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:50.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.001+0000 7f7dc5400700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7da40b2aa0 0x7f7da40b4fb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009900 tx=0x7f7db800b040 comp rx=0 tx=0).stop 2024-05-27T08:02:50.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.001+0000 7f7dc5400700 1 -- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dc01045e0 msgr2=0x7f7dc01a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:50.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.001+0000 7f7dc5400700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dc01045e0 0x7f7dc01a2330 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7db000b450 tx=0x7f7db000b760 comp rx=0 tx=0).stop 2024-05-27T08:02:50.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.002+0000 7f7dc5400700 1 -- 172.21.15.90:0/2519476466 shutdown_connections 2024-05-27T08:02:50.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.002+0000 7f7dc5400700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7da40b2aa0 0x7f7da40b4fb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:50.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.002+0000 7f7dc5400700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7dc0109860 0x7f7dc01ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:50.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.002+0000 7f7dc5400700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dc01045e0 0x7f7dc01a2330 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:50.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.002+0000 7f7dc5400700 1 --2- 172.21.15.90:0/2519476466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7dc0108f10 0x7f7dc01a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:50.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.002+0000 7f7dc5400700 1 -- 172.21.15.90:0/2519476466 >> 172.21.15.90:0/2519476466 conn(0x7f7dc0100030 msgr2=0x7f7dc0100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:50.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.002+0000 7f7dc5400700 1 -- 172.21.15.90:0/2519476466 shutdown_connections 2024-05-27T08:02:50.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:50.002+0000 7f7dc5400700 1 -- 172.21.15.90:0/2519476466 wait complete. 2024-05-27T08:02:50.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:50 smithi090 ceph-mon[31429]: pgmap v529: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-05-27T08:02:50.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:50 smithi129 ceph-mon[35254]: pgmap v529: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-05-27T08:02:50.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:50 smithi136 ceph-mon[35324]: pgmap v529: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 473 op/s 2024-05-27T08:02:51.767 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:02:53.170 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:02:53.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:52 smithi136 ceph-mon[35324]: pgmap v530: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 479 op/s 2024-05-27T08:02:53.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:52 smithi129 ceph-mon[35254]: pgmap v530: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 479 op/s 2024-05-27T08:02:53.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:52 smithi090 ceph-mon[31429]: pgmap v530: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 479 op/s 2024-05-27T08:02:53.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 755.0 s, 541.2 tps, lat 187.589 ms stddev 112.354 2024-05-27T08:02:54.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:54 smithi090 ceph-mon[31429]: pgmap v531: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 490 op/s 2024-05-27T08:02:54.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:54 smithi129 ceph-mon[35254]: pgmap v531: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 490 op/s 2024-05-27T08:02:54.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:54 smithi136 ceph-mon[35324]: pgmap v531: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 490 op/s 2024-05-27T08:02:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.212+0000 7fd2cfe00640 1 -- 172.21.15.90:0/4270668197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c810a1f0 msgr2=0x7fd2c810a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.212+0000 7fd2cfe00640 1 --2- 172.21.15.90:0/4270668197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c810a1f0 0x7fd2c810a600 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009860 tx=0x7fd2bc02a3d0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.213+0000 7fd2cfe00640 1 -- 172.21.15.90:0/4270668197 shutdown_connections 2024-05-27T08:02:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.213+0000 7fd2cfe00640 1 --2- 172.21.15.90:0/4270668197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2c811a8e0 0x7fd2c811cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.213+0000 7fd2cfe00640 1 --2- 172.21.15.90:0/4270668197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2c811d240 0x7fd2c811f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.213+0000 7fd2cfe00640 1 --2- 172.21.15.90:0/4270668197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c810a1f0 0x7fd2c810a600 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009860 tx=0x7fd2bc02a3d0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.213+0000 7fd2cfe00640 1 -- 172.21.15.90:0/4270668197 >> 172.21.15.90:0/4270668197 conn(0x7fd2c806e8f0 msgr2=0x7fd2c806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.214+0000 7fd2cfe00640 1 -- 172.21.15.90:0/4270668197 shutdown_connections 2024-05-27T08:02:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.214+0000 7fd2cfe00640 1 -- 172.21.15.90:0/4270668197 wait complete. 2024-05-27T08:02:57.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.216+0000 7fd2cfe00640 1 Processor -- start 2024-05-27T08:02:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.217+0000 7fd2cfe00640 1 -- start start 2024-05-27T08:02:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.219+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2c811a8e0 0x7fd2c81a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.219+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c811d240 0x7fd2c81a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.219+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2c81ade20 0x7fd2c81ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.220+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2c811a8e0 0x7fd2c81a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.220+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2c811a8e0 0x7fd2c81a96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39382/0 (socket says 172.21.15.90:39382) 2024-05-27T08:02:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.220+0000 7fd2cca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c811d240 0x7fd2c81a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.220+0000 7fd2cca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c811d240 0x7fd2c81a9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46708/0 (socket says 172.21.15.90:46708) 2024-05-27T08:02:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.220+0000 7fd2cde00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2c81ade20 0x7fd2c81ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.220+0000 7fd2cd400640 1 -- 172.21.15.90:0/3635752922 learned_addr learned my addr 172.21.15.90:0/3635752922 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.220+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c8122b20 con 0x7fd2c81ade20 2024-05-27T08:02:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.220+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c81229a0 con 0x7fd2c811a8e0 2024-05-27T08:02:57.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.220+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c8122ca0 con 0x7fd2c811d240 2024-05-27T08:02:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.222+0000 7fd2cca00640 1 -- 172.21.15.90:0/3635752922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2c811a8e0 msgr2=0x7fd2c81a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.222+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2c811a8e0 0x7fd2c81a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.222+0000 7fd2cca00640 1 -- 172.21.15.90:0/3635752922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2c81ade20 msgr2=0x7fd2c81ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.222+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2c81ade20 0x7fd2c81ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.222+0000 7fd2cca00640 1 -- 172.21.15.90:0/3635752922 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c81ae930 con 0x7fd2c811d240 2024-05-27T08:02:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.222+0000 7fd2cde00640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2c81ade20 0x7fd2c81ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:02:57.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.223+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c811d240 0x7fd2c81a9c20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400d720 tx=0x7fd2b400dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.225+0000 7fd2c6000640 1 -- 172.21.15.90:0/3635752922 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4017070 con 0x7fd2c811d240 2024-05-27T08:02:57.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.226+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c81aec20 con 0x7fd2c811d240 2024-05-27T08:02:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.226+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c81b6420 con 0x7fd2c811d240 2024-05-27T08:02:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.226+0000 7fd2c6000640 1 -- 172.21.15.90:0/3635752922 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2b400de30 con 0x7fd2c811d240 2024-05-27T08:02:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.227+0000 7fd2c6000640 1 -- 172.21.15.90:0/3635752922 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b40047a0 con 0x7fd2c811d240 2024-05-27T08:02:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.232+0000 7fd2d6dc2740 1 Processor -- start 2024-05-27T08:02:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.232+0000 7fd2d6dc2740 1 -- start start 2024-05-27T08:02:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.233+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b80d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.233+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b4422f0 0x55723b80dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.233+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b44a4d0 0x55723b80e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.233+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55723b77eb90 con 0x55723b44a4d0 2024-05-27T08:02:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.233+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55723b76e730 con 0x55723b4422f0 2024-05-27T08:02:57.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.233+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55723b772770 con 0x55723b46b1b0 2024-05-27T08:02:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.234+0000 7fd2cde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b80d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.234+0000 7fd2cde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b80d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46720/0 (socket says 172.21.15.90:46720) 2024-05-27T08:02:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.234+0000 7fd2cde00640 1 -- 172.21.15.90:0/1832708491 learned_addr learned my addr 172.21.15.90:0/1832708491 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.234+0000 7fd2cde00640 1 -- 172.21.15.90:0/1832708491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b4422f0 msgr2=0x55723b80dc30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:02:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.235+0000 7fd2cd400640 1 --2- 172.21.15.90:0/1832708491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b4422f0 0x55723b80dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.235+0000 7fd2cde00640 1 --2- 172.21.15.90:0/1832708491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b4422f0 0x55723b80dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.235+0000 7fd2cde00640 1 -- 172.21.15.90:0/1832708491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b44a4d0 msgr2=0x55723b80e170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:02:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.235+0000 7fd2cde00640 1 --2- 172.21.15.90:0/1832708491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b44a4d0 0x55723b80e170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.235+0000 7fd2cde00640 1 -- 172.21.15.90:0/1832708491 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55723b7078b0 con 0x55723b46b1b0 2024-05-27T08:02:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.235+0000 7fd2cde00640 1 --2- 172.21.15.90:0/1832708491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b80d6f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000e870 tx=0x7fd2c00095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1614eaf91649af47 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.236+0000 7fd2ba600640 1 -- 172.21.15.90:0/1832708491 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c000ddd0 con 0x55723b46b1b0 2024-05-27T08:02:57.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.236+0000 7fd2ba600640 1 -- 172.21.15.90:0/1832708491 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2c001c070 con 0x55723b46b1b0 2024-05-27T08:02:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.236+0000 7fd2ba600640 1 -- 172.21.15.90:0/1832708491 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c00165e0 con 0x55723b46b1b0 2024-05-27T08:02:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.237+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/1832708491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 msgr2=0x55723b80d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.237+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/1832708491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b80d6f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000e870 tx=0x7fd2c00095c0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.238+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/1832708491 shutdown_connections 2024-05-27T08:02:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.238+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/1832708491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b4422f0 0x55723b80dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.238+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/1832708491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b44a4d0 0x55723b80e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.238+0000 7fd2c6000640 1 -- 172.21.15.90:0/3635752922 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd2b4004220 con 0x7fd2c811d240 2024-05-27T08:02:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.238+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/1832708491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b80d6f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.238+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/1832708491 >> 172.21.15.90:0/1832708491 conn(0x55723b45f300 msgr2=0x55723b80a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.240+0000 7fd2c6000640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd2b00b2aa0 0x7fd2b00b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.241+0000 7fd2cd400640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd2b00b2aa0 0x7fd2b00b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.242+0000 7fd2c6000640 1 -- 172.21.15.90:0/3635752922 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7fd2b40a5870 con 0x7fd2c811d240 2024-05-27T08:02:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.242+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/1832708491 shutdown_connections 2024-05-27T08:02:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.243+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/1832708491 wait complete. 2024-05-27T08:02:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.243+0000 7fd2d6dc2740 1 -- start start 2024-05-27T08:02:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.244+0000 7fd2d6dc2740 1 Processor -- start 2024-05-27T08:02:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.244+0000 7fd2cd400640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd2b00b2aa0 0x7fd2b00b4fb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55723b80b0c0 tx=0x7fd2bc009450 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.244+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b4422f0 0x55723b828a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.244+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b46b1b0 0x55723b828f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.244+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 0x55723b834e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.244+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55723b454790 con 0x55723b46b1b0 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.244+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55723b716bc0 con 0x55723b429d80 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.244+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55723b82ba50 con 0x55723b4422f0 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.244+0000 7fd2d6dc2740 10 client.0 _ll_register_callbacks cb 0x55723b6f4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.245+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b4422f0 0x55723b828a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.245+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b4422f0 0x55723b828a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46722/0 (socket says 172.21.15.90:46722) 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.245+0000 7fd2cd400640 1 -- 172.21.15.90:0/3264109892 learned_addr learned my addr 172.21.15.90:0/3264109892 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.245+0000 7fd2cde00640 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 0x55723b834e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.246+0000 7fd2cd400640 1 -- 172.21.15.90:0/3264109892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 msgr2=0x55723b834e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.246+0000 7fd2cd400640 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 0x55723b834e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.246+0000 7fd2cd400640 1 -- 172.21.15.90:0/3264109892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b46b1b0 msgr2=0x55723b828f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:02:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.247+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b46b1b0 0x55723b828f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.248+0000 7fd2cd400640 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b46b1b0 0x55723b828f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.248+0000 7fd2cd400640 1 -- 172.21.15.90:0/3264109892 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55723b483210 con 0x55723b4422f0 2024-05-27T08:02:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.249+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b46b1b0 0x55723b828f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:57.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:56 smithi136 ceph-mon[35324]: pgmap v532: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 608 op/s 2024-05-27T08:02:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.250+0000 7fd2cd400640 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b4422f0 0x55723b828a00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc034000 tx=0x7fd2bc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45130becb69d8a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.250+0000 7fd2aea00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:02:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.252+0000 7fd2aea00640 1 -- 172.21.15.90:0/3264109892 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc073670 con 0x55723b4422f0 2024-05-27T08:02:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.253+0000 7fd2aea00640 1 -- 172.21.15.90:0/3264109892 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7fd2bc073810 con 0x55723b4422f0 2024-05-27T08:02:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.253+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3264109892 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2c001c070 con 0x55723b4422f0 2024-05-27T08:02:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.254+0000 7fd2aea00640 1 -- 172.21.15.90:0/3264109892 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd2bc073a10 con 0x55723b4422f0 2024-05-27T08:02:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.254+0000 7fd2d6dc2740 10 client.27919 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.254+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3264109892 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55723b54f620 con 0x55723b4422f0 2024-05-27T08:02:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.255+0000 7fd2aea00640 1 -- 172.21.15.90:0/3264109892 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2c001c070 con 0x55723b4422f0 2024-05-27T08:02:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.257+0000 7fd2aea00640 1 -- 172.21.15.90:0/3264109892 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fd2bc044030 con 0x55723b4422f0 2024-05-27T08:02:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.257+0000 7fd2d6dc2740 10 client.27919 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.257+0000 7fd2d6dc2740 10 client.27919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.257+0000 7fd2d6dc2740 20 client.27919 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.257+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55723b3ff1c0 0x55723b83ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.257+0000 7fd2d6dc2740 4 client.27919 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.257+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3264109892 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55723b780ae0 con 0x55723b3ff1c0 2024-05-27T08:02:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.258+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55723b3ff1c0 0x55723b83ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.261+0000 7fd2aea00640 10 client.27919 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.261+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55723b3ff1c0 0x55723b83ad60 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d9975e155b312f server_cookie=bb3ece50c3bde3ac in_seq=0 out_seq=0 2024-05-27T08:02:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.263+0000 7fd2aea00640 1 -- 172.21.15.90:0/3264109892 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55723b780ae0 con 0x55723b3ff1c0 2024-05-27T08:02:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.263+0000 7fd2aea00640 10 client.27919 handle_command_reply: tid=0 2024-05-27T08:02:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.263+0000 7fd2d6dc2740 1 client.27919 shutdown 2024-05-27T08:02:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.264+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3264109892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b4422f0 msgr2=0x55723b828a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.264+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b4422f0 0x55723b828a00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc034000 tx=0x7fd2bc005f90 comp rx=0 tx=0).stop 2024-05-27T08:02:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.265+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3264109892 shutdown_connections 2024-05-27T08:02:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.265+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55723b3ff1c0 0x55723b83ad60 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.265+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 0x55723b834e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.265+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b46b1b0 0x55723b828f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.265+0000 7fd2aea00640 0 client.27919 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.265+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3264109892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b4422f0 0x55723b828a00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.265+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3264109892 >> 172.21.15.90:0/3264109892 conn(0x55723b45f300 msgr2=0x55723b80f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.265+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3264109892 shutdown_connections 2024-05-27T08:02:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.266+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3264109892 wait complete. 2024-05-27T08:02:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.267+0000 7fd2d6dc2740 20 client.27919 trim_cache size 0 max 16384 2024-05-27T08:02:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.299+0000 7fd2d6dc2740 1 Processor -- start 2024-05-27T08:02:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.303+0000 7fd2d6dc2740 1 -- start start 2024-05-27T08:02:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.303+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b429d80 0x55723b82b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.303+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 0x55723b80ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.303+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b46b1b0 0x55723b8226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.303+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a80119b0 con 0x55723b4422f0 2024-05-27T08:02:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.303+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a800ea00 con 0x55723b46b1b0 2024-05-27T08:02:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.303+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8010c10 con 0x55723b429d80 2024-05-27T08:02:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.304+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b429d80 0x55723b82b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.304+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b429d80 0x55723b82b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46726/0 (socket says 172.21.15.90:46726) 2024-05-27T08:02:57.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.304+0000 7fd2cd400640 1 -- 172.21.15.90:0/3344472240 learned_addr learned my addr 172.21.15.90:0/3344472240 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.304+0000 7fd2cde00640 1 --2- 172.21.15.90:0/3344472240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b46b1b0 0x55723b8226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.304+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3344472240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 0x55723b80ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.304+0000 7fd2cd400640 1 -- 172.21.15.90:0/3344472240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b46b1b0 msgr2=0x55723b8226a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.304+0000 7fd2cd400640 1 --2- 172.21.15.90:0/3344472240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b46b1b0 0x55723b8226a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.305+0000 7fd2cd400640 1 -- 172.21.15.90:0/3344472240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 msgr2=0x55723b80ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.305+0000 7fd2cd400640 1 --2- 172.21.15.90:0/3344472240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 0x55723b80ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.305+0000 7fd2cd400640 1 -- 172.21.15.90:0/3344472240 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55723b804050 con 0x55723b429d80 2024-05-27T08:02:57.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.305+0000 7fd2cd400640 1 --2- 172.21.15.90:0/3344472240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b429d80 0x55723b82b130 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc03d9c0 tx=0x7fd2bc005490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c5cb90e85813e9e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.306+0000 7fd2b9c00640 1 -- 172.21.15.90:0/3344472240 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc03dd50 con 0x55723b429d80 2024-05-27T08:02:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.306+0000 7fd2b9c00640 1 -- 172.21.15.90:0/3344472240 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2bc06e070 con 0x55723b429d80 2024-05-27T08:02:57.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.306+0000 7fd2b9c00640 1 -- 172.21.15.90:0/3344472240 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc004330 con 0x55723b429d80 2024-05-27T08:02:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.308+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3344472240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b429d80 msgr2=0x55723b82b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.308+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3344472240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b429d80 0x55723b82b130 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc03d9c0 tx=0x7fd2bc005490 comp rx=0 tx=0).stop 2024-05-27T08:02:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.308+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3344472240 shutdown_connections 2024-05-27T08:02:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.308+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3344472240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b46b1b0 0x55723b8226a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.308+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3344472240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 0x55723b80ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.308+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3344472240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b429d80 0x55723b82b130 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.308+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3344472240 >> 172.21.15.90:0/3344472240 conn(0x55723b45f300 msgr2=0x55723b828cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.309+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3344472240 shutdown_connections 2024-05-27T08:02:57.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.309+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3344472240 wait complete. 2024-05-27T08:02:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.310+0000 7fd2d6dc2740 1 -- start start 2024-05-27T08:02:57.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.310+0000 7fd2d6dc2740 1 Processor -- start 2024-05-27T08:02:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.312+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 0x55723b89b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.312+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 0x55723b89bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.312+0000 7fd2d6dc2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b82c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.312+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8010540 con 0x55723b4422f0 2024-05-27T08:02:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.312+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a800f0d0 con 0x55723b429d80 2024-05-27T08:02:57.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.312+0000 7fd2d6dc2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a800f7a0 con 0x55723b46b1b0 2024-05-27T08:02:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.312+0000 7fd2d6dc2740 10 client.0 _ll_register_callbacks cb 0x7fd2bc0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:57.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.313+0000 7fd2cca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 0x55723b89bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:56 smithi129 ceph-mon[35254]: pgmap v532: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 608 op/s 2024-05-27T08:02:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.313+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 0x55723b89b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.313+0000 7fd2cde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b82c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.313+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 0x55723b89b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:39398/0 (socket says 172.21.15.90:39398) 2024-05-27T08:02:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.313+0000 7fd2cd400640 1 -- 172.21.15.90:0/3249749538 learned_addr learned my addr 172.21.15.90:0/3249749538 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.314+0000 7fd2cca00640 1 -- 172.21.15.90:0/3249749538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 msgr2=0x55723b82c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.315+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b82c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.315+0000 7fd2cca00640 1 -- 172.21.15.90:0/3249749538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 msgr2=0x55723b89b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.315+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 0x55723b89b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.315+0000 7fd2cca00640 1 -- 172.21.15.90:0/3249749538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55723b82c9a0 con 0x55723b4422f0 2024-05-27T08:02:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.315+0000 7fd2cde00640 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b82c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:02:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.315+0000 7fd2cca00640 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 0x55723b89bc50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40df6f0 tx=0x7fd2b40ae750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c38b8c78ef5db8b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.317+0000 7fd2aea00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.317+0000 7fd2aea00640 1 -- 172.21.15.90:0/3249749538 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b40a5a70 con 0x55723b4422f0 2024-05-27T08:02:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.317+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3249749538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2bc06e070 con 0x55723b4422f0 2024-05-27T08:02:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.318+0000 7fd2aea00640 1 -- 172.21.15.90:0/3249749538 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7fd2b40a5510 con 0x55723b4422f0 2024-05-27T08:02:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.319+0000 7fd2aea00640 1 -- 172.21.15.90:0/3249749538 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd2b40a5c10 con 0x55723b4422f0 2024-05-27T08:02:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.319+0000 7fd2d6dc2740 10 client.18159 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.319+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3249749538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55723b8a2d70 con 0x55723b4422f0 2024-05-27T08:02:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.321+0000 7fd2aea00640 1 -- 172.21.15.90:0/3249749538 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2bc06e070 con 0x55723b4422f0 2024-05-27T08:02:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.321+0000 7fd2aea00640 1 -- 172.21.15.90:0/3249749538 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fd2b40df970 con 0x55723b4422f0 2024-05-27T08:02:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.321+0000 7fd2d6dc2740 10 client.18159 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.321+0000 7fd2d6dc2740 10 client.18159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.321+0000 7fd2d6dc2740 20 client.18159 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.321+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55723b8a34e0 0x55723b8a5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.321+0000 7fd2d6dc2740 4 client.18159 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:02:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.321+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3249749538 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55723b77a2c0 con 0x55723b8a34e0 2024-05-27T08:02:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.322+0000 7fd2cd400640 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55723b8a34e0 0x55723b8a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.325+0000 7fd2cd400640 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55723b8a34e0 0x55723b8a5900 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1324013cf50ae7eb server_cookie=971ccd6deb751258 in_seq=0 out_seq=0 2024-05-27T08:02:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:57.325+0000 7fd2aea00640 10 client.18159 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:57.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:56 smithi090 ceph-mon[31429]: pgmap v532: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 608 op/s 2024-05-27T08:02:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.153+0000 7fd2aea00640 1 -- 172.21.15.90:0/3249749538 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55723b77a2c0 con 0x55723b8a34e0 2024-05-27T08:02:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.153+0000 7fd2aea00640 10 client.18159 handle_command_reply: tid=0 2024-05-27T08:02:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.153+0000 7fd2d6dc2740 1 client.18159 shutdown 2024-05-27T08:02:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.154+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3249749538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 msgr2=0x55723b89bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.154+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 0x55723b89bc50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40df6f0 tx=0x7fd2b40ae750 comp rx=0 tx=0).stop 2024-05-27T08:02:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.154+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3249749538 shutdown_connections 2024-05-27T08:02:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.154+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55723b8a34e0 0x55723b8a5900 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:58.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.154+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55723b46b1b0 0x55723b82c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.155+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55723b4422f0 0x55723b89bc50 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.155+0000 7fd2d6dc2740 1 --2- 172.21.15.90:0/3249749538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55723b429d80 0x55723b89b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.155+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3249749538 >> 172.21.15.90:0/3249749538 conn(0x55723b45f300 msgr2=0x55723b828c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.155+0000 7fd2aea00640 0 client.18159 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.155+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3249749538 shutdown_connections 2024-05-27T08:02:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.155+0000 7fd2d6dc2740 1 -- 172.21.15.90:0/3249749538 wait complete. 2024-05-27T08:02:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.156+0000 7fd2d6dc2740 20 client.18159 trim_cache size 0 max 16384 2024-05-27T08:02:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:02:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:02:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:02:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd2b00b2aa0 msgr2=0x7fd2b00b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd2b00b2aa0 0x7fd2b00b4fb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55723b80b0c0 tx=0x7fd2bc009450 comp rx=0 tx=0).stop 2024-05-27T08:02:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c811d240 msgr2=0x7fd2c81a9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c811d240 0x7fd2c81a9c20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400d720 tx=0x7fd2b400dbe0 comp rx=0 tx=0).stop 2024-05-27T08:02:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 shutdown_connections 2024-05-27T08:02:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd2b00b2aa0 0x7fd2b00b4fb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2c81ade20 0x7fd2c81ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c811d240 0x7fd2c81a9c20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 --2- 172.21.15.90:0/3635752922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd2c811a8e0 0x7fd2c81a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:58.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 >> 172.21.15.90:0/3635752922 conn(0x7fd2c806e8f0 msgr2=0x7fd2c811acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.159+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 shutdown_connections 2024-05-27T08:02:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:02:58.160+0000 7fd2cfe00640 1 -- 172.21.15.90:0/3635752922 wait complete. 2024-05-27T08:02:58.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:02:57 smithi136 ceph-mon[35324]: pgmap v533: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 462 op/s 2024-05-27T08:02:58.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:02:57 smithi090 ceph-mon[31429]: pgmap v533: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 462 op/s 2024-05-27T08:02:58.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:02:57 smithi129 ceph-mon[35254]: pgmap v533: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.0 MiB/s wr, 462 op/s 2024-05-27T08:02:58.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 760.0 s, 518.6 tps, lat 192.921 ms stddev 130.379 2024-05-27T08:02:59.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.201+0000 7f0bd1c00700 1 -- 172.21.15.90:0/1771419902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc1043f0 msgr2=0x7f0bcc104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.201+0000 7f0bd1c00700 1 --2- 172.21.15.90:0/1771419902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc1043f0 0x7f0bcc104800 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0099c0 tx=0x7f0bbc009cd0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.201+0000 7f0bd1c00700 1 -- 172.21.15.90:0/1771419902 shutdown_connections 2024-05-27T08:02:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.201+0000 7f0bd1c00700 1 --2- 172.21.15.90:0/1771419902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc109670 0x7f0bcc110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.201+0000 7f0bd1c00700 1 --2- 172.21.15.90:0/1771419902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc1043f0 0x7f0bcc104800 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.201+0000 7f0bd1c00700 1 --2- 172.21.15.90:0/1771419902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc108d20 0x7f0bcc109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.201+0000 7f0bd1c00700 1 -- 172.21.15.90:0/1771419902 >> 172.21.15.90:0/1771419902 conn(0x7f0bcc079d00 msgr2=0x7f0bcc07a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:59.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.203+0000 7f0bd1c00700 1 -- 172.21.15.90:0/1771419902 shutdown_connections 2024-05-27T08:02:59.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.203+0000 7f0bd1c00700 1 -- 172.21.15.90:0/1771419902 wait complete. 2024-05-27T08:02:59.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.204+0000 7f0bd1c00700 1 Processor -- start 2024-05-27T08:02:59.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.205+0000 7f0bd1c00700 1 -- start start 2024-05-27T08:02:59.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.205+0000 7f0bd1c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc1043f0 0x7f0bcc199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.205+0000 7f0bd1c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc108d20 0x7f0bcc199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.205+0000 7f0bd1c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc109670 0x7f0bcc1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.205+0000 7f0bd1c00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1141a0 con 0x7f0bcc1043f0 2024-05-27T08:02:59.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.205+0000 7f0bd1c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc114020 con 0x7f0bcc109670 2024-05-27T08:02:59.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.205+0000 7f0bd1c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc114320 con 0x7f0bcc108d20 2024-05-27T08:02:59.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.206+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc108d20 0x7f0bcc199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.206+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc108d20 0x7f0bcc199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36086/0 (socket says 172.21.15.90:36086) 2024-05-27T08:02:59.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.206+0000 7f0bcb400700 1 -- 172.21.15.90:0/4137775470 learned_addr learned my addr 172.21.15.90:0/4137775470 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:59.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.206+0000 7f0bd1200700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc109670 0x7f0bcc1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.207+0000 7f0bcb400700 1 -- 172.21.15.90:0/4137775470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc109670 msgr2=0x7f0bcc1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:59.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.207+0000 7f0bcb400700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc109670 0x7f0bcc1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.207+0000 7f0bcb400700 1 -- 172.21.15.90:0/4137775470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc1043f0 msgr2=0x7f0bcc199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:59.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.207+0000 7f0bcb400700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc1043f0 0x7f0bcc199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.207+0000 7f0bcb400700 1 -- 172.21.15.90:0/4137775470 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc1a0c10 con 0x7f0bcc108d20 2024-05-27T08:02:59.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.208+0000 7f0bcb400700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc108d20 0x7f0bcc199d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800d980 tx=0x7f0bb800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:59.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.209+0000 7f0bc8c00700 1 -- 172.21.15.90:0/4137775470 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8017070 con 0x7f0bcc108d20 2024-05-27T08:02:59.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.209+0000 7f0bc8c00700 1 -- 172.21.15.90:0/4137775470 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bb8009dc0 con 0x7f0bcc108d20 2024-05-27T08:02:59.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.209+0000 7f0bd1c00700 1 -- 172.21.15.90:0/4137775470 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc1a0f00 con 0x7f0bcc108d20 2024-05-27T08:02:59.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.209+0000 7f0bd1c00700 1 -- 172.21.15.90:0/4137775470 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc1a1490 con 0x7f0bcc108d20 2024-05-27T08:02:59.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.209+0000 7f0bc8c00700 1 -- 172.21.15.90:0/4137775470 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800f3c0 con 0x7f0bcc108d20 2024-05-27T08:02:59.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.218+0000 7f0bc8c00700 1 -- 172.21.15.90:0/4137775470 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0bb8009930 con 0x7f0bcc108d20 2024-05-27T08:02:59.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.220+0000 7f0bc8c00700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a70 0x7f0bb00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.220+0000 7f0bc8c00700 1 -- 172.21.15.90:0/4137775470 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f0bb80e2490 con 0x7f0bcc108d20 2024-05-27T08:02:59.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.222+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a70 0x7f0bb00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.223+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a70 0x7f0bb00b4f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006010 tx=0x7f0bbc000bc0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:59.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.277+0000 7f0be1ceeb80 1 Processor -- start 2024-05-27T08:02:59.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.277+0000 7f0be1ceeb80 1 -- start start 2024-05-27T08:02:59.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.278+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.278+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6410 0x5581e2ec0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.278+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 0x5581e2ec3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.278+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5581e2de4e40 con 0x5581e2eb6410 2024-05-27T08:02:59.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.278+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5581e2dd49e0 con 0x5581e2eb6820 2024-05-27T08:02:59.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.278+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5581e2dd8a20 con 0x5581e2d37e40 2024-05-27T08:02:59.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.279+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.279+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36088/0 (socket says 172.21.15.90:36088) 2024-05-27T08:02:59.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.279+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3644215857 learned_addr learned my addr 172.21.15.90:0/3644215857 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:59.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.279+0000 7f0bcb400700 1 --2- 172.21.15.90:0/3644215857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6410 0x5581e2ec0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.280+0000 7f0bd1200700 1 --2- 172.21.15.90:0/3644215857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 0x5581e2ec3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.281+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3644215857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 msgr2=0x5581e2ec3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:59.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.281+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/3644215857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 0x5581e2ec3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.281+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3644215857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6410 msgr2=0x5581e2ec0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:59.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.281+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/3644215857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6410 0x5581e2ec0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.281+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3644215857 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e2d9d420 con 0x5581e2d37e40 2024-05-27T08:02:59.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.281+0000 7f0bcb400700 1 --2- 172.21.15.90:0/3644215857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6410 0x5581e2ec0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:02:59.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.281+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/3644215857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb5ed0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0046c0 tx=0x7f0bbc01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb197a0d1ed6041b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:59.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.282+0000 7f0bc1600700 1 -- 172.21.15.90:0/3644215857 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc031070 con 0x5581e2d37e40 2024-05-27T08:02:59.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.282+0000 7f0bc1600700 1 -- 172.21.15.90:0/3644215857 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bbc0095d0 con 0x5581e2d37e40 2024-05-27T08:02:59.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.282+0000 7f0bc1600700 1 -- 172.21.15.90:0/3644215857 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc023700 con 0x5581e2d37e40 2024-05-27T08:02:59.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.284+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3644215857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 msgr2=0x5581e2eb5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:59.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.284+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3644215857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb5ed0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0046c0 tx=0x7f0bbc01c000 comp rx=0 tx=0).stop 2024-05-27T08:02:59.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.285+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3644215857 shutdown_connections 2024-05-27T08:02:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.285+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3644215857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 0x5581e2ec3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.285+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3644215857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb5ed0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.285+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3644215857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6410 0x5581e2ec0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.285+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3644215857 >> 172.21.15.90:0/3644215857 conn(0x5581e2d97fc0 msgr2=0x5581e2eb1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.286+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3644215857 shutdown_connections 2024-05-27T08:02:59.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.286+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3644215857 wait complete. 2024-05-27T08:02:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.287+0000 7f0be1ceeb80 1 -- start start 2024-05-27T08:02:59.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.288+0000 7f0be1ceeb80 1 Processor -- start 2024-05-27T08:02:59.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.290+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2d37e40 0x5581e2ec05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.290+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2eb6410 0x5581e2ec0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.291+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 0x5581e2ec1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.291+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5581e2cdafa0 con 0x5581e2d37e40 2024-05-27T08:02:59.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.291+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5581e2d75760 con 0x5581e2eb6820 2024-05-27T08:02:59.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.291+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5581e2ec3930 con 0x5581e2eb6410 2024-05-27T08:02:59.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.292+0000 7f0be1ceeb80 10 client.0 _ll_register_callbacks cb 0x5581e2d7ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:59.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.292+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2d37e40 0x5581e2ec05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.293+0000 7f0bd1200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 0x5581e2ec1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.294+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2eb6410 0x5581e2ec0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.294+0000 7f0bd1200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 0x5581e2ec1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52852/0 (socket says 172.21.15.90:52852) 2024-05-27T08:02:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.294+0000 7f0bd1200700 1 -- 172.21.15.90:0/3098076524 learned_addr learned my addr 172.21.15.90:0/3098076524 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:59.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.294+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2eb6410 0x5581e2ec0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36098/0 (socket says 172.21.15.90:36098) 2024-05-27T08:02:59.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.296+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3098076524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2eb6410 msgr2=0x5581e2ec0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:59.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.296+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2eb6410 0x5581e2ec0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.296+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3098076524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 msgr2=0x5581e2ec1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:59.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.296+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 0x5581e2ec1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.296+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3098076524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e2d248d0 con 0x5581e2d37e40 2024-05-27T08:02:59.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.297+0000 7f0bb6000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:02:59.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.298+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2d37e40 0x5581e2ec05f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc01e040 tx=0x7f0bbc0237a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54891e7a9a6fe45 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.299+0000 7f0bb6000700 1 -- 172.21.15.90:0/3098076524 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc031040 con 0x5581e2d37e40 2024-05-27T08:02:59.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.299+0000 7f0bb6000700 1 -- 172.21.15.90:0/3098076524 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f0bbc023cc0 con 0x5581e2d37e40 2024-05-27T08:02:59.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.299+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3098076524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bbc0095d0 con 0x5581e2d37e40 2024-05-27T08:02:59.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.301+0000 7f0bb6000700 1 -- 172.21.15.90:0/3098076524 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f0bbc022430 con 0x5581e2d37e40 2024-05-27T08:02:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.301+0000 7f0be1ceeb80 10 client.18165 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:59.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.301+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3098076524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5581e2da5060 con 0x5581e2d37e40 2024-05-27T08:02:59.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.302+0000 7f0bb6000700 1 -- 172.21.15.90:0/3098076524 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bbc0095d0 con 0x5581e2d37e40 2024-05-27T08:02:59.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.302+0000 7f0bb6000700 1 -- 172.21.15.90:0/3098076524 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f0bbc02b030 con 0x5581e2d37e40 2024-05-27T08:02:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.302+0000 7f0be1ceeb80 10 client.18165 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.302+0000 7f0be1ceeb80 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.303+0000 7f0be1ceeb80 20 client.18165 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.303+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5581e2ed06f0 0x5581e2ed2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.303+0000 7f0be1ceeb80 4 client.18165 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:02:59.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.303+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3098076524 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581e2de6d90 con 0x5581e2ed06f0 2024-05-27T08:02:59.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.304+0000 7f0bcb400700 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5581e2ed06f0 0x5581e2ed2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.305+0000 7f0bb6000700 10 client.18165 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:59.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.305+0000 7f0bcb400700 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5581e2ed06f0 0x5581e2ed2b10 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a564a69850744ba server_cookie=cfc3588184c53f89 in_seq=0 out_seq=0 2024-05-27T08:02:59.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.308+0000 7f0bb6000700 1 -- 172.21.15.90:0/3098076524 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5581e2de6d90 con 0x5581e2ed06f0 2024-05-27T08:02:59.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.308+0000 7f0bb6000700 10 client.18165 handle_command_reply: tid=0 2024-05-27T08:02:59.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.309+0000 7f0be1ceeb80 1 client.18165 shutdown 2024-05-27T08:02:59.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.310+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3098076524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2d37e40 msgr2=0x5581e2ec05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:59.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.310+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2d37e40 0x5581e2ec05f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc01e040 tx=0x7f0bbc0237a0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.310+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3098076524 shutdown_connections 2024-05-27T08:02:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.310+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5581e2ed06f0 0x5581e2ed2b10 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.310+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6820 0x5581e2ec1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.310+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2d37e40 0x5581e2ec05f0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.310+0000 7f0bb6000700 0 client.18165 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.310+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3098076524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2eb6410 0x5581e2ec0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.310+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3098076524 >> 172.21.15.90:0/3098076524 conn(0x5581e2d97fc0 msgr2=0x5581e2eb0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:59.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.312+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3098076524 shutdown_connections 2024-05-27T08:02:59.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.312+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3098076524 wait complete. 2024-05-27T08:02:59.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.312+0000 7f0be1ceeb80 20 client.18165 trim_cache size 0 max 16384 2024-05-27T08:02:59.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.342+0000 7f0be1ceeb80 1 Processor -- start 2024-05-27T08:02:59.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.343+0000 7f0be1ceeb80 1 -- start start 2024-05-27T08:02:59.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.343+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.343+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 0x5581e2eba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.343+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 0x5581e2ebbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.343+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1c0df0 con 0x5581e2eb6820 2024-05-27T08:02:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.343+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1bde40 con 0x5581e2eb6410 2024-05-27T08:02:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.343+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1c0050 con 0x5581e2d37e40 2024-05-27T08:02:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.344+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.344+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36106/0 (socket says 172.21.15.90:36106) 2024-05-27T08:02:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.344+0000 7f0bcbe00700 1 -- 172.21.15.90:0/1220321884 learned_addr learned my addr 172.21.15.90:0/1220321884 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.344+0000 7f0bcb400700 1 --2- 172.21.15.90:0/1220321884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 0x5581e2eba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.344+0000 7f0bd1200700 1 --2- 172.21.15.90:0/1220321884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 0x5581e2ebbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.345+0000 7f0bcbe00700 1 -- 172.21.15.90:0/1220321884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 msgr2=0x5581e2eba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:59.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.345+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/1220321884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 0x5581e2eba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.345+0000 7f0bcbe00700 1 -- 172.21.15.90:0/1220321884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 msgr2=0x5581e2ebbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:59.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.345+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/1220321884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 0x5581e2ebbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.345+0000 7f0bcbe00700 1 -- 172.21.15.90:0/1220321884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e2eb3350 con 0x5581e2d37e40 2024-05-27T08:02:59.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.345+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/1220321884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb1e30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00fa80 tx=0x7f0bbc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c9e5e6a3019e46c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:59.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.346+0000 7f0bc0c00700 1 -- 172.21.15.90:0/1220321884 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc046070 con 0x5581e2d37e40 2024-05-27T08:02:59.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.346+0000 7f0bc0c00700 1 -- 172.21.15.90:0/1220321884 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bbc03a070 con 0x5581e2d37e40 2024-05-27T08:02:59.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.346+0000 7f0bc0c00700 1 -- 172.21.15.90:0/1220321884 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc031040 con 0x5581e2d37e40 2024-05-27T08:02:59.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.348+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/1220321884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 msgr2=0x5581e2eb1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:59.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.348+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/1220321884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb1e30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00fa80 tx=0x7f0bbc009310 comp rx=0 tx=0).stop 2024-05-27T08:02:59.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.348+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/1220321884 shutdown_connections 2024-05-27T08:02:59.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.348+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/1220321884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 0x5581e2ebbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.348+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/1220321884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2eb1e30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.348+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/1220321884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 0x5581e2eba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.348+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/1220321884 >> 172.21.15.90:0/1220321884 conn(0x5581e2d97fc0 msgr2=0x5581e2ebedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:59.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.348+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/1220321884 shutdown_connections 2024-05-27T08:02:59.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.348+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/1220321884 wait complete. 2024-05-27T08:02:59.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.349+0000 7f0be1ceeb80 1 -- start start 2024-05-27T08:02:59.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.349+0000 7f0be1ceeb80 1 Processor -- start 2024-05-27T08:02:59.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.350+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2ed5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.350+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 0x5581e2ed33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.350+0000 7f0be1ceeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 0x5581e2ed38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.351+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1bf980 con 0x5581e2eb6820 2024-05-27T08:02:59.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.351+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1be510 con 0x5581e2eb6410 2024-05-27T08:02:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.351+0000 7f0be1ceeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc1bebe0 con 0x5581e2d37e40 2024-05-27T08:02:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.351+0000 7f0be1ceeb80 10 client.0 _ll_register_callbacks cb 0x5581e2d7ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:02:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.352+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 0x5581e2ed33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.352+0000 7f0bd1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 0x5581e2ed38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.352+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2ed5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.352+0000 7f0bcbe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2ed5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:36120/0 (socket says 172.21.15.90:36120) 2024-05-27T08:02:59.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.352+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3357469034 learned_addr learned my addr 172.21.15.90:0/3357469034 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:02:59.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.352+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 0x5581e2ed33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:52864/0 (socket says 172.21.15.90:52864) 2024-05-27T08:02:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.352+0000 7f0bd1200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 0x5581e2ed38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46428/0 (socket says 172.21.15.90:46428) 2024-05-27T08:02:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.354+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3357469034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 msgr2=0x5581e2ed33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:59.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.354+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 0x5581e2ed33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.354+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3357469034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 msgr2=0x5581e2ed38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:02:59.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.354+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 0x5581e2ed38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.354+0000 7f0bcbe00700 1 -- 172.21.15.90:0/3357469034 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e2ed3e20 con 0x5581e2d37e40 2024-05-27T08:02:59.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.355+0000 7f0bcbe00700 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2ed5fd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00b9f0 tx=0x7f0bbc023720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=477e30f7bd8a1df9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:02:59.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.355+0000 7f0bb6000700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:02:59.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.356+0000 7f0bb6000700 1 -- 172.21.15.90:0/3357469034 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc04f060 con 0x5581e2d37e40 2024-05-27T08:02:59.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.356+0000 7f0bb6000700 1 -- 172.21.15.90:0/3357469034 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9480+0+0 (secure 0 0 0) 0x7f0bbc00f460 con 0x5581e2d37e40 2024-05-27T08:02:59.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.356+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3357469034 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bbc03a070 con 0x5581e2d37e40 2024-05-27T08:02:59.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.358+0000 7f0bb6000700 1 -- 172.21.15.90:0/3357469034 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f0bbc056050 con 0x5581e2d37e40 2024-05-27T08:02:59.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.358+0000 7f0be1ceeb80 10 client.27943 fetch_fsmap learned FSMap version 64 2024-05-27T08:02:59.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.358+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3357469034 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5581e2f3abf0 con 0x5581e2d37e40 2024-05-27T08:02:59.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.359+0000 7f0bb6000700 1 -- 172.21.15.90:0/3357469034 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bbc03a070 con 0x5581e2d37e40 2024-05-27T08:02:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.359+0000 7f0bb6000700 1 -- 172.21.15.90:0/3357469034 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f0bbc02b040 con 0x5581e2d37e40 2024-05-27T08:02:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.359+0000 7f0be1ceeb80 10 client.27943 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:02:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.359+0000 7f0be1ceeb80 10 client.27943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:02:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.359+0000 7f0be1ceeb80 20 client.27943 populate_metadata read hostname 'smithi090' 2024-05-27T08:02:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.359+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5581e2f3b140 0x5581e2f3d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:02:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.359+0000 7f0be1ceeb80 4 client.27943 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:02:59.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.359+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3357469034 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581e2de0570 con 0x5581e2f3b140 2024-05-27T08:02:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.360+0000 7f0bcb400700 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5581e2f3b140 0x5581e2f3d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:02:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.360+0000 7f0bb6000700 10 client.27943 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.360+0000 7f0bcb400700 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5581e2f3b140 0x5581e2f3d560 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf35cd356d931700 server_cookie=b1328c3a96613b5c in_seq=0 out_seq=0 2024-05-27T08:02:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.361+0000 7f0bb6000700 1 -- 172.21.15.90:0/3357469034 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581e2de0570 con 0x5581e2f3b140 2024-05-27T08:02:59.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.361+0000 7f0bb6000700 10 client.27943 handle_command_reply: tid=0 2024-05-27T08:02:59.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.361+0000 7f0be1ceeb80 1 client.27943 shutdown 2024-05-27T08:02:59.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.362+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3357469034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 msgr2=0x5581e2ed5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:59.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.362+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2ed5fd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00b9f0 tx=0x7f0bbc023720 comp rx=0 tx=0).stop 2024-05-27T08:02:59.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.365+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3357469034 shutdown_connections 2024-05-27T08:02:59.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.365+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5581e2f3b140 0x5581e2f3d560 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.365+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5581e2eb6820 0x5581e2ed38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.365+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581e2d37e40 0x5581e2ed5fd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.365+0000 7f0be1ceeb80 1 --2- 172.21.15.90:0/3357469034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581e2eb6410 0x5581e2ed33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.365+0000 7f0bb6000700 0 client.27943 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:02:59.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.365+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3357469034 >> 172.21.15.90:0/3357469034 conn(0x5581e2d97fc0 msgr2=0x5581e2ecc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:59.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.366+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3357469034 shutdown_connections 2024-05-27T08:02:59.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.366+0000 7f0be1ceeb80 1 -- 172.21.15.90:0/3357469034 wait complete. 2024-05-27T08:02:59.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.367+0000 7f0be1ceeb80 20 client.27943 trim_cache size 0 max 16384 2024-05-27T08:02:59.370 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:02:59.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.368+0000 7f0bd1c00700 1 -- 172.21.15.90:0/4137775470 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a70 msgr2=0x7f0bb00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:59.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.368+0000 7f0bd1c00700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a70 0x7f0bb00b4f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006010 tx=0x7f0bbc000bc0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.368+0000 7f0bd1c00700 1 -- 172.21.15.90:0/4137775470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc108d20 msgr2=0x7f0bcc199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:02:59.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.368+0000 7f0bd1c00700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc108d20 0x7f0bcc199d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800d980 tx=0x7f0bb800dc90 comp rx=0 tx=0).stop 2024-05-27T08:02:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.369+0000 7f0bd1c00700 1 -- 172.21.15.90:0/4137775470 shutdown_connections 2024-05-27T08:02:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.369+0000 7f0bd1c00700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f0bb00b2a70 0x7f0bb00b4f80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.369+0000 7f0bd1c00700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0bcc109670 0x7f0bcc1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.369+0000 7f0bd1c00700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0bcc1043f0 0x7f0bcc199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.369+0000 7f0bd1c00700 1 --2- 172.21.15.90:0/4137775470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bcc108d20 0x7f0bcc199d80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:02:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.369+0000 7f0bd1c00700 1 -- 172.21.15.90:0/4137775470 >> 172.21.15.90:0/4137775470 conn(0x7f0bcc079d00 msgr2=0x7f0bcc102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:02:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.369+0000 7f0bd1c00700 1 -- 172.21.15.90:0/4137775470 shutdown_connections 2024-05-27T08:02:59.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:02:59.370+0000 7f0bd1c00700 1 -- 172.21.15.90:0/4137775470 wait complete. 2024-05-27T08:03:01.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:00 smithi136 ceph-mon[35324]: pgmap v534: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.7 MiB/s wr, 667 op/s 2024-05-27T08:03:01.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:00 smithi129 ceph-mon[35254]: pgmap v534: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.7 MiB/s wr, 667 op/s 2024-05-27T08:03:01.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:00 smithi090 ceph-mon[31429]: pgmap v534: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.7 MiB/s wr, 667 op/s 2024-05-27T08:03:01.526 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:03:01.526 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:03:02.081 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:02.262 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:02 smithi090 ceph-mon[31429]: pgmap v535: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.5 MiB/s wr, 530 op/s 2024-05-27T08:03:02.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:02 smithi129 ceph-mon[35254]: pgmap v535: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.5 MiB/s wr, 530 op/s 2024-05-27T08:03:02.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:02 smithi136 ceph-mon[35324]: pgmap v535: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.5 MiB/s wr, 530 op/s 2024-05-27T08:03:03.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 765.0 s, 592.8 tps, lat 164.436 ms stddev 134.939 2024-05-27T08:03:05.217 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:04 smithi090 ceph-mon[31429]: osdmap e122: 12 total, 12 up, 12 in 2024-05-27T08:03:05.217 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:04 smithi090 ceph-mon[31429]: pgmap v537: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.4 MiB/s wr, 601 op/s 2024-05-27T08:03:05.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:04 smithi136 ceph-mon[35324]: osdmap e122: 12 total, 12 up, 12 in 2024-05-27T08:03:05.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:04 smithi136 ceph-mon[35324]: pgmap v537: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.4 MiB/s wr, 601 op/s 2024-05-27T08:03:05.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:04 smithi129 ceph-mon[35254]: osdmap e122: 12 total, 12 up, 12 in 2024-05-27T08:03:05.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:04 smithi129 ceph-mon[35254]: pgmap v537: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.4 MiB/s wr, 601 op/s 2024-05-27T08:03:06.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:06 smithi129 ceph-mon[35254]: pgmap v538: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.8 MiB/s wr, 572 op/s 2024-05-27T08:03:06.331 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:06 smithi090 ceph-mon[31429]: pgmap v538: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.8 MiB/s wr, 572 op/s 2024-05-27T08:03:06.504 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:06 smithi136 ceph-mon[35324]: pgmap v538: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.8 MiB/s wr, 572 op/s 2024-05-27T08:03:08.217 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:08 smithi090 ceph-mon[31429]: pgmap v539: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.9 MiB/s wr, 564 op/s 2024-05-27T08:03:08.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:08 smithi129 ceph-mon[35254]: pgmap v539: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.9 MiB/s wr, 564 op/s 2024-05-27T08:03:08.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:08 smithi136 ceph-mon[35324]: pgmap v539: 129 pgs: 11 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 89 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.9 MiB/s wr, 564 op/s 2024-05-27T08:03:08.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 770.0 s, 409.6 tps, lat 248.901 ms stddev 404.781 2024-05-27T08:03:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.290+0000 7f3c9fc00700 1 -- 172.21.15.90:0/4182608161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c981034b0 msgr2=0x7f3c981038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.290+0000 7f3c9fc00700 1 --2- 172.21.15.90:0/4182608161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c981034b0 0x7f3c981038c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009920 tx=0x7f3c8c009c30 comp rx=0 tx=0).stop 2024-05-27T08:03:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.291+0000 7f3c9fc00700 1 -- 172.21.15.90:0/4182608161 shutdown_connections 2024-05-27T08:03:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.291+0000 7f3c9fc00700 1 --2- 172.21.15.90:0/4182608161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c98108730 0x7f3c98110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.291+0000 7f3c9fc00700 1 --2- 172.21.15.90:0/4182608161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c981034b0 0x7f3c981038c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.291+0000 7f3c9fc00700 1 --2- 172.21.15.90:0/4182608161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c98107de0 0x7f3c981081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.291+0000 7f3c9fc00700 1 -- 172.21.15.90:0/4182608161 >> 172.21.15.90:0/4182608161 conn(0x7f3c980fedb0 msgr2=0x7f3c981011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:09.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.292+0000 7f3c9fc00700 1 -- 172.21.15.90:0/4182608161 shutdown_connections 2024-05-27T08:03:09.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.293+0000 7f3c9fc00700 1 -- 172.21.15.90:0/4182608161 wait complete. 2024-05-27T08:03:09.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.295+0000 7f3c9fc00700 1 Processor -- start 2024-05-27T08:03:09.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.295+0000 7f3c9fc00700 1 -- start start 2024-05-27T08:03:09.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.297+0000 7f3c9fc00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c981034b0 0x7f3c9819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.297+0000 7f3c9fc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c98107de0 0x7f3c9819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.297+0000 7f3c9fc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c98108730 0x7f3c9819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.299+0000 7f3c9fc00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3c98113260 con 0x7f3c981034b0 2024-05-27T08:03:09.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.299+0000 7f3c9fc00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3c981130e0 con 0x7f3c98107de0 2024-05-27T08:03:09.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.299+0000 7f3c9fc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3c981133e0 con 0x7f3c98108730 2024-05-27T08:03:09.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.299+0000 7f3c9dc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c98108730 0x7f3c9819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.299+0000 7f3c9dc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c98108730 0x7f3c9819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39176/0 (socket says 172.21.15.90:39176) 2024-05-27T08:03:09.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.299+0000 7f3c9dc00700 1 -- 172.21.15.90:0/3809422495 learned_addr learned my addr 172.21.15.90:0/3809422495 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:09.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.300+0000 7f3c97e00700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c98107de0 0x7f3c9819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.300+0000 7f3c9dc00700 1 -- 172.21.15.90:0/3809422495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c98107de0 msgr2=0x7f3c9819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:09.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.300+0000 7f3c9dc00700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c98107de0 0x7f3c9819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.300+0000 7f3c9dc00700 1 -- 172.21.15.90:0/3809422495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c981034b0 msgr2=0x7f3c9819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:09.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.300+0000 7f3c9dc00700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c981034b0 0x7f3c9819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.300+0000 7f3c9dc00700 1 -- 172.21.15.90:0/3809422495 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9819fb10 con 0x7f3c98108730 2024-05-27T08:03:09.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.303+0000 7f3c9dc00700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c98108730 0x7f3c9819f5d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3c90007ee0 tx=0x7f3c9000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:09.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.304+0000 7f3c95600700 1 -- 172.21.15.90:0/3809422495 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000dbd0 con 0x7f3c98108730 2024-05-27T08:03:09.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.304+0000 7f3c9fc00700 1 -- 172.21.15.90:0/3809422495 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9819fe00 con 0x7f3c98108730 2024-05-27T08:03:09.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.304+0000 7f3c95600700 1 -- 172.21.15.90:0/3809422495 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c9000dd70 con 0x7f3c98108730 2024-05-27T08:03:09.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.305+0000 7f3c9fc00700 1 -- 172.21.15.90:0/3809422495 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c981a0280 con 0x7f3c98108730 2024-05-27T08:03:09.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.305+0000 7f3c95600700 1 -- 172.21.15.90:0/3809422495 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000dbd0 con 0x7f3c98108730 2024-05-27T08:03:09.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.318+0000 7f3c95600700 1 -- 172.21.15.90:0/3809422495 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3c9001a020 con 0x7f3c98108730 2024-05-27T08:03:09.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.324+0000 7f3c95600700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3c800b2c70 0x7f3c800b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.324+0000 7f3c95600700 1 -- 172.21.15.90:0/3809422495 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9584+0+0 (secure 0 0 0) 0x7f3c900e2bd0 con 0x7f3c98108730 2024-05-27T08:03:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.326+0000 7f3c9d200700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3c800b2c70 0x7f3c800b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.332+0000 7f3c9d200700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3c800b2c70 0x7f3c800b5180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c005f40 tx=0x7f3c8c009f80 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.334+0000 7f3cae68cb80 1 Processor -- start 2024-05-27T08:03:09.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.335+0000 7f3cae68cb80 1 -- start start 2024-05-27T08:03:09.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.337+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 0x560b5e0353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.338+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5deb6ec0 0x560b5e0358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.339+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e042190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.339+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560b5df63ec0 con 0x560b5deb6ec0 2024-05-27T08:03:09.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.339+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560b5df53a60 con 0x560b5e035ef0 2024-05-27T08:03:09.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.339+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b5df57aa0 con 0x560b5dfa2a70 2024-05-27T08:03:09.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.339+0000 7f3c9d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 0x560b5e0353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.339+0000 7f3c9d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 0x560b5e0353b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39186/0 (socket says 172.21.15.90:39186) 2024-05-27T08:03:09.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.339+0000 7f3c9d200700 1 -- 172.21.15.90:0/1693433964 learned_addr learned my addr 172.21.15.90:0/1693433964 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:09.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.339+0000 7f3c9dc00700 1 --2- 172.21.15.90:0/1693433964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e042190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.340+0000 7f3c9d200700 1 -- 172.21.15.90:0/1693433964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 msgr2=0x560b5e042190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:09.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.340+0000 7f3c9d200700 1 --2- 172.21.15.90:0/1693433964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e042190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.340+0000 7f3c9d200700 1 -- 172.21.15.90:0/1693433964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5deb6ec0 msgr2=0x560b5e0358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:09.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.340+0000 7f3c9d200700 1 --2- 172.21.15.90:0/1693433964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5deb6ec0 0x560b5e0358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.340+0000 7f3c9d200700 1 -- 172.21.15.90:0/1693433964 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5df1c4a0 con 0x560b5dfa2a70 2024-05-27T08:03:09.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.341+0000 7f3c9d200700 1 --2- 172.21.15.90:0/1693433964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 0x560b5e0353b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c016000 tx=0x7f3c8c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee2786b9119eace server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:09.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.343+0000 7f3c8a000700 1 -- 172.21.15.90:0/1693433964 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c00fdf0 con 0x560b5dfa2a70 2024-05-27T08:03:09.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.343+0000 7f3c8a000700 1 -- 172.21.15.90:0/1693433964 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c8c00f870 con 0x560b5dfa2a70 2024-05-27T08:03:09.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.345+0000 7f3cae68cb80 1 -- 172.21.15.90:0/1693433964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 msgr2=0x560b5e0353b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:09.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.345+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/1693433964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 0x560b5e0353b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c016000 tx=0x7f3c8c01c000 comp rx=0 tx=0).stop 2024-05-27T08:03:09.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.345+0000 7f3cae68cb80 1 -- 172.21.15.90:0/1693433964 shutdown_connections 2024-05-27T08:03:09.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.345+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/1693433964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e042190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.345+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/1693433964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 0x560b5e0353b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.345+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/1693433964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5deb6ec0 0x560b5e0358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.345+0000 7f3cae68cb80 1 -- 172.21.15.90:0/1693433964 >> 172.21.15.90:0/1693433964 conn(0x560b5df50410 msgr2=0x560b5e031540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:09.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.345+0000 7f3cae68cb80 1 -- 172.21.15.90:0/1693433964 shutdown_connections 2024-05-27T08:03:09.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.345+0000 7f3cae68cb80 1 -- 172.21.15.90:0/1693433964 wait complete. 2024-05-27T08:03:09.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.346+0000 7f3cae68cb80 1 -- start start 2024-05-27T08:03:09.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.347+0000 7f3cae68cb80 1 Processor -- start 2024-05-27T08:03:09.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.348+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e04e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.348+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5dfa2a70 0x560b5e04e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.348+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e048810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.348+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560b5de5a020 con 0x560b5dfa2a70 2024-05-27T08:03:09.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.348+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560b5def47e0 con 0x560b5e035ef0 2024-05-27T08:03:09.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.348+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b5e042c40 con 0x560b5deb6ec0 2024-05-27T08:03:09.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.348+0000 7f3cae68cb80 10 client.0 _ll_register_callbacks cb 0x560b5defdcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:09.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.349+0000 7f3c9d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e04e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.349+0000 7f3c9d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e04e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39196/0 (socket says 172.21.15.90:39196) 2024-05-27T08:03:09.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.349+0000 7f3c9d200700 1 -- 172.21.15.90:0/871358221 learned_addr learned my addr 172.21.15.90:0/871358221 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:09.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.349+0000 7f3c9d200700 1 -- 172.21.15.90:0/871358221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 msgr2=0x560b5e048810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:03:09.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.349+0000 7f3c9d200700 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e048810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.350+0000 7f3c9d200700 1 -- 172.21.15.90:0/871358221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5dfa2a70 msgr2=0x560b5e04e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:09.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.350+0000 7f3c9d200700 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5dfa2a70 0x560b5e04e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.350+0000 7f3c9d200700 1 -- 172.21.15.90:0/871358221 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5dea3950 con 0x560b5deb6ec0 2024-05-27T08:03:09.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.350+0000 7f3c9d200700 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e04e410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000c00 tx=0x7f3c8c005320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff6f047ba57c642 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:09.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.350+0000 7f3c7ea00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:03:09.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.354+0000 7f3c7ea00700 1 -- 172.21.15.90:0/871358221 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c036060 con 0x560b5deb6ec0 2024-05-27T08:03:09.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.354+0000 7f3c7ea00700 1 -- 172.21.15.90:0/871358221 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9584+0+0 (secure 0 0 0) 0x7f3c8c00b560 con 0x560b5deb6ec0 2024-05-27T08:03:09.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.355+0000 7f3cae68cb80 1 -- 172.21.15.90:0/871358221 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c8c00f870 con 0x560b5deb6ec0 2024-05-27T08:03:09.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.355+0000 7f3c7ea00700 1 -- 172.21.15.90:0/871358221 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3c8c00b760 con 0x560b5deb6ec0 2024-05-27T08:03:09.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.356+0000 7f3cae68cb80 10 client.27955 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:09.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.356+0000 7f3cae68cb80 1 -- 172.21.15.90:0/871358221 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560b5df240e0 con 0x560b5deb6ec0 2024-05-27T08:03:09.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.356+0000 7f3c7ea00700 1 -- 172.21.15.90:0/871358221 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c8c00f870 con 0x560b5deb6ec0 2024-05-27T08:03:09.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.359+0000 7f3c7ea00700 1 -- 172.21.15.90:0/871358221 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f3c8c02d580 con 0x560b5deb6ec0 2024-05-27T08:03:09.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.359+0000 7f3cae68cb80 10 client.27955 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:09.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.359+0000 7f3cae68cb80 10 client.27955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:09.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.359+0000 7f3cae68cb80 20 client.27955 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:09.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.359+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b5e058330 0x560b5e05a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.359+0000 7f3cae68cb80 4 client.27955 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:09.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.359+0000 7f3cae68cb80 1 -- 172.21.15.90:0/871358221 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b5df65e10 con 0x560b5e058330 2024-05-27T08:03:09.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.360+0000 7f3c97e00700 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b5e058330 0x560b5e05a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.362+0000 7f3c97e00700 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b5e058330 0x560b5e05a750 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1236b023419806 server_cookie=6e4a131f49345efb in_seq=0 out_seq=0 2024-05-27T08:03:09.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.362+0000 7f3c7ea00700 10 client.27955 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:09.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.366+0000 7f3c7ea00700 1 -- 172.21.15.90:0/871358221 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560b5df65e10 con 0x560b5e058330 2024-05-27T08:03:09.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.367+0000 7f3c7ea00700 10 client.27955 handle_command_reply: tid=0 2024-05-27T08:03:09.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.367+0000 7f3cae68cb80 1 client.27955 shutdown 2024-05-27T08:03:09.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.367+0000 7f3cae68cb80 1 -- 172.21.15.90:0/871358221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 msgr2=0x560b5e04e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:09.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.367+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e04e410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000c00 tx=0x7f3c8c005320 comp rx=0 tx=0).stop 2024-05-27T08:03:09.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.368+0000 7f3cae68cb80 1 -- 172.21.15.90:0/871358221 shutdown_connections 2024-05-27T08:03:09.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.368+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b5e058330 0x560b5e05a750 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.368+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e048810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.368+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e04e410 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.368+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/871358221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5dfa2a70 0x560b5e04e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.368+0000 7f3c7ea00700 0 client.27955 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:09.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.368+0000 7f3cae68cb80 1 -- 172.21.15.90:0/871358221 >> 172.21.15.90:0/871358221 conn(0x560b5df50410 msgr2=0x560b5e035390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:09.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.368+0000 7f3cae68cb80 1 -- 172.21.15.90:0/871358221 shutdown_connections 2024-05-27T08:03:09.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.368+0000 7f3cae68cb80 1 -- 172.21.15.90:0/871358221 wait complete. 2024-05-27T08:03:09.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.368+0000 7f3cae68cb80 20 client.27955 trim_cache size 0 max 16384 2024-05-27T08:03:09.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.410+0000 7f3cae68cb80 1 Processor -- start 2024-05-27T08:03:09.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.410+0000 7f3cae68cb80 1 -- start start 2024-05-27T08:03:09.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.411+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e0316e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.411+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5dfa2a70 0x560b5e05fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.411+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e060330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.411+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78010a60 con 0x560b5dfa2a70 2024-05-27T08:03:09.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.411+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7800dab0 con 0x560b5e035ef0 2024-05-27T08:03:09.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.411+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7800fcc0 con 0x560b5deb6ec0 2024-05-27T08:03:09.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.411+0000 7f3c9d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e0316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.411+0000 7f3c9d200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e0316e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39200/0 (socket says 172.21.15.90:39200) 2024-05-27T08:03:09.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.411+0000 7f3c9d200700 1 -- 172.21.15.90:0/3849072457 learned_addr learned my addr 172.21.15.90:0/3849072457 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:09.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.412+0000 7f3c9dc00700 1 --2- 172.21.15.90:0/3849072457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e060330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.412+0000 7f3c9d200700 1 -- 172.21.15.90:0/3849072457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 msgr2=0x560b5e060330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:09.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.412+0000 7f3c9d200700 1 --2- 172.21.15.90:0/3849072457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e060330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.412+0000 7f3c9d200700 1 -- 172.21.15.90:0/3849072457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5dfa2a70 msgr2=0x560b5e05fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:09.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.412+0000 7f3c9d200700 1 --2- 172.21.15.90:0/3849072457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5dfa2a70 0x560b5e05fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.412+0000 7f3c9d200700 1 -- 172.21.15.90:0/3849072457 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b5e032080 con 0x560b5deb6ec0 2024-05-27T08:03:09.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.413+0000 7f3c9d200700 1 --2- 172.21.15.90:0/3849072457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e0316e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c02daf0 tx=0x7f3c8c005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13cac15279896f4d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.414+0000 7f3c89600700 1 -- 172.21.15.90:0/3849072457 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c036040 con 0x560b5deb6ec0 2024-05-27T08:03:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.414+0000 7f3c89600700 1 -- 172.21.15.90:0/3849072457 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c8c00f870 con 0x560b5deb6ec0 2024-05-27T08:03:09.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.414+0000 7f3c89600700 1 -- 172.21.15.90:0/3849072457 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c040080 con 0x560b5deb6ec0 2024-05-27T08:03:09.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.416+0000 7f3cae68cb80 1 -- 172.21.15.90:0/3849072457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 msgr2=0x560b5e0316e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:09.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.416+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/3849072457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e0316e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c02daf0 tx=0x7f3c8c005ce0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.416+0000 7f3cae68cb80 1 -- 172.21.15.90:0/3849072457 shutdown_connections 2024-05-27T08:03:09.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.416+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/3849072457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e035ef0 0x560b5e060330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.416+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/3849072457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5deb6ec0 0x560b5e0316e0 secure :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c02daf0 tx=0x7f3c8c005ce0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.416+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/3849072457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5dfa2a70 0x560b5e05fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.416+0000 7f3cae68cb80 1 -- 172.21.15.90:0/3849072457 >> 172.21.15.90:0/3849072457 conn(0x560b5df50410 msgr2=0x560b5e030fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:09.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.419+0000 7f3cae68cb80 1 -- 172.21.15.90:0/3849072457 shutdown_connections 2024-05-27T08:03:09.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.419+0000 7f3cae68cb80 1 -- 172.21.15.90:0/3849072457 wait complete. 2024-05-27T08:03:09.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.420+0000 7f3cae68cb80 1 -- start start 2024-05-27T08:03:09.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.420+0000 7f3cae68cb80 1 Processor -- start 2024-05-27T08:03:09.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.423+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 0x560b5e0506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.423+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5e035ef0 0x560b5e050be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.423+0000 7f3cae68cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e051120 0x560b5e0c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.424+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7800f5f0 con 0x560b5e035ef0 2024-05-27T08:03:09.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.424+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7800e180 con 0x560b5e051120 2024-05-27T08:03:09.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.424+0000 7f3cae68cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7800e850 con 0x560b5dfa2a70 2024-05-27T08:03:09.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.424+0000 7f3cae68cb80 10 client.0 _ll_register_callbacks cb 0x560b5defdcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:09.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.424+0000 7f3c9dc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e051120 0x560b5e0c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.424+0000 7f3c9dc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e051120 0x560b5e0c1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:41698/0 (socket says 172.21.15.90:41698) 2024-05-27T08:03:09.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.424+0000 7f3c9dc00700 1 -- 172.21.15.90:0/4116422915 learned_addr learned my addr 172.21.15.90:0/4116422915 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:09.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.424+0000 7f3c9d200700 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 0x560b5e0506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.425+0000 7f3c9dc00700 1 -- 172.21.15.90:0/4116422915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 msgr2=0x560b5e0506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:09.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.425+0000 7f3c9dc00700 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 0x560b5e0506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.425+0000 7f3c9dc00700 1 -- 172.21.15.90:0/4116422915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5e035ef0 msgr2=0x560b5e050be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:03:09.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.425+0000 7f3c97e00700 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5e035ef0 0x560b5e050be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.425+0000 7f3c9dc00700 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5e035ef0 0x560b5e050be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.425+0000 7f3c9dc00700 1 -- 172.21.15.90:0/4116422915 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b5e0c2280 con 0x560b5e051120 2024-05-27T08:03:09.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.425+0000 7f3cae68cb80 1 -- 172.21.15.90:0/4116422915 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c8c00f870 con 0x560b5e051120 2024-05-27T08:03:09.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.426+0000 7f3c7ea00700 10 client.37880 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:03:09.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.426+0000 7f3c9dc00700 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e051120 0x560b5e0c1cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000d7b0 tx=0x7f3c900b2370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b221bd02c17c732 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:09.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.426+0000 7f3c7ea00700 1 -- 172.21.15.90:0/4116422915 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c900c3060 con 0x560b5e051120 2024-05-27T08:03:09.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.426+0000 7f3c7ea00700 1 -- 172.21.15.90:0/4116422915 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9584+0+0 (secure 0 0 0) 0x7f3c900a6080 con 0x560b5e051120 2024-05-27T08:03:09.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.427+0000 7f3c7ea00700 1 -- 172.21.15.90:0/4116422915 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3c900c3060 con 0x560b5e051120 2024-05-27T08:03:09.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.427+0000 7f3cae68cb80 10 client.37880 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:09.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.427+0000 7f3cae68cb80 1 -- 172.21.15.90:0/4116422915 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({fsmap=64}) -- 0x560b5e0c34a0 con 0x560b5e051120 2024-05-27T08:03:09.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.429+0000 7f3c7ea00700 1 -- 172.21.15.90:0/4116422915 <== mon.2 v2:172.21.15.129:3300/0 4 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f3c900a9cf0 con 0x560b5e051120 2024-05-27T08:03:09.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.430+0000 7f3cae68cb80 10 client.37880 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:09.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.430+0000 7f3cae68cb80 10 client.37880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:09.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.430+0000 7f3cae68cb80 20 client.37880 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:09.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.430+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b5e0c3a40 0x560b5e0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:09.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.430+0000 7f3cae68cb80 4 client.37880 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:03:09.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.430+0000 7f3cae68cb80 1 -- 172.21.15.90:0/4116422915 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560b5df5f5f0 con 0x560b5e0c3a40 2024-05-27T08:03:09.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.431+0000 7f3c9d200700 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b5e0c3a40 0x560b5e0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:09.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.432+0000 7f3c9d200700 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b5e0c3a40 0x560b5e0c5eb0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20b1aee860087d9 server_cookie=248e101f43512ed in_seq=0 out_seq=0 2024-05-27T08:03:09.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.432+0000 7f3c7ea00700 10 client.37880 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:09.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.434+0000 7f3c7ea00700 1 -- 172.21.15.90:0/4116422915 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560b5df5f5f0 con 0x560b5e0c3a40 2024-05-27T08:03:09.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.434+0000 7f3c7ea00700 10 client.37880 handle_command_reply: tid=0 2024-05-27T08:03:09.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.434+0000 7f3cae68cb80 1 client.37880 shutdown 2024-05-27T08:03:09.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.435+0000 7f3cae68cb80 1 -- 172.21.15.90:0/4116422915 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e051120 msgr2=0x560b5e0c1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:09.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.435+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e051120 0x560b5e0c1cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000d7b0 tx=0x7f3c900b2370 comp rx=0 tx=0).stop 2024-05-27T08:03:09.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.435+0000 7f3cae68cb80 1 -- 172.21.15.90:0/4116422915 shutdown_connections 2024-05-27T08:03:09.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.435+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x560b5e0c3a40 0x560b5e0c5eb0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.435+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b5e051120 0x560b5e0c1cc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.435+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b5dfa2a70 0x560b5e0506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.435+0000 7f3cae68cb80 1 --2- 172.21.15.90:0/4116422915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560b5e035ef0 0x560b5e050be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.435+0000 7f3cae68cb80 1 -- 172.21.15.90:0/4116422915 >> 172.21.15.90:0/4116422915 conn(0x560b5df50410 msgr2=0x560b5e046cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:09.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.435+0000 7f3c7ea00700 0 client.37880 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:09.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.435+0000 7f3cae68cb80 1 -- 172.21.15.90:0/4116422915 shutdown_connections 2024-05-27T08:03:09.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.436+0000 7f3cae68cb80 1 -- 172.21.15.90:0/4116422915 wait complete. 2024-05-27T08:03:09.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.436+0000 7f3cae68cb80 20 client.37880 trim_cache size 0 max 16384 2024-05-27T08:03:09.441 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:03:09.441 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T08:03:09.441 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "120a1307-1825-42dc-8dfe-1572b7dc80c9", 2024-05-27T08:03:09.441 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T08:03:09.441 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:03:09.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 -- 172.21.15.90:0/3809422495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3c800b2c70 msgr2=0x7f3c800b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:09.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3c800b2c70 0x7f3c800b5180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c005f40 tx=0x7f3c8c009f80 comp rx=0 tx=0).stop 2024-05-27T08:03:09.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 -- 172.21.15.90:0/3809422495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c98108730 msgr2=0x7f3c9819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:09.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c98108730 0x7f3c9819f5d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3c90007ee0 tx=0x7f3c9000d3b0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 -- 172.21.15.90:0/3809422495 shutdown_connections 2024-05-27T08:03:09.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c98108730 0x7f3c9819f5d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c981034b0 0x7f3c9819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3c800b2c70 0x7f3c800b5180 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c005f40 tx=0x7f3c8c009f80 comp rx=0 tx=0).stop 2024-05-27T08:03:09.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 --2- 172.21.15.90:0/3809422495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3c98107de0 0x7f3c9819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:09.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 -- 172.21.15.90:0/3809422495 >> 172.21.15.90:0/3809422495 conn(0x7f3c980fedb0 msgr2=0x7f3c9806ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:09.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.441+0000 7f3c9fc00700 1 -- 172.21.15.90:0/3809422495 shutdown_connections 2024-05-27T08:03:09.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:09.442+0000 7f3c9fc00700 1 -- 172.21.15.90:0/3809422495 wait complete. 2024-05-27T08:03:10.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:10 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T08:03:10.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:10 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:10.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:10 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T08:03:10.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:10 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:10.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:10 smithi090 ceph-mon[31429]: pgmap v540: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.5 MiB/s wr, 459 op/s 2024-05-27T08:03:10.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:10 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T08:03:10.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:10 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:10.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:10 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T08:03:10.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:10 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:10.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:10 smithi129 ceph-mon[35254]: pgmap v540: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.5 MiB/s wr, 459 op/s 2024-05-27T08:03:10.366 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:03:10.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:10 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T08:03:10.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:10 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:10.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:10 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T08:03:10.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:10 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T08:03:10.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:10 smithi136 ceph-mon[35324]: pgmap v540: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.5 MiB/s wr, 459 op/s 2024-05-27T08:03:11.159 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:11.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.564+0000 7f6c3fe00640 1 -- 172.21.15.90:0/513933230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c38114af0 msgr2=0x7f6c38116f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.564+0000 7f6c3fe00640 1 --2- 172.21.15.90:0/513933230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c38114af0 0x7f6c38116f10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009860 tx=0x7f6c3002a340 comp rx=0 tx=0).stop 2024-05-27T08:03:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.564+0000 7f6c3fe00640 1 -- 172.21.15.90:0/513933230 shutdown_connections 2024-05-27T08:03:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.564+0000 7f6c3fe00640 1 --2- 172.21.15.90:0/513933230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c38114af0 0x7f6c38116f10 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.564+0000 7f6c3fe00640 1 --2- 172.21.15.90:0/513933230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c38112190 0x7f6c381145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.564+0000 7f6c3fe00640 1 --2- 172.21.15.90:0/513933230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c38077610 0x7f6c38077a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.564+0000 7f6c3fe00640 1 -- 172.21.15.90:0/513933230 >> 172.21.15.90:0/513933230 conn(0x7f6c38101520 msgr2=0x7f6c38103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.566+0000 7f6c3fe00640 1 -- 172.21.15.90:0/513933230 shutdown_connections 2024-05-27T08:03:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.566+0000 7f6c3fe00640 1 -- 172.21.15.90:0/513933230 wait complete. 2024-05-27T08:03:11.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.567+0000 7f6c3fe00640 1 Processor -- start 2024-05-27T08:03:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.568+0000 7f6c3fe00640 1 -- start start 2024-05-27T08:03:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.568+0000 7f6c3fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c38077610 0x7f6c381a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.568+0000 7f6c3fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c38112190 0x7f6c381a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.568+0000 7f6c3fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c38114af0 0x7f6c381a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.568+0000 7f6c3fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3811a330 con 0x7f6c38112190 2024-05-27T08:03:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.569+0000 7f6c3fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3811a1b0 con 0x7f6c38114af0 2024-05-27T08:03:11.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.569+0000 7f6c3fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3811a4b0 con 0x7f6c38077610 2024-05-27T08:03:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.569+0000 7f6c3ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c38112190 0x7f6c381a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.569+0000 7f6c3ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c38112190 0x7f6c381a1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46820/0 (socket says 172.21.15.90:46820) 2024-05-27T08:03:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.569+0000 7f6c3ca00640 1 -- 172.21.15.90:0/3332136268 learned_addr learned my addr 172.21.15.90:0/3332136268 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.570+0000 7f6c3de00640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c38114af0 0x7f6c381a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.570+0000 7f6c3ca00640 1 -- 172.21.15.90:0/3332136268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c38077610 msgr2=0x7f6c381a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.570+0000 7f6c3ca00640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c38077610 0x7f6c381a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.570+0000 7f6c3ca00640 1 -- 172.21.15.90:0/3332136268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c38114af0 msgr2=0x7f6c381a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.570+0000 7f6c3ca00640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c38114af0 0x7f6c381a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.570+0000 7f6c3ca00640 1 -- 172.21.15.90:0/3332136268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c381a5c30 con 0x7f6c38112190 2024-05-27T08:03:11.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.571+0000 7f6c3ca00640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c38112190 0x7f6c381a1330 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400ebc0 tx=0x7f6c2400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.571+0000 7f6c36000640 1 -- 172.21.15.90:0/3332136268 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24004d20 con 0x7f6c38112190 2024-05-27T08:03:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.571+0000 7f6c36000640 1 -- 172.21.15.90:0/3332136268 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c240054b0 con 0x7f6c38112190 2024-05-27T08:03:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.571+0000 7f6c36000640 1 -- 172.21.15.90:0/3332136268 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24010760 con 0x7f6c38112190 2024-05-27T08:03:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.572+0000 7f6c3fe00640 1 -- 172.21.15.90:0/3332136268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c381a5f20 con 0x7f6c38112190 2024-05-27T08:03:11.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.572+0000 7f6c3fe00640 1 -- 172.21.15.90:0/3332136268 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c381a6420 con 0x7f6c38112190 2024-05-27T08:03:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.588+0000 7f6c46e50740 1 Processor -- start 2024-05-27T08:03:11.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.589+0000 7f6c46e50740 1 -- start start 2024-05-27T08:03:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.590+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31613b1b0 0x55a3164dd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.590+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3161122f0 0x55a3164ddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.590+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 0x55a3164de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.590+0000 7f6c46e50740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a31644eb90 con 0x55a3161122f0 2024-05-27T08:03:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.590+0000 7f6c46e50740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a31643e730 con 0x55a31613b1b0 2024-05-27T08:03:11.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.590+0000 7f6c46e50740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a316442770 con 0x55a31611a4d0 2024-05-27T08:03:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.591+0000 7f6c36000640 1 -- 172.21.15.90:0/3332136268 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6c24010900 con 0x7f6c38112190 2024-05-27T08:03:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.591+0000 7f6c3de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31613b1b0 0x55a3164dd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.591+0000 7f6c3de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31613b1b0 0x55a3164dd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:41720/0 (socket says 172.21.15.90:41720) 2024-05-27T08:03:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.591+0000 7f6c3de00640 1 -- 172.21.15.90:0/2690744074 learned_addr learned my addr 172.21.15.90:0/2690744074 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.592+0000 7f6c3d400640 1 --2- 172.21.15.90:0/2690744074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3161122f0 0x55a3164ddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.592+0000 7f6c3ca00640 1 --2- 172.21.15.90:0/2690744074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 0x55a3164de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.593+0000 7f6c3de00640 1 -- 172.21.15.90:0/2690744074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 msgr2=0x55a3164de2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.593+0000 7f6c3de00640 1 --2- 172.21.15.90:0/2690744074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 0x55a3164de2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.593+0000 7f6c3de00640 1 -- 172.21.15.90:0/2690744074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3161122f0 msgr2=0x55a3164ddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.593+0000 7f6c3de00640 1 --2- 172.21.15.90:0/2690744074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3161122f0 0x55a3164ddda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.593+0000 7f6c3de00640 1 -- 172.21.15.90:0/2690744074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3163d78b0 con 0x55a31613b1b0 2024-05-27T08:03:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.593+0000 7f6c3ca00640 1 --2- 172.21.15.90:0/2690744074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 0x55a3164de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:11.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.593+0000 7f6c3de00640 1 --2- 172.21.15.90:0/2690744074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31613b1b0 0x55a3164dd860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009780 tx=0x7f6c300095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a344cad0e5ae066e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.594+0000 7f6c36000640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c1c0b2a20 0x7f6c1c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.594+0000 7f6c36000640 1 -- 172.21.15.90:0/3332136268 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9584+0+0 (secure 0 0 0) 0x7f6c240e33c0 con 0x7f6c38112190 2024-05-27T08:03:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.594+0000 7f6c2e600640 1 -- 172.21.15.90:0/2690744074 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30040020 con 0x55a31613b1b0 2024-05-27T08:03:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.594+0000 7f6c2e600640 1 -- 172.21.15.90:0/2690744074 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c300044d0 con 0x55a31613b1b0 2024-05-27T08:03:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.594+0000 7f6c2e600640 1 -- 172.21.15.90:0/2690744074 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30037040 con 0x55a31613b1b0 2024-05-27T08:03:11.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.595+0000 7f6c3d400640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c1c0b2a20 0x7f6c1c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.596+0000 7f6c46e50740 1 -- 172.21.15.90:0/2690744074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31613b1b0 msgr2=0x55a3164dd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.596+0000 7f6c46e50740 1 --2- 172.21.15.90:0/2690744074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31613b1b0 0x55a3164dd860 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009780 tx=0x7f6c300095a0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.596+0000 7f6c3d400640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c1c0b2a20 0x7f6c1c0b4f30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005ab0 tx=0x7f6c28005a20 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.599+0000 7f6c46e50740 1 -- 172.21.15.90:0/2690744074 shutdown_connections 2024-05-27T08:03:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.599+0000 7f6c46e50740 1 --2- 172.21.15.90:0/2690744074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 0x55a3164de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.599+0000 7f6c46e50740 1 --2- 172.21.15.90:0/2690744074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3161122f0 0x55a3164ddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.599+0000 7f6c46e50740 1 --2- 172.21.15.90:0/2690744074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31613b1b0 0x55a3164dd860 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.599+0000 7f6c46e50740 1 -- 172.21.15.90:0/2690744074 >> 172.21.15.90:0/2690744074 conn(0x55a31612f300 msgr2=0x55a3164da010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.599+0000 7f6c46e50740 1 -- 172.21.15.90:0/2690744074 shutdown_connections 2024-05-27T08:03:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.599+0000 7f6c46e50740 1 -- 172.21.15.90:0/2690744074 wait complete. 2024-05-27T08:03:11.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.600+0000 7f6c46e50740 1 -- start start 2024-05-27T08:03:11.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.601+0000 7f6c46e50740 1 Processor -- start 2024-05-27T08:03:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.601+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 0x55a3164e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.602+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 0x55a3164e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.602+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31613b1b0 0x55a3164f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.602+0000 7f6c46e50740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a316124790 con 0x55a31613b1b0 2024-05-27T08:03:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.602+0000 7f6c46e50740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a3163e6bc0 con 0x55a3161122f0 2024-05-27T08:03:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.602+0000 7f6c46e50740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a3164ea930 con 0x55a31611a4d0 2024-05-27T08:03:11.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.602+0000 7f6c46e50740 10 client.0 _ll_register_callbacks cb 0x55a3163c4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.602+0000 7f6c3d400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 0x55a3164e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.602+0000 7f6c3d400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 0x55a3164e79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:41732/0 (socket says 172.21.15.90:41732) 2024-05-27T08:03:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.602+0000 7f6c3d400640 1 -- 172.21.15.90:0/3629052114 learned_addr learned my addr 172.21.15.90:0/3629052114 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.602+0000 7f6c3ca00640 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 0x55a3164e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.603+0000 7f6c3de00640 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31613b1b0 0x55a3164f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.603+0000 7f6c3d400640 1 -- 172.21.15.90:0/3629052114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 msgr2=0x55a3164e7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.603+0000 7f6c3d400640 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 0x55a3164e7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.603+0000 7f6c3d400640 1 -- 172.21.15.90:0/3629052114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31613b1b0 msgr2=0x55a3164f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.603+0000 7f6c3d400640 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31613b1b0 0x55a3164f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.603+0000 7f6c3d400640 1 -- 172.21.15.90:0/3629052114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a316153210 con 0x55a3161122f0 2024-05-27T08:03:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.603+0000 7f6c3ca00640 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 0x55a3164e7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.604+0000 7f6c3d400640 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 0x55a3164e79f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005fd0 tx=0x7f6c28009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4666c724f19ece82 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:11.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.604+0000 7f6c22a00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:03:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.605+0000 7f6c22a00640 1 -- 172.21.15.90:0/3629052114 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28023020 con 0x55a3161122f0 2024-05-27T08:03:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.605+0000 7f6c22a00640 1 -- 172.21.15.90:0/3629052114 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9584+0+0 (secure 0 0 0) 0x7f6c28004250 con 0x55a3161122f0 2024-05-27T08:03:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.605+0000 7f6c46e50740 1 -- 172.21.15.90:0/3629052114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c300044d0 con 0x55a3161122f0 2024-05-27T08:03:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.606+0000 7f6c22a00640 1 -- 172.21.15.90:0/3629052114 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f6c2802e050 con 0x55a3161122f0 2024-05-27T08:03:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.606+0000 7f6c46e50740 10 client.37889 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:11.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.606+0000 7f6c46e50740 1 -- 172.21.15.90:0/3629052114 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a31621f620 con 0x55a3161122f0 2024-05-27T08:03:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.606+0000 7f6c22a00640 1 -- 172.21.15.90:0/3629052114 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c300044d0 con 0x55a3161122f0 2024-05-27T08:03:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.607+0000 7f6c22a00640 1 -- 172.21.15.90:0/3629052114 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f6c2800f040 con 0x55a3161122f0 2024-05-27T08:03:11.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.607+0000 7f6c46e50740 10 client.37889 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.607+0000 7f6c46e50740 10 client.37889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.607+0000 7f6c46e50740 20 client.37889 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.607+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a3160f9d80 0x55a31650ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.607+0000 7f6c46e50740 4 client.37889 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.607+0000 7f6c46e50740 1 -- 172.21.15.90:0/3629052114 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a316450ae0 con 0x55a3160f9d80 2024-05-27T08:03:11.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.608+0000 7f6c3ca00640 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a3160f9d80 0x55a31650ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.610+0000 7f6c22a00640 10 client.37889 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:11.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.610+0000 7f6c3ca00640 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a3160f9d80 0x55a31650ae00 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e00920096e12821 server_cookie=811c02b6a77e7c3b in_seq=0 out_seq=0 2024-05-27T08:03:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.614+0000 7f6c22a00640 1 -- 172.21.15.90:0/3629052114 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a316450ae0 con 0x55a3160f9d80 2024-05-27T08:03:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.614+0000 7f6c22a00640 10 client.37889 handle_command_reply: tid=0 2024-05-27T08:03:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.614+0000 7f6c46e50740 1 client.37889 shutdown 2024-05-27T08:03:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.614+0000 7f6c46e50740 1 -- 172.21.15.90:0/3629052114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 msgr2=0x55a3164e79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.614+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 0x55a3164e79f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005fd0 tx=0x7f6c28009260 comp rx=0 tx=0).stop 2024-05-27T08:03:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.615+0000 7f6c46e50740 1 -- 172.21.15.90:0/3629052114 shutdown_connections 2024-05-27T08:03:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.615+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a3160f9d80 0x55a31650ae00 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.615+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31613b1b0 0x55a3164f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.615+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31611a4d0 0x55a3164e7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.615+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3629052114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 0x55a3164e79f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.615+0000 7f6c46e50740 1 -- 172.21.15.90:0/3629052114 >> 172.21.15.90:0/3629052114 conn(0x55a31612f300 msgr2=0x55a3164dd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.615+0000 7f6c22a00640 0 client.37889 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.615+0000 7f6c46e50740 1 -- 172.21.15.90:0/3629052114 shutdown_connections 2024-05-27T08:03:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.616+0000 7f6c46e50740 1 -- 172.21.15.90:0/3629052114 wait complete. 2024-05-27T08:03:11.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.616+0000 7f6c46e50740 20 client.37889 trim_cache size 0 max 16384 2024-05-27T08:03:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.646+0000 7f6c46e50740 1 Processor -- start 2024-05-27T08:03:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.647+0000 7f6c46e50740 1 -- start start 2024-05-27T08:03:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.648+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 0x55a3164ea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.648+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31611a4d0 0x55a3164ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.648+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164d9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.648+0000 7f6c46e50740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38070180 con 0x55a31611a4d0 2024-05-27T08:03:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.648+0000 7f6c46e50740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3806d1d0 con 0x55a3161122f0 2024-05-27T08:03:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.648+0000 7f6c46e50740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3806f3e0 con 0x55a31613b1b0 2024-05-27T08:03:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.648+0000 7f6c3ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31611a4d0 0x55a3164ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.648+0000 7f6c3ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31611a4d0 0x55a3164ddd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46846/0 (socket says 172.21.15.90:46846) 2024-05-27T08:03:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.648+0000 7f6c3ca00640 1 -- 172.21.15.90:0/3280188117 learned_addr learned my addr 172.21.15.90:0/3280188117 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.648+0000 7f6c3d400640 1 --2- 172.21.15.90:0/3280188117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 0x55a3164ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.649+0000 7f6c3de00640 1 --2- 172.21.15.90:0/3280188117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164d9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.649+0000 7f6c3de00640 1 -- 172.21.15.90:0/3280188117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 msgr2=0x55a3164ea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.649+0000 7f6c3de00640 1 --2- 172.21.15.90:0/3280188117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 0x55a3164ea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.649+0000 7f6c3de00640 1 -- 172.21.15.90:0/3280188117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31611a4d0 msgr2=0x55a3164ddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.649+0000 7f6c3de00640 1 --2- 172.21.15.90:0/3280188117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31611a4d0 0x55a3164ddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.649+0000 7f6c3de00640 1 -- 172.21.15.90:0/3280188117 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3164e8290 con 0x55a31613b1b0 2024-05-27T08:03:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.650+0000 7f6c3de00640 1 --2- 172.21.15.90:0/3280188117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164d9330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6c30005d80 tx=0x7f6c30005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab4f956b3e79aed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:11.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.651+0000 7f6c2dc00640 1 -- 172.21.15.90:0/3280188117 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30037070 con 0x55a31613b1b0 2024-05-27T08:03:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.651+0000 7f6c2dc00640 1 -- 172.21.15.90:0/3280188117 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c30030df0 con 0x55a31613b1b0 2024-05-27T08:03:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.651+0000 7f6c2dc00640 1 -- 172.21.15.90:0/3280188117 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30040040 con 0x55a31613b1b0 2024-05-27T08:03:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.653+0000 7f6c46e50740 1 -- 172.21.15.90:0/3280188117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 msgr2=0x55a3164d9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.653+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3280188117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164d9330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6c30005d80 tx=0x7f6c30005fb0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.653+0000 7f6c46e50740 1 -- 172.21.15.90:0/3280188117 shutdown_connections 2024-05-27T08:03:11.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.653+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3280188117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164d9330 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.653+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3280188117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a31611a4d0 0x55a3164ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.653+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3280188117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a3161122f0 0x55a3164ea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.654+0000 7f6c46e50740 1 -- 172.21.15.90:0/3280188117 >> 172.21.15.90:0/3280188117 conn(0x55a31612f300 msgr2=0x55a3164dd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.654+0000 7f6c46e50740 1 -- 172.21.15.90:0/3280188117 shutdown_connections 2024-05-27T08:03:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.654+0000 7f6c46e50740 1 -- 172.21.15.90:0/3280188117 wait complete. 2024-05-27T08:03:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.655+0000 7f6c46e50740 1 -- start start 2024-05-27T08:03:11.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.655+0000 7f6c46e50740 1 Processor -- start 2024-05-27T08:03:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.656+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3161122f0 0x55a3164f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.656+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31611a4d0 0x55a3164eab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.656+0000 7f6c46e50740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.656+0000 7f6c46e50740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3806ed10 con 0x55a3161122f0 2024-05-27T08:03:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.657+0000 7f6c3de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.657+0000 7f6c3de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164eb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39276/0 (socket says 172.21.15.90:39276) 2024-05-27T08:03:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.657+0000 7f6c3de00640 1 -- 172.21.15.90:0/3735093360 learned_addr learned my addr 172.21.15.90:0/3735093360 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.657+0000 7f6c46e50740 1 -- 172.21.15.90:0/3735093360 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3806d8a0 con 0x55a31611a4d0 2024-05-27T08:03:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.657+0000 7f6c46e50740 1 -- 172.21.15.90:0/3735093360 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3806df70 con 0x55a31613b1b0 2024-05-27T08:03:11.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.657+0000 7f6c46e50740 10 client.0 _ll_register_callbacks cb 0x7f6c28016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.657+0000 7f6c3d400640 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3161122f0 0x55a3164f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.657+0000 7f6c3ca00640 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31611a4d0 0x55a3164eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.658+0000 7f6c3de00640 1 -- 172.21.15.90:0/3735093360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31611a4d0 msgr2=0x55a3164eab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.658+0000 7f6c3de00640 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31611a4d0 0x55a3164eab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.658+0000 7f6c3de00640 1 -- 172.21.15.90:0/3735093360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3161122f0 msgr2=0x55a3164f0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.658+0000 7f6c3de00640 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3161122f0 0x55a3164f0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.658+0000 7f6c3de00640 1 -- 172.21.15.90:0/3735093360 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3164eb7b0 con 0x55a31613b1b0 2024-05-27T08:03:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.659+0000 7f6c3de00640 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164eb0c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6c30034040 tx=0x7f6c30030720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39812f08ba69f1e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.659+0000 7f6c22a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:03:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.660+0000 7f6c22a00640 1 -- 172.21.15.90:0/3735093360 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30037070 con 0x55a31613b1b0 2024-05-27T08:03:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.660+0000 7f6c46e50740 1 -- 172.21.15.90:0/3735093360 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c30030df0 con 0x55a31613b1b0 2024-05-27T08:03:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.660+0000 7f6c22a00640 1 -- 172.21.15.90:0/3735093360 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9584+0+0 (secure 0 0 0) 0x7f6c30004260 con 0x55a31613b1b0 2024-05-27T08:03:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.660+0000 7f6c22a00640 1 -- 172.21.15.90:0/3735093360 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f6c30059050 con 0x55a31613b1b0 2024-05-27T08:03:11.663 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.660+0000 7f6c46e50740 10 client.27970 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.660+0000 7f6c46e50740 1 -- 172.21.15.90:0/3735093360 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a316572330 con 0x55a31613b1b0 2024-05-27T08:03:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.661+0000 7f6c22a00640 1 -- 172.21.15.90:0/3735093360 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6c30030df0 con 0x55a31613b1b0 2024-05-27T08:03:11.664 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.662+0000 7f6c22a00640 1 -- 172.21.15.90:0/3735093360 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f6c30004a30 con 0x55a31613b1b0 2024-05-27T08:03:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.662+0000 7f6c46e50740 10 client.27970 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.662+0000 7f6c46e50740 10 client.27970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.662+0000 7f6c46e50740 20 client.27970 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.662+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a3160cf1c0 0x55a316574860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.662+0000 7f6c46e50740 4 client.27970 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:03:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.662+0000 7f6c46e50740 1 -- 172.21.15.90:0/3735093360 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a31644a2c0 con 0x55a3160cf1c0 2024-05-27T08:03:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.663+0000 7f6c3d400640 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a3160cf1c0 0x55a316574860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.666+0000 7f6c3d400640 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a3160cf1c0 0x55a316574860 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44370d2ce3a8e0ee server_cookie=97178e15ba86bae3 in_seq=0 out_seq=0 2024-05-27T08:03:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:11.666+0000 7f6c22a00640 10 client.27970 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:12.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:12 smithi129 ceph-mon[35254]: pgmap v541: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.2 MiB/s wr, 499 op/s 2024-05-27T08:03:12.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:12 smithi090 ceph-mon[31429]: pgmap v541: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.2 MiB/s wr, 499 op/s 2024-05-27T08:03:12.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:12 smithi136 ceph-mon[35324]: pgmap v541: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 7.2 MiB/s wr, 499 op/s 2024-05-27T08:03:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c22a00640 1 -- 172.21.15.90:0/3735093360 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a31644a2c0 con 0x55a3160cf1c0 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c22a00640 10 client.27970 handle_command_reply: tid=0 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c46e50740 1 client.27970 shutdown 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c46e50740 1 -- 172.21.15.90:0/3735093360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 msgr2=0x55a3164eb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164eb0c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6c30034040 tx=0x7f6c30030720 comp rx=0 tx=0).stop 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c46e50740 1 -- 172.21.15.90:0/3735093360 shutdown_connections 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55a3160cf1c0 0x55a316574860 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a31611a4d0 0x55a3164eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a31613b1b0 0x55a3164eb0c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c46e50740 1 --2- 172.21.15.90:0/3735093360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a3161122f0 0x55a3164f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c46e50740 1 -- 172.21.15.90:0/3735093360 >> 172.21.15.90:0/3735093360 conn(0x55a31612f300 msgr2=0x55a3164e7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c22a00640 0 client.27970 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.915+0000 7f6c46e50740 1 -- 172.21.15.90:0/3735093360 shutdown_connections 2024-05-27T08:03:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.916+0000 7f6c46e50740 1 -- 172.21.15.90:0/3735093360 wait complete. 2024-05-27T08:03:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.916+0000 7f6c46e50740 20 client.27970 trim_cache size 0 max 16384 2024-05-27T08:03:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:03:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:03:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:03:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.917+0000 7f6c3fe00640 1 -- 172.21.15.90:0/3332136268 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c1c0b2a20 msgr2=0x7f6c1c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.917+0000 7f6c3fe00640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c1c0b2a20 0x7f6c1c0b4f30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005ab0 tx=0x7f6c28005a20 comp rx=0 tx=0).stop 2024-05-27T08:03:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.917+0000 7f6c3fe00640 1 -- 172.21.15.90:0/3332136268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c38112190 msgr2=0x7f6c381a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.917+0000 7f6c3fe00640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c38112190 0x7f6c381a1330 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400ebc0 tx=0x7f6c2400c6a0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.917+0000 7f6c3fe00640 1 -- 172.21.15.90:0/3332136268 shutdown_connections 2024-05-27T08:03:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.917+0000 7f6c3fe00640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f6c1c0b2a20 0x7f6c1c0b4f30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.917+0000 7f6c3fe00640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c38114af0 0x7f6c381a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.917+0000 7f6c3fe00640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c38112190 0x7f6c381a1330 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.917+0000 7f6c3fe00640 1 --2- 172.21.15.90:0/3332136268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c38077610 0x7f6c381a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.917+0000 7f6c3fe00640 1 -- 172.21.15.90:0/3332136268 >> 172.21.15.90:0/3332136268 conn(0x7f6c38101520 msgr2=0x7f6c38113510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.918+0000 7f6c3fe00640 1 -- 172.21.15.90:0/3332136268 shutdown_connections 2024-05-27T08:03:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:12.918+0000 7f6c3fe00640 1 -- 172.21.15.90:0/3332136268 wait complete. 2024-05-27T08:03:13.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:13 smithi090 ceph-mon[31429]: osdmap e123: 12 total, 12 up, 12 in 2024-05-27T08:03:13.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:13 smithi136 ceph-mon[35324]: osdmap e123: 12 total, 12 up, 12 in 2024-05-27T08:03:13.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:13 smithi129 ceph-mon[35254]: osdmap e123: 12 total, 12 up, 12 in 2024-05-27T08:03:13.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 775.0 s, 444.8 tps, lat 224.497 ms stddev 151.948 2024-05-27T08:03:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:14 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T08:03:14.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:14 smithi090 ceph-mon[31429]: pgmap v543: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.3 MiB/s wr, 533 op/s 2024-05-27T08:03:14.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:14 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T08:03:14.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:14 smithi136 ceph-mon[35324]: pgmap v543: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.3 MiB/s wr, 533 op/s 2024-05-27T08:03:14.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:14 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T08:03:14.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:14 smithi129 ceph-mon[35254]: pgmap v543: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.3 MiB/s wr, 533 op/s 2024-05-27T08:03:16.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:16 smithi090 ceph-mon[31429]: pgmap v544: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 7.1 MiB/s wr, 566 op/s 2024-05-27T08:03:16.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:16 smithi129 ceph-mon[35254]: pgmap v544: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 7.1 MiB/s wr, 566 op/s 2024-05-27T08:03:16.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:16 smithi136 ceph-mon[35324]: pgmap v544: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 7.1 MiB/s wr, 566 op/s 2024-05-27T08:03:18.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:18 smithi129 ceph-mon[35254]: pgmap v545: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 7.5 MiB/s wr, 575 op/s 2024-05-27T08:03:18.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:18 smithi090 ceph-mon[31429]: pgmap v545: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 7.5 MiB/s wr, 575 op/s 2024-05-27T08:03:18.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:18 smithi136 ceph-mon[35324]: pgmap v545: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 7.5 MiB/s wr, 575 op/s 2024-05-27T08:03:18.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 780.0 s, 520.8 tps, lat 188.399 ms stddev 194.142 2024-05-27T08:03:19.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.232+0000 7f395b200700 1 -- 172.21.15.90:0/3450311238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f395409a860 msgr2=0x7f395409ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:19.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.232+0000 7f395b200700 1 --2- 172.21.15.90:0/3450311238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f395409a860 0x7f395409ac70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f395c06a3b0 tx=0x7f395c064d30 comp rx=0 tx=0).stop 2024-05-27T08:03:19.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.233+0000 7f395b200700 1 -- 172.21.15.90:0/3450311238 shutdown_connections 2024-05-27T08:03:19.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.233+0000 7f395b200700 1 --2- 172.21.15.90:0/3450311238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3954095180 0x7f39540a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.233+0000 7f395b200700 1 --2- 172.21.15.90:0/3450311238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f395409a860 0x7f395409ac70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.233+0000 7f395b200700 1 --2- 172.21.15.90:0/3450311238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3954094830 0x7f3954094c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.233+0000 7f395b200700 1 -- 172.21.15.90:0/3450311238 >> 172.21.15.90:0/3450311238 conn(0x7f395400a390 msgr2=0x7f395400a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:19.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.233+0000 7f395b200700 1 -- 172.21.15.90:0/3450311238 shutdown_connections 2024-05-27T08:03:19.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.233+0000 7f395b200700 1 -- 172.21.15.90:0/3450311238 wait complete. 2024-05-27T08:03:19.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.235+0000 7f395b200700 1 Processor -- start 2024-05-27T08:03:19.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.236+0000 7f395b200700 1 -- start start 2024-05-27T08:03:19.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.236+0000 7f395b200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3954094830 0x7f395412ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.236+0000 7f395b200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3954095180 0x7f395412b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.237+0000 7f395b200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395409a860 0x7f395412f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.237+0000 7f395b200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f39540a4430 con 0x7f3954095180 2024-05-27T08:03:19.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.237+0000 7f395b200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39540a42b0 con 0x7f395409a860 2024-05-27T08:03:19.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.237+0000 7f395b200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f39540a45b0 con 0x7f3954094830 2024-05-27T08:03:19.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.237+0000 7f3959400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3954095180 0x7f395412b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.237+0000 7f3959e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3954094830 0x7f395412ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.237+0000 7f3959e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3954094830 0x7f395412ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44222/0 (socket says 172.21.15.90:44222) 2024-05-27T08:03:19.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.237+0000 7f3959e00700 1 -- 172.21.15.90:0/893110598 learned_addr learned my addr 172.21.15.90:0/893110598 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:19.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.237+0000 7f395a800700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395409a860 0x7f395412f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.238+0000 7f3959e00700 1 -- 172.21.15.90:0/893110598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395409a860 msgr2=0x7f395412f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:19.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.238+0000 7f3959e00700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395409a860 0x7f395412f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.238+0000 7f3959e00700 1 -- 172.21.15.90:0/893110598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3954095180 msgr2=0x7f395412b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:19.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.238+0000 7f3959e00700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3954095180 0x7f395412b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.238+0000 7f3959e00700 1 -- 172.21.15.90:0/893110598 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395412fc60 con 0x7f3954094830 2024-05-27T08:03:19.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.238+0000 7f3959e00700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3954094830 0x7f395412ae40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f395c02ec00 tx=0x7f395c0519b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:19.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.239+0000 7f3952a00700 1 -- 172.21.15.90:0/893110598 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c093020 con 0x7f3954094830 2024-05-27T08:03:19.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.239+0000 7f3952a00700 1 -- 172.21.15.90:0/893110598 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f395c06ad60 con 0x7f3954094830 2024-05-27T08:03:19.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.239+0000 7f3952a00700 1 -- 172.21.15.90:0/893110598 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c07a450 con 0x7f3954094830 2024-05-27T08:03:19.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.240+0000 7f395b200700 1 -- 172.21.15.90:0/893110598 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395412fef0 con 0x7f3954094830 2024-05-27T08:03:19.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.240+0000 7f395b200700 1 -- 172.21.15.90:0/893110598 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3954130350 con 0x7f3954094830 2024-05-27T08:03:19.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.247+0000 7f3952a00700 1 -- 172.21.15.90:0/893110598 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f395c096020 con 0x7f3954094830 2024-05-27T08:03:19.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.249+0000 7f3952a00700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f39400b2b40 0x7f39400b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.249+0000 7f3952a00700 1 -- 172.21.15.90:0/893110598 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f395c1594e0 con 0x7f3954094830 2024-05-27T08:03:19.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.250+0000 7f3959400700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f39400b2b40 0x7f39400b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.252+0000 7f3959400700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f39400b2b40 0x7f39400b5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3954092600 tx=0x7f394c00a380 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:19.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.262+0000 7f396fc46b80 1 Processor -- start 2024-05-27T08:03:19.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.262+0000 7f396fc46b80 1 -- start start 2024-05-27T08:03:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.263+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70f7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.263+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f70f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.263+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 0x55b6f71036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.263+0000 7f396fc46b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b6f7024e40 con 0x55b6f6f77e40 2024-05-27T08:03:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.263+0000 7f396fc46b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b6f70149e0 con 0x55b6f70639f0 2024-05-27T08:03:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.263+0000 7f396fc46b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b6f7018a20 con 0x55b6f71012c0 2024-05-27T08:03:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.263+0000 7f3959400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f70f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.264+0000 7f3959400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f70f7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36108/0 (socket says 172.21.15.90:36108) 2024-05-27T08:03:19.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.264+0000 7f3959400700 1 -- 172.21.15.90:0/1163207862 learned_addr learned my addr 172.21.15.90:0/1163207862 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:19.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.264+0000 7f395a800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 0x55b6f71036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.264+0000 7f3959e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.266+0000 7f3959400700 1 -- 172.21.15.90:0/1163207862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 msgr2=0x55b6f71036e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:19.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.266+0000 7f3959400700 1 --2- 172.21.15.90:0/1163207862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 0x55b6f71036e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.266+0000 7f3959400700 1 -- 172.21.15.90:0/1163207862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 msgr2=0x55b6f70f7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:19.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.266+0000 7f3959400700 1 --2- 172.21.15.90:0/1163207862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70f7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.266+0000 7f3959400700 1 -- 172.21.15.90:0/1163207862 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f6fdd420 con 0x55b6f6f77e40 2024-05-27T08:03:19.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.267+0000 7f3959400700 1 --2- 172.21.15.90:0/1163207862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f70f7c90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f394c00fe70 tx=0x7f394c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808759a1e9ffdc90 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:19.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.267+0000 7f394b400700 1 -- 172.21.15.90:0/1163207862 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c016070 con 0x55b6f6f77e40 2024-05-27T08:03:19.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.268+0000 7f394b400700 1 -- 172.21.15.90:0/1163207862 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f394c014070 con 0x55b6f6f77e40 2024-05-27T08:03:19.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.268+0000 7f394b400700 1 -- 172.21.15.90:0/1163207862 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c010c90 con 0x55b6f6f77e40 2024-05-27T08:03:19.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.269+0000 7f396fc46b80 1 -- 172.21.15.90:0/1163207862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 msgr2=0x55b6f70f7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:19.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.270+0000 7f396fc46b80 1 --2- 172.21.15.90:0/1163207862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f70f7c90 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f394c00fe70 tx=0x7f394c012000 comp rx=0 tx=0).stop 2024-05-27T08:03:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.271+0000 7f396fc46b80 1 -- 172.21.15.90:0/1163207862 shutdown_connections 2024-05-27T08:03:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.271+0000 7f396fc46b80 1 --2- 172.21.15.90:0/1163207862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 0x55b6f71036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.271+0000 7f396fc46b80 1 --2- 172.21.15.90:0/1163207862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70f7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.271+0000 7f396fc46b80 1 --2- 172.21.15.90:0/1163207862 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f70f7c90 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.271+0000 7f396fc46b80 1 -- 172.21.15.90:0/1163207862 >> 172.21.15.90:0/1163207862 conn(0x55b6f7011390 msgr2=0x55b6f70f2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:19.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.271+0000 7f396fc46b80 1 -- 172.21.15.90:0/1163207862 shutdown_connections 2024-05-27T08:03:19.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.271+0000 7f396fc46b80 1 -- 172.21.15.90:0/1163207862 wait complete. 2024-05-27T08:03:19.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.272+0000 7f396fc46b80 1 -- start start 2024-05-27T08:03:19.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.272+0000 7f396fc46b80 1 Processor -- start 2024-05-27T08:03:19.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.273+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f6f77e40 0x55b6f70fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.273+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70fe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.273+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f71012c0 0x55b6f70fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.273+0000 7f396fc46b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b6f6f1afa0 con 0x55b6f71012c0 2024-05-27T08:03:19.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.273+0000 7f396fc46b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b6f6fb5760 con 0x55b6f70639f0 2024-05-27T08:03:19.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.273+0000 7f396fc46b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b6f7114e10 con 0x55b6f6f77e40 2024-05-27T08:03:19.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.273+0000 7f396fc46b80 10 client.0 _ll_register_callbacks cb 0x55b6f6fbec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:19.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.275+0000 7f395a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f71012c0 0x55b6f70fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.275+0000 7f3959400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.275+0000 7f3959400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70fe790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43476/0 (socket says 172.21.15.90:43476) 2024-05-27T08:03:19.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.275+0000 7f3959400700 1 -- 172.21.15.90:0/3416003158 learned_addr learned my addr 172.21.15.90:0/3416003158 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:19.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.275+0000 7f3959e00700 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f6f77e40 0x55b6f70fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.276+0000 7f395a800700 1 -- 172.21.15.90:0/3416003158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f6f77e40 msgr2=0x55b6f70fe250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:19.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.276+0000 7f395a800700 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f6f77e40 0x55b6f70fe250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.276+0000 7f395a800700 1 -- 172.21.15.90:0/3416003158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 msgr2=0x55b6f70fe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:19.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.276+0000 7f395a800700 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70fe790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.276+0000 7f395a800700 1 -- 172.21.15.90:0/3416003158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f6f648d0 con 0x55b6f71012c0 2024-05-27T08:03:19.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.277+0000 7f393fe00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:03:19.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.277+0000 7f395a800700 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f71012c0 0x55b6f70fecd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3944005fd0 tx=0x7f3944004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e933e0eef5260d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:19.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.278+0000 7f393fe00700 1 -- 172.21.15.90:0/3416003158 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944003a90 con 0x55b6f71012c0 2024-05-27T08:03:19.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.279+0000 7f393fe00700 1 -- 172.21.15.90:0/3416003158 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f3944003890 con 0x55b6f71012c0 2024-05-27T08:03:19.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.279+0000 7f396fc46b80 1 -- 172.21.15.90:0/3416003158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f394c014070 con 0x55b6f71012c0 2024-05-27T08:03:19.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.279+0000 7f393fe00700 1 -- 172.21.15.90:0/3416003158 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f3944003a90 con 0x55b6f71012c0 2024-05-27T08:03:19.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.280+0000 7f396fc46b80 10 client.18201 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:19.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.280+0000 7f396fc46b80 1 -- 172.21.15.90:0/3416003158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b6f6fe5060 con 0x55b6f71012c0 2024-05-27T08:03:19.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.281+0000 7f393fe00700 1 -- 172.21.15.90:0/3416003158 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f394401e070 con 0x55b6f71012c0 2024-05-27T08:03:19.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.281+0000 7f393fe00700 1 -- 172.21.15.90:0/3416003158 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f3944023450 con 0x55b6f71012c0 2024-05-27T08:03:19.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.282+0000 7f396fc46b80 10 client.18201 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:19.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.282+0000 7f396fc46b80 10 client.18201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:19.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.282+0000 7f396fc46b80 20 client.18201 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:19.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.282+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b6f7122120 0x55b6f71245b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.282+0000 7f396fc46b80 4 client.18201 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:19.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.283+0000 7f396fc46b80 1 -- 172.21.15.90:0/3416003158 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f394c014070 con 0x55b6f7122120 2024-05-27T08:03:19.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.283+0000 7f3959e00700 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b6f7122120 0x55b6f71245b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.284+0000 7f393fe00700 10 client.18201 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:19.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.284+0000 7f3959e00700 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b6f7122120 0x55b6f71245b0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2135ee103269a5fd server_cookie=3cd58fe4de5e88e5 in_seq=0 out_seq=0 2024-05-27T08:03:19.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.286+0000 7f393fe00700 1 -- 172.21.15.90:0/3416003158 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x7f394c014070 con 0x55b6f7122120 2024-05-27T08:03:19.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.286+0000 7f393fe00700 10 client.18201 handle_command_reply: tid=0 2024-05-27T08:03:19.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.286+0000 7f396fc46b80 1 client.18201 shutdown 2024-05-27T08:03:19.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.287+0000 7f396fc46b80 1 -- 172.21.15.90:0/3416003158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f71012c0 msgr2=0x55b6f70fecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:19.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.287+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f71012c0 0x55b6f70fecd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3944005fd0 tx=0x7f3944004320 comp rx=0 tx=0).stop 2024-05-27T08:03:19.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.287+0000 7f396fc46b80 1 -- 172.21.15.90:0/3416003158 shutdown_connections 2024-05-27T08:03:19.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.287+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b6f7122120 0x55b6f71245b0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.287+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f71012c0 0x55b6f70fecd0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.287+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f6f77e40 0x55b6f70fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.287+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3416003158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70fe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.287+0000 7f396fc46b80 1 -- 172.21.15.90:0/3416003158 >> 172.21.15.90:0/3416003158 conn(0x55b6f7011390 msgr2=0x55b6f70f2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:19.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.287+0000 7f393fe00700 0 client.18201 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:19.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.287+0000 7f396fc46b80 1 -- 172.21.15.90:0/3416003158 shutdown_connections 2024-05-27T08:03:19.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.288+0000 7f396fc46b80 1 -- 172.21.15.90:0/3416003158 wait complete. 2024-05-27T08:03:19.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.288+0000 7f396fc46b80 20 client.18201 trim_cache size 0 max 16384 2024-05-27T08:03:19.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.327+0000 7f396fc46b80 1 Processor -- start 2024-05-27T08:03:19.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.328+0000 7f396fc46b80 1 -- start start 2024-05-27T08:03:19.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.328+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f711deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.328+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70f6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.328+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 0x55b6f7129db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.328+0000 7f396fc46b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3954141f00 con 0x55b6f6f77e40 2024-05-27T08:03:19.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.328+0000 7f396fc46b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f395413ef50 con 0x55b6f70639f0 2024-05-27T08:03:19.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.328+0000 7f396fc46b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3954141160 con 0x55b6f71012c0 2024-05-27T08:03:19.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.328+0000 7f3959e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f711deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.329+0000 7f3959e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f711deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36118/0 (socket says 172.21.15.90:36118) 2024-05-27T08:03:19.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.329+0000 7f3959e00700 1 -- 172.21.15.90:0/2532367883 learned_addr learned my addr 172.21.15.90:0/2532367883 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:19.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.329+0000 7f3959400700 1 --2- 172.21.15.90:0/2532367883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.329+0000 7f395a800700 1 --2- 172.21.15.90:0/2532367883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 0x55b6f7129db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.330+0000 7f3959e00700 1 -- 172.21.15.90:0/2532367883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 msgr2=0x55b6f7129db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:19.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.330+0000 7f3959e00700 1 --2- 172.21.15.90:0/2532367883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 0x55b6f7129db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.330+0000 7f3959e00700 1 -- 172.21.15.90:0/2532367883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 msgr2=0x55b6f70f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:19.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.330+0000 7f3959e00700 1 --2- 172.21.15.90:0/2532367883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.330+0000 7f3959e00700 1 -- 172.21.15.90:0/2532367883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f7121ef0 con 0x55b6f6f77e40 2024-05-27T08:03:19.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.331+0000 7f395a800700 1 --2- 172.21.15.90:0/2532367883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 0x55b6f7129db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:19.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.331+0000 7f3959e00700 1 --2- 172.21.15.90:0/2532367883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f711deb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f395c0525c0 tx=0x7f395c06af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56880aa420c65ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:19.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.331+0000 7f394aa00700 1 -- 172.21.15.90:0/2532367883 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c120c90 con 0x55b6f6f77e40 2024-05-27T08:03:19.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.332+0000 7f394aa00700 1 -- 172.21.15.90:0/2532367883 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f395c120590 con 0x55b6f6f77e40 2024-05-27T08:03:19.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.332+0000 7f394aa00700 1 -- 172.21.15.90:0/2532367883 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c1590c0 con 0x55b6f6f77e40 2024-05-27T08:03:19.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.333+0000 7f396fc46b80 1 -- 172.21.15.90:0/2532367883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 msgr2=0x55b6f711deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:19.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.333+0000 7f396fc46b80 1 --2- 172.21.15.90:0/2532367883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f711deb0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f395c0525c0 tx=0x7f395c06af10 comp rx=0 tx=0).stop 2024-05-27T08:03:19.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.333+0000 7f396fc46b80 1 -- 172.21.15.90:0/2532367883 shutdown_connections 2024-05-27T08:03:19.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.333+0000 7f396fc46b80 1 --2- 172.21.15.90:0/2532367883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f71012c0 0x55b6f7129db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.333+0000 7f396fc46b80 1 --2- 172.21.15.90:0/2532367883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f6f77e40 0x55b6f711deb0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.333+0000 7f396fc46b80 1 --2- 172.21.15.90:0/2532367883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f70639f0 0x55b6f70f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.333+0000 7f396fc46b80 1 -- 172.21.15.90:0/2532367883 >> 172.21.15.90:0/2532367883 conn(0x55b6f7011390 msgr2=0x55b6f70f6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:19.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.334+0000 7f396fc46b80 1 -- 172.21.15.90:0/2532367883 shutdown_connections 2024-05-27T08:03:19.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.334+0000 7f396fc46b80 1 -- 172.21.15.90:0/2532367883 wait complete. 2024-05-27T08:03:19.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.335+0000 7f396fc46b80 1 -- start start 2024-05-27T08:03:19.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.335+0000 7f396fc46b80 1 Processor -- start 2024-05-27T08:03:19.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.336+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f70639f0 0x55b6f711a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.336+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f71012c0 0x55b6f711a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.336+0000 7f396fc46b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f7185c60 0x55b6f7186070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.336+0000 7f396fc46b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3954140a90 con 0x55b6f7185c60 2024-05-27T08:03:19.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.336+0000 7f396fc46b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f395413f620 con 0x55b6f71012c0 2024-05-27T08:03:19.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.336+0000 7f396fc46b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f395413fcf0 con 0x55b6f70639f0 2024-05-27T08:03:19.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.336+0000 7f396fc46b80 10 client.0 _ll_register_callbacks cb 0x55b6f6fbec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:19.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.337+0000 7f3959e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f70639f0 0x55b6f711a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.337+0000 7f3959e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f70639f0 0x55b6f711a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44250/0 (socket says 172.21.15.90:44250) 2024-05-27T08:03:19.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.337+0000 7f3959e00700 1 -- 172.21.15.90:0/3900881233 learned_addr learned my addr 172.21.15.90:0/3900881233 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:19.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.337+0000 7f3959400700 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f71012c0 0x55b6f711a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.337+0000 7f395a800700 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f7185c60 0x55b6f7186070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.338+0000 7f3959e00700 1 -- 172.21.15.90:0/3900881233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f71012c0 msgr2=0x55b6f711a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:19.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.338+0000 7f3959e00700 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f71012c0 0x55b6f711a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.338+0000 7f3959e00700 1 -- 172.21.15.90:0/3900881233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f7185c60 msgr2=0x55b6f7186070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:19.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.338+0000 7f3959e00700 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f7185c60 0x55b6f7186070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.338+0000 7f3959e00700 1 -- 172.21.15.90:0/3900881233 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f7188960 con 0x55b6f70639f0 2024-05-27T08:03:19.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.339+0000 7f3959400700 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f71012c0 0x55b6f711a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:19.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.339+0000 7f395a800700 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f7185c60 0x55b6f7186070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:19.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.340+0000 7f3959e00700 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f70639f0 0x55b6f711a0a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f395c11d800 tx=0x7f395c11c580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e30124a73421bb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:19.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.340+0000 7f393fe00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:03:19.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.340+0000 7f393fe00700 1 -- 172.21.15.90:0/3900881233 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c120c90 con 0x55b6f70639f0 2024-05-27T08:03:19.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.341+0000 7f393fe00700 1 -- 172.21.15.90:0/3900881233 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f395c06a8d0 con 0x55b6f70639f0 2024-05-27T08:03:19.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.341+0000 7f396fc46b80 1 -- 172.21.15.90:0/3900881233 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f395c120590 con 0x55b6f70639f0 2024-05-27T08:03:19.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.342+0000 7f393fe00700 1 -- 172.21.15.90:0/3900881233 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f395c06aad0 con 0x55b6f70639f0 2024-05-27T08:03:19.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.342+0000 7f396fc46b80 10 client.27985 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:19.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.343+0000 7f396fc46b80 1 -- 172.21.15.90:0/3900881233 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b6f719e4d0 con 0x55b6f70639f0 2024-05-27T08:03:19.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.343+0000 7f393fe00700 1 -- 172.21.15.90:0/3900881233 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f395c120590 con 0x55b6f70639f0 2024-05-27T08:03:19.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.347+0000 7f393fe00700 1 -- 172.21.15.90:0/3900881233 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f395c085ce0 con 0x55b6f70639f0 2024-05-27T08:03:19.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.347+0000 7f396fc46b80 10 client.27985 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:19.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.347+0000 7f396fc46b80 10 client.27985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:19.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.347+0000 7f396fc46b80 20 client.27985 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:19.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.347+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b6f719ea30 0x55b6f71a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:19.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.347+0000 7f396fc46b80 4 client.27985 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:03:19.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.347+0000 7f396fc46b80 1 -- 172.21.15.90:0/3900881233 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6f7026d90 con 0x55b6f719ea30 2024-05-27T08:03:19.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.348+0000 7f3959400700 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b6f719ea30 0x55b6f71a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:19.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.350+0000 7f393fe00700 10 client.27985 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:19.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.350+0000 7f3959400700 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b6f719ea30 0x55b6f71a0e50 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab3ad47019b97534 server_cookie=9916f958023e8403 in_seq=0 out_seq=0 2024-05-27T08:03:19.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.353+0000 7f393fe00700 1 -- 172.21.15.90:0/3900881233 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b6f7026d90 con 0x55b6f719ea30 2024-05-27T08:03:19.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.353+0000 7f393fe00700 10 client.27985 handle_command_reply: tid=0 2024-05-27T08:03:19.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.353+0000 7f396fc46b80 1 client.27985 shutdown 2024-05-27T08:03:19.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f396fc46b80 1 -- 172.21.15.90:0/3900881233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f70639f0 msgr2=0x55b6f711a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:19.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f70639f0 0x55b6f711a0a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f395c11d800 tx=0x7f395c11c580 comp rx=0 tx=0).stop 2024-05-27T08:03:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f396fc46b80 1 -- 172.21.15.90:0/3900881233 shutdown_connections 2024-05-27T08:03:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55b6f719ea30 0x55b6f71a0e50 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b6f7185c60 0x55b6f7186070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b6f70639f0 0x55b6f711a0a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f396fc46b80 1 --2- 172.21.15.90:0/3900881233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b6f71012c0 0x55b6f711a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f393fe00700 0 client.27985 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:19.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f396fc46b80 1 -- 172.21.15.90:0/3900881233 >> 172.21.15.90:0/3900881233 conn(0x55b6f7011390 msgr2=0x55b6f70fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f396fc46b80 1 -- 172.21.15.90:0/3900881233 shutdown_connections 2024-05-27T08:03:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.355+0000 7f396fc46b80 1 -- 172.21.15.90:0/3900881233 wait complete. 2024-05-27T08:03:19.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.356+0000 7f396fc46b80 20 client.27985 trim_cache size 0 max 16384 2024-05-27T08:03:19.359 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:03:19.359 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T08:03:19.359 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T08:03:19.359 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:03:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.357+0000 7f395b200700 1 -- 172.21.15.90:0/893110598 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f39400b2b40 msgr2=0x7f39400b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.357+0000 7f395b200700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f39400b2b40 0x7f39400b5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3954092600 tx=0x7f394c00a380 comp rx=0 tx=0).stop 2024-05-27T08:03:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.357+0000 7f395b200700 1 -- 172.21.15.90:0/893110598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3954094830 msgr2=0x7f395412ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.357+0000 7f395b200700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3954094830 0x7f395412ae40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f395c02ec00 tx=0x7f395c0519b0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.357+0000 7f395b200700 1 -- 172.21.15.90:0/893110598 shutdown_connections 2024-05-27T08:03:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.357+0000 7f395b200700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f39400b2b40 0x7f39400b5050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.357+0000 7f395b200700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f395409a860 0x7f395412f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.357+0000 7f395b200700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3954094830 0x7f395412ae40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.357+0000 7f395b200700 1 --2- 172.21.15.90:0/893110598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3954095180 0x7f395412b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.357+0000 7f395b200700 1 -- 172.21.15.90:0/893110598 >> 172.21.15.90:0/893110598 conn(0x7f395400a390 msgr2=0x7f3954093100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.358+0000 7f395b200700 1 -- 172.21.15.90:0/893110598 shutdown_connections 2024-05-27T08:03:19.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:19.358+0000 7f395b200700 1 -- 172.21.15.90:0/893110598 wait complete. 2024-05-27T08:03:20.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:20 smithi090 ceph-mon[31429]: pgmap v546: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.0 MiB/s wr, 594 op/s 2024-05-27T08:03:20.384 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:03:20.384 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:03:20.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:20 smithi136 ceph-mon[35324]: pgmap v546: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.0 MiB/s wr, 594 op/s 2024-05-27T08:03:20.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:20 smithi129 ceph-mon[35254]: pgmap v546: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 8.0 MiB/s wr, 594 op/s 2024-05-27T08:03:21.104 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:22.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:22 smithi129 ceph-mon[35254]: pgmap v547: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.4 MiB/s wr, 560 op/s 2024-05-27T08:03:22.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:22 smithi090 ceph-mon[31429]: pgmap v547: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.4 MiB/s wr, 560 op/s 2024-05-27T08:03:22.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:22 smithi136 ceph-mon[35324]: pgmap v547: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.4 MiB/s wr, 560 op/s 2024-05-27T08:03:23.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 785.0 s, 473.8 tps, lat 211.971 ms stddev 215.082 2024-05-27T08:03:24.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.074+0000 7f9cab200700 1 -- 172.21.15.90:0/2777373384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca409d560 msgr2=0x7f9ca40a99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:24.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.074+0000 7f9cab200700 1 --2- 172.21.15.90:0/2777373384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca409d560 0x7f9ca40a99d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9cac069810 tx=0x7f9cac06a480 comp rx=0 tx=0).stop 2024-05-27T08:03:24.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.075+0000 7f9cab200700 1 -- 172.21.15.90:0/2777373384 shutdown_connections 2024-05-27T08:03:24.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.075+0000 7f9cab200700 1 --2- 172.21.15.90:0/2777373384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca409d560 0x7f9ca40a99d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:24.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.075+0000 7f9cab200700 1 --2- 172.21.15.90:0/2777373384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ca40a2c50 0x7f9ca40a3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:24.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.076+0000 7f9cab200700 1 --2- 172.21.15.90:0/2777373384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ca409cc10 0x7f9ca409d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:24.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.076+0000 7f9cab200700 1 -- 172.21.15.90:0/2777373384 >> 172.21.15.90:0/2777373384 conn(0x7f9ca400ac60 msgr2=0x7f9ca400b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:24.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.076+0000 7f9cab200700 1 -- 172.21.15.90:0/2777373384 shutdown_connections 2024-05-27T08:03:24.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.076+0000 7f9cab200700 1 -- 172.21.15.90:0/2777373384 wait complete. 2024-05-27T08:03:24.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.079+0000 7f9cab200700 1 Processor -- start 2024-05-27T08:03:24.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.079+0000 7f9cab200700 1 -- start start 2024-05-27T08:03:24.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.080+0000 7f9cab200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca409cc10 0x7f9ca4133520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:24.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.080+0000 7f9cab200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ca409d560 0x7f9ca4133ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:24.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.080+0000 7f9cab200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ca40a2c50 0x7f9ca4137d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.081+0000 7f9cab200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca40acb30 con 0x7f9ca409d560 2024-05-27T08:03:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.081+0000 7f9ca9e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca409cc10 0x7f9ca4133520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.081+0000 7f9ca9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ca409d560 0x7f9ca4133ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.081+0000 7f9ca9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ca409d560 0x7f9ca4133ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36138/0 (socket says 172.21.15.90:36138) 2024-05-27T08:03:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.081+0000 7f9ca9400700 1 -- 172.21.15.90:0/824617592 learned_addr learned my addr 172.21.15.90:0/824617592 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.081+0000 7f9ca9e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca409cc10 0x7f9ca4133520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:44268/0 (socket says 172.21.15.90:44268) 2024-05-27T08:03:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.081+0000 7f9caa800700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ca40a2c50 0x7f9ca4137d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.082+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca40ac9b0 con 0x7f9ca40a2c50 2024-05-27T08:03:24.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.082+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca40accb0 con 0x7f9ca409cc10 2024-05-27T08:03:24.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.083+0000 7f9caa800700 1 -- 172.21.15.90:0/824617592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca409cc10 msgr2=0x7f9ca4133520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:24.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.083+0000 7f9caa800700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca409cc10 0x7f9ca4133520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:24.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.083+0000 7f9caa800700 1 -- 172.21.15.90:0/824617592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ca409d560 msgr2=0x7f9ca4133ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:24.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.083+0000 7f9caa800700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ca409d560 0x7f9ca4133ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:24.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.083+0000 7f9caa800700 1 -- 172.21.15.90:0/824617592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca41383f0 con 0x7f9ca40a2c50 2024-05-27T08:03:24.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.084+0000 7f9caa800700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ca40a2c50 0x7f9ca4137d50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ca409dbd0 tx=0x7f9cac0806c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:24.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.084+0000 7f9ca2a00700 1 -- 172.21.15.90:0/824617592 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac08e070 con 0x7f9ca40a2c50 2024-05-27T08:03:24.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.085+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca4138680 con 0x7f9ca40a2c50 2024-05-27T08:03:24.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.085+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca4138b80 con 0x7f9ca40a2c50 2024-05-27T08:03:24.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.086+0000 7f9ca2a00700 1 -- 172.21.15.90:0/824617592 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9cac080d90 con 0x7f9ca40a2c50 2024-05-27T08:03:24.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.086+0000 7f9ca2a00700 1 -- 172.21.15.90:0/824617592 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac085040 con 0x7f9ca40a2c50 2024-05-27T08:03:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.088+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8c005330 con 0x7f9ca40a2c50 2024-05-27T08:03:24.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.100+0000 7f9ca2a00700 1 -- 172.21.15.90:0/824617592 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9cac069d80 con 0x7f9ca40a2c50 2024-05-27T08:03:24.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.106+0000 7f9ca2a00700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9c900b2a70 0x7f9c900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:24.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.106+0000 7f9ca9e00700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9c900b2a70 0x7f9c900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:24.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.107+0000 7f9ca2a00700 1 -- 172.21.15.90:0/824617592 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f9cac1515c0 con 0x7f9ca40a2c50 2024-05-27T08:03:24.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.109+0000 7f9ca9e00700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9c900b2a70 0x7f9c900b4f80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ca409df10 tx=0x7f9c9c007400 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:24.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.117+0000 7f9ca2a00700 1 -- 172.21.15.90:0/824617592 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ca4138b80 con 0x7f9ca40a2c50 2024-05-27T08:03:24.306 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:24 smithi136 ceph-mon[35324]: pgmap v548: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.8 MiB/s wr, 536 op/s 2024-05-27T08:03:24.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:24 smithi129 ceph-mon[35254]: pgmap v548: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.8 MiB/s wr, 536 op/s 2024-05-27T08:03:24.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:24 smithi090 ceph-mon[31429]: pgmap v548: 129 pgs: 129 active+clean; 3.1 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.8 MiB/s wr, 536 op/s 2024-05-27T08:03:24.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.523+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c8c005620 con 0x7f9ca40a2c50 2024-05-27T08:03:24.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.525+0000 7f9ca2a00700 1 -- 172.21.15.90:0/824617592 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+9456 (secure 0 0 0) 0x7f9cac118b70 con 0x7f9ca40a2c50 2024-05-27T08:03:24.528 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:03:24.528 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":64,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:56:32.161929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T08:03:24.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.530+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9c900b2a70 msgr2=0x7f9c900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.530+0000 7f9cab200700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9c900b2a70 0x7f9c900b4f80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ca409df10 tx=0x7f9c9c007400 comp rx=0 tx=0).stop 2024-05-27T08:03:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.530+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ca40a2c50 msgr2=0x7f9ca4137d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.530+0000 7f9cab200700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ca40a2c50 0x7f9ca4137d50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ca409dbd0 tx=0x7f9cac0806c0 comp rx=0 tx=0).stop 2024-05-27T08:03:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.531+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 shutdown_connections 2024-05-27T08:03:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.531+0000 7f9cab200700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9c900b2a70 0x7f9c900b4f80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.531+0000 7f9cab200700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9ca40a2c50 0x7f9ca4137d50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.531+0000 7f9cab200700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca409cc10 0x7f9ca4133520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.531+0000 7f9cab200700 1 --2- 172.21.15.90:0/824617592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ca409d560 0x7f9ca4133ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.531+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 >> 172.21.15.90:0/824617592 conn(0x7f9ca400ac60 msgr2=0x7f9ca409b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.532+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 shutdown_connections 2024-05-27T08:03:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:24.532+0000 7f9cab200700 1 -- 172.21.15.90:0/824617592 wait complete. 2024-05-27T08:03:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 64 2024-05-27T08:03:25.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:25 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/824617592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:03:25.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:25 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/824617592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:03:25.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:25 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/824617592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:03:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.212+0000 7fe5c9800640 1 -- 172.21.15.90:0/801733970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 msgr2=0x7fe5c4108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.212+0000 7fe5c9800640 1 --2- 172.21.15.90:0/801733970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c4108490 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009ca0 tx=0x7fe5bc02a8c0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.213+0000 7fe5c9800640 1 -- 172.21.15.90:0/801733970 shutdown_connections 2024-05-27T08:03:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.213+0000 7fe5c9800640 1 --2- 172.21.15.90:0/801733970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c4108490 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.213+0000 7fe5c9800640 1 --2- 172.21.15.90:0/801733970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 0x7fe5c41079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.213+0000 7fe5c9800640 1 --2- 172.21.15.90:0/801733970 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 0x7fe5c4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.213+0000 7fe5c9800640 1 -- 172.21.15.90:0/801733970 >> 172.21.15.90:0/801733970 conn(0x7fe5c40782d0 msgr2=0x7fe5c4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.213+0000 7fe5c9800640 1 -- 172.21.15.90:0/801733970 shutdown_connections 2024-05-27T08:03:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.213+0000 7fe5c9800640 1 -- 172.21.15.90:0/801733970 wait complete. 2024-05-27T08:03:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.216+0000 7fe5c9800640 1 Processor -- start 2024-05-27T08:03:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.217+0000 7fe5c9800640 1 -- start start 2024-05-27T08:03:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.218+0000 7fe5c9800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 0x7fe5c41a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.219+0000 7fe5c9800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 0x7fe5c41a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.219+0000 7fe5c9800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.220+0000 7fe5c9800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c411a350 con 0x7fe5c41075d0 2024-05-27T08:03:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.220+0000 7fe5c9800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c411a1d0 con 0x7fe5c4108000 2024-05-27T08:03:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.220+0000 7fe5c9800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c411a4d0 con 0x7fe5c4102b80 2024-05-27T08:03:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.220+0000 7fe5c3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.220+0000 7fe5c3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43540/0 (socket says 172.21.15.90:43540) 2024-05-27T08:03:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.220+0000 7fe5c2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 0x7fe5c41a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.220+0000 7fe5c3400640 1 -- 172.21.15.90:0/440969574 learned_addr learned my addr 172.21.15.90:0/440969574 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.221+0000 7fe5c3400640 1 -- 172.21.15.90:0/440969574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 msgr2=0x7fe5c41a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.221+0000 7fe5c3400640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 0x7fe5c41a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.221+0000 7fe5c3400640 1 -- 172.21.15.90:0/440969574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 msgr2=0x7fe5c41a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.221+0000 7fe5c3400640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 0x7fe5c41a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.221+0000 7fe5c3400640 1 -- 172.21.15.90:0/440969574 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c41a5c30 con 0x7fe5c4108000 2024-05-27T08:03:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.222+0000 7fe5c3400640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc031040 tx=0x7fe5bc032990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.222+0000 7fe5bb400640 1 -- 172.21.15.90:0/440969574 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc032cb0 con 0x7fe5c4108000 2024-05-27T08:03:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.223+0000 7fe5bb400640 1 -- 172.21.15.90:0/440969574 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5bc032e50 con 0x7fe5c4108000 2024-05-27T08:03:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.223+0000 7fe5bb400640 1 -- 172.21.15.90:0/440969574 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc030ad0 con 0x7fe5c4108000 2024-05-27T08:03:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.223+0000 7fe5c9800640 1 -- 172.21.15.90:0/440969574 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c41a5ec0 con 0x7fe5c4108000 2024-05-27T08:03:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.223+0000 7fe5c9800640 1 -- 172.21.15.90:0/440969574 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c41a63c0 con 0x7fe5c4108000 2024-05-27T08:03:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.226+0000 7fe5d0976740 1 Processor -- start 2024-05-27T08:03:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.226+0000 7fe5d0976740 1 -- start start 2024-05-27T08:03:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.226+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2d81b0 0x556ccf67a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.227+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf67ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.227+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf67b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.227+0000 7fe5d0976740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556ccf5ebb90 con 0x556ccf2b74d0 2024-05-27T08:03:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.227+0000 7fe5d0976740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ccf5db730 con 0x556ccf2af2f0 2024-05-27T08:03:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.227+0000 7fe5d0976740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556ccf5df770 con 0x556ccf2d81b0 2024-05-27T08:03:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.227+0000 7fe5c2000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf67b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.228+0000 7fe5c2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf67ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.228+0000 7fe5c2000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf67b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36172/0 (socket says 172.21.15.90:36172) 2024-05-27T08:03:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.228+0000 7fe5c2000640 1 -- 172.21.15.90:0/1479774730 learned_addr learned my addr 172.21.15.90:0/1479774730 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.228+0000 7fe5c2000640 1 --2- 172.21.15.90:0/1479774730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2d81b0 0x556ccf67a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.230+0000 7fe5c2000640 1 -- 172.21.15.90:0/1479774730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2d81b0 msgr2=0x556ccf67a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.230+0000 7fe5c2000640 1 --2- 172.21.15.90:0/1479774730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2d81b0 0x556ccf67a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.230+0000 7fe5c2000640 1 -- 172.21.15.90:0/1479774730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 msgr2=0x556ccf67ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.230+0000 7fe5c2000640 1 --2- 172.21.15.90:0/1479774730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf67ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.230+0000 7fe5c2000640 1 -- 172.21.15.90:0/1479774730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ccf5748b0 con 0x556ccf2b74d0 2024-05-27T08:03:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.231+0000 7fe5c2000640 1 --2- 172.21.15.90:0/1479774730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf67b2e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0007fd0 tx=0x7fe5b0007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb41fb0ec48ec124 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.232+0000 7fe5bb400640 1 -- 172.21.15.90:0/440969574 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe5bc030c70 con 0x7fe5c4108000 2024-05-27T08:03:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.232+0000 7fe5afe00640 1 -- 172.21.15.90:0/1479774730 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b000d030 con 0x556ccf2b74d0 2024-05-27T08:03:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.233+0000 7fe5afe00640 1 -- 172.21.15.90:0/1479774730 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5b0006df0 con 0x556ccf2b74d0 2024-05-27T08:03:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.234+0000 7fe5afe00640 1 -- 172.21.15.90:0/1479774730 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b000d030 con 0x556ccf2b74d0 2024-05-27T08:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.234+0000 7fe5d0976740 1 -- 172.21.15.90:0/1479774730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 msgr2=0x556ccf67b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.235+0000 7fe5d0976740 1 --2- 172.21.15.90:0/1479774730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf67b2e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0007fd0 tx=0x7fe5b0007e80 comp rx=0 tx=0).stop 2024-05-27T08:03:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.235+0000 7fe5d0976740 1 -- 172.21.15.90:0/1479774730 shutdown_connections 2024-05-27T08:03:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.235+0000 7fe5d0976740 1 --2- 172.21.15.90:0/1479774730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf67ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.235+0000 7fe5d0976740 1 --2- 172.21.15.90:0/1479774730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf67b2e0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.235+0000 7fe5d0976740 1 --2- 172.21.15.90:0/1479774730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2d81b0 0x556ccf67a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.235+0000 7fe5d0976740 1 -- 172.21.15.90:0/1479774730 >> 172.21.15.90:0/1479774730 conn(0x556ccf2cc300 msgr2=0x556ccf677010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.236+0000 7fe5bb400640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b2aa0 0x7fe5a80b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.236+0000 7fe5d0976740 1 -- 172.21.15.90:0/1479774730 shutdown_connections 2024-05-27T08:03:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.236+0000 7fe5bb400640 1 -- 172.21.15.90:0/440969574 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7fe5bc103f70 con 0x7fe5c4108000 2024-05-27T08:03:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.236+0000 7fe5c2a00640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b2aa0 0x7fe5a80b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.237+0000 7fe5d0976740 1 -- 172.21.15.90:0/1479774730 wait complete. 2024-05-27T08:03:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.238+0000 7fe5d0976740 1 -- start start 2024-05-27T08:03:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.238+0000 7fe5c2a00640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b2aa0 0x7fe5a80b4fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x556ccf67bc10 tx=0x7fe5b4005f90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.239+0000 7fe5d0976740 1 Processor -- start 2024-05-27T08:03:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.239+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2af2f0 0x556ccf684960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.240+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf684ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.240+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2d81b0 0x556ccf685420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.240+0000 7fe5d0976740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556ccf2c1790 con 0x556ccf2b74d0 2024-05-27T08:03:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.240+0000 7fe5d0976740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ccf583bc0 con 0x556ccf2d81b0 2024-05-27T08:03:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.240+0000 7fe5d0976740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556ccf687a30 con 0x556ccf2af2f0 2024-05-27T08:03:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.240+0000 7fe5d0976740 10 client.0 _ll_register_callbacks cb 0x556ccf561400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.240+0000 7fe5c2000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf684ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.240+0000 7fe5c2000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf684ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36174/0 (socket says 172.21.15.90:36174) 2024-05-27T08:03:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.240+0000 7fe5c2000640 1 -- 172.21.15.90:0/2847691654 learned_addr learned my addr 172.21.15.90:0/2847691654 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.241+0000 7fe5c2000640 1 -- 172.21.15.90:0/2847691654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2af2f0 msgr2=0x556ccf684960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:03:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.241+0000 7fe5c2000640 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2af2f0 0x556ccf684960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.241+0000 7fe5c2000640 1 -- 172.21.15.90:0/2847691654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2d81b0 msgr2=0x556ccf685420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:03:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.241+0000 7fe5c2000640 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2d81b0 0x556ccf685420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.247 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.241+0000 7fe5c2000640 1 -- 172.21.15.90:0/2847691654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ccf2f0210 con 0x556ccf2b74d0 2024-05-27T08:03:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.242+0000 7fe5c2000640 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf684ea0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0001e50 tx=0x7fe5b0000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de41cf1fec0e653 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.242+0000 7fe5acc00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:03:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.243+0000 7fe5acc00640 1 -- 172.21.15.90:0/2847691654 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b000d050 con 0x556ccf2b74d0 2024-05-27T08:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.243+0000 7fe5acc00640 1 -- 172.21.15.90:0/2847691654 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7fe5b0007470 con 0x556ccf2b74d0 2024-05-27T08:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.243+0000 7fe5d0976740 1 -- 172.21.15.90:0/2847691654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b0006df0 con 0x556ccf2b74d0 2024-05-27T08:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.243+0000 7fe5acc00640 1 -- 172.21.15.90:0/2847691654 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe5b001d040 con 0x556ccf2b74d0 2024-05-27T08:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.243+0000 7fe5d0976740 10 client.18219 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.243+0000 7fe5d0976740 1 -- 172.21.15.90:0/2847691654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x556ccf3bc620 con 0x556ccf2b74d0 2024-05-27T08:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.244+0000 7fe5acc00640 1 -- 172.21.15.90:0/2847691654 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5b0006df0 con 0x556ccf2b74d0 2024-05-27T08:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.244+0000 7fe5acc00640 1 -- 172.21.15.90:0/2847691654 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe5b0020070 con 0x556ccf2b74d0 2024-05-27T08:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.244+0000 7fe5d0976740 10 client.18219 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.244+0000 7fe5d0976740 10 client.18219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.245+0000 7fe5d0976740 20 client.18219 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.245+0000 7fe5d0976740 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556ccf296d80 0x556ccf696d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.245+0000 7fe5d0976740 4 client.18219 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.245+0000 7fe5d0976740 1 -- 172.21.15.90:0/2847691654 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ccf5edae0 con 0x556ccf296d80 2024-05-27T08:03:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.246+0000 7fe5c2a00640 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556ccf296d80 0x556ccf696d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.249+0000 7fe5c2a00640 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556ccf296d80 0x556ccf696d50 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20287a400278950 server_cookie=78184124205a457c in_seq=0 out_seq=0 2024-05-27T08:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.249+0000 7fe5acc00640 10 client.18219 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.251+0000 7fe5acc00640 1 -- 172.21.15.90:0/2847691654 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x556ccf5edae0 con 0x556ccf296d80 2024-05-27T08:03:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.251+0000 7fe5acc00640 10 client.18219 handle_command_reply: tid=0 2024-05-27T08:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.251+0000 7fe5d0976740 1 client.18219 shutdown 2024-05-27T08:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.251+0000 7fe5d0976740 1 -- 172.21.15.90:0/2847691654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 msgr2=0x556ccf684ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.251+0000 7fe5d0976740 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf684ea0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0001e50 tx=0x7fe5b0000f60 comp rx=0 tx=0).stop 2024-05-27T08:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.252+0000 7fe5d0976740 1 -- 172.21.15.90:0/2847691654 shutdown_connections 2024-05-27T08:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.252+0000 7fe5d0976740 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556ccf296d80 0x556ccf696d50 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.252+0000 7fe5d0976740 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2d81b0 0x556ccf685420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.252+0000 7fe5d0976740 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2b74d0 0x556ccf684ea0 secure :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0001e50 tx=0x7fe5b0000f60 comp rx=0 tx=0).stop 2024-05-27T08:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.252+0000 7fe5d0976740 1 --2- 172.21.15.90:0/2847691654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2af2f0 0x556ccf684960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.252+0000 7fe5d0976740 1 -- 172.21.15.90:0/2847691654 >> 172.21.15.90:0/2847691654 conn(0x556ccf2cc300 msgr2=0x556ccf67c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.252+0000 7fe5acc00640 0 client.18219 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.252+0000 7fe5d0976740 1 -- 172.21.15.90:0/2847691654 shutdown_connections 2024-05-27T08:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.252+0000 7fe5d0976740 1 -- 172.21.15.90:0/2847691654 wait complete. 2024-05-27T08:03:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.253+0000 7fe5d0976740 20 client.18219 trim_cache size 0 max 16384 2024-05-27T08:03:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.301+0000 7fe5d0976740 1 Processor -- start 2024-05-27T08:03:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.304+0000 7fe5d0976740 1 -- start start 2024-05-27T08:03:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.305+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf687090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.305+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2d81b0 0x556ccf67e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.306+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf26c1c0 0x556ccf67e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.306+0000 7fe5d0976740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b9370 con 0x556ccf26c1c0 2024-05-27T08:03:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.306+0000 7fe5d0976740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b63c0 con 0x556ccf2af2f0 2024-05-27T08:03:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.306+0000 7fe5d0976740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b85d0 con 0x556ccf2d81b0 2024-05-27T08:03:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.307+0000 7fe5c2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf687090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.307+0000 7fe5c2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf687090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43548/0 (socket says 172.21.15.90:43548) 2024-05-27T08:03:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.307+0000 7fe5c2a00640 1 -- 172.21.15.90:0/904345183 learned_addr learned my addr 172.21.15.90:0/904345183 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.307+0000 7fe5c3400640 1 --2- 172.21.15.90:0/904345183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf26c1c0 0x556ccf67e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.307+0000 7fe5c2000640 1 --2- 172.21.15.90:0/904345183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2d81b0 0x556ccf67e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.308+0000 7fe5c3400640 1 -- 172.21.15.90:0/904345183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2d81b0 msgr2=0x556ccf67e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.308+0000 7fe5c3400640 1 --2- 172.21.15.90:0/904345183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2d81b0 0x556ccf67e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.308+0000 7fe5c3400640 1 -- 172.21.15.90:0/904345183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 msgr2=0x556ccf687090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.308+0000 7fe5c3400640 1 --2- 172.21.15.90:0/904345183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf687090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.308+0000 7fe5c3400640 1 -- 172.21.15.90:0/904345183 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ccf296fb0 con 0x556ccf26c1c0 2024-05-27T08:03:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.308+0000 7fe5c3400640 1 --2- 172.21.15.90:0/904345183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf26c1c0 0x556ccf67e9c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc1013a0 tx=0x7fe5bc0ca880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ca8080004aff0e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.309+0000 7fe5af400640 1 -- 172.21.15.90:0/904345183 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0cb630 con 0x556ccf26c1c0 2024-05-27T08:03:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.309+0000 7fe5af400640 1 -- 172.21.15.90:0/904345183 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5bc036030 con 0x556ccf26c1c0 2024-05-27T08:03:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.309+0000 7fe5af400640 1 -- 172.21.15.90:0/904345183 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0c7360 con 0x556ccf26c1c0 2024-05-27T08:03:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.311+0000 7fe5d0976740 1 -- 172.21.15.90:0/904345183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf26c1c0 msgr2=0x556ccf67e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.311+0000 7fe5d0976740 1 --2- 172.21.15.90:0/904345183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf26c1c0 0x556ccf67e9c0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc1013a0 tx=0x7fe5bc0ca880 comp rx=0 tx=0).stop 2024-05-27T08:03:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.312+0000 7fe5d0976740 1 -- 172.21.15.90:0/904345183 shutdown_connections 2024-05-27T08:03:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.312+0000 7fe5d0976740 1 --2- 172.21.15.90:0/904345183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf26c1c0 0x556ccf67e9c0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.312+0000 7fe5d0976740 1 --2- 172.21.15.90:0/904345183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf2d81b0 0x556ccf67e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.312+0000 7fe5d0976740 1 --2- 172.21.15.90:0/904345183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf687090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.312+0000 7fe5d0976740 1 -- 172.21.15.90:0/904345183 >> 172.21.15.90:0/904345183 conn(0x556ccf2cc300 msgr2=0x556ccf684c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.312+0000 7fe5d0976740 1 -- 172.21.15.90:0/904345183 shutdown_connections 2024-05-27T08:03:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.312+0000 7fe5d0976740 1 -- 172.21.15.90:0/904345183 wait complete. 2024-05-27T08:03:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.313+0000 7fe5d0976740 1 -- start start 2024-05-27T08:03:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.313+0000 7fe5d0976740 1 Processor -- start 2024-05-27T08:03:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.315+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf26c1c0 0x556ccf68d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.315+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf687d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.315+0000 7fe5d0976740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2d81b0 0x556ccf688270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.315+0000 7fe5d0976740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b7f00 con 0x556ccf2d81b0 2024-05-27T08:03:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.315+0000 7fe5d0976740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b6a90 con 0x556ccf2af2f0 2024-05-27T08:03:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.315+0000 7fe5d0976740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c41b7160 con 0x556ccf26c1c0 2024-05-27T08:03:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.315+0000 7fe5d0976740 10 client.0 _ll_register_callbacks cb 0x7fe5b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.315+0000 7fe5c3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2d81b0 0x556ccf688270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.315+0000 7fe5c3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2d81b0 0x556ccf688270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36184/0 (socket says 172.21.15.90:36184) 2024-05-27T08:03:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.315+0000 7fe5c3400640 1 -- 172.21.15.90:0/138690391 learned_addr learned my addr 172.21.15.90:0/138690391 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.316+0000 7fe5c2a00640 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf26c1c0 0x556ccf68d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.316+0000 7fe5c2000640 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf687d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.317+0000 7fe5c3400640 1 -- 172.21.15.90:0/138690391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf26c1c0 msgr2=0x556ccf68d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.317+0000 7fe5c3400640 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf26c1c0 0x556ccf68d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.317+0000 7fe5c3400640 1 -- 172.21.15.90:0/138690391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 msgr2=0x556ccf687d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.317+0000 7fe5c3400640 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf687d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.317+0000 7fe5c3400640 1 -- 172.21.15.90:0/138690391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ccf6888e0 con 0x556ccf2d81b0 2024-05-27T08:03:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.317+0000 7fe5c2a00640 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf26c1c0 0x556ccf68d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.317+0000 7fe5c3400640 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2d81b0 0x556ccf688270 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005e30 tx=0x7fe5bc030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78924739b4f6385 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.317+0000 7fe5acc00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:03:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.318+0000 7fe5acc00640 1 -- 172.21.15.90:0/138690391 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc101720 con 0x556ccf2d81b0 2024-05-27T08:03:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.318+0000 7fe5acc00640 1 -- 172.21.15.90:0/138690391 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7fe5bc0d3900 con 0x556ccf2d81b0 2024-05-27T08:03:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.319+0000 7fe5d0976740 1 -- 172.21.15.90:0/138690391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5bc036030 con 0x556ccf2d81b0 2024-05-27T08:03:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.319+0000 7fe5acc00640 1 -- 172.21.15.90:0/138690391 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fe5bc0cf2e0 con 0x556ccf2d81b0 2024-05-27T08:03:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.319+0000 7fe5d0976740 10 client.18225 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:26.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.319+0000 7fe5d0976740 1 -- 172.21.15.90:0/138690391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x556ccf6fe220 con 0x556ccf2d81b0 2024-05-27T08:03:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.320+0000 7fe5acc00640 1 -- 172.21.15.90:0/138690391 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5bc036030 con 0x556ccf2d81b0 2024-05-27T08:03:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.320+0000 7fe5acc00640 1 -- 172.21.15.90:0/138690391 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fe5bc0c68a0 con 0x556ccf2d81b0 2024-05-27T08:03:26.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.321+0000 7fe5d0976740 10 client.18225 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.321+0000 7fe5d0976740 10 client.18225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.321+0000 7fe5d0976740 20 client.18225 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:26.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.321+0000 7fe5d0976740 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556ccf6fe7a0 0x556ccf700bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.321+0000 7fe5d0976740 4 client.18225 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:03:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.321+0000 7fe5d0976740 1 -- 172.21.15.90:0/138690391 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ccf5e72c0 con 0x556ccf6fe7a0 2024-05-27T08:03:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.322+0000 7fe5c2a00640 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556ccf6fe7a0 0x556ccf700bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.326+0000 7fe5c2a00640 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556ccf6fe7a0 0x556ccf700bc0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3070e64046764522 server_cookie=25f4f3c5518cf02d in_seq=0 out_seq=0 2024-05-27T08:03:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:26.326+0000 7fe5acc00640 10 client.18225 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:26.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:26 smithi090 ceph-mon[31429]: pgmap v549: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.2 MiB/s wr, 603 op/s 2024-05-27T08:03:26.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:26 smithi136 ceph-mon[35324]: pgmap v549: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.2 MiB/s wr, 603 op/s 2024-05-27T08:03:26.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:26 smithi129 ceph-mon[35254]: pgmap v549: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 8.2 MiB/s wr, 603 op/s 2024-05-27T08:03:27.482 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T08:03:28.296 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:28.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:28 smithi129 ceph-mon[35254]: pgmap v550: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 427 op/s 2024-05-27T08:03:28.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:28 smithi090 ceph-mon[31429]: pgmap v550: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 427 op/s 2024-05-27T08:03:28.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:28 smithi136 ceph-mon[35324]: pgmap v550: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.0 MiB/s wr, 427 op/s 2024-05-27T08:03:28.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 790.0 s, 499.5 tps, lat 203.547 ms stddev 118.727 2024-05-27T08:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.920+0000 7fe5acc00640 1 -- 172.21.15.90:0/138690391 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556ccf5e72c0 con 0x556ccf6fe7a0 2024-05-27T08:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.920+0000 7fe5acc00640 10 client.18225 handle_command_reply: tid=0 2024-05-27T08:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.920+0000 7fe5d0976740 1 client.18225 shutdown 2024-05-27T08:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.921+0000 7fe5d0976740 1 -- 172.21.15.90:0/138690391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2d81b0 msgr2=0x556ccf688270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.921+0000 7fe5d0976740 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2d81b0 0x556ccf688270 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc005e30 tx=0x7fe5bc030fd0 comp rx=0 tx=0).stop 2024-05-27T08:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.922+0000 7fe5d0976740 1 -- 172.21.15.90:0/138690391 shutdown_connections 2024-05-27T08:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.922+0000 7fe5d0976740 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x556ccf6fe7a0 0x556ccf700bc0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.922+0000 7fe5d0976740 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ccf2af2f0 0x556ccf687d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.922+0000 7fe5d0976740 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556ccf2d81b0 0x556ccf688270 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.922+0000 7fe5d0976740 1 --2- 172.21.15.90:0/138690391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ccf26c1c0 0x556ccf68d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.922+0000 7fe5d0976740 1 -- 172.21.15.90:0/138690391 >> 172.21.15.90:0/138690391 conn(0x556ccf2cc300 msgr2=0x556ccf67a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.923+0000 7fe5acc00640 0 client.18225 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.923+0000 7fe5d0976740 1 -- 172.21.15.90:0/138690391 shutdown_connections 2024-05-27T08:03:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.923+0000 7fe5d0976740 1 -- 172.21.15.90:0/138690391 wait complete. 2024-05-27T08:03:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.924+0000 7fe5d0976740 20 client.18225 trim_cache size 0 max 16384 2024-05-27T08:03:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:03:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:03:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:03:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.925+0000 7fe5c9800640 1 -- 172.21.15.90:0/440969574 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b2aa0 msgr2=0x7fe5a80b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.925+0000 7fe5c9800640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b2aa0 0x7fe5a80b4fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x556ccf67bc10 tx=0x7fe5b4005f90 comp rx=0 tx=0).stop 2024-05-27T08:03:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.926+0000 7fe5c9800640 1 -- 172.21.15.90:0/440969574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 msgr2=0x7fe5c41a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.926+0000 7fe5c9800640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc031040 tx=0x7fe5bc032990 comp rx=0 tx=0).stop 2024-05-27T08:03:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.927+0000 7fe5c9800640 1 -- 172.21.15.90:0/440969574 shutdown_connections 2024-05-27T08:03:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.927+0000 7fe5c9800640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fe5a80b2aa0 0x7fe5a80b4fb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.927+0000 7fe5c9800640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5c4108000 0x7fe5c41a55f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.927+0000 7fe5c9800640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5c41075d0 0x7fe5c41a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.927+0000 7fe5c9800640 1 --2- 172.21.15.90:0/440969574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c4102b80 0x7fe5c41a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.927+0000 7fe5c9800640 1 -- 172.21.15.90:0/440969574 >> 172.21.15.90:0/440969574 conn(0x7fe5c40782d0 msgr2=0x7fe5c41051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.927+0000 7fe5c9800640 1 -- 172.21.15.90:0/440969574 shutdown_connections 2024-05-27T08:03:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:28.927+0000 7fe5c9800640 1 -- 172.21.15.90:0/440969574 wait complete. 2024-05-27T08:03:30.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:30 smithi129 ceph-mon[35254]: pgmap v551: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.6 MiB/s wr, 550 op/s 2024-05-27T08:03:30.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:30 smithi090 ceph-mon[31429]: pgmap v551: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.6 MiB/s wr, 550 op/s 2024-05-27T08:03:30.502 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:30 smithi136 ceph-mon[35324]: pgmap v551: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.6 MiB/s wr, 550 op/s 2024-05-27T08:03:31.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.878+0000 7fb700000700 1 -- 172.21.15.90:0/3196762829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8111bc0 msgr2=0x7fb6f8111fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:31.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.878+0000 7fb700000700 1 --2- 172.21.15.90:0/3196762829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8111bc0 0x7fb6f8111fd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009990 tx=0x7fb6ec009ca0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.878+0000 7fb700000700 1 -- 172.21.15.90:0/3196762829 shutdown_connections 2024-05-27T08:03:31.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.878+0000 7fb700000700 1 --2- 172.21.15.90:0/3196762829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f810b270 0x7fb6f81198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.878+0000 7fb700000700 1 --2- 172.21.15.90:0/3196762829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8111bc0 0x7fb6f8111fd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.878+0000 7fb700000700 1 --2- 172.21.15.90:0/3196762829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f810a920 0x7fb6f810ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.878+0000 7fb700000700 1 -- 172.21.15.90:0/3196762829 >> 172.21.15.90:0/3196762829 conn(0x7fb6f806f4c0 msgr2=0x7fb6f806f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:31.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.879+0000 7fb700000700 1 -- 172.21.15.90:0/3196762829 shutdown_connections 2024-05-27T08:03:31.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.879+0000 7fb700000700 1 -- 172.21.15.90:0/3196762829 wait complete. 2024-05-27T08:03:31.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.881+0000 7fb700000700 1 Processor -- start 2024-05-27T08:03:31.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.881+0000 7fb700000700 1 -- start start 2024-05-27T08:03:31.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.881+0000 7fb700000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f810a920 0x7fb6f811a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.882+0000 7fb700000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f810b270 0x7fb6f811a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.882+0000 7fb700000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8111bc0 0x7fb6f811aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.882+0000 7fb700000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f811eeb0 con 0x7fb6f810b270 2024-05-27T08:03:31.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.882+0000 7fb700000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f811ed30 con 0x7fb6f8111bc0 2024-05-27T08:03:31.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.882+0000 7fb700000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f811f030 con 0x7fb6f810a920 2024-05-27T08:03:31.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.882+0000 7fb6fd600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f810a920 0x7fb6f811a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.882+0000 7fb6fd600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f810a920 0x7fb6f811a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:45494/0 (socket says 172.21.15.90:45494) 2024-05-27T08:03:31.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.882+0000 7fb6fd600700 1 -- 172.21.15.90:0/163192370 learned_addr learned my addr 172.21.15.90:0/163192370 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:31.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.882+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f810b270 0x7fb6f811a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.882+0000 7fb6fe000700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8111bc0 0x7fb6f811aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.883+0000 7fb6fd600700 1 -- 172.21.15.90:0/163192370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8111bc0 msgr2=0x7fb6f811aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:31.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.883+0000 7fb6fd600700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8111bc0 0x7fb6f811aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.883+0000 7fb6fd600700 1 -- 172.21.15.90:0/163192370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f810b270 msgr2=0x7fb6f811a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:31.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.883+0000 7fb6fd600700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f810b270 0x7fb6f811a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.883+0000 7fb6fd600700 1 -- 172.21.15.90:0/163192370 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f81b5e40 con 0x7fb6f810a920 2024-05-27T08:03:31.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.884+0000 7fb6fd600700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f810a920 0x7fb6f811a030 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec003760 tx=0x7fb6ec0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:31.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.884+0000 7fb6f6000700 1 -- 172.21.15.90:0/163192370 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec026070 con 0x7fb6f810a920 2024-05-27T08:03:31.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.884+0000 7fb6f6000700 1 -- 172.21.15.90:0/163192370 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6ec00fbc0 con 0x7fb6f810a920 2024-05-27T08:03:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.884+0000 7fb6f6000700 1 -- 172.21.15.90:0/163192370 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec021bc0 con 0x7fb6f810a920 2024-05-27T08:03:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.884+0000 7fb700000700 1 -- 172.21.15.90:0/163192370 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f81b6070 con 0x7fb6f810a920 2024-05-27T08:03:31.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.885+0000 7fb700000700 1 -- 172.21.15.90:0/163192370 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f81b65d0 con 0x7fb6f810a920 2024-05-27T08:03:31.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.891+0000 7fb6f6000700 1 -- 172.21.15.90:0/163192370 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb6ec0050d0 con 0x7fb6f810a920 2024-05-27T08:03:31.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.893+0000 7fb6f6000700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6e00b2b80 0x7fb6e00b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.894+0000 7fb6f6000700 1 -- 172.21.15.90:0/163192370 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7fb6ec0e42a0 con 0x7fb6f810a920 2024-05-27T08:03:31.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.894+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6e00b2b80 0x7fb6e00b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.900+0000 7fb70ead3b80 1 Processor -- start 2024-05-27T08:03:31.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.900+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6e00b2b80 0x7fb6e00b5090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6f806faf0 tx=0x7fb6e4009190 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:31.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.900+0000 7fb70ead3b80 1 -- start start 2024-05-27T08:03:31.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.901+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e39be9f0 0x5645e3a51330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.901+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e38d2e40 0x5645e3a51870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.901+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e3a51e70 0x5645e3a5e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.901+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5645e397fe40 con 0x5645e3a51e70 2024-05-27T08:03:31.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.901+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5645e396f9e0 con 0x5645e38d2e40 2024-05-27T08:03:31.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.901+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5645e3973a20 con 0x5645e39be9f0 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.901+0000 7fb6fcc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e38d2e40 0x5645e3a51870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.901+0000 7fb6fcc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e38d2e40 0x5645e3a51870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53570/0 (socket says 172.21.15.90:53570) 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.901+0000 7fb6fcc00700 1 -- 172.21.15.90:0/1822743196 learned_addr learned my addr 172.21.15.90:0/1822743196 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.902+0000 7fb6fcc00700 1 -- 172.21.15.90:0/1822743196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e39be9f0 msgr2=0x5645e3a51330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.902+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/1822743196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e39be9f0 0x5645e3a51330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.902+0000 7fb6fcc00700 1 -- 172.21.15.90:0/1822743196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e3a51e70 msgr2=0x5645e3a5e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.902+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/1822743196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e3a51e70 0x5645e3a5e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.902+0000 7fb6fcc00700 1 -- 172.21.15.90:0/1822743196 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e3938420 con 0x5645e38d2e40 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.902+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/1822743196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e38d2e40 0x5645e3a51870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009fd0 tx=0x7fb6e4011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cecb82fceadeb8be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.903+0000 7fb6eaa00700 1 -- 172.21.15.90:0/1822743196 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e401b040 con 0x5645e38d2e40 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.903+0000 7fb6eaa00700 1 -- 172.21.15.90:0/1822743196 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6e4024070 con 0x5645e38d2e40 2024-05-27T08:03:31.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.903+0000 7fb6eaa00700 1 -- 172.21.15.90:0/1822743196 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40136b0 con 0x5645e38d2e40 2024-05-27T08:03:31.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.907+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1822743196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e38d2e40 msgr2=0x5645e3a51870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:31.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.907+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1822743196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e38d2e40 0x5645e3a51870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009fd0 tx=0x7fb6e4011000 comp rx=0 tx=0).stop 2024-05-27T08:03:31.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.907+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1822743196 shutdown_connections 2024-05-27T08:03:31.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.907+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1822743196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e3a51e70 0x5645e3a5e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.907+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1822743196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e39be9f0 0x5645e3a51330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.907+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1822743196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e38d2e40 0x5645e3a51870 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.907+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1822743196 >> 172.21.15.90:0/1822743196 conn(0x5645e396c390 msgr2=0x5645e3a4d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:31.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.907+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1822743196 shutdown_connections 2024-05-27T08:03:31.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.907+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1822743196 wait complete. 2024-05-27T08:03:31.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.908+0000 7fb70ead3b80 1 -- start start 2024-05-27T08:03:31.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.909+0000 7fb70ead3b80 1 Processor -- start 2024-05-27T08:03:31.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.910+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e38d2e40 0x5645e3a5b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.910+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e39be9f0 0x5645e3a5bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.910+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 0x5645e3a5c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.911+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5645e3875fa0 con 0x5645e39be9f0 2024-05-27T08:03:31.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.911+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5645e3910760 con 0x5645e3a51e70 2024-05-27T08:03:31.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.911+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5645e3a5eb80 con 0x5645e38d2e40 2024-05-27T08:03:31.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.911+0000 7fb70ead3b80 10 client.0 _ll_register_callbacks cb 0x5645e3919c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:31.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.911+0000 7fb6fe000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 0x5645e3a5c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.911+0000 7fb6fe000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 0x5645e3a5c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53576/0 (socket says 172.21.15.90:53576) 2024-05-27T08:03:31.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.911+0000 7fb6fe000700 1 -- 172.21.15.90:0/1198950371 learned_addr learned my addr 172.21.15.90:0/1198950371 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:31.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.911+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e39be9f0 0x5645e3a5bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.911+0000 7fb6fd600700 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e38d2e40 0x5645e3a5b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.912+0000 7fb6fe000700 1 -- 172.21.15.90:0/1198950371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e38d2e40 msgr2=0x5645e3a5b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:31.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.912+0000 7fb6fe000700 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e38d2e40 0x5645e3a5b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.912+0000 7fb6fe000700 1 -- 172.21.15.90:0/1198950371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e39be9f0 msgr2=0x5645e3a5bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:31.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.912+0000 7fb6fe000700 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e39be9f0 0x5645e3a5bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.912+0000 7fb6fe000700 1 -- 172.21.15.90:0/1198950371 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e38bf8d0 con 0x5645e3a51e70 2024-05-27T08:03:31.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.913+0000 7fb6df400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:03:31.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.913+0000 7fb6fe000700 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 0x5645e3a5c170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000ab40 tx=0x7fb6f000af00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685400c124acae0c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:31.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.913+0000 7fb6df400700 1 -- 172.21.15.90:0/1198950371 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0018020 con 0x5645e3a51e70 2024-05-27T08:03:31.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.913+0000 7fb6df400700 1 -- 172.21.15.90:0/1198950371 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7fb6f000ea30 con 0x5645e3a51e70 2024-05-27T08:03:31.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.913+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1198950371 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e4024070 con 0x5645e3a51e70 2024-05-27T08:03:31.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.914+0000 7fb6df400700 1 -- 172.21.15.90:0/1198950371 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb6f000ec50 con 0x5645e3a51e70 2024-05-27T08:03:31.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.914+0000 7fb70ead3b80 10 client.37943 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:31.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.914+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1198950371 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5645e3940060 con 0x5645e3a51e70 2024-05-27T08:03:31.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.915+0000 7fb6df400700 1 -- 172.21.15.90:0/1198950371 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6e4024070 con 0x5645e3a51e70 2024-05-27T08:03:31.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.915+0000 7fb6df400700 1 -- 172.21.15.90:0/1198950371 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fb6f001c3a0 con 0x5645e3a51e70 2024-05-27T08:03:31.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.915+0000 7fb70ead3b80 10 client.37943 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:31.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.915+0000 7fb70ead3b80 10 client.37943 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T08:03:31.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.915+0000 7fb70ead3b80 20 client.37943 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:31.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.915+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5645e3a6bd40 0x5645e3a6e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.916+0000 7fb70ead3b80 4 client.37943 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:31.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.916+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1198950371 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645e3981d90 con 0x5645e3a6bd40 2024-05-27T08:03:31.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.916+0000 7fb6fd600700 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5645e3a6bd40 0x5645e3a6e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.917+0000 7fb6df400700 10 client.37943 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T08:03:31.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.917+0000 7fb6fd600700 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5645e3a6bd40 0x5645e3a6e190 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c7020b8a12825083 server_cookie=deb0cda0585806cf in_seq=0 out_seq=0 2024-05-27T08:03:31.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.918+0000 7fb6df400700 1 -- 172.21.15.90:0/1198950371 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5645e3981d90 con 0x5645e3a6bd40 2024-05-27T08:03:31.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.918+0000 7fb6df400700 10 client.37943 handle_command_reply: tid=0 2024-05-27T08:03:31.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.918+0000 7fb70ead3b80 1 client.37943 shutdown 2024-05-27T08:03:31.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.919+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1198950371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 msgr2=0x5645e3a5c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:31.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.919+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 0x5645e3a5c170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000ab40 tx=0x7fb6f000af00 comp rx=0 tx=0).stop 2024-05-27T08:03:31.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.919+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1198950371 shutdown_connections 2024-05-27T08:03:31.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.919+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5645e3a6bd40 0x5645e3a6e190 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.919+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 0x5645e3a5c170 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.919+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e38d2e40 0x5645e3a5b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.919+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1198950371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e39be9f0 0x5645e3a5bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.919+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1198950371 >> 172.21.15.90:0/1198950371 conn(0x5645e396c390 msgr2=0x5645e3a513b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:31.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.919+0000 7fb6df400700 0 client.37943 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T08:03:31.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.920+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1198950371 shutdown_connections 2024-05-27T08:03:31.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.920+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1198950371 wait complete. 2024-05-27T08:03:31.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.920+0000 7fb70ead3b80 20 client.37943 trim_cache size 0 max 16384 2024-05-27T08:03:31.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.944+0000 7fb70ead3b80 1 Processor -- start 2024-05-27T08:03:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.944+0000 7fb70ead3b80 1 -- start start 2024-05-27T08:03:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.945+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3a5a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.945+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e39be9f0 0x5645e3a4d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.945+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 0x5645e3a67cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.945+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d80107c0 con 0x5645e38d2e40 2024-05-27T08:03:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.945+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d800d810 con 0x5645e3a51e70 2024-05-27T08:03:31.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.945+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d800fa20 con 0x5645e39be9f0 2024-05-27T08:03:31.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.945+0000 7fb6fd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3a5a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.945+0000 7fb6fd600700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3a5a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37196/0 (socket says 172.21.15.90:37196) 2024-05-27T08:03:31.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.946+0000 7fb6fe000700 1 --2- 172.21.15.90:0/3858674021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 0x5645e3a67cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.946+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/3858674021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e39be9f0 0x5645e3a4d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.946+0000 7fb6fd600700 1 -- 172.21.15.90:0/3858674021 learned_addr learned my addr 172.21.15.90:0/3858674021 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:31.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.947+0000 7fb6fcc00700 1 -- 172.21.15.90:0/3858674021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 msgr2=0x5645e3a67cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:31.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.947+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/3858674021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 0x5645e3a67cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.947+0000 7fb6fcc00700 1 -- 172.21.15.90:0/3858674021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 msgr2=0x5645e3a5a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:31.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.947+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/3858674021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3a5a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.947+0000 7fb6fcc00700 1 -- 172.21.15.90:0/3858674021 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e3a52a50 con 0x5645e39be9f0 2024-05-27T08:03:31.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.947+0000 7fb6fd600700 1 --2- 172.21.15.90:0/3858674021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3a5a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:31.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.948+0000 7fb6fcc00700 1 --2- 172.21.15.90:0/3858674021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e39be9f0 0x5645e3a4d590 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4015fd0 tx=0x7fb6e4006e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e1a537b6c5c03c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:31.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.949+0000 7fb6ea000700 1 -- 172.21.15.90:0/3858674021 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4015780 con 0x5645e39be9f0 2024-05-27T08:03:31.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.949+0000 7fb6ea000700 1 -- 172.21.15.90:0/3858674021 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6e4024050 con 0x5645e39be9f0 2024-05-27T08:03:31.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.949+0000 7fb6ea000700 1 -- 172.21.15.90:0/3858674021 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4032ba0 con 0x5645e39be9f0 2024-05-27T08:03:31.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.950+0000 7fb70ead3b80 1 -- 172.21.15.90:0/3858674021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e39be9f0 msgr2=0x5645e3a4d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:31.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.950+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/3858674021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e39be9f0 0x5645e3a4d590 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4015fd0 tx=0x7fb6e4006e10 comp rx=0 tx=0).stop 2024-05-27T08:03:31.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.951+0000 7fb70ead3b80 1 -- 172.21.15.90:0/3858674021 shutdown_connections 2024-05-27T08:03:31.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.951+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/3858674021 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e3a51e70 0x5645e3a67cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.951+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/3858674021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3a5a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.951+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/3858674021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e39be9f0 0x5645e3a4d590 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.951+0000 7fb70ead3b80 1 -- 172.21.15.90:0/3858674021 >> 172.21.15.90:0/3858674021 conn(0x5645e396c390 msgr2=0x5645e3a4ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:31.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.951+0000 7fb70ead3b80 1 -- 172.21.15.90:0/3858674021 shutdown_connections 2024-05-27T08:03:31.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.951+0000 7fb70ead3b80 1 -- 172.21.15.90:0/3858674021 wait complete. 2024-05-27T08:03:31.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.952+0000 7fb70ead3b80 1 -- start start 2024-05-27T08:03:31.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.952+0000 7fb70ead3b80 1 Processor -- start 2024-05-27T08:03:31.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.954+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3acefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.954+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e39be9f0 0x5645e3acf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.954+0000 7fb70ead3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e3a51e70 0x5645e3a62e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.954+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d800f350 con 0x5645e38d2e40 2024-05-27T08:03:31.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.954+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d800dee0 con 0x5645e39be9f0 2024-05-27T08:03:31.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.954+0000 7fb70ead3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d800e5b0 con 0x5645e3a51e70 2024-05-27T08:03:31.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.955+0000 7fb70ead3b80 10 client.0 _ll_register_callbacks cb 0x5645e3919c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.955+0000 7fb6fe000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e3a51e70 0x5645e3a62e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.955+0000 7fb6fe000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e3a51e70 0x5645e3a62e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:45546/0 (socket says 172.21.15.90:45546) 2024-05-27T08:03:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.955+0000 7fb6fe000700 1 -- 172.21.15.90:0/1382925638 learned_addr learned my addr 172.21.15.90:0/1382925638 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.955+0000 7fb6fd600700 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3acefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.956+0000 7fb6fe000700 1 -- 172.21.15.90:0/1382925638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e39be9f0 msgr2=0x5645e3acf4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:03:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.956+0000 7fb6fe000700 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e39be9f0 0x5645e3acf4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.956+0000 7fb6fe000700 1 -- 172.21.15.90:0/1382925638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 msgr2=0x5645e3acefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.956+0000 7fb6fe000700 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3acefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.956+0000 7fb6fe000700 1 -- 172.21.15.90:0/1382925638 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e3a633d0 con 0x5645e3a51e70 2024-05-27T08:03:31.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.956+0000 7fb6fd600700 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3acefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:31.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.957+0000 7fb6fe000700 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e3a51e70 0x5645e3a62e90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0000f80 tx=0x7fb6f000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7913e2842cd28a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:31.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.957+0000 7fb6df400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:03:31.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.957+0000 7fb6df400700 1 -- 172.21.15.90:0/1382925638 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0018040 con 0x5645e3a51e70 2024-05-27T08:03:31.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.957+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1382925638 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6e4024050 con 0x5645e3a51e70 2024-05-27T08:03:31.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.957+0000 7fb6df400700 1 -- 172.21.15.90:0/1382925638 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7fb6f000e9f0 con 0x5645e3a51e70 2024-05-27T08:03:31.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.958+0000 7fb6df400700 1 -- 172.21.15.90:0/1382925638 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb6f0018040 con 0x5645e3a51e70 2024-05-27T08:03:31.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.958+0000 7fb70ead3b80 10 client.28018 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:31.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.958+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1382925638 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5645e3a64650 con 0x5645e3a51e70 2024-05-27T08:03:31.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.958+0000 7fb6df400700 1 -- 172.21.15.90:0/1382925638 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6e4024050 con 0x5645e3a51e70 2024-05-27T08:03:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.959+0000 7fb6df400700 1 -- 172.21.15.90:0/1382925638 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fb6f0007900 con 0x5645e3a51e70 2024-05-27T08:03:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.959+0000 7fb70ead3b80 10 client.28018 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.959+0000 7fb70ead3b80 10 client.28018 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T08:03:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.959+0000 7fb70ead3b80 20 client.28018 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.959+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5645e3ae7830 0x5645e3ae9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.959+0000 7fb70ead3b80 4 client.28018 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:03:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.959+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1382925638 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645e397b570 con 0x5645e3ae7830 2024-05-27T08:03:31.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.959+0000 7fb6fd600700 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5645e3ae7830 0x5645e3ae9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.961+0000 7fb6fd600700 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5645e3ae7830 0x5645e3ae9c50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff94be877e8241f7 server_cookie=479c2b019d2782fc in_seq=0 out_seq=0 2024-05-27T08:03:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.961+0000 7fb6df400700 10 client.28018 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T08:03:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.962+0000 7fb6df400700 1 -- 172.21.15.90:0/1382925638 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645e397b570 con 0x5645e3ae7830 2024-05-27T08:03:31.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.962+0000 7fb6df400700 10 client.28018 handle_command_reply: tid=0 2024-05-27T08:03:31.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.962+0000 7fb70ead3b80 1 client.28018 shutdown 2024-05-27T08:03:31.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.963+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1382925638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e3a51e70 msgr2=0x5645e3a62e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:31.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.963+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e3a51e70 0x5645e3a62e90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0000f80 tx=0x7fb6f000ee70 comp rx=0 tx=0).stop 2024-05-27T08:03:31.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.963+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1382925638 shutdown_connections 2024-05-27T08:03:31.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.963+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5645e3ae7830 0x5645e3ae9c50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.963+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645e3a51e70 0x5645e3a62e90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.963+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5645e38d2e40 0x5645e3acefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.963+0000 7fb70ead3b80 1 --2- 172.21.15.90:0/1382925638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645e39be9f0 0x5645e3acf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.963+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1382925638 >> 172.21.15.90:0/1382925638 conn(0x5645e396c390 msgr2=0x5645e3a5b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.963+0000 7fb6df400700 0 client.28018 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T08:03:31.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.965+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1382925638 shutdown_connections 2024-05-27T08:03:31.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.965+0000 7fb70ead3b80 1 -- 172.21.15.90:0/1382925638 wait complete. 2024-05-27T08:03:31.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.965+0000 7fb70ead3b80 20 client.28018 trim_cache size 0 max 16384 2024-05-27T08:03:31.968 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:03:31.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.967+0000 7fb700000700 1 -- 172.21.15.90:0/163192370 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6e00b2b80 msgr2=0x7fb6e00b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:31.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.967+0000 7fb700000700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6e00b2b80 0x7fb6e00b5090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6f806faf0 tx=0x7fb6e4009190 comp rx=0 tx=0).stop 2024-05-27T08:03:31.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.967+0000 7fb700000700 1 -- 172.21.15.90:0/163192370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f810a920 msgr2=0x7fb6f811a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:31.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.967+0000 7fb700000700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f810a920 0x7fb6f811a030 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec003760 tx=0x7fb6ec0057c0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.967+0000 7fb700000700 1 -- 172.21.15.90:0/163192370 shutdown_connections 2024-05-27T08:03:31.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.967+0000 7fb700000700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6e00b2b80 0x7fb6e00b5090 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.967+0000 7fb700000700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6f8111bc0 0x7fb6f811aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.967+0000 7fb700000700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6f810a920 0x7fb6f811a030 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.967+0000 7fb700000700 1 --2- 172.21.15.90:0/163192370 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f810b270 0x7fb6f811a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.967+0000 7fb700000700 1 -- 172.21.15.90:0/163192370 >> 172.21.15.90:0/163192370 conn(0x7fb6f806f4c0 msgr2=0x7fb6f81106b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.968+0000 7fb700000700 1 -- 172.21.15.90:0/163192370 shutdown_connections 2024-05-27T08:03:31.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:31.968+0000 7fb700000700 1 -- 172.21.15.90:0/163192370 wait complete. 2024-05-27T08:03:32.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:32 smithi090 ceph-mon[31429]: pgmap v552: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 480 op/s 2024-05-27T08:03:32.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:32 smithi129 ceph-mon[35254]: pgmap v552: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 480 op/s 2024-05-27T08:03:32.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:32 smithi136 ceph-mon[35324]: pgmap v552: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 480 op/s 2024-05-27T08:03:32.963 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T08:03:33.680 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 795.0 s, 540.7 tps, lat 183.823 ms stddev 133.539 2024-05-27T08:03:34.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:34 smithi090 ceph-mon[31429]: pgmap v553: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 480 op/s 2024-05-27T08:03:34.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:34 smithi129 ceph-mon[35254]: pgmap v553: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 480 op/s 2024-05-27T08:03:34.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:34 smithi136 ceph-mon[35324]: pgmap v553: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 480 op/s 2024-05-27T08:03:36.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:36 smithi129 ceph-mon[35254]: pgmap v554: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.0 MiB/s wr, 572 op/s 2024-05-27T08:03:36.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:36 smithi090 ceph-mon[31429]: pgmap v554: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.0 MiB/s wr, 572 op/s 2024-05-27T08:03:36.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:36 smithi136 ceph-mon[35324]: pgmap v554: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.0 MiB/s wr, 572 op/s 2024-05-27T08:03:38.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:38 smithi129 ceph-mon[35254]: pgmap v555: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 413 op/s 2024-05-27T08:03:38.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:38 smithi090 ceph-mon[31429]: pgmap v555: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 413 op/s 2024-05-27T08:03:38.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:38 smithi136 ceph-mon[35324]: pgmap v555: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 413 op/s 2024-05-27T08:03:38.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 800.0 s, 477.2 tps, lat 209.120 ms stddev 160.413 2024-05-27T08:03:40.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.264+0000 7f8a67200700 1 -- 172.21.15.90:0/3862786500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a68109670 msgr2=0x7f8a68110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:40.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.264+0000 7f8a67200700 1 --2- 172.21.15.90:0/3862786500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a68109670 0x7f8a68110fd0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000b0d0 tx=0x7f8a6000b3e0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.264+0000 7f8a67200700 1 -- 172.21.15.90:0/3862786500 shutdown_connections 2024-05-27T08:03:40.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.264+0000 7f8a67200700 1 --2- 172.21.15.90:0/3862786500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a68109670 0x7f8a68110fd0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.264+0000 7f8a67200700 1 --2- 172.21.15.90:0/3862786500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a681043f0 0x7f8a68104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.264+0000 7f8a67200700 1 --2- 172.21.15.90:0/3862786500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108d20 0x7f8a68109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.264+0000 7f8a67200700 1 -- 172.21.15.90:0/3862786500 >> 172.21.15.90:0/3862786500 conn(0x7f8a68079d00 msgr2=0x7f8a6807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:40.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.264+0000 7f8a67200700 1 -- 172.21.15.90:0/3862786500 shutdown_connections 2024-05-27T08:03:40.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.264+0000 7f8a67200700 1 -- 172.21.15.90:0/3862786500 wait complete. 2024-05-27T08:03:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.265+0000 7f8a67200700 1 Processor -- start 2024-05-27T08:03:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.265+0000 7f8a67200700 1 -- start start 2024-05-27T08:03:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.265+0000 7f8a67200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a681043f0 0x7f8a681a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.265+0000 7f8a67200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108d20 0x7f8a681a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.265+0000 7f8a67200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a68109670 0x7f8a6819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.265+0000 7f8a67200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68114150 con 0x7f8a68109670 2024-05-27T08:03:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.265+0000 7f8a67200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68113fd0 con 0x7f8a681043f0 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.265+0000 7f8a67200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681142d0 con 0x7f8a68108d20 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.266+0000 7f8a66800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a68109670 0x7f8a6819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.266+0000 7f8a66800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a68109670 0x7f8a6819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54894/0 (socket says 172.21.15.90:54894) 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.266+0000 7f8a66800700 1 -- 172.21.15.90:0/1794997523 learned_addr learned my addr 172.21.15.90:0/1794997523 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.266+0000 7f8a65400700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108d20 0x7f8a681a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.266+0000 7f8a65e00700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a681043f0 0x7f8a681a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.266+0000 7f8a66800700 1 -- 172.21.15.90:0/1794997523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108d20 msgr2=0x7f8a681a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.266+0000 7f8a66800700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108d20 0x7f8a681a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.266+0000 7f8a66800700 1 -- 172.21.15.90:0/1794997523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a681043f0 msgr2=0x7f8a681a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.266+0000 7f8a66800700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a681043f0 0x7f8a681a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.266+0000 7f8a66800700 1 -- 172.21.15.90:0/1794997523 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6819b080 con 0x7f8a68109670 2024-05-27T08:03:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.267+0000 7f8a66800700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a68109670 0x7f8a6819a9d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009850 tx=0x7f8a60007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.267+0000 7f8a5ea00700 1 -- 172.21.15.90:0/1794997523 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6001f070 con 0x7f8a68109670 2024-05-27T08:03:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.267+0000 7f8a5ea00700 1 -- 172.21.15.90:0/1794997523 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a60005500 con 0x7f8a68109670 2024-05-27T08:03:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.267+0000 7f8a67200700 1 -- 172.21.15.90:0/1794997523 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6819b310 con 0x7f8a68109670 2024-05-27T08:03:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.267+0000 7f8a5ea00700 1 -- 172.21.15.90:0/1794997523 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6001b8f0 con 0x7f8a68109670 2024-05-27T08:03:40.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.267+0000 7f8a67200700 1 -- 172.21.15.90:0/1794997523 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6819b7f0 con 0x7f8a68109670 2024-05-27T08:03:40.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.271+0000 7f8a5ea00700 1 -- 172.21.15.90:0/1794997523 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8a60022070 con 0x7f8a68109670 2024-05-27T08:03:40.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.272+0000 7f8a5ea00700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a4c0b29f0 0x7f8a4c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.272+0000 7f8a5ea00700 1 -- 172.21.15.90:0/1794997523 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f8a6000f020 con 0x7f8a68109670 2024-05-27T08:03:40.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.272+0000 7f8a65e00700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a4c0b29f0 0x7f8a4c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.273+0000 7f8a65e00700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a4c0b29f0 0x7f8a4c0b4f00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8a6819bb10 tx=0x7f8a5800b410 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:40.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:40 smithi090 ceph-mon[31429]: pgmap v556: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.5 MiB/s wr, 541 op/s 2024-05-27T08:03:40.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:40 smithi136 ceph-mon[35324]: pgmap v556: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.5 MiB/s wr, 541 op/s 2024-05-27T08:03:40.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:40 smithi129 ceph-mon[35254]: pgmap v556: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.5 MiB/s wr, 541 op/s 2024-05-27T08:03:40.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.948+0000 7f8a7bbcbb80 1 Processor -- start 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.948+0000 7f8a7bbcbb80 1 -- start start 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.949+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 0x55e8e0dd8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.949+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.949+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd96d0 0x55e8e0dea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.949+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e8e0d07e40 con 0x55e8e0dd96d0 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.949+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8e0cf79e0 con 0x55e8e0c5ae40 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.949+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e8e0cfba20 con 0x55e8e0dd92c0 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.949+0000 7f8a66800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd96d0 0x55e8e0dea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.949+0000 7f8a66800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd96d0 0x55e8e0dea430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54906/0 (socket says 172.21.15.90:54906) 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.949+0000 7f8a66800700 1 -- 172.21.15.90:0/2461034899 learned_addr learned my addr 172.21.15.90:0/2461034899 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.949+0000 7f8a65e00700 1 --2- 172.21.15.90:0/2461034899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 0x55e8e0dd8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a65400700 1 --2- 172.21.15.90:0/2461034899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a66800700 1 -- 172.21.15.90:0/2461034899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 msgr2=0x55e8e0de7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a66800700 1 --2- 172.21.15.90:0/2461034899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a66800700 1 -- 172.21.15.90:0/2461034899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 msgr2=0x55e8e0dd8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a66800700 1 --2- 172.21.15.90:0/2461034899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 0x55e8e0dd8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a66800700 1 -- 172.21.15.90:0/2461034899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e0cc0420 con 0x55e8e0dd96d0 2024-05-27T08:03:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a65400700 1 --2- 172.21.15.90:0/2461034899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:40.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a66800700 1 --2- 172.21.15.90:0/2461034899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd96d0 0x55e8e0dea430 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8a600e10c0 tx=0x7f8a600aa5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617eeb1d210179a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:40.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a53400700 1 -- 172.21.15.90:0/2461034899 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a600aad40 con 0x55e8e0dd96d0 2024-05-27T08:03:40.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a53400700 1 -- 172.21.15.90:0/2461034899 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a600af0f0 con 0x55e8e0dd96d0 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.950+0000 7f8a53400700 1 -- 172.21.15.90:0/2461034899 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a600aafa0 con 0x55e8e0dd96d0 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.951+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/2461034899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd96d0 msgr2=0x55e8e0dea430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.951+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/2461034899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd96d0 0x55e8e0dea430 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8a600e10c0 tx=0x7f8a600aa5b0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.952+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/2461034899 shutdown_connections 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.952+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/2461034899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd96d0 0x55e8e0dea430 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.952+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/2461034899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 0x55e8e0dd8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.952+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/2461034899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.952+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/2461034899 >> 172.21.15.90:0/2461034899 conn(0x55e8e0cbafc0 msgr2=0x55e8e0dd4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.952+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/2461034899 shutdown_connections 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.952+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/2461034899 wait complete. 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.952+0000 7f8a7bbcbb80 1 -- start start 2024-05-27T08:03:40.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.952+0000 7f8a7bbcbb80 1 Processor -- start 2024-05-27T08:03:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.953+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.953+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.953+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 0x55e8e0de8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.953+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e8e0bfdfa0 con 0x55e8e0c5ae40 2024-05-27T08:03:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.953+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8e0c98760 con 0x55e8e0dd96d0 2024-05-27T08:03:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.953+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e8e0deab60 con 0x55e8e0dd92c0 2024-05-27T08:03:40.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.953+0000 7f8a7bbcbb80 10 client.0 _ll_register_callbacks cb 0x55e8e0ca1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:40.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.953+0000 7f8a65e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.953+0000 7f8a65e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54912/0 (socket says 172.21.15.90:54912) 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.953+0000 7f8a65e00700 1 -- 172.21.15.90:0/458791735 learned_addr learned my addr 172.21.15.90:0/458791735 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.954+0000 7f8a66800700 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 0x55e8e0de8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.954+0000 7f8a65400700 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.954+0000 7f8a65e00700 1 -- 172.21.15.90:0/458791735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 msgr2=0x55e8e0de7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.954+0000 7f8a65e00700 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.954+0000 7f8a65e00700 1 -- 172.21.15.90:0/458791735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 msgr2=0x55e8e0de8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.954+0000 7f8a65e00700 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 0x55e8e0de8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.954+0000 7f8a65e00700 1 -- 172.21.15.90:0/458791735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e0c478d0 con 0x55e8e0c5ae40 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.954+0000 7f8a66800700 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 0x55e8e0de8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:40.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.954+0000 7f8a65e00700 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de79e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8a58011770 tx=0x7f8a580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d9b2b93f63a6d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.954+0000 7f8a47e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a47e00700 1 -- 172.21.15.90:0/458791735 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58028670 con 0x55e8e0c5ae40 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/458791735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a600af0f0 con 0x55e8e0c5ae40 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a47e00700 1 -- 172.21.15.90:0/458791735 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f8a5801f430 con 0x55e8e0c5ae40 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a47e00700 1 -- 172.21.15.90:0/458791735 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8a58028810 con 0x55e8e0c5ae40 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a7bbcbb80 10 client.18249 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/458791735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e8e0cc8060 con 0x55e8e0c5ae40 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a47e00700 1 -- 172.21.15.90:0/458791735 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a600af0f0 con 0x55e8e0c5ae40 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a47e00700 1 -- 172.21.15.90:0/458791735 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f8a5801fb30 con 0x55e8e0c5ae40 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a7bbcbb80 10 client.18249 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a7bbcbb80 10 client.18249 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T08:03:40.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.955+0000 7f8a7bbcbb80 20 client.18249 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:40.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.956+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e8e0df7c30 0x55e8e0dfa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.956+0000 7f8a7bbcbb80 4 client.18249 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:40.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.956+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/458791735 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8e0d09d90 con 0x55e8e0df7c30 2024-05-27T08:03:40.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.956+0000 7f8a65400700 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e8e0df7c30 0x55e8e0dfa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.957+0000 7f8a47e00700 10 client.18249 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T08:03:40.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.957+0000 7f8a65400700 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e8e0df7c30 0x55e8e0dfa0e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fb6deb63ed3e0af2 server_cookie=59d82f23189debdf in_seq=0 out_seq=0 2024-05-27T08:03:40.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a47e00700 1 -- 172.21.15.90:0/458791735 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e8e0d09d90 con 0x55e8e0df7c30 2024-05-27T08:03:40.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a47e00700 10 client.18249 handle_command_reply: tid=0 2024-05-27T08:03:40.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 client.18249 shutdown 2024-05-27T08:03:40.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/458791735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 msgr2=0x55e8e0de79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de79e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8a58011770 tx=0x7f8a580047c0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/458791735 shutdown_connections 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e8e0df7c30 0x55e8e0dfa0e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 0x55e8e0de8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de79e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/458791735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/458791735 >> 172.21.15.90:0/458791735 conn(0x55e8e0cbafc0 msgr2=0x55e8e0dd3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a47e00700 0 client.18249 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/458791735 shutdown_connections 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.959+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/458791735 wait complete. 2024-05-27T08:03:40.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.960+0000 7f8a7bbcbb80 20 client.18249 trim_cache size 0 max 16384 2024-05-27T08:03:40.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.977+0000 7f8a7bbcbb80 1 Processor -- start 2024-05-27T08:03:40.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.977+0000 7f8a7bbcbb80 1 -- start start 2024-05-27T08:03:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.977+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.977+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.977+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.977+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681afbb0 con 0x55e8e0c5ae40 2024-05-27T08:03:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.977+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681acc00 con 0x55e8e0dd96d0 2024-05-27T08:03:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.977+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681aee10 con 0x55e8e0dd92c0 2024-05-27T08:03:40.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a65e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a65e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54920/0 (socket says 172.21.15.90:54920) 2024-05-27T08:03:40.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a65e00700 1 -- 172.21.15.90:0/4176094206 learned_addr learned my addr 172.21.15.90:0/4176094206 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:40.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a65400700 1 --2- 172.21.15.90:0/4176094206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a66800700 1 --2- 172.21.15.90:0/4176094206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a65e00700 1 -- 172.21.15.90:0/4176094206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 msgr2=0x55e8e0de31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:40.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a65e00700 1 --2- 172.21.15.90:0/4176094206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a65e00700 1 -- 172.21.15.90:0/4176094206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 msgr2=0x55e8e0ddf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:40.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a65e00700 1 --2- 172.21.15.90:0/4176094206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a65e00700 1 -- 172.21.15.90:0/4176094206 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e0de5f50 con 0x55e8e0c5ae40 2024-05-27T08:03:40.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.978+0000 7f8a66800700 1 --2- 172.21.15.90:0/4176094206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:40.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.979+0000 7f8a65e00700 1 --2- 172.21.15.90:0/4176094206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de6940 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8a5801fea0 tx=0x7f8a58010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1c19ee2d675689 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:40.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.980+0000 7f8a52a00700 1 -- 172.21.15.90:0/4176094206 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5803c020 con 0x55e8e0c5ae40 2024-05-27T08:03:40.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.980+0000 7f8a52a00700 1 -- 172.21.15.90:0/4176094206 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a58023070 con 0x55e8e0c5ae40 2024-05-27T08:03:40.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.980+0000 7f8a52a00700 1 -- 172.21.15.90:0/4176094206 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5801b330 con 0x55e8e0c5ae40 2024-05-27T08:03:40.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.981+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/4176094206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 msgr2=0x55e8e0de6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:40.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.981+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/4176094206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de6940 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8a5801fea0 tx=0x7f8a58010ec0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.981+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/4176094206 shutdown_connections 2024-05-27T08:03:40.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.981+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/4176094206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.981+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/4176094206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0c5ae40 0x55e8e0de6940 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.981+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/4176094206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd92c0 0x55e8e0de31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.981+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/4176094206 >> 172.21.15.90:0/4176094206 conn(0x55e8e0cbafc0 msgr2=0x55e8e0de81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:40.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.981+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/4176094206 shutdown_connections 2024-05-27T08:03:40.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.982+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/4176094206 wait complete. 2024-05-27T08:03:40.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.982+0000 7f8a7bbcbb80 1 -- start start 2024-05-27T08:03:40.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.983+0000 7f8a7bbcbb80 1 Processor -- start 2024-05-27T08:03:40.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.984+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 0x55e8e0ddbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.984+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd92c0 0x55e8e0ddc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.984+0000 7f8a7bbcbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:40.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.984+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681ae740 con 0x55e8e0dd92c0 2024-05-27T08:03:40.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.984+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681ad2d0 con 0x55e8e0c5ae40 2024-05-27T08:03:40.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.987+0000 7f8a7bbcbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681ad9a0 con 0x55e8e0dd96d0 2024-05-27T08:03:40.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.987+0000 7f8a7bbcbb80 10 client.0 _ll_register_callbacks cb 0x55e8e0ca1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:40.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.989+0000 7f8a66800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.989+0000 7f8a65e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 0x55e8e0ddbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.989+0000 7f8a66800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:60684/0 (socket says 172.21.15.90:60684) 2024-05-27T08:03:40.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.989+0000 7f8a65e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 0x55e8e0ddbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:44054/0 (socket says 172.21.15.90:44054) 2024-05-27T08:03:40.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.989+0000 7f8a66800700 1 -- 172.21.15.90:0/40845516 learned_addr learned my addr 172.21.15.90:0/40845516 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:40.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.990+0000 7f8a65400700 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd92c0 0x55e8e0ddc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:40.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.990+0000 7f8a65e00700 1 -- 172.21.15.90:0/40845516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd96d0 msgr2=0x55e8e0ddc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:40.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.990+0000 7f8a65e00700 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.990+0000 7f8a65e00700 1 -- 172.21.15.90:0/40845516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd92c0 msgr2=0x55e8e0ddc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:40.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.990+0000 7f8a65e00700 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd92c0 0x55e8e0ddc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:40.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.990+0000 7f8a65e00700 1 -- 172.21.15.90:0/40845516 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e0deeee0 con 0x55e8e0c5ae40 2024-05-27T08:03:40.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.991+0000 7f8a66800700 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:40.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.991+0000 7f8a65400700 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd92c0 0x55e8e0ddc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.992+0000 7f8a65e00700 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 0x55e8e0ddbeb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8a58000c00 tx=0x7f8a5801bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e7be38835410d11 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.992+0000 7f8a47e00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:03:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.993+0000 7f8a47e00700 1 -- 172.21.15.90:0/40845516 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58044020 con 0x55e8e0c5ae40 2024-05-27T08:03:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.993+0000 7f8a47e00700 1 -- 172.21.15.90:0/40845516 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f8a58028dc0 con 0x55e8e0c5ae40 2024-05-27T08:03:40.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.993+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/40845516 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a58023070 con 0x55e8e0c5ae40 2024-05-27T08:03:40.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.994+0000 7f8a47e00700 1 -- 172.21.15.90:0/40845516 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8a5803c040 con 0x55e8e0c5ae40 2024-05-27T08:03:40.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.994+0000 7f8a7bbcbb80 10 client.37961 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:40.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.994+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/40845516 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e8e0df00b0 con 0x55e8e0c5ae40 2024-05-27T08:03:40.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.995+0000 7f8a47e00700 1 -- 172.21.15.90:0/40845516 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a58023070 con 0x55e8e0c5ae40 2024-05-27T08:03:40.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.998+0000 7f8a47e00700 1 -- 172.21.15.90:0/40845516 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f8a580053b0 con 0x55e8e0c5ae40 2024-05-27T08:03:41.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.998+0000 7f8a7bbcbb80 10 client.37961 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:41.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.998+0000 7f8a7bbcbb80 10 client.37961 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T08:03:41.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.999+0000 7f8a7bbcbb80 20 client.37961 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:41.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.999+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e8e0df0700 0x55e8e0e68f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:41.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.999+0000 7f8a7bbcbb80 4 client.37961 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:03:41.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:40.999+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/40845516 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8e0d03570 con 0x55e8e0df0700 2024-05-27T08:03:41.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.000+0000 7f8a65400700 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e8e0df0700 0x55e8e0e68f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:41.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.005+0000 7f8a47e00700 10 client.37961 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T08:03:41.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.005+0000 7f8a65400700 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e8e0df0700 0x55e8e0e68f70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90bd29a623005bfe server_cookie=7f42e21b830b7ae1 in_seq=0 out_seq=0 2024-05-27T08:03:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a47e00700 1 -- 172.21.15.90:0/40845516 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8e0d03570 con 0x55e8e0df0700 2024-05-27T08:03:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a47e00700 10 client.37961 handle_command_reply: tid=0 2024-05-27T08:03:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a7bbcbb80 1 client.37961 shutdown 2024-05-27T08:03:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/40845516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 msgr2=0x55e8e0ddbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 0x55e8e0ddbeb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8a58000c00 tx=0x7f8a5801bc80 comp rx=0 tx=0).stop 2024-05-27T08:03:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/40845516 shutdown_connections 2024-05-27T08:03:41.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55e8e0df0700 0x55e8e0e68f70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e8e0dd96d0 0x55e8e0ddc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8e0c5ae40 0x55e8e0ddbeb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a7bbcbb80 1 --2- 172.21.15.90:0/40845516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e8e0dd92c0 0x55e8e0ddc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.008+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/40845516 >> 172.21.15.90:0/40845516 conn(0x55e8e0cbafc0 msgr2=0x55e8e0dd4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:41.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.009+0000 7f8a47e00700 0 client.37961 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T08:03:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.009+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/40845516 shutdown_connections 2024-05-27T08:03:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.009+0000 7f8a7bbcbb80 1 -- 172.21.15.90:0/40845516 wait complete. 2024-05-27T08:03:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.009+0000 7f8a7bbcbb80 20 client.37961 trim_cache size 0 max 16384 2024-05-27T08:03:41.012 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:03:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.012+0000 7f8a67200700 1 -- 172.21.15.90:0/1794997523 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a4c0b29f0 msgr2=0x7f8a4c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.012+0000 7f8a67200700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a4c0b29f0 0x7f8a4c0b4f00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8a6819bb10 tx=0x7f8a5800b410 comp rx=0 tx=0).stop 2024-05-27T08:03:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.012+0000 7f8a67200700 1 -- 172.21.15.90:0/1794997523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a68109670 msgr2=0x7f8a6819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.012+0000 7f8a67200700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a68109670 0x7f8a6819a9d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009850 tx=0x7f8a60007d90 comp rx=0 tx=0).stop 2024-05-27T08:03:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.012+0000 7f8a67200700 1 -- 172.21.15.90:0/1794997523 shutdown_connections 2024-05-27T08:03:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.012+0000 7f8a67200700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a68109670 0x7f8a6819a9d0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.012+0000 7f8a67200700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a4c0b29f0 0x7f8a4c0b4f00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.012+0000 7f8a67200700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a681043f0 0x7f8a681a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.012+0000 7f8a67200700 1 --2- 172.21.15.90:0/1794997523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108d20 0x7f8a681a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.012+0000 7f8a67200700 1 -- 172.21.15.90:0/1794997523 >> 172.21.15.90:0/1794997523 conn(0x7f8a68079d00 msgr2=0x7f8a68101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.013+0000 7f8a67200700 1 -- 172.21.15.90:0/1794997523 shutdown_connections 2024-05-27T08:03:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:41.013+0000 7f8a67200700 1 -- 172.21.15.90:0/1794997523 wait complete. 2024-05-27T08:03:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.317+0000 7f30c2600640 1 -- 172.21.15.90:0/4196047030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30b4096af0 msgr2=0x7f30b4096f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.317+0000 7f30c2600640 1 --2- 172.21.15.90:0/4196047030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30b4096af0 0x7f30b4096f00 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009900 tx=0x7f30a802a3f0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.318+0000 7f30c2600640 1 -- 172.21.15.90:0/4196047030 shutdown_connections 2024-05-27T08:03:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.318+0000 7f30c2600640 1 --2- 172.21.15.90:0/4196047030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30b40a1380 0x7f30b40a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.318+0000 7f30c2600640 1 --2- 172.21.15.90:0/4196047030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30b4096af0 0x7f30b4096f00 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.318+0000 7f30c2600640 1 --2- 172.21.15.90:0/4196047030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b409ca20 0x7f30b40965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.318+0000 7f30c2600640 1 -- 172.21.15.90:0/4196047030 >> 172.21.15.90:0/4196047030 conn(0x7f30b400c120 msgr2=0x7f30b400c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.318+0000 7f30c2600640 1 -- 172.21.15.90:0/4196047030 shutdown_connections 2024-05-27T08:03:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.318+0000 7f30c2600640 1 -- 172.21.15.90:0/4196047030 wait complete. 2024-05-27T08:03:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.320+0000 7f30c2600640 1 Processor -- start 2024-05-27T08:03:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.321+0000 7f30c2600640 1 -- start start 2024-05-27T08:03:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.323+0000 7f30c2600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b4096af0 0x7f30b40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.323+0000 7f30c2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30b409ca20 0x7f30b40a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.323+0000 7f30c2600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30b40a1380 0x7f30b40a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.323+0000 7f30c2600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f30b40092a0 con 0x7f30b409ca20 2024-05-27T08:03:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.323+0000 7f30c2600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4009120 con 0x7f30b40a1380 2024-05-27T08:03:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.323+0000 7f30c2600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4009420 con 0x7f30b4096af0 2024-05-27T08:03:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.323+0000 7f30c1200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b4096af0 0x7f30b40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.323+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30b409ca20 0x7f30b40a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.323+0000 7f30c1200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b4096af0 0x7f30b40a6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:60700/0 (socket says 172.21.15.90:60700) 2024-05-27T08:03:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.323+0000 7f30c1200640 1 -- 172.21.15.90:0/56352048 learned_addr learned my addr 172.21.15.90:0/56352048 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.324+0000 7f30c1c00640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30b40a1380 0x7f30b40a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.324+0000 7f30bbe00640 1 -- 172.21.15.90:0/56352048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b4096af0 msgr2=0x7f30b40a6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.324+0000 7f30bbe00640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b4096af0 0x7f30b40a6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.324+0000 7f30bbe00640 1 -- 172.21.15.90:0/56352048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30b40a1380 msgr2=0x7f30b40a6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.324+0000 7f30bbe00640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30b40a1380 0x7f30b40a6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.324+0000 7f30bbe00640 1 -- 172.21.15.90:0/56352048 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b4150a90 con 0x7f30b409ca20 2024-05-27T08:03:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.325+0000 7f30bbe00640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30b409ca20 0x7f30b40a6790 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f30a80057c0 tx=0x7f30a80305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.328+0000 7f30b9600640 1 -- 172.21.15.90:0/56352048 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a803f030 con 0x7f30b409ca20 2024-05-27T08:03:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.328+0000 7f30b9600640 1 -- 172.21.15.90:0/56352048 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30a8030de0 con 0x7f30b409ca20 2024-05-27T08:03:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.328+0000 7f30b9600640 1 -- 172.21.15.90:0/56352048 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8035070 con 0x7f30b409ca20 2024-05-27T08:03:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.328+0000 7f30c2600640 1 -- 172.21.15.90:0/56352048 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b4150d20 con 0x7f30b409ca20 2024-05-27T08:03:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.328+0000 7f30c2600640 1 -- 172.21.15.90:0/56352048 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b4151190 con 0x7f30b409ca20 2024-05-27T08:03:42.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.339+0000 7f30cab97740 1 Processor -- start 2024-05-27T08:03:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.340+0000 7f30cab97740 1 -- start start 2024-05-27T08:03:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.340+0000 7f30cab97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 0x55e9f4593a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.340+0000 7f30cab97740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41c82f0 0x55e9f4593fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.340+0000 7f30b9600640 1 -- 172.21.15.90:0/56352048 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f30a8042030 con 0x7f30b409ca20 2024-05-27T08:03:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.340+0000 7f30cab97740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 0x55e9f4594510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.340+0000 7f30cab97740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e9f4504b90 con 0x55e9f41d04d0 2024-05-27T08:03:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.340+0000 7f30cab97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e9f44f4730 con 0x55e9f41f11b0 2024-05-27T08:03:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.340+0000 7f30cab97740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e9f44f8770 con 0x55e9f41c82f0 2024-05-27T08:03:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.341+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 0x55e9f4594510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.341+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 0x55e9f4594510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54946/0 (socket says 172.21.15.90:54946) 2024-05-27T08:03:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.341+0000 7f30c1c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 0x55e9f4593a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.342+0000 7f30c1200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41c82f0 0x55e9f4593fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.342+0000 7f30c1c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 0x55e9f4593a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:44074/0 (socket says 172.21.15.90:44074) 2024-05-27T08:03:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.342+0000 7f30c1c00640 1 -- 172.21.15.90:0/1131050694 learned_addr learned my addr 172.21.15.90:0/1131050694 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.342+0000 7f30bbe00640 1 -- 172.21.15.90:0/1131050694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41c82f0 msgr2=0x55e9f4593fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.342+0000 7f30bbe00640 1 --2- 172.21.15.90:0/1131050694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41c82f0 0x55e9f4593fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.343+0000 7f30bbe00640 1 -- 172.21.15.90:0/1131050694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 msgr2=0x55e9f4593a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.343+0000 7f30bbe00640 1 --2- 172.21.15.90:0/1131050694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 0x55e9f4593a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.343+0000 7f30bbe00640 1 -- 172.21.15.90:0/1131050694 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f448d8b0 con 0x55e9f41d04d0 2024-05-27T08:03:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.343+0000 7f30b9600640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30a40b2b70 0x7f30a40b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.343+0000 7f30c1c00640 1 --2- 172.21.15.90:0/1131050694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 0x55e9f4593a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.344+0000 7f30bbe00640 1 --2- 172.21.15.90:0/1131050694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 0x55e9f4594510 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f30a8100830 tx=0x7f30a8109510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32457dd0177884fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.345+0000 7f30c1200640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30a40b2b70 0x7f30a40b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.347+0000 7f30b9600640 1 -- 172.21.15.90:0/56352048 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f30a8103510 con 0x7f30b409ca20 2024-05-27T08:03:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.347+0000 7f30c1200640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30a40b2b70 0x7f30a40b5080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55e9f4594e40 tx=0x7f30b0005f90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.347+0000 7f30ae000640 1 -- 172.21.15.90:0/1131050694 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8110070 con 0x55e9f41d04d0 2024-05-27T08:03:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.348+0000 7f30ae000640 1 -- 172.21.15.90:0/1131050694 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30a80c9ed0 con 0x55e9f41d04d0 2024-05-27T08:03:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.348+0000 7f30ae000640 1 -- 172.21.15.90:0/1131050694 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a80c8ed0 con 0x55e9f41d04d0 2024-05-27T08:03:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.349+0000 7f30cab97740 1 -- 172.21.15.90:0/1131050694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 msgr2=0x55e9f4594510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:42.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.349+0000 7f30cab97740 1 --2- 172.21.15.90:0/1131050694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 0x55e9f4594510 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f30a8100830 tx=0x7f30a8109510 comp rx=0 tx=0).stop 2024-05-27T08:03:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.349+0000 7f30cab97740 1 -- 172.21.15.90:0/1131050694 shutdown_connections 2024-05-27T08:03:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.349+0000 7f30cab97740 1 --2- 172.21.15.90:0/1131050694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 0x55e9f4594510 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.349+0000 7f30cab97740 1 --2- 172.21.15.90:0/1131050694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41c82f0 0x55e9f4593fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.349+0000 7f30cab97740 1 --2- 172.21.15.90:0/1131050694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 0x55e9f4593a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.349+0000 7f30cab97740 1 -- 172.21.15.90:0/1131050694 >> 172.21.15.90:0/1131050694 conn(0x55e9f41e5300 msgr2=0x55e9f4590240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.350+0000 7f30cab97740 1 -- 172.21.15.90:0/1131050694 shutdown_connections 2024-05-27T08:03:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.350+0000 7f30cab97740 1 -- 172.21.15.90:0/1131050694 wait complete. 2024-05-27T08:03:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.350+0000 7f30cab97740 1 -- start start 2024-05-27T08:03:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.351+0000 7f30cab97740 1 Processor -- start 2024-05-27T08:03:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.352+0000 7f30cab97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41c82f0 0x55e9f459df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.352+0000 7f30cab97740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 0x55e9f459e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.352+0000 7f30cab97740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 0x55e9f45a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.352+0000 7f30cab97740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e9f41da790 con 0x55e9f41d04d0 2024-05-27T08:03:42.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.352+0000 7f30cab97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e9f449cbc0 con 0x55e9f41c82f0 2024-05-27T08:03:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.352+0000 7f30cab97740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e9f45a0a40 con 0x55e9f41f11b0 2024-05-27T08:03:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.352+0000 7f30cab97740 10 client.0 _ll_register_callbacks cb 0x55e9f447a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.352+0000 7f30c1200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41c82f0 0x55e9f459df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.352+0000 7f30c1200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41c82f0 0x55e9f459df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:44078/0 (socket says 172.21.15.90:44078) 2024-05-27T08:03:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.352+0000 7f30c1200640 1 -- 172.21.15.90:0/2702260364 learned_addr learned my addr 172.21.15.90:0/2702260364 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.353+0000 7f30c1c00640 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 0x55e9f45a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.353+0000 7f30c1200640 1 -- 172.21.15.90:0/2702260364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 msgr2=0x55e9f45a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.353+0000 7f30c1200640 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 0x55e9f45a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.353+0000 7f30c1200640 1 -- 172.21.15.90:0/2702260364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 msgr2=0x55e9f459e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.353+0000 7f30c1200640 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 0x55e9f459e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.353+0000 7f30c1200640 1 -- 172.21.15.90:0/2702260364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f4209210 con 0x55e9f41c82f0 2024-05-27T08:03:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.354+0000 7f30c1200640 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41c82f0 0x55e9f459df00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f30b0004620 tx=0x7f30b0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8be69082ec9b29ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.354+0000 7f30a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:03:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.355+0000 7f30a2a00640 1 -- 172.21.15.90:0/2702260364 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b001f700 con 0x55e9f41c82f0 2024-05-27T08:03:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.355+0000 7f30a2a00640 1 -- 172.21.15.90:0/2702260364 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f30b001f8a0 con 0x55e9f41c82f0 2024-05-27T08:03:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.355+0000 7f30cab97740 1 -- 172.21.15.90:0/2702260364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30a80c9ed0 con 0x55e9f41c82f0 2024-05-27T08:03:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.356+0000 7f30a2a00640 1 -- 172.21.15.90:0/2702260364 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f30b001faa0 con 0x55e9f41c82f0 2024-05-27T08:03:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.356+0000 7f30cab97740 10 client.37973 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.356+0000 7f30cab97740 1 -- 172.21.15.90:0/2702260364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e9f42d5620 con 0x55e9f41c82f0 2024-05-27T08:03:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.357+0000 7f30a2a00640 1 -- 172.21.15.90:0/2702260364 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30a80c9ed0 con 0x55e9f41c82f0 2024-05-27T08:03:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.357+0000 7f30a2a00640 1 -- 172.21.15.90:0/2702260364 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f30b0027070 con 0x55e9f41c82f0 2024-05-27T08:03:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.357+0000 7f30cab97740 10 client.37973 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.357+0000 7f30cab97740 10 client.37973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.357+0000 7f30cab97740 20 client.37973 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.357+0000 7f30cab97740 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e9f41afd80 0x55e9f45b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.357+0000 7f30cab97740 4 client.37973 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.357+0000 7f30cab97740 1 -- 172.21.15.90:0/2702260364 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9f4506ae0 con 0x55e9f41afd80 2024-05-27T08:03:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.357+0000 7f30bbe00640 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e9f41afd80 0x55e9f45b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.358+0000 7f30bbe00640 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e9f41afd80 0x55e9f45b00e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d353c03ef2c1ab server_cookie=dfa3d919fd788567 in_seq=0 out_seq=0 2024-05-27T08:03:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.358+0000 7f30a2a00640 10 client.37973 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.359+0000 7f30a2a00640 1 -- 172.21.15.90:0/2702260364 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55e9f4506ae0 con 0x55e9f41afd80 2024-05-27T08:03:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.359+0000 7f30a2a00640 10 client.37973 handle_command_reply: tid=0 2024-05-27T08:03:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.360+0000 7f30cab97740 1 client.37973 shutdown 2024-05-27T08:03:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.360+0000 7f30cab97740 1 -- 172.21.15.90:0/2702260364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41c82f0 msgr2=0x55e9f459df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.360+0000 7f30cab97740 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41c82f0 0x55e9f459df00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f30b0004620 tx=0x7f30b0004730 comp rx=0 tx=0).stop 2024-05-27T08:03:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.361+0000 7f30cab97740 1 -- 172.21.15.90:0/2702260364 shutdown_connections 2024-05-27T08:03:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.361+0000 7f30cab97740 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e9f41afd80 0x55e9f45b00e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.361+0000 7f30cab97740 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 0x55e9f45a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.361+0000 7f30cab97740 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41d04d0 0x55e9f459e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.361+0000 7f30cab97740 1 --2- 172.21.15.90:0/2702260364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41c82f0 0x55e9f459df00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.361+0000 7f30a2a00640 0 client.37973 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.361+0000 7f30cab97740 1 -- 172.21.15.90:0/2702260364 >> 172.21.15.90:0/2702260364 conn(0x55e9f41e5300 msgr2=0x55e9f45957b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.361+0000 7f30cab97740 1 -- 172.21.15.90:0/2702260364 shutdown_connections 2024-05-27T08:03:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.361+0000 7f30cab97740 1 -- 172.21.15.90:0/2702260364 wait complete. 2024-05-27T08:03:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.361+0000 7f30cab97740 20 client.37973 trim_cache size 0 max 16384 2024-05-27T08:03:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.401+0000 7f30cab97740 1 Processor -- start 2024-05-27T08:03:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.401+0000 7f30cab97740 1 -- start start 2024-05-27T08:03:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.402+0000 7f30cab97740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f459f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.402+0000 7f30cab97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41d04d0 0x55e9f45a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.402+0000 7f30cab97740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 0x55e9f45aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.402+0000 7f30cab97740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f30b415cd00 con 0x55e9f41c82f0 2024-05-27T08:03:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.402+0000 7f30cab97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4159d50 con 0x55e9f41d04d0 2024-05-27T08:03:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.402+0000 7f30cab97740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30b415bf60 con 0x55e9f41f11b0 2024-05-27T08:03:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.403+0000 7f30c1c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 0x55e9f45aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.403+0000 7f30c1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f459f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.403+0000 7f30c1c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 0x55e9f45aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:60734/0 (socket says 172.21.15.90:60734) 2024-05-27T08:03:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.403+0000 7f30c1c00640 1 -- 172.21.15.90:0/3615136731 learned_addr learned my addr 172.21.15.90:0/3615136731 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.403+0000 7f30bbe00640 1 --2- 172.21.15.90:0/3615136731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41d04d0 0x55e9f45a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.404+0000 7f30c1200640 1 -- 172.21.15.90:0/3615136731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 msgr2=0x55e9f45aa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.404+0000 7f30c1200640 1 --2- 172.21.15.90:0/3615136731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 0x55e9f45aa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.404+0000 7f30c1200640 1 -- 172.21.15.90:0/3615136731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41d04d0 msgr2=0x55e9f45a9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.404+0000 7f30c1200640 1 --2- 172.21.15.90:0/3615136731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41d04d0 0x55e9f45a9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.404+0000 7f30c1200640 1 -- 172.21.15.90:0/3615136731 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f45aa560 con 0x55e9f41c82f0 2024-05-27T08:03:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.404+0000 7f30c1c00640 1 --2- 172.21.15.90:0/3615136731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 0x55e9f45aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.405+0000 7f30c1200640 1 --2- 172.21.15.90:0/3615136731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f459f640 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f30b00277c0 tx=0x7f30b0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c021de725ea1f885 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.407+0000 7f30ad600640 1 -- 172.21.15.90:0/3615136731 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b001f900 con 0x55e9f41c82f0 2024-05-27T08:03:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.407+0000 7f30ad600640 1 -- 172.21.15.90:0/3615136731 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30b0027d40 con 0x55e9f41c82f0 2024-05-27T08:03:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.407+0000 7f30ad600640 1 -- 172.21.15.90:0/3615136731 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b001f900 con 0x55e9f41c82f0 2024-05-27T08:03:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.409+0000 7f30cab97740 1 -- 172.21.15.90:0/3615136731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 msgr2=0x55e9f459f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:42.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.409+0000 7f30cab97740 1 --2- 172.21.15.90:0/3615136731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f459f640 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f30b00277c0 tx=0x7f30b0000bc0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.412+0000 7f30cab97740 1 -- 172.21.15.90:0/3615136731 shutdown_connections 2024-05-27T08:03:42.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.412+0000 7f30cab97740 1 --2- 172.21.15.90:0/3615136731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41d04d0 0x55e9f45a9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.413+0000 7f30cab97740 1 --2- 172.21.15.90:0/3615136731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41f11b0 0x55e9f45aa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.413+0000 7f30cab97740 1 --2- 172.21.15.90:0/3615136731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f459f640 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.413+0000 7f30cab97740 1 -- 172.21.15.90:0/3615136731 >> 172.21.15.90:0/3615136731 conn(0x55e9f41e5300 msgr2=0x55e9f459dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.413+0000 7f30cab97740 1 -- 172.21.15.90:0/3615136731 shutdown_connections 2024-05-27T08:03:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.413+0000 7f30cab97740 1 -- 172.21.15.90:0/3615136731 wait complete. 2024-05-27T08:03:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.414+0000 7f30cab97740 1 -- start start 2024-05-27T08:03:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.414+0000 7f30cab97740 1 Processor -- start 2024-05-27T08:03:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.415+0000 7f30cab97740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f45a1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.415+0000 7f30cab97740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41d04d0 0x55e9f45a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.415+0000 7f30cab97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 0x55e9f46167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.415+0000 7f30cab97740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f30b415b890 con 0x55e9f41c82f0 2024-05-27T08:03:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.415+0000 7f30cab97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30b415a420 con 0x55e9f41f11b0 2024-05-27T08:03:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.415+0000 7f30cab97740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30b415aaf0 con 0x55e9f41d04d0 2024-05-27T08:03:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.415+0000 7f30c1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f45a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.416+0000 7f30cab97740 10 client.0 _ll_register_callbacks cb 0x7f30b0009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:42.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.415+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41d04d0 0x55e9f45a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.416+0000 7f30bbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41d04d0 0x55e9f45a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:60744/0 (socket says 172.21.15.90:60744) 2024-05-27T08:03:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.416+0000 7f30bbe00640 1 -- 172.21.15.90:0/2447139657 learned_addr learned my addr 172.21.15.90:0/2447139657 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.416+0000 7f30c1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f45a1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54970/0 (socket says 172.21.15.90:54970) 2024-05-27T08:03:42.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.416+0000 7f30c1c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 0x55e9f46167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.417+0000 7f30c1200640 1 -- 172.21.15.90:0/2447139657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41d04d0 msgr2=0x55e9f45a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.418+0000 7f30c1200640 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41d04d0 0x55e9f45a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.418+0000 7f30c1200640 1 -- 172.21.15.90:0/2447139657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 msgr2=0x55e9f46167a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.418+0000 7f30c1200640 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 0x55e9f46167a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.418+0000 7f30c1200640 1 -- 172.21.15.90:0/2447139657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f4616ce0 con 0x55e9f41c82f0 2024-05-27T08:03:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.418+0000 7f30bbe00640 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41d04d0 0x55e9f45a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.418+0000 7f30c1200640 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f45a1290 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f30b00277c0 tx=0x7f30b0027640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c2d599c7a1f2b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:42.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.418+0000 7f30a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:03:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.419+0000 7f30a2a00640 1 -- 172.21.15.90:0/2447139657 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b0043060 con 0x55e9f41c82f0 2024-05-27T08:03:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.419+0000 7f30cab97740 1 -- 172.21.15.90:0/2447139657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30b0027d40 con 0x55e9f41c82f0 2024-05-27T08:03:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.419+0000 7f30a2a00640 1 -- 172.21.15.90:0/2447139657 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f30b0027a00 con 0x55e9f41c82f0 2024-05-27T08:03:42.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.420+0000 7f30a2a00640 1 -- 172.21.15.90:0/2447139657 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f30b0038030 con 0x55e9f41c82f0 2024-05-27T08:03:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.420+0000 7f30cab97740 10 client.18267 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.420+0000 7f30cab97740 1 -- 172.21.15.90:0/2447139657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55e9f4617eb0 con 0x55e9f41c82f0 2024-05-27T08:03:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.421+0000 7f30a2a00640 1 -- 172.21.15.90:0/2447139657 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30b0027d40 con 0x55e9f41c82f0 2024-05-27T08:03:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.421+0000 7f30a2a00640 1 -- 172.21.15.90:0/2447139657 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f30b0027040 con 0x55e9f41c82f0 2024-05-27T08:03:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.421+0000 7f30cab97740 10 client.18267 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.421+0000 7f30cab97740 10 client.18267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.421+0000 7f30cab97740 20 client.18267 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.421+0000 7f30cab97740 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e9f41851c0 0x55e9f461a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.421+0000 7f30cab97740 4 client.18267 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:03:42.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.421+0000 7f30cab97740 1 -- 172.21.15.90:0/2447139657 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e9f45002c0 con 0x55e9f41851c0 2024-05-27T08:03:42.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.423+0000 7f30bbe00640 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e9f41851c0 0x55e9f461a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.426+0000 7f30bbe00640 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e9f41851c0 0x55e9f461a4c0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8034461dceffac13 server_cookie=737e536c87366193 in_seq=0 out_seq=0 2024-05-27T08:03:42.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.426+0000 7f30a2a00640 10 client.18267 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:42.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:42 smithi090 ceph-mon[31429]: pgmap v557: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 469 op/s 2024-05-27T08:03:42.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:42 smithi136 ceph-mon[35324]: pgmap v557: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 469 op/s 2024-05-27T08:03:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.502+0000 7f30a2a00640 1 -- 172.21.15.90:0/2447139657 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e9f45002c0 con 0x55e9f41851c0 2024-05-27T08:03:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.502+0000 7f30a2a00640 10 client.18267 handle_command_reply: tid=0 2024-05-27T08:03:42.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.502+0000 7f30cab97740 1 client.18267 shutdown 2024-05-27T08:03:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.503+0000 7f30cab97740 1 -- 172.21.15.90:0/2447139657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 msgr2=0x55e9f45a1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.503+0000 7f30cab97740 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f45a1290 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f30b00277c0 tx=0x7f30b0027640 comp rx=0 tx=0).stop 2024-05-27T08:03:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.504+0000 7f30cab97740 1 -- 172.21.15.90:0/2447139657 shutdown_connections 2024-05-27T08:03:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.504+0000 7f30cab97740 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55e9f41851c0 0x55e9f461a4c0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.504+0000 7f30cab97740 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9f41f11b0 0x55e9f46167a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.504+0000 7f30cab97740 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e9f41d04d0 0x55e9f45a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.504+0000 7f30a2a00640 0 client.18267 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.504+0000 7f30cab97740 1 --2- 172.21.15.90:0/2447139657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e9f41c82f0 0x55e9f45a1290 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.504+0000 7f30cab97740 1 -- 172.21.15.90:0/2447139657 >> 172.21.15.90:0/2447139657 conn(0x55e9f41e5300 msgr2=0x55e9f4593820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.506+0000 7f30cab97740 1 -- 172.21.15.90:0/2447139657 shutdown_connections 2024-05-27T08:03:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.507+0000 7f30cab97740 1 -- 172.21.15.90:0/2447139657 wait complete. 2024-05-27T08:03:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.507+0000 7f30cab97740 20 client.18267 trim_cache size 0 max 16384 2024-05-27T08:03:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:03:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:03:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:03:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.509+0000 7f30c2600640 1 -- 172.21.15.90:0/56352048 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30a40b2b70 msgr2=0x7f30a40b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.509+0000 7f30c2600640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30a40b2b70 0x7f30a40b5080 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55e9f4594e40 tx=0x7f30b0005f90 comp rx=0 tx=0).stop 2024-05-27T08:03:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.509+0000 7f30c2600640 1 -- 172.21.15.90:0/56352048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30b409ca20 msgr2=0x7f30b40a6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.509+0000 7f30c2600640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30b409ca20 0x7f30b40a6790 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f30a80057c0 tx=0x7f30a80305d0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.510+0000 7f30c2600640 1 -- 172.21.15.90:0/56352048 shutdown_connections 2024-05-27T08:03:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.510+0000 7f30c2600640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f30a40b2b70 0x7f30a40b5080 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.510+0000 7f30c2600640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30b40a1380 0x7f30b40a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.510+0000 7f30c2600640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30b409ca20 0x7f30b40a6790 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.510+0000 7f30c2600640 1 --2- 172.21.15.90:0/56352048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b4096af0 0x7f30b40a6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.510+0000 7f30c2600640 1 -- 172.21.15.90:0/56352048 >> 172.21.15.90:0/56352048 conn(0x7f30b400c120 msgr2=0x7f30b40980b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.511+0000 7f30c2600640 1 -- 172.21.15.90:0/56352048 shutdown_connections 2024-05-27T08:03:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:42.511+0000 7f30c2600640 1 -- 172.21.15.90:0/56352048 wait complete. 2024-05-27T08:03:42.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:42 smithi129 ceph-mon[35254]: pgmap v557: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 469 op/s 2024-05-27T08:03:42.626 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:03:43.426 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:43.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 805.0 s, 580.7 tps, lat 174.256 ms stddev 102.881 2024-05-27T08:03:44.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:44 smithi090 ceph-mon[31429]: pgmap v558: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 462 op/s 2024-05-27T08:03:44.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:44 smithi136 ceph-mon[35324]: pgmap v558: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 462 op/s 2024-05-27T08:03:44.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:44 smithi129 ceph-mon[35254]: pgmap v558: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 462 op/s 2024-05-27T08:03:45.319 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:45 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:03:45.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:45 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:03:45.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:45 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:03:46.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:46 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:46 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:46 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:46 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:46 smithi090 ceph-mon[31429]: pgmap v559: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 570 op/s 2024-05-27T08:03:46.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:46 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:46 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:46 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:46 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.752 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:46 smithi136 ceph-mon[35324]: pgmap v559: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 570 op/s 2024-05-27T08:03:46.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:46 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:46 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:46 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:46 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:46.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:46 smithi129 ceph-mon[35254]: pgmap v559: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 570 op/s 2024-05-27T08:03:47.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:47 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:03:47.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:47 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:03:47.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:47 smithi136 ceph-mon[35324]: pgmap v560: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 489 op/s 2024-05-27T08:03:47.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:47 smithi136 ceph-mon[35324]: pgmap v561: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 588 op/s 2024-05-27T08:03:47.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:47 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:47.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:47 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:03:47.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:47 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:03:47.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:47 smithi129 ceph-mon[35254]: pgmap v560: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 489 op/s 2024-05-27T08:03:47.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:47 smithi129 ceph-mon[35254]: pgmap v561: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 588 op/s 2024-05-27T08:03:47.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:47 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:47.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:47 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:03:47.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:47 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:03:47.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:47 smithi090 ceph-mon[31429]: pgmap v560: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.5 MiB/s wr, 489 op/s 2024-05-27T08:03:47.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:47 smithi090 ceph-mon[31429]: pgmap v561: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.9 MiB/s wr, 588 op/s 2024-05-27T08:03:47.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:47 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:03:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 810.0 s, 423.8 tps, lat 225.843 ms stddev 193.409 2024-05-27T08:03:49.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.423+0000 7f7534400700 1 -- 172.21.15.90:0/4165685673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752c103240 msgr2=0x7f752c103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:49.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.423+0000 7f7534400700 1 --2- 172.21.15.90:0/4165685673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752c103240 0x7f752c103650 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7520009990 tx=0x7f7520009ca0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.424+0000 7f7534400700 1 -- 172.21.15.90:0/4165685673 shutdown_connections 2024-05-27T08:03:49.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.424+0000 7f7534400700 1 --2- 172.21.15.90:0/4165685673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f752c108690 0x7f752c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.424+0000 7f7534400700 1 --2- 172.21.15.90:0/4165685673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752c103240 0x7f752c103650 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.424+0000 7f7534400700 1 --2- 172.21.15.90:0/4165685673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c103b90 0x7f752c1080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.424+0000 7f7534400700 1 -- 172.21.15.90:0/4165685673 >> 172.21.15.90:0/4165685673 conn(0x7f752c0fedb0 msgr2=0x7f752c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.424+0000 7f7534400700 1 -- 172.21.15.90:0/4165685673 shutdown_connections 2024-05-27T08:03:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.424+0000 7f7534400700 1 -- 172.21.15.90:0/4165685673 wait complete. 2024-05-27T08:03:49.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.426+0000 7f7534400700 1 Processor -- start 2024-05-27T08:03:49.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.427+0000 7f7534400700 1 -- start start 2024-05-27T08:03:49.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.428+0000 7f7534400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752c103240 0x7f752c0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.429+0000 7f7534400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f752c103b90 0x7f752c06e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.429+0000 7f7534400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c108690 0x7f752c06eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.430+0000 7f7534400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f752c11a960 con 0x7f752c103240 2024-05-27T08:03:49.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.430+0000 7f7534400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f752c11a7e0 con 0x7f752c108690 2024-05-27T08:03:49.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.431+0000 7f7534400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f752c11aae0 con 0x7f752c103b90 2024-05-27T08:03:49.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.431+0000 7f7531a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752c103240 0x7f752c0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.431+0000 7f7532400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c108690 0x7f752c06eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.431+0000 7f7531000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f752c103b90 0x7f752c06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.431+0000 7f7532400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c108690 0x7f752c06eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:41976/0 (socket says 172.21.15.90:41976) 2024-05-27T08:03:49.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.431+0000 7f7532400700 1 -- 172.21.15.90:0/2336413985 learned_addr learned my addr 172.21.15.90:0/2336413985 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:49.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.433+0000 7f7531a00700 1 -- 172.21.15.90:0/2336413985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f752c103b90 msgr2=0x7f752c06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:49.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.433+0000 7f7531a00700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f752c103b90 0x7f752c06e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.433+0000 7f7531a00700 1 -- 172.21.15.90:0/2336413985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c108690 msgr2=0x7f752c06eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:49.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.433+0000 7f7531a00700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c108690 0x7f752c06eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.433+0000 7f7531a00700 1 -- 172.21.15.90:0/2336413985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c072ed0 con 0x7f752c103240 2024-05-27T08:03:49.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.436+0000 7f7531a00700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752c103240 0x7f752c0753a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f75200098e0 tx=0x7f75200057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:49.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.438+0000 7f752a000700 1 -- 172.21.15.90:0/2336413985 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520026070 con 0x7f752c103240 2024-05-27T08:03:49.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.439+0000 7f752a000700 1 -- 172.21.15.90:0/2336413985 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f752000fe20 con 0x7f752c103240 2024-05-27T08:03:49.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.439+0000 7f752a000700 1 -- 172.21.15.90:0/2336413985 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520021e10 con 0x7f752c103240 2024-05-27T08:03:49.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.439+0000 7f7534400700 1 -- 172.21.15.90:0/2336413985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c073160 con 0x7f752c103240 2024-05-27T08:03:49.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.439+0000 7f7534400700 1 -- 172.21.15.90:0/2336413985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c0735f0 con 0x7f752c103240 2024-05-27T08:03:49.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.448+0000 7f752a000700 1 -- 172.21.15.90:0/2336413985 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7520009d10 con 0x7f752c103240 2024-05-27T08:03:49.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.450+0000 7f752a000700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75140b2b20 0x7f75140b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.451+0000 7f752a000700 1 -- 172.21.15.90:0/2336413985 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f75200e3340 con 0x7f752c103240 2024-05-27T08:03:49.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.452+0000 7f7531000700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75140b2b20 0x7f75140b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.456+0000 7f7531000700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75140b2b20 0x7f75140b5030 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f752c101710 tx=0x7f7518009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:49.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.466+0000 7f7542e48b80 1 Processor -- start 2024-05-27T08:03:49.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.467+0000 7f7542e48b80 1 -- start start 2024-05-27T08:03:49.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.467+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 0x55a8e5932330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.468+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e5932870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.468+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.468+0000 7f7542e48b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a8e5860e40 con 0x55a8e589f9f0 2024-05-27T08:03:49.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.468+0000 7f7542e48b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a8e58509e0 con 0x55a8e5932e70 2024-05-27T08:03:49.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.468+0000 7f7542e48b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a8e5854a20 con 0x55a8e57b3e40 2024-05-27T08:03:49.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.468+0000 7f7532400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.469+0000 7f7532400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:41980/0 (socket says 172.21.15.90:41980) 2024-05-27T08:03:49.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.469+0000 7f7532400700 1 -- 172.21.15.90:0/2042795844 learned_addr learned my addr 172.21.15.90:0/2042795844 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:49.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.469+0000 7f7531a00700 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 0x55a8e5932330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.469+0000 7f7531000700 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e5932870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.469+0000 7f7532400700 1 -- 172.21.15.90:0/2042795844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 msgr2=0x55a8e5932870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:49.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.469+0000 7f7532400700 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e5932870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.469+0000 7f7532400700 1 -- 172.21.15.90:0/2042795844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 msgr2=0x55a8e5932330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:49.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.470+0000 7f7532400700 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 0x55a8e5932330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.470+0000 7f7532400700 1 -- 172.21.15.90:0/2042795844 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e5819420 con 0x55a8e5932e70 2024-05-27T08:03:49.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.470+0000 7f7531000700 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e5932870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:49.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.470+0000 7f7531a00700 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 0x55a8e5932330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:49.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.471+0000 7f7532400700 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593f110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f752400e680 tx=0x7f7524009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a06a201dad1f12c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:49.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.472+0000 7f751ea00700 1 -- 172.21.15.90:0/2042795844 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524019030 con 0x55a8e5932e70 2024-05-27T08:03:49.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.473+0000 7f751ea00700 1 -- 172.21.15.90:0/2042795844 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75240093a0 con 0x55a8e5932e70 2024-05-27T08:03:49.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.473+0000 7f751ea00700 1 -- 172.21.15.90:0/2042795844 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524007500 con 0x55a8e5932e70 2024-05-27T08:03:49.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.475+0000 7f7542e48b80 1 -- 172.21.15.90:0/2042795844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 msgr2=0x55a8e593f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:49.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.475+0000 7f7542e48b80 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593f110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f752400e680 tx=0x7f7524009920 comp rx=0 tx=0).stop 2024-05-27T08:03:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.475+0000 7f7542e48b80 1 -- 172.21.15.90:0/2042795844 shutdown_connections 2024-05-27T08:03:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.475+0000 7f7542e48b80 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593f110 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.475+0000 7f7542e48b80 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 0x55a8e5932330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.475+0000 7f7542e48b80 1 --2- 172.21.15.90:0/2042795844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e5932870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.475+0000 7f7542e48b80 1 -- 172.21.15.90:0/2042795844 >> 172.21.15.90:0/2042795844 conn(0x55a8e584d390 msgr2=0x55a8e592e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:49.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.477+0000 7f7542e48b80 1 -- 172.21.15.90:0/2042795844 shutdown_connections 2024-05-27T08:03:49.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.477+0000 7f7542e48b80 1 -- 172.21.15.90:0/2042795844 wait complete. 2024-05-27T08:03:49.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.478+0000 7f7542e48b80 1 -- start start 2024-05-27T08:03:49.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.478+0000 7f7542e48b80 1 Processor -- start 2024-05-27T08:03:49.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.480+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e593c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.480+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 0x55a8e593cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.481+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.481+0000 7f7542e48b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a8e5756fa0 con 0x55a8e589f9f0 2024-05-27T08:03:49.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.481+0000 7f7542e48b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a8e57f1760 con 0x55a8e5932e70 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.481+0000 7f7542e48b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a8e593fbd0 con 0x55a8e57b3e40 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.482+0000 7f7542e48b80 10 client.0 _ll_register_callbacks cb 0x55a8e57fac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.482+0000 7f7532400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.481+0000 7f7531a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e593c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.482+0000 7f7531a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e593c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59016/0 (socket says 172.21.15.90:59016) 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.482+0000 7f7532400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:41994/0 (socket says 172.21.15.90:41994) 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.482+0000 7f7532400700 1 -- 172.21.15.90:0/2900526507 learned_addr learned my addr 172.21.15.90:0/2900526507 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.483+0000 7f7531000700 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 0x55a8e593cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.484+0000 7f7532400700 1 -- 172.21.15.90:0/2900526507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 msgr2=0x55a8e593c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.484+0000 7f7532400700 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e593c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.484+0000 7f7532400700 1 -- 172.21.15.90:0/2900526507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 msgr2=0x55a8e593cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:49.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.484+0000 7f7532400700 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 0x55a8e593cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.484+0000 7f7532400700 1 -- 172.21.15.90:0/2900526507 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e57a08d0 con 0x55a8e5932e70 2024-05-27T08:03:49.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.484+0000 7f7531a00700 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e593c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:49.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.484+0000 7f7531000700 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 0x55a8e593cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:49.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.485+0000 7f7532400700 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593d0c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7524005b90 tx=0x7f75240077a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d04ae6b109b6a84f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:49.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.485+0000 7f7513400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:03:49.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.491+0000 7f7513400700 1 -- 172.21.15.90:0/2900526507 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524007dd0 con 0x55a8e5932e70 2024-05-27T08:03:49.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.491+0000 7f7513400700 1 -- 172.21.15.90:0/2900526507 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f7524003a00 con 0x55a8e5932e70 2024-05-27T08:03:49.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.491+0000 7f7542e48b80 1 -- 172.21.15.90:0/2900526507 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75240093a0 con 0x55a8e5932e70 2024-05-27T08:03:49.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.492+0000 7f7513400700 1 -- 172.21.15.90:0/2900526507 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7524003c20 con 0x55a8e5932e70 2024-05-27T08:03:49.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.492+0000 7f7542e48b80 10 client.37988 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:49.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.492+0000 7f7542e48b80 1 -- 172.21.15.90:0/2900526507 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a8e5821060 con 0x55a8e5932e70 2024-05-27T08:03:49.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.493+0000 7f7513400700 1 -- 172.21.15.90:0/2900526507 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75240093a0 con 0x55a8e5932e70 2024-05-27T08:03:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.495+0000 7f7513400700 1 -- 172.21.15.90:0/2900526507 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f7524003dc0 con 0x55a8e5932e70 2024-05-27T08:03:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.496+0000 7f7542e48b80 10 client.37988 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.496+0000 7f7542e48b80 10 client.37988 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:03:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.496+0000 7f7542e48b80 20 client.37988 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.496+0000 7f7542e48b80 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a8e594c970 0x55a8e594edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.496+0000 7f7542e48b80 4 client.37988 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:49.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.496+0000 7f7542e48b80 1 -- 172.21.15.90:0/2900526507 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8e5862d90 con 0x55a8e594c970 2024-05-27T08:03:49.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.496+0000 7f7531a00700 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a8e594c970 0x55a8e594edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.497+0000 7f7531a00700 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a8e594c970 0x55a8e594edd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da216544a47d052f server_cookie=3c350ff58e81fe85 in_seq=0 out_seq=0 2024-05-27T08:03:49.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.497+0000 7f7513400700 10 client.37988 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T08:03:49.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.498+0000 7f7513400700 1 -- 172.21.15.90:0/2900526507 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55a8e5862d90 con 0x55a8e594c970 2024-05-27T08:03:49.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.498+0000 7f7513400700 10 client.37988 handle_command_reply: tid=0 2024-05-27T08:03:49.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.499+0000 7f7542e48b80 1 client.37988 shutdown 2024-05-27T08:03:49.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.499+0000 7f7542e48b80 1 -- 172.21.15.90:0/2900526507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 msgr2=0x55a8e593d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:49.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.499+0000 7f7542e48b80 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593d0c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7524005b90 tx=0x7f75240077a0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.499+0000 7f7542e48b80 1 -- 172.21.15.90:0/2900526507 shutdown_connections 2024-05-27T08:03:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.499+0000 7f7542e48b80 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a8e594c970 0x55a8e594edd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.499+0000 7f7542e48b80 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e5932e70 0x55a8e593d0c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.499+0000 7f7542e48b80 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e57b3e40 0x55a8e593c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.499+0000 7f7542e48b80 1 --2- 172.21.15.90:0/2900526507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e589f9f0 0x55a8e593cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.499+0000 7f7542e48b80 1 -- 172.21.15.90:0/2900526507 >> 172.21.15.90:0/2900526507 conn(0x55a8e584d390 msgr2=0x55a8e5932310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.500+0000 7f7513400700 0 client.37988 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T08:03:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.500+0000 7f7542e48b80 1 -- 172.21.15.90:0/2900526507 shutdown_connections 2024-05-27T08:03:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.500+0000 7f7542e48b80 1 -- 172.21.15.90:0/2900526507 wait complete. 2024-05-27T08:03:49.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.500+0000 7f7542e48b80 20 client.37988 trim_cache size 0 max 16384 2024-05-27T08:03:49.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.540+0000 7f7542e48b80 1 Processor -- start 2024-05-27T08:03:49.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.541+0000 7f7542e48b80 1 -- start start 2024-05-27T08:03:49.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.542+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e5932830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.542+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 0x55a8e5937e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.542+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e5932e70 0x55a8e593b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.542+0000 7f7542e48b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1c0cb0 con 0x55a8e5932e70 2024-05-27T08:03:49.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.542+0000 7f7542e48b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1bdd00 con 0x55a8e57b3e40 2024-05-27T08:03:49.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.542+0000 7f7542e48b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1bff10 con 0x55a8e589f9f0 2024-05-27T08:03:49.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.542+0000 7f7531000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 0x55a8e5937e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.542+0000 7f7531000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 0x55a8e5937e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59032/0 (socket says 172.21.15.90:59032) 2024-05-27T08:03:49.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.542+0000 7f7531000700 1 -- 172.21.15.90:0/1122896592 learned_addr learned my addr 172.21.15.90:0/1122896592 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:49.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.543+0000 7f7531a00700 1 --2- 172.21.15.90:0/1122896592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e5932830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.544+0000 7f7531a00700 1 -- 172.21.15.90:0/1122896592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 msgr2=0x55a8e5937e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:49.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.544+0000 7f7531a00700 1 --2- 172.21.15.90:0/1122896592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 0x55a8e5937e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.544+0000 7f7531a00700 1 -- 172.21.15.90:0/1122896592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e5932e70 msgr2=0x55a8e593b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:49.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.544+0000 7f7531a00700 1 --2- 172.21.15.90:0/1122896592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e5932e70 0x55a8e593b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.544+0000 7f7531a00700 1 -- 172.21.15.90:0/1122896592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e592a240 con 0x55a8e57b3e40 2024-05-27T08:03:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.547+0000 7f7531a00700 1 --2- 172.21.15.90:0/1122896592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e5932830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f75200e0660 tx=0x7f7520009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81d7225a9ec68e63 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:49.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.548+0000 7f751e000700 1 -- 172.21.15.90:0/1122896592 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75200e3110 con 0x55a8e57b3e40 2024-05-27T08:03:49.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.548+0000 7f751e000700 1 -- 172.21.15.90:0/1122896592 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a8e5862d90 con 0x55a8e57b3e40 2024-05-27T08:03:49.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.548+0000 7f751e000700 1 -- 172.21.15.90:0/1122896592 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75200b2bc0 con 0x55a8e57b3e40 2024-05-27T08:03:49.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.549+0000 7f7542e48b80 1 -- 172.21.15.90:0/1122896592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 msgr2=0x55a8e5932830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:49.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.549+0000 7f7542e48b80 1 --2- 172.21.15.90:0/1122896592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e5932830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f75200e0660 tx=0x7f7520009ca0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.550+0000 7f7542e48b80 1 -- 172.21.15.90:0/1122896592 shutdown_connections 2024-05-27T08:03:49.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.550+0000 7f7542e48b80 1 --2- 172.21.15.90:0/1122896592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e5932e70 0x55a8e593b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.550+0000 7f7542e48b80 1 --2- 172.21.15.90:0/1122896592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e5932830 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.550+0000 7f7542e48b80 1 --2- 172.21.15.90:0/1122896592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 0x55a8e5937e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.550+0000 7f7542e48b80 1 -- 172.21.15.90:0/1122896592 >> 172.21.15.90:0/1122896592 conn(0x55a8e584d390 msgr2=0x55a8e5932120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.551+0000 7f7542e48b80 1 -- 172.21.15.90:0/1122896592 shutdown_connections 2024-05-27T08:03:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.551+0000 7f7542e48b80 1 -- 172.21.15.90:0/1122896592 wait complete. 2024-05-27T08:03:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.551+0000 7f7542e48b80 1 -- start start 2024-05-27T08:03:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.552+0000 7f7542e48b80 1 Processor -- start 2024-05-27T08:03:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.554+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e593fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.555+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 0x55a8e59403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.555+0000 7f7542e48b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e5932e70 0x55a8e59408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.555+0000 7f7531a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e593fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.555+0000 7f7531a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e593fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:42008/0 (socket says 172.21.15.90:42008) 2024-05-27T08:03:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.555+0000 7f7531a00700 1 -- 172.21.15.90:0/1633458517 learned_addr learned my addr 172.21.15.90:0/1633458517 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.555+0000 7f7532400700 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e5932e70 0x55a8e59408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.556+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1bf840 con 0x55a8e5932e70 2024-05-27T08:03:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.556+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1be3d0 con 0x55a8e57b3e40 2024-05-27T08:03:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.556+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1beaa0 con 0x55a8e589f9f0 2024-05-27T08:03:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.556+0000 7f7542e48b80 10 client.0 _ll_register_callbacks cb 0x55a8e57fac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.557+0000 7f7531000700 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 0x55a8e59403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.557+0000 7f7531a00700 1 -- 172.21.15.90:0/1633458517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 msgr2=0x55a8e59403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:49.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.557+0000 7f7531a00700 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 0x55a8e59403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.558+0000 7f7531a00700 1 -- 172.21.15.90:0/1633458517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e5932e70 msgr2=0x55a8e59408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:49.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.558+0000 7f7531a00700 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e5932e70 0x55a8e59408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.558+0000 7f7531a00700 1 -- 172.21.15.90:0/1633458517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e59b5640 con 0x55a8e57b3e40 2024-05-27T08:03:49.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.558+0000 7f7532400700 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e5932e70 0x55a8e59408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:49.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.559+0000 7f7513400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:03:49.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.559+0000 7f7531a00700 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e593fe70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f75200a6300 tx=0x7f75200e0b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15f8e8f6b92a1497 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:49.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.559+0000 7f7513400700 1 -- 172.21.15.90:0/1633458517 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75200ae630 con 0x55a8e57b3e40 2024-05-27T08:03:49.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.560+0000 7f7513400700 1 -- 172.21.15.90:0/1633458517 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f75200a6870 con 0x55a8e57b3e40 2024-05-27T08:03:49.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.560+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8e5862d90 con 0x55a8e57b3e40 2024-05-27T08:03:49.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.561+0000 7f7513400700 1 -- 172.21.15.90:0/1633458517 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f75200b3050 con 0x55a8e57b3e40 2024-05-27T08:03:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.561+0000 7f7542e48b80 10 client.37994 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.561+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55a8e59b68a0 con 0x55a8e57b3e40 2024-05-27T08:03:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.562+0000 7f7513400700 1 -- 172.21.15.90:0/1633458517 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a8e5862d90 con 0x55a8e57b3e40 2024-05-27T08:03:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.562+0000 7f7513400700 1 -- 172.21.15.90:0/1633458517 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f7520029030 con 0x55a8e57b3e40 2024-05-27T08:03:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.562+0000 7f7542e48b80 10 client.37994 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.562+0000 7f7542e48b80 10 client.37994 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:03:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.562+0000 7f7542e48b80 20 client.37994 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.562+0000 7f7542e48b80 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a8e59b6d80 0x55a8e59b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.562+0000 7f7542e48b80 4 client.37994 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:03:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.562+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8e585c570 con 0x55a8e59b6d80 2024-05-27T08:03:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.565+0000 7f7531000700 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a8e59b6d80 0x55a8e59b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:49.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.567+0000 7f7513400700 10 client.37994 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T08:03:49.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.567+0000 7f7531000700 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a8e59b6d80 0x55a8e59b91a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bd270359a422aee server_cookie=a99d63c4118eddd4 in_seq=0 out_seq=0 2024-05-27T08:03:49.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7513400700 1 -- 172.21.15.90:0/1633458517 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8e585c570 con 0x55a8e59b6d80 2024-05-27T08:03:49.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7513400700 10 client.37994 handle_command_reply: tid=0 2024-05-27T08:03:49.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7542e48b80 1 client.37994 shutdown 2024-05-27T08:03:49.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 msgr2=0x55a8e593fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:49.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7542e48b80 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e593fe70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f75200a6300 tx=0x7f75200e0b90 comp rx=0 tx=0).stop 2024-05-27T08:03:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 shutdown_connections 2024-05-27T08:03:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7542e48b80 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55a8e59b6d80 0x55a8e59b91a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7542e48b80 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8e5932e70 0x55a8e59408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7542e48b80 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8e57b3e40 0x55a8e593fe70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7542e48b80 1 --2- 172.21.15.90:0/1633458517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a8e589f9f0 0x55a8e59403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7513400700 0 client.37994 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T08:03:49.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.568+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 >> 172.21.15.90:0/1633458517 conn(0x55a8e584d390 msgr2=0x55a8e593c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:49.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.569+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 shutdown_connections 2024-05-27T08:03:49.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.569+0000 7f7542e48b80 1 -- 172.21.15.90:0/1633458517 wait complete. 2024-05-27T08:03:49.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.569+0000 7f7542e48b80 20 client.37994 trim_cache size 0 max 16384 2024-05-27T08:03:49.571 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:03:49.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.570+0000 7f7534400700 1 -- 172.21.15.90:0/2336413985 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75140b2b20 msgr2=0x7f75140b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:49.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.570+0000 7f7534400700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75140b2b20 0x7f75140b5030 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f752c101710 tx=0x7f7518009260 comp rx=0 tx=0).stop 2024-05-27T08:03:49.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.570+0000 7f7534400700 1 -- 172.21.15.90:0/2336413985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752c103240 msgr2=0x7f752c0753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:49.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.570+0000 7f7534400700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752c103240 0x7f752c0753a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f75200098e0 tx=0x7f75200057c0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.571+0000 7f7534400700 1 -- 172.21.15.90:0/2336413985 shutdown_connections 2024-05-27T08:03:49.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.571+0000 7f7534400700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75140b2b20 0x7f75140b5030 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.571+0000 7f7534400700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c108690 0x7f752c06eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.571+0000 7f7534400700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f752c103240 0x7f752c0753a0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.571+0000 7f7534400700 1 --2- 172.21.15.90:0/2336413985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f752c103b90 0x7f752c06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:49.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.571+0000 7f7534400700 1 -- 172.21.15.90:0/2336413985 >> 172.21.15.90:0/2336413985 conn(0x7f752c0fedb0 msgr2=0x7f752c106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:49.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.572+0000 7f7534400700 1 -- 172.21.15.90:0/2336413985 shutdown_connections 2024-05-27T08:03:49.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:49.572+0000 7f7534400700 1 -- 172.21.15.90:0/2336413985 wait complete. 2024-05-27T08:03:50.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:49 smithi090 ceph-mon[31429]: pgmap v562: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 466 op/s 2024-05-27T08:03:50.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:49 smithi136 ceph-mon[35324]: pgmap v562: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 466 op/s 2024-05-27T08:03:50.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:49 smithi129 ceph-mon[35254]: pgmap v562: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.4 MiB/s wr, 466 op/s 2024-05-27T08:03:50.625 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:03:51.347 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:03:52.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:51 smithi090 ceph-mon[31429]: pgmap v563: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 518 op/s 2024-05-27T08:03:52.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:51 smithi136 ceph-mon[35324]: pgmap v563: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 518 op/s 2024-05-27T08:03:52.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:51 smithi129 ceph-mon[35254]: pgmap v563: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 518 op/s 2024-05-27T08:03:53.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 815.0 s, 535.8 tps, lat 192.240 ms stddev 123.123 2024-05-27T08:03:54.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:53 smithi090 ceph-mon[31429]: pgmap v564: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 485 op/s 2024-05-27T08:03:54.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:53 smithi136 ceph-mon[35324]: pgmap v564: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 485 op/s 2024-05-27T08:03:54.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:53 smithi129 ceph-mon[35254]: pgmap v564: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 485 op/s 2024-05-27T08:03:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.814+0000 7f7776400640 1 -- 172.21.15.90:0/1404067165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f777010a760 msgr2=0x7f777010ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.814+0000 7f7776400640 1 --2- 172.21.15.90:0/1404067165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f777010a760 0x7f777010ab70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7760009900 tx=0x7f776002a3f0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.814+0000 7f7776400640 1 -- 172.21.15.90:0/1404067165 shutdown_connections 2024-05-27T08:03:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.814+0000 7f7776400640 1 --2- 172.21.15.90:0/1404067165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7770116120 0x7f7770118540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.814+0000 7f7776400640 1 --2- 172.21.15.90:0/1404067165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7770118bd0 0x7f777011b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.814+0000 7f7776400640 1 --2- 172.21.15.90:0/1404067165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f777010a760 0x7f777010ab70 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.814+0000 7f7776400640 1 -- 172.21.15.90:0/1404067165 >> 172.21.15.90:0/1404067165 conn(0x7f777006eda0 msgr2=0x7f777006f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.814+0000 7f7776400640 1 -- 172.21.15.90:0/1404067165 shutdown_connections 2024-05-27T08:03:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.814+0000 7f7776400640 1 -- 172.21.15.90:0/1404067165 wait complete. 2024-05-27T08:03:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.815+0000 7f7776400640 1 Processor -- start 2024-05-27T08:03:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.815+0000 7f7776400640 1 -- start start 2024-05-27T08:03:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.816+0000 7f7776400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f777010a760 0x7f77701a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.816+0000 7f7776400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7770116120 0x7f77701a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.816+0000 7f7776400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7770118bd0 0x7f77701a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.816+0000 7f7776400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f777011e6d0 con 0x7f7770116120 2024-05-27T08:03:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.816+0000 7f7776400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f777011e550 con 0x7f7770118bd0 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.816+0000 7f7776400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f777011e850 con 0x7f777010a760 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.816+0000 7f7767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7770116120 0x7f77701a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.816+0000 7f7767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7770116120 0x7f77701a5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44220/0 (socket says 172.21.15.90:44220) 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.816+0000 7f7767e00640 1 -- 172.21.15.90:0/2848585340 learned_addr learned my addr 172.21.15.90:0/2848585340 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.816+0000 7f776f400640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f777010a760 0x7f77701a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f776fe00640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7770118bd0 0x7f77701a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f7767e00640 1 -- 172.21.15.90:0/2848585340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f777010a760 msgr2=0x7f77701a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f7767e00640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f777010a760 0x7f77701a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f7767e00640 1 -- 172.21.15.90:0/2848585340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7770118bd0 msgr2=0x7f77701a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f7767e00640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7770118bd0 0x7f77701a99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f7767e00640 1 -- 172.21.15.90:0/2848585340 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77701a9fb0 con 0x7f7770116120 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f776f400640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f777010a760 0x7f77701a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:03:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f7767e00640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7770116120 0x7f77701a5730 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f775800c8f0 tx=0x7f775800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f776cc00640 1 -- 172.21.15.90:0/2848585340 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758007bb0 con 0x7f7770116120 2024-05-27T08:03:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f776cc00640 1 -- 172.21.15.90:0/2848585340 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7758007d50 con 0x7f7770116120 2024-05-27T08:03:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f776cc00640 1 -- 172.21.15.90:0/2848585340 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580186f0 con 0x7f7770116120 2024-05-27T08:03:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f7776400640 1 -- 172.21.15.90:0/2848585340 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77701aa2a0 con 0x7f7770116120 2024-05-27T08:03:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.817+0000 7f7776400640 1 -- 172.21.15.90:0/2848585340 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77701aa7f0 con 0x7f7770116120 2024-05-27T08:03:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.821+0000 7f777d38e740 1 Processor -- start 2024-05-27T08:03:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.821+0000 7f777d38e740 1 -- start start 2024-05-27T08:03:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.821+0000 7f777d38e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56512990e1b0 0x565129cb0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.821+0000 7f777d38e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298e52f0 0x565129cb0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.821+0000 7f777d38e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cb12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.821+0000 7f777d38e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x565129c21b90 con 0x5651298ed4d0 2024-05-27T08:03:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.821+0000 7f777d38e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565129c11730 con 0x5651298e52f0 2024-05-27T08:03:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.821+0000 7f777d38e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x565129c15770 con 0x56512990e1b0 2024-05-27T08:03:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.821+0000 7f776cc00640 1 -- 172.21.15.90:0/2848585340 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f775800f450 con 0x7f7770116120 2024-05-27T08:03:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f7767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cb12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f7767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cb12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44234/0 (socket says 172.21.15.90:44234) 2024-05-27T08:03:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f7767e00640 1 -- 172.21.15.90:0/733371283 learned_addr learned my addr 172.21.15.90:0/733371283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f776fe00640 1 --2- 172.21.15.90:0/733371283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56512990e1b0 0x565129cb0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f776f400640 1 --2- 172.21.15.90:0/733371283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298e52f0 0x565129cb0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f7767e00640 1 -- 172.21.15.90:0/733371283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56512990e1b0 msgr2=0x565129cb0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f7767e00640 1 --2- 172.21.15.90:0/733371283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56512990e1b0 0x565129cb0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f7767e00640 1 -- 172.21.15.90:0/733371283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298e52f0 msgr2=0x565129cb0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f7767e00640 1 --2- 172.21.15.90:0/733371283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298e52f0 0x565129cb0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f7767e00640 1 -- 172.21.15.90:0/733371283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565129baa8b0 con 0x5651298ed4d0 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.822+0000 7f7767e00640 1 --2- 172.21.15.90:0/733371283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cb12e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f77580dfe20 tx=0x7f77580e6df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c855975cb5357438 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.823+0000 7f7764c00640 1 -- 172.21.15.90:0/733371283 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580f2020 con 0x5651298ed4d0 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.823+0000 7f7764c00640 1 -- 172.21.15.90:0/733371283 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77580ebd60 con 0x5651298ed4d0 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.823+0000 7f7764c00640 1 -- 172.21.15.90:0/733371283 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580ed5f0 con 0x5651298ed4d0 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.823+0000 7f776cc00640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f77540b2950 0x7f77540b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.823+0000 7f776cc00640 1 -- 172.21.15.90:0/2848585340 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f77580e2bc0 con 0x7f7770116120 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.823+0000 7f776fe00640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f77540b2950 0x7f77540b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.823+0000 7f777d38e740 1 -- 172.21.15.90:0/733371283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 msgr2=0x565129cb12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.823+0000 7f777d38e740 1 --2- 172.21.15.90:0/733371283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cb12e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f77580dfe20 tx=0x7f77580e6df0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.824+0000 7f777d38e740 1 -- 172.21.15.90:0/733371283 shutdown_connections 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.824+0000 7f777d38e740 1 --2- 172.21.15.90:0/733371283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298e52f0 0x565129cb0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.824+0000 7f777d38e740 1 --2- 172.21.15.90:0/733371283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cb12e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.824+0000 7f777d38e740 1 --2- 172.21.15.90:0/733371283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56512990e1b0 0x565129cb0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.824+0000 7f777d38e740 1 -- 172.21.15.90:0/733371283 >> 172.21.15.90:0/733371283 conn(0x565129902300 msgr2=0x565129cad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.824+0000 7f777d38e740 1 -- 172.21.15.90:0/733371283 shutdown_connections 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.824+0000 7f777d38e740 1 -- 172.21.15.90:0/733371283 wait complete. 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.824+0000 7f776fe00640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f77540b2950 0x7f77540b4e60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x565129cad9d0 tx=0x7f7768007600 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.824+0000 7f777d38e740 1 -- start start 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f777d38e740 1 Processor -- start 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f777d38e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298e52f0 0x565129cbaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f777d38e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cbafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f777d38e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56512990e1b0 0x565129cc6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f777d38e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5651298f7790 con 0x5651298ed4d0 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f777d38e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565129bb9bc0 con 0x56512990e1b0 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f777d38e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x565129cbd990 con 0x5651298e52f0 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f777d38e740 10 client.0 _ll_register_callbacks cb 0x565129b97400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f7767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cbafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f7767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cbafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44244/0 (socket says 172.21.15.90:44244) 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.825+0000 7f7767e00640 1 -- 172.21.15.90:0/2819611528 learned_addr learned my addr 172.21.15.90:0/2819611528 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f776f400640 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298e52f0 0x565129cbaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f7767e00640 1 -- 172.21.15.90:0/2819611528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298e52f0 msgr2=0x565129cbaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f7767e00640 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298e52f0 0x565129cbaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f7767e00640 1 -- 172.21.15.90:0/2819611528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56512990e1b0 msgr2=0x565129cc6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f7767e00640 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56512990e1b0 0x565129cc6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f7767e00640 1 -- 172.21.15.90:0/2819611528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565129926210 con 0x5651298ed4d0 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f7767e00640 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cbafc0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f77580a9770 tx=0x7f775800f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a761ab655cfdb85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f775d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f775d600640 1 -- 172.21.15.90:0/2819611528 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580f2070 con 0x5651298ed4d0 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f775d600640 1 -- 172.21.15.90:0/2819611528 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f77580ebb60 con 0x5651298ed4d0 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f777d38e740 1 -- 172.21.15.90:0/2819611528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77580ebd60 con 0x5651298ed4d0 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.826+0000 7f775d600640 1 -- 172.21.15.90:0/2819611528 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f77580a5930 con 0x5651298ed4d0 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.827+0000 7f777d38e740 10 client.18291 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.827+0000 7f777d38e740 1 -- 172.21.15.90:0/2819611528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5651299f2620 con 0x5651298ed4d0 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.827+0000 7f775d600640 1 -- 172.21.15.90:0/2819611528 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77580ebd60 con 0x5651298ed4d0 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.827+0000 7f775d600640 1 -- 172.21.15.90:0/2819611528 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f77580e0160 con 0x5651298ed4d0 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.827+0000 7f777d38e740 10 client.18291 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.827+0000 7f777d38e740 10 client.18291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.827+0000 7f777d38e740 20 client.18291 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.827+0000 7f777d38e740 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5651298ccd80 0x565129cccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.827+0000 7f777d38e740 4 client.18291 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.827+0000 7f777d38e740 1 -- 172.21.15.90:0/2819611528 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565129c23ae0 con 0x5651298ccd80 2024-05-27T08:03:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.828+0000 7f776f400640 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5651298ccd80 0x565129cccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.829+0000 7f776f400640 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5651298ccd80 0x565129cccb10 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a66dcfac4eef0d server_cookie=61c0c638e729e258 in_seq=0 out_seq=0 2024-05-27T08:03:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.829+0000 7f775d600640 10 client.18291 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f775d600640 1 -- 172.21.15.90:0/2819611528 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x565129c23ae0 con 0x5651298ccd80 2024-05-27T08:03:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f775d600640 10 client.18291 handle_command_reply: tid=0 2024-05-27T08:03:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 client.18291 shutdown 2024-05-27T08:03:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 -- 172.21.15.90:0/2819611528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 msgr2=0x565129cbafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cbafc0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f77580a9770 tx=0x7f775800f970 comp rx=0 tx=0).stop 2024-05-27T08:03:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 -- 172.21.15.90:0/2819611528 shutdown_connections 2024-05-27T08:03:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5651298ccd80 0x565129cccb10 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56512990e1b0 0x565129cc6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f775d600640 0 client.18291 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298ed4d0 0x565129cbafc0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 --2- 172.21.15.90:0/2819611528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298e52f0 0x565129cbaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 -- 172.21.15.90:0/2819611528 >> 172.21.15.90:0/2819611528 conn(0x565129902300 msgr2=0x565129cb2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 -- 172.21.15.90:0/2819611528 shutdown_connections 2024-05-27T08:03:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 1 -- 172.21.15.90:0/2819611528 wait complete. 2024-05-27T08:03:55.839 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.831+0000 7f777d38e740 20 client.18291 trim_cache size 0 max 16384 2024-05-27T08:03:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.847+0000 7f777d38e740 1 Processor -- start 2024-05-27T08:03:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.847+0000 7f777d38e740 1 -- start start 2024-05-27T08:03:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.847+0000 7f777d38e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298e52f0 0x565129cba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.847+0000 7f777d38e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298ed4d0 0x565129cc6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.847+0000 7f777d38e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56512990e1b0 0x565129ccfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.847+0000 7f777d38e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f77701bd580 con 0x5651298e52f0 2024-05-27T08:03:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.847+0000 7f777d38e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77701ba5d0 con 0x56512990e1b0 2024-05-27T08:03:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.847+0000 7f777d38e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77701bc7e0 con 0x5651298ed4d0 2024-05-27T08:03:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.847+0000 7f776f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298e52f0 0x565129cba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.848+0000 7f776f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298e52f0 0x565129cba540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44254/0 (socket says 172.21.15.90:44254) 2024-05-27T08:03:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.848+0000 7f776f400640 1 -- 172.21.15.90:0/3747767255 learned_addr learned my addr 172.21.15.90:0/3747767255 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.848+0000 7f7767e00640 1 --2- 172.21.15.90:0/3747767255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298ed4d0 0x565129cc6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.848+0000 7f776fe00640 1 --2- 172.21.15.90:0/3747767255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56512990e1b0 0x565129ccfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.848+0000 7f776f400640 1 -- 172.21.15.90:0/3747767255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298ed4d0 msgr2=0x565129cc6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.848+0000 7f776f400640 1 --2- 172.21.15.90:0/3747767255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298ed4d0 0x565129cc6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.848+0000 7f776f400640 1 -- 172.21.15.90:0/3747767255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56512990e1b0 msgr2=0x565129ccfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.848+0000 7f776f400640 1 --2- 172.21.15.90:0/3747767255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56512990e1b0 0x565129ccfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.848+0000 7f776f400640 1 -- 172.21.15.90:0/3747767255 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565129cadb70 con 0x5651298e52f0 2024-05-27T08:03:55.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.849+0000 7f776f400640 1 --2- 172.21.15.90:0/3747767255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298e52f0 0x565129cba540 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7760032fd0 tx=0x7f7760009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d928340dd845a549 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.849+0000 7f775fe00640 1 -- 172.21.15.90:0/3747767255 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760048020 con 0x5651298e52f0 2024-05-27T08:03:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.849+0000 7f775fe00640 1 -- 172.21.15.90:0/3747767255 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77600044d0 con 0x5651298e52f0 2024-05-27T08:03:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.849+0000 7f775fe00640 1 -- 172.21.15.90:0/3747767255 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760035040 con 0x5651298e52f0 2024-05-27T08:03:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.850+0000 7f777d38e740 1 -- 172.21.15.90:0/3747767255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298e52f0 msgr2=0x565129cba540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.850+0000 7f777d38e740 1 --2- 172.21.15.90:0/3747767255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298e52f0 0x565129cba540 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7760032fd0 tx=0x7f7760009640 comp rx=0 tx=0).stop 2024-05-27T08:03:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.850+0000 7f777d38e740 1 -- 172.21.15.90:0/3747767255 shutdown_connections 2024-05-27T08:03:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.850+0000 7f777d38e740 1 --2- 172.21.15.90:0/3747767255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56512990e1b0 0x565129ccfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.850+0000 7f777d38e740 1 --2- 172.21.15.90:0/3747767255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298ed4d0 0x565129cc6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.850+0000 7f777d38e740 1 --2- 172.21.15.90:0/3747767255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651298e52f0 0x565129cba540 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.850+0000 7f777d38e740 1 -- 172.21.15.90:0/3747767255 >> 172.21.15.90:0/3747767255 conn(0x565129902300 msgr2=0x565129cb1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.850+0000 7f777d38e740 1 -- 172.21.15.90:0/3747767255 shutdown_connections 2024-05-27T08:03:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.850+0000 7f777d38e740 1 -- 172.21.15.90:0/3747767255 wait complete. 2024-05-27T08:03:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.850+0000 7f777d38e740 1 -- start start 2024-05-27T08:03:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f777d38e740 1 Processor -- start 2024-05-27T08:03:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f777d38e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298e52f0 0x565129ccd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f777d38e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298ed4d0 0x565129ccd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f777d38e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56512990e1b0 0x565129d2cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f777d38e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f77701bc110 con 0x56512990e1b0 2024-05-27T08:03:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f777d38e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77701baca0 con 0x5651298ed4d0 2024-05-27T08:03:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f777d38e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77701bb370 con 0x5651298e52f0 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f777d38e740 10 client.0 _ll_register_callbacks cb 0x7f77680074a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f776fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56512990e1b0 0x565129d2cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f776fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56512990e1b0 0x565129d2cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44266/0 (socket says 172.21.15.90:44266) 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.851+0000 7f776fe00640 1 -- 172.21.15.90:0/1391130767 learned_addr learned my addr 172.21.15.90:0/1391130767 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f776f400640 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298e52f0 0x565129ccd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f7767e00640 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298ed4d0 0x565129ccd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f776fe00640 1 -- 172.21.15.90:0/1391130767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298e52f0 msgr2=0x565129ccd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f776fe00640 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298e52f0 0x565129ccd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f776fe00640 1 -- 172.21.15.90:0/1391130767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298ed4d0 msgr2=0x565129ccd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f776fe00640 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298ed4d0 0x565129ccd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f776fe00640 1 -- 172.21.15.90:0/1391130767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565129d2d460 con 0x56512990e1b0 2024-05-27T08:03:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f7767e00640 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298ed4d0 0x565129ccd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f776fe00640 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56512990e1b0 0x565129d2cf20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f776801ee20 tx=0x7f7768004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a35e697a8af77f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f775d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f775d600640 1 -- 172.21.15.90:0/1391130767 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776800ecc0 con 0x56512990e1b0 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f777d38e740 1 -- 172.21.15.90:0/1391130767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77600044d0 con 0x56512990e1b0 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.852+0000 7f775d600640 1 -- 172.21.15.90:0/1391130767 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7f7768003970 con 0x56512990e1b0 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f775d600640 1 -- 172.21.15.90:0/1391130767 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f776800ee60 con 0x56512990e1b0 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f777d38e740 10 client.18297 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f777d38e740 1 -- 172.21.15.90:0/1391130767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x565129d347f0 con 0x56512990e1b0 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f775d600640 1 -- 172.21.15.90:0/1391130767 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f77600044d0 con 0x56512990e1b0 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f775d600640 1 -- 172.21.15.90:0/1391130767 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f7768028400 con 0x56512990e1b0 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f777d38e740 10 client.18297 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f777d38e740 10 client.18297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:03:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f777d38e740 20 client.18297 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f777d38e740 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5651298a21c0 0x565129d36fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f777d38e740 4 client.18297 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:03:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.853+0000 7f777d38e740 1 -- 172.21.15.90:0/1391130767 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565129c1d2c0 con 0x5651298a21c0 2024-05-27T08:03:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.854+0000 7f776f400640 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5651298a21c0 0x565129d36fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.855+0000 7f775d600640 10 client.18297 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:55.855+0000 7f776f400640 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5651298a21c0 0x565129d36fb0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93bbabde3d34ece6 server_cookie=bfa186460c69b5a3 in_seq=0 out_seq=0 2024-05-27T08:03:56.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:55 smithi090 ceph-mon[31429]: pgmap v565: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 466 op/s 2024-05-27T08:03:56.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:55 smithi136 ceph-mon[35324]: pgmap v565: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 466 op/s 2024-05-27T08:03:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.296+0000 7f775d600640 1 -- 172.21.15.90:0/1391130767 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565129c1d2c0 con 0x5651298a21c0 2024-05-27T08:03:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.296+0000 7f775d600640 10 client.18297 handle_command_reply: tid=0 2024-05-27T08:03:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.296+0000 7f777d38e740 1 client.18297 shutdown 2024-05-27T08:03:56.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.299+0000 7f777d38e740 1 -- 172.21.15.90:0/1391130767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56512990e1b0 msgr2=0x565129d2cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.299+0000 7f777d38e740 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56512990e1b0 0x565129d2cf20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f776801ee20 tx=0x7f7768004ab0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.299+0000 7f777d38e740 1 -- 172.21.15.90:0/1391130767 shutdown_connections 2024-05-27T08:03:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.299+0000 7f777d38e740 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5651298a21c0 0x565129d36fb0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.299+0000 7f777d38e740 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5651298ed4d0 0x565129ccd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.299+0000 7f777d38e740 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56512990e1b0 0x565129d2cf20 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.299+0000 7f777d38e740 1 --2- 172.21.15.90:0/1391130767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5651298e52f0 0x565129ccd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.299+0000 7f777d38e740 1 -- 172.21.15.90:0/1391130767 >> 172.21.15.90:0/1391130767 conn(0x565129902300 msgr2=0x565129cb0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.300+0000 7f775d600640 0 client.18297 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:03:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.300+0000 7f777d38e740 1 -- 172.21.15.90:0/1391130767 shutdown_connections 2024-05-27T08:03:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.300+0000 7f777d38e740 1 -- 172.21.15.90:0/1391130767 wait complete. 2024-05-27T08:03:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.300+0000 7f777d38e740 20 client.18297 trim_cache size 0 max 16384 2024-05-27T08:03:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:03:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:03:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:03:56.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.303+0000 7f7776400640 1 -- 172.21.15.90:0/2848585340 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f77540b2950 msgr2=0x7f77540b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.303+0000 7f7776400640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f77540b2950 0x7f77540b4e60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x565129cad9d0 tx=0x7f7768007600 comp rx=0 tx=0).stop 2024-05-27T08:03:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.303+0000 7f7776400640 1 -- 172.21.15.90:0/2848585340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7770116120 msgr2=0x7f77701a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.303+0000 7f7776400640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7770116120 0x7f77701a5730 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f775800c8f0 tx=0x7f775800cdb0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.304+0000 7f7776400640 1 -- 172.21.15.90:0/2848585340 shutdown_connections 2024-05-27T08:03:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.304+0000 7f7776400640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f77540b2950 0x7f77540b4e60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.304+0000 7f7776400640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7770118bd0 0x7f77701a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.304+0000 7f7776400640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7770116120 0x7f77701a5730 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.304+0000 7f7776400640 1 --2- 172.21.15.90:0/2848585340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f777010a760 0x7f77701a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.304+0000 7f7776400640 1 -- 172.21.15.90:0/2848585340 >> 172.21.15.90:0/2848585340 conn(0x7f777006eda0 msgr2=0x7f7770116530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.305+0000 7f7776400640 1 -- 172.21.15.90:0/2848585340 shutdown_connections 2024-05-27T08:03:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:03:56.305+0000 7f7776400640 1 -- 172.21.15.90:0/2848585340 wait complete. 2024-05-27T08:03:56.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:55 smithi129 ceph-mon[35254]: pgmap v565: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.5 MiB/s wr, 466 op/s 2024-05-27T08:03:56.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.945+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2999057549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8c8075d20 msgr2=0x7fd8c8076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:56.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.945+0000 7fd8cfa00700 1 --2- 172.21.15.90:0/2999057549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8c8075d20 0x7fd8c8076130 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0098f0 tx=0x7fd8bc009c00 comp rx=0 tx=0).stop 2024-05-27T08:03:56.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.946+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2999057549 shutdown_connections 2024-05-27T08:03:56.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.946+0000 7fd8cfa00700 1 --2- 172.21.15.90:0/2999057549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c8110b90 0x7fd8c8112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.946+0000 7fd8cfa00700 1 --2- 172.21.15.90:0/2999057549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8c8075d20 0x7fd8c8076130 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.946+0000 7fd8cfa00700 1 --2- 172.21.15.90:0/2999057549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8c8076670 0x7fd8c8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.946+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2999057549 >> 172.21.15.90:0/2999057549 conn(0x7fd8c8100030 msgr2=0x7fd8c8102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:56.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.946+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2999057549 shutdown_connections 2024-05-27T08:03:56.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.946+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2999057549 wait complete. 2024-05-27T08:03:56.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.949+0000 7fd8cfa00700 1 Processor -- start 2024-05-27T08:03:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.950+0000 7fd8cfa00700 1 -- start start 2024-05-27T08:03:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.953+0000 7fd8cfa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8c8075d20 0x7fd8c810daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.953+0000 7fd8cfa00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8c8076670 0x7fd8c810dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.953+0000 7fd8cfa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c8110b90 0x7fd8c810e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.954+0000 7fd8cfa00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81165b0 con 0x7fd8c8075d20 2024-05-27T08:03:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.954+0000 7fd8cfa00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8116430 con 0x7fd8c8110b90 2024-05-27T08:03:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.954+0000 7fd8cfa00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8116730 con 0x7fd8c8076670 2024-05-27T08:03:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.954+0000 7fd8cd000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8c8075d20 0x7fd8c810daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.954+0000 7fd8cda00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c8110b90 0x7fd8c810e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.954+0000 7fd8cda00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c8110b90 0x7fd8c810e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:42078/0 (socket says 172.21.15.90:42078) 2024-05-27T08:03:56.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.954+0000 7fd8cda00700 1 -- 172.21.15.90:0/2801079739 learned_addr learned my addr 172.21.15.90:0/2801079739 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:56.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.954+0000 7fd8c7e00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8c8076670 0x7fd8c810dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.955+0000 7fd8c7e00700 1 -- 172.21.15.90:0/2801079739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c8110b90 msgr2=0x7fd8c810e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:56.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.955+0000 7fd8c7e00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c8110b90 0x7fd8c810e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.955+0000 7fd8c7e00700 1 -- 172.21.15.90:0/2801079739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8c8075d20 msgr2=0x7fd8c810daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.955+0000 7fd8c7e00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8c8075d20 0x7fd8c810daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.955+0000 7fd8c7e00700 1 -- 172.21.15.90:0/2801079739 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c81b5cb0 con 0x7fd8c8076670 2024-05-27T08:03:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.955+0000 7fd8cd000700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8c8075d20 0x7fd8c810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:03:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.956+0000 7fd8c7e00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8c8076670 0x7fd8c810dfe0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400d950 tx=0x7fd8b400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.956+0000 7fd8c5600700 1 -- 172.21.15.90:0/2801079739 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4016070 con 0x7fd8c8076670 2024-05-27T08:03:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.957+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2801079739 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c81b5fa0 con 0x7fd8c8076670 2024-05-27T08:03:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.957+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2801079739 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c81b6450 con 0x7fd8c8076670 2024-05-27T08:03:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.957+0000 7fd8c5600700 1 -- 172.21.15.90:0/2801079739 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8b400fd20 con 0x7fd8c8076670 2024-05-27T08:03:56.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.957+0000 7fd8c5600700 1 -- 172.21.15.90:0/2801079739 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400e3b0 con 0x7fd8c8076670 2024-05-27T08:03:56.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.968+0000 7fd8c5600700 1 -- 172.21.15.90:0/2801079739 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd8b4019070 con 0x7fd8c8076670 2024-05-27T08:03:56.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.970+0000 7fd8de5d2b80 1 Processor -- start 2024-05-27T08:03:56.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.971+0000 7fd8de5d2b80 1 -- start start 2024-05-27T08:03:56.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.971+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e4542c9f0 0x558e454bf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.971+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 0x558e454bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.971+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e454bfe70 0x558e454cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.971+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558e453ede40 con 0x558e4542c9f0 2024-05-27T08:03:56.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.971+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558e453dd9e0 con 0x558e454bfe70 2024-05-27T08:03:56.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.971+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558e453e1a20 con 0x558e45340e40 2024-05-27T08:03:56.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.971+0000 7fd8c5600700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8b00b2af0 0x7fd8b00b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.972+0000 7fd8c5600700 1 -- 172.21.15.90:0/2801079739 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7fd8b40e2690 con 0x7fd8c8076670 2024-05-27T08:03:56.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.972+0000 7fd8cda00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8b00b2af0 0x7fd8b00b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.972+0000 7fd8c7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e454bfe70 0x558e454cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.972+0000 7fd8c7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e454bfe70 0x558e454cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:42084/0 (socket says 172.21.15.90:42084) 2024-05-27T08:03:56.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.972+0000 7fd8c7e00700 1 -- 172.21.15.90:0/4290534962 learned_addr learned my addr 172.21.15.90:0/4290534962 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:56.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.973+0000 7fd8cd000700 1 --2- 172.21.15.90:0/4290534962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 0x558e454bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.973+0000 7fd8cda00700 1 --2- 172.21.15.90:0/4290534962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e4542c9f0 0x558e454bf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.974+0000 7fd8cd000700 1 -- 172.21.15.90:0/4290534962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e454bfe70 msgr2=0x558e454cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:56.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.974+0000 7fd8cd000700 1 --2- 172.21.15.90:0/4290534962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e454bfe70 0x558e454cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.974+0000 7fd8cd000700 1 -- 172.21.15.90:0/4290534962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e4542c9f0 msgr2=0x558e454bf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:56.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.974+0000 7fd8cd000700 1 --2- 172.21.15.90:0/4290534962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e4542c9f0 0x558e454bf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.974+0000 7fd8cd000700 1 -- 172.21.15.90:0/4290534962 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e453a6420 con 0x558e45340e40 2024-05-27T08:03:56.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.975+0000 7fd8cd000700 1 --2- 172.21.15.90:0/4290534962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 0x558e454bf870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009810 tx=0x7fd8bc009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=181320603f1917be server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:56.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.976+0000 7fd8cda00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8b00b2af0 0x7fd8b00b5000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009fd0 tx=0x7fd8c0008b80 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:56.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.976+0000 7fd8ba000700 1 -- 172.21.15.90:0/4290534962 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00f6e0 con 0x558e45340e40 2024-05-27T08:03:56.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.976+0000 7fd8ba000700 1 -- 172.21.15.90:0/4290534962 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8bc028070 con 0x558e45340e40 2024-05-27T08:03:56.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.976+0000 7fd8ba000700 1 -- 172.21.15.90:0/4290534962 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc023670 con 0x558e45340e40 2024-05-27T08:03:56.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.978+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/4290534962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 msgr2=0x558e454bf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:56.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.978+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/4290534962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 0x558e454bf870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009810 tx=0x7fd8bc009400 comp rx=0 tx=0).stop 2024-05-27T08:03:56.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.979+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/4290534962 shutdown_connections 2024-05-27T08:03:56.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.979+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/4290534962 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e454bfe70 0x558e454cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.979+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/4290534962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e4542c9f0 0x558e454bf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.979+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/4290534962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 0x558e454bf870 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.979+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/4290534962 >> 172.21.15.90:0/4290534962 conn(0x558e453da390 msgr2=0x558e454bb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:56.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.979+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/4290534962 shutdown_connections 2024-05-27T08:03:56.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.979+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/4290534962 wait complete. 2024-05-27T08:03:56.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.980+0000 7fd8de5d2b80 1 -- start start 2024-05-27T08:03:56.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.981+0000 7fd8de5d2b80 1 Processor -- start 2024-05-27T08:03:56.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.982+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 0x558e454c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.982+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.982+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e454bfe70 0x558e454c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.982+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558e452e3fa0 con 0x558e454bfe70 2024-05-27T08:03:56.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.982+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558e4537e760 con 0x558e4542c9f0 2024-05-27T08:03:56.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.982+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558e454ddb30 con 0x558e45340e40 2024-05-27T08:03:56.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.982+0000 7fd8de5d2b80 10 client.0 _ll_register_callbacks cb 0x558e45387c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.983+0000 7fd8cd000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 0x558e454c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.983+0000 7fd8c7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.983+0000 7fd8cd000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 0x558e454c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59114/0 (socket says 172.21.15.90:59114) 2024-05-27T08:03:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.983+0000 7fd8cd000700 1 -- 172.21.15.90:0/657606893 learned_addr learned my addr 172.21.15.90:0/657606893 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:56.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.983+0000 7fd8cda00700 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e454bfe70 0x558e454c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.985+0000 7fd8cda00700 1 -- 172.21.15.90:0/657606893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 msgr2=0x558e454c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:56.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.985+0000 7fd8cda00700 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 0x558e454c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.985+0000 7fd8cda00700 1 -- 172.21.15.90:0/657606893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 msgr2=0x558e454c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:56.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.985+0000 7fd8cda00700 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.985+0000 7fd8cda00700 1 -- 172.21.15.90:0/657606893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e4532d8d0 con 0x558e454bfe70 2024-05-27T08:03:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.987+0000 7fd8cda00700 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e454bfe70 0x558e454c9fd0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009990 tx=0x7fd8c00096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e7a834e00a1671 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:56.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.987+0000 7fd8aea00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:03:56.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.988+0000 7fd8aea00700 1 -- 172.21.15.90:0/657606893 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0025070 con 0x558e454bfe70 2024-05-27T08:03:56.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.988+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/657606893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8bc028070 con 0x558e454bfe70 2024-05-27T08:03:56.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.988+0000 7fd8aea00700 1 -- 172.21.15.90:0/657606893 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7fd8c0021480 con 0x558e454bfe70 2024-05-27T08:03:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.988+0000 7fd8aea00700 1 -- 172.21.15.90:0/657606893 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd8c001c040 con 0x558e454bfe70 2024-05-27T08:03:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.988+0000 7fd8de5d2b80 10 client.18309 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:56.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.988+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/657606893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558e453ae060 con 0x558e454bfe70 2024-05-27T08:03:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.989+0000 7fd8aea00700 1 -- 172.21.15.90:0/657606893 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8bc028070 con 0x558e454bfe70 2024-05-27T08:03:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.989+0000 7fd8aea00700 1 -- 172.21.15.90:0/657606893 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fd8c0018620 con 0x558e454bfe70 2024-05-27T08:03:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.989+0000 7fd8de5d2b80 10 client.18309 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.989+0000 7fd8de5d2b80 10 client.18309 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:03:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.989+0000 7fd8de5d2b80 20 client.18309 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.989+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558e454eab60 0x558e454ecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.989+0000 7fd8de5d2b80 4 client.18309 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:03:56.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.989+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/657606893 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e453efd90 con 0x558e454eab60 2024-05-27T08:03:56.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.990+0000 7fd8cd000700 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558e454eab60 0x558e454ecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:56.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.992+0000 7fd8cd000700 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558e454eab60 0x558e454ecff0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc0f5ee13a02a65f server_cookie=537d35c434b6748a in_seq=0 out_seq=0 2024-05-27T08:03:56.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.992+0000 7fd8aea00700 10 client.18309 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T08:03:56.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.994+0000 7fd8aea00700 1 -- 172.21.15.90:0/657606893 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558e453efd90 con 0x558e454eab60 2024-05-27T08:03:56.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.994+0000 7fd8aea00700 10 client.18309 handle_command_reply: tid=0 2024-05-27T08:03:56.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.994+0000 7fd8de5d2b80 1 client.18309 shutdown 2024-05-27T08:03:56.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.995+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/657606893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e454bfe70 msgr2=0x558e454c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:56.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.995+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e454bfe70 0x558e454c9fd0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009990 tx=0x7fd8c00096e0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.995+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/657606893 shutdown_connections 2024-05-27T08:03:56.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.995+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e454bfe70 0x558e454c9fd0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.995+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e45340e40 0x558e454c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.995+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558e454eab60 0x558e454ecff0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.995+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/657606893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:56.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.995+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/657606893 >> 172.21.15.90:0/657606893 conn(0x558e453da390 msgr2=0x558e454bf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:56.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.995+0000 7fd8aea00700 0 client.18309 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T08:03:56.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.996+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/657606893 shutdown_connections 2024-05-27T08:03:56.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.996+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/657606893 wait complete. 2024-05-27T08:03:56.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:56.996+0000 7fd8de5d2b80 20 client.18309 trim_cache size 0 max 16384 2024-05-27T08:03:57.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.026+0000 7fd8de5d2b80 1 Processor -- start 2024-05-27T08:03:57.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.026+0000 7fd8de5d2b80 1 -- start start 2024-05-27T08:03:57.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.027+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 0x558e454c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:57.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.027+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454bf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:57.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.027+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:57.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.027+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8010840 con 0x558e45340e40 2024-05-27T08:03:57.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.027+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a800d890 con 0x558e4542c9f0 2024-05-27T08:03:57.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.027+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a800faa0 con 0x558e454bfe70 2024-05-27T08:03:57.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.028+0000 7fd8cda00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:57.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.028+0000 7fd8cda00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454e6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59124/0 (socket says 172.21.15.90:59124) 2024-05-27T08:03:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.028+0000 7fd8cda00700 1 -- 172.21.15.90:0/3092527972 learned_addr learned my addr 172.21.15.90:0/3092527972 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.028+0000 7fd8c7e00700 1 --2- 172.21.15.90:0/3092527972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454bf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.028+0000 7fd8cd000700 1 --2- 172.21.15.90:0/3092527972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 0x558e454c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.029+0000 7fd8cda00700 1 -- 172.21.15.90:0/3092527972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 msgr2=0x558e454bf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.029+0000 7fd8cda00700 1 --2- 172.21.15.90:0/3092527972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454bf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.029+0000 7fd8cda00700 1 -- 172.21.15.90:0/3092527972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 msgr2=0x558e454c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.029+0000 7fd8cda00700 1 --2- 172.21.15.90:0/3092527972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 0x558e454c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.029+0000 7fd8cda00700 1 -- 172.21.15.90:0/3092527972 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e454ca560 con 0x558e454bfe70 2024-05-27T08:03:57.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.030+0000 7fd8cda00700 1 --2- 172.21.15.90:0/3092527972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454e6c30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0018ac0 tx=0x7fd8c0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f245d1dcc2968b88 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:57.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.030+0000 7fd8b9600700 1 -- 172.21.15.90:0/3092527972 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0025040 con 0x558e454bfe70 2024-05-27T08:03:57.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.030+0000 7fd8b9600700 1 -- 172.21.15.90:0/3092527972 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8c0021480 con 0x558e454bfe70 2024-05-27T08:03:57.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.030+0000 7fd8b9600700 1 -- 172.21.15.90:0/3092527972 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c002a6a0 con 0x558e454bfe70 2024-05-27T08:03:57.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.032+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3092527972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 msgr2=0x558e454e6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.032+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/3092527972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454e6c30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0018ac0 tx=0x7fd8c0000f40 comp rx=0 tx=0).stop 2024-05-27T08:03:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.032+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3092527972 shutdown_connections 2024-05-27T08:03:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.032+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/3092527972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454e6c30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.032+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/3092527972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 0x558e454c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.032+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/3092527972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454bf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.032+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3092527972 >> 172.21.15.90:0/3092527972 conn(0x558e453da390 msgr2=0x558e454bf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.033+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3092527972 shutdown_connections 2024-05-27T08:03:57.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.033+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3092527972 wait complete. 2024-05-27T08:03:57.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.034+0000 7fd8de5d2b80 1 -- start start 2024-05-27T08:03:57.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.034+0000 7fd8de5d2b80 1 Processor -- start 2024-05-27T08:03:57.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.035+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 0x558e454c31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:57.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.035+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454c3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:57.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.035+0000 7fd8de5d2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:57.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.035+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a800f3d0 con 0x558e45340e40 2024-05-27T08:03:57.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.035+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a800df60 con 0x558e4542c9f0 2024-05-27T08:03:57.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.035+0000 7fd8de5d2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a800e630 con 0x558e454bfe70 2024-05-27T08:03:57.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.035+0000 7fd8de5d2b80 10 client.0 _ll_register_callbacks cb 0x558e45387c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:03:57.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.035+0000 7fd8cda00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:57.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.036+0000 7fd8c7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.036+0000 7fd8cda00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454c3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:59130/0 (socket says 172.21.15.90:59130) 2024-05-27T08:03:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.036+0000 7fd8cda00700 1 -- 172.21.15.90:0/3010468421 learned_addr learned my addr 172.21.15.90:0/3010468421 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:03:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.036+0000 7fd8cd000700 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 0x558e454c31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.036+0000 7fd8cda00700 1 -- 172.21.15.90:0/3010468421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 msgr2=0x558e454c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.036+0000 7fd8cda00700 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454c3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.037+0000 7fd8cda00700 1 -- 172.21.15.90:0/3010468421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 msgr2=0x558e454c31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:03:57.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.037+0000 7fd8cda00700 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 0x558e454c31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.037+0000 7fd8cda00700 1 -- 172.21.15.90:0/3010468421 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e454e1e20 con 0x558e454bfe70 2024-05-27T08:03:57.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.037+0000 7fd8cd000700 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 0x558e454c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:03:57.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.038+0000 7fd8cda00700 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454c3c60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0007160 tx=0x7fd8c00218e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21a2193c2bc5a524 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:03:57.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.038+0000 7fd8aea00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:03:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.038+0000 7fd8aea00700 1 -- 172.21.15.90:0/3010468421 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c003e070 con 0x558e454bfe70 2024-05-27T08:03:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.038+0000 7fd8aea00700 1 -- 172.21.15.90:0/3010468421 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9496+0+0 (secure 0 0 0) 0x7fd8c002a440 con 0x558e454bfe70 2024-05-27T08:03:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.038+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3010468421 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8c0021480 con 0x558e454bfe70 2024-05-27T08:03:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.039+0000 7fd8aea00700 1 -- 172.21.15.90:0/3010468421 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fd8c0025040 con 0x558e454bfe70 2024-05-27T08:03:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.039+0000 7fd8de5d2b80 10 client.28102 fetch_fsmap learned FSMap version 64 2024-05-27T08:03:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.039+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3010468421 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558e454e3080 con 0x558e454bfe70 2024-05-27T08:03:57.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.039+0000 7fd8aea00700 1 -- 172.21.15.90:0/3010468421 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8c0021480 con 0x558e454bfe70 2024-05-27T08:03:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.040+0000 7fd8aea00700 1 -- 172.21.15.90:0/3010468421 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fd8c0043560 con 0x558e454bfe70 2024-05-27T08:03:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.040+0000 7fd8de5d2b80 10 client.28102 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:03:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.040+0000 7fd8de5d2b80 10 client.28102 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:03:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.040+0000 7fd8de5d2b80 20 client.28102 populate_metadata read hostname 'smithi090' 2024-05-27T08:03:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.040+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558e454e3790 0x558e45557690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:03:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.040+0000 7fd8de5d2b80 4 client.28102 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:03:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.040+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3010468421 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e453e9570 con 0x558e454e3790 2024-05-27T08:03:57.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.041+0000 7fd8cd000700 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558e454e3790 0x558e45557690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:03:57.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.043+0000 7fd8cd000700 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558e454e3790 0x558e45557690 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f260df50224a47e1 server_cookie=853f2954566a4280 in_seq=0 out_seq=0 2024-05-27T08:03:57.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.043+0000 7fd8aea00700 10 client.28102 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T08:03:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.043+0000 7fd8aea00700 1 -- 172.21.15.90:0/3010468421 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e453e9570 con 0x558e454e3790 2024-05-27T08:03:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.043+0000 7fd8aea00700 10 client.28102 handle_command_reply: tid=0 2024-05-27T08:03:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.043+0000 7fd8de5d2b80 1 client.28102 shutdown 2024-05-27T08:03:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3010468421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 msgr2=0x558e454c3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454c3c60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0007160 tx=0x7fd8c00218e0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3010468421 shutdown_connections 2024-05-27T08:03:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558e454bfe70 0x558e454c3c60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558e45340e40 0x558e454c31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x558e454e3790 0x558e45557690 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 1 --2- 172.21.15.90:0/3010468421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e4542c9f0 0x558e454c3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8aea00700 0 client.28102 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T08:03:57.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3010468421 >> 172.21.15.90:0/3010468421 conn(0x558e453da390 msgr2=0x558e454c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:57.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3010468421 shutdown_connections 2024-05-27T08:03:57.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 1 -- 172.21.15.90:0/3010468421 wait complete. 2024-05-27T08:03:57.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.044+0000 7fd8de5d2b80 20 client.28102 trim_cache size 0 max 16384 2024-05-27T08:03:57.047 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:03:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2801079739 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8b00b2af0 msgr2=0x7fd8b00b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8b00b2af0 0x7fd8b00b5000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009fd0 tx=0x7fd8c0008b80 comp rx=0 tx=0).stop 2024-05-27T08:03:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2801079739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8c8076670 msgr2=0x7fd8c810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:03:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8c8076670 0x7fd8c810dfe0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400d950 tx=0x7fd8b400dc60 comp rx=0 tx=0).stop 2024-05-27T08:03:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2801079739 shutdown_connections 2024-05-27T08:03:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fd8b00b2af0 0x7fd8b00b5000 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c8110b90 0x7fd8c810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8c8075d20 0x7fd8c810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 --2- 172.21.15.90:0/2801079739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8c8076670 0x7fd8c810dfe0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:03:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2801079739 >> 172.21.15.90:0/2801079739 conn(0x7fd8c8100030 msgr2=0x7fd8c8101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:03:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2801079739 shutdown_connections 2024-05-27T08:03:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:03:57.047+0000 7fd8cfa00700 1 -- 172.21.15.90:0/2801079739 wait complete. 2024-05-27T08:03:57.919 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:57 smithi090 ceph-mon[31429]: pgmap v566: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 486 op/s 2024-05-27T08:03:58.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:03:58.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:57 smithi136 ceph-mon[35324]: pgmap v566: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 486 op/s 2024-05-27T08:03:58.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:57 smithi129 ceph-mon[35254]: pgmap v566: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 MiB/s wr, 486 op/s 2024-05-27T08:03:58.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 820.0 s, 460.8 tps, lat 215.787 ms stddev 189.468 2024-05-27T08:03:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:00.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:03:59 smithi090 ceph-mon[31429]: pgmap v567: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 454 op/s 2024-05-27T08:04:00.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:03:59 smithi136 ceph-mon[35324]: pgmap v567: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 454 op/s 2024-05-27T08:04:00.317 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:03:59 smithi129 ceph-mon[35254]: pgmap v567: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 454 op/s 2024-05-27T08:04:02.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:01 smithi090 ceph-mon[31429]: pgmap v568: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 569 op/s 2024-05-27T08:04:02.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:01 smithi136 ceph-mon[35324]: pgmap v568: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 569 op/s 2024-05-27T08:04:02.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:01 smithi129 ceph-mon[35254]: pgmap v568: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 569 op/s 2024-05-27T08:04:03.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 825.0 s, 493.8 tps, lat 201.869 ms stddev 190.350 2024-05-27T08:04:04.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:03 smithi090 ceph-mon[31429]: pgmap v569: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 506 op/s 2024-05-27T08:04:04.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:03 smithi136 ceph-mon[35324]: pgmap v569: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 506 op/s 2024-05-27T08:04:04.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:03 smithi129 ceph-mon[35254]: pgmap v569: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.8 MiB/s wr, 506 op/s 2024-05-27T08:04:05.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:04 smithi136 ceph-mon[35324]: osdmap e124: 12 total, 12 up, 12 in 2024-05-27T08:04:05.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:04 smithi129 ceph-mon[35254]: osdmap e124: 12 total, 12 up, 12 in 2024-05-27T08:04:05.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:04 smithi090 ceph-mon[31429]: osdmap e124: 12 total, 12 up, 12 in 2024-05-27T08:04:06.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:05 smithi136 ceph-mon[35324]: pgmap v571: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.5 MiB/s wr, 553 op/s 2024-05-27T08:04:06.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:05 smithi129 ceph-mon[35254]: pgmap v571: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.5 MiB/s wr, 553 op/s 2024-05-27T08:04:06.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:05 smithi090 ceph-mon[31429]: pgmap v571: 129 pgs: 7 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 102 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.5 MiB/s wr, 553 op/s 2024-05-27T08:04:06.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.533+0000 7fc331800700 1 -- 172.21.15.90:0/4278000019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c113910 msgr2=0x7fc32c113d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:06.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.533+0000 7fc331800700 1 --2- 172.21.15.90:0/4278000019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c113910 0x7fc32c113d20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc31400b0a0 tx=0x7fc31400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.534+0000 7fc331800700 1 -- 172.21.15.90:0/4278000019 shutdown_connections 2024-05-27T08:04:06.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.534+0000 7fc331800700 1 --2- 172.21.15.90:0/4278000019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc32c114260 0x7fc32c11bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.534+0000 7fc331800700 1 --2- 172.21.15.90:0/4278000019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c10efe0 0x7fc32c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.534+0000 7fc331800700 1 --2- 172.21.15.90:0/4278000019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c113910 0x7fc32c113d20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.534+0000 7fc331800700 1 -- 172.21.15.90:0/4278000019 >> 172.21.15.90:0/4278000019 conn(0x7fc32c10ab50 msgr2=0x7fc32c10cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:06.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.537+0000 7fc331800700 1 -- 172.21.15.90:0/4278000019 shutdown_connections 2024-05-27T08:04:06.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.537+0000 7fc331800700 1 -- 172.21.15.90:0/4278000019 wait complete. 2024-05-27T08:04:06.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.539+0000 7fc331800700 1 Processor -- start 2024-05-27T08:04:06.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.539+0000 7fc331800700 1 -- start start 2024-05-27T08:04:06.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.540+0000 7fc331800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc32c10efe0 0x7fc32c1190a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.540+0000 7fc331800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c113910 0x7fc32c1195e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.540+0000 7fc331800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c114260 0x7fc32c119b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.540+0000 7fc331800700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c11eef0 con 0x7fc32c10efe0 2024-05-27T08:04:06.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.540+0000 7fc331800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c11ed70 con 0x7fc32c114260 2024-05-27T08:04:06.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.540+0000 7fc331800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c11f070 con 0x7fc32c113910 2024-05-27T08:04:06.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.541+0000 7fc32aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc32c10efe0 0x7fc32c1190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.541+0000 7fc32aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc32c10efe0 0x7fc32c1190a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37342/0 (socket says 172.21.15.90:37342) 2024-05-27T08:04:06.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.541+0000 7fc32aa00700 1 -- 172.21.15.90:0/1373507790 learned_addr learned my addr 172.21.15.90:0/1373507790 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:06.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.541+0000 7fc32a000700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c113910 0x7fc32c1195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.541+0000 7fc32a000700 1 -- 172.21.15.90:0/1373507790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c114260 msgr2=0x7fc32c119b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:06.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.541+0000 7fc32a000700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c114260 0x7fc32c119b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.542+0000 7fc32a000700 1 -- 172.21.15.90:0/1373507790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc32c10efe0 msgr2=0x7fc32c1190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:06.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.542+0000 7fc32a000700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc32c10efe0 0x7fc32c1190a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.542+0000 7fc32a000700 1 -- 172.21.15.90:0/1373507790 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c1b5e80 con 0x7fc32c113910 2024-05-27T08:04:06.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.542+0000 7fc32aa00700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc32c10efe0 0x7fc32c1190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:06.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.542+0000 7fc32a000700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c113910 0x7fc32c1195e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc314000f80 tx=0x7fc3140096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:06.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.543+0000 7fc323400700 1 -- 172.21.15.90:0/1373507790 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314021070 con 0x7fc32c113910 2024-05-27T08:04:06.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.543+0000 7fc323400700 1 -- 172.21.15.90:0/1373507790 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc314007c80 con 0x7fc32c113910 2024-05-27T08:04:06.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.543+0000 7fc331800700 1 -- 172.21.15.90:0/1373507790 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c1b6110 con 0x7fc32c113910 2024-05-27T08:04:06.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.543+0000 7fc331800700 1 -- 172.21.15.90:0/1373507790 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c1b65c0 con 0x7fc32c113910 2024-05-27T08:04:06.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.544+0000 7fc323400700 1 -- 172.21.15.90:0/1373507790 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3140047c0 con 0x7fc32c113910 2024-05-27T08:04:06.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.549+0000 7fc323400700 1 -- 172.21.15.90:0/1373507790 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc314003a00 con 0x7fc32c113910 2024-05-27T08:04:06.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.552+0000 7fc323400700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc30c0b2c00 0x7fc30c0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.552+0000 7fc323400700 1 -- 172.21.15.90:0/1373507790 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9584+0+0 (secure 0 0 0) 0x7fc3140e3c50 con 0x7fc32c113910 2024-05-27T08:04:06.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.553+0000 7fc32aa00700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc30c0b2c00 0x7fc30c0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.555+0000 7fc32aa00700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc30c0b2c00 0x7fc30c0b5110 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc324009ea0 tx=0x7fc324009210 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:06.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.814+0000 7fc3402e7b80 1 Processor -- start 2024-05-27T08:04:06.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.815+0000 7fc3402e7b80 1 -- start start 2024-05-27T08:04:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.820+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de572e40 0x55f8de6f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.820+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de6f7d30 0x55f8de6f1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.820+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f1db0 0x55f8de6f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.820+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f8de61fe40 con 0x55f8de6f7d30 2024-05-27T08:04:06.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.820+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f8de60f9e0 con 0x55f8de572e40 2024-05-27T08:04:06.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.820+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f8de613a20 con 0x55f8de6f1db0 2024-05-27T08:04:06.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.821+0000 7fc32b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f1db0 0x55f8de6f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.821+0000 7fc32b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f1db0 0x55f8de6f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:51268/0 (socket says 172.21.15.90:51268) 2024-05-27T08:04:06.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.821+0000 7fc32b400700 1 -- 172.21.15.90:0/2255278286 learned_addr learned my addr 172.21.15.90:0/2255278286 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:06.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.821+0000 7fc32a000700 1 --2- 172.21.15.90:0/2255278286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de6f7d30 0x55f8de6f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.822+0000 7fc32b400700 1 -- 172.21.15.90:0/2255278286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de572e40 msgr2=0x55f8de6f77f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:04:06.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.822+0000 7fc32b400700 1 --2- 172.21.15.90:0/2255278286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de572e40 0x55f8de6f77f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.822+0000 7fc32b400700 1 -- 172.21.15.90:0/2255278286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de6f7d30 msgr2=0x55f8de6f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:06.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.822+0000 7fc32b400700 1 --2- 172.21.15.90:0/2255278286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de6f7d30 0x55f8de6f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.822+0000 7fc32b400700 1 -- 172.21.15.90:0/2255278286 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8de5d8420 con 0x55f8de6f1db0 2024-05-27T08:04:06.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.823+0000 7fc32b400700 1 --2- 172.21.15.90:0/2255278286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f1db0 0x55f8de6f21c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0097f0 tx=0x7fc31c009b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5cc58d1fe89224 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:06.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.823+0000 7fc31be00700 1 -- 172.21.15.90:0/2255278286 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c015020 con 0x55f8de6f1db0 2024-05-27T08:04:06.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.823+0000 7fc31be00700 1 -- 172.21.15.90:0/2255278286 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc31c011cc0 con 0x55f8de6f1db0 2024-05-27T08:04:06.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.823+0000 7fc31be00700 1 -- 172.21.15.90:0/2255278286 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00bd00 con 0x55f8de6f1db0 2024-05-27T08:04:06.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.825+0000 7fc3402e7b80 1 -- 172.21.15.90:0/2255278286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f1db0 msgr2=0x55f8de6f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:06.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.825+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/2255278286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f1db0 0x55f8de6f21c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0097f0 tx=0x7fc31c009b00 comp rx=0 tx=0).stop 2024-05-27T08:04:06.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.825+0000 7fc3402e7b80 1 -- 172.21.15.90:0/2255278286 shutdown_connections 2024-05-27T08:04:06.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.825+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/2255278286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f1db0 0x55f8de6f21c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.825+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/2255278286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de572e40 0x55f8de6f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.825+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/2255278286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de6f7d30 0x55f8de6f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.825+0000 7fc3402e7b80 1 -- 172.21.15.90:0/2255278286 >> 172.21.15.90:0/2255278286 conn(0x55f8de5d2fc0 msgr2=0x55f8de6ec8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:06.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.825+0000 7fc3402e7b80 1 -- 172.21.15.90:0/2255278286 shutdown_connections 2024-05-27T08:04:06.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.825+0000 7fc3402e7b80 1 -- 172.21.15.90:0/2255278286 wait complete. 2024-05-27T08:04:06.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.826+0000 7fc3402e7b80 1 -- start start 2024-05-27T08:04:06.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.826+0000 7fc3402e7b80 1 Processor -- start 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.827+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 0x55f8de6fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.827+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de6f1db0 0x55f8de6fba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.827+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6fbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.827+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f8de515fa0 con 0x55f8de572e40 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.827+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f8de5b0760 con 0x55f8de6f1db0 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.827+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f8de6fe800 con 0x55f8de6f7d30 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.827+0000 7fc3402e7b80 10 client.0 _ll_register_callbacks cb 0x55f8de5b9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.828+0000 7fc32aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 0x55f8de6fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.828+0000 7fc32aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 0x55f8de6fb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37360/0 (socket says 172.21.15.90:37360) 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.828+0000 7fc32aa00700 1 -- 172.21.15.90:0/3528863327 learned_addr learned my addr 172.21.15.90:0/3528863327 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.829+0000 7fc32aa00700 1 -- 172.21.15.90:0/3528863327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 msgr2=0x55f8de6fbf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:04:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.829+0000 7fc32aa00700 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6fbf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.829+0000 7fc32aa00700 1 -- 172.21.15.90:0/3528863327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de6f1db0 msgr2=0x55f8de6fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.830+0000 7fc32aa00700 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de6f1db0 0x55f8de6fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.830+0000 7fc32aa00700 1 -- 172.21.15.90:0/3528863327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8de55f8d0 con 0x55f8de572e40 2024-05-27T08:04:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.830+0000 7fc32aa00700 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 0x55f8de6fb4d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc324015a90 tx=0x7fc3240047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21d5bece28a51ef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.830+0000 7fc318c00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:04:06.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.831+0000 7fc318c00700 1 -- 172.21.15.90:0/3528863327 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324023020 con 0x55f8de572e40 2024-05-27T08:04:06.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.831+0000 7fc318c00700 1 -- 172.21.15.90:0/3528863327 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9584+0+0 (secure 0 0 0) 0x7fc3240134d0 con 0x55f8de572e40 2024-05-27T08:04:06.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.831+0000 7fc3402e7b80 1 -- 172.21.15.90:0/3528863327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc31c011cc0 con 0x55f8de572e40 2024-05-27T08:04:06.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.832+0000 7fc318c00700 1 -- 172.21.15.90:0/3528863327 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc32401b040 con 0x55f8de572e40 2024-05-27T08:04:06.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.833+0000 7fc3402e7b80 10 client.18318 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:06.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.833+0000 7fc3402e7b80 1 -- 172.21.15.90:0/3528863327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f8de5e0060 con 0x55f8de572e40 2024-05-27T08:04:06.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.834+0000 7fc318c00700 1 -- 172.21.15.90:0/3528863327 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc31c011cc0 con 0x55f8de572e40 2024-05-27T08:04:06.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.834+0000 7fc318c00700 1 -- 172.21.15.90:0/3528863327 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fc32401e030 con 0x55f8de572e40 2024-05-27T08:04:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.834+0000 7fc3402e7b80 10 client.18318 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.834+0000 7fc3402e7b80 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.834+0000 7fc3402e7b80 20 client.18318 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.834+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8de71c8b0 0x55f8de71eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.835+0000 7fc3402e7b80 4 client.18318 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.835+0000 7fc3402e7b80 1 -- 172.21.15.90:0/3528863327 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8de621d90 con 0x55f8de71c8b0 2024-05-27T08:04:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.837+0000 7fc32a000700 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8de71c8b0 0x55f8de71eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.839+0000 7fc32a000700 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8de71c8b0 0x55f8de71eda0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad58959cf02d5562 server_cookie=3632839203ee7918 in_seq=0 out_seq=0 2024-05-27T08:04:06.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.839+0000 7fc318c00700 10 client.18318 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:06.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.841+0000 7fc318c00700 1 -- 172.21.15.90:0/3528863327 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f8de621d90 con 0x55f8de71c8b0 2024-05-27T08:04:06.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.841+0000 7fc318c00700 10 client.18318 handle_command_reply: tid=0 2024-05-27T08:04:06.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.841+0000 7fc3402e7b80 1 client.18318 shutdown 2024-05-27T08:04:06.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.842+0000 7fc3402e7b80 1 -- 172.21.15.90:0/3528863327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 msgr2=0x55f8de6fb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:06.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.842+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 0x55f8de6fb4d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc324015a90 tx=0x7fc3240047e0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.842+0000 7fc3402e7b80 1 -- 172.21.15.90:0/3528863327 shutdown_connections 2024-05-27T08:04:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.842+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8de71c8b0 0x55f8de71eda0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.842+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6fbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.842+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 0x55f8de6fb4d0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.843+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/3528863327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de6f1db0 0x55f8de6fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.843+0000 7fc3402e7b80 1 -- 172.21.15.90:0/3528863327 >> 172.21.15.90:0/3528863327 conn(0x55f8de5d2fc0 msgr2=0x55f8de6ec580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.843+0000 7fc318c00700 0 client.18318 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.843+0000 7fc3402e7b80 1 -- 172.21.15.90:0/3528863327 shutdown_connections 2024-05-27T08:04:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.843+0000 7fc3402e7b80 1 -- 172.21.15.90:0/3528863327 wait complete. 2024-05-27T08:04:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.843+0000 7fc3402e7b80 20 client.18318 trim_cache size 0 max 16384 2024-05-27T08:04:06.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.872+0000 7fc3402e7b80 1 Processor -- start 2024-05-27T08:04:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.872+0000 7fc3402e7b80 1 -- start start 2024-05-27T08:04:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.873+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 0x55f8de6f1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.873+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de6f1db0 0x55f8de6fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.873+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6f5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.873+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc308010900 con 0x55f8de572e40 2024-05-27T08:04:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.873+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc30800d950 con 0x55f8de6f1db0 2024-05-27T08:04:06.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.873+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc30800fb60 con 0x55f8de6f7d30 2024-05-27T08:04:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.874+0000 7fc32aa00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 0x55f8de6f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.874+0000 7fc32b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.874+0000 7fc32b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6f5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:51272/0 (socket says 172.21.15.90:51272) 2024-05-27T08:04:06.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.874+0000 7fc32b400700 1 -- 172.21.15.90:0/4108685834 learned_addr learned my addr 172.21.15.90:0/4108685834 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:06.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.875+0000 7fc32b400700 1 -- 172.21.15.90:0/4108685834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de6f1db0 msgr2=0x55f8de6fd970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:04:06.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.875+0000 7fc32b400700 1 --2- 172.21.15.90:0/4108685834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de6f1db0 0x55f8de6fd970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.875+0000 7fc32b400700 1 -- 172.21.15.90:0/4108685834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 msgr2=0x55f8de6f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:06.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.875+0000 7fc32b400700 1 --2- 172.21.15.90:0/4108685834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 0x55f8de6f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.875+0000 7fc32b400700 1 -- 172.21.15.90:0/4108685834 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8de6f9a40 con 0x55f8de6f7d30 2024-05-27T08:04:06.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.875+0000 7fc32b400700 1 --2- 172.21.15.90:0/4108685834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6f5850 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0117f0 tx=0x7fc31c011820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=172ae5a7a5ebb3fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:06.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.876+0000 7fc31b400700 1 -- 172.21.15.90:0/4108685834 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c015050 con 0x55f8de6f7d30 2024-05-27T08:04:06.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.876+0000 7fc31b400700 1 -- 172.21.15.90:0/4108685834 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc31c01e070 con 0x55f8de6f7d30 2024-05-27T08:04:06.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.876+0000 7fc31b400700 1 -- 172.21.15.90:0/4108685834 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c023e00 con 0x55f8de6f7d30 2024-05-27T08:04:06.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.878+0000 7fc3402e7b80 1 -- 172.21.15.90:0/4108685834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 msgr2=0x55f8de6f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:06.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.878+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/4108685834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6f5850 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0117f0 tx=0x7fc31c011820 comp rx=0 tx=0).stop 2024-05-27T08:04:06.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.878+0000 7fc3402e7b80 1 -- 172.21.15.90:0/4108685834 shutdown_connections 2024-05-27T08:04:06.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.878+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/4108685834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6f5850 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.878+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/4108685834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de572e40 0x55f8de6f1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.878+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/4108685834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de6f1db0 0x55f8de6fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.878+0000 7fc3402e7b80 1 -- 172.21.15.90:0/4108685834 >> 172.21.15.90:0/4108685834 conn(0x55f8de5d2fc0 msgr2=0x55f8de6f5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:06.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.879+0000 7fc3402e7b80 1 -- 172.21.15.90:0/4108685834 shutdown_connections 2024-05-27T08:04:06.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.879+0000 7fc3402e7b80 1 -- 172.21.15.90:0/4108685834 wait complete. 2024-05-27T08:04:06.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.880+0000 7fc3402e7b80 1 -- start start 2024-05-27T08:04:06.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.880+0000 7fc3402e7b80 1 Processor -- start 2024-05-27T08:04:06.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.882+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de572e40 0x55f8de703f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.883+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de6f1db0 0x55f8de6fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.883+0000 7fc3402e7b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.883+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc30800f490 con 0x55f8de6f1db0 2024-05-27T08:04:06.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.883+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc30800e020 con 0x55f8de572e40 2024-05-27T08:04:06.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.883+0000 7fc3402e7b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc30800e6f0 con 0x55f8de6f7d30 2024-05-27T08:04:06.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.883+0000 7fc32aa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de572e40 0x55f8de703f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.884+0000 7fc32b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.884+0000 7fc32b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6fcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:51276/0 (socket says 172.21.15.90:51276) 2024-05-27T08:04:06.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.884+0000 7fc32b400700 1 -- 172.21.15.90:0/184855963 learned_addr learned my addr 172.21.15.90:0/184855963 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:06.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.884+0000 7fc3402e7b80 10 client.0 _ll_register_callbacks cb 0x55f8de5b9c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:06.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.885+0000 7fc32a000700 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de6f1db0 0x55f8de6fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.885+0000 7fc32b400700 1 -- 172.21.15.90:0/184855963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de572e40 msgr2=0x55f8de703f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:06.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.885+0000 7fc32b400700 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de572e40 0x55f8de703f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.885+0000 7fc32b400700 1 -- 172.21.15.90:0/184855963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de6f1db0 msgr2=0x55f8de6fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:06.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.885+0000 7fc32b400700 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de6f1db0 0x55f8de6fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.885+0000 7fc32b400700 1 -- 172.21.15.90:0/184855963 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8de6fd3e0 con 0x55f8de6f7d30 2024-05-27T08:04:06.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.888+0000 7fc32b400700 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6fcea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0119c0 tx=0x7fc31c0236a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f61a88753783610b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:06.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.888+0000 7fc318c00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:04:06.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.888+0000 7fc318c00700 1 -- 172.21.15.90:0/184855963 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c023a40 con 0x55f8de6f7d30 2024-05-27T08:04:06.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.889+0000 7fc318c00700 1 -- 172.21.15.90:0/184855963 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9584+0+0 (secure 0 0 0) 0x7fc31c023be0 con 0x55f8de6f7d30 2024-05-27T08:04:06.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.889+0000 7fc3402e7b80 1 -- 172.21.15.90:0/184855963 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc31c01e070 con 0x55f8de6f7d30 2024-05-27T08:04:06.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.890+0000 7fc318c00700 1 -- 172.21.15.90:0/184855963 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc31c023de0 con 0x55f8de6f7d30 2024-05-27T08:04:06.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.890+0000 7fc3402e7b80 10 client.28114 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:06.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.890+0000 7fc3402e7b80 1 -- 172.21.15.90:0/184855963 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f8de786fd0 con 0x55f8de6f7d30 2024-05-27T08:04:06.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.891+0000 7fc318c00700 1 -- 172.21.15.90:0/184855963 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc31c01e070 con 0x55f8de6f7d30 2024-05-27T08:04:06.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.893+0000 7fc318c00700 1 -- 172.21.15.90:0/184855963 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fc31c02c3f0 con 0x55f8de6f7d30 2024-05-27T08:04:06.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.893+0000 7fc3402e7b80 10 client.28114 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:06.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.893+0000 7fc3402e7b80 10 client.28114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:06.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.893+0000 7fc3402e7b80 20 client.28114 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:06.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.893+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8de7874f0 0x55f8de789910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:06.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.893+0000 7fc3402e7b80 4 client.28114 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:04:06.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.893+0000 7fc3402e7b80 1 -- 172.21.15.90:0/184855963 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f8de61b570 con 0x55f8de7874f0 2024-05-27T08:04:06.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.894+0000 7fc32aa00700 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8de7874f0 0x55f8de789910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:06.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.896+0000 7fc318c00700 10 client.28114 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:06.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.896+0000 7fc32aa00700 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8de7874f0 0x55f8de789910 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f86a7101469c3fe2 server_cookie=f840f3a9ff7a3dec in_seq=0 out_seq=0 2024-05-27T08:04:06.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.896+0000 7fc318c00700 1 -- 172.21.15.90:0/184855963 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f8de61b570 con 0x55f8de7874f0 2024-05-27T08:04:06.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.896+0000 7fc318c00700 10 client.28114 handle_command_reply: tid=0 2024-05-27T08:04:06.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.896+0000 7fc3402e7b80 1 client.28114 shutdown 2024-05-27T08:04:06.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.897+0000 7fc3402e7b80 1 -- 172.21.15.90:0/184855963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 msgr2=0x55f8de6fcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:06.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.897+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6fcea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0119c0 tx=0x7fc31c0236a0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.897+0000 7fc3402e7b80 1 -- 172.21.15.90:0/184855963 shutdown_connections 2024-05-27T08:04:06.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.897+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f8de7874f0 0x55f8de789910 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.899+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f8de6f7d30 0x55f8de6fcea0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.899+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f8de572e40 0x55f8de703f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.899+0000 7fc318c00700 0 client.28114 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:06.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.899+0000 7fc3402e7b80 1 --2- 172.21.15.90:0/184855963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f8de6f1db0 0x55f8de6fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.899+0000 7fc3402e7b80 1 -- 172.21.15.90:0/184855963 >> 172.21.15.90:0/184855963 conn(0x55f8de5d2fc0 msgr2=0x55f8de6f6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:06.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.900+0000 7fc3402e7b80 1 -- 172.21.15.90:0/184855963 shutdown_connections 2024-05-27T08:04:06.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.900+0000 7fc3402e7b80 1 -- 172.21.15.90:0/184855963 wait complete. 2024-05-27T08:04:06.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.900+0000 7fc3402e7b80 20 client.28114 trim_cache size 0 max 16384 2024-05-27T08:04:06.903 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:04:06.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.901+0000 7fc331800700 1 -- 172.21.15.90:0/1373507790 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc30c0b2c00 msgr2=0x7fc30c0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:06.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.901+0000 7fc331800700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc30c0b2c00 0x7fc30c0b5110 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc324009ea0 tx=0x7fc324009210 comp rx=0 tx=0).stop 2024-05-27T08:04:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.901+0000 7fc331800700 1 -- 172.21.15.90:0/1373507790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c113910 msgr2=0x7fc32c1195e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.901+0000 7fc331800700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c113910 0x7fc32c1195e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc314000f80 tx=0x7fc3140096e0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.902+0000 7fc331800700 1 -- 172.21.15.90:0/1373507790 shutdown_connections 2024-05-27T08:04:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.902+0000 7fc331800700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc30c0b2c00 0x7fc30c0b5110 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.902+0000 7fc331800700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc32c114260 0x7fc32c119b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.902+0000 7fc331800700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc32c10efe0 0x7fc32c1190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.902+0000 7fc331800700 1 --2- 172.21.15.90:0/1373507790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc32c113910 0x7fc32c1195e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.902+0000 7fc331800700 1 -- 172.21.15.90:0/1373507790 >> 172.21.15.90:0/1373507790 conn(0x7fc32c10ab50 msgr2=0x7fc32c1130d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.902+0000 7fc331800700 1 -- 172.21.15.90:0/1373507790 shutdown_connections 2024-05-27T08:04:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:06.902+0000 7fc331800700 1 -- 172.21.15.90:0/1373507790 wait complete. 2024-05-27T08:04:08.318 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:08 smithi129 ceph-mon[35254]: pgmap v572: 129 pgs: 9 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 100 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 516 op/s 2024-05-27T08:04:08.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:08 smithi090 ceph-mon[31429]: pgmap v572: 129 pgs: 9 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 100 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 516 op/s 2024-05-27T08:04:08.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:08 smithi136 ceph-mon[35324]: pgmap v572: 129 pgs: 9 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 100 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.9 MiB/s wr, 516 op/s 2024-05-27T08:04:08.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 830.0 s, 422.8 tps, lat 240.281 ms stddev 292.445 2024-05-27T08:04:09.051 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:04:09.051 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:04:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.649+0000 7f8a9f600640 1 -- 172.21.15.90:0/287415261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a9810b890 msgr2=0x7f8a9810dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.649+0000 7f8a9f600640 1 --2- 172.21.15.90:0/287415261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a9810b890 0x7f8a9810dcb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009900 tx=0x7f8a8802a480 comp rx=0 tx=0).stop 2024-05-27T08:04:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.649+0000 7f8a9f600640 1 -- 172.21.15.90:0/287415261 shutdown_connections 2024-05-27T08:04:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.649+0000 7f8a9f600640 1 --2- 172.21.15.90:0/287415261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a9810e1f0 0x7f8a981106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.649+0000 7f8a9f600640 1 --2- 172.21.15.90:0/287415261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a9810b890 0x7f8a9810dcb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.649+0000 7f8a9f600640 1 --2- 172.21.15.90:0/287415261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a98104e30 0x7f8a98105240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.649+0000 7f8a9f600640 1 -- 172.21.15.90:0/287415261 >> 172.21.15.90:0/287415261 conn(0x7f8a980782d0 msgr2=0x7f8a98078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.650+0000 7f8a9f600640 1 -- 172.21.15.90:0/287415261 shutdown_connections 2024-05-27T08:04:09.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.650+0000 7f8a9f600640 1 -- 172.21.15.90:0/287415261 wait complete. 2024-05-27T08:04:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.651+0000 7f8a9f600640 1 Processor -- start 2024-05-27T08:04:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.652+0000 7f8a9f600640 1 -- start start 2024-05-27T08:04:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.652+0000 7f8a9f600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a98104e30 0x7f8a981a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.652+0000 7f8a9f600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a9810b890 0x7f8a981a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.652+0000 7f8a9f600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a9810e1f0 0x7f8a981acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.652+0000 7f8a9f600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98113d90 con 0x7f8a9810b890 2024-05-27T08:04:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.652+0000 7f8a9f600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98113c10 con 0x7f8a9810e1f0 2024-05-27T08:04:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.652+0000 7f8a9f600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98113f10 con 0x7f8a98104e30 2024-05-27T08:04:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.653+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a9810b890 0x7f8a981a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.653+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a9810b890 0x7f8a981a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45542/0 (socket says 172.21.15.90:45542) 2024-05-27T08:04:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.653+0000 7f8a97e00640 1 -- 172.21.15.90:0/2916374365 learned_addr learned my addr 172.21.15.90:0/2916374365 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.653+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a98104e30 0x7f8a981a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.653+0000 7f8a9d600640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a9810e1f0 0x7f8a981acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.654+0000 7f8a97e00640 1 -- 172.21.15.90:0/2916374365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a98104e30 msgr2=0x7f8a981a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.654+0000 7f8a97e00640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a98104e30 0x7f8a981a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.654+0000 7f8a97e00640 1 -- 172.21.15.90:0/2916374365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a9810e1f0 msgr2=0x7f8a981acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.654+0000 7f8a97e00640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a9810e1f0 0x7f8a981acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.654+0000 7f8a97e00640 1 -- 172.21.15.90:0/2916374365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a98199260 con 0x7f8a9810b890 2024-05-27T08:04:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.654+0000 7f8a97e00640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a9810b890 0x7f8a981a2190 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8a8802a9f0 tx=0x7f8a88003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.656+0000 7f8a95600640 1 -- 172.21.15.90:0/2916374365 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8803f070 con 0x7f8a9810b890 2024-05-27T08:04:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.656+0000 7f8a95600640 1 -- 172.21.15.90:0/2916374365 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a8803baf0 con 0x7f8a9810b890 2024-05-27T08:04:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.656+0000 7f8a95600640 1 -- 172.21.15.90:0/2916374365 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88035030 con 0x7f8a9810b890 2024-05-27T08:04:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.657+0000 7f8a9f600640 1 -- 172.21.15.90:0/2916374365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a981994f0 con 0x7f8a9810b890 2024-05-27T08:04:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.657+0000 7f8a9f600640 1 -- 172.21.15.90:0/2916374365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a98199960 con 0x7f8a9810b890 2024-05-27T08:04:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.664+0000 7f8aa6585740 1 Processor -- start 2024-05-27T08:04:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.664+0000 7f8aa6585740 1 -- start start 2024-05-27T08:04:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.665+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.665+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.665+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348410d4d0 0x5634844d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.665+0000 7f8aa6585740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563484441b90 con 0x5634841052f0 2024-05-27T08:04:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.665+0000 7f8aa6585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563484431730 con 0x56348410d4d0 2024-05-27T08:04:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.665+0000 7f8aa6585740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563484435770 con 0x56348412e1b0 2024-05-27T08:04:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.666+0000 7f8a9d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.666+0000 7f8a9d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:43696/0 (socket says 172.21.15.90:43696) 2024-05-27T08:04:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.666+0000 7f8a9d600640 1 -- 172.21.15.90:0/1863417752 learned_addr learned my addr 172.21.15.90:0/1863417752 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.666+0000 7f8a9d600640 1 -- 172.21.15.90:0/1863417752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348410d4d0 msgr2=0x5634844d12e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:04:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.666+0000 7f8a9d600640 1 --2- 172.21.15.90:0/1863417752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348410d4d0 0x5634844d12e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.666+0000 7f8a9d600640 1 -- 172.21.15.90:0/1863417752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 msgr2=0x5634844d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.666+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/1863417752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.667+0000 7f8a9d600640 1 --2- 172.21.15.90:0/1863417752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.667+0000 7f8a9d600640 1 -- 172.21.15.90:0/1863417752 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634843ca8b0 con 0x56348412e1b0 2024-05-27T08:04:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.667+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/1863417752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:09.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.669+0000 7f8a9d600640 1 --2- 172.21.15.90:0/1863417752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844d0860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8a840099e0 tx=0x7f8a84009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2eaa202e8bd74a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.670+0000 7f8a8dc00640 1 -- 172.21.15.90:0/1863417752 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84017070 con 0x56348412e1b0 2024-05-27T08:04:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.670+0000 7f8a8dc00640 1 -- 172.21.15.90:0/1863417752 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a84013470 con 0x56348412e1b0 2024-05-27T08:04:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.670+0000 7f8a8dc00640 1 -- 172.21.15.90:0/1863417752 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84012630 con 0x56348412e1b0 2024-05-27T08:04:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.672+0000 7f8aa6585740 1 -- 172.21.15.90:0/1863417752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 msgr2=0x5634844d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:09.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.673+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1863417752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844d0860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8a840099e0 tx=0x7f8a84009ea0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.673+0000 7f8aa6585740 1 -- 172.21.15.90:0/1863417752 shutdown_connections 2024-05-27T08:04:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.673+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1863417752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348410d4d0 0x5634844d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.673+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1863417752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.673+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1863417752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844d0860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.673+0000 7f8aa6585740 1 -- 172.21.15.90:0/1863417752 >> 172.21.15.90:0/1863417752 conn(0x563484122300 msgr2=0x5634844cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.674+0000 7f8a95600640 1 -- 172.21.15.90:0/2916374365 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8a88042070 con 0x7f8a9810b890 2024-05-27T08:04:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.675+0000 7f8aa6585740 1 -- 172.21.15.90:0/1863417752 shutdown_connections 2024-05-27T08:04:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.676+0000 7f8aa6585740 1 -- 172.21.15.90:0/1863417752 wait complete. 2024-05-27T08:04:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.676+0000 7f8a95600640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a7c0b29b0 0x7f8a7c0b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.676+0000 7f8aa6585740 1 -- start start 2024-05-27T08:04:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.677+0000 7f8aa6585740 1 Processor -- start 2024-05-27T08:04:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.677+0000 7f8a9d600640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a7c0b29b0 0x7f8a7c0b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.678+0000 7f8a95600640 1 -- 172.21.15.90:0/2916374365 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9584+0+0 (secure 0 0 0) 0x7f8a88103260 con 0x7f8a9810b890 2024-05-27T08:04:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.678+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634841052f0 0x5634844dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.678+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 0x5634844db210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.679+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56348412e1b0 0x5634844f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.679+0000 7f8aa6585740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563484117790 con 0x56348412e1b0 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.679+0000 7f8aa6585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5634843d9bc0 con 0x5634841052f0 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.679+0000 7f8aa6585740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5634844ee9a0 con 0x56348410d4d0 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.679+0000 7f8aa6585740 10 client.0 _ll_register_callbacks cb 0x5634843b7400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.679+0000 7f8a9d600640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a7c0b29b0 0x7f8a7c0b4ec0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8a84013e90 tx=0x7f8a840047d0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.679+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 0x5634844db210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.679+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 0x5634844db210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:43710/0 (socket says 172.21.15.90:43710) 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.679+0000 7f8a97e00640 1 -- 172.21.15.90:0/2423004095 learned_addr learned my addr 172.21.15.90:0/2423004095 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.679+0000 7f8a9d600640 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56348412e1b0 0x5634844f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.680+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634841052f0 0x5634844dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.680+0000 7f8a97e00640 1 -- 172.21.15.90:0/2423004095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634841052f0 msgr2=0x5634844dacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.680+0000 7f8a97e00640 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634841052f0 0x5634844dacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.680+0000 7f8a97e00640 1 -- 172.21.15.90:0/2423004095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56348412e1b0 msgr2=0x5634844f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.680+0000 7f8a97e00640 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56348412e1b0 0x5634844f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.680+0000 7f8a97e00640 1 -- 172.21.15.90:0/2423004095 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563484146210 con 0x56348410d4d0 2024-05-27T08:04:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.681+0000 7f8a9d600640 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56348412e1b0 0x5634844f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:09.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.681+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634841052f0 0x5634844dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.681+0000 7f8a97e00640 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 0x5634844db210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8a880c6160 tx=0x7f8a880c9cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f5e1060d39d54e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.681+0000 7f8a82000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:04:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.682+0000 7f8a82000640 1 -- 172.21.15.90:0/2423004095 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880ca0e0 con 0x56348410d4d0 2024-05-27T08:04:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.682+0000 7f8aa6585740 1 -- 172.21.15.90:0/2423004095 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a84013470 con 0x56348410d4d0 2024-05-27T08:04:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.682+0000 7f8a82000640 1 -- 172.21.15.90:0/2423004095 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9584+0+0 (secure 0 0 0) 0x7f8a880c66d0 con 0x56348410d4d0 2024-05-27T08:04:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.683+0000 7f8a82000640 1 -- 172.21.15.90:0/2423004095 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8a880ca0e0 con 0x56348410d4d0 2024-05-27T08:04:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.686+0000 7f8aa6585740 10 client.28126 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.686+0000 7f8aa6585740 1 -- 172.21.15.90:0/2423004095 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x563484212620 con 0x56348410d4d0 2024-05-27T08:04:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.690+0000 7f8a82000640 1 -- 172.21.15.90:0/2423004095 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a84013470 con 0x56348410d4d0 2024-05-27T08:04:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.690+0000 7f8a82000640 1 -- 172.21.15.90:0/2423004095 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f8a880c6370 con 0x56348410d4d0 2024-05-27T08:04:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.690+0000 7f8aa6585740 10 client.28126 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.690+0000 7f8aa6585740 10 client.28126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.690+0000 7f8aa6585740 20 client.28126 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.691+0000 7f8aa6585740 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5634840ecd80 0x5634844fdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.692+0000 7f8aa6585740 4 client.28126 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.692+0000 7f8aa6585740 1 -- 172.21.15.90:0/2423004095 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563484443ae0 con 0x5634840ecd80 2024-05-27T08:04:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.692+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5634840ecd80 0x5634844fdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.694+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5634840ecd80 0x5634844fdcc0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f804c6a426506f server_cookie=608a1db29522225 in_seq=0 out_seq=0 2024-05-27T08:04:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.694+0000 7f8a82000640 10 client.28126 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.696+0000 7f8a82000640 1 -- 172.21.15.90:0/2423004095 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x563484443ae0 con 0x5634840ecd80 2024-05-27T08:04:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.696+0000 7f8a82000640 10 client.28126 handle_command_reply: tid=0 2024-05-27T08:04:09.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.696+0000 7f8aa6585740 1 client.28126 shutdown 2024-05-27T08:04:09.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.697+0000 7f8aa6585740 1 -- 172.21.15.90:0/2423004095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 msgr2=0x5634844db210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.697+0000 7f8aa6585740 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 0x5634844db210 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8a880c6160 tx=0x7f8a880c9cb0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.697+0000 7f8aa6585740 1 -- 172.21.15.90:0/2423004095 shutdown_connections 2024-05-27T08:04:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.697+0000 7f8aa6585740 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5634840ecd80 0x5634844fdcc0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.697+0000 7f8aa6585740 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56348412e1b0 0x5634844f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.697+0000 7f8aa6585740 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 0x5634844db210 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.697+0000 7f8aa6585740 1 --2- 172.21.15.90:0/2423004095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5634841052f0 0x5634844dacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.697+0000 7f8aa6585740 1 -- 172.21.15.90:0/2423004095 >> 172.21.15.90:0/2423004095 conn(0x563484122300 msgr2=0x5634844d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.697+0000 7f8a82000640 0 client.28126 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.698+0000 7f8aa6585740 1 -- 172.21.15.90:0/2423004095 shutdown_connections 2024-05-27T08:04:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.698+0000 7f8aa6585740 1 -- 172.21.15.90:0/2423004095 wait complete. 2024-05-27T08:04:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.698+0000 7f8aa6585740 20 client.28126 trim_cache size 0 max 16384 2024-05-27T08:04:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.748+0000 7f8aa6585740 1 Processor -- start 2024-05-27T08:04:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.749+0000 7f8aa6585740 1 -- start start 2024-05-27T08:04:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.749+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.749+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348410d4d0 0x5634844db030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.749+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.749+0000 7f8aa6585740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98070290 con 0x5634841052f0 2024-05-27T08:04:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.749+0000 7f8aa6585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9806d2e0 con 0x56348410d4d0 2024-05-27T08:04:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.749+0000 7f8aa6585740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9806f4f0 con 0x56348412e1b0 2024-05-27T08:04:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.750+0000 7f8a9d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.751+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348410d4d0 0x5634844db030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.751+0000 7f8a9cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.751+0000 7f8a9cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45578/0 (socket says 172.21.15.90:45578) 2024-05-27T08:04:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.751+0000 7f8a9cc00640 1 -- 172.21.15.90:0/4166149767 learned_addr learned my addr 172.21.15.90:0/4166149767 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.751+0000 7f8a9d600640 1 -- 172.21.15.90:0/4166149767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348410d4d0 msgr2=0x5634844db030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.751+0000 7f8a9d600640 1 --2- 172.21.15.90:0/4166149767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348410d4d0 0x5634844db030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.751+0000 7f8a9d600640 1 -- 172.21.15.90:0/4166149767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 msgr2=0x5634844d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.751+0000 7f8a9d600640 1 --2- 172.21.15.90:0/4166149767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.751+0000 7f8a9d600640 1 -- 172.21.15.90:0/4166149767 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634844d9030 con 0x56348412e1b0 2024-05-27T08:04:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.751+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/4166149767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.753+0000 7f8a9d600640 1 --2- 172.21.15.90:0/4166149767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844f7a70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8a84004700 tx=0x7f8a84008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adae3eb6de78853e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.754+0000 7f8a8d200640 1 -- 172.21.15.90:0/4166149767 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8401cba0 con 0x56348412e1b0 2024-05-27T08:04:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.754+0000 7f8a8d200640 1 -- 172.21.15.90:0/4166149767 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a8401cd40 con 0x56348412e1b0 2024-05-27T08:04:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.754+0000 7f8a8d200640 1 -- 172.21.15.90:0/4166149767 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8404b6a0 con 0x56348412e1b0 2024-05-27T08:04:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.755+0000 7f8aa6585740 1 -- 172.21.15.90:0/4166149767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 msgr2=0x5634844f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.755+0000 7f8aa6585740 1 --2- 172.21.15.90:0/4166149767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844f7a70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8a84004700 tx=0x7f8a84008040 comp rx=0 tx=0).stop 2024-05-27T08:04:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.757+0000 7f8aa6585740 1 -- 172.21.15.90:0/4166149767 shutdown_connections 2024-05-27T08:04:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.758+0000 7f8aa6585740 1 --2- 172.21.15.90:0/4166149767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348410d4d0 0x5634844db030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.758+0000 7f8aa6585740 1 --2- 172.21.15.90:0/4166149767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348412e1b0 0x5634844f7a70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.758+0000 7f8aa6585740 1 --2- 172.21.15.90:0/4166149767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.758+0000 7f8aa6585740 1 -- 172.21.15.90:0/4166149767 >> 172.21.15.90:0/4166149767 conn(0x563484122300 msgr2=0x5634844dcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:09.761 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.759+0000 7f8aa6585740 1 -- 172.21.15.90:0/4166149767 shutdown_connections 2024-05-27T08:04:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.760+0000 7f8aa6585740 1 -- 172.21.15.90:0/4166149767 wait complete. 2024-05-27T08:04:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.761+0000 7f8aa6585740 1 -- start start 2024-05-27T08:04:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.762+0000 7f8aa6585740 1 Processor -- start 2024-05-27T08:04:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.765+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.765+0000 7f8aa6585740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 0x5634844d5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.765+0000 7f8a9cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.765+0000 7f8a9cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45580/0 (socket says 172.21.15.90:45580) 2024-05-27T08:04:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.765+0000 7f8a9cc00640 1 -- 172.21.15.90:0/1589098488 learned_addr learned my addr 172.21.15.90:0/1589098488 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.766+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348412e1b0 0x5634844f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.766+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9806ee20 con 0x5634841052f0 2024-05-27T08:04:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.766+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9806d9b0 con 0x56348412e1b0 2024-05-27T08:04:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.766+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9806e080 con 0x56348410d4d0 2024-05-27T08:04:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.766+0000 7f8aa6585740 10 client.0 _ll_register_callbacks cb 0x7f8a88005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.767+0000 7f8a9d600640 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348412e1b0 0x5634844f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.767+0000 7f8a97e00640 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 0x5634844d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.767+0000 7f8a9cc00640 1 -- 172.21.15.90:0/1589098488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 msgr2=0x5634844d5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.767+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 0x5634844d5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.767+0000 7f8a9cc00640 1 -- 172.21.15.90:0/1589098488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348412e1b0 msgr2=0x5634844f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.767+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348412e1b0 0x5634844f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.767+0000 7f8a9cc00640 1 -- 172.21.15.90:0/1589098488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634844d5a10 con 0x5634841052f0 2024-05-27T08:04:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.768+0000 7f8a9cc00640 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d4c10 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8a90003b00 tx=0x7f8a900128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ae49cecc2696a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.768+0000 7f8a82000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:04:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.770+0000 7f8a82000640 1 -- 172.21.15.90:0/1589098488 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9001b820 con 0x5634841052f0 2024-05-27T08:04:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.770+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a8401cd40 con 0x5634841052f0 2024-05-27T08:04:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.770+0000 7f8a82000640 1 -- 172.21.15.90:0/1589098488 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9584+0+0 (secure 0 0 0) 0x7f8a9001b9c0 con 0x5634841052f0 2024-05-27T08:04:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.771+0000 7f8a82000640 1 -- 172.21.15.90:0/1589098488 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f8a9001bbc0 con 0x5634841052f0 2024-05-27T08:04:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.771+0000 7f8aa6585740 10 client.18336 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.771+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5634844f42e0 con 0x5634841052f0 2024-05-27T08:04:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.773+0000 7f8a82000640 1 -- 172.21.15.90:0/1589098488 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a8401cd40 con 0x5634841052f0 2024-05-27T08:04:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.773+0000 7f8a82000640 1 -- 172.21.15.90:0/1589098488 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f8a9002d4f0 con 0x5634841052f0 2024-05-27T08:04:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.773+0000 7f8aa6585740 10 client.18336 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.773+0000 7f8aa6585740 10 client.18336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.773+0000 7f8aa6585740 20 client.18336 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.773+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5634840c21c0 0x563484568330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.773+0000 7f8aa6585740 4 client.18336 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:04:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.773+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56348443d2c0 con 0x5634840c21c0 2024-05-27T08:04:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.774+0000 7f8a97e00640 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5634840c21c0 0x563484568330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.776+0000 7f8a97e00640 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5634840c21c0 0x563484568330 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70a474c4b77dc9f server_cookie=f2884ee9dbcc588e in_seq=0 out_seq=0 2024-05-27T08:04:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.776+0000 7f8a82000640 10 client.18336 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.848+0000 7f8a82000640 1 -- 172.21.15.90:0/1589098488 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56348443d2c0 con 0x5634840c21c0 2024-05-27T08:04:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.848+0000 7f8a82000640 10 client.18336 handle_command_reply: tid=0 2024-05-27T08:04:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.848+0000 7f8aa6585740 1 client.18336 shutdown 2024-05-27T08:04:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.850+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 msgr2=0x5634844d4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.850+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d4c10 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8a90003b00 tx=0x7f8a900128e0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.852+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 shutdown_connections 2024-05-27T08:04:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.852+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5634840c21c0 0x563484568330 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.852+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56348412e1b0 0x5634844f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.852+0000 7f8a82000640 0 client.18336 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.852+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56348410d4d0 0x5634844d5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.852+0000 7f8aa6585740 1 --2- 172.21.15.90:0/1589098488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5634841052f0 0x5634844d4c10 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.852+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 >> 172.21.15.90:0/1589098488 conn(0x563484122300 msgr2=0x5634844cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.853+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 shutdown_connections 2024-05-27T08:04:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.853+0000 7f8aa6585740 1 -- 172.21.15.90:0/1589098488 wait complete. 2024-05-27T08:04:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.853+0000 7f8aa6585740 20 client.18336 trim_cache size 0 max 16384 2024-05-27T08:04:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:04:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:04:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:04:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 -- 172.21.15.90:0/2916374365 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a7c0b29b0 msgr2=0x7f8a7c0b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a7c0b29b0 0x7f8a7c0b4ec0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8a84013e90 tx=0x7f8a840047d0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 -- 172.21.15.90:0/2916374365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a9810b890 msgr2=0x7f8a981a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a9810b890 0x7f8a981a2190 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8a8802a9f0 tx=0x7f8a88003c00 comp rx=0 tx=0).stop 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 -- 172.21.15.90:0/2916374365 shutdown_connections 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f8a7c0b29b0 0x7f8a7c0b4ec0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a9810e1f0 0x7f8a981acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a9810b890 0x7f8a981a2190 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 --2- 172.21.15.90:0/2916374365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a98104e30 0x7f8a981a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 -- 172.21.15.90:0/2916374365 >> 172.21.15.90:0/2916374365 conn(0x7f8a980782d0 msgr2=0x7f8a98102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 -- 172.21.15.90:0/2916374365 shutdown_connections 2024-05-27T08:04:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:09.856+0000 7f8a9f600640 1 -- 172.21.15.90:0/2916374365 wait complete. 2024-05-27T08:04:10.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:10 smithi129 ceph-mon[35254]: pgmap v573: 129 pgs: 9 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 100 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 494 op/s 2024-05-27T08:04:10.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:10 smithi090 ceph-mon[31429]: pgmap v573: 129 pgs: 9 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 100 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 494 op/s 2024-05-27T08:04:10.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:10 smithi136 ceph-mon[35324]: pgmap v573: 129 pgs: 9 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 100 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.7 MiB/s wr, 494 op/s 2024-05-27T08:04:10.735 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:12.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:12 smithi129 ceph-mon[35254]: pgmap v574: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 436 op/s 2024-05-27T08:04:12.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:12 smithi090 ceph-mon[31429]: pgmap v574: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 436 op/s 2024-05-27T08:04:12.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:12 smithi136 ceph-mon[35324]: pgmap v574: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 MiB/s wr, 436 op/s 2024-05-27T08:04:13.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:13 smithi090 ceph-mon[31429]: osdmap e125: 12 total, 12 up, 12 in 2024-05-27T08:04:13.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:13 smithi090 ceph-mon[31429]: pgmap v576: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-05-27T08:04:13.750 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:13 smithi136 ceph-mon[35324]: osdmap e125: 12 total, 12 up, 12 in 2024-05-27T08:04:13.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:13 smithi136 ceph-mon[35324]: pgmap v576: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-05-27T08:04:13.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:13 smithi129 ceph-mon[35254]: osdmap e125: 12 total, 12 up, 12 in 2024-05-27T08:04:13.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:13 smithi129 ceph-mon[35254]: pgmap v576: 129 pgs: 129 active+clean; 3.1 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.4 MiB/s wr, 395 op/s 2024-05-27T08:04:13.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 835.0 s, 537.6 tps, lat 186.070 ms stddev 95.226 2024-05-27T08:04:16.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:15 smithi090 ceph-mon[31429]: pgmap v577: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 MiB/s wr, 461 op/s 2024-05-27T08:04:16.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:15 smithi136 ceph-mon[35324]: pgmap v577: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 MiB/s wr, 461 op/s 2024-05-27T08:04:16.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:15 smithi129 ceph-mon[35254]: pgmap v577: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 MiB/s wr, 461 op/s 2024-05-27T08:04:16.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.939+0000 7f01ede00700 1 -- 172.21.15.90:0/3649378064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e8110980 msgr2=0x7f01e8110d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:16.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.939+0000 7f01ede00700 1 --2- 172.21.15.90:0/3649378064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e8110980 0x7f01e8110d90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f01e000b0a0 tx=0x7f01e000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.940+0000 7f01ede00700 1 -- 172.21.15.90:0/3649378064 shutdown_connections 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.940+0000 7f01ede00700 1 --2- 172.21.15.90:0/3649378064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e810b2a0 0x7f01e8117820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.940+0000 7f01ede00700 1 --2- 172.21.15.90:0/3649378064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e8110980 0x7f01e8110d90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.940+0000 7f01ede00700 1 --2- 172.21.15.90:0/3649378064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01e810a950 0x7f01e810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.940+0000 7f01ede00700 1 -- 172.21.15.90:0/3649378064 >> 172.21.15.90:0/3649378064 conn(0x7f01e8076d20 msgr2=0x7f01e8077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.941+0000 7f01ede00700 1 -- 172.21.15.90:0/3649378064 shutdown_connections 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.941+0000 7f01ede00700 1 -- 172.21.15.90:0/3649378064 wait complete. 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.942+0000 7f01ede00700 1 Processor -- start 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.943+0000 7f01ede00700 1 -- start start 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.943+0000 7f01ede00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e810a950 0x7f01e8075390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.943+0000 7f01ede00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e810b2a0 0x7f01e806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.943+0000 7f01ede00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01e8110980 0x7f01e806eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.943+0000 7f01ede00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f01e811aa50 con 0x7f01e810a950 2024-05-27T08:04:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.943+0000 7f01ede00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01e811a8d0 con 0x7f01e8110980 2024-05-27T08:04:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.943+0000 7f01ede00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01e811abd0 con 0x7f01e810b2a0 2024-05-27T08:04:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.944+0000 7f01e6a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e810a950 0x7f01e8075390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.944+0000 7f01e7400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01e8110980 0x7f01e806eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.944+0000 7f01e7400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01e8110980 0x7f01e806eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:41234/0 (socket says 172.21.15.90:41234) 2024-05-27T08:04:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.944+0000 7f01e7400700 1 -- 172.21.15.90:0/941332113 learned_addr learned my addr 172.21.15.90:0/941332113 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.944+0000 7f01e6000700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e810b2a0 0x7f01e806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.945+0000 7f01e6a00700 1 -- 172.21.15.90:0/941332113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e810b2a0 msgr2=0x7f01e806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.945+0000 7f01e6a00700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e810b2a0 0x7f01e806e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.945+0000 7f01e6a00700 1 -- 172.21.15.90:0/941332113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01e8110980 msgr2=0x7f01e806eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.945+0000 7f01e6a00700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01e8110980 0x7f01e806eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.945+0000 7f01e6a00700 1 -- 172.21.15.90:0/941332113 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e8072ec0 con 0x7f01e810a950 2024-05-27T08:04:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.945+0000 7f01e7400700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01e8110980 0x7f01e806eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.945+0000 7f01e6a00700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e810a950 0x7f01e8075390 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f01e000b070 tx=0x7f01e0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:16.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.946+0000 7f01eca00700 1 -- 172.21.15.90:0/941332113 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0021070 con 0x7f01e810a950 2024-05-27T08:04:16.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.946+0000 7f01eca00700 1 -- 172.21.15.90:0/941332113 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01e0007b80 con 0x7f01e810a950 2024-05-27T08:04:16.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.946+0000 7f01eca00700 1 -- 172.21.15.90:0/941332113 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0021070 con 0x7f01e810a950 2024-05-27T08:04:16.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.947+0000 7f01ede00700 1 -- 172.21.15.90:0/941332113 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e8073150 con 0x7f01e810a950 2024-05-27T08:04:16.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.948+0000 7f01ede00700 1 -- 172.21.15.90:0/941332113 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e80734f0 con 0x7f01e810a950 2024-05-27T08:04:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.954+0000 7f01eca00700 1 -- 172.21.15.90:0/941332113 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f01e0024070 con 0x7f01e810a950 2024-05-27T08:04:16.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.956+0000 7f01eca00700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f01cc0b2ca0 0x7f01cc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:16.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.957+0000 7f01eca00700 1 -- 172.21.15.90:0/941332113 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f01e000f020 con 0x7f01e810a950 2024-05-27T08:04:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.958+0000 7f01e6000700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f01cc0b2ca0 0x7f01cc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:16.959+0000 7f01e6000700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f01cc0b2ca0 0x7f01cc0b51b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f01e8108780 tx=0x7f01d8009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.088+0000 7f01fc884b80 1 Processor -- start 2024-05-27T08:04:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.089+0000 7f01fc884b80 1 -- start start 2024-05-27T08:04:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.089+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c042870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.090+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c042db0 0x56018c03c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.090+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c03d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.090+0000 7f01fc884b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56018bf6aec0 con 0x56018c042db0 2024-05-27T08:04:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.090+0000 7f01fc884b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56018bf5aa60 con 0x56018bebdec0 2024-05-27T08:04:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.090+0000 7f01fc884b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56018bf5eaa0 con 0x56018c03ce30 2024-05-27T08:04:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.090+0000 7f01e6a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c042870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.090+0000 7f01e6a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c042870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:41238/0 (socket says 172.21.15.90:41238) 2024-05-27T08:04:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.090+0000 7f01e6a00700 1 -- 172.21.15.90:0/1690688785 learned_addr learned my addr 172.21.15.90:0/1690688785 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.090+0000 7f01e7400700 1 --2- 172.21.15.90:0/1690688785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c03d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.091+0000 7f01e6000700 1 --2- 172.21.15.90:0/1690688785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c042db0 0x56018c03c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.091+0000 7f01e6a00700 1 -- 172.21.15.90:0/1690688785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 msgr2=0x56018c03d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.091+0000 7f01e6a00700 1 --2- 172.21.15.90:0/1690688785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c03d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.091+0000 7f01e6a00700 1 -- 172.21.15.90:0/1690688785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c042db0 msgr2=0x56018c03c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.091+0000 7f01e6a00700 1 --2- 172.21.15.90:0/1690688785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c042db0 0x56018c03c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.091+0000 7f01e6a00700 1 -- 172.21.15.90:0/1690688785 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018bf234a0 con 0x56018bebdec0 2024-05-27T08:04:17.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.092+0000 7f01e6a00700 1 --2- 172.21.15.90:0/1690688785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c042870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f01e00a7220 tx=0x7f01e00ae750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ca5b881b55b44c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.093+0000 7f01dcc00700 1 -- 172.21.15.90:0/1690688785 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00a69a0 con 0x56018bebdec0 2024-05-27T08:04:17.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.093+0000 7f01dcc00700 1 -- 172.21.15.90:0/1690688785 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01e00075c0 con 0x56018bebdec0 2024-05-27T08:04:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.093+0000 7f01dcc00700 1 -- 172.21.15.90:0/1690688785 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00a6b40 con 0x56018bebdec0 2024-05-27T08:04:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.094+0000 7f01fc884b80 1 -- 172.21.15.90:0/1690688785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 msgr2=0x56018c042870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.094+0000 7f01fc884b80 1 --2- 172.21.15.90:0/1690688785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c042870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f01e00a7220 tx=0x7f01e00ae750 comp rx=0 tx=0).stop 2024-05-27T08:04:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.094+0000 7f01fc884b80 1 -- 172.21.15.90:0/1690688785 shutdown_connections 2024-05-27T08:04:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.094+0000 7f01fc884b80 1 --2- 172.21.15.90:0/1690688785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c03d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.094+0000 7f01fc884b80 1 --2- 172.21.15.90:0/1690688785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c042870 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.094+0000 7f01fc884b80 1 --2- 172.21.15.90:0/1690688785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c042db0 0x56018c03c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.094+0000 7f01fc884b80 1 -- 172.21.15.90:0/1690688785 >> 172.21.15.90:0/1690688785 conn(0x56018bf1e040 msgr2=0x56018c037950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.095+0000 7f01fc884b80 1 -- 172.21.15.90:0/1690688785 shutdown_connections 2024-05-27T08:04:17.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.095+0000 7f01fc884b80 1 -- 172.21.15.90:0/1690688785 wait complete. 2024-05-27T08:04:17.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.095+0000 7f01fc884b80 1 -- start start 2024-05-27T08:04:17.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.096+0000 7f01fc884b80 1 Processor -- start 2024-05-27T08:04:17.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.097+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c046580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.097+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c03ce30 0x56018c046ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.097+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c042db0 0x56018c047000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.098+0000 7f01fc884b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56018be61020 con 0x56018c03ce30 2024-05-27T08:04:17.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.098+0000 7f01fc884b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56018befb7e0 con 0x56018bebdec0 2024-05-27T08:04:17.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.098+0000 7f01fc884b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56018c049640 con 0x56018c042db0 2024-05-27T08:04:17.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.098+0000 7f01e6a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c046580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.098+0000 7f01fc884b80 10 client.0 _ll_register_callbacks cb 0x56018bf04cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:17.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.098+0000 7f01e6a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c046580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:41240/0 (socket says 172.21.15.90:41240) 2024-05-27T08:04:17.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.098+0000 7f01e6a00700 1 -- 172.21.15.90:0/2933138268 learned_addr learned my addr 172.21.15.90:0/2933138268 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:17.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.098+0000 7f01e7400700 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c042db0 0x56018c047000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.099+0000 7f01e6000700 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c03ce30 0x56018c046ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.099+0000 7f01e7400700 1 -- 172.21.15.90:0/2933138268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 msgr2=0x56018c046580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.099+0000 7f01e7400700 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c046580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.099+0000 7f01e7400700 1 -- 172.21.15.90:0/2933138268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c03ce30 msgr2=0x56018c046ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.099+0000 7f01e7400700 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c03ce30 0x56018c046ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.099+0000 7f01e7400700 1 -- 172.21.15.90:0/2933138268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018beaa950 con 0x56018c042db0 2024-05-27T08:04:17.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.099+0000 7f01e6000700 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c03ce30 0x56018c046ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:17.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.100+0000 7f01e7400700 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c042db0 0x56018c047000 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f01d4011930 tx=0x7f01d4011c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f31bb4c773ab88 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.100+0000 7f01d1600700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:04:17.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.101+0000 7f01d1600700 1 -- 172.21.15.90:0/2933138268 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d400f940 con 0x56018c042db0 2024-05-27T08:04:17.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.101+0000 7f01d1600700 1 -- 172.21.15.90:0/2933138268 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f01d400fae0 con 0x56018c042db0 2024-05-27T08:04:17.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.101+0000 7f01fc884b80 1 -- 172.21.15.90:0/2933138268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01e00075c0 con 0x56018c042db0 2024-05-27T08:04:17.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.102+0000 7f01d1600700 1 -- 172.21.15.90:0/2933138268 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f01d400fce0 con 0x56018c042db0 2024-05-27T08:04:17.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.102+0000 7f01fc884b80 10 client.28144 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:17.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.102+0000 7f01fc884b80 1 -- 172.21.15.90:0/2933138268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56018bf2b0e0 con 0x56018c042db0 2024-05-27T08:04:17.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.103+0000 7f01d1600700 1 -- 172.21.15.90:0/2933138268 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01e00075c0 con 0x56018c042db0 2024-05-27T08:04:17.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.104+0000 7f01d1600700 1 -- 172.21.15.90:0/2933138268 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f01d401d400 con 0x56018c042db0 2024-05-27T08:04:17.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.104+0000 7f01fc884b80 10 client.28144 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:17.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.104+0000 7f01fc884b80 10 client.28144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:17.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.104+0000 7f01fc884b80 20 client.28144 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:17.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.104+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56018c056570 0x56018c058a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.104+0000 7f01fc884b80 4 client.28144 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:17.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.104+0000 7f01fc884b80 1 -- 172.21.15.90:0/2933138268 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56018bf6ce10 con 0x56018c056570 2024-05-27T08:04:17.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.105+0000 7f01e6a00700 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56018c056570 0x56018c058a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.108+0000 7f01e6a00700 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56018c056570 0x56018c058a00 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68de4dfe122d393c server_cookie=1f0a949ae31aa74c in_seq=0 out_seq=0 2024-05-27T08:04:17.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.108+0000 7f01d1600700 10 client.28144 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:17.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.111+0000 7f01d1600700 1 -- 172.21.15.90:0/2933138268 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56018bf6ce10 con 0x56018c056570 2024-05-27T08:04:17.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.111+0000 7f01d1600700 10 client.28144 handle_command_reply: tid=0 2024-05-27T08:04:17.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.111+0000 7f01fc884b80 1 client.28144 shutdown 2024-05-27T08:04:17.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.112+0000 7f01fc884b80 1 -- 172.21.15.90:0/2933138268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c042db0 msgr2=0x56018c047000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.112+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c042db0 0x56018c047000 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f01d4011930 tx=0x7f01d4011c40 comp rx=0 tx=0).stop 2024-05-27T08:04:17.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.112+0000 7f01fc884b80 1 -- 172.21.15.90:0/2933138268 shutdown_connections 2024-05-27T08:04:17.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.112+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56018c056570 0x56018c058a00 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.112+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c042db0 0x56018c047000 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.112+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c046580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.112+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2933138268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c03ce30 0x56018c046ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.112+0000 7f01fc884b80 1 -- 172.21.15.90:0/2933138268 >> 172.21.15.90:0/2933138268 conn(0x56018bf1e040 msgr2=0x56018c036bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.113+0000 7f01d1600700 0 client.28144 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:17.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.113+0000 7f01fc884b80 1 -- 172.21.15.90:0/2933138268 shutdown_connections 2024-05-27T08:04:17.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.113+0000 7f01fc884b80 1 -- 172.21.15.90:0/2933138268 wait complete. 2024-05-27T08:04:17.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.114+0000 7f01fc884b80 20 client.28144 trim_cache size 0 max 16384 2024-05-27T08:04:17.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.155+0000 7f01fc884b80 1 Processor -- start 2024-05-27T08:04:17.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.156+0000 7f01fc884b80 1 -- start start 2024-05-27T08:04:17.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.156+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018bebdec0 0x56018c03c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.156+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c046f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.156+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018c042db0 0x56018c041d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.156+0000 7f01fc884b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81c0ce0 con 0x56018bebdec0 2024-05-27T08:04:17.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.156+0000 7f01fc884b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81bdd30 con 0x56018c042db0 2024-05-27T08:04:17.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.156+0000 7f01fc884b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81bff40 con 0x56018c03ce30 2024-05-27T08:04:17.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.157+0000 7f01e6000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c046f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.158+0000 7f01e6000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c046f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:43768/0 (socket says 172.21.15.90:43768) 2024-05-27T08:04:17.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.158+0000 7f01e6000700 1 -- 172.21.15.90:0/3175416429 learned_addr learned my addr 172.21.15.90:0/3175416429 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:17.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.158+0000 7f01e7400700 1 --2- 172.21.15.90:0/3175416429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018c042db0 0x56018c041d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.159+0000 7f01e6000700 1 -- 172.21.15.90:0/3175416429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018c042db0 msgr2=0x56018c041d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.159+0000 7f01e6000700 1 --2- 172.21.15.90:0/3175416429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018c042db0 0x56018c041d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.159+0000 7f01e6000700 1 -- 172.21.15.90:0/3175416429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018bebdec0 msgr2=0x56018c03c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.159+0000 7f01e6000700 1 --2- 172.21.15.90:0/3175416429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018bebdec0 0x56018c03c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.159+0000 7f01e6000700 1 -- 172.21.15.90:0/3175416429 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018c044af0 con 0x56018c03ce30 2024-05-27T08:04:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.160+0000 7f01e6000700 1 --2- 172.21.15.90:0/3175416429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c046f70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f01d8011e00 tx=0x7f01d8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb830a0642060d81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.160+0000 7f01d3e00700 1 -- 172.21.15.90:0/3175416429 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8003740 con 0x56018c03ce30 2024-05-27T08:04:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.160+0000 7f01d3e00700 1 -- 172.21.15.90:0/3175416429 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01d8003d70 con 0x56018c03ce30 2024-05-27T08:04:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.160+0000 7f01d3e00700 1 -- 172.21.15.90:0/3175416429 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d801ddf0 con 0x56018c03ce30 2024-05-27T08:04:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.162+0000 7f01fc884b80 1 -- 172.21.15.90:0/3175416429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 msgr2=0x56018c046f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.162+0000 7f01fc884b80 1 --2- 172.21.15.90:0/3175416429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c046f70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f01d8011e00 tx=0x7f01d8014000 comp rx=0 tx=0).stop 2024-05-27T08:04:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.163+0000 7f01fc884b80 1 -- 172.21.15.90:0/3175416429 shutdown_connections 2024-05-27T08:04:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.163+0000 7f01fc884b80 1 --2- 172.21.15.90:0/3175416429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018c042db0 0x56018c041d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.163+0000 7f01fc884b80 1 --2- 172.21.15.90:0/3175416429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018bebdec0 0x56018c03c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.163+0000 7f01fc884b80 1 --2- 172.21.15.90:0/3175416429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c046f70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.164+0000 7f01fc884b80 1 -- 172.21.15.90:0/3175416429 >> 172.21.15.90:0/3175416429 conn(0x56018bf1e040 msgr2=0x56018c036bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.165+0000 7f01fc884b80 1 -- 172.21.15.90:0/3175416429 shutdown_connections 2024-05-27T08:04:17.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.166+0000 7f01fc884b80 1 -- 172.21.15.90:0/3175416429 wait complete. 2024-05-27T08:04:17.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.167+0000 7f01fc884b80 1 -- start start 2024-05-27T08:04:17.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.167+0000 7f01fc884b80 1 Processor -- start 2024-05-27T08:04:17.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.169+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c04e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.169+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c04ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.169+0000 7f01fc884b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c042db0 0x56018c04b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.169+0000 7f01fc884b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81bf870 con 0x56018c042db0 2024-05-27T08:04:17.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.169+0000 7f01fc884b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81be400 con 0x56018bebdec0 2024-05-27T08:04:17.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.169+0000 7f01fc884b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01e81bead0 con 0x56018c03ce30 2024-05-27T08:04:17.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.169+0000 7f01fc884b80 10 client.0 _ll_register_callbacks cb 0x56018bf04cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:17.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.171+0000 7f01e6a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c04e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.171+0000 7f01e6a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c04e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:41268/0 (socket says 172.21.15.90:41268) 2024-05-27T08:04:17.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.171+0000 7f01e6a00700 1 -- 172.21.15.90:0/2919444625 learned_addr learned my addr 172.21.15.90:0/2919444625 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:17.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.171+0000 7f01e6000700 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c04ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.172+0000 7f01e6a00700 1 -- 172.21.15.90:0/2919444625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 msgr2=0x56018c04ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.172+0000 7f01e6a00700 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c04ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.172+0000 7f01e6a00700 1 -- 172.21.15.90:0/2919444625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c042db0 msgr2=0x56018c04b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:17.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.172+0000 7f01e6a00700 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c042db0 0x56018c04b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.172+0000 7f01e6a00700 1 -- 172.21.15.90:0/2919444625 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56018c04bcb0 con 0x56018bebdec0 2024-05-27T08:04:17.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.173+0000 7f01e6a00700 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c04e7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f01e00a7220 tx=0x7f01e000b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5129c16dd44b12b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:17.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.173+0000 7f01d1600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:04:17.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.173+0000 7f01d1600700 1 -- 172.21.15.90:0/2919444625 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00a6d10 con 0x56018bebdec0 2024-05-27T08:04:17.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.173+0000 7f01d1600700 1 -- 172.21.15.90:0/2919444625 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f01e00e0810 con 0x56018bebdec0 2024-05-27T08:04:17.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.174+0000 7f01fc884b80 1 -- 172.21.15.90:0/2919444625 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d8003d70 con 0x56018bebdec0 2024-05-27T08:04:17.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.174+0000 7f01d1600700 1 -- 172.21.15.90:0/2919444625 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f01e00a6eb0 con 0x56018bebdec0 2024-05-27T08:04:17.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.174+0000 7f01fc884b80 10 client.38057 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:17.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.174+0000 7f01fc884b80 1 -- 172.21.15.90:0/2919444625 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56018c0c1780 con 0x56018bebdec0 2024-05-27T08:04:17.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.175+0000 7f01d1600700 1 -- 172.21.15.90:0/2919444625 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01d8003d70 con 0x56018bebdec0 2024-05-27T08:04:17.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.179+0000 7f01d1600700 1 -- 172.21.15.90:0/2919444625 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f01e0011070 con 0x56018bebdec0 2024-05-27T08:04:17.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.179+0000 7f01fc884b80 10 client.38057 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:17.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.179+0000 7f01fc884b80 10 client.38057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:17.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.179+0000 7f01fc884b80 20 client.38057 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:17.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.179+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56018c0c1cb0 0x56018c0c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:17.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.179+0000 7f01fc884b80 4 client.38057 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:04:17.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.179+0000 7f01fc884b80 1 -- 172.21.15.90:0/2919444625 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56018bf665f0 con 0x56018c0c1cb0 2024-05-27T08:04:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.180+0000 7f01e6000700 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56018c0c1cb0 0x56018c0c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.182+0000 7f01e6000700 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56018c0c1cb0 0x56018c0c40d0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c704fc27947c768d server_cookie=4e6d6c4d2a0e0c4d in_seq=0 out_seq=0 2024-05-27T08:04:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.182+0000 7f01d1600700 10 client.38057 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.184+0000 7f01d1600700 1 -- 172.21.15.90:0/2919444625 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56018bf665f0 con 0x56018c0c1cb0 2024-05-27T08:04:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.184+0000 7f01d1600700 10 client.38057 handle_command_reply: tid=0 2024-05-27T08:04:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.184+0000 7f01fc884b80 1 client.38057 shutdown 2024-05-27T08:04:17.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.186+0000 7f01fc884b80 1 -- 172.21.15.90:0/2919444625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 msgr2=0x56018c04e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.186+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c04e7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f01e00a7220 tx=0x7f01e000b700 comp rx=0 tx=0).stop 2024-05-27T08:04:17.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.187+0000 7f01fc884b80 1 -- 172.21.15.90:0/2919444625 shutdown_connections 2024-05-27T08:04:17.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.187+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56018c0c1cb0 0x56018c0c40d0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.187+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56018c042db0 0x56018c04b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.187+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56018bebdec0 0x56018c04e7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.187+0000 7f01fc884b80 1 --2- 172.21.15.90:0/2919444625 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56018c03ce30 0x56018c04ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.187+0000 7f01d1600700 0 client.38057 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:17.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.187+0000 7f01fc884b80 1 -- 172.21.15.90:0/2919444625 >> 172.21.15.90:0/2919444625 conn(0x56018bf1e040 msgr2=0x56018c046cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.187+0000 7f01fc884b80 1 -- 172.21.15.90:0/2919444625 shutdown_connections 2024-05-27T08:04:17.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.188+0000 7f01fc884b80 1 -- 172.21.15.90:0/2919444625 wait complete. 2024-05-27T08:04:17.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.188+0000 7f01fc884b80 20 client.38057 trim_cache size 0 max 16384 2024-05-27T08:04:17.191 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:04:17.191 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T08:04:17.192 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "4236b987-b19a-42a2-bdeb-c130a9a54693", 2024-05-27T08:04:17.192 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T08:04:17.192 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:04:17.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 -- 172.21.15.90:0/941332113 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f01cc0b2ca0 msgr2=0x7f01cc0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f01cc0b2ca0 0x7f01cc0b51b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f01e8108780 tx=0x7f01d8009260 comp rx=0 tx=0).stop 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 -- 172.21.15.90:0/941332113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e810a950 msgr2=0x7f01e8075390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e810a950 0x7f01e8075390 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f01e000b070 tx=0x7f01e0004d00 comp rx=0 tx=0).stop 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 -- 172.21.15.90:0/941332113 shutdown_connections 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f01cc0b2ca0 0x7f01cc0b51b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01e8110980 0x7f01e806eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e810a950 0x7f01e8075390 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 --2- 172.21.15.90:0/941332113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e810b2a0 0x7f01e806e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 -- 172.21.15.90:0/941332113 >> 172.21.15.90:0/941332113 conn(0x7f01e8076d20 msgr2=0x7f01e81091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 -- 172.21.15.90:0/941332113 shutdown_connections 2024-05-27T08:04:17.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:17.190+0000 7f01ede00700 1 -- 172.21.15.90:0/941332113 wait complete. 2024-05-27T08:04:18.106 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:04:18.204 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:17 smithi090 ceph-mon[31429]: pgmap v578: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 504 op/s 2024-05-27T08:04:18.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:17 smithi136 ceph-mon[35324]: pgmap v578: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 504 op/s 2024-05-27T08:04:18.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:17 smithi129 ceph-mon[35254]: pgmap v578: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.0 MiB/s wr, 504 op/s 2024-05-27T08:04:18.821 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:18.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 840.0 s, 528.6 tps, lat 186.137 ms stddev 142.700 2024-05-27T08:04:19.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:18 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T08:04:19.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:18 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:19.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:18 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T08:04:19.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:18 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:19.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:18 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T08:04:19.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:18 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:19.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:18 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T08:04:19.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:18 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:19.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:18 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T08:04:19.317 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:18 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:19.317 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:18 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T08:04:19.317 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:18 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T08:04:20.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:19 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T08:04:20.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:19 smithi090 ceph-mon[31429]: pgmap v579: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.3 MiB/s wr, 557 op/s 2024-05-27T08:04:20.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:19 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T08:04:20.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:19 smithi136 ceph-mon[35324]: pgmap v579: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.3 MiB/s wr, 557 op/s 2024-05-27T08:04:20.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:19 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T08:04:20.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:19 smithi129 ceph-mon[35254]: pgmap v579: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.3 MiB/s wr, 557 op/s 2024-05-27T08:04:22.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:21 smithi090 ceph-mon[31429]: pgmap v580: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.6 MiB/s wr, 643 op/s 2024-05-27T08:04:22.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:21 smithi136 ceph-mon[35324]: pgmap v580: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.6 MiB/s wr, 643 op/s 2024-05-27T08:04:22.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:21 smithi129 ceph-mon[35254]: pgmap v580: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.6 MiB/s wr, 643 op/s 2024-05-27T08:04:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.146+0000 7f2967200640 1 -- 172.21.15.90:0/4090880681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2968075480 msgr2=0x7f2968075890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.146+0000 7f2967200640 1 --2- 172.21.15.90:0/4090880681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2968075480 0x7f2968075890 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2958009900 tx=0x7f295802a480 comp rx=0 tx=0).stop 2024-05-27T08:04:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.146+0000 7f2967200640 1 -- 172.21.15.90:0/4090880681 shutdown_connections 2024-05-27T08:04:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.146+0000 7f2967200640 1 --2- 172.21.15.90:0/4090880681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2968114a70 0x7f2968116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.146+0000 7f2967200640 1 --2- 172.21.15.90:0/4090880681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2968112110 0x7f2968114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.146+0000 7f2967200640 1 --2- 172.21.15.90:0/4090880681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2968075480 0x7f2968075890 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.146+0000 7f2967200640 1 -- 172.21.15.90:0/4090880681 >> 172.21.15.90:0/4090880681 conn(0x7f2968079140 msgr2=0x7f2968079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.146+0000 7f2967200640 1 -- 172.21.15.90:0/4090880681 shutdown_connections 2024-05-27T08:04:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.147+0000 7f2967200640 1 -- 172.21.15.90:0/4090880681 wait complete. 2024-05-27T08:04:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.147+0000 7f2967200640 1 Processor -- start 2024-05-27T08:04:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.148+0000 7f2967200640 1 -- start start 2024-05-27T08:04:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.148+0000 7f2967200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2968075480 0x7f29681a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.148+0000 7f2967200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2968112110 0x7f29681a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.148+0000 7f2967200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2968114a70 0x7f29681a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.148+0000 7f2967200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f296811a1c0 con 0x7f2968114a70 2024-05-27T08:04:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.148+0000 7f2967200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f296811a040 con 0x7f2968112110 2024-05-27T08:04:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.148+0000 7f2967200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f296811a340 con 0x7f2968075480 2024-05-27T08:04:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.148+0000 7f2966800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2968114a70 0x7f29681a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.148+0000 7f2966800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2968114a70 0x7f29681a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39194/0 (socket says 172.21.15.90:39194) 2024-05-27T08:04:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.148+0000 7f2966800640 1 -- 172.21.15.90:0/4259747211 learned_addr learned my addr 172.21.15.90:0/4259747211 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f2965e00640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2968075480 0x7f29681a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f2965400640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2968112110 0x7f29681a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f2966800640 1 -- 172.21.15.90:0/4259747211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2968075480 msgr2=0x7f29681a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f2966800640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2968075480 0x7f29681a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f2966800640 1 -- 172.21.15.90:0/4259747211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2968112110 msgr2=0x7f29681a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f2966800640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2968112110 0x7f29681a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f2966800640 1 -- 172.21.15.90:0/4259747211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29681a1160 con 0x7f2968114a70 2024-05-27T08:04:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f2966800640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2968114a70 0x7f29681a0ac0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f296000cad0 tx=0x7f296000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f295ea00640 1 -- 172.21.15.90:0/4259747211 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296000ae20 con 0x7f2968114a70 2024-05-27T08:04:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f295ea00640 1 -- 172.21.15.90:0/4259747211 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2960005020 con 0x7f2968114a70 2024-05-27T08:04:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f2967200640 1 -- 172.21.15.90:0/4259747211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29681a1450 con 0x7f2968114a70 2024-05-27T08:04:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f295ea00640 1 -- 172.21.15.90:0/4259747211 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296000f6b0 con 0x7f2968114a70 2024-05-27T08:04:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.149+0000 7f2967200640 1 -- 172.21.15.90:0/4259747211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29681ad730 con 0x7f2968114a70 2024-05-27T08:04:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.153+0000 7f29739cf740 1 Processor -- start 2024-05-27T08:04:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.154+0000 7f295ea00640 1 -- 172.21.15.90:0/4259747211 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f29600051d0 con 0x7f2968114a70 2024-05-27T08:04:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.154+0000 7f29739cf740 1 -- start start 2024-05-27T08:04:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f29739cf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de61da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f29739cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de2522f0 0x5631de61dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f29739cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de25a4d0 0x5631de61e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f29739cf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5631de58eb90 con 0x5631de27b1b0 2024-05-27T08:04:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f29739cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5631de57e730 con 0x5631de25a4d0 2024-05-27T08:04:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f29739cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631de582770 con 0x5631de2522f0 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f2965400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de61da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f2965400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de61da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39206/0 (socket says 172.21.15.90:39206) 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f2965400640 1 -- 172.21.15.90:0/2249731891 learned_addr learned my addr 172.21.15.90:0/2249731891 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f2965400640 1 -- 172.21.15.90:0/2249731891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de2522f0 msgr2=0x5631de61dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f2965400640 1 --2- 172.21.15.90:0/2249731891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de2522f0 0x5631de61dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f2965400640 1 -- 172.21.15.90:0/2249731891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de25a4d0 msgr2=0x5631de61e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f2965400640 1 --2- 172.21.15.90:0/2249731891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de25a4d0 0x5631de61e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.155+0000 7f2965400640 1 -- 172.21.15.90:0/2249731891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631de5178b0 con 0x5631de27b1b0 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.156+0000 7f295ea00640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f294c0b29d0 0x7f294c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.156+0000 7f2965400640 1 --2- 172.21.15.90:0/2249731891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de61da60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f295400a9a0 tx=0x7f295400ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34b2db9ec247cee server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.156+0000 7f295ea00640 1 -- 172.21.15.90:0/4259747211 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f29600e2e00 con 0x7f2968114a70 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.156+0000 7f2965e00640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f294c0b29d0 0x7f294c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.156+0000 7f2953000640 1 -- 172.21.15.90:0/2249731891 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295400f040 con 0x5631de27b1b0 2024-05-27T08:04:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.156+0000 7f2953000640 1 -- 172.21.15.90:0/2249731891 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2954018070 con 0x5631de27b1b0 2024-05-27T08:04:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.156+0000 7f2953000640 1 -- 172.21.15.90:0/2249731891 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954013e10 con 0x5631de27b1b0 2024-05-27T08:04:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.157+0000 7f29739cf740 1 -- 172.21.15.90:0/2249731891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 msgr2=0x5631de61da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.157+0000 7f2965e00640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f294c0b29d0 0x7f294c0b4ee0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f29681a1c00 tx=0x7f2958005a20 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.157+0000 7f29739cf740 1 --2- 172.21.15.90:0/2249731891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de61da60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f295400a9a0 tx=0x7f295400ae60 comp rx=0 tx=0).stop 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.157+0000 7f29739cf740 1 -- 172.21.15.90:0/2249731891 shutdown_connections 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.157+0000 7f29739cf740 1 --2- 172.21.15.90:0/2249731891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de25a4d0 0x5631de61e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.157+0000 7f29739cf740 1 --2- 172.21.15.90:0/2249731891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de2522f0 0x5631de61dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.157+0000 7f29739cf740 1 --2- 172.21.15.90:0/2249731891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de61da60 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.157+0000 7f29739cf740 1 -- 172.21.15.90:0/2249731891 >> 172.21.15.90:0/2249731891 conn(0x5631de26f300 msgr2=0x5631de61a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.157+0000 7f29739cf740 1 -- 172.21.15.90:0/2249731891 shutdown_connections 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.157+0000 7f29739cf740 1 -- 172.21.15.90:0/2249731891 wait complete. 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.158+0000 7f29739cf740 1 -- start start 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.158+0000 7f29739cf740 1 Processor -- start 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.158+0000 7f29739cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 0x5631de627f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.158+0000 7f29739cf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de25a4d0 0x5631de628460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.158+0000 7f29739cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de27b1b0 0x5631de633ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.158+0000 7f29739cf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5631de264790 con 0x5631de25a4d0 2024-05-27T08:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.158+0000 7f29739cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5631de526bc0 con 0x5631de2522f0 2024-05-27T08:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.158+0000 7f29739cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631de62ab90 con 0x5631de27b1b0 2024-05-27T08:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.158+0000 7f29739cf740 10 client.0 _ll_register_callbacks cb 0x5631de504400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de25a4d0 0x5631de628460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de25a4d0 0x5631de628460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39214/0 (socket says 172.21.15.90:39214) 2024-05-27T08:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965400640 1 -- 172.21.15.90:0/3855684810 learned_addr learned my addr 172.21.15.90:0/3855684810 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965e00640 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 0x5631de627f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2966800640 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de27b1b0 0x5631de633ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965400640 1 -- 172.21.15.90:0/3855684810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de27b1b0 msgr2=0x5631de633ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965400640 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de27b1b0 0x5631de633ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965400640 1 -- 172.21.15.90:0/3855684810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 msgr2=0x5631de627f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965400640 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 0x5631de627f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965400640 1 -- 172.21.15.90:0/3855684810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631de293210 con 0x5631de25a4d0 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965e00640 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 0x5631de627f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2965400640 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de25a4d0 0x5631de628460 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f295400a120 tx=0x7f2954013940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae9dd2213c614ed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2947400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2947400640 1 -- 172.21.15.90:0/3855684810 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954023020 con 0x5631de25a4d0 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f2947400640 1 -- 172.21.15.90:0/3855684810 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f295400ea10 con 0x5631de25a4d0 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.159+0000 7f29739cf740 1 -- 172.21.15.90:0/3855684810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2954018070 con 0x5631de25a4d0 2024-05-27T08:04:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f2947400640 1 -- 172.21.15.90:0/3855684810 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f295402e050 con 0x5631de25a4d0 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f29739cf740 10 client.18357 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f29739cf740 1 -- 172.21.15.90:0/3855684810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5631de35f620 con 0x5631de25a4d0 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f2947400640 1 -- 172.21.15.90:0/3855684810 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2954018070 con 0x5631de25a4d0 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f2947400640 1 -- 172.21.15.90:0/3855684810 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f29540273a0 con 0x5631de25a4d0 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f29739cf740 10 client.18357 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f29739cf740 10 client.18357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f29739cf740 20 client.18357 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f29739cf740 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631de239d80 0x5631de639fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f29739cf740 4 client.18357 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.160+0000 7f29739cf740 1 -- 172.21.15.90:0/3855684810 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631de590ae0 con 0x5631de239d80 2024-05-27T08:04:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.161+0000 7f2965e00640 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631de239d80 0x5631de639fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.162+0000 7f2965e00640 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631de239d80 0x5631de639fc0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26ee88955115f0f server_cookie=87417e871caffe4e in_seq=0 out_seq=0 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.162+0000 7f2947400640 10 client.18357 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f2947400640 1 -- 172.21.15.90:0/3855684810 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5631de590ae0 con 0x5631de239d80 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f2947400640 10 client.18357 handle_command_reply: tid=0 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f29739cf740 1 client.18357 shutdown 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f29739cf740 1 -- 172.21.15.90:0/3855684810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de25a4d0 msgr2=0x5631de628460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f29739cf740 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de25a4d0 0x5631de628460 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f295400a120 tx=0x7f2954013940 comp rx=0 tx=0).stop 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f29739cf740 1 -- 172.21.15.90:0/3855684810 shutdown_connections 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f29739cf740 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631de239d80 0x5631de639fc0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f29739cf740 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de27b1b0 0x5631de633ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f29739cf740 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de25a4d0 0x5631de628460 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f29739cf740 1 --2- 172.21.15.90:0/3855684810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 0x5631de627f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f2947400640 0 client.18357 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f29739cf740 1 -- 172.21.15.90:0/3855684810 >> 172.21.15.90:0/3855684810 conn(0x5631de26f300 msgr2=0x5631de61d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.164+0000 7f29739cf740 1 -- 172.21.15.90:0/3855684810 shutdown_connections 2024-05-27T08:04:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.165+0000 7f29739cf740 1 -- 172.21.15.90:0/3855684810 wait complete. 2024-05-27T08:04:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.165+0000 7f29739cf740 20 client.18357 trim_cache size 0 max 16384 2024-05-27T08:04:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.180+0000 7f29739cf740 1 Processor -- start 2024-05-27T08:04:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.180+0000 7f29739cf740 1 -- start start 2024-05-27T08:04:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f29739cf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de2522f0 0x5631de623520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f29739cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de25a4d0 0x5631de629b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f29739cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de27b1b0 0x5631de62a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f29739cf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f29681b9180 con 0x5631de2522f0 2024-05-27T08:04:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f29739cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29681b61d0 con 0x5631de27b1b0 2024-05-27T08:04:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f29739cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f29681b83e0 con 0x5631de25a4d0 2024-05-27T08:04:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f2965e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de2522f0 0x5631de623520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f2965e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de2522f0 0x5631de623520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39222/0 (socket says 172.21.15.90:39222) 2024-05-27T08:04:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f2965e00640 1 -- 172.21.15.90:0/2405448372 learned_addr learned my addr 172.21.15.90:0/2405448372 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f2965e00640 1 -- 172.21.15.90:0/2405448372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de25a4d0 msgr2=0x5631de629b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f2965e00640 1 --2- 172.21.15.90:0/2405448372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de25a4d0 0x5631de629b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f2965e00640 1 -- 172.21.15.90:0/2405448372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de27b1b0 msgr2=0x5631de62a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f2965e00640 1 --2- 172.21.15.90:0/2405448372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de27b1b0 0x5631de62a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.181+0000 7f2965e00640 1 -- 172.21.15.90:0/2405448372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631de61ae00 con 0x5631de2522f0 2024-05-27T08:04:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.182+0000 7f2965e00640 1 --2- 172.21.15.90:0/2405448372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de2522f0 0x5631de623520 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f29580036a0 tx=0x7f2958041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51eb424f111b842 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.182+0000 7f2952600640 1 -- 172.21.15.90:0/2405448372 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958003e60 con 0x5631de2522f0 2024-05-27T08:04:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.182+0000 7f2952600640 1 -- 172.21.15.90:0/2405448372 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2958055070 con 0x5631de2522f0 2024-05-27T08:04:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.182+0000 7f2952600640 1 -- 172.21.15.90:0/2405448372 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295803c910 con 0x5631de2522f0 2024-05-27T08:04:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 -- 172.21.15.90:0/2405448372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de2522f0 msgr2=0x5631de623520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 --2- 172.21.15.90:0/2405448372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de2522f0 0x5631de623520 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f29580036a0 tx=0x7f2958041000 comp rx=0 tx=0).stop 2024-05-27T08:04:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 -- 172.21.15.90:0/2405448372 shutdown_connections 2024-05-27T08:04:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 --2- 172.21.15.90:0/2405448372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de27b1b0 0x5631de62a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 --2- 172.21.15.90:0/2405448372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de25a4d0 0x5631de629b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 --2- 172.21.15.90:0/2405448372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de2522f0 0x5631de623520 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 -- 172.21.15.90:0/2405448372 >> 172.21.15.90:0/2405448372 conn(0x5631de26f300 msgr2=0x5631de629430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 -- 172.21.15.90:0/2405448372 shutdown_connections 2024-05-27T08:04:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 -- 172.21.15.90:0/2405448372 wait complete. 2024-05-27T08:04:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 -- start start 2024-05-27T08:04:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.183+0000 7f29739cf740 1 Processor -- start 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f29739cf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 0x5631de62cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f29739cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de25a4d0 0x5631de62d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f29739cf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de62d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f29739cf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f29681b7d10 con 0x5631de27b1b0 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f29739cf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f29681b68a0 con 0x5631de2522f0 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f29739cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f29681b6f70 con 0x5631de25a4d0 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f29739cf740 10 client.0 _ll_register_callbacks cb 0x7f295400a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f2966800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de62d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f2966800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de62d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39236/0 (socket says 172.21.15.90:39236) 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f2966800640 1 -- 172.21.15.90:0/3992904158 learned_addr learned my addr 172.21.15.90:0/3992904158 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f2965400640 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de25a4d0 0x5631de62d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f2965e00640 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 0x5631de62cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f2966800640 1 -- 172.21.15.90:0/3992904158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de25a4d0 msgr2=0x5631de62d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f2966800640 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de25a4d0 0x5631de62d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f2966800640 1 -- 172.21.15.90:0/3992904158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 msgr2=0x5631de62cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f2966800640 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 0x5631de62cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.184+0000 7f2966800640 1 -- 172.21.15.90:0/3992904158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631de6a0770 con 0x5631de27b1b0 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.185+0000 7f2966800640 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de62d780 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f29600e01e0 tx=0x7f29600a9820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e96c66c7f47017 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.185+0000 7f2947400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.185+0000 7f2947400640 1 -- 172.21.15.90:0/3992904158 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296001d040 con 0x5631de27b1b0 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.185+0000 7f29739cf740 1 -- 172.21.15.90:0/3992904158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2958055070 con 0x5631de27b1b0 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.185+0000 7f2947400640 1 -- 172.21.15.90:0/3992904158 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f29600b28f0 con 0x5631de27b1b0 2024-05-27T08:04:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.185+0000 7f2947400640 1 -- 172.21.15.90:0/3992904158 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f29600c43e0 con 0x5631de27b1b0 2024-05-27T08:04:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.185+0000 7f29739cf740 10 client.18363 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.185+0000 7f29739cf740 1 -- 172.21.15.90:0/3992904158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5631de6a1a40 con 0x5631de27b1b0 2024-05-27T08:04:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.185+0000 7f2947400640 1 -- 172.21.15.90:0/3992904158 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2958055070 con 0x5631de27b1b0 2024-05-27T08:04:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.185+0000 7f2947400640 1 -- 172.21.15.90:0/3992904158 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f29600a89e0 con 0x5631de27b1b0 2024-05-27T08:04:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.186+0000 7f29739cf740 10 client.18363 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.186+0000 7f29739cf740 10 client.18363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.186+0000 7f29739cf740 20 client.18363 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.186+0000 7f29739cf740 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631de20f1c0 0x5631de6a4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.186+0000 7f29739cf740 4 client.18363 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:04:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.186+0000 7f29739cf740 1 -- 172.21.15.90:0/3992904158 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631de58a2c0 con 0x5631de20f1c0 2024-05-27T08:04:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.186+0000 7f2965e00640 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631de20f1c0 0x5631de6a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.188+0000 7f2965e00640 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631de20f1c0 0x5631de6a4200 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe253ab803d2f803 server_cookie=f2be068fe84d60c1 in_seq=0 out_seq=0 2024-05-27T08:04:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.188+0000 7f2947400640 10 client.18363 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:23.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.310+0000 7f2947400640 1 -- 172.21.15.90:0/3992904158 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5631de58a2c0 con 0x5631de20f1c0 2024-05-27T08:04:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.310+0000 7f2947400640 10 client.18363 handle_command_reply: tid=0 2024-05-27T08:04:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.310+0000 7f29739cf740 1 client.18363 shutdown 2024-05-27T08:04:23.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.311+0000 7f29739cf740 1 -- 172.21.15.90:0/3992904158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 msgr2=0x5631de62d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.311+0000 7f29739cf740 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de62d780 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f29600e01e0 tx=0x7f29600a9820 comp rx=0 tx=0).stop 2024-05-27T08:04:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.311+0000 7f29739cf740 1 -- 172.21.15.90:0/3992904158 shutdown_connections 2024-05-27T08:04:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.311+0000 7f29739cf740 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5631de20f1c0 0x5631de6a4200 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.311+0000 7f29739cf740 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5631de27b1b0 0x5631de62d780 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.311+0000 7f29739cf740 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631de25a4d0 0x5631de62d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.311+0000 7f29739cf740 1 --2- 172.21.15.90:0/3992904158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5631de2522f0 0x5631de62cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.311+0000 7f2947400640 0 client.18363 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:23.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.311+0000 7f29739cf740 1 -- 172.21.15.90:0/3992904158 >> 172.21.15.90:0/3992904158 conn(0x5631de26f300 msgr2=0x5631de63e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.312+0000 7f29739cf740 1 -- 172.21.15.90:0/3992904158 shutdown_connections 2024-05-27T08:04:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.312+0000 7f29739cf740 1 -- 172.21.15.90:0/3992904158 wait complete. 2024-05-27T08:04:23.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.313+0000 7f29739cf740 20 client.18363 trim_cache size 0 max 16384 2024-05-27T08:04:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:04:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:04:23.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:04:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.320+0000 7f2967200640 1 -- 172.21.15.90:0/4259747211 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f294c0b29d0 msgr2=0x7f294c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.321+0000 7f2967200640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f294c0b29d0 0x7f294c0b4ee0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f29681a1c00 tx=0x7f2958005a20 comp rx=0 tx=0).stop 2024-05-27T08:04:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.321+0000 7f2967200640 1 -- 172.21.15.90:0/4259747211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2968114a70 msgr2=0x7f29681a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:23.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.321+0000 7f2967200640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2968114a70 0x7f29681a0ac0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f296000cad0 tx=0x7f296000cf90 comp rx=0 tx=0).stop 2024-05-27T08:04:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.321+0000 7f2967200640 1 -- 172.21.15.90:0/4259747211 shutdown_connections 2024-05-27T08:04:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.321+0000 7f2967200640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f294c0b29d0 0x7f294c0b4ee0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.321+0000 7f2967200640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2968112110 0x7f29681a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.321+0000 7f2967200640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2968114a70 0x7f29681a0ac0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.321+0000 7f2967200640 1 --2- 172.21.15.90:0/4259747211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2968075480 0x7f29681a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.321+0000 7f2967200640 1 -- 172.21.15.90:0/4259747211 >> 172.21.15.90:0/4259747211 conn(0x7f2968079140 msgr2=0x7f2968112520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.322+0000 7f2967200640 1 -- 172.21.15.90:0/4259747211 shutdown_connections 2024-05-27T08:04:23.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:23.322+0000 7f2967200640 1 -- 172.21.15.90:0/4259747211 wait complete. 2024-05-27T08:04:23.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:23 smithi136 ceph-mon[35324]: pgmap v581: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.4 MiB/s wr, 633 op/s 2024-05-27T08:04:23.793 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:23 smithi090 ceph-mon[31429]: pgmap v581: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.4 MiB/s wr, 633 op/s 2024-05-27T08:04:23.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:23 smithi129 ceph-mon[35254]: pgmap v581: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.4 MiB/s wr, 633 op/s 2024-05-27T08:04:23.829 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 845.0 s, 485.0 tps, lat 209.372 ms stddev 166.217 2024-05-27T08:04:24.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.172+0000 7f316c400700 1 -- 172.21.15.90:0/2355491404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3164111240 msgr2=0x7f316410ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:24.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.172+0000 7f316c400700 1 --2- 172.21.15.90:0/2355491404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3164111240 0x7f316410ae80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3150009990 tx=0x7f3150009ca0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.172+0000 7f316c400700 1 -- 172.21.15.90:0/2355491404 shutdown_connections 2024-05-27T08:04:24.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.172+0000 7f316c400700 1 --2- 172.21.15.90:0/2355491404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f316410b4c0 0x7f316407c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.172+0000 7f316c400700 1 --2- 172.21.15.90:0/2355491404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3164110da0 0x7f316410a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.172+0000 7f316c400700 1 --2- 172.21.15.90:0/2355491404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3164111240 0x7f316410ae80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.172+0000 7f316c400700 1 -- 172.21.15.90:0/2355491404 >> 172.21.15.90:0/2355491404 conn(0x7f316406f4c0 msgr2=0x7f316406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:24.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.172+0000 7f316c400700 1 -- 172.21.15.90:0/2355491404 shutdown_connections 2024-05-27T08:04:24.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.172+0000 7f316c400700 1 -- 172.21.15.90:0/2355491404 wait complete. 2024-05-27T08:04:24.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.173+0000 7f316c400700 1 Processor -- start 2024-05-27T08:04:24.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.173+0000 7f316c400700 1 -- start start 2024-05-27T08:04:24.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.174+0000 7f316c400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f316410b4c0 0x7f3164077b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.174+0000 7f316c400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3164110da0 0x7f3164078120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.174+0000 7f316c400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3164111240 0x7f31641c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.174+0000 7f316c400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3164085210 con 0x7f3164111240 2024-05-27T08:04:24.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.174+0000 7f316c400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3164085090 con 0x7f316410b4c0 2024-05-27T08:04:24.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.174+0000 7f316c400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3164085390 con 0x7f3164110da0 2024-05-27T08:04:24.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.175+0000 7f3169000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3164110da0 0x7f3164078120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.175+0000 7f3169000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3164110da0 0x7f3164078120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56594/0 (socket says 172.21.15.90:56594) 2024-05-27T08:04:24.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.175+0000 7f3169000700 1 -- 172.21.15.90:0/3727689167 learned_addr learned my addr 172.21.15.90:0/3727689167 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:24.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.175+0000 7f3169000700 1 -- 172.21.15.90:0/3727689167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f316410b4c0 msgr2=0x7f3164077b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:24.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.175+0000 7f3169000700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f316410b4c0 0x7f3164077b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.175+0000 7f3169000700 1 -- 172.21.15.90:0/3727689167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3164111240 msgr2=0x7f31641c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:24.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.175+0000 7f3169000700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3164111240 0x7f31641c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.176+0000 7f3169000700 1 -- 172.21.15.90:0/3727689167 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31641c2ab0 con 0x7f3164110da0 2024-05-27T08:04:24.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.176+0000 7f3169000700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3164110da0 0x7f3164078120 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3150006010 tx=0x7f31500057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:24.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.177+0000 7f3162000700 1 -- 172.21.15.90:0/3727689167 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150026070 con 0x7f3164110da0 2024-05-27T08:04:24.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.177+0000 7f316c400700 1 -- 172.21.15.90:0/3727689167 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31641c2ce0 con 0x7f3164110da0 2024-05-27T08:04:24.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.177+0000 7f316c400700 1 -- 172.21.15.90:0/3727689167 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31641c3190 con 0x7f3164110da0 2024-05-27T08:04:24.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.177+0000 7f3162000700 1 -- 172.21.15.90:0/3727689167 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3150021410 con 0x7f3164110da0 2024-05-27T08:04:24.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.177+0000 7f3162000700 1 -- 172.21.15.90:0/3727689167 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150017650 con 0x7f3164110da0 2024-05-27T08:04:24.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.185+0000 7f3162000700 1 -- 172.21.15.90:0/3727689167 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3150029070 con 0x7f3164110da0 2024-05-27T08:04:24.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.192+0000 7f3162000700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f314c0b2a70 0x7f314c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.192+0000 7f3162000700 1 -- 172.21.15.90:0/3727689167 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f31500e47a0 con 0x7f3164110da0 2024-05-27T08:04:24.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.192+0000 7f3169a00700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f314c0b2a70 0x7f314c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.213+0000 7f317affdb80 1 Processor -- start 2024-05-27T08:04:24.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.213+0000 7f3169a00700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f314c0b2a70 0x7f314c0b4f80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f31640799d0 tx=0x7f3158009250 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:24.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.214+0000 7f317affdb80 1 -- start start 2024-05-27T08:04:24.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.215+0000 7f317affdb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462269f0 0x559d462b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.215+0000 7f317affdb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 0x559d462b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.216+0000 7f317affdb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.216+0000 7f317affdb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559d461e7e40 con 0x559d462b9db0 2024-05-27T08:04:24.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.216+0000 7f317affdb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559d461d79e0 con 0x559d462269f0 2024-05-27T08:04:24.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.216+0000 7f3169a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462269f0 0x559d462b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.216+0000 7f3169a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462269f0 0x559d462b9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43790/0 (socket says 172.21.15.90:43790) 2024-05-27T08:04:24.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.216+0000 7f3169a00700 1 -- 172.21.15.90:0/1672010239 learned_addr learned my addr 172.21.15.90:0/1672010239 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:24.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.216+0000 7f317affdb80 1 -- 172.21.15.90:0/1672010239 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d461dba20 con 0x559d4613ae40 2024-05-27T08:04:24.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.218+0000 7f316a400700 1 --2- 172.21.15.90:0/1672010239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.218+0000 7f3169000700 1 --2- 172.21.15.90:0/1672010239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 0x559d462b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.218+0000 7f3169a00700 1 -- 172.21.15.90:0/1672010239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 msgr2=0x559d462b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:24.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.218+0000 7f3169a00700 1 --2- 172.21.15.90:0/1672010239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 0x559d462b97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.218+0000 7f3169a00700 1 -- 172.21.15.90:0/1672010239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 msgr2=0x559d462ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:24.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.218+0000 7f3169a00700 1 --2- 172.21.15.90:0/1672010239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.218+0000 7f3169a00700 1 -- 172.21.15.90:0/1672010239 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d461a0420 con 0x559d462269f0 2024-05-27T08:04:24.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.219+0000 7f316a400700 1 --2- 172.21.15.90:0/1672010239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:24.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.219+0000 7f3169a00700 1 --2- 172.21.15.90:0/1672010239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462269f0 0x559d462b9250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f315800cd90 tx=0x7f315800f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeff9156dd4248d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:24.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.220+0000 7f3156a00700 1 -- 172.21.15.90:0/1672010239 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158019420 con 0x559d462269f0 2024-05-27T08:04:24.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.222+0000 7f3156a00700 1 -- 172.21.15.90:0/1672010239 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3158019a50 con 0x559d462269f0 2024-05-27T08:04:24.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.222+0000 7f3156a00700 1 -- 172.21.15.90:0/1672010239 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158017870 con 0x559d462269f0 2024-05-27T08:04:24.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.224+0000 7f317affdb80 1 -- 172.21.15.90:0/1672010239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462269f0 msgr2=0x559d462b9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:24.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.224+0000 7f317affdb80 1 --2- 172.21.15.90:0/1672010239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462269f0 0x559d462b9250 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f315800cd90 tx=0x7f315800f000 comp rx=0 tx=0).stop 2024-05-27T08:04:24.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.224+0000 7f317affdb80 1 -- 172.21.15.90:0/1672010239 shutdown_connections 2024-05-27T08:04:24.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.224+0000 7f317affdb80 1 --2- 172.21.15.90:0/1672010239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.224+0000 7f317affdb80 1 --2- 172.21.15.90:0/1672010239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462269f0 0x559d462b9250 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.224+0000 7f317affdb80 1 --2- 172.21.15.90:0/1672010239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 0x559d462b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.224+0000 7f317affdb80 1 -- 172.21.15.90:0/1672010239 >> 172.21.15.90:0/1672010239 conn(0x559d461d4390 msgr2=0x559d462b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:24.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.225+0000 7f317affdb80 1 -- 172.21.15.90:0/1672010239 shutdown_connections 2024-05-27T08:04:24.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.225+0000 7f317affdb80 1 -- 172.21.15.90:0/1672010239 wait complete. 2024-05-27T08:04:24.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.226+0000 7f317affdb80 1 -- start start 2024-05-27T08:04:24.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.226+0000 7f317affdb80 1 Processor -- start 2024-05-27T08:04:24.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.227+0000 7f317affdb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 0x559d462c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.227+0000 7f317affdb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462269f0 0x559d462c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.227+0000 7f317affdb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.227+0000 7f317affdb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559d460ddfa0 con 0x559d462b9db0 2024-05-27T08:04:24.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.227+0000 7f317affdb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559d46178760 con 0x559d4613ae40 2024-05-27T08:04:24.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.227+0000 7f317affdb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d462cf3e0 con 0x559d462269f0 2024-05-27T08:04:24.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.227+0000 7f317affdb80 10 client.0 _ll_register_callbacks cb 0x559d46181c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:24.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.228+0000 7f316a400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.228+0000 7f316a400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39274/0 (socket says 172.21.15.90:39274) 2024-05-27T08:04:24.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.228+0000 7f316a400700 1 -- 172.21.15.90:0/1711407663 learned_addr learned my addr 172.21.15.90:0/1711407663 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:24.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.228+0000 7f3169a00700 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 0x559d462c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.229+0000 7f3169000700 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462269f0 0x559d462c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.229+0000 7f316a400700 1 -- 172.21.15.90:0/1711407663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462269f0 msgr2=0x559d462c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:24.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.229+0000 7f316a400700 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462269f0 0x559d462c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.229+0000 7f316a400700 1 -- 172.21.15.90:0/1711407663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 msgr2=0x559d462c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:24.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.229+0000 7f316a400700 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 0x559d462c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.229+0000 7f316a400700 1 -- 172.21.15.90:0/1711407663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d461278d0 con 0x559d462b9db0 2024-05-27T08:04:24.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.229+0000 7f3169000700 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462269f0 0x559d462c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:24.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.230+0000 7f316a400700 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ca420 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f315c010620 tx=0x7f315c010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449579c8d0881a67 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:24.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.230+0000 7f314b400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:04:24.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.231+0000 7f314b400700 1 -- 172.21.15.90:0/1711407663 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c012860 con 0x559d462b9db0 2024-05-27T08:04:24.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.231+0000 7f317affdb80 1 -- 172.21.15.90:0/1711407663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3158019a50 con 0x559d462b9db0 2024-05-27T08:04:24.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.232+0000 7f314b400700 1 -- 172.21.15.90:0/1711407663 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f315c012a00 con 0x559d462b9db0 2024-05-27T08:04:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.232+0000 7f314b400700 1 -- 172.21.15.90:0/1711407663 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f315c012c20 con 0x559d462b9db0 2024-05-27T08:04:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.232+0000 7f317affdb80 10 client.18366 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:24.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.232+0000 7f317affdb80 1 -- 172.21.15.90:0/1711407663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x559d461a8060 con 0x559d462b9db0 2024-05-27T08:04:24.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.233+0000 7f314b400700 1 -- 172.21.15.90:0/1711407663 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3158019a50 con 0x559d462b9db0 2024-05-27T08:04:24.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.234+0000 7f314b400700 1 -- 172.21.15.90:0/1711407663 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f315c00f040 con 0x559d462b9db0 2024-05-27T08:04:24.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.235+0000 7f317affdb80 10 client.18366 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:24.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.235+0000 7f317affdb80 10 client.18366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:24.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.235+0000 7f317affdb80 20 client.18366 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.235+0000 7f317affdb80 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559d462dc5e0 0x559d462dea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.235+0000 7f317affdb80 4 client.18366 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.235+0000 7f317affdb80 1 -- 172.21.15.90:0/1711407663 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d461e9d90 con 0x559d462dc5e0 2024-05-27T08:04:24.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.236+0000 7f3169a00700 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559d462dc5e0 0x559d462dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.237+0000 7f3169a00700 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559d462dc5e0 0x559d462dea50 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbe453f4e6fa15dd server_cookie=2bc95f2233e4960a in_seq=0 out_seq=0 2024-05-27T08:04:24.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.237+0000 7f314b400700 10 client.18366 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:24.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.238+0000 7f314b400700 1 -- 172.21.15.90:0/1711407663 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x559d461e9d90 con 0x559d462dc5e0 2024-05-27T08:04:24.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.238+0000 7f314b400700 10 client.18366 handle_command_reply: tid=0 2024-05-27T08:04:24.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.239+0000 7f317affdb80 1 client.18366 shutdown 2024-05-27T08:04:24.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.239+0000 7f317affdb80 1 -- 172.21.15.90:0/1711407663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 msgr2=0x559d462ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:24.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.239+0000 7f317affdb80 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ca420 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f315c010620 tx=0x7f315c010930 comp rx=0 tx=0).stop 2024-05-27T08:04:24.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.239+0000 7f317affdb80 1 -- 172.21.15.90:0/1711407663 shutdown_connections 2024-05-27T08:04:24.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.239+0000 7f317affdb80 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559d462dc5e0 0x559d462dea50 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.239+0000 7f317affdb80 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462b9db0 0x559d462ca420 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.239+0000 7f317affdb80 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 0x559d462c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.239+0000 7f317affdb80 1 --2- 172.21.15.90:0/1711407663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462269f0 0x559d462c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.239+0000 7f317affdb80 1 -- 172.21.15.90:0/1711407663 >> 172.21.15.90:0/1711407663 conn(0x559d461d4390 msgr2=0x559d462b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:24.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.240+0000 7f314b400700 0 client.18366 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:24.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.240+0000 7f317affdb80 1 -- 172.21.15.90:0/1711407663 shutdown_connections 2024-05-27T08:04:24.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.241+0000 7f317affdb80 1 -- 172.21.15.90:0/1711407663 wait complete. 2024-05-27T08:04:24.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.241+0000 7f317affdb80 20 client.18366 trim_cache size 0 max 16384 2024-05-27T08:04:24.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.281+0000 7f317affdb80 1 Processor -- start 2024-05-27T08:04:24.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.282+0000 7f317affdb80 1 -- start start 2024-05-27T08:04:24.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.283+0000 7f317affdb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 0x559d462c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.283+0000 7f317affdb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 0x559d462d84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.284+0000 7f317affdb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462b9db0 0x559d462e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.284+0000 7f317affdb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f31440107c0 con 0x559d462269f0 2024-05-27T08:04:24.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.284+0000 7f317affdb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f314400d810 con 0x559d462b9db0 2024-05-27T08:04:24.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.284+0000 7f317affdb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f314400fa20 con 0x559d4613ae40 2024-05-27T08:04:24.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.284+0000 7f3169a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 0x559d462c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.284+0000 7f3169a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 0x559d462c4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:56608/0 (socket says 172.21.15.90:56608) 2024-05-27T08:04:24.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.284+0000 7f3169a00700 1 -- 172.21.15.90:0/1526899818 learned_addr learned my addr 172.21.15.90:0/1526899818 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:24.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.284+0000 7f3169000700 1 --2- 172.21.15.90:0/1526899818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 0x559d462d84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.286+0000 7f316a400700 1 --2- 172.21.15.90:0/1526899818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462b9db0 0x559d462e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.286+0000 7f3169a00700 1 -- 172.21.15.90:0/1526899818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462b9db0 msgr2=0x559d462e4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:24.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.286+0000 7f3169a00700 1 --2- 172.21.15.90:0/1526899818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462b9db0 0x559d462e4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.286+0000 7f3169a00700 1 -- 172.21.15.90:0/1526899818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 msgr2=0x559d462d84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:24.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.286+0000 7f3169a00700 1 --2- 172.21.15.90:0/1526899818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 0x559d462d84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.286+0000 7f3169a00700 1 -- 172.21.15.90:0/1526899818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d462ca790 con 0x559d4613ae40 2024-05-27T08:04:24.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.287+0000 7f3169a00700 1 --2- 172.21.15.90:0/1526899818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 0x559d462c4fc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f315800cd30 tx=0x7f31580173b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79e638380b52ad78 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:24.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.287+0000 7f3156000700 1 -- 172.21.15.90:0/1526899818 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31580199e0 con 0x559d4613ae40 2024-05-27T08:04:24.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.288+0000 7f3156000700 1 -- 172.21.15.90:0/1526899818 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3158017950 con 0x559d4613ae40 2024-05-27T08:04:24.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.288+0000 7f3156000700 1 -- 172.21.15.90:0/1526899818 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315802d680 con 0x559d4613ae40 2024-05-27T08:04:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.289+0000 7f317affdb80 1 -- 172.21.15.90:0/1526899818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 msgr2=0x559d462c4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.289+0000 7f317affdb80 1 --2- 172.21.15.90:0/1526899818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 0x559d462c4fc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f315800cd30 tx=0x7f31580173b0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.289+0000 7f317affdb80 1 -- 172.21.15.90:0/1526899818 shutdown_connections 2024-05-27T08:04:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.289+0000 7f317affdb80 1 --2- 172.21.15.90:0/1526899818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d462b9db0 0x559d462e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.289+0000 7f317affdb80 1 --2- 172.21.15.90:0/1526899818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d4613ae40 0x559d462c4fc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.289+0000 7f317affdb80 1 --2- 172.21.15.90:0/1526899818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 0x559d462d84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.289+0000 7f317affdb80 1 -- 172.21.15.90:0/1526899818 >> 172.21.15.90:0/1526899818 conn(0x559d461d4390 msgr2=0x559d462c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.290+0000 7f317affdb80 1 -- 172.21.15.90:0/1526899818 shutdown_connections 2024-05-27T08:04:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.290+0000 7f317affdb80 1 -- 172.21.15.90:0/1526899818 wait complete. 2024-05-27T08:04:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.290+0000 7f317affdb80 1 -- start start 2024-05-27T08:04:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.291+0000 7f317affdb80 1 Processor -- start 2024-05-27T08:04:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.292+0000 7f317affdb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 0x559d4633f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.292+0000 7f317affdb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 0x559d4633f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.292+0000 7f317affdb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462b9db0 0x559d462cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.292+0000 7f317affdb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f314400f350 con 0x559d462269f0 2024-05-27T08:04:24.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.292+0000 7f317affdb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f314400dee0 con 0x559d4613ae40 2024-05-27T08:04:24.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.292+0000 7f317affdb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f314400e5b0 con 0x559d462b9db0 2024-05-27T08:04:24.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.292+0000 7f317affdb80 10 client.0 _ll_register_callbacks cb 0x559d46181c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:24.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.292+0000 7f3169a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 0x559d4633f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.292+0000 7f3169a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 0x559d4633f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:43804/0 (socket says 172.21.15.90:43804) 2024-05-27T08:04:24.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.292+0000 7f3169a00700 1 -- 172.21.15.90:0/3018923210 learned_addr learned my addr 172.21.15.90:0/3018923210 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:24.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.293+0000 7f316a400700 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462b9db0 0x559d462cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.294+0000 7f3169a00700 1 -- 172.21.15.90:0/3018923210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462b9db0 msgr2=0x559d462cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.294+0000 7f3169a00700 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462b9db0 0x559d462cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.294+0000 7f3169a00700 1 -- 172.21.15.90:0/3018923210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 msgr2=0x559d4633f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.294+0000 7f3169000700 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 0x559d4633f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.295+0000 7f3169a00700 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 0x559d4633f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.295+0000 7f3169a00700 1 -- 172.21.15.90:0/3018923210 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d462d02b0 con 0x559d4613ae40 2024-05-27T08:04:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.295+0000 7f3169000700 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 0x559d4633f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.296+0000 7f3169a00700 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 0x559d4633f430 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3158009e80 tx=0x7f3158019520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cda2252f6a6b76 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.296+0000 7f314b400700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:04:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.296+0000 7f314b400700 1 -- 172.21.15.90:0/3018923210 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158017650 con 0x559d4613ae40 2024-05-27T08:04:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.297+0000 7f314b400700 1 -- 172.21.15.90:0/3018923210 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f3158017b00 con 0x559d4613ae40 2024-05-27T08:04:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.298+0000 7f317affdb80 1 -- 172.21.15.90:0/3018923210 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3158017950 con 0x559d4613ae40 2024-05-27T08:04:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.298+0000 7f314b400700 1 -- 172.21.15.90:0/3018923210 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f315804b050 con 0x559d4613ae40 2024-05-27T08:04:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.298+0000 7f317affdb80 10 client.38072 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.298+0000 7f317affdb80 1 -- 172.21.15.90:0/3018923210 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x559d46346af0 con 0x559d4613ae40 2024-05-27T08:04:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.299+0000 7f314b400700 1 -- 172.21.15.90:0/3018923210 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3158017950 con 0x559d4613ae40 2024-05-27T08:04:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.299+0000 7f314b400700 1 -- 172.21.15.90:0/3018923210 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f315802d420 con 0x559d4613ae40 2024-05-27T08:04:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.299+0000 7f317affdb80 10 client.38072 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:24.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.299+0000 7f317affdb80 10 client.38072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:24.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.299+0000 7f317affdb80 20 client.38072 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:24.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.299+0000 7f317affdb80 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559d46347070 0x559d46349490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:24.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.299+0000 7f317affdb80 4 client.38072 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:04:24.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.299+0000 7f317affdb80 1 -- 172.21.15.90:0/3018923210 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d461e3570 con 0x559d46347070 2024-05-27T08:04:24.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.300+0000 7f3169000700 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559d46347070 0x559d46349490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:24.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.302+0000 7f3169000700 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559d46347070 0x559d46349490 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47068e7062c9cbd3 server_cookie=6cb8050917c6c75d in_seq=0 out_seq=0 2024-05-27T08:04:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f314b400700 10 client.38072 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f314b400700 1 -- 172.21.15.90:0/3018923210 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d461e3570 con 0x559d46347070 2024-05-27T08:04:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f314b400700 10 client.38072 handle_command_reply: tid=0 2024-05-27T08:04:24.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 client.38072 shutdown 2024-05-27T08:04:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 -- 172.21.15.90:0/3018923210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 msgr2=0x559d4633f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 0x559d4633f430 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3158009e80 tx=0x7f3158019520 comp rx=0 tx=0).stop 2024-05-27T08:04:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 -- 172.21.15.90:0/3018923210 shutdown_connections 2024-05-27T08:04:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x559d46347070 0x559d46349490 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d462b9db0 0x559d462cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d4613ae40 0x559d4633f430 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 --2- 172.21.15.90:0/3018923210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559d462269f0 0x559d4633f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 -- 172.21.15.90:0/3018923210 >> 172.21.15.90:0/3018923210 conn(0x559d461d4390 msgr2=0x559d462c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:24.308 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:04:24.308 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T08:04:24.308 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T08:04:24.309 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:04:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f314b400700 0 client.38072 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 -- 172.21.15.90:0/3018923210 shutdown_connections 2024-05-27T08:04:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 1 -- 172.21.15.90:0/3018923210 wait complete. 2024-05-27T08:04:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.304+0000 7f317affdb80 20 client.38072 trim_cache size 0 max 16384 2024-05-27T08:04:24.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 -- 172.21.15.90:0/3727689167 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f314c0b2a70 msgr2=0x7f314c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f314c0b2a70 0x7f314c0b4f80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f31640799d0 tx=0x7f3158009250 comp rx=0 tx=0).stop 2024-05-27T08:04:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 -- 172.21.15.90:0/3727689167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3164110da0 msgr2=0x7f3164078120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3164110da0 0x7f3164078120 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3150006010 tx=0x7f31500057c0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 -- 172.21.15.90:0/3727689167 shutdown_connections 2024-05-27T08:04:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3164111240 0x7f31641c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f314c0b2a70 0x7f314c0b4f80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f316410b4c0 0x7f3164077b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 --2- 172.21.15.90:0/3727689167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3164110da0 0x7f3164078120 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 -- 172.21.15.90:0/3727689167 >> 172.21.15.90:0/3727689167 conn(0x7f316406f4c0 msgr2=0x7f316407d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 -- 172.21.15.90:0/3727689167 shutdown_connections 2024-05-27T08:04:24.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:24.307+0000 7f316c400700 1 -- 172.21.15.90:0/3727689167 wait complete. 2024-05-27T08:04:25.318 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:04:25.318 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:04:26.110 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:26.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:25 smithi090 ceph-mon[31429]: pgmap v582: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.6 MiB/s wr, 657 op/s 2024-05-27T08:04:26.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:25 smithi136 ceph-mon[35324]: pgmap v582: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.6 MiB/s wr, 657 op/s 2024-05-27T08:04:26.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:25 smithi129 ceph-mon[35254]: pgmap v582: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.6 MiB/s wr, 657 op/s 2024-05-27T08:04:28.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:27 smithi090 ceph-mon[31429]: pgmap v583: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.2 MiB/s wr, 566 op/s 2024-05-27T08:04:28.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:27 smithi136 ceph-mon[35324]: pgmap v583: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.2 MiB/s wr, 566 op/s 2024-05-27T08:04:28.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:27 smithi129 ceph-mon[35254]: pgmap v583: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.2 MiB/s wr, 566 op/s 2024-05-27T08:04:28.830 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 850.0 s, 489.1 tps, lat 183.445 ms stddev 75.466 2024-05-27T08:04:29.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:29 smithi090 ceph-mon[31429]: pgmap v584: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.6 MiB/s wr, 550 op/s 2024-05-27T08:04:30.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:29 smithi136 ceph-mon[35324]: pgmap v584: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.6 MiB/s wr, 550 op/s 2024-05-27T08:04:30.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:29 smithi129 ceph-mon[35254]: pgmap v584: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 5.6 MiB/s wr, 550 op/s 2024-05-27T08:04:31.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:31 smithi090 ceph-mon[31429]: pgmap v585: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.4 MiB/s wr, 592 op/s 2024-05-27T08:04:31.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.993+0000 7ffa05c00700 1 -- 172.21.15.90:0/3061805420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa0010b540 msgr2=0x7ffa00119890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:31.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.993+0000 7ffa05c00700 1 --2- 172.21.15.90:0/3061805420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa0010b540 0x7ffa00119890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009990 tx=0x7ff9f0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:04:31.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.993+0000 7ffa05c00700 1 -- 172.21.15.90:0/3061805420 shutdown_connections 2024-05-27T08:04:31.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.994+0000 7ffa05c00700 1 --2- 172.21.15.90:0/3061805420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa0010b540 0x7ffa00119890 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:31.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.994+0000 7ffa05c00700 1 --2- 172.21.15.90:0/3061805420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa00111bb0 0x7ffa00111fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:31.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.994+0000 7ffa05c00700 1 --2- 172.21.15.90:0/3061805420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa0010abf0 0x7ffa0010b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:31.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.994+0000 7ffa05c00700 1 -- 172.21.15.90:0/3061805420 >> 172.21.15.90:0/3061805420 conn(0x7ffa0006f980 msgr2=0x7ffa0006fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:31.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.994+0000 7ffa05c00700 1 -- 172.21.15.90:0/3061805420 shutdown_connections 2024-05-27T08:04:31.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.994+0000 7ffa05c00700 1 -- 172.21.15.90:0/3061805420 wait complete. 2024-05-27T08:04:31.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.995+0000 7ffa05c00700 1 Processor -- start 2024-05-27T08:04:31.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.996+0000 7ffa05c00700 1 -- start start 2024-05-27T08:04:32.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.997+0000 7ffa05c00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa0010abf0 0x7ffa001af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:32.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.998+0000 7ffa05c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0010b540 0x7ffa001afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:32.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.998+0000 7ffa05c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa00111bb0 0x7ffa001a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:32.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.998+0000 7ff9fea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa0010abf0 0x7ffa001af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.999+0000 7ff9fe000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0010b540 0x7ffa001afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.999+0000 7ff9ff400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa00111bb0 0x7ffa001a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.999+0000 7ff9ff400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa00111bb0 0x7ffa001a99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:54192/0 (socket says 172.21.15.90:54192) 2024-05-27T08:04:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.999+0000 7ff9fea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa0010abf0 0x7ffa001af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54066/0 (socket says 172.21.15.90:54066) 2024-05-27T08:04:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.999+0000 7ff9fe000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0010b540 0x7ffa001afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:39482/0 (socket says 172.21.15.90:39482) 2024-05-27T08:04:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:31.999+0000 7ff9ff400700 1 -- 172.21.15.90:0/642365019 learned_addr learned my addr 172.21.15.90:0/642365019 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.000+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0011aa10 con 0x7ffa0010abf0 2024-05-27T08:04:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.000+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0011a890 con 0x7ffa00111bb0 2024-05-27T08:04:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.000+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0011ab90 con 0x7ffa0010b540 2024-05-27T08:04:32.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.001+0000 7ff9fe000700 1 -- 172.21.15.90:0/642365019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa00111bb0 msgr2=0x7ffa001a99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:32.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.001+0000 7ff9fe000700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa00111bb0 0x7ffa001a99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:32.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.001+0000 7ff9fe000700 1 -- 172.21.15.90:0/642365019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa0010abf0 msgr2=0x7ffa001af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:32.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.001+0000 7ff9fe000700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa0010abf0 0x7ffa001af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:32.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.001+0000 7ff9fe000700 1 -- 172.21.15.90:0/642365019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa001aa090 con 0x7ffa0010b540 2024-05-27T08:04:32.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.003+0000 7ff9fe000700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0010b540 0x7ffa001afd00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00da20 tx=0x7ff9ec00dd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:32.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.005+0000 7ff9f7400700 1 -- 172.21.15.90:0/642365019 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec01a070 con 0x7ffa0010b540 2024-05-27T08:04:32.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.005+0000 7ff9f7400700 1 -- 172.21.15.90:0/642365019 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff9ec00ce00 con 0x7ffa0010b540 2024-05-27T08:04:32.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.005+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa001aa380 con 0x7ffa0010b540 2024-05-27T08:04:32.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.005+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa001b6650 con 0x7ffa0010b540 2024-05-27T08:04:32.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.006+0000 7ff9f7400700 1 -- 172.21.15.90:0/642365019 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec015dc0 con 0x7ffa0010b540 2024-05-27T08:04:32.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.010+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e0005330 con 0x7ffa0010b540 2024-05-27T08:04:32.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.015+0000 7ff9f7400700 1 -- 172.21.15.90:0/642365019 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff9ec01d070 con 0x7ffa0010b540 2024-05-27T08:04:32.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.016+0000 7ff9f7400700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff9e40b2c00 0x7ff9e40b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:32.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.017+0000 7ff9f7400700 1 -- 172.21.15.90:0/642365019 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7ff9ec0e1b40 con 0x7ffa0010b540 2024-05-27T08:04:32.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.018+0000 7ff9fea00700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff9e40b2c00 0x7ff9e40b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:32.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.019+0000 7ff9fea00700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff9e40b2c00 0x7ff9e40b5110 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa001aab20 tx=0x7ff9f80094f0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:32.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.026+0000 7ff9f7400700 1 -- 172.21.15.90:0/642365019 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9ec0a90f0 con 0x7ffa0010b540 2024-05-27T08:04:32.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:31 smithi136 ceph-mon[35324]: pgmap v585: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.4 MiB/s wr, 592 op/s 2024-05-27T08:04:32.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:31 smithi129 ceph-mon[35254]: pgmap v585: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 6.4 MiB/s wr, 592 op/s 2024-05-27T08:04:32.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.562+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9e00058f0 con 0x7ffa0010b540 2024-05-27T08:04:32.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.564+0000 7ff9f7400700 1 -- 172.21.15.90:0/642365019 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+9456 (secure 0 0 0) 0x7ff9ec0b1170 con 0x7ffa0010b540 2024-05-27T08:04:32.572 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:04:32.572 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":64,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:56:32.161929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T08:04:32.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.571+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff9e40b2c00 msgr2=0x7ff9e40b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:32.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.571+0000 7ffa05c00700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff9e40b2c00 0x7ff9e40b5110 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa001aab20 tx=0x7ff9f80094f0 comp rx=0 tx=0).stop 2024-05-27T08:04:32.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.571+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0010b540 msgr2=0x7ffa001afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:32.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.571+0000 7ffa05c00700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0010b540 0x7ffa001afd00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00da20 tx=0x7ff9ec00dd30 comp rx=0 tx=0).stop 2024-05-27T08:04:32.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.571+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 shutdown_connections 2024-05-27T08:04:32.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.571+0000 7ffa05c00700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff9e40b2c00 0x7ff9e40b5110 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:32.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.571+0000 7ffa05c00700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa00111bb0 0x7ffa001a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:32.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.571+0000 7ffa05c00700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa0010abf0 0x7ffa001af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:32.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.571+0000 7ffa05c00700 1 --2- 172.21.15.90:0/642365019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa0010b540 0x7ffa001afd00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:32.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.571+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 >> 172.21.15.90:0/642365019 conn(0x7ffa0006f980 msgr2=0x7ffa00110420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:32.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.572+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 shutdown_connections 2024-05-27T08:04:32.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:32.572+0000 7ffa05c00700 1 -- 172.21.15.90:0/642365019 wait complete. 2024-05-27T08:04:32.585 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 64 2024-05-27T08:04:33.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:32 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/642365019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:04:33.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:32 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/642365019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:04:33.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:32 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/642365019' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:04:33.557 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T08:04:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 855.0 s, 524.3 tps, lat 210.848 ms stddev 143.150 2024-05-27T08:04:33.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:33 smithi090 ceph-mon[31429]: pgmap v586: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.6 MiB/s wr, 427 op/s 2024-05-27T08:04:34.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:33 smithi136 ceph-mon[35324]: pgmap v586: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.6 MiB/s wr, 427 op/s 2024-05-27T08:04:34.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:33 smithi129 ceph-mon[35254]: pgmap v586: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.6 MiB/s wr, 427 op/s 2024-05-27T08:04:34.338 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:36.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:35 smithi136 ceph-mon[35324]: pgmap v587: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 515 op/s 2024-05-27T08:04:36.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:35 smithi129 ceph-mon[35254]: pgmap v587: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 515 op/s 2024-05-27T08:04:36.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:35 smithi090 ceph-mon[31429]: pgmap v587: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 515 op/s 2024-05-27T08:04:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.566+0000 7fb6d8c00640 1 -- 172.21.15.90:0/2027875964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d41041f0 msgr2=0x7fb6d41105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.566+0000 7fb6d8c00640 1 --2- 172.21.15.90:0/2027875964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d41041f0 0x7fb6d41105c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009ca0 tx=0x7fb6cc02a8e0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.566+0000 7fb6d8c00640 1 -- 172.21.15.90:0/2027875964 shutdown_connections 2024-05-27T08:04:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.566+0000 7fb6d8c00640 1 --2- 172.21.15.90:0/2027875964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d4109f40 0x7fb6d4103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.566+0000 7fb6d8c00640 1 --2- 172.21.15.90:0/2027875964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d41041f0 0x7fb6d41105c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.566+0000 7fb6d8c00640 1 --2- 172.21.15.90:0/2027875964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6d4109aa0 0x7fb6d4103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.566+0000 7fb6d8c00640 1 -- 172.21.15.90:0/2027875964 >> 172.21.15.90:0/2027875964 conn(0x7fb6d40791a0 msgr2=0x7fb6d40795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.566+0000 7fb6d8c00640 1 -- 172.21.15.90:0/2027875964 shutdown_connections 2024-05-27T08:04:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.566+0000 7fb6d8c00640 1 -- 172.21.15.90:0/2027875964 wait complete. 2024-05-27T08:04:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.567+0000 7fb6d8c00640 1 Processor -- start 2024-05-27T08:04:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.567+0000 7fb6d8c00640 1 -- start start 2024-05-27T08:04:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.567+0000 7fb6d8c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d41041f0 0x7fb6d40760d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.567+0000 7fb6d8c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6d4109aa0 0x7fb6d4076610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.567+0000 7fb6d8c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d4109f40 0x7fb6d4076b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.567+0000 7fb6d8c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4113c90 con 0x7fb6d41041f0 2024-05-27T08:04:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.567+0000 7fb6d8c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4113b10 con 0x7fb6d4109f40 2024-05-27T08:04:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.567+0000 7fb6d8c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d4113e10 con 0x7fb6d4109aa0 2024-05-27T08:04:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.568+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d41041f0 0x7fb6d40760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.568+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d41041f0 0x7fb6d40760d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54082/0 (socket says 172.21.15.90:54082) 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.568+0000 7fb6d3400640 1 -- 172.21.15.90:0/3947343513 learned_addr learned my addr 172.21.15.90:0/3947343513 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.568+0000 7fb6d2a00640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6d4109aa0 0x7fb6d4076610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.568+0000 7fb6d3400640 1 -- 172.21.15.90:0/3947343513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6d4109aa0 msgr2=0x7fb6d4076610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.568+0000 7fb6d3400640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6d4109aa0 0x7fb6d4076610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.568+0000 7fb6d3400640 1 -- 172.21.15.90:0/3947343513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d4109f40 msgr2=0x7fb6d4076b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.568+0000 7fb6d3400640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d4109f40 0x7fb6d4076b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.568+0000 7fb6d3400640 1 -- 172.21.15.90:0/3947343513 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d41a8a90 con 0x7fb6d41041f0 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.568+0000 7fb6d3400640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d41041f0 0x7fb6d40760d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400b450 tx=0x7fb6c400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.569+0000 7fb6cbe00640 1 -- 172.21.15.90:0/3947343513 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4015020 con 0x7fb6d41041f0 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.569+0000 7fb6cbe00640 1 -- 172.21.15.90:0/3947343513 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6c4011d20 con 0x7fb6d41041f0 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.569+0000 7fb6cbe00640 1 -- 172.21.15.90:0/3947343513 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4010e10 con 0x7fb6d41041f0 2024-05-27T08:04:36.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.569+0000 7fb6d8c00640 1 -- 172.21.15.90:0/3947343513 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d41a8d80 con 0x7fb6d41041f0 2024-05-27T08:04:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.569+0000 7fb6d8c00640 1 -- 172.21.15.90:0/3947343513 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d41a9210 con 0x7fb6d41041f0 2024-05-27T08:04:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.572+0000 7fb6e1380740 1 Processor -- start 2024-05-27T08:04:36.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.572+0000 7fb6e1380740 1 -- start start 2024-05-27T08:04:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.572+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 0x561df56b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.572+0000 7fb6cbe00640 1 -- 172.21.15.90:0/3947343513 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb6c4018020 con 0x7fb6d41041f0 2024-05-27T08:04:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.572+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52e72f0 0x561df56b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.572+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 0x561df56b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.572+0000 7fb6e1380740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561df5623b90 con 0x561df52e72f0 2024-05-27T08:04:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.572+0000 7fb6e1380740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561df5613730 con 0x561df52ef4d0 2024-05-27T08:04:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.572+0000 7fb6e1380740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561df5617770 con 0x561df53101b0 2024-05-27T08:04:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.573+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52e72f0 0x561df56b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.573+0000 7fb6d3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52e72f0 0x561df56b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54094/0 (socket says 172.21.15.90:54094) 2024-05-27T08:04:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.573+0000 7fb6d3400640 1 -- 172.21.15.90:0/2622893441 learned_addr learned my addr 172.21.15.90:0/2622893441 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.573+0000 7fb6d3e00640 1 --2- 172.21.15.90:0/2622893441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 0x561df56b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.573+0000 7fb6d2a00640 1 --2- 172.21.15.90:0/2622893441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 0x561df56b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.573+0000 7fb6d3400640 1 -- 172.21.15.90:0/2622893441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 msgr2=0x561df56b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.573+0000 7fb6d3400640 1 --2- 172.21.15.90:0/2622893441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 0x561df56b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.573+0000 7fb6d3400640 1 -- 172.21.15.90:0/2622893441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 msgr2=0x561df56b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.573+0000 7fb6d3400640 1 --2- 172.21.15.90:0/2622893441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 0x561df56b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.573+0000 7fb6d3400640 1 -- 172.21.15.90:0/2622893441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df55ac8b0 con 0x561df52e72f0 2024-05-27T08:04:36.577 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.574+0000 7fb6d3400640 1 --2- 172.21.15.90:0/2622893441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52e72f0 0x561df56b2fa0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40df7d0 tx=0x7fb6c40e8b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea04b4630dae8fcc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.574+0000 7fb6bfe00640 1 -- 172.21.15.90:0/2622893441 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40f5070 con 0x561df52e72f0 2024-05-27T08:04:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.574+0000 7fb6bfe00640 1 -- 172.21.15.90:0/2622893441 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6c40e4070 con 0x561df52e72f0 2024-05-27T08:04:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.574+0000 7fb6bfe00640 1 -- 172.21.15.90:0/2622893441 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40ea940 con 0x561df52e72f0 2024-05-27T08:04:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.574+0000 7fb6cbe00640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6b80b29d0 0x7fb6b80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.574+0000 7fb6cbe00640 1 -- 172.21.15.90:0/3947343513 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fb6c40e2460 con 0x7fb6d41041f0 2024-05-27T08:04:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.574+0000 7fb6d3e00640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6b80b29d0 0x7fb6b80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.575+0000 7fb6e1380740 1 -- 172.21.15.90:0/2622893441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52e72f0 msgr2=0x561df56b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.575+0000 7fb6e1380740 1 --2- 172.21.15.90:0/2622893441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52e72f0 0x561df56b2fa0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40df7d0 tx=0x7fb6c40e8b10 comp rx=0 tx=0).stop 2024-05-27T08:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.575+0000 7fb6e1380740 1 -- 172.21.15.90:0/2622893441 shutdown_connections 2024-05-27T08:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.575+0000 7fb6e1380740 1 --2- 172.21.15.90:0/2622893441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 0x561df56b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.575+0000 7fb6e1380740 1 --2- 172.21.15.90:0/2622893441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52e72f0 0x561df56b2fa0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.575+0000 7fb6e1380740 1 --2- 172.21.15.90:0/2622893441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 0x561df56b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.575+0000 7fb6e1380740 1 -- 172.21.15.90:0/2622893441 >> 172.21.15.90:0/2622893441 conn(0x561df5304300 msgr2=0x561df56af210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.575+0000 7fb6d3e00640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6b80b29d0 0x7fb6b80b4ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009a60 tx=0x7fb6cc009950 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.575+0000 7fb6e1380740 1 -- 172.21.15.90:0/2622893441 shutdown_connections 2024-05-27T08:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.575+0000 7fb6e1380740 1 -- 172.21.15.90:0/2622893441 wait complete. 2024-05-27T08:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.576+0000 7fb6e1380740 1 -- start start 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.576+0000 7fb6e1380740 1 Processor -- start 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.576+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52e72f0 0x561df56bcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.576+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56bd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.576+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 0x561df56c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.576+0000 7fb6e1380740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561df52f9790 con 0x561df52ef4d0 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.576+0000 7fb6e1380740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561df55bbbc0 con 0x561df52e72f0 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.576+0000 7fb6e1380740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561df56bfb90 con 0x561df53101b0 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.576+0000 7fb6e1380740 10 client.0 _ll_register_callbacks cb 0x561df5599400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56bd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56bd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54110/0 (socket says 172.21.15.90:54110) 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d2a00640 1 -- 172.21.15.90:0/4272991573 learned_addr learned my addr 172.21.15.90:0/4272991573 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d3400640 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52e72f0 0x561df56bcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d3e00640 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 0x561df56c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d2a00640 1 -- 172.21.15.90:0/4272991573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 msgr2=0x561df56c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d2a00640 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 0x561df56c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d2a00640 1 -- 172.21.15.90:0/4272991573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52e72f0 msgr2=0x561df56bcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d2a00640 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52e72f0 0x561df56bcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d2a00640 1 -- 172.21.15.90:0/4272991573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df5328210 con 0x561df52ef4d0 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d3e00640 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 0x561df56c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6d2a00640 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56bd3e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000cfd0 tx=0x7fb6c000e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9d96bef5b40c69 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.577+0000 7fb6bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6bcc00640 1 -- 172.21.15.90:0/4272991573 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0018020 con 0x561df52ef4d0 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6bcc00640 1 -- 172.21.15.90:0/4272991573 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fb6c0013660 con 0x561df52ef4d0 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6e1380740 1 -- 172.21.15.90:0/4272991573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c40e4070 con 0x561df52ef4d0 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6bcc00640 1 -- 172.21.15.90:0/4272991573 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb6c0023050 con 0x561df52ef4d0 2024-05-27T08:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6e1380740 10 client.18387 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6e1380740 1 -- 172.21.15.90:0/4272991573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561df53f4620 con 0x561df52ef4d0 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6bcc00640 1 -- 172.21.15.90:0/4272991573 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6c40e4070 con 0x561df52ef4d0 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6bcc00640 1 -- 172.21.15.90:0/4272991573 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fb6c0014770 con 0x561df52ef4d0 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6e1380740 10 client.18387 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6e1380740 10 client.18387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6e1380740 20 client.18387 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6e1380740 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561df52ced80 0x561df56cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6e1380740 4 client.18387 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.578+0000 7fb6e1380740 1 -- 172.21.15.90:0/4272991573 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561df5625ae0 con 0x561df52ced80 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.579+0000 7fb6d3400640 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561df52ced80 0x561df56cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.580+0000 7fb6d3400640 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561df52ced80 0x561df56cf150 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb9f89e0bfc6115 server_cookie=8be866f05ddf3b5 in_seq=0 out_seq=0 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.580+0000 7fb6bcc00640 10 client.18387 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.582+0000 7fb6bcc00640 1 -- 172.21.15.90:0/4272991573 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561df5625ae0 con 0x561df52ced80 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.582+0000 7fb6bcc00640 10 client.18387 handle_command_reply: tid=0 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.582+0000 7fb6e1380740 1 client.18387 shutdown 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.582+0000 7fb6e1380740 1 -- 172.21.15.90:0/4272991573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 msgr2=0x561df56bd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.582+0000 7fb6e1380740 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56bd3e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000cfd0 tx=0x7fb6c000e660 comp rx=0 tx=0).stop 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.583+0000 7fb6e1380740 1 -- 172.21.15.90:0/4272991573 shutdown_connections 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.583+0000 7fb6e1380740 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561df52ced80 0x561df56cf150 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.583+0000 7fb6e1380740 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df53101b0 0x561df56c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.583+0000 7fb6e1380740 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56bd3e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.583+0000 7fb6e1380740 1 --2- 172.21.15.90:0/4272991573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52e72f0 0x561df56bcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.583+0000 7fb6bcc00640 0 client.18387 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.583+0000 7fb6e1380740 1 -- 172.21.15.90:0/4272991573 >> 172.21.15.90:0/4272991573 conn(0x561df5304300 msgr2=0x561df56b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.583+0000 7fb6e1380740 1 -- 172.21.15.90:0/4272991573 shutdown_connections 2024-05-27T08:04:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.583+0000 7fb6e1380740 1 -- 172.21.15.90:0/4272991573 wait complete. 2024-05-27T08:04:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.583+0000 7fb6e1380740 20 client.18387 trim_cache size 0 max 16384 2024-05-27T08:04:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.598+0000 7fb6e1380740 1 Processor -- start 2024-05-27T08:04:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.598+0000 7fb6e1380740 1 -- start start 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df52e72f0 0x561df56bbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df53101b0 0x561df56c9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6e1380740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b4dd0 con 0x561df52ef4d0 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6e1380740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b1e20 con 0x561df53101b0 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6e1380740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b4030 con 0x561df52e72f0 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6d2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54124/0 (socket says 172.21.15.90:54124) 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6d2a00640 1 -- 172.21.15.90:0/351417861 learned_addr learned my addr 172.21.15.90:0/351417861 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6d2a00640 1 -- 172.21.15.90:0/351417861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df52e72f0 msgr2=0x561df56bbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6d2a00640 1 --2- 172.21.15.90:0/351417861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df52e72f0 0x561df56bbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6d2a00640 1 -- 172.21.15.90:0/351417861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df53101b0 msgr2=0x561df56c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6d2a00640 1 --2- 172.21.15.90:0/351417861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df53101b0 0x561df56c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.599+0000 7fb6d2a00640 1 -- 172.21.15.90:0/351417861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df56b3e40 con 0x561df52ef4d0 2024-05-27T08:04:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.600+0000 7fb6d2a00640 1 --2- 172.21.15.90:0/351417861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56c8c30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0014d80 tx=0x7fb6c000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6d725157248b03 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.600+0000 7fb6bf400640 1 -- 172.21.15.90:0/351417861 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0018070 con 0x561df52ef4d0 2024-05-27T08:04:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.600+0000 7fb6bf400640 1 -- 172.21.15.90:0/351417861 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6c000c270 con 0x561df52ef4d0 2024-05-27T08:04:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.600+0000 7fb6bf400640 1 -- 172.21.15.90:0/351417861 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0013bc0 con 0x561df52ef4d0 2024-05-27T08:04:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.601+0000 7fb6e1380740 1 -- 172.21.15.90:0/351417861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 msgr2=0x561df56c8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.601+0000 7fb6e1380740 1 --2- 172.21.15.90:0/351417861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56c8c30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0014d80 tx=0x7fb6c000cbd0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.601+0000 7fb6e1380740 1 -- 172.21.15.90:0/351417861 shutdown_connections 2024-05-27T08:04:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.601+0000 7fb6e1380740 1 --2- 172.21.15.90:0/351417861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df53101b0 0x561df56c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.601+0000 7fb6e1380740 1 --2- 172.21.15.90:0/351417861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df52ef4d0 0x561df56c8c30 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.601+0000 7fb6e1380740 1 --2- 172.21.15.90:0/351417861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df52e72f0 0x561df56bbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.601+0000 7fb6e1380740 1 -- 172.21.15.90:0/351417861 >> 172.21.15.90:0/351417861 conn(0x561df5304300 msgr2=0x561df56b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.601+0000 7fb6e1380740 1 -- 172.21.15.90:0/351417861 shutdown_connections 2024-05-27T08:04:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.601+0000 7fb6e1380740 1 -- 172.21.15.90:0/351417861 wait complete. 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.601+0000 7fb6e1380740 1 -- start start 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6e1380740 1 Processor -- start 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df52e72f0 0x561df56c0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 0x561df56c0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6e1380740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df53101b0 0x561df572f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6e1380740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b3960 con 0x561df53101b0 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6e1380740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b24f0 con 0x561df52ef4d0 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6e1380740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d41b2bc0 con 0x561df52e72f0 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6e1380740 10 client.0 _ll_register_callbacks cb 0x7fb6c0006f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6d3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df53101b0 0x561df572f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6d3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df53101b0 0x561df572f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54134/0 (socket says 172.21.15.90:54134) 2024-05-27T08:04:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.602+0000 7fb6d3e00640 1 -- 172.21.15.90:0/1089890676 learned_addr learned my addr 172.21.15.90:0/1089890676 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6d2a00640 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 0x561df56c0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6d3400640 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df52e72f0 0x561df56c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6d3e00640 1 -- 172.21.15.90:0/1089890676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df52e72f0 msgr2=0x561df56c0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6d3e00640 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df52e72f0 0x561df56c0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6d3e00640 1 -- 172.21.15.90:0/1089890676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 msgr2=0x561df56c0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6d3e00640 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 0x561df56c0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6d3e00640 1 -- 172.21.15.90:0/1089890676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df572fde0 con 0x561df53101b0 2024-05-27T08:04:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6d2a00640 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 0x561df56c0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6d3400640 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df52e72f0 0x561df56c0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6d3e00640 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df53101b0 0x561df572f7a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009ca0 tx=0x7fb6cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287546edb532b5f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6bcc00640 1 -- 172.21.15.90:0/1089890676 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc0305e0 con 0x561df53101b0 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6e1380740 1 -- 172.21.15.90:0/1089890676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6c000c270 con 0x561df53101b0 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.603+0000 7fb6bcc00640 1 -- 172.21.15.90:0/1089890676 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fb6cc030780 con 0x561df53101b0 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6bcc00640 1 -- 172.21.15.90:0/1089890676 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb6cc05a050 con 0x561df53101b0 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6e1380740 10 client.18393 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6e1380740 1 -- 172.21.15.90:0/1089890676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561df5737160 con 0x561df53101b0 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6bcc00640 1 -- 172.21.15.90:0/1089890676 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6c000c270 con 0x561df53101b0 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6bcc00640 1 -- 172.21.15.90:0/1089890676 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fb6cc0518f0 con 0x561df53101b0 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6e1380740 10 client.18393 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6e1380740 10 client.18393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6e1380740 20 client.18393 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6e1380740 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561df52a41c0 0x561df5739900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6e1380740 4 client.18393 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:04:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.604+0000 7fb6e1380740 1 -- 172.21.15.90:0/1089890676 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561df561f2c0 con 0x561df52a41c0 2024-05-27T08:04:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.605+0000 7fb6d3400640 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561df52a41c0 0x561df5739900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.606+0000 7fb6bcc00640 10 client.18393 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.606+0000 7fb6d3400640 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561df52a41c0 0x561df5739900 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=765b1412519b067f server_cookie=1c22cf897243b8c2 in_seq=0 out_seq=0 2024-05-27T08:04:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.808+0000 7fb6bcc00640 1 -- 172.21.15.90:0/1089890676 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561df561f2c0 con 0x561df52a41c0 2024-05-27T08:04:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.808+0000 7fb6bcc00640 10 client.18393 handle_command_reply: tid=0 2024-05-27T08:04:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.808+0000 7fb6e1380740 1 client.18393 shutdown 2024-05-27T08:04:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.809+0000 7fb6e1380740 1 -- 172.21.15.90:0/1089890676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df53101b0 msgr2=0x561df572f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.809+0000 7fb6e1380740 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df53101b0 0x561df572f7a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009ca0 tx=0x7fb6cc005fd0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.809+0000 7fb6e1380740 1 -- 172.21.15.90:0/1089890676 shutdown_connections 2024-05-27T08:04:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.809+0000 7fb6e1380740 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561df52a41c0 0x561df5739900 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.809+0000 7fb6e1380740 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df52ef4d0 0x561df56c0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.809+0000 7fb6e1380740 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561df53101b0 0x561df572f7a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.809+0000 7fb6e1380740 1 --2- 172.21.15.90:0/1089890676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561df52e72f0 0x561df56c0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.809+0000 7fb6e1380740 1 -- 172.21.15.90:0/1089890676 >> 172.21.15.90:0/1089890676 conn(0x561df5304300 msgr2=0x561df56b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.809+0000 7fb6bcc00640 0 client.18393 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.810+0000 7fb6e1380740 1 -- 172.21.15.90:0/1089890676 shutdown_connections 2024-05-27T08:04:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.810+0000 7fb6e1380740 1 -- 172.21.15.90:0/1089890676 wait complete. 2024-05-27T08:04:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.810+0000 7fb6e1380740 20 client.18393 trim_cache size 0 max 16384 2024-05-27T08:04:36.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:04:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:04:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:04:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.811+0000 7fb6d8c00640 1 -- 172.21.15.90:0/3947343513 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6b80b29d0 msgr2=0x7fb6b80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.811+0000 7fb6d8c00640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6b80b29d0 0x7fb6b80b4ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009a60 tx=0x7fb6cc009950 comp rx=0 tx=0).stop 2024-05-27T08:04:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.811+0000 7fb6d8c00640 1 -- 172.21.15.90:0/3947343513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d41041f0 msgr2=0x7fb6d40760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.811+0000 7fb6d8c00640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d41041f0 0x7fb6d40760d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400b450 tx=0x7fb6c400b910 comp rx=0 tx=0).stop 2024-05-27T08:04:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.812+0000 7fb6d8c00640 1 -- 172.21.15.90:0/3947343513 shutdown_connections 2024-05-27T08:04:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.812+0000 7fb6d8c00640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb6b80b29d0 0x7fb6b80b4ee0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.812+0000 7fb6d8c00640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6d4109f40 0x7fb6d4076b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.812+0000 7fb6d8c00640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb6d4109aa0 0x7fb6d4076610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.812+0000 7fb6d8c00640 1 --2- 172.21.15.90:0/3947343513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d41041f0 0x7fb6d40760d0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.812+0000 7fb6d8c00640 1 -- 172.21.15.90:0/3947343513 >> 172.21.15.90:0/3947343513 conn(0x7fb6d40791a0 msgr2=0x7fb6d4100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.813+0000 7fb6d8c00640 1 -- 172.21.15.90:0/3947343513 shutdown_connections 2024-05-27T08:04:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:36.814+0000 7fb6d8c00640 1 -- 172.21.15.90:0/3947343513 wait complete. 2024-05-27T08:04:38.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:37 smithi136 ceph-mon[35324]: pgmap v588: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 495 op/s 2024-05-27T08:04:38.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:37 smithi090 ceph-mon[31429]: pgmap v588: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 495 op/s 2024-05-27T08:04:38.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:37 smithi129 ceph-mon[35254]: pgmap v588: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 495 op/s 2024-05-27T08:04:38.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 860.0 s, 444.8 tps, lat 223.662 ms stddev 202.938 2024-05-27T08:04:40.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:39 smithi136 ceph-mon[35324]: pgmap v589: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 470 op/s 2024-05-27T08:04:40.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:39 smithi129 ceph-mon[35254]: pgmap v589: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 470 op/s 2024-05-27T08:04:40.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:39 smithi090 ceph-mon[31429]: pgmap v589: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 470 op/s 2024-05-27T08:04:42.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.151+0000 7ff10a800700 1 -- 172.21.15.90:0/4205339719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff104114070 msgr2=0x7ff10411b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:42.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.151+0000 7ff10a800700 1 --2- 172.21.15.90:0/4205339719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff104114070 0x7ff10411b9d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b0a0 tx=0x7ff0fc00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.151+0000 7ff10a800700 1 -- 172.21.15.90:0/4205339719 shutdown_connections 2024-05-27T08:04:42.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.151+0000 7ff10a800700 1 --2- 172.21.15.90:0/4205339719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff104114070 0x7ff10411b9d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.151+0000 7ff10a800700 1 --2- 172.21.15.90:0/4205339719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff10410edf0 0x7ff10410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.151+0000 7ff10a800700 1 --2- 172.21.15.90:0/4205339719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104113720 0x7ff104113b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.151+0000 7ff10a800700 1 -- 172.21.15.90:0/4205339719 >> 172.21.15.90:0/4205339719 conn(0x7ff10410a980 msgr2=0x7ff10410cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:42.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.152+0000 7ff10a800700 1 -- 172.21.15.90:0/4205339719 shutdown_connections 2024-05-27T08:04:42.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.152+0000 7ff10a800700 1 -- 172.21.15.90:0/4205339719 wait complete. 2024-05-27T08:04:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.154+0000 7ff10a800700 1 Processor -- start 2024-05-27T08:04:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.154+0000 7ff10a800700 1 -- start start 2024-05-27T08:04:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.155+0000 7ff10a800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff10410edf0 0x7ff104118dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.156+0000 7ff10a800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104113720 0x7ff104119300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.156+0000 7ff109400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff10410edf0 0x7ff104118dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.156+0000 7ff109400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff10410edf0 0x7ff104118dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:46804/0 (socket says 172.21.15.90:46804) 2024-05-27T08:04:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.156+0000 7ff109400700 1 -- 172.21.15.90:0/2726709205 learned_addr learned my addr 172.21.15.90:0/2726709205 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.156+0000 7ff10a800700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff104114070 0x7ff104119840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.156+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff10411eb90 con 0x7ff104113720 2024-05-27T08:04:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.156+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff10411ea10 con 0x7ff10410edf0 2024-05-27T08:04:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.156+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff10411ed10 con 0x7ff104114070 2024-05-27T08:04:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.157+0000 7ff109e00700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff104114070 0x7ff104119840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.157+0000 7ff108a00700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104113720 0x7ff104119300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.157+0000 7ff109e00700 1 -- 172.21.15.90:0/2726709205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff10410edf0 msgr2=0x7ff104118dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.157+0000 7ff109e00700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff10410edf0 0x7ff104118dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.157+0000 7ff109e00700 1 -- 172.21.15.90:0/2726709205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104113720 msgr2=0x7ff104119300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.158+0000 7ff109e00700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104113720 0x7ff104119300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.158+0000 7ff109e00700 1 -- 172.21.15.90:0/2726709205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1041ab270 con 0x7ff104114070 2024-05-27T08:04:42.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.159+0000 7ff109e00700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff104114070 0x7ff104119840 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b070 tx=0x7ff0fc004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:42.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.160+0000 7ff102000700 1 -- 172.21.15.90:0/2726709205 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc021070 con 0x7ff104114070 2024-05-27T08:04:42.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.160+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1041ab4a0 con 0x7ff104114070 2024-05-27T08:04:42.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.161+0000 7ff102000700 1 -- 172.21.15.90:0/2726709205 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0fc007b80 con 0x7ff104114070 2024-05-27T08:04:42.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.162+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1041ab9e0 con 0x7ff104114070 2024-05-27T08:04:42.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.162+0000 7ff102000700 1 -- 172.21.15.90:0/2726709205 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0052c0 con 0x7ff104114070 2024-05-27T08:04:42.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.169+0000 7ff102000700 1 -- 172.21.15.90:0/2726709205 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff0fc024070 con 0x7ff104114070 2024-05-27T08:04:42.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.173+0000 7ff102000700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff0ec0b2a70 0x7ff0ec0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.173+0000 7ff102000700 1 -- 172.21.15.90:0/2726709205 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7ff0fc0e3a70 con 0x7ff104114070 2024-05-27T08:04:42.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.174+0000 7ff109400700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff0ec0b2a70 0x7ff0ec0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.177+0000 7ff109400700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff0ec0b2a70 0x7ff0ec0b4f80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009cb0 tx=0x7ff0f8009260 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:42.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.215+0000 7ff11a9afb80 1 Processor -- start 2024-05-27T08:04:42.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.217+0000 7ff11a9afb80 1 -- start start 2024-05-27T08:04:42.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.220+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf194e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.220+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.221+0000 7ff109400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.221+0000 7ff109400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf194e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:48744/0 (socket says 172.21.15.90:48744) 2024-05-27T08:04:42.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.223+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.223+0000 7ff108a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.224+0000 7ff108a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf19fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59312/0 (socket says 172.21.15.90:59312) 2024-05-27T08:04:42.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.224+0000 7ff109400700 1 -- 172.21.15.90:0/3357946352 learned_addr learned my addr 172.21.15.90:0/3357946352 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:42.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.224+0000 7ff109e00700 1 --2- 172.21.15.90:0/3357946352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.225+0000 7ff11a9afb80 1 -- 172.21.15.90:0/3357946352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5610cf0c3e40 con 0x5610cf1953a0 2024-05-27T08:04:42.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.225+0000 7ff11a9afb80 1 -- 172.21.15.90:0/3357946352 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5610cf0b39e0 con 0x5610cf1957b0 2024-05-27T08:04:42.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.225+0000 7ff11a9afb80 1 -- 172.21.15.90:0/3357946352 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5610cf0b7a20 con 0x5610cf016e40 2024-05-27T08:04:42.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.226+0000 7ff108a00700 1 -- 172.21.15.90:0/3357946352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 msgr2=0x5610cf194e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:42.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.226+0000 7ff108a00700 1 --2- 172.21.15.90:0/3357946352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf194e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.226+0000 7ff108a00700 1 -- 172.21.15.90:0/3357946352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 msgr2=0x5610cf1a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:42.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.226+0000 7ff108a00700 1 --2- 172.21.15.90:0/3357946352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.227+0000 7ff108a00700 1 -- 172.21.15.90:0/3357946352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610cf07c420 con 0x5610cf1953a0 2024-05-27T08:04:42.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.227+0000 7ff108a00700 1 --2- 172.21.15.90:0/3357946352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf19fb10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff104113150 tx=0x7ff0f0009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed15981115126d3d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:42.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.229+0000 7ff0f6a00700 1 -- 172.21.15.90:0/3357946352 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0016070 con 0x5610cf1953a0 2024-05-27T08:04:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.229+0000 7ff0f6a00700 1 -- 172.21.15.90:0/3357946352 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0f0005020 con 0x5610cf1953a0 2024-05-27T08:04:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.229+0000 7ff0f6a00700 1 -- 172.21.15.90:0/3357946352 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f000e9f0 con 0x5610cf1953a0 2024-05-27T08:04:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.230+0000 7ff11a9afb80 1 -- 172.21.15.90:0/3357946352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 msgr2=0x5610cf19fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:42.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.230+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/3357946352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf19fb10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff104113150 tx=0x7ff0f0009d70 comp rx=0 tx=0).stop 2024-05-27T08:04:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.230+0000 7ff11a9afb80 1 -- 172.21.15.90:0/3357946352 shutdown_connections 2024-05-27T08:04:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.230+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/3357946352 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.230+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/3357946352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf194e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.230+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/3357946352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf19fb10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.230+0000 7ff11a9afb80 1 -- 172.21.15.90:0/3357946352 >> 172.21.15.90:0/3357946352 conn(0x5610cf076fc0 msgr2=0x5610cf190d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:42.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.232+0000 7ff11a9afb80 1 -- 172.21.15.90:0/3357946352 shutdown_connections 2024-05-27T08:04:42.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.232+0000 7ff11a9afb80 1 -- 172.21.15.90:0/3357946352 wait complete. 2024-05-27T08:04:42.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.233+0000 7ff11a9afb80 1 -- start start 2024-05-27T08:04:42.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.233+0000 7ff11a9afb80 1 Processor -- start 2024-05-27T08:04:42.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.234+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf016e40 0x5610cf19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.235+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.235+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1957b0 0x5610cf1a0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.235+0000 7ff11a9afb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5610cefb9fa0 con 0x5610cf1953a0 2024-05-27T08:04:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.235+0000 7ff11a9afb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5610cf054760 con 0x5610cf016e40 2024-05-27T08:04:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.235+0000 7ff11a9afb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5610cf1a29d0 con 0x5610cf1957b0 2024-05-27T08:04:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.235+0000 7ff11a9afb80 10 client.0 _ll_register_callbacks cb 0x5610cf05dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.236+0000 7ff109e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1957b0 0x5610cf1a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.236+0000 7ff109e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1957b0 0x5610cf1a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:48760/0 (socket says 172.21.15.90:48760) 2024-05-27T08:04:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.236+0000 7ff109e00700 1 -- 172.21.15.90:0/2516322303 learned_addr learned my addr 172.21.15.90:0/2516322303 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.236+0000 7ff109400700 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf016e40 0x5610cf19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.236+0000 7ff109e00700 1 -- 172.21.15.90:0/2516322303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf016e40 msgr2=0x5610cf19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.236+0000 7ff109e00700 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf016e40 0x5610cf19f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.236+0000 7ff109e00700 1 -- 172.21.15.90:0/2516322303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 msgr2=0x5610cf19faf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:04:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.237+0000 7ff108a00700 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.237+0000 7ff109e00700 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf19faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.237+0000 7ff109e00700 1 -- 172.21.15.90:0/2516322303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610cf0038d0 con 0x5610cf1957b0 2024-05-27T08:04:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.237+0000 7ff109e00700 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1957b0 0x5610cf1a0030 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0e0ea0 tx=0x7ff0fc0aa230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df4193a197840853 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.237+0000 7ff0eb400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:04:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.237+0000 7ff0eb400700 1 -- 172.21.15.90:0/2516322303 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc02b070 con 0x5610cf1957b0 2024-05-27T08:04:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.238+0000 7ff0eb400700 1 -- 172.21.15.90:0/2516322303 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7ff0fc0b10b0 con 0x5610cf1957b0 2024-05-27T08:04:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.238+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2516322303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f0005020 con 0x5610cf1957b0 2024-05-27T08:04:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.238+0000 7ff0eb400700 1 -- 172.21.15.90:0/2516322303 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff0fc0c4050 con 0x5610cf1957b0 2024-05-27T08:04:42.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.239+0000 7ff11a9afb80 10 client.28183 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.239+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2516322303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5610cf084060 con 0x5610cf1957b0 2024-05-27T08:04:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.239+0000 7ff0eb400700 1 -- 172.21.15.90:0/2516322303 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0f0005020 con 0x5610cf1957b0 2024-05-27T08:04:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.240+0000 7ff0eb400700 1 -- 172.21.15.90:0/2516322303 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7ff0fc011070 con 0x5610cf1957b0 2024-05-27T08:04:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.241+0000 7ff11a9afb80 10 client.28183 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.241+0000 7ff11a9afb80 10 client.28183 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T08:04:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.241+0000 7ff11a9afb80 20 client.28183 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.241+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5610cf1af8c0 0x5610cf1b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.241+0000 7ff11a9afb80 4 client.28183 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.241+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2516322303 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610cf0c5d90 con 0x5610cf1af8c0 2024-05-27T08:04:42.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.241+0000 7ff109400700 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5610cf1af8c0 0x5610cf1b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.243+0000 7ff0eb400700 10 client.28183 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T08:04:42.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.243+0000 7ff109400700 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5610cf1af8c0 0x5610cf1b1d50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd4e31e4bc256a91 server_cookie=25e33232365cc29d in_seq=0 out_seq=0 2024-05-27T08:04:42.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.244+0000 7ff0eb400700 1 -- 172.21.15.90:0/2516322303 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5610cf0c5d90 con 0x5610cf1af8c0 2024-05-27T08:04:42.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.244+0000 7ff0eb400700 10 client.28183 handle_command_reply: tid=0 2024-05-27T08:04:42.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.244+0000 7ff11a9afb80 1 client.28183 shutdown 2024-05-27T08:04:42.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2516322303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1957b0 msgr2=0x5610cf1a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:42.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1957b0 0x5610cf1a0030 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0e0ea0 tx=0x7ff0fc0aa230 comp rx=0 tx=0).stop 2024-05-27T08:04:42.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2516322303 shutdown_connections 2024-05-27T08:04:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5610cf1af8c0 0x5610cf1b1d50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1957b0 0x5610cf1a0030 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf016e40 0x5610cf19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff0eb400700 0 client.28183 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T08:04:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2516322303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf19faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2516322303 >> 172.21.15.90:0/2516322303 conn(0x5610cf076fc0 msgr2=0x5610cf18fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2516322303 shutdown_connections 2024-05-27T08:04:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2516322303 wait complete. 2024-05-27T08:04:42.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.245+0000 7ff11a9afb80 20 client.28183 trim_cache size 0 max 16384 2024-05-27T08:04:42.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.274+0000 7ff11a9afb80 1 Processor -- start 2024-05-27T08:04:42.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.274+0000 7ff11a9afb80 1 -- start start 2024-05-27T08:04:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:42 smithi090 ceph-mon[31429]: pgmap v590: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 512 op/s 2024-05-27T08:04:42.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.274+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf016e40 0x5610cf19add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.275+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1953a0 0x5610cf1aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.275+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.275+0000 7ff11a9afb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff1041c9260 con 0x5610cf016e40 2024-05-27T08:04:42.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.275+0000 7ff11a9afb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff1041c62b0 con 0x5610cf1957b0 2024-05-27T08:04:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.275+0000 7ff11a9afb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1041c84c0 con 0x5610cf1953a0 2024-05-27T08:04:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.276+0000 7ff109400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf016e40 0x5610cf19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.276+0000 7ff108a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1953a0 0x5610cf1aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.276+0000 7ff108a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1953a0 0x5610cf1aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:48774/0 (socket says 172.21.15.90:48774) 2024-05-27T08:04:42.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.276+0000 7ff108a00700 1 -- 172.21.15.90:0/168720828 learned_addr learned my addr 172.21.15.90:0/168720828 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.276+0000 7ff109e00700 1 --2- 172.21.15.90:0/168720828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.277+0000 7ff109e00700 1 -- 172.21.15.90:0/168720828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1953a0 msgr2=0x5610cf1aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.277+0000 7ff109e00700 1 --2- 172.21.15.90:0/168720828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1953a0 0x5610cf1aba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.277+0000 7ff109e00700 1 -- 172.21.15.90:0/168720828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf016e40 msgr2=0x5610cf19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.277+0000 7ff109e00700 1 --2- 172.21.15.90:0/168720828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf016e40 0x5610cf19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.277+0000 7ff109e00700 1 -- 172.21.15.90:0/168720828 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610cf19fdc0 con 0x5610cf1957b0 2024-05-27T08:04:42.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.277+0000 7ff108a00700 1 --2- 172.21.15.90:0/168720828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1953a0 0x5610cf1aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:42.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.278+0000 7ff109e00700 1 --2- 172.21.15.90:0/168720828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a1220 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0e3310 tx=0x7ff0fc00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dc3dd6cf4ee8158 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:42.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.278+0000 7ff0f6000700 1 -- 172.21.15.90:0/168720828 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc02b070 con 0x5610cf1957b0 2024-05-27T08:04:42.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.279+0000 7ff0f6000700 1 -- 172.21.15.90:0/168720828 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0fc0aa520 con 0x5610cf1957b0 2024-05-27T08:04:42.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.279+0000 7ff0f6000700 1 -- 172.21.15.90:0/168720828 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0b1310 con 0x5610cf1957b0 2024-05-27T08:04:42.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.282+0000 7ff11a9afb80 1 -- 172.21.15.90:0/168720828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 msgr2=0x5610cf1a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:42.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.282+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/168720828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a1220 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0e3310 tx=0x7ff0fc00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.282+0000 7ff11a9afb80 1 -- 172.21.15.90:0/168720828 shutdown_connections 2024-05-27T08:04:42.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.283+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/168720828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a1220 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.283+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/168720828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf016e40 0x5610cf19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.283+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/168720828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf1953a0 0x5610cf1aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.283+0000 7ff11a9afb80 1 -- 172.21.15.90:0/168720828 >> 172.21.15.90:0/168720828 conn(0x5610cf076fc0 msgr2=0x5610cf1a0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:42.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.283+0000 7ff11a9afb80 1 -- 172.21.15.90:0/168720828 shutdown_connections 2024-05-27T08:04:42.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.283+0000 7ff11a9afb80 1 -- 172.21.15.90:0/168720828 wait complete. 2024-05-27T08:04:42.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.284+0000 7ff11a9afb80 1 -- start start 2024-05-27T08:04:42.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.285+0000 7ff11a9afb80 1 Processor -- start 2024-05-27T08:04:42.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.287+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf1a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.287+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf1a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.287+0000 7ff11a9afb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.287+0000 7ff11a9afb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff1041c7df0 con 0x5610cf1953a0 2024-05-27T08:04:42.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.287+0000 7ff11a9afb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff1041c6980 con 0x5610cf1957b0 2024-05-27T08:04:42.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.287+0000 7ff11a9afb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1041c7050 con 0x5610cf016e40 2024-05-27T08:04:42.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.287+0000 7ff11a9afb80 10 client.0 _ll_register_callbacks cb 0x5610cf05dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:42.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.289+0000 7ff109400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf1a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.289+0000 7ff109400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf1a7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:48786/0 (socket says 172.21.15.90:48786) 2024-05-27T08:04:42.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.289+0000 7ff109400700 1 -- 172.21.15.90:0/2197902869 learned_addr learned my addr 172.21.15.90:0/2197902869 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:42.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.289+0000 7ff108a00700 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf1a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.290+0000 7ff109e00700 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.290+0000 7ff109400700 1 -- 172.21.15.90:0/2197902869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 msgr2=0x5610cf1a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:42.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.290+0000 7ff109400700 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.290+0000 7ff109400700 1 -- 172.21.15.90:0/2197902869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 msgr2=0x5610cf1a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:42.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.290+0000 7ff109400700 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf1a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.290+0000 7ff109400700 1 -- 172.21.15.90:0/2197902869 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610cf1a3c20 con 0x5610cf016e40 2024-05-27T08:04:42.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.291+0000 7ff109400700 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf1a7f90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x5610cf1916d0 tx=0x7ff0f80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c80a629f239e1ce server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:42.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.291+0000 7ff0eb400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:04:42.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.293+0000 7ff0eb400700 1 -- 172.21.15.90:0/2197902869 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f802d070 con 0x5610cf016e40 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.293+0000 7ff0eb400700 1 -- 172.21.15.90:0/2197902869 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7ff0f80294c0 con 0x5610cf016e40 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.293+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2197902869 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0fc0aa520 con 0x5610cf016e40 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.295+0000 7ff0eb400700 1 -- 172.21.15.90:0/2197902869 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7ff0f8037050 con 0x5610cf016e40 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.295+0000 7ff11a9afb80 10 client.28189 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.295+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2197902869 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5610cf219ff0 con 0x5610cf016e40 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.295+0000 7ff0eb400700 1 -- 172.21.15.90:0/2197902869 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0fc0aa520 con 0x5610cf016e40 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.296+0000 7ff0eb400700 1 -- 172.21.15.90:0/2197902869 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7ff0f8032940 con 0x5610cf016e40 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.296+0000 7ff11a9afb80 10 client.28189 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.296+0000 7ff11a9afb80 10 client.28189 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.296+0000 7ff11a9afb80 20 client.28189 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:42.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.296+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5610cf21a7b0 0x5610cf21cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.296+0000 7ff11a9afb80 4 client.28189 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:04:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.296+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2197902869 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610cf0bf570 con 0x5610cf21a7b0 2024-05-27T08:04:42.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.297+0000 7ff108a00700 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5610cf21a7b0 0x5610cf21cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:42.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.300+0000 7ff108a00700 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5610cf21a7b0 0x5610cf21cbd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6762f7ca0769b0af server_cookie=626594d9ec8a008b in_seq=0 out_seq=0 2024-05-27T08:04:42.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.300+0000 7ff0eb400700 10 client.28189 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T08:04:42.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.301+0000 7ff0eb400700 1 -- 172.21.15.90:0/2197902869 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610cf0bf570 con 0x5610cf21a7b0 2024-05-27T08:04:42.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.301+0000 7ff0eb400700 10 client.28189 handle_command_reply: tid=0 2024-05-27T08:04:42.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.304+0000 7ff11a9afb80 1 client.28189 shutdown 2024-05-27T08:04:42.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.304+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2197902869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 msgr2=0x5610cf1a7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:42.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.304+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf1a7f90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x5610cf1916d0 tx=0x7ff0f80047c0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.304+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2197902869 shutdown_connections 2024-05-27T08:04:42.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.304+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5610cf21a7b0 0x5610cf21cbd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.304+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5610cf1957b0 0x5610cf1a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.304+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5610cf016e40 0x5610cf1a7f90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.304+0000 7ff11a9afb80 1 --2- 172.21.15.90:0/2197902869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5610cf1953a0 0x5610cf1a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.304+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2197902869 >> 172.21.15.90:0/2197902869 conn(0x5610cf076fc0 msgr2=0x5610cf19e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:42.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.305+0000 7ff0eb400700 0 client.28189 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T08:04:42.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.305+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2197902869 shutdown_connections 2024-05-27T08:04:42.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.305+0000 7ff11a9afb80 1 -- 172.21.15.90:0/2197902869 wait complete. 2024-05-27T08:04:42.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.305+0000 7ff11a9afb80 20 client.28189 trim_cache size 0 max 16384 2024-05-27T08:04:42.307 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:04:42.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.307+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff0ec0b2a70 msgr2=0x7ff0ec0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:42.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.307+0000 7ff10a800700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff0ec0b2a70 0x7ff0ec0b4f80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009cb0 tx=0x7ff0f8009260 comp rx=0 tx=0).stop 2024-05-27T08:04:42.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.307+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff104114070 msgr2=0x7ff104119840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:42.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.307+0000 7ff10a800700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff104114070 0x7ff104119840 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b070 tx=0x7ff0fc004c80 comp rx=0 tx=0).stop 2024-05-27T08:04:42.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.308+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 shutdown_connections 2024-05-27T08:04:42.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.308+0000 7ff10a800700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff104114070 0x7ff104119840 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.308+0000 7ff10a800700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff0ec0b2a70 0x7ff0ec0b4f80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.308+0000 7ff10a800700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff10410edf0 0x7ff104118dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.308+0000 7ff10a800700 1 --2- 172.21.15.90:0/2726709205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104113720 0x7ff104119300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:42.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.308+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 >> 172.21.15.90:0/2726709205 conn(0x7ff10410a980 msgr2=0x7ff104111790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:42.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.309+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 shutdown_connections 2024-05-27T08:04:42.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:42.309+0000 7ff10a800700 1 -- 172.21.15.90:0/2726709205 wait complete. 2024-05-27T08:04:42.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:42 smithi129 ceph-mon[35254]: pgmap v590: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 512 op/s 2024-05-27T08:04:42.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:42 smithi136 ceph-mon[35324]: pgmap v590: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.7 MiB/s wr, 512 op/s 2024-05-27T08:04:43.240 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T08:04:43.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 865.0 s, 459.6 tps, lat 215.065 ms stddev 165.174 2024-05-27T08:04:44.027 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:44.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:44 smithi129 ceph-mon[35254]: pgmap v591: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-05-27T08:04:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:44 smithi090 ceph-mon[31429]: pgmap v591: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-05-27T08:04:44.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:44 smithi136 ceph-mon[35324]: pgmap v591: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.6 MiB/s wr, 416 op/s 2024-05-27T08:04:46.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:46 smithi129 ceph-mon[35254]: pgmap v592: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 521 op/s 2024-05-27T08:04:46.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:46 smithi090 ceph-mon[31429]: pgmap v592: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 521 op/s 2024-05-27T08:04:46.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:46 smithi136 ceph-mon[35324]: pgmap v592: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.9 MiB/s wr, 521 op/s 2024-05-27T08:04:48.317 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:48 smithi129 ceph-mon[35254]: pgmap v593: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 475 op/s 2024-05-27T08:04:48.318 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:48 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:04:48.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:48 smithi090 ceph-mon[31429]: pgmap v593: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 475 op/s 2024-05-27T08:04:48.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:48 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:04:48.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:48 smithi136 ceph-mon[35324]: pgmap v593: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 475 op/s 2024-05-27T08:04:48.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:48 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:04:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 870.0 s, 427.8 tps, lat 232.544 ms stddev 204.065 2024-05-27T08:04:49.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:49 smithi129 ceph-mon[35254]: pgmap v594: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 434 op/s 2024-05-27T08:04:49.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:49 smithi090 ceph-mon[31429]: pgmap v594: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 434 op/s 2024-05-27T08:04:49.501 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:49 smithi136 ceph-mon[35324]: pgmap v594: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 MiB/s wr, 434 op/s 2024-05-27T08:04:49.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.630+0000 7f75d6200700 1 -- 172.21.15.90:0/2240388364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d010e8e0 msgr2=0x7f75d0110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:49.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.630+0000 7f75d6200700 1 --2- 172.21.15.90:0/2240388364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d010e8e0 0x7f75d0110dc0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009f90 tx=0x7f75c800b330 comp rx=0 tx=0).stop 2024-05-27T08:04:49.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.631+0000 7f75d6200700 1 -- 172.21.15.90:0/2240388364 shutdown_connections 2024-05-27T08:04:49.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.631+0000 7f75d6200700 1 --2- 172.21.15.90:0/2240388364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d010e8e0 0x7f75d0110dc0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.631+0000 7f75d6200700 1 --2- 172.21.15.90:0/2240388364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75d01054b0 0x7f75d01058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.631+0000 7f75d6200700 1 --2- 172.21.15.90:0/2240388364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d0105e00 0x7f75d010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.631+0000 7f75d6200700 1 -- 172.21.15.90:0/2240388364 >> 172.21.15.90:0/2240388364 conn(0x7f75d00fedb0 msgr2=0x7f75d01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:49.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.631+0000 7f75d6200700 1 -- 172.21.15.90:0/2240388364 shutdown_connections 2024-05-27T08:04:49.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.631+0000 7f75d6200700 1 -- 172.21.15.90:0/2240388364 wait complete. 2024-05-27T08:04:49.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.634+0000 7f75d6200700 1 Processor -- start 2024-05-27T08:04:49.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.634+0000 7f75d6200700 1 -- start start 2024-05-27T08:04:49.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.635+0000 7f75d6200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d01054b0 0x7f75d01a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.635+0000 7f75d6200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d0105e00 0x7f75d01a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.635+0000 7f75d6200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75d010e8e0 0x7f75d01a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.635+0000 7f75d6200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75d01121e0 con 0x7f75d0105e00 2024-05-27T08:04:49.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.635+0000 7f75d6200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0112060 con 0x7f75d01054b0 2024-05-27T08:04:49.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.635+0000 7f75d6200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75d0112360 con 0x7f75d010e8e0 2024-05-27T08:04:49.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.636+0000 7f75cfe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75d010e8e0 0x7f75d01a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.636+0000 7f75cea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d0105e00 0x7f75d01a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.636+0000 7f75cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d01054b0 0x7f75d01a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.637+0000 7f75cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d01054b0 0x7f75d01a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37400/0 (socket says 172.21.15.90:37400) 2024-05-27T08:04:49.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.637+0000 7f75cea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d0105e00 0x7f75d01a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45286/0 (socket says 172.21.15.90:45286) 2024-05-27T08:04:49.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.637+0000 7f75cf400700 1 -- 172.21.15.90:0/758308520 learned_addr learned my addr 172.21.15.90:0/758308520 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:49.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.638+0000 7f75cf400700 1 -- 172.21.15.90:0/758308520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75d010e8e0 msgr2=0x7f75d01a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:49.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.638+0000 7f75cf400700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75d010e8e0 0x7f75d01a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.638+0000 7f75cf400700 1 -- 172.21.15.90:0/758308520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d0105e00 msgr2=0x7f75d01a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:49.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.638+0000 7f75cf400700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d0105e00 0x7f75d01a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.638+0000 7f75cf400700 1 -- 172.21.15.90:0/758308520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d01a6180 con 0x7f75d01054b0 2024-05-27T08:04:49.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.639+0000 7f75cf400700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d01054b0 0x7f75d01a1300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f75c000e9c0 tx=0x7f75c000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:49.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.639+0000 7f75c7e00700 1 -- 172.21.15.90:0/758308520 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c000cbd0 con 0x7f75d01054b0 2024-05-27T08:04:49.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.639+0000 7f75d6200700 1 -- 172.21.15.90:0/758308520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d01a6470 con 0x7f75d01054b0 2024-05-27T08:04:49.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.639+0000 7f75d6200700 1 -- 172.21.15.90:0/758308520 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d0198850 con 0x7f75d01054b0 2024-05-27T08:04:49.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.640+0000 7f75c7e00700 1 -- 172.21.15.90:0/758308520 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75c000ee60 con 0x7f75d01054b0 2024-05-27T08:04:49.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.640+0000 7f75c7e00700 1 -- 172.21.15.90:0/758308520 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c00054b0 con 0x7f75d01054b0 2024-05-27T08:04:49.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.647+0000 7f75c7e00700 1 -- 172.21.15.90:0/758308520 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f75c0021440 con 0x7f75d01054b0 2024-05-27T08:04:49.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.651+0000 7f75c7e00700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75b40b2bf0 0x7f75b40b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.652+0000 7f75c7e00700 1 -- 172.21.15.90:0/758308520 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f75c0005250 con 0x7f75d01054b0 2024-05-27T08:04:49.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.653+0000 7f75cea00700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75b40b2bf0 0x7f75b40b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.662+0000 7f75cea00700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75b40b2bf0 0x7f75b40b5100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f75d01022b0 tx=0x7f75bc011040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:49.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.669+0000 7f75e4d0cb80 1 Processor -- start 2024-05-27T08:04:49.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.670+0000 7f75e4d0cb80 1 -- start start 2024-05-27T08:04:49.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.672+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.672+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d27bd30 0x55dc5d275870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.672+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d275db0 0x55dc5d2761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.672+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d1a3e40 con 0x55dc5d27bd30 2024-05-27T08:04:49.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.673+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d1939e0 con 0x55dc5d0f6e40 2024-05-27T08:04:49.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.673+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d197a20 con 0x55dc5d275db0 2024-05-27T08:04:49.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.673+0000 7f75cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.673+0000 7f75cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37404/0 (socket says 172.21.15.90:37404) 2024-05-27T08:04:49.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.673+0000 7f75cf400700 1 -- 172.21.15.90:0/2617516587 learned_addr learned my addr 172.21.15.90:0/2617516587 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:49.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.673+0000 7f75cfe00700 1 --2- 172.21.15.90:0/2617516587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d275db0 0x55dc5d2761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.674+0000 7f75cfe00700 1 -- 172.21.15.90:0/2617516587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 msgr2=0x55dc5d27b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:49.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.674+0000 7f75cfe00700 1 --2- 172.21.15.90:0/2617516587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.674+0000 7f75cfe00700 1 -- 172.21.15.90:0/2617516587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d27bd30 msgr2=0x55dc5d275870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:49.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.674+0000 7f75cfe00700 1 --2- 172.21.15.90:0/2617516587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d27bd30 0x55dc5d275870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.674+0000 7f75cfe00700 1 -- 172.21.15.90:0/2617516587 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5d15c420 con 0x55dc5d275db0 2024-05-27T08:04:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.675+0000 7f75cfe00700 1 --2- 172.21.15.90:0/2617516587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d275db0 0x55dc5d2761c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b6c0 tx=0x7f75c80092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c8dd2257389a6ef server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.675+0000 7f75c4c00700 1 -- 172.21.15.90:0/2617516587 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8020070 con 0x55dc5d275db0 2024-05-27T08:04:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.675+0000 7f75c4c00700 1 -- 172.21.15.90:0/2617516587 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75c8003c00 con 0x55dc5d275db0 2024-05-27T08:04:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.675+0000 7f75c4c00700 1 -- 172.21.15.90:0/2617516587 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8007a50 con 0x55dc5d275db0 2024-05-27T08:04:49.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.677+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/2617516587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d275db0 msgr2=0x55dc5d2761c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:49.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.677+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/2617516587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d275db0 0x55dc5d2761c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f75c800b6c0 tx=0x7f75c80092e0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.678+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/2617516587 shutdown_connections 2024-05-27T08:04:49.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.678+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/2617516587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d275db0 0x55dc5d2761c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.678+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/2617516587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.678+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/2617516587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d27bd30 0x55dc5d275870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.678+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/2617516587 >> 172.21.15.90:0/2617516587 conn(0x55dc5d156fc0 msgr2=0x55dc5d2708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:49.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.678+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/2617516587 shutdown_connections 2024-05-27T08:04:49.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.678+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/2617516587 wait complete. 2024-05-27T08:04:49.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.679+0000 7f75e4d0cb80 1 -- start start 2024-05-27T08:04:49.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.679+0000 7f75e4d0cb80 1 Processor -- start 2024-05-27T08:04:49.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.681+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.681+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d27fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.681+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.681+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d099fa0 con 0x55dc5d275db0 2024-05-27T08:04:49.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.681+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d134760 con 0x55dc5d0f6e40 2024-05-27T08:04:49.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.681+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc5d2827a0 con 0x55dc5d27bd30 2024-05-27T08:04:49.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.681+0000 7f75e4d0cb80 10 client.0 _ll_register_callbacks cb 0x55dc5d13dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.682+0000 7f75cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.682+0000 7f75cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37416/0 (socket says 172.21.15.90:37416) 2024-05-27T08:04:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.682+0000 7f75cf400700 1 -- 172.21.15.90:0/1630627677 learned_addr learned my addr 172.21.15.90:0/1630627677 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.683+0000 7f75cfe00700 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.683+0000 7f75cea00700 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d27fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.685+0000 7f75cf400700 1 -- 172.21.15.90:0/1630627677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 msgr2=0x55dc5d27ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.685+0000 7f75cf400700 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.685+0000 7f75cf400700 1 -- 172.21.15.90:0/1630627677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 msgr2=0x55dc5d27fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.685+0000 7f75cf400700 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d27fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.685+0000 7f75cf400700 1 -- 172.21.15.90:0/1630627677 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5d0e38d0 con 0x55dc5d0f6e40 2024-05-27T08:04:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.685+0000 7f75cfe00700 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:04:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.685+0000 7f75cea00700 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d27fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.686+0000 7f75cf400700 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27f510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f75c00a7a60 tx=0x7f75c00aa430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1edda95859e6301 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.686+0000 7f75b9600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:04:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.687+0000 7f75b9600700 1 -- 172.21.15.90:0/1630627677 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c00b3650 con 0x55dc5d0f6e40 2024-05-27T08:04:49.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.687+0000 7f75b9600700 1 -- 172.21.15.90:0/1630627677 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f75c00aaa50 con 0x55dc5d0f6e40 2024-05-27T08:04:49.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.687+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1630627677 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75c8003c00 con 0x55dc5d0f6e40 2024-05-27T08:04:49.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.688+0000 7f75b9600700 1 -- 172.21.15.90:0/1630627677 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f75c00aac50 con 0x55dc5d0f6e40 2024-05-27T08:04:49.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.688+0000 7f75e4d0cb80 10 client.38114 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:49.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.689+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1630627677 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55dc5d164060 con 0x55dc5d0f6e40 2024-05-27T08:04:49.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.691+0000 7f75b9600700 1 -- 172.21.15.90:0/1630627677 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75c8003c00 con 0x55dc5d0f6e40 2024-05-27T08:04:49.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.691+0000 7f75b9600700 1 -- 172.21.15.90:0/1630627677 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f75c0005490 con 0x55dc5d0f6e40 2024-05-27T08:04:49.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.691+0000 7f75e4d0cb80 10 client.38114 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:49.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.691+0000 7f75e4d0cb80 10 client.38114 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T08:04:49.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.691+0000 7f75e4d0cb80 20 client.38114 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:49.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.691+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55dc5d28f910 0x55dc5d291da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.691+0000 7f75e4d0cb80 4 client.38114 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:49.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.691+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1630627677 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc5d1a5d90 con 0x55dc5d28f910 2024-05-27T08:04:49.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.692+0000 7f75cea00700 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55dc5d28f910 0x55dc5d291da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.693+0000 7f75b9600700 10 client.38114 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T08:04:49.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.693+0000 7f75cea00700 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55dc5d28f910 0x55dc5d291da0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=16564e8b9f246f63 server_cookie=153e172a69cdc93a in_seq=0 out_seq=0 2024-05-27T08:04:49.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.694+0000 7f75b9600700 1 -- 172.21.15.90:0/1630627677 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55dc5d1a5d90 con 0x55dc5d28f910 2024-05-27T08:04:49.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.694+0000 7f75b9600700 10 client.38114 handle_command_reply: tid=0 2024-05-27T08:04:49.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.695+0000 7f75e4d0cb80 1 client.38114 shutdown 2024-05-27T08:04:49.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.697+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1630627677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 msgr2=0x55dc5d27f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:49.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.697+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27f510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f75c00a7a60 tx=0x7f75c00aa430 comp rx=0 tx=0).stop 2024-05-27T08:04:49.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.698+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1630627677 shutdown_connections 2024-05-27T08:04:49.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.698+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55dc5d28f910 0x55dc5d291da0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.698+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.698+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d0f6e40 0x55dc5d27f510 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.698+0000 7f75b9600700 0 client.38114 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T08:04:49.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.698+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/1630627677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d27fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.698+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1630627677 >> 172.21.15.90:0/1630627677 conn(0x55dc5d156fc0 msgr2=0x55dc5d270580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:49.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.698+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1630627677 shutdown_connections 2024-05-27T08:04:49.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.698+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1630627677 wait complete. 2024-05-27T08:04:49.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.698+0000 7f75e4d0cb80 20 client.38114 trim_cache size 0 max 16384 2024-05-27T08:04:49.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.727+0000 7f75e4d0cb80 1 Processor -- start 2024-05-27T08:04:49.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.728+0000 7f75e4d0cb80 1 -- start start 2024-05-27T08:04:49.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.728+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d275870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.729+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.729+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d270b60 0x55dc5d281870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.729+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75b0010810 con 0x55dc5d275db0 2024-05-27T08:04:49.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.729+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75b000d860 con 0x55dc5d27bd30 2024-05-27T08:04:49.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.729+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75b000fa70 con 0x55dc5d270b60 2024-05-27T08:04:49.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.729+0000 7f75cea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.729+0000 7f75cfe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d270b60 0x55dc5d281870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.729+0000 7f75cea00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37428/0 (socket says 172.21.15.90:37428) 2024-05-27T08:04:49.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.729+0000 7f75cea00700 1 -- 172.21.15.90:0/1462971220 learned_addr learned my addr 172.21.15.90:0/1462971220 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:49.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.729+0000 7f75cfe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d270b60 0x55dc5d281870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33194/0 (socket says 172.21.15.90:33194) 2024-05-27T08:04:49.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.730+0000 7f75cf400700 1 --2- 172.21.15.90:0/1462971220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d275870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.730+0000 7f75cea00700 1 -- 172.21.15.90:0/1462971220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d270b60 msgr2=0x55dc5d281870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:49.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.730+0000 7f75cea00700 1 --2- 172.21.15.90:0/1462971220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d270b60 0x55dc5d281870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.730+0000 7f75cea00700 1 -- 172.21.15.90:0/1462971220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 msgr2=0x55dc5d275870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:49.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.730+0000 7f75cea00700 1 --2- 172.21.15.90:0/1462971220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d275870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.730+0000 7f75cea00700 1 -- 172.21.15.90:0/1462971220 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5d281db0 con 0x55dc5d27bd30 2024-05-27T08:04:49.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.731+0000 7f75cea00700 1 --2- 172.21.15.90:0/1462971220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55dc5d271550 tx=0x7f75bc016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e7ca8c51a67411c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:49.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.732+0000 7f75bbe00700 1 -- 172.21.15.90:0/1462971220 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc01a9e0 con 0x55dc5d27bd30 2024-05-27T08:04:49.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.732+0000 7f75bbe00700 1 -- 172.21.15.90:0/1462971220 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75bc01ab80 con 0x55dc5d27bd30 2024-05-27T08:04:49.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.733+0000 7f75bbe00700 1 -- 172.21.15.90:0/1462971220 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc026400 con 0x55dc5d27bd30 2024-05-27T08:04:49.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.734+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1462971220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d27bd30 msgr2=0x55dc5d27ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:49.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.734+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/1462971220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55dc5d271550 tx=0x7f75bc016000 comp rx=0 tx=0).stop 2024-05-27T08:04:49.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.737+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1462971220 shutdown_connections 2024-05-27T08:04:49.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.737+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/1462971220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d270b60 0x55dc5d281870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.737+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/1462971220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d275870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.737+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/1462971220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d27bd30 0x55dc5d27ff00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.737+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1462971220 >> 172.21.15.90:0/1462971220 conn(0x55dc5d156fc0 msgr2=0x55dc5d27e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:49.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.739+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1462971220 shutdown_connections 2024-05-27T08:04:49.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.739+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/1462971220 wait complete. 2024-05-27T08:04:49.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.740+0000 7f75e4d0cb80 1 -- start start 2024-05-27T08:04:49.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.740+0000 7f75e4d0cb80 1 Processor -- start 2024-05-27T08:04:49.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.741+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d270b60 0x55dc5d287f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.741+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d296b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.741+0000 7f75e4d0cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 0x55dc5d2970b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.741+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75b000f3a0 con 0x55dc5d275db0 2024-05-27T08:04:49.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.741+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f75b000df30 con 0x55dc5d270b60 2024-05-27T08:04:49.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.741+0000 7f75e4d0cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75b000e600 con 0x55dc5d27bd30 2024-05-27T08:04:49.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.741+0000 7f75e4d0cb80 10 client.0 _ll_register_callbacks cb 0x55dc5d13dc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:49.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.742+0000 7f75cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d270b60 0x55dc5d287f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.743+0000 7f75cf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d270b60 0x55dc5d287f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37442/0 (socket says 172.21.15.90:37442) 2024-05-27T08:04:49.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.743+0000 7f75cf400700 1 -- 172.21.15.90:0/632553168 learned_addr learned my addr 172.21.15.90:0/632553168 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:49.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.743+0000 7f75cfe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 0x55dc5d2970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.743+0000 7f75cea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d296b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.744+0000 7f75cf400700 1 -- 172.21.15.90:0/632553168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 msgr2=0x55dc5d2970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:49.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.744+0000 7f75cf400700 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 0x55dc5d2970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.744+0000 7f75cf400700 1 -- 172.21.15.90:0/632553168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 msgr2=0x55dc5d296b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:49.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.744+0000 7f75cf400700 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d296b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.744+0000 7f75cf400700 1 -- 172.21.15.90:0/632553168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc5d2975f0 con 0x55dc5d270b60 2024-05-27T08:04:49.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.745+0000 7f75cf400700 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d270b60 0x55dc5d287f90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75c00a7a60 tx=0x7f75c00a9f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2825c596ef080f21 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:49.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.745+0000 7f75b9600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:04:49.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.746+0000 7f75b9600700 1 -- 172.21.15.90:0/632553168 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c00b3650 con 0x55dc5d270b60 2024-05-27T08:04:49.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.746+0000 7f75b9600700 1 -- 172.21.15.90:0/632553168 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f75c00a6de0 con 0x55dc5d270b60 2024-05-27T08:04:49.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.746+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/632553168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75bc01ab80 con 0x55dc5d270b60 2024-05-27T08:04:49.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.747+0000 7f75b9600700 1 -- 172.21.15.90:0/632553168 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f75c00ab8f0 con 0x55dc5d270b60 2024-05-27T08:04:49.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.747+0000 7f75e4d0cb80 10 client.38120 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:49.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.747+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/632553168 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55dc5d2fa0b0 con 0x55dc5d270b60 2024-05-27T08:04:49.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.749+0000 7f75b9600700 1 -- 172.21.15.90:0/632553168 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75bc01ab80 con 0x55dc5d270b60 2024-05-27T08:04:49.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.749+0000 7f75b9600700 1 -- 172.21.15.90:0/632553168 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f75c00abcc0 con 0x55dc5d270b60 2024-05-27T08:04:49.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.749+0000 7f75e4d0cb80 10 client.38120 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:49.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.750+0000 7f75e4d0cb80 10 client.38120 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T08:04:49.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.750+0000 7f75e4d0cb80 20 client.38120 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:49.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.750+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55dc5d2fa5a0 0x55dc5d2fc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:49.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.750+0000 7f75e4d0cb80 4 client.38120 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:04:49.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.750+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/632553168 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc5d19f570 con 0x55dc5d2fa5a0 2024-05-27T08:04:49.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.751+0000 7f75cea00700 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55dc5d2fa5a0 0x55dc5d2fc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:49.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.753+0000 7f75cea00700 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55dc5d2fa5a0 0x55dc5d2fc9c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=547b131ce65cbaa3 server_cookie=b8fc944238e3adaa in_seq=0 out_seq=0 2024-05-27T08:04:49.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.753+0000 7f75b9600700 10 client.38120 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T08:04:49.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.754+0000 7f75b9600700 1 -- 172.21.15.90:0/632553168 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc5d19f570 con 0x55dc5d2fa5a0 2024-05-27T08:04:49.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.754+0000 7f75b9600700 10 client.38120 handle_command_reply: tid=0 2024-05-27T08:04:49.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.754+0000 7f75e4d0cb80 1 client.38120 shutdown 2024-05-27T08:04:49.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.755+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/632553168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d270b60 msgr2=0x55dc5d287f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:49.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.755+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d270b60 0x55dc5d287f90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75c00a7a60 tx=0x7f75c00a9f60 comp rx=0 tx=0).stop 2024-05-27T08:04:49.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.755+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/632553168 shutdown_connections 2024-05-27T08:04:49.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.755+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x55dc5d2fa5a0 0x55dc5d2fc9c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.755+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc5d27bd30 0x55dc5d2970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.755+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dc5d270b60 0x55dc5d287f90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.755+0000 7f75e4d0cb80 1 --2- 172.21.15.90:0/632553168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55dc5d275db0 0x55dc5d296b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.755+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/632553168 >> 172.21.15.90:0/632553168 conn(0x55dc5d156fc0 msgr2=0x55dc5d279140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:49.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.756+0000 7f75b9600700 0 client.38120 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T08:04:49.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.756+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/632553168 shutdown_connections 2024-05-27T08:04:49.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.756+0000 7f75e4d0cb80 1 -- 172.21.15.90:0/632553168 wait complete. 2024-05-27T08:04:49.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.756+0000 7f75e4d0cb80 20 client.38120 trim_cache size 0 max 16384 2024-05-27T08:04:49.759 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:04:49.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 -- 172.21.15.90:0/758308520 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75b40b2bf0 msgr2=0x7f75b40b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:49.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75b40b2bf0 0x7f75b40b5100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f75d01022b0 tx=0x7f75bc011040 comp rx=0 tx=0).stop 2024-05-27T08:04:49.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 -- 172.21.15.90:0/758308520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d01054b0 msgr2=0x7f75d01a1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:49.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d01054b0 0x7f75d01a1300 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f75c000e9c0 tx=0x7f75c000ecd0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 -- 172.21.15.90:0/758308520 shutdown_connections 2024-05-27T08:04:49.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75d010e8e0 0x7f75d01a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f75b40b2bf0 0x7f75b40b5100 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75d01054b0 0x7f75d01a1300 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 --2- 172.21.15.90:0/758308520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75d0105e00 0x7f75d01a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:49.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 -- 172.21.15.90:0/758308520 >> 172.21.15.90:0/758308520 conn(0x7f75d00fedb0 msgr2=0x7f75d00ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:49.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 -- 172.21.15.90:0/758308520 shutdown_connections 2024-05-27T08:04:49.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:49.758+0000 7f75d6200700 1 -- 172.21.15.90:0/758308520 wait complete. 2024-05-27T08:04:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.193+0000 7f1f7dc00640 1 -- 172.21.15.90:0/4021849278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f70094f50 msgr2=0x7f1f70095360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.193+0000 7f1f7dc00640 1 --2- 172.21.15.90:0/4021849278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f70094f50 0x7f1f70095360 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009930 tx=0x7f1f6402a420 comp rx=0 tx=0).stop 2024-05-27T08:04:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.194+0000 7f1f7dc00640 1 -- 172.21.15.90:0/4021849278 shutdown_connections 2024-05-27T08:04:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.194+0000 7f1f7dc00640 1 --2- 172.21.15.90:0/4021849278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f700a13d0 0x7f1f700a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.194+0000 7f1f7dc00640 1 --2- 172.21.15.90:0/4021849278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f70094f50 0x7f1f70095360 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.194+0000 7f1f7dc00640 1 --2- 172.21.15.90:0/4021849278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7009b930 0x7f1f70094a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.194+0000 7f1f7dc00640 1 -- 172.21.15.90:0/4021849278 >> 172.21.15.90:0/4021849278 conn(0x7f1f7000a740 msgr2=0x7f1f7000ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.194+0000 7f1f7dc00640 1 -- 172.21.15.90:0/4021849278 shutdown_connections 2024-05-27T08:04:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.194+0000 7f1f7dc00640 1 -- 172.21.15.90:0/4021849278 wait complete. 2024-05-27T08:04:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.195+0000 7f1f7dc00640 1 Processor -- start 2024-05-27T08:04:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.195+0000 7f1f7dc00640 1 -- start start 2024-05-27T08:04:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f7dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f70094f50 0x7f1f70095bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f7dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f7009b930 0x7f1f70096130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f7dc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f700a13d0 0x7f1f70096670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f7dc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1f700a6b30 con 0x7f1f700a13d0 2024-05-27T08:04:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f7dc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f700a69b0 con 0x7f1f70094f50 2024-05-27T08:04:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f7dc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f700a6cb0 con 0x7f1f7009b930 2024-05-27T08:04:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f7d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f700a13d0 0x7f1f70096670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f7d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f700a13d0 0x7f1f70096670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45336/0 (socket says 172.21.15.90:45336) 2024-05-27T08:04:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f7d200640 1 -- 172.21.15.90:0/2440999164 learned_addr learned my addr 172.21.15.90:0/2440999164 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f77400640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f7009b930 0x7f1f70096130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.196+0000 7f1f77e00640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f70094f50 0x7f1f70095bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.197+0000 7f1f7d200640 1 -- 172.21.15.90:0/2440999164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f7009b930 msgr2=0x7f1f70096130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.197+0000 7f1f7d200640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f7009b930 0x7f1f70096130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.197+0000 7f1f7d200640 1 -- 172.21.15.90:0/2440999164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f70094f50 msgr2=0x7f1f70095bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.197+0000 7f1f7d200640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f70094f50 0x7f1f70095bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.197+0000 7f1f7d200640 1 -- 172.21.15.90:0/2440999164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7013b8f0 con 0x7f1f700a13d0 2024-05-27T08:04:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.197+0000 7f1f7d200640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f700a13d0 0x7f1f70096670 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1f78050b30 tx=0x7f1f7807a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.198+0000 7f1f74c00640 1 -- 172.21.15.90:0/2440999164 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f780692b0 con 0x7f1f700a13d0 2024-05-27T08:04:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.198+0000 7f1f7dc00640 1 -- 172.21.15.90:0/2440999164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7013bbe0 con 0x7f1f700a13d0 2024-05-27T08:04:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.198+0000 7f1f74c00640 1 -- 172.21.15.90:0/2440999164 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f7807ada0 con 0x7f1f700a13d0 2024-05-27T08:04:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.198+0000 7f1f74c00640 1 -- 172.21.15.90:0/2440999164 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7807f5e0 con 0x7f1f700a13d0 2024-05-27T08:04:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.198+0000 7f1f7dc00640 1 -- 172.21.15.90:0/2440999164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f7013c0b0 con 0x7f1f700a13d0 2024-05-27T08:04:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.202+0000 7f1f861cc740 1 Processor -- start 2024-05-27T08:04:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.203+0000 7f1f74c00640 1 -- 172.21.15.90:0/2440999164 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1f78080420 con 0x7f1f700a13d0 2024-05-27T08:04:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.204+0000 7f1f861cc740 1 -- start start 2024-05-27T08:04:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.204+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d51860ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.204+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5182432f0 0x55d51860efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.204+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51824b4d0 0x55d51860f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.204+0000 7f1f861cc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d51857fb90 con 0x55d51826c1b0 2024-05-27T08:04:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.204+0000 7f1f861cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d51856f730 con 0x55d51824b4d0 2024-05-27T08:04:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.204+0000 7f1f861cc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d518573770 con 0x55d5182432f0 2024-05-27T08:04:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f77400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d51860ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f77400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d51860ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45352/0 (socket says 172.21.15.90:45352) 2024-05-27T08:04:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f77400640 1 -- 172.21.15.90:0/1453042206 learned_addr learned my addr 172.21.15.90:0/1453042206 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:50.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f74c00640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1f600b29e0 0x7f1f600b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f77e00640 1 --2- 172.21.15.90:0/1453042206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5182432f0 0x55d51860efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f74c00640 1 -- 172.21.15.90:0/2440999164 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f1f7814eda0 con 0x7f1f700a13d0 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f77400640 1 --2- 172.21.15.90:0/1453042206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51824b4d0 0x55d51860f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f7d200640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1f600b29e0 0x7f1f600b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f77400640 1 -- 172.21.15.90:0/1453042206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5182432f0 msgr2=0x55d51860efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f77400640 1 --2- 172.21.15.90:0/1453042206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5182432f0 0x55d51860efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f77400640 1 -- 172.21.15.90:0/1453042206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51824b4d0 msgr2=0x55d51860f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f77400640 1 --2- 172.21.15.90:0/1453042206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51824b4d0 0x55d51860f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.205+0000 7f1f77400640 1 -- 172.21.15.90:0/1453042206 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5185088b0 con 0x55d51826c1b0 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.206+0000 7f1f77400640 1 --2- 172.21.15.90:0/1453042206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d51860ea90 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1f640057c0 tx=0x7f1f64005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36721e3522729577 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.206+0000 7f1f7d200640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1f600b29e0 0x7f1f600b4ef0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1f78115d50 tx=0x7f1f78068d90 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.206+0000 7f1f69600640 1 -- 172.21.15.90:0/1453042206 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64035070 con 0x55d51826c1b0 2024-05-27T08:04:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.206+0000 7f1f69600640 1 -- 172.21.15.90:0/1453042206 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f64003c80 con 0x55d51826c1b0 2024-05-27T08:04:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.206+0000 7f1f69600640 1 -- 172.21.15.90:0/1453042206 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64031c30 con 0x55d51826c1b0 2024-05-27T08:04:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.207+0000 7f1f861cc740 1 -- 172.21.15.90:0/1453042206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 msgr2=0x55d51860ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.207+0000 7f1f861cc740 1 --2- 172.21.15.90:0/1453042206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d51860ea90 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1f640057c0 tx=0x7f1f64005a40 comp rx=0 tx=0).stop 2024-05-27T08:04:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.207+0000 7f1f861cc740 1 -- 172.21.15.90:0/1453042206 shutdown_connections 2024-05-27T08:04:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.207+0000 7f1f861cc740 1 --2- 172.21.15.90:0/1453042206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51824b4d0 0x55d51860f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.207+0000 7f1f861cc740 1 --2- 172.21.15.90:0/1453042206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5182432f0 0x55d51860efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.207+0000 7f1f861cc740 1 --2- 172.21.15.90:0/1453042206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d51860ea90 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.207+0000 7f1f861cc740 1 -- 172.21.15.90:0/1453042206 >> 172.21.15.90:0/1453042206 conn(0x55d518260300 msgr2=0x55d51860b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.207+0000 7f1f861cc740 1 -- 172.21.15.90:0/1453042206 shutdown_connections 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.207+0000 7f1f861cc740 1 -- 172.21.15.90:0/1453042206 wait complete. 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.208+0000 7f1f861cc740 1 -- start start 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.208+0000 7f1f861cc740 1 Processor -- start 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.208+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5182432f0 0x55d518618c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.208+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d51824b4d0 0x55d518619170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518624f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f861cc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d518255790 con 0x55d51826c1b0 2024-05-27T08:04:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f861cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d518517bc0 con 0x55d5182432f0 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f861cc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d51861bb60 con 0x55d51824b4d0 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f861cc740 10 client.0 _ll_register_callbacks cb 0x55d5184f5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f77400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518624f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f77400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518624f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45366/0 (socket says 172.21.15.90:45366) 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f77400640 1 -- 172.21.15.90:0/4066329030 learned_addr learned my addr 172.21.15.90:0/4066329030 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f77e00640 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d51824b4d0 0x55d518619170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f77400640 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5182432f0 0x55d518618c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f77400640 1 -- 172.21.15.90:0/4066329030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d51824b4d0 msgr2=0x55d518619170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f77400640 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d51824b4d0 0x55d518619170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f77400640 1 -- 172.21.15.90:0/4066329030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5182432f0 msgr2=0x55d518618c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f77400640 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5182432f0 0x55d518618c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.209+0000 7f1f77400640 1 -- 172.21.15.90:0/4066329030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d518284210 con 0x55d51826c1b0 2024-05-27T08:04:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.210+0000 7f1f77400640 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518624f20 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f1f64000c00 tx=0x7f1f64009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978df845f517fd3c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.210+0000 7f1f5e000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.210+0000 7f1f5e000640 1 -- 172.21.15.90:0/4066329030 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64035040 con 0x55d51826c1b0 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.210+0000 7f1f5e000640 1 -- 172.21.15.90:0/4066329030 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f1f640319b0 con 0x55d51826c1b0 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.210+0000 7f1f861cc740 1 -- 172.21.15.90:0/4066329030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f64003c80 con 0x55d51826c1b0 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.210+0000 7f1f5e000640 1 -- 172.21.15.90:0/4066329030 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f1f640317a0 con 0x55d51826c1b0 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.210+0000 7f1f861cc740 10 client.18429 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.210+0000 7f1f861cc740 1 -- 172.21.15.90:0/4066329030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55d518350620 con 0x55d51826c1b0 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.211+0000 7f1f5e000640 1 -- 172.21.15.90:0/4066329030 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f64003c80 con 0x55d51826c1b0 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.211+0000 7f1f5e000640 1 -- 172.21.15.90:0/4066329030 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f1f64047690 con 0x55d51826c1b0 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.211+0000 7f1f861cc740 10 client.18429 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.211+0000 7f1f861cc740 10 client.18429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.211+0000 7f1f861cc740 20 client.18429 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:50.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.211+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d51822ad80 0x55d51862acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.211+0000 7f1f861cc740 4 client.18429 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:50.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.211+0000 7f1f861cc740 1 -- 172.21.15.90:0/4066329030 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d518581ae0 con 0x55d51822ad80 2024-05-27T08:04:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.212+0000 7f1f77e00640 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d51822ad80 0x55d51862acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.213+0000 7f1f77e00640 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d51822ad80 0x55d51862acf0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce88992719de9e9d server_cookie=2ddb1f636267af99 in_seq=0 out_seq=0 2024-05-27T08:04:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.213+0000 7f1f5e000640 10 client.18429 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.214+0000 7f1f5e000640 1 -- 172.21.15.90:0/4066329030 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55d518581ae0 con 0x55d51822ad80 2024-05-27T08:04:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.214+0000 7f1f5e000640 10 client.18429 handle_command_reply: tid=0 2024-05-27T08:04:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.214+0000 7f1f861cc740 1 client.18429 shutdown 2024-05-27T08:04:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 1 -- 172.21.15.90:0/4066329030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 msgr2=0x55d518624f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518624f20 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f1f64000c00 tx=0x7f1f64009680 comp rx=0 tx=0).stop 2024-05-27T08:04:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 1 -- 172.21.15.90:0/4066329030 shutdown_connections 2024-05-27T08:04:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d51822ad80 0x55d51862acf0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518624f20 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f5e000640 0 client.18429 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d51824b4d0 0x55d518619170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4066329030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5182432f0 0x55d518618c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 1 -- 172.21.15.90:0/4066329030 >> 172.21.15.90:0/4066329030 conn(0x55d518260300 msgr2=0x55d51860e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 1 -- 172.21.15.90:0/4066329030 shutdown_connections 2024-05-27T08:04:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 1 -- 172.21.15.90:0/4066329030 wait complete. 2024-05-27T08:04:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.215+0000 7f1f861cc740 20 client.18429 trim_cache size 0 max 16384 2024-05-27T08:04:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f861cc740 1 Processor -- start 2024-05-27T08:04:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f861cc740 1 -- start start 2024-05-27T08:04:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5182432f0 0x55d518619180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51824b4d0 0x55d518624c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d51826c1b0 0x55d5186251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f861cc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70147bd0 con 0x55d51824b4d0 2024-05-27T08:04:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f861cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70144c20 con 0x55d5182432f0 2024-05-27T08:04:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f861cc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70146e30 con 0x55d51826c1b0 2024-05-27T08:04:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f77e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51824b4d0 0x55d518624c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f77e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51824b4d0 0x55d518624c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45376/0 (socket says 172.21.15.90:45376) 2024-05-27T08:04:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.231+0000 7f1f77e00640 1 -- 172.21.15.90:0/3416510835 learned_addr learned my addr 172.21.15.90:0/3416510835 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.232+0000 7f1f77e00640 1 -- 172.21.15.90:0/3416510835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d51826c1b0 msgr2=0x55d5186251d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.232+0000 7f1f77e00640 1 --2- 172.21.15.90:0/3416510835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d51826c1b0 0x55d5186251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.232+0000 7f1f77e00640 1 -- 172.21.15.90:0/3416510835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5182432f0 msgr2=0x55d518619180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:50.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.232+0000 7f1f77e00640 1 --2- 172.21.15.90:0/3416510835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5182432f0 0x55d518619180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.232+0000 7f1f77e00640 1 -- 172.21.15.90:0/3416510835 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d518625710 con 0x55d51824b4d0 2024-05-27T08:04:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.232+0000 7f1f77e00640 1 --2- 172.21.15.90:0/3416510835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51824b4d0 0x55d518624c90 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c004630 tx=0x7f1f6c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517bdd116fef35bd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.232+0000 7f1f68c00640 1 -- 172.21.15.90:0/3416510835 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c003e40 con 0x55d51824b4d0 2024-05-27T08:04:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.232+0000 7f1f68c00640 1 -- 172.21.15.90:0/3416510835 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f6c01c900 con 0x55d51824b4d0 2024-05-27T08:04:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.232+0000 7f1f68c00640 1 -- 172.21.15.90:0/3416510835 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c010af0 con 0x55d51824b4d0 2024-05-27T08:04:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.233+0000 7f1f861cc740 1 -- 172.21.15.90:0/3416510835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51824b4d0 msgr2=0x55d518624c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.233+0000 7f1f861cc740 1 --2- 172.21.15.90:0/3416510835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51824b4d0 0x55d518624c90 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c004630 tx=0x7f1f6c00ced0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.234+0000 7f1f861cc740 1 -- 172.21.15.90:0/3416510835 shutdown_connections 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.234+0000 7f1f861cc740 1 --2- 172.21.15.90:0/3416510835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d51826c1b0 0x55d5186251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.234+0000 7f1f861cc740 1 --2- 172.21.15.90:0/3416510835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51824b4d0 0x55d518624c90 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.234+0000 7f1f861cc740 1 --2- 172.21.15.90:0/3416510835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d5182432f0 0x55d518619180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.234+0000 7f1f861cc740 1 -- 172.21.15.90:0/3416510835 >> 172.21.15.90:0/3416510835 conn(0x55d518260300 msgr2=0x55d51861ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.234+0000 7f1f861cc740 1 -- 172.21.15.90:0/3416510835 shutdown_connections 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.234+0000 7f1f861cc740 1 -- 172.21.15.90:0/3416510835 wait complete. 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.234+0000 7f1f861cc740 1 -- start start 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.234+0000 7f1f861cc740 1 Processor -- start 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5182432f0 0x55d51861e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51824b4d0 0x55d51861e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f861cc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518691380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f861cc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70146760 con 0x55d51826c1b0 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f861cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f701452f0 con 0x55d51824b4d0 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f861cc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f701459c0 con 0x55d5182432f0 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f861cc740 10 client.0 _ll_register_callbacks cb 0x7f1f6c00c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f77400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518691380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f77400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518691380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45378/0 (socket says 172.21.15.90:45378) 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f77400640 1 -- 172.21.15.90:0/4142318088 learned_addr learned my addr 172.21.15.90:0/4142318088 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f77e00640 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51824b4d0 0x55d51861e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.235+0000 7f1f77400640 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5182432f0 0x55d51861e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.236+0000 7f1f77400640 1 -- 172.21.15.90:0/4142318088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5182432f0 msgr2=0x55d51861e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.236+0000 7f1f77400640 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5182432f0 0x55d51861e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.236+0000 7f1f77400640 1 -- 172.21.15.90:0/4142318088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51824b4d0 msgr2=0x55d51861e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.236+0000 7f1f77400640 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51824b4d0 0x55d51861e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.236+0000 7f1f77400640 1 -- 172.21.15.90:0/4142318088 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5186918c0 con 0x55d51826c1b0 2024-05-27T08:04:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.236+0000 7f1f77400640 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518691380 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1f640057c0 tx=0x7f1f64005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4940bc6c4d230cf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.236+0000 7f1f5e000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:04:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.236+0000 7f1f5e000640 1 -- 172.21.15.90:0/4142318088 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64035040 con 0x55d51826c1b0 2024-05-27T08:04:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.236+0000 7f1f861cc740 1 -- 172.21.15.90:0/4142318088 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f6c01c900 con 0x55d51826c1b0 2024-05-27T08:04:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.236+0000 7f1f5e000640 1 -- 172.21.15.90:0/4142318088 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f1f64047420 con 0x55d51826c1b0 2024-05-27T08:04:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f5e000640 1 -- 172.21.15.90:0/4142318088 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f1f64009780 con 0x55d51826c1b0 2024-05-27T08:04:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f861cc740 10 client.18435 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f861cc740 1 -- 172.21.15.90:0/4142318088 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55d5186929a0 con 0x55d51826c1b0 2024-05-27T08:04:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f5e000640 1 -- 172.21.15.90:0/4142318088 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f6c01c900 con 0x55d51826c1b0 2024-05-27T08:04:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f5e000640 1 -- 172.21.15.90:0/4142318088 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f1f640035f0 con 0x55d51826c1b0 2024-05-27T08:04:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f861cc740 10 client.18435 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f861cc740 10 client.18435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:04:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f861cc740 20 client.18435 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d5182001c0 0x55d518694d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f861cc740 4 client.18435 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:04:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.237+0000 7f1f861cc740 1 -- 172.21.15.90:0/4142318088 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d51857b2c0 con 0x55d5182001c0 2024-05-27T08:04:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.238+0000 7f1f77e00640 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d5182001c0 0x55d518694d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.239+0000 7f1f5e000640 10 client.18435 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.239+0000 7f1f77e00640 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d5182001c0 0x55d518694d40 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6109d2721388d6bc server_cookie=9b52c6ea41b9ea27 in_seq=0 out_seq=0 2024-05-27T08:04:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.557+0000 7f1f5e000640 1 -- 172.21.15.90:0/4142318088 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d51857b2c0 con 0x55d5182001c0 2024-05-27T08:04:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.557+0000 7f1f5e000640 10 client.18435 handle_command_reply: tid=0 2024-05-27T08:04:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.557+0000 7f1f861cc740 1 client.18435 shutdown 2024-05-27T08:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.557+0000 7f1f861cc740 1 -- 172.21.15.90:0/4142318088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 msgr2=0x55d518691380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.557+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518691380 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f1f640057c0 tx=0x7f1f64005a20 comp rx=0 tx=0).stop 2024-05-27T08:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.558+0000 7f1f861cc740 1 -- 172.21.15.90:0/4142318088 shutdown_connections 2024-05-27T08:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.558+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55d5182001c0 0x55d518694d40 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.558+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51824b4d0 0x55d51861e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.558+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51826c1b0 0x55d518691380 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.558+0000 7f1f861cc740 1 --2- 172.21.15.90:0/4142318088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d5182432f0 0x55d51861e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.558+0000 7f1f861cc740 1 -- 172.21.15.90:0/4142318088 >> 172.21.15.90:0/4142318088 conn(0x55d518260300 msgr2=0x55d51861a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.558+0000 7f1f5e000640 0 client.18435 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.558+0000 7f1f861cc740 1 -- 172.21.15.90:0/4142318088 shutdown_connections 2024-05-27T08:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.559+0000 7f1f861cc740 1 -- 172.21.15.90:0/4142318088 wait complete. 2024-05-27T08:04:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.559+0000 7f1f861cc740 20 client.18435 trim_cache size 0 max 16384 2024-05-27T08:04:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:04:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:04:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:04:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.561+0000 7f1f7dc00640 1 -- 172.21.15.90:0/2440999164 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1f600b29e0 msgr2=0x7f1f600b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.561+0000 7f1f7dc00640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1f600b29e0 0x7f1f600b4ef0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1f78115d50 tx=0x7f1f78068d90 comp rx=0 tx=0).stop 2024-05-27T08:04:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.561+0000 7f1f7dc00640 1 -- 172.21.15.90:0/2440999164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f700a13d0 msgr2=0x7f1f70096670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.561+0000 7f1f7dc00640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f700a13d0 0x7f1f70096670 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1f78050b30 tx=0x7f1f7807a4b0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.562+0000 7f1f7dc00640 1 -- 172.21.15.90:0/2440999164 shutdown_connections 2024-05-27T08:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.562+0000 7f1f7dc00640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1f600b29e0 0x7f1f600b4ef0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.562+0000 7f1f7dc00640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f700a13d0 0x7f1f70096670 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.562+0000 7f1f7dc00640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f7009b930 0x7f1f70096130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.562+0000 7f1f7dc00640 1 --2- 172.21.15.90:0/2440999164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f70094f50 0x7f1f70095bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.562+0000 7f1f7dc00640 1 -- 172.21.15.90:0/2440999164 >> 172.21.15.90:0/2440999164 conn(0x7f1f7000a740 msgr2=0x7f1f70093820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.564+0000 7f1f7dc00640 1 -- 172.21.15.90:0/2440999164 shutdown_connections 2024-05-27T08:04:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:04:50.564+0000 7f1f7dc00640 1 -- 172.21.15.90:0/2440999164 wait complete. 2024-05-27T08:04:50.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:50 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:04:50.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:50 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:04:50.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:50 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:04:50.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:50 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:04:50.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:50 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:04:50.751 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:50 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:04:50.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:50 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:04:50.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:50 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:04:50.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:50 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:04:51.211 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:04:51.659 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:51 smithi090 ceph-mon[31429]: pgmap v595: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-27T08:04:51.659 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:51 smithi090 ceph-mon[31429]: pgmap v596: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-05-27T08:04:51.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:51 smithi129 ceph-mon[35254]: pgmap v595: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-27T08:04:51.816 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:51 smithi129 ceph-mon[35254]: pgmap v596: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-05-27T08:04:51.994 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:04:52.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:51 smithi136 ceph-mon[35324]: pgmap v595: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-05-27T08:04:52.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:51 smithi136 ceph-mon[35324]: pgmap v596: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 459 op/s 2024-05-27T08:04:53.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:53 smithi129 ceph-mon[35254]: pgmap v597: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 509 op/s 2024-05-27T08:04:53.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 875.0 s, 489.4 tps, lat 208.719 ms stddev 131.229 2024-05-27T08:04:53.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:53 smithi090 ceph-mon[31429]: pgmap v597: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 509 op/s 2024-05-27T08:04:54.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:53 smithi136 ceph-mon[35324]: pgmap v597: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.6 MiB/s wr, 509 op/s 2024-05-27T08:04:55.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:55 smithi090 ceph-mon[31429]: pgmap v598: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 484 op/s 2024-05-27T08:04:55.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:55 smithi129 ceph-mon[35254]: pgmap v598: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 484 op/s 2024-05-27T08:04:56.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:55 smithi136 ceph-mon[35324]: pgmap v598: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.5 MiB/s wr, 484 op/s 2024-05-27T08:04:57.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:57 smithi129 ceph-mon[35254]: pgmap v599: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.9 MiB/s wr, 481 op/s 2024-05-27T08:04:57.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:57 smithi090 ceph-mon[31429]: pgmap v599: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.9 MiB/s wr, 481 op/s 2024-05-27T08:04:58.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:57 smithi136 ceph-mon[35324]: pgmap v599: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.9 MiB/s wr, 481 op/s 2024-05-27T08:04:58.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.105+0000 7f9d9de00700 1 -- 172.21.15.90:0/1920140284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d98109360 msgr2=0x7f9d98109770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:58.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.105+0000 7f9d9de00700 1 --2- 172.21.15.90:0/1920140284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d98109360 0x7f9d98109770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009990 tx=0x7f9d88009ca0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.106+0000 7f9d9de00700 1 -- 172.21.15.90:0/1920140284 shutdown_connections 2024-05-27T08:04:58.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.106+0000 7f9d9de00700 1 --2- 172.21.15.90:0/1920140284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d98104290 0x7f9d98110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.106+0000 7f9d9de00700 1 --2- 172.21.15.90:0/1920140284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d98109360 0x7f9d98109770 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.106+0000 7f9d9de00700 1 --2- 172.21.15.90:0/1920140284 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d98103940 0x7f9d98103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.106+0000 7f9d9de00700 1 -- 172.21.15.90:0/1920140284 >> 172.21.15.90:0/1920140284 conn(0x7f9d980fe500 msgr2=0x7f9d98100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:58.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.107+0000 7f9d9de00700 1 -- 172.21.15.90:0/1920140284 shutdown_connections 2024-05-27T08:04:58.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.107+0000 7f9d9de00700 1 -- 172.21.15.90:0/1920140284 wait complete. 2024-05-27T08:04:58.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.109+0000 7f9d9de00700 1 Processor -- start 2024-05-27T08:04:58.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.109+0000 7f9d9de00700 1 -- start start 2024-05-27T08:04:58.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.110+0000 7f9d9de00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d98103940 0x7f9d9810d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.110+0000 7f9d9de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d98104290 0x7f9d9810d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.110+0000 7f9d9de00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d98109360 0x7f9d9810dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.110+0000 7f9d9de00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9d98112230 con 0x7f9d98103940 2024-05-27T08:04:58.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.110+0000 7f9d9de00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981120b0 con 0x7f9d98109360 2024-05-27T08:04:58.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.110+0000 7f9d9de00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981123b0 con 0x7f9d98104290 2024-05-27T08:04:58.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d96000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d98104290 0x7f9d9810d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d96000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d98104290 0x7f9d9810d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:33286/0 (socket says 172.21.15.90:33286) 2024-05-27T08:04:58.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d96000700 1 -- 172.21.15.90:0/849178134 learned_addr learned my addr 172.21.15.90:0/849178134 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:58.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d97400700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d98109360 0x7f9d9810dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d96a00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d98103940 0x7f9d9810d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d96000700 1 -- 172.21.15.90:0/849178134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d98109360 msgr2=0x7f9d9810dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d96000700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d98109360 0x7f9d9810dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d96000700 1 -- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d98103940 msgr2=0x7f9d9810d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d96000700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d98103940 0x7f9d9810d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d96000700 1 -- 172.21.15.90:0/849178134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d981ad690 con 0x7f9d98104290 2024-05-27T08:04:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.111+0000 7f9d96a00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d98103940 0x7f9d9810d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.112+0000 7f9d96000700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d98104290 0x7f9d9810d830 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c9e0 tx=0x7f9d8400cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.113+0000 7f9d9ca00700 1 -- 172.21.15.90:0/849178134 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84007aa0 con 0x7f9d98104290 2024-05-27T08:04:58.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.114+0000 7f9d9ca00700 1 -- 172.21.15.90:0/849178134 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d84007c40 con 0x7f9d98104290 2024-05-27T08:04:58.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.114+0000 7f9d9ca00700 1 -- 172.21.15.90:0/849178134 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84018660 con 0x7f9d98104290 2024-05-27T08:04:58.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.114+0000 7f9d9de00700 1 -- 172.21.15.90:0/849178134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d981ad860 con 0x7f9d98104290 2024-05-27T08:04:58.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.114+0000 7f9d9de00700 1 -- 172.21.15.90:0/849178134 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d981adcf0 con 0x7f9d98104290 2024-05-27T08:04:58.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.120+0000 7f9d9ca00700 1 -- 172.21.15.90:0/849178134 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9d8400f970 con 0x7f9d98104290 2024-05-27T08:04:58.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.124+0000 7f9d9ca00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9d7c0b2bc0 0x7f9d7c0b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.125+0000 7f9d9ca00700 1 -- 172.21.15.90:0/849178134 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f9d840e2950 con 0x7f9d98104290 2024-05-27T08:04:58.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.125+0000 7f9d96a00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9d7c0b2bc0 0x7f9d7c0b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.126+0000 7f9d96a00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9d7c0b2bc0 0x7f9d7c0b50d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d880059d0 tx=0x7f9d88005940 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:58.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.129+0000 7f9dac885b80 1 Processor -- start 2024-05-27T08:04:58.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.130+0000 7f9dac885b80 1 -- start start 2024-05-27T08:04:58.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.131+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 0x55fb694eb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.131+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694e5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.131+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 0x55fb694e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.131+0000 7f9dac885b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fb69413e40 con 0x55fb69366e40 2024-05-27T08:04:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.131+0000 7f9dac885b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb694039e0 con 0x55fb694e5db0 2024-05-27T08:04:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.131+0000 7f9dac885b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fb69407a20 con 0x55fb694ebd30 2024-05-27T08:04:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.131+0000 7f9d97400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 0x55fb694e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.131+0000 7f9d97400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 0x55fb694e61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37530/0 (socket says 172.21.15.90:37530) 2024-05-27T08:04:58.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.131+0000 7f9d97400700 1 -- 172.21.15.90:0/3856220200 learned_addr learned my addr 172.21.15.90:0/3856220200 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:58.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.132+0000 7f9d96000700 1 --2- 172.21.15.90:0/3856220200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694e5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.132+0000 7f9d96a00700 1 --2- 172.21.15.90:0/3856220200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 0x55fb694eb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.132+0000 7f9d96000700 1 -- 172.21.15.90:0/3856220200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 msgr2=0x55fb694e61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:58.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.132+0000 7f9d96000700 1 --2- 172.21.15.90:0/3856220200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 0x55fb694e61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.132+0000 7f9d96000700 1 -- 172.21.15.90:0/3856220200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 msgr2=0x55fb694eb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:58.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.132+0000 7f9d96000700 1 --2- 172.21.15.90:0/3856220200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 0x55fb694eb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.132+0000 7f9d96000700 1 -- 172.21.15.90:0/3856220200 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb693cc420 con 0x55fb694ebd30 2024-05-27T08:04:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.133+0000 7f9d96000700 1 --2- 172.21.15.90:0/3856220200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694e5870 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9d840dfc70 tx=0x7f9d840a9080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=757498029d7ef8f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.134+0000 7f9d8cc00700 1 -- 172.21.15.90:0/3856220200 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840adbb0 con 0x55fb694ebd30 2024-05-27T08:04:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.134+0000 7f9d8cc00700 1 -- 172.21.15.90:0/3856220200 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d840b2660 con 0x55fb694ebd30 2024-05-27T08:04:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.134+0000 7f9d8cc00700 1 -- 172.21.15.90:0/3856220200 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d840b21d0 con 0x55fb694ebd30 2024-05-27T08:04:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.135+0000 7f9dac885b80 1 -- 172.21.15.90:0/3856220200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 msgr2=0x55fb694e5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.135+0000 7f9dac885b80 1 --2- 172.21.15.90:0/3856220200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694e5870 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9d840dfc70 tx=0x7f9d840a9080 comp rx=0 tx=0).stop 2024-05-27T08:04:58.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.136+0000 7f9dac885b80 1 -- 172.21.15.90:0/3856220200 shutdown_connections 2024-05-27T08:04:58.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.136+0000 7f9dac885b80 1 --2- 172.21.15.90:0/3856220200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 0x55fb694e61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.136+0000 7f9dac885b80 1 --2- 172.21.15.90:0/3856220200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 0x55fb694eb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.136+0000 7f9dac885b80 1 --2- 172.21.15.90:0/3856220200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694e5870 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.136+0000 7f9dac885b80 1 -- 172.21.15.90:0/3856220200 >> 172.21.15.90:0/3856220200 conn(0x55fb693c6fc0 msgr2=0x55fb694e08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:58.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.136+0000 7f9dac885b80 1 -- 172.21.15.90:0/3856220200 shutdown_connections 2024-05-27T08:04:58.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.136+0000 7f9dac885b80 1 -- 172.21.15.90:0/3856220200 wait complete. 2024-05-27T08:04:58.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.137+0000 7f9dac885b80 1 -- start start 2024-05-27T08:04:58.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.137+0000 7f9dac885b80 1 Processor -- start 2024-05-27T08:04:58.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.138+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.138+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 0x55fb694efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.138+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694effe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.138+0000 7f9dac885b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fb69309fa0 con 0x55fb694e5db0 2024-05-27T08:04:58.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.138+0000 7f9dac885b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fb693a4760 con 0x55fb69366e40 2024-05-27T08:04:58.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.138+0000 7f9dac885b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fb694f2800 con 0x55fb694ebd30 2024-05-27T08:04:58.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.138+0000 7f9dac885b80 10 client.0 _ll_register_callbacks cb 0x55fb693adc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:58.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.139+0000 7f9d96a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.139+0000 7f9d96a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694ef560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37546/0 (socket says 172.21.15.90:37546) 2024-05-27T08:04:58.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.139+0000 7f9d96a00700 1 -- 172.21.15.90:0/257574019 learned_addr learned my addr 172.21.15.90:0/257574019 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:58.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.139+0000 7f9d97400700 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694effe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.140+0000 7f9d96000700 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 0x55fb694efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.140+0000 7f9d96a00700 1 -- 172.21.15.90:0/257574019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 msgr2=0x55fb694effe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:58.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.140+0000 7f9d96a00700 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694effe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.140+0000 7f9d96a00700 1 -- 172.21.15.90:0/257574019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 msgr2=0x55fb694efaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:58.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.140+0000 7f9d96a00700 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 0x55fb694efaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.140+0000 7f9d96a00700 1 -- 172.21.15.90:0/257574019 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb693538d0 con 0x55fb69366e40 2024-05-27T08:04:58.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.141+0000 7f9d96000700 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 0x55fb694efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:04:58.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.141+0000 7f9d96a00700 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694ef560 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009990 tx=0x7f9d88009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94b01f381ae5123f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:58.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.141+0000 7f9d81600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:04:58.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.142+0000 7f9d81600700 1 -- 172.21.15.90:0/257574019 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88024420 con 0x55fb69366e40 2024-05-27T08:04:58.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.142+0000 7f9d81600700 1 -- 172.21.15.90:0/257574019 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f9d8801e030 con 0x55fb69366e40 2024-05-27T08:04:58.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.143+0000 7f9dac885b80 1 -- 172.21.15.90:0/257574019 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d840b2660 con 0x55fb69366e40 2024-05-27T08:04:58.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.144+0000 7f9d81600700 1 -- 172.21.15.90:0/257574019 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f9d880245c0 con 0x55fb69366e40 2024-05-27T08:04:58.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.145+0000 7f9dac885b80 10 client.38138 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:58.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.145+0000 7f9dac885b80 1 -- 172.21.15.90:0/257574019 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55fb693d4060 con 0x55fb69366e40 2024-05-27T08:04:58.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.145+0000 7f9d81600700 1 -- 172.21.15.90:0/257574019 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d840b2660 con 0x55fb69366e40 2024-05-27T08:04:58.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.146+0000 7f9d81600700 1 -- 172.21.15.90:0/257574019 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f9d88023820 con 0x55fb69366e40 2024-05-27T08:04:58.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.146+0000 7f9dac885b80 10 client.38138 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:58.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.146+0000 7f9dac885b80 10 client.38138 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:04:58.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.146+0000 7f9dac885b80 20 client.38138 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:58.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.146+0000 7f9dac885b80 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55fb695108a0 0x55fb69512d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.146+0000 7f9dac885b80 4 client.38138 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:04:58.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.146+0000 7f9dac885b80 1 -- 172.21.15.90:0/257574019 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb69415d90 con 0x55fb695108a0 2024-05-27T08:04:58.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.147+0000 7f9d96000700 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55fb695108a0 0x55fb69512d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.149+0000 7f9d81600700 10 client.38138 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T08:04:58.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.150+0000 7f9d96000700 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55fb695108a0 0x55fb69512d70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca4b3587f625714e server_cookie=5995636bd71063b5 in_seq=0 out_seq=0 2024-05-27T08:04:58.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.151+0000 7f9d81600700 1 -- 172.21.15.90:0/257574019 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55fb69415d90 con 0x55fb695108a0 2024-05-27T08:04:58.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.151+0000 7f9d81600700 10 client.38138 handle_command_reply: tid=0 2024-05-27T08:04:58.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.152+0000 7f9dac885b80 1 client.38138 shutdown 2024-05-27T08:04:58.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.152+0000 7f9dac885b80 1 -- 172.21.15.90:0/257574019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 msgr2=0x55fb694ef560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:58.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.152+0000 7f9dac885b80 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694ef560 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009990 tx=0x7f9d88009310 comp rx=0 tx=0).stop 2024-05-27T08:04:58.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.153+0000 7f9dac885b80 1 -- 172.21.15.90:0/257574019 shutdown_connections 2024-05-27T08:04:58.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.153+0000 7f9dac885b80 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55fb695108a0 0x55fb69512d70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.153+0000 7f9dac885b80 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694effe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.153+0000 7f9dac885b80 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694ef560 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.153+0000 7f9dac885b80 1 --2- 172.21.15.90:0/257574019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 0x55fb694efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.153+0000 7f9dac885b80 1 -- 172.21.15.90:0/257574019 >> 172.21.15.90:0/257574019 conn(0x55fb693c6fc0 msgr2=0x55fb694e0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:58.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.153+0000 7f9d81600700 0 client.38138 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T08:04:58.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.153+0000 7f9dac885b80 1 -- 172.21.15.90:0/257574019 shutdown_connections 2024-05-27T08:04:58.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.153+0000 7f9dac885b80 1 -- 172.21.15.90:0/257574019 wait complete. 2024-05-27T08:04:58.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.154+0000 7f9dac885b80 20 client.38138 trim_cache size 0 max 16384 2024-05-27T08:04:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.185+0000 7f9dac885b80 1 Processor -- start 2024-05-27T08:04:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.185+0000 7f9dac885b80 1 -- start start 2024-05-27T08:04:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.185+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 0x55fb694e5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.186+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 0x55fb69512f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.186+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.186+0000 7f9dac885b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981afd50 con 0x55fb69366e40 2024-05-27T08:04:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.186+0000 7f9dac885b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9811e750 con 0x55fb694e5db0 2024-05-27T08:04:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.186+0000 7f9dac885b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981aefb0 con 0x55fb694ebd30 2024-05-27T08:04:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.186+0000 7f9d96a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 0x55fb694e5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.186+0000 7f9d97400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.186+0000 7f9d96a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 0x55fb694e5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45412/0 (socket says 172.21.15.90:45412) 2024-05-27T08:04:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.186+0000 7f9d96a00700 1 -- 172.21.15.90:0/2371582088 learned_addr learned my addr 172.21.15.90:0/2371582088 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.187+0000 7f9d96000700 1 --2- 172.21.15.90:0/2371582088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 0x55fb69512f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.187+0000 7f9d96a00700 1 -- 172.21.15.90:0/2371582088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 msgr2=0x55fb694f1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.187+0000 7f9d96a00700 1 --2- 172.21.15.90:0/2371582088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694f1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.187+0000 7f9d96a00700 1 -- 172.21.15.90:0/2371582088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 msgr2=0x55fb69512f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.187+0000 7f9d96a00700 1 --2- 172.21.15.90:0/2371582088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 0x55fb69512f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.187+0000 7f9d96a00700 1 -- 172.21.15.90:0/2371582088 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb694f2360 con 0x55fb69366e40 2024-05-27T08:04:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.187+0000 7f9d96000700 1 --2- 172.21.15.90:0/2371582088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 0x55fb69512f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:04:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.188+0000 7f9d96a00700 1 --2- 172.21.15.90:0/2371582088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 0x55fb694e5870 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9d88006010 tx=0x7f9d88009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca01f8326696f5cc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.189+0000 7f9d83e00700 1 -- 172.21.15.90:0/2371582088 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88023e80 con 0x55fb69366e40 2024-05-27T08:04:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.189+0000 7f9d83e00700 1 -- 172.21.15.90:0/2371582088 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d880238a0 con 0x55fb69366e40 2024-05-27T08:04:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.189+0000 7f9d83e00700 1 -- 172.21.15.90:0/2371582088 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88005020 con 0x55fb69366e40 2024-05-27T08:04:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.191+0000 7f9dac885b80 1 -- 172.21.15.90:0/2371582088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 msgr2=0x55fb694e5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:58.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.191+0000 7f9dac885b80 1 --2- 172.21.15.90:0/2371582088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 0x55fb694e5870 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9d88006010 tx=0x7f9d88009ca0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.192+0000 7f9dac885b80 1 -- 172.21.15.90:0/2371582088 shutdown_connections 2024-05-27T08:04:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.192+0000 7f9dac885b80 1 --2- 172.21.15.90:0/2371582088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb694f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.192+0000 7f9dac885b80 1 --2- 172.21.15.90:0/2371582088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb69366e40 0x55fb694e5870 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.192+0000 7f9dac885b80 1 --2- 172.21.15.90:0/2371582088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb694e5db0 0x55fb69512f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.192+0000 7f9dac885b80 1 -- 172.21.15.90:0/2371582088 >> 172.21.15.90:0/2371582088 conn(0x55fb693c6fc0 msgr2=0x55fb694e9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:58.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.192+0000 7f9dac885b80 1 -- 172.21.15.90:0/2371582088 shutdown_connections 2024-05-27T08:04:58.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.193+0000 7f9dac885b80 1 -- 172.21.15.90:0/2371582088 wait complete. 2024-05-27T08:04:58.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.194+0000 7f9dac885b80 1 -- start start 2024-05-27T08:04:58.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.194+0000 7f9dac885b80 1 Processor -- start 2024-05-27T08:04:58.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.197+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.197+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 0x55fb694f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.198+0000 7f9d96a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.198+0000 7f9d96a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:37560/0 (socket says 172.21.15.90:37560) 2024-05-27T08:04:58.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.198+0000 7f9dac885b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb69517520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.198+0000 7f9d96a00700 1 -- 172.21.15.90:0/264871864 learned_addr learned my addr 172.21.15.90:0/264871864 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:04:58.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.198+0000 7f9d96000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 0x55fb694f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.198+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9d981ae8e0 con 0x55fb694e5db0 2024-05-27T08:04:58.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.198+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9811ee20 con 0x55fb69366e40 2024-05-27T08:04:58.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.198+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9811f4f0 con 0x55fb694ebd30 2024-05-27T08:04:58.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.198+0000 7f9dac885b80 10 client.0 _ll_register_callbacks cb 0x55fb693adc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:04:58.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.200+0000 7f9d97400700 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb69517520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.200+0000 7f9d96a00700 1 -- 172.21.15.90:0/264871864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 msgr2=0x55fb69517520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:58.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.200+0000 7f9d96a00700 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb69517520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.200+0000 7f9d96a00700 1 -- 172.21.15.90:0/264871864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 msgr2=0x55fb694f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:04:58.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.200+0000 7f9d96a00700 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 0x55fb694f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.200+0000 7f9d96a00700 1 -- 172.21.15.90:0/264871864 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb69518f60 con 0x55fb69366e40 2024-05-27T08:04:58.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.201+0000 7f9d96a00700 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694f7a50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9d8801f040 tx=0x7f9d88023ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce716e00b81771e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:04:58.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.201+0000 7f9d81600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:04:58.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.201+0000 7f9d81600700 1 -- 172.21.15.90:0/264871864 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d88044050 con 0x55fb69366e40 2024-05-27T08:04:58.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.201+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d880238a0 con 0x55fb69366e40 2024-05-27T08:04:58.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.201+0000 7f9d81600700 1 -- 172.21.15.90:0/264871864 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f9d8803d040 con 0x55fb69366e40 2024-05-27T08:04:58.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.202+0000 7f9d81600700 1 -- 172.21.15.90:0/264871864 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f9d88004c70 con 0x55fb69366e40 2024-05-27T08:04:58.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.202+0000 7f9dac885b80 10 client.38144 fetch_fsmap learned FSMap version 64 2024-05-27T08:04:58.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.202+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55fb6957a730 con 0x55fb69366e40 2024-05-27T08:04:58.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.202+0000 7f9d81600700 1 -- 172.21.15.90:0/264871864 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9d880238a0 con 0x55fb69366e40 2024-05-27T08:04:58.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.202+0000 7f9d81600700 1 -- 172.21.15.90:0/264871864 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f9d8803f070 con 0x55fb69366e40 2024-05-27T08:04:58.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.202+0000 7f9dac885b80 10 client.38144 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:04:58.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.203+0000 7f9dac885b80 10 client.38144 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:04:58.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.203+0000 7f9dac885b80 20 client.38144 populate_metadata read hostname 'smithi090' 2024-05-27T08:04:58.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.203+0000 7f9dac885b80 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55fb6957abe0 0x55fb6957d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:04:58.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.203+0000 7f9dac885b80 4 client.38144 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:04:58.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.203+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb6940f570 con 0x55fb6957abe0 2024-05-27T08:04:58.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.205+0000 7f9d96000700 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55fb6957abe0 0x55fb6957d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:04:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.209+0000 7f9d96000700 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55fb6957abe0 0x55fb6957d000 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aff531245508a3bb server_cookie=77b84849e8448670 in_seq=0 out_seq=0 2024-05-27T08:04:58.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.209+0000 7f9d81600700 10 client.38144 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.209+0000 7f9d81600700 1 -- 172.21.15.90:0/264871864 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb6940f570 con 0x55fb6957abe0 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.209+0000 7f9d81600700 10 client.38144 handle_command_reply: tid=0 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.209+0000 7f9dac885b80 1 client.38144 shutdown 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 msgr2=0x55fb694f7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9dac885b80 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694f7a50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9d8801f040 tx=0x7f9d88023ac0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 shutdown_connections 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9dac885b80 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55fb6957abe0 0x55fb6957d000 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9dac885b80 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb694ebd30 0x55fb69517520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9dac885b80 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fb69366e40 0x55fb694f7a50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9dac885b80 1 --2- 172.21.15.90:0/264871864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb694e5db0 0x55fb694f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9d81600700 0 client.38144 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T08:04:58.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 >> 172.21.15.90:0/264871864 conn(0x55fb693c6fc0 msgr2=0x55fb694ef500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:58.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 shutdown_connections 2024-05-27T08:04:58.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.210+0000 7f9dac885b80 1 -- 172.21.15.90:0/264871864 wait complete. 2024-05-27T08:04:58.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.211+0000 7f9dac885b80 20 client.38144 trim_cache size 0 max 16384 2024-05-27T08:04:58.213 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:04:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.212+0000 7f9d9de00700 1 -- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9d7c0b2bc0 msgr2=0x7f9d7c0b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.212+0000 7f9d9de00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9d7c0b2bc0 0x7f9d7c0b50d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9d880059d0 tx=0x7f9d88005940 comp rx=0 tx=0).stop 2024-05-27T08:04:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.212+0000 7f9d9de00700 1 -- 172.21.15.90:0/849178134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d98104290 msgr2=0x7f9d9810d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:04:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.212+0000 7f9d9de00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d98104290 0x7f9d9810d830 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400c9e0 tx=0x7f9d8400cda0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.212+0000 7f9d9de00700 1 -- 172.21.15.90:0/849178134 shutdown_connections 2024-05-27T08:04:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.212+0000 7f9d9de00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9d7c0b2bc0 0x7f9d7c0b50d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.212+0000 7f9d9de00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9d98109360 0x7f9d9810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.212+0000 7f9d9de00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9d98103940 0x7f9d9810d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.212+0000 7f9d9de00700 1 --2- 172.21.15.90:0/849178134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d98104290 0x7f9d9810d830 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:04:58.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.212+0000 7f9d9de00700 1 -- 172.21.15.90:0/849178134 >> 172.21.15.90:0/849178134 conn(0x7f9d980fe500 msgr2=0x7f9d98100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:04:58.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.213+0000 7f9d9de00700 1 -- 172.21.15.90:0/849178134 shutdown_connections 2024-05-27T08:04:58.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:04:58.214+0000 7f9d9de00700 1 -- 172.21.15.90:0/849178134 wait complete. 2024-05-27T08:04:58.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 880.0 s, 517.2 tps, lat 192.132 ms stddev 97.956 2024-05-27T08:04:59.582 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:04:59.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:04:59 smithi090 ceph-mon[31429]: pgmap v600: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.6 MiB/s wr, 441 op/s 2024-05-27T08:05:00.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:04:59 smithi136 ceph-mon[35324]: pgmap v600: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.6 MiB/s wr, 441 op/s 2024-05-27T08:05:00.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:04:59 smithi129 ceph-mon[35254]: pgmap v600: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.6 MiB/s wr, 441 op/s 2024-05-27T08:05:00.163 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:01.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:01 smithi090 ceph-mon[31429]: pgmap v601: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 511 op/s 2024-05-27T08:05:02.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:01 smithi136 ceph-mon[35324]: pgmap v601: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 511 op/s 2024-05-27T08:05:02.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:01 smithi129 ceph-mon[35254]: pgmap v601: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.7 MiB/s wr, 511 op/s 2024-05-27T08:05:03.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:03 smithi090 ceph-mon[31429]: pgmap v602: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-05-27T08:05:03.830 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 885.0 s, 553.9 tps, lat 177.407 ms stddev 183.137 2024-05-27T08:05:03.871 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:03 smithi129 ceph-mon[35254]: pgmap v602: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-05-27T08:05:03.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.917+0000 7f940be00640 1 -- 172.21.15.90:0/2809647801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9404112180 msgr2=0x7f94041145a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:03.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.917+0000 7f940be00640 1 --2- 172.21.15.90:0/2809647801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9404112180 0x7f94041145a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009900 tx=0x7f93f002a400 comp rx=0 tx=0).stop 2024-05-27T08:05:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.917+0000 7f940be00640 1 -- 172.21.15.90:0/2809647801 shutdown_connections 2024-05-27T08:05:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.917+0000 7f940be00640 1 --2- 172.21.15.90:0/2809647801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404114ae0 0x7f9404116f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.917+0000 7f940be00640 1 --2- 172.21.15.90:0/2809647801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9404112180 0x7f94041145a0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.917+0000 7f940be00640 1 --2- 172.21.15.90:0/2809647801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404077610 0x7f9404077a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.917+0000 7f940be00640 1 -- 172.21.15.90:0/2809647801 >> 172.21.15.90:0/2809647801 conn(0x7f9404101530 msgr2=0x7f9404103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.918+0000 7f940be00640 1 -- 172.21.15.90:0/2809647801 shutdown_connections 2024-05-27T08:05:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.918+0000 7f940be00640 1 -- 172.21.15.90:0/2809647801 wait complete. 2024-05-27T08:05:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.919+0000 7f940be00640 1 Processor -- start 2024-05-27T08:05:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.920+0000 7f940be00640 1 -- start start 2024-05-27T08:05:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.920+0000 7f940be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404077610 0x7f940410ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.920+0000 7f940be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9404112180 0x7f940410b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.920+0000 7f940be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404114ae0 0x7f940410b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.920+0000 7f940be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f940411a330 con 0x7f9404112180 2024-05-27T08:05:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.920+0000 7f940be00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f940411a1b0 con 0x7f9404114ae0 2024-05-27T08:05:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.920+0000 7f940be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f940411a4b0 con 0x7f9404077610 2024-05-27T08:05:03.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.921+0000 7f9409e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404114ae0 0x7f940410b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.921+0000 7f9409e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404114ae0 0x7f940410b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50886/0 (socket says 172.21.15.90:50886) 2024-05-27T08:05:03.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.921+0000 7f9409e00640 1 -- 172.21.15.90:0/1388862147 learned_addr learned my addr 172.21.15.90:0/1388862147 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.921+0000 7f9409400640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404077610 0x7f940410ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.922+0000 7f9409400640 1 -- 172.21.15.90:0/1388862147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404114ae0 msgr2=0x7f940410b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.922+0000 7f9409400640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404114ae0 0x7f940410b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.922+0000 7f9409400640 1 -- 172.21.15.90:0/1388862147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9404112180 msgr2=0x7f940410b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.922+0000 7f9409400640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9404112180 0x7f940410b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.922+0000 7f9409400640 1 -- 172.21.15.90:0/1388862147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940410f810 con 0x7f9404077610 2024-05-27T08:05:03.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.923+0000 7f9409400640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404077610 0x7f940410ae60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f93f800e9f0 tx=0x7f93f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.924+0000 7f9402000640 1 -- 172.21.15.90:0/1388862147 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f800ccc0 con 0x7f9404077610 2024-05-27T08:05:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.924+0000 7f9402000640 1 -- 172.21.15.90:0/1388862147 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f80108c0 con 0x7f9404077610 2024-05-27T08:05:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.924+0000 7f940be00640 1 -- 172.21.15.90:0/1388862147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940410fb00 con 0x7f9404077610 2024-05-27T08:05:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.924+0000 7f9402000640 1 -- 172.21.15.90:0/1388862147 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8018710 con 0x7f9404077610 2024-05-27T08:05:03.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.924+0000 7f940be00640 1 -- 172.21.15.90:0/1388862147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9404110070 con 0x7f9404077610 2024-05-27T08:05:03.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.929+0000 7f9412dcd740 1 Processor -- start 2024-05-27T08:05:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.930+0000 7f9412dcd740 1 -- start start 2024-05-27T08:05:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.931+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600729031b0 0x560072ca5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.931+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728da2f0 0x560072ca5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.931+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072ca62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.931+0000 7f9412dcd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560072c16b90 con 0x5600729031b0 2024-05-27T08:05:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.931+0000 7f9412dcd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560072c06730 con 0x5600728da2f0 2024-05-27T08:05:03.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.931+0000 7f9412dcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560072c0a770 con 0x5600728e24d0 2024-05-27T08:05:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.931+0000 7f9402000640 1 -- 172.21.15.90:0/1388862147 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f93f80188b0 con 0x7f9404077610 2024-05-27T08:05:03.933 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.932+0000 7f9408a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072ca62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.932+0000 7f9408a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072ca62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42412/0 (socket says 172.21.15.90:42412) 2024-05-27T08:05:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.932+0000 7f9408a00640 1 -- 172.21.15.90:0/1592722755 learned_addr learned my addr 172.21.15.90:0/1592722755 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.932+0000 7f9409400640 1 --2- 172.21.15.90:0/1592722755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728da2f0 0x560072ca5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.933+0000 7f9408a00640 1 -- 172.21.15.90:0/1592722755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728da2f0 msgr2=0x560072ca5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.933+0000 7f9408a00640 1 --2- 172.21.15.90:0/1592722755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728da2f0 0x560072ca5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.933+0000 7f9408a00640 1 -- 172.21.15.90:0/1592722755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600729031b0 msgr2=0x560072ca5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.933+0000 7f9408a00640 1 --2- 172.21.15.90:0/1592722755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600729031b0 0x560072ca5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.933+0000 7f9408a00640 1 -- 172.21.15.90:0/1592722755 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560072b9f8b0 con 0x5600728e24d0 2024-05-27T08:05:03.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.934+0000 7f9402000640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f93ec0b2a20 0x7f93ec0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.934+0000 7f9402000640 1 -- 172.21.15.90:0/1388862147 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f93f80e2f00 con 0x7f9404077610 2024-05-27T08:05:03.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.934+0000 7f9409e00640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f93ec0b2a20 0x7f93ec0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:03.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.935+0000 7f9409e00640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f93ec0b2a20 0x7f93ec0b4f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x560072ca29d0 tx=0x7f93fc009b50 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:03.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.936+0000 7f9408a00640 1 --2- 172.21.15.90:0/1592722755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072ca62e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f93f00057c0 tx=0x7f93f00040d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6643e8384fb2b62 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.940+0000 7f93f6600640 1 -- 172.21.15.90:0/1592722755 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f003e030 con 0x5600728e24d0 2024-05-27T08:05:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.940+0000 7f93f6600640 1 -- 172.21.15.90:0/1592722755 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f0004240 con 0x5600728e24d0 2024-05-27T08:05:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.940+0000 7f93f6600640 1 -- 172.21.15.90:0/1592722755 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0035040 con 0x5600728e24d0 2024-05-27T08:05:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.943+0000 7f9412dcd740 1 -- 172.21.15.90:0/1592722755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 msgr2=0x560072ca62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.943+0000 7f9412dcd740 1 --2- 172.21.15.90:0/1592722755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072ca62e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f93f00057c0 tx=0x7f93f00040d0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.943+0000 7f9412dcd740 1 -- 172.21.15.90:0/1592722755 shutdown_connections 2024-05-27T08:05:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.943+0000 7f9412dcd740 1 --2- 172.21.15.90:0/1592722755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728da2f0 0x560072ca5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.943+0000 7f9412dcd740 1 --2- 172.21.15.90:0/1592722755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072ca62e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.943+0000 7f9412dcd740 1 --2- 172.21.15.90:0/1592722755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600729031b0 0x560072ca5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.943+0000 7f9412dcd740 1 -- 172.21.15.90:0/1592722755 >> 172.21.15.90:0/1592722755 conn(0x5600728f7300 msgr2=0x560072ca2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:03.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.944+0000 7f9412dcd740 1 -- 172.21.15.90:0/1592722755 shutdown_connections 2024-05-27T08:05:03.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.944+0000 7f9412dcd740 1 -- 172.21.15.90:0/1592722755 wait complete. 2024-05-27T08:05:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.946+0000 7f9412dcd740 1 -- start start 2024-05-27T08:05:03.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.946+0000 7f9412dcd740 1 Processor -- start 2024-05-27T08:05:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.947+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 0x560072caf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.947+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072caff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.949+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.949+0000 7f9412dcd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5600728ec790 con 0x5600728da2f0 2024-05-27T08:05:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.949+0000 7f9412dcd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560072baebc0 con 0x5600729031b0 2024-05-27T08:05:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.949+0000 7f9412dcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560072cb2930 con 0x5600728e24d0 2024-05-27T08:05:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.949+0000 7f9412dcd740 10 client.0 _ll_register_callbacks cb 0x560072b8c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.950+0000 7f9409e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.950+0000 7f9409e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50896/0 (socket says 172.21.15.90:50896) 2024-05-27T08:05:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.950+0000 7f9409e00640 1 -- 172.21.15.90:0/364854719 learned_addr learned my addr 172.21.15.90:0/364854719 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.950+0000 7f9408a00640 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072caff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.950+0000 7f9409400640 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 0x560072caf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.951+0000 7f9409400640 1 -- 172.21.15.90:0/364854719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 msgr2=0x560072caff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.951+0000 7f9409400640 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072caff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.951+0000 7f9409400640 1 -- 172.21.15.90:0/364854719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 msgr2=0x560072cbbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.951+0000 7f9409400640 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.951+0000 7f9409400640 1 -- 172.21.15.90:0/364854719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56007291b210 con 0x5600728da2f0 2024-05-27T08:05:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.952+0000 7f9409400640 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 0x560072caf9d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f93f80e0280 tx=0x7f93f80e6640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4fe99889a66380 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.952+0000 7f93eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.952+0000 7f93eaa00640 1 -- 172.21.15.90:0/364854719 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f80a9920 con 0x5600728da2f0 2024-05-27T08:05:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.953+0000 7f93eaa00640 1 -- 172.21.15.90:0/364854719 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f93f80b27e0 con 0x5600728da2f0 2024-05-27T08:05:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.953+0000 7f9412dcd740 1 -- 172.21.15.90:0/364854719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93f0004240 con 0x5600728da2f0 2024-05-27T08:05:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.954+0000 7f93eaa00640 1 -- 172.21.15.90:0/364854719 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f93f80aa510 con 0x5600728da2f0 2024-05-27T08:05:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.955+0000 7f9412dcd740 10 client.18453 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.955+0000 7f9412dcd740 1 -- 172.21.15.90:0/364854719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5600729e7620 con 0x5600728da2f0 2024-05-27T08:05:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.955+0000 7f93eaa00640 1 -- 172.21.15.90:0/364854719 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f0004240 con 0x5600728da2f0 2024-05-27T08:05:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.961+0000 7f93eaa00640 1 -- 172.21.15.90:0/364854719 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f93f8018ba0 con 0x5600728da2f0 2024-05-27T08:05:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.961+0000 7f9412dcd740 10 client.18453 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.961+0000 7f9412dcd740 10 client.18453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.961+0000 7f9412dcd740 20 client.18453 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.961+0000 7f9412dcd740 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5600728c1d80 0x560072cc1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.961+0000 7f9412dcd740 4 client.18453 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.961+0000 7f9412dcd740 1 -- 172.21.15.90:0/364854719 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560072c18ae0 con 0x5600728c1d80 2024-05-27T08:05:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.963+0000 7f9408a00640 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5600728c1d80 0x560072cc1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.964+0000 7f9408a00640 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5600728c1d80 0x560072cc1cc0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8318cd0e961c03b5 server_cookie=f6247cad525ca145 in_seq=0 out_seq=0 2024-05-27T08:05:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.965+0000 7f93eaa00640 10 client.18453 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:03.970 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.969+0000 7f93eaa00640 1 -- 172.21.15.90:0/364854719 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x560072c18ae0 con 0x5600728c1d80 2024-05-27T08:05:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.969+0000 7f93eaa00640 10 client.18453 handle_command_reply: tid=0 2024-05-27T08:05:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.970+0000 7f9412dcd740 1 client.18453 shutdown 2024-05-27T08:05:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.976+0000 7f9412dcd740 1 -- 172.21.15.90:0/364854719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 msgr2=0x560072caf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.976+0000 7f9412dcd740 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 0x560072caf9d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f93f80e0280 tx=0x7f93f80e6640 comp rx=0 tx=0).stop 2024-05-27T08:05:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.976+0000 7f9412dcd740 1 -- 172.21.15.90:0/364854719 shutdown_connections 2024-05-27T08:05:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.976+0000 7f9412dcd740 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5600728c1d80 0x560072cc1cc0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.976+0000 7f9412dcd740 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.976+0000 7f9412dcd740 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072caff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.976+0000 7f9412dcd740 1 --2- 172.21.15.90:0/364854719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 0x560072caf9d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.976+0000 7f9412dcd740 1 -- 172.21.15.90:0/364854719 >> 172.21.15.90:0/364854719 conn(0x5600728f7300 msgr2=0x560072ca7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.977+0000 7f93eaa00640 0 client.18453 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.977+0000 7f9412dcd740 1 -- 172.21.15.90:0/364854719 shutdown_connections 2024-05-27T08:05:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.978+0000 7f9412dcd740 1 -- 172.21.15.90:0/364854719 wait complete. 2024-05-27T08:05:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:03.978+0000 7f9412dcd740 20 client.18453 trim_cache size 0 max 16384 2024-05-27T08:05:04.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:03 smithi136 ceph-mon[35324]: pgmap v602: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-05-27T08:05:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.011+0000 7f9412dcd740 1 Processor -- start 2024-05-27T08:05:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.011+0000 7f9412dcd740 1 -- start start 2024-05-27T08:05:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.012+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 0x560072caff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.012+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072cb10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.012+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.012+0000 7f9412dcd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9404123700 con 0x5600728da2f0 2024-05-27T08:05:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.012+0000 7f9412dcd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f940406a270 con 0x5600729031b0 2024-05-27T08:05:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.012+0000 7f9412dcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9404122960 con 0x5600728e24d0 2024-05-27T08:05:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.013+0000 7f9409e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.013+0000 7f9409e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50900/0 (socket says 172.21.15.90:50900) 2024-05-27T08:05:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.013+0000 7f9409e00640 1 -- 172.21.15.90:0/2036501886 learned_addr learned my addr 172.21.15.90:0/2036501886 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.013+0000 7f9408a00640 1 --2- 172.21.15.90:0/2036501886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072cb10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.013+0000 7f9409e00640 1 -- 172.21.15.90:0/2036501886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 msgr2=0x560072cb10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.013+0000 7f9409e00640 1 --2- 172.21.15.90:0/2036501886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072cb10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.013+0000 7f9409400640 1 --2- 172.21.15.90:0/2036501886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 0x560072caff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.013+0000 7f9409e00640 1 -- 172.21.15.90:0/2036501886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 msgr2=0x560072caff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.014+0000 7f9409e00640 1 --2- 172.21.15.90:0/2036501886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 0x560072caff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.014+0000 7f9409e00640 1 -- 172.21.15.90:0/2036501886 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560072cb1fa0 con 0x5600729031b0 2024-05-27T08:05:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.015+0000 7f9409400640 1 --2- 172.21.15.90:0/2036501886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 0x560072caff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.015+0000 7f9409e00640 1 --2- 172.21.15.90:0/2036501886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbb9d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f93fc01c790 tx=0x7f93fc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a739f8c9012c9f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.018+0000 7f93f5c00640 1 -- 172.21.15.90:0/2036501886 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc02a070 con 0x5600729031b0 2024-05-27T08:05:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.018+0000 7f93f5c00640 1 -- 172.21.15.90:0/2036501886 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93fc01ad70 con 0x5600729031b0 2024-05-27T08:05:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.018+0000 7f93f5c00640 1 -- 172.21.15.90:0/2036501886 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc025820 con 0x5600729031b0 2024-05-27T08:05:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.019+0000 7f9412dcd740 1 -- 172.21.15.90:0/2036501886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 msgr2=0x560072cbb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.019+0000 7f9412dcd740 1 --2- 172.21.15.90:0/2036501886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbb9d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f93fc01c790 tx=0x7f93fc014000 comp rx=0 tx=0).stop 2024-05-27T08:05:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.019+0000 7f9412dcd740 1 -- 172.21.15.90:0/2036501886 shutdown_connections 2024-05-27T08:05:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.019+0000 7f9412dcd740 1 --2- 172.21.15.90:0/2036501886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600729031b0 0x560072cbb9d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.019+0000 7f9412dcd740 1 --2- 172.21.15.90:0/2036501886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728e24d0 0x560072cb10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.019+0000 7f9412dcd740 1 --2- 172.21.15.90:0/2036501886 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600728da2f0 0x560072caff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.019+0000 7f9412dcd740 1 -- 172.21.15.90:0/2036501886 >> 172.21.15.90:0/2036501886 conn(0x5600728f7300 msgr2=0x560072caac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.019+0000 7f9412dcd740 1 -- 172.21.15.90:0/2036501886 shutdown_connections 2024-05-27T08:05:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.020+0000 7f9412dcd740 1 -- 172.21.15.90:0/2036501886 wait complete. 2024-05-27T08:05:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.020+0000 7f9412dcd740 1 -- start start 2024-05-27T08:05:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.021+0000 7f9412dcd740 1 Processor -- start 2024-05-27T08:05:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.021+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728da2f0 0x560072cb31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.021+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728e24d0 0x560072cb36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.021+0000 7f9412dcd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600729031b0 0x560072d28350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.021+0000 7f9412dcd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9404122290 con 0x5600729031b0 2024-05-27T08:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.021+0000 7f9412dcd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9404120e20 con 0x5600728e24d0 2024-05-27T08:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.021+0000 7f9412dcd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94041214f0 con 0x5600728da2f0 2024-05-27T08:05:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.021+0000 7f9412dcd740 10 client.0 _ll_register_callbacks cb 0x7f93f80e6740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.022+0000 7f9409400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728da2f0 0x560072cb31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.022+0000 7f9409e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600729031b0 0x560072d28350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.022+0000 7f9408a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728e24d0 0x560072cb36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.022+0000 7f9408a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728e24d0 0x560072cb36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50916/0 (socket says 172.21.15.90:50916) 2024-05-27T08:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.022+0000 7f9408a00640 1 -- 172.21.15.90:0/2599769868 learned_addr learned my addr 172.21.15.90:0/2599769868 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.023+0000 7f9409e00640 1 -- 172.21.15.90:0/2599769868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728da2f0 msgr2=0x560072cb31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.023+0000 7f9409e00640 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728da2f0 0x560072cb31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.023+0000 7f9409e00640 1 -- 172.21.15.90:0/2599769868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728e24d0 msgr2=0x560072cb36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.023+0000 7f9409e00640 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728e24d0 0x560072cb36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.023+0000 7f9409e00640 1 -- 172.21.15.90:0/2599769868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560072d28890 con 0x5600729031b0 2024-05-27T08:05:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.024+0000 7f9409400640 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728da2f0 0x560072cb31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.024+0000 7f9408a00640 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728e24d0 0x560072cb36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:05:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.024+0000 7f9409e00640 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600729031b0 0x560072d28350 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x560072cb0a20 tx=0x7f93fc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5e475271c24149 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.025+0000 7f93eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.025+0000 7f93eaa00640 1 -- 172.21.15.90:0/2599769868 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc02a040 con 0x5600729031b0 2024-05-27T08:05:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.025+0000 7f9412dcd740 1 -- 172.21.15.90:0/2599769868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93fc01ad70 con 0x5600729031b0 2024-05-27T08:05:04.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.025+0000 7f93eaa00640 1 -- 172.21.15.90:0/2599769868 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f93fc01a8e0 con 0x5600729031b0 2024-05-27T08:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.026+0000 7f93eaa00640 1 -- 172.21.15.90:0/2599769868 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f93fc03d050 con 0x5600729031b0 2024-05-27T08:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.026+0000 7f9412dcd740 10 client.18456 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.026+0000 7f9412dcd740 1 -- 172.21.15.90:0/2599769868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x560072d29a30 con 0x5600729031b0 2024-05-27T08:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.026+0000 7f93eaa00640 1 -- 172.21.15.90:0/2599769868 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93fc01ad70 con 0x5600729031b0 2024-05-27T08:05:04.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.026+0000 7f93eaa00640 1 -- 172.21.15.90:0/2599769868 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f93fc038030 con 0x5600729031b0 2024-05-27T08:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.027+0000 7f9412dcd740 10 client.18456 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.027+0000 7f9412dcd740 10 client.18456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.027+0000 7f9412dcd740 20 client.18456 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.027+0000 7f9412dcd740 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5600728971c0 0x560072d2c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.027+0000 7f9412dcd740 4 client.18456 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:05:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.027+0000 7f9412dcd740 1 -- 172.21.15.90:0/2599769868 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560072c122c0 con 0x5600728971c0 2024-05-27T08:05:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.028+0000 7f9409400640 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5600728971c0 0x560072d2c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.032+0000 7f93eaa00640 10 client.18456 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.032+0000 7f9409400640 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5600728971c0 0x560072d2c1a0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b6a53669596ac9 server_cookie=26a789cde7f2ea7f in_seq=0 out_seq=0 2024-05-27T08:05:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.161+0000 7f93eaa00640 1 -- 172.21.15.90:0/2599769868 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560072c122c0 con 0x5600728971c0 2024-05-27T08:05:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.161+0000 7f93eaa00640 10 client.18456 handle_command_reply: tid=0 2024-05-27T08:05:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.161+0000 7f9412dcd740 1 client.18456 shutdown 2024-05-27T08:05:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.162+0000 7f9412dcd740 1 -- 172.21.15.90:0/2599769868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600729031b0 msgr2=0x560072d28350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.162+0000 7f9412dcd740 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600729031b0 0x560072d28350 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x560072cb0a20 tx=0x7f93fc01c500 comp rx=0 tx=0).stop 2024-05-27T08:05:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.162+0000 7f9412dcd740 1 -- 172.21.15.90:0/2599769868 shutdown_connections 2024-05-27T08:05:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.162+0000 7f9412dcd740 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5600728971c0 0x560072d2c1a0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.163+0000 7f9412dcd740 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5600728e24d0 0x560072cb36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.163+0000 7f9412dcd740 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5600729031b0 0x560072d28350 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.163+0000 7f9412dcd740 1 --2- 172.21.15.90:0/2599769868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5600728da2f0 0x560072cb31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.163+0000 7f9412dcd740 1 -- 172.21.15.90:0/2599769868 >> 172.21.15.90:0/2599769868 conn(0x5600728f7300 msgr2=0x560072ca55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.163+0000 7f93eaa00640 0 client.18456 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.163+0000 7f9412dcd740 1 -- 172.21.15.90:0/2599769868 shutdown_connections 2024-05-27T08:05:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.163+0000 7f9412dcd740 1 -- 172.21.15.90:0/2599769868 wait complete. 2024-05-27T08:05:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.163+0000 7f9412dcd740 20 client.18456 trim_cache size 0 max 16384 2024-05-27T08:05:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:05:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:05:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:05:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.165+0000 7f940be00640 1 -- 172.21.15.90:0/1388862147 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f93ec0b2a20 msgr2=0x7f93ec0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.165+0000 7f940be00640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f93ec0b2a20 0x7f93ec0b4f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x560072ca29d0 tx=0x7f93fc009b50 comp rx=0 tx=0).stop 2024-05-27T08:05:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.165+0000 7f940be00640 1 -- 172.21.15.90:0/1388862147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404077610 msgr2=0x7f940410ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.165+0000 7f940be00640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404077610 0x7f940410ae60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f93f800e9f0 tx=0x7f93f800eeb0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.165+0000 7f940be00640 1 -- 172.21.15.90:0/1388862147 shutdown_connections 2024-05-27T08:05:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.165+0000 7f940be00640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f93ec0b2a20 0x7f93ec0b4f30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.165+0000 7f940be00640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9404114ae0 0x7f940410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.165+0000 7f940be00640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9404112180 0x7f940410b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.165+0000 7f940be00640 1 --2- 172.21.15.90:0/1388862147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404077610 0x7f940410ae60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.165+0000 7f940be00640 1 -- 172.21.15.90:0/1388862147 >> 172.21.15.90:0/1388862147 conn(0x7f9404101530 msgr2=0x7f9404101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.166+0000 7f940be00640 1 -- 172.21.15.90:0/1388862147 shutdown_connections 2024-05-27T08:05:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:04.166+0000 7f940be00640 1 -- 172.21.15.90:0/1388862147 wait complete. 2024-05-27T08:05:04.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.563+0000 7f004e000700 1 -- 172.21.15.90:0/838822044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0048109860 msgr2=0x7f00481111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.563+0000 7f004e000700 1 --2- 172.21.15.90:0/838822044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0048109860 0x7f00481111c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0040009fc0 tx=0x7f004000b330 comp rx=0 tx=0).stop 2024-05-27T08:05:04.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.564+0000 7f004e000700 1 -- 172.21.15.90:0/838822044 shutdown_connections 2024-05-27T08:05:04.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.564+0000 7f004e000700 1 --2- 172.21.15.90:0/838822044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0048109860 0x7f00481111c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.564+0000 7f004e000700 1 --2- 172.21.15.90:0/838822044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00481045e0 0x7f00481049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.565+0000 7f004e000700 1 --2- 172.21.15.90:0/838822044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0048108f10 0x7f0048109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.565+0000 7f004e000700 1 -- 172.21.15.90:0/838822044 >> 172.21.15.90:0/838822044 conn(0x7f0048100030 msgr2=0x7f0048102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:04.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.567+0000 7f004e000700 1 -- 172.21.15.90:0/838822044 shutdown_connections 2024-05-27T08:05:04.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.567+0000 7f004e000700 1 -- 172.21.15.90:0/838822044 wait complete. 2024-05-27T08:05:04.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.569+0000 7f004e000700 1 Processor -- start 2024-05-27T08:05:04.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.569+0000 7f004e000700 1 -- start start 2024-05-27T08:05:04.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.570+0000 7f004e000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00481045e0 0x7f00480775a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.570+0000 7f004e000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0048108f10 0x7f00480726d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.570+0000 7f004e000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0048109860 0x7f0048072c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.570+0000 7f004e000700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0048114390 con 0x7f0048109860 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.570+0000 7f004e000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0048114210 con 0x7f00481045e0 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.570+0000 7f004e000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0048114510 con 0x7f0048108f10 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.570+0000 7f0046a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0048108f10 0x7f00480726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.570+0000 7f0046a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0048108f10 0x7f00480726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42472/0 (socket says 172.21.15.90:42472) 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.570+0000 7f0046a00700 1 -- 172.21.15.90:0/1926309403 learned_addr learned my addr 172.21.15.90:0/1926309403 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.570+0000 7f0047400700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00481045e0 0x7f00480775a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.571+0000 7f0047e00700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0048109860 0x7f0048072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.571+0000 7f0046a00700 1 -- 172.21.15.90:0/1926309403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00481045e0 msgr2=0x7f00480775a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.571+0000 7f0046a00700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00481045e0 0x7f00480775a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.571+0000 7f0046a00700 1 -- 172.21.15.90:0/1926309403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0048109860 msgr2=0x7f0048072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.571+0000 7f0046a00700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0048109860 0x7f0048072c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.571+0000 7f0046a00700 1 -- 172.21.15.90:0/1926309403 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00480734a0 con 0x7f0048108f10 2024-05-27T08:05:04.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.572+0000 7f0046a00700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0048108f10 0x7f00480726d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f003400d980 tx=0x7f003400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:04.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.577+0000 7f003fe00700 1 -- 172.21.15.90:0/1926309403 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034017070 con 0x7f0048108f10 2024-05-27T08:05:04.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.577+0000 7f003fe00700 1 -- 172.21.15.90:0/1926309403 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0034010460 con 0x7f0048108f10 2024-05-27T08:05:04.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.577+0000 7f004e000700 1 -- 172.21.15.90:0/1926309403 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00481b1970 con 0x7f0048108f10 2024-05-27T08:05:04.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.577+0000 7f004e000700 1 -- 172.21.15.90:0/1926309403 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00481b1eb0 con 0x7f0048108f10 2024-05-27T08:05:04.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.577+0000 7f003fe00700 1 -- 172.21.15.90:0/1926309403 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003400de60 con 0x7f0048108f10 2024-05-27T08:05:04.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.586+0000 7f003fe00700 1 -- 172.21.15.90:0/1926309403 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f003400f620 con 0x7f0048108f10 2024-05-27T08:05:04.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.588+0000 7f003fe00700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f002c0b2b40 0x7f002c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.589+0000 7f003fe00700 1 -- 172.21.15.90:0/1926309403 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f00340e3290 con 0x7f0048108f10 2024-05-27T08:05:04.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.589+0000 7f0047400700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f002c0b2b40 0x7f002c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.609+0000 7f0047400700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f002c0b2b40 0x7f002c0b5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0038005950 tx=0x7f00380058e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:04.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.613+0000 7f005cb31b80 1 Processor -- start 2024-05-27T08:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.613+0000 7f005cb31b80 1 -- start start 2024-05-27T08:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.613+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 0x5646eb13a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.613+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 0x5646eb13a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.613+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb14f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.613+0000 7f005cb31b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5646eb068e40 con 0x5646eb0a79f0 2024-05-27T08:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.613+0000 7f005cb31b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5646eb0589e0 con 0x5646eafbbe40 2024-05-27T08:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.614+0000 7f005cb31b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5646eb05ca20 con 0x5646eb13adb0 2024-05-27T08:05:04.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.616+0000 7f0047400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 0x5646eb13a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.616+0000 7f0046a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 0x5646eb13a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.616+0000 7f0047e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.616+0000 7f0047e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb14f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42474/0 (socket says 172.21.15.90:42474) 2024-05-27T08:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.616+0000 7f0047e00700 1 -- 172.21.15.90:0/3416103820 learned_addr learned my addr 172.21.15.90:0/3416103820 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.618+0000 7f0047e00700 1 -- 172.21.15.90:0/3416103820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 msgr2=0x5646eb13a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.618+0000 7f0047e00700 1 --2- 172.21.15.90:0/3416103820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 0x5646eb13a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.618+0000 7f0047e00700 1 -- 172.21.15.90:0/3416103820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 msgr2=0x5646eb13a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.618+0000 7f0047e00700 1 --2- 172.21.15.90:0/3416103820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 0x5646eb13a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.618+0000 7f0047e00700 1 -- 172.21.15.90:0/3416103820 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646eb021420 con 0x5646eb13adb0 2024-05-27T08:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.620+0000 7f0047e00700 1 --2- 172.21.15.90:0/3416103820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb14f940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0040009f90 tx=0x7f0040009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=918f6a7d76f561a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.620+0000 7f003cc00700 1 -- 172.21.15.90:0/3416103820 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004001f030 con 0x5646eb13adb0 2024-05-27T08:05:04.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.620+0000 7f003cc00700 1 -- 172.21.15.90:0/3416103820 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f004000bcb0 con 0x5646eb13adb0 2024-05-27T08:05:04.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.620+0000 7f003cc00700 1 -- 172.21.15.90:0/3416103820 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040007c40 con 0x5646eb13adb0 2024-05-27T08:05:04.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.621+0000 7f005cb31b80 1 -- 172.21.15.90:0/3416103820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 msgr2=0x5646eb14f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.621+0000 7f005cb31b80 1 --2- 172.21.15.90:0/3416103820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb14f940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0040009f90 tx=0x7f0040009790 comp rx=0 tx=0).stop 2024-05-27T08:05:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.624+0000 7f005cb31b80 1 -- 172.21.15.90:0/3416103820 shutdown_connections 2024-05-27T08:05:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.624+0000 7f005cb31b80 1 --2- 172.21.15.90:0/3416103820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb14f940 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.624+0000 7f005cb31b80 1 --2- 172.21.15.90:0/3416103820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 0x5646eb13a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.624+0000 7f005cb31b80 1 --2- 172.21.15.90:0/3416103820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 0x5646eb13a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.624+0000 7f005cb31b80 1 -- 172.21.15.90:0/3416103820 >> 172.21.15.90:0/3416103820 conn(0x5646eb055390 msgr2=0x5646eb1364c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:04.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.625+0000 7f005cb31b80 1 -- 172.21.15.90:0/3416103820 shutdown_connections 2024-05-27T08:05:04.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.625+0000 7f005cb31b80 1 -- 172.21.15.90:0/3416103820 wait complete. 2024-05-27T08:05:04.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.626+0000 7f005cb31b80 1 -- start start 2024-05-27T08:05:04.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.626+0000 7f005cb31b80 1 Processor -- start 2024-05-27T08:05:04.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.628+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 0x5646eb14ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.628+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb0a79f0 0x5646eb14d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.628+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb13adb0 0x5646eb14d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.629+0000 7f0046a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb0a79f0 0x5646eb14d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.629+0000 7f0047e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb13adb0 0x5646eb14d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.629+0000 7f0047e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb13adb0 0x5646eb14d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50940/0 (socket says 172.21.15.90:50940) 2024-05-27T08:05:04.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.629+0000 7f0047e00700 1 -- 172.21.15.90:0/635586067 learned_addr learned my addr 172.21.15.90:0/635586067 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:04.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.630+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5646eaf5efa0 con 0x5646eafbbe40 2024-05-27T08:05:04.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.630+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5646eaff9760 con 0x5646eb13adb0 2024-05-27T08:05:04.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.630+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5646eb150380 con 0x5646eb0a79f0 2024-05-27T08:05:04.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.630+0000 7f0047400700 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 0x5646eb14ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.630+0000 7f005cb31b80 10 client.0 _ll_register_callbacks cb 0x5646eb002c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:04.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.631+0000 7f0047e00700 1 -- 172.21.15.90:0/635586067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb0a79f0 msgr2=0x5646eb14d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.631+0000 7f0047e00700 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb0a79f0 0x5646eb14d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.631+0000 7f0047e00700 1 -- 172.21.15.90:0/635586067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 msgr2=0x5646eb14ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.631+0000 7f0047e00700 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 0x5646eb14ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.631+0000 7f0047e00700 1 -- 172.21.15.90:0/635586067 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646eafa88d0 con 0x5646eb13adb0 2024-05-27T08:05:04.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.632+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004000bcb0 con 0x5646eb13adb0 2024-05-27T08:05:04.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.633+0000 7f0047400700 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 0x5646eb14ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:05:04.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.633+0000 7f0047e00700 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb13adb0 0x5646eb14d890 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f004000b770 tx=0x7f0040007ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a23994b05e0567 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:04.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.633+0000 7f0031600700 10 client.38174 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:05:04.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.634+0000 7f0031600700 1 -- 172.21.15.90:0/635586067 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040028020 con 0x5646eb13adb0 2024-05-27T08:05:04.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.634+0000 7f0031600700 1 -- 172.21.15.90:0/635586067 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f0040007c70 con 0x5646eb13adb0 2024-05-27T08:05:04.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.634+0000 7f0031600700 1 -- 172.21.15.90:0/635586067 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f004001f040 con 0x5646eb13adb0 2024-05-27T08:05:04.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.634+0000 7f005cb31b80 10 client.38174 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:04.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.634+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({fsmap=64}) -- 0x5646eb029060 con 0x5646eb13adb0 2024-05-27T08:05:04.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.635+0000 7f0031600700 1 -- 172.21.15.90:0/635586067 <== mon.2 v2:172.21.15.129:3300/0 4 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f0040011070 con 0x5646eb13adb0 2024-05-27T08:05:04.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.635+0000 7f005cb31b80 10 client.38174 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:04.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.635+0000 7f005cb31b80 10 client.38174 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:05:04.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.635+0000 7f005cb31b80 20 client.38174 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:04.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.635+0000 7f005cb31b80 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5646eb15d070 0x5646eb15f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.635+0000 7f005cb31b80 4 client.38174 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:04.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.635+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646eb06ad90 con 0x5646eb15d070 2024-05-27T08:05:04.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.636+0000 7f0047400700 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5646eb15d070 0x5646eb15f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.639+0000 7f0031600700 10 client.38174 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T08:05:04.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.639+0000 7f0047400700 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5646eb15d070 0x5646eb15f490 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70b64d465e3f25bd server_cookie=648be866a85f5082 in_seq=0 out_seq=0 2024-05-27T08:05:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.642+0000 7f0031600700 1 -- 172.21.15.90:0/635586067 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5646eb06ad90 con 0x5646eb15d070 2024-05-27T08:05:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.642+0000 7f0031600700 10 client.38174 handle_command_reply: tid=0 2024-05-27T08:05:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.642+0000 7f005cb31b80 1 client.38174 shutdown 2024-05-27T08:05:04.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.643+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb13adb0 msgr2=0x5646eb14d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.643+0000 7f005cb31b80 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb13adb0 0x5646eb14d890 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f004000b770 tx=0x7f0040007ad0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.643+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 shutdown_connections 2024-05-27T08:05:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.643+0000 7f005cb31b80 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5646eb15d070 0x5646eb15f490 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.643+0000 7f005cb31b80 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb13adb0 0x5646eb14d890 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.643+0000 7f005cb31b80 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 0x5646eb14ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.643+0000 7f005cb31b80 1 --2- 172.21.15.90:0/635586067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb0a79f0 0x5646eb14d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.643+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 >> 172.21.15.90:0/635586067 conn(0x5646eb055390 msgr2=0x5646eb136010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.643+0000 7f0031600700 0 client.38174 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T08:05:04.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.647+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 shutdown_connections 2024-05-27T08:05:04.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.647+0000 7f005cb31b80 1 -- 172.21.15.90:0/635586067 wait complete. 2024-05-27T08:05:04.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.647+0000 7f005cb31b80 20 client.38174 trim_cache size 0 max 16384 2024-05-27T08:05:04.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.677+0000 7f005cb31b80 1 Processor -- start 2024-05-27T08:05:04.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.678+0000 7f005cb31b80 1 -- start start 2024-05-27T08:05:04.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.678+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 0x5646eb148630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.679+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 0x5646eb14bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.679+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb159480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.680+0000 7f005cb31b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f00280107e0 con 0x5646eb0a79f0 2024-05-27T08:05:04.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.680+0000 7f005cb31b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f002800d830 con 0x5646eafbbe40 2024-05-27T08:05:04.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.680+0000 7f005cb31b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f002800fa40 con 0x5646eb13adb0 2024-05-27T08:05:04.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.681+0000 7f0047400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 0x5646eb148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.681+0000 7f0046a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 0x5646eb14bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.681+0000 7f0046a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 0x5646eb14bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49314/0 (socket says 172.21.15.90:49314) 2024-05-27T08:05:04.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.681+0000 7f0046a00700 1 -- 172.21.15.90:0/392138374 learned_addr learned my addr 172.21.15.90:0/392138374 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:04.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.682+0000 7f0046a00700 1 -- 172.21.15.90:0/392138374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 msgr2=0x5646eb159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.682+0000 7f0047e00700 1 --2- 172.21.15.90:0/392138374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.682+0000 7f0046a00700 1 --2- 172.21.15.90:0/392138374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.682+0000 7f0046a00700 1 -- 172.21.15.90:0/392138374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 msgr2=0x5646eb148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.682+0000 7f0046a00700 1 --2- 172.21.15.90:0/392138374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 0x5646eb148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.682+0000 7f0046a00700 1 -- 172.21.15.90:0/392138374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646eb1502c0 con 0x5646eb0a79f0 2024-05-27T08:05:04.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.683+0000 7f0047400700 1 --2- 172.21.15.90:0/392138374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 0x5646eb148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:04.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.683+0000 7f0047e00700 1 --2- 172.21.15.90:0/392138374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:04.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.683+0000 7f0046a00700 1 --2- 172.21.15.90:0/392138374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 0x5646eb14bd70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f00340aa130 tx=0x7f00340aa9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175769157a0aba4f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:04.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.684+0000 7f0033e00700 1 -- 172.21.15.90:0/392138374 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034020050 con 0x5646eb0a79f0 2024-05-27T08:05:04.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.685+0000 7f0033e00700 1 -- 172.21.15.90:0/392138374 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00340b3000 con 0x5646eb0a79f0 2024-05-27T08:05:04.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.685+0000 7f0033e00700 1 -- 172.21.15.90:0/392138374 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00340a9400 con 0x5646eb0a79f0 2024-05-27T08:05:04.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.686+0000 7f005cb31b80 1 -- 172.21.15.90:0/392138374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 msgr2=0x5646eb14bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.686+0000 7f005cb31b80 1 --2- 172.21.15.90:0/392138374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 0x5646eb14bd70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f00340aa130 tx=0x7f00340aa9f0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.686+0000 7f005cb31b80 1 -- 172.21.15.90:0/392138374 shutdown_connections 2024-05-27T08:05:04.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.686+0000 7f005cb31b80 1 --2- 172.21.15.90:0/392138374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.686+0000 7f005cb31b80 1 --2- 172.21.15.90:0/392138374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eafbbe40 0x5646eb148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.686+0000 7f005cb31b80 1 --2- 172.21.15.90:0/392138374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eb0a79f0 0x5646eb14bd70 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.686+0000 7f005cb31b80 1 -- 172.21.15.90:0/392138374 >> 172.21.15.90:0/392138374 conn(0x5646eb055390 msgr2=0x5646eb13bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:04.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.687+0000 7f005cb31b80 1 -- 172.21.15.90:0/392138374 shutdown_connections 2024-05-27T08:05:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.687+0000 7f005cb31b80 1 -- 172.21.15.90:0/392138374 wait complete. 2024-05-27T08:05:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.688+0000 7f005cb31b80 1 -- start start 2024-05-27T08:05:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.688+0000 7f005cb31b80 1 Processor -- start 2024-05-27T08:05:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.689+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 0x5646eb155d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.689+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb0a79f0 0x5646eb147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.690+0000 7f005cb31b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb147c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.690+0000 7f005cb31b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f002800f370 con 0x5646eafbbe40 2024-05-27T08:05:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.690+0000 7f005cb31b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f002800df00 con 0x5646eb0a79f0 2024-05-27T08:05:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.690+0000 7f005cb31b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f002800e5d0 con 0x5646eb13adb0 2024-05-27T08:05:04.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.690+0000 7f005cb31b80 10 client.0 _ll_register_callbacks cb 0x5646eb002c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:04.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.690+0000 7f0047400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 0x5646eb155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.690+0000 7f0046a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb0a79f0 0x5646eb147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.690+0000 7f0046a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb0a79f0 0x5646eb147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:50962/0 (socket says 172.21.15.90:50962) 2024-05-27T08:05:04.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.690+0000 7f0046a00700 1 -- 172.21.15.90:0/3978926281 learned_addr learned my addr 172.21.15.90:0/3978926281 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:04.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.691+0000 7f0047e00700 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.691+0000 7f0046a00700 1 -- 172.21.15.90:0/3978926281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 msgr2=0x5646eb147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.691+0000 7f0046a00700 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.691+0000 7f0046a00700 1 -- 172.21.15.90:0/3978926281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 msgr2=0x5646eb155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:04.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.691+0000 7f0046a00700 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 0x5646eb155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.691+0000 7f0046a00700 1 -- 172.21.15.90:0/3978926281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646eb1483a0 con 0x5646eb0a79f0 2024-05-27T08:05:04.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.692+0000 7f0046a00700 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb0a79f0 0x5646eb147660 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f00340a7410 tx=0x7f003400b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f67342843872541 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:04.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.692+0000 7f0031600700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:05:04.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.692+0000 7f0031600700 1 -- 172.21.15.90:0/3978926281 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00340b2d70 con 0x5646eb0a79f0 2024-05-27T08:05:04.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.693+0000 7f0031600700 1 -- 172.21.15.90:0/3978926281 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f0034009b40 con 0x5646eb0a79f0 2024-05-27T08:05:04.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.693+0000 7f005cb31b80 1 -- 172.21.15.90:0/3978926281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00340b3000 con 0x5646eb0a79f0 2024-05-27T08:05:04.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.694+0000 7f0031600700 1 -- 172.21.15.90:0/3978926281 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f0034020030 con 0x5646eb0a79f0 2024-05-27T08:05:04.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.694+0000 7f005cb31b80 10 client.38177 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:04.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.694+0000 7f005cb31b80 1 -- 172.21.15.90:0/3978926281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5646eb1c7930 con 0x5646eb0a79f0 2024-05-27T08:05:04.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.694+0000 7f0031600700 1 -- 172.21.15.90:0/3978926281 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0034009d40 con 0x5646eb0a79f0 2024-05-27T08:05:04.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.694+0000 7f0031600700 1 -- 172.21.15.90:0/3978926281 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f00340a8e70 con 0x5646eb0a79f0 2024-05-27T08:05:04.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.694+0000 7f005cb31b80 10 client.38177 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:04.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.695+0000 7f005cb31b80 10 client.38177 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:05:04.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.695+0000 7f005cb31b80 20 client.38177 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:04.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.695+0000 7f005cb31b80 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5646eb1c7e20 0x5646eb1ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:04.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.695+0000 7f005cb31b80 4 client.38177 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:05:04.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.695+0000 7f005cb31b80 1 -- 172.21.15.90:0/3978926281 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f00340b3000 con 0x5646eb1c7e20 2024-05-27T08:05:04.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.695+0000 7f0047400700 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5646eb1c7e20 0x5646eb1ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:04.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.697+0000 7f0031600700 10 client.38177 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T08:05:04.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.697+0000 7f0047400700 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5646eb1c7e20 0x5646eb1ca240 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db284c2578a0a4a1 server_cookie=67d9757c4bfc1c9c in_seq=0 out_seq=0 2024-05-27T08:05:04.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.698+0000 7f0031600700 1 -- 172.21.15.90:0/3978926281 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f00340b3000 con 0x5646eb1c7e20 2024-05-27T08:05:04.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.699+0000 7f0031600700 10 client.38177 handle_command_reply: tid=0 2024-05-27T08:05:04.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.699+0000 7f005cb31b80 1 client.38177 shutdown 2024-05-27T08:05:04.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.700+0000 7f005cb31b80 1 -- 172.21.15.90:0/3978926281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb0a79f0 msgr2=0x5646eb147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.700+0000 7f005cb31b80 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb0a79f0 0x5646eb147660 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f00340a7410 tx=0x7f003400b4f0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.703+0000 7f005cb31b80 1 -- 172.21.15.90:0/3978926281 shutdown_connections 2024-05-27T08:05:04.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.703+0000 7f005cb31b80 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5646eb13adb0 0x5646eb147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.703+0000 7f005cb31b80 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5646eafbbe40 0x5646eb155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.703+0000 7f005cb31b80 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x5646eb1c7e20 0x5646eb1ca240 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.703+0000 7f005cb31b80 1 --2- 172.21.15.90:0/3978926281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5646eb0a79f0 0x5646eb147660 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.703+0000 7f005cb31b80 1 -- 172.21.15.90:0/3978926281 >> 172.21.15.90:0/3978926281 conn(0x5646eb055390 msgr2=0x5646eb14e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:04.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.703+0000 7f0031600700 0 client.38177 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T08:05:04.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.705+0000 7f005cb31b80 1 -- 172.21.15.90:0/3978926281 shutdown_connections 2024-05-27T08:05:04.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.706+0000 7f005cb31b80 1 -- 172.21.15.90:0/3978926281 wait complete. 2024-05-27T08:05:04.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.706+0000 7f005cb31b80 20 client.38177 trim_cache size 0 max 16384 2024-05-27T08:05:04.708 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:05:04.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.709+0000 7f004e000700 1 -- 172.21.15.90:0/1926309403 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f002c0b2b40 msgr2=0x7f002c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.709+0000 7f004e000700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f002c0b2b40 0x7f002c0b5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0038005950 tx=0x7f00380058e0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.709+0000 7f004e000700 1 -- 172.21.15.90:0/1926309403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0048108f10 msgr2=0x7f00480726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:04.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.709+0000 7f004e000700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0048108f10 0x7f00480726d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f003400d980 tx=0x7f003400dd40 comp rx=0 tx=0).stop 2024-05-27T08:05:04.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.709+0000 7f004e000700 1 -- 172.21.15.90:0/1926309403 shutdown_connections 2024-05-27T08:05:04.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.709+0000 7f004e000700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0048109860 0x7f0048072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.709+0000 7f004e000700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f002c0b2b40 0x7f002c0b5050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.709+0000 7f004e000700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f00481045e0 0x7f00480775a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.709+0000 7f004e000700 1 --2- 172.21.15.90:0/1926309403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0048108f10 0x7f00480726d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:04.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.709+0000 7f004e000700 1 -- 172.21.15.90:0/1926309403 >> 172.21.15.90:0/1926309403 conn(0x7f0048100030 msgr2=0x7f0048108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:04.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.710+0000 7f004e000700 1 -- 172.21.15.90:0/1926309403 shutdown_connections 2024-05-27T08:05:04.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:04.710+0000 7f004e000700 1 -- 172.21.15.90:0/1926309403 wait complete. 2024-05-27T08:05:05.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:05 smithi090 ceph-mon[31429]: pgmap v603: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 563 op/s 2024-05-27T08:05:06.262 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:05 smithi129 ceph-mon[35254]: pgmap v603: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 563 op/s 2024-05-27T08:05:06.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:05 smithi136 ceph-mon[35324]: pgmap v603: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.9 MiB/s wr, 563 op/s 2024-05-27T08:05:07.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:07 smithi090 ceph-mon[31429]: pgmap v604: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-05-27T08:05:07.956 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:05:08.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:07 smithi136 ceph-mon[35324]: pgmap v604: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-05-27T08:05:08.068 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:07 smithi129 ceph-mon[35254]: pgmap v604: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.6 MiB/s wr, 560 op/s 2024-05-27T08:05:08.685 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:08.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 890.0 s, 433.5 tps, lat 222.924 ms stddev 185.986 2024-05-27T08:05:09.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:09 smithi090 ceph-mon[31429]: pgmap v605: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 491 op/s 2024-05-27T08:05:10.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:09 smithi136 ceph-mon[35324]: pgmap v605: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 491 op/s 2024-05-27T08:05:10.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:09 smithi129 ceph-mon[35254]: pgmap v605: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 5.0 MiB/s wr, 491 op/s 2024-05-27T08:05:11.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:11 smithi090 ceph-mon[31429]: pgmap v606: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 627 op/s 2024-05-27T08:05:12.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:11 smithi136 ceph-mon[35324]: pgmap v606: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 627 op/s 2024-05-27T08:05:12.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:11 smithi129 ceph-mon[35254]: pgmap v606: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 627 op/s 2024-05-27T08:05:13.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 895.0 s, 437.4 tps, lat 223.567 ms stddev 182.577 2024-05-27T08:05:13.967 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:13 smithi090 ceph-mon[31429]: pgmap v607: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-05-27T08:05:14.002 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:13 smithi136 ceph-mon[35324]: pgmap v607: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-05-27T08:05:14.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:13 smithi129 ceph-mon[35254]: pgmap v607: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-05-27T08:05:14.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.576+0000 7f99e7e00700 1 -- 172.21.15.90:0/1889723691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e01045e0 msgr2=0x7f99e01049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.576+0000 7f99e7e00700 1 --2- 172.21.15.90:0/1889723691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e01045e0 0x7f99e01049f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009f90 tx=0x7f99d400b330 comp rx=0 tx=0).stop 2024-05-27T08:05:14.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.577+0000 7f99e7e00700 1 -- 172.21.15.90:0/1889723691 shutdown_connections 2024-05-27T08:05:14.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.577+0000 7f99e7e00700 1 --2- 172.21.15.90:0/1889723691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99e0109860 0x7f99e01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.577+0000 7f99e7e00700 1 --2- 172.21.15.90:0/1889723691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e01045e0 0x7f99e01049f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.577+0000 7f99e7e00700 1 --2- 172.21.15.90:0/1889723691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99e0108f10 0x7f99e0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.577+0000 7f99e7e00700 1 -- 172.21.15.90:0/1889723691 >> 172.21.15.90:0/1889723691 conn(0x7f99e0100030 msgr2=0x7f99e0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.578+0000 7f99e7e00700 1 -- 172.21.15.90:0/1889723691 shutdown_connections 2024-05-27T08:05:14.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.578+0000 7f99e7e00700 1 -- 172.21.15.90:0/1889723691 wait complete. 2024-05-27T08:05:14.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.579+0000 7f99e7e00700 1 Processor -- start 2024-05-27T08:05:14.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.580+0000 7f99e7e00700 1 -- start start 2024-05-27T08:05:14.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.580+0000 7f99e7e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99e01045e0 0x7f99e01a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.580+0000 7f99e7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99e0108f10 0x7f99e01a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.580+0000 7f99e7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e0109860 0x7f99e01ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.580+0000 7f99e7e00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0114390 con 0x7f99e01045e0 2024-05-27T08:05:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.580+0000 7f99e7e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0114210 con 0x7f99e0109860 2024-05-27T08:05:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.580+0000 7f99e7e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0114510 con 0x7f99e0108f10 2024-05-27T08:05:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.581+0000 7f99e5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99e01045e0 0x7f99e01a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.581+0000 7f99e5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99e01045e0 0x7f99e01a2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55778/0 (socket says 172.21.15.90:55778) 2024-05-27T08:05:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.581+0000 7f99e5400700 1 -- 172.21.15.90:0/136499601 learned_addr learned my addr 172.21.15.90:0/136499601 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:14.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.582+0000 7f99e5400700 1 -- 172.21.15.90:0/136499601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99e0108f10 msgr2=0x7f99e01a25d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T08:05:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.582+0000 7f99e5400700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99e0108f10 0x7f99e01a25d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.582+0000 7f99e5400700 1 -- 172.21.15.90:0/136499601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e0109860 msgr2=0x7f99e01ad0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T08:05:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.582+0000 7f99e5400700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e0109860 0x7f99e01ad0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.582+0000 7f99e5400700 1 -- 172.21.15.90:0/136499601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e01ad630 con 0x7f99e01045e0 2024-05-27T08:05:14.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.583+0000 7f99e5400700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99e01045e0 0x7f99e01a2070 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f99d400cd00 tx=0x7f99d4009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.584+0000 7f99de000700 1 -- 172.21.15.90:0/136499601 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4021070 con 0x7f99e01045e0 2024-05-27T08:05:14.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.584+0000 7f99de000700 1 -- 172.21.15.90:0/136499601 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d4007ac0 con 0x7f99e01045e0 2024-05-27T08:05:14.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.584+0000 7f99de000700 1 -- 172.21.15.90:0/136499601 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d40052a0 con 0x7f99e01045e0 2024-05-27T08:05:14.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.584+0000 7f99e7e00700 1 -- 172.21.15.90:0/136499601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e01ad800 con 0x7f99e01045e0 2024-05-27T08:05:14.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.584+0000 7f99e7e00700 1 -- 172.21.15.90:0/136499601 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e01adcc0 con 0x7f99e01045e0 2024-05-27T08:05:14.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.592+0000 7f99de000700 1 -- 172.21.15.90:0/136499601 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f99d4024070 con 0x7f99e01045e0 2024-05-27T08:05:14.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.595+0000 7f99de000700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f99c80b2af0 0x7f99c80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.596+0000 7f99de000700 1 -- 172.21.15.90:0/136499601 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f99d400f020 con 0x7f99e01045e0 2024-05-27T08:05:14.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.597+0000 7f99e4a00700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f99c80b2af0 0x7f99c80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.608+0000 7f99e4a00700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f99c80b2af0 0x7f99c80b5000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99e0077ad0 tx=0x7f99cc009310 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.611+0000 7f99f68feb80 1 Processor -- start 2024-05-27T08:05:14.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.611+0000 7f99f68feb80 1 -- start start 2024-05-27T08:05:14.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.612+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.612+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b5d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.612+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b5ddb0 0x55f918b5e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.612+0000 7f99f68feb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f918a8be40 con 0x55f918b63d30 2024-05-27T08:05:14.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.612+0000 7f99f68feb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f918a7b9e0 con 0x55f9189dee40 2024-05-27T08:05:14.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.612+0000 7f99f68feb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f918a7fa20 con 0x55f918b5ddb0 2024-05-27T08:05:14.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.613+0000 7f99e5400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.613+0000 7f99e5400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b637f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53186/0 (socket says 172.21.15.90:53186) 2024-05-27T08:05:14.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.613+0000 7f99e5400700 1 -- 172.21.15.90:0/1746580505 learned_addr learned my addr 172.21.15.90:0/1746580505 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:14.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.613+0000 7f99e5400700 1 -- 172.21.15.90:0/1746580505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b5ddb0 msgr2=0x55f918b5e1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.614+0000 7f99e4a00700 1 --2- 172.21.15.90:0/1746580505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b5d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.614+0000 7f99e5e00700 1 --2- 172.21.15.90:0/1746580505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b5ddb0 0x55f918b5e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.614+0000 7f99e5400700 1 --2- 172.21.15.90:0/1746580505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b5ddb0 0x55f918b5e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.614+0000 7f99e5400700 1 -- 172.21.15.90:0/1746580505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 msgr2=0x55f918b5d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.614+0000 7f99e5400700 1 --2- 172.21.15.90:0/1746580505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b5d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.614+0000 7f99e5400700 1 -- 172.21.15.90:0/1746580505 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f918a44420 con 0x55f9189dee40 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.615+0000 7f99e4a00700 1 --2- 172.21.15.90:0/1746580505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b5d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.615+0000 7f99e5400700 1 --2- 172.21.15.90:0/1746580505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b637f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f99d40a7ee0 tx=0x7f99d40ab5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d6ca60f9a63970e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.615+0000 7f99d2a00700 1 -- 172.21.15.90:0/1746580505 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d40abe60 con 0x55f9189dee40 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.616+0000 7f99d2a00700 1 -- 172.21.15.90:0/1746580505 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d40b0210 con 0x55f9189dee40 2024-05-27T08:05:14.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.616+0000 7f99d2a00700 1 -- 172.21.15.90:0/1746580505 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d40ac350 con 0x55f9189dee40 2024-05-27T08:05:14.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.618+0000 7f99f68feb80 1 -- 172.21.15.90:0/1746580505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 msgr2=0x55f918b637f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.618+0000 7f99f68feb80 1 --2- 172.21.15.90:0/1746580505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b637f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f99d40a7ee0 tx=0x7f99d40ab5d0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.618+0000 7f99f68feb80 1 -- 172.21.15.90:0/1746580505 shutdown_connections 2024-05-27T08:05:14.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.618+0000 7f99f68feb80 1 --2- 172.21.15.90:0/1746580505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b5ddb0 0x55f918b5e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.619+0000 7f99f68feb80 1 --2- 172.21.15.90:0/1746580505 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b637f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.619+0000 7f99f68feb80 1 --2- 172.21.15.90:0/1746580505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b5d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.619+0000 7f99f68feb80 1 -- 172.21.15.90:0/1746580505 >> 172.21.15.90:0/1746580505 conn(0x55f918a3efc0 msgr2=0x55f918b588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.619+0000 7f99f68feb80 1 -- 172.21.15.90:0/1746580505 shutdown_connections 2024-05-27T08:05:14.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.619+0000 7f99f68feb80 1 -- 172.21.15.90:0/1746580505 wait complete. 2024-05-27T08:05:14.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.620+0000 7f99f68feb80 1 -- start start 2024-05-27T08:05:14.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.620+0000 7f99f68feb80 1 Processor -- start 2024-05-27T08:05:14.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.621+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9189dee40 0x55f918b67530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.621+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f918b5ddb0 0x55f918b67a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.621+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.621+0000 7f99f68feb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f918981fa0 con 0x55f9189dee40 2024-05-27T08:05:14.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.621+0000 7f99f68feb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f918a1c760 con 0x55f918b5ddb0 2024-05-27T08:05:14.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.621+0000 7f99f68feb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f918b7b5d0 con 0x55f918b63d30 2024-05-27T08:05:14.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.621+0000 7f99f68feb80 10 client.0 _ll_register_callbacks cb 0x55f918a25c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:14.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.622+0000 7f99e5400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9189dee40 0x55f918b67530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.622+0000 7f99e5e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.622+0000 7f99e5e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b67fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50436/0 (socket says 172.21.15.90:50436) 2024-05-27T08:05:14.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.622+0000 7f99e5e00700 1 -- 172.21.15.90:0/2379540927 learned_addr learned my addr 172.21.15.90:0/2379540927 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:14.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.622+0000 7f99e4a00700 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f918b5ddb0 0x55f918b67a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.623+0000 7f99e5400700 1 -- 172.21.15.90:0/2379540927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 msgr2=0x55f918b67fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.623+0000 7f99e5400700 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b67fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.623+0000 7f99e5400700 1 -- 172.21.15.90:0/2379540927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f918b5ddb0 msgr2=0x55f918b67a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.623+0000 7f99e5400700 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f918b5ddb0 0x55f918b67a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.623+0000 7f99e5400700 1 -- 172.21.15.90:0/2379540927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9189cb8d0 con 0x55f9189dee40 2024-05-27T08:05:14.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.625+0000 7f99e5400700 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9189dee40 0x55f918b67530 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f99d40a7ee0 tx=0x7f99d40b49d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32dd725405b2653 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.625+0000 7f99c7400700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:14.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.625+0000 7f99c7400700 1 -- 172.21.15.90:0/2379540927 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d40b4c30 con 0x55f9189dee40 2024-05-27T08:05:14.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.625+0000 7f99c7400700 1 -- 172.21.15.90:0/2379540927 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f99d40ab2d0 con 0x55f9189dee40 2024-05-27T08:05:14.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.626+0000 7f99f68feb80 1 -- 172.21.15.90:0/2379540927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99d40b0210 con 0x55f9189dee40 2024-05-27T08:05:14.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.626+0000 7f99c7400700 1 -- 172.21.15.90:0/2379540927 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f99d40ac310 con 0x55f9189dee40 2024-05-27T08:05:14.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.627+0000 7f99f68feb80 10 client.18480 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:14.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.627+0000 7f99f68feb80 1 -- 172.21.15.90:0/2379540927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f918a4c060 con 0x55f9189dee40 2024-05-27T08:05:14.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.627+0000 7f99c7400700 1 -- 172.21.15.90:0/2379540927 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d40b0210 con 0x55f9189dee40 2024-05-27T08:05:14.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.627+0000 7f99c7400700 1 -- 172.21.15.90:0/2379540927 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f99d408e030 con 0x55f9189dee40 2024-05-27T08:05:14.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.628+0000 7f99f68feb80 10 client.18480 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:14.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.628+0000 7f99f68feb80 10 client.18480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:14.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.628+0000 7f99f68feb80 20 client.18480 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:14.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.628+0000 7f99f68feb80 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f918b88000 0x55f918b8a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.628+0000 7f99f68feb80 4 client.18480 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:14.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.628+0000 7f99f68feb80 1 -- 172.21.15.90:0/2379540927 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f918a8dd90 con 0x55f918b88000 2024-05-27T08:05:14.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.628+0000 7f99e4a00700 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f918b88000 0x55f918b8a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.630+0000 7f99c7400700 10 client.18480 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:14.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.630+0000 7f99e4a00700 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f918b88000 0x55f918b8a420 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=799a276908bbe454 server_cookie=14c9038b5eebbfcc in_seq=0 out_seq=0 2024-05-27T08:05:14.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.632+0000 7f99c7400700 1 -- 172.21.15.90:0/2379540927 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55f918a8dd90 con 0x55f918b88000 2024-05-27T08:05:14.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.632+0000 7f99c7400700 10 client.18480 handle_command_reply: tid=0 2024-05-27T08:05:14.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.632+0000 7f99f68feb80 1 client.18480 shutdown 2024-05-27T08:05:14.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.633+0000 7f99f68feb80 1 -- 172.21.15.90:0/2379540927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9189dee40 msgr2=0x55f918b67530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.633+0000 7f99f68feb80 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9189dee40 0x55f918b67530 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f99d40a7ee0 tx=0x7f99d40b49d0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.633+0000 7f99f68feb80 1 -- 172.21.15.90:0/2379540927 shutdown_connections 2024-05-27T08:05:14.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.633+0000 7f99f68feb80 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f918b88000 0x55f918b8a420 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.633+0000 7f99f68feb80 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b67fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.633+0000 7f99f68feb80 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f9189dee40 0x55f918b67530 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.633+0000 7f99f68feb80 1 --2- 172.21.15.90:0/2379540927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f918b5ddb0 0x55f918b67a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.633+0000 7f99f68feb80 1 -- 172.21.15.90:0/2379540927 >> 172.21.15.90:0/2379540927 conn(0x55f918a3efc0 msgr2=0x55f918b58580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.633+0000 7f99c7400700 0 client.18480 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:14.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.634+0000 7f99f68feb80 1 -- 172.21.15.90:0/2379540927 shutdown_connections 2024-05-27T08:05:14.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.634+0000 7f99f68feb80 1 -- 172.21.15.90:0/2379540927 wait complete. 2024-05-27T08:05:14.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.634+0000 7f99f68feb80 20 client.18480 trim_cache size 0 max 16384 2024-05-27T08:05:14.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.663+0000 7f99f68feb80 1 Processor -- start 2024-05-27T08:05:14.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.663+0000 7f99f68feb80 1 -- start start 2024-05-27T08:05:14.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.663+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9189dee40 0x55f918b5d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.663+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f918b5ddb0 0x55f918b66490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.663+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b69980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.663+0000 7f99f68feb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01afe20 con 0x55f918b63d30 2024-05-27T08:05:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.663+0000 7f99f68feb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0074870 con 0x55f918b5ddb0 2024-05-27T08:05:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.663+0000 7f99f68feb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01af080 con 0x55f9189dee40 2024-05-27T08:05:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.664+0000 7f99e5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b69980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.664+0000 7f99e5e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b69980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55814/0 (socket says 172.21.15.90:55814) 2024-05-27T08:05:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.664+0000 7f99e5e00700 1 -- 172.21.15.90:0/3243654869 learned_addr learned my addr 172.21.15.90:0/3243654869 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.664+0000 7f99e5400700 1 --2- 172.21.15.90:0/3243654869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9189dee40 0x55f918b5d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.665+0000 7f99e5e00700 1 -- 172.21.15.90:0/3243654869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9189dee40 msgr2=0x55f918b5d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.665+0000 7f99e5e00700 1 --2- 172.21.15.90:0/3243654869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9189dee40 0x55f918b5d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.665+0000 7f99e5e00700 1 -- 172.21.15.90:0/3243654869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f918b5ddb0 msgr2=0x55f918b66490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.665+0000 7f99e5e00700 1 --2- 172.21.15.90:0/3243654869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f918b5ddb0 0x55f918b66490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.665+0000 7f99e5e00700 1 -- 172.21.15.90:0/3243654869 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f918b68320 con 0x55f918b63d30 2024-05-27T08:05:14.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.665+0000 7f99e5400700 1 --2- 172.21.15.90:0/3243654869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9189dee40 0x55f918b5d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:14.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.665+0000 7f99e5e00700 1 --2- 172.21.15.90:0/3243654869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b69980 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f99d80045b0 tx=0x7f99d8009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a024e40c2b5ec0ed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.666+0000 7f99d2000700 1 -- 172.21.15.90:0/3243654869 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8004a30 con 0x55f918b63d30 2024-05-27T08:05:14.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.667+0000 7f99d2000700 1 -- 172.21.15.90:0/3243654869 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d80054b0 con 0x55f918b63d30 2024-05-27T08:05:14.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.667+0000 7f99d2000700 1 -- 172.21.15.90:0/3243654869 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8003ba0 con 0x55f918b63d30 2024-05-27T08:05:14.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.668+0000 7f99f68feb80 1 -- 172.21.15.90:0/3243654869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 msgr2=0x55f918b69980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.668+0000 7f99f68feb80 1 --2- 172.21.15.90:0/3243654869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b69980 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f99d80045b0 tx=0x7f99d8009a90 comp rx=0 tx=0).stop 2024-05-27T08:05:14.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.668+0000 7f99f68feb80 1 -- 172.21.15.90:0/3243654869 shutdown_connections 2024-05-27T08:05:14.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.668+0000 7f99f68feb80 1 --2- 172.21.15.90:0/3243654869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b63d30 0x55f918b69980 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.668+0000 7f99f68feb80 1 --2- 172.21.15.90:0/3243654869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f9189dee40 0x55f918b5d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.668+0000 7f99f68feb80 1 --2- 172.21.15.90:0/3243654869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f918b5ddb0 0x55f918b66490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.669+0000 7f99f68feb80 1 -- 172.21.15.90:0/3243654869 >> 172.21.15.90:0/3243654869 conn(0x55f918a3efc0 msgr2=0x55f918b61140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.669+0000 7f99f68feb80 1 -- 172.21.15.90:0/3243654869 shutdown_connections 2024-05-27T08:05:14.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.669+0000 7f99f68feb80 1 -- 172.21.15.90:0/3243654869 wait complete. 2024-05-27T08:05:14.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.670+0000 7f99f68feb80 1 -- start start 2024-05-27T08:05:14.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.670+0000 7f99f68feb80 1 Processor -- start 2024-05-27T08:05:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.671+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.671+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b5ddb0 0x55f918b80720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.671+0000 7f99f68feb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b80d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.671+0000 7f99f68feb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01ae9b0 con 0x55f918b5ddb0 2024-05-27T08:05:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.671+0000 7f99f68feb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0074f40 con 0x55f9189dee40 2024-05-27T08:05:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.671+0000 7f99f68feb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0075610 con 0x55f918b63d30 2024-05-27T08:05:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.672+0000 7f99f68feb80 10 client.0 _ll_register_callbacks cb 0x55f918a25c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:14.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.672+0000 7f99e5e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b80d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.672+0000 7f99e5e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b80d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50454/0 (socket says 172.21.15.90:50454) 2024-05-27T08:05:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.672+0000 7f99e5e00700 1 -- 172.21.15.90:0/1666398832 learned_addr learned my addr 172.21.15.90:0/1666398832 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.672+0000 7f99e4a00700 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b5ddb0 0x55f918b80720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.673+0000 7f99e5400700 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.673+0000 7f99e5e00700 1 -- 172.21.15.90:0/1666398832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 msgr2=0x55f918b902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.673+0000 7f99e5e00700 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b902f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.673+0000 7f99e5e00700 1 -- 172.21.15.90:0/1666398832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b5ddb0 msgr2=0x55f918b80720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:14.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.673+0000 7f99e5e00700 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b5ddb0 0x55f918b80720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.673+0000 7f99e5e00700 1 -- 172.21.15.90:0/1666398832 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f918b8eaf0 con 0x55f918b63d30 2024-05-27T08:05:14.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.674+0000 7f99e5400700 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:14.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.674+0000 7f99e4a00700 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b5ddb0 0x55f918b80720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:14.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.675+0000 7f99e5e00700 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b80d20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f99d8005250 tx=0x7f99d8005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c27871d159123161 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:14.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.675+0000 7f99c7400700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:05:14.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.676+0000 7f99c7400700 1 -- 172.21.15.90:0/1666398832 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8004d20 con 0x55f918b63d30 2024-05-27T08:05:14.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.676+0000 7f99c7400700 1 -- 172.21.15.90:0/1666398832 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f99d80037e0 con 0x55f918b63d30 2024-05-27T08:05:14.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.678+0000 7f99f68feb80 1 -- 172.21.15.90:0/1666398832 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99d80054b0 con 0x55f918b63d30 2024-05-27T08:05:14.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.679+0000 7f99c7400700 1 -- 172.21.15.90:0/1666398832 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f99d80039e0 con 0x55f918b63d30 2024-05-27T08:05:14.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.679+0000 7f99f68feb80 10 client.28279 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:14.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.680+0000 7f99f68feb80 1 -- 172.21.15.90:0/1666398832 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55f918bf1b90 con 0x55f918b63d30 2024-05-27T08:05:14.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.680+0000 7f99c7400700 1 -- 172.21.15.90:0/1666398832 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99d80054b0 con 0x55f918b63d30 2024-05-27T08:05:14.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.680+0000 7f99c7400700 1 -- 172.21.15.90:0/1666398832 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f99d8019030 con 0x55f918b63d30 2024-05-27T08:05:14.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.681+0000 7f99f68feb80 10 client.28279 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:14.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.681+0000 7f99f68feb80 10 client.28279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:14.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.681+0000 7f99f68feb80 20 client.28279 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:14.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.681+0000 7f99f68feb80 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f918bf2170 0x55f918bf4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:14.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.681+0000 7f99f68feb80 4 client.28279 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:05:14.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.681+0000 7f99f68feb80 1 -- 172.21.15.90:0/1666398832 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f918a87570 con 0x55f918bf2170 2024-05-27T08:05:14.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.682+0000 7f99e5400700 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f918bf2170 0x55f918bf4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:14.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.683+0000 7f99e5400700 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f918bf2170 0x55f918bf4590 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94794307f85cdc27 server_cookie=458ea7c71b18a278 in_seq=0 out_seq=0 2024-05-27T08:05:14.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.683+0000 7f99c7400700 10 client.28279 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.683+0000 7f99c7400700 1 -- 172.21.15.90:0/1666398832 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f918a87570 con 0x55f918bf2170 2024-05-27T08:05:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.683+0000 7f99c7400700 10 client.28279 handle_command_reply: tid=0 2024-05-27T08:05:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.683+0000 7f99f68feb80 1 client.28279 shutdown 2024-05-27T08:05:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.684+0000 7f99f68feb80 1 -- 172.21.15.90:0/1666398832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 msgr2=0x55f918b80d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.684+0000 7f99f68feb80 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b80d20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f99d8005250 tx=0x7f99d8005810 comp rx=0 tx=0).stop 2024-05-27T08:05:14.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.684+0000 7f99f68feb80 1 -- 172.21.15.90:0/1666398832 shutdown_connections 2024-05-27T08:05:14.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.684+0000 7f99f68feb80 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55f918bf2170 0x55f918bf4590 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.684+0000 7f99f68feb80 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f918b63d30 0x55f918b80d20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.684+0000 7f99f68feb80 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f9189dee40 0x55f918b902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.684+0000 7f99f68feb80 1 --2- 172.21.15.90:0/1666398832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f918b5ddb0 0x55f918b80720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.684+0000 7f99f68feb80 1 -- 172.21.15.90:0/1666398832 >> 172.21.15.90:0/1666398832 conn(0x55f918a3efc0 msgr2=0x55f918b61140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.684+0000 7f99c7400700 0 client.28279 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.685+0000 7f99f68feb80 1 -- 172.21.15.90:0/1666398832 shutdown_connections 2024-05-27T08:05:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.685+0000 7f99f68feb80 1 -- 172.21.15.90:0/1666398832 wait complete. 2024-05-27T08:05:14.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.685+0000 7f99f68feb80 20 client.28279 trim_cache size 0 max 16384 2024-05-27T08:05:14.689 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:05:14.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.687+0000 7f99e7e00700 1 -- 172.21.15.90:0/136499601 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f99c80b2af0 msgr2=0x7f99c80b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.687+0000 7f99e7e00700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f99c80b2af0 0x7f99c80b5000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f99e0077ad0 tx=0x7f99cc009310 comp rx=0 tx=0).stop 2024-05-27T08:05:14.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.687+0000 7f99e7e00700 1 -- 172.21.15.90:0/136499601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99e01045e0 msgr2=0x7f99e01a2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:14.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.687+0000 7f99e7e00700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99e01045e0 0x7f99e01a2070 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f99d400cd00 tx=0x7f99d4009510 comp rx=0 tx=0).stop 2024-05-27T08:05:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.687+0000 7f99e7e00700 1 -- 172.21.15.90:0/136499601 shutdown_connections 2024-05-27T08:05:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.687+0000 7f99e7e00700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f99c80b2af0 0x7f99c80b5000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.687+0000 7f99e7e00700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99e0109860 0x7f99e01ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.687+0000 7f99e7e00700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99e01045e0 0x7f99e01a2070 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.687+0000 7f99e7e00700 1 --2- 172.21.15.90:0/136499601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99e0108f10 0x7f99e01a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:14.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.687+0000 7f99e7e00700 1 -- 172.21.15.90:0/136499601 >> 172.21.15.90:0/136499601 conn(0x7f99e0100030 msgr2=0x7f99e01087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:14.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.688+0000 7f99e7e00700 1 -- 172.21.15.90:0/136499601 shutdown_connections 2024-05-27T08:05:14.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:14.689+0000 7f99e7e00700 1 -- 172.21.15.90:0/136499601 wait complete. 2024-05-27T08:05:15.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:15 smithi090 ceph-mon[31429]: pgmap v608: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 483 op/s 2024-05-27T08:05:16.000 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:15 smithi136 ceph-mon[35324]: pgmap v608: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 483 op/s 2024-05-27T08:05:16.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:15 smithi129 ceph-mon[35254]: pgmap v608: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 483 op/s 2024-05-27T08:05:16.599 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-27T08:05:16.599 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-27T08:05:17.915 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:17.967 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:17 smithi090 ceph-mon[31429]: pgmap v609: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.0 MiB/s wr, 473 op/s 2024-05-27T08:05:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.973+0000 7f57f0e00640 1 -- 172.21.15.90:0/2992765174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57ec11a8e0 msgr2=0x7f57ec11cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.973+0000 7f57f0e00640 1 --2- 172.21.15.90:0/2992765174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57ec11a8e0 0x7f57ec11cd00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009ca0 tx=0x7f57e402a960 comp rx=0 tx=0).stop 2024-05-27T08:05:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.975+0000 7f57f0e00640 1 -- 172.21.15.90:0/2992765174 shutdown_connections 2024-05-27T08:05:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.975+0000 7f57f0e00640 1 --2- 172.21.15.90:0/2992765174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57ec11d240 0x7f57ec11f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.975+0000 7f57f0e00640 1 --2- 172.21.15.90:0/2992765174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57ec11a8e0 0x7f57ec11cd00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.975+0000 7f57f0e00640 1 --2- 172.21.15.90:0/2992765174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57ec10a1f0 0x7f57ec10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.975+0000 7f57f0e00640 1 -- 172.21.15.90:0/2992765174 >> 172.21.15.90:0/2992765174 conn(0x7f57ec06e8f0 msgr2=0x7f57ec06ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.976+0000 7f57f0e00640 1 -- 172.21.15.90:0/2992765174 shutdown_connections 2024-05-27T08:05:17.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.976+0000 7f57f0e00640 1 -- 172.21.15.90:0/2992765174 wait complete. 2024-05-27T08:05:17.980 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.979+0000 7f57f0e00640 1 Processor -- start 2024-05-27T08:05:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.979+0000 7f57f0e00640 1 -- start start 2024-05-27T08:05:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.980+0000 7f57f0e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57ec10a1f0 0x7f57ec1a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.981+0000 7f57f0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57ec11a8e0 0x7f57ec1a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.981+0000 7f57e9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57ec10a1f0 0x7f57ec1a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.981+0000 7f57e9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57ec10a1f0 0x7f57ec1a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53222/0 (socket says 172.21.15.90:53222) 2024-05-27T08:05:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.981+0000 7f57f0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57ec11d240 0x7f57ec1addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.981+0000 7f57f0e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec122b20 con 0x7f57ec11a8e0 2024-05-27T08:05:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.981+0000 7f57f0e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec1229a0 con 0x7f57ec10a1f0 2024-05-27T08:05:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.981+0000 7f57f0e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec122ca0 con 0x7f57ec11d240 2024-05-27T08:05:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.981+0000 7f57e9e00640 1 -- 172.21.15.90:0/1672302444 learned_addr learned my addr 172.21.15.90:0/1672302444 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.982+0000 7f57ea800640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57ec11d240 0x7f57ec1addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.982+0000 7f57e9e00640 1 -- 172.21.15.90:0/1672302444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57ec11d240 msgr2=0x7f57ec1addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.982+0000 7f57e9e00640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57ec11d240 0x7f57ec1addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.982+0000 7f57e9e00640 1 -- 172.21.15.90:0/1672302444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57ec11a8e0 msgr2=0x7f57ec1a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.982+0000 7f57e9e00640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57ec11a8e0 0x7f57ec1a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.982+0000 7f57e9e00640 1 -- 172.21.15.90:0/1672302444 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec1ae3f0 con 0x7f57ec10a1f0 2024-05-27T08:05:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.982+0000 7f57ea800640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57ec11d240 0x7f57ec1addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.984+0000 7f57e9e00640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57ec10a1f0 0x7f57ec1a95e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f57d400d950 tx=0x7f57d400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:17.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.985+0000 7f57e2a00640 1 -- 172.21.15.90:0/1672302444 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4017070 con 0x7f57ec10a1f0 2024-05-27T08:05:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.985+0000 7f57e2a00640 1 -- 172.21.15.90:0/1672302444 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57d4009c30 con 0x7f57ec10a1f0 2024-05-27T08:05:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.985+0000 7f57f0e00640 1 -- 172.21.15.90:0/1672302444 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ec1ae6e0 con 0x7f57ec10a1f0 2024-05-27T08:05:17.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.985+0000 7f57f0e00640 1 -- 172.21.15.90:0/1672302444 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ec1b6060 con 0x7f57ec10a1f0 2024-05-27T08:05:17.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.985+0000 7f57e2a00640 1 -- 172.21.15.90:0/1672302444 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d400f620 con 0x7f57ec10a1f0 2024-05-27T08:05:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.990+0000 7f57f7dce740 1 Processor -- start 2024-05-27T08:05:17.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.991+0000 7f57f7dce740 1 -- start start 2024-05-27T08:05:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.992+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bec41b0 0x55c28c266860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.992+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 0x55c28c266da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.992+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bea34d0 0x55c28c2672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.992+0000 7f57f7dce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c28c1d7b90 con 0x55c28bea34d0 2024-05-27T08:05:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.992+0000 7f57f7dce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c28c1c7730 con 0x55c28be9b2f0 2024-05-27T08:05:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.992+0000 7f57f7dce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c28c1cb770 con 0x55c28bec41b0 2024-05-27T08:05:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.994+0000 7f57ea800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bec41b0 0x55c28c266860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.994+0000 7f57e9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bea34d0 0x55c28c2672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.994+0000 7f57e9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bea34d0 0x55c28c2672e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55864/0 (socket says 172.21.15.90:55864) 2024-05-27T08:05:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.994+0000 7f57e9400640 1 -- 172.21.15.90:0/4178416073 learned_addr learned my addr 172.21.15.90:0/4178416073 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.995+0000 7f57e9400640 1 -- 172.21.15.90:0/4178416073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bec41b0 msgr2=0x55c28c266860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.995+0000 7f57e9400640 1 --2- 172.21.15.90:0/4178416073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bec41b0 0x55c28c266860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.995+0000 7f57e9400640 1 -- 172.21.15.90:0/4178416073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 msgr2=0x55c28c266da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-27T08:05:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.995+0000 7f57e9400640 1 --2- 172.21.15.90:0/4178416073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 0x55c28c266da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.995+0000 7f57e9400640 1 -- 172.21.15.90:0/4178416073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28c1608b0 con 0x55c28bea34d0 2024-05-27T08:05:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.996+0000 7f57e9400640 1 --2- 172.21.15.90:0/4178416073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bea34d0 0x55c28c2672e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f57e40094b0 tx=0x7f57e4009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b724b0e6bbc57 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.996+0000 7f57db400640 1 -- 172.21.15.90:0/4178416073 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4034040 con 0x55c28bea34d0 2024-05-27T08:05:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.996+0000 7f57db400640 1 -- 172.21.15.90:0/4178416073 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57e4003e00 con 0x55c28bea34d0 2024-05-27T08:05:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.997+0000 7f57e2a00640 1 -- 172.21.15.90:0/1672302444 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f57d4010460 con 0x7f57ec10a1f0 2024-05-27T08:05:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.997+0000 7f57f7dce740 1 -- 172.21.15.90:0/4178416073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bea34d0 msgr2=0x55c28c2672e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.997+0000 7f57f7dce740 1 --2- 172.21.15.90:0/4178416073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bea34d0 0x55c28c2672e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f57e40094b0 tx=0x7f57e4009a00 comp rx=0 tx=0).stop 2024-05-27T08:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.998+0000 7f57f7dce740 1 -- 172.21.15.90:0/4178416073 shutdown_connections 2024-05-27T08:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.998+0000 7f57f7dce740 1 --2- 172.21.15.90:0/4178416073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 0x55c28c266da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.998+0000 7f57f7dce740 1 --2- 172.21.15.90:0/4178416073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bea34d0 0x55c28c2672e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.998+0000 7f57f7dce740 1 --2- 172.21.15.90:0/4178416073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bec41b0 0x55c28c266860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.998+0000 7f57f7dce740 1 -- 172.21.15.90:0/4178416073 >> 172.21.15.90:0/4178416073 conn(0x55c28beb8300 msgr2=0x55c28c263010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:18.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:17 smithi136 ceph-mon[35324]: pgmap v609: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.0 MiB/s wr, 473 op/s 2024-05-27T08:05:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.998+0000 7f57f7dce740 1 -- 172.21.15.90:0/4178416073 shutdown_connections 2024-05-27T08:05:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.998+0000 7f57f7dce740 1 -- 172.21.15.90:0/4178416073 wait complete. 2024-05-27T08:05:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:17.999+0000 7f57f7dce740 1 -- start start 2024-05-27T08:05:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.000+0000 7f57f7dce740 1 Processor -- start 2024-05-27T08:05:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.001+0000 7f57e2a00640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57d00b2960 0x7f57d00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.002+0000 7f57e2a00640 1 -- 172.21.15.90:0/1672302444 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f57d40e25c0 con 0x7f57ec10a1f0 2024-05-27T08:05:18.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.002+0000 7f57ea800640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57d00b2960 0x7f57d00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.004+0000 7f57ea800640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57d00b2960 0x7f57d00b4e70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f57dc005fd0 tx=0x7f57dc005cf0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:18.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.004+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 0x55c28c2708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.005+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 0x55c28c270e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.007 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.005+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c271340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.005+0000 7f57f7dce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c28bead790 con 0x55c28bec41b0 2024-05-27T08:05:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.005+0000 7f57f7dce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c28c16fbc0 con 0x55c28be9b2f0 2024-05-27T08:05:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.005+0000 7f57f7dce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c28c2739d0 con 0x55c28bea34d0 2024-05-27T08:05:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.005+0000 7f57e9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 0x55c28c2708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.006+0000 7f57e9e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 0x55c28c2708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53232/0 (socket says 172.21.15.90:53232) 2024-05-27T08:05:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.006+0000 7f57e9e00640 1 -- 172.21.15.90:0/3897324669 learned_addr learned my addr 172.21.15.90:0/3897324669 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.006+0000 7f57e9400640 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 0x55c28c270e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.009 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.006+0000 7f57f7dce740 10 client.0 _ll_register_callbacks cb 0x55c28c14d400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.007+0000 7f57ea800640 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c271340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.007+0000 7f57e9e00640 1 -- 172.21.15.90:0/3897324669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 msgr2=0x55c28c270e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.007+0000 7f57e9e00640 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 0x55c28c270e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.007+0000 7f57e9e00640 1 -- 172.21.15.90:0/3897324669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 msgr2=0x55c28c271340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.010 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.007+0000 7f57e9e00640 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c271340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.007+0000 7f57e9e00640 1 -- 172.21.15.90:0/3897324669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28bedc210 con 0x55c28be9b2f0 2024-05-27T08:05:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.008+0000 7f57e9e00640 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 0x55c28c2708c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009610 tx=0x7f57d40dfe60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd2dde4f281f9b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.008+0000 7f57cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:05:18.011 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.008+0000 7f57cfe00640 1 -- 172.21.15.90:0/3897324669 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4020070 con 0x55c28be9b2f0 2024-05-27T08:05:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.009+0000 7f57cfe00640 1 -- 172.21.15.90:0/3897324669 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f57d40e2340 con 0x55c28be9b2f0 2024-05-27T08:05:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.009+0000 7f57f7dce740 1 -- 172.21.15.90:0/3897324669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57e4003e00 con 0x55c28be9b2f0 2024-05-27T08:05:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.009+0000 7f57cfe00640 1 -- 172.21.15.90:0/3897324669 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f57d40a9b70 con 0x55c28be9b2f0 2024-05-27T08:05:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.009+0000 7f57f7dce740 10 client.38198 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.010+0000 7f57f7dce740 1 -- 172.21.15.90:0/3897324669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55c28bfa8620 con 0x55c28be9b2f0 2024-05-27T08:05:18.012 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.010+0000 7f57cfe00640 1 -- 172.21.15.90:0/3897324669 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57e4003e00 con 0x55c28be9b2f0 2024-05-27T08:05:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.012+0000 7f57cfe00640 1 -- 172.21.15.90:0/3897324669 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f57d40b0ba0 con 0x55c28be9b2f0 2024-05-27T08:05:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.012+0000 7f57f7dce740 10 client.38198 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.012+0000 7f57f7dce740 10 client.38198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.012+0000 7f57f7dce740 20 client.38198 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.012+0000 7f57f7dce740 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c28be82d80 0x55c28c282da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.012+0000 7f57f7dce740 4 client.38198 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:18.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.012+0000 7f57f7dce740 1 -- 172.21.15.90:0/3897324669 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c28c1d9ae0 con 0x55c28be82d80 2024-05-27T08:05:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.012+0000 7f57e9400640 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c28be82d80 0x55c28c282da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.013+0000 7f57e9400640 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c28be82d80 0x55c28c282da0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f56cf2bbb1db4f server_cookie=d3e51312f8fda655 in_seq=0 out_seq=0 2024-05-27T08:05:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.013+0000 7f57cfe00640 10 client.38198 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.014+0000 7f57cfe00640 1 -- 172.21.15.90:0/3897324669 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55c28c1d9ae0 con 0x55c28be82d80 2024-05-27T08:05:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.015+0000 7f57cfe00640 10 client.38198 handle_command_reply: tid=0 2024-05-27T08:05:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.015+0000 7f57f7dce740 1 client.38198 shutdown 2024-05-27T08:05:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.016+0000 7f57f7dce740 1 -- 172.21.15.90:0/3897324669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 msgr2=0x55c28c2708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.016+0000 7f57f7dce740 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 0x55c28c2708c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009610 tx=0x7f57d40dfe60 comp rx=0 tx=0).stop 2024-05-27T08:05:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.016+0000 7f57f7dce740 1 -- 172.21.15.90:0/3897324669 shutdown_connections 2024-05-27T08:05:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.016+0000 7f57f7dce740 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c28be82d80 0x55c28c282da0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.016+0000 7f57f7dce740 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c271340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.016+0000 7f57f7dce740 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 0x55c28c270e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.016+0000 7f57f7dce740 1 --2- 172.21.15.90:0/3897324669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be9b2f0 0x55c28c2708c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.016+0000 7f57f7dce740 1 -- 172.21.15.90:0/3897324669 >> 172.21.15.90:0/3897324669 conn(0x55c28beb8300 msgr2=0x55c28c268580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:18.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.016+0000 7f57cfe00640 0 client.38198 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.017+0000 7f57f7dce740 1 -- 172.21.15.90:0/3897324669 shutdown_connections 2024-05-27T08:05:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.018+0000 7f57f7dce740 1 -- 172.21.15.90:0/3897324669 wait complete. 2024-05-27T08:05:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.018+0000 7f57f7dce740 20 client.38198 trim_cache size 0 max 16384 2024-05-27T08:05:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.051+0000 7f57f7dce740 1 Processor -- start 2024-05-27T08:05:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.051+0000 7f57f7dce740 1 -- start start 2024-05-27T08:05:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.052+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 0x55c28c262330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.052+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c272a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.052+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be581c0 0x55c28c271160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.052+0000 7f57f7dce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8011aa0 con 0x55c28bec41b0 2024-05-27T08:05:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.052+0000 7f57f7dce740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57c800eaf0 con 0x55c28be581c0 2024-05-27T08:05:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.052+0000 7f57f7dce740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8010d00 con 0x55c28bea34d0 2024-05-27T08:05:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.053+0000 7f57ea800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be581c0 0x55c28c271160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.053+0000 7f57ea800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be581c0 0x55c28c271160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53236/0 (socket says 172.21.15.90:53236) 2024-05-27T08:05:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.053+0000 7f57ea800640 1 -- 172.21.15.90:0/2043503060 learned_addr learned my addr 172.21.15.90:0/2043503060 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.053+0000 7f57e9e00640 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 0x55c28c262330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.054+0000 7f57e9400640 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c272a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.055+0000 7f57ea800640 1 -- 172.21.15.90:0/2043503060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 msgr2=0x55c28c262330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.055+0000 7f57ea800640 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 0x55c28c262330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.055+0000 7f57ea800640 1 -- 172.21.15.90:0/2043503060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 msgr2=0x55c28c272a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.055+0000 7f57ea800640 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c272a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.055+0000 7f57ea800640 1 -- 172.21.15.90:0/2043503060 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28c25cfc0 con 0x55c28be581c0 2024-05-27T08:05:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.056+0000 7f57e9400640 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c272a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:05:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.056+0000 7f57e9e00640 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 0x55c28c262330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:05:18.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.056+0000 7f57ea800640 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be581c0 0x55c28c271160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0124a0 tx=0x7f57dc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f55ab65a57a8858 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.057+0000 7f57daa00640 1 -- 172.21.15.90:0/2043503060 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc028670 con 0x55c28be581c0 2024-05-27T08:05:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.057+0000 7f57daa00640 1 -- 172.21.15.90:0/2043503060 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57dc023070 con 0x55c28be581c0 2024-05-27T08:05:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.057+0000 7f57daa00640 1 -- 172.21.15.90:0/2043503060 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc01fba0 con 0x55c28be581c0 2024-05-27T08:05:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.058+0000 7f57f7dce740 1 -- 172.21.15.90:0/2043503060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be581c0 msgr2=0x55c28c271160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.058+0000 7f57f7dce740 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be581c0 0x55c28c271160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0124a0 tx=0x7f57dc014000 comp rx=0 tx=0).stop 2024-05-27T08:05:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.059+0000 7f57f7dce740 1 -- 172.21.15.90:0/2043503060 shutdown_connections 2024-05-27T08:05:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.059+0000 7f57f7dce740 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28be581c0 0x55c28c271160 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.059+0000 7f57f7dce740 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c272a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.059+0000 7f57f7dce740 1 --2- 172.21.15.90:0/2043503060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28bea34d0 0x55c28c262330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.059+0000 7f57f7dce740 1 -- 172.21.15.90:0/2043503060 >> 172.21.15.90:0/2043503060 conn(0x55c28beb8300 msgr2=0x55c28c2665f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.059+0000 7f57f7dce740 1 -- 172.21.15.90:0/2043503060 shutdown_connections 2024-05-27T08:05:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.059+0000 7f57f7dce740 1 -- 172.21.15.90:0/2043503060 wait complete. 2024-05-27T08:05:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.060+0000 7f57f7dce740 1 -- start start 2024-05-27T08:05:18.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.060+0000 7f57f7dce740 1 Processor -- start 2024-05-27T08:05:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.061+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28be581c0 0x55c28c278fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.061+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28bea34d0 0x55c28c279510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.061+0000 7f57f7dce740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c26abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57e9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28be581c0 0x55c28c278fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57ea800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c26abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57e9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28be581c0 0x55c28c278fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:50520/0 (socket says 172.21.15.90:50520) 2024-05-27T08:05:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57ea800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c26abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55890/0 (socket says 172.21.15.90:55890) 2024-05-27T08:05:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57e9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28bea34d0 0x55c28c279510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57f7dce740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8010630 con 0x55c28bec41b0 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57e9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28bea34d0 0x55c28c279510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:53252/0 (socket says 172.21.15.90:53252) 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57f7dce740 1 -- 172.21.15.90:0/1992877127 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57c800f1c0 con 0x55c28bea34d0 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57e9400640 1 -- 172.21.15.90:0/1992877127 learned_addr learned my addr 172.21.15.90:0/1992877127 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57f7dce740 1 -- 172.21.15.90:0/1992877127 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57c800f890 con 0x55c28be581c0 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57ea800640 1 -- 172.21.15.90:0/1992877127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28be581c0 msgr2=0x55c28c278fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57ea800640 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28be581c0 0x55c28c278fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57ea800640 1 -- 172.21.15.90:0/1992877127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28bea34d0 msgr2=0x55c28c279510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57ea800640 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28bea34d0 0x55c28c279510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57ea800640 1 -- 172.21.15.90:0/1992877127 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28c26b380 con 0x55c28bec41b0 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.062+0000 7f57f7dce740 10 client.0 _ll_register_callbacks cb 0x7f57d400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:18.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.063+0000 7f57f7dce740 1 -- 172.21.15.90:0/1992877127 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57dc023070 con 0x55c28bec41b0 2024-05-27T08:05:18.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:17 smithi129 ceph-mon[35254]: pgmap v609: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.0 MiB/s wr, 473 op/s 2024-05-27T08:05:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.063+0000 7f57ea800640 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c26abf0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00b810 tx=0x7f57dc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff2a68e675f731d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.063+0000 7f57cfe00640 10 client.18495 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.064+0000 7f57cfe00640 1 -- 172.21.15.90:0/1992877127 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc037060 con 0x55c28bec41b0 2024-05-27T08:05:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.064+0000 7f57cfe00640 1 -- 172.21.15.90:0/1992877127 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f57dc01fb30 con 0x55c28bec41b0 2024-05-27T08:05:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.065+0000 7f57cfe00640 1 -- 172.21.15.90:0/1992877127 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f57dc037060 con 0x55c28bec41b0 2024-05-27T08:05:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.065+0000 7f57f7dce740 10 client.18495 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.065+0000 7f57f7dce740 1 -- 172.21.15.90:0/1992877127 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({fsmap=64}) -- 0x55c28c2784e0 con 0x55c28bec41b0 2024-05-27T08:05:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.066+0000 7f57cfe00640 1 -- 172.21.15.90:0/1992877127 <== mon.0 v2:172.21.15.90:3300/0 4 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f57dc031030 con 0x55c28bec41b0 2024-05-27T08:05:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.066+0000 7f57f7dce740 10 client.18495 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.066+0000 7f57f7dce740 10 client.18495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.066+0000 7f57f7dce740 20 client.18495 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.066+0000 7f57f7dce740 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c28c2fc480 0x55c28c2fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.066+0000 7f57f7dce740 4 client.18495 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:05:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.066+0000 7f57f7dce740 1 -- 172.21.15.90:0/1992877127 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c28c1d32c0 con 0x55c28c2fc480 2024-05-27T08:05:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.067+0000 7f57e9e00640 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c28c2fc480 0x55c28c2fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.068+0000 7f57e9e00640 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c28c2fc480 0x55c28c2fe8a0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de38a159b6f32673 server_cookie=cc452d48f22ed41f in_seq=0 out_seq=0 2024-05-27T08:05:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:18.068+0000 7f57cfe00640 10 client.18495 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:18.828 INFO:teuthology.orchestra.run.smithi090.stderr:progress: 900.0 s, 475.2 tps, lat 210.466 ms stddev 187.552 2024-05-27T08:05:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.010+0000 7f57cfe00640 1 -- 172.21.15.90:0/1992877127 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c28c1d32c0 con 0x55c28c2fc480 2024-05-27T08:05:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.011+0000 7f57cfe00640 10 client.18495 handle_command_reply: tid=0 2024-05-27T08:05:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.011+0000 7f57f7dce740 1 client.18495 shutdown 2024-05-27T08:05:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.012+0000 7f57f7dce740 1 -- 172.21.15.90:0/1992877127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 msgr2=0x55c28c26abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.016+0000 7f57f7dce740 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c26abf0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00b810 tx=0x7f57dc005740 comp rx=0 tx=0).stop 2024-05-27T08:05:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.016+0000 7f57f7dce740 1 -- 172.21.15.90:0/1992877127 shutdown_connections 2024-05-27T08:05:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.016+0000 7f57f7dce740 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55c28c2fc480 0x55c28c2fe8a0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.016+0000 7f57f7dce740 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c28bea34d0 0x55c28c279510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.016+0000 7f57f7dce740 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c28bec41b0 0x55c28c26abf0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.016+0000 7f57f7dce740 1 --2- 172.21.15.90:0/1992877127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28be581c0 0x55c28c278fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.016+0000 7f57f7dce740 1 -- 172.21.15.90:0/1992877127 >> 172.21.15.90:0/1992877127 conn(0x55c28beb8300 msgr2=0x55c28c267020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.016+0000 7f57cfe00640 0 client.18495 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.016+0000 7f57f7dce740 1 -- 172.21.15.90:0/1992877127 shutdown_connections 2024-05-27T08:05:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.017+0000 7f57f7dce740 1 -- 172.21.15.90:0/1992877127 wait complete. 2024-05-27T08:05:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.017+0000 7f57f7dce740 20 client.18495 trim_cache size 0 max 16384 2024-05-27T08:05:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:05:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:05:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:05:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.018+0000 7f57f0e00640 1 -- 172.21.15.90:0/1672302444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57d00b2960 msgr2=0x7f57d00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.018+0000 7f57f0e00640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57d00b2960 0x7f57d00b4e70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f57dc005fd0 tx=0x7f57dc005cf0 comp rx=0 tx=0).stop 2024-05-27T08:05:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.018+0000 7f57f0e00640 1 -- 172.21.15.90:0/1672302444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57ec10a1f0 msgr2=0x7f57ec1a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.018+0000 7f57f0e00640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57ec10a1f0 0x7f57ec1a95e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f57d400d950 tx=0x7f57d400de10 comp rx=0 tx=0).stop 2024-05-27T08:05:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.019+0000 7f57f0e00640 1 -- 172.21.15.90:0/1672302444 shutdown_connections 2024-05-27T08:05:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.019+0000 7f57f0e00640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f57d00b2960 0x7f57d00b4e70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.019+0000 7f57f0e00640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57ec11d240 0x7f57ec1addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.019+0000 7f57f0e00640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57ec11a8e0 0x7f57ec1a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.019+0000 7f57f0e00640 1 --2- 172.21.15.90:0/1672302444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f57ec10a1f0 0x7f57ec1a95e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.019+0000 7f57f0e00640 1 -- 172.21.15.90:0/1672302444 >> 172.21.15.90:0/1672302444 conn(0x7f57ec06e8f0 msgr2=0x7f57ec11acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.019+0000 7f57f0e00640 1 -- 172.21.15.90:0/1672302444 shutdown_connections 2024-05-27T08:05:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:19.019+0000 7f57f0e00640 1 -- 172.21.15.90:0/1672302444 wait complete. 2024-05-27T08:05:19.363 INFO:teuthology.orchestra.run.smithi090.stdout:transaction type: 2024-05-27T08:05:19.363 INFO:teuthology.orchestra.run.smithi090.stdout:scaling factor: 32 2024-05-27T08:05:19.363 INFO:teuthology.orchestra.run.smithi090.stdout:query mode: simple 2024-05-27T08:05:19.363 INFO:teuthology.orchestra.run.smithi090.stdout:number of clients: 100 2024-05-27T08:05:19.363 INFO:teuthology.orchestra.run.smithi090.stdout:number of threads: 4 2024-05-27T08:05:19.364 INFO:teuthology.orchestra.run.smithi090.stdout:duration: 900 s 2024-05-27T08:05:19.364 INFO:teuthology.orchestra.run.smithi090.stdout:number of transactions actually processed: 470453 2024-05-27T08:05:19.364 INFO:teuthology.orchestra.run.smithi090.stdout:latency average = 191.184 ms 2024-05-27T08:05:19.364 INFO:teuthology.orchestra.run.smithi090.stdout:latency stddev = 239.748 ms 2024-05-27T08:05:19.364 INFO:teuthology.orchestra.run.smithi090.stdout:tps = 522.415088 (including connections establishing) 2024-05-27T08:05:19.364 INFO:teuthology.orchestra.run.smithi090.stdout:tps = 522.428291 (excluding connections establishing) 2024-05-27T08:05:19.370 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-05-27T08:05:19.944 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:19 smithi090 ceph-mon[31429]: pgmap v610: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 381 op/s 2024-05-27T08:05:20.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:19 smithi136 ceph-mon[35324]: pgmap v610: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 381 op/s 2024-05-27T08:05:20.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:19 smithi129 ceph-mon[35254]: pgmap v610: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.0 MiB/s wr, 381 op/s 2024-05-27T08:05:21.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:21 smithi090 ceph-mon[31429]: pgmap v611: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 513 op/s 2024-05-27T08:05:22.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:21 smithi136 ceph-mon[35324]: pgmap v611: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 513 op/s 2024-05-27T08:05:22.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:21 smithi129 ceph-mon[35254]: pgmap v611: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 513 op/s 2024-05-27T08:05:22.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.074+0000 7f12a9200700 1 -- 172.21.15.90:0/3265508078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12a41054b0 msgr2=0x7f12a41058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:22.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.074+0000 7f12a9200700 1 --2- 172.21.15.90:0/3265508078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12a41054b0 0x7f12a41058c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f1290009990 tx=0x7f1290009ca0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.074+0000 7f12a9200700 1 -- 172.21.15.90:0/3265508078 shutdown_connections 2024-05-27T08:05:22.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.074+0000 7f12a9200700 1 --2- 172.21.15.90:0/3265508078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12a410e8e0 0x7f12a4110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.074+0000 7f12a9200700 1 --2- 172.21.15.90:0/3265508078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12a41054b0 0x7f12a41058c0 secure :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f1290009990 tx=0x7f1290009ca0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.074+0000 7f12a9200700 1 --2- 172.21.15.90:0/3265508078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12a4105e00 0x7f12a410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.074+0000 7f12a9200700 1 -- 172.21.15.90:0/3265508078 >> 172.21.15.90:0/3265508078 conn(0x7f12a40fedb0 msgr2=0x7f12a41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:22.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.075+0000 7f12a9200700 1 -- 172.21.15.90:0/3265508078 shutdown_connections 2024-05-27T08:05:22.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.075+0000 7f12a9200700 1 -- 172.21.15.90:0/3265508078 wait complete. 2024-05-27T08:05:22.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.077+0000 7f12a9200700 1 Processor -- start 2024-05-27T08:05:22.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.077+0000 7f12a9200700 1 -- start start 2024-05-27T08:05:22.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.079+0000 7f12a9200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12a4105e00 0x7f12a41a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.079+0000 7f12a9200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12a410e8e0 0x7f12a41a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.080+0000 7f12a9200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12a41ab080 0x7f12a41ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.080+0000 7f12a9200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4114330 con 0x7f12a4105e00 2024-05-27T08:05:22.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.080+0000 7f12a9200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f12a41141b0 con 0x7f12a41ab080 2024-05-27T08:05:22.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.080+0000 7f12a9200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f12a41144b0 con 0x7f12a410e8e0 2024-05-27T08:05:22.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.080+0000 7f12a1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12a4105e00 0x7f12a41a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.080+0000 7f12a1400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12a410e8e0 0x7f12a41a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.080+0000 7f12a1400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12a410e8e0 0x7f12a41a27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42500/0 (socket says 172.21.15.90:42500) 2024-05-27T08:05:22.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.080+0000 7f12a1400700 1 -- 172.21.15.90:0/206190980 learned_addr learned my addr 172.21.15.90:0/206190980 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:22.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.080+0000 7f12a2800700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12a41ab080 0x7f12a41ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.081+0000 7f12a1400700 1 -- 172.21.15.90:0/206190980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12a41ab080 msgr2=0x7f12a41ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:22.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.081+0000 7f12a1400700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12a41ab080 0x7f12a41ad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.081+0000 7f12a1400700 1 -- 172.21.15.90:0/206190980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12a4105e00 msgr2=0x7f12a41a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:22.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.081+0000 7f12a1400700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12a4105e00 0x7f12a41a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.081+0000 7f12a1400700 1 -- 172.21.15.90:0/206190980 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a41adba0 con 0x7f12a410e8e0 2024-05-27T08:05:22.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.082+0000 7f12a1400700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12a410e8e0 0x7f12a41a27e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f129400d950 tx=0x7f129400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:22.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.082+0000 7f129aa00700 1 -- 172.21.15.90:0/206190980 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294016070 con 0x7f12a410e8e0 2024-05-27T08:05:22.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.082+0000 7f129aa00700 1 -- 172.21.15.90:0/206190980 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f129400fd20 con 0x7f12a410e8e0 2024-05-27T08:05:22.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.082+0000 7f129aa00700 1 -- 172.21.15.90:0/206190980 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129400e3b0 con 0x7f12a410e8e0 2024-05-27T08:05:22.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.083+0000 7f12a9200700 1 -- 172.21.15.90:0/206190980 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a41ade90 con 0x7f12a410e8e0 2024-05-27T08:05:22.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.083+0000 7f12a9200700 1 -- 172.21.15.90:0/206190980 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a41ae290 con 0x7f12a410e8e0 2024-05-27T08:05:22.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.091+0000 7f129aa00700 1 -- 172.21.15.90:0/206190980 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1294019070 con 0x7f12a410e8e0 2024-05-27T08:05:22.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.094+0000 7f129aa00700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f12880b2a70 0x7f12880b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.094+0000 7f129aa00700 1 -- 172.21.15.90:0/206190980 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f12940e2330 con 0x7f12a410e8e0 2024-05-27T08:05:22.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.094+0000 7f12a1e00700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f12880b2a70 0x7f12880b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.097+0000 7f12a1e00700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f12880b2a70 0x7f12880b4f80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f12900098e0 tx=0x7f129000b540 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:22.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.669+0000 7f12b7e45b80 1 Processor -- start 2024-05-27T08:05:22.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.669+0000 7f12b7e45b80 1 -- start start 2024-05-27T08:05:22.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.670+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab423b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.670+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.670+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 0x56278ab4f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.670+0000 7f12a1e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.670+0000 7f12a1e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab423b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48530/0 (socket says 172.21.15.90:48530) 2024-05-27T08:05:22.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.670+0000 7f12a1400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.670+0000 7f12a1400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42506/0 (socket says 172.21.15.90:42506) 2024-05-27T08:05:22.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.675+0000 7f12a2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 0x56278ab4f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.676+0000 7f12a2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 0x56278ab4f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51516/0 (socket says 172.21.15.90:51516) 2024-05-27T08:05:22.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.679+0000 7f12a1e00700 1 -- 172.21.15.90:0/2597474145 learned_addr learned my addr 172.21.15.90:0/2597474145 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:22.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.679+0000 7f12b7e45b80 1 -- 172.21.15.90:0/2597474145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56278aa70ec0 con 0x56278ab42ef0 2024-05-27T08:05:22.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.680+0000 7f12b7e45b80 1 -- 172.21.15.90:0/2597474145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56278aa60a60 con 0x56278aaafa70 2024-05-27T08:05:22.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.680+0000 7f12b7e45b80 1 -- 172.21.15.90:0/2597474145 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56278aa64aa0 con 0x56278a9c3ec0 2024-05-27T08:05:22.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.681+0000 7f12a2800700 1 -- 172.21.15.90:0/2597474145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 msgr2=0x56278ab428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:22.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.681+0000 7f12a2800700 1 --2- 172.21.15.90:0/2597474145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.681+0000 7f12a2800700 1 -- 172.21.15.90:0/2597474145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 msgr2=0x56278ab423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:22.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.681+0000 7f12a2800700 1 --2- 172.21.15.90:0/2597474145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.681+0000 7f12a2800700 1 -- 172.21.15.90:0/2597474145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278aa294a0 con 0x56278ab42ef0 2024-05-27T08:05:22.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.682+0000 7f12a2800700 1 --2- 172.21.15.90:0/2597474145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 0x56278ab4f190 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f129c003e30 tx=0x7f129c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab8d70dcf7bb7b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:22.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.682+0000 7f128f400700 1 -- 172.21.15.90:0/2597474145 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c004bb0 con 0x56278ab42ef0 2024-05-27T08:05:22.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.683+0000 7f128f400700 1 -- 172.21.15.90:0/2597474145 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f129c004800 con 0x56278ab42ef0 2024-05-27T08:05:22.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.683+0000 7f128f400700 1 -- 172.21.15.90:0/2597474145 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c007c80 con 0x56278ab42ef0 2024-05-27T08:05:22.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.685+0000 7f12b7e45b80 1 -- 172.21.15.90:0/2597474145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 msgr2=0x56278ab4f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:22.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.685+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/2597474145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 0x56278ab4f190 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f129c003e30 tx=0x7f129c009c20 comp rx=0 tx=0).stop 2024-05-27T08:05:22.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.685+0000 7f12b7e45b80 1 -- 172.21.15.90:0/2597474145 shutdown_connections 2024-05-27T08:05:22.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.685+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/2597474145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 0x56278ab4f190 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.685+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/2597474145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.685+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/2597474145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.686+0000 7f12b7e45b80 1 -- 172.21.15.90:0/2597474145 >> 172.21.15.90:0/2597474145 conn(0x56278aa5d410 msgr2=0x56278ab3e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:22.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.688+0000 7f12b7e45b80 1 -- 172.21.15.90:0/2597474145 shutdown_connections 2024-05-27T08:05:22.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.688+0000 7f12b7e45b80 1 -- 172.21.15.90:0/2597474145 wait complete. 2024-05-27T08:05:22.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.689+0000 7f12b7e45b80 1 -- start start 2024-05-27T08:05:22.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.689+0000 7f12b7e45b80 1 Processor -- start 2024-05-27T08:05:22.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.691+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 0x56278ab4c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.691+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab4cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.691+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278ab42ef0 0x56278ab4d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.692+0000 7f12a1400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab4cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.692+0000 7f12a2800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278ab42ef0 0x56278ab4d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.692+0000 7f12a1400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab4cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48536/0 (socket says 172.21.15.90:48536) 2024-05-27T08:05:22.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.694+0000 7f12a1400700 1 -- 172.21.15.90:0/3682468055 learned_addr learned my addr 172.21.15.90:0/3682468055 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:22.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.692+0000 7f12b7e45b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56278a967020 con 0x56278a9c3ec0 2024-05-27T08:05:22.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.694+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3682468055 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56278aa017e0 con 0x56278aaafa70 2024-05-27T08:05:22.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.694+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3682468055 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56278ab4fc00 con 0x56278ab42ef0 2024-05-27T08:05:22.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.694+0000 7f12b7e45b80 10 client.0 _ll_register_callbacks cb 0x56278aa0acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:22.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.693+0000 7f12a1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 0x56278ab4c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.694+0000 7f12a1400700 1 -- 172.21.15.90:0/3682468055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278ab42ef0 msgr2=0x56278ab4d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:22.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.696+0000 7f12a1400700 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278ab42ef0 0x56278ab4d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.696+0000 7f12a1400700 1 -- 172.21.15.90:0/3682468055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 msgr2=0x56278ab4c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:22.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.696+0000 7f12a1400700 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 0x56278ab4c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.696+0000 7f12a1400700 1 -- 172.21.15.90:0/3682468055 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278a9b0950 con 0x56278aaafa70 2024-05-27T08:05:22.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.696+0000 7f12a1e00700 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 0x56278ab4c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:22.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.696+0000 7f12a2800700 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278ab42ef0 0x56278ab4d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:22.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.697+0000 7f12a1400700 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab4cc90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f12940a91d0 tx=0x7f12940a9780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14458efaf4461c00 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:22.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.697+0000 7f1283e00700 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-05-27T08:05:22.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.701+0000 7f1283e00700 1 -- 172.21.15.90:0/3682468055 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294020040 con 0x56278aaafa70 2024-05-27T08:05:22.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.701+0000 7f1283e00700 1 -- 172.21.15.90:0/3682468055 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f12940ad170 con 0x56278aaafa70 2024-05-27T08:05:22.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.701+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3682468055 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f129c004800 con 0x56278aaafa70 2024-05-27T08:05:22.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.704+0000 7f1283e00700 1 -- 172.21.15.90:0/3682468055 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f12940a9240 con 0x56278aaafa70 2024-05-27T08:05:22.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.704+0000 7f12b7e45b80 10 client.38216 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:22.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.704+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3682468055 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56278aa310e0 con 0x56278aaafa70 2024-05-27T08:05:22.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.706+0000 7f1283e00700 1 -- 172.21.15.90:0/3682468055 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f129c004800 con 0x56278aaafa70 2024-05-27T08:05:22.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.706+0000 7f1283e00700 1 -- 172.21.15.90:0/3682468055 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f12940c3bc0 con 0x56278aaafa70 2024-05-27T08:05:22.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.706+0000 7f12b7e45b80 10 client.38216 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:22.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.706+0000 7f12b7e45b80 10 client.38216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:22.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.706+0000 7f12b7e45b80 20 client.38216 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:22.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.706+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56278ab5ce60 0x56278ab5f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.706+0000 7f12b7e45b80 4 client.38216 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:22.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.706+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3682468055 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56278aa72e10 con 0x56278ab5ce60 2024-05-27T08:05:22.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.708+0000 7f12a1e00700 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56278ab5ce60 0x56278ab5f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.709+0000 7f1283e00700 10 client.38216 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:22.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.709+0000 7f12a1e00700 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56278ab5ce60 0x56278ab5f2d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=978c2f388ad4d8 server_cookie=5e2ecfd4ade379c7 in_seq=0 out_seq=0 2024-05-27T08:05:22.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.713+0000 7f1283e00700 1 -- 172.21.15.90:0/3682468055 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56278aa72e10 con 0x56278ab5ce60 2024-05-27T08:05:22.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.713+0000 7f1283e00700 10 client.38216 handle_command_reply: tid=0 2024-05-27T08:05:22.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.713+0000 7f12b7e45b80 1 client.38216 shutdown 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3682468055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 msgr2=0x56278ab4cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab4cc90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f12940a91d0 tx=0x7f12940a9780 comp rx=0 tx=0).stop 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3682468055 shutdown_connections 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56278ab5ce60 0x56278ab5f2d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278ab42ef0 0x56278ab4d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 0x56278ab4c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3682468055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab4cc90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3682468055 >> 172.21.15.90:0/3682468055 conn(0x56278aa5d410 msgr2=0x56278ab423d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f1283e00700 0 client.38216 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3682468055 shutdown_connections 2024-05-27T08:05:22.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.714+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3682468055 wait complete. 2024-05-27T08:05:22.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.715+0000 7f12b7e45b80 20 client.38216 trim_cache size 0 max 16384 2024-05-27T08:05:22.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.755+0000 7f12b7e45b80 1 Processor -- start 2024-05-27T08:05:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.755+0000 7f12b7e45b80 1 -- start start 2024-05-27T08:05:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.755+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 0x56278ab47f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.755+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278aaafa70 0x56278ab44980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.755+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278ab42ef0 0x56278ab42810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.755+0000 7f12b7e45b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1284010720 con 0x56278a9c3ec0 2024-05-27T08:05:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.755+0000 7f12b7e45b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f128400d770 con 0x56278ab42ef0 2024-05-27T08:05:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.755+0000 7f12b7e45b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f128400f980 con 0x56278aaafa70 2024-05-27T08:05:22.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.757+0000 7f12a1400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278aaafa70 0x56278ab44980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.757+0000 7f12a2800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278ab42ef0 0x56278ab42810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.757+0000 7f12a2800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278ab42ef0 0x56278ab42810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:48538/0 (socket says 172.21.15.90:48538) 2024-05-27T08:05:22.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.758+0000 7f12a2800700 1 -- 172.21.15.90:0/3993318427 learned_addr learned my addr 172.21.15.90:0/3993318427 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:22.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.758+0000 7f12a1e00700 1 --2- 172.21.15.90:0/3993318427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 0x56278ab47f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.759+0000 7f12a1400700 1 -- 172.21.15.90:0/3993318427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278ab42ef0 msgr2=0x56278ab42810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:22.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.759+0000 7f12a1400700 1 --2- 172.21.15.90:0/3993318427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278ab42ef0 0x56278ab42810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.759+0000 7f12a1400700 1 -- 172.21.15.90:0/3993318427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 msgr2=0x56278ab47f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.760+0000 7f12a1400700 1 --2- 172.21.15.90:0/3993318427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 0x56278ab47f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.760+0000 7f12a1400700 1 -- 172.21.15.90:0/3993318427 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278ab3a2c0 con 0x56278aaafa70 2024-05-27T08:05:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.761+0000 7f12a2800700 1 --2- 172.21.15.90:0/3993318427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278ab42ef0 0x56278ab42810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:05:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.761+0000 7f12a1e00700 1 --2- 172.21.15.90:0/3993318427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 0x56278ab47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:22.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.764+0000 7f12a1400700 1 --2- 172.21.15.90:0/3993318427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278aaafa70 0x56278ab44980 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f12940c3800 tx=0x7f12940a8dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80c7371caacbd360 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:22.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.766+0000 7f128ea00700 1 -- 172.21.15.90:0/3993318427 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294020070 con 0x56278aaafa70 2024-05-27T08:05:22.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.766+0000 7f128ea00700 1 -- 172.21.15.90:0/3993318427 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12940c38f0 con 0x56278aaafa70 2024-05-27T08:05:22.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.766+0000 7f128ea00700 1 -- 172.21.15.90:0/3993318427 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12940ad170 con 0x56278aaafa70 2024-05-27T08:05:22.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.767+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3993318427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278aaafa70 msgr2=0x56278ab44980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:22.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.767+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3993318427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278aaafa70 0x56278ab44980 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f12940c3800 tx=0x7f12940a8dc0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.768+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3993318427 shutdown_connections 2024-05-27T08:05:22.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.768+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3993318427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278ab42ef0 0x56278ab42810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.768+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3993318427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278a9c3ec0 0x56278ab47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.768+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3993318427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278aaafa70 0x56278ab44980 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.768+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3993318427 >> 172.21.15.90:0/3993318427 conn(0x56278aa5d410 msgr2=0x56278ab4c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:22.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.771+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3993318427 shutdown_connections 2024-05-27T08:05:22.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.771+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3993318427 wait complete. 2024-05-27T08:05:22.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.772+0000 7f12b7e45b80 1 -- start start 2024-05-27T08:05:22.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.772+0000 7f12b7e45b80 1 Processor -- start 2024-05-27T08:05:22.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.773+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.774+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab55610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.774+0000 7f12b7e45b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 0x56278ab4fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.774+0000 7f12b7e45b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f128400f2b0 con 0x56278ab42ef0 2024-05-27T08:05:22.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.775+0000 7f12b7e45b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f128400de40 con 0x56278aaafa70 2024-05-27T08:05:22.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.775+0000 7f12a1e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.775+0000 7f12a1e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:42524/0 (socket says 172.21.15.90:42524) 2024-05-27T08:05:22.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.775+0000 7f12a1e00700 1 -- 172.21.15.90:0/3421874340 learned_addr learned my addr 172.21.15.90:0/3421874340 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:22.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.775+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3421874340 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f128400e510 con 0x56278a9c3ec0 2024-05-27T08:05:22.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.775+0000 7f12a1400700 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab55610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.776+0000 7f12b7e45b80 10 client.0 _ll_register_callbacks cb 0x56278aa0acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:22.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.776+0000 7f12a2800700 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 0x56278ab4fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.777+0000 7f12a1e00700 1 -- 172.21.15.90:0/3421874340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 msgr2=0x56278ab55610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:22.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.777+0000 7f12a1e00700 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab55610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.777+0000 7f12a1e00700 1 -- 172.21.15.90:0/3421874340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 msgr2=0x56278ab4fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:22.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.777+0000 7f12a1e00700 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 0x56278ab4fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.777+0000 7f12a1e00700 1 -- 172.21.15.90:0/3421874340 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278ab50600 con 0x56278a9c3ec0 2024-05-27T08:05:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.778+0000 7f12a1e00700 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab550d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1290009990 tx=0x7f1290009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f0426e05719e8ed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.778+0000 7f1283e00700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-05-27T08:05:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.778+0000 7f1283e00700 1 -- 172.21.15.90:0/3421874340 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12900037c0 con 0x56278a9c3ec0 2024-05-27T08:05:22.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.778+0000 7f1283e00700 1 -- 172.21.15.90:0/3421874340 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f1290003960 con 0x56278a9c3ec0 2024-05-27T08:05:22.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.779+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3421874340 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12940c38f0 con 0x56278a9c3ec0 2024-05-27T08:05:22.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.780+0000 7f1283e00700 1 -- 172.21.15.90:0/3421874340 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f1290003b60 con 0x56278a9c3ec0 2024-05-27T08:05:22.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.780+0000 7f12b7e45b80 10 client.28312 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:22.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.780+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3421874340 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56278abc8420 con 0x56278a9c3ec0 2024-05-27T08:05:22.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.780+0000 7f1283e00700 1 -- 172.21.15.90:0/3421874340 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12940c38f0 con 0x56278a9c3ec0 2024-05-27T08:05:22.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.784+0000 7f1283e00700 1 -- 172.21.15.90:0/3421874340 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f1290022070 con 0x56278a9c3ec0 2024-05-27T08:05:22.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.784+0000 7f12b7e45b80 10 client.28312 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:22.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.784+0000 7f12b7e45b80 10 client.28312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:22.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.784+0000 7f12b7e45b80 20 client.28312 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:22.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.784+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56278abc8b30 0x56278abcaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:22.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.784+0000 7f12b7e45b80 4 client.28312 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-27T08:05:22.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.784+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3421874340 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56278aa6c5f0 con 0x56278abc8b30 2024-05-27T08:05:22.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.785+0000 7f12a1400700 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56278abc8b30 0x56278abcaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:22.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.785+0000 7f1283e00700 10 client.28312 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:22.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.785+0000 7f12a1400700 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56278abc8b30 0x56278abcaf50 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b156e2f2b94339b server_cookie=cc69e80ec85fe1db in_seq=0 out_seq=0 2024-05-27T08:05:22.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.794+0000 7f1283e00700 1 -- 172.21.15.90:0/3421874340 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56278aa6c5f0 con 0x56278abc8b30 2024-05-27T08:05:22.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.794+0000 7f1283e00700 10 client.28312 handle_command_reply: tid=0 2024-05-27T08:05:22.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.794+0000 7f12b7e45b80 1 client.28312 shutdown 2024-05-27T08:05:22.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.794+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3421874340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 msgr2=0x56278ab550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:22.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.794+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab550d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1290009990 tx=0x7f1290009ca0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.795+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3421874340 shutdown_connections 2024-05-27T08:05:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.795+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x56278abc8b30 0x56278abcaf50 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.795+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56278ab42ef0 0x56278ab4fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.795+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278a9c3ec0 0x56278ab550d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.795+0000 7f12b7e45b80 1 --2- 172.21.15.90:0/3421874340 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56278aaafa70 0x56278ab55610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.795+0000 7f1283e00700 0 client.28312 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:22.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.795+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3421874340 >> 172.21.15.90:0/3421874340 conn(0x56278aa5d410 msgr2=0x56278ab4cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:22.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.796+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3421874340 shutdown_connections 2024-05-27T08:05:22.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.796+0000 7f12b7e45b80 1 -- 172.21.15.90:0/3421874340 wait complete. 2024-05-27T08:05:22.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.796+0000 7f12b7e45b80 20 client.28312 trim_cache size 0 max 16384 2024-05-27T08:05:22.799 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:05:22.799 INFO:teuthology.orchestra.run.smithi090.stdout: "return_code": 0, 2024-05-27T08:05:22.799 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_tag": "9dd455bd-f6b1-466c-a80f-8aea426829ea", 2024-05-27T08:05:22.799 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": "asynchronous" 2024-05-27T08:05:22.799 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:05:22.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.798+0000 7f12a9200700 1 -- 172.21.15.90:0/206190980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f12880b2a70 msgr2=0x7f12880b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:22.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.798+0000 7f12a9200700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f12880b2a70 0x7f12880b4f80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f12900098e0 tx=0x7f129000b540 comp rx=0 tx=0).stop 2024-05-27T08:05:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.798+0000 7f12a9200700 1 -- 172.21.15.90:0/206190980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12a410e8e0 msgr2=0x7f12a41a27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.798+0000 7f12a9200700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12a410e8e0 0x7f12a41a27e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f129400d950 tx=0x7f129400dc60 comp rx=0 tx=0).stop 2024-05-27T08:05:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.798+0000 7f12a9200700 1 -- 172.21.15.90:0/206190980 shutdown_connections 2024-05-27T08:05:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.798+0000 7f12a9200700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f12880b2a70 0x7f12880b4f80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.798+0000 7f12a9200700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f12a41ab080 0x7f12a41ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.798+0000 7f12a9200700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12a4105e00 0x7f12a41a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.798+0000 7f12a9200700 1 --2- 172.21.15.90:0/206190980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12a410e8e0 0x7f12a41a27e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:22.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.798+0000 7f12a9200700 1 -- 172.21.15.90:0/206190980 >> 172.21.15.90:0/206190980 conn(0x7f12a40fedb0 msgr2=0x7f12a4100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:22.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.799+0000 7f12a9200700 1 -- 172.21.15.90:0/206190980 shutdown_connections 2024-05-27T08:05:22.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:22.801+0000 7f12a9200700 1 -- 172.21.15.90:0/206190980 wait complete. 2024-05-27T08:05:23.754 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-27T08:05:23.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:23 smithi090 ceph-mon[31429]: pgmap v612: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 470 op/s 2024-05-27T08:05:23.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:23 smithi090 ceph-mon[31429]: scrub queued for path: / 2024-05-27T08:05:23.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:23 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:23.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:23 smithi090 ceph-mon[31429]: scrub summary: active paths [/] 2024-05-27T08:05:24.003 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:23 smithi136 ceph-mon[35324]: pgmap v612: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 470 op/s 2024-05-27T08:05:24.003 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:23 smithi136 ceph-mon[35324]: scrub queued for path: / 2024-05-27T08:05:24.003 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:23 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:24.003 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:23 smithi136 ceph-mon[35324]: scrub summary: active paths [/] 2024-05-27T08:05:24.066 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:23 smithi129 ceph-mon[35254]: pgmap v612: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.8 MiB/s wr, 470 op/s 2024-05-27T08:05:24.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:23 smithi129 ceph-mon[35254]: scrub queued for path: / 2024-05-27T08:05:24.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:23 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:24.067 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:23 smithi129 ceph-mon[35254]: scrub summary: active paths [/] 2024-05-27T08:05:24.496 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:26.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:25 smithi136 ceph-mon[35324]: pgmap v613: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 649 op/s 2024-05-27T08:05:26.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:25 smithi129 ceph-mon[35254]: pgmap v613: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 649 op/s 2024-05-27T08:05:26.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:25 smithi090 ceph-mon[31429]: pgmap v613: 129 pgs: 129 active+clean; 3.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.3 MiB/s wr, 649 op/s 2024-05-27T08:05:28.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:27 smithi136 ceph-mon[35324]: pgmap v614: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 721 op/s 2024-05-27T08:05:28.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:27 smithi129 ceph-mon[35254]: pgmap v614: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 721 op/s 2024-05-27T08:05:28.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:27 smithi090 ceph-mon[31429]: pgmap v614: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 MiB/s wr, 721 op/s 2024-05-27T08:05:29.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:29 smithi090 ceph-mon[31429]: pgmap v615: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 753 op/s 2024-05-27T08:05:29.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:29 smithi090 ceph-mon[31429]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:29.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:29 smithi090 ceph-mon[31429]: scrub summary: idle 2024-05-27T08:05:30.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:29 smithi136 ceph-mon[35324]: pgmap v615: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 753 op/s 2024-05-27T08:05:30.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:29 smithi136 ceph-mon[35324]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:30.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:29 smithi136 ceph-mon[35324]: scrub summary: idle 2024-05-27T08:05:30.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:29 smithi129 ceph-mon[35254]: pgmap v615: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.9 MiB/s wr, 753 op/s 2024-05-27T08:05:30.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:29 smithi129 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-05-27T08:05:30.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:29 smithi129 ceph-mon[35254]: scrub summary: idle 2024-05-27T08:05:30.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.959+0000 7fc5b8a00700 1 -- 172.21.15.90:0/883656451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b41029b0 msgr2=0x7fc5b4102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:30.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.959+0000 7fc5b8a00700 1 --2- 172.21.15.90:0/883656451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b41029b0 0x7fc5b4102dc0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00aaa0 tx=0x7fc59c00adb0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.959+0000 7fc5b8a00700 1 -- 172.21.15.90:0/883656451 shutdown_connections 2024-05-27T08:05:30.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.959+0000 7fc5b8a00700 1 --2- 172.21.15.90:0/883656451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5b4103300 0x7fc5b410f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.959+0000 7fc5b8a00700 1 --2- 172.21.15.90:0/883656451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5b41089e0 0x7fc5b4108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.959+0000 7fc5b8a00700 1 --2- 172.21.15.90:0/883656451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b41029b0 0x7fc5b4102dc0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.959+0000 7fc5b8a00700 1 -- 172.21.15.90:0/883656451 >> 172.21.15.90:0/883656451 conn(0x7fc5b40fe6e0 msgr2=0x7fc5b4100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:30.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.959+0000 7fc5b8a00700 1 -- 172.21.15.90:0/883656451 shutdown_connections 2024-05-27T08:05:30.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.959+0000 7fc5b8a00700 1 -- 172.21.15.90:0/883656451 wait complete. 2024-05-27T08:05:30.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b8a00700 1 Processor -- start 2024-05-27T08:05:30.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b8a00700 1 -- start start 2024-05-27T08:05:30.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b8a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5b41029b0 0x7fc5b419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b8a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5b4103300 0x7fc5b419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b41089e0 0x7fc5b41a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b8a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41121e0 con 0x7fc5b41089e0 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b8a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b4112060 con 0x7fc5b4103300 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b8a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b4112360 con 0x7fc5b41029b0 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b41089e0 0x7fc5b41a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b41089e0 0x7fc5b41a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51326/0 (socket says 172.21.15.90:51326) 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.961+0000 7fc5b2800700 1 -- 172.21.15.90:0/1742728881 learned_addr learned my addr 172.21.15.90:0/1742728881 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.962+0000 7fc5abe00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5b4103300 0x7fc5b419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.962+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5b41029b0 0x7fc5b419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.962+0000 7fc5b2800700 1 -- 172.21.15.90:0/1742728881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5b41029b0 msgr2=0x7fc5b419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.962+0000 7fc5b2800700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5b41029b0 0x7fc5b419cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.962+0000 7fc5b2800700 1 -- 172.21.15.90:0/1742728881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5b4103300 msgr2=0x7fc5b419d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:30.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.962+0000 7fc5b2800700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5b4103300 0x7fc5b419d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.962+0000 7fc5b2800700 1 -- 172.21.15.90:0/1742728881 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b41a1eb0 con 0x7fc5b41089e0 2024-05-27T08:05:30.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.962+0000 7fc5b2800700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b41089e0 0x7fc5b41a1810 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00d980 tx=0x7fc5ac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:30.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.963+0000 7fc5aaa00700 1 -- 172.21.15.90:0/1742728881 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac017070 con 0x7fc5b41089e0 2024-05-27T08:05:30.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.963+0000 7fc5aaa00700 1 -- 172.21.15.90:0/1742728881 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5ac009dd0 con 0x7fc5b41089e0 2024-05-27T08:05:30.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.963+0000 7fc5aaa00700 1 -- 172.21.15.90:0/1742728881 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac005260 con 0x7fc5b41089e0 2024-05-27T08:05:30.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.963+0000 7fc5b8a00700 1 -- 172.21.15.90:0/1742728881 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b41a21a0 con 0x7fc5b41089e0 2024-05-27T08:05:30.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.963+0000 7fc5b8a00700 1 -- 172.21.15.90:0/1742728881 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b41a9ac0 con 0x7fc5b41089e0 2024-05-27T08:05:30.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.967+0000 7fc5aaa00700 1 -- 172.21.15.90:0/1742728881 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc5ac009940 con 0x7fc5b41089e0 2024-05-27T08:05:30.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.969+0000 7fc5aaa00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc5980b29f0 0x7fc5980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.969+0000 7fc5aaa00700 1 -- 172.21.15.90:0/1742728881 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fc5ac0e2a80 con 0x7fc5b41089e0 2024-05-27T08:05:30.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.969+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc5980b29f0 0x7fc5980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.970+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc5980b29f0 0x7fc5980b4f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4103af0 tx=0x7fc5a4009040 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:30.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.976+0000 7fc5c7495b80 1 Processor -- start 2024-05-27T08:05:30.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.976+0000 7fc5c7495b80 1 -- start start 2024-05-27T08:05:30.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.976+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2a58e40 0x561db2bdd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.976+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 0x561db2bd7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.976+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2bd81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.976+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561db2b05e40 con 0x561db2bd7db0 2024-05-27T08:05:30.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.976+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561db2af59e0 con 0x561db2a58e40 2024-05-27T08:05:30.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.976+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561db2af9a20 con 0x561db2bddd30 2024-05-27T08:05:30.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.977+0000 7fc5b2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2bd81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.977+0000 7fc5abe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 0x561db2bd7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.977+0000 7fc5b1e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2a58e40 0x561db2bdd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.977+0000 7fc5b2800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2bd81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51330/0 (socket says 172.21.15.90:51330) 2024-05-27T08:05:30.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.977+0000 7fc5abe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 0x561db2bd7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.90:46776/0 (socket says 172.21.15.90:46776) 2024-05-27T08:05:30.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.977+0000 7fc5abe00700 1 -- 172.21.15.90:0/2065428391 learned_addr learned my addr 172.21.15.90:0/2065428391 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.978+0000 7fc5b2800700 1 -- 172.21.15.90:0/2065428391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 msgr2=0x561db2bd7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.978+0000 7fc5b2800700 1 --2- 172.21.15.90:0/2065428391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 0x561db2bd7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.978+0000 7fc5b2800700 1 -- 172.21.15.90:0/2065428391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2a58e40 msgr2=0x561db2bdd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.978+0000 7fc5b2800700 1 --2- 172.21.15.90:0/2065428391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2a58e40 0x561db2bdd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.978+0000 7fc5b2800700 1 -- 172.21.15.90:0/2065428391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db2abe420 con 0x561db2bd7db0 2024-05-27T08:05:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.978+0000 7fc5b2800700 1 --2- 172.21.15.90:0/2065428391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2bd81c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0dfda0 tx=0x7fc5ac0a91b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64408dfa8351e8fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.978+0000 7fc5a3400700 1 -- 172.21.15.90:0/2065428391 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0aa030 con 0x561db2bd7db0 2024-05-27T08:05:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.978+0000 7fc5a3400700 1 -- 172.21.15.90:0/2065428391 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5ac08fe50 con 0x561db2bd7db0 2024-05-27T08:05:30.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.979+0000 7fc5a3400700 1 -- 172.21.15.90:0/2065428391 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0cabe0 con 0x561db2bd7db0 2024-05-27T08:05:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.979+0000 7fc5c7495b80 1 -- 172.21.15.90:0/2065428391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 msgr2=0x561db2bd81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.979+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/2065428391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2bd81c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0dfda0 tx=0x7fc5ac0a91b0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.980+0000 7fc5c7495b80 1 -- 172.21.15.90:0/2065428391 shutdown_connections 2024-05-27T08:05:30.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.980+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/2065428391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2bd81c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.980+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/2065428391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2a58e40 0x561db2bdd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.980+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/2065428391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 0x561db2bd7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.980+0000 7fc5c7495b80 1 -- 172.21.15.90:0/2065428391 >> 172.21.15.90:0/2065428391 conn(0x561db2ab8fc0 msgr2=0x561db2bd28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.980+0000 7fc5c7495b80 1 -- 172.21.15.90:0/2065428391 shutdown_connections 2024-05-27T08:05:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.980+0000 7fc5c7495b80 1 -- 172.21.15.90:0/2065428391 wait complete. 2024-05-27T08:05:30.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.980+0000 7fc5c7495b80 1 -- start start 2024-05-27T08:05:30.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.981+0000 7fc5c7495b80 1 Processor -- start 2024-05-27T08:05:30.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.981+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2a58e40 0x561db2be1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.981+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2be1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.981+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bddd30 0x561db2be20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.981+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561db29fbfa0 con 0x561db2bd7db0 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.981+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561db2a96760 con 0x561db2bddd30 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.981+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561db2be4800 con 0x561db2a58e40 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.981+0000 7fc5c7495b80 10 client.0 _ll_register_callbacks cb 0x561db2a9fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.982+0000 7fc5abe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2be1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.982+0000 7fc5abe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2be1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51340/0 (socket says 172.21.15.90:51340) 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.982+0000 7fc5abe00700 1 -- 172.21.15.90:0/684386147 learned_addr learned my addr 172.21.15.90:0/684386147 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.982+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2a58e40 0x561db2be1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.982+0000 7fc5b2800700 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bddd30 0x561db2be20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.982+0000 7fc5abe00700 1 -- 172.21.15.90:0/684386147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2a58e40 msgr2=0x561db2be1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:30.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.982+0000 7fc5abe00700 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2a58e40 0x561db2be1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.982+0000 7fc5abe00700 1 -- 172.21.15.90:0/684386147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bddd30 msgr2=0x561db2be20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:30.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.982+0000 7fc5abe00700 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bddd30 0x561db2be20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.982+0000 7fc5abe00700 1 -- 172.21.15.90:0/684386147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db2a458d0 con 0x561db2bd7db0 2024-05-27T08:05:30.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.983+0000 7fc5abe00700 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2be1b90 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc59c000c00 tx=0x7fc59c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9d46d340b71fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:30.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.983+0000 7fc597e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:30.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.987+0000 7fc597e00700 1 -- 172.21.15.90:0/684386147 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c025070 con 0x561db2bd7db0 2024-05-27T08:05:30.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.987+0000 7fc597e00700 1 -- 172.21.15.90:0/684386147 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fc59c00cd00 con 0x561db2bd7db0 2024-05-27T08:05:30.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.987+0000 7fc5c7495b80 1 -- 172.21.15.90:0/684386147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ac08fe50 con 0x561db2bd7db0 2024-05-27T08:05:30.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.988+0000 7fc597e00700 1 -- 172.21.15.90:0/684386147 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc59c02f050 con 0x561db2bd7db0 2024-05-27T08:05:30.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.988+0000 7fc5c7495b80 10 client.18519 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:30.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.988+0000 7fc5c7495b80 1 -- 172.21.15.90:0/684386147 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561db2ac6060 con 0x561db2bd7db0 2024-05-27T08:05:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.988+0000 7fc597e00700 1 -- 172.21.15.90:0/684386147 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5ac08fe50 con 0x561db2bd7db0 2024-05-27T08:05:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.989+0000 7fc597e00700 1 -- 172.21.15.90:0/684386147 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fc59c00b040 con 0x561db2bd7db0 2024-05-27T08:05:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.989+0000 7fc5c7495b80 10 client.18519 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.989+0000 7fc5c7495b80 10 client.18519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.989+0000 7fc5c7495b80 20 client.18519 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.989+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561db2bf19f0 0x561db2bf3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:30.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.989+0000 7fc5c7495b80 4 client.18519 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:30.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.989+0000 7fc5c7495b80 1 -- 172.21.15.90:0/684386147 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561db2b07d90 con 0x561db2bf19f0 2024-05-27T08:05:30.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.990+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561db2bf19f0 0x561db2bf3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:30.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.991+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561db2bf19f0 0x561db2bf3e40 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4146eceda9bf4eb5 server_cookie=3e1814084536e6f3 in_seq=0 out_seq=0 2024-05-27T08:05:30.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.991+0000 7fc597e00700 10 client.18519 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:30.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.992+0000 7fc597e00700 1 -- 172.21.15.90:0/684386147 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x561db2b07d90 con 0x561db2bf19f0 2024-05-27T08:05:30.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.992+0000 7fc597e00700 10 client.18519 handle_command_reply: tid=0 2024-05-27T08:05:30.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 client.18519 shutdown 2024-05-27T08:05:30.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 -- 172.21.15.90:0/684386147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 msgr2=0x561db2be1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:30.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2be1b90 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc59c000c00 tx=0x7fc59c003730 comp rx=0 tx=0).stop 2024-05-27T08:05:30.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 -- 172.21.15.90:0/684386147 shutdown_connections 2024-05-27T08:05:30.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561db2bf19f0 0x561db2bf3e40 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bddd30 0x561db2be20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2a58e40 0x561db2be1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/684386147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2bd7db0 0x561db2be1b90 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:30.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc597e00700 0 client.18519 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:30.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 -- 172.21.15.90:0/684386147 >> 172.21.15.90:0/684386147 conn(0x561db2ab8fc0 msgr2=0x561db2bd2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:30.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 -- 172.21.15.90:0/684386147 shutdown_connections 2024-05-27T08:05:30.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.993+0000 7fc5c7495b80 1 -- 172.21.15.90:0/684386147 wait complete. 2024-05-27T08:05:30.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:30.994+0000 7fc5c7495b80 20 client.18519 trim_cache size 0 max 16384 2024-05-27T08:05:31.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.020+0000 7fc5c7495b80 1 Processor -- start 2024-05-27T08:05:31.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.020+0000 7fc5c7495b80 1 -- start start 2024-05-27T08:05:31.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.021+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bd7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:31.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.021+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bd7db0 0x561db2be05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:31.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.021+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bddd30 0x561db2bdbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:31.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.021+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41b4190 con 0x561db2a58e40 2024-05-27T08:05:31.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.021+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41b11e0 con 0x561db2bddd30 2024-05-27T08:05:31.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.021+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41b33f0 con 0x561db2bd7db0 2024-05-27T08:05:31.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.021+0000 7fc5b1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bd7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:31.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.021+0000 7fc5b1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bd7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51344/0 (socket says 172.21.15.90:51344) 2024-05-27T08:05:31.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.021+0000 7fc5b1e00700 1 -- 172.21.15.90:0/3432842232 learned_addr learned my addr 172.21.15.90:0/3432842232 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:31.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.022+0000 7fc5b2800700 1 --2- 172.21.15.90:0/3432842232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bddd30 0x561db2bdbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:31.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.022+0000 7fc5b1e00700 1 -- 172.21.15.90:0/3432842232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bd7db0 msgr2=0x561db2be05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:31.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.022+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/3432842232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bd7db0 0x561db2be05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.022+0000 7fc5b1e00700 1 -- 172.21.15.90:0/3432842232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bddd30 msgr2=0x561db2bdbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:31.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.022+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/3432842232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bddd30 0x561db2bdbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.022+0000 7fc5b1e00700 1 -- 172.21.15.90:0/3432842232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db2be2710 con 0x561db2a58e40 2024-05-27T08:05:31.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.022+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/3432842232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bd7870 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4007b70 tx=0x7fc5a4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6417647ee767513 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:31.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.023+0000 7fc5a2a00700 1 -- 172.21.15.90:0/3432842232 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4024410 con 0x561db2a58e40 2024-05-27T08:05:31.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.023+0000 7fc5a2a00700 1 -- 172.21.15.90:0/3432842232 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5a4030070 con 0x561db2a58e40 2024-05-27T08:05:31.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.023+0000 7fc5a2a00700 1 -- 172.21.15.90:0/3432842232 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a402a2c0 con 0x561db2a58e40 2024-05-27T08:05:31.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.024+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3432842232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 msgr2=0x561db2bd7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:31.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.024+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/3432842232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bd7870 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4007b70 tx=0x7fc5a4015000 comp rx=0 tx=0).stop 2024-05-27T08:05:31.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.024+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3432842232 shutdown_connections 2024-05-27T08:05:31.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.024+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/3432842232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bddd30 0x561db2bdbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.024+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/3432842232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bd7870 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.024+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/3432842232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bd7db0 0x561db2be05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.024+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3432842232 >> 172.21.15.90:0/3432842232 conn(0x561db2ab8fc0 msgr2=0x561db2be1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:31.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.024+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3432842232 shutdown_connections 2024-05-27T08:05:31.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.024+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3432842232 wait complete. 2024-05-27T08:05:31.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.025+0000 7fc5c7495b80 1 -- start start 2024-05-27T08:05:31.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.025+0000 7fc5c7495b80 1 Processor -- start 2024-05-27T08:05:31.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:31.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bd7db0 0x561db2bf8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:31.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5c7495b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 0x561db2bf94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:31.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41b2d20 con 0x561db2a58e40 2024-05-27T08:05:31.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41b18b0 con 0x561db2bd7db0 2024-05-27T08:05:31.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5c7495b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41b1f80 con 0x561db2bddd30 2024-05-27T08:05:31.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5c7495b80 10 client.0 _ll_register_callbacks cb 0x561db2a9fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:31.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5b1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:31.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5b1e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bea1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51354/0 (socket says 172.21.15.90:51354) 2024-05-27T08:05:31.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5b1e00700 1 -- 172.21.15.90:0/3099717538 learned_addr learned my addr 172.21.15.90:0/3099717538 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:31.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.026+0000 7fc5abe00700 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bd7db0 0x561db2bf8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:31.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.027+0000 7fc5b2800700 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 0x561db2bf94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:31.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.027+0000 7fc5b1e00700 1 -- 172.21.15.90:0/3099717538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 msgr2=0x561db2bf94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:31.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.027+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 0x561db2bf94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.027+0000 7fc5b1e00700 1 -- 172.21.15.90:0/3099717538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bd7db0 msgr2=0x561db2bf8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:31.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.027+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bd7db0 0x561db2bf8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.027+0000 7fc5b1e00700 1 -- 172.21.15.90:0/3099717538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db2bf9c60 con 0x561db2a58e40 2024-05-27T08:05:31.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.027+0000 7fc5b1e00700 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bea1e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009670 tx=0x7fc5a40247e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8299c94cf278a71e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:31.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.027+0000 7fc597e00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:31.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.028+0000 7fc597e00700 1 -- 172.21.15.90:0/3099717538 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a403c640 con 0x561db2a58e40 2024-05-27T08:05:31.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.028+0000 7fc597e00700 1 -- 172.21.15.90:0/3099717538 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fc5a403c7e0 con 0x561db2a58e40 2024-05-27T08:05:31.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.028+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3099717538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5a4030070 con 0x561db2a58e40 2024-05-27T08:05:31.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.028+0000 7fc597e00700 1 -- 172.21.15.90:0/3099717538 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc5a403ca20 con 0x561db2a58e40 2024-05-27T08:05:31.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.028+0000 7fc5c7495b80 10 client.18525 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:31.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.028+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3099717538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x561db2c5c490 con 0x561db2a58e40 2024-05-27T08:05:31.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.028+0000 7fc597e00700 1 -- 172.21.15.90:0/3099717538 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc5a4030070 con 0x561db2a58e40 2024-05-27T08:05:31.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.028+0000 7fc597e00700 1 -- 172.21.15.90:0/3099717538 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fc5a403a020 con 0x561db2a58e40 2024-05-27T08:05:31.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.029+0000 7fc5c7495b80 10 client.18525 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:31.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.029+0000 7fc5c7495b80 10 client.18525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:31.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.029+0000 7fc5c7495b80 20 client.18525 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:31.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.029+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561db2c5cb10 0x561db2c5ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:31.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.029+0000 7fc5c7495b80 4 client.18525 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-27T08:05:31.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.029+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3099717538 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561db2b01570 con 0x561db2c5cb10 2024-05-27T08:05:31.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.029+0000 7fc5abe00700 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561db2c5cb10 0x561db2c5ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:31.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.030+0000 7fc5abe00700 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561db2c5cb10 0x561db2c5ef30 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f131b62e2a3272 server_cookie=2195075a77d7d3c2 in_seq=0 out_seq=0 2024-05-27T08:05:31.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.030+0000 7fc597e00700 10 client.18525 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:31.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.036+0000 7fc597e00700 1 -- 172.21.15.90:0/3099717538 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561db2b01570 con 0x561db2c5cb10 2024-05-27T08:05:31.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.036+0000 7fc597e00700 10 client.18525 handle_command_reply: tid=0 2024-05-27T08:05:31.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.036+0000 7fc5c7495b80 1 client.18525 shutdown 2024-05-27T08:05:31.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.036+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3099717538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 msgr2=0x561db2bea1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:31.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bea1e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009670 tx=0x7fc5a40247e0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3099717538 shutdown_connections 2024-05-27T08:05:31.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x561db2c5cb10 0x561db2c5ef30 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561db2bddd30 0x561db2bf94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561db2a58e40 0x561db2bea1e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc5c7495b80 1 --2- 172.21.15.90:0/3099717538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561db2bd7db0 0x561db2bf8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc597e00700 0 client.18525 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:31.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3099717538 >> 172.21.15.90:0/3099717538 conn(0x561db2ab8fc0 msgr2=0x561db2be1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:31.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3099717538 shutdown_connections 2024-05-27T08:05:31.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc5c7495b80 1 -- 172.21.15.90:0/3099717538 wait complete. 2024-05-27T08:05:31.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.037+0000 7fc5c7495b80 20 client.18525 trim_cache size 0 max 16384 2024-05-27T08:05:31.041 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:05:31.041 INFO:teuthology.orchestra.run.smithi090.stdout: "status": "no active scrubs running", 2024-05-27T08:05:31.041 INFO:teuthology.orchestra.run.smithi090.stdout: "scrubs": {} 2024-05-27T08:05:31.041 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:05:31.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 -- 172.21.15.90:0/1742728881 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc5980b29f0 msgr2=0x7fc5980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:31.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc5980b29f0 0x7fc5980b4f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4103af0 tx=0x7fc5a4009040 comp rx=0 tx=0).stop 2024-05-27T08:05:31.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 -- 172.21.15.90:0/1742728881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b41089e0 msgr2=0x7fc5b41a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:31.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b41089e0 0x7fc5b41a1810 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00d980 tx=0x7fc5ac00dd40 comp rx=0 tx=0).stop 2024-05-27T08:05:31.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 -- 172.21.15.90:0/1742728881 shutdown_connections 2024-05-27T08:05:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b41089e0 0x7fc5b41a1810 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5b41029b0 0x7fc5b419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc5980b29f0 0x7fc5980b4f00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 --2- 172.21.15.90:0/1742728881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5b4103300 0x7fc5b419d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 -- 172.21.15.90:0/1742728881 >> 172.21.15.90:0/1742728881 conn(0x7fc5b40fe6e0 msgr2=0x7fc5b4107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 -- 172.21.15.90:0/1742728881 shutdown_connections 2024-05-27T08:05:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:31.038+0000 7fc5b8a00700 1 -- 172.21.15.90:0/1742728881 wait complete. 2024-05-27T08:05:32.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:31 smithi136 ceph-mon[35324]: pgmap v616: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.1 MiB/s wr, 932 op/s 2024-05-27T08:05:32.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:31 smithi129 ceph-mon[35254]: pgmap v616: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.1 MiB/s wr, 932 op/s 2024-05-27T08:05:32.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:31 smithi090 ceph-mon[31429]: pgmap v616: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 8.1 MiB/s wr, 932 op/s 2024-05-27T08:05:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.278+0000 7fde5d800640 1 -- 172.21.15.90:0/2397481982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde58112e30 msgr2=0x7fde58113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.278+0000 7fde5d800640 1 --2- 172.21.15.90:0/2397481982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde58112e30 0x7fde58113260 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fde44009900 tx=0x7fde4402a410 comp rx=0 tx=0).stop 2024-05-27T08:05:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.278+0000 7fde5d800640 1 -- 172.21.15.90:0/2397481982 shutdown_connections 2024-05-27T08:05:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.278+0000 7fde5d800640 1 --2- 172.21.15.90:0/2397481982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde581138f0 0x7fde5811b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.278+0000 7fde5d800640 1 --2- 172.21.15.90:0/2397481982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde58112e30 0x7fde58113260 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.278+0000 7fde5d800640 1 --2- 172.21.15.90:0/2397481982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde5810e8a0 0x7fde5810ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.278+0000 7fde5d800640 1 -- 172.21.15.90:0/2397481982 >> 172.21.15.90:0/2397481982 conn(0x7fde58109ea0 msgr2=0x7fde5810c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.278+0000 7fde5d800640 1 -- 172.21.15.90:0/2397481982 shutdown_connections 2024-05-27T08:05:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.278+0000 7fde5d800640 1 -- 172.21.15.90:0/2397481982 wait complete. 2024-05-27T08:05:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.279+0000 7fde5d800640 1 Processor -- start 2024-05-27T08:05:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.279+0000 7fde5d800640 1 -- start start 2024-05-27T08:05:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.279+0000 7fde5d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde5810e8a0 0x7fde581a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.279+0000 7fde5d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde58112e30 0x7fde581a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.279+0000 7fde5d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde581138f0 0x7fde581a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.279+0000 7fde5d800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde5811e680 con 0x7fde5810e8a0 2024-05-27T08:05:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde5d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fde5811e500 con 0x7fde58112e30 2024-05-27T08:05:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde5d800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fde5811e800 con 0x7fde581138f0 2024-05-27T08:05:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde56a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde5810e8a0 0x7fde581a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde56a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde5810e8a0 0x7fde581a5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51384/0 (socket says 172.21.15.90:51384) 2024-05-27T08:05:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde56a00640 1 -- 172.21.15.90:0/848305922 learned_addr learned my addr 172.21.15.90:0/848305922 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde56000640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde58112e30 0x7fde581a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde57400640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde581138f0 0x7fde581a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde56a00640 1 -- 172.21.15.90:0/848305922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde581138f0 msgr2=0x7fde581a9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde56a00640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde581138f0 0x7fde581a9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde56a00640 1 -- 172.21.15.90:0/848305922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde58112e30 msgr2=0x7fde581a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde56a00640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde58112e30 0x7fde581a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde56a00640 1 -- 172.21.15.90:0/848305922 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde581a9fe0 con 0x7fde5810e8a0 2024-05-27T08:05:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde57400640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde581138f0 0x7fde581a9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.280+0000 7fde56000640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde58112e30 0x7fde581a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.281+0000 7fde56a00640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde5810e8a0 0x7fde581a5200 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c8c0 tx=0x7fde4800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.281+0000 7fde4f400640 1 -- 172.21.15.90:0/848305922 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48004220 con 0x7fde5810e8a0 2024-05-27T08:05:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.281+0000 7fde4f400640 1 -- 172.21.15.90:0/848305922 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde48007e40 con 0x7fde5810e8a0 2024-05-27T08:05:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.281+0000 7fde5d800640 1 -- 172.21.15.90:0/848305922 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde581aa2d0 con 0x7fde5810e8a0 2024-05-27T08:05:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.281+0000 7fde4f400640 1 -- 172.21.15.90:0/848305922 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48004a00 con 0x7fde5810e8a0 2024-05-27T08:05:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.281+0000 7fde5d800640 1 -- 172.21.15.90:0/848305922 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde581aa820 con 0x7fde5810e8a0 2024-05-27T08:05:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.285+0000 7fde64769740 1 Processor -- start 2024-05-27T08:05:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.285+0000 7fde64769740 1 -- start start 2024-05-27T08:05:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde64769740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededdd1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde64769740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededdd1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde64769740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededdd22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde64769740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ededd42b90 con 0x55ededa2f1b0 2024-05-27T08:05:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde64769740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ededd32730 con 0x55ededa062f0 2024-05-27T08:05:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde64769740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ededd36770 con 0x55ededa0e4d0 2024-05-27T08:05:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededdd1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde4f400640 1 -- 172.21.15.90:0/848305922 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fde480079b0 con 0x7fde5810e8a0 2024-05-27T08:05:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededdd1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51392/0 (socket says 172.21.15.90:51392) 2024-05-27T08:05:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde57400640 1 -- 172.21.15.90:0/2890898055 learned_addr learned my addr 172.21.15.90:0/2890898055 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde56000640 1 --2- 172.21.15.90:0/2890898055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededdd22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.286+0000 7fde57400640 1 -- 172.21.15.90:0/2890898055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 msgr2=0x55ededdd22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.287+0000 7fde57400640 1 --2- 172.21.15.90:0/2890898055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededdd22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.287+0000 7fde57400640 1 -- 172.21.15.90:0/2890898055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 msgr2=0x55ededdd1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.287+0000 7fde57400640 1 --2- 172.21.15.90:0/2890898055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededdd1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.287+0000 7fde57400640 1 -- 172.21.15.90:0/2890898055 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ededccb8b0 con 0x55ededa2f1b0 2024-05-27T08:05:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.287+0000 7fde56000640 1 --2- 172.21.15.90:0/2890898055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededdd22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.287+0000 7fde57400640 1 --2- 172.21.15.90:0/2890898055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededdd1860 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fde50008de0 tx=0x7fde5000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324ccff858a0fe3a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.287+0000 7fde43e00640 1 -- 172.21.15.90:0/2890898055 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5000f830 con 0x55ededa2f1b0 2024-05-27T08:05:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.287+0000 7fde43e00640 1 -- 172.21.15.90:0/2890898055 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde50018480 con 0x55ededa2f1b0 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.287+0000 7fde43e00640 1 -- 172.21.15.90:0/2890898055 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5000fe60 con 0x55ededa2f1b0 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.288+0000 7fde4f400640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fde3c0b2950 0x7fde3c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.288+0000 7fde64769740 1 -- 172.21.15.90:0/2890898055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 msgr2=0x55ededdd1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.288+0000 7fde64769740 1 --2- 172.21.15.90:0/2890898055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededdd1860 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fde50008de0 tx=0x7fde5000d710 comp rx=0 tx=0).stop 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.288+0000 7fde4f400640 1 -- 172.21.15.90:0/848305922 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fde480e27e0 con 0x7fde5810e8a0 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.288+0000 7fde56a00640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fde3c0b2950 0x7fde3c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.289+0000 7fde64769740 1 -- 172.21.15.90:0/2890898055 shutdown_connections 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.289+0000 7fde64769740 1 --2- 172.21.15.90:0/2890898055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededdd1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.289+0000 7fde64769740 1 --2- 172.21.15.90:0/2890898055 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededdd22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.289+0000 7fde64769740 1 --2- 172.21.15.90:0/2890898055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededdd1860 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.289+0000 7fde64769740 1 -- 172.21.15.90:0/2890898055 >> 172.21.15.90:0/2890898055 conn(0x55ededa23300 msgr2=0x55ededdce010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.289+0000 7fde64769740 1 -- 172.21.15.90:0/2890898055 shutdown_connections 2024-05-27T08:05:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.289+0000 7fde64769740 1 -- 172.21.15.90:0/2890898055 wait complete. 2024-05-27T08:05:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.289+0000 7fde56a00640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fde3c0b2950 0x7fde3c0b4e60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55ededdcf350 tx=0x7fde480a56c0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.289+0000 7fde64769740 1 -- start start 2024-05-27T08:05:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.290+0000 7fde64769740 1 Processor -- start 2024-05-27T08:05:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.290+0000 7fde64769740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa062f0 0x55ededddb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.290+0000 7fde64769740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa0e4d0 0x55ededddbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.290+0000 7fde64769740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa2f1b0 0x55ededde7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.290+0000 7fde64769740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ededa18790 con 0x55ededa062f0 2024-05-27T08:05:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.290+0000 7fde64769740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ededcdabc0 con 0x55ededa0e4d0 2024-05-27T08:05:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.290+0000 7fde64769740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ededdde990 con 0x55ededa2f1b0 2024-05-27T08:05:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.290+0000 7fde64769740 10 client.0 _ll_register_callbacks cb 0x55ededcb8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.291+0000 7fde57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa062f0 0x55ededddb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.291+0000 7fde57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa062f0 0x55ededddb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51408/0 (socket says 172.21.15.90:51408) 2024-05-27T08:05:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.291+0000 7fde57400640 1 -- 172.21.15.90:0/2703356787 learned_addr learned my addr 172.21.15.90:0/2703356787 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.291+0000 7fde56000640 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa0e4d0 0x55ededddbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.291+0000 7fde57400640 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa2f1b0 0x55ededde7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.291+0000 7fde57400640 1 -- 172.21.15.90:0/2703356787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa2f1b0 msgr2=0x55ededde7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.291+0000 7fde57400640 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa2f1b0 0x55ededde7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.291+0000 7fde57400640 1 -- 172.21.15.90:0/2703356787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa0e4d0 msgr2=0x55ededddbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.291+0000 7fde57400640 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa0e4d0 0x55ededddbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.291+0000 7fde57400640 1 -- 172.21.15.90:0/2703356787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ededa47210 con 0x55ededa062f0 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.292+0000 7fde57400640 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa062f0 0x55ededddb9e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fde50018d00 tx=0x7fde50018e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b4517b73d48be0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.292+0000 7fde40c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.292+0000 7fde40c00640 1 -- 172.21.15.90:0/2703356787 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5000fa40 con 0x55ededa062f0 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.292+0000 7fde40c00640 1 -- 172.21.15.90:0/2703356787 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fde5000fbe0 con 0x55ededa062f0 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.292+0000 7fde64769740 1 -- 172.21.15.90:0/2703356787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde50018480 con 0x55ededa062f0 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.292+0000 7fde40c00640 1 -- 172.21.15.90:0/2703356787 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fde50005040 con 0x55ededa062f0 2024-05-27T08:05:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.292+0000 7fde64769740 10 client.18537 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.292+0000 7fde64769740 1 -- 172.21.15.90:0/2703356787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55ededb13620 con 0x55ededa062f0 2024-05-27T08:05:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.293+0000 7fde40c00640 1 -- 172.21.15.90:0/2703356787 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde50018480 con 0x55ededa062f0 2024-05-27T08:05:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.293+0000 7fde40c00640 1 -- 172.21.15.90:0/2703356787 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fde50026020 con 0x55ededa062f0 2024-05-27T08:05:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.293+0000 7fde64769740 10 client.18537 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.293+0000 7fde64769740 10 client.18537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.293+0000 7fde64769740 20 client.18537 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.293+0000 7fde64769740 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eded9edd80 0x55ededdedc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.293+0000 7fde64769740 4 client.18537 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.293+0000 7fde64769740 1 -- 172.21.15.90:0/2703356787 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ededd44ae0 con 0x55eded9edd80 2024-05-27T08:05:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.294+0000 7fde56000640 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eded9edd80 0x55ededdedc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.295+0000 7fde40c00640 10 client.18537 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.295+0000 7fde56000640 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eded9edd80 0x55ededdedc60 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b724524c069f7704 server_cookie=bc3f004aa75f44f7 in_seq=0 out_seq=0 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde40c00640 1 -- 172.21.15.90:0/2703356787 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55ededd44ae0 con 0x55eded9edd80 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde40c00640 10 client.18537 handle_command_reply: tid=0 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde64769740 1 client.18537 shutdown 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde64769740 1 -- 172.21.15.90:0/2703356787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa062f0 msgr2=0x55ededddb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde64769740 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa062f0 0x55ededddb9e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fde50018d00 tx=0x7fde50018e00 comp rx=0 tx=0).stop 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde64769740 1 -- 172.21.15.90:0/2703356787 shutdown_connections 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde64769740 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eded9edd80 0x55ededdedc60 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde64769740 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa0e4d0 0x55ededddbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde64769740 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa2f1b0 0x55ededde7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde64769740 1 --2- 172.21.15.90:0/2703356787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa062f0 0x55ededddb9e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde40c00640 0 client.18537 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde64769740 1 -- 172.21.15.90:0/2703356787 >> 172.21.15.90:0/2703356787 conn(0x55ededa23300 msgr2=0x55ededdd1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.297+0000 7fde64769740 1 -- 172.21.15.90:0/2703356787 shutdown_connections 2024-05-27T08:05:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.298+0000 7fde64769740 1 -- 172.21.15.90:0/2703356787 wait complete. 2024-05-27T08:05:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.298+0000 7fde64769740 20 client.18537 trim_cache size 0 max 16384 2024-05-27T08:05:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.316+0000 7fde64769740 1 Processor -- start 2024-05-27T08:05:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.316+0000 7fde64769740 1 -- start start 2024-05-27T08:05:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.317+0000 7fde64769740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededddd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.317+0000 7fde64769740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededde7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.317+0000 7fde64769740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.317+0000 7fde64769740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde581bd730 con 0x55ededa2f1b0 2024-05-27T08:05:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.317+0000 7fde64769740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fde581ba780 con 0x55ededa062f0 2024-05-27T08:05:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.317+0000 7fde64769740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fde581bc990 con 0x55ededa0e4d0 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.317+0000 7fde57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.317+0000 7fde57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51410/0 (socket says 172.21.15.90:51410) 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.317+0000 7fde57400640 1 -- 172.21.15.90:0/2488685283 learned_addr learned my addr 172.21.15.90:0/2488685283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.317+0000 7fde57400640 1 --2- 172.21.15.90:0/2488685283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededddd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.318+0000 7fde56000640 1 --2- 172.21.15.90:0/2488685283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededde7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.318+0000 7fde57400640 1 -- 172.21.15.90:0/2488685283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 msgr2=0x55ededde7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.318+0000 7fde57400640 1 --2- 172.21.15.90:0/2488685283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededde7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.318+0000 7fde57400640 1 -- 172.21.15.90:0/2488685283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 msgr2=0x55ededddd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.318+0000 7fde57400640 1 --2- 172.21.15.90:0/2488685283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededddd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.318+0000 7fde57400640 1 -- 172.21.15.90:0/2488685283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ededddc280 con 0x55ededa2f1b0 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.318+0000 7fde57400640 1 --2- 172.21.15.90:0/2488685283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde7f70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fde50000f80 tx=0x7fde5001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65eca24a0d74782f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.318+0000 7fde43400640 1 -- 172.21.15.90:0/2488685283 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50018480 con 0x55ededa2f1b0 2024-05-27T08:05:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.318+0000 7fde43400640 1 -- 172.21.15.90:0/2488685283 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde5001d030 con 0x55ededa2f1b0 2024-05-27T08:05:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.318+0000 7fde43400640 1 -- 172.21.15.90:0/2488685283 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5000b930 con 0x55ededa2f1b0 2024-05-27T08:05:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.319+0000 7fde64769740 1 -- 172.21.15.90:0/2488685283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 msgr2=0x55ededde7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.319+0000 7fde64769740 1 --2- 172.21.15.90:0/2488685283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde7f70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fde50000f80 tx=0x7fde5001b040 comp rx=0 tx=0).stop 2024-05-27T08:05:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.319+0000 7fde64769740 1 -- 172.21.15.90:0/2488685283 shutdown_connections 2024-05-27T08:05:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.319+0000 7fde64769740 1 --2- 172.21.15.90:0/2488685283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde7f70 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.319+0000 7fde64769740 1 --2- 172.21.15.90:0/2488685283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededde7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.319+0000 7fde64769740 1 --2- 172.21.15.90:0/2488685283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededddd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.319+0000 7fde64769740 1 -- 172.21.15.90:0/2488685283 >> 172.21.15.90:0/2488685283 conn(0x55ededa23300 msgr2=0x55ededddd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.319+0000 7fde64769740 1 -- 172.21.15.90:0/2488685283 shutdown_connections 2024-05-27T08:05:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.319+0000 7fde64769740 1 -- 172.21.15.90:0/2488685283 wait complete. 2024-05-27T08:05:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.320+0000 7fde64769740 1 -- start start 2024-05-27T08:05:32.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.320+0000 7fde64769740 1 Processor -- start 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde64769740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededde0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde64769740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededde1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde64769740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde64769740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde581bc2c0 con 0x55ededa2f1b0 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde64769740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fde581bae50 con 0x55ededa062f0 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde64769740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fde581bb520 con 0x55ededa0e4d0 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde64769740 10 client.0 _ll_register_callbacks cb 0x7fde50008c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde57400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51420/0 (socket says 172.21.15.90:51420) 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde57400640 1 -- 172.21.15.90:0/1126952053 learned_addr learned my addr 172.21.15.90:0/1126952053 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde57400640 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededde0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde56000640 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededde1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde57400640 1 -- 172.21.15.90:0/1126952053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 msgr2=0x55ededde1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde57400640 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededde1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.321+0000 7fde57400640 1 -- 172.21.15.90:0/1126952053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 msgr2=0x55ededde0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.322+0000 7fde57400640 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededde0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.322+0000 7fde57400640 1 -- 172.21.15.90:0/1126952053 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edede54400 con 0x55ededa2f1b0 2024-05-27T08:05:32.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.322+0000 7fde57400640 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde1560 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fde50008d00 tx=0x7fde50008b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25302991e5322200 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.322+0000 7fde40c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.322+0000 7fde40c00640 1 -- 172.21.15.90:0/1126952053 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50008280 con 0x55ededa2f1b0 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.322+0000 7fde40c00640 1 -- 172.21.15.90:0/1126952053 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fde5000b3a0 con 0x55ededa2f1b0 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.322+0000 7fde64769740 1 -- 172.21.15.90:0/1126952053 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde5001d030 con 0x55ededa2f1b0 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.322+0000 7fde40c00640 1 -- 172.21.15.90:0/1126952053 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fde50008730 con 0x55ededa2f1b0 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.323+0000 7fde64769740 10 client.18543 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.323+0000 7fde64769740 1 -- 172.21.15.90:0/1126952053 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55edede55670 con 0x55ededa2f1b0 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.323+0000 7fde40c00640 1 -- 172.21.15.90:0/1126952053 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde5001d030 con 0x55ededa2f1b0 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.323+0000 7fde40c00640 1 -- 172.21.15.90:0/1126952053 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fde50018ce0 con 0x55ededa2f1b0 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.323+0000 7fde64769740 10 client.18543 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.323+0000 7fde64769740 10 client.18543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.323+0000 7fde64769740 20 client.18543 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.323+0000 7fde64769740 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eded9c31c0 0x55edede57d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.323+0000 7fde64769740 4 client.18543 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-27T08:05:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.323+0000 7fde64769740 1 -- 172.21.15.90:0/1126952053 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ededdd2130 con 0x55eded9c31c0 2024-05-27T08:05:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.324+0000 7fde56000640 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eded9c31c0 0x55edede57d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.325+0000 7fde56000640 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eded9c31c0 0x55edede57d30 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a990c9b21b3f3b server_cookie=c52b778e3cc4245 in_seq=0 out_seq=0 2024-05-27T08:05:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.325+0000 7fde40c00640 10 client.18543 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.409+0000 7fde40c00640 1 -- 172.21.15.90:0/1126952053 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ededdd2130 con 0x55eded9c31c0 2024-05-27T08:05:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.409+0000 7fde40c00640 10 client.18543 handle_command_reply: tid=0 2024-05-27T08:05:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.409+0000 7fde64769740 1 client.18543 shutdown 2024-05-27T08:05:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.409+0000 7fde64769740 1 -- 172.21.15.90:0/1126952053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 msgr2=0x55ededde1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.410+0000 7fde64769740 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde1560 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fde50008d00 tx=0x7fde50008b30 comp rx=0 tx=0).stop 2024-05-27T08:05:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.410+0000 7fde64769740 1 -- 172.21.15.90:0/1126952053 shutdown_connections 2024-05-27T08:05:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.410+0000 7fde64769740 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x55eded9c31c0 0x55edede57d30 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.410+0000 7fde64769740 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ededa2f1b0 0x55ededde1560 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.410+0000 7fde64769740 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ededa0e4d0 0x55ededde1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.410+0000 7fde64769740 1 --2- 172.21.15.90:0/1126952053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ededa062f0 0x55ededde0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.410+0000 7fde40c00640 0 client.18543 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.410+0000 7fde64769740 1 -- 172.21.15.90:0/1126952053 >> 172.21.15.90:0/1126952053 conn(0x55ededa23300 msgr2=0x55ededddb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.410+0000 7fde64769740 1 -- 172.21.15.90:0/1126952053 shutdown_connections 2024-05-27T08:05:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.411+0000 7fde64769740 1 -- 172.21.15.90:0/1126952053 wait complete. 2024-05-27T08:05:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.411+0000 7fde64769740 20 client.18543 trim_cache size 0 max 16384 2024-05-27T08:05:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-05-27T08:05:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-05-27T08:05:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-05-27T08:05:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 -- 172.21.15.90:0/848305922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fde3c0b2950 msgr2=0x7fde3c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fde3c0b2950 0x7fde3c0b4e60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55ededdcf350 tx=0x7fde480a56c0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 -- 172.21.15.90:0/848305922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde5810e8a0 msgr2=0x7fde581a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde5810e8a0 0x7fde581a5200 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c8c0 tx=0x7fde4800cd80 comp rx=0 tx=0).stop 2024-05-27T08:05:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 -- 172.21.15.90:0/848305922 shutdown_connections 2024-05-27T08:05:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fde3c0b2950 0x7fde3c0b4e60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fde58112e30 0x7fde581a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde581138f0 0x7fde581a9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 --2- 172.21.15.90:0/848305922 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde5810e8a0 0x7fde581a5200 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 -- 172.21.15.90:0/848305922 >> 172.21.15.90:0/848305922 conn(0x7fde58109ea0 msgr2=0x7fde5811a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 -- 172.21.15.90:0/848305922 shutdown_connections 2024-05-27T08:05:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-05-27T08:05:32.412+0000 7fde5d800640 1 -- 172.21.15.90:0/848305922 wait complete. 2024-05-27T08:05:32.927 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-27T08:05:32.928 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:05:33.391 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:33.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:33 smithi090 ceph-mon[31429]: pgmap v617: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.9 MiB/s wr, 815 op/s 2024-05-27T08:05:34.001 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:33 smithi136 ceph-mon[35324]: pgmap v617: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.9 MiB/s wr, 815 op/s 2024-05-27T08:05:34.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:33 smithi129 ceph-mon[35254]: pgmap v617: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.9 MiB/s wr, 815 op/s 2024-05-27T08:05:34.291 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-05-27T08:05:35.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.892+0000 7ff4a2200700 1 -- 172.21.15.90:0/279768428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff49c10b890 msgr2=0x7ff49c1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:35.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.892+0000 7ff4a2200700 1 --2- 172.21.15.90:0/279768428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff49c10b890 0x7ff49c1131f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7ff49400b0d0 tx=0x7ff49400b3e0 comp rx=0 tx=0).stop 2024-05-27T08:05:35.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.892+0000 7ff4a2200700 1 -- 172.21.15.90:0/279768428 shutdown_connections 2024-05-27T08:05:35.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.892+0000 7ff4a2200700 1 --2- 172.21.15.90:0/279768428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff49c10b890 0x7ff49c1131f0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:35.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.892+0000 7ff4a2200700 1 --2- 172.21.15.90:0/279768428 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff49c106610 0x7ff49c106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:35.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.892+0000 7ff4a2200700 1 --2- 172.21.15.90:0/279768428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c10af40 0x7ff49c10b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:35.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.892+0000 7ff4a2200700 1 -- 172.21.15.90:0/279768428 >> 172.21.15.90:0/279768428 conn(0x7ff49c102020 msgr2=0x7ff49c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:35.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.892+0000 7ff4a2200700 1 -- 172.21.15.90:0/279768428 shutdown_connections 2024-05-27T08:05:35.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.892+0000 7ff4a2200700 1 -- 172.21.15.90:0/279768428 wait complete. 2024-05-27T08:05:35.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.893+0000 7ff4a2200700 1 Processor -- start 2024-05-27T08:05:35.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.894+0000 7ff4a2200700 1 -- start start 2024-05-27T08:05:35.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.894+0000 7ff4a2200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c106610 0x7ff49c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:35.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.894+0000 7ff4a2200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff49c10af40 0x7ff49c19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:35.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.894+0000 7ff4a2200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff49c10b890 0x7ff49c1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:35.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.894+0000 7ff4a2200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c1163c0 con 0x7ff49c10b890 2024-05-27T08:05:35.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.894+0000 7ff4a2200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c116240 con 0x7ff49c10af40 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.894+0000 7ff4a2200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c116540 con 0x7ff49c106610 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.894+0000 7ff4a1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff49c10b890 0x7ff49c1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff4a1800700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff49c10b890 0x7ff49c1a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51432/0 (socket says 172.21.15.90:51432) 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff4a1800700 1 -- 172.21.15.90:0/860950558 learned_addr learned my addr 172.21.15.90:0/860950558 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff4a0e00700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c106610 0x7ff49c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff49be00700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff49c10af40 0x7ff49c19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff4a1800700 1 -- 172.21.15.90:0/860950558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c106610 msgr2=0x7ff49c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff4a1800700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c106610 0x7ff49c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff4a1800700 1 -- 172.21.15.90:0/860950558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff49c10af40 msgr2=0x7ff49c19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff4a1800700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff49c10af40 0x7ff49c19d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff4a1800700 1 -- 172.21.15.90:0/860950558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c1a1c00 con 0x7ff49c10b890 2024-05-27T08:05:35.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff49be00700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff49c10af40 0x7ff49c19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:35.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.895+0000 7ff4a1800700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff49c10b890 0x7ff49c1a1560 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff494007d10 tx=0x7ff494007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:35.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.896+0000 7ff499600700 1 -- 172.21.15.90:0/860950558 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49401f070 con 0x7ff49c10b890 2024-05-27T08:05:35.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.896+0000 7ff499600700 1 -- 172.21.15.90:0/860950558 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff494003e10 con 0x7ff49c10b890 2024-05-27T08:05:35.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.896+0000 7ff4a2200700 1 -- 172.21.15.90:0/860950558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c1a1e90 con 0x7ff49c10b890 2024-05-27T08:05:35.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.896+0000 7ff4a2200700 1 -- 172.21.15.90:0/860950558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c1a9810 con 0x7ff49c10b890 2024-05-27T08:05:35.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.896+0000 7ff499600700 1 -- 172.21.15.90:0/860950558 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49401b740 con 0x7ff49c10b890 2024-05-27T08:05:35.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.897+0000 7ff4a2200700 1 -- 172.21.15.90:0/860950558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c0512b0 con 0x7ff49c10b890 2024-05-27T08:05:35.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.901+0000 7ff499600700 1 -- 172.21.15.90:0/860950558 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff494022070 con 0x7ff49c10b890 2024-05-27T08:05:35.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.903+0000 7ff499600700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4840b2a70 0x7ff4840b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:35.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.903+0000 7ff4a0e00700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4840b2a70 0x7ff4840b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:35.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.904+0000 7ff499600700 1 -- 172.21.15.90:0/860950558 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7ff49400f020 con 0x7ff49c10b890 2024-05-27T08:05:35.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.904+0000 7ff4a0e00700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4840b2a70 0x7ff4840b4f80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff490005950 tx=0x7ff49000b410 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:35.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:35.907+0000 7ff499600700 1 -- 172.21.15.90:0/860950558 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff494029020 con 0x7ff49c10b890 2024-05-27T08:05:35.951 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/: 2024-05-27T08:05:35.952 INFO:teuthology.orchestra.run.smithi090.stdout:total 2 2024-05-27T08:05:35.952 INFO:teuthology.orchestra.run.smithi090.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708303 May 27 07:46 . 2024-05-27T08:05:35.952 INFO:teuthology.orchestra.run.smithi090.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1439708303 May 27 07:46 .. 2024-05-27T08:05:35.952 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708196 May 27 08:05 data 2024-05-27T08:05:35.952 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:35.952 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data: 2024-05-27T08:05:35.952 INFO:teuthology.orchestra.run.smithi090.stdout:total 46 2024-05-27T08:05:35.952 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708196 May 27 08:05 . 2024-05-27T08:05:35.952 INFO:teuthology.orchestra.run.smithi090.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708303 May 27 07:46 .. 2024-05-27T08:05:35.952 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 566482485 May 27 07:49 base 2024-05-27T08:05:35.953 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 May 27 07:47 current_logfiles 2024-05-27T08:05:35.953 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 27 07:47 global 2024-05-27T08:05:35.953 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1047 May 27 07:47 log 2024-05-27T08:05:35.953 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_commit_ts 2024-05-27T08:05:35.953 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_dynshmem 2024-05-27T08:05:35.953 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 May 27 07:46 pg_hba.conf 2024-05-27T08:05:35.953 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 May 27 07:46 pg_ident.conf 2024-05-27T08:05:35.953 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 27 08:05 pg_logical 2024-05-27T08:05:35.953 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:46 pg_multixact 2024-05-27T08:05:35.954 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_notify 2024-05-27T08:05:35.954 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_replslot 2024-05-27T08:05:35.954 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_serial 2024-05-27T08:05:35.954 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_snapshots 2024-05-27T08:05:35.954 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 27 08:05 pg_stat 2024-05-27T08:05:35.954 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 08:05 pg_stat_tmp 2024-05-27T08:05:35.954 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 08:05 pg_subtrans 2024-05-27T08:05:35.954 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_tblspc 2024-05-27T08:05:35.954 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 pg_twophase 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 27 07:46 PG_VERSION 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 872415232 May 27 08:05 pg_wal 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 27 07:46 pg_xact 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 May 27 07:46 postgresql.auto.conf 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 May 27 07:46 postgresql.conf 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 May 27 07:47 postmaster.opts 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/base: 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout:total 3 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 566482485 May 27 07:49 . 2024-05-27T08:05:35.955 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708196 May 27 08:05 .. 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 27 07:47 1 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 27 07:47 13493 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 550556207 May 27 07:54 13494 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:50 pgsql_tmp 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/base/1: 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout:total 7778 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 27 07:47 . 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 566482485 May 27 07:49 .. 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 112 2024-05-27T08:05:35.956 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 113 2024-05-27T08:05:35.957 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:47 1247 2024-05-27T08:05:35.957 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1247_fsm 2024-05-27T08:05:35.957 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1247_vm 2024-05-27T08:05:35.957 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 27 07:47 1249 2024-05-27T08:05:35.957 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1249_fsm 2024-05-27T08:05:35.957 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1249_vm 2024-05-27T08:05:35.958 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 27 07:47 1255 2024-05-27T08:05:35.958 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1255_fsm 2024-05-27T08:05:35.958 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1255_vm 2024-05-27T08:05:35.958 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 27 07:47 1259 2024-05-27T08:05:35.958 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1259_fsm 2024-05-27T08:05:35.958 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1259_vm 2024-05-27T08:05:35.958 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 27 07:47 13344 2024-05-27T08:05:35.958 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13344_fsm 2024-05-27T08:05:35.958 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13344_vm 2024-05-27T08:05:35.959 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13346 2024-05-27T08:05:35.959 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13348 2024-05-27T08:05:35.959 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349 2024-05-27T08:05:35.959 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13349_fsm 2024-05-27T08:05:35.959 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349_vm 2024-05-27T08:05:35.959 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13351 2024-05-27T08:05:35.960 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13353 2024-05-27T08:05:35.960 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354 2024-05-27T08:05:35.960 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13354_fsm 2024-05-27T08:05:35.960 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354_vm 2024-05-27T08:05:35.960 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13356 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13358 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13359_fsm 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359_vm 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13361 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13363 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 1417 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 1418 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 174 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 175 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2187 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2224 2024-05-27T08:05:35.961 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2328 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2336 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2337 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2579 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2600 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2600_fsm 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2600_vm 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2601_fsm 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601_vm 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2602 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2602_fsm 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2602_vm 2024-05-27T08:05:35.962 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2603 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2603_fsm 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2603_vm 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2604 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2605 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2605_fsm 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2605_vm 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2606 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2606_fsm 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2606_vm 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2607 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2607_fsm 2024-05-27T08:05:35.963 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2607_vm 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 27 07:47 2608 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2608_fsm 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2608_vm 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 27 07:47 2609 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2609_fsm 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2609_vm 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2610 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2610_fsm 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2610_vm 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2611 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612 2024-05-27T08:05:35.964 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2612_fsm 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612_vm 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2613 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2615_fsm 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615_vm 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2616 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2616_fsm 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2616_vm 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 27 07:47 2617 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2617_fsm 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2617_vm 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 27 07:47 2618 2024-05-27T08:05:35.965 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2618_fsm 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2618_vm 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 27 07:47 2619 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2619_fsm 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2619_vm 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2620 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2650 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2651 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2652 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2653 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2654 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2655 2024-05-27T08:05:35.966 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2656 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2657 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 27 07:47 2658 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:47 2659 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2660 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2661 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2662 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2663 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2664 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2665 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2666 2024-05-27T08:05:35.967 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2667 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2668 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2669 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2670 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 27 07:47 2673 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 27 07:47 2674 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 27 07:47 2675 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2678 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2679 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2680 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2681 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2682 2024-05-27T08:05:35.968 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2683 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2684 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2685 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2686 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2687 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2688 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2689 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 27 07:47 2690 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 27 07:47 2691 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2692 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2693 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2696 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2699 2024-05-27T08:05:35.969 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2701 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2702 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2703 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2704 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2753 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2753_fsm 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2753_vm 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2754 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2755 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2756 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2757 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2830 2024-05-27T08:05:35.970 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2831 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2832 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2833 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2834 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2835 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2836 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2837 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 27 07:47 2838 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2838_fsm 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2838_vm 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2839 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840_fsm 2024-05-27T08:05:35.971 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2840_vm 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2841 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2995 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2996 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3079_fsm 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079_vm 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3080 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3081 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3085 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3118 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3119 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 27 07:47 3164 2024-05-27T08:05:35.972 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3256 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3257 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3258 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3350 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3351 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3379 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3380 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3381 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3394 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3394_fsm 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3394_vm 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3395 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3429 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3430 2024-05-27T08:05:35.973 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3431 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3433 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3439 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3440 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3455 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 27 07:47 3456 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3456_fsm 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3456_vm 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3466 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3467 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3468 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3501 2024-05-27T08:05:35.974 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3502 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3503 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3534 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3541_fsm 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541_vm 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3542 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3574 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3575 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3576 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3596 2024-05-27T08:05:35.975 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3597 2024-05-27T08:05:35.976 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3598 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3599 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3600_fsm 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600_vm 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3601_fsm 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601_vm 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3602_fsm 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602_vm 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603 2024-05-27T08:05:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603_fsm 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3603_vm 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3604 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3605 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3606 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3607 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3608 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 3609 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3712 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3764_fsm 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764_vm 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3766 2024-05-27T08:05:35.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3767 2024-05-27T08:05:35.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3997 2024-05-27T08:05:35.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4143 2024-05-27T08:05:35.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4144 2024-05-27T08:05:35.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4145 2024-05-27T08:05:35.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4146 2024-05-27T08:05:35.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4147 2024-05-27T08:05:35.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4148 2024-05-27T08:05:35.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4149 2024-05-27T08:05:35.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4150 2024-05-27T08:05:35.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4151 2024-05-27T08:05:36.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:35 smithi129 ceph-mon[35254]: pgmap v618: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.3 MiB/s wr, 779 op/s 2024-05-27T08:05:36.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:35 smithi090 ceph-mon[31429]: pgmap v618: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.3 MiB/s wr, 779 op/s 2024-05-27T08:05:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.223+0000 7ff4a2200700 1 -- 172.21.15.90:0/860950558 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff49c19e090 con 0x7ff49c10b890 2024-05-27T08:05:36.317 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:35 smithi136 ceph-mon[35324]: pgmap v618: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.3 MiB/s wr, 779 op/s 2024-05-27T08:05:36.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.224+0000 7ff499600700 1 -- 172.21.15.90:0/860950558 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+9456 (secure 0 0 0) 0x7ff4940ab940 con 0x7ff49c10b890 2024-05-27T08:05:36.318 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:36.318 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":64,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T07:41:11.710115+0000","modified":"2024-05-27T07:56:32.161929+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34472,"mds_1":34442,"mds_2":34424,"mds_3":34406,"mds_4":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14556":{"gid":14556,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/767662982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":767662982},{"type":"v1","addr":"172.21.15.90:6839","nonce":767662982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14562":{"gid":14562,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/24895501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":24895501},{"type":"v1","addr":"172.21.15.90:6841","nonce":24895501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/1595668128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1595668128},{"type":"v1","addr":"172.21.15.136:6833","nonce":1595668128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/1419480441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1419480441},{"type":"v1","addr":"172.21.15.136:6835","nonce":1419480441}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":4,"incarnation":26,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6839/2113925564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2113925564},{"type":"v1","addr":"172.21.15.136:6839","nonce":2113925564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34406":{"gid":34406,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":37,"addr":"172.21.15.129:6833/3131589000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3131589000},{"type":"v1","addr":"172.21.15.129:6833","nonce":3131589000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":34,"addr":"172.21.15.129:6835/1894603892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1894603892},{"type":"v1","addr":"172.21.15.129:6835","nonce":1894603892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34442":{"gid":34442,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":30,"addr":"172.21.15.129:6837/2346696366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":2346696366},{"type":"v1","addr":"172.21.15.129:6837","nonce":2346696366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34472":{"gid":34472,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.129:6839/2893900972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2893900972},{"type":"v1","addr":"172.21.15.129:6839","nonce":2893900972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34472,"qdb_cluster":[34442,34472,34424,34406,24478]},"id":1}]} 2024-05-27T08:05:36.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.227+0000 7ff4a2200700 1 -- 172.21.15.90:0/860950558 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4840b2a70 msgr2=0x7ff4840b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:36.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.227+0000 7ff4a2200700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4840b2a70 0x7ff4840b4f80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff490005950 tx=0x7ff49000b410 comp rx=0 tx=0).stop 2024-05-27T08:05:36.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.227+0000 7ff4a2200700 1 -- 172.21.15.90:0/860950558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff49c10b890 msgr2=0x7ff49c1a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:36.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.227+0000 7ff4a2200700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff49c10b890 0x7ff49c1a1560 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff494007d10 tx=0x7ff494007d40 comp rx=0 tx=0).stop 2024-05-27T08:05:36.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.227+0000 7ff4a2200700 1 -- 172.21.15.90:0/860950558 shutdown_connections 2024-05-27T08:05:36.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.227+0000 7ff4a2200700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff49c10b890 0x7ff49c1a1560 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:36.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.227+0000 7ff4a2200700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff49c106610 0x7ff49c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:36.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.227+0000 7ff4a2200700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7ff4840b2a70 0x7ff4840b4f80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:36.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.227+0000 7ff4a2200700 1 --2- 172.21.15.90:0/860950558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff49c10af40 0x7ff49c19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:36.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.227+0000 7ff4a2200700 1 -- 172.21.15.90:0/860950558 >> 172.21.15.90:0/860950558 conn(0x7ff49c102020 msgr2=0x7ff49c10a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:36.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.228+0000 7ff4a2200700 1 -- 172.21.15.90:0/860950558 shutdown_connections 2024-05-27T08:05:36.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:36.228+0000 7ff4a2200700 1 -- 172.21.15.90:0/860950558 wait complete. 2024-05-27T08:05:36.320 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 64 2024-05-27T08:05:37.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:36 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/860950558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:05:37.097 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:36 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/860950558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:05:37.199 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-27T08:05:37.236 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4152 2024-05-27T08:05:37.236 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4153 2024-05-27T08:05:37.236 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4154 2024-05-27T08:05:37.236 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4155 2024-05-27T08:05:37.236 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4156 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4157 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4158 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4159 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4160 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4161 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4162 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4163 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4164 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4165 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4166 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4167 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4168 2024-05-27T08:05:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4169 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4170 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4171 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4172 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4173 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4174 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 5002 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 548 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 549 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 6102 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 6104 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 6106 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6110 2024-05-27T08:05:37.238 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6111 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6112 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6113 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6117 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 826 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 827 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 828 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 27 07:47 pg_filenode.map 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 27 07:47 PG_VERSION 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:total 7778 2024-05-27T08:05:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 May 27 07:47 . 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 566482485 May 27 07:49 .. 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 112 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 113 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:47 1247 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1247_fsm 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1247_vm 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 27 07:47 1249 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1249_fsm 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1249_vm 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 27 07:47 1255 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1255_fsm 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1255_vm 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 27 07:47 1259 2024-05-27T08:05:37.240 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1259_fsm 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1259_vm 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 27 07:47 13344 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13344_fsm 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13344_vm 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13346 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13348 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13349_fsm 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349_vm 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13351 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13353 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13354_fsm 2024-05-27T08:05:37.241 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354_vm 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13356 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13358 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13359_fsm 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359_vm 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13361 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13363 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 1417 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 1418 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 174 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 175 2024-05-27T08:05:37.242 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2187 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2224 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2328 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2336 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2337 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2579 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2600 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2600_fsm 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2600_vm 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2601_fsm 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601_vm 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2602 2024-05-27T08:05:37.243 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2602_fsm 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2602_vm 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2603 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2603_fsm 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2603_vm 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2604 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2605 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2605_fsm 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2605_vm 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2606 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2606_fsm 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2606_vm 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2607 2024-05-27T08:05:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2607_fsm 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2607_vm 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 27 07:47 2608 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2608_fsm 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2608_vm 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 27 07:47 2609 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2609_fsm 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2609_vm 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2610 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2610_fsm 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2610_vm 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2611 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612 2024-05-27T08:05:37.245 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2612_fsm 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612_vm 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2613 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2615_fsm 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615_vm 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2616 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2616_fsm 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2616_vm 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 27 07:47 2617 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2617_fsm 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2617_vm 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 27 07:47 2618 2024-05-27T08:05:37.246 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2618_fsm 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2618_vm 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 May 27 07:47 2619 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2619_fsm 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2619_vm 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2620 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2650 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2651 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2652 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2653 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2654 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2655 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2656 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2657 2024-05-27T08:05:37.247 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 27 07:47 2658 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:47 2659 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2660 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2661 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2662 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2663 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2664 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2665 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2666 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2667 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2668 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2669 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2670 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 27 07:47 2673 2024-05-27T08:05:37.248 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 27 07:47 2674 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 27 07:47 2675 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2678 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2679 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2680 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2681 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2682 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2683 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2684 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2685 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2686 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2687 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2688 2024-05-27T08:05:37.249 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2689 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 27 07:47 2690 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 27 07:47 2691 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2692 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2693 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2696 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2699 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2701 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2702 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2703 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2704 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2753 2024-05-27T08:05:37.250 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2753_fsm 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2753_vm 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2754 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2755 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2756 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2757 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2830 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2831 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2832 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2833 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2834 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2835 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2836 2024-05-27T08:05:37.251 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2837 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 27 07:47 2838 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2838_fsm 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2838_vm 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2839 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840_fsm 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2840_vm 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2841 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2995 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2996 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3079_fsm 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079_vm 2024-05-27T08:05:37.252 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3080 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3081 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3085 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3118 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3119 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 27 07:47 3164 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3256 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3257 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3258 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3350 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3351 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3379 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3380 2024-05-27T08:05:37.253 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3381 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3394 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3394_fsm 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3394_vm 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3395 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3429 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3430 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3431 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3433 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3439 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3440 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3455 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 27 07:47 3456 2024-05-27T08:05:37.254 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3456_fsm 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3456_vm 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3466 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3467 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3468 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3501 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3502 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3503 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3534 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3541_fsm 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541_vm 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3542 2024-05-27T08:05:37.255 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3574 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3575 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3576 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3596 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3597 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3598 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3599 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3600_fsm 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600_vm 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3601_fsm 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601_vm 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602 2024-05-27T08:05:37.256 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3602_fsm 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602_vm 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603_fsm 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3603_vm 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3604 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3605 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3606 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3607 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3608 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 3609 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3712 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3764_fsm 2024-05-27T08:05:37.257 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764_vm 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3766 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3767 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3997 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4143 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4144 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4145 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4146 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4147 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4148 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4149 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4150 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4151 2024-05-27T08:05:37.258 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4152 2024-05-27T08:05:37.259 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4153 2024-05-27T08:05:37.259 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:36 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/860950558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:05:37.661 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:37.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:37 smithi090 ceph-mon[31429]: pgmap v619: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.4 MiB/s wr, 559 op/s 2024-05-27T08:05:38.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:37 smithi129 ceph-mon[35254]: pgmap v619: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.4 MiB/s wr, 559 op/s 2024-05-27T08:05:38.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:37 smithi136 ceph-mon[35324]: pgmap v619: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.4 MiB/s wr, 559 op/s 2024-05-27T08:05:38.568 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4154 2024-05-27T08:05:38.568 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4155 2024-05-27T08:05:38.568 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4156 2024-05-27T08:05:38.568 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4157 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4158 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4159 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4160 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4161 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4162 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4163 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4164 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4165 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4166 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4167 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4168 2024-05-27T08:05:38.569 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4169 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4170 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4171 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4172 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4173 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4174 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 5002 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 548 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 549 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6102 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6104 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6106 2024-05-27T08:05:38.570 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6110 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6111 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6112 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6113 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6117 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 826 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 827 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 828 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 27 07:47 pg_filenode.map 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 27 07:47 PG_VERSION 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-05-27T08:05:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:total 537655 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 550556207 May 27 07:54 . 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 566482485 May 27 07:49 .. 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 112 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 113 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:50 1247 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1247_fsm 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:50 1247_vm 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 May 27 07:51 1249 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:50 1249_fsm 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:50 1249_vm 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 May 27 07:47 1255 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1255_fsm 2024-05-27T08:05:38.572 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1255_vm 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 May 27 08:05 1259 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1259_fsm 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:50 1259_vm 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 May 27 07:47 13344 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13344_fsm 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13344_vm 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13346 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13348 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13349_fsm 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13349_vm 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13351 2024-05-27T08:05:38.573 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13353 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13354_fsm 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13354_vm 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13356 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13358 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 13359_fsm 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13359_vm 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 13361 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 13363 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 1417 2024-05-27T08:05:38.574 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 1418 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 438927360 May 27 08:05 16396 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 May 27 08:04 16396_fsm 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 08:05 16396_vm 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2523136 May 27 08:05 16397 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 08:05 16397_fsm 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 08:05 16397_vm 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4038656 May 27 08:05 16399 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 08:05 16399_fsm 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 08:05 16399_vm 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 08:05 16400 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 27 08:04 16402 2024-05-27T08:05:38.575 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 May 27 08:05 16404 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24690688 May 27 08:05 16406 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 08:05 16406_fsm 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 08:03 16406_vm 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 174 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 175 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2187 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2224 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2328 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2336 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2337 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:50 2579 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2600 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2600_fsm 2024-05-27T08:05:38.576 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2600_vm 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2601_fsm 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2601_vm 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2602 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2602_fsm 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2602_vm 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2603 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2603_fsm 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2603_vm 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2604 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2605 2024-05-27T08:05:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2605_fsm 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2605_vm 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:50 2606 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2606_fsm 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:50 2606_vm 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2607 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2607_fsm 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2607_vm 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 May 27 07:50 2608 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2608_fsm 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:50 2608_vm 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 May 27 07:47 2609 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2609_fsm 2024-05-27T08:05:38.578 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2609_vm 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:51 2610 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2610_fsm 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:50 2610_vm 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2611 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2612_fsm 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2612_vm 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2613 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2615_fsm 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2615_vm 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2616 2024-05-27T08:05:38.579 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2616_fsm 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2616_vm 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 May 27 07:47 2617 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2617_fsm 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2617_vm 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 May 27 07:47 2618 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2618_fsm 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2618_vm 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 May 27 08:05 2619 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 08:02 2619_fsm 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:50 2619_vm 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2620 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2650 2024-05-27T08:05:38.580 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2651 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2652 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2653 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2654 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2655 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2656 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2657 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 27 07:50 2658 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 May 27 07:51 2659 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2660 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2661 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:51 2662 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:51 2663 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:50 2664 2024-05-27T08:05:38.581 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:50 2665 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:50 2666 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:50 2667 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2668 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2669 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2670 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 May 27 07:50 2673 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 May 27 07:50 2674 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 May 27 07:47 2675 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:51 2678 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:51 2679 2024-05-27T08:05:38.582 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2680 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2681 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2682 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2683 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2684 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2685 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2686 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2687 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 2688 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 07:47 2689 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 May 27 07:47 2690 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 May 27 07:47 2691 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2692 2024-05-27T08:05:38.583 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2693 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 08:04 2696 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2699 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2701 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2702 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:50 2703 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:50 2704 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2753 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2753_fsm 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2753_vm 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2754 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2755 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2756 2024-05-27T08:05:38.584 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 2757 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2830 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2831 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2832 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2833 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2834 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2835 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2836 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2837 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 May 27 07:47 2838 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2838_fsm 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2838_vm 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2839 2024-05-27T08:05:38.585 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2840_fsm 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2840_vm 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2841 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 2995 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2996 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3079_fsm 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3079_vm 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3080 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3081 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:47 3085 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3118 2024-05-27T08:05:38.586 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3119 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 May 27 07:47 3164 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3256 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3257 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3258 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3350 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3351 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3379 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3380 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3381 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3394 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3394_fsm 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3394_vm 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3395 2024-05-27T08:05:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3429 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3430 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3431 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3433 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3439 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3440 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 May 27 07:50 3455 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 May 27 07:47 3456 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3456_fsm 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3456_vm 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3466 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3467 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3468 2024-05-27T08:05:38.588 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3501 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3502 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3503 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3534 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3541_fsm 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3541_vm 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3542 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3574 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3575 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3576 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3596 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3597 2024-05-27T08:05:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 3598 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3599 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3600_fsm 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3600_vm 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3601_fsm 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3601_vm 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3602_fsm 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3602_vm 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603 2024-05-27T08:05:38.590 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3603_fsm 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3603_vm 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3604 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3605 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3606 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3607 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3608 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 May 27 07:47 3609 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3712 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 3764_fsm 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3764_vm 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3766 2024-05-27T08:05:38.591 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 3767 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3997 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4143 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4144 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4145 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4146 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4147 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4148 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4149 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4150 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4151 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4152 2024-05-27T08:05:38.592 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4153 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4154 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4155 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4156 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4157 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4158 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4159 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4160 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4161 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4162 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4163 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4164 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4165 2024-05-27T08:05:38.593 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4166 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4167 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4168 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4169 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4170 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4171 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4172 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 4173 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4174 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 5002 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 548 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 549 2024-05-27T08:05:38.594 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6102 2024-05-27T08:05:38.595 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6104 2024-05-27T08:05:38.595 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 6106 2024-05-27T08:05:38.595 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6110 2024-05-27T08:05:38.924 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6111 2024-05-27T08:05:38.924 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6112 2024-05-27T08:05:38.924 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6113 2024-05-27T08:05:38.924 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6117 2024-05-27T08:05:38.924 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:47 826 2024-05-27T08:05:38.924 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 827 2024-05-27T08:05:38.924 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 828 2024-05-27T08:05:38.924 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 27 07:47 pg_filenode.map 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 May 27 07:47 pg_internal.init 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 May 27 07:47 PG_VERSION 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:50 . 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 566482485 May 27 07:49 .. 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/global: 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout:total 561 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 May 27 07:47 . 2024-05-27T08:05:38.925 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708196 May 27 08:05 .. 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1213 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1213_fsm 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1213_vm 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1214 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1214_fsm 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1214_vm 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 1232 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 1233 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1260 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1260_fsm 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1260_vm 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1261 2024-05-27T08:05:38.926 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1261_fsm 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1261_vm 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1262 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 1262_fsm 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 1262_vm 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2396 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 May 27 07:47 2396_fsm 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2396_vm 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2397 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2671 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2672 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2676 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2677 2024-05-27T08:05:38.927 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2694 2024-05-27T08:05:38.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2695 2024-05-27T08:05:38.978 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2697 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:47 2698 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2846 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2847 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2964 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2965 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 2966 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 2967 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 3592 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 3593 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4060 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4061 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4175 2024-05-27T08:05:38.979 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4176 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4177 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4178 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4181 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4182 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4183 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4184 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 4185 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 4186 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 6000 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6001 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6002 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 6100 2024-05-27T08:05:38.980 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6114 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:47 6115 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 08:05 pg_control 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 May 27 07:47 pg_filenode.map 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 May 27 07:47 pg_internal.init 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/log: 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:total 3 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 May 27 07:47 . 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1118 May 27 08:05 postgresql-Mon.log 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-05-27T08:05:38.981 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:38.982 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:38.982 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:38.982 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:38.982 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-05-27T08:05:38.982 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:38.982 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:38.982 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:38.982 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:38.982 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-05-27T08:05:38.982 INFO:teuthology.orchestra.run.smithi090.stdout:total 3 2024-05-27T08:05:39.146 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 27 08:05 . 2024-05-27T08:05:39.146 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.146 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 mappings 2024-05-27T08:05:39.146 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 27 08:05 replorigin_checkpoint 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 snapshots 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 27 08:05 .. 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 May 27 08:05 .. 2024-05-27T08:05:39.147 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout:total 2 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:46 . 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 members 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 offsets 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout:total 9 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 . 2024-05-27T08:05:39.148 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:46 .. 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 0000 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout:total 9 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:46 . 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 May 27 07:46 .. 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 May 27 07:52 0000 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:39.149 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.150 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:total 16 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 May 27 08:05 . 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 May 27 08:05 db_0.stat 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 May 27 08:05 db_13494.stat 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 May 27 08:05 global.stat 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.151 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-05-27T08:05:39.152 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:39.152 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 08:05 . 2024-05-27T08:05:39.152 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.152 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.152 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-05-27T08:05:39.152 INFO:teuthology.orchestra.run.smithi090.stdout:total 49 2024-05-27T08:05:39.152 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 08:05 . 2024-05-27T08:05:39.152 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.467 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 May 27 08:05 0007 2024-05-27T08:05:39.467 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.467 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-05-27T08:05:39.467 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:total 851970 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 872415232 May 27 08:05 . 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.468 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:05 000000010000000000000070 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:54 000000010000000000000071 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:59 000000010000000000000072 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:01 000000010000000000000073 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 000000010000000000000074 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:00 000000010000000000000075 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:57 000000010000000000000076 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:02 000000010000000000000077 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:59 000000010000000000000078 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 000000010000000000000079 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:00 00000001000000000000007A 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 00000001000000000000007B 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 00000001000000000000007C 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:01 00000001000000000000007D 2024-05-27T08:05:39.469 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 00000001000000000000007E 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:59 00000001000000000000007F 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 000000010000000000000080 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:57 000000010000000000000081 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 000000010000000000000082 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:01 000000010000000000000083 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:59 000000010000000000000084 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:01 000000010000000000000085 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 000000010000000000000086 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:59 000000010000000000000087 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 000000010000000000000088 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:00 000000010000000000000089 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:59 00000001000000000000008A 2024-05-27T08:05:39.470 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:00 00000001000000000000008B 2024-05-27T08:05:39.494 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 00000001000000000000008C 2024-05-27T08:05:39.494 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 00000001000000000000008D 2024-05-27T08:05:39.494 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:57 00000001000000000000008E 2024-05-27T08:05:39.494 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 07:58 00000001000000000000008F 2024-05-27T08:05:39.494 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:03 000000010000000000000090 2024-05-27T08:05:39.494 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:03 000000010000000000000091 2024-05-27T08:05:39.494 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:02 000000010000000000000092 2024-05-27T08:05:39.494 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:03 000000010000000000000093 2024-05-27T08:05:39.494 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:04 000000010000000000000094 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:03 000000010000000000000095 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:04 000000010000000000000096 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:02 000000010000000000000097 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:03 000000010000000000000098 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:03 000000010000000000000099 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:04 00000001000000000000009A 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:05 00000001000000000000009B 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:02 00000001000000000000009C 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:04 00000001000000000000009D 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:03 00000001000000000000009E 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:04 00000001000000000000009F 2024-05-27T08:05:39.495 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:04 0000000100000000000000A0 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:03 0000000100000000000000A1 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:02 0000000100000000000000A2 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 May 27 08:04 0000000100000000000000A3 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 archive_status 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:total 1 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 May 27 07:46 . 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 872415232 May 27 08:05 .. 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:total 121 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 27 07:46 . 2024-05-27T08:05:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1439708267 May 27 08:05 .. 2024-05-27T08:05:39.497 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 May 27 08:05 0000 2024-05-27T08:05:39.503 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-27T08:05:39.514 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T08:05:39.572 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-27T08:05:39.593 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-27T08:05:39.593 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T08:05:39.593 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T08:05:39.594 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-27T08:05:39.704 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-27T08:05:39.789 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-27T08:05:39.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.903+0000 7f475ee00700 1 -- 172.21.15.90:0/2250550428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47581027d0 msgr2=0x7f4758102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:39.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.903+0000 7f475ee00700 1 --2- 172.21.15.90:0/2250550428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47581027d0 0x7f4758102be0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4744009990 tx=0x7f4744009ca0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.903+0000 7f475ee00700 1 -- 172.21.15.90:0/2250550428 shutdown_connections 2024-05-27T08:05:39.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.903+0000 7f475ee00700 1 --2- 172.21.15.90:0/2250550428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4758103120 0x7f4758111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.903+0000 7f475ee00700 1 --2- 172.21.15.90:0/2250550428 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f475810a3d0 0x7f475810a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.903+0000 7f475ee00700 1 --2- 172.21.15.90:0/2250550428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47581027d0 0x7f4758102be0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.903+0000 7f475ee00700 1 -- 172.21.15.90:0/2250550428 >> 172.21.15.90:0/2250550428 conn(0x7f47580fe6e0 msgr2=0x7f4758100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:39.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.903+0000 7f475ee00700 1 -- 172.21.15.90:0/2250550428 shutdown_connections 2024-05-27T08:05:39.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.903+0000 7f475ee00700 1 -- 172.21.15.90:0/2250550428 wait complete. 2024-05-27T08:05:39.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.904+0000 7f475ee00700 1 Processor -- start 2024-05-27T08:05:39.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.904+0000 7f475ee00700 1 -- start start 2024-05-27T08:05:39.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f475ee00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47581027d0 0x7f475819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f475ee00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758103120 0x7f475819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f475ee00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f475810a3d0 0x7f475819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f475ee00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4758114350 con 0x7f47581027d0 2024-05-27T08:05:39.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f475ee00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47581141d0 con 0x7f4758103120 2024-05-27T08:05:39.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f475ee00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47581144d0 con 0x7f475810a3d0 2024-05-27T08:05:39.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f4757e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47581027d0 0x7f475819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f4757e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47581027d0 0x7f475819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48084/0 (socket says 172.21.15.90:48084) 2024-05-27T08:05:39.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f4757e00700 1 -- 172.21.15.90:0/3670171345 learned_addr learned my addr 172.21.15.90:0/3670171345 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:39.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f4757400700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758103120 0x7f475819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.905+0000 7f475ce00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f475810a3d0 0x7f475819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f4757e00700 1 -- 172.21.15.90:0/3670171345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f475810a3d0 msgr2=0x7f475819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:39.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f4757e00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f475810a3d0 0x7f475819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f4757e00700 1 -- 172.21.15.90:0/3670171345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758103120 msgr2=0x7f475819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:39.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f4757e00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758103120 0x7f475819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f4757e00700 1 -- 172.21.15.90:0/3670171345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475819fc20 con 0x7f47581027d0 2024-05-27T08:05:39.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f4757400700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758103120 0x7f475819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:39.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f475ce00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f475810a3d0 0x7f475819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:39.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f4757e00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47581027d0 0x7f475819ae00 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f474800c9b0 tx=0x7f474800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:39.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f4754c00700 1 -- 172.21.15.90:0/3670171345 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748007b00 con 0x7f47581027d0 2024-05-27T08:05:39.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f4754c00700 1 -- 172.21.15.90:0/3670171345 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4748007ca0 con 0x7f47581027d0 2024-05-27T08:05:39.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f475ee00700 1 -- 172.21.15.90:0/3670171345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475819ff10 con 0x7f47581027d0 2024-05-27T08:05:39.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.906+0000 7f4754c00700 1 -- 172.21.15.90:0/3670171345 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748018660 con 0x7f47581027d0 2024-05-27T08:05:39.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.907+0000 7f475ee00700 1 -- 172.21.15.90:0/3670171345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47581a0450 con 0x7f47581027d0 2024-05-27T08:05:39.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.910+0000 7f4754c00700 1 -- 172.21.15.90:0/3670171345 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f47480188c0 con 0x7f47581027d0 2024-05-27T08:05:39.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.911+0000 7f476d9cfb80 1 Processor -- start 2024-05-27T08:05:39.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.912+0000 7f476d9cfb80 1 -- start start 2024-05-27T08:05:39.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.912+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6b9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.912+0000 7f4754c00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f473c0b29f0 0x7f473c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.912+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6b9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.912+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6b9e70 0x5630ff6c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.912+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5630ff5e7e40 con 0x5630ff53ae40 2024-05-27T08:05:39.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.912+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5630ff5d79e0 con 0x5630ff6269f0 2024-05-27T08:05:39.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.912+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5630ff5dba20 con 0x5630ff6b9e70 2024-05-27T08:05:39.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.912+0000 7f4754c00700 1 -- 172.21.15.90:0/3670171345 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f47480e2a90 con 0x7f47581027d0 2024-05-27T08:05:39.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.912+0000 7f4757e00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f473c0b29f0 0x7f473c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f475ce00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f4757400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f475ce00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6b9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.90:44228/0 (socket says 172.21.15.90:44228) 2024-05-27T08:05:39.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f475ce00700 1 -- 172.21.15.90:0/3656548106 learned_addr learned my addr 172.21.15.90:0/3656548106 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:39.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f475ce00700 1 --2- 172.21.15.90:0/3656548106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6b9e70 0x5630ff6c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f4757400700 1 -- 172.21.15.90:0/3656548106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6b9e70 msgr2=0x5630ff6c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:39.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f4757400700 1 --2- 172.21.15.90:0/3656548106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6b9e70 0x5630ff6c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f4757400700 1 -- 172.21.15.90:0/3656548106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 msgr2=0x5630ff6b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:39.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f4757400700 1 --2- 172.21.15.90:0/3656548106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6b9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f4757400700 1 -- 172.21.15.90:0/3656548106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ff5a0420 con 0x5630ff53ae40 2024-05-27T08:05:39.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.913+0000 7f4757e00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f473c0b29f0 0x7f473c0b4f00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f474800cfd0 tx=0x7f474800fc40 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:39.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.914+0000 7f4757400700 1 --2- 172.21.15.90:0/3656548106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6b9870 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4744000c00 tx=0x7f47440096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec247b46edcd820 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.914+0000 7f474d600700 1 -- 172.21.15.90:0/3656548106 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744025070 con 0x5630ff53ae40 2024-05-27T08:05:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.914+0000 7f474d600700 1 -- 172.21.15.90:0/3656548106 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4744003c50 con 0x5630ff53ae40 2024-05-27T08:05:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.914+0000 7f474d600700 1 -- 172.21.15.90:0/3656548106 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474400b7d0 con 0x5630ff53ae40 2024-05-27T08:05:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.915+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3656548106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 msgr2=0x5630ff6b9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.915+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3656548106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6b9870 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4744000c00 tx=0x7f47440096e0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.915+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3656548106 shutdown_connections 2024-05-27T08:05:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.915+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3656548106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6b9e70 0x5630ff6c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.915+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3656548106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.915+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3656548106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6b9870 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.915+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3656548106 >> 172.21.15.90:0/3656548106 conn(0x5630ff5d4390 msgr2=0x5630ff6b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:39.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.915+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3656548106 shutdown_connections 2024-05-27T08:05:39.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.916+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3656548106 wait complete. 2024-05-27T08:05:39.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.916+0000 7f476d9cfb80 1 -- start start 2024-05-27T08:05:39.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.916+0000 7f476d9cfb80 1 Processor -- start 2024-05-27T08:05:39.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.917+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.917+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.917+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6b9e70 0x5630ff6c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.917+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5630ff4ddfa0 con 0x5630ff53ae40 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.917+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5630ff578760 con 0x5630ff6269f0 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.917+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5630ff6c6b20 con 0x5630ff6b9e70 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.917+0000 7f476d9cfb80 10 client.0 _ll_register_callbacks cb 0x5630ff581c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.917+0000 7f475ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.917+0000 7f475ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6c36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48110/0 (socket says 172.21.15.90:48110) 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.917+0000 7f475ce00700 1 -- 172.21.15.90:0/3149351901 learned_addr learned my addr 172.21.15.90:0/3149351901 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.918+0000 7f4757400700 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.918+0000 7f475ce00700 1 -- 172.21.15.90:0/3149351901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6b9e70 msgr2=0x5630ff6c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.918+0000 7f475ce00700 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6b9e70 0x5630ff6c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.918+0000 7f475ce00700 1 -- 172.21.15.90:0/3149351901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 msgr2=0x5630ff6c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:39.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.918+0000 7f475ce00700 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.918+0000 7f475ce00700 1 -- 172.21.15.90:0/3149351901 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ff5278d0 con 0x5630ff53ae40 2024-05-27T08:05:39.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.918+0000 7f475ce00700 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6c36a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4750007db0 tx=0x7f4750017630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d405af1635f392 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:39.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.918+0000 7f4742000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:39.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.918+0000 7f4742000700 1 -- 172.21.15.90:0/3149351901 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750017cf0 con 0x5630ff53ae40 2024-05-27T08:05:39.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.918+0000 7f4742000700 1 -- 172.21.15.90:0/3149351901 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f4750019400 con 0x5630ff53ae40 2024-05-27T08:05:39.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.919+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3149351901 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4744003c50 con 0x5630ff53ae40 2024-05-27T08:05:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.919+0000 7f4742000700 1 -- 172.21.15.90:0/3149351901 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4750027050 con 0x5630ff53ae40 2024-05-27T08:05:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.919+0000 7f476d9cfb80 10 client.18561 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.919+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3149351901 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5630ff5a8060 con 0x5630ff53ae40 2024-05-27T08:05:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.919+0000 7f4742000700 1 -- 172.21.15.90:0/3149351901 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4744003c50 con 0x5630ff53ae40 2024-05-27T08:05:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.919+0000 7f4742000700 1 -- 172.21.15.90:0/3149351901 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f4750018040 con 0x5630ff53ae40 2024-05-27T08:05:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.920+0000 7f476d9cfb80 10 client.18561 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.920+0000 7f476d9cfb80 10 client.18561 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T08:05:39.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.920+0000 7f476d9cfb80 20 client.18561 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:39.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.920+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5630ff6d7fc0 0x5630ff6da440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.920+0000 7f476d9cfb80 4 client.18561 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:39.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.920+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3149351901 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630ff5e9d90 con 0x5630ff6d7fc0 2024-05-27T08:05:39.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.920+0000 7f4757400700 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5630ff6d7fc0 0x5630ff6da440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.921+0000 7f4757400700 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5630ff6d7fc0 0x5630ff6da440 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3cebceaa4715080 server_cookie=9bbc6036e9b26030 in_seq=0 out_seq=0 2024-05-27T08:05:39.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.921+0000 7f4742000700 10 client.18561 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T08:05:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.923+0000 7f4742000700 1 -- 172.21.15.90:0/3149351901 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x5630ff5e9d90 con 0x5630ff6d7fc0 2024-05-27T08:05:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.923+0000 7f4742000700 10 client.18561 handle_command_reply: tid=0 2024-05-27T08:05:39.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.923+0000 7f476d9cfb80 1 client.18561 shutdown 2024-05-27T08:05:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3149351901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 msgr2=0x5630ff6c36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6c36a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4750007db0 tx=0x7f4750017630 comp rx=0 tx=0).stop 2024-05-27T08:05:39.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3149351901 shutdown_connections 2024-05-27T08:05:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5630ff6d7fc0 0x5630ff6da440 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6b9e70 0x5630ff6c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff53ae40 0x5630ff6c36a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3149351901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3149351901 >> 172.21.15.90:0/3149351901 conn(0x5630ff5d4390 msgr2=0x5630ff6b9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f4742000700 0 client.18561 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T08:05:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3149351901 shutdown_connections 2024-05-27T08:05:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3149351901 wait complete. 2024-05-27T08:05:39.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.924+0000 7f476d9cfb80 20 client.18561 trim_cache size 0 max 16384 2024-05-27T08:05:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.941+0000 7f476d9cfb80 1 Processor -- start 2024-05-27T08:05:39.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.941+0000 7f476d9cfb80 1 -- start start 2024-05-27T08:05:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.942+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff53ae40 0x5630ff6bee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.942+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.942+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6bd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.942+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f47581afde0 con 0x5630ff6b9e70 2024-05-27T08:05:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.942+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47581ace30 con 0x5630ff6269f0 2024-05-27T08:05:39.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.942+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47581af040 con 0x5630ff53ae40 2024-05-27T08:05:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.942+0000 7f475ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6bd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.942+0000 7f475ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6bd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48116/0 (socket says 172.21.15.90:48116) 2024-05-27T08:05:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.942+0000 7f475ce00700 1 -- 172.21.15.90:0/3278274573 learned_addr learned my addr 172.21.15.90:0/3278274573 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.943+0000 7f4757400700 1 --2- 172.21.15.90:0/3278274573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.942+0000 7f475ce00700 1 --2- 172.21.15.90:0/3278274573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff53ae40 0x5630ff6bee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.943+0000 7f475ce00700 1 -- 172.21.15.90:0/3278274573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff53ae40 msgr2=0x5630ff6bee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:39.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.943+0000 7f475ce00700 1 --2- 172.21.15.90:0/3278274573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff53ae40 0x5630ff6bee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.943+0000 7f475ce00700 1 -- 172.21.15.90:0/3278274573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 msgr2=0x5630ff6b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:39.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.943+0000 7f475ce00700 1 --2- 172.21.15.90:0/3278274573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.943+0000 7f475ce00700 1 -- 172.21.15.90:0/3278274573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ff6b1240 con 0x5630ff6b9e70 2024-05-27T08:05:39.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.943+0000 7f475ce00700 1 --2- 172.21.15.90:0/3278274573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6bd5f0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4750000f80 tx=0x7f475000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb5d46a6e433a8b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.943+0000 7f474cc00700 1 -- 172.21.15.90:0/3278274573 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750018e40 con 0x5630ff6b9e70 2024-05-27T08:05:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.944+0000 7f474cc00700 1 -- 172.21.15.90:0/3278274573 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4750016a20 con 0x5630ff6b9e70 2024-05-27T08:05:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.944+0000 7f474cc00700 1 -- 172.21.15.90:0/3278274573 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47500166d0 con 0x5630ff6b9e70 2024-05-27T08:05:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.945+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3278274573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 msgr2=0x5630ff6bd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.945+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3278274573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6bd5f0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4750000f80 tx=0x7f475000b040 comp rx=0 tx=0).stop 2024-05-27T08:05:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.945+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3278274573 shutdown_connections 2024-05-27T08:05:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.945+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3278274573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6bd5f0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.945+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3278274573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff53ae40 0x5630ff6bee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.945+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/3278274573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff6269f0 0x5630ff6b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.945+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3278274573 >> 172.21.15.90:0/3278274573 conn(0x5630ff5d4390 msgr2=0x5630ff6bcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:39.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.945+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3278274573 shutdown_connections 2024-05-27T08:05:39.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.945+0000 7f476d9cfb80 1 -- 172.21.15.90:0/3278274573 wait complete. 2024-05-27T08:05:39.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.945+0000 7f476d9cfb80 1 -- start start 2024-05-27T08:05:39.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.946+0000 7f476d9cfb80 1 Processor -- start 2024-05-27T08:05:39.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.946+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff53ae40 0x5630ff6d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.946+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6269f0 0x5630ff6cb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.946+0000 7f476d9cfb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.946+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f47581ae970 con 0x5630ff6b9e70 2024-05-27T08:05:39.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.946+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47581ad500 con 0x5630ff53ae40 2024-05-27T08:05:39.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.946+0000 7f476d9cfb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47581adbd0 con 0x5630ff6269f0 2024-05-27T08:05:39.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.946+0000 7f476d9cfb80 10 client.0 _ll_register_callbacks cb 0x5630ff581c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f475ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f475ce00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48130/0 (socket says 172.21.15.90:48130) 2024-05-27T08:05:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f475ce00700 1 -- 172.21.15.90:0/900647676 learned_addr learned my addr 172.21.15.90:0/900647676 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f475ce00700 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff53ae40 0x5630ff6d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f4757400700 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6269f0 0x5630ff6cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f475ce00700 1 -- 172.21.15.90:0/900647676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6269f0 msgr2=0x5630ff6cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f475ce00700 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6269f0 0x5630ff6cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f475ce00700 1 -- 172.21.15.90:0/900647676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff53ae40 msgr2=0x5630ff6d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f475ce00700 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff53ae40 0x5630ff6d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f475ce00700 1 -- 172.21.15.90:0/900647676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ff6cbee0 con 0x5630ff6b9e70 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f475ce00700 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6cb740 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x5630ff6bf370 tx=0x7f4750007a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66112e26cb96bb91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.947+0000 7f4742000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.948+0000 7f4742000700 1 -- 172.21.15.90:0/900647676 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475003d060 con 0x5630ff6b9e70 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.948+0000 7f4742000700 1 -- 172.21.15.90:0/900647676 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f4750016c90 con 0x5630ff6b9e70 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.948+0000 7f476d9cfb80 1 -- 172.21.15.90:0/900647676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4750016a20 con 0x5630ff6b9e70 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.948+0000 7f4742000700 1 -- 172.21.15.90:0/900647676 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f4750016e90 con 0x5630ff6b9e70 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.948+0000 7f476d9cfb80 10 client.18567 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.948+0000 7f476d9cfb80 1 -- 172.21.15.90:0/900647676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5630ff741a50 con 0x5630ff6b9e70 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.948+0000 7f4742000700 1 -- 172.21.15.90:0/900647676 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4750016a20 con 0x5630ff6b9e70 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.949+0000 7f4742000700 1 -- 172.21.15.90:0/900647676 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f47500384c0 con 0x5630ff6b9e70 2024-05-27T08:05:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.949+0000 7f476d9cfb80 10 client.18567 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.949+0000 7f476d9cfb80 10 client.18567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.i 2024-05-27T08:05:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.949+0000 7f476d9cfb80 20 client.18567 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.949+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5630ff742050 0x5630ff744470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.949+0000 7f476d9cfb80 4 client.18567 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:05:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.949+0000 7f476d9cfb80 1 -- 172.21.15.90:0/900647676 --> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630ff5e3570 con 0x5630ff742050 2024-05-27T08:05:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.949+0000 7f4757400700 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5630ff742050 0x5630ff744470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.950+0000 7f4757400700 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5630ff742050 0x5630ff744470 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ca13030d2411e886 server_cookie=abc9bc00e38798b1 in_seq=0 out_seq=0 2024-05-27T08:05:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.950+0000 7f4742000700 10 client.18567 ms_handle_connect on v2:172.21.15.136:6838/2113925564 2024-05-27T08:05:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f4742000700 1 -- 172.21.15.90:0/900647676 <== mds.4 v2:172.21.15.136:6838/2113925564 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630ff5e3570 con 0x5630ff742050 2024-05-27T08:05:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f4742000700 10 client.18567 handle_command_reply: tid=0 2024-05-27T08:05:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 client.18567 shutdown 2024-05-27T08:05:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 -- 172.21.15.90:0/900647676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 msgr2=0x5630ff6cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6cb740 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x5630ff6bf370 tx=0x7f4750007a70 comp rx=0 tx=0).stop 2024-05-27T08:05:39.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 -- 172.21.15.90:0/900647676 shutdown_connections 2024-05-27T08:05:39.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.136:6838/2113925564,v1:172.21.15.136:6839/2113925564] conn(0x5630ff742050 0x5630ff744470 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5630ff6b9e70 0x5630ff6cb740 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630ff53ae40 0x5630ff6d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 --2- 172.21.15.90:0/900647676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5630ff6269f0 0x5630ff6cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f4742000700 0 client.18567 ms_handle_reset on v2:172.21.15.136:6838/2113925564 2024-05-27T08:05:39.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 -- 172.21.15.90:0/900647676 >> 172.21.15.90:0/900647676 conn(0x5630ff5d4390 msgr2=0x5630ff6d3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:39.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 -- 172.21.15.90:0/900647676 shutdown_connections 2024-05-27T08:05:39.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.951+0000 7f476d9cfb80 1 -- 172.21.15.90:0/900647676 wait complete. 2024-05-27T08:05:39.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f476d9cfb80 20 client.18567 trim_cache size 0 max 16384 2024-05-27T08:05:39.955 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:05:39.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f475ee00700 1 -- 172.21.15.90:0/3670171345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f473c0b29f0 msgr2=0x7f473c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:39.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f475ee00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f473c0b29f0 0x7f473c0b4f00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f474800cfd0 tx=0x7f474800fc40 comp rx=0 tx=0).stop 2024-05-27T08:05:39.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f475ee00700 1 -- 172.21.15.90:0/3670171345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47581027d0 msgr2=0x7f475819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:39.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f475ee00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47581027d0 0x7f475819ae00 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f474800c9b0 tx=0x7f474800cd70 comp rx=0 tx=0).stop 2024-05-27T08:05:39.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f475ee00700 1 -- 172.21.15.90:0/3670171345 shutdown_connections 2024-05-27T08:05:39.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f475ee00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f475810a3d0 0x7f475819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f475ee00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47581027d0 0x7f475819ae00 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f475ee00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f473c0b29f0 0x7f473c0b4f00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f475ee00700 1 --2- 172.21.15.90:0/3670171345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758103120 0x7f475819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:39.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.952+0000 7f475ee00700 1 -- 172.21.15.90:0/3670171345 >> 172.21.15.90:0/3670171345 conn(0x7f47580fe6e0 msgr2=0x7f4758108100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:39.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.953+0000 7f475ee00700 1 -- 172.21.15.90:0/3670171345 shutdown_connections 2024-05-27T08:05:39.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:39.953+0000 7f475ee00700 1 -- 172.21.15.90:0/3670171345 wait complete. 2024-05-27T08:05:40.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:39 smithi090 ceph-mon[31429]: pgmap v620: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.2 MiB/s wr, 431 op/s 2024-05-27T08:05:40.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.054+0000 7f1ccf400640 1 -- 172.21.15.90:0/2880792202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc8106610 msgr2=0x7f1cc81129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:40.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.054+0000 7f1ccf400640 1 --2- 172.21.15.90:0/2880792202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc8106610 0x7f1cc81129e0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009ca0 tx=0x7f1cc002a8c0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.054+0000 7f1ccf400640 1 -- 172.21.15.90:0/2880792202 shutdown_connections 2024-05-27T08:05:40.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.054+0000 7f1ccf400640 1 --2- 172.21.15.90:0/2880792202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc8106610 0x7f1cc81129e0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.054+0000 7f1ccf400640 1 --2- 172.21.15.90:0/2880792202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cc810c360 0x7f1cc8105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.054+0000 7f1ccf400640 1 --2- 172.21.15.90:0/2880792202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc810bec0 0x7f1cc8105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.054+0000 7f1ccf400640 1 -- 172.21.15.90:0/2880792202 >> 172.21.15.90:0/2880792202 conn(0x7f1cc81016b0 msgr2=0x7f1cc8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:40.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.054+0000 7f1ccf400640 1 -- 172.21.15.90:0/2880792202 shutdown_connections 2024-05-27T08:05:40.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.054+0000 7f1ccf400640 1 -- 172.21.15.90:0/2880792202 wait complete. 2024-05-27T08:05:40.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.055+0000 7f1ccf400640 1 Processor -- start 2024-05-27T08:05:40.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.055+0000 7f1ccf400640 1 -- start start 2024-05-27T08:05:40.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cc8106610 0x7f1cc81a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:40.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccf400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc810bec0 0x7f1cc81a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc810c360 0x7f1cc819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccf400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8115fb0 con 0x7f1cc810c360 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccf400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8115e30 con 0x7f1cc810bec0 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccf400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc8116130 con 0x7f1cc8106610 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc810c360 0x7f1cc819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc810c360 0x7f1cc819c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48152/0 (socket says 172.21.15.90:48152) 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccd400640 1 -- 172.21.15.90:0/3098142788 learned_addr learned my addr 172.21.15.90:0/3098142788 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccca00640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cc8106610 0x7f1cc81a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1cc7e00640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc810bec0 0x7f1cc81a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.056+0000 7f1ccd400640 1 -- 172.21.15.90:0/3098142788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cc8106610 msgr2=0x7f1cc81a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1ccd400640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cc8106610 0x7f1cc81a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1ccd400640 1 -- 172.21.15.90:0/3098142788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc810bec0 msgr2=0x7f1cc81a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1ccd400640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc810bec0 0x7f1cc81a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1ccd400640 1 -- 172.21.15.90:0/3098142788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc819cef0 con 0x7f1cc810c360 2024-05-27T08:05:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1cc7e00640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc810bec0 0x7f1cc81a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1ccd400640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc810c360 0x7f1cc819c810 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0031040 tx=0x7f1cc0032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1cc5600640 1 -- 172.21.15.90:0/3098142788 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0004040 con 0x7f1cc810c360 2024-05-27T08:05:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1cc5600640 1 -- 172.21.15.90:0/3098142788 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cc0004670 con 0x7f1cc810c360 2024-05-27T08:05:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1ccf400640 1 -- 172.21.15.90:0/3098142788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc819d180 con 0x7f1cc810c360 2024-05-27T08:05:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1cc5600640 1 -- 172.21.15.90:0/3098142788 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0030d90 con 0x7f1cc810c360 2024-05-27T08:05:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.057+0000 7f1ccf400640 1 -- 172.21.15.90:0/3098142788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc81a93e0 con 0x7f1cc810c360 2024-05-27T08:05:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.059+0000 7f1ccf400640 1 -- 172.21.15.90:0/3098142788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc81071a0 con 0x7f1cc810c360 2024-05-27T08:05:40.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.061+0000 7f1cc5600640 1 -- 172.21.15.90:0/3098142788 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1cc00305c0 con 0x7f1cc810c360 2024-05-27T08:05:40.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.063+0000 7f1cc5600640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1cac0b29d0 0x7f1cac0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:40.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.064+0000 7f1cc5600640 1 -- 172.21.15.90:0/3098142788 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f1cc01040f0 con 0x7f1cc810c360 2024-05-27T08:05:40.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.064+0000 7f1ccca00640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1cac0b29d0 0x7f1cac0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:40.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:39 smithi129 ceph-mon[35254]: pgmap v620: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.2 MiB/s wr, 431 op/s 2024-05-27T08:05:40.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.065+0000 7f1ccca00640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1cac0b29d0 0x7f1cac0b4ee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8005fd0 tx=0x7f1cb8005e40 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:40.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.069+0000 7f1cc5600640 1 -- 172.21.15.90:0/3098142788 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1cc00098b0 con 0x7f1cc810c360 2024-05-27T08:05:40.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:39 smithi136 ceph-mon[35324]: pgmap v620: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 3.2 MiB/s wr, 431 op/s 2024-05-27T08:05:40.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.301+0000 7f1ccf400640 1 -- 172.21.15.90:0/3098142788 --> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1cc810e040 con 0x7f1cac0b29d0 2024-05-27T08:05:40.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.317+0000 7f1cc5600640 1 -- 172.21.15.90:0/3098142788 <== mgr.34967 v2:172.21.15.90:6800/4256436582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f1cb800ea60 con 0x7f1cac0b29d0 2024-05-27T08:05:40.318 INFO:teuthology.orchestra.run.smithi090.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-27T00:00:00", "created": "2024-05-27T07:46:46", "first": "2024-05-27T07:47:00", "last": "2024-05-27T08:05:00", "last_pruned": "2024-05-27T08:04:00", "created_count": 19, "pruned_count": 12, "active": true} 2024-05-27T08:05:40.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 -- 172.21.15.90:0/3098142788 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1cac0b29d0 msgr2=0x7f1cac0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:40.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1cac0b29d0 0x7f1cac0b4ee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8005fd0 tx=0x7f1cb8005e40 comp rx=0 tx=0).stop 2024-05-27T08:05:40.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 -- 172.21.15.90:0/3098142788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc810c360 msgr2=0x7f1cc819c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:40.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc810c360 0x7f1cc819c810 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0031040 tx=0x7f1cc0032bc0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 -- 172.21.15.90:0/3098142788 shutdown_connections 2024-05-27T08:05:40.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f1cac0b29d0 0x7f1cac0b4ee0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cc810bec0 0x7f1cc81a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc810c360 0x7f1cc819c810 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 --2- 172.21.15.90:0/3098142788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cc8106610 0x7f1cc81a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 -- 172.21.15.90:0/3098142788 >> 172.21.15.90:0/3098142788 conn(0x7f1cc81016b0 msgr2=0x7f1cc8103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:40.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 -- 172.21.15.90:0/3098142788 shutdown_connections 2024-05-27T08:05:40.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.321+0000 7f1ccf400640 1 -- 172.21.15.90:0/3098142788 wait complete. 2024-05-27T08:05:40.350 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-27T08:05:40.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.621+0000 7fa422400640 1 -- 172.21.15.90:0/263394066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41c10beb0 msgr2=0x7fa41c105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:40.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.621+0000 7fa422400640 1 --2- 172.21.15.90:0/263394066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41c10beb0 0x7fa41c105a50 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009ca0 tx=0x7fa40c02a8c0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.622+0000 7fa422400640 1 -- 172.21.15.90:0/263394066 shutdown_connections 2024-05-27T08:05:40.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.622+0000 7fa422400640 1 --2- 172.21.15.90:0/263394066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c10c350 0x7fa41c105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.622+0000 7fa422400640 1 --2- 172.21.15.90:0/263394066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa41c106600 0x7fa41c1129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.622+0000 7fa422400640 1 --2- 172.21.15.90:0/263394066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41c10beb0 0x7fa41c105a50 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.622+0000 7fa422400640 1 -- 172.21.15.90:0/263394066 >> 172.21.15.90:0/263394066 conn(0x7fa41c1016b0 msgr2=0x7fa41c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:40.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.622+0000 7fa422400640 1 -- 172.21.15.90:0/263394066 shutdown_connections 2024-05-27T08:05:40.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.622+0000 7fa422400640 1 -- 172.21.15.90:0/263394066 wait complete. 2024-05-27T08:05:40.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.623+0000 7fa422400640 1 Processor -- start 2024-05-27T08:05:40.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.623+0000 7fa422400640 1 -- start start 2024-05-27T08:05:40.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.624+0000 7fa422400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c106600 0x7fa41c1a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:40.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.624+0000 7fa422400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa41c10beb0 0x7fa41c1a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:40.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.624+0000 7fa422400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41c10c350 0x7fa41c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:40.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.624+0000 7fa422400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c115f90 con 0x7fa41c10c350 2024-05-27T08:05:40.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.624+0000 7fa422400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c115e10 con 0x7fa41c106600 2024-05-27T08:05:40.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.624+0000 7fa422400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c116110 con 0x7fa41c10beb0 2024-05-27T08:05:40.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.624+0000 7fa41be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41c10c350 0x7fa41c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:40.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.624+0000 7fa41be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41c10c350 0x7fa41c19c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48170/0 (socket says 172.21.15.90:48170) 2024-05-27T08:05:40.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.624+0000 7fa41be00640 1 -- 172.21.15.90:0/3324598091 learned_addr learned my addr 172.21.15.90:0/3324598091 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:40.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa41b400640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c106600 0x7fa41c1a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:40.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa41aa00640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa41c10beb0 0x7fa41c1a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:40.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa41be00640 1 -- 172.21.15.90:0/3324598091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa41c10beb0 msgr2=0x7fa41c1a2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:40.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa41be00640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa41c10beb0 0x7fa41c1a2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa41be00640 1 -- 172.21.15.90:0/3324598091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c106600 msgr2=0x7fa41c1a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:40.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa41be00640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c106600 0x7fa41c1a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa41be00640 1 -- 172.21.15.90:0/3324598091 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c19cee0 con 0x7fa41c10c350 2024-05-27T08:05:40.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa41be00640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41c10c350 0x7fa41c19c870 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa41400c970 tx=0x7fa41400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:40.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa413e00640 1 -- 172.21.15.90:0/3324598091 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414007bb0 con 0x7fa41c10c350 2024-05-27T08:05:40.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa413e00640 1 -- 172.21.15.90:0/3324598091 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa414007d50 con 0x7fa41c10c350 2024-05-27T08:05:40.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa422400640 1 -- 172.21.15.90:0/3324598091 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41c19d1d0 con 0x7fa41c10c350 2024-05-27T08:05:40.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.625+0000 7fa413e00640 1 -- 172.21.15.90:0/3324598091 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414018660 con 0x7fa41c10c350 2024-05-27T08:05:40.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.626+0000 7fa422400640 1 -- 172.21.15.90:0/3324598091 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41c19d6d0 con 0x7fa41c10c350 2024-05-27T08:05:40.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.627+0000 7fa422400640 1 -- 172.21.15.90:0/3324598091 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41c107190 con 0x7fa41c10c350 2024-05-27T08:05:40.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.629+0000 7fa413e00640 1 -- 172.21.15.90:0/3324598091 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa41400f970 con 0x7fa41c10c350 2024-05-27T08:05:40.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.631+0000 7fa413e00640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa4000b2950 0x7fa4000b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:40.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.631+0000 7fa413e00640 1 -- 172.21.15.90:0/3324598091 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fa4140e2d00 con 0x7fa41c10c350 2024-05-27T08:05:40.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.631+0000 7fa41b400640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa4000b2950 0x7fa4000b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:40.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.632+0000 7fa41b400640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa4000b2950 0x7fa4000b4e60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa40c006010 tx=0x7fa40c000f40 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:40.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.634+0000 7fa413e00640 1 -- 172.21.15.90:0/3324598091 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa414054df0 con 0x7fa41c10c350 2024-05-27T08:05:40.864 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-27T08:05:40.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.870+0000 7fa422400640 1 -- 172.21.15.90:0/3324598091 --> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa41c1021d0 con 0x7fa4000b2950 2024-05-27T08:05:40.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.886+0000 7fa413e00640 1 -- 172.21.15.90:0/3324598091 <== mgr.34967 v2:172.21.15.90:6800/4256436582 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa41c1021d0 con 0x7fa4000b2950 2024-05-27T08:05:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/ 1m 6m3h 2024-05-27T08:05:40.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 -- 172.21.15.90:0/3324598091 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa4000b2950 msgr2=0x7fa4000b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:40.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa4000b2950 0x7fa4000b4e60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa40c006010 tx=0x7fa40c000f40 comp rx=0 tx=0).stop 2024-05-27T08:05:40.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 -- 172.21.15.90:0/3324598091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41c10c350 msgr2=0x7fa41c19c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:40.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41c10c350 0x7fa41c19c870 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa41400c970 tx=0x7fa41400ce30 comp rx=0 tx=0).stop 2024-05-27T08:05:40.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 -- 172.21.15.90:0/3324598091 shutdown_connections 2024-05-27T08:05:40.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fa4000b2950 0x7fa4000b4e60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa41c10c350 0x7fa41c19c870 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa41c10beb0 0x7fa41c1a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 --2- 172.21.15.90:0/3324598091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa41c106600 0x7fa41c1a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:40.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 -- 172.21.15.90:0/3324598091 >> 172.21.15.90:0/3324598091 conn(0x7fa41c1016b0 msgr2=0x7fa41c101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:40.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 -- 172.21.15.90:0/3324598091 shutdown_connections 2024-05-27T08:05:40.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:40.888+0000 7fa422400640 1 -- 172.21.15.90:0/3324598091 wait complete. 2024-05-27T08:05:40.917 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-27T08:05:40.932 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-27T08:05:41.299 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:42.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:41 smithi129 ceph-mon[35254]: from='client.18573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:05:42.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:41 smithi129 ceph-mon[35254]: pgmap v621: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.2 MiB/s wr, 310 op/s 2024-05-27T08:05:42.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:41 smithi129 ceph-mon[35254]: from='client.18579 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:05:42.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:41 smithi090 ceph-mon[31429]: from='client.18573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:05:42.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:41 smithi090 ceph-mon[31429]: pgmap v621: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.2 MiB/s wr, 310 op/s 2024-05-27T08:05:42.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:41 smithi090 ceph-mon[31429]: from='client.18579 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:05:42.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:41 smithi136 ceph-mon[35324]: from='client.18573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:05:42.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:41 smithi136 ceph-mon[35324]: pgmap v621: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.2 MiB/s wr, 310 op/s 2024-05-27T08:05:42.252 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:41 smithi136 ceph-mon[35324]: from='client.18579 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:05:43.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.483+0000 7feb65a00700 1 -- 172.21.15.90:0/2847567977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb60076670 msgr2=0x7feb60110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:43.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.483+0000 7feb65a00700 1 --2- 172.21.15.90:0/2847567977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb60076670 0x7feb60110650 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7feb48009990 tx=0x7feb48009ca0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.483+0000 7feb65a00700 1 -- 172.21.15.90:0/2847567977 shutdown_connections 2024-05-27T08:05:43.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.483+0000 7feb65a00700 1 --2- 172.21.15.90:0/2847567977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb60110b90 0x7feb60112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.483+0000 7feb65a00700 1 --2- 172.21.15.90:0/2847567977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb60075d20 0x7feb60076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.483+0000 7feb65a00700 1 --2- 172.21.15.90:0/2847567977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb60076670 0x7feb60110650 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.483+0000 7feb65a00700 1 -- 172.21.15.90:0/2847567977 >> 172.21.15.90:0/2847567977 conn(0x7feb60100030 msgr2=0x7feb60102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:43.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.483+0000 7feb65a00700 1 -- 172.21.15.90:0/2847567977 shutdown_connections 2024-05-27T08:05:43.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.483+0000 7feb65a00700 1 -- 172.21.15.90:0/2847567977 wait complete. 2024-05-27T08:05:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.484+0000 7feb65a00700 1 Processor -- start 2024-05-27T08:05:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.485+0000 7feb65a00700 1 -- start start 2024-05-27T08:05:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.485+0000 7feb65a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb60075d20 0x7feb6019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.485+0000 7feb65a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb60076670 0x7feb6019d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.485+0000 7feb65a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb60110b90 0x7feb601a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.485+0000 7feb65a00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feb60116540 con 0x7feb60076670 2024-05-27T08:05:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.485+0000 7feb65a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feb601163c0 con 0x7feb60075d20 2024-05-27T08:05:43.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.485+0000 7feb65a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feb601166c0 con 0x7feb60110b90 2024-05-27T08:05:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.485+0000 7feb56000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb60076670 0x7feb6019d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.485+0000 7feb56000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb60076670 0x7feb6019d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48188/0 (socket says 172.21.15.90:48188) 2024-05-27T08:05:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.485+0000 7feb56000700 1 -- 172.21.15.90:0/1001861251 learned_addr learned my addr 172.21.15.90:0/1001861251 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.486+0000 7feb5ea00700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb60075d20 0x7feb6019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.486+0000 7feb5f400700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb60110b90 0x7feb601a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.486+0000 7feb56000700 1 -- 172.21.15.90:0/1001861251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb60110b90 msgr2=0x7feb601a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.486+0000 7feb56000700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb60110b90 0x7feb601a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.486+0000 7feb56000700 1 -- 172.21.15.90:0/1001861251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb60075d20 msgr2=0x7feb6019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.486+0000 7feb56000700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb60075d20 0x7feb6019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.486+0000 7feb56000700 1 -- 172.21.15.90:0/1001861251 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb601a1d70 con 0x7feb60076670 2024-05-27T08:05:43.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.486+0000 7feb56000700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb60076670 0x7feb6019d4f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7feb480057f0 tx=0x7feb48009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:43.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.487+0000 7feb57e00700 1 -- 172.21.15.90:0/1001861251 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48027070 con 0x7feb60076670 2024-05-27T08:05:43.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.487+0000 7feb57e00700 1 -- 172.21.15.90:0/1001861251 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb4800fd60 con 0x7feb60076670 2024-05-27T08:05:43.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.487+0000 7feb65a00700 1 -- 172.21.15.90:0/1001861251 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb601a2000 con 0x7feb60076670 2024-05-27T08:05:43.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.487+0000 7feb57e00700 1 -- 172.21.15.90:0/1001861251 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48022de0 con 0x7feb60076670 2024-05-27T08:05:43.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.487+0000 7feb65a00700 1 -- 172.21.15.90:0/1001861251 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb601a24e0 con 0x7feb60076670 2024-05-27T08:05:43.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.491+0000 7feb57e00700 1 -- 172.21.15.90:0/1001861251 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7feb4802a070 con 0x7feb60076670 2024-05-27T08:05:43.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.493+0000 7feb57e00700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feb440b29f0 0x7feb440b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.493+0000 7feb57e00700 1 -- 172.21.15.90:0/1001861251 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7feb480e39e0 con 0x7feb60076670 2024-05-27T08:05:43.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.493+0000 7feb5ea00700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feb440b29f0 0x7feb440b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.493+0000 7feb744aeb80 1 Processor -- start 2024-05-27T08:05:43.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.494+0000 7feb744aeb80 1 -- start start 2024-05-27T08:05:43.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.494+0000 7feb5ea00700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feb440b29f0 0x7feb440b4f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb50005ea0 tx=0x7feb50005e30 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:43.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.494+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa56f9f0 0x558baa602330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.494+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa483e40 0x558baa602870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.494+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 0x558baa60f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.494+0000 7feb744aeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558baa530e40 con 0x558baa56f9f0 2024-05-27T08:05:43.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.494+0000 7feb744aeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558baa5209e0 con 0x558baa602e70 2024-05-27T08:05:43.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.494+0000 7feb744aeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558baa524a20 con 0x558baa483e40 2024-05-27T08:05:43.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.494+0000 7feb5ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa56f9f0 0x558baa602330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.495+0000 7feb5ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa56f9f0 0x558baa602330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48190/0 (socket says 172.21.15.90:48190) 2024-05-27T08:05:43.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.495+0000 7feb5ea00700 1 -- 172.21.15.90:0/302500132 learned_addr learned my addr 172.21.15.90:0/302500132 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:43.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.495+0000 7feb5f400700 1 --2- 172.21.15.90:0/302500132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 0x558baa60f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.495+0000 7feb56000700 1 --2- 172.21.15.90:0/302500132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa483e40 0x558baa602870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.495+0000 7feb5ea00700 1 -- 172.21.15.90:0/302500132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa483e40 msgr2=0x558baa602870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:43.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.495+0000 7feb5ea00700 1 --2- 172.21.15.90:0/302500132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa483e40 0x558baa602870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.495+0000 7feb5ea00700 1 -- 172.21.15.90:0/302500132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 msgr2=0x558baa60f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:43.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.495+0000 7feb5ea00700 1 --2- 172.21.15.90:0/302500132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 0x558baa60f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.495+0000 7feb5ea00700 1 -- 172.21.15.90:0/302500132 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558baa4e9420 con 0x558baa56f9f0 2024-05-27T08:05:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.495+0000 7feb5ea00700 1 --2- 172.21.15.90:0/302500132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa56f9f0 0x558baa602330 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7feb5000a520 tx=0x7feb5000af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5319d8e03faa038 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.496+0000 7feb4fe00700 1 -- 172.21.15.90:0/302500132 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50010d80 con 0x558baa56f9f0 2024-05-27T08:05:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.496+0000 7feb4fe00700 1 -- 172.21.15.90:0/302500132 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb50022070 con 0x558baa56f9f0 2024-05-27T08:05:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.496+0000 7feb4fe00700 1 -- 172.21.15.90:0/302500132 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5000f620 con 0x558baa56f9f0 2024-05-27T08:05:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.497+0000 7feb744aeb80 1 -- 172.21.15.90:0/302500132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa56f9f0 msgr2=0x558baa602330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.497+0000 7feb744aeb80 1 --2- 172.21.15.90:0/302500132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa56f9f0 0x558baa602330 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7feb5000a520 tx=0x7feb5000af90 comp rx=0 tx=0).stop 2024-05-27T08:05:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.497+0000 7feb744aeb80 1 -- 172.21.15.90:0/302500132 shutdown_connections 2024-05-27T08:05:43.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.497+0000 7feb744aeb80 1 --2- 172.21.15.90:0/302500132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 0x558baa60f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.497+0000 7feb744aeb80 1 --2- 172.21.15.90:0/302500132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa56f9f0 0x558baa602330 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.497+0000 7feb744aeb80 1 --2- 172.21.15.90:0/302500132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa483e40 0x558baa602870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.497+0000 7feb744aeb80 1 -- 172.21.15.90:0/302500132 >> 172.21.15.90:0/302500132 conn(0x558baa51d390 msgr2=0x558baa5fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:43.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.497+0000 7feb744aeb80 1 -- 172.21.15.90:0/302500132 shutdown_connections 2024-05-27T08:05:43.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.497+0000 7feb744aeb80 1 -- 172.21.15.90:0/302500132 wait complete. 2024-05-27T08:05:43.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.498+0000 7feb744aeb80 1 -- start start 2024-05-27T08:05:43.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.498+0000 7feb744aeb80 1 Processor -- start 2024-05-27T08:05:43.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 0x558baa60c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa60cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa60d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb744aeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558baa426fa0 con 0x558baa602e70 2024-05-27T08:05:43.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb744aeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558baa4c1760 con 0x558baa483e40 2024-05-27T08:05:43.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb744aeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558baa60fb80 con 0x558baa56f9f0 2024-05-27T08:05:43.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb744aeb80 10 client.0 _ll_register_callbacks cb 0x558baa4cac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb5f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa60d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb5f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa60d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48204/0 (socket says 172.21.15.90:48204) 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb5f400700 1 -- 172.21.15.90:0/2036447494 learned_addr learned my addr 172.21.15.90:0/2036447494 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb56000700 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa60cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.499+0000 7feb5ea00700 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 0x558baa60c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb5f400700 1 -- 172.21.15.90:0/2036447494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 msgr2=0x558baa60cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb5f400700 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa60cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb5f400700 1 -- 172.21.15.90:0/2036447494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 msgr2=0x558baa60c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb5f400700 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 0x558baa60c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb5f400700 1 -- 172.21.15.90:0/2036447494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558baa4708d0 con 0x558baa602e70 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb5ea00700 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 0x558baa60c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb5f400700 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa60d1d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7feb58010b30 tx=0x7feb58010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72367a28e47ec1c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:43.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb4cc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:43.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb4cc00700 1 -- 172.21.15.90:0/2036447494 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5801a020 con 0x558baa602e70 2024-05-27T08:05:43.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb4cc00700 1 -- 172.21.15.90:0/2036447494 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7feb5800ea60 con 0x558baa602e70 2024-05-27T08:05:43.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.500+0000 7feb744aeb80 1 -- 172.21.15.90:0/2036447494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb50022070 con 0x558baa602e70 2024-05-27T08:05:43.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.501+0000 7feb4cc00700 1 -- 172.21.15.90:0/2036447494 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7feb58025050 con 0x558baa602e70 2024-05-27T08:05:43.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.501+0000 7feb744aeb80 10 client.18591 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:43.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.501+0000 7feb744aeb80 1 -- 172.21.15.90:0/2036447494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558baa4f1060 con 0x558baa602e70 2024-05-27T08:05:43.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.501+0000 7feb4cc00700 1 -- 172.21.15.90:0/2036447494 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb50022070 con 0x558baa602e70 2024-05-27T08:05:43.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.502+0000 7feb4cc00700 1 -- 172.21.15.90:0/2036447494 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7feb58015b40 con 0x558baa602e70 2024-05-27T08:05:43.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.502+0000 7feb744aeb80 10 client.18591 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:43.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.502+0000 7feb744aeb80 10 client.18591 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T08:05:43.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.502+0000 7feb744aeb80 20 client.18591 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:43.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.502+0000 7feb744aeb80 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x558baa61cbb0 0x558baa61f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.502+0000 7feb744aeb80 4 client.18591 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:43.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.502+0000 7feb744aeb80 1 -- 172.21.15.90:0/2036447494 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558baa532d90 con 0x558baa61cbb0 2024-05-27T08:05:43.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.503+0000 7feb5ea00700 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x558baa61cbb0 0x558baa61f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.504+0000 7feb5ea00700 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x558baa61cbb0 0x558baa61f000 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a18e43fb7fec0c1b server_cookie=7be56fddb892b781 in_seq=0 out_seq=0 2024-05-27T08:05:43.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.504+0000 7feb4cc00700 10 client.18591 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T08:05:43.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb4cc00700 1 -- 172.21.15.90:0/2036447494 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558baa532d90 con 0x558baa61cbb0 2024-05-27T08:05:43.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb4cc00700 10 client.18591 handle_command_reply: tid=0 2024-05-27T08:05:43.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 client.18591 shutdown 2024-05-27T08:05:43.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 -- 172.21.15.90:0/2036447494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 msgr2=0x558baa60d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:43.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa60d1d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7feb58010b30 tx=0x7feb58010ef0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 -- 172.21.15.90:0/2036447494 shutdown_connections 2024-05-27T08:05:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x558baa61cbb0 0x558baa61f000 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa60d1d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 0x558baa60c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 --2- 172.21.15.90:0/2036447494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa60cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 -- 172.21.15.90:0/2036447494 >> 172.21.15.90:0/2036447494 conn(0x558baa51d390 msgr2=0x558baa602450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb4cc00700 0 client.18591 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T08:05:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 -- 172.21.15.90:0/2036447494 shutdown_connections 2024-05-27T08:05:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 1 -- 172.21.15.90:0/2036447494 wait complete. 2024-05-27T08:05:43.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.506+0000 7feb744aeb80 20 client.18591 trim_cache size 0 max 16384 2024-05-27T08:05:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.523+0000 7feb744aeb80 1 Processor -- start 2024-05-27T08:05:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.523+0000 7feb744aeb80 1 -- start start 2024-05-27T08:05:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.524+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 0x558baa607f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.524+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa618c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.524+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa6045e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.524+0000 7feb744aeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feb601b4110 con 0x558baa602e70 2024-05-27T08:05:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.524+0000 7feb744aeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feb601b1160 con 0x558baa483e40 2024-05-27T08:05:43.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.524+0000 7feb744aeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feb601b3370 con 0x558baa56f9f0 2024-05-27T08:05:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.524+0000 7feb5f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa6045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.524+0000 7feb5f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa6045e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48218/0 (socket says 172.21.15.90:48218) 2024-05-27T08:05:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.524+0000 7feb5f400700 1 -- 172.21.15.90:0/1176607888 learned_addr learned my addr 172.21.15.90:0/1176607888 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.524+0000 7feb5ea00700 1 --2- 172.21.15.90:0/1176607888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 0x558baa607f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.525+0000 7feb56000700 1 --2- 172.21.15.90:0/1176607888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa618c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.525+0000 7feb5f400700 1 -- 172.21.15.90:0/1176607888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 msgr2=0x558baa618c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.525+0000 7feb5f400700 1 --2- 172.21.15.90:0/1176607888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa618c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.525+0000 7feb5f400700 1 -- 172.21.15.90:0/1176607888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 msgr2=0x558baa607f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:43.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.525+0000 7feb5f400700 1 --2- 172.21.15.90:0/1176607888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 0x558baa607f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.525+0000 7feb5f400700 1 -- 172.21.15.90:0/1176607888 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558baa60acc0 con 0x558baa602e70 2024-05-27T08:05:43.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.525+0000 7feb5f400700 1 --2- 172.21.15.90:0/1176607888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa6045e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7feb58000f80 tx=0x7feb58007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0caeafdd0eb5e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:43.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.526+0000 7feb4f400700 1 -- 172.21.15.90:0/1176607888 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5802e070 con 0x558baa602e70 2024-05-27T08:05:43.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.526+0000 7feb4f400700 1 -- 172.21.15.90:0/1176607888 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb58027420 con 0x558baa602e70 2024-05-27T08:05:43.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.526+0000 7feb4f400700 1 -- 172.21.15.90:0/1176607888 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5801a040 con 0x558baa602e70 2024-05-27T08:05:43.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.527+0000 7feb744aeb80 1 -- 172.21.15.90:0/1176607888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 msgr2=0x558baa6045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:43.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.527+0000 7feb744aeb80 1 --2- 172.21.15.90:0/1176607888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa6045e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7feb58000f80 tx=0x7feb58007c70 comp rx=0 tx=0).stop 2024-05-27T08:05:43.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.527+0000 7feb744aeb80 1 -- 172.21.15.90:0/1176607888 shutdown_connections 2024-05-27T08:05:43.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.527+0000 7feb744aeb80 1 --2- 172.21.15.90:0/1176607888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa602e70 0x558baa6045e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.527+0000 7feb744aeb80 1 --2- 172.21.15.90:0/1176607888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa483e40 0x558baa607f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.527+0000 7feb744aeb80 1 --2- 172.21.15.90:0/1176607888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa618c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.527+0000 7feb744aeb80 1 -- 172.21.15.90:0/1176607888 >> 172.21.15.90:0/1176607888 conn(0x558baa51d390 msgr2=0x558baa603ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.527+0000 7feb744aeb80 1 -- 172.21.15.90:0/1176607888 shutdown_connections 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.527+0000 7feb744aeb80 1 -- 172.21.15.90:0/1176607888 wait complete. 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.527+0000 7feb744aeb80 1 -- start start 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.528+0000 7feb744aeb80 1 Processor -- start 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.528+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa483e40 0x558baa606860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.528+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa606da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.528+0000 7feb744aeb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 0x558baa613e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.528+0000 7feb744aeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feb601b2ca0 con 0x558baa483e40 2024-05-27T08:05:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.528+0000 7feb744aeb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feb601b1830 con 0x558baa602e70 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.528+0000 7feb744aeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feb601b1f00 con 0x558baa56f9f0 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.528+0000 7feb744aeb80 10 client.0 _ll_register_callbacks cb 0x558baa4cac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.528+0000 7feb5ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa483e40 0x558baa606860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb5ea00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa483e40 0x558baa606860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48234/0 (socket says 172.21.15.90:48234) 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb5ea00700 1 -- 172.21.15.90:0/3765407178 learned_addr learned my addr 172.21.15.90:0/3765407178 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb5f400700 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 0x558baa613e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb56000700 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa606da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb5ea00700 1 -- 172.21.15.90:0/3765407178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 msgr2=0x558baa606da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb5ea00700 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa606da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb5ea00700 1 -- 172.21.15.90:0/3765407178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 msgr2=0x558baa613e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb5ea00700 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 0x558baa613e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb5ea00700 1 -- 172.21.15.90:0/3765407178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558baa614350 con 0x558baa483e40 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb5f400700 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 0x558baa613e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb5ea00700 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa483e40 0x558baa606860 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7feb5000a9e0 tx=0x7feb5000a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa5efcb8faf2446 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.529+0000 7feb4cc00700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.530+0000 7feb4cc00700 1 -- 172.21.15.90:0/3765407178 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50004710 con 0x558baa483e40 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.530+0000 7feb4cc00700 1 -- 172.21.15.90:0/3765407178 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7feb500048b0 con 0x558baa483e40 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.530+0000 7feb744aeb80 1 -- 172.21.15.90:0/3765407178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb58027420 con 0x558baa483e40 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.530+0000 7feb4cc00700 1 -- 172.21.15.90:0/3765407178 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7feb50040050 con 0x558baa483e40 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.530+0000 7feb744aeb80 10 client.18597 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.530+0000 7feb744aeb80 1 -- 172.21.15.90:0/3765407178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558baa615590 con 0x558baa483e40 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.530+0000 7feb4cc00700 1 -- 172.21.15.90:0/3765407178 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb58027420 con 0x558baa483e40 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.531+0000 7feb4cc00700 1 -- 172.21.15.90:0/3765407178 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7feb5001da70 con 0x558baa483e40 2024-05-27T08:05:43.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.531+0000 7feb744aeb80 10 client.18597 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:43.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.531+0000 7feb744aeb80 10 client.18597 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-05-27T08:05:43.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.531+0000 7feb744aeb80 20 client.18597 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:43.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.531+0000 7feb744aeb80 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x558baa6876d0 0x558baa689af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:43.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.531+0000 7feb744aeb80 4 client.18597 mds_command: new command op to 34406 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:05:43.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.531+0000 7feb744aeb80 1 -- 172.21.15.90:0/3765407178 --> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558baa5fd120 con 0x558baa6876d0 2024-05-27T08:05:43.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.531+0000 7feb56000700 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x558baa6876d0 0x558baa689af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:43.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.532+0000 7feb56000700 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x558baa6876d0 0x558baa689af0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a57bf00641bca80 server_cookie=3b76ce63fbf59533 in_seq=0 out_seq=0 2024-05-27T08:05:43.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.532+0000 7feb4cc00700 10 client.18597 ms_handle_connect on v2:172.21.15.129:6832/3131589000 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb4cc00700 1 -- 172.21.15.90:0/3765407178 <== mds.3 v2:172.21.15.129:6832/3131589000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558baa5fd120 con 0x558baa6876d0 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb4cc00700 10 client.18597 handle_command_reply: tid=0 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb744aeb80 1 client.18597 shutdown 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb744aeb80 1 -- 172.21.15.90:0/3765407178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa483e40 msgr2=0x558baa606860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb744aeb80 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa483e40 0x558baa606860 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7feb5000a9e0 tx=0x7feb5000a410 comp rx=0 tx=0).stop 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb744aeb80 1 -- 172.21.15.90:0/3765407178 shutdown_connections 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb744aeb80 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.129:6832/3131589000,v1:172.21.15.129:6833/3131589000] conn(0x558baa6876d0 0x558baa689af0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb744aeb80 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558baa602e70 0x558baa613e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb744aeb80 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558baa483e40 0x558baa606860 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb744aeb80 1 --2- 172.21.15.90:0/3765407178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558baa56f9f0 0x558baa606da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb744aeb80 1 -- 172.21.15.90:0/3765407178 >> 172.21.15.90:0/3765407178 conn(0x558baa51d390 msgr2=0x558baa60c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.533+0000 7feb4cc00700 0 client.18597 ms_handle_reset on v2:172.21.15.129:6832/3131589000 2024-05-27T08:05:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.534+0000 7feb744aeb80 1 -- 172.21.15.90:0/3765407178 shutdown_connections 2024-05-27T08:05:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.534+0000 7feb744aeb80 1 -- 172.21.15.90:0/3765407178 wait complete. 2024-05-27T08:05:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.534+0000 7feb744aeb80 20 client.18597 trim_cache size 0 max 16384 2024-05-27T08:05:43.541 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:05:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.534+0000 7feb65a00700 1 -- 172.21.15.90:0/1001861251 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feb440b29f0 msgr2=0x7feb440b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.534+0000 7feb65a00700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feb440b29f0 0x7feb440b4f00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7feb50005ea0 tx=0x7feb50005e30 comp rx=0 tx=0).stop 2024-05-27T08:05:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.534+0000 7feb65a00700 1 -- 172.21.15.90:0/1001861251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb60076670 msgr2=0x7feb6019d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.534+0000 7feb65a00700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb60076670 0x7feb6019d4f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7feb480057f0 tx=0x7feb48009f90 comp rx=0 tx=0).stop 2024-05-27T08:05:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.535+0000 7feb65a00700 1 -- 172.21.15.90:0/1001861251 shutdown_connections 2024-05-27T08:05:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.535+0000 7feb65a00700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb60110b90 0x7feb601a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.535+0000 7feb65a00700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7feb440b29f0 0x7feb440b4f00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.535+0000 7feb65a00700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb60075d20 0x7feb6019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.535+0000 7feb65a00700 1 --2- 172.21.15.90:0/1001861251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb60076670 0x7feb6019d4f0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:43.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.535+0000 7feb65a00700 1 -- 172.21.15.90:0/1001861251 >> 172.21.15.90:0/1001861251 conn(0x7feb60100030 msgr2=0x7feb60100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:43.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.535+0000 7feb65a00700 1 -- 172.21.15.90:0/1001861251 shutdown_connections 2024-05-27T08:05:43.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:43.535+0000 7feb65a00700 1 -- 172.21.15.90:0/1001861251 wait complete. 2024-05-27T08:05:44.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:43 smithi129 ceph-mon[35254]: pgmap v622: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 806 KiB/s wr, 118 op/s 2024-05-27T08:05:44.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:43 smithi090 ceph-mon[31429]: pgmap v622: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 806 KiB/s wr, 118 op/s 2024-05-27T08:05:44.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:43 smithi136 ceph-mon[35324]: pgmap v622: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 806 KiB/s wr, 118 op/s 2024-05-27T08:05:44.416 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-27T08:05:44.852 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:46.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:45 smithi129 ceph-mon[35254]: pgmap v623: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 665 KiB/s wr, 97 op/s 2024-05-27T08:05:46.081 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:45 smithi090 ceph-mon[31429]: pgmap v623: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 665 KiB/s wr, 97 op/s 2024-05-27T08:05:46.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:45 smithi136 ceph-mon[35324]: pgmap v623: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 665 KiB/s wr, 97 op/s 2024-05-27T08:05:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.017+0000 7fdeffe00700 1 -- 172.21.15.90:0/994752664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf001025d0 msgr2=0x7fdf001029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.017+0000 7fdeffe00700 1 --2- 172.21.15.90:0/994752664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf001025d0 0x7fdf001029e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009990 tx=0x7fdee8009ca0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.017+0000 7fdeffe00700 1 -- 172.21.15.90:0/994752664 shutdown_connections 2024-05-27T08:05:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.017+0000 7fdeffe00700 1 --2- 172.21.15.90:0/994752664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf00102f20 0x7fdf00110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.017+0000 7fdeffe00700 1 --2- 172.21.15.90:0/994752664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf00109140 0x7fdf00109550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.017+0000 7fdeffe00700 1 --2- 172.21.15.90:0/994752664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf001025d0 0x7fdf001029e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.017+0000 7fdeffe00700 1 -- 172.21.15.90:0/994752664 >> 172.21.15.90:0/994752664 conn(0x7fdf000783a0 msgr2=0x7fdf000787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.017+0000 7fdeffe00700 1 -- 172.21.15.90:0/994752664 shutdown_connections 2024-05-27T08:05:47.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.017+0000 7fdeffe00700 1 -- 172.21.15.90:0/994752664 wait complete. 2024-05-27T08:05:47.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.018+0000 7fdeffe00700 1 Processor -- start 2024-05-27T08:05:47.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.019+0000 7fdeffe00700 1 -- start start 2024-05-27T08:05:47.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.019+0000 7fdeffe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf001025d0 0x7fdf0019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.019+0000 7fdeffe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf00102f20 0x7fdf0019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.019+0000 7fdeffe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf00109140 0x7fdf001a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.019+0000 7fdeffe00700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf001142e0 con 0x7fdf00102f20 2024-05-27T08:05:47.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.019+0000 7fdeffe00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf00114160 con 0x7fdf001025d0 2024-05-27T08:05:47.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.019+0000 7fdeffe00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdf00114460 con 0x7fdf00109140 2024-05-27T08:05:47.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdef6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf00102f20 0x7fdf0019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdef6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf00102f20 0x7fdf0019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48252/0 (socket says 172.21.15.90:48252) 2024-05-27T08:05:47.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdef6000700 1 -- 172.21.15.90:0/4107643720 learned_addr learned my addr 172.21.15.90:0/4107643720 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:47.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdefea00700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf001025d0 0x7fdf0019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdeff400700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf00109140 0x7fdf001a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdef6000700 1 -- 172.21.15.90:0/4107643720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf00109140 msgr2=0x7fdf001a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdef6000700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf00109140 0x7fdf001a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdef6000700 1 -- 172.21.15.90:0/4107643720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf001025d0 msgr2=0x7fdf0019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdef6000700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf001025d0 0x7fdf0019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdef6000700 1 -- 172.21.15.90:0/4107643720 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf001a1cb0 con 0x7fdf00102f20 2024-05-27T08:05:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.020+0000 7fdef6000700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf00102f20 0x7fdf0019d430 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fdee8000c00 tx=0x7fdee80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.021+0000 7fdf04a00700 1 -- 172.21.15.90:0/4107643720 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8027070 con 0x7fdf00102f20 2024-05-27T08:05:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.021+0000 7fdf04a00700 1 -- 172.21.15.90:0/4107643720 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdee800fc80 con 0x7fdf00102f20 2024-05-27T08:05:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.021+0000 7fdeffe00700 1 -- 172.21.15.90:0/4107643720 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf001a1f40 con 0x7fdf00102f20 2024-05-27T08:05:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.021+0000 7fdf04a00700 1 -- 172.21.15.90:0/4107643720 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee8022d30 con 0x7fdf00102f20 2024-05-27T08:05:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.021+0000 7fdeffe00700 1 -- 172.21.15.90:0/4107643720 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf001a2470 con 0x7fdf00102f20 2024-05-27T08:05:47.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.025+0000 7fdf04a00700 1 -- 172.21.15.90:0/4107643720 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdee802a070 con 0x7fdf00102f20 2024-05-27T08:05:47.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.026+0000 7fdf148f4b80 1 Processor -- start 2024-05-27T08:05:47.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.026+0000 7fdf148f4b80 1 -- start start 2024-05-27T08:05:47.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.027+0000 7fdf04a00700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdee40b2a70 0x7fdee40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.027+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b8ea9f0 0x55b56b97e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.027+0000 7fdf04a00700 1 -- 172.21.15.90:0/4107643720 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fdee80e3980 con 0x7fdf00102f20 2024-05-27T08:05:47.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.027+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b97ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.027+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b98a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.027+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b56b8abe40 con 0x55b56b9882c0 2024-05-27T08:05:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.027+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b56b89b9e0 con 0x55b56b7fee40 2024-05-27T08:05:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.027+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b56b89fa20 con 0x55b56b8ea9f0 2024-05-27T08:05:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.027+0000 7fdeff400700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdee40b2a70 0x7fdee40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdeff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b98a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdeff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b98a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48266/0 (socket says 172.21.15.90:48266) 2024-05-27T08:05:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdeff400700 1 -- 172.21.15.90:0/2455581269 learned_addr learned my addr 172.21.15.90:0/2455581269 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdef6000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b97ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdefea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b8ea9f0 0x55b56b97e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdeff400700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdee40b2a70 0x7fdee40b4f80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdef8012440 tx=0x7fdef8012800 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdeff400700 1 -- 172.21.15.90:0/2455581269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b8ea9f0 msgr2=0x55b56b97e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdeff400700 1 --2- 172.21.15.90:0/2455581269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b8ea9f0 0x55b56b97e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdeff400700 1 -- 172.21.15.90:0/2455581269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 msgr2=0x55b56b97ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdeff400700 1 --2- 172.21.15.90:0/2455581269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b97ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.028+0000 7fdeff400700 1 -- 172.21.15.90:0/2455581269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56b864420 con 0x55b56b9882c0 2024-05-27T08:05:47.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.029+0000 7fdeff400700 1 --2- 172.21.15.90:0/2455581269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b98a6e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fdef800c3e0 tx=0x7fdef801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7f5a7b3c213163 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.029+0000 7fdef4c00700 1 -- 172.21.15.90:0/2455581269 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8004020 con 0x55b56b9882c0 2024-05-27T08:05:47.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.029+0000 7fdef4c00700 1 -- 172.21.15.90:0/2455581269 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdef8003680 con 0x55b56b9882c0 2024-05-27T08:05:47.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.029+0000 7fdef4c00700 1 -- 172.21.15.90:0/2455581269 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef801e620 con 0x55b56b9882c0 2024-05-27T08:05:47.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.030+0000 7fdf148f4b80 1 -- 172.21.15.90:0/2455581269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 msgr2=0x55b56b98a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.030+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/2455581269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b98a6e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fdef800c3e0 tx=0x7fdef801a000 comp rx=0 tx=0).stop 2024-05-27T08:05:47.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.030+0000 7fdf148f4b80 1 -- 172.21.15.90:0/2455581269 shutdown_connections 2024-05-27T08:05:47.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.030+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/2455581269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b98a6e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.030+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/2455581269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b8ea9f0 0x55b56b97e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.030+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/2455581269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b97ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.030+0000 7fdf148f4b80 1 -- 172.21.15.90:0/2455581269 >> 172.21.15.90:0/2455581269 conn(0x55b56b898390 msgr2=0x55b56b979710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.030+0000 7fdf148f4b80 1 -- 172.21.15.90:0/2455581269 shutdown_connections 2024-05-27T08:05:47.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.030+0000 7fdf148f4b80 1 -- 172.21.15.90:0/2455581269 wait complete. 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.031+0000 7fdf148f4b80 1 -- start start 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.031+0000 7fdf148f4b80 1 Processor -- start 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.032+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b9851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.032+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b9856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.032+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b9882c0 0x55b56b985c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.032+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b56b7a1fa0 con 0x55b56b8ea9f0 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.032+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b56b83c760 con 0x55b56b7fee40 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.032+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b56b98ada0 con 0x55b56b9882c0 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.032+0000 7fdf148f4b80 10 client.0 _ll_register_callbacks cb 0x55b56b845c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.032+0000 7fdef6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b9856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.032+0000 7fdef6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b9856f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48280/0 (socket says 172.21.15.90:48280) 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.032+0000 7fdef6000700 1 -- 172.21.15.90:0/1298698735 learned_addr learned my addr 172.21.15.90:0/1298698735 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:47.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdefea00700 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b9851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdef6000700 1 -- 172.21.15.90:0/1298698735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b9882c0 msgr2=0x55b56b985c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdef6000700 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b9882c0 0x55b56b985c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdef6000700 1 -- 172.21.15.90:0/1298698735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 msgr2=0x55b56b9851b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdef6000700 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b9851b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdef6000700 1 -- 172.21.15.90:0/1298698735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56b7eb8d0 con 0x55b56b8ea9f0 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdef6000700 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b9856f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fdee80aa850 tx=0x7fdee8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d466ae8df127ecf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdeed600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdeed600700 1 -- 172.21.15.90:0/1298698735 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee80aa8c0 con 0x55b56b8ea9f0 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdeed600700 1 -- 172.21.15.90:0/1298698735 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fdee80a6eb0 con 0x55b56b8ea9f0 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.033+0000 7fdf148f4b80 1 -- 172.21.15.90:0/1298698735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdef8003680 con 0x55b56b8ea9f0 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.034+0000 7fdeed600700 1 -- 172.21.15.90:0/1298698735 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fdee80aec70 con 0x55b56b8ea9f0 2024-05-27T08:05:47.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.034+0000 7fdf148f4b80 10 client.18609 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:47.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.034+0000 7fdf148f4b80 1 -- 172.21.15.90:0/1298698735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b56b86c060 con 0x55b56b8ea9f0 2024-05-27T08:05:47.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.034+0000 7fdeed600700 1 -- 172.21.15.90:0/1298698735 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdef8003680 con 0x55b56b8ea9f0 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.034+0000 7fdeed600700 1 -- 172.21.15.90:0/1298698735 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fdee80b3290 con 0x55b56b8ea9f0 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.034+0000 7fdf148f4b80 10 client.18609 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.034+0000 7fdf148f4b80 10 client.18609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.034+0000 7fdf148f4b80 20 client.18609 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.035+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55b56b997b40 0x55b56b999fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.035+0000 7fdf148f4b80 4 client.18609 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.035+0000 7fdf148f4b80 1 -- 172.21.15.90:0/1298698735 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b56b8add90 con 0x55b56b997b40 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.035+0000 7fdefea00700 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55b56b997b40 0x55b56b999fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.036+0000 7fdefea00700 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55b56b997b40 0x55b56b999fa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e9a9b99f651298f server_cookie=efb04ce980557ffe in_seq=0 out_seq=0 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.036+0000 7fdeed600700 10 client.18609 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.038+0000 7fdeed600700 1 -- 172.21.15.90:0/1298698735 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x55b56b8add90 con 0x55b56b997b40 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.038+0000 7fdeed600700 10 client.18609 handle_command_reply: tid=0 2024-05-27T08:05:47.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.038+0000 7fdf148f4b80 1 client.18609 shutdown 2024-05-27T08:05:47.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.038+0000 7fdf148f4b80 1 -- 172.21.15.90:0/1298698735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 msgr2=0x55b56b9856f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.038+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b9856f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fdee80aa850 tx=0x7fdee8009ca0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.039+0000 7fdf148f4b80 1 -- 172.21.15.90:0/1298698735 shutdown_connections 2024-05-27T08:05:47.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.039+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55b56b997b40 0x55b56b999fa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.039+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b9882c0 0x55b56b985c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.039+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b9851b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.039+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/1298698735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b9856f0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.039+0000 7fdf148f4b80 1 -- 172.21.15.90:0/1298698735 >> 172.21.15.90:0/1298698735 conn(0x55b56b898390 msgr2=0x55b56b97e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.039+0000 7fdeed600700 0 client.18609 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T08:05:47.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.039+0000 7fdf148f4b80 1 -- 172.21.15.90:0/1298698735 shutdown_connections 2024-05-27T08:05:47.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.039+0000 7fdf148f4b80 1 -- 172.21.15.90:0/1298698735 wait complete. 2024-05-27T08:05:47.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.039+0000 7fdf148f4b80 20 client.18609 trim_cache size 0 max 16384 2024-05-27T08:05:47.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.056+0000 7fdf148f4b80 1 Processor -- start 2024-05-27T08:05:47.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.056+0000 7fdf148f4b80 1 -- start start 2024-05-27T08:05:47.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.056+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b979740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.056+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b8ea9f0 0x55b56b993e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.056+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b99f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.056+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf001b3e40 con 0x55b56b9882c0 2024-05-27T08:05:47.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.056+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf001b0e90 con 0x55b56b7fee40 2024-05-27T08:05:47.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.056+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdf001b30a0 con 0x55b56b8ea9f0 2024-05-27T08:05:47.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.057+0000 7fdeff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b99f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.057+0000 7fdeff400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b99f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48296/0 (socket says 172.21.15.90:48296) 2024-05-27T08:05:47.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.057+0000 7fdeff400700 1 -- 172.21.15.90:0/2874088518 learned_addr learned my addr 172.21.15.90:0/2874088518 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:47.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.057+0000 7fdefea00700 1 --2- 172.21.15.90:0/2874088518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b979740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.057+0000 7fdef6000700 1 --2- 172.21.15.90:0/2874088518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b8ea9f0 0x55b56b993e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.057+0000 7fdeff400700 1 -- 172.21.15.90:0/2874088518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b8ea9f0 msgr2=0x55b56b993e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.057+0000 7fdeff400700 1 --2- 172.21.15.90:0/2874088518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b8ea9f0 0x55b56b993e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.057+0000 7fdeff400700 1 -- 172.21.15.90:0/2874088518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 msgr2=0x55b56b979740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.057+0000 7fdeff400700 1 --2- 172.21.15.90:0/2874088518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b979740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.057+0000 7fdeff400700 1 -- 172.21.15.90:0/2874088518 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56b9880c0 con 0x55b56b9882c0 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.058+0000 7fdeff400700 1 --2- 172.21.15.90:0/2874088518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b99f760 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fdef800cdd0 tx=0x7fdef8005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9500b86070bb202 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.058+0000 7fdeefe00700 1 -- 172.21.15.90:0/2874088518 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8004950 con 0x55b56b9882c0 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.058+0000 7fdeefe00700 1 -- 172.21.15.90:0/2874088518 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdef800ca20 con 0x55b56b9882c0 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.058+0000 7fdeefe00700 1 -- 172.21.15.90:0/2874088518 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef801e680 con 0x55b56b9882c0 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.059+0000 7fdf148f4b80 1 -- 172.21.15.90:0/2874088518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 msgr2=0x55b56b99f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.059+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/2874088518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b99f760 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fdef800cdd0 tx=0x7fdef8005d90 comp rx=0 tx=0).stop 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.059+0000 7fdf148f4b80 1 -- 172.21.15.90:0/2874088518 shutdown_connections 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.059+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/2874088518 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b9882c0 0x55b56b99f760 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.059+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/2874088518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b979740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.059+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/2874088518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b8ea9f0 0x55b56b993e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.059+0000 7fdf148f4b80 1 -- 172.21.15.90:0/2874088518 >> 172.21.15.90:0/2874088518 conn(0x55b56b898390 msgr2=0x55b56b97e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.059+0000 7fdf148f4b80 1 -- 172.21.15.90:0/2874088518 shutdown_connections 2024-05-27T08:05:47.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.059+0000 7fdf148f4b80 1 -- 172.21.15.90:0/2874088518 wait complete. 2024-05-27T08:05:47.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.060+0000 7fdf148f4b80 1 -- start start 2024-05-27T08:05:47.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.060+0000 7fdf148f4b80 1 Processor -- start 2024-05-27T08:05:47.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b990800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b98b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdf148f4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b9882c0 0x55b56b98b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf001b29d0 con 0x55b56b8ea9f0 2024-05-27T08:05:47.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf001b1560 con 0x55b56b7fee40 2024-05-27T08:05:47.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdf148f4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdf001b1c30 con 0x55b56b9882c0 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdf148f4b80 10 client.0 _ll_register_callbacks cb 0x55b56b845c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdef6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b98b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdef6000700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b98b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48300/0 (socket says 172.21.15.90:48300) 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdef6000700 1 -- 172.21.15.90:0/760540499 learned_addr learned my addr 172.21.15.90:0/760540499 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdeff400700 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b9882c0 0x55b56b98b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdefea00700 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b990800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdef6000700 1 -- 172.21.15.90:0/760540499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b9882c0 msgr2=0x55b56b98b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdef6000700 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b9882c0 0x55b56b98b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdef6000700 1 -- 172.21.15.90:0/760540499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 msgr2=0x55b56b990800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdef6000700 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b990800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.061+0000 7fdef6000700 1 -- 172.21.15.90:0/760540499 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56b98bd80 con 0x55b56b8ea9f0 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.062+0000 7fdef6000700 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b98b020 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fdee80a7b30 tx=0x7fdee8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48ae5e703cf1447 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.062+0000 7fdeed600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.062+0000 7fdeed600700 1 -- 172.21.15.90:0/760540499 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee80aa8c0 con 0x55b56b8ea9f0 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.062+0000 7fdeed600700 1 -- 172.21.15.90:0/760540499 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fdee80a6eb0 con 0x55b56b8ea9f0 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.062+0000 7fdf148f4b80 1 -- 172.21.15.90:0/760540499 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdef800ca20 con 0x55b56b8ea9f0 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.062+0000 7fdeed600700 1 -- 172.21.15.90:0/760540499 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fdee80a6780 con 0x55b56b8ea9f0 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.062+0000 7fdf148f4b80 10 client.18615 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.062+0000 7fdf148f4b80 1 -- 172.21.15.90:0/760540499 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55b56ba09e00 con 0x55b56b8ea9f0 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.063+0000 7fdeed600700 1 -- 172.21.15.90:0/760540499 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdef800ca20 con 0x55b56b8ea9f0 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.063+0000 7fdeed600700 1 -- 172.21.15.90:0/760540499 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fdee80c4ba0 con 0x55b56b8ea9f0 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.063+0000 7fdf148f4b80 10 client.18615 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.063+0000 7fdf148f4b80 10 client.18615 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.063+0000 7fdf148f4b80 20 client.18615 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.063+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55b56ba0a3d0 0x55b56ba0c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.063+0000 7fdf148f4b80 4 client.18615 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:05:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.063+0000 7fdf148f4b80 1 -- 172.21.15.90:0/760540499 --> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b56b8a7570 con 0x55b56ba0a3d0 2024-05-27T08:05:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.064+0000 7fdefea00700 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55b56ba0a3d0 0x55b56ba0c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.064+0000 7fdefea00700 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55b56ba0a3d0 0x55b56ba0c7f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=542b1305df30eeab server_cookie=73f7ac5942cb5d92 in_seq=0 out_seq=0 2024-05-27T08:05:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.064+0000 7fdeed600700 10 client.18615 ms_handle_connect on v2:172.21.15.129:6834/1894603892 2024-05-27T08:05:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdeed600700 1 -- 172.21.15.90:0/760540499 <== mds.2 v2:172.21.15.129:6834/1894603892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b56b8a7570 con 0x55b56ba0a3d0 2024-05-27T08:05:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdeed600700 10 client.18615 handle_command_reply: tid=0 2024-05-27T08:05:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdf148f4b80 1 client.18615 shutdown 2024-05-27T08:05:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdf148f4b80 1 -- 172.21.15.90:0/760540499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 msgr2=0x55b56b98b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b98b020 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fdee80a7b30 tx=0x7fdee8005e80 comp rx=0 tx=0).stop 2024-05-27T08:05:47.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdf148f4b80 1 -- 172.21.15.90:0/760540499 shutdown_connections 2024-05-27T08:05:47.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.129:6834/1894603892,v1:172.21.15.129:6835/1894603892] conn(0x55b56ba0a3d0 0x55b56ba0c7f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b56b9882c0 0x55b56b98b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b56b7fee40 0x55b56b990800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdf148f4b80 1 --2- 172.21.15.90:0/760540499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b56b8ea9f0 0x55b56b98b020 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdf148f4b80 1 -- 172.21.15.90:0/760540499 >> 172.21.15.90:0/760540499 conn(0x55b56b898390 msgr2=0x55b56b97e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.065+0000 7fdeed600700 0 client.18615 ms_handle_reset on v2:172.21.15.129:6834/1894603892 2024-05-27T08:05:47.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.066+0000 7fdf148f4b80 1 -- 172.21.15.90:0/760540499 shutdown_connections 2024-05-27T08:05:47.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.066+0000 7fdf148f4b80 1 -- 172.21.15.90:0/760540499 wait complete. 2024-05-27T08:05:47.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.066+0000 7fdf148f4b80 20 client.18615 trim_cache size 0 max 16384 2024-05-27T08:05:47.070 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:05:47.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 -- 172.21.15.90:0/4107643720 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdee40b2a70 msgr2=0x7fdee40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdee40b2a70 0x7fdee40b4f80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdef8012440 tx=0x7fdef8012800 comp rx=0 tx=0).stop 2024-05-27T08:05:47.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 -- 172.21.15.90:0/4107643720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf00102f20 msgr2=0x7fdf0019d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:47.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf00102f20 0x7fdf0019d430 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fdee8000c00 tx=0x7fdee80036f0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 -- 172.21.15.90:0/4107643720 shutdown_connections 2024-05-27T08:05:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf00109140 0x7fdf001a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fdee40b2a70 0x7fdee40b4f80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf001025d0 0x7fdf0019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 --2- 172.21.15.90:0/4107643720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf00102f20 0x7fdf0019d430 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 -- 172.21.15.90:0/4107643720 >> 172.21.15.90:0/4107643720 conn(0x7fdf000783a0 msgr2=0x7fdf00110ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 -- 172.21.15.90:0/4107643720 shutdown_connections 2024-05-27T08:05:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:47.067+0000 7fdeffe00700 1 -- 172.21.15.90:0/4107643720 wait complete. 2024-05-27T08:05:48.018 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-27T08:05:48.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:47 smithi129 ceph-mon[35254]: pgmap v624: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 136 KiB/s wr, 25 op/s 2024-05-27T08:05:48.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:47 smithi090 ceph-mon[31429]: pgmap v624: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 136 KiB/s wr, 25 op/s 2024-05-27T08:05:48.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:47 smithi136 ceph-mon[35324]: pgmap v624: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 136 KiB/s wr, 25 op/s 2024-05-27T08:05:48.454 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:50.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:49 smithi129 ceph-mon[35254]: pgmap v625: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-05-27T08:05:50.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:49 smithi090 ceph-mon[31429]: pgmap v625: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-05-27T08:05:50.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:49 smithi136 ceph-mon[35324]: pgmap v625: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 20 KiB/s wr, 17 op/s 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.719+0000 7fc6c9400700 1 -- 172.21.15.90:0/276356588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c4109670 msgr2=0x7fc6c4110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.719+0000 7fc6c9400700 1 --2- 172.21.15.90:0/276356588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c4109670 0x7fc6c4110fd0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00b0d0 tx=0x7fc6bc00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.720+0000 7fc6c9400700 1 -- 172.21.15.90:0/276356588 shutdown_connections 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.720+0000 7fc6c9400700 1 --2- 172.21.15.90:0/276356588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c4109670 0x7fc6c4110fd0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.720+0000 7fc6c9400700 1 --2- 172.21.15.90:0/276356588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c41043f0 0x7fc6c4104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.720+0000 7fc6c9400700 1 --2- 172.21.15.90:0/276356588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c4108d20 0x7fc6c4109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.720+0000 7fc6c9400700 1 -- 172.21.15.90:0/276356588 >> 172.21.15.90:0/276356588 conn(0x7fc6c4079d00 msgr2=0x7fc6c407a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.720+0000 7fc6c9400700 1 -- 172.21.15.90:0/276356588 shutdown_connections 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.720+0000 7fc6c9400700 1 -- 172.21.15.90:0/276356588 wait complete. 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.721+0000 7fc6c9400700 1 Processor -- start 2024-05-27T08:05:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.721+0000 7fc6c9400700 1 -- start start 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.721+0000 7fc6c9400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c41043f0 0x7fc6c4075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.721+0000 7fc6c9400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c4108d20 0x7fc6c4075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.721+0000 7fc6c9400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c4109670 0x7fc6c4076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.721+0000 7fc6c9400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4114220 con 0x7fc6c41043f0 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.721+0000 7fc6c9400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41140a0 con 0x7fc6c4108d20 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.721+0000 7fc6c9400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41143a0 con 0x7fc6c4109670 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.722+0000 7fc6c3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c41043f0 0x7fc6c4075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.722+0000 7fc6c3e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c41043f0 0x7fc6c4075ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53430/0 (socket says 172.21.15.90:53430) 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.722+0000 7fc6c3e00700 1 -- 172.21.15.90:0/4066455431 learned_addr learned my addr 172.21.15.90:0/4066455431 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.722+0000 7fc6c3400700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c4108d20 0x7fc6c4075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.722+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c4109670 0x7fc6c4076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.722+0000 7fc6c3e00700 1 -- 172.21.15.90:0/4066455431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c4109670 msgr2=0x7fc6c4076530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.722+0000 7fc6c3e00700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c4109670 0x7fc6c4076530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.722+0000 7fc6c3e00700 1 -- 172.21.15.90:0/4066455431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c4108d20 msgr2=0x7fc6c4075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.722+0000 7fc6c3e00700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c4108d20 0x7fc6c4075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.722+0000 7fc6c3e00700 1 -- 172.21.15.90:0/4066455431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c41a8f80 con 0x7fc6c41043f0 2024-05-27T08:05:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.723+0000 7fc6c3e00700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c41043f0 0x7fc6c4075ab0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400b530 tx=0x7fc6b400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.723+0000 7fc6c0c00700 1 -- 172.21.15.90:0/4066455431 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4015020 con 0x7fc6c41043f0 2024-05-27T08:05:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.723+0000 7fc6c0c00700 1 -- 172.21.15.90:0/4066455431 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6b4011cc0 con 0x7fc6c41043f0 2024-05-27T08:05:50.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.723+0000 7fc6c9400700 1 -- 172.21.15.90:0/4066455431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c41a9270 con 0x7fc6c41043f0 2024-05-27T08:05:50.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.723+0000 7fc6c0c00700 1 -- 172.21.15.90:0/4066455431 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4010d30 con 0x7fc6c41043f0 2024-05-27T08:05:50.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.723+0000 7fc6c9400700 1 -- 172.21.15.90:0/4066455431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c41a97b0 con 0x7fc6c41043f0 2024-05-27T08:05:50.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.728+0000 7fc6c0c00700 1 -- 172.21.15.90:0/4066455431 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc6b4018020 con 0x7fc6c41043f0 2024-05-27T08:05:50.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.729+0000 7fc6d9532b80 1 Processor -- start 2024-05-27T08:05:50.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.729+0000 7fc6d9532b80 1 -- start start 2024-05-27T08:05:50.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.729+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.729+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a8f9410 0x56199a903ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.729+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 0x56199a906020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.729+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56199a827e40 con 0x56199a77ae40 2024-05-27T08:05:50.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.729+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56199a8179e0 con 0x56199a8f9410 2024-05-27T08:05:50.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.729+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56199a81ba20 con 0x56199a8f9820 2024-05-27T08:05:50.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8f8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53434/0 (socket says 172.21.15.90:53434) 2024-05-27T08:05:50.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c8a00700 1 -- 172.21.15.90:0/2961165442 learned_addr learned my addr 172.21.15.90:0/2961165442 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:50.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c3400700 1 --2- 172.21.15.90:0/2961165442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 0x56199a906020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c3e00700 1 --2- 172.21.15.90:0/2961165442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a8f9410 0x56199a903ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c8a00700 1 -- 172.21.15.90:0/2961165442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 msgr2=0x56199a906020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:50.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/2961165442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 0x56199a906020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c8a00700 1 -- 172.21.15.90:0/2961165442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a8f9410 msgr2=0x56199a903ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:50.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/2961165442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a8f9410 0x56199a903ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c8a00700 1 -- 172.21.15.90:0/2961165442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56199a7e0420 con 0x56199a77ae40 2024-05-27T08:05:50.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.730+0000 7fc6c3e00700 1 --2- 172.21.15.90:0/2961165442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a8f9410 0x56199a903ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:50.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.731+0000 7fc6c0c00700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6a80b29f0 0x7fc6a80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.731+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/2961165442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8f8ed0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009d60 tx=0x7fc6bc009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ad8123af3cd392 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:50.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.731+0000 7fc6c0c00700 1 -- 172.21.15.90:0/4066455431 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fc6b40e2120 con 0x7fc6c41043f0 2024-05-27T08:05:50.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.731+0000 7fc6c3e00700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6a80b29f0 0x7fc6a80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.731+0000 7fc6b9600700 1 -- 172.21.15.90:0/2961165442 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc020030 con 0x56199a77ae40 2024-05-27T08:05:50.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.731+0000 7fc6b9600700 1 -- 172.21.15.90:0/2961165442 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6bc0093f0 con 0x56199a77ae40 2024-05-27T08:05:50.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.731+0000 7fc6b9600700 1 -- 172.21.15.90:0/2961165442 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc005050 con 0x56199a77ae40 2024-05-27T08:05:50.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.732+0000 7fc6d9532b80 1 -- 172.21.15.90:0/2961165442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 msgr2=0x56199a8f8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:50.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.732+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/2961165442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8f8ed0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009d60 tx=0x7fc6bc009d90 comp rx=0 tx=0).stop 2024-05-27T08:05:50.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.732+0000 7fc6c3e00700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6a80b29f0 0x7fc6a80b4f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40df660 tx=0x7fc6b4004860 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:50.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.733+0000 7fc6d9532b80 1 -- 172.21.15.90:0/2961165442 shutdown_connections 2024-05-27T08:05:50.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.733+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/2961165442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 0x56199a906020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.733+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/2961165442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8f8ed0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.733+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/2961165442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a8f9410 0x56199a903ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.733+0000 7fc6d9532b80 1 -- 172.21.15.90:0/2961165442 >> 172.21.15.90:0/2961165442 conn(0x56199a7dafc0 msgr2=0x56199a8f4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:50.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.733+0000 7fc6d9532b80 1 -- 172.21.15.90:0/2961165442 shutdown_connections 2024-05-27T08:05:50.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.733+0000 7fc6d9532b80 1 -- 172.21.15.90:0/2961165442 wait complete. 2024-05-27T08:05:50.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.733+0000 7fc6d9532b80 1 -- start start 2024-05-27T08:05:50.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.734+0000 7fc6d9532b80 1 Processor -- start 2024-05-27T08:05:50.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.734+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a77ae40 0x56199a903570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.734+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 0x56199a903ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.734+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9820 0x56199a903ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.734+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56199a71dfa0 con 0x56199a8f9820 2024-05-27T08:05:50.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.734+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56199a7b8760 con 0x56199a77ae40 2024-05-27T08:05:50.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.734+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56199a906930 con 0x56199a8f9410 2024-05-27T08:05:50.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.734+0000 7fc6d9532b80 10 client.0 _ll_register_callbacks cb 0x56199a7c1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:50.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9820 0x56199a903ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9820 0x56199a903ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53440/0 (socket says 172.21.15.90:53440) 2024-05-27T08:05:50.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c8a00700 1 -- 172.21.15.90:0/3453995542 learned_addr learned my addr 172.21.15.90:0/3453995542 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:50.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c3400700 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 0x56199a903ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a77ae40 0x56199a903570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c8a00700 1 -- 172.21.15.90:0/3453995542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 msgr2=0x56199a903ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:50.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 0x56199a903ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c8a00700 1 -- 172.21.15.90:0/3453995542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a77ae40 msgr2=0x56199a903570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:50.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a77ae40 0x56199a903570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c8a00700 1 -- 172.21.15.90:0/3453995542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56199a7678d0 con 0x56199a8f9820 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9820 0x56199a903ff0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc006080 tx=0x7fc6bc0095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5274806bf365be65 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.735+0000 7fc6ae000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.736+0000 7fc6ae000700 1 -- 172.21.15.90:0/3453995542 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc020040 con 0x56199a8f9820 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.736+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3453995542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6bc0093f0 con 0x56199a8f9820 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.736+0000 7fc6ae000700 1 -- 172.21.15.90:0/3453995542 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fc6bc005230 con 0x56199a8f9820 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.736+0000 7fc6ae000700 1 -- 172.21.15.90:0/3453995542 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc6bc005430 con 0x56199a8f9820 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.736+0000 7fc6d9532b80 10 client.18627 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.736+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3453995542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56199a7e8060 con 0x56199a8f9820 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.736+0000 7fc6ae000700 1 -- 172.21.15.90:0/3453995542 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6bc0093f0 con 0x56199a8f9820 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.737+0000 7fc6ae000700 1 -- 172.21.15.90:0/3453995542 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fc6bc011070 con 0x56199a8f9820 2024-05-27T08:05:50.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.737+0000 7fc6d9532b80 10 client.18627 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:50.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.737+0000 7fc6d9532b80 10 client.18627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:05:50.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.737+0000 7fc6d9532b80 20 client.18627 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:50.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.737+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56199a9136a0 0x56199a915ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.737+0000 7fc6d9532b80 4 client.18627 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:50.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.737+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3453995542 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56199a829d90 con 0x56199a9136a0 2024-05-27T08:05:50.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.738+0000 7fc6c3400700 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56199a9136a0 0x56199a915ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.739+0000 7fc6c3400700 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56199a9136a0 0x56199a915ac0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e52b7f16809efd36 server_cookie=bdf5744483757957 in_seq=0 out_seq=0 2024-05-27T08:05:50.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.739+0000 7fc6ae000700 10 client.18627 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T08:05:50.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6ae000700 1 -- 172.21.15.90:0/3453995542 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x56199a829d90 con 0x56199a9136a0 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6ae000700 10 client.18627 handle_command_reply: tid=0 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6d9532b80 1 client.18627 shutdown 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3453995542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9820 msgr2=0x56199a903ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9820 0x56199a903ff0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc006080 tx=0x7fc6bc0095a0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3453995542 shutdown_connections 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9820 0x56199a903ff0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56199a9136a0 0x56199a915ac0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a77ae40 0x56199a903570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3453995542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 0x56199a903ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3453995542 >> 172.21.15.90:0/3453995542 conn(0x56199a7dafc0 msgr2=0x56199a8f3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.741+0000 7fc6ae000700 0 client.18627 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.742+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3453995542 shutdown_connections 2024-05-27T08:05:50.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.742+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3453995542 wait complete. 2024-05-27T08:05:50.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.742+0000 7fc6d9532b80 20 client.18627 trim_cache size 0 max 16384 2024-05-27T08:05:50.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.758+0000 7fc6d9532b80 1 Processor -- start 2024-05-27T08:05:50.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.758+0000 7fc6d9532b80 1 -- start start 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.759+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8fed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.759+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 0x56199a902af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.759+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a8f9820 0x56199a903ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.759+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b3f70 con 0x56199a77ae40 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.759+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b0fc0 con 0x56199a8f9820 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.759+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b31d0 con 0x56199a8f9410 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.759+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.759+0000 7fc6c8a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8fed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53450/0 (socket says 172.21.15.90:53450) 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.759+0000 7fc6c8a00700 1 -- 172.21.15.90:0/228282785 learned_addr learned my addr 172.21.15.90:0/228282785 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6c3400700 1 --2- 172.21.15.90:0/228282785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 0x56199a902af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6c8a00700 1 -- 172.21.15.90:0/228282785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 msgr2=0x56199a902af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:50.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/228282785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 0x56199a902af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6c8a00700 1 -- 172.21.15.90:0/228282785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a8f9820 msgr2=0x56199a903ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:50.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/228282785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a8f9820 0x56199a903ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6c8a00700 1 -- 172.21.15.90:0/228282785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56199a8f6350 con 0x56199a77ae40 2024-05-27T08:05:50.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6c3400700 1 --2- 172.21.15.90:0/228282785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 0x56199a902af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:50.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/228282785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8fed90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc01cc60 tx=0x7fc6bc00b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd03e867a3113523 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:50.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6b8c00700 1 -- 172.21.15.90:0/228282785 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc020070 con 0x56199a77ae40 2024-05-27T08:05:50.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6b8c00700 1 -- 172.21.15.90:0/228282785 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6bc0093f0 con 0x56199a77ae40 2024-05-27T08:05:50.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.760+0000 7fc6b8c00700 1 -- 172.21.15.90:0/228282785 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc0077d0 con 0x56199a77ae40 2024-05-27T08:05:50.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.761+0000 7fc6d9532b80 1 -- 172.21.15.90:0/228282785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 msgr2=0x56199a8fed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:50.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.761+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/228282785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8fed90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc01cc60 tx=0x7fc6bc00b050 comp rx=0 tx=0).stop 2024-05-27T08:05:50.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.761+0000 7fc6d9532b80 1 -- 172.21.15.90:0/228282785 shutdown_connections 2024-05-27T08:05:50.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.761+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/228282785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a8f9820 0x56199a903ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.761+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/228282785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a77ae40 0x56199a8fed90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.761+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/228282785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9410 0x56199a902af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.761+0000 7fc6d9532b80 1 -- 172.21.15.90:0/228282785 >> 172.21.15.90:0/228282785 conn(0x56199a7dafc0 msgr2=0x56199a8fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:50.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.761+0000 7fc6d9532b80 1 -- 172.21.15.90:0/228282785 shutdown_connections 2024-05-27T08:05:50.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.761+0000 7fc6d9532b80 1 -- 172.21.15.90:0/228282785 wait complete. 2024-05-27T08:05:50.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.762+0000 7fc6d9532b80 1 -- start start 2024-05-27T08:05:50.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.762+0000 7fc6d9532b80 1 Processor -- start 2024-05-27T08:05:50.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a77ae40 0x56199a976350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9410 0x56199a9768f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6d9532b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 0x56199a907160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b2b00 con 0x56199a8f9410 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b1690 con 0x56199a77ae40 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6d9532b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c41b1d60 con 0x56199a8f9820 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6d9532b80 10 client.0 _ll_register_callbacks cb 0x56199a7c1c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6c3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9410 0x56199a9768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6c3400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9410 0x56199a9768f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53460/0 (socket says 172.21.15.90:53460) 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6c3400700 1 -- 172.21.15.90:0/3866101614 learned_addr learned my addr 172.21.15.90:0/3866101614 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.763+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a77ae40 0x56199a976350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 0x56199a907160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6c3400700 1 -- 172.21.15.90:0/3866101614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 msgr2=0x56199a907160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6c3400700 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 0x56199a907160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6c3400700 1 -- 172.21.15.90:0/3866101614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a77ae40 msgr2=0x56199a976350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6c3400700 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a77ae40 0x56199a976350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6c3400700 1 -- 172.21.15.90:0/3866101614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56199a907810 con 0x56199a8f9410 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 0x56199a907160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6c3400700 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9410 0x56199a9768f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0005950 tx=0x7fc6b0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03f64de42f47a54 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6ae000700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6ae000700 1 -- 172.21.15.90:0/3866101614 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b001a6d0 con 0x56199a8f9410 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6ae000700 1 -- 172.21.15.90:0/3866101614 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fc6b001a870 con 0x56199a8f9410 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.764+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3866101614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6bc0093f0 con 0x56199a8f9410 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6ae000700 1 -- 172.21.15.90:0/3866101614 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fc6b002b050 con 0x56199a8f9410 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6d9532b80 10 client.18633 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3866101614 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x56199a97d9c0 con 0x56199a8f9410 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6ae000700 1 -- 172.21.15.90:0/3866101614 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc6bc0093f0 con 0x56199a8f9410 2024-05-27T08:05:50.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6ae000700 1 -- 172.21.15.90:0/3866101614 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7fc6b0023cd0 con 0x56199a8f9410 2024-05-27T08:05:50.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6d9532b80 10 client.18633 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:50.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6d9532b80 10 client.18633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-27T08:05:50.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6d9532b80 20 client.18633 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:50.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56199a97e150 0x56199a980570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:50.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6d9532b80 4 client.18633 mds_command: new command op to 34442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:05:50.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.765+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3866101614 --> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56199a823570 con 0x56199a97e150 2024-05-27T08:05:50.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.766+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56199a97e150 0x56199a980570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:50.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.767+0000 7fc6c8a00700 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56199a97e150 0x56199a980570 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5880b8c9f3c629c5 server_cookie=e8c741b9403f306 in_seq=0 out_seq=0 2024-05-27T08:05:50.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.767+0000 7fc6ae000700 10 client.18633 ms_handle_connect on v2:172.21.15.129:6836/2346696366 2024-05-27T08:05:50.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6ae000700 1 -- 172.21.15.90:0/3866101614 <== mds.1 v2:172.21.15.129:6836/2346696366 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56199a823570 con 0x56199a97e150 2024-05-27T08:05:50.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6ae000700 10 client.18633 handle_command_reply: tid=0 2024-05-27T08:05:50.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 client.18633 shutdown 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3866101614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9410 msgr2=0x56199a9768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9410 0x56199a9768f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0005950 tx=0x7fc6b0011a50 comp rx=0 tx=0).stop 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3866101614 shutdown_connections 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56199a8f9820 0x56199a907160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.129:6836/2346696366,v1:172.21.15.129:6837/2346696366] conn(0x56199a97e150 0x56199a980570 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56199a77ae40 0x56199a976350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 --2- 172.21.15.90:0/3866101614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56199a8f9410 0x56199a9768f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3866101614 >> 172.21.15.90:0/3866101614 conn(0x56199a7dafc0 msgr2=0x56199a901ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6ae000700 0 client.18633 ms_handle_reset on v2:172.21.15.129:6836/2346696366 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3866101614 shutdown_connections 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 1 -- 172.21.15.90:0/3866101614 wait complete. 2024-05-27T08:05:50.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.768+0000 7fc6d9532b80 20 client.18633 trim_cache size 0 max 16384 2024-05-27T08:05:50.772 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:05:50.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 -- 172.21.15.90:0/4066455431 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6a80b29f0 msgr2=0x7fc6a80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:50.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6a80b29f0 0x7fc6a80b4f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40df660 tx=0x7fc6b4004860 comp rx=0 tx=0).stop 2024-05-27T08:05:50.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 -- 172.21.15.90:0/4066455431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c41043f0 msgr2=0x7fc6c4075ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:50.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c41043f0 0x7fc6c4075ab0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400b530 tx=0x7fc6b400b8f0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 -- 172.21.15.90:0/4066455431 shutdown_connections 2024-05-27T08:05:50.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c4109670 0x7fc6c4076530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6c41043f0 0x7fc6c4075ab0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc6a80b29f0 0x7fc6a80b4f00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 --2- 172.21.15.90:0/4066455431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc6c4108d20 0x7fc6c4075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:50.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 -- 172.21.15.90:0/4066455431 >> 172.21.15.90:0/4066455431 conn(0x7fc6c4079d00 msgr2=0x7fc6c41015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:50.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 -- 172.21.15.90:0/4066455431 shutdown_connections 2024-05-27T08:05:50.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:50.769+0000 7fc6c9400700 1 -- 172.21.15.90:0/4066455431 wait complete. 2024-05-27T08:05:51.203 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:50 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:05:51.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:50 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:05:51.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:50 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:05:51.648 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-27T08:05:51.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:51 smithi090 ceph-mon[31429]: pgmap v626: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-05-27T08:05:52.068 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:05:52.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:51 smithi136 ceph-mon[35324]: pgmap v626: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-05-27T08:05:52.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:51 smithi129 ceph-mon[35254]: pgmap v626: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-05-27T08:05:53.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:52 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:05:53.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:52 smithi090 ceph-mon[31429]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:05:53.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:52 smithi090 ceph-mon[31429]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:05:53.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:52 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:05:53.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:52 smithi136 ceph-mon[35324]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:05:53.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:52 smithi136 ceph-mon[35324]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:05:53.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:52 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:05:53.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:52 smithi129 ceph-mon[35254]: from='mgr.34967 172.21.15.90:0/2837309187' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:05:53.316 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:52 smithi129 ceph-mon[35254]: from='mgr.34967 ' entity='mgr.x' 2024-05-27T08:05:54.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:53 smithi090 ceph-mon[31429]: pgmap v627: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-05-27T08:05:54.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:53 smithi090 ceph-mon[31429]: pgmap v628: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-05-27T08:05:54.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:53 smithi136 ceph-mon[35324]: pgmap v627: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-05-27T08:05:54.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:53 smithi136 ceph-mon[35324]: pgmap v628: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-05-27T08:05:54.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:53 smithi129 ceph-mon[35254]: pgmap v627: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-05-27T08:05:54.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:53 smithi129 ceph-mon[35254]: pgmap v628: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-05-27T08:05:54.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.577+0000 7f7886400700 1 -- 172.21.15.90:0/3459147998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7880108f10 msgr2=0x7f7880109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:54.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.577+0000 7f7886400700 1 --2- 172.21.15.90:0/3459147998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7880108f10 0x7f7880109320 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f78680099c0 tx=0x7f7868009cd0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.577+0000 7f7886400700 1 -- 172.21.15.90:0/3459147998 shutdown_connections 2024-05-27T08:05:54.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.577+0000 7f7886400700 1 --2- 172.21.15.90:0/3459147998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7880109860 0x7f78801111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.577+0000 7f7886400700 1 --2- 172.21.15.90:0/3459147998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78801045e0 0x7f78801049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.577+0000 7f7886400700 1 --2- 172.21.15.90:0/3459147998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7880108f10 0x7f7880109320 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.577+0000 7f7886400700 1 -- 172.21.15.90:0/3459147998 >> 172.21.15.90:0/3459147998 conn(0x7f7880100030 msgr2=0x7f7880102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:54.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.577+0000 7f7886400700 1 -- 172.21.15.90:0/3459147998 shutdown_connections 2024-05-27T08:05:54.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.577+0000 7f7886400700 1 -- 172.21.15.90:0/3459147998 wait complete. 2024-05-27T08:05:54.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.579+0000 7f7886400700 1 Processor -- start 2024-05-27T08:05:54.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.579+0000 7f7886400700 1 -- start start 2024-05-27T08:05:54.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.579+0000 7f7886400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78801045e0 0x7f7880075ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.579+0000 7f7886400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7880108f10 0x7f7880075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.579+0000 7f7886400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7880109860 0x7f7880076530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.579+0000 7f7886400700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7880114370 con 0x7f7880109860 2024-05-27T08:05:54.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.579+0000 7f7886400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78801141f0 con 0x7f78801045e0 2024-05-27T08:05:54.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.579+0000 7f7886400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78801144f0 con 0x7f7880108f10 2024-05-27T08:05:54.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.580+0000 7f787fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7880109860 0x7f7880076530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.580+0000 7f787fe00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7880109860 0x7f7880076530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53484/0 (socket says 172.21.15.90:53484) 2024-05-27T08:05:54.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.580+0000 7f787fe00700 1 -- 172.21.15.90:0/2180688515 learned_addr learned my addr 172.21.15.90:0/2180688515 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:54.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.580+0000 7f787f400700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78801045e0 0x7f7880075ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.580+0000 7f7876a00700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7880108f10 0x7f7880075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.580+0000 7f787fe00700 1 -- 172.21.15.90:0/2180688515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7880108f10 msgr2=0x7f7880075ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:54.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.580+0000 7f787fe00700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7880108f10 0x7f7880075ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.580+0000 7f787fe00700 1 -- 172.21.15.90:0/2180688515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78801045e0 msgr2=0x7f7880075ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:54.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.580+0000 7f787fe00700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78801045e0 0x7f7880075ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.580+0000 7f787fe00700 1 -- 172.21.15.90:0/2180688515 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78801a90f0 con 0x7f7880109860 2024-05-27T08:05:54.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.581+0000 7f787fe00700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7880109860 0x7f7880076530 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f787800c900 tx=0x7f787800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:54.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.581+0000 7f787cc00700 1 -- 172.21.15.90:0/2180688515 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78780078d0 con 0x7f7880109860 2024-05-27T08:05:54.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.581+0000 7f787cc00700 1 -- 172.21.15.90:0/2180688515 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f787800f450 con 0x7f7880109860 2024-05-27T08:05:54.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.581+0000 7f787cc00700 1 -- 172.21.15.90:0/2180688515 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878018660 con 0x7f7880109860 2024-05-27T08:05:54.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.581+0000 7f7886400700 1 -- 172.21.15.90:0/2180688515 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78801a93e0 con 0x7f7880109860 2024-05-27T08:05:54.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.581+0000 7f7886400700 1 -- 172.21.15.90:0/2180688515 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78801a98a0 con 0x7f7880109860 2024-05-27T08:05:54.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.585+0000 7f787cc00700 1 -- 172.21.15.90:0/2180688515 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7878007a70 con 0x7f7880109860 2024-05-27T08:05:54.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.587+0000 7f7894f5eb80 1 Processor -- start 2024-05-27T08:05:54.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.587+0000 7f7894f5eb80 1 -- start start 2024-05-27T08:05:54.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.588+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x55806993d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.588+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55806993dd30 0x558069937870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.588+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558069937db0 0x5580699381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.588+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558069865e40 con 0x55806993dd30 2024-05-27T08:05:54.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.588+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5580698559e0 con 0x5580697b8e40 2024-05-27T08:05:54.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.588+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558069859a20 con 0x558069937db0 2024-05-27T08:05:54.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.588+0000 7f787cc00700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f78640b29f0 0x7f78640b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.588+0000 7f787f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55806993dd30 0x558069937870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.588+0000 7f787cc00700 1 -- 172.21.15.90:0/2180688515 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f78780e2830 con 0x7f7880109860 2024-05-27T08:05:54.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f787f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55806993dd30 0x558069937870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53496/0 (socket says 172.21.15.90:53496) 2024-05-27T08:05:54.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f787f400700 1 -- 172.21.15.90:0/1581063725 learned_addr learned my addr 172.21.15.90:0/1581063725 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:54.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f787fe00700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f78640b29f0 0x7f78640b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f7876a00700 1 --2- 172.21.15.90:0/1581063725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x55806993d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f7876a00700 1 --2- 172.21.15.90:0/1581063725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558069937db0 0x5580699381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f787f400700 1 -- 172.21.15.90:0/1581063725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558069937db0 msgr2=0x5580699381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f787f400700 1 --2- 172.21.15.90:0/1581063725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558069937db0 0x5580699381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f787f400700 1 -- 172.21.15.90:0/1581063725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 msgr2=0x55806993d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f787f400700 1 --2- 172.21.15.90:0/1581063725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x55806993d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f787f400700 1 -- 172.21.15.90:0/1581063725 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806981e420 con 0x55806993dd30 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f7876a00700 1 --2- 172.21.15.90:0/1581063725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558069937db0 0x5580699381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.589+0000 7f787fe00700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f78640b29f0 0x7f78640b4f00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f78780a5760 tx=0x7f7878005990 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.590+0000 7f787f400700 1 --2- 172.21.15.90:0/1581063725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55806993dd30 0x558069937870 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7870009990 tx=0x7f7870009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9728ffddafa8a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.590+0000 7f7874c00700 1 -- 172.21.15.90:0/1581063725 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870004780 con 0x55806993dd30 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.590+0000 7f7874c00700 1 -- 172.21.15.90:0/1581063725 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f787001a070 con 0x55806993dd30 2024-05-27T08:05:54.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.590+0000 7f7874c00700 1 -- 172.21.15.90:0/1581063725 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78700156e0 con 0x55806993dd30 2024-05-27T08:05:54.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.591+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1581063725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55806993dd30 msgr2=0x558069937870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:54.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.591+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/1581063725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55806993dd30 0x558069937870 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7870009990 tx=0x7f7870009ca0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.591+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1581063725 shutdown_connections 2024-05-27T08:05:54.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.591+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/1581063725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558069937db0 0x5580699381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.591+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/1581063725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x55806993d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.591+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/1581063725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55806993dd30 0x558069937870 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.591+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1581063725 >> 172.21.15.90:0/1581063725 conn(0x558069818fc0 msgr2=0x5580699328d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:54.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.591+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1581063725 shutdown_connections 2024-05-27T08:05:54.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.591+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1581063725 wait complete. 2024-05-27T08:05:54.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.592+0000 7f7894f5eb80 1 -- start start 2024-05-27T08:05:54.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.592+0000 7f7894f5eb80 1 Processor -- start 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.593+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x5580699414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.593+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x5580699419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.593+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x558069941f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.593+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55806975bfa0 con 0x558069937db0 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.593+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5580697f6760 con 0x5580697b8e40 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.593+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558069944800 con 0x55806993dd30 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.593+0000 7f7894f5eb80 10 client.0 _ll_register_callbacks cb 0x5580697ffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.593+0000 7f787f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x5580699419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.593+0000 7f787f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x5580699419e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53512/0 (socket says 172.21.15.90:53512) 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.593+0000 7f787f400700 1 -- 172.21.15.90:0/1778747320 learned_addr learned my addr 172.21.15.90:0/1778747320 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:54.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f7876a00700 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x5580699414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f7876a00700 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x558069941f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f787f400700 1 -- 172.21.15.90:0/1778747320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 msgr2=0x558069941f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f787f400700 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x558069941f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f787f400700 1 -- 172.21.15.90:0/1778747320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 msgr2=0x5580699414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f787f400700 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x5580699414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f787f400700 1 -- 172.21.15.90:0/1778747320 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580697a58d0 con 0x558069937db0 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f7876a00700 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x558069941f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f787f400700 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x5580699419e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7870004a60 tx=0x7f7870007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e689d8252d9bd41 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f786d600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.594+0000 7f786d600700 1 -- 172.21.15.90:0/1778747320 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870023070 con 0x558069937db0 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.595+0000 7f786d600700 1 -- 172.21.15.90:0/1778747320 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f7870015480 con 0x558069937db0 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.595+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1778747320 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f787001a070 con 0x558069937db0 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.595+0000 7f786d600700 1 -- 172.21.15.90:0/1778747320 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f787002f050 con 0x558069937db0 2024-05-27T08:05:54.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.595+0000 7f7894f5eb80 10 client.18645 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:54.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.595+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1778747320 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x558069826060 con 0x558069937db0 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.595+0000 7f786d600700 1 -- 172.21.15.90:0/1778747320 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f787001a070 con 0x558069937db0 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.595+0000 7f786d600700 1 -- 172.21.15.90:0/1778747320 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f78700051e0 con 0x558069937db0 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.595+0000 7f7894f5eb80 10 client.18645 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.595+0000 7f7894f5eb80 10 client.18645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.596+0000 7f7894f5eb80 20 client.18645 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.596+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558069951610 0x558069953ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.596+0000 7f7894f5eb80 4 client.18645 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.596+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1778747320 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558069867d90 con 0x558069951610 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.596+0000 7f7876a00700 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558069951610 0x558069953ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.598+0000 7f7876a00700 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558069951610 0x558069953ab0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c5cf8cf26533be server_cookie=468d607cbd3ee874 in_seq=0 out_seq=0 2024-05-27T08:05:54.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.598+0000 7f786d600700 10 client.18645 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:54.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f786d600700 1 -- 172.21.15.90:0/1778747320 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24041 (crc 0 0 0) 0x558069867d90 con 0x558069951610 2024-05-27T08:05:54.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f786d600700 10 client.18645 handle_command_reply: tid=0 2024-05-27T08:05:54.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f7894f5eb80 1 client.18645 shutdown 2024-05-27T08:05:54.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1778747320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 msgr2=0x5580699419e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:54.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x5580699419e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7870004a60 tx=0x7f7870007860 comp rx=0 tx=0).stop 2024-05-27T08:05:54.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1778747320 shutdown_connections 2024-05-27T08:05:54.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x558069951610 0x558069953ab0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x558069941f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x5580699414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/1778747320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x5580699419e0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1778747320 >> 172.21.15.90:0/1778747320 conn(0x558069818fc0 msgr2=0x558069932580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:54.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.600+0000 7f786d600700 0 client.18645 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:54.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.601+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1778747320 shutdown_connections 2024-05-27T08:05:54.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.601+0000 7f7894f5eb80 1 -- 172.21.15.90:0/1778747320 wait complete. 2024-05-27T08:05:54.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.601+0000 7f7894f5eb80 20 client.18645 trim_cache size 0 max 16384 2024-05-27T08:05:54.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.617+0000 7f7894f5eb80 1 Processor -- start 2024-05-27T08:05:54.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.617+0000 7f7894f5eb80 1 -- start start 2024-05-27T08:05:54.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580697b8e40 0x558069937870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558069937db0 0x55806993cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x55806994d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f78801b41d0 con 0x5580697b8e40 2024-05-27T08:05:54.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78801b1220 con 0x558069937db0 2024-05-27T08:05:54.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78801b3430 con 0x55806993dd30 2024-05-27T08:05:54.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7876a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580697b8e40 0x558069937870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7876a00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580697b8e40 0x558069937870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53514/0 (socket says 172.21.15.90:53514) 2024-05-27T08:05:54.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7876a00700 1 -- 172.21.15.90:0/689216040 learned_addr learned my addr 172.21.15.90:0/689216040 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:54.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f787f400700 1 --2- 172.21.15.90:0/689216040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558069937db0 0x55806993cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7876a00700 1 -- 172.21.15.90:0/689216040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 msgr2=0x55806994d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:54.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7876a00700 1 --2- 172.21.15.90:0/689216040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x55806994d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7876a00700 1 -- 172.21.15.90:0/689216040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558069937db0 msgr2=0x55806993cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:54.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.618+0000 7f7876a00700 1 --2- 172.21.15.90:0/689216040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558069937db0 0x55806993cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.619+0000 7f7876a00700 1 -- 172.21.15.90:0/689216040 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558069934300 con 0x5580697b8e40 2024-05-27T08:05:54.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.619+0000 7f7876a00700 1 --2- 172.21.15.90:0/689216040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580697b8e40 0x558069937870 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7868009990 tx=0x7f7868009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcf3470c5bf037 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:54.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.619+0000 7f786fe00700 1 -- 172.21.15.90:0/689216040 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786800b950 con 0x5580697b8e40 2024-05-27T08:05:54.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.619+0000 7f786fe00700 1 -- 172.21.15.90:0/689216040 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7868003680 con 0x5580697b8e40 2024-05-27T08:05:54.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.619+0000 7f786fe00700 1 -- 172.21.15.90:0/689216040 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786800f6c0 con 0x5580697b8e40 2024-05-27T08:05:54.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.620+0000 7f7894f5eb80 1 -- 172.21.15.90:0/689216040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580697b8e40 msgr2=0x558069937870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:54.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.620+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/689216040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580697b8e40 0x558069937870 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7868009990 tx=0x7f7868009310 comp rx=0 tx=0).stop 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.620+0000 7f7894f5eb80 1 -- 172.21.15.90:0/689216040 shutdown_connections 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.620+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/689216040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x55806994d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.620+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/689216040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5580697b8e40 0x558069937870 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.620+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/689216040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558069937db0 0x55806993cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.620+0000 7f7894f5eb80 1 -- 172.21.15.90:0/689216040 >> 172.21.15.90:0/689216040 conn(0x558069818fc0 msgr2=0x55806993fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.620+0000 7f7894f5eb80 1 -- 172.21.15.90:0/689216040 shutdown_connections 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.620+0000 7f7894f5eb80 1 -- 172.21.15.90:0/689216040 wait complete. 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.621+0000 7f7894f5eb80 1 -- start start 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.621+0000 7f7894f5eb80 1 Processor -- start 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x558069949a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x558069949f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f7894f5eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x558069958260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f78801b2d60 con 0x558069937db0 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f78801b18f0 con 0x5580697b8e40 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f7894f5eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78801b1fc0 con 0x55806993dd30 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f7894f5eb80 10 client.0 _ll_register_callbacks cb 0x5580697ffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f787f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x558069949f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f787f400700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x558069949f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53518/0 (socket says 172.21.15.90:53518) 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f787f400700 1 -- 172.21.15.90:0/928900456 learned_addr learned my addr 172.21.15.90:0/928900456 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f7876a00700 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x558069949a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f7876a00700 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x558069958260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f787f400700 1 -- 172.21.15.90:0/928900456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 msgr2=0x558069958260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.622+0000 7f787f400700 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x558069958260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f787f400700 1 -- 172.21.15.90:0/928900456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 msgr2=0x558069949a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:05:54.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f787f400700 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x558069949a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f787f400700 1 -- 172.21.15.90:0/928900456 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558069959c30 con 0x558069937db0 2024-05-27T08:05:54.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f7876a00700 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x558069949a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:54.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f787f400700 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x558069949f90 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7870000f80 tx=0x7f78700093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bdbe4af93d2cc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:54.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f786d600700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-05-27T08:05:54.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f786d600700 1 -- 172.21.15.90:0/928900456 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870023040 con 0x558069937db0 2024-05-27T08:05:54.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f7894f5eb80 1 -- 172.21.15.90:0/928900456 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7868003680 con 0x558069937db0 2024-05-27T08:05:54.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f786d600700 1 -- 172.21.15.90:0/928900456 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f7870007c60 con 0x558069937db0 2024-05-27T08:05:54.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f786d600700 1 -- 172.21.15.90:0/928900456 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7f7870007e60 con 0x558069937db0 2024-05-27T08:05:54.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f7894f5eb80 10 client.18651 fetch_fsmap learned FSMap version 64 2024-05-27T08:05:54.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.623+0000 7f7894f5eb80 1 -- 172.21.15.90:0/928900456 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x5580699bb4c0 con 0x558069937db0 2024-05-27T08:05:54.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.624+0000 7f786d600700 1 -- 172.21.15.90:0/928900456 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7868003680 con 0x558069937db0 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.624+0000 7f786d600700 1 -- 172.21.15.90:0/928900456 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 64) ==== 6559+0+0 (secure 0 0 0) 0x7f7870004ce0 con 0x558069937db0 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.624+0000 7f7894f5eb80 10 client.18651 fetch_fsmap finished waiting for FSMap version 64 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.624+0000 7f7894f5eb80 10 client.18651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.624+0000 7f7894f5eb80 20 client.18651 populate_metadata read hostname 'smithi090' 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.624+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580699bbae0 0x5580699bdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.624+0000 7f7894f5eb80 4 client.18651 mds_command: new command op to 34472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.624+0000 7f7894f5eb80 1 -- 172.21.15.90:0/928900456 --> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558069861570 con 0x5580699bbae0 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.625+0000 7f7876a00700 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580699bbae0 0x5580699bdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.626+0000 7f786d600700 10 client.18651 ms_handle_connect on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.626+0000 7f7876a00700 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580699bbae0 0x5580699bdf00 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=820a51355e8be940 server_cookie=6a688efc4365fe3c in_seq=0 out_seq=0 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.626+0000 7f786d600700 1 -- 172.21.15.90:0/928900456 <== mds.0 v2:172.21.15.129:6838/2893900972 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558069861570 con 0x5580699bbae0 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.626+0000 7f786d600700 10 client.18651 handle_command_reply: tid=0 2024-05-27T08:05:54.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.626+0000 7f7894f5eb80 1 client.18651 shutdown 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.626+0000 7f7894f5eb80 1 -- 172.21.15.90:0/928900456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 msgr2=0x558069949f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.626+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x558069949f90 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7870000f80 tx=0x7f78700093d0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.627+0000 7f7894f5eb80 1 -- 172.21.15.90:0/928900456 shutdown_connections 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.627+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.129:6838/2893900972,v1:172.21.15.129:6839/2893900972] conn(0x5580699bbae0 0x5580699bdf00 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.627+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55806993dd30 0x558069958260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.627+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580697b8e40 0x558069949a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.627+0000 7f7894f5eb80 1 --2- 172.21.15.90:0/928900456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558069937db0 0x558069949f90 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.627+0000 7f786d600700 0 client.18651 ms_handle_reset on v2:172.21.15.129:6838/2893900972 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.627+0000 7f7894f5eb80 1 -- 172.21.15.90:0/928900456 >> 172.21.15.90:0/928900456 conn(0x558069818fc0 msgr2=0x55806993fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.627+0000 7f7894f5eb80 1 -- 172.21.15.90:0/928900456 shutdown_connections 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.627+0000 7f7894f5eb80 1 -- 172.21.15.90:0/928900456 wait complete. 2024-05-27T08:05:54.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.627+0000 7f7894f5eb80 20 client.18651 trim_cache size 0 max 16384 2024-05-27T08:05:54.631 INFO:teuthology.orchestra.run.smithi090.stdout:[] 2024-05-27T08:05:54.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 -- 172.21.15.90:0/2180688515 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f78640b29f0 msgr2=0x7f78640b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f78640b29f0 0x7f78640b4f00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f78780a5760 tx=0x7f7878005990 comp rx=0 tx=0).stop 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 -- 172.21.15.90:0/2180688515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7880109860 msgr2=0x7f7880076530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7880109860 0x7f7880076530 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f787800c900 tx=0x7f787800cc10 comp rx=0 tx=0).stop 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 -- 172.21.15.90:0/2180688515 shutdown_connections 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7880109860 0x7f7880076530 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f78640b29f0 0x7f78640b4f00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78801045e0 0x7f7880075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 --2- 172.21.15.90:0/2180688515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7880108f10 0x7f7880075ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 -- 172.21.15.90:0/2180688515 >> 172.21.15.90:0/2180688515 conn(0x7f7880100030 msgr2=0x7f788010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 -- 172.21.15.90:0/2180688515 shutdown_connections 2024-05-27T08:05:54.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:54.628+0000 7f7886400700 1 -- 172.21.15.90:0/2180688515 wait complete. 2024-05-27T08:05:55.719 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-27T08:05:55.720 INFO:tasks.fwd_scrub:done joining 2024-05-27T08:05:55.720 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-27T08:05:55.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T08:05:55.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:55 smithi090 ceph-mon[31429]: pgmap v629: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-05-27T08:05:56.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.030+0000 7f7ad9c00640 1 -- 172.21.15.90:0/3251182129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad4112810 msgr2=0x7f7ad4118e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:56.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.030+0000 7f7ad9c00640 1 --2- 172.21.15.90:0/3251182129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad4112810 0x7f7ad4118e80 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009ca0 tx=0x7f7acc02a8c0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.031+0000 7f7ad9c00640 1 -- 172.21.15.90:0/3251182129 shutdown_connections 2024-05-27T08:05:56.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.031+0000 7f7ad9c00640 1 --2- 172.21.15.90:0/3251182129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ad4111d70 0x7f7ad4112180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.031+0000 7f7ad9c00640 1 --2- 172.21.15.90:0/3251182129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad4112810 0x7f7ad4118e80 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.031+0000 7f7ad9c00640 1 --2- 172.21.15.90:0/3251182129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ad410a740 0x7f7ad410ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.031+0000 7f7ad9c00640 1 -- 172.21.15.90:0/3251182129 >> 172.21.15.90:0/3251182129 conn(0x7f7ad406ed10 msgr2=0x7f7ad406f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:56.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.031+0000 7f7ad9c00640 1 -- 172.21.15.90:0/3251182129 shutdown_connections 2024-05-27T08:05:56.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.031+0000 7f7ad9c00640 1 -- 172.21.15.90:0/3251182129 wait complete. 2024-05-27T08:05:56.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.032+0000 7f7ad9c00640 1 Processor -- start 2024-05-27T08:05:56.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.032+0000 7f7ad9c00640 1 -- start start 2024-05-27T08:05:56.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.032+0000 7f7ad9c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a740 0x7f7ad410baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:56.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.032+0000 7f7ad9c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ad4111d70 0x7f7ad410c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:56.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.032+0000 7f7ad9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ad4112810 0x7f7ad410c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.032+0000 7f7ad9c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad411c530 con 0x7f7ad410a740 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad9c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad411c3b0 con 0x7f7ad4111d70 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad9c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad411c6b0 con 0x7f7ad4112810 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a740 0x7f7ad410baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a740 0x7f7ad410baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53546/0 (socket says 172.21.15.90:53546) 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad2a00640 1 -- 172.21.15.90:0/1432090233 learned_addr learned my addr 172.21.15.90:0/1432090233 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad2000640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ad4111d70 0x7f7ad410c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad3400640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ad4112810 0x7f7ad410c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad2a00640 1 -- 172.21.15.90:0/1432090233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ad4112810 msgr2=0x7f7ad410c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad2a00640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ad4112810 0x7f7ad410c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad2a00640 1 -- 172.21.15.90:0/1432090233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ad4111d70 msgr2=0x7f7ad410c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:56.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad2a00640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ad4111d70 0x7f7ad410c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.033+0000 7f7ad2a00640 1 -- 172.21.15.90:0/1432090233 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad41b1230 con 0x7f7ad410a740 2024-05-27T08:05:56.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.034+0000 7f7ad2a00640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a740 0x7f7ad410baf0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4005d00 tx=0x7f7ac400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:56.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.034+0000 7f7acb400640 1 -- 172.21.15.90:0/1432090233 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4017070 con 0x7f7ad410a740 2024-05-27T08:05:56.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.034+0000 7f7acb400640 1 -- 172.21.15.90:0/1432090233 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ac4009c30 con 0x7f7ad410a740 2024-05-27T08:05:56.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.034+0000 7f7acb400640 1 -- 172.21.15.90:0/1432090233 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400f620 con 0x7f7ad410a740 2024-05-27T08:05:56.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.034+0000 7f7ad9c00640 1 -- 172.21.15.90:0/1432090233 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad41b1520 con 0x7f7ad410a740 2024-05-27T08:05:56.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.034+0000 7f7ad9c00640 1 -- 172.21.15.90:0/1432090233 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad41b1a20 con 0x7f7ad410a740 2024-05-27T08:05:56.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.035+0000 7f7ad9c00640 1 -- 172.21.15.90:0/1432090233 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad4069de0 con 0x7f7ad410a740 2024-05-27T08:05:56.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.038+0000 7f7acb400640 1 -- 172.21.15.90:0/1432090233 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7ac401a070 con 0x7f7ad410a740 2024-05-27T08:05:56.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.040+0000 7f7acb400640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ab80b29e0 0x7f7ab80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:56.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.041+0000 7f7acb400640 1 -- 172.21.15.90:0/1432090233 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f7ac40e24a0 con 0x7f7ad410a740 2024-05-27T08:05:56.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.041+0000 7f7ad2000640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ab80b29e0 0x7f7ab80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:56.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.042+0000 7f7ad2000640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ab80b29e0 0x7f7ab80b4ef0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005fd0 tx=0x7f7ac0005850 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:56.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.044+0000 7f7acb400640 1 -- 172.21.15.90:0/1432090233 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ac4009de0 con 0x7f7ad410a740 2024-05-27T08:05:56.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:55 smithi136 ceph-mon[35324]: pgmap v629: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-05-27T08:05:56.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.253+0000 7f7ad9c00640 1 -- 172.21.15.90:0/1432090233 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ad410d020 con 0x7f7ad410a740 2024-05-27T08:05:56.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.257+0000 7f7acb400640 1 -- 172.21.15.90:0/1432090233 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+25290 (secure 0 0 0) 0x7f7ac40a9ad0 con 0x7f7ad410a740 2024-05-27T08:05:56.259 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:56.259 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":125,"fsid":"b0214938-1bfa-11ef-bc9b-c7b262605968","created":"2024-05-27T07:29:49.415763+0000","modified":"2024-05-27T08:04:12.438129+0000","last_up_change":"2024-05-27T07:39:47.953610+0000","last_in_change":"2024-05-27T07:39:20.788296+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T07:34:18.340712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T07:41:03.393870+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T07:41:07.610662+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":124,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-27T07:41:19.963468+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":124,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"81e03104-5d6d-4892-a8c1-6a186048c90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6803","nonce":70197447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6805","nonce":70197447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6809","nonce":70197447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":70197447},{"type":"v1","addr":"172.21.15.90:6807","nonce":70197447}]},"public_addr":"172.21.15.90:6803/70197447","cluster_addr":"172.21.15.90:6805/70197447","heartbeat_back_addr":"172.21.15.90:6809/70197447","heartbeat_front_addr":"172.21.15.90:6807/70197447","state":["exists","up"]},{"osd":1,"uuid":"0e44f3ac-ac7c-40fc-b7e7-8b42257a3b0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6801","nonce":3393795676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6803","nonce":3393795676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6807","nonce":3393795676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3393795676},{"type":"v1","addr":"172.21.15.129:6805","nonce":3393795676}]},"public_addr":"172.21.15.129:6801/3393795676","cluster_addr":"172.21.15.129:6803/3393795676","heartbeat_back_addr":"172.21.15.129:6807/3393795676","heartbeat_front_addr":"172.21.15.129:6805/3393795676","state":["exists","up"]},{"osd":2,"uuid":"4b859166-8b57-42a1-852f-93a38ab9102c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6801","nonce":1421934201}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6803","nonce":1421934201}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6807","nonce":1421934201}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1421934201},{"type":"v1","addr":"172.21.15.136:6805","nonce":1421934201}]},"public_addr":"172.21.15.136:6801/1421934201","cluster_addr":"172.21.15.136:6803/1421934201","heartbeat_back_addr":"172.21.15.136:6807/1421934201","heartbeat_front_addr":"172.21.15.136:6805/1421934201","state":["exists","up"]},{"osd":3,"uuid":"a92e6906-1cfb-4f68-9ab7-2bc35221478f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6811","nonce":1546698486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6813","nonce":1546698486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6817","nonce":1546698486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":1546698486},{"type":"v1","addr":"172.21.15.90:6815","nonce":1546698486}]},"public_addr":"172.21.15.90:6811/1546698486","cluster_addr":"172.21.15.90:6813/1546698486","heartbeat_back_addr":"172.21.15.90:6817/1546698486","heartbeat_front_addr":"172.21.15.90:6815/1546698486","state":["exists","up"]},{"osd":4,"uuid":"a730ea3e-47fa-46e2-b3af-dba7d3ac49c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6809","nonce":2737784840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6811","nonce":2737784840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6815","nonce":2737784840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2737784840},{"type":"v1","addr":"172.21.15.129:6813","nonce":2737784840}]},"public_addr":"172.21.15.129:6809/2737784840","cluster_addr":"172.21.15.129:6811/2737784840","heartbeat_back_addr":"172.21.15.129:6815/2737784840","heartbeat_front_addr":"172.21.15.129:6813/2737784840","state":["exists","up"]},{"osd":5,"uuid":"8d6ab904-e641-4ebf-98f8-39396f44f205","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6809","nonce":3317770816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6811","nonce":3317770816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6815","nonce":3317770816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3317770816},{"type":"v1","addr":"172.21.15.136:6813","nonce":3317770816}]},"public_addr":"172.21.15.136:6809/3317770816","cluster_addr":"172.21.15.136:6811/3317770816","heartbeat_back_addr":"172.21.15.136:6815/3317770816","heartbeat_front_addr":"172.21.15.136:6813/3317770816","state":["exists","up"]},{"osd":6,"uuid":"f675d248-884d-4bbb-88ab-6edab1eafcda","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6819","nonce":476520683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6821","nonce":476520683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6825","nonce":476520683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":476520683},{"type":"v1","addr":"172.21.15.90:6823","nonce":476520683}]},"public_addr":"172.21.15.90:6819/476520683","cluster_addr":"172.21.15.90:6821/476520683","heartbeat_back_addr":"172.21.15.90:6825/476520683","heartbeat_front_addr":"172.21.15.90:6823/476520683","state":["exists","up"]},{"osd":7,"uuid":"22a66106-a43b-4eb3-adf2-5291094db025","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6817","nonce":1361444791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6819","nonce":1361444791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6823","nonce":1361444791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1361444791},{"type":"v1","addr":"172.21.15.129:6821","nonce":1361444791}]},"public_addr":"172.21.15.129:6817/1361444791","cluster_addr":"172.21.15.129:6819/1361444791","heartbeat_back_addr":"172.21.15.129:6823/1361444791","heartbeat_front_addr":"172.21.15.129:6821/1361444791","state":["exists","up"]},{"osd":8,"uuid":"4148e784-16f8-48b6-8521-ed7556044566","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6817","nonce":2341317946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6819","nonce":2341317946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6823","nonce":2341317946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":2341317946},{"type":"v1","addr":"172.21.15.136:6821","nonce":2341317946}]},"public_addr":"172.21.15.136:6817/2341317946","cluster_addr":"172.21.15.136:6819/2341317946","heartbeat_back_addr":"172.21.15.136:6823/2341317946","heartbeat_front_addr":"172.21.15.136:6821/2341317946","state":["exists","up"]},{"osd":9,"uuid":"14e386d1-ab82-4e8d-baa4-14d02137a192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6827","nonce":4197259682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6829","nonce":4197259682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6833","nonce":4197259682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4197259682},{"type":"v1","addr":"172.21.15.90:6831","nonce":4197259682}]},"public_addr":"172.21.15.90:6827/4197259682","cluster_addr":"172.21.15.90:6829/4197259682","heartbeat_back_addr":"172.21.15.90:6833/4197259682","heartbeat_front_addr":"172.21.15.90:6831/4197259682","state":["exists","up"]},{"osd":10,"uuid":"5da4c9b4-fc8c-43f4-a2a3-a4883c7f3bb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6825","nonce":95466853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6827","nonce":95466853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6831","nonce":95466853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":95466853},{"type":"v1","addr":"172.21.15.129:6829","nonce":95466853}]},"public_addr":"172.21.15.129:6825/95466853","cluster_addr":"172.21.15.129:6827/95466853","heartbeat_back_addr":"172.21.15.129:6831/95466853","heartbeat_front_addr":"172.21.15.129:6829/95466853","state":["exists","up"]},{"osd":11,"uuid":"d748f6bf-b26a-4a17-9b1b-51e78f7d5708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6825","nonce":2683318661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6827","nonce":2683318661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6831","nonce":2683318661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2683318661},{"type":"v1","addr":"172.21.15.136:6829","nonce":2683318661}]},"public_addr":"172.21.15.136:6825/2683318661","cluster_addr":"172.21.15.136:6827/2683318661","heartbeat_back_addr":"172.21.15.136:6831/2683318661","heartbeat_front_addr":"172.21.15.136:6829/2683318661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:03.220488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:33:39.101467+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:14.268729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:34:50.002829+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:35:26.668522+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:02.576152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:36:38.957277+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:17.133701+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:37:53.924572+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:38:30.269074+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:07.731933+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T07:39:45.659088+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":7,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6801/2854405112":"2024-05-28T07:46:26.839737+0000","172.21.15.90:6800/2854405112":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/3526000120":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/836242224":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/2932841075":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/1476836587":"2024-05-28T07:30:58.154487+0000","172.21.15.90:0/3178069601":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6801/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/2742579272":"2024-05-28T07:30:58.154487+0000","172.21.15.90:6800/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2593454412":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/3534244243":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2692415447":"2024-05-28T07:46:26.839737+0000","172.21.15.90:0/2575484267":"2024-05-28T07:30:23.548910+0000","172.21.15.90:6801/410158065":"2024-05-28T07:30:23.548910+0000","172.21.15.90:0/2626688812":"2024-05-28T07:30:58.154487+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]},{"pool":4,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:05:56.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 -- 172.21.15.90:0/1432090233 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ab80b29e0 msgr2=0x7f7ab80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:56.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ab80b29e0 0x7f7ab80b4ef0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0005fd0 tx=0x7f7ac0005850 comp rx=0 tx=0).stop 2024-05-27T08:05:56.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 -- 172.21.15.90:0/1432090233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a740 msgr2=0x7f7ad410baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:56.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a740 0x7f7ad410baf0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4005d00 tx=0x7f7ac400dde0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 -- 172.21.15.90:0/1432090233 shutdown_connections 2024-05-27T08:05:56.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ab80b29e0 0x7f7ab80b4ef0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ad4111d70 0x7f7ad410c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ad4112810 0x7f7ad410c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 --2- 172.21.15.90:0/1432090233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ad410a740 0x7f7ad410baf0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 -- 172.21.15.90:0/1432090233 >> 172.21.15.90:0/1432090233 conn(0x7f7ad406ed10 msgr2=0x7f7ad4073710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 -- 172.21.15.90:0/1432090233 shutdown_connections 2024-05-27T08:05:56.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.260+0000 7f7ad9c00640 1 -- 172.21.15.90:0/1432090233 wait complete. 2024-05-27T08:05:56.287 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T07:34:18.340712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T07:41:03.393870+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T07:41:07.610662+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-27T07:41:19.963468+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-27T08:05:56.288 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T08:05:56.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:55 smithi129 ceph-mon[35254]: pgmap v629: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-05-27T08:05:56.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.563+0000 7f79bee00640 1 -- 172.21.15.90:0/752867583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b8115060 msgr2=0x7f79b8117500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:56.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.563+0000 7f79bee00640 1 --2- 172.21.15.90:0/752867583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b8115060 0x7f79b8117500 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f79b000adb0 tx=0x7f79b002b7a0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.563+0000 7f79bee00640 1 -- 172.21.15.90:0/752867583 shutdown_connections 2024-05-27T08:05:56.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.563+0000 7f79bee00640 1 --2- 172.21.15.90:0/752867583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79b810f300 0x7f79b810f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.563+0000 7f79bee00640 1 --2- 172.21.15.90:0/752867583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b8115060 0x7f79b8117500 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.563+0000 7f79bee00640 1 --2- 172.21.15.90:0/752867583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79b810ad30 0x7f79b810edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.563+0000 7f79bee00640 1 -- 172.21.15.90:0/752867583 >> 172.21.15.90:0/752867583 conn(0x7f79b806ed70 msgr2=0x7f79b806f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:56.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.563+0000 7f79bee00640 1 -- 172.21.15.90:0/752867583 shutdown_connections 2024-05-27T08:05:56.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.563+0000 7f79bee00640 1 -- 172.21.15.90:0/752867583 wait complete. 2024-05-27T08:05:56.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.564+0000 7f79bee00640 1 Processor -- start 2024-05-27T08:05:56.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.564+0000 7f79bee00640 1 -- start start 2024-05-27T08:05:56.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.564+0000 7f79bee00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79b810ad30 0x7f79b81a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:56.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.564+0000 7f79bee00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b810f300 0x7f79b81a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:56.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.564+0000 7f79bee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79b8115060 0x7f79b81a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.564+0000 7f79bee00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f79b811a2a0 con 0x7f79b810f300 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.564+0000 7f79bee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f79b811a120 con 0x7f79b8115060 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.564+0000 7f79bee00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f79b811a420 con 0x7f79b810ad30 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.565+0000 7f79b7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b810f300 0x7f79b81a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.565+0000 7f79b7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b810f300 0x7f79b81a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53568/0 (socket says 172.21.15.90:53568) 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.565+0000 7f79b7400640 1 -- 172.21.15.90:0/2387358166 learned_addr learned my addr 172.21.15.90:0/2387358166 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.565+0000 7f79b7e00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79b810ad30 0x7f79b81a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.565+0000 7f79bce00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79b8115060 0x7f79b81a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.565+0000 7f79b7400640 1 -- 172.21.15.90:0/2387358166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79b810ad30 msgr2=0x7f79b81a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.565+0000 7f79b7400640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79b810ad30 0x7f79b81a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.565+0000 7f79b7400640 1 -- 172.21.15.90:0/2387358166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79b8115060 msgr2=0x7f79b81a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.565+0000 7f79b7400640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79b8115060 0x7f79b81a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.565+0000 7f79b7400640 1 -- 172.21.15.90:0/2387358166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b81a9fd0 con 0x7f79b810f300 2024-05-27T08:05:56.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.566+0000 7f79b7400640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b810f300 0x7f79b81a56d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f79a400d770 tx=0x7f79a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:56.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.566+0000 7f79b4c00640 1 -- 172.21.15.90:0/2387358166 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a4017070 con 0x7f79b810f300 2024-05-27T08:05:56.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.566+0000 7f79b4c00640 1 -- 172.21.15.90:0/2387358166 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f79a4009c30 con 0x7f79b810f300 2024-05-27T08:05:56.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.566+0000 7f79b4c00640 1 -- 172.21.15.90:0/2387358166 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a400f620 con 0x7f79b810f300 2024-05-27T08:05:56.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.566+0000 7f79bee00640 1 -- 172.21.15.90:0/2387358166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b81aa2c0 con 0x7f79b810f300 2024-05-27T08:05:56.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.566+0000 7f79bee00640 1 -- 172.21.15.90:0/2387358166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b81b1c40 con 0x7f79b810f300 2024-05-27T08:05:56.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.567+0000 7f79bee00640 1 -- 172.21.15.90:0/2387358166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b8069e30 con 0x7f79b810f300 2024-05-27T08:05:56.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.571+0000 7f79b4c00640 1 -- 172.21.15.90:0/2387358166 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f79a401a070 con 0x7f79b810f300 2024-05-27T08:05:56.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.572+0000 7f79b4c00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f799c0b29e0 0x7f799c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:56.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.572+0000 7f79b4c00640 1 -- 172.21.15.90:0/2387358166 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f79a40e2510 con 0x7f79b810f300 2024-05-27T08:05:56.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.573+0000 7f79b7e00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f799c0b29e0 0x7f799c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:56.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.573+0000 7f79b7e00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f799c0b29e0 0x7f799c0b4ef0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005fd0 tx=0x7f79a8005850 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:56.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.573+0000 7f79b4c00640 1 -- 172.21.15.90:0/2387358166 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79a4009de0 con 0x7f79b810f300 2024-05-27T08:05:56.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.783+0000 7f79bee00640 1 -- 172.21.15.90:0/2387358166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f79b81b1e70 con 0x7f79b810f300 2024-05-27T08:05:56.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.785+0000 7f79b4c00640 1 -- 172.21.15.90:0/2387358166 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) ==== 93+0+10 (secure 0 0 0) 0x7f79a40a9b40 con 0x7f79b810f300 2024-05-27T08:05:56.786 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-05-27T08:05:56.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 -- 172.21.15.90:0/2387358166 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f799c0b29e0 msgr2=0x7f799c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:56.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f799c0b29e0 0x7f799c0b4ef0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005fd0 tx=0x7f79a8005850 comp rx=0 tx=0).stop 2024-05-27T08:05:56.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 -- 172.21.15.90:0/2387358166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b810f300 msgr2=0x7f79b81a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:56.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b810f300 0x7f79b81a56d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f79a400d770 tx=0x7f79a400dc30 comp rx=0 tx=0).stop 2024-05-27T08:05:56.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 -- 172.21.15.90:0/2387358166 shutdown_connections 2024-05-27T08:05:56.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f799c0b29e0 0x7f799c0b4ef0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79b8115060 0x7f79b81a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79b810f300 0x7f79b81a56d0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 --2- 172.21.15.90:0/2387358166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79b810ad30 0x7f79b81a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:56.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 -- 172.21.15.90:0/2387358166 >> 172.21.15.90:0/2387358166 conn(0x7f79b806ed70 msgr2=0x7f79b810dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:56.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 -- 172.21.15.90:0/2387358166 shutdown_connections 2024-05-27T08:05:56.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:56.788+0000 7f79bee00640 1 -- 172.21.15.90:0/2387358166 wait complete. 2024-05-27T08:05:56.817 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T08:05:57.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.081+0000 7fc156200640 1 -- 172.21.15.90:0/1357593334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc150109a60 msgr2=0x7fc150103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:57.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.081+0000 7fc156200640 1 --2- 172.21.15.90:0/1357593334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc150109a60 0x7fc150103660 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc144009900 tx=0x7fc14402a3f0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.081+0000 7fc156200640 1 -- 172.21.15.90:0/1357593334 shutdown_connections 2024-05-27T08:05:57.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.081+0000 7fc156200640 1 --2- 172.21.15.90:0/1357593334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc150109f00 0x7fc150103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.081+0000 7fc156200640 1 --2- 172.21.15.90:0/1357593334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc150104210 0x7fc1501104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.081+0000 7fc156200640 1 --2- 172.21.15.90:0/1357593334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc150109a60 0x7fc150103660 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.081+0000 7fc156200640 1 -- 172.21.15.90:0/1357593334 >> 172.21.15.90:0/1357593334 conn(0x7fc1500ff2c0 msgr2=0x7fc1501016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:57.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.081+0000 7fc156200640 1 -- 172.21.15.90:0/1357593334 shutdown_connections 2024-05-27T08:05:57.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.081+0000 7fc156200640 1 -- 172.21.15.90:0/1357593334 wait complete. 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.082+0000 7fc156200640 1 Processor -- start 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.083+0000 7fc156200640 1 -- start start 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.083+0000 7fc156200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc150104210 0x7fc15019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.083+0000 7fc156200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc150109a60 0x7fc15019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.083+0000 7fc156200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc150109f00 0x7fc15019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.083+0000 7fc156200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc150113b60 con 0x7fc150109a60 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.083+0000 7fc156200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1501139e0 con 0x7fc150104210 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.083+0000 7fc156200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc150113ce0 con 0x7fc150109f00 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.083+0000 7fc14fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc150109a60 0x7fc15019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.083+0000 7fc14fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc150109a60 0x7fc15019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53590/0 (socket says 172.21.15.90:53590) 2024-05-27T08:05:57.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.083+0000 7fc14fe00640 1 -- 172.21.15.90:0/76640933 learned_addr learned my addr 172.21.15.90:0/76640933 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:57.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc154e00640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc150104210 0x7fc15019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:57.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc14fe00640 1 -- 172.21.15.90:0/76640933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc150109f00 msgr2=0x7fc15019ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:57.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc14fe00640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc150109f00 0x7fc15019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc14fe00640 1 -- 172.21.15.90:0/76640933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc150104210 msgr2=0x7fc15019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:57.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc14fe00640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc150104210 0x7fc15019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc14fe00640 1 -- 172.21.15.90:0/76640933 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15019f3c0 con 0x7fc150109a60 2024-05-27T08:05:57.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc154e00640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc150104210 0x7fc15019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:57.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc14fe00640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc150109a60 0x7fc15019ab40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00c9a0 tx=0x7fc13c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:57.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc14d600640 1 -- 172.21.15.90:0/76640933 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c007c70 con 0x7fc150109a60 2024-05-27T08:05:57.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc14d600640 1 -- 172.21.15.90:0/76640933 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc13c007e10 con 0x7fc150109a60 2024-05-27T08:05:57.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc14d600640 1 -- 172.21.15.90:0/76640933 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c018780 con 0x7fc150109a60 2024-05-27T08:05:57.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.084+0000 7fc156200640 1 -- 172.21.15.90:0/76640933 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15019f6b0 con 0x7fc150109a60 2024-05-27T08:05:57.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.085+0000 7fc156200640 1 -- 172.21.15.90:0/76640933 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15019fbb0 con 0x7fc150109a60 2024-05-27T08:05:57.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.086+0000 7fc156200640 1 -- 172.21.15.90:0/76640933 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15006a150 con 0x7fc150109a60 2024-05-27T08:05:57.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.088+0000 7fc14d600640 1 -- 172.21.15.90:0/76640933 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc13c00fb20 con 0x7fc150109a60 2024-05-27T08:05:57.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.090+0000 7fc14d600640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc1380b29d0 0x7fc1380b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:57.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.090+0000 7fc14d600640 1 -- 172.21.15.90:0/76640933 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7fc13c0e2c00 con 0x7fc150109a60 2024-05-27T08:05:57.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.091+0000 7fc154e00640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc1380b29d0 0x7fc1380b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:57.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.091+0000 7fc154e00640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc1380b29d0 0x7fc1380b4ee0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc144005ab0 tx=0x7fc144005fd0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:57.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.093+0000 7fc14d600640 1 -- 172.21.15.90:0/76640933 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc13c014030 con 0x7fc150109a60 2024-05-27T08:05:57.153 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:56 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1432090233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:05:57.153 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:56 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2387358166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:05:57.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:56 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1432090233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:05:57.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:56 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2387358166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:05:57.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.298+0000 7fc156200640 1 -- 172.21.15.90:0/76640933 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc1501a72b0 con 0x7fc150109a60 2024-05-27T08:05:57.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.299+0000 7fc14d600640 1 -- 172.21.15.90:0/76640933 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) ==== 104+0+11 (secure 0 0 0) 0x7fc13c0aa2c0 con 0x7fc150109a60 2024-05-27T08:05:57.301 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-05-27T08:05:57.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 -- 172.21.15.90:0/76640933 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc1380b29d0 msgr2=0x7fc1380b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:57.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc1380b29d0 0x7fc1380b4ee0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc144005ab0 tx=0x7fc144005fd0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 -- 172.21.15.90:0/76640933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc150109a60 msgr2=0x7fc15019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:57.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc150109a60 0x7fc15019ab40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00c9a0 tx=0x7fc13c00ce60 comp rx=0 tx=0).stop 2024-05-27T08:05:57.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 -- 172.21.15.90:0/76640933 shutdown_connections 2024-05-27T08:05:57.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fc1380b29d0 0x7fc1380b4ee0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc150109f00 0x7fc15019ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc150109a60 0x7fc15019ab40 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 --2- 172.21.15.90:0/76640933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc150104210 0x7fc15019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 -- 172.21.15.90:0/76640933 >> 172.21.15.90:0/76640933 conn(0x7fc1500ff2c0 msgr2=0x7fc1500ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:57.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 -- 172.21.15.90:0/76640933 shutdown_connections 2024-05-27T08:05:57.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.302+0000 7fc156200640 1 -- 172.21.15.90:0/76640933 wait complete. 2024-05-27T08:05:57.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:56 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1432090233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:05:57.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:56 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2387358166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:05:57.331 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T08:05:57.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.597+0000 7f9f58000640 1 -- 172.21.15.90:0/233296837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f50104380 msgr2=0x7f9f50110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:57.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.597+0000 7f9f58000640 1 --2- 172.21.15.90:0/233296837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f50104380 0x7f9f50110750 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009ca0 tx=0x7f9f4802a8c0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.598+0000 7f9f58000640 1 -- 172.21.15.90:0/233296837 shutdown_connections 2024-05-27T08:05:57.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.598+0000 7f9f58000640 1 --2- 172.21.15.90:0/233296837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f5010a0d0 0x7f9f50103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.598+0000 7f9f58000640 1 --2- 172.21.15.90:0/233296837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f50104380 0x7f9f50110750 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.598+0000 7f9f58000640 1 --2- 172.21.15.90:0/233296837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f50109c30 0x7f9f501037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.598+0000 7f9f58000640 1 -- 172.21.15.90:0/233296837 >> 172.21.15.90:0/233296837 conn(0x7f9f500ff330 msgr2=0x7f9f50101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:57.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.598+0000 7f9f58000640 1 -- 172.21.15.90:0/233296837 shutdown_connections 2024-05-27T08:05:57.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.598+0000 7f9f58000640 1 -- 172.21.15.90:0/233296837 wait complete. 2024-05-27T08:05:57.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.599+0000 7f9f58000640 1 Processor -- start 2024-05-27T08:05:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.599+0000 7f9f58000640 1 -- start start 2024-05-27T08:05:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.599+0000 7f9f58000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f50104380 0x7f9f500760d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f58000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f50109c30 0x7f9f50076610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f58000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f5010a0d0 0x7f9f50076b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f58000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50113d40 con 0x7f9f50104380 2024-05-27T08:05:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f58000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50113bc0 con 0x7f9f5010a0d0 2024-05-27T08:05:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f58000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50113ec0 con 0x7f9f50109c30 2024-05-27T08:05:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f55600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f50104380 0x7f9f500760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f55600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f50104380 0x7f9f500760d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53620/0 (socket says 172.21.15.90:53620) 2024-05-27T08:05:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f55600640 1 -- 172.21.15.90:0/4006341120 learned_addr learned my addr 172.21.15.90:0/4006341120 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:57.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f54c00640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f50109c30 0x7f9f50076610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:57.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f55600640 1 -- 172.21.15.90:0/4006341120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f50109c30 msgr2=0x7f9f50076610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:57.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f55600640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f50109c30 0x7f9f50076610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f55600640 1 -- 172.21.15.90:0/4006341120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f5010a0d0 msgr2=0x7f9f50076b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:57.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f55600640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f5010a0d0 0x7f9f50076b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.600+0000 7f9f55600640 1 -- 172.21.15.90:0/4006341120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f501a8a60 con 0x7f9f50104380 2024-05-27T08:05:57.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.601+0000 7f9f55600640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f50104380 0x7f9f500760d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400d740 tx=0x7f9f4400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:57.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.601+0000 7f9f4e000640 1 -- 172.21.15.90:0/4006341120 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44017070 con 0x7f9f50104380 2024-05-27T08:05:57.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.601+0000 7f9f4e000640 1 -- 172.21.15.90:0/4006341120 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f4400de50 con 0x7f9f50104380 2024-05-27T08:05:57.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.601+0000 7f9f4e000640 1 -- 172.21.15.90:0/4006341120 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44004c60 con 0x7f9f50104380 2024-05-27T08:05:57.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.601+0000 7f9f58000640 1 -- 172.21.15.90:0/4006341120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f501a8d50 con 0x7f9f50104380 2024-05-27T08:05:57.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.601+0000 7f9f58000640 1 -- 172.21.15.90:0/4006341120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f501a91e0 con 0x7f9f50104380 2024-05-27T08:05:57.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.602+0000 7f9f58000640 1 -- 172.21.15.90:0/4006341120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5006a280 con 0x7f9f50104380 2024-05-27T08:05:57.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.605+0000 7f9f4e000640 1 -- 172.21.15.90:0/4006341120 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9f44004e00 con 0x7f9f50104380 2024-05-27T08:05:57.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.606+0000 7f9f4e000640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9f380b29e0 0x7f9f380b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:57.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.606+0000 7f9f4e000640 1 -- 172.21.15.90:0/4006341120 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f9f440e2820 con 0x7f9f50104380 2024-05-27T08:05:57.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.606+0000 7f9f54c00640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9f380b29e0 0x7f9f380b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:57.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.607+0000 7f9f54c00640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9f380b29e0 0x7f9f380b4ef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9f50108e60 tx=0x7f9f3c005ed0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:57.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.609+0000 7f9f4e000640 1 -- 172.21.15.90:0/4006341120 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f440a9e50 con 0x7f9f50104380 2024-05-27T08:05:57.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.818+0000 7f9f58000640 1 -- 172.21.15.90:0/4006341120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9f5002dc20 con 0x7f9f50104380 2024-05-27T08:05:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.819+0000 7f9f4e000640 1 -- 172.21.15.90:0/4006341120 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) ==== 100+0+11 (secure 0 0 0) 0x7f9f440a9750 con 0x7f9f50104380 2024-05-27T08:05:57.821 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-05-27T08:05:57.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 -- 172.21.15.90:0/4006341120 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9f380b29e0 msgr2=0x7f9f380b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:57.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9f380b29e0 0x7f9f380b4ef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9f50108e60 tx=0x7f9f3c005ed0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 -- 172.21.15.90:0/4006341120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f50104380 msgr2=0x7f9f500760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:57.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f50104380 0x7f9f500760d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400d740 tx=0x7f9f4400dc00 comp rx=0 tx=0).stop 2024-05-27T08:05:57.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 -- 172.21.15.90:0/4006341120 shutdown_connections 2024-05-27T08:05:57.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f9f380b29e0 0x7f9f380b4ef0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f5010a0d0 0x7f9f50076b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f50109c30 0x7f9f50076610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 --2- 172.21.15.90:0/4006341120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f50104380 0x7f9f500760d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:57.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 -- 172.21.15.90:0/4006341120 >> 172.21.15.90:0/4006341120 conn(0x7f9f500ff330 msgr2=0x7f9f50101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:57.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 -- 172.21.15.90:0/4006341120 shutdown_connections 2024-05-27T08:05:57.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:57.822+0000 7f9f58000640 1 -- 172.21.15.90:0/4006341120 wait complete. 2024-05-27T08:05:57.849 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-27T08:05:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.115+0000 7f7edfe00640 1 -- 172.21.15.90:0/576426746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ee010f330 msgr2=0x7f7ee0116ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.115+0000 7f7edfe00640 1 --2- 172.21.15.90:0/576426746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ee010f330 0x7f7ee0116ad0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009ca0 tx=0x7f7ed802a8c0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.115+0000 7f7edfe00640 1 -- 172.21.15.90:0/576426746 shutdown_connections 2024-05-27T08:05:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.115+0000 7f7edfe00640 1 --2- 172.21.15.90:0/576426746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ee010f330 0x7f7ee0116ad0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.115+0000 7f7edfe00640 1 --2- 172.21.15.90:0/576426746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ee010e870 0x7f7ee010eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.115+0000 7f7edfe00640 1 --2- 172.21.15.90:0/576426746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ee010a2e0 0x7f7ee010a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.115+0000 7f7edfe00640 1 -- 172.21.15.90:0/576426746 >> 172.21.15.90:0/576426746 conn(0x7f7ee0076c80 msgr2=0x7f7ee00770c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:58.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.115+0000 7f7edfe00640 1 -- 172.21.15.90:0/576426746 shutdown_connections 2024-05-27T08:05:58.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.115+0000 7f7edfe00640 1 -- 172.21.15.90:0/576426746 wait complete. 2024-05-27T08:05:58.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.116+0000 7f7edfe00640 1 Processor -- start 2024-05-27T08:05:58.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.116+0000 7f7edfe00640 1 -- start start 2024-05-27T08:05:58.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.116+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ee010a2e0 0x7f7ee01a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.116+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ee010e870 0x7f7ee01a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.116+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ee010f330 0x7f7ee01a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.116+0000 7f7edfe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee011a120 con 0x7f7ee010e870 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.116+0000 7f7edfe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0119fa0 con 0x7f7ee010a2e0 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.116+0000 7f7edfe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee011a2a0 con 0x7f7ee010f330 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7ed7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ee010e870 0x7f7ee01a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7ed7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ee010e870 0x7f7ee01a6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53636/0 (socket says 172.21.15.90:53636) 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7ed7e00640 1 -- 172.21.15.90:0/1191187861 learned_addr learned my addr 172.21.15.90:0/1191187861 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7edf400640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ee010f330 0x7f7ee01a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7edea00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ee010a2e0 0x7f7ee01a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7ed7e00640 1 -- 172.21.15.90:0/1191187861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ee010f330 msgr2=0x7f7ee01a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7ed7e00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ee010f330 0x7f7ee01a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7ed7e00640 1 -- 172.21.15.90:0/1191187861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ee010a2e0 msgr2=0x7f7ee01a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7ed7e00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ee010a2e0 0x7f7ee01a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7ed7e00640 1 -- 172.21.15.90:0/1191187861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee01a0f90 con 0x7f7ee010e870 2024-05-27T08:05:58.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.117+0000 7f7edf400640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ee010f330 0x7f7ee01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:05:58.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.118+0000 7f7ed7e00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ee010e870 0x7f7ee01a6cd0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c970 tx=0x7f7ec800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:58.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.118+0000 7f7ed7400640 1 -- 172.21.15.90:0/1191187861 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8014070 con 0x7f7ee010e870 2024-05-27T08:05:58.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.118+0000 7f7ed7400640 1 -- 172.21.15.90:0/1191187861 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ec8004440 con 0x7f7ee010e870 2024-05-27T08:05:58.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.118+0000 7f7edfe00640 1 -- 172.21.15.90:0/1191187861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee01a1280 con 0x7f7ee010e870 2024-05-27T08:05:58.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.118+0000 7f7ed7400640 1 -- 172.21.15.90:0/1191187861 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8003d40 con 0x7f7ee010e870 2024-05-27T08:05:58.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.118+0000 7f7edfe00640 1 -- 172.21.15.90:0/1191187861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee01a17d0 con 0x7f7ee010e870 2024-05-27T08:05:58.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.119+0000 7f7edfe00640 1 -- 172.21.15.90:0/1191187861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee006a150 con 0x7f7ee010e870 2024-05-27T08:05:58.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.122+0000 7f7ed7400640 1 -- 172.21.15.90:0/1191187861 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7ec8017070 con 0x7f7ee010e870 2024-05-27T08:05:58.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.123+0000 7f7ed7400640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ec40b2950 0x7f7ec40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.123+0000 7f7ed7400640 1 -- 172.21.15.90:0/1191187861 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f7ec80e2ce0 con 0x7f7ee010e870 2024-05-27T08:05:58.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.123+0000 7f7edea00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ec40b2950 0x7f7ec40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.124+0000 7f7edea00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ec40b2950 0x7f7ec40b4e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0005fd0 tx=0x7f7ed0005850 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:58.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.126+0000 7f7ed7400640 1 -- 172.21.15.90:0/1191187861 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ee01a17d0 con 0x7f7ee010e870 2024-05-27T08:05:58.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:57 smithi090 ceph-mon[31429]: pgmap v630: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-27T08:05:58.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:57 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/76640933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T08:05:58.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:57 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/4006341120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T08:05:58.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:57 smithi136 ceph-mon[35324]: pgmap v630: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-27T08:05:58.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:57 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/76640933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T08:05:58.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:57 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/4006341120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T08:05:58.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:57 smithi129 ceph-mon[35254]: pgmap v630: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-05-27T08:05:58.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:57 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/76640933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T08:05:58.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:57 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/4006341120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T08:05:58.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.335+0000 7f7edfe00640 1 -- 172.21.15.90:0/1191187861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7ee002dc20 con 0x7f7ee010e870 2024-05-27T08:05:58.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.336+0000 7f7ed7400640 1 -- 172.21.15.90:0/1191187861 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v125) ==== 103+0+11 (secure 0 0 0) 0x7f7ec801e020 con 0x7f7ee010e870 2024-05-27T08:05:58.338 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-05-27T08:05:58.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 -- 172.21.15.90:0/1191187861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ec40b2950 msgr2=0x7f7ec40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ec40b2950 0x7f7ec40b4e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0005fd0 tx=0x7f7ed0005850 comp rx=0 tx=0).stop 2024-05-27T08:05:58.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 -- 172.21.15.90:0/1191187861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ee010e870 msgr2=0x7f7ee01a6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ee010e870 0x7f7ee01a6cd0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c970 tx=0x7f7ec800ce30 comp rx=0 tx=0).stop 2024-05-27T08:05:58.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 -- 172.21.15.90:0/1191187861 shutdown_connections 2024-05-27T08:05:58.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f7ec40b2950 0x7f7ec40b4e60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ee010f330 0x7f7ee01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ee010e870 0x7f7ee01a6cd0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 --2- 172.21.15.90:0/1191187861 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ee010a2e0 0x7f7ee01a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 -- 172.21.15.90:0/1191187861 >> 172.21.15.90:0/1191187861 conn(0x7f7ee0076c80 msgr2=0x7f7ee0107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:58.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.340+0000 7f7edfe00640 1 -- 172.21.15.90:0/1191187861 shutdown_connections 2024-05-27T08:05:58.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.341+0000 7f7edfe00640 1 -- 172.21.15.90:0/1191187861 wait complete. 2024-05-27T08:05:58.370 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-27T08:05:58.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.636+0000 7f3bf2600640 1 -- 172.21.15.90:0/1335914935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec10c2f0 msgr2=0x7f3bec105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.636+0000 7f3bf2600640 1 --2- 172.21.15.90:0/1335914935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec10c2f0 0x7f3bec105f30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009ca0 tx=0x7f3bdc02a8c0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.636+0000 7f3bf2600640 1 -- 172.21.15.90:0/1335914935 shutdown_connections 2024-05-27T08:05:58.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.636+0000 7f3bf2600640 1 --2- 172.21.15.90:0/1335914935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bec1065a0 0x7f3bec112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.636+0000 7f3bf2600640 1 --2- 172.21.15.90:0/1335914935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec10c2f0 0x7f3bec105f30 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.636+0000 7f3bf2600640 1 --2- 172.21.15.90:0/1335914935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bec10be50 0x7f3bec1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.636+0000 7f3bf2600640 1 -- 172.21.15.90:0/1335914935 >> 172.21.15.90:0/1335914935 conn(0x7f3bec077080 msgr2=0x7f3bec0774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:05:58.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.636+0000 7f3bf2600640 1 -- 172.21.15.90:0/1335914935 shutdown_connections 2024-05-27T08:05:58.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.637+0000 7f3bf2600640 1 -- 172.21.15.90:0/1335914935 wait complete. 2024-05-27T08:05:58.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.638+0000 7f3bf2600640 1 Processor -- start 2024-05-27T08:05:58.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.638+0000 7f3bf2600640 1 -- start start 2024-05-27T08:05:58.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.638+0000 7f3bf2600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bec1065a0 0x7f3bec1a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.638+0000 7f3bf2600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bec10be50 0x7f3bec1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.638+0000 7f3bf2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec10c2f0 0x7f3bec19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.638+0000 7f3bf2600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec115f40 con 0x7f3bec10c2f0 2024-05-27T08:05:58.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.638+0000 7f3bf2600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec115dc0 con 0x7f3bec10be50 2024-05-27T08:05:58.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.638+0000 7f3bf2600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3bec1160c0 con 0x7f3bec1065a0 2024-05-27T08:05:58.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.638+0000 7f3bebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec10c2f0 0x7f3bec19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.639+0000 7f3bebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec10c2f0 0x7f3bec19c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53648/0 (socket says 172.21.15.90:53648) 2024-05-27T08:05:58.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.639+0000 7f3bebe00640 1 -- 172.21.15.90:0/2903827256 learned_addr learned my addr 172.21.15.90:0/2903827256 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:05:58.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.639+0000 7f3beaa00640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bec10be50 0x7f3bec1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.639+0000 7f3beb400640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bec1065a0 0x7f3bec1a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.639+0000 7f3bebe00640 1 -- 172.21.15.90:0/2903827256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bec1065a0 msgr2=0x7f3bec1a2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.639+0000 7f3bebe00640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bec1065a0 0x7f3bec1a2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.639+0000 7f3bebe00640 1 -- 172.21.15.90:0/2903827256 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bec10be50 msgr2=0x7f3bec1a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:05:58.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.639+0000 7f3bebe00640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bec10be50 0x7f3bec1a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:05:58.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.639+0000 7f3bebe00640 1 -- 172.21.15.90:0/2903827256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bec19cea0 con 0x7f3bec10c2f0 2024-05-27T08:05:58.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.639+0000 7f3bebe00640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec10c2f0 0x7f3bec19c7c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3be400d750 tx=0x7f3be400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.640+0000 7f3be3e00640 1 -- 172.21.15.90:0/2903827256 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4017070 con 0x7f3bec10c2f0 2024-05-27T08:05:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.640+0000 7f3be3e00640 1 -- 172.21.15.90:0/2903827256 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3be40054b0 con 0x7f3bec10c2f0 2024-05-27T08:05:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.640+0000 7f3bf2600640 1 -- 172.21.15.90:0/2903827256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bec19d190 con 0x7f3bec10c2f0 2024-05-27T08:05:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.640+0000 7f3be3e00640 1 -- 172.21.15.90:0/2903827256 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be4004b90 con 0x7f3bec10c2f0 2024-05-27T08:05:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.640+0000 7f3bf2600640 1 -- 172.21.15.90:0/2903827256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bec1a93e0 con 0x7f3bec10c2f0 2024-05-27T08:05:58.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.641+0000 7f3bf2600640 1 -- 172.21.15.90:0/2903827256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bec107130 con 0x7f3bec10c2f0 2024-05-27T08:05:58.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.645+0000 7f3be3e00640 1 -- 172.21.15.90:0/2903827256 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3be4004d30 con 0x7f3bec10c2f0 2024-05-27T08:05:58.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.646+0000 7f3be3e00640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3bd00b29e0 0x7f3bd00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:05:58.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.647+0000 7f3be3e00640 1 -- 172.21.15.90:0/2903827256 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9496+0+0 (secure 0 0 0) 0x7f3be40e28f0 con 0x7f3bec10c2f0 2024-05-27T08:05:58.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.647+0000 7f3beb400640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3bd00b29e0 0x7f3bd00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:05:58.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.648+0000 7f3beb400640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3bd00b29e0 0x7f3bd00b4ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4005fd0 tx=0x7f3bd400bf30 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:05:58.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.649+0000 7f3be3e00640 1 -- 172.21.15.90:0/2903827256 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3be40a9f20 con 0x7f3bec10c2f0 2024-05-27T08:05:58.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.910+0000 7f3bf2600640 1 -- 172.21.15.90:0/2903827256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f3bec06a2b0 con 0x7f3bec10c2f0 2024-05-27T08:05:58.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.940+0000 7f3be3e00640 1 -- 172.21.15.90:0/2903827256 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816174 (secure 0 0 0) 0x7f3bec06a2b0 con 0x7f3bec10c2f0 2024-05-27T08:05:58.959 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 26, 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: "active_gid": 34967, 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: "active_name": "x", 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: "active_addrs": { 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:6800", 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 4256436582 2024-05-27T08:05:58.960 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:6801", 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 4256436582 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: "active_addr": "172.21.15.90:6801/4256436582", 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: "active_change": "2024-05-27T07:46:26.839992+0000", 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: "active_mgr_features": 4540701547738038271, 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: "available": true, 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: "standbys": [ 2024-05-27T08:05:58.961 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "gid": 25000, 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "z", 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_features": 4540701547738038271, 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:58.962 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.963 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:58.964 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.965 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.966 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.967 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.968 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.969 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.970 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.971 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.972 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.973 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.974 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:58.975 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:58.976 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.977 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-05-27T08:05:58.978 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T08:05:58.979 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T08:05:58.980 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.981 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.982 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.983 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-05-27T08:05:58.984 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.985 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-05-27T08:05:58.986 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.987 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.988 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.989 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:58.990 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.991 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.992 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.993 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-05-27T08:05:58.994 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T08:05:58.995 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T08:05:58.996 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.997 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.998 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:58.999 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-05-27T08:05:59.000 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.001 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.002 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.003 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.004 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.005 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.006 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-05-27T08:05:59.007 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.008 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.009 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.010 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.011 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.012 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.013 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.014 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.015 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.016 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.017 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.018 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.019 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.020 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.021 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.022 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.023 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:05:59.024 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-05-27T08:05:59.025 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.026 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.027 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.029 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.030 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-05-27T08:05:59.031 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-05-27T08:05:59.032 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.033 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:05:59.034 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T08:05:59.035 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.036 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.039 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.040 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.041 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.042 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.043 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.044 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.045 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.046 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.047 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.048 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.049 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.050 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T08:05:59.051 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.052 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.053 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.054 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.055 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.056 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.057 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.058 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-05-27T08:05:59.059 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.060 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.061 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.062 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.063 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.064 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-05-27T08:05:59.066 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.067 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.068 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.069 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.070 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.071 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.072 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-05-27T08:05:59.073 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.074 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-05-27T08:05:59.075 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.076 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.077 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.078 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.079 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.080 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T08:05:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T08:05:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T08:05:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T08:05:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T08:05:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T08:05:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T08:05:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T08:05:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T08:05:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T08:05:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T08:05:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-05-27T08:05:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-05-27T08:05:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-05-27T08:05:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-05-27T08:05:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-05-27T08:05:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:05:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:05:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:05:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T08:05:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-05-27T08:05:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-05-27T08:05:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:05:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-05-27T08:05:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-05-27T08:05:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-05-27T08:05:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-05-27T08:05:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T08:05:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-05-27T08:05:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-05-27T08:05:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-05-27T08:05:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-05-27T08:05:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-05-27T08:05:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-05-27T08:05:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T08:05:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-05-27T08:05:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:05:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-05-27T08:05:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-05-27T08:05:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:05:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-05-27T08:05:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-05-27T08:05:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-05-27T08:05:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.343 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-05-27T08:05:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-05-27T08:05:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-05-27T08:05:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-05-27T08:05:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-05-27T08:05:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-05-27T08:05:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-05-27T08:05:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-05-27T08:05:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-05-27T08:05:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-05-27T08:05:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-05-27T08:05:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-05-27T08:05:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smb", 2024-05-27T08:05:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "update_orchestration": { 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "update_orchestration", 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-05-27T08:05:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-05-27T08:05:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-05-27T08:05:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-05-27T08:05:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T08:05:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-05-27T08:05:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-05-27T08:05:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T08:05:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T08:05:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-05-27T08:05:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-05-27T08:05:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-05-27T08:05:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-05-27T08:05:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-05-27T08:05:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T08:05:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-05-27T08:05:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-05-27T08:05:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T08:05:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T08:05:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-05-27T08:05:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-05-27T08:05:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-05-27T08:05:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-05-27T08:05:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "gid": 34961, 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "y", 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_features": 4540701547738038271, 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-05-27T08:05:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-05-27T08:05:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-05-27T08:05:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-05-27T08:05:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-05-27T08:05:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-05-27T08:05:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-05-27T08:05:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-05-27T08:05:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-05-27T08:05:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:05:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-05-27T08:05:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-05-27T08:05:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T08:05:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:05:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-05-27T08:05:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-05-27T08:05:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:05:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-05-27T08:05:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:05:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-05-27T08:05:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T08:05:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-05-27T08:05:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-05-27T08:05:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-05-27T08:05:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-05-27T08:05:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-05-27T08:05:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-05-27T08:05:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-05-27T08:05:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.547 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-05-27T08:05:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T08:05:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T08:05:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-05-27T08:05:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-05-27T08:05:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:05:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T08:05:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-05-27T08:05:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-05-27T08:05:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-05-27T08:05:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-05-27T08:05:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-05-27T08:05:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-05-27T08:05:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:05:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-05-27T08:05:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:05:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-05-27T08:05:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-05-27T08:05:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-05-27T08:05:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-05-27T08:05:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-05-27T08:05:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-05-27T08:05:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T08:05:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T08:05:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T08:05:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-05-27T08:05:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-05-27T08:05:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-05-27T08:05:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-05-27T08:05:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-05-27T08:05:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T08:05:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-05-27T08:05:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-05-27T08:05:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-05-27T08:05:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T08:05:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T08:05:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-05-27T08:05:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-05-27T08:05:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T08:05:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-05-27T08:05:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-05-27T08:05:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-05-27T08:05:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-05-27T08:05:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T08:05:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T08:05:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-05-27T08:05:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-05-27T08:05:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T08:05:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-05-27T08:05:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-05-27T08:05:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T08:05:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-05-27T08:05:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-05-27T08:05:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-05-27T08:05:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-05-27T08:05:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-05-27T08:05:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T08:05:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T08:05:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T08:05:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T08:05:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T08:05:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T08:05:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T08:05:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-05-27T08:05:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T08:05:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T08:05:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T08:05:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T08:05:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T08:05:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T08:05:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T08:05:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T08:05:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T08:05:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T08:05:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T08:05:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T08:05:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T08:05:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T08:05:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T08:05:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T08:05:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-05-27T08:05:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T08:05:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T08:05:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T08:05:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T08:05:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T08:05:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-05-27T08:05:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-05-27T08:05:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-05-27T08:05:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-05-27T08:05:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-05-27T08:05:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-05-27T08:05:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-05-27T08:05:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-05-27T08:05:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-05-27T08:05:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-05-27T08:05:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T08:05:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-05-27T08:05:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-05-27T08:05:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-05-27T08:05:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-05-27T08:05:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T08:05:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.751 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T08:05:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-05-27T08:05:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-05-27T08:05:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-05-27T08:05:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-05-27T08:05:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-05-27T08:05:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-05-27T08:05:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:05:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:05:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:05:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-05-27T08:05:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-05-27T08:05:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-05-27T08:05:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-05-27T08:05:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-05-27T08:05:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-05-27T08:05:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-05-27T08:05:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-05-27T08:05:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T08:05:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-05-27T08:05:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-05-27T08:05:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T08:05:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.782 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-05-27T08:05:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-05-27T08:05:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-05-27T08:05:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-05-27T08:05:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-05-27T08:05:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T08:05:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-05-27T08:05:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-05-27T08:05:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T08:05:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-05-27T08:05:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-05-27T08:05:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-05-27T08:05:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-05-27T08:05:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-05-27T08:05:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-05-27T08:05:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-05-27T08:05:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-05-27T08:05:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T08:05:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.902 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.902 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.902 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.902 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.902 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.902 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.902 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.902 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.902 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.903 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-05-27T08:05:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-05-27T08:05:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-05-27T08:05:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-05-27T08:05:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-05-27T08:05:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-05-27T08:05:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T08:05:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-05-27T08:05:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-05-27T08:05:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smb", 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:05:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:05:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "update_orchestration": { 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "update_orchestration", 2024-05-27T08:06:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-05-27T08:06:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-05-27T08:06:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T08:06:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T08:06:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-05-27T08:06:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-05-27T08:06:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-05-27T08:06:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:06:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:06:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-05-27T08:06:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-05-27T08:06:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-05-27T08:06:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T08:06:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-05-27T08:06:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-05-27T08:06:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "modules": [ 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "dashboard", 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "iostat", 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "nfs", 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "restful", 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "snap_schedule" 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T08:06:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-05-27T08:06:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-05-27T08:06:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-05-27T08:06:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-05-27T08:06:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-05-27T08:06:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-05-27T08:06:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-05-27T08:06:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-05-27T08:06:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-05-27T08:06:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-05-27T08:06:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T08:06:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-05-27T08:06:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-05-27T08:06:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T08:06:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:06:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T08:06:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:06:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-05-27T08:06:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T08:06:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-05-27T08:06:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T08:06:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:06:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-05-27T08:06:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-05-27T08:06:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:06:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T08:06:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T08:06:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-05-27T08:06:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-05-27T08:06:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T08:06:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T08:06:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-05-27T08:06:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-05-27T08:06:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-05-27T08:06:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T08:06:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-05-27T08:06:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-05-27T08:06:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-05-27T08:06:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-05-27T08:06:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-05-27T08:06:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-05-27T08:06:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-05-27T08:06:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T08:06:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-05-27T08:06:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-05-27T08:06:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-05-27T08:06:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-05-27T08:06:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T08:06:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-05-27T08:06:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T08:06:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-05-27T08:06:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-05-27T08:06:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-05-27T08:06:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-05-27T08:06:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-05-27T08:06:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-05-27T08:06:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T08:06:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-05-27T08:06:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-05-27T08:06:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-05-27T08:06:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T08:06:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T08:06:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-05-27T08:06:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-05-27T08:06:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T08:06:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-05-27T08:06:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-05-27T08:06:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T08:06:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T08:06:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T08:06:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-05-27T08:06:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T08:06:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T08:06:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T08:06:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T08:06:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T08:06:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T08:06:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T08:06:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T08:06:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T08:06:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T08:06:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T08:06:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T08:06:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-05-27T08:06:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-05-27T08:06:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-05-27T08:06:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-05-27T08:06:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-05-27T08:06:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-05-27T08:06:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-05-27T08:06:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-05-27T08:06:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-05-27T08:06:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-05-27T08:06:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-05-27T08:06:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-05-27T08:06:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-05-27T08:06:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-05-27T08:06:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-05-27T08:06:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-05-27T08:06:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:06:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:06:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-05-27T08:06:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T08:06:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-05-27T08:06:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-05-27T08:06:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-05-27T08:06:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-05-27T08:06:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-05-27T08:06:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-05-27T08:06:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-05-27T08:06:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-05-27T08:06:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:06:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-05-27T08:06:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-05-27T08:06:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-05-27T08:06:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-05-27T08:06:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-05-27T08:06:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-05-27T08:06:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-05-27T08:06:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:06:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T08:06:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-05-27T08:06:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T08:06:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T08:06:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-05-27T08:06:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-05-27T08:06:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-05-27T08:06:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-05-27T08:06:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.551 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.552 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.553 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.554 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.555 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.556 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.557 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.558 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.558 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.558 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.558 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.558 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.558 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.558 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.558 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.558 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.558 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.559 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.560 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.561 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.562 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.563 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.564 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.565 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.567 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.568 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.569 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.569 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.569 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.569 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.569 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.569 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.569 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.569 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.569 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.569 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-05-27T08:06:00.571 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.572 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.573 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.574 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.574 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.574 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.574 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-05-27T08:06:00.574 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-05-27T08:06:00.574 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.574 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.574 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.574 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.574 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.575 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.576 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-05-27T08:06:00.577 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.578 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.579 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-05-27T08:06:00.580 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.581 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-05-27T08:06:00.582 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.583 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.584 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.585 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smb", 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.586 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.587 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.589 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.590 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.590 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.590 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.590 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.590 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.590 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.590 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.590 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.590 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.590 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.591 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.592 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.593 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.594 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.595 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "update_orchestration": { 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "update_orchestration", 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.597 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.598 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.599 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.600 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.601 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.602 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.603 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.604 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.605 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.606 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.607 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.608 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.609 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.610 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.611 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.612 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-05-27T08:06:00.613 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.614 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.615 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.616 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.617 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.618 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.619 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.620 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.621 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.622 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.623 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.624 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.627 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.628 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.629 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.630 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.631 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.632 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-05-27T08:06:00.633 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.634 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.635 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.636 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.637 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.638 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.639 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-05-27T08:06:00.640 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.641 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.642 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.643 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.644 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.645 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.646 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.647 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.648 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.649 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.650 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.651 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.652 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.653 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.654 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.655 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.656 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.657 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.658 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.659 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.660 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.661 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.662 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.663 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.664 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.665 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.666 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.667 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T08:06:00.669 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.670 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.671 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-05-27T08:06:00.672 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.673 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-05-27T08:06:00.674 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.675 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.676 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.677 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-05-27T08:06:00.678 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.679 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.680 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.681 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.682 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.683 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-05-27T08:06:00.684 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: "services": { 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: "dashboard": "https://172.21.15.90:8443/" 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: "always_on_modules": { 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: "octopus": [ 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-05-27T08:06:00.685 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "pacific": [ 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-05-27T08:06:00.686 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "quincy": [ 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-05-27T08:06:00.687 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "reef": [ 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-05-27T08:06:00.688 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "squid": [ 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.689 INFO:teuthology.orchestra.run.smithi090.stdout: "last_failure_osd_epoch": 92, 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: "active_clients": [ 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3504395597 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.690 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "libcephsqlite", 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 318882068 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-05-27T08:06:00.691 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3034137665 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 4138761333 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:06:00.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 4082587902 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-05-27T08:06:00.693 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-05-27T08:06:00.694 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-05-27T08:06:00.694 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 1278729432 2024-05-27T08:06:00.694 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.694 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:06:00.694 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:00.694 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-05-27T08:06:00.694 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:06:00.695 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:58 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1191187861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T08:06:00.695 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:59 smithi129 ceph-mon[35254]: pgmap v631: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-27T08:06:00.695 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:05:59 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2903827256' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T08:06:00.695 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:58 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1191187861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T08:06:00.695 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:59 smithi136 ceph-mon[35324]: pgmap v631: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-27T08:06:00.696 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:05:59 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2903827256' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T08:06:00.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.957+0000 7f3bf2600640 1 -- 172.21.15.90:0/2903827256 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3bd00b29e0 msgr2=0x7f3bd00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.957+0000 7f3bf2600640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3bd00b29e0 0x7f3bd00b4ef0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4005fd0 tx=0x7f3bd400bf30 comp rx=0 tx=0).stop 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.957+0000 7f3bf2600640 1 -- 172.21.15.90:0/2903827256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec10c2f0 msgr2=0x7f3bec19c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.957+0000 7f3bf2600640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec10c2f0 0x7f3bec19c7c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3be400d750 tx=0x7f3be400dc10 comp rx=0 tx=0).stop 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.958+0000 7f3bf2600640 1 -- 172.21.15.90:0/2903827256 shutdown_connections 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.958+0000 7f3bf2600640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3bd00b29e0 0x7f3bd00b4ef0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.958+0000 7f3bf2600640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3bec10be50 0x7f3bec1a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.958+0000 7f3bf2600640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bec10c2f0 0x7f3bec19c7c0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.958+0000 7f3bf2600640 1 --2- 172.21.15.90:0/2903827256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bec1065a0 0x7f3bec1a2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.958+0000 7f3bf2600640 1 -- 172.21.15.90:0/2903827256 >> 172.21.15.90:0/2903827256 conn(0x7f3bec077080 msgr2=0x7f3bec1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.958+0000 7f3bf2600640 1 -- 172.21.15.90:0/2903827256 shutdown_connections 2024-05-27T08:06:00.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:05:58.958+0000 7f3bf2600640 1 -- 172.21.15.90:0/2903827256 wait complete. 2024-05-27T08:06:00.749 DEBUG:tasks.check_counter:Getting stats from a 2024-05-27T08:06:00.749 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-27T08:06:00.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:58 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1191187861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-27T08:06:00.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:59 smithi090 ceph-mon[31429]: pgmap v631: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.7 KiB/s wr, 7 op/s 2024-05-27T08:06:00.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:05:59 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2903827256' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T08:06:01.201 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:01.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:01 smithi090 ceph-mon[31429]: pgmap v632: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-27T08:06:02.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:01 smithi136 ceph-mon[35324]: pgmap v632: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-27T08:06:02.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:01 smithi129 ceph-mon[35254]: pgmap v632: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-05-27T08:06:04.087 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:03 smithi090 ceph-mon[31429]: pgmap v633: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-27T08:06:04.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:03 smithi136 ceph-mon[35324]: pgmap v633: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-27T08:06:04.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:03 smithi129 ceph-mon[35254]: pgmap v633: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-05-27T08:06:04.631 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 2819, 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 2819, 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 846861, 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 895619, 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 6, 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.558801205, 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.181420718, 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.129671504, 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.185609319, 2024-05-27T08:06:04.632 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2818, 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.487276239, 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000172915 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.633 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 1181, 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 1486, 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 291006, 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 1059070, 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 8, 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 5, 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.284138523, 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.103234407, 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.053739712, 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.097595445, 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:06:04.634 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1483, 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.277534945, 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000187144 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 684080 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:06:04.635 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 484, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 444, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 1999308, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 256206, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.179704674, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.077398110, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.105214235, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.020208137, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 443, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.076869341, 2024-05-27T08:06:04.636 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000173519 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1999040, 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 255888 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-05-27T08:06:04.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 6514, 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.387996293, 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000059563 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-05-27T08:06:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 0, 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 0, 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 0, 2024-05-27T08:06:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 1, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-05-27T08:06:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:06:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 0, 2024-05-27T08:06:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 0, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 0 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 0, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 0, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 0, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 0, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 0, 2024-05-27T08:06:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_path": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_inode": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-05-27T08:06:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 0, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 8, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 0, 2024-05-27T08:06:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 2, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 2, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 4196824, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 4196683, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 8 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 3, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 3, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 0, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 1, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 1, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 0, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 0, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 0, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 0, 2024-05-27T08:06:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 0, 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 0, 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 0, 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 117632, 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 215440 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 0, 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 0, 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 0, 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 0, 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 0, 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-05-27T08:06:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.652 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.653 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.654 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.654 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-05-27T08:06:04.654 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.654 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.655 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.656 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-05-27T08:06:04.657 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.658 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.659 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.659 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-05-27T08:06:04.659 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.659 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.659 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.659 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.659 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-05-27T08:06:04.659 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.659 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.659 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.660 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 2, 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 4, 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 2, 2024-05-27T08:06:04.662 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 4, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 0, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 0, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 4480, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:06:04.663 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:06:04.664 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-05-27T08:06:04.665 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 198961, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 56, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:06:04.666 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 9512, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 182, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 8512, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 13, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.667 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 3995, 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 21, 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 3, 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 3995, 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3995, 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 6.203905800, 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001552917 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-05-27T08:06:04.668 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3995, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 3996 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 3995, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 3995, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 0, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 2580, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 0, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 1413, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 0, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 0, 2024-05-27T08:06:04.669 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-05-27T08:06:04.670 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-05-27T08:06:04.670 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-05-27T08:06:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-05-27T08:06:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 0, 2024-05-27T08:06:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-05-27T08:06:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-05-27T08:06:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-05-27T08:06:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 1, 2024-05-27T08:06:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 0, 2024-05-27T08:06:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:06:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-05-27T08:06:04.671 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-05-27T08:06:04.672 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 125, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 36, 2024-05-27T08:06:04.673 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 5, 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 7, 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 2, 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 0, 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 2, 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 0 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 0, 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-05-27T08:06:04.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "osd-slow-ops": { 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 0, 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 0, 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 0, 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-05-27T08:06:04.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 4480, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 2706560, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 4480, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 4480, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2706560, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.676 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 3995, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 2525, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 3995, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 2525, 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-05-27T08:06:04.677 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-05-27T08:06:04.678 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 3995, 2024-05-27T08:06:04.679 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 3995, 2024-05-27T08:06:04.679 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 3995, 2024-05-27T08:06:04.679 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 3995, 2024-05-27T08:06:04.679 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-05-27T08:06:04.679 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.679 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.679 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.679 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:04.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-05-27T08:06:04.680 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x55c142c82ea0": { 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-05-27T08:06:04.681 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-05-27T08:06:04.682 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-05-27T08:06:04.682 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-05-27T08:06:04.682 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-05-27T08:06:04.682 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-05-27T08:06:04.682 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-05-27T08:06:04.682 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-05-27T08:06:04.682 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-05-27T08:06:04.682 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:04.682 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-05-27T08:06:04.682 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-05-27T08:06:05.777 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-27T08:06:05.777 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-27T08:06:05.778 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-27T08:06:05.778 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-27T08:06:05.778 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-27T08:06:05.778 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:06:05.778 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:06:05.778 DEBUG:tasks.check_counter:Getting stats from d 2024-05-27T08:06:05.778 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-27T08:06:06.207 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:06.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:05 smithi090 ceph-mon[31429]: pgmap v634: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-27T08:06:06.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:05 smithi136 ceph-mon[35324]: pgmap v634: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-27T08:06:06.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:05 smithi129 ceph-mon[35254]: pgmap v634: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-05-27T08:06:07.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:07 smithi090 ceph-mon[31429]: pgmap v635: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-05-27T08:06:08.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:07 smithi136 ceph-mon[35324]: pgmap v635: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-05-27T08:06:08.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:07 smithi129 ceph-mon[35254]: pgmap v635: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-05-27T08:06:08.745 INFO:teuthology.orchestra.run.smithi090.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-05-27T08:06:09.774 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-27T08:06:09.775 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi090 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump' 2024-05-27T08:06:09.777 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-27T08:06:09.789 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-27T08:06:09.789 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T08:06:09.790 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T08:06:09.871 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T08:06:09.871 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T08:06:09.958 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T08:06:10.069 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:09 smithi090 ceph-mon[31429]: pgmap v636: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-05-27T08:06:10.069 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:09 smithi090 ceph-mon[31429]: osdmap e126: 12 total, 12 up, 12 in 2024-05-27T08:06:10.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:09 smithi136 ceph-mon[35324]: pgmap v636: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-05-27T08:06:10.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:09 smithi136 ceph-mon[35324]: osdmap e126: 12 total, 12 up, 12 in 2024-05-27T08:06:10.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:09 smithi129 ceph-mon[35254]: pgmap v636: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-05-27T08:06:10.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:09 smithi129 ceph-mon[35254]: osdmap e126: 12 total, 12 up, 12 in 2024-05-27T08:06:10.410 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:12.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:11 smithi090 ceph-mon[31429]: pgmap v638: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:06:12.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:11 smithi136 ceph-mon[35324]: pgmap v638: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:06:12.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:11 smithi129 ceph-mon[35254]: pgmap v638: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:06:12.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.899+0000 7f3eae200700 1 -- 172.21.15.90:0/4119053003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8115170 msgr2=0x7f3ea8117590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.899+0000 7f3eae200700 1 --2- 172.21.15.90:0/4119053003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8115170 0x7f3ea8117590 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000b0a0 tx=0x7f3ea000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.900+0000 7f3eae200700 1 -- 172.21.15.90:0/4119053003 shutdown_connections 2024-05-27T08:06:12.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.900+0000 7f3eae200700 1 --2- 172.21.15.90:0/4119053003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8115170 0x7f3ea8117590 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.900+0000 7f3eae200700 1 --2- 172.21.15.90:0/4119053003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea8075b50 0x7f3ea8075f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.900+0000 7f3eae200700 1 --2- 172.21.15.90:0/4119053003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ea80764a0 0x7f3ea8114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.900+0000 7f3eae200700 1 -- 172.21.15.90:0/4119053003 >> 172.21.15.90:0/4119053003 conn(0x7f3ea8079d00 msgr2=0x7f3ea807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:12.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.900+0000 7f3eae200700 1 -- 172.21.15.90:0/4119053003 shutdown_connections 2024-05-27T08:06:12.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.900+0000 7f3eae200700 1 -- 172.21.15.90:0/4119053003 wait complete. 2024-05-27T08:06:12.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.901+0000 7f3eae200700 1 Processor -- start 2024-05-27T08:06:12.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.901+0000 7f3eae200700 1 -- start start 2024-05-27T08:06:12.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3eae200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8075b50 0x7f3ea81a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3eae200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ea80764a0 0x7f3ea81a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3eae200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea8115170 0x7f3ea81a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3eae200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea811a720 con 0x7f3ea8075b50 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3eae200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea811a5a0 con 0x7f3ea80764a0 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3eae200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea811a8a0 con 0x7f3ea8115170 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3eace00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8075b50 0x7f3ea81a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3eace00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8075b50 0x7f3ea81a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33394/0 (socket says 172.21.15.90:33394) 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3eace00700 1 -- 172.21.15.90:0/2514920288 learned_addr learned my addr 172.21.15.90:0/2514920288 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3ea7e00700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ea80764a0 0x7f3ea81a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.902+0000 7f3ead800700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea8115170 0x7f3ea81a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3eace00700 1 -- 172.21.15.90:0/2514920288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea8115170 msgr2=0x7f3ea81a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3eace00700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea8115170 0x7f3ea81a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3eace00700 1 -- 172.21.15.90:0/2514920288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ea80764a0 msgr2=0x7f3ea81a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:12.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3eace00700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ea80764a0 0x7f3ea81a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:12.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3eace00700 1 -- 172.21.15.90:0/2514920288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea81a5fc0 con 0x7f3ea8075b50 2024-05-27T08:06:12.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3ead800700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea8115170 0x7f3ea81a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:06:12.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3eace00700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8075b50 0x7f3ea81a10f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00d950 tx=0x7f3e9c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:12.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3ea5600700 1 -- 172.21.15.90:0/2514920288 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c016070 con 0x7f3ea8075b50 2024-05-27T08:06:12.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3ea5600700 1 -- 172.21.15.90:0/2514920288 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e9c010460 con 0x7f3ea8075b50 2024-05-27T08:06:12.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3ea5600700 1 -- 172.21.15.90:0/2514920288 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00f620 con 0x7f3ea8075b50 2024-05-27T08:06:12.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.903+0000 7f3eae200700 1 -- 172.21.15.90:0/2514920288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea81a6250 con 0x7f3ea8075b50 2024-05-27T08:06:12.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.904+0000 7f3eae200700 1 -- 172.21.15.90:0/2514920288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea81adbd0 con 0x7f3ea8075b50 2024-05-27T08:06:12.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.905+0000 7f3eae200700 1 -- 172.21.15.90:0/2514920288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea8077290 con 0x7f3ea8075b50 2024-05-27T08:06:12.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.907+0000 7f3ea5600700 1 -- 172.21.15.90:0/2514920288 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3e9c019070 con 0x7f3ea8075b50 2024-05-27T08:06:12.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.909+0000 7f3ea5600700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e900b2a70 0x7f3e900b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:12.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.909+0000 7f3ea5600700 1 -- 172.21.15.90:0/2514920288 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9584+0+0 (secure 0 0 0) 0x7f3e9c0e2600 con 0x7f3ea8075b50 2024-05-27T08:06:12.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.909+0000 7f3ea7e00700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e900b2a70 0x7f3e900b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:12.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.910+0000 7f3ea7e00700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e900b2a70 0x7f3e900b4f80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3e94005950 tx=0x7f3e940058e0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:12.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:12.912+0000 7f3ea5600700 1 -- 172.21.15.90:0/2514920288 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e9c0a9bc0 con 0x7f3ea8075b50 2024-05-27T08:06:13.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.163+0000 7f3eae200700 1 -- 172.21.15.90:0/2514920288 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3ea81a2450 con 0x7f3ea8075b50 2024-05-27T08:06:13.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.164+0000 7f3ea5600700 1 -- 172.21.15.90:0/2514920288 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v126) ==== 81+0+943 (secure 0 0 0) 0x7f3e9c0a94c0 con 0x7f3ea8075b50 2024-05-27T08:06:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/2854405112 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/2854405112 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3526000120 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/836242224 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2932841075 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1476836587 2024-05-28T07:30:58.154487+0000 2024-05-27T08:06:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3178069601 2024-05-28T07:30:58.154487+0000 2024-05-27T08:06:13.166 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/2742579272 2024-05-28T07:30:58.154487+0000 2024-05-27T08:06:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/2742579272 2024-05-28T07:30:58.154487+0000 2024-05-27T08:06:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/410158065 2024-05-28T07:30:23.548910+0000 2024-05-27T08:06:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2593454412 2024-05-28T07:30:23.548910+0000 2024-05-27T08:06:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3534244243 2024-05-28T07:30:23.548910+0000 2024-05-27T08:06:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2692415447 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2575484267 2024-05-28T07:30:23.548910+0000 2024-05-27T08:06:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/410158065 2024-05-28T07:30:23.548910+0000 2024-05-27T08:06:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2626688812 2024-05-28T07:30:58.154487+0000 2024-05-27T08:06:13.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.167+0000 7f3eae200700 1 -- 172.21.15.90:0/2514920288 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e900b2a70 msgr2=0x7f3e900b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:13.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.167+0000 7f3eae200700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e900b2a70 0x7f3e900b4f80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3e94005950 tx=0x7f3e940058e0 comp rx=0 tx=0).stop 2024-05-27T08:06:13.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.167+0000 7f3eae200700 1 -- 172.21.15.90:0/2514920288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8075b50 msgr2=0x7f3ea81a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:13.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.167+0000 7f3eae200700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8075b50 0x7f3ea81a10f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00d950 tx=0x7f3e9c00dd10 comp rx=0 tx=0).stop 2024-05-27T08:06:13.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.168+0000 7f3eae200700 1 -- 172.21.15.90:0/2514920288 shutdown_connections 2024-05-27T08:06:13.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.168+0000 7f3eae200700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea8115170 0x7f3ea81a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:13.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.168+0000 7f3eae200700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ea8075b50 0x7f3ea81a10f0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:13.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.168+0000 7f3eae200700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7f3e900b2a70 0x7f3e900b4f80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:13.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.168+0000 7f3eae200700 1 --2- 172.21.15.90:0/2514920288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ea80764a0 0x7f3ea81a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:13.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.168+0000 7f3eae200700 1 -- 172.21.15.90:0/2514920288 >> 172.21.15.90:0/2514920288 conn(0x7f3ea8079d00 msgr2=0x7f3ea8100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:13.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.168+0000 7f3eae200700 1 -- 172.21.15.90:0/2514920288 shutdown_connections 2024-05-27T08:06:13.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:13.168+0000 7f3eae200700 1 -- 172.21.15.90:0/2514920288 wait complete. 2024-05-27T08:06:13.170 INFO:teuthology.orchestra.run.smithi090.stderr:listed 16 entries 2024-05-27T08:06:14.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:13 smithi090 ceph-mon[31429]: pgmap v639: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:06:14.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:13 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/2514920288' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:06:14.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:13 smithi136 ceph-mon[35324]: pgmap v639: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:06:14.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:13 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/2514920288' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:06:14.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:13 smithi129 ceph-mon[35254]: pgmap v639: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:06:14.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:13 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/2514920288' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:06:14.921 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-05-27T08:06:14.922 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T08:06:14.982 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T08:06:15.431 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:06:16.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:15 smithi090 ceph-mon[31429]: pgmap v640: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-05-27T08:06:16.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:15 smithi136 ceph-mon[35324]: pgmap v640: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-05-27T08:06:16.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:15 smithi129 ceph-mon[35254]: pgmap v640: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 30 KiB/s wr, 11 op/s 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.941+0000 7fb52f200700 1 -- 172.21.15.90:0/3574946071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb528109840 msgr2=0x7fb5281111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.941+0000 7fb52f200700 1 --2- 172.21.15.90:0/3574946071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb528109840 0x7fb5281111a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb52000b0d0 tx=0x7fb52000b3e0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.941+0000 7fb52f200700 1 -- 172.21.15.90:0/3574946071 shutdown_connections 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.941+0000 7fb52f200700 1 --2- 172.21.15.90:0/3574946071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb528109840 0x7fb5281111a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.941+0000 7fb52f200700 1 --2- 172.21.15.90:0/3574946071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5281045c0 0x7fb5281049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.941+0000 7fb52f200700 1 --2- 172.21.15.90:0/3574946071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb528108ef0 0x7fb528109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.941+0000 7fb52f200700 1 -- 172.21.15.90:0/3574946071 >> 172.21.15.90:0/3574946071 conn(0x7fb5280ffe90 msgr2=0x7fb5281022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.941+0000 7fb52f200700 1 -- 172.21.15.90:0/3574946071 shutdown_connections 2024-05-27T08:06:17.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.941+0000 7fb52f200700 1 -- 172.21.15.90:0/3574946071 wait complete. 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.942+0000 7fb52f200700 1 Processor -- start 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.942+0000 7fb52f200700 1 -- start start 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.943+0000 7fb52f200700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5281045c0 0x7fb52819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.943+0000 7fb52f200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb528108ef0 0x7fb52819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.943+0000 7fb52f200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb528109840 0x7fb5281a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.943+0000 7fb52f200700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb528114320 con 0x7fb5281045c0 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.943+0000 7fb52f200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281141a0 con 0x7fb528109840 2024-05-27T08:06:17.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.943+0000 7fb52f200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281144a0 con 0x7fb528108ef0 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.943+0000 7fb527e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5281045c0 0x7fb52819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.943+0000 7fb527e00700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5281045c0 0x7fb52819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33406/0 (socket says 172.21.15.90:33406) 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.943+0000 7fb527e00700 1 -- 172.21.15.90:0/1065505347 learned_addr learned my addr 172.21.15.90:0/1065505347 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.943+0000 7fb527400700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb528108ef0 0x7fb52819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.944+0000 7fb52d200700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb528109840 0x7fb5281a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.944+0000 7fb527e00700 1 -- 172.21.15.90:0/1065505347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb528108ef0 msgr2=0x7fb52819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.944+0000 7fb527e00700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb528108ef0 0x7fb52819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.944+0000 7fb527e00700 1 -- 172.21.15.90:0/1065505347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb528109840 msgr2=0x7fb5281a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.944+0000 7fb527e00700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb528109840 0x7fb5281a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.944+0000 7fb527e00700 1 -- 172.21.15.90:0/1065505347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5281a1dc0 con 0x7fb5281045c0 2024-05-27T08:06:17.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.944+0000 7fb527e00700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5281045c0 0x7fb52819cef0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb51800d950 tx=0x7fb51800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.944+0000 7fb524c00700 1 -- 172.21.15.90:0/1065505347 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518016070 con 0x7fb5281045c0 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.944+0000 7fb524c00700 1 -- 172.21.15.90:0/1065505347 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb518010460 con 0x7fb5281045c0 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.944+0000 7fb524c00700 1 -- 172.21.15.90:0/1065505347 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51800f620 con 0x7fb5281045c0 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.945+0000 7fb52f200700 1 -- 172.21.15.90:0/1065505347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5281a20b0 con 0x7fb5281045c0 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.945+0000 7fb52f200700 1 -- 172.21.15.90:0/1065505347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5281a99d0 con 0x7fb5281045c0 2024-05-27T08:06:17.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.946+0000 7fb52f200700 1 -- 172.21.15.90:0/1065505347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5280512b0 con 0x7fb5281045c0 2024-05-27T08:06:17.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.948+0000 7fb524c00700 1 -- 172.21.15.90:0/1065505347 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb518019070 con 0x7fb5281045c0 2024-05-27T08:06:17.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.950+0000 7fb524c00700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 0x7fb50c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:06:17.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.950+0000 7fb524c00700 1 -- 172.21.15.90:0/1065505347 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9496+0+0 (secure 0 0 0) 0x7fb5180e2580 con 0x7fb5281045c0 2024-05-27T08:06:17.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.950+0000 7fb527400700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 0x7fb50c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:06:17.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.951+0000 7fb527400700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 0x7fb50c0b4f80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb514005fd0 tx=0x7fb514005db0 comp rx=0 tx=0).ready entity=mgr.34967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:06:17.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:17.955+0000 7fb524c00700 1 -- 172.21.15.90:0/1065505347 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5180a9b90 con 0x7fb5281045c0 2024-05-27T08:06:18.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.210+0000 7fb52f200700 1 -- 172.21.15.90:0/1065505347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb5281a9ba0 con 0x7fb5281045c0 2024-05-27T08:06:18.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:17 smithi090 ceph-mon[31429]: pgmap v641: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-27T08:06:18.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:17 smithi090 ceph-mon[31429]: osdmap e127: 12 total, 12 up, 12 in 2024-05-27T08:06:18.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.211+0000 7fb524c00700 1 -- 172.21.15.90:0/1065505347 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v127) ==== 81+0+943 (secure 0 0 0) 0x7fb5180a9490 con 0x7fb5281045c0 2024-05-27T08:06:18.213 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/2854405112 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:18.213 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/2854405112 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:18.213 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3526000120 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/836242224 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2932841075 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1476836587 2024-05-28T07:30:58.154487+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3178069601 2024-05-28T07:30:58.154487+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/2742579272 2024-05-28T07:30:58.154487+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/2742579272 2024-05-28T07:30:58.154487+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/410158065 2024-05-28T07:30:23.548910+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2593454412 2024-05-28T07:30:23.548910+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3534244243 2024-05-28T07:30:23.548910+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2692415447 2024-05-28T07:46:26.839737+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2575484267 2024-05-28T07:30:23.548910+0000 2024-05-27T08:06:18.214 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/410158065 2024-05-28T07:30:23.548910+0000 2024-05-27T08:06:18.215 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2626688812 2024-05-28T07:30:58.154487+0000 2024-05-27T08:06:18.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.215+0000 7fb52f200700 1 -- 172.21.15.90:0/1065505347 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 msgr2=0x7fb50c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:18.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.215+0000 7fb52f200700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 0x7fb50c0b4f80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb514005fd0 tx=0x7fb514005db0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.215+0000 7fb52f200700 1 -- 172.21.15.90:0/1065505347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5281045c0 msgr2=0x7fb52819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:06:18.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.215+0000 7fb52f200700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5281045c0 0x7fb52819cef0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb51800d950 tx=0x7fb51800dd10 comp rx=0 tx=0).stop 2024-05-27T08:06:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.216+0000 7fb52f200700 1 -- 172.21.15.90:0/1065505347 shutdown_connections 2024-05-27T08:06:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.216+0000 7fb52f200700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.90:6800/4256436582,v1:172.21.15.90:6801/4256436582] conn(0x7fb50c0b2a70 0x7fb50c0b4f80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.216+0000 7fb52f200700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb528109840 0x7fb5281a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.216+0000 7fb52f200700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5281045c0 0x7fb52819cef0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.216+0000 7fb52f200700 1 --2- 172.21.15.90:0/1065505347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb528108ef0 0x7fb52819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:06:18.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.216+0000 7fb52f200700 1 -- 172.21.15.90:0/1065505347 >> 172.21.15.90:0/1065505347 conn(0x7fb5280ffe90 msgr2=0x7fb52810a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:06:18.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.216+0000 7fb52f200700 1 -- 172.21.15.90:0/1065505347 shutdown_connections 2024-05-27T08:06:18.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-05-27T08:06:18.216+0000 7fb52f200700 1 -- 172.21.15.90:0/1065505347 wait complete. 2024-05-27T08:06:18.220 INFO:teuthology.orchestra.run.smithi090.stderr:listed 16 entries 2024-05-27T08:06:18.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:17 smithi136 ceph-mon[35324]: pgmap v641: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-27T08:06:18.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:17 smithi136 ceph-mon[35324]: osdmap e127: 12 total, 12 up, 12 in 2024-05-27T08:06:18.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:17 smithi129 ceph-mon[35254]: pgmap v641: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-27T08:06:18.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:17 smithi129 ceph-mon[35254]: osdmap e127: 12 total, 12 up, 12 in 2024-05-27T08:06:19.077 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi090.front.sepia.ceph.com... 2024-05-27T08:06:19.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:06:19.077 DEBUG:teuthology.orchestra.run.smithi090:> sudo fusermount -u /tmp/cephfs 2024-05-27T08:06:19.181 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[90454]: fuse finished with error 0 and tester_r 0 2024-05-27T08:06:19.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:18 smithi090 ceph-mon[31429]: from='client.? 172.21.15.90:0/1065505347' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:06:19.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:18 smithi136 ceph-mon[35324]: from='client.? 172.21.15.90:0/1065505347' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:06:19.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:18 smithi129 ceph-mon[35254]: from='client.? 172.21.15.90:0/1065505347' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:06:19.457 INFO:teuthology.orchestra.run:waiting for 300 2024-05-27T08:06:20.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:19 smithi090 ceph-mon[31429]: pgmap v643: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-05-27T08:06:20.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:19 smithi136 ceph-mon[35324]: pgmap v643: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-05-27T08:06:20.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:19 smithi129 ceph-mon[35254]: pgmap v643: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-05-27T08:06:22.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:21 smithi090 ceph-mon[31429]: pgmap v644: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-05-27T08:06:22.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:21 smithi136 ceph-mon[35324]: pgmap v644: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-05-27T08:06:22.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:21 smithi129 ceph-mon[35254]: pgmap v644: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-05-27T08:06:24.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:23 smithi136 ceph-mon[35324]: pgmap v645: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-05-27T08:06:24.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:23 smithi129 ceph-mon[35254]: pgmap v645: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-05-27T08:06:24.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:23 smithi090 ceph-mon[31429]: pgmap v645: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-05-27T08:06:25.558 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T08:06:25.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:06:25.559 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-05-27T08:06:25.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:06:25.632 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-05-27T08:06:25.731 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-05-27T08:06:25.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:06:25.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-05-27T08:06:25.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:06:25.820 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-05-27T08:06:25.927 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-27T08:06:25.940 INFO:tasks.cephadm:Teardown begin 2024-05-27T08:06:25.941 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/cephadm.py", line 2270, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi090 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump' 2024-05-27T08:06:25.942 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:06:26.023 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:06:26.077 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:25 smithi090 ceph-mon[31429]: pgmap v646: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-05-27T08:06:26.224 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:25 smithi129 ceph-mon[35254]: pgmap v646: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-05-27T08:06:26.240 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:06:26.250 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:25 smithi136 ceph-mon[35324]: pgmap v646: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-05-27T08:06:26.425 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-27T08:06:26.425 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T08:06:26.498 DEBUG:teuthology.orchestra.run.smithi129:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T08:06:26.555 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T08:06:26.611 INFO:tasks.cephadm:Stopping all daemons... 2024-05-27T08:06:26.611 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-27T08:06:26.611 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.a 2024-05-27T08:06:26.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:26 smithi090 systemd[1]: Stopping Ceph mon.a for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:06:27.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a[31425]: 2024-05-27T08:06:27.195+0000 7fdce6c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:06:27.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a[31425]: 2024-05-27T08:06:27.195+0000 7fdce6c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-27T08:06:27.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:27 smithi090 podman[133400]: 2024-05-27 08:06:27.329402981 +0000 UTC m=+0.280105546 container died 217e3dbbb3134b31b0553cfe6ba5d74730b6c90e0cfd8fd03f76cb29fffb8223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-27T08:06:28.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:27 smithi090 podman[133400]: 2024-05-27 08:06:27.895717379 +0000 UTC m=+0.846419941 container cleanup 217e3dbbb3134b31b0553cfe6ba5d74730b6c90e0cfd8fd03f76cb29fffb8223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-27T08:06:28.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:27 smithi090 bash[133400]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a 2024-05-27T08:06:28.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:May 27 08:06:28 smithi090 podman[133414]: 2024-05-27 08:06:28.132557075 +0000 UTC m=+0.790190828 container remove 217e3dbbb3134b31b0553cfe6ba5d74730b6c90e0cfd8fd03f76cb29fffb8223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-27T08:06:28.449 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.a.service' 2024-05-27T08:06:28.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:06:28.579 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-27T08:06:28.579 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-27T08:06:28.579 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.b 2024-05-27T08:06:29.065 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:28 smithi129 systemd[1]: Stopping Ceph mon.b for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:06:29.565 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:29 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-b[35250]: 2024-05-27T08:06:29.240+0000 7f2c65a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:06:29.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:29 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-b[35250]: 2024-05-27T08:06:29.240+0000 7f2c65a00700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-27T08:06:29.566 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:29 smithi129 podman[72014]: 2024-05-27 08:06:29.415932664 +0000 UTC m=+0.369097420 container died 714573e46108e9f94f2c5e187085efcddfc01454ca1d0f990d16a014d64cddb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-b, org.label-schema.build-date=20240521, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-27T08:06:30.082 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:30 smithi129 podman[72014]: 2024-05-27 08:06:30.073973853 +0000 UTC m=+1.027138606 container cleanup 714573e46108e9f94f2c5e187085efcddfc01454ca1d0f990d16a014d64cddb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-27T08:06:30.082 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:30 smithi129 bash[72014]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-b 2024-05-27T08:06:30.815 INFO:journalctl@ceph.mon.b.smithi129.stdout:May 27 08:06:30 smithi129 podman[72025]: 2024-05-27 08:06:30.384377663 +0000 UTC m=+0.952821131 container remove 714573e46108e9f94f2c5e187085efcddfc01454ca1d0f990d16a014d64cddb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-b, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-27T08:06:30.921 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.b.service' 2024-05-27T08:06:31.060 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:06:31.061 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-27T08:06:31.061 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-27T08:06:31.061 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.c 2024-05-27T08:06:31.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:31 smithi136 systemd[1]: Stopping Ceph mon.c for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:06:32.100 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-c[35319]: 2024-05-27T08:06:31.840+0000 7f30fe800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:06:32.100 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-c[35319]: 2024-05-27T08:06:31.840+0000 7f30fe800700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-27T08:06:32.390 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:32 smithi136 podman[73931]: 2024-05-27 08:06:32.099359869 +0000 UTC m=+0.503045198 container died 8986a2754a27244f159dae5aa81c04cc2b5896b038b6e40fc308967753e37f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:06:32.650 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:32 smithi136 podman[73931]: 2024-05-27 08:06:32.627683457 +0000 UTC m=+1.031368789 container cleanup 8986a2754a27244f159dae5aa81c04cc2b5896b038b6e40fc308967753e37f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-c, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:06:32.651 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:32 smithi136 bash[73931]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-c 2024-05-27T08:06:33.251 INFO:journalctl@ceph.mon.c.smithi136.stdout:May 27 08:06:32 smithi136 podman[73943]: 2024-05-27 08:06:32.954891286 +0000 UTC m=+0.838430825 container remove 8986a2754a27244f159dae5aa81c04cc2b5896b038b6e40fc308967753e37f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-05-27T08:06:33.492 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mon.c.service' 2024-05-27T08:06:33.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:06:33.617 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-27T08:06:33.618 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-27T08:06:33.618 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.x 2024-05-27T08:06:33.964 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 08:06:33 smithi090 systemd[1]: Stopping Ceph mgr.x for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:06:34.465 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 08:06:34 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T08:06:34.184+0000 7f4f0f200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:06:34.465 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 08:06:34 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x[31657]: 2024-05-27T08:06:34.184+0000 7f4f0f200700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-27T08:06:34.465 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 08:06:34 smithi090 podman[133520]: 2024-05-27 08:06:34.289934041 +0000 UTC m=+0.298620294 container died da1e5558b99ab619c617cb673d704721834096a11216cde713f8d5f33ca1034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:06:35.214 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 08:06:34 smithi090 podman[133520]: 2024-05-27 08:06:34.954966867 +0000 UTC m=+0.963653083 container cleanup da1e5558b99ab619c617cb673d704721834096a11216cde713f8d5f33ca1034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T08:06:35.214 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 08:06:34 smithi090 bash[133520]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x 2024-05-27T08:06:35.214 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 08:06:35 smithi090 podman[133532]: 2024-05-27 08:06:35.183361579 +0000 UTC m=+0.864764134 container remove da1e5558b99ab619c617cb673d704721834096a11216cde713f8d5f33ca1034d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-x, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T08:06:35.622 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.x.service' 2024-05-27T08:06:35.686 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 08:06:35 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.x.service: Deactivated successfully. 2024-05-27T08:06:35.686 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 08:06:35 smithi090 systemd[1]: Stopped Ceph mgr.x for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T08:06:35.686 INFO:journalctl@ceph.mgr.x.smithi090.stdout:May 27 08:06:35 smithi090 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.x.service: Consumed 2min 56.698s CPU time. 2024-05-27T08:06:35.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:06:35.731 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-27T08:06:35.732 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-27T08:06:35.732 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.y 2024-05-27T08:06:36.065 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 08:06:35 smithi129 systemd[1]: Stopping Ceph mgr.y for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:06:36.740 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 08:06:36 smithi129 podman[72146]: 2024-05-27 08:06:36.376522519 +0000 UTC m=+0.207781005 container died d65cd30f1ca7885796c1f524960c7b5d47667eada5f6ac848b02eae2c2f57c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux ) 2024-05-27T08:06:37.065 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 08:06:36 smithi129 podman[72146]: 2024-05-27 08:06:36.922356731 +0000 UTC m=+0.753615213 container cleanup d65cd30f1ca7885796c1f524960c7b5d47667eada5f6ac848b02eae2c2f57c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS) 2024-05-27T08:06:37.065 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 08:06:36 smithi129 bash[72146]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y 2024-05-27T08:06:37.565 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 08:06:37 smithi129 podman[72158]: 2024-05-27 08:06:37.174131575 +0000 UTC m=+0.777709971 container remove d65cd30f1ca7885796c1f524960c7b5d47667eada5f6ac848b02eae2c2f57c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-05-27T08:06:37.565 INFO:journalctl@ceph.mgr.y.smithi129.stdout:May 27 08:06:37 smithi129 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-27T08:06:37.662 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.y.service' 2024-05-27T08:06:37.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:06:37.774 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-27T08:06:37.775 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-27T08:06:37.775 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.z 2024-05-27T08:06:38.251 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 08:06:37 smithi136 systemd[1]: Stopping Ceph mgr.z for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:06:38.751 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 08:06:38 smithi136 podman[74053]: 2024-05-27 08:06:38.473224983 +0000 UTC m=+0.226043177 container died 550d9598331ba373b37628fec767c806384e1b0d46e628974528999503b002df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:06:39.146 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 08:06:38 smithi136 podman[74053]: 2024-05-27 08:06:38.950780797 +0000 UTC m=+0.703598924 container cleanup 550d9598331ba373b37628fec767c806384e1b0d46e628974528999503b002df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-27T08:06:39.146 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 08:06:38 smithi136 bash[74053]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z 2024-05-27T08:06:39.500 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 08:06:39 smithi136 podman[74067]: 2024-05-27 08:06:39.143834782 +0000 UTC m=+0.651803798 container remove 550d9598331ba373b37628fec767c806384e1b0d46e628974528999503b002df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-27T08:06:39.501 INFO:journalctl@ceph.mgr.z.smithi136.stdout:May 27 08:06:39 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-27T08:06:39.620 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mgr.z.service' 2024-05-27T08:06:39.726 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:06:39.727 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-27T08:06:39.727 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-27T08:06:39.727 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.0 2024-05-27T08:06:40.214 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:39 smithi090 systemd[1]: Stopping Ceph osd.0 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:06:40.715 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0[45853]: 2024-05-27T08:06:40.358+0000 7f6a1a200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:06:40.715 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0[45853]: 2024-05-27T08:06:40.358+0000 7f6a1a200700 -1 osd.0 127 *** Got signal Terminated *** 2024-05-27T08:06:40.715 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0[45853]: 2024-05-27T08:06:40.358+0000 7f6a1a200700 -1 osd.0 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:06:46.830 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:46 smithi090 podman[133653]: 2024-05-27 08:06:46.502938538 +0000 UTC m=+6.356542814 container died cd049f999030bf227bcbf18992b65e79567027f788a1853f572428a2e5966934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True) 2024-05-27T08:06:47.464 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:47 smithi090 podman[133653]: 2024-05-27 08:06:47.090878453 +0000 UTC m=+6.944482756 container cleanup cd049f999030bf227bcbf18992b65e79567027f788a1853f572428a2e5966934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-27T08:06:47.464 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:47 smithi090 bash[133653]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0 2024-05-27T08:06:47.964 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:47 smithi090 podman[133671]: 2024-05-27 08:06:47.535437157 +0000 UTC m=+1.002301982 container remove cd049f999030bf227bcbf18992b65e79567027f788a1853f572428a2e5966934 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:06:48.450 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:48 smithi090 podman[133763]: 2024-05-27 08:06:48.068943366 +0000 UTC m=+0.034349210 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:06:48.450 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:48 smithi090 podman[133763]: 2024-05-27 08:06:48.178165593 +0000 UTC m=+0.143571475 container create d70bafbbe2311214537b6adeded1d42d6adcff70ae115f531de94fa46c1754bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-27T08:06:48.715 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:48 smithi090 podman[133763]: 2024-05-27 08:06:48.44919361 +0000 UTC m=+0.414599492 container init d70bafbbe2311214537b6adeded1d42d6adcff70ae115f531de94fa46c1754bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:06:48.715 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:48 smithi090 podman[133763]: 2024-05-27 08:06:48.460283733 +0000 UTC m=+0.425689598 container start d70bafbbe2311214537b6adeded1d42d6adcff70ae115f531de94fa46c1754bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0) 2024-05-27T08:06:48.715 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:48 smithi090 podman[133763]: 2024-05-27 08:06:48.572982509 +0000 UTC m=+0.538388397 container attach d70bafbbe2311214537b6adeded1d42d6adcff70ae115f531de94fa46c1754bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0-deactivate, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-27T08:06:49.634 INFO:journalctl@ceph.osd.0.smithi090.stdout:May 27 08:06:49 smithi090 podman[133763]: 2024-05-27 08:06:49.279611817 +0000 UTC m=+1.245017719 container died d70bafbbe2311214537b6adeded1d42d6adcff70ae115f531de94fa46c1754bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:06:50.065 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.0.service' 2024-05-27T08:06:50.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:06:50.179 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-27T08:06:50.179 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-27T08:06:50.179 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.1 2024-05-27T08:06:50.565 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:50 smithi129 systemd[1]: Stopping Ceph osd.1 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:06:51.066 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1[39822]: 2024-05-27T08:06:50.800+0000 7f7ae8c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:06:51.066 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1[39822]: 2024-05-27T08:06:50.800+0000 7f7ae8c00700 -1 osd.1 127 *** Got signal Terminated *** 2024-05-27T08:06:51.067 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1[39822]: 2024-05-27T08:06:50.800+0000 7f7ae8c00700 -1 osd.1 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:06:57.171 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:56 smithi129 podman[72273]: 2024-05-27 08:06:56.821077808 +0000 UTC m=+6.210167176 container died fa599bc94cdc7a8a74f73d8dbd52bc2647bcaa9f97c38e1c2198d2e6367a494e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-27T08:06:57.565 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:57 smithi129 podman[72273]: 2024-05-27 08:06:57.319974752 +0000 UTC m=+6.709064130 container cleanup fa599bc94cdc7a8a74f73d8dbd52bc2647bcaa9f97c38e1c2198d2e6367a494e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:06:57.565 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:57 smithi129 bash[72273]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1 2024-05-27T08:06:58.064 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:57 smithi129 podman[72290]: 2024-05-27 08:06:57.663535539 +0000 UTC m=+0.812838036 container remove fa599bc94cdc7a8a74f73d8dbd52bc2647bcaa9f97c38e1c2198d2e6367a494e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-27T08:06:58.565 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:58 smithi129 podman[72379]: 2024-05-27 08:06:58.194023237 +0000 UTC m=+0.031933068 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:06:58.565 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:58 smithi129 podman[72379]: 2024-05-27 08:06:58.314652902 +0000 UTC m=+0.152562769 container create 81cc8729321124ac6e2249d72cf40477dccbcb83caec466a456ff42f2f999e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:06:59.065 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:58 smithi129 podman[72379]: 2024-05-27 08:06:58.635585641 +0000 UTC m=+0.473495551 container init 81cc8729321124ac6e2249d72cf40477dccbcb83caec466a456ff42f2f999e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:06:59.066 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:58 smithi129 podman[72379]: 2024-05-27 08:06:58.646982071 +0000 UTC m=+0.484891945 container start 81cc8729321124ac6e2249d72cf40477dccbcb83caec466a456ff42f2f999e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T08:06:59.066 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:58 smithi129 podman[72379]: 2024-05-27 08:06:58.784450159 +0000 UTC m=+0.622360059 container attach 81cc8729321124ac6e2249d72cf40477dccbcb83caec466a456ff42f2f999e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-27T08:07:00.065 INFO:journalctl@ceph.osd.1.smithi129.stdout:May 27 08:06:59 smithi129 podman[72379]: 2024-05-27 08:06:59.78749163 +0000 UTC m=+1.625401498 container died 81cc8729321124ac6e2249d72cf40477dccbcb83caec466a456ff42f2f999e0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS) 2024-05-27T08:07:00.612 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.1.service' 2024-05-27T08:07:00.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:07:00.721 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-27T08:07:00.721 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-27T08:07:00.721 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.2 2024-05-27T08:07:01.223 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:00 smithi136 systemd[1]: Stopping Ceph osd.2 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:07:01.501 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2[40317]: 2024-05-27T08:07:01.300+0000 7f8646c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:07:01.501 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2[40317]: 2024-05-27T08:07:01.300+0000 7f8646c00700 -1 osd.2 127 *** Got signal Terminated *** 2024-05-27T08:07:01.501 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2[40317]: 2024-05-27T08:07:01.300+0000 7f8646c00700 -1 osd.2 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:07:07.751 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:07 smithi136 podman[74176]: 2024-05-27 08:07:07.451188434 +0000 UTC m=+6.297680662 container died 37b3dd9d139ef1976c89dcbfb182f55f8e37490f8ee99545de1110e84f16943c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-27T08:07:08.132 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:07 smithi136 podman[74176]: 2024-05-27 08:07:07.88681581 +0000 UTC m=+6.733308035 container cleanup 37b3dd9d139ef1976c89dcbfb182f55f8e37490f8ee99545de1110e84f16943c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-05-27T08:07:08.132 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:07 smithi136 bash[74176]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2 2024-05-27T08:07:08.133 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:07.958+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:08.500 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:08 smithi136 podman[74203]: 2024-05-27 08:07:08.130087788 +0000 UTC m=+0.642221948 container remove 37b3dd9d139ef1976c89dcbfb182f55f8e37490f8ee99545de1110e84f16943c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-27T08:07:09.086 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:08 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:08.995+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:09.087 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:08 smithi136 podman[74292]: 2024-05-27 08:07:08.687574607 +0000 UTC m=+0.035473435 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:07:09.087 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:08 smithi136 podman[74292]: 2024-05-27 08:07:08.81450212 +0000 UTC m=+0.162400970 container create 51a0bca862eeded02c25fbfc77e31528afcddd6fad6de59e57dcc23548c66626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T08:07:09.223 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:08 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:08.932+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:09.501 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:09 smithi136 podman[74292]: 2024-05-27 08:07:09.124303094 +0000 UTC m=+0.472201971 container init 51a0bca862eeded02c25fbfc77e31528afcddd6fad6de59e57dcc23548c66626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T08:07:09.501 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:09 smithi136 podman[74292]: 2024-05-27 08:07:09.136490797 +0000 UTC m=+0.484389639 container start 51a0bca862eeded02c25fbfc77e31528afcddd6fad6de59e57dcc23548c66626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T08:07:09.501 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:09 smithi136 podman[74292]: 2024-05-27 08:07:09.284503338 +0000 UTC m=+0.632402195 container attach 51a0bca862eeded02c25fbfc77e31528afcddd6fad6de59e57dcc23548c66626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-27T08:07:09.502 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:09 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:09.084+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:10.251 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:10 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:10.106+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:10.251 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:09 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:09.998+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:10.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:09 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:09.971+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:10.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:10 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:10.042+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:10.660 INFO:journalctl@ceph.osd.2.smithi136.stdout:May 27 08:07:10 smithi136 podman[74317]: 2024-05-27 08:07:10.371642593 +0000 UTC m=+0.034995292 container died 51a0bca862eeded02c25fbfc77e31528afcddd6fad6de59e57dcc23548c66626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:07:11.077 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.2.service' 2024-05-27T08:07:11.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:07:11.189 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-27T08:07:11.189 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-27T08:07:11.189 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.3 2024-05-27T08:07:11.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:10 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:10.997+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:11.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:11 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:11.013+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:11.500 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:11 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:11.045+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:11.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:11 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:11.094+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:11.587 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:11 smithi090 systemd[1]: Stopping Ceph osd.3 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:07:11.964 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:11 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3[52519]: 2024-05-27T08:07:11.764+0000 7f667fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:07:11.965 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:11 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3[52519]: 2024-05-27T08:07:11.764+0000 7f667fe00700 -1 osd.3 127 *** Got signal Terminated *** 2024-05-27T08:07:11.965 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:11 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3[52519]: 2024-05-27T08:07:11.764+0000 7f667fe00700 -1 osd.3 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:07:11.965 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:11 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:11.585+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:12.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:11 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:11.635+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:12.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:11 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:11.997+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:12.439 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:12.044+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:12.500 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:12 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:12.017+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:12.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:12 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:12.100+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:12.714 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3[52519]: 2024-05-27T08:07:12.437+0000 7f6676600700 -1 osd.3 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.561099+0000 front 2024-05-27T08:06:40.561252+0000 (oldest deadline 2024-05-27T08:07:11.763120+0000) 2024-05-27T08:07:12.714 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:12.555+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:13.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:12 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:12.608+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:13.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:13 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:13.007+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:13.435 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:13 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:13.080+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:13.500 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:13 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:13.020+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:13.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:13 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:13.092+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:13.714 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:13 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:13.539+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:13.714 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:13 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3[52519]: 2024-05-27T08:07:13.433+0000 7f6676600700 -1 osd.3 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.561099+0000 front 2024-05-27T08:06:40.561252+0000 (oldest deadline 2024-05-27T08:07:11.763120+0000) 2024-05-27T08:07:14.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:13 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:13.627+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:14.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:13 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:13.996+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:14.442 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:14.063+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:14.500 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:14.027+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:14.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:14.111+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:14.714 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3[52519]: 2024-05-27T08:07:14.440+0000 7f6676600700 -1 osd.3 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.561099+0000 front 2024-05-27T08:06:40.561252+0000 (oldest deadline 2024-05-27T08:07:11.763120+0000) 2024-05-27T08:07:14.714 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:14.560+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:15.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:14.661+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:15.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:15 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:15.038+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:15.432 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:15.096+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:15.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:15.020+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:15.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:15.098+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:15.714 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3[52519]: 2024-05-27T08:07:15.431+0000 7f6676600700 -1 osd.3 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.561099+0000 front 2024-05-27T08:06:40.561252+0000 (oldest deadline 2024-05-27T08:07:11.763120+0000) 2024-05-27T08:07:15.714 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:15.520+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:15.983 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:15.678+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:16.251 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:15.981+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:16.251 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:16 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:16.102+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:16.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:16 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:16.010+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:16.440 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:16.075+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:16.714 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3[52519]: 2024-05-27T08:07:16.438+0000 7f6676600700 -1 osd.3 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.561099+0000 front 2024-05-27T08:06:40.561252+0000 (oldest deadline 2024-05-27T08:07:11.763120+0000) 2024-05-27T08:07:16.714 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:16.526+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:17.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:16 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:16.655+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:17.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:17.020+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:17.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:17.088+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:17.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:17.028+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:17.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:17.104+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:17.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:17 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:17.511+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:18.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:17.625+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:18.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:17.997+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:18.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:18.017+0000 7f4594800700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.766577+0000 front 2024-05-27T08:06:54.766638+0000 (oldest deadline 2024-05-27T08:07:17.568143+0000) 2024-05-27T08:07:18.321 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:18 smithi090 podman[133889]: 2024-05-27 08:07:18.007476068 +0000 UTC m=+6.391246815 container died e569ebc33c4fac670d080ef6fcc0810e7de5b50f92e0b4c95a76b05c3428e8fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-27T08:07:18.321 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:18.046+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:18.321 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:18.046+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:18.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:18.073+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:18.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:18.150+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:18.573 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:18 smithi090 podman[133889]: 2024-05-27 08:07:18.544958447 +0000 UTC m=+6.928729154 container cleanup e569ebc33c4fac670d080ef6fcc0810e7de5b50f92e0b4c95a76b05c3428e8fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-27T08:07:18.573 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:18 smithi090 bash[133889]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3 2024-05-27T08:07:18.573 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:18.532+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:18.574 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:18 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:18.532+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:19.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:18.654+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:19.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:18.654+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:19.214 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:18 smithi090 podman[133908]: 2024-05-27 08:07:18.872114138 +0000 UTC m=+0.843246651 container remove e569ebc33c4fac670d080ef6fcc0810e7de5b50f92e0b4c95a76b05c3428e8fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-27T08:07:19.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:19.040+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:19.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:19.040+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:19.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:18.964+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:19.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:18.971+0000 7f4594800700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.766577+0000 front 2024-05-27T08:06:54.766638+0000 (oldest deadline 2024-05-27T08:07:17.568143+0000) 2024-05-27T08:07:19.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:19.066+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:19.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:19.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:19.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:19.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:19.936 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:19.548+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:19.936 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:19 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:19.548+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:19.936 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:19 smithi090 podman[133995]: 2024-05-27 08:07:19.659831531 +0000 UTC m=+0.034172968 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:07:19.937 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:19 smithi090 podman[133995]: 2024-05-27 08:07:19.773304831 +0000 UTC m=+0.147646305 container create 8d973a5e95f552ad1a56633941739989ee190d537143724e765e74f9ad800d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-27T08:07:19.946 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:19.602+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:20.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:19.705+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:20.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:19.705+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:20.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:20.018+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:20.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:20.018+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:20.215 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:20 smithi090 podman[133995]: 2024-05-27 08:07:20.045604245 +0000 UTC m=+0.419945762 container init 8d973a5e95f552ad1a56633941739989ee190d537143724e765e74f9ad800d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T08:07:20.215 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:20 smithi090 podman[133995]: 2024-05-27 08:07:20.056415171 +0000 UTC m=+0.430756640 container start 8d973a5e95f552ad1a56633941739989ee190d537143724e765e74f9ad800d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-05-27T08:07:20.215 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:20 smithi090 podman[133995]: 2024-05-27 08:07:20.144379935 +0000 UTC m=+0.518721475 container attach 8d973a5e95f552ad1a56633941739989ee190d537143724e765e74f9ad800d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240521) 2024-05-27T08:07:20.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:19.978+0000 7f4594800700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.766577+0000 front 2024-05-27T08:06:54.766638+0000 (oldest deadline 2024-05-27T08:07:17.568143+0000) 2024-05-27T08:07:20.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:19.944+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:20.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:19.944+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:20.500 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:20.051+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:20.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:20.150+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:20.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:20.150+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:20.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:20.560+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:20.853 INFO:journalctl@ceph.osd.3.smithi090.stdout:May 27 08:07:20 smithi090 podman[134021]: 2024-05-27 08:07:20.567192132 +0000 UTC m=+0.031256960 container died 8d973a5e95f552ad1a56633941739989ee190d537143724e765e74f9ad800d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:07:20.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:20.525+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:20.853 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:20 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:20.525+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:21.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:20.701+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:21.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:20.701+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:21.210 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:21.052+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:21.210 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:21.052+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:21.252 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.3.service' 2024-05-27T08:07:21.314 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:20.931+0000 7f4594800700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.766577+0000 front 2024-05-27T08:06:54.766638+0000 (oldest deadline 2024-05-27T08:07:17.568143+0000) 2024-05-27T08:07:21.317 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:20.936+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:21.317 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:20.936+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:21.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:07:21.402 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-27T08:07:21.402 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-27T08:07:21.402 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.4 2024-05-27T08:07:21.500 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:21.063+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:21.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:21.169+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:21.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:21.169+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:21.815 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:21 smithi129 systemd[1]: Stopping Ceph osd.4 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:07:21.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:21.576+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:21.880 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:21.517+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:21.880 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:21 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:21.517+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:22.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:21.697+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:22.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:21.697+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:22.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:22 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:22.025+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:22.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:22 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:22.025+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:22.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:21.980+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:22.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:21.980+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:22.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:21.965+0000 7f4594800700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.766577+0000 front 2024-05-27T08:06:54.766638+0000 (oldest deadline 2024-05-27T08:07:17.568143+0000) 2024-05-27T08:07:22.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:22.050+0000 7f4599000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:07:22.316 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:22.050+0000 7f4599000700 -1 osd.4 127 *** Got signal Terminated *** 2024-05-27T08:07:22.316 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:22.050+0000 7f4599000700 -1 osd.4 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:07:22.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:22.200+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:22.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:22.200+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:22.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:22.107+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:22.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:22.107+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:22.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:22.553+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:22.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:22 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:22.484+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:22.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:22 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:22.484+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:23.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:22.656+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:23.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:22.656+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:23.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:22.964+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:23.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:22.964+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:23.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:22.987+0000 7f4594800700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.766577+0000 front 2024-05-27T08:06:54.766638+0000 (oldest deadline 2024-05-27T08:07:17.568143+0000) 2024-05-27T08:07:23.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:23 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:23.044+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:23.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:23 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:23.044+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:23.500 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:23.236+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:23.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:23.236+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:23.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:23.070+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:23.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:23.070+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:23.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:23 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:23.507+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:23.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:23 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:23.532+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:23.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:23 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:23.532+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:24.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:23.628+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:24.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:23.628+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:24.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:24.006+0000 7f4594800700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.766577+0000 front 2024-05-27T08:06:54.766638+0000 (oldest deadline 2024-05-27T08:07:17.568143+0000) 2024-05-27T08:07:24.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:23 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:23.999+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:24.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:23.999+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:24.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:24 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:24.018+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:24.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:24 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:24.018+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:24.500 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:24.237+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:24.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:24.237+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:24.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:24.054+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:24.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:24.054+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:24.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:24.555+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:24.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:24 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:24.520+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:24.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:24 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:24.520+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:25.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:24.608+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:25.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:24.608+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:25.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:25.033+0000 7f4594800700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.766577+0000 front 2024-05-27T08:06:54.766638+0000 (oldest deadline 2024-05-27T08:07:17.568143+0000) 2024-05-27T08:07:25.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:25.028+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:25.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:25.028+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:25.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:24 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:24.980+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:25.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:24 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:24.980+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:25.500 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:25.194+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:25.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:25.194+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:25.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:25.084+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:25.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:25.084+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:25.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:25 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:25.554+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:25.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:25 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:25.554+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:25.995 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:25.591+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:26.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:25.588+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:26.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:25.588+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:26.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:25.993+0000 7f4594800700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.766577+0000 front 2024-05-27T08:06:54.766638+0000 (oldest deadline 2024-05-27T08:07:17.568143+0000) 2024-05-27T08:07:26.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:25.995+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:26.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:25.995+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:26.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:26 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:26.021+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:26.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:26 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:26.021+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:26.500 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:26.157+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:26.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:26.157+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:26.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:26.085+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:26.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:26.085+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:26.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:26.547+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:26.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:26 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:26.582+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:26.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:26 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:26.582+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:27.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:26.625+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:27.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:26.625+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:27.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4[46809]: 2024-05-27T08:07:26.946+0000 7f4594800700 -1 osd.4 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.766577+0000 front 2024-05-27T08:06:54.766638+0000 (oldest deadline 2024-05-27T08:07:17.568143+0000) 2024-05-27T08:07:27.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:27 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:27.021+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:27.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:27 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:27.021+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:27.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:27.000+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:27.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:27.000+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:27.500 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:27.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:27.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:27.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:27.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:27.090+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:27.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:27.090+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:27.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:27 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:27.499+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:27.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:27.554+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:27.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:27.554+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:28.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:27.591+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:28.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:27.591+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:28.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:28 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:28.059+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:28.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:28 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:28.059+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:28.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:27.965+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:28.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:27 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:27.965+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:28.500 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:28.070+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:28.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:28.070+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:28.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:28.178+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:28.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:28.178+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:28.692 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:28 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:28.498+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:28.692 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:28 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:28.498+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:28.693 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:28 smithi129 podman[72497]: 2024-05-27 08:07:28.321647202 +0000 UTC m=+6.466624320 container died 220487bdf3ed1a2f30147a152bc66ace68e8315c50a264a5604df6016e81f456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T08:07:28.934 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:28 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:28.597+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:28.934 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:28 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:28.597+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:29.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:28.554+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:29.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:28.554+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:29.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:28.554+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:29.039 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:28 smithi129 podman[72497]: 2024-05-27 08:07:28.810709972 +0000 UTC m=+6.955687047 container cleanup 220487bdf3ed1a2f30147a152bc66ace68e8315c50a264a5604df6016e81f456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T08:07:29.039 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:28 smithi129 bash[72497]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4 2024-05-27T08:07:29.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:28 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:28.931+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:29.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:28 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:28.931+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:29.315 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:29 smithi129 podman[72518]: 2024-05-27 08:07:29.03727019 +0000 UTC m=+0.702984748 container remove 220487bdf3ed1a2f30147a152bc66ace68e8315c50a264a5604df6016e81f456 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-27T08:07:29.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:29 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:29.095+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:29.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:29 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:29.095+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:29.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:29.198+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:29.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:29.198+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:29.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:29.099+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:29.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:29.099+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:29.815 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:29 smithi129 podman[72604]: 2024-05-27 08:07:29.685328139 +0000 UTC m=+0.033079380 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:07:29.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:29 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:29.543+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:29.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:29 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:29.543+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:29.936 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:29.553+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:29.936 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:29.553+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:29.936 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:29.553+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:30.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:29.527+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:30.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:29.527+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:30.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:29.527+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:30.136 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:30 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:30.070+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:30.136 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:30 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:30.070+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:30.136 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:29 smithi129 podman[72604]: 2024-05-27 08:07:29.838496168 +0000 UTC m=+0.186247447 container create 125721d2e39eb607c2589edae40acde89df4de5ff9881473f0d16d68d000d6cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux ) 2024-05-27T08:07:30.136 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:30 smithi129 podman[72604]: 2024-05-27 08:07:30.134863445 +0000 UTC m=+0.482614777 container init 125721d2e39eb607c2589edae40acde89df4de5ff9881473f0d16d68d000d6cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T08:07:30.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:29.934+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:30.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:29 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:29.934+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:30.493 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:30.110+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:30.493 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:30.110+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:30.493 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:30.110+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.168852+0000 front 2024-05-27T08:07:05.168700+0000 (oldest deadline 2024-05-27T08:07:29.769596+0000) 2024-05-27T08:07:30.493 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:30.194+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:30.493 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:30.194+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:30.493 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:30.194+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:30.565 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:30 smithi129 podman[72604]: 2024-05-27 08:07:30.146386508 +0000 UTC m=+0.494137775 container start 125721d2e39eb607c2589edae40acde89df4de5ff9881473f0d16d68d000d6cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:07:30.565 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:30 smithi129 podman[72604]: 2024-05-27 08:07:30.233394704 +0000 UTC m=+0.581146029 container attach 125721d2e39eb607c2589edae40acde89df4de5ff9881473f0d16d68d000d6cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0) 2024-05-27T08:07:30.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:30.490+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:30.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:30.490+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:30.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:30.490+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:30.902 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:30 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:30.512+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:30.903 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:30 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:30.512+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:30.903 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:30 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:30.512+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:30.983 INFO:journalctl@ceph.osd.4.smithi129.stdout:May 27 08:07:30 smithi129 podman[72604]: 2024-05-27 08:07:30.602582379 +0000 UTC m=+0.950333670 container died 125721d2e39eb607c2589edae40acde89df4de5ff9881473f0d16d68d000d6cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-27T08:07:30.984 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:30 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:30.563+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:30.984 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:30 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:30.563+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:31.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:30 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:30.900+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:31.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:30 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:30.900+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:31.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:31 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:31.071+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:31.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:31 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:31.071+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:31.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:31 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:31.071+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:31.436 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.4.service' 2024-05-27T08:07:31.450 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:31.119+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:31.451 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:31.119+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:31.451 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:31.119+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.168852+0000 front 2024-05-27T08:07:05.168700+0000 (oldest deadline 2024-05-27T08:07:29.769596+0000) 2024-05-27T08:07:31.451 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:31.164+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:31.451 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:31.164+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:31.451 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:31.164+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:31.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:07:31.544 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-27T08:07:31.544 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-27T08:07:31.544 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.5 2024-05-27T08:07:31.750 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:31 smithi136 systemd[1]: Stopping Ceph osd.5 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:07:31.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:31.448+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:31.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:31.448+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:31.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:31.448+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:31.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:31 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:31.517+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:31.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:31 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:31.517+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:31.900 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:31 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:31.554+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:31.900 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:31 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:31.554+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:31.900 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:31 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:31.554+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:32.160 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:32.083+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:32.160 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:32.083+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:32.160 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:32.083+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.168852+0000 front 2024-05-27T08:07:05.168700+0000 (oldest deadline 2024-05-27T08:07:29.769596+0000) 2024-05-27T08:07:32.160 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:32.087+0000 7fd0bf200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:07:32.160 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:32.087+0000 7fd0bf200700 -1 osd.5 127 *** Got signal Terminated *** 2024-05-27T08:07:32.160 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:32.087+0000 7fd0bf200700 -1 osd.5 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:07:32.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:31 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:31.897+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:32.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:31 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:31.897+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:32.462 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:32.203+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:32.462 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:32.203+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:32.462 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:32.203+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:32.471 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:32 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:32.115+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:32.471 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:32 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:32.115+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:32.471 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:32 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:32.115+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:32.750 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:32.459+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:32.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:32.459+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:32.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:32.459+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:32.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:32 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:32.469+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:32.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:32 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:32.469+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:32.883 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:32.544+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:32.883 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:32.544+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:32.883 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:32.544+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:33.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:32.880+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:33.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:32.880+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:33.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:32 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:32.880+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:33.256 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:33 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:33.143+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:33.256 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:33 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:33.143+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:33.256 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:33 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:33.143+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:33.485 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:33.196+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:33.485 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:33.196+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:33.485 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:33.196+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:33.486 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:33.086+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:33.486 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:33.086+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:33.486 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:33.086+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.168852+0000 front 2024-05-27T08:07:05.168700+0000 (oldest deadline 2024-05-27T08:07:29.769596+0000) 2024-05-27T08:07:33.565 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:33 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:33.451+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:33.565 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:33 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:33.451+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:33.750 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:33.483+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:33.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:33.483+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:33.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:33.483+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:33.889 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:33 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:33.523+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:33.889 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:33 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:33.523+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:33.889 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:33 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:33.523+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:34.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:33 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:33.887+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:34.236 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:33 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:33.887+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:34.236 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:33 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:33.887+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:34.446 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:34 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:34.113+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:34.447 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:34 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:34.113+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:34.447 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:34 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:34.113+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:34.468 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:34.068+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:34.468 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:34.068+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:34.468 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:34.068+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.168852+0000 front 2024-05-27T08:07:05.168700+0000 (oldest deadline 2024-05-27T08:07:29.769596+0000) 2024-05-27T08:07:34.468 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:34.191+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:34.468 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:34.191+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:34.468 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:34.191+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:34.750 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:34.466+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:34.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:34.466+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:34.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:34.466+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:34.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:34 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:34.443+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:34.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:34 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:34.443+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:34.921 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:34 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:34.566+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:34.922 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:34 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:34.566+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:34.922 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:34 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:34.566+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:35.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:34 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:34.919+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:35.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:34 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:34.919+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:35.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:34 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:34.919+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:35.495 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:35 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:35.073+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:35.495 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:35 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:35.073+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:35.495 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:35 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:35.073+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:35.499 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:35.226+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:35.499 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:35.226+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:35.499 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:35.226+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:35.499 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:35.019+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:35.499 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:35.019+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:35.499 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:35.019+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.168852+0000 front 2024-05-27T08:07:05.168700+0000 (oldest deadline 2024-05-27T08:07:29.769596+0000) 2024-05-27T08:07:35.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:35.497+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:35.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:35.497+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:35.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:35.497+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:35.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:35 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:35.492+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:35.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:35 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:35.492+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:35.918 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:35 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:35.583+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:35.918 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:35 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:35.583+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:35.918 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:35 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:35.583+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:36.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:35 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:35.916+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:36.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:35 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:35.916+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:36.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:35 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:35.916+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:36.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:36 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:36.038+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:36.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:36 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:36.038+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:36.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:36 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:36.038+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:36.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:36.229+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:36.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:36.229+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:36.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:36.229+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:36.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:36.044+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:43.754844+0000 front 2024-05-27T08:06:43.754944+0000 (oldest deadline 2024-05-27T08:07:07.157533+0000) 2024-05-27T08:07:36.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:36.044+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:52.264614+0000 front 2024-05-27T08:06:52.264683+0000 (oldest deadline 2024-05-27T08:07:21.666455+0000) 2024-05-27T08:07:36.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5[47329]: 2024-05-27T08:07:36.044+0000 7fd0baa00700 -1 osd.5 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.168852+0000 front 2024-05-27T08:07:05.168700+0000 (oldest deadline 2024-05-27T08:07:29.769596+0000) 2024-05-27T08:07:36.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:36 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:36.507+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:36.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:36 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:36.507+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:36.922 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:36 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:36.550+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:36.922 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:36 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:36.550+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:36.922 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:36 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:36.550+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:37.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:36.500+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:37.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:36.500+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:37.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:36.500+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:37.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:36 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:36.920+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:37.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:36 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:36.920+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:37.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:36 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:36.920+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:37.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:37 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:37.009+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:37.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:37 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:37.009+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:37.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:37 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:37.009+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:37.461 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:37 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:37.195+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:37.461 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:37 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:37.195+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:37.461 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:37 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:37.195+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:37.750 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:37 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:37.459+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:37.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:37 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:37.459+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:37.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:37 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:37.459+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:37.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:37 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:37.535+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:37.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:37 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:37.535+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:37.899 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:37 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:37.549+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:37.899 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:37 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:37.549+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:37.899 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:37 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:37.549+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:38.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:37 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:37.897+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:38.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:37 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:37.897+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:38.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:37 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:37.897+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:38.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:37 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:37.964+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:38.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:37 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:37.964+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:38.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:37 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:37.964+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:38.500 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:38 smithi136 podman[74399]: 2024-05-27 08:07:38.439674354 +0000 UTC m=+6.492138114 container died 755538b44e88fa2aa4431fc7de9c39b402e0f214ced4ca4a5ff9ab8f0bb3dfda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-27T08:07:38.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:38 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:38.201+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:38.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:38 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:38.201+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:38.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:38 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:38.201+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:38.758 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:38 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:38.503+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:38.758 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:38 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:38.503+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:38.758 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:38 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:38.503+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:38.905 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:38 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:38.515+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:38.905 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:38 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:38.515+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:38.905 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:38 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:38.515+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:38.905 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:38 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:38.515+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:38.943 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:38 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:38.585+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:38.943 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:38 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:38.585+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:39.205 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:38 smithi136 podman[74399]: 2024-05-27 08:07:38.890278625 +0000 UTC m=+6.942742362 container cleanup 755538b44e88fa2aa4431fc7de9c39b402e0f214ced4ca4a5ff9ab8f0bb3dfda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T08:07:39.205 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:38 smithi136 bash[74399]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5 2024-05-27T08:07:39.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:38 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:38.903+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:39.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:38 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:38.903+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:39.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:38 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:38.903+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:39.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:38 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:38.941+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:39.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:38 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:38.941+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:39.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:38 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:38.941+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:39.500 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:39.202+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:39.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:39.202+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:39.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:39.202+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:39.501 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:39 smithi136 podman[74428]: 2024-05-27 08:07:39.233492904 +0000 UTC m=+0.766278191 container remove 755538b44e88fa2aa4431fc7de9c39b402e0f214ced4ca4a5ff9ab8f0bb3dfda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:07:39.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:39 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:39.544+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:39.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:39 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:39.544+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:39.946 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:39 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:39.543+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:39.946 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:39 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:39.543+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:39.946 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:39 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:39.543+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:39.946 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:39 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:39.543+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:40.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:39.516+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:40.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:39.516+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:40.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:39 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:39.516+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:40.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:39 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:39.944+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:40.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:39 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:39.944+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:40.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:39 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:39.944+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:40.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:39 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:39.903+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:40.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:39 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:39.903+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:40.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:39 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:39.903+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:40.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:39 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:39.903+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:40.423 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:40 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:40.217+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:40.424 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:40 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:40.217+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:40.424 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:40 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:40.217+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:40.424 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:40 smithi136 podman[74518]: 2024-05-27 08:07:39.994467434 +0000 UTC m=+0.033932979 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:07:40.424 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:40 smithi136 podman[74518]: 2024-05-27 08:07:40.118038095 +0000 UTC m=+0.157503677 container create de8525221be882b13e1177aabc2ee27b9a6f2a189e6866399f856d7112f7fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-05-27T08:07:40.424 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:40 smithi136 podman[74518]: 2024-05-27 08:07:40.422418071 +0000 UTC m=+0.461883663 container init de8525221be882b13e1177aabc2ee27b9a6f2a189e6866399f856d7112f7fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T08:07:40.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:40 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:40.489+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:40.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:40 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:40.489+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:40.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:40 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:40.489+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:40.751 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:40 smithi136 podman[74518]: 2024-05-27 08:07:40.433834962 +0000 UTC m=+0.473300581 container start de8525221be882b13e1177aabc2ee27b9a6f2a189e6866399f856d7112f7fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-05-27T08:07:40.751 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:40 smithi136 podman[74518]: 2024-05-27 08:07:40.529342548 +0000 UTC m=+0.568808137 container attach de8525221be882b13e1177aabc2ee27b9a6f2a189e6866399f856d7112f7fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:07:40.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:40 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:40.543+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:40.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:40 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:40.543+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:40.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:40 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:40.543+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:40.955 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:40.583+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:40.955 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:40.583+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:40.955 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:40.583+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:40.955 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:40.583+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:41.204 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:40 smithi136 podman[74544]: 2024-05-27 08:07:40.936029053 +0000 UTC m=+0.031708141 container died de8525221be882b13e1177aabc2ee27b9a6f2a189e6866399f856d7112f7fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:07:41.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:40.953+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:41.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:40.953+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:41.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:40.953+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:41.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:40 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:40.953+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.200993+0000 front 2024-05-27T08:07:16.201394+0000 (oldest deadline 2024-05-27T08:07:40.901261+0000) 2024-05-27T08:07:41.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:40 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:40.863+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:41.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:40 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:40.863+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:41.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:40 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:40.863+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:41.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:40 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:40.863+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:41.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:41 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:41.202+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:41.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:41 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:41.202+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:41.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:41 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:41.202+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:41.469 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:41 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:41.202+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:41.686 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.5.service' 2024-05-27T08:07:41.750 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:41 smithi136 podman[74544]: 2024-05-27 08:07:41.641958935 +0000 UTC m=+0.737638030 container remove de8525221be882b13e1177aabc2ee27b9a6f2a189e6866399f856d7112f7fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-05-27T08:07:41.751 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:41 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.5.service: Deactivated successfully. 2024-05-27T08:07:41.751 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:41 smithi136 systemd[1]: Stopped Ceph osd.5 for b0214938-1bfa-11ef-bc9b-c7b262605968. 2024-05-27T08:07:41.751 INFO:journalctl@ceph.osd.5.smithi136.stdout:May 27 08:07:41 smithi136 systemd[1]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.5.service: Consumed 17min 19.456s CPU time. 2024-05-27T08:07:41.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:41 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:41.467+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:41.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:41 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:41.467+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:41.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:41 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:41.467+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:41.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:07:41.799 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-27T08:07:41.799 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-27T08:07:41.799 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.6 2024-05-27T08:07:41.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:41 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:41.555+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:41.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:41 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:41.555+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:41.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:41 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:41.555+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:41.910 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:41 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:41.631+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:41.910 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:41 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:41.631+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:41.911 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:41 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:41.631+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:41.911 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:41 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:41.631+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:42.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:41 smithi090 systemd[1]: Stopping Ceph osd.6 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:07:42.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:41 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:41.960+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:42.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:41 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:41.960+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:42.215 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:41 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:41.960+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:42.215 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:41 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:41.960+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.200993+0000 front 2024-05-27T08:07:16.201394+0000 (oldest deadline 2024-05-27T08:07:40.901261+0000) 2024-05-27T08:07:42.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:41 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:41.875+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:42.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:41 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:41.875+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:42.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:41 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:41.875+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:42.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:41 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:41.875+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:42.501 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:42 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:42.444+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:42.501 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:42 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:42.444+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:42.501 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:42 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:42.444+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:42.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:42 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:42.224+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:42.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:42 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:42.224+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:42.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:42 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:42.224+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:42.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:42 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:42.224+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:42.555 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:42.301+0000 7fe339e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:07:42.555 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:42.301+0000 7fe339e00700 -1 osd.6 127 *** Got signal Terminated *** 2024-05-27T08:07:42.555 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:42.301+0000 7fe339e00700 -1 osd.6 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:07:42.848 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:42 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:42.587+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:42.849 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:42 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:42.587+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:42.849 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:42 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:42.587+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:42.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:42.653+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:42.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:42.653+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:42.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:42.653+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:42.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:42.653+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:43.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:42 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:42.846+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:43.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:42 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:42.846+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:43.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:42 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:42.846+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:43.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:42 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:42.846+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:43.460 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:43 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:43.206+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:43.461 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:43 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:43.206+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:43.461 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:43 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:43.206+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:43.461 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:43 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:43.206+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:43.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:42.962+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:43.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:42.962+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:43.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:42.962+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:43.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:42.962+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.200993+0000 front 2024-05-27T08:07:16.201394+0000 (oldest deadline 2024-05-27T08:07:40.901261+0000) 2024-05-27T08:07:43.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:43 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:43.459+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:43.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:43 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:43.459+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:43.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:43 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:43.459+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:43.847 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:43.577+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:43.848 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:43.577+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:43.848 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:43.577+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:43.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:43 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:43.646+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:43.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:43 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:43.646+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:43.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:43 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:43.646+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:43.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:43 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:43.646+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:44.302 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:43.845+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:44.302 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:43.845+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:44.302 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:43.845+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:44.302 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:43 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:43.845+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:44.459 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:44.182+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:44.459 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:44.182+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:44.459 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:44.182+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:44.459 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:44.182+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:44.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:44.012+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:44.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:44.012+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:44.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:44.012+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:44.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:44.012+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.200993+0000 front 2024-05-27T08:07:16.201394+0000 (oldest deadline 2024-05-27T08:07:40.901261+0000) 2024-05-27T08:07:44.750 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:44.457+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:44.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:44.457+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:44.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:44 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:44.457+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:44.897 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:44 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:44.617+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:44.897 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:44 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:44.617+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:44.897 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:44 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:44.617+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:44.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:44.647+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:44.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:44.647+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:44.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:44.647+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:44.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:44 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:44.647+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:45.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:44 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:44.895+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:45.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:44 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:44.895+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:45.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:44 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:44.895+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:45.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:44 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:44.895+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:45.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:45.003+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:45.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:45.003+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:45.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:45.003+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:45.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:45.003+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.200993+0000 front 2024-05-27T08:07:16.201394+0000 (oldest deadline 2024-05-27T08:07:40.901261+0000) 2024-05-27T08:07:45.500 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:45 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:45.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:45.500 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:45 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:45.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:45.500 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:45 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:45.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:45.500 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:45 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:45.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:45.750 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:45 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:45.498+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:45.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:45 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:45.498+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:45.751 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:45 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:45.498+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:45.899 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:45 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:45.603+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:45.899 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:45 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:45.603+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:45.899 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:45 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:45.603+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:45.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:45.634+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:45.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:45.634+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:45.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:45.634+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:45.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:45.634+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:46.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:45 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:45.897+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:46.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:45 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:45.897+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:46.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:45 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:45.897+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:46.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:45 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:45.897+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:46.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:45.963+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:46.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:45.963+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:46.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:45.963+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:46.464 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:45 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:45.963+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.200993+0000 front 2024-05-27T08:07:16.201394+0000 (oldest deadline 2024-05-27T08:07:40.901261+0000) 2024-05-27T08:07:46.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:46 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:46.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:46.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:46 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:46.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:46.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:46 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:46.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:46.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:46 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:46.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:46.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:46 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:46.643+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:46.927 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:46 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:46.643+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:46.928 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:46 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:46.643+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:46.941 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:46 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:46.628+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:46.941 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:46 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:46.628+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:46.941 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:46 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:46.628+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:46.941 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:46 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:46.628+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:47.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:46 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:46.518+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:47.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:46 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:46.518+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:47.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:46 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:46.518+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:47.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:46 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:46.939+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.987105+0000 front 2024-05-27T08:06:42.987175+0000 (oldest deadline 2024-05-27T08:07:09.388301+0000) 2024-05-27T08:07:47.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:46 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:46.939+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.691239+0000 front 2024-05-27T08:06:51.690520+0000 (oldest deadline 2024-05-27T08:07:17.491997+0000) 2024-05-27T08:07:47.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:46 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:46.939+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.096647+0000 front 2024-05-27T08:07:02.096754+0000 (oldest deadline 2024-05-27T08:07:32.098618+0000) 2024-05-27T08:07:47.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:46 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6[59124]: 2024-05-27T08:07:46.939+0000 7fe330600700 -1 osd.6 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.200993+0000 front 2024-05-27T08:07:16.201394+0000 (oldest deadline 2024-05-27T08:07:40.901261+0000) 2024-05-27T08:07:47.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:46 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:46.925+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:47.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:46 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:46.925+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:47.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:46 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:46.925+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:47.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:46 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:46.925+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:47.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:47 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:47.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:47.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:47 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:47.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:47.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:47 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:47.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:47.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:47 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:47.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:47.911 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:47 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:47.646+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:47.911 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:47 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:47.646+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:47.912 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:47 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:47.646+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:47.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:47 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:47.602+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:47.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:47 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:47.602+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:47.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:47 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:47.602+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:47.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:47 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:47.602+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:48.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:47 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:47.548+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:48.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:47 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:47.548+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:48.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:47 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:47.548+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:48.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:47 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:47.909+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:48.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:47 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:47.909+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:48.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:47 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:47.909+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:48.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:47 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:47.909+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:48.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:48 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:48.223+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:48.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:48 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:48.223+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:48.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:48 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:48.223+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:48.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:48 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:48.223+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:48.714 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:48 smithi090 podman[134112]: 2024-05-27 08:07:48.424119341 +0000 UTC m=+6.262945022 container died 47393c4e2cb47a5bce5d124a864c62ab2482f05c7fa2818a2f85781b9773f964 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-27T08:07:48.714 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:48 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:48.615+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:48.715 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:48 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:48.615+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:48.715 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:48 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:48.615+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:48.715 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:48 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:48.615+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:49.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:48 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:48.599+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:49.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:48 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:48.599+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:49.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:48 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:48.599+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:49.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:48 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:48.917+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:49.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:48 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:48.917+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:49.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:48 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:48.917+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:49.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:48 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:48.917+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:49.066 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:48 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:48.673+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:49.066 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:48 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:48.673+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:49.066 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:48 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:48.673+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:49.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:49 smithi090 podman[134112]: 2024-05-27 08:07:49.024005131 +0000 UTC m=+6.862830808 container cleanup 47393c4e2cb47a5bce5d124a864c62ab2482f05c7fa2818a2f85781b9773f964 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS) 2024-05-27T08:07:49.214 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:49 smithi090 bash[134112]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6 2024-05-27T08:07:49.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:49 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:49.197+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:49.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:49 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:49.197+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:49.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:49 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:49.197+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:49.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:49 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:49.197+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:49.661 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:49 smithi090 podman[134132]: 2024-05-27 08:07:49.242290053 +0000 UTC m=+0.799455803 container remove 47393c4e2cb47a5bce5d124a864c62ab2482f05c7fa2818a2f85781b9773f964 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:07:49.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:49 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:49.659+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:49.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:49 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:49.659+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:49.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:49 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:49.659+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:49.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:49 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:49.659+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:50.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:49 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:49.607+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:50.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:49 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:49.607+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:50.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:49 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:49.607+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:50.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:49 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:49.607+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:07:50.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:49 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:49.703+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:50.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:49 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:49.703+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:50.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:49 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:49.703+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:50.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:49 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:49.877+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:50.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:49 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:49.877+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:50.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:49 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:49.877+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:50.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:49 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:49.877+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:50.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:49 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:49.877+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:07:50.284 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:50 smithi090 podman[134218]: 2024-05-27 08:07:49.917088128 +0000 UTC m=+0.030930156 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:07:50.284 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:50 smithi090 podman[134218]: 2024-05-27 08:07:50.019745373 +0000 UTC m=+0.133587433 container create 0f040fb095ceee5fbd3b4d4bf9c07ab267bbcd5d27ca612930b37426e2f8b6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-27T08:07:50.284 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:50 smithi090 podman[134218]: 2024-05-27 08:07:50.282307282 +0000 UTC m=+0.396149325 container init 0f040fb095ceee5fbd3b4d4bf9c07ab267bbcd5d27ca612930b37426e2f8b6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:07:50.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:50 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:50.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:50.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:50 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:50.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:50.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:50 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:50.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:50.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:50 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:50.192+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:50.633 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:50 smithi090 podman[134218]: 2024-05-27 08:07:50.292956717 +0000 UTC m=+0.406798772 container start 0f040fb095ceee5fbd3b4d4bf9c07ab267bbcd5d27ca612930b37426e2f8b6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-27T08:07:50.634 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:50 smithi090 podman[134218]: 2024-05-27 08:07:50.456386018 +0000 UTC m=+0.570228138 container attach 0f040fb095ceee5fbd3b4d4bf9c07ab267bbcd5d27ca612930b37426e2f8b6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-27T08:07:50.964 INFO:journalctl@ceph.osd.6.smithi090.stdout:May 27 08:07:50 smithi090 podman[134218]: 2024-05-27 08:07:50.744247857 +0000 UTC m=+0.858089923 container died 0f040fb095ceee5fbd3b4d4bf9c07ab267bbcd5d27ca612930b37426e2f8b6e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:07:50.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:50 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:50.630+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:50.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:50 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:50.630+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:50.965 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:50 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:50.630+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:50.965 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:50 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:50.630+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:50.965 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:50 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:50.630+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:07:51.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:50 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:50.628+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:51.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:50 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:50.628+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:51.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:50 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:50.628+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:51.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:50 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:50.628+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:07:51.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:50.712+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:51.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:50.712+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:51.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:50.712+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:51.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:50.869+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:51.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:50.869+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:51.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:50.869+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:51.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:50.869+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:51.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:50 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:50.869+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:07:51.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:51 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:51.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:51.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:51 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:51.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:51.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:51 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:51.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:51.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:51 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:51.145+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:51.614 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.6.service' 2024-05-27T08:07:51.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:07:51.725 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-27T08:07:51.725 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-27T08:07:51.726 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.7 2024-05-27T08:07:51.817 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:51.727+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:51.818 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:51.727+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:51.818 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:51.727+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:51.818 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:51.727+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:21.350152+0000 front 2024-05-27T08:07:21.350287+0000 (oldest deadline 2024-05-27T08:07:50.751775+0000) 2024-05-27T08:07:51.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:51 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:51.660+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:51.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:51 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:51.660+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:51.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:51 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:51.660+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:51.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:51 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:51.660+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:51.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:51 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:51.660+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:07:52.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:51 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:51.593+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:52.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:51 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:51.593+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:52.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:51 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:51.593+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:52.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:51 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:51.593+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:07:52.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:51 smithi129 systemd[1]: Stopping Ceph osd.7 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:07:52.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:51.868+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:52.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:51.868+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:52.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:51.868+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:52.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:51.868+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:52.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:51.868+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:07:52.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:52 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:52.170+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:52.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:52 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:52.170+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:52.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:52 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:52.170+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:52.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:52 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:52.170+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:52.565 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:52.289+0000 7f1d72a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:07:52.565 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:52.289+0000 7f1d72a00700 -1 osd.7 127 *** Got signal Terminated *** 2024-05-27T08:07:52.565 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:52.289+0000 7f1d72a00700 -1 osd.7 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:07:52.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:52 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:52.701+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:52.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:52 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:52.701+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:52.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:52 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:52.701+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:52.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:52 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:52.701+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:52.965 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:52 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:52.701+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:07:53.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:52 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:52.599+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:53.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:52 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:52.599+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:53.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:52 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:52.599+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:53.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:52 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:52.599+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:07:53.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:52.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:53.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:52.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:53.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:52.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:53.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:52.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:53.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:52.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:07:53.066 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:52.721+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:53.066 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:52.721+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:53.066 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:52.721+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:53.067 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:52 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:52.721+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:21.350152+0000 front 2024-05-27T08:07:21.350287+0000 (oldest deadline 2024-05-27T08:07:50.751775+0000) 2024-05-27T08:07:53.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:53 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:53.207+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:53.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:53 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:53.207+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:53.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:53 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:53.207+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:53.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:53 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:53.207+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:53.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:53 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:53.207+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:07:54.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:53 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:53.629+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:54.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:53 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:53.629+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:54.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:53 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:53.629+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:54.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:53 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:53.629+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:07:54.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:53 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:53.718+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:54.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:53 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:53.718+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:54.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:53 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:53.718+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:54.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:53 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:53.718+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:21.350152+0000 front 2024-05-27T08:07:21.350287+0000 (oldest deadline 2024-05-27T08:07:50.751775+0000) 2024-05-27T08:07:54.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:53 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:53.831+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:54.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:53 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:53.831+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:54.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:53 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:53.831+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:54.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:53 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:53.831+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:54.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:53 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:53.831+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:07:54.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:53 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:53.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:54.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:53 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:53.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:54.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:53 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:53.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:54.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:53 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:53.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:54.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:53 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:53.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:07:54.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:54 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:54.225+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:54.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:54 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:54.225+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:54.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:54 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:54.225+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:54.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:54 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:54.225+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:54.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:54 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:54.225+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:07:55.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:54 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:54.680+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:55.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:54 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:54.680+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:55.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:54 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:54.680+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:55.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:54 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:54.680+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:07:55.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:54 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:54.744+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:55.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:54 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:54.744+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:55.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:54 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:54.744+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:55.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:54 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:54.744+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:21.350152+0000 front 2024-05-27T08:07:21.350287+0000 (oldest deadline 2024-05-27T08:07:50.751775+0000) 2024-05-27T08:07:55.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:54 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:54.844+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:55.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:54 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:54.844+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:55.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:54 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:54.844+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:55.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:54 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:54.844+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:55.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:54 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:54.844+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:07:55.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:54 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:54.763+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:55.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:54 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:54.763+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:55.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:54 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:54.763+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:55.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:54 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:54.763+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:55.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:54 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:54.763+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:07:55.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:55 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:55.237+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:55.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:55 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:55.237+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:55.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:55 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:55.237+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:55.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:55 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:55.237+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:55.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:55 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:55.237+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:07:56.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:55 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:55.651+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:56.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:55 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:55.651+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:56.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:55 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:55.651+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:56.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:55 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:55.651+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:07:56.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:55 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:55.762+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:56.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:55 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:55.762+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:56.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:55 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:55.762+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:56.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:55 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:55.762+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:21.350152+0000 front 2024-05-27T08:07:21.350287+0000 (oldest deadline 2024-05-27T08:07:50.751775+0000) 2024-05-27T08:07:56.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:55 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:55.864+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:56.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:55 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:55.864+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:56.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:55 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:55.864+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:56.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:55 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:55.864+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:56.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:55 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:55.864+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:07:56.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:55 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:55.798+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:56.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:55 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:55.798+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:56.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:55 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:55.798+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:56.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:55 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:55.798+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:56.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:55 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:55.798+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:07:56.649 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:56 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:56.265+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:56.649 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:56 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:56.265+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:56.649 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:56 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:56.265+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:56.649 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:56 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:56.265+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:56.650 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:56 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:56.265+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:07:57.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:56 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:56.647+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:57.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:56 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:56.647+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:57.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:56 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:56.647+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:57.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:56 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:56.647+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:07:57.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:56 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:56.740+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.438569+0000 front 2024-05-27T08:06:53.438358+0000 (oldest deadline 2024-05-27T08:07:18.738567+0000) 2024-05-27T08:07:57.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:56 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:56.740+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:04.640979+0000 front 2024-05-27T08:07:04.641102+0000 (oldest deadline 2024-05-27T08:07:28.042568+0000) 2024-05-27T08:07:57.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:56 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:56.740+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:14.447347+0000 front 2024-05-27T08:07:14.446909+0000 (oldest deadline 2024-05-27T08:07:39.746905+0000) 2024-05-27T08:07:57.065 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:56 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7[53474]: 2024-05-27T08:07:56.740+0000 7f1d6e200700 -1 osd.7 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:21.350152+0000 front 2024-05-27T08:07:21.350287+0000 (oldest deadline 2024-05-27T08:07:50.751775+0000) 2024-05-27T08:07:57.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:56 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:56.851+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:57.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:56 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:56.851+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:57.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:56 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:56.851+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:57.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:56 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:56.851+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:57.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:56 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:56.851+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:07:57.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:56.774+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:57.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:56.774+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:57.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:56.774+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:57.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:56.774+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:57.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:56 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:56.774+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:07:57.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:57 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:57.282+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:57.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:57 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:57.282+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:57.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:57 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:57.282+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:57.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:57 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:57.282+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:57.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:57 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:57.282+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:07:58.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:57 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:57.692+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:58.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:57 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:57.692+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:58.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:57 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:57.692+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:58.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:57 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:57.692+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:07:58.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:57.771+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:58.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:57.771+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:58.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:57.771+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:58.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:57.771+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:58.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:57 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:57.771+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:07:58.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:57 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:57.867+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:58.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:57 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:57.867+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:58.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:57 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:57.867+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:58.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:57 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:57.867+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:58.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:57 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:57.867+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:07:58.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:58 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:58.292+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:58.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:58 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:58.292+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:58.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:58 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:58.292+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:58.695 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:58 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:58.292+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:58.695 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:58 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:58.292+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:07:58.879 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:58 smithi129 podman[72722]: 2024-05-27 08:07:58.60124661 +0000 UTC m=+6.452870105 container died 097c27e12a30ac7f40d932afa6aad890e95be2d6df7f2b4b30d742926a3f58e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-27T08:07:59.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:58 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:58.693+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:07:59.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:58 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:58.693+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:07:59.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:58 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:58.693+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:07:59.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:58 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:58.693+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:07:59.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:58 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:58.796+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:07:59.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:58 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:58.796+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:07:59.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:58 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:58.796+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:07:59.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:58 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:58.796+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:07:59.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:58 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:58.796+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:07:59.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:58 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:58.796+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:07:59.315 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:59 smithi129 podman[72722]: 2024-05-27 08:07:59.066923897 +0000 UTC m=+6.918547407 container cleanup 097c27e12a30ac7f40d932afa6aad890e95be2d6df7f2b4b30d742926a3f58e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:07:59.315 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:59 smithi129 bash[72722]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7 2024-05-27T08:07:59.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:58 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:58.873+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:07:59.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:58 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:58.873+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:07:59.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:58 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:58.873+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:07:59.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:58 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:58.873+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:07:59.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:58 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:58.873+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:07:59.665 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:59 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:59.254+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:07:59.665 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:59 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:59.254+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:07:59.666 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:59 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:59.254+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:07:59.666 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:59 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:59.254+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:07:59.666 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:07:59 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:07:59.254+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:07:59.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:07:59 smithi129 podman[72742]: 2024-05-27 08:07:59.377104324 +0000 UTC m=+0.755120421 container remove 097c27e12a30ac7f40d932afa6aad890e95be2d6df7f2b4b30d742926a3f58e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:08:00.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:59 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:59.664+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:00.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:59 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:59.664+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:00.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:59 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:59.664+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:00.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:59 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:59.664+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:00.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:07:59 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:07:59.664+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:00.190 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:59 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:59.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:00.191 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:59 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:59.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:00.191 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:59 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:59.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:00.191 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:59 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:59.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:00.191 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:59 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:59.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:00.191 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:07:59 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:07:59.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:00.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:59 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:59.761+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:00.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:59 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:59.761+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:00.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:59 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:59.761+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:00.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:59 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:59.761+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:00.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:59 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:59.761+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:00.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:07:59 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:07:59.761+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:00.558 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:08:00 smithi129 podman[72826]: 2024-05-27 08:08:00.09771651 +0000 UTC m=+0.036582056 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:08:00.559 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:08:00 smithi129 podman[72826]: 2024-05-27 08:08:00.22852616 +0000 UTC m=+0.167391725 container create 493a9dd7c59a34e1670dcb620b7ecdbcf09212aa499026aa8d5a544404bc873e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-05-27T08:08:00.559 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:08:00 smithi129 podman[72826]: 2024-05-27 08:08:00.557420728 +0000 UTC m=+0.496286274 container init 493a9dd7c59a34e1670dcb620b7ecdbcf09212aa499026aa8d5a544404bc873e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:08:00.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:00 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:00.260+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:08:00.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:00 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:00.260+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:08:00.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:00 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:00.260+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:08:00.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:00 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:00.260+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:08:00.694 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:00 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:00.260+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:08:00.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:08:00 smithi129 podman[72826]: 2024-05-27 08:08:00.568497419 +0000 UTC m=+0.507362953 container start 493a9dd7c59a34e1670dcb620b7ecdbcf09212aa499026aa8d5a544404bc873e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-27T08:08:00.815 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:08:00 smithi129 podman[72826]: 2024-05-27 08:08:00.681129248 +0000 UTC m=+0.619994817 container attach 493a9dd7c59a34e1670dcb620b7ecdbcf09212aa499026aa8d5a544404bc873e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:08:01.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:00 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:00.692+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:01.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:00 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:00.692+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:01.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:00 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:00.692+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:01.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:00 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:00.692+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:01.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:00 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:00.692+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:01.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:00.759+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:01.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:00.759+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:01.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:00.759+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:01.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:00.759+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:01.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:00.759+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:01.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:00 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:00.759+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:01.315 INFO:journalctl@ceph.osd.7.smithi129.stdout:May 27 08:08:01 smithi129 podman[72852]: 2024-05-27 08:08:01.06549567 +0000 UTC m=+0.034984797 container died 493a9dd7c59a34e1670dcb620b7ecdbcf09212aa499026aa8d5a544404bc873e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-05-27T08:08:01.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:00 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:00.824+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:01.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:00 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:00.824+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:01.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:00 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:00.824+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:01.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:00 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:00.824+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:01.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:00 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:00.824+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:01.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:00 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:00.824+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:01.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:01.226+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:08:01.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:01.226+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:08:01.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:01.226+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:08:01.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:01.226+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:08:01.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:01.226+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:08:01.861 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.7.service' 2024-05-27T08:08:01.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:08:01.962 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-27T08:08:01.962 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-27T08:08:01.962 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.8 2024-05-27T08:08:02.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:01.740+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:02.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:01.740+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:02.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:01.740+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:02.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:01.740+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:02.000 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:01 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:01.740+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:02.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:01 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:01.811+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:02.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:01 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:01.811+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:02.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:01 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:01.811+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:02.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:01 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:01.811+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:02.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:01 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:01.811+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:02.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:01 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:01.811+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:02.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:01 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:01.802+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:02.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:01 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:01.802+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:02.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:01 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:01.802+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:02.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:01 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:01.802+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:02.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:01 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:01.802+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:02.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:01 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:01.802+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:02.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:02 smithi136 systemd[1]: Stopping Ceph osd.8 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:08:02.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:02.240+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:08:02.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:02.240+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:08:02.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:02.240+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:08:02.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:02.240+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:08:02.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:02.240+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:08:03.001 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:02.583+0000 7f027be00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:08:03.001 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:02.583+0000 7f027be00700 -1 osd.8 127 *** Got signal Terminated *** 2024-05-27T08:08:03.001 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:02.583+0000 7f027be00700 -1 osd.8 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:08:03.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:02.737+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:03.002 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:02.737+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:03.002 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:02.737+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:03.002 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:02.737+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:03.002 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:02 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:02.737+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:03.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:02 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:02.800+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:03.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:02 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:02.800+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:03.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:02 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:02.800+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:03.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:02 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:02.800+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:03.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:02 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:02.800+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:03.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:02 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:02.800+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:03.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:02.768+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:03.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:02.768+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:03.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:02.768+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:03.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:02.768+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:03.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:02.768+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:03.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:02 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:02.768+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:03.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:03.230+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:08:03.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:03.230+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:08:03.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:03.230+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:08:03.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:03.230+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:08:03.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:03.230+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:08:03.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:03.230+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:35.614984+0000 front 2024-05-27T08:07:35.615043+0000 (oldest deadline 2024-05-27T08:08:03.216232+0000) 2024-05-27T08:08:04.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:03.740+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:04.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:03.740+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:04.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:03.740+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:04.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:03.740+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:04.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:03 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:03.740+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:04.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:03 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:03.772+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:04.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:03 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:03.772+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:04.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:03 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:03.772+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:04.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:03 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:03.772+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:04.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:03 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:03.772+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:04.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:03 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:03.772+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:04.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:03 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:03.833+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:04.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:03 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:03.833+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:04.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:03 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:03.833+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:04.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:03 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:03.833+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:04.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:03 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:03.833+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:04.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:03 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:03.833+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:04.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:04.184+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:08:04.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:04.184+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:08:04.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:04.184+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:08:04.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:04.184+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:08:04.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:04.184+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:08:04.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:04.184+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:35.614984+0000 front 2024-05-27T08:07:35.615043+0000 (oldest deadline 2024-05-27T08:08:03.216232+0000) 2024-05-27T08:08:05.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:04.736+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:05.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:04.736+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:05.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:04.736+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:05.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:04.736+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:05.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:04 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:04.736+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:05.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:04 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:04.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:05.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:04 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:04.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:05.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:04 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:04.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:05.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:04 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:04.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:05.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:04 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:04.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:05.066 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:04 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:04.861+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:05.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:04 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:04.816+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:05.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:04 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:04.816+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:05.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:04 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:04.816+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:05.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:04 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:04.816+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:05.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:04 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:04.816+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:05.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:04 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:04.816+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:05.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:05.159+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:08:05.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:05.159+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:08:05.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:05.159+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:08:05.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:05.159+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:08:05.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:05.159+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:08:05.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:05.159+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:35.614984+0000 front 2024-05-27T08:07:35.615043+0000 (oldest deadline 2024-05-27T08:08:03.216232+0000) 2024-05-27T08:08:06.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:05.741+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:06.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:05.741+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:06.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:05.741+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:06.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:05.741+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:06.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:05 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:05.741+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:06.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:05 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:05.773+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:06.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:05 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:05.773+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:06.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:05 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:05.773+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:06.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:05 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:05.773+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:06.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:05 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:05.773+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:06.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:05 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:05.773+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:06.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:05 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:05.871+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:06.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:05 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:05.871+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:06.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:05 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:05.871+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:06.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:05 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:05.871+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:06.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:05 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:05.871+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:06.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:05 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:05.871+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:06.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:06.189+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:08:06.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:06.189+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:08:06.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:06.189+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:08:06.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:06.189+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:08:06.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:06.189+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:08:06.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:06.189+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:35.614984+0000 front 2024-05-27T08:07:35.615043+0000 (oldest deadline 2024-05-27T08:08:03.216232+0000) 2024-05-27T08:08:07.150 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:06.753+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:07.150 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:06.753+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:07.150 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:06.753+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:07.151 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:06.753+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:07.151 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:06 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:06.753+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:07.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:06 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:06.766+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:07.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:06 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:06.766+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:07.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:06 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:06.766+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:07.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:06 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:06.766+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:07.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:06 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:06.766+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:07.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:06 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:06.766+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:07.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:06 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:06.879+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:07.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:06 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:06.879+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:07.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:06 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:06.879+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:07.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:06 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:06.879+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:07.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:06 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:06.879+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:07.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:06 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:06.879+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:07.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:07.148+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:42.993701+0000 front 2024-05-27T08:06:42.993835+0000 (oldest deadline 2024-05-27T08:07:08.293964+0000) 2024-05-27T08:08:07.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:07.148+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.196779+0000 front 2024-05-27T08:06:54.196579+0000 (oldest deadline 2024-05-27T08:07:18.296673+0000) 2024-05-27T08:08:07.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:07.148+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:03.599291+0000 front 2024-05-27T08:07:03.599016+0000 (oldest deadline 2024-05-27T08:07:29.498666+0000) 2024-05-27T08:08:07.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:07.148+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:15.304259+0000 front 2024-05-27T08:07:15.304375+0000 (oldest deadline 2024-05-27T08:07:40.604078+0000) 2024-05-27T08:08:07.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:07.148+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.808971+0000 front 2024-05-27T08:07:25.809085+0000 (oldest deadline 2024-05-27T08:07:52.210825+0000) 2024-05-27T08:08:07.501 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8[54134]: 2024-05-27T08:08:07.148+0000 7f0277600700 -1 osd.8 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:35.614984+0000 front 2024-05-27T08:07:35.615043+0000 (oldest deadline 2024-05-27T08:08:03.216232+0000) 2024-05-27T08:08:08.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:07.730+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:08.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:07.730+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:08.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:07.730+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:08.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:07.730+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:08.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:07.730+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:08.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:07 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:07.730+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:08.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:07.775+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:08.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:07.775+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:08.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:07.775+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:08.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:07.775+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:08.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:07 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:07.775+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:08.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:07 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:07.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:08.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:07 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:07.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:08.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:07 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:07.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:08.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:07 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:07.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:08.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:07 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:07.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:08.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:07 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:07.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:09.001 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:08 smithi136 podman[74626]: 2024-05-27 08:08:08.664105876 +0000 UTC m=+6.244693672 container died 610c11351dfaa90ab6cc2f0c3b8b4c08cd0edf29498ed3b9038cd9fdb541c483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-27T08:08:09.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:08 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:08.811+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:09.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:08 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:08.811+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:09.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:08 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:08.811+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:09.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:08 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:08.811+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:09.001 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:08 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:08.811+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:09.002 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:08 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:08.811+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:09.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:08 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:08.744+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:09.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:08 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:08.744+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:09.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:08 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:08.744+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:09.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:08 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:08.744+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:09.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:08 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:08.744+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:09.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:08 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:08.744+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:09.217 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:08 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:08.841+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:09.217 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:08 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:08.841+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:09.217 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:08 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:08.841+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:09.217 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:08 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:08.841+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:09.218 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:08 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:08.842+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:09.218 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:08 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:08.842+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:09.496 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:09 smithi136 podman[74626]: 2024-05-27 08:08:09.210064994 +0000 UTC m=+6.790652807 container cleanup 610c11351dfaa90ab6cc2f0c3b8b4c08cd0edf29498ed3b9038cd9fdb541c483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-27T08:08:09.496 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:09 smithi136 bash[74626]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8 2024-05-27T08:08:09.750 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:09 smithi136 podman[74643]: 2024-05-27 08:08:09.494603348 +0000 UTC m=+0.804265460 container remove 610c11351dfaa90ab6cc2f0c3b8b4c08cd0edf29498ed3b9038cd9fdb541c483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:08:10.116 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:09 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:09.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:10.116 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:09 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:09.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:10.116 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:09 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:09.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:10.117 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:09 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:09.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:10.117 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:09 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:09.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:10.117 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:09 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:09.737+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:10.235 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:09 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:09.890+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:10.235 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:09 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:09.890+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:10.235 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:09 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:09.890+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:10.235 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:09 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:09.890+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:10.235 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:09 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:09.890+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:10.235 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:09 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:09.890+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:10.235 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:09 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:09.890+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:10.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:09 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:09.826+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:10.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:09 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:09.826+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:10.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:09 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:09.826+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:10.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:09 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:09.826+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:10.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:09 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:09.826+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:10.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:09 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:09.826+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:10.635 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:10 smithi136 podman[74730]: 2024-05-27 08:08:10.231994279 +0000 UTC m=+0.031191643 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:08:10.635 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:10 smithi136 podman[74730]: 2024-05-27 08:08:10.371500509 +0000 UTC m=+0.170697910 container create 420e1ae27542c0a05e760885f9dda185643c7a6194b3f1929785dcadf5654524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-05-27T08:08:10.635 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:10 smithi136 podman[74730]: 2024-05-27 08:08:10.63342001 +0000 UTC m=+0.432617454 container init 420e1ae27542c0a05e760885f9dda185643c7a6194b3f1929785dcadf5654524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T08:08:10.892 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:10 smithi136 podman[74730]: 2024-05-27 08:08:10.643661822 +0000 UTC m=+0.442859217 container start 420e1ae27542c0a05e760885f9dda185643c7a6194b3f1929785dcadf5654524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-27T08:08:10.892 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:10 smithi136 podman[74730]: 2024-05-27 08:08:10.715741318 +0000 UTC m=+0.514938722 container attach 420e1ae27542c0a05e760885f9dda185643c7a6194b3f1929785dcadf5654524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-27T08:08:10.892 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:10 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:10.824+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:10.893 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:10 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:10.824+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:10.893 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:10 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:10.824+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:10.893 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:10 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:10.824+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:10.893 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:10 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:10.825+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:10.893 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:10 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:10.825+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:11.153 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:10 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:10.732+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:11.153 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:10 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:10.732+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:11.153 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:10 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:10.732+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:11.153 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:10 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:10.732+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:11.153 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:10 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:10.732+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:11.153 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:10 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:10.732+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:11.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:10 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:10.902+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:11.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:10 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:10.902+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:11.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:10 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:10.902+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:11.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:10 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:10.902+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:11.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:10 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:10.902+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:11.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:10 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:10.902+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:11.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:10 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:10.902+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:11.448 INFO:journalctl@ceph.osd.8.smithi136.stdout:May 27 08:08:11 smithi136 podman[74760]: 2024-05-27 08:08:11.145140188 +0000 UTC m=+0.033907666 container died 420e1ae27542c0a05e760885f9dda185643c7a6194b3f1929785dcadf5654524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-27T08:08:11.817 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:11 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:11.741+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:11.817 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:11 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:11.741+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:11.817 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:11 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:11.741+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:11.817 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:11 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:11.741+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:11.817 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:11 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:11.741+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:11.817 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:11 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:11.741+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:11.840 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.8.service' 2024-05-27T08:08:11.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:08:11.977 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-27T08:08:11.977 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-27T08:08:11.977 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.9 2024-05-27T08:08:12.132 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:11 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:11.846+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:12.132 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:11 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:11.846+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:12.132 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:11 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:11.846+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:12.132 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:11 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:11.846+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:12.132 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:11 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:11.846+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:12.132 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:11 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:11.846+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:12.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:11 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:11.882+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:12.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:11 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:11.882+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:12.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:11 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:11.882+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:12.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:11 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:11.882+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:12.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:11 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:11.882+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:12.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:11 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:11.882+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:12.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:11 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:11.882+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:12.464 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:12 smithi090 systemd[1]: Stopping Ceph osd.9 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:08:12.763 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:12.507+0000 7f36ede00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:08:12.763 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:12.507+0000 7f36ede00700 -1 osd.9 127 *** Got signal Terminated *** 2024-05-27T08:08:12.763 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:12.507+0000 7f36ede00700 -1 osd.9 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:08:12.763 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:12.750+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:12.764 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:12.750+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:12.764 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:12.750+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:12.764 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:12.750+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:12.764 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:12.750+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:12.764 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:12 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:12.750+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:13.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:12 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:12.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:13.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:12 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:12.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:13.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:12 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:12.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:13.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:12 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:12.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:13.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:12 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:12.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:13.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:12 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:12.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:13.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:12 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:12.866+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:13.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:12 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:12.866+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:13.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:12 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:12.866+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:13.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:12 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:12.866+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:13.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:12 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:12.866+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:13.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:12 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:12.866+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:13.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:12 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:12.866+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:14.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:13 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:13.793+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:14.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:13 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:13.793+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:14.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:13 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:13.793+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:14.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:13 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:13.793+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:14.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:13 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:13.793+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:14.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:13 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:13.793+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:14.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:13 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:13.888+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:14.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:13 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:13.888+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:14.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:13 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:13.888+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:14.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:13 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:13.888+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:14.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:13 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:13.888+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:14.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:13 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:13.888+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:14.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:13 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:13.850+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:14.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:13 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:13.850+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:14.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:13 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:13.850+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:14.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:13 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:13.850+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:14.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:13 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:13.850+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:14.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:13 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:13.850+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:14.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:13 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:13.850+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:15.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:14.808+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:15.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:14.808+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:15.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:14.808+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:15.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:14.808+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:15.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:14.808+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:15.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:14 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:14.808+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:15.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:14.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:15.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:14.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:15.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:14.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:15.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:14.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:15.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:14.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:15.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:14 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:14.891+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:15.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:14 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:14.854+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:15.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:14 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:14.854+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:15.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:14 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:14.854+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:15.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:14 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:14.854+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:15.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:14 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:14.854+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:15.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:14 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:14.854+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:15.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:14 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:14.854+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:16.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:15.817+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:16.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:15.817+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:16.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:15.817+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:16.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:15.817+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:16.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:15.817+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:16.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:15 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:15.817+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:16.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:15.855+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:16.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:15.855+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:16.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:15.855+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:16.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:15.855+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:16.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:15.855+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:16.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:15 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:15.855+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:16.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:15 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:15.825+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:16.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:15 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:15.825+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:16.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:15 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:15.825+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:16.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:15 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:15.825+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:16.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:15 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:15.825+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:16.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:15 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:15.825+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:16.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:15 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:15.825+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:17.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:16.806+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.956560+0000 front 2024-05-27T08:06:41.955611+0000 (oldest deadline 2024-05-27T08:07:10.756613+0000) 2024-05-27T08:08:17.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:16.806+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:51.259108+0000 front 2024-05-27T08:06:51.258453+0000 (oldest deadline 2024-05-27T08:07:17.659571+0000) 2024-05-27T08:08:17.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:16.806+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.863638+0000 front 2024-05-27T08:07:02.863733+0000 (oldest deadline 2024-05-27T08:07:28.763730+0000) 2024-05-27T08:08:17.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:16.806+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:12.168481+0000 front 2024-05-27T08:07:12.168622+0000 (oldest deadline 2024-05-27T08:07:38.068639+0000) 2024-05-27T08:08:17.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:16.806+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:26.173503+0000 front 2024-05-27T08:07:26.173875+0000 (oldest deadline 2024-05-27T08:07:50.273427+0000) 2024-05-27T08:08:17.215 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:16 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9[65827]: 2024-05-27T08:08:16.806+0000 7f36e9600700 -1 osd.9 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.375406+0000 front 2024-05-27T08:07:34.375249+0000 (oldest deadline 2024-05-27T08:07:58.474916+0000) 2024-05-27T08:08:17.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:16 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:16.896+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:17.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:16 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:16.896+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:17.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:16 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:16.896+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:17.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:16 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:16.896+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:17.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:16 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:16.896+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:17.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:16 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:16.896+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:17.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:16 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:16.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:17.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:16 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:16.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:17.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:16 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:16.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:17.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:16 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:16.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:17.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:16 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:16.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:17.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:16 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:16.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:17.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:16 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:16.858+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:18.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:17.921+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:18.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:17.921+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:18.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:17.921+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:18.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:17.921+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:18.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:17.921+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:18.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:17 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:17.921+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:18.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:17.893+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:18.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:17.893+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:18.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:17.893+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:18.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:17.893+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:18.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:17.893+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:18.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:17.893+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:18.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:17 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:17.893+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:19.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:18.894+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:19.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:18.894+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:19.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:18.894+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:19.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:18.894+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:19.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:18.894+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:19.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:18 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:18.894+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:19.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:18.915+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:19.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:18.915+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:19.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:18.915+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:19.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:18.915+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:19.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:18.915+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:19.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:18.915+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:19.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:18 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:18.915+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:19.326 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:19 smithi090 podman[134336]: 2024-05-27 08:08:19.034465556 +0000 UTC m=+6.674305324 container died b1ecf289ef211efeb62cea53cdf829782d01b6d7d0fefe33e701ae0d0eb4c2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD) 2024-05-27T08:08:19.580 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:19 smithi090 podman[134336]: 2024-05-27 08:08:19.560138435 +0000 UTC m=+7.199978166 container cleanup b1ecf289ef211efeb62cea53cdf829782d01b6d7d0fefe33e701ae0d0eb4c2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS) 2024-05-27T08:08:19.581 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:19 smithi090 bash[134336]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9 2024-05-27T08:08:19.964 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:19 smithi090 podman[134355]: 2024-05-27 08:08:19.747086248 +0000 UTC m=+0.683715679 container remove b1ecf289ef211efeb62cea53cdf829782d01b6d7d0fefe33e701ae0d0eb4c2de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-27T08:08:20.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:19.865+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:20.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:19.865+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:20.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:19.865+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:20.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:19.865+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:20.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:19.865+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:20.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:19 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:19.865+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:20.292 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:19.933+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:20.292 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:19.933+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:20.292 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:19.933+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:20.292 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:19.933+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:20.292 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:19.933+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:20.292 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:19.933+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:20.292 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:19 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:19.933+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:20.837 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:20 smithi090 podman[134444]: 2024-05-27 08:08:20.581462086 +0000 UTC m=+0.119844976 container create 6fead367cd87b16c01b010fc970edd72b0342d8952f901615c1cc1f86106f6bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T08:08:20.837 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:20 smithi090 podman[134444]: 2024-05-27 08:08:20.495590327 +0000 UTC m=+0.033973196 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:08:20.837 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:20 smithi090 podman[134444]: 2024-05-27 08:08:20.835600223 +0000 UTC m=+0.373983156 container init 6fead367cd87b16c01b010fc970edd72b0342d8952f901615c1cc1f86106f6bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux ) 2024-05-27T08:08:21.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:20 smithi090 podman[134444]: 2024-05-27 08:08:20.847288258 +0000 UTC m=+0.385671174 container start 6fead367cd87b16c01b010fc970edd72b0342d8952f901615c1cc1f86106f6bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-27T08:08:21.214 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:20 smithi090 podman[134444]: 2024-05-27 08:08:20.942992417 +0000 UTC m=+0.481375356 container attach 6fead367cd87b16c01b010fc970edd72b0342d8952f901615c1cc1f86106f6bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-27T08:08:21.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:20.908+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:21.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:20.908+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:21.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:20.908+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:21.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:20.908+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:21.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:20.908+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:21.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:20.908+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:21.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:20.940+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:21.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:20.940+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:21.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:20.940+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:21.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:20.940+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:21.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:20.940+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:21.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:20.940+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:21.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:20 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:20.940+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:21.591 INFO:journalctl@ceph.osd.9.smithi090.stdout:May 27 08:08:21 smithi090 podman[134444]: 2024-05-27 08:08:21.301791257 +0000 UTC m=+0.840174159 container died 6fead367cd87b16c01b010fc970edd72b0342d8952f901615c1cc1f86106f6bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T08:08:21.927 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.9.service' 2024-05-27T08:08:22.041 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:08:22.042 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-27T08:08:22.042 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-27T08:08:22.043 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.10 2024-05-27T08:08:22.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:21.873+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:22.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:21.873+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:22.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:21.873+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:22.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:21.873+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:22.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:21.873+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:22.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:21.873+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:22.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:21 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:21.873+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:22.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:21.981+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:22.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:21.981+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:22.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:21.981+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:22.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:21.981+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:22.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:21.981+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:22.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:21.981+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:22.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:21 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:21.981+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:22.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 systemd[1]: Stopping Ceph osd.10 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:08:22.990 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:22.620+0000 7fa25fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:08:22.990 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:22.620+0000 7fa25fe00700 -1 osd.10 127 *** Got signal Terminated *** 2024-05-27T08:08:22.990 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:22.620+0000 7fa25fe00700 -1 osd.10 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:08:23.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:22.884+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:23.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:22.884+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:23.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:22.884+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:23.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:22.884+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:23.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:22.884+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:23.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:22.884+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:23.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:22 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:22.884+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:23.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:22.987+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:23.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:22.987+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:23.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:22.987+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:23.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:22.987+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:23.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:22.987+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:23.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:22.987+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:23.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:22 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:22.987+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:24.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:23.929+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:24.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:23.929+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:24.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:23.929+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:24.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:23.929+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:24.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:23.929+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:24.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:23.929+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:24.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:23 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:23.929+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:24.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:24.025+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:24.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:24.025+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:24.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:24.025+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:24.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:24.025+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:24.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:24.025+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:24.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:24.025+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:24.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:24 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:24.025+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:25.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:24.932+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:25.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:24.932+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:25.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:24.932+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:25.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:24.932+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:25.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:24.932+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:25.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:24.932+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:25.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:24 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:24.932+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:25.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.008+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:25.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.008+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:25.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.008+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:25.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.008+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:25.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.008+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:25.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.008+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:25.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:25 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.008+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:26.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:25.952+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:26.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:25.952+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:26.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:25.952+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:26.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:25.952+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:26.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:25.952+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:26.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:25.952+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:26.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:25.952+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:26.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.999+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:26.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.999+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:26.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.999+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:26.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.999+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:26.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.999+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:26.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.999+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:26.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:26 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:25.999+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:27.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:26.972+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:27.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:26.972+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:27.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:26.972+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:27.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:26.972+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:27.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:26.972+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:27.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:26.972+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:27.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:26 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:26.972+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:27.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:27 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:27.041+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:41.242869+0000 front 2024-05-27T08:06:41.243461+0000 (oldest deadline 2024-05-27T08:07:08.843962+0000) 2024-05-27T08:08:27.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:27 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:27.041+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:54.048772+0000 front 2024-05-27T08:06:54.048872+0000 (oldest deadline 2024-05-27T08:07:19.251136+0000) 2024-05-27T08:08:27.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:27 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:27.041+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:05.655059+0000 front 2024-05-27T08:07:05.655246+0000 (oldest deadline 2024-05-27T08:07:30.355350+0000) 2024-05-27T08:08:27.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:27 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:27.041+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-05-27T08:07:16.659043+0000 front 2024-05-27T08:07:16.659261+0000 (oldest deadline 2024-05-27T08:07:39.559103+0000) 2024-05-27T08:08:27.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:27 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:27.041+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:25.863612+0000 front 2024-05-27T08:07:25.863665+0000 (oldest deadline 2024-05-27T08:07:49.264481+0000) 2024-05-27T08:08:27.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:27 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:27.041+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.368571+0000 front 2024-05-27T08:07:34.368358+0000 (oldest deadline 2024-05-27T08:07:59.068108+0000) 2024-05-27T08:08:27.316 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:27 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10[60122]: 2024-05-27T08:08:27.041+0000 7fa25b600700 -1 osd.10 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.071294+0000 front 2024-05-27T08:07:46.071442+0000 (oldest deadline 2024-05-27T08:08:09.570921+0000) 2024-05-27T08:08:28.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:27.936+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:28.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:27.936+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:28.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:27.936+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:28.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:27.936+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:28.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:27.936+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:28.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:27.936+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:28.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:27 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:27.936+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:29.052 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:28 smithi129 podman[72943]: 2024-05-27 08:08:28.708519134 +0000 UTC m=+6.232817515 container died bcead1a9ea6e7db6fea658cf4f88ce48fdf05d3dd2ef13b342e473348cccc890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:08:29.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:28.928+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:29.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:28.928+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:29.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:28.928+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:29.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:28.928+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:29.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:28.928+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:29.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:28.928+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:29.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:28 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:28.928+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:29.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:29 smithi129 podman[72943]: 2024-05-27 08:08:29.188743664 +0000 UTC m=+6.713042035 container cleanup bcead1a9ea6e7db6fea658cf4f88ce48fdf05d3dd2ef13b342e473348cccc890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T08:08:29.315 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:29 smithi129 bash[72943]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10 2024-05-27T08:08:29.815 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:29 smithi129 podman[72959]: 2024-05-27 08:08:29.448357578 +0000 UTC m=+0.712792175 container remove bcead1a9ea6e7db6fea658cf4f88ce48fdf05d3dd2ef13b342e473348cccc890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T08:08:30.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:29.902+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:30.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:29.902+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:30.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:29.902+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:30.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:29.902+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:30.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:29.902+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:30.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:29.902+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:30.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:29.902+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:30.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:29 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:29.902+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-05-27T08:08:04.701031+0000 front 2024-05-27T08:08:04.700910+0000 (oldest deadline 2024-05-27T08:08:29.400738+0000) 2024-05-27T08:08:30.565 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:30 smithi129 podman[73045]: 2024-05-27 08:08:30.213703276 +0000 UTC m=+0.034925272 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:08:30.565 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:30 smithi129 podman[73045]: 2024-05-27 08:08:30.315844233 +0000 UTC m=+0.137066259 container create f434701b8413e24cf60b39e331a9a58b0f3d47807e513463a88557bc1500afea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240521, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:08:31.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:30 smithi129 podman[73045]: 2024-05-27 08:08:30.661688756 +0000 UTC m=+0.482910792 container init f434701b8413e24cf60b39e331a9a58b0f3d47807e513463a88557bc1500afea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-27T08:08:31.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:30 smithi129 podman[73045]: 2024-05-27 08:08:30.674027931 +0000 UTC m=+0.495249942 container start f434701b8413e24cf60b39e331a9a58b0f3d47807e513463a88557bc1500afea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:08:31.065 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:30 smithi129 podman[73045]: 2024-05-27 08:08:30.760459327 +0000 UTC m=+0.581681328 container attach f434701b8413e24cf60b39e331a9a58b0f3d47807e513463a88557bc1500afea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:08:31.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:30.919+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:31.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:30.919+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:31.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:30.919+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:31.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:30.919+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:31.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:30.919+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:31.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:30.919+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:31.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:30.919+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:31.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:30 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:30.919+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-05-27T08:08:04.701031+0000 front 2024-05-27T08:08:04.700910+0000 (oldest deadline 2024-05-27T08:08:29.400738+0000) 2024-05-27T08:08:31.471 INFO:journalctl@ceph.osd.10.smithi129.stdout:May 27 08:08:31 smithi129 podman[73045]: 2024-05-27 08:08:31.133224815 +0000 UTC m=+0.954446850 container died f434701b8413e24cf60b39e331a9a58b0f3d47807e513463a88557bc1500afea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-27T08:08:31.848 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.10.service' 2024-05-27T08:08:31.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:08:31.953 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-27T08:08:31.953 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-27T08:08:31.953 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.11 2024-05-27T08:08:32.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:31.898+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:32.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:31.898+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:32.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:31.898+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:32.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:31.898+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:32.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:31.898+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:32.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:31.898+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:32.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:31.898+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:32.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:31 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:31.898+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-05-27T08:08:04.701031+0000 front 2024-05-27T08:08:04.700910+0000 (oldest deadline 2024-05-27T08:08:29.400738+0000) 2024-05-27T08:08:32.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 systemd[1]: Stopping Ceph osd.11 for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:08:32.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.510+0000 7fbb4c600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:08:32.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.510+0000 7fbb4c600700 -1 osd.11 127 *** Got signal Terminated *** 2024-05-27T08:08:32.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.510+0000 7fbb4c600700 -1 osd.11 127 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:08:32.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.945+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:32.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.945+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:32.948 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.945+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:33.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.945+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:33.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.945+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:33.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.945+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:33.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.945+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:33.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:32 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:32.945+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-05-27T08:08:04.701031+0000 front 2024-05-27T08:08:04.700910+0000 (oldest deadline 2024-05-27T08:08:29.400738+0000) 2024-05-27T08:08:34.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:33.897+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:34.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:33.897+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:34.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:33.897+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:34.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:33.897+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:34.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:33.897+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:34.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:33.897+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:34.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:33.897+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:34.252 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:33.897+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-05-27T08:08:04.701031+0000 front 2024-05-27T08:08:04.700910+0000 (oldest deadline 2024-05-27T08:08:29.400738+0000) 2024-05-27T08:08:35.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:34.937+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:35.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:34.937+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:35.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:34.937+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:35.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:34.937+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:35.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:34.937+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:35.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:34.937+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:35.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:34.937+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:35.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:34 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:34.937+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-05-27T08:08:04.701031+0000 front 2024-05-27T08:08:04.700910+0000 (oldest deadline 2024-05-27T08:08:29.400738+0000) 2024-05-27T08:08:36.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:35.900+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:36.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:35.900+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:36.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:35.900+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:36.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:35.900+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:36.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:35.900+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:36.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:35.900+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:36.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:35.900+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:36.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:35 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:35.900+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-05-27T08:08:04.701031+0000 front 2024-05-27T08:08:04.700910+0000 (oldest deadline 2024-05-27T08:08:29.400738+0000) 2024-05-27T08:08:37.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:36.879+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-05-27T08:06:40.866851+0000 front 2024-05-27T08:06:40.867003+0000 (oldest deadline 2024-05-27T08:07:11.469352+0000) 2024-05-27T08:08:37.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:36.879+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-05-27T08:06:53.173404+0000 front 2024-05-27T08:06:53.172875+0000 (oldest deadline 2024-05-27T08:07:18.473013+0000) 2024-05-27T08:08:37.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:36.879+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-05-27T08:07:02.477761+0000 front 2024-05-27T08:07:02.477846+0000 (oldest deadline 2024-05-27T08:07:28.378035+0000) 2024-05-27T08:08:37.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:36.879+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-05-27T08:07:22.886863+0000 front 2024-05-27T08:07:22.886931+0000 (oldest deadline 2024-05-27T08:07:48.787134+0000) 2024-05-27T08:08:37.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:36.879+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-05-27T08:07:34.492366+0000 front 2024-05-27T08:07:34.492438+0000 (oldest deadline 2024-05-27T08:07:59.192605+0000) 2024-05-27T08:08:37.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:36.879+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-05-27T08:07:46.795551+0000 front 2024-05-27T08:07:46.795753+0000 (oldest deadline 2024-05-27T08:08:08.495612+0000) 2024-05-27T08:08:37.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:36.879+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-05-27T08:07:57.199145+0000 front 2024-05-27T08:07:57.198884+0000 (oldest deadline 2024-05-27T08:08:21.298604+0000) 2024-05-27T08:08:37.251 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:36 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11[61190]: 2024-05-27T08:08:36.879+0000 7fbb42e00700 -1 osd.11 127 heartbeat_check: no reply from 172.21.15.136:6820 osd.8 since back 2024-05-27T08:08:04.701031+0000 front 2024-05-27T08:08:04.700910+0000 (oldest deadline 2024-05-27T08:08:29.400738+0000) 2024-05-27T08:08:39.111 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:38 smithi136 podman[74848]: 2024-05-27 08:08:38.787669296 +0000 UTC m=+6.416519862 container died 46575a5aaf85f88fd62dd845be0598045bd8ad44e2bee6d8827748c1d379b757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T08:08:39.489 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:39 smithi136 podman[74848]: 2024-05-27 08:08:39.220382019 +0000 UTC m=+6.849232581 container cleanup 46575a5aaf85f88fd62dd845be0598045bd8ad44e2bee6d8827748c1d379b757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T08:08:39.489 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:39 smithi136 bash[74848]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11 2024-05-27T08:08:39.750 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:39 smithi136 podman[74868]: 2024-05-27 08:08:39.488044724 +0000 UTC m=+0.687779106 container remove 46575a5aaf85f88fd62dd845be0598045bd8ad44e2bee6d8827748c1d379b757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS) 2024-05-27T08:08:40.523 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:40 smithi136 podman[74956]: 2024-05-27 08:08:40.168932578 +0000 UTC m=+0.033908125 image pull e36ea4fc8a5807e8d2701370e350a0de3856538797f7fba64cfce7e344bd6799 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6 2024-05-27T08:08:40.524 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:40 smithi136 podman[74956]: 2024-05-27 08:08:40.276691009 +0000 UTC m=+0.141666607 container create ad3be577f0b261d160403dec9695b9b88d3b4c087c8ce01cd501a03b2650a055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521) 2024-05-27T08:08:40.524 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:40 smithi136 podman[74956]: 2024-05-27 08:08:40.521966692 +0000 UTC m=+0.386942283 container init ad3be577f0b261d160403dec9695b9b88d3b4c087c8ce01cd501a03b2650a055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T08:08:40.985 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:40 smithi136 podman[74956]: 2024-05-27 08:08:40.53196918 +0000 UTC m=+0.396944798 container start ad3be577f0b261d160403dec9695b9b88d3b4c087c8ce01cd501a03b2650a055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-27T08:08:40.986 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:40 smithi136 podman[74956]: 2024-05-27 08:08:40.604112982 +0000 UTC m=+0.469088606 container attach ad3be577f0b261d160403dec9695b9b88d3b4c087c8ce01cd501a03b2650a055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-27T08:08:41.250 INFO:journalctl@ceph.osd.11.smithi136.stdout:May 27 08:08:40 smithi136 podman[74956]: 2024-05-27 08:08:40.984792027 +0000 UTC m=+0.849767572 container died ad3be577f0b261d160403dec9695b9b88d3b4c087c8ce01cd501a03b2650a055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-27T08:08:41.655 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@osd.11.service' 2024-05-27T08:08:41.760 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:08:41.760 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-27T08:08:41.760 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-27T08:08:41.761 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.a 2024-05-27T08:08:42.214 INFO:journalctl@ceph.mds.a.smithi090.stdout:May 27 08:08:41 smithi090 systemd[1]: Stopping Ceph mds.a for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:08:42.553 INFO:journalctl@ceph.mds.a.smithi090.stdout:May 27 08:08:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-a[72820]: 2024-05-27T08:08:42.315+0000 7f9125400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:08:42.553 INFO:journalctl@ceph.mds.a.smithi090.stdout:May 27 08:08:42 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-a[72820]: 2024-05-27T08:08:42.315+0000 7f9125400700 -1 mds.a *** got signal Terminated *** 2024-05-27T08:08:46.038 INFO:journalctl@ceph.mds.a.smithi090.stdout:May 27 08:08:45 smithi090 podman[134547]: 2024-05-27 08:08:45.769730504 +0000 UTC m=+3.568852772 container died f2ede0e852007cbaf3cc039e237d904c79e2c3042d09a84a1583664d87e719ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:08:46.448 INFO:journalctl@ceph.mds.a.smithi090.stdout:May 27 08:08:46 smithi090 podman[134547]: 2024-05-27 08:08:46.202260106 +0000 UTC m=+4.001382367 container cleanup f2ede0e852007cbaf3cc039e237d904c79e2c3042d09a84a1583664d87e719ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T08:08:46.449 INFO:journalctl@ceph.mds.a.smithi090.stdout:May 27 08:08:46 smithi090 bash[134547]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-a 2024-05-27T08:08:46.714 INFO:journalctl@ceph.mds.a.smithi090.stdout:May 27 08:08:46 smithi090 podman[134572]: 2024-05-27 08:08:46.446984614 +0000 UTC m=+0.657624027 container remove f2ede0e852007cbaf3cc039e237d904c79e2c3042d09a84a1583664d87e719ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-a, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-27T08:08:47.008 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.a.service' 2024-05-27T08:08:47.117 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:08:47.118 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-27T08:08:47.118 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-27T08:08:47.118 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d 2024-05-27T08:08:47.255 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.d.service' 2024-05-27T08:08:47.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:08:47.351 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-27T08:08:47.351 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-27T08:08:47.351 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.g 2024-05-27T08:08:47.714 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 08:08:47 smithi090 systemd[1]: Stopping Ceph mds.g for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:08:48.464 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 08:08:48 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-g[72404]: 2024-05-27T08:08:48.009+0000 7fa6b3200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:08:48.464 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 08:08:48 smithi090 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-g[72404]: 2024-05-27T08:08:48.009+0000 7fa6b3200700 -1 mds.g *** got signal Terminated *** 2024-05-27T08:08:49.407 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 08:08:49 smithi090 podman[134713]: 2024-05-27 08:08:49.144428025 +0000 UTC m=+1.370398486 container died 37b82db357a226fa30efec82b638b0acf291c459e2cdf33e812a5d5abe48a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-g, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:08:49.714 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 08:08:49 smithi090 podman[134713]: 2024-05-27 08:08:49.592856572 +0000 UTC m=+1.818827022 container cleanup 37b82db357a226fa30efec82b638b0acf291c459e2cdf33e812a5d5abe48a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5) 2024-05-27T08:08:49.714 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 08:08:49 smithi090 bash[134713]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-g 2024-05-27T08:08:50.214 INFO:journalctl@ceph.mds.g.smithi090.stdout:May 27 08:08:49 smithi090 podman[134725]: 2024-05-27 08:08:49.836160508 +0000 UTC m=+0.673002713 container remove 37b82db357a226fa30efec82b638b0acf291c459e2cdf33e812a5d5abe48a047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90) 2024-05-27T08:08:50.306 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.g.service' 2024-05-27T08:08:50.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:08:50.400 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-27T08:08:50.401 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-27T08:08:50.401 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j 2024-05-27T08:08:50.543 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.j.service' 2024-05-27T08:08:50.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:08:50.647 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-27T08:08:50.647 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-27T08:08:50.647 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.b 2024-05-27T08:08:51.065 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 08:08:50 smithi129 systemd[1]: Stopping Ceph mds.b for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:08:51.565 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 08:08:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b[65300]: 2024-05-27T08:08:51.247+0000 7f8489000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:08:51.565 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 08:08:51 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b[65300]: 2024-05-27T08:08:51.247+0000 7f8489000700 -1 mds.b *** got signal Terminated *** 2024-05-27T08:08:58.406 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 08:08:58 smithi129 podman[73163]: 2024-05-27 08:08:58.106493918 +0000 UTC m=+6.996041936 container died d2c1c48c587b0486490ba25dd8e795c0ca11879a64868bbac2bd5ff185aea35a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-05-27T08:08:58.786 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 08:08:58 smithi129 podman[73163]: 2024-05-27 08:08:58.525363844 +0000 UTC m=+7.414911862 container cleanup d2c1c48c587b0486490ba25dd8e795c0ca11879a64868bbac2bd5ff185aea35a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:08:58.786 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 08:08:58 smithi129 bash[73163]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b 2024-05-27T08:08:59.064 INFO:journalctl@ceph.mds.b.smithi129.stdout:May 27 08:08:58 smithi129 podman[73176]: 2024-05-27 08:08:58.784616485 +0000 UTC m=+0.639806933 container remove d2c1c48c587b0486490ba25dd8e795c0ca11879a64868bbac2bd5ff185aea35a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-27T08:08:59.296 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.b.service' 2024-05-27T08:08:59.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:08:59.403 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-27T08:08:59.403 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-27T08:08:59.403 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.e 2024-05-27T08:08:59.814 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 08:08:59 smithi129 systemd[1]: Stopping Ceph mds.e for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:09:00.315 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 08:09:00 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-e[64685]: 2024-05-27T08:09:00.013+0000 7f6729600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:09:00.315 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 08:09:00 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-e[64685]: 2024-05-27T08:09:00.013+0000 7f6729600700 -1 mds.e *** got signal Terminated *** 2024-05-27T08:09:05.563 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 08:09:05 smithi129 podman[73283]: 2024-05-27 08:09:05.277022627 +0000 UTC m=+5.416495920 container died 7f2d3fbd6f3076203e80762f0382e6df5549ecf0f273dde6213361d36dd10378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-e, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-05-27T08:09:05.815 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 08:09:05 smithi129 podman[73283]: 2024-05-27 08:09:05.688515025 +0000 UTC m=+5.827988312 container cleanup 7f2d3fbd6f3076203e80762f0382e6df5549ecf0f273dde6213361d36dd10378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:09:05.815 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 08:09:05 smithi129 bash[73283]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-e 2024-05-27T08:09:06.315 INFO:journalctl@ceph.mds.e.smithi129.stdout:May 27 08:09:05 smithi129 podman[73297]: 2024-05-27 08:09:05.973134769 +0000 UTC m=+0.678668463 container remove 7f2d3fbd6f3076203e80762f0382e6df5549ecf0f273dde6213361d36dd10378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-27T08:09:06.434 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.e.service' 2024-05-27T08:09:06.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:09:06.540 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-27T08:09:06.541 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-27T08:09:06.541 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.h 2024-05-27T08:09:06.815 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 08:09:06 smithi129 systemd[1]: Stopping Ceph mds.h for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:09:07.565 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 08:09:07 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-h[64989]: 2024-05-27T08:09:07.107+0000 7f0a6f000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:09:07.565 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 08:09:07 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-h[64989]: 2024-05-27T08:09:07.107+0000 7f0a6f000700 -1 mds.h *** got signal Terminated *** 2024-05-27T08:09:12.558 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 08:09:12 smithi129 podman[73403]: 2024-05-27 08:09:12.267066154 +0000 UTC m=+5.295492968 container died 37f5f08697cf618a164706ae58025204dcbca4b7d03561aae8e7f82b61a20aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-h, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-27T08:09:12.815 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 08:09:12 smithi129 podman[73403]: 2024-05-27 08:09:12.684033809 +0000 UTC m=+5.712460617 container cleanup 37f5f08697cf618a164706ae58025204dcbca4b7d03561aae8e7f82b61a20aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-27T08:09:12.815 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 08:09:12 smithi129 bash[73403]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-h 2024-05-27T08:09:13.314 INFO:journalctl@ceph.mds.h.smithi129.stdout:May 27 08:09:12 smithi129 podman[73428]: 2024-05-27 08:09:12.97705341 +0000 UTC m=+0.693293837 container remove 37f5f08697cf618a164706ae58025204dcbca4b7d03561aae8e7f82b61a20aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T08:09:13.472 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.h.service' 2024-05-27T08:09:13.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:09:13.581 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-27T08:09:13.581 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-27T08:09:13.581 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.k 2024-05-27T08:09:14.064 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 08:09:13 smithi129 systemd[1]: Stopping Ceph mds.k for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:09:14.564 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 08:09:14 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-k[64375]: 2024-05-27T08:09:14.212+0000 7f993dc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:09:14.565 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 08:09:14 smithi129 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-k[64375]: 2024-05-27T08:09:14.212+0000 7f993dc00700 -1 mds.k *** got signal Terminated *** 2024-05-27T08:09:18.651 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 08:09:18 smithi129 podman[73537]: 2024-05-27 08:09:18.325832761 +0000 UTC m=+4.248482911 container died 446c1375720aca54b2ca1777d833303bfd4487436a85fc29b60dfe7c84649563 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-k, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-27T08:09:18.957 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 08:09:18 smithi129 podman[73537]: 2024-05-27 08:09:18.763489511 +0000 UTC m=+4.686139651 container cleanup 446c1375720aca54b2ca1777d833303bfd4487436a85fc29b60dfe7c84649563 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T08:09:18.957 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 08:09:18 smithi129 bash[73537]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-k 2024-05-27T08:09:19.314 INFO:journalctl@ceph.mds.k.smithi129.stdout:May 27 08:09:18 smithi129 podman[73551]: 2024-05-27 08:09:18.955859962 +0000 UTC m=+0.604050087 container remove 446c1375720aca54b2ca1777d833303bfd4487436a85fc29b60dfe7c84649563 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-k, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD) 2024-05-27T08:09:19.480 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.k.service' 2024-05-27T08:09:19.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:09:19.584 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-27T08:09:19.584 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-27T08:09:19.584 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c 2024-05-27T08:09:19.691 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.c.service' 2024-05-27T08:09:19.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:09:19.795 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-27T08:09:19.795 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-27T08:09:19.795 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.f 2024-05-27T08:09:20.250 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 08:09:19 smithi136 systemd[1]: Stopping Ceph mds.f for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:09:20.591 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 08:09:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-f[65732]: 2024-05-27T08:09:20.392+0000 7fb9f2800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:09:20.591 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 08:09:20 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-f[65732]: 2024-05-27T08:09:20.392+0000 7fb9f2800700 -1 mds.f *** got signal Terminated *** 2024-05-27T08:09:23.836 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 08:09:23 smithi136 podman[75109]: 2024-05-27 08:09:23.532736489 +0000 UTC m=+3.268315653 container died 1f254d421592e4d00f769a07edca15cd7a2a75996d8fc2b8bd6980af270641d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-27T08:09:24.250 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 08:09:23 smithi136 podman[75109]: 2024-05-27 08:09:23.969536471 +0000 UTC m=+3.705115604 container cleanup 1f254d421592e4d00f769a07edca15cd7a2a75996d8fc2b8bd6980af270641d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:09:24.251 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 08:09:23 smithi136 bash[75109]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-f 2024-05-27T08:09:24.743 INFO:journalctl@ceph.mds.f.smithi136.stdout:May 27 08:09:24 smithi136 podman[75133]: 2024-05-27 08:09:24.274682381 +0000 UTC m=+0.730568531 container remove 1f254d421592e4d00f769a07edca15cd7a2a75996d8fc2b8bd6980af270641d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T08:09:24.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.f.service' 2024-05-27T08:09:24.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:09:24.889 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-27T08:09:24.889 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-27T08:09:24.889 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.i 2024-05-27T08:09:25.456 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 08:09:25 smithi136 systemd[1]: Stopping Ceph mds.i for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:09:25.750 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 08:09:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-i[66332]: 2024-05-27T08:09:25.454+0000 7fbce3800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:09:25.750 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 08:09:25 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-i[66332]: 2024-05-27T08:09:25.454+0000 7fbce3800700 -1 mds.i *** got signal Terminated *** 2024-05-27T08:09:31.517 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 08:09:31 smithi136 podman[75239]: 2024-05-27 08:09:31.255847819 +0000 UTC m=+5.905812855 container died 8a9624ce7aa2241feddc3891b4cbd66f25e383606001130866c9d6c5f828a8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-i, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-27T08:09:31.825 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 08:09:31 smithi136 podman[75239]: 2024-05-27 08:09:31.614825161 +0000 UTC m=+6.264790189 container cleanup 8a9624ce7aa2241feddc3891b4cbd66f25e383606001130866c9d6c5f828a8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:09:31.825 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 08:09:31 smithi136 bash[75239]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-i 2024-05-27T08:09:32.250 INFO:journalctl@ceph.mds.i.smithi136.stdout:May 27 08:09:31 smithi136 podman[75254]: 2024-05-27 08:09:31.824315146 +0000 UTC m=+0.548923948 container remove 8a9624ce7aa2241feddc3891b4cbd66f25e383606001130866c9d6c5f828a8eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-i, ceph=True, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:09:32.359 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.i.service' 2024-05-27T08:09:32.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:09:32.460 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-27T08:09:32.460 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-27T08:09:32.460 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.l 2024-05-27T08:09:33.000 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 08:09:32 smithi136 systemd[1]: Stopping Ceph mds.l for b0214938-1bfa-11ef-bc9b-c7b262605968... 2024-05-27T08:09:33.500 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 08:09:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-l[65418]: 2024-05-27T08:09:33.084+0000 7f8a08200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:09:33.500 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 08:09:33 smithi136 ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-l[65418]: 2024-05-27T08:09:33.084+0000 7f8a08200700 -1 mds.l *** got signal Terminated *** 2024-05-27T08:09:37.660 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 08:09:37 smithi136 podman[75362]: 2024-05-27 08:09:37.349959654 +0000 UTC m=+4.413929802 container died d8a33d5999d14dbf015667b2f29d72a243f96f44ffa4eda0847cde7e13eb3cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-l, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-27T08:09:37.936 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 08:09:37 smithi136 podman[75362]: 2024-05-27 08:09:37.767823205 +0000 UTC m=+4.831793317 container cleanup d8a33d5999d14dbf015667b2f29d72a243f96f44ffa4eda0847cde7e13eb3cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-l, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, org.label-schema.schema-version=1.0) 2024-05-27T08:09:37.937 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 08:09:37 smithi136 bash[75362]: ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-l 2024-05-27T08:09:38.250 INFO:journalctl@ceph.mds.l.smithi136.stdout:May 27 08:09:37 smithi136 podman[75375]: 2024-05-27 08:09:37.935266402 +0000 UTC m=+0.565564725 container remove d8a33d5999d14dbf015667b2f29d72a243f96f44ffa4eda0847cde7e13eb3cbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e6a098888d5cb5eb3200725617291c1104881c62c1d90236d9eab3f8efd213b6, name=ceph-b0214938-1bfa-11ef-bc9b-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, RELEASE=wip-vshankar-testing-20240521.063558-debug-38f9d90, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T08:09:38.434 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b0214938-1bfa-11ef-bc9b-c7b262605968@mds.l.service' 2024-05-27T08:09:38.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:09:38.537 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-27T08:09:38.537 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T08:09:38.855 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T08:09:43.903 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T08:09:44.220 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T08:09:49.102 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T08:09:49.418 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T08:09:54.276 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:09:54.366 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:09:54.461 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:09:54.553 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-27T08:09:54.555 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662/remote/smithi090/crash 2024-05-27T08:09:54.556 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T08:09:54.667 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T08:09:54.667 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-05-27T08:09:54.672 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662/remote/smithi129/crash 2024-05-27T08:09:54.673 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T08:09:54.772 INFO:teuthology.orchestra.run.smithi129.stderr:tar: /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T08:09:54.772 INFO:teuthology.orchestra.run.smithi129.stderr:tar: Error is not recoverable: exiting now 2024-05-27T08:09:54.775 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662/remote/smithi136/crash 2024-05-27T08:09:54.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T08:09:54.872 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T08:09:54.872 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-05-27T08:09:54.876 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-27T08:09:54.876 DEBUG:teuthology.orchestra.run.smithi090:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-27T08:09:55.055 INFO:teuthology.orchestra.run.smithi090.stdout:2024-05-27T07:57:27.317310+0000 mon.a (mon.0) 1465 : cluster [WRN] Health check failed: 3 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-05-27T08:09:55.057 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-27T08:09:55.057 INFO:tasks.cephadm:Compressing logs... 2024-05-27T08:09:55.057 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T08:09:55.062 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T08:09:55.070 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T08:09:55.161 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-05-27T08:09:55.162 INFO:teuthology.orchestra.run.smithi136.stderr: /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T08:09:55.162 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-05-27T08:09:55.163 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-05-27T08:09:55.164 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T08:09:55.165 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-05-27T08:09:55.166 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-05-27T08:09:55.168 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-05-27T08:09:55.169 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-05-27T08:09:55.169 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T08:09:55.169 INFO:teuthology.orchestra.run.smithi090.stderr:: No such file or directory 2024-05-27T08:09:55.169 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T08:09:55.170 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-05-27T08:09:55.171 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T08:09:55.172 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-05-27T08:09:55.172 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-05-27T08:09:55.172 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-05-27T08:09:55.173 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-05-27T08:09:55.173 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-05-27T08:09:55.174 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T08:09:55.175 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-05-27T08:09:55.175 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-05-27T08:09:55.175 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T08:09:55.176 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-05-27T08:09:55.176 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-05-27T08:09:55.177 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-05-27T08:09:55.179 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T08:09:55.180 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-05-27T08:09:55.180 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-05-27T08:09:55.181 INFO:teuthology.orchestra.run.smithi129.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-27T08:09:55.181 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T08:09:55.181 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T08:09:55.183 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T08:09:55.183 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-05-27T08:09:55.183 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-05-27T08:09:55.183 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-05-27T08:09:55.184 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-05-27T08:09:55.184 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-05-27T08:09:55.185 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-05-27T08:09:55.186 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T08:09:55.186 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-27T08:09:55.187 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-05-27T08:09:55.187 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-05-27T08:09:55.188 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-05-27T08:09:55.189 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-05-27T08:09:55.190 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T08:09:55.191 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-05-27T08:09:55.192 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T08:09:55.193 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-05-27T08:09:55.195 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-05-27T08:09:55.195 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T08:09:55.199 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-05-27T08:09:55.323 INFO:teuthology.orchestra.run.smithi136.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T08:09:55.350 INFO:teuthology.orchestra.run.smithi136.stderr: 83.7% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T08:09:55.359 INFO:teuthology.orchestra.run.smithi129.stderr: 83.8% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T08:09:55.360 INFO:teuthology.orchestra.run.smithi136.stderr: 90.3% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T08:09:55.387 INFO:teuthology.orchestra.run.smithi129.stderr: 90.8% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T08:09:55.472 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-27T08:09:55.486 INFO:teuthology.orchestra.run.smithi090.stderr: 87.3% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T08:09:55.503 INFO:teuthology.orchestra.run.smithi090.stderr: 84.2% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T08:09:55.575 INFO:teuthology.orchestra.run.smithi129.stderr: 87.3% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T08:09:55.575 INFO:teuthology.orchestra.run.smithi136.stderr: 94.9% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-05-27T08:09:55.578 INFO:teuthology.orchestra.run.smithi136.stderr: 94.8% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-05-27T08:09:55.599 INFO:teuthology.orchestra.run.smithi136.stderr: 90.9% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T08:09:55.611 INFO:teuthology.orchestra.run.smithi129.stderr: 90.3% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T08:09:55.630 INFO:teuthology.orchestra.run.smithi136.stderr: 87.3% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T08:09:55.728 INFO:teuthology.orchestra.run.smithi090.stderr: 90.9% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T08:09:55.987 INFO:teuthology.orchestra.run.smithi129.stderr: 91.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T08:09:56.080 INFO:teuthology.orchestra.run.smithi136.stderr: 93.3% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-05-27T08:09:56.238 INFO:teuthology.orchestra.run.smithi129.stderr: 93.2% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-05-27T08:09:56.257 INFO:teuthology.orchestra.run.smithi090.stderr: 90.1% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T08:09:56.280 INFO:teuthology.orchestra.run.smithi090.stderr: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T08:09:56.650 INFO:teuthology.orchestra.run.smithi090.stderr: 94.8% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-05-27T08:09:56.667 INFO:teuthology.orchestra.run.smithi090.stderr: 94.7% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-05-27T08:09:57.444 INFO:teuthology.orchestra.run.smithi129.stderr: 93.2% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-05-27T08:09:57.709 INFO:teuthology.orchestra.run.smithi129.stderr: 93.2% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-05-27T08:09:58.264 INFO:teuthology.orchestra.run.smithi136.stderr: 91.6% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-05-27T08:09:59.181 INFO:teuthology.orchestra.run.smithi136.stderr: 94.8% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-05-27T08:09:59.581 INFO:teuthology.orchestra.run.smithi129.stderr: 91.6% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-05-27T08:10:00.289 INFO:teuthology.orchestra.run.smithi090.stderr: 90.2% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-05-27T08:10:00.988 INFO:teuthology.orchestra.run.smithi090.stderr: 94.2% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-05-27T08:10:02.160 INFO:teuthology.orchestra.run.smithi090.stderr: 93.3% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-05-27T08:10:10.053 INFO:teuthology.orchestra.run.smithi090.stderr: 90.1% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-05-27T08:11:11.447 INFO:teuthology.orchestra.run.smithi129.stderr: 93.9% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-05-27T08:14:08.085 INFO:teuthology.orchestra.run.smithi129.stderr: 91.0% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-05-27T08:14:29.840 INFO:teuthology.orchestra.run.smithi129.stderr: 91.1% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-05-27T08:15:34.864 INFO:teuthology.orchestra.run.smithi129.stderr: 90.9% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-05-27T08:15:36.833 INFO:teuthology.orchestra.run.smithi136.stderr: 91.0% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-05-27T08:15:48.135 INFO:teuthology.orchestra.run.smithi129.stderr: 91.0% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-05-27T08:15:48.162 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-05-27T08:15:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:real 5m53.021s 2024-05-27T08:15:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:user 8m44.630s 2024-05-27T08:15:48.163 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m56.728s 2024-05-27T08:16:26.493 INFO:teuthology.orchestra.run.smithi136.stderr: 91.0% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-05-27T08:17:08.638 INFO:teuthology.orchestra.run.smithi136.stderr: 91.1% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-05-27T08:17:32.741 INFO:teuthology.orchestra.run.smithi136.stderr: 90.8% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-05-27T08:17:32.747 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-05-27T08:17:32.747 INFO:teuthology.orchestra.run.smithi136.stderr:real 7m37.620s 2024-05-27T08:17:32.747 INFO:teuthology.orchestra.run.smithi136.stderr:user 10m51.170s 2024-05-27T08:17:32.747 INFO:teuthology.orchestra.run.smithi136.stderr:sys 1m10.778s 2024-05-27T08:20:49.607 INFO:teuthology.orchestra.run.smithi090.stderr: 91.0% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-05-27T08:21:32.812 INFO:teuthology.orchestra.run.smithi090.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-05-27T08:21:54.602 INFO:teuthology.orchestra.run.smithi090.stderr: 90.9% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-05-27T08:22:06.026 INFO:teuthology.orchestra.run.smithi090.stderr: 90.9% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-05-27T08:23:04.171 INFO:teuthology.orchestra.run.smithi090.stderr: 90.8% -- replaced with /var/log/ceph/b0214938-1bfa-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-05-27T08:23:04.185 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-05-27T08:23:04.185 INFO:teuthology.orchestra.run.smithi090.stderr:real 13m9.074s 2024-05-27T08:23:04.185 INFO:teuthology.orchestra.run.smithi090.stderr:user 19m35.872s 2024-05-27T08:23:04.185 INFO:teuthology.orchestra.run.smithi090.stderr:sys 2m28.353s 2024-05-27T08:23:04.186 INFO:tasks.cephadm:Archiving logs... 2024-05-27T08:23:04.186 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662/remote/smithi090/log 2024-05-27T08:23:04.188 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T08:24:18.194 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662/remote/smithi129/log 2024-05-27T08:24:18.196 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T08:25:09.668 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662/remote/smithi136/log 2024-05-27T08:25:09.669 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T08:26:05.724 INFO:tasks.cephadm:Removing cluster... 2024-05-27T08:26:05.725 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 --force 2024-05-27T08:26:06.163 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T08:26:08.077 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 --force 2024-05-27T08:26:08.506 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T08:26:10.137 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 --force 2024-05-27T08:26:10.638 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: b0214938-1bfa-11ef-bc9b-c7b262605968 2024-05-27T08:26:12.409 INFO:tasks.cephadm:Removing cephadm ... 2024-05-27T08:26:12.409 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:12.482 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:12.540 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:12.598 INFO:tasks.cephadm:Teardown complete 2024-05-27T08:26:12.599 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-27T08:26:12.610 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/install/__init__.py", line 632, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_38f9d900c83c63799fbdbe61acc9a11b0d3554a6/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi090 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump' 2024-05-27T08:26:12.611 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-27T08:26:12.612 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T08:26:12.619 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T08:26:12.627 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T08:26:12.799 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-27T08:26:12.810 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-27T08:26:12.811 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:26:12.819 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:26:12.827 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:26:12.864 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:26:12.875 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:26:12.881 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:26:12.975 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:26:12.976 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-05-27T08:26:12.976 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 +690us[ +629us] +/- 61ms 2024-05-27T08:26:12.976 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 -656us[ -656us] +/- 51ms 2024-05-27T08:26:12.976 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +45us[ -17us] +/- 75ms 2024-05-27T08:26:12.976 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:26:13.006 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:26:13.006 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-05-27T08:26:13.007 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv01.front.sepia.ceph.com 3 6 377 45 +696us[ +695us] +/- 61ms 2024-05-27T08:26:13.007 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -644us[ -645us] +/- 51ms 2024-05-27T08:26:13.007 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +55us[ +53us] +/- 75ms 2024-05-27T08:26:13.007 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:26:13.031 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:26:13.031 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-05-27T08:26:13.031 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 +720us[ +712us] +/- 61ms 2024-05-27T08:26:13.031 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 -634us[ -634us] +/- 51ms 2024-05-27T08:26:13.031 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -25us[ -25us] +/- 75ms 2024-05-27T08:26:13.031 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:26:13.035 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-27T08:26:13.046 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-27T08:26:13.047 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-27T08:26:13.101 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T08:26:13.427 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T08:26:13.660 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T08:26:13.888 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:26:13.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:26:13.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:26:14.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:26:14.082 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:26:14.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:26:14.174 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-27T08:26:14.185 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-27T08:26:14.267 INFO:teuthology.task.internal:Duration was 4005.861614 seconds 2024-05-27T08:26:14.268 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-27T08:26:14.304 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-27T08:26:14.304 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T08:26:14.312 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T08:26:14.320 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T08:26:14.468 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:26:14.470 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:26:14.481 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:26:14.931 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-27T08:26:14.931 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-05-27T08:26:14.931 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T08:26:15.010 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-05-27T08:26:15.010 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T08:26:15.087 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-05-27T08:26:15.087 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T08:26:15.165 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-27T08:26:15.165 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T08:26:15.173 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T08:26:15.181 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T08:26:15.261 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-27T08:26:15.261 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T08:26:15.266 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T08:26:15.272 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T08:26:16.712 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-27T08:26:16.726 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-27T08:26:16.726 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T08:26:16.893 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T08:26:16.986 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T08:26:17.075 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-27T08:26:17.086 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:26:17.095 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:26:17.102 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:26:17.201 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-05-27T08:26:17.217 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-05-27T08:26:17.223 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-05-27T08:26:17.289 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:26:17.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:26:17.345 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:26:17.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:26:17.399 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:26:17.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:26:17.454 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-27T08:26:17.466 INFO:teuthology.task.internal:Transferring archived files... 2024-05-27T08:26:17.468 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662/remote/smithi090 2024-05-27T08:26:17.468 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T08:26:17.600 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662/remote/smithi129 2024-05-27T08:26:17.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T08:26:17.698 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-27_05:52:49-fs-wip-vshankar-testing-20240521.063558-debug-testing-default-smithi/7727662/remote/smithi136 2024-05-27T08:26:17.698 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T08:26:17.795 INFO:teuthology.task.internal:Removing archive directory... 2024-05-27T08:26:17.796 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T08:26:17.804 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T08:26:17.811 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T08:26:17.868 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-27T08:26:17.881 INFO:teuthology.task.internal:Not uploading archives. 2024-05-27T08:26:17.881 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-27T08:26:17.918 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-27T08:26:17.918 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T08:26:17.926 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T08:26:17.934 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T08:26:17.976 INFO:teuthology.orchestra.run.smithi090.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 08:26 /home/ubuntu/cephtest 2024-05-27T08:26:17.983 INFO:teuthology.orchestra.run.smithi129.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 08:26 /home/ubuntu/cephtest 2024-05-27T08:26:17.990 INFO:teuthology.orchestra.run.smithi136.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 08:26 /home/ubuntu/cephtest 2024-05-27T08:26:17.994 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-27T08:26:18.006 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-27T08:26:18.058 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi090 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump' 2024-05-27T08:26:18.062 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/postgres}} duration: 4005.8616144657135 failure_reason: 'Command failed on smithi090 with status 22: ''sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:38f9d900c83c63799fbdbe61acc9a11b0d3554a6 shell --fsid b0214938-1bfa-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump''' flavor: default mon.a-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.b-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 mon.c-kernel-sha1: b9459a26b0888785d05c47deb35abc9d0354ab93 owner: scheduled_vshankar@teuthology status: fail success: false 2024-05-27T08:26:18.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T08:26:18.156 INFO:teuthology.run:FAIL